* [PATCH 0/2] arm64 head.S fixes
@ 2024-04-15 7:54 Ard Biesheuvel
2024-04-15 7:54 ` [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround Ard Biesheuvel
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Ard Biesheuvel @ 2024-04-15 7:54 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Ard Biesheuvel, Catalin Marinas, Will Deacon, Marc Zyngier,
Mark Rutland
From: Ard Biesheuvel <ardb@kernel.org>
Ensure the MMU is off before changing HCR_EL2.E2H when entering at EL2.
Drop MMU disable workarounds where they are not needed.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Ard Biesheuvel (2):
arm64/head: Drop unnecessary pre-disable-MMU workaround
arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H
arch/arm64/kernel/head.S | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
--
2.44.0.683.g7961c838ac-goog
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround 2024-04-15 7:54 [PATCH 0/2] arm64 head.S fixes Ard Biesheuvel @ 2024-04-15 7:54 ` Ard Biesheuvel 2024-04-15 8:20 ` Marc Zyngier 2024-04-15 8:29 ` Mark Rutland 2024-04-15 7:54 ` [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H Ard Biesheuvel 2024-04-18 17:02 ` [PATCH 0/2] arm64 head.S fixes Catalin Marinas 2 siblings, 2 replies; 8+ messages in thread From: Ard Biesheuvel @ 2024-04-15 7:54 UTC (permalink / raw) To: linux-arm-kernel Cc: Ard Biesheuvel, Catalin Marinas, Will Deacon, Marc Zyngier, Mark Rutland From: Ard Biesheuvel <ardb@kernel.org> The Falkor erratum that results in the need for an ISB before clearing the M bit in SCTLR_ELx only applies to execution at exception level x, and so the workaround is not needed when disabling the EL1 MMU while running at EL2. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> --- arch/arm64/kernel/head.S | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 06234c3a15f3..b8bbd72cb194 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) cbz x0, 2f /* Set a sane SCTLR_EL1, the VHE way */ - pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x1 mov x2, #BOOT_CPU_FLAG_E2H b 3f 2: - pre_disable_mmu_workaround msr sctlr_el1, x1 mov x2, xzr 3: -- 2.44.0.683.g7961c838ac-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround 2024-04-15 7:54 ` [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround Ard Biesheuvel @ 2024-04-15 8:20 ` Marc Zyngier 2024-04-15 8:29 ` Mark Rutland 1 sibling, 0 replies; 8+ messages in thread From: Marc Zyngier @ 2024-04-15 8:20 UTC (permalink / raw) To: Ard Biesheuvel Cc: linux-arm-kernel, Ard Biesheuvel, Catalin Marinas, Will Deacon, Mark Rutland On Mon, 15 Apr 2024 08:54:14 +0100, Ard Biesheuvel <ardb+git@google.com> wrote: > > From: Ard Biesheuvel <ardb@kernel.org> > > The Falkor erratum that results in the need for an ISB before clearing > the M bit in SCTLR_ELx only applies to execution at exception level x, > and so the workaround is not needed when disabling the EL1 MMU while > running at EL2. > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> > --- > arch/arm64/kernel/head.S | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index 06234c3a15f3..b8bbd72cb194 100644 > --- a/arch/arm64/kernel/head.S > +++ b/arch/arm64/kernel/head.S > @@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) > cbz x0, 2f > > /* Set a sane SCTLR_EL1, the VHE way */ > - pre_disable_mmu_workaround > msr_s SYS_SCTLR_EL12, x1 > mov x2, #BOOT_CPU_FLAG_E2H > b 3f > > 2: > - pre_disable_mmu_workaround > msr sctlr_el1, x1 > mov x2, xzr > 3: Acked-by: Marc Zyngier <maz@kernel.org> M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround 2024-04-15 7:54 ` [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround Ard Biesheuvel 2024-04-15 8:20 ` Marc Zyngier @ 2024-04-15 8:29 ` Mark Rutland 1 sibling, 0 replies; 8+ messages in thread From: Mark Rutland @ 2024-04-15 8:29 UTC (permalink / raw) To: Ard Biesheuvel Cc: linux-arm-kernel, Ard Biesheuvel, Catalin Marinas, Will Deacon, Marc Zyngier On Mon, Apr 15, 2024 at 09:54:14AM +0200, Ard Biesheuvel wrote: > From: Ard Biesheuvel <ardb@kernel.org> > > The Falkor erratum that results in the need for an ISB before clearing > the M bit in SCTLR_ELx only applies to execution at exception level x, > and so the workaround is not needed when disabling the EL1 MMU while > running at EL2. > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Mark Rutland <mark.rutland@arm.com> Mark. > --- > arch/arm64/kernel/head.S | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index 06234c3a15f3..b8bbd72cb194 100644 > --- a/arch/arm64/kernel/head.S > +++ b/arch/arm64/kernel/head.S > @@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) > cbz x0, 2f > > /* Set a sane SCTLR_EL1, the VHE way */ > - pre_disable_mmu_workaround > msr_s SYS_SCTLR_EL12, x1 > mov x2, #BOOT_CPU_FLAG_E2H > b 3f > > 2: > - pre_disable_mmu_workaround > msr sctlr_el1, x1 > mov x2, xzr > 3: > -- > 2.44.0.683.g7961c838ac-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H 2024-04-15 7:54 [PATCH 0/2] arm64 head.S fixes Ard Biesheuvel 2024-04-15 7:54 ` [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround Ard Biesheuvel @ 2024-04-15 7:54 ` Ard Biesheuvel 2024-04-15 8:22 ` Marc Zyngier 2024-04-15 8:32 ` Mark Rutland 2024-04-18 17:02 ` [PATCH 0/2] arm64 head.S fixes Catalin Marinas 2 siblings, 2 replies; 8+ messages in thread From: Ard Biesheuvel @ 2024-04-15 7:54 UTC (permalink / raw) To: linux-arm-kernel Cc: Ard Biesheuvel, Catalin Marinas, Will Deacon, Marc Zyngier, Mark Rutland From: Ard Biesheuvel <ardb@kernel.org> Even though the boot protocol stipulates otherwise, an exception has been made for the EFI stub, and entering the core kernel with the MMU enabled is permitted. This allows a substantial amount of cache maintenance to be elided, wich is significant when fast boot times are critical (e.g., for booting micro-VMs) Once the initial ID map has been populated, the MMU is disabled as part of the logic sequence that puts all system registers into a known state. Any code that needs to execute within the window where the MMU is off is cleaned to the PoC explicitly, which includes all of HYP text when entering at EL2. However, the current sequence of initializing the EL2 system registers is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2 is reprogrammed, and this means that a VHE-to-nVHE switch may occur while the MMU is enabled. This switch causes some system registers as well as page table descriptors to be interpreted in a different way, potentially resulting in spurious exceptions relating to MMU translation. So disable the MMU explicitly first when entering in EL2 with the MMU and caches enabled. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> --- arch/arm64/kernel/head.S | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index b8bbd72cb194..cb68adcabe07 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) adr_l x1, __hyp_text_end adr_l x2, dcache_clean_poc blr x2 + + mov_q x0, INIT_SCTLR_EL2_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb 0: mov_q x0, HCR_HOST_NVHE_FLAGS -- 2.44.0.683.g7961c838ac-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H 2024-04-15 7:54 ` [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H Ard Biesheuvel @ 2024-04-15 8:22 ` Marc Zyngier 2024-04-15 8:32 ` Mark Rutland 1 sibling, 0 replies; 8+ messages in thread From: Marc Zyngier @ 2024-04-15 8:22 UTC (permalink / raw) To: Ard Biesheuvel Cc: linux-arm-kernel, Ard Biesheuvel, Catalin Marinas, Will Deacon, Mark Rutland On Mon, 15 Apr 2024 08:54:15 +0100, Ard Biesheuvel <ardb+git@google.com> wrote: > > From: Ard Biesheuvel <ardb@kernel.org> > > Even though the boot protocol stipulates otherwise, an exception has > been made for the EFI stub, and entering the core kernel with the MMU > enabled is permitted. This allows a substantial amount of cache > maintenance to be elided, wich is significant when fast boot times are > critical (e.g., for booting micro-VMs) > > Once the initial ID map has been populated, the MMU is disabled as part > of the logic sequence that puts all system registers into a known state. > Any code that needs to execute within the window where the MMU is off is > cleaned to the PoC explicitly, which includes all of HYP text when > entering at EL2. > > However, the current sequence of initializing the EL2 system registers > is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2 > is reprogrammed, and this means that a VHE-to-nVHE switch may occur > while the MMU is enabled. This switch causes some system registers as > well as page table descriptors to be interpreted in a different way, > potentially resulting in spurious exceptions relating to MMU > translation. > > So disable the MMU explicitly first when entering in EL2 with the MMU > and caches enabled. > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> > --- > arch/arm64/kernel/head.S | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index b8bbd72cb194..cb68adcabe07 100644 > --- a/arch/arm64/kernel/head.S > +++ b/arch/arm64/kernel/head.S > @@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) > adr_l x1, __hyp_text_end > adr_l x2, dcache_clean_poc > blr x2 > + > + mov_q x0, INIT_SCTLR_EL2_MMU_OFF > + pre_disable_mmu_workaround > + msr sctlr_el2, x0 > + isb > 0: > mov_q x0, HCR_HOST_NVHE_FLAGS > Acked-by: Marc Zyngier <maz@kernel.org> M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H 2024-04-15 7:54 ` [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H Ard Biesheuvel 2024-04-15 8:22 ` Marc Zyngier @ 2024-04-15 8:32 ` Mark Rutland 1 sibling, 0 replies; 8+ messages in thread From: Mark Rutland @ 2024-04-15 8:32 UTC (permalink / raw) To: Ard Biesheuvel Cc: linux-arm-kernel, Ard Biesheuvel, Catalin Marinas, Will Deacon, Marc Zyngier On Mon, Apr 15, 2024 at 09:54:15AM +0200, Ard Biesheuvel wrote: > From: Ard Biesheuvel <ardb@kernel.org> > > Even though the boot protocol stipulates otherwise, an exception has > been made for the EFI stub, and entering the core kernel with the MMU > enabled is permitted. This allows a substantial amount of cache > maintenance to be elided, wich is significant when fast boot times are > critical (e.g., for booting micro-VMs) > > Once the initial ID map has been populated, the MMU is disabled as part > of the logic sequence that puts all system registers into a known state. > Any code that needs to execute within the window where the MMU is off is > cleaned to the PoC explicitly, which includes all of HYP text when > entering at EL2. > > However, the current sequence of initializing the EL2 system registers > is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2 > is reprogrammed, and this means that a VHE-to-nVHE switch may occur > while the MMU is enabled. This switch causes some system registers as > well as page table descriptors to be interpreted in a different way, > potentially resulting in spurious exceptions relating to MMU > translation. > > So disable the MMU explicitly first when entering in EL2 with the MMU > and caches enabled. > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Mark Rutland <mark.rutland@arm.com> Mark. > --- > arch/arm64/kernel/head.S | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index b8bbd72cb194..cb68adcabe07 100644 > --- a/arch/arm64/kernel/head.S > +++ b/arch/arm64/kernel/head.S > @@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) > adr_l x1, __hyp_text_end > adr_l x2, dcache_clean_poc > blr x2 > + > + mov_q x0, INIT_SCTLR_EL2_MMU_OFF > + pre_disable_mmu_workaround > + msr sctlr_el2, x0 > + isb > 0: > mov_q x0, HCR_HOST_NVHE_FLAGS > > -- > 2.44.0.683.g7961c838ac-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/2] arm64 head.S fixes 2024-04-15 7:54 [PATCH 0/2] arm64 head.S fixes Ard Biesheuvel 2024-04-15 7:54 ` [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround Ard Biesheuvel 2024-04-15 7:54 ` [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H Ard Biesheuvel @ 2024-04-18 17:02 ` Catalin Marinas 2 siblings, 0 replies; 8+ messages in thread From: Catalin Marinas @ 2024-04-18 17:02 UTC (permalink / raw) To: linux-arm-kernel, Ard Biesheuvel Cc: Will Deacon, Ard Biesheuvel, Marc Zyngier, Mark Rutland On Mon, 15 Apr 2024 09:54:13 +0200, Ard Biesheuvel wrote: > Ensure the MMU is off before changing HCR_EL2.E2H when entering at EL2. > Drop MMU disable workarounds where they are not needed. > > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will@kernel.org> > Cc: Marc Zyngier <maz@kernel.org> > Cc: Mark Rutland <mark.rutland@arm.com> > > [...] Applied to arm64 (for-next/fixes), thanks! [1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround https://git.kernel.org/arm64/c/2b504e162037 [2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H https://git.kernel.org/arm64/c/34e526cb7d46 I added a Fixes tag and cc stable on the second patch, it goes all the way to 6.3 (this should actually be 6.6 LTS). -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-04-18 17:02 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-04-15 7:54 [PATCH 0/2] arm64 head.S fixes Ard Biesheuvel 2024-04-15 7:54 ` [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround Ard Biesheuvel 2024-04-15 8:20 ` Marc Zyngier 2024-04-15 8:29 ` Mark Rutland 2024-04-15 7:54 ` [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H Ard Biesheuvel 2024-04-15 8:22 ` Marc Zyngier 2024-04-15 8:32 ` Mark Rutland 2024-04-18 17:02 ` [PATCH 0/2] arm64 head.S fixes Catalin Marinas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).