[PATCH v3] KVM: arm64: Optimize protected mode with FWB and DIC

[PATCH v3] KVM: arm64: Optimize protected mode with FWB and DIC

[Mostafa Saleh]

KVM opportunistically enables FWB if supported by the system for guest
VMs, which allows it to elude cache maintenance for data as they are
forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return.
However in protected mode, before calling __clean_dcache_guest_page()
it loops over the range and fix_map/unmap it, issuing TLB
invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
functions issue dsb() for the unmap path.

Similarly for DIC, we can omit the call in
invalidate_icache_guest_page()

No isb() needed as ERET is context synchronization event.

Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
index 4e329e39a695..39aa8911f62c 100644
--- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
+++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
@@ -261,11 +261,18 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/* See comment in __clean_dcache_guest_page() */
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return;
+
 	__apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
 static void invalidate_icache_guest_page(void *va, size_t size)
 {
+	if (alternative_has_cap_unlikely(ARM64_HAS_CACHE_DIC))
+		return;
+
 	__apply_guest_page(va, size, __invalidate_icache_guest_page);
 }
 
-- 
2.55.0.229.g6434b31f56-goog

Re: [PATCH v3] KVM: arm64: Optimize protected mode with FWB and DIC

[Fuad Tabba]

On Thu, 23 Jul 2026 at 17:20, Mostafa Saleh <smostafa@google.com> wrote:
>
> KVM opportunistically enables FWB if supported by the system for guest
> VMs, which allows it to elude cache maintenance for data as they are
> forced to be cacheable from stage-2.
> In that case, __clean_dcache_guest_page() will immediately return.
> However in protected mode, before calling __clean_dcache_guest_page()
> it loops over the range and fix_map/unmap it, issuing TLB
> invalidations, dsb() and isb() unnecessarily.
>
> This can be optimized by returning early if FWB is supported,
> kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> functions issue dsb() for the unmap path.
>
> Similarly for DIC, we can omit the call in
> invalidate_icache_guest_page()
>
> No isb() needed as ERET is context synchronization event.
>
> Signed-off-by: Mostafa Saleh <smostafa@google.com>

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad


> ---
>  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> index 4e329e39a695..39aa8911f62c 100644
> --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> @@ -261,11 +261,18 @@ static void __apply_guest_page(void *va, size_t size,
>
>  static void clean_dcache_guest_page(void *va, size_t size)
>  {
> +       /* See comment in __clean_dcache_guest_page() */
> +       if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
> +               return;
> +
>         __apply_guest_page(va, size, __clean_dcache_guest_page);
>  }
>
>  static void invalidate_icache_guest_page(void *va, size_t size)
>  {
> +       if (alternative_has_cap_unlikely(ARM64_HAS_CACHE_DIC))
> +               return;
> +
>         __apply_guest_page(va, size, __invalidate_icache_guest_page);
>  }
>
> --
> 2.55.0.229.g6434b31f56-goog
>