[PATCH] KVM: arm64: Optimize protected mode and FWB

[PATCH] KVM: arm64: Optimize protected mode and FWB

[Mostafa Saleh]

KVM opportunistically enables FWB if supported by the system for guest
VMs, which allows it to elude cache maintenance for data as they are
forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return.
However in protected mode, before calling __clean_dcache_guest_page()
it loops over the range and fix_map/unmap it, issuing TLB
invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
functions issue dsb() for the unmap path.

Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
index 4e329e39a695..6e9229106a25 100644
--- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
+++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
@@ -261,6 +261,10 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/* See __clean_dcache_guest_page() */
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return;
+
 	__apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
-- 
2.55.0.229.g6434b31f56-goog

Re: [PATCH] KVM: arm64: Optimize protected mode and FWB

[Fuad Tabba]

On Mon, 20 Jul 2026 at 21:35, Mostafa Saleh <smostafa@google.com> wrote:
>
> KVM opportunistically enables FWB if supported by the system for guest
> VMs, which allows it to elude cache maintenance for data as they are
> forced to be cacheable from stage-2.
> In that case, __clean_dcache_guest_page() will immediately return.
> However in protected mode, before calling __clean_dcache_guest_page()
> it loops over the range and fix_map/unmap it, issuing TLB
> invalidations, dsb() and isb() unnecessarily.
>
> This can be optimized by returning early if FWB is supported,
> kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> functions issue dsb() for the unmap path.
>
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> ---
>  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> index 4e329e39a695..6e9229106a25 100644
> --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> @@ -261,6 +261,10 @@ static void __apply_guest_page(void *va, size_t size,
>
>  static void clean_dcache_guest_page(void *va, size_t size)
>  {
> +       /* See __clean_dcache_guest_page() */

This looks good to me. One comment: this file also has
hyp_poison_page(), which deliberately avoids
__clean_dcache_guest_page() because the FWB elision would be wrong
there (it even notes "Prefer kvm_flush_dcache_to_poc() over
__clean_dcache_guest_page()"). Given the two opposite treatments in
the same file, could this comment say why eliding is safe here, e.g.
that the consumer is the guest via the FWB-forced stage-2? It would
save the next reader (if they're anything like me) from reconciling
the two.

That fuller comment could also replace "See
__clean_dcache_guest_page()", which mostly just points back at the
function passed in the __apply_guest_page() call below.

With that fixed:

Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>

Cheers,
/fuad

> +       if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
> +               return;
> +
>         __apply_guest_page(va, size, __clean_dcache_guest_page);
>  }
>
> --
> 2.55.0.229.g6434b31f56-goog
>

Re: [PATCH] KVM: arm64: Optimize protected mode and FWB

[Mostafa Saleh]

On Tue, Jul 21, 2026 at 08:49:26AM +0100, Fuad Tabba wrote:
> On Mon, 20 Jul 2026 at 21:35, Mostafa Saleh <smostafa@google.com> wrote:
> >
> > KVM opportunistically enables FWB if supported by the system for guest
> > VMs, which allows it to elude cache maintenance for data as they are
> > forced to be cacheable from stage-2.
> > In that case, __clean_dcache_guest_page() will immediately return.
> > However in protected mode, before calling __clean_dcache_guest_page()
> > it loops over the range and fix_map/unmap it, issuing TLB
> > invalidations, dsb() and isb() unnecessarily.
> >
> > This can be optimized by returning early if FWB is supported,
> > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > functions issue dsb() for the unmap path.
> >
> > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > ---
> >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > index 4e329e39a695..6e9229106a25 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > @@ -261,6 +261,10 @@ static void __apply_guest_page(void *va, size_t size,
> >
> >  static void clean_dcache_guest_page(void *va, size_t size)
> >  {
> > +       /* See __clean_dcache_guest_page() */
> 
> This looks good to me. One comment: this file also has
> hyp_poison_page(), which deliberately avoids
> __clean_dcache_guest_page() because the FWB elision would be wrong
> there (it even notes "Prefer kvm_flush_dcache_to_poc() over
> __clean_dcache_guest_page()"). Given the two opposite treatments in
> the same file, could this comment say why eliding is safe here, e.g.
> that the consumer is the guest via the FWB-forced stage-2? It would
> save the next reader (if they're anything like me) from reconciling
> the two.

__clean_dcache_guest_page() already have this comment.
	/*
	 * With FWB, we ensure that the guest always accesses memory using
	 * cacheable attributes, and we don't have to clean to PoC when
	 * faulting in pages. Furthermore, FWB implies IDC, so cleaning to
	 * PoU is not required either in this case.
	 */

I will add comment to clarify guest vs host:
	/*
	 * Guest stage-2 uses FWB if it exists, in that case it is
	 * safe to elide CMOs.
	 * Unlike the host stage-2 which never have FWB enabled.
	 */

Thanks,
Mostafa

> 
> That fuller comment could also replace "See
> __clean_dcache_guest_page()", which mostly just points back at the
> function passed in the __apply_guest_page() call below.
> 
> With that fixed:
> 
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> 
> Cheers,
> /fuad
> 
> > +       if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
> > +               return;
> > +
> >         __apply_guest_page(va, size, __clean_dcache_guest_page);
> >  }
> >
> > --
> > 2.55.0.229.g6434b31f56-goog
> >