[PATCH v2] KVM: arm64: Optimize protected mode and FWB

[PATCH v2] KVM: arm64: Optimize protected mode and FWB

[Mostafa Saleh]

KVM opportunistically enables FWB if supported by the system for guest
VMs, which allows it to elude cache maintenance for data as they are
forced to be cacheable from stage-2.
In that case, __clean_dcache_guest_page() will immediately return.
However in protected mode, before calling __clean_dcache_guest_page()
it loops over the range and fix_map/unmap it, issuing TLB
invalidations, dsb() and isb() unnecessarily.

This can be optimized by returning early if FWB is supported,
kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
functions issue dsb() for the unmap path.

Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
index 4e329e39a695..049a09be2bbf 100644
--- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
+++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
@@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
 
 static void clean_dcache_guest_page(void *va, size_t size)
 {
+	/*
+	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
+	 * In contrast, the host stage-2 never has FWB enabled.
+	 */
+	if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
+		return;
+
 	__apply_guest_page(va, size, __clean_dcache_guest_page);
 }
 
-- 
2.55.0.229.g6434b31f56-goog

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

[Marc Zyngier]

On Tue, 21 Jul 2026 09:56:45 +0100,
Mostafa Saleh <smostafa@google.com> wrote:
> 
> KVM opportunistically enables FWB if supported by the system for guest
> VMs, which allows it to elude cache maintenance for data as they are
> forced to be cacheable from stage-2.
> In that case, __clean_dcache_guest_page() will immediately return.
> However in protected mode, before calling __clean_dcache_guest_page()
> it loops over the range and fix_map/unmap it, issuing TLB
> invalidations, dsb() and isb() unnecessarily.
> 
> This can be optimized by returning early if FWB is supported,
> kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> functions issue dsb() for the unmap path.
> 
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> index 4e329e39a695..049a09be2bbf 100644
> --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
>  
>  static void clean_dcache_guest_page(void *va, size_t size)
>  {
> +	/*
> +	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> +	 * In contrast, the host stage-2 never has FWB enabled.

Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
available.

Not sure that changes anything, but the comment is factually wrong.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

[Mostafa Saleh]

On Tue, Jul 21, 2026 at 11:10:07AM +0100, Marc Zyngier wrote:
> On Tue, 21 Jul 2026 09:56:45 +0100,
> Mostafa Saleh <smostafa@google.com> wrote:
> > 
> > KVM opportunistically enables FWB if supported by the system for guest
> > VMs, which allows it to elude cache maintenance for data as they are
> > forced to be cacheable from stage-2.
> > In that case, __clean_dcache_guest_page() will immediately return.
> > However in protected mode, before calling __clean_dcache_guest_page()
> > it loops over the range and fix_map/unmap it, issuing TLB
> > invalidations, dsb() and isb() unnecessarily.
> > 
> > This can be optimized by returning early if FWB is supported,
> > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > functions issue dsb() for the unmap path.
> > 
> > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> > ---
> >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > index 4e329e39a695..049a09be2bbf 100644
> > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
> >  
> >  static void clean_dcache_guest_page(void *va, size_t size)
> >  {
> > +	/*
> > +	 * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> > +	 * In contrast, the host stage-2 never has FWB enabled.
> 
> Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
> over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
> available.
> 
> Not sure that changes anything, but the comment is factually wrong.

Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not
change the patch, I will need to update the comment.

Thanks,
Mostafa

> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

[Fuad Tabba]

On Tue, 21 Jul 2026 at 11:19, Mostafa Saleh <smostafa@google.com> wrote:
>
> On Tue, Jul 21, 2026 at 11:10:07AM +0100, Marc Zyngier wrote:
> > On Tue, 21 Jul 2026 09:56:45 +0100,
> > Mostafa Saleh <smostafa@google.com> wrote:
> > >
> > > KVM opportunistically enables FWB if supported by the system for guest
> > > VMs, which allows it to elude cache maintenance for data as they are
> > > forced to be cacheable from stage-2.
> > > In that case, __clean_dcache_guest_page() will immediately return.
> > > However in protected mode, before calling __clean_dcache_guest_page()
> > > it loops over the range and fix_map/unmap it, issuing TLB
> > > invalidations, dsb() and isb() unnecessarily.
> > >
> > > This can be optimized by returning early if FWB is supported,
> > > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > > functions issue dsb() for the unmap path.
> > >
> > > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > > Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> > > ---
> > >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > >
> > > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > index 4e329e39a695..049a09be2bbf 100644
> > > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
> > >
> > >  static void clean_dcache_guest_page(void *va, size_t size)
> > >  {
> > > +   /*
> > > +    * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> > > +    * In contrast, the host stage-2 never has FWB enabled.
> >
> > Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
> > over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
> > available.
> >
> > Not sure that changes anything, but the comment is factually wrong.
>
> Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not
> change the patch, I will need to update the comment.

Sorry, this confusing is partially my fault. The original comment in v1

>  /* See __clean_dcache_guest_page() */

wasn't very clear to me, since I wasn't sure what I was looking for in
__clean_dcache_guest_page(). Talking to Mostafa in person, he's
referring to the comment _inside_ __clean_dcache_guest_page. I think
the best fix is to rephase as

+  /* See comment in __clean_dcache_guest_page() */

This would save duplication and having to repeat the comment.

Cheers,
/fuad

> Thanks,
> Mostafa
>
> >
> > Thanks,
> >
> >       M.
> >
> > --
> > Without deviation from the norm, progress is not possible.

Re: [PATCH v2] KVM: arm64: Optimize protected mode and FWB

[Mostafa Saleh]

On Wed, Jul 22, 2026 at 12:51:35PM +0100, Fuad Tabba wrote:
> On Tue, 21 Jul 2026 at 11:19, Mostafa Saleh <smostafa@google.com> wrote:
> >
> > On Tue, Jul 21, 2026 at 11:10:07AM +0100, Marc Zyngier wrote:
> > > On Tue, 21 Jul 2026 09:56:45 +0100,
> > > Mostafa Saleh <smostafa@google.com> wrote:
> > > >
> > > > KVM opportunistically enables FWB if supported by the system for guest
> > > > VMs, which allows it to elude cache maintenance for data as they are
> > > > forced to be cacheable from stage-2.
> > > > In that case, __clean_dcache_guest_page() will immediately return.
> > > > However in protected mode, before calling __clean_dcache_guest_page()
> > > > it loops over the range and fix_map/unmap it, issuing TLB
> > > > invalidations, dsb() and isb() unnecessarily.
> > > >
> > > > This can be optimized by returning early if FWB is supported,
> > > > kvm_pgtable_stage2_map() already issues dsb() and tlb invalidation
> > > > functions issue dsb() for the unmap path.
> > > >
> > > > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > > > Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
> > > > ---
> > > >  arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++
> > > >  1 file changed, 7 insertions(+)
> > > >
> > > > diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > > index 4e329e39a695..049a09be2bbf 100644
> > > > --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > > +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
> > > > @@ -261,6 +261,13 @@ static void __apply_guest_page(void *va, size_t size,
> > > >
> > > >  static void clean_dcache_guest_page(void *va, size_t size)
> > > >  {
> > > > +   /*
> > > > +    * Guest stage-2 uses FWB if available, making it safe to elide CMOs.
> > > > +    * In contrast, the host stage-2 never has FWB enabled.
> > >
> > > Beeeeeeep!!! Since a373930ec9406 ("KVM: arm64: Switch pKVM host S2
> > > over to KVM_PGTABLE_S2_AS_S1"), FWB is enabled everywhere when
> > > available.
> > >
> > > Not sure that changes anything, but the comment is factually wrong.
> >
> > Ah, I see it uses KVM_PGTABLE_S2_AS_S1 in that case, that should not
> > change the patch, I will need to update the comment.
> 
> Sorry, this confusing is partially my fault. The original comment in v1
> 
> >  /* See __clean_dcache_guest_page() */
> 
> wasn't very clear to me, since I wasn't sure what I was looking for in
> __clean_dcache_guest_page(). Talking to Mostafa in person, he's
> referring to the comment _inside_ __clean_dcache_guest_page. I think
> the best fix is to rephase as
> 
> +  /* See comment in __clean_dcache_guest_page() */
> 
> This would save duplication and having to repeat the comment.
> 

Makes sense, also thinking about this, I think we should do the
same for invalidate_icache_guest_page() and ARM64_HAS_CACHE_DIC.

I can include that in the same patch.

Thanks,
Mostafa