[PATCH v2] KVM: arm64: Move data barrier to end of split walk

[PATCH v2] KVM: arm64: Move data barrier to end of split walk

[Colton Lewis]

This DSB guarantees page table updates have been made visible to the
hardware table walker. Moving the DSB from stage2_split_walker() to
after the walk is finished in kvm_pgtable_stage2_split() results in a
roughly 70% reduction in Clear Dirty Log Time in
dirty_log_perf_test (modified to use eager page splitting) when using
huge pages. This gain holds steady through a range of vcpus
used (tested 1-64) and memory used (tested 1-64GB).

This is safe to do because nothing else is using the page tables while
they are still being mapped and this is how other page table walkers
already function. None of them have a data barrier in the walker
itself because relative ordering of table PTEs to table contents comes
from the release semantics of stage2_make_pte().

Signed-off-by: Colton Lewis <coltonlewis@google.com>
---

v2:
  * Added more information about purpose of DSB to commit message
  * Rebased to v6.11-rc2

v1:
https://lore.kernel.org/kvmarm/20240718223519.1673835-1-coltonlewis@google.com/

 arch/arm64/kvm/hyp/pgtable.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 9e2bbee77491..9788af2ca8c0 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -1547,7 +1547,6 @@ static int stage2_split_walker(const struct kvm_pgtable_visit_ctx *ctx,
 	 */
 	new = kvm_init_table_pte(childp, mm_ops);
 	stage2_make_pte(ctx, new);
-	dsb(ishst);
 	return 0;
 }

@@ -1559,8 +1558,11 @@ int kvm_pgtable_stage2_split(struct kvm_pgtable *pgt, u64 addr, u64 size,
 		.flags	= KVM_PGTABLE_WALK_LEAF,
 		.arg	= mc,
 	};
+	int ret;

-	return kvm_pgtable_walk(pgt, addr, size, &walker);
+	ret = kvm_pgtable_walk(pgt, addr, size, &walker);
+	dsb(ishst);
+	return ret;
 }

 int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_s2_mmu *mmu,
--
2.46.0.76.ge559c4bf1a-goog

Re: [PATCH v2] KVM: arm64: Move data barrier to end of split walk

[Oliver Upton]

Hi Colton,

On Thu, Aug 08, 2024 at 05:42:43PM +0000, Colton Lewis wrote:
> This DSB guarantees page table updates have been made visible to the
> hardware table walker. Moving the DSB from stage2_split_walker() to
> after the walk is finished in kvm_pgtable_stage2_split() results in a
> roughly 70% reduction in Clear Dirty Log Time in
> dirty_log_perf_test (modified to use eager page splitting) when using
> huge pages. This gain holds steady through a range of vcpus
> used (tested 1-64) and memory used (tested 1-64GB).

Would you have time to put together a patch for the dirty_log_perf_test
changes you've made? This would be quite valuable for testing future
improvements to eager page splitting.

> This is safe to do because nothing else is using the page tables while
> they are still being mapped and this is how other page table walkers
> already function. None of them have a data barrier in the walker
> itself because relative ordering of table PTEs to table contents comes
> from the release semantics of stage2_make_pte().
> 
> Signed-off-by: Colton Lewis <coltonlewis@google.com>

The diff itself looks good to me, so:

Acked-by: Oliver Upton <oliver.upton@linux.dev>

-- 
Thanks,
Oliver

Re: [PATCH v2] KVM: arm64: Move data barrier to end of split walk

[Colton Lewis]

Oliver Upton <oliver.upton@linux.dev> writes:

> Hi Colton,

> On Thu, Aug 08, 2024 at 05:42:43PM +0000, Colton Lewis wrote:
>> This DSB guarantees page table updates have been made visible to the
>> hardware table walker. Moving the DSB from stage2_split_walker() to
>> after the walk is finished in kvm_pgtable_stage2_split() results in a
>> roughly 70% reduction in Clear Dirty Log Time in
>> dirty_log_perf_test (modified to use eager page splitting) when using
>> huge pages. This gain holds steady through a range of vcpus
>> used (tested 1-64) and memory used (tested 1-64GB).

> Would you have time to put together a patch for the dirty_log_perf_test
> changes you've made? This would be quite valuable for testing future
> improvements to eager page splitting.

I can do that.

>> This is safe to do because nothing else is using the page tables while
>> they are still being mapped and this is how other page table walkers
>> already function. None of them have a data barrier in the walker
>> itself because relative ordering of table PTEs to table contents comes
>> from the release semantics of stage2_make_pte().

>> Signed-off-by: Colton Lewis <coltonlewis@google.com>

> The diff itself looks good to me, so:

> Acked-by: Oliver Upton <oliver.upton@linux.dev>

> --
> Thanks,
> Oliver

Re: (subset) [PATCH v2] KVM: arm64: Move data barrier to end of split walk

[Marc Zyngier]

On Thu, 08 Aug 2024 17:42:43 +0000, Colton Lewis wrote:
> This DSB guarantees page table updates have been made visible to the
> hardware table walker. Moving the DSB from stage2_split_walker() to
> after the walk is finished in kvm_pgtable_stage2_split() results in a
> roughly 70% reduction in Clear Dirty Log Time in
> dirty_log_perf_test (modified to use eager page splitting) when using
> huge pages. This gain holds steady through a range of vcpus
> used (tested 1-64) and memory used (tested 1-64GB).
> 
> [...]

Applied to next, thanks!

[1/1] KVM: arm64: Move data barrier to end of split walk
      commit: 38753cbc4dca431d4354319c7481f6bd1a212baf

Cheers,

	M.
-- 
Without deviation from the norm, progress is not possible.