[PATCH v1 0/2] KVM: arm64: Use folio for THP support

[PATCH v1 0/2] KVM: arm64: Use folio for THP support

[Vincent Donnefort]

With the introduction of folios for transparent huge pages [1], we
can rely on this support to identify if a page is backed by a huge one,
saving a page table walk.

[1] https://lkml.kernel.org/r/20220504182857.4013401-3-willy@infradead.org

Vincent Donnefort (2):
  KVM: arm64: Do not transfer page refcount for THP adjustment
  KVM: arm64: Use folio for THP adjustment

 arch/arm64/kvm/mmu.c | 79 ++------------------------------------------
 1 file changed, 3 insertions(+), 76 deletions(-)


base-commit: ce9ecca0238b140b88f43859b211c9fdfd8e5b70
-- 
2.42.0.515.g380fc7ccd1-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v1 1/2] KVM: arm64: Do not transfer page refcount for THP adjustment

[Vincent Donnefort]

With folios, GUP affects a refcount common to all pages forming the THP.
There is therefore no need to move the refcount from the tail to the
head page. Under the hood it decrements and increments
the same counter.

Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 587a104f66c3..de5e5148ef5d 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1295,28 +1295,8 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
 		if (sz < PMD_SIZE)
 			return PAGE_SIZE;
 
-		/*
-		 * The address we faulted on is backed by a transparent huge
-		 * page.  However, because we map the compound huge page and
-		 * not the individual tail page, we need to transfer the
-		 * refcount to the head page.  We have to be careful that the
-		 * THP doesn't start to split while we are adjusting the
-		 * refcounts.
-		 *
-		 * We are sure this doesn't happen, because mmu_invalidate_retry
-		 * was successful and we are holding the mmu_lock, so if this
-		 * THP is trying to split, it will be blocked in the mmu
-		 * notifier before touching any of the pages, specifically
-		 * before being able to call __split_huge_page_refcount().
-		 *
-		 * We can therefore safely transfer the refcount from PG_tail
-		 * to PG_head and switch the pfn from a tail page to the head
-		 * page accordingly.
-		 */
 		*ipap &= PMD_MASK;
-		kvm_release_pfn_clean(pfn);
 		pfn &= ~(PTRS_PER_PMD - 1);
-		get_page(pfn_to_page(pfn));
 		*pfnp = pfn;
 
 		return PMD_SIZE;
-- 
2.42.0.515.g380fc7ccd1-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v1 2/2] KVM: arm64: Use folio for THP adjustment

[Vincent Donnefort]

Now transparent huge pages use folios, we can check efficiently if a
page is mapped by a block by checking the folio size. This is saving a
page table walk.

It is safe to read the folio in this path. We've just increased its
refcount (GUP from __gfn_to_pfn_memslot()). This will prevent attempts
of splitting the huge page.

Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index de5e5148ef5d..69fcbcc7aca5 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -791,51 +791,6 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,
 	return 0;
 }
 
-static struct kvm_pgtable_mm_ops kvm_user_mm_ops = {
-	/* We shouldn't need any other callback to walk the PT */
-	.phys_to_virt		= kvm_host_va,
-};
-
-static int get_user_mapping_size(struct kvm *kvm, u64 addr)
-{
-	struct kvm_pgtable pgt = {
-		.pgd		= (kvm_pteref_t)kvm->mm->pgd,
-		.ia_bits	= vabits_actual,
-		.start_level	= (KVM_PGTABLE_MAX_LEVELS -
-				   CONFIG_PGTABLE_LEVELS),
-		.mm_ops		= &kvm_user_mm_ops,
-	};
-	unsigned long flags;
-	kvm_pte_t pte = 0;	/* Keep GCC quiet... */
-	u32 level = ~0;
-	int ret;
-
-	/*
-	 * Disable IRQs so that we hazard against a concurrent
-	 * teardown of the userspace page tables (which relies on
-	 * IPI-ing threads).
-	 */
-	local_irq_save(flags);
-	ret = kvm_pgtable_get_leaf(&pgt, addr, &pte, &level);
-	local_irq_restore(flags);
-
-	if (ret)
-		return ret;
-
-	/*
-	 * Not seeing an error, but not updating level? Something went
-	 * deeply wrong...
-	 */
-	if (WARN_ON(level >= KVM_PGTABLE_MAX_LEVELS))
-		return -EFAULT;
-
-	/* Oops, the userspace PTs are gone... Replay the fault */
-	if (!kvm_pte_valid(pte))
-		return -EAGAIN;
-
-	return BIT(ARM64_HW_PGTABLE_LEVEL_SHIFT(level));
-}
-
 static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = {
 	.zalloc_page		= stage2_memcache_zalloc_page,
 	.zalloc_pages_exact	= kvm_s2_zalloc_pages_exact,
@@ -1274,7 +1229,7 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,
  *
  * Returns the size of the mapping.
  */
-static long
+static unsigned long
 transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
 			    unsigned long hva, kvm_pfn_t *pfnp,
 			    phys_addr_t *ipap)
@@ -1287,10 +1242,7 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
 	 * block map is contained within the memslot.
 	 */
 	if (fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) {
-		int sz = get_user_mapping_size(kvm, hva);
-
-		if (sz < 0)
-			return sz;
+		size_t sz = folio_size(pfn_folio(pfn));
 
 		if (sz < PMD_SIZE)
 			return PAGE_SIZE;
@@ -1385,7 +1337,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	kvm_pfn_t pfn;
 	bool logging_active = memslot_is_logging(memslot);
 	unsigned long fault_level = kvm_vcpu_trap_get_fault_level(vcpu);
-	long vma_pagesize, fault_granule;
+	unsigned long vma_pagesize, fault_granule;
 	enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
 	struct kvm_pgtable *pgt;
 
@@ -1530,11 +1482,6 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 			vma_pagesize = transparent_hugepage_adjust(kvm, memslot,
 								   hva, &pfn,
 								   &fault_ipa);
-
-		if (vma_pagesize < 0) {
-			ret = vma_pagesize;
-			goto out_unlock;
-		}
 	}
 
 	if (fault_status != ESR_ELx_FSC_PERM && !device && kvm_has_mte(kvm)) {
-- 
2.42.0.515.g380fc7ccd1-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 0/2] KVM: arm64: Use folio for THP support

[Oliver Upton]

Vincent,

On Tue, Sep 26, 2023 at 07:19:30PM +0100, Vincent Donnefort wrote:
> With the introduction of folios for transparent huge pages [1], we
> can rely on this support to identify if a page is backed by a huge one,
> saving a page table walk.

Better yet, this gets rid of a particular walk that has had some rather
hairy bugs in the past :)

> [1] https://lkml.kernel.org/r/20220504182857.4013401-3-willy@infradead.org

nit: for the sake of posterity, it might be good to stick a reference to
commit cb196ee1ef39 ("mm/huge_memory: convert do_huge_pmd_anonymous_page()
to use vma_alloc_folio()") in the changelogs.

-- 
Thanks,
Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 1/2] KVM: arm64: Do not transfer page refcount for THP adjustment

[Matthew Wilcox]

On Tue, Sep 26, 2023 at 07:19:31PM +0100, Vincent Donnefort wrote:
> With folios, GUP affects a refcount common to all pages forming the THP.
> There is therefore no need to move the refcount from the tail to the
> head page. Under the hood it decrements and increments
> the same counter.

FYI, this isn't new to folios.  get_page() has _always_ operated on the
head page of a compound page.  Here's how it worked in 2.6.12-rc2:

+static inline void get_page(struct page *page)
+{
+       if (unlikely(PageCompound(page)))
+               page = (struct page *)page->private;
+       atomic_inc(&page->_count);
+}

I can't be bothered to switch over to the linux-fullhistory tree and
show how it evolved, because i think this sufficiently makes the point --
you can backport this patch as far as you like.

Folios just make it obvious that this is what's going on.  And that's a
good thing!  Making the MM less inscrutable is a win for everybody.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 0/2] KVM: arm64: Use folio for THP support

[Vincent Donnefort]

On Wed, Sep 27, 2023 at 06:30:40AM +0000, Oliver Upton wrote:
> Vincent,
> 
> On Tue, Sep 26, 2023 at 07:19:30PM +0100, Vincent Donnefort wrote:
> > With the introduction of folios for transparent huge pages [1], we
> > can rely on this support to identify if a page is backed by a huge one,
> > saving a page table walk.
> 
> Better yet, this gets rid of a particular walk that has had some rather
> hairy bugs in the past :)
> 
> > [1] https://lkml.kernel.org/r/20220504182857.4013401-3-willy@infradead.org
> 
> nit: for the sake of posterity, it might be good to stick a reference to
> commit cb196ee1ef39 ("mm/huge_memory: convert do_huge_pmd_anonymous_page()
> to use vma_alloc_folio()") in the changelogs.

I'll add that reference in the following version.

> 
> -- 
> Thanks,
> Oliver

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v1 1/2] KVM: arm64: Do not transfer page refcount for THP adjustment

[Vincent Donnefort]

On Wed, Sep 27, 2023 at 08:00:28AM +0100, Matthew Wilcox wrote:
> On Tue, Sep 26, 2023 at 07:19:31PM +0100, Vincent Donnefort wrote:
> > With folios, GUP affects a refcount common to all pages forming the THP.
> > There is therefore no need to move the refcount from the tail to the
> > head page. Under the hood it decrements and increments
> > the same counter.
> 
> FYI, this isn't new to folios.  get_page() has _always_ operated on the
> head page of a compound page.  Here's how it worked in 2.6.12-rc2:
> 
> +static inline void get_page(struct page *page)
> +{
> +       if (unlikely(PageCompound(page)))
> +               page = (struct page *)page->private;
> +       atomic_inc(&page->_count);
> +}
> 
> I can't be bothered to switch over to the linux-fullhistory tree and
> show how it evolved, because i think this sufficiently makes the point --
> you can backport this patch as far as you like.

Thanks for having a look!

I'll refine that description in the next version.

> 
> Folios just make it obvious that this is what's going on.  And that's a
> good thing!  Making the MM less inscrutable is a win for everybody.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel