Re: [PATCH v3 00/21] KVM: arm64: Rewrite page-table code and fault handling

Will Deacon
2020-09-03T12:18:22Z

On Thu, Sep 03, 2020 at 09:48:18PM +1000, Gavin Shan wrote:
> On 9/3/20 9:13 PM, Gavin Shan wrote:
> > On 9/3/20 5:34 PM, Gavin Shan wrote:
> > > On 8/25/20 7:39 PM, Will Deacon wrote:
> > > > Hello folks,
> > > >
> > > > This is version three of the KVM page-table rework that I previously posted
> > > > here:
> > > >
> > > >    v1: https://lore.kernel.org/r/20200730153406.25136-1-will@kernel.org
> > > >    v2: https://lore.kernel.org/r/20200818132818.16065-1-will@kernel.org
> > > >
> > > > Changes since v2 include:
> > > >
> > > >    * Rebased onto -rc2, which includes the conflicting OOM blocking fixes
> > > >    * Dropped the patch trying to "fix" the memcache in kvm_phys_addr_ioremap()
> > > >
> > >
> > > It's really nice work, making the code unified/simplified greatly.
> > > However, it seems it doesn't work well with HugeTLBfs. Please refer
> > > to the following test result and see if you have quick idea, or I
> > > can debug it a bit :)

Nice testing matrix, and thanks for reporting the problem!

> > > Machine         Host                     Guest              Result
> > > ===============================================================
> > > ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 64KB                    64KB     passed
> > >               THP:       disabled
> > >               HugeTLB:   disabled
> > > ---------------------------------------------------------------
> > > ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 64KB                    64KB     passed
> > >               THP:       enabled
> > >               HugeTLB:   disabled
> > > ----------------------------------------------------------------
> > > ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Fail[1]
> > >               PAGE_SIZE: 64KB                    64KB     Fail[1]
> > >               THP:       disabled
> > >               HugeTLB:   enabled
> > > ---------------------------------------------------------------
> > > ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 4KB                     64KB     Passed
> > >               THP:       disabled
> > >               HugeTLB:   disabled
> > > ---------------------------------------------------------------
> > > ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
> > >               PAGE_SIZE: 4KB                     64KB     Passed
> > >               THP:       enabled
> > >               HugeTLB:   disabled
> > > --------------------------------------------------------------
> > > ThunderX2    VA_BITS:   39           PAGE_SIZE: 4KB     Fail[2]
> > >               PAGE_SIZE: 4KB                    64KB     Fail[2]
> > >               THP:       disabled
> > >               HugeTLB:   enabled
> > >
> >
> > I debugged the code and found the issue is caused by the following
> > patch.
> >
> > [PATCH v3 06/21] KVM: arm64: Add support for stage-2 map()/unmap() in generic page-table

(I think this is just a symptom of the page-table being out of whack)

> Sorry that the guest could hang sometimes with above changes. I have no idea what
> has been happening before I'm going to debug for more.. I'm pasting the used command
> and output from guest.

Can you try the diff below, please? I think we can end up sticking down a
huge-page-sized mapping at an unaligned address, which causes us both to
overmap and also to fail to use the huge granule for a block mapping.

Cheers,

Will

--->8

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index f28e03dcb897..3bff942e5f33 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -737,11 +737,11 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
bool exec_fault;
bool device = false;
unsigned long mmu_seq;
- gfn_t gfn = fault_ipa >> PAGE_SHIFT;
struct kvm *kvm = vcpu->kvm;
struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
struct vm_area_struct *vma;
short vma_shift;
+ gfn_t gfn;
kvm_pfn_t pfn;
bool logging_active = memslot_is_logging(memslot);
unsigned long vma_pagesize;
@@ -780,7 +780,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
}

if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
- gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT;
+ fault_ipa &= huge_page_mask(hstate_vma(vma));
+
+ gfn = fault_ipa >> PAGE_SHIFT;
mmap_read_unlock(current->mm);

if (fault_status != FSC_PERM) {