Re: [PATCH v3 00/21] KVM: arm64: Rewrite page-table code and fault handling

Gavin Shan
2020-09-04T00:53:26Z

Hi Will,

On 9/3/20 10:16 PM, Will Deacon wrote:
> On Thu, Sep 03, 2020 at 09:48:18PM +1000, Gavin Shan wrote:
>> On 9/3/20 9:13 PM, Gavin Shan wrote:
>>> On 9/3/20 5:34 PM, Gavin Shan wrote:
>>>> On 8/25/20 7:39 PM, Will Deacon wrote:
>>>>> Hello folks,
>>>>>
>>>>> This is version three of the KVM page-table rework that I previously posted
>>>>> here:
>>>>>
>>>>>    v1: https://lore.kernel.org/r/20200730153406.25136-1-will@kernel.org
>>>>>    v2: https://lore.kernel.org/r/20200818132818.16065-1-will@kernel.org
>>>>>
>>>>> Changes since v2 include:
>>>>>
>>>>>    * Rebased onto -rc2, which includes the conflicting OOM blocking fixes
>>>>>    * Dropped the patch trying to "fix" the memcache in kvm_phys_addr_ioremap()
>>>>>
>>>>
>>>> It's really nice work, making the code unified/simplified greatly.
>>>> However, it seems it doesn't work well with HugeTLBfs. Please refer
>>>> to the following test result and see if you have quick idea, or I
>>>> can debug it a bit :)
>
> Nice testing matrix, and thanks for reporting the problem!
>
>>>> Machine         Host                     Guest              Result
>>>> ===============================================================
>>>> ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
>>>>               PAGE_SIZE: 64KB                    64KB     passed
>>>>               THP:       disabled
>>>>               HugeTLB:   disabled
>>>> ---------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Passed
>>>>               PAGE_SIZE: 64KB                    64KB     passed
>>>>               THP:       enabled
>>>>               HugeTLB:   disabled
>>>> ----------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   42           PAGE_SIZE:  4KB     Fail[1]
>>>>               PAGE_SIZE: 64KB                    64KB     Fail[1]
>>>>               THP:       disabled
>>>>               HugeTLB:   enabled
>>>> ---------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
>>>>               PAGE_SIZE: 4KB                     64KB     Passed
>>>>               THP:       disabled
>>>>               HugeTLB:   disabled
>>>> ---------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   39           PAGE_SIZE:  4KB     Passed
>>>>               PAGE_SIZE: 4KB                     64KB     Passed
>>>>               THP:       enabled
>>>>               HugeTLB:   disabled
>>>> --------------------------------------------------------------
>>>> ThunderX2    VA_BITS:   39           PAGE_SIZE: 4KB     Fail[2]
>>>>               PAGE_SIZE: 4KB                    64KB     Fail[2]
>>>>               THP:       disabled
>>>>               HugeTLB:   enabled
>>>>
>>>
>>> I debugged the code and found the issue is caused by the following
>>> patch.
>>>
>>> [PATCH v3 06/21] KVM: arm64: Add support for stage-2 map()/unmap() in generic page-table
>
> (I think this is just a symptom of the page-table being out of whack)
>
>> Sorry that the guest could hang sometimes with above changes. I have no idea what
>> has been happening before I'm going to debug for more.. I'm pasting the used command
>> and output from guest.
>
> Can you try the diff below, please? I think we can end up sticking down a
> huge-page-sized mapping at an unaligned address, which causes us both to
> overmap and also to fail to use the huge granule for a block mapping.
>

Since the the following changes have been folded to v4, I reran the test cases
on v4 and everything works fine.

Thanks,
Gavin

> --->8
>
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index f28e03dcb897..3bff942e5f33 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -737,11 +737,11 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> bool exec_fault;
> bool device = false;
> unsigned long mmu_seq;
> - gfn_t gfn = fault_ipa >> PAGE_SHIFT;
> struct kvm *kvm = vcpu->kvm;
> struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
> struct vm_area_struct *vma;
> short vma_shift;
> + gfn_t gfn;
> kvm_pfn_t pfn;
> bool logging_active = memslot_is_logging(memslot);
> unsigned long vma_pagesize;
> @@ -780,7 +780,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> }
>
> if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE)
> - gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT;
> + fault_ipa &= huge_page_mask(hstate_vma(vma));
> +
> + gfn = fault_ipa >> PAGE_SHIFT;
> mmap_read_unlock(current->mm);
>
> if (fault_status != FSC_PERM) {