From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 710F1C433ED for ; Wed, 14 Apr 2021 09:07:08 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D536361176 for ; Wed, 14 Apr 2021 09:07:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D536361176 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kV7Qehn+Pdot0u3uGXSMymJweuVtlvFnvE467/yFZ6I=; b=mPOfEq10jjeZTSLwpgXTo8mr8 KSg2nw+jCVLquJehoyf9rEgas6B6CJh+7/jFgrATtO9+arh1Qw4IHnTgdw8wtRWOGTHTS3xTFln8X 1j+2Re3q4tZTQRyTEtu1DIfSfr0QLAtREk7ApHNYiuj2fAZvatGMmTMNbIGyIZAcXnkY4GaNvgX8b MpqXYJfbSCsCL5nXZmUgTcK3BMaWcgTbWuLmhExJb3EliSRN2OgeT8SVmZijbVIn/r+ZtIg720tz2 jWpVt8AkJyNnLEiDFOzdxzZFeOFq5fznV1Ac3CrydjSowjSeiUOR9Wfv0ymal4JQPZuw2F0yu7r4i XG8+4qiSQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWbSU-00C5jG-AN; Wed, 14 Apr 2021 09:05:14 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWbSR-00C5iz-Fv for linux-arm-kernel@desiato.infradead.org; Wed, 14 Apr 2021 09:05:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:MIME-Version:References: In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=nulrpTQErVYfdj9rHVB/DYt2sUAP84+fBtNthfkp3Bc=; b=VMn79wOqWL1QApTDKLi3TjVhYA 2dpNThKFyEQLHX+vdZXBUm94YOWO310jOEcDihGRBagiHRQ3ALSQZwf40odHMPNqzDF6h9AlNSUtQ JAC/Z+d8bI352X+vVCWVxnRhfsdc0zbp7GpG4wmPu57fZIzxzQ2XLQRp2lFaoJeT003dJ+3aXIT2o UIH4vMxgMZgdu6xmQfXmW5TVsMzIwS9NMWv5ZOUql+GBeVW/RRNUm5G6UXAmz2OMVUYo6gx7hO8ZP nGRuz0wvBiOXn2Tm3jpYeedbghqNpc5Ebzg/mFtfG9iXe9aGEXollKNk14Zp1kIF5hutYH74ELVRS QKfHkSJg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWbSO-007dl6-Cj for linux-arm-kernel@lists.infradead.org; Wed, 14 Apr 2021 09:05:10 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 034A761166; Wed, 14 Apr 2021 09:05:08 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lWbSL-007OqK-TI; Wed, 14 Apr 2021 10:05:06 +0100 Date: Wed, 14 Apr 2021 10:05:05 +0100 Message-ID: <87pmyxme2m.wl-maz@kernel.org> From: Marc Zyngier To: Keqian Zhu Cc: , , , , , Santosh Shukla Subject: Re: [PATCH v3 2/2] kvm/arm64: Try stage2 block mapping for host device MMIO In-Reply-To: <20210414065109.8616-3-zhukeqian1@huawei.com> References: <20210414065109.8616-1-zhukeqian1@huawei.com> <20210414065109.8616-3-zhukeqian1@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: zhukeqian1@huawei.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, wanghaibin.wang@huawei.com, sashukla@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210414_020508_503307_C3408267 X-CRM114-Status: GOOD ( 40.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org + Santosh, who found some interesting bugs in that area before. On Wed, 14 Apr 2021 07:51:09 +0100, Keqian Zhu wrote: > > The MMIO region of a device maybe huge (GB level), try to use > block mapping in stage2 to speedup both map and unmap. > > Compared to normal memory mapping, we should consider two more > points when try block mapping for MMIO region: > > 1. For normal memory mapping, the PA(host physical address) and > HVA have same alignment within PUD_SIZE or PMD_SIZE when we use > the HVA to request hugepage, so we don't need to consider PA > alignment when verifing block mapping. But for device memory > mapping, the PA and HVA may have different alignment. > > 2. For normal memory mapping, we are sure hugepage size properly > fit into vma, so we don't check whether the mapping size exceeds > the boundary of vma. But for device memory mapping, we should pay > attention to this. > > This adds device_rough_page_shift() to check these two points when > selecting block mapping size. > > Signed-off-by: Keqian Zhu > --- > arch/arm64/kvm/mmu.c | 37 +++++++++++++++++++++++++++++++++---- > 1 file changed, 33 insertions(+), 4 deletions(-) > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > index c59af5ca01b0..1a6d96169d60 100644 > --- a/arch/arm64/kvm/mmu.c > +++ b/arch/arm64/kvm/mmu.c > @@ -624,6 +624,31 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb) > send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current); > } > > +/* > + * Find a max mapping size that properly insides the vma. And hva and pa must > + * have the same alignment to this mapping size. It's rough as there are still > + * other restrictions, will be checked by fault_supports_stage2_huge_mapping(). > + */ > +static short device_rough_page_shift(struct vm_area_struct *vma, > + unsigned long hva) My earlier question still stands. Under which circumstances would this function return something that is *not* the final mapping size? I really don't see a reason why this would not return the final mapping size. > +{ > + phys_addr_t pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); > + > +#ifndef __PAGETABLE_PMD_FOLDED > + if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && > + ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start && > + ALIGN(hva, PUD_SIZE) <= vma->vm_end) > + return PUD_SHIFT; > +#endif > + > + if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && > + ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start && > + ALIGN(hva, PMD_SIZE) <= vma->vm_end) > + return PMD_SHIFT; > + > + return PAGE_SHIFT; > +} > + > static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, > unsigned long hva, > unsigned long map_size) > @@ -769,7 +794,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > return -EFAULT; > } > > - /* Let's check if we will get back a huge page backed by hugetlbfs */ > + /* > + * Let's check if we will get back a huge page backed by hugetlbfs, or > + * get block mapping for device MMIO region. > + */ > mmap_read_lock(current->mm); > vma = find_vma_intersection(current->mm, hva, hva + 1); > if (unlikely(!vma)) { > @@ -780,11 +808,12 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > if (is_vm_hugetlb_page(vma)) > vma_shift = huge_page_shift(hstate_vma(vma)); > + else if (vma->vm_flags & VM_PFNMAP) > + vma_shift = device_rough_page_shift(vma, hva); What prevents a VMA from having both VM_HUGETLB and VM_PFNMAP? This is pretty unlikely, but I'd like to see this case catered for. > else > vma_shift = PAGE_SHIFT; > > - if (logging_active || > - (vma->vm_flags & VM_PFNMAP)) { > + if (logging_active) { > force_pte = true; > vma_shift = PAGE_SHIFT; > } > @@ -855,7 +884,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > if (kvm_is_device_pfn(pfn)) { > device = true; > - force_pte = true; > + force_pte = (vma_pagesize == PAGE_SIZE); Why do we need to set force_pte if we are already dealing with PAGE_SIZE? I guess you are doing this for the sake of avoiding the call to transparent_hugepage_adjust(), right? I'd rather you simply don't try to upgrade a device mapping by explicitly checking for this and keep force_pte for *memory* exclusively. Santosh, can you please take a look at this series and try to see if the problem you fixed in [1] (which ended up as commit 91a2c34b7d6f) is still OK with this series? > } else if (logging_active && !write_fault) { > /* > * Only actually map the page as writable if this was a write Thanks, M. [1] https://lore.kernel.org/kvmarm/1603711447-11998-1-git-send-email-sashukla@nvidia.com/ -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel