From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4F8BC433ED for ; Wed, 7 Apr 2021 13:20:31 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9673361394 for ; Wed, 7 Apr 2021 13:20:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9673361394 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5Zc1ryuQ3nxY71CdOIC8vyNlN4hVBaeIE/KQ44xN1gU=; b=LuKSvKzNssIAb/MZKMcV7bspc UGD/Ad8nT8scaIuJqyKYnO5gizPo07jRHuuhvg4NDscmexh7dY99OvfC0RJ7FRYjtuY/TGPJmKOo9 lSeEdOXvWOJlfQfRP7tD+WQ7jwqYRJ4PK0g+WBRl07djDQrNmzWfYgHs4Mq7dyWml6fJGDNsKfkm/ OU2GQ8aYUcJP6dNsXho/pgn2p6qgsoUk3DbhJGTompfVsP2MC+GjugIficOryfDibKwsx6gVO+K7n kcA/wGuPmwsFuvul8HL/30aEwX62b1Jeme/bx6Gn59eY6JWBakuCPf5/1i35Pc6MIp9aryUYvT9qa nDFy89CoA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lU84Z-0051mG-P5; Wed, 07 Apr 2021 13:18:21 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lU84O-0051lN-V4 for linux-arm-kernel@lists.infradead.org; Wed, 07 Apr 2021 13:18:13 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4475E61369; Wed, 7 Apr 2021 13:18:07 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lU84L-0064x4-8f; Wed, 07 Apr 2021 14:18:05 +0100 Date: Wed, 07 Apr 2021 14:18:04 +0100 Message-ID: <878s5up71v.wl-maz@kernel.org> From: Marc Zyngier To: Keqian Zhu Cc: , , , , Will Deacon , Catalin Marinas , Mark Rutland , James Morse , Suzuki K Poulose , Julien Thierry , , , , Subject: Re: [RFC PATCH v2 2/2] kvm/arm64: Try stage2 block mapping for host device MMIO In-Reply-To: <20210316134338.18052-3-zhukeqian1@huawei.com> References: <20210316134338.18052-1-zhukeqian1@huawei.com> <20210316134338.18052-3-zhukeqian1@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: zhukeqian1@huawei.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, james.morse@arm.com, suzuki.poulose@arm.com, julien.thierry.kdev@gmail.com, wanghaibin.wang@huawei.com, jiangkunkun@huawei.com, yuzenghui@huawei.com, lushenming@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210407_141811_655617_468E4D32 X-CRM114-Status: GOOD ( 42.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 16 Mar 2021 13:43:38 +0000, Keqian Zhu wrote: > > The MMIO region of a device maybe huge (GB level), try to use > block mapping in stage2 to speedup both map and unmap. > > Compared to normal memory mapping, we should consider two more > points when try block mapping for MMIO region: > > 1. For normal memory mapping, the PA(host physical address) and > HVA have same alignment within PUD_SIZE or PMD_SIZE when we use > the HVA to request hugepage, so we don't need to consider PA > alignment when verifing block mapping. But for device memory > mapping, the PA and HVA may have different alignment. > > 2. For normal memory mapping, we are sure hugepage size properly > fit into vma, so we don't check whether the mapping size exceeds > the boundary of vma. But for device memory mapping, we should pay > attention to this. > > This adds device_rough_page_shift() to check these two points when > selecting block mapping size. > > Signed-off-by: Keqian Zhu > --- > > Mainly for RFC, not fully tested. I will fully test it when the > code logic is well accepted. > > --- > arch/arm64/kvm/mmu.c | 42 ++++++++++++++++++++++++++++++++++++++---- > 1 file changed, 38 insertions(+), 4 deletions(-) > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > index c59af5ca01b0..224aa15eb4d9 100644 > --- a/arch/arm64/kvm/mmu.c > +++ b/arch/arm64/kvm/mmu.c > @@ -624,6 +624,36 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb) > send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current); > } > > +/* > + * Find a mapping size that properly insides the intersection of vma and > + * memslot. And hva and pa have the same alignment to this mapping size. > + * It's rough because there are still other restrictions, which will be > + * checked by the following fault_supports_stage2_huge_mapping(). I don't think these restrictions make complete sense to me. If this is a PFNMAP VMA, we should use the biggest mapping size that covers the VMA, and not more than the VMA. > + */ > +static short device_rough_page_shift(struct kvm_memory_slot *memslot, > + struct vm_area_struct *vma, > + unsigned long hva) > +{ > + size_t size = memslot->npages * PAGE_SIZE; > + hva_t sec_start = max(memslot->userspace_addr, vma->vm_start); > + hva_t sec_end = min(memslot->userspace_addr + size, vma->vm_end); > + phys_addr_t pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); > + > +#ifndef __PAGETABLE_PMD_FOLDED > + if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && > + ALIGN_DOWN(hva, PUD_SIZE) >= sec_start && > + ALIGN(hva, PUD_SIZE) <= sec_end) > + return PUD_SHIFT; > +#endif > + > + if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && > + ALIGN_DOWN(hva, PMD_SIZE) >= sec_start && > + ALIGN(hva, PMD_SIZE) <= sec_end) > + return PMD_SHIFT; > + > + return PAGE_SHIFT; > +} > + > static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, > unsigned long hva, > unsigned long map_size) > @@ -769,7 +799,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > return -EFAULT; > } > > - /* Let's check if we will get back a huge page backed by hugetlbfs */ > + /* > + * Let's check if we will get back a huge page backed by hugetlbfs, or > + * get block mapping for device MMIO region. > + */ > mmap_read_lock(current->mm); > vma = find_vma_intersection(current->mm, hva, hva + 1); > if (unlikely(!vma)) { > @@ -780,11 +813,12 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > if (is_vm_hugetlb_page(vma)) > vma_shift = huge_page_shift(hstate_vma(vma)); > + else if (vma->vm_flags & VM_PFNMAP) > + vma_shift = device_rough_page_shift(memslot, vma, hva); > else > vma_shift = PAGE_SHIFT; > > - if (logging_active || > - (vma->vm_flags & VM_PFNMAP)) { > + if (logging_active) { > force_pte = true; > vma_shift = PAGE_SHIFT; But why should we downgrade to page-size mappings if logging? This is a device, and you aren't moving the device around, are you? Or is your device actually memory with a device mapping that you are trying to migrate? > } > @@ -855,7 +889,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > if (kvm_is_device_pfn(pfn)) { > device = true; > - force_pte = true; > + force_pte = (vma_pagesize == PAGE_SIZE); > } else if (logging_active && !write_fault) { > /* > * Only actually map the page as writable if this was a write > -- > 2.19.1 > > Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel