From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC08FC352BE for ; Fri, 17 Apr 2020 09:10:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA4A8221EA for ; Fri, 17 Apr 2020 09:10:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ZU2siCAn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA4A8221EA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=blpb5uppUYr2ZoVMuGCyIcuQwqSRHQ0/lzI1raAj7E4=; b=ZU2siCAnBFbrKq 8apMcwZgBNAMyDht312JqsTcPggbFbNa7Svk30iVAJ4XRPVw0NqHKIribktEA+GhANMPOeJ2sPzAt zxKbkKARgFd8Zk4xdO2gk3o4v0q+bJLHnjgayuYBnRaB5MydoO8gXmPxR+AzE+fm7puk1PlELv2sJ hGjh7OFeCRVVjdGGfCjrEon+s9lSM5pmgel7N6jqpBwTa9GMa4DohMY/66fVvI42fwVffxQVNHDx4 ZPrPwp3oJZkwhkm0ioqA086DlOLyuV5X+tjwSkTx/jHGNHbEq5BzMWdsvspK6BVRnHP8RD73xIGnT /TQREqhJaB0/rb9KqmFQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jPN0z-00055u-LU; Fri, 17 Apr 2020 09:10:25 +0000 Received: from szxga04-in.huawei.com ([45.249.212.190] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jPN0v-00054y-P5 for linux-arm-kernel@lists.infradead.org; Fri, 17 Apr 2020 09:10:23 +0000 Received: from DGGEMS409-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 2063C4944C6BE347FA91; Fri, 17 Apr 2020 17:10:15 +0800 (CST) Received: from [127.0.0.1] (10.173.221.230) by DGGEMS409-HUB.china.huawei.com (10.3.19.209) with Microsoft SMTP Server id 14.3.487.0; Fri, 17 Apr 2020 17:10:07 +0800 Subject: Re: [PATCH v2] KVM/arm64: Support enabling dirty log gradually in small chunks To: Paolo Bonzini , Marc Zyngier References: <20200413122023.52583-1-zhukeqian1@huawei.com> <20200416160939.7e9c1621@why> <442f288e-2934-120c-4994-5357e3e9216b@redhat.com> From: zhukeqian Message-ID: <3e3ce7dd-af13-6daa-9ccf-747405d448cc@huawei.com> Date: Fri, 17 Apr 2020 17:10:05 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: <442f288e-2934-120c-4994-5357e3e9216b@redhat.com> X-Originating-IP: [10.173.221.230] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200417_021021_990613_0FE59654 X-CRM114-Status: GOOD ( 16.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, Sean Christopherson , James Morse , Julien Thierry , Jay Zhou , wanghaibin.wang@huawei.com, Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Paolo, On 2020/4/16 23:55, Paolo Bonzini wrote: > On 16/04/20 17:09, Marc Zyngier wrote: >> On Wed, 15 Apr 2020 18:13:56 +0200 >> Paolo Bonzini wrote: >> >>> On 13/04/20 14:20, Keqian Zhu wrote: >>>> There is already support of enabling dirty log graually in small chunks >>>> for x86 in commit 3c9bd4006bfc ("KVM: x86: enable dirty log gradually in >>>> small chunks"). This adds support for arm64. >>>> >>>> x86 still writes protect all huge pages when DIRTY_LOG_INITIALLY_ALL_SET >>>> is eanbled. However, for arm64, both huge pages and normal pages can be >>>> write protected gradually by userspace. >>>> >>>> Under the Huawei Kunpeng 920 2.6GHz platform, I did some tests on 128G >>>> Linux VMs with different page size. The memory pressure is 127G in each >>>> case. The time taken of memory_global_dirty_log_start in QEMU is listed >>>> below: >>>> >>>> Page Size Before After Optimization >>>> 4K 650ms 1.8ms >>>> 2M 4ms 1.8ms >>>> 1G 2ms 1.8ms >>>> >>>> Besides the time reduction, the biggest income is that we will minimize >>>> the performance side effect (because of dissloving huge pages and marking >>>> memslots dirty) on guest after enabling dirty log. >>>> >>>> Signed-off-by: Keqian Zhu >>>> --- >>>> Documentation/virt/kvm/api.rst | 2 +- >>>> arch/arm64/include/asm/kvm_host.h | 3 +++ >>>> virt/kvm/arm/mmu.c | 12 ++++++++++-- >>>> 3 files changed, 14 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst >>>> index efbbe570aa9b..0017f63fa44f 100644 >>>> --- a/Documentation/virt/kvm/api.rst >>>> +++ b/Documentation/virt/kvm/api.rst >>>> @@ -5777,7 +5777,7 @@ will be initialized to 1 when created. This also improves performance because >>>> dirty logging can be enabled gradually in small chunks on the first call >>>> to KVM_CLEAR_DIRTY_LOG. KVM_DIRTY_LOG_INITIALLY_SET depends on >>>> KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE (it is also only available on >>>> -x86 for now). >>>> +x86 and arm64 for now). >>>> >>>> KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 was previously available under the name >>>> KVM_CAP_MANUAL_DIRTY_LOG_PROTECT, but the implementation had bugs that make >>>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h >>>> index 32c8a675e5a4..a723f84fab83 100644 >>>> --- a/arch/arm64/include/asm/kvm_host.h >>>> +++ b/arch/arm64/include/asm/kvm_host.h >>>> @@ -46,6 +46,9 @@ >>>> #define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3) >>>> #define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4) >>>> >>>> +#define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ >>>> + KVM_DIRTY_LOG_INITIALLY_SET) >>>> + >>>> DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); >>>> >>>> extern unsigned int kvm_sve_max_vl; >>>> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c >>>> index e3b9ee268823..1077f653a611 100644 >>>> --- a/virt/kvm/arm/mmu.c >>>> +++ b/virt/kvm/arm/mmu.c >>>> @@ -2265,8 +2265,16 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, >>>> * allocated dirty_bitmap[], dirty pages will be be tracked while the >>>> * memory slot is write protected. >>>> */ >>>> - if (change != KVM_MR_DELETE && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) >>>> - kvm_mmu_wp_memory_region(kvm, mem->slot); >>>> + if (change != KVM_MR_DELETE && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { >>>> + /* >>>> + * If we're with initial-all-set, we don't need to write >>>> + * protect any pages because they're all reported as dirty. >>>> + * Huge pages and normal pages will be write protect gradually. >>>> + */ >>>> + if (!kvm_dirty_log_manual_protect_and_init_set(kvm)) { >>>> + kvm_mmu_wp_memory_region(kvm, mem->slot); >>>> + } >>>> + } >>>> } >>>> >>>> int kvm_arch_prepare_memory_region(struct kvm *kvm, >>>> >>> >>> Marc, what is the status of this patch? >> >> I just had a look at it. Is there any urgency for merging it? > > No, I thought I was still replying to the v1. Sorry that patch v1 is dropped. Because I realized that stage2 page tables will be unmapped during VM reboot, or they are not established soon after migration, so stage2 page tables can not be used to decide whether a page is needed to migrate. Thanks, Keqian > > Paolo > > > . > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel