From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F2CF9C25B0E for ; Fri, 19 Aug 2022 08:01:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZmsQ3XTCPO4jzyPWkTV5xPc5PZCJ03UwR7sMMiunOro=; b=3zfeqqQHLRNnCv xyARQQKT0YU37APusZMZ10ZqZ55pNZaCIpFaj+/4LPo7wEkcTuLItGrt2NiGc1LHXIz7WZmhffCGR KF9hMGulln3eRhnDEOZvlkCoLeF8IIFrfmnEo6wjk6RmWAFAA7bo3d59bqsUZaFLfMxT/GbLgqyOb za83izRSYotVhwc8vZBZKnFc0dJH6ycvq3ib3s1gxI8cwEhMSF941a0x3Ig7Bd1qSUIlDZahNAqI5 atlsTEd40+zPAT9ECLZeuIxU+nXKLJOpGab54sSggoS5P6tBpXSM3NPWbjQ/USBs6A5gmHwVzzLha pwvFMIrAQ0HzyaVb3drQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oOwvb-003itQ-Co; Fri, 19 Aug 2022 08:00:27 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oOwvX-003iqt-Dq for linux-arm-kernel@lists.infradead.org; Fri, 19 Aug 2022 08:00:25 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5E3BD616C7; Fri, 19 Aug 2022 08:00:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6791C433B5; Fri, 19 Aug 2022 08:00:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1660896021; bh=dXB7Dmr6Olnqm8i/ag5x25/j/xOuKOYp7Zqy3xYbF1k=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=bPzGRNloxET7jCjbmwuPekfurJU+YIanGSes6+FQxRyOaXDN7NleDvVk0kCSGm+mj Nvy+u3VuCbER8OL+RbU4KnLqA6HRIfjiQ7u8R5gsfEHDNpnzogYoaC5TjC7KhI9nGl 3u6j0LMPpZx/z/OYomSHy1cBWXgHYDxR1vDcP0ZfPUIOgSJjWLT9uKP94QaW4rEyq0 Uw510IEA5cxLKQKuP025ixhomp3R0zWPbgu8jqG/fWOoVRppxVNpkOQSMy+fdBPeIl fYw4MU6pTlU+kolvweUV5rWvIr/LtPt8krly8NS3/oMs0OxtW71SdawKlOlX4geM01 EzczQKfP6wZiA== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oOwvT-004D7a-Dh; Fri, 19 Aug 2022 09:00:19 +0100 Date: Fri, 19 Aug 2022 09:00:18 +0100 Message-ID: <87lerkwtm5.wl-maz@kernel.org> From: Marc Zyngier To: Gavin Shan Cc: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: Re: [PATCH v1 1/5] KVM: arm64: Enable ring-based dirty memory tracking In-Reply-To: <20220819005601.198436-2-gshan@redhat.com> References: <20220819005601.198436-1-gshan@redhat.com> <20220819005601.198436-2-gshan@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gshan@redhat.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220819_010023_584409_E7E8CF1D X-CRM114-Status: GOOD ( 30.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 19 Aug 2022 01:55:57 +0100, Gavin Shan wrote: > > The ring-based dirty memory tracking has been available and enabled > on x86 for a while. The feature is beneficial when the number of > dirty pages is small in a checkpointing system or live migration > scenario. More details can be found from fb04a1eddb1a ("KVM: X86: > Implement ring-based dirty memory tracking"). > > This enables the ring-based dirty memory tracking on ARM64. It's > notable that no extra reserved ring entries are needed on ARM64 > because the huge pages are always split into base pages when page > dirty tracking is enabled. Can you please elaborate on this? Adding a per-CPU ring of course results in extra memory allocation, so there must be a subtle x86-specific detail that I'm not aware of... > > Signed-off-by: Gavin Shan > --- > Documentation/virt/kvm/api.rst | 2 +- > arch/arm64/include/uapi/asm/kvm.h | 1 + > arch/arm64/kvm/Kconfig | 1 + > arch/arm64/kvm/arm.c | 8 ++++++++ > 4 files changed, 11 insertions(+), 1 deletion(-) > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > index abd7c32126ce..19fa1ac017ed 100644 > --- a/Documentation/virt/kvm/api.rst > +++ b/Documentation/virt/kvm/api.rst > @@ -8022,7 +8022,7 @@ regardless of what has actually been exposed through the CPUID leaf. > 8.29 KVM_CAP_DIRTY_LOG_RING > --------------------------- > > -:Architectures: x86 > +:Architectures: x86, arm64 > :Parameters: args[0] - size of the dirty log ring > > KVM is capable of tracking dirty memory using ring buffers that are > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h > index 3bb134355874..7e04b0b8d2b2 100644 > --- a/arch/arm64/include/uapi/asm/kvm.h > +++ b/arch/arm64/include/uapi/asm/kvm.h > @@ -43,6 +43,7 @@ > #define __KVM_HAVE_VCPU_EVENTS > > #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 > +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 For context, the documentation says: - if KVM_CAP_DIRTY_LOG_RING is available, a number of pages at KVM_DIRTY_LOG_PAGE_OFFSET * PAGE_SIZE. [...] What is the reason for picking this particular value? > > #define KVM_REG_SIZE(id) \ > (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT)) > diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig > index 815cc118c675..0309b2d0f2da 100644 > --- a/arch/arm64/kvm/Kconfig > +++ b/arch/arm64/kvm/Kconfig > @@ -32,6 +32,7 @@ menuconfig KVM > select KVM_VFIO > select HAVE_KVM_EVENTFD > select HAVE_KVM_IRQFD > + select HAVE_KVM_DIRTY_RING > select HAVE_KVM_MSI > select HAVE_KVM_IRQCHIP > select HAVE_KVM_IRQ_ROUTING > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index 986cee6fbc7f..3de6b9b39db7 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -866,6 +866,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) > if (!ret) > ret = 1; > > + /* Force vcpu exit if its dirty ring is soft-full */ > + if (unlikely(vcpu->kvm->dirty_ring_size && > + kvm_dirty_ring_soft_full(&vcpu->dirty_ring))) { > + vcpu->run->exit_reason = KVM_EXIT_DIRTY_RING_FULL; > + trace_kvm_dirty_ring_exit(vcpu); > + ret = 0; > + } > + Why can't this be moved to kvm_vcpu_exit_request() instead? I would also very much like the check to be made a common helper with x86. A seemingly approach would be to make this a request on dirty log insertion, and avoid the whole "check the log size" on every run, which adds pointless overhead to unsuspecting users (aka everyone). Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel