From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C4CA0C32772 for ; Tue, 23 Aug 2022 14:46:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AsFA6k54uSxXwxV5si05rwzzO+R45M4LlZ2JqbEDWQQ=; b=qjDbCcXcOYpn7E zUBuPVWC7tDZfvIDp7RQjOdTbxxo/BgVSSI2hQl49rxVI8fGTUZQhmOcjz+YjyvcblDQfS+ddfhFd tUTWFAC7qf/hbmCIIzoCqDdGmFFuCGYWg42/SciSh58XLF4tRo/MrN61/kND1Rr0MNdtLWDBs3adY 2r6xxxXFHwiyig3YtBdtTWaUbwpKmOczzcmy7LQOY0pNCVGPWfvs+Ecj2eLZUBgIbN40bJLRJT1pr Al7pPlUhQW8CeCLANqxTpqaUQkqyGLzK+MhP/VtiTJ8GdGCQNxgDsaUBvsMd701ICCKTSYg72OxSc SqKTYk0WoxlXOLCma5Vw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQV9I-006hh6-ST; Tue, 23 Aug 2022 14:45:00 +0000 Received: from out1.migadu.com ([91.121.223.63]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQV9E-006hdQ-Hm for linux-arm-kernel@lists.infradead.org; Tue, 23 Aug 2022 14:44:58 +0000 Date: Tue, 23 Aug 2022 09:44:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1661265889; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wYTxuO09xQHamH8HB9DanmNLqgTyS7iN+66gFB/iGcs=; b=KzS2iPpdIzfHDC3yepdSeojEKRrJeBGV61JZxpWQCWAB5NKd2i2D94pZKSuDM3G2eJ+izs YH21AVjrNOZyMSuuQy6w0ir/+3q7zjahK2l6toOyZkSk/VUDgyVAMtk9rvwdQk0szY003u jUrrEx4Pa3dRbNvCt5aPDinF0ofcJvA= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: Marc Zyngier Cc: Gavin Shan , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: Re: [PATCH v1 1/5] KVM: arm64: Enable ring-based dirty memory tracking Message-ID: References: <20220819005601.198436-1-gshan@redhat.com> <20220819005601.198436-2-gshan@redhat.com> <87lerkwtm5.wl-maz@kernel.org> <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> <87fshovtu0.wl-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87fshovtu0.wl-maz@kernel.org> X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220823_074457_202220_ABF8D710 X-CRM114-Status: GOOD ( 26.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Aug 22, 2022 at 10:42:15PM +0100, Marc Zyngier wrote: > Hi Gavin, > > On Mon, 22 Aug 2022 02:58:20 +0100, > Gavin Shan wrote: > > > > Hi Marc, > > > > On 8/19/22 6:00 PM, Marc Zyngier wrote: > > > On Fri, 19 Aug 2022 01:55:57 +0100, > > > Gavin Shan wrote: > > >> > > >> The ring-based dirty memory tracking has been available and enabled > > >> on x86 for a while. The feature is beneficial when the number of > > >> dirty pages is small in a checkpointing system or live migration > > >> scenario. More details can be found from fb04a1eddb1a ("KVM: X86: > > >> Implement ring-based dirty memory tracking"). > > >> > > >> This enables the ring-based dirty memory tracking on ARM64. It's > > >> notable that no extra reserved ring entries are needed on ARM64 > > >> because the huge pages are always split into base pages when page > > >> dirty tracking is enabled. > > > > > > Can you please elaborate on this? Adding a per-CPU ring of course > > > results in extra memory allocation, so there must be a subtle > > > x86-specific detail that I'm not aware of... > > > > > > > Sure. I guess it's helpful to explain how it works in next revision. > > Something like below: > > > > This enables the ring-based dirty memory tracking on ARM64. The feature > > is enabled by CONFIG_HAVE_KVM_DIRTY_RING, detected and enabled by > > CONFIG_HAVE_KVM_DIRTY_RING. A ring buffer is created on every vcpu and > > each entry is described by 'struct kvm_dirty_gfn'. The ring buffer is > > pushed by host when page becomes dirty and pulled by userspace. A vcpu > > exit is forced when the ring buffer becomes full. The ring buffers on > > all vcpus can be reset by ioctl command KVM_RESET_DIRTY_RINGS. > > > > Yes, I think so. Adding a per-CPU ring results in extra memory allocation. > > However, it's avoiding synchronization among multiple vcpus when dirty > > pages happen on multiple vcpus. More discussion can be found from [1] > > Oh, I totally buy the relaxation of the synchronisation (though I > doubt this will have any visible effect until we have something like > Oliver's patches to allow parallel faulting). > Heh, yeah I need to get that out the door. I'll also note that Gavin's changes are still relevant without that series, as we do write unprotect in parallel at PTE granularity after commit f783ef1c0e82 ("KVM: arm64: Add fast path to handle permission relaxation during dirty logging"). -- Thanks, Oliver _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel