From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C4EDEC32772 for ; Tue, 23 Aug 2022 13:59:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IwMTIy7mfe8t4Dm8nVwh55loejCWeT/r2uEhDsAcsiY=; b=bjFJf0JIB/OeXF Du5WZuz+hXayjBeoDXrPFADKDTrjR6ZLI+on+PRyaiqKUN/fXVtaGHDA6NIgZ+fX10zYb56p+aLXi nfnkeM+zYJbajR+6oS61QhP86mOKMGsgJMWBoQXoeQurDmW4sRcJHD/VFVaDxmLDX2nXIsau0SHfE ZLf/V1ykhemm2wV5sNvx879onqjqn5yBT95fpsZbB4o11cP7ffOxzhz2hAW9nAuYXIKJEe8Lfwg6r 4cUUYDLC0zRgrYbzLSZsXIjwdfC/f/LhL46IaEYQxzxhneo4mReKhb4/fE5xTvkEpVAvbRfNsTKQs kEEQOYyVJxhlh2cABnxw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQUQH-006UQR-8A; Tue, 23 Aug 2022 13:58:29 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQUQD-006UOk-C3 for linux-arm-kernel@lists.infradead.org; Tue, 23 Aug 2022 13:58:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1661263104; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YHVefDGXU83xWUJyLerizfLITJD1h0bvpZUJLhnlk24=; b=AXO1SNmNaxNAq3ngFOkjpwKd9OjFxUtQlCxgejQIg8YZhdDKJzOyAzsl0mGVGXKUuO3hpU dxBSGLCI7QSqq5QwrWgBYdTx5HvdDZpa7Fm4cD8qFL0wMh389YJCml7DFA11upm7Hy0FHT o9nsIdQoVq8WoDhDI1ANKETuLGKrX5A= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-612-9g-Nzuy7O7SCmZGP8zOlTQ-1; Tue, 23 Aug 2022 09:58:23 -0400 X-MC-Unique: 9g-Nzuy7O7SCmZGP8zOlTQ-1 Received: by mail-qv1-f71.google.com with SMTP id c10-20020a0ce64a000000b00496b34088f3so7516122qvn.15 for ; Tue, 23 Aug 2022 06:58:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc; bh=YHVefDGXU83xWUJyLerizfLITJD1h0bvpZUJLhnlk24=; b=IrkE8MoG1LEyudfwnIYWGs27TfjUgvS8c6otpiuzSFIBE6DUy5IdqZrbBYB6GQuRhf MAfQtuzayqPYeg9/qb97Ydoi3pbO1+JgJYDpoHE6mAWhFspov0ZLkT5mSZucdNb4EoGV MPuFkV0MAeM/niA1YpK0kLGJKpqEd3YYxIYwtmovon66ebY9vRCb7xmlfCMICAXT5pLX WN5lnf6kbiq6rcB3MEjUzL0e05BLRr3FqbWqDf+hOOzXSBH3qCIkvav0WlyAwBgoq8h6 gLONDKqMYz95uwyvGhtmOX/adYdDnV5jIW0vepM7yU+KYaB2dXS7vM2Uqjwiw0jaGBfJ IV2w== X-Gm-Message-State: ACgBeo3051SGejDowOGZZvBTYRieO3XdJLzRyZUn68HtyPPDZ337FBUs QQvVcfkUoVRUuM0AGVSXN2PhlXb4mZ28/whgjKtmRbh8mp7Y29wXAAk+4iGxNf7B4zvv6wC8Ydk kzFVUSwP4SRw7pzMo/NBWYkscqq3plZ5FIng= X-Received: by 2002:ac8:5b15:0:b0:343:6789:193a with SMTP id m21-20020ac85b15000000b003436789193amr19117278qtw.647.1661263102675; Tue, 23 Aug 2022 06:58:22 -0700 (PDT) X-Google-Smtp-Source: AA6agR7gd2yWDFVr17SP2RU0l4eAn5No/A726n2hPHBkND28yI2ySau6/YWpTWAdi0/4trPHGWQO6g== X-Received: by 2002:ac8:5b15:0:b0:343:6789:193a with SMTP id m21-20020ac85b15000000b003436789193amr19117247qtw.647.1661263102418; Tue, 23 Aug 2022 06:58:22 -0700 (PDT) Received: from xz-m1.local (bras-base-aurron9127w-grc-35-70-27-3-10.dsl.bell.ca. [70.27.3.10]) by smtp.gmail.com with ESMTPSA id g7-20020a05620a40c700b006a6ebde4799sm12544646qko.90.2022.08.23.06.58.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Aug 2022 06:58:21 -0700 (PDT) Date: Tue, 23 Aug 2022 09:58:19 -0400 From: Peter Xu To: Gavin Shan Cc: Marc Zyngier , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: Re: [PATCH v1 1/5] KVM: arm64: Enable ring-based dirty memory tracking Message-ID: References: <20220819005601.198436-1-gshan@redhat.com> <20220819005601.198436-2-gshan@redhat.com> <87lerkwtm5.wl-maz@kernel.org> <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> <87fshovtu0.wl-maz@kernel.org> <171d0159-4698-354b-8b2f-49d920d03b1b@redhat.com> MIME-Version: 1.0 In-Reply-To: <171d0159-4698-354b-8b2f-49d920d03b1b@redhat.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220823_065825_503849_16AEB5D3 X-CRM114-Status: GOOD ( 21.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Aug 23, 2022 at 03:22:17PM +1000, Gavin Shan wrote: > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > > index 986cee6fbc7f..0b41feb6fb7d 100644 > > --- a/arch/arm64/kvm/arm.c > > +++ b/arch/arm64/kvm/arm.c > > @@ -747,6 +747,12 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu) > > if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) > > return kvm_vcpu_suspend(vcpu); > > + > > + if (kvm_check_request(KVM_REQ_RING_SOFT_FULL, vcpu)) { > > + vcpu->run->exit_reason = KVM_EXIT_DIRTY_RING_FULL; > > + trace_kvm_dirty_ring_exit(vcpu); > > + return 0; > > + } > > } > > return 1; > > diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c > > index f4c2a6eb1666..08b2f01164fa 100644 > > --- a/virt/kvm/dirty_ring.c > > +++ b/virt/kvm/dirty_ring.c > > @@ -149,6 +149,7 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring) > > void kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset) > > { > > + struct kvm_vcpu *vcpu = container_of(ring, struct kvm_vcpu, dirty_ring); > > struct kvm_dirty_gfn *entry; > > /* It should never get full */ > > @@ -166,6 +167,9 @@ void kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset) > > kvm_dirty_gfn_set_dirtied(entry); > > ring->dirty_index++; > > trace_kvm_dirty_ring_push(ring, slot, offset); > > + > > + if (kvm_dirty_ring_soft_full(vcpu)) > > + kvm_make_request(KVM_REQ_RING_SOFT_FULL, vcpu); > > } > > struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring, u32 offset) > > > > Ok, thanks for the details, Marc. I will adopt your code in next revision :) Note that there can be a slight difference with the old/new code, in that an (especially malicious) userapp can logically ignore the DIRTY_RING_FULL vmexit and keep kicking VCPU_RUN with the new code. Unlike the old code, the 2nd/3rd/... KVM_RUN will still run in the new code until the next dirty pfn being pushed to the ring, then it'll request ring full exit again. Each time it exits the ring grows 1. At last iiuc it can easily hit the ring full and trigger the warning at the entry of kvm_dirty_ring_push(): /* It should never get full */ WARN_ON_ONCE(kvm_dirty_ring_full(ring)); We did that because kvm_dirty_ring_push() was previously designed to not be able to fail at all (e.g., in the old bitmap world we never will fail too). We can't because we can't lose any dirty page or migration could silently fail too (consider when we do user exit due to ring full and migration just completed; there could be unsynced pages on src/dst). So even though the old approach will need to read kvm->dirty_ring_size for every entrance which is a pity, it will avoid issue above. Side note: for x86 the dirty ring check was put at the entrance not because it needs to be the highest priority - it should really be the same when check kvm requests. It's just that it'll be the fastest way to fail properly if needed before loading mmu, disabling preemption/irq, etc. Thanks, -- Peter Xu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel