From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Mon, 18 Sep 2023 17:08:40 -0700 Subject: [RFC PATCH v12 10/33] KVM: Set the stage for handling only shared mappings in mmu_notifier events In-Reply-To: <20230918180754.iomoaqnw75j3rrxb@amd.com> References: <20230914015531.1419405-1-seanjc@google.com> <20230914015531.1419405-11-seanjc@google.com> <20230918180754.iomoaqnw75j3rrxb@amd.com> Message-ID: List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Mon, Sep 18, 2023, Michael Roth wrote: > On Wed, Sep 13, 2023 at 06:55:08PM -0700, Sean Christopherson wrote: > > Add flags to "struct kvm_gfn_range" to let notifier events target only > > shared and only private mappings, and write up the existing mmu_notifier > > events to be shared-only (private memory is never associated with a > > userspace virtual address, i.e. can't be reached via mmu_notifiers). > > > > Add two flags so that KVM can handle the three possibilities (shared, > > private, and shared+private) without needing something like a tri-state > > enum. > > > > Link: https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB at google.com > > Signed-off-by: Sean Christopherson > > --- > > include/linux/kvm_host.h | 2 ++ > > virt/kvm/kvm_main.c | 7 +++++++ > > 2 files changed, 9 insertions(+) > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > index d8c6ce6c8211..b5373cee2b08 100644 > > --- a/include/linux/kvm_host.h > > +++ b/include/linux/kvm_host.h > > @@ -263,6 +263,8 @@ struct kvm_gfn_range { > > gfn_t start; > > gfn_t end; > > union kvm_mmu_notifier_arg arg; > > + bool only_private; > > + bool only_shared; > > bool may_block; > > }; > > bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index 174de2789657..a41f8658dfe0 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -635,6 +635,13 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm, > > * the second or later invocation of the handler). > > */ > > gfn_range.arg = range->arg; > > + > > + /* > > + * HVA-based notifications aren't relevant to private > > + * mappings as they don't have a userspace mapping. > > + */ > > + gfn_range.only_private = false; > > + gfn_range.only_shared = true; > > gfn_range.may_block = range->may_block; > > Who is supposed to read only_private/only_shared? Is it supposed to be > plumbed onto arch code and handled specially there? Yeah, that's the idea. Though I don't know that it's worth using for SNP, the cost of checking the RMP may be higher than just eating the extra faults. > I ask because I see elsewhere you have: > > /* > * If one or more memslots were found and thus zapped, notify arch code > * that guest memory has been reclaimed. This needs to be done *after* > * dropping mmu_lock, as x86's reclaim path is slooooow. > */ > if (__kvm_handle_hva_range(kvm, &hva_range).found_memslot) > kvm_arch_guest_memory_reclaimed(kvm); > > and if there are any MMU notifier events that touch HVAs, then > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() will get called, > which causes the performance issues for SEV and SNP that Ashish had brought > up. Technically that would only need to happen if there are GPAs in that > memslot that aren't currently backed by gmem pages (and then gmem could handle > its own wbinvd_on_all_cpus() (or maybe clflush per-page)). > > Actually, even if there are shared pages in the GPA range, the > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() can be skipped for > guests that only use gmem pages for private memory. Is that acceptable? Yes, that was my original plan. I may have forgotten that exact plan at one point or another and not communicated it well. But the idea is definitely that if a VM type, a.k.a. SNP guests, is required to use gmem for private memory, then there's no need to blast WBINVD because barring a KVM bug, the mmu_notifier event can't have freed private memory, even if it *did* zap SPTEs. For gmem, if KVM doesn't precisely zap only shared SPTEs for SNP (is that even possible to do race-free?), then KVM needs to blast WBINVD when freeing memory from gmem even if there are no SPTEs. But that seems like a non-issue for a well-behaved setup because the odds of there being *zero* SPTEs should be nil. > Just trying to figure out where this only_private/only_shared handling ties > into that (or if it's a separate thing entirely). It's mostly a TDX thing. I threw it in this series mostly to "formally" document that the mmu_notifier path only affects shared mappings. If the code causes confusion without the TDX context, and won't be used by SNP, we can and should drop it from the initial merge and have it go along with the TDX series. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE2B6180 for ; Tue, 19 Sep 2023 00:08:43 +0000 (UTC) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-59c12d31d04so36887157b3.1 for ; Mon, 18 Sep 2023 17:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695082122; x=1695686922; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=GmkZKmC1wLS7I9Kuqm9W8ruPNvs1lFHUbwBTnnTYxNHX51xEX7shNX0U0tVAy9IE7L jLOaAw5jhD6ETEro/vsaQHwSQ7RyDjhqZpYxto/F0mryCzI+WTmG9h/GzH4NMn/UdJzt eW9/qgYaizfC+WJuuBtUMG0BXA9uyDKXJo/li6WSXtDpSv7nHIGjbiyIgeSV9je6cYv3 YLu4qEP7DML0sRCdzc+k+7IApoT7gd9xXXlgGEmR6XKk3EItoEMx9crRlyeVoY8Jtvz0 PDKObbytslyzP/MGSrgrzlz+EC+3c10Shsx8qBxxb7gyQNzFIYn1K137VGK8aqJJl8Jj m5QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695082122; x=1695686922; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=a6RqAwsidGxtZlxI+JVmLINLkm1T/RBluCQl/aqwO7C1jtnpulxkUPnow+XfDjB8uT JKIroy6Vieq976280usxbKXvDCG5ZjvsuFrW5fqcleGfkrnhXM6QJX1FIlPF9vJ2JoTD TGE/8pWreiNCXgEwDiFVn4QOhItHyg1t8DSEKM9mcq8AEVgK2I2FbUDwn1jZioyIJDUs gAAeNEoPIA/qhGtYy5mWEMrLAfd0CSdAEMafztldbbqSFILU7MWgiVDeU/U3H/LW8yRJ 0kb4lPgCECw+ncltPJVmrbh/KJl7LhFuFzve4jTPhG2pyV6boKRDc3vpBYTaQImbwcu7 Sn3g== X-Gm-Message-State: AOJu0YyLGsOZg1hRoUrf8CyItM5ZwbX2NzcZrUw/OXK1u4J1kPxwN2Uy 4Gk6E/mp8Kzb9PLiHffLncdszSQK8h4= X-Google-Smtp-Source: AGHT+IHHNvt5OesaYTZ4SrOkO6G/E0GpOB67x3jgD/5Z7IEbZFEyHH5t/MUiil/50JGkevI3xvv2hT/ge04= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:11cf:b0:d80:23ff:ae7f with SMTP id n15-20020a05690211cf00b00d8023ffae7fmr237508ybu.4.1695082122578; Mon, 18 Sep 2023 17:08:42 -0700 (PDT) Date: Mon, 18 Sep 2023 17:08:40 -0700 In-Reply-To: <20230918180754.iomoaqnw75j3rrxb@amd.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20230914015531.1419405-1-seanjc@google.com> <20230914015531.1419405-11-seanjc@google.com> <20230918180754.iomoaqnw75j3rrxb@amd.com> Message-ID: Subject: Re: [RFC PATCH v12 10/33] KVM: Set the stage for handling only shared mappings in mmu_notifier events From: Sean Christopherson To: Michael Roth Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Anish Moorthy , Yu Zhang , Isaku Yamahata , Xu Yilun , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" , Ashish Kalra Content-Type: text/plain; charset="us-ascii" On Mon, Sep 18, 2023, Michael Roth wrote: > On Wed, Sep 13, 2023 at 06:55:08PM -0700, Sean Christopherson wrote: > > Add flags to "struct kvm_gfn_range" to let notifier events target only > > shared and only private mappings, and write up the existing mmu_notifier > > events to be shared-only (private memory is never associated with a > > userspace virtual address, i.e. can't be reached via mmu_notifiers). > > > > Add two flags so that KVM can handle the three possibilities (shared, > > private, and shared+private) without needing something like a tri-state > > enum. > > > > Link: https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB@google.com > > Signed-off-by: Sean Christopherson > > --- > > include/linux/kvm_host.h | 2 ++ > > virt/kvm/kvm_main.c | 7 +++++++ > > 2 files changed, 9 insertions(+) > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > index d8c6ce6c8211..b5373cee2b08 100644 > > --- a/include/linux/kvm_host.h > > +++ b/include/linux/kvm_host.h > > @@ -263,6 +263,8 @@ struct kvm_gfn_range { > > gfn_t start; > > gfn_t end; > > union kvm_mmu_notifier_arg arg; > > + bool only_private; > > + bool only_shared; > > bool may_block; > > }; > > bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index 174de2789657..a41f8658dfe0 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -635,6 +635,13 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm, > > * the second or later invocation of the handler). > > */ > > gfn_range.arg = range->arg; > > + > > + /* > > + * HVA-based notifications aren't relevant to private > > + * mappings as they don't have a userspace mapping. > > + */ > > + gfn_range.only_private = false; > > + gfn_range.only_shared = true; > > gfn_range.may_block = range->may_block; > > Who is supposed to read only_private/only_shared? Is it supposed to be > plumbed onto arch code and handled specially there? Yeah, that's the idea. Though I don't know that it's worth using for SNP, the cost of checking the RMP may be higher than just eating the extra faults. > I ask because I see elsewhere you have: > > /* > * If one or more memslots were found and thus zapped, notify arch code > * that guest memory has been reclaimed. This needs to be done *after* > * dropping mmu_lock, as x86's reclaim path is slooooow. > */ > if (__kvm_handle_hva_range(kvm, &hva_range).found_memslot) > kvm_arch_guest_memory_reclaimed(kvm); > > and if there are any MMU notifier events that touch HVAs, then > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() will get called, > which causes the performance issues for SEV and SNP that Ashish had brought > up. Technically that would only need to happen if there are GPAs in that > memslot that aren't currently backed by gmem pages (and then gmem could handle > its own wbinvd_on_all_cpus() (or maybe clflush per-page)). > > Actually, even if there are shared pages in the GPA range, the > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() can be skipped for > guests that only use gmem pages for private memory. Is that acceptable? Yes, that was my original plan. I may have forgotten that exact plan at one point or another and not communicated it well. But the idea is definitely that if a VM type, a.k.a. SNP guests, is required to use gmem for private memory, then there's no need to blast WBINVD because barring a KVM bug, the mmu_notifier event can't have freed private memory, even if it *did* zap SPTEs. For gmem, if KVM doesn't precisely zap only shared SPTEs for SNP (is that even possible to do race-free?), then KVM needs to blast WBINVD when freeing memory from gmem even if there are no SPTEs. But that seems like a non-issue for a well-behaved setup because the odds of there being *zero* SPTEs should be nil. > Just trying to figure out where this only_private/only_shared handling ties > into that (or if it's a separate thing entirely). It's mostly a TDX thing. I threw it in this series mostly to "formally" document that the mmu_notifier path only affects shared mappings. If the code causes confusion without the TDX context, and won't be used by SNP, we can and should drop it from the initial merge and have it go along with the TDX series. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84989CD3423 for ; Tue, 19 Sep 2023 00:08:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=gtzIca1vFbO9eOfVblOXMXTHmcXYWgGUGXpJGf0Z4RA=; b=jgg0ivofSS4mRUThkkTFm0k6UQ NMEA4XpDQxxXdRjbJ46uXlh/A/QcUITniRtEMQCNNlDFlujI14yfNx28nqv7FYnQH1fI5rRszK/U+ CfrN/3VKz882c/sDoUSVwvHDAkkYxP3yQ00op8n7ewqzz37XYq6ZRehtZkr78usIZ9bq0SQPve8N/ wmiaT9ZmOMLh/vO6njRe47TYT8Qyn2AhPrJrGV9T7B6ynyRO0Uu8UEo9cH1dVCnt/fuxeSIS4APjd hzrwWU97IcHBd+AoJwtd4fP6zo5HJMF8OFQVRAfbrcNZMQyRWGbNEQb2z/jTLBSo+rEUCCp+IoJCh lQ+Q9XSA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qiOIK-00GZFD-1j; Tue, 19 Sep 2023 00:08:48 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qiOIH-00GZDs-1P for linux-riscv@lists.infradead.org; Tue, 19 Sep 2023 00:08:47 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-59c27703cc6so32443927b3.2 for ; Mon, 18 Sep 2023 17:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695082123; x=1695686923; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=a65hwXrfQyHXgDYe6XnxKZNx6Vn70ATf8JIavZCEqAWzw7esLm3cM/ae6QK4xsb9Rn BRTF9rCvXtCzMYOWedGjJlp4yeqO/JNCC/3dH5orgyF3ISY1GPdoGLd/2MCJFhDiAP4w xn0Ow7H6TJ9/iY7piZrFZNl8PlL60kCiA+MR/6w2vIfrtWcdwzaRGsIizxoEd4kmVSCv PzKKjI0TikoOVMEFxjElWQWuXub0hahVDzkzB2gpBdx+554CU//gbGcy4ECFAPeHdk/o mirjiVu/ySiLajC9o8B0WThZuPRoIP28+J0p5ODhb3t6uAopW9hVsxBQalzWc3yExLhX 16MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695082123; x=1695686923; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=v8GjgQ8zAMulHV1869Tfin+rf+n1PI68gc6+u9pqqDgM3kCRpsDpqgt7Ad/+2gSqb0 dLAOl1Tb0rtnbjoNTLLhfw1vrJiplGg9ZJjUKW13RYMAtHptEJNFUOV+sRrmOclzdQZp I0y6j0djngwKdWxAu/BPPXxAjKquZNDrOD4myyz/JYtgNBEOCuCcCD4nunN5QABJb8Xe //LKy1KAlNKKnq3zZAF/uyQpUEPpKf73s92VQ0Qp1Vv77EqeAov90gsPZReFPym3LZkL B9Ootjtp7e/K1Blt6UbrjPKOyr5rYRJdKTqdQyR97HSkmT7xfWvkKeY5KxRq1+YqypDE k+Gw== X-Gm-Message-State: AOJu0YzvAYDkaXBQDo6Eo7iABPaQloGZ9l7ojR5AlV2FzJht6/oC1/9P WED978pOAxZfAfEyDYzZ6p93YZyoiLo= X-Google-Smtp-Source: AGHT+IHHNvt5OesaYTZ4SrOkO6G/E0GpOB67x3jgD/5Z7IEbZFEyHH5t/MUiil/50JGkevI3xvv2hT/ge04= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:11cf:b0:d80:23ff:ae7f with SMTP id n15-20020a05690211cf00b00d8023ffae7fmr237508ybu.4.1695082122578; Mon, 18 Sep 2023 17:08:42 -0700 (PDT) Date: Mon, 18 Sep 2023 17:08:40 -0700 In-Reply-To: <20230918180754.iomoaqnw75j3rrxb@amd.com> Mime-Version: 1.0 References: <20230914015531.1419405-1-seanjc@google.com> <20230914015531.1419405-11-seanjc@google.com> <20230918180754.iomoaqnw75j3rrxb@amd.com> Message-ID: Subject: Re: [RFC PATCH v12 10/33] KVM: Set the stage for handling only shared mappings in mmu_notifier events From: Sean Christopherson To: Michael Roth Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Anish Moorthy , Yu Zhang , Isaku Yamahata , Xu Yilun , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" , Ashish Kalra X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230918_170845_495515_D03EEB7B X-CRM114-Status: GOOD ( 37.88 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On Mon, Sep 18, 2023, Michael Roth wrote: > On Wed, Sep 13, 2023 at 06:55:08PM -0700, Sean Christopherson wrote: > > Add flags to "struct kvm_gfn_range" to let notifier events target only > > shared and only private mappings, and write up the existing mmu_notifier > > events to be shared-only (private memory is never associated with a > > userspace virtual address, i.e. can't be reached via mmu_notifiers). > > > > Add two flags so that KVM can handle the three possibilities (shared, > > private, and shared+private) without needing something like a tri-state > > enum. > > > > Link: https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB@google.com > > Signed-off-by: Sean Christopherson > > --- > > include/linux/kvm_host.h | 2 ++ > > virt/kvm/kvm_main.c | 7 +++++++ > > 2 files changed, 9 insertions(+) > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > index d8c6ce6c8211..b5373cee2b08 100644 > > --- a/include/linux/kvm_host.h > > +++ b/include/linux/kvm_host.h > > @@ -263,6 +263,8 @@ struct kvm_gfn_range { > > gfn_t start; > > gfn_t end; > > union kvm_mmu_notifier_arg arg; > > + bool only_private; > > + bool only_shared; > > bool may_block; > > }; > > bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index 174de2789657..a41f8658dfe0 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -635,6 +635,13 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm, > > * the second or later invocation of the handler). > > */ > > gfn_range.arg = range->arg; > > + > > + /* > > + * HVA-based notifications aren't relevant to private > > + * mappings as they don't have a userspace mapping. > > + */ > > + gfn_range.only_private = false; > > + gfn_range.only_shared = true; > > gfn_range.may_block = range->may_block; > > Who is supposed to read only_private/only_shared? Is it supposed to be > plumbed onto arch code and handled specially there? Yeah, that's the idea. Though I don't know that it's worth using for SNP, the cost of checking the RMP may be higher than just eating the extra faults. > I ask because I see elsewhere you have: > > /* > * If one or more memslots were found and thus zapped, notify arch code > * that guest memory has been reclaimed. This needs to be done *after* > * dropping mmu_lock, as x86's reclaim path is slooooow. > */ > if (__kvm_handle_hva_range(kvm, &hva_range).found_memslot) > kvm_arch_guest_memory_reclaimed(kvm); > > and if there are any MMU notifier events that touch HVAs, then > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() will get called, > which causes the performance issues for SEV and SNP that Ashish had brought > up. Technically that would only need to happen if there are GPAs in that > memslot that aren't currently backed by gmem pages (and then gmem could handle > its own wbinvd_on_all_cpus() (or maybe clflush per-page)). > > Actually, even if there are shared pages in the GPA range, the > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() can be skipped for > guests that only use gmem pages for private memory. Is that acceptable? Yes, that was my original plan. I may have forgotten that exact plan at one point or another and not communicated it well. But the idea is definitely that if a VM type, a.k.a. SNP guests, is required to use gmem for private memory, then there's no need to blast WBINVD because barring a KVM bug, the mmu_notifier event can't have freed private memory, even if it *did* zap SPTEs. For gmem, if KVM doesn't precisely zap only shared SPTEs for SNP (is that even possible to do race-free?), then KVM needs to blast WBINVD when freeing memory from gmem even if there are no SPTEs. But that seems like a non-issue for a well-behaved setup because the odds of there being *zero* SPTEs should be nil. > Just trying to figure out where this only_private/only_shared handling ties > into that (or if it's a separate thing entirely). It's mostly a TDX thing. I threw it in this series mostly to "formally" document that the mmu_notifier path only affects shared mappings. If the code causes confusion without the TDX context, and won't be used by SNP, we can and should drop it from the initial merge and have it go along with the TDX series. _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75C56CD3424 for ; Tue, 19 Sep 2023 00:09:43 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=PQV0vXOf; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4RqMVF6FSzz3cN1 for ; Tue, 19 Sep 2023 10:09:41 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=PQV0vXOf; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=flex--seanjc.bounces.google.com (client-ip=2607:f8b0:4864:20::114a; helo=mail-yw1-x114a.google.com; envelope-from=3iuyizqykdkkbnjwslpxxpun.lxvurwdgyyl-mneurbcb.xiujkb.xap@flex--seanjc.bounces.google.com; receiver=lists.ozlabs.org) Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4RqMTB50nnz3cMH for ; Tue, 19 Sep 2023 10:08:45 +1000 (AEST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-58f9db8bc1dso58674887b3.3 for ; Mon, 18 Sep 2023 17:08:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695082122; x=1695686922; darn=lists.ozlabs.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=PQV0vXOfgEzAmPHDw6YhuysKnA8wHakbDzDUHvnNM0sXufKo/TXEyyn9MD2sMtMGmJ WFUJGng5k8UpiRx1qM63DBLFFhitSKsOrjlYSVdhFLe9yUBRbh9gH7temRahnNzbuIex OojLIxIGPradsgsVU5RAOeVgajw+WGFsLL+KIvzW/1L8MKAObGI7BW46lNmmcGvQaDQw 9Hrt/Djeg1HijpyRr/DaZ4DQC+nCKGbB7gDy+0LVvyJpV9Ni1W/huDL7znfFUxhAtqSG 63BylD2JTqkQrdKvPC2itVloOhk2jDEzdP3xfG8w02QL8GbzwN6yDsZufLRCdzoWmMXQ CU5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695082122; x=1695686922; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=SeqFbIaSpMEPyHKAr+PIyRul6iudD3jV8PeX1K2efaAex2APCFWtV/9L5h5gdrI0Tn xmYMR4xNWPAAqclYUGvEHUronU0FCLu7IMrmDMXd4hGOS6EaHpy7/vWyK6GClLNLyMRg L2ZUDWCFldsNSzNzWjyKdpwx2JFBtJi3lZCLtd1Z5WXI7A7k/7ASYIWvznzciAT3faiC MABsSWh9tste9eaY9bFjhon5qppu5RzIc70Xwo082XD+P7M0U+FALnpilS8wC3Cgqrbo McyCKI9qyRpMkn3KKMeVTeKqzdfDm2K7xOeO66NhYglXo7xn+85KPefRwyYvPJrBq4LO PQBg== X-Gm-Message-State: AOJu0Yw2M/6Qp3JYaapqo8kGitSdl5bh7B0cXMlfaa2QUyArCphad/yW ll8PLsH5rQeIZU9nXqQeB2nd1MZsqAY= X-Google-Smtp-Source: AGHT+IHHNvt5OesaYTZ4SrOkO6G/E0GpOB67x3jgD/5Z7IEbZFEyHH5t/MUiil/50JGkevI3xvv2hT/ge04= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:11cf:b0:d80:23ff:ae7f with SMTP id n15-20020a05690211cf00b00d8023ffae7fmr237508ybu.4.1695082122578; Mon, 18 Sep 2023 17:08:42 -0700 (PDT) Date: Mon, 18 Sep 2023 17:08:40 -0700 In-Reply-To: <20230918180754.iomoaqnw75j3rrxb@amd.com> Mime-Version: 1.0 References: <20230914015531.1419405-1-seanjc@google.com> <20230914015531.1419405-11-seanjc@google.com> <20230918180754.iomoaqnw75j3rrxb@amd.com> Message-ID: Subject: Re: [RFC PATCH v12 10/33] KVM: Set the stage for handling only shared mappings in mmu_notifier events From: Sean Christopherson To: Michael Roth Content-Type: text/plain; charset="us-ascii" X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, David Hildenbrand , Yu Zhang , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chao Peng , linux-riscv@lists.infradead.org, Isaku Yamahata , Paul Moore , Marc Zyngier , Huacai Chen , James Morris , "Matthew Wilcox \(Oracle\)" , Wang , Fuad Tabba , Jarkko Sakkinen , "Serge E. Hallyn" , Maciej Szmigiero , Albert Ou , Vlastimil Babka , Ackerley Tng , Paul Walmsley , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Isaku Yamahata , Quentin Perret , Ashish Kalra , Liam Merwick , linux-mips@vger.kernel.org, Oliver Upton , linux-security-module@vger.kernel.org, Palmer Dabbelt , "Kirill A . Shutemov" , kvm-riscv@lists.infradead.org, Anup Patel , linux-fsdevel@vger.kernel.org, Paolo Bonzini , Andrew Morton , Vishal Annapurve , linuxppc-dev@lists.ozlabs.org, Xu Yilun , Anish Moorthy Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, Sep 18, 2023, Michael Roth wrote: > On Wed, Sep 13, 2023 at 06:55:08PM -0700, Sean Christopherson wrote: > > Add flags to "struct kvm_gfn_range" to let notifier events target only > > shared and only private mappings, and write up the existing mmu_notifier > > events to be shared-only (private memory is never associated with a > > userspace virtual address, i.e. can't be reached via mmu_notifiers). > > > > Add two flags so that KVM can handle the three possibilities (shared, > > private, and shared+private) without needing something like a tri-state > > enum. > > > > Link: https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB@google.com > > Signed-off-by: Sean Christopherson > > --- > > include/linux/kvm_host.h | 2 ++ > > virt/kvm/kvm_main.c | 7 +++++++ > > 2 files changed, 9 insertions(+) > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > index d8c6ce6c8211..b5373cee2b08 100644 > > --- a/include/linux/kvm_host.h > > +++ b/include/linux/kvm_host.h > > @@ -263,6 +263,8 @@ struct kvm_gfn_range { > > gfn_t start; > > gfn_t end; > > union kvm_mmu_notifier_arg arg; > > + bool only_private; > > + bool only_shared; > > bool may_block; > > }; > > bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index 174de2789657..a41f8658dfe0 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -635,6 +635,13 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm, > > * the second or later invocation of the handler). > > */ > > gfn_range.arg = range->arg; > > + > > + /* > > + * HVA-based notifications aren't relevant to private > > + * mappings as they don't have a userspace mapping. > > + */ > > + gfn_range.only_private = false; > > + gfn_range.only_shared = true; > > gfn_range.may_block = range->may_block; > > Who is supposed to read only_private/only_shared? Is it supposed to be > plumbed onto arch code and handled specially there? Yeah, that's the idea. Though I don't know that it's worth using for SNP, the cost of checking the RMP may be higher than just eating the extra faults. > I ask because I see elsewhere you have: > > /* > * If one or more memslots were found and thus zapped, notify arch code > * that guest memory has been reclaimed. This needs to be done *after* > * dropping mmu_lock, as x86's reclaim path is slooooow. > */ > if (__kvm_handle_hva_range(kvm, &hva_range).found_memslot) > kvm_arch_guest_memory_reclaimed(kvm); > > and if there are any MMU notifier events that touch HVAs, then > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() will get called, > which causes the performance issues for SEV and SNP that Ashish had brought > up. Technically that would only need to happen if there are GPAs in that > memslot that aren't currently backed by gmem pages (and then gmem could handle > its own wbinvd_on_all_cpus() (or maybe clflush per-page)). > > Actually, even if there are shared pages in the GPA range, the > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() can be skipped for > guests that only use gmem pages for private memory. Is that acceptable? Yes, that was my original plan. I may have forgotten that exact plan at one point or another and not communicated it well. But the idea is definitely that if a VM type, a.k.a. SNP guests, is required to use gmem for private memory, then there's no need to blast WBINVD because barring a KVM bug, the mmu_notifier event can't have freed private memory, even if it *did* zap SPTEs. For gmem, if KVM doesn't precisely zap only shared SPTEs for SNP (is that even possible to do race-free?), then KVM needs to blast WBINVD when freeing memory from gmem even if there are no SPTEs. But that seems like a non-issue for a well-behaved setup because the odds of there being *zero* SPTEs should be nil. > Just trying to figure out where this only_private/only_shared handling ties > into that (or if it's a separate thing entirely). It's mostly a TDX thing. I threw it in this series mostly to "formally" document that the mmu_notifier path only affects shared mappings. If the code causes confusion without the TDX context, and won't be used by SNP, we can and should drop it from the initial merge and have it go along with the TDX series. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F24FFCD3423 for ; Tue, 19 Sep 2023 00:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=5/XrYqRJGFQjrMkTjxsUr/5RFkbTU4Yxlc3Xr9O8bQo=; b=qh/xvhVAyx/XC1MORfPIJf2BtJ QcJzv/hr0y3Az7WaoSPyH+7XlJHG8mYiKJuqYIlXJeW1qQWjEgE6bfZa3117cCP4f8NS4+fasowk7 LYoqeRYvS6P5RYd+FWLxbeAuk/AM4joZns4kUt+5P09x4IjqH2kUmwYLg+EFmbYssCI0Nq421Xl5k mSxxS0tpvFUO61tGaFsPkQKwP30mL5Hol8HoCbSni4A7OB1DsSSMv8KI7ZTwbyMyGkaoEAIM54/Xr 0SjmpNAkecvxp0ZzNTSHupUywfWYvNAzBjaFr64F+771Hhvl2lKUoN/56v/V75s59TPFokRQHjURz IYHEK5AA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qiOIL-00GZFN-0G; Tue, 19 Sep 2023 00:08:49 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qiOIH-00GZDt-1a for linux-arm-kernel@lists.infradead.org; Tue, 19 Sep 2023 00:08:47 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-59c12d31d04so36887167b3.1 for ; Mon, 18 Sep 2023 17:08:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695082123; x=1695686923; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=a65hwXrfQyHXgDYe6XnxKZNx6Vn70ATf8JIavZCEqAWzw7esLm3cM/ae6QK4xsb9Rn BRTF9rCvXtCzMYOWedGjJlp4yeqO/JNCC/3dH5orgyF3ISY1GPdoGLd/2MCJFhDiAP4w xn0Ow7H6TJ9/iY7piZrFZNl8PlL60kCiA+MR/6w2vIfrtWcdwzaRGsIizxoEd4kmVSCv PzKKjI0TikoOVMEFxjElWQWuXub0hahVDzkzB2gpBdx+554CU//gbGcy4ECFAPeHdk/o mirjiVu/ySiLajC9o8B0WThZuPRoIP28+J0p5ODhb3t6uAopW9hVsxBQalzWc3yExLhX 16MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695082123; x=1695686923; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dInZugQi4I3jT3AsjQuL7R2MGGG9K+dIBnr86/ph3q8=; b=O36Ea+VqQtHlzgkN9XyL4jdByIb67cCqBHBu/6n7I8d/Xmp593zoM1x8dOjlhiGpg2 NZ3JckpJJtZNylZxplhCAZVwZoV+hYeKDopvwGD7f2t1/rq3jeDerVYGlvHVem6bGzry 4wx/NLXasGvxZ1g04HmqgxIqcDOXN3MdTkgy4C5jWQH5yE6fuo3Qa7p6Qpmjem7Pq41A IhITxqmN5xcWHlLR2z27K3TiMetnnDKxUXDJx0LTYMMrNl93cvrR6gqR8mvyOPgQIjRV o/bljHvoxNAYDZpZuEVRhF/RC+NSTFTzO0zRyC6AmL6lTXx0uKMtKcylv40DvfuQJEs2 hUOQ== X-Gm-Message-State: AOJu0Yxcsb8uNIhXYqLnHtXPW66IXVTzB5cBhS6vD18mCUYY2szciLMX Qwe/PMjT7/zJTLYEbdL2KS24MwoAFeM= X-Google-Smtp-Source: AGHT+IHHNvt5OesaYTZ4SrOkO6G/E0GpOB67x3jgD/5Z7IEbZFEyHH5t/MUiil/50JGkevI3xvv2hT/ge04= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:11cf:b0:d80:23ff:ae7f with SMTP id n15-20020a05690211cf00b00d8023ffae7fmr237508ybu.4.1695082122578; Mon, 18 Sep 2023 17:08:42 -0700 (PDT) Date: Mon, 18 Sep 2023 17:08:40 -0700 In-Reply-To: <20230918180754.iomoaqnw75j3rrxb@amd.com> Mime-Version: 1.0 References: <20230914015531.1419405-1-seanjc@google.com> <20230914015531.1419405-11-seanjc@google.com> <20230918180754.iomoaqnw75j3rrxb@amd.com> Message-ID: Subject: Re: [RFC PATCH v12 10/33] KVM: Set the stage for handling only shared mappings in mmu_notifier events From: Sean Christopherson To: Michael Roth Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Anish Moorthy , Yu Zhang , Isaku Yamahata , Xu Yilun , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" , Ashish Kalra X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230918_170845_530213_5EAC77D3 X-CRM114-Status: GOOD ( 39.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Sep 18, 2023, Michael Roth wrote: > On Wed, Sep 13, 2023 at 06:55:08PM -0700, Sean Christopherson wrote: > > Add flags to "struct kvm_gfn_range" to let notifier events target only > > shared and only private mappings, and write up the existing mmu_notifier > > events to be shared-only (private memory is never associated with a > > userspace virtual address, i.e. can't be reached via mmu_notifiers). > > > > Add two flags so that KVM can handle the three possibilities (shared, > > private, and shared+private) without needing something like a tri-state > > enum. > > > > Link: https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB@google.com > > Signed-off-by: Sean Christopherson > > --- > > include/linux/kvm_host.h | 2 ++ > > virt/kvm/kvm_main.c | 7 +++++++ > > 2 files changed, 9 insertions(+) > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > index d8c6ce6c8211..b5373cee2b08 100644 > > --- a/include/linux/kvm_host.h > > +++ b/include/linux/kvm_host.h > > @@ -263,6 +263,8 @@ struct kvm_gfn_range { > > gfn_t start; > > gfn_t end; > > union kvm_mmu_notifier_arg arg; > > + bool only_private; > > + bool only_shared; > > bool may_block; > > }; > > bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index 174de2789657..a41f8658dfe0 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -635,6 +635,13 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm, > > * the second or later invocation of the handler). > > */ > > gfn_range.arg = range->arg; > > + > > + /* > > + * HVA-based notifications aren't relevant to private > > + * mappings as they don't have a userspace mapping. > > + */ > > + gfn_range.only_private = false; > > + gfn_range.only_shared = true; > > gfn_range.may_block = range->may_block; > > Who is supposed to read only_private/only_shared? Is it supposed to be > plumbed onto arch code and handled specially there? Yeah, that's the idea. Though I don't know that it's worth using for SNP, the cost of checking the RMP may be higher than just eating the extra faults. > I ask because I see elsewhere you have: > > /* > * If one or more memslots were found and thus zapped, notify arch code > * that guest memory has been reclaimed. This needs to be done *after* > * dropping mmu_lock, as x86's reclaim path is slooooow. > */ > if (__kvm_handle_hva_range(kvm, &hva_range).found_memslot) > kvm_arch_guest_memory_reclaimed(kvm); > > and if there are any MMU notifier events that touch HVAs, then > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() will get called, > which causes the performance issues for SEV and SNP that Ashish had brought > up. Technically that would only need to happen if there are GPAs in that > memslot that aren't currently backed by gmem pages (and then gmem could handle > its own wbinvd_on_all_cpus() (or maybe clflush per-page)). > > Actually, even if there are shared pages in the GPA range, the > kvm_arch_guest_memory_reclaimed()->wbinvd_on_all_cpus() can be skipped for > guests that only use gmem pages for private memory. Is that acceptable? Yes, that was my original plan. I may have forgotten that exact plan at one point or another and not communicated it well. But the idea is definitely that if a VM type, a.k.a. SNP guests, is required to use gmem for private memory, then there's no need to blast WBINVD because barring a KVM bug, the mmu_notifier event can't have freed private memory, even if it *did* zap SPTEs. For gmem, if KVM doesn't precisely zap only shared SPTEs for SNP (is that even possible to do race-free?), then KVM needs to blast WBINVD when freeing memory from gmem even if there are no SPTEs. But that seems like a non-issue for a well-behaved setup because the odds of there being *zero* SPTEs should be nil. > Just trying to figure out where this only_private/only_shared handling ties > into that (or if it's a separate thing entirely). It's mostly a TDX thing. I threw it in this series mostly to "formally" document that the mmu_notifier path only affects shared mappings. If the code causes confusion without the TDX context, and won't be used by SNP, we can and should drop it from the initial merge and have it go along with the TDX series. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel