From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71598C43460 for ; Mon, 19 Apr 2021 16:01:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D47AA613CA for ; Mon, 19 Apr 2021 16:01:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D47AA613CA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3F8DD6B006C; Mon, 19 Apr 2021 12:01:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A9D56B006E; Mon, 19 Apr 2021 12:01:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D8E86B0070; Mon, 19 Apr 2021 12:01:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id EFADF6B006C for ; Mon, 19 Apr 2021 12:01:53 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9E6208249980 for ; Mon, 19 Apr 2021 16:01:53 +0000 (UTC) X-FDA: 78049582506.12.817527D Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf18.hostedemail.com (Postfix) with ESMTP id 92BED2000391 for ; Mon, 19 Apr 2021 16:01:53 +0000 (UTC) Received: by mail-pg1-f182.google.com with SMTP id f29so24550135pgm.8 for ; Mon, 19 Apr 2021 09:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=D+scCxaF/k3XnD2sscAAiRoJE5CZQjd/qu5mFzNH7gg=; b=i6RwnX5xqtSrZgM5rvmPeRmsfWNxhgopBYEi7XmYe2x+pC/K41omM/bvXmWAVKQdQY lzyWCwQbk11DZ3Geq6gmuYxwk0ELNs7jLPwq+wZYFINt7uY6uHeRzZeqXSB6GuVtClOA T/QiaTut2wOhP/aP+L6oWxJPjRQX/NcOKzvG3dsWlR6j98FCd85cnQHmT+GLhSUQ+eLK sTJRB51lTBIsffNdMn97QnCUyrA6Ego1GbUSI1/LFLZlqp9RfFKBr1WqljqJyIHL2DX1 oJfkMteb3JLrkZxNM6c8IJPJiXC4uWjDX/CntY5u9HWERN4m8Otuu8VxDKFGVIw3genw SV6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=D+scCxaF/k3XnD2sscAAiRoJE5CZQjd/qu5mFzNH7gg=; b=V9AN2CIjm6oSTgcKR5yk393dOQejrOVruVRNyq6SVEIz8I7pnO3tKnawoODYBFnN6d NPNw/NhiwORCJWVMIRs3Tps3QLWDDG45LkifrRz9Fdrli/5PdqHuEzx3+uoSCn28Z4zC 7mQYLCOG7nssoK9E4EyIg8hEVdIPgLyCO78LvrF4f32WNTmDGPEcgmA8t5S2y5g9/6Wk 6eF8mj1HKnFTCsO25uLWJ4kn6MiCm0Tv+8Imf0dbPcMDu2ZNydunbrF89BSKL6Iokk2O jO5847GnOLMYByB7VkzOW/bjzb0tfuEfGVjp92ADee6nap11+HuJouuozlVACx+IM7mb M6Nw== X-Gm-Message-State: AOAM53011FhCMjtC71ttxQR8/COZfS1QNxHH0qYnagnL6qbTXM+oSjzX iu7Uuhaq/qPfhohiaDLqiA2UFg== X-Google-Smtp-Source: ABdhPJwKrlTLKHLyKD4kCvVe40T/+hXPu2uy1xeYbvr2iGbcZ8V8usWVWgakEo//6lEuJdOUeoFa8g== X-Received: by 2002:a62:7d07:0:b029:21b:d1bc:f6c8 with SMTP id y7-20020a627d070000b029021bd1bcf6c8mr20258928pfc.45.1618848110621; Mon, 19 Apr 2021 09:01:50 -0700 (PDT) Received: from google.com (240.111.247.35.bc.googleusercontent.com. [35.247.111.240]) by smtp.gmail.com with ESMTPSA id l17sm13229762pgi.66.2021.04.19.09.01.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Apr 2021 09:01:49 -0700 (PDT) Date: Mon, 19 Apr 2021 16:01:46 +0000 From: Sean Christopherson To: "Kirill A. Shutemov" Cc: "Kirill A. Shutemov" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Jim Mattson , David Rientjes , "Edgecombe, Rick P" , "Kleen, Andi" , "Yamahata, Isaku" , Erdem Aktas , Steve Rutherford , Peter Gonda , David Hildenbrand , x86@kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [RFCv2 13/13] KVM: unmap guest memory using poisoned pages Message-ID: References: <20210416154106.23721-1-kirill.shutemov@linux.intel.com> <20210416154106.23721-14-kirill.shutemov@linux.intel.com> <20210419142602.khjbzktk5tk5l6lk@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210419142602.khjbzktk5tk5l6lk@box.shutemov.name> X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 92BED2000391 X-Stat-Signature: ihgjthygmhhknf1yn6w9qikgo8mfwwiy Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail-pg1-f182.google.com; client-ip=209.85.215.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618848113-876178 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 19, 2021, Kirill A. Shutemov wrote: > On Fri, Apr 16, 2021 at 05:30:30PM +0000, Sean Christopherson wrote: > > I like the idea of using "special" PTE value to denote guest private memory, > > e.g. in this RFC, HWPOISON. But I strongly dislike having KVM involved in the > > manipulation of the special flag/value. > > > > Today, userspace owns the gfn->hva translations and the kernel effectively owns > > the hva->pfn translations (with input from userspace). KVM just connects the > > dots. > > > > Having KVM own the shared/private transitions means KVM is now part owner of the > > entire gfn->hva->pfn translation, i.e. KVM is effectively now a secondary MMU > > and a co-owner of the primary MMU. This creates locking madness, e.g. KVM taking > > mmap_sem for write, mmu_lock under page lock, etc..., and also takes control away > > from userspace. E.g. userspace strategy could be to use a separate backing/pool > > for shared memory and change the gfn->hva translation (memslots) in reaction to > > a shared/private conversion. Automatically swizzling things in KVM takes away > > that option. > > > > IMO, KVM should be entirely "passive" in this process, e.g. the guest shares or > > protects memory, userspace calls into the kernel to change state, and the kernel > > manages the page tables to prevent bad actors. KVM simply does the plumbing for > > the guest page tables. > > That's a new perspective for me. Very interesting. > > Let's see how it can look like: > > - KVM only allows poisoned pages (or whatever flag we end up using for > protection) in the private mappings. SIGBUS otherwise. > > - Poisoned pages must be tied to the KVM instance to be allowed in the > private mappings. Like kvm->id in the current prototype. SIGBUS > otherwise. > > - Pages get poisoned on fault in if the VMA has a new vmflag set. > > - Fault in of a poisoned page leads to hwpoison entry. Userspace cannot > access such pages. > > - Poisoned pages produced this way get unpoisoned on free. > > - The new VMA flag set by userspace. mprotect(2)? Ya, or mmap(), though I'm not entirely sure a VMA flag would suffice. The notion of the page being private is tied to the PFN, which would suggest "struct page" needs to be involved. But fundamentally the private pages, are well, private. They can't be shared across processes, so I think we could (should?) require the VMA to always be MAP_PRIVATE. Does that buy us enough to rely on the VMA alone? I.e. is that enough to prevent userspace and unaware kernel code from acquiring a reference to the underlying page? > - Add a new GUP flag to retrive such pages from the userspace mapping. > Used only for private mapping population. > - Shared gfn ranges managed by userspace, based on hypercalls from the > guest. > > - Shared mappings get populated via normal VMA. Any poisoned pages here > would lead to SIGBUS. > > So far it looks pretty straight-forward. > > The only thing that I don't understand is at way point the page gets tied > to the KVM instance. Currently we do it just before populating shadow > entries, but it would not work with the new scheme: as we poison pages > on fault it they may never get inserted into shadow entries. That's not > good as we rely on the info to unpoison page on free. Can you elaborate on what you mean by "unpoison"? If the page is never actually mapped into the guest, then its poisoned status is nothing more than a software flag, i.e. nothing extra needs to be done on free. If the page is mapped into the guest, then KVM can be made responsible for reinitializing the page with keyid=0 when the page is removed from the guest. The TDX Module prevents mapping the same PFN into multiple guests, so the kernel doesn't actually have to care _which_ KVM instance(s) is associated with a page, it only needs to prevent installing valid PTEs in the host page tables. > Maybe we should tie VMA to the KVM instance on setting the vmflags? > I donno. > > Any comments? > > -- > Kirill A. Shutemov