From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B315E7AD74 for ; Tue, 3 Oct 2023 15:59:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DCC86B020E; Tue, 3 Oct 2023 11:59:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98D3E6B020F; Tue, 3 Oct 2023 11:59:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82DFC6B0213; Tue, 3 Oct 2023 11:59:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 751CF6B020E for ; Tue, 3 Oct 2023 11:59:32 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 436841CA1BB for ; Tue, 3 Oct 2023 15:59:32 +0000 (UTC) X-FDA: 81304610184.25.EB6CE63 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf03.hostedemail.com (Postfix) with ESMTP id 276A120024 for ; Tue, 3 Oct 2023 15:59:29 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=F8Nf+FZi; spf=pass (imf03.hostedemail.com: domain of 3YDocZQYKCHUlXTgcVZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--seanjc.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3YDocZQYKCHUlXTgcVZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696348770; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xP89tk8jeFUDqiHwgtcyinENLLmFDFAo+sp2wFgmRBw=; b=kTRTAyjfEztWKK3QUSX35Dl6/XQvdXMMZMU5o2jWdfN//gC+cZ2WGaRpWkrgIHnW0tNM5F 7zwak64ZIEneXff6atbF2xI4hsljqP+fOzdyeGrjOtjVJi9JUI7DtJ7qdEaSHAyMAxJ8lt Ip8s8icxMLZm1lYNXywfa61sCw2GviQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696348770; a=rsa-sha256; cv=none; b=2h4I/X9H+Bf8USYd2nXkHbsvvx9H50mTprp4G5DhC7LiMHisTb1OXhPqiDpZEwXWEfHuBT 8aaDsjiZuEvAtz8VzW0Uy8R4omzpbxWdZz+OEMkWQWyEkN3A8q71CWkgOy1RZjuRLglJ4b a0N/mzwSl9C/frnbvDvWU4+5T+Xfd3s= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=F8Nf+FZi; spf=pass (imf03.hostedemail.com: domain of 3YDocZQYKCHUlXTgcVZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--seanjc.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3YDocZQYKCHUlXTgcVZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-d814634fe4bso1294144276.1 for ; Tue, 03 Oct 2023 08:59:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696348769; x=1696953569; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xP89tk8jeFUDqiHwgtcyinENLLmFDFAo+sp2wFgmRBw=; b=F8Nf+FZi1YlZxgSd+Vjn8S8B4Af7tkQv3uy+o0vmBM6EbjUhqUH7ADfxRu3Kbfvhla AlY8rbgpzGocJqA0eA8tfZQ0rdJJuCY1nNkCXTsI5jI2b/7C/s8vsefr8Xv0SozTijK7 +q3Y9i31jL6p5h6zmIxqe05YmBLL0t57rMmPBbHljTbl+nlIG0uvhVk/tC9AnNwhH/46 3w4+FJlOwh1nsvDU5XZJpMbUpM35gImFcoLqA0oa1qOHE38K21r+GVKzXevq1ClJww0O kkESdZchaHVYGtVSocPO6oF32knm1wqX6I0NJjGL2Zd9THwkw9R6eg7S6v4yvFPoKax6 btnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696348769; x=1696953569; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xP89tk8jeFUDqiHwgtcyinENLLmFDFAo+sp2wFgmRBw=; b=SEeNHdPhQycYuvke4nIy33VISXrH0jwRMt9EOPXqqe1xZmdNhpcN1jLugDHZzdSq17 Mtazb84npVZYpN9Vw780BpKuA3TEEjJa7giES6ELgiC+WiC8zVPUsViUeiKKHPKEbSL4 zbOR5O4dUSsQFeutyASBTk+Yir8C+jKuShdGJil3hX9k5gprUoaBqiV1w7OQ8kyuTW3l z7Vascyw1bDYcrDMZsvCt2vUgYEpTvymwtHpHymouTbfqnjoAuI+vm12x/GXGRm4Tblq XKR2rJczOyA5E9L1Cthk6MhfcwhEn9ESWWiutXkuD7wPe1qH6eHsDUkpqXNJjQKOkA8T eR2w== X-Gm-Message-State: AOJu0YwqziHOLGBO1FBldhjb6ycT5YfdXlbfYTqoInD1e/XDZJJAe3NF 4jrNWoMD45eDlc8BcbafZQMwlET8/QI= X-Google-Smtp-Source: AGHT+IHlsTPxnmIroL8cntBugIBpa7VGBcPZjKPldlj/dItcEWdxSvIkKSkj76hPSiTFqWvviFkle9Hc81s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:3604:0:b0:d7a:c85c:725b with SMTP id d4-20020a253604000000b00d7ac85c725bmr227114yba.7.1696348768967; Tue, 03 Oct 2023 08:59:28 -0700 (PDT) Date: Tue, 3 Oct 2023 08:59:27 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230914015531.1419405-1-seanjc@google.com> <20230914015531.1419405-12-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v12 11/33] KVM: Introduce per-page memory attributes From: Sean Christopherson To: Fuad Tabba Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Jarkko Sakkinen , Anish Moorthy , Yu Zhang , Isaku Yamahata , Xu Yilun , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" Content-Type: text/plain; charset="us-ascii" X-Stat-Signature: 8mf9uwc1fa5ywt8z4xaghmocdecxgfa6 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 276A120024 X-Rspam-User: X-HE-Tag: 1696348769-449218 X-HE-Meta: U2FsdGVkX19fBG1ThSUejBpiYdcyPxJeyDbEsixNYx7Dg0bmkd83AzfL6MzmBisd2sUOkB10qECS6ukx3+ys4PZx195I8Bi4LaATnxHfyeXc1E+Ri2K2vXEMYNc+7NYWZvCOxiolqqkAAvCryiEW++atSSnX98HJjBP1YeJK3U9psDBovhywoCJvWkJEHL8jJuqnDyw1lAH2uQFW9fBushS/5RcgDM6hgzrgdf22SbX1dmRV/8Ye92GZ6PFo00+fcO7TWBrwuMKuwnaCQBah3o2KftSsaPj7ROC12PZ+CCevVmcDetqaDowmUP5RhyiDlcGGKtHUqCKocatZbPEjUCDCN5stVSZ0yKm9/HplSSvLdhYwKy2AEapRvmCNg0i71mZAFApbmMCsLpacE7IPkryp07RxUeQaySMVWGgDS1enEZ3D3O5LDjrakCoM8lv1YqKKI7Ps7l2DnhOrgPuINhZEge1RedfkjuSzNPoMhRzduLCs5aL9aknsgkfKAP1R1w1cEdx9sQYmpVXx+kw0ZRdQfp/yk/puzsYZR1MXXkmlPfg2XQQugrP/IxrLMx9+zEohwDqUaSUw/fhi2KaXNaSxSk2o/vwaCbgDcUQHNGeu6J6eKKK07kVuxulJIT627t/qNY8WKa25+A2rhvE0oBT4Dm/MipR/z1rVoRCxctD9VutvWWtlG4NbVxJ4L20+jYk0RaPHQuQl5j5d19mz8DfDvbHKzPNuAwF6CyQQOMGURwWgh33XhnwSDLw3FxUyyJEPP5yA9pwi6M2RruQ5jZv4DeKCaTV+OqqxXT4cVRNh5n7unwhIXRlglNroVudGJXFQy6GZhA8Ltcl4WyxJtm+VlLhcf3JQ5H5iQMi3xEZQxdn0kKmsTjrxydto+GAHmV6G4+xM9AL0XrMjnol0kS4PXT+zqWtx49sWkMwEvm9YtiCkhFn2fYLKvYHSuNw9I9tssk/dup+f+Tutajv HPWt8Q7j SswxJs3V8ahBZLYPtvVOhEU8Iz2aLYwfs6WouDYFUuZBv68GyFxsjVHsHa+I7G0O8xBGI8c++PsxsnMkUhumUiRrwImTZGRvZuwf7NGpvzFRVVQtGxoxOaY03hWjvnk7Nj9WgkvJyazjvd+rTN9MpEITnQEhp6LdOqelZBUrizj7LTjWUnq/Fv+8/zU7B0G1sbJEgP3WgVWs2jLQcVd0yLvOGwJTCJlSx4Z50WmQ7EWRB5o7I6Q6o5wkF/xD5g2/9o210djV+WB0PEBU38e4KwqT2esyGSJkCa1yWIcLrtkq27YF6/oB8iIQ6z75vBmgZ3lWf/uwZhJqeg5kJY6xuYPcgxgJDqyZS2k6u6YDvZg2MrArkl8qKxbm+Zn2mf6R2bglAePNLVX9QiwUavILzSaamaTfCeiI1yUB/+3ny+x/AcCvyNiYuoqj5EMmgvEUBkiGFf/nVcjnPq8K7zS/ejDPWHFEmoBwktLmlaId7hWSdqa0VjkSLfAteyzBfGWali+kmQK9ew+BLxz6lgIOVBcFaPrMemz/eDtbcR/8zHjO9kge7FRUOwAAF+3qzYPI7qYvykAsvm4Szx7RvEIm5PB9pq5FgE3PHMCJZF/SXWSL0uhw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Oct 03, 2023, Fuad Tabba wrote: > Hi, > > > diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h > > index d2d913acf0df..f8642ff2eb9d 100644 > > --- a/include/uapi/linux/kvm.h > > +++ b/include/uapi/linux/kvm.h > > @@ -1227,6 +1227,7 @@ struct kvm_ppc_resize_hpt { > > #define KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE 228 > > #define KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES 229 > > #define KVM_CAP_USER_MEMORY2 230 > > +#define KVM_CAP_MEMORY_ATTRIBUTES 231 > > > > #ifdef KVM_CAP_IRQ_ROUTING > > > > @@ -2293,4 +2294,17 @@ struct kvm_s390_zpci_op { > > /* flags for kvm_s390_zpci_op->u.reg_aen.flags */ > > #define KVM_S390_ZPCIOP_REGAEN_HOST (1 << 0) > > > > +/* Available with KVM_CAP_MEMORY_ATTRIBUTES */ > > +#define KVM_GET_SUPPORTED_MEMORY_ATTRIBUTES _IOR(KVMIO, 0xd2, __u64) > > +#define KVM_SET_MEMORY_ATTRIBUTES _IOW(KVMIO, 0xd3, struct kvm_memory_attributes) > > + > > +struct kvm_memory_attributes { > > + __u64 address; > > + __u64 size; > > + __u64 attributes; > > + __u64 flags; > > +}; > > + > > +#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) > > + > > In pKVM, we don't want to allow setting (or clearing) of PRIVATE/SHARED > attributes from userspace. Why not? The whole thing falls apart if userspace doesn't *know* the state of a page, and the only way for userspace to know the state of a page at a given moment in time is if userspace controls the attributes. E.g. even if KVM were to provide a way for userspace to query attributes, the attributes exposed to usrspace would become stale the instant KVM drops slots_lock (or whatever lock protects the attributes) since userspace couldn't prevent future changes. Why does pKVM need to prevent userspace from stating *its* view of attributes? If the goal is to reduce memory overhead, that can be solved by using an internal, non-ABI attributes flag to track pKVM's view of SHARED vs. PRIVATE. If the guest attempts to access memory where pKVM and userspace don't agree on the state, generate an exit to userspace. Or kill the guest. Or do something else entirely. > However, we'd like to use the attributes xarray to track the sharing state of > guest pages at the host kernel. > > Moreover, we'd rather the default guest page state be PRIVATE, and > only specify which pages are shared. All pKVM guest pages start off as > private, and the majority will remain so. I would rather optimize kvm_vm_set_mem_attributes() to generate range-based xarray entries, at which point it shouldn't matter all that much whether PRIVATE or SHARED is the default "empty" state. We opted not to do that for the initial merge purely to keep the code as simple as possible (which is obviously still not exactly simple). With range-based xarray entries, the cost of tagging huge chunks of memory as PRIVATE should be a non-issue. And if that's not enough for whatever reason, I would rather define the polarity of PRIVATE on a per-VM basis, but only for internal storage. > I'm not sure if this is the best way to do this: One idea would be to move > the definition of KVM_MEMORY_ATTRIBUTE_PRIVATE to > arch/*/include/asm/kvm_host.h, which is where kvm_arch_supported_attributes() > lives as well. This would allow different architectures to specify their own > attributes (i.e., instead we'd have a KVM_MEMORY_ATTRIBUTE_SHARED for pKVM). > This wouldn't help in terms of preventing userspace from clearing attributes > (i.e., setting a 0 attribute) though. > > The other thing, which we need for pKVM anyway, is to make > kvm_vm_set_mem_attributes() global, so that it can be called from outside of > kvm_main.c (already have a local patch for this that declares it in > kvm_host.h), That's no problem, but I am definitely opposed to KVM modifying attributes that are owned by userspace. > and not gate this function by KVM_GENERIC_MEMORY_ATTRIBUTES. As above, I am opposed to pKVM having a completely different ABI for managing PRIVATE vs. SHARED. I have no objection to pKVM using unclaimed flags in the attributes to store extra metadata, but if KVM_SET_MEMORY_ATTRIBUTES doesn't work for pKVM, then we've failed miserably and should revist the uAPI.