From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Fri, 11 Aug 2023 10:44:11 -0700 Subject: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory In-Reply-To: References: <20230718234512.1690985-13-seanjc@google.com> Message-ID: List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Thu, Aug 10, 2023, Vishal Annapurve wrote: > On Tue, Aug 8, 2023 at 2:13?PM Sean Christopherson wrote: > > ... > > > > + When binding a memslot to the file, if a kvm pointer exists, it must > > > be the same kvm as the one in this binding > > > + When the binding to the last memslot is removed from a file, NULL the > > > kvm pointer. > > > > Nullifying the KVM pointer isn't sufficient, because without additional actions > > userspace could extract data from a VM by deleting its memslots and then binding > > the guest_memfd to an attacker controlled VM. Or more likely with TDX and SNP, > > induce badness by coercing KVM into mapping memory into a guest with the wrong > > ASID/HKID. > > > > TDX/SNP have mechanisms i.e. PAMT/RMP tables to ensure that the same > memory is not assigned to two different VMs. One of the main reasons we pivoted away from using a flag in "struct page" to indicate that a page was private was so that KVM could enforce 1:1 VM:page ownership *without* relying on hardware. And FWIW, the PAMT provides no protection in this specific case because KVM does TDH.MEM.PAGE.REMOVE when zapping S-EPT entries, and that marks the page clear in the PAMT. The danger there is that physical memory is still encrypted with the guest's HKID, and so mapping the memory into a different VM, which might not be a TDX guest!, could lead to corruption and/or poison #MCs. The HKID issues wouldn't be a problem if v15 is merged as-is, because zapping S-EPT entries also fully purges and reclaims the page, but as we discussed in one of the many threads, reclaiming physical memory should be tied to the inode, i.e. to memory truly being freed, and not to S-EPTs being zapped. And there is a very good reason for wanting to do that, as it allows KVM to do the expensive cache flush + clear outside of mmu_lock. > Deleting memslots should also clear out the contents of the memory as the EPT > tables will be zapped in the process No, deleting a memslot should not clear memory. As I said in my previous response, the fact that zapping S-EPT entries is destructive is a limitiation of TDX, not a feature we want to apply to other VM types. And that's not even a fundamental property of TDX, e.g. TDX could remove the limitation, at the cost of consuming quite a bit more memory, by tracking the exact owner by HKID in the PAMT and decoupling S-EPT entries from page ownership. Or in theory, KVM could workaround the limitation by only doing TDH.MEM.RANGE.BLOCK when zapping S-EPTs. Hmm, that might actually be worth looking at. > and the host will reclaim the memory. There are no guarantees that the host will reclaim the memory. E.g. QEMU will delete and re-create memslots for "regular" VMs when emulating option ROMs. Even if that use case is nonsensical for confidential VMs (and it probably is nonsensical), I don't want to define KVM's ABI based on what we *think* userspace will do. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9559C1172E for ; Fri, 11 Aug 2023 17:44:14 +0000 (UTC) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-586a5ac5c29so26337647b3.1 for ; Fri, 11 Aug 2023 10:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=z4avZMYBxF/USDP2G14KA+P8N8yreov1u2JOHwfc1TVZdl6Iz+YTKH19wDx7mUMq9M o4u6qon1QiMk8DVxh93qeN2Mz6fZGsUFtzOg33bB4Mo7KFsOypRjJRdzz9Y8WFCiY85q br+fPgBO/on1YU5soxnymBtm9OPZyICHaFKZfYSg9Ropko2mo0wKLVwIok1lVrWFw51i jjhtl/HK+nEz+jNhJe7imTF+VBA6+NGFZjFm7+hHqXCkfP3+rYY1oaYSypJ0v6009kFT lq7bEwknPOhWjDhET6YMFrJ/3n1YTZ5hf1bP/IHZB/5/XDyHPzuDGnqfF1kEhe1cc6AD 1nfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=di63/cIWEdTJiO+Uszc6f7vMisFY/Z8fzJ2ntOQDE+sq3kGpT+dqEWfsa9wX5en5OZ VLH/UTJ3+y5qK6N0HGpqpIqtsmufEvTjzEhvt5aUq0wdFrkikMkE2Gcw5jAXZ/zmjzsy +4ibQlQZ1DXBX6vsF9SHEfMZO1gul911EHPnTWS1Ce/9WkhGiJnci6U5Uqu8ewREEELg ICeZUelQc80fQbvPLwBfap93D3foRlQAnMddodC+wcANHdH1hyCFkmxd8pzcwrrW0HME 2cY22HrRX4QvJ4c63gveaUhIr0kdylybEQ36dJt8xoW5nRNXCdfDz1n6A29lfJB8nUwJ 7LdA== X-Gm-Message-State: AOJu0Yy/mPVqEoUYH+0aajYxG/vJ5ZCebDlOVVwidF/FW0NqnQuB3Qbf MpA/GxLF7IElCeOJ1dWZS9Z58rlZRhE= X-Google-Smtp-Source: AGHT+IFlh+eXwF8rMcD2luIRCX1nl8Idt0F+6stO6hsWWfidj415wHUgqoJc8kVAwNGB8y2mQW1tGjk2lPA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:7443:0:b0:d20:7752:e384 with SMTP id p64-20020a257443000000b00d207752e384mr41859ybc.3.1691775853469; Fri, 11 Aug 2023 10:44:13 -0700 (PDT) Date: Fri, 11 Aug 2023 10:44:11 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Cc: Ackerley Tng , pbonzini@redhat.com, maz@kernel.org, oliver.upton@linux.dev, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, willy@infradead.org, akpm@linux-foundation.org, paul@paul-moore.com, jmorris@namei.org, serge@hallyn.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, chao.p.peng@linux.intel.com, tabba@google.com, jarkko@kernel.org, yu.c.zhang@linux.intel.com, mail@maciej.szmigiero.name, vbabka@suse.cz, david@redhat.com, qperret@google.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Thu, Aug 10, 2023, Vishal Annapurve wrote: > On Tue, Aug 8, 2023 at 2:13=E2=80=AFPM Sean Christopherson wrote: > > ... >=20 > > > + When binding a memslot to the file, if a kvm pointer exists, it mus= t > > > be the same kvm as the one in this binding > > > + When the binding to the last memslot is removed from a file, NULL t= he > > > kvm pointer. > > > > Nullifying the KVM pointer isn't sufficient, because without additional= actions > > userspace could extract data from a VM by deleting its memslots and the= n binding > > the guest_memfd to an attacker controlled VM. Or more likely with TDX = and SNP, > > induce badness by coercing KVM into mapping memory into a guest with th= e wrong > > ASID/HKID. > > >=20 > TDX/SNP have mechanisms i.e. PAMT/RMP tables to ensure that the same > memory is not assigned to two different VMs. One of the main reasons we pivoted away from using a flag in "struct page" = to indicate that a page was private was so that KVM could enforce 1:1 VM:page = ownership *without* relying on hardware. And FWIW, the PAMT provides no protection in this specific case because KVM= does TDH.MEM.PAGE.REMOVE when zapping S-EPT entries, and that marks the page cle= ar in the PAMT. The danger there is that physical memory is still encrypted with= the guest's HKID, and so mapping the memory into a different VM, which might no= t be a TDX guest!, could lead to corruption and/or poison #MCs. The HKID issues wouldn't be a problem if v15 is merged as-is, because zappi= ng S-EPT entries also fully purges and reclaims the page, but as we discussed = in one of the many threads, reclaiming physical memory should be tied to the i= node, i.e. to memory truly being freed, and not to S-EPTs being zapped. And ther= e is a very good reason for wanting to do that, as it allows KVM to do the expen= sive cache flush + clear outside of mmu_lock. > Deleting memslots should also clear out the contents of the memory as the= EPT > tables will be zapped in the process No, deleting a memslot should not clear memory. As I said in my previous r= esponse, the fact that zapping S-EPT entries is destructive is a limitiation of TDX,= not a feature we want to apply to other VM types. And that's not even a fundamen= tal property of TDX, e.g. TDX could remove the limitation, at the cost of consu= ming quite a bit more memory, by tracking the exact owner by HKID in the PAMT an= d decoupling S-EPT entries from page ownership. Or in theory, KVM could workaround the limitation by only doing TDH.MEM.RAN= GE.BLOCK when zapping S-EPTs. Hmm, that might actually be worth looking at. > and the host will reclaim the memory. There are no guarantees that the host will reclaim the memory. E.g. QEMU w= ill delete and re-create memslots for "regular" VMs when emulating option ROMs.= Even if that use case is nonsensical for confidential VMs (and it probably is no= nsensical), I don't want to define KVM's ABI based on what we *think* userspace will do= . From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45CC7EB64DD for ; Fri, 11 Aug 2023 17:44:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=yZsNbW4XKEyUOgAffsi6ATW0NIedhZ5Gc21hG4mKfKI=; b=KEP9P/NioJzukv4Cehl9xHu0ju 4IXKZZ49pg9rtnX6PYhY+OoT0RJVj86viI9Fc5aDm8W2rFo1wD35r9qlfgwOmveeMRB0skSbiODQT OGTI1b2c4ATK7TQ94jyXjMwrEZFIP6d9PCYdt106CKIVUlKg8zZtcNRujk/hPk0nJmwwp523rson+ FH1qUKucx65LyZ4ZxwlErelqDuewetp0eiyXvv8fAOxULPtQLGJue8Lle7KuUO18P2iLq4w+umHd/ QzHu0PLCEFKMsNWIV1Osimfy8zmywcPDtk1j8/YebpReImwxSR/B5QtXbp2y+9Akpz5bbr9iiYlhM nibN+kJw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qUWBO-00BGaV-0F; Fri, 11 Aug 2023 17:44:18 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qUWBL-00BGYS-1p for linux-riscv@lists.infradead.org; Fri, 11 Aug 2023 17:44:16 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-583d1d0de65so26422187b3.0 for ; Fri, 11 Aug 2023 10:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=z4avZMYBxF/USDP2G14KA+P8N8yreov1u2JOHwfc1TVZdl6Iz+YTKH19wDx7mUMq9M o4u6qon1QiMk8DVxh93qeN2Mz6fZGsUFtzOg33bB4Mo7KFsOypRjJRdzz9Y8WFCiY85q br+fPgBO/on1YU5soxnymBtm9OPZyICHaFKZfYSg9Ropko2mo0wKLVwIok1lVrWFw51i jjhtl/HK+nEz+jNhJe7imTF+VBA6+NGFZjFm7+hHqXCkfP3+rYY1oaYSypJ0v6009kFT lq7bEwknPOhWjDhET6YMFrJ/3n1YTZ5hf1bP/IHZB/5/XDyHPzuDGnqfF1kEhe1cc6AD 1nfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=HDBZo1Dao3lNTiIDbH4XuWaQld87Brs3uUxxEEtXA82g81a/25KVwTkhJgxhwv/+it o4uUE41szwEbYYax9S2CHr0LiXtPVhVB+0ytgliw9jtnUmPD6E3WZ1BmA/2GntkY7YDg /p6+TdZVTAzRgu6+FqvFiZLuoileIosPjo62dRcllirDhHCRO89PbnoAolOGSu9Ecz0d XBVbAbJjGVfikdyshgqiRLziURVgqm7SjNWIf2xOr1CpNQ9OPX1F1uDPn2JnUMN7Xj1y 0sCN0fvHWH0RqXB2iB0cV4m6JAPtN1Iqs4RPguBJyUtkkeZEvGjhDVsIrItshyVgliQt Vh+w== X-Gm-Message-State: AOJu0Ywz2ze+V8V+zPke1/CApcsX/45A71UY4djytqw1S727vD0JGzxW 6lAfmNfw8BD3oUsYvjnPRS0vlohZG40= X-Google-Smtp-Source: AGHT+IFlh+eXwF8rMcD2luIRCX1nl8Idt0F+6stO6hsWWfidj415wHUgqoJc8kVAwNGB8y2mQW1tGjk2lPA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:7443:0:b0:d20:7752:e384 with SMTP id p64-20020a257443000000b00d207752e384mr41859ybc.3.1691775853469; Fri, 11 Aug 2023 10:44:13 -0700 (PDT) Date: Fri, 11 Aug 2023 10:44:11 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Cc: Ackerley Tng , pbonzini@redhat.com, maz@kernel.org, oliver.upton@linux.dev, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, willy@infradead.org, akpm@linux-foundation.org, paul@paul-moore.com, jmorris@namei.org, serge@hallyn.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, chao.p.peng@linux.intel.com, tabba@google.com, jarkko@kernel.org, yu.c.zhang@linux.intel.com, mail@maciej.szmigiero.name, vbabka@suse.cz, david@redhat.com, qperret@google.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230811_104415_610995_332C078E X-CRM114-Status: GOOD ( 20.57 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gVGh1LCBBdWcgMTAsIDIwMjMsIFZpc2hhbCBBbm5hcHVydmUgd3JvdGU6Cj4gT24gVHVlLCBB dWcgOCwgMjAyMyBhdCAyOjEz4oCvUE0gU2VhbiBDaHJpc3RvcGhlcnNvbiA8c2VhbmpjQGdvb2ds ZS5jb20+IHdyb3RlOgo+ID4gLi4uCj4gCj4gPiA+ICsgV2hlbiBiaW5kaW5nIGEgbWVtc2xvdCB0 byB0aGUgZmlsZSwgaWYgYSBrdm0gcG9pbnRlciBleGlzdHMsIGl0IG11c3QKPiA+ID4gICBiZSB0 aGUgc2FtZSBrdm0gYXMgdGhlIG9uZSBpbiB0aGlzIGJpbmRpbmcKPiA+ID4gKyBXaGVuIHRoZSBi aW5kaW5nIHRvIHRoZSBsYXN0IG1lbXNsb3QgaXMgcmVtb3ZlZCBmcm9tIGEgZmlsZSwgTlVMTCB0 aGUKPiA+ID4gICBrdm0gcG9pbnRlci4KPiA+Cj4gPiBOdWxsaWZ5aW5nIHRoZSBLVk0gcG9pbnRl ciBpc24ndCBzdWZmaWNpZW50LCBiZWNhdXNlIHdpdGhvdXQgYWRkaXRpb25hbCBhY3Rpb25zCj4g PiB1c2Vyc3BhY2UgY291bGQgZXh0cmFjdCBkYXRhIGZyb20gYSBWTSBieSBkZWxldGluZyBpdHMg bWVtc2xvdHMgYW5kIHRoZW4gYmluZGluZwo+ID4gdGhlIGd1ZXN0X21lbWZkIHRvIGFuIGF0dGFj a2VyIGNvbnRyb2xsZWQgVk0uICBPciBtb3JlIGxpa2VseSB3aXRoIFREWCBhbmQgU05QLAo+ID4g aW5kdWNlIGJhZG5lc3MgYnkgY29lcmNpbmcgS1ZNIGludG8gbWFwcGluZyBtZW1vcnkgaW50byBh IGd1ZXN0IHdpdGggdGhlIHdyb25nCj4gPiBBU0lEL0hLSUQuCj4gPgo+IAo+IFREWC9TTlAgaGF2 ZSBtZWNoYW5pc21zIGkuZS4gUEFNVC9STVAgdGFibGVzIHRvIGVuc3VyZSB0aGF0IHRoZSBzYW1l Cj4gbWVtb3J5IGlzIG5vdCBhc3NpZ25lZCB0byB0d28gZGlmZmVyZW50IFZNcy4KCk9uZSBvZiB0 aGUgbWFpbiByZWFzb25zIHdlIHBpdm90ZWQgYXdheSBmcm9tIHVzaW5nIGEgZmxhZyBpbiAic3Ry dWN0IHBhZ2UiIHRvCmluZGljYXRlIHRoYXQgYSBwYWdlIHdhcyBwcml2YXRlIHdhcyBzbyB0aGF0 IEtWTSBjb3VsZCBlbmZvcmNlIDE6MSBWTTpwYWdlIG93bmVyc2hpcAoqd2l0aG91dCogcmVseWlu ZyBvbiBoYXJkd2FyZS4KCkFuZCBGV0lXLCB0aGUgUEFNVCBwcm92aWRlcyBubyBwcm90ZWN0aW9u IGluIHRoaXMgc3BlY2lmaWMgY2FzZSBiZWNhdXNlIEtWTSBkb2VzClRESC5NRU0uUEFHRS5SRU1P VkUgd2hlbiB6YXBwaW5nIFMtRVBUIGVudHJpZXMsIGFuZCB0aGF0IG1hcmtzIHRoZSBwYWdlIGNs ZWFyIGluCnRoZSBQQU1ULiAgVGhlIGRhbmdlciB0aGVyZSBpcyB0aGF0IHBoeXNpY2FsIG1lbW9y eSBpcyBzdGlsbCBlbmNyeXB0ZWQgd2l0aCB0aGUKZ3Vlc3QncyBIS0lELCBhbmQgc28gbWFwcGlu ZyB0aGUgbWVtb3J5IGludG8gYSBkaWZmZXJlbnQgVk0sIHdoaWNoIG1pZ2h0IG5vdCBiZQphIFRE WCBndWVzdCEsIGNvdWxkIGxlYWQgdG8gY29ycnVwdGlvbiBhbmQvb3IgcG9pc29uICNNQ3MuCgpU aGUgSEtJRCBpc3N1ZXMgd291bGRuJ3QgYmUgYSBwcm9ibGVtIGlmIHYxNSBpcyBtZXJnZWQgYXMt aXMsIGJlY2F1c2UgemFwcGluZwpTLUVQVCBlbnRyaWVzIGFsc28gZnVsbHkgcHVyZ2VzIGFuZCBy ZWNsYWltcyB0aGUgcGFnZSwgYnV0IGFzIHdlIGRpc2N1c3NlZCBpbgpvbmUgb2YgdGhlIG1hbnkg dGhyZWFkcywgcmVjbGFpbWluZyBwaHlzaWNhbCBtZW1vcnkgc2hvdWxkIGJlIHRpZWQgdG8gdGhl IGlub2RlLAppLmUuIHRvIG1lbW9yeSB0cnVseSBiZWluZyBmcmVlZCwgYW5kIG5vdCB0byBTLUVQ VHMgYmVpbmcgemFwcGVkLiAgQW5kIHRoZXJlIGlzCmEgdmVyeSBnb29kIHJlYXNvbiBmb3Igd2Fu dGluZyB0byBkbyB0aGF0LCBhcyBpdCBhbGxvd3MgS1ZNIHRvIGRvIHRoZSBleHBlbnNpdmUKY2Fj aGUgZmx1c2ggKyBjbGVhciBvdXRzaWRlIG9mIG1tdV9sb2NrLgoKPiBEZWxldGluZyBtZW1zbG90 cyBzaG91bGQgYWxzbyBjbGVhciBvdXQgdGhlIGNvbnRlbnRzIG9mIHRoZSBtZW1vcnkgYXMgdGhl IEVQVAo+IHRhYmxlcyB3aWxsIGJlIHphcHBlZCBpbiB0aGUgcHJvY2VzcwoKTm8sIGRlbGV0aW5n IGEgbWVtc2xvdCBzaG91bGQgbm90IGNsZWFyIG1lbW9yeS4gIEFzIEkgc2FpZCBpbiBteSBwcmV2 aW91cyByZXNwb25zZSwKdGhlIGZhY3QgdGhhdCB6YXBwaW5nIFMtRVBUIGVudHJpZXMgaXMgZGVz dHJ1Y3RpdmUgaXMgYSBsaW1pdGlhdGlvbiBvZiBURFgsIG5vdCBhCmZlYXR1cmUgd2Ugd2FudCB0 byBhcHBseSB0byBvdGhlciBWTSB0eXBlcy4gIEFuZCB0aGF0J3Mgbm90IGV2ZW4gYSBmdW5kYW1l bnRhbApwcm9wZXJ0eSBvZiBURFgsIGUuZy4gVERYIGNvdWxkIHJlbW92ZSB0aGUgbGltaXRhdGlv biwgYXQgdGhlIGNvc3Qgb2YgY29uc3VtaW5nCnF1aXRlIGEgYml0IG1vcmUgbWVtb3J5LCBieSB0 cmFja2luZyB0aGUgZXhhY3Qgb3duZXIgYnkgSEtJRCBpbiB0aGUgUEFNVCBhbmQKZGVjb3VwbGlu ZyBTLUVQVCBlbnRyaWVzIGZyb20gcGFnZSBvd25lcnNoaXAuCgpPciBpbiB0aGVvcnksIEtWTSBj b3VsZCB3b3JrYXJvdW5kIHRoZSBsaW1pdGF0aW9uIGJ5IG9ubHkgZG9pbmcgVERILk1FTS5SQU5H RS5CTE9DSwp3aGVuIHphcHBpbmcgUy1FUFRzLiAgSG1tLCB0aGF0IG1pZ2h0IGFjdHVhbGx5IGJl IHdvcnRoIGxvb2tpbmcgYXQuCgo+IGFuZCB0aGUgaG9zdCB3aWxsIHJlY2xhaW0gdGhlIG1lbW9y eS4KClRoZXJlIGFyZSBubyBndWFyYW50ZWVzIHRoYXQgdGhlIGhvc3Qgd2lsbCByZWNsYWltIHRo ZSBtZW1vcnkuICBFLmcuIFFFTVUgd2lsbApkZWxldGUgYW5kIHJlLWNyZWF0ZSBtZW1zbG90cyBm b3IgInJlZ3VsYXIiIFZNcyB3aGVuIGVtdWxhdGluZyBvcHRpb24gUk9Ncy4gIEV2ZW4KaWYgdGhh dCB1c2UgY2FzZSBpcyBub25zZW5zaWNhbCBmb3IgY29uZmlkZW50aWFsIFZNcyAoYW5kIGl0IHBy b2JhYmx5IGlzIG5vbnNlbnNpY2FsKSwKSSBkb24ndCB3YW50IHRvIGRlZmluZSBLVk0ncyBBQkkg YmFzZWQgb24gd2hhdCB3ZSAqdGhpbmsqIHVzZXJzcGFjZSB3aWxsIGRvLgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBs aXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVh ZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B85DFEB64DD for ; Fri, 11 Aug 2023 17:45:11 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20221208 header.b=z4avZMYB; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4RMrm56bh3z3c1L for ; Sat, 12 Aug 2023 03:45:09 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20221208 header.b=z4avZMYB; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=flex--seanjc.bounces.google.com (client-ip=2607:f8b0:4864:20::b49; helo=mail-yb1-xb49.google.com; envelope-from=3bxpwzaykdnooa6jf8ckkcha.8kihejqtll8-9arheopo.kvh67o.knc@flex--seanjc.bounces.google.com; receiver=lists.ozlabs.org) Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4RMrl80Ftfz2ydR for ; Sat, 12 Aug 2023 03:44:18 +1000 (AEST) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-c5fc972760eso2331909276.1 for ; Fri, 11 Aug 2023 10:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=z4avZMYBxF/USDP2G14KA+P8N8yreov1u2JOHwfc1TVZdl6Iz+YTKH19wDx7mUMq9M o4u6qon1QiMk8DVxh93qeN2Mz6fZGsUFtzOg33bB4Mo7KFsOypRjJRdzz9Y8WFCiY85q br+fPgBO/on1YU5soxnymBtm9OPZyICHaFKZfYSg9Ropko2mo0wKLVwIok1lVrWFw51i jjhtl/HK+nEz+jNhJe7imTF+VBA6+NGFZjFm7+hHqXCkfP3+rYY1oaYSypJ0v6009kFT lq7bEwknPOhWjDhET6YMFrJ/3n1YTZ5hf1bP/IHZB/5/XDyHPzuDGnqfF1kEhe1cc6AD 1nfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=fPoPA9bv4l1kYcIVrnAWOaTNuduyB+sTosUoJodraP6UeDtll3m12O/XUvGaLs37Ks mMnW8T3IYoJNrGjR7VpltH4gYeB+0uBLPpsSve+cWThkvfT7WbNhZAcJrAtuXBtWEh/c Z1UQTFsxuveSJIJ8ciA1TycD/2fHcwWv6IEg5rCYVtuZ7gGu2T8MyeCKrbyvZo3TdDMj 19cn6HpwT+xxpBvRNN/Piwjs+crWuhbTtJAY6SaS0DEeG8LCuugY4uc5V7n5KZu1GwBF Qf1My4sFQpGXlCueTYGJlvQavlm3XICrsDtQ/+3hxY7eRa0YdLslKG/F+bos6P8btOCC O99w== X-Gm-Message-State: AOJu0YyBZFQlIoInP1T6S5GPLI+3PNT7slGu1WBkUzbNCQ4OorSyZVBG UoNBLMKNvr7Zfr3/PheCiW2/guKhRvU= X-Google-Smtp-Source: AGHT+IFlh+eXwF8rMcD2luIRCX1nl8Idt0F+6stO6hsWWfidj415wHUgqoJc8kVAwNGB8y2mQW1tGjk2lPA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:7443:0:b0:d20:7752:e384 with SMTP id p64-20020a257443000000b00d207752e384mr41859ybc.3.1691775853469; Fri, 11 Aug 2023 10:44:13 -0700 (PDT) Date: Fri, 11 Aug 2023 10:44:11 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, david@redhat.com, yu.c.zhang@linux.intel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, chao.p.peng@linux.intel.com, linux-riscv@lists.infradead.org, isaku.yamahata@gmail.com, paul@paul-moore.com, maz@kernel.org, chenhuacai@kernel.org, jmorris@namei.org, willy@infradead.org, wei.w.wang@intel.com, tabba@google.com, jarkko@kernel.org, serge@hallyn.com, mail@maciej.szmigiero.name, aou@eecs.berkeley.edu, vbabka@suse.cz, michael.roth@amd.com, Ackerley Tng , paul.walmsley@sifive.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, qperret@google.com, liam.merwick@oracle.com, linux-mips@vger.kernel.org, oliver.upton@linux.dev, linux-security-module@vger.kernel.org, palmer@dabbelt.com, kvm-riscv@lists.infradead.org, anup@brainfault.org, linux-fsdevel@vger.kernel.org, pbonzini@redhat.com, akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org, kirill.shutemov@linux.intel.com Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Thu, Aug 10, 2023, Vishal Annapurve wrote: > On Tue, Aug 8, 2023 at 2:13=E2=80=AFPM Sean Christopherson wrote: > > ... >=20 > > > + When binding a memslot to the file, if a kvm pointer exists, it mus= t > > > be the same kvm as the one in this binding > > > + When the binding to the last memslot is removed from a file, NULL t= he > > > kvm pointer. > > > > Nullifying the KVM pointer isn't sufficient, because without additional= actions > > userspace could extract data from a VM by deleting its memslots and the= n binding > > the guest_memfd to an attacker controlled VM. Or more likely with TDX = and SNP, > > induce badness by coercing KVM into mapping memory into a guest with th= e wrong > > ASID/HKID. > > >=20 > TDX/SNP have mechanisms i.e. PAMT/RMP tables to ensure that the same > memory is not assigned to two different VMs. One of the main reasons we pivoted away from using a flag in "struct page" = to indicate that a page was private was so that KVM could enforce 1:1 VM:page = ownership *without* relying on hardware. And FWIW, the PAMT provides no protection in this specific case because KVM= does TDH.MEM.PAGE.REMOVE when zapping S-EPT entries, and that marks the page cle= ar in the PAMT. The danger there is that physical memory is still encrypted with= the guest's HKID, and so mapping the memory into a different VM, which might no= t be a TDX guest!, could lead to corruption and/or poison #MCs. The HKID issues wouldn't be a problem if v15 is merged as-is, because zappi= ng S-EPT entries also fully purges and reclaims the page, but as we discussed = in one of the many threads, reclaiming physical memory should be tied to the i= node, i.e. to memory truly being freed, and not to S-EPTs being zapped. And ther= e is a very good reason for wanting to do that, as it allows KVM to do the expen= sive cache flush + clear outside of mmu_lock. > Deleting memslots should also clear out the contents of the memory as the= EPT > tables will be zapped in the process No, deleting a memslot should not clear memory. As I said in my previous r= esponse, the fact that zapping S-EPT entries is destructive is a limitiation of TDX,= not a feature we want to apply to other VM types. And that's not even a fundamen= tal property of TDX, e.g. TDX could remove the limitation, at the cost of consu= ming quite a bit more memory, by tracking the exact owner by HKID in the PAMT an= d decoupling S-EPT entries from page ownership. Or in theory, KVM could workaround the limitation by only doing TDH.MEM.RAN= GE.BLOCK when zapping S-EPTs. Hmm, that might actually be worth looking at. > and the host will reclaim the memory. There are no guarantees that the host will reclaim the memory. E.g. QEMU w= ill delete and re-create memslots for "regular" VMs when emulating option ROMs.= Even if that use case is nonsensical for confidential VMs (and it probably is no= nsensical), I don't want to define KVM's ABI based on what we *think* userspace will do= . From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3EC2EEB64DD for ; Fri, 11 Aug 2023 17:44:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FZknWQQMPRLyjAp7QWfxw7GJW0/vx1Fr2G++Z1Nk/IA=; b=VD9QnutOleLDVJcv7Q3kRg9b6q pwPhRu+B35cojj7MNfy6hbO+zWk1iE04HHskTZNSb3uoCtz4WwxFrWeEjR9l2epfZsghyL+Ih7WKZ ehLK5bP9o7j6amBAaag90aCXeAaSfplA454OkGyslX32AdkWSmNt1tRRYXvzq+VwGqH/gMv9iDLlW AZzzrdporvVDAFPn7ZhY32mab/G4rCChwwwQQvy81lNL6Z5nt8i/E4jqS1A2OdRUnymU+IPeNob87 ZFXJoIWNq6f8APqPk9DRMsVJ3HmOhwEvN6YhObJv4sny09hR/kCkB5/5Q8qNCHpmVqLZ+qO1m8IRD ujTVR66Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qUWBS-00BGcQ-2r; Fri, 11 Aug 2023 17:44:22 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qUWBL-00BGYR-2c for linux-arm-kernel@lists.infradead.org; Fri, 11 Aug 2023 17:44:17 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d6412374defso1998095276.0 for ; Fri, 11 Aug 2023 10:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=z4avZMYBxF/USDP2G14KA+P8N8yreov1u2JOHwfc1TVZdl6Iz+YTKH19wDx7mUMq9M o4u6qon1QiMk8DVxh93qeN2Mz6fZGsUFtzOg33bB4Mo7KFsOypRjJRdzz9Y8WFCiY85q br+fPgBO/on1YU5soxnymBtm9OPZyICHaFKZfYSg9Ropko2mo0wKLVwIok1lVrWFw51i jjhtl/HK+nEz+jNhJe7imTF+VBA6+NGFZjFm7+hHqXCkfP3+rYY1oaYSypJ0v6009kFT lq7bEwknPOhWjDhET6YMFrJ/3n1YTZ5hf1bP/IHZB/5/XDyHPzuDGnqfF1kEhe1cc6AD 1nfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691775853; x=1692380653; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=y5tf/etP9591sBoXXP0u0lhmE2sGhQYCIj4rXej+PgY=; b=TZioe0zMDuq68/jbuEeNFUK6gj/EOFnZlCrOghvHpkG196rP9x5LOa2J3cqNEnLJ4L bW6ftAXQ2DtCG5IF0k3bvXJbdWPJZcRPk+SgfCUbKEzSuj2vPvLM2Ia2nxFael+yrWrP DjkhN7WF97jbUa1Ue1QeILH8qdyq7nVExUY9HxmrhjAZsTNbjSrRqSRYWxhUJEKMT+zP SPK6fu7Czwm4uN6wBwy86Gz6rsS/vDDDPxP+Y601zRa+ewe1D0ztPNnX0Bc/qGD3exLA 2G1m3zZbfjPZiI8LNZNvMUbCwxq5pqhVJDKQpDTGszAGIvlC1tELUZWEqhLD80TMb7r5 cU1g== X-Gm-Message-State: AOJu0YwGHNtURyqloS6zMBVOEfnSnvLagoUgZyB2TNj85jCzubUUUaIp 3KBQflqsSnbd7rT49EMqMjj4oWlY+/U= X-Google-Smtp-Source: AGHT+IFlh+eXwF8rMcD2luIRCX1nl8Idt0F+6stO6hsWWfidj415wHUgqoJc8kVAwNGB8y2mQW1tGjk2lPA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:7443:0:b0:d20:7752:e384 with SMTP id p64-20020a257443000000b00d207752e384mr41859ybc.3.1691775853469; Fri, 11 Aug 2023 10:44:13 -0700 (PDT) Date: Fri, 11 Aug 2023 10:44:11 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Cc: Ackerley Tng , pbonzini@redhat.com, maz@kernel.org, oliver.upton@linux.dev, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, willy@infradead.org, akpm@linux-foundation.org, paul@paul-moore.com, jmorris@namei.org, serge@hallyn.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, chao.p.peng@linux.intel.com, tabba@google.com, jarkko@kernel.org, yu.c.zhang@linux.intel.com, mail@maciej.szmigiero.name, vbabka@suse.cz, david@redhat.com, qperret@google.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230811_104415_855135_BF3E6C58 X-CRM114-Status: GOOD ( 22.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVGh1LCBBdWcgMTAsIDIwMjMsIFZpc2hhbCBBbm5hcHVydmUgd3JvdGU6Cj4gT24gVHVlLCBB dWcgOCwgMjAyMyBhdCAyOjEz4oCvUE0gU2VhbiBDaHJpc3RvcGhlcnNvbiA8c2VhbmpjQGdvb2ds ZS5jb20+IHdyb3RlOgo+ID4gLi4uCj4gCj4gPiA+ICsgV2hlbiBiaW5kaW5nIGEgbWVtc2xvdCB0 byB0aGUgZmlsZSwgaWYgYSBrdm0gcG9pbnRlciBleGlzdHMsIGl0IG11c3QKPiA+ID4gICBiZSB0 aGUgc2FtZSBrdm0gYXMgdGhlIG9uZSBpbiB0aGlzIGJpbmRpbmcKPiA+ID4gKyBXaGVuIHRoZSBi aW5kaW5nIHRvIHRoZSBsYXN0IG1lbXNsb3QgaXMgcmVtb3ZlZCBmcm9tIGEgZmlsZSwgTlVMTCB0 aGUKPiA+ID4gICBrdm0gcG9pbnRlci4KPiA+Cj4gPiBOdWxsaWZ5aW5nIHRoZSBLVk0gcG9pbnRl ciBpc24ndCBzdWZmaWNpZW50LCBiZWNhdXNlIHdpdGhvdXQgYWRkaXRpb25hbCBhY3Rpb25zCj4g PiB1c2Vyc3BhY2UgY291bGQgZXh0cmFjdCBkYXRhIGZyb20gYSBWTSBieSBkZWxldGluZyBpdHMg bWVtc2xvdHMgYW5kIHRoZW4gYmluZGluZwo+ID4gdGhlIGd1ZXN0X21lbWZkIHRvIGFuIGF0dGFj a2VyIGNvbnRyb2xsZWQgVk0uICBPciBtb3JlIGxpa2VseSB3aXRoIFREWCBhbmQgU05QLAo+ID4g aW5kdWNlIGJhZG5lc3MgYnkgY29lcmNpbmcgS1ZNIGludG8gbWFwcGluZyBtZW1vcnkgaW50byBh IGd1ZXN0IHdpdGggdGhlIHdyb25nCj4gPiBBU0lEL0hLSUQuCj4gPgo+IAo+IFREWC9TTlAgaGF2 ZSBtZWNoYW5pc21zIGkuZS4gUEFNVC9STVAgdGFibGVzIHRvIGVuc3VyZSB0aGF0IHRoZSBzYW1l Cj4gbWVtb3J5IGlzIG5vdCBhc3NpZ25lZCB0byB0d28gZGlmZmVyZW50IFZNcy4KCk9uZSBvZiB0 aGUgbWFpbiByZWFzb25zIHdlIHBpdm90ZWQgYXdheSBmcm9tIHVzaW5nIGEgZmxhZyBpbiAic3Ry dWN0IHBhZ2UiIHRvCmluZGljYXRlIHRoYXQgYSBwYWdlIHdhcyBwcml2YXRlIHdhcyBzbyB0aGF0 IEtWTSBjb3VsZCBlbmZvcmNlIDE6MSBWTTpwYWdlIG93bmVyc2hpcAoqd2l0aG91dCogcmVseWlu ZyBvbiBoYXJkd2FyZS4KCkFuZCBGV0lXLCB0aGUgUEFNVCBwcm92aWRlcyBubyBwcm90ZWN0aW9u IGluIHRoaXMgc3BlY2lmaWMgY2FzZSBiZWNhdXNlIEtWTSBkb2VzClRESC5NRU0uUEFHRS5SRU1P VkUgd2hlbiB6YXBwaW5nIFMtRVBUIGVudHJpZXMsIGFuZCB0aGF0IG1hcmtzIHRoZSBwYWdlIGNs ZWFyIGluCnRoZSBQQU1ULiAgVGhlIGRhbmdlciB0aGVyZSBpcyB0aGF0IHBoeXNpY2FsIG1lbW9y eSBpcyBzdGlsbCBlbmNyeXB0ZWQgd2l0aCB0aGUKZ3Vlc3QncyBIS0lELCBhbmQgc28gbWFwcGlu ZyB0aGUgbWVtb3J5IGludG8gYSBkaWZmZXJlbnQgVk0sIHdoaWNoIG1pZ2h0IG5vdCBiZQphIFRE WCBndWVzdCEsIGNvdWxkIGxlYWQgdG8gY29ycnVwdGlvbiBhbmQvb3IgcG9pc29uICNNQ3MuCgpU aGUgSEtJRCBpc3N1ZXMgd291bGRuJ3QgYmUgYSBwcm9ibGVtIGlmIHYxNSBpcyBtZXJnZWQgYXMt aXMsIGJlY2F1c2UgemFwcGluZwpTLUVQVCBlbnRyaWVzIGFsc28gZnVsbHkgcHVyZ2VzIGFuZCBy ZWNsYWltcyB0aGUgcGFnZSwgYnV0IGFzIHdlIGRpc2N1c3NlZCBpbgpvbmUgb2YgdGhlIG1hbnkg dGhyZWFkcywgcmVjbGFpbWluZyBwaHlzaWNhbCBtZW1vcnkgc2hvdWxkIGJlIHRpZWQgdG8gdGhl IGlub2RlLAppLmUuIHRvIG1lbW9yeSB0cnVseSBiZWluZyBmcmVlZCwgYW5kIG5vdCB0byBTLUVQ VHMgYmVpbmcgemFwcGVkLiAgQW5kIHRoZXJlIGlzCmEgdmVyeSBnb29kIHJlYXNvbiBmb3Igd2Fu dGluZyB0byBkbyB0aGF0LCBhcyBpdCBhbGxvd3MgS1ZNIHRvIGRvIHRoZSBleHBlbnNpdmUKY2Fj aGUgZmx1c2ggKyBjbGVhciBvdXRzaWRlIG9mIG1tdV9sb2NrLgoKPiBEZWxldGluZyBtZW1zbG90 cyBzaG91bGQgYWxzbyBjbGVhciBvdXQgdGhlIGNvbnRlbnRzIG9mIHRoZSBtZW1vcnkgYXMgdGhl IEVQVAo+IHRhYmxlcyB3aWxsIGJlIHphcHBlZCBpbiB0aGUgcHJvY2VzcwoKTm8sIGRlbGV0aW5n IGEgbWVtc2xvdCBzaG91bGQgbm90IGNsZWFyIG1lbW9yeS4gIEFzIEkgc2FpZCBpbiBteSBwcmV2 aW91cyByZXNwb25zZSwKdGhlIGZhY3QgdGhhdCB6YXBwaW5nIFMtRVBUIGVudHJpZXMgaXMgZGVz dHJ1Y3RpdmUgaXMgYSBsaW1pdGlhdGlvbiBvZiBURFgsIG5vdCBhCmZlYXR1cmUgd2Ugd2FudCB0 byBhcHBseSB0byBvdGhlciBWTSB0eXBlcy4gIEFuZCB0aGF0J3Mgbm90IGV2ZW4gYSBmdW5kYW1l bnRhbApwcm9wZXJ0eSBvZiBURFgsIGUuZy4gVERYIGNvdWxkIHJlbW92ZSB0aGUgbGltaXRhdGlv biwgYXQgdGhlIGNvc3Qgb2YgY29uc3VtaW5nCnF1aXRlIGEgYml0IG1vcmUgbWVtb3J5LCBieSB0 cmFja2luZyB0aGUgZXhhY3Qgb3duZXIgYnkgSEtJRCBpbiB0aGUgUEFNVCBhbmQKZGVjb3VwbGlu ZyBTLUVQVCBlbnRyaWVzIGZyb20gcGFnZSBvd25lcnNoaXAuCgpPciBpbiB0aGVvcnksIEtWTSBj b3VsZCB3b3JrYXJvdW5kIHRoZSBsaW1pdGF0aW9uIGJ5IG9ubHkgZG9pbmcgVERILk1FTS5SQU5H RS5CTE9DSwp3aGVuIHphcHBpbmcgUy1FUFRzLiAgSG1tLCB0aGF0IG1pZ2h0IGFjdHVhbGx5IGJl IHdvcnRoIGxvb2tpbmcgYXQuCgo+IGFuZCB0aGUgaG9zdCB3aWxsIHJlY2xhaW0gdGhlIG1lbW9y eS4KClRoZXJlIGFyZSBubyBndWFyYW50ZWVzIHRoYXQgdGhlIGhvc3Qgd2lsbCByZWNsYWltIHRo ZSBtZW1vcnkuICBFLmcuIFFFTVUgd2lsbApkZWxldGUgYW5kIHJlLWNyZWF0ZSBtZW1zbG90cyBm b3IgInJlZ3VsYXIiIFZNcyB3aGVuIGVtdWxhdGluZyBvcHRpb24gUk9Ncy4gIEV2ZW4KaWYgdGhh dCB1c2UgY2FzZSBpcyBub25zZW5zaWNhbCBmb3IgY29uZmlkZW50aWFsIFZNcyAoYW5kIGl0IHBy b2JhYmx5IGlzIG5vbnNlbnNpY2FsKSwKSSBkb24ndCB3YW50IHRvIGRlZmluZSBLVk0ncyBBQkkg YmFzZWQgb24gd2hhdCB3ZSAqdGhpbmsqIHVzZXJzcGFjZSB3aWxsIGRvLgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWls aW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0 cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=