From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Wed, 19 Jul 2023 10:47:41 -0700 Subject: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory In-Reply-To: References: <20230718234512.1690985-1-seanjc@google.com> <20230718234512.1690985-13-seanjc@google.com> Message-ID: List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Wed, Jul 19, 2023, Vishal Annapurve wrote: > On Tue, Jul 18, 2023 at 4:49?PM Sean Christopherson wrote: > > ... > > +static int kvm_gmem_error_page(struct address_space *mapping, struct page *page) > > +{ > > + struct list_head *gmem_list = &mapping->private_list; > > + struct kvm_memory_slot *slot; > > + struct kvm_gmem *gmem; > > + unsigned long index; > > + pgoff_t start, end; > > + gfn_t gfn; > > + > > + filemap_invalidate_lock_shared(mapping); > > + > > + start = page->index; > > + end = start + thp_nr_pages(page); > > + > > + list_for_each_entry(gmem, gmem_list, entry) { > > + xa_for_each_range(&gmem->bindings, index, slot, start, end - 1) { > > + for (gfn = start; gfn < end; gfn++) { > > + if (WARN_ON_ONCE(gfn < slot->base_gfn || > > + gfn >= slot->base_gfn + slot->npages)) > > + continue; > > + > > + /* > > + * FIXME: Tell userspace that the *private* > > + * memory encountered an error. > > + */ > > + send_sig_mceerr(BUS_MCEERR_AR, > > + (void __user *)gfn_to_hva_memslot(slot, gfn), > > + PAGE_SHIFT, current); > > Does it make sense to replicate what happens with MCE handling on > tmpfs backed guest memory: > 1) Unmap gpa from guest > 2) On the next guest EPT fault, exit to userspace to handle/log the > mce error for the gpa. Hmm, yes, that would be much better. Ah, and kvm_gmem_get_pfn() needs to check folio_test_hwpoison() and potentially PageHWPoison(). E.g. if the folio is huge, KVM needs to restrict the mapping to order-0 (target page isn't poisoned), or return KVM_PFN_ERR_HWPOISON (taget page IS poisoned). Alternatively, KVM could punch a hole in kvm_gmem_error_page(), but I don't think we want to do that because that would prevent forwarding the #MC to the guest. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F42361FB54 for ; Wed, 19 Jul 2023 17:47:44 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-ca8e5f39e09so1760321276.0 for ; Wed, 19 Jul 2023 10:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689788864; x=1692380864; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=ArpFLxX4pd+nE6yhxk4kfQsyY2cBj0283wyB2UFD4pQNpbkrc+ZAbqxw+z+OgClanI 8Bmj2Z7j2c1NhNgCvBdv83cjgOHHCOOMDHYftHtZeAL3vo412/Xbeep/S+CdApdBGNVE aduoGH7mk12lxFjmYh58eVW/1Ksc0OT9NLaSC1w5eMBSYs+0sTNoL7N0w3aZ0n2P+aH0 VnFxLOrHPJ9r2ezOXOfj0cBYblwBN0Ye2w9Cgtba6HWxXo2yyLAo7PNbRfr5GAzKlZhF ic8thdmStV5nJQreXCY0jlMVC8WTwDKkeNH3l7/QHaf5X+Z1Dsac59JBXIEJlOoDRy4l I3Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689788864; x=1692380864; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=ahuJV6p0jT9DjzetHeR9MhVUm/XQ3B8bjT3xDxhnZ1qEz5UAmYLva9ugM9345Npl1E /uN3O4EEYV/B2OeCxB4RO95lxAEQRztS/CKDDE0gCL9i01oRSPFmLdl2NROtt5kCeyog uEj/+wiIt5hUmQvpzBQCsJjnDn3uLI/CkMvfWBlXzfZNyJLgDxyl6vFI+XvHCjNeglnc CTCNcwPl81Dqn+xm3m6chvMmWrQYR+i+TndZH+EolXhgbydP3ijVISkcoRZ9GVTbRInf dKFC857KpdWHjxLiN1G6bstcudm19r+NysLsob7LoOJEiNXnf2QnkjTNHJlIrNHszRfN 5w/A== X-Gm-Message-State: ABy/qLZi/w34srXjD56gIdhdv1vt134r6bRiUToQBYkTj6rg7cy1yBgg 58mmxnP8/2wKqC8YymMbYO6U0oj4QkE= X-Google-Smtp-Source: APBJJlFlYUVvIIObTojpJ8UchQ4CwQtQGAVrCh++KpJ7LgJpdZRIA9xQwbGLQUVWTMKWnIrBZUJv99cvfhs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:f509:0:b0:ca3:3341:6315 with SMTP id a9-20020a25f509000000b00ca333416315mr42238ybe.0.1689788863726; Wed, 19 Jul 2023 10:47:43 -0700 (PDT) Date: Wed, 19 Jul 2023 10:47:41 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20230718234512.1690985-1-seanjc@google.com> <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Yu Zhang , Ackerley Tng , Maciej Szmigiero , Vlastimil Babka , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Wed, Jul 19, 2023, Vishal Annapurve wrote: > On Tue, Jul 18, 2023 at 4:49=E2=80=AFPM Sean Christopherson wrote: > > ... > > +static int kvm_gmem_error_page(struct address_space *mapping, struct p= age *page) > > +{ > > + struct list_head *gmem_list =3D &mapping->private_list; > > + struct kvm_memory_slot *slot; > > + struct kvm_gmem *gmem; > > + unsigned long index; > > + pgoff_t start, end; > > + gfn_t gfn; > > + > > + filemap_invalidate_lock_shared(mapping); > > + > > + start =3D page->index; > > + end =3D start + thp_nr_pages(page); > > + > > + list_for_each_entry(gmem, gmem_list, entry) { > > + xa_for_each_range(&gmem->bindings, index, slot, start, = end - 1) { > > + for (gfn =3D start; gfn < end; gfn++) { > > + if (WARN_ON_ONCE(gfn < slot->base_gfn |= | > > + gfn >=3D slot->base_gfn= + slot->npages)) > > + continue; > > + > > + /* > > + * FIXME: Tell userspace that the *priv= ate* > > + * memory encountered an error. > > + */ > > + send_sig_mceerr(BUS_MCEERR_AR, > > + (void __user *)gfn_to_h= va_memslot(slot, gfn), > > + PAGE_SHIFT, current); >=20 > Does it make sense to replicate what happens with MCE handling on > tmpfs backed guest memory: > 1) Unmap gpa from guest > 2) On the next guest EPT fault, exit to userspace to handle/log the > mce error for the gpa. Hmm, yes, that would be much better. Ah, and kvm_gmem_get_pfn() needs to c= heck folio_test_hwpoison() and potentially PageHWPoison(). E.g. if the folio is= huge, KVM needs to restrict the mapping to order-0 (target page isn't poisoned), = or return KVM_PFN_ERR_HWPOISON (taget page IS poisoned). Alternatively, KVM could punch a hole in kvm_gmem_error_page(), but I don't= think we want to do that because that would prevent forwarding the #MC to the gue= st. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 77EA8C001DE for ; Wed, 19 Jul 2023 17:47:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=meP7ojMNYD/wjRgbpefQO2QNv9ULnSvPLu0+zyhIMsA=; b=KJz9ADtwH0R3osJUhQOqo3ZCbr QdD5xPPeJMTtzti7dlKabSPqpEGvXHNuKVFP9ovAYhHq57FjIORzcOJFr800Nup+QyrfCBBFXQVCV wcOJxzLeJFjYv4TN3qCzTvk6B/avhYmjm/ELT/c5LJuVbgjb6Vfi/LEE30piePmB4y1NQFbaplkQj 6TPuknZ0sQNZU8RivebvJVFiX247A2ET0v0kXTE8OofGP+W9HUwTab2k8196HumM4QfxtrRz7YI/u SJXW48XaVupJKpyFS5N+gvaCfqSW5lTt3vNUIxykO6FA0e0k48ER4GZ/hl/+Gm3aLKnth12Ry2h9/ qJZklTcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qMBHE-008P3S-2t; Wed, 19 Jul 2023 17:47:52 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qMBHA-008OxI-29 for linux-riscv@lists.infradead.org; Wed, 19 Jul 2023 17:47:50 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-ca8e5f39e09so1760322276.0 for ; Wed, 19 Jul 2023 10:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689788864; x=1692380864; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=ArpFLxX4pd+nE6yhxk4kfQsyY2cBj0283wyB2UFD4pQNpbkrc+ZAbqxw+z+OgClanI 8Bmj2Z7j2c1NhNgCvBdv83cjgOHHCOOMDHYftHtZeAL3vo412/Xbeep/S+CdApdBGNVE aduoGH7mk12lxFjmYh58eVW/1Ksc0OT9NLaSC1w5eMBSYs+0sTNoL7N0w3aZ0n2P+aH0 VnFxLOrHPJ9r2ezOXOfj0cBYblwBN0Ye2w9Cgtba6HWxXo2yyLAo7PNbRfr5GAzKlZhF ic8thdmStV5nJQreXCY0jlMVC8WTwDKkeNH3l7/QHaf5X+Z1Dsac59JBXIEJlOoDRy4l I3Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689788864; x=1692380864; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=HvTzcoglGo8bjf4c7P5T8LkP7rariXchJ+kBH6tLY7Hg6FQG1AgGnkk11i2KvQl8AL kyvPRJT5OlCA2PeGn+GIQ+BspurK3MotwVVcjk85lyGDBxroro7M4TGxNlBHLi3MKDxI r+RmtsspTxbkO9eeijNDeYY0vz1u7ZEo2wfTAzmsRFA4GuLYNQABX/2N9ZakVz8LDxJ6 gG+xyOds9tHMgrob0IimN2nWmbCvN9Mf6aPfbDdPtx6hom0JXb6vAPObYRSRnR0J2A/l 6xRQuItMuvo9oB1/q8vetQa+yB4BllxFwMO2Na3/ozJgBm4EGvZvDx/uAoDz3rsb0WUz BLQw== X-Gm-Message-State: ABy/qLZKXzXOdm/QQEM4tVVg4lnB8OevuRGfXmPPibp3jQ7xGI+dJ/3T BdzjVnzUqwUVcpofCCKkdk/9lVvRF2c= X-Google-Smtp-Source: APBJJlFlYUVvIIObTojpJ8UchQ4CwQtQGAVrCh++KpJ7LgJpdZRIA9xQwbGLQUVWTMKWnIrBZUJv99cvfhs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:f509:0:b0:ca3:3341:6315 with SMTP id a9-20020a25f509000000b00ca333416315mr42238ybe.0.1689788863726; Wed, 19 Jul 2023 10:47:43 -0700 (PDT) Date: Wed, 19 Jul 2023 10:47:41 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230718234512.1690985-1-seanjc@google.com> <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Yu Zhang , Ackerley Tng , Maciej Szmigiero , Vlastimil Babka , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230719_104748_727508_AFD1A8F9 X-CRM114-Status: GOOD ( 16.68 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBKdWwgMTksIDIwMjMsIFZpc2hhbCBBbm5hcHVydmUgd3JvdGU6Cj4gT24gVHVlLCBK dWwgMTgsIDIwMjMgYXQgNDo0OeKAr1BNIFNlYW4gQ2hyaXN0b3BoZXJzb24gPHNlYW5qY0Bnb29n bGUuY29tPiB3cm90ZToKPiA+IC4uLgo+ID4gK3N0YXRpYyBpbnQga3ZtX2dtZW1fZXJyb3JfcGFn ZShzdHJ1Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZywgc3RydWN0IHBhZ2UgKnBhZ2UpCj4gPiAr ewo+ID4gKyAgICAgICBzdHJ1Y3QgbGlzdF9oZWFkICpnbWVtX2xpc3QgPSAmbWFwcGluZy0+cHJp dmF0ZV9saXN0Owo+ID4gKyAgICAgICBzdHJ1Y3Qga3ZtX21lbW9yeV9zbG90ICpzbG90Owo+ID4g KyAgICAgICBzdHJ1Y3Qga3ZtX2dtZW0gKmdtZW07Cj4gPiArICAgICAgIHVuc2lnbmVkIGxvbmcg aW5kZXg7Cj4gPiArICAgICAgIHBnb2ZmX3Qgc3RhcnQsIGVuZDsKPiA+ICsgICAgICAgZ2ZuX3Qg Z2ZuOwo+ID4gKwo+ID4gKyAgICAgICBmaWxlbWFwX2ludmFsaWRhdGVfbG9ja19zaGFyZWQobWFw cGluZyk7Cj4gPiArCj4gPiArICAgICAgIHN0YXJ0ID0gcGFnZS0+aW5kZXg7Cj4gPiArICAgICAg IGVuZCA9IHN0YXJ0ICsgdGhwX25yX3BhZ2VzKHBhZ2UpOwo+ID4gKwo+ID4gKyAgICAgICBsaXN0 X2Zvcl9lYWNoX2VudHJ5KGdtZW0sIGdtZW1fbGlzdCwgZW50cnkpIHsKPiA+ICsgICAgICAgICAg ICAgICB4YV9mb3JfZWFjaF9yYW5nZSgmZ21lbS0+YmluZGluZ3MsIGluZGV4LCBzbG90LCBzdGFy dCwgZW5kIC0gMSkgewo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgZm9yIChnZm4gPSBzdGFy dDsgZ2ZuIDwgZW5kOyBnZm4rKykgewo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBpZiAoV0FSTl9PTl9PTkNFKGdmbiA8IHNsb3QtPmJhc2VfZ2ZuIHx8Cj4gPiArICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBnZm4gPj0gc2xvdC0+YmFzZV9n Zm4gKyBzbG90LT5ucGFnZXMpKQo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIGNvbnRpbnVlOwo+ID4gKwo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAvKgo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKiBGSVhNRTogVGVsbCB1 c2Vyc3BhY2UgdGhhdCB0aGUgKnByaXZhdGUqCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAqIG1lbW9yeSBlbmNvdW50ZXJlZCBhbiBlcnJvci4KPiA+ICsgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICovCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHNlbmRfc2lnX21jZWVycihCVVNfTUNFRVJSX0FSLAo+ID4gKyAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHZvaWQgX191c2VyICopZ2ZuX3RvX2h2YV9tZW1z bG90KHNsb3QsIGdmbiksCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBQQUdFX1NISUZULCBjdXJyZW50KTsKPiAKPiBEb2VzIGl0IG1ha2Ugc2Vuc2Ug dG8gcmVwbGljYXRlIHdoYXQgaGFwcGVucyB3aXRoIE1DRSBoYW5kbGluZyBvbgo+IHRtcGZzIGJh Y2tlZCBndWVzdCBtZW1vcnk6Cj4gMSkgVW5tYXAgZ3BhIGZyb20gZ3Vlc3QKPiAyKSBPbiB0aGUg bmV4dCBndWVzdCBFUFQgZmF1bHQsIGV4aXQgdG8gdXNlcnNwYWNlIHRvIGhhbmRsZS9sb2cgdGhl Cj4gbWNlIGVycm9yIGZvciB0aGUgZ3BhLgoKSG1tLCB5ZXMsIHRoYXQgd291bGQgYmUgbXVjaCBi ZXR0ZXIuICBBaCwgYW5kIGt2bV9nbWVtX2dldF9wZm4oKSBuZWVkcyB0byBjaGVjawpmb2xpb190 ZXN0X2h3cG9pc29uKCkgYW5kIHBvdGVudGlhbGx5IFBhZ2VIV1BvaXNvbigpLiAgRS5nLiBpZiB0 aGUgZm9saW8gaXMgaHVnZSwKS1ZNIG5lZWRzIHRvIHJlc3RyaWN0IHRoZSBtYXBwaW5nIHRvIG9y ZGVyLTAgKHRhcmdldCBwYWdlIGlzbid0IHBvaXNvbmVkKSwgb3IKcmV0dXJuIEtWTV9QRk5fRVJS X0hXUE9JU09OICh0YWdldCBwYWdlIElTIHBvaXNvbmVkKS4KCkFsdGVybmF0aXZlbHksIEtWTSBj b3VsZCBwdW5jaCBhIGhvbGUgaW4ga3ZtX2dtZW1fZXJyb3JfcGFnZSgpLCBidXQgSSBkb24ndCB0 aGluawp3ZSB3YW50IHRvIGRvIHRoYXQgYmVjYXVzZSB0aGF0IHdvdWxkIHByZXZlbnQgZm9yd2Fy ZGluZyB0aGUgI01DIHRvIHRoZSBndWVzdC4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmxpbnV4LXJpc2N2IG1haWxpbmcgbGlzdApsaW51eC1yaXNjdkBs aXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlz dGluZm8vbGludXgtcmlzY3YK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9403DC00528 for ; Wed, 19 Jul 2023 17:48:44 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20221208 header.b=ArpFLxX4; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4R5jwq1bVvz3bNp for ; Thu, 20 Jul 2023 03:48:43 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20221208 header.b=ArpFLxX4; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=flex--seanjc.bounces.google.com (client-ip=2607:f8b0:4864:20::b49; helo=mail-yb1-xb49.google.com; envelope-from=3vyg4zaykdba8uq3zsw44w1u.s421y3ad55s-tub1y898.4f1qr8.47w@flex--seanjc.bounces.google.com; receiver=lists.ozlabs.org) Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4R5jvl232vz3c1m for ; Thu, 20 Jul 2023 03:47:46 +1000 (AEST) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-c361777c7f7so1771337276.0 for ; Wed, 19 Jul 2023 10:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689788864; x=1692380864; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=ArpFLxX4pd+nE6yhxk4kfQsyY2cBj0283wyB2UFD4pQNpbkrc+ZAbqxw+z+OgClanI 8Bmj2Z7j2c1NhNgCvBdv83cjgOHHCOOMDHYftHtZeAL3vo412/Xbeep/S+CdApdBGNVE aduoGH7mk12lxFjmYh58eVW/1Ksc0OT9NLaSC1w5eMBSYs+0sTNoL7N0w3aZ0n2P+aH0 VnFxLOrHPJ9r2ezOXOfj0cBYblwBN0Ye2w9Cgtba6HWxXo2yyLAo7PNbRfr5GAzKlZhF ic8thdmStV5nJQreXCY0jlMVC8WTwDKkeNH3l7/QHaf5X+Z1Dsac59JBXIEJlOoDRy4l I3Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689788864; x=1692380864; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=ZIWH1o0Y+iUq66BRcNZ5CYsxJ/ekH54WA7yspWA7Yxc1PrH7ohgYdu3gkQHTomQJNl XkyOVJiuyQFbu1KJTv+tlOr9lnNjHp2TDR8YiHbChxCQzjvd9UefHZwRjKz5gdQ506PO bqjEL5a944UvZaQy0/aoPfnjchbSAaZmPGSiq+iu5CtcEZS4tzID+VBzTD+Ov9mNLDkK AWd6YmqFLgYWjk/2Y1O7S6ya5i3Wu1jU/cX9iKz32RBWvVeqtv9duq6qheuTj8DQUMWC OP4xqv7pYYXtzV3bM78ZSDFTyBhc9GeZn5wrAXxcg0ye9sqDlqGp+S6FEbomYlkuQlLy Bp2w== X-Gm-Message-State: ABy/qLbj/APY+RfvVNRDZDxsIHzbV3oLcRDbEZ/PV5xaCxyedUZik1Dz lstTaNPR3eJlynzsR5Y/TYYpyNL4fwg= X-Google-Smtp-Source: APBJJlFlYUVvIIObTojpJ8UchQ4CwQtQGAVrCh++KpJ7LgJpdZRIA9xQwbGLQUVWTMKWnIrBZUJv99cvfhs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:f509:0:b0:ca3:3341:6315 with SMTP id a9-20020a25f509000000b00ca333416315mr42238ybe.0.1689788863726; Wed, 19 Jul 2023 10:47:43 -0700 (PDT) Date: Wed, 19 Jul 2023 10:47:41 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230718234512.1690985-1-seanjc@google.com> <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, David Hildenbrand , Yu Zhang , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chao Peng , linux-riscv@lists.infradead.org, Isaku Yamahata , Paul Moore , Marc Zyngier , Huacai Chen , James Morris , "Matthew Wilcox \(Oracle\)" , Wang , Fuad Tabba , Jarkko Sakkinen , "Serge E. Hallyn" , Maciej Szmigiero , Albert Ou , Vlastimil Babka , Michael Roth , Ackerley Tng , Paul Walmsley , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Quentin Perret , Liam Merwick , linux-mips@vger.kernel.org, Oliver Upton , linux-security-module@vger.kernel.org, Palmer Dabbelt , kvm-riscv@lists.infradead.org, Anup Patel , linux-fsdevel@vger.kernel.org, Paolo Bonzini , Andrew Morton , linuxppc-dev@lists.ozlabs.org, "Kirill A . Shutemov" Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, Jul 19, 2023, Vishal Annapurve wrote: > On Tue, Jul 18, 2023 at 4:49=E2=80=AFPM Sean Christopherson wrote: > > ... > > +static int kvm_gmem_error_page(struct address_space *mapping, struct p= age *page) > > +{ > > + struct list_head *gmem_list =3D &mapping->private_list; > > + struct kvm_memory_slot *slot; > > + struct kvm_gmem *gmem; > > + unsigned long index; > > + pgoff_t start, end; > > + gfn_t gfn; > > + > > + filemap_invalidate_lock_shared(mapping); > > + > > + start =3D page->index; > > + end =3D start + thp_nr_pages(page); > > + > > + list_for_each_entry(gmem, gmem_list, entry) { > > + xa_for_each_range(&gmem->bindings, index, slot, start, = end - 1) { > > + for (gfn =3D start; gfn < end; gfn++) { > > + if (WARN_ON_ONCE(gfn < slot->base_gfn |= | > > + gfn >=3D slot->base_gfn= + slot->npages)) > > + continue; > > + > > + /* > > + * FIXME: Tell userspace that the *priv= ate* > > + * memory encountered an error. > > + */ > > + send_sig_mceerr(BUS_MCEERR_AR, > > + (void __user *)gfn_to_h= va_memslot(slot, gfn), > > + PAGE_SHIFT, current); >=20 > Does it make sense to replicate what happens with MCE handling on > tmpfs backed guest memory: > 1) Unmap gpa from guest > 2) On the next guest EPT fault, exit to userspace to handle/log the > mce error for the gpa. Hmm, yes, that would be much better. Ah, and kvm_gmem_get_pfn() needs to c= heck folio_test_hwpoison() and potentially PageHWPoison(). E.g. if the folio is= huge, KVM needs to restrict the mapping to order-0 (target page isn't poisoned), = or return KVM_PFN_ERR_HWPOISON (taget page IS poisoned). Alternatively, KVM could punch a hole in kvm_gmem_error_page(), but I don't= think we want to do that because that would prevent forwarding the #MC to the gue= st. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6276C001B0 for ; Wed, 19 Jul 2023 17:48:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=DrOAKPkUTP24Zlb/q8ffNteH0HP3ZnhEhFGsRMmnNjg=; b=hBCOEuO7Kb123ZeSMKNNYdSZYw yLbbPxiqPrEBf+3qtWfwVKBoYTPzSmqlAbYMgMUGuucANCkl6A6+NniyCZFL0zkb+RpQwFygYBZH4 J8P0fgNMspQJvvAlpuRbFYfcZlv1L6kcqVA6zQd9uyLiLEwZ/7D95yaRONqWk/lwzu+7x1ci88EPY /05CI1Oq1TC6uFGRA8tCmHCOQ/9rC0fP7OWKYzxXkQp/XvFaeaG+F28gbARZj0Es+W5kO6QG1lqFx 8rT2qMcBWEjXTQZXLEfc6LmnDSB76m/st4BmFuCQ/UbMqoHLabJijz2ui8uaooGplhqgO0eNWzSWu OEQ9oOrw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qMBHE-008P2v-14; Wed, 19 Jul 2023 17:47:52 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qMBHA-008OxH-28 for linux-arm-kernel@lists.infradead.org; Wed, 19 Jul 2023 17:47:50 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-c926075a50cso1740887276.1 for ; Wed, 19 Jul 2023 10:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689788863; x=1692380863; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=X9ynW0ej8481ncb+JbSIL8euZe2mr5ac0SQN0lLCVOPeUWz8gBSwzlmF4TS+pmsHcD ZHvVpvYsxfXcuGN+7iAna+BjVzlQUdXfzbNxlCS0j99kNjn4uiKYJhxoW/wLZRZhGth6 SbpazT26VqyLHMq6YpzbXDxi+oMw3HYZStI2pQTTUoLLoHXhI/Fn5BDIc8VD0DBHnb5o tff0t0H4OQwlS8KfnNmKRSLobGStUKs9j2oknxZFFagCPmr5LE+cTEI4/QNQaG1bCtL1 cIfSi8x3D97snfz5OpAaFyM4T1btS6Fnjj3oZZvzbB3+DHeSFy4bNTnAooGpizTyMK76 Nfxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689788863; x=1692380863; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ZrpSlfvy8CWa29HQgq0qreuIOXt/Wldn9lf0s+IQ19g=; b=dh2BjcLKWkPxaLnY0mnCPhN+EY09g4nOXARsbQyBaoMZcdBG8QlIFb2rA3v5rJplg8 jGL1OP6K5TF39TkuE8akYYnFKuHyLpdys5kb/bmu+RWJl75xFdzm+B51wN9c5WqUlHJe e3/OFXQIyfT9HpMGUs4sk8cRkVjqv/gWQShs/g5h6CTYUz+iMht1KsRLBDCK3c9uOAb7 G4q9HYKlNFsxxI4HyUnEPiDVBaTaJHI2zrGsh9OqKFGU3mD49yAme6t5dr9Vwx4oEvr4 RvnBMVXirz0ifZuyfxWdmsJUqH7AzNbpd1sPdG/tZv1Rq+mTognUW1amag0C61PKbh+z 4CUg== X-Gm-Message-State: ABy/qLZVvTXTOSwFJOqYRYr45V/C68yktcaIbzI6X3tcEUUwFKSuWxKm hmTXfcIpya23RqzLC3z9lYDKx/Mh5Eo= X-Google-Smtp-Source: APBJJlFlYUVvIIObTojpJ8UchQ4CwQtQGAVrCh++KpJ7LgJpdZRIA9xQwbGLQUVWTMKWnIrBZUJv99cvfhs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:f509:0:b0:ca3:3341:6315 with SMTP id a9-20020a25f509000000b00ca333416315mr42238ybe.0.1689788863726; Wed, 19 Jul 2023 10:47:43 -0700 (PDT) Date: Wed, 19 Jul 2023 10:47:41 -0700 In-Reply-To: Mime-Version: 1.0 References: <20230718234512.1690985-1-seanjc@google.com> <20230718234512.1690985-13-seanjc@google.com> Message-ID: Subject: Re: [RFC PATCH v11 12/29] KVM: Add KVM_CREATE_GUEST_MEMFD ioctl() for guest-specific backing memory From: Sean Christopherson To: Vishal Annapurve Cc: Paolo Bonzini , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Matthew Wilcox (Oracle)" , Andrew Morton , Paul Moore , James Morris , "Serge E. Hallyn" , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-security-module@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Peng , Fuad Tabba , Jarkko Sakkinen , Yu Zhang , Ackerley Tng , Maciej Szmigiero , Vlastimil Babka , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230719_104748_725773_99E67D0F X-CRM114-Status: GOOD ( 18.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBKdWwgMTksIDIwMjMsIFZpc2hhbCBBbm5hcHVydmUgd3JvdGU6Cj4gT24gVHVlLCBK dWwgMTgsIDIwMjMgYXQgNDo0OeKAr1BNIFNlYW4gQ2hyaXN0b3BoZXJzb24gPHNlYW5qY0Bnb29n bGUuY29tPiB3cm90ZToKPiA+IC4uLgo+ID4gK3N0YXRpYyBpbnQga3ZtX2dtZW1fZXJyb3JfcGFn ZShzdHJ1Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZywgc3RydWN0IHBhZ2UgKnBhZ2UpCj4gPiAr ewo+ID4gKyAgICAgICBzdHJ1Y3QgbGlzdF9oZWFkICpnbWVtX2xpc3QgPSAmbWFwcGluZy0+cHJp dmF0ZV9saXN0Owo+ID4gKyAgICAgICBzdHJ1Y3Qga3ZtX21lbW9yeV9zbG90ICpzbG90Owo+ID4g KyAgICAgICBzdHJ1Y3Qga3ZtX2dtZW0gKmdtZW07Cj4gPiArICAgICAgIHVuc2lnbmVkIGxvbmcg aW5kZXg7Cj4gPiArICAgICAgIHBnb2ZmX3Qgc3RhcnQsIGVuZDsKPiA+ICsgICAgICAgZ2ZuX3Qg Z2ZuOwo+ID4gKwo+ID4gKyAgICAgICBmaWxlbWFwX2ludmFsaWRhdGVfbG9ja19zaGFyZWQobWFw cGluZyk7Cj4gPiArCj4gPiArICAgICAgIHN0YXJ0ID0gcGFnZS0+aW5kZXg7Cj4gPiArICAgICAg IGVuZCA9IHN0YXJ0ICsgdGhwX25yX3BhZ2VzKHBhZ2UpOwo+ID4gKwo+ID4gKyAgICAgICBsaXN0 X2Zvcl9lYWNoX2VudHJ5KGdtZW0sIGdtZW1fbGlzdCwgZW50cnkpIHsKPiA+ICsgICAgICAgICAg ICAgICB4YV9mb3JfZWFjaF9yYW5nZSgmZ21lbS0+YmluZGluZ3MsIGluZGV4LCBzbG90LCBzdGFy dCwgZW5kIC0gMSkgewo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgZm9yIChnZm4gPSBzdGFy dDsgZ2ZuIDwgZW5kOyBnZm4rKykgewo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBpZiAoV0FSTl9PTl9PTkNFKGdmbiA8IHNsb3QtPmJhc2VfZ2ZuIHx8Cj4gPiArICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBnZm4gPj0gc2xvdC0+YmFzZV9n Zm4gKyBzbG90LT5ucGFnZXMpKQo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIGNvbnRpbnVlOwo+ID4gKwo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAvKgo+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKiBGSVhNRTogVGVsbCB1 c2Vyc3BhY2UgdGhhdCB0aGUgKnByaXZhdGUqCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAqIG1lbW9yeSBlbmNvdW50ZXJlZCBhbiBlcnJvci4KPiA+ICsgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICovCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHNlbmRfc2lnX21jZWVycihCVVNfTUNFRVJSX0FSLAo+ID4gKyAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgKHZvaWQgX191c2VyICopZ2ZuX3RvX2h2YV9tZW1z bG90KHNsb3QsIGdmbiksCj4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBQQUdFX1NISUZULCBjdXJyZW50KTsKPiAKPiBEb2VzIGl0IG1ha2Ugc2Vuc2Ug dG8gcmVwbGljYXRlIHdoYXQgaGFwcGVucyB3aXRoIE1DRSBoYW5kbGluZyBvbgo+IHRtcGZzIGJh Y2tlZCBndWVzdCBtZW1vcnk6Cj4gMSkgVW5tYXAgZ3BhIGZyb20gZ3Vlc3QKPiAyKSBPbiB0aGUg bmV4dCBndWVzdCBFUFQgZmF1bHQsIGV4aXQgdG8gdXNlcnNwYWNlIHRvIGhhbmRsZS9sb2cgdGhl Cj4gbWNlIGVycm9yIGZvciB0aGUgZ3BhLgoKSG1tLCB5ZXMsIHRoYXQgd291bGQgYmUgbXVjaCBi ZXR0ZXIuICBBaCwgYW5kIGt2bV9nbWVtX2dldF9wZm4oKSBuZWVkcyB0byBjaGVjawpmb2xpb190 ZXN0X2h3cG9pc29uKCkgYW5kIHBvdGVudGlhbGx5IFBhZ2VIV1BvaXNvbigpLiAgRS5nLiBpZiB0 aGUgZm9saW8gaXMgaHVnZSwKS1ZNIG5lZWRzIHRvIHJlc3RyaWN0IHRoZSBtYXBwaW5nIHRvIG9y ZGVyLTAgKHRhcmdldCBwYWdlIGlzbid0IHBvaXNvbmVkKSwgb3IKcmV0dXJuIEtWTV9QRk5fRVJS X0hXUE9JU09OICh0YWdldCBwYWdlIElTIHBvaXNvbmVkKS4KCkFsdGVybmF0aXZlbHksIEtWTSBj b3VsZCBwdW5jaCBhIGhvbGUgaW4ga3ZtX2dtZW1fZXJyb3JfcGFnZSgpLCBidXQgSSBkb24ndCB0 aGluawp3ZSB3YW50IHRvIGRvIHRoYXQgYmVjYXVzZSB0aGF0IHdvdWxkIHByZXZlbnQgZm9yd2Fy ZGluZyB0aGUgI01DIHRvIHRoZSBndWVzdC4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFy bS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9t YWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK