From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55F7ABE4E for ; Wed, 31 May 2023 08:54:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AA25C433EF; Wed, 31 May 2023 08:54:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685523262; bh=Sqn3M6dSHNJEqYtJZIRvu64F2FVcoaakUNxFMdpXTJM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=HeZE/tncj++dXrRuRsxqnZCmtcksVbV7Hepd7mOs3iyEhSFtfJw3dIWRumbiqkTOe mezWymVS6m21keQT4VzaKFRqXXKXZEMEmGPXs6OwGkQccCm6Wkxm1OxXQhOmIgKXwm mTlIY9WFxLp/3fa313tORgmYQcBqp/ImPNKA4lVFwt4RrXU9n/KLfbfnpZ9XSM3bD6 KFzu4kW7vodftmR7gtt/LQO4Y84V4w9K6yRPaXlHN/rEKLfjxlemTqGeCO0Dv9IxL7 Se0zOH4kYHXT8agpYpc6AfMu82+OnHm5df7bXZiOnHtY5Aqd/v0q7Vd/zjx0BBVg4b 0x6rCxIggPvsg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q4Hb1-001Z7H-UP; Wed, 31 May 2023 09:54:20 +0100 Date: Wed, 31 May 2023 09:54:19 +0100 Message-ID: <86y1l4c24k.wl-maz@kernel.org> From: Marc Zyngier To: Raghavendra Rao Ananta Cc: Oliver Upton , James Morse , Suzuki K Poulose , Ricardo Koller , Paolo Bonzini , Jing Zhang , Colton Lewis , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH v4 6/6] KVM: arm64: Use TLBI range-based intructions for unmap In-Reply-To: References: <20230519005231.3027912-1-rananta@google.com> <20230519005231.3027912-7-rananta@google.com> <87ttvvjk5q.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: rananta@google.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, ricarkol@google.com, pbonzini@redhat.com, jingzhangos@google.com, coltonlewis@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Tue, 30 May 2023 22:35:57 +0100, Raghavendra Rao Ananta wrote: >=20 > On Mon, May 29, 2023 at 7:18=E2=80=AFAM Marc Zyngier wro= te: > > > > On Fri, 19 May 2023 01:52:31 +0100, > > Raghavendra Rao Ananta wrote: > > > > > > The current implementation of the stage-2 unmap walker traverses > > > the given range and, as a part of break-before-make, performs > > > TLB invalidations with a DSB for every PTE. A multitude of this > > > combination could cause a performance bottleneck. > > > > > > Hence, if the system supports FEAT_TLBIRANGE, defer the TLB > > > invalidations until the entire walk is finished, and then > > > use range-based instructions to invalidate the TLBs in one go. > > > Condition this upon S2FWB in order to avoid walking the page-table > > > again to perform the CMOs after issuing the TLBI. > > > > But that's the real bottleneck. TLBIs are cheap compared to CMOs, even > > on remarkably bad implementations. What is your plan to fix this? > > > Correct me if I'm wrong, but my understanding was that a multiple > issuance of TLBI + DSB was the bottleneck, and this patch tries to > avoid this by issuing only one TLBI + DSB at the end. At least on some of the machines I have access to, CMOs are fare more expensive than TLBIs, and they are the ones causing slowdowns. Your system shows a different behaviour, and that's fine, but you can't draw a general conclusion from it. > > > > > > Rename stage2_put_pte() to stage2_unmap_put_pte() as the function > > > now serves the stage-2 unmap walker specifically, rather than > > > acting generic. > > > > > > Signed-off-by: Raghavendra Rao Ananta > > > --- > > > arch/arm64/kvm/hyp/pgtable.c | 35 ++++++++++++++++++++++++++++++----- > > > 1 file changed, 30 insertions(+), 5 deletions(-) > > > > > > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtabl= e.c > > > index b8f0dbd12f773..5832ee3418fb0 100644 > > > --- a/arch/arm64/kvm/hyp/pgtable.c > > > +++ b/arch/arm64/kvm/hyp/pgtable.c > > > @@ -771,16 +771,34 @@ static void stage2_make_pte(const struct kvm_pg= table_visit_ctx *ctx, kvm_pte_t n > > > smp_store_release(ctx->ptep, new); > > > } > > > > > > -static void stage2_put_pte(const struct kvm_pgtable_visit_ctx *ctx, = struct kvm_s2_mmu *mmu, > > > - struct kvm_pgtable_mm_ops *mm_ops) > > > +static bool stage2_unmap_defer_tlb_flush(struct kvm_pgtable *pgt) > > > { > > > + /* > > > + * If FEAT_TLBIRANGE is implemented, defer the individial PTE > > > + * TLB invalidations until the entire walk is finished, and > > > + * then use the range-based TLBI instructions to do the > > > + * invalidations. Condition this upon S2FWB in order to avoid > > > + * a page-table walk again to perform the CMOs after TLBI. > > > + */ > > > + return system_supports_tlb_range() && stage2_has_fwb(pgt); > > > +} > > > + > > > +static void stage2_unmap_put_pte(const struct kvm_pgtable_visit_ctx = *ctx, > > > + struct kvm_s2_mmu *mmu, > > > + struct kvm_pgtable_mm_ops *mm_ops) > > > +{ > > > + struct kvm_pgtable *pgt =3D ctx->arg; > > > + > > > /* > > > * Clear the existing PTE, and perform break-before-make with > > > * TLB maintenance if it was valid. > > > */ > > > if (kvm_pte_valid(ctx->old)) { > > > kvm_clear_pte(ctx->ptep); > > > - kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, ctx->addr, = ctx->level); > > > + > > > + if (!stage2_unmap_defer_tlb_flush(pgt)) > > > + kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, > > > + ctx->addr, ctx->level); > > > > This really doesn't match the comment anymore. > > > Right, I can re-write this in the next spin. >=20 > > Overall, I'm very concerned that we lose the consistency property that > > the current code has: once called, the TLBs and the page tables are > > synchronised. > > > > Yes, this patch looks correct. But it is also really fragile. > > > Yeah, we were a little skeptical about this too. Till v2, we had a > different implementation in which we had an independent fast unmap > path that disconnects the PTE hierarchy if the unmap range was exactly > KVM_PGTABLE_MIN_BLOCK_LEVEL [1]. But this had some problems, and we > pivoted to the current implementation. Can we at least have some sort of runtime assertions that at the point we release the write lock, the TLBs have been invalidated? Even if that's tied to some debug config. >=20 > > > } > > > > > > mm_ops->put_page(ctx->ptep); > > > @@ -1015,7 +1033,7 @@ static int stage2_unmap_walker(const struct kvm= _pgtable_visit_ctx *ctx, > > > * block entry and rely on the remaining portions being faulted > > > * back lazily. > > > */ > > > - stage2_put_pte(ctx, mmu, mm_ops); > > > + stage2_unmap_put_pte(ctx, mmu, mm_ops); > > > > > > if (need_flush && mm_ops->dcache_clean_inval_poc) > > > mm_ops->dcache_clean_inval_poc(kvm_pte_follow(ctx->old,= mm_ops), > > > @@ -1029,13 +1047,20 @@ static int stage2_unmap_walker(const struct k= vm_pgtable_visit_ctx *ctx, > > > > > > int kvm_pgtable_stage2_unmap(struct kvm_pgtable *pgt, u64 addr, u64 = size) > > > { > > > + int ret; > > > struct kvm_pgtable_walker walker =3D { > > > .cb =3D stage2_unmap_walker, > > > .arg =3D pgt, > > > .flags =3D KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TA= BLE_POST, > > > }; > > > > > > - return kvm_pgtable_walk(pgt, addr, size, &walker); > > > + ret =3D kvm_pgtable_walk(pgt, addr, size, &walker); > > > + if (stage2_unmap_defer_tlb_flush(pgt)) > > > + /* Perform the deferred TLB invalidations */ > > > + kvm_call_hyp(__kvm_tlb_flush_vmid_range, pgt->mmu, > > > + addr, addr + size); > > > > This "kvm_call_hyp(__kvm_tlb_flush_vmid_range,...)" could do with a > > wrapper from the point where you introduce it. > > > Sorry, I didn't get this comment. Do you mind elaborating on it? All I'm saying is that you should have a wrapper like: void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, phys_addr_t base, size_t size) { kvm_call_hyp(__kvm_tlb_flush_vmid_range, mmu, base, base + size); } and use it throughout the code. Thanks, M. --=20 Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A6CDC77B73 for ; Wed, 31 May 2023 08:54:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yige4m2P8fRAGjiuapHuvMqHcLVEqLwH08jra91FGfs=; b=xA4E79ppcGItg5 KXb0xf2aeptHzaMU+ysnCpE4TzhOLdkRVTo8kFqzD86AA2YCA+7bxwQ8S94IEpxDftI27igzFmwKk U4hCfJBh+nwNcygtwkfyRNMNbi34+c7/1bmNLJUmbUwVa/hPUmxEuuew/bOWXAlJwosxpurYMRV6Y DuMCgc1uD8SimSJx1SCaI8ls/0lAkl+j01PcOpz5R1qzOaWnF8eml16aMa3an36exBNQG1qwtcXLW msi4+DjgFxnopwRH+t6id6m89Tsa15YtxnqSAeyWtblbZX9i6JS/SCTea5D2PNHKSgYEwTBSuG+2k zVu9lNtK/fOnGGEZ6ypw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q4Hb8-00GlJd-1n; Wed, 31 May 2023 08:54:26 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q4Hb5-00GlHU-1Q for linux-arm-kernel@lists.infradead.org; Wed, 31 May 2023 08:54:25 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B777B63854; Wed, 31 May 2023 08:54:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AA25C433EF; Wed, 31 May 2023 08:54:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685523262; bh=Sqn3M6dSHNJEqYtJZIRvu64F2FVcoaakUNxFMdpXTJM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=HeZE/tncj++dXrRuRsxqnZCmtcksVbV7Hepd7mOs3iyEhSFtfJw3dIWRumbiqkTOe mezWymVS6m21keQT4VzaKFRqXXKXZEMEmGPXs6OwGkQccCm6Wkxm1OxXQhOmIgKXwm mTlIY9WFxLp/3fa313tORgmYQcBqp/ImPNKA4lVFwt4RrXU9n/KLfbfnpZ9XSM3bD6 KFzu4kW7vodftmR7gtt/LQO4Y84V4w9K6yRPaXlHN/rEKLfjxlemTqGeCO0Dv9IxL7 Se0zOH4kYHXT8agpYpc6AfMu82+OnHm5df7bXZiOnHtY5Aqd/v0q7Vd/zjx0BBVg4b 0x6rCxIggPvsg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1q4Hb1-001Z7H-UP; Wed, 31 May 2023 09:54:20 +0100 Date: Wed, 31 May 2023 09:54:19 +0100 Message-ID: <86y1l4c24k.wl-maz@kernel.org> From: Marc Zyngier To: Raghavendra Rao Ananta Cc: Oliver Upton , James Morse , Suzuki K Poulose , Ricardo Koller , Paolo Bonzini , Jing Zhang , Colton Lewis , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH v4 6/6] KVM: arm64: Use TLBI range-based intructions for unmap In-Reply-To: References: <20230519005231.3027912-1-rananta@google.com> <20230519005231.3027912-7-rananta@google.com> <87ttvvjk5q.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: rananta@google.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, ricarkol@google.com, pbonzini@redhat.com, jingzhangos@google.com, coltonlewis@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230531_015423_556249_9968DEEF X-CRM114-Status: GOOD ( 51.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVHVlLCAzMCBNYXkgMjAyMyAyMjozNTo1NyArMDEwMCwKUmFnaGF2ZW5kcmEgUmFvIEFuYW50 YSA8cmFuYW50YUBnb29nbGUuY29tPiB3cm90ZToKPiAKPiBPbiBNb24sIE1heSAyOSwgMjAyMyBh dCA3OjE44oCvQU0gTWFyYyBaeW5naWVyIDxtYXpAa2VybmVsLm9yZz4gd3JvdGU6Cj4gPgo+ID4g T24gRnJpLCAxOSBNYXkgMjAyMyAwMTo1MjozMSArMDEwMCwKPiA+IFJhZ2hhdmVuZHJhIFJhbyBB bmFudGEgPHJhbmFudGFAZ29vZ2xlLmNvbT4gd3JvdGU6Cj4gPiA+Cj4gPiA+IFRoZSBjdXJyZW50 IGltcGxlbWVudGF0aW9uIG9mIHRoZSBzdGFnZS0yIHVubWFwIHdhbGtlciB0cmF2ZXJzZXMKPiA+ ID4gdGhlIGdpdmVuIHJhbmdlIGFuZCwgYXMgYSBwYXJ0IG9mIGJyZWFrLWJlZm9yZS1tYWtlLCBw ZXJmb3Jtcwo+ID4gPiBUTEIgaW52YWxpZGF0aW9ucyB3aXRoIGEgRFNCIGZvciBldmVyeSBQVEUu IEEgbXVsdGl0dWRlIG9mIHRoaXMKPiA+ID4gY29tYmluYXRpb24gY291bGQgY2F1c2UgYSBwZXJm b3JtYW5jZSBib3R0bGVuZWNrLgo+ID4gPgo+ID4gPiBIZW5jZSwgaWYgdGhlIHN5c3RlbSBzdXBw b3J0cyBGRUFUX1RMQklSQU5HRSwgZGVmZXIgdGhlIFRMQgo+ID4gPiBpbnZhbGlkYXRpb25zIHVu dGlsIHRoZSBlbnRpcmUgd2FsayBpcyBmaW5pc2hlZCwgYW5kIHRoZW4KPiA+ID4gdXNlIHJhbmdl LWJhc2VkIGluc3RydWN0aW9ucyB0byBpbnZhbGlkYXRlIHRoZSBUTEJzIGluIG9uZSBnby4KPiA+ ID4gQ29uZGl0aW9uIHRoaXMgdXBvbiBTMkZXQiBpbiBvcmRlciB0byBhdm9pZCB3YWxraW5nIHRo ZSBwYWdlLXRhYmxlCj4gPiA+IGFnYWluIHRvIHBlcmZvcm0gdGhlIENNT3MgYWZ0ZXIgaXNzdWlu ZyB0aGUgVExCSS4KPiA+Cj4gPiBCdXQgdGhhdCdzIHRoZSByZWFsIGJvdHRsZW5lY2suIFRMQklz IGFyZSBjaGVhcCBjb21wYXJlZCB0byBDTU9zLCBldmVuCj4gPiBvbiByZW1hcmthYmx5IGJhZCBp bXBsZW1lbnRhdGlvbnMuIFdoYXQgaXMgeW91ciBwbGFuIHRvIGZpeCB0aGlzPwo+ID4KPiBDb3Jy ZWN0IG1lIGlmIEknbSB3cm9uZywgYnV0IG15IHVuZGVyc3RhbmRpbmcgd2FzIHRoYXQgYSBtdWx0 aXBsZQo+IGlzc3VhbmNlIG9mIFRMQkkgKyBEU0Igd2FzIHRoZSBib3R0bGVuZWNrLCBhbmQgdGhp cyBwYXRjaCB0cmllcyB0bwo+IGF2b2lkIHRoaXMgYnkgaXNzdWluZyBvbmx5IG9uZSBUTEJJICsg RFNCIGF0IHRoZSBlbmQuCgpBdCBsZWFzdCBvbiBzb21lIG9mIHRoZSBtYWNoaW5lcyBJIGhhdmUg YWNjZXNzIHRvLCBDTU9zIGFyZSBmYXJlIG1vcmUKZXhwZW5zaXZlIHRoYW4gVExCSXMsIGFuZCB0 aGV5IGFyZSB0aGUgb25lcyBjYXVzaW5nIHNsb3dkb3ducy4gWW91cgpzeXN0ZW0gc2hvd3MgYSBk aWZmZXJlbnQgYmVoYXZpb3VyLCBhbmQgdGhhdCdzIGZpbmUsIGJ1dCB5b3UgY2FuJ3QKZHJhdyBh IGdlbmVyYWwgY29uY2x1c2lvbiBmcm9tIGl0LgoKPiA+ID4KPiA+ID4gUmVuYW1lIHN0YWdlMl9w dXRfcHRlKCkgdG8gc3RhZ2UyX3VubWFwX3B1dF9wdGUoKSBhcyB0aGUgZnVuY3Rpb24KPiA+ID4g bm93IHNlcnZlcyB0aGUgc3RhZ2UtMiB1bm1hcCB3YWxrZXIgc3BlY2lmaWNhbGx5LCByYXRoZXIg dGhhbgo+ID4gPiBhY3RpbmcgZ2VuZXJpYy4KPiA+ID4KPiA+ID4gU2lnbmVkLW9mZi1ieTogUmFn aGF2ZW5kcmEgUmFvIEFuYW50YSA8cmFuYW50YUBnb29nbGUuY29tPgo+ID4gPiAtLS0KPiA+ID4g IGFyY2gvYXJtNjQva3ZtL2h5cC9wZ3RhYmxlLmMgfCAzNSArKysrKysrKysrKysrKysrKysrKysr KysrKysrKystLS0tLQo+ID4gPiAgMSBmaWxlIGNoYW5nZWQsIDMwIGluc2VydGlvbnMoKyksIDUg ZGVsZXRpb25zKC0pCj4gPiA+Cj4gPiA+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2t2bS9oeXAv cGd0YWJsZS5jIGIvYXJjaC9hcm02NC9rdm0vaHlwL3BndGFibGUuYwo+ID4gPiBpbmRleCBiOGYw ZGJkMTJmNzczLi41ODMyZWUzNDE4ZmIwIDEwMDY0NAo+ID4gPiAtLS0gYS9hcmNoL2FybTY0L2t2 bS9oeXAvcGd0YWJsZS5jCj4gPiA+ICsrKyBiL2FyY2gvYXJtNjQva3ZtL2h5cC9wZ3RhYmxlLmMK PiA+ID4gQEAgLTc3MSwxNiArNzcxLDM0IEBAIHN0YXRpYyB2b2lkIHN0YWdlMl9tYWtlX3B0ZShj b25zdCBzdHJ1Y3Qga3ZtX3BndGFibGVfdmlzaXRfY3R4ICpjdHgsIGt2bV9wdGVfdCBuCj4gPiA+ ICAgICAgIHNtcF9zdG9yZV9yZWxlYXNlKGN0eC0+cHRlcCwgbmV3KTsKPiA+ID4gIH0KPiA+ID4K PiA+ID4gLXN0YXRpYyB2b2lkIHN0YWdlMl9wdXRfcHRlKGNvbnN0IHN0cnVjdCBrdm1fcGd0YWJs ZV92aXNpdF9jdHggKmN0eCwgc3RydWN0IGt2bV9zMl9tbXUgKm1tdSwKPiA+ID4gLSAgICAgICAg ICAgICAgICAgICAgICAgIHN0cnVjdCBrdm1fcGd0YWJsZV9tbV9vcHMgKm1tX29wcykKPiA+ID4g K3N0YXRpYyBib29sIHN0YWdlMl91bm1hcF9kZWZlcl90bGJfZmx1c2goc3RydWN0IGt2bV9wZ3Rh YmxlICpwZ3QpCj4gPiA+ICB7Cj4gPiA+ICsgICAgIC8qCj4gPiA+ICsgICAgICAqIElmIEZFQVRf VExCSVJBTkdFIGlzIGltcGxlbWVudGVkLCBkZWZlciB0aGUgaW5kaXZpZGlhbCBQVEUKPiA+ID4g KyAgICAgICogVExCIGludmFsaWRhdGlvbnMgdW50aWwgdGhlIGVudGlyZSB3YWxrIGlzIGZpbmlz aGVkLCBhbmQKPiA+ID4gKyAgICAgICogdGhlbiB1c2UgdGhlIHJhbmdlLWJhc2VkIFRMQkkgaW5z dHJ1Y3Rpb25zIHRvIGRvIHRoZQo+ID4gPiArICAgICAgKiBpbnZhbGlkYXRpb25zLiBDb25kaXRp b24gdGhpcyB1cG9uIFMyRldCIGluIG9yZGVyIHRvIGF2b2lkCj4gPiA+ICsgICAgICAqIGEgcGFn ZS10YWJsZSB3YWxrIGFnYWluIHRvIHBlcmZvcm0gdGhlIENNT3MgYWZ0ZXIgVExCSS4KPiA+ID4g KyAgICAgICovCj4gPiA+ICsgICAgIHJldHVybiBzeXN0ZW1fc3VwcG9ydHNfdGxiX3JhbmdlKCkg JiYgc3RhZ2UyX2hhc19md2IocGd0KTsKPiA+ID4gK30KPiA+ID4gKwo+ID4gPiArc3RhdGljIHZv aWQgc3RhZ2UyX3VubWFwX3B1dF9wdGUoY29uc3Qgc3RydWN0IGt2bV9wZ3RhYmxlX3Zpc2l0X2N0 eCAqY3R4LAo+ID4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3Qga3ZtX3My X21tdSAqbW11LAo+ID4gPiArICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3Qga3Zt X3BndGFibGVfbW1fb3BzICptbV9vcHMpCj4gPiA+ICt7Cj4gPiA+ICsgICAgIHN0cnVjdCBrdm1f cGd0YWJsZSAqcGd0ID0gY3R4LT5hcmc7Cj4gPiA+ICsKPiA+ID4gICAgICAgLyoKPiA+ID4gICAg ICAgICogQ2xlYXIgdGhlIGV4aXN0aW5nIFBURSwgYW5kIHBlcmZvcm0gYnJlYWstYmVmb3JlLW1h a2Ugd2l0aAo+ID4gPiAgICAgICAgKiBUTEIgbWFpbnRlbmFuY2UgaWYgaXQgd2FzIHZhbGlkLgo+ ID4gPiAgICAgICAgKi8KPiA+ID4gICAgICAgaWYgKGt2bV9wdGVfdmFsaWQoY3R4LT5vbGQpKSB7 Cj4gPiA+ICAgICAgICAgICAgICAga3ZtX2NsZWFyX3B0ZShjdHgtPnB0ZXApOwo+ID4gPiAtICAg ICAgICAgICAgIGt2bV9jYWxsX2h5cChfX2t2bV90bGJfZmx1c2hfdm1pZF9pcGEsIG1tdSwgY3R4 LT5hZGRyLCBjdHgtPmxldmVsKTsKPiA+ID4gKwo+ID4gPiArICAgICAgICAgICAgIGlmICghc3Rh Z2UyX3VubWFwX2RlZmVyX3RsYl9mbHVzaChwZ3QpKQo+ID4gPiArICAgICAgICAgICAgICAgICAg ICAga3ZtX2NhbGxfaHlwKF9fa3ZtX3RsYl9mbHVzaF92bWlkX2lwYSwgbW11LAo+ID4gPiArICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGN0eC0+YWRkciwgY3R4LT5sZXZlbCk7 Cj4gPgo+ID4gVGhpcyByZWFsbHkgZG9lc24ndCBtYXRjaCB0aGUgY29tbWVudCBhbnltb3JlLgo+ ID4KPiBSaWdodCwgSSBjYW4gcmUtd3JpdGUgdGhpcyBpbiB0aGUgbmV4dCBzcGluLgo+IAo+ID4g T3ZlcmFsbCwgSSdtIHZlcnkgY29uY2VybmVkIHRoYXQgd2UgbG9zZSB0aGUgY29uc2lzdGVuY3kg cHJvcGVydHkgdGhhdAo+ID4gdGhlIGN1cnJlbnQgY29kZSBoYXM6IG9uY2UgY2FsbGVkLCB0aGUg VExCcyBhbmQgdGhlIHBhZ2UgdGFibGVzIGFyZQo+ID4gc3luY2hyb25pc2VkLgo+ID4KPiA+IFll cywgdGhpcyBwYXRjaCBsb29rcyBjb3JyZWN0LiBCdXQgaXQgaXMgYWxzbyByZWFsbHkgZnJhZ2ls ZS4KPiA+Cj4gWWVhaCwgd2Ugd2VyZSBhIGxpdHRsZSBza2VwdGljYWwgYWJvdXQgdGhpcyB0b28u IFRpbGwgdjIsIHdlIGhhZCBhCj4gZGlmZmVyZW50IGltcGxlbWVudGF0aW9uIGluIHdoaWNoIHdl IGhhZCBhbiBpbmRlcGVuZGVudCBmYXN0IHVubWFwCj4gcGF0aCB0aGF0IGRpc2Nvbm5lY3RzIHRo ZSBQVEUgaGllcmFyY2h5IGlmIHRoZSB1bm1hcCByYW5nZSB3YXMgZXhhY3RseQo+IEtWTV9QR1RB QkxFX01JTl9CTE9DS19MRVZFTCBbMV0uICBCdXQgdGhpcyBoYWQgc29tZSBwcm9ibGVtcywgYW5k IHdlCj4gcGl2b3RlZCB0byB0aGUgY3VycmVudCBpbXBsZW1lbnRhdGlvbi4KCkNhbiB3ZSBhdCBs ZWFzdCBoYXZlIHNvbWUgc29ydCBvZiBydW50aW1lIGFzc2VydGlvbnMgdGhhdCBhdCB0aGUgcG9p bnQKd2UgcmVsZWFzZSB0aGUgd3JpdGUgbG9jaywgdGhlIFRMQnMgaGF2ZSBiZWVuIGludmFsaWRh dGVkPyBFdmVuIGlmCnRoYXQncyB0aWVkIHRvIHNvbWUgZGVidWcgY29uZmlnLgoKPiAKPiA+ID4g ICAgICAgfQo+ID4gPgo+ID4gPiAgICAgICBtbV9vcHMtPnB1dF9wYWdlKGN0eC0+cHRlcCk7Cj4g PiA+IEBAIC0xMDE1LDcgKzEwMzMsNyBAQCBzdGF0aWMgaW50IHN0YWdlMl91bm1hcF93YWxrZXIo Y29uc3Qgc3RydWN0IGt2bV9wZ3RhYmxlX3Zpc2l0X2N0eCAqY3R4LAo+ID4gPiAgICAgICAgKiBi bG9jayBlbnRyeSBhbmQgcmVseSBvbiB0aGUgcmVtYWluaW5nIHBvcnRpb25zIGJlaW5nIGZhdWx0 ZWQKPiA+ID4gICAgICAgICogYmFjayBsYXppbHkuCj4gPiA+ICAgICAgICAqLwo+ID4gPiAtICAg ICBzdGFnZTJfcHV0X3B0ZShjdHgsIG1tdSwgbW1fb3BzKTsKPiA+ID4gKyAgICAgc3RhZ2UyX3Vu bWFwX3B1dF9wdGUoY3R4LCBtbXUsIG1tX29wcyk7Cj4gPiA+Cj4gPiA+ICAgICAgIGlmIChuZWVk X2ZsdXNoICYmIG1tX29wcy0+ZGNhY2hlX2NsZWFuX2ludmFsX3BvYykKPiA+ID4gICAgICAgICAg ICAgICBtbV9vcHMtPmRjYWNoZV9jbGVhbl9pbnZhbF9wb2Moa3ZtX3B0ZV9mb2xsb3coY3R4LT5v bGQsIG1tX29wcyksCj4gPiA+IEBAIC0xMDI5LDEzICsxMDQ3LDIwIEBAIHN0YXRpYyBpbnQgc3Rh Z2UyX3VubWFwX3dhbGtlcihjb25zdCBzdHJ1Y3Qga3ZtX3BndGFibGVfdmlzaXRfY3R4ICpjdHgs Cj4gPiA+Cj4gPiA+ICBpbnQga3ZtX3BndGFibGVfc3RhZ2UyX3VubWFwKHN0cnVjdCBrdm1fcGd0 YWJsZSAqcGd0LCB1NjQgYWRkciwgdTY0IHNpemUpCj4gPiA+ICB7Cj4gPiA+ICsgICAgIGludCBy ZXQ7Cj4gPiA+ICAgICAgIHN0cnVjdCBrdm1fcGd0YWJsZV93YWxrZXIgd2Fsa2VyID0gewo+ID4g PiAgICAgICAgICAgICAgIC5jYiAgICAgPSBzdGFnZTJfdW5tYXBfd2Fsa2VyLAo+ID4gPiAgICAg ICAgICAgICAgIC5hcmcgICAgPSBwZ3QsCj4gPiA+ICAgICAgICAgICAgICAgLmZsYWdzICA9IEtW TV9QR1RBQkxFX1dBTEtfTEVBRiB8IEtWTV9QR1RBQkxFX1dBTEtfVEFCTEVfUE9TVCwKPiA+ID4g ICAgICAgfTsKPiA+ID4KPiA+ID4gLSAgICAgcmV0dXJuIGt2bV9wZ3RhYmxlX3dhbGsocGd0LCBh ZGRyLCBzaXplLCAmd2Fsa2VyKTsKPiA+ID4gKyAgICAgcmV0ID0ga3ZtX3BndGFibGVfd2Fsayhw Z3QsIGFkZHIsIHNpemUsICZ3YWxrZXIpOwo+ID4gPiArICAgICBpZiAoc3RhZ2UyX3VubWFwX2Rl ZmVyX3RsYl9mbHVzaChwZ3QpKQo+ID4gPiArICAgICAgICAgICAgIC8qIFBlcmZvcm0gdGhlIGRl ZmVycmVkIFRMQiBpbnZhbGlkYXRpb25zICovCj4gPiA+ICsgICAgICAgICAgICAga3ZtX2NhbGxf aHlwKF9fa3ZtX3RsYl9mbHVzaF92bWlkX3JhbmdlLCBwZ3QtPm1tdSwKPiA+ID4gKyAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgYWRkciwgYWRkciArIHNpemUpOwo+ID4KPiA+IFRoaXMgImt2 bV9jYWxsX2h5cChfX2t2bV90bGJfZmx1c2hfdm1pZF9yYW5nZSwuLi4pIiBjb3VsZCBkbyB3aXRo IGEKPiA+IHdyYXBwZXIgZnJvbSB0aGUgcG9pbnQgd2hlcmUgeW91IGludHJvZHVjZSBpdC4KPiA+ Cj4gU29ycnksIEkgZGlkbid0IGdldCB0aGlzIGNvbW1lbnQuIERvIHlvdSBtaW5kIGVsYWJvcmF0 aW5nIG9uIGl0PwoKQWxsIEknbSBzYXlpbmcgaXMgdGhhdCB5b3Ugc2hvdWxkIGhhdmUgYSB3cmFw cGVyIGxpa2U6Cgp2b2lkIGt2bV90bGJfZmx1c2hfdm1pZF9yYW5nZShzdHJ1Y3Qga3ZtX3MyX21t dSAqbW11LAoJCQkgICAgICBwaHlzX2FkZHJfdCBiYXNlLCBzaXplX3Qgc2l6ZSkKewoJa3ZtX2Nh bGxfaHlwKF9fa3ZtX3RsYl9mbHVzaF92bWlkX3JhbmdlLAoJCSAgICAgbW11LCBiYXNlLCBiYXNl ICsgc2l6ZSk7Cn0KCmFuZCB1c2UgaXQgdGhyb3VnaG91dCB0aGUgY29kZS4KClRoYW5rcywKCglN LgoKLS0gCldpdGhvdXQgZGV2aWF0aW9uIGZyb20gdGhlIG5vcm0sIHByb2dyZXNzIGlzIG5vdCBw b3NzaWJsZS4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f CmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5m cmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xp bnV4LWFybS1rZXJuZWwK