From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 165D413FD6F; Fri, 26 Apr 2024 10:26:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714127192; cv=none; b=Y1HYO+iQAjeDCKtfu+sExm0ebYlQLKA4MvYnis8a/Y7uUlGiUXhwG8hFksr2REXk3eaS5xOR1rWljeOexgilTeSTY7T3d18qXinvpHepAS22Z0XeWPPBzNcW9wIZbgAOp3jhQ9JnnUxahKvhVqOgIgF9dDHbW/7fo7L6gbF6oQk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714127192; c=relaxed/simple; bh=Mp1x52ASowv5NbAjsAc/PHMwV+sGt7uEmgf5kXMNu0o=; h=From:To:Cc:Subject:In-Reply-To:References:Date:Message-ID: MIME-Version:Content-Type; b=mml0/N31l43DSKSB4HpemxAM+hS/af8vVvrJLLjMTLQmItLoJZKA0AO7D54lBfomRCqFKfbzNIjZ3b3uJmbf5BWGC56uOj+BGVceebSJtFN5AJTOyWs5xQzuhNlNKQfm9TrHMa1oH2Y+iYsOBYqJigZzXIFdoTXw5HKc+UBaGOs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GIQp+fw1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GIQp+fw1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C7CFC113CD; Fri, 26 Apr 2024 10:26:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714127191; bh=Mp1x52ASowv5NbAjsAc/PHMwV+sGt7uEmgf5kXMNu0o=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=GIQp+fw1b+bCzFwKnZvtyr3gBd7s0dqMzzIfhjgPNDFBppuN0ni2nWziKZb0lXZoE 4Pl3XK0JIHC/wq2VFGZL2vRnKrYiXcfaOKY8FdZ9BMFMryc8MfDVgp5l4thjfYv4Vp hviMf8HKOnJU5xITrRXjCDZQGlLz8BXMdfpeFge4IDXmLHNHedtn+CH6E9pLLHpPLd +InkwL49wttyCk3bNG/X05HdLjagge4ED/TgNE/fP5QNst/bHmKVULAyVzNM3MIIyK 1TfXSaHwLDSkp7SwOQD4+wM9gxS4ftKLBhIdNQjBwgoxWALPOlT855UgnJTyPe1/m4 ultrnqWHzAPkA== From: Puranjay Mohan To: Andrii Nakryiko Cc: Catalin Marinas , Will Deacon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Zi Shen Lim , Xu Kuohai , Florent Revest , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH bpf-next v2 2/2] bpf, arm64: inline bpf_get_smp_processor_id() helper In-Reply-To: References: <20240424173550.16359-1-puranjay@kernel.org> <20240424173550.16359-3-puranjay@kernel.org> Date: Fri, 26 Apr 2024 10:26:28 +0000 Message-ID: Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Andrii Nakryiko writes: > On Thu, Apr 25, 2024 at 11:56=E2=80=AFAM Puranjay Mohan wrote: >> >> Andrii Nakryiko writes: >> >> > On Thu, Apr 25, 2024 at 3:14=E2=80=AFAM Puranjay Mohan wrote: >> >> >> >> Andrii Nakryiko writes: >> >> >> >> > On Wed, Apr 24, 2024 at 10:36=E2=80=AFAM Puranjay Mohan wrote: >> >> >> >> >> >> As ARM64 JIT now implements BPF_MOV64_PERCPU_REG instruction, inli= ne >> >> >> bpf_get_smp_processor_id(). >> >> >> >> >> >> ARM64 uses the per-cpu variable cpu_number to store the cpu id. >> >> >> >> >> >> Here is how the BPF and ARM64 JITed assembly changes after this co= mmit: >> >> >> >> >> >> BPF >> >> >> =3D=3D=3D=3D=3D >> >> >> BEFORE AFTER >> >> >> -------- ------- >> >> >> >> >> >> int cpu =3D bpf_get_smp_processor_id(); int cpu =3D bpf_= get_smp_processor_id(); >> >> >> (85) call bpf_get_smp_processor_id#229032 (18) r0 =3D 0xffff= 800082072008 >> >> >> (bf) r0 =3D r0 >> >> > >> >> > nit: hmm, you are probably using a bit outdated bpftool, it should = be >> >> > emitted as: >> >> > >> >> > (bf) r0 =3D &(void __percpu *)(r0) >> >> >> >> Yes, I was using the bpftool shipped with the distro. I tried it again >> >> with the latest bpftool and it emitted this as expected. >> > >> > Cool, would be nice to update the commit message with the right syntax >> > for next revision, thanks! >> > >> >> Sure, will do. >> >> >> >> >> > >> >> >> (61) r0 =3D *(u32 = *)(r0 +0) >> >> >> >> >> >> ARM64 JIT >> >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D >> >> >> >> >> >> BEFORE AFTER >> >> >> -------- ------- >> >> >> >> >> >> int cpu =3D bpf_get_smp_processor_id(); int cpu =3D bpf_get_s= mp_processor_id(); >> >> >> mov x10, #0xfffffffffffff4d0 mov x7, #0xffff8000= ffffffff >> >> >> movk x10, #0x802b, lsl #16 movk x7, #0x8207, ls= l #16 >> >> >> movk x10, #0x8000, lsl #32 movk x7, #0x2008 >> >> >> blr x10 mrs x10, tpidr_el1 >> >> >> add x7, x0, #0x0 add x7, x7, x10 >> >> >> ldr w7, [x7] >> >> >> >> >> >> Performance improvement using benchmark[1] >> >> >> >> >> >> BEFORE AFTER >> >> >> -------- ------- >> >> >> >> >> >> glob-arr-inc : 23.817 =C2=B1 0.019M/s glob-arr-inc : = 24.631 =C2=B1 0.027M/s >> >> >> arr-inc : 23.253 =C2=B1 0.019M/s arr-inc : = 23.742 =C2=B1 0.023M/s >> >> >> hash-inc : 12.258 =C2=B1 0.010M/s hash-inc : = 12.625 =C2=B1 0.004M/s >> >> >> >> >> >> [1] https://github.com/anakryiko/linux/commit/8dec900975ef >> >> >> >> >> >> Signed-off-by: Puranjay Mohan >> >> >> --- >> >> >> kernel/bpf/verifier.c | 11 ++++++++++- >> >> >> 1 file changed, 10 insertions(+), 1 deletion(-) >> >> >> >> >> > >> >> > Besides the nits, lgtm. >> >> > >> >> > Acked-by: Andrii Nakryiko >> >> > >> >> >> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c >> >> >> index 9715c88cc025..3373be261889 100644 >> >> >> --- a/kernel/bpf/verifier.c >> >> >> +++ b/kernel/bpf/verifier.c >> >> >> @@ -20205,7 +20205,7 @@ static int do_misc_fixups(struct bpf_verif= ier_env *env) >> >> >> goto next_insn; >> >> >> } >> >> >> >> >> >> -#ifdef CONFIG_X86_64 >> >> >> +#if defined(CONFIG_X86_64) || defined(CONFIG_ARM64) >> >> > >> >> > I think you can drop this, we are protected by >> >> > bpf_jit_supports_percpu_insn() check and newly added inner #if/#elif >> >> > checks? >> >> >> >> If I remove this and later add support of percpu_insn on RISCV without >> >> inlining bpf_get_smp_processor_id() then it will cause problems here >> >> right? because then the last 5-6 lines inside this if(){} will be >> >> executed for RISCV. >> > >> > Just add >> > >> > #else >> > return -EFAULT; >> >> I don't think we can return. > > ah, because it's not an error condition, right > >> >> > #endif >> > >> > ? >> > >> > I'm trying to avoid this duplication of the defined(CONFIG_xxx) checks >> > for supported architectures. >> >> Does the following look correct? >> >> I will do it like this: >> >> /* Implement bpf_get_smp_processor_id() inline. */ >> if (insn->imm =3D=3D BPF_FUNC_get_smp_processor_id && >> prog->jit_requested && bpf_jit_supports_percpu_insn(= )) { >> /* BPF_FUNC_get_smp_processor_id inlining is an >> * optimization, so if pcpu_hot.cpu_number is ev= er >> * changed in some incompatible and hard to supp= ort >> * way, it's fine to back out this inlining logic >> */ >> #if defined(CONFIG_X86_64) >> insn_buf[0] =3D BPF_MOV32_IMM(BPF_REG_0, (u32)(u= nsigned long)&pcpu_hot.cpu_number); >> insn_buf[1] =3D BPF_MOV64_PERCPU_REG(BPF_REG_0, = BPF_REG_0); >> insn_buf[2] =3D BPF_LDX_MEM(BPF_W, BPF_REG_0, BP= F_REG_0, 0); >> cnt =3D 3; >> #elif defined(CONFIG_ARM64) >> struct bpf_insn cpu_number_addr[2] =3D { BPF_LD_= IMM64(BPF_REG_0, (u64)&cpu_number) }; >> >> insn_buf[0] =3D cpu_number_addr[0]; >> insn_buf[1] =3D cpu_number_addr[1]; >> insn_buf[2] =3D BPF_MOV64_PERCPU_REG(BPF_REG_0, = BPF_REG_0); >> insn_buf[3] =3D BPF_LDX_MEM(BPF_W, BPF_REG_0, BP= F_REG_0, 0); >> cnt =3D 4; >> #else >> goto next_insn; >> #endif > > yep, I just wrote a large comment about goto next_insns above and then > saw you already proposed that :) Yep, I think this is the way. > >> new_prog =3D bpf_patch_insn_data(env, i + delta,= insn_buf, cnt); >> if (!new_prog) >> return -ENOMEM; >> >> delta +=3D cnt - 1; >> env->prog =3D prog =3D new_prog; >> insn =3D new_prog->insnsi + i + delta; >> goto next_insn; >> } >> >> >> >> >> >> > >> >> >> /* Implement bpf_get_smp_processor_id() inline. */ >> >> >> if (insn->imm =3D=3D BPF_FUNC_get_smp_processor_id= && >> >> >> prog->jit_requested && bpf_jit_supports_percpu= _insn()) { >> >> >> @@ -20214,11 +20214,20 @@ static int do_misc_fixups(struct bpf_ver= ifier_env *env) >> >> >> * changed in some incompatible and hard t= o support >> >> >> * way, it's fine to back out this inlinin= g logic >> >> >> */ >> >> >> +#if defined(CONFIG_X86_64) >> >> >> insn_buf[0] =3D BPF_MOV32_IMM(BPF_REG_0, (= u32)(unsigned long)&pcpu_hot.cpu_number); >> >> >> insn_buf[1] =3D BPF_MOV64_PERCPU_REG(BPF_R= EG_0, BPF_REG_0); >> >> >> insn_buf[2] =3D BPF_LDX_MEM(BPF_W, BPF_REG= _0, BPF_REG_0, 0); >> >> >> cnt =3D 3; >> >> >> +#elif defined(CONFIG_ARM64) >> >> >> + struct bpf_insn cpu_number_addr[2] =3D { B= PF_LD_IMM64(BPF_REG_0, (u64)&cpu_number) }; >> >> >> >> >> > >> >> > this &cpu_number offset is not guaranteed to be within 4GB on arm64? >> >> >> >> Unfortunately, the per-cpu section is not placed in the first 4GB and >> >> therefore the per-cpu pointers are not 32-bit on ARM64. >> > >> > I see. It might make sense to turn x86-64 code into using MOV64_IMM as >> > well to keep more of the logic common. Then it will be just the >> > difference of an offset that's loaded. Give it a try? >> >> I think MOV64_IMM would have more overhead than MOV32_IMM and if we can >> use it in x86-64 we should keep doing it that way. Wdyt? > > My assumption (which I didn't check) was that BPF JITs should optimize > such MOV64_IMM that have a constant fitting within 32-bits with a > faster and smaller instruction. But I'm fine leaving it as is, of > course. You are right. I verified that the JITs will optimize this if the imm is 32-bit. So, I will make it common in the next version. Also, for the readers, we are discussing: 1) BPF_MOV32_IMM : This moves a 32 bit imm into a register and zero-extends it. 2) BPF_LD_IMM64 : This moves(loads) a 64 bit imm into a register. The JITs will optimize this to a BPF_MOV32_IMM, if the imm is 32-bit. Not to be confused with : 3) BPF_MOV64_IMM: This also works with a 32-bit imm but will sign extend it to 64-bit rather than zero-extend. Thanks, Puranjay From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44D3EC4345F for ; Fri, 26 Apr 2024 10:26:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References :In-Reply-To:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9nMg0cx1IGit8mwfqQB4Q8V6xkHbXejfzCxw5twbZ58=; b=ZctYFqdBfkyBp8 926em5TI+YA2O0VisMYEEoEHHCNO3GpiATJs+vVPPl4yPoqXm+U1DyiE1tK0dpY2Lq6dscP8ow4b9 5IyIGzIZku+4Mkkhl2IjfiMfOUt+T174S614d3xw3HmcoUYHU5I+eImRLP4EogUi2h9YubgYWMxnX Sdf/AR8LI1kfvvzv2v3DSqZPunA+2M0BmnkAuu758t+Fju5wio2NTOcqtc0HaMJbWz0K+WEVoOlLo EDPON9Gf2yKiIVftT5uBoRrx2mxvNt2lKcIX70rVin64lo0USbrRm2KTP8zQBosVSb2yOKUHI5i0D lM5pahZFD9Df5afFEa6A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s0Ims-0000000C8Vs-0dep; Fri, 26 Apr 2024 10:26:38 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s0Imo-0000000C8V7-0AwM for linux-arm-kernel@lists.infradead.org; Fri, 26 Apr 2024 10:26:36 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0729F6200B; Fri, 26 Apr 2024 10:26:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C7CFC113CD; Fri, 26 Apr 2024 10:26:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1714127191; bh=Mp1x52ASowv5NbAjsAc/PHMwV+sGt7uEmgf5kXMNu0o=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=GIQp+fw1b+bCzFwKnZvtyr3gBd7s0dqMzzIfhjgPNDFBppuN0ni2nWziKZb0lXZoE 4Pl3XK0JIHC/wq2VFGZL2vRnKrYiXcfaOKY8FdZ9BMFMryc8MfDVgp5l4thjfYv4Vp hviMf8HKOnJU5xITrRXjCDZQGlLz8BXMdfpeFge4IDXmLHNHedtn+CH6E9pLLHpPLd +InkwL49wttyCk3bNG/X05HdLjagge4ED/TgNE/fP5QNst/bHmKVULAyVzNM3MIIyK 1TfXSaHwLDSkp7SwOQD4+wM9gxS4ftKLBhIdNQjBwgoxWALPOlT855UgnJTyPe1/m4 ultrnqWHzAPkA== From: Puranjay Mohan To: Andrii Nakryiko Cc: Catalin Marinas , Will Deacon , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Zi Shen Lim , Xu Kuohai , Florent Revest , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH bpf-next v2 2/2] bpf, arm64: inline bpf_get_smp_processor_id() helper In-Reply-To: References: <20240424173550.16359-1-puranjay@kernel.org> <20240424173550.16359-3-puranjay@kernel.org> Date: Fri, 26 Apr 2024 10:26:28 +0000 Message-ID: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240426_032634_195846_63851D7B X-CRM114-Status: GOOD ( 39.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org QW5kcmlpIE5ha3J5aWtvIDxhbmRyaWkubmFrcnlpa29AZ21haWwuY29tPiB3cml0ZXM6Cgo+IE9u IFRodSwgQXByIDI1LCAyMDI0IGF0IDExOjU24oCvQU0gUHVyYW5qYXkgTW9oYW4gPHB1cmFuamF5 QGtlcm5lbC5vcmc+IHdyb3RlOgo+Pgo+PiBBbmRyaWkgTmFrcnlpa28gPGFuZHJpaS5uYWtyeWlr b0BnbWFpbC5jb20+IHdyaXRlczoKPj4KPj4gPiBPbiBUaHUsIEFwciAyNSwgMjAyNCBhdCAzOjE0 4oCvQU0gUHVyYW5qYXkgTW9oYW4gPHB1cmFuamF5QGtlcm5lbC5vcmc+IHdyb3RlOgo+PiA+Pgo+ PiA+PiBBbmRyaWkgTmFrcnlpa28gPGFuZHJpaS5uYWtyeWlrb0BnbWFpbC5jb20+IHdyaXRlczoK Pj4gPj4KPj4gPj4gPiBPbiBXZWQsIEFwciAyNCwgMjAyNCBhdCAxMDozNuKAr0FNIFB1cmFuamF5 IE1vaGFuIDxwdXJhbmpheUBrZXJuZWwub3JnPiB3cm90ZToKPj4gPj4gPj4KPj4gPj4gPj4gQXMg QVJNNjQgSklUIG5vdyBpbXBsZW1lbnRzIEJQRl9NT1Y2NF9QRVJDUFVfUkVHIGluc3RydWN0aW9u LCBpbmxpbmUKPj4gPj4gPj4gYnBmX2dldF9zbXBfcHJvY2Vzc29yX2lkKCkuCj4+ID4+ID4+Cj4+ ID4+ID4+IEFSTTY0IHVzZXMgdGhlIHBlci1jcHUgdmFyaWFibGUgY3B1X251bWJlciB0byBzdG9y ZSB0aGUgY3B1IGlkLgo+PiA+PiA+Pgo+PiA+PiA+PiBIZXJlIGlzIGhvdyB0aGUgQlBGIGFuZCBB Uk02NCBKSVRlZCBhc3NlbWJseSBjaGFuZ2VzIGFmdGVyIHRoaXMgY29tbWl0Ogo+PiA+PiA+Pgo+ PiA+PiA+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEJQRgo+PiA+ PiA+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPT09PT0KPj4gPj4g Pj4gICAgICAgICAgICAgICBCRUZPUkUgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBBRlRFUgo+PiA+PiA+PiAgICAgICAgICAgICAgLS0tLS0tLS0gICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgLS0tLS0tLQo+PiA+PiA+Pgo+PiA+PiA+PiBpbnQgY3B1ID0g YnBmX2dldF9zbXBfcHJvY2Vzc29yX2lkKCk7ICAgICAgICAgICBpbnQgY3B1ID0gYnBmX2dldF9z bXBfcHJvY2Vzc29yX2lkKCk7Cj4+ID4+ID4+ICg4NSkgY2FsbCBicGZfZ2V0X3NtcF9wcm9jZXNz b3JfaWQjMjI5MDMyICAgICAgICgxOCkgcjAgPSAweGZmZmY4MDAwODIwNzIwMDgKPj4gPj4gPj4g ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKGJmKSByMCA9 IHIwCj4+ID4+ID4KPj4gPj4gPiBuaXQ6IGhtbSwgeW91IGFyZSBwcm9iYWJseSB1c2luZyBhIGJp dCBvdXRkYXRlZCBicGZ0b29sLCBpdCBzaG91bGQgYmUKPj4gPj4gPiBlbWl0dGVkIGFzOgo+PiA+ PiA+Cj4+ID4+ID4gKGJmKSByMCA9ICYodm9pZCBfX3BlcmNwdSAqKShyMCkKPj4gPj4KPj4gPj4g WWVzLCBJIHdhcyB1c2luZyB0aGUgYnBmdG9vbCBzaGlwcGVkIHdpdGggdGhlIGRpc3Ryby4gSSB0 cmllZCBpdCBhZ2Fpbgo+PiA+PiB3aXRoIHRoZSBsYXRlc3QgYnBmdG9vbCBhbmQgaXQgZW1pdHRl ZCB0aGlzIGFzIGV4cGVjdGVkLgo+PiA+Cj4+ID4gQ29vbCwgd291bGQgYmUgbmljZSB0byB1cGRh dGUgdGhlIGNvbW1pdCBtZXNzYWdlIHdpdGggdGhlIHJpZ2h0IHN5bnRheAo+PiA+IGZvciBuZXh0 IHJldmlzaW9uLCB0aGFua3MhCj4+ID4KPj4KPj4gU3VyZSwgd2lsbCBkby4KPj4KPj4gPj4KPj4g Pj4gPgo+PiA+PiA+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAoNjEpIHIwID0gKih1MzIgKikocjAgKzApCj4+ID4+ID4+Cj4+ID4+ID4+ICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQVJNNjQgSklUCj4+ID4+ID4+ICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICA9PT09PT09PT09PQo+PiA+PiA+Pgo+PiA+PiA+ PiAgICAgICAgICAgICAgIEJFRk9SRSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIEFGVEVSCj4+ID4+ID4+ICAgICAgICAgICAgICAtLS0tLS0tLSAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAtLS0tLS0tCj4+ID4+ID4+Cj4+ID4+ID4+IGludCBjcHUgPSBi cGZfZ2V0X3NtcF9wcm9jZXNzb3JfaWQoKTsgICAgICBpbnQgY3B1ID0gYnBmX2dldF9zbXBfcHJv Y2Vzc29yX2lkKCk7Cj4+ID4+ID4+IG1vdiAgICAgeDEwLCAjMHhmZmZmZmZmZmZmZmZmNGQwICAg ICAgICAgICBtb3YgICAgIHg3LCAjMHhmZmZmODAwMGZmZmZmZmZmCj4+ID4+ID4+IG1vdmsgICAg eDEwLCAjMHg4MDJiLCBsc2wgIzE2ICAgICAgICAgICAgICBtb3ZrICAgIHg3LCAjMHg4MjA3LCBs c2wgIzE2Cj4+ID4+ID4+IG1vdmsgICAgeDEwLCAjMHg4MDAwLCBsc2wgIzMyICAgICAgICAgICAg ICBtb3ZrICAgIHg3LCAjMHgyMDA4Cj4+ID4+ID4+IGJsciAgICAgeDEwICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBtcnMgICAgIHgxMCwgdHBpZHJfZWwxCj4+ID4+ID4+IGFkZCAgICAg eDcsIHgwLCAjMHgwICAgICAgICAgICAgICAgICAgICAgICBhZGQgICAgIHg3LCB4NywgeDEwCj4+ ID4+ID4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsZHIgICAg IHc3LCBbeDddCj4+ID4+ID4+Cj4+ID4+ID4+IFBlcmZvcm1hbmNlIGltcHJvdmVtZW50IHVzaW5n IGJlbmNobWFya1sxXQo+PiA+PiA+Pgo+PiA+PiA+PiAgICAgICAgICAgICAgQkVGT1JFICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgQUZURVIKPj4gPj4gPj4gICAgICAgICAg ICAgLS0tLS0tLS0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLS0tLS0tLQo+ PiA+PiA+Pgo+PiA+PiA+PiBnbG9iLWFyci1pbmMgICA6ICAgMjMuODE3IMKxIDAuMDE5TS9zICAg ICAgZ2xvYi1hcnItaW5jICAgOiAgIDI0LjYzMSDCsSAwLjAyN00vcwo+PiA+PiA+PiBhcnItaW5j ICAgICAgICA6ICAgMjMuMjUzIMKxIDAuMDE5TS9zICAgICAgYXJyLWluYyAgICAgICAgOiAgIDIz Ljc0MiDCsSAwLjAyM00vcwo+PiA+PiA+PiBoYXNoLWluYyAgICAgICA6ICAgMTIuMjU4IMKxIDAu MDEwTS9zICAgICAgaGFzaC1pbmMgICAgICAgOiAgIDEyLjYyNSDCsSAwLjAwNE0vcwo+PiA+PiA+ Pgo+PiA+PiA+PiBbMV0gaHR0cHM6Ly9naXRodWIuY29tL2FuYWtyeWlrby9saW51eC9jb21taXQv OGRlYzkwMDk3NWVmCj4+ID4+ID4+Cj4+ID4+ID4+IFNpZ25lZC1vZmYtYnk6IFB1cmFuamF5IE1v aGFuIDxwdXJhbmpheUBrZXJuZWwub3JnPgo+PiA+PiA+PiAtLS0KPj4gPj4gPj4gIGtlcm5lbC9i cGYvdmVyaWZpZXIuYyB8IDExICsrKysrKysrKystCj4+ID4+ID4+ICAxIGZpbGUgY2hhbmdlZCwg MTAgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQo+PiA+PiA+Pgo+PiA+PiA+Cj4+ID4+ID4g QmVzaWRlcyB0aGUgbml0cywgbGd0bS4KPj4gPj4gPgo+PiA+PiA+IEFja2VkLWJ5OiBBbmRyaWkg TmFrcnlpa28gPGFuZHJpaUBrZXJuZWwub3JnPgo+PiA+PiA+Cj4+ID4+ID4+IGRpZmYgLS1naXQg YS9rZXJuZWwvYnBmL3ZlcmlmaWVyLmMgYi9rZXJuZWwvYnBmL3ZlcmlmaWVyLmMKPj4gPj4gPj4g aW5kZXggOTcxNWM4OGNjMDI1Li4zMzczYmUyNjE4ODkgMTAwNjQ0Cj4+ID4+ID4+IC0tLSBhL2tl cm5lbC9icGYvdmVyaWZpZXIuYwo+PiA+PiA+PiArKysgYi9rZXJuZWwvYnBmL3ZlcmlmaWVyLmMK Pj4gPj4gPj4gQEAgLTIwMjA1LDcgKzIwMjA1LDcgQEAgc3RhdGljIGludCBkb19taXNjX2ZpeHVw cyhzdHJ1Y3QgYnBmX3ZlcmlmaWVyX2VudiAqZW52KQo+PiA+PiA+PiAgICAgICAgICAgICAgICAg ICAgICAgICBnb3RvIG5leHRfaW5zbjsKPj4gPj4gPj4gICAgICAgICAgICAgICAgIH0KPj4gPj4g Pj4KPj4gPj4gPj4gLSNpZmRlZiBDT05GSUdfWDg2XzY0Cj4+ID4+ID4+ICsjaWYgZGVmaW5lZChD T05GSUdfWDg2XzY0KSB8fCBkZWZpbmVkKENPTkZJR19BUk02NCkKPj4gPj4gPgo+PiA+PiA+IEkg dGhpbmsgeW91IGNhbiBkcm9wIHRoaXMsIHdlIGFyZSBwcm90ZWN0ZWQgYnkKPj4gPj4gPiBicGZf aml0X3N1cHBvcnRzX3BlcmNwdV9pbnNuKCkgY2hlY2sgYW5kIG5ld2x5IGFkZGVkIGlubmVyICNp Zi8jZWxpZgo+PiA+PiA+IGNoZWNrcz8KPj4gPj4KPj4gPj4gSWYgSSByZW1vdmUgdGhpcyBhbmQg bGF0ZXIgYWRkIHN1cHBvcnQgb2YgcGVyY3B1X2luc24gb24gUklTQ1Ygd2l0aG91dAo+PiA+PiBp bmxpbmluZyBicGZfZ2V0X3NtcF9wcm9jZXNzb3JfaWQoKSB0aGVuIGl0IHdpbGwgY2F1c2UgcHJv YmxlbXMgaGVyZQo+PiA+PiByaWdodD8gYmVjYXVzZSB0aGVuIHRoZSBsYXN0IDUtNiBsaW5lcyBp bnNpZGUgdGhpcyBpZigpe30gd2lsbCBiZQo+PiA+PiBleGVjdXRlZCBmb3IgUklTQ1YuCj4+ID4K Pj4gPiBKdXN0IGFkZAo+PiA+Cj4+ID4gI2Vsc2UKPj4gPiByZXR1cm4gLUVGQVVMVDsKPj4KPj4g SSBkb24ndCB0aGluayB3ZSBjYW4gcmV0dXJuLgo+Cj4gYWgsIGJlY2F1c2UgaXQncyBub3QgYW4g ZXJyb3IgY29uZGl0aW9uLCByaWdodAo+Cj4+Cj4+ID4gI2VuZGlmCj4+ID4KPj4gPiA/Cj4+ID4K Pj4gPiBJJ20gdHJ5aW5nIHRvIGF2b2lkIHRoaXMgZHVwbGljYXRpb24gb2YgdGhlIGRlZmluZWQo Q09ORklHX3h4eCkgY2hlY2tzCj4+ID4gZm9yIHN1cHBvcnRlZCBhcmNoaXRlY3R1cmVzLgo+Pgo+ PiBEb2VzIHRoZSBmb2xsb3dpbmcgbG9vayBjb3JyZWN0Pwo+Pgo+PiBJIHdpbGwgZG8gaXQgbGlr ZSB0aGlzOgo+Pgo+PiAgICAgICAgICAgICAgICAgLyogSW1wbGVtZW50IGJwZl9nZXRfc21wX3By b2Nlc3Nvcl9pZCgpIGlubGluZS4gKi8KPj4gICAgICAgICAgICAgICAgIGlmIChpbnNuLT5pbW0g PT0gQlBGX0ZVTkNfZ2V0X3NtcF9wcm9jZXNzb3JfaWQgJiYKPj4gICAgICAgICAgICAgICAgICAg ICBwcm9nLT5qaXRfcmVxdWVzdGVkICYmIGJwZl9qaXRfc3VwcG9ydHNfcGVyY3B1X2luc24oKSkg ewo+PiAgICAgICAgICAgICAgICAgICAgICAgICAvKiBCUEZfRlVOQ19nZXRfc21wX3Byb2Nlc3Nv cl9pZCBpbmxpbmluZyBpcyBhbgo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgKiBvcHRpbWl6 YXRpb24sIHNvIGlmIHBjcHVfaG90LmNwdV9udW1iZXIgaXMgZXZlcgo+PiAgICAgICAgICAgICAg ICAgICAgICAgICAgKiBjaGFuZ2VkIGluIHNvbWUgaW5jb21wYXRpYmxlIGFuZCBoYXJkIHRvIHN1 cHBvcnQKPj4gICAgICAgICAgICAgICAgICAgICAgICAgICogd2F5LCBpdCdzIGZpbmUgdG8gYmFj ayBvdXQgdGhpcyBpbmxpbmluZyBsb2dpYwo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgKi8K Pj4gI2lmIGRlZmluZWQoQ09ORklHX1g4Nl82NCkKPj4gICAgICAgICAgICAgICAgICAgICAgICAg aW5zbl9idWZbMF0gPSBCUEZfTU9WMzJfSU1NKEJQRl9SRUdfMCwgKHUzMikodW5zaWduZWQgbG9u ZykmcGNwdV9ob3QuY3B1X251bWJlcik7Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgIGluc25f YnVmWzFdID0gQlBGX01PVjY0X1BFUkNQVV9SRUcoQlBGX1JFR18wLCBCUEZfUkVHXzApOwo+PiAg ICAgICAgICAgICAgICAgICAgICAgICBpbnNuX2J1ZlsyXSA9IEJQRl9MRFhfTUVNKEJQRl9XLCBC UEZfUkVHXzAsIEJQRl9SRUdfMCwgMCk7Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgIGNudCA9 IDM7Cj4+ICNlbGlmIGRlZmluZWQoQ09ORklHX0FSTTY0KQo+PiAgICAgICAgICAgICAgICAgICAg ICAgICBzdHJ1Y3QgYnBmX2luc24gY3B1X251bWJlcl9hZGRyWzJdID0geyBCUEZfTERfSU1NNjQo QlBGX1JFR18wLCAodTY0KSZjcHVfbnVtYmVyKSB9Owo+Pgo+PiAgICAgICAgICAgICAgICAgICAg ICAgICBpbnNuX2J1ZlswXSA9IGNwdV9udW1iZXJfYWRkclswXTsKPj4gICAgICAgICAgICAgICAg ICAgICAgICAgaW5zbl9idWZbMV0gPSBjcHVfbnVtYmVyX2FkZHJbMV07Cj4+ICAgICAgICAgICAg ICAgICAgICAgICAgIGluc25fYnVmWzJdID0gQlBGX01PVjY0X1BFUkNQVV9SRUcoQlBGX1JFR18w LCBCUEZfUkVHXzApOwo+PiAgICAgICAgICAgICAgICAgICAgICAgICBpbnNuX2J1ZlszXSA9IEJQ Rl9MRFhfTUVNKEJQRl9XLCBCUEZfUkVHXzAsIEJQRl9SRUdfMCwgMCk7Cj4+ICAgICAgICAgICAg ICAgICAgICAgICAgIGNudCA9IDQ7Cj4+ICNlbHNlCj4+ICAgICAgICAgICAgICAgICAgICAgICAg IGdvdG8gbmV4dF9pbnNuOwo+PiAjZW5kaWYKPgo+IHllcCwgSSBqdXN0IHdyb3RlIGEgbGFyZ2Ug Y29tbWVudCBhYm91dCBnb3RvIG5leHRfaW5zbnMgYWJvdmUgYW5kIHRoZW4KPiBzYXcgeW91IGFs cmVhZHkgcHJvcG9zZWQgdGhhdCA6KSBZZXAsIEkgdGhpbmsgdGhpcyBpcyB0aGUgd2F5Lgo+Cj4+ ICAgICAgICAgICAgICAgICAgICAgICAgIG5ld19wcm9nID0gYnBmX3BhdGNoX2luc25fZGF0YShl bnYsIGkgKyBkZWx0YSwgaW5zbl9idWYsIGNudCk7Cj4+ICAgICAgICAgICAgICAgICAgICAgICAg IGlmICghbmV3X3Byb2cpCj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmV0dXJu IC1FTk9NRU07Cj4+Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgIGRlbHRhICAgICs9IGNudCAt IDE7Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgIGVudi0+cHJvZyA9IHByb2cgPSBuZXdfcHJv ZzsKPj4gICAgICAgICAgICAgICAgICAgICAgICAgaW5zbiAgICAgID0gbmV3X3Byb2ctPmluc25z aSArIGkgKyBkZWx0YTsKPj4gICAgICAgICAgICAgICAgICAgICAgICAgZ290byBuZXh0X2luc247 Cj4+ICAgICAgICAgICAgICAgICB9Cj4+Cj4+Cj4+ID4+Cj4+ID4+ID4KPj4gPj4gPj4gICAgICAg ICAgICAgICAgIC8qIEltcGxlbWVudCBicGZfZ2V0X3NtcF9wcm9jZXNzb3JfaWQoKSBpbmxpbmUu ICovCj4+ID4+ID4+ICAgICAgICAgICAgICAgICBpZiAoaW5zbi0+aW1tID09IEJQRl9GVU5DX2dl dF9zbXBfcHJvY2Vzc29yX2lkICYmCj4+ID4+ID4+ICAgICAgICAgICAgICAgICAgICAgcHJvZy0+ aml0X3JlcXVlc3RlZCAmJiBicGZfaml0X3N1cHBvcnRzX3BlcmNwdV9pbnNuKCkpIHsKPj4gPj4g Pj4gQEAgLTIwMjE0LDExICsyMDIxNCwyMCBAQCBzdGF0aWMgaW50IGRvX21pc2NfZml4dXBzKHN0 cnVjdCBicGZfdmVyaWZpZXJfZW52ICplbnYpCj4+ID4+ID4+ICAgICAgICAgICAgICAgICAgICAg ICAgICAqIGNoYW5nZWQgaW4gc29tZSBpbmNvbXBhdGlibGUgYW5kIGhhcmQgdG8gc3VwcG9ydAo+ PiA+PiA+PiAgICAgICAgICAgICAgICAgICAgICAgICAgKiB3YXksIGl0J3MgZmluZSB0byBiYWNr IG91dCB0aGlzIGlubGluaW5nIGxvZ2ljCj4+ID4+ID4+ICAgICAgICAgICAgICAgICAgICAgICAg ICAqLwo+PiA+PiA+PiArI2lmIGRlZmluZWQoQ09ORklHX1g4Nl82NCkKPj4gPj4gPj4gICAgICAg ICAgICAgICAgICAgICAgICAgaW5zbl9idWZbMF0gPSBCUEZfTU9WMzJfSU1NKEJQRl9SRUdfMCwg KHUzMikodW5zaWduZWQgbG9uZykmcGNwdV9ob3QuY3B1X251bWJlcik7Cj4+ID4+ID4+ICAgICAg ICAgICAgICAgICAgICAgICAgIGluc25fYnVmWzFdID0gQlBGX01PVjY0X1BFUkNQVV9SRUcoQlBG X1JFR18wLCBCUEZfUkVHXzApOwo+PiA+PiA+PiAgICAgICAgICAgICAgICAgICAgICAgICBpbnNu X2J1ZlsyXSA9IEJQRl9MRFhfTUVNKEJQRl9XLCBCUEZfUkVHXzAsIEJQRl9SRUdfMCwgMCk7Cj4+ ID4+ID4+ICAgICAgICAgICAgICAgICAgICAgICAgIGNudCA9IDM7Cj4+ID4+ID4+ICsjZWxpZiBk ZWZpbmVkKENPTkZJR19BUk02NCkKPj4gPj4gPj4gKyAgICAgICAgICAgICAgICAgICAgICAgc3Ry dWN0IGJwZl9pbnNuIGNwdV9udW1iZXJfYWRkclsyXSA9IHsgQlBGX0xEX0lNTTY0KEJQRl9SRUdf MCwgKHU2NCkmY3B1X251bWJlcikgfTsKPj4gPj4gPj4KPj4gPj4gPgo+PiA+PiA+IHRoaXMgJmNw dV9udW1iZXIgb2Zmc2V0IGlzIG5vdCBndWFyYW50ZWVkIHRvIGJlIHdpdGhpbiA0R0Igb24gYXJt NjQ/Cj4+ID4+Cj4+ID4+IFVuZm9ydHVuYXRlbHksIHRoZSBwZXItY3B1IHNlY3Rpb24gaXMgbm90 IHBsYWNlZCBpbiB0aGUgZmlyc3QgNEdCIGFuZAo+PiA+PiB0aGVyZWZvcmUgdGhlIHBlci1jcHUg cG9pbnRlcnMgYXJlIG5vdCAzMi1iaXQgb24gQVJNNjQuCj4+ID4KPj4gPiBJIHNlZS4gSXQgbWln aHQgbWFrZSBzZW5zZSB0byB0dXJuIHg4Ni02NCBjb2RlIGludG8gdXNpbmcgTU9WNjRfSU1NIGFz Cj4+ID4gd2VsbCB0byBrZWVwIG1vcmUgb2YgdGhlIGxvZ2ljIGNvbW1vbi4gVGhlbiBpdCB3aWxs IGJlIGp1c3QgdGhlCj4+ID4gZGlmZmVyZW5jZSBvZiBhbiBvZmZzZXQgdGhhdCdzIGxvYWRlZC4g R2l2ZSBpdCBhIHRyeT8KPj4KPj4gSSB0aGluayBNT1Y2NF9JTU0gd291bGQgaGF2ZSBtb3JlIG92 ZXJoZWFkIHRoYW4gTU9WMzJfSU1NIGFuZCBpZiB3ZSBjYW4KPj4gdXNlIGl0IGluIHg4Ni02NCB3 ZSBzaG91bGQga2VlcCBkb2luZyBpdCB0aGF0IHdheS4gV2R5dD8KPgo+IE15IGFzc3VtcHRpb24g KHdoaWNoIEkgZGlkbid0IGNoZWNrKSB3YXMgdGhhdCBCUEYgSklUcyBzaG91bGQgb3B0aW1pemUK PiBzdWNoIE1PVjY0X0lNTSB0aGF0IGhhdmUgYSBjb25zdGFudCBmaXR0aW5nIHdpdGhpbiAzMi1i aXRzIHdpdGggYQo+IGZhc3RlciBhbmQgc21hbGxlciBpbnN0cnVjdGlvbi4gQnV0IEknbSBmaW5l IGxlYXZpbmcgaXQgYXMgaXMsIG9mCj4gY291cnNlLgoKWW91IGFyZSByaWdodC4gSSB2ZXJpZmll ZCB0aGF0IHRoZSBKSVRzIHdpbGwgb3B0aW1pemUgdGhpcyBpZiB0aGUgaW1tIGlzCjMyLWJpdC4g U28sIEkgd2lsbCBtYWtlIGl0IGNvbW1vbiBpbiB0aGUgbmV4dCB2ZXJzaW9uLgoKQWxzbywgZm9y IHRoZSByZWFkZXJzLCB3ZSBhcmUgZGlzY3Vzc2luZzoKCjEpIEJQRl9NT1YzMl9JTU0gOiBUaGlz IG1vdmVzIGEgMzIgYml0IGltbSBpbnRvIGEgcmVnaXN0ZXIgYW5kCiAgICAgICAgICAgICAgICAg ICB6ZXJvLWV4dGVuZHMgaXQuCgoyKSBCUEZfTERfSU1NNjQgOiBUaGlzIG1vdmVzKGxvYWRzKSBh IDY0IGJpdCBpbW0gaW50byBhIHJlZ2lzdGVyLiBUaGUKICAgICAgICAgICAgICAgICAgSklUcyB3 aWxsIG9wdGltaXplIHRoaXMgdG8gYSBCUEZfTU9WMzJfSU1NLCBpZiB0aGUgaW1tCiAgICAgICAg ICAgICAgICAgIGlzIDMyLWJpdC4KCk5vdCB0byBiZSBjb25mdXNlZCB3aXRoIDoKMykgQlBGX01P VjY0X0lNTTogVGhpcyBhbHNvIHdvcmtzIHdpdGggYSAzMi1iaXQgaW1tIGJ1dCB3aWxsIHNpZ24g ZXh0ZW5kCiAgICAgICAgICAgICAgICAgIGl0IHRvIDY0LWJpdCByYXRoZXIgdGhhbiB6ZXJvLWV4 dGVuZC4KClRoYW5rcywKUHVyYW5qYXkKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1r ZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWls bWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK