From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1F909CDB471 for ; Mon, 22 Jun 2026 08:07:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:CC:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7l3MIp31Oqn6l35ANG2oeSNGQ5mPwFEIt+3BHRhEgFs=; b=gmGJzbHOl0jzguZXDMHQGqGs47 1Z60ZEZJXNDtQFsHho650h3gc95FJJqKcCD9Dwdjph3V8yoicbcKPkjdbd8SPY213wvqQQ0qiXckr NcrQcMUDBo/P2aTioP3XjNoBh9WOBBlqTRfw2111OQYMXFge/XqwYMHuZJK8+Ne52zph4wLqGQIYs LDjjjCNOidj25XQnVod0gWE96ELiHTN3oZ3ct0feX6YpwXKFHEhve482Kvc7nOyw25A8ZM8RhlIA/ duo2YhDkxNLyO5QdJvAlgz94bIV1lH2MFCANe+Rklqxi5hoBctVGfbtVa2x7I4VhkIGWLPVtOfG2f h32ZPGNA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wbZgM-00000004dms-01ts; Mon, 22 Jun 2026 08:07:02 +0000 Received: from canpmsgout06.his.huawei.com ([113.46.200.221]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wbZgB-00000004dS1-2bDi; Mon, 22 Jun 2026 08:06:54 +0000 dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=7l3MIp31Oqn6l35ANG2oeSNGQ5mPwFEIt+3BHRhEgFs=; b=JVcKeZfRvP8T8eSthBTDPagvkz4L0tUeCdqiAVGv4Eh3T3SVTQXrwmvOHMtG0cUUpo0xQ5/O4 aR6aIwZIftD9mfmnCd56hixc1MZIztHFios6RgBUF+IkCieMOqGm+NmJXzvSoKNNHpGsbDuVE/n scbJtcuZ3SQZ7BdsHKOvpH4= Received: from mail.maildlp.com (unknown [172.19.163.104]) by canpmsgout06.his.huawei.com (SkyGuard) with ESMTPS id 4gkLCQ6hV0zRhVP; Mon, 22 Jun 2026 15:57:38 +0800 (CST) Received: from dggpemf500011.china.huawei.com (unknown [7.185.36.131]) by mail.maildlp.com (Postfix) with ESMTPS id AB1BA40363; Mon, 22 Jun 2026 16:06:41 +0800 (CST) Received: from [10.67.109.254] (10.67.109.254) by dggpemf500011.china.huawei.com (7.185.36.131) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Mon, 22 Jun 2026 16:06:39 +0800 Message-ID: <2a34ff43-78a0-4d99-885a-0e7f560634ca@huawei.com> Date: Mon, 22 Jun 2026 16:06:38 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH RFC 3/3] arm64: Add HOTPLUG_PARALLEL support for secondary CPUs To: Will Deacon CC: Michael Kelley , "catalin.marinas@arm.com" , "tsbogend@alpha.franken.de" , "pjw@kernel.org" , "palmer@dabbelt.com" , "aou@eecs.berkeley.edu" , "alex@ghiti.fr" , "tglx@kernel.org" , "mingo@redhat.com" , "bp@alien8.de" , "dave.hansen@linux.intel.com" , "hpa@zytor.com" , "peterz@infradead.org" , "kees@kernel.org" , "nathan@kernel.org" , "linusw@kernel.org" , "ojeda@kernel.org" , "david.kaplan@amd.com" , "lukas.bulwahn@redhat.com" , "ryan.roberts@arm.com" , "maz@kernel.org" , "timothy.hayes@arm.com" , "lpieralisi@kernel.org" , "thuth@redhat.com" , "oupton@kernel.org" , "yeoreum.yun@arm.com" , "miko.lenczewski@arm.com" , "broonie@kernel.org" , "kevin.brodsky@arm.com" , "james.clark@linaro.org" , "tabba@google.com" , "mrigendra.chaubey@gmail.com" , "arnd@arndb.de" , "anshuman.khandual@arm.com" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-mips@vger.kernel.org" , "linux-riscv@lists.infradead.org" References: <20260611133809.3854977-1-ruanjinjie@huawei.com> <20260611133809.3854977-4-ruanjinjie@huawei.com> From: Jinjie Ruan In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.67.109.254] X-ClientProxiedBy: kwepems200001.china.huawei.com (7.221.188.67) To dggpemf500011.china.huawei.com (7.185.36.131) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260622_010652_287017_C53210AA X-CRM114-Status: GOOD ( 26.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 6/18/2026 8:21 PM, Will Deacon wrote: > Hi Jinjie, > > On Mon, Jun 15, 2026 at 04:51:48PM +0800, Jinjie Ruan wrote: >> On 6/12/2026 11:45 PM, Michael Kelley wrote: >>> From: Jinjie Ruan Sent: Thursday, June 11, 2026 6:38 AM >>>> >>>> Support for parallel secondary CPU bringup is already utilized by x86, >>>> MIPS, and RISC-V. This patch brings this capability to the arm64 >>>> architecture. >>>> >>>> Rework the global `secondary_data` accessed during early boot into >>>> a per-CPU array. This array maps logical CPU IDs to MPIDR_EL1 values, >>>> enabling the early boot code in head.S to resolve each secondary CPU's >>>> logical ID concurrently. >>>> >>>> To fully enable HOTPLUG_PARALLEL, this patch implements: >>>> 1) An arm64-specific arch_cpuhp_kick_ap_alive() handler. >>>> 2) Callbacks to cpuhp_ap_sync_alive() inside secondary_start_kernel(). >>>> >>>> Successfully tested on QEMU ARM64 virt machine (KVM on, 128 vCPUs). >>>> >>>> | test kernel | secondary CPUs boot time | >>>> | --------------------- | -------------------- | >>>> | Without this patch | 155.672 | >>>> | cpuhp.parallel=0 | 62.897 | >>>> | cpuhp.parallel=1 | 166.703 | >>> >>> The last two rows seem mixed up. I would expect parallel=0 to >>> result in a longer boot time. >> >> Hi, Michael, >> >> The results are correct and not mixed up. >> >> Compared to the original non‑HOTPLUG_PARALLEL approach, the advantage of >> cpuhp.parallel=0 lies in its use of cpu_relax(`yield` on arm64) instead >> of the wait_for_completion_timeout() mechanism (which may cause sleep >> and context switching). This significantly reduces the overhead of VM >> exits and context switches in a KVM guest, thereby cutting the secondary >> CPU boot time by more than half. > > I don't think that's a particularly compelling reason to enable this for > arm64, in all honesty. The yield instruction typically doesn't do > anything on actual arm64 silicon, so this probably means that you're > introducing busy-loops which tend to be bad for power and scalability. After updating the implementation in v2, the performance gains are primarily observed on actual hardware. > > I implemented this a while ago [1] but didn't manage to see much in terms > of performance improvement and so I didn't bother to send the patches out As shown in v2 below, on actual hardware, this results in a 40%–60% reduction in boot time. Bringup Time Comparison (ms, lower is better): | Platform | Baseline| P=0 | P=1 | Delta(%)| | --------------------- | ------- | ------- | ------ | ------- | | 64-core ATF QEMU | 2075.8 | 2080.7 | 1653.4 | 20.34% | | 192-core server(HIP12)| 14619.2 | 14619.1 | 8589.4 | 41.21% | | 32-core board | 2776.5 | 2881.0 | 1045.0 | 62.36% | Link: https://lore.kernel.org/all/20260618092444.1316336-5-ruanjinjie@huawei.com/ > after talking about it at KVM forum [2]. However, as mentioned at the end > of that talk, it _is_ still useful for confidential VMs using PSCI so > let me dust off my old series and send it out to see what you think. > > It relies on PSCI v0.2, which means we don't need the NR_CPUS size array > for secondary_data and I also have some support for error handling (it > doesn't look like you handle __early_cpu_boot_status properly). I need some time to look closely at your patch. Alternatively, I will integrate your changes, re-test everything on actual hardware, and then send out a revised version. > > It looks like I could include your first patch, though! Thank you very much. > > Will > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=cpu-hotplug It seems that the following patch removing `rcutree_report_cpu_starting()` will reintroduce the original issue as commit ce3d31ad3cac ("arm64/smp: Move rcu_cpu_starting() earlier") soloved. Link: https://web.git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/commit/?h=cpu-hotplug&id=bba4b62f45f2614bf6085e6cd3f233528f85bf26 Indeed, I also noticed that the invocation order of rcutree_report_cpu_starting() on arm64 is somewhat suboptimal. It hinders the implementation of parallel bringup on arm64 and could potentially lead to RCU stalls. Link: https://lore.kernel.org/all/20260618092444.1316336-4-ruanjinjie@huawei.com/ [ 0.329017] smp: Bringing up secondary CPUs ... [ 0.343628] Detected VIPT I-cache on CPU1 [ 0.343788] [ 0.343806] ============================= [ 0.343816] WARNING: suspicious RCU usage [ 0.343966] 7.1.0-rc1-g27c1871848a2 #109 Not tainted [ 0.344087] ----------------------------- [ 0.344098] kernel/locking/lockdep.c:3801 RCU-list traversed in non-reader section!! [ 0.344112] [ 0.344112] other info that might help us debug this: [ 0.344112] [ 0.344135] [ 0.344135] RCU used illegally from offline CPU! [ 0.344135] rcu_scheduler_active = 1, debug_locks = 1 [ 0.344174] no locks held by swapper/1/0. [ 0.344204] [ 0.344204] stack backtrace: [ 0.344611] CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 7.1.0-rc1-g27c1871848a2 #109 PREEMPT [ 0.344707] Hardware name: linux,dummy-virt (DT) [ 0.345267] Call trace: [ 0.345436] show_stack+0x18/0x24 (C) [ 0.345593] dump_stack_lvl+0x90/0xd0 [ 0.345620] dump_stack+0x18/0x24 [ 0.345639] lockdep_rcu_suspicious+0x170/0x234 [ 0.345665] __lock_acquire+0xdd4/0x2078 [ 0.345688] lock_acquire+0x1c4/0x3f0 [ 0.345711] _raw_spin_lock_irqsave+0x60/0x88 [ 0.345736] down_trylock+0x18/0x48 [ 0.345758] __down_trylock_console_sem+0x38/0xc4 [ 0.345782] vprintk_emit+0x23c/0x3d0 [ 0.345802] vprintk_default+0x38/0x44 [ 0.345822] vprintk+0x28/0x34 [ 0.345841] _printk+0x5c/0x84 [ 0.345864] cpuinfo_store_cpu+0x174/0x298 [ 0.345884] secondary_start_kernel+0xbc/0x150 [ 0.345905] __secondary_switched+0xc0/0xc4 [ 0.350307] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 [ 0.350523] GICv3: CPU1: using allocated LPI pending table @0x00000001042f0000 [ 0.351303] CPU1: Booted secondary processor 0x0000000001 [0x410fd034] [ 0.387425] Detected VIPT I-cache on CPU2 > [2] https://www.youtube.com/watch?v=Q6kOshnnQuE > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C2F3CDB471 for ; Mon, 22 Jun 2026 08:07:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:CC:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ffzqfHM/l6NOfSx+vGWxy53P7f3TCYuZeGdinmIsn5w=; b=H+F6r4SzuUPOcR WM2CusCAksfV84DHeV6nlr0dBKdWkeRCSf+XkVJ/5gWrIn9+WT0w8fIGCgyXriTh3rikOLbiIj6Y3 GPh9/KHrAsIpD1+fpJWQzmsjczp72rMA1W8aPx617A6dnUFGYhh3GDykmzvq11BtUK5fB+/jn+CNq mDGaDTiJZy4AxdYs84mYqGpUa3hgkYo0ToKSuoY+ZEFNvstC9Klyocy81QYEVU4qP+SDroO4zWkW7 iaxlYU2Mk57XOUBBSLidSRXVPykBpSXkunKQUWg0jcFUNGxH9JAg6Med9jfP7stMq+97fhjBAiyyP oPkRp+dpSWU+8SlCtY4g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wbZgM-00000004dn8-0Sdv; Mon, 22 Jun 2026 08:07:02 +0000 Received: from canpmsgout06.his.huawei.com ([113.46.200.221]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wbZgB-00000004dS1-2bDi; Mon, 22 Jun 2026 08:06:54 +0000 dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=7l3MIp31Oqn6l35ANG2oeSNGQ5mPwFEIt+3BHRhEgFs=; b=JVcKeZfRvP8T8eSthBTDPagvkz4L0tUeCdqiAVGv4Eh3T3SVTQXrwmvOHMtG0cUUpo0xQ5/O4 aR6aIwZIftD9mfmnCd56hixc1MZIztHFios6RgBUF+IkCieMOqGm+NmJXzvSoKNNHpGsbDuVE/n scbJtcuZ3SQZ7BdsHKOvpH4= Received: from mail.maildlp.com (unknown [172.19.163.104]) by canpmsgout06.his.huawei.com (SkyGuard) with ESMTPS id 4gkLCQ6hV0zRhVP; Mon, 22 Jun 2026 15:57:38 +0800 (CST) Received: from dggpemf500011.china.huawei.com (unknown [7.185.36.131]) by mail.maildlp.com (Postfix) with ESMTPS id AB1BA40363; Mon, 22 Jun 2026 16:06:41 +0800 (CST) Received: from [10.67.109.254] (10.67.109.254) by dggpemf500011.china.huawei.com (7.185.36.131) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Mon, 22 Jun 2026 16:06:39 +0800 Message-ID: <2a34ff43-78a0-4d99-885a-0e7f560634ca@huawei.com> Date: Mon, 22 Jun 2026 16:06:38 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH RFC 3/3] arm64: Add HOTPLUG_PARALLEL support for secondary CPUs To: Will Deacon CC: Michael Kelley , "catalin.marinas@arm.com" , "tsbogend@alpha.franken.de" , "pjw@kernel.org" , "palmer@dabbelt.com" , "aou@eecs.berkeley.edu" , "alex@ghiti.fr" , "tglx@kernel.org" , "mingo@redhat.com" , "bp@alien8.de" , "dave.hansen@linux.intel.com" , "hpa@zytor.com" , "peterz@infradead.org" , "kees@kernel.org" , "nathan@kernel.org" , "linusw@kernel.org" , "ojeda@kernel.org" , "david.kaplan@amd.com" , "lukas.bulwahn@redhat.com" , "ryan.roberts@arm.com" , "maz@kernel.org" , "timothy.hayes@arm.com" , "lpieralisi@kernel.org" , "thuth@redhat.com" , "oupton@kernel.org" , "yeoreum.yun@arm.com" , "miko.lenczewski@arm.com" , "broonie@kernel.org" , "kevin.brodsky@arm.com" , "james.clark@linaro.org" , "tabba@google.com" , "mrigendra.chaubey@gmail.com" , "arnd@arndb.de" , "anshuman.khandual@arm.com" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-mips@vger.kernel.org" , "linux-riscv@lists.infradead.org" References: <20260611133809.3854977-1-ruanjinjie@huawei.com> <20260611133809.3854977-4-ruanjinjie@huawei.com> From: Jinjie Ruan In-Reply-To: X-Originating-IP: [10.67.109.254] X-ClientProxiedBy: kwepems200001.china.huawei.com (7.221.188.67) To dggpemf500011.china.huawei.com (7.185.36.131) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260622_010652_287017_C53210AA X-CRM114-Status: GOOD ( 26.34 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org CgpPbiA2LzE4LzIwMjYgODoyMSBQTSwgV2lsbCBEZWFjb24gd3JvdGU6Cj4gSGkgSmluamllLAo+ IAo+IE9uIE1vbiwgSnVuIDE1LCAyMDI2IGF0IDA0OjUxOjQ4UE0gKzA4MDAsIEppbmppZSBSdWFu IHdyb3RlOgo+PiBPbiA2LzEyLzIwMjYgMTE6NDUgUE0sIE1pY2hhZWwgS2VsbGV5IHdyb3RlOgo+ Pj4gRnJvbTogSmluamllIFJ1YW4gPHJ1YW5qaW5qaWVAaHVhd2VpLmNvbT4gU2VudDogVGh1cnNk YXksIEp1bmUgMTEsIDIwMjYgNjozOCBBTQo+Pj4+Cj4+Pj4gU3VwcG9ydCBmb3IgcGFyYWxsZWwg c2Vjb25kYXJ5IENQVSBicmluZ3VwIGlzIGFscmVhZHkgdXRpbGl6ZWQgYnkgeDg2LAo+Pj4+IE1J UFMsIGFuZCBSSVNDLVYuIFRoaXMgcGF0Y2ggYnJpbmdzIHRoaXMgY2FwYWJpbGl0eSB0byB0aGUg YXJtNjQKPj4+PiBhcmNoaXRlY3R1cmUuCj4+Pj4KPj4+PiBSZXdvcmsgdGhlIGdsb2JhbCBgc2Vj b25kYXJ5X2RhdGFgIGFjY2Vzc2VkIGR1cmluZyBlYXJseSBib290IGludG8KPj4+PiBhIHBlci1D UFUgYXJyYXkuIFRoaXMgYXJyYXkgbWFwcyBsb2dpY2FsIENQVSBJRHMgdG8gTVBJRFJfRUwxIHZh bHVlcywKPj4+PiBlbmFibGluZyB0aGUgZWFybHkgYm9vdCBjb2RlIGluIGhlYWQuUyB0byByZXNv bHZlIGVhY2ggc2Vjb25kYXJ5IENQVSdzCj4+Pj4gbG9naWNhbCBJRCBjb25jdXJyZW50bHkuCj4+ Pj4KPj4+PiBUbyBmdWxseSBlbmFibGUgSE9UUExVR19QQVJBTExFTCwgdGhpcyBwYXRjaCBpbXBs ZW1lbnRzOgo+Pj4+IDEpIEFuIGFybTY0LXNwZWNpZmljIGFyY2hfY3B1aHBfa2lja19hcF9hbGl2 ZSgpIGhhbmRsZXIuCj4+Pj4gMikgQ2FsbGJhY2tzIHRvIGNwdWhwX2FwX3N5bmNfYWxpdmUoKSBp bnNpZGUgc2Vjb25kYXJ5X3N0YXJ0X2tlcm5lbCgpLgo+Pj4+Cj4+Pj4gU3VjY2Vzc2Z1bGx5IHRl c3RlZCBvbiBRRU1VIEFSTTY0IHZpcnQgbWFjaGluZSAoS1ZNIG9uLCAxMjggdkNQVXMpLgo+Pj4+ Cj4+Pj4gfCAgICAgdGVzdCBrZXJuZWwJICAgfCBzZWNvbmRhcnkgQ1BVcyBib290IHRpbWUgfAo+ Pj4+IHwgIC0tLS0tLS0tLS0tLS0tLS0tLS0tLSAgIHwJLS0tLS0tLS0tLS0tLS0tLS0tLS0gIHwK Pj4+PiB8ICAgV2l0aG91dCB0aGlzIHBhdGNoICAgICB8CQkxNTUuNjcyCSAgICAgIHwKPj4+PiB8 ICAgY3B1aHAucGFyYWxsZWw9MAkgICB8CQk2Mi44OTcJICAgICAgfAo+Pj4+IHwgICBjcHVocC5w YXJhbGxlbD0xCSAgIHwJCTE2Ni43MDMJICAgICAgfAo+Pj4KPj4+IFRoZSBsYXN0IHR3byByb3dz IHNlZW0gbWl4ZWQgdXAuIEkgd291bGQgZXhwZWN0IHBhcmFsbGVsPTAgdG8KPj4+IHJlc3VsdCBp biBhIGxvbmdlciBib290IHRpbWUuCj4+Cj4+IEhpLCBNaWNoYWVsLAo+Pgo+PiBUaGUgcmVzdWx0 cyBhcmUgY29ycmVjdCBhbmQgbm90IG1peGVkIHVwLgo+Pgo+PiBDb21wYXJlZCB0byB0aGUgb3Jp Z2luYWwgbm9u4oCRSE9UUExVR19QQVJBTExFTCBhcHByb2FjaCwgdGhlIGFkdmFudGFnZSBvZgo+ PiBjcHVocC5wYXJhbGxlbD0wIGxpZXMgaW4gaXRzIHVzZSBvZiBjcHVfcmVsYXgoYHlpZWxkYCBv biBhcm02NCkgaW5zdGVhZAo+PiBvZiB0aGUgd2FpdF9mb3JfY29tcGxldGlvbl90aW1lb3V0KCkg bWVjaGFuaXNtICh3aGljaCBtYXkgY2F1c2Ugc2xlZXAKPj4gYW5kIGNvbnRleHQgc3dpdGNoaW5n KS4gVGhpcyBzaWduaWZpY2FudGx5IHJlZHVjZXMgdGhlIG92ZXJoZWFkIG9mIFZNCj4+IGV4aXRz IGFuZCBjb250ZXh0IHN3aXRjaGVzIGluIGEgS1ZNIGd1ZXN0LCB0aGVyZWJ5IGN1dHRpbmcgdGhl IHNlY29uZGFyeQo+PiBDUFUgYm9vdCB0aW1lIGJ5IG1vcmUgdGhhbiBoYWxmLgo+IAo+IEkgZG9u J3QgdGhpbmsgdGhhdCdzIGEgcGFydGljdWxhcmx5IGNvbXBlbGxpbmcgcmVhc29uIHRvIGVuYWJs ZSB0aGlzIGZvcgo+IGFybTY0LCBpbiBhbGwgaG9uZXN0eS4gVGhlIHlpZWxkIGluc3RydWN0aW9u IHR5cGljYWxseSBkb2Vzbid0IGRvCj4gYW55dGhpbmcgb24gYWN0dWFsIGFybTY0IHNpbGljb24s IHNvIHRoaXMgcHJvYmFibHkgbWVhbnMgdGhhdCB5b3UncmUKPiBpbnRyb2R1Y2luZyBidXN5LWxv b3BzIHdoaWNoIHRlbmQgdG8gYmUgYmFkIGZvciBwb3dlciBhbmQgc2NhbGFiaWxpdHkuCgpBZnRl ciB1cGRhdGluZyB0aGUgaW1wbGVtZW50YXRpb24gaW4gdjIsIHRoZSBwZXJmb3JtYW5jZSBnYWlu cyBhcmUKcHJpbWFyaWx5IG9ic2VydmVkIG9uIGFjdHVhbCBoYXJkd2FyZS4KCj4gCj4gSSBpbXBs ZW1lbnRlZCB0aGlzIGEgd2hpbGUgYWdvIFsxXSBidXQgZGlkbid0IG1hbmFnZSB0byBzZWUgbXVj aCBpbiB0ZXJtcwo+IG9mIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50IGFuZCBzbyBJIGRpZG4ndCBi b3RoZXIgdG8gc2VuZCB0aGUgcGF0Y2hlcyBvdXQKCkFzIHNob3duIGluIHYyIGJlbG93LCBvbiBh Y3R1YWwgaGFyZHdhcmUsIHRoaXMgcmVzdWx0cyBpbiBhIDQwJeKAkzYwJQpyZWR1Y3Rpb24gaW4g Ym9vdCB0aW1lLgoKQnJpbmd1cCBUaW1lIENvbXBhcmlzb24gKG1zLCBsb3dlciBpcyBiZXR0ZXIp OgoKfCAgICAgUGxhdGZvcm0JCXwgQmFzZWxpbmV8ICAgUD0wICAgfCAgIFA9MSAgfCBEZWx0YSgl KXwKfCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0gfCAtLS0tLS0tIHwgLS0tLS0tLSB8IC0tLS0tLSB8 IC0tLS0tLS0gfAp8IDY0LWNvcmUgQVRGIFFFTVUJfCAyMDc1LjggIHwgMjA4MC43ICB8IDE2NTMu NCB8IDIwLjM0JSAgfAp8IDE5Mi1jb3JlIHNlcnZlcihISVAxMil8IDE0NjE5LjIgfCAxNDYxOS4x IHwgODU4OS40IHwgNDEuMjElICB8CnwgMzItY29yZSBib2FyZAkgICAgICAgIHwgMjc3Ni41ICB8 IDI4ODEuMCAgfCAxMDQ1LjAgfCA2Mi4zNiUgIHwKCkxpbms6Cmh0dHBzOi8vbG9yZS5rZXJuZWwu b3JnL2FsbC8yMDI2MDYxODA5MjQ0NC4xMzE2MzM2LTUtcnVhbmppbmppZUBodWF3ZWkuY29tLwoK PiBhZnRlciB0YWxraW5nIGFib3V0IGl0IGF0IEtWTSBmb3J1bSBbMl0uIEhvd2V2ZXIsIGFzIG1l bnRpb25lZCBhdCB0aGUgZW5kCj4gb2YgdGhhdCB0YWxrLCBpdCBfaXNfIHN0aWxsIHVzZWZ1bCBm b3IgY29uZmlkZW50aWFsIFZNcyB1c2luZyBQU0NJIHNvCj4gbGV0IG1lIGR1c3Qgb2ZmIG15IG9s ZCBzZXJpZXMgYW5kIHNlbmQgaXQgb3V0IHRvIHNlZSB3aGF0IHlvdSB0aGluay4KPiAKPiBJdCBy ZWxpZXMgb24gUFNDSSB2MC4yLCB3aGljaCBtZWFucyB3ZSBkb24ndCBuZWVkIHRoZSBOUl9DUFVT IHNpemUgYXJyYXkKPiBmb3Igc2Vjb25kYXJ5X2RhdGEgYW5kIEkgYWxzbyBoYXZlIHNvbWUgc3Vw cG9ydCBmb3IgZXJyb3IgaGFuZGxpbmcgKGl0Cj4gZG9lc24ndCBsb29rIGxpa2UgeW91IGhhbmRs ZSBfX2Vhcmx5X2NwdV9ib290X3N0YXR1cyBwcm9wZXJseSkuCgpJIG5lZWQgc29tZSB0aW1lIHRv IGxvb2sgY2xvc2VseSBhdCB5b3VyIHBhdGNoLiBBbHRlcm5hdGl2ZWx5LCBJIHdpbGwKaW50ZWdy YXRlIHlvdXIgY2hhbmdlcywgcmUtdGVzdCBldmVyeXRoaW5nIG9uIGFjdHVhbCBoYXJkd2FyZSwg YW5kIHRoZW4Kc2VuZCBvdXQgYSByZXZpc2VkIHZlcnNpb24uCgo+IAo+IEl0IGxvb2tzIGxpa2Ug SSBjb3VsZCBpbmNsdWRlIHlvdXIgZmlyc3QgcGF0Y2gsIHRob3VnaCEKClRoYW5rIHlvdSB2ZXJ5 IG11Y2guCgo+IAo+IFdpbGwKPiAKPiBbMV0gaHR0cHM6Ly9naXQua2VybmVsLm9yZy9wdWIvc2Nt L2xpbnV4L2tlcm5lbC9naXQvd2lsbC9saW51eC5naXQvbG9nLz9oPWNwdS1ob3RwbHVnCgpJdCBz ZWVtcyB0aGF0IHRoZSBmb2xsb3dpbmcgcGF0Y2ggcmVtb3ZpbmcKYHJjdXRyZWVfcmVwb3J0X2Nw dV9zdGFydGluZygpYCB3aWxsIHJlaW50cm9kdWNlIHRoZSBvcmlnaW5hbCBpc3N1ZSBhcwpjb21t aXQgY2UzZDMxYWQzY2FjICgiYXJtNjQvc21wOiBNb3ZlCnJjdV9jcHVfc3RhcnRpbmcoKSBlYXJs aWVyIikgc29sb3ZlZC4KCkxpbms6Cmh0dHBzOi8vd2ViLmdpdC5rZXJuZWwub3JnL3B1Yi9zY20v bGludXgva2VybmVsL2dpdC93aWxsL2xpbnV4LmdpdC9jb21taXQvP2g9Y3B1LWhvdHBsdWcmaWQ9 YmJhNGI2MmY0NWYyNjE0YmY2MDg1ZTZjZDNmMjMzNTI4Zjg1YmYyNgoKSW5kZWVkLCBJIGFsc28g bm90aWNlZCB0aGF0IHRoZSBpbnZvY2F0aW9uIG9yZGVyIG9mCnJjdXRyZWVfcmVwb3J0X2NwdV9z dGFydGluZygpIG9uIGFybTY0IGlzIHNvbWV3aGF0IHN1Ym9wdGltYWwuIEl0CmhpbmRlcnMgdGhl IGltcGxlbWVudGF0aW9uIG9mIHBhcmFsbGVsIGJyaW5ndXAgb24gYXJtNjQgYW5kIGNvdWxkCnBv dGVudGlhbGx5IGxlYWQgdG8gUkNVIHN0YWxscy4KCkxpbms6Cmh0dHBzOi8vbG9yZS5rZXJuZWwu b3JnL2FsbC8yMDI2MDYxODA5MjQ0NC4xMzE2MzM2LTQtcnVhbmppbmppZUBodWF3ZWkuY29tLwoK WyAgICAwLjMyOTAxN10gc21wOiBCcmluZ2luZyB1cCBzZWNvbmRhcnkgQ1BVcyAuLi4KWyAgICAw LjM0MzYyOF0gRGV0ZWN0ZWQgVklQVCBJLWNhY2hlIG9uIENQVTEKWyAgICAwLjM0Mzc4OF0KWyAg ICAwLjM0MzgwNl0gPT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KWyAgICAwLjM0MzgxNl0g V0FSTklORzogc3VzcGljaW91cyBSQ1UgdXNhZ2UKWyAgICAwLjM0Mzk2Nl0gNy4xLjAtcmMxLWcy N2MxODcxODQ4YTIgIzEwOSBOb3QgdGFpbnRlZApbICAgIDAuMzQ0MDg3XSAtLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQpbICAgIDAuMzQ0MDk4XSBrZXJuZWwvbG9ja2luZy9sb2NrZGVwLmM6 MzgwMSBSQ1UtbGlzdCB0cmF2ZXJzZWQgaW4Kbm9uLXJlYWRlciBzZWN0aW9uISEKWyAgICAwLjM0 NDExMl0KWyAgICAwLjM0NDExMl0gb3RoZXIgaW5mbyB0aGF0IG1pZ2h0IGhlbHAgdXMgZGVidWcg dGhpczoKWyAgICAwLjM0NDExMl0KWyAgICAwLjM0NDEzNV0KWyAgICAwLjM0NDEzNV0gUkNVIHVz ZWQgaWxsZWdhbGx5IGZyb20gb2ZmbGluZSBDUFUhClsgICAgMC4zNDQxMzVdIHJjdV9zY2hlZHVs ZXJfYWN0aXZlID0gMSwgZGVidWdfbG9ja3MgPSAxClsgICAgMC4zNDQxNzRdIG5vIGxvY2tzIGhl bGQgYnkgc3dhcHBlci8xLzAuClsgICAgMC4zNDQyMDRdClsgICAgMC4zNDQyMDRdIHN0YWNrIGJh Y2t0cmFjZToKWyAgICAwLjM0NDYxMV0gQ1BVOiAxIFVJRDogMCBQSUQ6IDAgQ29tbTogc3dhcHBl ci8xIE5vdCB0YWludGVkCjcuMS4wLXJjMS1nMjdjMTg3MTg0OGEyICMxMDkgUFJFRU1QVApbICAg IDAuMzQ0NzA3XSBIYXJkd2FyZSBuYW1lOiBsaW51eCxkdW1teS12aXJ0IChEVCkKWyAgICAwLjM0 NTI2N10gQ2FsbCB0cmFjZToKWyAgICAwLjM0NTQzNl0gIHNob3dfc3RhY2srMHgxOC8weDI0IChD KQpbICAgIDAuMzQ1NTkzXSAgZHVtcF9zdGFja19sdmwrMHg5MC8weGQwClsgICAgMC4zNDU2MjBd ICBkdW1wX3N0YWNrKzB4MTgvMHgyNApbICAgIDAuMzQ1NjM5XSAgbG9ja2RlcF9yY3Vfc3VzcGlj aW91cysweDE3MC8weDIzNApbICAgIDAuMzQ1NjY1XSAgX19sb2NrX2FjcXVpcmUrMHhkZDQvMHgy MDc4ClsgICAgMC4zNDU2ODhdICBsb2NrX2FjcXVpcmUrMHgxYzQvMHgzZjAKWyAgICAwLjM0NTcx MV0gIF9yYXdfc3Bpbl9sb2NrX2lycXNhdmUrMHg2MC8weDg4ClsgICAgMC4zNDU3MzZdICBkb3du X3RyeWxvY2srMHgxOC8weDQ4ClsgICAgMC4zNDU3NThdICBfX2Rvd25fdHJ5bG9ja19jb25zb2xl X3NlbSsweDM4LzB4YzQKWyAgICAwLjM0NTc4Ml0gIHZwcmludGtfZW1pdCsweDIzYy8weDNkMApb ICAgIDAuMzQ1ODAyXSAgdnByaW50a19kZWZhdWx0KzB4MzgvMHg0NApbICAgIDAuMzQ1ODIyXSAg dnByaW50aysweDI4LzB4MzQKWyAgICAwLjM0NTg0MV0gIF9wcmludGsrMHg1Yy8weDg0ClsgICAg MC4zNDU4NjRdICBjcHVpbmZvX3N0b3JlX2NwdSsweDE3NC8weDI5OApbICAgIDAuMzQ1ODg0XSAg c2Vjb25kYXJ5X3N0YXJ0X2tlcm5lbCsweGJjLzB4MTUwClsgICAgMC4zNDU5MDVdICBfX3NlY29u ZGFyeV9zd2l0Y2hlZCsweGMwLzB4YzQKWyAgICAwLjM1MDMwN10gR0lDdjM6IENQVTE6IGZvdW5k IHJlZGlzdHJpYnV0b3IgMSByZWdpb24KMDoweDAwMDAwMDAwMDgwYzAwMDAKWyAgICAwLjM1MDUy M10gR0lDdjM6IENQVTE6IHVzaW5nIGFsbG9jYXRlZCBMUEkgcGVuZGluZyB0YWJsZQpAMHgwMDAw MDAwMTA0MmYwMDAwClsgICAgMC4zNTEzMDNdIENQVTE6IEJvb3RlZCBzZWNvbmRhcnkgcHJvY2Vz c29yIDB4MDAwMDAwMDAwMSBbMHg0MTBmZDAzNF0KWyAgICAwLjM4NzQyNV0gRGV0ZWN0ZWQgVklQ VCBJLWNhY2hlIG9uIENQVTIKCgo+IFsyXSBodHRwczovL3d3dy55b3V0dWJlLmNvbS93YXRjaD92 PVE2a09zaG5uUXVFCj4gCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJh ZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51 eC1yaXNjdgo=