From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ryan Roberts Subject: Re: [PATCH v1 00/10] variable-order, large folios for anonymous memory Date: Tue, 27 Jun 2023 10:59:22 +0100 Message-ID: <1fb0c4cb-a709-de20-d643-32ed43550059@arm.com> References: <20230626171430.3167004-1-ryan.roberts@arm.com> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: List-ID: Content-Type: text/plain; charset="utf-8" To: Yu Zhao Cc: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Geert Uytterhoeven , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-m68k@lists On 27/06/2023 08:49, Yu Zhao wrote: > On Mon, Jun 26, 2023 at 9:30 PM Yu Zhao wrote: >> >> On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts wrote: >>> >>> Hi All, >>> >>> Following on from the previous RFCv2 [1], this series implements variable order, >>> large folios for anonymous memory. The objective of this is to improve >>> performance by allocating larger chunks of memory during anonymous page faults: >>> >>> - Since SW (the kernel) is dealing with larger chunks of memory than base >>> pages, there are efficiency savings to be had; fewer page faults, batched PTE >>> and RMAP manipulation, fewer items on lists, etc. In short, we reduce kernel >>> overhead. This should benefit all architectures. >>> - Since we are now mapping physically contiguous chunks of memory, we can take >>> advantage of HW TLB compression techniques. A reduction in TLB pressure >>> speeds up kernel and user space. arm64 systems have 2 mechanisms to coalesce >>> TLB entries; "the contiguous bit" (architectural) and HPA (uarch). >>> >>> This patch set deals with the SW side of things only and based on feedback from >>> the RFC, aims to be the most minimal initial change, upon which future >>> incremental changes can be added. For this reason, the new behaviour is hidden >>> behind a new Kconfig switch, CONFIG_LARGE_ANON_FOLIO, which is disabled by >>> default. Although the code has been refactored to parameterize the desired order >>> of the allocation, when the feature is disabled (by forcing the order to be >>> always 0) my performance tests measure no regression. So I'm hoping this will be >>> a suitable mechanism to allow incremental submissions to the kernel without >>> affecting the rest of the world. >>> >>> The patches are based on top of v6.4 plus Matthew Wilcox's set_ptes() series >>> [2], which is a hard dependency. I'm not sure of Matthew's exact plans for >>> getting that series into the kernel, but I'm hoping we can start the review >>> process on this patch set independently. I have a branch at [3]. >>> >>> I've posted a separate series concerning the HW part (contpte mapping) for arm64 >>> at [4]. >>> >>> >>> Performance >>> ----------- >>> >>> Below results show 2 benchmarks; kernel compilation and speedometer 2.0 (a >>> javascript benchmark running in Chromium). Both cases are running on Ampere >>> Altra with 1 NUMA node enabled, Ubuntu 22.04 and XFS filesystem. Each benchmark >>> is repeated 15 times over 5 reboots and averaged. >>> >>> All improvements are relative to baseline-4k. 'anonfolio-basic' is this series. >>> 'anonfolio' is the full patch set similar to the RFC with the additional changes >>> to the extra 3 fault paths. The rest of the configs are described at [4]. >>> >>> Kernel Compilation (smaller is better): >>> >>> | kernel | real-time | kern-time | user-time | >>> |:----------------|------------:|------------:|------------:| >>> | baseline-4k | 0.0% | 0.0% | 0.0% | >>> | anonfolio-basic | -5.3% | -42.9% | -0.6% | >>> | anonfolio | -5.4% | -46.0% | -0.3% | >>> | contpte | -6.8% | -45.7% | -2.1% | >>> | exefolio | -8.4% | -46.4% | -3.7% | >>> | baseline-16k | -8.7% | -49.2% | -3.7% | >>> | baseline-64k | -10.5% | -66.0% | -3.5% | >>> >>> Speedometer 2.0 (bigger is better): >>> >>> | kernel | runs_per_min | >>> |:----------------|---------------:| >>> | baseline-4k | 0.0% | >>> | anonfolio-basic | 0.7% | >>> | anonfolio | 1.2% | >>> | contpte | 3.1% | >>> | exefolio | 4.2% | >>> | baseline-16k | 5.3% | >> >> Thanks for pushing this forward! >> >>> Changes since RFCv2 >>> ------------------- >>> >>> - Simplified series to bare minimum (on David Hildenbrand's advice) >> >> My impression is that this series still includes many pieces that can >> be split out and discussed separately with followup series. >> >> (I skipped 04/10 and will look at it tomorrow.) > > I went through the series twice. Here what I think a bare minimum > series (easier to review/debug/land) would look like: > 1. a new arch specific function providing a prefered order within (0, > PMD_ORDER). > 2. an extended anon folio alloc API taking that order (02/10, partially). > 3. an updated folio_add_new_anon_rmap() covering the large() && > !pmd_mappable() case (similar to 04/10). > 4. s/folio_test_pmd_mappable/folio_test_large/ in page_remove_rmap() > (06/10, reviewed-by provided). > 5. finally, use the extended anon folio alloc API with the arch > preferred order in do_anonymous_page() (10/10, partially). > > The rest can be split out into separate series and move forward in > parallel with probably a long list of things we need/want to do. Thanks for the fadt review - I really appreciate it! I've responded to many of your comments. I'd appreciate if we can close those points then I will work up a v2. Thanks, Ryan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A9A5C0015E for ; Tue, 27 Jun 2023 10:03:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232018AbjF0KDP (ORCPT ); Tue, 27 Jun 2023 06:03:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232022AbjF0KC4 (ORCPT ); Tue, 27 Jun 2023 06:02:56 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BD2972D63; Tue, 27 Jun 2023 02:59:33 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 756CC2F4; Tue, 27 Jun 2023 03:00:10 -0700 (PDT) Received: from [10.1.30.74] (C02Z41KALVDN.cambridge.arm.com [10.1.30.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A99B83F64C; Tue, 27 Jun 2023 02:59:23 -0700 (PDT) Message-ID: <1fb0c4cb-a709-de20-d643-32ed43550059@arm.com> Date: Tue, 27 Jun 2023 10:59:22 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v1 00/10] variable-order, large folios for anonymous memory To: Yu Zhao Cc: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Geert Uytterhoeven , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org References: <20230626171430.3167004-1-ryan.roberts@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-ia64@vger.kernel.org On 27/06/2023 08:49, Yu Zhao wrote: > On Mon, Jun 26, 2023 at 9:30 PM Yu Zhao wrote: >> >> On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts wrote: >>> >>> Hi All, >>> >>> Following on from the previous RFCv2 [1], this series implements variable order, >>> large folios for anonymous memory. The objective of this is to improve >>> performance by allocating larger chunks of memory during anonymous page faults: >>> >>> - Since SW (the kernel) is dealing with larger chunks of memory than base >>> pages, there are efficiency savings to be had; fewer page faults, batched PTE >>> and RMAP manipulation, fewer items on lists, etc. In short, we reduce kernel >>> overhead. This should benefit all architectures. >>> - Since we are now mapping physically contiguous chunks of memory, we can take >>> advantage of HW TLB compression techniques. A reduction in TLB pressure >>> speeds up kernel and user space. arm64 systems have 2 mechanisms to coalesce >>> TLB entries; "the contiguous bit" (architectural) and HPA (uarch). >>> >>> This patch set deals with the SW side of things only and based on feedback from >>> the RFC, aims to be the most minimal initial change, upon which future >>> incremental changes can be added. For this reason, the new behaviour is hidden >>> behind a new Kconfig switch, CONFIG_LARGE_ANON_FOLIO, which is disabled by >>> default. Although the code has been refactored to parameterize the desired order >>> of the allocation, when the feature is disabled (by forcing the order to be >>> always 0) my performance tests measure no regression. So I'm hoping this will be >>> a suitable mechanism to allow incremental submissions to the kernel without >>> affecting the rest of the world. >>> >>> The patches are based on top of v6.4 plus Matthew Wilcox's set_ptes() series >>> [2], which is a hard dependency. I'm not sure of Matthew's exact plans for >>> getting that series into the kernel, but I'm hoping we can start the review >>> process on this patch set independently. I have a branch at [3]. >>> >>> I've posted a separate series concerning the HW part (contpte mapping) for arm64 >>> at [4]. >>> >>> >>> Performance >>> ----------- >>> >>> Below results show 2 benchmarks; kernel compilation and speedometer 2.0 (a >>> javascript benchmark running in Chromium). Both cases are running on Ampere >>> Altra with 1 NUMA node enabled, Ubuntu 22.04 and XFS filesystem. Each benchmark >>> is repeated 15 times over 5 reboots and averaged. >>> >>> All improvements are relative to baseline-4k. 'anonfolio-basic' is this series. >>> 'anonfolio' is the full patch set similar to the RFC with the additional changes >>> to the extra 3 fault paths. The rest of the configs are described at [4]. >>> >>> Kernel Compilation (smaller is better): >>> >>> | kernel | real-time | kern-time | user-time | >>> |:----------------|------------:|------------:|------------:| >>> | baseline-4k | 0.0% | 0.0% | 0.0% | >>> | anonfolio-basic | -5.3% | -42.9% | -0.6% | >>> | anonfolio | -5.4% | -46.0% | -0.3% | >>> | contpte | -6.8% | -45.7% | -2.1% | >>> | exefolio | -8.4% | -46.4% | -3.7% | >>> | baseline-16k | -8.7% | -49.2% | -3.7% | >>> | baseline-64k | -10.5% | -66.0% | -3.5% | >>> >>> Speedometer 2.0 (bigger is better): >>> >>> | kernel | runs_per_min | >>> |:----------------|---------------:| >>> | baseline-4k | 0.0% | >>> | anonfolio-basic | 0.7% | >>> | anonfolio | 1.2% | >>> | contpte | 3.1% | >>> | exefolio | 4.2% | >>> | baseline-16k | 5.3% | >> >> Thanks for pushing this forward! >> >>> Changes since RFCv2 >>> ------------------- >>> >>> - Simplified series to bare minimum (on David Hildenbrand's advice) >> >> My impression is that this series still includes many pieces that can >> be split out and discussed separately with followup series. >> >> (I skipped 04/10 and will look at it tomorrow.) > > I went through the series twice. Here what I think a bare minimum > series (easier to review/debug/land) would look like: > 1. a new arch specific function providing a prefered order within (0, > PMD_ORDER). > 2. an extended anon folio alloc API taking that order (02/10, partially). > 3. an updated folio_add_new_anon_rmap() covering the large() && > !pmd_mappable() case (similar to 04/10). > 4. s/folio_test_pmd_mappable/folio_test_large/ in page_remove_rmap() > (06/10, reviewed-by provided). > 5. finally, use the extended anon folio alloc API with the arch > preferred order in do_anonymous_page() (10/10, partially). > > The rest can be split out into separate series and move forward in > parallel with probably a long list of things we need/want to do. Thanks for the fadt review - I really appreciate it! I've responded to many of your comments. I'd appreciate if we can close those points then I will work up a v2. Thanks, Ryan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B71D9EB64DC for ; Tue, 27 Jun 2023 09:59:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JhfFZo8d7vIcBbGN7InGu4slByFoKkIXxFwUkfHrH2M=; b=r6RiszWoKo4gcW KgOqTwos07ld2dP8QbGMnvNHO0fVigIKdI1Wu4Dks7Jc/9Q1Adr6apTkOwTfJhlNZWiTM5ANLvfad z6gOV1xY46mEa2WP/EbCiIGRDAKZHigWvPHrvGpbNU1K/c7j+lP4YzbYc9PmYJqNzqa4ehpk3JcZy ZtjklgPIHprHYfq5bdgjnODd/fZdCi00eSBZ2m9sEOyoEgR4HxjQu6nMeAfZbrBdisZRhDvfQMQh8 gWBfSzfa9RWouaCnzveE1ZDenQMDnVmRVMfixVD+ZtBI+3AkucEg5bXu7syB93jeiSUA1eBejoP6V BpoADoU3utzgUkqM9dDQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qE5Tu-00Cl4w-30; Tue, 27 Jun 2023 09:59:30 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qE5Tr-00Cl3a-1m for linux-arm-kernel@lists.infradead.org; Tue, 27 Jun 2023 09:59:29 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 756CC2F4; Tue, 27 Jun 2023 03:00:10 -0700 (PDT) Received: from [10.1.30.74] (C02Z41KALVDN.cambridge.arm.com [10.1.30.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A99B83F64C; Tue, 27 Jun 2023 02:59:23 -0700 (PDT) Message-ID: <1fb0c4cb-a709-de20-d643-32ed43550059@arm.com> Date: Tue, 27 Jun 2023 10:59:22 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v1 00/10] variable-order, large folios for anonymous memory To: Yu Zhao Cc: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Geert Uytterhoeven , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org References: <20230626171430.3167004-1-ryan.roberts@arm.com> From: Ryan Roberts In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230627_025927_682534_323924FA X-CRM114-Status: GOOD ( 26.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMjcvMDYvMjAyMyAwODo0OSwgWXUgWmhhbyB3cm90ZToKPiBPbiBNb24sIEp1biAyNiwgMjAy MyBhdCA5OjMw4oCvUE0gWXUgWmhhbyA8eXV6aGFvQGdvb2dsZS5jb20+IHdyb3RlOgo+Pgo+PiBP biBNb24sIEp1biAyNiwgMjAyMyBhdCAxMToxNOKAr0FNIFJ5YW4gUm9iZXJ0cyA8cnlhbi5yb2Jl cnRzQGFybS5jb20+IHdyb3RlOgo+Pj4KPj4+IEhpIEFsbCwKPj4+Cj4+PiBGb2xsb3dpbmcgb24g ZnJvbSB0aGUgcHJldmlvdXMgUkZDdjIgWzFdLCB0aGlzIHNlcmllcyBpbXBsZW1lbnRzIHZhcmlh YmxlIG9yZGVyLAo+Pj4gbGFyZ2UgZm9saW9zIGZvciBhbm9ueW1vdXMgbWVtb3J5LiBUaGUgb2Jq ZWN0aXZlIG9mIHRoaXMgaXMgdG8gaW1wcm92ZQo+Pj4gcGVyZm9ybWFuY2UgYnkgYWxsb2NhdGlu ZyBsYXJnZXIgY2h1bmtzIG9mIG1lbW9yeSBkdXJpbmcgYW5vbnltb3VzIHBhZ2UgZmF1bHRzOgo+ Pj4KPj4+ICAtIFNpbmNlIFNXICh0aGUga2VybmVsKSBpcyBkZWFsaW5nIHdpdGggbGFyZ2VyIGNo dW5rcyBvZiBtZW1vcnkgdGhhbiBiYXNlCj4+PiAgICBwYWdlcywgdGhlcmUgYXJlIGVmZmljaWVu Y3kgc2F2aW5ncyB0byBiZSBoYWQ7IGZld2VyIHBhZ2UgZmF1bHRzLCBiYXRjaGVkIFBURQo+Pj4g ICAgYW5kIFJNQVAgbWFuaXB1bGF0aW9uLCBmZXdlciBpdGVtcyBvbiBsaXN0cywgZXRjLiBJbiBz aG9ydCwgd2UgcmVkdWNlIGtlcm5lbAo+Pj4gICAgb3ZlcmhlYWQuIFRoaXMgc2hvdWxkIGJlbmVm aXQgYWxsIGFyY2hpdGVjdHVyZXMuCj4+PiAgLSBTaW5jZSB3ZSBhcmUgbm93IG1hcHBpbmcgcGh5 c2ljYWxseSBjb250aWd1b3VzIGNodW5rcyBvZiBtZW1vcnksIHdlIGNhbiB0YWtlCj4+PiAgICBh ZHZhbnRhZ2Ugb2YgSFcgVExCIGNvbXByZXNzaW9uIHRlY2huaXF1ZXMuIEEgcmVkdWN0aW9uIGlu IFRMQiBwcmVzc3VyZQo+Pj4gICAgc3BlZWRzIHVwIGtlcm5lbCBhbmQgdXNlciBzcGFjZS4gYXJt NjQgc3lzdGVtcyBoYXZlIDIgbWVjaGFuaXNtcyB0byBjb2FsZXNjZQo+Pj4gICAgVExCIGVudHJp ZXM7ICJ0aGUgY29udGlndW91cyBiaXQiIChhcmNoaXRlY3R1cmFsKSBhbmQgSFBBICh1YXJjaCku Cj4+Pgo+Pj4gVGhpcyBwYXRjaCBzZXQgZGVhbHMgd2l0aCB0aGUgU1cgc2lkZSBvZiB0aGluZ3Mg b25seSBhbmQgYmFzZWQgb24gZmVlZGJhY2sgZnJvbQo+Pj4gdGhlIFJGQywgYWltcyB0byBiZSB0 aGUgbW9zdCBtaW5pbWFsIGluaXRpYWwgY2hhbmdlLCB1cG9uIHdoaWNoIGZ1dHVyZQo+Pj4gaW5j cmVtZW50YWwgY2hhbmdlcyBjYW4gYmUgYWRkZWQuIEZvciB0aGlzIHJlYXNvbiwgdGhlIG5ldyBi ZWhhdmlvdXIgaXMgaGlkZGVuCj4+PiBiZWhpbmQgYSBuZXcgS2NvbmZpZyBzd2l0Y2gsIENPTkZJ R19MQVJHRV9BTk9OX0ZPTElPLCB3aGljaCBpcyBkaXNhYmxlZCBieQo+Pj4gZGVmYXVsdC4gQWx0 aG91Z2ggdGhlIGNvZGUgaGFzIGJlZW4gcmVmYWN0b3JlZCB0byBwYXJhbWV0ZXJpemUgdGhlIGRl c2lyZWQgb3JkZXIKPj4+IG9mIHRoZSBhbGxvY2F0aW9uLCB3aGVuIHRoZSBmZWF0dXJlIGlzIGRp c2FibGVkIChieSBmb3JjaW5nIHRoZSBvcmRlciB0byBiZQo+Pj4gYWx3YXlzIDApIG15IHBlcmZv cm1hbmNlIHRlc3RzIG1lYXN1cmUgbm8gcmVncmVzc2lvbi4gU28gSSdtIGhvcGluZyB0aGlzIHdp bGwgYmUKPj4+IGEgc3VpdGFibGUgbWVjaGFuaXNtIHRvIGFsbG93IGluY3JlbWVudGFsIHN1Ym1p c3Npb25zIHRvIHRoZSBrZXJuZWwgd2l0aG91dAo+Pj4gYWZmZWN0aW5nIHRoZSByZXN0IG9mIHRo ZSB3b3JsZC4KPj4+Cj4+PiBUaGUgcGF0Y2hlcyBhcmUgYmFzZWQgb24gdG9wIG9mIHY2LjQgcGx1 cyBNYXR0aGV3IFdpbGNveCdzIHNldF9wdGVzKCkgc2VyaWVzCj4+PiBbMl0sIHdoaWNoIGlzIGEg aGFyZCBkZXBlbmRlbmN5LiBJJ20gbm90IHN1cmUgb2YgTWF0dGhldydzIGV4YWN0IHBsYW5zIGZv cgo+Pj4gZ2V0dGluZyB0aGF0IHNlcmllcyBpbnRvIHRoZSBrZXJuZWwsIGJ1dCBJJ20gaG9waW5n IHdlIGNhbiBzdGFydCB0aGUgcmV2aWV3Cj4+PiBwcm9jZXNzIG9uIHRoaXMgcGF0Y2ggc2V0IGlu ZGVwZW5kZW50bHkuIEkgaGF2ZSBhIGJyYW5jaCBhdCBbM10uCj4+Pgo+Pj4gSSd2ZSBwb3N0ZWQg YSBzZXBhcmF0ZSBzZXJpZXMgY29uY2VybmluZyB0aGUgSFcgcGFydCAoY29udHB0ZSBtYXBwaW5n KSBmb3IgYXJtNjQKPj4+IGF0IFs0XS4KPj4+Cj4+Pgo+Pj4gUGVyZm9ybWFuY2UKPj4+IC0tLS0t LS0tLS0tCj4+Pgo+Pj4gQmVsb3cgcmVzdWx0cyBzaG93IDIgYmVuY2htYXJrczsga2VybmVsIGNv bXBpbGF0aW9uIGFuZCBzcGVlZG9tZXRlciAyLjAgKGEKPj4+IGphdmFzY3JpcHQgYmVuY2htYXJr IHJ1bm5pbmcgaW4gQ2hyb21pdW0pLiBCb3RoIGNhc2VzIGFyZSBydW5uaW5nIG9uIEFtcGVyZQo+ Pj4gQWx0cmEgd2l0aCAxIE5VTUEgbm9kZSBlbmFibGVkLCBVYnVudHUgMjIuMDQgYW5kIFhGUyBm aWxlc3lzdGVtLiBFYWNoIGJlbmNobWFyawo+Pj4gaXMgcmVwZWF0ZWQgMTUgdGltZXMgb3ZlciA1 IHJlYm9vdHMgYW5kIGF2ZXJhZ2VkLgo+Pj4KPj4+IEFsbCBpbXByb3ZlbWVudHMgYXJlIHJlbGF0 aXZlIHRvIGJhc2VsaW5lLTRrLiAnYW5vbmZvbGlvLWJhc2ljJyBpcyB0aGlzIHNlcmllcy4KPj4+ ICdhbm9uZm9saW8nIGlzIHRoZSBmdWxsIHBhdGNoIHNldCBzaW1pbGFyIHRvIHRoZSBSRkMgd2l0 aCB0aGUgYWRkaXRpb25hbCBjaGFuZ2VzCj4+PiB0byB0aGUgZXh0cmEgMyBmYXVsdCBwYXRocy4g VGhlIHJlc3Qgb2YgdGhlIGNvbmZpZ3MgYXJlIGRlc2NyaWJlZCBhdCBbNF0uCj4+Pgo+Pj4gS2Vy bmVsIENvbXBpbGF0aW9uIChzbWFsbGVyIGlzIGJldHRlcik6Cj4+Pgo+Pj4gfCBrZXJuZWwgICAg ICAgICAgfCAgIHJlYWwtdGltZSB8ICAga2Vybi10aW1lIHwgICB1c2VyLXRpbWUgfAo+Pj4gfDot LS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLTp8LS0tLS0tLS0tLS0tOnwtLS0tLS0tLS0tLS06 fAo+Pj4gfCBiYXNlbGluZS00ayAgICAgfCAgICAgICAgMC4wJSB8ICAgICAgICAwLjAlIHwgICAg ICAgIDAuMCUgfAo+Pj4gfCBhbm9uZm9saW8tYmFzaWMgfCAgICAgICAtNS4zJSB8ICAgICAgLTQy LjklIHwgICAgICAgLTAuNiUgfAo+Pj4gfCBhbm9uZm9saW8gICAgICAgfCAgICAgICAtNS40JSB8 ICAgICAgLTQ2LjAlIHwgICAgICAgLTAuMyUgfAo+Pj4gfCBjb250cHRlICAgICAgICAgfCAgICAg ICAtNi44JSB8ICAgICAgLTQ1LjclIHwgICAgICAgLTIuMSUgfAo+Pj4gfCBleGVmb2xpbyAgICAg ICAgfCAgICAgICAtOC40JSB8ICAgICAgLTQ2LjQlIHwgICAgICAgLTMuNyUgfAo+Pj4gfCBiYXNl bGluZS0xNmsgICAgfCAgICAgICAtOC43JSB8ICAgICAgLTQ5LjIlIHwgICAgICAgLTMuNyUgfAo+ Pj4gfCBiYXNlbGluZS02NGsgICAgfCAgICAgIC0xMC41JSB8ICAgICAgLTY2LjAlIHwgICAgICAg LTMuNSUgfAo+Pj4KPj4+IFNwZWVkb21ldGVyIDIuMCAoYmlnZ2VyIGlzIGJldHRlcik6Cj4+Pgo+ Pj4gfCBrZXJuZWwgICAgICAgICAgfCAgIHJ1bnNfcGVyX21pbiB8Cj4+PiB8Oi0tLS0tLS0tLS0t LS0tLS18LS0tLS0tLS0tLS0tLS0tOnwKPj4+IHwgYmFzZWxpbmUtNGsgICAgIHwgICAgICAgICAg IDAuMCUgfAo+Pj4gfCBhbm9uZm9saW8tYmFzaWMgfCAgICAgICAgICAgMC43JSB8Cj4+PiB8IGFu b25mb2xpbyAgICAgICB8ICAgICAgICAgICAxLjIlIHwKPj4+IHwgY29udHB0ZSAgICAgICAgIHwg ICAgICAgICAgIDMuMSUgfAo+Pj4gfCBleGVmb2xpbyAgICAgICAgfCAgICAgICAgICAgNC4yJSB8 Cj4+PiB8IGJhc2VsaW5lLTE2ayAgICB8ICAgICAgICAgICA1LjMlIHwKPj4KPj4gVGhhbmtzIGZv ciBwdXNoaW5nIHRoaXMgZm9yd2FyZCEKPj4KPj4+IENoYW5nZXMgc2luY2UgUkZDdjIKPj4+IC0t LS0tLS0tLS0tLS0tLS0tLS0KPj4+Cj4+PiAgIC0gU2ltcGxpZmllZCBzZXJpZXMgdG8gYmFyZSBt aW5pbXVtIChvbiBEYXZpZCBIaWxkZW5icmFuZCdzIGFkdmljZSkKPj4KPj4gTXkgaW1wcmVzc2lv biBpcyB0aGF0IHRoaXMgc2VyaWVzIHN0aWxsIGluY2x1ZGVzIG1hbnkgcGllY2VzIHRoYXQgY2Fu Cj4+IGJlIHNwbGl0IG91dCBhbmQgZGlzY3Vzc2VkIHNlcGFyYXRlbHkgd2l0aCBmb2xsb3d1cCBz ZXJpZXMuCj4+Cj4+IChJIHNraXBwZWQgMDQvMTAgYW5kIHdpbGwgbG9vayBhdCBpdCB0b21vcnJv dy4pCj4gCj4gSSB3ZW50IHRocm91Z2ggdGhlIHNlcmllcyB0d2ljZS4gSGVyZSB3aGF0IEkgdGhp bmsgYSBiYXJlIG1pbmltdW0KPiBzZXJpZXMgKGVhc2llciB0byByZXZpZXcvZGVidWcvbGFuZCkg d291bGQgbG9vayBsaWtlOgo+IDEuIGEgbmV3IGFyY2ggc3BlY2lmaWMgZnVuY3Rpb24gcHJvdmlk aW5nIGEgcHJlZmVyZWQgb3JkZXIgd2l0aGluICgwLAo+IFBNRF9PUkRFUikuCj4gMi4gYW4gZXh0 ZW5kZWQgYW5vbiBmb2xpbyBhbGxvYyBBUEkgdGFraW5nIHRoYXQgb3JkZXIgKDAyLzEwLCBwYXJ0 aWFsbHkpLgo+IDMuIGFuIHVwZGF0ZWQgZm9saW9fYWRkX25ld19hbm9uX3JtYXAoKSBjb3Zlcmlu ZyB0aGUgbGFyZ2UoKSAmJgo+ICFwbWRfbWFwcGFibGUoKSBjYXNlIChzaW1pbGFyIHRvIDA0LzEw KS4KPiA0LiBzL2ZvbGlvX3Rlc3RfcG1kX21hcHBhYmxlL2ZvbGlvX3Rlc3RfbGFyZ2UvIGluIHBh Z2VfcmVtb3ZlX3JtYXAoKQo+ICgwNi8xMCwgcmV2aWV3ZWQtYnkgcHJvdmlkZWQpLgo+IDUuIGZp bmFsbHksIHVzZSB0aGUgZXh0ZW5kZWQgYW5vbiBmb2xpbyBhbGxvYyBBUEkgd2l0aCB0aGUgYXJj aAo+IHByZWZlcnJlZCBvcmRlciBpbiBkb19hbm9ueW1vdXNfcGFnZSgpICgxMC8xMCwgcGFydGlh bGx5KS4KPiAKPiBUaGUgcmVzdCBjYW4gYmUgc3BsaXQgb3V0IGludG8gc2VwYXJhdGUgc2VyaWVz IGFuZCBtb3ZlIGZvcndhcmQgaW4KPiBwYXJhbGxlbCB3aXRoIHByb2JhYmx5IGEgbG9uZyBsaXN0 IG9mIHRoaW5ncyB3ZSBuZWVkL3dhbnQgdG8gZG8uCgpUaGFua3MgZm9yIHRoZSBmYWR0IHJldmll dyAtIEkgcmVhbGx5IGFwcHJlY2lhdGUgaXQhCgpJJ3ZlIHJlc3BvbmRlZCB0byBtYW55IG9mIHlv dXIgY29tbWVudHMuIEknZCBhcHByZWNpYXRlIGlmIHdlIGNhbiBjbG9zZSB0aG9zZQpwb2ludHMg dGhlbiBJIHdpbGwgd29yayB1cCBhIHYyLgoKVGhhbmtzLApSeWFuCgoKCl9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGlu ZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMu aW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK