From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ryan Roberts Subject: Re: [PATCH v1 00/10] variable-order, large folios for anonymous memory Date: Thu, 29 Jun 2023 16:28:16 +0100 Message-ID: <7ed253e7-7245-32bc-e5ac-6d037b711b49@arm.com> References: <20230626171430.3167004-1-ryan.roberts@arm.com> <1fb0c4cb-a709-de20-d643-32ed43550059@arm.com> Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: List-ID: Content-Type: text/plain; charset="utf-8" To: Yu Zhao , Yin Fengwei Cc: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , David Hildenbrand , Catalin Marinas , Will Deacon , Geert Uytterhoeven , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org On 28/06/2023 19:22, Yu Zhao wrote: > On Tue, Jun 27, 2023 at 3:59 AM Ryan Roberts wrote: >> >> On 27/06/2023 08:49, Yu Zhao wrote: >>> On Mon, Jun 26, 2023 at 9:30 PM Yu Zhao wrote: >>>> >>>> On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts wrote: >>>>> >>>>> Hi All, >>>>> >>>>> Following on from the previous RFCv2 [1], this series implements variable order, >>>>> large folios for anonymous memory. The objective of this is to improve >>>>> performance by allocating larger chunks of memory during anonymous page faults: >>>>> >>>>> - Since SW (the kernel) is dealing with larger chunks of memory than base >>>>> pages, there are efficiency savings to be had; fewer page faults, batched PTE >>>>> and RMAP manipulation, fewer items on lists, etc. In short, we reduce kernel >>>>> overhead. This should benefit all architectures. >>>>> - Since we are now mapping physically contiguous chunks of memory, we can take >>>>> advantage of HW TLB compression techniques. A reduction in TLB pressure >>>>> speeds up kernel and user space. arm64 systems have 2 mechanisms to coalesce >>>>> TLB entries; "the contiguous bit" (architectural) and HPA (uarch). >>>>> >>>>> This patch set deals with the SW side of things only and based on feedback from >>>>> the RFC, aims to be the most minimal initial change, upon which future >>>>> incremental changes can be added. For this reason, the new behaviour is hidden >>>>> behind a new Kconfig switch, CONFIG_LARGE_ANON_FOLIO, which is disabled by >>>>> default. Although the code has been refactored to parameterize the desired order >>>>> of the allocation, when the feature is disabled (by forcing the order to be >>>>> always 0) my performance tests measure no regression. So I'm hoping this will be >>>>> a suitable mechanism to allow incremental submissions to the kernel without >>>>> affecting the rest of the world. >>>>> >>>>> The patches are based on top of v6.4 plus Matthew Wilcox's set_ptes() series >>>>> [2], which is a hard dependency. I'm not sure of Matthew's exact plans for >>>>> getting that series into the kernel, but I'm hoping we can start the review >>>>> process on this patch set independently. I have a branch at [3]. >>>>> >>>>> I've posted a separate series concerning the HW part (contpte mapping) for arm64 >>>>> at [4]. >>>>> >>>>> >>>>> Performance >>>>> ----------- >>>>> >>>>> Below results show 2 benchmarks; kernel compilation and speedometer 2.0 (a >>>>> javascript benchmark running in Chromium). Both cases are running on Ampere >>>>> Altra with 1 NUMA node enabled, Ubuntu 22.04 and XFS filesystem. Each benchmark >>>>> is repeated 15 times over 5 reboots and averaged. >>>>> >>>>> All improvements are relative to baseline-4k. 'anonfolio-basic' is this series. >>>>> 'anonfolio' is the full patch set similar to the RFC with the additional changes >>>>> to the extra 3 fault paths. The rest of the configs are described at [4]. >>>>> >>>>> Kernel Compilation (smaller is better): >>>>> >>>>> | kernel | real-time | kern-time | user-time | >>>>> |:----------------|------------:|------------:|------------:| >>>>> | baseline-4k | 0.0% | 0.0% | 0.0% | >>>>> | anonfolio-basic | -5.3% | -42.9% | -0.6% | >>>>> | anonfolio | -5.4% | -46.0% | -0.3% | >>>>> | contpte | -6.8% | -45.7% | -2.1% | >>>>> | exefolio | -8.4% | -46.4% | -3.7% | >>>>> | baseline-16k | -8.7% | -49.2% | -3.7% | >>>>> | baseline-64k | -10.5% | -66.0% | -3.5% | >>>>> >>>>> Speedometer 2.0 (bigger is better): >>>>> >>>>> | kernel | runs_per_min | >>>>> |:----------------|---------------:| >>>>> | baseline-4k | 0.0% | >>>>> | anonfolio-basic | 0.7% | >>>>> | anonfolio | 1.2% | >>>>> | contpte | 3.1% | >>>>> | exefolio | 4.2% | >>>>> | baseline-16k | 5.3% | >>>> >>>> Thanks for pushing this forward! >>>> >>>>> Changes since RFCv2 >>>>> ------------------- >>>>> >>>>> - Simplified series to bare minimum (on David Hildenbrand's advice) >>>> >>>> My impression is that this series still includes many pieces that can >>>> be split out and discussed separately with followup series. >>>> >>>> (I skipped 04/10 and will look at it tomorrow.) >>> >>> I went through the series twice. Here what I think a bare minimum >>> series (easier to review/debug/land) would look like: > > === > >>> 1. a new arch specific function providing a prefered order within (0, >>> PMD_ORDER). >>> 2. an extended anon folio alloc API taking that order (02/10, partially). >>> 3. an updated folio_add_new_anon_rmap() covering the large() && >>> !pmd_mappable() case (similar to 04/10). >>> 4. s/folio_test_pmd_mappable/folio_test_large/ in page_remove_rmap() >>> (06/10, reviewed-by provided). >>> 5. finally, use the extended anon folio alloc API with the arch >>> preferred order in do_anonymous_page() (10/10, partially). > > === > >>> The rest can be split out into separate series and move forward in >>> parallel with probably a long list of things we need/want to do. >> >> Thanks for the fadt review - I really appreciate it! >> >> I've responded to many of your comments. I'd appreciate if we can close those >> points then I will work up a v2. > > Thanks! > > Based on the latest discussion here [1], my original list above can be > optionally reduced to 4 patches: item 2 can be quashed into item 5. > > Also please make sure we have only one global (apply to all archs) > Kconfig option, and it should be added in item 5: > > if TRANSPARENT_HUGEPAGE > config FLEXIBLE/VARIABLE_THP # or whatever name you see fit > end if Naming is always the hardest part. I've been calling it LARGE_ANON_FOLIO up until now. But I think you are right that we should show that it is related to THP, so I'll go with FLEXIBLE_THP for v2, and let people shout if they hate it. If we are not letting the arch declare that it supports FLEXIBLE_THP, then I think we need the default version of arch_wants_pte_order() to return a value higher than 0 (which is what I have it returning at the moment). Because otherwise, for an arch that hasn't defined its own version of arch_wants_pte_order(), FLEXIBLE_THP on vs off will give the same result. So I propose to set the default to ilog2(SZ_64K >> PAGE_SHIFT). Shout if you have any concerns. > > (How many new Kconfig options added within arch/arm64/ is not a concern of MM.) > > And please make sure it's disabled by default, Done because we are still > missing many important functions, e.g., I don't think we can mlock() > when large() && !pmd_mappable(), see mlock_pte_range() and > mlock_vma_folio(). We can fix it along with many things later, but we > need to present a plan and a schedule now. Otherwise, there would be > pushback if we try to land the series without supporting mlock(). There are other areas that I'm aware off. I'll put together a table and send it out once I have v2 out the door (hopefully tomorrow or Monday). Hopefully we can work together to fill it in and figure out who can do what? I'm certainly planning to continue to push this work forwards beyond this initial patch set. Thanks, Ryan > > Do you or Fengwei plan to take on it? (I personally don't.) If not, > I'll try to find someone from our team to look at it. (It'd be more > scalable if we have a coordinated group of people individually solving > different problems.) > > [1] https://lore.kernel.org/r/b2c81404-67df-f841-ef02-919e841f49f2@arm.com/ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80EB5EB64D9 for ; Thu, 29 Jun 2023 15:29:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p0Uch+jly3z2h/wN3UDxwXj4ZbgMM5lvIWVdRXO7AgM=; b=fhYCsO/NTazGeX xm6J0GR1EDSN+fWb+aMclDDnrqw6R9LBz6Td6faps0AePPS0F4opWMKEkRIfDxiHfJnv3quSXIlvA ykwnExGte2CAqur4hgaD2zTz8CLGxgr+nbrLQb5x7/XmhkhVss382rPVUBglnt/Cn5I5EdBMo4Fme C4Fe3NJZaFZlbxAhR7pZJhDzmWt2BrM6s41dJrBAXodpxoXVqBg68jYH0lPP/kxYriJuPRJbnEsdp IdonN+7ySQJkdZ16kJjp7XqXki/RvxMcXzQgB313fRhYbj2o14v+f9Sf1P0YZZ6X5w7FwCD6893We Lt+alByu4JkKTpN0j/Fw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qEtZS-001QSx-0S; Thu, 29 Jun 2023 15:28:34 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qEtZJ-001QRb-2M for linux-arm-kernel@lists.infradead.org; Thu, 29 Jun 2023 15:28:33 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 35E38C14; Thu, 29 Jun 2023 08:29:04 -0700 (PDT) Received: from [10.1.27.40] (C02Z41KALVDN.cambridge.arm.com [10.1.27.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BF5103F64C; Thu, 29 Jun 2023 08:28:17 -0700 (PDT) Message-ID: <7ed253e7-7245-32bc-e5ac-6d037b711b49@arm.com> Date: Thu, 29 Jun 2023 16:28:16 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v1 00/10] variable-order, large folios for anonymous memory To: Yu Zhao , Yin Fengwei Cc: Andrew Morton , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , David Hildenbrand , Catalin Marinas , Will Deacon , Geert Uytterhoeven , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org References: <20230626171430.3167004-1-ryan.roberts@arm.com> <1fb0c4cb-a709-de20-d643-32ed43550059@arm.com> From: Ryan Roberts In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230629_082825_883805_C9F7366D X-CRM114-Status: GOOD ( 39.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMjgvMDYvMjAyMyAxOToyMiwgWXUgWmhhbyB3cm90ZToKPiBPbiBUdWUsIEp1biAyNywgMjAy MyBhdCAzOjU54oCvQU0gUnlhbiBSb2JlcnRzIDxyeWFuLnJvYmVydHNAYXJtLmNvbT4gd3JvdGU6 Cj4+Cj4+IE9uIDI3LzA2LzIwMjMgMDg6NDksIFl1IFpoYW8gd3JvdGU6Cj4+PiBPbiBNb24sIEp1 biAyNiwgMjAyMyBhdCA5OjMw4oCvUE0gWXUgWmhhbyA8eXV6aGFvQGdvb2dsZS5jb20+IHdyb3Rl Ogo+Pj4+Cj4+Pj4gT24gTW9uLCBKdW4gMjYsIDIwMjMgYXQgMTE6MTTigK9BTSBSeWFuIFJvYmVy dHMgPHJ5YW4ucm9iZXJ0c0Bhcm0uY29tPiB3cm90ZToKPj4+Pj4KPj4+Pj4gSGkgQWxsLAo+Pj4+ Pgo+Pj4+PiBGb2xsb3dpbmcgb24gZnJvbSB0aGUgcHJldmlvdXMgUkZDdjIgWzFdLCB0aGlzIHNl cmllcyBpbXBsZW1lbnRzIHZhcmlhYmxlIG9yZGVyLAo+Pj4+PiBsYXJnZSBmb2xpb3MgZm9yIGFu b255bW91cyBtZW1vcnkuIFRoZSBvYmplY3RpdmUgb2YgdGhpcyBpcyB0byBpbXByb3ZlCj4+Pj4+ IHBlcmZvcm1hbmNlIGJ5IGFsbG9jYXRpbmcgbGFyZ2VyIGNodW5rcyBvZiBtZW1vcnkgZHVyaW5n IGFub255bW91cyBwYWdlIGZhdWx0czoKPj4+Pj4KPj4+Pj4gIC0gU2luY2UgU1cgKHRoZSBrZXJu ZWwpIGlzIGRlYWxpbmcgd2l0aCBsYXJnZXIgY2h1bmtzIG9mIG1lbW9yeSB0aGFuIGJhc2UKPj4+ Pj4gICAgcGFnZXMsIHRoZXJlIGFyZSBlZmZpY2llbmN5IHNhdmluZ3MgdG8gYmUgaGFkOyBmZXdl ciBwYWdlIGZhdWx0cywgYmF0Y2hlZCBQVEUKPj4+Pj4gICAgYW5kIFJNQVAgbWFuaXB1bGF0aW9u LCBmZXdlciBpdGVtcyBvbiBsaXN0cywgZXRjLiBJbiBzaG9ydCwgd2UgcmVkdWNlIGtlcm5lbAo+ Pj4+PiAgICBvdmVyaGVhZC4gVGhpcyBzaG91bGQgYmVuZWZpdCBhbGwgYXJjaGl0ZWN0dXJlcy4K Pj4+Pj4gIC0gU2luY2Ugd2UgYXJlIG5vdyBtYXBwaW5nIHBoeXNpY2FsbHkgY29udGlndW91cyBj aHVua3Mgb2YgbWVtb3J5LCB3ZSBjYW4gdGFrZQo+Pj4+PiAgICBhZHZhbnRhZ2Ugb2YgSFcgVExC IGNvbXByZXNzaW9uIHRlY2huaXF1ZXMuIEEgcmVkdWN0aW9uIGluIFRMQiBwcmVzc3VyZQo+Pj4+ PiAgICBzcGVlZHMgdXAga2VybmVsIGFuZCB1c2VyIHNwYWNlLiBhcm02NCBzeXN0ZW1zIGhhdmUg MiBtZWNoYW5pc21zIHRvIGNvYWxlc2NlCj4+Pj4+ICAgIFRMQiBlbnRyaWVzOyAidGhlIGNvbnRp Z3VvdXMgYml0IiAoYXJjaGl0ZWN0dXJhbCkgYW5kIEhQQSAodWFyY2gpLgo+Pj4+Pgo+Pj4+PiBU aGlzIHBhdGNoIHNldCBkZWFscyB3aXRoIHRoZSBTVyBzaWRlIG9mIHRoaW5ncyBvbmx5IGFuZCBi YXNlZCBvbiBmZWVkYmFjayBmcm9tCj4+Pj4+IHRoZSBSRkMsIGFpbXMgdG8gYmUgdGhlIG1vc3Qg bWluaW1hbCBpbml0aWFsIGNoYW5nZSwgdXBvbiB3aGljaCBmdXR1cmUKPj4+Pj4gaW5jcmVtZW50 YWwgY2hhbmdlcyBjYW4gYmUgYWRkZWQuIEZvciB0aGlzIHJlYXNvbiwgdGhlIG5ldyBiZWhhdmlv dXIgaXMgaGlkZGVuCj4+Pj4+IGJlaGluZCBhIG5ldyBLY29uZmlnIHN3aXRjaCwgQ09ORklHX0xB UkdFX0FOT05fRk9MSU8sIHdoaWNoIGlzIGRpc2FibGVkIGJ5Cj4+Pj4+IGRlZmF1bHQuIEFsdGhv dWdoIHRoZSBjb2RlIGhhcyBiZWVuIHJlZmFjdG9yZWQgdG8gcGFyYW1ldGVyaXplIHRoZSBkZXNp cmVkIG9yZGVyCj4+Pj4+IG9mIHRoZSBhbGxvY2F0aW9uLCB3aGVuIHRoZSBmZWF0dXJlIGlzIGRp c2FibGVkIChieSBmb3JjaW5nIHRoZSBvcmRlciB0byBiZQo+Pj4+PiBhbHdheXMgMCkgbXkgcGVy Zm9ybWFuY2UgdGVzdHMgbWVhc3VyZSBubyByZWdyZXNzaW9uLiBTbyBJJ20gaG9waW5nIHRoaXMg d2lsbCBiZQo+Pj4+PiBhIHN1aXRhYmxlIG1lY2hhbmlzbSB0byBhbGxvdyBpbmNyZW1lbnRhbCBz dWJtaXNzaW9ucyB0byB0aGUga2VybmVsIHdpdGhvdXQKPj4+Pj4gYWZmZWN0aW5nIHRoZSByZXN0 IG9mIHRoZSB3b3JsZC4KPj4+Pj4KPj4+Pj4gVGhlIHBhdGNoZXMgYXJlIGJhc2VkIG9uIHRvcCBv ZiB2Ni40IHBsdXMgTWF0dGhldyBXaWxjb3gncyBzZXRfcHRlcygpIHNlcmllcwo+Pj4+PiBbMl0s IHdoaWNoIGlzIGEgaGFyZCBkZXBlbmRlbmN5LiBJJ20gbm90IHN1cmUgb2YgTWF0dGhldydzIGV4 YWN0IHBsYW5zIGZvcgo+Pj4+PiBnZXR0aW5nIHRoYXQgc2VyaWVzIGludG8gdGhlIGtlcm5lbCwg YnV0IEknbSBob3Bpbmcgd2UgY2FuIHN0YXJ0IHRoZSByZXZpZXcKPj4+Pj4gcHJvY2VzcyBvbiB0 aGlzIHBhdGNoIHNldCBpbmRlcGVuZGVudGx5LiBJIGhhdmUgYSBicmFuY2ggYXQgWzNdLgo+Pj4+ Pgo+Pj4+PiBJJ3ZlIHBvc3RlZCBhIHNlcGFyYXRlIHNlcmllcyBjb25jZXJuaW5nIHRoZSBIVyBw YXJ0IChjb250cHRlIG1hcHBpbmcpIGZvciBhcm02NAo+Pj4+PiBhdCBbNF0uCj4+Pj4+Cj4+Pj4+ Cj4+Pj4+IFBlcmZvcm1hbmNlCj4+Pj4+IC0tLS0tLS0tLS0tCj4+Pj4+Cj4+Pj4+IEJlbG93IHJl c3VsdHMgc2hvdyAyIGJlbmNobWFya3M7IGtlcm5lbCBjb21waWxhdGlvbiBhbmQgc3BlZWRvbWV0 ZXIgMi4wIChhCj4+Pj4+IGphdmFzY3JpcHQgYmVuY2htYXJrIHJ1bm5pbmcgaW4gQ2hyb21pdW0p LiBCb3RoIGNhc2VzIGFyZSBydW5uaW5nIG9uIEFtcGVyZQo+Pj4+PiBBbHRyYSB3aXRoIDEgTlVN QSBub2RlIGVuYWJsZWQsIFVidW50dSAyMi4wNCBhbmQgWEZTIGZpbGVzeXN0ZW0uIEVhY2ggYmVu Y2htYXJrCj4+Pj4+IGlzIHJlcGVhdGVkIDE1IHRpbWVzIG92ZXIgNSByZWJvb3RzIGFuZCBhdmVy YWdlZC4KPj4+Pj4KPj4+Pj4gQWxsIGltcHJvdmVtZW50cyBhcmUgcmVsYXRpdmUgdG8gYmFzZWxp bmUtNGsuICdhbm9uZm9saW8tYmFzaWMnIGlzIHRoaXMgc2VyaWVzLgo+Pj4+PiAnYW5vbmZvbGlv JyBpcyB0aGUgZnVsbCBwYXRjaCBzZXQgc2ltaWxhciB0byB0aGUgUkZDIHdpdGggdGhlIGFkZGl0 aW9uYWwgY2hhbmdlcwo+Pj4+PiB0byB0aGUgZXh0cmEgMyBmYXVsdCBwYXRocy4gVGhlIHJlc3Qg b2YgdGhlIGNvbmZpZ3MgYXJlIGRlc2NyaWJlZCBhdCBbNF0uCj4+Pj4+Cj4+Pj4+IEtlcm5lbCBD b21waWxhdGlvbiAoc21hbGxlciBpcyBiZXR0ZXIpOgo+Pj4+Pgo+Pj4+PiB8IGtlcm5lbCAgICAg ICAgICB8ICAgcmVhbC10aW1lIHwgICBrZXJuLXRpbWUgfCAgIHVzZXItdGltZSB8Cj4+Pj4+IHw6 LS0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS06fC0tLS0tLS0tLS0tLTp8LS0tLS0tLS0tLS0t OnwKPj4+Pj4gfCBiYXNlbGluZS00ayAgICAgfCAgICAgICAgMC4wJSB8ICAgICAgICAwLjAlIHwg ICAgICAgIDAuMCUgfAo+Pj4+PiB8IGFub25mb2xpby1iYXNpYyB8ICAgICAgIC01LjMlIHwgICAg ICAtNDIuOSUgfCAgICAgICAtMC42JSB8Cj4+Pj4+IHwgYW5vbmZvbGlvICAgICAgIHwgICAgICAg LTUuNCUgfCAgICAgIC00Ni4wJSB8ICAgICAgIC0wLjMlIHwKPj4+Pj4gfCBjb250cHRlICAgICAg ICAgfCAgICAgICAtNi44JSB8ICAgICAgLTQ1LjclIHwgICAgICAgLTIuMSUgfAo+Pj4+PiB8IGV4 ZWZvbGlvICAgICAgICB8ICAgICAgIC04LjQlIHwgICAgICAtNDYuNCUgfCAgICAgICAtMy43JSB8 Cj4+Pj4+IHwgYmFzZWxpbmUtMTZrICAgIHwgICAgICAgLTguNyUgfCAgICAgIC00OS4yJSB8ICAg ICAgIC0zLjclIHwKPj4+Pj4gfCBiYXNlbGluZS02NGsgICAgfCAgICAgIC0xMC41JSB8ICAgICAg LTY2LjAlIHwgICAgICAgLTMuNSUgfAo+Pj4+Pgo+Pj4+PiBTcGVlZG9tZXRlciAyLjAgKGJpZ2dl ciBpcyBiZXR0ZXIpOgo+Pj4+Pgo+Pj4+PiB8IGtlcm5lbCAgICAgICAgICB8ICAgcnVuc19wZXJf bWluIHwKPj4+Pj4gfDotLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLTp8Cj4+Pj4+IHwg YmFzZWxpbmUtNGsgICAgIHwgICAgICAgICAgIDAuMCUgfAo+Pj4+PiB8IGFub25mb2xpby1iYXNp YyB8ICAgICAgICAgICAwLjclIHwKPj4+Pj4gfCBhbm9uZm9saW8gICAgICAgfCAgICAgICAgICAg MS4yJSB8Cj4+Pj4+IHwgY29udHB0ZSAgICAgICAgIHwgICAgICAgICAgIDMuMSUgfAo+Pj4+PiB8 IGV4ZWZvbGlvICAgICAgICB8ICAgICAgICAgICA0LjIlIHwKPj4+Pj4gfCBiYXNlbGluZS0xNmsg ICAgfCAgICAgICAgICAgNS4zJSB8Cj4+Pj4KPj4+PiBUaGFua3MgZm9yIHB1c2hpbmcgdGhpcyBm b3J3YXJkIQo+Pj4+Cj4+Pj4+IENoYW5nZXMgc2luY2UgUkZDdjIKPj4+Pj4gLS0tLS0tLS0tLS0t LS0tLS0tLQo+Pj4+Pgo+Pj4+PiAgIC0gU2ltcGxpZmllZCBzZXJpZXMgdG8gYmFyZSBtaW5pbXVt IChvbiBEYXZpZCBIaWxkZW5icmFuZCdzIGFkdmljZSkKPj4+Pgo+Pj4+IE15IGltcHJlc3Npb24g aXMgdGhhdCB0aGlzIHNlcmllcyBzdGlsbCBpbmNsdWRlcyBtYW55IHBpZWNlcyB0aGF0IGNhbgo+ Pj4+IGJlIHNwbGl0IG91dCBhbmQgZGlzY3Vzc2VkIHNlcGFyYXRlbHkgd2l0aCBmb2xsb3d1cCBz ZXJpZXMuCj4+Pj4KPj4+PiAoSSBza2lwcGVkIDA0LzEwIGFuZCB3aWxsIGxvb2sgYXQgaXQgdG9t b3Jyb3cuKQo+Pj4KPj4+IEkgd2VudCB0aHJvdWdoIHRoZSBzZXJpZXMgdHdpY2UuIEhlcmUgd2hh dCBJIHRoaW5rIGEgYmFyZSBtaW5pbXVtCj4+PiBzZXJpZXMgKGVhc2llciB0byByZXZpZXcvZGVi dWcvbGFuZCkgd291bGQgbG9vayBsaWtlOgo+IAo+ID09PQo+IAo+Pj4gMS4gYSBuZXcgYXJjaCBz cGVjaWZpYyBmdW5jdGlvbiBwcm92aWRpbmcgYSBwcmVmZXJlZCBvcmRlciB3aXRoaW4gKDAsCj4+ PiBQTURfT1JERVIpLgo+Pj4gMi4gYW4gZXh0ZW5kZWQgYW5vbiBmb2xpbyBhbGxvYyBBUEkgdGFr aW5nIHRoYXQgb3JkZXIgKDAyLzEwLCBwYXJ0aWFsbHkpLgo+Pj4gMy4gYW4gdXBkYXRlZCBmb2xp b19hZGRfbmV3X2Fub25fcm1hcCgpIGNvdmVyaW5nIHRoZSBsYXJnZSgpICYmCj4+PiAhcG1kX21h cHBhYmxlKCkgY2FzZSAoc2ltaWxhciB0byAwNC8xMCkuCj4+PiA0LiBzL2ZvbGlvX3Rlc3RfcG1k X21hcHBhYmxlL2ZvbGlvX3Rlc3RfbGFyZ2UvIGluIHBhZ2VfcmVtb3ZlX3JtYXAoKQo+Pj4gKDA2 LzEwLCByZXZpZXdlZC1ieSBwcm92aWRlZCkuCj4+PiA1LiBmaW5hbGx5LCB1c2UgdGhlIGV4dGVu ZGVkIGFub24gZm9saW8gYWxsb2MgQVBJIHdpdGggdGhlIGFyY2gKPj4+IHByZWZlcnJlZCBvcmRl ciBpbiBkb19hbm9ueW1vdXNfcGFnZSgpICgxMC8xMCwgcGFydGlhbGx5KS4KPiAKPiA9PT0KPiAK Pj4+IFRoZSByZXN0IGNhbiBiZSBzcGxpdCBvdXQgaW50byBzZXBhcmF0ZSBzZXJpZXMgYW5kIG1v dmUgZm9yd2FyZCBpbgo+Pj4gcGFyYWxsZWwgd2l0aCBwcm9iYWJseSBhIGxvbmcgbGlzdCBvZiB0 aGluZ3Mgd2UgbmVlZC93YW50IHRvIGRvLgo+Pgo+PiBUaGFua3MgZm9yIHRoZSBmYWR0IHJldmll dyAtIEkgcmVhbGx5IGFwcHJlY2lhdGUgaXQhCj4+Cj4+IEkndmUgcmVzcG9uZGVkIHRvIG1hbnkg b2YgeW91ciBjb21tZW50cy4gSSdkIGFwcHJlY2lhdGUgaWYgd2UgY2FuIGNsb3NlIHRob3NlCj4+ IHBvaW50cyB0aGVuIEkgd2lsbCB3b3JrIHVwIGEgdjIuCj4gCj4gVGhhbmtzIQo+IAo+IEJhc2Vk IG9uIHRoZSBsYXRlc3QgZGlzY3Vzc2lvbiBoZXJlIFsxXSwgbXkgb3JpZ2luYWwgbGlzdCBhYm92 ZSBjYW4gYmUKPiBvcHRpb25hbGx5IHJlZHVjZWQgdG8gNCBwYXRjaGVzOiBpdGVtIDIgY2FuIGJl IHF1YXNoZWQgaW50byBpdGVtIDUuCj4gCj4gQWxzbyBwbGVhc2UgbWFrZSBzdXJlIHdlIGhhdmUg b25seSBvbmUgZ2xvYmFsIChhcHBseSB0byBhbGwgYXJjaHMpCj4gS2NvbmZpZyBvcHRpb24sIGFu ZCBpdCBzaG91bGQgYmUgYWRkZWQgaW4gaXRlbSA1Ogo+IAo+ICAgaWYgVFJBTlNQQVJFTlRfSFVH RVBBR0UKPiAgICAgY29uZmlnIEZMRVhJQkxFL1ZBUklBQkxFX1RIUCAjIG9yIHdoYXRldmVyIG5h bWUgeW91IHNlZSBmaXQKPiAgIGVuZCBpZgoKTmFtaW5nIGlzIGFsd2F5cyB0aGUgaGFyZGVzdCBw YXJ0LiBJJ3ZlIGJlZW4gY2FsbGluZyBpdCBMQVJHRV9BTk9OX0ZPTElPIHVwCnVudGlsIG5vdy4g QnV0IEkgdGhpbmsgeW91IGFyZSByaWdodCB0aGF0IHdlIHNob3VsZCBzaG93IHRoYXQgaXQgaXMg cmVsYXRlZCB0bwpUSFAsIHNvIEknbGwgZ28gd2l0aCBGTEVYSUJMRV9USFAgZm9yIHYyLCBhbmQg bGV0IHBlb3BsZSBzaG91dCBpZiB0aGV5IGhhdGUgaXQuCgpJZiB3ZSBhcmUgbm90IGxldHRpbmcg dGhlIGFyY2ggZGVjbGFyZSB0aGF0IGl0IHN1cHBvcnRzIEZMRVhJQkxFX1RIUCwgdGhlbiBJCnRo aW5rIHdlIG5lZWQgdGhlIGRlZmF1bHQgdmVyc2lvbiBvZiBhcmNoX3dhbnRzX3B0ZV9vcmRlcigp IHRvIHJldHVybiBhIHZhbHVlCmhpZ2hlciB0aGFuIDAgKHdoaWNoIGlzIHdoYXQgSSBoYXZlIGl0 IHJldHVybmluZyBhdCB0aGUgbW9tZW50KS4gQmVjYXVzZQpvdGhlcndpc2UsIGZvciBhbiBhcmNo IHRoYXQgaGFzbid0IGRlZmluZWQgaXRzIG93biB2ZXJzaW9uIG9mCmFyY2hfd2FudHNfcHRlX29y ZGVyKCksIEZMRVhJQkxFX1RIUCBvbiB2cyBvZmYgd2lsbCBnaXZlIHRoZSBzYW1lIHJlc3VsdC4g U28gSQpwcm9wb3NlIHRvIHNldCB0aGUgZGVmYXVsdCB0byBpbG9nMihTWl82NEsgPj4gUEFHRV9T SElGVCkuIFNob3V0IGlmIHlvdSBoYXZlIGFueQpjb25jZXJucy4KCj4gCj4gKEhvdyBtYW55IG5l dyBLY29uZmlnIG9wdGlvbnMgYWRkZWQgd2l0aGluIGFyY2gvYXJtNjQvIGlzIG5vdCBhIGNvbmNl cm4gb2YgTU0uKQo+IAo+IEFuZCBwbGVhc2UgbWFrZSBzdXJlIGl0J3MgZGlzYWJsZWQgYnkgZGVm YXVsdCwKCkRvbmUKCiBiZWNhdXNlIHdlIGFyZSBzdGlsbAo+IG1pc3NpbmcgbWFueSBpbXBvcnRh bnQgZnVuY3Rpb25zLCBlLmcuLCBJIGRvbid0IHRoaW5rIHdlIGNhbiBtbG9jaygpCj4gd2hlbiBs YXJnZSgpICYmICFwbWRfbWFwcGFibGUoKSwgc2VlIG1sb2NrX3B0ZV9yYW5nZSgpIGFuZAo+IG1s b2NrX3ZtYV9mb2xpbygpLiBXZSBjYW4gZml4IGl0IGFsb25nIHdpdGggbWFueSB0aGluZ3MgbGF0 ZXIsIGJ1dCB3ZQo+IG5lZWQgdG8gcHJlc2VudCBhIHBsYW4gYW5kIGEgc2NoZWR1bGUgbm93LiBP dGhlcndpc2UsIHRoZXJlIHdvdWxkIGJlCj4gcHVzaGJhY2sgaWYgd2UgdHJ5IHRvIGxhbmQgdGhl IHNlcmllcyB3aXRob3V0IHN1cHBvcnRpbmcgbWxvY2soKS4KClRoZXJlIGFyZSBvdGhlciBhcmVh cyB0aGF0IEknbSBhd2FyZSBvZmYuIEknbGwgcHV0IHRvZ2V0aGVyIGEgdGFibGUgYW5kIHNlbmQg aXQKb3V0IG9uY2UgSSBoYXZlIHYyIG91dCB0aGUgZG9vciAoaG9wZWZ1bGx5IHRvbW9ycm93IG9y IE1vbmRheSkuIEhvcGVmdWxseSB3ZSBjYW4Kd29yayB0b2dldGhlciB0byBmaWxsIGl0IGluIGFu ZCBmaWd1cmUgb3V0IHdobyBjYW4gZG8gd2hhdD8gSSdtIGNlcnRhaW5seQpwbGFubmluZyB0byBj b250aW51ZSB0byBwdXNoIHRoaXMgd29yayBmb3J3YXJkcyBiZXlvbmQgdGhpcyBpbml0aWFsIHBh dGNoIHNldC4KClRoYW5rcywKUnlhbgoKPiAKPiBEbyB5b3Ugb3IgRmVuZ3dlaSBwbGFuIHRvIHRh a2Ugb24gaXQ/IChJIHBlcnNvbmFsbHkgZG9uJ3QuKSBJZiBub3QsCj4gSSdsbCB0cnkgdG8gZmlu ZCBzb21lb25lIGZyb20gb3VyIHRlYW0gdG8gbG9vayBhdCBpdC4gKEl0J2QgYmUgbW9yZQo+IHNj YWxhYmxlIGlmIHdlIGhhdmUgYSBjb29yZGluYXRlZCBncm91cCBvZiBwZW9wbGUgaW5kaXZpZHVh bGx5IHNvbHZpbmcKPiBkaWZmZXJlbnQgcHJvYmxlbXMuKQo+IAo+IFsxXSBodHRwczovL2xvcmUu a2VybmVsLm9yZy9yL2IyYzgxNDA0LTY3ZGYtZjg0MS1lZjAyLTkxOWU4NDFmNDlmMkBhcm0uY29t LwoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4 LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFk Lm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFy bS1rZXJuZWwK