From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D6B7C67861 for ; Tue, 9 Apr 2024 14:46:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=I3Gpn/OL0iRCouvdyqM8uGuOtVW6Q/sXmlIQ+4brP98=; b=HXcDx9q3L113ai JOav41QslLPvruJiOR6A51HbIs25ZkhAkGz252zK+fPPtg4/YWB/yO/i7mS2owmRle/pqeQLGVrE1 GlkSc2zjdfIcGkoVPwQcmmu9dFYIlWUGZ19D/ZYnhZ3/IBqG3vk/+/L0kSjGkJWmE+XQIeDT+ctMt YdfGGD4iKQa1vD/dUA4L+OW9Z4TjKpLrJcy/D5rnY4ksN87AvglWIVWReVn4yVoXFzqNBLxQLXY+M euNlG0fsh6EfV1k4D5G3oqdxKoCTNCZYnCTWdlAHlf8LpI1cpKIj5AeVx+cxMmmJldt2rtoeH7BdQ udH5rl3n/zyYqOoEbRbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruCjJ-00000002Uii-3vzc; Tue, 09 Apr 2024 14:45:46 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruCj6-00000002UXI-2kut for linux-arm-kernel@lists.infradead.org; Tue, 09 Apr 2024 14:45:38 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712673917; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=tHbxdpXJCUAXYf/KwECE5AEhHgEmc9VwJ4s91typnE0=; b=cXf8JHCt6XCFhQYwbdfZhNfYoKSEIBWuFrgs5n2vtfFwuzmG8BL0g6wMembTZPrTLyCbcD GFMyJeqhjgNY7pUnz8PJkhAuP8987VurvTxeMpQhlr7+Cet1Ahp2BiwnUSnaRVrgWxgc0G SHvl1wlD1lv55z4QHGdrDIhqvqBlOX8= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-131-DRrmE5T1MDGv_LjCVKZB_w-1; Tue, 09 Apr 2024 10:45:15 -0400 X-MC-Unique: DRrmE5T1MDGv_LjCVKZB_w-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3455cbdea2cso1293171f8f.0 for ; Tue, 09 Apr 2024 07:45:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712673914; x=1713278714; h=content-transfer-encoding:in-reply-to:organization:autocrypt :content-language:from:references:cc:to:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=tHbxdpXJCUAXYf/KwECE5AEhHgEmc9VwJ4s91typnE0=; b=fm8pFbl6Muyj9LqI3fUuOsGSfWO98grkDiJs1yMltSTxeKFInwHBhFYu5xMhOgtzEK d50bfDhFEakW67atlgFhHUMFYfFoRaRm1bNXIZvT1Wjb2zFfn8yn1rAYINlR/kKiXHYv VlYb8knU0PRRHjWOYW6zve9KTuNAKd0MMNdF+m+pL3Bou95i0BxiMj87rUgMQJeevaHw xy3+x+wm4VVAr2KkF/1ZFySsDx1/3FyUJ9CpmxgID+DjGuF1ihNQ1ibmM7GdcvO5+9nY PLr8PA47ynpmPfBAkQwt3qL9ahYkpUpxa95JscXMvrGY35nRESme+w0vwhMG16I+77Ew KFjA== X-Forwarded-Encrypted: i=1; AJvYcCUilFHu/qz+vYzScGNcfx0g96Hpap2MDC38XIXDDkBlQd6Bp6kBskKbTpOzV6LuFBcuZfma/WezDlKa9u3upSBDeR8I9KbRrTxvhLCu7BuC9iZ32o4= X-Gm-Message-State: AOJu0YzoHdSDdz5gkVjMpxTAKuW0NZs33UVycU0ebq85swQMjNDtMiHL mmSlKmvF3icZsOzO9F7ZJnAJvEZ9/WBOuVbN1TK7M1V9yCJaNOPa5b4zoYlAaYqz5Rv8FD/IOa4 D6Aye3JAI/uIV8UjplCoOcSuwmpBeeKq+YvaRMWwp2tzIPkasl+mGK1iRjmlnW9Vdlm1VSKeo X-Received: by 2002:adf:e8ce:0:b0:343:7f4b:6da5 with SMTP id k14-20020adfe8ce000000b003437f4b6da5mr2640911wrn.17.1712673914419; Tue, 09 Apr 2024 07:45:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHClZQ7rP2xial2U5KejMrDnzBhx0WZ6sqaUIzXjZB2Wd4foOsIYetGLrhxwOuydlGYv+PN2Q== X-Received: by 2002:adf:e8ce:0:b0:343:7f4b:6da5 with SMTP id k14-20020adfe8ce000000b003437f4b6da5mr2640887wrn.17.1712673913904; Tue, 09 Apr 2024 07:45:13 -0700 (PDT) Received: from ?IPV6:2003:cb:c70a:be00:a285:bc76:307d:4eaa? (p200300cbc70abe00a285bc76307d4eaa.dip0.t-ipconnect.de. [2003:cb:c70a:be00:a285:bc76:307d:4eaa]) by smtp.gmail.com with ESMTPSA id d14-20020adfe84e000000b00343bd6716b3sm11631867wrn.99.2024.04.09.07.45.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 09 Apr 2024 07:45:13 -0700 (PDT) Message-ID: <156cf812-a2de-4480-80dc-31c38ca0eb57@redhat.com> Date: Tue, 9 Apr 2024 16:45:12 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 0/4] Speed up boot with faster linear map creation To: Ryan Roberts , Itaru Kitayama Cc: Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Donald Dutile , Eric Chanudet , Linux ARM , "linux-kernel@vger.kernel.org" References: <20240404143308.2224141-1-ryan.roberts@arm.com> <533adb77-8c2b-40db-84cb-88de77ab92bb@arm.com> <1d5abb48-08a8-4d83-a681-6915bc7b6907@arm.com> <268FBD1C-B102-4726-A7F4-1125123BDA7A@linux.dev> <5e4dc2fe-2945-4fc5-a533-c8b2d04668a0@redhat.com> From: David Hildenbrand Autocrypt: addr=david@redhat.com; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt WNyWQQ== Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240409_074533_787493_DD6E5058 X-CRM114-Status: GOOD ( 17.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMDkuMDQuMjQgMTY6MzksIFJ5YW4gUm9iZXJ0cyB3cm90ZToKPiBPbiAwOS8wNC8yMDI0IDE1 OjI5LCBEYXZpZCBIaWxkZW5icmFuZCB3cm90ZToKPj4gT24gMDkuMDQuMjQgMTY6MTMsIFJ5YW4g Um9iZXJ0cyB3cm90ZToKPj4+IE9uIDA5LzA0LzIwMjQgMTI6NTEsIERhdmlkIEhpbGRlbmJyYW5k IHdyb3RlOgo+Pj4+IE9uIDA5LjA0LjI0IDEzOjI5LCBEYXZpZCBIaWxkZW5icmFuZCB3cm90ZToK Pj4+Pj4gT24gMDkuMDQuMjQgMTM6MjIsIERhdmlkIEhpbGRlbmJyYW5kIHdyb3RlOgo+Pj4+Pj4g T24gMDkuMDQuMjQgMTI6MTMsIEl0YXJ1IEtpdGF5YW1hIHdyb3RlOgo+Pj4+Pj4+Cj4+Pj4+Pj4K Pj4+Pj4+Pj4gT24gQXByIDksIDIwMjQsIGF0IDE5OjA0LCBSeWFuIFJvYmVydHMgPHJ5YW4ucm9i ZXJ0c0Bhcm0uY29tPiB3cm90ZToKPj4+Pj4+Pj4KPj4+Pj4+Pj4gT24gMDkvMDQvMjAyNCAwMTox MCwgSXRhcnUgS2l0YXlhbWEgd3JvdGU6Cj4+Pj4+Pj4+PiBIaSBSeWFuLAo+Pj4+Pj4+Pj4KPj4+ Pj4+Pj4+PiBPbiBBcHIgOCwgMjAyNCwgYXQgMTY6MzAsIFJ5YW4gUm9iZXJ0cyA8cnlhbi5yb2Jl cnRzQGFybS5jb20+IHdyb3RlOgo+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4gT24gMDYvMDQvMjAyNCAx MTozMSwgSXRhcnUgS2l0YXlhbWEgd3JvdGU6Cj4+Pj4+Pj4+Pj4+IEhpIFJ5YW4sCj4+Pj4+Pj4+ Pj4+Cj4+Pj4+Pj4+Pj4+IE9uIFNhdCwgQXByIDA2LCAyMDI0IGF0IDA5OjMyOjM0QU0gKzAxMDAs IFJ5YW4gUm9iZXJ0cyB3cm90ZToKPj4+Pj4+Pj4+Pj4+IEhpIEl0YXJ1LAo+Pj4+Pj4+Pj4+Pj4K Pj4+Pj4+Pj4+Pj4+IE9uIDA1LzA0LzIwMjQgMDg6MzksIEl0YXJ1IEtpdGF5YW1hIHdyb3RlOgo+ Pj4+Pj4+Pj4+Pj4+IE9uIFRodSwgQXByIDA0LCAyMDI0IGF0IDAzOjMzOjA0UE0gKzAxMDAsIFJ5 YW4gUm9iZXJ0cyB3cm90ZToKPj4+Pj4+Pj4+Pj4+Pj4gSGkgQWxsLAo+Pj4+Pj4+Pj4+Pj4+Pgo+ Pj4+Pj4+Pj4+Pj4+PiBJdCB0dXJucyBvdXQgdGhhdCBjcmVhdGluZyB0aGUgbGluZWFyIG1hcCBj YW4gdGFrZSBhIHNpZ25pZmljYW50Cj4+Pj4+Pj4+Pj4+Pj4+IHByb3BvcnRpb24gb2YKPj4+Pj4+ Pj4+Pj4+Pj4gdGhlIHRvdGFsIGJvb3QgdGltZSwgZXNwZWNpYWxseSB3aGVuIHJvZGF0YT1mdWxs LiBBbmQgbW9zdCBvZiB0aGUKPj4+Pj4+Pj4+Pj4+Pj4gdGltZSBpcyBzcGVudAo+Pj4+Pj4+Pj4+ Pj4+PiB3YWl0aW5nIG9uIHN1cGVyZmx1b3VzIHRsYiBpbnZhbGlkYXRpb24gYW5kIG1lbW9yeSBi YXJyaWVycy4gVGhpcwo+Pj4+Pj4+Pj4+Pj4+PiBzZXJpZXMgcmV3b3Jrcwo+Pj4+Pj4+Pj4+Pj4+ PiB0aGUga2VybmVsIHBndGFibGUgZ2VuZXJhdGlvbiBjb2RlIHRvIHNpZ25pZmljYW50bHkgcmVk dWNlIHRoZSBudW1iZXIKPj4+Pj4+Pj4+Pj4+Pj4gb2YgdGhvc2UKPj4+Pj4+Pj4+Pj4+Pj4gVExC SXMsIElTQnMgYW5kIERTQnMuIFNlZSBlYWNoIHBhdGNoIGZvciBkZXRhaWxzLgo+Pj4+Pj4+Pj4+ Pj4+Pgo+Pj4+Pj4+Pj4+Pj4+PiBUaGUgYmVsb3cgc2hvd3MgdGhlIGV4ZWN1dGlvbiB0aW1lIG9m IG1hcF9tZW0oKSBhY3Jvc3MgYSBjb3VwbGUgb2YKPj4+Pj4+Pj4+Pj4+Pj4gZGlmZmVyZW50Cj4+ Pj4+Pj4+Pj4+Pj4+IHN5c3RlbXMgd2l0aCBkaWZmZXJlbnQgUkFNIGNvbmZpZ3VyYXRpb25zLiBX ZSBtZWFzdXJlIGFmdGVyIGFwcGx5aW5nCj4+Pj4+Pj4+Pj4+Pj4+IGVhY2ggcGF0Y2gKPj4+Pj4+ Pj4+Pj4+Pj4gYW5kIHNob3cgdGhlIGltcHJvdmVtZW50IHJlbGF0aXZlIHRvIGJhc2UgKHY2Ljkt cmMyKToKPj4+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCB8IEFwcGxlIE0yIFZNIHwgQW1wZXJlIEFsdHJhfCBBbXBlcmUgQWx0 cmF8IEFtcGVyZQo+Pj4+Pj4+Pj4+Pj4+PiBBbHRyYQo+Pj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHwgVk0sIDE2R8KgwqDCoMKgIHwgVk0sIDY0R8Kg wqDCoMKgIHwgVk0sIDI1NkfCoMKgwqAgfCBNZXRhbCwKPj4+Pj4+Pj4+Pj4+Pj4gNTEyRwo+Pj4+ Pj4+Pj4+Pj4+PiAtLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tfC0t LS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLQo+Pj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHzCoMKgIG1zwqDCoMKgICglKSB8wqDCoCBtc8KgwqDCoCAo JSkgfMKgwqAgbXPCoMKgwqAgKCUpIHwKPj4+Pj4+Pj4+Pj4+Pj4gbXPCoMKgwqAgKCUpCj4+Pj4+ Pj4+Pj4+Pj4+IC0tLS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS18LS0t LS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tCj4+Pj4+Pj4+Pj4+Pj4+IGJhc2XCoMKgwqDCoMKgwqDC oMKgwqDCoCB8wqAgMTUzwqDCoCAoMCUpIHwgMjIyN8KgwqAgKDAlKSB8IDg3OTjCoMKgICgwJSkg fCAxNzQ0Mgo+Pj4+Pj4+Pj4+Pj4+PiAoMCUpCj4+Pj4+Pj4+Pj4+Pj4+IG5vLWNvbnQtcmVtYXDC oCB8wqDCoCA3NyAoLTQ5JSkgfMKgIDQzMSAoLTgxJSkgfCAxNzI3ICgtODAlKSB8wqAgMzc5Ngo+ Pj4+Pj4+Pj4+Pj4+PiAoLTc4JSkKPj4+Pj4+Pj4+Pj4+Pj4gYmF0Y2gtYmFycmllcnMgfMKgwqAg MTMgKC05MiUpIHzCoCAxNjIgKC05MyUpIHzCoCA2NTUgKC05MyUpIHzCoCAxNjU2Cj4+Pj4+Pj4+ Pj4+Pj4+ICgtOTElKQo+Pj4+Pj4+Pj4+Pj4+PiBuby1hbGxvYy1yZW1hcCB8wqDCoCAxMSAoLTkz JSkgfMKgIDEwOSAoLTk1JSkgfMKgIDQ0OSAoLTk1JSkgfMKgIDEyNTcKPj4+Pj4+Pj4+Pj4+Pj4g KC05MyUpCj4+Pj4+Pj4+Pj4+Pj4+IGxhenktdW5tYXDCoMKgwqDCoCB8wqDCoMKgIDYgKC05NiUp IHzCoMKgIDYxICgtOTclKSB8wqAgMjU3ICgtOTclKSB8wqDCoCA4MzgKPj4+Pj4+Pj4+Pj4+Pj4g KC05NSUpCj4+Pj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+Pj4+IFRoaXMgc2VyaWVzIGFwcGxpZXMg b24gdG9wIG9mIHY2LjktcmMyLiBBbGwgbW0gc2VsZnRlc3RzIHBhc3MuIEkndmUKPj4+Pj4+Pj4+ Pj4+Pj4gY29tcGlsZSBhbmQKPj4+Pj4+Pj4+Pj4+Pj4gYm9vdCB0ZXN0ZWQgdmFyaW91cyBQQUdF X1NJWkUgYW5kIFZBIHNpemUgY29uZmlncy4KPj4+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+Pj4g LS0tCj4+Pj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+Pj4+IENoYW5nZXMgc2luY2UgdjEgWzFdCj4+ Pj4+Pj4+Pj4+Pj4+ID09PT09PT09PT09PT09PT09PT09Cj4+Pj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+ Pj4+Pj4+ICDCoMKgwqDCoMKgIC0gQWRkZWQgVGVzdGVkLWJ5IHRhZ3MgKHRoYW5rcyB0byBFcmlj IGFuZCBJdGFydSkKPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgLSBSZW5hbWVkIF9fX3NldF9w dGUoKSAtPiBfX3NldF9wdGVfbm9zeW5jKCkgKHBlciBBcmQpCj4+Pj4+Pj4+Pj4+Pj4+ICDCoMKg wqDCoMKgIC0gUmVvcmRlcmVkIHBhdGNoZXMgKGJpZ2dlc3QgaW1wYWN0ICYgbGVhc3QgY29udHJv dmVyc2lhbCBmaXJzdCkKPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgLSBSZW9yZGVyZWQgYWxs b2MvbWFwL3VubWFwIGZ1bmN0aW9ucyBpbiBtbXUuYyB0byBhaWQgcmVhZGVyCj4+Pj4+Pj4+Pj4+ Pj4+ICDCoMKgwqDCoMKgIC0gcHRlX2NsZWFyKCkgLT4gX19wdGVfY2xlYXIoKSBpbiBjbGVhcl9m aXhtYXBfbm9zeW5jKCkKPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgLSBSZXZlcnRlZCBnZW5l cmljIHA0ZF9pbmRleCgpIHdoaWNoIGNhdXNlZCB4ODYgYnVpbGQgZXJyb3IuCj4+Pj4+Pj4+Pj4+ Pj4+IFJlcGxhY2VkIHdpdGgKPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoMKgIHVuY29uZGl0 aW9uYWwgcDRkX2luZGV4KCkgZGVmaW5lIHVuZGVyIGFybTY0Lgo+Pj4+Pj4+Pj4+Pj4+Pgo+Pj4+ Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Pj4+PiBbMV0KPj4+Pj4+Pj4+Pj4+Pj4gaHR0cHM6Ly9sb3Jl Lmtlcm5lbC5vcmcvbGludXgtYXJtLWtlcm5lbC8yMDI0MDMyNjEwMTQ0OC4zNDUzNjI2LTEtcnlh bi5yb2JlcnRzQGFybS5jb20vPGh0dHBzOi8vbG9yZS5rZXJuZWwub3JnL2xpbnV4LWFybS1rZXJu ZWwvMjAyNDAzMjYxMDE0NDguMzQ1MzYyNi0xLXJ5YW4ucm9iZXJ0c0Bhcm0uY29tLz4KPj4+Pj4+ Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+Pj4gVGhhbmtzLAo+Pj4+Pj4+Pj4+Pj4+PiBSeWFuCj4+Pj4+ Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+Pj4+IFJ5YW4gUm9iZXJ0cyAoNCk6 Cj4+Pj4+Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgIGFybTY0OiBtbTogRG9uJ3QgcmVtYXAgcGd0YWJs ZXMgcGVyLWNvbnQocHRlfHBtZCkgYmxvY2sKPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgYXJt NjQ6IG1tOiBCYXRjaCBkc2IgYW5kIGlzYiB3aGVuIHBvcHVsYXRpbmcgcGd0YWJsZXMKPj4+Pj4+ Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgYXJtNjQ6IG1tOiBEb24ndCByZW1hcCBwZ3RhYmxlcyBmb3Ig YWxsb2NhdGUgdnMgcG9wdWxhdGUKPj4+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqAgYXJtNjQ6IG1t OiBMYXppbHkgY2xlYXIgcHRlIHRhYmxlIG1hcHBpbmdzIGZyb20gZml4bWFwCj4+Pj4+Pj4+Pj4+ Pj4+Cj4+Pj4+Pj4+Pj4+Pj4+IGFyY2gvYXJtNjQvaW5jbHVkZS9hc20vZml4bWFwLmjCoCB8wqDC oCA1ICstCj4+Pj4+Pj4+Pj4+Pj4+IGFyY2gvYXJtNjQvaW5jbHVkZS9hc20vbW11LmjCoMKgwqDC oCB8wqDCoCA4ICsKPj4+Pj4+Pj4+Pj4+Pj4gYXJjaC9hcm02NC9pbmNsdWRlL2FzbS9wZ3RhYmxl LmggfMKgIDEzICstCj4+Pj4+Pj4+Pj4+Pj4+IGFyY2gvYXJtNjQva2VybmVsL2NwdWZlYXR1cmUu Y8KgwqAgfMKgIDEwICstCj4+Pj4+Pj4+Pj4+Pj4+IGFyY2gvYXJtNjQvbW0vZml4bWFwLmPCoMKg wqDCoMKgwqDCoMKgwqDCoCB8wqAgMTEgKwo+Pj4+Pj4+Pj4+Pj4+PiBhcmNoL2FybTY0L21tL21t dS5jwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfCAzNzcgKysrKysrKysrKysrKysrKysrKysr KystLS0tLS0tLQo+Pj4+Pj4+Pj4+Pj4+PiA2IGZpbGVzIGNoYW5nZWQsIDMxOSBpbnNlcnRpb25z KCspLCAxMDUgZGVsZXRpb25zKC0pCj4+Pj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+Pj4+IC0tIAo+ Pj4+Pj4+Pj4+Pj4+PiAyLjI1LjEKPj4+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+ Pj4+Pj4+IEkndmUgYnVpbGQgYW5kIGJvb3QgdGVzdGVkIHRoZSB2MiBvbiBGVlAsIGJhc2UgaXMg dGFrZW4gZnJvbSB5b3VyCj4+Pj4+Pj4+Pj4+Pj4gbGludXgtcnIgcmVwby4gUnVubmluZyBydW5f dm10ZXN0cy5zaCBvbiB2MiBsZWZ0IHNvbWUgZ3VwIGxvbmd0ZXJtIG5vdAo+Pj4+Pj4+Pj4+Pj4+ IG9rcywgd291bGQgeW91IHRha2UgYSBsb29rIGF0IGl0PyBUaGUgbW0ga3NlZmx0ZXN0cyB1c2Vk IGlzIGZyb20geW91cgo+Pj4+Pj4+Pj4+Pj4+IGxpbnV4LXJyIHJlcG8gdG9vLgo+Pj4+Pj4+Pj4+ Pj4KPj4+Pj4+Pj4+Pj4+IFRoYW5rcyBmb3IgdGFraW5nIGEgbG9vayBhdCB0aGlzLgo+Pj4+Pj4+ Pj4+Pj4KPj4+Pj4+Pj4+Pj4+IEkgY2FuJ3QgcmVwcm9kdWNlIHlvdXIgaXNzdWUgdW5mb3J0dW5h dGVseTsgc3RlcHMgYXMgZm9sbG93cyBvbiBBcHBsZQo+Pj4+Pj4+Pj4+Pj4gTTIgVk06Cj4+Pj4+ Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Pj4gQ29uZmlnOiBhcm02NCBkZWZjb25maWcgKyB0aGUgZm9sbG93 aW5nOgo+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+ICMgU3F1YXNoZnMgZm9yIHNuYXBzLCB4ZnMg Zm9yIGxhcmdlIGZpbGUgZm9saW9zLgo+Pj4+Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVu YWJsZSBDT05GSUdfU1FVQVNIRlNfTFo0Cj4+Pj4+Pj4+Pj4+PiAuL3NjcmlwdHMvY29uZmlnIC0t ZW5hYmxlIENPTkZJR19TUVVBU0hGU19MWk8KPj4+Pj4+Pj4+Pj4+IC4vc2NyaXB0cy9jb25maWcg LS1lbmFibGUgQ09ORklHX1NRVUFTSEZTX1haCj4+Pj4+Pj4+Pj4+PiAuL3NjcmlwdHMvY29uZmln IC0tZW5hYmxlIENPTkZJR19TUVVBU0hGU19aU1RECj4+Pj4+Pj4+Pj4+PiAuL3NjcmlwdHMvY29u ZmlnIC0tZW5hYmxlIENPTkZJR19YRlNfRlMKPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiAjIEZv ciBnZW5lcmFsIG1tIGRlYnVnLgo+Pj4+Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJs ZSBDT05GSUdfREVCVUdfVk0KPj4+Pj4+Pj4+Pj4+IC4vc2NyaXB0cy9jb25maWcgLS1lbmFibGUg Q09ORklHX0RFQlVHX1ZNX01BUExFX1RSRUUKPj4+Pj4+Pj4+Pj4+IC4vc2NyaXB0cy9jb25maWcg LS1lbmFibGUgQ09ORklHX0RFQlVHX1ZNX1JCCj4+Pj4+Pj4+Pj4+PiAuL3NjcmlwdHMvY29uZmln IC0tZW5hYmxlIENPTkZJR19ERUJVR19WTV9QR0ZMQUdTCj4+Pj4+Pj4+Pj4+PiAuL3NjcmlwdHMv Y29uZmlnIC0tZW5hYmxlIENPTkZJR19ERUJVR19WTV9QR1RBQkxFCj4+Pj4+Pj4+Pj4+PiAuL3Nj cmlwdHMvY29uZmlnIC0tZW5hYmxlIENPTkZJR19QQUdFX1RBQkxFX0NIRUNLCj4+Pj4+Pj4+Pj4+ Pgo+Pj4+Pj4+Pj4+Pj4gIyBGb3IgbW0gc2VsZnRlc3RzLgo+Pj4+Pj4+Pj4+Pj4gLi9zY3JpcHRz L2NvbmZpZyAtLWVuYWJsZSBDT05GSUdfVVNFUkZBVUxURkQKPj4+Pj4+Pj4+Pj4+IC4vc2NyaXB0 cy9jb25maWcgLS1lbmFibGUgQ09ORklHX1RFU1RfVk1BTExPQwo+Pj4+Pj4+Pj4+Pj4gLi9zY3Jp cHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdfR1VQX1RFU1QKPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+ Pj4+PiBSdW5uaW5nIG9uIFZNIHdpdGggMTJHIG1lbW9yeSwgc3BsaXQgYWNyb3NzIDIgKGVtdWxh dGVkKSBOVU1BIG5vZGVzCj4+Pj4+Pj4+Pj4+PiAobmVlZGVkIGJ5Cj4+Pj4+Pj4+Pj4+PiBzb21l IG1tIHNlbGZ0ZXN0cyksIHdpdGgga2VybmVsIGNvbW1hbmQgbGluZSB0byByZXNlcnZlIGh1Z2V0 bGJzIGFuZAo+Pj4+Pj4+Pj4+Pj4gb3RoZXIKPj4+Pj4+Pj4+Pj4+IGZlYXR1cmVzIHJlcXVpcmVk IGJ5IHNvbWUgbW0gc2VsZnRlc3RzOgo+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+ICIKPj4+Pj4+ Pj4+Pj4+IHRyYW5zcGFyZW50X2h1Z2VwYWdlPW1hZHZpc2UgZWFybHljb24gcm9vdD0vZGV2L3Zk YTIgc2VjcmV0bWVtLmVuYWJsZQo+Pj4+Pj4+Pj4+Pj4gaHVnZXBhZ2Vzej0xRyBodWdlcGFnZXM9 MDoyLDE6MiBodWdlcGFnZXN6PTMyTSBodWdlcGFnZXM9MDoyLDE6Mgo+Pj4+Pj4+Pj4+Pj4gZGVm YXVsdF9odWdlcGFnZXN6PTJNIGh1Z2VwYWdlcz0wOjY0LDE6NjQgaHVnZXBhZ2Vzej02NEsKPj4+ Pj4+Pj4+Pj4+IGh1Z2VwYWdlcz0wOjIsMToyCj4+Pj4+Pj4+Pj4+PiAiCj4+Pj4+Pj4+Pj4+Pgo+ Pj4+Pj4+Pj4+Pj4gVWJ1bnR1IHVzZXJzcGFjZSBydW5uaW5nIG9mZiBYRlMgcm9vdGZzLiBCdWls ZCBhbmQgcnVuIG1tIHNlbGZ0ZXN0cwo+Pj4+Pj4+Pj4+Pj4gZnJvbSBzYW1lCj4+Pj4+Pj4+Pj4+ PiBnaXQgdHJlZS4KPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Pj4gQWx0aG91 Z2ggSSBkb24ndCB0aGluayBhbnkgb2YgdGhpcyBjb25maWcgc2hvdWxkIG1ha2UgYSBkaWZmZXJl bmNlIHRvCj4+Pj4+Pj4+Pj4+PiBndXBfbG9uZ3Rlcm0uCj4+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+ Pj4gTG9va3MgbGlrZSB5b3VyIGVycm9ycyBhcmUgYWxsICJmdHJ1bmNhdGUoKSBmYWlsZWQiLiBJ J3ZlIHNlZW4gdGhpcwo+Pj4+Pj4+Pj4+Pj4gcHJvYmxlbSBvbgo+Pj4+Pj4+Pj4+Pj4gb3VyIENJ IHN5c3RlbS4gVGhlcmUgaXQgaXMgZHVlIHRvIHJ1bm5pbmcgdGhlIHRlc3RzIGZyb20gTkZTIGZp bGUKPj4+Pj4+Pj4+Pj4+IHN5c3RlbS4gV2hhdAo+Pj4+Pj4+Pj4+Pj4gZmlsZXN5c3RlbSBhcmUg eW91IHVzaW5nPyBQZXJoYXBzIHlvdSBhcmUgc2hhcmluZyBpbnRvIHRoZSBGVlAgdXNpbmcKPj4+ Pj4+Pj4+Pj4+IDlwPyBUaGF0Cj4+Pj4+Pj4+Pj4+PiBtaWdodCBhbHNvIGJlIHByb2JsZW1hdGlj Lgo+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+PiBUaGF0IHdhcyBpdC4gVGhpcyB0aW1lIEkgYm9vdGVk IHVwIHRoZSBrZXJuZWwgaW5jbHVkaW5nIHlvdXIgc2VyaWVzIG9uCj4+Pj4+Pj4+Pj4+IFFFTVUg b24gbXkgTTEgYW5kIGV4ZWN1dGVkIHRoZSBndXBfbG9uZ3Rlcm0gcHJvZ3JhbSB3aXRob3V0IHRo ZSBmdHJ1bmNhdGUKPj4+Pj4+Pj4+Pj4gZmFpbHVyZXMuIFdoZW4gdGVzdGluZyB5b3VyIGtlcm5l bCBvbiBGVlAsIEkgd2FzIGV4ZWN1dGluZyB0aGUgc2NyaXB0Cj4+Pj4+Pj4+Pj4+IGZyb20gdGhl IEZWUCdzIGhvc3QgZmlsZXN5c3RlbSB1c2luZyA5cC4KPj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+IEkn bSBub3Qgc3VyZSBleGFjdGx5IHdoYXQgdGhlIHJvb3QgY2F1c2UgaXMuIFBlcmhhcHMgdGhlcmUg aXNuJ3QgZW5vdWdoCj4+Pj4+Pj4+Pj4gc3BhY2Ugb24KPj4+Pj4+Pj4+PiB0aGUgZGlzaz8gSXQg bWlnaHQgYmUgd29ydGggZW5oYW5jaW5nIHRoZSBlcnJvciBsb2cgdG8gcHJvdmlkZSB0aGUKPj4+ Pj4+Pj4+PiBlcnJubyBpbgo+Pj4+Pj4+Pj4+IHRvb2xzL3Rlc3Rpbmcvc2VsZnRlc3RzL21tL2d1 cF9sb25ndGVybS5jLgo+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4gQXR0YWNoZWQgaXMg dGhlIHN0cmFjZeKAmWQgZ3VwX2xvbmd0ZXJtIGV4ZWN1dGlvbmcgbG9nIG9uIHlvdXIKPj4+Pj4+ Pj4+IHBndGFibGUtYm9vdC1zcGVlZHVwLXYyIGtlcm5lbC4KPj4+Pj4+Pj4KPj4+Pj4+Pj4gU29y cnkgYXJlIHlvdSBzYXlpbmcgdGhhdCBpdCBvbmx5IGZhaWxzIHdpdGggdGhlIHBndGFibGUtYm9v dC1zcGVlZHVwLXYyCj4+Pj4+Pj4+IHBhdGNoCj4+Pj4+Pj4+IHNldCBhcHBsaWVkPyBJIHRob3Vn aHQgd2UgcHJldmlvdXNseSBjb25jbHVkZWQgdGhhdCBpdCB3YXMgaW5kZXBlbmRlbnQgb2YKPj4+ Pj4+Pj4gdGhhdD8KPj4+Pj4+Pj4gSSB3YXMgdW5kZXIgdGhlIGltcHJlc3Npb24gdGhhdCBpdCB3 YXMgZmlsZXN5c3RlbSByZWxhdGVkIGFuZCBub3Qgc29tZXRoaW5nCj4+Pj4+Pj4+IHRoYXQKPj4+ Pj4+Pj4gSSB3YXMgcGxhbm5pbmcgdG8gaW52ZXN0aWdhdGUuCj4+Pj4+Pj4KPj4+Pj4+PiBObywg aXJyZXNwZWN0aXZlIG9mIHRoZSBrZXJuZWwsIGlmIHVzaW5nIDlwIG9uIEZWUCB0aGUgdGVzdCBw cm9ncmFtIGZhaWxzLgo+Pj4+Pj4+IEl0IGlzIGluZGVlZCA5cCBmaWxlc3lzdGVtIHJlbGF0ZWQs IGFzIEkgc3dpdGNoZWQgdG8gdXNpbmcgTkZTIGFsbCB0aGUKPj4+Pj4+PiBpc3N1ZXMgYXJlIGdv bmUuCj4+Pj4+Pgo+Pj4+Pj4gRGlkIGl0IG5ldmVyIHdvcmsgb24gOXA/IElmIHNvLCB3ZSBtaWdo dCBoYXZlIHRvIFNLSVAgdGhhdCB0ZXN0Lgo+Pj4+Pj4KPj4+Pj4+IG9wZW5hdChBVF9GRENXRCwg Imd1cF9sb25ndGVybS5jX3RtcGZpbGVfQkxib090IiwgT19SRFdSfE9fQ1JFQVR8T19FWENMLAo+ Pj4+Pj4gMDYwMCkgPSAzCj4+Pj4+PiB1bmxpbmthdChBVF9GRENXRCwgImd1cF9sb25ndGVybS5j X3RtcGZpbGVfQkxib090IiwgMCkgPSAwCj4+Pj4+PiBmc3RhdGZzKDMsIDB4ZmZmZmU1MDVhODQw KcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgID0gLTEgRU9QTk9UU1VQUCAoT3BlcmF0aW9uIG5v dAo+Pj4+Pj4gc3VwcG9ydGVkKQo+Pj4+Pj4gZnRydW5jYXRlKDMsIDQwOTYpwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgID0gLTEgRU5PRU5UIChObyBzdWNoIGZpbGUg b3IKPj4+Pj4+IGRpcmVjdG9yeSkKPj4+Pj4KPj4+Pj4gTm90ZTogSSdtIHdvbmRlcmluZyBpZiB0 aGUgdW5saW5rYXQgaGVyZSBpcyB0aGUgcHJvYmxlbSB0aGF0IG1ha2VzCj4+Pj4+IGZ0cnVuY2F0 ZSgpIHdpdGggOXAgcmVzdWx0IGluIHdlaXJkIGVycm9ycyAoZS5nLiwgdGhlIGh5cGVydmlzb3IK Pj4+Pj4gdW5saW5rZWQgdGhlIGZpbGUgYW5kIGNhbm5vdCByZW9wZW4gaXQgZm9yIHRoZSBmc3Rh dGZzL2Z0cnVuY2F0ZS4gLi4uCj4+Pj4+IHdoaWNoIGdpdmVzIHVzIHdlaXJkIGVycm9ycyBoZXJl KS4KPj4+Pj4KPj4+Pj4gVGhlbiwgd2Ugc2hvdWxkIGxvb2t1cCB0aGUgZnMgdHlwZSBpbiBydW5f d2l0aF9sb2NhbF90bXBmaWxlKCkgYmVmb3JlCj4+Pj4+IHRoZSB1bmxpbmsoKSBhbmQgc2ltcGx5 IHNraXAgdGhlIHRlc3QgaWYgaXQgaXMgOXAuCj4+Pj4KPj4+PiBUaGUgdW5saW5rIHdpdGggOXAg bW9zdCBjZXJ0YWlubHkgd2FzIGEga25vd24gaXNzdWUgaW4gdGhlIHBhc3Q6Cj4+Pj4KPj4+PiBo dHRwczovL2dpdGxhYi5jb20vcWVtdS1wcm9qZWN0L3FlbXUvLS9pc3N1ZXMvMTAzCj4+Pj4KPj4+ PiBNYXliZSBpdCdzIHN0aWxsIGFuIGlzc3VlIHdpdGggb2xkZXIgaHlwZXJ2aXNvcnMgKFFFTVU/ KT8gT3IgaXQgd2FzIG5ldmVyCj4+Pj4gY29tcGxldGVseSByZXNvbHZlZD8KPj4+Cj4+PiBJIGJl bGlldmUgSXRhcnUgaXMgcnVubmluZyBvbiBGVlAgKEZpeGVkIFZpcnR1YWwgUGxhdGZvcm0gLSAi ZmFzdCBtb2RlbCIgLQo+Pj4gQXJtJ3MgYXJjaGl0ZWN0dXJlIGVtdWxhdG9yKS4gU28gUUVNVSB3 b24ndCBiZSBpbnZvbHZlZCBoZXJlLiBUaGUgRlZQIGVtdWxhdGVzCj4+PiBhIDlwIGRldmljZSwg c28gcGVyaGFwcyB0aGUgYnVnIGlzIGluIHRoZXJlLgo+Pgo+PiBWZXJ5IGxpa2VseS4KPj4KPj4+ Cj4+PiBOb3RlIHRoYXQgSSBzZWUgbG90cyBvZiAiZmFsbG9jYXRlKCkgZmFpbGVkIiBmYWlsdXJl cyBpbiBndXBfbG9uZ3Rlcm0gd2hlbgo+Pj4gcnVubmluZyBvbiBvdXIgQ0kgc3lzdGVtLiBUaGlz IGlzIGEgY29tcGxldGVseSBkaWZmZXJlbnQgc2V0dXA7IFJlYWwgSFcgd2l0aAo+Pj4gTGludXgg cnVubmluZyBiYXJlIG1ldGFsIHVzaW5nIGFuIE5GUyByb290ZnMuIEknbSBub3Qgc3VyZSBpZiB0 aGlzIGlzIHJlbGF0ZWQuCj4+PiBMb2dzIHNob3cgaXQgZmFpbGluZyBjb25zaXN0ZW50bHkgZm9y IHRoZSAidG1wZmlsZSIgYW5kICJsb2NhbCB0bXBmaWxlIiB0ZXN0Cj4+PiBjb25maWdzLiBJIGFs c28gc2VlIGEgY291cGxlIG9mIHRoZXNlIGZhaWxzIGluIHRoZSBjb3cgdGVzdHMuCj4+Cj4+IFdo YXQgaXMgdGhlIGZhbGxvY2F0ZSgpIGVycm5vIHlvdSBhcmUgZ2V0dGluZz8gc3RyYWNlIGxvZyB3 b3VsZCBoZWxwICh0byBzZWUgaWYKPj4gc3RhdGZzIGFsc28gZmFpbHMgYWxyZWFkeSkhIExpa2Vs eSBhIHNpbWlsYXIgTkZTIGlzc3VlLgo+IAo+IFVuZm9ydHVuYXRlbHkgdGhpcyBpcyBhIHN5c3Rl bSBJIGRvbid0IGhhdmUgYWNjZXNzIHRvLiBJJ3ZlIHJlcXVlc3RlZCBzb21lIG9mCj4gdGhpcyB0 cmlhZ2UgdG8gYmUgZG9uZSwgYnV0IGl0cyBmYWlybHkgbG93IHByaW9yaXR5IHVuZm9ydHVuYXRl bHkuCgpUbyB3b3JrIGFyb3VuZCB0aGVzZSBCVUdzICg/KSBlbHNld2hlcmUsIHdlIGNvdWxkIHNp bXBseSBza2lwIHRoZSB0ZXN0IAppZiBnZXRfZnNfdHlwZSgpIGlzIG5vdCBhYmxlIHRvIGRldGVj dCB0aGUgRlMgdHlwZS4gTGlrZWx5IHRoYXQncyBhbiAKZWFybHkgaW5kaWNhdG9yIHRoYXQgdGhl IHVubGluaygpIG1lc3NlZCBzb21ldGhpbmcgdXAuCgouLi4gZG9lc24ndCBmZWVsIHJpZ2h0LCB0 aG91Z2guCgotLSAKQ2hlZXJzLAoKRGF2aWQgLyBkaGlsZGVuYgoKCl9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBs aXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5m cmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D33D782D99 for ; Tue, 9 Apr 2024 14:45:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712673919; cv=none; b=gPBRXmlyrYUpnOj8GGz+wAXsTSgbFS+kPe2rjyo1TGpl1eB1XR0xouiyZO7+I8IpfE1T95IBDmwEvHQZwW52sxLJ2D7q72dBuqTq0ByO/9Fv8C/S3bi/Jzu5p+uHe4yRLejD20kNlWdGau7gmhxO79UHPXvtKBVhaiLTfTVqcYo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712673919; c=relaxed/simple; bh=aGqz41U3y9/pJtpcePlc6MaIjtPa7lwPthlt+Cl5SsA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=IlJNuqnbn6+Mon513AqblLbrMwGHnQdMvrbb9r3GeRBJk09lFkfCJKHxcn4K7QYLFBTg6eUK55xI64cNoL6LaGiXv6m1nQU37WDe0/ZiPf/jiKh+Vbvn7T0UK7Ei7Ir/8/LAks+4CxHF511vNxAGLYRd/j78jtM5Hmy2itRco4E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cXf8JHCt; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cXf8JHCt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712673917; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=tHbxdpXJCUAXYf/KwECE5AEhHgEmc9VwJ4s91typnE0=; b=cXf8JHCt6XCFhQYwbdfZhNfYoKSEIBWuFrgs5n2vtfFwuzmG8BL0g6wMembTZPrTLyCbcD GFMyJeqhjgNY7pUnz8PJkhAuP8987VurvTxeMpQhlr7+Cet1Ahp2BiwnUSnaRVrgWxgc0G SHvl1wlD1lv55z4QHGdrDIhqvqBlOX8= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-393-2aT3VqbgO-qLPRfOwtdDGw-1; Tue, 09 Apr 2024 10:45:15 -0400 X-MC-Unique: 2aT3VqbgO-qLPRfOwtdDGw-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-345cc6c2367so1158977f8f.0 for ; Tue, 09 Apr 2024 07:45:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712673914; x=1713278714; h=content-transfer-encoding:in-reply-to:organization:autocrypt :content-language:from:references:cc:to:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=tHbxdpXJCUAXYf/KwECE5AEhHgEmc9VwJ4s91typnE0=; b=E102WmN4kh34TitPE1qBweNdJHBiu0hyN8ctYDlCEJSB3T4w/dTX4ho8xLfgO/tVXi GY1aE9clyS0OZrPnZHozNP4pCMZxqiQ7Btu6DC3rhlnI1QWq+3DzcELbNk2vD0Y1SbqW Al24dLXPT6DU8+sWV5pXxtJYWJRrPTS/w7VeVXt9X4VxwTZJdjm4G3KkQ/yGtp/+PDpu kCeoMi36AtY7frrUZ5Ia6ky2/KqekR97qc6wwfirdFLW0IDdGbWIR3TxEUo797vFBTfT up8uCHcBmKR7gzwO9m1CUBdX62Mx/94fCowilGN0QWhPzw9QAMQPMb+Cm5bjw8zqqUxK W7xg== X-Forwarded-Encrypted: i=1; AJvYcCXJHGGBLRWveuawM0kQnhdQdur7JXexfBOeFH05yVJs7hY/raQUUTi7/tHV5+h0ug5uTyNvXhWo7jQfkjeTgZCA764CHCMufp8ThUoq X-Gm-Message-State: AOJu0YwpLhLwkVYKwnP5me0grJ/PfOpyBFOIboWIjVv+0fogVqEWS4ZB RLa+Gkshm9vObHrLVMYeElpgaNbUWMoTLzuYSmt5CFUlIyTob/KarD/8QPNJISfvdXEVbtyWZBZ bP44SkxZbtuuaqLAXy4SxPel9sx5sUoYQ1BKia9p7EZ0sqnY29waxO1v2MrpPW8cNixGkDA== X-Received: by 2002:adf:e8ce:0:b0:343:7f4b:6da5 with SMTP id k14-20020adfe8ce000000b003437f4b6da5mr2640913wrn.17.1712673914420; Tue, 09 Apr 2024 07:45:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHClZQ7rP2xial2U5KejMrDnzBhx0WZ6sqaUIzXjZB2Wd4foOsIYetGLrhxwOuydlGYv+PN2Q== X-Received: by 2002:adf:e8ce:0:b0:343:7f4b:6da5 with SMTP id k14-20020adfe8ce000000b003437f4b6da5mr2640887wrn.17.1712673913904; Tue, 09 Apr 2024 07:45:13 -0700 (PDT) Received: from ?IPV6:2003:cb:c70a:be00:a285:bc76:307d:4eaa? (p200300cbc70abe00a285bc76307d4eaa.dip0.t-ipconnect.de. [2003:cb:c70a:be00:a285:bc76:307d:4eaa]) by smtp.gmail.com with ESMTPSA id d14-20020adfe84e000000b00343bd6716b3sm11631867wrn.99.2024.04.09.07.45.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 09 Apr 2024 07:45:13 -0700 (PDT) Message-ID: <156cf812-a2de-4480-80dc-31c38ca0eb57@redhat.com> Date: Tue, 9 Apr 2024 16:45:12 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 0/4] Speed up boot with faster linear map creation To: Ryan Roberts , Itaru Kitayama Cc: Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Donald Dutile , Eric Chanudet , Linux ARM , "linux-kernel@vger.kernel.org" References: <20240404143308.2224141-1-ryan.roberts@arm.com> <533adb77-8c2b-40db-84cb-88de77ab92bb@arm.com> <1d5abb48-08a8-4d83-a681-6915bc7b6907@arm.com> <268FBD1C-B102-4726-A7F4-1125123BDA7A@linux.dev> <5e4dc2fe-2945-4fc5-a533-c8b2d04668a0@redhat.com> From: David Hildenbrand Content-Language: en-US Autocrypt: addr=david@redhat.com; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt WNyWQQ== Organization: Red Hat In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 09.04.24 16:39, Ryan Roberts wrote: > On 09/04/2024 15:29, David Hildenbrand wrote: >> On 09.04.24 16:13, Ryan Roberts wrote: >>> On 09/04/2024 12:51, David Hildenbrand wrote: >>>> On 09.04.24 13:29, David Hildenbrand wrote: >>>>> On 09.04.24 13:22, David Hildenbrand wrote: >>>>>> On 09.04.24 12:13, Itaru Kitayama wrote: >>>>>>> >>>>>>> >>>>>>>> On Apr 9, 2024, at 19:04, Ryan Roberts wrote: >>>>>>>> >>>>>>>> On 09/04/2024 01:10, Itaru Kitayama wrote: >>>>>>>>> Hi Ryan, >>>>>>>>> >>>>>>>>>> On Apr 8, 2024, at 16:30, Ryan Roberts wrote: >>>>>>>>>> >>>>>>>>>> On 06/04/2024 11:31, Itaru Kitayama wrote: >>>>>>>>>>> Hi Ryan, >>>>>>>>>>> >>>>>>>>>>> On Sat, Apr 06, 2024 at 09:32:34AM +0100, Ryan Roberts wrote: >>>>>>>>>>>> Hi Itaru, >>>>>>>>>>>> >>>>>>>>>>>> On 05/04/2024 08:39, Itaru Kitayama wrote: >>>>>>>>>>>>> On Thu, Apr 04, 2024 at 03:33:04PM +0100, Ryan Roberts wrote: >>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>> >>>>>>>>>>>>>> It turns out that creating the linear map can take a significant >>>>>>>>>>>>>> proportion of >>>>>>>>>>>>>> the total boot time, especially when rodata=full. And most of the >>>>>>>>>>>>>> time is spent >>>>>>>>>>>>>> waiting on superfluous tlb invalidation and memory barriers. This >>>>>>>>>>>>>> series reworks >>>>>>>>>>>>>> the kernel pgtable generation code to significantly reduce the number >>>>>>>>>>>>>> of those >>>>>>>>>>>>>> TLBIs, ISBs and DSBs. See each patch for details. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The below shows the execution time of map_mem() across a couple of >>>>>>>>>>>>>> different >>>>>>>>>>>>>> systems with different RAM configurations. We measure after applying >>>>>>>>>>>>>> each patch >>>>>>>>>>>>>> and show the improvement relative to base (v6.9-rc2): >>>>>>>>>>>>>> >>>>>>>>>>>>>>                    | Apple M2 VM | Ampere Altra| Ampere Altra| Ampere >>>>>>>>>>>>>> Altra >>>>>>>>>>>>>>                    | VM, 16G     | VM, 64G     | VM, 256G    | Metal, >>>>>>>>>>>>>> 512G >>>>>>>>>>>>>> ---------------|-------------|-------------|-------------|------------- >>>>>>>>>>>>>>                    |   ms    (%) |   ms    (%) |   ms    (%) | >>>>>>>>>>>>>> ms    (%) >>>>>>>>>>>>>> ---------------|-------------|-------------|-------------|------------- >>>>>>>>>>>>>> base           |  153   (0%) | 2227   (0%) | 8798   (0%) | 17442 >>>>>>>>>>>>>> (0%) >>>>>>>>>>>>>> no-cont-remap  |   77 (-49%) |  431 (-81%) | 1727 (-80%) |  3796 >>>>>>>>>>>>>> (-78%) >>>>>>>>>>>>>> batch-barriers |   13 (-92%) |  162 (-93%) |  655 (-93%) |  1656 >>>>>>>>>>>>>> (-91%) >>>>>>>>>>>>>> no-alloc-remap |   11 (-93%) |  109 (-95%) |  449 (-95%) |  1257 >>>>>>>>>>>>>> (-93%) >>>>>>>>>>>>>> lazy-unmap     |    6 (-96%) |   61 (-97%) |  257 (-97%) |   838 >>>>>>>>>>>>>> (-95%) >>>>>>>>>>>>>> >>>>>>>>>>>>>> This series applies on top of v6.9-rc2. All mm selftests pass. I've >>>>>>>>>>>>>> compile and >>>>>>>>>>>>>> boot tested various PAGE_SIZE and VA size configs. >>>>>>>>>>>>>> >>>>>>>>>>>>>> --- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Changes since v1 [1] >>>>>>>>>>>>>> ==================== >>>>>>>>>>>>>> >>>>>>>>>>>>>>       - Added Tested-by tags (thanks to Eric and Itaru) >>>>>>>>>>>>>>       - Renamed ___set_pte() -> __set_pte_nosync() (per Ard) >>>>>>>>>>>>>>       - Reordered patches (biggest impact & least controversial first) >>>>>>>>>>>>>>       - Reordered alloc/map/unmap functions in mmu.c to aid reader >>>>>>>>>>>>>>       - pte_clear() -> __pte_clear() in clear_fixmap_nosync() >>>>>>>>>>>>>>       - Reverted generic p4d_index() which caused x86 build error. >>>>>>>>>>>>>> Replaced with >>>>>>>>>>>>>>         unconditional p4d_index() define under arm64. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> https://lore.kernel.org/linux-arm-kernel/20240326101448.3453626-1-ryan.roberts@arm.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Ryan >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Ryan Roberts (4): >>>>>>>>>>>>>>       arm64: mm: Don't remap pgtables per-cont(pte|pmd) block >>>>>>>>>>>>>>       arm64: mm: Batch dsb and isb when populating pgtables >>>>>>>>>>>>>>       arm64: mm: Don't remap pgtables for allocate vs populate >>>>>>>>>>>>>>       arm64: mm: Lazily clear pte table mappings from fixmap >>>>>>>>>>>>>> >>>>>>>>>>>>>> arch/arm64/include/asm/fixmap.h  |   5 +- >>>>>>>>>>>>>> arch/arm64/include/asm/mmu.h     |   8 + >>>>>>>>>>>>>> arch/arm64/include/asm/pgtable.h |  13 +- >>>>>>>>>>>>>> arch/arm64/kernel/cpufeature.c   |  10 +- >>>>>>>>>>>>>> arch/arm64/mm/fixmap.c           |  11 + >>>>>>>>>>>>>> arch/arm64/mm/mmu.c              | 377 +++++++++++++++++++++++-------- >>>>>>>>>>>>>> 6 files changed, 319 insertions(+), 105 deletions(-) >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> 2.25.1 >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I've build and boot tested the v2 on FVP, base is taken from your >>>>>>>>>>>>> linux-rr repo. Running run_vmtests.sh on v2 left some gup longterm not >>>>>>>>>>>>> oks, would you take a look at it? The mm ksefltests used is from your >>>>>>>>>>>>> linux-rr repo too. >>>>>>>>>>>> >>>>>>>>>>>> Thanks for taking a look at this. >>>>>>>>>>>> >>>>>>>>>>>> I can't reproduce your issue unfortunately; steps as follows on Apple >>>>>>>>>>>> M2 VM: >>>>>>>>>>>> >>>>>>>>>>>> Config: arm64 defconfig + the following: >>>>>>>>>>>> >>>>>>>>>>>> # Squashfs for snaps, xfs for large file folios. >>>>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_LZ4 >>>>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_LZO >>>>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_XZ >>>>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_ZSTD >>>>>>>>>>>> ./scripts/config --enable CONFIG_XFS_FS >>>>>>>>>>>> >>>>>>>>>>>> # For general mm debug. >>>>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM >>>>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_MAPLE_TREE >>>>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_RB >>>>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_PGFLAGS >>>>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_PGTABLE >>>>>>>>>>>> ./scripts/config --enable CONFIG_PAGE_TABLE_CHECK >>>>>>>>>>>> >>>>>>>>>>>> # For mm selftests. >>>>>>>>>>>> ./scripts/config --enable CONFIG_USERFAULTFD >>>>>>>>>>>> ./scripts/config --enable CONFIG_TEST_VMALLOC >>>>>>>>>>>> ./scripts/config --enable CONFIG_GUP_TEST >>>>>>>>>>>> >>>>>>>>>>>> Running on VM with 12G memory, split across 2 (emulated) NUMA nodes >>>>>>>>>>>> (needed by >>>>>>>>>>>> some mm selftests), with kernel command line to reserve hugetlbs and >>>>>>>>>>>> other >>>>>>>>>>>> features required by some mm selftests: >>>>>>>>>>>> >>>>>>>>>>>> " >>>>>>>>>>>> transparent_hugepage=madvise earlycon root=/dev/vda2 secretmem.enable >>>>>>>>>>>> hugepagesz=1G hugepages=0:2,1:2 hugepagesz=32M hugepages=0:2,1:2 >>>>>>>>>>>> default_hugepagesz=2M hugepages=0:64,1:64 hugepagesz=64K >>>>>>>>>>>> hugepages=0:2,1:2 >>>>>>>>>>>> " >>>>>>>>>>>> >>>>>>>>>>>> Ubuntu userspace running off XFS rootfs. Build and run mm selftests >>>>>>>>>>>> from same >>>>>>>>>>>> git tree. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Although I don't think any of this config should make a difference to >>>>>>>>>>>> gup_longterm. >>>>>>>>>>>> >>>>>>>>>>>> Looks like your errors are all "ftruncate() failed". I've seen this >>>>>>>>>>>> problem on >>>>>>>>>>>> our CI system. There it is due to running the tests from NFS file >>>>>>>>>>>> system. What >>>>>>>>>>>> filesystem are you using? Perhaps you are sharing into the FVP using >>>>>>>>>>>> 9p? That >>>>>>>>>>>> might also be problematic. >>>>>>>>>>> >>>>>>>>>>> That was it. This time I booted up the kernel including your series on >>>>>>>>>>> QEMU on my M1 and executed the gup_longterm program without the ftruncate >>>>>>>>>>> failures. When testing your kernel on FVP, I was executing the script >>>>>>>>>>> from the FVP's host filesystem using 9p. >>>>>>>>>> >>>>>>>>>> I'm not sure exactly what the root cause is. Perhaps there isn't enough >>>>>>>>>> space on >>>>>>>>>> the disk? It might be worth enhancing the error log to provide the >>>>>>>>>> errno in >>>>>>>>>> tools/testing/selftests/mm/gup_longterm.c. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Attached is the strace’d gup_longterm executiong log on your >>>>>>>>> pgtable-boot-speedup-v2 kernel. >>>>>>>> >>>>>>>> Sorry are you saying that it only fails with the pgtable-boot-speedup-v2 >>>>>>>> patch >>>>>>>> set applied? I thought we previously concluded that it was independent of >>>>>>>> that? >>>>>>>> I was under the impression that it was filesystem related and not something >>>>>>>> that >>>>>>>> I was planning to investigate. >>>>>>> >>>>>>> No, irrespective of the kernel, if using 9p on FVP the test program fails. >>>>>>> It is indeed 9p filesystem related, as I switched to using NFS all the >>>>>>> issues are gone. >>>>>> >>>>>> Did it never work on 9p? If so, we might have to SKIP that test. >>>>>> >>>>>> openat(AT_FDCWD, "gup_longterm.c_tmpfile_BLboOt", O_RDWR|O_CREAT|O_EXCL, >>>>>> 0600) = 3 >>>>>> unlinkat(AT_FDCWD, "gup_longterm.c_tmpfile_BLboOt", 0) = 0 >>>>>> fstatfs(3, 0xffffe505a840)              = -1 EOPNOTSUPP (Operation not >>>>>> supported) >>>>>> ftruncate(3, 4096)                      = -1 ENOENT (No such file or >>>>>> directory) >>>>> >>>>> Note: I'm wondering if the unlinkat here is the problem that makes >>>>> ftruncate() with 9p result in weird errors (e.g., the hypervisor >>>>> unlinked the file and cannot reopen it for the fstatfs/ftruncate. ... >>>>> which gives us weird errors here). >>>>> >>>>> Then, we should lookup the fs type in run_with_local_tmpfile() before >>>>> the unlink() and simply skip the test if it is 9p. >>>> >>>> The unlink with 9p most certainly was a known issue in the past: >>>> >>>> https://gitlab.com/qemu-project/qemu/-/issues/103 >>>> >>>> Maybe it's still an issue with older hypervisors (QEMU?)? Or it was never >>>> completely resolved? >>> >>> I believe Itaru is running on FVP (Fixed Virtual Platform - "fast model" - >>> Arm's architecture emulator). So QEMU won't be involved here. The FVP emulates >>> a 9p device, so perhaps the bug is in there. >> >> Very likely. >> >>> >>> Note that I see lots of "fallocate() failed" failures in gup_longterm when >>> running on our CI system. This is a completely different setup; Real HW with >>> Linux running bare metal using an NFS rootfs. I'm not sure if this is related. >>> Logs show it failing consistently for the "tmpfile" and "local tmpfile" test >>> configs. I also see a couple of these fails in the cow tests. >> >> What is the fallocate() errno you are getting? strace log would help (to see if >> statfs also fails already)! Likely a similar NFS issue. > > Unfortunately this is a system I don't have access to. I've requested some of > this triage to be done, but its fairly low priority unfortunately. To work around these BUGs (?) elsewhere, we could simply skip the test if get_fs_type() is not able to detect the FS type. Likely that's an early indicator that the unlink() messed something up. ... doesn't feel right, though. -- Cheers, David / dhildenb