From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A4E3ACD1296 for ; Tue, 9 Apr 2024 14:35:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MgV5bmYIsfLthmkAnOngsVnjkEN9iALM3Sf31bmEVho=; b=bEvlefjPUVHZZ+ hSu+c0I8Z8ENBxz+2retX7SOmtKnjMQ09b0Jbe0BZf1QUMl32i1bNzT5Z0Neo4Auf/z89QnuzbxQk wOKodhYbzxgpXxFMHzME0w7+zU/LyvoDM+suVXC4qg90DXG9VB7SESsMxHWmPoUdtFRKCLYl2FNis iMsyCVNyL1ZeJ2UstXH+OveoZcaJIIrqJM/sKaLMy3w8yBxfnl7dOjFELQHj/7/sLxevxBiM1rCHT wEP2Z3tlwOUFhMTkcW9zcInI+UNHtcHYZGIsqcKkJhgm5X/cVZE2O+utddVDYLuErMFeBv2QyCpVR YUrsYseyhP1970u6OKXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruCZ3-00000002ROJ-22eE; Tue, 09 Apr 2024 14:35:09 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruCYs-00000002RKJ-2HO6 for linux-arm-kernel@lists.infradead.org; Tue, 09 Apr 2024 14:35:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712673297; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=uSy4hmOLYZFzlKt9kcrYgfbIexHYMkegeYTn4mQv5d8=; b=W828GwKV2GNtlJeTi8U4+RkPuuKTUJgnof19sEHpHZ6v35k3kehBfZKglAfgmPKlNlPhni xYHRCOkcMd2Yqx+Fp7wtDelTc479iuYlmMwgvJAB5pvmZOZPvSw0RGRZgTKS6OXD9bd1fk H/CGbLdkGFPLTonQo+veRkYNxABZU3A= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-QgLe56LQPfOn7wBfBHuK4g-1; Tue, 09 Apr 2024 10:34:55 -0400 X-MC-Unique: QgLe56LQPfOn7wBfBHuK4g-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-41401f598cfso33337015e9.2 for ; Tue, 09 Apr 2024 07:34:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712672989; x=1713277789; h=content-transfer-encoding:in-reply-to:organization:autocrypt :content-language:from:references:cc:to:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=uSy4hmOLYZFzlKt9kcrYgfbIexHYMkegeYTn4mQv5d8=; b=sK+C8oVNIMTnOqHbeqpmMaoxEyTQzMvuJYe2zUyNZ+knUGfIAyCyhSZmC8MvhwS2B4 4LuBUcN8jiQOxIyrk4m3Q934nBu125K8+yb8+qLUdfrER6S4DDVQB4ucVzCVwPYLfwF8 6j0PG9t9N7BvR6Z5i+dKetio5ZCIL/3j83nozeC4NEhfdv9E4JCYKctjMxrlp8ij7iIu bM45gZ3eH5aDva4NM1U2SppdQ/FwPYVR8/IAA269EANC7MHl6l+oInCKAcYAlG9f85+b d/cwpBC7y5vdvTezqaBp6fjS/jLHqUxEWg9BLC8t6P/oWoqJsdMNlPqTz9aBXhXOMThQ 591A== X-Forwarded-Encrypted: i=1; AJvYcCU6Z8dtJjPMp+C1I15IRmbZgrCoCJ/rUW3xRKQhzXXJuvuHvFulSMNn7UnAaseUh5gDO4o7fd7stGbTLqtkNncMzptWM3Kev9Hweyv7mb83FERXhWY= X-Gm-Message-State: AOJu0YzyE17EJDdUglGhDJXEVyXrEBCwxdlpVzvmodpiGeuwPaXWq+lh 81PwzXMMdgKPXdHeRTur31/KjVU2G+lg5ZLaK+hjYjMlw6LdPIKAAxoo4+qfndSDrMU4Q4LJdoP CnPtvqu+R9yTuSezp5oevWKOm1d+RFHhaHsRzKe00kn12RNYrjtnsPnIZ9bZOsdf+R2+Sket3 X-Received: by 2002:a05:600c:1f81:b0:416:1c9f:3a3f with SMTP id je1-20020a05600c1f8100b004161c9f3a3fmr8161228wmb.9.1712672989218; Tue, 09 Apr 2024 07:29:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFuVQgn4xrJl3abG1X8zRrA4z6AnUi7eOyhM0YvqA/qEe3DHF7sDFrxH4yOljhuuybBI/bKwA== X-Received: by 2002:a05:600c:1f81:b0:416:1c9f:3a3f with SMTP id je1-20020a05600c1f8100b004161c9f3a3fmr8161206wmb.9.1712672988730; Tue, 09 Apr 2024 07:29:48 -0700 (PDT) Received: from ?IPV6:2003:cb:c70a:be00:a285:bc76:307d:4eaa? (p200300cbc70abe00a285bc76307d4eaa.dip0.t-ipconnect.de. [2003:cb:c70a:be00:a285:bc76:307d:4eaa]) by smtp.gmail.com with ESMTPSA id bi8-20020a05600c3d8800b00416b92dceaesm859398wmb.11.2024.04.09.07.29.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 09 Apr 2024 07:29:48 -0700 (PDT) Message-ID: Date: Tue, 9 Apr 2024 16:29:47 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 0/4] Speed up boot with faster linear map creation To: Ryan Roberts , Itaru Kitayama Cc: Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Donald Dutile , Eric Chanudet , Linux ARM , "linux-kernel@vger.kernel.org" References: <20240404143308.2224141-1-ryan.roberts@arm.com> <533adb77-8c2b-40db-84cb-88de77ab92bb@arm.com> <1d5abb48-08a8-4d83-a681-6915bc7b6907@arm.com> <268FBD1C-B102-4726-A7F4-1125123BDA7A@linux.dev> <5e4dc2fe-2945-4fc5-a533-c8b2d04668a0@redhat.com> From: David Hildenbrand Autocrypt: addr=david@redhat.com; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt WNyWQQ== Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240409_073458_825486_F340EBC3 X-CRM114-Status: GOOD ( 20.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMDkuMDQuMjQgMTY6MTMsIFJ5YW4gUm9iZXJ0cyB3cm90ZToKPiBPbiAwOS8wNC8yMDI0IDEy OjUxLCBEYXZpZCBIaWxkZW5icmFuZCB3cm90ZToKPj4gT24gMDkuMDQuMjQgMTM6MjksIERhdmlk IEhpbGRlbmJyYW5kIHdyb3RlOgo+Pj4gT24gMDkuMDQuMjQgMTM6MjIsIERhdmlkIEhpbGRlbmJy YW5kIHdyb3RlOgo+Pj4+IE9uIDA5LjA0LjI0IDEyOjEzLCBJdGFydSBLaXRheWFtYSB3cm90ZToK Pj4+Pj4KPj4+Pj4KPj4+Pj4+IE9uIEFwciA5LCAyMDI0LCBhdCAxOTowNCwgUnlhbiBSb2JlcnRz IDxyeWFuLnJvYmVydHNAYXJtLmNvbT4gd3JvdGU6Cj4+Pj4+Pgo+Pj4+Pj4gT24gMDkvMDQvMjAy NCAwMToxMCwgSXRhcnUgS2l0YXlhbWEgd3JvdGU6Cj4+Pj4+Pj4gSGkgUnlhbiwKPj4+Pj4+Pgo+ Pj4+Pj4+PiBPbiBBcHIgOCwgMjAyNCwgYXQgMTY6MzAsIFJ5YW4gUm9iZXJ0cyA8cnlhbi5yb2Jl cnRzQGFybS5jb20+IHdyb3RlOgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBPbiAwNi8wNC8yMDI0IDExOjMx LCBJdGFydSBLaXRheWFtYSB3cm90ZToKPj4+Pj4+Pj4+IEhpIFJ5YW4sCj4+Pj4+Pj4+Pgo+Pj4+ Pj4+Pj4gT24gU2F0LCBBcHIgMDYsIDIwMjQgYXQgMDk6MzI6MzRBTSArMDEwMCwgUnlhbiBSb2Jl cnRzIHdyb3RlOgo+Pj4+Pj4+Pj4+IEhpIEl0YXJ1LAo+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4gT24g MDUvMDQvMjAyNCAwODozOSwgSXRhcnUgS2l0YXlhbWEgd3JvdGU6Cj4+Pj4+Pj4+Pj4+IE9uIFRo dSwgQXByIDA0LCAyMDI0IGF0IDAzOjMzOjA0UE0gKzAxMDAsIFJ5YW4gUm9iZXJ0cyB3cm90ZToK Pj4+Pj4+Pj4+Pj4+IEhpIEFsbCwKPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiBJdCB0dXJucyBv dXQgdGhhdCBjcmVhdGluZyB0aGUgbGluZWFyIG1hcCBjYW4gdGFrZSBhIHNpZ25pZmljYW50Cj4+ Pj4+Pj4+Pj4+PiBwcm9wb3J0aW9uIG9mCj4+Pj4+Pj4+Pj4+PiB0aGUgdG90YWwgYm9vdCB0aW1l LCBlc3BlY2lhbGx5IHdoZW4gcm9kYXRhPWZ1bGwuIEFuZCBtb3N0IG9mIHRoZQo+Pj4+Pj4+Pj4+ Pj4gdGltZSBpcyBzcGVudAo+Pj4+Pj4+Pj4+Pj4gd2FpdGluZyBvbiBzdXBlcmZsdW91cyB0bGIg aW52YWxpZGF0aW9uIGFuZCBtZW1vcnkgYmFycmllcnMuIFRoaXMKPj4+Pj4+Pj4+Pj4+IHNlcmll cyByZXdvcmtzCj4+Pj4+Pj4+Pj4+PiB0aGUga2VybmVsIHBndGFibGUgZ2VuZXJhdGlvbiBjb2Rl IHRvIHNpZ25pZmljYW50bHkgcmVkdWNlIHRoZSBudW1iZXIKPj4+Pj4+Pj4+Pj4+IG9mIHRob3Nl Cj4+Pj4+Pj4+Pj4+PiBUTEJJcywgSVNCcyBhbmQgRFNCcy4gU2VlIGVhY2ggcGF0Y2ggZm9yIGRl dGFpbHMuCj4+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Pj4gVGhlIGJlbG93IHNob3dzIHRoZSBleGVj dXRpb24gdGltZSBvZiBtYXBfbWVtKCkgYWNyb3NzIGEgY291cGxlIG9mCj4+Pj4+Pj4+Pj4+PiBk aWZmZXJlbnQKPj4+Pj4+Pj4+Pj4+IHN5c3RlbXMgd2l0aCBkaWZmZXJlbnQgUkFNIGNvbmZpZ3Vy YXRpb25zLiBXZSBtZWFzdXJlIGFmdGVyIGFwcGx5aW5nCj4+Pj4+Pj4+Pj4+PiBlYWNoIHBhdGNo Cj4+Pj4+Pj4+Pj4+PiBhbmQgc2hvdyB0aGUgaW1wcm92ZW1lbnQgcmVsYXRpdmUgdG8gYmFzZSAo djYuOS1yYzIpOgo+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIHwgQXBwbGUgTTIgVk0gfCBBbXBlcmUgQWx0cmF8IEFtcGVyZSBBbHRy YXwgQW1wZXJlCj4+Pj4+Pj4+Pj4+PiBBbHRyYQo+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgfCBWTSwgMTZHwqDCoMKgwqAgfCBWTSwgNjRHwqDCoMKgwqAg fCBWTSwgMjU2R8KgwqDCoCB8IE1ldGFsLAo+Pj4+Pj4+Pj4+Pj4gNTEyRwo+Pj4+Pj4+Pj4+Pj4g LS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0t fC0tLS0tLS0tLS0tLS0KPj4+Pj4+Pj4+Pj4+ICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIHzCoMKgIG1zwqDCoMKgICglKSB8wqDCoCBtc8KgwqDCoCAoJSkgfMKgwqAgbXPCoMKg wqAgKCUpIHwKPj4+Pj4+Pj4+Pj4+IG1zwqDCoMKgICglKQo+Pj4+Pj4+Pj4+Pj4gLS0tLS0tLS0t LS0tLS0tfC0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLXwtLS0tLS0tLS0tLS0tfC0tLS0tLS0t LS0tLS0KPj4+Pj4+Pj4+Pj4+IGJhc2XCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqAgMTUzwqDCoCAo MCUpIHwgMjIyN8KgwqAgKDAlKSB8IDg3OTjCoMKgICgwJSkgfCAxNzQ0MsKgwqAgKDAlKQo+Pj4+ Pj4+Pj4+Pj4gbm8tY29udC1yZW1hcMKgIHzCoMKgIDc3ICgtNDklKSB8wqAgNDMxICgtODElKSB8 IDE3MjcgKC04MCUpIHzCoCAzNzk2ICgtNzglKQo+Pj4+Pj4+Pj4+Pj4gYmF0Y2gtYmFycmllcnMg fMKgwqAgMTMgKC05MiUpIHzCoCAxNjIgKC05MyUpIHzCoCA2NTUgKC05MyUpIHzCoCAxNjU2ICgt OTElKQo+Pj4+Pj4+Pj4+Pj4gbm8tYWxsb2MtcmVtYXAgfMKgwqAgMTEgKC05MyUpIHzCoCAxMDkg KC05NSUpIHzCoCA0NDkgKC05NSUpIHzCoCAxMjU3ICgtOTMlKQo+Pj4+Pj4+Pj4+Pj4gbGF6eS11 bm1hcMKgwqDCoMKgIHzCoMKgwqAgNiAoLTk2JSkgfMKgwqAgNjEgKC05NyUpIHzCoCAyNTcgKC05 NyUpIHzCoMKgIDgzOCAoLTk1JSkKPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiBUaGlzIHNlcmll cyBhcHBsaWVzIG9uIHRvcCBvZiB2Ni45LXJjMi4gQWxsIG1tIHNlbGZ0ZXN0cyBwYXNzLiBJJ3Zl Cj4+Pj4+Pj4+Pj4+PiBjb21waWxlIGFuZAo+Pj4+Pj4+Pj4+Pj4gYm9vdCB0ZXN0ZWQgdmFyaW91 cyBQQUdFX1NJWkUgYW5kIFZBIHNpemUgY29uZmlncy4KPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+ PiAtLS0KPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiBDaGFuZ2VzIHNpbmNlIHYxIFsxXQo+Pj4+ Pj4+Pj4+Pj4gPT09PT09PT09PT09PT09PT09PT0KPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiAg wqDCoMKgwqAgLSBBZGRlZCBUZXN0ZWQtYnkgdGFncyAodGhhbmtzIHRvIEVyaWMgYW5kIEl0YXJ1 KQo+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgIC0gUmVuYW1lZCBfX19zZXRfcHRlKCkgLT4gX19zZXRf cHRlX25vc3luYygpIChwZXIgQXJkKQo+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgIC0gUmVvcmRlcmVk IHBhdGNoZXMgKGJpZ2dlc3QgaW1wYWN0ICYgbGVhc3QgY29udHJvdmVyc2lhbCBmaXJzdCkKPj4+ Pj4+Pj4+Pj4+ICDCoMKgwqDCoCAtIFJlb3JkZXJlZCBhbGxvYy9tYXAvdW5tYXAgZnVuY3Rpb25z IGluIG1tdS5jIHRvIGFpZCByZWFkZXIKPj4+Pj4+Pj4+Pj4+ICDCoMKgwqDCoCAtIHB0ZV9jbGVh cigpIC0+IF9fcHRlX2NsZWFyKCkgaW4gY2xlYXJfZml4bWFwX25vc3luYygpCj4+Pj4+Pj4+Pj4+ PiAgwqDCoMKgwqAgLSBSZXZlcnRlZCBnZW5lcmljIHA0ZF9pbmRleCgpIHdoaWNoIGNhdXNlZCB4 ODYgYnVpbGQgZXJyb3IuCj4+Pj4+Pj4+Pj4+PiBSZXBsYWNlZCB3aXRoCj4+Pj4+Pj4+Pj4+PiAg wqDCoMKgwqDCoMKgIHVuY29uZGl0aW9uYWwgcDRkX2luZGV4KCkgZGVmaW5lIHVuZGVyIGFybTY0 Lgo+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiBbMV0KPj4+Pj4+Pj4+Pj4+ IGh0dHBzOi8vbG9yZS5rZXJuZWwub3JnL2xpbnV4LWFybS1rZXJuZWwvMjAyNDAzMjYxMDE0NDgu MzQ1MzYyNi0xLXJ5YW4ucm9iZXJ0c0Bhcm0uY29tLzxodHRwczovL2xvcmUua2VybmVsLm9yZy9s aW51eC1hcm0ta2VybmVsLzIwMjQwMzI2MTAxNDQ4LjM0NTM2MjYtMS1yeWFuLnJvYmVydHNAYXJt LmNvbS8+Cj4+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Pj4gVGhhbmtzLAo+Pj4+Pj4+Pj4+Pj4gUnlh bgo+Pj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4+PiBSeWFuIFJvYmVydHMgKDQp Ogo+Pj4+Pj4+Pj4+Pj4gIMKgwqDCoMKgIGFybTY0OiBtbTogRG9uJ3QgcmVtYXAgcGd0YWJsZXMg cGVyLWNvbnQocHRlfHBtZCkgYmxvY2sKPj4+Pj4+Pj4+Pj4+ICDCoMKgwqDCoCBhcm02NDogbW06 IEJhdGNoIGRzYiBhbmQgaXNiIHdoZW4gcG9wdWxhdGluZyBwZ3RhYmxlcwo+Pj4+Pj4+Pj4+Pj4g IMKgwqDCoMKgIGFybTY0OiBtbTogRG9uJ3QgcmVtYXAgcGd0YWJsZXMgZm9yIGFsbG9jYXRlIHZz IHBvcHVsYXRlCj4+Pj4+Pj4+Pj4+PiAgwqDCoMKgwqAgYXJtNjQ6IG1tOiBMYXppbHkgY2xlYXIg cHRlIHRhYmxlIG1hcHBpbmdzIGZyb20gZml4bWFwCj4+Pj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Pj4g YXJjaC9hcm02NC9pbmNsdWRlL2FzbS9maXhtYXAuaMKgIHzCoMKgIDUgKy0KPj4+Pj4+Pj4+Pj4+ IGFyY2gvYXJtNjQvaW5jbHVkZS9hc20vbW11LmjCoMKgwqDCoCB8wqDCoCA4ICsKPj4+Pj4+Pj4+ Pj4+IGFyY2gvYXJtNjQvaW5jbHVkZS9hc20vcGd0YWJsZS5oIHzCoCAxMyArLQo+Pj4+Pj4+Pj4+ Pj4gYXJjaC9hcm02NC9rZXJuZWwvY3B1ZmVhdHVyZS5jwqDCoCB8wqAgMTAgKy0KPj4+Pj4+Pj4+ Pj4+IGFyY2gvYXJtNjQvbW0vZml4bWFwLmPCoMKgwqDCoMKgwqDCoMKgwqDCoCB8wqAgMTEgKwo+ Pj4+Pj4+Pj4+Pj4gYXJjaC9hcm02NC9tbS9tbXUuY8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IHwgMzc3ICsrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0KPj4+Pj4+Pj4+Pj4+IDYgZmls ZXMgY2hhbmdlZCwgMzE5IGluc2VydGlvbnMoKyksIDEwNSBkZWxldGlvbnMoLSkKPj4+Pj4+Pj4+ Pj4+Cj4+Pj4+Pj4+Pj4+PiAtLSAKPj4+Pj4+Pj4+Pj4+IDIuMjUuMQo+Pj4+Pj4+Pj4+Pj4KPj4+ Pj4+Pj4+Pj4KPj4+Pj4+Pj4+Pj4gSSd2ZSBidWlsZCBhbmQgYm9vdCB0ZXN0ZWQgdGhlIHYyIG9u IEZWUCwgYmFzZSBpcyB0YWtlbiBmcm9tIHlvdXIKPj4+Pj4+Pj4+Pj4gbGludXgtcnIgcmVwby4g UnVubmluZyBydW5fdm10ZXN0cy5zaCBvbiB2MiBsZWZ0IHNvbWUgZ3VwIGxvbmd0ZXJtIG5vdAo+ Pj4+Pj4+Pj4+PiBva3MsIHdvdWxkIHlvdSB0YWtlIGEgbG9vayBhdCBpdD8gVGhlIG1tIGtzZWZs dGVzdHMgdXNlZCBpcyBmcm9tIHlvdXIKPj4+Pj4+Pj4+Pj4gbGludXgtcnIgcmVwbyB0b28uCj4+ Pj4+Pj4+Pj4KPj4+Pj4+Pj4+PiBUaGFua3MgZm9yIHRha2luZyBhIGxvb2sgYXQgdGhpcy4KPj4+ Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+IEkgY2FuJ3QgcmVwcm9kdWNlIHlvdXIgaXNzdWUgdW5mb3J0dW5h dGVseTsgc3RlcHMgYXMgZm9sbG93cyBvbiBBcHBsZQo+Pj4+Pj4+Pj4+IE0yIFZNOgo+Pj4+Pj4+ Pj4+Cj4+Pj4+Pj4+Pj4gQ29uZmlnOiBhcm02NCBkZWZjb25maWcgKyB0aGUgZm9sbG93aW5nOgo+ Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4gIyBTcXVhc2hmcyBmb3Igc25hcHMsIHhmcyBmb3IgbGFyZ2Ug ZmlsZSBmb2xpb3MuCj4+Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdf U1FVQVNIRlNfTFo0Cj4+Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdf U1FVQVNIRlNfTFpPCj4+Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdf U1FVQVNIRlNfWFoKPj4+Pj4+Pj4+PiAuL3NjcmlwdHMvY29uZmlnIC0tZW5hYmxlIENPTkZJR19T UVVBU0hGU19aU1RECj4+Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdf WEZTX0ZTCj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+PiAjIEZvciBnZW5lcmFsIG1tIGRlYnVnLgo+Pj4+ Pj4+Pj4+IC4vc2NyaXB0cy9jb25maWcgLS1lbmFibGUgQ09ORklHX0RFQlVHX1ZNCj4+Pj4+Pj4+ Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdfREVCVUdfVk1fTUFQTEVfVFJFRQo+ Pj4+Pj4+Pj4+IC4vc2NyaXB0cy9jb25maWcgLS1lbmFibGUgQ09ORklHX0RFQlVHX1ZNX1JCCj4+ Pj4+Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdfREVCVUdfVk1fUEdGTEFH Uwo+Pj4+Pj4+Pj4+IC4vc2NyaXB0cy9jb25maWcgLS1lbmFibGUgQ09ORklHX0RFQlVHX1ZNX1BH VEFCTEUKPj4+Pj4+Pj4+PiAuL3NjcmlwdHMvY29uZmlnIC0tZW5hYmxlIENPTkZJR19QQUdFX1RB QkxFX0NIRUNLCj4+Pj4+Pj4+Pj4KPj4+Pj4+Pj4+PiAjIEZvciBtbSBzZWxmdGVzdHMuCj4+Pj4+ Pj4+Pj4gLi9zY3JpcHRzL2NvbmZpZyAtLWVuYWJsZSBDT05GSUdfVVNFUkZBVUxURkQKPj4+Pj4+ Pj4+PiAuL3NjcmlwdHMvY29uZmlnIC0tZW5hYmxlIENPTkZJR19URVNUX1ZNQUxMT0MKPj4+Pj4+ Pj4+PiAuL3NjcmlwdHMvY29uZmlnIC0tZW5hYmxlIENPTkZJR19HVVBfVEVTVAo+Pj4+Pj4+Pj4+ Cj4+Pj4+Pj4+Pj4gUnVubmluZyBvbiBWTSB3aXRoIDEyRyBtZW1vcnksIHNwbGl0IGFjcm9zcyAy IChlbXVsYXRlZCkgTlVNQSBub2Rlcwo+Pj4+Pj4+Pj4+IChuZWVkZWQgYnkKPj4+Pj4+Pj4+PiBz b21lIG1tIHNlbGZ0ZXN0cyksIHdpdGgga2VybmVsIGNvbW1hbmQgbGluZSB0byByZXNlcnZlIGh1 Z2V0bGJzIGFuZCBvdGhlcgo+Pj4+Pj4+Pj4+IGZlYXR1cmVzIHJlcXVpcmVkIGJ5IHNvbWUgbW0g c2VsZnRlc3RzOgo+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4gIgo+Pj4+Pj4+Pj4+IHRyYW5zcGFyZW50 X2h1Z2VwYWdlPW1hZHZpc2UgZWFybHljb24gcm9vdD0vZGV2L3ZkYTIgc2VjcmV0bWVtLmVuYWJs ZQo+Pj4+Pj4+Pj4+IGh1Z2VwYWdlc3o9MUcgaHVnZXBhZ2VzPTA6MiwxOjIgaHVnZXBhZ2Vzej0z Mk0gaHVnZXBhZ2VzPTA6MiwxOjIKPj4+Pj4+Pj4+PiBkZWZhdWx0X2h1Z2VwYWdlc3o9Mk0gaHVn ZXBhZ2VzPTA6NjQsMTo2NCBodWdlcGFnZXN6PTY0SyBodWdlcGFnZXM9MDoyLDE6Mgo+Pj4+Pj4+ Pj4+ICIKPj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+IFVidW50dSB1c2Vyc3BhY2UgcnVubmluZyBvZmYg WEZTIHJvb3Rmcy4gQnVpbGQgYW5kIHJ1biBtbSBzZWxmdGVzdHMKPj4+Pj4+Pj4+PiBmcm9tIHNh bWUKPj4+Pj4+Pj4+PiBnaXQgdHJlZS4KPj4+Pj4+Pj4+Pgo+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+Pj4g QWx0aG91Z2ggSSBkb24ndCB0aGluayBhbnkgb2YgdGhpcyBjb25maWcgc2hvdWxkIG1ha2UgYSBk aWZmZXJlbmNlIHRvCj4+Pj4+Pj4+Pj4gZ3VwX2xvbmd0ZXJtLgo+Pj4+Pj4+Pj4+Cj4+Pj4+Pj4+ Pj4gTG9va3MgbGlrZSB5b3VyIGVycm9ycyBhcmUgYWxsICJmdHJ1bmNhdGUoKSBmYWlsZWQiLiBJ J3ZlIHNlZW4gdGhpcwo+Pj4+Pj4+Pj4+IHByb2JsZW0gb24KPj4+Pj4+Pj4+PiBvdXIgQ0kgc3lz dGVtLiBUaGVyZSBpdCBpcyBkdWUgdG8gcnVubmluZyB0aGUgdGVzdHMgZnJvbSBORlMgZmlsZQo+ Pj4+Pj4+Pj4+IHN5c3RlbS4gV2hhdAo+Pj4+Pj4+Pj4+IGZpbGVzeXN0ZW0gYXJlIHlvdSB1c2lu Zz8gUGVyaGFwcyB5b3UgYXJlIHNoYXJpbmcgaW50byB0aGUgRlZQIHVzaW5nCj4+Pj4+Pj4+Pj4g OXA/IFRoYXQKPj4+Pj4+Pj4+PiBtaWdodCBhbHNvIGJlIHByb2JsZW1hdGljLgo+Pj4+Pj4+Pj4K Pj4+Pj4+Pj4+IFRoYXQgd2FzIGl0LiBUaGlzIHRpbWUgSSBib290ZWQgdXAgdGhlIGtlcm5lbCBp bmNsdWRpbmcgeW91ciBzZXJpZXMgb24KPj4+Pj4+Pj4+IFFFTVUgb24gbXkgTTEgYW5kIGV4ZWN1 dGVkIHRoZSBndXBfbG9uZ3Rlcm0gcHJvZ3JhbSB3aXRob3V0IHRoZSBmdHJ1bmNhdGUKPj4+Pj4+ Pj4+IGZhaWx1cmVzLiBXaGVuIHRlc3RpbmcgeW91ciBrZXJuZWwgb24gRlZQLCBJIHdhcyBleGVj dXRpbmcgdGhlIHNjcmlwdAo+Pj4+Pj4+Pj4gZnJvbSB0aGUgRlZQJ3MgaG9zdCBmaWxlc3lzdGVt IHVzaW5nIDlwLgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBJJ20gbm90IHN1cmUgZXhhY3RseSB3aGF0IHRo ZSByb290IGNhdXNlIGlzLiBQZXJoYXBzIHRoZXJlIGlzbid0IGVub3VnaAo+Pj4+Pj4+PiBzcGFj ZSBvbgo+Pj4+Pj4+PiB0aGUgZGlzaz8gSXQgbWlnaHQgYmUgd29ydGggZW5oYW5jaW5nIHRoZSBl cnJvciBsb2cgdG8gcHJvdmlkZSB0aGUgZXJybm8gaW4KPj4+Pj4+Pj4gdG9vbHMvdGVzdGluZy9z ZWxmdGVzdHMvbW0vZ3VwX2xvbmd0ZXJtLmMuCj4+Pj4+Pj4+Cj4+Pj4+Pj4KPj4+Pj4+PiBBdHRh Y2hlZCBpcyB0aGUgc3RyYWNl4oCZZCBndXBfbG9uZ3Rlcm0gZXhlY3V0aW9uZyBsb2cgb24geW91 cgo+Pj4+Pj4+IHBndGFibGUtYm9vdC1zcGVlZHVwLXYyIGtlcm5lbC4KPj4+Pj4+Cj4+Pj4+PiBT b3JyeSBhcmUgeW91IHNheWluZyB0aGF0IGl0IG9ubHkgZmFpbHMgd2l0aCB0aGUgcGd0YWJsZS1i b290LXNwZWVkdXAtdjIgcGF0Y2gKPj4+Pj4+IHNldCBhcHBsaWVkPyBJIHRob3VnaHQgd2UgcHJl dmlvdXNseSBjb25jbHVkZWQgdGhhdCBpdCB3YXMgaW5kZXBlbmRlbnQgb2YKPj4+Pj4+IHRoYXQ/ Cj4+Pj4+PiBJIHdhcyB1bmRlciB0aGUgaW1wcmVzc2lvbiB0aGF0IGl0IHdhcyBmaWxlc3lzdGVt IHJlbGF0ZWQgYW5kIG5vdCBzb21ldGhpbmcKPj4+Pj4+IHRoYXQKPj4+Pj4+IEkgd2FzIHBsYW5u aW5nIHRvIGludmVzdGlnYXRlLgo+Pj4+Pgo+Pj4+PiBObywgaXJyZXNwZWN0aXZlIG9mIHRoZSBr ZXJuZWwsIGlmIHVzaW5nIDlwIG9uIEZWUCB0aGUgdGVzdCBwcm9ncmFtIGZhaWxzLgo+Pj4+PiBJ dCBpcyBpbmRlZWQgOXAgZmlsZXN5c3RlbSByZWxhdGVkLCBhcyBJIHN3aXRjaGVkIHRvIHVzaW5n IE5GUyBhbGwgdGhlCj4+Pj4+IGlzc3VlcyBhcmUgZ29uZS4KPj4+Pgo+Pj4+IERpZCBpdCBuZXZl ciB3b3JrIG9uIDlwPyBJZiBzbywgd2UgbWlnaHQgaGF2ZSB0byBTS0lQIHRoYXQgdGVzdC4KPj4+ Pgo+Pj4+IG9wZW5hdChBVF9GRENXRCwgImd1cF9sb25ndGVybS5jX3RtcGZpbGVfQkxib090Iiwg T19SRFdSfE9fQ1JFQVR8T19FWENMLAo+Pj4+IDA2MDApID0gMwo+Pj4+IHVubGlua2F0KEFUX0ZE Q1dELCAiZ3VwX2xvbmd0ZXJtLmNfdG1wZmlsZV9CTGJvT3QiLCAwKSA9IDAKPj4+PiBmc3RhdGZz KDMsIDB4ZmZmZmU1MDVhODQwKcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgID0gLTEgRU9QTk9U U1VQUCAoT3BlcmF0aW9uIG5vdAo+Pj4+IHN1cHBvcnRlZCkKPj4+PiBmdHJ1bmNhdGUoMywgNDA5 NinCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgPSAtMSBFTk9FTlQg KE5vIHN1Y2ggZmlsZSBvciBkaXJlY3RvcnkpCj4+Pgo+Pj4gTm90ZTogSSdtIHdvbmRlcmluZyBp ZiB0aGUgdW5saW5rYXQgaGVyZSBpcyB0aGUgcHJvYmxlbSB0aGF0IG1ha2VzCj4+PiBmdHJ1bmNh dGUoKSB3aXRoIDlwIHJlc3VsdCBpbiB3ZWlyZCBlcnJvcnMgKGUuZy4sIHRoZSBoeXBlcnZpc29y Cj4+PiB1bmxpbmtlZCB0aGUgZmlsZSBhbmQgY2Fubm90IHJlb3BlbiBpdCBmb3IgdGhlIGZzdGF0 ZnMvZnRydW5jYXRlLiAuLi4KPj4+IHdoaWNoIGdpdmVzIHVzIHdlaXJkIGVycm9ycyBoZXJlKS4K Pj4+Cj4+PiBUaGVuLCB3ZSBzaG91bGQgbG9va3VwIHRoZSBmcyB0eXBlIGluIHJ1bl93aXRoX2xv Y2FsX3RtcGZpbGUoKSBiZWZvcmUKPj4+IHRoZSB1bmxpbmsoKSBhbmQgc2ltcGx5IHNraXAgdGhl IHRlc3QgaWYgaXQgaXMgOXAuCj4+Cj4+IFRoZSB1bmxpbmsgd2l0aCA5cCBtb3N0IGNlcnRhaW5s eSB3YXMgYSBrbm93biBpc3N1ZSBpbiB0aGUgcGFzdDoKPj4KPj4gaHR0cHM6Ly9naXRsYWIuY29t L3FlbXUtcHJvamVjdC9xZW11Ly0vaXNzdWVzLzEwMwo+Pgo+PiBNYXliZSBpdCdzIHN0aWxsIGFu IGlzc3VlIHdpdGggb2xkZXIgaHlwZXJ2aXNvcnMgKFFFTVU/KT8gT3IgaXQgd2FzIG5ldmVyCj4+ IGNvbXBsZXRlbHkgcmVzb2x2ZWQ/Cj4gCj4gSSBiZWxpZXZlIEl0YXJ1IGlzIHJ1bm5pbmcgb24g RlZQIChGaXhlZCBWaXJ0dWFsIFBsYXRmb3JtIC0gImZhc3QgbW9kZWwiIC0gQXJtJ3MgYXJjaGl0 ZWN0dXJlIGVtdWxhdG9yKS4gU28gUUVNVSB3b24ndCBiZSBpbnZvbHZlZCBoZXJlLiBUaGUgRlZQ IGVtdWxhdGVzIGEgOXAgZGV2aWNlLCBzbyBwZXJoYXBzIHRoZSBidWcgaXMgaW4gdGhlcmUuCgpW ZXJ5IGxpa2VseS4KCj4gCj4gTm90ZSB0aGF0IEkgc2VlIGxvdHMgb2YgImZhbGxvY2F0ZSgpIGZh aWxlZCIgZmFpbHVyZXMgaW4gZ3VwX2xvbmd0ZXJtIHdoZW4gcnVubmluZyBvbiBvdXIgQ0kgc3lz dGVtLiBUaGlzIGlzIGEgY29tcGxldGVseSBkaWZmZXJlbnQgc2V0dXA7IFJlYWwgSFcgd2l0aCBM aW51eCBydW5uaW5nIGJhcmUgbWV0YWwgdXNpbmcgYW4gTkZTIHJvb3Rmcy4gSSdtIG5vdCBzdXJl IGlmIHRoaXMgaXMgcmVsYXRlZC4gTG9ncyBzaG93IGl0IGZhaWxpbmcgY29uc2lzdGVudGx5IGZv ciB0aGUgInRtcGZpbGUiIGFuZCAibG9jYWwgdG1wZmlsZSIgdGVzdCBjb25maWdzLiBJIGFsc28g c2VlIGEgY291cGxlIG9mIHRoZXNlIGZhaWxzIGluIHRoZSBjb3cgdGVzdHMuCgpXaGF0IGlzIHRo ZSBmYWxsb2NhdGUoKSBlcnJubyB5b3UgYXJlIGdldHRpbmc/IHN0cmFjZSBsb2cgd291bGQgaGVs cCAodG8gCnNlZSBpZiBzdGF0ZnMgYWxzbyBmYWlscyBhbHJlYWR5KSEgTGlrZWx5IGEgc2ltaWxh ciBORlMgaXNzdWUuCgoKLS0gCkNoZWVycywKCkRhdmlkIC8gZGhpbGRlbmIKCgpfX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1h aWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xp c3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FDD512FF67 for ; Tue, 9 Apr 2024 14:29:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712672995; cv=none; b=uqWgdeAtZf6rOzSp4sqruT+gvhKgh/mwPtfy94NXkEAhRWKf3ZAtRxl4+b/5dI1VN+aPlGnfxBcDwrtV6DUii+Fsh18NbZTPlOYqmAcDGHZGHxbyTjJMO3syHUpsGfcU1j8yZsuY9dcjlY1mN4o+oR5OAlpo+wcBiKvyyEh0mn4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712672995; c=relaxed/simple; bh=Z6b2MbrbT8ld6BFHGwVcVdJMPJOykt+9dgbOtxM03xc=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=hiER6I3BzPn0PPoya677i5RYZ8GX5pPEi+F1qGSeoAhXOy6Ord1gDCklAOXt3wofa/94q+g1laii1UX5/dAbA0GmuT3zP4kFPTkpKZLsErNcZmPhU/nq13+uHssVnujxi4NEiUDxuPpOrZMoYYiiIIKTa7vbnvf6lHACZCrjhTM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GAt6eymV; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GAt6eymV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712672991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=uSy4hmOLYZFzlKt9kcrYgfbIexHYMkegeYTn4mQv5d8=; b=GAt6eymV/8K7DDEk8VcPgxatvtMmLPd6bTA43LeZPkIkSwcgJlixCtW61CNn8m2E0DgE/a xhDFFpK8kpiCkTITJJJUz52v8l1+4n81c7eZwH6+ev848OQXWcjk1t9HE8ppc9kB9KPlC+ fSfEtLud+CkZ70ZL0V73EKCE4gZwZIo= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-33-aio_mmNjNHele97r3Pnmtw-1; Tue, 09 Apr 2024 10:29:50 -0400 X-MC-Unique: aio_mmNjNHele97r3Pnmtw-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-414aa7bd274so33383305e9.1 for ; Tue, 09 Apr 2024 07:29:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712672989; x=1713277789; h=content-transfer-encoding:in-reply-to:organization:autocrypt :content-language:from:references:cc:to:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=uSy4hmOLYZFzlKt9kcrYgfbIexHYMkegeYTn4mQv5d8=; b=HAmyjGuC0khpBU0zumo0UApf7guXr60OGLb0yxJKfdKlcFFOm0orbUySTGDcgudRui hqgfdBV2IXWf6ylY4+CoQoHNzot68T/942VOR2IW1mEas3CCVTdme9bietm4bsGwc1Of IZEU+H7t1yoG5LOz0YJZOZX+vyjuFuHB21S7JLCo3tMng0VVzniJFcPIik8359Yn9fhk LTivt9Tdg+C1nMmnziY0tjItXgY1227Ytss7X94yNEJ4jyB6H9sGq7aLUDUE1o1Dkg0E K+v4wVBurXjxhKGjeNAG1sSYB59vX3iX8FxCSvRZPDCgozSRWozCWmqnMxUWSIbK3Yw1 fn7A== X-Forwarded-Encrypted: i=1; AJvYcCX/C8vR9wWRiOReShC+BR8o2FUePbcXIWlIEKvp3U/6mZt0EopG+vD6uzheRPhOPQ7mEhNL8CN+qLUuOwfjTN81q3amc4WRcq2//xkb X-Gm-Message-State: AOJu0YzbTmup+DjAPzQ6Xljm2O1P+TaZhRyEa5ivP2GOH7mJwGUpxMa4 WXtWsVkCd9FL/6dg/i6rYIzVLJnOojAMYJfBiOQcsXRt76qbQy6GxfTbCopNcHAAC+TRrPKyA1h rSa3jS1DXcjLJtU4q2KD6KlIrcaJXy6nWbqE3QxdaJMckWrF17yYQ/gElKCXd8w== X-Received: by 2002:a05:600c:1f81:b0:416:1c9f:3a3f with SMTP id je1-20020a05600c1f8100b004161c9f3a3fmr8161229wmb.9.1712672989219; Tue, 09 Apr 2024 07:29:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFuVQgn4xrJl3abG1X8zRrA4z6AnUi7eOyhM0YvqA/qEe3DHF7sDFrxH4yOljhuuybBI/bKwA== X-Received: by 2002:a05:600c:1f81:b0:416:1c9f:3a3f with SMTP id je1-20020a05600c1f8100b004161c9f3a3fmr8161206wmb.9.1712672988730; Tue, 09 Apr 2024 07:29:48 -0700 (PDT) Received: from ?IPV6:2003:cb:c70a:be00:a285:bc76:307d:4eaa? (p200300cbc70abe00a285bc76307d4eaa.dip0.t-ipconnect.de. [2003:cb:c70a:be00:a285:bc76:307d:4eaa]) by smtp.gmail.com with ESMTPSA id bi8-20020a05600c3d8800b00416b92dceaesm859398wmb.11.2024.04.09.07.29.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 09 Apr 2024 07:29:48 -0700 (PDT) Message-ID: Date: Tue, 9 Apr 2024 16:29:47 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 0/4] Speed up boot with faster linear map creation To: Ryan Roberts , Itaru Kitayama Cc: Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Donald Dutile , Eric Chanudet , Linux ARM , "linux-kernel@vger.kernel.org" References: <20240404143308.2224141-1-ryan.roberts@arm.com> <533adb77-8c2b-40db-84cb-88de77ab92bb@arm.com> <1d5abb48-08a8-4d83-a681-6915bc7b6907@arm.com> <268FBD1C-B102-4726-A7F4-1125123BDA7A@linux.dev> <5e4dc2fe-2945-4fc5-a533-c8b2d04668a0@redhat.com> From: David Hildenbrand Content-Language: en-US Autocrypt: addr=david@redhat.com; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt WNyWQQ== Organization: Red Hat In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 09.04.24 16:13, Ryan Roberts wrote: > On 09/04/2024 12:51, David Hildenbrand wrote: >> On 09.04.24 13:29, David Hildenbrand wrote: >>> On 09.04.24 13:22, David Hildenbrand wrote: >>>> On 09.04.24 12:13, Itaru Kitayama wrote: >>>>> >>>>> >>>>>> On Apr 9, 2024, at 19:04, Ryan Roberts wrote: >>>>>> >>>>>> On 09/04/2024 01:10, Itaru Kitayama wrote: >>>>>>> Hi Ryan, >>>>>>> >>>>>>>> On Apr 8, 2024, at 16:30, Ryan Roberts wrote: >>>>>>>> >>>>>>>> On 06/04/2024 11:31, Itaru Kitayama wrote: >>>>>>>>> Hi Ryan, >>>>>>>>> >>>>>>>>> On Sat, Apr 06, 2024 at 09:32:34AM +0100, Ryan Roberts wrote: >>>>>>>>>> Hi Itaru, >>>>>>>>>> >>>>>>>>>> On 05/04/2024 08:39, Itaru Kitayama wrote: >>>>>>>>>>> On Thu, Apr 04, 2024 at 03:33:04PM +0100, Ryan Roberts wrote: >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> It turns out that creating the linear map can take a significant >>>>>>>>>>>> proportion of >>>>>>>>>>>> the total boot time, especially when rodata=full. And most of the >>>>>>>>>>>> time is spent >>>>>>>>>>>> waiting on superfluous tlb invalidation and memory barriers. This >>>>>>>>>>>> series reworks >>>>>>>>>>>> the kernel pgtable generation code to significantly reduce the number >>>>>>>>>>>> of those >>>>>>>>>>>> TLBIs, ISBs and DSBs. See each patch for details. >>>>>>>>>>>> >>>>>>>>>>>> The below shows the execution time of map_mem() across a couple of >>>>>>>>>>>> different >>>>>>>>>>>> systems with different RAM configurations. We measure after applying >>>>>>>>>>>> each patch >>>>>>>>>>>> and show the improvement relative to base (v6.9-rc2): >>>>>>>>>>>> >>>>>>>>>>>>                   | Apple M2 VM | Ampere Altra| Ampere Altra| Ampere >>>>>>>>>>>> Altra >>>>>>>>>>>>                   | VM, 16G     | VM, 64G     | VM, 256G    | Metal, >>>>>>>>>>>> 512G >>>>>>>>>>>> ---------------|-------------|-------------|-------------|------------- >>>>>>>>>>>>                   |   ms    (%) |   ms    (%) |   ms    (%) | >>>>>>>>>>>> ms    (%) >>>>>>>>>>>> ---------------|-------------|-------------|-------------|------------- >>>>>>>>>>>> base           |  153   (0%) | 2227   (0%) | 8798   (0%) | 17442   (0%) >>>>>>>>>>>> no-cont-remap  |   77 (-49%) |  431 (-81%) | 1727 (-80%) |  3796 (-78%) >>>>>>>>>>>> batch-barriers |   13 (-92%) |  162 (-93%) |  655 (-93%) |  1656 (-91%) >>>>>>>>>>>> no-alloc-remap |   11 (-93%) |  109 (-95%) |  449 (-95%) |  1257 (-93%) >>>>>>>>>>>> lazy-unmap     |    6 (-96%) |   61 (-97%) |  257 (-97%) |   838 (-95%) >>>>>>>>>>>> >>>>>>>>>>>> This series applies on top of v6.9-rc2. All mm selftests pass. I've >>>>>>>>>>>> compile and >>>>>>>>>>>> boot tested various PAGE_SIZE and VA size configs. >>>>>>>>>>>> >>>>>>>>>>>> --- >>>>>>>>>>>> >>>>>>>>>>>> Changes since v1 [1] >>>>>>>>>>>> ==================== >>>>>>>>>>>> >>>>>>>>>>>>      - Added Tested-by tags (thanks to Eric and Itaru) >>>>>>>>>>>>      - Renamed ___set_pte() -> __set_pte_nosync() (per Ard) >>>>>>>>>>>>      - Reordered patches (biggest impact & least controversial first) >>>>>>>>>>>>      - Reordered alloc/map/unmap functions in mmu.c to aid reader >>>>>>>>>>>>      - pte_clear() -> __pte_clear() in clear_fixmap_nosync() >>>>>>>>>>>>      - Reverted generic p4d_index() which caused x86 build error. >>>>>>>>>>>> Replaced with >>>>>>>>>>>>        unconditional p4d_index() define under arm64. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> [1] >>>>>>>>>>>> https://lore.kernel.org/linux-arm-kernel/20240326101448.3453626-1-ryan.roberts@arm.com/ >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ryan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Ryan Roberts (4): >>>>>>>>>>>>      arm64: mm: Don't remap pgtables per-cont(pte|pmd) block >>>>>>>>>>>>      arm64: mm: Batch dsb and isb when populating pgtables >>>>>>>>>>>>      arm64: mm: Don't remap pgtables for allocate vs populate >>>>>>>>>>>>      arm64: mm: Lazily clear pte table mappings from fixmap >>>>>>>>>>>> >>>>>>>>>>>> arch/arm64/include/asm/fixmap.h  |   5 +- >>>>>>>>>>>> arch/arm64/include/asm/mmu.h     |   8 + >>>>>>>>>>>> arch/arm64/include/asm/pgtable.h |  13 +- >>>>>>>>>>>> arch/arm64/kernel/cpufeature.c   |  10 +- >>>>>>>>>>>> arch/arm64/mm/fixmap.c           |  11 + >>>>>>>>>>>> arch/arm64/mm/mmu.c              | 377 +++++++++++++++++++++++-------- >>>>>>>>>>>> 6 files changed, 319 insertions(+), 105 deletions(-) >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> 2.25.1 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I've build and boot tested the v2 on FVP, base is taken from your >>>>>>>>>>> linux-rr repo. Running run_vmtests.sh on v2 left some gup longterm not >>>>>>>>>>> oks, would you take a look at it? The mm ksefltests used is from your >>>>>>>>>>> linux-rr repo too. >>>>>>>>>> >>>>>>>>>> Thanks for taking a look at this. >>>>>>>>>> >>>>>>>>>> I can't reproduce your issue unfortunately; steps as follows on Apple >>>>>>>>>> M2 VM: >>>>>>>>>> >>>>>>>>>> Config: arm64 defconfig + the following: >>>>>>>>>> >>>>>>>>>> # Squashfs for snaps, xfs for large file folios. >>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_LZ4 >>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_LZO >>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_XZ >>>>>>>>>> ./scripts/config --enable CONFIG_SQUASHFS_ZSTD >>>>>>>>>> ./scripts/config --enable CONFIG_XFS_FS >>>>>>>>>> >>>>>>>>>> # For general mm debug. >>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM >>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_MAPLE_TREE >>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_RB >>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_PGFLAGS >>>>>>>>>> ./scripts/config --enable CONFIG_DEBUG_VM_PGTABLE >>>>>>>>>> ./scripts/config --enable CONFIG_PAGE_TABLE_CHECK >>>>>>>>>> >>>>>>>>>> # For mm selftests. >>>>>>>>>> ./scripts/config --enable CONFIG_USERFAULTFD >>>>>>>>>> ./scripts/config --enable CONFIG_TEST_VMALLOC >>>>>>>>>> ./scripts/config --enable CONFIG_GUP_TEST >>>>>>>>>> >>>>>>>>>> Running on VM with 12G memory, split across 2 (emulated) NUMA nodes >>>>>>>>>> (needed by >>>>>>>>>> some mm selftests), with kernel command line to reserve hugetlbs and other >>>>>>>>>> features required by some mm selftests: >>>>>>>>>> >>>>>>>>>> " >>>>>>>>>> transparent_hugepage=madvise earlycon root=/dev/vda2 secretmem.enable >>>>>>>>>> hugepagesz=1G hugepages=0:2,1:2 hugepagesz=32M hugepages=0:2,1:2 >>>>>>>>>> default_hugepagesz=2M hugepages=0:64,1:64 hugepagesz=64K hugepages=0:2,1:2 >>>>>>>>>> " >>>>>>>>>> >>>>>>>>>> Ubuntu userspace running off XFS rootfs. Build and run mm selftests >>>>>>>>>> from same >>>>>>>>>> git tree. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Although I don't think any of this config should make a difference to >>>>>>>>>> gup_longterm. >>>>>>>>>> >>>>>>>>>> Looks like your errors are all "ftruncate() failed". I've seen this >>>>>>>>>> problem on >>>>>>>>>> our CI system. There it is due to running the tests from NFS file >>>>>>>>>> system. What >>>>>>>>>> filesystem are you using? Perhaps you are sharing into the FVP using >>>>>>>>>> 9p? That >>>>>>>>>> might also be problematic. >>>>>>>>> >>>>>>>>> That was it. This time I booted up the kernel including your series on >>>>>>>>> QEMU on my M1 and executed the gup_longterm program without the ftruncate >>>>>>>>> failures. When testing your kernel on FVP, I was executing the script >>>>>>>>> from the FVP's host filesystem using 9p. >>>>>>>> >>>>>>>> I'm not sure exactly what the root cause is. Perhaps there isn't enough >>>>>>>> space on >>>>>>>> the disk? It might be worth enhancing the error log to provide the errno in >>>>>>>> tools/testing/selftests/mm/gup_longterm.c. >>>>>>>> >>>>>>> >>>>>>> Attached is the strace’d gup_longterm executiong log on your >>>>>>> pgtable-boot-speedup-v2 kernel. >>>>>> >>>>>> Sorry are you saying that it only fails with the pgtable-boot-speedup-v2 patch >>>>>> set applied? I thought we previously concluded that it was independent of >>>>>> that? >>>>>> I was under the impression that it was filesystem related and not something >>>>>> that >>>>>> I was planning to investigate. >>>>> >>>>> No, irrespective of the kernel, if using 9p on FVP the test program fails. >>>>> It is indeed 9p filesystem related, as I switched to using NFS all the >>>>> issues are gone. >>>> >>>> Did it never work on 9p? If so, we might have to SKIP that test. >>>> >>>> openat(AT_FDCWD, "gup_longterm.c_tmpfile_BLboOt", O_RDWR|O_CREAT|O_EXCL, >>>> 0600) = 3 >>>> unlinkat(AT_FDCWD, "gup_longterm.c_tmpfile_BLboOt", 0) = 0 >>>> fstatfs(3, 0xffffe505a840)              = -1 EOPNOTSUPP (Operation not >>>> supported) >>>> ftruncate(3, 4096)                      = -1 ENOENT (No such file or directory) >>> >>> Note: I'm wondering if the unlinkat here is the problem that makes >>> ftruncate() with 9p result in weird errors (e.g., the hypervisor >>> unlinked the file and cannot reopen it for the fstatfs/ftruncate. ... >>> which gives us weird errors here). >>> >>> Then, we should lookup the fs type in run_with_local_tmpfile() before >>> the unlink() and simply skip the test if it is 9p. >> >> The unlink with 9p most certainly was a known issue in the past: >> >> https://gitlab.com/qemu-project/qemu/-/issues/103 >> >> Maybe it's still an issue with older hypervisors (QEMU?)? Or it was never >> completely resolved? > > I believe Itaru is running on FVP (Fixed Virtual Platform - "fast model" - Arm's architecture emulator). So QEMU won't be involved here. The FVP emulates a 9p device, so perhaps the bug is in there. Very likely. > > Note that I see lots of "fallocate() failed" failures in gup_longterm when running on our CI system. This is a completely different setup; Real HW with Linux running bare metal using an NFS rootfs. I'm not sure if this is related. Logs show it failing consistently for the "tmpfile" and "local tmpfile" test configs. I also see a couple of these fails in the cow tests. What is the fallocate() errno you are getting? strace log would help (to see if statfs also fails already)! Likely a similar NFS issue. -- Cheers, David / dhildenb