From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CE9CCD4F2B for ; Fri, 22 Sep 2023 08:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232328AbjIVIgX (ORCPT ); Fri, 22 Sep 2023 04:36:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229476AbjIVIgX (ORCPT ); Fri, 22 Sep 2023 04:36:23 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 279009E; Fri, 22 Sep 2023 01:36:15 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B69FDA7; Fri, 22 Sep 2023 01:36:52 -0700 (PDT) Received: from [10.57.65.11] (unknown [10.57.65.11]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2F7F03F67D; Fri, 22 Sep 2023 01:36:09 -0700 (PDT) Message-ID: Date: Fri, 22 Sep 2023 09:36:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 3/8] riscv: hugetlb: Convert set_huge_pte_at() to take vma Content-Language: en-GB To: Alexandre Ghiti , Catalin Marinas , Will Deacon , "James E.J. Bottomley" , Helge Deller , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , "David S. Miller" , Arnd Bergmann , Mike Kravetz , Muchun Song , SeongJae Park , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Anshuman Khandual , Peter Xu , Axel Rasmussen , Qi Zheng Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org References: <20230921162007.1630149-1-ryan.roberts@arm.com> <20230921162007.1630149-4-ryan.roberts@arm.com> <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> From: Ryan Roberts In-Reply-To: <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-parisc@vger.kernel.org On 22/09/2023 08:54, Alexandre Ghiti wrote: > Hi Ryan, > > On 21/09/2023 18:20, Ryan Roberts wrote: >> In order to fix a bug, arm64 needs access to the vma inside it's >> implementation of set_huge_pte_at(). Provide for this by converting the >> mm parameter to be a vma. Any implementations that require the mm can >> access it via vma->vm_mm. >> >> This commit makes the required riscv modifications. Separate commits >> update the other arches and core code, before the actual bug is fixed in >> arm64. >> >> No behavioral changes intended. >> >> Signed-off-by: Ryan Roberts >> --- >>   arch/riscv/include/asm/hugetlb.h | 2 +- >>   arch/riscv/mm/hugetlbpage.c      | 3 ++- >>   2 files changed, 3 insertions(+), 2 deletions(-) >> >> diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h >> index 34e24f078cc1..be1ac8582bc2 100644 >> --- a/arch/riscv/include/asm/hugetlb.h >> +++ b/arch/riscv/include/asm/hugetlb.h >> @@ -17,7 +17,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, >>               pte_t *ptep, unsigned long sz); >>     #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT >> -void set_huge_pte_at(struct mm_struct *mm, >> +void set_huge_pte_at(struct vm_area_struct *vma, >>                unsigned long addr, pte_t *ptep, pte_t pte); >>     #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR >> diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c >> index 96225a8533ad..7cdbf0960772 100644 >> --- a/arch/riscv/mm/hugetlbpage.c >> +++ b/arch/riscv/mm/hugetlbpage.c >> @@ -177,11 +177,12 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int >> shift, vm_flags_t flags) >>       return entry; >>   } >>   -void set_huge_pte_at(struct mm_struct *mm, >> +void set_huge_pte_at(struct vm_area_struct *vma, >>                unsigned long addr, >>                pte_t *ptep, >>                pte_t pte) >>   { >> +    struct mm_struct *mm = vma->vm_mm; >>       int i, pte_num; >>         if (!pte_napot(pte)) { > > > You can add: > > Reviewed-by: Alexandre Ghiti Thanks! > > I realize that we may have the same issue with our contig pte implementation > (called napot in riscv) as we don't handle swap/migration entries at all. So I > guess we need something similar, and I'll implement it (unless you want to do it > of course, but I guess it's easier for me to test). Yes -I'll leave you to do the riscv part. > One (maybe stupid) question > though: wouldn't it be possible to extract the contig pte size from the value of > ptep instead of using a vma? Not for arm64: We support contpmd, pmd and contpte entries as backing for the logical huge pte, depending on size. So without the size, we can't distinguish between a coincidentally-aligned pmd entry vs a contpmd entry (which is just a fixed size block of pmd entries). Discussion with Christophe on the powerpc patch triggered some thinking; There is theoretical problem with my current approach because there is one call site in the core code that calls set_huge_pte_at(&init_mm). I've changed that to: struct vm_area_struct vma = TLB_FLUSH_VMA(&init_mm, 0); set_huge_pte_at(&vma); knowing that this will never actually get called for arm64 because we return PAGE_SIZE for arch_vmap_pte_range_map_size() and all other arches just take the mm and ignore the rest of the vma. So it's safe, but fragile. But it looks like riscv overrides arch_vmap_pte_range_map_size() and therefore the call will be made there. And if riscv also needs to determine the size from the vma, then bang. So I'm going to rework it to continue to pass the mm in, but also add a size parameter. Then it's totally safe. Will post a v2 later today. Thanks, Ryan > > Thanks, > > Alex > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A695CD4F2B for ; Fri, 22 Sep 2023 08:36:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ju5BMWdJPKS2ywfoY/A8ZYIlpIZH/+gyX+8qG0UfEb4=; b=xDJitZ/Tc42P9q 3a+bOlB+wgkRnLWIy76NvuLlM3aBgFK5W8lFqNOqWbEypAAwfaQFW9zMOvu3KyUvJrCFWmeFBM372 kf7hQg/0c8zVe6rUamWdOk6flIxLwC8NHY8Vj7PDb1duMdT/3Seneh+XIdO0TeKYGXhl3sKhdk5z9 2Qv/85J/N28gYyk1+bEtzsL8hpnmaxES+PS3LFCS8fF+7vu6JlnlsUQY/HOqVk6OY4cDrBvri2ajq CkWqc4Jjh0eOPP0wgEICgXiWDrrqWGnFiJ2wiPhhBUxocwP7oU4TbnGGEX600HFoe+BWvNvV3j8kh 9BTa13oaOsIm9OGC9nPg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qjbeD-008Qr6-0x; Fri, 22 Sep 2023 08:36:25 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qjbe8-008Qoj-2J; Fri, 22 Sep 2023 08:36:23 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B69FDA7; Fri, 22 Sep 2023 01:36:52 -0700 (PDT) Received: from [10.57.65.11] (unknown [10.57.65.11]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2F7F03F67D; Fri, 22 Sep 2023 01:36:09 -0700 (PDT) Message-ID: Date: Fri, 22 Sep 2023 09:36:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 3/8] riscv: hugetlb: Convert set_huge_pte_at() to take vma Content-Language: en-GB To: Alexandre Ghiti , Catalin Marinas , Will Deacon , "James E.J. Bottomley" , Helge Deller , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , "David S. Miller" , Arnd Bergmann , Mike Kravetz , Muchun Song , SeongJae Park , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Anshuman Khandual , Peter Xu , Axel Rasmussen , Qi Zheng Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org References: <20230921162007.1630149-1-ryan.roberts@arm.com> <20230921162007.1630149-4-ryan.roberts@arm.com> <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> From: Ryan Roberts In-Reply-To: <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230922_013620_863697_561704DE X-CRM114-Status: GOOD ( 29.02 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gMjIvMDkvMjAyMyAwODo1NCwgQWxleGFuZHJlIEdoaXRpIHdyb3RlOgo+IEhpIFJ5YW4sCj4g Cj4gT24gMjEvMDkvMjAyMyAxODoyMCwgUnlhbiBSb2JlcnRzIHdyb3RlOgo+PiBJbiBvcmRlciB0 byBmaXggYSBidWcsIGFybTY0IG5lZWRzIGFjY2VzcyB0byB0aGUgdm1hIGluc2lkZSBpdCdzCj4+ IGltcGxlbWVudGF0aW9uIG9mIHNldF9odWdlX3B0ZV9hdCgpLiBQcm92aWRlIGZvciB0aGlzIGJ5 IGNvbnZlcnRpbmcgdGhlCj4+IG1tIHBhcmFtZXRlciB0byBiZSBhIHZtYS4gQW55IGltcGxlbWVu dGF0aW9ucyB0aGF0IHJlcXVpcmUgdGhlIG1tIGNhbgo+PiBhY2Nlc3MgaXQgdmlhIHZtYS0+dm1f bW0uCj4+Cj4+IFRoaXMgY29tbWl0IG1ha2VzIHRoZSByZXF1aXJlZCByaXNjdiBtb2RpZmljYXRp b25zLiBTZXBhcmF0ZSBjb21taXRzCj4+IHVwZGF0ZSB0aGUgb3RoZXIgYXJjaGVzIGFuZCBjb3Jl IGNvZGUsIGJlZm9yZSB0aGUgYWN0dWFsIGJ1ZyBpcyBmaXhlZCBpbgo+PiBhcm02NC4KPj4KPj4g Tm8gYmVoYXZpb3JhbCBjaGFuZ2VzIGludGVuZGVkLgo+Pgo+PiBTaWduZWQtb2ZmLWJ5OiBSeWFu IFJvYmVydHMgPHJ5YW4ucm9iZXJ0c0Bhcm0uY29tPgo+PiAtLS0KPj4gwqAgYXJjaC9yaXNjdi9p bmNsdWRlL2FzbS9odWdldGxiLmggfCAyICstCj4+IMKgIGFyY2gvcmlzY3YvbW0vaHVnZXRsYnBh Z2UuY8KgwqDCoMKgwqAgfCAzICsrLQo+PiDCoCAyIGZpbGVzIGNoYW5nZWQsIDMgaW5zZXJ0aW9u cygrKSwgMiBkZWxldGlvbnMoLSkKPj4KPj4gZGlmZiAtLWdpdCBhL2FyY2gvcmlzY3YvaW5jbHVk ZS9hc20vaHVnZXRsYi5oIGIvYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9odWdldGxiLmgKPj4gaW5k ZXggMzRlMjRmMDc4Y2MxLi5iZTFhYzg1ODJiYzIgMTAwNjQ0Cj4+IC0tLSBhL2FyY2gvcmlzY3Yv aW5jbHVkZS9hc20vaHVnZXRsYi5oCj4+ICsrKyBiL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vaHVn ZXRsYi5oCj4+IEBAIC0xNyw3ICsxNyw3IEBAIHZvaWQgaHVnZV9wdGVfY2xlYXIoc3RydWN0IG1t X3N0cnVjdCAqbW0sIHVuc2lnbmVkIGxvbmcgYWRkciwKPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgcHRlX3QgKnB0ZXAsIHVuc2lnbmVkIGxvbmcgc3opOwo+PiDCoCDCoCAjZGVmaW5lIF9f SEFWRV9BUkNIX0hVR0VfU0VUX0hVR0VfUFRFX0FUCj4+IC12b2lkIHNldF9odWdlX3B0ZV9hdChz dHJ1Y3QgbW1fc3RydWN0ICptbSwKPj4gK3ZvaWQgc2V0X2h1Z2VfcHRlX2F0KHN0cnVjdCB2bV9h cmVhX3N0cnVjdCAqdm1hLAo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVk IGxvbmcgYWRkciwgcHRlX3QgKnB0ZXAsIHB0ZV90IHB0ZSk7Cj4+IMKgIMKgICNkZWZpbmUgX19I QVZFX0FSQ0hfSFVHRV9QVEVQX0dFVF9BTkRfQ0xFQVIKPj4gZGlmZiAtLWdpdCBhL2FyY2gvcmlz Y3YvbW0vaHVnZXRsYnBhZ2UuYyBiL2FyY2gvcmlzY3YvbW0vaHVnZXRsYnBhZ2UuYwo+PiBpbmRl eCA5NjIyNWE4NTMzYWQuLjdjZGJmMDk2MDc3MiAxMDA2NDQKPj4gLS0tIGEvYXJjaC9yaXNjdi9t bS9odWdldGxicGFnZS5jCj4+ICsrKyBiL2FyY2gvcmlzY3YvbW0vaHVnZXRsYnBhZ2UuYwo+PiBA QCAtMTc3LDExICsxNzcsMTIgQEAgcHRlX3QgYXJjaF9tYWtlX2h1Z2VfcHRlKHB0ZV90IGVudHJ5 LCB1bnNpZ25lZCBpbnQKPj4gc2hpZnQsIHZtX2ZsYWdzX3QgZmxhZ3MpCj4+IMKgwqDCoMKgwqAg cmV0dXJuIGVudHJ5Owo+PiDCoCB9Cj4+IMKgIC12b2lkIHNldF9odWdlX3B0ZV9hdChzdHJ1Y3Qg bW1fc3RydWN0ICptbSwKPj4gK3ZvaWQgc2V0X2h1Z2VfcHRlX2F0KHN0cnVjdCB2bV9hcmVhX3N0 cnVjdCAqdm1hLAo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVkIGxvbmcg YWRkciwKPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwdGVfdCAqcHRlcCwKPj4gwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwdGVfdCBwdGUpCj4+IMKgIHsKPj4gK8KgwqDCoCBz dHJ1Y3QgbW1fc3RydWN0ICptbSA9IHZtYS0+dm1fbW07Cj4+IMKgwqDCoMKgwqAgaW50IGksIHB0 ZV9udW07Cj4+IMKgIMKgwqDCoMKgwqAgaWYgKCFwdGVfbmFwb3QocHRlKSkgewo+IAo+IAo+IFlv dSBjYW4gYWRkOgo+IAo+IFJldmlld2VkLWJ5OiBBbGV4YW5kcmUgR2hpdGkgPGFsZXhnaGl0aUBy aXZvc2luYy5jb20+CgpUaGFua3MhCgo+IAo+IEkgcmVhbGl6ZSB0aGF0IHdlIG1heSBoYXZlIHRo ZSBzYW1lIGlzc3VlIHdpdGggb3VyIGNvbnRpZyBwdGUgaW1wbGVtZW50YXRpb24KPiAoY2FsbGVk IG5hcG90IGluIHJpc2N2KSBhcyB3ZSBkb24ndCBoYW5kbGUgc3dhcC9taWdyYXRpb24gZW50cmll cyBhdCBhbGwuIFNvIEkKPiBndWVzcyB3ZSBuZWVkIHNvbWV0aGluZyBzaW1pbGFyLCBhbmQgSSds bCBpbXBsZW1lbnQgaXQgKHVubGVzcyB5b3Ugd2FudCB0byBkbyBpdAo+IG9mIGNvdXJzZSwgYnV0 IEkgZ3Vlc3MgaXQncyBlYXNpZXIgZm9yIG1lIHRvIHRlc3QpLiAKClllcyAtSSdsbCBsZWF2ZSB5 b3UgdG8gZG8gdGhlIHJpc2N2IHBhcnQuCgo+IE9uZSAobWF5YmUgc3R1cGlkKSBxdWVzdGlvbgo+ IHRob3VnaDogd291bGRuJ3QgaXQgYmUgcG9zc2libGUgdG8gZXh0cmFjdCB0aGUgY29udGlnIHB0 ZSBzaXplIGZyb20gdGhlIHZhbHVlIG9mCj4gcHRlcCBpbnN0ZWFkIG9mIHVzaW5nIGEgdm1hPwoK Tm90IGZvciBhcm02NDogV2Ugc3VwcG9ydCBjb250cG1kLCBwbWQgYW5kIGNvbnRwdGUgZW50cmll cyBhcyBiYWNraW5nIGZvciB0aGUKbG9naWNhbCBodWdlIHB0ZSwgZGVwZW5kaW5nIG9uIHNpemUu IFNvIHdpdGhvdXQgdGhlIHNpemUsIHdlIGNhbid0IGRpc3Rpbmd1aXNoCmJldHdlZW4gYSBjb2lu Y2lkZW50YWxseS1hbGlnbmVkIHBtZCBlbnRyeSB2cyBhIGNvbnRwbWQgZW50cnkgKHdoaWNoIGlz IGp1c3QgYQpmaXhlZCBzaXplIGJsb2NrIG9mIHBtZCBlbnRyaWVzKS4KCkRpc2N1c3Npb24gd2l0 aCBDaHJpc3RvcGhlIG9uIHRoZSBwb3dlcnBjIHBhdGNoIHRyaWdnZXJlZCBzb21lIHRoaW5raW5n OyBUaGVyZQppcyB0aGVvcmV0aWNhbCBwcm9ibGVtIHdpdGggbXkgY3VycmVudCBhcHByb2FjaCBi ZWNhdXNlIHRoZXJlIGlzIG9uZSBjYWxsIHNpdGUKaW4gdGhlIGNvcmUgY29kZSB0aGF0IGNhbGxz IHNldF9odWdlX3B0ZV9hdCgmaW5pdF9tbSkuIEkndmUgY2hhbmdlZCB0aGF0IHRvOgoKICBzdHJ1 Y3Qgdm1fYXJlYV9zdHJ1Y3Qgdm1hID0gVExCX0ZMVVNIX1ZNQSgmaW5pdF9tbSwgMCk7CiAgc2V0 X2h1Z2VfcHRlX2F0KCZ2bWEpOwoKa25vd2luZyB0aGF0IHRoaXMgd2lsbCBuZXZlciBhY3R1YWxs eSBnZXQgY2FsbGVkIGZvciBhcm02NCBiZWNhdXNlIHdlIHJldHVybgpQQUdFX1NJWkUgZm9yIGFy Y2hfdm1hcF9wdGVfcmFuZ2VfbWFwX3NpemUoKSBhbmQgYWxsIG90aGVyIGFyY2hlcyBqdXN0IHRh a2UgdGhlCm1tIGFuZCBpZ25vcmUgdGhlIHJlc3Qgb2YgdGhlIHZtYS4gU28gaXQncyBzYWZlLCBi dXQgZnJhZ2lsZS4KCkJ1dCBpdCBsb29rcyBsaWtlIHJpc2N2IG92ZXJyaWRlcyBhcmNoX3ZtYXBf cHRlX3JhbmdlX21hcF9zaXplKCkgYW5kIHRoZXJlZm9yZQp0aGUgY2FsbCB3aWxsIGJlIG1hZGUg dGhlcmUuIEFuZCBpZiByaXNjdiBhbHNvIG5lZWRzIHRvIGRldGVybWluZSB0aGUgc2l6ZSBmcm9t CnRoZSB2bWEsIHRoZW4gYmFuZy4KClNvIEknbSBnb2luZyB0byByZXdvcmsgaXQgdG8gY29udGlu dWUgdG8gcGFzcyB0aGUgbW0gaW4sIGJ1dCBhbHNvIGFkZCBhIHNpemUKcGFyYW1ldGVyLiBUaGVu IGl0J3MgdG90YWxseSBzYWZlLiBXaWxsIHBvc3QgYSB2MiBsYXRlciB0b2RheS4KClRoYW5rcywK UnlhbgoKPiAKPiBUaGFua3MsCj4gCj4gQWxleAo+IAoKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LXJpc2N2IG1haWxpbmcgbGlzdApsaW51eC1y aXNjdkBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxt YW4vbGlzdGluZm8vbGludXgtcmlzY3YK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 687E6CD4F2B for ; Fri, 22 Sep 2023 08:40:50 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4RsQhd1FZDz3jTC for ; Fri, 22 Sep 2023 18:40:49 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=arm.com (client-ip=217.140.110.172; helo=foss.arm.com; envelope-from=ryan.roberts@arm.com; receiver=lists.ozlabs.org) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lists.ozlabs.org (Postfix) with ESMTP id 4RsQc26csKz3ffb for ; Fri, 22 Sep 2023 18:36:48 +1000 (AEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B69FDA7; Fri, 22 Sep 2023 01:36:52 -0700 (PDT) Received: from [10.57.65.11] (unknown [10.57.65.11]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2F7F03F67D; Fri, 22 Sep 2023 01:36:09 -0700 (PDT) Message-ID: Date: Fri, 22 Sep 2023 09:36:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 3/8] riscv: hugetlb: Convert set_huge_pte_at() to take vma Content-Language: en-GB To: Alexandre Ghiti , Catalin Marinas , Will Deacon , "James E.J. Bottomley" , Helge Deller , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , "David S. Miller" , Arnd Bergmann , Mike Kravetz , Muchun Song , SeongJae Park , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Anshuman Khandual , Peter Xu , Axel Rasmussen , Qi Zheng References: <20230921162007.1630149-1-ryan.roberts@arm.com> <20230921162007.1630149-4-ryan.roberts@arm.com> <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> From: Ryan Roberts In-Reply-To: <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-s390@vger.kernel.org, linux-parisc@vger.kernel.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, linux-mm@kvack.org, sparclinux@vger.kernel.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On 22/09/2023 08:54, Alexandre Ghiti wrote: > Hi Ryan, > > On 21/09/2023 18:20, Ryan Roberts wrote: >> In order to fix a bug, arm64 needs access to the vma inside it's >> implementation of set_huge_pte_at(). Provide for this by converting the >> mm parameter to be a vma. Any implementations that require the mm can >> access it via vma->vm_mm. >> >> This commit makes the required riscv modifications. Separate commits >> update the other arches and core code, before the actual bug is fixed in >> arm64. >> >> No behavioral changes intended. >> >> Signed-off-by: Ryan Roberts >> --- >>   arch/riscv/include/asm/hugetlb.h | 2 +- >>   arch/riscv/mm/hugetlbpage.c      | 3 ++- >>   2 files changed, 3 insertions(+), 2 deletions(-) >> >> diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h >> index 34e24f078cc1..be1ac8582bc2 100644 >> --- a/arch/riscv/include/asm/hugetlb.h >> +++ b/arch/riscv/include/asm/hugetlb.h >> @@ -17,7 +17,7 @@ void huge_pte_clear(struct mm_struct *mm, unsigned long addr, >>               pte_t *ptep, unsigned long sz); >>     #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT >> -void set_huge_pte_at(struct mm_struct *mm, >> +void set_huge_pte_at(struct vm_area_struct *vma, >>                unsigned long addr, pte_t *ptep, pte_t pte); >>     #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR >> diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c >> index 96225a8533ad..7cdbf0960772 100644 >> --- a/arch/riscv/mm/hugetlbpage.c >> +++ b/arch/riscv/mm/hugetlbpage.c >> @@ -177,11 +177,12 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int >> shift, vm_flags_t flags) >>       return entry; >>   } >>   -void set_huge_pte_at(struct mm_struct *mm, >> +void set_huge_pte_at(struct vm_area_struct *vma, >>                unsigned long addr, >>                pte_t *ptep, >>                pte_t pte) >>   { >> +    struct mm_struct *mm = vma->vm_mm; >>       int i, pte_num; >>         if (!pte_napot(pte)) { > > > You can add: > > Reviewed-by: Alexandre Ghiti Thanks! > > I realize that we may have the same issue with our contig pte implementation > (called napot in riscv) as we don't handle swap/migration entries at all. So I > guess we need something similar, and I'll implement it (unless you want to do it > of course, but I guess it's easier for me to test). Yes -I'll leave you to do the riscv part. > One (maybe stupid) question > though: wouldn't it be possible to extract the contig pte size from the value of > ptep instead of using a vma? Not for arm64: We support contpmd, pmd and contpte entries as backing for the logical huge pte, depending on size. So without the size, we can't distinguish between a coincidentally-aligned pmd entry vs a contpmd entry (which is just a fixed size block of pmd entries). Discussion with Christophe on the powerpc patch triggered some thinking; There is theoretical problem with my current approach because there is one call site in the core code that calls set_huge_pte_at(&init_mm). I've changed that to: struct vm_area_struct vma = TLB_FLUSH_VMA(&init_mm, 0); set_huge_pte_at(&vma); knowing that this will never actually get called for arm64 because we return PAGE_SIZE for arch_vmap_pte_range_map_size() and all other arches just take the mm and ignore the rest of the vma. So it's safe, but fragile. But it looks like riscv overrides arch_vmap_pte_range_map_size() and therefore the call will be made there. And if riscv also needs to determine the size from the vma, then bang. So I'm going to rework it to continue to pass the mm in, but also add a size parameter. Then it's totally safe. Will post a v2 later today. Thanks, Ryan > > Thanks, > > Alex > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDEFCCD4F32 for ; Fri, 22 Sep 2023 08:36:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GZvlRbK4qu5vzsDyB/N5p3KkMuP7n0ozlBCsCOA6kD4=; b=XT5yF/HOjPcI/U JZDAquzmgYD+AWMY7klZymu9x2Agh+Ps6DRBvd/JaGr9dg6S+jVoX6d4g4yTaKOxEi5KyfpgqEbA4 AkKYczMRb/HgYT/J+9LopOk89Y8jGNmBQajy585I/N6mzze79d3Zes9irhihDWmXCjpJZUOuNZXrX Nr9YsQuDfvqC4OsevxuND/pxwRisaCsri/trnZqGGmvNN45HE8uy0eVE5HvouP7tjADkLkyI37CIO PX0hMAQOakMeo5+Wz9KFc5020sV9TbTCHJ9JL0dZbjldWgO3JU0ni/pJL4ndiNJGtIEzJp0ZOziGu Llz+IV/LOmo4ufD87e0A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qjbeC-008Qqy-2e; Fri, 22 Sep 2023 08:36:24 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qjbe8-008Qoj-2J; Fri, 22 Sep 2023 08:36:23 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8B69FDA7; Fri, 22 Sep 2023 01:36:52 -0700 (PDT) Received: from [10.57.65.11] (unknown [10.57.65.11]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2F7F03F67D; Fri, 22 Sep 2023 01:36:09 -0700 (PDT) Message-ID: Date: Fri, 22 Sep 2023 09:36:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 3/8] riscv: hugetlb: Convert set_huge_pte_at() to take vma Content-Language: en-GB To: Alexandre Ghiti , Catalin Marinas , Will Deacon , "James E.J. Bottomley" , Helge Deller , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Gerald Schaefer , "David S. Miller" , Arnd Bergmann , Mike Kravetz , Muchun Song , SeongJae Park , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Anshuman Khandual , Peter Xu , Axel Rasmussen , Qi Zheng Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org References: <20230921162007.1630149-1-ryan.roberts@arm.com> <20230921162007.1630149-4-ryan.roberts@arm.com> <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> From: Ryan Roberts In-Reply-To: <7bbceed4-c5f6-42d4-5d94-060032b73385@ghiti.fr> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230922_013620_863697_561704DE X-CRM114-Status: GOOD ( 29.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMjIvMDkvMjAyMyAwODo1NCwgQWxleGFuZHJlIEdoaXRpIHdyb3RlOgo+IEhpIFJ5YW4sCj4g Cj4gT24gMjEvMDkvMjAyMyAxODoyMCwgUnlhbiBSb2JlcnRzIHdyb3RlOgo+PiBJbiBvcmRlciB0 byBmaXggYSBidWcsIGFybTY0IG5lZWRzIGFjY2VzcyB0byB0aGUgdm1hIGluc2lkZSBpdCdzCj4+ IGltcGxlbWVudGF0aW9uIG9mIHNldF9odWdlX3B0ZV9hdCgpLiBQcm92aWRlIGZvciB0aGlzIGJ5 IGNvbnZlcnRpbmcgdGhlCj4+IG1tIHBhcmFtZXRlciB0byBiZSBhIHZtYS4gQW55IGltcGxlbWVu dGF0aW9ucyB0aGF0IHJlcXVpcmUgdGhlIG1tIGNhbgo+PiBhY2Nlc3MgaXQgdmlhIHZtYS0+dm1f bW0uCj4+Cj4+IFRoaXMgY29tbWl0IG1ha2VzIHRoZSByZXF1aXJlZCByaXNjdiBtb2RpZmljYXRp b25zLiBTZXBhcmF0ZSBjb21taXRzCj4+IHVwZGF0ZSB0aGUgb3RoZXIgYXJjaGVzIGFuZCBjb3Jl IGNvZGUsIGJlZm9yZSB0aGUgYWN0dWFsIGJ1ZyBpcyBmaXhlZCBpbgo+PiBhcm02NC4KPj4KPj4g Tm8gYmVoYXZpb3JhbCBjaGFuZ2VzIGludGVuZGVkLgo+Pgo+PiBTaWduZWQtb2ZmLWJ5OiBSeWFu IFJvYmVydHMgPHJ5YW4ucm9iZXJ0c0Bhcm0uY29tPgo+PiAtLS0KPj4gwqAgYXJjaC9yaXNjdi9p bmNsdWRlL2FzbS9odWdldGxiLmggfCAyICstCj4+IMKgIGFyY2gvcmlzY3YvbW0vaHVnZXRsYnBh Z2UuY8KgwqDCoMKgwqAgfCAzICsrLQo+PiDCoCAyIGZpbGVzIGNoYW5nZWQsIDMgaW5zZXJ0aW9u cygrKSwgMiBkZWxldGlvbnMoLSkKPj4KPj4gZGlmZiAtLWdpdCBhL2FyY2gvcmlzY3YvaW5jbHVk ZS9hc20vaHVnZXRsYi5oIGIvYXJjaC9yaXNjdi9pbmNsdWRlL2FzbS9odWdldGxiLmgKPj4gaW5k ZXggMzRlMjRmMDc4Y2MxLi5iZTFhYzg1ODJiYzIgMTAwNjQ0Cj4+IC0tLSBhL2FyY2gvcmlzY3Yv aW5jbHVkZS9hc20vaHVnZXRsYi5oCj4+ICsrKyBiL2FyY2gvcmlzY3YvaW5jbHVkZS9hc20vaHVn ZXRsYi5oCj4+IEBAIC0xNyw3ICsxNyw3IEBAIHZvaWQgaHVnZV9wdGVfY2xlYXIoc3RydWN0IG1t X3N0cnVjdCAqbW0sIHVuc2lnbmVkIGxvbmcgYWRkciwKPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgcHRlX3QgKnB0ZXAsIHVuc2lnbmVkIGxvbmcgc3opOwo+PiDCoCDCoCAjZGVmaW5lIF9f SEFWRV9BUkNIX0hVR0VfU0VUX0hVR0VfUFRFX0FUCj4+IC12b2lkIHNldF9odWdlX3B0ZV9hdChz dHJ1Y3QgbW1fc3RydWN0ICptbSwKPj4gK3ZvaWQgc2V0X2h1Z2VfcHRlX2F0KHN0cnVjdCB2bV9h cmVhX3N0cnVjdCAqdm1hLAo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVk IGxvbmcgYWRkciwgcHRlX3QgKnB0ZXAsIHB0ZV90IHB0ZSk7Cj4+IMKgIMKgICNkZWZpbmUgX19I QVZFX0FSQ0hfSFVHRV9QVEVQX0dFVF9BTkRfQ0xFQVIKPj4gZGlmZiAtLWdpdCBhL2FyY2gvcmlz Y3YvbW0vaHVnZXRsYnBhZ2UuYyBiL2FyY2gvcmlzY3YvbW0vaHVnZXRsYnBhZ2UuYwo+PiBpbmRl eCA5NjIyNWE4NTMzYWQuLjdjZGJmMDk2MDc3MiAxMDA2NDQKPj4gLS0tIGEvYXJjaC9yaXNjdi9t bS9odWdldGxicGFnZS5jCj4+ICsrKyBiL2FyY2gvcmlzY3YvbW0vaHVnZXRsYnBhZ2UuYwo+PiBA QCAtMTc3LDExICsxNzcsMTIgQEAgcHRlX3QgYXJjaF9tYWtlX2h1Z2VfcHRlKHB0ZV90IGVudHJ5 LCB1bnNpZ25lZCBpbnQKPj4gc2hpZnQsIHZtX2ZsYWdzX3QgZmxhZ3MpCj4+IMKgwqDCoMKgwqAg cmV0dXJuIGVudHJ5Owo+PiDCoCB9Cj4+IMKgIC12b2lkIHNldF9odWdlX3B0ZV9hdChzdHJ1Y3Qg bW1fc3RydWN0ICptbSwKPj4gK3ZvaWQgc2V0X2h1Z2VfcHRlX2F0KHN0cnVjdCB2bV9hcmVhX3N0 cnVjdCAqdm1hLAo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVuc2lnbmVkIGxvbmcg YWRkciwKPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwdGVfdCAqcHRlcCwKPj4gwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwdGVfdCBwdGUpCj4+IMKgIHsKPj4gK8KgwqDCoCBz dHJ1Y3QgbW1fc3RydWN0ICptbSA9IHZtYS0+dm1fbW07Cj4+IMKgwqDCoMKgwqAgaW50IGksIHB0 ZV9udW07Cj4+IMKgIMKgwqDCoMKgwqAgaWYgKCFwdGVfbmFwb3QocHRlKSkgewo+IAo+IAo+IFlv dSBjYW4gYWRkOgo+IAo+IFJldmlld2VkLWJ5OiBBbGV4YW5kcmUgR2hpdGkgPGFsZXhnaGl0aUBy aXZvc2luYy5jb20+CgpUaGFua3MhCgo+IAo+IEkgcmVhbGl6ZSB0aGF0IHdlIG1heSBoYXZlIHRo ZSBzYW1lIGlzc3VlIHdpdGggb3VyIGNvbnRpZyBwdGUgaW1wbGVtZW50YXRpb24KPiAoY2FsbGVk IG5hcG90IGluIHJpc2N2KSBhcyB3ZSBkb24ndCBoYW5kbGUgc3dhcC9taWdyYXRpb24gZW50cmll cyBhdCBhbGwuIFNvIEkKPiBndWVzcyB3ZSBuZWVkIHNvbWV0aGluZyBzaW1pbGFyLCBhbmQgSSds bCBpbXBsZW1lbnQgaXQgKHVubGVzcyB5b3Ugd2FudCB0byBkbyBpdAo+IG9mIGNvdXJzZSwgYnV0 IEkgZ3Vlc3MgaXQncyBlYXNpZXIgZm9yIG1lIHRvIHRlc3QpLiAKClllcyAtSSdsbCBsZWF2ZSB5 b3UgdG8gZG8gdGhlIHJpc2N2IHBhcnQuCgo+IE9uZSAobWF5YmUgc3R1cGlkKSBxdWVzdGlvbgo+ IHRob3VnaDogd291bGRuJ3QgaXQgYmUgcG9zc2libGUgdG8gZXh0cmFjdCB0aGUgY29udGlnIHB0 ZSBzaXplIGZyb20gdGhlIHZhbHVlIG9mCj4gcHRlcCBpbnN0ZWFkIG9mIHVzaW5nIGEgdm1hPwoK Tm90IGZvciBhcm02NDogV2Ugc3VwcG9ydCBjb250cG1kLCBwbWQgYW5kIGNvbnRwdGUgZW50cmll cyBhcyBiYWNraW5nIGZvciB0aGUKbG9naWNhbCBodWdlIHB0ZSwgZGVwZW5kaW5nIG9uIHNpemUu IFNvIHdpdGhvdXQgdGhlIHNpemUsIHdlIGNhbid0IGRpc3Rpbmd1aXNoCmJldHdlZW4gYSBjb2lu Y2lkZW50YWxseS1hbGlnbmVkIHBtZCBlbnRyeSB2cyBhIGNvbnRwbWQgZW50cnkgKHdoaWNoIGlz IGp1c3QgYQpmaXhlZCBzaXplIGJsb2NrIG9mIHBtZCBlbnRyaWVzKS4KCkRpc2N1c3Npb24gd2l0 aCBDaHJpc3RvcGhlIG9uIHRoZSBwb3dlcnBjIHBhdGNoIHRyaWdnZXJlZCBzb21lIHRoaW5raW5n OyBUaGVyZQppcyB0aGVvcmV0aWNhbCBwcm9ibGVtIHdpdGggbXkgY3VycmVudCBhcHByb2FjaCBi ZWNhdXNlIHRoZXJlIGlzIG9uZSBjYWxsIHNpdGUKaW4gdGhlIGNvcmUgY29kZSB0aGF0IGNhbGxz IHNldF9odWdlX3B0ZV9hdCgmaW5pdF9tbSkuIEkndmUgY2hhbmdlZCB0aGF0IHRvOgoKICBzdHJ1 Y3Qgdm1fYXJlYV9zdHJ1Y3Qgdm1hID0gVExCX0ZMVVNIX1ZNQSgmaW5pdF9tbSwgMCk7CiAgc2V0 X2h1Z2VfcHRlX2F0KCZ2bWEpOwoKa25vd2luZyB0aGF0IHRoaXMgd2lsbCBuZXZlciBhY3R1YWxs eSBnZXQgY2FsbGVkIGZvciBhcm02NCBiZWNhdXNlIHdlIHJldHVybgpQQUdFX1NJWkUgZm9yIGFy Y2hfdm1hcF9wdGVfcmFuZ2VfbWFwX3NpemUoKSBhbmQgYWxsIG90aGVyIGFyY2hlcyBqdXN0IHRh a2UgdGhlCm1tIGFuZCBpZ25vcmUgdGhlIHJlc3Qgb2YgdGhlIHZtYS4gU28gaXQncyBzYWZlLCBi dXQgZnJhZ2lsZS4KCkJ1dCBpdCBsb29rcyBsaWtlIHJpc2N2IG92ZXJyaWRlcyBhcmNoX3ZtYXBf cHRlX3JhbmdlX21hcF9zaXplKCkgYW5kIHRoZXJlZm9yZQp0aGUgY2FsbCB3aWxsIGJlIG1hZGUg dGhlcmUuIEFuZCBpZiByaXNjdiBhbHNvIG5lZWRzIHRvIGRldGVybWluZSB0aGUgc2l6ZSBmcm9t CnRoZSB2bWEsIHRoZW4gYmFuZy4KClNvIEknbSBnb2luZyB0byByZXdvcmsgaXQgdG8gY29udGlu dWUgdG8gcGFzcyB0aGUgbW0gaW4sIGJ1dCBhbHNvIGFkZCBhIHNpemUKcGFyYW1ldGVyLiBUaGVu IGl0J3MgdG90YWxseSBzYWZlLiBXaWxsIHBvc3QgYSB2MiBsYXRlciB0b2RheS4KClRoYW5rcywK UnlhbgoKPiAKPiBUaGFua3MsCj4gCj4gQWxleAo+IAoKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0Cmxp bnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFk Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK