From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-172.mta1.migadu.com (out-172.mta1.migadu.com [95.215.58.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16EC336EAB2 for ; Thu, 29 Jan 2026 06:54:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769669677; cv=none; b=Z6lq2gNFFlyMHagsU5Yifg1hXTofBTSgh41C/rhcdXePj+6UUNQPS+HIVbAAIIKLpsHwmO8oJusdbDqhhHGJoRiLpN02j6XmLqJK2BtnMp0FjER1bx1e4T+70dN/4RJw2rrvvRiHYi2iRD1SSwERgAAiE2YzxXzrDAd1I1szLKE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769669677; c=relaxed/simple; bh=RbsUYwMnVebVInSib7GA/5OJfo9x/STjQcP0XPPt4mA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=ksJBVYSCao1URxbhHrgUfoaw+VzlE/XnQQteE+98OXKpuL2foEPLzAhUyLRilBcIWVt/Aic0AaW0+1YPrIwbhzgG1UQp37XExNZZBJC3tHEHagmYHMEcvh4q8vHrrB2cluxGAmIjSelbIXgiz85uZqWVPkpoLhBN7Wfd4JBeBvA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=n5geKtvs; arc=none smtp.client-ip=95.215.58.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="n5geKtvs" Message-ID: <0db8c993-a525-438f-9d7b-94f63bfa0aa4@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1769669662; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i4JRgDeBr4jab+v0AQMoJMPJjVR8MkEaj+rH1zUeTz4=; b=n5geKtvskBt5UeHtUN3tjKwhwYQQWLClgpDulimgfAcl+2JC++CzR0HifKg5bhU1Jcjsp5 j+4xd+xIwQHvnZDELrYAHClQ62HxwFa6zJbY2EgtbaVxX5Rlrx3nJYrdjqq/V5wJ1C7J8N aKBzpwi/wUAI7cpkGYqzoCq6dJaN4Ys= Date: Thu, 29 Jan 2026 14:54:07 +0800 Precedence: bulk X-Mailing-List: linux-doc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Subject: Re: [PATCHv5 11/17] mm/hugetlb: Remove fake head pages To: Kiryl Shutsemau Cc: Oscar Salvador , Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Zi Yan , Baoquan He , Michal Hocko , Johannes Weiner , Jonathan Corbet , Huacai Chen , WANG Xuerui , Palmer Dabbelt , Paul Walmsley , Albert Ou , Alexandre Ghiti , kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, Andrew Morton , David Hildenbrand , Matthew Wilcox , Usama Arif , Frank van der Linden References: <20260128135500.22121-1-kas@kernel.org> <20260128135500.22121-12-kas@kernel.org> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Muchun Song In-Reply-To: <20260128135500.22121-12-kas@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT On 2026/1/28 21:54, Kiryl Shutsemau wrote: > HugeTLB Vmemmap Optimization (HVO) reduces memory usage by freeing most > vmemmap pages for huge pages and remapping the freed range to a single > page containing the struct page metadata. > > With the new mask-based compound_info encoding (for power-of-2 struct > page sizes), all tail pages of the same order are now identical > regardless of which compound page they belong to. This means the tail > pages can be truly shared without fake heads. > > Allocate a single page of initialized tail struct pages per NUMA node > per order in the vmemmap_tails[] array in pglist_data. All huge pages of > that order on the node share this tail page, mapped read-only into their > vmemmap. The head page remains unique per huge page. > > Redefine MAX_FOLIO_ORDER using ilog2(). The define has to produce a > compile-constant as it is used to specify vmemmap_tail array size. > For some reason, compiler is not able to solve get_order() at > compile-time, but ilog2() works. > > Avoid PUD_ORDER to define MAX_FOLIO_ORDER as it adds dependency to > which generates hard-to-break include loop. > > This eliminates fake heads while maintaining the same memory savings, > and simplifies compound_head() by removing fake head detection. > > Signed-off-by: Kiryl Shutsemau > --- > include/linux/mmzone.h | 18 +++++++++++++++-- > mm/hugetlb_vmemmap.c | 36 ++++++++++++++++++++++++++++++++-- > mm/sparse-vmemmap.c | 44 ++++++++++++++++++++++++++++++++++-------- > 3 files changed, 86 insertions(+), 12 deletions(-) > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index 192143b5cdc0..698091c74dbb 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -81,13 +81,17 @@ > * currently expect (see CONFIG_HAVE_GIGANTIC_FOLIOS): with hugetlb, we expect > * no folios larger than 16 GiB on 64bit and 1 GiB on 32bit. > */ > -#define MAX_FOLIO_ORDER get_order(IS_ENABLED(CONFIG_64BIT) ? SZ_16G : SZ_1G) > +#ifdef CONFIG_64BIT > +#define MAX_FOLIO_ORDER (ilog2(SZ_16G) - PAGE_SHIFT) > +#else > +#define MAX_FOLIO_ORDER (ilog2(SZ_1G) - PAGE_SHIFT) > +#endif > #else > /* > * Without hugetlb, gigantic folios that are bigger than a single PUD are > * currently impossible. > */ > -#define MAX_FOLIO_ORDER PUD_ORDER > +#define MAX_FOLIO_ORDER (PUD_SHIFT - PAGE_SHIFT) > #endif > > #define MAX_FOLIO_NR_PAGES (1UL << MAX_FOLIO_ORDER) > @@ -1402,6 +1406,13 @@ struct memory_failure_stats { > }; > #endif > > +/* > + * vmemmap optimization (like HVO) is only possible for page orders that fill > + * two or more pages with struct pages. > + */ > +#define VMEMMAP_TAIL_MIN_ORDER (ilog2(2 * PAGE_SIZE / sizeof(struct page))) > +#define NR_VMEMMAP_TAILS (MAX_FOLIO_ORDER - VMEMMAP_TAIL_MIN_ORDER + 1) > + > /* > * On NUMA machines, each NUMA node would have a pg_data_t to describe > * it's memory layout. On UMA machines there is a single pglist_data which > @@ -1550,6 +1561,9 @@ typedef struct pglist_data { > #ifdef CONFIG_MEMORY_FAILURE > struct memory_failure_stats mf_stats; > #endif > +#ifdef CONFIG_SPARSEMEM_VMEMMAP > + unsigned long vmemmap_tails[NR_VMEMMAP_TAILS]; We should record "struct page" instead of pfn, I'll explain below. > +#endif > } pg_data_t; > > #define node_present_pages(nid) (NODE_DATA(nid)->node_present_pages) > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > index a39a301e08b9..f5f42b92dd7d 100644 > --- a/mm/hugetlb_vmemmap.c > +++ b/mm/hugetlb_vmemmap.c > @@ -19,6 +19,7 @@ > > #include > #include "hugetlb_vmemmap.h" > +#include "internal.h" > > /** > * struct vmemmap_remap_walk - walk vmemmap page table > @@ -505,6 +506,34 @@ static bool vmemmap_should_optimize_folio(const struct hstate *h, struct folio * > return true; > } > > +static struct page *vmemmap_get_tail(unsigned int order, int node) > +{ > + unsigned long pfn; > + unsigned int idx; > + struct page *tail, *p; > + > + idx = order - VMEMMAP_TAIL_MIN_ORDER; > + pfn = READ_ONCE(NODE_DATA(node)->vmemmap_tails[idx]); > + if (pfn) You’ve assumed that a valid PFN can never be zero, but that isn’t guaranteed.  If we store the `struct page` pointer instead, the issue disappears: its virtual address is never NULL. Moreover, we only convert back and forth with pfn_to_page()/page_to_pfn(); we never dereference any member of the structure, so we don’t have to care whether `struct page` has been initialized yet during early boot (it is safe for us to get page in sparse-vmemmap.c). > + return pfn_to_page(pfn); > + > + tail = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0); > + if (!tail) > + return NULL; > + > + p = page_to_virt(tail); > + for (int i = 0; i < PAGE_SIZE / sizeof(struct page); i++) > + prep_compound_tail(p + i, NULL, order); > + > + pfn = PHYS_PFN(virt_to_phys(p)); > + if (cmpxchg(&NODE_DATA(node)->vmemmap_tails[idx], 0, pfn)) { > + __free_page(tail); > + pfn = READ_ONCE(NODE_DATA(node)->vmemmap_tails[idx]); > + } > + > + return pfn_to_page(pfn); > +} > + > static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, > struct folio *folio, > struct list_head *vmemmap_pages, > @@ -520,6 +549,11 @@ static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, > if (!vmemmap_should_optimize_folio(h, folio)) > return ret; > > + nid = folio_nid(folio); > + vmemmap_tail = vmemmap_get_tail(h->order, nid); > + if (!vmemmap_tail) > + return -ENOMEM; > + > static_branch_inc(&hugetlb_optimize_vmemmap_key); > > if (flags & VMEMMAP_SYNCHRONIZE_RCU) > @@ -537,7 +571,6 @@ static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, > */ > folio_set_hugetlb_vmemmap_optimized(folio); > > - nid = folio_nid(folio); > vmemmap_head = alloc_pages_node(nid, GFP_KERNEL, 0); > if (!vmemmap_head) { > ret = -ENOMEM; > @@ -548,7 +581,6 @@ static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, > list_add(&vmemmap_head->lru, vmemmap_pages); > memmap_pages_add(1); > > - vmemmap_tail = vmemmap_head; > vmemmap_start = (unsigned long)&folio->page; > vmemmap_end = vmemmap_start + hugetlb_vmemmap_size(h); > > diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c > index 37522d6cb398..23abd06f1a4e 100644 > --- a/mm/sparse-vmemmap.c > +++ b/mm/sparse-vmemmap.c > @@ -378,16 +378,45 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigned long end, > } > } > > -/* > - * Populate vmemmap pages HVO-style. The first page contains the head > - * page and needed tail pages, the other ones are mirrors of the first > - * page. > - */ > +static __meminit unsigned long vmemmap_get_tail(unsigned int order, int node) > +{ > + unsigned long pfn; > + unsigned int idx; > + struct page *p; > + > + BUG_ON(order < VMEMMAP_TAIL_MIN_ORDER); > + BUG_ON(order > MAX_FOLIO_ORDER); > + > + idx = order - VMEMMAP_TAIL_MIN_ORDER; > + pfn = NODE_DATA(node)->vmemmap_tails[idx];              ^ Why you added a space here? > + if (pfn) > + return pfn; > + > + p = vmemmap_alloc_block_zero(PAGE_SIZE, node); > + if (!p) > + return 0; > + > + for (int i = 0; i < PAGE_SIZE / sizeof(struct page); i++) > + prep_compound_tail(p + i, NULL, order); > + > + pfn = PHYS_PFN(virt_to_phys(p)); > + NODE_DATA(node)->vmemmap_tails[idx] = pfn; > + > + return pfn; > +} > + > int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end, > int node, unsigned long headsize) > { > + unsigned long maddr, len, tail_pfn; > + unsigned int order; > pte_t *pte; > - unsigned long maddr; > + > + len = end - addr; > + order = ilog2(len * sizeof(struct page) / PAGE_SIZE); > + tail_pfn = vmemmap_get_tail(order, node); > + if (!tail_pfn) > + return -ENOMEM; > > for (maddr = addr; maddr < addr + headsize; maddr += PAGE_SIZE) { > pte = vmemmap_populate_address(maddr, node, NULL, -1, 0); > @@ -398,8 +427,7 @@ int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end, > /* > * Reuse the last page struct page mapped above for the rest. > */ > - return vmemmap_populate_range(maddr, end, node, NULL, > - pte_pfn(ptep_get(pte)), 0); > + return vmemmap_populate_range(maddr, end, node, NULL, tail_pfn, 0); > } > > void __weak __meminit vmemmap_set_pmd(pmd_t *pmd, void *p, int node, From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C601D358CF for ; Thu, 29 Jan 2026 06:55:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nuU36ifMzR/NmgkuJVETCYxLQBBDXB0yl/pggGMgN80=; b=TA6IhSCEKJezFU B2E1Zhfn4YnLH7y7ojOs609/urU58337LTZWutDlKqQI51rEjOFnh4VWSkTaeCPBBshZeqhXijaF8 4V5J6dWB426zoGLU1C3UDriXtxadOZH625MQL8NtHeSawRAF26JhpOrAWpB8jFt58pH4RKFSBahKx uIU9k/qIbnz+KtnXAKBbFhMBDMAslz84rELXcK8ZCArMTQYYWSCdbbvtcBtt5VRgR1bk30WhfSzk3 jn1ubbnj5uZY2g8M2MKMRVIAWPCogAVZMrjIGZ8EDLMXE3Eee1GTe+vl0BQ70dOL3d9d/QWkeIqFo TRygQf+pF+AEab1D0Sfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1vlLvM-0000000HLPA-3MqP; Thu, 29 Jan 2026 06:54:40 +0000 Received: from out-177.mta1.migadu.com ([2001:41d0:203:375::b1]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1vlLvI-0000000HLOp-1xDW for linux-riscv@lists.infradead.org; Thu, 29 Jan 2026 06:54:39 +0000 Message-ID: <0db8c993-a525-438f-9d7b-94f63bfa0aa4@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1769669662; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i4JRgDeBr4jab+v0AQMoJMPJjVR8MkEaj+rH1zUeTz4=; b=n5geKtvskBt5UeHtUN3tjKwhwYQQWLClgpDulimgfAcl+2JC++CzR0HifKg5bhU1Jcjsp5 j+4xd+xIwQHvnZDELrYAHClQ62HxwFa6zJbY2EgtbaVxX5Rlrx3nJYrdjqq/V5wJ1C7J8N aKBzpwi/wUAI7cpkGYqzoCq6dJaN4Ys= Date: Thu, 29 Jan 2026 14:54:07 +0800 MIME-Version: 1.0 Subject: Re: [PATCHv5 11/17] mm/hugetlb: Remove fake head pages To: Kiryl Shutsemau Cc: Oscar Salvador , Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Zi Yan , Baoquan He , Michal Hocko , Johannes Weiner , Jonathan Corbet , Huacai Chen , WANG Xuerui , Palmer Dabbelt , Paul Walmsley , Albert Ou , Alexandre Ghiti , kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, Andrew Morton , David Hildenbrand , Matthew Wilcox , Usama Arif , Frank van der Linden References: <20260128135500.22121-1-kas@kernel.org> <20260128135500.22121-12-kas@kernel.org> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Muchun Song In-Reply-To: <20260128135500.22121-12-kas@kernel.org> X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260128_225437_506112_D51051C6 X-CRM114-Status: GOOD ( 32.80 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org CgpPbiAyMDI2LzEvMjggMjE6NTQsIEtpcnlsIFNodXRzZW1hdSB3cm90ZToKPiBIdWdlVExCIFZt ZW1tYXAgT3B0aW1pemF0aW9uIChIVk8pIHJlZHVjZXMgbWVtb3J5IHVzYWdlIGJ5IGZyZWVpbmcg bW9zdAo+IHZtZW1tYXAgcGFnZXMgZm9yIGh1Z2UgcGFnZXMgYW5kIHJlbWFwcGluZyB0aGUgZnJl ZWQgcmFuZ2UgdG8gYSBzaW5nbGUKPiBwYWdlIGNvbnRhaW5pbmcgdGhlIHN0cnVjdCBwYWdlIG1l dGFkYXRhLgo+Cj4gV2l0aCB0aGUgbmV3IG1hc2stYmFzZWQgY29tcG91bmRfaW5mbyBlbmNvZGlu ZyAoZm9yIHBvd2VyLW9mLTIgc3RydWN0Cj4gcGFnZSBzaXplcyksIGFsbCB0YWlsIHBhZ2VzIG9m IHRoZSBzYW1lIG9yZGVyIGFyZSBub3cgaWRlbnRpY2FsCj4gcmVnYXJkbGVzcyBvZiB3aGljaCBj b21wb3VuZCBwYWdlIHRoZXkgYmVsb25nIHRvLiBUaGlzIG1lYW5zIHRoZSB0YWlsCj4gcGFnZXMg Y2FuIGJlIHRydWx5IHNoYXJlZCB3aXRob3V0IGZha2UgaGVhZHMuCj4KPiBBbGxvY2F0ZSBhIHNp bmdsZSBwYWdlIG9mIGluaXRpYWxpemVkIHRhaWwgc3RydWN0IHBhZ2VzIHBlciBOVU1BIG5vZGUK PiBwZXIgb3JkZXIgaW4gdGhlIHZtZW1tYXBfdGFpbHNbXSBhcnJheSBpbiBwZ2xpc3RfZGF0YS4g QWxsIGh1Z2UgcGFnZXMgb2YKPiB0aGF0IG9yZGVyIG9uIHRoZSBub2RlIHNoYXJlIHRoaXMgdGFp bCBwYWdlLCBtYXBwZWQgcmVhZC1vbmx5IGludG8gdGhlaXIKPiB2bWVtbWFwLiBUaGUgaGVhZCBw YWdlIHJlbWFpbnMgdW5pcXVlIHBlciBodWdlIHBhZ2UuCj4KPiBSZWRlZmluZSBNQVhfRk9MSU9f T1JERVIgdXNpbmcgaWxvZzIoKS4gVGhlIGRlZmluZSBoYXMgdG8gcHJvZHVjZSBhCj4gY29tcGls ZS1jb25zdGFudCBhcyBpdCBpcyB1c2VkIHRvIHNwZWNpZnkgdm1lbW1hcF90YWlsIGFycmF5IHNp emUuCj4gRm9yIHNvbWUgcmVhc29uLCBjb21waWxlciBpcyBub3QgYWJsZSB0byBzb2x2ZSBnZXRf b3JkZXIoKSBhdAo+IGNvbXBpbGUtdGltZSwgYnV0IGlsb2cyKCkgd29ya3MuCj4KPiBBdm9pZCBQ VURfT1JERVIgdG8gZGVmaW5lIE1BWF9GT0xJT19PUkRFUiBhcyBpdCBhZGRzIGRlcGVuZGVuY3kg dG8KPiA8bGludXgvcGd0YWJsZS5oPiB3aGljaCBnZW5lcmF0ZXMgaGFyZC10by1icmVhayBpbmNs dWRlIGxvb3AuCj4KPiBUaGlzIGVsaW1pbmF0ZXMgZmFrZSBoZWFkcyB3aGlsZSBtYWludGFpbmlu ZyB0aGUgc2FtZSBtZW1vcnkgc2F2aW5ncywKPiBhbmQgc2ltcGxpZmllcyBjb21wb3VuZF9oZWFk KCkgYnkgcmVtb3ZpbmcgZmFrZSBoZWFkIGRldGVjdGlvbi4KPgo+IFNpZ25lZC1vZmYtYnk6IEtp cnlsIFNodXRzZW1hdSA8a2FzQGtlcm5lbC5vcmc+Cj4gLS0tCj4gICBpbmNsdWRlL2xpbnV4L21t em9uZS5oIHwgMTggKysrKysrKysrKysrKysrLS0KPiAgIG1tL2h1Z2V0bGJfdm1lbW1hcC5jICAg fCAzNiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tCj4gICBtbS9zcGFyc2Utdm1l bW1hcC5jICAgIHwgNDQgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0t Cj4gICAzIGZpbGVzIGNoYW5nZWQsIDg2IGluc2VydGlvbnMoKyksIDEyIGRlbGV0aW9ucygtKQo+ Cj4gZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbW16b25lLmggYi9pbmNsdWRlL2xpbnV4L21t em9uZS5oCj4gaW5kZXggMTkyMTQzYjVjZGMwLi42OTgwOTFjNzRkYmIgMTAwNjQ0Cj4gLS0tIGEv aW5jbHVkZS9saW51eC9tbXpvbmUuaAo+ICsrKyBiL2luY2x1ZGUvbGludXgvbW16b25lLmgKPiBA QCAtODEsMTMgKzgxLDE3IEBACj4gICAgKiBjdXJyZW50bHkgZXhwZWN0IChzZWUgQ09ORklHX0hB VkVfR0lHQU5USUNfRk9MSU9TKTogd2l0aCBodWdldGxiLCB3ZSBleHBlY3QKPiAgICAqIG5vIGZv bGlvcyBsYXJnZXIgdGhhbiAxNiBHaUIgb24gNjRiaXQgYW5kIDEgR2lCIG9uIDMyYml0Lgo+ICAg ICovCj4gLSNkZWZpbmUgTUFYX0ZPTElPX09SREVSCQlnZXRfb3JkZXIoSVNfRU5BQkxFRChDT05G SUdfNjRCSVQpID8gU1pfMTZHIDogU1pfMUcpCj4gKyNpZmRlZiBDT05GSUdfNjRCSVQKPiArI2Rl ZmluZSBNQVhfRk9MSU9fT1JERVIJCShpbG9nMihTWl8xNkcpIC0gUEFHRV9TSElGVCkKPiArI2Vs c2UKPiArI2RlZmluZSBNQVhfRk9MSU9fT1JERVIJCShpbG9nMihTWl8xRykgLSBQQUdFX1NISUZU KQo+ICsjZW5kaWYKPiAgICNlbHNlCj4gICAvKgo+ICAgICogV2l0aG91dCBodWdldGxiLCBnaWdh bnRpYyBmb2xpb3MgdGhhdCBhcmUgYmlnZ2VyIHRoYW4gYSBzaW5nbGUgUFVEIGFyZQo+ICAgICog Y3VycmVudGx5IGltcG9zc2libGUuCj4gICAgKi8KPiAtI2RlZmluZSBNQVhfRk9MSU9fT1JERVIJ CVBVRF9PUkRFUgo+ICsjZGVmaW5lIE1BWF9GT0xJT19PUkRFUgkJKFBVRF9TSElGVCAtIFBBR0Vf U0hJRlQpCj4gICAjZW5kaWYKPiAgIAo+ICAgI2RlZmluZSBNQVhfRk9MSU9fTlJfUEFHRVMJKDFV TCA8PCBNQVhfRk9MSU9fT1JERVIpCj4gQEAgLTE0MDIsNiArMTQwNiwxMyBAQCBzdHJ1Y3QgbWVt b3J5X2ZhaWx1cmVfc3RhdHMgewo+ICAgfTsKPiAgICNlbmRpZgo+ICAgCj4gKy8qCj4gKyAqIHZt ZW1tYXAgb3B0aW1pemF0aW9uIChsaWtlIEhWTykgaXMgb25seSBwb3NzaWJsZSBmb3IgcGFnZSBv cmRlcnMgdGhhdCBmaWxsCj4gKyAqIHR3byBvciBtb3JlIHBhZ2VzIHdpdGggc3RydWN0IHBhZ2Vz Lgo+ICsgKi8KPiArI2RlZmluZSBWTUVNTUFQX1RBSUxfTUlOX09SREVSIChpbG9nMigyICogUEFH RV9TSVpFIC8gc2l6ZW9mKHN0cnVjdCBwYWdlKSkpCj4gKyNkZWZpbmUgTlJfVk1FTU1BUF9UQUlM UyAoTUFYX0ZPTElPX09SREVSIC0gVk1FTU1BUF9UQUlMX01JTl9PUkRFUiArIDEpCj4gKwo+ICAg LyoKPiAgICAqIE9uIE5VTUEgbWFjaGluZXMsIGVhY2ggTlVNQSBub2RlIHdvdWxkIGhhdmUgYSBw Z19kYXRhX3QgdG8gZGVzY3JpYmUKPiAgICAqIGl0J3MgbWVtb3J5IGxheW91dC4gT24gVU1BIG1h Y2hpbmVzIHRoZXJlIGlzIGEgc2luZ2xlIHBnbGlzdF9kYXRhIHdoaWNoCj4gQEAgLTE1NTAsNiAr MTU2MSw5IEBAIHR5cGVkZWYgc3RydWN0IHBnbGlzdF9kYXRhIHsKPiAgICNpZmRlZiBDT05GSUdf TUVNT1JZX0ZBSUxVUkUKPiAgIAlzdHJ1Y3QgbWVtb3J5X2ZhaWx1cmVfc3RhdHMgbWZfc3RhdHM7 Cj4gICAjZW5kaWYKPiArI2lmZGVmIENPTkZJR19TUEFSU0VNRU1fVk1FTU1BUAo+ICsJdW5zaWdu ZWQgbG9uZyB2bWVtbWFwX3RhaWxzW05SX1ZNRU1NQVBfVEFJTFNdOwoKV2Ugc2hvdWxkIHJlY29y ZCAic3RydWN0IHBhZ2UiIGluc3RlYWQgb2YgcGZuLCBJJ2xsIGV4cGxhaW4gYmVsb3cuCgo+ICsj ZW5kaWYKPiAgIH0gcGdfZGF0YV90Owo+ICAgCj4gICAjZGVmaW5lIG5vZGVfcHJlc2VudF9wYWdl cyhuaWQpCShOT0RFX0RBVEEobmlkKS0+bm9kZV9wcmVzZW50X3BhZ2VzKQo+IGRpZmYgLS1naXQg YS9tbS9odWdldGxiX3ZtZW1tYXAuYyBiL21tL2h1Z2V0bGJfdm1lbW1hcC5jCj4gaW5kZXggYTM5 YTMwMWUwOGI5Li5mNWY0MmI5MmRkN2QgMTAwNjQ0Cj4gLS0tIGEvbW0vaHVnZXRsYl92bWVtbWFw LmMKPiArKysgYi9tbS9odWdldGxiX3ZtZW1tYXAuYwo+IEBAIC0xOSw2ICsxOSw3IEBACj4gICAK PiAgICNpbmNsdWRlIDxhc20vdGxiZmx1c2guaD4KPiAgICNpbmNsdWRlICJodWdldGxiX3ZtZW1t YXAuaCIKPiArI2luY2x1ZGUgImludGVybmFsLmgiCj4gICAKPiAgIC8qKgo+ICAgICogc3RydWN0 IHZtZW1tYXBfcmVtYXBfd2FsayAtIHdhbGsgdm1lbW1hcCBwYWdlIHRhYmxlCj4gQEAgLTUwNSw2 ICs1MDYsMzQgQEAgc3RhdGljIGJvb2wgdm1lbW1hcF9zaG91bGRfb3B0aW1pemVfZm9saW8oY29u c3Qgc3RydWN0IGhzdGF0ZSAqaCwgc3RydWN0IGZvbGlvICoKPiAgIAlyZXR1cm4gdHJ1ZTsKPiAg IH0KPiAgIAo+ICtzdGF0aWMgc3RydWN0IHBhZ2UgKnZtZW1tYXBfZ2V0X3RhaWwodW5zaWduZWQg aW50IG9yZGVyLCBpbnQgbm9kZSkKPiArewo+ICsJdW5zaWduZWQgbG9uZyBwZm47Cj4gKwl1bnNp Z25lZCBpbnQgaWR4Owo+ICsJc3RydWN0IHBhZ2UgKnRhaWwsICpwOwo+ICsKPiArCWlkeCA9IG9y ZGVyIC0gVk1FTU1BUF9UQUlMX01JTl9PUkRFUjsKPiArCXBmbiA9IFJFQURfT05DRShOT0RFX0RB VEEobm9kZSktPnZtZW1tYXBfdGFpbHNbaWR4XSk7Cj4gKwlpZiAocGZuKQoKWW914oCZdmUgYXNz dW1lZCB0aGF0IGEgdmFsaWQgUEZOIGNhbiBuZXZlciBiZSB6ZXJvLCBidXQgdGhhdAppc27igJl0 IGd1YXJhbnRlZWQuwqDCoElmIHdlIHN0b3JlIHRoZSBgc3RydWN0IHBhZ2VgIHBvaW50ZXIKaW5z dGVhZCwgdGhlIGlzc3VlIGRpc2FwcGVhcnM6IGl0cyB2aXJ0dWFsIGFkZHJlc3MgaXMgbmV2ZXIK TlVMTC4KCk1vcmVvdmVyLCB3ZSBvbmx5IGNvbnZlcnQgYmFjayBhbmQgZm9ydGggd2l0aCBwZm5f dG9fcGFnZSgpL3BhZ2VfdG9fcGZuKCk7CndlIG5ldmVyIGRlcmVmZXJlbmNlIGFueSBtZW1iZXIg b2YgdGhlIHN0cnVjdHVyZSwgc28gd2UgZG9u4oCZdApoYXZlIHRvIGNhcmUgd2hldGhlciBgc3Ry dWN0IHBhZ2VgIGhhcyBiZWVuIGluaXRpYWxpemVkIHlldApkdXJpbmcgZWFybHkgYm9vdCAoaXQg aXMgc2FmZSBmb3IgdXMgdG8gZ2V0IHBhZ2UgaW4gc3BhcnNlLXZtZW1tYXAuYykuCgo+ICsJCXJl dHVybiBwZm5fdG9fcGFnZShwZm4pOwo+ICsKPiArCXRhaWwgPSBhbGxvY19wYWdlc19ub2RlKG5v ZGUsIEdGUF9LRVJORUwgfCBfX0dGUF9aRVJPLCAwKTsKPiArCWlmICghdGFpbCkKPiArCQlyZXR1 cm4gTlVMTDsKPiArCj4gKwlwID0gcGFnZV90b192aXJ0KHRhaWwpOwo+ICsJZm9yIChpbnQgaSA9 IDA7IGkgPCBQQUdFX1NJWkUgLyBzaXplb2Yoc3RydWN0IHBhZ2UpOyBpKyspCj4gKwkJcHJlcF9j b21wb3VuZF90YWlsKHAgKyBpLCBOVUxMLCBvcmRlcik7Cj4gKwo+ICsJcGZuID0gUEhZU19QRk4o dmlydF90b19waHlzKHApKTsKPiArCWlmIChjbXB4Y2hnKCZOT0RFX0RBVEEobm9kZSktPnZtZW1t YXBfdGFpbHNbaWR4XSwgMCwgcGZuKSkgewo+ICsJCV9fZnJlZV9wYWdlKHRhaWwpOwo+ICsJCXBm biA9IFJFQURfT05DRShOT0RFX0RBVEEobm9kZSktPnZtZW1tYXBfdGFpbHNbaWR4XSk7Cj4gKwl9 Cj4gKwo+ICsJcmV0dXJuIHBmbl90b19wYWdlKHBmbik7Cj4gK30KPiArCj4gICBzdGF0aWMgaW50 IF9faHVnZXRsYl92bWVtbWFwX29wdGltaXplX2ZvbGlvKGNvbnN0IHN0cnVjdCBoc3RhdGUgKmgs Cj4gICAJCQkJCSAgICBzdHJ1Y3QgZm9saW8gKmZvbGlvLAo+ICAgCQkJCQkgICAgc3RydWN0IGxp c3RfaGVhZCAqdm1lbW1hcF9wYWdlcywKPiBAQCAtNTIwLDYgKzU0OSwxMSBAQCBzdGF0aWMgaW50 IF9faHVnZXRsYl92bWVtbWFwX29wdGltaXplX2ZvbGlvKGNvbnN0IHN0cnVjdCBoc3RhdGUgKmgs Cj4gICAJaWYgKCF2bWVtbWFwX3Nob3VsZF9vcHRpbWl6ZV9mb2xpbyhoLCBmb2xpbykpCj4gICAJ CXJldHVybiByZXQ7Cj4gICAKPiArCW5pZCA9IGZvbGlvX25pZChmb2xpbyk7Cj4gKwl2bWVtbWFw X3RhaWwgPSB2bWVtbWFwX2dldF90YWlsKGgtPm9yZGVyLCBuaWQpOwo+ICsJaWYgKCF2bWVtbWFw X3RhaWwpCj4gKwkJcmV0dXJuIC1FTk9NRU07Cj4gKwo+ICAgCXN0YXRpY19icmFuY2hfaW5jKCZo dWdldGxiX29wdGltaXplX3ZtZW1tYXBfa2V5KTsKPiAgIAo+ICAgCWlmIChmbGFncyAmIFZNRU1N QVBfU1lOQ0hST05JWkVfUkNVKQo+IEBAIC01MzcsNyArNTcxLDYgQEAgc3RhdGljIGludCBfX2h1 Z2V0bGJfdm1lbW1hcF9vcHRpbWl6ZV9mb2xpbyhjb25zdCBzdHJ1Y3QgaHN0YXRlICpoLAo+ICAg CSAqLwo+ICAgCWZvbGlvX3NldF9odWdldGxiX3ZtZW1tYXBfb3B0aW1pemVkKGZvbGlvKTsKPiAg IAo+IC0JbmlkID0gZm9saW9fbmlkKGZvbGlvKTsKPiAgIAl2bWVtbWFwX2hlYWQgPSBhbGxvY19w YWdlc19ub2RlKG5pZCwgR0ZQX0tFUk5FTCwgMCk7Cj4gICAJaWYgKCF2bWVtbWFwX2hlYWQpIHsK PiAgIAkJcmV0ID0gLUVOT01FTTsKPiBAQCAtNTQ4LDcgKzU4MSw2IEBAIHN0YXRpYyBpbnQgX19o dWdldGxiX3ZtZW1tYXBfb3B0aW1pemVfZm9saW8oY29uc3Qgc3RydWN0IGhzdGF0ZSAqaCwKPiAg IAlsaXN0X2FkZCgmdm1lbW1hcF9oZWFkLT5scnUsIHZtZW1tYXBfcGFnZXMpOwo+ICAgCW1lbW1h cF9wYWdlc19hZGQoMSk7Cj4gICAKPiAtCXZtZW1tYXBfdGFpbAk9IHZtZW1tYXBfaGVhZDsKPiAg IAl2bWVtbWFwX3N0YXJ0CT0gKHVuc2lnbmVkIGxvbmcpJmZvbGlvLT5wYWdlOwo+ICAgCXZtZW1t YXBfZW5kCT0gdm1lbW1hcF9zdGFydCArIGh1Z2V0bGJfdm1lbW1hcF9zaXplKGgpOwo+ICAgCj4g ZGlmZiAtLWdpdCBhL21tL3NwYXJzZS12bWVtbWFwLmMgYi9tbS9zcGFyc2Utdm1lbW1hcC5jCj4g aW5kZXggMzc1MjJkNmNiMzk4Li4yM2FiZDA2ZjFhNGUgMTAwNjQ0Cj4gLS0tIGEvbW0vc3BhcnNl LXZtZW1tYXAuYwo+ICsrKyBiL21tL3NwYXJzZS12bWVtbWFwLmMKPiBAQCAtMzc4LDE2ICszNzgs NDUgQEAgdm9pZCB2bWVtbWFwX3dycHJvdGVjdF9odm8odW5zaWduZWQgbG9uZyBhZGRyLCB1bnNp Z25lZCBsb25nIGVuZCwKPiAgIAl9Cj4gICB9Cj4gICAKPiAtLyoKPiAtICogUG9wdWxhdGUgdm1l bW1hcCBwYWdlcyBIVk8tc3R5bGUuIFRoZSBmaXJzdCBwYWdlIGNvbnRhaW5zIHRoZSBoZWFkCj4g LSAqIHBhZ2UgYW5kIG5lZWRlZCB0YWlsIHBhZ2VzLCB0aGUgb3RoZXIgb25lcyBhcmUgbWlycm9y cyBvZiB0aGUgZmlyc3QKPiAtICogcGFnZS4KPiAtICovCj4gK3N0YXRpYyBfX21lbWluaXQgdW5z aWduZWQgbG9uZyB2bWVtbWFwX2dldF90YWlsKHVuc2lnbmVkIGludCBvcmRlciwgaW50IG5vZGUp Cj4gK3sKPiArCXVuc2lnbmVkIGxvbmcgcGZuOwo+ICsJdW5zaWduZWQgaW50IGlkeDsKPiArCXN0 cnVjdCBwYWdlICpwOwo+ICsKPiArCUJVR19PTihvcmRlciA8IFZNRU1NQVBfVEFJTF9NSU5fT1JE RVIpOwo+ICsJQlVHX09OKG9yZGVyID4gTUFYX0ZPTElPX09SREVSKTsKPiArCj4gKwlpZHggPSBv cmRlciAtIFZNRU1NQVBfVEFJTF9NSU5fT1JERVI7Cj4gKwlwZm4gPSAgTk9ERV9EQVRBKG5vZGUp LT52bWVtbWFwX3RhaWxzW2lkeF07CiDCoCDCoCDCoCDCoCDCoCDCoCDCoF4KV2h5IHlvdSBhZGRl ZCBhIHNwYWNlIGhlcmU/Cgo+ICsJaWYgKHBmbikKPiArCQlyZXR1cm4gcGZuOwo+ICsKPiArCXAg PSB2bWVtbWFwX2FsbG9jX2Jsb2NrX3plcm8oUEFHRV9TSVpFLCBub2RlKTsKPiArCWlmICghcCkK PiArCQlyZXR1cm4gMDsKPiArCj4gKwlmb3IgKGludCBpID0gMDsgaSA8IFBBR0VfU0laRSAvIHNp emVvZihzdHJ1Y3QgcGFnZSk7IGkrKykKPiArCQlwcmVwX2NvbXBvdW5kX3RhaWwocCArIGksIE5V TEwsIG9yZGVyKTsKPiArCj4gKwlwZm4gPSBQSFlTX1BGTih2aXJ0X3RvX3BoeXMocCkpOwo+ICsJ Tk9ERV9EQVRBKG5vZGUpLT52bWVtbWFwX3RhaWxzW2lkeF0gPSBwZm47Cj4gKwo+ICsJcmV0dXJu IHBmbjsKPiArfQo+ICsKPiAgIGludCBfX21lbWluaXQgdm1lbW1hcF9wb3B1bGF0ZV9odm8odW5z aWduZWQgbG9uZyBhZGRyLCB1bnNpZ25lZCBsb25nIGVuZCwKPiAgIAkJCQkgICAgICAgaW50IG5v ZGUsIHVuc2lnbmVkIGxvbmcgaGVhZHNpemUpCj4gICB7Cj4gKwl1bnNpZ25lZCBsb25nIG1hZGRy LCBsZW4sIHRhaWxfcGZuOwo+ICsJdW5zaWduZWQgaW50IG9yZGVyOwo+ICAgCXB0ZV90ICpwdGU7 Cj4gLQl1bnNpZ25lZCBsb25nIG1hZGRyOwo+ICsKPiArCWxlbiA9IGVuZCAtIGFkZHI7Cj4gKwlv cmRlciA9IGlsb2cyKGxlbiAqIHNpemVvZihzdHJ1Y3QgcGFnZSkgLyBQQUdFX1NJWkUpOwo+ICsJ dGFpbF9wZm4gPSB2bWVtbWFwX2dldF90YWlsKG9yZGVyLCBub2RlKTsKPiArCWlmICghdGFpbF9w Zm4pCj4gKwkJcmV0dXJuIC1FTk9NRU07Cj4gICAKPiAgIAlmb3IgKG1hZGRyID0gYWRkcjsgbWFk ZHIgPCBhZGRyICsgaGVhZHNpemU7IG1hZGRyICs9IFBBR0VfU0laRSkgewo+ICAgCQlwdGUgPSB2 bWVtbWFwX3BvcHVsYXRlX2FkZHJlc3MobWFkZHIsIG5vZGUsIE5VTEwsIC0xLCAwKTsKPiBAQCAt Mzk4LDggKzQyNyw3IEBAIGludCBfX21lbWluaXQgdm1lbW1hcF9wb3B1bGF0ZV9odm8odW5zaWdu ZWQgbG9uZyBhZGRyLCB1bnNpZ25lZCBsb25nIGVuZCwKPiAgIAkvKgo+ICAgCSAqIFJldXNlIHRo ZSBsYXN0IHBhZ2Ugc3RydWN0IHBhZ2UgbWFwcGVkIGFib3ZlIGZvciB0aGUgcmVzdC4KPiAgIAkg Ki8KPiAtCXJldHVybiB2bWVtbWFwX3BvcHVsYXRlX3JhbmdlKG1hZGRyLCBlbmQsIG5vZGUsIE5V TEwsCj4gLQkJCQkJcHRlX3BmbihwdGVwX2dldChwdGUpKSwgMCk7Cj4gKwlyZXR1cm4gdm1lbW1h cF9wb3B1bGF0ZV9yYW5nZShtYWRkciwgZW5kLCBub2RlLCBOVUxMLCB0YWlsX3BmbiwgMCk7Cj4g ICB9Cj4gICAKPiAgIHZvaWQgX193ZWFrIF9fbWVtaW5pdCB2bWVtbWFwX3NldF9wbWQocG1kX3Qg KnBtZCwgdm9pZCAqcCwgaW50IG5vZGUsCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxp c3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0 aW5mby9saW51eC1yaXNjdgo=