From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03844C7115C for ; Wed, 25 Jun 2025 12:19:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=csN3NGyJz0Plq+DSLlEX2Pakhu0xabELB8Wjy8HXAeM=; b=UvtYpffjCnMkEQhlhK6w0sWlJm THPR2qy/3T31Efs7QI+QrJU80n+QMhCVLZXVS7DmXD1AX4jUnqEgehOpZ/gQJBfUHv42y1+5XTQ77 1TQctankeDiUmEG0+JSpmw4uGDtbTbb35EXmhJqGVA9edJO/eYLV24cd9AiQO8wghiCiCPme3rf7i TVlGrQGN8lUxghpXmXqshQv6xlKshaKw9sw22ALGrg+Vj3ZyI/WFohD1BcRYKDKx6jFkUgQ8ka3eU W6iWhG2j2B60zlVLwuKGla7K0bmuLFenTcOvmK+stK1RQ8fiveWvx099SOsg2lLu5tSi+OUZ8UY2K wXQgIRLA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1uUP6Q-00000008bJd-3MDk; Wed, 25 Jun 2025 12:19:47 +0000 Received: from out-182.mta1.migadu.com ([95.215.58.182]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1uUNfx-00000008MYw-1lv5 for linux-arm-kernel@lists.infradead.org; Wed, 25 Jun 2025 10:48:22 +0000 Message-ID: <938c4726-b93e-46df-bceb-65c7574714a6@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1750848488; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=csN3NGyJz0Plq+DSLlEX2Pakhu0xabELB8Wjy8HXAeM=; b=BWB+hSf9s40JKUag99JUhdHNhY+hi/eWwJYO4YD9eHSuRniIiKbey15lYX6sZe0B1ZFnok sZPk4HhuguE7ZuMeAVTkLE7Zes+IVipMQAhigbH1G++5xpJpTMC8j8KZxnXpNdNO2AFCWs FaaRff8WSDn24xARtP5xY/nLqjGRjxI= Date: Wed, 25 Jun 2025 18:47:55 +0800 MIME-Version: 1.0 Subject: Re: [PATCH v4 3/4] mm: Support batched unmap for lazyfree large folios during reclamation Content-Language: en-US To: David Hildenbrand Cc: 21cnbao@gmail.com, akpm@linux-foundation.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, kasong@tencent.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, lorenzo.stoakes@oracle.com, ryan.roberts@arm.com, v-songbaohua@oppo.com, x86@kernel.org, ying.huang@intel.com, zhengtangquan@oppo.com, Lance Yang References: <2c19a6cf-0b42-477b-a672-ed8c1edd4267@redhat.com> <20250624162503.78957-1-ioworker0@gmail.com> <27d174e0-c209-4851-825a-0baeb56df86f@redhat.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Lance Yang In-Reply-To: <27d174e0-c209-4851-825a-0baeb56df86f@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250625_034821_607682_D9479786 X-CRM114-Status: GOOD ( 34.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 2025/6/25 18:00, David Hildenbrand wrote: > On 24.06.25 18:25, Lance Yang wrote: >> On 2025/6/24 23:34, David Hildenbrand wrote: >>> On 24.06.25 17:26, Lance Yang wrote: >>>> On 2025/6/24 20:55, David Hildenbrand wrote: >>>>> On 14.02.25 10:30, Barry Song wrote: >>>>>> From: Barry Song >>>> [...] >>>>>> diff --git a/mm/rmap.c b/mm/rmap.c >>>>>> index 89e51a7a9509..8786704bd466 100644 >>>>>> --- a/mm/rmap.c >>>>>> +++ b/mm/rmap.c >>>>>> @@ -1781,6 +1781,25 @@ void folio_remove_rmap_pud(struct folio >>>>>> *folio, >>>>>> struct page *page, >>>>>>     #endif >>>>>>     } >>>>>> +/* We support batch unmapping of PTEs for lazyfree large folios */ >>>>>> +static inline bool can_batch_unmap_folio_ptes(unsigned long addr, >>>>>> +            struct folio *folio, pte_t *ptep) >>>>>> +{ >>>>>> +    const fpb_t fpb_flags = FPB_IGNORE_DIRTY | >>>>>> FPB_IGNORE_SOFT_DIRTY; >>>>>> +    int max_nr = folio_nr_pages(folio); >>>>> >>>>> Let's assume we have the first page of a folio mapped at the last page >>>>> table entry in our page table. >>>> >>>> Good point. I'm curious if it is something we've seen in practice ;) >>> >>> I challenge you to write a reproducer :P I assume it might be doable >>> through simple mremap(). >>> >>>> >>>>> >>>>> What prevents folio_pte_batch() from reading outside the page table? >>>> >>>> Assuming such a scenario is possible, to prevent any chance of an >>>> out-of-bounds read, how about this change: >>>> >>>> diff --git a/mm/rmap.c b/mm/rmap.c >>>> index fb63d9256f09..9aeae811a38b 100644 >>>> --- a/mm/rmap.c >>>> +++ b/mm/rmap.c >>>> @@ -1852,6 +1852,25 @@ static inline bool >>>> can_batch_unmap_folio_ptes(unsigned long addr, >>>>        const fpb_t fpb_flags = FPB_IGNORE_DIRTY | >>>> FPB_IGNORE_SOFT_DIRTY; >>>>        int max_nr = folio_nr_pages(folio); >>>>        pte_t pte = ptep_get(ptep); >>>> +    unsigned long end_addr; >>>> + >>>> +    /* >>>> +     * To batch unmap, the entire folio's PTEs must be contiguous >>>> +     * and mapped within the same PTE page table, which corresponds to >>>> +     * a single PMD entry. Before calling folio_pte_batch(), which >>>> does >>>> +     * not perform boundary checks itself, we must verify that the >>>> +     * address range covered by the folio does not cross a PMD >>>> boundary. >>>> +     */ >>>> +    end_addr = addr + (max_nr * PAGE_SIZE) - 1; >>>> + >>>> +    /* >>>> +     * A fast way to check for a PMD boundary cross is to align both >>>> +     * the start and end addresses to the PMD boundary and see if they >>>> +     * are different. If they are, the range spans across at least two >>>> +     * different PMD-managed regions. >>>> +     */ >>>> +    if ((addr & PMD_MASK) != (end_addr & PMD_MASK)) >>>> +        return false; >>> >>> You should not be messing with max_nr = folio_nr_pages(folio) here at >>> all. folio_pte_batch() takes care of that. >>> >>> Also, way too many comments ;) >>> >>> You may only batch within a single VMA and within a single page table. >>> >>> So simply align the addr up to the next PMD, and make sure it does not >>> exceed the vma end. >>> >>> ALIGN and friends can help avoiding excessive comments. >> >> Thanks! How about this updated version based on your suggestion: >> >> diff --git a/mm/rmap.c b/mm/rmap.c >> index fb63d9256f09..241d55a92a47 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -1847,12 +1847,25 @@ void folio_remove_rmap_pud(struct folio >> *folio, struct page *page, >>   /* We support batch unmapping of PTEs for lazyfree large folios */ >>   static inline bool can_batch_unmap_folio_ptes(unsigned long addr, >> -            struct folio *folio, pte_t *ptep) >> +                          struct folio *folio, pte_t *ptep, >> +                          struct vm_area_struct *vma) >>   { >>       const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; >> +    unsigned long next_pmd, vma_end, end_addr; >>       int max_nr = folio_nr_pages(folio); >>       pte_t pte = ptep_get(ptep); >> +    /* >> +     * Limit the batch scan within a single VMA and within a single >> +     * page table. >> +     */ >> +    vma_end = vma->vm_end; >> +    next_pmd = ALIGN(addr + 1, PMD_SIZE); >> +    end_addr = addr + (unsigned long)max_nr * PAGE_SIZE; >> + >> +    if (end_addr > min(next_pmd, vma_end)) >> +        return false; > > May I suggest that we clean all that up as we fix it? Yeah, that looks much better. Thanks for the suggestion! > > Maybe something like this: > > diff --git a/mm/rmap.c b/mm/rmap.c > index 3b74bb19c11dd..11fbddc6ad8d6 100644 > --- a/mm/rmap.c > +++ b/mm/rmap.c > @@ -1845,23 +1845,38 @@ void folio_remove_rmap_pud(struct folio *folio, > struct page *page, >  #endif >  } > > -/* We support batch unmapping of PTEs for lazyfree large folios */ > -static inline bool can_batch_unmap_folio_ptes(unsigned long addr, > -                       struct folio *folio, pte_t *ptep) > +static inline unsigned int folio_unmap_pte_batch(struct folio *folio, > +               struct page_vma_mapped_walk *pvmw, enum ttu_flags flags, > +               pte_t pte) >  { >         const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; > -       int max_nr = folio_nr_pages(folio); > -       pte_t pte = ptep_get(ptep); > +       struct vm_area_struct *vma = pvmw->vma; > +       unsigned long end_addr, addr = pvmw->address; > +       unsigned int max_nr; > + > +       if (flags & TTU_HWPOISON) > +               return 1; > +       if (!folio_test_large(folio)) > +               return 1; > + > +       /* We may only batch within a single VMA and a single page > table. */ > +       end_addr = min_t(unsigned long, ALIGN(addr + 1, PMD_SIZE), vma- > >vm_end); > +       max_nr = (end_addr - addr) >> PAGE_SHIFT; > > +       /* We only support lazyfree batching for now ... */ >         if (!folio_test_anon(folio) || folio_test_swapbacked(folio)) > -               return false; > +               return 1; >         if (pte_unused(pte)) > -               return false; > -       if (pte_pfn(pte) != folio_pfn(folio)) > -               return false; > +               return 1; > +       /* ... where we must be able to batch the whole folio. */ > +       if (pte_pfn(pte) != folio_pfn(folio) || max_nr != > folio_nr_pages(folio)) > +               return 1; > +       max_nr = folio_pte_batch(folio, addr, pvmw->pte, pte, max_nr, > fpb_flags, > +                                NULL, NULL, NULL); > > -       return folio_pte_batch(folio, addr, ptep, pte, max_nr, > fpb_flags, NULL, > -                              NULL, NULL) == max_nr; > +       if (max_nr != folio_nr_pages(folio)) > +               return 1; > +       return max_nr; >  } > >  /* > @@ -2024,9 +2039,7 @@ static bool try_to_unmap_one(struct folio *folio, > struct vm_area_struct *vma, >                         if (pte_dirty(pteval)) >                                 folio_mark_dirty(folio); >                 } else if (likely(pte_present(pteval))) { > -                       if (folio_test_large(folio) && !(flags & > TTU_HWPOISON) && > -                           can_batch_unmap_folio_ptes(address, folio, > pvmw.pte)) > -                               nr_pages = folio_nr_pages(folio); > +                       nr_pages = folio_unmap_pte_batch(folio, &pvmw, > flags, pteval); >                         end_addr = address + nr_pages * PAGE_SIZE; >                         flush_cache_range(vma, address, end_addr); > > > Note that I don't quite understand why we have to batch the whole thing > or fallback to > individual pages. Why can't we perform other batches that span only some > PTEs? What's special > about 1 PTE vs. 2 PTEs vs. all PTEs? That's a good point about the "all-or-nothing" batching logic ;) It seems the "all-or-nothing" approach is specific to the lazyfree use case, which needs to unmap the entire folio for reclamation. If that's not possible, it falls back to the single-page slow path. Also, supporting partial batches would be useful, but not common case I guess, so let's leave it as is ;p Thanks, Lance > > > Can someone enlighten me why that is required? > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF9FEC7EE30 for ; Wed, 25 Jun 2025 12:17:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=671UdArY+ch9PUxJTY4nvTtJaTaPkK4bAfvgbs3Bm3M=; b=mHVQpRkNY0F4tu N37JbxvV27rvrepDMmsleO3EquuM0JdKHTPrTfqc4yQB0OsUmGAaAGoHXWrLZALtXyoqqHv4EN2WU 6J8k89msC7pUtghant5SPjgc0LmPsoCS8Bq+clcjKDsgVfKuGJ0k+IJBMakbGruFCIOcqmSOtd/DT DO4wCMPgi1hDiLFIGNdEgahTvpyMGV/AVo0K6t48sS8WH6doVGVe5vcvZrEsDwRFisfojOCBjc45A RGAgTH6C/2705WDa3Z14T0GWpILY+2qz4BrvdQ3CjGDVI2b6lzmaq6hoAfFwz6zUbtNxOCpvVBqoc NzW9YA5QPAm76V7V1EbQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1uUP41-00000008azt-46Kr; Wed, 25 Jun 2025 12:17:17 +0000 Received: from out-172.mta1.migadu.com ([2001:41d0:203:375::ac]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1uUNfm-00000008MXY-4AtK for linux-riscv@lists.infradead.org; Wed, 25 Jun 2025 10:48:12 +0000 Message-ID: <938c4726-b93e-46df-bceb-65c7574714a6@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1750848488; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=csN3NGyJz0Plq+DSLlEX2Pakhu0xabELB8Wjy8HXAeM=; b=BWB+hSf9s40JKUag99JUhdHNhY+hi/eWwJYO4YD9eHSuRniIiKbey15lYX6sZe0B1ZFnok sZPk4HhuguE7ZuMeAVTkLE7Zes+IVipMQAhigbH1G++5xpJpTMC8j8KZxnXpNdNO2AFCWs FaaRff8WSDn24xARtP5xY/nLqjGRjxI= Date: Wed, 25 Jun 2025 18:47:55 +0800 MIME-Version: 1.0 Subject: Re: [PATCH v4 3/4] mm: Support batched unmap for lazyfree large folios during reclamation Content-Language: en-US To: David Hildenbrand Cc: 21cnbao@gmail.com, akpm@linux-foundation.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, kasong@tencent.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, lorenzo.stoakes@oracle.com, ryan.roberts@arm.com, v-songbaohua@oppo.com, x86@kernel.org, ying.huang@intel.com, zhengtangquan@oppo.com, Lance Yang References: <2c19a6cf-0b42-477b-a672-ed8c1edd4267@redhat.com> <20250624162503.78957-1-ioworker0@gmail.com> <27d174e0-c209-4851-825a-0baeb56df86f@redhat.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Lance Yang In-Reply-To: <27d174e0-c209-4851-825a-0baeb56df86f@redhat.com> X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250625_034811_449321_8B575065 X-CRM114-Status: GOOD ( 33.55 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org CgpPbiAyMDI1LzYvMjUgMTg6MDAsIERhdmlkIEhpbGRlbmJyYW5kIHdyb3RlOgo+IE9uIDI0LjA2 LjI1IDE4OjI1LCBMYW5jZSBZYW5nIHdyb3RlOgo+PiBPbiAyMDI1LzYvMjQgMjM6MzQsIERhdmlk IEhpbGRlbmJyYW5kIHdyb3RlOgo+Pj4gT24gMjQuMDYuMjUgMTc6MjYsIExhbmNlIFlhbmcgd3Jv dGU6Cj4+Pj4gT24gMjAyNS82LzI0IDIwOjU1LCBEYXZpZCBIaWxkZW5icmFuZCB3cm90ZToKPj4+ Pj4gT24gMTQuMDIuMjUgMTA6MzAsIEJhcnJ5IFNvbmcgd3JvdGU6Cj4+Pj4+PiBGcm9tOiBCYXJy eSBTb25nIDx2LXNvbmdiYW9odWFAb3Bwby5jb20+Cj4+Pj4gWy4uLl0KPj4+Pj4+IGRpZmYgLS1n aXQgYS9tbS9ybWFwLmMgYi9tbS9ybWFwLmMKPj4+Pj4+IGluZGV4IDg5ZTUxYTdhOTUwOS4uODc4 NjcwNGJkNDY2IDEwMDY0NAo+Pj4+Pj4gLS0tIGEvbW0vcm1hcC5jCj4+Pj4+PiArKysgYi9tbS9y bWFwLmMKPj4+Pj4+IEBAIC0xNzgxLDYgKzE3ODEsMjUgQEAgdm9pZCBmb2xpb19yZW1vdmVfcm1h cF9wdWQoc3RydWN0IGZvbGlvIAo+Pj4+Pj4gKmZvbGlvLAo+Pj4+Pj4gc3RydWN0IHBhZ2UgKnBh Z2UsCj4+Pj4+PiDCoMKgwqAgI2VuZGlmCj4+Pj4+PiDCoMKgwqAgfQo+Pj4+Pj4gKy8qIFdlIHN1 cHBvcnQgYmF0Y2ggdW5tYXBwaW5nIG9mIFBURXMgZm9yIGxhenlmcmVlIGxhcmdlIGZvbGlvcyAq Lwo+Pj4+Pj4gK3N0YXRpYyBpbmxpbmUgYm9vbCBjYW5fYmF0Y2hfdW5tYXBfZm9saW9fcHRlcyh1 bnNpZ25lZCBsb25nIGFkZHIsCj4+Pj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3Qg Zm9saW8gKmZvbGlvLCBwdGVfdCAqcHRlcCkKPj4+Pj4+ICt7Cj4+Pj4+PiArwqDCoMKgIGNvbnN0 IGZwYl90IGZwYl9mbGFncyA9IEZQQl9JR05PUkVfRElSVFkgfCAKPj4+Pj4+IEZQQl9JR05PUkVf U09GVF9ESVJUWTsKPj4+Pj4+ICvCoMKgwqAgaW50IG1heF9uciA9IGZvbGlvX25yX3BhZ2VzKGZv bGlvKTsKPj4+Pj4KPj4+Pj4gTGV0J3MgYXNzdW1lIHdlIGhhdmUgdGhlIGZpcnN0IHBhZ2Ugb2Yg YSBmb2xpbyBtYXBwZWQgYXQgdGhlIGxhc3QgcGFnZQo+Pj4+PiB0YWJsZSBlbnRyeSBpbiBvdXIg cGFnZSB0YWJsZS4KPj4+Pgo+Pj4+IEdvb2QgcG9pbnQuIEknbSBjdXJpb3VzIGlmIGl0IGlzIHNv bWV0aGluZyB3ZSd2ZSBzZWVuIGluIHByYWN0aWNlIDspCj4+Pgo+Pj4gSSBjaGFsbGVuZ2UgeW91 IHRvIHdyaXRlIGEgcmVwcm9kdWNlciA6UCBJIGFzc3VtZSBpdCBtaWdodCBiZSBkb2FibGUKPj4+ IHRocm91Z2ggc2ltcGxlIG1yZW1hcCgpLgo+Pj4KPj4+Pgo+Pj4+Pgo+Pj4+PiBXaGF0IHByZXZl bnRzIGZvbGlvX3B0ZV9iYXRjaCgpIGZyb20gcmVhZGluZyBvdXRzaWRlIHRoZSBwYWdlIHRhYmxl Pwo+Pj4+Cj4+Pj4gQXNzdW1pbmcgc3VjaCBhIHNjZW5hcmlvIGlzIHBvc3NpYmxlLCB0byBwcmV2 ZW50IGFueSBjaGFuY2Ugb2YgYW4KPj4+PiBvdXQtb2YtYm91bmRzIHJlYWQsIGhvdyBhYm91dCB0 aGlzIGNoYW5nZToKPj4+Pgo+Pj4+IGRpZmYgLS1naXQgYS9tbS9ybWFwLmMgYi9tbS9ybWFwLmMK Pj4+PiBpbmRleCBmYjYzZDkyNTZmMDkuLjlhZWFlODExYTM4YiAxMDA2NDQKPj4+PiAtLS0gYS9t bS9ybWFwLmMKPj4+PiArKysgYi9tbS9ybWFwLmMKPj4+PiBAQCAtMTg1Miw2ICsxODUyLDI1IEBA IHN0YXRpYyBpbmxpbmUgYm9vbAo+Pj4+IGNhbl9iYXRjaF91bm1hcF9mb2xpb19wdGVzKHVuc2ln bmVkIGxvbmcgYWRkciwKPj4+PiDCoMKgwqDCoMKgwqAgY29uc3QgZnBiX3QgZnBiX2ZsYWdzID0g RlBCX0lHTk9SRV9ESVJUWSB8IAo+Pj4+IEZQQl9JR05PUkVfU09GVF9ESVJUWTsKPj4+PiDCoMKg wqDCoMKgwqAgaW50IG1heF9uciA9IGZvbGlvX25yX3BhZ2VzKGZvbGlvKTsKPj4+PiDCoMKgwqDC oMKgwqAgcHRlX3QgcHRlID0gcHRlcF9nZXQocHRlcCk7Cj4+Pj4gK8KgwqDCoCB1bnNpZ25lZCBs b25nIGVuZF9hZGRyOwo+Pj4+ICsKPj4+PiArwqDCoMKgIC8qCj4+Pj4gK8KgwqDCoMKgICogVG8g YmF0Y2ggdW5tYXAsIHRoZSBlbnRpcmUgZm9saW8ncyBQVEVzIG11c3QgYmUgY29udGlndW91cwo+ Pj4+ICvCoMKgwqDCoCAqIGFuZCBtYXBwZWQgd2l0aGluIHRoZSBzYW1lIFBURSBwYWdlIHRhYmxl LCB3aGljaCBjb3JyZXNwb25kcyB0bwo+Pj4+ICvCoMKgwqDCoCAqIGEgc2luZ2xlIFBNRCBlbnRy eS4gQmVmb3JlIGNhbGxpbmcgZm9saW9fcHRlX2JhdGNoKCksIHdoaWNoIAo+Pj4+IGRvZXMKPj4+ PiArwqDCoMKgwqAgKiBub3QgcGVyZm9ybSBib3VuZGFyeSBjaGVja3MgaXRzZWxmLCB3ZSBtdXN0 IHZlcmlmeSB0aGF0IHRoZQo+Pj4+ICvCoMKgwqDCoCAqIGFkZHJlc3MgcmFuZ2UgY292ZXJlZCBi eSB0aGUgZm9saW8gZG9lcyBub3QgY3Jvc3MgYSBQTUQgCj4+Pj4gYm91bmRhcnkuCj4+Pj4gK8Kg wqDCoMKgICovCj4+Pj4gK8KgwqDCoCBlbmRfYWRkciA9IGFkZHIgKyAobWF4X25yICogUEFHRV9T SVpFKSAtIDE7Cj4+Pj4gKwo+Pj4+ICvCoMKgwqAgLyoKPj4+PiArwqDCoMKgwqAgKiBBIGZhc3Qg d2F5IHRvIGNoZWNrIGZvciBhIFBNRCBib3VuZGFyeSBjcm9zcyBpcyB0byBhbGlnbiBib3RoCj4+ Pj4gK8KgwqDCoMKgICogdGhlIHN0YXJ0IGFuZCBlbmQgYWRkcmVzc2VzIHRvIHRoZSBQTUQgYm91 bmRhcnkgYW5kIHNlZSBpZiB0aGV5Cj4+Pj4gK8KgwqDCoMKgICogYXJlIGRpZmZlcmVudC4gSWYg dGhleSBhcmUsIHRoZSByYW5nZSBzcGFucyBhY3Jvc3MgYXQgbGVhc3QgdHdvCj4+Pj4gK8KgwqDC oMKgICogZGlmZmVyZW50IFBNRC1tYW5hZ2VkIHJlZ2lvbnMuCj4+Pj4gK8KgwqDCoMKgICovCj4+ Pj4gK8KgwqDCoCBpZiAoKGFkZHIgJiBQTURfTUFTSykgIT0gKGVuZF9hZGRyICYgUE1EX01BU0sp KQo+Pj4+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4gZmFsc2U7Cj4+Pgo+Pj4gWW91IHNob3VsZCBu b3QgYmUgbWVzc2luZyB3aXRoIG1heF9uciA9IGZvbGlvX25yX3BhZ2VzKGZvbGlvKSBoZXJlIGF0 Cj4+PiBhbGwuIGZvbGlvX3B0ZV9iYXRjaCgpIHRha2VzIGNhcmUgb2YgdGhhdC4KPj4+Cj4+PiBB bHNvLCB3YXkgdG9vIG1hbnkgY29tbWVudHMgOykKPj4+Cj4+PiBZb3UgbWF5IG9ubHkgYmF0Y2gg d2l0aGluIGEgc2luZ2xlIFZNQSBhbmQgd2l0aGluIGEgc2luZ2xlIHBhZ2UgdGFibGUuCj4+Pgo+ Pj4gU28gc2ltcGx5IGFsaWduIHRoZSBhZGRyIHVwIHRvIHRoZSBuZXh0IFBNRCwgYW5kIG1ha2Ug c3VyZSBpdCBkb2VzIG5vdAo+Pj4gZXhjZWVkIHRoZSB2bWEgZW5kLgo+Pj4KPj4+IEFMSUdOIGFu ZCBmcmllbmRzIGNhbiBoZWxwIGF2b2lkaW5nIGV4Y2Vzc2l2ZSBjb21tZW50cy4KPj4KPj4gVGhh bmtzISBIb3cgYWJvdXQgdGhpcyB1cGRhdGVkIHZlcnNpb24gYmFzZWQgb24geW91ciBzdWdnZXN0 aW9uOgo+Pgo+PiBkaWZmIC0tZ2l0IGEvbW0vcm1hcC5jIGIvbW0vcm1hcC5jCj4+IGluZGV4IGZi NjNkOTI1NmYwOS4uMjQxZDU1YTkyYTQ3IDEwMDY0NAo+PiAtLS0gYS9tbS9ybWFwLmMKPj4gKysr IGIvbW0vcm1hcC5jCj4+IEBAIC0xODQ3LDEyICsxODQ3LDI1IEBAIHZvaWQgZm9saW9fcmVtb3Zl X3JtYXBfcHVkKHN0cnVjdCBmb2xpbyAKPj4gKmZvbGlvLCBzdHJ1Y3QgcGFnZSAqcGFnZSwKPj4g wqAgLyogV2Ugc3VwcG9ydCBiYXRjaCB1bm1hcHBpbmcgb2YgUFRFcyBmb3IgbGF6eWZyZWUgbGFy Z2UgZm9saW9zICovCj4+IMKgIHN0YXRpYyBpbmxpbmUgYm9vbCBjYW5fYmF0Y2hfdW5tYXBfZm9s aW9fcHRlcyh1bnNpZ25lZCBsb25nIGFkZHIsCj4+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN0 cnVjdCBmb2xpbyAqZm9saW8sIHB0ZV90ICpwdGVwKQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc3RydWN0IGZvbGlvICpmb2xpbywgcHRlX3Qg KnB0ZXAsCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCBzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSkKPj4gwqAgewo+PiDCoMKgwqDCoMKgIGNv bnN0IGZwYl90IGZwYl9mbGFncyA9IEZQQl9JR05PUkVfRElSVFkgfCBGUEJfSUdOT1JFX1NPRlRf RElSVFk7Cj4+ICvCoMKgwqAgdW5zaWduZWQgbG9uZyBuZXh0X3BtZCwgdm1hX2VuZCwgZW5kX2Fk ZHI7Cj4+IMKgwqDCoMKgwqAgaW50IG1heF9uciA9IGZvbGlvX25yX3BhZ2VzKGZvbGlvKTsKPj4g wqDCoMKgwqDCoCBwdGVfdCBwdGUgPSBwdGVwX2dldChwdGVwKTsKPj4gK8KgwqDCoCAvKgo+PiAr wqDCoMKgwqAgKiBMaW1pdCB0aGUgYmF0Y2ggc2NhbiB3aXRoaW4gYSBzaW5nbGUgVk1BIGFuZCB3 aXRoaW4gYSBzaW5nbGUKPj4gK8KgwqDCoMKgICogcGFnZSB0YWJsZS4KPj4gK8KgwqDCoMKgICov Cj4+ICvCoMKgwqAgdm1hX2VuZCA9IHZtYS0+dm1fZW5kOwo+PiArwqDCoMKgIG5leHRfcG1kID0g QUxJR04oYWRkciArIDEsIFBNRF9TSVpFKTsKPj4gK8KgwqDCoCBlbmRfYWRkciA9IGFkZHIgKyAo dW5zaWduZWQgbG9uZyltYXhfbnIgKiBQQUdFX1NJWkU7Cj4+ICsKPj4gK8KgwqDCoCBpZiAoZW5k X2FkZHIgPiBtaW4obmV4dF9wbWQsIHZtYV9lbmQpKQo+PiArwqDCoMKgwqDCoMKgwqAgcmV0dXJu IGZhbHNlOwo+IAo+IE1heSBJIHN1Z2dlc3QgdGhhdCB3ZSBjbGVhbiBhbGwgdGhhdCB1cCBhcyB3 ZSBmaXggaXQ/CgpZZWFoLCB0aGF0IGxvb2tzIG11Y2ggYmV0dGVyLiBUaGFua3MgZm9yIHRoZSBz dWdnZXN0aW9uIQoKPiAKPiBNYXliZSBzb21ldGhpbmcgbGlrZSB0aGlzOgo+IAo+IGRpZmYgLS1n aXQgYS9tbS9ybWFwLmMgYi9tbS9ybWFwLmMKPiBpbmRleCAzYjc0YmIxOWMxMWRkLi4xMWZiZGRj NmFkOGQ2IDEwMDY0NAo+IC0tLSBhL21tL3JtYXAuYwo+ICsrKyBiL21tL3JtYXAuYwo+IEBAIC0x ODQ1LDIzICsxODQ1LDM4IEBAIHZvaWQgZm9saW9fcmVtb3ZlX3JtYXBfcHVkKHN0cnVjdCBmb2xp byAqZm9saW8sIAo+IHN0cnVjdCBwYWdlICpwYWdlLAo+ICDCoCNlbmRpZgo+ICDCoH0KPiAKPiAt LyogV2Ugc3VwcG9ydCBiYXRjaCB1bm1hcHBpbmcgb2YgUFRFcyBmb3IgbGF6eWZyZWUgbGFyZ2Ug Zm9saW9zICovCj4gLXN0YXRpYyBpbmxpbmUgYm9vbCBjYW5fYmF0Y2hfdW5tYXBfZm9saW9fcHRl cyh1bnNpZ25lZCBsb25nIGFkZHIsCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIHN0cnVjdCBmb2xpbyAqZm9saW8sIHB0ZV90ICpwdGVwKQo+ICtzdGF0aWMg aW5saW5lIHVuc2lnbmVkIGludCBmb2xpb191bm1hcF9wdGVfYmF0Y2goc3RydWN0IGZvbGlvICpm b2xpbywKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgcGFnZV92bWFfbWFw cGVkX3dhbGsgKnB2bXcsIGVudW0gdHR1X2ZsYWdzIGZsYWdzLAo+ICvCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIHB0ZV90IHB0ZSkKPiAgwqB7Cj4gIMKgwqDCoMKgwqDCoMKgIGNvbnN0IGZw Yl90IGZwYl9mbGFncyA9IEZQQl9JR05PUkVfRElSVFkgfCBGUEJfSUdOT1JFX1NPRlRfRElSVFk7 Cj4gLcKgwqDCoMKgwqDCoCBpbnQgbWF4X25yID0gZm9saW9fbnJfcGFnZXMoZm9saW8pOwo+IC3C oMKgwqDCoMKgwqAgcHRlX3QgcHRlID0gcHRlcF9nZXQocHRlcCk7Cj4gK8KgwqDCoMKgwqDCoCBz dHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSA9IHB2bXctPnZtYTsKPiArwqDCoMKgwqDCoMKgIHVu c2lnbmVkIGxvbmcgZW5kX2FkZHIsIGFkZHIgPSBwdm13LT5hZGRyZXNzOwo+ICvCoMKgwqDCoMKg wqAgdW5zaWduZWQgaW50IG1heF9ucjsKPiArCj4gK8KgwqDCoMKgwqDCoCBpZiAoZmxhZ3MgJiBU VFVfSFdQT0lTT04pCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDE7Cj4g K8KgwqDCoMKgwqDCoCBpZiAoIWZvbGlvX3Rlc3RfbGFyZ2UoZm9saW8pKQo+ICvCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiAxOwo+ICsKPiArwqDCoMKgwqDCoMKgIC8qIFdlIG1h eSBvbmx5IGJhdGNoIHdpdGhpbiBhIHNpbmdsZSBWTUEgYW5kIGEgc2luZ2xlIHBhZ2UgCj4gdGFi bGUuICovCj4gK8KgwqDCoMKgwqDCoCBlbmRfYWRkciA9IG1pbl90KHVuc2lnbmVkIGxvbmcsIEFM SUdOKGFkZHIgKyAxLCBQTURfU0laRSksIHZtYS0gCj4gID52bV9lbmQpOwo+ICvCoMKgwqDCoMKg wqAgbWF4X25yID0gKGVuZF9hZGRyIC0gYWRkcikgPj4gUEFHRV9TSElGVDsKPiAKPiArwqDCoMKg wqDCoMKgIC8qIFdlIG9ubHkgc3VwcG9ydCBsYXp5ZnJlZSBiYXRjaGluZyBmb3Igbm93IC4uLiAq Lwo+ICDCoMKgwqDCoMKgwqDCoCBpZiAoIWZvbGlvX3Rlc3RfYW5vbihmb2xpbykgfHwgZm9saW9f dGVzdF9zd2FwYmFja2VkKGZvbGlvKSkKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBy ZXR1cm4gZmFsc2U7Cj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDE7Cj4g IMKgwqDCoMKgwqDCoMKgIGlmIChwdGVfdW51c2VkKHB0ZSkpCj4gLcKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgcmV0dXJuIGZhbHNlOwo+IC3CoMKgwqDCoMKgwqAgaWYgKHB0ZV9wZm4ocHRl KSAhPSBmb2xpb19wZm4oZm9saW8pKQo+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJl dHVybiBmYWxzZTsKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMTsKPiAr wqDCoMKgwqDCoMKgIC8qIC4uLiB3aGVyZSB3ZSBtdXN0IGJlIGFibGUgdG8gYmF0Y2ggdGhlIHdo b2xlIGZvbGlvLiAqLwo+ICvCoMKgwqDCoMKgwqAgaWYgKHB0ZV9wZm4ocHRlKSAhPSBmb2xpb19w Zm4oZm9saW8pIHx8IG1heF9uciAhPSAKPiBmb2xpb19ucl9wYWdlcyhmb2xpbykpCj4gK8KgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDE7Cj4gK8KgwqDCoMKgwqDCoCBtYXhfbnIg PSBmb2xpb19wdGVfYmF0Y2goZm9saW8sIGFkZHIsIHB2bXctPnB0ZSwgcHRlLCBtYXhfbnIsIAo+ IGZwYl9mbGFncywKPiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgTlVMTCwgTlVMTCwgTlVMTCk7Cj4gCj4gLcKgwqDCoMKgwqDC oCByZXR1cm4gZm9saW9fcHRlX2JhdGNoKGZvbGlvLCBhZGRyLCBwdGVwLCBwdGUsIG1heF9uciwg Cj4gZnBiX2ZsYWdzLCBOVUxMLAo+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIE5VTEwsIE5VTEwpID09IG1heF9ucjsKPiArwqDCoMKg wqDCoMKgIGlmIChtYXhfbnIgIT0gZm9saW9fbnJfcGFnZXMoZm9saW8pKQo+ICvCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiAxOwo+ICvCoMKgwqDCoMKgwqAgcmV0dXJuIG1heF9u cjsKPiAgwqB9Cj4gCj4gIMKgLyoKPiBAQCAtMjAyNCw5ICsyMDM5LDcgQEAgc3RhdGljIGJvb2wg dHJ5X3RvX3VubWFwX29uZShzdHJ1Y3QgZm9saW8gKmZvbGlvLCAKPiBzdHJ1Y3Qgdm1fYXJlYV9z dHJ1Y3QgKnZtYSwKPiAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoCBpZiAocHRlX2RpcnR5KHB0ZXZhbCkpCj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGZvbGlvX21hcmtfZGlydHkoZm9s aW8pOwo+ICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgfSBlbHNlIGlmIChsaWtlbHko cHRlX3ByZXNlbnQocHRldmFsKSkpIHsKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgaWYgKGZvbGlvX3Rlc3RfbGFyZ2UoZm9saW8pICYmICEoZmxhZ3MgJiAK PiBUVFVfSFdQT0lTT04pICYmCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgY2FuX2JhdGNoX3VubWFwX2ZvbGlvX3B0ZXMoYWRkcmVzcywgZm9s aW8sIAo+IHB2bXcucHRlKSkKPiAtwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIG5yX3BhZ2VzID0gZm9saW9fbnJfcGFnZXMoZm9saW8p Owo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBucl9wYWdl cyA9IGZvbGlvX3VubWFwX3B0ZV9iYXRjaChmb2xpbywgJnB2bXcsIAo+IGZsYWdzLCBwdGV2YWwp Owo+ICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGVuZF9h ZGRyID0gYWRkcmVzcyArIG5yX3BhZ2VzICogUEFHRV9TSVpFOwo+ICDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGZsdXNoX2NhY2hlX3JhbmdlKHZtYSwgYWRk cmVzcywgZW5kX2FkZHIpOwo+IAo+IAo+IE5vdGUgdGhhdCBJIGRvbid0IHF1aXRlIHVuZGVyc3Rh bmQgd2h5IHdlIGhhdmUgdG8gYmF0Y2ggdGhlIHdob2xlIHRoaW5nIAo+IG9yIGZhbGxiYWNrIHRv Cj4gaW5kaXZpZHVhbCBwYWdlcy4gV2h5IGNhbid0IHdlIHBlcmZvcm0gb3RoZXIgYmF0Y2hlcyB0 aGF0IHNwYW4gb25seSBzb21lIAo+IFBURXM/IFdoYXQncyBzcGVjaWFsCj4gYWJvdXQgMSBQVEUg dnMuIDIgUFRFcyB2cy4gYWxsIFBURXM/CgpUaGF0J3MgYSBnb29kIHBvaW50IGFib3V0IHRoZSAi YWxsLW9yLW5vdGhpbmciIGJhdGNoaW5nIGxvZ2ljIDspCgpJdCBzZWVtcyB0aGUgImFsbC1vci1u b3RoaW5nIiBhcHByb2FjaCBpcyBzcGVjaWZpYyB0byB0aGUgbGF6eWZyZWUgdXNlCmNhc2UsIHdo aWNoIG5lZWRzIHRvIHVubWFwIHRoZSBlbnRpcmUgZm9saW8gZm9yIHJlY2xhbWF0aW9uLiBJZiB0 aGF0J3MKbm90IHBvc3NpYmxlLCBpdCBmYWxscyBiYWNrIHRvIHRoZSBzaW5nbGUtcGFnZSBzbG93 IHBhdGguCgpBbHNvLCBzdXBwb3J0aW5nIHBhcnRpYWwgYmF0Y2hlcyB3b3VsZCBiZSB1c2VmdWws IGJ1dCBub3QgY29tbW9uIGNhc2UKSSBndWVzcywgc28gbGV0J3MgbGVhdmUgaXQgYXMgaXMgO3AK ClRoYW5rcywKTGFuY2UKCj4gCj4gCj4gQ2FuIHNvbWVvbmUgZW5saWdodGVuIG1lIHdoeSB0aGF0 IGlzIHJlcXVpcmVkPwo+IAoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fCmxpbnV4LXJpc2N2IG1haWxpbmcgbGlzdApsaW51eC1yaXNjdkBsaXN0cy5pbmZy YWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGlu dXgtcmlzY3YK