From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9994CA9ED3 for ; Mon, 4 Nov 2019 23:49:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 74FD6214E0 for ; Mon, 4 Nov 2019 23:49:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X/bC82UW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729945AbfKDXtk (ORCPT ); Mon, 4 Nov 2019 18:49:40 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:39350 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729916AbfKDXtk (ORCPT ); Mon, 4 Nov 2019 18:49:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572911379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ls7Zx2AkPrMXy+LPLj5cuy1SX8JbSG4GMzPShaYe1yw=; b=X/bC82UWeFKjrAylGSFPS1m1bVGp8DkLnZI5Vvc+9RsaPuwgjjTrTZ6U1mb2Pxf8QSnDRu 4VF/0gIsLyOlSs3c0chs4ed48bj/Ynkw6aX7jOiFRw6UVaHEQz5hvHPHcxNUf+Ovx5lOZ4 q4jEQzfF1zoBWTsy3aLFyuxDtA4euDo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-397-aDWO2QFVNLO1s54amWQi-Q-1; Mon, 04 Nov 2019 18:49:35 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F0E0D107ACC2; Mon, 4 Nov 2019 23:49:30 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.178]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D177919C4F; Mon, 4 Nov 2019 23:49:21 +0000 (UTC) Date: Mon, 4 Nov 2019 18:49:20 -0500 From: Jerome Glisse To: John Hubbard Cc: Andrew Morton , Al Viro , Alex Williamson , Benjamin Herrenschmidt , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , Christoph Hellwig , Dan Williams , Daniel Vetter , Dave Chinner , David Airlie , "David S . Miller" , Ira Weiny , Jan Kara , Jason Gunthorpe , Jens Axboe , Jonathan Corbet , Magnus Karlsson , Mauro Carvalho Chehab , Michael Ellerman , Michal Hocko , Mike Kravetz , Paul Mackerras , Shuah Khan , Vlastimil Babka , bpf@vger.kernel.org, dri-devel@lists.freedesktop.org, kvm@vger.kernel.org, linux-block@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, linux-rdma@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, netdev@vger.kernel.org, linux-mm@kvack.org, LKML Subject: Re: [PATCH v2 12/18] mm/gup: track FOLL_PIN pages Message-ID: <20191104234920.GA18515@redhat.com> References: <20191103211813.213227-1-jhubbard@nvidia.com> <20191103211813.213227-13-jhubbard@nvidia.com> <20191104185238.GG5134@redhat.com> <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> MIME-Version: 1.0 In-Reply-To: <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: aDWO2QFVNLO1s54amWQi-Q-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Mon, Nov 04, 2019 at 02:49:18PM -0800, John Hubbard wrote: > On 11/4/19 10:52 AM, Jerome Glisse wrote: > > On Sun, Nov 03, 2019 at 01:18:07PM -0800, John Hubbard wrote: > >> Add tracking of pages that were pinned via FOLL_PIN. > >> > >> As mentioned in the FOLL_PIN documentation, callers who effectively se= t > >> FOLL_PIN are required to ultimately free such pages via put_user_page(= ). > >> The effect is similar to FOLL_GET, and may be thought of as "FOLL_GET > >> for DIO and/or RDMA use". > >> > >> Pages that have been pinned via FOLL_PIN are identifiable via a > >> new function call: > >> > >> bool page_dma_pinned(struct page *page); > >> > >> What to do in response to encountering such a page, is left to later > >> patchsets. There is discussion about this in [1]. > >> > >> This also changes a BUG_ON(), to a WARN_ON(), in follow_page_mask(). > >> > >> This also has a couple of trivial, non-functional change fixes to > >> try_get_compound_head(). That function got moved to the top of the > >> file. > >=20 > > Maybe split that as a separate trivial patch. >=20 >=20 > Will do. >=20 >=20 > >=20 > >> > >> This includes the following fix from Ira Weiny: > >> > >> DAX requires detection of a page crossing to a ref count of 1. Fix th= is > >> for GUP pages by introducing put_devmap_managed_user_page() which > >> accounts for GUP_PIN_COUNTING_BIAS now used by GUP. > >=20 > > Please do the put_devmap_managed_page() changes in a separate > > patch, it would be a lot easier to follow, also on that front > > see comments below. >=20 >=20 > Oh! OK. It makes sense when you say it out loud. :) >=20 >=20 > ... > >> +static inline bool put_devmap_managed_page(struct page *page) > >> +{ > >> +=09bool is_devmap =3D page_is_devmap_managed(page); > >> + > >> +=09if (is_devmap) { > >> +=09=09int count =3D page_ref_dec_return(page); > >> + > >> +=09=09__put_devmap_managed_page(page, count); > >> +=09} > >> + > >> +=09return is_devmap; > >> +} > >=20 > > I think the __put_devmap_managed_page() should be rename > > to free_devmap_managed_page() and that the count !=3D 1 > > case move to this inline function ie: > >=20 > > static inline bool put_devmap_managed_page(struct page *page) > > { > > =09bool is_devmap =3D page_is_devmap_managed(page); > >=20 > > =09if (is_devmap) { > > =09=09int count =3D page_ref_dec_return(page); > >=20 > > =09=09/* > > =09=09 * If refcount is 1 then page is freed and refcount is stable as = nobody > > =09=09 * holds a reference on the page. > > =09=09 */ > > =09=09if (count =3D=3D 1) > > =09=09=09free_devmap_managed_page(page, count); > > =09=09else if (!count) > > =09=09=09__put_page(page); > > =09} > >=20 > > =09return is_devmap; > > } > >=20 >=20 > Thanks, that does look cleaner and easier to read. >=20 > >=20 > >> + > >> #else /* CONFIG_DEV_PAGEMAP_OPS */ > >> static inline bool put_devmap_managed_page(struct page *page) > >> { > >> @@ -1038,6 +1051,8 @@ static inline __must_check bool try_get_page(str= uct page *page) > >> =09return true; > >> } > >> =20 > >> +__must_check bool user_page_ref_inc(struct page *page); > >> + > >=20 > > What about having it as an inline here as it is pretty small. >=20 >=20 > You mean move it to a static inline function in mm.h? It's worse than it= =20 > looks, though: *everything* that it calls is also a static function, loca= l > to gup.c. So I'd have to expose both try_get_compound_head() and > __update_proc_vmstat(). And that also means calling mod_node_page_state()= from > mm.h, and it goes south right about there. :) Ok fair enough > ... =20 > >> +/** > >> + * page_dma_pinned() - report if a page is pinned by a call to pin_us= er_pages*() > >> + * or pin_longterm_pages*() > >> + * @page:=09pointer to page to be queried. > >> + * @Return:=09True, if it is likely that the page has been "dma-pinne= d". > >> + *=09=09False, if the page is definitely not dma-pinned. > >> + */ > >=20 > > Maybe add a small comment about wrap around :) >=20 >=20 > I don't *think* the count can wrap around, due to the checks in user_page= _ref_inc(). >=20 > But it's true that the documentation is a little light here...What did yo= u have=20 > in mind? About false positive case (and how unlikely they are) and that wrap around is properly handle. Maybe just a pointer to the documentation so that people know they can go look there for details. I know my brain tend to forget where to look for things so i like to be constantly reminded hey the doc is Documentations/foobar :) > > [...] > >=20 > >> @@ -1930,12 +2028,20 @@ static int __gup_device_huge(unsigned long pfn= , unsigned long addr, > >> =20 > >> =09=09pgmap =3D get_dev_pagemap(pfn, pgmap); > >> =09=09if (unlikely(!pgmap)) { > >> -=09=09=09undo_dev_pagemap(nr, nr_start, pages); > >> +=09=09=09undo_dev_pagemap(nr, nr_start, flags, pages); > >> =09=09=09return 0; > >> =09=09} > >> =09=09SetPageReferenced(page); > >> =09=09pages[*nr] =3D page; > >> -=09=09get_page(page); > >> + > >> +=09=09if (flags & FOLL_PIN) { > >> +=09=09=09if (unlikely(!user_page_ref_inc(page))) { > >> +=09=09=09=09undo_dev_pagemap(nr, nr_start, flags, pages); > >> +=09=09=09=09return 0; > >> +=09=09=09} > >=20 > > Maybe add a comment about a case that should never happens ie > > user_page_ref_inc() fails after the second iteration of the > > loop as it would be broken and a bug to call undo_dev_pagemap() > > after the first iteration of that loop. > >=20 > > Also i believe that this should never happens as if first > > iteration succeed than __page_cache_add_speculative() will > > succeed for all the iterations. > >=20 > > Note that the pgmap case above follows that too ie the call to > > get_dev_pagemap() can only fail on first iteration of the loop, > > well i assume you can never have a huge device page that span > > different pgmap ie different devices (which is a reasonable > > assumption). So maybe this code needs fixing ie : > >=20 > > =09=09pgmap =3D get_dev_pagemap(pfn, pgmap); > > =09=09if (unlikely(!pgmap)) > > =09=09=09return 0; > >=20 > >=20 >=20 > OK, yes that does make sense. And I think a comment is adequate, > no need to check for bugs during every tail page iteration. So how=20 > about this, as a preliminary patch: Actualy i thought about it and i think that there is pgmap per section and thus maybe one device can have multiple pgmap and that would be an issue for page bigger than section size (ie bigger than 128MB iirc). I will go double check that, but maybe Dan can chime in. In any case my comment above is correct for the page ref increment, if the first one succeed than others will too or otherwise it means someone is doing too many put_page()/ put_user_page() which is _bad_ :) >=20 > diff --git a/mm/gup.c b/mm/gup.c > index 8f236a335ae9..a4a81e125832 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -1892,17 +1892,18 @@ static int gup_pte_range(pmd_t pmd, unsigned long= addr, unsigned long end, > static int __gup_device_huge(unsigned long pfn, unsigned long addr, > unsigned long end, struct page **pages, int *nr) > { > - int nr_start =3D *nr; > - struct dev_pagemap *pgmap =3D NULL; > + /* > + * Huge pages should never cross dev_pagemap boundaries. Therefor= e, use > + * this same pgmap for the entire huge page. > + */ > + struct dev_pagemap *pgmap =3D get_dev_pagemap(pfn, NULL); > + > + if (unlikely(!pgmap)) > + return 0; > =20 > do { > struct page *page =3D pfn_to_page(pfn); > =20 > - pgmap =3D get_dev_pagemap(pfn, pgmap); > - if (unlikely(!pgmap)) { > - undo_dev_pagemap(nr, nr_start, pages); > - return 0; > - } > SetPageReferenced(page); > pages[*nr] =3D page; > get_page(page); >=20 >=20 >=20 >=20 > >> +=09=09} else > >> +=09=09=09get_page(page); > >> + > >> =09=09(*nr)++; > >> =09=09pfn++; > >> =09} while (addr +=3D PAGE_SIZE, addr !=3D end); > >=20 > > [...] > >=20 > >> @@ -2409,7 +2540,7 @@ static int internal_get_user_pages_fast(unsigned= long start, int nr_pages, > >> =09unsigned long addr, len, end; > >> =09int nr =3D 0, ret =3D 0; > >> =20 > >> -=09if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM))) > >> +=09if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_P= IN))) > >=20 > > Maybe add a comments to explain, something like: > >=20 > > /* > > * The only flags allowed here are: FOLL_WRITE, FOLL_LONGTERM, FOLL_PIN > > * > > * Note that get_user_pages_fast() imply FOLL_GET flag by default but > > * callers can over-ride this default to pin case by setting FOLL_PIN. > > */ >=20 > Good idea. Here's the draft now: >=20 > /* > * The only flags allowed here are: FOLL_WRITE, FOLL_LONGTERM, FOLL_PIN. > * > * Note that get_user_pages_fast() implies FOLL_GET flag by default, but > * callers can override this default by setting FOLL_PIN instead of > * FOLL_GET. > */ > if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_PIN))) > return -EINVAL; Looks good to me. ... Cheers, J=E9r=F4me From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 554A9CA9EC9 for ; Mon, 4 Nov 2019 23:52:27 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A3252089C for ; Mon, 4 Nov 2019 23:52:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X/bC82UW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A3252089C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 476V2v6zt4zF1fp for ; Tue, 5 Nov 2019 10:52:23 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=207.211.31.120; helo=us-smtp-1.mimecast.com; envelope-from=jglisse@redhat.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="X/bC82UW"; dkim-atps=neutral Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 476Tzq55NZzF3qh for ; Tue, 5 Nov 2019 10:49:42 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572911379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ls7Zx2AkPrMXy+LPLj5cuy1SX8JbSG4GMzPShaYe1yw=; b=X/bC82UWeFKjrAylGSFPS1m1bVGp8DkLnZI5Vvc+9RsaPuwgjjTrTZ6U1mb2Pxf8QSnDRu 4VF/0gIsLyOlSs3c0chs4ed48bj/Ynkw6aX7jOiFRw6UVaHEQz5hvHPHcxNUf+Ovx5lOZ4 q4jEQzfF1zoBWTsy3aLFyuxDtA4euDo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-397-aDWO2QFVNLO1s54amWQi-Q-1; Mon, 04 Nov 2019 18:49:35 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F0E0D107ACC2; Mon, 4 Nov 2019 23:49:30 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.178]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D177919C4F; Mon, 4 Nov 2019 23:49:21 +0000 (UTC) Date: Mon, 4 Nov 2019 18:49:20 -0500 From: Jerome Glisse To: John Hubbard Subject: Re: [PATCH v2 12/18] mm/gup: track FOLL_PIN pages Message-ID: <20191104234920.GA18515@redhat.com> References: <20191103211813.213227-1-jhubbard@nvidia.com> <20191103211813.213227-13-jhubbard@nvidia.com> <20191104185238.GG5134@redhat.com> <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> MIME-Version: 1.0 In-Reply-To: <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: aDWO2QFVNLO1s54amWQi-Q-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Jonathan Corbet , linux-rdma@vger.kernel.org, Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, Alex Williamson , Al Viro , Dan Williams , Mauro Carvalho Chehab , bpf@vger.kernel.org, Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, LKML , Daniel Vetter , linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, Nov 04, 2019 at 02:49:18PM -0800, John Hubbard wrote: > On 11/4/19 10:52 AM, Jerome Glisse wrote: > > On Sun, Nov 03, 2019 at 01:18:07PM -0800, John Hubbard wrote: > >> Add tracking of pages that were pinned via FOLL_PIN. > >> > >> As mentioned in the FOLL_PIN documentation, callers who effectively se= t > >> FOLL_PIN are required to ultimately free such pages via put_user_page(= ). > >> The effect is similar to FOLL_GET, and may be thought of as "FOLL_GET > >> for DIO and/or RDMA use". > >> > >> Pages that have been pinned via FOLL_PIN are identifiable via a > >> new function call: > >> > >> bool page_dma_pinned(struct page *page); > >> > >> What to do in response to encountering such a page, is left to later > >> patchsets. There is discussion about this in [1]. > >> > >> This also changes a BUG_ON(), to a WARN_ON(), in follow_page_mask(). > >> > >> This also has a couple of trivial, non-functional change fixes to > >> try_get_compound_head(). That function got moved to the top of the > >> file. > >=20 > > Maybe split that as a separate trivial patch. >=20 >=20 > Will do. >=20 >=20 > >=20 > >> > >> This includes the following fix from Ira Weiny: > >> > >> DAX requires detection of a page crossing to a ref count of 1. Fix th= is > >> for GUP pages by introducing put_devmap_managed_user_page() which > >> accounts for GUP_PIN_COUNTING_BIAS now used by GUP. > >=20 > > Please do the put_devmap_managed_page() changes in a separate > > patch, it would be a lot easier to follow, also on that front > > see comments below. >=20 >=20 > Oh! OK. It makes sense when you say it out loud. :) >=20 >=20 > ... > >> +static inline bool put_devmap_managed_page(struct page *page) > >> +{ > >> +=09bool is_devmap =3D page_is_devmap_managed(page); > >> + > >> +=09if (is_devmap) { > >> +=09=09int count =3D page_ref_dec_return(page); > >> + > >> +=09=09__put_devmap_managed_page(page, count); > >> +=09} > >> + > >> +=09return is_devmap; > >> +} > >=20 > > I think the __put_devmap_managed_page() should be rename > > to free_devmap_managed_page() and that the count !=3D 1 > > case move to this inline function ie: > >=20 > > static inline bool put_devmap_managed_page(struct page *page) > > { > > =09bool is_devmap =3D page_is_devmap_managed(page); > >=20 > > =09if (is_devmap) { > > =09=09int count =3D page_ref_dec_return(page); > >=20 > > =09=09/* > > =09=09 * If refcount is 1 then page is freed and refcount is stable as = nobody > > =09=09 * holds a reference on the page. > > =09=09 */ > > =09=09if (count =3D=3D 1) > > =09=09=09free_devmap_managed_page(page, count); > > =09=09else if (!count) > > =09=09=09__put_page(page); > > =09} > >=20 > > =09return is_devmap; > > } > >=20 >=20 > Thanks, that does look cleaner and easier to read. >=20 > >=20 > >> + > >> #else /* CONFIG_DEV_PAGEMAP_OPS */ > >> static inline bool put_devmap_managed_page(struct page *page) > >> { > >> @@ -1038,6 +1051,8 @@ static inline __must_check bool try_get_page(str= uct page *page) > >> =09return true; > >> } > >> =20 > >> +__must_check bool user_page_ref_inc(struct page *page); > >> + > >=20 > > What about having it as an inline here as it is pretty small. >=20 >=20 > You mean move it to a static inline function in mm.h? It's worse than it= =20 > looks, though: *everything* that it calls is also a static function, loca= l > to gup.c. So I'd have to expose both try_get_compound_head() and > __update_proc_vmstat(). And that also means calling mod_node_page_state()= from > mm.h, and it goes south right about there. :) Ok fair enough > ... =20 > >> +/** > >> + * page_dma_pinned() - report if a page is pinned by a call to pin_us= er_pages*() > >> + * or pin_longterm_pages*() > >> + * @page:=09pointer to page to be queried. > >> + * @Return:=09True, if it is likely that the page has been "dma-pinne= d". > >> + *=09=09False, if the page is definitely not dma-pinned. > >> + */ > >=20 > > Maybe add a small comment about wrap around :) >=20 >=20 > I don't *think* the count can wrap around, due to the checks in user_page= _ref_inc(). >=20 > But it's true that the documentation is a little light here...What did yo= u have=20 > in mind? About false positive case (and how unlikely they are) and that wrap around is properly handle. Maybe just a pointer to the documentation so that people know they can go look there for details. I know my brain tend to forget where to look for things so i like to be constantly reminded hey the doc is Documentations/foobar :) > > [...] > >=20 > >> @@ -1930,12 +2028,20 @@ static int __gup_device_huge(unsigned long pfn= , unsigned long addr, > >> =20 > >> =09=09pgmap =3D get_dev_pagemap(pfn, pgmap); > >> =09=09if (unlikely(!pgmap)) { > >> -=09=09=09undo_dev_pagemap(nr, nr_start, pages); > >> +=09=09=09undo_dev_pagemap(nr, nr_start, flags, pages); > >> =09=09=09return 0; > >> =09=09} > >> =09=09SetPageReferenced(page); > >> =09=09pages[*nr] =3D page; > >> -=09=09get_page(page); > >> + > >> +=09=09if (flags & FOLL_PIN) { > >> +=09=09=09if (unlikely(!user_page_ref_inc(page))) { > >> +=09=09=09=09undo_dev_pagemap(nr, nr_start, flags, pages); > >> +=09=09=09=09return 0; > >> +=09=09=09} > >=20 > > Maybe add a comment about a case that should never happens ie > > user_page_ref_inc() fails after the second iteration of the > > loop as it would be broken and a bug to call undo_dev_pagemap() > > after the first iteration of that loop. > >=20 > > Also i believe that this should never happens as if first > > iteration succeed than __page_cache_add_speculative() will > > succeed for all the iterations. > >=20 > > Note that the pgmap case above follows that too ie the call to > > get_dev_pagemap() can only fail on first iteration of the loop, > > well i assume you can never have a huge device page that span > > different pgmap ie different devices (which is a reasonable > > assumption). So maybe this code needs fixing ie : > >=20 > > =09=09pgmap =3D get_dev_pagemap(pfn, pgmap); > > =09=09if (unlikely(!pgmap)) > > =09=09=09return 0; > >=20 > >=20 >=20 > OK, yes that does make sense. And I think a comment is adequate, > no need to check for bugs during every tail page iteration. So how=20 > about this, as a preliminary patch: Actualy i thought about it and i think that there is pgmap per section and thus maybe one device can have multiple pgmap and that would be an issue for page bigger than section size (ie bigger than 128MB iirc). I will go double check that, but maybe Dan can chime in. In any case my comment above is correct for the page ref increment, if the first one succeed than others will too or otherwise it means someone is doing too many put_page()/ put_user_page() which is _bad_ :) >=20 > diff --git a/mm/gup.c b/mm/gup.c > index 8f236a335ae9..a4a81e125832 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -1892,17 +1892,18 @@ static int gup_pte_range(pmd_t pmd, unsigned long= addr, unsigned long end, > static int __gup_device_huge(unsigned long pfn, unsigned long addr, > unsigned long end, struct page **pages, int *nr) > { > - int nr_start =3D *nr; > - struct dev_pagemap *pgmap =3D NULL; > + /* > + * Huge pages should never cross dev_pagemap boundaries. Therefor= e, use > + * this same pgmap for the entire huge page. > + */ > + struct dev_pagemap *pgmap =3D get_dev_pagemap(pfn, NULL); > + > + if (unlikely(!pgmap)) > + return 0; > =20 > do { > struct page *page =3D pfn_to_page(pfn); > =20 > - pgmap =3D get_dev_pagemap(pfn, pgmap); > - if (unlikely(!pgmap)) { > - undo_dev_pagemap(nr, nr_start, pages); > - return 0; > - } > SetPageReferenced(page); > pages[*nr] =3D page; > get_page(page); >=20 >=20 >=20 >=20 > >> +=09=09} else > >> +=09=09=09get_page(page); > >> + > >> =09=09(*nr)++; > >> =09=09pfn++; > >> =09} while (addr +=3D PAGE_SIZE, addr !=3D end); > >=20 > > [...] > >=20 > >> @@ -2409,7 +2540,7 @@ static int internal_get_user_pages_fast(unsigned= long start, int nr_pages, > >> =09unsigned long addr, len, end; > >> =09int nr =3D 0, ret =3D 0; > >> =20 > >> -=09if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM))) > >> +=09if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_P= IN))) > >=20 > > Maybe add a comments to explain, something like: > >=20 > > /* > > * The only flags allowed here are: FOLL_WRITE, FOLL_LONGTERM, FOLL_PIN > > * > > * Note that get_user_pages_fast() imply FOLL_GET flag by default but > > * callers can over-ride this default to pin case by setting FOLL_PIN. > > */ >=20 > Good idea. Here's the draft now: >=20 > /* > * The only flags allowed here are: FOLL_WRITE, FOLL_LONGTERM, FOLL_PIN. > * > * Note that get_user_pages_fast() implies FOLL_GET flag by default, but > * callers can override this default by setting FOLL_PIN instead of > * FOLL_GET. > */ > if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_PIN))) > return -EINVAL; Looks good to me. ... Cheers, J=E9r=F4me From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [PATCH v2 12/18] mm/gup: track FOLL_PIN pages Date: Mon, 4 Nov 2019 18:49:20 -0500 Message-ID: <20191104234920.GA18515@redhat.com> References: <20191103211813.213227-1-jhubbard@nvidia.com> <20191103211813.213227-13-jhubbard@nvidia.com> <20191104185238.GG5134@redhat.com> <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> Content-Disposition: inline Sender: netdev-owner@vger.kernel.org To: John Hubbard Cc: Andrew Morton , Al Viro , Alex Williamson , Benjamin Herrenschmidt , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , Christoph Hellwig , Dan Williams , Daniel Vetter , Dave Chinner , David Airlie , "David S . Miller" , Ira Weiny , Jan Kara , Jason Gunthorpe , Jens Axboe , Jonathan Corbet , Magnus Karlsson , Mauro Carvalho Chehab , Michael Ellerman , Michal Hocko List-Id: dri-devel@lists.freedesktop.org On Mon, Nov 04, 2019 at 02:49:18PM -0800, John Hubbard wrote: > On 11/4/19 10:52 AM, Jerome Glisse wrote: > > On Sun, Nov 03, 2019 at 01:18:07PM -0800, John Hubbard wrote: > >> Add tracking of pages that were pinned via FOLL_PIN. > >> > >> As mentioned in the FOLL_PIN documentation, callers who effectively se= t > >> FOLL_PIN are required to ultimately free such pages via put_user_page(= ). > >> The effect is similar to FOLL_GET, and may be thought of as "FOLL_GET > >> for DIO and/or RDMA use". > >> > >> Pages that have been pinned via FOLL_PIN are identifiable via a > >> new function call: > >> > >> bool page_dma_pinned(struct page *page); > >> > >> What to do in response to encountering such a page, is left to later > >> patchsets. There is discussion about this in [1]. > >> > >> This also changes a BUG_ON(), to a WARN_ON(), in follow_page_mask(). > >> > >> This also has a couple of trivial, non-functional change fixes to > >> try_get_compound_head(). That function got moved to the top of the > >> file. > >=20 > > Maybe split that as a separate trivial patch. >=20 >=20 > Will do. >=20 >=20 > >=20 > >> > >> This includes the following fix from Ira Weiny: > >> > >> DAX requires detection of a page crossing to a ref count of 1. Fix th= is > >> for GUP pages by introducing put_devmap_managed_user_page() which > >> accounts for GUP_PIN_COUNTING_BIAS now used by GUP. > >=20 > > Please do the put_devmap_managed_page() changes in a separate > > patch, it would be a lot easier to follow, also on that front > > see comments below. >=20 >=20 > Oh! OK. It makes sense when you say it out loud. :) >=20 >=20 > ... > >> +static inline bool put_devmap_managed_page(struct page *page) > >> +{ > >> +=09bool is_devmap =3D page_is_devmap_managed(page); > >> + > >> +=09if (is_devmap) { > >> +=09=09int count =3D page_ref_dec_return(page); > >> + > >> +=09=09__put_devmap_managed_page(page, count); > >> +=09} > >> + > >> +=09return is_devmap; > >> +} > >=20 > > I think the __put_devmap_managed_page() should be rename > > to free_devmap_managed_page() and that the count !=3D 1 > > case move to this inline function ie: > >=20 > > static inline bool put_devmap_managed_page(struct page *page) > > { > > =09bool is_devmap =3D page_is_devmap_managed(page); > >=20 > > =09if (is_devmap) { > > =09=09int count =3D page_ref_dec_return(page); > >=20 > > =09=09/* > > =09=09 * If refcount is 1 then page is freed and refcount is stable as = nobody > > =09=09 * holds a reference on the page. > > =09=09 */ > > =09=09if (count =3D=3D 1) > > =09=09=09free_devmap_managed_page(page, count); > > =09=09else if (!count) > > =09=09=09__put_page(page); > > =09} > >=20 > > =09return is_devmap; > > } > >=20 >=20 > Thanks, that does look cleaner and easier to read. >=20 > >=20 > >> + > >> #else /* CONFIG_DEV_PAGEMAP_OPS */ > >> static inline bool put_devmap_managed_page(struct page *page) > >> { > >> @@ -1038,6 +1051,8 @@ static inline __must_check bool try_get_page(str= uct page *page) > >> =09return true; > >> } > >> =20 > >> +__must_check bool user_page_ref_inc(struct page *page); > >> + > >=20 > > What about having it as an inline here as it is pretty small. >=20 >=20 > You mean move it to a static inline function in mm.h? It's worse than it= =20 > looks, though: *everything* that it calls is also a static function, loca= l > to gup.c. So I'd have to expose both try_get_compound_head() and > __update_proc_vmstat(). And that also means calling mod_node_page_state()= from > mm.h, and it goes south right about there. :) Ok fair enough > ... =20 > >> +/** > >> + * page_dma_pinned() - report if a page is pinned by a call to pin_us= er_pages*() > >> + * or pin_longterm_pages*() > >> + * @page:=09pointer to page to be queried. > >> + * @Return:=09True, if it is likely that the page has been "dma-pinne= d". > >> + *=09=09False, if the page is definitely not dma-pinned. > >> + */ > >=20 > > Maybe add a small comment about wrap around :) >=20 >=20 > I don't *think* the count can wrap around, due to the checks in user_page= _ref_inc(). >=20 > But it's true that the documentation is a little light here...What did yo= u have=20 > in mind? About false positive case (and how unlikely they are) and that wrap around is properly handle. Maybe just a pointer to the documentation so that people know they can go look there for details. I know my brain tend to forget where to look for things so i like to be constantly reminded hey the doc is Documentations/foobar :) > > [...] > >=20 > >> @@ -1930,12 +2028,20 @@ static int __gup_device_huge(unsigned long pfn= , unsigned long addr, > >> =20 > >> =09=09pgmap =3D get_dev_pagemap(pfn, pgmap); > >> =09=09if (unlikely(!pgmap)) { > >> -=09=09=09undo_dev_pagemap(nr, nr_start, pages); > >> +=09=09=09undo_dev_pagemap(nr, nr_start, flags, pages); > >> =09=09=09return 0; > >> =09=09} > >> =09=09SetPageReferenced(page); > >> =09=09pages[*nr] =3D page; > >> -=09=09get_page(page); > >> + > >> +=09=09if (flags & FOLL_PIN) { > >> +=09=09=09if (unlikely(!user_page_ref_inc(page))) { > >> +=09=09=09=09undo_dev_pagemap(nr, nr_start, flags, pages); > >> +=09=09=09=09return 0; > >> +=09=09=09} > >=20 > > Maybe add a comment about a case that should never happens ie > > user_page_ref_inc() fails after the second iteration of the > > loop as it would be broken and a bug to call undo_dev_pagemap() > > after the first iteration of that loop. > >=20 > > Also i believe that this should never happens as if first > > iteration succeed than __page_cache_add_speculative() will > > succeed for all the iterations. > >=20 > > Note that the pgmap case above follows that too ie the call to > > get_dev_pagemap() can only fail on first iteration of the loop, > > well i assume you can never have a huge device page that span > > different pgmap ie different devices (which is a reasonable > > assumption). So maybe this code needs fixing ie : > >=20 > > =09=09pgmap =3D get_dev_pagemap(pfn, pgmap); > > =09=09if (unlikely(!pgmap)) > > =09=09=09return 0; > >=20 > >=20 >=20 > OK, yes that does make sense. And I think a comment is adequate, > no need to check for bugs during every tail page iteration. So how=20 > about this, as a preliminary patch: Actualy i thought about it and i think that there is pgmap per section and thus maybe one device can have multiple pgmap and that would be an issue for page bigger than section size (ie bigger than 128MB iirc). I will go double check that, but maybe Dan can chime in. In any case my comment above is correct for the page ref increment, if the first one succeed than others will too or otherwise it means someone is doing too many put_page()/ put_user_page() which is _bad_ :) >=20 > diff --git a/mm/gup.c b/mm/gup.c > index 8f236a335ae9..a4a81e125832 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -1892,17 +1892,18 @@ static int gup_pte_range(pmd_t pmd, unsigned long= addr, unsigned long end, > static int __gup_device_huge(unsigned long pfn, unsigned long addr, > unsigned long end, struct page **pages, int *nr) > { > - int nr_start =3D *nr; > - struct dev_pagemap *pgmap =3D NULL; > + /* > + * Huge pages should never cross dev_pagemap boundaries. Therefor= e, use > + * this same pgmap for the entire huge page. > + */ > + struct dev_pagemap *pgmap =3D get_dev_pagemap(pfn, NULL); > + > + if (unlikely(!pgmap)) > + return 0; > =20 > do { > struct page *page =3D pfn_to_page(pfn); > =20 > - pgmap =3D get_dev_pagemap(pfn, pgmap); > - if (unlikely(!pgmap)) { > - undo_dev_pagemap(nr, nr_start, pages); > - return 0; > - } > SetPageReferenced(page); > pages[*nr] =3D page; > get_page(page); >=20 >=20 >=20 >=20 > >> +=09=09} else > >> +=09=09=09get_page(page); > >> + > >> =09=09(*nr)++; > >> =09=09pfn++; > >> =09} while (addr +=3D PAGE_SIZE, addr !=3D end); > >=20 > > [...] > >=20 > >> @@ -2409,7 +2540,7 @@ static int internal_get_user_pages_fast(unsigned= long start, int nr_pages, > >> =09unsigned long addr, len, end; > >> =09int nr =3D 0, ret =3D 0; > >> =20 > >> -=09if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM))) > >> +=09if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_P= IN))) > >=20 > > Maybe add a comments to explain, something like: > >=20 > > /* > > * The only flags allowed here are: FOLL_WRITE, FOLL_LONGTERM, FOLL_PIN > > * > > * Note that get_user_pages_fast() imply FOLL_GET flag by default but > > * callers can over-ride this default to pin case by setting FOLL_PIN. > > */ >=20 > Good idea. Here's the draft now: >=20 > /* > * The only flags allowed here are: FOLL_WRITE, FOLL_LONGTERM, FOLL_PIN. > * > * Note that get_user_pages_fast() implies FOLL_GET flag by default, but > * callers can override this default by setting FOLL_PIN instead of > * FOLL_GET. > */ > if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_PIN))) > return -EINVAL; Looks good to me. ... Cheers, J=E9r=F4me From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 582B1CA9ED5 for ; Mon, 4 Nov 2019 23:49:42 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3278F21E6F for ; Mon, 4 Nov 2019 23:49:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3278F21E6F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A7C896E8B7; Mon, 4 Nov 2019 23:49:41 +0000 (UTC) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2D2946E8B7 for ; Mon, 4 Nov 2019 23:49:40 +0000 (UTC) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-397-aDWO2QFVNLO1s54amWQi-Q-1; Mon, 04 Nov 2019 18:49:35 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F0E0D107ACC2; Mon, 4 Nov 2019 23:49:30 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.178]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D177919C4F; Mon, 4 Nov 2019 23:49:21 +0000 (UTC) Date: Mon, 4 Nov 2019 18:49:20 -0500 From: Jerome Glisse To: John Hubbard Subject: Re: [PATCH v2 12/18] mm/gup: track FOLL_PIN pages Message-ID: <20191104234920.GA18515@redhat.com> References: <20191103211813.213227-1-jhubbard@nvidia.com> <20191103211813.213227-13-jhubbard@nvidia.com> <20191104185238.GG5134@redhat.com> <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> MIME-Version: 1.0 In-Reply-To: <7821cf87-75a8-45e2-cf28-f85b62192416@nvidia.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: aDWO2QFVNLO1s54amWQi-Q-1 X-Mimecast-Spam-Score: 0 Content-Disposition: inline X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572911379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ls7Zx2AkPrMXy+LPLj5cuy1SX8JbSG4GMzPShaYe1yw=; b=X/bC82UWeFKjrAylGSFPS1m1bVGp8DkLnZI5Vvc+9RsaPuwgjjTrTZ6U1mb2Pxf8QSnDRu 4VF/0gIsLyOlSs3c0chs4ed48bj/Ynkw6aX7jOiFRw6UVaHEQz5hvHPHcxNUf+Ovx5lOZ4 q4jEQzfF1zoBWTsy3aLFyuxDtA4euDo= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Jonathan Corbet , linux-rdma@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, Alex Williamson , Al Viro , Dan Williams , Mauro Carvalho Chehab , bpf@vger.kernel.org, Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, LKML , linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191104234920.Zz0bGbopqz_Kfy9M77UCuVoaW-24HaZnEbSpfAx71vg@z> T24gTW9uLCBOb3YgMDQsIDIwMTkgYXQgMDI6NDk6MThQTSAtMDgwMCwgSm9obiBIdWJiYXJkIHdy b3RlOgo+IE9uIDExLzQvMTkgMTA6NTIgQU0sIEplcm9tZSBHbGlzc2Ugd3JvdGU6Cj4gPiBPbiBT dW4sIE5vdiAwMywgMjAxOSBhdCAwMToxODowN1BNIC0wODAwLCBKb2huIEh1YmJhcmQgd3JvdGU6 Cj4gPj4gQWRkIHRyYWNraW5nIG9mIHBhZ2VzIHRoYXQgd2VyZSBwaW5uZWQgdmlhIEZPTExfUElO Lgo+ID4+Cj4gPj4gQXMgbWVudGlvbmVkIGluIHRoZSBGT0xMX1BJTiBkb2N1bWVudGF0aW9uLCBj YWxsZXJzIHdobyBlZmZlY3RpdmVseSBzZXQKPiA+PiBGT0xMX1BJTiBhcmUgcmVxdWlyZWQgdG8g dWx0aW1hdGVseSBmcmVlIHN1Y2ggcGFnZXMgdmlhIHB1dF91c2VyX3BhZ2UoKS4KPiA+PiBUaGUg ZWZmZWN0IGlzIHNpbWlsYXIgdG8gRk9MTF9HRVQsIGFuZCBtYXkgYmUgdGhvdWdodCBvZiBhcyAi Rk9MTF9HRVQKPiA+PiBmb3IgRElPIGFuZC9vciBSRE1BIHVzZSIuCj4gPj4KPiA+PiBQYWdlcyB0 aGF0IGhhdmUgYmVlbiBwaW5uZWQgdmlhIEZPTExfUElOIGFyZSBpZGVudGlmaWFibGUgdmlhIGEK PiA+PiBuZXcgZnVuY3Rpb24gY2FsbDoKPiA+Pgo+ID4+ICAgIGJvb2wgcGFnZV9kbWFfcGlubmVk KHN0cnVjdCBwYWdlICpwYWdlKTsKPiA+Pgo+ID4+IFdoYXQgdG8gZG8gaW4gcmVzcG9uc2UgdG8g ZW5jb3VudGVyaW5nIHN1Y2ggYSBwYWdlLCBpcyBsZWZ0IHRvIGxhdGVyCj4gPj4gcGF0Y2hzZXRz LiBUaGVyZSBpcyBkaXNjdXNzaW9uIGFib3V0IHRoaXMgaW4gWzFdLgo+ID4+Cj4gPj4gVGhpcyBh bHNvIGNoYW5nZXMgYSBCVUdfT04oKSwgdG8gYSBXQVJOX09OKCksIGluIGZvbGxvd19wYWdlX21h c2soKS4KPiA+Pgo+ID4+IFRoaXMgYWxzbyBoYXMgYSBjb3VwbGUgb2YgdHJpdmlhbCwgbm9uLWZ1 bmN0aW9uYWwgY2hhbmdlIGZpeGVzIHRvCj4gPj4gdHJ5X2dldF9jb21wb3VuZF9oZWFkKCkuIFRo YXQgZnVuY3Rpb24gZ290IG1vdmVkIHRvIHRoZSB0b3Agb2YgdGhlCj4gPj4gZmlsZS4KPiA+IAo+ ID4gTWF5YmUgc3BsaXQgdGhhdCBhcyBhIHNlcGFyYXRlIHRyaXZpYWwgcGF0Y2guCj4gCj4gCj4g V2lsbCBkby4KPiAKPiAKPiA+IAo+ID4+Cj4gPj4gVGhpcyBpbmNsdWRlcyB0aGUgZm9sbG93aW5n IGZpeCBmcm9tIElyYSBXZWlueToKPiA+Pgo+ID4+IERBWCByZXF1aXJlcyBkZXRlY3Rpb24gb2Yg YSBwYWdlIGNyb3NzaW5nIHRvIGEgcmVmIGNvdW50IG9mIDEuICBGaXggdGhpcwo+ID4+IGZvciBH VVAgcGFnZXMgYnkgaW50cm9kdWNpbmcgcHV0X2Rldm1hcF9tYW5hZ2VkX3VzZXJfcGFnZSgpIHdo aWNoCj4gPj4gYWNjb3VudHMgZm9yIEdVUF9QSU5fQ09VTlRJTkdfQklBUyBub3cgdXNlZCBieSBH VVAuCj4gPiAKPiA+IFBsZWFzZSBkbyB0aGUgcHV0X2Rldm1hcF9tYW5hZ2VkX3BhZ2UoKSBjaGFu Z2VzIGluIGEgc2VwYXJhdGUKPiA+IHBhdGNoLCBpdCB3b3VsZCBiZSBhIGxvdCBlYXNpZXIgdG8g Zm9sbG93LCBhbHNvIG9uIHRoYXQgZnJvbnQKPiA+IHNlZSBjb21tZW50cyBiZWxvdy4KPiAKPiAK PiBPaCEgT0suIEl0IG1ha2VzIHNlbnNlIHdoZW4geW91IHNheSBpdCBvdXQgbG91ZC4gOikKPiAK PiAKPiAuLi4KPiA+PiArc3RhdGljIGlubGluZSBib29sIHB1dF9kZXZtYXBfbWFuYWdlZF9wYWdl KHN0cnVjdCBwYWdlICpwYWdlKQo+ID4+ICt7Cj4gPj4gKwlib29sIGlzX2Rldm1hcCA9IHBhZ2Vf aXNfZGV2bWFwX21hbmFnZWQocGFnZSk7Cj4gPj4gKwo+ID4+ICsJaWYgKGlzX2Rldm1hcCkgewo+ ID4+ICsJCWludCBjb3VudCA9IHBhZ2VfcmVmX2RlY19yZXR1cm4ocGFnZSk7Cj4gPj4gKwo+ID4+ ICsJCV9fcHV0X2Rldm1hcF9tYW5hZ2VkX3BhZ2UocGFnZSwgY291bnQpOwo+ID4+ICsJfQo+ID4+ ICsKPiA+PiArCXJldHVybiBpc19kZXZtYXA7Cj4gPj4gK30KPiA+IAo+ID4gSSB0aGluayB0aGUg X19wdXRfZGV2bWFwX21hbmFnZWRfcGFnZSgpIHNob3VsZCBiZSByZW5hbWUKPiA+IHRvIGZyZWVf ZGV2bWFwX21hbmFnZWRfcGFnZSgpIGFuZCB0aGF0IHRoZSBjb3VudCAhPSAxCj4gPiBjYXNlIG1v dmUgdG8gdGhpcyBpbmxpbmUgZnVuY3Rpb24gaWU6Cj4gPiAKPiA+IHN0YXRpYyBpbmxpbmUgYm9v bCBwdXRfZGV2bWFwX21hbmFnZWRfcGFnZShzdHJ1Y3QgcGFnZSAqcGFnZSkKPiA+IHsKPiA+IAli b29sIGlzX2Rldm1hcCA9IHBhZ2VfaXNfZGV2bWFwX21hbmFnZWQocGFnZSk7Cj4gPiAKPiA+IAlp ZiAoaXNfZGV2bWFwKSB7Cj4gPiAJCWludCBjb3VudCA9IHBhZ2VfcmVmX2RlY19yZXR1cm4ocGFn ZSk7Cj4gPiAKPiA+IAkJLyoKPiA+IAkJICogSWYgcmVmY291bnQgaXMgMSB0aGVuIHBhZ2UgaXMg ZnJlZWQgYW5kIHJlZmNvdW50IGlzIHN0YWJsZSBhcyBub2JvZHkKPiA+IAkJICogaG9sZHMgYSBy ZWZlcmVuY2Ugb24gdGhlIHBhZ2UuCj4gPiAJCSAqLwo+ID4gCQlpZiAoY291bnQgPT0gMSkKPiA+ IAkJCWZyZWVfZGV2bWFwX21hbmFnZWRfcGFnZShwYWdlLCBjb3VudCk7Cj4gPiAJCWVsc2UgaWYg KCFjb3VudCkKPiA+IAkJCV9fcHV0X3BhZ2UocGFnZSk7Cj4gPiAJfQo+ID4gCj4gPiAJcmV0dXJu IGlzX2Rldm1hcDsKPiA+IH0KPiA+IAo+IAo+IFRoYW5rcywgdGhhdCBkb2VzIGxvb2sgY2xlYW5l ciBhbmQgZWFzaWVyIHRvIHJlYWQuCj4gCj4gPiAKPiA+PiArCj4gPj4gICNlbHNlIC8qIENPTkZJ R19ERVZfUEFHRU1BUF9PUFMgKi8KPiA+PiAgc3RhdGljIGlubGluZSBib29sIHB1dF9kZXZtYXBf bWFuYWdlZF9wYWdlKHN0cnVjdCBwYWdlICpwYWdlKQo+ID4+ICB7Cj4gPj4gQEAgLTEwMzgsNiAr MTA1MSw4IEBAIHN0YXRpYyBpbmxpbmUgX19tdXN0X2NoZWNrIGJvb2wgdHJ5X2dldF9wYWdlKHN0 cnVjdCBwYWdlICpwYWdlKQo+ID4+ICAJcmV0dXJuIHRydWU7Cj4gPj4gIH0KPiA+PiAgCj4gPj4g K19fbXVzdF9jaGVjayBib29sIHVzZXJfcGFnZV9yZWZfaW5jKHN0cnVjdCBwYWdlICpwYWdlKTsK PiA+PiArCj4gPiAKPiA+IFdoYXQgYWJvdXQgaGF2aW5nIGl0IGFzIGFuIGlubGluZSBoZXJlIGFz IGl0IGlzIHByZXR0eSBzbWFsbC4KPiAKPiAKPiBZb3UgbWVhbiBtb3ZlIGl0IHRvIGEgc3RhdGlj IGlubGluZSBmdW5jdGlvbiBpbiBtbS5oPyBJdCdzIHdvcnNlIHRoYW4gaXQgCj4gbG9va3MsIHRo b3VnaDogKmV2ZXJ5dGhpbmcqIHRoYXQgaXQgY2FsbHMgaXMgYWxzbyBhIHN0YXRpYyBmdW5jdGlv biwgbG9jYWwKPiB0byBndXAuYy4gU28gSSdkIGhhdmUgdG8gZXhwb3NlIGJvdGggdHJ5X2dldF9j b21wb3VuZF9oZWFkKCkgYW5kCj4gX191cGRhdGVfcHJvY192bXN0YXQoKS4gQW5kIHRoYXQgYWxz byBtZWFucyBjYWxsaW5nIG1vZF9ub2RlX3BhZ2Vfc3RhdGUoKSBmcm9tCj4gbW0uaCwgYW5kIGl0 IGdvZXMgc291dGggcmlnaHQgYWJvdXQgdGhlcmUuIDopCgpPayBmYWlyIGVub3VnaAoKPiAuLi4g IAo+ID4+ICsvKioKPiA+PiArICogcGFnZV9kbWFfcGlubmVkKCkgLSByZXBvcnQgaWYgYSBwYWdl IGlzIHBpbm5lZCBieSBhIGNhbGwgdG8gcGluX3VzZXJfcGFnZXMqKCkKPiA+PiArICogb3IgcGlu X2xvbmd0ZXJtX3BhZ2VzKigpCj4gPj4gKyAqIEBwYWdlOglwb2ludGVyIHRvIHBhZ2UgdG8gYmUg cXVlcmllZC4KPiA+PiArICogQFJldHVybjoJVHJ1ZSwgaWYgaXQgaXMgbGlrZWx5IHRoYXQgdGhl IHBhZ2UgaGFzIGJlZW4gImRtYS1waW5uZWQiLgo+ID4+ICsgKgkJRmFsc2UsIGlmIHRoZSBwYWdl IGlzIGRlZmluaXRlbHkgbm90IGRtYS1waW5uZWQuCj4gPj4gKyAqLwo+ID4gCj4gPiBNYXliZSBh ZGQgYSBzbWFsbCBjb21tZW50IGFib3V0IHdyYXAgYXJvdW5kIDopCj4gCj4gCj4gSSBkb24ndCAq dGhpbmsqIHRoZSBjb3VudCBjYW4gd3JhcCBhcm91bmQsIGR1ZSB0byB0aGUgY2hlY2tzIGluIHVz ZXJfcGFnZV9yZWZfaW5jKCkuCj4gCj4gQnV0IGl0J3MgdHJ1ZSB0aGF0IHRoZSBkb2N1bWVudGF0 aW9uIGlzIGEgbGl0dGxlIGxpZ2h0IGhlcmUuLi5XaGF0IGRpZCB5b3UgaGF2ZSAKPiBpbiBtaW5k PwoKQWJvdXQgZmFsc2UgcG9zaXRpdmUgY2FzZSAoYW5kIGhvdyB1bmxpa2VseSB0aGV5IGFyZSkg YW5kIHRoYXQgd3JhcAphcm91bmQgaXMgcHJvcGVybHkgaGFuZGxlLiBNYXliZSBqdXN0IGEgcG9p bnRlciB0byB0aGUgZG9jdW1lbnRhdGlvbgpzbyB0aGF0IHBlb3BsZSBrbm93IHRoZXkgY2FuIGdv IGxvb2sgdGhlcmUgZm9yIGRldGFpbHMuIEkga25vdyBteQpicmFpbiB0ZW5kIHRvIGZvcmdldCB3 aGVyZSB0byBsb29rIGZvciB0aGluZ3Mgc28gaSBsaWtlIHRvIGJlIGNvbnN0YW50bHkKcmVtaW5k ZWQgaGV5IHRoZSBkb2MgaXMgRG9jdW1lbnRhdGlvbnMvZm9vYmFyIDopCgo+ID4gWy4uLl0KPiA+ IAo+ID4+IEBAIC0xOTMwLDEyICsyMDI4LDIwIEBAIHN0YXRpYyBpbnQgX19ndXBfZGV2aWNlX2h1 Z2UodW5zaWduZWQgbG9uZyBwZm4sIHVuc2lnbmVkIGxvbmcgYWRkciwKPiA+PiAgCj4gPj4gIAkJ cGdtYXAgPSBnZXRfZGV2X3BhZ2VtYXAocGZuLCBwZ21hcCk7Cj4gPj4gIAkJaWYgKHVubGlrZWx5 KCFwZ21hcCkpIHsKPiA+PiAtCQkJdW5kb19kZXZfcGFnZW1hcChuciwgbnJfc3RhcnQsIHBhZ2Vz KTsKPiA+PiArCQkJdW5kb19kZXZfcGFnZW1hcChuciwgbnJfc3RhcnQsIGZsYWdzLCBwYWdlcyk7 Cj4gPj4gIAkJCXJldHVybiAwOwo+ID4+ICAJCX0KPiA+PiAgCQlTZXRQYWdlUmVmZXJlbmNlZChw YWdlKTsKPiA+PiAgCQlwYWdlc1sqbnJdID0gcGFnZTsKPiA+PiAtCQlnZXRfcGFnZShwYWdlKTsK PiA+PiArCj4gPj4gKwkJaWYgKGZsYWdzICYgRk9MTF9QSU4pIHsKPiA+PiArCQkJaWYgKHVubGlr ZWx5KCF1c2VyX3BhZ2VfcmVmX2luYyhwYWdlKSkpIHsKPiA+PiArCQkJCXVuZG9fZGV2X3BhZ2Vt YXAobnIsIG5yX3N0YXJ0LCBmbGFncywgcGFnZXMpOwo+ID4+ICsJCQkJcmV0dXJuIDA7Cj4gPj4g KwkJCX0KPiA+IAo+ID4gTWF5YmUgYWRkIGEgY29tbWVudCBhYm91dCBhIGNhc2UgdGhhdCBzaG91 bGQgbmV2ZXIgaGFwcGVucyBpZQo+ID4gdXNlcl9wYWdlX3JlZl9pbmMoKSBmYWlscyBhZnRlciB0 aGUgc2Vjb25kIGl0ZXJhdGlvbiBvZiB0aGUKPiA+IGxvb3AgYXMgaXQgd291bGQgYmUgYnJva2Vu IGFuZCBhIGJ1ZyB0byBjYWxsIHVuZG9fZGV2X3BhZ2VtYXAoKQo+ID4gYWZ0ZXIgdGhlIGZpcnN0 IGl0ZXJhdGlvbiBvZiB0aGF0IGxvb3AuCj4gPiAKPiA+IEFsc28gaSBiZWxpZXZlIHRoYXQgdGhp cyBzaG91bGQgbmV2ZXIgaGFwcGVucyBhcyBpZiBmaXJzdAo+ID4gaXRlcmF0aW9uIHN1Y2NlZWQg dGhhbiBfX3BhZ2VfY2FjaGVfYWRkX3NwZWN1bGF0aXZlKCkgd2lsbAo+ID4gc3VjY2VlZCBmb3Ig YWxsIHRoZSBpdGVyYXRpb25zLgo+ID4gCj4gPiBOb3RlIHRoYXQgdGhlIHBnbWFwIGNhc2UgYWJv dmUgZm9sbG93cyB0aGF0IHRvbyBpZSB0aGUgY2FsbCB0bwo+ID4gZ2V0X2Rldl9wYWdlbWFwKCkg Y2FuIG9ubHkgZmFpbCBvbiBmaXJzdCBpdGVyYXRpb24gb2YgdGhlIGxvb3AsCj4gPiB3ZWxsIGkg YXNzdW1lIHlvdSBjYW4gbmV2ZXIgaGF2ZSBhIGh1Z2UgZGV2aWNlIHBhZ2UgdGhhdCBzcGFuCj4g PiBkaWZmZXJlbnQgcGdtYXAgaWUgZGlmZmVyZW50IGRldmljZXMgKHdoaWNoIGlzIGEgcmVhc29u YWJsZQo+ID4gYXNzdW1wdGlvbikuIFNvIG1heWJlIHRoaXMgY29kZSBuZWVkcyBmaXhpbmcgaWUg Ogo+ID4gCj4gPiAJCXBnbWFwID0gZ2V0X2Rldl9wYWdlbWFwKHBmbiwgcGdtYXApOwo+ID4gCQlp ZiAodW5saWtlbHkoIXBnbWFwKSkKPiA+IAkJCXJldHVybiAwOwo+ID4gCj4gPiAKPiAKPiBPSywg eWVzIHRoYXQgZG9lcyBtYWtlIHNlbnNlLiBBbmQgSSB0aGluayBhIGNvbW1lbnQgaXMgYWRlcXVh dGUsCj4gbm8gbmVlZCB0byBjaGVjayBmb3IgYnVncyBkdXJpbmcgZXZlcnkgdGFpbCBwYWdlIGl0 ZXJhdGlvbi4gU28gaG93IAo+IGFib3V0IHRoaXMsIGFzIGEgcHJlbGltaW5hcnkgcGF0Y2g6CgpB Y3R1YWx5IGkgdGhvdWdodCBhYm91dCBpdCBhbmQgaSB0aGluayB0aGF0IHRoZXJlIGlzIHBnbWFw CnBlciBzZWN0aW9uIGFuZCB0aHVzIG1heWJlIG9uZSBkZXZpY2UgY2FuIGhhdmUgbXVsdGlwbGUg cGdtYXAKYW5kIHRoYXQgd291bGQgYmUgYW4gaXNzdWUgZm9yIHBhZ2UgYmlnZ2VyIHRoYW4gc2Vj dGlvbiBzaXplCihpZSBiaWdnZXIgdGhhbiAxMjhNQiBpaXJjKS4gSSB3aWxsIGdvIGRvdWJsZSBj aGVjayB0aGF0LCBidXQKbWF5YmUgRGFuIGNhbiBjaGltZSBpbi4KCkluIGFueSBjYXNlIG15IGNv bW1lbnQgYWJvdmUgaXMgY29ycmVjdCBmb3IgdGhlIHBhZ2UgcmVmCmluY3JlbWVudCwgaWYgdGhl IGZpcnN0IG9uZSBzdWNjZWVkIHRoYW4gb3RoZXJzIHdpbGwgdG9vCm9yIG90aGVyd2lzZSBpdCBt ZWFucyBzb21lb25lIGlzIGRvaW5nIHRvbyBtYW55IHB1dF9wYWdlKCkvCnB1dF91c2VyX3BhZ2Uo KSB3aGljaCBpcyBfYmFkXyA6KQoKPiAKPiBkaWZmIC0tZ2l0IGEvbW0vZ3VwLmMgYi9tbS9ndXAu Ywo+IGluZGV4IDhmMjM2YTMzNWFlOS4uYTRhODFlMTI1ODMyIDEwMDY0NAo+IC0tLSBhL21tL2d1 cC5jCj4gKysrIGIvbW0vZ3VwLmMKPiBAQCAtMTg5MiwxNyArMTg5MiwxOCBAQCBzdGF0aWMgaW50 IGd1cF9wdGVfcmFuZ2UocG1kX3QgcG1kLCB1bnNpZ25lZCBsb25nIGFkZHIsIHVuc2lnbmVkIGxv bmcgZW5kLAo+ICBzdGF0aWMgaW50IF9fZ3VwX2RldmljZV9odWdlKHVuc2lnbmVkIGxvbmcgcGZu LCB1bnNpZ25lZCBsb25nIGFkZHIsCj4gICAgICAgICAgICAgICAgIHVuc2lnbmVkIGxvbmcgZW5k LCBzdHJ1Y3QgcGFnZSAqKnBhZ2VzLCBpbnQgKm5yKQo+ICB7Cj4gLSAgICAgICBpbnQgbnJfc3Rh cnQgPSAqbnI7Cj4gLSAgICAgICBzdHJ1Y3QgZGV2X3BhZ2VtYXAgKnBnbWFwID0gTlVMTDsKPiAr ICAgICAgIC8qCj4gKyAgICAgICAgKiBIdWdlIHBhZ2VzIHNob3VsZCBuZXZlciBjcm9zcyBkZXZf cGFnZW1hcCBib3VuZGFyaWVzLiBUaGVyZWZvcmUsIHVzZQo+ICsgICAgICAgICogdGhpcyBzYW1l IHBnbWFwIGZvciB0aGUgZW50aXJlIGh1Z2UgcGFnZS4KPiArICAgICAgICAqLwo+ICsgICAgICAg c3RydWN0IGRldl9wYWdlbWFwICpwZ21hcCA9IGdldF9kZXZfcGFnZW1hcChwZm4sIE5VTEwpOwo+ ICsKPiArICAgICAgIGlmICh1bmxpa2VseSghcGdtYXApKQo+ICsgICAgICAgICAgICAgICByZXR1 cm4gMDsKPiAgCj4gICAgICAgICBkbyB7Cj4gICAgICAgICAgICAgICAgIHN0cnVjdCBwYWdlICpw YWdlID0gcGZuX3RvX3BhZ2UocGZuKTsKPiAgCj4gLSAgICAgICAgICAgICAgIHBnbWFwID0gZ2V0 X2Rldl9wYWdlbWFwKHBmbiwgcGdtYXApOwo+IC0gICAgICAgICAgICAgICBpZiAodW5saWtlbHko IXBnbWFwKSkgewo+IC0gICAgICAgICAgICAgICAgICAgICAgIHVuZG9fZGV2X3BhZ2VtYXAobnIs IG5yX3N0YXJ0LCBwYWdlcyk7Cj4gLSAgICAgICAgICAgICAgICAgICAgICAgcmV0dXJuIDA7Cj4g LSAgICAgICAgICAgICAgIH0KPiAgICAgICAgICAgICAgICAgU2V0UGFnZVJlZmVyZW5jZWQocGFn ZSk7Cj4gICAgICAgICAgICAgICAgIHBhZ2VzWypucl0gPSBwYWdlOwo+ICAgICAgICAgICAgICAg ICBnZXRfcGFnZShwYWdlKTsKPiAKPiAKPiAKPiAKPiA+PiArCQl9IGVsc2UKPiA+PiArCQkJZ2V0 X3BhZ2UocGFnZSk7Cj4gPj4gKwo+ID4+ICAJCSgqbnIpKys7Cj4gPj4gIAkJcGZuKys7Cj4gPj4g IAl9IHdoaWxlIChhZGRyICs9IFBBR0VfU0laRSwgYWRkciAhPSBlbmQpOwo+ID4gCj4gPiBbLi4u XQo+ID4gCj4gPj4gQEAgLTI0MDksNyArMjU0MCw3IEBAIHN0YXRpYyBpbnQgaW50ZXJuYWxfZ2V0 X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsCj4gPj4g IAl1bnNpZ25lZCBsb25nIGFkZHIsIGxlbiwgZW5kOwo+ID4+ICAJaW50IG5yID0gMCwgcmV0ID0g MDsKPiA+PiAgCj4gPj4gLQlpZiAoV0FSTl9PTl9PTkNFKGd1cF9mbGFncyAmIH4oRk9MTF9XUklU RSB8IEZPTExfTE9OR1RFUk0pKSkKPiA+PiArCWlmIChXQVJOX09OX09OQ0UoZ3VwX2ZsYWdzICYg fihGT0xMX1dSSVRFIHwgRk9MTF9MT05HVEVSTSB8IEZPTExfUElOKSkpCj4gPiAKPiA+IE1heWJl IGFkZCBhIGNvbW1lbnRzIHRvIGV4cGxhaW4sIHNvbWV0aGluZyBsaWtlOgo+ID4gCj4gPiAvKgo+ ID4gICogVGhlIG9ubHkgZmxhZ3MgYWxsb3dlZCBoZXJlIGFyZTogRk9MTF9XUklURSwgRk9MTF9M T05HVEVSTSwgRk9MTF9QSU4KPiA+ICAqCj4gPiAgKiBOb3RlIHRoYXQgZ2V0X3VzZXJfcGFnZXNf ZmFzdCgpIGltcGx5IEZPTExfR0VUIGZsYWcgYnkgZGVmYXVsdCBidXQKPiA+ICAqIGNhbGxlcnMg Y2FuIG92ZXItcmlkZSB0aGlzIGRlZmF1bHQgdG8gcGluIGNhc2UgYnkgc2V0dGluZyBGT0xMX1BJ Ti4KPiA+ICAqLwo+IAo+IEdvb2QgaWRlYS4gSGVyZSdzIHRoZSBkcmFmdCBub3c6Cj4gCj4gLyoK PiAgKiBUaGUgb25seSBmbGFncyBhbGxvd2VkIGhlcmUgYXJlOiBGT0xMX1dSSVRFLCBGT0xMX0xP TkdURVJNLCBGT0xMX1BJTi4KPiAgKgo+ICAqIE5vdGUgdGhhdCBnZXRfdXNlcl9wYWdlc19mYXN0 KCkgaW1wbGllcyBGT0xMX0dFVCBmbGFnIGJ5IGRlZmF1bHQsIGJ1dAo+ICAqIGNhbGxlcnMgY2Fu IG92ZXJyaWRlIHRoaXMgZGVmYXVsdCBieSBzZXR0aW5nIEZPTExfUElOIGluc3RlYWQgb2YKPiAg KiBGT0xMX0dFVC4KPiAgKi8KPiBpZiAoV0FSTl9PTl9PTkNFKGd1cF9mbGFncyAmIH4oRk9MTF9X UklURSB8IEZPTExfTE9OR1RFUk0gfCBGT0xMX1BJTikpKQo+ICAgICAgICAgcmV0dXJuIC1FSU5W QUw7CgpMb29rcyBnb29kIHRvIG1lLgoKLi4uCgpDaGVlcnMsCkrDqXLDtG1lCgpfX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpkcmktZGV2ZWwgbWFpbGluZyBs aXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVz a3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9kcmktZGV2ZWw=