From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B661C4332D for ; Fri, 20 Mar 2020 08:10:36 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 472B120739 for ; Fri, 20 Mar 2020 08:10:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="niE0QFwA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 472B120739 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9D3476EAE1; Fri, 20 Mar 2020 08:10:34 +0000 (UTC) Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by gabe.freedesktop.org (Postfix) with ESMTPS id CB5A26EAAB for ; Fri, 20 Mar 2020 00:03:48 +0000 (UTC) Received: by mail-qk1-x741.google.com with SMTP id f3so5250392qkh.1 for ; Thu, 19 Mar 2020 17:03:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=1pJxpq8bFhAdtFvjymRkDlT+zzgC3t93bLWt4+LdA0o=; b=niE0QFwAScsJNksiL1RD0EoZyWLTkmIFeNZTwtNcFLkMjDayi0bxMD56+ydBpxjFer ccfFIEFWOAqX7S6DF8dy2B90sjOJd11kjQNv+zzxmWUfQ+NuUvGWeapza5y7m3aL1wJG j3hvcJFjjm2RHXKkVcIUMT/Unai34kkTifnPtPSjJsdNLmUD0nhajqJhX+Agn9Kz3t8d MseyS+KCBRWIY/Clnpsqfu2H+lIZb7sticYp/HJnFGSWY7gUq7c/Zqf45npz2Z8DGQ78 mXMSeVQdS57T+ekPAXfTCqZyqZmCir8LMmtQu7NDGFXY1YDgmRsH7Uq3cAhojnHv5w1y bUgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=1pJxpq8bFhAdtFvjymRkDlT+zzgC3t93bLWt4+LdA0o=; b=hSGsN2fgYYabMoNTucrTiz/8lVe2mgyJwf2C/NiOuETimK1cJDKz0zyPpK4LbMlSqN 2RA2861sADk8QCpGFV9DkDa43L/iYZzH89/ZTRHiBxnPAYXQpqNtz158EIgAP5rzyy5A D1yYvD3eNfKsZz8VqjBu/xgaWIz9tqr5bwOBfz+oePRVVMUbZm65bSyOjutWL0H2+oCx oZFIQBqvw2TC4JLdEZTnPCLxXnDtunALQwdDl/dCSqYXEuJzO+a7wiUa8qBqYYi4gtyr mTNc4kdjs7Cb5HW6dsiFnqUIIr8OUNX2jH4WSftF0RlyynbwAN3U42K6pm9O8urObBR6 kSmg== X-Gm-Message-State: ANhLgQ0dojVHBmyo24I8DVN6fiz00S90BfHlK9IuqVRmkTT22BRpDSIh PzCdw1Pd4i4XSH67ZCRD1LSyTw== X-Google-Smtp-Source: ADFU+vvmojx05al3/dBJDAgC1QulIXZbUAIDZwSQ+speDrKVTJZR9ajSBjpmE5SDOXsMKVejlONOTg== X-Received: by 2002:a37:b44:: with SMTP id 65mr5064085qkl.201.1584662627565; Thu, 19 Mar 2020 17:03:47 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id h11sm139725qtr.38.2020.03.19.17.03.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 17:03:47 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jF58b-00029b-D5; Thu, 19 Mar 2020 21:03:45 -0300 Date: Thu, 19 Mar 2020 21:03:45 -0300 From: Jason Gunthorpe To: Ralph Campbell Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-ID: <20200320000345.GO20941@ziepe.ca> References: <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> <20200319181716.GK20941@ziepe.ca> <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Mailman-Approved-At: Fri, 20 Mar 2020 08:10:27 +0000 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, kvm-ppc@vger.kernel.org, Christoph Hellwig , linux-mm@kvack.org, Jerome Glisse , Ben Skeggs , linux-kselftest@vger.kernel.org, Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" T24gVGh1LCBNYXIgMTksIDIwMjAgYXQgMDM6NTY6NTBQTSAtMDcwMCwgUmFscGggQ2FtcGJlbGwg d3JvdGU6Cj4gQWRkaW5nIGxpbnV4LWtzZWxmdGVzdEB2Z2VyLmtlcm5lbC5vcmcgZm9yIHRoZSB0 ZXN0IGNvbmZpZyBxdWVzdGlvbi4KPiAKPiBPbiAzLzE5LzIwIDExOjE3IEFNLCBKYXNvbiBHdW50 aG9ycGUgd3JvdGU6Cj4gPiBPbiBUdWUsIE1hciAxNywgMjAyMCBhdCAwNDoxNDozMVBNIC0wNzAw LCBSYWxwaCBDYW1wYmVsbCB3cm90ZToKPiA+ID4gCj4gPiA+IE9uIDMvMTcvMjAgNTo1OSBBTSwg Q2hyaXN0b3BoIEhlbGx3aWcgd3JvdGU6Cj4gPiA+ID4gT24gVHVlLCBNYXIgMTcsIDIwMjAgYXQg MDk6NDc6NTVBTSAtMDMwMCwgSmFzb24gR3VudGhvcnBlIHdyb3RlOgo+ID4gPiA+ID4gSSd2ZSBi ZWVuIHVzaW5nIHY3IG9mIFJhbHBoJ3MgdGVzdGVyIGFuZCBpdCBpcyB3b3JraW5nIHdlbGwgLSBp dCBoYXMKPiA+ID4gPiA+IERFVklDRV9QUklWQVRFIHN1cHBvcnQgc28gSSB0aGluayBpdCBjYW4g dGVzdCB0aGlzIGZsb3cgdG9vLiBSYWxwaCBhcmUKPiA+ID4gPiA+IHlvdSBhYmxlPwo+ID4gPiA+ ID4gCj4gPiA+ID4gPiBUaGlzIGh1bmsgc2VlbXMgdHJpdmlhbCBlbm91Z2ggdG8gbWUsIGNhbiB3 ZSBpbmNsdWRlIGl0IG5vdz8KPiA+ID4gPiAKPiA+ID4gPiBJIGNhbiBzZW5kIGEgc2VwYXJhdGUg cGF0Y2ggZm9yIGl0IG9uY2UgdGhlIHRlc3RlciBjb3ZlcnMgaXQuICBJIGRvbid0Cj4gPiA+ID4g d2FudCB0byBhZGQgaXQgdG8gdGhlIG9yaWdpbmFsIHBhdGNoIGFzIGl0IGlzIGEgc2lnbmlmaWNh bnQgYmVoYXZpb3IKPiA+ID4gPiBjaGFuZ2UgY29tcGFyZWQgdG8gdGhlIGV4aXN0aW5nIGNvZGUu Cj4gPiA+ID4gCj4gPiA+IAo+ID4gPiBBdHRhY2hlZCBpcyBhbiB1cGRhdGVkIHZlcnNpb24gb2Yg bXkgSE1NIHRlc3RzIGJhc2VkIG9uIGxpbnV4LTUuNi4wLXJjNi4KPiA+ID4gSSByYW4gdGhpcyBP SyB3aXRoIEphc29uJ3MgOCsxIEhNTSBwYXRjaGVzLCBDaHJpc3RvcGgncyAxLTUgbWlzYyBITU0g Y2xlYW4gdXBzLAo+ID4gPiBhbmQgQ2hyaXN0b3BoJ3MgMS00IGRldmljZSBwcml2YXRlIHBhZ2Ug Y2hhbmdlcyBhcHBsaWVkLgo+ID4gCj4gPiBJJ2QgbGlrZSB0byBnZXQgdGhpcyB0byBtZXJnYWJs ZSwgaXQgbG9va3MgcHJldHR5IGdvb2Qgbm93LCBidXQgSSBoYXZlCj4gPiBubyBpZGVhIGFib3V0 IHNlbGZ0ZXN0cyAtIGFuZCBJJ20gc3RydWdnbGluZyB0byBldmVuIGNvbXBpbGUgdGhlIHRvb2xz Cj4gPiBkaXIKPiA+IAo+ID4gPiBkaWZmIC0tZ2l0IGEvbGliL0tjb25maWcuZGVidWcgYi9saWIv S2NvbmZpZy5kZWJ1Zwo+ID4gPiBpbmRleCA2OWRlZjRhOWRmMDAuLjRkMjJjZTc4NzlhNyAxMDA2 NDQKPiA+ID4gKysrIGIvbGliL0tjb25maWcuZGVidWcKPiA+ID4gQEAgLTIxNjIsNiArMjE2Miwx OCBAQCBjb25maWcgVEVTVF9NRU1JTklUCj4gPiA+ICAgCSAgSWYgdW5zdXJlLCBzYXkgTi4KPiA+ ID4gK2NvbmZpZyBURVNUX0hNTQo+ID4gPiArCXRyaXN0YXRlICJUZXN0IEhNTSAoSGV0ZXJvZ2Vu ZW91cyBNZW1vcnkgTWFuYWdlbWVudCkiCj4gPiA+ICsJZGVwZW5kcyBvbiBERVZJQ0VfUFJJVkFU RQo+ID4gPiArCXNlbGVjdCBITU1fTUlSUk9SCj4gPiA+ICsgICAgICAgIHNlbGVjdCBNTVVfTk9U SUZJRVIKPiA+IAo+ID4gZXh0cmEgc3BhY2VzCj4gCj4gV2lsbCBmaXggaW4gdjguCj4gCj4gPiBJ biBnZW5lcmFsIEkgd29uZGVyIGlmIGl0IGV2ZW4gbWFrZXMgc2Vuc2UgdGhhdCBERVZJQ0VfUFJJ VkFURSBpcyB1c2VyCj4gPiBzZWxlY3RhYmxlPwo+IAo+IFNob3VsZCB0ZXN0cyBlbmFibGUgdGhl IGZlYXR1cmUgb3IgdGhlIGZlYXR1cmUgZW5hYmxlIHRoZSB0ZXN0Pwo+IElNSE8sIGlmIHRoZSBm ZWF0dXJlIGlzIGJlaW5nIGNvbXBpbGVkIGludG8gdGhlIGtlcm5lbCwgdGhhdCBzaG91bGQKPiBl bmFibGUgdGhlIG1lbnUgaXRlbSBmb3IgdGhlIHRlc3QuIElmIHRoZSBmZWF0dXJlIGlzbid0IHNl bGVjdGVkLAo+IG5vIG5lZWQgdG8gdGVzdCBpdCA6LSkKCkkgbWVudCBpZiBERVZJQ0VfUFJJVkFU RSBzaG91bGQgYmUgYSB1c2VyIHNlbGVjdGFibGUgb3B0aW9uIGF0IGFsbCwgb3IKc2hvdWxkIGl0 IGJlIHR1cm5lZCBvbiB3aGVuIGEgZHJpdmVyIGxpa2Ugbm91dmVhdSBpcyBzZWxlY3RlZC4KCklz IHRoZXJlIHNvbWUgZG93bnNpZGUgdG8gZW5hYmxpbmcgREVWSUNFX1BSSVZBVEU/Cgo+ID4gVGhl IG5vdGlmaWVyIGhvbGRzIGEgbW1ncmFiLCBubyBuZWVkIGZvciBhbm90aGVyIG9uZQo+IAo+IE9L LiBJJ2xsIHJlcGxhY2UgZG1pcnJvci0+bW0gd2l0aCBkbWlycm9yLT5ub3RpZmllci5tbS4KClJp Z2h0IHRoYXQgaXMgZ29vZCB0b28KCj4gPiA+ICsJZmlscC0+cHJpdmF0ZV9kYXRhID0gZG1pcnJv cjsKPiA+IAo+ID4gTm90IHN1cmUgd2hhdCB0aGlzIGNvbW1lbnQgbWVhbnMKPiAKPiBJJ2xsIGNo YW5nZSB0aGUgY29tbWVudCB0bzoKPiAJICAvKgo+ICAgICAgICAgICogVGhlIGZpcnN0IG9wZW4g b2YgdGhlIGRldmljZSBjaGFyYWN0ZXIgZmlsZSByZWdpc3RlcnMgdGhlIGFkZHJlc3MKPiAgICAg ICAgICAqIHNwYWNlIG9mIHRoZSBwcm9jZXNzIGRvaW5nIHRoZSBvcGVuKCkgc3lzdGVtIGNhbGwg d2l0aCB0aGUgZGV2aWNlLgo+ICAgICAgICAgICogU3Vic2VxdWVudCBmaWxlIG9wZW5zIGJ5IG90 aGVyIHByb2Nlc3NlcyB3aWxsIGhhdmUgYWNjZXNzIHRvIHRoZQo+ICAgICAgICAgICogZmlyc3Qg cHJvY2VzcycgYWRkcmVzcyBzcGFjZS4KPiAgICAgICAgICAqLwoKSG93IGRvZXMgdGhpcyBoYXBw ZW4/IFRoZSBmdW5jdGlvbiBsb29rcyBsaWtlIGl0IGFsd2F5cyBkb2VzIHRoZSBzYW1lIHRoaW5n Cgo+ID4gPiArc3RhdGljIGJvb2wgZG1pcnJvcl9pbnRlcnZhbF9pbnZhbGlkYXRlKHN0cnVjdCBt bXVfaW50ZXJ2YWxfbm90aWZpZXIgKm1uaSwKPiA+ID4gKwkJCQljb25zdCBzdHJ1Y3QgbW11X25v dGlmaWVyX3JhbmdlICpyYW5nZSwKPiA+ID4gKwkJCQl1bnNpZ25lZCBsb25nIGN1cl9zZXEpCj4g PiA+ICt7Cj4gPiA+ICsJc3RydWN0IGRtaXJyb3IgKmRtaXJyb3IgPSBjb250YWluZXJfb2YobW5p LCBzdHJ1Y3QgZG1pcnJvciwgbm90aWZpZXIpOwo+ID4gPiArCXN0cnVjdCBtbV9zdHJ1Y3QgKm1t ID0gZG1pcnJvci0+bW07Cj4gPiA+ICsKPiA+ID4gKwkvKgo+ID4gPiArCSAqIElmIHRoZSBwcm9j ZXNzIGRvZXNuJ3QgZXhpc3QsIHdlIGRvbid0IG5lZWQgdG8gaW52YWxpZGF0ZSB0aGUKPiA+ID4g KwkgKiBkZXZpY2UgcGFnZSB0YWJsZSBzaW5jZSB0aGUgYWRkcmVzcyBzcGFjZSB3aWxsIGJlIHRv cm4gZG93bi4KPiA+ID4gKwkgKi8KPiA+ID4gKwlpZiAoIW1tZ2V0X25vdF96ZXJvKG1tKSkKPiA+ ID4gKwkJcmV0dXJuIHRydWU7Cj4gPiAKPiA+IFdoeT8gRG9uJ3QgdGhlIG5vdGlmaWVycyBwcm92 aWRlIGZvciB0aGlzIGFscmVhZHkuCj4gPiAKPiA+IG1tZ2V0X25vdF96ZXJvKCkgaXMgcmVxdWly ZWQgYmVmb3JlIGNhbGxpbmcgaG1tX3JhbmdlX2ZhdWx0KCkgdGhvdWdoCgpPaC4uLiBUaGlzIGlz IHRoZSBpbnZhbGlkYXRlX2FsbCBwYXRoIGR1cmluZyBpbnZhbGlkYXRpb24KCklNSE8geW91IHNo b3VsZCB0ZXN0IHRoZSBpbnZhbGlkYXRpb24gcmVhc29uIGluIHRoZSByYW5nZSB0byBleGNsdWRl CnRoaXMuCgpCdXQgeGFfZXJhc2UgbG9va3MgdG90YWxseSBzYWZlIHNvIHRoZXJlIHNob3VsZCBi ZSBubyByZWFzb24gdG8gZG8KdGhhdC4KCj4gVGhpcyBpcyBhIHdvcmthcm91bmQgZm9yIGEgcHJv YmxlbSBJIGRvbid0IHF1aXRlIHVuZGVyc3RhbmQuCj4gSWYgeW91IGNoYW5nZSB0b29scy90ZXN0 aW5nL3NlbGZ0ZXN0cy92bS9obW0tdGVzdHMuYyBsaW5lIDg2OCB0bwo+IAlBU1NFUlRfRVEocmV0 LCAtMSk7Cj4gVGhlbiB0aGUgdGVzdCB3aWxsIGFib3J0LCBjb3JlIGR1bXAsIGFuZCBjYXVzZSB0 d28gcHJvYmxlbXMsCj4gMSkgdGhlIG1pZ3JhdGVkIHBhZ2Ugd2lsbCBiZSBmYXVsdGVkIGJhY2sg dG8gc3lzdGVtIG1lbW9yeSBpbiBvcmRlciB0byB3cml0ZQo+ICAgIGl0IHRvIHRoZSBjb3JlIGR1 bXAuIFRoaXMgdHJpZ2dlcnMgbG9ja2RlcF9hc3NlcnRfaGVsZCgmd2Fsay5tbS0+bW1hcF9zZW0p Cj4gICAgaW4gd2Fsa19wYWdlX3JhbmdlKCkuCgpIYXMgdGhlIG1pZ3JhdGlvbiBzdHVmZiBiZWNv bWUgZW50YW5nbGVkIHdpdGggdGhlIHhhcnJheT8KCj4gWyAgMTM3Ljk4MDcxOF0gQ29kZTogODAg MmYgMWEgODMgYzYgMDUgZTkgOGQgN2IgMDEgMDEgZTggM2UgYjEgYjEgZmUgZTkgMDUgZmYgZmYg ZmYgNjYgNjYgMmUgMGYgMWYgODQgMDAgMDAgMDAgMDAgMDAgMGYgMWYgMDAgNDEgNTYgNDEgNTUg NDEgNTQgNTUgPDQ4PiA4OSBmZCA1MyA0YyA4ZCA2ZCAxMCBlOCAzYyBmYyBmZiBmZiA0OSA4OSBj NCA0YyA4OSBlMCA4MyBlMCAwMwo+IFsgIDEzNy45OTk0NjFdIFJTUDogMDAxODpmZmZmYzkwMDAx NWU3N2M4IEVGTEFHUzogMDAwMDAyNDYgT1JJR19SQVg6IGZmZmZmZmZmZmZmZmZmMTMKPiBbICAx MzguMDA3MDI4XSBSQVg6IGZmZmY4ODg2ZTUwOGM0MDggUkJYOiAwMDAwMDAwMDAwMDAwMDAwIFJD WDogZmZmZmZmZmY4MjYyNmM4OQo+IFsgIDEzOC4wMTQxNTldIFJEWDogZGZmZmZjMDAwMDAwMDAw MCBSU0k6IDAwMDAwMDAwMDAwMDAwMDAgUkRJOiBmZmZmYzkwMDAxNWU3OGEwCj4gWyAgMTM4LjAy MTI5M10gUkJQOiBmZmZmYzkwMDAxNWU3OGEwIFIwODogZmZmZmZmZmY4MTE0NjFjNCBSMDk6IGZm ZmZmNTIwMDAyYmNmMTcKPiBbICAxMzguMDI4NDI2XSBSMTA6IGZmZmZmNTIwMDAyYmNmMTYgUjEx OiAwMDAwMDAwMDAwMDAwMDAzIFIxMjogMDAwMDAwMDAwMjYwNmQxMAo+IFsgIDEzOC4wMzU1NTdd IFIxMzogZmZmZjg4ODZlNTA4YzQ0OCBSMTQ6IDAwMDAwMDAwMDAwMDAwMzEgUjE1OiBmZmZmZmZm ZmEwNjU0NmEwCj4gWyAgMTM4LjA0MjcwMV0gID8gZG9fcmF3X3NwaW5fbG9jaysweDEwNC8weDFk MAo+IFsgIDEzOC4wNDY4ODhdICA/IHhhc19zdG9yZSsweDE5LzB4YTYwCj4gWyAgMTM4LjA1MDM5 MF0gIHhhc19zdG9yZSsweDViMy8weGE2MAo+IFsgIDEzOC4wNTM4MDZdICA/IHJlZ2lzdGVyX2xv Y2tfY2xhc3MrMHg4NjAvMHg4NjAKPiBbICAxMzguMDU4MjY3XSAgX194YV9lcmFzZSsweDk2LzB4 MTEwCj4gWyAgMTM4LjA2MTY3M10gID8geGFzX3N0b3JlKzB4YTYwLzB4YTYwCj4gWyAgMTM4LjA2 NTI2N10gIHhhX2VyYXNlKzB4MTkvMHgzMAoKb2gsIGl0IGlzIGRvaW5nIHRoaXM6CgpzdGF0aWMg dm9pZCBtbl9pdHJlZV9yZWxlYXNlKHN0cnVjdCBtbXVfbm90aWZpZXJfc3Vic2NyaXB0aW9ucyAq c3Vic2NyaXB0aW9ucywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgbW1fc3Ry dWN0ICptbSkKewogICAgICAgIHN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgcmFuZ2UgPSB7CiAg ICAgICAgICAgICAgICAuZmxhZ3MgPSBNTVVfTk9USUZJRVJfUkFOR0VfQkxPQ0tBQkxFLAogICAg ICAgICAgICAgICAgLmV2ZW50ID0gTU1VX05PVElGWV9SRUxFQVNFLAogICAgICAgICAgICAgICAg Lm1tID0gbW0sCiAgICAgICAgICAgICAgICAuc3RhcnQgPSAwLAogICAgICAgICAgICAgICAgLmVu ZCA9IFVMT05HX01BWCwKICAgICAgICB9OwoKaWUgaXQgaXMgc2l0dGluZyBkb2luZyBhIGh1Z2Ug bnVtYmVyIG9mIHhhX2VyYXNlcywgSSBzdXBwb3NlLiBQcm9iYWJseQppbiBub3JtYWwgZXhpdCB0 aGUgbm90aWZpZXIgaXMgcmVtb3ZlZCBiZWZvcmUgdGhlIG1tIGlzIGRlc3Ryb3llZC4KClRoZSB4 YV9lcmFzZSBuZWVkcyB0byBiZSBhIGJpdCBzbWFydGVyIHRvIGp1bXAgb3ZlciBnYXBzIGluIHRo ZSB0cmVlCnBlcmhhcHMgc29tZQoKeGFfZm9yX2VhY2goKQogICB4YV9lcmFzZSgpCgpwYXR0ZXJu PwoKPiA+IEFsc28gSSBnZXQgdGhpczoKPiA+IAo+ID4gbGliL3Rlc3RfaG1tLmM6IEluIGZ1bmN0 aW9uIOKAmGRtaXJyb3JfZGV2bWVtX2ZhdWx0X2FsbG9jX2FuZF9jb3B54oCZOgo+ID4gbGliL3Rl c3RfaG1tLmM6MTA0MToyNTogd2FybmluZzogdW51c2VkIHZhcmlhYmxlIOKAmHZtYeKAmSBbLVd1 bnVzZWQtdmFyaWFibGVdCj4gPiAgIDEwNDEgfCAgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEg PSBhcmdzLT52bWE7Cj4gPiAKPiA+IEJ1dCB0aGlzIGlzIGEga2VybmVsIGJ1ZywgZHVlIHRvIGFs bG9jX3BhZ2Vfdm1hIGJlaW5nIGEgI2RlZmluZSBub3QgYQo+ID4gc3RhdGljIGlubGluZSBhbmQg bWUgaGF2aW5nIENPTkZJR19OVU1BIG9mZiBpbiB0aGlzIC5jb25maWcKPiAKPiBGaXhlZC4KCmlu IGdmcC5oPwoKSmFzb24KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KYW1kLWdmeCBtYWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcK aHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Date: Fri, 20 Mar 2020 00:03:45 +0000 Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-Id: <20200320000345.GO20941@ziepe.ca> List-Id: References: <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> <20200319181716.GK20941@ziepe.ca> <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> In-Reply-To: <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit To: Ralph Campbell Cc: Christoph Hellwig , Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= , Ben Skeggs , Jerome Glisse , kvm-ppc@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org On Thu, Mar 19, 2020 at 03:56:50PM -0700, Ralph Campbell wrote: > Adding linux-kselftest@vger.kernel.org for the test config question. > > On 3/19/20 11:17 AM, Jason Gunthorpe wrote: > > On Tue, Mar 17, 2020 at 04:14:31PM -0700, Ralph Campbell wrote: > > > > > > On 3/17/20 5:59 AM, Christoph Hellwig wrote: > > > > On Tue, Mar 17, 2020 at 09:47:55AM -0300, Jason Gunthorpe wrote: > > > > > I've been using v7 of Ralph's tester and it is working well - it has > > > > > DEVICE_PRIVATE support so I think it can test this flow too. Ralph are > > > > > you able? > > > > > > > > > > This hunk seems trivial enough to me, can we include it now? > > > > > > > > I can send a separate patch for it once the tester covers it. I don't > > > > want to add it to the original patch as it is a significant behavior > > > > change compared to the existing code. > > > > > > > > > > Attached is an updated version of my HMM tests based on linux-5.6.0-rc6. > > > I ran this OK with Jason's 8+1 HMM patches, Christoph's 1-5 misc HMM clean ups, > > > and Christoph's 1-4 device private page changes applied. > > > > I'd like to get this to mergable, it looks pretty good now, but I have > > no idea about selftests - and I'm struggling to even compile the tools > > dir > > > > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > > > index 69def4a9df00..4d22ce7879a7 100644 > > > +++ b/lib/Kconfig.debug > > > @@ -2162,6 +2162,18 @@ config TEST_MEMINIT > > > If unsure, say N. > > > +config TEST_HMM > > > + tristate "Test HMM (Heterogeneous Memory Management)" > > > + depends on DEVICE_PRIVATE > > > + select HMM_MIRROR > > > + select MMU_NOTIFIER > > > > extra spaces > > Will fix in v8. > > > In general I wonder if it even makes sense that DEVICE_PRIVATE is user > > selectable? > > Should tests enable the feature or the feature enable the test? > IMHO, if the feature is being compiled into the kernel, that should > enable the menu item for the test. If the feature isn't selected, > no need to test it :-) I ment if DEVICE_PRIVATE should be a user selectable option at all, or should it be turned on when a driver like nouveau is selected. Is there some downside to enabling DEVICE_PRIVATE? > > The notifier holds a mmgrab, no need for another one > > OK. I'll replace dmirror->mm with dmirror->notifier.mm. Right that is good too > > > + filp->private_data = dmirror; > > > > Not sure what this comment means > > I'll change the comment to: > /* > * The first open of the device character file registers the address > * space of the process doing the open() system call with the device. > * Subsequent file opens by other processes will have access to the > * first process' address space. > */ How does this happen? The function looks like it always does the same thing > > > +static bool dmirror_interval_invalidate(struct mmu_interval_notifier *mni, > > > + const struct mmu_notifier_range *range, > > > + unsigned long cur_seq) > > > +{ > > > + struct dmirror *dmirror = container_of(mni, struct dmirror, notifier); > > > + struct mm_struct *mm = dmirror->mm; > > > + > > > + /* > > > + * If the process doesn't exist, we don't need to invalidate the > > > + * device page table since the address space will be torn down. > > > + */ > > > + if (!mmget_not_zero(mm)) > > > + return true; > > > > Why? Don't the notifiers provide for this already. > > > > mmget_not_zero() is required before calling hmm_range_fault() though Oh... This is the invalidate_all path during invalidation IMHO you should test the invalidation reason in the range to exclude this. But xa_erase looks totally safe so there should be no reason to do that. > This is a workaround for a problem I don't quite understand. > If you change tools/testing/selftests/vm/hmm-tests.c line 868 to > ASSERT_EQ(ret, -1); > Then the test will abort, core dump, and cause two problems, > 1) the migrated page will be faulted back to system memory in order to write > it to the core dump. This triggers lockdep_assert_held(&walk.mm->mmap_sem) > in walk_page_range(). Has the migration stuff become entangled with the xarray? > [ 137.980718] Code: 80 2f 1a 83 c6 05 e9 8d 7b 01 01 e8 3e b1 b1 fe e9 05 ff ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 41 56 41 55 41 54 55 <48> 89 fd 53 4c 8d 6d 10 e8 3c fc ff ff 49 89 c4 4c 89 e0 83 e0 03 > [ 137.999461] RSP: 0018:ffffc900015e77c8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 > [ 138.007028] RAX: ffff8886e508c408 RBX: 0000000000000000 RCX: ffffffff82626c89 > [ 138.014159] RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffffc900015e78a0 > [ 138.021293] RBP: ffffc900015e78a0 R08: ffffffff811461c4 R09: fffff520002bcf17 > [ 138.028426] R10: fffff520002bcf16 R11: 0000000000000003 R12: 0000000002606d10 > [ 138.035557] R13: ffff8886e508c448 R14: 0000000000000031 R15: ffffffffa06546a0 > [ 138.042701] ? do_raw_spin_lock+0x104/0x1d0 > [ 138.046888] ? xas_store+0x19/0xa60 > [ 138.050390] xas_store+0x5b3/0xa60 > [ 138.053806] ? register_lock_class+0x860/0x860 > [ 138.058267] __xa_erase+0x96/0x110 > [ 138.061673] ? xas_store+0xa60/0xa60 > [ 138.065267] xa_erase+0x19/0x30 oh, it is doing this: static void mn_itree_release(struct mmu_notifier_subscriptions *subscriptions, struct mm_struct *mm) { struct mmu_notifier_range range = { .flags = MMU_NOTIFIER_RANGE_BLOCKABLE, .event = MMU_NOTIFY_RELEASE, .mm = mm, .start = 0, .end = ULONG_MAX, }; ie it is sitting doing a huge number of xa_erases, I suppose. Probably in normal exit the notifier is removed before the mm is destroyed. The xa_erase needs to be a bit smarter to jump over gaps in the tree perhaps some xa_for_each() xa_erase() pattern? > > Also I get this: > > > > lib/test_hmm.c: In function ‘dmirror_devmem_fault_alloc_and_copy’: > > lib/test_hmm.c:1041:25: warning: unused variable ‘vma’ [-Wunused-variable] > > 1041 | struct vm_area_struct *vma = args->vma; > > > > But this is a kernel bug, due to alloc_page_vma being a #define not a > > static inline and me having CONFIG_NUMA off in this .config > > Fixed. in gfp.h? Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31E77C4332B for ; Fri, 20 Mar 2020 00:03:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DF1D620409 for ; Fri, 20 Mar 2020 00:03:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="niE0QFwA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727260AbgCTADt (ORCPT ); Thu, 19 Mar 2020 20:03:49 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:35844 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727256AbgCTADt (ORCPT ); Thu, 19 Mar 2020 20:03:49 -0400 Received: by mail-qk1-f193.google.com with SMTP id d11so5220855qko.3 for ; Thu, 19 Mar 2020 17:03:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=1pJxpq8bFhAdtFvjymRkDlT+zzgC3t93bLWt4+LdA0o=; b=niE0QFwAScsJNksiL1RD0EoZyWLTkmIFeNZTwtNcFLkMjDayi0bxMD56+ydBpxjFer ccfFIEFWOAqX7S6DF8dy2B90sjOJd11kjQNv+zzxmWUfQ+NuUvGWeapza5y7m3aL1wJG j3hvcJFjjm2RHXKkVcIUMT/Unai34kkTifnPtPSjJsdNLmUD0nhajqJhX+Agn9Kz3t8d MseyS+KCBRWIY/Clnpsqfu2H+lIZb7sticYp/HJnFGSWY7gUq7c/Zqf45npz2Z8DGQ78 mXMSeVQdS57T+ekPAXfTCqZyqZmCir8LMmtQu7NDGFXY1YDgmRsH7Uq3cAhojnHv5w1y bUgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=1pJxpq8bFhAdtFvjymRkDlT+zzgC3t93bLWt4+LdA0o=; b=jpE4D7B8WO4FoyBVyU5GuUWW8sieDAj+VQxn5IYP0xFVsQIo2uUkrbNcxB0lG/s9/O 0raksInYGFGyUtTj92xY3LvocD+DOvZnV9YHmmeWcusVsBz17FjEA4fncl5Rbw7E+tmn G/CyJ7OGNEJyjltCZpGwBkNtU4JrNKGpzcgQjRZWkNRPCNK/kkGNU9/TvIUNWtX4xnBi C8qSMmH73kTwbf5i4cNPCEw2llDJfL2ocaNqkPQVQKAIBY1gTXfjNAXzLxWuf4hF039d 7gt3PnD3hw8nrYZhlM04BXV1laptJIo9n9NiU2stk+CgXokTf81/NeCp6na33uptrgkj J5hA== X-Gm-Message-State: ANhLgQ1kf3/jstW5Jz+kNn70H22MYhB6GK+Gnq+IymiOKGc0B3Q99kiY cjJlJAw6Swf3DXGV8NbmPbvxVw== X-Google-Smtp-Source: ADFU+vvmojx05al3/dBJDAgC1QulIXZbUAIDZwSQ+speDrKVTJZR9ajSBjpmE5SDOXsMKVejlONOTg== X-Received: by 2002:a37:b44:: with SMTP id 65mr5064085qkl.201.1584662627565; Thu, 19 Mar 2020 17:03:47 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id h11sm139725qtr.38.2020.03.19.17.03.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 17:03:47 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jF58b-00029b-D5; Thu, 19 Mar 2020 21:03:45 -0300 Date: Thu, 19 Mar 2020 21:03:45 -0300 From: Jason Gunthorpe To: Ralph Campbell Cc: Christoph Hellwig , Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= , Ben Skeggs , Jerome Glisse , kvm-ppc@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-ID: <20200320000345.GO20941@ziepe.ca> References: <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> <20200319181716.GK20941@ziepe.ca> <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org On Thu, Mar 19, 2020 at 03:56:50PM -0700, Ralph Campbell wrote: > Adding linux-kselftest@vger.kernel.org for the test config question. > > On 3/19/20 11:17 AM, Jason Gunthorpe wrote: > > On Tue, Mar 17, 2020 at 04:14:31PM -0700, Ralph Campbell wrote: > > > > > > On 3/17/20 5:59 AM, Christoph Hellwig wrote: > > > > On Tue, Mar 17, 2020 at 09:47:55AM -0300, Jason Gunthorpe wrote: > > > > > I've been using v7 of Ralph's tester and it is working well - it has > > > > > DEVICE_PRIVATE support so I think it can test this flow too. Ralph are > > > > > you able? > > > > > > > > > > This hunk seems trivial enough to me, can we include it now? > > > > > > > > I can send a separate patch for it once the tester covers it. I don't > > > > want to add it to the original patch as it is a significant behavior > > > > change compared to the existing code. > > > > > > > > > > Attached is an updated version of my HMM tests based on linux-5.6.0-rc6. > > > I ran this OK with Jason's 8+1 HMM patches, Christoph's 1-5 misc HMM clean ups, > > > and Christoph's 1-4 device private page changes applied. > > > > I'd like to get this to mergable, it looks pretty good now, but I have > > no idea about selftests - and I'm struggling to even compile the tools > > dir > > > > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > > > index 69def4a9df00..4d22ce7879a7 100644 > > > +++ b/lib/Kconfig.debug > > > @@ -2162,6 +2162,18 @@ config TEST_MEMINIT > > > If unsure, say N. > > > +config TEST_HMM > > > + tristate "Test HMM (Heterogeneous Memory Management)" > > > + depends on DEVICE_PRIVATE > > > + select HMM_MIRROR > > > + select MMU_NOTIFIER > > > > extra spaces > > Will fix in v8. > > > In general I wonder if it even makes sense that DEVICE_PRIVATE is user > > selectable? > > Should tests enable the feature or the feature enable the test? > IMHO, if the feature is being compiled into the kernel, that should > enable the menu item for the test. If the feature isn't selected, > no need to test it :-) I ment if DEVICE_PRIVATE should be a user selectable option at all, or should it be turned on when a driver like nouveau is selected. Is there some downside to enabling DEVICE_PRIVATE? > > The notifier holds a mmgrab, no need for another one > > OK. I'll replace dmirror->mm with dmirror->notifier.mm. Right that is good too > > > + filp->private_data = dmirror; > > > > Not sure what this comment means > > I'll change the comment to: > /* > * The first open of the device character file registers the address > * space of the process doing the open() system call with the device. > * Subsequent file opens by other processes will have access to the > * first process' address space. > */ How does this happen? The function looks like it always does the same thing > > > +static bool dmirror_interval_invalidate(struct mmu_interval_notifier *mni, > > > + const struct mmu_notifier_range *range, > > > + unsigned long cur_seq) > > > +{ > > > + struct dmirror *dmirror = container_of(mni, struct dmirror, notifier); > > > + struct mm_struct *mm = dmirror->mm; > > > + > > > + /* > > > + * If the process doesn't exist, we don't need to invalidate the > > > + * device page table since the address space will be torn down. > > > + */ > > > + if (!mmget_not_zero(mm)) > > > + return true; > > > > Why? Don't the notifiers provide for this already. > > > > mmget_not_zero() is required before calling hmm_range_fault() though Oh... This is the invalidate_all path during invalidation IMHO you should test the invalidation reason in the range to exclude this. But xa_erase looks totally safe so there should be no reason to do that. > This is a workaround for a problem I don't quite understand. > If you change tools/testing/selftests/vm/hmm-tests.c line 868 to > ASSERT_EQ(ret, -1); > Then the test will abort, core dump, and cause two problems, > 1) the migrated page will be faulted back to system memory in order to write > it to the core dump. This triggers lockdep_assert_held(&walk.mm->mmap_sem) > in walk_page_range(). Has the migration stuff become entangled with the xarray? > [ 137.980718] Code: 80 2f 1a 83 c6 05 e9 8d 7b 01 01 e8 3e b1 b1 fe e9 05 ff ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 41 56 41 55 41 54 55 <48> 89 fd 53 4c 8d 6d 10 e8 3c fc ff ff 49 89 c4 4c 89 e0 83 e0 03 > [ 137.999461] RSP: 0018:ffffc900015e77c8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 > [ 138.007028] RAX: ffff8886e508c408 RBX: 0000000000000000 RCX: ffffffff82626c89 > [ 138.014159] RDX: dffffc0000000000 RSI: 0000000000000000 RDI: ffffc900015e78a0 > [ 138.021293] RBP: ffffc900015e78a0 R08: ffffffff811461c4 R09: fffff520002bcf17 > [ 138.028426] R10: fffff520002bcf16 R11: 0000000000000003 R12: 0000000002606d10 > [ 138.035557] R13: ffff8886e508c448 R14: 0000000000000031 R15: ffffffffa06546a0 > [ 138.042701] ? do_raw_spin_lock+0x104/0x1d0 > [ 138.046888] ? xas_store+0x19/0xa60 > [ 138.050390] xas_store+0x5b3/0xa60 > [ 138.053806] ? register_lock_class+0x860/0x860 > [ 138.058267] __xa_erase+0x96/0x110 > [ 138.061673] ? xas_store+0xa60/0xa60 > [ 138.065267] xa_erase+0x19/0x30 oh, it is doing this: static void mn_itree_release(struct mmu_notifier_subscriptions *subscriptions, struct mm_struct *mm) { struct mmu_notifier_range range = { .flags = MMU_NOTIFIER_RANGE_BLOCKABLE, .event = MMU_NOTIFY_RELEASE, .mm = mm, .start = 0, .end = ULONG_MAX, }; ie it is sitting doing a huge number of xa_erases, I suppose. Probably in normal exit the notifier is removed before the mm is destroyed. The xa_erase needs to be a bit smarter to jump over gaps in the tree perhaps some xa_for_each() xa_erase() pattern? > > Also I get this: > > > > lib/test_hmm.c: In function ‘dmirror_devmem_fault_alloc_and_copy’: > > lib/test_hmm.c:1041:25: warning: unused variable ‘vma’ [-Wunused-variable] > > 1041 | struct vm_area_struct *vma = args->vma; > > > > But this is a kernel bug, due to alloc_page_vma being a #define not a > > static inline and me having CONFIG_NUMA off in this .config > > Fixed. in gfp.h? Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6F84C43333 for ; Fri, 20 Mar 2020 08:01:26 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7232520739 for ; Fri, 20 Mar 2020 08:01:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="niE0QFwA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7232520739 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4AF1B89E8C; Fri, 20 Mar 2020 08:01:25 +0000 (UTC) Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) by gabe.freedesktop.org (Postfix) with ESMTPS id CACE76EA68 for ; Fri, 20 Mar 2020 00:03:48 +0000 (UTC) Received: by mail-qk1-x743.google.com with SMTP id l25so309954qki.7 for ; Thu, 19 Mar 2020 17:03:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=1pJxpq8bFhAdtFvjymRkDlT+zzgC3t93bLWt4+LdA0o=; b=niE0QFwAScsJNksiL1RD0EoZyWLTkmIFeNZTwtNcFLkMjDayi0bxMD56+ydBpxjFer ccfFIEFWOAqX7S6DF8dy2B90sjOJd11kjQNv+zzxmWUfQ+NuUvGWeapza5y7m3aL1wJG j3hvcJFjjm2RHXKkVcIUMT/Unai34kkTifnPtPSjJsdNLmUD0nhajqJhX+Agn9Kz3t8d MseyS+KCBRWIY/Clnpsqfu2H+lIZb7sticYp/HJnFGSWY7gUq7c/Zqf45npz2Z8DGQ78 mXMSeVQdS57T+ekPAXfTCqZyqZmCir8LMmtQu7NDGFXY1YDgmRsH7Uq3cAhojnHv5w1y bUgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=1pJxpq8bFhAdtFvjymRkDlT+zzgC3t93bLWt4+LdA0o=; b=QEHswFDSQi/vyZnd/M3bRBULuSzu60YLw3gi7Niq4tdbpwOfV08Wkqj+1JsmugFD42 Dru14jtRHXUZ5MCqBUGciO78u5XYen2BhMQynDmV5wXY+95GxRTPlj2W2dmZ/0rV5ENk JFVTUwIdYXnY4Aqk4+Dw3FCWn89SrQGSSlLjIy6F0bLEpx2mvvnYfFmaASLtqrUMXWgK MqIEWXaqHv1Et92jW2a22/aF8zBUw4syht0YJHSt1yGbuGhgpzJbCZW8JPD1o76/QvIF QExxKm4rFIVh98U0hvALLsRvAVyyq027B8vvtnG5jfQwWDFVPz+CDOhxtIhJE0BcgfmY oGlQ== X-Gm-Message-State: ANhLgQ0W89Ap0vR5OkhPMZIXTfWE+Ud3FkiF1JXyUhQ5+uoUfUSzaKMg XWK1IK4AqosBhHUortAiFAqOeg== X-Google-Smtp-Source: ADFU+vvmojx05al3/dBJDAgC1QulIXZbUAIDZwSQ+speDrKVTJZR9ajSBjpmE5SDOXsMKVejlONOTg== X-Received: by 2002:a37:b44:: with SMTP id 65mr5064085qkl.201.1584662627565; Thu, 19 Mar 2020 17:03:47 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id h11sm139725qtr.38.2020.03.19.17.03.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 19 Mar 2020 17:03:47 -0700 (PDT) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1jF58b-00029b-D5; Thu, 19 Mar 2020 21:03:45 -0300 Date: Thu, 19 Mar 2020 21:03:45 -0300 From: Jason Gunthorpe To: Ralph Campbell Subject: Re: [PATCH 3/4] mm: simplify device private page handling in hmm_range_fault Message-ID: <20200320000345.GO20941@ziepe.ca> References: <20200316193216.920734-4-hch@lst.de> <7256f88d-809e-4aba-3c46-a223bd8cc521@nvidia.com> <20200317121536.GQ20941@ziepe.ca> <20200317122445.GA11662@lst.de> <20200317122813.GA11866@lst.de> <20200317124755.GR20941@ziepe.ca> <20200317125955.GA12847@lst.de> <24fca825-3b0f-188f-bcf2-fadcf3a9f05a@nvidia.com> <20200319181716.GK20941@ziepe.ca> <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <89e33770-a0ab-e1ec-d5e5-535edefd3fd3@nvidia.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Mailman-Approved-At: Fri, 20 Mar 2020 08:01:25 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, kvm-ppc@vger.kernel.org, Christoph Hellwig , linux-mm@kvack.org, Jerome Glisse , Ben Skeggs , linux-kselftest@vger.kernel.org, Dan Williams , Bharata B Rao , Christian =?utf-8?B?S8O2bmln?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T24gVGh1LCBNYXIgMTksIDIwMjAgYXQgMDM6NTY6NTBQTSAtMDcwMCwgUmFscGggQ2FtcGJlbGwg d3JvdGU6Cj4gQWRkaW5nIGxpbnV4LWtzZWxmdGVzdEB2Z2VyLmtlcm5lbC5vcmcgZm9yIHRoZSB0 ZXN0IGNvbmZpZyBxdWVzdGlvbi4KPiAKPiBPbiAzLzE5LzIwIDExOjE3IEFNLCBKYXNvbiBHdW50 aG9ycGUgd3JvdGU6Cj4gPiBPbiBUdWUsIE1hciAxNywgMjAyMCBhdCAwNDoxNDozMVBNIC0wNzAw LCBSYWxwaCBDYW1wYmVsbCB3cm90ZToKPiA+ID4gCj4gPiA+IE9uIDMvMTcvMjAgNTo1OSBBTSwg Q2hyaXN0b3BoIEhlbGx3aWcgd3JvdGU6Cj4gPiA+ID4gT24gVHVlLCBNYXIgMTcsIDIwMjAgYXQg MDk6NDc6NTVBTSAtMDMwMCwgSmFzb24gR3VudGhvcnBlIHdyb3RlOgo+ID4gPiA+ID4gSSd2ZSBi ZWVuIHVzaW5nIHY3IG9mIFJhbHBoJ3MgdGVzdGVyIGFuZCBpdCBpcyB3b3JraW5nIHdlbGwgLSBp dCBoYXMKPiA+ID4gPiA+IERFVklDRV9QUklWQVRFIHN1cHBvcnQgc28gSSB0aGluayBpdCBjYW4g dGVzdCB0aGlzIGZsb3cgdG9vLiBSYWxwaCBhcmUKPiA+ID4gPiA+IHlvdSBhYmxlPwo+ID4gPiA+ ID4gCj4gPiA+ID4gPiBUaGlzIGh1bmsgc2VlbXMgdHJpdmlhbCBlbm91Z2ggdG8gbWUsIGNhbiB3 ZSBpbmNsdWRlIGl0IG5vdz8KPiA+ID4gPiAKPiA+ID4gPiBJIGNhbiBzZW5kIGEgc2VwYXJhdGUg cGF0Y2ggZm9yIGl0IG9uY2UgdGhlIHRlc3RlciBjb3ZlcnMgaXQuICBJIGRvbid0Cj4gPiA+ID4g d2FudCB0byBhZGQgaXQgdG8gdGhlIG9yaWdpbmFsIHBhdGNoIGFzIGl0IGlzIGEgc2lnbmlmaWNh bnQgYmVoYXZpb3IKPiA+ID4gPiBjaGFuZ2UgY29tcGFyZWQgdG8gdGhlIGV4aXN0aW5nIGNvZGUu Cj4gPiA+ID4gCj4gPiA+IAo+ID4gPiBBdHRhY2hlZCBpcyBhbiB1cGRhdGVkIHZlcnNpb24gb2Yg bXkgSE1NIHRlc3RzIGJhc2VkIG9uIGxpbnV4LTUuNi4wLXJjNi4KPiA+ID4gSSByYW4gdGhpcyBP SyB3aXRoIEphc29uJ3MgOCsxIEhNTSBwYXRjaGVzLCBDaHJpc3RvcGgncyAxLTUgbWlzYyBITU0g Y2xlYW4gdXBzLAo+ID4gPiBhbmQgQ2hyaXN0b3BoJ3MgMS00IGRldmljZSBwcml2YXRlIHBhZ2Ug Y2hhbmdlcyBhcHBsaWVkLgo+ID4gCj4gPiBJJ2QgbGlrZSB0byBnZXQgdGhpcyB0byBtZXJnYWJs ZSwgaXQgbG9va3MgcHJldHR5IGdvb2Qgbm93LCBidXQgSSBoYXZlCj4gPiBubyBpZGVhIGFib3V0 IHNlbGZ0ZXN0cyAtIGFuZCBJJ20gc3RydWdnbGluZyB0byBldmVuIGNvbXBpbGUgdGhlIHRvb2xz Cj4gPiBkaXIKPiA+IAo+ID4gPiBkaWZmIC0tZ2l0IGEvbGliL0tjb25maWcuZGVidWcgYi9saWIv S2NvbmZpZy5kZWJ1Zwo+ID4gPiBpbmRleCA2OWRlZjRhOWRmMDAuLjRkMjJjZTc4NzlhNyAxMDA2 NDQKPiA+ID4gKysrIGIvbGliL0tjb25maWcuZGVidWcKPiA+ID4gQEAgLTIxNjIsNiArMjE2Miwx OCBAQCBjb25maWcgVEVTVF9NRU1JTklUCj4gPiA+ICAgCSAgSWYgdW5zdXJlLCBzYXkgTi4KPiA+ ID4gK2NvbmZpZyBURVNUX0hNTQo+ID4gPiArCXRyaXN0YXRlICJUZXN0IEhNTSAoSGV0ZXJvZ2Vu ZW91cyBNZW1vcnkgTWFuYWdlbWVudCkiCj4gPiA+ICsJZGVwZW5kcyBvbiBERVZJQ0VfUFJJVkFU RQo+ID4gPiArCXNlbGVjdCBITU1fTUlSUk9SCj4gPiA+ICsgICAgICAgIHNlbGVjdCBNTVVfTk9U SUZJRVIKPiA+IAo+ID4gZXh0cmEgc3BhY2VzCj4gCj4gV2lsbCBmaXggaW4gdjguCj4gCj4gPiBJ biBnZW5lcmFsIEkgd29uZGVyIGlmIGl0IGV2ZW4gbWFrZXMgc2Vuc2UgdGhhdCBERVZJQ0VfUFJJ VkFURSBpcyB1c2VyCj4gPiBzZWxlY3RhYmxlPwo+IAo+IFNob3VsZCB0ZXN0cyBlbmFibGUgdGhl IGZlYXR1cmUgb3IgdGhlIGZlYXR1cmUgZW5hYmxlIHRoZSB0ZXN0Pwo+IElNSE8sIGlmIHRoZSBm ZWF0dXJlIGlzIGJlaW5nIGNvbXBpbGVkIGludG8gdGhlIGtlcm5lbCwgdGhhdCBzaG91bGQKPiBl bmFibGUgdGhlIG1lbnUgaXRlbSBmb3IgdGhlIHRlc3QuIElmIHRoZSBmZWF0dXJlIGlzbid0IHNl bGVjdGVkLAo+IG5vIG5lZWQgdG8gdGVzdCBpdCA6LSkKCkkgbWVudCBpZiBERVZJQ0VfUFJJVkFU RSBzaG91bGQgYmUgYSB1c2VyIHNlbGVjdGFibGUgb3B0aW9uIGF0IGFsbCwgb3IKc2hvdWxkIGl0 IGJlIHR1cm5lZCBvbiB3aGVuIGEgZHJpdmVyIGxpa2Ugbm91dmVhdSBpcyBzZWxlY3RlZC4KCklz IHRoZXJlIHNvbWUgZG93bnNpZGUgdG8gZW5hYmxpbmcgREVWSUNFX1BSSVZBVEU/Cgo+ID4gVGhl IG5vdGlmaWVyIGhvbGRzIGEgbW1ncmFiLCBubyBuZWVkIGZvciBhbm90aGVyIG9uZQo+IAo+IE9L LiBJJ2xsIHJlcGxhY2UgZG1pcnJvci0+bW0gd2l0aCBkbWlycm9yLT5ub3RpZmllci5tbS4KClJp Z2h0IHRoYXQgaXMgZ29vZCB0b28KCj4gPiA+ICsJZmlscC0+cHJpdmF0ZV9kYXRhID0gZG1pcnJv cjsKPiA+IAo+ID4gTm90IHN1cmUgd2hhdCB0aGlzIGNvbW1lbnQgbWVhbnMKPiAKPiBJJ2xsIGNo YW5nZSB0aGUgY29tbWVudCB0bzoKPiAJICAvKgo+ICAgICAgICAgICogVGhlIGZpcnN0IG9wZW4g b2YgdGhlIGRldmljZSBjaGFyYWN0ZXIgZmlsZSByZWdpc3RlcnMgdGhlIGFkZHJlc3MKPiAgICAg ICAgICAqIHNwYWNlIG9mIHRoZSBwcm9jZXNzIGRvaW5nIHRoZSBvcGVuKCkgc3lzdGVtIGNhbGwg d2l0aCB0aGUgZGV2aWNlLgo+ICAgICAgICAgICogU3Vic2VxdWVudCBmaWxlIG9wZW5zIGJ5IG90 aGVyIHByb2Nlc3NlcyB3aWxsIGhhdmUgYWNjZXNzIHRvIHRoZQo+ICAgICAgICAgICogZmlyc3Qg cHJvY2VzcycgYWRkcmVzcyBzcGFjZS4KPiAgICAgICAgICAqLwoKSG93IGRvZXMgdGhpcyBoYXBw ZW4/IFRoZSBmdW5jdGlvbiBsb29rcyBsaWtlIGl0IGFsd2F5cyBkb2VzIHRoZSBzYW1lIHRoaW5n Cgo+ID4gPiArc3RhdGljIGJvb2wgZG1pcnJvcl9pbnRlcnZhbF9pbnZhbGlkYXRlKHN0cnVjdCBt bXVfaW50ZXJ2YWxfbm90aWZpZXIgKm1uaSwKPiA+ID4gKwkJCQljb25zdCBzdHJ1Y3QgbW11X25v dGlmaWVyX3JhbmdlICpyYW5nZSwKPiA+ID4gKwkJCQl1bnNpZ25lZCBsb25nIGN1cl9zZXEpCj4g PiA+ICt7Cj4gPiA+ICsJc3RydWN0IGRtaXJyb3IgKmRtaXJyb3IgPSBjb250YWluZXJfb2YobW5p LCBzdHJ1Y3QgZG1pcnJvciwgbm90aWZpZXIpOwo+ID4gPiArCXN0cnVjdCBtbV9zdHJ1Y3QgKm1t ID0gZG1pcnJvci0+bW07Cj4gPiA+ICsKPiA+ID4gKwkvKgo+ID4gPiArCSAqIElmIHRoZSBwcm9j ZXNzIGRvZXNuJ3QgZXhpc3QsIHdlIGRvbid0IG5lZWQgdG8gaW52YWxpZGF0ZSB0aGUKPiA+ID4g KwkgKiBkZXZpY2UgcGFnZSB0YWJsZSBzaW5jZSB0aGUgYWRkcmVzcyBzcGFjZSB3aWxsIGJlIHRv cm4gZG93bi4KPiA+ID4gKwkgKi8KPiA+ID4gKwlpZiAoIW1tZ2V0X25vdF96ZXJvKG1tKSkKPiA+ ID4gKwkJcmV0dXJuIHRydWU7Cj4gPiAKPiA+IFdoeT8gRG9uJ3QgdGhlIG5vdGlmaWVycyBwcm92 aWRlIGZvciB0aGlzIGFscmVhZHkuCj4gPiAKPiA+IG1tZ2V0X25vdF96ZXJvKCkgaXMgcmVxdWly ZWQgYmVmb3JlIGNhbGxpbmcgaG1tX3JhbmdlX2ZhdWx0KCkgdGhvdWdoCgpPaC4uLiBUaGlzIGlz IHRoZSBpbnZhbGlkYXRlX2FsbCBwYXRoIGR1cmluZyBpbnZhbGlkYXRpb24KCklNSE8geW91IHNo b3VsZCB0ZXN0IHRoZSBpbnZhbGlkYXRpb24gcmVhc29uIGluIHRoZSByYW5nZSB0byBleGNsdWRl CnRoaXMuCgpCdXQgeGFfZXJhc2UgbG9va3MgdG90YWxseSBzYWZlIHNvIHRoZXJlIHNob3VsZCBi ZSBubyByZWFzb24gdG8gZG8KdGhhdC4KCj4gVGhpcyBpcyBhIHdvcmthcm91bmQgZm9yIGEgcHJv YmxlbSBJIGRvbid0IHF1aXRlIHVuZGVyc3RhbmQuCj4gSWYgeW91IGNoYW5nZSB0b29scy90ZXN0 aW5nL3NlbGZ0ZXN0cy92bS9obW0tdGVzdHMuYyBsaW5lIDg2OCB0bwo+IAlBU1NFUlRfRVEocmV0 LCAtMSk7Cj4gVGhlbiB0aGUgdGVzdCB3aWxsIGFib3J0LCBjb3JlIGR1bXAsIGFuZCBjYXVzZSB0 d28gcHJvYmxlbXMsCj4gMSkgdGhlIG1pZ3JhdGVkIHBhZ2Ugd2lsbCBiZSBmYXVsdGVkIGJhY2sg dG8gc3lzdGVtIG1lbW9yeSBpbiBvcmRlciB0byB3cml0ZQo+ICAgIGl0IHRvIHRoZSBjb3JlIGR1 bXAuIFRoaXMgdHJpZ2dlcnMgbG9ja2RlcF9hc3NlcnRfaGVsZCgmd2Fsay5tbS0+bW1hcF9zZW0p Cj4gICAgaW4gd2Fsa19wYWdlX3JhbmdlKCkuCgpIYXMgdGhlIG1pZ3JhdGlvbiBzdHVmZiBiZWNv bWUgZW50YW5nbGVkIHdpdGggdGhlIHhhcnJheT8KCj4gWyAgMTM3Ljk4MDcxOF0gQ29kZTogODAg MmYgMWEgODMgYzYgMDUgZTkgOGQgN2IgMDEgMDEgZTggM2UgYjEgYjEgZmUgZTkgMDUgZmYgZmYg ZmYgNjYgNjYgMmUgMGYgMWYgODQgMDAgMDAgMDAgMDAgMDAgMGYgMWYgMDAgNDEgNTYgNDEgNTUg NDEgNTQgNTUgPDQ4PiA4OSBmZCA1MyA0YyA4ZCA2ZCAxMCBlOCAzYyBmYyBmZiBmZiA0OSA4OSBj NCA0YyA4OSBlMCA4MyBlMCAwMwo+IFsgIDEzNy45OTk0NjFdIFJTUDogMDAxODpmZmZmYzkwMDAx NWU3N2M4IEVGTEFHUzogMDAwMDAyNDYgT1JJR19SQVg6IGZmZmZmZmZmZmZmZmZmMTMKPiBbICAx MzguMDA3MDI4XSBSQVg6IGZmZmY4ODg2ZTUwOGM0MDggUkJYOiAwMDAwMDAwMDAwMDAwMDAwIFJD WDogZmZmZmZmZmY4MjYyNmM4OQo+IFsgIDEzOC4wMTQxNTldIFJEWDogZGZmZmZjMDAwMDAwMDAw MCBSU0k6IDAwMDAwMDAwMDAwMDAwMDAgUkRJOiBmZmZmYzkwMDAxNWU3OGEwCj4gWyAgMTM4LjAy MTI5M10gUkJQOiBmZmZmYzkwMDAxNWU3OGEwIFIwODogZmZmZmZmZmY4MTE0NjFjNCBSMDk6IGZm ZmZmNTIwMDAyYmNmMTcKPiBbICAxMzguMDI4NDI2XSBSMTA6IGZmZmZmNTIwMDAyYmNmMTYgUjEx OiAwMDAwMDAwMDAwMDAwMDAzIFIxMjogMDAwMDAwMDAwMjYwNmQxMAo+IFsgIDEzOC4wMzU1NTdd IFIxMzogZmZmZjg4ODZlNTA4YzQ0OCBSMTQ6IDAwMDAwMDAwMDAwMDAwMzEgUjE1OiBmZmZmZmZm ZmEwNjU0NmEwCj4gWyAgMTM4LjA0MjcwMV0gID8gZG9fcmF3X3NwaW5fbG9jaysweDEwNC8weDFk MAo+IFsgIDEzOC4wNDY4ODhdICA/IHhhc19zdG9yZSsweDE5LzB4YTYwCj4gWyAgMTM4LjA1MDM5 MF0gIHhhc19zdG9yZSsweDViMy8weGE2MAo+IFsgIDEzOC4wNTM4MDZdICA/IHJlZ2lzdGVyX2xv Y2tfY2xhc3MrMHg4NjAvMHg4NjAKPiBbICAxMzguMDU4MjY3XSAgX194YV9lcmFzZSsweDk2LzB4 MTEwCj4gWyAgMTM4LjA2MTY3M10gID8geGFzX3N0b3JlKzB4YTYwLzB4YTYwCj4gWyAgMTM4LjA2 NTI2N10gIHhhX2VyYXNlKzB4MTkvMHgzMAoKb2gsIGl0IGlzIGRvaW5nIHRoaXM6CgpzdGF0aWMg dm9pZCBtbl9pdHJlZV9yZWxlYXNlKHN0cnVjdCBtbXVfbm90aWZpZXJfc3Vic2NyaXB0aW9ucyAq c3Vic2NyaXB0aW9ucywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgbW1fc3Ry dWN0ICptbSkKewogICAgICAgIHN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgcmFuZ2UgPSB7CiAg ICAgICAgICAgICAgICAuZmxhZ3MgPSBNTVVfTk9USUZJRVJfUkFOR0VfQkxPQ0tBQkxFLAogICAg ICAgICAgICAgICAgLmV2ZW50ID0gTU1VX05PVElGWV9SRUxFQVNFLAogICAgICAgICAgICAgICAg Lm1tID0gbW0sCiAgICAgICAgICAgICAgICAuc3RhcnQgPSAwLAogICAgICAgICAgICAgICAgLmVu ZCA9IFVMT05HX01BWCwKICAgICAgICB9OwoKaWUgaXQgaXMgc2l0dGluZyBkb2luZyBhIGh1Z2Ug bnVtYmVyIG9mIHhhX2VyYXNlcywgSSBzdXBwb3NlLiBQcm9iYWJseQppbiBub3JtYWwgZXhpdCB0 aGUgbm90aWZpZXIgaXMgcmVtb3ZlZCBiZWZvcmUgdGhlIG1tIGlzIGRlc3Ryb3llZC4KClRoZSB4 YV9lcmFzZSBuZWVkcyB0byBiZSBhIGJpdCBzbWFydGVyIHRvIGp1bXAgb3ZlciBnYXBzIGluIHRo ZSB0cmVlCnBlcmhhcHMgc29tZQoKeGFfZm9yX2VhY2goKQogICB4YV9lcmFzZSgpCgpwYXR0ZXJu PwoKPiA+IEFsc28gSSBnZXQgdGhpczoKPiA+IAo+ID4gbGliL3Rlc3RfaG1tLmM6IEluIGZ1bmN0 aW9uIOKAmGRtaXJyb3JfZGV2bWVtX2ZhdWx0X2FsbG9jX2FuZF9jb3B54oCZOgo+ID4gbGliL3Rl c3RfaG1tLmM6MTA0MToyNTogd2FybmluZzogdW51c2VkIHZhcmlhYmxlIOKAmHZtYeKAmSBbLVd1 bnVzZWQtdmFyaWFibGVdCj4gPiAgIDEwNDEgfCAgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEg PSBhcmdzLT52bWE7Cj4gPiAKPiA+IEJ1dCB0aGlzIGlzIGEga2VybmVsIGJ1ZywgZHVlIHRvIGFs bG9jX3BhZ2Vfdm1hIGJlaW5nIGEgI2RlZmluZSBub3QgYQo+ID4gc3RhdGljIGlubGluZSBhbmQg bWUgaGF2aW5nIENPTkZJR19OVU1BIG9mZiBpbiB0aGlzIC5jb25maWcKPiAKPiBGaXhlZC4KCmlu IGdmcC5oPwoKSmFzb24KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX18KZHJpLWRldmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Au b3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRl dmVsCg==