From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [PATCH hmm 02/15] mm/mmu_notifier: add an interval tree notifier Date: Mon, 21 Oct 2019 14:30:56 -0400 Message-ID: <20191021183056.GA3177@redhat.com> References: <20191015181242.8343-1-jgg@ziepe.ca> <20191015181242.8343-3-jgg@ziepe.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20191015181242.8343-3-jgg@ziepe.ca> Content-Disposition: inline List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Jason Gunthorpe Cc: Andrea Arcangeli , Ralph Campbell , linux-rdma@vger.kernel.org, John Hubbard , Felix.Kuehling@amd.com, dri-devel@lists.freedesktop.org, Michal Hocko , linux-mm@kvack.org, Jason Gunthorpe , amd-gfx@lists.freedesktop.org, Ben Skeggs List-Id: amd-gfx.lists.freedesktop.org T24gVHVlLCBPY3QgMTUsIDIwMTkgYXQgMDM6MTI6MjlQTSAtMDMwMCwgSmFzb24gR3VudGhvcnBl IHdyb3RlOgo+IEZyb206IEphc29uIEd1bnRob3JwZSA8amdnQG1lbGxhbm94LmNvbT4KPiAKPiBP ZiB0aGUgMTMgdXNlcnMgb2YgbW11X25vdGlmaWVycywgOCBvZiB0aGVtIHVzZSBvbmx5Cj4gaW52 YWxpZGF0ZV9yYW5nZV9zdGFydC9lbmQoKSBhbmQgaW1tZWRpYXRlbHkgaW50ZXJzZWN0IHRoZQo+ IG1tdV9ub3RpZmllcl9yYW5nZSB3aXRoIHNvbWUga2luZCBvZiBpbnRlcm5hbCBsaXN0IG9mIFZB cy4gIDQgdXNlIGFuCj4gaW50ZXJ2YWwgdHJlZSAoaTkxNV9nZW0sIHJhZGVvbl9tbiwgdW1lbV9v ZHAsIGhmaTEpLiA0IHVzZSBhIGxpbmtlZCBsaXN0Cj4gb2Ygc29tZSBraW5kIChzY2lmX2RtYSwg dmhvc3QsIGdudGRldiwgaG1tKQo+IAo+IEFuZCB0aGUgcmVtYWluaW5nIDUgZWl0aGVyIGRvbid0 IHVzZSBpbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KCkgb3IgZG8gc29tZQo+IHNwZWNpYWwgdGhpbmcg d2l0aCBpdC4KPiAKPiBJdCB0dXJucyBvdXQgdGhhdCBidWlsZGluZyBhIGNvcnJlY3Qgc2NoZW1l IHdpdGggYW4gaW50ZXJ2YWwgdHJlZSBpcwo+IHByZXR0eSBjb21wbGljYXRlZCwgcGFydGljdWxh cmx5IGlmIHRoZSB1c2UgY2FzZSBpcyBzeW5jaHJvbml6aW5nIGFnYWluc3QKPiBhbm90aGVyIHRo cmVhZCBkb2luZyBnZXRfdXNlcl9wYWdlcygpLiAgTWFueSBvZiB0aGVzZSBpbXBsZW1lbnRhdGlv bnMgaGF2ZQo+IHZhcmlvdXMgc3VidGxlIGFuZCBkaWZmaWN1bHQgdG8gZml4IHJhY2VzLgo+IAo+ IFRoaXMgYXBwcm9hY2ggcHV0cyB0aGUgaW50ZXJ2YWwgdHJlZSBhcyBjb21tb24gY29kZSBhdCB0 aGUgdG9wIG9mIHRoZSBtbXUKPiBub3RpZmllciBjYWxsIHRyZWUgYW5kIGltcGxlbWVudHMgYSBz aGFyZWFibGUgbG9ja2luZyBzY2hlbWUuCj4gCj4gSXQgaW5jbHVkZXM6Cj4gIC0gQW4gaW50ZXJ2 YWwgdHJlZSB0cmFja2luZyBWQSByYW5nZXMsIHdpdGggcGVyLXJhbmdlIGNhbGxiYWNrcwo+ICAt IEEgcmVhZC93cml0ZSBsb2NraW5nIHNjaGVtZSBmb3IgdGhlIGludGVydmFsIHRyZWUgdGhhdCBh dm9pZHMKPiAgICBzbGVlcGluZyBpbiB0aGUgbm90aWZpZXIgcGF0aCAoZm9yIE9PTSBraWxsZXIp Cj4gIC0gQSBzZXF1ZW5jZSBjb3VudGVyIGJhc2VkIGNvbGxpc2lvbi1yZXRyeSBsb2NraW5nIHNj aGVtZSB0byB0ZWxsCj4gICAgZGV2aWNlIHBhZ2UgZmF1bHQgdGhhdCBhIFZBIHJhbmdlIGlzIGJl aW5nIGNvbmN1cnJlbnRseSBpbnZhbGlkYXRlZC4KPiAKPiBUaGlzIGlzIGJhc2VkIG9uIHZhcmlv dXMgaWRlYXM6Cj4gLSBobW0gYWNjdW11bGF0ZXMgaW52YWxpZGF0ZWQgVkEgcmFuZ2VzIGFuZCBy ZWxlYXNlcyB0aGVtIHdoZW4gYWxsCj4gICBpbnZhbGlkYXRlcyBhcmUgZG9uZSwgdmlhIGFjdGl2 ZV9pbnZhbGlkYXRlX3JhbmdlcyBjb3VudC4KPiAgIFRoaXMgYXBwcm9hY2ggYXZvaWRzIGhhdmlu ZyB0byBpbnRlcnNlY3QgdGhlIGludGVydmFsIHRyZWUgdHdpY2UgKGFzCj4gICB1bWVtX29kcCBk b2VzKSBhdCB0aGUgcG90ZW50aWFsIGNvc3Qgb2YgYSBsb25nZXIgZGV2aWNlIHBhZ2UgZmF1bHQu Cj4gCj4gLSBrdm0vdW1lbV9vZHAgdXNlIGEgc2VxdWVuY2UgY291bnRlciB0byBkcml2ZSB0aGUg Y29sbGlzaW9uIHJldHJ5LAo+ICAgdmlhIGludmFsaWRhdGVfc2VxCj4gCj4gLSBhIGRlZmVycmVk IHdvcmsgdG9kbyBsaXN0IG9uIHVubG9jayBzY2hlbWUgbGlrZSBSVE5MLCB2aWEgZGVmZXJyZWRf bGlzdC4KPiAgIFRoaXMgbWFrZXMgYWRkaW5nL3JlbW92aW5nIGludGVydmFsIHRyZWUgbWVtYmVy cyBtb3JlIGRldGVybWluaXN0aWMKPiAKPiAtIHNlcWxvY2ssIGV4Y2VwdCB0aGlzIHZlcnNpb24g bWFrZXMgdGhlIHNlcWxvY2sgaWRlYSBtdWx0aS1ob2xkZXIgb24gdGhlCj4gICB3cml0ZSBzaWRl IGJ5IHByb3RlY3RpbmcgaXQgd2l0aCBhY3RpdmVfaW52YWxpZGF0ZV9yYW5nZXMgYW5kIGEgc3Bp bmxvY2sKPiAKPiBUbyBtaW5pbWl6ZSBNTSBvdmVyaGVhZCB3aGVuIG9ubHkgdGhlIGludGVydmFs IHRyZWUgaXMgYmVpbmcgdXNlZCwgdGhlCj4gZW50aXJlIFNSQ1UgYW5kIGhsaXN0IG92ZXJoZWFk cyBhcmUgZHJvcHBlZCB1c2luZyBzb21lIHNpbXBsZQo+IGJyYW5jaGVzLiBTaW1pbGFybHkgdGhl IGludGVydmFsIHRyZWUgb3ZlcmhlYWQgaXMgZHJvcHBlZCB3aGVuIGluIGhsaXN0Cj4gbW9kZS4K PiAKPiBUaGUgb3ZlcmhlYWQgZnJvbSB0aGUgbWFuZGF0b3J5IHNwaW5sb2NrIGlzIGJyb2FkbHkg dGhlIHNhbWUgYXMgbW9zdCBvZgo+IGV4aXN0aW5nIHVzZXJzIHdoaWNoIGFscmVhZHkgaGFkIGEg bG9jayAob3IgdHdvKSBvZiBzb21lIHNvcnQgb24gdGhlCj4gaW52YWxpZGF0aW9uIHBhdGguCj4g Cj4gQ2M6IEFuZHJlYSBBcmNhbmdlbGkgPGFhcmNhbmdlQHJlZGhhdC5jb20+Cj4gQ2M6IE1pY2hh bCBIb2NrbyA8bWhvY2tvQGtlcm5lbC5vcmc+Cj4gU2lnbmVkLW9mZi1ieTogSmFzb24gR3VudGhv cnBlIDxqZ2dAbWVsbGFub3guY29tPgo+IC0tLQo+ICBpbmNsdWRlL2xpbnV4L21tdV9ub3RpZmll ci5oIHwgIDc4ICsrKysrKwo+ICBtbS9LY29uZmlnICAgICAgICAgICAgICAgICAgIHwgICAxICsK PiAgbW0vbW11X25vdGlmaWVyLmMgICAgICAgICAgICB8IDUyOSArKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrKystLQo+ICAzIGZpbGVzIGNoYW5nZWQsIDU4MyBpbnNlcnRpb25zKCspLCAy NSBkZWxldGlvbnMoLSkKPiAKPiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbXVfbm90aWZp ZXIuaCBiL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmgKPiBpbmRleCAxMmJkNjAzZDMxOGNl Ny4uYmMyYjEyNDgzZGUxMjcgMTAwNjQ0Cj4gLS0tIGEvaW5jbHVkZS9saW51eC9tbXVfbm90aWZp ZXIuaAo+ICsrKyBiL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmgKPiBAQCAtNiwxMCArNiwx MiBAQAo+ICAjaW5jbHVkZSA8bGludXgvc3BpbmxvY2suaD4KPiAgI2luY2x1ZGUgPGxpbnV4L21t X3R5cGVzLmg+Cj4gICNpbmNsdWRlIDxsaW51eC9zcmN1Lmg+Cj4gKyNpbmNsdWRlIDxsaW51eC9p bnRlcnZhbF90cmVlLmg+Cj4gIAo+ICBzdHJ1Y3QgbW11X25vdGlmaWVyX21tOwo+ICBzdHJ1Y3Qg bW11X25vdGlmaWVyOwo+ICBzdHJ1Y3QgbW11X25vdGlmaWVyX3JhbmdlOwo+ICtzdHJ1Y3QgbW11 X3JhbmdlX25vdGlmaWVyOwo+ICAKPiAgLyoqCj4gICAqIGVudW0gbW11X25vdGlmaWVyX2V2ZW50 IC0gcmVhc29uIGZvciB0aGUgbW11IG5vdGlmaWVyIGNhbGxiYWNrCj4gQEAgLTMyLDYgKzM0LDkg QEAgc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZTsKPiAgICogYWNjZXNzIGZsYWdzKS4gVXNlciBz aG91bGQgc29mdCBkaXJ0eSB0aGUgcGFnZSBpbiB0aGUgZW5kIGNhbGxiYWNrIHRvIG1ha2UKPiAg ICogc3VyZSB0aGF0IGFueW9uZSByZWx5aW5nIG9uIHNvZnQgZGlydHluZXNzIGNhdGNoIHBhZ2Vz IHRoYXQgbWlnaHQgYmUgd3JpdHRlbgo+ICAgKiB0aHJvdWdoIG5vbiBDUFUgbWFwcGluZ3MuCj4g KyAqCj4gKyAqIEBNTVVfTk9USUZZX1JFTEVBU0U6IHVzZWQgZHVyaW5nIG1tdV9yYW5nZV9ub3Rp ZmllciBpbnZhbGlkYXRlIHRvIHNpZ25hbCB0aGF0Cj4gKyAqIHRoZSBtbSByZWZjb3VudCBpcyB6 ZXJvIGFuZCB0aGUgcmFuZ2UgaXMgbm8gbG9uZ2VyIGFjY2Vzc2libGUuCj4gICAqLwo+ICBlbnVt IG1tdV9ub3RpZmllcl9ldmVudCB7Cj4gIAlNTVVfTk9USUZZX1VOTUFQID0gMCwKPiBAQCAtMzks NiArNDQsNyBAQCBlbnVtIG1tdV9ub3RpZmllcl9ldmVudCB7Cj4gIAlNTVVfTk9USUZZX1BST1RF Q1RJT05fVk1BLAo+ICAJTU1VX05PVElGWV9QUk9URUNUSU9OX1BBR0UsCj4gIAlNTVVfTk9USUZZ X1NPRlRfRElSVFksCj4gKwlNTVVfTk9USUZZX1JFTEVBU0UsCj4gIH07Cj4gIAo+ICAjZGVmaW5l IE1NVV9OT1RJRklFUl9SQU5HRV9CTE9DS0FCTEUgKDEgPDwgMCkKPiBAQCAtMjIyLDYgKzIyOCwy NSBAQCBzdHJ1Y3QgbW11X25vdGlmaWVyIHsKPiAgCXVuc2lnbmVkIGludCB1c2VyczsKPiAgfTsK PiAgCj4gKy8qKgo+ICsgKiBzdHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyX29wcwo+ICsgKiBAaW52 YWxpZGF0ZTogVXBvbiByZXR1cm4gdGhlIGNhbGxlciBtdXN0IHN0b3AgdXNpbmcgYW55IFNQVEVz IHdpdGhpbiB0aGlzCj4gKyAqICAgICAgICAgICAgICByYW5nZSwgdGhpcyBmdW5jdGlvbiBjYW4g c2xlZXAuIFJldHVybiBmYWxzZSBpZiBibG9ja2luZyB3YXMKPiArICogICAgICAgICAgICAgIHJl cXVpcmVkIGJ1dCByYW5nZSBpcyBub24tYmxvY2tpbmcKPiArICovCj4gK3N0cnVjdCBtbXVfcmFu Z2Vfbm90aWZpZXJfb3BzIHsKPiArCWJvb2wgKCppbnZhbGlkYXRlKShzdHJ1Y3QgbW11X3Jhbmdl X25vdGlmaWVyICptcm4sCj4gKwkJCSAgIGNvbnN0IHN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2Ug KnJhbmdlKTsKPiArfTsKPiArCj4gK3N0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgewo+ICsJc3Ry dWN0IGludGVydmFsX3RyZWVfbm9kZSBpbnRlcnZhbF90cmVlOwo+ICsJY29uc3Qgc3RydWN0IG1t dV9yYW5nZV9ub3RpZmllcl9vcHMgKm9wczsKPiArCXN0cnVjdCBobGlzdF9ub2RlIGRlZmVycmVk X2l0ZW07Cj4gKwl1bnNpZ25lZCBsb25nIGludmFsaWRhdGVfc2VxOwo+ICsJc3RydWN0IG1tX3N0 cnVjdCAqbW07Cj4gK307Cj4gKwo+ICAjaWZkZWYgQ09ORklHX01NVV9OT1RJRklFUgo+ICAKPiAg I2lmZGVmIENPTkZJR19MT0NLREVQCj4gQEAgLTI2Myw2ICsyODgsNTkgQEAgZXh0ZXJuIGludCBf X21tdV9ub3RpZmllcl9yZWdpc3RlcihzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKPiAgCQkJCSAg IHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKTsKPiAgZXh0ZXJuIHZvaWQgbW11X25vdGlmaWVyX3VucmVn aXN0ZXIoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCj4gIAkJCQkgICAgc3RydWN0IG1tX3N0cnVj dCAqbW0pOwo+ICsKPiArdW5zaWduZWQgbG9uZyBtbXVfcmFuZ2VfcmVhZF9iZWdpbihzdHJ1Y3Qg bW11X3JhbmdlX25vdGlmaWVyICptcm4pOwo+ICtpbnQgbW11X3JhbmdlX25vdGlmaWVyX2luc2Vy dChzdHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyICptcm4sCj4gKwkJCSAgICAgIHVuc2lnbmVkIGxv bmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAo+ICsJCQkgICAgICBzdHJ1Y3QgbW1fc3Ry dWN0ICptbSk7Cj4gK2ludCBtbXVfcmFuZ2Vfbm90aWZpZXJfaW5zZXJ0X2xvY2tlZChzdHJ1Y3Qg bW11X3JhbmdlX25vdGlmaWVyICptcm4sCj4gKwkJCQkgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQs IHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAo+ICsJCQkJICAgICBzdHJ1Y3QgbW1fc3RydWN0ICptbSk7 Cj4gK3ZvaWQgbW11X3JhbmdlX25vdGlmaWVyX3JlbW92ZShzdHJ1Y3QgbW11X3JhbmdlX25vdGlm aWVyICptcm4pOwo+ICsKPiArLyoqCj4gKyAqIG1tdV9yYW5nZV9yZWFkX3JldHJ5IC0gRW5kIGEg cmVhZCBzaWRlIGNyaXRpY2FsIHNlY3Rpb24gYWdhaW5zdCBhIFZBIHJhbmdlCj4gKyAqIG1ybjog VGhlIHJhbmdlIHVuZGVyIGxvY2sKPiArICogc2VxOiBUaGUgcmV0dXJuIG9mIHRoZSBwYWlyZWQg bW11X3JhbmdlX3JlYWRfYmVnaW4oKQo+ICsgKgo+ICsgKiBUaGlzIE1VU1QgYmUgY2FsbGVkIHVu ZGVyIGEgdXNlciBwcm92aWRlZCBsb2NrIHRoYXQgaXMgYWxzbyBoZWxkCj4gKyAqIHVuY29uZGl0 aW9uYWxseSBieSBvcC0+aW52YWxpZGF0ZSgpLiBUaGF0IGxvY2sgcHJvdmlkZXMgdGhlIHJlcXVp cmVkIFNNUAo+ICsgKiBiYXJyaWVyIGZvciBoYW5kbGluZyBpbnZhbGlkYXRlX3NlcS4KPiArICoK PiArICogRWFjaCBjYWxsIHNob3VsZCBiZSBwYWlyZWQgd2l0aCBhIHNpbmdsZSBtbXVfcmFuZ2Vf cmVhZF9iZWdpbigpIGFuZAo+ICsgKiBzaG91bGQgYmUgdXNlZCB0byBjb25jbHVkZSB0aGUgcmVh ZCBzaWRlLgo+ICsgKgo+ICsgKiBSZXR1cm5zIHRydWUgaWYgYW4gaW52YWxpZGF0aW9uIGNvbGxp ZGVkIHdpdGggdGhpcyBjcml0aWNhbCBzZWN0aW9uLCBhbmQKPiArICogdGhlIGNhbGxlciBzaG91 bGQgcmV0cnkuCj4gKyAqLwo+ICtzdGF0aWMgaW5saW5lIGJvb2wgbW11X3JhbmdlX3JlYWRfcmV0 cnkoc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJuLAo+ICsJCQkJCXVuc2lnbmVkIGxvbmcg c2VxKQo+ICt7Cj4gKwlyZXR1cm4gUkVBRF9PTkNFKG1ybi0+aW52YWxpZGF0ZV9zZXEpICE9IHNl cTsKPiArfQoKV2hhdCBhYm91dCBjYWxsaW5nIHRoaXMgbW11X3JhbmdlX3JlYWRfZW5kKCkgaW5z dGVhZCA/IFRvIG1hdGNoCndpdGggdGhlIG1tdV9yYW5nZV9yZWFkX2JlZ2luKCkuCgoKPiArCj4g Ky8qKgo+ICsgKiBtbXVfcmFuZ2VfY2hlY2tfcmV0cnkgLSBUZXN0IGlmIGEgY29sbGlzaW9uIGhh cyBvY2N1cnJlZAo+ICsgKiBtcm46IFRoZSByYW5nZSB1bmRlciBsb2NrCj4gKyAqIHNlcTogVGhl IHJldHVybiBvZiB0aGUgbWF0Y2hpbmcgbW11X3JhbmdlX3JlYWRfYmVnaW4oKQo+ICsgKgo+ICsg KiBUaGlzIGNhbiBiZSB1c2VkIGluIHRoZSBjcml0aWNhbCBzZWN0aW9uIGJldHdlZW4gbW11X3Jh bmdlX3JlYWRfYmVnaW4oKSBhbmQKPiArICogbW11X3JhbmdlX3JlYWRfcmV0cnkoKS4gIEEgcmV0 dXJuIG9mIHRydWUgaW5kaWNhdGVzIGFuIGludmFsaWRhdGlvbiBoYXMKPiArICogY29sbGlkZWQg d2l0aCB0aGlzIGxvY2sgYW5kIGEgZnV0dXJlIG1tdV9yYW5nZV9yZWFkX3JldHJ5KCkgd2lsbCBy ZXR1cm4KPiArICogdHJ1ZS4KPiArICoKPiArICogRmFsc2UgaXMgbm90IHJlbGlhYmxlIGFuZCBv bmx5IHN1Z2dlc3RzIGEgY29sbGlzaW9uIGhhcyBub3QgaGFwcGVuZWQuIEl0Cj4gKyAqIGNhbiBi ZSBjYWxsZWQgbWFueSB0aW1lcyBhbmQgZG9lcyBub3QgaGF2ZSB0byBob2xkIHRoZSB1c2VyIHBy b3ZpZGVkIGxvY2suCj4gKyAqCj4gKyAqIFRoaXMgY2FsbCBjYW4gYmUgdXNlZCBhcyBwYXJ0IG9m IGxvb3BzIGFuZCBvdGhlciBleHBlbnNpdmUgb3BlcmF0aW9ucyB0bwo+ICsgKiBleHBlZGl0ZSBh IHJldHJ5Lgo+ICsgKi8KPiArc3RhdGljIGlubGluZSBib29sIG1tdV9yYW5nZV9jaGVja19yZXRy eShzdHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyICptcm4sCj4gKwkJCQkJIHVuc2lnbmVkIGxvbmcg c2VxKQo+ICt7Cj4gKwlyZXR1cm4gUkVBRF9PTkNFKG1ybi0+aW52YWxpZGF0ZV9zZXEpICE9IHNl cTsKPiArfQo+ICsKPiAgZXh0ZXJuIHZvaWQgX19tbXVfbm90aWZpZXJfbW1fZGVzdHJveShzdHJ1 Y3QgbW1fc3RydWN0ICptbSk7Cj4gIGV4dGVybiB2b2lkIF9fbW11X25vdGlmaWVyX3JlbGVhc2Uo c3RydWN0IG1tX3N0cnVjdCAqbW0pOwo+ICBleHRlcm4gaW50IF9fbW11X25vdGlmaWVyX2NsZWFy X2ZsdXNoX3lvdW5nKHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAo+IGRpZmYgLS1naXQgYS9tbS9LY29u ZmlnIGIvbW0vS2NvbmZpZwo+IGluZGV4IGE1ZGFlOWE3ZWI1MTBhLi5kMGI1MDQ2ZDlhZWZmZCAx MDA2NDQKPiAtLS0gYS9tbS9LY29uZmlnCj4gKysrIGIvbW0vS2NvbmZpZwo+IEBAIC0yODQsNiAr Mjg0LDcgQEAgY29uZmlnIFZJUlRfVE9fQlVTCj4gIGNvbmZpZyBNTVVfTk9USUZJRVIKPiAgCWJv b2wKPiAgCXNlbGVjdCBTUkNVCj4gKwlzZWxlY3QgSU5URVJWQUxfVFJFRQo+ICAKPiAgY29uZmln IEtTTQo+ICAJYm9vbCAiRW5hYmxlIEtTTSBmb3IgcGFnZSBtZXJnaW5nIgo+IGRpZmYgLS1naXQg YS9tbS9tbXVfbm90aWZpZXIuYyBiL21tL21tdV9ub3RpZmllci5jCj4gaW5kZXggMzY3NjcwY2Zk MDJiN2IuLjVlNWU3NWViY2RlNGFmIDEwMDY0NAo+IC0tLSBhL21tL21tdV9ub3RpZmllci5jCj4g KysrIGIvbW0vbW11X25vdGlmaWVyLmMKPiBAQCAtMTIsNiArMTIsNyBAQAo+ICAjaW5jbHVkZSA8 bGludXgvZXhwb3J0Lmg+Cj4gICNpbmNsdWRlIDxsaW51eC9tbS5oPgo+ICAjaW5jbHVkZSA8bGlu dXgvZXJyLmg+Cj4gKyNpbmNsdWRlIDxsaW51eC9pbnRlcnZhbF90cmVlLmg+Cj4gICNpbmNsdWRl IDxsaW51eC9zcmN1Lmg+Cj4gICNpbmNsdWRlIDxsaW51eC9yY3VwZGF0ZS5oPgo+ICAjaW5jbHVk ZSA8bGludXgvc2NoZWQuaD4KPiBAQCAtMzYsMTAgKzM3LDI0MyBAQCBzdHJ1Y3QgbG9ja2RlcF9t YXAgX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydF9tYXAgPSB7Cj4gIHN0cnVj dCBtbXVfbm90aWZpZXJfbW0gewo+ICAJLyogYWxsIG1tdSBub3RpZmllcnMgcmVnaXN0ZXJlZCBp biB0aGlzIG1tIGFyZSBxdWV1ZWQgaW4gdGhpcyBsaXN0ICovCj4gIAlzdHJ1Y3QgaGxpc3RfaGVh ZCBsaXN0Owo+ICsJYm9vbCBoYXNfaW50ZXJ2YWw7Cj4gIAkvKiB0byBzZXJpYWxpemUgdGhlIGxp c3QgbW9kaWZpY2F0aW9ucyBhbmQgaGxpc3RfdW5oYXNoZWQgKi8KPiAgCXNwaW5sb2NrX3QgbG9j azsKPiArCXVuc2lnbmVkIGxvbmcgaW52YWxpZGF0ZV9zZXE7Cj4gKwl1bnNpZ25lZCBsb25nIGFj dGl2ZV9pbnZhbGlkYXRlX3JhbmdlczsKPiArCXN0cnVjdCByYl9yb290X2NhY2hlZCBpdHJlZTsK PiArCXdhaXRfcXVldWVfaGVhZF90IHdxOwo+ICsJc3RydWN0IGhsaXN0X2hlYWQgZGVmZXJyZWRf bGlzdDsKPiAgfTsKPiAgCj4gKy8qCj4gKyAqIFRoaXMgaXMgYSBjb2xsaXNpb24tcmV0cnkgcmVh ZC1zaWRlL3dyaXRlLXNpZGUgJ2xvY2snLCBhIGxvdCBsaWtlIGEKPiArICogc2VxY291bnQsIGhv d2V2ZXIgdGhpcyBhbGxvd3MgbXVsdGlwbGUgd3JpdGUtc2lkZXMgdG8gaG9sZCBpdCBhdAo+ICsg KiBvbmNlLiBDb25jZXB0dWFsbHkgdGhlIHdyaXRlIHNpZGUgaXMgcHJvdGVjdGluZyB0aGUgdmFs dWVzIG9mIHRoZSBQVEVzIGluCj4gKyAqIHRoaXMgbW0sIHN1Y2ggdGhhdCBQVEVTIGNhbm5vdCBi ZSByZWFkIGludG8gU1BURXMgd2hpbGUgYW55IHdyaXRlciBleGlzdHMuCj4gKyAqCj4gKyAqIE5v dGUgdGhhdCB0aGUgY29yZSBtbSBjcmVhdGVzIG5lc3RlZCBpbnZhbGlkYXRlX3JhbmdlX3N0YXJ0 KCkvZW5kKCkgcmVnaW9ucwo+ICsgKiB3aXRoaW4gdGhlIHNhbWUgdGhyZWFkLCBhbmQgcnVucyBp bnZhbGlkYXRlX3JhbmdlX3N0YXJ0KCkvZW5kKCkgaW4gcGFyYWxsZWwKPiArICogb24gbXVsdGlw bGUgQ1BVcy4gVGhpcyBpcyBkZXNpZ25lZCB0byBub3QgcmVkdWNlIGNvbmN1cnJlbmN5IG9yIGJs b2NrCj4gKyAqIHByb2dyZXNzIG9uIHRoZSBtbSBzaWRlLgo+ICsgKgo+ICsgKiBBcyBhIHNlY29u ZGFyeSBmdW5jdGlvbiwgaG9sZGluZyB0aGUgZnVsbCB3cml0ZSBzaWRlIGFsc28gc2VydmVzIHRv IHByZXZlbnQKPiArICogd3JpdGVycyBmb3IgdGhlIGl0cmVlLCB0aGlzIGlzIGFuIG9wdGltaXph dGlvbiB0byBhdm9pZCBleHRyYSBsb2NraW5nCj4gKyAqIGR1cmluZyBpbnZhbGlkYXRlX3Jhbmdl X3N0YXJ0L2VuZCBub3RpZmllcnMuCj4gKyAqCj4gKyAqIFRoZSB3cml0ZSBzaWRlIGhhcyB0d28g c3RhdGVzLCBmdWxseSBleGNsdWRlZDoKPiArICogIC0gbW0tPmFjdGl2ZV9pbnZhbGlkYXRlX3Jh bmdlcyAhPSAwCj4gKyAqICAtIG1ubi0+aW52YWxpZGF0ZV9zZXEgJiAxID09IFRydWUKPiArICog IC0gc29tZSByYW5nZSBvbiB0aGUgbW1fc3RydWN0IGlzIGJlaW5nIGludmFsaWRhdGVkCj4gKyAq ICAtIHRoZSBpdHJlZSBpcyBub3QgYWxsb3dlZCB0byBjaGFuZ2UKPiArICoKPiArICogQW5kIHBh cnRpYWxseSBleGNsdWRlZDoKPiArICogIC0gbW0tPmFjdGl2ZV9pbnZhbGlkYXRlX3JhbmdlcyAh PSAwCj4gKyAqICAtIHNvbWUgcmFuZ2Ugb24gdGhlIG1tX3N0cnVjdCBpcyBiZWluZyBpbnZhbGlk YXRlZAo+ICsgKiAgLSB0aGUgaXRyZWUgaXMgYWxsb3dlZCB0byBjaGFuZ2UKPiArICoKPiArICog VGhlIGxhdGVyIHN0YXRlIGF2b2lkcyBzb21lIGV4cGVuc2l2ZSB3b3JrIG9uIGludl9lbmQgaW4g dGhlIGNvbW1vbiBjYXNlIG9mCj4gKyAqIG5vIG1ybiBtb25pdG9yaW5nIHRoZSBWQS4KPiArICov Cj4gK3N0YXRpYyBib29sIG1uX2l0cmVlX2lzX2ludmFsaWRhdGluZyhzdHJ1Y3QgbW11X25vdGlm aWVyX21tICptbW5fbW0pCj4gK3sKPiArCWxvY2tkZXBfYXNzZXJ0X2hlbGQoJm1tbl9tbS0+bG9j ayk7Cj4gKwlyZXR1cm4gbW1uX21tLT5pbnZhbGlkYXRlX3NlcSAmIDE7Cj4gK30KPiArCj4gK3N0 YXRpYyBzdHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyICoKPiArbW5faXRyZWVfaW52X3N0YXJ0X3Jh bmdlKHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSwKPiArCQkJIGNvbnN0IHN0cnVjdCBt bXVfbm90aWZpZXJfcmFuZ2UgKnJhbmdlLAo+ICsJCQkgdW5zaWduZWQgbG9uZyAqc2VxKQo+ICt7 Cj4gKwlzdHJ1Y3QgaW50ZXJ2YWxfdHJlZV9ub2RlICpub2RlOwo+ICsJc3RydWN0IG1tdV9yYW5n ZV9ub3RpZmllciAqcmVzID0gTlVMTDsKPiArCj4gKwlzcGluX2xvY2soJm1tbl9tbS0+bG9jayk7 Cj4gKwltbW5fbW0tPmFjdGl2ZV9pbnZhbGlkYXRlX3JhbmdlcysrOwo+ICsJbm9kZSA9IGludGVy dmFsX3RyZWVfaXRlcl9maXJzdCgmbW1uX21tLT5pdHJlZSwgcmFuZ2UtPnN0YXJ0LAo+ICsJCQkJ CXJhbmdlLT5lbmQgLSAxKTsKPiArCWlmIChub2RlKSB7Cj4gKwkJbW1uX21tLT5pbnZhbGlkYXRl X3NlcSB8PSAxOwo+ICsJCXJlcyA9IGNvbnRhaW5lcl9vZihub2RlLCBzdHJ1Y3QgbW11X3Jhbmdl X25vdGlmaWVyLAo+ICsJCQkJICAgaW50ZXJ2YWxfdHJlZSk7Cj4gKwl9Cj4gKwo+ICsJKnNlcSA9 IG1tbl9tbS0+aW52YWxpZGF0ZV9zZXE7Cj4gKwlzcGluX3VubG9jaygmbW1uX21tLT5sb2NrKTsK PiArCXJldHVybiByZXM7Cj4gK30KPiArCj4gK3N0YXRpYyBzdHJ1Y3QgbW11X3JhbmdlX25vdGlm aWVyICoKPiArbW5faXRyZWVfaW52X25leHQoc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJu LAo+ICsJCSAgY29uc3Qgc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCj4gK3sKPiAr CXN0cnVjdCBpbnRlcnZhbF90cmVlX25vZGUgKm5vZGU7Cj4gKwo+ICsJbm9kZSA9IGludGVydmFs X3RyZWVfaXRlcl9uZXh0KCZtcm4tPmludGVydmFsX3RyZWUsIHJhbmdlLT5zdGFydCwKPiArCQkJ CSAgICAgICByYW5nZS0+ZW5kIC0gMSk7Cj4gKwlpZiAoIW5vZGUpCj4gKwkJcmV0dXJuIE5VTEw7 Cj4gKwlyZXR1cm4gY29udGFpbmVyX29mKG5vZGUsIHN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIs IGludGVydmFsX3RyZWUpOwo+ICt9Cj4gKwo+ICtzdGF0aWMgdm9pZCBtbl9pdHJlZV9pbnZfZW5k KHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSkKPiArewo+ICsJc3RydWN0IG1tdV9yYW5n ZV9ub3RpZmllciAqbXJuOwo+ICsJc3RydWN0IGhsaXN0X25vZGUgKm5leHQ7Cj4gKwlib29sIG5l ZWRfd2FrZSA9IGZhbHNlOwo+ICsKPiArCXNwaW5fbG9jaygmbW1uX21tLT5sb2NrKTsKPiArCWlm ICgtLW1tbl9tbS0+YWN0aXZlX2ludmFsaWRhdGVfcmFuZ2VzIHx8Cj4gKwkgICAgIW1uX2l0cmVl X2lzX2ludmFsaWRhdGluZyhtbW5fbW0pKSB7Cj4gKwkJc3Bpbl91bmxvY2soJm1tbl9tbS0+bG9j ayk7Cj4gKwkJcmV0dXJuOwo+ICsJfQo+ICsKPiArCW1tbl9tbS0+aW52YWxpZGF0ZV9zZXErKzsK PiArCW5lZWRfd2FrZSA9IHRydWU7Cj4gKwo+ICsJLyoKPiArCSAqIFRoZSBpbnZfZW5kIGluY29y cG9yYXRlcyBhIGRlZmVycmVkIG1lY2hhbmlzbSBsaWtlIHJ0bmwuIEFkZHMgYW5kCgpUaGUgcnRu bCByZWZlcmVuY2UgaXMgbG9zdCBvbiBwZW9wbGUgdW5mYW1pbGlhciB3aXRoIHRoZSBuZXR3b3Jr IDopCmNvZGUgbWF5YmUgbGlrZSBydG5sX2xvY2soKS9ydG5sX3VubG9jaygpIHNvIHBlb3BsZSBo YXZlIGEgY2hhbmNlIHRvCmdyZXAgdGhlIHJpZ2h0IGZ1bmN0aW9uLiBBc3N1bWluZyBpIGFtIG15 c2VsZiBnZXR0aW5nIHRoZSByaWdodApyZWZlcmVuY2UgOikKCj4gKwkgKiByZW1vdmVzIGFyZSBx dWV1ZWQgdW50aWwgdGhlIGZpbmFsIGludl9lbmQgaGFwcGVucyB0aGVuIHRoZXkgYXJlCj4gKwkg KiBwcm9ncmVzc2VkLiBUaGlzIGFycmFuZ2VtZW50IGZvciB0cmVlIHVwZGF0ZXMgaXMgdXNlZCB0 byBhdm9pZAo+ICsJICogdXNpbmcgYSBibG9ja2luZyBsb2NrIGR1cmluZyBpbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0Lgo+ICsJICovCj4gKwlobGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlKG1ybiwgbmV4 dCwgJm1tbl9tbS0+ZGVmZXJyZWRfbGlzdCwKPiArCQkJCSAgZGVmZXJyZWRfaXRlbSkgewo+ICsJ CWlmIChSQl9FTVBUWV9OT0RFKCZtcm4tPmludGVydmFsX3RyZWUucmIpKQo+ICsJCQlpbnRlcnZh bF90cmVlX2luc2VydCgmbXJuLT5pbnRlcnZhbF90cmVlLAo+ICsJCQkJCSAgICAgJm1tbl9tbS0+ aXRyZWUpOwo+ICsJCWVsc2UKPiArCQkJaW50ZXJ2YWxfdHJlZV9yZW1vdmUoJm1ybi0+aW50ZXJ2 YWxfdHJlZSwKPiArCQkJCQkgICAgICZtbW5fbW0tPml0cmVlKTsKPiArCQlobGlzdF9kZWwoJm1y bi0+ZGVmZXJyZWRfaXRlbSk7Cj4gKwl9Cj4gKwlzcGluX3VubG9jaygmbW1uX21tLT5sb2NrKTsK PiArCj4gKwkvKgo+ICsJICogVE9ETzogU2luY2Ugd2UgYWxyZWFkeSBoYXZlIGEgc3BpbmxvY2sg YWJvdmUsIHRoaXMgd291bGQgYmUgZmFzdGVyCj4gKwkgKiBhcyB3YWtlX3VwX3EKPiArCSAqLwo+ ICsJaWYgKG5lZWRfd2FrZSkKPiArCQl3YWtlX3VwX2FsbCgmbW1uX21tLT53cSk7Cj4gK30KPiAr Cj4gKy8qKgo+ICsgKiBtbXVfcmFuZ2VfcmVhZF9iZWdpbiAtIEJlZ2luIGEgcmVhZCBzaWRlIGNy aXRpY2FsIHNlY3Rpb24gYWdhaW5zdCBhIFZBIHJhbmdlCj4gKyAqIG1ybjogVGhlIHJhbmdlIHRv IGxvY2sKPiArICoKPiArICogbW11X3JhbmdlX3JlYWRfYmVnaW4oKS9tbXVfcmFuZ2VfcmVhZF9y ZXRyeSgpIGltcGxlbWVudCBhIGNvbGxpc2lvbi1yZXRyeQo+ICsgKiBsb2NraW5nIHNjaGVtZSBz aW1pbGFyIHRvIHNlcWNvdW50IGZvciB0aGUgVkEgcmFuZ2UgdW5kZXIgbXJuLiBJZiB0aGUgbW0K PiArICogaW52b2tlcyBpbnZhbGlkYXRpb24gZHVyaW5nIHRoZSBjcml0aWNhbCBzZWN0aW9uIHRo ZW4KPiArICogbW11X3JhbmdlX3JlYWRfcmV0cnkoKSB3aWxsIHJldHVybiB0cnVlLgo+ICsgKgo+ ICsgKiBUaGlzIGlzIHVzZWZ1bCB0byBvYnRhaW4gc2hhZG93IFBURXMgd2hlcmUgdGVhcmRvd24g b3Igc2V0dXAgb2YgdGhlIFNQVEVzCj4gKyAqIHJlcXVpcmUgYSBibG9ja2luZyBjb250ZXh0LiAg VGhlIGNyaXRpY2FsIHJlZ2lvbiBmb3JtZWQgYnkgdGhpcyBsb2NrIGNhbgo+ICsgKiBzbGVlcCwg YW5kIHRoZSByZXF1aXJlZCAndXNlcl9sb2NrJyBjYW4gYWxzbyBiZSBhIHNsZWVwaW5nIGxvY2su Cj4gKyAqCj4gKyAqIFRoZSBjYWxsZXIgaXMgcmVxdWlyZWQgdG8gcHJvdmlkZSBhICd1c2VyX2xv Y2snIHRvIHNlcmlhbGl6ZSBib3RoIHRlYXJkb3duCj4gKyAqIGFuZCBzZXR1cC4KPiArICoKPiAr ICogVGhlIHJldHVybiB2YWx1ZSBzaG91bGQgYmUgcGFzc2VkIHRvIG1tdV9yYW5nZV9yZWFkX3Jl dHJ5KCkuCj4gKyAqLwo+ICt1bnNpZ25lZCBsb25nIG1tdV9yYW5nZV9yZWFkX2JlZ2luKHN0cnVj dCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybikKPiArewo+ICsJc3RydWN0IG1tdV9ub3RpZmllcl9t bSAqbW1uX21tID0gbXJuLT5tbS0+bW11X25vdGlmaWVyX21tOwo+ICsJdW5zaWduZWQgbG9uZyBz ZXE7Cj4gKwlib29sIGlzX2ludmFsaWRhdGluZzsKPiArCj4gKwkvKgo+ICsJICogSWYgdGhlIG1y biBoYXMgYSBkaWZmZXJlbnQgc2VxIHZhbHVlIHVuZGVyIHRoZSB1c2VyX2xvY2sgdGhhbiB3ZQo+ ICsJICogc3RhcnRlZCB3aXRoIHRoZW4gaXQgaGFzIGNvbGxpZGVkLgo+ICsJICoKPiArCSAqIElm IHRoZSBtcm4gY3VycmVudGx5IGhhcyB0aGUgc2FtZSBzZXEgdmFsdWUgYXMgdGhlIG1tbl9tbSBz ZXEsIHRoZW4KPiArCSAqIGl0IGlzIGN1cnJlbnRseSBiZXR3ZWVuIGludmFsaWRhdGVfc3RhcnQv ZW5kIGFuZCBpcyBjb2xsaWRpbmcuCj4gKwkgKgo+ICsJICogVGhlIGxvY2tpbmcgbG9va3MgYnJv YWRseSBsaWtlIHRoaXM6Cj4gKwkgKiAgIG1uX3RyZWVfaW52YWxpZGF0ZV9zdGFydCgpOiAgICAg ICAgICBtbXVfcmFuZ2VfcmVhZF9iZWdpbigpOgo+ICsJICogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHNwaW5fbG9jawo+ICsJICogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICBzZXEgPSBSRUFEX09OQ0UobXJuLT5pbnZhbGlkYXRlX3NlcSk7 Cj4gKwkgKiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNlcSA9PSBt bW5fbW0tPmludmFsaWRhdGVfc2VxCj4gKwkgKiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgc3Bpbl91bmxvY2sKPiArCSAqICAgIHNwaW5fbG9jawo+ICsJICogICAgIHNl cSA9ICsrbW1uX21tLT5pbnZhbGlkYXRlX3NlcQo+ICsJICogICAgc3Bpbl91bmxvY2sKPiArCSAq ICAgIG1ybi0+aW52YWxpZGF0ZV9zZXEgPSBzZXEKPiArCSAqICAgICBvcC0+aW52YWxpZGF0ZV9y YW5nZSgpOgo+ICsJICogICAgICAgdXNlcl9sb2NrCj4gKwkgKiAgICAgICB1c2VyX3VubG9jawo+ ICsJICoKPiArCSAqICAgICAgICAgICAgICAgICAgICAgICAgICBbUmVxdWlyZWQ6IG1tdV9yYW5n ZV9yZWFkX3JldHJ5KCkgPT0gdHJ1ZV0KPiArCSAqCj4gKwkgKiAgIG1uX2l0cmVlX2ludl9lbmQo KToKPiArCSAqICAgIHNwaW5fbG9jawo+ICsJICogICAgIHNlcSA9ICsrbW1uX21tLT5pbnZhbGlk YXRlX3NlcQo+ICsJICogICAgc3Bpbl91bmxvY2sKPiArCSAqCj4gKwkgKiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB1c2VyX2xvY2sKPiArCSAqICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBtbXVfcmFuZ2VfcmVhZF9yZXRyeSgpOgo+ICsJICog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBSRUFEX09OQ0UobXJuLT5p bnZhbGlkYXRlX3NlcSkgIT0gc2VxCj4gKwkgKiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICB1c2VyX3VubG9jawo+ICsJICoKPiArCSAqIExvZ2ljYWxseSBtcm4tPmludmFs aWRhdGVfc2VxIGlzIGxvY2tlZCB1bmRlciB0aGUgdXNlciBwcm92aWRlZAo+ICsJICogbG9jaywg aG93ZXZlciB0aGUgd3JpdGUgaXMgcGxhY2VkIGJlZm9yZSB0aGF0IGxvY2sgZHVlIHRvIHRoZSB3 YXkKPiArCSAqIHRoZSBBUEkgaXMgbGF5ZXJlZC4KPiArCSAqCj4gKwkgKiBCYXJyaWVycyBhcmUg bm90IG5lZWRlZCBhcyBhbnkgcmFjZXMgaGVyZSBhcmUgY2xvc2VkIGJ5IGFuIGV2ZW50dWFsCj4g KwkgKiBtbXVfcmFuZ2VfcmVhZF9yZXRyeSgpLCB3aGljaCBwcm92aWRlcyBhIGJhcnJpZXIgdmlh IHRoZSB1c2VyX2xvY2suCj4gKwkgKi8KPiArCXNwaW5fbG9jaygmbW1uX21tLT5sb2NrKTsKPiAr CXNlcSA9IFJFQURfT05DRShtcm4tPmludmFsaWRhdGVfc2VxKTsKPiArCWlzX2ludmFsaWRhdGlu ZyA9IHNlcSA9PSBtbW5fbW0tPmludmFsaWRhdGVfc2VxOwo+ICsJc3Bpbl91bmxvY2soJm1tbl9t bS0+bG9jayk7Cj4gKwo+ICsJLyoKPiArCSAqIG1ybi0+aW52YWxpZGF0ZV9zZXEgaXMgYWx3YXlz IHNldCB0byBhbiBvZGQgdmFsdWUuIFRoaXMgZW5zdXJlcwo+ICsJICogdGhhdCBpZiBzZXEgZG9l cyB3cmFwIHdlIHdpbGwgYWx3YXlzIGNsZWFyIHRoZSBiZWxvdyBzbGVlcCBpbiBzb21lCj4gKwkg KiByZWFzb25hYmxlIHRpbWUgYXMgbW1uX21tLT5pbnZhbGlkYXRlX3NlcSBpcyBldmVuIGluIHRo ZSBpZGxlCj4gKwkgKiBzdGF0ZS4KCkkgdGhpbmsgdGhpcyBjb21tZW50IHNob3VsZCBiZSB3aXRo IHRoZSBzdHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyCmRlZmluaXRpb24gYW5kIHlvdSBzaG91bGQg anVzdCBwb2ludCB0byBpdCBmcm9tIGhlcmUgYXMgdGhlIHNhbWUKY29tbWVudCB3b3VsZCBiZSB1 c2VmdWwgZG93biBiZWxvdy4KCj4gKwkgKi8KPiArCWxvY2tfbWFwX2FjcXVpcmUoJl9fbW11X25v dGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbWFwKTsKPiArCWxvY2tfbWFwX3JlbGVhc2Uo Jl9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbWFwKTsKPiArCWlmIChpc19p bnZhbGlkYXRpbmcpCj4gKwkJd2FpdF9ldmVudChtbW5fbW0tPndxLAo+ICsJCQkgICBSRUFEX09O Q0UobW1uX21tLT5pbnZhbGlkYXRlX3NlcSkgIT0gc2VxKTsKPiArCj4gKwkvKgo+ICsJICogTm90 aWNlIHRoYXQgbW11X3JhbmdlX3JlYWRfcmV0cnkoKSBjYW4gYWxyZWFkeSBiZSB0cnVlIGF0IHRo aXMKPiArCSAqIHBvaW50LCBhdm9pZGluZyBsb29wcyBoZXJlIGFsbG93cyB0aGUgdXNlciBvZiB0 aGlzIGxvY2sgdG8gcHJvdmlkZQo+ICsJICogYSBnbG9iYWwgdGltZSBib3VuZC4KPiArCSAqLwo+ ICsKPiArCXJldHVybiBzZXE7Cj4gK30KPiArRVhQT1JUX1NZTUJPTF9HUEwobW11X3JhbmdlX3Jl YWRfYmVnaW4pOwo+ICsKPiArc3RhdGljIHZvaWQgbW5faXRyZWVfcmVsZWFzZShzdHJ1Y3QgbW11 X25vdGlmaWVyX21tICptbW5fbW0sCj4gKwkJCSAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0pCj4g K3sKPiArCXN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgcmFuZ2UgPSB7Cj4gKwkJLmZsYWdzID0g TU1VX05PVElGSUVSX1JBTkdFX0JMT0NLQUJMRSwKPiArCQkuZXZlbnQgPSBNTVVfTk9USUZZX1JF TEVBU0UsCj4gKwkJLm1tID0gbW0sCj4gKwkJLnN0YXJ0ID0gMCwKPiArCQkuZW5kID0gVUxPTkdf TUFYLAo+ICsJfTsKPiArCXN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybjsKPiArCXVuc2ln bmVkIGxvbmcgY3VyX3NlcTsKPiArCWJvb2wgcmV0Owo+ICsKPiArCWZvciAobXJuID0gbW5faXRy ZWVfaW52X3N0YXJ0X3JhbmdlKG1tbl9tbSwgJnJhbmdlLCAmY3VyX3NlcSk7IG1ybjsKPiArCSAg ICAgbXJuID0gbW5faXRyZWVfaW52X25leHQobXJuLCAmcmFuZ2UpKSB7Cj4gKwkJcmV0ID0gbXJu LT5vcHMtPmludmFsaWRhdGUobXJuLCAmcmFuZ2UpOwo+ICsJCVdBUk5fT04ocmV0KTsKPiArCX0K PiArCj4gKwltbl9pdHJlZV9pbnZfZW5kKG1tbl9tbSk7Cj4gK30KPiArCj4gIC8qCj4gICAqIFRo aXMgZnVuY3Rpb24gY2FuJ3QgcnVuIGNvbmN1cnJlbnRseSBhZ2FpbnN0IG1tdV9ub3RpZmllcl9y ZWdpc3Rlcgo+ICAgKiBiZWNhdXNlIG1tLT5tbV91c2VycyA+IDAgZHVyaW5nIG1tdV9ub3RpZmll cl9yZWdpc3RlciBhbmQgZXhpdF9tbWFwCj4gQEAgLTUyLDE3ICsyODYsMjQgQEAgc3RydWN0IG1t dV9ub3RpZmllcl9tbSB7Cj4gICAqIGNhbid0IGdvIGF3YXkgZnJvbSB1bmRlciB1cyBhcyBleGl0 X21tYXAgaG9sZHMgYW4gbW1fY291bnQgcGluCj4gICAqIGl0c2VsZi4KPiAgICovCj4gLXZvaWQg X19tbXVfbm90aWZpZXJfcmVsZWFzZShzdHJ1Y3QgbW1fc3RydWN0ICptbSkKPiArc3RhdGljIHZv aWQgbW5faGxpc3RfcmVsZWFzZShzdHJ1Y3QgbW11X25vdGlmaWVyX21tICptbW5fbW0sCj4gKwkJ CSAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0pCj4gIHsKPiAgCXN0cnVjdCBtbXVfbm90aWZpZXIg Km1uOwo+ICAJaW50IGlkOwo+ICAKPiArCWlmIChtbW5fbW0tPmhhc19pbnRlcnZhbCkKPiArCQlt bl9pdHJlZV9yZWxlYXNlKG1tbl9tbSwgbW0pOwo+ICsKPiArCWlmIChobGlzdF9lbXB0eSgmbW1u X21tLT5saXN0KSkKPiArCQlyZXR1cm47Cj4gKwo+ICAJLyoKPiAgCSAqIFNSQ1UgaGVyZSB3aWxs IGJsb2NrIG1tdV9ub3RpZmllcl91bnJlZ2lzdGVyIHVudGlsCj4gIAkgKiAtPnJlbGVhc2UgcmV0 dXJucy4KPiAgCSAqLwo+ICAJaWQgPSBzcmN1X3JlYWRfbG9jaygmc3JjdSk7Cj4gLQlobGlzdF9m b3JfZWFjaF9lbnRyeV9yY3UobW4sICZtbS0+bW11X25vdGlmaWVyX21tLT5saXN0LCBobGlzdCkK PiArCWhsaXN0X2Zvcl9lYWNoX2VudHJ5X3JjdShtbiwgJm1tbl9tbS0+bGlzdCwgaGxpc3QpCj4g IAkJLyoKPiAgCQkgKiBJZiAtPnJlbGVhc2UgcnVucyBiZWZvcmUgbW11X25vdGlmaWVyX3VucmVn aXN0ZXIgaXQgbXVzdCBiZQo+ICAJCSAqIGhhbmRsZWQsIGFzIGl0J3MgdGhlIG9ubHkgd2F5IGZv ciB0aGUgZHJpdmVyIHRvIGZsdXNoIGFsbAo+IEBAIC03Miw5ICszMTMsOSBAQCB2b2lkIF9fbW11 X25vdGlmaWVyX3JlbGVhc2Uoc3RydWN0IG1tX3N0cnVjdCAqbW0pCj4gIAkJaWYgKG1uLT5vcHMt PnJlbGVhc2UpCj4gIAkJCW1uLT5vcHMtPnJlbGVhc2UobW4sIG1tKTsKPiAgCj4gLQlzcGluX2xv Y2soJm1tLT5tbXVfbm90aWZpZXJfbW0tPmxvY2spOwo+IC0Jd2hpbGUgKHVubGlrZWx5KCFobGlz dF9lbXB0eSgmbW0tPm1tdV9ub3RpZmllcl9tbS0+bGlzdCkpKSB7Cj4gLQkJbW4gPSBobGlzdF9l bnRyeShtbS0+bW11X25vdGlmaWVyX21tLT5saXN0LmZpcnN0LAo+ICsJc3Bpbl9sb2NrKCZtbW5f bW0tPmxvY2spOwo+ICsJd2hpbGUgKHVubGlrZWx5KCFobGlzdF9lbXB0eSgmbW1uX21tLT5saXN0 KSkpIHsKPiArCQltbiA9IGhsaXN0X2VudHJ5KG1tbl9tbS0+bGlzdC5maXJzdCwKPiAgCQkJCSBz dHJ1Y3QgbW11X25vdGlmaWVyLAo+ICAJCQkJIGhsaXN0KTsKPiAgCQkvKgo+IEBAIC04NSw3ICsz MjYsNyBAQCB2b2lkIF9fbW11X25vdGlmaWVyX3JlbGVhc2Uoc3RydWN0IG1tX3N0cnVjdCAqbW0p Cj4gIAkJICovCj4gIAkJaGxpc3RfZGVsX2luaXRfcmN1KCZtbi0+aGxpc3QpOwo+ICAJfQo+IC0J c3Bpbl91bmxvY2soJm1tLT5tbXVfbm90aWZpZXJfbW0tPmxvY2spOwo+ICsJc3Bpbl91bmxvY2so Jm1tbl9tbS0+bG9jayk7Cj4gIAlzcmN1X3JlYWRfdW5sb2NrKCZzcmN1LCBpZCk7Cj4gIAo+ICAJ LyoKPiBAQCAtMTAwLDYgKzM0MSwxNyBAQCB2b2lkIF9fbW11X25vdGlmaWVyX3JlbGVhc2Uoc3Ry dWN0IG1tX3N0cnVjdCAqbW0pCj4gIAlzeW5jaHJvbml6ZV9zcmN1KCZzcmN1KTsKPiAgfQo+ICAK PiArdm9pZCBfX21tdV9ub3RpZmllcl9yZWxlYXNlKHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQo+ICt7 Cj4gKwlzdHJ1Y3QgbW11X25vdGlmaWVyX21tICptbW5fbW0gPSBtbS0+bW11X25vdGlmaWVyX21t Owo+ICsKPiArCWlmIChtbW5fbW0tPmhhc19pbnRlcnZhbCkKPiArCQltbl9pdHJlZV9yZWxlYXNl KG1tbl9tbSwgbW0pOwo+ICsKPiArCWlmICghaGxpc3RfZW1wdHkoJm1tbl9tbS0+bGlzdCkpCj4g KwkJbW5faGxpc3RfcmVsZWFzZShtbW5fbW0sIG1tKTsKPiArfQo+ICsKPiAgLyoKPiAgICogSWYg bm8geW91bmcgYml0ZmxhZyBpcyBzdXBwb3J0ZWQgYnkgdGhlIGhhcmR3YXJlLCAtPmNsZWFyX2Zs dXNoX3lvdW5nIGNhbgo+ICAgKiB1bm1hcCB0aGUgYWRkcmVzcyBhbmQgcmV0dXJuIDEgb3IgMCBk ZXBlbmRpbmcgaWYgdGhlIG1hcHBpbmcgcHJldmlvdXNseQo+IEBAIC0xNzIsMTQgKzQyNCw0MSBA QCB2b2lkIF9fbW11X25vdGlmaWVyX2NoYW5nZV9wdGUoc3RydWN0IG1tX3N0cnVjdCAqbW0sIHVu c2lnbmVkIGxvbmcgYWRkcmVzcywKPiAgCXNyY3VfcmVhZF91bmxvY2soJnNyY3UsIGlkKTsKPiAg fQo+ICAKPiAtaW50IF9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0 IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCj4gK3N0YXRpYyBpbnQgbW5faXRyZWVfaW52YWxp ZGF0ZShzdHJ1Y3QgbW11X25vdGlmaWVyX21tICptbW5fbW0sCj4gKwkJCQkgICAgIGNvbnN0IHN0 cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgKnJhbmdlKQo+ICt7Cj4gKwlzdHJ1Y3QgbW11X3Jhbmdl X25vdGlmaWVyICptcm47Cj4gKwl1bnNpZ25lZCBsb25nIGN1cl9zZXE7Cj4gKwo+ICsJZm9yICht cm4gPSBtbl9pdHJlZV9pbnZfc3RhcnRfcmFuZ2UobW1uX21tLCByYW5nZSwgJmN1cl9zZXEpOyBt cm47Cj4gKwkgICAgIG1ybiA9IG1uX2l0cmVlX2ludl9uZXh0KG1ybiwgcmFuZ2UpKSB7Cj4gKwkJ Ym9vbCByZXQ7Cj4gKwo+ICsJCVdSSVRFX09OQ0UobXJuLT5pbnZhbGlkYXRlX3NlcSwgY3VyX3Nl cSk7Cj4gKwkJcmV0ID0gbXJuLT5vcHMtPmludmFsaWRhdGUobXJuLCByYW5nZSk7Cj4gKwkJaWYg KCFyZXQgJiYgIVdBUk5fT04obW11X25vdGlmaWVyX3JhbmdlX2Jsb2NrYWJsZShyYW5nZSkpKQoK SXNuJ3QgdGhlIGxvZ2ljIHdyb25nIGhlcmUgPyBXZSB3YW50IHRvIHdhcm4gaWYgdGhlIHJhbmdl CndhcyBtYXJrIGFzIGJsb2NrYWJsZSBhbmQgaW52YWxpZGF0ZSByZXR1cm5lZCBmYWxzZS4gQWxz bwp3ZSB3ZW50IHRvIGJhY2tvZmYgbm8gbWF0dGVyIHdoYXQgaWYgdGhlIGludmFsaWRhdGUgcmV0 dXJuCmZhbHNlIGllOgogICAgaWYgKCFyZXQpIHsKICAgICAgICBXQVJOX09OKG1tdV9ub3RpZmll cl9yYW5nZV9ibG9ja2FibGUocmFuZ2UpKSkKICAgICAgICBnb3RvIG91dF93b3VsZF9ibG9jazsK ICAgIH0KCgo+ICsJCQlnb3RvIG91dF93b3VsZF9ibG9jazsKPiArCX0KPiArCXJldHVybiAwOwo+ ICsKPiArb3V0X3dvdWxkX2Jsb2NrOgo+ICsJLyoKPiArCSAqIE9uIC1FQUdBSU4gdGhlIG5vbi1i bG9ja2luZyBjYWxsZXIgaXMgbm90IGFsbG93ZWQgdG8gY2FsbAo+ICsJICogaW52YWxpZGF0ZV9y YW5nZV9lbmQoKQo+ICsJICovCj4gKwltbl9pdHJlZV9pbnZfZW5kKG1tbl9tbSk7Cj4gKwlyZXR1 cm4gLUVBR0FJTjsKPiArfQo+ICsKPiArc3RhdGljIGludCBtbl9obGlzdF9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSwKPiArCQkJCQkgICBzdHJ1 Y3QgbW11X25vdGlmaWVyX3JhbmdlICpyYW5nZSkKPiAgewo+ICAJc3RydWN0IG1tdV9ub3RpZmll ciAqbW47Cj4gIAlpbnQgcmV0ID0gMDsKPiAgCWludCBpZDsKPiAgCj4gIAlpZCA9IHNyY3VfcmVh ZF9sb2NrKCZzcmN1KTsKPiAtCWhsaXN0X2Zvcl9lYWNoX2VudHJ5X3JjdShtbiwgJnJhbmdlLT5t bS0+bW11X25vdGlmaWVyX21tLT5saXN0LCBobGlzdCkgewo+ICsJaGxpc3RfZm9yX2VhY2hfZW50 cnlfcmN1KG1uLCAmbW1uX21tLT5saXN0LCBobGlzdCkgewo+ICAJCWlmIChtbi0+b3BzLT5pbnZh bGlkYXRlX3JhbmdlX3N0YXJ0KSB7Cj4gIAkJCWludCBfcmV0Owo+ICAKPiBAQCAtMjAzLDE1ICs0 ODIsMzAgQEAgaW50IF9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0 IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCj4gIAlyZXR1cm4gcmV0Owo+ICB9Cj4gIAo+IC12 b2lkIF9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2VfZW5kKHN0cnVjdCBtbXVfbm90aWZp ZXJfcmFuZ2UgKnJhbmdlLAo+IC0JCQkJCSBib29sIG9ubHlfZW5kKQo+ICtpbnQgX19tbXVfbm90 aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyX3JhbmdlICpy YW5nZSkKPiArewo+ICsJc3RydWN0IG1tdV9ub3RpZmllcl9tbSAqbW1uX21tID0gcmFuZ2UtPm1t LT5tbXVfbm90aWZpZXJfbW07Cj4gKwlpbnQgcmV0ID0gMDsKPiArCj4gKwlpZiAobW1uX21tLT5o YXNfaW50ZXJ2YWwpIHsKPiArCQlyZXQgPSBtbl9pdHJlZV9pbnZhbGlkYXRlKG1tbl9tbSwgcmFu Z2UpOwo+ICsJCWlmIChyZXQpCj4gKwkJCXJldHVybiByZXQ7Cj4gKwl9Cj4gKwlpZiAoIWhsaXN0 X2VtcHR5KCZtbW5fbW0tPmxpc3QpKQo+ICsJCXJldHVybiBtbl9obGlzdF9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KG1tbl9tbSwgcmFuZ2UpOwo+ICsJcmV0dXJuIDA7Cj4gK30KPiArCj4gK3N0YXRp YyB2b2lkIG1uX2hsaXN0X2ludmFsaWRhdGVfZW5kKHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1t bl9tbSwKPiArCQkJCSAgICBzdHJ1Y3QgbW11X25vdGlmaWVyX3JhbmdlICpyYW5nZSwKPiArCQkJ CSAgICBib29sIG9ubHlfZW5kKQo+ICB7Cj4gIAlzdHJ1Y3QgbW11X25vdGlmaWVyICptbjsKPiAg CWludCBpZDsKPiAgCj4gLQlsb2NrX21hcF9hY3F1aXJlKCZfX21tdV9ub3RpZmllcl9pbnZhbGlk YXRlX3JhbmdlX3N0YXJ0X21hcCk7Cj4gIAlpZCA9IHNyY3VfcmVhZF9sb2NrKCZzcmN1KTsKPiAt CWhsaXN0X2Zvcl9lYWNoX2VudHJ5X3JjdShtbiwgJnJhbmdlLT5tbS0+bW11X25vdGlmaWVyX21t LT5saXN0LCBobGlzdCkgewo+ICsJaGxpc3RfZm9yX2VhY2hfZW50cnlfcmN1KG1uLCAmbW1uX21t LT5saXN0LCBobGlzdCkgewo+ICAJCS8qCj4gIAkJICogQ2FsbCBpbnZhbGlkYXRlX3JhbmdlIGhl cmUgdG9vIHRvIGF2b2lkIHRoZSBuZWVkIGZvciB0aGUKPiAgCQkgKiBzdWJzeXN0ZW0gb2YgaGF2 aW5nIHRvIHJlZ2lzdGVyIGFuIGludmFsaWRhdGVfcmFuZ2VfZW5kCj4gQEAgLTIzOCw2ICs1MzIs MTkgQEAgdm9pZCBfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX2VuZChzdHJ1Y3QgbW11 X25vdGlmaWVyX3JhbmdlICpyYW5nZSwKPiAgCQl9Cj4gIAl9Cj4gIAlzcmN1X3JlYWRfdW5sb2Nr KCZzcmN1LCBpZCk7Cj4gK30KPiArCj4gK3ZvaWQgX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9y YW5nZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UsCj4gKwkJCQkJIGJvb2wg b25seV9lbmQpCj4gK3sKPiArCXN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSA9IHJhbmdl LT5tbS0+bW11X25vdGlmaWVyX21tOwo+ICsKPiArCWxvY2tfbWFwX2FjcXVpcmUoJl9fbW11X25v dGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbWFwKTsKPiArCWlmIChtbW5fbW0tPmhhc19p bnRlcnZhbCkKPiArCQltbl9pdHJlZV9pbnZfZW5kKG1tbl9tbSk7Cj4gKwo+ICsJaWYgKCFobGlz dF9lbXB0eSgmbW1uX21tLT5saXN0KSkKPiArCQltbl9obGlzdF9pbnZhbGlkYXRlX2VuZChtbW5f bW0sIHJhbmdlLCBvbmx5X2VuZCk7Cj4gIAlsb2NrX21hcF9yZWxlYXNlKCZfX21tdV9ub3RpZmll cl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X21hcCk7Cj4gIH0KPiAgCj4gQEAgLTI1Niw4ICs1NjMs OSBAQCB2b2lkIF9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Uoc3RydWN0IG1tX3N0cnVj dCAqbW0sCj4gIH0KPiAgCj4gIC8qCj4gLSAqIFNhbWUgYXMgbW11X25vdGlmaWVyX3JlZ2lzdGVy IGJ1dCBoZXJlIHRoZSBjYWxsZXIgbXVzdCBob2xkIHRoZQo+IC0gKiBtbWFwX3NlbSBpbiB3cml0 ZSBtb2RlLgo+ICsgKiBTYW1lIGFzIG1tdV9ub3RpZmllcl9yZWdpc3RlciBidXQgaGVyZSB0aGUg Y2FsbGVyIG11c3QgaG9sZCB0aGUgbW1hcF9zZW0gaW4KPiArICogd3JpdGUgbW9kZS4gQSBOVUxM IG1uIHNpZ25hbHMgdGhlIG5vdGlmaWVyIGlzIGJlaW5nIHJlZ2lzdGVyZWQgZm9yIGl0cmVlCj4g KyAqIG1vZGUuCj4gICAqLwo+ICBpbnQgX19tbXVfbm90aWZpZXJfcmVnaXN0ZXIoc3RydWN0IG1t dV9ub3RpZmllciAqbW4sIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQo+ICB7Cj4gQEAgLTI3NCw5ICs1 ODIsNiBAQCBpbnQgX19tbXVfbm90aWZpZXJfcmVnaXN0ZXIoc3RydWN0IG1tdV9ub3RpZmllciAq bW4sIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQo+ICAJCWZzX3JlY2xhaW1fcmVsZWFzZShHRlBfS0VS TkVMKTsKPiAgCX0KPiAgCj4gLQltbi0+bW0gPSBtbTsKPiAtCW1uLT51c2VycyA9IDE7Cj4gLQo+ ICAJaWYgKCFtbS0+bW11X25vdGlmaWVyX21tKSB7Cj4gIAkJLyoKPiAgCQkgKiBrbWFsbG9jIGNh bm5vdCBiZSBjYWxsZWQgdW5kZXIgbW1fdGFrZV9hbGxfbG9ja3MoKSwgYnV0IHdlCj4gQEAgLTI4 NCwyMSArNTg5LDIyIEBAIGludCBfX21tdV9ub3RpZmllcl9yZWdpc3RlcihzdHJ1Y3QgbW11X25v dGlmaWVyICptbiwgc3RydWN0IG1tX3N0cnVjdCAqbW0pCj4gIAkJICogdGhlIHdyaXRlIHNpZGUg b2YgdGhlIG1tYXBfc2VtLgo+ICAJCSAqLwo+ICAJCW1tdV9ub3RpZmllcl9tbSA9Cj4gLQkJCWtt YWxsb2Moc2l6ZW9mKHN0cnVjdCBtbXVfbm90aWZpZXJfbW0pLCBHRlBfS0VSTkVMKTsKPiArCQkJ a3phbGxvYyhzaXplb2Yoc3RydWN0IG1tdV9ub3RpZmllcl9tbSksIEdGUF9LRVJORUwpOwo+ICAJ CWlmICghbW11X25vdGlmaWVyX21tKQo+ICAJCQlyZXR1cm4gLUVOT01FTTsKPiAgCj4gIAkJSU5J VF9ITElTVF9IRUFEKCZtbXVfbm90aWZpZXJfbW0tPmxpc3QpOwo+ICAJCXNwaW5fbG9ja19pbml0 KCZtbXVfbm90aWZpZXJfbW0tPmxvY2spOwo+ICsJCW1tdV9ub3RpZmllcl9tbS0+aW52YWxpZGF0 ZV9zZXEgPSAyOwoKV2h5IHN0YXJ0aW5nIGF0IDIgPwoKPiArCQltbXVfbm90aWZpZXJfbW0tPml0 cmVlID0gUkJfUk9PVF9DQUNIRUQ7Cj4gKwkJaW5pdF93YWl0cXVldWVfaGVhZCgmbW11X25vdGlm aWVyX21tLT53cSk7Cj4gKwkJSU5JVF9ITElTVF9IRUFEKCZtbXVfbm90aWZpZXJfbW0tPmRlZmVy cmVkX2xpc3QpOwo+ICAJfQo+ICAKPiAgCXJldCA9IG1tX3Rha2VfYWxsX2xvY2tzKG1tKTsKPiAg CWlmICh1bmxpa2VseShyZXQpKQo+ICAJCWdvdG8gb3V0X2NsZWFuOwo+ICAKPiAtCS8qIFBhaXJz IHdpdGggdGhlIG1tZHJvcCBpbiBtbXVfbm90aWZpZXJfdW5yZWdpc3Rlcl8qICovCj4gLQltbWdy YWIobW0pOwo+IC0KPiAgCS8qCj4gIAkgKiBTZXJpYWxpemUgdGhlIHVwZGF0ZSBhZ2FpbnN0IG1t dV9ub3RpZmllcl91bnJlZ2lzdGVyLiBBCj4gIAkgKiBzaWRlIG5vdGU6IG1tdV9ub3RpZmllcl9y ZWxlYXNlIGNhbid0IHJ1biBjb25jdXJyZW50bHkgd2l0aAo+IEBAIC0zMDYsMTMgKzYxMiwyNiBA QCBpbnQgX19tbXVfbm90aWZpZXJfcmVnaXN0ZXIoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sIHN0 cnVjdCBtbV9zdHJ1Y3QgKm1tKQo+ICAJICogY3VycmVudC0+bW0gb3IgZXhwbGljaXRseSB3aXRo IGdldF90YXNrX21tKCkgb3Igc2ltaWxhcikuCj4gIAkgKiBXZSBjYW4ndCByYWNlIGFnYWluc3Qg YW55IG90aGVyIG1tdSBub3RpZmllciBtZXRob2QgZWl0aGVyCj4gIAkgKiB0aGFua3MgdG8gbW1f dGFrZV9hbGxfbG9ja3MoKS4KPiArCSAqCj4gKwkgKiByZWxlYXNlIHNlbWFudGljcyBhcmUgcHJv dmlkZWQgZm9yIHVzZXJzIG5vdCBpbnNpZGUgYSBsb2NrIGNvdmVyZWQKPiArCSAqIGJ5IG1tX3Rh a2VfYWxsX2xvY2tzKCkuIGFjcXVpcmUgY2FuIG9ubHkgYmUgdXNlZCB3aGlsZSBob2xkaW5nIHRo ZQo+ICsJICogbW1ncmFiIG9yIG1tZ2V0LCBhbmQgaXMgc2FmZSBiZWNhdXNlIG9uY2UgY3JlYXRl ZCB0aGUKPiArCSAqIG1tdV9ub3RpZmlmZXJfbW0gaXMgbm90IGZyZWVkIHVudGlsIHRoZSBtbSBp cyBkZXN0cm95ZWQuCj4gIAkgKi8KPiAgCWlmIChtbXVfbm90aWZpZXJfbW0pCj4gLQkJbW0tPm1t dV9ub3RpZmllcl9tbSA9IG1tdV9ub3RpZmllcl9tbTsKPiArCQlzbXBfc3RvcmVfcmVsZWFzZSgm bW0tPm1tdV9ub3RpZmllcl9tbSwgbW11X25vdGlmaWVyX21tKTsKCkkgZG8gbm90IHVuZGVyc3Rh bmQgd2h5IHlvdSBuZWVkIHRoZSByZWxlYXNlIHNlbWFudGljcyBoZXJlLCB3ZQphcmUgdW5kZXIg dGhlIG1tYXBfc2VtIGluIHdyaXRlIG1vZGUgd2hlbiB3ZSByZWxlYXNlIGl0IHRoZSBsb2NrCmJh cnJpZXIgd2lsbCBtYWtlIHN1cmUgYW55b25lIGVsc2Ugc2VlcyB0aGUgbmV3IG1tdV9ub3RpZmll cl9tbQoKSSBmYWlsIHRvIHNlZSB0aGUgYmVuZWZpdCBvciBuZWVkIGZvciByZWxlYXNlL2FjcXVp cmUgc2VtYW50aWNzCmhlcmUuCgo+ICAKPiAtCXNwaW5fbG9jaygmbW0tPm1tdV9ub3RpZmllcl9t bS0+bG9jayk7Cj4gLQlobGlzdF9hZGRfaGVhZF9yY3UoJm1uLT5obGlzdCwgJm1tLT5tbXVfbm90 aWZpZXJfbW0tPmxpc3QpOwo+IC0Jc3Bpbl91bmxvY2soJm1tLT5tbXVfbm90aWZpZXJfbW0tPmxv Y2spOwo+ICsJaWYgKG1uKSB7Cj4gKwkJLyogUGFpcnMgd2l0aCB0aGUgbW1kcm9wIGluIG1tdV9u b3RpZmllcl91bnJlZ2lzdGVyXyogKi8KPiArCQltbWdyYWIobW0pOwo+ICsJCW1uLT5tbSA9IG1t Owo+ICsJCW1uLT51c2VycyA9IDE7Cj4gKwo+ICsJCXNwaW5fbG9jaygmbW0tPm1tdV9ub3RpZmll cl9tbS0+bG9jayk7Cj4gKwkJaGxpc3RfYWRkX2hlYWRfcmN1KCZtbi0+aGxpc3QsICZtbS0+bW11 X25vdGlmaWVyX21tLT5saXN0KTsKPiArCQlzcGluX3VubG9jaygmbW0tPm1tdV9ub3RpZmllcl9t bS0+bG9jayk7Cj4gKwl9IGVsc2UKPiArCQltbS0+bW11X25vdGlmaWVyX21tLT5oYXNfaW50ZXJ2 YWwgPSB0cnVlOwo+ICAKPiAgCW1tX2Ryb3BfYWxsX2xvY2tzKG1tKTsKPiAgCUJVR19PTihhdG9t aWNfcmVhZCgmbW0tPm1tX3VzZXJzKSA8PSAwKTsKPiBAQCAtNTI5LDYgKzg0OCwxNjYgQEAgdm9p ZCBtbXVfbm90aWZpZXJfcHV0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uKQo+ICB9Cj4gIEVYUE9S VF9TWU1CT0xfR1BMKG1tdV9ub3RpZmllcl9wdXQpOwo+ICAKPiArc3RhdGljIGludCBfX21tdV9y YW5nZV9ub3RpZmllcl9pbnNlcnQoc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJuLAo+ICsJ CQkJICAgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsCj4gKwkJCQkgICAgICAgdW5zaWduZWQgbG9u ZyBsZW5ndGgsCj4gKwkJCQkgICAgICAgc3RydWN0IG1tdV9ub3RpZmllcl9tbSAqbW1uX21tLAo+ ICsJCQkJICAgICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQo+ICt7Cj4gKwltcm4tPm1tID0gbW07 Cj4gKwlSQl9DTEVBUl9OT0RFKCZtcm4tPmludGVydmFsX3RyZWUucmIpOwo+ICsJbXJuLT5pbnRl cnZhbF90cmVlLnN0YXJ0ID0gc3RhcnQ7Cj4gKwkvKgo+ICsJICogTm90ZSB0aGF0IHRoZSByZXBy ZXNlbnRhdGlvbiBvZiB0aGUgaW50ZXJ2YWxzIGluIHRoZSBpbnRlcnZhbCB0cmVlCj4gKwkgKiBj b25zaWRlcnMgdGhlIGVuZGluZyBwb2ludCBhcyBjb250YWluZWQgaW4gdGhlIGludGVydmFsLgo+ ICsJICovCj4gKwlpZiAobGVuZ3RoID09IDAgfHwKPiArCSAgICBjaGVja19hZGRfb3ZlcmZsb3co c3RhcnQsIGxlbmd0aCAtIDEsICZtcm4tPmludGVydmFsX3RyZWUubGFzdCkpCj4gKwkJcmV0dXJu IC1FT1ZFUkZMT1c7Cj4gKwo+ICsJLyogcGFpcnMgd2l0aCBtbWRyb3AgaW4gbW11X3JhbmdlX25v dGlmaWVyX3JlbW92ZSgpICovCj4gKwltbWdyYWIobW0pOwo+ICsKPiArCS8qCj4gKwkgKiBJZiBz b21lIGludmFsaWRhdGVfcmFuZ2Vfc3RhcnQvZW5kIHJlZ2lvbiBpcyBnb2luZyBvbiBpbiBwYXJh bGxlbAo+ICsJICogd2UgZG9uJ3Qga25vdyB3aGF0IFZBIHJhbmdlcyBhcmUgYWZmZWN0ZWQsIHNv IHdlIG11c3QgYXNzdW1lIHRoaXMKPiArCSAqIG5ldyByYW5nZSBpcyBpbmNsdWRlZC4KPiArCSAq Cj4gKwkgKiBJZiB0aGUgaXRyZWUgaXMgaW52YWxpZGF0aW5nIHRoZW4gd2UgYXJlIG5vdCBhbGxv d2VkIHRvIGNoYW5nZQo+ICsJICogaXQuIFJldHJ5aW5nIHVudGlsIGludmFsaWRhdGlvbiBpcyBk b25lIGlzIHRyaWNreSBkdWUgdG8gdGhlCj4gKwkgKiBwb3NzaWJpbGl0eSBmb3IgbGl2ZSBsb2Nr LCBpbnN0ZWFkIGRlZmVyIHRoZSBhZGQgdG8gdGhlIHVubG9jayBzbwo+ICsJICogdGhpcyBhbGdv cml0aG0gaXMgZGV0ZXJtaW5pc3RpYy4KPiArCSAqCj4gKwkgKiBJbiBhbGwgY2FzZXMgdGhlIHZh bHVlIGZvciB0aGUgbXJuLT5tcl9pbnZhbGlkYXRlX3NlcSBzaG91bGQgYmUKPiArCSAqIG9kZCwg c2VlIG1tdV9yYW5nZV9yZWFkX2JlZ2luKCkKPiArCSAqLwo+ICsJc3Bpbl9sb2NrKCZtbW5fbW0t PmxvY2spOwo+ICsJaWYgKG1tbl9tbS0+YWN0aXZlX2ludmFsaWRhdGVfcmFuZ2VzKSB7Cj4gKwkJ aWYgKG1uX2l0cmVlX2lzX2ludmFsaWRhdGluZyhtbW5fbW0pKQo+ICsJCQlobGlzdF9hZGRfaGVh ZCgmbXJuLT5kZWZlcnJlZF9pdGVtLAo+ICsJCQkJICAgICAgICZtbW5fbW0tPmRlZmVycmVkX2xp c3QpOwo+ICsJCWVsc2Ugewo+ICsJCQltbW5fbW0tPmludmFsaWRhdGVfc2VxIHw9IDE7Cj4gKwkJ CWludGVydmFsX3RyZWVfaW5zZXJ0KCZtcm4tPmludGVydmFsX3RyZWUsCj4gKwkJCQkJICAgICAm bW1uX21tLT5pdHJlZSk7Cj4gKwkJfQo+ICsJCW1ybi0+aW52YWxpZGF0ZV9zZXEgPSBtbW5fbW0t PmludmFsaWRhdGVfc2VxOwo+ICsJfSBlbHNlIHsKPiArCQlXQVJOX09OKG1uX2l0cmVlX2lzX2lu dmFsaWRhdGluZyhtbW5fbW0pKTsKPiArCQltcm4tPmludmFsaWRhdGVfc2VxID0gbW1uX21tLT5p bnZhbGlkYXRlX3NlcSAtIDE7Cj4gKwkJaW50ZXJ2YWxfdHJlZV9pbnNlcnQoJm1ybi0+aW50ZXJ2 YWxfdHJlZSwgJm1tbl9tbS0+aXRyZWUpOwo+ICsJfQo+ICsJc3Bpbl91bmxvY2soJm1tbl9tbS0+ bG9jayk7Cj4gKwlyZXR1cm4gMDsKPiArfQo+ICsKPiArLyoqCj4gKyAqIG1tdV9yYW5nZV9ub3Rp Zmllcl9pbnNlcnQgLSBJbnNlcnQgYSByYW5nZSBub3RpZmllcgo+ICsgKiBAbXJuOiBSYW5nZSBu b3RpZmllciB0byByZWdpc3Rlcgo+ICsgKiBAc3RhcnQ6IFN0YXJ0aW5nIHZpcnR1YWwgYWRkcmVz cyB0byBtb25pdG9yCj4gKyAqIEBsZW5ndGg6IExlbmd0aCBvZiB0aGUgcmFuZ2UgdG8gbW9uaXRv cgo+ICsgKiBAbW0gOiBtbV9zdHJ1Y3QgdG8gYXR0YWNoIHRvCj4gKyAqCj4gKyAqIFRoaXMgZnVu Y3Rpb24gc3Vic2NyaWJlcyB0aGUgcmFuZ2Ugbm90aWZpZXIgZm9yIG5vdGlmaWNhdGlvbnMgZnJv bSB0aGUgbW0uCj4gKyAqIFVwb24gcmV0dXJuIHRoZSBvcHMgcmVsYXRlZCB0byBtbXVfcmFuZ2Vf bm90aWZpZXIgd2lsbCBiZSBjYWxsZWQgd2hlbmV2ZXIKPiArICogYW4gZXZlbnQgdGhhdCBpbnRl cnNlY3RzIHdpdGggdGhlIGdpdmVuIHJhbmdlIG9jY3Vycy4KPiArICoKPiArICogVXBvbiByZXR1 cm4gdGhlIHJhbmdlX25vdGlmaWVyIG1heSBub3QgYmUgcHJlc2VudCBpbiB0aGUgaW50ZXJ2YWwg dHJlZSB5ZXQuCj4gKyAqIFRoZSBjYWxsZXIgbXVzdCB1c2UgdGhlIG5vcm1hbCByYW5nZSBub3Rp ZmllciBsb2NraW5nIGZsb3cgdmlhCj4gKyAqIG1tdV9yYW5nZV9yZWFkX2JlZ2luKCkgdG8gZXN0 YWJsaXNoIFNQVEVzIGZvciB0aGlzIHJhbmdlLgo+ICsgKi8KPiAraW50IG1tdV9yYW5nZV9ub3Rp Zmllcl9pbnNlcnQoc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJuLAo+ICsJCQkgICAgICB1 bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGxlbmd0aCwKPiArCQkJICAgICAgc3Ry dWN0IG1tX3N0cnVjdCAqbW0pCj4gK3sKPiArCXN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9t bTsKPiArCWludCByZXQ7Cj4gKwo+ICsJbWlnaHRfbG9jaygmbW0tPm1tYXBfc2VtKTsKPiArCj4g KwltbW5fbW0gPSBzbXBfbG9hZF9hY3F1aXJlKCZtbS0+bW11X25vdGlmaWVyX21tKTsKPiArCWlm ICghbW1uX21tIHx8ICFtbW5fbW0tPmhhc19pbnRlcnZhbCkgewo+ICsJCXJldCA9IG1tdV9ub3Rp Zmllcl9yZWdpc3RlcihOVUxMLCBtbSk7Cj4gKwkJaWYgKHJldCkKPiArCQkJcmV0dXJuIHJldDsK PiArCQltbW5fbW0gPSBtbS0+bW11X25vdGlmaWVyX21tOwo+ICsJfQo+ICsJcmV0dXJuIF9fbW11 X3JhbmdlX25vdGlmaWVyX2luc2VydChtcm4sIHN0YXJ0LCBsZW5ndGgsIG1tbl9tbSwgbW0pOwo+ ICt9Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKG1tdV9yYW5nZV9ub3RpZmllcl9pbnNlcnQpOwo+ICsK PiAraW50IG1tdV9yYW5nZV9ub3RpZmllcl9pbnNlcnRfbG9ja2VkKHN0cnVjdCBtbXVfcmFuZ2Vf bm90aWZpZXIgKm1ybiwKPiArCQkJCSAgICAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQg bG9uZyBsZW5ndGgsCj4gKwkJCQkgICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQo+ICt7Cj4gKwlz dHJ1Y3QgbW11X25vdGlmaWVyX21tICptbW5fbW07Cj4gKwlpbnQgcmV0Owo+ICsKPiArCWxvY2tk ZXBfYXNzZXJ0X2hlbGRfd3JpdGUoJm1tLT5tbWFwX3NlbSk7Cj4gKwo+ICsJbW1uX21tID0gbW0t Pm1tdV9ub3RpZmllcl9tbTsKClNob3VsZG4ndCB5b3UgYmUgdXNpbmcgc21wX2xvYWRfYWNxdWly ZSgpID8KCj4gKwlpZiAoIW1tbl9tbSB8fCAhbW1uX21tLT5oYXNfaW50ZXJ2YWwpIHsKPiArCQly ZXQgPSBfX21tdV9ub3RpZmllcl9yZWdpc3RlcihOVUxMLCBtbSk7Cj4gKwkJaWYgKHJldCkKPiAr CQkJcmV0dXJuIHJldDsKPiArCQltbW5fbW0gPSBtbS0+bW11X25vdGlmaWVyX21tOwo+ICsJfQo+ ICsJcmV0dXJuIF9fbW11X3JhbmdlX25vdGlmaWVyX2luc2VydChtcm4sIHN0YXJ0LCBsZW5ndGgs IG1tbl9tbSwgbW0pOwo+ICt9Cj4gK0VYUE9SVF9TWU1CT0xfR1BMKG1tdV9yYW5nZV9ub3RpZmll cl9pbnNlcnRfbG9ja2VkKTsKPiArCj4gKy8qKgo+ICsgKiBtbXVfcmFuZ2Vfbm90aWZpZXJfcmVt b3ZlIC0gUmVtb3ZlIGEgcmFuZ2Ugbm90aWZpZXIKPiArICogQG1ybjogUmFuZ2Ugbm90aWZpZXIg dG8gdW5yZWdpc3Rlcgo+ICsgKgo+ICsgKiBUaGlzIGZ1bmN0aW9uIG11c3QgYmUgcGFpcmVkIHdp dGggbW11X3JhbmdlX25vdGlmaWVyX2luc2VydCgpLiBJdCBjYW5ub3QgYmUKPiArICogY2FsbGVk IGZyb20gYW55IG9wcyBjYWxsYmFjay4KPiArICoKPiArICogT25jZSB0aGlzIHJldHVybnMgb3Bz IGNhbGxiYWNrcyBhcmUgbm8gbG9uZ2VyIHJ1bm5pbmcgb24gb3RoZXIgQ1BVcyBhbmQKPiArICog d2lsbCBub3QgYmUgY2FsbGVkIGluIGZ1dHVyZS4KPiArICovCj4gK3ZvaWQgbW11X3JhbmdlX25v dGlmaWVyX3JlbW92ZShzdHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyICptcm4pCj4gK3sKPiArCXN0 cnVjdCBtbV9zdHJ1Y3QgKm1tID0gbXJuLT5tbTsKPiArCXN0cnVjdCBtbXVfbm90aWZpZXJfbW0g Km1tbl9tbSA9IG1tLT5tbXVfbm90aWZpZXJfbW07Cj4gKwl1bnNpZ25lZCBsb25nIHNlcSA9IDA7 Cj4gKwo+ICsJbWlnaHRfc2xlZXAoKTsKPiArCj4gKwlzcGluX2xvY2soJm1tbl9tbS0+bG9jayk7 Cj4gKwlpZiAobW5faXRyZWVfaXNfaW52YWxpZGF0aW5nKG1tbl9tbSkpIHsKPiArCQkvKgo+ICsJ CSAqIHJlbW92ZSBpcyBiZWluZyBjYWxsZWQgYWZ0ZXIgaW5zZXJ0IHB1dCB0aGlzIG9uIHRoZQo+ ICsJCSAqIGRlZmVycmVkIGxpc3QsIGJ1dCBiZWZvcmUgdGhlIGRlZmVycmVkIGxpc3Qgd2FzIHBy b2Nlc3NlZC4KPiArCQkgKi8KPiArCQlpZiAoUkJfRU1QVFlfTk9ERSgmbXJuLT5pbnRlcnZhbF90 cmVlLnJiKSkgewo+ICsJCQlobGlzdF9kZWwoJm1ybi0+ZGVmZXJyZWRfaXRlbSk7Cj4gKwkJfSBl bHNlIHsKPiArCQkJaGxpc3RfYWRkX2hlYWQoJm1ybi0+ZGVmZXJyZWRfaXRlbSwKPiArCQkJCSAg ICAgICAmbW1uX21tLT5kZWZlcnJlZF9saXN0KTsKPiArCQkJc2VxID0gbW1uX21tLT5pbnZhbGlk YXRlX3NlcTsKPiArCQl9Cj4gKwl9IGVsc2Ugewo+ICsJCVdBUk5fT04oUkJfRU1QVFlfTk9ERSgm bXJuLT5pbnRlcnZhbF90cmVlLnJiKSk7Cj4gKwkJaW50ZXJ2YWxfdHJlZV9yZW1vdmUoJm1ybi0+ aW50ZXJ2YWxfdHJlZSwgJm1tbl9tbS0+aXRyZWUpOwo+ICsJfQo+ICsJc3Bpbl91bmxvY2soJm1t bl9tbS0+bG9jayk7Cj4gKwo+ICsJLyoKPiArCSAqIFRoZSBwb3NzaWJsZSBzbGVlcCBvbiBwcm9n cmVzcyBpbiB0aGUgaW52YWxpZGF0aW9uIHJlcXVpcmVzIHRoZQo+ICsJICogY2FsbGVyIG5vdCBo b2xkIGFueSBsb2NrcyBoZWxkIGJ5IGludmFsaWRhdGlvbiBjYWxsYmFja3MuCj4gKwkgKi8KPiAr CWxvY2tfbWFwX2FjcXVpcmUoJl9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRf bWFwKTsKPiArCWxvY2tfbWFwX3JlbGVhc2UoJl9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFu Z2Vfc3RhcnRfbWFwKTsKPiArCWlmIChzZXEpCj4gKwkJd2FpdF9ldmVudChtbW5fbW0tPndxLAo+ ICsJCQkgICBSRUFEX09OQ0UobW1uX21tLT5pbnZhbGlkYXRlX3NlcSkgIT0gc2VxKTsKPiArCj4g KwkvKiBwYWlycyB3aXRoIG1tZ3JhYiBpbiBtbXVfcmFuZ2Vfbm90aWZpZXJfaW5zZXJ0KCkgKi8K PiArCW1tZHJvcChtbSk7Cj4gK30KPiArRVhQT1JUX1NZTUJPTF9HUEwobW11X3JhbmdlX25vdGlm aWVyX3JlbW92ZSk7Cj4gKwo+ICAvKioKPiAgICogbW11X25vdGlmaWVyX3N5bmNocm9uaXplIC0g RW5zdXJlIGFsbCBtbXVfbm90aWZpZXJzIGFyZSBmcmVlZAo+ICAgKgo+IC0tIAo+IDIuMjMuMAo+ IAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRl dmVsIG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8v bGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A28BCCA9EAF for ; Mon, 21 Oct 2019 18:31:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5B3892084C for ; Mon, 21 Oct 2019 18:31:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CS9L4JD/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728375AbfJUSbH (ORCPT ); Mon, 21 Oct 2019 14:31:07 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:53786 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727110AbfJUSbH (ORCPT ); Mon, 21 Oct 2019 14:31:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1571682665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LWzhTBYpGYPdSV91muDnmc/gB8n5LQ/uQ1I3j7xBs/s=; b=CS9L4JD/2HVI5THPLBsAr2QNwojiEtDDSILF1FEmLpfoOp1otJHwnT8XW1hb69tkhZ02s2 /opMF4u9/F8zXreD6EvY0f+Hm+2GiHeQ8ll+5Nw4erZj1YKb65NCCujG1qMSs7eeq6obtf 1LZ8sezoY9u7D9Iu9GIA8muQgyiBjB4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-360-XIrI2mA2OKecqcAoSJUmGQ-1; Mon, 21 Oct 2019 14:31:01 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A9A2980183E; Mon, 21 Oct 2019 18:30:59 +0000 (UTC) Received: from redhat.com (unknown [10.20.6.178]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 532FA194BE; Mon, 21 Oct 2019 18:30:58 +0000 (UTC) Date: Mon, 21 Oct 2019 14:30:56 -0400 From: Jerome Glisse To: Jason Gunthorpe Cc: Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Ben Skeggs , Jason Gunthorpe , Michal Hocko Subject: Re: [PATCH hmm 02/15] mm/mmu_notifier: add an interval tree notifier Message-ID: <20191021183056.GA3177@redhat.com> References: <20191015181242.8343-1-jgg@ziepe.ca> <20191015181242.8343-3-jgg@ziepe.ca> MIME-Version: 1.0 In-Reply-To: <20191015181242.8343-3-jgg@ziepe.ca> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: XIrI2mA2OKecqcAoSJUmGQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Tue, Oct 15, 2019 at 03:12:29PM -0300, Jason Gunthorpe wrote: > From: Jason Gunthorpe >=20 > Of the 13 users of mmu_notifiers, 8 of them use only > invalidate_range_start/end() and immediately intersect the > mmu_notifier_range with some kind of internal list of VAs. 4 use an > interval tree (i915_gem, radeon_mn, umem_odp, hfi1). 4 use a linked list > of some kind (scif_dma, vhost, gntdev, hmm) >=20 > And the remaining 5 either don't use invalidate_range_start() or do some > special thing with it. >=20 > It turns out that building a correct scheme with an interval tree is > pretty complicated, particularly if the use case is synchronizing against > another thread doing get_user_pages(). Many of these implementations hav= e > various subtle and difficult to fix races. >=20 > This approach puts the interval tree as common code at the top of the mmu > notifier call tree and implements a shareable locking scheme. >=20 > It includes: > - An interval tree tracking VA ranges, with per-range callbacks > - A read/write locking scheme for the interval tree that avoids > sleeping in the notifier path (for OOM killer) > - A sequence counter based collision-retry locking scheme to tell > device page fault that a VA range is being concurrently invalidated. >=20 > This is based on various ideas: > - hmm accumulates invalidated VA ranges and releases them when all > invalidates are done, via active_invalidate_ranges count. > This approach avoids having to intersect the interval tree twice (as > umem_odp does) at the potential cost of a longer device page fault. >=20 > - kvm/umem_odp use a sequence counter to drive the collision retry, > via invalidate_seq >=20 > - a deferred work todo list on unlock scheme like RTNL, via deferred_list= . > This makes adding/removing interval tree members more deterministic >=20 > - seqlock, except this version makes the seqlock idea multi-holder on the > write side by protecting it with active_invalidate_ranges and a spinloc= k >=20 > To minimize MM overhead when only the interval tree is being used, the > entire SRCU and hlist overheads are dropped using some simple > branches. Similarly the interval tree overhead is dropped when in hlist > mode. >=20 > The overhead from the mandatory spinlock is broadly the same as most of > existing users which already had a lock (or two) of some sort on the > invalidation path. >=20 > Cc: Andrea Arcangeli > Cc: Michal Hocko > Signed-off-by: Jason Gunthorpe > --- > include/linux/mmu_notifier.h | 78 ++++++ > mm/Kconfig | 1 + > mm/mmu_notifier.c | 529 +++++++++++++++++++++++++++++++++-- > 3 files changed, 583 insertions(+), 25 deletions(-) >=20 > diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h > index 12bd603d318ce7..bc2b12483de127 100644 > --- a/include/linux/mmu_notifier.h > +++ b/include/linux/mmu_notifier.h > @@ -6,10 +6,12 @@ > #include > #include > #include > +#include > =20 > struct mmu_notifier_mm; > struct mmu_notifier; > struct mmu_notifier_range; > +struct mmu_range_notifier; > =20 > /** > * enum mmu_notifier_event - reason for the mmu notifier callback > @@ -32,6 +34,9 @@ struct mmu_notifier_range; > * access flags). User should soft dirty the page in the end callback to= make > * sure that anyone relying on soft dirtyness catch pages that might be = written > * through non CPU mappings. > + * > + * @MMU_NOTIFY_RELEASE: used during mmu_range_notifier invalidate to sig= nal that > + * the mm refcount is zero and the range is no longer accessible. > */ > enum mmu_notifier_event { > =09MMU_NOTIFY_UNMAP =3D 0, > @@ -39,6 +44,7 @@ enum mmu_notifier_event { > =09MMU_NOTIFY_PROTECTION_VMA, > =09MMU_NOTIFY_PROTECTION_PAGE, > =09MMU_NOTIFY_SOFT_DIRTY, > +=09MMU_NOTIFY_RELEASE, > }; > =20 > #define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) > @@ -222,6 +228,25 @@ struct mmu_notifier { > =09unsigned int users; > }; > =20 > +/** > + * struct mmu_range_notifier_ops > + * @invalidate: Upon return the caller must stop using any SPTEs within = this > + * range, this function can sleep. Return false if blocking= was > + * required but range is non-blocking > + */ > +struct mmu_range_notifier_ops { > +=09bool (*invalidate)(struct mmu_range_notifier *mrn, > +=09=09=09 const struct mmu_notifier_range *range); > +}; > + > +struct mmu_range_notifier { > +=09struct interval_tree_node interval_tree; > +=09const struct mmu_range_notifier_ops *ops; > +=09struct hlist_node deferred_item; > +=09unsigned long invalidate_seq; > +=09struct mm_struct *mm; > +}; > + > #ifdef CONFIG_MMU_NOTIFIER > =20 > #ifdef CONFIG_LOCKDEP > @@ -263,6 +288,59 @@ extern int __mmu_notifier_register(struct mmu_notifi= er *mn, > =09=09=09=09 struct mm_struct *mm); > extern void mmu_notifier_unregister(struct mmu_notifier *mn, > =09=09=09=09 struct mm_struct *mm); > + > +unsigned long mmu_range_read_begin(struct mmu_range_notifier *mrn); > +int mmu_range_notifier_insert(struct mmu_range_notifier *mrn, > +=09=09=09 unsigned long start, unsigned long length, > +=09=09=09 struct mm_struct *mm); > +int mmu_range_notifier_insert_locked(struct mmu_range_notifier *mrn, > +=09=09=09=09 unsigned long start, unsigned long length, > +=09=09=09=09 struct mm_struct *mm); > +void mmu_range_notifier_remove(struct mmu_range_notifier *mrn); > + > +/** > + * mmu_range_read_retry - End a read side critical section against a VA = range > + * mrn: The range under lock > + * seq: The return of the paired mmu_range_read_begin() > + * > + * This MUST be called under a user provided lock that is also held > + * unconditionally by op->invalidate(). That lock provides the required = SMP > + * barrier for handling invalidate_seq. > + * > + * Each call should be paired with a single mmu_range_read_begin() and > + * should be used to conclude the read side. > + * > + * Returns true if an invalidation collided with this critical section, = and > + * the caller should retry. > + */ > +static inline bool mmu_range_read_retry(struct mmu_range_notifier *mrn, > +=09=09=09=09=09unsigned long seq) > +{ > +=09return READ_ONCE(mrn->invalidate_seq) !=3D seq; > +} What about calling this mmu_range_read_end() instead ? To match with the mmu_range_read_begin(). > + > +/** > + * mmu_range_check_retry - Test if a collision has occurred > + * mrn: The range under lock > + * seq: The return of the matching mmu_range_read_begin() > + * > + * This can be used in the critical section between mmu_range_read_begin= () and > + * mmu_range_read_retry(). A return of true indicates an invalidation h= as > + * collided with this lock and a future mmu_range_read_retry() will retu= rn > + * true. > + * > + * False is not reliable and only suggests a collision has not happened.= It > + * can be called many times and does not have to hold the user provided = lock. > + * > + * This call can be used as part of loops and other expensive operations= to > + * expedite a retry. > + */ > +static inline bool mmu_range_check_retry(struct mmu_range_notifier *mrn, > +=09=09=09=09=09 unsigned long seq) > +{ > +=09return READ_ONCE(mrn->invalidate_seq) !=3D seq; > +} > + > extern void __mmu_notifier_mm_destroy(struct mm_struct *mm); > extern void __mmu_notifier_release(struct mm_struct *mm); > extern int __mmu_notifier_clear_flush_young(struct mm_struct *mm, > diff --git a/mm/Kconfig b/mm/Kconfig > index a5dae9a7eb510a..d0b5046d9aeffd 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -284,6 +284,7 @@ config VIRT_TO_BUS > config MMU_NOTIFIER > =09bool > =09select SRCU > +=09select INTERVAL_TREE > =20 > config KSM > =09bool "Enable KSM for page merging" > diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c > index 367670cfd02b7b..5e5e75ebcde4af 100644 > --- a/mm/mmu_notifier.c > +++ b/mm/mmu_notifier.c > @@ -12,6 +12,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -36,10 +37,243 @@ struct lockdep_map __mmu_notifier_invalidate_range_s= tart_map =3D { > struct mmu_notifier_mm { > =09/* all mmu notifiers registered in this mm are queued in this list */ > =09struct hlist_head list; > +=09bool has_interval; > =09/* to serialize the list modifications and hlist_unhashed */ > =09spinlock_t lock; > +=09unsigned long invalidate_seq; > +=09unsigned long active_invalidate_ranges; > +=09struct rb_root_cached itree; > +=09wait_queue_head_t wq; > +=09struct hlist_head deferred_list; > }; > =20 > +/* > + * This is a collision-retry read-side/write-side 'lock', a lot like a > + * seqcount, however this allows multiple write-sides to hold it at > + * once. Conceptually the write side is protecting the values of the PTE= s in > + * this mm, such that PTES cannot be read into SPTEs while any writer ex= ists. > + * > + * Note that the core mm creates nested invalidate_range_start()/end() r= egions > + * within the same thread, and runs invalidate_range_start()/end() in pa= rallel > + * on multiple CPUs. This is designed to not reduce concurrency or block > + * progress on the mm side. > + * > + * As a secondary function, holding the full write side also serves to p= revent > + * writers for the itree, this is an optimization to avoid extra locking > + * during invalidate_range_start/end notifiers. > + * > + * The write side has two states, fully excluded: > + * - mm->active_invalidate_ranges !=3D 0 > + * - mnn->invalidate_seq & 1 =3D=3D True > + * - some range on the mm_struct is being invalidated > + * - the itree is not allowed to change > + * > + * And partially excluded: > + * - mm->active_invalidate_ranges !=3D 0 > + * - some range on the mm_struct is being invalidated > + * - the itree is allowed to change > + * > + * The later state avoids some expensive work on inv_end in the common c= ase of > + * no mrn monitoring the VA. > + */ > +static bool mn_itree_is_invalidating(struct mmu_notifier_mm *mmn_mm) > +{ > +=09lockdep_assert_held(&mmn_mm->lock); > +=09return mmn_mm->invalidate_seq & 1; > +} > + > +static struct mmu_range_notifier * > +mn_itree_inv_start_range(struct mmu_notifier_mm *mmn_mm, > +=09=09=09 const struct mmu_notifier_range *range, > +=09=09=09 unsigned long *seq) > +{ > +=09struct interval_tree_node *node; > +=09struct mmu_range_notifier *res =3D NULL; > + > +=09spin_lock(&mmn_mm->lock); > +=09mmn_mm->active_invalidate_ranges++; > +=09node =3D interval_tree_iter_first(&mmn_mm->itree, range->start, > +=09=09=09=09=09range->end - 1); > +=09if (node) { > +=09=09mmn_mm->invalidate_seq |=3D 1; > +=09=09res =3D container_of(node, struct mmu_range_notifier, > +=09=09=09=09 interval_tree); > +=09} > + > +=09*seq =3D mmn_mm->invalidate_seq; > +=09spin_unlock(&mmn_mm->lock); > +=09return res; > +} > + > +static struct mmu_range_notifier * > +mn_itree_inv_next(struct mmu_range_notifier *mrn, > +=09=09 const struct mmu_notifier_range *range) > +{ > +=09struct interval_tree_node *node; > + > +=09node =3D interval_tree_iter_next(&mrn->interval_tree, range->start, > +=09=09=09=09 range->end - 1); > +=09if (!node) > +=09=09return NULL; > +=09return container_of(node, struct mmu_range_notifier, interval_tree); > +} > + > +static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm) > +{ > +=09struct mmu_range_notifier *mrn; > +=09struct hlist_node *next; > +=09bool need_wake =3D false; > + > +=09spin_lock(&mmn_mm->lock); > +=09if (--mmn_mm->active_invalidate_ranges || > +=09 !mn_itree_is_invalidating(mmn_mm)) { > +=09=09spin_unlock(&mmn_mm->lock); > +=09=09return; > +=09} > + > +=09mmn_mm->invalidate_seq++; > +=09need_wake =3D true; > + > +=09/* > +=09 * The inv_end incorporates a deferred mechanism like rtnl. Adds and The rtnl reference is lost on people unfamiliar with the network :) code maybe like rtnl_lock()/rtnl_unlock() so people have a chance to grep the right function. Assuming i am myself getting the right reference :) > +=09 * removes are queued until the final inv_end happens then they are > +=09 * progressed. This arrangement for tree updates is used to avoid > +=09 * using a blocking lock during invalidate_range_start. > +=09 */ > +=09hlist_for_each_entry_safe(mrn, next, &mmn_mm->deferred_list, > +=09=09=09=09 deferred_item) { > +=09=09if (RB_EMPTY_NODE(&mrn->interval_tree.rb)) > +=09=09=09interval_tree_insert(&mrn->interval_tree, > +=09=09=09=09=09 &mmn_mm->itree); > +=09=09else > +=09=09=09interval_tree_remove(&mrn->interval_tree, > +=09=09=09=09=09 &mmn_mm->itree); > +=09=09hlist_del(&mrn->deferred_item); > +=09} > +=09spin_unlock(&mmn_mm->lock); > + > +=09/* > +=09 * TODO: Since we already have a spinlock above, this would be faster > +=09 * as wake_up_q > +=09 */ > +=09if (need_wake) > +=09=09wake_up_all(&mmn_mm->wq); > +} > + > +/** > + * mmu_range_read_begin - Begin a read side critical section against a V= A range > + * mrn: The range to lock > + * > + * mmu_range_read_begin()/mmu_range_read_retry() implement a collision-r= etry > + * locking scheme similar to seqcount for the VA range under mrn. If the= mm > + * invokes invalidation during the critical section then > + * mmu_range_read_retry() will return true. > + * > + * This is useful to obtain shadow PTEs where teardown or setup of the S= PTEs > + * require a blocking context. The critical region formed by this lock = can > + * sleep, and the required 'user_lock' can also be a sleeping lock. > + * > + * The caller is required to provide a 'user_lock' to serialize both tea= rdown > + * and setup. > + * > + * The return value should be passed to mmu_range_read_retry(). > + */ > +unsigned long mmu_range_read_begin(struct mmu_range_notifier *mrn) > +{ > +=09struct mmu_notifier_mm *mmn_mm =3D mrn->mm->mmu_notifier_mm; > +=09unsigned long seq; > +=09bool is_invalidating; > + > +=09/* > +=09 * If the mrn has a different seq value under the user_lock than we > +=09 * started with then it has collided. > +=09 * > +=09 * If the mrn currently has the same seq value as the mmn_mm seq, the= n > +=09 * it is currently between invalidate_start/end and is colliding. > +=09 * > +=09 * The locking looks broadly like this: > +=09 * mn_tree_invalidate_start(): mmu_range_read_begin(): > +=09 * spin_lock > +=09 * seq =3D READ_ONCE(mrn->in= validate_seq); > +=09 * seq =3D=3D mmn_mm->invali= date_seq > +=09 * spin_unlock > +=09 * spin_lock > +=09 * seq =3D ++mmn_mm->invalidate_seq > +=09 * spin_unlock > +=09 * mrn->invalidate_seq =3D seq > +=09 * op->invalidate_range(): > +=09 * user_lock > +=09 * user_unlock > +=09 * > +=09 * [Required: mmu_range_read_retry() =3D=3D = true] > +=09 * > +=09 * mn_itree_inv_end(): > +=09 * spin_lock > +=09 * seq =3D ++mmn_mm->invalidate_seq > +=09 * spin_unlock > +=09 * > +=09 * user_lock > +=09 * mmu_range_read_retry(): > +=09 * READ_ONCE(mrn->invalidate= _seq) !=3D seq > +=09 * user_unlock > +=09 * > +=09 * Logically mrn->invalidate_seq is locked under the user provided > +=09 * lock, however the write is placed before that lock due to the way > +=09 * the API is layered. > +=09 * > +=09 * Barriers are not needed as any races here are closed by an eventua= l > +=09 * mmu_range_read_retry(), which provides a barrier via the user_lock= . > +=09 */ > +=09spin_lock(&mmn_mm->lock); > +=09seq =3D READ_ONCE(mrn->invalidate_seq); > +=09is_invalidating =3D seq =3D=3D mmn_mm->invalidate_seq; > +=09spin_unlock(&mmn_mm->lock); > + > +=09/* > +=09 * mrn->invalidate_seq is always set to an odd value. This ensures > +=09 * that if seq does wrap we will always clear the below sleep in some > +=09 * reasonable time as mmn_mm->invalidate_seq is even in the idle > +=09 * state. I think this comment should be with the struct mmu_range_notifier definition and you should just point to it from here as the same comment would be useful down below. > +=09 */ > +=09lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); > +=09lock_map_release(&__mmu_notifier_invalidate_range_start_map); > +=09if (is_invalidating) > +=09=09wait_event(mmn_mm->wq, > +=09=09=09 READ_ONCE(mmn_mm->invalidate_seq) !=3D seq); > + > +=09/* > +=09 * Notice that mmu_range_read_retry() can already be true at this > +=09 * point, avoiding loops here allows the user of this lock to provide > +=09 * a global time bound. > +=09 */ > + > +=09return seq; > +} > +EXPORT_SYMBOL_GPL(mmu_range_read_begin); > + > +static void mn_itree_release(struct mmu_notifier_mm *mmn_mm, > +=09=09=09 struct mm_struct *mm) > +{ > +=09struct mmu_notifier_range range =3D { > +=09=09.flags =3D MMU_NOTIFIER_RANGE_BLOCKABLE, > +=09=09.event =3D MMU_NOTIFY_RELEASE, > +=09=09.mm =3D mm, > +=09=09.start =3D 0, > +=09=09.end =3D ULONG_MAX, > +=09}; > +=09struct mmu_range_notifier *mrn; > +=09unsigned long cur_seq; > +=09bool ret; > + > +=09for (mrn =3D mn_itree_inv_start_range(mmn_mm, &range, &cur_seq); mrn; > +=09 mrn =3D mn_itree_inv_next(mrn, &range)) { > +=09=09ret =3D mrn->ops->invalidate(mrn, &range); > +=09=09WARN_ON(ret); > +=09} > + > +=09mn_itree_inv_end(mmn_mm); > +} > + > /* > * This function can't run concurrently against mmu_notifier_register > * because mm->mm_users > 0 during mmu_notifier_register and exit_mmap > @@ -52,17 +286,24 @@ struct mmu_notifier_mm { > * can't go away from under us as exit_mmap holds an mm_count pin > * itself. > */ > -void __mmu_notifier_release(struct mm_struct *mm) > +static void mn_hlist_release(struct mmu_notifier_mm *mmn_mm, > +=09=09=09 struct mm_struct *mm) > { > =09struct mmu_notifier *mn; > =09int id; > =20 > +=09if (mmn_mm->has_interval) > +=09=09mn_itree_release(mmn_mm, mm); > + > +=09if (hlist_empty(&mmn_mm->list)) > +=09=09return; > + > =09/* > =09 * SRCU here will block mmu_notifier_unregister until > =09 * ->release returns. > =09 */ > =09id =3D srcu_read_lock(&srcu); > -=09hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) > +=09hlist_for_each_entry_rcu(mn, &mmn_mm->list, hlist) > =09=09/* > =09=09 * If ->release runs before mmu_notifier_unregister it must be > =09=09 * handled, as it's the only way for the driver to flush all > @@ -72,9 +313,9 @@ void __mmu_notifier_release(struct mm_struct *mm) > =09=09if (mn->ops->release) > =09=09=09mn->ops->release(mn, mm); > =20 > -=09spin_lock(&mm->mmu_notifier_mm->lock); > -=09while (unlikely(!hlist_empty(&mm->mmu_notifier_mm->list))) { > -=09=09mn =3D hlist_entry(mm->mmu_notifier_mm->list.first, > +=09spin_lock(&mmn_mm->lock); > +=09while (unlikely(!hlist_empty(&mmn_mm->list))) { > +=09=09mn =3D hlist_entry(mmn_mm->list.first, > =09=09=09=09 struct mmu_notifier, > =09=09=09=09 hlist); > =09=09/* > @@ -85,7 +326,7 @@ void __mmu_notifier_release(struct mm_struct *mm) > =09=09 */ > =09=09hlist_del_init_rcu(&mn->hlist); > =09} > -=09spin_unlock(&mm->mmu_notifier_mm->lock); > +=09spin_unlock(&mmn_mm->lock); > =09srcu_read_unlock(&srcu, id); > =20 > =09/* > @@ -100,6 +341,17 @@ void __mmu_notifier_release(struct mm_struct *mm) > =09synchronize_srcu(&srcu); > } > =20 > +void __mmu_notifier_release(struct mm_struct *mm) > +{ > +=09struct mmu_notifier_mm *mmn_mm =3D mm->mmu_notifier_mm; > + > +=09if (mmn_mm->has_interval) > +=09=09mn_itree_release(mmn_mm, mm); > + > +=09if (!hlist_empty(&mmn_mm->list)) > +=09=09mn_hlist_release(mmn_mm, mm); > +} > + > /* > * If no young bitflag is supported by the hardware, ->clear_flush_young= can > * unmap the address and return 1 or 0 depending if the mapping previous= ly > @@ -172,14 +424,41 @@ void __mmu_notifier_change_pte(struct mm_struct *mm= , unsigned long address, > =09srcu_read_unlock(&srcu, id); > } > =20 > -int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *ran= ge) > +static int mn_itree_invalidate(struct mmu_notifier_mm *mmn_mm, > +=09=09=09=09 const struct mmu_notifier_range *range) > +{ > +=09struct mmu_range_notifier *mrn; > +=09unsigned long cur_seq; > + > +=09for (mrn =3D mn_itree_inv_start_range(mmn_mm, range, &cur_seq); mrn; > +=09 mrn =3D mn_itree_inv_next(mrn, range)) { > +=09=09bool ret; > + > +=09=09WRITE_ONCE(mrn->invalidate_seq, cur_seq); > +=09=09ret =3D mrn->ops->invalidate(mrn, range); > +=09=09if (!ret && !WARN_ON(mmu_notifier_range_blockable(range))) Isn't the logic wrong here ? We want to warn if the range was mark as blockable and invalidate returned false. Also we went to backoff no matter what if the invalidate return false ie: if (!ret) { WARN_ON(mmu_notifier_range_blockable(range))) goto out_would_block; } > +=09=09=09goto out_would_block; > +=09} > +=09return 0; > + > +out_would_block: > +=09/* > +=09 * On -EAGAIN the non-blocking caller is not allowed to call > +=09 * invalidate_range_end() > +=09 */ > +=09mn_itree_inv_end(mmn_mm); > +=09return -EAGAIN; > +} > + > +static int mn_hlist_invalidate_range_start(struct mmu_notifier_mm *mmn_m= m, > +=09=09=09=09=09 struct mmu_notifier_range *range) > { > =09struct mmu_notifier *mn; > =09int ret =3D 0; > =09int id; > =20 > =09id =3D srcu_read_lock(&srcu); > -=09hlist_for_each_entry_rcu(mn, &range->mm->mmu_notifier_mm->list, hlist= ) { > +=09hlist_for_each_entry_rcu(mn, &mmn_mm->list, hlist) { > =09=09if (mn->ops->invalidate_range_start) { > =09=09=09int _ret; > =20 > @@ -203,15 +482,30 @@ int __mmu_notifier_invalidate_range_start(struct mm= u_notifier_range *range) > =09return ret; > } > =20 > -void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *rang= e, > -=09=09=09=09=09 bool only_end) > +int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *ran= ge) > +{ > +=09struct mmu_notifier_mm *mmn_mm =3D range->mm->mmu_notifier_mm; > +=09int ret =3D 0; > + > +=09if (mmn_mm->has_interval) { > +=09=09ret =3D mn_itree_invalidate(mmn_mm, range); > +=09=09if (ret) > +=09=09=09return ret; > +=09} > +=09if (!hlist_empty(&mmn_mm->list)) > +=09=09return mn_hlist_invalidate_range_start(mmn_mm, range); > +=09return 0; > +} > + > +static void mn_hlist_invalidate_end(struct mmu_notifier_mm *mmn_mm, > +=09=09=09=09 struct mmu_notifier_range *range, > +=09=09=09=09 bool only_end) > { > =09struct mmu_notifier *mn; > =09int id; > =20 > -=09lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); > =09id =3D srcu_read_lock(&srcu); > -=09hlist_for_each_entry_rcu(mn, &range->mm->mmu_notifier_mm->list, hlist= ) { > +=09hlist_for_each_entry_rcu(mn, &mmn_mm->list, hlist) { > =09=09/* > =09=09 * Call invalidate_range here too to avoid the need for the > =09=09 * subsystem of having to register an invalidate_range_end > @@ -238,6 +532,19 @@ void __mmu_notifier_invalidate_range_end(struct mmu_= notifier_range *range, > =09=09} > =09} > =09srcu_read_unlock(&srcu, id); > +} > + > +void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *rang= e, > +=09=09=09=09=09 bool only_end) > +{ > +=09struct mmu_notifier_mm *mmn_mm =3D range->mm->mmu_notifier_mm; > + > +=09lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); > +=09if (mmn_mm->has_interval) > +=09=09mn_itree_inv_end(mmn_mm); > + > +=09if (!hlist_empty(&mmn_mm->list)) > +=09=09mn_hlist_invalidate_end(mmn_mm, range, only_end); > =09lock_map_release(&__mmu_notifier_invalidate_range_start_map); > } > =20 > @@ -256,8 +563,9 @@ void __mmu_notifier_invalidate_range(struct mm_struct= *mm, > } > =20 > /* > - * Same as mmu_notifier_register but here the caller must hold the > - * mmap_sem in write mode. > + * Same as mmu_notifier_register but here the caller must hold the mmap_= sem in > + * write mode. A NULL mn signals the notifier is being registered for it= ree > + * mode. > */ > int __mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *m= m) > { > @@ -274,9 +582,6 @@ int __mmu_notifier_register(struct mmu_notifier *mn, = struct mm_struct *mm) > =09=09fs_reclaim_release(GFP_KERNEL); > =09} > =20 > -=09mn->mm =3D mm; > -=09mn->users =3D 1; > - > =09if (!mm->mmu_notifier_mm) { > =09=09/* > =09=09 * kmalloc cannot be called under mm_take_all_locks(), but we > @@ -284,21 +589,22 @@ int __mmu_notifier_register(struct mmu_notifier *mn= , struct mm_struct *mm) > =09=09 * the write side of the mmap_sem. > =09=09 */ > =09=09mmu_notifier_mm =3D > -=09=09=09kmalloc(sizeof(struct mmu_notifier_mm), GFP_KERNEL); > +=09=09=09kzalloc(sizeof(struct mmu_notifier_mm), GFP_KERNEL); > =09=09if (!mmu_notifier_mm) > =09=09=09return -ENOMEM; > =20 > =09=09INIT_HLIST_HEAD(&mmu_notifier_mm->list); > =09=09spin_lock_init(&mmu_notifier_mm->lock); > +=09=09mmu_notifier_mm->invalidate_seq =3D 2; Why starting at 2 ? > +=09=09mmu_notifier_mm->itree =3D RB_ROOT_CACHED; > +=09=09init_waitqueue_head(&mmu_notifier_mm->wq); > +=09=09INIT_HLIST_HEAD(&mmu_notifier_mm->deferred_list); > =09} > =20 > =09ret =3D mm_take_all_locks(mm); > =09if (unlikely(ret)) > =09=09goto out_clean; > =20 > -=09/* Pairs with the mmdrop in mmu_notifier_unregister_* */ > -=09mmgrab(mm); > - > =09/* > =09 * Serialize the update against mmu_notifier_unregister. A > =09 * side note: mmu_notifier_release can't run concurrently with > @@ -306,13 +612,26 @@ int __mmu_notifier_register(struct mmu_notifier *mn= , struct mm_struct *mm) > =09 * current->mm or explicitly with get_task_mm() or similar). > =09 * We can't race against any other mmu notifier method either > =09 * thanks to mm_take_all_locks(). > +=09 * > +=09 * release semantics are provided for users not inside a lock covered > +=09 * by mm_take_all_locks(). acquire can only be used while holding the > +=09 * mmgrab or mmget, and is safe because once created the > +=09 * mmu_notififer_mm is not freed until the mm is destroyed. > =09 */ > =09if (mmu_notifier_mm) > -=09=09mm->mmu_notifier_mm =3D mmu_notifier_mm; > +=09=09smp_store_release(&mm->mmu_notifier_mm, mmu_notifier_mm); I do not understand why you need the release semantics here, we are under the mmap_sem in write mode when we release it the lock barrier will make sure anyone else sees the new mmu_notifier_mm I fail to see the benefit or need for release/acquire semantics here. > =20 > -=09spin_lock(&mm->mmu_notifier_mm->lock); > -=09hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_mm->list); > -=09spin_unlock(&mm->mmu_notifier_mm->lock); > +=09if (mn) { > +=09=09/* Pairs with the mmdrop in mmu_notifier_unregister_* */ > +=09=09mmgrab(mm); > +=09=09mn->mm =3D mm; > +=09=09mn->users =3D 1; > + > +=09=09spin_lock(&mm->mmu_notifier_mm->lock); > +=09=09hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_mm->list); > +=09=09spin_unlock(&mm->mmu_notifier_mm->lock); > +=09} else > +=09=09mm->mmu_notifier_mm->has_interval =3D true; > =20 > =09mm_drop_all_locks(mm); > =09BUG_ON(atomic_read(&mm->mm_users) <=3D 0); > @@ -529,6 +848,166 @@ void mmu_notifier_put(struct mmu_notifier *mn) > } > EXPORT_SYMBOL_GPL(mmu_notifier_put); > =20 > +static int __mmu_range_notifier_insert(struct mmu_range_notifier *mrn, > +=09=09=09=09 unsigned long start, > +=09=09=09=09 unsigned long length, > +=09=09=09=09 struct mmu_notifier_mm *mmn_mm, > +=09=09=09=09 struct mm_struct *mm) > +{ > +=09mrn->mm =3D mm; > +=09RB_CLEAR_NODE(&mrn->interval_tree.rb); > +=09mrn->interval_tree.start =3D start; > +=09/* > +=09 * Note that the representation of the intervals in the interval tree > +=09 * considers the ending point as contained in the interval. > +=09 */ > +=09if (length =3D=3D 0 || > +=09 check_add_overflow(start, length - 1, &mrn->interval_tree.last)) > +=09=09return -EOVERFLOW; > + > +=09/* pairs with mmdrop in mmu_range_notifier_remove() */ > +=09mmgrab(mm); > + > +=09/* > +=09 * If some invalidate_range_start/end region is going on in parallel > +=09 * we don't know what VA ranges are affected, so we must assume this > +=09 * new range is included. > +=09 * > +=09 * If the itree is invalidating then we are not allowed to change > +=09 * it. Retrying until invalidation is done is tricky due to the > +=09 * possibility for live lock, instead defer the add to the unlock so > +=09 * this algorithm is deterministic. > +=09 * > +=09 * In all cases the value for the mrn->mr_invalidate_seq should be > +=09 * odd, see mmu_range_read_begin() > +=09 */ > +=09spin_lock(&mmn_mm->lock); > +=09if (mmn_mm->active_invalidate_ranges) { > +=09=09if (mn_itree_is_invalidating(mmn_mm)) > +=09=09=09hlist_add_head(&mrn->deferred_item, > +=09=09=09=09 &mmn_mm->deferred_list); > +=09=09else { > +=09=09=09mmn_mm->invalidate_seq |=3D 1; > +=09=09=09interval_tree_insert(&mrn->interval_tree, > +=09=09=09=09=09 &mmn_mm->itree); > +=09=09} > +=09=09mrn->invalidate_seq =3D mmn_mm->invalidate_seq; > +=09} else { > +=09=09WARN_ON(mn_itree_is_invalidating(mmn_mm)); > +=09=09mrn->invalidate_seq =3D mmn_mm->invalidate_seq - 1; > +=09=09interval_tree_insert(&mrn->interval_tree, &mmn_mm->itree); > +=09} > +=09spin_unlock(&mmn_mm->lock); > +=09return 0; > +} > + > +/** > + * mmu_range_notifier_insert - Insert a range notifier > + * @mrn: Range notifier to register > + * @start: Starting virtual address to monitor > + * @length: Length of the range to monitor > + * @mm : mm_struct to attach to > + * > + * This function subscribes the range notifier for notifications from th= e mm. > + * Upon return the ops related to mmu_range_notifier will be called when= ever > + * an event that intersects with the given range occurs. > + * > + * Upon return the range_notifier may not be present in the interval tre= e yet. > + * The caller must use the normal range notifier locking flow via > + * mmu_range_read_begin() to establish SPTEs for this range. > + */ > +int mmu_range_notifier_insert(struct mmu_range_notifier *mrn, > +=09=09=09 unsigned long start, unsigned long length, > +=09=09=09 struct mm_struct *mm) > +{ > +=09struct mmu_notifier_mm *mmn_mm; > +=09int ret; > + > +=09might_lock(&mm->mmap_sem); > + > +=09mmn_mm =3D smp_load_acquire(&mm->mmu_notifier_mm); > +=09if (!mmn_mm || !mmn_mm->has_interval) { > +=09=09ret =3D mmu_notifier_register(NULL, mm); > +=09=09if (ret) > +=09=09=09return ret; > +=09=09mmn_mm =3D mm->mmu_notifier_mm; > +=09} > +=09return __mmu_range_notifier_insert(mrn, start, length, mmn_mm, mm); > +} > +EXPORT_SYMBOL_GPL(mmu_range_notifier_insert); > + > +int mmu_range_notifier_insert_locked(struct mmu_range_notifier *mrn, > +=09=09=09=09 unsigned long start, unsigned long length, > +=09=09=09=09 struct mm_struct *mm) > +{ > +=09struct mmu_notifier_mm *mmn_mm; > +=09int ret; > + > +=09lockdep_assert_held_write(&mm->mmap_sem); > + > +=09mmn_mm =3D mm->mmu_notifier_mm; Shouldn't you be using smp_load_acquire() ? > +=09if (!mmn_mm || !mmn_mm->has_interval) { > +=09=09ret =3D __mmu_notifier_register(NULL, mm); > +=09=09if (ret) > +=09=09=09return ret; > +=09=09mmn_mm =3D mm->mmu_notifier_mm; > +=09} > +=09return __mmu_range_notifier_insert(mrn, start, length, mmn_mm, mm); > +} > +EXPORT_SYMBOL_GPL(mmu_range_notifier_insert_locked); > + > +/** > + * mmu_range_notifier_remove - Remove a range notifier > + * @mrn: Range notifier to unregister > + * > + * This function must be paired with mmu_range_notifier_insert(). It can= not be > + * called from any ops callback. > + * > + * Once this returns ops callbacks are no longer running on other CPUs a= nd > + * will not be called in future. > + */ > +void mmu_range_notifier_remove(struct mmu_range_notifier *mrn) > +{ > +=09struct mm_struct *mm =3D mrn->mm; > +=09struct mmu_notifier_mm *mmn_mm =3D mm->mmu_notifier_mm; > +=09unsigned long seq =3D 0; > + > +=09might_sleep(); > + > +=09spin_lock(&mmn_mm->lock); > +=09if (mn_itree_is_invalidating(mmn_mm)) { > +=09=09/* > +=09=09 * remove is being called after insert put this on the > +=09=09 * deferred list, but before the deferred list was processed. > +=09=09 */ > +=09=09if (RB_EMPTY_NODE(&mrn->interval_tree.rb)) { > +=09=09=09hlist_del(&mrn->deferred_item); > +=09=09} else { > +=09=09=09hlist_add_head(&mrn->deferred_item, > +=09=09=09=09 &mmn_mm->deferred_list); > +=09=09=09seq =3D mmn_mm->invalidate_seq; > +=09=09} > +=09} else { > +=09=09WARN_ON(RB_EMPTY_NODE(&mrn->interval_tree.rb)); > +=09=09interval_tree_remove(&mrn->interval_tree, &mmn_mm->itree); > +=09} > +=09spin_unlock(&mmn_mm->lock); > + > +=09/* > +=09 * The possible sleep on progress in the invalidation requires the > +=09 * caller not hold any locks held by invalidation callbacks. > +=09 */ > +=09lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); > +=09lock_map_release(&__mmu_notifier_invalidate_range_start_map); > +=09if (seq) > +=09=09wait_event(mmn_mm->wq, > +=09=09=09 READ_ONCE(mmn_mm->invalidate_seq) !=3D seq); > + > +=09/* pairs with mmgrab in mmu_range_notifier_insert() */ > +=09mmdrop(mm); > +} > +EXPORT_SYMBOL_GPL(mmu_range_notifier_remove); > + > /** > * mmu_notifier_synchronize - Ensure all mmu_notifiers are freed > * > --=20 > 2.23.0 >=20