From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michal Hocko Subject: [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Fri, 22 Jun 2018 17:02:42 +0200 Message-ID: <20180622150242.16558-1-mhocko@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: LKML Cc: Michal Hocko , kvm@vger.kernel.org, =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , David Airlie , Sudeep Dutt , dri-devel@lists.freedesktop.org, linux-mm@kvack.org, Andrea Arcangeli , "David (ChunMing) Zhou" , Dimitri Sivanich , linux-rdma@vger.kernel.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe , Doug Ledford , David Rientjes , xen-devel@lists.xenproject.org, intel-gfx@lists.freedesktop.org, =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Rodrigo Vivi , Boris Ostrovsky , Juergen Gross , Mike Marciniszyn , Dennis Dalessandro , Ashutosh List-Id: amd-gfx.lists.freedesktop.org RnJvbTogTWljaGFsIEhvY2tvIDxtaG9ja29Ac3VzZS5jb20+CgpUaGVyZSBhcmUgc2V2ZXJhbCBi bG9ja2FibGUgbW11IG5vdGlmaWVycyB3aGljaCBtaWdodCBzbGVlcCBpbgptbXVfbm90aWZpZXJf aW52YWxpZGF0ZV9yYW5nZV9zdGFydCBhbmQgdGhhdCBpcyBhIHByb2JsZW0gZm9yIHRoZQpvb21f cmVhcGVyIGJlY2F1c2UgaXQgbmVlZHMgdG8gZ3VhcmFudGVlIGEgZm9yd2FyZCBwcm9ncmVzcyBz byBpdCBjYW5ub3QKZGVwZW5kIG9uIGFueSBzbGVlcGFibGUgbG9ja3MuIEN1cnJlbnRseSB3ZSBz aW1wbHkgYmFjayBvZmYgYW5kIG1hcmsgYW4Kb29tIHZpY3RpbSB3aXRoIGJsb2NrYWJsZSBtbXUg bm90aWZpZXJzIGFzIGRvbmUgYWZ0ZXIgYSBzaG9ydCBzbGVlcC4KVGhhdCBjYW4gcmVzdWx0IGlu IHNlbGVjdGluZyBhIG5ldyBvb20gdmljdGltIHByZW1hdHVyZWx5IGJlY2F1c2UgdGhlCnByZXZp b3VzIG9uZSBzdGlsbCBoYXNuJ3QgdG9ybiBpdHMgbWVtb3J5IGRvd24geWV0LgoKV2UgY2FuIGRv IG11Y2ggYmV0dGVyIHRob3VnaC4gRXZlbiBpZiBtbXUgbm90aWZpZXJzIHVzZSBzbGVlcGFibGUg bG9ja3MKdGhlcmUgaXMgbm8gcmVhc29uIHRvIGF1dG9tYXRpY2FsbHkgYXNzdW1lIHRob3NlIGxv Y2tzIGFyZSBoZWxkLgpNb3Jlb3ZlciBtb3N0IG5vdGlmaWVycyBvbmx5IGNhcmUgYWJvdXQgYSBw b3J0aW9uIG9mIHRoZSBhZGRyZXNzCnNwYWNlLiBUaGlzIHBhdGNoIGhhbmRsZXMgdGhlIGZpcnN0 IHBhcnQgb2YgdGhlIHByb2JsZW0uCl9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3Rh cnQgZ2V0cyBhIGJsb2NrYWJsZSBmbGFnIGFuZApjYWxsYmFja3MgYXJlIG5vdCBhbGxvd2VkIHRv IHNsZWVwIGlmIHRoZSBmbGFnIGlzIHNldCB0byBmYWxzZS4gVGhpcyBpcwphY2hpZXZlZCBieSB1 c2luZyB0cnlsb2NrIGluc3RlYWQgb2YgdGhlIHNsZWVwYWJsZSBsb2NrIGZvciBtb3N0CmNhbGxi YWNrcy4gSSB0aGluayB3ZSBjYW4gaW1wcm92ZSB0aGF0IGV2ZW4gZnVydGhlciBiZWNhdXNlIHRo ZXJlIGlzCmEgY29tbW9uIHBhdHRlcm4gdG8gZG8gYSByYW5nZSBsb29rdXAgZmlyc3QgYW5kIHRo ZW4gZG8gc29tZXRoaW5nIGFib3V0CnRoYXQuIFRoZSBmaXJzdCBwYXJ0IGNhbiBiZSBkb25lIHdp dGhvdXQgYSBzbGVlcGluZyBsb2NrIEkgcHJlc3VtZS4KCkFueXdheSwgd2hhdCBkb2VzIHRoZSBv b21fcmVhcGVyIGRvIHdpdGggYWxsIHRoYXQ/IFdlIGRvIG5vdCBoYXZlIHRvCmZhaWwgcmlnaHQg YXdheS4gV2Ugc2ltcGx5IHJldHJ5IGlmIHRoZXJlIGlzIGF0IGxlYXN0IG9uZSBub3RpZmllciB3 aGljaApjb3VsZG4ndCBtYWtlIGFueSBwcm9ncmVzcy4gQSByZXRyeSBsb29wIGlzIGFscmVhZHkg aW1wbGVtZW50ZWQgdG8gd2FpdApmb3IgdGhlIG1tYXBfc2VtIGFuZCB0aGlzIGlzIGJhc2ljYWxs eSB0aGUgc2FtZSB0aGluZy4KCkNjOiAiRGF2aWQgKENodW5NaW5nKSBaaG91IiA8RGF2aWQxLlpo b3VAYW1kLmNvbT4KQ2M6IFBhb2xvIEJvbnppbmkgPHBib256aW5pQHJlZGhhdC5jb20+CkNjOiAi UmFkaW0gS3LEjW3DocWZIiA8cmtyY21hckByZWRoYXQuY29tPgpDYzogQWxleCBEZXVjaGVyIDxh bGV4YW5kZXIuZGV1Y2hlckBhbWQuY29tPgpDYzogIkNocmlzdGlhbiBLw7ZuaWciIDxjaHJpc3Rp YW4ua29lbmlnQGFtZC5jb20+CkNjOiBEYXZpZCBBaXJsaWUgPGFpcmxpZWRAbGludXguaWU+CkNj OiBKYW5pIE5pa3VsYSA8amFuaS5uaWt1bGFAbGludXguaW50ZWwuY29tPgpDYzogSm9vbmFzIExh aHRpbmVuIDxqb29uYXMubGFodGluZW5AbGludXguaW50ZWwuY29tPgpDYzogUm9kcmlnbyBWaXZp IDxyb2RyaWdvLnZpdmlAaW50ZWwuY29tPgpDYzogRG91ZyBMZWRmb3JkIDxkbGVkZm9yZEByZWRo YXQuY29tPgpDYzogSmFzb24gR3VudGhvcnBlIDxqZ2dAemllcGUuY2E+CkNjOiBNaWtlIE1hcmNp bmlzenluIDxtaWtlLm1hcmNpbmlzenluQGludGVsLmNvbT4KQ2M6IERlbm5pcyBEYWxlc3NhbmRy byA8ZGVubmlzLmRhbGVzc2FuZHJvQGludGVsLmNvbT4KQ2M6IFN1ZGVlcCBEdXR0IDxzdWRlZXAu ZHV0dEBpbnRlbC5jb20+CkNjOiBBc2h1dG9zaCBEaXhpdCA8YXNodXRvc2guZGl4aXRAaW50ZWwu Y29tPgpDYzogRGltaXRyaSBTaXZhbmljaCA8c2l2YW5pY2hAc2dpLmNvbT4KQ2M6IEJvcmlzIE9z dHJvdnNreSA8Ym9yaXMub3N0cm92c2t5QG9yYWNsZS5jb20+CkNjOiBKdWVyZ2VuIEdyb3NzIDxq Z3Jvc3NAc3VzZS5jb20+CkNjOiAiSsOpcsO0bWUgR2xpc3NlIiA8amdsaXNzZUByZWRoYXQuY29t PgpDYzogQW5kcmVhIEFyY2FuZ2VsaSA8YWFyY2FuZ2VAcmVkaGF0LmNvbT4KQ2M6IGt2bUB2Z2Vy Lmtlcm5lbC5vcmcgKG9wZW4gbGlzdDpLRVJORUwgVklSVFVBTCBNQUNISU5FIEZPUiBYODYgKEtW TS94ODYpKQpDYzogbGludXgta2VybmVsQHZnZXIua2VybmVsLm9yZyAob3BlbiBsaXN0Olg4NiBB UkNISVRFQ1RVUkUgKDMyLUJJVCBBTkQgNjQtQklUKSkKQ2M6IGFtZC1nZnhAbGlzdHMuZnJlZWRl c2t0b3Aub3JnIChvcGVuIGxpc3Q6UkFERU9OIGFuZCBBTURHUFUgRFJNIERSSVZFUlMpCkNjOiBk cmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnIChvcGVuIGxpc3Q6RFJNIERSSVZFUlMpCkNj OiBpbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnIChvcGVuIGxpc3Q6SU5URUwgRFJNIERS SVZFUlMgKGV4Y2x1ZGluZyBQb3Vsc2JvLCBNb29yZXN0b3cuLi4pCkNjOiBsaW51eC1yZG1hQHZn ZXIua2VybmVsLm9yZyAob3BlbiBsaXN0OklORklOSUJBTkQgU1VCU1lTVEVNKQpDYzogeGVuLWRl dmVsQGxpc3RzLnhlbnByb2plY3Qub3JnIChtb2RlcmF0ZWQgbGlzdDpYRU4gSFlQRVJWSVNPUiBJ TlRFUkZBQ0UpCkNjOiBsaW51eC1tbUBrdmFjay5vcmcgKG9wZW4gbGlzdDpITU0gLSBIZXRlcm9n ZW5lb3VzIE1lbW9yeSBNYW5hZ2VtZW50KQpSZXBvcnRlZC1ieTogRGF2aWQgUmllbnRqZXMgPHJp ZW50amVzQGdvb2dsZS5jb20+ClNpZ25lZC1vZmYtYnk6IE1pY2hhbCBIb2NrbyA8bWhvY2tvQHN1 c2UuY29tPgotLS0KCkhpLAp0aGlzIGlzIGFuIFJGQyBhbmQgbm90IHRlc3RlZCBhdCBhbGwuIEkg YW0gbm90IHZlcnkgZmFtaWxpYXIgd2l0aCB0aGUKbW11IG5vdGlmaWVycyBzZW1hbnRpY3MgdmVy eSBtdWNoIHNvIHRoaXMgaXMgYSBjcnVkZSBhdHRlbXB0IHRvIGFjaGlldmUKd2hhdCBJIG5lZWQg YmFzaWNhbGx5LiBJdCBtaWdodCBiZSBjb21wbGV0ZWx5IHdyb25nIGJ1dCBJIHdvdWxkIGxpa2UK dG8gZGlzY3VzcyB3aGF0IHdvdWxkIGJlIGEgYmV0dGVyIHdheSBpZiB0aGF0IGlzIHRoZSBjYXNl LgoKZ2V0X21haW50YWluZXJzIGdhdmUgbWUgcXVpdGUgbGFyZ2UgbGlzdCBvZiBwZW9wbGUgdG8g Q0Mgc28gSSBoYWQgdG8gdHJpbQppdCBkb3duLiBJZiB5b3UgdGhpbmsgSSBoYXZlIGZvcmdvdCBz b21lYm9keSwgcGxlYXNlIGxldCBtZSBrbm93CgpBbnkgZmVlZGJhY2sgaXMgaGlnaGx5IGFwcHJl Y2lhdGVkLgoKIGFyY2gveDg2L2t2bS94ODYuYyAgICAgICAgICAgICAgICAgICAgICB8ICA3ICsr KystLQogZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X21uLmMgIHwgMzMgKysrKysr KysrKysrKysrKysrKy0tLS0tLQogZHJpdmVycy9ncHUvZHJtL2k5MTUvaTkxNV9nZW1fdXNlcnB0 ci5jIHwgMTAgKysrKystLS0KIGRyaXZlcnMvZ3B1L2RybS9yYWRlb24vcmFkZW9uX21uLmMgICAg ICB8IDE1ICsrKysrKysrLS0tCiBkcml2ZXJzL2luZmluaWJhbmQvY29yZS91bWVtX29kcC5jICAg ICAgfCAxNSArKysrKysrKy0tLQogZHJpdmVycy9pbmZpbmliYW5kL2h3L2hmaTEvbW11X3JiLmMg ICAgIHwgIDcgKysrKy0tCiBkcml2ZXJzL21pc2MvbWljL3NjaWYvc2NpZl9kbWEuYyAgICAgICAg fCAgNyArKysrLS0KIGRyaXZlcnMvbWlzYy9zZ2ktZ3J1L2dydXRsYnB1cmdlLmMgICAgICB8ICA3 ICsrKystLQogZHJpdmVycy94ZW4vZ250ZGV2LmMgICAgICAgICAgICAgICAgICAgIHwgMTQgKysr KysrKystLS0KIGluY2x1ZGUvbGludXgva3ZtX2hvc3QuaCAgICAgICAgICAgICAgICB8ICAyICst CiBpbmNsdWRlL2xpbnV4L21tdV9ub3RpZmllci5oICAgICAgICAgICAgfCAxNSArKysrKysrKyst LQogbW0vaG1tLmMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgIDcgKysrKy0tCiBt bS9tbXVfbm90aWZpZXIuYyAgICAgICAgICAgICAgICAgICAgICAgfCAxNSArKysrKysrKy0tLQog bW0vb29tX2tpbGwuYyAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgMjkgKysrKysrKysrKyst LS0tLS0tLS0tLQogdmlydC9rdm0va3ZtX21haW4uYyAgICAgICAgICAgICAgICAgICAgIHwgMTIg KysrKysrLS0tCiAxNSBmaWxlcyBjaGFuZ2VkLCAxMzcgaW5zZXJ0aW9ucygrKSwgNTggZGVsZXRp b25zKC0pCgpkaWZmIC0tZ2l0IGEvYXJjaC94ODYva3ZtL3g4Ni5jIGIvYXJjaC94ODYva3ZtL3g4 Ni5jCmluZGV4IDZiY2VjYzMyNWU3ZS4uYWMwOGY1ZDcxMWJlIDEwMDY0NAotLS0gYS9hcmNoL3g4 Ni9rdm0veDg2LmMKKysrIGIvYXJjaC94ODYva3ZtL3g4Ni5jCkBAIC03MjAzLDggKzcyMDMsOSBA QCBzdGF0aWMgdm9pZCB2Y3B1X2xvYWRfZW9pX2V4aXRtYXAoc3RydWN0IGt2bV92Y3B1ICp2Y3B1 KQogCWt2bV94ODZfb3BzLT5sb2FkX2VvaV9leGl0bWFwKHZjcHUsIGVvaV9leGl0X2JpdG1hcCk7 CiB9CiAKLXZvaWQga3ZtX2FyY2hfbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Uoc3RydWN0 IGt2bSAqa3ZtLAotCQl1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCkKK2lu dCBrdm1fYXJjaF9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZShzdHJ1Y3Qga3ZtICprdm0s CisJCXVuc2lnbmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kLAorCQlib29sIGJsb2Nr YWJsZSkKIHsKIAl1bnNpZ25lZCBsb25nIGFwaWNfYWRkcmVzczsKIApAQCAtNzIxNSw2ICs3MjE2 LDggQEAgdm9pZCBrdm1fYXJjaF9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZShzdHJ1Y3Qg a3ZtICprdm0sCiAJYXBpY19hZGRyZXNzID0gZ2ZuX3RvX2h2YShrdm0sIEFQSUNfREVGQVVMVF9Q SFlTX0JBU0UgPj4gUEFHRV9TSElGVCk7CiAJaWYgKHN0YXJ0IDw9IGFwaWNfYWRkcmVzcyAmJiBh cGljX2FkZHJlc3MgPCBlbmQpCiAJCWt2bV9tYWtlX2FsbF9jcHVzX3JlcXVlc3Qoa3ZtLCBLVk1f UkVRX0FQSUNfUEFHRV9SRUxPQUQpOworCisJcmV0dXJuIDA7CiB9CiAKIHZvaWQga3ZtX3ZjcHVf cmVsb2FkX2FwaWNfYWNjZXNzX3BhZ2Uoc3RydWN0IGt2bV92Y3B1ICp2Y3B1KQpkaWZmIC0tZ2l0 IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X21uLmMgYi9kcml2ZXJzL2dwdS9k cm0vYW1kL2FtZGdwdS9hbWRncHVfbW4uYwppbmRleCA4M2UzNDRmYmI1MGEuLmQxMzhhNTI2ZmVm ZiAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X21uLmMKKysr IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X21uLmMKQEAgLTEzNiwxMiArMTM2 LDE4IEBAIHZvaWQgYW1kZ3B1X21uX3VubG9jayhzdHJ1Y3QgYW1kZ3B1X21uICptbikKICAqCiAg KiBUYWtlIHRoZSBybW4gcmVhZCBzaWRlIGxvY2suCiAgKi8KLXN0YXRpYyB2b2lkIGFtZGdwdV9t bl9yZWFkX2xvY2soc3RydWN0IGFtZGdwdV9tbiAqcm1uKQorc3RhdGljIGludCBhbWRncHVfbW5f cmVhZF9sb2NrKHN0cnVjdCBhbWRncHVfbW4gKnJtbiwgYm9vbCBibG9ja2FibGUpCiB7Ci0JbXV0 ZXhfbG9jaygmcm1uLT5yZWFkX2xvY2spOworCWlmIChibG9ja2FibGUpCisJCW11dGV4X2xvY2so JnJtbi0+cmVhZF9sb2NrKTsKKwllbHNlIGlmICghbXV0ZXhfdHJ5bG9jaygmcm1uLT5yZWFkX2xv Y2spKQorCQlyZXR1cm4gLUVBR0FJTjsKKwogCWlmIChhdG9taWNfaW5jX3JldHVybigmcm1uLT5y ZWN1cnNpb24pID09IDEpCiAJCWRvd25fcmVhZF9ub25fb3duZXIoJnJtbi0+bG9jayk7CiAJbXV0 ZXhfdW5sb2NrKCZybW4tPnJlYWRfbG9jayk7CisKKwlyZXR1cm4gMDsKIH0KIAogLyoqCkBAIC0x OTcsMTAgKzIwMywxMSBAQCBzdGF0aWMgdm9pZCBhbWRncHVfbW5faW52YWxpZGF0ZV9ub2RlKHN0 cnVjdCBhbWRncHVfbW5fbm9kZSAqbm9kZSwKICAqIFdlIGJsb2NrIGZvciBhbGwgQk9zIGJldHdl ZW4gc3RhcnQgYW5kIGVuZCB0byBiZSBpZGxlIGFuZAogICogdW5tYXAgdGhlbSBieSBtb3ZlIHRo ZW0gaW50byBzeXN0ZW0gZG9tYWluIGFnYWluLgogICovCi1zdGF0aWMgdm9pZCBhbWRncHVfbW5f aW52YWxpZGF0ZV9yYW5nZV9zdGFydF9nZngoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCitzdGF0 aWMgaW50IGFtZGdwdV9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X2dmeChzdHJ1Y3QgbW11X25v dGlmaWVyICptbiwKIAkJCQkJCSBzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkJCSB1bnNpZ25l ZCBsb25nIHN0YXJ0LAotCQkJCQkJIHVuc2lnbmVkIGxvbmcgZW5kKQorCQkJCQkJIHVuc2lnbmVk IGxvbmcgZW5kLAorCQkJCQkJIGJvb2wgYmxvY2thYmxlKQogewogCXN0cnVjdCBhbWRncHVfbW4g KnJtbiA9IGNvbnRhaW5lcl9vZihtbiwgc3RydWN0IGFtZGdwdV9tbiwgbW4pOwogCXN0cnVjdCBp bnRlcnZhbF90cmVlX25vZGUgKml0OwpAQCAtMjA4LDcgKzIxNSwxMSBAQCBzdGF0aWMgdm9pZCBh bWRncHVfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydF9nZngoc3RydWN0IG1tdV9ub3RpZmllciAq bW4sCiAJLyogbm90aWZpY2F0aW9uIGlzIGV4Y2x1c2l2ZSwgYnV0IGludGVydmFsIGlzIGluY2x1 c2l2ZSAqLwogCWVuZCAtPSAxOwogCi0JYW1kZ3B1X21uX3JlYWRfbG9jayhybW4pOworCS8qIFRP RE8gd2Ugc2hvdWxkIGJlIGFibGUgdG8gc3BsaXQgbG9ja2luZyBmb3IgaW50ZXJ2YWwgdHJlZSBh bmQKKwkgKiBhbWRncHVfbW5faW52YWxpZGF0ZV9ub2RlCisJICovCisJaWYgKGFtZGdwdV9tbl9y ZWFkX2xvY2socm1uLCBibG9ja2FibGUpKQorCQlyZXR1cm4gLUVBR0FJTjsKIAogCWl0ID0gaW50 ZXJ2YWxfdHJlZV9pdGVyX2ZpcnN0KCZybW4tPm9iamVjdHMsIHN0YXJ0LCBlbmQpOwogCXdoaWxl IChpdCkgewpAQCAtMjE5LDYgKzIzMCw4IEBAIHN0YXRpYyB2b2lkIGFtZGdwdV9tbl9pbnZhbGlk YXRlX3JhbmdlX3N0YXJ0X2dmeChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAogCQlhbWRncHVf bW5faW52YWxpZGF0ZV9ub2RlKG5vZGUsIHN0YXJ0LCBlbmQpOwogCX0KKworCXJldHVybiAwOwog fQogCiAvKioKQEAgLTIzMywxMCArMjQ2LDExIEBAIHN0YXRpYyB2b2lkIGFtZGdwdV9tbl9pbnZh bGlkYXRlX3JhbmdlX3N0YXJ0X2dmeChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKICAqIG5lY2Vz c2l0YXRlcyBldmljdGluZyBhbGwgdXNlci1tb2RlIHF1ZXVlcyBvZiB0aGUgcHJvY2Vzcy4gVGhl IEJPcwogICogYXJlIHJlc3RvcnRlZCBpbiBhbWRncHVfbW5faW52YWxpZGF0ZV9yYW5nZV9lbmRf aHNhLgogICovCi1zdGF0aWMgdm9pZCBhbWRncHVfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydF9o c2Eoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCitzdGF0aWMgaW50IGFtZGdwdV9tbl9pbnZhbGlk YXRlX3JhbmdlX3N0YXJ0X2hzYShzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJCQkJCSBzdHJ1 Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkJCSB1bnNpZ25lZCBsb25nIHN0YXJ0LAotCQkJCQkJIHVu c2lnbmVkIGxvbmcgZW5kKQorCQkJCQkJIHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCQkJIGJvb2wg YmxvY2thYmxlKQogewogCXN0cnVjdCBhbWRncHVfbW4gKnJtbiA9IGNvbnRhaW5lcl9vZihtbiwg c3RydWN0IGFtZGdwdV9tbiwgbW4pOwogCXN0cnVjdCBpbnRlcnZhbF90cmVlX25vZGUgKml0OwpA QCAtMjQ0LDcgKzI1OCw4IEBAIHN0YXRpYyB2b2lkIGFtZGdwdV9tbl9pbnZhbGlkYXRlX3Jhbmdl X3N0YXJ0X2hzYShzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkvKiBub3RpZmljYXRpb24gaXMg ZXhjbHVzaXZlLCBidXQgaW50ZXJ2YWwgaXMgaW5jbHVzaXZlICovCiAJZW5kIC09IDE7CiAKLQlh bWRncHVfbW5fcmVhZF9sb2NrKHJtbik7CisJaWYgKGFtZGdwdV9tbl9yZWFkX2xvY2socm1uLCBi bG9ja2FibGUpKQorCQlyZXR1cm4gLUVBR0FJTjsKIAogCWl0ID0gaW50ZXJ2YWxfdHJlZV9pdGVy X2ZpcnN0KCZybW4tPm9iamVjdHMsIHN0YXJ0LCBlbmQpOwogCXdoaWxlIChpdCkgewpAQCAtMjYy LDYgKzI3Nyw4IEBAIHN0YXRpYyB2b2lkIGFtZGdwdV9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0 X2hzYShzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJCQlhbWRncHVfYW1ka2ZkX2V2aWN0X3Vz ZXJwdHIobWVtLCBtbSk7CiAJCX0KIAl9CisKKwlyZXR1cm4gMDsKIH0KIAogLyoqCmRpZmYgLS1n aXQgYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pOTE1X2dlbV91c2VycHRyLmMgYi9kcml2ZXJzL2dw dS9kcm0vaTkxNS9pOTE1X2dlbV91c2VycHRyLmMKaW5kZXggODU0YmQ1MWI5NDc4Li41Mjg1ZGY5 MzMxZmEgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2k5MTVfZ2VtX3VzZXJwdHIu YworKysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pOTE1X2dlbV91c2VycHRyLmMKQEAgLTExMiwx MCArMTEyLDExIEBAIHN0YXRpYyB2b2lkIGRlbF9vYmplY3Qoc3RydWN0IGk5MTVfbW11X29iamVj dCAqbW8pCiAJbW8tPmF0dGFjaGVkID0gZmFsc2U7CiB9CiAKLXN0YXRpYyB2b2lkIGk5MTVfZ2Vt X3VzZXJwdHJfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICpf bW4sCitzdGF0aWMgaW50IGk5MTVfZ2VtX3VzZXJwdHJfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFy dChzdHJ1Y3QgbW11X25vdGlmaWVyICpfbW4sCiAJCQkJCQkgICAgICAgc3RydWN0IG1tX3N0cnVj dCAqbW0sCiAJCQkJCQkgICAgICAgdW5zaWduZWQgbG9uZyBzdGFydCwKLQkJCQkJCSAgICAgICB1 bnNpZ25lZCBsb25nIGVuZCkKKwkJCQkJCSAgICAgICB1bnNpZ25lZCBsb25nIGVuZCwKKwkJCQkJ CSAgICAgICBib29sIGJsb2NrYWJsZSkKIHsKIAlzdHJ1Y3QgaTkxNV9tbXVfbm90aWZpZXIgKm1u ID0KIAkJY29udGFpbmVyX29mKF9tbiwgc3RydWN0IGk5MTVfbW11X25vdGlmaWVyLCBtbik7CkBA IC0xMjQsNyArMTI1LDcgQEAgc3RhdGljIHZvaWQgaTkxNV9nZW1fdXNlcnB0cl9tbl9pbnZhbGlk YXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKl9tbiwKIAlMSVNUX0hFQUQoY2Fu Y2VsbGVkKTsKIAogCWlmIChSQl9FTVBUWV9ST09UKCZtbi0+b2JqZWN0cy5yYl9yb290KSkKLQkJ cmV0dXJuOworCQlyZXR1cm4gMDsKIAogCS8qIGludGVydmFsIHJhbmdlcyBhcmUgaW5jbHVzaXZl LCBidXQgaW52YWxpZGF0ZSByYW5nZSBpcyBleGNsdXNpdmUgKi8KIAllbmQtLTsKQEAgLTE1Miw3 ICsxNTMsOCBAQCBzdGF0aWMgdm9pZCBpOTE1X2dlbV91c2VycHRyX21uX2ludmFsaWRhdGVfcmFu Z2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqX21uLAogCQlkZWxfb2JqZWN0KG1vKTsKIAlz cGluX3VubG9jaygmbW4tPmxvY2spOwogCi0JaWYgKCFsaXN0X2VtcHR5KCZjYW5jZWxsZWQpKQor CS8qIFRPRE86IGNhbiB3ZSBza2lwIHdhaXRpbmcgaGVyZT8gKi8KKwlpZiAoIWxpc3RfZW1wdHko JmNhbmNlbGxlZCkgJiYgYmxvY2thYmxlKQogCQlmbHVzaF93b3JrcXVldWUobW4tPndxKTsKIH0K IApkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL3JhZGVvbi9yYWRlb25fbW4uYyBiL2RyaXZl cnMvZ3B1L2RybS9yYWRlb24vcmFkZW9uX21uLmMKaW5kZXggYWJkMjQ5NzVjOWIxLi5iNDdlODI4 YjcyNWQgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9yYWRlb24vcmFkZW9uX21uLmMKKysr IGIvZHJpdmVycy9ncHUvZHJtL3JhZGVvbi9yYWRlb25fbW4uYwpAQCAtMTE4LDEwICsxMTgsMTEg QEAgc3RhdGljIHZvaWQgcmFkZW9uX21uX3JlbGVhc2Uoc3RydWN0IG1tdV9ub3RpZmllciAqbW4s CiAgKiBXZSBibG9jayBmb3IgYWxsIEJPcyBiZXR3ZWVuIHN0YXJ0IGFuZCBlbmQgdG8gYmUgaWRs ZSBhbmQKICAqIHVubWFwIHRoZW0gYnkgbW92ZSB0aGVtIGludG8gc3lzdGVtIGRvbWFpbiBhZ2Fp bi4KICAqLwotc3RhdGljIHZvaWQgcmFkZW9uX21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3Ry dWN0IG1tdV9ub3RpZmllciAqbW4sCitzdGF0aWMgaW50IHJhZGVvbl9tbl9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQkJCQkgICAgIHN0cnVjdCBtbV9z dHJ1Y3QgKm1tLAogCQkJCQkgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsCi0JCQkJCSAgICAgdW5z aWduZWQgbG9uZyBlbmQpCisJCQkJCSAgICAgdW5zaWduZWQgbG9uZyBlbmQsCisJCQkJCSAgICAg Ym9vbCBibG9ja2FibGUpCiB7CiAJc3RydWN0IHJhZGVvbl9tbiAqcm1uID0gY29udGFpbmVyX29m KG1uLCBzdHJ1Y3QgcmFkZW9uX21uLCBtbik7CiAJc3RydWN0IHR0bV9vcGVyYXRpb25fY3R4IGN0 eCA9IHsgZmFsc2UsIGZhbHNlIH07CkBAIC0xMzAsNyArMTMxLDEzIEBAIHN0YXRpYyB2b2lkIHJh ZGVvbl9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAog CS8qIG5vdGlmaWNhdGlvbiBpcyBleGNsdXNpdmUsIGJ1dCBpbnRlcnZhbCBpcyBpbmNsdXNpdmUg Ki8KIAllbmQgLT0gMTsKIAotCW11dGV4X2xvY2soJnJtbi0+bG9jayk7CisJLyogVE9ETyB3ZSBz aG91bGQgYmUgYWJsZSB0byBzcGxpdCBsb2NraW5nIGZvciBpbnRlcnZhbCB0cmVlIGFuZAorCSAq IHRoZSB0ZWFyIGRvd24uCisJICovCisJaWYgKGJsb2NrYWJsZSkKKwkJbXV0ZXhfbG9jaygmcm1u LT5sb2NrKTsKKwllbHNlIGlmICghbXV0ZXhfdHJ5bG9jaygmcm1uLT5sb2NrKSkKKwkJcmV0dXJu IC1FQUdBSU47CiAKIAlpdCA9IGludGVydmFsX3RyZWVfaXRlcl9maXJzdCgmcm1uLT5vYmplY3Rz LCBzdGFydCwgZW5kKTsKIAl3aGlsZSAoaXQpIHsKQEAgLTE2Nyw2ICsxNzQsOCBAQCBzdGF0aWMg dm9pZCByYWRlb25fbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVy ICptbiwKIAl9CiAJCiAJbXV0ZXhfdW5sb2NrKCZybW4tPmxvY2spOworCisJcmV0dXJuIDA7CiB9 CiAKIHN0YXRpYyBjb25zdCBzdHJ1Y3QgbW11X25vdGlmaWVyX29wcyByYWRlb25fbW5fb3BzID0g ewpkaWZmIC0tZ2l0IGEvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvdW1lbV9vZHAuYyBiL2RyaXZl cnMvaW5maW5pYmFuZC9jb3JlL3VtZW1fb2RwLmMKaW5kZXggMTgyNDM2YjkyYmE5Li5mNjVmNmEy OWRhYWUgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL3VtZW1fb2RwLmMKKysr IGIvZHJpdmVycy9pbmZpbmliYW5kL2NvcmUvdW1lbV9vZHAuYwpAQCAtMjA3LDIyICsyMDcsMjkg QEAgc3RhdGljIGludCBpbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X3RyYW1wb2xpbmUoc3RydWN0IGli X3VtZW0gKml0ZW0sIHU2NCBzdGFydCwKIAlyZXR1cm4gMDsKIH0KIAotc3RhdGljIHZvaWQgaWJf dW1lbV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIg Km1uLAorc3RhdGljIGludCBpYl91bWVtX25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQo c3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJCQkJCQkgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0s CiAJCQkJCQkgICAgdW5zaWduZWQgbG9uZyBzdGFydCwKLQkJCQkJCSAgICB1bnNpZ25lZCBsb25n IGVuZCkKKwkJCQkJCSAgICB1bnNpZ25lZCBsb25nIGVuZCwKKwkJCQkJCSAgICBib29sIGJsb2Nr YWJsZSkKIHsKIAlzdHJ1Y3QgaWJfdWNvbnRleHQgKmNvbnRleHQgPSBjb250YWluZXJfb2YobW4s IHN0cnVjdCBpYl91Y29udGV4dCwgbW4pOwogCiAJaWYgKCFjb250ZXh0LT5pbnZhbGlkYXRlX3Jh bmdlKQotCQlyZXR1cm47CisJCXJldHVybiAwOworCisJaWYgKGJsb2NrYWJsZSkKKwkJZG93bl9y ZWFkKCZjb250ZXh0LT51bWVtX3J3c2VtKTsKKwllbHNlIGlmICghZG93bl9yZWFkX3RyeWxvY2so JmNvbnRleHQtPnVtZW1fcndzZW0pKQorCQlyZXR1cm4gLUVBR0FJTjsKIAogCWliX3Vjb250ZXh0 X25vdGlmaWVyX3N0YXJ0X2FjY291bnQoY29udGV4dCk7Ci0JZG93bl9yZWFkKCZjb250ZXh0LT51 bWVtX3J3c2VtKTsKIAlyYnRfaWJfdW1lbV9mb3JfZWFjaF9pbl9yYW5nZSgmY29udGV4dC0+dW1l bV90cmVlLCBzdGFydCwKIAkJCQkgICAgICBlbmQsCiAJCQkJICAgICAgaW52YWxpZGF0ZV9yYW5n ZV9zdGFydF90cmFtcG9saW5lLCBOVUxMKTsKIAl1cF9yZWFkKCZjb250ZXh0LT51bWVtX3J3c2Vt KTsKKworCXJldHVybiAwOwogfQogCiBzdGF0aWMgaW50IGludmFsaWRhdGVfcmFuZ2VfZW5kX3Ry YW1wb2xpbmUoc3RydWN0IGliX3VtZW0gKml0ZW0sIHU2NCBzdGFydCwKZGlmZiAtLWdpdCBhL2Ry aXZlcnMvaW5maW5pYmFuZC9ody9oZmkxL21tdV9yYi5jIGIvZHJpdmVycy9pbmZpbmliYW5kL2h3 L2hmaTEvbW11X3JiLmMKaW5kZXggNzBhY2VlZmUxNGQ1Li44NzgwNTYwZDE2MjMgMTAwNjQ0Ci0t LSBhL2RyaXZlcnMvaW5maW5pYmFuZC9ody9oZmkxL21tdV9yYi5jCisrKyBiL2RyaXZlcnMvaW5m aW5pYmFuZC9ody9oZmkxL21tdV9yYi5jCkBAIC0yODQsMTAgKzI4NCwxMSBAQCB2b2lkIGhmaTFf bW11X3JiX3JlbW92ZShzdHJ1Y3QgbW11X3JiX2hhbmRsZXIgKmhhbmRsZXIsCiAJaGFuZGxlci0+ b3BzLT5yZW1vdmUoaGFuZGxlci0+b3BzX2FyZywgbm9kZSk7CiB9CiAKLXN0YXRpYyB2b2lkIG1t dV9ub3RpZmllcl9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKK3N0YXRpYyBp bnQgbW11X25vdGlmaWVyX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQkJ CSAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0sCiAJCQkJICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0 LAotCQkJCSAgICAgdW5zaWduZWQgbG9uZyBlbmQpCisJCQkJICAgICB1bnNpZ25lZCBsb25nIGVu ZCwKKwkJCQkgICAgIGJvb2wgYmxvY2thYmxlKQogewogCXN0cnVjdCBtbXVfcmJfaGFuZGxlciAq aGFuZGxlciA9CiAJCWNvbnRhaW5lcl9vZihtbiwgc3RydWN0IG1tdV9yYl9oYW5kbGVyLCBtbik7 CkBAIC0zMTMsNiArMzE0LDggQEAgc3RhdGljIHZvaWQgbW11X25vdGlmaWVyX3JhbmdlX3N0YXJ0 KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCiAJaWYgKGFkZGVkKQogCQlxdWV1ZV93b3JrKGhh bmRsZXItPndxLCAmaGFuZGxlci0+ZGVsX3dvcmspOworCisJcmV0dXJuIDA7CiB9CiAKIC8qCmRp ZmYgLS1naXQgYS9kcml2ZXJzL21pc2MvbWljL3NjaWYvc2NpZl9kbWEuYyBiL2RyaXZlcnMvbWlz Yy9taWMvc2NpZi9zY2lmX2RtYS5jCmluZGV4IDYzZDYyNDZkNmRmZi4uZDk0MDU2OGJlZDg3IDEw MDY0NAotLS0gYS9kcml2ZXJzL21pc2MvbWljL3NjaWYvc2NpZl9kbWEuYworKysgYi9kcml2ZXJz L21pc2MvbWljL3NjaWYvc2NpZl9kbWEuYwpAQCAtMjAwLDE1ICsyMDAsMTggQEAgc3RhdGljIHZv aWQgc2NpZl9tbXVfbm90aWZpZXJfcmVsZWFzZShzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAlz Y2hlZHVsZV93b3JrKCZzY2lmX2luZm8ubWlzY193b3JrKTsKIH0KIAotc3RhdGljIHZvaWQgc2Np Zl9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVy ICptbiwKK3N0YXRpYyBpbnQgc2NpZl9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFy dChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJCQkJCSAgICAgc3RydWN0IG1tX3N0cnVjdCAq bW0sCiAJCQkJCQkgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsCi0JCQkJCQkgICAgIHVuc2lnbmVk IGxvbmcgZW5kKQorCQkJCQkJICAgICB1bnNpZ25lZCBsb25nIGVuZCwKKwkJCQkJCSAgICAgYm9v bCBibG9ja2FibGUpCiB7CiAJc3RydWN0IHNjaWZfbW11X25vdGlmCSptbW47CiAKIAltbW4gPSBj b250YWluZXJfb2YobW4sIHN0cnVjdCBzY2lmX21tdV9ub3RpZiwgZXBfbW11X25vdGlmaWVyKTsK IAlzY2lmX3JtYV9kZXN0cm95X3RjdyhtbW4sIHN0YXJ0LCBlbmQgLSBzdGFydCk7CisKKwlyZXR1 cm4gMAogfQogCiBzdGF0aWMgdm9pZCBzY2lmX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jhbmdl X2VuZChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKZGlmZiAtLWdpdCBhL2RyaXZlcnMvbWlzYy9z Z2ktZ3J1L2dydXRsYnB1cmdlLmMgYi9kcml2ZXJzL21pc2Mvc2dpLWdydS9ncnV0bGJwdXJnZS5j CmluZGV4IGEzNDU0ZWI1NmZiZi4uYmUyOGYwNWJmYWZhIDEwMDY0NAotLS0gYS9kcml2ZXJzL21p c2Mvc2dpLWdydS9ncnV0bGJwdXJnZS5jCisrKyBiL2RyaXZlcnMvbWlzYy9zZ2ktZ3J1L2dydXRs YnB1cmdlLmMKQEAgLTIxOSw5ICsyMTksMTAgQEAgdm9pZCBncnVfZmx1c2hfYWxsX3RsYihzdHJ1 Y3QgZ3J1X3N0YXRlICpncnUpCiAvKgogICogTU1VT1BTIG5vdGlmaWVyIGNhbGxvdXQgZnVuY3Rp b25zCiAgKi8KLXN0YXRpYyB2b2lkIGdydV9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBt bXVfbm90aWZpZXIgKm1uLAorc3RhdGljIGludCBncnVfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChz dHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJCQkgICAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0s Ci0JCQkJICAgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kKQorCQkJ CSAgICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCwKKwkJCQkgICAg ICAgYm9vbCBibG9ja2FibGUpCiB7CiAJc3RydWN0IGdydV9tbV9zdHJ1Y3QgKmdtcyA9IGNvbnRh aW5lcl9vZihtbiwgc3RydWN0IGdydV9tbV9zdHJ1Y3QsCiAJCQkJCQkgbXNfbm90aWZpZXIpOwpA QCAtMjMxLDYgKzIzMiw4IEBAIHN0YXRpYyB2b2lkIGdydV9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0 KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCWdydV9kYmcoZ3J1ZGV2LCAiZ21zICVwLCBzdGFy dCAweCVseCwgZW5kIDB4JWx4LCBhY3QgJWRcbiIsIGdtcywKIAkJc3RhcnQsIGVuZCwgYXRvbWlj X3JlYWQoJmdtcy0+bXNfcmFuZ2VfYWN0aXZlKSk7CiAJZ3J1X2ZsdXNoX3RsYl9yYW5nZShnbXMs IHN0YXJ0LCBlbmQgLSBzdGFydCk7CisKKwlyZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgZ3J1 X2ludmFsaWRhdGVfcmFuZ2VfZW5kKHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLApkaWZmIC0tZ2l0 IGEvZHJpdmVycy94ZW4vZ250ZGV2LmMgYi9kcml2ZXJzL3hlbi9nbnRkZXYuYwppbmRleCBiZDU2 NjUzYjliYmMuLjUwNzI0ZDA5ZmU1YyAxMDA2NDQKLS0tIGEvZHJpdmVycy94ZW4vZ250ZGV2LmMK KysrIGIvZHJpdmVycy94ZW4vZ250ZGV2LmMKQEAgLTQ2NSwxNCArNDY1LDIwIEBAIHN0YXRpYyB2 b2lkIHVubWFwX2lmX2luX3JhbmdlKHN0cnVjdCBncmFudF9tYXAgKm1hcCwKIAlXQVJOX09OKGVy cik7CiB9CiAKLXN0YXRpYyB2b2lkIG1uX2ludmxfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3Rp ZmllciAqbW4sCitzdGF0aWMgaW50IG1uX2ludmxfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3Rp ZmllciAqbW4sCiAJCQkJc3RydWN0IG1tX3N0cnVjdCAqbW0sCi0JCQkJdW5zaWduZWQgbG9uZyBz dGFydCwgdW5zaWduZWQgbG9uZyBlbmQpCisJCQkJdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWdu ZWQgbG9uZyBlbmQsCisJCQkJYm9vbCBibG9ja2FibGUpCiB7CiAJc3RydWN0IGdudGRldl9wcml2 ICpwcml2ID0gY29udGFpbmVyX29mKG1uLCBzdHJ1Y3QgZ250ZGV2X3ByaXYsIG1uKTsKIAlzdHJ1 Y3QgZ3JhbnRfbWFwICptYXA7CiAKLQltdXRleF9sb2NrKCZwcml2LT5sb2NrKTsKKwkvKiBUT0RP IGRvIHdlIHJlYWxseSBuZWVkIGEgbXV0ZXggaGVyZT8gKi8KKwlpZiAoYmxvY2thYmxlKQorCQlt dXRleF9sb2NrKCZwcml2LT5sb2NrKTsKKwllbHNlIGlmICghbXV0ZXhfdHJ5bG9jaygmcHJpdi0+ bG9jaykpCisJCXJldHVybiAtRUFHQUlOOworCiAJbGlzdF9mb3JfZWFjaF9lbnRyeShtYXAsICZw cml2LT5tYXBzLCBuZXh0KSB7CiAJCXVubWFwX2lmX2luX3JhbmdlKG1hcCwgc3RhcnQsIGVuZCk7 CiAJfQpAQCAtNDgwLDYgKzQ4Niw4IEBAIHN0YXRpYyB2b2lkIG1uX2ludmxfcmFuZ2Vfc3RhcnQo c3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJCXVubWFwX2lmX2luX3JhbmdlKG1hcCwgc3RhcnQs IGVuZCk7CiAJfQogCW11dGV4X3VubG9jaygmcHJpdi0+bG9jayk7CisKKwlyZXR1cm4gdHJ1ZTsK IH0KIAogc3RhdGljIHZvaWQgbW5fcmVsZWFzZShzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKZGlm ZiAtLWdpdCBhL2luY2x1ZGUvbGludXgva3ZtX2hvc3QuaCBiL2luY2x1ZGUvbGludXgva3ZtX2hv c3QuaAppbmRleCA0ZWU3YmM1NDhhODMuLmU0MTgxMDYzZTc1NSAxMDA2NDQKLS0tIGEvaW5jbHVk ZS9saW51eC9rdm1faG9zdC5oCisrKyBiL2luY2x1ZGUvbGludXgva3ZtX2hvc3QuaApAQCAtMTI3 NSw3ICsxMjc1LDcgQEAgc3RhdGljIGlubGluZSBsb25nIGt2bV9hcmNoX3ZjcHVfYXN5bmNfaW9j dGwoc3RydWN0IGZpbGUgKmZpbHAsCiB9CiAjZW5kaWYgLyogQ09ORklHX0hBVkVfS1ZNX1ZDUFVf QVNZTkNfSU9DVEwgKi8KIAotdm9pZCBrdm1fYXJjaF9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9y YW5nZShzdHJ1Y3Qga3ZtICprdm0sCitpbnQga3ZtX2FyY2hfbW11X25vdGlmaWVyX2ludmFsaWRh dGVfcmFuZ2Uoc3RydWN0IGt2bSAqa3ZtLAogCQl1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25l ZCBsb25nIGVuZCk7CiAKICNpZmRlZiBDT05GSUdfSEFWRV9LVk1fVkNQVV9SVU5fUElEX0NIQU5H RQpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbXVfbm90aWZpZXIuaCBiL2luY2x1ZGUvbGlu dXgvbW11X25vdGlmaWVyLmgKaW5kZXggMzkyZTZhZjgyNzAxLi4zNjk4Njc1MDFiZWQgMTAwNjQ0 Ci0tLSBhL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmgKKysrIGIvaW5jbHVkZS9saW51eC9t bXVfbm90aWZpZXIuaApAQCAtMjMwLDcgKzIzMCw4IEBAIGV4dGVybiBpbnQgX19tbXVfbm90aWZp ZXJfdGVzdF95b3VuZyhzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIGV4dGVybiB2b2lkIF9fbW11X25v dGlmaWVyX2NoYW5nZV9wdGUoc3RydWN0IG1tX3N0cnVjdCAqbW0sCiAJCQkJICAgICAgdW5zaWdu ZWQgbG9uZyBhZGRyZXNzLCBwdGVfdCBwdGUpOwogZXh0ZXJuIHZvaWQgX19tbXVfbm90aWZpZXJf aW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW1fc3RydWN0ICptbSwKLQkJCQkgIHVuc2ln bmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kKTsKKwkJCQkgIHVuc2lnbmVkIGxvbmcg c3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCSAgYm9vbCBibG9ja2FibGUpOwogZXh0ZXJu IHZvaWQgX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tX3N0cnVj dCAqbW0sCiAJCQkJICB1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCwKIAkJ CQkgIGJvb2wgb25seV9lbmQpOwpAQCAtMjgxLDcgKzI4MiwxNyBAQCBzdGF0aWMgaW5saW5lIHZv aWQgbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tX3N0cnVjdCAq bW0sCiAJCQkJICB1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCkKIHsKIAlp ZiAobW1faGFzX25vdGlmaWVycyhtbSkpCi0JCV9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFu Z2Vfc3RhcnQobW0sIHN0YXJ0LCBlbmQpOworCQlfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KG1tLCBzdGFydCwgZW5kLCB0cnVlKTsKK30KKworc3RhdGljIGlubGluZSBpbnQg bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbm9uYmxvY2soc3RydWN0IG1tX3N0 cnVjdCAqbW0sCisJCQkJICB1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCkK K3sKKwlpbnQgcmV0ID0gMDsKKwlpZiAobW1faGFzX25vdGlmaWVycyhtbSkpCisJCXJldCA9IF9f bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQobW0sIHN0YXJ0LCBlbmQsIGZhbHNl KTsKKworCXJldHVybiByZXQ7CiB9CiAKIHN0YXRpYyBpbmxpbmUgdm9pZCBtbXVfbm90aWZpZXJf aW52YWxpZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tX3N0cnVjdCAqbW0sCmRpZmYgLS1naXQgYS9t bS9obW0uYyBiL21tL2htbS5jCmluZGV4IGRlN2I2YmY3NzIwMS4uODFmZDU3YmQyNjM0IDEwMDY0 NAotLS0gYS9tbS9obW0uYworKysgYi9tbS9obW0uYwpAQCAtMTc3LDE2ICsxNzcsMTkgQEAgc3Rh dGljIHZvaWQgaG1tX3JlbGVhc2Uoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sIHN0cnVjdCBtbV9z dHJ1Y3QgKm1tKQogCXVwX3dyaXRlKCZobW0tPm1pcnJvcnNfc2VtKTsKIH0KIAotc3RhdGljIHZv aWQgaG1tX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCitz dGF0aWMgaW50IGhtbV9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIg Km1uLAogCQkJCSAgICAgICBzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkgICAgICAgdW5zaWdu ZWQgbG9uZyBzdGFydCwKLQkJCQkgICAgICAgdW5zaWduZWQgbG9uZyBlbmQpCisJCQkJICAgICAg IHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCSAgICAgICBib29sIGJsb2NrYWJsZSkKIHsKIAlzdHJ1 Y3QgaG1tICpobW0gPSBtbS0+aG1tOwogCiAJVk1fQlVHX09OKCFobW0pOwogCiAJYXRvbWljX2lu YygmaG1tLT5zZXF1ZW5jZSk7CisKKwlyZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgaG1tX2lu dmFsaWRhdGVfcmFuZ2VfZW5kKHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLApkaWZmIC0tZ2l0IGEv bW0vbW11X25vdGlmaWVyLmMgYi9tbS9tbXVfbm90aWZpZXIuYwppbmRleCBlZmY2Yjg4YTk5M2Yu LjMwY2M0MzEyMWRhOSAxMDA2NDQKLS0tIGEvbW0vbW11X25vdGlmaWVyLmMKKysrIGIvbW0vbW11 X25vdGlmaWVyLmMKQEAgLTE3NCwxOCArMTc0LDI1IEBAIHZvaWQgX19tbXVfbm90aWZpZXJfY2hh bmdlX3B0ZShzdHJ1Y3QgbW1fc3RydWN0ICptbSwgdW5zaWduZWQgbG9uZyBhZGRyZXNzLAogCXNy Y3VfcmVhZF91bmxvY2soJnNyY3UsIGlkKTsKIH0KIAotdm9pZCBfX21tdV9ub3RpZmllcl9pbnZh bGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAotCQkJCSAgdW5zaWduZWQg bG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQpCitpbnQgX19tbXVfbm90aWZpZXJfaW52YWxp ZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW1fc3RydWN0ICptbSwKKwkJCQkgIHVuc2lnbmVkIGxv bmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCSAgYm9vbCBibG9ja2FibGUpCiB7CiAJ c3RydWN0IG1tdV9ub3RpZmllciAqbW47CisJaW50IHJldCA9IDA7CiAJaW50IGlkOwogCiAJaWQg PSBzcmN1X3JlYWRfbG9jaygmc3JjdSk7CiAJaGxpc3RfZm9yX2VhY2hfZW50cnlfcmN1KG1uLCAm bW0tPm1tdV9ub3RpZmllcl9tbS0+bGlzdCwgaGxpc3QpIHsKLQkJaWYgKG1uLT5vcHMtPmludmFs aWRhdGVfcmFuZ2Vfc3RhcnQpCi0JCQltbi0+b3BzLT5pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KG1u LCBtbSwgc3RhcnQsIGVuZCk7CisJCWlmIChtbi0+b3BzLT5pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0 KSB7CisJCQlpbnQgX3JldCA9IG1uLT5vcHMtPmludmFsaWRhdGVfcmFuZ2Vfc3RhcnQobW4sIG1t LCBzdGFydCwgZW5kLCBibG9ja2FibGUpOworCQkJaWYgKF9yZXQpCisJCQkJcmV0ID0gX3JldDsK KwkJfQogCX0KIAlzcmN1X3JlYWRfdW5sb2NrKCZzcmN1LCBpZCk7CisKKwlyZXR1cm4gcmV0Owog fQogRVhQT1JUX1NZTUJPTF9HUEwoX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFy dCk7CiAKZGlmZiAtLWdpdCBhL21tL29vbV9raWxsLmMgYi9tbS9vb21fa2lsbC5jCmluZGV4IDg0 MDgxZTc3YmM1MS4uN2UwYzZlNzhhZTVjIDEwMDY0NAotLS0gYS9tbS9vb21fa2lsbC5jCisrKyBi L21tL29vbV9raWxsLmMKQEAgLTQ3OSw5ICs0NzksMTAgQEAgc3RhdGljIERFQ0xBUkVfV0FJVF9R VUVVRV9IRUFEKG9vbV9yZWFwZXJfd2FpdCk7CiBzdGF0aWMgc3RydWN0IHRhc2tfc3RydWN0ICpv b21fcmVhcGVyX2xpc3Q7CiBzdGF0aWMgREVGSU5FX1NQSU5MT0NLKG9vbV9yZWFwZXJfbG9jayk7 CiAKLXZvaWQgX19vb21fcmVhcF90YXNrX21tKHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQorYm9vbCBf X29vbV9yZWFwX3Rhc2tfbW0oc3RydWN0IG1tX3N0cnVjdCAqbW0pCiB7CiAJc3RydWN0IHZtX2Fy ZWFfc3RydWN0ICp2bWE7CisJYm9vbCByZXQgPSB0cnVlOwogCiAJLyoKIAkgKiBUZWxsIGFsbCB1 c2VycyBvZiBnZXRfdXNlci9jb3B5X2Zyb21fdXNlciBldGMuLi4gdGhhdCB0aGUgY29udGVudApA QCAtNTExLDEyICs1MTIsMTcgQEAgdm9pZCBfX29vbV9yZWFwX3Rhc2tfbW0oc3RydWN0IG1tX3N0 cnVjdCAqbW0pCiAJCQlzdHJ1Y3QgbW11X2dhdGhlciB0bGI7CiAKIAkJCXRsYl9nYXRoZXJfbW11 KCZ0bGIsIG1tLCBzdGFydCwgZW5kKTsKLQkJCW1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jhbmdl X3N0YXJ0KG1tLCBzdGFydCwgZW5kKTsKKwkJCWlmIChtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9y YW5nZV9zdGFydF9ub25ibG9jayhtbSwgc3RhcnQsIGVuZCkpIHsKKwkJCQlyZXQgPSBmYWxzZTsK KwkJCQljb250aW51ZTsKKwkJCX0KIAkJCXVubWFwX3BhZ2VfcmFuZ2UoJnRsYiwgdm1hLCBzdGFy dCwgZW5kLCBOVUxMKTsKIAkJCW1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX2VuZChtbSwg c3RhcnQsIGVuZCk7CiAJCQl0bGJfZmluaXNoX21tdSgmdGxiLCBzdGFydCwgZW5kKTsKIAkJfQog CX0KKworCXJldHVybiByZXQ7CiB9CiAKIHN0YXRpYyBib29sIG9vbV9yZWFwX3Rhc2tfbW0oc3Ry dWN0IHRhc2tfc3RydWN0ICp0c2ssIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQpAQCAtNTQ1LDE4ICs1 NTEsNiBAQCBzdGF0aWMgYm9vbCBvb21fcmVhcF90YXNrX21tKHN0cnVjdCB0YXNrX3N0cnVjdCAq dHNrLCBzdHJ1Y3QgbW1fc3RydWN0ICptbSkKIAkJZ290byB1bmxvY2tfb29tOwogCX0KIAotCS8q Ci0JICogSWYgdGhlIG1tIGhhcyBpbnZhbGlkYXRlX3tzdGFydCxlbmR9KCkgbm90aWZpZXJzIHRo YXQgY291bGQgYmxvY2ssCi0JICogc2xlZXAgdG8gZ2l2ZSB0aGUgb29tIHZpY3RpbSBzb21lIG1v cmUgdGltZS4KLQkgKiBUT0RPOiB3ZSByZWFsbHkgd2FudCB0byBnZXQgcmlkIG9mIHRoaXMgdWds eSBoYWNrIGFuZCBtYWtlIHN1cmUgdGhhdAotCSAqIG5vdGlmaWVycyBjYW5ub3QgYmxvY2sgZm9y IHVuYm91bmRlZCBhbW91bnQgb2YgdGltZQotCSAqLwotCWlmIChtbV9oYXNfYmxvY2thYmxlX2lu dmFsaWRhdGVfbm90aWZpZXJzKG1tKSkgewotCQl1cF9yZWFkKCZtbS0+bW1hcF9zZW0pOwotCQlz Y2hlZHVsZV90aW1lb3V0X2lkbGUoSFopOwotCQlnb3RvIHVubG9ja19vb207Ci0JfQotCiAJLyoK IAkgKiBNTUZfT09NX1NLSVAgaXMgc2V0IGJ5IGV4aXRfbW1hcCB3aGVuIHRoZSBPT00gcmVhcGVy IGNhbid0CiAJICogd29yayBvbiB0aGUgbW0gYW55bW9yZS4gVGhlIGNoZWNrIGZvciBNTUZfT09N X1NLSVAgbXVzdCBydW4KQEAgLTU3MSw3ICs1NjUsMTIgQEAgc3RhdGljIGJvb2wgb29tX3JlYXBf dGFza19tbShzdHJ1Y3QgdGFza19zdHJ1Y3QgKnRzaywgc3RydWN0IG1tX3N0cnVjdCAqbW0pCiAK IAl0cmFjZV9zdGFydF90YXNrX3JlYXBpbmcodHNrLT5waWQpOwogCi0JX19vb21fcmVhcF90YXNr X21tKG1tKTsKKwkvKiBmYWlsZWQgdG8gcmVhcCBwYXJ0IG9mIHRoZSBhZGRyZXNzIHNwYWNlLiBU cnkgYWdhaW4gbGF0ZXIgKi8KKwlpZiAoIV9fb29tX3JlYXBfdGFza19tbShtbSkpIHsKKwkJdXBf cmVhZCgmbW0tPm1tYXBfc2VtKTsKKwkJcmV0ID0gZmFsc2U7CisJCWdvdG8gb3V0X3VubG9jazsK Kwl9CiAKIAlwcl9pbmZvKCJvb21fcmVhcGVyOiByZWFwZWQgcHJvY2VzcyAlZCAoJXMpLCBub3cg YW5vbi1yc3M6JWx1a0IsIGZpbGUtcnNzOiVsdWtCLCBzaG1lbS1yc3M6JWx1a0JcbiIsCiAJCQl0 YXNrX3BpZF9ucih0c2spLCB0c2stPmNvbW0sCmRpZmYgLS1naXQgYS92aXJ0L2t2bS9rdm1fbWFp bi5jIGIvdmlydC9rdm0va3ZtX21haW4uYwppbmRleCBhZGEyMWY0N2YyMmIuLjZmN2U3MDlkMjk0 NCAxMDA2NDQKLS0tIGEvdmlydC9rdm0va3ZtX21haW4uYworKysgYi92aXJ0L2t2bS9rdm1fbWFp bi5jCkBAIC0xMzUsNyArMTM1LDcgQEAgc3RhdGljIHZvaWQga3ZtX3VldmVudF9ub3RpZnlfY2hh bmdlKHVuc2lnbmVkIGludCB0eXBlLCBzdHJ1Y3Qga3ZtICprdm0pOwogc3RhdGljIHVuc2lnbmVk IGxvbmcgbG9uZyBrdm1fY3JlYXRldm1fY291bnQ7CiBzdGF0aWMgdW5zaWduZWQgbG9uZyBsb25n IGt2bV9hY3RpdmVfdm1zOwogCi1fX3dlYWsgdm9pZCBrdm1fYXJjaF9tbXVfbm90aWZpZXJfaW52 YWxpZGF0ZV9yYW5nZShzdHJ1Y3Qga3ZtICprdm0sCitfX3dlYWsgaW50IGt2bV9hcmNoX21tdV9u b3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKHN0cnVjdCBrdm0gKmt2bSwKIAkJdW5zaWduZWQgbG9u ZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQpCiB7CiB9CkBAIC0zNTQsMTMgKzM1NCwxNSBAQCBz dGF0aWMgdm9pZCBrdm1fbW11X25vdGlmaWVyX2NoYW5nZV9wdGUoc3RydWN0IG1tdV9ub3RpZmll ciAqbW4sCiAJc3JjdV9yZWFkX3VubG9jaygma3ZtLT5zcmN1LCBpZHgpOwogfQogCi1zdGF0aWMg dm9pZCBrdm1fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9u b3RpZmllciAqbW4sCitzdGF0aWMgaW50IGt2bV9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5n ZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJCQkJCSAgICBzdHJ1Y3QgbW1fc3Ry dWN0ICptbSwKIAkJCQkJCSAgICB1bnNpZ25lZCBsb25nIHN0YXJ0LAotCQkJCQkJICAgIHVuc2ln bmVkIGxvbmcgZW5kKQorCQkJCQkJICAgIHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCQkJICAgIGJv b2wgYmxvY2thYmxlKQogewogCXN0cnVjdCBrdm0gKmt2bSA9IG1tdV9ub3RpZmllcl90b19rdm0o bW4pOwogCWludCBuZWVkX3RsYl9mbHVzaCA9IDAsIGlkeDsKKwlpbnQgcmV0OwogCiAJaWR4ID0g c3JjdV9yZWFkX2xvY2soJmt2bS0+c3JjdSk7CiAJc3Bpbl9sb2NrKCZrdm0tPm1tdV9sb2NrKTsK QEAgLTM3OCw5ICszODAsMTEgQEAgc3RhdGljIHZvaWQga3ZtX21tdV9ub3RpZmllcl9pbnZhbGlk YXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCiAJc3Bpbl91bmxvY2so Jmt2bS0+bW11X2xvY2spOwogCi0Ja3ZtX2FyY2hfbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFu Z2Uoa3ZtLCBzdGFydCwgZW5kKTsKKwlyZXQgPSBrdm1fYXJjaF9tbXVfbm90aWZpZXJfaW52YWxp ZGF0ZV9yYW5nZShrdm0sIHN0YXJ0LCBlbmQsIGJsb2NrYWJsZSk7CiAKIAlzcmN1X3JlYWRfdW5s b2NrKCZrdm0tPnNyY3UsIGlkeCk7CisKKwlyZXR1cm4gcmV0OwogfQogCiBzdGF0aWMgdm9pZCBr dm1fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2VfZW5kKHN0cnVjdCBtbXVfbm90aWZpZXIg Km1uLAotLSAKMi4xNy4xCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXwpJbnRlbC1nZnggbWFpbGluZyBsaXN0CkludGVsLWdmeEBsaXN0cy5mcmVlZGVza3Rv cC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9pbnRl bC1nZngK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f198.google.com (mail-wr0-f198.google.com [209.85.128.198]) by kanga.kvack.org (Postfix) with ESMTP id A49EC6B0003 for ; Fri, 22 Jun 2018 11:02:53 -0400 (EDT) Received: by mail-wr0-f198.google.com with SMTP id i14-v6so4721299wrq.1 for ; Fri, 22 Jun 2018 08:02:53 -0700 (PDT) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id s10-v6sor529130wmc.82.2018.06.22.08.02.51 for (Google Transport Security); Fri, 22 Jun 2018 08:02:51 -0700 (PDT) From: Michal Hocko Subject: [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Fri, 22 Jun 2018 17:02:42 +0200 Message-Id: <20180622150242.16558-1-mhocko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: LKML Cc: Michal Hocko , "David (ChunMing) Zhou" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Alex Deucher , =?UTF-8?q?Christian=20K=C3=B6nig?= , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Doug Ledford , Jason Gunthorpe , Mike Marciniszyn , Dennis Dalessandro , Sudeep Dutt , Ashutosh Dixit , Dimitri Sivanich , Boris Ostrovsky , Juergen Gross , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrea Arcangeli , kvm@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-rdma@vger.kernel.org, xen-devel@lists.xenproject.org, linux-mm@kvack.org, David Rientjes From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an oom victim with blockable mmu notifiers as done after a short sleep. That can result in selecting a new oom victim prematurely because the previous one still hasn't torn its memory down yet. We can do much better though. Even if mmu notifiers use sleepable locks there is no reason to automatically assume those locks are held. Moreover most notifiers only care about a portion of the address space. This patch handles the first part of the problem. __mmu_notifier_invalidate_range_start gets a blockable flag and callbacks are not allowed to sleep if the flag is set to false. This is achieved by using trylock instead of the sleepable lock for most callbacks. I think we can improve that even further because there is a common pattern to do a range lookup first and then do something about that. The first part can be done without a sleeping lock I presume. Anyway, what does the oom_reaper do with all that? We do not have to fail right away. We simply retry if there is at least one notifier which couldn't make any progress. A retry loop is already implemented to wait for the mmap_sem and this is basically the same thing. Cc: "David (ChunMing) Zhou" Cc: Paolo Bonzini Cc: "Radim KrA?mA!A?" Cc: Alex Deucher Cc: "Christian KA?nig" Cc: David Airlie Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Rodrigo Vivi Cc: Doug Ledford Cc: Jason Gunthorpe Cc: Mike Marciniszyn Cc: Dennis Dalessandro Cc: Sudeep Dutt Cc: Ashutosh Dixit Cc: Dimitri Sivanich Cc: Boris Ostrovsky Cc: Juergen Gross Cc: "JA(C)rA'me Glisse" Cc: Andrea Arcangeli Cc: kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE FOR X86 (KVM/x86)) Cc: linux-kernel@vger.kernel.org (open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)) Cc: amd-gfx@lists.freedesktop.org (open list:RADEON and AMDGPU DRM DRIVERS) Cc: dri-devel@lists.freedesktop.org (open list:DRM DRIVERS) Cc: intel-gfx@lists.freedesktop.org (open list:INTEL DRM DRIVERS (excluding Poulsbo, Moorestow...) Cc: linux-rdma@vger.kernel.org (open list:INFINIBAND SUBSYSTEM) Cc: xen-devel@lists.xenproject.org (moderated list:XEN HYPERVISOR INTERFACE) Cc: linux-mm@kvack.org (open list:HMM - Heterogeneous Memory Management) Reported-by: David Rientjes Signed-off-by: Michal Hocko --- Hi, this is an RFC and not tested at all. I am not very familiar with the mmu notifiers semantics very much so this is a crude attempt to achieve what I need basically. It might be completely wrong but I would like to discuss what would be a better way if that is the case. get_maintainers gave me quite large list of people to CC so I had to trim it down. If you think I have forgot somebody, please let me know Any feedback is highly appreciated. arch/x86/kvm/x86.c | 7 ++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 33 +++++++++++++++++++------ drivers/gpu/drm/i915/i915_gem_userptr.c | 10 +++++--- drivers/gpu/drm/radeon/radeon_mn.c | 15 ++++++++--- drivers/infiniband/core/umem_odp.c | 15 ++++++++--- drivers/infiniband/hw/hfi1/mmu_rb.c | 7 ++++-- drivers/misc/mic/scif/scif_dma.c | 7 ++++-- drivers/misc/sgi-gru/grutlbpurge.c | 7 ++++-- drivers/xen/gntdev.c | 14 ++++++++--- include/linux/kvm_host.h | 2 +- include/linux/mmu_notifier.h | 15 +++++++++-- mm/hmm.c | 7 ++++-- mm/mmu_notifier.c | 15 ++++++++--- mm/oom_kill.c | 29 +++++++++++----------- virt/kvm/kvm_main.c | 12 ++++++--- 15 files changed, 137 insertions(+), 58 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6bcecc325e7e..ac08f5d711be 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7203,8 +7203,9 @@ static void vcpu_load_eoi_exitmap(struct kvm_vcpu *vcpu) kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap); } -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, + bool blockable) { unsigned long apic_address; @@ -7215,6 +7216,8 @@ void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, apic_address = gfn_to_hva(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); if (start <= apic_address && apic_address < end) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); + + return 0; } void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 83e344fbb50a..d138a526feff 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -136,12 +136,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) * * Take the rmn read side lock. */ -static void amdgpu_mn_read_lock(struct amdgpu_mn *rmn) +static int amdgpu_mn_read_lock(struct amdgpu_mn *rmn, bool blockable) { - mutex_lock(&rmn->read_lock); + if (blockable) + mutex_lock(&rmn->read_lock); + else if (!mutex_trylock(&rmn->read_lock)) + return -EAGAIN; + if (atomic_inc_return(&rmn->recursion) == 1) down_read_non_owner(&rmn->lock); mutex_unlock(&rmn->read_lock); + + return 0; } /** @@ -197,10 +203,11 @@ static void amdgpu_mn_invalidate_node(struct amdgpu_mn_node *node, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -208,7 +215,11 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + /* TODO we should be able to split locking for interval tree and + * amdgpu_mn_invalidate_node + */ + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -219,6 +230,8 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, amdgpu_mn_invalidate_node(node, start, end); } + + return 0; } /** @@ -233,10 +246,11 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, * necessitates evicting all user-mode queues of the process. The BOs * are restorted in amdgpu_mn_invalidate_range_end_hsa. */ -static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -244,7 +258,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -262,6 +277,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, amdgpu_amdkfd_evict_userptr(mem, mm); } } + + return 0; } /** diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 854bd51b9478..5285df9331fa 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -112,10 +112,11 @@ static void del_object(struct i915_mmu_object *mo) mo->attached = false; } -static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, +static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct i915_mmu_notifier *mn = container_of(_mn, struct i915_mmu_notifier, mn); @@ -124,7 +125,7 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, LIST_HEAD(cancelled); if (RB_EMPTY_ROOT(&mn->objects.rb_root)) - return; + return 0; /* interval ranges are inclusive, but invalidate range is exclusive */ end--; @@ -152,7 +153,8 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, del_object(mo); spin_unlock(&mn->lock); - if (!list_empty(&cancelled)) + /* TODO: can we skip waiting here? */ + if (!list_empty(&cancelled) && blockable) flush_workqueue(mn->wq); } diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index abd24975c9b1..b47e828b725d 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -118,10 +118,11 @@ static void radeon_mn_release(struct mmu_notifier *mn, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, +static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct radeon_mn *rmn = container_of(mn, struct radeon_mn, mn); struct ttm_operation_ctx ctx = { false, false }; @@ -130,7 +131,13 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - mutex_lock(&rmn->lock); + /* TODO we should be able to split locking for interval tree and + * the tear down. + */ + if (blockable) + mutex_lock(&rmn->lock); + else if (!mutex_trylock(&rmn->lock)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -167,6 +174,8 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, } mutex_unlock(&rmn->lock); + + return 0; } static const struct mmu_notifier_ops radeon_mn_ops = { diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index 182436b92ba9..f65f6a29daae 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -207,22 +207,29 @@ static int invalidate_range_start_trampoline(struct ib_umem *item, u64 start, return 0; } -static void ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct ib_ucontext *context = container_of(mn, struct ib_ucontext, mn); if (!context->invalidate_range) - return; + return 0; + + if (blockable) + down_read(&context->umem_rwsem); + else if (!down_read_trylock(&context->umem_rwsem)) + return -EAGAIN; ib_ucontext_notifier_start_account(context); - down_read(&context->umem_rwsem); rbt_ib_umem_for_each_in_range(&context->umem_tree, start, end, invalidate_range_start_trampoline, NULL); up_read(&context->umem_rwsem); + + return 0; } static int invalidate_range_end_trampoline(struct ib_umem *item, u64 start, diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index 70aceefe14d5..8780560d1623 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -284,10 +284,11 @@ void hfi1_mmu_rb_remove(struct mmu_rb_handler *handler, handler->ops->remove(handler->ops_arg, node); } -static void mmu_notifier_range_start(struct mmu_notifier *mn, +static int mmu_notifier_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct mmu_rb_handler *handler = container_of(mn, struct mmu_rb_handler, mn); @@ -313,6 +314,8 @@ static void mmu_notifier_range_start(struct mmu_notifier *mn, if (added) queue_work(handler->wq, &handler->del_work); + + return 0; } /* diff --git a/drivers/misc/mic/scif/scif_dma.c b/drivers/misc/mic/scif/scif_dma.c index 63d6246d6dff..d940568bed87 100644 --- a/drivers/misc/mic/scif/scif_dma.c +++ b/drivers/misc/mic/scif/scif_dma.c @@ -200,15 +200,18 @@ static void scif_mmu_notifier_release(struct mmu_notifier *mn, schedule_work(&scif_info.misc_work); } -static void scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct scif_mmu_notif *mmn; mmn = container_of(mn, struct scif_mmu_notif, ep_mmu_notifier); scif_rma_destroy_tcw(mmn, start, end - start); + + return 0 } static void scif_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/misc/sgi-gru/grutlbpurge.c b/drivers/misc/sgi-gru/grutlbpurge.c index a3454eb56fbf..be28f05bfafa 100644 --- a/drivers/misc/sgi-gru/grutlbpurge.c +++ b/drivers/misc/sgi-gru/grutlbpurge.c @@ -219,9 +219,10 @@ void gru_flush_all_tlb(struct gru_state *gru) /* * MMUOPS notifier callout functions */ -static void gru_invalidate_range_start(struct mmu_notifier *mn, +static int gru_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gru_mm_struct *gms = container_of(mn, struct gru_mm_struct, ms_notifier); @@ -231,6 +232,8 @@ static void gru_invalidate_range_start(struct mmu_notifier *mn, gru_dbg(grudev, "gms %p, start 0x%lx, end 0x%lx, act %d\n", gms, start, end, atomic_read(&gms->ms_range_active)); gru_flush_tlb_range(gms, start, end - start); + + return 0; } static void gru_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index bd56653b9bbc..50724d09fe5c 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -465,14 +465,20 @@ static void unmap_if_in_range(struct grant_map *map, WARN_ON(err); } -static void mn_invl_range_start(struct mmu_notifier *mn, +static int mn_invl_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gntdev_priv *priv = container_of(mn, struct gntdev_priv, mn); struct grant_map *map; - mutex_lock(&priv->lock); + /* TODO do we really need a mutex here? */ + if (blockable) + mutex_lock(&priv->lock); + else if (!mutex_trylock(&priv->lock)) + return -EAGAIN; + list_for_each_entry(map, &priv->maps, next) { unmap_if_in_range(map, start, end); } @@ -480,6 +486,8 @@ static void mn_invl_range_start(struct mmu_notifier *mn, unmap_if_in_range(map, start, end); } mutex_unlock(&priv->lock); + + return true; } static void mn_release(struct mmu_notifier *mn, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4ee7bc548a83..e4181063e755 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1275,7 +1275,7 @@ static inline long kvm_arch_vcpu_async_ioctl(struct file *filp, } #endif /* CONFIG_HAVE_KVM_VCPU_ASYNC_IOCTL */ -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, unsigned long start, unsigned long end); #ifdef CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 392e6af82701..369867501bed 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -230,7 +230,8 @@ extern int __mmu_notifier_test_young(struct mm_struct *mm, extern void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, pte_t pte); extern void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end); + unsigned long start, unsigned long end, + bool blockable); extern void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, unsigned long start, unsigned long end, bool only_end); @@ -281,7 +282,17 @@ static inline void mmu_notifier_invalidate_range_start(struct mm_struct *mm, unsigned long start, unsigned long end) { if (mm_has_notifiers(mm)) - __mmu_notifier_invalidate_range_start(mm, start, end); + __mmu_notifier_invalidate_range_start(mm, start, end, true); +} + +static inline int mmu_notifier_invalidate_range_start_nonblock(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + int ret = 0; + if (mm_has_notifiers(mm)) + ret = __mmu_notifier_invalidate_range_start(mm, start, end, false); + + return ret; } static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, diff --git a/mm/hmm.c b/mm/hmm.c index de7b6bf77201..81fd57bd2634 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -177,16 +177,19 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) up_write(&hmm->mirrors_sem); } -static void hmm_invalidate_range_start(struct mmu_notifier *mn, +static int hmm_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); atomic_inc(&hmm->sequence); + + return 0; } static void hmm_invalidate_range_end(struct mmu_notifier *mn, diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index eff6b88a993f..30cc43121da9 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -174,18 +174,25 @@ void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, srcu_read_unlock(&srcu, id); } -void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end) +int __mmu_notifier_invalidate_range_start(struct mm_struct *mm, + unsigned long start, unsigned long end, + bool blockable) { struct mmu_notifier *mn; + int ret = 0; int id; id = srcu_read_lock(&srcu); hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) { - if (mn->ops->invalidate_range_start) - mn->ops->invalidate_range_start(mn, mm, start, end); + if (mn->ops->invalidate_range_start) { + int _ret = mn->ops->invalidate_range_start(mn, mm, start, end, blockable); + if (_ret) + ret = _ret; + } } srcu_read_unlock(&srcu, id); + + return ret; } EXPORT_SYMBOL_GPL(__mmu_notifier_invalidate_range_start); diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 84081e77bc51..7e0c6e78ae5c 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -479,9 +479,10 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); static struct task_struct *oom_reaper_list; static DEFINE_SPINLOCK(oom_reaper_lock); -void __oom_reap_task_mm(struct mm_struct *mm) +bool __oom_reap_task_mm(struct mm_struct *mm) { struct vm_area_struct *vma; + bool ret = true; /* * Tell all users of get_user/copy_from_user etc... that the content @@ -511,12 +512,17 @@ void __oom_reap_task_mm(struct mm_struct *mm) struct mmu_gather tlb; tlb_gather_mmu(&tlb, mm, start, end); - mmu_notifier_invalidate_range_start(mm, start, end); + if (mmu_notifier_invalidate_range_start_nonblock(mm, start, end)) { + ret = false; + continue; + } unmap_page_range(&tlb, vma, start, end, NULL); mmu_notifier_invalidate_range_end(mm, start, end); tlb_finish_mmu(&tlb, start, end); } } + + return ret; } static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) @@ -545,18 +551,6 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) goto unlock_oom; } - /* - * If the mm has invalidate_{start,end}() notifiers that could block, - * sleep to give the oom victim some more time. - * TODO: we really want to get rid of this ugly hack and make sure that - * notifiers cannot block for unbounded amount of time - */ - if (mm_has_blockable_invalidate_notifiers(mm)) { - up_read(&mm->mmap_sem); - schedule_timeout_idle(HZ); - goto unlock_oom; - } - /* * MMF_OOM_SKIP is set by exit_mmap when the OOM reaper can't * work on the mm anymore. The check for MMF_OOM_SKIP must run @@ -571,7 +565,12 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) trace_start_task_reaping(tsk->pid); - __oom_reap_task_mm(mm); + /* failed to reap part of the address space. Try again later */ + if (!__oom_reap_task_mm(mm)) { + up_read(&mm->mmap_sem); + ret = false; + goto out_unlock; + } pr_info("oom_reaper: reaped process %d (%s), now anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n", task_pid_nr(tsk), tsk->comm, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ada21f47f22b..6f7e709d2944 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -135,7 +135,7 @@ static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm); static unsigned long long kvm_createvm_count; static unsigned long long kvm_active_vms; -__weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, +__weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, unsigned long start, unsigned long end) { } @@ -354,13 +354,15 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn, srcu_read_unlock(&kvm->srcu, idx); } -static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct kvm *kvm = mmu_notifier_to_kvm(mn); int need_tlb_flush = 0, idx; + int ret; idx = srcu_read_lock(&kvm->srcu); spin_lock(&kvm->mmu_lock); @@ -378,9 +380,11 @@ static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, spin_unlock(&kvm->mmu_lock); - kvm_arch_mmu_notifier_invalidate_range(kvm, start, end); + ret = kvm_arch_mmu_notifier_invalidate_range(kvm, start, end, blockable); srcu_read_unlock(&kvm->srcu, idx); + + return ret; } static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=3.0 tests=MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A70BEC43144 for ; Fri, 22 Jun 2018 15:02:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 47BDA24451 for ; Fri, 22 Jun 2018 15:02:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 47BDA24451 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754157AbeFVPCz (ORCPT ); Fri, 22 Jun 2018 11:02:55 -0400 Received: from mail-wm0-f67.google.com ([74.125.82.67]:35635 "EHLO mail-wm0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751233AbeFVPCw (ORCPT ); Fri, 22 Jun 2018 11:02:52 -0400 Received: by mail-wm0-f67.google.com with SMTP id j15-v6so3017626wme.0; Fri, 22 Jun 2018 08:02:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=tKURP0Iqzw6WDEYIBjBI/IKNNwPWLBJ7NluyEljguLs=; b=adyF9cVd1oZfm5HMLCPshoMichtn91w7EMODcKAYFWINgsuMtDCZxyALy4OTWo36jy 5Hj2+99bvjTVhSEcJ81UpD4UCZBnpf8yxbozLfJo5+XFhBmRwy4p/1vMIujeEhLAYKtI m6onW153qtfuht8o+Z2Cie8uxpijIjNMhU7rw1nmKLKuZ3wtmLZteNkRM1e+s82QchlF lkV/7XSH6eGO+caXZvm431dAaxrYmzrujLxpZtZBV0IAPuHlAUk6RR5IDtOgFrx01x0L K8uT3G/gkMUC8NWjH6pYPeCjqWC5aqxIg/+tHKDeF7+kkShkRYbhgge++zWGG2UcjwQp JF1A== X-Gm-Message-State: APt69E2+5RCOItBQoRk/e++iU1nSMKdpm63atbwbxmJz3PGT9rnuOhZQ 4AmDZ4qwph9Yz4d1n9uVRVwKeUoZ X-Google-Smtp-Source: AAOMgpfJ3ny/7Bge5ODf1JQaH/j16mc3bNS+Ov41J5EMNqnXjYsSjp06qpfJ8FDlgo/UNnQ4Zc88Hw== X-Received: by 2002:a1c:e306:: with SMTP id a6-v6mr1890430wmh.15.1529679770844; Fri, 22 Jun 2018 08:02:50 -0700 (PDT) Received: from tiehlicka.suse.cz (ip-37-188-179-217.eurotel.cz. [37.188.179.217]) by smtp.gmail.com with ESMTPSA id d102-v6sm2623463wma.10.2018.06.22.08.02.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Jun 2018 08:02:49 -0700 (PDT) From: Michal Hocko To: LKML Cc: Michal Hocko , "David (ChunMing) Zhou" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Alex Deucher , =?UTF-8?q?Christian=20K=C3=B6nig?= , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Doug Ledford , Jason Gunthorpe , Mike Marciniszyn , Dennis Dalessandro , Sudeep Dutt , Ashutosh Dixit , Dimitri Sivanich , Boris Ostrovsky , Juergen Gross , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrea Arcangeli , kvm@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-rdma@vger.kernel.org, xen-devel@lists.xenproject.org, linux-mm@kvack.org, David Rientjes Subject: [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Fri, 22 Jun 2018 17:02:42 +0200 Message-Id: <20180622150242.16558-1-mhocko@kernel.org> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an oom victim with blockable mmu notifiers as done after a short sleep. That can result in selecting a new oom victim prematurely because the previous one still hasn't torn its memory down yet. We can do much better though. Even if mmu notifiers use sleepable locks there is no reason to automatically assume those locks are held. Moreover most notifiers only care about a portion of the address space. This patch handles the first part of the problem. __mmu_notifier_invalidate_range_start gets a blockable flag and callbacks are not allowed to sleep if the flag is set to false. This is achieved by using trylock instead of the sleepable lock for most callbacks. I think we can improve that even further because there is a common pattern to do a range lookup first and then do something about that. The first part can be done without a sleeping lock I presume. Anyway, what does the oom_reaper do with all that? We do not have to fail right away. We simply retry if there is at least one notifier which couldn't make any progress. A retry loop is already implemented to wait for the mmap_sem and this is basically the same thing. Cc: "David (ChunMing) Zhou" Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Alex Deucher Cc: "Christian König" Cc: David Airlie Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Rodrigo Vivi Cc: Doug Ledford Cc: Jason Gunthorpe Cc: Mike Marciniszyn Cc: Dennis Dalessandro Cc: Sudeep Dutt Cc: Ashutosh Dixit Cc: Dimitri Sivanich Cc: Boris Ostrovsky Cc: Juergen Gross Cc: "Jérôme Glisse" Cc: Andrea Arcangeli Cc: kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE FOR X86 (KVM/x86)) Cc: linux-kernel@vger.kernel.org (open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)) Cc: amd-gfx@lists.freedesktop.org (open list:RADEON and AMDGPU DRM DRIVERS) Cc: dri-devel@lists.freedesktop.org (open list:DRM DRIVERS) Cc: intel-gfx@lists.freedesktop.org (open list:INTEL DRM DRIVERS (excluding Poulsbo, Moorestow...) Cc: linux-rdma@vger.kernel.org (open list:INFINIBAND SUBSYSTEM) Cc: xen-devel@lists.xenproject.org (moderated list:XEN HYPERVISOR INTERFACE) Cc: linux-mm@kvack.org (open list:HMM - Heterogeneous Memory Management) Reported-by: David Rientjes Signed-off-by: Michal Hocko --- Hi, this is an RFC and not tested at all. I am not very familiar with the mmu notifiers semantics very much so this is a crude attempt to achieve what I need basically. It might be completely wrong but I would like to discuss what would be a better way if that is the case. get_maintainers gave me quite large list of people to CC so I had to trim it down. If you think I have forgot somebody, please let me know Any feedback is highly appreciated. arch/x86/kvm/x86.c | 7 ++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 33 +++++++++++++++++++------ drivers/gpu/drm/i915/i915_gem_userptr.c | 10 +++++--- drivers/gpu/drm/radeon/radeon_mn.c | 15 ++++++++--- drivers/infiniband/core/umem_odp.c | 15 ++++++++--- drivers/infiniband/hw/hfi1/mmu_rb.c | 7 ++++-- drivers/misc/mic/scif/scif_dma.c | 7 ++++-- drivers/misc/sgi-gru/grutlbpurge.c | 7 ++++-- drivers/xen/gntdev.c | 14 ++++++++--- include/linux/kvm_host.h | 2 +- include/linux/mmu_notifier.h | 15 +++++++++-- mm/hmm.c | 7 ++++-- mm/mmu_notifier.c | 15 ++++++++--- mm/oom_kill.c | 29 +++++++++++----------- virt/kvm/kvm_main.c | 12 ++++++--- 15 files changed, 137 insertions(+), 58 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6bcecc325e7e..ac08f5d711be 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7203,8 +7203,9 @@ static void vcpu_load_eoi_exitmap(struct kvm_vcpu *vcpu) kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap); } -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, + bool blockable) { unsigned long apic_address; @@ -7215,6 +7216,8 @@ void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, apic_address = gfn_to_hva(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); if (start <= apic_address && apic_address < end) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); + + return 0; } void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 83e344fbb50a..d138a526feff 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -136,12 +136,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) * * Take the rmn read side lock. */ -static void amdgpu_mn_read_lock(struct amdgpu_mn *rmn) +static int amdgpu_mn_read_lock(struct amdgpu_mn *rmn, bool blockable) { - mutex_lock(&rmn->read_lock); + if (blockable) + mutex_lock(&rmn->read_lock); + else if (!mutex_trylock(&rmn->read_lock)) + return -EAGAIN; + if (atomic_inc_return(&rmn->recursion) == 1) down_read_non_owner(&rmn->lock); mutex_unlock(&rmn->read_lock); + + return 0; } /** @@ -197,10 +203,11 @@ static void amdgpu_mn_invalidate_node(struct amdgpu_mn_node *node, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -208,7 +215,11 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + /* TODO we should be able to split locking for interval tree and + * amdgpu_mn_invalidate_node + */ + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -219,6 +230,8 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, amdgpu_mn_invalidate_node(node, start, end); } + + return 0; } /** @@ -233,10 +246,11 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, * necessitates evicting all user-mode queues of the process. The BOs * are restorted in amdgpu_mn_invalidate_range_end_hsa. */ -static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -244,7 +258,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -262,6 +277,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, amdgpu_amdkfd_evict_userptr(mem, mm); } } + + return 0; } /** diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 854bd51b9478..5285df9331fa 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -112,10 +112,11 @@ static void del_object(struct i915_mmu_object *mo) mo->attached = false; } -static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, +static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct i915_mmu_notifier *mn = container_of(_mn, struct i915_mmu_notifier, mn); @@ -124,7 +125,7 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, LIST_HEAD(cancelled); if (RB_EMPTY_ROOT(&mn->objects.rb_root)) - return; + return 0; /* interval ranges are inclusive, but invalidate range is exclusive */ end--; @@ -152,7 +153,8 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, del_object(mo); spin_unlock(&mn->lock); - if (!list_empty(&cancelled)) + /* TODO: can we skip waiting here? */ + if (!list_empty(&cancelled) && blockable) flush_workqueue(mn->wq); } diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index abd24975c9b1..b47e828b725d 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -118,10 +118,11 @@ static void radeon_mn_release(struct mmu_notifier *mn, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, +static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct radeon_mn *rmn = container_of(mn, struct radeon_mn, mn); struct ttm_operation_ctx ctx = { false, false }; @@ -130,7 +131,13 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - mutex_lock(&rmn->lock); + /* TODO we should be able to split locking for interval tree and + * the tear down. + */ + if (blockable) + mutex_lock(&rmn->lock); + else if (!mutex_trylock(&rmn->lock)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -167,6 +174,8 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, } mutex_unlock(&rmn->lock); + + return 0; } static const struct mmu_notifier_ops radeon_mn_ops = { diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index 182436b92ba9..f65f6a29daae 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -207,22 +207,29 @@ static int invalidate_range_start_trampoline(struct ib_umem *item, u64 start, return 0; } -static void ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct ib_ucontext *context = container_of(mn, struct ib_ucontext, mn); if (!context->invalidate_range) - return; + return 0; + + if (blockable) + down_read(&context->umem_rwsem); + else if (!down_read_trylock(&context->umem_rwsem)) + return -EAGAIN; ib_ucontext_notifier_start_account(context); - down_read(&context->umem_rwsem); rbt_ib_umem_for_each_in_range(&context->umem_tree, start, end, invalidate_range_start_trampoline, NULL); up_read(&context->umem_rwsem); + + return 0; } static int invalidate_range_end_trampoline(struct ib_umem *item, u64 start, diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index 70aceefe14d5..8780560d1623 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -284,10 +284,11 @@ void hfi1_mmu_rb_remove(struct mmu_rb_handler *handler, handler->ops->remove(handler->ops_arg, node); } -static void mmu_notifier_range_start(struct mmu_notifier *mn, +static int mmu_notifier_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct mmu_rb_handler *handler = container_of(mn, struct mmu_rb_handler, mn); @@ -313,6 +314,8 @@ static void mmu_notifier_range_start(struct mmu_notifier *mn, if (added) queue_work(handler->wq, &handler->del_work); + + return 0; } /* diff --git a/drivers/misc/mic/scif/scif_dma.c b/drivers/misc/mic/scif/scif_dma.c index 63d6246d6dff..d940568bed87 100644 --- a/drivers/misc/mic/scif/scif_dma.c +++ b/drivers/misc/mic/scif/scif_dma.c @@ -200,15 +200,18 @@ static void scif_mmu_notifier_release(struct mmu_notifier *mn, schedule_work(&scif_info.misc_work); } -static void scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct scif_mmu_notif *mmn; mmn = container_of(mn, struct scif_mmu_notif, ep_mmu_notifier); scif_rma_destroy_tcw(mmn, start, end - start); + + return 0 } static void scif_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/misc/sgi-gru/grutlbpurge.c b/drivers/misc/sgi-gru/grutlbpurge.c index a3454eb56fbf..be28f05bfafa 100644 --- a/drivers/misc/sgi-gru/grutlbpurge.c +++ b/drivers/misc/sgi-gru/grutlbpurge.c @@ -219,9 +219,10 @@ void gru_flush_all_tlb(struct gru_state *gru) /* * MMUOPS notifier callout functions */ -static void gru_invalidate_range_start(struct mmu_notifier *mn, +static int gru_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gru_mm_struct *gms = container_of(mn, struct gru_mm_struct, ms_notifier); @@ -231,6 +232,8 @@ static void gru_invalidate_range_start(struct mmu_notifier *mn, gru_dbg(grudev, "gms %p, start 0x%lx, end 0x%lx, act %d\n", gms, start, end, atomic_read(&gms->ms_range_active)); gru_flush_tlb_range(gms, start, end - start); + + return 0; } static void gru_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index bd56653b9bbc..50724d09fe5c 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -465,14 +465,20 @@ static void unmap_if_in_range(struct grant_map *map, WARN_ON(err); } -static void mn_invl_range_start(struct mmu_notifier *mn, +static int mn_invl_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gntdev_priv *priv = container_of(mn, struct gntdev_priv, mn); struct grant_map *map; - mutex_lock(&priv->lock); + /* TODO do we really need a mutex here? */ + if (blockable) + mutex_lock(&priv->lock); + else if (!mutex_trylock(&priv->lock)) + return -EAGAIN; + list_for_each_entry(map, &priv->maps, next) { unmap_if_in_range(map, start, end); } @@ -480,6 +486,8 @@ static void mn_invl_range_start(struct mmu_notifier *mn, unmap_if_in_range(map, start, end); } mutex_unlock(&priv->lock); + + return true; } static void mn_release(struct mmu_notifier *mn, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4ee7bc548a83..e4181063e755 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1275,7 +1275,7 @@ static inline long kvm_arch_vcpu_async_ioctl(struct file *filp, } #endif /* CONFIG_HAVE_KVM_VCPU_ASYNC_IOCTL */ -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, unsigned long start, unsigned long end); #ifdef CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 392e6af82701..369867501bed 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -230,7 +230,8 @@ extern int __mmu_notifier_test_young(struct mm_struct *mm, extern void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, pte_t pte); extern void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end); + unsigned long start, unsigned long end, + bool blockable); extern void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, unsigned long start, unsigned long end, bool only_end); @@ -281,7 +282,17 @@ static inline void mmu_notifier_invalidate_range_start(struct mm_struct *mm, unsigned long start, unsigned long end) { if (mm_has_notifiers(mm)) - __mmu_notifier_invalidate_range_start(mm, start, end); + __mmu_notifier_invalidate_range_start(mm, start, end, true); +} + +static inline int mmu_notifier_invalidate_range_start_nonblock(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + int ret = 0; + if (mm_has_notifiers(mm)) + ret = __mmu_notifier_invalidate_range_start(mm, start, end, false); + + return ret; } static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, diff --git a/mm/hmm.c b/mm/hmm.c index de7b6bf77201..81fd57bd2634 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -177,16 +177,19 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) up_write(&hmm->mirrors_sem); } -static void hmm_invalidate_range_start(struct mmu_notifier *mn, +static int hmm_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); atomic_inc(&hmm->sequence); + + return 0; } static void hmm_invalidate_range_end(struct mmu_notifier *mn, diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index eff6b88a993f..30cc43121da9 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -174,18 +174,25 @@ void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, srcu_read_unlock(&srcu, id); } -void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end) +int __mmu_notifier_invalidate_range_start(struct mm_struct *mm, + unsigned long start, unsigned long end, + bool blockable) { struct mmu_notifier *mn; + int ret = 0; int id; id = srcu_read_lock(&srcu); hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) { - if (mn->ops->invalidate_range_start) - mn->ops->invalidate_range_start(mn, mm, start, end); + if (mn->ops->invalidate_range_start) { + int _ret = mn->ops->invalidate_range_start(mn, mm, start, end, blockable); + if (_ret) + ret = _ret; + } } srcu_read_unlock(&srcu, id); + + return ret; } EXPORT_SYMBOL_GPL(__mmu_notifier_invalidate_range_start); diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 84081e77bc51..7e0c6e78ae5c 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -479,9 +479,10 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); static struct task_struct *oom_reaper_list; static DEFINE_SPINLOCK(oom_reaper_lock); -void __oom_reap_task_mm(struct mm_struct *mm) +bool __oom_reap_task_mm(struct mm_struct *mm) { struct vm_area_struct *vma; + bool ret = true; /* * Tell all users of get_user/copy_from_user etc... that the content @@ -511,12 +512,17 @@ void __oom_reap_task_mm(struct mm_struct *mm) struct mmu_gather tlb; tlb_gather_mmu(&tlb, mm, start, end); - mmu_notifier_invalidate_range_start(mm, start, end); + if (mmu_notifier_invalidate_range_start_nonblock(mm, start, end)) { + ret = false; + continue; + } unmap_page_range(&tlb, vma, start, end, NULL); mmu_notifier_invalidate_range_end(mm, start, end); tlb_finish_mmu(&tlb, start, end); } } + + return ret; } static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) @@ -545,18 +551,6 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) goto unlock_oom; } - /* - * If the mm has invalidate_{start,end}() notifiers that could block, - * sleep to give the oom victim some more time. - * TODO: we really want to get rid of this ugly hack and make sure that - * notifiers cannot block for unbounded amount of time - */ - if (mm_has_blockable_invalidate_notifiers(mm)) { - up_read(&mm->mmap_sem); - schedule_timeout_idle(HZ); - goto unlock_oom; - } - /* * MMF_OOM_SKIP is set by exit_mmap when the OOM reaper can't * work on the mm anymore. The check for MMF_OOM_SKIP must run @@ -571,7 +565,12 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) trace_start_task_reaping(tsk->pid); - __oom_reap_task_mm(mm); + /* failed to reap part of the address space. Try again later */ + if (!__oom_reap_task_mm(mm)) { + up_read(&mm->mmap_sem); + ret = false; + goto out_unlock; + } pr_info("oom_reaper: reaped process %d (%s), now anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n", task_pid_nr(tsk), tsk->comm, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ada21f47f22b..6f7e709d2944 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -135,7 +135,7 @@ static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm); static unsigned long long kvm_createvm_count; static unsigned long long kvm_active_vms; -__weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, +__weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, unsigned long start, unsigned long end) { } @@ -354,13 +354,15 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn, srcu_read_unlock(&kvm->srcu, idx); } -static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct kvm *kvm = mmu_notifier_to_kvm(mn); int need_tlb_flush = 0, idx; + int ret; idx = srcu_read_lock(&kvm->srcu); spin_lock(&kvm->mmu_lock); @@ -378,9 +380,11 @@ static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, spin_unlock(&kvm->mmu_lock); - kvm_arch_mmu_notifier_invalidate_range(kvm, start, end); + ret = kvm_arch_mmu_notifier_invalidate_range(kvm, start, end, blockable); srcu_read_unlock(&kvm->srcu, idx); + + return ret; } static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, -- 2.17.1