From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michal Hocko Subject: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Mon, 16 Jul 2018 13:50:58 +0200 Message-ID: <20180716115058.5559-1-mhocko@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Andrew Morton Cc: Michal Hocko , kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , David Airlie , Joonas Lahtinen , Sudeep Dutt , dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Andrea Arcangeli , "David (ChunMing) Zhou" , Dimitri Sivanich , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Jason Gunthorpe , Doug Ledford , David Rientjes , xen-devel-GuqFBffKawtpuQazS67q72D2FQJk+8+b@public.gmane.org, intel-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Jani Nikula , Leon Romanovsky , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Rodrigo Vivi , Boris Ostrovsky , Juergen RnJvbTogTWljaGFsIEhvY2tvIDxtaG9ja29Ac3VzZS5jb20+CgpUaGVyZSBhcmUgc2V2ZXJhbCBi bG9ja2FibGUgbW11IG5vdGlmaWVycyB3aGljaCBtaWdodCBzbGVlcCBpbgptbXVfbm90aWZpZXJf aW52YWxpZGF0ZV9yYW5nZV9zdGFydCBhbmQgdGhhdCBpcyBhIHByb2JsZW0gZm9yIHRoZQpvb21f cmVhcGVyIGJlY2F1c2UgaXQgbmVlZHMgdG8gZ3VhcmFudGVlIGEgZm9yd2FyZCBwcm9ncmVzcyBz byBpdCBjYW5ub3QKZGVwZW5kIG9uIGFueSBzbGVlcGFibGUgbG9ja3MuCgpDdXJyZW50bHkgd2Ug c2ltcGx5IGJhY2sgb2ZmIGFuZCBtYXJrIGFuIG9vbSB2aWN0aW0gd2l0aCBibG9ja2FibGUgbW11 Cm5vdGlmaWVycyBhcyBkb25lIGFmdGVyIGEgc2hvcnQgc2xlZXAuIFRoYXQgY2FuIHJlc3VsdCBp biBzZWxlY3RpbmcgYQpuZXcgb29tIHZpY3RpbSBwcmVtYXR1cmVseSBiZWNhdXNlIHRoZSBwcmV2 aW91cyBvbmUgc3RpbGwgaGFzbid0IHRvcm4KaXRzIG1lbW9yeSBkb3duIHlldC4KCldlIGNhbiBk byBtdWNoIGJldHRlciB0aG91Z2guIEV2ZW4gaWYgbW11IG5vdGlmaWVycyB1c2Ugc2xlZXBhYmxl IGxvY2tzCnRoZXJlIGlzIG5vIHJlYXNvbiB0byBhdXRvbWF0aWNhbGx5IGFzc3VtZSB0aG9zZSBs b2NrcyBhcmUgaGVsZC4KTW9yZW92ZXIgbWFqb3JpdHkgb2Ygbm90aWZpZXJzIG9ubHkgY2FyZSBh Ym91dCBhIHBvcnRpb24gb2YgdGhlIGFkZHJlc3MKc3BhY2UgYW5kIHRoZXJlIGlzIGFic29sdXRl bHkgemVybyByZWFzb24gdG8gZmFpbCB3aGVuIHdlIGFyZSB1bm1hcHBpbmcgYW4KdW5yZWxhdGVk IHJhbmdlLiBNYW55IG5vdGlmaWVycyBkbyByZWFsbHkgYmxvY2sgYW5kIHdhaXQgZm9yIEhXIHdo aWNoIGlzCmhhcmRlciB0byBoYW5kbGUgYW5kIHdlIGhhdmUgdG8gYmFpbCBvdXQgdGhvdWdoLgoK VGhpcyBwYXRjaCBoYW5kbGVzIHRoZSBsb3cgaGFuZ2luZyBmcnVpZC4gX19tbXVfbm90aWZpZXJf aW52YWxpZGF0ZV9yYW5nZV9zdGFydApnZXRzIGEgYmxvY2thYmxlIGZsYWcgYW5kIGNhbGxiYWNr cyBhcmUgbm90IGFsbG93ZWQgdG8gc2xlZXAgaWYgdGhlCmZsYWcgaXMgc2V0IHRvIGZhbHNlLiBU aGlzIGlzIGFjaGlldmVkIGJ5IHVzaW5nIHRyeWxvY2sgaW5zdGVhZCBvZiB0aGUKc2xlZXBhYmxl IGxvY2sgZm9yIG1vc3QgY2FsbGJhY2tzIGFuZCBjb250aW51ZSBhcyBsb25nIGFzIHdlIGRvIG5v dApibG9jayBkb3duIHRoZSBjYWxsIGNoYWluLgoKSSB0aGluayB3ZSBjYW4gaW1wcm92ZSB0aGF0 IGV2ZW4gZnVydGhlciBiZWNhdXNlIHRoZXJlIGlzIGEgY29tbW9uCnBhdHRlcm4gdG8gZG8gYSBy YW5nZSBsb29rdXAgZmlyc3QgYW5kIHRoZW4gZG8gc29tZXRoaW5nIGFib3V0IHRoYXQuClRoZSBm aXJzdCBwYXJ0IGNhbiBiZSBkb25lIHdpdGhvdXQgYSBzbGVlcGluZyBsb2NrIGluIG1vc3QgY2Fz ZXMgQUZBSUNTLgoKVGhlIG9vbV9yZWFwZXIgZW5kIHRoZW4gc2ltcGx5IHJldHJpZXMgaWYgdGhl cmUgaXMgYXQgbGVhc3Qgb25lIG5vdGlmaWVyCndoaWNoIGNvdWxkbid0IG1ha2UgYW55IHByb2dy ZXNzIGluICFibG9ja2FibGUgbW9kZS4gQSByZXRyeSBsb29wIGlzCmFscmVhZHkgaW1wbGVtZW50 ZWQgdG8gd2FpdCBmb3IgdGhlIG1tYXBfc2VtIGFuZCB0aGlzIGlzIGJhc2ljYWxseSB0aGUKc2Ft ZSB0aGluZy4KCkNoYW5nZXMgc2luY2UgcmZjIHYxCi0gZ3B1IG5vdGlmaWVycyBjYW4gc2xlZXAg d2hpbGUgd2FpdGluZyBmb3IgSFcgKGV2aWN0X3Byb2Nlc3NfcXVldWVzX2Nwc2NoCiAgb24gYSBs b2NrIGFuZCBhbWRncHVfbW5faW52YWxpZGF0ZV9ub2RlIG9uIHVuYm91bmQgdGltZW91dCkgbWFr ZSBzdXJlCiAgd2UgYmFpbCBvdXQgd2hlbiB3ZSBoYXZlIGFuIGludGVyc2VjdGluZyByYW5nZSBm b3Igc3RhcnRlcgotIG5vdGUgdGhhdCBhIG5vdGlmaWVyIGZhaWxlZCB0byB0aGUgbG9nIGZvciBl YXNpZXIgZGVidWdnaW5nCi0gYmFjayBvZmYgZWFybHkgaW4gaWJfdW1lbV9ub3RpZmllcl9pbnZh bGlkYXRlX3JhbmdlX3N0YXJ0IGlmIHRoZQogIGNhbGxiYWNrIGlzIGNhbGxlZAotIG1uX2ludmxf cmFuZ2Vfc3RhcnQgd2FpdHMgZm9yIGNvbXBsZXRpb24gZG93biB0aGUgdW5tYXBfZ3JhbnRfcGFn ZXMKICBwYXRoIHNvIHdlIGhhdmUgdG8gYmFjayBvZmYgZWFybHkgb24gb3ZlcmxhcHBpbmcgcmFu Z2VzCgpDYzogIkRhdmlkIChDaHVuTWluZykgWmhvdSIgPERhdmlkMS5aaG91QGFtZC5jb20+CkNj OiBQYW9sbyBCb256aW5pIDxwYm9uemluaUByZWRoYXQuY29tPgpDYzogIlJhZGltIEtyxI1tw6HF mSIgPHJrcmNtYXJAcmVkaGF0LmNvbT4KQ2M6IEFsZXggRGV1Y2hlciA8YWxleGFuZGVyLmRldWNo ZXJAYW1kLmNvbT4KQ2M6IERhdmlkIEFpcmxpZSA8YWlybGllZEBsaW51eC5pZT4KQ2M6IEphbmkg TmlrdWxhIDxqYW5pLm5pa3VsYUBsaW51eC5pbnRlbC5jb20+CkNjOiBKb29uYXMgTGFodGluZW4g PGpvb25hcy5sYWh0aW5lbkBsaW51eC5pbnRlbC5jb20+CkNjOiBSb2RyaWdvIFZpdmkgPHJvZHJp Z28udml2aUBpbnRlbC5jb20+CkNjOiBEb3VnIExlZGZvcmQgPGRsZWRmb3JkQHJlZGhhdC5jb20+ CkNjOiBKYXNvbiBHdW50aG9ycGUgPGpnZ0B6aWVwZS5jYT4KQ2M6IE1pa2UgTWFyY2luaXN6eW4g PG1pa2UubWFyY2luaXN6eW5AaW50ZWwuY29tPgpDYzogRGVubmlzIERhbGVzc2FuZHJvIDxkZW5u aXMuZGFsZXNzYW5kcm9AaW50ZWwuY29tPgpDYzogU3VkZWVwIER1dHQgPHN1ZGVlcC5kdXR0QGlu dGVsLmNvbT4KQ2M6IEFzaHV0b3NoIERpeGl0IDxhc2h1dG9zaC5kaXhpdEBpbnRlbC5jb20+CkNj OiBEaW1pdHJpIFNpdmFuaWNoIDxzaXZhbmljaEBzZ2kuY29tPgpDYzogQm9yaXMgT3N0cm92c2t5 IDxib3Jpcy5vc3Ryb3Zza3lAb3JhY2xlLmNvbT4KQ2M6IEp1ZXJnZW4gR3Jvc3MgPGpncm9zc0Bz dXNlLmNvbT4KQ2M6ICJKw6lyw7RtZSBHbGlzc2UiIDxqZ2xpc3NlQHJlZGhhdC5jb20+CkNjOiBB bmRyZWEgQXJjYW5nZWxpIDxhYXJjYW5nZUByZWRoYXQuY29tPgpDYzogRmVsaXggS3VlaGxpbmcg PGZlbGl4Lmt1ZWhsaW5nQGFtZC5jb20+CkNjOiBrdm1Admdlci5rZXJuZWwub3JnCkNjOiBsaW51 eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnCkNjOiBhbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9y ZwpDYzogZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpDYzogaW50ZWwtZ2Z4QGxpc3Rz LmZyZWVkZXNrdG9wLm9yZwpDYzogbGludXgtcmRtYUB2Z2VyLmtlcm5lbC5vcmcKQ2M6IHhlbi1k ZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZwpDYzogbGludXgtbW1Aa3ZhY2sub3JnCkFja2VkLWJ5 OiBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+ICMgQU1EIG5vdGlm aWVycwpBY2tlZC1ieTogTGVvbiBSb21hbm92c2t5IDxsZW9ucm9AbWVsbGFub3guY29tPiAjIG1s eCBhbmQgdW1lbV9vZHAKUmVwb3J0ZWQtYnk6IERhdmlkIFJpZW50amVzIDxyaWVudGplc0Bnb29n bGUuY29tPgpTaWduZWQtb2ZmLWJ5OiBNaWNoYWwgSG9ja28gPG1ob2Nrb0BzdXNlLmNvbT4KLS0t CgpIaSwKdGhlcmUgd2VyZSBubyBtYWpvciBvYmplY3Rpb25zIHdoZW4gSSBzZW50IHRoaXMgYXMg YW4gUkZDIHRoZSBsYXN0IHRpbWUKWzFdLiBJIHdhcyBob3BpbmcgZm9yIG1vcmUgZmVlZGJhY2sg aW4gdGhlIGRyaXZlcnMgbGFuZCBiZWNhdXNlIEkgYW0KdG91Y2hpbmcgdGhlIGNvZGUgSSBoYXZl IG5vIHdheSB0byB0ZXN0LiBPbiB0aGUgb3RoZXIgaGFuZCB0aGUgcGF0dGVybgppcyBxdWl0ZSBz aW1wbGUgYW5kIGNvbnNpc3RlbnQgb3ZlciBhbGwgdXNlcnMgc28gdGhlcmUgc2hvdWxkbid0IGJl CmFueSBsYXJnZSBzdXJwcmlzZXMgaG9wZWZ1bGx5LgoKQW55IGZ1cnRoZXIgcmV2aWV3IHdvdWxk IGJlIGhpZ2hseSBhcHByZWNpYXRlIG9mIGNvdXJzZS4gQnV0IGlzIHRoaXMKc29tZXRoaW5nIHRv IHB1dCBpbnRvIHRoZSBtbSB0cmVlIG5vdz8KClsxXSBodHRwOi8vbGttbC5rZXJuZWwub3JnL3Iv MjAxODA2MjcwNzQ0MjEuR0YzMjM0OEBkaGNwMjIuc3VzZS5jegoKCiBhcmNoL3g4Ni9rdm0veDg2 LmMgICAgICAgICAgICAgICAgICAgICAgfCAgNyArKy0tCiBkcml2ZXJzL2dwdS9kcm0vYW1kL2Ft ZGdwdS9hbWRncHVfbW4uYyAgfCA0MyArKysrKysrKysrKysrKysrKysrLS0tLS0KIGRyaXZlcnMv Z3B1L2RybS9pOTE1L2k5MTVfZ2VtX3VzZXJwdHIuYyB8IDEzICsrKysrKy0tCiBkcml2ZXJzL2dw dS9kcm0vcmFkZW9uL3JhZGVvbl9tbi5jICAgICAgfCAyMiArKysrKysrKysrKy0tCiBkcml2ZXJz L2luZmluaWJhbmQvY29yZS91bWVtX29kcC5jICAgICAgfCAzMyArKysrKysrKysrKysrKystLS0t CiBkcml2ZXJzL2luZmluaWJhbmQvaHcvaGZpMS9tbXVfcmIuYyAgICAgfCAxMSArKysrLS0tCiBk cml2ZXJzL2luZmluaWJhbmQvaHcvbWx4NS9vZHAuYyAgICAgICAgfCAgMiArLQogZHJpdmVycy9t aXNjL21pYy9zY2lmL3NjaWZfZG1hLmMgICAgICAgIHwgIDcgKystLQogZHJpdmVycy9taXNjL3Nn aS1ncnUvZ3J1dGxicHVyZ2UuYyAgICAgIHwgIDcgKystLQogZHJpdmVycy94ZW4vZ250ZGV2LmMg ICAgICAgICAgICAgICAgICAgIHwgNDQgKysrKysrKysrKysrKysrKysrKystLS0tLQogaW5jbHVk ZS9saW51eC9rdm1faG9zdC5oICAgICAgICAgICAgICAgIHwgIDQgKy0tCiBpbmNsdWRlL2xpbnV4 L21tdV9ub3RpZmllci5oICAgICAgICAgICAgfCAzNSArKysrKysrKysrKysrKystLS0tLQogaW5j bHVkZS9saW51eC9vb20uaCAgICAgICAgICAgICAgICAgICAgIHwgIDIgKy0KIGluY2x1ZGUvcmRt YS9pYl91bWVtX29kcC5oICAgICAgICAgICAgICB8ICAzICstCiBtbS9obW0uYyAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgfCAgNyArKy0tCiBtbS9tbWFwLmMgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfCAgMiArLQogbW0vbW11X25vdGlmaWVyLmMgICAgICAgICAgICAgICAg ICAgICAgIHwgMTkgKysrKysrKystLS0KIG1tL29vbV9raWxsLmMgICAgICAgICAgICAgICAgICAg ICAgICAgICB8IDI5ICsrKysrKysrLS0tLS0tLS0KIHZpcnQva3ZtL2t2bV9tYWluLmMgICAgICAg ICAgICAgICAgICAgICB8IDE1ICsrKysrKy0tLQogMTkgZmlsZXMgY2hhbmdlZCwgMjI1IGluc2Vy dGlvbnMoKyksIDgwIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2FyY2gveDg2L2t2bS94ODYu YyBiL2FyY2gveDg2L2t2bS94ODYuYwppbmRleCA2YmNlY2MzMjVlN2UuLmFjMDhmNWQ3MTFiZSAx MDA2NDQKLS0tIGEvYXJjaC94ODYva3ZtL3g4Ni5jCisrKyBiL2FyY2gveDg2L2t2bS94ODYuYwpA QCAtNzIwMyw4ICs3MjAzLDkgQEAgc3RhdGljIHZvaWQgdmNwdV9sb2FkX2VvaV9leGl0bWFwKHN0 cnVjdCBrdm1fdmNwdSAqdmNwdSkKIAlrdm1feDg2X29wcy0+bG9hZF9lb2lfZXhpdG1hcCh2Y3B1 LCBlb2lfZXhpdF9iaXRtYXApOwogfQogCi12b2lkIGt2bV9hcmNoX21tdV9ub3RpZmllcl9pbnZh bGlkYXRlX3JhbmdlKHN0cnVjdCBrdm0gKmt2bSwKLQkJdW5zaWduZWQgbG9uZyBzdGFydCwgdW5z aWduZWQgbG9uZyBlbmQpCitpbnQga3ZtX2FyY2hfbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFu Z2Uoc3RydWN0IGt2bSAqa3ZtLAorCQl1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25n IGVuZCwKKwkJYm9vbCBibG9ja2FibGUpCiB7CiAJdW5zaWduZWQgbG9uZyBhcGljX2FkZHJlc3M7 CiAKQEAgLTcyMTUsNiArNzIxNiw4IEBAIHZvaWQga3ZtX2FyY2hfbW11X25vdGlmaWVyX2ludmFs aWRhdGVfcmFuZ2Uoc3RydWN0IGt2bSAqa3ZtLAogCWFwaWNfYWRkcmVzcyA9IGdmbl90b19odmEo a3ZtLCBBUElDX0RFRkFVTFRfUEhZU19CQVNFID4+IFBBR0VfU0hJRlQpOwogCWlmIChzdGFydCA8 PSBhcGljX2FkZHJlc3MgJiYgYXBpY19hZGRyZXNzIDwgZW5kKQogCQlrdm1fbWFrZV9hbGxfY3B1 c19yZXF1ZXN0KGt2bSwgS1ZNX1JFUV9BUElDX1BBR0VfUkVMT0FEKTsKKworCXJldHVybiAwOwog fQogCiB2b2lkIGt2bV92Y3B1X3JlbG9hZF9hcGljX2FjY2Vzc19wYWdlKHN0cnVjdCBrdm1fdmNw dSAqdmNwdSkKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9t bi5jIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X21uLmMKaW5kZXggODNlMzQ0 ZmJiNTBhLi4zMzk5YTRhOTI3ZmIgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k Z3B1L2FtZGdwdV9tbi5jCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9t bi5jCkBAIC0xMzYsMTIgKzEzNiwxOCBAQCB2b2lkIGFtZGdwdV9tbl91bmxvY2soc3RydWN0IGFt ZGdwdV9tbiAqbW4pCiAgKgogICogVGFrZSB0aGUgcm1uIHJlYWQgc2lkZSBsb2NrLgogICovCi1z dGF0aWMgdm9pZCBhbWRncHVfbW5fcmVhZF9sb2NrKHN0cnVjdCBhbWRncHVfbW4gKnJtbikKK3N0 YXRpYyBpbnQgYW1kZ3B1X21uX3JlYWRfbG9jayhzdHJ1Y3QgYW1kZ3B1X21uICpybW4sIGJvb2wg YmxvY2thYmxlKQogewotCW11dGV4X2xvY2soJnJtbi0+cmVhZF9sb2NrKTsKKwlpZiAoYmxvY2th YmxlKQorCQltdXRleF9sb2NrKCZybW4tPnJlYWRfbG9jayk7CisJZWxzZSBpZiAoIW11dGV4X3Ry eWxvY2soJnJtbi0+cmVhZF9sb2NrKSkKKwkJcmV0dXJuIC1FQUdBSU47CisKIAlpZiAoYXRvbWlj X2luY19yZXR1cm4oJnJtbi0+cmVjdXJzaW9uKSA9PSAxKQogCQlkb3duX3JlYWRfbm9uX293bmVy KCZybW4tPmxvY2spOwogCW11dGV4X3VubG9jaygmcm1uLT5yZWFkX2xvY2spOworCisJcmV0dXJu IDA7CiB9CiAKIC8qKgpAQCAtMTk3LDEwICsyMDMsMTEgQEAgc3RhdGljIHZvaWQgYW1kZ3B1X21u X2ludmFsaWRhdGVfbm9kZShzdHJ1Y3QgYW1kZ3B1X21uX25vZGUgKm5vZGUsCiAgKiBXZSBibG9j ayBmb3IgYWxsIEJPcyBiZXR3ZWVuIHN0YXJ0IGFuZCBlbmQgdG8gYmUgaWRsZSBhbmQKICAqIHVu bWFwIHRoZW0gYnkgbW92ZSB0aGVtIGludG8gc3lzdGVtIGRvbWFpbiBhZ2Fpbi4KICAqLwotc3Rh dGljIHZvaWQgYW1kZ3B1X21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfZ2Z4KHN0cnVjdCBtbXVf bm90aWZpZXIgKm1uLAorc3RhdGljIGludCBhbWRncHVfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFy dF9nZngoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJCQkJCQkgc3RydWN0IG1tX3N0cnVjdCAq bW0sCiAJCQkJCQkgdW5zaWduZWQgbG9uZyBzdGFydCwKLQkJCQkJCSB1bnNpZ25lZCBsb25nIGVu ZCkKKwkJCQkJCSB1bnNpZ25lZCBsb25nIGVuZCwKKwkJCQkJCSBib29sIGJsb2NrYWJsZSkKIHsK IAlzdHJ1Y3QgYW1kZ3B1X21uICpybW4gPSBjb250YWluZXJfb2YobW4sIHN0cnVjdCBhbWRncHVf bW4sIG1uKTsKIAlzdHJ1Y3QgaW50ZXJ2YWxfdHJlZV9ub2RlICppdDsKQEAgLTIwOCwxNyArMjE1 LDI4IEBAIHN0YXRpYyB2b2lkIGFtZGdwdV9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X2dmeChz dHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkvKiBub3RpZmljYXRpb24gaXMgZXhjbHVzaXZlLCBi dXQgaW50ZXJ2YWwgaXMgaW5jbHVzaXZlICovCiAJZW5kIC09IDE7CiAKLQlhbWRncHVfbW5fcmVh ZF9sb2NrKHJtbik7CisJLyogVE9ETyB3ZSBzaG91bGQgYmUgYWJsZSB0byBzcGxpdCBsb2NraW5n IGZvciBpbnRlcnZhbCB0cmVlIGFuZAorCSAqIGFtZGdwdV9tbl9pbnZhbGlkYXRlX25vZGUKKwkg Ki8KKwlpZiAoYW1kZ3B1X21uX3JlYWRfbG9jayhybW4sIGJsb2NrYWJsZSkpCisJCXJldHVybiAt RUFHQUlOOwogCiAJaXQgPSBpbnRlcnZhbF90cmVlX2l0ZXJfZmlyc3QoJnJtbi0+b2JqZWN0cywg c3RhcnQsIGVuZCk7CiAJd2hpbGUgKGl0KSB7CiAJCXN0cnVjdCBhbWRncHVfbW5fbm9kZSAqbm9k ZTsKIAorCQlpZiAoIWJsb2NrYWJsZSkgeworCQkJYW1kZ3B1X21uX3JlYWRfdW5sb2NrKHJtbik7 CisJCQlyZXR1cm4gLUVBR0FJTjsKKwkJfQorCiAJCW5vZGUgPSBjb250YWluZXJfb2YoaXQsIHN0 cnVjdCBhbWRncHVfbW5fbm9kZSwgaXQpOwogCQlpdCA9IGludGVydmFsX3RyZWVfaXRlcl9uZXh0 KGl0LCBzdGFydCwgZW5kKTsKIAogCQlhbWRncHVfbW5faW52YWxpZGF0ZV9ub2RlKG5vZGUsIHN0 YXJ0LCBlbmQpOwogCX0KKworCXJldHVybiAwOwogfQogCiAvKioKQEAgLTIzMywxMCArMjUxLDEx IEBAIHN0YXRpYyB2b2lkIGFtZGdwdV9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X2dmeChzdHJ1 Y3QgbW11X25vdGlmaWVyICptbiwKICAqIG5lY2Vzc2l0YXRlcyBldmljdGluZyBhbGwgdXNlci1t b2RlIHF1ZXVlcyBvZiB0aGUgcHJvY2Vzcy4gVGhlIEJPcwogICogYXJlIHJlc3RvcnRlZCBpbiBh bWRncHVfbW5faW52YWxpZGF0ZV9yYW5nZV9lbmRfaHNhLgogICovCi1zdGF0aWMgdm9pZCBhbWRn cHVfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydF9oc2Eoc3RydWN0IG1tdV9ub3RpZmllciAqbW4s CitzdGF0aWMgaW50IGFtZGdwdV9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X2hzYShzdHJ1Y3Qg bW11X25vdGlmaWVyICptbiwKIAkJCQkJCSBzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkJCSB1 bnNpZ25lZCBsb25nIHN0YXJ0LAotCQkJCQkJIHVuc2lnbmVkIGxvbmcgZW5kKQorCQkJCQkJIHVu c2lnbmVkIGxvbmcgZW5kLAorCQkJCQkJIGJvb2wgYmxvY2thYmxlKQogewogCXN0cnVjdCBhbWRn cHVfbW4gKnJtbiA9IGNvbnRhaW5lcl9vZihtbiwgc3RydWN0IGFtZGdwdV9tbiwgbW4pOwogCXN0 cnVjdCBpbnRlcnZhbF90cmVlX25vZGUgKml0OwpAQCAtMjQ0LDEzICsyNjMsMTkgQEAgc3RhdGlj IHZvaWQgYW1kZ3B1X21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfaHNhKHN0cnVjdCBtbXVfbm90 aWZpZXIgKm1uLAogCS8qIG5vdGlmaWNhdGlvbiBpcyBleGNsdXNpdmUsIGJ1dCBpbnRlcnZhbCBp cyBpbmNsdXNpdmUgKi8KIAllbmQgLT0gMTsKIAotCWFtZGdwdV9tbl9yZWFkX2xvY2socm1uKTsK KwlpZiAoYW1kZ3B1X21uX3JlYWRfbG9jayhybW4sIGJsb2NrYWJsZSkpCisJCXJldHVybiAtRUFH QUlOOwogCiAJaXQgPSBpbnRlcnZhbF90cmVlX2l0ZXJfZmlyc3QoJnJtbi0+b2JqZWN0cywgc3Rh cnQsIGVuZCk7CiAJd2hpbGUgKGl0KSB7CiAJCXN0cnVjdCBhbWRncHVfbW5fbm9kZSAqbm9kZTsK IAkJc3RydWN0IGFtZGdwdV9ibyAqYm87CiAKKwkJaWYgKCFibG9ja2FibGUpIHsKKwkJCWFtZGdw dV9tbl9yZWFkX3VubG9jayhybW4pOworCQkJcmV0dXJuIC1FQUdBSU47CisJCX0KKwogCQlub2Rl ID0gY29udGFpbmVyX29mKGl0LCBzdHJ1Y3QgYW1kZ3B1X21uX25vZGUsIGl0KTsKIAkJaXQgPSBp bnRlcnZhbF90cmVlX2l0ZXJfbmV4dChpdCwgc3RhcnQsIGVuZCk7CiAKQEAgLTI2Miw2ICsyODcs OCBAQCBzdGF0aWMgdm9pZCBhbWRncHVfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydF9oc2Eoc3Ry dWN0IG1tdV9ub3RpZmllciAqbW4sCiAJCQkJYW1kZ3B1X2FtZGtmZF9ldmljdF91c2VycHRyKG1l bSwgbW0pOwogCQl9CiAJfQorCisJcmV0dXJuIDA7CiB9CiAKIC8qKgpkaWZmIC0tZ2l0IGEvZHJp dmVycy9ncHUvZHJtL2k5MTUvaTkxNV9nZW1fdXNlcnB0ci5jIGIvZHJpdmVycy9ncHUvZHJtL2k5 MTUvaTkxNV9nZW1fdXNlcnB0ci5jCmluZGV4IDg1NGJkNTFiOTQ3OC4uOWNiZmY2OGY2YjQxIDEw MDY0NAotLS0gYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pOTE1X2dlbV91c2VycHRyLmMKKysrIGIv ZHJpdmVycy9ncHUvZHJtL2k5MTUvaTkxNV9nZW1fdXNlcnB0ci5jCkBAIC0xMTIsMTAgKzExMiwx MSBAQCBzdGF0aWMgdm9pZCBkZWxfb2JqZWN0KHN0cnVjdCBpOTE1X21tdV9vYmplY3QgKm1vKQog CW1vLT5hdHRhY2hlZCA9IGZhbHNlOwogfQogCi1zdGF0aWMgdm9pZCBpOTE1X2dlbV91c2VycHRy X21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqX21uLAorc3Rh dGljIGludCBpOTE1X2dlbV91c2VycHRyX21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0 IG1tdV9ub3RpZmllciAqX21uLAogCQkJCQkJICAgICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAog CQkJCQkJICAgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsCi0JCQkJCQkgICAgICAgdW5zaWduZWQg bG9uZyBlbmQpCisJCQkJCQkgICAgICAgdW5zaWduZWQgbG9uZyBlbmQsCisJCQkJCQkgICAgICAg Ym9vbCBibG9ja2FibGUpCiB7CiAJc3RydWN0IGk5MTVfbW11X25vdGlmaWVyICptbiA9CiAJCWNv bnRhaW5lcl9vZihfbW4sIHN0cnVjdCBpOTE1X21tdV9ub3RpZmllciwgbW4pOwpAQCAtMTI0LDcg KzEyNSw3IEBAIHN0YXRpYyB2b2lkIGk5MTVfZ2VtX3VzZXJwdHJfbW5faW52YWxpZGF0ZV9yYW5n ZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICpfbW4sCiAJTElTVF9IRUFEKGNhbmNlbGxlZCk7 CiAKIAlpZiAoUkJfRU1QVFlfUk9PVCgmbW4tPm9iamVjdHMucmJfcm9vdCkpCi0JCXJldHVybjsK KwkJcmV0dXJuIDA7CiAKIAkvKiBpbnRlcnZhbCByYW5nZXMgYXJlIGluY2x1c2l2ZSwgYnV0IGlu dmFsaWRhdGUgcmFuZ2UgaXMgZXhjbHVzaXZlICovCiAJZW5kLS07CkBAIC0xMzIsNiArMTMzLDEw IEBAIHN0YXRpYyB2b2lkIGk5MTVfZ2VtX3VzZXJwdHJfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFy dChzdHJ1Y3QgbW11X25vdGlmaWVyICpfbW4sCiAJc3Bpbl9sb2NrKCZtbi0+bG9jayk7CiAJaXQg PSBpbnRlcnZhbF90cmVlX2l0ZXJfZmlyc3QoJm1uLT5vYmplY3RzLCBzdGFydCwgZW5kKTsKIAl3 aGlsZSAoaXQpIHsKKwkJaWYgKCFibG9ja2FibGUpIHsKKwkJCXNwaW5fdW5sb2NrKCZtbi0+bG9j ayk7CisJCQlyZXR1cm4gLUVBR0FJTjsKKwkJfQogCQkvKiBUaGUgbW11X29iamVjdCBpcyByZWxl YXNlZCBsYXRlIHdoZW4gZGVzdHJveWluZyB0aGUKIAkJICogR0VNIG9iamVjdCBzbyBpdCBpcyBl bnRpcmVseSBwb3NzaWJsZSB0byBnYWluIGEKIAkJICogcmVmZXJlbmNlIG9uIGFuIG9iamVjdCBp biB0aGUgcHJvY2VzcyBvZiBiZWluZyBmcmVlZApAQCAtMTU0LDYgKzE1OSw4IEBAIHN0YXRpYyB2 b2lkIGk5MTVfZ2VtX3VzZXJwdHJfbW5faW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11 X25vdGlmaWVyICpfbW4sCiAKIAlpZiAoIWxpc3RfZW1wdHkoJmNhbmNlbGxlZCkpCiAJCWZsdXNo X3dvcmtxdWV1ZShtbi0+d3EpOworCisJcmV0dXJuIDA7CiB9CiAKIHN0YXRpYyBjb25zdCBzdHJ1 Y3QgbW11X25vdGlmaWVyX29wcyBpOTE1X2dlbV91c2VycHRyX25vdGlmaWVyID0gewpkaWZmIC0t Z2l0IGEvZHJpdmVycy9ncHUvZHJtL3JhZGVvbi9yYWRlb25fbW4uYyBiL2RyaXZlcnMvZ3B1L2Ry bS9yYWRlb24vcmFkZW9uX21uLmMKaW5kZXggYWJkMjQ5NzVjOWIxLi5mOGIzNWRmNDRjNjAgMTAw NjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9yYWRlb24vcmFkZW9uX21uLmMKKysrIGIvZHJpdmVy cy9ncHUvZHJtL3JhZGVvbi9yYWRlb25fbW4uYwpAQCAtMTE4LDE5ICsxMTgsMjcgQEAgc3RhdGlj IHZvaWQgcmFkZW9uX21uX3JlbGVhc2Uoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAgKiBXZSBi bG9jayBmb3IgYWxsIEJPcyBiZXR3ZWVuIHN0YXJ0IGFuZCBlbmQgdG8gYmUgaWRsZSBhbmQKICAq IHVubWFwIHRoZW0gYnkgbW92ZSB0aGVtIGludG8gc3lzdGVtIGRvbWFpbiBhZ2Fpbi4KICAqLwot c3RhdGljIHZvaWQgcmFkZW9uX21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9u b3RpZmllciAqbW4sCitzdGF0aWMgaW50IHJhZGVvbl9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0 KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQkJCQkgICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1t LAogCQkJCQkgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsCi0JCQkJCSAgICAgdW5zaWduZWQgbG9u ZyBlbmQpCisJCQkJCSAgICAgdW5zaWduZWQgbG9uZyBlbmQsCisJCQkJCSAgICAgYm9vbCBibG9j a2FibGUpCiB7CiAJc3RydWN0IHJhZGVvbl9tbiAqcm1uID0gY29udGFpbmVyX29mKG1uLCBzdHJ1 Y3QgcmFkZW9uX21uLCBtbik7CiAJc3RydWN0IHR0bV9vcGVyYXRpb25fY3R4IGN0eCA9IHsgZmFs c2UsIGZhbHNlIH07CiAJc3RydWN0IGludGVydmFsX3RyZWVfbm9kZSAqaXQ7CisJaW50IHJldCA9 IDA7CiAKIAkvKiBub3RpZmljYXRpb24gaXMgZXhjbHVzaXZlLCBidXQgaW50ZXJ2YWwgaXMgaW5j bHVzaXZlICovCiAJZW5kIC09IDE7CiAKLQltdXRleF9sb2NrKCZybW4tPmxvY2spOworCS8qIFRP RE8gd2Ugc2hvdWxkIGJlIGFibGUgdG8gc3BsaXQgbG9ja2luZyBmb3IgaW50ZXJ2YWwgdHJlZSBh bmQKKwkgKiB0aGUgdGVhciBkb3duLgorCSAqLworCWlmIChibG9ja2FibGUpCisJCW11dGV4X2xv Y2soJnJtbi0+bG9jayk7CisJZWxzZSBpZiAoIW11dGV4X3RyeWxvY2soJnJtbi0+bG9jaykpCisJ CXJldHVybiAtRUFHQUlOOwogCiAJaXQgPSBpbnRlcnZhbF90cmVlX2l0ZXJfZmlyc3QoJnJtbi0+ b2JqZWN0cywgc3RhcnQsIGVuZCk7CiAJd2hpbGUgKGl0KSB7CkBAIC0xMzgsNiArMTQ2LDExIEBA IHN0YXRpYyB2b2lkIHJhZGVvbl9tbl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVf bm90aWZpZXIgKm1uLAogCQlzdHJ1Y3QgcmFkZW9uX2JvICpibzsKIAkJbG9uZyByOwogCisJCWlm ICghYmxvY2thYmxlKSB7CisJCQlyZXQgPSAtRUFHQUlOOworCQkJZ290byBvdXRfdW5sb2NrOwor CQl9CisKIAkJbm9kZSA9IGNvbnRhaW5lcl9vZihpdCwgc3RydWN0IHJhZGVvbl9tbl9ub2RlLCBp dCk7CiAJCWl0ID0gaW50ZXJ2YWxfdHJlZV9pdGVyX25leHQoaXQsIHN0YXJ0LCBlbmQpOwogCkBA IC0xNjYsNyArMTc5LDEwIEBAIHN0YXRpYyB2b2lkIHJhZGVvbl9tbl9pbnZhbGlkYXRlX3Jhbmdl X3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQl9CiAJfQogCQorb3V0X3VubG9jazoK IAltdXRleF91bmxvY2soJnJtbi0+bG9jayk7CisKKwlyZXR1cm4gcmV0OwogfQogCiBzdGF0aWMg Y29uc3Qgc3RydWN0IG1tdV9ub3RpZmllcl9vcHMgcmFkZW9uX21uX29wcyA9IHsKZGlmZiAtLWdp dCBhL2RyaXZlcnMvaW5maW5pYmFuZC9jb3JlL3VtZW1fb2RwLmMgYi9kcml2ZXJzL2luZmluaWJh bmQvY29yZS91bWVtX29kcC5jCmluZGV4IDE4MjQzNmI5MmJhOS4uNmVjNzQ4ZWNjZmY3IDEwMDY0 NAotLS0gYS9kcml2ZXJzL2luZmluaWJhbmQvY29yZS91bWVtX29kcC5jCisrKyBiL2RyaXZlcnMv aW5maW5pYmFuZC9jb3JlL3VtZW1fb2RwLmMKQEAgLTE4Niw2ICsxODYsNyBAQCBzdGF0aWMgdm9p ZCBpYl91bWVtX25vdGlmaWVyX3JlbGVhc2Uoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJcmJ0 X2liX3VtZW1fZm9yX2VhY2hfaW5fcmFuZ2UoJmNvbnRleHQtPnVtZW1fdHJlZSwgMCwKIAkJCQkg ICAgICBVTExPTkdfTUFYLAogCQkJCSAgICAgIGliX3VtZW1fbm90aWZpZXJfcmVsZWFzZV90cmFt cG9saW5lLAorCQkJCSAgICAgIHRydWUsCiAJCQkJICAgICAgTlVMTCk7CiAJdXBfcmVhZCgmY29u dGV4dC0+dW1lbV9yd3NlbSk7CiB9CkBAIC0yMDcsMjIgKzIwOCwzMSBAQCBzdGF0aWMgaW50IGlu dmFsaWRhdGVfcmFuZ2Vfc3RhcnRfdHJhbXBvbGluZShzdHJ1Y3QgaWJfdW1lbSAqaXRlbSwgdTY0 IHN0YXJ0LAogCXJldHVybiAwOwogfQogCi1zdGF0aWMgdm9pZCBpYl91bWVtX25vdGlmaWVyX2lu dmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCitzdGF0aWMgaW50 IGliX3VtZW1fbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlm aWVyICptbiwKIAkJCQkJCSAgICBzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkJCSAgICB1bnNp Z25lZCBsb25nIHN0YXJ0LAotCQkJCQkJICAgIHVuc2lnbmVkIGxvbmcgZW5kKQorCQkJCQkJICAg IHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCQkJICAgIGJvb2wgYmxvY2thYmxlKQogewogCXN0cnVj dCBpYl91Y29udGV4dCAqY29udGV4dCA9IGNvbnRhaW5lcl9vZihtbiwgc3RydWN0IGliX3Vjb250 ZXh0LCBtbik7CisJaW50IHJldDsKIAogCWlmICghY29udGV4dC0+aW52YWxpZGF0ZV9yYW5nZSkK LQkJcmV0dXJuOworCQlyZXR1cm4gMDsKKworCWlmIChibG9ja2FibGUpCisJCWRvd25fcmVhZCgm Y29udGV4dC0+dW1lbV9yd3NlbSk7CisJZWxzZSBpZiAoIWRvd25fcmVhZF90cnlsb2NrKCZjb250 ZXh0LT51bWVtX3J3c2VtKSkKKwkJcmV0dXJuIC1FQUdBSU47CiAKIAlpYl91Y29udGV4dF9ub3Rp Zmllcl9zdGFydF9hY2NvdW50KGNvbnRleHQpOwotCWRvd25fcmVhZCgmY29udGV4dC0+dW1lbV9y d3NlbSk7Ci0JcmJ0X2liX3VtZW1fZm9yX2VhY2hfaW5fcmFuZ2UoJmNvbnRleHQtPnVtZW1fdHJl ZSwgc3RhcnQsCisJcmV0ID0gcmJ0X2liX3VtZW1fZm9yX2VhY2hfaW5fcmFuZ2UoJmNvbnRleHQt PnVtZW1fdHJlZSwgc3RhcnQsCiAJCQkJICAgICAgZW5kLAotCQkJCSAgICAgIGludmFsaWRhdGVf cmFuZ2Vfc3RhcnRfdHJhbXBvbGluZSwgTlVMTCk7CisJCQkJICAgICAgaW52YWxpZGF0ZV9yYW5n ZV9zdGFydF90cmFtcG9saW5lLAorCQkJCSAgICAgIGJsb2NrYWJsZSwgTlVMTCk7CiAJdXBfcmVh ZCgmY29udGV4dC0+dW1lbV9yd3NlbSk7CisKKwlyZXR1cm4gcmV0OwogfQogCiBzdGF0aWMgaW50 IGludmFsaWRhdGVfcmFuZ2VfZW5kX3RyYW1wb2xpbmUoc3RydWN0IGliX3VtZW0gKml0ZW0sIHU2 NCBzdGFydCwKQEAgLTI0MiwxMCArMjUyLDE1IEBAIHN0YXRpYyB2b2lkIGliX3VtZW1fbm90aWZp ZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJaWYgKCFj b250ZXh0LT5pbnZhbGlkYXRlX3JhbmdlKQogCQlyZXR1cm47CiAKKwkvKgorCSAqIFRPRE86IHdl IGN1cnJlbnRseSBiYWlsIG91dCBpZiB0aGVyZSBpcyBhbnkgc2xlZXBhYmxlIHdvcmsgdG8gYmUg ZG9uZQorCSAqIGluIGliX3VtZW1fbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydCBzbyB3 ZSBzaG91bGRuJ3QgcmVhbGx5IGJsb2NrCisJICogaGVyZS4gQnV0IHRoaXMgaXMgdWdseSBhbmQg ZnJhZ2lsZS4KKwkgKi8KIAlkb3duX3JlYWQoJmNvbnRleHQtPnVtZW1fcndzZW0pOwogCXJidF9p Yl91bWVtX2Zvcl9lYWNoX2luX3JhbmdlKCZjb250ZXh0LT51bWVtX3RyZWUsIHN0YXJ0LAogCQkJ CSAgICAgIGVuZCwKLQkJCQkgICAgICBpbnZhbGlkYXRlX3JhbmdlX2VuZF90cmFtcG9saW5lLCBO VUxMKTsKKwkJCQkgICAgICBpbnZhbGlkYXRlX3JhbmdlX2VuZF90cmFtcG9saW5lLCB0cnVlLCBO VUxMKTsKIAl1cF9yZWFkKCZjb250ZXh0LT51bWVtX3J3c2VtKTsKIAlpYl91Y29udGV4dF9ub3Rp Zmllcl9lbmRfYWNjb3VudChjb250ZXh0KTsKIH0KQEAgLTc5OCw2ICs4MTMsNyBAQCBFWFBPUlRf U1lNQk9MKGliX3VtZW1fb2RwX3VubWFwX2RtYV9wYWdlcyk7CiBpbnQgcmJ0X2liX3VtZW1fZm9y X2VhY2hfaW5fcmFuZ2Uoc3RydWN0IHJiX3Jvb3RfY2FjaGVkICpyb290LAogCQkJCSAgdTY0IHN0 YXJ0LCB1NjQgbGFzdCwKIAkJCQkgIHVtZW1fY2FsbF9iYWNrIGNiLAorCQkJCSAgYm9vbCBibG9j a2FibGUsCiAJCQkJICB2b2lkICpjb29raWUpCiB7CiAJaW50IHJldF92YWwgPSAwOwpAQCAtODA5 LDYgKzgyNSw5IEBAIGludCByYnRfaWJfdW1lbV9mb3JfZWFjaF9pbl9yYW5nZShzdHJ1Y3QgcmJf cm9vdF9jYWNoZWQgKnJvb3QsCiAKIAlmb3IgKG5vZGUgPSByYnRfaWJfdW1lbV9pdGVyX2ZpcnN0 KHJvb3QsIHN0YXJ0LCBsYXN0IC0gMSk7CiAJCQlub2RlOyBub2RlID0gbmV4dCkgeworCQkvKiBU T0RPIG1vdmUgdGhlIGJsb2NrYWJsZSBkZWNpc2lvbiB1cCB0byB0aGUgY2FsbGJhY2sgKi8KKwkJ aWYgKCFibG9ja2FibGUpCisJCQlyZXR1cm4gLUVBR0FJTjsKIAkJbmV4dCA9IHJidF9pYl91bWVt X2l0ZXJfbmV4dChub2RlLCBzdGFydCwgbGFzdCAtIDEpOwogCQl1bWVtID0gY29udGFpbmVyX29m KG5vZGUsIHN0cnVjdCBpYl91bWVtX29kcCwgaW50ZXJ2YWxfdHJlZSk7CiAJCXJldF92YWwgPSBj Yih1bWVtLT51bWVtLCBzdGFydCwgbGFzdCwgY29va2llKSB8fCByZXRfdmFsOwpkaWZmIC0tZ2l0 IGEvZHJpdmVycy9pbmZpbmliYW5kL2h3L2hmaTEvbW11X3JiLmMgYi9kcml2ZXJzL2luZmluaWJh bmQvaHcvaGZpMS9tbXVfcmIuYwppbmRleCA3MGFjZWVmZTE0ZDUuLmUxYzc5OTZjMDE4ZSAxMDA2 NDQKLS0tIGEvZHJpdmVycy9pbmZpbmliYW5kL2h3L2hmaTEvbW11X3JiLmMKKysrIGIvZHJpdmVy cy9pbmZpbmliYW5kL2h3L2hmaTEvbW11X3JiLmMKQEAgLTY3LDkgKzY3LDkgQEAgc3RydWN0IG1t dV9yYl9oYW5kbGVyIHsKIAogc3RhdGljIHVuc2lnbmVkIGxvbmcgbW11X25vZGVfc3RhcnQoc3Ry dWN0IG1tdV9yYl9ub2RlICopOwogc3RhdGljIHVuc2lnbmVkIGxvbmcgbW11X25vZGVfbGFzdChz dHJ1Y3QgbW11X3JiX25vZGUgKik7Ci1zdGF0aWMgdm9pZCBtbXVfbm90aWZpZXJfcmFuZ2Vfc3Rh cnQoc3RydWN0IG1tdV9ub3RpZmllciAqLAorc3RhdGljIGludCBtbXVfbm90aWZpZXJfcmFuZ2Vf c3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqLAogCQkJCSAgICAgc3RydWN0IG1tX3N0cnVjdCAq LAotCQkJCSAgICAgdW5zaWduZWQgbG9uZywgdW5zaWduZWQgbG9uZyk7CisJCQkJICAgICB1bnNp Z25lZCBsb25nLCB1bnNpZ25lZCBsb25nLCBib29sKTsKIHN0YXRpYyBzdHJ1Y3QgbW11X3JiX25v ZGUgKl9fbW11X3JiX3NlYXJjaChzdHJ1Y3QgbW11X3JiX2hhbmRsZXIgKiwKIAkJCQkJICAgdW5z aWduZWQgbG9uZywgdW5zaWduZWQgbG9uZyk7CiBzdGF0aWMgdm9pZCBkb19yZW1vdmUoc3RydWN0 IG1tdV9yYl9oYW5kbGVyICpoYW5kbGVyLApAQCAtMjg0LDEwICsyODQsMTEgQEAgdm9pZCBoZmkx X21tdV9yYl9yZW1vdmUoc3RydWN0IG1tdV9yYl9oYW5kbGVyICpoYW5kbGVyLAogCWhhbmRsZXIt Pm9wcy0+cmVtb3ZlKGhhbmRsZXItPm9wc19hcmcsIG5vZGUpOwogfQogCi1zdGF0aWMgdm9pZCBt bXVfbm90aWZpZXJfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCitzdGF0aWMg aW50IG1tdV9ub3RpZmllcl9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJ CQkgICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAogCQkJCSAgICAgdW5zaWduZWQgbG9uZyBzdGFy dCwKLQkJCQkgICAgIHVuc2lnbmVkIGxvbmcgZW5kKQorCQkJCSAgICAgdW5zaWduZWQgbG9uZyBl bmQsCisJCQkJICAgICBib29sIGJsb2NrYWJsZSkKIHsKIAlzdHJ1Y3QgbW11X3JiX2hhbmRsZXIg KmhhbmRsZXIgPQogCQljb250YWluZXJfb2YobW4sIHN0cnVjdCBtbXVfcmJfaGFuZGxlciwgbW4p OwpAQCAtMzEzLDYgKzMxNCw4IEBAIHN0YXRpYyB2b2lkIG1tdV9ub3RpZmllcl9yYW5nZV9zdGFy dChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAogCWlmIChhZGRlZCkKIAkJcXVldWVfd29yayho YW5kbGVyLT53cSwgJmhhbmRsZXItPmRlbF93b3JrKTsKKworCXJldHVybiAwOwogfQogCiAvKgpk aWZmIC0tZ2l0IGEvZHJpdmVycy9pbmZpbmliYW5kL2h3L21seDUvb2RwLmMgYi9kcml2ZXJzL2lu ZmluaWJhbmQvaHcvbWx4NS9vZHAuYwppbmRleCBmMWE4N2E2OTBhNGMuLmQyMTZlMGQyOTIxZCAx MDA2NDQKLS0tIGEvZHJpdmVycy9pbmZpbmliYW5kL2h3L21seDUvb2RwLmMKKysrIGIvZHJpdmVy cy9pbmZpbmliYW5kL2h3L21seDUvb2RwLmMKQEAgLTQ4OCw3ICs0ODgsNyBAQCB2b2lkIG1seDVf aWJfZnJlZV9pbXBsaWNpdF9tcihzdHJ1Y3QgbWx4NV9pYl9tciAqaW1yKQogCiAJZG93bl9yZWFk KCZjdHgtPnVtZW1fcndzZW0pOwogCXJidF9pYl91bWVtX2Zvcl9lYWNoX2luX3JhbmdlKCZjdHgt PnVtZW1fdHJlZSwgMCwgVUxMT05HX01BWCwKLQkJCQkgICAgICBtcl9sZWFmX2ZyZWUsIGltcik7 CisJCQkJICAgICAgbXJfbGVhZl9mcmVlLCB0cnVlLCBpbXIpOwogCXVwX3JlYWQoJmN0eC0+dW1l bV9yd3NlbSk7CiAKIAl3YWl0X2V2ZW50KGltci0+cV9sZWFmX2ZyZWUsICFhdG9taWNfcmVhZCgm aW1yLT5udW1fbGVhZl9mcmVlKSk7CmRpZmYgLS1naXQgYS9kcml2ZXJzL21pc2MvbWljL3NjaWYv c2NpZl9kbWEuYyBiL2RyaXZlcnMvbWlzYy9taWMvc2NpZi9zY2lmX2RtYS5jCmluZGV4IDYzZDYy NDZkNmRmZi4uNjM2OWFlYWE3MDU2IDEwMDY0NAotLS0gYS9kcml2ZXJzL21pc2MvbWljL3NjaWYv c2NpZl9kbWEuYworKysgYi9kcml2ZXJzL21pc2MvbWljL3NjaWYvc2NpZl9kbWEuYwpAQCAtMjAw LDE1ICsyMDAsMTggQEAgc3RhdGljIHZvaWQgc2NpZl9tbXVfbm90aWZpZXJfcmVsZWFzZShzdHJ1 Y3QgbW11X25vdGlmaWVyICptbiwKIAlzY2hlZHVsZV93b3JrKCZzY2lmX2luZm8ubWlzY193b3Jr KTsKIH0KIAotc3RhdGljIHZvaWQgc2NpZl9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9z dGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKK3N0YXRpYyBpbnQgc2NpZl9tbXVfbm90aWZp ZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAkJCQkJ CSAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0sCiAJCQkJCQkgICAgIHVuc2lnbmVkIGxvbmcgc3Rh cnQsCi0JCQkJCQkgICAgIHVuc2lnbmVkIGxvbmcgZW5kKQorCQkJCQkJICAgICB1bnNpZ25lZCBs b25nIGVuZCwKKwkJCQkJCSAgICAgYm9vbCBibG9ja2FibGUpCiB7CiAJc3RydWN0IHNjaWZfbW11 X25vdGlmCSptbW47CiAKIAltbW4gPSBjb250YWluZXJfb2YobW4sIHN0cnVjdCBzY2lmX21tdV9u b3RpZiwgZXBfbW11X25vdGlmaWVyKTsKIAlzY2lmX3JtYV9kZXN0cm95X3RjdyhtbW4sIHN0YXJ0 LCBlbmQgLSBzdGFydCk7CisKKwlyZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgc2NpZl9tbXVf bm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCmRp ZmYgLS1naXQgYS9kcml2ZXJzL21pc2Mvc2dpLWdydS9ncnV0bGJwdXJnZS5jIGIvZHJpdmVycy9t aXNjL3NnaS1ncnUvZ3J1dGxicHVyZ2UuYwppbmRleCBhMzQ1NGViNTZmYmYuLmJlMjhmMDViZmFm YSAxMDA2NDQKLS0tIGEvZHJpdmVycy9taXNjL3NnaS1ncnUvZ3J1dGxicHVyZ2UuYworKysgYi9k cml2ZXJzL21pc2Mvc2dpLWdydS9ncnV0bGJwdXJnZS5jCkBAIC0yMTksOSArMjE5LDEwIEBAIHZv aWQgZ3J1X2ZsdXNoX2FsbF90bGIoc3RydWN0IGdydV9zdGF0ZSAqZ3J1KQogLyoKICAqIE1NVU9Q UyBub3RpZmllciBjYWxsb3V0IGZ1bmN0aW9ucwogICovCi1zdGF0aWMgdm9pZCBncnVfaW52YWxp ZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKK3N0YXRpYyBpbnQgZ3J1 X2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJCQkJICAg ICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAotCQkJCSAgICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0 LCB1bnNpZ25lZCBsb25nIGVuZCkKKwkJCQkgICAgICAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5z aWduZWQgbG9uZyBlbmQsCisJCQkJICAgICAgIGJvb2wgYmxvY2thYmxlKQogewogCXN0cnVjdCBn cnVfbW1fc3RydWN0ICpnbXMgPSBjb250YWluZXJfb2YobW4sIHN0cnVjdCBncnVfbW1fc3RydWN0 LAogCQkJCQkJIG1zX25vdGlmaWVyKTsKQEAgLTIzMSw2ICsyMzIsOCBAQCBzdGF0aWMgdm9pZCBn cnVfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAlncnVf ZGJnKGdydWRldiwgImdtcyAlcCwgc3RhcnQgMHglbHgsIGVuZCAweCVseCwgYWN0ICVkXG4iLCBn bXMsCiAJCXN0YXJ0LCBlbmQsIGF0b21pY19yZWFkKCZnbXMtPm1zX3JhbmdlX2FjdGl2ZSkpOwog CWdydV9mbHVzaF90bGJfcmFuZ2UoZ21zLCBzdGFydCwgZW5kIC0gc3RhcnQpOworCisJcmV0dXJu IDA7CiB9CiAKIHN0YXRpYyB2b2lkIGdydV9pbnZhbGlkYXRlX3JhbmdlX2VuZChzdHJ1Y3QgbW11 X25vdGlmaWVyICptbiwKZGlmZiAtLWdpdCBhL2RyaXZlcnMveGVuL2dudGRldi5jIGIvZHJpdmVy cy94ZW4vZ250ZGV2LmMKaW5kZXggYmQ1NjY1M2I5YmJjLi41NWI0ZjBlM2Y0ZDYgMTAwNjQ0Ci0t LSBhL2RyaXZlcnMveGVuL2dudGRldi5jCisrKyBiL2RyaXZlcnMveGVuL2dudGRldi5jCkBAIC00 NDEsMTggKzQ0MSwyNSBAQCBzdGF0aWMgY29uc3Qgc3RydWN0IHZtX29wZXJhdGlvbnNfc3RydWN0 IGdudGRldl92bW9wcyA9IHsKIAogLyogLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tICovCiAKK3N0YXRpYyBib29sIGluX3Jh bmdlKHN0cnVjdCBncmFudF9tYXAgKm1hcCwKKwkJCSAgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQs IHVuc2lnbmVkIGxvbmcgZW5kKQoreworCWlmICghbWFwLT52bWEpCisJCXJldHVybiBmYWxzZTsK KwlpZiAobWFwLT52bWEtPnZtX3N0YXJ0ID49IGVuZCkKKwkJcmV0dXJuIGZhbHNlOworCWlmICht YXAtPnZtYS0+dm1fZW5kIDw9IHN0YXJ0KQorCQlyZXR1cm4gZmFsc2U7CisKKwlyZXR1cm4gdHJ1 ZTsKK30KKwogc3RhdGljIHZvaWQgdW5tYXBfaWZfaW5fcmFuZ2Uoc3RydWN0IGdyYW50X21hcCAq bWFwLAogCQkJICAgICAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQpCiB7 CiAJdW5zaWduZWQgbG9uZyBtc3RhcnQsIG1lbmQ7CiAJaW50IGVycjsKIAotCWlmICghbWFwLT52 bWEpCi0JCXJldHVybjsKLQlpZiAobWFwLT52bWEtPnZtX3N0YXJ0ID49IGVuZCkKLQkJcmV0dXJu OwotCWlmIChtYXAtPnZtYS0+dm1fZW5kIDw9IHN0YXJ0KQotCQlyZXR1cm47CiAJbXN0YXJ0ID0g bWF4KHN0YXJ0LCBtYXAtPnZtYS0+dm1fc3RhcnQpOwogCW1lbmQgICA9IG1pbihlbmQsICAgbWFw LT52bWEtPnZtX2VuZCk7CiAJcHJfZGVidWcoIm1hcCAlZCslZCAoJWx4ICVseCksIHJhbmdlICVs eCAlbHgsIG1yYW5nZSAlbHggJWx4XG4iLApAQCAtNDY1LDIxICs0NzIsNDAgQEAgc3RhdGljIHZv aWQgdW5tYXBfaWZfaW5fcmFuZ2Uoc3RydWN0IGdyYW50X21hcCAqbWFwLAogCVdBUk5fT04oZXJy KTsKIH0KIAotc3RhdGljIHZvaWQgbW5faW52bF9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlm aWVyICptbiwKK3N0YXRpYyBpbnQgbW5faW52bF9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlm aWVyICptbiwKIAkJCQlzdHJ1Y3QgbW1fc3RydWN0ICptbSwKLQkJCQl1bnNpZ25lZCBsb25nIHN0 YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCkKKwkJCQl1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25l ZCBsb25nIGVuZCwKKwkJCQlib29sIGJsb2NrYWJsZSkKIHsKIAlzdHJ1Y3QgZ250ZGV2X3ByaXYg KnByaXYgPSBjb250YWluZXJfb2YobW4sIHN0cnVjdCBnbnRkZXZfcHJpdiwgbW4pOwogCXN0cnVj dCBncmFudF9tYXAgKm1hcDsKKwlpbnQgcmV0ID0gMDsKKworCS8qIFRPRE8gZG8gd2UgcmVhbGx5 IG5lZWQgYSBtdXRleCBoZXJlPyAqLworCWlmIChibG9ja2FibGUpCisJCW11dGV4X2xvY2soJnBy aXYtPmxvY2spOworCWVsc2UgaWYgKCFtdXRleF90cnlsb2NrKCZwcml2LT5sb2NrKSkKKwkJcmV0 dXJuIC1FQUdBSU47CiAKLQltdXRleF9sb2NrKCZwcml2LT5sb2NrKTsKIAlsaXN0X2Zvcl9lYWNo X2VudHJ5KG1hcCwgJnByaXYtPm1hcHMsIG5leHQpIHsKKwkJaWYgKGluX3JhbmdlKG1hcCwgc3Rh cnQsIGVuZCkpIHsKKwkJCXJldCA9IC1FQUdBSU47CisJCQlnb3RvIG91dF91bmxvY2s7CisJCX0K IAkJdW5tYXBfaWZfaW5fcmFuZ2UobWFwLCBzdGFydCwgZW5kKTsKIAl9CiAJbGlzdF9mb3JfZWFj aF9lbnRyeShtYXAsICZwcml2LT5mcmVlYWJsZV9tYXBzLCBuZXh0KSB7CisJCWlmIChpbl9yYW5n ZShtYXAsIHN0YXJ0LCBlbmQpKSB7CisJCQlyZXQgPSAtRUFHQUlOOworCQkJZ290byBvdXRfdW5s b2NrOworCQl9CiAJCXVubWFwX2lmX2luX3JhbmdlKG1hcCwgc3RhcnQsIGVuZCk7CiAJfQorCitv dXRfdW5sb2NrOgogCW11dGV4X3VubG9jaygmcHJpdi0+bG9jayk7CisKKwlyZXR1cm4gcmV0Owog fQogCiBzdGF0aWMgdm9pZCBtbl9yZWxlYXNlKHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLApkaWZm IC0tZ2l0IGEvaW5jbHVkZS9saW51eC9rdm1faG9zdC5oIGIvaW5jbHVkZS9saW51eC9rdm1faG9z dC5oCmluZGV4IDRlZTdiYzU0OGE4My4uMTQ4OTM1MDg1MTk0IDEwMDY0NAotLS0gYS9pbmNsdWRl L2xpbnV4L2t2bV9ob3N0LmgKKysrIGIvaW5jbHVkZS9saW51eC9rdm1faG9zdC5oCkBAIC0xMjc1 LDggKzEyNzUsOCBAQCBzdGF0aWMgaW5saW5lIGxvbmcga3ZtX2FyY2hfdmNwdV9hc3luY19pb2N0 bChzdHJ1Y3QgZmlsZSAqZmlscCwKIH0KICNlbmRpZiAvKiBDT05GSUdfSEFWRV9LVk1fVkNQVV9B U1lOQ19JT0NUTCAqLwogCi12b2lkIGt2bV9hcmNoX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jh bmdlKHN0cnVjdCBrdm0gKmt2bSwKLQkJdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9u ZyBlbmQpOworaW50IGt2bV9hcmNoX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKHN0cnVj dCBrdm0gKmt2bSwKKwkJdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQsIGJv b2wgYmxvY2thYmxlKTsKIAogI2lmZGVmIENPTkZJR19IQVZFX0tWTV9WQ1BVX1JVTl9QSURfQ0hB TkdFCiBpbnQga3ZtX2FyY2hfdmNwdV9ydW5fcGlkX2NoYW5nZShzdHJ1Y3Qga3ZtX3ZjcHUgKnZj cHUpOwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbXVfbm90aWZpZXIuaCBiL2luY2x1ZGUv bGludXgvbW11X25vdGlmaWVyLmgKaW5kZXggMzkyZTZhZjgyNzAxLi4yZWIxYTJkMDE3NTkgMTAw NjQ0Ci0tLSBhL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmgKKysrIGIvaW5jbHVkZS9saW51 eC9tbXVfbm90aWZpZXIuaApAQCAtMTUxLDEzICsxNTEsMTUgQEAgc3RydWN0IG1tdV9ub3RpZmll cl9vcHMgewogCSAqIGFkZHJlc3Mgc3BhY2UgYnV0IG1heSBzdGlsbCBiZSByZWZlcmVuY2VkIGJ5 IHNwdGVzIHVudGlsCiAJICogdGhlIGxhc3QgcmVmY291bnQgaXMgZHJvcHBlZC4KIAkgKgotCSAq IElmIGJvdGggb2YgdGhlc2UgY2FsbGJhY2tzIGNhbm5vdCBibG9jaywgYW5kIGludmFsaWRhdGVf cmFuZ2UKLQkgKiBjYW5ub3QgYmxvY2ssIG1tdV9ub3RpZmllcl9vcHMuZmxhZ3Mgc2hvdWxkIGhh dmUKLQkgKiBNTVVfSU5WQUxJREFURV9ET0VTX05PVF9CTE9DSyBzZXQuCisJICogSWYgYmxvY2th YmxlIGFyZ3VtZW50IGlzIHNldCB0byBmYWxzZSB0aGVuIHRoZSBjYWxsYmFjayBjYW5ub3QKKwkg KiBzbGVlcCBhbmQgaGFzIHRvIHJldHVybiB3aXRoIC1FQUdBSU4uIDAgc2hvdWxkIGJlIHJldHVy bmVkCisJICogb3RoZXJ3aXNlLgorCSAqCiAJICovCi0Jdm9pZCAoKmludmFsaWRhdGVfcmFuZ2Vf c3RhcnQpKHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAorCWludCAoKmludmFsaWRhdGVfcmFuZ2Vf c3RhcnQpKHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQkJCSAgICAgICBzdHJ1Y3QgbW1fc3Ry dWN0ICptbSwKLQkJCQkgICAgICAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBl bmQpOworCQkJCSAgICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIGVuZCwK KwkJCQkgICAgICAgYm9vbCBibG9ja2FibGUpOwogCXZvaWQgKCppbnZhbGlkYXRlX3JhbmdlX2Vu ZCkoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCiAJCQkJICAgICBzdHJ1Y3QgbW1fc3RydWN0ICpt bSwKIAkJCQkgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kKTsKQEAg LTIyOSw4ICsyMzEsOSBAQCBleHRlcm4gaW50IF9fbW11X25vdGlmaWVyX3Rlc3RfeW91bmcoc3Ry dWN0IG1tX3N0cnVjdCAqbW0sCiAJCQkJICAgICB1bnNpZ25lZCBsb25nIGFkZHJlc3MpOwogZXh0 ZXJuIHZvaWQgX19tbXVfbm90aWZpZXJfY2hhbmdlX3B0ZShzdHJ1Y3QgbW1fc3RydWN0ICptbSwK IAkJCQkgICAgICB1bnNpZ25lZCBsb25nIGFkZHJlc3MsIHB0ZV90IHB0ZSk7Ci1leHRlcm4gdm9p ZCBfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbV9zdHJ1Y3Qg Km1tLAotCQkJCSAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQpOworZXh0 ZXJuIGludCBfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbV9z dHJ1Y3QgKm1tLAorCQkJCSAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQs CisJCQkJICBib29sIGJsb2NrYWJsZSk7CiBleHRlcm4gdm9pZCBfX21tdV9ub3RpZmllcl9pbnZh bGlkYXRlX3JhbmdlX2VuZChzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkgIHVuc2lnbmVkIGxv bmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kLAogCQkJCSAgYm9vbCBvbmx5X2VuZCk7CkBAIC0y ODEsNyArMjg0LDE3IEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBtbXVfbm90aWZpZXJfaW52YWxpZGF0 ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkgIHVuc2lnbmVkIGxvbmcg c3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kKQogewogCWlmIChtbV9oYXNfbm90aWZpZXJzKG1tKSkK LQkJX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChtbSwgc3RhcnQsIGVuZCk7 CisJCV9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQobW0sIHN0YXJ0LCBlbmQs IHRydWUpOworfQorCitzdGF0aWMgaW5saW5lIGludCBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9y YW5nZV9zdGFydF9ub25ibG9jayhzdHJ1Y3QgbW1fc3RydWN0ICptbSwKKwkJCQkgIHVuc2lnbmVk IGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kKQoreworCWludCByZXQgPSAwOworCWlmICht bV9oYXNfbm90aWZpZXJzKG1tKSkKKwkJcmV0ID0gX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9y YW5nZV9zdGFydChtbSwgc3RhcnQsIGVuZCwgZmFsc2UpOworCisJcmV0dXJuIHJldDsKIH0KIAog c3RhdGljIGlubGluZSB2b2lkIG1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX2VuZChzdHJ1 Y3QgbW1fc3RydWN0ICptbSwKQEAgLTQ2MSw2ICs0NzQsMTIgQEAgc3RhdGljIGlubGluZSB2b2lk IG1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbV9zdHJ1Y3QgKm1t LAogewogfQogCitzdGF0aWMgaW5saW5lIGludCBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5n ZV9zdGFydF9ub25ibG9jayhzdHJ1Y3QgbW1fc3RydWN0ICptbSwKKwkJCQkgIHVuc2lnbmVkIGxv bmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgZW5kKQoreworCXJldHVybiAwOworfQorCiBzdGF0aWMg aW5saW5lIHZvaWQgbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2VfZW5kKHN0cnVjdCBtbV9z dHJ1Y3QgKm1tLAogCQkJCSAgdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQp CiB7CmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L29vbS5oIGIvaW5jbHVkZS9saW51eC9vb20u aAppbmRleCA2YWRhYzExM2U5NmQuLjkyZjcwZTRjNjI1MiAxMDA2NDQKLS0tIGEvaW5jbHVkZS9s aW51eC9vb20uaAorKysgYi9pbmNsdWRlL2xpbnV4L29vbS5oCkBAIC05NSw3ICs5NSw3IEBAIHN0 YXRpYyBpbmxpbmUgaW50IGNoZWNrX3N0YWJsZV9hZGRyZXNzX3NwYWNlKHN0cnVjdCBtbV9zdHJ1 Y3QgKm1tKQogCXJldHVybiAwOwogfQogCi12b2lkIF9fb29tX3JlYXBfdGFza19tbShzdHJ1Y3Qg bW1fc3RydWN0ICptbSk7Citib29sIF9fb29tX3JlYXBfdGFza19tbShzdHJ1Y3QgbW1fc3RydWN0 ICptbSk7CiAKIGV4dGVybiB1bnNpZ25lZCBsb25nIG9vbV9iYWRuZXNzKHN0cnVjdCB0YXNrX3N0 cnVjdCAqcCwKIAkJc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnLCBjb25zdCBub2RlbWFza190ICpu b2RlbWFzaywKZGlmZiAtLWdpdCBhL2luY2x1ZGUvcmRtYS9pYl91bWVtX29kcC5oIGIvaW5jbHVk ZS9yZG1hL2liX3VtZW1fb2RwLmgKaW5kZXggNmExN2Y4NTZmODQxLi4zODFjZGY1YTliZDEgMTAw NjQ0Ci0tLSBhL2luY2x1ZGUvcmRtYS9pYl91bWVtX29kcC5oCisrKyBiL2luY2x1ZGUvcmRtYS9p Yl91bWVtX29kcC5oCkBAIC0xMTksNyArMTE5LDggQEAgdHlwZWRlZiBpbnQgKCp1bWVtX2NhbGxf YmFjaykoc3RydWN0IGliX3VtZW0gKml0ZW0sIHU2NCBzdGFydCwgdTY0IGVuZCwKICAqLwogaW50 IHJidF9pYl91bWVtX2Zvcl9lYWNoX2luX3JhbmdlKHN0cnVjdCByYl9yb290X2NhY2hlZCAqcm9v dCwKIAkJCQkgIHU2NCBzdGFydCwgdTY0IGVuZCwKLQkJCQkgIHVtZW1fY2FsbF9iYWNrIGNiLCB2 b2lkICpjb29raWUpOworCQkJCSAgdW1lbV9jYWxsX2JhY2sgY2IsCisJCQkJICBib29sIGJsb2Nr YWJsZSwgdm9pZCAqY29va2llKTsKIAogLyoKICAqIEZpbmQgZmlyc3QgcmVnaW9uIGludGVyc2Vj dGluZyB3aXRoIGFkZHJlc3MgcmFuZ2UuCmRpZmYgLS1naXQgYS9tbS9obW0uYyBiL21tL2htbS5j CmluZGV4IGRlN2I2YmY3NzIwMS4uODFmZDU3YmQyNjM0IDEwMDY0NAotLS0gYS9tbS9obW0uYwor KysgYi9tbS9obW0uYwpAQCAtMTc3LDE2ICsxNzcsMTkgQEAgc3RhdGljIHZvaWQgaG1tX3JlbGVh c2Uoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQogCXVwX3dy aXRlKCZobW0tPm1pcnJvcnNfc2VtKTsKIH0KIAotc3RhdGljIHZvaWQgaG1tX2ludmFsaWRhdGVf cmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCitzdGF0aWMgaW50IGhtbV9pbnZh bGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQkJCSAgICAgICBz dHJ1Y3QgbW1fc3RydWN0ICptbSwKIAkJCQkgICAgICAgdW5zaWduZWQgbG9uZyBzdGFydCwKLQkJ CQkgICAgICAgdW5zaWduZWQgbG9uZyBlbmQpCisJCQkJICAgICAgIHVuc2lnbmVkIGxvbmcgZW5k LAorCQkJCSAgICAgICBib29sIGJsb2NrYWJsZSkKIHsKIAlzdHJ1Y3QgaG1tICpobW0gPSBtbS0+ aG1tOwogCiAJVk1fQlVHX09OKCFobW0pOwogCiAJYXRvbWljX2luYygmaG1tLT5zZXF1ZW5jZSk7 CisKKwlyZXR1cm4gMDsKIH0KIAogc3RhdGljIHZvaWQgaG1tX2ludmFsaWRhdGVfcmFuZ2VfZW5k KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLApkaWZmIC0tZ2l0IGEvbW0vbW1hcC5jIGIvbW0vbW1h cC5jCmluZGV4IGQxZWI4N2VmNGIxYS4uMzM2YmVlOGM0ZTI1IDEwMDY0NAotLS0gYS9tbS9tbWFw LmMKKysrIGIvbW0vbW1hcC5jCkBAIC0zMDc0LDcgKzMwNzQsNyBAQCB2b2lkIGV4aXRfbW1hcChz dHJ1Y3QgbW1fc3RydWN0ICptbSkKIAkJICogcmVsaWFibHkgdGVzdCBpdC4KIAkJICovCiAJCW11 dGV4X2xvY2soJm9vbV9sb2NrKTsKLQkJX19vb21fcmVhcF90YXNrX21tKG1tKTsKKwkJKHZvaWQp X19vb21fcmVhcF90YXNrX21tKG1tKTsKIAkJbXV0ZXhfdW5sb2NrKCZvb21fbG9jayk7CiAKIAkJ c2V0X2JpdChNTUZfT09NX1NLSVAsICZtbS0+ZmxhZ3MpOwpkaWZmIC0tZ2l0IGEvbW0vbW11X25v dGlmaWVyLmMgYi9tbS9tbXVfbm90aWZpZXIuYwppbmRleCBlZmY2Yjg4YTk5M2YuLjEwM2IyYjQ1 MDA0MyAxMDA2NDQKLS0tIGEvbW0vbW11X25vdGlmaWVyLmMKKysrIGIvbW0vbW11X25vdGlmaWVy LmMKQEAgLTE3NCwxOCArMTc0LDI5IEBAIHZvaWQgX19tbXVfbm90aWZpZXJfY2hhbmdlX3B0ZShz dHJ1Y3QgbW1fc3RydWN0ICptbSwgdW5zaWduZWQgbG9uZyBhZGRyZXNzLAogCXNyY3VfcmVhZF91 bmxvY2soJnNyY3UsIGlkKTsKIH0KIAotdm9pZCBfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAotCQkJCSAgdW5zaWduZWQgbG9uZyBzdGFy dCwgdW5zaWduZWQgbG9uZyBlbmQpCitpbnQgX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5n ZV9zdGFydChzdHJ1Y3QgbW1fc3RydWN0ICptbSwKKwkJCQkgIHVuc2lnbmVkIGxvbmcgc3RhcnQs IHVuc2lnbmVkIGxvbmcgZW5kLAorCQkJCSAgYm9vbCBibG9ja2FibGUpCiB7CiAJc3RydWN0IG1t dV9ub3RpZmllciAqbW47CisJaW50IHJldCA9IDA7CiAJaW50IGlkOwogCiAJaWQgPSBzcmN1X3Jl YWRfbG9jaygmc3JjdSk7CiAJaGxpc3RfZm9yX2VhY2hfZW50cnlfcmN1KG1uLCAmbW0tPm1tdV9u b3RpZmllcl9tbS0+bGlzdCwgaGxpc3QpIHsKLQkJaWYgKG1uLT5vcHMtPmludmFsaWRhdGVfcmFu Z2Vfc3RhcnQpCi0JCQltbi0+b3BzLT5pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KG1uLCBtbSwgc3Rh cnQsIGVuZCk7CisJCWlmIChtbi0+b3BzLT5pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KSB7CisJCQlp bnQgX3JldCA9IG1uLT5vcHMtPmludmFsaWRhdGVfcmFuZ2Vfc3RhcnQobW4sIG1tLCBzdGFydCwg ZW5kLCBibG9ja2FibGUpOworCQkJaWYgKF9yZXQpIHsKKwkJCQlwcl9pbmZvKCIlcFMgY2FsbGJh Y2sgZmFpbGVkIHdpdGggJWQgaW4gJXNibG9ja2FibGUgY29udGV4dC5cbiIsCisJCQkJCQltbi0+ b3BzLT5pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0LCBfcmV0LAorCQkJCQkJIWJsb2NrYWJsZSA/ICJu b24tIjogIiIpOworCQkJCXJldCA9IF9yZXQ7CisJCQl9CisJCX0KIAl9CiAJc3JjdV9yZWFkX3Vu bG9jaygmc3JjdSwgaWQpOworCisJcmV0dXJuIHJldDsKIH0KIEVYUE9SVF9TWU1CT0xfR1BMKF9f bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQpOwogCmRpZmYgLS1naXQgYS9tbS9v b21fa2lsbC5jIGIvbW0vb29tX2tpbGwuYwppbmRleCA4NDA4MWU3N2JjNTEuLjVhOTM2Y2YyNGQ3 OSAxMDA2NDQKLS0tIGEvbW0vb29tX2tpbGwuYworKysgYi9tbS9vb21fa2lsbC5jCkBAIC00Nzks OSArNDc5LDEwIEBAIHN0YXRpYyBERUNMQVJFX1dBSVRfUVVFVUVfSEVBRChvb21fcmVhcGVyX3dh aXQpOwogc3RhdGljIHN0cnVjdCB0YXNrX3N0cnVjdCAqb29tX3JlYXBlcl9saXN0Owogc3RhdGlj IERFRklORV9TUElOTE9DSyhvb21fcmVhcGVyX2xvY2spOwogCi12b2lkIF9fb29tX3JlYXBfdGFz a19tbShzdHJ1Y3QgbW1fc3RydWN0ICptbSkKK2Jvb2wgX19vb21fcmVhcF90YXNrX21tKHN0cnVj dCBtbV9zdHJ1Y3QgKm1tKQogewogCXN0cnVjdCB2bV9hcmVhX3N0cnVjdCAqdm1hOworCWJvb2wg cmV0ID0gdHJ1ZTsKIAogCS8qCiAJICogVGVsbCBhbGwgdXNlcnMgb2YgZ2V0X3VzZXIvY29weV9m cm9tX3VzZXIgZXRjLi4uIHRoYXQgdGhlIGNvbnRlbnQKQEAgLTUxMSwxMiArNTEyLDE3IEBAIHZv aWQgX19vb21fcmVhcF90YXNrX21tKHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQogCQkJc3RydWN0IG1t dV9nYXRoZXIgdGxiOwogCiAJCQl0bGJfZ2F0aGVyX21tdSgmdGxiLCBtbSwgc3RhcnQsIGVuZCk7 Ci0JCQltbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydChtbSwgc3RhcnQsIGVuZCk7 CisJCQlpZiAobW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbm9uYmxvY2sobW0s IHN0YXJ0LCBlbmQpKSB7CisJCQkJcmV0ID0gZmFsc2U7CisJCQkJY29udGludWU7CisJCQl9CiAJ CQl1bm1hcF9wYWdlX3JhbmdlKCZ0bGIsIHZtYSwgc3RhcnQsIGVuZCwgTlVMTCk7CiAJCQltbXVf bm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQobW0sIHN0YXJ0LCBlbmQpOwogCQkJdGxiX2Zp bmlzaF9tbXUoJnRsYiwgc3RhcnQsIGVuZCk7CiAJCX0KIAl9CisKKwlyZXR1cm4gcmV0OwogfQog CiBzdGF0aWMgYm9vbCBvb21fcmVhcF90YXNrX21tKHN0cnVjdCB0YXNrX3N0cnVjdCAqdHNrLCBz dHJ1Y3QgbW1fc3RydWN0ICptbSkKQEAgLTU0NSwxOCArNTUxLDYgQEAgc3RhdGljIGJvb2wgb29t X3JlYXBfdGFza19tbShzdHJ1Y3QgdGFza19zdHJ1Y3QgKnRzaywgc3RydWN0IG1tX3N0cnVjdCAq bW0pCiAJCWdvdG8gdW5sb2NrX29vbTsKIAl9CiAKLQkvKgotCSAqIElmIHRoZSBtbSBoYXMgaW52 YWxpZGF0ZV97c3RhcnQsZW5kfSgpIG5vdGlmaWVycyB0aGF0IGNvdWxkIGJsb2NrLAotCSAqIHNs ZWVwIHRvIGdpdmUgdGhlIG9vbSB2aWN0aW0gc29tZSBtb3JlIHRpbWUuCi0JICogVE9ETzogd2Ug cmVhbGx5IHdhbnQgdG8gZ2V0IHJpZCBvZiB0aGlzIHVnbHkgaGFjayBhbmQgbWFrZSBzdXJlIHRo YXQKLQkgKiBub3RpZmllcnMgY2Fubm90IGJsb2NrIGZvciB1bmJvdW5kZWQgYW1vdW50IG9mIHRp bWUKLQkgKi8KLQlpZiAobW1faGFzX2Jsb2NrYWJsZV9pbnZhbGlkYXRlX25vdGlmaWVycyhtbSkp IHsKLQkJdXBfcmVhZCgmbW0tPm1tYXBfc2VtKTsKLQkJc2NoZWR1bGVfdGltZW91dF9pZGxlKEha KTsKLQkJZ290byB1bmxvY2tfb29tOwotCX0KLQogCS8qCiAJICogTU1GX09PTV9TS0lQIGlzIHNl dCBieSBleGl0X21tYXAgd2hlbiB0aGUgT09NIHJlYXBlciBjYW4ndAogCSAqIHdvcmsgb24gdGhl IG1tIGFueW1vcmUuIFRoZSBjaGVjayBmb3IgTU1GX09PTV9TS0lQIG11c3QgcnVuCkBAIC01NzEs NyArNTY1LDEyIEBAIHN0YXRpYyBib29sIG9vbV9yZWFwX3Rhc2tfbW0oc3RydWN0IHRhc2tfc3Ry dWN0ICp0c2ssIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQogCiAJdHJhY2Vfc3RhcnRfdGFza19yZWFw aW5nKHRzay0+cGlkKTsKIAotCV9fb29tX3JlYXBfdGFza19tbShtbSk7CisJLyogZmFpbGVkIHRv IHJlYXAgcGFydCBvZiB0aGUgYWRkcmVzcyBzcGFjZS4gVHJ5IGFnYWluIGxhdGVyICovCisJaWYg KCFfX29vbV9yZWFwX3Rhc2tfbW0obW0pKSB7CisJCXVwX3JlYWQoJm1tLT5tbWFwX3NlbSk7CisJ CXJldCA9IGZhbHNlOworCQlnb3RvIHVubG9ja19vb207CisJfQogCiAJcHJfaW5mbygib29tX3Jl YXBlcjogcmVhcGVkIHByb2Nlc3MgJWQgKCVzKSwgbm93IGFub24tcnNzOiVsdWtCLCBmaWxlLXJz czolbHVrQiwgc2htZW0tcnNzOiVsdWtCXG4iLAogCQkJdGFza19waWRfbnIodHNrKSwgdHNrLT5j b21tLApkaWZmIC0tZ2l0IGEvdmlydC9rdm0va3ZtX21haW4uYyBiL3ZpcnQva3ZtL2t2bV9tYWlu LmMKaW5kZXggYWRhMjFmNDdmMjJiLi4xNmNlMzhmMTc4ZDEgMTAwNjQ0Ci0tLSBhL3ZpcnQva3Zt L2t2bV9tYWluLmMKKysrIGIvdmlydC9rdm0va3ZtX21haW4uYwpAQCAtMTM1LDkgKzEzNSwxMCBA QCBzdGF0aWMgdm9pZCBrdm1fdWV2ZW50X25vdGlmeV9jaGFuZ2UodW5zaWduZWQgaW50IHR5cGUs IHN0cnVjdCBrdm0gKmt2bSk7CiBzdGF0aWMgdW5zaWduZWQgbG9uZyBsb25nIGt2bV9jcmVhdGV2 bV9jb3VudDsKIHN0YXRpYyB1bnNpZ25lZCBsb25nIGxvbmcga3ZtX2FjdGl2ZV92bXM7CiAKLV9f d2VhayB2b2lkIGt2bV9hcmNoX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKHN0cnVjdCBr dm0gKmt2bSwKLQkJdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQpCitfX3dl YWsgaW50IGt2bV9hcmNoX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKHN0cnVjdCBrdm0g Kmt2bSwKKwkJdW5zaWduZWQgbG9uZyBzdGFydCwgdW5zaWduZWQgbG9uZyBlbmQsIGJvb2wgYmxv Y2thYmxlKQogeworCXJldHVybiAwOwogfQogCiBib29sIGt2bV9pc19yZXNlcnZlZF9wZm4oa3Zt X3Bmbl90IHBmbikKQEAgLTM1NCwxMyArMzU1LDE1IEBAIHN0YXRpYyB2b2lkIGt2bV9tbXVfbm90 aWZpZXJfY2hhbmdlX3B0ZShzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKIAlzcmN1X3JlYWRfdW5s b2NrKCZrdm0tPnNyY3UsIGlkeCk7CiB9CiAKLXN0YXRpYyB2b2lkIGt2bV9tbXVfbm90aWZpZXJf aW52YWxpZGF0ZV9yYW5nZV9zdGFydChzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwKK3N0YXRpYyBp bnQga3ZtX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90 aWZpZXIgKm1uLAogCQkJCQkJICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLAogCQkJCQkJICAgIHVu c2lnbmVkIGxvbmcgc3RhcnQsCi0JCQkJCQkgICAgdW5zaWduZWQgbG9uZyBlbmQpCisJCQkJCQkg ICAgdW5zaWduZWQgbG9uZyBlbmQsCisJCQkJCQkgICAgYm9vbCBibG9ja2FibGUpCiB7CiAJc3Ry dWN0IGt2bSAqa3ZtID0gbW11X25vdGlmaWVyX3RvX2t2bShtbik7CiAJaW50IG5lZWRfdGxiX2Zs dXNoID0gMCwgaWR4OworCWludCByZXQ7CiAKIAlpZHggPSBzcmN1X3JlYWRfbG9jaygma3ZtLT5z cmN1KTsKIAlzcGluX2xvY2soJmt2bS0+bW11X2xvY2spOwpAQCAtMzc4LDkgKzM4MSwxMSBAQCBz dGF0aWMgdm9pZCBrdm1fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0 IG1tdV9ub3RpZmllciAqbW4sCiAKIAlzcGluX3VubG9jaygma3ZtLT5tbXVfbG9jayk7CiAKLQlr dm1fYXJjaF9tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZShrdm0sIHN0YXJ0LCBlbmQpOwor CXJldCA9IGt2bV9hcmNoX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKGt2bSwgc3RhcnQs IGVuZCwgYmxvY2thYmxlKTsKIAogCXNyY3VfcmVhZF91bmxvY2soJmt2bS0+c3JjdSwgaWR4KTsK KworCXJldHVybiByZXQ7CiB9CiAKIHN0YXRpYyB2b2lkIGt2bV9tbXVfbm90aWZpZXJfaW52YWxp ZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sCi0tIAoyLjE4LjAKCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmFtZC1nZnggbWFpbGlu ZyBsaXN0CmFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRl c2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vYW1kLWdmeAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id C36EA6B0003 for ; Mon, 16 Jul 2018 07:51:12 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id c2-v6so15203627edi.20 for ; Mon, 16 Jul 2018 04:51:12 -0700 (PDT) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id q4-v6sor8178490eds.28.2018.07.16.04.51.10 for (Google Transport Security); Mon, 16 Jul 2018 04:51:10 -0700 (PDT) From: Michal Hocko Subject: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Mon, 16 Jul 2018 13:50:58 +0200 Message-Id: <20180716115058.5559-1-mhocko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: LKML , linux-mm@kvack.org, Michal Hocko , "David (ChunMing) Zhou" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Alex Deucher , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Doug Ledford , Jason Gunthorpe , Mike Marciniszyn , Dennis Dalessandro , Sudeep Dutt , Ashutosh Dixit , Dimitri Sivanich , Boris Ostrovsky , Juergen Gross , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrea Arcangeli , Felix Kuehling , kvm@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-rdma@vger.kernel.org, xen-devel@lists.xenproject.org, =?UTF-8?q?Christian=20K=C3=B6nig?= , David Rientjes , Leon Romanovsky From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an oom victim with blockable mmu notifiers as done after a short sleep. That can result in selecting a new oom victim prematurely because the previous one still hasn't torn its memory down yet. We can do much better though. Even if mmu notifiers use sleepable locks there is no reason to automatically assume those locks are held. Moreover majority of notifiers only care about a portion of the address space and there is absolutely zero reason to fail when we are unmapping an unrelated range. Many notifiers do really block and wait for HW which is harder to handle and we have to bail out though. This patch handles the low hanging fruid. __mmu_notifier_invalidate_range_start gets a blockable flag and callbacks are not allowed to sleep if the flag is set to false. This is achieved by using trylock instead of the sleepable lock for most callbacks and continue as long as we do not block down the call chain. I think we can improve that even further because there is a common pattern to do a range lookup first and then do something about that. The first part can be done without a sleeping lock in most cases AFAICS. The oom_reaper end then simply retries if there is at least one notifier which couldn't make any progress in !blockable mode. A retry loop is already implemented to wait for the mmap_sem and this is basically the same thing. Changes since rfc v1 - gpu notifiers can sleep while waiting for HW (evict_process_queues_cpsch on a lock and amdgpu_mn_invalidate_node on unbound timeout) make sure we bail out when we have an intersecting range for starter - note that a notifier failed to the log for easier debugging - back off early in ib_umem_notifier_invalidate_range_start if the callback is called - mn_invl_range_start waits for completion down the unmap_grant_pages path so we have to back off early on overlapping ranges Cc: "David (ChunMing) Zhou" Cc: Paolo Bonzini Cc: "Radim KrA?mA!A?" Cc: Alex Deucher Cc: David Airlie Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Rodrigo Vivi Cc: Doug Ledford Cc: Jason Gunthorpe Cc: Mike Marciniszyn Cc: Dennis Dalessandro Cc: Sudeep Dutt Cc: Ashutosh Dixit Cc: Dimitri Sivanich Cc: Boris Ostrovsky Cc: Juergen Gross Cc: "JA(C)rA'me Glisse" Cc: Andrea Arcangeli Cc: Felix Kuehling Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: amd-gfx@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org Cc: intel-gfx@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: xen-devel@lists.xenproject.org Cc: linux-mm@kvack.org Acked-by: Christian KA?nig # AMD notifiers Acked-by: Leon Romanovsky # mlx and umem_odp Reported-by: David Rientjes Signed-off-by: Michal Hocko --- Hi, there were no major objections when I sent this as an RFC the last time [1]. I was hoping for more feedback in the drivers land because I am touching the code I have no way to test. On the other hand the pattern is quite simple and consistent over all users so there shouldn't be any large surprises hopefully. Any further review would be highly appreciate of course. But is this something to put into the mm tree now? [1] http://lkml.kernel.org/r/20180627074421.GF32348@dhcp22.suse.cz arch/x86/kvm/x86.c | 7 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 43 +++++++++++++++++++----- drivers/gpu/drm/i915/i915_gem_userptr.c | 13 ++++++-- drivers/gpu/drm/radeon/radeon_mn.c | 22 +++++++++++-- drivers/infiniband/core/umem_odp.c | 33 +++++++++++++++---- drivers/infiniband/hw/hfi1/mmu_rb.c | 11 ++++--- drivers/infiniband/hw/mlx5/odp.c | 2 +- drivers/misc/mic/scif/scif_dma.c | 7 ++-- drivers/misc/sgi-gru/grutlbpurge.c | 7 ++-- drivers/xen/gntdev.c | 44 ++++++++++++++++++++----- include/linux/kvm_host.h | 4 +-- include/linux/mmu_notifier.h | 35 +++++++++++++++----- include/linux/oom.h | 2 +- include/rdma/ib_umem_odp.h | 3 +- mm/hmm.c | 7 ++-- mm/mmap.c | 2 +- mm/mmu_notifier.c | 19 ++++++++--- mm/oom_kill.c | 29 ++++++++-------- virt/kvm/kvm_main.c | 15 ++++++--- 19 files changed, 225 insertions(+), 80 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6bcecc325e7e..ac08f5d711be 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7203,8 +7203,9 @@ static void vcpu_load_eoi_exitmap(struct kvm_vcpu *vcpu) kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap); } -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, + bool blockable) { unsigned long apic_address; @@ -7215,6 +7216,8 @@ void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, apic_address = gfn_to_hva(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); if (start <= apic_address && apic_address < end) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); + + return 0; } void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 83e344fbb50a..3399a4a927fb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -136,12 +136,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) * * Take the rmn read side lock. */ -static void amdgpu_mn_read_lock(struct amdgpu_mn *rmn) +static int amdgpu_mn_read_lock(struct amdgpu_mn *rmn, bool blockable) { - mutex_lock(&rmn->read_lock); + if (blockable) + mutex_lock(&rmn->read_lock); + else if (!mutex_trylock(&rmn->read_lock)) + return -EAGAIN; + if (atomic_inc_return(&rmn->recursion) == 1) down_read_non_owner(&rmn->lock); mutex_unlock(&rmn->read_lock); + + return 0; } /** @@ -197,10 +203,11 @@ static void amdgpu_mn_invalidate_node(struct amdgpu_mn_node *node, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -208,17 +215,28 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + /* TODO we should be able to split locking for interval tree and + * amdgpu_mn_invalidate_node + */ + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { struct amdgpu_mn_node *node; + if (!blockable) { + amdgpu_mn_read_unlock(rmn); + return -EAGAIN; + } + node = container_of(it, struct amdgpu_mn_node, it); it = interval_tree_iter_next(it, start, end); amdgpu_mn_invalidate_node(node, start, end); } + + return 0; } /** @@ -233,10 +251,11 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, * necessitates evicting all user-mode queues of the process. The BOs * are restorted in amdgpu_mn_invalidate_range_end_hsa. */ -static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -244,13 +263,19 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { struct amdgpu_mn_node *node; struct amdgpu_bo *bo; + if (!blockable) { + amdgpu_mn_read_unlock(rmn); + return -EAGAIN; + } + node = container_of(it, struct amdgpu_mn_node, it); it = interval_tree_iter_next(it, start, end); @@ -262,6 +287,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, amdgpu_amdkfd_evict_userptr(mem, mm); } } + + return 0; } /** diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 854bd51b9478..9cbff68f6b41 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -112,10 +112,11 @@ static void del_object(struct i915_mmu_object *mo) mo->attached = false; } -static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, +static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct i915_mmu_notifier *mn = container_of(_mn, struct i915_mmu_notifier, mn); @@ -124,7 +125,7 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, LIST_HEAD(cancelled); if (RB_EMPTY_ROOT(&mn->objects.rb_root)) - return; + return 0; /* interval ranges are inclusive, but invalidate range is exclusive */ end--; @@ -132,6 +133,10 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, spin_lock(&mn->lock); it = interval_tree_iter_first(&mn->objects, start, end); while (it) { + if (!blockable) { + spin_unlock(&mn->lock); + return -EAGAIN; + } /* The mmu_object is released late when destroying the * GEM object so it is entirely possible to gain a * reference on an object in the process of being freed @@ -154,6 +159,8 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, if (!list_empty(&cancelled)) flush_workqueue(mn->wq); + + return 0; } static const struct mmu_notifier_ops i915_gem_userptr_notifier = { diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index abd24975c9b1..f8b35df44c60 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -118,19 +118,27 @@ static void radeon_mn_release(struct mmu_notifier *mn, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, +static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct radeon_mn *rmn = container_of(mn, struct radeon_mn, mn); struct ttm_operation_ctx ctx = { false, false }; struct interval_tree_node *it; + int ret = 0; /* notification is exclusive, but interval is inclusive */ end -= 1; - mutex_lock(&rmn->lock); + /* TODO we should be able to split locking for interval tree and + * the tear down. + */ + if (blockable) + mutex_lock(&rmn->lock); + else if (!mutex_trylock(&rmn->lock)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -138,6 +146,11 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct radeon_bo *bo; long r; + if (!blockable) { + ret = -EAGAIN; + goto out_unlock; + } + node = container_of(it, struct radeon_mn_node, it); it = interval_tree_iter_next(it, start, end); @@ -166,7 +179,10 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, } } +out_unlock: mutex_unlock(&rmn->lock); + + return ret; } static const struct mmu_notifier_ops radeon_mn_ops = { diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index 182436b92ba9..6ec748eccff7 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -186,6 +186,7 @@ static void ib_umem_notifier_release(struct mmu_notifier *mn, rbt_ib_umem_for_each_in_range(&context->umem_tree, 0, ULLONG_MAX, ib_umem_notifier_release_trampoline, + true, NULL); up_read(&context->umem_rwsem); } @@ -207,22 +208,31 @@ static int invalidate_range_start_trampoline(struct ib_umem *item, u64 start, return 0; } -static void ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct ib_ucontext *context = container_of(mn, struct ib_ucontext, mn); + int ret; if (!context->invalidate_range) - return; + return 0; + + if (blockable) + down_read(&context->umem_rwsem); + else if (!down_read_trylock(&context->umem_rwsem)) + return -EAGAIN; ib_ucontext_notifier_start_account(context); - down_read(&context->umem_rwsem); - rbt_ib_umem_for_each_in_range(&context->umem_tree, start, + ret = rbt_ib_umem_for_each_in_range(&context->umem_tree, start, end, - invalidate_range_start_trampoline, NULL); + invalidate_range_start_trampoline, + blockable, NULL); up_read(&context->umem_rwsem); + + return ret; } static int invalidate_range_end_trampoline(struct ib_umem *item, u64 start, @@ -242,10 +252,15 @@ static void ib_umem_notifier_invalidate_range_end(struct mmu_notifier *mn, if (!context->invalidate_range) return; + /* + * TODO: we currently bail out if there is any sleepable work to be done + * in ib_umem_notifier_invalidate_range_start so we shouldn't really block + * here. But this is ugly and fragile. + */ down_read(&context->umem_rwsem); rbt_ib_umem_for_each_in_range(&context->umem_tree, start, end, - invalidate_range_end_trampoline, NULL); + invalidate_range_end_trampoline, true, NULL); up_read(&context->umem_rwsem); ib_ucontext_notifier_end_account(context); } @@ -798,6 +813,7 @@ EXPORT_SYMBOL(ib_umem_odp_unmap_dma_pages); int rbt_ib_umem_for_each_in_range(struct rb_root_cached *root, u64 start, u64 last, umem_call_back cb, + bool blockable, void *cookie) { int ret_val = 0; @@ -809,6 +825,9 @@ int rbt_ib_umem_for_each_in_range(struct rb_root_cached *root, for (node = rbt_ib_umem_iter_first(root, start, last - 1); node; node = next) { + /* TODO move the blockable decision up to the callback */ + if (!blockable) + return -EAGAIN; next = rbt_ib_umem_iter_next(node, start, last - 1); umem = container_of(node, struct ib_umem_odp, interval_tree); ret_val = cb(umem->umem, start, last, cookie) || ret_val; diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index 70aceefe14d5..e1c7996c018e 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -67,9 +67,9 @@ struct mmu_rb_handler { static unsigned long mmu_node_start(struct mmu_rb_node *); static unsigned long mmu_node_last(struct mmu_rb_node *); -static void mmu_notifier_range_start(struct mmu_notifier *, +static int mmu_notifier_range_start(struct mmu_notifier *, struct mm_struct *, - unsigned long, unsigned long); + unsigned long, unsigned long, bool); static struct mmu_rb_node *__mmu_rb_search(struct mmu_rb_handler *, unsigned long, unsigned long); static void do_remove(struct mmu_rb_handler *handler, @@ -284,10 +284,11 @@ void hfi1_mmu_rb_remove(struct mmu_rb_handler *handler, handler->ops->remove(handler->ops_arg, node); } -static void mmu_notifier_range_start(struct mmu_notifier *mn, +static int mmu_notifier_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct mmu_rb_handler *handler = container_of(mn, struct mmu_rb_handler, mn); @@ -313,6 +314,8 @@ static void mmu_notifier_range_start(struct mmu_notifier *mn, if (added) queue_work(handler->wq, &handler->del_work); + + return 0; } /* diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c index f1a87a690a4c..d216e0d2921d 100644 --- a/drivers/infiniband/hw/mlx5/odp.c +++ b/drivers/infiniband/hw/mlx5/odp.c @@ -488,7 +488,7 @@ void mlx5_ib_free_implicit_mr(struct mlx5_ib_mr *imr) down_read(&ctx->umem_rwsem); rbt_ib_umem_for_each_in_range(&ctx->umem_tree, 0, ULLONG_MAX, - mr_leaf_free, imr); + mr_leaf_free, true, imr); up_read(&ctx->umem_rwsem); wait_event(imr->q_leaf_free, !atomic_read(&imr->num_leaf_free)); diff --git a/drivers/misc/mic/scif/scif_dma.c b/drivers/misc/mic/scif/scif_dma.c index 63d6246d6dff..6369aeaa7056 100644 --- a/drivers/misc/mic/scif/scif_dma.c +++ b/drivers/misc/mic/scif/scif_dma.c @@ -200,15 +200,18 @@ static void scif_mmu_notifier_release(struct mmu_notifier *mn, schedule_work(&scif_info.misc_work); } -static void scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct scif_mmu_notif *mmn; mmn = container_of(mn, struct scif_mmu_notif, ep_mmu_notifier); scif_rma_destroy_tcw(mmn, start, end - start); + + return 0; } static void scif_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/misc/sgi-gru/grutlbpurge.c b/drivers/misc/sgi-gru/grutlbpurge.c index a3454eb56fbf..be28f05bfafa 100644 --- a/drivers/misc/sgi-gru/grutlbpurge.c +++ b/drivers/misc/sgi-gru/grutlbpurge.c @@ -219,9 +219,10 @@ void gru_flush_all_tlb(struct gru_state *gru) /* * MMUOPS notifier callout functions */ -static void gru_invalidate_range_start(struct mmu_notifier *mn, +static int gru_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gru_mm_struct *gms = container_of(mn, struct gru_mm_struct, ms_notifier); @@ -231,6 +232,8 @@ static void gru_invalidate_range_start(struct mmu_notifier *mn, gru_dbg(grudev, "gms %p, start 0x%lx, end 0x%lx, act %d\n", gms, start, end, atomic_read(&gms->ms_range_active)); gru_flush_tlb_range(gms, start, end - start); + + return 0; } static void gru_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index bd56653b9bbc..55b4f0e3f4d6 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -441,18 +441,25 @@ static const struct vm_operations_struct gntdev_vmops = { /* ------------------------------------------------------------------ */ +static bool in_range(struct grant_map *map, + unsigned long start, unsigned long end) +{ + if (!map->vma) + return false; + if (map->vma->vm_start >= end) + return false; + if (map->vma->vm_end <= start) + return false; + + return true; +} + static void unmap_if_in_range(struct grant_map *map, unsigned long start, unsigned long end) { unsigned long mstart, mend; int err; - if (!map->vma) - return; - if (map->vma->vm_start >= end) - return; - if (map->vma->vm_end <= start) - return; mstart = max(start, map->vma->vm_start); mend = min(end, map->vma->vm_end); pr_debug("map %d+%d (%lx %lx), range %lx %lx, mrange %lx %lx\n", @@ -465,21 +472,40 @@ static void unmap_if_in_range(struct grant_map *map, WARN_ON(err); } -static void mn_invl_range_start(struct mmu_notifier *mn, +static int mn_invl_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gntdev_priv *priv = container_of(mn, struct gntdev_priv, mn); struct grant_map *map; + int ret = 0; + + /* TODO do we really need a mutex here? */ + if (blockable) + mutex_lock(&priv->lock); + else if (!mutex_trylock(&priv->lock)) + return -EAGAIN; - mutex_lock(&priv->lock); list_for_each_entry(map, &priv->maps, next) { + if (in_range(map, start, end)) { + ret = -EAGAIN; + goto out_unlock; + } unmap_if_in_range(map, start, end); } list_for_each_entry(map, &priv->freeable_maps, next) { + if (in_range(map, start, end)) { + ret = -EAGAIN; + goto out_unlock; + } unmap_if_in_range(map, start, end); } + +out_unlock: mutex_unlock(&priv->lock); + + return ret; } static void mn_release(struct mmu_notifier *mn, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4ee7bc548a83..148935085194 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1275,8 +1275,8 @@ static inline long kvm_arch_vcpu_async_ioctl(struct file *filp, } #endif /* CONFIG_HAVE_KVM_VCPU_ASYNC_IOCTL */ -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end); +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, bool blockable); #ifdef CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu); diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 392e6af82701..2eb1a2d01759 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -151,13 +151,15 @@ struct mmu_notifier_ops { * address space but may still be referenced by sptes until * the last refcount is dropped. * - * If both of these callbacks cannot block, and invalidate_range - * cannot block, mmu_notifier_ops.flags should have - * MMU_INVALIDATE_DOES_NOT_BLOCK set. + * If blockable argument is set to false then the callback cannot + * sleep and has to return with -EAGAIN. 0 should be returned + * otherwise. + * */ - void (*invalidate_range_start)(struct mmu_notifier *mn, + int (*invalidate_range_start)(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end); + unsigned long start, unsigned long end, + bool blockable); void (*invalidate_range_end)(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, unsigned long end); @@ -229,8 +231,9 @@ extern int __mmu_notifier_test_young(struct mm_struct *mm, unsigned long address); extern void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, pte_t pte); -extern void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end); +extern int __mmu_notifier_invalidate_range_start(struct mm_struct *mm, + unsigned long start, unsigned long end, + bool blockable); extern void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, unsigned long start, unsigned long end, bool only_end); @@ -281,7 +284,17 @@ static inline void mmu_notifier_invalidate_range_start(struct mm_struct *mm, unsigned long start, unsigned long end) { if (mm_has_notifiers(mm)) - __mmu_notifier_invalidate_range_start(mm, start, end); + __mmu_notifier_invalidate_range_start(mm, start, end, true); +} + +static inline int mmu_notifier_invalidate_range_start_nonblock(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + int ret = 0; + if (mm_has_notifiers(mm)) + ret = __mmu_notifier_invalidate_range_start(mm, start, end, false); + + return ret; } static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, @@ -461,6 +474,12 @@ static inline void mmu_notifier_invalidate_range_start(struct mm_struct *mm, { } +static inline int mmu_notifier_invalidate_range_start_nonblock(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + return 0; +} + static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, unsigned long start, unsigned long end) { diff --git a/include/linux/oom.h b/include/linux/oom.h index 6adac113e96d..92f70e4c6252 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -95,7 +95,7 @@ static inline int check_stable_address_space(struct mm_struct *mm) return 0; } -void __oom_reap_task_mm(struct mm_struct *mm); +bool __oom_reap_task_mm(struct mm_struct *mm); extern unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg, const nodemask_t *nodemask, diff --git a/include/rdma/ib_umem_odp.h b/include/rdma/ib_umem_odp.h index 6a17f856f841..381cdf5a9bd1 100644 --- a/include/rdma/ib_umem_odp.h +++ b/include/rdma/ib_umem_odp.h @@ -119,7 +119,8 @@ typedef int (*umem_call_back)(struct ib_umem *item, u64 start, u64 end, */ int rbt_ib_umem_for_each_in_range(struct rb_root_cached *root, u64 start, u64 end, - umem_call_back cb, void *cookie); + umem_call_back cb, + bool blockable, void *cookie); /* * Find first region intersecting with address range. diff --git a/mm/hmm.c b/mm/hmm.c index de7b6bf77201..81fd57bd2634 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -177,16 +177,19 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) up_write(&hmm->mirrors_sem); } -static void hmm_invalidate_range_start(struct mmu_notifier *mn, +static int hmm_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); atomic_inc(&hmm->sequence); + + return 0; } static void hmm_invalidate_range_end(struct mmu_notifier *mn, diff --git a/mm/mmap.c b/mm/mmap.c index d1eb87ef4b1a..336bee8c4e25 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3074,7 +3074,7 @@ void exit_mmap(struct mm_struct *mm) * reliably test it. */ mutex_lock(&oom_lock); - __oom_reap_task_mm(mm); + (void)__oom_reap_task_mm(mm); mutex_unlock(&oom_lock); set_bit(MMF_OOM_SKIP, &mm->flags); diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index eff6b88a993f..103b2b450043 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -174,18 +174,29 @@ void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, srcu_read_unlock(&srcu, id); } -void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end) +int __mmu_notifier_invalidate_range_start(struct mm_struct *mm, + unsigned long start, unsigned long end, + bool blockable) { struct mmu_notifier *mn; + int ret = 0; int id; id = srcu_read_lock(&srcu); hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) { - if (mn->ops->invalidate_range_start) - mn->ops->invalidate_range_start(mn, mm, start, end); + if (mn->ops->invalidate_range_start) { + int _ret = mn->ops->invalidate_range_start(mn, mm, start, end, blockable); + if (_ret) { + pr_info("%pS callback failed with %d in %sblockable context.\n", + mn->ops->invalidate_range_start, _ret, + !blockable ? "non-": ""); + ret = _ret; + } + } } srcu_read_unlock(&srcu, id); + + return ret; } EXPORT_SYMBOL_GPL(__mmu_notifier_invalidate_range_start); diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 84081e77bc51..5a936cf24d79 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -479,9 +479,10 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); static struct task_struct *oom_reaper_list; static DEFINE_SPINLOCK(oom_reaper_lock); -void __oom_reap_task_mm(struct mm_struct *mm) +bool __oom_reap_task_mm(struct mm_struct *mm) { struct vm_area_struct *vma; + bool ret = true; /* * Tell all users of get_user/copy_from_user etc... that the content @@ -511,12 +512,17 @@ void __oom_reap_task_mm(struct mm_struct *mm) struct mmu_gather tlb; tlb_gather_mmu(&tlb, mm, start, end); - mmu_notifier_invalidate_range_start(mm, start, end); + if (mmu_notifier_invalidate_range_start_nonblock(mm, start, end)) { + ret = false; + continue; + } unmap_page_range(&tlb, vma, start, end, NULL); mmu_notifier_invalidate_range_end(mm, start, end); tlb_finish_mmu(&tlb, start, end); } } + + return ret; } static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) @@ -545,18 +551,6 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) goto unlock_oom; } - /* - * If the mm has invalidate_{start,end}() notifiers that could block, - * sleep to give the oom victim some more time. - * TODO: we really want to get rid of this ugly hack and make sure that - * notifiers cannot block for unbounded amount of time - */ - if (mm_has_blockable_invalidate_notifiers(mm)) { - up_read(&mm->mmap_sem); - schedule_timeout_idle(HZ); - goto unlock_oom; - } - /* * MMF_OOM_SKIP is set by exit_mmap when the OOM reaper can't * work on the mm anymore. The check for MMF_OOM_SKIP must run @@ -571,7 +565,12 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) trace_start_task_reaping(tsk->pid); - __oom_reap_task_mm(mm); + /* failed to reap part of the address space. Try again later */ + if (!__oom_reap_task_mm(mm)) { + up_read(&mm->mmap_sem); + ret = false; + goto unlock_oom; + } pr_info("oom_reaper: reaped process %d (%s), now anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n", task_pid_nr(tsk), tsk->comm, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ada21f47f22b..16ce38f178d1 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -135,9 +135,10 @@ static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm); static unsigned long long kvm_createvm_count; static unsigned long long kvm_active_vms; -__weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) +__weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, bool blockable) { + return 0; } bool kvm_is_reserved_pfn(kvm_pfn_t pfn) @@ -354,13 +355,15 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn, srcu_read_unlock(&kvm->srcu, idx); } -static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct kvm *kvm = mmu_notifier_to_kvm(mn); int need_tlb_flush = 0, idx; + int ret; idx = srcu_read_lock(&kvm->srcu); spin_lock(&kvm->mmu_lock); @@ -378,9 +381,11 @@ static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, spin_unlock(&kvm->mmu_lock); - kvm_arch_mmu_notifier_invalidate_range(kvm, start, end); + ret = kvm_arch_mmu_notifier_invalidate_range(kvm, start, end, blockable); srcu_read_unlock(&kvm->srcu, idx); + + return ret; } static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, -- 2.18.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=3.0 tests=MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED53DECDFAA for ; Mon, 16 Jul 2018 11:51:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7C47920BED for ; Mon, 16 Jul 2018 11:51:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C47920BED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730092AbeGPMST (ORCPT ); Mon, 16 Jul 2018 08:18:19 -0400 Received: from mail-ed1-f67.google.com ([209.85.208.67]:43087 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727857AbeGPMST (ORCPT ); Mon, 16 Jul 2018 08:18:19 -0400 Received: by mail-ed1-f67.google.com with SMTP id b20-v6so4191707edt.10; Mon, 16 Jul 2018 04:51:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=yZQYW96hcBYgtIT4wvLccZa8Gw+DTsaiuUJjpwYTQLg=; b=K5P56FIrskxf3lIW4GZxm5x27QIC1kX+DRkdPjhpH3cVgGeH2pWshwDklBn/Fw1Gqz 6p2rUe01AXxtgThgw84q+z4Iyk8sBNTMgWMMa8Q1SnOtAuWSv1a2HcV/zVY735gp7G9J RNtjbROqdwk3ms2OCsCcO12+F9eSs7/meOZYQZIw/1GUEtn7DlKfk2zY5Mstxy8EK4+o E2w61d3AjPURJGZv6F4gCx5gs30pG3DhD4r/iACbVcmkWtwEzTGYoFTEVUY0mItkVfNs n1Gpm4pep44mKKLJsY4ZW+xC3pqJSd0WiZ8PtkmqZW4cjj4d11SDjV/ibFkXY73orWWH IY6g== X-Gm-Message-State: AOUpUlFXuAezizqS+uwfP+jSIhT1ed+uipxMGBoldgvj3cZA1ajEZL9O 3d+GVuPww7qv7F0uLv0ueecLLg== X-Google-Smtp-Source: AAOMgpfuejQzQ5+yFn50SrSqc5X/muYzI/eZjwyWE/KoSGpsUfjPuFf8l/IIQWcuvrAfFV9p6b/JKg== X-Received: by 2002:aa7:c0c9:: with SMTP id j9-v6mr17213574edp.135.1531741869717; Mon, 16 Jul 2018 04:51:09 -0700 (PDT) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id a5-v6sm6470219edr.1.2018.07.16.04.51.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Jul 2018 04:51:09 -0700 (PDT) From: Michal Hocko To: Andrew Morton Cc: LKML , , Michal Hocko , "David (ChunMing) Zhou" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Alex Deucher , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Doug Ledford , Jason Gunthorpe , Mike Marciniszyn , Dennis Dalessandro , Sudeep Dutt , Ashutosh Dixit , Dimitri Sivanich , Boris Ostrovsky , Juergen Gross , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrea Arcangeli , Felix Kuehling , kvm@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-rdma@vger.kernel.org, xen-devel@lists.xenproject.org, =?UTF-8?q?Christian=20K=C3=B6nig?= , David Rientjes , Leon Romanovsky Subject: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Mon, 16 Jul 2018 13:50:58 +0200 Message-Id: <20180716115058.5559-1-mhocko@kernel.org> X-Mailer: git-send-email 2.18.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Michal Hocko There are several blockable mmu notifiers which might sleep in mmu_notifier_invalidate_range_start and that is a problem for the oom_reaper because it needs to guarantee a forward progress so it cannot depend on any sleepable locks. Currently we simply back off and mark an oom victim with blockable mmu notifiers as done after a short sleep. That can result in selecting a new oom victim prematurely because the previous one still hasn't torn its memory down yet. We can do much better though. Even if mmu notifiers use sleepable locks there is no reason to automatically assume those locks are held. Moreover majority of notifiers only care about a portion of the address space and there is absolutely zero reason to fail when we are unmapping an unrelated range. Many notifiers do really block and wait for HW which is harder to handle and we have to bail out though. This patch handles the low hanging fruid. __mmu_notifier_invalidate_range_start gets a blockable flag and callbacks are not allowed to sleep if the flag is set to false. This is achieved by using trylock instead of the sleepable lock for most callbacks and continue as long as we do not block down the call chain. I think we can improve that even further because there is a common pattern to do a range lookup first and then do something about that. The first part can be done without a sleeping lock in most cases AFAICS. The oom_reaper end then simply retries if there is at least one notifier which couldn't make any progress in !blockable mode. A retry loop is already implemented to wait for the mmap_sem and this is basically the same thing. Changes since rfc v1 - gpu notifiers can sleep while waiting for HW (evict_process_queues_cpsch on a lock and amdgpu_mn_invalidate_node on unbound timeout) make sure we bail out when we have an intersecting range for starter - note that a notifier failed to the log for easier debugging - back off early in ib_umem_notifier_invalidate_range_start if the callback is called - mn_invl_range_start waits for completion down the unmap_grant_pages path so we have to back off early on overlapping ranges Cc: "David (ChunMing) Zhou" Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Alex Deucher Cc: David Airlie Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Rodrigo Vivi Cc: Doug Ledford Cc: Jason Gunthorpe Cc: Mike Marciniszyn Cc: Dennis Dalessandro Cc: Sudeep Dutt Cc: Ashutosh Dixit Cc: Dimitri Sivanich Cc: Boris Ostrovsky Cc: Juergen Gross Cc: "Jérôme Glisse" Cc: Andrea Arcangeli Cc: Felix Kuehling Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: amd-gfx@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org Cc: intel-gfx@lists.freedesktop.org Cc: linux-rdma@vger.kernel.org Cc: xen-devel@lists.xenproject.org Cc: linux-mm@kvack.org Acked-by: Christian König # AMD notifiers Acked-by: Leon Romanovsky # mlx and umem_odp Reported-by: David Rientjes Signed-off-by: Michal Hocko --- Hi, there were no major objections when I sent this as an RFC the last time [1]. I was hoping for more feedback in the drivers land because I am touching the code I have no way to test. On the other hand the pattern is quite simple and consistent over all users so there shouldn't be any large surprises hopefully. Any further review would be highly appreciate of course. But is this something to put into the mm tree now? [1] http://lkml.kernel.org/r/20180627074421.GF32348@dhcp22.suse.cz arch/x86/kvm/x86.c | 7 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 43 +++++++++++++++++++----- drivers/gpu/drm/i915/i915_gem_userptr.c | 13 ++++++-- drivers/gpu/drm/radeon/radeon_mn.c | 22 +++++++++++-- drivers/infiniband/core/umem_odp.c | 33 +++++++++++++++---- drivers/infiniband/hw/hfi1/mmu_rb.c | 11 ++++--- drivers/infiniband/hw/mlx5/odp.c | 2 +- drivers/misc/mic/scif/scif_dma.c | 7 ++-- drivers/misc/sgi-gru/grutlbpurge.c | 7 ++-- drivers/xen/gntdev.c | 44 ++++++++++++++++++++----- include/linux/kvm_host.h | 4 +-- include/linux/mmu_notifier.h | 35 +++++++++++++++----- include/linux/oom.h | 2 +- include/rdma/ib_umem_odp.h | 3 +- mm/hmm.c | 7 ++-- mm/mmap.c | 2 +- mm/mmu_notifier.c | 19 ++++++++--- mm/oom_kill.c | 29 ++++++++-------- virt/kvm/kvm_main.c | 15 ++++++--- 19 files changed, 225 insertions(+), 80 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6bcecc325e7e..ac08f5d711be 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7203,8 +7203,9 @@ static void vcpu_load_eoi_exitmap(struct kvm_vcpu *vcpu) kvm_x86_ops->load_eoi_exitmap(vcpu, eoi_exit_bitmap); } -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, + bool blockable) { unsigned long apic_address; @@ -7215,6 +7216,8 @@ void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, apic_address = gfn_to_hva(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); if (start <= apic_address && apic_address < end) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); + + return 0; } void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c index 83e344fbb50a..3399a4a927fb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c @@ -136,12 +136,18 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) * * Take the rmn read side lock. */ -static void amdgpu_mn_read_lock(struct amdgpu_mn *rmn) +static int amdgpu_mn_read_lock(struct amdgpu_mn *rmn, bool blockable) { - mutex_lock(&rmn->read_lock); + if (blockable) + mutex_lock(&rmn->read_lock); + else if (!mutex_trylock(&rmn->read_lock)) + return -EAGAIN; + if (atomic_inc_return(&rmn->recursion) == 1) down_read_non_owner(&rmn->lock); mutex_unlock(&rmn->read_lock); + + return 0; } /** @@ -197,10 +203,11 @@ static void amdgpu_mn_invalidate_node(struct amdgpu_mn_node *node, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -208,17 +215,28 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + /* TODO we should be able to split locking for interval tree and + * amdgpu_mn_invalidate_node + */ + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { struct amdgpu_mn_node *node; + if (!blockable) { + amdgpu_mn_read_unlock(rmn); + return -EAGAIN; + } + node = container_of(it, struct amdgpu_mn_node, it); it = interval_tree_iter_next(it, start, end); amdgpu_mn_invalidate_node(node, start, end); } + + return 0; } /** @@ -233,10 +251,11 @@ static void amdgpu_mn_invalidate_range_start_gfx(struct mmu_notifier *mn, * necessitates evicting all user-mode queues of the process. The BOs * are restorted in amdgpu_mn_invalidate_range_end_hsa. */ -static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, +static int amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct amdgpu_mn *rmn = container_of(mn, struct amdgpu_mn, mn); struct interval_tree_node *it; @@ -244,13 +263,19 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, /* notification is exclusive, but interval is inclusive */ end -= 1; - amdgpu_mn_read_lock(rmn); + if (amdgpu_mn_read_lock(rmn, blockable)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { struct amdgpu_mn_node *node; struct amdgpu_bo *bo; + if (!blockable) { + amdgpu_mn_read_unlock(rmn); + return -EAGAIN; + } + node = container_of(it, struct amdgpu_mn_node, it); it = interval_tree_iter_next(it, start, end); @@ -262,6 +287,8 @@ static void amdgpu_mn_invalidate_range_start_hsa(struct mmu_notifier *mn, amdgpu_amdkfd_evict_userptr(mem, mm); } } + + return 0; } /** diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c index 854bd51b9478..9cbff68f6b41 100644 --- a/drivers/gpu/drm/i915/i915_gem_userptr.c +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c @@ -112,10 +112,11 @@ static void del_object(struct i915_mmu_object *mo) mo->attached = false; } -static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, +static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct i915_mmu_notifier *mn = container_of(_mn, struct i915_mmu_notifier, mn); @@ -124,7 +125,7 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, LIST_HEAD(cancelled); if (RB_EMPTY_ROOT(&mn->objects.rb_root)) - return; + return 0; /* interval ranges are inclusive, but invalidate range is exclusive */ end--; @@ -132,6 +133,10 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, spin_lock(&mn->lock); it = interval_tree_iter_first(&mn->objects, start, end); while (it) { + if (!blockable) { + spin_unlock(&mn->lock); + return -EAGAIN; + } /* The mmu_object is released late when destroying the * GEM object so it is entirely possible to gain a * reference on an object in the process of being freed @@ -154,6 +159,8 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, if (!list_empty(&cancelled)) flush_workqueue(mn->wq); + + return 0; } static const struct mmu_notifier_ops i915_gem_userptr_notifier = { diff --git a/drivers/gpu/drm/radeon/radeon_mn.c b/drivers/gpu/drm/radeon/radeon_mn.c index abd24975c9b1..f8b35df44c60 100644 --- a/drivers/gpu/drm/radeon/radeon_mn.c +++ b/drivers/gpu/drm/radeon/radeon_mn.c @@ -118,19 +118,27 @@ static void radeon_mn_release(struct mmu_notifier *mn, * We block for all BOs between start and end to be idle and * unmap them by move them into system domain again. */ -static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, +static int radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct radeon_mn *rmn = container_of(mn, struct radeon_mn, mn); struct ttm_operation_ctx ctx = { false, false }; struct interval_tree_node *it; + int ret = 0; /* notification is exclusive, but interval is inclusive */ end -= 1; - mutex_lock(&rmn->lock); + /* TODO we should be able to split locking for interval tree and + * the tear down. + */ + if (blockable) + mutex_lock(&rmn->lock); + else if (!mutex_trylock(&rmn->lock)) + return -EAGAIN; it = interval_tree_iter_first(&rmn->objects, start, end); while (it) { @@ -138,6 +146,11 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, struct radeon_bo *bo; long r; + if (!blockable) { + ret = -EAGAIN; + goto out_unlock; + } + node = container_of(it, struct radeon_mn_node, it); it = interval_tree_iter_next(it, start, end); @@ -166,7 +179,10 @@ static void radeon_mn_invalidate_range_start(struct mmu_notifier *mn, } } +out_unlock: mutex_unlock(&rmn->lock); + + return ret; } static const struct mmu_notifier_ops radeon_mn_ops = { diff --git a/drivers/infiniband/core/umem_odp.c b/drivers/infiniband/core/umem_odp.c index 182436b92ba9..6ec748eccff7 100644 --- a/drivers/infiniband/core/umem_odp.c +++ b/drivers/infiniband/core/umem_odp.c @@ -186,6 +186,7 @@ static void ib_umem_notifier_release(struct mmu_notifier *mn, rbt_ib_umem_for_each_in_range(&context->umem_tree, 0, ULLONG_MAX, ib_umem_notifier_release_trampoline, + true, NULL); up_read(&context->umem_rwsem); } @@ -207,22 +208,31 @@ static int invalidate_range_start_trampoline(struct ib_umem *item, u64 start, return 0; } -static void ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int ib_umem_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct ib_ucontext *context = container_of(mn, struct ib_ucontext, mn); + int ret; if (!context->invalidate_range) - return; + return 0; + + if (blockable) + down_read(&context->umem_rwsem); + else if (!down_read_trylock(&context->umem_rwsem)) + return -EAGAIN; ib_ucontext_notifier_start_account(context); - down_read(&context->umem_rwsem); - rbt_ib_umem_for_each_in_range(&context->umem_tree, start, + ret = rbt_ib_umem_for_each_in_range(&context->umem_tree, start, end, - invalidate_range_start_trampoline, NULL); + invalidate_range_start_trampoline, + blockable, NULL); up_read(&context->umem_rwsem); + + return ret; } static int invalidate_range_end_trampoline(struct ib_umem *item, u64 start, @@ -242,10 +252,15 @@ static void ib_umem_notifier_invalidate_range_end(struct mmu_notifier *mn, if (!context->invalidate_range) return; + /* + * TODO: we currently bail out if there is any sleepable work to be done + * in ib_umem_notifier_invalidate_range_start so we shouldn't really block + * here. But this is ugly and fragile. + */ down_read(&context->umem_rwsem); rbt_ib_umem_for_each_in_range(&context->umem_tree, start, end, - invalidate_range_end_trampoline, NULL); + invalidate_range_end_trampoline, true, NULL); up_read(&context->umem_rwsem); ib_ucontext_notifier_end_account(context); } @@ -798,6 +813,7 @@ EXPORT_SYMBOL(ib_umem_odp_unmap_dma_pages); int rbt_ib_umem_for_each_in_range(struct rb_root_cached *root, u64 start, u64 last, umem_call_back cb, + bool blockable, void *cookie) { int ret_val = 0; @@ -809,6 +825,9 @@ int rbt_ib_umem_for_each_in_range(struct rb_root_cached *root, for (node = rbt_ib_umem_iter_first(root, start, last - 1); node; node = next) { + /* TODO move the blockable decision up to the callback */ + if (!blockable) + return -EAGAIN; next = rbt_ib_umem_iter_next(node, start, last - 1); umem = container_of(node, struct ib_umem_odp, interval_tree); ret_val = cb(umem->umem, start, last, cookie) || ret_val; diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index 70aceefe14d5..e1c7996c018e 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -67,9 +67,9 @@ struct mmu_rb_handler { static unsigned long mmu_node_start(struct mmu_rb_node *); static unsigned long mmu_node_last(struct mmu_rb_node *); -static void mmu_notifier_range_start(struct mmu_notifier *, +static int mmu_notifier_range_start(struct mmu_notifier *, struct mm_struct *, - unsigned long, unsigned long); + unsigned long, unsigned long, bool); static struct mmu_rb_node *__mmu_rb_search(struct mmu_rb_handler *, unsigned long, unsigned long); static void do_remove(struct mmu_rb_handler *handler, @@ -284,10 +284,11 @@ void hfi1_mmu_rb_remove(struct mmu_rb_handler *handler, handler->ops->remove(handler->ops_arg, node); } -static void mmu_notifier_range_start(struct mmu_notifier *mn, +static int mmu_notifier_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct mmu_rb_handler *handler = container_of(mn, struct mmu_rb_handler, mn); @@ -313,6 +314,8 @@ static void mmu_notifier_range_start(struct mmu_notifier *mn, if (added) queue_work(handler->wq, &handler->del_work); + + return 0; } /* diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c index f1a87a690a4c..d216e0d2921d 100644 --- a/drivers/infiniband/hw/mlx5/odp.c +++ b/drivers/infiniband/hw/mlx5/odp.c @@ -488,7 +488,7 @@ void mlx5_ib_free_implicit_mr(struct mlx5_ib_mr *imr) down_read(&ctx->umem_rwsem); rbt_ib_umem_for_each_in_range(&ctx->umem_tree, 0, ULLONG_MAX, - mr_leaf_free, imr); + mr_leaf_free, true, imr); up_read(&ctx->umem_rwsem); wait_event(imr->q_leaf_free, !atomic_read(&imr->num_leaf_free)); diff --git a/drivers/misc/mic/scif/scif_dma.c b/drivers/misc/mic/scif/scif_dma.c index 63d6246d6dff..6369aeaa7056 100644 --- a/drivers/misc/mic/scif/scif_dma.c +++ b/drivers/misc/mic/scif/scif_dma.c @@ -200,15 +200,18 @@ static void scif_mmu_notifier_release(struct mmu_notifier *mn, schedule_work(&scif_info.misc_work); } -static void scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int scif_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct scif_mmu_notif *mmn; mmn = container_of(mn, struct scif_mmu_notif, ep_mmu_notifier); scif_rma_destroy_tcw(mmn, start, end - start); + + return 0; } static void scif_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/misc/sgi-gru/grutlbpurge.c b/drivers/misc/sgi-gru/grutlbpurge.c index a3454eb56fbf..be28f05bfafa 100644 --- a/drivers/misc/sgi-gru/grutlbpurge.c +++ b/drivers/misc/sgi-gru/grutlbpurge.c @@ -219,9 +219,10 @@ void gru_flush_all_tlb(struct gru_state *gru) /* * MMUOPS notifier callout functions */ -static void gru_invalidate_range_start(struct mmu_notifier *mn, +static int gru_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gru_mm_struct *gms = container_of(mn, struct gru_mm_struct, ms_notifier); @@ -231,6 +232,8 @@ static void gru_invalidate_range_start(struct mmu_notifier *mn, gru_dbg(grudev, "gms %p, start 0x%lx, end 0x%lx, act %d\n", gms, start, end, atomic_read(&gms->ms_range_active)); gru_flush_tlb_range(gms, start, end - start); + + return 0; } static void gru_invalidate_range_end(struct mmu_notifier *mn, diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index bd56653b9bbc..55b4f0e3f4d6 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -441,18 +441,25 @@ static const struct vm_operations_struct gntdev_vmops = { /* ------------------------------------------------------------------ */ +static bool in_range(struct grant_map *map, + unsigned long start, unsigned long end) +{ + if (!map->vma) + return false; + if (map->vma->vm_start >= end) + return false; + if (map->vma->vm_end <= start) + return false; + + return true; +} + static void unmap_if_in_range(struct grant_map *map, unsigned long start, unsigned long end) { unsigned long mstart, mend; int err; - if (!map->vma) - return; - if (map->vma->vm_start >= end) - return; - if (map->vma->vm_end <= start) - return; mstart = max(start, map->vma->vm_start); mend = min(end, map->vma->vm_end); pr_debug("map %d+%d (%lx %lx), range %lx %lx, mrange %lx %lx\n", @@ -465,21 +472,40 @@ static void unmap_if_in_range(struct grant_map *map, WARN_ON(err); } -static void mn_invl_range_start(struct mmu_notifier *mn, +static int mn_invl_range_start(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end) + unsigned long start, unsigned long end, + bool blockable) { struct gntdev_priv *priv = container_of(mn, struct gntdev_priv, mn); struct grant_map *map; + int ret = 0; + + /* TODO do we really need a mutex here? */ + if (blockable) + mutex_lock(&priv->lock); + else if (!mutex_trylock(&priv->lock)) + return -EAGAIN; - mutex_lock(&priv->lock); list_for_each_entry(map, &priv->maps, next) { + if (in_range(map, start, end)) { + ret = -EAGAIN; + goto out_unlock; + } unmap_if_in_range(map, start, end); } list_for_each_entry(map, &priv->freeable_maps, next) { + if (in_range(map, start, end)) { + ret = -EAGAIN; + goto out_unlock; + } unmap_if_in_range(map, start, end); } + +out_unlock: mutex_unlock(&priv->lock); + + return ret; } static void mn_release(struct mmu_notifier *mn, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4ee7bc548a83..148935085194 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1275,8 +1275,8 @@ static inline long kvm_arch_vcpu_async_ioctl(struct file *filp, } #endif /* CONFIG_HAVE_KVM_VCPU_ASYNC_IOCTL */ -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end); +int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, bool blockable); #ifdef CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu); diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 392e6af82701..2eb1a2d01759 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -151,13 +151,15 @@ struct mmu_notifier_ops { * address space but may still be referenced by sptes until * the last refcount is dropped. * - * If both of these callbacks cannot block, and invalidate_range - * cannot block, mmu_notifier_ops.flags should have - * MMU_INVALIDATE_DOES_NOT_BLOCK set. + * If blockable argument is set to false then the callback cannot + * sleep and has to return with -EAGAIN. 0 should be returned + * otherwise. + * */ - void (*invalidate_range_start)(struct mmu_notifier *mn, + int (*invalidate_range_start)(struct mmu_notifier *mn, struct mm_struct *mm, - unsigned long start, unsigned long end); + unsigned long start, unsigned long end, + bool blockable); void (*invalidate_range_end)(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, unsigned long end); @@ -229,8 +231,9 @@ extern int __mmu_notifier_test_young(struct mm_struct *mm, unsigned long address); extern void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, pte_t pte); -extern void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end); +extern int __mmu_notifier_invalidate_range_start(struct mm_struct *mm, + unsigned long start, unsigned long end, + bool blockable); extern void __mmu_notifier_invalidate_range_end(struct mm_struct *mm, unsigned long start, unsigned long end, bool only_end); @@ -281,7 +284,17 @@ static inline void mmu_notifier_invalidate_range_start(struct mm_struct *mm, unsigned long start, unsigned long end) { if (mm_has_notifiers(mm)) - __mmu_notifier_invalidate_range_start(mm, start, end); + __mmu_notifier_invalidate_range_start(mm, start, end, true); +} + +static inline int mmu_notifier_invalidate_range_start_nonblock(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + int ret = 0; + if (mm_has_notifiers(mm)) + ret = __mmu_notifier_invalidate_range_start(mm, start, end, false); + + return ret; } static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, @@ -461,6 +474,12 @@ static inline void mmu_notifier_invalidate_range_start(struct mm_struct *mm, { } +static inline int mmu_notifier_invalidate_range_start_nonblock(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + return 0; +} + static inline void mmu_notifier_invalidate_range_end(struct mm_struct *mm, unsigned long start, unsigned long end) { diff --git a/include/linux/oom.h b/include/linux/oom.h index 6adac113e96d..92f70e4c6252 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -95,7 +95,7 @@ static inline int check_stable_address_space(struct mm_struct *mm) return 0; } -void __oom_reap_task_mm(struct mm_struct *mm); +bool __oom_reap_task_mm(struct mm_struct *mm); extern unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg, const nodemask_t *nodemask, diff --git a/include/rdma/ib_umem_odp.h b/include/rdma/ib_umem_odp.h index 6a17f856f841..381cdf5a9bd1 100644 --- a/include/rdma/ib_umem_odp.h +++ b/include/rdma/ib_umem_odp.h @@ -119,7 +119,8 @@ typedef int (*umem_call_back)(struct ib_umem *item, u64 start, u64 end, */ int rbt_ib_umem_for_each_in_range(struct rb_root_cached *root, u64 start, u64 end, - umem_call_back cb, void *cookie); + umem_call_back cb, + bool blockable, void *cookie); /* * Find first region intersecting with address range. diff --git a/mm/hmm.c b/mm/hmm.c index de7b6bf77201..81fd57bd2634 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -177,16 +177,19 @@ static void hmm_release(struct mmu_notifier *mn, struct mm_struct *mm) up_write(&hmm->mirrors_sem); } -static void hmm_invalidate_range_start(struct mmu_notifier *mn, +static int hmm_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct hmm *hmm = mm->hmm; VM_BUG_ON(!hmm); atomic_inc(&hmm->sequence); + + return 0; } static void hmm_invalidate_range_end(struct mmu_notifier *mn, diff --git a/mm/mmap.c b/mm/mmap.c index d1eb87ef4b1a..336bee8c4e25 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3074,7 +3074,7 @@ void exit_mmap(struct mm_struct *mm) * reliably test it. */ mutex_lock(&oom_lock); - __oom_reap_task_mm(mm); + (void)__oom_reap_task_mm(mm); mutex_unlock(&oom_lock); set_bit(MMF_OOM_SKIP, &mm->flags); diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index eff6b88a993f..103b2b450043 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -174,18 +174,29 @@ void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, srcu_read_unlock(&srcu, id); } -void __mmu_notifier_invalidate_range_start(struct mm_struct *mm, - unsigned long start, unsigned long end) +int __mmu_notifier_invalidate_range_start(struct mm_struct *mm, + unsigned long start, unsigned long end, + bool blockable) { struct mmu_notifier *mn; + int ret = 0; int id; id = srcu_read_lock(&srcu); hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) { - if (mn->ops->invalidate_range_start) - mn->ops->invalidate_range_start(mn, mm, start, end); + if (mn->ops->invalidate_range_start) { + int _ret = mn->ops->invalidate_range_start(mn, mm, start, end, blockable); + if (_ret) { + pr_info("%pS callback failed with %d in %sblockable context.\n", + mn->ops->invalidate_range_start, _ret, + !blockable ? "non-": ""); + ret = _ret; + } + } } srcu_read_unlock(&srcu, id); + + return ret; } EXPORT_SYMBOL_GPL(__mmu_notifier_invalidate_range_start); diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 84081e77bc51..5a936cf24d79 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -479,9 +479,10 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); static struct task_struct *oom_reaper_list; static DEFINE_SPINLOCK(oom_reaper_lock); -void __oom_reap_task_mm(struct mm_struct *mm) +bool __oom_reap_task_mm(struct mm_struct *mm) { struct vm_area_struct *vma; + bool ret = true; /* * Tell all users of get_user/copy_from_user etc... that the content @@ -511,12 +512,17 @@ void __oom_reap_task_mm(struct mm_struct *mm) struct mmu_gather tlb; tlb_gather_mmu(&tlb, mm, start, end); - mmu_notifier_invalidate_range_start(mm, start, end); + if (mmu_notifier_invalidate_range_start_nonblock(mm, start, end)) { + ret = false; + continue; + } unmap_page_range(&tlb, vma, start, end, NULL); mmu_notifier_invalidate_range_end(mm, start, end); tlb_finish_mmu(&tlb, start, end); } } + + return ret; } static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) @@ -545,18 +551,6 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) goto unlock_oom; } - /* - * If the mm has invalidate_{start,end}() notifiers that could block, - * sleep to give the oom victim some more time. - * TODO: we really want to get rid of this ugly hack and make sure that - * notifiers cannot block for unbounded amount of time - */ - if (mm_has_blockable_invalidate_notifiers(mm)) { - up_read(&mm->mmap_sem); - schedule_timeout_idle(HZ); - goto unlock_oom; - } - /* * MMF_OOM_SKIP is set by exit_mmap when the OOM reaper can't * work on the mm anymore. The check for MMF_OOM_SKIP must run @@ -571,7 +565,12 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) trace_start_task_reaping(tsk->pid); - __oom_reap_task_mm(mm); + /* failed to reap part of the address space. Try again later */ + if (!__oom_reap_task_mm(mm)) { + up_read(&mm->mmap_sem); + ret = false; + goto unlock_oom; + } pr_info("oom_reaper: reaped process %d (%s), now anon-rss:%lukB, file-rss:%lukB, shmem-rss:%lukB\n", task_pid_nr(tsk), tsk->comm, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ada21f47f22b..16ce38f178d1 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -135,9 +135,10 @@ static void kvm_uevent_notify_change(unsigned int type, struct kvm *kvm); static unsigned long long kvm_createvm_count; static unsigned long long kvm_active_vms; -__weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) +__weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, + unsigned long start, unsigned long end, bool blockable) { + return 0; } bool kvm_is_reserved_pfn(kvm_pfn_t pfn) @@ -354,13 +355,15 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn, srcu_read_unlock(&kvm->srcu, idx); } -static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, +static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, struct mm_struct *mm, unsigned long start, - unsigned long end) + unsigned long end, + bool blockable) { struct kvm *kvm = mmu_notifier_to_kvm(mn); int need_tlb_flush = 0, idx; + int ret; idx = srcu_read_lock(&kvm->srcu); spin_lock(&kvm->mmu_lock); @@ -378,9 +381,11 @@ static void kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, spin_unlock(&kvm->mmu_lock); - kvm_arch_mmu_notifier_invalidate_range(kvm, start, end); + ret = kvm_arch_mmu_notifier_invalidate_range(kvm, start, end, blockable); srcu_read_unlock(&kvm->srcu, idx); + + return ret; } static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn, -- 2.18.0