From mboxrd@z Thu Jan 1 00:00:00 1970 From: jglisse-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org Subject: [PATCH 1/2] mm/mmu_notifier: avoid double notification when it is useless v2 Date: Mon, 16 Oct 2017 23:10:02 -0400 Message-ID: <20171017031003.7481-2-jglisse@redhat.com> References: <20171017031003.7481-1-jglisse@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20171017031003.7481-1-jglisse-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org Cc: Andrea Arcangeli , Stephen Rothwell , Joerg Roedel , Benjamin Herrenschmidt , Andrew Donnellan , linuxppc-dev-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , linux-next-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Michael Ellerman , Alistair Popple , Andrew Morton , Linus Torvalds , David Woodhouse List-Id: iommu@lists.linux-foundation.org RnJvbTogSsOpcsO0bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+CgpUaGlzIHBhdGNoIG9u bHkgYWZmZWN0cyB1c2VycyBvZiBtbXVfbm90aWZpZXItPmludmFsaWRhdGVfcmFuZ2UgY2FsbGJh Y2sKd2hpY2ggYXJlIGRldmljZSBkcml2ZXJzIHJlbGF0ZWQgdG8gQVRTL1BBU0lELCBDQVBJLCBJ T01NVXYyLCBTVk0gLi4uCmFuZCBpdCBpcyBhbiBvcHRpbWl6YXRpb24gZm9yIHRob3NlIHVzZXJz LiBFdmVyeW9uZSBlbHNlIGlzIHVuYWZmZWN0ZWQKYnkgaXQuCgpXaGVuIGNsZWFyaW5nIGEgcHRl L3BtZCB3ZSBhcmUgZ2l2ZW4gYSBjaG9pY2UgdG8gbm90aWZ5IHRoZSBldmVudCB1bmRlcgp0aGUg cGFnZSB0YWJsZSBsb2NrIChub3RpZnkgdmVyc2lvbiBvZiAqX2NsZWFyX2ZsdXNoIGhlbHBlcnMg ZG8gY2FsbCB0aGUKbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2UpLiBCdXQgdGhhdCBub3Rp ZmljYXRpb24gaXMgbm90IG5lY2Vzc2FyeSBpbgphbGwgY2FzZXMuCgpUaGlzIHBhdGNoZXMgcmVt b3ZlIGFsbW9zdCBhbGwgY2FzZXMgd2hlcmUgaXQgaXMgdXNlbGVzcyB0byBoYXZlIGEgY2FsbAp0 byBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZSBiZWZvcmUgbW11X25vdGlmaWVyX2ludmFs aWRhdGVfcmFuZ2VfZW5kLgpJdCBhbHNvIGFkZHMgZG9jdW1lbnRhdGlvbiBpbiBhbGwgdGhvc2Ug Y2FzZXMgZXhwbGFpbmluZyB3aHkuCgpCZWxvdyBpcyBhIG1vcmUgaW4gZGVwdGggYW5hbHlzaXMg b2Ygd2h5IHRoaXMgaXMgZmluZSB0byBkbyB0aGlzOgoKRm9yIHNlY29uZGFyeSBUTEIgKG5vbiBD UFUgVExCKSBsaWtlIElPTU1VIFRMQiBvciBkZXZpY2UgVExCICh3aGVuIGRldmljZQp1c2UgdGhp bmcgbGlrZSBBVFMvUEFTSUQgdG8gZ2V0IHRoZSBJT01NVSB0byB3YWxrIHRoZSBDUFUgcGFnZSB0 YWJsZSB0bwphY2Nlc3MgYSBwcm9jZXNzIHZpcnR1YWwgYWRkcmVzcyBzcGFjZSkuIFRoZXJlIGlz IG9ubHkgMiBjYXNlcyB3aGVuIHlvdQpuZWVkIHRvIG5vdGlmeSB0aG9zZSBzZWNvbmRhcnkgVExC IHdoaWxlIGhvbGRpbmcgcGFnZSB0YWJsZSBsb2NrIHdoZW4KY2xlYXJpbmcgYSBwdGUvcG1kOgoK ICBBKSBwYWdlIGJhY2tpbmcgYWRkcmVzcyBpcyBmcmVlIGJlZm9yZSBtbXVfbm90aWZpZXJfaW52 YWxpZGF0ZV9yYW5nZV9lbmQKICBCKSBhIHBhZ2UgdGFibGUgZW50cnkgaXMgdXBkYXRlZCB0byBw b2ludCB0byBhIG5ldyBwYWdlIChDT1csIHdyaXRlIGZhdWx0CiAgICAgb24gemVybyBwYWdlLCBf X3JlcGxhY2VfcGFnZSgpLCAuLi4pCgpDYXNlIEEgaXMgb2J2aW91cyB5b3UgZG8gbm90IHdhbnQg dG8gdGFrZSB0aGUgcmlzayBmb3IgdGhlIGRldmljZSB0byB3cml0ZQp0byBhIHBhZ2UgdGhhdCBt aWdodCBub3cgYmUgdXNlZCBieSBzb21ldGhpbmcgY29tcGxldGVseSBkaWZmZXJlbnQuCgpDYXNl IEIgaXMgbW9yZSBzdWJ0bGUuIEZvciBjb3JyZWN0bmVzcyBpdCByZXF1aXJlcyB0aGUgZm9sbG93 aW5nIHNlcXVlbmNlCnRvIGhhcHBlbjoKICAtIHRha2UgcGFnZSB0YWJsZSBsb2NrCiAgLSBjbGVh ciBwYWdlIHRhYmxlIGVudHJ5IGFuZCBub3RpZnkgKHBtZC9wdGVfaHVnZV9jbGVhcl9mbHVzaF9u b3RpZnkoKSkKICAtIHNldCBwYWdlIHRhYmxlIGVudHJ5IHRvIHBvaW50IHRvIG5ldyBwYWdlCgpJ ZiBjbGVhcmluZyB0aGUgcGFnZSB0YWJsZSBlbnRyeSBpcyBub3QgZm9sbG93ZWQgYnkgYSBub3Rp ZnkgYmVmb3JlIHNldHRpbmcKdGhlIG5ldyBwdGUvcG1kIHZhbHVlIHRoZW4geW91IGNhbiBicmVh ayBtZW1vcnkgbW9kZWwgbGlrZSBDMTEgb3IgQysrMTEgZm9yCnRoZSBkZXZpY2UuCgpDb25zaWRl ciB0aGUgZm9sbG93aW5nIHNjZW5hcmlvIChkZXZpY2UgdXNlIGEgZmVhdHVyZSBzaW1pbGFyIHRv IEFUUy8KUEFTSUQpOgoKVHdvIGFkZHJlc3MgYWRkckEgYW5kIGFkZHJCIHN1Y2ggdGhhdCB8YWRk ckEgLSBhZGRyQnwgPj0gUEFHRV9TSVpFIHdlCmFzc3VtZSB0aGV5IGFyZSB3cml0ZSBwcm90ZWN0 ZWQgZm9yIENPVyAob3RoZXIgY2FzZSBvZiBCIGFwcGx5IHRvbykuCgpbVGltZSBOXSAtLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LQpDUFUtdGhyZWFkLTAgIHt0cnkgdG8gd3JpdGUgdG8gYWRkckF9CkNQVS10aHJlYWQtMSAge3Ry eSB0byB3cml0ZSB0byBhZGRyQn0KQ1BVLXRocmVhZC0yICB7fQpDUFUtdGhyZWFkLTMgIHt9CkRF Vi10aHJlYWQtMCAge3JlYWQgYWRkckEgYW5kIHBvcHVsYXRlIGRldmljZSBUTEJ9CkRFVi10aHJl YWQtMiAge3JlYWQgYWRkckIgYW5kIHBvcHVsYXRlIGRldmljZSBUTEJ9CltUaW1lIE4rMV0gLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tCkNQVS10aHJlYWQtMCAge0NPV19zdGVwMDoge21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KGFkZHJBKX19CkNQVS10aHJlYWQtMSAge0NPV19zdGVwMDoge21tdV9ub3RpZmll cl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KGFkZHJCKX19CkNQVS10aHJlYWQtMiAge30KQ1BVLXRo cmVhZC0zICB7fQpERVYtdGhyZWFkLTAgIHt9CkRFVi10aHJlYWQtMiAge30KW1RpbWUgTisyXSAt LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0KQ1BVLXRocmVhZC0wICB7Q09XX3N0ZXAxOiB7dXBkYXRlIHBhZ2UgdGFibGUgcG9pbnQg dG8gbmV3IHBhZ2UgZm9yIGFkZHJBfX0KQ1BVLXRocmVhZC0xICB7Q09XX3N0ZXAxOiB7dXBkYXRl IHBhZ2UgdGFibGUgcG9pbnQgdG8gbmV3IHBhZ2UgZm9yIGFkZHJCfX0KQ1BVLXRocmVhZC0yICB7 fQpDUFUtdGhyZWFkLTMgIHt9CkRFVi10aHJlYWQtMCAge30KREVWLXRocmVhZC0yICB7fQpbVGlt ZSBOKzNdIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLQpDUFUtdGhyZWFkLTAgIHtwcmVlbXB0ZWR9CkNQVS10aHJlYWQtMSAge3By ZWVtcHRlZH0KQ1BVLXRocmVhZC0yICB7d3JpdGUgdG8gYWRkckEgd2hpY2ggaXMgYSB3cml0ZSB0 byBuZXcgcGFnZX0KQ1BVLXRocmVhZC0zICB7fQpERVYtdGhyZWFkLTAgIHt9CkRFVi10aHJlYWQt MiAge30KW1RpbWUgTiszXSAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KQ1BVLXRocmVhZC0wICB7cHJlZW1wdGVkfQpDUFUtdGhy ZWFkLTEgIHtwcmVlbXB0ZWR9CkNQVS10aHJlYWQtMiAge30KQ1BVLXRocmVhZC0zICB7d3JpdGUg dG8gYWRkckIgd2hpY2ggaXMgYSB3cml0ZSB0byBuZXcgcGFnZX0KREVWLXRocmVhZC0wICB7fQpE RVYtdGhyZWFkLTIgIHt9CltUaW1lIE4rNF0gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCkNQVS10aHJlYWQtMCAge3ByZWVtcHRl ZH0KQ1BVLXRocmVhZC0xICB7Q09XX3N0ZXAzOiB7bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFu Z2VfZW5kKGFkZHJCKX19CkNQVS10aHJlYWQtMiAge30KQ1BVLXRocmVhZC0zICB7fQpERVYtdGhy ZWFkLTAgIHt9CkRFVi10aHJlYWQtMiAge30KW1RpbWUgTis1XSAtLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KQ1BVLXRocmVhZC0w ICB7cHJlZW1wdGVkfQpDUFUtdGhyZWFkLTEgIHt9CkNQVS10aHJlYWQtMiAge30KQ1BVLXRocmVh ZC0zICB7fQpERVYtdGhyZWFkLTAgIHtyZWFkIGFkZHJBIGZyb20gb2xkIHBhZ2V9CkRFVi10aHJl YWQtMiAge3JlYWQgYWRkckIgZnJvbSBuZXcgcGFnZX0KClNvIGhlcmUgYmVjYXVzZSBhdCB0aW1l IE4rMiB0aGUgY2xlYXIgcGFnZSB0YWJsZSBlbnRyeSB3YXMgbm90IHBhaXIgd2l0aCBhCm5vdGlm aWNhdGlvbiB0byBpbnZhbGlkYXRlIHRoZSBzZWNvbmRhcnkgVExCLCB0aGUgZGV2aWNlIHNlZSB0 aGUgbmV3IHZhbHVlCmZvciBhZGRyQiBiZWZvcmUgc2VpbmcgdGhlIG5ldyB2YWx1ZSBmb3IgYWRk ckEuIFRoaXMgYnJlYWsgdG90YWwgbWVtb3J5Cm9yZGVyaW5nIGZvciB0aGUgZGV2aWNlLgoKV2hl biBjaGFuZ2luZyBhIHB0ZSB0byB3cml0ZSBwcm90ZWN0IG9yIHRvIHBvaW50IHRvIGEgbmV3IHdy aXRlIHByb3RlY3RlZApwYWdlIHdpdGggc2FtZSBjb250ZW50IChLU00pIGl0IGlzIG9rIHRvIGRl bGF5IGludmFsaWRhdGVfcmFuZ2UgY2FsbGJhY2sgdG8KbW11X25vdGlmaWVyX2ludmFsaWRhdGVf cmFuZ2VfZW5kKCkgb3V0c2lkZSB0aGUgcGFnZSB0YWJsZSBsb2NrLiBUaGlzIGlzCnRydWUgZXZl biBpZiB0aGUgdGhyZWFkIGRvaW5nIHBhZ2UgdGFibGUgdXBkYXRlIGlzIHByZWVtcHRlZCByaWdo dCBhZnRlcgpyZWxlYXNpbmcgcGFnZSB0YWJsZSBsb2NrIGJlZm9yZSBjYWxsaW5nIG1tdV9ub3Rp Zmllcl9pbnZhbGlkYXRlX3JhbmdlX2VuZAoKQ2hhbmdlZCBzaW5jZSB2MToKICAtIHR5cG9zICh0 aGFua3MgdG8gQW5kcmVhKQogIC0gQXZvaWQgdW5uZWNlc3NhcnkgcHJlY2F1dGlvbiBpbiB0cnlf dG9fdW5tYXAoKSAoQW5kcmVhKQogIC0gQmUgbW9yZSBjb25zZXJ2YXRpdmUgaW4gdHJ5X3RvX3Vu bWFwX29uZSgpCgpTaWduZWQtb2ZmLWJ5OiBKw6lyw7RtZSBHbGlzc2UgPGpnbGlzc2VAcmVkaGF0 LmNvbT4KQ2M6IEFuZHJlYSBBcmNhbmdlbGkgPGFhcmNhbmdlQHJlZGhhdC5jb20+CkNjOiBOYWRh diBBbWl0IDxuYWRhdi5hbWl0QGdtYWlsLmNvbT4KQ2M6IExpbnVzIFRvcnZhbGRzIDx0b3J2YWxk c0BsaW51eC1mb3VuZGF0aW9uLm9yZz4KQ2M6IEFuZHJldyBNb3J0b24gPGFrcG1AbGludXgtZm91 bmRhdGlvbi5vcmc+CkNjOiBKb2VyZyBSb2VkZWwgPGpyb2VkZWxAc3VzZS5kZT4KQ2M6IFN1cmF2 ZWUgU3V0aGlrdWxwYW5pdCA8c3VyYXZlZS5zdXRoaWt1bHBhbml0QGFtZC5jb20+CkNjOiBEYXZp ZCBXb29kaG91c2UgPGR3bXcyQGluZnJhZGVhZC5vcmc+CkNjOiBBbGlzdGFpciBQb3BwbGUgPGFs aXN0YWlyQHBvcHBsZS5pZC5hdT4KQ2M6IE1pY2hhZWwgRWxsZXJtYW4gPG1wZUBlbGxlcm1hbi5p ZC5hdT4KQ2M6IEJlbmphbWluIEhlcnJlbnNjaG1pZHQgPGJlbmhAa2VybmVsLmNyYXNoaW5nLm9y Zz4KQ2M6IFN0ZXBoZW4gUm90aHdlbGwgPHNmckBjYW5iLmF1dWcub3JnLmF1PgpDYzogQW5kcmV3 IERvbm5lbGxhbiA8YW5kcmV3LmRvbm5lbGxhbkBhdTEuaWJtLmNvbT4KCkNjOiBpb21tdUBsaXN0 cy5saW51eC1mb3VuZGF0aW9uLm9yZwpDYzogbGludXhwcGMtZGV2QGxpc3RzLm96bGFicy5vcmcK Q2M6IGxpbnV4LW5leHRAdmdlci5rZXJuZWwub3JnCi0tLQogRG9jdW1lbnRhdGlvbi92bS9tbXVf bm90aWZpZXIudHh0IHwgOTMgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr CiBmcy9kYXguYyAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgOSArKystCiBpbmNsdWRlL2xp bnV4L21tdV9ub3RpZmllci5oICAgICAgfCAgMyArLQogbW0vaHVnZV9tZW1vcnkuYyAgICAgICAg ICAgICAgICAgIHwgMjAgKysrKysrKy0tCiBtbS9odWdldGxiLmMgICAgICAgICAgICAgICAgICAg ICAgfCAxNiArKysrKy0tCiBtbS9rc20uYyAgICAgICAgICAgICAgICAgICAgICAgICAgfCAxNSAr KysrKystCiBtbS9ybWFwLmMgICAgICAgICAgICAgICAgICAgICAgICAgfCA1OSArKysrKysrKysr KysrKysrKysrKysrLS0tCiA3IGZpbGVzIGNoYW5nZWQsIDE5OCBpbnNlcnRpb25zKCspLCAxNyBk ZWxldGlvbnMoLSkKIGNyZWF0ZSBtb2RlIDEwMDY0NCBEb2N1bWVudGF0aW9uL3ZtL21tdV9ub3Rp Zmllci50eHQKCmRpZmYgLS1naXQgYS9Eb2N1bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQg Yi9Eb2N1bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQKbmV3IGZpbGUgbW9kZSAxMDA2NDQK aW5kZXggMDAwMDAwMDAwMDAwLi4yM2I0NjI1NjZiYjcKLS0tIC9kZXYvbnVsbAorKysgYi9Eb2N1 bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQKQEAgLTAsMCArMSw5MyBAQAorV2hlbiBkbyB5 b3UgbmVlZCB0byBub3RpZnkgaW5zaWRlIHBhZ2UgdGFibGUgbG9jayA/CisKK1doZW4gY2xlYXJp bmcgYSBwdGUvcG1kIHdlIGFyZSBnaXZlbiBhIGNob2ljZSB0byBub3RpZnkgdGhlIGV2ZW50IHRo cm91Z2gKKyhub3RpZnkgdmVyc2lvbiBvZiAqX2NsZWFyX2ZsdXNoIGNhbGwgbW11X25vdGlmaWVy X2ludmFsaWRhdGVfcmFuZ2UpIHVuZGVyCit0aGUgcGFnZSB0YWJsZSBsb2NrLiBCdXQgdGhhdCBu b3RpZmljYXRpb24gaXMgbm90IG5lY2Vzc2FyeSBpbiBhbGwgY2FzZXMuCisKK0ZvciBzZWNvbmRh cnkgVExCIChub24gQ1BVIFRMQikgbGlrZSBJT01NVSBUTEIgb3IgZGV2aWNlIFRMQiAod2hlbiBk ZXZpY2UgdXNlCit0aGluZyBsaWtlIEFUUy9QQVNJRCB0byBnZXQgdGhlIElPTU1VIHRvIHdhbGsg dGhlIENQVSBwYWdlIHRhYmxlIHRvIGFjY2VzcyBhCitwcm9jZXNzIHZpcnR1YWwgYWRkcmVzcyBz cGFjZSkuIFRoZXJlIGlzIG9ubHkgMiBjYXNlcyB3aGVuIHlvdSBuZWVkIHRvIG5vdGlmeQordGhv c2Ugc2Vjb25kYXJ5IFRMQiB3aGlsZSBob2xkaW5nIHBhZ2UgdGFibGUgbG9jayB3aGVuIGNsZWFy aW5nIGEgcHRlL3BtZDoKKworICBBKSBwYWdlIGJhY2tpbmcgYWRkcmVzcyBpcyBmcmVlIGJlZm9y ZSBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQoKQorICBCKSBhIHBhZ2UgdGFibGUg ZW50cnkgaXMgdXBkYXRlZCB0byBwb2ludCB0byBhIG5ldyBwYWdlIChDT1csIHdyaXRlIGZhdWx0 CisgICAgIG9uIHplcm8gcGFnZSwgX19yZXBsYWNlX3BhZ2UoKSwgLi4uKQorCitDYXNlIEEgaXMg b2J2aW91cyB5b3UgZG8gbm90IHdhbnQgdG8gdGFrZSB0aGUgcmlzayBmb3IgdGhlIGRldmljZSB0 byB3cml0ZSB0bworYSBwYWdlIHRoYXQgbWlnaHQgbm93IGJlIHVzZWQgYnkgc29tZSBjb21wbGV0 ZWx5IGRpZmZlcmVudCB0YXNrLgorCitDYXNlIEIgaXMgbW9yZSBzdWJ0bGUuIEZvciBjb3JyZWN0 bmVzcyBpdCByZXF1aXJlcyB0aGUgZm9sbG93aW5nIHNlcXVlbmNlIHRvCitoYXBwZW46CisgIC0g dGFrZSBwYWdlIHRhYmxlIGxvY2sKKyAgLSBjbGVhciBwYWdlIHRhYmxlIGVudHJ5IGFuZCBub3Rp ZnkgKFtwbWQvcHRlXXBfaHVnZV9jbGVhcl9mbHVzaF9ub3RpZnkoKSkKKyAgLSBzZXQgcGFnZSB0 YWJsZSBlbnRyeSB0byBwb2ludCB0byBuZXcgcGFnZQorCitJZiBjbGVhcmluZyB0aGUgcGFnZSB0 YWJsZSBlbnRyeSBpcyBub3QgZm9sbG93ZWQgYnkgYSBub3RpZnkgYmVmb3JlIHNldHRpbmcKK3Ro ZSBuZXcgcHRlL3BtZCB2YWx1ZSB0aGVuIHlvdSBjYW4gYnJlYWsgbWVtb3J5IG1vZGVsIGxpa2Ug QzExIG9yIEMrKzExIGZvcgordGhlIGRldmljZS4KKworQ29uc2lkZXIgdGhlIGZvbGxvd2luZyBz Y2VuYXJpbyAoZGV2aWNlIHVzZSBhIGZlYXR1cmUgc2ltaWxhciB0byBBVFMvUEFTSUQpOgorCitU d28gYWRkcmVzcyBhZGRyQSBhbmQgYWRkckIgc3VjaCB0aGF0IHxhZGRyQSAtIGFkZHJCfCA+PSBQ QUdFX1NJWkUgd2UgYXNzdW1lCit0aGV5IGFyZSB3cml0ZSBwcm90ZWN0ZWQgZm9yIENPVyAob3Ro ZXIgY2FzZSBvZiBCIGFwcGx5IHRvbykuCisKK1tUaW1lIE5dIC0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCitDUFUtdGhy ZWFkLTAgIHt0cnkgdG8gd3JpdGUgdG8gYWRkckF9CitDUFUtdGhyZWFkLTEgIHt0cnkgdG8gd3Jp dGUgdG8gYWRkckJ9CitDUFUtdGhyZWFkLTIgIHt9CitDUFUtdGhyZWFkLTMgIHt9CitERVYtdGhy ZWFkLTAgIHtyZWFkIGFkZHJBIGFuZCBwb3B1bGF0ZSBkZXZpY2UgVExCfQorREVWLXRocmVhZC0y ICB7cmVhZCBhZGRyQiBhbmQgcG9wdWxhdGUgZGV2aWNlIFRMQn0KK1tUaW1lIE4rMV0gLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tCitDUFUtdGhyZWFkLTAgIHtDT1dfc3RlcDA6IHttbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9y YW5nZV9zdGFydChhZGRyQSl9fQorQ1BVLXRocmVhZC0xICB7Q09XX3N0ZXAwOiB7bW11X25vdGlm aWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoYWRkckIpfX0KK0NQVS10aHJlYWQtMiAge30KK0NQ VS10aHJlYWQtMyAge30KK0RFVi10aHJlYWQtMCAge30KK0RFVi10aHJlYWQtMiAge30KK1tUaW1l IE4rMl0gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tCitDUFUtdGhyZWFkLTAgIHtDT1dfc3RlcDE6IHt1cGRhdGUgcGFnZSB0 YWJsZSB0byBwb2ludCB0byBuZXcgcGFnZSBmb3IgYWRkckF9fQorQ1BVLXRocmVhZC0xICB7Q09X X3N0ZXAxOiB7dXBkYXRlIHBhZ2UgdGFibGUgdG8gcG9pbnQgdG8gbmV3IHBhZ2UgZm9yIGFkZHJC fX0KK0NQVS10aHJlYWQtMiAge30KK0NQVS10aHJlYWQtMyAge30KK0RFVi10aHJlYWQtMCAge30K K0RFVi10aHJlYWQtMiAge30KK1tUaW1lIE4rM10gLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCitDUFUtdGhyZWFkLTAgIHtw cmVlbXB0ZWR9CitDUFUtdGhyZWFkLTEgIHtwcmVlbXB0ZWR9CitDUFUtdGhyZWFkLTIgIHt3cml0 ZSB0byBhZGRyQSB3aGljaCBpcyBhIHdyaXRlIHRvIG5ldyBwYWdlfQorQ1BVLXRocmVhZC0zICB7 fQorREVWLXRocmVhZC0wICB7fQorREVWLXRocmVhZC0yICB7fQorW1RpbWUgTiszXSAtLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0KK0NQVS10aHJlYWQtMCAge3ByZWVtcHRlZH0KK0NQVS10aHJlYWQtMSAge3ByZWVtcHRlZH0K K0NQVS10aHJlYWQtMiAge30KK0NQVS10aHJlYWQtMyAge3dyaXRlIHRvIGFkZHJCIHdoaWNoIGlz IGEgd3JpdGUgdG8gbmV3IHBhZ2V9CitERVYtdGhyZWFkLTAgIHt9CitERVYtdGhyZWFkLTIgIHt9 CitbVGltZSBOKzRdIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLQorQ1BVLXRocmVhZC0wICB7cHJlZW1wdGVkfQorQ1BVLXRo cmVhZC0xICB7Q09XX3N0ZXAzOiB7bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2VfZW5kKGFk ZHJCKX19CitDUFUtdGhyZWFkLTIgIHt9CitDUFUtdGhyZWFkLTMgIHt9CitERVYtdGhyZWFkLTAg IHt9CitERVYtdGhyZWFkLTIgIHt9CitbVGltZSBOKzVdIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQorQ1BVLXRocmVhZC0w ICB7cHJlZW1wdGVkfQorQ1BVLXRocmVhZC0xICB7fQorQ1BVLXRocmVhZC0yICB7fQorQ1BVLXRo cmVhZC0zICB7fQorREVWLXRocmVhZC0wICB7cmVhZCBhZGRyQSBmcm9tIG9sZCBwYWdlfQorREVW LXRocmVhZC0yICB7cmVhZCBhZGRyQiBmcm9tIG5ldyBwYWdlfQorCitTbyBoZXJlIGJlY2F1c2Ug YXQgdGltZSBOKzIgdGhlIGNsZWFyIHBhZ2UgdGFibGUgZW50cnkgd2FzIG5vdCBwYWlyIHdpdGgg YQorbm90aWZpY2F0aW9uIHRvIGludmFsaWRhdGUgdGhlIHNlY29uZGFyeSBUTEIsIHRoZSBkZXZp Y2Ugc2VlIHRoZSBuZXcgdmFsdWUgZm9yCithZGRyQiBiZWZvcmUgc2VpbmcgdGhlIG5ldyB2YWx1 ZSBmb3IgYWRkckEuIFRoaXMgYnJlYWsgdG90YWwgbWVtb3J5IG9yZGVyaW5nCitmb3IgdGhlIGRl dmljZS4KKworV2hlbiBjaGFuZ2luZyBhIHB0ZSB0byB3cml0ZSBwcm90ZWN0IG9yIHRvIHBvaW50 IHRvIGEgbmV3IHdyaXRlIHByb3RlY3RlZCBwYWdlCit3aXRoIHNhbWUgY29udGVudCAoS1NNKSBp dCBpcyBmaW5lIHRvIGRlbGF5IHRoZSBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZQorY2Fs bCB0byBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQoKSBvdXRzaWRlIHRoZSBwYWdl IHRhYmxlIGxvY2suIFRoaXMKK2lzIHRydWUgZXZlbiBpZiB0aGUgdGhyZWFkIGRvaW5nIHRoZSBw YWdlIHRhYmxlIHVwZGF0ZSBpcyBwcmVlbXB0ZWQgcmlnaHQgYWZ0ZXIKK3JlbGVhc2luZyBwYWdl IHRhYmxlIGxvY2sgYnV0IGJlZm9yZSBjYWxsIG1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jhbmdl X2VuZCgpLgpkaWZmIC0tZ2l0IGEvZnMvZGF4LmMgYi9mcy9kYXguYwppbmRleCBmM2E0NGE3YzE0 YjMuLjllYzc5NzQyNGU0ZiAxMDA2NDQKLS0tIGEvZnMvZGF4LmMKKysrIGIvZnMvZGF4LmMKQEAg LTYxNCw2ICs2MTQsMTMgQEAgc3RhdGljIHZvaWQgZGF4X21hcHBpbmdfZW50cnlfbWtjbGVhbihz dHJ1Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZywKIAkJaWYgKGZvbGxvd19wdGVfcG1kKHZtYS0+ dm1fbW0sIGFkZHJlc3MsICZzdGFydCwgJmVuZCwgJnB0ZXAsICZwbWRwLCAmcHRsKSkKIAkJCWNv bnRpbnVlOwogCisJCS8qCisJCSAqIE5vIG5lZWQgdG8gY2FsbCBtbXVfbm90aWZpZXJfaW52YWxp ZGF0ZV9yYW5nZSgpIGFzIHdlIGFyZQorCQkgKiBkb3duZ3JhZGluZyBwYWdlIHRhYmxlIHByb3Rl Y3Rpb24gbm90IGNoYW5naW5nIGl0IHRvIHBvaW50CisJCSAqIHRvIGEgbmV3IHBhZ2UuCisJCSAq CisJCSAqIFNlZSBEb2N1bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQKKwkJICovCiAJCWlm IChwbWRwKSB7CiAjaWZkZWYgQ09ORklHX0ZTX0RBWF9QTUQKIAkJCXBtZF90IHBtZDsKQEAgLTYy OCw3ICs2MzUsNiBAQCBzdGF0aWMgdm9pZCBkYXhfbWFwcGluZ19lbnRyeV9ta2NsZWFuKHN0cnVj dCBhZGRyZXNzX3NwYWNlICptYXBwaW5nLAogCQkJcG1kID0gcG1kX3dycHJvdGVjdChwbWQpOwog CQkJcG1kID0gcG1kX21rY2xlYW4ocG1kKTsKIAkJCXNldF9wbWRfYXQodm1hLT52bV9tbSwgYWRk cmVzcywgcG1kcCwgcG1kKTsKLQkJCW1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKHZtYS0+ dm1fbW0sIHN0YXJ0LCBlbmQpOwogdW5sb2NrX3BtZDoKIAkJCXNwaW5fdW5sb2NrKHB0bCk7CiAj ZW5kaWYKQEAgLTY0Myw3ICs2NDksNiBAQCBzdGF0aWMgdm9pZCBkYXhfbWFwcGluZ19lbnRyeV9t a2NsZWFuKHN0cnVjdCBhZGRyZXNzX3NwYWNlICptYXBwaW5nLAogCQkJcHRlID0gcHRlX3dycHJv dGVjdChwdGUpOwogCQkJcHRlID0gcHRlX21rY2xlYW4ocHRlKTsKIAkJCXNldF9wdGVfYXQodm1h LT52bV9tbSwgYWRkcmVzcywgcHRlcCwgcHRlKTsKLQkJCW1tdV9ub3RpZmllcl9pbnZhbGlkYXRl X3JhbmdlKHZtYS0+dm1fbW0sIHN0YXJ0LCBlbmQpOwogdW5sb2NrX3B0ZToKIAkJCXB0ZV91bm1h cF91bmxvY2socHRlcCwgcHRsKTsKIAkJfQpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbXVf bm90aWZpZXIuaCBiL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmgKaW5kZXggNjg2NmU4MTI2 OTgyLi40OWM5MjVjOTZiOGEgMTAwNjQ0Ci0tLSBhL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVy LmgKKysrIGIvaW5jbHVkZS9saW51eC9tbXVfbm90aWZpZXIuaApAQCAtMTU1LDcgKzE1NSw4IEBA IHN0cnVjdCBtbXVfbm90aWZpZXJfb3BzIHsKIAkgKiBzaGFyZWQgcGFnZS10YWJsZXMsIGl0IG5v dCBuZWNlc3NhcnkgdG8gaW1wbGVtZW50IHRoZQogCSAqIGludmFsaWRhdGVfcmFuZ2Vfc3RhcnQo KS9lbmQoKSBub3RpZmllcnMsIGFzCiAJICogaW52YWxpZGF0ZV9yYW5nZSgpIGFscmVhZCBjYXRj aGVzIHRoZSBwb2ludHMgaW4gdGltZSB3aGVuIGFuCi0JICogZXh0ZXJuYWwgVExCIHJhbmdlIG5l ZWRzIHRvIGJlIGZsdXNoZWQuCisJICogZXh0ZXJuYWwgVExCIHJhbmdlIG5lZWRzIHRvIGJlIGZs dXNoZWQuIEZvciBtb3JlIGluIGRlcHRoCisJICogZGlzY3Vzc2lvbiBvbiB0aGlzIHNlZSBEb2N1 bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQKIAkgKgogCSAqIFRoZSBpbnZhbGlkYXRlX3Jh bmdlKCkgZnVuY3Rpb24gaXMgY2FsbGVkIHVuZGVyIHRoZSBwdGwKIAkgKiBzcGluLWxvY2sgYW5k IG5vdCBhbGxvd2VkIHRvIHNsZWVwLgpkaWZmIC0tZ2l0IGEvbW0vaHVnZV9tZW1vcnkuYyBiL21t L2h1Z2VfbWVtb3J5LmMKaW5kZXggYzAzN2QzZDM0OTUwLi5mZjViYzY0N2I1MWQgMTAwNjQ0Ci0t LSBhL21tL2h1Z2VfbWVtb3J5LmMKKysrIGIvbW0vaHVnZV9tZW1vcnkuYwpAQCAtMTE4Niw4ICsx MTg2LDE1IEBAIHN0YXRpYyBpbnQgZG9faHVnZV9wbWRfd3BfcGFnZV9mYWxsYmFjayhzdHJ1Y3Qg dm1fZmF1bHQgKnZtZiwgcG1kX3Qgb3JpZ19wbWQsCiAJCWdvdG8gb3V0X2ZyZWVfcGFnZXM7CiAJ Vk1fQlVHX09OX1BBR0UoIVBhZ2VIZWFkKHBhZ2UpLCBwYWdlKTsKIAorCS8qCisJICogTGVhdmUg cG1kIGVtcHR5IHVudGlsIHB0ZSBpcyBmaWxsZWQgbm90ZSB3ZSBtdXN0IG5vdGlmeSBoZXJlIGFz CisJICogY29uY3VycmVudCBDUFUgdGhyZWFkIG1pZ2h0IHdyaXRlIHRvIG5ldyBwYWdlIGJlZm9y ZSB0aGUgY2FsbCB0bworCSAqIG1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX2VuZCgpIGhh cHBlbnMgd2hpY2ggY2FuIGxlYWQgdG8gYQorCSAqIGRldmljZSBzZWVpbmcgbWVtb3J5IHdyaXRl IGluIGRpZmZlcmVudCBvcmRlciB0aGFuIENQVS4KKwkgKgorCSAqIFNlZSBEb2N1bWVudGF0aW9u L3ZtL21tdV9ub3RpZmllci50eHQKKwkgKi8KIAlwbWRwX2h1Z2VfY2xlYXJfZmx1c2hfbm90aWZ5 KHZtYSwgaGFkZHIsIHZtZi0+cG1kKTsKLQkvKiBsZWF2ZSBwbWQgZW1wdHkgdW50aWwgcHRlIGlz IGZpbGxlZCAqLwogCiAJcGd0YWJsZSA9IHBndGFibGVfdHJhbnNfaHVnZV93aXRoZHJhdyh2bWEt PnZtX21tLCB2bWYtPnBtZCk7CiAJcG1kX3BvcHVsYXRlKHZtYS0+dm1fbW0sICZfcG1kLCBwZ3Rh YmxlKTsKQEAgLTIwMjYsOCArMjAzMywxNSBAQCBzdGF0aWMgdm9pZCBfX3NwbGl0X2h1Z2VfemVy b19wYWdlX3BtZChzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSwKIAlwbWRfdCBfcG1kOwogCWlu dCBpOwogCi0JLyogbGVhdmUgcG1kIGVtcHR5IHVudGlsIHB0ZSBpcyBmaWxsZWQgKi8KLQlwbWRw X2h1Z2VfY2xlYXJfZmx1c2hfbm90aWZ5KHZtYSwgaGFkZHIsIHBtZCk7CisJLyoKKwkgKiBMZWF2 ZSBwbWQgZW1wdHkgdW50aWwgcHRlIGlzIGZpbGxlZCBub3RlIHRoYXQgaXQgaXMgZmluZSB0byBk ZWxheQorCSAqIG5vdGlmaWNhdGlvbiB1bnRpbCBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5n ZV9lbmQoKSBhcyB3ZSBhcmUKKwkgKiByZXBsYWNpbmcgYSB6ZXJvIHBtZCB3cml0ZSBwcm90ZWN0 ZWQgcGFnZSB3aXRoIGEgemVybyBwdGUgd3JpdGUKKwkgKiBwcm90ZWN0ZWQgcGFnZS4KKwkgKgor CSAqIFNlZSBEb2N1bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQKKwkgKi8KKwlwbWRwX2h1 Z2VfY2xlYXJfZmx1c2godm1hLCBoYWRkciwgcG1kKTsKIAogCXBndGFibGUgPSBwZ3RhYmxlX3Ry YW5zX2h1Z2Vfd2l0aGRyYXcobW0sIHBtZCk7CiAJcG1kX3BvcHVsYXRlKG1tLCAmX3BtZCwgcGd0 YWJsZSk7CmRpZmYgLS1naXQgYS9tbS9odWdldGxiLmMgYi9tbS9odWdldGxiLmMKaW5kZXggMTc2 OGVmYTRjNTAxLi42M2E2M2YxYjUzNmMgMTAwNjQ0Ci0tLSBhL21tL2h1Z2V0bGIuYworKysgYi9t bS9odWdldGxiLmMKQEAgLTMyNTQsOSArMzI1NCwxNCBAQCBpbnQgY29weV9odWdldGxiX3BhZ2Vf cmFuZ2Uoc3RydWN0IG1tX3N0cnVjdCAqZHN0LCBzdHJ1Y3QgbW1fc3RydWN0ICpzcmMsCiAJCQlz ZXRfaHVnZV9zd2FwX3B0ZV9hdChkc3QsIGFkZHIsIGRzdF9wdGUsIGVudHJ5LCBzeik7CiAJCX0g ZWxzZSB7CiAJCQlpZiAoY293KSB7CisJCQkJLyoKKwkJCQkgKiBObyBuZWVkIHRvIG5vdGlmeSBh cyB3ZSBhcmUgZG93bmdyYWRpbmcgcGFnZQorCQkJCSAqIHRhYmxlIHByb3RlY3Rpb24gbm90IGNo YW5naW5nIGl0IHRvIHBvaW50CisJCQkJICogdG8gYSBuZXcgcGFnZS4KKwkJCQkgKgorCQkJCSAq IFNlZSBEb2N1bWVudGF0aW9uL3ZtL21tdV9ub3RpZmllci50eHQKKwkJCQkgKi8KIAkJCQlodWdl X3B0ZXBfc2V0X3dycHJvdGVjdChzcmMsIGFkZHIsIHNyY19wdGUpOwotCQkJCW1tdV9ub3RpZmll cl9pbnZhbGlkYXRlX3JhbmdlKHNyYywgbW11bl9zdGFydCwKLQkJCQkJCQkJICAgbW11bl9lbmQp OwogCQkJfQogCQkJZW50cnkgPSBodWdlX3B0ZXBfZ2V0KHNyY19wdGUpOwogCQkJcHRlcGFnZSA9 IHB0ZV9wYWdlKGVudHJ5KTsKQEAgLTQyODgsNyArNDI5MywxMiBAQCB1bnNpZ25lZCBsb25nIGh1 Z2V0bGJfY2hhbmdlX3Byb3RlY3Rpb24oc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEsCiAJICog YW5kIHRoYXQgcGFnZSB0YWJsZSBiZSByZXVzZWQgYW5kIGZpbGxlZCB3aXRoIGp1bmsuCiAJICov CiAJZmx1c2hfaHVnZXRsYl90bGJfcmFuZ2Uodm1hLCBzdGFydCwgZW5kKTsKLQltbXVfbm90aWZp ZXJfaW52YWxpZGF0ZV9yYW5nZShtbSwgc3RhcnQsIGVuZCk7CisJLyoKKwkgKiBObyBuZWVkIHRv IGNhbGwgbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2UoKSB3ZSBhcmUgZG93bmdyYWRpbmcK KwkgKiBwYWdlIHRhYmxlIHByb3RlY3Rpb24gbm90IGNoYW5naW5nIGl0IHRvIHBvaW50IHRvIGEg bmV3IHBhZ2UuCisJICoKKwkgKiBTZWUgRG9jdW1lbnRhdGlvbi92bS9tbXVfbm90aWZpZXIudHh0 CisJICovCiAJaV9tbWFwX3VubG9ja193cml0ZSh2bWEtPnZtX2ZpbGUtPmZfbWFwcGluZyk7CiAJ bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2VfZW5kKG1tLCBzdGFydCwgZW5kKTsKIApkaWZm IC0tZ2l0IGEvbW0va3NtLmMgYi9tbS9rc20uYwppbmRleCA2Y2I2MGY0NmNjZTUuLmJlOGY0NTc2 Zjg0MiAxMDA2NDQKLS0tIGEvbW0va3NtLmMKKysrIGIvbW0va3NtLmMKQEAgLTEwNTIsOCArMTA1 MiwxMyBAQCBzdGF0aWMgaW50IHdyaXRlX3Byb3RlY3RfcGFnZShzdHJ1Y3Qgdm1fYXJlYV9zdHJ1 Y3QgKnZtYSwgc3RydWN0IHBhZ2UgKnBhZ2UsCiAJCSAqIFNvIHdlIGNsZWFyIHRoZSBwdGUgYW5k IGZsdXNoIHRoZSB0bGIgYmVmb3JlIHRoZSBjaGVjawogCQkgKiB0aGlzIGFzc3VyZSB1cyB0aGF0 IG5vIE9fRElSRUNUIGNhbiBoYXBwZW4gYWZ0ZXIgdGhlIGNoZWNrCiAJCSAqIG9yIGluIHRoZSBt aWRkbGUgb2YgdGhlIGNoZWNrLgorCQkgKgorCQkgKiBObyBuZWVkIHRvIG5vdGlmeSBhcyB3ZSBh cmUgZG93bmdyYWRpbmcgcGFnZSB0YWJsZSB0byByZWFkCisJCSAqIG9ubHkgbm90IGNoYW5naW5n IGl0IHRvIHBvaW50IHRvIGEgbmV3IHBhZ2UuCisJCSAqCisJCSAqIFNlZSBEb2N1bWVudGF0aW9u L3ZtL21tdV9ub3RpZmllci50eHQKIAkJICovCi0JCWVudHJ5ID0gcHRlcF9jbGVhcl9mbHVzaF9u b3RpZnkodm1hLCBwdm13LmFkZHJlc3MsIHB2bXcucHRlKTsKKwkJZW50cnkgPSBwdGVwX2NsZWFy X2ZsdXNoKHZtYSwgcHZtdy5hZGRyZXNzLCBwdm13LnB0ZSk7CiAJCS8qCiAJCSAqIENoZWNrIHRo YXQgbm8gT19ESVJFQ1Qgb3Igc2ltaWxhciBJL08gaXMgaW4gcHJvZ3Jlc3Mgb24gdGhlCiAJCSAq IHBhZ2UKQEAgLTExMzYsNyArMTE0MSwxMyBAQCBzdGF0aWMgaW50IHJlcGxhY2VfcGFnZShzdHJ1 Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSwgc3RydWN0IHBhZ2UgKnBhZ2UsCiAJfQogCiAJZmx1c2hf Y2FjaGVfcGFnZSh2bWEsIGFkZHIsIHB0ZV9wZm4oKnB0ZXApKTsKLQlwdGVwX2NsZWFyX2ZsdXNo X25vdGlmeSh2bWEsIGFkZHIsIHB0ZXApOworCS8qCisJICogTm8gbmVlZCB0byBub3RpZnkgYXMg d2UgYXJlIHJlcGxhY2luZyBhIHJlYWQgb25seSBwYWdlIHdpdGggYW5vdGhlcgorCSAqIHJlYWQg b25seSBwYWdlIHdpdGggdGhlIHNhbWUgY29udGVudC4KKwkgKgorCSAqIFNlZSBEb2N1bWVudGF0 aW9uL3ZtL21tdV9ub3RpZmllci50eHQKKwkgKi8KKwlwdGVwX2NsZWFyX2ZsdXNoKHZtYSwgYWRk ciwgcHRlcCk7CiAJc2V0X3B0ZV9hdF9ub3RpZnkobW0sIGFkZHIsIHB0ZXAsIG5ld3B0ZSk7CiAK IAlwYWdlX3JlbW92ZV9ybWFwKHBhZ2UsIGZhbHNlKTsKZGlmZiAtLWdpdCBhL21tL3JtYXAuYyBi L21tL3JtYXAuYwppbmRleCAwNjE4MjYyNzg1MjAuLjZiNWEwZjIxOWFjMCAxMDA2NDQKLS0tIGEv bW0vcm1hcC5jCisrKyBiL21tL3JtYXAuYwpAQCAtOTM3LDEwICs5MzcsMTUgQEAgc3RhdGljIGJv b2wgcGFnZV9ta2NsZWFuX29uZShzdHJ1Y3QgcGFnZSAqcGFnZSwgc3RydWN0IHZtX2FyZWFfc3Ry dWN0ICp2bWEsCiAjZW5kaWYKIAkJfQogCi0JCWlmIChyZXQpIHsKLQkJCW1tdV9ub3RpZmllcl9p bnZhbGlkYXRlX3JhbmdlKHZtYS0+dm1fbW0sIGNzdGFydCwgY2VuZCk7CisJCS8qCisJCSAqIE5v IG5lZWQgdG8gY2FsbCBtbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZSgpIGFzIHdlIGFyZQor CQkgKiBkb3duZ3JhZGluZyBwYWdlIHRhYmxlIHByb3RlY3Rpb24gbm90IGNoYW5naW5nIGl0IHRv IHBvaW50CisJCSAqIHRvIGEgbmV3IHBhZ2UuCisJCSAqCisJCSAqIFNlZSBEb2N1bWVudGF0aW9u L3ZtL21tdV9ub3RpZmllci50eHQKKwkJICovCisJCWlmIChyZXQpCiAJCQkoKmNsZWFuZWQpKys7 Ci0JCX0KIAl9CiAKIAltbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9lbmQodm1hLT52bV9t bSwgc3RhcnQsIGVuZCk7CkBAIC0xNDI0LDYgKzE0MjksMTAgQEAgc3RhdGljIGJvb2wgdHJ5X3Rv X3VubWFwX29uZShzdHJ1Y3QgcGFnZSAqcGFnZSwgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEs CiAJCQlpZiAocHRlX3NvZnRfZGlydHkocHRldmFsKSkKIAkJCQlzd3BfcHRlID0gcHRlX3N3cF9t a3NvZnRfZGlydHkoc3dwX3B0ZSk7CiAJCQlzZXRfcHRlX2F0KG1tLCBwdm13LmFkZHJlc3MsIHB2 bXcucHRlLCBzd3BfcHRlKTsKKwkJCS8qCisJCQkgKiBObyBuZWVkIHRvIGludmFsaWRhdGUgaGVy ZSBpdCB3aWxsIHN5bmNocm9uaXplIG9uCisJCQkgKiBhZ2FpbnN0IHRoZSBzcGVjaWFsIHN3YXAg bWlncmF0aW9uIHB0ZS4KKwkJCSAqLwogCQkJZ290byBkaXNjYXJkOwogCQl9CiAKQEAgLTE0ODEs NiArMTQ5MCw5IEBAIHN0YXRpYyBib29sIHRyeV90b191bm1hcF9vbmUoc3RydWN0IHBhZ2UgKnBh Z2UsIHN0cnVjdCB2bV9hcmVhX3N0cnVjdCAqdm1hLAogCQkJICogd2lsbCB0YWtlIGNhcmUgb2Yg dGhlIHJlc3QuCiAJCQkgKi8KIAkJCWRlY19tbV9jb3VudGVyKG1tLCBtbV9jb3VudGVyKHBhZ2Up KTsKKwkJCS8qIFdlIGhhdmUgdG8gaW52YWxpZGF0ZSBhcyB3ZSBjbGVhcmVkIHRoZSBwdGUgKi8K KwkJCW1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKG1tLCBhZGRyZXNzLAorCQkJCQkJICAg ICAgYWRkcmVzcyArIFBBR0VfU0laRSk7CiAJCX0gZWxzZSBpZiAoSVNfRU5BQkxFRChDT05GSUdf TUlHUkFUSU9OKSAmJgogCQkJCShmbGFncyAmIChUVFVfTUlHUkFUSU9OfFRUVV9TUExJVF9GUkVF WkUpKSkgewogCQkJc3dwX2VudHJ5X3QgZW50cnk7CkBAIC0xNDk2LDYgKzE1MDgsMTAgQEAgc3Rh dGljIGJvb2wgdHJ5X3RvX3VubWFwX29uZShzdHJ1Y3QgcGFnZSAqcGFnZSwgc3RydWN0IHZtX2Fy ZWFfc3RydWN0ICp2bWEsCiAJCQlpZiAocHRlX3NvZnRfZGlydHkocHRldmFsKSkKIAkJCQlzd3Bf cHRlID0gcHRlX3N3cF9ta3NvZnRfZGlydHkoc3dwX3B0ZSk7CiAJCQlzZXRfcHRlX2F0KG1tLCBh ZGRyZXNzLCBwdm13LnB0ZSwgc3dwX3B0ZSk7CisJCQkvKgorCQkJICogTm8gbmVlZCB0byBpbnZh bGlkYXRlIGhlcmUgaXQgd2lsbCBzeW5jaHJvbml6ZSBvbgorCQkJICogYWdhaW5zdCB0aGUgc3Bl Y2lhbCBzd2FwIG1pZ3JhdGlvbiBwdGUuCisJCQkgKi8KIAkJfSBlbHNlIGlmIChQYWdlQW5vbihw YWdlKSkgewogCQkJc3dwX2VudHJ5X3QgZW50cnkgPSB7IC52YWwgPSBwYWdlX3ByaXZhdGUoc3Vi cGFnZSkgfTsKIAkJCXB0ZV90IHN3cF9wdGU7CkBAIC0xNTA3LDYgKzE1MjMsOCBAQCBzdGF0aWMg Ym9vbCB0cnlfdG9fdW5tYXBfb25lKHN0cnVjdCBwYWdlICpwYWdlLCBzdHJ1Y3Qgdm1fYXJlYV9z dHJ1Y3QgKnZtYSwKIAkJCQlXQVJOX09OX09OQ0UoMSk7CiAJCQkJcmV0ID0gZmFsc2U7CiAJCQkJ LyogV2UgaGF2ZSB0byBpbnZhbGlkYXRlIGFzIHdlIGNsZWFyZWQgdGhlIHB0ZSAqLworCQkJCW1t dV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKG1tLCBhZGRyZXNzLAorCQkJCQkJCWFkZHJlc3Mg KyBQQUdFX1NJWkUpOwogCQkJCXBhZ2Vfdm1hX21hcHBlZF93YWxrX2RvbmUoJnB2bXcpOwogCQkJ CWJyZWFrOwogCQkJfQpAQCAtMTUxNCw2ICsxNTMyLDkgQEAgc3RhdGljIGJvb2wgdHJ5X3RvX3Vu bWFwX29uZShzdHJ1Y3QgcGFnZSAqcGFnZSwgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEsCiAJ CQkvKiBNQURWX0ZSRUUgcGFnZSBjaGVjayAqLwogCQkJaWYgKCFQYWdlU3dhcEJhY2tlZChwYWdl KSkgewogCQkJCWlmICghUGFnZURpcnR5KHBhZ2UpKSB7CisJCQkJCS8qIEludmFsaWRhdGUgYXMg d2UgY2xlYXJlZCB0aGUgcHRlICovCisJCQkJCW1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jhbmdl KG1tLAorCQkJCQkJYWRkcmVzcywgYWRkcmVzcyArIFBBR0VfU0laRSk7CiAJCQkJCWRlY19tbV9j b3VudGVyKG1tLCBNTV9BTk9OUEFHRVMpOwogCQkJCQlnb3RvIGRpc2NhcmQ7CiAJCQkJfQpAQCAt MTU0NywxMyArMTU2OCwzOSBAQCBzdGF0aWMgYm9vbCB0cnlfdG9fdW5tYXBfb25lKHN0cnVjdCBw YWdlICpwYWdlLCBzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSwKIAkJCWlmIChwdGVfc29mdF9k aXJ0eShwdGV2YWwpKQogCQkJCXN3cF9wdGUgPSBwdGVfc3dwX21rc29mdF9kaXJ0eShzd3BfcHRl KTsKIAkJCXNldF9wdGVfYXQobW0sIGFkZHJlc3MsIHB2bXcucHRlLCBzd3BfcHRlKTsKLQkJfSBl bHNlCisJCQkvKiBJbnZhbGlkYXRlIGFzIHdlIGNsZWFyZWQgdGhlIHB0ZSAqLworCQkJbW11X25v dGlmaWVyX2ludmFsaWRhdGVfcmFuZ2UobW0sIGFkZHJlc3MsCisJCQkJCQkgICAgICBhZGRyZXNz ICsgUEFHRV9TSVpFKTsKKwkJfSBlbHNlIHsKKwkJCS8qCisJCQkgKiBXZSBzaG91bGQgbm90IG5l ZWQgdG8gbm90aWZ5IGhlcmUgYXMgd2UgcmVhY2ggdGhpcworCQkJICogY2FzZSBvbmx5IGZyb20g ZnJlZXplX3BhZ2UoKSBpdHNlbGYgb25seSBjYWxsIGZyb20KKwkJCSAqIHNwbGl0X2h1Z2VfcGFn ZV90b19saXN0KCkgc28gZXZlcnl0aGluZyBiZWxvdyBtdXN0CisJCQkgKiBiZSB0cnVlOgorCQkJ ICogICAtIHBhZ2UgaXMgbm90IGFub255bW91cworCQkJICogICAtIHBhZ2UgaXMgbG9ja2VkCisJ CQkgKgorCQkJICogU28gYXMgaXQgaXMgYSBsb2NrZWQgZmlsZSBiYWNrIHBhZ2UgdGh1cyBpdCBj YW4gbm90CisJCQkgKiBiZSByZW1vdmUgZnJvbSB0aGUgcGFnZSBjYWNoZSBhbmQgcmVwbGFjZSBi eSBhIG5ldworCQkJICogcGFnZSBiZWZvcmUgbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vf ZW5kIHNvIG5vCisJCQkgKiBjb25jdXJyZW50IHRocmVhZCBtaWdodCB1cGRhdGUgaXRzIHBhZ2Ug dGFibGUgdG8KKwkJCSAqIHBvaW50IGF0IG5ldyBwYWdlIHdoaWxlIGEgZGV2aWNlIHN0aWxsIGlz IHVzaW5nIHRoaXMKKwkJCSAqIHBhZ2UuCisJCQkgKgorCQkJICogU2VlIERvY3VtZW50YXRpb24v dm0vbW11X25vdGlmaWVyLnR4dAorCQkJICovCiAJCQlkZWNfbW1fY291bnRlcihtbSwgbW1fY291 bnRlcl9maWxlKHBhZ2UpKTsKKwkJfQogZGlzY2FyZDoKKwkJLyoKKwkJICogTm8gbmVlZCB0byBj YWxsIG1tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlKCkgaXQgaGFzIGJlCisJCSAqIGRvbmUg YWJvdmUgZm9yIGFsbCBjYXNlcyByZXF1aXJpbmcgaXQgdG8gaGFwcGVuIHVuZGVyIHBhZ2UKKwkJ ICogdGFibGUgbG9jayBiZWZvcmUgbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2VfZW5kKCkK KwkJICoKKwkJICogU2VlIERvY3VtZW50YXRpb24vdm0vbW11X25vdGlmaWVyLnR4dAorCQkgKi8K IAkJcGFnZV9yZW1vdmVfcm1hcChzdWJwYWdlLCBQYWdlSHVnZShwYWdlKSk7CiAJCXB1dF9wYWdl KHBhZ2UpOwotCQltbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZShtbSwgYWRkcmVzcywKLQkJ CQkJICAgICAgYWRkcmVzcyArIFBBR0VfU0laRSk7CiAJfQogCiAJbW11X25vdGlmaWVyX2ludmFs aWRhdGVfcmFuZ2VfZW5kKHZtYS0+dm1fbW0sIHN0YXJ0LCBlbmQpOwotLSAKMi4xMy42CgpfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwppb21tdSBtYWlsaW5n IGxpc3QKaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51 eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3yGKtx5cdxzDrD6 for ; Tue, 17 Oct 2017 14:10:17 +1100 (AEDT) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrea Arcangeli , Nadav Amit , Linus Torvalds , Andrew Morton , Joerg Roedel , Suravee Suthikulpanit , David Woodhouse , Alistair Popple , Michael Ellerman , Benjamin Herrenschmidt , Stephen Rothwell , Andrew Donnellan , iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, linux-next@vger.kernel.org Subject: [PATCH 1/2] mm/mmu_notifier: avoid double notification when it is useless v2 Date: Mon, 16 Oct 2017 23:10:02 -0400 Message-Id: <20171017031003.7481-2-jglisse@redhat.com> In-Reply-To: <20171017031003.7481-1-jglisse@redhat.com> References: <20171017031003.7481-1-jglisse@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Jérôme Glisse This patch only affects users of mmu_notifier->invalidate_range callback which are device drivers related to ATS/PASID, CAPI, IOMMUv2, SVM ... and it is an optimization for those users. Everyone else is unaffected by it. When clearing a pte/pmd we are given a choice to notify the event under the page table lock (notify version of *_clear_flush helpers do call the mmu_notifier_invalidate_range). But that notification is not necessary in all cases. This patches remove almost all cases where it is useless to have a call to mmu_notifier_invalidate_range before mmu_notifier_invalidate_range_end. It also adds documentation in all those cases explaining why. Below is a more in depth analysis of why this is fine to do this: For secondary TLB (non CPU TLB) like IOMMU TLB or device TLB (when device use thing like ATS/PASID to get the IOMMU to walk the CPU page table to access a process virtual address space). There is only 2 cases when you need to notify those secondary TLB while holding page table lock when clearing a pte/pmd: A) page backing address is free before mmu_notifier_invalidate_range_end B) a page table entry is updated to point to a new page (COW, write fault on zero page, __replace_page(), ...) Case A is obvious you do not want to take the risk for the device to write to a page that might now be used by something completely different. Case B is more subtle. For correctness it requires the following sequence to happen: - take page table lock - clear page table entry and notify (pmd/pte_huge_clear_flush_notify()) - set page table entry to point to new page If clearing the page table entry is not followed by a notify before setting the new pte/pmd value then you can break memory model like C11 or C++11 for the device. Consider the following scenario (device use a feature similar to ATS/ PASID): Two address addrA and addrB such that |addrA - addrB| >= PAGE_SIZE we assume they are write protected for COW (other case of B apply too). [Time N] ----------------------------------------------------------------- CPU-thread-0 {try to write to addrA} CPU-thread-1 {try to write to addrB} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {read addrA and populate device TLB} DEV-thread-2 {read addrB and populate device TLB} [Time N+1] --------------------------------------------------------------- CPU-thread-0 {COW_step0: {mmu_notifier_invalidate_range_start(addrA)}} CPU-thread-1 {COW_step0: {mmu_notifier_invalidate_range_start(addrB)}} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+2] --------------------------------------------------------------- CPU-thread-0 {COW_step1: {update page table point to new page for addrA}} CPU-thread-1 {COW_step1: {update page table point to new page for addrB}} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+3] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {preempted} CPU-thread-2 {write to addrA which is a write to new page} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+3] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {preempted} CPU-thread-2 {} CPU-thread-3 {write to addrB which is a write to new page} DEV-thread-0 {} DEV-thread-2 {} [Time N+4] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {COW_step3: {mmu_notifier_invalidate_range_end(addrB)}} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+5] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {read addrA from old page} DEV-thread-2 {read addrB from new page} So here because at time N+2 the clear page table entry was not pair with a notification to invalidate the secondary TLB, the device see the new value for addrB before seing the new value for addrA. This break total memory ordering for the device. When changing a pte to write protect or to point to a new write protected page with same content (KSM) it is ok to delay invalidate_range callback to mmu_notifier_invalidate_range_end() outside the page table lock. This is true even if the thread doing page table update is preempted right after releasing page table lock before calling mmu_notifier_invalidate_range_end Changed since v1: - typos (thanks to Andrea) - Avoid unnecessary precaution in try_to_unmap() (Andrea) - Be more conservative in try_to_unmap_one() Signed-off-by: Jérôme Glisse Cc: Andrea Arcangeli Cc: Nadav Amit Cc: Linus Torvalds Cc: Andrew Morton Cc: Joerg Roedel Cc: Suravee Suthikulpanit Cc: David Woodhouse Cc: Alistair Popple Cc: Michael Ellerman Cc: Benjamin Herrenschmidt Cc: Stephen Rothwell Cc: Andrew Donnellan Cc: iommu@lists.linux-foundation.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-next@vger.kernel.org --- Documentation/vm/mmu_notifier.txt | 93 +++++++++++++++++++++++++++++++++++++++ fs/dax.c | 9 +++- include/linux/mmu_notifier.h | 3 +- mm/huge_memory.c | 20 +++++++-- mm/hugetlb.c | 16 +++++-- mm/ksm.c | 15 ++++++- mm/rmap.c | 59 ++++++++++++++++++++++--- 7 files changed, 198 insertions(+), 17 deletions(-) create mode 100644 Documentation/vm/mmu_notifier.txt diff --git a/Documentation/vm/mmu_notifier.txt b/Documentation/vm/mmu_notifier.txt new file mode 100644 index 000000000000..23b462566bb7 --- /dev/null +++ b/Documentation/vm/mmu_notifier.txt @@ -0,0 +1,93 @@ +When do you need to notify inside page table lock ? + +When clearing a pte/pmd we are given a choice to notify the event through +(notify version of *_clear_flush call mmu_notifier_invalidate_range) under +the page table lock. But that notification is not necessary in all cases. + +For secondary TLB (non CPU TLB) like IOMMU TLB or device TLB (when device use +thing like ATS/PASID to get the IOMMU to walk the CPU page table to access a +process virtual address space). There is only 2 cases when you need to notify +those secondary TLB while holding page table lock when clearing a pte/pmd: + + A) page backing address is free before mmu_notifier_invalidate_range_end() + B) a page table entry is updated to point to a new page (COW, write fault + on zero page, __replace_page(), ...) + +Case A is obvious you do not want to take the risk for the device to write to +a page that might now be used by some completely different task. + +Case B is more subtle. For correctness it requires the following sequence to +happen: + - take page table lock + - clear page table entry and notify ([pmd/pte]p_huge_clear_flush_notify()) + - set page table entry to point to new page + +If clearing the page table entry is not followed by a notify before setting +the new pte/pmd value then you can break memory model like C11 or C++11 for +the device. + +Consider the following scenario (device use a feature similar to ATS/PASID): + +Two address addrA and addrB such that |addrA - addrB| >= PAGE_SIZE we assume +they are write protected for COW (other case of B apply too). + +[Time N] -------------------------------------------------------------------- +CPU-thread-0 {try to write to addrA} +CPU-thread-1 {try to write to addrB} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {read addrA and populate device TLB} +DEV-thread-2 {read addrB and populate device TLB} +[Time N+1] ------------------------------------------------------------------ +CPU-thread-0 {COW_step0: {mmu_notifier_invalidate_range_start(addrA)}} +CPU-thread-1 {COW_step0: {mmu_notifier_invalidate_range_start(addrB)}} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+2] ------------------------------------------------------------------ +CPU-thread-0 {COW_step1: {update page table to point to new page for addrA}} +CPU-thread-1 {COW_step1: {update page table to point to new page for addrB}} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+3] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {preempted} +CPU-thread-2 {write to addrA which is a write to new page} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+3] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {preempted} +CPU-thread-2 {} +CPU-thread-3 {write to addrB which is a write to new page} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+4] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {COW_step3: {mmu_notifier_invalidate_range_end(addrB)}} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+5] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {read addrA from old page} +DEV-thread-2 {read addrB from new page} + +So here because at time N+2 the clear page table entry was not pair with a +notification to invalidate the secondary TLB, the device see the new value for +addrB before seing the new value for addrA. This break total memory ordering +for the device. + +When changing a pte to write protect or to point to a new write protected page +with same content (KSM) it is fine to delay the mmu_notifier_invalidate_range +call to mmu_notifier_invalidate_range_end() outside the page table lock. This +is true even if the thread doing the page table update is preempted right after +releasing page table lock but before call mmu_notifier_invalidate_range_end(). diff --git a/fs/dax.c b/fs/dax.c index f3a44a7c14b3..9ec797424e4f 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -614,6 +614,13 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, if (follow_pte_pmd(vma->vm_mm, address, &start, &end, &ptep, &pmdp, &ptl)) continue; + /* + * No need to call mmu_notifier_invalidate_range() as we are + * downgrading page table protection not changing it to point + * to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ if (pmdp) { #ifdef CONFIG_FS_DAX_PMD pmd_t pmd; @@ -628,7 +635,6 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, pmd = pmd_wrprotect(pmd); pmd = pmd_mkclean(pmd); set_pmd_at(vma->vm_mm, address, pmdp, pmd); - mmu_notifier_invalidate_range(vma->vm_mm, start, end); unlock_pmd: spin_unlock(ptl); #endif @@ -643,7 +649,6 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, pte = pte_wrprotect(pte); pte = pte_mkclean(pte); set_pte_at(vma->vm_mm, address, ptep, pte); - mmu_notifier_invalidate_range(vma->vm_mm, start, end); unlock_pte: pte_unmap_unlock(ptep, ptl); } diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 6866e8126982..49c925c96b8a 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -155,7 +155,8 @@ struct mmu_notifier_ops { * shared page-tables, it not necessary to implement the * invalidate_range_start()/end() notifiers, as * invalidate_range() alread catches the points in time when an - * external TLB range needs to be flushed. + * external TLB range needs to be flushed. For more in depth + * discussion on this see Documentation/vm/mmu_notifier.txt * * The invalidate_range() function is called under the ptl * spin-lock and not allowed to sleep. diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c037d3d34950..ff5bc647b51d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1186,8 +1186,15 @@ static int do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, pmd_t orig_pmd, goto out_free_pages; VM_BUG_ON_PAGE(!PageHead(page), page); + /* + * Leave pmd empty until pte is filled note we must notify here as + * concurrent CPU thread might write to new page before the call to + * mmu_notifier_invalidate_range_end() happens which can lead to a + * device seeing memory write in different order than CPU. + * + * See Documentation/vm/mmu_notifier.txt + */ pmdp_huge_clear_flush_notify(vma, haddr, vmf->pmd); - /* leave pmd empty until pte is filled */ pgtable = pgtable_trans_huge_withdraw(vma->vm_mm, vmf->pmd); pmd_populate(vma->vm_mm, &_pmd, pgtable); @@ -2026,8 +2033,15 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, pmd_t _pmd; int i; - /* leave pmd empty until pte is filled */ - pmdp_huge_clear_flush_notify(vma, haddr, pmd); + /* + * Leave pmd empty until pte is filled note that it is fine to delay + * notification until mmu_notifier_invalidate_range_end() as we are + * replacing a zero pmd write protected page with a zero pte write + * protected page. + * + * See Documentation/vm/mmu_notifier.txt + */ + pmdp_huge_clear_flush(vma, haddr, pmd); pgtable = pgtable_trans_huge_withdraw(mm, pmd); pmd_populate(mm, &_pmd, pgtable); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1768efa4c501..63a63f1b536c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3254,9 +3254,14 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, set_huge_swap_pte_at(dst, addr, dst_pte, entry, sz); } else { if (cow) { + /* + * No need to notify as we are downgrading page + * table protection not changing it to point + * to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ huge_ptep_set_wrprotect(src, addr, src_pte); - mmu_notifier_invalidate_range(src, mmun_start, - mmun_end); } entry = huge_ptep_get(src_pte); ptepage = pte_page(entry); @@ -4288,7 +4293,12 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * and that page table be reused and filled with junk. */ flush_hugetlb_tlb_range(vma, start, end); - mmu_notifier_invalidate_range(mm, start, end); + /* + * No need to call mmu_notifier_invalidate_range() we are downgrading + * page table protection not changing it to point to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ i_mmap_unlock_write(vma->vm_file->f_mapping); mmu_notifier_invalidate_range_end(mm, start, end); diff --git a/mm/ksm.c b/mm/ksm.c index 6cb60f46cce5..be8f4576f842 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1052,8 +1052,13 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, * So we clear the pte and flush the tlb before the check * this assure us that no O_DIRECT can happen after the check * or in the middle of the check. + * + * No need to notify as we are downgrading page table to read + * only not changing it to point to a new page. + * + * See Documentation/vm/mmu_notifier.txt */ - entry = ptep_clear_flush_notify(vma, pvmw.address, pvmw.pte); + entry = ptep_clear_flush(vma, pvmw.address, pvmw.pte); /* * Check that no O_DIRECT or similar I/O is in progress on the * page @@ -1136,7 +1141,13 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, } flush_cache_page(vma, addr, pte_pfn(*ptep)); - ptep_clear_flush_notify(vma, addr, ptep); + /* + * No need to notify as we are replacing a read only page with another + * read only page with the same content. + * + * See Documentation/vm/mmu_notifier.txt + */ + ptep_clear_flush(vma, addr, ptep); set_pte_at_notify(mm, addr, ptep, newpte); page_remove_rmap(page, false); diff --git a/mm/rmap.c b/mm/rmap.c index 061826278520..6b5a0f219ac0 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -937,10 +937,15 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, #endif } - if (ret) { - mmu_notifier_invalidate_range(vma->vm_mm, cstart, cend); + /* + * No need to call mmu_notifier_invalidate_range() as we are + * downgrading page table protection not changing it to point + * to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ + if (ret) (*cleaned)++; - } } mmu_notifier_invalidate_range_end(vma->vm_mm, start, end); @@ -1424,6 +1429,10 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); set_pte_at(mm, pvmw.address, pvmw.pte, swp_pte); + /* + * No need to invalidate here it will synchronize on + * against the special swap migration pte. + */ goto discard; } @@ -1481,6 +1490,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * will take care of the rest. */ dec_mm_counter(mm, mm_counter(page)); + /* We have to invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); } else if (IS_ENABLED(CONFIG_MIGRATION) && (flags & (TTU_MIGRATION|TTU_SPLIT_FREEZE))) { swp_entry_t entry; @@ -1496,6 +1508,10 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); + /* + * No need to invalidate here it will synchronize on + * against the special swap migration pte. + */ } else if (PageAnon(page)) { swp_entry_t entry = { .val = page_private(subpage) }; pte_t swp_pte; @@ -1507,6 +1523,8 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, WARN_ON_ONCE(1); ret = false; /* We have to invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); page_vma_mapped_walk_done(&pvmw); break; } @@ -1514,6 +1532,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, /* MADV_FREE page check */ if (!PageSwapBacked(page)) { if (!PageDirty(page)) { + /* Invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, + address, address + PAGE_SIZE); dec_mm_counter(mm, MM_ANONPAGES); goto discard; } @@ -1547,13 +1568,39 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); - } else + /* Invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); + } else { + /* + * We should not need to notify here as we reach this + * case only from freeze_page() itself only call from + * split_huge_page_to_list() so everything below must + * be true: + * - page is not anonymous + * - page is locked + * + * So as it is a locked file back page thus it can not + * be remove from the page cache and replace by a new + * page before mmu_notifier_invalidate_range_end so no + * concurrent thread might update its page table to + * point at new page while a device still is using this + * page. + * + * See Documentation/vm/mmu_notifier.txt + */ dec_mm_counter(mm, mm_counter_file(page)); + } discard: + /* + * No need to call mmu_notifier_invalidate_range() it has be + * done above for all cases requiring it to happen under page + * table lock before mmu_notifier_invalidate_range_end() + * + * See Documentation/vm/mmu_notifier.txt + */ page_remove_rmap(subpage, PageHuge(page)); put_page(page); - mmu_notifier_invalidate_range(mm, address, - address + PAGE_SIZE); } mmu_notifier_invalidate_range_end(vma->vm_mm, start, end); -- 2.13.6 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by kanga.kvack.org (Postfix) with ESMTP id C03696B0253 for ; Mon, 16 Oct 2017 23:10:17 -0400 (EDT) Received: by mail-qk0-f198.google.com with SMTP id o187so513294qke.1 for ; Mon, 16 Oct 2017 20:10:17 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id r20si4331qke.267.2017.10.16.20.10.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Oct 2017 20:10:16 -0700 (PDT) From: jglisse@redhat.com Subject: [PATCH 1/2] mm/mmu_notifier: avoid double notification when it is useless v2 Date: Mon, 16 Oct 2017 23:10:02 -0400 Message-Id: <20171017031003.7481-2-jglisse@redhat.com> In-Reply-To: <20171017031003.7481-1-jglisse@redhat.com> References: <20171017031003.7481-1-jglisse@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: owner-linux-mm@kvack.org List-ID: To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Andrea Arcangeli , Nadav Amit , Linus Torvalds , Andrew Morton , Joerg Roedel , Suravee Suthikulpanit , David Woodhouse , Alistair Popple , Michael Ellerman , Benjamin Herrenschmidt , Stephen Rothwell , Andrew Donnellan , iommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, linux-next@vger.kernel.org From: JA(C)rA'me Glisse This patch only affects users of mmu_notifier->invalidate_range callback which are device drivers related to ATS/PASID, CAPI, IOMMUv2, SVM ... and it is an optimization for those users. Everyone else is unaffected by it. When clearing a pte/pmd we are given a choice to notify the event under the page table lock (notify version of *_clear_flush helpers do call the mmu_notifier_invalidate_range). But that notification is not necessary in all cases. This patches remove almost all cases where it is useless to have a call to mmu_notifier_invalidate_range before mmu_notifier_invalidate_range_end. It also adds documentation in all those cases explaining why. Below is a more in depth analysis of why this is fine to do this: For secondary TLB (non CPU TLB) like IOMMU TLB or device TLB (when device use thing like ATS/PASID to get the IOMMU to walk the CPU page table to access a process virtual address space). There is only 2 cases when you need to notify those secondary TLB while holding page table lock when clearing a pte/pmd: A) page backing address is free before mmu_notifier_invalidate_range_end B) a page table entry is updated to point to a new page (COW, write fault on zero page, __replace_page(), ...) Case A is obvious you do not want to take the risk for the device to write to a page that might now be used by something completely different. Case B is more subtle. For correctness it requires the following sequence to happen: - take page table lock - clear page table entry and notify (pmd/pte_huge_clear_flush_notify()) - set page table entry to point to new page If clearing the page table entry is not followed by a notify before setting the new pte/pmd value then you can break memory model like C11 or C++11 for the device. Consider the following scenario (device use a feature similar to ATS/ PASID): Two address addrA and addrB such that |addrA - addrB| >= PAGE_SIZE we assume they are write protected for COW (other case of B apply too). [Time N] ----------------------------------------------------------------- CPU-thread-0 {try to write to addrA} CPU-thread-1 {try to write to addrB} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {read addrA and populate device TLB} DEV-thread-2 {read addrB and populate device TLB} [Time N+1] --------------------------------------------------------------- CPU-thread-0 {COW_step0: {mmu_notifier_invalidate_range_start(addrA)}} CPU-thread-1 {COW_step0: {mmu_notifier_invalidate_range_start(addrB)}} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+2] --------------------------------------------------------------- CPU-thread-0 {COW_step1: {update page table point to new page for addrA}} CPU-thread-1 {COW_step1: {update page table point to new page for addrB}} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+3] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {preempted} CPU-thread-2 {write to addrA which is a write to new page} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+3] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {preempted} CPU-thread-2 {} CPU-thread-3 {write to addrB which is a write to new page} DEV-thread-0 {} DEV-thread-2 {} [Time N+4] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {COW_step3: {mmu_notifier_invalidate_range_end(addrB)}} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {} DEV-thread-2 {} [Time N+5] --------------------------------------------------------------- CPU-thread-0 {preempted} CPU-thread-1 {} CPU-thread-2 {} CPU-thread-3 {} DEV-thread-0 {read addrA from old page} DEV-thread-2 {read addrB from new page} So here because at time N+2 the clear page table entry was not pair with a notification to invalidate the secondary TLB, the device see the new value for addrB before seing the new value for addrA. This break total memory ordering for the device. When changing a pte to write protect or to point to a new write protected page with same content (KSM) it is ok to delay invalidate_range callback to mmu_notifier_invalidate_range_end() outside the page table lock. This is true even if the thread doing page table update is preempted right after releasing page table lock before calling mmu_notifier_invalidate_range_end Changed since v1: - typos (thanks to Andrea) - Avoid unnecessary precaution in try_to_unmap() (Andrea) - Be more conservative in try_to_unmap_one() Signed-off-by: JA(C)rA'me Glisse Cc: Andrea Arcangeli Cc: Nadav Amit Cc: Linus Torvalds Cc: Andrew Morton Cc: Joerg Roedel Cc: Suravee Suthikulpanit Cc: David Woodhouse Cc: Alistair Popple Cc: Michael Ellerman Cc: Benjamin Herrenschmidt Cc: Stephen Rothwell Cc: Andrew Donnellan Cc: iommu@lists.linux-foundation.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-next@vger.kernel.org --- Documentation/vm/mmu_notifier.txt | 93 +++++++++++++++++++++++++++++++++++++++ fs/dax.c | 9 +++- include/linux/mmu_notifier.h | 3 +- mm/huge_memory.c | 20 +++++++-- mm/hugetlb.c | 16 +++++-- mm/ksm.c | 15 ++++++- mm/rmap.c | 59 ++++++++++++++++++++++--- 7 files changed, 198 insertions(+), 17 deletions(-) create mode 100644 Documentation/vm/mmu_notifier.txt diff --git a/Documentation/vm/mmu_notifier.txt b/Documentation/vm/mmu_notifier.txt new file mode 100644 index 000000000000..23b462566bb7 --- /dev/null +++ b/Documentation/vm/mmu_notifier.txt @@ -0,0 +1,93 @@ +When do you need to notify inside page table lock ? + +When clearing a pte/pmd we are given a choice to notify the event through +(notify version of *_clear_flush call mmu_notifier_invalidate_range) under +the page table lock. But that notification is not necessary in all cases. + +For secondary TLB (non CPU TLB) like IOMMU TLB or device TLB (when device use +thing like ATS/PASID to get the IOMMU to walk the CPU page table to access a +process virtual address space). There is only 2 cases when you need to notify +those secondary TLB while holding page table lock when clearing a pte/pmd: + + A) page backing address is free before mmu_notifier_invalidate_range_end() + B) a page table entry is updated to point to a new page (COW, write fault + on zero page, __replace_page(), ...) + +Case A is obvious you do not want to take the risk for the device to write to +a page that might now be used by some completely different task. + +Case B is more subtle. For correctness it requires the following sequence to +happen: + - take page table lock + - clear page table entry and notify ([pmd/pte]p_huge_clear_flush_notify()) + - set page table entry to point to new page + +If clearing the page table entry is not followed by a notify before setting +the new pte/pmd value then you can break memory model like C11 or C++11 for +the device. + +Consider the following scenario (device use a feature similar to ATS/PASID): + +Two address addrA and addrB such that |addrA - addrB| >= PAGE_SIZE we assume +they are write protected for COW (other case of B apply too). + +[Time N] -------------------------------------------------------------------- +CPU-thread-0 {try to write to addrA} +CPU-thread-1 {try to write to addrB} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {read addrA and populate device TLB} +DEV-thread-2 {read addrB and populate device TLB} +[Time N+1] ------------------------------------------------------------------ +CPU-thread-0 {COW_step0: {mmu_notifier_invalidate_range_start(addrA)}} +CPU-thread-1 {COW_step0: {mmu_notifier_invalidate_range_start(addrB)}} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+2] ------------------------------------------------------------------ +CPU-thread-0 {COW_step1: {update page table to point to new page for addrA}} +CPU-thread-1 {COW_step1: {update page table to point to new page for addrB}} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+3] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {preempted} +CPU-thread-2 {write to addrA which is a write to new page} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+3] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {preempted} +CPU-thread-2 {} +CPU-thread-3 {write to addrB which is a write to new page} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+4] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {COW_step3: {mmu_notifier_invalidate_range_end(addrB)}} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {} +DEV-thread-2 {} +[Time N+5] ------------------------------------------------------------------ +CPU-thread-0 {preempted} +CPU-thread-1 {} +CPU-thread-2 {} +CPU-thread-3 {} +DEV-thread-0 {read addrA from old page} +DEV-thread-2 {read addrB from new page} + +So here because at time N+2 the clear page table entry was not pair with a +notification to invalidate the secondary TLB, the device see the new value for +addrB before seing the new value for addrA. This break total memory ordering +for the device. + +When changing a pte to write protect or to point to a new write protected page +with same content (KSM) it is fine to delay the mmu_notifier_invalidate_range +call to mmu_notifier_invalidate_range_end() outside the page table lock. This +is true even if the thread doing the page table update is preempted right after +releasing page table lock but before call mmu_notifier_invalidate_range_end(). diff --git a/fs/dax.c b/fs/dax.c index f3a44a7c14b3..9ec797424e4f 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -614,6 +614,13 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, if (follow_pte_pmd(vma->vm_mm, address, &start, &end, &ptep, &pmdp, &ptl)) continue; + /* + * No need to call mmu_notifier_invalidate_range() as we are + * downgrading page table protection not changing it to point + * to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ if (pmdp) { #ifdef CONFIG_FS_DAX_PMD pmd_t pmd; @@ -628,7 +635,6 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, pmd = pmd_wrprotect(pmd); pmd = pmd_mkclean(pmd); set_pmd_at(vma->vm_mm, address, pmdp, pmd); - mmu_notifier_invalidate_range(vma->vm_mm, start, end); unlock_pmd: spin_unlock(ptl); #endif @@ -643,7 +649,6 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping, pte = pte_wrprotect(pte); pte = pte_mkclean(pte); set_pte_at(vma->vm_mm, address, ptep, pte); - mmu_notifier_invalidate_range(vma->vm_mm, start, end); unlock_pte: pte_unmap_unlock(ptep, ptl); } diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 6866e8126982..49c925c96b8a 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -155,7 +155,8 @@ struct mmu_notifier_ops { * shared page-tables, it not necessary to implement the * invalidate_range_start()/end() notifiers, as * invalidate_range() alread catches the points in time when an - * external TLB range needs to be flushed. + * external TLB range needs to be flushed. For more in depth + * discussion on this see Documentation/vm/mmu_notifier.txt * * The invalidate_range() function is called under the ptl * spin-lock and not allowed to sleep. diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c037d3d34950..ff5bc647b51d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1186,8 +1186,15 @@ static int do_huge_pmd_wp_page_fallback(struct vm_fault *vmf, pmd_t orig_pmd, goto out_free_pages; VM_BUG_ON_PAGE(!PageHead(page), page); + /* + * Leave pmd empty until pte is filled note we must notify here as + * concurrent CPU thread might write to new page before the call to + * mmu_notifier_invalidate_range_end() happens which can lead to a + * device seeing memory write in different order than CPU. + * + * See Documentation/vm/mmu_notifier.txt + */ pmdp_huge_clear_flush_notify(vma, haddr, vmf->pmd); - /* leave pmd empty until pte is filled */ pgtable = pgtable_trans_huge_withdraw(vma->vm_mm, vmf->pmd); pmd_populate(vma->vm_mm, &_pmd, pgtable); @@ -2026,8 +2033,15 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, pmd_t _pmd; int i; - /* leave pmd empty until pte is filled */ - pmdp_huge_clear_flush_notify(vma, haddr, pmd); + /* + * Leave pmd empty until pte is filled note that it is fine to delay + * notification until mmu_notifier_invalidate_range_end() as we are + * replacing a zero pmd write protected page with a zero pte write + * protected page. + * + * See Documentation/vm/mmu_notifier.txt + */ + pmdp_huge_clear_flush(vma, haddr, pmd); pgtable = pgtable_trans_huge_withdraw(mm, pmd); pmd_populate(mm, &_pmd, pgtable); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1768efa4c501..63a63f1b536c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3254,9 +3254,14 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, set_huge_swap_pte_at(dst, addr, dst_pte, entry, sz); } else { if (cow) { + /* + * No need to notify as we are downgrading page + * table protection not changing it to point + * to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ huge_ptep_set_wrprotect(src, addr, src_pte); - mmu_notifier_invalidate_range(src, mmun_start, - mmun_end); } entry = huge_ptep_get(src_pte); ptepage = pte_page(entry); @@ -4288,7 +4293,12 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, * and that page table be reused and filled with junk. */ flush_hugetlb_tlb_range(vma, start, end); - mmu_notifier_invalidate_range(mm, start, end); + /* + * No need to call mmu_notifier_invalidate_range() we are downgrading + * page table protection not changing it to point to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ i_mmap_unlock_write(vma->vm_file->f_mapping); mmu_notifier_invalidate_range_end(mm, start, end); diff --git a/mm/ksm.c b/mm/ksm.c index 6cb60f46cce5..be8f4576f842 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1052,8 +1052,13 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, * So we clear the pte and flush the tlb before the check * this assure us that no O_DIRECT can happen after the check * or in the middle of the check. + * + * No need to notify as we are downgrading page table to read + * only not changing it to point to a new page. + * + * See Documentation/vm/mmu_notifier.txt */ - entry = ptep_clear_flush_notify(vma, pvmw.address, pvmw.pte); + entry = ptep_clear_flush(vma, pvmw.address, pvmw.pte); /* * Check that no O_DIRECT or similar I/O is in progress on the * page @@ -1136,7 +1141,13 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, } flush_cache_page(vma, addr, pte_pfn(*ptep)); - ptep_clear_flush_notify(vma, addr, ptep); + /* + * No need to notify as we are replacing a read only page with another + * read only page with the same content. + * + * See Documentation/vm/mmu_notifier.txt + */ + ptep_clear_flush(vma, addr, ptep); set_pte_at_notify(mm, addr, ptep, newpte); page_remove_rmap(page, false); diff --git a/mm/rmap.c b/mm/rmap.c index 061826278520..6b5a0f219ac0 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -937,10 +937,15 @@ static bool page_mkclean_one(struct page *page, struct vm_area_struct *vma, #endif } - if (ret) { - mmu_notifier_invalidate_range(vma->vm_mm, cstart, cend); + /* + * No need to call mmu_notifier_invalidate_range() as we are + * downgrading page table protection not changing it to point + * to a new page. + * + * See Documentation/vm/mmu_notifier.txt + */ + if (ret) (*cleaned)++; - } } mmu_notifier_invalidate_range_end(vma->vm_mm, start, end); @@ -1424,6 +1429,10 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); set_pte_at(mm, pvmw.address, pvmw.pte, swp_pte); + /* + * No need to invalidate here it will synchronize on + * against the special swap migration pte. + */ goto discard; } @@ -1481,6 +1490,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * will take care of the rest. */ dec_mm_counter(mm, mm_counter(page)); + /* We have to invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); } else if (IS_ENABLED(CONFIG_MIGRATION) && (flags & (TTU_MIGRATION|TTU_SPLIT_FREEZE))) { swp_entry_t entry; @@ -1496,6 +1508,10 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); + /* + * No need to invalidate here it will synchronize on + * against the special swap migration pte. + */ } else if (PageAnon(page)) { swp_entry_t entry = { .val = page_private(subpage) }; pte_t swp_pte; @@ -1507,6 +1523,8 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, WARN_ON_ONCE(1); ret = false; /* We have to invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); page_vma_mapped_walk_done(&pvmw); break; } @@ -1514,6 +1532,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, /* MADV_FREE page check */ if (!PageSwapBacked(page)) { if (!PageDirty(page)) { + /* Invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, + address, address + PAGE_SIZE); dec_mm_counter(mm, MM_ANONPAGES); goto discard; } @@ -1547,13 +1568,39 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); - } else + /* Invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); + } else { + /* + * We should not need to notify here as we reach this + * case only from freeze_page() itself only call from + * split_huge_page_to_list() so everything below must + * be true: + * - page is not anonymous + * - page is locked + * + * So as it is a locked file back page thus it can not + * be remove from the page cache and replace by a new + * page before mmu_notifier_invalidate_range_end so no + * concurrent thread might update its page table to + * point at new page while a device still is using this + * page. + * + * See Documentation/vm/mmu_notifier.txt + */ dec_mm_counter(mm, mm_counter_file(page)); + } discard: + /* + * No need to call mmu_notifier_invalidate_range() it has be + * done above for all cases requiring it to happen under page + * table lock before mmu_notifier_invalidate_range_end() + * + * See Documentation/vm/mmu_notifier.txt + */ page_remove_rmap(subpage, PageHuge(page)); put_page(page); - mmu_notifier_invalidate_range(mm, address, - address + PAGE_SIZE); } mmu_notifier_invalidate_range_end(vma->vm_mm, start, end); -- 2.13.6 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org