From mboxrd@z Thu Jan 1 00:00:00 1970 From: "ira.weiny--- via dri-devel" Subject: [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool' Date: Wed, 13 Feb 2019 15:04:51 -0800 Message-ID: <20190213230455.5605-4-ira.weiny@intel.com> References: <20190211201643.7599-1-ira.weiny@intel.com> <20190213230455.5605-1-ira.weiny@intel.com> Reply-To: ira.weiny@intel.com Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20190213230455.5605-1-ira.weiny@intel.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, kvm@vger.kernel.org, linux-fpga@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-media@vger.kernel.org, linux-scsi@vger.kernel.org, devel@driverdev.osuosl.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-fbdev@vger.kernel.org, xen-devel@lists.xenproject.org, devel@lists.orangefs.org, linux-mm@kvack.org, ceph-devel@vger.kernel.org, rds-devel@oss.oracle.com Cc: Martin Brandenburg , Rich Felker , David Hildenbrand , David Airlie , Jason Wang , =?UTF-8?q?Kai=20M=C3=A4kisara?= , Michal Hocko , Ira Weiny , Ben Chan , Rob Springer , Todd Poynor , Yoshinori Sato , "Michael S. Tsirkin" , Jason Gunthorpe , Ingo Molnar , Matt Porter , Wu Hao , Alan Tull , John Hubbard , "James E.J. Bottomley" , Alex Williamson , Stefano Stabellini , Borislav Petkov , Alexander Viro List-Id: ceph-devel.vger.kernel.org RnJvbTogSXJhIFdlaW55IDxpcmEud2VpbnlAaW50ZWwuY29tPgoKVG8gZmFjaWxpdGF0ZSBhZGRp dGlvbmFsIG9wdGlvbnMgdG8gZ2V0X3VzZXJfcGFnZXNfZmFzdCgpIGNoYW5nZSB0aGUKc2luZ3Vs YXIgd3JpdGUgcGFyYW1ldGVyIHRvIGJlIGd1cF9mbGFncy4KClRoaXMgcGF0Y2ggZG9lcyBub3Qg Y2hhbmdlIGFueSBmdW5jdGlvbmFsaXR5LiAgTmV3IGZ1bmN0aW9uYWxpdHkgd2lsbApmb2xsb3cg aW4gc3Vic2VxdWVudCBwYXRjaGVzLgoKU29tZSBvZiB0aGUgZ2V0X3VzZXJfcGFnZXNfZmFzdCgp IGNhbGwgc2l0ZXMgd2VyZSB1bmNoYW5nZWQgYmVjYXVzZSB0aGV5CmFscmVhZHkgcGFzc2VkIEZP TExfV1JJVEUgb3IgMCBmb3IgdGhlIHdyaXRlIHBhcmFtZXRlci4KClNpZ25lZC1vZmYtYnk6IEly YSBXZWlueSA8aXJhLndlaW55QGludGVsLmNvbT4KLS0tCiBhcmNoL21pcHMvbW0vZ3VwLmMgICAg ICAgICAgICAgICAgICAgICAgICAgfCAxMSArKysrKystLS0tLQogYXJjaC9wb3dlcnBjL2t2bS9i b29rM3NfNjRfbW11X2h2LmMgICAgICAgIHwgIDQgKystLQogYXJjaC9wb3dlcnBjL2t2bS9lNTAw X21tdS5jICAgICAgICAgICAgICAgIHwgIDIgKy0KIGFyY2gvcG93ZXJwYy9tbS9tbXVfY29udGV4 dF9pb21tdS5jICAgICAgICB8ICA0ICsrLS0KIGFyY2gvczM5MC9rdm0vaW50ZXJydXB0LmMgICAg ICAgICAgICAgICAgICB8ICAyICstCiBhcmNoL3MzOTAvbW0vZ3VwLmMgICAgICAgICAgICAgICAg ICAgICAgICAgfCAxMiArKysrKystLS0tLS0KIGFyY2gvc2gvbW0vZ3VwLmMgICAgICAgICAgICAg ICAgICAgICAgICAgICB8IDExICsrKysrKy0tLS0tCiBhcmNoL3NwYXJjL21tL2d1cC5jICAgICAg ICAgICAgICAgICAgICAgICAgfCAgOSArKysrKy0tLS0KIGFyY2gveDg2L2t2bS9wYWdpbmdfdG1w bC5oICAgICAgICAgICAgICAgICB8ICAyICstCiBhcmNoL3g4Ni9rdm0vc3ZtLmMgICAgICAgICAg ICAgICAgICAgICAgICAgfCAgMiArLQogZHJpdmVycy9mcGdhL2RmbC1hZnUtZG1hLXJlZ2lvbi5j ICAgICAgICAgIHwgIDIgKy0KIGRyaXZlcnMvZ3B1L2RybS92aWEvdmlhX2RtYWJsaXQuYyAgICAg ICAgICB8ICAzICsrLQogZHJpdmVycy9pbmZpbmliYW5kL2h3L2hmaTEvdXNlcl9wYWdlcy5jICAg IHwgIDMgKystCiBkcml2ZXJzL21pc2MvZ2Vud3FlL2NhcmRfdXRpbHMuYyAgICAgICAgICAgfCAg MiArLQogZHJpdmVycy9taXNjL3Ztd192bWNpL3ZtY2lfaG9zdC5jICAgICAgICAgIHwgIDIgKy0K IGRyaXZlcnMvbWlzYy92bXdfdm1jaS92bWNpX3F1ZXVlX3BhaXIuYyAgICB8ICA2ICsrKystLQog ZHJpdmVycy9wbGF0Zm9ybS9nb2xkZmlzaC9nb2xkZmlzaF9waXBlLmMgIHwgIDMgKystCiBkcml2 ZXJzL3JhcGlkaW8vZGV2aWNlcy9yaW9fbXBvcnRfY2Rldi5jICAgfCAgNCArKystCiBkcml2ZXJz L3NidXMvY2hhci9vcmFkYXguYyAgICAgICAgICAgICAgICAgfCAgMiArLQogZHJpdmVycy9zY3Np L3N0LmMgICAgICAgICAgICAgICAgICAgICAgICAgIHwgIDMgKystCiBkcml2ZXJzL3N0YWdpbmcv Z2Fza2V0L2dhc2tldF9wYWdlX3RhYmxlLmMgfCAgNCArKy0tCiBkcml2ZXJzL3RlZS90ZWVfc2ht LmMgICAgICAgICAgICAgICAgICAgICAgfCAgMiArLQogZHJpdmVycy92ZmlvL3ZmaW9faW9tbXVf c3BhcHJfdGNlLmMgICAgICAgIHwgIDMgKystCiBkcml2ZXJzL3Zob3N0L3Zob3N0LmMgICAgICAg ICAgICAgICAgICAgICAgfCAgMiArLQogZHJpdmVycy92aWRlby9mYmRldi9wdnIyZmIuYyAgICAg ICAgICAgICAgIHwgIDIgKy0KIGRyaXZlcnMvdmlydC9mc2xfaHlwZXJ2aXNvci5jICAgICAgICAg ICAgICB8ICAyICstCiBkcml2ZXJzL3hlbi9nbnRkZXYuYyAgICAgICAgICAgICAgICAgICAgICAg fCAgMiArLQogZnMvb3JhbmdlZnMvb3JhbmdlZnMtYnVmbWFwLmMgICAgICAgICAgICAgIHwgIDIg Ky0KIGluY2x1ZGUvbGludXgvbW0uaCAgICAgICAgICAgICAgICAgICAgICAgICB8ICA0ICsrLS0K IGtlcm5lbC9mdXRleC5jICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAyICstCiBsaWIv aW92X2l0ZXIuYyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgNyArKysrKy0tCiBtbS9n dXAuYyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAxMCArKysrKy0tLS0tCiBt bS91dGlsLmMgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgOCArKysrLS0tLQog bmV0L2NlcGgvcGFnZXZlYy5jICAgICAgICAgICAgICAgICAgICAgICAgIHwgIDIgKy0KIG5ldC9y ZHMvaW5mby5jICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAyICstCiBuZXQvcmRzL3Jk bWEuYyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgMyArKy0KIDM2IGZpbGVzIGNoYW5n ZWQsIDgxIGluc2VydGlvbnMoKyksIDY1IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2FyY2gv bWlwcy9tbS9ndXAuYyBiL2FyY2gvbWlwcy9tbS9ndXAuYwppbmRleCAwZDE0ZTBkOGVhY2YuLjRj MmI0NDgzNjgzYyAxMDA2NDQKLS0tIGEvYXJjaC9taXBzL21tL2d1cC5jCisrKyBiL2FyY2gvbWlw cy9tbS9ndXAuYwpAQCAtMjM1LDcgKzIzNSw3IEBAIGludCBfX2dldF91c2VyX3BhZ2VzX2Zhc3Qo dW5zaWduZWQgbG9uZyBzdGFydCwgaW50IG5yX3BhZ2VzLCBpbnQgd3JpdGUsCiAgKiBnZXRfdXNl cl9wYWdlc19mYXN0KCkgLSBwaW4gdXNlciBwYWdlcyBpbiBtZW1vcnkKICAqIEBzdGFydDoJc3Rh cnRpbmcgdXNlciBhZGRyZXNzCiAgKiBAbnJfcGFnZXM6CW51bWJlciBvZiBwYWdlcyBmcm9tIHN0 YXJ0IHRvIHBpbgotICogQHdyaXRlOgl3aGV0aGVyIHBhZ2VzIHdpbGwgYmUgd3JpdHRlbiB0bwor ICogQGd1cF9mbGFnczoJZmxhZ3MgbW9kaWZ5aW5nIHBpbiBiZWhhdmlvdXIKICAqIEBwYWdlczoJ YXJyYXkgdGhhdCByZWNlaXZlcyBwb2ludGVycyB0byB0aGUgcGFnZXMgcGlubmVkLgogICoJCVNo b3VsZCBiZSBhdCBsZWFzdCBucl9wYWdlcyBsb25nLgogICoKQEAgLTI0Nyw4ICsyNDcsOCBAQCBp bnQgX19nZXRfdXNlcl9wYWdlc19mYXN0KHVuc2lnbmVkIGxvbmcgc3RhcnQsIGludCBucl9wYWdl cywgaW50IHdyaXRlLAogICogcmVxdWVzdGVkLiBJZiBucl9wYWdlcyBpcyAwIG9yIG5lZ2F0aXZl LCByZXR1cm5zIDAuIElmIG5vIHBhZ2VzCiAgKiB3ZXJlIHBpbm5lZCwgcmV0dXJucyAtZXJybm8u CiAgKi8KLWludCBnZXRfdXNlcl9wYWdlc19mYXN0KHVuc2lnbmVkIGxvbmcgc3RhcnQsIGludCBu cl9wYWdlcywgaW50IHdyaXRlLAotCQkJc3RydWN0IHBhZ2UgKipwYWdlcykKK2ludCBnZXRfdXNl cl9wYWdlc19mYXN0KHVuc2lnbmVkIGxvbmcgc3RhcnQsIGludCBucl9wYWdlcywKKwkJCXVuc2ln bmVkIGludCBndXBfZmxhZ3MsIHN0cnVjdCBwYWdlICoqcGFnZXMpCiB7CiAJc3RydWN0IG1tX3N0 cnVjdCAqbW0gPSBjdXJyZW50LT5tbTsKIAl1bnNpZ25lZCBsb25nIGFkZHIsIGxlbiwgZW5kOwpA QCAtMjczLDcgKzI3Myw4IEBAIGludCBnZXRfdXNlcl9wYWdlc19mYXN0KHVuc2lnbmVkIGxvbmcg c3RhcnQsIGludCBucl9wYWdlcywgaW50IHdyaXRlLAogCQluZXh0ID0gcGdkX2FkZHJfZW5kKGFk ZHIsIGVuZCk7CiAJCWlmIChwZ2Rfbm9uZShwZ2QpKQogCQkJZ290byBzbG93OwotCQlpZiAoIWd1 cF9wdWRfcmFuZ2UocGdkLCBhZGRyLCBuZXh0LCB3cml0ZSwgcGFnZXMsICZucikpCisJCWlmICgh Z3VwX3B1ZF9yYW5nZShwZ2QsIGFkZHIsIG5leHQsIGd1cF9mbGFncyAmIEZPTExfV1JJVEUsCisJ CQkJICAgcGFnZXMsICZucikpCiAJCQlnb3RvIHNsb3c7CiAJfSB3aGlsZSAocGdkcCsrLCBhZGRy ID0gbmV4dCwgYWRkciAhPSBlbmQpOwogCWxvY2FsX2lycV9lbmFibGUoKTsKQEAgLTI4OSw3ICsy OTAsNyBAQCBpbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQg bnJfcGFnZXMsIGludCB3cml0ZSwKIAlwYWdlcyArPSBucjsKIAogCXJldCA9IGdldF91c2VyX3Bh Z2VzX3VubG9ja2VkKHN0YXJ0LCAoZW5kIC0gc3RhcnQpID4+IFBBR0VfU0hJRlQsCi0JCQkJICAg ICAgcGFnZXMsIHdyaXRlID8gRk9MTF9XUklURSA6IDApOworCQkJCSAgICAgIHBhZ2VzLCBndXBf ZmxhZ3MpOwogCiAJLyogSGF2ZSB0byBiZSBhIGJpdCBjYXJlZnVsIHdpdGggcmV0dXJuIHZhbHVl cyAqLwogCWlmIChuciA+IDApIHsKZGlmZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9rdm0vYm9vazNz XzY0X21tdV9odi5jIGIvYXJjaC9wb3dlcnBjL2t2bS9ib29rM3NfNjRfbW11X2h2LmMKaW5kZXgg YmQyZGNmYmYwMGNkLi44ZmNiMGE5MjFlNDYgMTAwNjQ0Ci0tLSBhL2FyY2gvcG93ZXJwYy9rdm0v Ym9vazNzXzY0X21tdV9odi5jCisrKyBiL2FyY2gvcG93ZXJwYy9rdm0vYm9vazNzXzY0X21tdV9o di5jCkBAIC01ODIsNyArNTgyLDcgQEAgaW50IGt2bXBwY19ib29rM3NfaHZfcGFnZV9mYXVsdChz dHJ1Y3Qga3ZtX3J1biAqcnVuLCBzdHJ1Y3Qga3ZtX3ZjcHUgKnZjcHUsCiAJLyogSWYgd3JpdGlu ZyAhPSAwLCB0aGVuIHRoZSBIUFRFIG11c3QgYWxsb3cgd3JpdGluZywgaWYgd2UgZ2V0IGhlcmUg Ki8KIAl3cml0ZV9vayA9IHdyaXRpbmc7CiAJaHZhID0gZ2ZuX3RvX2h2YV9tZW1zbG90KG1lbXNs b3QsIGdmbik7Ci0JbnBhZ2VzID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChodmEsIDEsIHdyaXRpbmcs IHBhZ2VzKTsKKwlucGFnZXMgPSBnZXRfdXNlcl9wYWdlc19mYXN0KGh2YSwgMSwgd3JpdGluZyA/ IEZPTExfV1JJVEUgOiAwLCBwYWdlcyk7CiAJaWYgKG5wYWdlcyA8IDEpIHsKIAkJLyogQ2hlY2sg aWYgaXQncyBhbiBJL08gbWFwcGluZyAqLwogCQlkb3duX3JlYWQoJmN1cnJlbnQtPm1tLT5tbWFw X3NlbSk7CkBAIC0xMTc1LDcgKzExNzUsNyBAQCB2b2lkICprdm1wcGNfcGluX2d1ZXN0X3BhZ2Uo c3RydWN0IGt2bSAqa3ZtLCB1bnNpZ25lZCBsb25nIGdwYSwKIAlpZiAoIW1lbXNsb3QgfHwgKG1l bXNsb3QtPmZsYWdzICYgS1ZNX01FTVNMT1RfSU5WQUxJRCkpCiAJCWdvdG8gZXJyOwogCWh2YSA9 IGdmbl90b19odmFfbWVtc2xvdChtZW1zbG90LCBnZm4pOwotCW5wYWdlcyA9IGdldF91c2VyX3Bh Z2VzX2Zhc3QoaHZhLCAxLCAxLCBwYWdlcyk7CisJbnBhZ2VzID0gZ2V0X3VzZXJfcGFnZXNfZmFz dChodmEsIDEsIEZPTExfV1JJVEUsIHBhZ2VzKTsKIAlpZiAobnBhZ2VzIDwgMSkKIAkJZ290byBl cnI7CiAJcGFnZSA9IHBhZ2VzWzBdOwpkaWZmIC0tZ2l0IGEvYXJjaC9wb3dlcnBjL2t2bS9lNTAw X21tdS5jIGIvYXJjaC9wb3dlcnBjL2t2bS9lNTAwX21tdS5jCmluZGV4IDI0Mjk2ZjRjYWRjNi4u ZTBhZjUzZmQ3OGM1IDEwMDY0NAotLS0gYS9hcmNoL3Bvd2VycGMva3ZtL2U1MDBfbW11LmMKKysr IGIvYXJjaC9wb3dlcnBjL2t2bS9lNTAwX21tdS5jCkBAIC03ODMsNyArNzgzLDcgQEAgaW50IGt2 bV92Y3B1X2lvY3RsX2NvbmZpZ190bGIoc3RydWN0IGt2bV92Y3B1ICp2Y3B1LAogCWlmICghcGFn ZXMpCiAJCXJldHVybiAtRU5PTUVNOwogCi0JcmV0ID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChjZmct PmFycmF5LCBudW1fcGFnZXMsIDEsIHBhZ2VzKTsKKwlyZXQgPSBnZXRfdXNlcl9wYWdlc19mYXN0 KGNmZy0+YXJyYXksIG51bV9wYWdlcywgRk9MTF9XUklURSwgcGFnZXMpOwogCWlmIChyZXQgPCAw KQogCQlnb3RvIGZyZWVfcGFnZXM7CiAKZGlmZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9tbS9tbXVf Y29udGV4dF9pb21tdS5jIGIvYXJjaC9wb3dlcnBjL21tL21tdV9jb250ZXh0X2lvbW11LmMKaW5k ZXggYTcxMmE2NTBhOGI2Li5hY2IwOTkwYzgzNjQgMTAwNjQ0Ci0tLSBhL2FyY2gvcG93ZXJwYy9t bS9tbXVfY29udGV4dF9pb21tdS5jCisrKyBiL2FyY2gvcG93ZXJwYy9tbS9tbXVfY29udGV4dF9p b21tdS5jCkBAIC0xOTAsNyArMTkwLDcgQEAgc3RhdGljIGxvbmcgbW1faW9tbXVfZG9fYWxsb2Mo c3RydWN0IG1tX3N0cnVjdCAqbW0sIHVuc2lnbmVkIGxvbmcgdWEsCiAJZm9yIChpID0gMDsgaSA8 IGVudHJpZXM7ICsraSkgewogCQljdXJfdWEgPSB1YSArIChpIDw8IFBBR0VfU0hJRlQpOwogCQlp ZiAoMSAhPSBnZXRfdXNlcl9wYWdlc19mYXN0KGN1cl91YSwKLQkJCQkJMS8qIHBhZ2VzICovLCAx LyogaXN3cml0ZSAqLywgJnBhZ2UpKSB7CisJCQkJCTEvKiBwYWdlcyAqLywgRk9MTF9XUklURSwg JnBhZ2UpKSB7CiAJCQlyZXQgPSAtRUZBVUxUOwogCQkJZm9yIChqID0gMDsgaiA8IGk7ICsraikK IAkJCQlwdXRfcGFnZShwZm5fdG9fcGFnZShtZW0tPmhwYXNbal0gPj4KQEAgLTIwOSw3ICsyMDks NyBAQCBzdGF0aWMgbG9uZyBtbV9pb21tdV9kb19hbGxvYyhzdHJ1Y3QgbW1fc3RydWN0ICptbSwg dW5zaWduZWQgbG9uZyB1YSwKIAkJCWlmIChtbV9pb21tdV9tb3ZlX3BhZ2VfZnJvbV9jbWEocGFn ZSkpCiAJCQkJZ290byBwb3B1bGF0ZTsKIAkJCWlmICgxICE9IGdldF91c2VyX3BhZ2VzX2Zhc3Qo Y3VyX3VhLAotCQkJCQkJMS8qIHBhZ2VzICovLCAxLyogaXN3cml0ZSAqLywKKwkJCQkJCTEvKiBw YWdlcyAqLywgRk9MTF9XUklURSwKIAkJCQkJCSZwYWdlKSkgewogCQkJCXJldCA9IC1FRkFVTFQ7 CiAJCQkJZm9yIChqID0gMDsgaiA8IGk7ICsraikKZGlmZiAtLWdpdCBhL2FyY2gvczM5MC9rdm0v aW50ZXJydXB0LmMgYi9hcmNoL3MzOTAva3ZtL2ludGVycnVwdC5jCmluZGV4IGZjYjU1YjAyOTkw ZS4uNjlkOTM2NmI5NjZjIDEwMDY0NAotLS0gYS9hcmNoL3MzOTAva3ZtL2ludGVycnVwdC5jCisr KyBiL2FyY2gvczM5MC9rdm0vaW50ZXJydXB0LmMKQEAgLTIyNzgsNyArMjI3OCw3IEBAIHN0YXRp YyBpbnQga3ZtX3MzOTBfYWRhcHRlcl9tYXAoc3RydWN0IGt2bSAqa3ZtLCB1bnNpZ25lZCBpbnQg aWQsIF9fdTY0IGFkZHIpCiAJCXJldCA9IC1FRkFVTFQ7CiAJCWdvdG8gb3V0OwogCX0KLQlyZXQg PSBnZXRfdXNlcl9wYWdlc19mYXN0KG1hcC0+YWRkciwgMSwgMSwgJm1hcC0+cGFnZSk7CisJcmV0 ID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChtYXAtPmFkZHIsIDEsIEZPTExfV1JJVEUsICZtYXAtPnBh Z2UpOwogCWlmIChyZXQgPCAwKQogCQlnb3RvIG91dDsKIAlCVUdfT04ocmV0ICE9IDEpOwpkaWZm IC0tZ2l0IGEvYXJjaC9zMzkwL21tL2d1cC5jIGIvYXJjaC9zMzkwL21tL2d1cC5jCmluZGV4IDI4 MDlkMTFjN2EyOC4uMGE2ZmFmM2Q5OTYwIDEwMDY0NAotLS0gYS9hcmNoL3MzOTAvbW0vZ3VwLmMK KysrIGIvYXJjaC9zMzkwL21tL2d1cC5jCkBAIC0yNjUsNyArMjY1LDcgQEAgaW50IF9fZ2V0X3Vz ZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsIGludCB3cml0 ZSwKICAqIGdldF91c2VyX3BhZ2VzX2Zhc3QoKSAtIHBpbiB1c2VyIHBhZ2VzIGluIG1lbW9yeQog ICogQHN0YXJ0OglzdGFydGluZyB1c2VyIGFkZHJlc3MKICAqIEBucl9wYWdlczoJbnVtYmVyIG9m IHBhZ2VzIGZyb20gc3RhcnQgdG8gcGluCi0gKiBAd3JpdGU6CXdoZXRoZXIgcGFnZXMgd2lsbCBi ZSB3cml0dGVuIHRvCisgKiBAZ3VwX2ZsYWdzOglmbGFncyBtb2RpZnlpbmcgcGluIGJlaGF2aW91 cgogICogQHBhZ2VzOglhcnJheSB0aGF0IHJlY2VpdmVzIHBvaW50ZXJzIHRvIHRoZSBwYWdlcyBw aW5uZWQuCiAgKgkJU2hvdWxkIGJlIGF0IGxlYXN0IG5yX3BhZ2VzIGxvbmcuCiAgKgpAQCAtMjc3 LDIyICsyNzcsMjIgQEAgaW50IF9fZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0 YXJ0LCBpbnQgbnJfcGFnZXMsIGludCB3cml0ZSwKICAqIHJlcXVlc3RlZC4gSWYgbnJfcGFnZXMg aXMgMCBvciBuZWdhdGl2ZSwgcmV0dXJucyAwLiBJZiBubyBwYWdlcwogICogd2VyZSBwaW5uZWQs IHJldHVybnMgLWVycm5vLgogICovCi1pbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBs b25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsIGludCB3cml0ZSwKLQkJCXN0cnVjdCBwYWdlICoqcGFn ZXMpCitpbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJf cGFnZXMsCisJCQl1bnNpZ25lZCBpbnQgZ3VwX2ZsYWdzLCBzdHJ1Y3QgcGFnZSAqKnBhZ2VzKQog ewogCWludCBuciwgcmV0OwogCiAJbWlnaHRfc2xlZXAoKTsKIAlzdGFydCAmPSBQQUdFX01BU0s7 Ci0JbnIgPSBfX2dldF91c2VyX3BhZ2VzX2Zhc3Qoc3RhcnQsIG5yX3BhZ2VzLCB3cml0ZSwgcGFn ZXMpOworCW5yID0gX19nZXRfdXNlcl9wYWdlc19mYXN0KHN0YXJ0LCBucl9wYWdlcywgZ3VwX2Zs YWdzICYgRk9MTF9XUklURSwKKwkJCQkgICBwYWdlcyk7CiAJaWYgKG5yID09IG5yX3BhZ2VzKQog CQlyZXR1cm4gbnI7CiAKIAkvKiBUcnkgdG8gZ2V0IHRoZSByZW1haW5pbmcgcGFnZXMgd2l0aCBn ZXRfdXNlcl9wYWdlcyAqLwogCXN0YXJ0ICs9IG5yIDw8IFBBR0VfU0hJRlQ7CiAJcGFnZXMgKz0g bnI7Ci0JcmV0ID0gZ2V0X3VzZXJfcGFnZXNfdW5sb2NrZWQoc3RhcnQsIG5yX3BhZ2VzIC0gbnIs IHBhZ2VzLAotCQkJCSAgICAgIHdyaXRlID8gRk9MTF9XUklURSA6IDApOworCXJldCA9IGdldF91 c2VyX3BhZ2VzX3VubG9ja2VkKHN0YXJ0LCBucl9wYWdlcyAtIG5yLCBwYWdlcywgZ3VwX2ZsYWdz KTsKIAkvKiBIYXZlIHRvIGJlIGEgYml0IGNhcmVmdWwgd2l0aCByZXR1cm4gdmFsdWVzICovCiAJ aWYgKG5yID4gMCkKIAkJcmV0ID0gKHJldCA8IDApID8gbnIgOiByZXQgKyBucjsKZGlmZiAtLWdp dCBhL2FyY2gvc2gvbW0vZ3VwLmMgYi9hcmNoL3NoL21tL2d1cC5jCmluZGV4IDNlMjdmNmQxZjFl Yy4uMjc3Yzg4MmY3NDg5IDEwMDY0NAotLS0gYS9hcmNoL3NoL21tL2d1cC5jCisrKyBiL2FyY2gv c2gvbW0vZ3VwLmMKQEAgLTIwNCw3ICsyMDQsNyBAQCBpbnQgX19nZXRfdXNlcl9wYWdlc19mYXN0 KHVuc2lnbmVkIGxvbmcgc3RhcnQsIGludCBucl9wYWdlcywgaW50IHdyaXRlLAogICogZ2V0X3Vz ZXJfcGFnZXNfZmFzdCgpIC0gcGluIHVzZXIgcGFnZXMgaW4gbWVtb3J5CiAgKiBAc3RhcnQ6CXN0 YXJ0aW5nIHVzZXIgYWRkcmVzcwogICogQG5yX3BhZ2VzOgludW1iZXIgb2YgcGFnZXMgZnJvbSBz dGFydCB0byBwaW4KLSAqIEB3cml0ZToJd2hldGhlciBwYWdlcyB3aWxsIGJlIHdyaXR0ZW4gdG8K KyAqIEBndXBfZmxhZ3M6CWZsYWdzIG1vZGlmeWluZyBwaW4gYmVoYXZpb3VyCiAgKiBAcGFnZXM6 CWFycmF5IHRoYXQgcmVjZWl2ZXMgcG9pbnRlcnMgdG8gdGhlIHBhZ2VzIHBpbm5lZC4KICAqCQlT aG91bGQgYmUgYXQgbGVhc3QgbnJfcGFnZXMgbG9uZy4KICAqCkBAIC0yMTYsOCArMjE2LDggQEAg aW50IF9fZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJfcGFn ZXMsIGludCB3cml0ZSwKICAqIHJlcXVlc3RlZC4gSWYgbnJfcGFnZXMgaXMgMCBvciBuZWdhdGl2 ZSwgcmV0dXJucyAwLiBJZiBubyBwYWdlcwogICogd2VyZSBwaW5uZWQsIHJldHVybnMgLWVycm5v LgogICovCi1pbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQg bnJfcGFnZXMsIGludCB3cml0ZSwKLQkJCXN0cnVjdCBwYWdlICoqcGFnZXMpCitpbnQgZ2V0X3Vz ZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsCisJCQl1bnNp Z25lZCBpbnQgZ3VwX2ZsYWdzLCBzdHJ1Y3QgcGFnZSAqKnBhZ2VzKQogewogCXN0cnVjdCBtbV9z dHJ1Y3QgKm1tID0gY3VycmVudC0+bW07CiAJdW5zaWduZWQgbG9uZyBhZGRyLCBsZW4sIGVuZDsK QEAgLTI0MSw3ICsyNDEsOCBAQCBpbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25n IHN0YXJ0LCBpbnQgbnJfcGFnZXMsIGludCB3cml0ZSwKIAkJbmV4dCA9IHBnZF9hZGRyX2VuZChh ZGRyLCBlbmQpOwogCQlpZiAocGdkX25vbmUocGdkKSkKIAkJCWdvdG8gc2xvdzsKLQkJaWYgKCFn dXBfcHVkX3JhbmdlKHBnZCwgYWRkciwgbmV4dCwgd3JpdGUsIHBhZ2VzLCAmbnIpKQorCQlpZiAo IWd1cF9wdWRfcmFuZ2UocGdkLCBhZGRyLCBuZXh0LCBndXBfZmxhZ3MgJiBGT0xMX1dSSVRFLAor CQkJCSAgIHBhZ2VzLCAmbnIpKQogCQkJZ290byBzbG93OwogCX0gd2hpbGUgKHBnZHArKywgYWRk ciA9IG5leHQsIGFkZHIgIT0gZW5kKTsKIAlsb2NhbF9pcnFfZW5hYmxlKCk7CkBAIC0yNjEsNyAr MjYyLDcgQEAgaW50IGdldF91c2VyX3BhZ2VzX2Zhc3QodW5zaWduZWQgbG9uZyBzdGFydCwgaW50 IG5yX3BhZ2VzLCBpbnQgd3JpdGUsCiAKIAkJcmV0ID0gZ2V0X3VzZXJfcGFnZXNfdW5sb2NrZWQo c3RhcnQsCiAJCQkoZW5kIC0gc3RhcnQpID4+IFBBR0VfU0hJRlQsIHBhZ2VzLAotCQkJd3JpdGUg PyBGT0xMX1dSSVRFIDogMCk7CisJCQlndXBfZmxhZ3MpOwogCiAJCS8qIEhhdmUgdG8gYmUgYSBi aXQgY2FyZWZ1bCB3aXRoIHJldHVybiB2YWx1ZXMgKi8KIAkJaWYgKG5yID4gMCkgewpkaWZmIC0t Z2l0IGEvYXJjaC9zcGFyYy9tbS9ndXAuYyBiL2FyY2gvc3BhcmMvbW0vZ3VwLmMKaW5kZXggYWVl NmRiYTgzZDBlLi4xZTc3MGE1MTdkNGEgMTAwNjQ0Ci0tLSBhL2FyY2gvc3BhcmMvbW0vZ3VwLmMK KysrIGIvYXJjaC9zcGFyYy9tbS9ndXAuYwpAQCAtMjQ1LDggKzI0NSw4IEBAIGludCBfX2dldF91 c2VyX3BhZ2VzX2Zhc3QodW5zaWduZWQgbG9uZyBzdGFydCwgaW50IG5yX3BhZ2VzLCBpbnQgd3Jp dGUsCiAJcmV0dXJuIG5yOwogfQogCi1pbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBs b25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsIGludCB3cml0ZSwKLQkJCXN0cnVjdCBwYWdlICoqcGFn ZXMpCitpbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJf cGFnZXMsCisJCQl1bnNpZ25lZCBpbnQgZ3VwX2ZsYWdzLCBzdHJ1Y3QgcGFnZSAqKnBhZ2VzKQog ewogCXN0cnVjdCBtbV9zdHJ1Y3QgKm1tID0gY3VycmVudC0+bW07CiAJdW5zaWduZWQgbG9uZyBh ZGRyLCBsZW4sIGVuZDsKQEAgLTMwMyw3ICszMDMsOCBAQCBpbnQgZ2V0X3VzZXJfcGFnZXNfZmFz dCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsIGludCB3cml0ZSwKIAkJbmV4dCA9 IHBnZF9hZGRyX2VuZChhZGRyLCBlbmQpOwogCQlpZiAocGdkX25vbmUocGdkKSkKIAkJCWdvdG8g c2xvdzsKLQkJaWYgKCFndXBfcHVkX3JhbmdlKHBnZCwgYWRkciwgbmV4dCwgd3JpdGUsIHBhZ2Vz LCAmbnIpKQorCQlpZiAoIWd1cF9wdWRfcmFuZ2UocGdkLCBhZGRyLCBuZXh0LCBndXBfZmxhZ3Mg JiBGT0xMX1dSSVRFLAorCQkJCSAgIHBhZ2VzLCAmbnIpKQogCQkJZ290byBzbG93OwogCX0gd2hp bGUgKHBnZHArKywgYWRkciA9IG5leHQsIGFkZHIgIT0gZW5kKTsKIApAQCAtMzI0LDcgKzMyNSw3 IEBAIGludCBnZXRfdXNlcl9wYWdlc19mYXN0KHVuc2lnbmVkIGxvbmcgc3RhcnQsIGludCBucl9w YWdlcywgaW50IHdyaXRlLAogCiAJCXJldCA9IGdldF91c2VyX3BhZ2VzX3VubG9ja2VkKHN0YXJ0 LAogCQkJKGVuZCAtIHN0YXJ0KSA+PiBQQUdFX1NISUZULCBwYWdlcywKLQkJCXdyaXRlID8gRk9M TF9XUklURSA6IDApOworCQkJZ3VwX2ZsYWdzKTsKIAogCQkvKiBIYXZlIHRvIGJlIGEgYml0IGNh cmVmdWwgd2l0aCByZXR1cm4gdmFsdWVzICovCiAJCWlmIChuciA+IDApIHsKZGlmZiAtLWdpdCBh L2FyY2gveDg2L2t2bS9wYWdpbmdfdG1wbC5oIGIvYXJjaC94ODYva3ZtL3BhZ2luZ190bXBsLmgK aW5kZXggNmJkY2EzOTgyOWJjLi4wODcxNTAzNGUzMTUgMTAwNjQ0Ci0tLSBhL2FyY2gveDg2L2t2 bS9wYWdpbmdfdG1wbC5oCisrKyBiL2FyY2gveDg2L2t2bS9wYWdpbmdfdG1wbC5oCkBAIC0xNDAs NyArMTQwLDcgQEAgc3RhdGljIGludCBGTkFNRShjbXB4Y2hnX2dwdGUpKHN0cnVjdCBrdm1fdmNw dSAqdmNwdSwgc3RydWN0IGt2bV9tbXUgKm1tdSwKIAlwdF9lbGVtZW50X3QgKnRhYmxlOwogCXN0 cnVjdCBwYWdlICpwYWdlOwogCi0JbnBhZ2VzID0gZ2V0X3VzZXJfcGFnZXNfZmFzdCgodW5zaWdu ZWQgbG9uZylwdGVwX3VzZXIsIDEsIDEsICZwYWdlKTsKKwlucGFnZXMgPSBnZXRfdXNlcl9wYWdl c19mYXN0KCh1bnNpZ25lZCBsb25nKXB0ZXBfdXNlciwgMSwgRk9MTF9XUklURSwgJnBhZ2UpOwog CS8qIENoZWNrIGlmIHRoZSB1c2VyIGlzIGRvaW5nIHNvbWV0aGluZyBtZWFuaW5nbGVzcy4gKi8K IAlpZiAodW5saWtlbHkobnBhZ2VzICE9IDEpKQogCQlyZXR1cm4gLUVGQVVMVDsKZGlmZiAtLWdp dCBhL2FyY2gveDg2L2t2bS9zdm0uYyBiL2FyY2gveDg2L2t2bS9zdm0uYwppbmRleCBmMTNhM2Ey NGQzNjAuLjE3MzU5NmEwMjBjYiAxMDA2NDQKLS0tIGEvYXJjaC94ODYva3ZtL3N2bS5jCisrKyBi L2FyY2gveDg2L2t2bS9zdm0uYwpAQCAtMTgwMyw3ICsxODAzLDcgQEAgc3RhdGljIHN0cnVjdCBw YWdlICoqc2V2X3Bpbl9tZW1vcnkoc3RydWN0IGt2bSAqa3ZtLCB1bnNpZ25lZCBsb25nIHVhZGRy LAogCQlyZXR1cm4gTlVMTDsKIAogCS8qIFBpbiB0aGUgdXNlciB2aXJ0dWFsIGFkZHJlc3MuICov Ci0JbnBpbm5lZCA9IGdldF91c2VyX3BhZ2VzX2Zhc3QodWFkZHIsIG5wYWdlcywgd3JpdGUgPyBG T0xMX1dSSVRFIDogMCwgcGFnZXMpOworCW5waW5uZWQgPSBnZXRfdXNlcl9wYWdlc19mYXN0KHVh ZGRyLCBucGFnZXMsIEZPTExfV1JJVEUsIHBhZ2VzKTsKIAlpZiAobnBpbm5lZCAhPSBucGFnZXMp IHsKIAkJcHJfZXJyKCJTRVY6IEZhaWx1cmUgbG9ja2luZyAlbHUgcGFnZXMuXG4iLCBucGFnZXMp OwogCQlnb3RvIGVycjsKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZnBnYS9kZmwtYWZ1LWRtYS1yZWdp b24uYyBiL2RyaXZlcnMvZnBnYS9kZmwtYWZ1LWRtYS1yZWdpb24uYwppbmRleCBlMThhNzg2ZmM5 NDMuLmM0Mzg3MjJiZjRlMSAxMDA2NDQKLS0tIGEvZHJpdmVycy9mcGdhL2RmbC1hZnUtZG1hLXJl Z2lvbi5jCisrKyBiL2RyaXZlcnMvZnBnYS9kZmwtYWZ1LWRtYS1yZWdpb24uYwpAQCAtMTAyLDcg KzEwMiw3IEBAIHN0YXRpYyBpbnQgYWZ1X2RtYV9waW5fcGFnZXMoc3RydWN0IGRmbF9mZWF0dXJl X3BsYXRmb3JtX2RhdGEgKnBkYXRhLAogCQlnb3RvIHVubG9ja192bTsKIAl9CiAKLQlwaW5uZWQg PSBnZXRfdXNlcl9wYWdlc19mYXN0KHJlZ2lvbi0+dXNlcl9hZGRyLCBucGFnZXMsIDEsCisJcGlu bmVkID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChyZWdpb24tPnVzZXJfYWRkciwgbnBhZ2VzLCBGT0xM X1dSSVRFLAogCQkJCSAgICAgcmVnaW9uLT5wYWdlcyk7CiAJaWYgKHBpbm5lZCA8IDApIHsKIAkJ cmV0ID0gcGlubmVkOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL3ZpYS92aWFfZG1hYmxp dC5jIGIvZHJpdmVycy9ncHUvZHJtL3ZpYS92aWFfZG1hYmxpdC5jCmluZGV4IDM0NWJkYTQ0OTRl MS4uMGM4YjA5NjAyOTEwIDEwMDY0NAotLS0gYS9kcml2ZXJzL2dwdS9kcm0vdmlhL3ZpYV9kbWFi bGl0LmMKKysrIGIvZHJpdmVycy9ncHUvZHJtL3ZpYS92aWFfZG1hYmxpdC5jCkBAIC0yMzksNyAr MjM5LDggQEAgdmlhX2xvY2tfYWxsX2RtYV9wYWdlcyhkcm1fdmlhX3NnX2luZm9fdCAqdnNnLCAg ZHJtX3ZpYV9kbWFibGl0X3QgKnhmZXIpCiAJaWYgKE5VTEwgPT0gdnNnLT5wYWdlcykKIAkJcmV0 dXJuIC1FTk9NRU07CiAJcmV0ID0gZ2V0X3VzZXJfcGFnZXNfZmFzdCgodW5zaWduZWQgbG9uZyl4 ZmVyLT5tZW1fYWRkciwKLQkJCXZzZy0+bnVtX3BhZ2VzLCB2c2ctPmRpcmVjdGlvbiA9PSBETUFf RlJPTV9ERVZJQ0UsCisJCQl2c2ctPm51bV9wYWdlcywKKwkJCXZzZy0+ZGlyZWN0aW9uID09IERN QV9GUk9NX0RFVklDRSA/IEZPTExfV1JJVEUgOiAwLAogCQkJdnNnLT5wYWdlcyk7CiAJaWYgKHJl dCAhPSB2c2ctPm51bV9wYWdlcykgewogCQlpZiAocmV0IDwgMCkKZGlmZiAtLWdpdCBhL2RyaXZl cnMvaW5maW5pYmFuZC9ody9oZmkxL3VzZXJfcGFnZXMuYyBiL2RyaXZlcnMvaW5maW5pYmFuZC9o dy9oZmkxL3VzZXJfcGFnZXMuYwppbmRleCAyNGI1OTJjNjUyMmUuLjc4Y2NhY2FmOTdkMCAxMDA2 NDQKLS0tIGEvZHJpdmVycy9pbmZpbmliYW5kL2h3L2hmaTEvdXNlcl9wYWdlcy5jCisrKyBiL2Ry aXZlcnMvaW5maW5pYmFuZC9ody9oZmkxL3VzZXJfcGFnZXMuYwpAQCAtMTA1LDcgKzEwNSw4IEBA IGludCBoZmkxX2FjcXVpcmVfdXNlcl9wYWdlcyhzdHJ1Y3QgbW1fc3RydWN0ICptbSwgdW5zaWdu ZWQgbG9uZyB2YWRkciwgc2l6ZV90IG5wCiB7CiAJaW50IHJldDsKIAotCXJldCA9IGdldF91c2Vy X3BhZ2VzX2Zhc3QodmFkZHIsIG5wYWdlcywgd3JpdGFibGUsIHBhZ2VzKTsKKwlyZXQgPSBnZXRf dXNlcl9wYWdlc19mYXN0KHZhZGRyLCBucGFnZXMsIHdyaXRhYmxlID8gRk9MTF9XUklURSA6IDAs CisJCQkJICBwYWdlcyk7CiAJaWYgKHJldCA8IDApCiAJCXJldHVybiByZXQ7CiAKZGlmZiAtLWdp dCBhL2RyaXZlcnMvbWlzYy9nZW53cWUvY2FyZF91dGlscy5jIGIvZHJpdmVycy9taXNjL2dlbndx ZS9jYXJkX3V0aWxzLmMKaW5kZXggMjUyNjVmZDBmZDZlLi44OWNmZjlkMTAxMmIgMTAwNjQ0Ci0t LSBhL2RyaXZlcnMvbWlzYy9nZW53cWUvY2FyZF91dGlscy5jCisrKyBiL2RyaXZlcnMvbWlzYy9n ZW53cWUvY2FyZF91dGlscy5jCkBAIC02MDMsNyArNjAzLDcgQEAgaW50IGdlbndxZV91c2VyX3Zt YXAoc3RydWN0IGdlbndxZV9kZXYgKmNkLCBzdHJ1Y3QgZG1hX21hcHBpbmcgKm0sIHZvaWQgKnVh ZGRyLAogCS8qIHBpbiB1c2VyIHBhZ2VzIGluIG1lbW9yeSAqLwogCXJjID0gZ2V0X3VzZXJfcGFn ZXNfZmFzdChkYXRhICYgUEFHRV9NQVNLLCAvKiBwYWdlIGFsaWduZWQgYWRkciAqLwogCQkJCSBt LT5ucl9wYWdlcywKLQkJCQkgbS0+d3JpdGUsCQkvKiByZWFkYWJsZS93cml0YWJsZSAqLworCQkJ CSBtLT53cml0ZSA/IEZPTExfV1JJVEUgOiAwLAkvKiByZWFkYWJsZS93cml0YWJsZSAqLwogCQkJ CSBtLT5wYWdlX2xpc3QpOwkvKiBwdHJzIHRvIHBhZ2VzICovCiAJaWYgKHJjIDwgMCkKIAkJZ290 byBmYWlsX2dldF91c2VyX3BhZ2VzOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9taXNjL3Ztd192bWNp L3ZtY2lfaG9zdC5jIGIvZHJpdmVycy9taXNjL3Ztd192bWNpL3ZtY2lfaG9zdC5jCmluZGV4IDk5 N2Y5MjU0M2RkNC4uNDIyZDA4ZGEzMjQ0IDEwMDY0NAotLS0gYS9kcml2ZXJzL21pc2Mvdm13X3Zt Y2kvdm1jaV9ob3N0LmMKKysrIGIvZHJpdmVycy9taXNjL3Ztd192bWNpL3ZtY2lfaG9zdC5jCkBA IC0yNDIsNyArMjQyLDcgQEAgc3RhdGljIGludCB2bWNpX2hvc3Rfc2V0dXBfbm90aWZ5KHN0cnVj dCB2bWNpX2N0eCAqY29udGV4dCwKIAkvKgogCSAqIExvY2sgcGh5c2ljYWwgcGFnZSBiYWNraW5n IGEgZ2l2ZW4gdXNlciBWQS4KIAkgKi8KLQlyZXR2YWwgPSBnZXRfdXNlcl9wYWdlc19mYXN0KHV2 YSwgMSwgMSwgJmNvbnRleHQtPm5vdGlmeV9wYWdlKTsKKwlyZXR2YWwgPSBnZXRfdXNlcl9wYWdl c19mYXN0KHV2YSwgMSwgRk9MTF9XUklURSwgJmNvbnRleHQtPm5vdGlmeV9wYWdlKTsKIAlpZiAo cmV0dmFsICE9IDEpIHsKIAkJY29udGV4dC0+bm90aWZ5X3BhZ2UgPSBOVUxMOwogCQlyZXR1cm4g Vk1DSV9FUlJPUl9HRU5FUklDOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9taXNjL3Ztd192bWNpL3Zt Y2lfcXVldWVfcGFpci5jIGIvZHJpdmVycy9taXNjL3Ztd192bWNpL3ZtY2lfcXVldWVfcGFpci5j CmluZGV4IDI2NGY0ZWQ4ZWVmMi4uYzUzOTZlZTMyZTUxIDEwMDY0NAotLS0gYS9kcml2ZXJzL21p c2Mvdm13X3ZtY2kvdm1jaV9xdWV1ZV9wYWlyLmMKKysrIGIvZHJpdmVycy9taXNjL3Ztd192bWNp L3ZtY2lfcXVldWVfcGFpci5jCkBAIC02NjYsNyArNjY2LDggQEAgc3RhdGljIGludCBxcF9ob3N0 X2dldF91c2VyX21lbW9yeSh1NjQgcHJvZHVjZV91dmEsCiAJaW50IGVyciA9IFZNQ0lfU1VDQ0VT UzsKIAogCXJldHZhbCA9IGdldF91c2VyX3BhZ2VzX2Zhc3QoKHVpbnRwdHJfdCkgcHJvZHVjZV91 dmEsCi0JCQkJICAgICBwcm9kdWNlX3EtPmtlcm5lbF9pZi0+bnVtX3BhZ2VzLCAxLAorCQkJCSAg ICAgcHJvZHVjZV9xLT5rZXJuZWxfaWYtPm51bV9wYWdlcywKKwkJCQkgICAgIEZPTExfV1JJVEUs CiAJCQkJICAgICBwcm9kdWNlX3EtPmtlcm5lbF9pZi0+dS5oLmhlYWRlcl9wYWdlKTsKIAlpZiAo cmV0dmFsIDwgKGludClwcm9kdWNlX3EtPmtlcm5lbF9pZi0+bnVtX3BhZ2VzKSB7CiAJCXByX2Rl YnVnKCJnZXRfdXNlcl9wYWdlc19mYXN0KHByb2R1Y2UpIGZhaWxlZCAocmV0dmFsPSVkKSIsCkBA IC02NzgsNyArNjc5LDggQEAgc3RhdGljIGludCBxcF9ob3N0X2dldF91c2VyX21lbW9yeSh1NjQg cHJvZHVjZV91dmEsCiAJfQogCiAJcmV0dmFsID0gZ2V0X3VzZXJfcGFnZXNfZmFzdCgodWludHB0 cl90KSBjb25zdW1lX3V2YSwKLQkJCQkgICAgIGNvbnN1bWVfcS0+a2VybmVsX2lmLT5udW1fcGFn ZXMsIDEsCisJCQkJICAgICBjb25zdW1lX3EtPmtlcm5lbF9pZi0+bnVtX3BhZ2VzLAorCQkJCSAg ICAgRk9MTF9XUklURSwKIAkJCQkgICAgIGNvbnN1bWVfcS0+a2VybmVsX2lmLT51LmguaGVhZGVy X3BhZ2UpOwogCWlmIChyZXR2YWwgPCAoaW50KWNvbnN1bWVfcS0+a2VybmVsX2lmLT5udW1fcGFn ZXMpIHsKIAkJcHJfZGVidWcoImdldF91c2VyX3BhZ2VzX2Zhc3QoY29uc3VtZSkgZmFpbGVkIChy ZXR2YWw9JWQpIiwKZGlmZiAtLWdpdCBhL2RyaXZlcnMvcGxhdGZvcm0vZ29sZGZpc2gvZ29sZGZp c2hfcGlwZS5jIGIvZHJpdmVycy9wbGF0Zm9ybS9nb2xkZmlzaC9nb2xkZmlzaF9waXBlLmMKaW5k ZXggMzIxYmM2NzNjNDE3Li5jZWYwMTMzYWE0N2EgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvcGxhdGZv cm0vZ29sZGZpc2gvZ29sZGZpc2hfcGlwZS5jCisrKyBiL2RyaXZlcnMvcGxhdGZvcm0vZ29sZGZp c2gvZ29sZGZpc2hfcGlwZS5jCkBAIC0yNzQsNyArMjc0LDggQEAgc3RhdGljIGludCBwaW5fdXNl cl9wYWdlcyh1bnNpZ25lZCBsb25nIGZpcnN0X3BhZ2UsCiAJCSppdGVyX2xhc3RfcGFnZV9zaXpl ID0gbGFzdF9wYWdlX3NpemU7CiAJfQogCi0JcmV0ID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChmaXJz dF9wYWdlLCByZXF1ZXN0ZWRfcGFnZXMsICFpc193cml0ZSwKKwlyZXQgPSBnZXRfdXNlcl9wYWdl c19mYXN0KGZpcnN0X3BhZ2UsIHJlcXVlc3RlZF9wYWdlcywKKwkJCQkgICFpc193cml0ZSA/IEZP TExfV1JJVEUgOiAwLAogCQkJCSAgcGFnZXMpOwogCWlmIChyZXQgPD0gMCkKIAkJcmV0dXJuIC1F RkFVTFQ7CmRpZmYgLS1naXQgYS9kcml2ZXJzL3JhcGlkaW8vZGV2aWNlcy9yaW9fbXBvcnRfY2Rl di5jIGIvZHJpdmVycy9yYXBpZGlvL2RldmljZXMvcmlvX21wb3J0X2NkZXYuYwppbmRleCBjYmU0 NjdmZjFhYmEuLmY2ODFiM2U5ZTk3MCAxMDA2NDQKLS0tIGEvZHJpdmVycy9yYXBpZGlvL2Rldmlj ZXMvcmlvX21wb3J0X2NkZXYuYworKysgYi9kcml2ZXJzL3JhcGlkaW8vZGV2aWNlcy9yaW9fbXBv cnRfY2Rldi5jCkBAIC04NjgsNyArODY4LDkgQEAgcmlvX2RtYV90cmFuc2ZlcihzdHJ1Y3QgZmls ZSAqZmlscCwgdTMyIHRyYW5zZmVyX21vZGUsCiAKIAkJcGlubmVkID0gZ2V0X3VzZXJfcGFnZXNf ZmFzdCgKIAkJCQkodW5zaWduZWQgbG9uZyl4ZmVyLT5sb2NfYWRkciAmIFBBR0VfTUFTSywKLQkJ CQlucl9wYWdlcywgZGlyID09IERNQV9GUk9NX0RFVklDRSwgcGFnZV9saXN0KTsKKwkJCQlucl9w YWdlcywKKwkJCQlkaXIgPT0gRE1BX0ZST01fREVWSUNFID8gRk9MTF9XUklURSA6IDAsCisJCQkJ cGFnZV9saXN0KTsKIAogCQlpZiAocGlubmVkICE9IG5yX3BhZ2VzKSB7CiAJCQlpZiAocGlubmVk IDwgMCkgewpkaWZmIC0tZ2l0IGEvZHJpdmVycy9zYnVzL2NoYXIvb3JhZGF4LmMgYi9kcml2ZXJz L3NidXMvY2hhci9vcmFkYXguYwppbmRleCA2NTE2YmMzY2I1OGIuLjc5MGFhMTQ4NjcwZCAxMDA2 NDQKLS0tIGEvZHJpdmVycy9zYnVzL2NoYXIvb3JhZGF4LmMKKysrIGIvZHJpdmVycy9zYnVzL2No YXIvb3JhZGF4LmMKQEAgLTQzNyw3ICs0MzcsNyBAQCBzdGF0aWMgaW50IGRheF9sb2NrX3BhZ2Uo dm9pZCAqdmEsIHN0cnVjdCBwYWdlICoqcCkKIAogCWRheF9kYmcoInV2YSAlcCIsIHZhKTsKIAot CXJldCA9IGdldF91c2VyX3BhZ2VzX2Zhc3QoKHVuc2lnbmVkIGxvbmcpdmEsIDEsIDEsIHApOwor CXJldCA9IGdldF91c2VyX3BhZ2VzX2Zhc3QoKHVuc2lnbmVkIGxvbmcpdmEsIDEsIEZPTExfV1JJ VEUsIHApOwogCWlmIChyZXQgPT0gMSkgewogCQlkYXhfZGJnKCJsb2NrZWQgcGFnZSAlcCwgZm9y IFZBICVwIiwgKnAsIHZhKTsKIAkJcmV0dXJuIDA7CmRpZmYgLS1naXQgYS9kcml2ZXJzL3Njc2kv c3QuYyBiL2RyaXZlcnMvc2NzaS9zdC5jCmluZGV4IDdmZjIyZDNmMDNlMy4uODcxYjI1OTE0YzA3 IDEwMDY0NAotLS0gYS9kcml2ZXJzL3Njc2kvc3QuYworKysgYi9kcml2ZXJzL3Njc2kvc3QuYwpA QCAtNDkxOCw3ICs0OTE4LDggQEAgc3RhdGljIGludCBzZ2xfbWFwX3VzZXJfcGFnZXMoc3RydWN0 IHN0X2J1ZmZlciAqU1RicCwKIAogICAgICAgICAvKiBUcnkgdG8gZmF1bHQgaW4gYWxsIG9mIHRo ZSBuZWNlc3NhcnkgcGFnZXMgKi8KICAgICAgICAgLyogcnc9PVJFQUQgbWVhbnMgcmVhZCBmcm9t IGRyaXZlLCB3cml0ZSBpbnRvIG1lbW9yeSBhcmVhICovCi0JcmVzID0gZ2V0X3VzZXJfcGFnZXNf ZmFzdCh1YWRkciwgbnJfcGFnZXMsIHJ3ID09IFJFQUQsIHBhZ2VzKTsKKwlyZXMgPSBnZXRfdXNl cl9wYWdlc19mYXN0KHVhZGRyLCBucl9wYWdlcywgcncgPT0gUkVBRCA/IEZPTExfV1JJVEUgOiAw LAorCQkJCSAgcGFnZXMpOwogCiAJLyogRXJyb3JzIGFuZCBubyBwYWdlIG1hcHBlZCBzaG91bGQg cmV0dXJuIGhlcmUgKi8KIAlpZiAocmVzIDwgbnJfcGFnZXMpCmRpZmYgLS1naXQgYS9kcml2ZXJz L3N0YWdpbmcvZ2Fza2V0L2dhc2tldF9wYWdlX3RhYmxlLmMgYi9kcml2ZXJzL3N0YWdpbmcvZ2Fz a2V0L2dhc2tldF9wYWdlX3RhYmxlLmMKaW5kZXggMjY3NTVkOWNhNDFkLi5mNjdmZGYxZDM4MTcg MTAwNjQ0Ci0tLSBhL2RyaXZlcnMvc3RhZ2luZy9nYXNrZXQvZ2Fza2V0X3BhZ2VfdGFibGUuYwor KysgYi9kcml2ZXJzL3N0YWdpbmcvZ2Fza2V0L2dhc2tldF9wYWdlX3RhYmxlLmMKQEAgLTQ4Niw4 ICs0ODYsOCBAQCBzdGF0aWMgaW50IGdhc2tldF9wZXJmb3JtX21hcHBpbmcoc3RydWN0IGdhc2tl dF9wYWdlX3RhYmxlICpwZ190YmwsCiAJCQlwdGVzW2ldLmRtYV9hZGRyID0gcGdfdGJsLT5jb2hl cmVudF9wYWdlc1swXS5wYWRkciArCiAJCQkJCSAgIG9mZiArIGkgKiBQQUdFX1NJWkU7CiAJCX0g ZWxzZSB7Ci0JCQlyZXQgPSBnZXRfdXNlcl9wYWdlc19mYXN0KHBhZ2VfYWRkciAtIG9mZnNldCwg MSwgMSwKLQkJCQkJCSAgJnBhZ2UpOworCQkJcmV0ID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChwYWdl X2FkZHIgLSBvZmZzZXQsIDEsCisJCQkJCQkgIEZPTExfV1JJVEUsICZwYWdlKTsKIAogCQkJaWYg KHJldCA8PSAwKSB7CiAJCQkJZGV2X2VycihwZ190YmwtPmRldmljZSwKZGlmZiAtLWdpdCBhL2Ry aXZlcnMvdGVlL3RlZV9zaG0uYyBiL2RyaXZlcnMvdGVlL3RlZV9zaG0uYwppbmRleCAwYjlhYjFk MGRkNDUuLjQ5ZmQ3MzEyZTJhYSAxMDA2NDQKLS0tIGEvZHJpdmVycy90ZWUvdGVlX3NobS5jCisr KyBiL2RyaXZlcnMvdGVlL3RlZV9zaG0uYwpAQCAtMjczLDcgKzI3Myw3IEBAIHN0cnVjdCB0ZWVf c2htICp0ZWVfc2htX3JlZ2lzdGVyKHN0cnVjdCB0ZWVfY29udGV4dCAqY3R4LCB1bnNpZ25lZCBs b25nIGFkZHIsCiAJCWdvdG8gZXJyOwogCX0KIAotCXJjID0gZ2V0X3VzZXJfcGFnZXNfZmFzdChz dGFydCwgbnVtX3BhZ2VzLCAxLCBzaG0tPnBhZ2VzKTsKKwlyYyA9IGdldF91c2VyX3BhZ2VzX2Zh c3Qoc3RhcnQsIG51bV9wYWdlcywgRk9MTF9XUklURSwgc2htLT5wYWdlcyk7CiAJaWYgKHJjID4g MCkKIAkJc2htLT5udW1fcGFnZXMgPSByYzsKIAlpZiAocmMgIT0gbnVtX3BhZ2VzKSB7CmRpZmYg LS1naXQgYS9kcml2ZXJzL3ZmaW8vdmZpb19pb21tdV9zcGFwcl90Y2UuYyBiL2RyaXZlcnMvdmZp by92ZmlvX2lvbW11X3NwYXByX3RjZS5jCmluZGV4IGM0MjQ5MTMzMjRlMy4uYTRiMTBiYjQwODZi IDEwMDY0NAotLS0gYS9kcml2ZXJzL3ZmaW8vdmZpb19pb21tdV9zcGFwcl90Y2UuYworKysgYi9k cml2ZXJzL3ZmaW8vdmZpb19pb21tdV9zcGFwcl90Y2UuYwpAQCAtNTMyLDcgKzUzMiw4IEBAIHN0 YXRpYyBpbnQgdGNlX2lvbW11X3VzZV9wYWdlKHVuc2lnbmVkIGxvbmcgdGNlLCB1bnNpZ25lZCBs b25nICpocGEpCiAJZW51bSBkbWFfZGF0YV9kaXJlY3Rpb24gZGlyZWN0aW9uID0gaW9tbXVfdGNl X2RpcmVjdGlvbih0Y2UpOwogCiAJaWYgKGdldF91c2VyX3BhZ2VzX2Zhc3QodGNlICYgUEFHRV9N QVNLLCAxLAotCQkJZGlyZWN0aW9uICE9IERNQV9UT19ERVZJQ0UsICZwYWdlKSAhPSAxKQorCQkJ ZGlyZWN0aW9uICE9IERNQV9UT19ERVZJQ0UgPyBGT0xMX1dSSVRFIDogMCwKKwkJCSZwYWdlKSAh PSAxKQogCQlyZXR1cm4gLUVGQVVMVDsKIAogCSpocGEgPSBfX3BhKCh1bnNpZ25lZCBsb25nKSBw YWdlX2FkZHJlc3MocGFnZSkpOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy92aG9zdC92aG9zdC5jIGIv ZHJpdmVycy92aG9zdC92aG9zdC5jCmluZGV4IDI0YTEyOWZjZGQ2MS4uNzI2ODViMTY1OWZmIDEw MDY0NAotLS0gYS9kcml2ZXJzL3Zob3N0L3Zob3N0LmMKKysrIGIvZHJpdmVycy92aG9zdC92aG9z dC5jCkBAIC0xNzAwLDcgKzE3MDAsNyBAQCBzdGF0aWMgaW50IHNldF9iaXRfdG9fdXNlcihpbnQg bnIsIHZvaWQgX191c2VyICphZGRyKQogCWludCBiaXQgPSBuciArIChsb2cgJSBQQUdFX1NJWkUp ICogODsKIAlpbnQgcjsKIAotCXIgPSBnZXRfdXNlcl9wYWdlc19mYXN0KGxvZywgMSwgMSwgJnBh Z2UpOworCXIgPSBnZXRfdXNlcl9wYWdlc19mYXN0KGxvZywgMSwgRk9MTF9XUklURSwgJnBhZ2Up OwogCWlmIChyIDwgMCkKIAkJcmV0dXJuIHI7CiAJQlVHX09OKHIgIT0gMSk7CmRpZmYgLS1naXQg YS9kcml2ZXJzL3ZpZGVvL2ZiZGV2L3B2cjJmYi5jIGIvZHJpdmVycy92aWRlby9mYmRldi9wdnIy ZmIuYwppbmRleCA4YTUzZDFkZTYxMWQuLjQxMzkwYzhlMGY2NyAxMDA2NDQKLS0tIGEvZHJpdmVy cy92aWRlby9mYmRldi9wdnIyZmIuYworKysgYi9kcml2ZXJzL3ZpZGVvL2ZiZGV2L3B2cjJmYi5j CkBAIC02ODYsNyArNjg2LDcgQEAgc3RhdGljIHNzaXplX3QgcHZyMmZiX3dyaXRlKHN0cnVjdCBm Yl9pbmZvICppbmZvLCBjb25zdCBjaGFyICpidWYsCiAJaWYgKCFwYWdlcykKIAkJcmV0dXJuIC1F Tk9NRU07CiAKLQlyZXQgPSBnZXRfdXNlcl9wYWdlc19mYXN0KCh1bnNpZ25lZCBsb25nKWJ1Ziwg bnJfcGFnZXMsIHRydWUsIHBhZ2VzKTsKKwlyZXQgPSBnZXRfdXNlcl9wYWdlc19mYXN0KCh1bnNp Z25lZCBsb25nKWJ1ZiwgbnJfcGFnZXMsIEZPTExfV1JJVEUsIHBhZ2VzKTsKIAlpZiAocmV0IDwg bnJfcGFnZXMpIHsKIAkJbnJfcGFnZXMgPSByZXQ7CiAJCXJldCA9IC1FSU5WQUw7CmRpZmYgLS1n aXQgYS9kcml2ZXJzL3ZpcnQvZnNsX2h5cGVydmlzb3IuYyBiL2RyaXZlcnMvdmlydC9mc2xfaHlw ZXJ2aXNvci5jCmluZGV4IDhiYTcyNmU2MDBlOS4uNjQ0NmJjYWI0MTg1IDEwMDY0NAotLS0gYS9k cml2ZXJzL3ZpcnQvZnNsX2h5cGVydmlzb3IuYworKysgYi9kcml2ZXJzL3ZpcnQvZnNsX2h5cGVy dmlzb3IuYwpAQCAtMjQ0LDcgKzI0NCw3IEBAIHN0YXRpYyBsb25nIGlvY3RsX21lbWNweShzdHJ1 Y3QgZnNsX2h2X2lvY3RsX21lbWNweSBfX3VzZXIgKnApCiAKIAkvKiBHZXQgdGhlIHBoeXNpY2Fs IGFkZHJlc3NlcyBvZiB0aGUgc291cmNlIGJ1ZmZlciAqLwogCW51bV9waW5uZWQgPSBnZXRfdXNl cl9wYWdlc19mYXN0KHBhcmFtLmxvY2FsX3ZhZGRyIC0gbGJfb2Zmc2V0LAotCQludW1fcGFnZXMs IHBhcmFtLnNvdXJjZSAhPSAtMSwgcGFnZXMpOworCQludW1fcGFnZXMsIHBhcmFtLnNvdXJjZSAh PSAtMSA/IEZPTExfV1JJVEUgOiAwLCBwYWdlcyk7CiAKIAlpZiAobnVtX3Bpbm5lZCAhPSBudW1f cGFnZXMpIHsKIAkJLyogZ2V0X3VzZXJfcGFnZXMoKSBmYWlsZWQgKi8KZGlmZiAtLWdpdCBhL2Ry aXZlcnMveGVuL2dudGRldi5jIGIvZHJpdmVycy94ZW4vZ250ZGV2LmMKaW5kZXggNWVmYzVlZWU5 NTQ0Li43YjQ3ZjFlNmFhYjQgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMveGVuL2dudGRldi5jCisrKyBi L2RyaXZlcnMveGVuL2dudGRldi5jCkBAIC04NTIsNyArODUyLDcgQEAgc3RhdGljIGludCBnbnRk ZXZfZ2V0X3BhZ2Uoc3RydWN0IGdudGRldl9jb3B5X2JhdGNoICpiYXRjaCwgdm9pZCBfX3VzZXIg KnZpcnQsCiAJdW5zaWduZWQgbG9uZyB4ZW5fcGZuOwogCWludCByZXQ7CiAKLQlyZXQgPSBnZXRf dXNlcl9wYWdlc19mYXN0KGFkZHIsIDEsIHdyaXRlYWJsZSwgJnBhZ2UpOworCXJldCA9IGdldF91 c2VyX3BhZ2VzX2Zhc3QoYWRkciwgMSwgd3JpdGVhYmxlID8gRk9MTF9XUklURSA6IDAsICZwYWdl KTsKIAlpZiAocmV0IDwgMCkKIAkJcmV0dXJuIHJldDsKIApkaWZmIC0tZ2l0IGEvZnMvb3Jhbmdl ZnMvb3JhbmdlZnMtYnVmbWFwLmMgYi9mcy9vcmFuZ2Vmcy9vcmFuZ2Vmcy1idWZtYXAuYwppbmRl eCA0NDNiY2Q4YzNjMTkuLjVhN2M0ZmRhNjgyZiAxMDA2NDQKLS0tIGEvZnMvb3JhbmdlZnMvb3Jh bmdlZnMtYnVmbWFwLmMKKysrIGIvZnMvb3JhbmdlZnMvb3JhbmdlZnMtYnVmbWFwLmMKQEAgLTI2 OSw3ICsyNjksNyBAQCBvcmFuZ2Vmc19idWZtYXBfbWFwKHN0cnVjdCBvcmFuZ2Vmc19idWZtYXAg KmJ1Zm1hcCwKIAogCS8qIG1hcCB0aGUgcGFnZXMgKi8KIAlyZXQgPSBnZXRfdXNlcl9wYWdlc19m YXN0KCh1bnNpZ25lZCBsb25nKXVzZXJfZGVzYy0+cHRyLAotCQkJICAgICBidWZtYXAtPnBhZ2Vf Y291bnQsIDEsIGJ1Zm1hcC0+cGFnZV9hcnJheSk7CisJCQkgICAgIGJ1Zm1hcC0+cGFnZV9jb3Vu dCwgRk9MTF9XUklURSwgYnVmbWFwLT5wYWdlX2FycmF5KTsKIAogCWlmIChyZXQgPCAwKQogCQly ZXR1cm4gcmV0OwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbS5oIGIvaW5jbHVkZS9saW51 eC9tbS5oCmluZGV4IDA1YTEwNWQ5ZDRjMy4uOGUxZjNjZDc0ODJhIDEwMDY0NAotLS0gYS9pbmNs dWRlL2xpbnV4L21tLmgKKysrIGIvaW5jbHVkZS9saW51eC9tbS5oCkBAIC0xNTM3LDggKzE1Mzcs OCBAQCBsb25nIGdldF91c2VyX3BhZ2VzX2xvY2tlZCh1bnNpZ25lZCBsb25nIHN0YXJ0LCB1bnNp Z25lZCBsb25nIG5yX3BhZ2VzLAogbG9uZyBnZXRfdXNlcl9wYWdlc191bmxvY2tlZCh1bnNpZ25l ZCBsb25nIHN0YXJ0LCB1bnNpZ25lZCBsb25nIG5yX3BhZ2VzLAogCQkgICAgc3RydWN0IHBhZ2Ug KipwYWdlcywgdW5zaWduZWQgaW50IGd1cF9mbGFncyk7CiAKLWludCBnZXRfdXNlcl9wYWdlc19m YXN0KHVuc2lnbmVkIGxvbmcgc3RhcnQsIGludCBucl9wYWdlcywgaW50IHdyaXRlLAotCQkJc3Ry dWN0IHBhZ2UgKipwYWdlcyk7CitpbnQgZ2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25n IHN0YXJ0LCBpbnQgbnJfcGFnZXMsCisJCQl1bnNpZ25lZCBpbnQgZ3VwX2ZsYWdzLCBzdHJ1Y3Qg cGFnZSAqKnBhZ2VzKTsKIAogLyogQ29udGFpbmVyIGZvciBwaW5uZWQgcGZucyAvIHBhZ2VzICov CiBzdHJ1Y3QgZnJhbWVfdmVjdG9yIHsKZGlmZiAtLWdpdCBhL2tlcm5lbC9mdXRleC5jIGIva2Vy bmVsL2Z1dGV4LmMKaW5kZXggZmRkMzEyZGEwOTkyLi5lMTAyMDk5NDZmOGIgMTAwNjQ0Ci0tLSBh L2tlcm5lbC9mdXRleC5jCisrKyBiL2tlcm5lbC9mdXRleC5jCkBAIC01NDYsNyArNTQ2LDcgQEAg Z2V0X2Z1dGV4X2tleSh1MzIgX191c2VyICp1YWRkciwgaW50IGZzaGFyZWQsIHVuaW9uIGZ1dGV4 X2tleSAqa2V5LCBlbnVtIGZ1dGV4X2EKIAlpZiAodW5saWtlbHkoc2hvdWxkX2ZhaWxfZnV0ZXgo ZnNoYXJlZCkpKQogCQlyZXR1cm4gLUVGQVVMVDsKIAotCWVyciA9IGdldF91c2VyX3BhZ2VzX2Zh c3QoYWRkcmVzcywgMSwgMSwgJnBhZ2UpOworCWVyciA9IGdldF91c2VyX3BhZ2VzX2Zhc3QoYWRk cmVzcywgMSwgRk9MTF9XUklURSwgJnBhZ2UpOwogCS8qCiAJICogSWYgd3JpdGUgYWNjZXNzIGlz IG5vdCByZXF1aXJlZCAoZWcuIEZVVEVYX1dBSVQpLCB0cnkKIAkgKiBhbmQgZ2V0IHJlYWQtb25s eSBhY2Nlc3MuCmRpZmYgLS1naXQgYS9saWIvaW92X2l0ZXIuYyBiL2xpYi9pb3ZfaXRlci5jCmlu ZGV4IGJlNGJkNjI3Y2FmMC4uNmRiYWUwNjkyNzE5IDEwMDY0NAotLS0gYS9saWIvaW92X2l0ZXIu YworKysgYi9saWIvaW92X2l0ZXIuYwpAQCAtMTI4MCw3ICsxMjgwLDkgQEAgc3NpemVfdCBpb3Zf aXRlcl9nZXRfcGFnZXMoc3RydWN0IGlvdl9pdGVyICppLAogCQkJbGVuID0gbWF4cGFnZXMgKiBQ QUdFX1NJWkU7CiAJCWFkZHIgJj0gfihQQUdFX1NJWkUgLSAxKTsKIAkJbiA9IERJVl9ST1VORF9V UChsZW4sIFBBR0VfU0laRSk7Ci0JCXJlcyA9IGdldF91c2VyX3BhZ2VzX2Zhc3QoYWRkciwgbiwg aW92X2l0ZXJfcncoaSkgIT0gV1JJVEUsIHBhZ2VzKTsKKwkJcmVzID0gZ2V0X3VzZXJfcGFnZXNf ZmFzdChhZGRyLCBuLAorCQkJCWlvdl9pdGVyX3J3KGkpICE9IFdSSVRFID8gIEZPTExfV1JJVEUg OiAwLAorCQkJCXBhZ2VzKTsKIAkJaWYgKHVubGlrZWx5KHJlcyA8IDApKQogCQkJcmV0dXJuIHJl czsKIAkJcmV0dXJuIChyZXMgPT0gbiA/IGxlbiA6IHJlcyAqIFBBR0VfU0laRSkgLSAqc3RhcnQ7 CkBAIC0xMzYxLDcgKzEzNjMsOCBAQCBzc2l6ZV90IGlvdl9pdGVyX2dldF9wYWdlc19hbGxvYyhz dHJ1Y3QgaW92X2l0ZXIgKmksCiAJCXAgPSBnZXRfcGFnZXNfYXJyYXkobik7CiAJCWlmICghcCkK IAkJCXJldHVybiAtRU5PTUVNOwotCQlyZXMgPSBnZXRfdXNlcl9wYWdlc19mYXN0KGFkZHIsIG4s IGlvdl9pdGVyX3J3KGkpICE9IFdSSVRFLCBwKTsKKwkJcmVzID0gZ2V0X3VzZXJfcGFnZXNfZmFz dChhZGRyLCBuLAorCQkJCWlvdl9pdGVyX3J3KGkpICE9IFdSSVRFID8gIEZPTExfV1JJVEUgOiAw LCBwKTsKIAkJaWYgKHVubGlrZWx5KHJlcyA8IDApKSB7CiAJCQlrdmZyZWUocCk7CiAJCQlyZXR1 cm4gcmVzOwpkaWZmIC0tZ2l0IGEvbW0vZ3VwLmMgYi9tbS9ndXAuYwppbmRleCA2ODEzODgyMzYx MDYuLjZmMzJkMzZiM2M1YiAxMDA2NDQKLS0tIGEvbW0vZ3VwLmMKKysrIGIvbW0vZ3VwLmMKQEAg LTE4NjMsNyArMTg2Myw3IEBAIGludCBfX2dldF91c2VyX3BhZ2VzX2Zhc3QodW5zaWduZWQgbG9u ZyBzdGFydCwgaW50IG5yX3BhZ2VzLCBpbnQgd3JpdGUsCiAgKiBnZXRfdXNlcl9wYWdlc19mYXN0 KCkgLSBwaW4gdXNlciBwYWdlcyBpbiBtZW1vcnkKICAqIEBzdGFydDoJc3RhcnRpbmcgdXNlciBh ZGRyZXNzCiAgKiBAbnJfcGFnZXM6CW51bWJlciBvZiBwYWdlcyBmcm9tIHN0YXJ0IHRvIHBpbgot ICogQHdyaXRlOgl3aGV0aGVyIHBhZ2VzIHdpbGwgYmUgd3JpdHRlbiB0bworICogQGd1cF9mbGFn czoJZmxhZ3MgbW9kaWZ5aW5nIHBpbiBiZWhhdmlvdXIKICAqIEBwYWdlczoJYXJyYXkgdGhhdCBy ZWNlaXZlcyBwb2ludGVycyB0byB0aGUgcGFnZXMgcGlubmVkLgogICoJCVNob3VsZCBiZSBhdCBs ZWFzdCBucl9wYWdlcyBsb25nLgogICoKQEAgLTE4NzUsOCArMTg3NSw4IEBAIGludCBfX2dldF91 c2VyX3BhZ2VzX2Zhc3QodW5zaWduZWQgbG9uZyBzdGFydCwgaW50IG5yX3BhZ2VzLCBpbnQgd3Jp dGUsCiAgKiByZXF1ZXN0ZWQuIElmIG5yX3BhZ2VzIGlzIDAgb3IgbmVnYXRpdmUsIHJldHVybnMg MC4gSWYgbm8gcGFnZXMKICAqIHdlcmUgcGlubmVkLCByZXR1cm5zIC1lcnJuby4KICAqLwotaW50 IGdldF91c2VyX3BhZ2VzX2Zhc3QodW5zaWduZWQgbG9uZyBzdGFydCwgaW50IG5yX3BhZ2VzLCBp bnQgd3JpdGUsCi0JCQlzdHJ1Y3QgcGFnZSAqKnBhZ2VzKQoraW50IGdldF91c2VyX3BhZ2VzX2Zh c3QodW5zaWduZWQgbG9uZyBzdGFydCwgaW50IG5yX3BhZ2VzLAorCQkJdW5zaWduZWQgaW50IGd1 cF9mbGFncywgc3RydWN0IHBhZ2UgKipwYWdlcykKIHsKIAl1bnNpZ25lZCBsb25nIGFkZHIsIGxl biwgZW5kOwogCWludCBuciA9IDAsIHJldCA9IDA7CkBAIC0xODk0LDcgKzE4OTQsNyBAQCBpbnQg Z2V0X3VzZXJfcGFnZXNfZmFzdCh1bnNpZ25lZCBsb25nIHN0YXJ0LCBpbnQgbnJfcGFnZXMsIGlu dCB3cml0ZSwKIAogCWlmIChndXBfZmFzdF9wZXJtaXR0ZWQoc3RhcnQsIG5yX3BhZ2VzKSkgewog CQlsb2NhbF9pcnFfZGlzYWJsZSgpOwotCQlndXBfcGdkX3JhbmdlKGFkZHIsIGVuZCwgd3JpdGUg PyBGT0xMX1dSSVRFIDogMCwgcGFnZXMsICZucik7CisJCWd1cF9wZ2RfcmFuZ2UoYWRkciwgZW5k LCBndXBfZmxhZ3MsIHBhZ2VzLCAmbnIpOwogCQlsb2NhbF9pcnFfZW5hYmxlKCk7CiAJCXJldCA9 IG5yOwogCX0KQEAgLTE5MDUsNyArMTkwNSw3IEBAIGludCBnZXRfdXNlcl9wYWdlc19mYXN0KHVu c2lnbmVkIGxvbmcgc3RhcnQsIGludCBucl9wYWdlcywgaW50IHdyaXRlLAogCQlwYWdlcyArPSBu cjsKIAogCQlyZXQgPSBnZXRfdXNlcl9wYWdlc191bmxvY2tlZChzdGFydCwgbnJfcGFnZXMgLSBu ciwgcGFnZXMsCi0JCQkJd3JpdGUgPyBGT0xMX1dSSVRFIDogMCk7CisJCQkJCSAgICAgIGd1cF9m bGFncyk7CiAKIAkJLyogSGF2ZSB0byBiZSBhIGJpdCBjYXJlZnVsIHdpdGggcmV0dXJuIHZhbHVl cyAqLwogCQlpZiAobnIgPiAwKSB7CmRpZmYgLS1naXQgYS9tbS91dGlsLmMgYi9tbS91dGlsLmMK aW5kZXggMWVhMDU1MTM4MDQzLi4wMWZmZTE0NWM2MmIgMTAwNjQ0Ci0tLSBhL21tL3V0aWwuYwor KysgYi9tbS91dGlsLmMKQEAgLTMwNiw3ICszMDYsNyBAQCBFWFBPUlRfU1lNQk9MX0dQTChfX2dl dF91c2VyX3BhZ2VzX2Zhc3QpOwogICogZ2V0X3VzZXJfcGFnZXNfZmFzdCgpIC0gcGluIHVzZXIg cGFnZXMgaW4gbWVtb3J5CiAgKiBAc3RhcnQ6CXN0YXJ0aW5nIHVzZXIgYWRkcmVzcwogICogQG5y X3BhZ2VzOgludW1iZXIgb2YgcGFnZXMgZnJvbSBzdGFydCB0byBwaW4KLSAqIEB3cml0ZToJd2hl dGhlciBwYWdlcyB3aWxsIGJlIHdyaXR0ZW4gdG8KKyAqIEBndXBfZmxhZ3M6CWZsYWdzIG1vZGlm eWluZyBwaW4gYmVoYXZpb3VyCiAgKiBAcGFnZXM6CWFycmF5IHRoYXQgcmVjZWl2ZXMgcG9pbnRl cnMgdG8gdGhlIHBhZ2VzIHBpbm5lZC4KICAqCQlTaG91bGQgYmUgYXQgbGVhc3QgbnJfcGFnZXMg bG9uZy4KICAqCkBAIC0zMjcsMTAgKzMyNywxMCBAQCBFWFBPUlRfU1lNQk9MX0dQTChfX2dldF91 c2VyX3BhZ2VzX2Zhc3QpOwogICogZ2V0X3VzZXJfcGFnZXNfZmFzdCBzaW1wbHkgZmFsbHMgYmFj ayB0byBnZXRfdXNlcl9wYWdlcy4KICAqLwogaW50IF9fd2VhayBnZXRfdXNlcl9wYWdlc19mYXN0 KHVuc2lnbmVkIGxvbmcgc3RhcnQsCi0JCQkJaW50IG5yX3BhZ2VzLCBpbnQgd3JpdGUsIHN0cnVj dCBwYWdlICoqcGFnZXMpCisJCQkJaW50IG5yX3BhZ2VzLCB1bnNpZ25lZCBpbnQgZ3VwX2ZsYWdz LAorCQkJCXN0cnVjdCBwYWdlICoqcGFnZXMpCiB7Ci0JcmV0dXJuIGdldF91c2VyX3BhZ2VzX3Vu bG9ja2VkKHN0YXJ0LCBucl9wYWdlcywgcGFnZXMsCi0JCQkJICAgICAgIHdyaXRlID8gRk9MTF9X UklURSA6IDApOworCXJldHVybiBnZXRfdXNlcl9wYWdlc191bmxvY2tlZChzdGFydCwgbnJfcGFn ZXMsIHBhZ2VzLCBndXBfZmxhZ3MpOwogfQogRVhQT1JUX1NZTUJPTF9HUEwoZ2V0X3VzZXJfcGFn ZXNfZmFzdCk7CiAKZGlmZiAtLWdpdCBhL25ldC9jZXBoL3BhZ2V2ZWMuYyBiL25ldC9jZXBoL3Bh Z2V2ZWMuYwppbmRleCBkMzczNmY1YmZmZWMuLjc0Y2FmYzAxNDJlYSAxMDA2NDQKLS0tIGEvbmV0 L2NlcGgvcGFnZXZlYy5jCisrKyBiL25ldC9jZXBoL3BhZ2V2ZWMuYwpAQCAtMjcsNyArMjcsNyBA QCBzdHJ1Y3QgcGFnZSAqKmNlcGhfZ2V0X2RpcmVjdF9wYWdlX3ZlY3Rvcihjb25zdCB2b2lkIF9f dXNlciAqZGF0YSwKIAl3aGlsZSAoZ290IDwgbnVtX3BhZ2VzKSB7CiAJCXJjID0gZ2V0X3VzZXJf cGFnZXNfZmFzdCgKIAkJICAgICh1bnNpZ25lZCBsb25nKWRhdGEgKyAoKHVuc2lnbmVkIGxvbmcp Z290ICogUEFHRV9TSVpFKSwKLQkJICAgIG51bV9wYWdlcyAtIGdvdCwgd3JpdGVfcGFnZSwgcGFn ZXMgKyBnb3QpOworCQkgICAgbnVtX3BhZ2VzIC0gZ290LCB3cml0ZV9wYWdlID8gRk9MTF9XUklU RSA6IDAsIHBhZ2VzICsgZ290KTsKIAkJaWYgKHJjIDwgMCkKIAkJCWJyZWFrOwogCQlCVUdfT04o cmMgPT0gMCk7CmRpZmYgLS1naXQgYS9uZXQvcmRzL2luZm8uYyBiL25ldC9yZHMvaW5mby5jCmlu ZGV4IGUzNjdhOTdhMThjOC4uMDNmNmZkNTZkMjM3IDEwMDY0NAotLS0gYS9uZXQvcmRzL2luZm8u YworKysgYi9uZXQvcmRzL2luZm8uYwpAQCAtMTkzLDcgKzE5Myw3IEBAIGludCByZHNfaW5mb19n ZXRzb2Nrb3B0KHN0cnVjdCBzb2NrZXQgKnNvY2ssIGludCBvcHRuYW1lLCBjaGFyIF9fdXNlciAq b3B0dmFsLAogCQlyZXQgPSAtRU5PTUVNOwogCQlnb3RvIG91dDsKIAl9Ci0JcmV0ID0gZ2V0X3Vz ZXJfcGFnZXNfZmFzdChzdGFydCwgbnJfcGFnZXMsIDEsIHBhZ2VzKTsKKwlyZXQgPSBnZXRfdXNl cl9wYWdlc19mYXN0KHN0YXJ0LCBucl9wYWdlcywgRk9MTF9XUklURSwgcGFnZXMpOwogCWlmIChy ZXQgIT0gbnJfcGFnZXMpIHsKIAkJaWYgKHJldCA+IDApCiAJCQlucl9wYWdlcyA9IHJldDsKZGlm ZiAtLWdpdCBhL25ldC9yZHMvcmRtYS5jIGIvbmV0L3Jkcy9yZG1hLmMKaW5kZXggMTgyYWI4NDMw NTk0Li5iMzQwZWQ0ZmM0M2EgMTAwNjQ0Ci0tLSBhL25ldC9yZHMvcmRtYS5jCisrKyBiL25ldC9y ZHMvcmRtYS5jCkBAIC0xNTgsNyArMTU4LDggQEAgc3RhdGljIGludCByZHNfcGluX3BhZ2VzKHVu c2lnbmVkIGxvbmcgdXNlcl9hZGRyLCB1bnNpZ25lZCBpbnQgbnJfcGFnZXMsCiB7CiAJaW50IHJl dDsKIAotCXJldCA9IGdldF91c2VyX3BhZ2VzX2Zhc3QodXNlcl9hZGRyLCBucl9wYWdlcywgd3Jp dGUsIHBhZ2VzKTsKKwlyZXQgPSBnZXRfdXNlcl9wYWdlc19mYXN0KHVzZXJfYWRkciwgbnJfcGFn ZXMsIHdyaXRlID8gRk9MTF9XUklURSA6IDAsCisJCQkJICBwYWdlcyk7CiAKIAlpZiAocmV0ID49 IDAgJiYgcmV0IDwgbnJfcGFnZXMpIHsKIAkJd2hpbGUgKHJldC0tKQotLSAKMi4yMC4xCgpfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpkcmktZGV2ZWwgbWFp bGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5m cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9kcmktZGV2ZWw= From mboxrd@z Thu Jan 1 00:00:00 1970 From: ira.weiny@intel.com Date: Wed, 13 Feb 2019 23:04:51 +0000 Subject: [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool' Message-Id: <20190213230455.5605-4-ira.weiny@intel.com> List-Id: References: <20190211201643.7599-1-ira.weiny@intel.com> <20190213230455.5605-1-ira.weiny@intel.com> In-Reply-To: <20190213230455.5605-1-ira.weiny@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, kvm@vger.kernel.org, linux-fpga@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-media@vger.kernel.org, linux-scsi@vger.kernel.org, devel@driverdev.osuosl.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-fbdev@vger.kernel.org, xen-devel@lists.xenproject.org, devel@lists.orangefs.org, linux-mm@kvack.org, ceph-devel@vger.kernel.org, rds-devel@oss.oracle.com Cc: Martin Brandenburg , Rich Felker , David Hildenbrand , David Airlie , Jason Wang , =?UTF-8?q?Kai=20M=C3=A4kisara?= , Michal Hocko , Ira Weiny , Ben Chan , Rob Springer , Todd Poynor , Yoshinori Sato , "Michael S. Tsirkin" , Jason Gunthorpe , Ingo Molnar , Matt Porter , Wu Hao , Alan Tull , John Hubbard , "James E.J. Bottomley" , Alex Williamson , Stefano Stabellini , Borislav Petkov From: Ira Weiny To facilitate additional options to get_user_pages_fast() change the singular write parameter to be gup_flags. This patch does not change any functionality. New functionality will follow in subsequent patches. Some of the get_user_pages_fast() call sites were unchanged because they already passed FOLL_WRITE or 0 for the write parameter. Signed-off-by: Ira Weiny --- arch/mips/mm/gup.c | 11 ++++++----- arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 ++-- arch/powerpc/kvm/e500_mmu.c | 2 +- arch/powerpc/mm/mmu_context_iommu.c | 4 ++-- arch/s390/kvm/interrupt.c | 2 +- arch/s390/mm/gup.c | 12 ++++++------ arch/sh/mm/gup.c | 11 ++++++----- arch/sparc/mm/gup.c | 9 +++++---- arch/x86/kvm/paging_tmpl.h | 2 +- arch/x86/kvm/svm.c | 2 +- drivers/fpga/dfl-afu-dma-region.c | 2 +- drivers/gpu/drm/via/via_dmablit.c | 3 ++- drivers/infiniband/hw/hfi1/user_pages.c | 3 ++- drivers/misc/genwqe/card_utils.c | 2 +- drivers/misc/vmw_vmci/vmci_host.c | 2 +- drivers/misc/vmw_vmci/vmci_queue_pair.c | 6 ++++-- drivers/platform/goldfish/goldfish_pipe.c | 3 ++- drivers/rapidio/devices/rio_mport_cdev.c | 4 +++- drivers/sbus/char/oradax.c | 2 +- drivers/scsi/st.c | 3 ++- drivers/staging/gasket/gasket_page_table.c | 4 ++-- drivers/tee/tee_shm.c | 2 +- drivers/vfio/vfio_iommu_spapr_tce.c | 3 ++- drivers/vhost/vhost.c | 2 +- drivers/video/fbdev/pvr2fb.c | 2 +- drivers/virt/fsl_hypervisor.c | 2 +- drivers/xen/gntdev.c | 2 +- fs/orangefs/orangefs-bufmap.c | 2 +- include/linux/mm.h | 4 ++-- kernel/futex.c | 2 +- lib/iov_iter.c | 7 +++++-- mm/gup.c | 10 +++++----- mm/util.c | 8 ++++---- net/ceph/pagevec.c | 2 +- net/rds/info.c | 2 +- net/rds/rdma.c | 3 ++- 36 files changed, 81 insertions(+), 65 deletions(-) diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c index 0d14e0d8eacf..4c2b4483683c 100644 --- a/arch/mips/mm/gup.c +++ b/arch/mips/mm/gup.c @@ -235,7 +235,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -247,8 +247,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -273,7 +273,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); local_irq_enable(); @@ -289,7 +290,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, pages += nr; ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, - pages, write ? FOLL_WRITE : 0); + pages, gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index bd2dcfbf00cd..8fcb0a921e46 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -582,7 +582,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, /* If writing != 0, then the HPTE must allow writing, if we get here */ write_ok = writing; hva = gfn_to_hva_memslot(memslot, gfn); - npages = get_user_pages_fast(hva, 1, writing, pages); + npages = get_user_pages_fast(hva, 1, writing ? FOLL_WRITE : 0, pages); if (npages < 1) { /* Check if it's an I/O mapping */ down_read(¤t->mm->mmap_sem); @@ -1175,7 +1175,7 @@ void *kvmppc_pin_guest_page(struct kvm *kvm, unsigned long gpa, if (!memslot || (memslot->flags & KVM_MEMSLOT_INVALID)) goto err; hva = gfn_to_hva_memslot(memslot, gfn); - npages = get_user_pages_fast(hva, 1, 1, pages); + npages = get_user_pages_fast(hva, 1, FOLL_WRITE, pages); if (npages < 1) goto err; page = pages[0]; diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c index 24296f4cadc6..e0af53fd78c5 100644 --- a/arch/powerpc/kvm/e500_mmu.c +++ b/arch/powerpc/kvm/e500_mmu.c @@ -783,7 +783,7 @@ int kvm_vcpu_ioctl_config_tlb(struct kvm_vcpu *vcpu, if (!pages) return -ENOMEM; - ret = get_user_pages_fast(cfg->array, num_pages, 1, pages); + ret = get_user_pages_fast(cfg->array, num_pages, FOLL_WRITE, pages); if (ret < 0) goto free_pages; diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index a712a650a8b6..acb0990c8364 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -190,7 +190,7 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, for (i = 0; i < entries; ++i) { cur_ua = ua + (i << PAGE_SHIFT); if (1 != get_user_pages_fast(cur_ua, - 1/* pages */, 1/* iswrite */, &page)) { + 1/* pages */, FOLL_WRITE, &page)) { ret = -EFAULT; for (j = 0; j < i; ++j) put_page(pfn_to_page(mem->hpas[j] >> @@ -209,7 +209,7 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, if (mm_iommu_move_page_from_cma(page)) goto populate; if (1 != get_user_pages_fast(cur_ua, - 1/* pages */, 1/* iswrite */, + 1/* pages */, FOLL_WRITE, &page)) { ret = -EFAULT; for (j = 0; j < i; ++j) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index fcb55b02990e..69d9366b966c 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2278,7 +2278,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) ret = -EFAULT; goto out; } - ret = get_user_pages_fast(map->addr, 1, 1, &map->page); + ret = get_user_pages_fast(map->addr, 1, FOLL_WRITE, &map->page); if (ret < 0) goto out; BUG_ON(ret != 1); diff --git a/arch/s390/mm/gup.c b/arch/s390/mm/gup.c index 2809d11c7a28..0a6faf3d9960 100644 --- a/arch/s390/mm/gup.c +++ b/arch/s390/mm/gup.c @@ -265,7 +265,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -277,22 +277,22 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { int nr, ret; might_sleep(); start &= PAGE_MASK; - nr = __get_user_pages_fast(start, nr_pages, write, pages); + nr = __get_user_pages_fast(start, nr_pages, gup_flags & FOLL_WRITE, + pages); if (nr = nr_pages) return nr; /* Try to get the remaining pages with get_user_pages */ start += nr << PAGE_SHIFT; pages += nr; - ret = get_user_pages_unlocked(start, nr_pages - nr, pages, - write ? FOLL_WRITE : 0); + ret = get_user_pages_unlocked(start, nr_pages - nr, pages, gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) ret = (ret < 0) ? nr : ret + nr; diff --git a/arch/sh/mm/gup.c b/arch/sh/mm/gup.c index 3e27f6d1f1ec..277c882f7489 100644 --- a/arch/sh/mm/gup.c +++ b/arch/sh/mm/gup.c @@ -204,7 +204,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -216,8 +216,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -241,7 +241,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); local_irq_enable(); @@ -261,7 +262,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c index aee6dba83d0e..1e770a517d4a 100644 --- a/arch/sparc/mm/gup.c +++ b/arch/sparc/mm/gup.c @@ -245,8 +245,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, return nr; } -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -303,7 +303,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); @@ -324,7 +325,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/x86/kvm/paging_tmpl.h b/arch/x86/kvm/paging_tmpl.h index 6bdca39829bc..08715034e315 100644 --- a/arch/x86/kvm/paging_tmpl.h +++ b/arch/x86/kvm/paging_tmpl.h @@ -140,7 +140,7 @@ static int FNAME(cmpxchg_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, pt_element_t *table; struct page *page; - npages = get_user_pages_fast((unsigned long)ptep_user, 1, 1, &page); + npages = get_user_pages_fast((unsigned long)ptep_user, 1, FOLL_WRITE, &page); /* Check if the user is doing something meaningless. */ if (unlikely(npages != 1)) return -EFAULT; diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index f13a3a24d360..173596a020cb 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1803,7 +1803,7 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, return NULL; /* Pin the user virtual address. */ - npinned = get_user_pages_fast(uaddr, npages, write ? FOLL_WRITE : 0, pages); + npinned = get_user_pages_fast(uaddr, npages, FOLL_WRITE, pages); if (npinned != npages) { pr_err("SEV: Failure locking %lu pages.\n", npages); goto err; diff --git a/drivers/fpga/dfl-afu-dma-region.c b/drivers/fpga/dfl-afu-dma-region.c index e18a786fc943..c438722bf4e1 100644 --- a/drivers/fpga/dfl-afu-dma-region.c +++ b/drivers/fpga/dfl-afu-dma-region.c @@ -102,7 +102,7 @@ static int afu_dma_pin_pages(struct dfl_feature_platform_data *pdata, goto unlock_vm; } - pinned = get_user_pages_fast(region->user_addr, npages, 1, + pinned = get_user_pages_fast(region->user_addr, npages, FOLL_WRITE, region->pages); if (pinned < 0) { ret = pinned; diff --git a/drivers/gpu/drm/via/via_dmablit.c b/drivers/gpu/drm/via/via_dmablit.c index 345bda4494e1..0c8b09602910 100644 --- a/drivers/gpu/drm/via/via_dmablit.c +++ b/drivers/gpu/drm/via/via_dmablit.c @@ -239,7 +239,8 @@ via_lock_all_dma_pages(drm_via_sg_info_t *vsg, drm_via_dmablit_t *xfer) if (NULL = vsg->pages) return -ENOMEM; ret = get_user_pages_fast((unsigned long)xfer->mem_addr, - vsg->num_pages, vsg->direction = DMA_FROM_DEVICE, + vsg->num_pages, + vsg->direction = DMA_FROM_DEVICE ? FOLL_WRITE : 0, vsg->pages); if (ret != vsg->num_pages) { if (ret < 0) diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index 24b592c6522e..78ccacaf97d0 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -105,7 +105,8 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np { int ret; - ret = get_user_pages_fast(vaddr, npages, writable, pages); + ret = get_user_pages_fast(vaddr, npages, writable ? FOLL_WRITE : 0, + pages); if (ret < 0) return ret; diff --git a/drivers/misc/genwqe/card_utils.c b/drivers/misc/genwqe/card_utils.c index 25265fd0fd6e..89cff9d1012b 100644 --- a/drivers/misc/genwqe/card_utils.c +++ b/drivers/misc/genwqe/card_utils.c @@ -603,7 +603,7 @@ int genwqe_user_vmap(struct genwqe_dev *cd, struct dma_mapping *m, void *uaddr, /* pin user pages in memory */ rc = get_user_pages_fast(data & PAGE_MASK, /* page aligned addr */ m->nr_pages, - m->write, /* readable/writable */ + m->write ? FOLL_WRITE : 0, /* readable/writable */ m->page_list); /* ptrs to pages */ if (rc < 0) goto fail_get_user_pages; diff --git a/drivers/misc/vmw_vmci/vmci_host.c b/drivers/misc/vmw_vmci/vmci_host.c index 997f92543dd4..422d08da3244 100644 --- a/drivers/misc/vmw_vmci/vmci_host.c +++ b/drivers/misc/vmw_vmci/vmci_host.c @@ -242,7 +242,7 @@ static int vmci_host_setup_notify(struct vmci_ctx *context, /* * Lock physical page backing a given user VA. */ - retval = get_user_pages_fast(uva, 1, 1, &context->notify_page); + retval = get_user_pages_fast(uva, 1, FOLL_WRITE, &context->notify_page); if (retval != 1) { context->notify_page = NULL; return VMCI_ERROR_GENERIC; diff --git a/drivers/misc/vmw_vmci/vmci_queue_pair.c b/drivers/misc/vmw_vmci/vmci_queue_pair.c index 264f4ed8eef2..c5396ee32e51 100644 --- a/drivers/misc/vmw_vmci/vmci_queue_pair.c +++ b/drivers/misc/vmw_vmci/vmci_queue_pair.c @@ -666,7 +666,8 @@ static int qp_host_get_user_memory(u64 produce_uva, int err = VMCI_SUCCESS; retval = get_user_pages_fast((uintptr_t) produce_uva, - produce_q->kernel_if->num_pages, 1, + produce_q->kernel_if->num_pages, + FOLL_WRITE, produce_q->kernel_if->u.h.header_page); if (retval < (int)produce_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(produce) failed (retval=%d)", @@ -678,7 +679,8 @@ static int qp_host_get_user_memory(u64 produce_uva, } retval = get_user_pages_fast((uintptr_t) consume_uva, - consume_q->kernel_if->num_pages, 1, + consume_q->kernel_if->num_pages, + FOLL_WRITE, consume_q->kernel_if->u.h.header_page); if (retval < (int)consume_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(consume) failed (retval=%d)", diff --git a/drivers/platform/goldfish/goldfish_pipe.c b/drivers/platform/goldfish/goldfish_pipe.c index 321bc673c417..cef0133aa47a 100644 --- a/drivers/platform/goldfish/goldfish_pipe.c +++ b/drivers/platform/goldfish/goldfish_pipe.c @@ -274,7 +274,8 @@ static int pin_user_pages(unsigned long first_page, *iter_last_page_size = last_page_size; } - ret = get_user_pages_fast(first_page, requested_pages, !is_write, + ret = get_user_pages_fast(first_page, requested_pages, + !is_write ? FOLL_WRITE : 0, pages); if (ret <= 0) return -EFAULT; diff --git a/drivers/rapidio/devices/rio_mport_cdev.c b/drivers/rapidio/devices/rio_mport_cdev.c index cbe467ff1aba..f681b3e9e970 100644 --- a/drivers/rapidio/devices/rio_mport_cdev.c +++ b/drivers/rapidio/devices/rio_mport_cdev.c @@ -868,7 +868,9 @@ rio_dma_transfer(struct file *filp, u32 transfer_mode, pinned = get_user_pages_fast( (unsigned long)xfer->loc_addr & PAGE_MASK, - nr_pages, dir = DMA_FROM_DEVICE, page_list); + nr_pages, + dir = DMA_FROM_DEVICE ? FOLL_WRITE : 0, + page_list); if (pinned != nr_pages) { if (pinned < 0) { diff --git a/drivers/sbus/char/oradax.c b/drivers/sbus/char/oradax.c index 6516bc3cb58b..790aa148670d 100644 --- a/drivers/sbus/char/oradax.c +++ b/drivers/sbus/char/oradax.c @@ -437,7 +437,7 @@ static int dax_lock_page(void *va, struct page **p) dax_dbg("uva %p", va); - ret = get_user_pages_fast((unsigned long)va, 1, 1, p); + ret = get_user_pages_fast((unsigned long)va, 1, FOLL_WRITE, p); if (ret = 1) { dax_dbg("locked page %p, for VA %p", *p, va); return 0; diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c index 7ff22d3f03e3..871b25914c07 100644 --- a/drivers/scsi/st.c +++ b/drivers/scsi/st.c @@ -4918,7 +4918,8 @@ static int sgl_map_user_pages(struct st_buffer *STbp, /* Try to fault in all of the necessary pages */ /* rw=READ means read from drive, write into memory area */ - res = get_user_pages_fast(uaddr, nr_pages, rw = READ, pages); + res = get_user_pages_fast(uaddr, nr_pages, rw = READ ? FOLL_WRITE : 0, + pages); /* Errors and no page mapped should return here */ if (res < nr_pages) diff --git a/drivers/staging/gasket/gasket_page_table.c b/drivers/staging/gasket/gasket_page_table.c index 26755d9ca41d..f67fdf1d3817 100644 --- a/drivers/staging/gasket/gasket_page_table.c +++ b/drivers/staging/gasket/gasket_page_table.c @@ -486,8 +486,8 @@ static int gasket_perform_mapping(struct gasket_page_table *pg_tbl, ptes[i].dma_addr = pg_tbl->coherent_pages[0].paddr + off + i * PAGE_SIZE; } else { - ret = get_user_pages_fast(page_addr - offset, 1, 1, - &page); + ret = get_user_pages_fast(page_addr - offset, 1, + FOLL_WRITE, &page); if (ret <= 0) { dev_err(pg_tbl->device, diff --git a/drivers/tee/tee_shm.c b/drivers/tee/tee_shm.c index 0b9ab1d0dd45..49fd7312e2aa 100644 --- a/drivers/tee/tee_shm.c +++ b/drivers/tee/tee_shm.c @@ -273,7 +273,7 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, goto err; } - rc = get_user_pages_fast(start, num_pages, 1, shm->pages); + rc = get_user_pages_fast(start, num_pages, FOLL_WRITE, shm->pages); if (rc > 0) shm->num_pages = rc; if (rc != num_pages) { diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c index c424913324e3..a4b10bb4086b 100644 --- a/drivers/vfio/vfio_iommu_spapr_tce.c +++ b/drivers/vfio/vfio_iommu_spapr_tce.c @@ -532,7 +532,8 @@ static int tce_iommu_use_page(unsigned long tce, unsigned long *hpa) enum dma_data_direction direction = iommu_tce_direction(tce); if (get_user_pages_fast(tce & PAGE_MASK, 1, - direction != DMA_TO_DEVICE, &page) != 1) + direction != DMA_TO_DEVICE ? FOLL_WRITE : 0, + &page) != 1) return -EFAULT; *hpa = __pa((unsigned long) page_address(page)); diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 24a129fcdd61..72685b1659ff 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1700,7 +1700,7 @@ static int set_bit_to_user(int nr, void __user *addr) int bit = nr + (log % PAGE_SIZE) * 8; int r; - r = get_user_pages_fast(log, 1, 1, &page); + r = get_user_pages_fast(log, 1, FOLL_WRITE, &page); if (r < 0) return r; BUG_ON(r != 1); diff --git a/drivers/video/fbdev/pvr2fb.c b/drivers/video/fbdev/pvr2fb.c index 8a53d1de611d..41390c8e0f67 100644 --- a/drivers/video/fbdev/pvr2fb.c +++ b/drivers/video/fbdev/pvr2fb.c @@ -686,7 +686,7 @@ static ssize_t pvr2fb_write(struct fb_info *info, const char *buf, if (!pages) return -ENOMEM; - ret = get_user_pages_fast((unsigned long)buf, nr_pages, true, pages); + ret = get_user_pages_fast((unsigned long)buf, nr_pages, FOLL_WRITE, pages); if (ret < nr_pages) { nr_pages = ret; ret = -EINVAL; diff --git a/drivers/virt/fsl_hypervisor.c b/drivers/virt/fsl_hypervisor.c index 8ba726e600e9..6446bcab4185 100644 --- a/drivers/virt/fsl_hypervisor.c +++ b/drivers/virt/fsl_hypervisor.c @@ -244,7 +244,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) /* Get the physical addresses of the source buffer */ num_pinned = get_user_pages_fast(param.local_vaddr - lb_offset, - num_pages, param.source != -1, pages); + num_pages, param.source != -1 ? FOLL_WRITE : 0, pages); if (num_pinned != num_pages) { /* get_user_pages() failed */ diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 5efc5eee9544..7b47f1e6aab4 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -852,7 +852,7 @@ static int gntdev_get_page(struct gntdev_copy_batch *batch, void __user *virt, unsigned long xen_pfn; int ret; - ret = get_user_pages_fast(addr, 1, writeable, &page); + ret = get_user_pages_fast(addr, 1, writeable ? FOLL_WRITE : 0, &page); if (ret < 0) return ret; diff --git a/fs/orangefs/orangefs-bufmap.c b/fs/orangefs/orangefs-bufmap.c index 443bcd8c3c19..5a7c4fda682f 100644 --- a/fs/orangefs/orangefs-bufmap.c +++ b/fs/orangefs/orangefs-bufmap.c @@ -269,7 +269,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap, /* map the pages */ ret = get_user_pages_fast((unsigned long)user_desc->ptr, - bufmap->page_count, 1, bufmap->page_array); + bufmap->page_count, FOLL_WRITE, bufmap->page_array); if (ret < 0) return ret; diff --git a/include/linux/mm.h b/include/linux/mm.h index 05a105d9d4c3..8e1f3cd7482a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1537,8 +1537,8 @@ long get_user_pages_locked(unsigned long start, unsigned long nr_pages, long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages); +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages); /* Container for pinned pfns / pages */ struct frame_vector { diff --git a/kernel/futex.c b/kernel/futex.c index fdd312da0992..e10209946f8b 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -546,7 +546,7 @@ get_futex_key(u32 __user *uaddr, int fshared, union futex_key *key, enum futex_a if (unlikely(should_fail_futex(fshared))) return -EFAULT; - err = get_user_pages_fast(address, 1, 1, &page); + err = get_user_pages_fast(address, 1, FOLL_WRITE, &page); /* * If write access is not required (eg. FUTEX_WAIT), try * and get read-only access. diff --git a/lib/iov_iter.c b/lib/iov_iter.c index be4bd627caf0..6dbae0692719 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1280,7 +1280,9 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, len = maxpages * PAGE_SIZE; addr &= ~(PAGE_SIZE - 1); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, iov_iter_rw(i) != WRITE, pages); + res = get_user_pages_fast(addr, n, + iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, + pages); if (unlikely(res < 0)) return res; return (res = n ? len : res * PAGE_SIZE) - *start; @@ -1361,7 +1363,8 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, iov_iter_rw(i) != WRITE, p); + res = get_user_pages_fast(addr, n, + iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, p); if (unlikely(res < 0)) { kvfree(p); return res; diff --git a/mm/gup.c b/mm/gup.c index 681388236106..6f32d36b3c5b 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1863,7 +1863,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -1875,8 +1875,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { unsigned long addr, len, end; int nr = 0, ret = 0; @@ -1894,7 +1894,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, if (gup_fast_permitted(start, nr_pages)) { local_irq_disable(); - gup_pgd_range(addr, end, write ? FOLL_WRITE : 0, pages, &nr); + gup_pgd_range(addr, end, gup_flags, pages, &nr); local_irq_enable(); ret = nr; } @@ -1905,7 +1905,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, pages += nr; ret = get_user_pages_unlocked(start, nr_pages - nr, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/mm/util.c b/mm/util.c index 1ea055138043..01ffe145c62b 100644 --- a/mm/util.c +++ b/mm/util.c @@ -306,7 +306,7 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast); * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -327,10 +327,10 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast); * get_user_pages_fast simply falls back to get_user_pages. */ int __weak get_user_pages_fast(unsigned long start, - int nr_pages, int write, struct page **pages) + int nr_pages, unsigned int gup_flags, + struct page **pages) { - return get_user_pages_unlocked(start, nr_pages, pages, - write ? FOLL_WRITE : 0); + return get_user_pages_unlocked(start, nr_pages, pages, gup_flags); } EXPORT_SYMBOL_GPL(get_user_pages_fast); diff --git a/net/ceph/pagevec.c b/net/ceph/pagevec.c index d3736f5bffec..74cafc0142ea 100644 --- a/net/ceph/pagevec.c +++ b/net/ceph/pagevec.c @@ -27,7 +27,7 @@ struct page **ceph_get_direct_page_vector(const void __user *data, while (got < num_pages) { rc = get_user_pages_fast( (unsigned long)data + ((unsigned long)got * PAGE_SIZE), - num_pages - got, write_page, pages + got); + num_pages - got, write_page ? FOLL_WRITE : 0, pages + got); if (rc < 0) break; BUG_ON(rc = 0); diff --git a/net/rds/info.c b/net/rds/info.c index e367a97a18c8..03f6fd56d237 100644 --- a/net/rds/info.c +++ b/net/rds/info.c @@ -193,7 +193,7 @@ int rds_info_getsockopt(struct socket *sock, int optname, char __user *optval, ret = -ENOMEM; goto out; } - ret = get_user_pages_fast(start, nr_pages, 1, pages); + ret = get_user_pages_fast(start, nr_pages, FOLL_WRITE, pages); if (ret != nr_pages) { if (ret > 0) nr_pages = ret; diff --git a/net/rds/rdma.c b/net/rds/rdma.c index 182ab8430594..b340ed4fc43a 100644 --- a/net/rds/rdma.c +++ b/net/rds/rdma.c @@ -158,7 +158,8 @@ static int rds_pin_pages(unsigned long user_addr, unsigned int nr_pages, { int ret; - ret = get_user_pages_fast(user_addr, nr_pages, write, pages); + ret = get_user_pages_fast(user_addr, nr_pages, write ? FOLL_WRITE : 0, + pages); if (ret >= 0 && ret < nr_pages) { while (ret--) -- 2.20.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: From: ira.weiny@intel.com Subject: [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool' Date: Wed, 13 Feb 2019 15:04:51 -0800 Message-Id: <20190213230455.5605-4-ira.weiny@intel.com> In-Reply-To: <20190213230455.5605-1-ira.weiny@intel.com> References: <20190211201643.7599-1-ira.weiny@intel.com> <20190213230455.5605-1-ira.weiny@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit To: linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, kvm@vger.kernel.org, linux-fpga@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-media@vger.kernel.org, linux-scsi@vger.kernel.org, devel@driverdev.osuosl.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-fbdev@vger.kernel.org, xen-devel@lists.xenproject.org, devel@lists.orangefs.org, linux-mm@kvack.org, ceph-devel@vger.kernel.org, rds-devel@oss.oracle.com Cc: Ira Weiny , John Hubbard , David Hildenbrand , Cornelia Huck , Yoshinori Sato , Rich Felker , "David S. Miller" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Joerg Roedel , Wu Hao , Alan Tull , Moritz Fischer , David Airlie , Daniel Vetter , Jason Gunthorpe , Dennis Dalessandro , Christian Benvenuti , Mauro Carvalho Chehab , Matt Porter , Alexandre Bounine , =?UTF-8?q?Kai=20M=C3=A4kisara?= , "James E.J. Bottomley" , "Martin K. Petersen" , Rob Springer , Todd Poynor , Ben Chan , Jens Wiklander , Alex Williamson , "Michael S. Tsirkin" , Jason Wang , Bartlomiej Zolnierkiewicz , Stefano Stabellini , Martin Brandenburg , Peter Zijlstra , Alexander Viro , Andrew Morton , Michal Hocko , "Kirill A. Shutemov" List-ID: From: Ira Weiny To facilitate additional options to get_user_pages_fast() change the singular write parameter to be gup_flags. This patch does not change any functionality. New functionality will follow in subsequent patches. Some of the get_user_pages_fast() call sites were unchanged because they already passed FOLL_WRITE or 0 for the write parameter. Signed-off-by: Ira Weiny --- arch/mips/mm/gup.c | 11 ++++++----- arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 ++-- arch/powerpc/kvm/e500_mmu.c | 2 +- arch/powerpc/mm/mmu_context_iommu.c | 4 ++-- arch/s390/kvm/interrupt.c | 2 +- arch/s390/mm/gup.c | 12 ++++++------ arch/sh/mm/gup.c | 11 ++++++----- arch/sparc/mm/gup.c | 9 +++++---- arch/x86/kvm/paging_tmpl.h | 2 +- arch/x86/kvm/svm.c | 2 +- drivers/fpga/dfl-afu-dma-region.c | 2 +- drivers/gpu/drm/via/via_dmablit.c | 3 ++- drivers/infiniband/hw/hfi1/user_pages.c | 3 ++- drivers/misc/genwqe/card_utils.c | 2 +- drivers/misc/vmw_vmci/vmci_host.c | 2 +- drivers/misc/vmw_vmci/vmci_queue_pair.c | 6 ++++-- drivers/platform/goldfish/goldfish_pipe.c | 3 ++- drivers/rapidio/devices/rio_mport_cdev.c | 4 +++- drivers/sbus/char/oradax.c | 2 +- drivers/scsi/st.c | 3 ++- drivers/staging/gasket/gasket_page_table.c | 4 ++-- drivers/tee/tee_shm.c | 2 +- drivers/vfio/vfio_iommu_spapr_tce.c | 3 ++- drivers/vhost/vhost.c | 2 +- drivers/video/fbdev/pvr2fb.c | 2 +- drivers/virt/fsl_hypervisor.c | 2 +- drivers/xen/gntdev.c | 2 +- fs/orangefs/orangefs-bufmap.c | 2 +- include/linux/mm.h | 4 ++-- kernel/futex.c | 2 +- lib/iov_iter.c | 7 +++++-- mm/gup.c | 10 +++++----- mm/util.c | 8 ++++---- net/ceph/pagevec.c | 2 +- net/rds/info.c | 2 +- net/rds/rdma.c | 3 ++- 36 files changed, 81 insertions(+), 65 deletions(-) diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c index 0d14e0d8eacf..4c2b4483683c 100644 --- a/arch/mips/mm/gup.c +++ b/arch/mips/mm/gup.c @@ -235,7 +235,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -247,8 +247,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -273,7 +273,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); local_irq_enable(); @@ -289,7 +290,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, pages += nr; ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, - pages, write ? FOLL_WRITE : 0); + pages, gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index bd2dcfbf00cd..8fcb0a921e46 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -582,7 +582,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, /* If writing != 0, then the HPTE must allow writing, if we get here */ write_ok = writing; hva = gfn_to_hva_memslot(memslot, gfn); - npages = get_user_pages_fast(hva, 1, writing, pages); + npages = get_user_pages_fast(hva, 1, writing ? FOLL_WRITE : 0, pages); if (npages < 1) { /* Check if it's an I/O mapping */ down_read(¤t->mm->mmap_sem); @@ -1175,7 +1175,7 @@ void *kvmppc_pin_guest_page(struct kvm *kvm, unsigned long gpa, if (!memslot || (memslot->flags & KVM_MEMSLOT_INVALID)) goto err; hva = gfn_to_hva_memslot(memslot, gfn); - npages = get_user_pages_fast(hva, 1, 1, pages); + npages = get_user_pages_fast(hva, 1, FOLL_WRITE, pages); if (npages < 1) goto err; page = pages[0]; diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c index 24296f4cadc6..e0af53fd78c5 100644 --- a/arch/powerpc/kvm/e500_mmu.c +++ b/arch/powerpc/kvm/e500_mmu.c @@ -783,7 +783,7 @@ int kvm_vcpu_ioctl_config_tlb(struct kvm_vcpu *vcpu, if (!pages) return -ENOMEM; - ret = get_user_pages_fast(cfg->array, num_pages, 1, pages); + ret = get_user_pages_fast(cfg->array, num_pages, FOLL_WRITE, pages); if (ret < 0) goto free_pages; diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index a712a650a8b6..acb0990c8364 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -190,7 +190,7 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, for (i = 0; i < entries; ++i) { cur_ua = ua + (i << PAGE_SHIFT); if (1 != get_user_pages_fast(cur_ua, - 1/* pages */, 1/* iswrite */, &page)) { + 1/* pages */, FOLL_WRITE, &page)) { ret = -EFAULT; for (j = 0; j < i; ++j) put_page(pfn_to_page(mem->hpas[j] >> @@ -209,7 +209,7 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, if (mm_iommu_move_page_from_cma(page)) goto populate; if (1 != get_user_pages_fast(cur_ua, - 1/* pages */, 1/* iswrite */, + 1/* pages */, FOLL_WRITE, &page)) { ret = -EFAULT; for (j = 0; j < i; ++j) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index fcb55b02990e..69d9366b966c 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2278,7 +2278,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) ret = -EFAULT; goto out; } - ret = get_user_pages_fast(map->addr, 1, 1, &map->page); + ret = get_user_pages_fast(map->addr, 1, FOLL_WRITE, &map->page); if (ret < 0) goto out; BUG_ON(ret != 1); diff --git a/arch/s390/mm/gup.c b/arch/s390/mm/gup.c index 2809d11c7a28..0a6faf3d9960 100644 --- a/arch/s390/mm/gup.c +++ b/arch/s390/mm/gup.c @@ -265,7 +265,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -277,22 +277,22 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { int nr, ret; might_sleep(); start &= PAGE_MASK; - nr = __get_user_pages_fast(start, nr_pages, write, pages); + nr = __get_user_pages_fast(start, nr_pages, gup_flags & FOLL_WRITE, + pages); if (nr == nr_pages) return nr; /* Try to get the remaining pages with get_user_pages */ start += nr << PAGE_SHIFT; pages += nr; - ret = get_user_pages_unlocked(start, nr_pages - nr, pages, - write ? FOLL_WRITE : 0); + ret = get_user_pages_unlocked(start, nr_pages - nr, pages, gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) ret = (ret < 0) ? nr : ret + nr; diff --git a/arch/sh/mm/gup.c b/arch/sh/mm/gup.c index 3e27f6d1f1ec..277c882f7489 100644 --- a/arch/sh/mm/gup.c +++ b/arch/sh/mm/gup.c @@ -204,7 +204,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -216,8 +216,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -241,7 +241,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); local_irq_enable(); @@ -261,7 +262,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c index aee6dba83d0e..1e770a517d4a 100644 --- a/arch/sparc/mm/gup.c +++ b/arch/sparc/mm/gup.c @@ -245,8 +245,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, return nr; } -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -303,7 +303,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); @@ -324,7 +325,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/x86/kvm/paging_tmpl.h b/arch/x86/kvm/paging_tmpl.h index 6bdca39829bc..08715034e315 100644 --- a/arch/x86/kvm/paging_tmpl.h +++ b/arch/x86/kvm/paging_tmpl.h @@ -140,7 +140,7 @@ static int FNAME(cmpxchg_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, pt_element_t *table; struct page *page; - npages = get_user_pages_fast((unsigned long)ptep_user, 1, 1, &page); + npages = get_user_pages_fast((unsigned long)ptep_user, 1, FOLL_WRITE, &page); /* Check if the user is doing something meaningless. */ if (unlikely(npages != 1)) return -EFAULT; diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index f13a3a24d360..173596a020cb 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1803,7 +1803,7 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, return NULL; /* Pin the user virtual address. */ - npinned = get_user_pages_fast(uaddr, npages, write ? FOLL_WRITE : 0, pages); + npinned = get_user_pages_fast(uaddr, npages, FOLL_WRITE, pages); if (npinned != npages) { pr_err("SEV: Failure locking %lu pages.\n", npages); goto err; diff --git a/drivers/fpga/dfl-afu-dma-region.c b/drivers/fpga/dfl-afu-dma-region.c index e18a786fc943..c438722bf4e1 100644 --- a/drivers/fpga/dfl-afu-dma-region.c +++ b/drivers/fpga/dfl-afu-dma-region.c @@ -102,7 +102,7 @@ static int afu_dma_pin_pages(struct dfl_feature_platform_data *pdata, goto unlock_vm; } - pinned = get_user_pages_fast(region->user_addr, npages, 1, + pinned = get_user_pages_fast(region->user_addr, npages, FOLL_WRITE, region->pages); if (pinned < 0) { ret = pinned; diff --git a/drivers/gpu/drm/via/via_dmablit.c b/drivers/gpu/drm/via/via_dmablit.c index 345bda4494e1..0c8b09602910 100644 --- a/drivers/gpu/drm/via/via_dmablit.c +++ b/drivers/gpu/drm/via/via_dmablit.c @@ -239,7 +239,8 @@ via_lock_all_dma_pages(drm_via_sg_info_t *vsg, drm_via_dmablit_t *xfer) if (NULL == vsg->pages) return -ENOMEM; ret = get_user_pages_fast((unsigned long)xfer->mem_addr, - vsg->num_pages, vsg->direction == DMA_FROM_DEVICE, + vsg->num_pages, + vsg->direction == DMA_FROM_DEVICE ? FOLL_WRITE : 0, vsg->pages); if (ret != vsg->num_pages) { if (ret < 0) diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index 24b592c6522e..78ccacaf97d0 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -105,7 +105,8 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np { int ret; - ret = get_user_pages_fast(vaddr, npages, writable, pages); + ret = get_user_pages_fast(vaddr, npages, writable ? FOLL_WRITE : 0, + pages); if (ret < 0) return ret; diff --git a/drivers/misc/genwqe/card_utils.c b/drivers/misc/genwqe/card_utils.c index 25265fd0fd6e..89cff9d1012b 100644 --- a/drivers/misc/genwqe/card_utils.c +++ b/drivers/misc/genwqe/card_utils.c @@ -603,7 +603,7 @@ int genwqe_user_vmap(struct genwqe_dev *cd, struct dma_mapping *m, void *uaddr, /* pin user pages in memory */ rc = get_user_pages_fast(data & PAGE_MASK, /* page aligned addr */ m->nr_pages, - m->write, /* readable/writable */ + m->write ? FOLL_WRITE : 0, /* readable/writable */ m->page_list); /* ptrs to pages */ if (rc < 0) goto fail_get_user_pages; diff --git a/drivers/misc/vmw_vmci/vmci_host.c b/drivers/misc/vmw_vmci/vmci_host.c index 997f92543dd4..422d08da3244 100644 --- a/drivers/misc/vmw_vmci/vmci_host.c +++ b/drivers/misc/vmw_vmci/vmci_host.c @@ -242,7 +242,7 @@ static int vmci_host_setup_notify(struct vmci_ctx *context, /* * Lock physical page backing a given user VA. */ - retval = get_user_pages_fast(uva, 1, 1, &context->notify_page); + retval = get_user_pages_fast(uva, 1, FOLL_WRITE, &context->notify_page); if (retval != 1) { context->notify_page = NULL; return VMCI_ERROR_GENERIC; diff --git a/drivers/misc/vmw_vmci/vmci_queue_pair.c b/drivers/misc/vmw_vmci/vmci_queue_pair.c index 264f4ed8eef2..c5396ee32e51 100644 --- a/drivers/misc/vmw_vmci/vmci_queue_pair.c +++ b/drivers/misc/vmw_vmci/vmci_queue_pair.c @@ -666,7 +666,8 @@ static int qp_host_get_user_memory(u64 produce_uva, int err = VMCI_SUCCESS; retval = get_user_pages_fast((uintptr_t) produce_uva, - produce_q->kernel_if->num_pages, 1, + produce_q->kernel_if->num_pages, + FOLL_WRITE, produce_q->kernel_if->u.h.header_page); if (retval < (int)produce_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(produce) failed (retval=%d)", @@ -678,7 +679,8 @@ static int qp_host_get_user_memory(u64 produce_uva, } retval = get_user_pages_fast((uintptr_t) consume_uva, - consume_q->kernel_if->num_pages, 1, + consume_q->kernel_if->num_pages, + FOLL_WRITE, consume_q->kernel_if->u.h.header_page); if (retval < (int)consume_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(consume) failed (retval=%d)", diff --git a/drivers/platform/goldfish/goldfish_pipe.c b/drivers/platform/goldfish/goldfish_pipe.c index 321bc673c417..cef0133aa47a 100644 --- a/drivers/platform/goldfish/goldfish_pipe.c +++ b/drivers/platform/goldfish/goldfish_pipe.c @@ -274,7 +274,8 @@ static int pin_user_pages(unsigned long first_page, *iter_last_page_size = last_page_size; } - ret = get_user_pages_fast(first_page, requested_pages, !is_write, + ret = get_user_pages_fast(first_page, requested_pages, + !is_write ? FOLL_WRITE : 0, pages); if (ret <= 0) return -EFAULT; diff --git a/drivers/rapidio/devices/rio_mport_cdev.c b/drivers/rapidio/devices/rio_mport_cdev.c index cbe467ff1aba..f681b3e9e970 100644 --- a/drivers/rapidio/devices/rio_mport_cdev.c +++ b/drivers/rapidio/devices/rio_mport_cdev.c @@ -868,7 +868,9 @@ rio_dma_transfer(struct file *filp, u32 transfer_mode, pinned = get_user_pages_fast( (unsigned long)xfer->loc_addr & PAGE_MASK, - nr_pages, dir == DMA_FROM_DEVICE, page_list); + nr_pages, + dir == DMA_FROM_DEVICE ? FOLL_WRITE : 0, + page_list); if (pinned != nr_pages) { if (pinned < 0) { diff --git a/drivers/sbus/char/oradax.c b/drivers/sbus/char/oradax.c index 6516bc3cb58b..790aa148670d 100644 --- a/drivers/sbus/char/oradax.c +++ b/drivers/sbus/char/oradax.c @@ -437,7 +437,7 @@ static int dax_lock_page(void *va, struct page **p) dax_dbg("uva %p", va); - ret = get_user_pages_fast((unsigned long)va, 1, 1, p); + ret = get_user_pages_fast((unsigned long)va, 1, FOLL_WRITE, p); if (ret == 1) { dax_dbg("locked page %p, for VA %p", *p, va); return 0; diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c index 7ff22d3f03e3..871b25914c07 100644 --- a/drivers/scsi/st.c +++ b/drivers/scsi/st.c @@ -4918,7 +4918,8 @@ static int sgl_map_user_pages(struct st_buffer *STbp, /* Try to fault in all of the necessary pages */ /* rw==READ means read from drive, write into memory area */ - res = get_user_pages_fast(uaddr, nr_pages, rw == READ, pages); + res = get_user_pages_fast(uaddr, nr_pages, rw == READ ? FOLL_WRITE : 0, + pages); /* Errors and no page mapped should return here */ if (res < nr_pages) diff --git a/drivers/staging/gasket/gasket_page_table.c b/drivers/staging/gasket/gasket_page_table.c index 26755d9ca41d..f67fdf1d3817 100644 --- a/drivers/staging/gasket/gasket_page_table.c +++ b/drivers/staging/gasket/gasket_page_table.c @@ -486,8 +486,8 @@ static int gasket_perform_mapping(struct gasket_page_table *pg_tbl, ptes[i].dma_addr = pg_tbl->coherent_pages[0].paddr + off + i * PAGE_SIZE; } else { - ret = get_user_pages_fast(page_addr - offset, 1, 1, - &page); + ret = get_user_pages_fast(page_addr - offset, 1, + FOLL_WRITE, &page); if (ret <= 0) { dev_err(pg_tbl->device, diff --git a/drivers/tee/tee_shm.c b/drivers/tee/tee_shm.c index 0b9ab1d0dd45..49fd7312e2aa 100644 --- a/drivers/tee/tee_shm.c +++ b/drivers/tee/tee_shm.c @@ -273,7 +273,7 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, goto err; } - rc = get_user_pages_fast(start, num_pages, 1, shm->pages); + rc = get_user_pages_fast(start, num_pages, FOLL_WRITE, shm->pages); if (rc > 0) shm->num_pages = rc; if (rc != num_pages) { diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c index c424913324e3..a4b10bb4086b 100644 --- a/drivers/vfio/vfio_iommu_spapr_tce.c +++ b/drivers/vfio/vfio_iommu_spapr_tce.c @@ -532,7 +532,8 @@ static int tce_iommu_use_page(unsigned long tce, unsigned long *hpa) enum dma_data_direction direction = iommu_tce_direction(tce); if (get_user_pages_fast(tce & PAGE_MASK, 1, - direction != DMA_TO_DEVICE, &page) != 1) + direction != DMA_TO_DEVICE ? FOLL_WRITE : 0, + &page) != 1) return -EFAULT; *hpa = __pa((unsigned long) page_address(page)); diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 24a129fcdd61..72685b1659ff 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1700,7 +1700,7 @@ static int set_bit_to_user(int nr, void __user *addr) int bit = nr + (log % PAGE_SIZE) * 8; int r; - r = get_user_pages_fast(log, 1, 1, &page); + r = get_user_pages_fast(log, 1, FOLL_WRITE, &page); if (r < 0) return r; BUG_ON(r != 1); diff --git a/drivers/video/fbdev/pvr2fb.c b/drivers/video/fbdev/pvr2fb.c index 8a53d1de611d..41390c8e0f67 100644 --- a/drivers/video/fbdev/pvr2fb.c +++ b/drivers/video/fbdev/pvr2fb.c @@ -686,7 +686,7 @@ static ssize_t pvr2fb_write(struct fb_info *info, const char *buf, if (!pages) return -ENOMEM; - ret = get_user_pages_fast((unsigned long)buf, nr_pages, true, pages); + ret = get_user_pages_fast((unsigned long)buf, nr_pages, FOLL_WRITE, pages); if (ret < nr_pages) { nr_pages = ret; ret = -EINVAL; diff --git a/drivers/virt/fsl_hypervisor.c b/drivers/virt/fsl_hypervisor.c index 8ba726e600e9..6446bcab4185 100644 --- a/drivers/virt/fsl_hypervisor.c +++ b/drivers/virt/fsl_hypervisor.c @@ -244,7 +244,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) /* Get the physical addresses of the source buffer */ num_pinned = get_user_pages_fast(param.local_vaddr - lb_offset, - num_pages, param.source != -1, pages); + num_pages, param.source != -1 ? FOLL_WRITE : 0, pages); if (num_pinned != num_pages) { /* get_user_pages() failed */ diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 5efc5eee9544..7b47f1e6aab4 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -852,7 +852,7 @@ static int gntdev_get_page(struct gntdev_copy_batch *batch, void __user *virt, unsigned long xen_pfn; int ret; - ret = get_user_pages_fast(addr, 1, writeable, &page); + ret = get_user_pages_fast(addr, 1, writeable ? FOLL_WRITE : 0, &page); if (ret < 0) return ret; diff --git a/fs/orangefs/orangefs-bufmap.c b/fs/orangefs/orangefs-bufmap.c index 443bcd8c3c19..5a7c4fda682f 100644 --- a/fs/orangefs/orangefs-bufmap.c +++ b/fs/orangefs/orangefs-bufmap.c @@ -269,7 +269,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap, /* map the pages */ ret = get_user_pages_fast((unsigned long)user_desc->ptr, - bufmap->page_count, 1, bufmap->page_array); + bufmap->page_count, FOLL_WRITE, bufmap->page_array); if (ret < 0) return ret; diff --git a/include/linux/mm.h b/include/linux/mm.h index 05a105d9d4c3..8e1f3cd7482a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1537,8 +1537,8 @@ long get_user_pages_locked(unsigned long start, unsigned long nr_pages, long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages); +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages); /* Container for pinned pfns / pages */ struct frame_vector { diff --git a/kernel/futex.c b/kernel/futex.c index fdd312da0992..e10209946f8b 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -546,7 +546,7 @@ get_futex_key(u32 __user *uaddr, int fshared, union futex_key *key, enum futex_a if (unlikely(should_fail_futex(fshared))) return -EFAULT; - err = get_user_pages_fast(address, 1, 1, &page); + err = get_user_pages_fast(address, 1, FOLL_WRITE, &page); /* * If write access is not required (eg. FUTEX_WAIT), try * and get read-only access. diff --git a/lib/iov_iter.c b/lib/iov_iter.c index be4bd627caf0..6dbae0692719 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1280,7 +1280,9 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, len = maxpages * PAGE_SIZE; addr &= ~(PAGE_SIZE - 1); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, iov_iter_rw(i) != WRITE, pages); + res = get_user_pages_fast(addr, n, + iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, + pages); if (unlikely(res < 0)) return res; return (res == n ? len : res * PAGE_SIZE) - *start; @@ -1361,7 +1363,8 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, iov_iter_rw(i) != WRITE, p); + res = get_user_pages_fast(addr, n, + iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, p); if (unlikely(res < 0)) { kvfree(p); return res; diff --git a/mm/gup.c b/mm/gup.c index 681388236106..6f32d36b3c5b 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1863,7 +1863,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -1875,8 +1875,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { unsigned long addr, len, end; int nr = 0, ret = 0; @@ -1894,7 +1894,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, if (gup_fast_permitted(start, nr_pages)) { local_irq_disable(); - gup_pgd_range(addr, end, write ? FOLL_WRITE : 0, pages, &nr); + gup_pgd_range(addr, end, gup_flags, pages, &nr); local_irq_enable(); ret = nr; } @@ -1905,7 +1905,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, pages += nr; ret = get_user_pages_unlocked(start, nr_pages - nr, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/mm/util.c b/mm/util.c index 1ea055138043..01ffe145c62b 100644 --- a/mm/util.c +++ b/mm/util.c @@ -306,7 +306,7 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast); * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -327,10 +327,10 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast); * get_user_pages_fast simply falls back to get_user_pages. */ int __weak get_user_pages_fast(unsigned long start, - int nr_pages, int write, struct page **pages) + int nr_pages, unsigned int gup_flags, + struct page **pages) { - return get_user_pages_unlocked(start, nr_pages, pages, - write ? FOLL_WRITE : 0); + return get_user_pages_unlocked(start, nr_pages, pages, gup_flags); } EXPORT_SYMBOL_GPL(get_user_pages_fast); diff --git a/net/ceph/pagevec.c b/net/ceph/pagevec.c index d3736f5bffec..74cafc0142ea 100644 --- a/net/ceph/pagevec.c +++ b/net/ceph/pagevec.c @@ -27,7 +27,7 @@ struct page **ceph_get_direct_page_vector(const void __user *data, while (got < num_pages) { rc = get_user_pages_fast( (unsigned long)data + ((unsigned long)got * PAGE_SIZE), - num_pages - got, write_page, pages + got); + num_pages - got, write_page ? FOLL_WRITE : 0, pages + got); if (rc < 0) break; BUG_ON(rc == 0); diff --git a/net/rds/info.c b/net/rds/info.c index e367a97a18c8..03f6fd56d237 100644 --- a/net/rds/info.c +++ b/net/rds/info.c @@ -193,7 +193,7 @@ int rds_info_getsockopt(struct socket *sock, int optname, char __user *optval, ret = -ENOMEM; goto out; } - ret = get_user_pages_fast(start, nr_pages, 1, pages); + ret = get_user_pages_fast(start, nr_pages, FOLL_WRITE, pages); if (ret != nr_pages) { if (ret > 0) nr_pages = ret; diff --git a/net/rds/rdma.c b/net/rds/rdma.c index 182ab8430594..b340ed4fc43a 100644 --- a/net/rds/rdma.c +++ b/net/rds/rdma.c @@ -158,7 +158,8 @@ static int rds_pin_pages(unsigned long user_addr, unsigned int nr_pages, { int ret; - ret = get_user_pages_fast(user_addr, nr_pages, write, pages); + ret = get_user_pages_fast(user_addr, nr_pages, write ? FOLL_WRITE : 0, + pages); if (ret >= 0 && ret < nr_pages) { while (ret--) -- 2.20.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 849F6C43381 for ; Wed, 13 Feb 2019 23:23:05 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 065F3218AC for ; Wed, 13 Feb 2019 23:23:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 065F3218AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 440Ftv11DNzDqGG for ; Thu, 14 Feb 2019 10:23:03 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=intel.com (client-ip=134.134.136.65; helo=mga03.intel.com; envelope-from=ira.weiny@intel.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 440FVX2j3CzDqVP for ; Thu, 14 Feb 2019 10:05:24 +1100 (AEDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Feb 2019 15:05:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,366,1544515200"; d="scan'208";a="138415605" Received: from iweiny-desk2.sc.intel.com ([10.3.52.157]) by orsmga001.jf.intel.com with ESMTP; 13 Feb 2019 15:05:21 -0800 From: ira.weiny@intel.com To: linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, kvm@vger.kernel.org, linux-fpga@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org, linux-media@vger.kernel.org, linux-scsi@vger.kernel.org, devel@driverdev.osuosl.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-fbdev@vger.kernel.org, xen-devel@lists.xenproject.org, devel@lists.orangefs.org, linux-mm@kvack.org, ceph-devel@vger.kernel.org, rds-devel@oss.oracle.com Subject: [PATCH V2 3/7] mm/gup: Change GUP fast to use flags rather than a write 'bool' Date: Wed, 13 Feb 2019 15:04:51 -0800 Message-Id: <20190213230455.5605-4-ira.weiny@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190213230455.5605-1-ira.weiny@intel.com> References: <20190211201643.7599-1-ira.weiny@intel.com> <20190213230455.5605-1-ira.weiny@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Thu, 14 Feb 2019 10:16:37 +1100 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Martin Brandenburg , Rich Felker , David Hildenbrand , David Airlie , Jason Wang , =?UTF-8?q?Kai=20M=C3=A4kisara?= , Michal Hocko , Ira Weiny , Ben Chan , Rob Springer , Todd Poynor , Yoshinori Sato , Joerg Roedel , "Michael S. Tsirkin" , Jason Gunthorpe , Ingo Molnar , Wu Hao , Alan Tull , John Hubbard , "James E.J. Bottomley" , Alex Williamson , Stefano Stabellini , Borislav Petkov , Alexander Viro , Thomas Gleixner , Mauro Carvalho Chehab , Peter Zijlstra , "Martin K. Petersen" , Dennis Dalessandro , Cornelia Huck , Jens Wiklander , Alexandre Bounine , Bartlomiej Zolnierkiewicz , Daniel Vetter , Moritz Fischer , Andrew Morton , "David S. Miller" , "Kirill A. Shutemov" , Christian Benvenuti Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Ira Weiny To facilitate additional options to get_user_pages_fast() change the singular write parameter to be gup_flags. This patch does not change any functionality. New functionality will follow in subsequent patches. Some of the get_user_pages_fast() call sites were unchanged because they already passed FOLL_WRITE or 0 for the write parameter. Signed-off-by: Ira Weiny --- arch/mips/mm/gup.c | 11 ++++++----- arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 ++-- arch/powerpc/kvm/e500_mmu.c | 2 +- arch/powerpc/mm/mmu_context_iommu.c | 4 ++-- arch/s390/kvm/interrupt.c | 2 +- arch/s390/mm/gup.c | 12 ++++++------ arch/sh/mm/gup.c | 11 ++++++----- arch/sparc/mm/gup.c | 9 +++++---- arch/x86/kvm/paging_tmpl.h | 2 +- arch/x86/kvm/svm.c | 2 +- drivers/fpga/dfl-afu-dma-region.c | 2 +- drivers/gpu/drm/via/via_dmablit.c | 3 ++- drivers/infiniband/hw/hfi1/user_pages.c | 3 ++- drivers/misc/genwqe/card_utils.c | 2 +- drivers/misc/vmw_vmci/vmci_host.c | 2 +- drivers/misc/vmw_vmci/vmci_queue_pair.c | 6 ++++-- drivers/platform/goldfish/goldfish_pipe.c | 3 ++- drivers/rapidio/devices/rio_mport_cdev.c | 4 +++- drivers/sbus/char/oradax.c | 2 +- drivers/scsi/st.c | 3 ++- drivers/staging/gasket/gasket_page_table.c | 4 ++-- drivers/tee/tee_shm.c | 2 +- drivers/vfio/vfio_iommu_spapr_tce.c | 3 ++- drivers/vhost/vhost.c | 2 +- drivers/video/fbdev/pvr2fb.c | 2 +- drivers/virt/fsl_hypervisor.c | 2 +- drivers/xen/gntdev.c | 2 +- fs/orangefs/orangefs-bufmap.c | 2 +- include/linux/mm.h | 4 ++-- kernel/futex.c | 2 +- lib/iov_iter.c | 7 +++++-- mm/gup.c | 10 +++++----- mm/util.c | 8 ++++---- net/ceph/pagevec.c | 2 +- net/rds/info.c | 2 +- net/rds/rdma.c | 3 ++- 36 files changed, 81 insertions(+), 65 deletions(-) diff --git a/arch/mips/mm/gup.c b/arch/mips/mm/gup.c index 0d14e0d8eacf..4c2b4483683c 100644 --- a/arch/mips/mm/gup.c +++ b/arch/mips/mm/gup.c @@ -235,7 +235,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -247,8 +247,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -273,7 +273,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); local_irq_enable(); @@ -289,7 +290,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, pages += nr; ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, - pages, write ? FOLL_WRITE : 0); + pages, gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index bd2dcfbf00cd..8fcb0a921e46 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -582,7 +582,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, /* If writing != 0, then the HPTE must allow writing, if we get here */ write_ok = writing; hva = gfn_to_hva_memslot(memslot, gfn); - npages = get_user_pages_fast(hva, 1, writing, pages); + npages = get_user_pages_fast(hva, 1, writing ? FOLL_WRITE : 0, pages); if (npages < 1) { /* Check if it's an I/O mapping */ down_read(¤t->mm->mmap_sem); @@ -1175,7 +1175,7 @@ void *kvmppc_pin_guest_page(struct kvm *kvm, unsigned long gpa, if (!memslot || (memslot->flags & KVM_MEMSLOT_INVALID)) goto err; hva = gfn_to_hva_memslot(memslot, gfn); - npages = get_user_pages_fast(hva, 1, 1, pages); + npages = get_user_pages_fast(hva, 1, FOLL_WRITE, pages); if (npages < 1) goto err; page = pages[0]; diff --git a/arch/powerpc/kvm/e500_mmu.c b/arch/powerpc/kvm/e500_mmu.c index 24296f4cadc6..e0af53fd78c5 100644 --- a/arch/powerpc/kvm/e500_mmu.c +++ b/arch/powerpc/kvm/e500_mmu.c @@ -783,7 +783,7 @@ int kvm_vcpu_ioctl_config_tlb(struct kvm_vcpu *vcpu, if (!pages) return -ENOMEM; - ret = get_user_pages_fast(cfg->array, num_pages, 1, pages); + ret = get_user_pages_fast(cfg->array, num_pages, FOLL_WRITE, pages); if (ret < 0) goto free_pages; diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index a712a650a8b6..acb0990c8364 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -190,7 +190,7 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, for (i = 0; i < entries; ++i) { cur_ua = ua + (i << PAGE_SHIFT); if (1 != get_user_pages_fast(cur_ua, - 1/* pages */, 1/* iswrite */, &page)) { + 1/* pages */, FOLL_WRITE, &page)) { ret = -EFAULT; for (j = 0; j < i; ++j) put_page(pfn_to_page(mem->hpas[j] >> @@ -209,7 +209,7 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, if (mm_iommu_move_page_from_cma(page)) goto populate; if (1 != get_user_pages_fast(cur_ua, - 1/* pages */, 1/* iswrite */, + 1/* pages */, FOLL_WRITE, &page)) { ret = -EFAULT; for (j = 0; j < i; ++j) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index fcb55b02990e..69d9366b966c 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2278,7 +2278,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) ret = -EFAULT; goto out; } - ret = get_user_pages_fast(map->addr, 1, 1, &map->page); + ret = get_user_pages_fast(map->addr, 1, FOLL_WRITE, &map->page); if (ret < 0) goto out; BUG_ON(ret != 1); diff --git a/arch/s390/mm/gup.c b/arch/s390/mm/gup.c index 2809d11c7a28..0a6faf3d9960 100644 --- a/arch/s390/mm/gup.c +++ b/arch/s390/mm/gup.c @@ -265,7 +265,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -277,22 +277,22 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { int nr, ret; might_sleep(); start &= PAGE_MASK; - nr = __get_user_pages_fast(start, nr_pages, write, pages); + nr = __get_user_pages_fast(start, nr_pages, gup_flags & FOLL_WRITE, + pages); if (nr == nr_pages) return nr; /* Try to get the remaining pages with get_user_pages */ start += nr << PAGE_SHIFT; pages += nr; - ret = get_user_pages_unlocked(start, nr_pages - nr, pages, - write ? FOLL_WRITE : 0); + ret = get_user_pages_unlocked(start, nr_pages - nr, pages, gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) ret = (ret < 0) ? nr : ret + nr; diff --git a/arch/sh/mm/gup.c b/arch/sh/mm/gup.c index 3e27f6d1f1ec..277c882f7489 100644 --- a/arch/sh/mm/gup.c +++ b/arch/sh/mm/gup.c @@ -204,7 +204,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -216,8 +216,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -241,7 +241,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); local_irq_enable(); @@ -261,7 +262,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c index aee6dba83d0e..1e770a517d4a 100644 --- a/arch/sparc/mm/gup.c +++ b/arch/sparc/mm/gup.c @@ -245,8 +245,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, return nr; } -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { struct mm_struct *mm = current->mm; unsigned long addr, len, end; @@ -303,7 +303,8 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) goto slow; - if (!gup_pud_range(pgd, addr, next, write, pages, &nr)) + if (!gup_pud_range(pgd, addr, next, gup_flags & FOLL_WRITE, + pages, &nr)) goto slow; } while (pgdp++, addr = next, addr != end); @@ -324,7 +325,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, ret = get_user_pages_unlocked(start, (end - start) >> PAGE_SHIFT, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/arch/x86/kvm/paging_tmpl.h b/arch/x86/kvm/paging_tmpl.h index 6bdca39829bc..08715034e315 100644 --- a/arch/x86/kvm/paging_tmpl.h +++ b/arch/x86/kvm/paging_tmpl.h @@ -140,7 +140,7 @@ static int FNAME(cmpxchg_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, pt_element_t *table; struct page *page; - npages = get_user_pages_fast((unsigned long)ptep_user, 1, 1, &page); + npages = get_user_pages_fast((unsigned long)ptep_user, 1, FOLL_WRITE, &page); /* Check if the user is doing something meaningless. */ if (unlikely(npages != 1)) return -EFAULT; diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index f13a3a24d360..173596a020cb 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1803,7 +1803,7 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, return NULL; /* Pin the user virtual address. */ - npinned = get_user_pages_fast(uaddr, npages, write ? FOLL_WRITE : 0, pages); + npinned = get_user_pages_fast(uaddr, npages, FOLL_WRITE, pages); if (npinned != npages) { pr_err("SEV: Failure locking %lu pages.\n", npages); goto err; diff --git a/drivers/fpga/dfl-afu-dma-region.c b/drivers/fpga/dfl-afu-dma-region.c index e18a786fc943..c438722bf4e1 100644 --- a/drivers/fpga/dfl-afu-dma-region.c +++ b/drivers/fpga/dfl-afu-dma-region.c @@ -102,7 +102,7 @@ static int afu_dma_pin_pages(struct dfl_feature_platform_data *pdata, goto unlock_vm; } - pinned = get_user_pages_fast(region->user_addr, npages, 1, + pinned = get_user_pages_fast(region->user_addr, npages, FOLL_WRITE, region->pages); if (pinned < 0) { ret = pinned; diff --git a/drivers/gpu/drm/via/via_dmablit.c b/drivers/gpu/drm/via/via_dmablit.c index 345bda4494e1..0c8b09602910 100644 --- a/drivers/gpu/drm/via/via_dmablit.c +++ b/drivers/gpu/drm/via/via_dmablit.c @@ -239,7 +239,8 @@ via_lock_all_dma_pages(drm_via_sg_info_t *vsg, drm_via_dmablit_t *xfer) if (NULL == vsg->pages) return -ENOMEM; ret = get_user_pages_fast((unsigned long)xfer->mem_addr, - vsg->num_pages, vsg->direction == DMA_FROM_DEVICE, + vsg->num_pages, + vsg->direction == DMA_FROM_DEVICE ? FOLL_WRITE : 0, vsg->pages); if (ret != vsg->num_pages) { if (ret < 0) diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index 24b592c6522e..78ccacaf97d0 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -105,7 +105,8 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np { int ret; - ret = get_user_pages_fast(vaddr, npages, writable, pages); + ret = get_user_pages_fast(vaddr, npages, writable ? FOLL_WRITE : 0, + pages); if (ret < 0) return ret; diff --git a/drivers/misc/genwqe/card_utils.c b/drivers/misc/genwqe/card_utils.c index 25265fd0fd6e..89cff9d1012b 100644 --- a/drivers/misc/genwqe/card_utils.c +++ b/drivers/misc/genwqe/card_utils.c @@ -603,7 +603,7 @@ int genwqe_user_vmap(struct genwqe_dev *cd, struct dma_mapping *m, void *uaddr, /* pin user pages in memory */ rc = get_user_pages_fast(data & PAGE_MASK, /* page aligned addr */ m->nr_pages, - m->write, /* readable/writable */ + m->write ? FOLL_WRITE : 0, /* readable/writable */ m->page_list); /* ptrs to pages */ if (rc < 0) goto fail_get_user_pages; diff --git a/drivers/misc/vmw_vmci/vmci_host.c b/drivers/misc/vmw_vmci/vmci_host.c index 997f92543dd4..422d08da3244 100644 --- a/drivers/misc/vmw_vmci/vmci_host.c +++ b/drivers/misc/vmw_vmci/vmci_host.c @@ -242,7 +242,7 @@ static int vmci_host_setup_notify(struct vmci_ctx *context, /* * Lock physical page backing a given user VA. */ - retval = get_user_pages_fast(uva, 1, 1, &context->notify_page); + retval = get_user_pages_fast(uva, 1, FOLL_WRITE, &context->notify_page); if (retval != 1) { context->notify_page = NULL; return VMCI_ERROR_GENERIC; diff --git a/drivers/misc/vmw_vmci/vmci_queue_pair.c b/drivers/misc/vmw_vmci/vmci_queue_pair.c index 264f4ed8eef2..c5396ee32e51 100644 --- a/drivers/misc/vmw_vmci/vmci_queue_pair.c +++ b/drivers/misc/vmw_vmci/vmci_queue_pair.c @@ -666,7 +666,8 @@ static int qp_host_get_user_memory(u64 produce_uva, int err = VMCI_SUCCESS; retval = get_user_pages_fast((uintptr_t) produce_uva, - produce_q->kernel_if->num_pages, 1, + produce_q->kernel_if->num_pages, + FOLL_WRITE, produce_q->kernel_if->u.h.header_page); if (retval < (int)produce_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(produce) failed (retval=%d)", @@ -678,7 +679,8 @@ static int qp_host_get_user_memory(u64 produce_uva, } retval = get_user_pages_fast((uintptr_t) consume_uva, - consume_q->kernel_if->num_pages, 1, + consume_q->kernel_if->num_pages, + FOLL_WRITE, consume_q->kernel_if->u.h.header_page); if (retval < (int)consume_q->kernel_if->num_pages) { pr_debug("get_user_pages_fast(consume) failed (retval=%d)", diff --git a/drivers/platform/goldfish/goldfish_pipe.c b/drivers/platform/goldfish/goldfish_pipe.c index 321bc673c417..cef0133aa47a 100644 --- a/drivers/platform/goldfish/goldfish_pipe.c +++ b/drivers/platform/goldfish/goldfish_pipe.c @@ -274,7 +274,8 @@ static int pin_user_pages(unsigned long first_page, *iter_last_page_size = last_page_size; } - ret = get_user_pages_fast(first_page, requested_pages, !is_write, + ret = get_user_pages_fast(first_page, requested_pages, + !is_write ? FOLL_WRITE : 0, pages); if (ret <= 0) return -EFAULT; diff --git a/drivers/rapidio/devices/rio_mport_cdev.c b/drivers/rapidio/devices/rio_mport_cdev.c index cbe467ff1aba..f681b3e9e970 100644 --- a/drivers/rapidio/devices/rio_mport_cdev.c +++ b/drivers/rapidio/devices/rio_mport_cdev.c @@ -868,7 +868,9 @@ rio_dma_transfer(struct file *filp, u32 transfer_mode, pinned = get_user_pages_fast( (unsigned long)xfer->loc_addr & PAGE_MASK, - nr_pages, dir == DMA_FROM_DEVICE, page_list); + nr_pages, + dir == DMA_FROM_DEVICE ? FOLL_WRITE : 0, + page_list); if (pinned != nr_pages) { if (pinned < 0) { diff --git a/drivers/sbus/char/oradax.c b/drivers/sbus/char/oradax.c index 6516bc3cb58b..790aa148670d 100644 --- a/drivers/sbus/char/oradax.c +++ b/drivers/sbus/char/oradax.c @@ -437,7 +437,7 @@ static int dax_lock_page(void *va, struct page **p) dax_dbg("uva %p", va); - ret = get_user_pages_fast((unsigned long)va, 1, 1, p); + ret = get_user_pages_fast((unsigned long)va, 1, FOLL_WRITE, p); if (ret == 1) { dax_dbg("locked page %p, for VA %p", *p, va); return 0; diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c index 7ff22d3f03e3..871b25914c07 100644 --- a/drivers/scsi/st.c +++ b/drivers/scsi/st.c @@ -4918,7 +4918,8 @@ static int sgl_map_user_pages(struct st_buffer *STbp, /* Try to fault in all of the necessary pages */ /* rw==READ means read from drive, write into memory area */ - res = get_user_pages_fast(uaddr, nr_pages, rw == READ, pages); + res = get_user_pages_fast(uaddr, nr_pages, rw == READ ? FOLL_WRITE : 0, + pages); /* Errors and no page mapped should return here */ if (res < nr_pages) diff --git a/drivers/staging/gasket/gasket_page_table.c b/drivers/staging/gasket/gasket_page_table.c index 26755d9ca41d..f67fdf1d3817 100644 --- a/drivers/staging/gasket/gasket_page_table.c +++ b/drivers/staging/gasket/gasket_page_table.c @@ -486,8 +486,8 @@ static int gasket_perform_mapping(struct gasket_page_table *pg_tbl, ptes[i].dma_addr = pg_tbl->coherent_pages[0].paddr + off + i * PAGE_SIZE; } else { - ret = get_user_pages_fast(page_addr - offset, 1, 1, - &page); + ret = get_user_pages_fast(page_addr - offset, 1, + FOLL_WRITE, &page); if (ret <= 0) { dev_err(pg_tbl->device, diff --git a/drivers/tee/tee_shm.c b/drivers/tee/tee_shm.c index 0b9ab1d0dd45..49fd7312e2aa 100644 --- a/drivers/tee/tee_shm.c +++ b/drivers/tee/tee_shm.c @@ -273,7 +273,7 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, goto err; } - rc = get_user_pages_fast(start, num_pages, 1, shm->pages); + rc = get_user_pages_fast(start, num_pages, FOLL_WRITE, shm->pages); if (rc > 0) shm->num_pages = rc; if (rc != num_pages) { diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c index c424913324e3..a4b10bb4086b 100644 --- a/drivers/vfio/vfio_iommu_spapr_tce.c +++ b/drivers/vfio/vfio_iommu_spapr_tce.c @@ -532,7 +532,8 @@ static int tce_iommu_use_page(unsigned long tce, unsigned long *hpa) enum dma_data_direction direction = iommu_tce_direction(tce); if (get_user_pages_fast(tce & PAGE_MASK, 1, - direction != DMA_TO_DEVICE, &page) != 1) + direction != DMA_TO_DEVICE ? FOLL_WRITE : 0, + &page) != 1) return -EFAULT; *hpa = __pa((unsigned long) page_address(page)); diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 24a129fcdd61..72685b1659ff 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1700,7 +1700,7 @@ static int set_bit_to_user(int nr, void __user *addr) int bit = nr + (log % PAGE_SIZE) * 8; int r; - r = get_user_pages_fast(log, 1, 1, &page); + r = get_user_pages_fast(log, 1, FOLL_WRITE, &page); if (r < 0) return r; BUG_ON(r != 1); diff --git a/drivers/video/fbdev/pvr2fb.c b/drivers/video/fbdev/pvr2fb.c index 8a53d1de611d..41390c8e0f67 100644 --- a/drivers/video/fbdev/pvr2fb.c +++ b/drivers/video/fbdev/pvr2fb.c @@ -686,7 +686,7 @@ static ssize_t pvr2fb_write(struct fb_info *info, const char *buf, if (!pages) return -ENOMEM; - ret = get_user_pages_fast((unsigned long)buf, nr_pages, true, pages); + ret = get_user_pages_fast((unsigned long)buf, nr_pages, FOLL_WRITE, pages); if (ret < nr_pages) { nr_pages = ret; ret = -EINVAL; diff --git a/drivers/virt/fsl_hypervisor.c b/drivers/virt/fsl_hypervisor.c index 8ba726e600e9..6446bcab4185 100644 --- a/drivers/virt/fsl_hypervisor.c +++ b/drivers/virt/fsl_hypervisor.c @@ -244,7 +244,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p) /* Get the physical addresses of the source buffer */ num_pinned = get_user_pages_fast(param.local_vaddr - lb_offset, - num_pages, param.source != -1, pages); + num_pages, param.source != -1 ? FOLL_WRITE : 0, pages); if (num_pinned != num_pages) { /* get_user_pages() failed */ diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 5efc5eee9544..7b47f1e6aab4 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -852,7 +852,7 @@ static int gntdev_get_page(struct gntdev_copy_batch *batch, void __user *virt, unsigned long xen_pfn; int ret; - ret = get_user_pages_fast(addr, 1, writeable, &page); + ret = get_user_pages_fast(addr, 1, writeable ? FOLL_WRITE : 0, &page); if (ret < 0) return ret; diff --git a/fs/orangefs/orangefs-bufmap.c b/fs/orangefs/orangefs-bufmap.c index 443bcd8c3c19..5a7c4fda682f 100644 --- a/fs/orangefs/orangefs-bufmap.c +++ b/fs/orangefs/orangefs-bufmap.c @@ -269,7 +269,7 @@ orangefs_bufmap_map(struct orangefs_bufmap *bufmap, /* map the pages */ ret = get_user_pages_fast((unsigned long)user_desc->ptr, - bufmap->page_count, 1, bufmap->page_array); + bufmap->page_count, FOLL_WRITE, bufmap->page_array); if (ret < 0) return ret; diff --git a/include/linux/mm.h b/include/linux/mm.h index 05a105d9d4c3..8e1f3cd7482a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1537,8 +1537,8 @@ long get_user_pages_locked(unsigned long start, unsigned long nr_pages, long get_user_pages_unlocked(unsigned long start, unsigned long nr_pages, struct page **pages, unsigned int gup_flags); -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages); +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages); /* Container for pinned pfns / pages */ struct frame_vector { diff --git a/kernel/futex.c b/kernel/futex.c index fdd312da0992..e10209946f8b 100644 --- a/kernel/futex.c +++ b/kernel/futex.c @@ -546,7 +546,7 @@ get_futex_key(u32 __user *uaddr, int fshared, union futex_key *key, enum futex_a if (unlikely(should_fail_futex(fshared))) return -EFAULT; - err = get_user_pages_fast(address, 1, 1, &page); + err = get_user_pages_fast(address, 1, FOLL_WRITE, &page); /* * If write access is not required (eg. FUTEX_WAIT), try * and get read-only access. diff --git a/lib/iov_iter.c b/lib/iov_iter.c index be4bd627caf0..6dbae0692719 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1280,7 +1280,9 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, len = maxpages * PAGE_SIZE; addr &= ~(PAGE_SIZE - 1); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, iov_iter_rw(i) != WRITE, pages); + res = get_user_pages_fast(addr, n, + iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, + pages); if (unlikely(res < 0)) return res; return (res == n ? len : res * PAGE_SIZE) - *start; @@ -1361,7 +1363,8 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, iov_iter_rw(i) != WRITE, p); + res = get_user_pages_fast(addr, n, + iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, p); if (unlikely(res < 0)) { kvfree(p); return res; diff --git a/mm/gup.c b/mm/gup.c index 681388236106..6f32d36b3c5b 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1863,7 +1863,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -1875,8 +1875,8 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, * requested. If nr_pages is 0 or negative, returns 0. If no pages * were pinned, returns -errno. */ -int get_user_pages_fast(unsigned long start, int nr_pages, int write, - struct page **pages) +int get_user_pages_fast(unsigned long start, int nr_pages, + unsigned int gup_flags, struct page **pages) { unsigned long addr, len, end; int nr = 0, ret = 0; @@ -1894,7 +1894,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, if (gup_fast_permitted(start, nr_pages)) { local_irq_disable(); - gup_pgd_range(addr, end, write ? FOLL_WRITE : 0, pages, &nr); + gup_pgd_range(addr, end, gup_flags, pages, &nr); local_irq_enable(); ret = nr; } @@ -1905,7 +1905,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write, pages += nr; ret = get_user_pages_unlocked(start, nr_pages - nr, pages, - write ? FOLL_WRITE : 0); + gup_flags); /* Have to be a bit careful with return values */ if (nr > 0) { diff --git a/mm/util.c b/mm/util.c index 1ea055138043..01ffe145c62b 100644 --- a/mm/util.c +++ b/mm/util.c @@ -306,7 +306,7 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast); * get_user_pages_fast() - pin user pages in memory * @start: starting user address * @nr_pages: number of pages from start to pin - * @write: whether pages will be written to + * @gup_flags: flags modifying pin behaviour * @pages: array that receives pointers to the pages pinned. * Should be at least nr_pages long. * @@ -327,10 +327,10 @@ EXPORT_SYMBOL_GPL(__get_user_pages_fast); * get_user_pages_fast simply falls back to get_user_pages. */ int __weak get_user_pages_fast(unsigned long start, - int nr_pages, int write, struct page **pages) + int nr_pages, unsigned int gup_flags, + struct page **pages) { - return get_user_pages_unlocked(start, nr_pages, pages, - write ? FOLL_WRITE : 0); + return get_user_pages_unlocked(start, nr_pages, pages, gup_flags); } EXPORT_SYMBOL_GPL(get_user_pages_fast); diff --git a/net/ceph/pagevec.c b/net/ceph/pagevec.c index d3736f5bffec..74cafc0142ea 100644 --- a/net/ceph/pagevec.c +++ b/net/ceph/pagevec.c @@ -27,7 +27,7 @@ struct page **ceph_get_direct_page_vector(const void __user *data, while (got < num_pages) { rc = get_user_pages_fast( (unsigned long)data + ((unsigned long)got * PAGE_SIZE), - num_pages - got, write_page, pages + got); + num_pages - got, write_page ? FOLL_WRITE : 0, pages + got); if (rc < 0) break; BUG_ON(rc == 0); diff --git a/net/rds/info.c b/net/rds/info.c index e367a97a18c8..03f6fd56d237 100644 --- a/net/rds/info.c +++ b/net/rds/info.c @@ -193,7 +193,7 @@ int rds_info_getsockopt(struct socket *sock, int optname, char __user *optval, ret = -ENOMEM; goto out; } - ret = get_user_pages_fast(start, nr_pages, 1, pages); + ret = get_user_pages_fast(start, nr_pages, FOLL_WRITE, pages); if (ret != nr_pages) { if (ret > 0) nr_pages = ret; diff --git a/net/rds/rdma.c b/net/rds/rdma.c index 182ab8430594..b340ed4fc43a 100644 --- a/net/rds/rdma.c +++ b/net/rds/rdma.c @@ -158,7 +158,8 @@ static int rds_pin_pages(unsigned long user_addr, unsigned int nr_pages, { int ret; - ret = get_user_pages_fast(user_addr, nr_pages, write, pages); + ret = get_user_pages_fast(user_addr, nr_pages, write ? FOLL_WRITE : 0, + pages); if (ret >= 0 && ret < nr_pages) { while (ret--) -- 2.20.1