From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18663C47082 for ; Mon, 31 May 2021 17:23:04 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C489E610FC for ; Mon, 31 May 2021 17:23:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C489E610FC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 717866E96E; Mon, 31 May 2021 17:22:51 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 471A26E96C; Mon, 31 May 2021 17:22:49 +0000 (UTC) IronPort-SDR: Dfs6vwlEkP0xkWVUCan6whSQ26/d7rGVlpXfEMRnyKYL/AErGI1gYTFcA94a0+z4jq0gyIaOjD EmoykgAHCGBQ== X-IronPort-AV: E=McAfee;i="6200,9189,10001"; a="201520206" X-IronPort-AV: E=Sophos;i="5.83,237,1616482800"; d="scan'208";a="201520206" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2021 10:22:48 -0700 IronPort-SDR: Uz8DGq9v3SWEoo2gQLFRuu+2VQo2VsBUdEPubvqGYh1Tq9q2Lmcqap6g/Ie25oAVHUDEOfYol3 jD5NgMkFvyoA== X-IronPort-AV: E=Sophos;i="5.83,237,1616482800"; d="scan'208";a="416210992" Received: from fnygreen-mobl1.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.142]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2021 10:22:47 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Mon, 31 May 2021 19:22:19 +0200 Message-Id: <20210531172228.70846-7-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210531172228.70846-1-thomas.hellstrom@linux.intel.com> References: <20210531172228.70846-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v8 06/15] drm/ttm: Add a generic TTM memcpy move for page-based iomem X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , =?UTF-8?q?Christian=20K=C3=B6nig?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" VGhlIGludGVybmFsIHR0bV9ib191dGlsIG1lbWNweSB1c2VzIGlvcmVtYXAgZnVuY3Rpb25hbGl0 eSwgYW5kIHdoaWxlIGl0CnByb2JhYmx5IG1pZ2h0IGJlIHBvc3NpYmxlIHRvIHVzZSBpdCBmb3Ig Y29weWluZyBpbi0gYW5kIG91dCBvZgpzZ2xpc3QgcmVwcmVzZW50ZWQgaW8gbWVtb3J5LCB1c2lu ZyBpb19tZW1fcmVzZXJ2ZSgpIC8gaW9fbWVtX2ZyZWUoKQpjYWxsYmFja3MsIHRoYXQgd291bGQg Y2F1c2UgcHJvYmxlbXMgd2l0aCBmYXVsdCgpLgpJbnN0ZWFkLCBpbXBsZW1lbnQgYSBtZXRob2Qg bWFwcGluZyBwYWdlLWJ5LXBhZ2UgdXNpbmcga21hcF9sb2NhbCgpCnNlbWFudGljcy4gQXMgYW4g YWRkaXRpb25hbCBiZW5lZml0IHdlIHRoZW4gYXZvaWQgdGhlIG9jY2FzaW9uYWwgZ2xvYmFsClRM QiBmbHVzaGVzIG9mIGlvcmVtYXAoKSBhbmQgY29uc3VtaW5nIGlvcmVtYXAgc3BhY2UsIGVsaW1p bmF0aW9uIG9mIGEKY3JpdGljYWwgcG9pbnQgb2YgZmFpbHVyZSBhbmQgd2l0aCBhIHNsaWdodCBj aGFuZ2Ugb2Ygc2VtYW50aWNzIHdlIGNvdWxkCmFsc28gcHVzaCB0aGUgbWVtY3B5IG91dCBhc3lu YyBmb3IgdGVzdGluZyBhbmQgYXN5bmMgZHJpdmVyIGRldmVsb3BtZW50CnB1cnBvc2VzLgoKQSBz cGVjaWFsIGxpbmVhciBpb21lbSBpdGVyYXRvciBpcyBpbnRyb2R1Y2VkIGludGVybmFsbHkgdG8g bWltaWMgdGhlCm9sZCBpb3JlbWFwIGJlaGF2aW91ciBmb3IgY29kZS1wYXRocyB0aGF0IGNhbid0 IGltbWVkaWF0ZWx5IGJlIHBvcnRlZApvdmVyLiBUaGlzIGFkZHMgdG8gdGhlIGNvZGUgc2l6ZSBh bmQgc2hvdWxkIGJlIGNvbnNpZGVyZWQgYSB0ZW1wb3JhcnkKc29sdXRpb24uCgpMb29raW5nIGF0 IHRoZSBjb2RlIHdlIGhhdmUgYSBsb3Qgb2YgY2hlY2tzIGZvciBpb21hcCB0YWdnZWQgcG9pbnRl cnMuCklkZWFsbHkgd2Ugc2hvdWxkIGV4dGVuZCB0aGUgY29yZSBtZW1yZW1hcCBmdW5jdGlvbnMg dG8gYWxzbyBhY2NlcHQKdW5jYWNoZWQgbWVtb3J5IGFuZCBrbWFwX2xvY2FsIGZ1bmN0aW9uYWxp dHkuIFRoZW4gd2UgY291bGQgc3RyaXAgYQpsb3Qgb2YgY29kZS4KCkNjOiBDaHJpc3RpYW4gS8O2 bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+ClNpZ25lZC1vZmYtYnk6IFRob21hcyBIZWxs c3Ryw7ZtIDx0aG9tYXMuaGVsbHN0cm9tQGxpbnV4LmludGVsLmNvbT4KUmV2aWV3ZWQtYnk6IENo cmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4KLS0tCnYzOgotIFNwbGl0 IHVwIGluIHZhcmlvdXMgVFRNIGZpbGVzIGFuZCBhZGRyZXNzZWQgcmV2aWV3IGNvbW1lbnRzIGJ5 CiAgQ2hyaXN0aWFuIEvDtm5pZy4gVGVzdGVkIGFuZCBmaXhlZCBsZWdhY3kgaW9tYXAgbWVtY3B5 IHBhdGggb24gaTkxNS4KdjQ6Ci0gRml4IGFuIHVuaW5pdGlhbGl6ZWQgdmFyaWFibGUKICBSZXBv cnRlZCBieToga2VybmVsIHRlc3Qgcm9ib3QgPGxrcEBpbnRlbC5jb20+CiAgUmVwb3J0ZWQgYnk6 IERhbiBDYXJwZW50ZXIgPGRhbi5jYXJwZW50ZXJAb3JhY2xlLmNvbT4KLSBNaW5vciBjaGFuZ2Ug dG8gdGhlIHR0bV9tb3ZlX21lbWNweSgpIGludGVyZmFjZS4KLSBHcmFjZWZ1bGx5IGhhbmRsZSBs YWNrIG9mIG1lbXJlbWFwKCkgc3VwcG9ydCBvbiBtZW1jcHkKICAoUmVwb3J0ZWQgYnkgTWF0dGhl dyBBdWxkKQotIE1pbm9yIHN0eWxlIGZpeCAoUmVwb3J0ZWQgYnkgTWF0dGhldyBBdWxkKQp2ODoK LSBDaGFuZ2UgbnVtX3BhZ2VzIGZyb20gcGdvZmZfdCB0byB1aW50MzJfdCAoU3VnZ2VzdGVkIGJ5 IENocmlzdGlhbiBLw7ZuaWcpCi0tLQogZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fYm9fdXRpbC5j ICB8IDI4MCArKysrKysrKysrLS0tLS0tLS0tLS0tLS0tLS0tLQogZHJpdmVycy9ncHUvZHJtL3R0 bS90dG1fbW9kdWxlLmMgICB8ICAzNSArKysrCiBkcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9yZXNv dXJjZS5jIHwgMTkzICsrKysrKysrKysrKysrKysrKysrCiBkcml2ZXJzL2dwdS9kcm0vdHRtL3R0 bV90dC5jICAgICAgIHwgIDQyICsrKysrCiBpbmNsdWRlL2RybS90dG0vdHRtX2JvX2RyaXZlci5o ICAgIHwgIDI4ICsrKwogaW5jbHVkZS9kcm0vdHRtL3R0bV9jYWNoaW5nLmggICAgICB8ICAgMiAr CiBpbmNsdWRlL2RybS90dG0vdHRtX2ttYXBfaXRlci5oICAgIHwgIDYxICsrKysrKysKIGluY2x1 ZGUvZHJtL3R0bS90dG1fcmVzb3VyY2UuaCAgICAgfCAgNjEgKysrKysrKwogaW5jbHVkZS9kcm0v dHRtL3R0bV90dC5oICAgICAgICAgICB8ICAxNiArKwogOSBmaWxlcyBjaGFuZ2VkLCA1MzYgaW5z ZXJ0aW9ucygrKSwgMTgyIGRlbGV0aW9ucygtKQogY3JlYXRlIG1vZGUgMTAwNjQ0IGluY2x1ZGUv ZHJtL3R0bS90dG1fa21hcF9pdGVyLmgKCmRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vdHRt L3R0bV9ib191dGlsLmMgYi9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9ib191dGlsLmMKaW5kZXgg YWU4YjYxNDYwNzI0Li5hODAwOTk4YTEyYjAgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS90 dG0vdHRtX2JvX3V0aWwuYworKysgYi9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9ib191dGlsLmMK QEAgLTcyLDE5MCArNzIsMTI2IEBAIHZvaWQgdHRtX21lbV9pb19mcmVlKHN0cnVjdCB0dG1fZGV2 aWNlICpiZGV2LAogCW1lbS0+YnVzLmFkZHIgPSBOVUxMOwogfQogCi1zdGF0aWMgaW50IHR0bV9y ZXNvdXJjZV9pb3JlbWFwKHN0cnVjdCB0dG1fZGV2aWNlICpiZGV2LAotCQkJICAgICAgIHN0cnVj dCB0dG1fcmVzb3VyY2UgKm1lbSwKLQkJCSAgICAgICB2b2lkICoqdmlydHVhbCkKKy8qKgorICog dHRtX21vdmVfbWVtY3B5IC0gSGVscGVyIHRvIHBlcmZvcm0gYSBtZW1jcHkgdHRtIG1vdmUgb3Bl cmF0aW9uLgorICogQGJvOiBUaGUgc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0LgorICogQG5ld19t ZW06IFRoZSBzdHJ1Y3QgdHRtX3Jlc291cmNlIHdlJ3JlIG1vdmluZyB0byAoY29weSBkZXN0aW5h dGlvbikuCisgKiBAbmV3X2l0ZXI6IEEgc3RydWN0IHR0bV9rbWFwX2l0ZXIgcmVwcmVzZW50aW5n IHRoZSBkZXN0aW5hdGlvbiByZXNvdXJjZS4KKyAqIEBzcmNfaXRlcjogQSBzdHJ1Y3QgdHRtX2tt YXBfaXRlciByZXByZXNlbnRpbmcgdGhlIHNvdXJjZSByZXNvdXJjZS4KKyAqCisgKiBUaGlzIGZ1 bmN0aW9uIGlzIGludGVuZGVkIHRvIGJlIGFibGUgdG8gbW92ZSBvdXQgYXN5bmMgdW5kZXIgYQor ICogZG1hLWZlbmNlIGlmIGRlc2lyZWQuCisgKi8KK3ZvaWQgdHRtX21vdmVfbWVtY3B5KHN0cnVj dCB0dG1fYnVmZmVyX29iamVjdCAqYm8sCisJCSAgICAgdTMyIG51bV9wYWdlcywKKwkJICAgICBz dHJ1Y3QgdHRtX2ttYXBfaXRlciAqZHN0X2l0ZXIsCisJCSAgICAgc3RydWN0IHR0bV9rbWFwX2l0 ZXIgKnNyY19pdGVyKQogewotCWludCByZXQ7Ci0Jdm9pZCAqYWRkcjsKLQotCSp2aXJ0dWFsID0g TlVMTDsKLQlyZXQgPSB0dG1fbWVtX2lvX3Jlc2VydmUoYmRldiwgbWVtKTsKLQlpZiAocmV0IHx8 ICFtZW0tPmJ1cy5pc19pb21lbSkKLQkJcmV0dXJuIHJldDsKKwljb25zdCBzdHJ1Y3QgdHRtX2tt YXBfaXRlcl9vcHMgKmRzdF9vcHMgPSBkc3RfaXRlci0+b3BzOworCWNvbnN0IHN0cnVjdCB0dG1f a21hcF9pdGVyX29wcyAqc3JjX29wcyA9IHNyY19pdGVyLT5vcHM7CisJc3RydWN0IHR0bV90dCAq dHRtID0gYm8tPnR0bTsKKwlzdHJ1Y3QgZG1hX2J1Zl9tYXAgc3JjX21hcCwgZHN0X21hcDsKKwlw Z29mZl90IGk7CiAKLQlpZiAobWVtLT5idXMuYWRkcikgewotCQlhZGRyID0gbWVtLT5idXMuYWRk cjsKLQl9IGVsc2UgewotCQlzaXplX3QgYnVzX3NpemUgPSAoc2l6ZV90KW1lbS0+bnVtX3BhZ2Vz IDw8IFBBR0VfU0hJRlQ7CisJLyogU2luZ2xlIFRUTSBtb3ZlLiBOT1AgKi8KKwlpZiAoZHN0X29w cy0+bWFwc190dCAmJiBzcmNfb3BzLT5tYXBzX3R0KQorCQlyZXR1cm47CiAKLQkJaWYgKG1lbS0+ YnVzLmNhY2hpbmcgPT0gdHRtX3dyaXRlX2NvbWJpbmVkKQotCQkJYWRkciA9IGlvcmVtYXBfd2Mo bWVtLT5idXMub2Zmc2V0LCBidXNfc2l6ZSk7Ci0jaWZkZWYgQ09ORklHX1g4NgotCQllbHNlIGlm IChtZW0tPmJ1cy5jYWNoaW5nID09IHR0bV9jYWNoZWQpCi0JCQlhZGRyID0gaW9yZW1hcF9jYWNo ZShtZW0tPmJ1cy5vZmZzZXQsIGJ1c19zaXplKTsKLSNlbmRpZgotCQllbHNlCi0JCQlhZGRyID0g aW9yZW1hcChtZW0tPmJ1cy5vZmZzZXQsIGJ1c19zaXplKTsKLQkJaWYgKCFhZGRyKSB7Ci0JCQl0 dG1fbWVtX2lvX2ZyZWUoYmRldiwgbWVtKTsKLQkJCXJldHVybiAtRU5PTUVNOworCS8qIERvbid0 IG1vdmUgbm9uZXhpc3RlbnQgZGF0YS4gQ2xlYXIgZGVzdGluYXRpb24gaW5zdGVhZC4gKi8KKwlp ZiAoc3JjX29wcy0+bWFwc190dCAmJiAoIXR0bSB8fCAhdHRtX3R0X2lzX3BvcHVsYXRlZCh0dG0p KSkgeworCQlpZiAodHRtICYmICEodHRtLT5wYWdlX2ZsYWdzICYgVFRNX1BBR0VfRkxBR19aRVJP X0FMTE9DKSkKKwkJCXJldHVybjsKKworCQlmb3IgKGkgPSAwOyBpIDwgbnVtX3BhZ2VzOyArK2kp IHsKKwkJCWRzdF9vcHMtPm1hcF9sb2NhbChkc3RfaXRlciwgJmRzdF9tYXAsIGkpOworCQkJaWYg KGRzdF9tYXAuaXNfaW9tZW0pCisJCQkJbWVtc2V0X2lvKGRzdF9tYXAudmFkZHJfaW9tZW0sIDAs IFBBR0VfU0laRSk7CisJCQllbHNlCisJCQkJbWVtc2V0KGRzdF9tYXAudmFkZHIsIDAsIFBBR0Vf U0laRSk7CisJCQlpZiAoZHN0X29wcy0+dW5tYXBfbG9jYWwpCisJCQkJZHN0X29wcy0+dW5tYXBf bG9jYWwoZHN0X2l0ZXIsICZkc3RfbWFwKTsKIAkJfQorCQlyZXR1cm47CiAJfQotCSp2aXJ0dWFs ID0gYWRkcjsKLQlyZXR1cm4gMDsKLX0KLQotc3RhdGljIHZvaWQgdHRtX3Jlc291cmNlX2lvdW5t YXAoc3RydWN0IHR0bV9kZXZpY2UgKmJkZXYsCi0JCQkJc3RydWN0IHR0bV9yZXNvdXJjZSAqbWVt LAotCQkJCXZvaWQgKnZpcnR1YWwpCi17Ci0JaWYgKHZpcnR1YWwgJiYgbWVtLT5idXMuYWRkciA9 PSBOVUxMKQotCQlpb3VubWFwKHZpcnR1YWwpOwotCXR0bV9tZW1faW9fZnJlZShiZGV2LCBtZW0p OwotfQotCi1zdGF0aWMgaW50IHR0bV9jb3B5X2lvX3BhZ2Uodm9pZCAqZHN0LCB2b2lkICpzcmMs IHVuc2lnbmVkIGxvbmcgcGFnZSkKLXsKLQl1aW50MzJfdCAqZHN0UCA9Ci0JICAgICh1aW50MzJf dCAqKSAoKHVuc2lnbmVkIGxvbmcpZHN0ICsgKHBhZ2UgPDwgUEFHRV9TSElGVCkpOwotCXVpbnQz Ml90ICpzcmNQID0KLQkgICAgKHVpbnQzMl90ICopICgodW5zaWduZWQgbG9uZylzcmMgKyAocGFn ZSA8PCBQQUdFX1NISUZUKSk7Ci0KLQlpbnQgaTsKLQlmb3IgKGkgPSAwOyBpIDwgUEFHRV9TSVpF IC8gc2l6ZW9mKHVpbnQzMl90KTsgKytpKQotCQlpb3dyaXRlMzIoaW9yZWFkMzIoc3JjUCsrKSwg ZHN0UCsrKTsKLQlyZXR1cm4gMDsKLX0KLQotc3RhdGljIGludCB0dG1fY29weV9pb190dG1fcGFn ZShzdHJ1Y3QgdHRtX3R0ICp0dG0sIHZvaWQgKnNyYywKLQkJCQl1bnNpZ25lZCBsb25nIHBhZ2Us Ci0JCQkJcGdwcm90X3QgcHJvdCkKLXsKLQlzdHJ1Y3QgcGFnZSAqZCA9IHR0bS0+cGFnZXNbcGFn ZV07Ci0Jdm9pZCAqZHN0OwotCi0JaWYgKCFkKQotCQlyZXR1cm4gLUVOT01FTTsKLQotCXNyYyA9 ICh2b2lkICopKCh1bnNpZ25lZCBsb25nKXNyYyArIChwYWdlIDw8IFBBR0VfU0hJRlQpKTsKLQlk c3QgPSBrbWFwX2F0b21pY19wcm90KGQsIHByb3QpOwotCWlmICghZHN0KQotCQlyZXR1cm4gLUVO T01FTTsKLQotCW1lbWNweV9mcm9taW8oZHN0LCBzcmMsIFBBR0VfU0laRSk7Ci0KLQlrdW5tYXBf YXRvbWljKGRzdCk7Ci0KLQlyZXR1cm4gMDsKLX0KLQotc3RhdGljIGludCB0dG1fY29weV90dG1f aW9fcGFnZShzdHJ1Y3QgdHRtX3R0ICp0dG0sIHZvaWQgKmRzdCwKLQkJCQl1bnNpZ25lZCBsb25n IHBhZ2UsCi0JCQkJcGdwcm90X3QgcHJvdCkKLXsKLQlzdHJ1Y3QgcGFnZSAqcyA9IHR0bS0+cGFn ZXNbcGFnZV07Ci0Jdm9pZCAqc3JjOwogCi0JaWYgKCFzKQotCQlyZXR1cm4gLUVOT01FTTsKLQot CWRzdCA9ICh2b2lkICopKCh1bnNpZ25lZCBsb25nKWRzdCArIChwYWdlIDw8IFBBR0VfU0hJRlQp KTsKLQlzcmMgPSBrbWFwX2F0b21pY19wcm90KHMsIHByb3QpOwotCWlmICghc3JjKQotCQlyZXR1 cm4gLUVOT01FTTsKLQotCW1lbWNweV90b2lvKGRzdCwgc3JjLCBQQUdFX1NJWkUpOwotCi0Ja3Vu bWFwX2F0b21pYyhzcmMpOworCWZvciAoaSA9IDA7IGkgPCBudW1fcGFnZXM7ICsraSkgeworCQlk c3Rfb3BzLT5tYXBfbG9jYWwoZHN0X2l0ZXIsICZkc3RfbWFwLCBpKTsKKwkJc3JjX29wcy0+bWFw X2xvY2FsKHNyY19pdGVyLCAmc3JjX21hcCwgaSk7CisKKwkJaWYgKCFzcmNfbWFwLmlzX2lvbWVt ICYmICFkc3RfbWFwLmlzX2lvbWVtKSB7CisJCQltZW1jcHkoZHN0X21hcC52YWRkciwgc3JjX21h cC52YWRkciwgUEFHRV9TSVpFKTsKKwkJfSBlbHNlIGlmICghc3JjX21hcC5pc19pb21lbSkgewor CQkJZG1hX2J1Zl9tYXBfbWVtY3B5X3RvKCZkc3RfbWFwLCBzcmNfbWFwLnZhZGRyLAorCQkJCQkg ICAgICBQQUdFX1NJWkUpOworCQl9IGVsc2UgaWYgKCFkc3RfbWFwLmlzX2lvbWVtKSB7CisJCQlt ZW1jcHlfZnJvbWlvKGRzdF9tYXAudmFkZHIsIHNyY19tYXAudmFkZHJfaW9tZW0sCisJCQkJICAg ICAgUEFHRV9TSVpFKTsKKwkJfSBlbHNlIHsKKwkJCWludCBqOworCQkJdTMyIF9faW9tZW0gKnNy YyA9IHNyY19tYXAudmFkZHJfaW9tZW07CisJCQl1MzIgX19pb21lbSAqZHN0ID0gZHN0X21hcC52 YWRkcl9pb21lbTsKIAotCXJldHVybiAwOworCQkJZm9yIChqID0gMDsgaiA8IChQQUdFX1NJWkUg LyBzaXplb2YodTMyKSk7ICsraikKKwkJCQlpb3dyaXRlMzIoaW9yZWFkMzIoc3JjKyspLCBkc3Qr Kyk7CisJCX0KKwkJaWYgKHNyY19vcHMtPnVubWFwX2xvY2FsKQorCQkJc3JjX29wcy0+dW5tYXBf bG9jYWwoc3JjX2l0ZXIsICZzcmNfbWFwKTsKKwkJaWYgKGRzdF9vcHMtPnVubWFwX2xvY2FsKQor CQkJZHN0X29wcy0+dW5tYXBfbG9jYWwoZHN0X2l0ZXIsICZkc3RfbWFwKTsKKwl9CiB9CitFWFBP UlRfU1lNQk9MKHR0bV9tb3ZlX21lbWNweSk7CiAKIGludCB0dG1fYm9fbW92ZV9tZW1jcHkoc3Ry dWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywKIAkJICAgICAgIHN0cnVjdCB0dG1fb3BlcmF0aW9u X2N0eCAqY3R4LAotCQkgICAgICAgc3RydWN0IHR0bV9yZXNvdXJjZSAqbmV3X21lbSkKKwkJICAg ICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKmRzdF9tZW0pCiB7CiAJc3RydWN0IHR0bV9kZXZpY2Ug KmJkZXYgPSBiby0+YmRldjsKLQlzdHJ1Y3QgdHRtX3Jlc291cmNlX21hbmFnZXIgKm1hbiA9IHR0 bV9tYW5hZ2VyX3R5cGUoYmRldiwgbmV3X21lbS0+bWVtX3R5cGUpOworCXN0cnVjdCB0dG1fcmVz b3VyY2VfbWFuYWdlciAqZHN0X21hbiA9CisJCXR0bV9tYW5hZ2VyX3R5cGUoYm8tPmJkZXYsIGRz dF9tZW0tPm1lbV90eXBlKTsKIAlzdHJ1Y3QgdHRtX3R0ICp0dG0gPSBiby0+dHRtOwotCXN0cnVj dCB0dG1fcmVzb3VyY2UgKm9sZF9tZW0gPSAmYm8tPm1lbTsKLQlzdHJ1Y3QgdHRtX3Jlc291cmNl IG9sZF9jb3B5ID0gKm9sZF9tZW07Ci0Jdm9pZCAqb2xkX2lvbWFwOwotCXZvaWQgKm5ld19pb21h cDsKLQlpbnQgcmV0OwotCXVuc2lnbmVkIGxvbmcgaTsKLQotCXJldCA9IHR0bV9ib193YWl0X2N0 eChibywgY3R4KTsKLQlpZiAocmV0KQotCQlyZXR1cm4gcmV0OwotCi0JcmV0ID0gdHRtX3Jlc291 cmNlX2lvcmVtYXAoYmRldiwgb2xkX21lbSwgJm9sZF9pb21hcCk7Ci0JaWYgKHJldCkKLQkJcmV0 dXJuIHJldDsKLQlyZXQgPSB0dG1fcmVzb3VyY2VfaW9yZW1hcChiZGV2LCBuZXdfbWVtLCAmbmV3 X2lvbWFwKTsKLQlpZiAocmV0KQotCQlnb3RvIG91dDsKLQotCS8qCi0JICogU2luZ2xlIFRUTSBt b3ZlLiBOT1AuCi0JICovCi0JaWYgKG9sZF9pb21hcCA9PSBOVUxMICYmIG5ld19pb21hcCA9PSBO VUxMKQotCQlnb3RvIG91dDI7Ci0KLQkvKgotCSAqIERvbid0IG1vdmUgbm9uZXhpc3RlbnQgZGF0 YS4gQ2xlYXIgZGVzdGluYXRpb24gaW5zdGVhZC4KLQkgKi8KLQlpZiAob2xkX2lvbWFwID09IE5V TEwgJiYKLQkgICAgKHR0bSA9PSBOVUxMIHx8ICghdHRtX3R0X2lzX3BvcHVsYXRlZCh0dG0pICYm Ci0JCQkgICAgICEodHRtLT5wYWdlX2ZsYWdzICYgVFRNX1BBR0VfRkxBR19TV0FQUEVEKSkpKSB7 Ci0JCW1lbXNldF9pbyhuZXdfaW9tYXAsIDAsIG5ld19tZW0tPm51bV9wYWdlcypQQUdFX1NJWkUp OwotCQlnb3RvIG91dDI7Ci0JfQorCXN0cnVjdCB0dG1fcmVzb3VyY2UgKnNyY19tZW0gPSAmYm8t Pm1lbTsKKwlzdHJ1Y3QgdHRtX3Jlc291cmNlX21hbmFnZXIgKnNyY19tYW4gPQorCQl0dG1fbWFu YWdlcl90eXBlKGJkZXYsIHNyY19tZW0tPm1lbV90eXBlKTsKKwlzdHJ1Y3QgdHRtX3Jlc291cmNl IHNyY19jb3B5ID0gKnNyY19tZW07CisJdW5pb24geworCQlzdHJ1Y3QgdHRtX2ttYXBfaXRlcl90 dCB0dDsKKwkJc3RydWN0IHR0bV9rbWFwX2l0ZXJfbGluZWFyX2lvIGlvOworCX0gX2RzdF9pdGVy LCBfc3JjX2l0ZXI7CisJc3RydWN0IHR0bV9rbWFwX2l0ZXIgKmRzdF9pdGVyLCAqc3JjX2l0ZXI7 CisJaW50IHJldCA9IDA7CiAKLQkvKgotCSAqIFRUTSBtaWdodCBiZSBudWxsIGZvciBtb3ZlcyB3 aXRoaW4gdGhlIHNhbWUgcmVnaW9uLgotCSAqLwotCWlmICh0dG0pIHsKKwlpZiAodHRtICYmICgo dHRtLT5wYWdlX2ZsYWdzICYgVFRNX1BBR0VfRkxBR19TV0FQUEVEKSB8fAorCQkgICAgZHN0X21h bi0+dXNlX3R0KSkgewogCQlyZXQgPSB0dG1fdHRfcG9wdWxhdGUoYmRldiwgdHRtLCBjdHgpOwog CQlpZiAocmV0KQotCQkJZ290byBvdXQxOworCQkJcmV0dXJuIHJldDsKIAl9CiAKLQlmb3IgKGkg PSAwOyBpIDwgbmV3X21lbS0+bnVtX3BhZ2VzOyArK2kpIHsKLQkJaWYgKG9sZF9pb21hcCA9PSBO VUxMKSB7Ci0JCQlwZ3Byb3RfdCBwcm90ID0gdHRtX2lvX3Byb3QoYm8sIG9sZF9tZW0sIFBBR0Vf S0VSTkVMKTsKLQkJCXJldCA9IHR0bV9jb3B5X3R0bV9pb19wYWdlKHR0bSwgbmV3X2lvbWFwLCBp LAotCQkJCQkJICAgcHJvdCk7Ci0JCX0gZWxzZSBpZiAobmV3X2lvbWFwID09IE5VTEwpIHsKLQkJ CXBncHJvdF90IHByb3QgPSB0dG1faW9fcHJvdChibywgbmV3X21lbSwgUEFHRV9LRVJORUwpOwot CQkJcmV0ID0gdHRtX2NvcHlfaW9fdHRtX3BhZ2UodHRtLCBvbGRfaW9tYXAsIGksCi0JCQkJCQkg ICBwcm90KTsKLQkJfSBlbHNlIHsKLQkJCXJldCA9IHR0bV9jb3B5X2lvX3BhZ2UobmV3X2lvbWFw LCBvbGRfaW9tYXAsIGkpOwotCQl9Ci0JCWlmIChyZXQpCi0JCQlnb3RvIG91dDE7CisJZHN0X2l0 ZXIgPSB0dG1fa21hcF9pdGVyX2xpbmVhcl9pb19pbml0KCZfZHN0X2l0ZXIuaW8sIGJkZXYsIGRz dF9tZW0pOworCWlmIChQVFJfRVJSKGRzdF9pdGVyKSA9PSAtRUlOVkFMICYmIGRzdF9tYW4tPnVz ZV90dCkKKwkJZHN0X2l0ZXIgPSB0dG1fa21hcF9pdGVyX3R0X2luaXQoJl9kc3RfaXRlci50dCwg Ym8tPnR0bSk7CisJaWYgKElTX0VSUihkc3RfaXRlcikpCisJCXJldHVybiBQVFJfRVJSKGRzdF9p dGVyKTsKKworCXNyY19pdGVyID0gdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9faW5pdCgmX3NyY19p dGVyLmlvLCBiZGV2LCBzcmNfbWVtKTsKKwlpZiAoUFRSX0VSUihzcmNfaXRlcikgPT0gLUVJTlZB TCAmJiBzcmNfbWFuLT51c2VfdHQpCisJCXNyY19pdGVyID0gdHRtX2ttYXBfaXRlcl90dF9pbml0 KCZfc3JjX2l0ZXIudHQsIGJvLT50dG0pOworCWlmIChJU19FUlIoc3JjX2l0ZXIpKSB7CisJCXJl dCA9IFBUUl9FUlIoc3JjX2l0ZXIpOworCQlnb3RvIG91dF9zcmNfaXRlcjsKIAl9Ci0JbWIoKTsK LW91dDI6Ci0Jb2xkX2NvcHkgPSAqb2xkX21lbTsKIAotCXR0bV9ib19hc3NpZ25fbWVtKGJvLCBu ZXdfbWVtKTsKLQotCWlmICghbWFuLT51c2VfdHQpCi0JCXR0bV9ib190dF9kZXN0cm95KGJvKTsK Kwl0dG1fbW92ZV9tZW1jcHkoYm8sIGRzdF9tZW0tPm51bV9wYWdlcywgZHN0X2l0ZXIsIHNyY19p dGVyKTsKKwlzcmNfY29weSA9ICpzcmNfbWVtOworCXR0bV9ib19tb3ZlX3N5bmNfY2xlYW51cChi bywgZHN0X21lbSk7CiAKLW91dDE6Ci0JdHRtX3Jlc291cmNlX2lvdW5tYXAoYmRldiwgb2xkX21l bSwgbmV3X2lvbWFwKTsKLW91dDoKLQl0dG1fcmVzb3VyY2VfaW91bm1hcChiZGV2LCAmb2xkX2Nv cHksIG9sZF9pb21hcCk7CisJaWYgKCFzcmNfaXRlci0+b3BzLT5tYXBzX3R0KQorCQl0dG1fa21h cF9pdGVyX2xpbmVhcl9pb19maW5pKCZfc3JjX2l0ZXIuaW8sIGJkZXYsICZzcmNfY29weSk7Citv dXRfc3JjX2l0ZXI6CisJaWYgKCFkc3RfaXRlci0+b3BzLT5tYXBzX3R0KQorCQl0dG1fa21hcF9p dGVyX2xpbmVhcl9pb19maW5pKCZfZHN0X2l0ZXIuaW8sIGJkZXYsIGRzdF9tZW0pOwogCi0JLyoK LQkgKiBPbiBlcnJvciwga2VlcCB0aGUgbW0gbm9kZSEKLQkgKi8KLQlpZiAoIXJldCkKLQkJdHRt X3Jlc291cmNlX2ZyZWUoYm8sICZvbGRfY29weSk7CiAJcmV0dXJuIHJldDsKIH0KIEVYUE9SVF9T WU1CT0wodHRtX2JvX21vdmVfbWVtY3B5KTsKQEAgLTMzNiwyNyArMjcyLDcgQEAgcGdwcm90X3Qg dHRtX2lvX3Byb3Qoc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywgc3RydWN0IHR0bV9yZXNv dXJjZSAqcmVzLAogCW1hbiA9IHR0bV9tYW5hZ2VyX3R5cGUoYm8tPmJkZXYsIHJlcy0+bWVtX3R5 cGUpOwogCWNhY2hpbmcgPSBtYW4tPnVzZV90dCA/IGJvLT50dG0tPmNhY2hpbmcgOiByZXMtPmJ1 cy5jYWNoaW5nOwogCi0JLyogQ2FjaGVkIG1hcHBpbmdzIG5lZWQgbm8gYWRqdXN0bWVudCAqLwot CWlmIChjYWNoaW5nID09IHR0bV9jYWNoZWQpCi0JCXJldHVybiB0bXA7Ci0KLSNpZiBkZWZpbmVk KF9faTM4Nl9fKSB8fCBkZWZpbmVkKF9feDg2XzY0X18pCi0JaWYgKGNhY2hpbmcgPT0gdHRtX3dy aXRlX2NvbWJpbmVkKQotCQl0bXAgPSBwZ3Byb3Rfd3JpdGVjb21iaW5lKHRtcCk7Ci0JZWxzZSBp ZiAoYm9vdF9jcHVfZGF0YS54ODYgPiAzKQotCQl0bXAgPSBwZ3Byb3Rfbm9uY2FjaGVkKHRtcCk7 Ci0jZW5kaWYKLSNpZiBkZWZpbmVkKF9faWE2NF9fKSB8fCBkZWZpbmVkKF9fYXJtX18pIHx8IGRl ZmluZWQoX19hYXJjaDY0X18pIHx8IFwKLSAgICBkZWZpbmVkKF9fcG93ZXJwY19fKSB8fCBkZWZp bmVkKF9fbWlwc19fKQotCWlmIChjYWNoaW5nID09IHR0bV93cml0ZV9jb21iaW5lZCkKLQkJdG1w ID0gcGdwcm90X3dyaXRlY29tYmluZSh0bXApOwotCWVsc2UKLQkJdG1wID0gcGdwcm90X25vbmNh Y2hlZCh0bXApOwotI2VuZGlmCi0jaWYgZGVmaW5lZChfX3NwYXJjX18pCi0JdG1wID0gcGdwcm90 X25vbmNhY2hlZCh0bXApOwotI2VuZGlmCi0JcmV0dXJuIHRtcDsKKwlyZXR1cm4gdHRtX3Byb3Rf ZnJvbV9jYWNoaW5nKGNhY2hpbmcsIHRtcCk7CiB9CiBFWFBPUlRfU1lNQk9MKHR0bV9pb19wcm90 KTsKIApkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fbW9kdWxlLmMgYi9kcml2 ZXJzL2dwdS9kcm0vdHRtL3R0bV9tb2R1bGUuYwppbmRleCA1NmIwZWZkYmExYTkuLjk5N2M0NThm NjhhOSAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fbW9kdWxlLmMKKysrIGIv ZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fbW9kdWxlLmMKQEAgLTMxLDEyICszMSw0NyBAQAogICov CiAjaW5jbHVkZSA8bGludXgvbW9kdWxlLmg+CiAjaW5jbHVkZSA8bGludXgvZGV2aWNlLmg+Cisj aW5jbHVkZSA8bGludXgvcGd0YWJsZS5oPgogI2luY2x1ZGUgPGxpbnV4L3NjaGVkLmg+CiAjaW5j bHVkZSA8bGludXgvZGVidWdmcy5oPgogI2luY2x1ZGUgPGRybS9kcm1fc3lzZnMuaD4KKyNpbmNs dWRlIDxkcm0vdHRtL3R0bV9jYWNoaW5nLmg+CiAKICNpbmNsdWRlICJ0dG1fbW9kdWxlLmgiCiAK Ky8qKgorICogdHRtX3Byb3RfZnJvbV9jYWNoaW5nIC0gTW9kaWZ5IHRoZSBwYWdlIHByb3RlY3Rp b24gYWNjb3JkaW5nIHRvIHRoZQorICogdHRtIGNhY2luZyBtb2RlCisgKiBAY2FjaGluZzogVGhl IHR0bSBjYWNoaW5nIG1vZGUKKyAqIEB0bXA6IFRoZSBvcmlnaW5hbCBwYWdlIHByb3RlY3Rpb24K KyAqCisgKiBSZXR1cm46IFRoZSBtb2RpZmllZCBwYWdlIHByb3RlY3Rpb24KKyAqLworcGdwcm90 X3QgdHRtX3Byb3RfZnJvbV9jYWNoaW5nKGVudW0gdHRtX2NhY2hpbmcgY2FjaGluZywgcGdwcm90 X3QgdG1wKQoreworCS8qIENhY2hlZCBtYXBwaW5ncyBuZWVkIG5vIGFkanVzdG1lbnQgKi8KKwlp ZiAoY2FjaGluZyA9PSB0dG1fY2FjaGVkKQorCQlyZXR1cm4gdG1wOworCisjaWYgZGVmaW5lZChf X2kzODZfXykgfHwgZGVmaW5lZChfX3g4Nl82NF9fKQorCWlmIChjYWNoaW5nID09IHR0bV93cml0 ZV9jb21iaW5lZCkKKwkJdG1wID0gcGdwcm90X3dyaXRlY29tYmluZSh0bXApOworCWVsc2UgaWYg KGJvb3RfY3B1X2RhdGEueDg2ID4gMykKKwkJdG1wID0gcGdwcm90X25vbmNhY2hlZCh0bXApOwor I2VuZGlmCisjaWYgZGVmaW5lZChfX2lhNjRfXykgfHwgZGVmaW5lZChfX2FybV9fKSB8fCBkZWZp bmVkKF9fYWFyY2g2NF9fKSB8fCBcCisJZGVmaW5lZChfX3Bvd2VycGNfXykgfHwgZGVmaW5lZChf X21pcHNfXykKKwlpZiAoY2FjaGluZyA9PSB0dG1fd3JpdGVfY29tYmluZWQpCisJCXRtcCA9IHBn cHJvdF93cml0ZWNvbWJpbmUodG1wKTsKKwllbHNlCisJCXRtcCA9IHBncHJvdF9ub25jYWNoZWQo dG1wKTsKKyNlbmRpZgorI2lmIGRlZmluZWQoX19zcGFyY19fKQorCXRtcCA9IHBncHJvdF9ub25j YWNoZWQodG1wKTsKKyNlbmRpZgorCXJldHVybiB0bXA7Cit9CisKIHN0cnVjdCBkZW50cnkgKnR0 bV9kZWJ1Z2ZzX3Jvb3Q7CiAKIHN0YXRpYyBpbnQgX19pbml0IHR0bV9pbml0KHZvaWQpCmRpZmYg LS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV9yZXNvdXJjZS5jIGIvZHJpdmVycy9ncHUv ZHJtL3R0bS90dG1fcmVzb3VyY2UuYwppbmRleCA1OWUyYjcxNTdlNDEuLjkzMWJjZDFhOWI5YiAx MDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fcmVzb3VyY2UuYworKysgYi9kcml2 ZXJzL2dwdS9kcm0vdHRtL3R0bV9yZXNvdXJjZS5jCkBAIC0yMiw2ICsyMiwxMCBAQAogICogQXV0 aG9yczogQ2hyaXN0aWFuIEvDtm5pZwogICovCiAKKyNpbmNsdWRlIDxsaW51eC9kbWEtYnVmLW1h cC5oPgorI2luY2x1ZGUgPGxpbnV4L2lvLW1hcHBpbmcuaD4KKyNpbmNsdWRlIDxsaW51eC9zY2F0 dGVybGlzdC5oPgorCiAjaW5jbHVkZSA8ZHJtL3R0bS90dG1fcmVzb3VyY2UuaD4KICNpbmNsdWRl IDxkcm0vdHRtL3R0bV9ib19kcml2ZXIuaD4KIApAQCAtMTQ3LDMgKzE1MSwxOTIgQEAgdm9pZCB0 dG1fcmVzb3VyY2VfbWFuYWdlcl9kZWJ1ZyhzdHJ1Y3QgdHRtX3Jlc291cmNlX21hbmFnZXIgKm1h biwKIAkJbWFuLT5mdW5jLT5kZWJ1ZyhtYW4sIHApOwogfQogRVhQT1JUX1NZTUJPTCh0dG1fcmVz b3VyY2VfbWFuYWdlcl9kZWJ1Zyk7CisKK3N0YXRpYyB2b2lkIHR0bV9rbWFwX2l0ZXJfaW9tYXBf bWFwX2xvY2FsKHN0cnVjdCB0dG1fa21hcF9pdGVyICppdGVyLAorCQkJCQkgIHN0cnVjdCBkbWFf YnVmX21hcCAqZG1hcCwKKwkJCQkJICBwZ29mZl90IGkpCit7CisJc3RydWN0IHR0bV9rbWFwX2l0 ZXJfaW9tYXAgKml0ZXJfaW8gPQorCQljb250YWluZXJfb2YoaXRlciwgdHlwZW9mKCppdGVyX2lv KSwgYmFzZSk7CisJdm9pZCBfX2lvbWVtICphZGRyOworCityZXRyeToKKwl3aGlsZSAoaSA+PSBp dGVyX2lvLT5jYWNoZS5lbmQpIHsKKwkJaXRlcl9pby0+Y2FjaGUuc2cgPSBpdGVyX2lvLT5jYWNo ZS5zZyA/CisJCQlzZ19uZXh0KGl0ZXJfaW8tPmNhY2hlLnNnKSA6IGl0ZXJfaW8tPnN0LT5zZ2w7 CisJCWl0ZXJfaW8tPmNhY2hlLmkgPSBpdGVyX2lvLT5jYWNoZS5lbmQ7CisJCWl0ZXJfaW8tPmNh Y2hlLmVuZCArPSBzZ19kbWFfbGVuKGl0ZXJfaW8tPmNhY2hlLnNnKSA+PgorCQkJUEFHRV9TSElG VDsKKwkJaXRlcl9pby0+Y2FjaGUub2ZmcyA9IHNnX2RtYV9hZGRyZXNzKGl0ZXJfaW8tPmNhY2hl LnNnKSAtCisJCQlpdGVyX2lvLT5zdGFydDsKKwl9CisKKwlpZiAoaSA8IGl0ZXJfaW8tPmNhY2hl LmkpIHsKKwkJaXRlcl9pby0+Y2FjaGUuZW5kID0gMDsKKwkJaXRlcl9pby0+Y2FjaGUuc2cgPSBO VUxMOworCQlnb3RvIHJldHJ5OworCX0KKworCWFkZHIgPSBpb19tYXBwaW5nX21hcF9sb2NhbF93 YyhpdGVyX2lvLT5pb21hcCwgaXRlcl9pby0+Y2FjaGUub2ZmcyArCisJCQkJICAgICAgICgoKHJl c291cmNlX3NpemVfdClpIC0gaXRlcl9pby0+Y2FjaGUuaSkKKwkJCQkJPDwgUEFHRV9TSElGVCkp OworCWRtYV9idWZfbWFwX3NldF92YWRkcl9pb21lbShkbWFwLCBhZGRyKTsKK30KKworc3RhdGlj IHZvaWQgdHRtX2ttYXBfaXRlcl9pb21hcF91bm1hcF9sb2NhbChzdHJ1Y3QgdHRtX2ttYXBfaXRl ciAqaXRlciwKKwkJCQkJICAgIHN0cnVjdCBkbWFfYnVmX21hcCAqbWFwKQoreworCWlvX21hcHBp bmdfdW5tYXBfbG9jYWwobWFwLT52YWRkcl9pb21lbSk7Cit9CisKK3N0YXRpYyBjb25zdCBzdHJ1 Y3QgdHRtX2ttYXBfaXRlcl9vcHMgdHRtX2ttYXBfaXRlcl9pb19vcHMgPSB7CisJLm1hcF9sb2Nh bCA9ICB0dG1fa21hcF9pdGVyX2lvbWFwX21hcF9sb2NhbCwKKwkudW5tYXBfbG9jYWwgPSB0dG1f a21hcF9pdGVyX2lvbWFwX3VubWFwX2xvY2FsLAorCS5tYXBzX3R0ID0gZmFsc2UsCit9OworCisv KioKKyAqIHR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCAtIEluaXRpYWxpemUgYSBzdHJ1Y3QgdHRt X2ttYXBfaXRlcl9pb21hcAorICogQGl0ZXJfaW86IFRoZSBzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9p b21hcCB0byBpbml0aWFsaXplLgorICogQGlvbWFwOiBUaGUgc3RydWN0IGlvX21hcHBpbmcgcmVw cmVzZW50aW5nIHRoZSB1bmRlcmx5aW5nIGxpbmVhciBpb19tZW1vcnkuCisgKiBAc3Q6IHNnX3Rh YmxlIGludG8gQGlvbWFwLCByZXByZXNlbnRpbmcgdGhlIG1lbW9yeSBvZiB0aGUgc3RydWN0Cisg KiB0dG1fcmVzb3VyY2UuCisgKiBAc3RhcnQ6IE9mZnNldCB0aGF0IG5lZWRzIHRvIGJlIHN1YnRy YWN0ZWQgZnJvbSBAc3QgdG8gbWFrZQorICogc2dfZG1hX2FkZHJlc3Moc3QtPnNnbCkgLSBAc3Rh cnQgPT0gMCBmb3IgQGlvbWFwIHN0YXJ0LgorICoKKyAqIFJldHVybjogUG9pbnRlciB0byB0aGUg ZW1iZWRkZWQgc3RydWN0IHR0bV9rbWFwX2l0ZXIuCisgKi8KK3N0cnVjdCB0dG1fa21hcF9pdGVy ICoKK3R0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdChzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9pb21hcCAq aXRlcl9pbywKKwkJCSBzdHJ1Y3QgaW9fbWFwcGluZyAqaW9tYXAsCisJCQkgc3RydWN0IHNnX3Rh YmxlICpzdCwKKwkJCSByZXNvdXJjZV9zaXplX3Qgc3RhcnQpCit7CisJaXRlcl9pby0+YmFzZS5v cHMgPSAmdHRtX2ttYXBfaXRlcl9pb19vcHM7CisJaXRlcl9pby0+aW9tYXAgPSBpb21hcDsKKwlp dGVyX2lvLT5zdCA9IHN0OworCWl0ZXJfaW8tPnN0YXJ0ID0gc3RhcnQ7CisJbWVtc2V0KCZpdGVy X2lvLT5jYWNoZSwgMCwgc2l6ZW9mKGl0ZXJfaW8tPmNhY2hlKSk7CisKKwlyZXR1cm4gJml0ZXJf aW8tPmJhc2U7Cit9CitFWFBPUlRfU1lNQk9MKHR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCk7CisK Ky8qKgorICogRE9DOiBMaW5lYXIgaW8gaXRlcmF0b3IKKyAqCisgKiBUaGlzIGNvZGUgc2hvdWxk IGRpZSBpbiB0aGUgbm90IHRvbyBuZWFyIGZ1dHVyZS4gQmVzdCB3b3VsZCBiZSBpZiB3ZSBjb3Vs ZAorICogbWFrZSBpby1tYXBwaW5nIHVzZSBtZW1yZW1hcCBmb3IgYWxsIGlvIG1lbW9yeSwgYW5k IGhhdmUgbWVtcmVtYXAKKyAqIGltcGxlbWVudCBhIGttYXBfbG9jYWwgZnVuY3Rpb25hbGl0eS4g V2UgY291bGQgdGhlbiBzdHJpcCBhIGh1Z2UgYW1vdW50IG9mCisgKiBjb2RlLiBUaGVzZSBsaW5l YXIgaW8gaXRlcmF0b3JzIGFyZSBpbXBsZW1lbnRlZCB0byBtaW1pYyBvbGQgZnVuY3Rpb25hbGl0 eSwKKyAqIGFuZCB0aGV5IGRvbid0IHVzZSBrbWFwX2xvY2FsIHNlbWFudGljcyBhdCBhbGwgaW50 ZXJuYWxseS4gUmF0aGVyIGlvcmVtYXAgb3IKKyAqIGZyaWVuZHMsIGFuZCBhdCBsZWFzdCBvbiAz Mi1iaXQgdGhleSBhZGQgZ2xvYmFsIFRMQiBmbHVzaGVzIGFuZCBwb2ludHMKKyAqIG9mIGZhaWx1 cmUuCisgKi8KKworc3RhdGljIHZvaWQgdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9fbWFwX2xvY2Fs KHN0cnVjdCB0dG1fa21hcF9pdGVyICppdGVyLAorCQkJCQkgICAgICBzdHJ1Y3QgZG1hX2J1Zl9t YXAgKmRtYXAsCisJCQkJCSAgICAgIHBnb2ZmX3QgaSkKK3sKKwlzdHJ1Y3QgdHRtX2ttYXBfaXRl cl9saW5lYXJfaW8gKml0ZXJfaW8gPQorCQljb250YWluZXJfb2YoaXRlciwgdHlwZW9mKCppdGVy X2lvKSwgYmFzZSk7CisKKwkqZG1hcCA9IGl0ZXJfaW8tPmRtYXA7CisJZG1hX2J1Zl9tYXBfaW5j cihkbWFwLCBpICogUEFHRV9TSVpFKTsKK30KKworc3RhdGljIGNvbnN0IHN0cnVjdCB0dG1fa21h cF9pdGVyX29wcyB0dG1fa21hcF9pdGVyX2xpbmVhcl9pb19vcHMgPSB7CisJLm1hcF9sb2NhbCA9 ICB0dG1fa21hcF9pdGVyX2xpbmVhcl9pb19tYXBfbG9jYWwsCisJLm1hcHNfdHQgPSBmYWxzZSwK K307CisKKy8qKgorICogdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9faW5pdCAtIEluaXRpYWxpemUg YW4gaXRlcmF0b3IgZm9yIGxpbmVhciBpbyBtZW1vcnkKKyAqIEBpdGVyX2lvOiBUaGUgaXRlcmF0 b3IgdG8gaW5pdGlhbGl6ZQorICogQGJkZXY6IFRoZSBUVE0gZGV2aWNlCisgKiBAbWVtOiBUaGUg dHRtIHJlc291cmNlIHJlcHJlc2VudGluZyB0aGUgaW9tYXAuCisgKgorICogVGhpcyBmdW5jdGlv biBpcyBmb3IgaW50ZXJuYWwgVFRNIHVzZSBvbmx5LiBJdCBzZXRzIHVwIGEgbWVtY3B5IGttYXAg aXRlcmF0b3IKKyAqIHBvaW50aW5nIGF0IGEgbGluZWFyIGNodW5rIG9mIGlvIG1lbW9yeS4KKyAq CisgKiBSZXR1cm46IEEgcG9pbnRlciB0byB0aGUgZW1iZWRkZWQgc3RydWN0IHR0bV9rbWFwX2l0 ZXIgb3IgZXJyb3IgcG9pbnRlciBvbgorICogZmFpbHVyZS4KKyAqLworc3RydWN0IHR0bV9rbWFw X2l0ZXIgKgordHRtX2ttYXBfaXRlcl9saW5lYXJfaW9faW5pdChzdHJ1Y3QgdHRtX2ttYXBfaXRl cl9saW5lYXJfaW8gKml0ZXJfaW8sCisJCQkgICAgIHN0cnVjdCB0dG1fZGV2aWNlICpiZGV2LAor CQkJICAgICBzdHJ1Y3QgdHRtX3Jlc291cmNlICptZW0pCit7CisJaW50IHJldDsKKworCXJldCA9 IHR0bV9tZW1faW9fcmVzZXJ2ZShiZGV2LCBtZW0pOworCWlmIChyZXQpCisJCWdvdG8gb3V0X2Vy cjsKKwlpZiAoIW1lbS0+YnVzLmlzX2lvbWVtKSB7CisJCXJldCA9IC1FSU5WQUw7CisJCWdvdG8g b3V0X2lvX2ZyZWU7CisJfQorCisJaWYgKG1lbS0+YnVzLmFkZHIpIHsKKwkJZG1hX2J1Zl9tYXBf c2V0X3ZhZGRyKCZpdGVyX2lvLT5kbWFwLCBtZW0tPmJ1cy5hZGRyKTsKKwkJaXRlcl9pby0+bmVl ZHNfdW5tYXAgPSBmYWxzZTsKKwl9IGVsc2UgeworCQlzaXplX3QgYnVzX3NpemUgPSAoc2l6ZV90 KW1lbS0+bnVtX3BhZ2VzIDw8IFBBR0VfU0hJRlQ7CisKKwkJaXRlcl9pby0+bmVlZHNfdW5tYXAg PSB0cnVlOworCQltZW1zZXQoJml0ZXJfaW8tPmRtYXAsIDAsIHNpemVvZihpdGVyX2lvLT5kbWFw KSk7CisJCWlmIChtZW0tPmJ1cy5jYWNoaW5nID09IHR0bV93cml0ZV9jb21iaW5lZCkKKwkJCWRt YV9idWZfbWFwX3NldF92YWRkcl9pb21lbSgmaXRlcl9pby0+ZG1hcCwKKwkJCQkJCSAgICBpb3Jl bWFwX3djKG1lbS0+YnVzLm9mZnNldCwKKwkJCQkJCQkgICAgICAgYnVzX3NpemUpKTsKKwkJZWxz ZSBpZiAobWVtLT5idXMuY2FjaGluZyA9PSB0dG1fY2FjaGVkKQorCQkJZG1hX2J1Zl9tYXBfc2V0 X3ZhZGRyKCZpdGVyX2lvLT5kbWFwLAorCQkJCQkgICAgICBtZW1yZW1hcChtZW0tPmJ1cy5vZmZz ZXQsIGJ1c19zaXplLAorCQkJCQkJICAgICAgIE1FTVJFTUFQX1dCIHwKKwkJCQkJCSAgICAgICBN RU1SRU1BUF9XVCB8CisJCQkJCQkgICAgICAgTUVNUkVNQVBfV0MpKTsKKworCQkvKiBJZiB1bmNh Y2hlZCByZXF1ZXN0ZWQgb3IgaWYgbWFwcGluZyBjYWNoZWQgb3Igd2MgZmFpbGVkICovCisJCWlm IChkbWFfYnVmX21hcF9pc19udWxsKCZpdGVyX2lvLT5kbWFwKSkKKwkJCWRtYV9idWZfbWFwX3Nl dF92YWRkcl9pb21lbSgmaXRlcl9pby0+ZG1hcCwKKwkJCQkJCSAgICBpb3JlbWFwKG1lbS0+YnVz Lm9mZnNldCwKKwkJCQkJCQkgICAgYnVzX3NpemUpKTsKKworCQlpZiAoZG1hX2J1Zl9tYXBfaXNf bnVsbCgmaXRlcl9pby0+ZG1hcCkpIHsKKwkJCXJldCA9IC1FTk9NRU07CisJCQlnb3RvIG91dF9p b19mcmVlOworCQl9CisJfQorCisJaXRlcl9pby0+YmFzZS5vcHMgPSAmdHRtX2ttYXBfaXRlcl9s aW5lYXJfaW9fb3BzOworCXJldHVybiAmaXRlcl9pby0+YmFzZTsKKworb3V0X2lvX2ZyZWU6CisJ dHRtX21lbV9pb19mcmVlKGJkZXYsIG1lbSk7CitvdXRfZXJyOgorCXJldHVybiBFUlJfUFRSKHJl dCk7Cit9CisKKy8qKgorICogdHRtX2ttYXBfaXRlcl9saW5lYXJfaW9fZmluaSAtIENsZWFuIHVw IGFuIGl0ZXJhdG9yIGZvciBsaW5lYXIgaW8gbWVtb3J5CisgKiBAaXRlcl9pbzogVGhlIGl0ZXJh dG9yIHRvIGluaXRpYWxpemUKKyAqIEBiZGV2OiBUaGUgVFRNIGRldmljZQorICogQG1lbTogVGhl IHR0bSByZXNvdXJjZSByZXByZXNlbnRpbmcgdGhlIGlvbWFwLgorICoKKyAqIFRoaXMgZnVuY3Rp b24gaXMgZm9yIGludGVybmFsIFRUTSB1c2Ugb25seS4gSXQgY2xlYW5zIHVwIGEgbWVtY3B5IGtt YXAKKyAqIGl0ZXJhdG9yIGluaXRpYWxpemVkIGJ5IHR0bV9rbWFwX2l0ZXJfbGluZWFyX2lvX2lu aXQuCisgKi8KK3ZvaWQKK3R0bV9rbWFwX2l0ZXJfbGluZWFyX2lvX2Zpbmkoc3RydWN0IHR0bV9r bWFwX2l0ZXJfbGluZWFyX2lvICppdGVyX2lvLAorCQkJICAgICBzdHJ1Y3QgdHRtX2RldmljZSAq YmRldiwKKwkJCSAgICAgc3RydWN0IHR0bV9yZXNvdXJjZSAqbWVtKQoreworCWlmIChpdGVyX2lv LT5uZWVkc191bm1hcCAmJiBkbWFfYnVmX21hcF9pc19zZXQoJml0ZXJfaW8tPmRtYXApKSB7CisJ CWlmIChpdGVyX2lvLT5kbWFwLmlzX2lvbWVtKQorCQkJaW91bm1hcChpdGVyX2lvLT5kbWFwLnZh ZGRyX2lvbWVtKTsKKwkJZWxzZQorCQkJbWVtdW5tYXAoaXRlcl9pby0+ZG1hcC52YWRkcik7CisJ fQorCisJdHRtX21lbV9pb19mcmVlKGJkZXYsIG1lbSk7Cit9CmRpZmYgLS1naXQgYS9kcml2ZXJz L2dwdS9kcm0vdHRtL3R0bV90dC5jIGIvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fdHQuYwppbmRl eCA1MzllMDIzMmNiM2IuLjBlNDEyMjcxMTZiMSAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJt L3R0bS90dG1fdHQuYworKysgYi9kcml2ZXJzL2dwdS9kcm0vdHRtL3R0bV90dC5jCkBAIC00MzMs MyArNDMzLDQ1IEBAIHZvaWQgdHRtX3R0X21ncl9pbml0KHVuc2lnbmVkIGxvbmcgbnVtX3BhZ2Vz LCB1bnNpZ25lZCBsb25nIG51bV9kbWEzMl9wYWdlcykKIAlpZiAoIXR0bV9kbWEzMl9wYWdlc19s aW1pdCkKIAkJdHRtX2RtYTMyX3BhZ2VzX2xpbWl0ID0gbnVtX2RtYTMyX3BhZ2VzOwogfQorCitz dGF0aWMgdm9pZCB0dG1fa21hcF9pdGVyX3R0X21hcF9sb2NhbChzdHJ1Y3QgdHRtX2ttYXBfaXRl ciAqaXRlciwKKwkJCQkgICAgICAgc3RydWN0IGRtYV9idWZfbWFwICpkbWFwLAorCQkJCSAgICAg ICBwZ29mZl90IGkpCit7CisJc3RydWN0IHR0bV9rbWFwX2l0ZXJfdHQgKml0ZXJfdHQgPQorCQlj b250YWluZXJfb2YoaXRlciwgdHlwZW9mKCppdGVyX3R0KSwgYmFzZSk7CisKKwlkbWFfYnVmX21h cF9zZXRfdmFkZHIoZG1hcCwga21hcF9sb2NhbF9wYWdlX3Byb3QoaXRlcl90dC0+dHQtPnBhZ2Vz W2ldLAorCQkJCQkJCSBpdGVyX3R0LT5wcm90KSk7Cit9CisKK3N0YXRpYyB2b2lkIHR0bV9rbWFw X2l0ZXJfdHRfdW5tYXBfbG9jYWwoc3RydWN0IHR0bV9rbWFwX2l0ZXIgKml0ZXIsCisJCQkJCSBz dHJ1Y3QgZG1hX2J1Zl9tYXAgKm1hcCkKK3sKKwlrdW5tYXBfbG9jYWwobWFwLT52YWRkcik7Cit9 CisKK3N0YXRpYyBjb25zdCBzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9vcHMgdHRtX2ttYXBfaXRlcl90 dF9vcHMgPSB7CisJLm1hcF9sb2NhbCA9IHR0bV9rbWFwX2l0ZXJfdHRfbWFwX2xvY2FsLAorCS51 bm1hcF9sb2NhbCA9IHR0bV9rbWFwX2l0ZXJfdHRfdW5tYXBfbG9jYWwsCisJLm1hcHNfdHQgPSB0 cnVlLAorfTsKKworLyoqCisgKiB0dG1fa21hcF9pdGVyX3R0X2luaXQgLSBJbml0aWFsaXplIGEg c3RydWN0IHR0bV9rbWFwX2l0ZXJfdHQKKyAqIEBpdGVyX3R0OiBUaGUgc3RydWN0IHR0bV9rbWFw X2l0ZXJfdHQgdG8gaW5pdGlhbGl6ZS4KKyAqIEB0dDogU3RydWN0IHR0bV90dCBob2xkaW5nIHBh Z2UgcG9pbnRlcnMgb2YgdGhlIHN0cnVjdCB0dG1fcmVzb3VyY2UuCisgKgorICogUmV0dXJuOiBQ b2ludGVyIHRvIHRoZSBlbWJlZGRlZCBzdHJ1Y3QgdHRtX2ttYXBfaXRlci4KKyAqLworc3RydWN0 IHR0bV9rbWFwX2l0ZXIgKgordHRtX2ttYXBfaXRlcl90dF9pbml0KHN0cnVjdCB0dG1fa21hcF9p dGVyX3R0ICppdGVyX3R0LAorCQkgICAgICBzdHJ1Y3QgdHRtX3R0ICp0dCkKK3sKKwlpdGVyX3R0 LT5iYXNlLm9wcyA9ICZ0dG1fa21hcF9pdGVyX3R0X29wczsKKwlpdGVyX3R0LT50dCA9IHR0Owor CWl0ZXJfdHQtPnByb3QgPSB0dG1fcHJvdF9mcm9tX2NhY2hpbmcodHQtPmNhY2hpbmcsIFBBR0Vf S0VSTkVMKTsKKworCXJldHVybiAmaXRlcl90dC0+YmFzZTsKK30KK0VYUE9SVF9TWU1CT0wodHRt X2ttYXBfaXRlcl90dF9pbml0KTsKZGlmZiAtLWdpdCBhL2luY2x1ZGUvZHJtL3R0bS90dG1fYm9f ZHJpdmVyLmggYi9pbmNsdWRlL2RybS90dG0vdHRtX2JvX2RyaXZlci5oCmluZGV4IGRiY2NhYzk1 N2Y4Zi4uNzI1MTE5OTIyNjFhIDEwMDY0NAotLS0gYS9pbmNsdWRlL2RybS90dG0vdHRtX2JvX2Ry aXZlci5oCisrKyBiL2luY2x1ZGUvZHJtL3R0bS90dG1fYm9fZHJpdmVyLmgKQEAgLTQwLDYgKzQw LDcgQEAKICNpbmNsdWRlIDxkcm0vdHRtL3R0bV9kZXZpY2UuaD4KIAogI2luY2x1ZGUgInR0bV9i b19hcGkuaCIKKyNpbmNsdWRlICJ0dG1fa21hcF9pdGVyLmgiCiAjaW5jbHVkZSAidHRtX3BsYWNl bWVudC5oIgogI2luY2x1ZGUgInR0bV90dC5oIgogI2luY2x1ZGUgInR0bV9wb29sLmgiCkBAIC0y NzIsNiArMjczLDIzIEBAIGludCB0dG1fYm9fbW92ZV9hY2NlbF9jbGVhbnVwKHN0cnVjdCB0dG1f YnVmZmVyX29iamVjdCAqYm8sCiAJCQkgICAgICBib29sIHBpcGVsaW5lLAogCQkJICAgICAgc3Ry dWN0IHR0bV9yZXNvdXJjZSAqbmV3X21lbSk7CiAKKy8qKgorICogdHRtX2JvX21vdmVfYWNjZWxf Y2xlYW51cC4KKyAqCisgKiBAYm86IEEgcG9pbnRlciB0byBhIHN0cnVjdCB0dG1fYnVmZmVyX29i amVjdC4KKyAqIEBuZXdfbWVtOiBzdHJ1Y3QgdHRtX3Jlc291cmNlIGluZGljYXRpbmcgd2hlcmUg dG8gbW92ZS4KKyAqCisgKiBTcGVjaWFsIGNhc2Ugb2YgdHRtX2JvX21vdmVfYWNjZWxfY2xlYW51 cCB3aGVyZSB0aGUgYm8gaXMgZ3VhcmFudGVlZAorICogYnkgdGhlIGNhbGxlciB0byBiZSBpZGxl LiBUeXBpY2FsbHkgdXNlZCBhZnRlciBtZW1jcHkgYnVmZmVyIG1vdmVzLgorICovCitzdGF0aWMg aW5saW5lIHZvaWQgdHRtX2JvX21vdmVfc3luY19jbGVhbnVwKHN0cnVjdCB0dG1fYnVmZmVyX29i amVjdCAqYm8sCisJCQkJCSAgICBzdHJ1Y3QgdHRtX3Jlc291cmNlICpuZXdfbWVtKQoreworCWlu dCByZXQgPSB0dG1fYm9fbW92ZV9hY2NlbF9jbGVhbnVwKGJvLCBOVUxMLCB0cnVlLCBmYWxzZSwg bmV3X21lbSk7CisKKwlXQVJOX09OKHJldCk7Cit9CisKIC8qKgogICogdHRtX2JvX3BpcGVsaW5l X2d1dHRpbmcuCiAgKgpAQCAtMzMyLDQgKzM1MCwxNCBAQCBpbnQgdHRtX3JhbmdlX21hbl9pbml0 KHN0cnVjdCB0dG1fZGV2aWNlICpiZGV2LAogaW50IHR0bV9yYW5nZV9tYW5fZmluaShzdHJ1Y3Qg dHRtX2RldmljZSAqYmRldiwKIAkJICAgICAgIHVuc2lnbmVkIHR5cGUpOwogCit2b2lkIHR0bV9t b3ZlX21lbWNweShzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvLAorCQkgICAgIHUzMiBudW1f cGFnZXMsCisJCSAgICAgc3RydWN0IHR0bV9rbWFwX2l0ZXIgKmRzdF9pdGVyLAorCQkgICAgIHN0 cnVjdCB0dG1fa21hcF9pdGVyICpzcmNfaXRlcik7CisKK3N0cnVjdCB0dG1fa21hcF9pdGVyICoK K3R0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdChzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9pb21hcCAqaXRl cl9pbywKKwkJCSBzdHJ1Y3QgaW9fbWFwcGluZyAqaW9tYXAsCisJCQkgc3RydWN0IHNnX3RhYmxl ICpzdCwKKwkJCSByZXNvdXJjZV9zaXplX3Qgc3RhcnQpOwogI2VuZGlmCmRpZmYgLS1naXQgYS9p bmNsdWRlL2RybS90dG0vdHRtX2NhY2hpbmcuaCBiL2luY2x1ZGUvZHJtL3R0bS90dG1fY2FjaGlu Zy5oCmluZGV4IGEwYjRhNDlmYTQzMi4uM2M5ZGQ2NWY1YWFmIDEwMDY0NAotLS0gYS9pbmNsdWRl L2RybS90dG0vdHRtX2NhY2hpbmcuaAorKysgYi9pbmNsdWRlL2RybS90dG0vdHRtX2NhY2hpbmcu aApAQCAtMzMsNCArMzMsNiBAQCBlbnVtIHR0bV9jYWNoaW5nIHsKIAl0dG1fY2FjaGVkCiB9Owog CitwZ3Byb3RfdCB0dG1fcHJvdF9mcm9tX2NhY2hpbmcoZW51bSB0dG1fY2FjaGluZyBjYWNoaW5n LCBwZ3Byb3RfdCB0bXApOworCiAjZW5kaWYKZGlmZiAtLWdpdCBhL2luY2x1ZGUvZHJtL3R0bS90 dG1fa21hcF9pdGVyLmggYi9pbmNsdWRlL2RybS90dG0vdHRtX2ttYXBfaXRlci5oCm5ldyBmaWxl IG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAwMDAwMC4uOGJiMDBmZDM5ZDZjCi0tLSAvZGV2L251 bGwKKysrIGIvaW5jbHVkZS9kcm0vdHRtL3R0bV9rbWFwX2l0ZXIuaApAQCAtMCwwICsxLDYxIEBA CisvKiBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogTUlUICovCisvKgorICogQ29weXJpZ2h0IMKp IDIwMjEgSW50ZWwgQ29ycG9yYXRpb24KKyAqLworI2lmbmRlZiBfX1RUTV9LTUFQX0lURVJfSF9f CisjZGVmaW5lIF9fVFRNX0tNQVBfSVRFUl9IX18KKworI2luY2x1ZGUgPGxpbnV4L3R5cGVzLmg+ CisKK3N0cnVjdCB0dG1fa21hcF9pdGVyOworc3RydWN0IGRtYV9idWZfbWFwOworCisvKioKKyAq IHN0cnVjdCB0dG1fa21hcF9pdGVyX29wcyAtIE9wcyBzdHJ1Y3R1cmUgZm9yIGEgc3RydWN0Cisg KiB0dG1fa21hcF9pdGVyLgorICogQG1hcHNfdHQ6IFdoZXRoZXIgdGhlIGl0ZXJhdG9yIG1hcHMg VFQgbWVtb3J5IGRpcmVjdGx5LCBhcyBvcHBvc2VkCisgKiBtYXBwaW5nIGEgVFQgdGhyb3VnaCBh biBhcGVydHVyZS4gQm90aCB0aGVzZSBtb2RlcyBoYXZlCisgKiBzdHJ1Y3QgdHRtX3Jlc291cmNl X21hbmFnZXI6OnVzZV90dCBzZXQsIGJ1dCB0aGUgbGF0dGVyIHR5cGljYWxseQorICogcmV0dXJu cyBpc19pb21lbSA9PSB0cnVlIGZyb20gdHRtX21lbV9pb19yZXNlcnZlLgorICovCitzdHJ1Y3Qg dHRtX2ttYXBfaXRlcl9vcHMgeworCS8qKgorCSAqIGttYXBfbG9jYWwoKSAtIE1hcCBhIFBBR0Vf U0laRSBwYXJ0IG9mIHRoZSByZXNvdXJjZSB1c2luZworCSAqIGttYXBfbG9jYWwgc2VtYW50aWNz LgorCSAqIEByZXNfaXRlcjogUG9pbnRlciB0byB0aGUgc3RydWN0IHR0bV9rbWFwX2l0ZXIgcmVw cmVzZW50aW5nCisJICogdGhlIHJlc291cmNlLgorCSAqIEBkbWFwOiBUaGUgc3RydWN0IGRtYV9i dWZfbWFwIGhvbGRpbmcgdGhlIHZpcnR1YWwgYWRkcmVzcyBhZnRlcgorCSAqIHRoZSBvcGVyYXRp b24uCisJICogQGk6IFRoZSBsb2NhdGlvbiB3aXRoaW4gdGhlIHJlc291cmNlIHRvIG1hcC4gUEFH RV9TSVpFIGdyYW51bGFyaXR5LgorCSAqLworCXZvaWQgKCptYXBfbG9jYWwpKHN0cnVjdCB0dG1f a21hcF9pdGVyICpyZXNfaXRlciwKKwkJCSAgc3RydWN0IGRtYV9idWZfbWFwICpkbWFwLCBwZ29m Zl90IGkpOworCS8qKgorCSAqIHVubWFwX2xvY2FsKCkgLSBVbm1hcCBhIFBBR0VfU0laRSBwYXJ0 IG9mIHRoZSByZXNvdXJjZSBwcmV2aW91c2x5CisJICogbWFwcGVkIHVzaW5nIGttYXBfbG9jYWwu CisJICogQHJlc19pdGVyOiBQb2ludGVyIHRvIHRoZSBzdHJ1Y3QgdHRtX2ttYXBfaXRlciByZXBy ZXNlbnRpbmcKKwkgKiB0aGUgcmVzb3VyY2UuCisJICogQGRtYXA6IFRoZSBzdHJ1Y3QgZG1hX2J1 Zl9tYXAgaG9sZGluZyB0aGUgdmlydHVhbCBhZGRyZXNzIGFmdGVyCisJICogdGhlIG9wZXJhdGlv bi4KKwkgKi8KKwl2b2lkICgqdW5tYXBfbG9jYWwpKHN0cnVjdCB0dG1fa21hcF9pdGVyICpyZXNf aXRlciwKKwkJCSAgICBzdHJ1Y3QgZG1hX2J1Zl9tYXAgKmRtYXApOworCWJvb2wgbWFwc190dDsK K307CisKKy8qKgorICogc3RydWN0IHR0bV9rbWFwX2l0ZXIgLSBJdGVyYXRvciBmb3Iga21hcF9s b2NhbCB0eXBlIG9wZXJhdGlvbnMgb24gYQorICogcmVzb3VyY2UuCisgKiBAb3BzOiBQb2ludGVy IHRvIHRoZSBvcGVyYXRpb25zIHN0cnVjdC4KKyAqCisgKiBUaGlzIHN0cnVjdCBpcyBpbnRlbmRl ZCB0byBiZSBlbWJlZGRlZCBpbiBhIHJlc291cmNlLXNwZWNpZmljIHNwZWNpYWxpemF0aW9uCisg KiBpbXBsZW1lbnRpbmcgb3BlcmF0aW9ucyBmb3IgdGhlIHJlc291cmNlLgorICoKKyAqIE5vdGhp bmcgc3RvcHMgdXMgZnJvbSBleHRlbmRpbmcgdGhlIG9wZXJhdGlvbnMgdG8gdm1hcCwgdm1hcF9w Zm4gZXRjLAorICogcmVwbGFjaW5nIHNvbWUgb3IgcGFydHMgb2YgdGhlIHR0bV9ib191dGlsLiBj cHUtbWFwIGZ1bmN0aW9uYWxpdHkuCisgKi8KK3N0cnVjdCB0dG1fa21hcF9pdGVyIHsKKwljb25z dCBzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9vcHMgKm9wczsKK307CisKKyNlbmRpZiAvKiBfX1RUTV9L TUFQX0lURVJfSF9fICovCmRpZmYgLS1naXQgYS9pbmNsdWRlL2RybS90dG0vdHRtX3Jlc291cmNl LmggYi9pbmNsdWRlL2RybS90dG0vdHRtX3Jlc291cmNlLmgKaW5kZXggODkwYjlkMzY5NTE5Li5i OGRjMGJkYjBkYTUgMTAwNjQ0Ci0tLSBhL2luY2x1ZGUvZHJtL3R0bS90dG1fcmVzb3VyY2UuaAor KysgYi9pbmNsdWRlL2RybS90dG0vdHRtX3Jlc291cmNlLmgKQEAgLTI3LDkgKzI3LDExIEBACiAK ICNpbmNsdWRlIDxsaW51eC90eXBlcy5oPgogI2luY2x1ZGUgPGxpbnV4L211dGV4Lmg+CisjaW5j bHVkZSA8bGludXgvZG1hLWJ1Zi1tYXAuaD4KICNpbmNsdWRlIDxsaW51eC9kbWEtZmVuY2UuaD4K ICNpbmNsdWRlIDxkcm0vZHJtX3ByaW50Lmg+CiAjaW5jbHVkZSA8ZHJtL3R0bS90dG1fY2FjaGlu Zy5oPgorI2luY2x1ZGUgPGRybS90dG0vdHRtX2ttYXBfaXRlci5oPgogCiAjZGVmaW5lIFRUTV9N QVhfQk9fUFJJT1JJVFkJNFUKIApAQCAtMzgsNiArNDAsMTAgQEAgc3RydWN0IHR0bV9yZXNvdXJj ZV9tYW5hZ2VyOwogc3RydWN0IHR0bV9yZXNvdXJjZTsKIHN0cnVjdCB0dG1fcGxhY2U7CiBzdHJ1 Y3QgdHRtX2J1ZmZlcl9vYmplY3Q7CitzdHJ1Y3QgZG1hX2J1Zl9tYXA7CitzdHJ1Y3QgaW9fbWFw cGluZzsKK3N0cnVjdCBzZ190YWJsZTsKK3N0cnVjdCBzY2F0dGVybGlzdDsKIAogc3RydWN0IHR0 bV9yZXNvdXJjZV9tYW5hZ2VyX2Z1bmMgewogCS8qKgpAQCAtMTc2LDYgKzE4Miw0NSBAQCBzdHJ1 Y3QgdHRtX3Jlc291cmNlIHsKIAlzdHJ1Y3QgdHRtX2J1c19wbGFjZW1lbnQgYnVzOwogfTsKIAor LyoqCisgKiBzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9pb21hcCAtIFNwZWNpYWxpemF0aW9uIGZvciBh IHN0cnVjdCBpb19tYXBwaW5nICsKKyAqIHN0cnVjdCBzZ190YWJsZSBiYWNrZWQgc3RydWN0IHR0 bV9yZXNvdXJjZS4KKyAqIEBiYXNlOiBFbWJlZGRlZCBzdHJ1Y3QgdHRtX2ttYXBfaXRlciBwcm92 aWRpbmcgdGhlIHVzYWdlIGludGVyZmFjZS4KKyAqIEBpb21hcDogc3RydWN0IGlvX21hcHBpbmcg cmVwcmVzZW50aW5nIHRoZSB1bmRlcmx5aW5nIGxpbmVhciBpb19tZW1vcnkuCisgKiBAc3Q6IHNn X3RhYmxlIGludG8gQGlvbWFwLCByZXByZXNlbnRpbmcgdGhlIG1lbW9yeSBvZiB0aGUgc3RydWN0 IHR0bV9yZXNvdXJjZS4KKyAqIEBzdGFydDogT2Zmc2V0IHRoYXQgbmVlZHMgdG8gYmUgc3VidHJh Y3RlZCBmcm9tIEBzdCB0byBtYWtlCisgKiBzZ19kbWFfYWRkcmVzcyhzdC0+c2dsKSAtIEBzdGFy dCA9PSAwIGZvciBAaW9tYXAgc3RhcnQuCisgKiBAY2FjaGU6IFNjYXR0ZXJsaXN0IHRyYXZlcnNh bCBjYWNoZSBmb3IgZmFzdCBsb29rdXBzLgorICogQGNhY2hlLnNnOiBQb2ludGVyIHRvIHRoZSBj dXJyZW50bHkgY2FjaGVkIHNjYXR0ZXJsaXN0IHNlZ21lbnQuCisgKiBAY2FjaGUuaTogRmlyc3Qg aW5kZXggb2YgQHNnLiBQQUdFX1NJWkUgZ3JhbnVsYXJpdHkuCisgKiBAY2FjaGUuZW5kOiBMYXN0 IGluZGV4ICsgMSBvZiBAc2cuIFBBR0VfU0laRSBncmFudWxhcml0eS4KKyAqIEBjYWNoZS5vZmZz OiBGaXJzdCBvZmZzZXQgaW50byBAaW9tYXAgb2YgQHNnLiBQQUdFX1NJWkUgZ3JhbnVsYXJpdHku CisgKi8KK3N0cnVjdCB0dG1fa21hcF9pdGVyX2lvbWFwIHsKKwlzdHJ1Y3QgdHRtX2ttYXBfaXRl ciBiYXNlOworCXN0cnVjdCBpb19tYXBwaW5nICppb21hcDsKKwlzdHJ1Y3Qgc2dfdGFibGUgKnN0 OworCXJlc291cmNlX3NpemVfdCBzdGFydDsKKwlzdHJ1Y3QgeworCQlzdHJ1Y3Qgc2NhdHRlcmxp c3QgKnNnOworCQlwZ29mZl90IGk7CisJCXBnb2ZmX3QgZW5kOworCQlwZ29mZl90IG9mZnM7CisJ fSBjYWNoZTsKK307CisKKy8qKgorICogc3RydWN0IHR0bV9rbWFwX2l0ZXJfbGluZWFyX2lvIC0g SXRlcmF0b3Igc3BlY2lhbGl6YXRpb24gZm9yIGxpbmVhciBpbworICogQGJhc2U6IFRoZSBiYXNl IGl0ZXJhdG9yCisgKiBAZG1hcDogUG9pbnRzIHRvIHRoZSBzdGFydGluZyBhZGRyZXNzIG9mIHRo ZSByZWdpb24KKyAqIEBuZWVkc191bm1hcDogV2hldGhlciB3ZSBuZWVkIHRvIHVubWFwIG9uIGZp bmkKKyAqLworc3RydWN0IHR0bV9rbWFwX2l0ZXJfbGluZWFyX2lvIHsKKwlzdHJ1Y3QgdHRtX2tt YXBfaXRlciBiYXNlOworCXN0cnVjdCBkbWFfYnVmX21hcCBkbWFwOworCWJvb2wgbmVlZHNfdW5t YXA7Cit9OworCiAvKioKICAqIHR0bV9yZXNvdXJjZV9tYW5hZ2VyX3NldF91c2VkCiAgKgpAQCAt MjM3LDQgKzI4MiwyMCBAQCBpbnQgdHRtX3Jlc291cmNlX21hbmFnZXJfZXZpY3RfYWxsKHN0cnVj dCB0dG1fZGV2aWNlICpiZGV2LAogdm9pZCB0dG1fcmVzb3VyY2VfbWFuYWdlcl9kZWJ1ZyhzdHJ1 Y3QgdHRtX3Jlc291cmNlX21hbmFnZXIgKm1hbiwKIAkJCQlzdHJ1Y3QgZHJtX3ByaW50ZXIgKnAp OwogCitzdHJ1Y3QgdHRtX2ttYXBfaXRlciAqCit0dG1fa21hcF9pdGVyX2lvbWFwX2luaXQoc3Ry dWN0IHR0bV9rbWFwX2l0ZXJfaW9tYXAgKml0ZXJfaW8sCisJCQkgc3RydWN0IGlvX21hcHBpbmcg KmlvbWFwLAorCQkJIHN0cnVjdCBzZ190YWJsZSAqc3QsCisJCQkgcmVzb3VyY2Vfc2l6ZV90IHN0 YXJ0KTsKKworc3RydWN0IHR0bV9rbWFwX2l0ZXJfbGluZWFyX2lvOworCitzdHJ1Y3QgdHRtX2tt YXBfaXRlciAqCit0dG1fa21hcF9pdGVyX2xpbmVhcl9pb19pbml0KHN0cnVjdCB0dG1fa21hcF9p dGVyX2xpbmVhcl9pbyAqaXRlcl9pbywKKwkJCSAgICAgc3RydWN0IHR0bV9kZXZpY2UgKmJkZXYs CisJCQkgICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKm1lbSk7CisKK3ZvaWQgdHRtX2ttYXBfaXRl cl9saW5lYXJfaW9fZmluaShzdHJ1Y3QgdHRtX2ttYXBfaXRlcl9saW5lYXJfaW8gKml0ZXJfaW8s CisJCQkJICBzdHJ1Y3QgdHRtX2RldmljZSAqYmRldiwKKwkJCQkgIHN0cnVjdCB0dG1fcmVzb3Vy Y2UgKm1lbSk7CiAjZW5kaWYKZGlmZiAtLWdpdCBhL2luY2x1ZGUvZHJtL3R0bS90dG1fdHQuaCBi L2luY2x1ZGUvZHJtL3R0bS90dG1fdHQuaAppbmRleCAxMzRkMDllZjc3NjYuLjMxMDIwNTlkYjcy NiAxMDA2NDQKLS0tIGEvaW5jbHVkZS9kcm0vdHRtL3R0bV90dC5oCisrKyBiL2luY2x1ZGUvZHJt L3R0bS90dG1fdHQuaApAQCAtMjksNiArMjksNyBAQAogCiAjaW5jbHVkZSA8bGludXgvdHlwZXMu aD4KICNpbmNsdWRlIDxkcm0vdHRtL3R0bV9jYWNoaW5nLmg+CisjaW5jbHVkZSA8ZHJtL3R0bS90 dG1fa21hcF9pdGVyLmg+CiAKIHN0cnVjdCB0dG1fYm9fZGV2aWNlOwogc3RydWN0IHR0bV90dDsK QEAgLTY5LDYgKzcwLDE4IEBAIHN0cnVjdCB0dG1fdHQgewogCWVudW0gdHRtX2NhY2hpbmcgY2Fj aGluZzsKIH07CiAKKy8qKgorICogc3RydWN0IHR0bV9rbWFwX2l0ZXJfdHQgLSBTcGVjaWFsaXph dGlvbiBvZiBhIG1hcHBpZyBpdGVyYXRvciBmb3IgYSB0dC4KKyAqIEBiYXNlOiBFbWJlZGRlZCBz dHJ1Y3QgdHRtX2ttYXBfaXRlciBwcm92aWRpbmcgdGhlIHVzYWdlIGludGVyZmFjZQorICogQHR0 OiBDYWNoZWQgc3RydWN0IHR0bV90dC4KKyAqIEBwcm90OiBDYWNoZWQgcGFnZSBwcm90ZWN0aW9u IGZvciBtYXBwaW5nLgorICovCitzdHJ1Y3QgdHRtX2ttYXBfaXRlcl90dCB7CisJc3RydWN0IHR0 bV9rbWFwX2l0ZXIgYmFzZTsKKwlzdHJ1Y3QgdHRtX3R0ICp0dDsKKwlwZ3Byb3RfdCBwcm90Owor fTsKKwogc3RhdGljIGlubGluZSBib29sIHR0bV90dF9pc19wb3B1bGF0ZWQoc3RydWN0IHR0bV90 dCAqdHQpCiB7CiAJcmV0dXJuIHR0LT5wYWdlX2ZsYWdzICYgVFRNX1BBR0VfRkxBR19QUklWX1BP UFVMQVRFRDsKQEAgLTE1OSw2ICsxNzIsOSBAQCB2b2lkIHR0bV90dF91bnBvcHVsYXRlKHN0cnVj dCB0dG1fZGV2aWNlICpiZGV2LCBzdHJ1Y3QgdHRtX3R0ICp0dG0pOwogCiB2b2lkIHR0bV90dF9t Z3JfaW5pdCh1bnNpZ25lZCBsb25nIG51bV9wYWdlcywgdW5zaWduZWQgbG9uZyBudW1fZG1hMzJf cGFnZXMpOwogCitzdHJ1Y3QgdHRtX2ttYXBfaXRlciAqdHRtX2ttYXBfaXRlcl90dF9pbml0KHN0 cnVjdCB0dG1fa21hcF9pdGVyX3R0ICppdGVyX3R0LAorCQkJCQkgICAgc3RydWN0IHR0bV90dCAq dHQpOworCiAjaWYgSVNfRU5BQkxFRChDT05GSUdfQUdQKQogI2luY2x1ZGUgPGxpbnV4L2FncF9i YWNrZW5kLmg+CiAKLS0gCjIuMzEuMQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJl ZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGlu Zm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DF1DC4708F for ; Mon, 31 May 2021 17:23:08 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 26ED1610A8 for ; Mon, 31 May 2021 17:23:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26ED1610A8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7ECF16E972; Mon, 31 May 2021 17:22:54 +0000 (UTC) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by gabe.freedesktop.org (Postfix) with ESMTPS id 471A26E96C; Mon, 31 May 2021 17:22:49 +0000 (UTC) IronPort-SDR: Dfs6vwlEkP0xkWVUCan6whSQ26/d7rGVlpXfEMRnyKYL/AErGI1gYTFcA94a0+z4jq0gyIaOjD EmoykgAHCGBQ== X-IronPort-AV: E=McAfee;i="6200,9189,10001"; a="201520206" X-IronPort-AV: E=Sophos;i="5.83,237,1616482800"; d="scan'208";a="201520206" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2021 10:22:48 -0700 IronPort-SDR: Uz8DGq9v3SWEoo2gQLFRuu+2VQo2VsBUdEPubvqGYh1Tq9q2Lmcqap6g/Ie25oAVHUDEOfYol3 jD5NgMkFvyoA== X-IronPort-AV: E=Sophos;i="5.83,237,1616482800"; d="scan'208";a="416210992" Received: from fnygreen-mobl1.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.142]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2021 10:22:47 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v8 06/15] drm/ttm: Add a generic TTM memcpy move for page-based iomem Date: Mon, 31 May 2021 19:22:19 +0200 Message-Id: <20210531172228.70846-7-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210531172228.70846-1-thomas.hellstrom@linux.intel.com> References: <20210531172228.70846-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , =?UTF-8?q?Christian=20K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The internal ttm_bo_util memcpy uses ioremap functionality, and while it probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free() callbacks, that would cause problems with fault(). Instead, implement a method mapping page-by-page using kmap_local() semantics. As an additional benefit we then avoid the occasional global TLB flushes of ioremap() and consuming ioremap space, elimination of a critical point of failure and with a slight change of semantics we could also push the memcpy out async for testing and async driver development purposes. A special linear iomem iterator is introduced internally to mimic the old ioremap behaviour for code-paths that can't immediately be ported over. This adds to the code size and should be considered a temporary solution. Looking at the code we have a lot of checks for iomap tagged pointers. Ideally we should extend the core memremap functions to also accept uncached memory and kmap_local functionality. Then we could strip a lot of code. Cc: Christian König Signed-off-by: Thomas Hellström Reviewed-by: Christian König --- v3: - Split up in various TTM files and addressed review comments by Christian König. Tested and fixed legacy iomap memcpy path on i915. v4: - Fix an uninitialized variable Reported by: kernel test robot Reported by: Dan Carpenter - Minor change to the ttm_move_memcpy() interface. - Gracefully handle lack of memremap() support on memcpy (Reported by Matthew Auld) - Minor style fix (Reported by Matthew Auld) v8: - Change num_pages from pgoff_t to uint32_t (Suggested by Christian König) --- drivers/gpu/drm/ttm/ttm_bo_util.c | 280 ++++++++++------------------- drivers/gpu/drm/ttm/ttm_module.c | 35 ++++ drivers/gpu/drm/ttm/ttm_resource.c | 193 ++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_tt.c | 42 +++++ include/drm/ttm/ttm_bo_driver.h | 28 +++ include/drm/ttm/ttm_caching.h | 2 + include/drm/ttm/ttm_kmap_iter.h | 61 +++++++ include/drm/ttm/ttm_resource.h | 61 +++++++ include/drm/ttm/ttm_tt.h | 16 ++ 9 files changed, 536 insertions(+), 182 deletions(-) create mode 100644 include/drm/ttm/ttm_kmap_iter.h diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index ae8b61460724..a800998a12b0 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -72,190 +72,126 @@ void ttm_mem_io_free(struct ttm_device *bdev, mem->bus.addr = NULL; } -static int ttm_resource_ioremap(struct ttm_device *bdev, - struct ttm_resource *mem, - void **virtual) +/** + * ttm_move_memcpy - Helper to perform a memcpy ttm move operation. + * @bo: The struct ttm_buffer_object. + * @new_mem: The struct ttm_resource we're moving to (copy destination). + * @new_iter: A struct ttm_kmap_iter representing the destination resource. + * @src_iter: A struct ttm_kmap_iter representing the source resource. + * + * This function is intended to be able to move out async under a + * dma-fence if desired. + */ +void ttm_move_memcpy(struct ttm_buffer_object *bo, + u32 num_pages, + struct ttm_kmap_iter *dst_iter, + struct ttm_kmap_iter *src_iter) { - int ret; - void *addr; - - *virtual = NULL; - ret = ttm_mem_io_reserve(bdev, mem); - if (ret || !mem->bus.is_iomem) - return ret; + const struct ttm_kmap_iter_ops *dst_ops = dst_iter->ops; + const struct ttm_kmap_iter_ops *src_ops = src_iter->ops; + struct ttm_tt *ttm = bo->ttm; + struct dma_buf_map src_map, dst_map; + pgoff_t i; - if (mem->bus.addr) { - addr = mem->bus.addr; - } else { - size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT; + /* Single TTM move. NOP */ + if (dst_ops->maps_tt && src_ops->maps_tt) + return; - if (mem->bus.caching == ttm_write_combined) - addr = ioremap_wc(mem->bus.offset, bus_size); -#ifdef CONFIG_X86 - else if (mem->bus.caching == ttm_cached) - addr = ioremap_cache(mem->bus.offset, bus_size); -#endif - else - addr = ioremap(mem->bus.offset, bus_size); - if (!addr) { - ttm_mem_io_free(bdev, mem); - return -ENOMEM; + /* Don't move nonexistent data. Clear destination instead. */ + if (src_ops->maps_tt && (!ttm || !ttm_tt_is_populated(ttm))) { + if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC)) + return; + + for (i = 0; i < num_pages; ++i) { + dst_ops->map_local(dst_iter, &dst_map, i); + if (dst_map.is_iomem) + memset_io(dst_map.vaddr_iomem, 0, PAGE_SIZE); + else + memset(dst_map.vaddr, 0, PAGE_SIZE); + if (dst_ops->unmap_local) + dst_ops->unmap_local(dst_iter, &dst_map); } + return; } - *virtual = addr; - return 0; -} - -static void ttm_resource_iounmap(struct ttm_device *bdev, - struct ttm_resource *mem, - void *virtual) -{ - if (virtual && mem->bus.addr == NULL) - iounmap(virtual); - ttm_mem_io_free(bdev, mem); -} - -static int ttm_copy_io_page(void *dst, void *src, unsigned long page) -{ - uint32_t *dstP = - (uint32_t *) ((unsigned long)dst + (page << PAGE_SHIFT)); - uint32_t *srcP = - (uint32_t *) ((unsigned long)src + (page << PAGE_SHIFT)); - - int i; - for (i = 0; i < PAGE_SIZE / sizeof(uint32_t); ++i) - iowrite32(ioread32(srcP++), dstP++); - return 0; -} - -static int ttm_copy_io_ttm_page(struct ttm_tt *ttm, void *src, - unsigned long page, - pgprot_t prot) -{ - struct page *d = ttm->pages[page]; - void *dst; - - if (!d) - return -ENOMEM; - - src = (void *)((unsigned long)src + (page << PAGE_SHIFT)); - dst = kmap_atomic_prot(d, prot); - if (!dst) - return -ENOMEM; - - memcpy_fromio(dst, src, PAGE_SIZE); - - kunmap_atomic(dst); - - return 0; -} - -static int ttm_copy_ttm_io_page(struct ttm_tt *ttm, void *dst, - unsigned long page, - pgprot_t prot) -{ - struct page *s = ttm->pages[page]; - void *src; - if (!s) - return -ENOMEM; - - dst = (void *)((unsigned long)dst + (page << PAGE_SHIFT)); - src = kmap_atomic_prot(s, prot); - if (!src) - return -ENOMEM; - - memcpy_toio(dst, src, PAGE_SIZE); - - kunmap_atomic(src); + for (i = 0; i < num_pages; ++i) { + dst_ops->map_local(dst_iter, &dst_map, i); + src_ops->map_local(src_iter, &src_map, i); + + if (!src_map.is_iomem && !dst_map.is_iomem) { + memcpy(dst_map.vaddr, src_map.vaddr, PAGE_SIZE); + } else if (!src_map.is_iomem) { + dma_buf_map_memcpy_to(&dst_map, src_map.vaddr, + PAGE_SIZE); + } else if (!dst_map.is_iomem) { + memcpy_fromio(dst_map.vaddr, src_map.vaddr_iomem, + PAGE_SIZE); + } else { + int j; + u32 __iomem *src = src_map.vaddr_iomem; + u32 __iomem *dst = dst_map.vaddr_iomem; - return 0; + for (j = 0; j < (PAGE_SIZE / sizeof(u32)); ++j) + iowrite32(ioread32(src++), dst++); + } + if (src_ops->unmap_local) + src_ops->unmap_local(src_iter, &src_map); + if (dst_ops->unmap_local) + dst_ops->unmap_local(dst_iter, &dst_map); + } } +EXPORT_SYMBOL(ttm_move_memcpy); int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem) + struct ttm_resource *dst_mem) { struct ttm_device *bdev = bo->bdev; - struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type); + struct ttm_resource_manager *dst_man = + ttm_manager_type(bo->bdev, dst_mem->mem_type); struct ttm_tt *ttm = bo->ttm; - struct ttm_resource *old_mem = &bo->mem; - struct ttm_resource old_copy = *old_mem; - void *old_iomap; - void *new_iomap; - int ret; - unsigned long i; - - ret = ttm_bo_wait_ctx(bo, ctx); - if (ret) - return ret; - - ret = ttm_resource_ioremap(bdev, old_mem, &old_iomap); - if (ret) - return ret; - ret = ttm_resource_ioremap(bdev, new_mem, &new_iomap); - if (ret) - goto out; - - /* - * Single TTM move. NOP. - */ - if (old_iomap == NULL && new_iomap == NULL) - goto out2; - - /* - * Don't move nonexistent data. Clear destination instead. - */ - if (old_iomap == NULL && - (ttm == NULL || (!ttm_tt_is_populated(ttm) && - !(ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)))) { - memset_io(new_iomap, 0, new_mem->num_pages*PAGE_SIZE); - goto out2; - } + struct ttm_resource *src_mem = &bo->mem; + struct ttm_resource_manager *src_man = + ttm_manager_type(bdev, src_mem->mem_type); + struct ttm_resource src_copy = *src_mem; + union { + struct ttm_kmap_iter_tt tt; + struct ttm_kmap_iter_linear_io io; + } _dst_iter, _src_iter; + struct ttm_kmap_iter *dst_iter, *src_iter; + int ret = 0; - /* - * TTM might be null for moves within the same region. - */ - if (ttm) { + if (ttm && ((ttm->page_flags & TTM_PAGE_FLAG_SWAPPED) || + dst_man->use_tt)) { ret = ttm_tt_populate(bdev, ttm, ctx); if (ret) - goto out1; + return ret; } - for (i = 0; i < new_mem->num_pages; ++i) { - if (old_iomap == NULL) { - pgprot_t prot = ttm_io_prot(bo, old_mem, PAGE_KERNEL); - ret = ttm_copy_ttm_io_page(ttm, new_iomap, i, - prot); - } else if (new_iomap == NULL) { - pgprot_t prot = ttm_io_prot(bo, new_mem, PAGE_KERNEL); - ret = ttm_copy_io_ttm_page(ttm, old_iomap, i, - prot); - } else { - ret = ttm_copy_io_page(new_iomap, old_iomap, i); - } - if (ret) - goto out1; + dst_iter = ttm_kmap_iter_linear_io_init(&_dst_iter.io, bdev, dst_mem); + if (PTR_ERR(dst_iter) == -EINVAL && dst_man->use_tt) + dst_iter = ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm); + if (IS_ERR(dst_iter)) + return PTR_ERR(dst_iter); + + src_iter = ttm_kmap_iter_linear_io_init(&_src_iter.io, bdev, src_mem); + if (PTR_ERR(src_iter) == -EINVAL && src_man->use_tt) + src_iter = ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm); + if (IS_ERR(src_iter)) { + ret = PTR_ERR(src_iter); + goto out_src_iter; } - mb(); -out2: - old_copy = *old_mem; - ttm_bo_assign_mem(bo, new_mem); - - if (!man->use_tt) - ttm_bo_tt_destroy(bo); + ttm_move_memcpy(bo, dst_mem->num_pages, dst_iter, src_iter); + src_copy = *src_mem; + ttm_bo_move_sync_cleanup(bo, dst_mem); -out1: - ttm_resource_iounmap(bdev, old_mem, new_iomap); -out: - ttm_resource_iounmap(bdev, &old_copy, old_iomap); + if (!src_iter->ops->maps_tt) + ttm_kmap_iter_linear_io_fini(&_src_iter.io, bdev, &src_copy); +out_src_iter: + if (!dst_iter->ops->maps_tt) + ttm_kmap_iter_linear_io_fini(&_dst_iter.io, bdev, dst_mem); - /* - * On error, keep the mm node! - */ - if (!ret) - ttm_resource_free(bo, &old_copy); return ret; } EXPORT_SYMBOL(ttm_bo_move_memcpy); @@ -336,27 +272,7 @@ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res, man = ttm_manager_type(bo->bdev, res->mem_type); caching = man->use_tt ? bo->ttm->caching : res->bus.caching; - /* Cached mappings need no adjustment */ - if (caching == ttm_cached) - return tmp; - -#if defined(__i386__) || defined(__x86_64__) - if (caching == ttm_write_combined) - tmp = pgprot_writecombine(tmp); - else if (boot_cpu_data.x86 > 3) - tmp = pgprot_noncached(tmp); -#endif -#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \ - defined(__powerpc__) || defined(__mips__) - if (caching == ttm_write_combined) - tmp = pgprot_writecombine(tmp); - else - tmp = pgprot_noncached(tmp); -#endif -#if defined(__sparc__) - tmp = pgprot_noncached(tmp); -#endif - return tmp; + return ttm_prot_from_caching(caching, tmp); } EXPORT_SYMBOL(ttm_io_prot); diff --git a/drivers/gpu/drm/ttm/ttm_module.c b/drivers/gpu/drm/ttm/ttm_module.c index 56b0efdba1a9..997c458f68a9 100644 --- a/drivers/gpu/drm/ttm/ttm_module.c +++ b/drivers/gpu/drm/ttm/ttm_module.c @@ -31,12 +31,47 @@ */ #include #include +#include #include #include #include +#include #include "ttm_module.h" +/** + * ttm_prot_from_caching - Modify the page protection according to the + * ttm cacing mode + * @caching: The ttm caching mode + * @tmp: The original page protection + * + * Return: The modified page protection + */ +pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp) +{ + /* Cached mappings need no adjustment */ + if (caching == ttm_cached) + return tmp; + +#if defined(__i386__) || defined(__x86_64__) + if (caching == ttm_write_combined) + tmp = pgprot_writecombine(tmp); + else if (boot_cpu_data.x86 > 3) + tmp = pgprot_noncached(tmp); +#endif +#if defined(__ia64__) || defined(__arm__) || defined(__aarch64__) || \ + defined(__powerpc__) || defined(__mips__) + if (caching == ttm_write_combined) + tmp = pgprot_writecombine(tmp); + else + tmp = pgprot_noncached(tmp); +#endif +#if defined(__sparc__) + tmp = pgprot_noncached(tmp); +#endif + return tmp; +} + struct dentry *ttm_debugfs_root; static int __init ttm_init(void) diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 59e2b7157e41..931bcd1a9b9b 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -22,6 +22,10 @@ * Authors: Christian König */ +#include +#include +#include + #include #include @@ -147,3 +151,192 @@ void ttm_resource_manager_debug(struct ttm_resource_manager *man, man->func->debug(man, p); } EXPORT_SYMBOL(ttm_resource_manager_debug); + +static void ttm_kmap_iter_iomap_map_local(struct ttm_kmap_iter *iter, + struct dma_buf_map *dmap, + pgoff_t i) +{ + struct ttm_kmap_iter_iomap *iter_io = + container_of(iter, typeof(*iter_io), base); + void __iomem *addr; + +retry: + while (i >= iter_io->cache.end) { + iter_io->cache.sg = iter_io->cache.sg ? + sg_next(iter_io->cache.sg) : iter_io->st->sgl; + iter_io->cache.i = iter_io->cache.end; + iter_io->cache.end += sg_dma_len(iter_io->cache.sg) >> + PAGE_SHIFT; + iter_io->cache.offs = sg_dma_address(iter_io->cache.sg) - + iter_io->start; + } + + if (i < iter_io->cache.i) { + iter_io->cache.end = 0; + iter_io->cache.sg = NULL; + goto retry; + } + + addr = io_mapping_map_local_wc(iter_io->iomap, iter_io->cache.offs + + (((resource_size_t)i - iter_io->cache.i) + << PAGE_SHIFT)); + dma_buf_map_set_vaddr_iomem(dmap, addr); +} + +static void ttm_kmap_iter_iomap_unmap_local(struct ttm_kmap_iter *iter, + struct dma_buf_map *map) +{ + io_mapping_unmap_local(map->vaddr_iomem); +} + +static const struct ttm_kmap_iter_ops ttm_kmap_iter_io_ops = { + .map_local = ttm_kmap_iter_iomap_map_local, + .unmap_local = ttm_kmap_iter_iomap_unmap_local, + .maps_tt = false, +}; + +/** + * ttm_kmap_iter_iomap_init - Initialize a struct ttm_kmap_iter_iomap + * @iter_io: The struct ttm_kmap_iter_iomap to initialize. + * @iomap: The struct io_mapping representing the underlying linear io_memory. + * @st: sg_table into @iomap, representing the memory of the struct + * ttm_resource. + * @start: Offset that needs to be subtracted from @st to make + * sg_dma_address(st->sgl) - @start == 0 for @iomap start. + * + * Return: Pointer to the embedded struct ttm_kmap_iter. + */ +struct ttm_kmap_iter * +ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io, + struct io_mapping *iomap, + struct sg_table *st, + resource_size_t start) +{ + iter_io->base.ops = &ttm_kmap_iter_io_ops; + iter_io->iomap = iomap; + iter_io->st = st; + iter_io->start = start; + memset(&iter_io->cache, 0, sizeof(iter_io->cache)); + + return &iter_io->base; +} +EXPORT_SYMBOL(ttm_kmap_iter_iomap_init); + +/** + * DOC: Linear io iterator + * + * This code should die in the not too near future. Best would be if we could + * make io-mapping use memremap for all io memory, and have memremap + * implement a kmap_local functionality. We could then strip a huge amount of + * code. These linear io iterators are implemented to mimic old functionality, + * and they don't use kmap_local semantics at all internally. Rather ioremap or + * friends, and at least on 32-bit they add global TLB flushes and points + * of failure. + */ + +static void ttm_kmap_iter_linear_io_map_local(struct ttm_kmap_iter *iter, + struct dma_buf_map *dmap, + pgoff_t i) +{ + struct ttm_kmap_iter_linear_io *iter_io = + container_of(iter, typeof(*iter_io), base); + + *dmap = iter_io->dmap; + dma_buf_map_incr(dmap, i * PAGE_SIZE); +} + +static const struct ttm_kmap_iter_ops ttm_kmap_iter_linear_io_ops = { + .map_local = ttm_kmap_iter_linear_io_map_local, + .maps_tt = false, +}; + +/** + * ttm_kmap_iter_linear_io_init - Initialize an iterator for linear io memory + * @iter_io: The iterator to initialize + * @bdev: The TTM device + * @mem: The ttm resource representing the iomap. + * + * This function is for internal TTM use only. It sets up a memcpy kmap iterator + * pointing at a linear chunk of io memory. + * + * Return: A pointer to the embedded struct ttm_kmap_iter or error pointer on + * failure. + */ +struct ttm_kmap_iter * +ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io, + struct ttm_device *bdev, + struct ttm_resource *mem) +{ + int ret; + + ret = ttm_mem_io_reserve(bdev, mem); + if (ret) + goto out_err; + if (!mem->bus.is_iomem) { + ret = -EINVAL; + goto out_io_free; + } + + if (mem->bus.addr) { + dma_buf_map_set_vaddr(&iter_io->dmap, mem->bus.addr); + iter_io->needs_unmap = false; + } else { + size_t bus_size = (size_t)mem->num_pages << PAGE_SHIFT; + + iter_io->needs_unmap = true; + memset(&iter_io->dmap, 0, sizeof(iter_io->dmap)); + if (mem->bus.caching == ttm_write_combined) + dma_buf_map_set_vaddr_iomem(&iter_io->dmap, + ioremap_wc(mem->bus.offset, + bus_size)); + else if (mem->bus.caching == ttm_cached) + dma_buf_map_set_vaddr(&iter_io->dmap, + memremap(mem->bus.offset, bus_size, + MEMREMAP_WB | + MEMREMAP_WT | + MEMREMAP_WC)); + + /* If uncached requested or if mapping cached or wc failed */ + if (dma_buf_map_is_null(&iter_io->dmap)) + dma_buf_map_set_vaddr_iomem(&iter_io->dmap, + ioremap(mem->bus.offset, + bus_size)); + + if (dma_buf_map_is_null(&iter_io->dmap)) { + ret = -ENOMEM; + goto out_io_free; + } + } + + iter_io->base.ops = &ttm_kmap_iter_linear_io_ops; + return &iter_io->base; + +out_io_free: + ttm_mem_io_free(bdev, mem); +out_err: + return ERR_PTR(ret); +} + +/** + * ttm_kmap_iter_linear_io_fini - Clean up an iterator for linear io memory + * @iter_io: The iterator to initialize + * @bdev: The TTM device + * @mem: The ttm resource representing the iomap. + * + * This function is for internal TTM use only. It cleans up a memcpy kmap + * iterator initialized by ttm_kmap_iter_linear_io_init. + */ +void +ttm_kmap_iter_linear_io_fini(struct ttm_kmap_iter_linear_io *iter_io, + struct ttm_device *bdev, + struct ttm_resource *mem) +{ + if (iter_io->needs_unmap && dma_buf_map_is_set(&iter_io->dmap)) { + if (iter_io->dmap.is_iomem) + iounmap(iter_io->dmap.vaddr_iomem); + else + memunmap(iter_io->dmap.vaddr); + } + + ttm_mem_io_free(bdev, mem); +} diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 539e0232cb3b..0e41227116b1 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -433,3 +433,45 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages) if (!ttm_dma32_pages_limit) ttm_dma32_pages_limit = num_dma32_pages; } + +static void ttm_kmap_iter_tt_map_local(struct ttm_kmap_iter *iter, + struct dma_buf_map *dmap, + pgoff_t i) +{ + struct ttm_kmap_iter_tt *iter_tt = + container_of(iter, typeof(*iter_tt), base); + + dma_buf_map_set_vaddr(dmap, kmap_local_page_prot(iter_tt->tt->pages[i], + iter_tt->prot)); +} + +static void ttm_kmap_iter_tt_unmap_local(struct ttm_kmap_iter *iter, + struct dma_buf_map *map) +{ + kunmap_local(map->vaddr); +} + +static const struct ttm_kmap_iter_ops ttm_kmap_iter_tt_ops = { + .map_local = ttm_kmap_iter_tt_map_local, + .unmap_local = ttm_kmap_iter_tt_unmap_local, + .maps_tt = true, +}; + +/** + * ttm_kmap_iter_tt_init - Initialize a struct ttm_kmap_iter_tt + * @iter_tt: The struct ttm_kmap_iter_tt to initialize. + * @tt: Struct ttm_tt holding page pointers of the struct ttm_resource. + * + * Return: Pointer to the embedded struct ttm_kmap_iter. + */ +struct ttm_kmap_iter * +ttm_kmap_iter_tt_init(struct ttm_kmap_iter_tt *iter_tt, + struct ttm_tt *tt) +{ + iter_tt->base.ops = &ttm_kmap_iter_tt_ops; + iter_tt->tt = tt; + iter_tt->prot = ttm_prot_from_caching(tt->caching, PAGE_KERNEL); + + return &iter_tt->base; +} +EXPORT_SYMBOL(ttm_kmap_iter_tt_init); diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index dbccac957f8f..72511992261a 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -40,6 +40,7 @@ #include #include "ttm_bo_api.h" +#include "ttm_kmap_iter.h" #include "ttm_placement.h" #include "ttm_tt.h" #include "ttm_pool.h" @@ -272,6 +273,23 @@ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo, bool pipeline, struct ttm_resource *new_mem); +/** + * ttm_bo_move_accel_cleanup. + * + * @bo: A pointer to a struct ttm_buffer_object. + * @new_mem: struct ttm_resource indicating where to move. + * + * Special case of ttm_bo_move_accel_cleanup where the bo is guaranteed + * by the caller to be idle. Typically used after memcpy buffer moves. + */ +static inline void ttm_bo_move_sync_cleanup(struct ttm_buffer_object *bo, + struct ttm_resource *new_mem) +{ + int ret = ttm_bo_move_accel_cleanup(bo, NULL, true, false, new_mem); + + WARN_ON(ret); +} + /** * ttm_bo_pipeline_gutting. * @@ -332,4 +350,14 @@ int ttm_range_man_init(struct ttm_device *bdev, int ttm_range_man_fini(struct ttm_device *bdev, unsigned type); +void ttm_move_memcpy(struct ttm_buffer_object *bo, + u32 num_pages, + struct ttm_kmap_iter *dst_iter, + struct ttm_kmap_iter *src_iter); + +struct ttm_kmap_iter * +ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io, + struct io_mapping *iomap, + struct sg_table *st, + resource_size_t start); #endif diff --git a/include/drm/ttm/ttm_caching.h b/include/drm/ttm/ttm_caching.h index a0b4a49fa432..3c9dd65f5aaf 100644 --- a/include/drm/ttm/ttm_caching.h +++ b/include/drm/ttm/ttm_caching.h @@ -33,4 +33,6 @@ enum ttm_caching { ttm_cached }; +pgprot_t ttm_prot_from_caching(enum ttm_caching caching, pgprot_t tmp); + #endif diff --git a/include/drm/ttm/ttm_kmap_iter.h b/include/drm/ttm/ttm_kmap_iter.h new file mode 100644 index 000000000000..8bb00fd39d6c --- /dev/null +++ b/include/drm/ttm/ttm_kmap_iter.h @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2021 Intel Corporation + */ +#ifndef __TTM_KMAP_ITER_H__ +#define __TTM_KMAP_ITER_H__ + +#include + +struct ttm_kmap_iter; +struct dma_buf_map; + +/** + * struct ttm_kmap_iter_ops - Ops structure for a struct + * ttm_kmap_iter. + * @maps_tt: Whether the iterator maps TT memory directly, as opposed + * mapping a TT through an aperture. Both these modes have + * struct ttm_resource_manager::use_tt set, but the latter typically + * returns is_iomem == true from ttm_mem_io_reserve. + */ +struct ttm_kmap_iter_ops { + /** + * kmap_local() - Map a PAGE_SIZE part of the resource using + * kmap_local semantics. + * @res_iter: Pointer to the struct ttm_kmap_iter representing + * the resource. + * @dmap: The struct dma_buf_map holding the virtual address after + * the operation. + * @i: The location within the resource to map. PAGE_SIZE granularity. + */ + void (*map_local)(struct ttm_kmap_iter *res_iter, + struct dma_buf_map *dmap, pgoff_t i); + /** + * unmap_local() - Unmap a PAGE_SIZE part of the resource previously + * mapped using kmap_local. + * @res_iter: Pointer to the struct ttm_kmap_iter representing + * the resource. + * @dmap: The struct dma_buf_map holding the virtual address after + * the operation. + */ + void (*unmap_local)(struct ttm_kmap_iter *res_iter, + struct dma_buf_map *dmap); + bool maps_tt; +}; + +/** + * struct ttm_kmap_iter - Iterator for kmap_local type operations on a + * resource. + * @ops: Pointer to the operations struct. + * + * This struct is intended to be embedded in a resource-specific specialization + * implementing operations for the resource. + * + * Nothing stops us from extending the operations to vmap, vmap_pfn etc, + * replacing some or parts of the ttm_bo_util. cpu-map functionality. + */ +struct ttm_kmap_iter { + const struct ttm_kmap_iter_ops *ops; +}; + +#endif /* __TTM_KMAP_ITER_H__ */ diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 890b9d369519..b8dc0bdb0da5 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -27,9 +27,11 @@ #include #include +#include #include #include #include +#include #define TTM_MAX_BO_PRIORITY 4U @@ -38,6 +40,10 @@ struct ttm_resource_manager; struct ttm_resource; struct ttm_place; struct ttm_buffer_object; +struct dma_buf_map; +struct io_mapping; +struct sg_table; +struct scatterlist; struct ttm_resource_manager_func { /** @@ -176,6 +182,45 @@ struct ttm_resource { struct ttm_bus_placement bus; }; +/** + * struct ttm_kmap_iter_iomap - Specialization for a struct io_mapping + + * struct sg_table backed struct ttm_resource. + * @base: Embedded struct ttm_kmap_iter providing the usage interface. + * @iomap: struct io_mapping representing the underlying linear io_memory. + * @st: sg_table into @iomap, representing the memory of the struct ttm_resource. + * @start: Offset that needs to be subtracted from @st to make + * sg_dma_address(st->sgl) - @start == 0 for @iomap start. + * @cache: Scatterlist traversal cache for fast lookups. + * @cache.sg: Pointer to the currently cached scatterlist segment. + * @cache.i: First index of @sg. PAGE_SIZE granularity. + * @cache.end: Last index + 1 of @sg. PAGE_SIZE granularity. + * @cache.offs: First offset into @iomap of @sg. PAGE_SIZE granularity. + */ +struct ttm_kmap_iter_iomap { + struct ttm_kmap_iter base; + struct io_mapping *iomap; + struct sg_table *st; + resource_size_t start; + struct { + struct scatterlist *sg; + pgoff_t i; + pgoff_t end; + pgoff_t offs; + } cache; +}; + +/** + * struct ttm_kmap_iter_linear_io - Iterator specialization for linear io + * @base: The base iterator + * @dmap: Points to the starting address of the region + * @needs_unmap: Whether we need to unmap on fini + */ +struct ttm_kmap_iter_linear_io { + struct ttm_kmap_iter base; + struct dma_buf_map dmap; + bool needs_unmap; +}; + /** * ttm_resource_manager_set_used * @@ -237,4 +282,20 @@ int ttm_resource_manager_evict_all(struct ttm_device *bdev, void ttm_resource_manager_debug(struct ttm_resource_manager *man, struct drm_printer *p); +struct ttm_kmap_iter * +ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io, + struct io_mapping *iomap, + struct sg_table *st, + resource_size_t start); + +struct ttm_kmap_iter_linear_io; + +struct ttm_kmap_iter * +ttm_kmap_iter_linear_io_init(struct ttm_kmap_iter_linear_io *iter_io, + struct ttm_device *bdev, + struct ttm_resource *mem); + +void ttm_kmap_iter_linear_io_fini(struct ttm_kmap_iter_linear_io *iter_io, + struct ttm_device *bdev, + struct ttm_resource *mem); #endif diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h index 134d09ef7766..3102059db726 100644 --- a/include/drm/ttm/ttm_tt.h +++ b/include/drm/ttm/ttm_tt.h @@ -29,6 +29,7 @@ #include #include +#include struct ttm_bo_device; struct ttm_tt; @@ -69,6 +70,18 @@ struct ttm_tt { enum ttm_caching caching; }; +/** + * struct ttm_kmap_iter_tt - Specialization of a mappig iterator for a tt. + * @base: Embedded struct ttm_kmap_iter providing the usage interface + * @tt: Cached struct ttm_tt. + * @prot: Cached page protection for mapping. + */ +struct ttm_kmap_iter_tt { + struct ttm_kmap_iter base; + struct ttm_tt *tt; + pgprot_t prot; +}; + static inline bool ttm_tt_is_populated(struct ttm_tt *tt) { return tt->page_flags & TTM_PAGE_FLAG_PRIV_POPULATED; @@ -159,6 +172,9 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm); void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages); +struct ttm_kmap_iter *ttm_kmap_iter_tt_init(struct ttm_kmap_iter_tt *iter_tt, + struct ttm_tt *tt); + #if IS_ENABLED(CONFIG_AGP) #include -- 2.31.1