From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17741C43462 for ; Tue, 11 May 2021 13:26:22 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B64176191D for ; Tue, 11 May 2021 13:26:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B64176191D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D3C226EA3A; Tue, 11 May 2021 13:26:17 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1FD496EA3D; Tue, 11 May 2021 13:26:16 +0000 (UTC) IronPort-SDR: V8TH/naUq8SSqd6Wfpvb/YizZiijXC+dGLPUXAGm2cgESABK+d8FHHvU25meOXa3KdITdrIscV 3agVenfU9uyQ== X-IronPort-AV: E=McAfee;i="6200,9189,9980"; a="196345411" X-IronPort-AV: E=Sophos;i="5.82,290,1613462400"; d="scan'208";a="196345411" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2021 06:26:16 -0700 IronPort-SDR: kIC6BdFzd2UKWk4SBmWo7ZBuvyiLPWOLATdPmkj/dVATdbcQ+walh6Rg9dYq8yn0gcP7qKEoIa jP91SjsgjGhg== X-IronPort-AV: E=Sophos;i="5.82,290,1613462400"; d="scan'208";a="468905434" Received: from kjeldbeg-mobl2.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.168]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2021 06:26:14 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Tue, 11 May 2021 15:25:23 +0200 Message-Id: <20210511132525.377190-6-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210511132525.377190-1-thomas.hellstrom@linux.intel.com> References: <20210511132525.377190-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 5/7] drm/i915/ttm, drm/ttm: Add a generic TTM memcpy move for page-based iomem X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , =?UTF-8?q?Christian=20K=C3=B6nig?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" VGhlIGludGVybmFsIHR0bV9ib191dGlsIG1lbWNweSB1c2VzIHZtYXAgZnVuY3Rpb25hbGl0eSwg YW5kIHdoaWxlIGl0CnByb2JhYmx5IG1pZ2h0IGJlIHBvc3NpYmxlIHRvIHVzZSBpdCBmb3IgY29w eWluZyBpbi0gYW5kIG91dCBvZgpzZ2xpc3QgcmVwcmVzZW50ZWQgaW8gbWVtb3J5LCB1c2luZyBp b19tZW1fcmVzZXJ2ZSgpIC8gaW9fbWVtX2ZyZWUoKQpjYWxsYmFja3MsIHRoYXQgd291bGQgY2F1 c2UgcHJvYmxlbXMgd2l0aCBmYXVsdCgpLgpJbnN0ZWFkLCBpbXBsZW1lbnQgYSBtZXRob2QgbWFw cGluZyBwYWdlLWJ5LXBhZ2UgdXNpbmcga21hcF9sb2NhbCgpCnNlbWFudGljcy4gQXMgYW4gYWRk aXRpb25hbCBiZW5lZml0IHdlIHRoZW4gYXZvaWQgdGhlIG9jY2FzaW9uYWwgZ2xvYmFsClRMQiBm bHVzaGVzIG9mIHZtYXAoKSBhbmQgY29uc3VtaW5nIHZtYXAgc3BhY2UsIGVsaW1pbmF0aW9uIG9m IGEgY3JpdGljYWwKcG9pbnQgb2YgZmFpbHVyZSBhbmQgd2l0aCBhIHNsaWdodCBjaGFuZ2Ugb2Yg c2VtYW50aWNzIHdlIGNvdWxkIGFsc28gcHVzaAp0aGUgbWVtY3B5IG91dCBhc3luYyBmb3IgdGVz dGluZyBhbmQgYXN5bmMgZHJpdmVyIGRldmVscG1lbnQgcHVycG9zZXMuClB1c2hpbmcgb3V0IGFz eW5jIGNhbiBiZSBkb25lIHNpbmNlIHRoZXJlIGlzIG5vIG1lbW9yeSBhbGxvY2F0aW9uIGdvaW5n IG9uCnRoYXQgY291bGQgdmlvbGF0ZSB0aGUgZG1hX2ZlbmNlIGxvY2tkZXAgcnVsZXMuCgpOb3Rl IHRoYXQgZHJpdmVycyB0aGF0IGRvbid0IHdhbnQgdG8gdXNlIHN0cnVjdCBpb19tYXBwaW5nIGJ1 dCByZWxpZXMgb24KbWVtcmVtYXAgZnVuY3Rpb25hbGl0eSwgYW5kIHRoYXQgZG9uJ3Qgd2FudCB0 byB1c2Ugc2NhdHRlcmxpc3RzIGZvcgpWUkFNIG1heSB3ZWxsIGRlZmluZSBzcGVjaWFsaXplZCAo aG9wZWZ1bGx5IHJldXNhYmxlKSBpdGVyYXRvcnMgZm9yIHRoZWlyCnBhcnRpY3VsYXIgZW52aXJv bm1lbnQuCgpDYzogQ2hyaXN0aWFuIEvDtm5pZyA8Y2hyaXN0aWFuLmtvZW5pZ0BhbWQuY29tPgpT aWduZWQtb2ZmLWJ5OiBUaG9tYXMgSGVsbHN0csO2bSA8dGhvbWFzLmhlbGxzdHJvbUBsaW51eC5p bnRlbC5jb20+Ci0tLQogZHJpdmVycy9ncHUvZHJtL2k5MTUvTWFrZWZpbGUgICAgICAgICAgICAg ICAgIHwgICAxICsKIC4uLi9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX3R0bV9ib191dGlsLmMg ICB8IDE1NSArKysrKysrKysrKysrKysrKysKIC4uLi9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2Vt X3R0bV9ib191dGlsLmggICB8IDE0MSArKysrKysrKysrKysrKysrCiBkcml2ZXJzL2dwdS9kcm0v dHRtL3R0bV9iby5jICAgICAgICAgICAgICAgICAgfCAgIDEgKwogNCBmaWxlcyBjaGFuZ2VkLCAy OTggaW5zZXJ0aW9ucygrKQogY3JlYXRlIG1vZGUgMTAwNjQ0IGRyaXZlcnMvZ3B1L2RybS9pOTE1 L2dlbS9pOTE1X2dlbV90dG1fYm9fdXRpbC5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgZHJpdmVycy9n cHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX3R0bV9ib191dGlsLmgKCmRpZmYgLS1naXQgYS9kcml2 ZXJzL2dwdS9kcm0vaTkxNS9NYWtlZmlsZSBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L01ha2VmaWxl CmluZGV4IGNiODgyMzU3MDk5Ni4uOTU4Y2NjMWVkZmVkIDEwMDY0NAotLS0gYS9kcml2ZXJzL2dw dS9kcm0vaTkxNS9NYWtlZmlsZQorKysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9NYWtlZmlsZQpA QCAtMTU1LDYgKzE1NSw3IEBAIGdlbS15ICs9IFwKIAlnZW0vaTkxNV9nZW1fc3RvbGVuLm8gXAog CWdlbS9pOTE1X2dlbV90aHJvdHRsZS5vIFwKIAlnZW0vaTkxNV9nZW1fdGlsaW5nLm8gXAorCWdl bS9pOTE1X2dlbV90dG1fYm9fdXRpbC5vIFwKIAlnZW0vaTkxNV9nZW1fdXNlcnB0ci5vIFwKIAln ZW0vaTkxNV9nZW1fd2FpdC5vIFwKIAlnZW0vaTkxNV9nZW1mcy5vCmRpZmYgLS1naXQgYS9kcml2 ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtX2JvX3V0aWwuYyBiL2RyaXZlcnMvZ3B1 L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG1fYm9fdXRpbC5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0 CmluZGV4IDAwMDAwMDAwMDAwMC4uMTExNmQ3ZGYxNDYxCi0tLSAvZGV2L251bGwKKysrIGIvZHJp dmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX3R0bV9ib191dGlsLmMKQEAgLTAsMCArMSwx NTUgQEAKKy8vIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBNSVQKKy8qCisgKiBDb3B5cmlnaHQg wqkgMjAyMSBJbnRlbCBDb3Jwb3JhdGlvbgorICovCisKKy8qKgorICogRE9DOiBVc2FnZSBhbmQg aW50ZW50aW9ucy4KKyAqCisgKiBUaGlzIGZpbGUgY29udGFpbnMgZnVuY3Rpb25hbGl0eSB0aGF0 IHdlIG1pZ2h0IHdhbnQgdG8gbW92ZSBpbnRvCisgKiB0dG1fYm9fdXRpbC5jIGlmIHRoZXJlIGlz IGEgY29tbW9uIGludGVyZXN0LgorICogQ3VycmVudGx5IGEga21hcF9sb2NhbCBvbmx5IG1lbWNw eSB3aXRoIHN1cHBvcnQgZm9yIHBhZ2UtYmFzZWQgaW9tZW0gcmVnaW9ucywKKyAqIGFuZCBmYXN0 IG1lbWNweSBmcm9tIHdyaXRlLWNvbWJpbmVkIG1lbW9yeS4KKyAqLworCisjaW5jbHVkZSA8bGlu dXgvZG1hLWJ1Zi1tYXAuaD4KKyNpbmNsdWRlIDxsaW51eC9oaWdobWVtLmg+CisjaW5jbHVkZSA8 bGludXgvaW8tbWFwcGluZy5oPgorI2luY2x1ZGUgPGxpbnV4L3NjYXR0ZXJsaXN0Lmg+CisKKyNp bmNsdWRlICJpOTE1X21lbWNweS5oIgorCisjaW5jbHVkZSAiZ2VtL2k5MTVfZ2VtX3R0bV9ib191 dGlsLmgiCisKK3N0YXRpYyB2b2lkIGk5MTVfdHRtX2ttYXBfaXRlcl90dF9rbWFwX2xvY2FsKHN0 cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIgKml0ZXIsCisJCQkJCSAgICAgc3RydWN0IGRtYV9idWZf bWFwICpkbWFwLAorCQkJCQkgICAgIHBnb2ZmX3QgaSkKK3sKKwlzdHJ1Y3QgaTkxNV90dG1fa21h cF9pdGVyX3R0ICppdGVyX3R0ID0KKwkJY29udGFpbmVyX29mKGl0ZXIsIHR5cGVvZigqaXRlcl90 dCksIGJhc2UpOworCisJZG1hX2J1Zl9tYXBfc2V0X3ZhZGRyKGRtYXAsIGttYXBfbG9jYWxfcGFn ZShpdGVyX3R0LT50dC0+cGFnZXNbaV0pKTsKK30KKworc3RhdGljIHZvaWQgaTkxNV90dG1fa21h cF9pdGVyX2lvbWFwX2ttYXBfbG9jYWwoc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlciAqaXRlciwK KwkJCQkJCXN0cnVjdCBkbWFfYnVmX21hcCAqZG1hcCwKKwkJCQkJCXBnb2ZmX3QgaSkKK3sKKwlz dHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyX2lvbWFwICppdGVyX2lvID0KKwkJY29udGFpbmVyX29m KGl0ZXIsIHR5cGVvZigqaXRlcl9pbyksIGJhc2UpOworCXZvaWQgX19pb21lbSAqYWRkcjsKKwor cmV0cnk6CisJd2hpbGUgKGkgPj0gaXRlcl9pby0+Y2FjaGUuZW5kKSB7CisJCWl0ZXJfaW8tPmNh Y2hlLnNnID0gaXRlcl9pby0+Y2FjaGUuc2cgPworCQkJc2dfbmV4dChpdGVyX2lvLT5jYWNoZS5z ZykgOiBpdGVyX2lvLT5zdC0+c2dsOworCQlpdGVyX2lvLT5jYWNoZS5pID0gaXRlcl9pby0+Y2Fj aGUuZW5kOworCQlpdGVyX2lvLT5jYWNoZS5lbmQgKz0gc2dfZG1hX2xlbihpdGVyX2lvLT5jYWNo ZS5zZykgPj4KKwkJCVBBR0VfU0hJRlQ7CisJCWl0ZXJfaW8tPmNhY2hlLm9mZnMgPSBzZ19kbWFf YWRkcmVzcyhpdGVyX2lvLT5jYWNoZS5zZykgLQorCQkJaXRlcl9pby0+c3RhcnQ7CisJfQorCisJ aWYgKGkgPCBpdGVyX2lvLT5jYWNoZS5pKSB7CisJCWl0ZXJfaW8tPmNhY2hlLmVuZCA9IDA7CisJ CWl0ZXJfaW8tPmNhY2hlLnNnID0gTlVMTDsKKwkJZ290byByZXRyeTsKKwl9CisKKwlhZGRyID0g aW9fbWFwcGluZ19tYXBfbG9jYWxfd2MoaXRlcl9pby0+aW9tYXAsIGl0ZXJfaW8tPmNhY2hlLm9m ZnMgKworCQkJCSAgICAgICAoKChyZXNvdXJjZV9zaXplX3QpaSAtIGl0ZXJfaW8tPmNhY2hlLmkp CisJCQkJCTw8IFBBR0VfU0hJRlQpKTsKKwlkbWFfYnVmX21hcF9zZXRfdmFkZHJfaW9tZW0oZG1h cCwgYWRkcik7Cit9CisKK3N0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXJfb3BzIGk5MTVfdHRtX2tt YXBfaXRlcl90dF9vcHMgPSB7CisJLmttYXBfbG9jYWwgPSBpOTE1X3R0bV9rbWFwX2l0ZXJfdHRf a21hcF9sb2NhbAorfTsKKworc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlcl9vcHMgaTkxNV90dG1f a21hcF9pdGVyX2lvX29wcyA9IHsKKwkua21hcF9sb2NhbCA9ICBpOTE1X3R0bV9rbWFwX2l0ZXJf aW9tYXBfa21hcF9sb2NhbAorfTsKKworc3RhdGljIHZvaWQga3VubWFwX2xvY2FsX2RtYV9idWZf bWFwKHN0cnVjdCBkbWFfYnVmX21hcCAqbWFwKQoreworCWlmIChtYXAtPmlzX2lvbWVtKQorCQlp b19tYXBwaW5nX3VubWFwX2xvY2FsKG1hcC0+dmFkZHJfaW9tZW0pOworCWVsc2UKKwkJa3VubWFw X2xvY2FsKG1hcC0+dmFkZHIpOworfQorCisvKioKKyAqIGk5MTVfdHRtX21vdmVfbWVtY3B5IC0g SGVscGVyIHRvIHBlcmZvcm0gYSBtZW1jcHkgdHRtIG1vdmUgb3BlcmF0aW9uLgorICogQGJvOiBU aGUgc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0LgorICogQG5ld19tZW06IFRoZSBzdHJ1Y3QgdHRt X3Jlc291cmNlIHdlJ3JlIG1vdmluZyB0byAoY29weSBkZXN0aW5hdGlvbikuCisgKiBAbmV3X2tt YXA6IEEgc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlciByZXByZXNlbnRpbmcgdGhlIGRlc3RpbmF0 aW9uIHJlc291cmNlLgorICogQG9sZF9rbWFwOiBBIHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIg cmVwcmVzZW50aW5nIHRoZSBzb3VyY2UgcmVzb3VyY2UuCisgKi8KK3ZvaWQgaTkxNV90dG1fbW92 ZV9tZW1jcHkoc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywKKwkJCSAgc3RydWN0IHR0bV9y ZXNvdXJjZSAqbmV3X21lbSwKKwkJCSAgc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlciAqbmV3X2tt YXAsCisJCQkgIHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIgKm9sZF9rbWFwKQoreworCXN0cnVj dCB0dG1fZGV2aWNlICpiZGV2ID0gYm8tPmJkZXY7CisJc3RydWN0IHR0bV9yZXNvdXJjZV9tYW5h Z2VyICptYW4gPSB0dG1fbWFuYWdlcl90eXBlKGJkZXYsIG5ld19tZW0tPm1lbV90eXBlKTsKKwlz dHJ1Y3QgdHRtX3R0ICp0dG0gPSBiby0+dHRtOworCXN0cnVjdCB0dG1fcmVzb3VyY2UgKm9sZF9t ZW0gPSAmYm8tPm1lbTsKKwlzdHJ1Y3QgdHRtX3Jlc291cmNlIG9sZF9jb3B5ID0gKm9sZF9tZW07 CisJc3RydWN0IHR0bV9yZXNvdXJjZV9tYW5hZ2VyICpvbGRfbWFuID0gdHRtX21hbmFnZXJfdHlw ZShiZGV2LCBvbGRfbWVtLT5tZW1fdHlwZSk7CisJc3RydWN0IGRtYV9idWZfbWFwIG9sZF9tYXAs IG5ld19tYXA7CisJcGdvZmZfdCBpOworCisJLyogRm9yIHRoZSBwYWdlLWJhc2VkIGFsbG9jYXRv ciB3ZSBuZWVkIHNndGFibGUgaXRlcmF0b3JzIGFzIHdlbGwuKi8KKworCS8qIFNpbmdsZSBUVE0g bW92ZS4gTk9QICovCisJaWYgKG9sZF9tYW4tPnVzZV90dCAmJiBtYW4tPnVzZV90dCkKKwkJZ290 byBkb25lOworCisJLyogRG9uJ3QgbW92ZSBub25leGlzdGVudCBkYXRhLiBDbGVhciBkZXN0aW5h dGlvbiBpbnN0ZWFkLiAqLworCWlmIChvbGRfbWFuLT51c2VfdHQgJiYgIW1hbi0+dXNlX3R0ICYm CisJICAgICh0dG0gPT0gTlVMTCB8fCAhdHRtX3R0X2lzX3BvcHVsYXRlZCh0dG0pKSkgeworCQlp ZiAodHRtICYmICEodHRtLT5wYWdlX2ZsYWdzICYgVFRNX1BBR0VfRkxBR19aRVJPX0FMTE9DKSkK KwkJCWdvdG8gZG9uZTsKKworCQlmb3IgKGkgPSAwOyBpIDwgbmV3X21lbS0+bnVtX3BhZ2VzOyAr K2kpIHsKKwkJCW5ld19rbWFwLT5vcHMtPmttYXBfbG9jYWwobmV3X2ttYXAsICZuZXdfbWFwLCBp KTsKKwkJCW1lbXNldF9pbyhuZXdfbWFwLnZhZGRyX2lvbWVtLCAwLCBQQUdFX1NJWkUpOworCQkJ a3VubWFwX2xvY2FsX2RtYV9idWZfbWFwKCZuZXdfbWFwKTsKKwkJfQorCQlnb3RvIGRvbmU7CisJ fQorCisJZm9yIChpID0gMDsgaSA8IG5ld19tZW0tPm51bV9wYWdlczsgKytpKSB7CisJCW5ld19r bWFwLT5vcHMtPmttYXBfbG9jYWwobmV3X2ttYXAsICZuZXdfbWFwLCBpKTsKKwkJb2xkX2ttYXAt Pm9wcy0+a21hcF9sb2NhbChvbGRfa21hcCwgJm9sZF9tYXAsIGkpOworCQlpZiAoIW9sZF9tYXAu aXNfaW9tZW0gfHwKKwkJICAgICFpOTE1X21lbWNweV9mcm9tX3djKG5ld19tYXAudmFkZHIsIG9s ZF9tYXAudmFkZHIsIFBBR0VfU0laRSkpIHsKKwkJCWlmICghb2xkX21hcC5pc19pb21lbSkgewor CQkJCWRtYV9idWZfbWFwX21lbWNweV90bygmbmV3X21hcCwgb2xkX21hcC52YWRkciwKKwkJCQkJ CSAgICAgIFBBR0VfU0laRSk7CisJCQl9IGVsc2UgaWYgKCFuZXdfbWFwLmlzX2lvbWVtKSB7CisJ CQkJbWVtY3B5X2Zyb21pbyhuZXdfbWFwLnZhZGRyLCBvbGRfbWFwLnZhZGRyX2lvbWVtLAorCQkJ CQkgICAgICBQQUdFX1NJWkUpOworCQkJfSBlbHNlIHsKKwkJCQlwZ29mZl90IGo7CisJCQkJdTMy IF9faW9tZW0gKnNyYyA9IG9sZF9tYXAudmFkZHJfaW9tZW07CisJCQkJdTMyIF9faW9tZW0gKmRz dCA9IG5ld19tYXAudmFkZHJfaW9tZW07CisKKwkJCQlmb3IgKGogPSAwOyBqIDwgKFBBR0VfU0la RSA+PiAyKTsgKytqKQorCQkJCQlpb3dyaXRlMzIoaW9yZWFkMzIoc3JjKyspLCBkc3QrKyk7CisJ CQl9CisJCX0KKwkJa3VubWFwX2xvY2FsX2RtYV9idWZfbWFwKCZvbGRfbWFwKTsKKwkJa3VubWFw X2xvY2FsX2RtYV9idWZfbWFwKCZuZXdfbWFwKTsKKwl9CisKK2RvbmU6CisJb2xkX2NvcHkgPSAq b2xkX21lbTsKKworCXR0bV9ib19hc3NpZ25fbWVtKGJvLCBuZXdfbWVtKTsKKworCWlmICghbWFu LT51c2VfdHQpCisJCXR0bV9ib190dF9kZXN0cm95KGJvKTsKKworCXR0bV9yZXNvdXJjZV9mcmVl KGJvLCAmb2xkX2NvcHkpOworfQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2Vt L2k5MTVfZ2VtX3R0bV9ib191dGlsLmggYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9n ZW1fdHRtX2JvX3V0aWwuaApuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAwMDAu LjgyYzkyMTc2NzE4ZAotLS0gL2Rldi9udWxsCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dl bS9pOTE1X2dlbV90dG1fYm9fdXRpbC5oCkBAIC0wLDAgKzEsMTQxIEBACisvKiBTUERYLUxpY2Vu c2UtSWRlbnRpZmllcjogTUlUICovCisvKgorICogQ29weXJpZ2h0IMKpIDIwMjEgSW50ZWwgQ29y cG9yYXRpb24KKyAqLworCisvKgorICogVGhpcyBmaWxlcyBjb250YWlucyBmdW5jdGlvbmFsaXR5 IHRoYXQgd2UgbWlnaHQgd2FudCB0byBtb3ZlIGludG8KKyAqIHR0bV9ib191dGlsLmMgaWYgdGhl cmUgaXMgYSBjb21tb24gaW50ZXJlc3QuCisgKi8KKyNpZm5kZWYgX0k5MTVfR0VNX1RUTV9CT19V VElMX0hfCisjZGVmaW5lIF9JOTE1X0dFTV9UVE1fQk9fVVRJTF9IXworCisjaW5jbHVkZSA8ZHJt L3R0bS90dG1fYm9fZHJpdmVyLmg+CitzdHJ1Y3QgZG1hX2J1Zl9tYXA7CitzdHJ1Y3QgaW9fbWFw cGluZzsKK3N0cnVjdCBzZ190YWJsZTsKK3N0cnVjdCBzY2F0dGVybGlzdDsKKworc3RydWN0IHR0 bV90dDsKK3N0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXI7CisKKy8qKgorICogc3RydWN0IGk5MTVf dHRtX2ttYXBfaXRlcl9vcHMgLSBPcHMgc3RydWN0dXJlIGZvciBhIHN0cnVjdAorICogaTkxNV90 dG1fa21hcF9pdGVyLgorICovCitzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyX29wcyB7CisJLyoq CisJICoga21hcF9sb2NhbCAtIE1hcCBhIFBBR0VfU0laRSBwYXJ0IG9mIHRoZSByZXNvdXJjZSB1 c2luZworCSAqIGttYXBfbG9jYWwgc2VtYW50aWNzLgorCSAqIEByZXNfa21hcDogUG9pbnRlciB0 byB0aGUgc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlciByZXByZXNlbnRpbmcKKwkgKiB0aGUgcmVz b3VyY2UuCisJICogQGRtYXA6IFRoZSBzdHJ1Y3QgZG1hX2J1Zl9tYXAgaG9sZGluZyB0aGUgdmly dHVhbCBhZGRyZXNzIGFmdGVyCisJICogdGhlIG9wZXJhdGlvbi4KKwkgKiBAaTogVGhlIGxvY2F0 aW9uIHdpdGhpbiB0aGUgcmVzb3VyY2UgdG8gbWFwLiBQQUdFX1NJWkUgZ3JhbnVsYXJpdHkuCisJ ICovCisJdm9pZCAoKmttYXBfbG9jYWwpKHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIgKnJlc19r bWFwLAorCQkJICAgc3RydWN0IGRtYV9idWZfbWFwICpkbWFwLCBwZ29mZl90IGkpOworfTsKKwor LyoqCisgKiBzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyIC0gSXRlcmF0b3IgZm9yIGttYXBfbG9j YWwgdHlwZSBvcGVyYXRpb25zIG9uIGEKKyAqIHJlc291cmNlLgorICogQG9wczogUG9pbnRlciB0 byB0aGUgb3BlcmF0aW9ucyBzdHJ1Y3QuCisgKgorICogVGhpcyBzdHJ1Y3QgaXMgaW50ZW5kZWQg dG8gYmUgZW1iZWRkZWQgaW4gYSByZXNvdXJjZS1zcGVjaWZpYyBzcGVjaWFsaXphdGlvbgorICog aW1wbGVtZW50aW5nIG9wZXJhdGlvbnMgZm9yIHRoZSByZXNvdXJjZS4KKyAqCisgKiBOb3RoaW5n IHN0b3BzIHVzIGZyb20gZXh0ZW5kaW5nIHRoZSBvcGVyYXRpb25zIHRvIHZtYXAsIHZtYXBfcGZu IGV0YywKKyAqIHJlcGxhY2luZyBzb21lIG9yIHBhcnRzIG9mIHRoZSB0dG1fYm9fdXRpbC4gY3B1 LW1hcCBmdW5jdGlvbmFsaXR5LgorICovCitzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyIHsKKwlj b25zdCBzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyX29wcyAqb3BzOworfTsKKworLyoqCisgKiBz dHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyX3R0IC0gU3BlY2lhbGl6YXRpb24gZm9yIGEgdHQgKHBh Z2UpIGJhY2tlZCBzdHJ1Y3QKKyAqIHR0bV9yZXNvdXJjZS4KKyAqIEBiYXNlOiBFbWJlZGRlZCBz dHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyIHByb3ZpZGluZyB0aGUgdXNhZ2UgaW50ZXJmYWNlCisg KiBAdHQ6IENhY2hlZCBzdHJ1Y3QgdHRtX3R0LgorICovCitzdHJ1Y3QgaTkxNV90dG1fa21hcF9p dGVyX3R0IHsKKwlzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyIGJhc2U7CisJc3RydWN0IHR0bV90 dCAqdHQ7Cit9OworCisvKioKKyAqIHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXJfaW9tYXAgLSBT cGVjaWFsaXphdGlvbiBmb3IgYSBzdHJ1Y3QgaW9fbWFwcGluZyArCisgKiBzdHJ1Y3Qgc2dfdGFi bGUgYmFja2VkIHN0cnVjdCB0dG1fcmVzb3VyY2UuCisgKiBAYmFzZTogRW1iZWRkZWQgc3RydWN0 IGk5MTVfdHRtX2ttYXBfaXRlciBwcm92aWRpbmcgdGhlIHVzYWdlIGludGVyZmFjZS4KKyAqIEBp b21hcDogc3RydWN0IGlvX21hcHBpbmcgcmVwcmVzZW50aW5nIHRoZSB1bmRlcmx5aW5nIGxpbmVh ciBpb19tZW1vcnkuCisgKiBAc3Q6IHNnX3RhYmxlIGludG8gQGlvbWFwLCByZXByZXNlbnRpbmcg dGhlIG1lbW9yeSBvZiB0aGUgc3RydWN0IHR0bV9yZXNvdXJjZS4KKyAqIEBzdGFydDogT2Zmc2V0 IHRoYXQgbmVlZHMgdG8gYmUgc3VidHJhY3RlZCBmcm9tIEBzdCB0byBtYWtlCisgKiBzZ19kbWFf YWRkcmVzcyhzdC0+c2dsKSAtIEBzdGFydCA9PSAwIGZvciBAaW9tYXAgc3RhcnQuCisgKiBAY2Fj aGU6IFNjYXR0ZXJsaXN0IHRyYXZlcnNhbCBjYWNoZSBmb3IgZmFzdCBsb29rdXBzLgorICogQGNh Y2hlLnNnOiBQb2ludGVyIHRvIHRoZSBjdXJyZW50bHkgY2FjaGVkIHNjYXR0ZXJsaXN0IHNlZ21l bnQuCisgKiBAY2FjaGUuaTogRmlyc3QgaW5kZXggb2YgQHNnLiBQQUdFX1NJWkUgZ3JhbnVsYXJp dHkuCisgKiBAY2FjaGUuZW5kOiBMYXN0IGluZGV4ICsgMSBvZiBAc2cuIFBBR0VfU0laRSBncmFu dWxhcml0eS4KKyAqIEBjYWNoZS5vZmZzOiBGaXJzdCBvZmZzZXQgaW50byBAaW9tYXAgb2YgQHNn LiBQQUdFX1NJWkUgZ3JhbnVsYXJpdHkuCisgKi8KK3N0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXJf aW9tYXAgeworCXN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIgYmFzZTsKKwlzdHJ1Y3QgaW9fbWFw cGluZyAqaW9tYXA7CisJc3RydWN0IHNnX3RhYmxlICpzdDsKKwlyZXNvdXJjZV9zaXplX3Qgc3Rh cnQ7CisJc3RydWN0IHsKKwkJc3RydWN0IHNjYXR0ZXJsaXN0ICpzZzsKKwkJcGdvZmZfdCBpOwor CQlwZ29mZl90IGVuZDsKKwkJcGdvZmZfdCBvZmZzOworCX0gY2FjaGU7Cit9OworCitleHRlcm4g c3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlcl9vcHMgaTkxNV90dG1fa21hcF9pdGVyX3R0X29wczsK K2V4dGVybiBzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyX29wcyBpOTE1X3R0bV9rbWFwX2l0ZXJf aW9fb3BzOworCisvKioKKyAqIGk5MTVfdHRtX2ttYXBfaXRlcl9pb21hcF9pbml0IC0gSW5pdGlh bGl6ZSBhIHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXJfaW9tYXAKKyAqIEBpdGVyX2lvOiBUaGUg c3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlcl9pb21hcCB0byBpbml0aWFsaXplLgorICogQGlvbWFw OiBUaGUgc3RydWN0IGlvX21hcHBpbmcgcmVwcmVzZW50aW5nIHRoZSB1bmRlcmx5aW5nIGxpbmVh ciBpb19tZW1vcnkuCisgKiBAc3Q6IHNnX3RhYmxlIGludG8gQGlvbWFwLCByZXByZXNlbnRpbmcg dGhlIG1lbW9yeSBvZiB0aGUgc3RydWN0CisgKiB0dG1fcmVzb3VyY2UuCisgKiBAc3RhcnQ6IE9m ZnNldCB0aGF0IG5lZWRzIHRvIGJlIHN1YnRyYWN0ZWQgZnJvbSBAc3QgdG8gbWFrZQorICogc2df ZG1hX2FkZHJlc3Moc3QtPnNnbCkgLSBAc3RhcnQgPT0gMCBmb3IgQGlvbWFwIHN0YXJ0LgorICoK KyAqIFJldHVybjogUG9pbnRlciB0byB0aGUgZW1iZWRkZWQgc3RydWN0IGk5MTVfdHRtX2ttYXBf aXRlci4KKyAqLworc3RhdGljIGlubGluZSBzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyICoKK2k5 MTVfdHRtX2ttYXBfaXRlcl9pb21hcF9pbml0KHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXJfaW9t YXAgKml0ZXJfaW8sCisJCQkgICAgICBzdHJ1Y3QgaW9fbWFwcGluZyAqaW9tYXAsCisJCQkgICAg ICBzdHJ1Y3Qgc2dfdGFibGUgKnN0LAorCQkJICAgICAgcmVzb3VyY2Vfc2l6ZV90IHN0YXJ0KQor eworCWl0ZXJfaW8tPmJhc2Uub3BzID0gJmk5MTVfdHRtX2ttYXBfaXRlcl9pb19vcHM7CisJaXRl cl9pby0+aW9tYXAgPSBpb21hcDsKKwlpdGVyX2lvLT5zdCA9IHN0OworCWl0ZXJfaW8tPnN0YXJ0 ID0gc3RhcnQ7CisJbWVtc2V0KCZpdGVyX2lvLT5jYWNoZSwgMCwgc2l6ZW9mKGl0ZXJfaW8tPmNh Y2hlKSk7CisJcmV0dXJuICZpdGVyX2lvLT5iYXNlOworfQorCisvKioKKyAqIHR0bV9rbWFwX2l0 ZXJfdHRfaW5pdCAtIEluaXRpYWxpemUgYSBzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyX3R0Cisg KiBAaXRlcl90dDogVGhlIHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXJfdHQgdG8gaW5pdGlhbGl6 ZS4KKyAqIEB0dDogU3RydWN0IHR0bV90dCBob2xkaW5nIHBhZ2UgcG9pbnRlcnMgb2YgdGhlIHN0 cnVjdCB0dG1fcmVzb3VyY2UuCisgKgorICogUmV0dXJuOiBQb2ludGVyIHRvIHRoZSBlbWJlZGRl ZCBzdHJ1Y3QgaTkxNV90dG1fa21hcF9pdGVyLgorICovCitzdGF0aWMgaW5saW5lIHN0cnVjdCBp OTE1X3R0bV9rbWFwX2l0ZXIgKgoraTkxNV90dG1fa21hcF9pdGVyX3R0X2luaXQoc3RydWN0IGk5 MTVfdHRtX2ttYXBfaXRlcl90dCAqaXRlcl90dCwKKwkJCSAgIHN0cnVjdCB0dG1fdHQgKnR0KQor eworCWl0ZXJfdHQtPmJhc2Uub3BzID0gJmk5MTVfdHRtX2ttYXBfaXRlcl90dF9vcHM7CisJaXRl cl90dC0+dHQgPSB0dDsKKwlyZXR1cm4gJml0ZXJfdHQtPmJhc2U7Cit9CisKK3ZvaWQgaTkxNV90 dG1fbW92ZV9tZW1jcHkoc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywKKwkJCSAgc3RydWN0 IHR0bV9yZXNvdXJjZSAqbmV3X21lbSwKKwkJCSAgc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlciAq bmV3X2l0ZXIsCisJCQkgIHN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIgKm9sZF9pdGVyKTsKKyNl bmRpZgpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL3R0bS90dG1fYm8uYyBiL2RyaXZlcnMv Z3B1L2RybS90dG0vdHRtX2JvLmMKaW5kZXggY2ExYjA5OGI2YTU2Li40NDc5YzU1YWFhMWQgMTAw NjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS90dG0vdHRtX2JvLmMKKysrIGIvZHJpdmVycy9ncHUv ZHJtL3R0bS90dG1fYm8uYwpAQCAtMTIyMSwzICsxMjIxLDQgQEAgdm9pZCB0dG1fYm9fdHRfZGVz dHJveShzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvKQogCXR0bV90dF9kZXN0cm95KGJvLT5i ZGV2LCBiby0+dHRtKTsKIAliby0+dHRtID0gTlVMTDsKIH0KK0VYUE9SVF9TWU1CT0wodHRtX2Jv X3R0X2Rlc3Ryb3kpOwotLSAKMi4zMC4yCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fXwpJbnRlbC1nZnggbWFpbGluZyBsaXN0CkludGVsLWdmeEBsaXN0cy5m cmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0 aW5mby9pbnRlbC1nZngK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C48B7C43470 for ; Tue, 11 May 2021 13:26:22 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7314C6191C for ; Tue, 11 May 2021 13:26:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7314C6191C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 051496EA3B; Tue, 11 May 2021 13:26:18 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1FD496EA3D; Tue, 11 May 2021 13:26:16 +0000 (UTC) IronPort-SDR: V8TH/naUq8SSqd6Wfpvb/YizZiijXC+dGLPUXAGm2cgESABK+d8FHHvU25meOXa3KdITdrIscV 3agVenfU9uyQ== X-IronPort-AV: E=McAfee;i="6200,9189,9980"; a="196345411" X-IronPort-AV: E=Sophos;i="5.82,290,1613462400"; d="scan'208";a="196345411" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2021 06:26:16 -0700 IronPort-SDR: kIC6BdFzd2UKWk4SBmWo7ZBuvyiLPWOLATdPmkj/dVATdbcQ+walh6Rg9dYq8yn0gcP7qKEoIa jP91SjsgjGhg== X-IronPort-AV: E=Sophos;i="5.82,290,1613462400"; d="scan'208";a="468905434" Received: from kjeldbeg-mobl2.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.168]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 May 2021 06:26:14 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 5/7] drm/i915/ttm, drm/ttm: Add a generic TTM memcpy move for page-based iomem Date: Tue, 11 May 2021 15:25:23 +0200 Message-Id: <20210511132525.377190-6-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210511132525.377190-1-thomas.hellstrom@linux.intel.com> References: <20210511132525.377190-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , =?UTF-8?q?Christian=20K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The internal ttm_bo_util memcpy uses vmap functionality, and while it probably might be possible to use it for copying in- and out of sglist represented io memory, using io_mem_reserve() / io_mem_free() callbacks, that would cause problems with fault(). Instead, implement a method mapping page-by-page using kmap_local() semantics. As an additional benefit we then avoid the occasional global TLB flushes of vmap() and consuming vmap space, elimination of a critical point of failure and with a slight change of semantics we could also push the memcpy out async for testing and async driver develpment purposes. Pushing out async can be done since there is no memory allocation going on that could violate the dma_fence lockdep rules. Note that drivers that don't want to use struct io_mapping but relies on memremap functionality, and that don't want to use scatterlists for VRAM may well define specialized (hopefully reusable) iterators for their particular environment. Cc: Christian König Signed-off-by: Thomas Hellström --- drivers/gpu/drm/i915/Makefile | 1 + .../gpu/drm/i915/gem/i915_gem_ttm_bo_util.c | 155 ++++++++++++++++++ .../gpu/drm/i915/gem/i915_gem_ttm_bo_util.h | 141 ++++++++++++++++ drivers/gpu/drm/ttm/ttm_bo.c | 1 + 4 files changed, 298 insertions(+) create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.c create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.h diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile index cb8823570996..958ccc1edfed 100644 --- a/drivers/gpu/drm/i915/Makefile +++ b/drivers/gpu/drm/i915/Makefile @@ -155,6 +155,7 @@ gem-y += \ gem/i915_gem_stolen.o \ gem/i915_gem_throttle.o \ gem/i915_gem_tiling.o \ + gem/i915_gem_ttm_bo_util.o \ gem/i915_gem_userptr.o \ gem/i915_gem_wait.o \ gem/i915_gemfs.o diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.c new file mode 100644 index 000000000000..1116d7df1461 --- /dev/null +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.c @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2021 Intel Corporation + */ + +/** + * DOC: Usage and intentions. + * + * This file contains functionality that we might want to move into + * ttm_bo_util.c if there is a common interest. + * Currently a kmap_local only memcpy with support for page-based iomem regions, + * and fast memcpy from write-combined memory. + */ + +#include +#include +#include +#include + +#include "i915_memcpy.h" + +#include "gem/i915_gem_ttm_bo_util.h" + +static void i915_ttm_kmap_iter_tt_kmap_local(struct i915_ttm_kmap_iter *iter, + struct dma_buf_map *dmap, + pgoff_t i) +{ + struct i915_ttm_kmap_iter_tt *iter_tt = + container_of(iter, typeof(*iter_tt), base); + + dma_buf_map_set_vaddr(dmap, kmap_local_page(iter_tt->tt->pages[i])); +} + +static void i915_ttm_kmap_iter_iomap_kmap_local(struct i915_ttm_kmap_iter *iter, + struct dma_buf_map *dmap, + pgoff_t i) +{ + struct i915_ttm_kmap_iter_iomap *iter_io = + container_of(iter, typeof(*iter_io), base); + void __iomem *addr; + +retry: + while (i >= iter_io->cache.end) { + iter_io->cache.sg = iter_io->cache.sg ? + sg_next(iter_io->cache.sg) : iter_io->st->sgl; + iter_io->cache.i = iter_io->cache.end; + iter_io->cache.end += sg_dma_len(iter_io->cache.sg) >> + PAGE_SHIFT; + iter_io->cache.offs = sg_dma_address(iter_io->cache.sg) - + iter_io->start; + } + + if (i < iter_io->cache.i) { + iter_io->cache.end = 0; + iter_io->cache.sg = NULL; + goto retry; + } + + addr = io_mapping_map_local_wc(iter_io->iomap, iter_io->cache.offs + + (((resource_size_t)i - iter_io->cache.i) + << PAGE_SHIFT)); + dma_buf_map_set_vaddr_iomem(dmap, addr); +} + +struct i915_ttm_kmap_iter_ops i915_ttm_kmap_iter_tt_ops = { + .kmap_local = i915_ttm_kmap_iter_tt_kmap_local +}; + +struct i915_ttm_kmap_iter_ops i915_ttm_kmap_iter_io_ops = { + .kmap_local = i915_ttm_kmap_iter_iomap_kmap_local +}; + +static void kunmap_local_dma_buf_map(struct dma_buf_map *map) +{ + if (map->is_iomem) + io_mapping_unmap_local(map->vaddr_iomem); + else + kunmap_local(map->vaddr); +} + +/** + * i915_ttm_move_memcpy - Helper to perform a memcpy ttm move operation. + * @bo: The struct ttm_buffer_object. + * @new_mem: The struct ttm_resource we're moving to (copy destination). + * @new_kmap: A struct i915_ttm_kmap_iter representing the destination resource. + * @old_kmap: A struct i915_ttm_kmap_iter representing the source resource. + */ +void i915_ttm_move_memcpy(struct ttm_buffer_object *bo, + struct ttm_resource *new_mem, + struct i915_ttm_kmap_iter *new_kmap, + struct i915_ttm_kmap_iter *old_kmap) +{ + struct ttm_device *bdev = bo->bdev; + struct ttm_resource_manager *man = ttm_manager_type(bdev, new_mem->mem_type); + struct ttm_tt *ttm = bo->ttm; + struct ttm_resource *old_mem = &bo->mem; + struct ttm_resource old_copy = *old_mem; + struct ttm_resource_manager *old_man = ttm_manager_type(bdev, old_mem->mem_type); + struct dma_buf_map old_map, new_map; + pgoff_t i; + + /* For the page-based allocator we need sgtable iterators as well.*/ + + /* Single TTM move. NOP */ + if (old_man->use_tt && man->use_tt) + goto done; + + /* Don't move nonexistent data. Clear destination instead. */ + if (old_man->use_tt && !man->use_tt && + (ttm == NULL || !ttm_tt_is_populated(ttm))) { + if (ttm && !(ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC)) + goto done; + + for (i = 0; i < new_mem->num_pages; ++i) { + new_kmap->ops->kmap_local(new_kmap, &new_map, i); + memset_io(new_map.vaddr_iomem, 0, PAGE_SIZE); + kunmap_local_dma_buf_map(&new_map); + } + goto done; + } + + for (i = 0; i < new_mem->num_pages; ++i) { + new_kmap->ops->kmap_local(new_kmap, &new_map, i); + old_kmap->ops->kmap_local(old_kmap, &old_map, i); + if (!old_map.is_iomem || + !i915_memcpy_from_wc(new_map.vaddr, old_map.vaddr, PAGE_SIZE)) { + if (!old_map.is_iomem) { + dma_buf_map_memcpy_to(&new_map, old_map.vaddr, + PAGE_SIZE); + } else if (!new_map.is_iomem) { + memcpy_fromio(new_map.vaddr, old_map.vaddr_iomem, + PAGE_SIZE); + } else { + pgoff_t j; + u32 __iomem *src = old_map.vaddr_iomem; + u32 __iomem *dst = new_map.vaddr_iomem; + + for (j = 0; j < (PAGE_SIZE >> 2); ++j) + iowrite32(ioread32(src++), dst++); + } + } + kunmap_local_dma_buf_map(&old_map); + kunmap_local_dma_buf_map(&new_map); + } + +done: + old_copy = *old_mem; + + ttm_bo_assign_mem(bo, new_mem); + + if (!man->use_tt) + ttm_bo_tt_destroy(bo); + + ttm_resource_free(bo, &old_copy); +} diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.h b/drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.h new file mode 100644 index 000000000000..82c92176718d --- /dev/null +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_bo_util.h @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2021 Intel Corporation + */ + +/* + * This files contains functionality that we might want to move into + * ttm_bo_util.c if there is a common interest. + */ +#ifndef _I915_GEM_TTM_BO_UTIL_H_ +#define _I915_GEM_TTM_BO_UTIL_H_ + +#include +struct dma_buf_map; +struct io_mapping; +struct sg_table; +struct scatterlist; + +struct ttm_tt; +struct i915_ttm_kmap_iter; + +/** + * struct i915_ttm_kmap_iter_ops - Ops structure for a struct + * i915_ttm_kmap_iter. + */ +struct i915_ttm_kmap_iter_ops { + /** + * kmap_local - Map a PAGE_SIZE part of the resource using + * kmap_local semantics. + * @res_kmap: Pointer to the struct i915_ttm_kmap_iter representing + * the resource. + * @dmap: The struct dma_buf_map holding the virtual address after + * the operation. + * @i: The location within the resource to map. PAGE_SIZE granularity. + */ + void (*kmap_local)(struct i915_ttm_kmap_iter *res_kmap, + struct dma_buf_map *dmap, pgoff_t i); +}; + +/** + * struct i915_ttm_kmap_iter - Iterator for kmap_local type operations on a + * resource. + * @ops: Pointer to the operations struct. + * + * This struct is intended to be embedded in a resource-specific specialization + * implementing operations for the resource. + * + * Nothing stops us from extending the operations to vmap, vmap_pfn etc, + * replacing some or parts of the ttm_bo_util. cpu-map functionality. + */ +struct i915_ttm_kmap_iter { + const struct i915_ttm_kmap_iter_ops *ops; +}; + +/** + * struct i915_ttm_kmap_iter_tt - Specialization for a tt (page) backed struct + * ttm_resource. + * @base: Embedded struct i915_ttm_kmap_iter providing the usage interface + * @tt: Cached struct ttm_tt. + */ +struct i915_ttm_kmap_iter_tt { + struct i915_ttm_kmap_iter base; + struct ttm_tt *tt; +}; + +/** + * struct i915_ttm_kmap_iter_iomap - Specialization for a struct io_mapping + + * struct sg_table backed struct ttm_resource. + * @base: Embedded struct i915_ttm_kmap_iter providing the usage interface. + * @iomap: struct io_mapping representing the underlying linear io_memory. + * @st: sg_table into @iomap, representing the memory of the struct ttm_resource. + * @start: Offset that needs to be subtracted from @st to make + * sg_dma_address(st->sgl) - @start == 0 for @iomap start. + * @cache: Scatterlist traversal cache for fast lookups. + * @cache.sg: Pointer to the currently cached scatterlist segment. + * @cache.i: First index of @sg. PAGE_SIZE granularity. + * @cache.end: Last index + 1 of @sg. PAGE_SIZE granularity. + * @cache.offs: First offset into @iomap of @sg. PAGE_SIZE granularity. + */ +struct i915_ttm_kmap_iter_iomap { + struct i915_ttm_kmap_iter base; + struct io_mapping *iomap; + struct sg_table *st; + resource_size_t start; + struct { + struct scatterlist *sg; + pgoff_t i; + pgoff_t end; + pgoff_t offs; + } cache; +}; + +extern struct i915_ttm_kmap_iter_ops i915_ttm_kmap_iter_tt_ops; +extern struct i915_ttm_kmap_iter_ops i915_ttm_kmap_iter_io_ops; + +/** + * i915_ttm_kmap_iter_iomap_init - Initialize a struct i915_ttm_kmap_iter_iomap + * @iter_io: The struct i915_ttm_kmap_iter_iomap to initialize. + * @iomap: The struct io_mapping representing the underlying linear io_memory. + * @st: sg_table into @iomap, representing the memory of the struct + * ttm_resource. + * @start: Offset that needs to be subtracted from @st to make + * sg_dma_address(st->sgl) - @start == 0 for @iomap start. + * + * Return: Pointer to the embedded struct i915_ttm_kmap_iter. + */ +static inline struct i915_ttm_kmap_iter * +i915_ttm_kmap_iter_iomap_init(struct i915_ttm_kmap_iter_iomap *iter_io, + struct io_mapping *iomap, + struct sg_table *st, + resource_size_t start) +{ + iter_io->base.ops = &i915_ttm_kmap_iter_io_ops; + iter_io->iomap = iomap; + iter_io->st = st; + iter_io->start = start; + memset(&iter_io->cache, 0, sizeof(iter_io->cache)); + return &iter_io->base; +} + +/** + * ttm_kmap_iter_tt_init - Initialize a struct i915_ttm_kmap_iter_tt + * @iter_tt: The struct i915_ttm_kmap_iter_tt to initialize. + * @tt: Struct ttm_tt holding page pointers of the struct ttm_resource. + * + * Return: Pointer to the embedded struct i915_ttm_kmap_iter. + */ +static inline struct i915_ttm_kmap_iter * +i915_ttm_kmap_iter_tt_init(struct i915_ttm_kmap_iter_tt *iter_tt, + struct ttm_tt *tt) +{ + iter_tt->base.ops = &i915_ttm_kmap_iter_tt_ops; + iter_tt->tt = tt; + return &iter_tt->base; +} + +void i915_ttm_move_memcpy(struct ttm_buffer_object *bo, + struct ttm_resource *new_mem, + struct i915_ttm_kmap_iter *new_iter, + struct i915_ttm_kmap_iter *old_iter); +#endif diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index ca1b098b6a56..4479c55aaa1d 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1221,3 +1221,4 @@ void ttm_bo_tt_destroy(struct ttm_buffer_object *bo) ttm_tt_destroy(bo->bdev, bo->ttm); bo->ttm = NULL; } +EXPORT_SYMBOL(ttm_bo_tt_destroy); -- 2.30.2