From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ED2AC49361 for ; Thu, 17 Jun 2021 06:31:02 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 30921613E7 for ; Thu, 17 Jun 2021 06:31:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 30921613E7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 336A56E885; Thu, 17 Jun 2021 06:30:54 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 688776E87E; Thu, 17 Jun 2021 06:30:52 +0000 (UTC) IronPort-SDR: oJBfYBAO+V0aSbWwNivWXjO/o5H5BgWOSpAnNTErnXPEwX7M7mdMnJp9x3nf7/rFGqDLtitV0a zzy2zatkhc2Q== X-IronPort-AV: E=McAfee;i="6200,9189,10017"; a="203287852" X-IronPort-AV: E=Sophos;i="5.83,278,1616482800"; d="scan'208";a="203287852" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2021 23:30:52 -0700 IronPort-SDR: T3S2SxYpMi4hJAjTh4MQb2+dXSMnWN3VfRRmdvQMPd2zWXigJ9Or7H7AdLcSJ8h9/TIjkkrq4N CcVuGZmKnSbA== X-IronPort-AV: E=Sophos;i="5.83,278,1616482800"; d="scan'208";a="554302750" Received: from vanderss-mobl.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.193]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2021 23:30:50 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Thu, 17 Jun 2021 08:30:16 +0200 Message-Id: <20210617063018.92802-11-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210617063018.92802-1-thomas.hellstrom@linux.intel.com> References: <20210617063018.92802-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v5 10/12] drm/i915/ttm: accelerated move implementation X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.auld@intel.com, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" RnJvbTogUmFtYWxpbmdhbSBDIDxyYW1hbGluZ2FtLmNAaW50ZWwuY29tPgoKSW52b2tlcyB0aGUg cGlwZWxpbmVkIHBhZ2UgbWlncmF0aW9uIHRocm91Z2ggYmx0LCBmb3IKaTkxNV90dG1fbW92ZSBy ZXF1ZXN0cyBvZiBldmljdGlvbiBhbmQgYWxzbyBvYmogY2xlYXIuCgpTaWduZWQtb2ZmLWJ5OiBS YW1hbGluZ2FtIEMgPHJhbWFsaW5nYW0uY0BpbnRlbC5jb20+ClJldmlld2VkLWJ5OiBUaG9tYXMg SGVsbHN0csO2bSA8dGhvbWFzLmhlbGxzdHJvbUBsaW51eC5pbnRlbC5jb20+Ci0tLQp2MjoKIC0g c3ViZnVuY3Rpb24gZm9yIGFjY2VsX21vdmUgKFRob21hcykKIC0gZW5naW5lX3BtX2dldC9wdXQg YXJvdW5kIGNvbnRleHRfbW92ZS9jbGVhciAoVGhvbWFzKQogLSBJbnZhbGlkYXRpb24gYXQgYWNj ZWxfY2xlYXIgKFRob21hcykKdjM6CiAtIGNvbmZsaWN0IHJlc29sdXRpb24gcy8mYm8tPm1lbS9i by0+cmVzb3VyY2UvZwp2NDoKIC0gVGltZW91dCBpcyBzZXQgZm9yIE1BWF9TQ0hFRFVMRV9USU1F T1VUIChUaG9tYXMpCiAtIHMvVFRNX1BMX1BSSVYvSTkxNV9QTF9MTUVNMCAoVGhvbWFzKQotLS0K IGRyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG0uYyB8IDg4ICsrKysrKysrKysr KysrKysrKysrKy0tLS0KIDEgZmlsZSBjaGFuZ2VkLCA3NSBpbnNlcnRpb25zKCspLCAxMyBkZWxl dGlvbnMoLSkKCmRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1f dHRtLmMgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtLmMKaW5kZXggOTM2 NmIxOGQxYmM2Li5kZjQ2NTM1Y2NhNDcgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1 L2dlbS9pOTE1X2dlbV90dG0uYworKysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9n ZW1fdHRtLmMKQEAgLTE1LDYgKzE1LDkgQEAKICNpbmNsdWRlICJnZW0vaTkxNV9nZW1fdHRtLmgi CiAjaW5jbHVkZSAiZ2VtL2k5MTVfZ2VtX21tYW4uaCIKIAorI2luY2x1ZGUgImd0L2ludGVsX21p Z3JhdGUuaCIKKyNpbmNsdWRlICJndC9pbnRlbF9lbmdpbmVfcG0uaCIKKwogI2RlZmluZSBJOTE1 X1BMX0xNRU0wIFRUTV9QTF9QUklWCiAjZGVmaW5lIEk5MTVfUExfU1lTVEVNIFRUTV9QTF9TWVNU RU0KICNkZWZpbmUgSTkxNV9QTF9TVE9MRU4gVFRNX1BMX1ZSQU0KQEAgLTMyNiw2ICszMjksNjIg QEAgaTkxNV90dG1fcmVzb3VyY2VfZ2V0X3N0KHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpv YmosCiAJcmV0dXJuIGludGVsX3JlZ2lvbl90dG1fcmVzb3VyY2VfdG9fc3Qob2JqLT5tbS5yZWdp b24sIHJlcyk7CiB9CiAKK3N0YXRpYyBpbnQgaTkxNV90dG1fYWNjZWxfbW92ZShzdHJ1Y3QgdHRt X2J1ZmZlcl9vYmplY3QgKmJvLAorCQkJICAgICAgIHN0cnVjdCB0dG1fcmVzb3VyY2UgKmRzdF9t ZW0sCisJCQkgICAgICAgc3RydWN0IHNnX3RhYmxlICpkc3Rfc3QpCit7CisJc3RydWN0IGRybV9p OTE1X3ByaXZhdGUgKmk5MTUgPSBjb250YWluZXJfb2YoYm8tPmJkZXYsIHR5cGVvZigqaTkxNSks CisJCQkJCQkgICAgIGJkZXYpOworCXN0cnVjdCB0dG1fcmVzb3VyY2VfbWFuYWdlciAqc3JjX21h biA9CisJCXR0bV9tYW5hZ2VyX3R5cGUoYm8tPmJkZXYsIGJvLT5yZXNvdXJjZS0+bWVtX3R5cGUp OworCXN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmogPSBpOTE1X3R0bV90b19nZW0oYm8p OworCXN0cnVjdCBzZ190YWJsZSAqc3JjX3N0OworCXN0cnVjdCBpOTE1X3JlcXVlc3QgKnJxOwor CWludCByZXQ7CisKKwlpZiAoIWk5MTUtPmd0Lm1pZ3JhdGUuY29udGV4dCkKKwkJcmV0dXJuIC1F SU5WQUw7CisKKwlpZiAoIWJvLT50dG0gfHwgIXR0bV90dF9pc19wb3B1bGF0ZWQoYm8tPnR0bSkp IHsKKwkJaWYgKGJvLT50eXBlID09IHR0bV9ib190eXBlX2tlcm5lbCkKKwkJCXJldHVybiAtRUlO VkFMOworCisJCWlmIChiby0+dHRtICYmCisJCSAgICAhKGJvLT50dG0tPnBhZ2VfZmxhZ3MgJiBU VE1fUEFHRV9GTEFHX1pFUk9fQUxMT0MpKQorCQkJcmV0dXJuIDA7CisKKwkJaW50ZWxfZW5naW5l X3BtX2dldChpOTE1LT5ndC5taWdyYXRlLmNvbnRleHQtPmVuZ2luZSk7CisJCXJldCA9IGludGVs X2NvbnRleHRfbWlncmF0ZV9jbGVhcihpOTE1LT5ndC5taWdyYXRlLmNvbnRleHQsIE5VTEwsCisJ CQkJCQkgIGRzdF9zdC0+c2dsLCBJOTE1X0NBQ0hFX05PTkUsCisJCQkJCQkgIGRzdF9tZW0tPm1l bV90eXBlID49IEk5MTVfUExfTE1FTTAsCisJCQkJCQkgIDAsICZycSk7CisKKwkJaWYgKCFyZXQg JiYgcnEpIHsKKwkJCWk5MTVfcmVxdWVzdF93YWl0KHJxLCAwLCBNQVhfU0NIRURVTEVfVElNRU9V VCk7CisJCQlpOTE1X3JlcXVlc3RfcHV0KHJxKTsKKwkJfQorCQlpbnRlbF9lbmdpbmVfcG1fcHV0 KGk5MTUtPmd0Lm1pZ3JhdGUuY29udGV4dC0+ZW5naW5lKTsKKwl9IGVsc2UgeworCQlzcmNfc3Qg PSBzcmNfbWFuLT51c2VfdHQgPyBpOTE1X3R0bV90dF9nZXRfc3QoYm8tPnR0bSkgOgorCQkJCQkJ b2JqLT50dG0uY2FjaGVkX2lvX3N0OworCisJCWludGVsX2VuZ2luZV9wbV9nZXQoaTkxNS0+Z3Qu bWlncmF0ZS5jb250ZXh0LT5lbmdpbmUpOworCQlyZXQgPSBpbnRlbF9jb250ZXh0X21pZ3JhdGVf Y29weShpOTE1LT5ndC5taWdyYXRlLmNvbnRleHQsCisJCQkJCQkgTlVMTCwgc3JjX3N0LT5zZ2ws IEk5MTVfQ0FDSEVfTk9ORSwKKwkJCQkJCSBiby0+cmVzb3VyY2UtPm1lbV90eXBlID49IEk5MTVf UExfTE1FTTAsCisJCQkJCQkgZHN0X3N0LT5zZ2wsIEk5MTVfQ0FDSEVfTk9ORSwKKwkJCQkJCSBk c3RfbWVtLT5tZW1fdHlwZSA+PSBJOTE1X1BMX0xNRU0wLAorCQkJCQkJICZycSk7CisJCWlmICgh cmV0ICYmIHJxKSB7CisJCQlpOTE1X3JlcXVlc3Rfd2FpdChycSwgMCwgTUFYX1NDSEVEVUxFX1RJ TUVPVVQpOworCQkJaTkxNV9yZXF1ZXN0X3B1dChycSk7CisJCX0KKwkJaW50ZWxfZW5naW5lX3Bt X3B1dChpOTE1LT5ndC5taWdyYXRlLmNvbnRleHQtPmVuZ2luZSk7CisJfQorCisJcmV0dXJuIHJl dDsKK30KKwogc3RhdGljIGludCBpOTE1X3R0bV9tb3ZlKHN0cnVjdCB0dG1fYnVmZmVyX29iamVj dCAqYm8sIGJvb2wgZXZpY3QsCiAJCQkgc3RydWN0IHR0bV9vcGVyYXRpb25fY3R4ICpjdHgsCiAJ CQkgc3RydWN0IHR0bV9yZXNvdXJjZSAqZHN0X21lbSwKQEAgLTM3NiwxOSArNDM1LDIyIEBAIHN0 YXRpYyBpbnQgaTkxNV90dG1fbW92ZShzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvLCBib29s IGV2aWN0LAogCWlmIChJU19FUlIoZHN0X3N0KSkKIAkJcmV0dXJuIFBUUl9FUlIoZHN0X3N0KTsK IAotCS8qIElmIHdlIHN0YXJ0IG1hcHBpbmcgR0dUVCwgd2UgY2FuIG5vIGxvbmdlciB1c2UgbWFu Ojp1c2VfdHQgaGVyZS4gKi8KLQlkc3RfaXRlciA9IGRzdF9tYW4tPnVzZV90dCA/Ci0JCXR0bV9r bWFwX2l0ZXJfdHRfaW5pdCgmX2RzdF9pdGVyLnR0LCBiby0+dHRtKSA6Ci0JCXR0bV9rbWFwX2l0 ZXJfaW9tYXBfaW5pdCgmX2RzdF9pdGVyLmlvLCAmZHN0X3JlZy0+aW9tYXAsCi0JCQkJCSBkc3Rf c3QsIGRzdF9yZWctPnJlZ2lvbi5zdGFydCk7Ci0KLQlzcmNfaXRlciA9IHNyY19tYW4tPnVzZV90 dCA/Ci0JCXR0bV9rbWFwX2l0ZXJfdHRfaW5pdCgmX3NyY19pdGVyLnR0LCBiby0+dHRtKSA6Ci0J CXR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCgmX3NyY19pdGVyLmlvLCAmc3JjX3JlZy0+aW9tYXAs Ci0JCQkJCSBvYmotPnR0bS5jYWNoZWRfaW9fc3QsCi0JCQkJCSBzcmNfcmVnLT5yZWdpb24uc3Rh cnQpOwotCi0JdHRtX21vdmVfbWVtY3B5KGJvLCBkc3RfbWVtLT5udW1fcGFnZXMsIGRzdF9pdGVy LCBzcmNfaXRlcik7CisJcmV0ID0gaTkxNV90dG1fYWNjZWxfbW92ZShibywgZHN0X21lbSwgZHN0 X3N0KTsKKwlpZiAocmV0KSB7CisJCS8qIElmIHdlIHN0YXJ0IG1hcHBpbmcgR0dUVCwgd2UgY2Fu IG5vIGxvbmdlciB1c2UgbWFuOjp1c2VfdHQgaGVyZS4gKi8KKwkJZHN0X2l0ZXIgPSBkc3RfbWFu LT51c2VfdHQgPworCQkJdHRtX2ttYXBfaXRlcl90dF9pbml0KCZfZHN0X2l0ZXIudHQsIGJvLT50 dG0pIDoKKwkJCXR0bV9rbWFwX2l0ZXJfaW9tYXBfaW5pdCgmX2RzdF9pdGVyLmlvLCAmZHN0X3Jl Zy0+aW9tYXAsCisJCQkJCQkgZHN0X3N0LCBkc3RfcmVnLT5yZWdpb24uc3RhcnQpOworCisJCXNy Y19pdGVyID0gc3JjX21hbi0+dXNlX3R0ID8KKwkJCXR0bV9rbWFwX2l0ZXJfdHRfaW5pdCgmX3Ny Y19pdGVyLnR0LCBiby0+dHRtKSA6CisJCQl0dG1fa21hcF9pdGVyX2lvbWFwX2luaXQoJl9zcmNf aXRlci5pbywgJnNyY19yZWctPmlvbWFwLAorCQkJCQkJIG9iai0+dHRtLmNhY2hlZF9pb19zdCwK KwkJCQkJCSBzcmNfcmVnLT5yZWdpb24uc3RhcnQpOworCisJCXR0bV9tb3ZlX21lbWNweShibywg ZHN0X21lbS0+bnVtX3BhZ2VzLCBkc3RfaXRlciwgc3JjX2l0ZXIpOworCX0KIAl0dG1fYm9fbW92 ZV9zeW5jX2NsZWFudXAoYm8sIGRzdF9tZW0pOwogCWk5MTVfdHRtX2ZyZWVfY2FjaGVkX2lvX3N0 KG9iaik7CiAKLS0gCjIuMzEuMQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRl c2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8v aW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C676AC48BE5 for ; Thu, 17 Jun 2021 06:31:11 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 93920613E9 for ; Thu, 17 Jun 2021 06:31:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93920613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EB95B6E888; Thu, 17 Jun 2021 06:31:03 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 688776E87E; Thu, 17 Jun 2021 06:30:52 +0000 (UTC) IronPort-SDR: oJBfYBAO+V0aSbWwNivWXjO/o5H5BgWOSpAnNTErnXPEwX7M7mdMnJp9x3nf7/rFGqDLtitV0a zzy2zatkhc2Q== X-IronPort-AV: E=McAfee;i="6200,9189,10017"; a="203287852" X-IronPort-AV: E=Sophos;i="5.83,278,1616482800"; d="scan'208";a="203287852" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2021 23:30:52 -0700 IronPort-SDR: T3S2SxYpMi4hJAjTh4MQb2+dXSMnWN3VfRRmdvQMPd2zWXigJ9Or7H7AdLcSJ8h9/TIjkkrq4N CcVuGZmKnSbA== X-IronPort-AV: E=Sophos;i="5.83,278,1616482800"; d="scan'208";a="554302750" Received: from vanderss-mobl.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.193]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2021 23:30:50 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v5 10/12] drm/i915/ttm: accelerated move implementation Date: Thu, 17 Jun 2021 08:30:16 +0200 Message-Id: <20210617063018.92802-11-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210617063018.92802-1-thomas.hellstrom@linux.intel.com> References: <20210617063018.92802-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.auld@intel.com, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Ramalingam C Invokes the pipelined page migration through blt, for i915_ttm_move requests of eviction and also obj clear. Signed-off-by: Ramalingam C Reviewed-by: Thomas Hellström --- v2: - subfunction for accel_move (Thomas) - engine_pm_get/put around context_move/clear (Thomas) - Invalidation at accel_clear (Thomas) v3: - conflict resolution s/&bo->mem/bo->resource/g v4: - Timeout is set for MAX_SCHEDULE_TIMEOUT (Thomas) - s/TTM_PL_PRIV/I915_PL_LMEM0 (Thomas) --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 88 +++++++++++++++++++++---- 1 file changed, 75 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 9366b18d1bc6..df46535cca47 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -15,6 +15,9 @@ #include "gem/i915_gem_ttm.h" #include "gem/i915_gem_mman.h" +#include "gt/intel_migrate.h" +#include "gt/intel_engine_pm.h" + #define I915_PL_LMEM0 TTM_PL_PRIV #define I915_PL_SYSTEM TTM_PL_SYSTEM #define I915_PL_STOLEN TTM_PL_VRAM @@ -326,6 +329,62 @@ i915_ttm_resource_get_st(struct drm_i915_gem_object *obj, return intel_region_ttm_resource_to_st(obj->mm.region, res); } +static int i915_ttm_accel_move(struct ttm_buffer_object *bo, + struct ttm_resource *dst_mem, + struct sg_table *dst_st) +{ + struct drm_i915_private *i915 = container_of(bo->bdev, typeof(*i915), + bdev); + struct ttm_resource_manager *src_man = + ttm_manager_type(bo->bdev, bo->resource->mem_type); + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + struct sg_table *src_st; + struct i915_request *rq; + int ret; + + if (!i915->gt.migrate.context) + return -EINVAL; + + if (!bo->ttm || !ttm_tt_is_populated(bo->ttm)) { + if (bo->type == ttm_bo_type_kernel) + return -EINVAL; + + if (bo->ttm && + !(bo->ttm->page_flags & TTM_PAGE_FLAG_ZERO_ALLOC)) + return 0; + + intel_engine_pm_get(i915->gt.migrate.context->engine); + ret = intel_context_migrate_clear(i915->gt.migrate.context, NULL, + dst_st->sgl, I915_CACHE_NONE, + dst_mem->mem_type >= I915_PL_LMEM0, + 0, &rq); + + if (!ret && rq) { + i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT); + i915_request_put(rq); + } + intel_engine_pm_put(i915->gt.migrate.context->engine); + } else { + src_st = src_man->use_tt ? i915_ttm_tt_get_st(bo->ttm) : + obj->ttm.cached_io_st; + + intel_engine_pm_get(i915->gt.migrate.context->engine); + ret = intel_context_migrate_copy(i915->gt.migrate.context, + NULL, src_st->sgl, I915_CACHE_NONE, + bo->resource->mem_type >= I915_PL_LMEM0, + dst_st->sgl, I915_CACHE_NONE, + dst_mem->mem_type >= I915_PL_LMEM0, + &rq); + if (!ret && rq) { + i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT); + i915_request_put(rq); + } + intel_engine_pm_put(i915->gt.migrate.context->engine); + } + + return ret; +} + static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *dst_mem, @@ -376,19 +435,22 @@ static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, if (IS_ERR(dst_st)) return PTR_ERR(dst_st); - /* If we start mapping GGTT, we can no longer use man::use_tt here. */ - dst_iter = dst_man->use_tt ? - ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm) : - ttm_kmap_iter_iomap_init(&_dst_iter.io, &dst_reg->iomap, - dst_st, dst_reg->region.start); - - src_iter = src_man->use_tt ? - ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm) : - ttm_kmap_iter_iomap_init(&_src_iter.io, &src_reg->iomap, - obj->ttm.cached_io_st, - src_reg->region.start); - - ttm_move_memcpy(bo, dst_mem->num_pages, dst_iter, src_iter); + ret = i915_ttm_accel_move(bo, dst_mem, dst_st); + if (ret) { + /* If we start mapping GGTT, we can no longer use man::use_tt here. */ + dst_iter = dst_man->use_tt ? + ttm_kmap_iter_tt_init(&_dst_iter.tt, bo->ttm) : + ttm_kmap_iter_iomap_init(&_dst_iter.io, &dst_reg->iomap, + dst_st, dst_reg->region.start); + + src_iter = src_man->use_tt ? + ttm_kmap_iter_tt_init(&_src_iter.tt, bo->ttm) : + ttm_kmap_iter_iomap_init(&_src_iter.io, &src_reg->iomap, + obj->ttm.cached_io_st, + src_reg->region.start); + + ttm_move_memcpy(bo, dst_mem->num_pages, dst_iter, src_iter); + } ttm_bo_move_sync_cleanup(bo, dst_mem); i915_ttm_free_cached_io_st(obj); -- 2.31.1