From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95E3CC47098 for ; Tue, 1 Jun 2021 07:47:34 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5D3746124B for ; Tue, 1 Jun 2021 07:47:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D3746124B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E01D56E89E; Tue, 1 Jun 2021 07:47:19 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 606C56E89F; Tue, 1 Jun 2021 07:47:17 +0000 (UTC) IronPort-SDR: AhGa/lbsKTMIPvylY5wReSwoYPEz3w7/RTr1PY/J88Ua/jpColjAIwQRS/uOukIzDsemim3YK5 ZBJL1IfseLwA== X-IronPort-AV: E=McAfee;i="6200,9189,10001"; a="203494076" X-IronPort-AV: E=Sophos;i="5.83,239,1616482800"; d="scan'208";a="203494076" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jun 2021 00:47:16 -0700 IronPort-SDR: W6Q5ZxWLHElpZov5nPYHYiDd8Y84gkxMapLX6jUwjwwl1Yx/icCC+/KbsGqvQuhIfz40ufi0p1 hygOTauwEycg== X-IronPort-AV: E=Sophos;i="5.83,239,1616482800"; d="scan'208";a="549648708" Received: from clillies-mobl.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.18]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jun 2021 00:47:14 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Tue, 1 Jun 2021 09:46:46 +0200 Message-Id: <20210601074654.3103-8-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210601074654.3103-1-thomas.hellstrom@linux.intel.com> References: <20210601074654.3103-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v9 07/15] drm: Add a prefetching memcpy_from_wc X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , =?UTF-8?q?Christian=20K=C3=B6nig?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" UmVhZGluZyBvdXQgb2Ygd3JpdGUtY29tYmluaW5nIG1hcHBlZCBtZW1vcnkgaXMgdHlwaWNhbGx5 IHZlcnkgc2xvdwpzaW5jZSB0aGUgQ1BVIGRvZXNuJ3QgcHJlZmV0Y2guIEhvd2V2ZXIgc29tZSBh cmNocyBoYXZlIHNwZWNpYWwKaW5zdHJ1Y3Rpb25zIHRvIGRvIHRoaXMuCgpTbyBhZGQgYSBiZXN0 LWVmZm9ydCBtZW1jcHlfZnJvbV93YyB0YWtpbmcgZG1hLWJ1Zi1tYXAgcG9pbnRlcgphcmd1bWVu dHMgdGhhdCBhdHRlbXB0cyB0byB1c2UgYSBmYXN0IHByZWZldGNoaW5nIG1lbWNweSBhbmQKb3Ro ZXJ3aXNlIGZhbGxzIGJhY2sgdG8gb3JkaW5hcnkgbWVtY29waWVzLCB0YWtpbmcgdGhlIGlvbWVt IHRhZ2dpbmcKaW50byBhY2NvdW50LgoKVGhlIGNvZGUgaXMgbGFyZ2VseSBjb3BpZWQgZnJvbSBp OTE1X21lbWNweV9mcm9tX3djLgoKQ2M6IERhbmllbCBWZXR0ZXIgPGRhbmllbEBmZndsbC5jaD4K Q2M6IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4KU3VnZ2VzdGVk LWJ5OiBEYW5pZWwgVmV0dGVyIDxkYW5pZWxAZmZ3bGwuY2g+ClNpZ25lZC1vZmYtYnk6IFRob21h cyBIZWxsc3Ryw7ZtIDx0aG9tYXMuaGVsbHN0cm9tQGxpbnV4LmludGVsLmNvbT4KQWNrZWQtYnk6 IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4KQWNrZWQtYnk6IERh bmllbCBWZXR0ZXIgPGRhbmllbEBmZndsbC5jaD4KLS0tCnY3OgotIFBlcmZvcm0gYSBtZW1jcHkg ZXZlbiBpZiB3YXJuaW5nIHdpdGggaW5faW50ZXJydXB0KCkuIFN1Z2dlc3RlZCBieQogIENocmlz dGlhbiBLw7ZuaWcuCi0gRml4IGNvbXBpbGF0aW9uIGZhaWx1cmUgb24gIVg4NiAoUmVwb3J0ZWQg Ynkga2VybmVsIHRlc3Qgcm9ib3QKICBsa3BAaW50ZWwuY29tKQp2ODoKLSBTa2lwIGtlcm5lbGRv YyBmb3IgZHJtX21lbWNweV9pbml0X2Vhcmx5KCkKLSBFeHBvcnQgZHJtX21lbWNweV9mcm9tX3dj KCkgYWxzbyBmb3Igbm9uLXg4Ni4KLS0tCiBEb2N1bWVudGF0aW9uL2dwdS9kcm0tbW0ucnN0IHwg ICAyICstCiBkcml2ZXJzL2dwdS9kcm0vZHJtX2NhY2hlLmMgIHwgMTQ4ICsrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrCiBkcml2ZXJzL2dwdS9kcm0vZHJtX2Rydi5jICAgIHwgICAy ICsKIGluY2x1ZGUvZHJtL2RybV9jYWNoZS5oICAgICAgfCAgIDcgKysKIDQgZmlsZXMgY2hhbmdl ZCwgMTU4IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKCmRpZmYgLS1naXQgYS9Eb2N1bWVu dGF0aW9uL2dwdS9kcm0tbW0ucnN0IGIvRG9jdW1lbnRhdGlvbi9ncHUvZHJtLW1tLnJzdAppbmRl eCAyMWJlNmRlYWRjMTIuLmM2NjA1OGM1YmNlNyAxMDA2NDQKLS0tIGEvRG9jdW1lbnRhdGlvbi9n cHUvZHJtLW1tLnJzdAorKysgYi9Eb2N1bWVudGF0aW9uL2dwdS9kcm0tbW0ucnN0CkBAIC00Njks NyArNDY5LDcgQEAgRFJNIE1NIFJhbmdlIEFsbG9jYXRvciBGdW5jdGlvbiBSZWZlcmVuY2VzCiAu LiBrZXJuZWwtZG9jOjogZHJpdmVycy9ncHUvZHJtL2RybV9tbS5jCiAgICA6ZXhwb3J0OgogCi1E Uk0gQ2FjaGUgSGFuZGxpbmcKK0RSTSBDYWNoZSBIYW5kbGluZyBhbmQgRmFzdCBXQyBtZW1jcHko KQogPT09PT09PT09PT09PT09PT09CiAKIC4uIGtlcm5lbC1kb2M6OiBkcml2ZXJzL2dwdS9kcm0v ZHJtX2NhY2hlLmMKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9kcm1fY2FjaGUuYyBiL2Ry aXZlcnMvZ3B1L2RybS9kcm1fY2FjaGUuYwppbmRleCA3OWE1MGVmMTI1MGYuLjU0NjU5OWYxOWE5 MyAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2RybV9jYWNoZS5jCisrKyBiL2RyaXZlcnMv Z3B1L2RybS9kcm1fY2FjaGUuYwpAQCAtMjgsNiArMjgsNyBAQAogICogQXV0aG9yczogVGhvbWFz IEhlbGxzdHLDtm0gPHRob21hcy1hdC10dW5nc3RlbmdyYXBoaWNzLWRvdC1jb20+CiAgKi8KIAor I2luY2x1ZGUgPGxpbnV4L2RtYS1idWYtbWFwLmg+CiAjaW5jbHVkZSA8bGludXgvZXhwb3J0Lmg+ CiAjaW5jbHVkZSA8bGludXgvaGlnaG1lbS5oPgogI2luY2x1ZGUgPGxpbnV4L21lbV9lbmNyeXB0 Lmg+CkBAIC0zNSw2ICszNiw5IEBACiAKICNpbmNsdWRlIDxkcm0vZHJtX2NhY2hlLmg+CiAKKy8q IEEgc21hbGwgYm91bmNlIGJ1ZmZlciB0aGF0IGZpdHMgb24gdGhlIHN0YWNrLiAqLworI2RlZmlu ZSBNRU1DUFlfQk9VTkNFX1NJWkUgMTI4CisKICNpZiBkZWZpbmVkKENPTkZJR19YODYpCiAjaW5j bHVkZSA8YXNtL3NtcC5oPgogCkBAIC0yMDksMyArMjEzLDE0NyBAQCBib29sIGRybV9uZWVkX3N3 aW90bGIoaW50IGRtYV9iaXRzKQogCXJldHVybiBtYXhfaW9tZW0gPiAoKHU2NCkxIDw8IGRtYV9i aXRzKTsKIH0KIEVYUE9SVF9TWU1CT0woZHJtX25lZWRfc3dpb3RsYik7CisKK3N0YXRpYyB2b2lk IG1lbWNweV9mYWxsYmFjayhzdHJ1Y3QgZG1hX2J1Zl9tYXAgKmRzdCwKKwkJCSAgICBjb25zdCBz dHJ1Y3QgZG1hX2J1Zl9tYXAgKnNyYywKKwkJCSAgICB1bnNpZ25lZCBsb25nIGxlbikKK3sKKwlp ZiAoIWRzdC0+aXNfaW9tZW0gJiYgIXNyYy0+aXNfaW9tZW0pIHsKKwkJbWVtY3B5KGRzdC0+dmFk ZHIsIHNyYy0+dmFkZHIsIGxlbik7CisJfSBlbHNlIGlmICghc3JjLT5pc19pb21lbSkgeworCQlk bWFfYnVmX21hcF9tZW1jcHlfdG8oZHN0LCBzcmMtPnZhZGRyLCBsZW4pOworCX0gZWxzZSBpZiAo IWRzdC0+aXNfaW9tZW0pIHsKKwkJbWVtY3B5X2Zyb21pbyhkc3QtPnZhZGRyLCBzcmMtPnZhZGRy X2lvbWVtLCBsZW4pOworCX0gZWxzZSB7CisJCS8qCisJCSAqIEJvdW5jZSBzaXplIGlzIG5vdCBw ZXJmb3JtYW5jZSB0dW5lZCwgYnV0IHVzaW5nIGEKKwkJICogYm91bmNlIGJ1ZmZlciBsaWtlIHRo aXMgaXMgc2lnbmlmaWNhbnRseSBmYXN0ZXIgdGhhbgorCQkgKiByZXNvcnRpbmcgdG8gaW9yZWFk eHgoKSArIGlvd3JpdGV4eCgpLgorCQkgKi8KKwkJY2hhciBib3VuY2VbTUVNQ1BZX0JPVU5DRV9T SVpFXTsKKwkJdm9pZCBfX2lvbWVtICpfc3JjID0gc3JjLT52YWRkcl9pb21lbTsKKwkJdm9pZCBf X2lvbWVtICpfZHN0ID0gZHN0LT52YWRkcl9pb21lbTsKKworCQl3aGlsZSAobGVuID49IE1FTUNQ WV9CT1VOQ0VfU0laRSkgeworCQkJbWVtY3B5X2Zyb21pbyhib3VuY2UsIF9zcmMsIE1FTUNQWV9C T1VOQ0VfU0laRSk7CisJCQltZW1jcHlfdG9pbyhfZHN0LCBib3VuY2UsIE1FTUNQWV9CT1VOQ0Vf U0laRSk7CisJCQlfc3JjICs9IE1FTUNQWV9CT1VOQ0VfU0laRTsKKwkJCV9kc3QgKz0gTUVNQ1BZ X0JPVU5DRV9TSVpFOworCQkJbGVuIC09IE1FTUNQWV9CT1VOQ0VfU0laRTsKKwkJfQorCQlpZiAo bGVuKSB7CisJCQltZW1jcHlfZnJvbWlvKGJvdW5jZSwgX3NyYywgTUVNQ1BZX0JPVU5DRV9TSVpF KTsKKwkJCW1lbWNweV90b2lvKF9kc3QsIGJvdW5jZSwgTUVNQ1BZX0JPVU5DRV9TSVpFKTsKKwkJ fQorCX0KK30KKworI2lmZGVmIENPTkZJR19YODYKKworc3RhdGljIERFRklORV9TVEFUSUNfS0VZ X0ZBTFNFKGhhc19tb3ZudGRxYSk7CisKK3N0YXRpYyB2b2lkIF9fbWVtY3B5X250ZHFhKHZvaWQg KmRzdCwgY29uc3Qgdm9pZCAqc3JjLCB1bnNpZ25lZCBsb25nIGxlbikKK3sKKwlrZXJuZWxfZnB1 X2JlZ2luKCk7CisKKwl3aGlsZSAobGVuID49IDQpIHsKKwkJYXNtKCJtb3ZudGRxYQkoJTApLCAl JXhtbTBcbiIKKwkJICAgICJtb3ZudGRxYSAxNiglMCksICUleG1tMVxuIgorCQkgICAgIm1vdm50 ZHFhIDMyKCUwKSwgJSV4bW0yXG4iCisJCSAgICAibW92bnRkcWEgNDgoJTApLCAlJXhtbTNcbiIK KwkJICAgICJtb3ZhcHMgJSV4bW0wLCAgICglMSlcbiIKKwkJICAgICJtb3ZhcHMgJSV4bW0xLCAx NiglMSlcbiIKKwkJICAgICJtb3ZhcHMgJSV4bW0yLCAzMiglMSlcbiIKKwkJICAgICJtb3ZhcHMg JSV4bW0zLCA0OCglMSlcbiIKKwkJICAgIDo6ICJyIiAoc3JjKSwgInIiIChkc3QpIDogIm1lbW9y eSIpOworCQlzcmMgKz0gNjQ7CisJCWRzdCArPSA2NDsKKwkJbGVuIC09IDQ7CisJfQorCXdoaWxl IChsZW4tLSkgeworCQlhc20oIm1vdm50ZHFhICglMCksICUleG1tMFxuIgorCQkgICAgIm1vdmFw cyAlJXhtbTAsICglMSlcbiIKKwkJICAgIDo6ICJyIiAoc3JjKSwgInIiIChkc3QpIDogIm1lbW9y eSIpOworCQlzcmMgKz0gMTY7CisJCWRzdCArPSAxNjsKKwl9CisKKwlrZXJuZWxfZnB1X2VuZCgp OworfQorCisvKgorICogX19kcm1fbWVtY3B5X2Zyb21fd2MgY29waWVzIEBsZW4gYnl0ZXMgZnJv bSBAc3JjIHRvIEBkc3QgdXNpbmcKKyAqIG5vbi10ZW1wb3JhbCBpbnN0cnVjdGlvbnMgd2hlcmUg YXZhaWxhYmxlLiBOb3RlIHRoYXQgYWxsIGFyZ3VtZW50cworICogKEBzcmMsIEBkc3QpIG11c3Qg YmUgYWxpZ25lZCB0byAxNiBieXRlcyBhbmQgQGxlbiBtdXN0IGJlIGEgbXVsdGlwbGUKKyAqIG9m IDE2LgorICovCitzdGF0aWMgdm9pZCBfX2RybV9tZW1jcHlfZnJvbV93Yyh2b2lkICpkc3QsIGNv bnN0IHZvaWQgKnNyYywgdW5zaWduZWQgbG9uZyBsZW4pCit7CisJaWYgKHVubGlrZWx5KCgodW5z aWduZWQgbG9uZylkc3QgfCAodW5zaWduZWQgbG9uZylzcmMgfCBsZW4pICYgMTUpKQorCQltZW1j cHkoZHN0LCBzcmMsIGxlbik7CisJZWxzZSBpZiAobGlrZWx5KGxlbikpCisJCV9fbWVtY3B5X250 ZHFhKGRzdCwgc3JjLCBsZW4gPj4gNCk7Cit9CisKKy8qKgorICogZHJtX21lbWNweV9mcm9tX3dj IC0gUGVyZm9ybSB0aGUgZmFzdGVzdCBhdmFpbGFibGUgbWVtY3B5IGZyb20gYSBzb3VyY2UKKyAq IHRoYXQgbWF5IGJlIFdDLgorICogQGRzdDogVGhlIGRlc3RpbmF0aW9uIHBvaW50ZXIKKyAqIEBz cmM6IFRoZSBzb3VyY2UgcG9pbnRlcgorICogQGxlbjogVGhlIHNpemUgb2YgdGhlIGFyZWEgbyB0 cmFuc2ZlciBpbiBieXRlcworICoKKyAqIFRyaWVzIGFuIGFyY2ggb3B0aW1pemVkIG1lbWNweSBm b3IgcHJlZmV0Y2hpbmcgcmVhZGluZyBvdXQgb2YgYSBXQyByZWdpb24sCisgKiBhbmQgaWYgbm8g c3VjaCBiZWFzdCBpcyBhdmFpbGFibGUsIGZhbGxzIGJhY2sgdG8gYSBub3JtYWwgbWVtY3B5Lgor ICovCit2b2lkIGRybV9tZW1jcHlfZnJvbV93YyhzdHJ1Y3QgZG1hX2J1Zl9tYXAgKmRzdCwKKwkJ CWNvbnN0IHN0cnVjdCBkbWFfYnVmX21hcCAqc3JjLAorCQkJdW5zaWduZWQgbG9uZyBsZW4pCit7 CisJaWYgKFdBUk5fT04oaW5faW50ZXJydXB0KCkpKSB7CisJCW1lbWNweV9mYWxsYmFjayhkc3Qs IHNyYywgbGVuKTsKKwkJcmV0dXJuOworCX0KKworCWlmIChzdGF0aWNfYnJhbmNoX2xpa2VseSgm aGFzX21vdm50ZHFhKSkgeworCQlfX2RybV9tZW1jcHlfZnJvbV93Yyhkc3QtPmlzX2lvbWVtID8K KwkJCQkgICAgICh2b2lkIF9fZm9yY2UgKilkc3QtPnZhZGRyX2lvbWVtIDoKKwkJCQkgICAgIGRz dC0+dmFkZHIsCisJCQkJICAgICBzcmMtPmlzX2lvbWVtID8KKwkJCQkgICAgICh2b2lkIGNvbnN0 IF9fZm9yY2UgKilzcmMtPnZhZGRyX2lvbWVtIDoKKwkJCQkgICAgIHNyYy0+dmFkZHIsCisJCQkJ ICAgICBsZW4pOworCQlyZXR1cm47CisJfQorCisJbWVtY3B5X2ZhbGxiYWNrKGRzdCwgc3JjLCBs ZW4pOworfQorRVhQT1JUX1NZTUJPTChkcm1fbWVtY3B5X2Zyb21fd2MpOworCisvKgorICogZHJt X21lbWNweV9pbml0X2Vhcmx5IC0gT25lIHRpbWUgaW5pdGlhbGl6YXRpb24gb2YgdGhlIFdDIG1l bWNweSBjb2RlCisgKi8KK3ZvaWQgZHJtX21lbWNweV9pbml0X2Vhcmx5KHZvaWQpCit7CisJLyoK KwkgKiBTb21lIGh5cGVydmlzb3JzIChlLmcuIEtWTSkgZG9uJ3Qgc3VwcG9ydCBWRVgtcHJlZml4 IGluc3RydWN0aW9ucworCSAqIGVtdWxhdGlvbi4gU28gZG9uJ3QgZW5hYmxlIG1vdm50ZHFhIGlu IGh5cGVydmlzb3IgZ3Vlc3QuCisJICovCisJaWYgKHN0YXRpY19jcHVfaGFzKFg4Nl9GRUFUVVJF X1hNTTRfMSkgJiYKKwkgICAgIWJvb3RfY3B1X2hhcyhYODZfRkVBVFVSRV9IWVBFUlZJU09SKSkK KwkJc3RhdGljX2JyYW5jaF9lbmFibGUoJmhhc19tb3ZudGRxYSk7Cit9CisjZWxzZQordm9pZCBk cm1fbWVtY3B5X2Zyb21fd2Moc3RydWN0IGRtYV9idWZfbWFwICpkc3QsCisJCQljb25zdCBzdHJ1 Y3QgZG1hX2J1Zl9tYXAgKnNyYywKKwkJCXVuc2lnbmVkIGxvbmcgbGVuKQoreworCVdBUk5fT04o aW5faW50ZXJydXB0KCkpOworCisJbWVtY3B5X2ZhbGxiYWNrKGRzdCwgc3JjLCBsZW4pOworfQor RVhQT1JUX1NZTUJPTChkcm1fbWVtY3B5X2Zyb21fd2MpOworCit2b2lkIGRybV9tZW1jcHlfaW5p dF9lYXJseSh2b2lkKQoreworfQorI2VuZGlmIC8qIENPTkZJR19YODYgKi8KZGlmZiAtLWdpdCBh L2RyaXZlcnMvZ3B1L2RybS9kcm1fZHJ2LmMgYi9kcml2ZXJzL2dwdS9kcm0vZHJtX2Rydi5jCmlu ZGV4IDNkOGQ2OGE5OGI5NS4uODgwNGVjN2QzMjE1IDEwMDY0NAotLS0gYS9kcml2ZXJzL2dwdS9k cm0vZHJtX2Rydi5jCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9kcm1fZHJ2LmMKQEAgLTM1LDYgKzM1 LDcgQEAKICNpbmNsdWRlIDxsaW51eC9zbGFiLmg+CiAjaW5jbHVkZSA8bGludXgvc3JjdS5oPgog CisjaW5jbHVkZSA8ZHJtL2RybV9jYWNoZS5oPgogI2luY2x1ZGUgPGRybS9kcm1fY2xpZW50Lmg+ CiAjaW5jbHVkZSA8ZHJtL2RybV9jb2xvcl9tZ210Lmg+CiAjaW5jbHVkZSA8ZHJtL2RybV9kcnYu aD4KQEAgLTEwNDEsNiArMTA0Miw3IEBAIHN0YXRpYyBpbnQgX19pbml0IGRybV9jb3JlX2luaXQo dm9pZCkKIAogCWRybV9jb25uZWN0b3JfaWRhX2luaXQoKTsKIAlpZHJfaW5pdCgmZHJtX21pbm9y c19pZHIpOworCWRybV9tZW1jcHlfaW5pdF9lYXJseSgpOwogCiAJcmV0ID0gZHJtX3N5c2ZzX2lu aXQoKTsKIAlpZiAocmV0IDwgMCkgewpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9kcm0vZHJtX2NhY2hl LmggYi9pbmNsdWRlL2RybS9kcm1fY2FjaGUuaAppbmRleCBlOWFkNDg2M2Q5MTUuLmNjOWRlMTYz MmRkMyAxMDA2NDQKLS0tIGEvaW5jbHVkZS9kcm0vZHJtX2NhY2hlLmgKKysrIGIvaW5jbHVkZS9k cm0vZHJtX2NhY2hlLmgKQEAgLTM1LDYgKzM1LDggQEAKIAogI2luY2x1ZGUgPGxpbnV4L3NjYXR0 ZXJsaXN0Lmg+CiAKK3N0cnVjdCBkbWFfYnVmX21hcDsKKwogdm9pZCBkcm1fY2xmbHVzaF9wYWdl cyhzdHJ1Y3QgcGFnZSAqcGFnZXNbXSwgdW5zaWduZWQgbG9uZyBudW1fcGFnZXMpOwogdm9pZCBk cm1fY2xmbHVzaF9zZyhzdHJ1Y3Qgc2dfdGFibGUgKnN0KTsKIHZvaWQgZHJtX2NsZmx1c2hfdmly dF9yYW5nZSh2b2lkICphZGRyLCB1bnNpZ25lZCBsb25nIGxlbmd0aCk7CkBAIC03MCw0ICs3Miw5 IEBAIHN0YXRpYyBpbmxpbmUgYm9vbCBkcm1fYXJjaF9jYW5fd2NfbWVtb3J5KHZvaWQpCiAjZW5k aWYKIH0KIAordm9pZCBkcm1fbWVtY3B5X2luaXRfZWFybHkodm9pZCk7CisKK3ZvaWQgZHJtX21l bWNweV9mcm9tX3djKHN0cnVjdCBkbWFfYnVmX21hcCAqZHN0LAorCQkJY29uc3Qgc3RydWN0IGRt YV9idWZfbWFwICpzcmMsCisJCQl1bnNpZ25lZCBsb25nIGxlbik7CiAjZW5kaWYKLS0gCjIuMzEu MQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KSW50ZWwt Z2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8v bGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48D8EC47097 for ; Tue, 1 Jun 2021 07:47:32 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 056D660698 for ; Tue, 1 Jun 2021 07:47:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 056D660698 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 473C66E8A2; Tue, 1 Jun 2021 07:47:18 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 606C56E89F; Tue, 1 Jun 2021 07:47:17 +0000 (UTC) IronPort-SDR: AhGa/lbsKTMIPvylY5wReSwoYPEz3w7/RTr1PY/J88Ua/jpColjAIwQRS/uOukIzDsemim3YK5 ZBJL1IfseLwA== X-IronPort-AV: E=McAfee;i="6200,9189,10001"; a="203494076" X-IronPort-AV: E=Sophos;i="5.83,239,1616482800"; d="scan'208";a="203494076" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jun 2021 00:47:16 -0700 IronPort-SDR: W6Q5ZxWLHElpZov5nPYHYiDd8Y84gkxMapLX6jUwjwwl1Yx/icCC+/KbsGqvQuhIfz40ufi0p1 hygOTauwEycg== X-IronPort-AV: E=Sophos;i="5.83,239,1616482800"; d="scan'208";a="549648708" Received: from clillies-mobl.ger.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.18]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jun 2021 00:47:14 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v9 07/15] drm: Add a prefetching memcpy_from_wc Date: Tue, 1 Jun 2021 09:46:46 +0200 Message-Id: <20210601074654.3103-8-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210601074654.3103-1-thomas.hellstrom@linux.intel.com> References: <20210601074654.3103-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , =?UTF-8?q?Christian=20K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Reading out of write-combining mapped memory is typically very slow since the CPU doesn't prefetch. However some archs have special instructions to do this. So add a best-effort memcpy_from_wc taking dma-buf-map pointer arguments that attempts to use a fast prefetching memcpy and otherwise falls back to ordinary memcopies, taking the iomem tagging into account. The code is largely copied from i915_memcpy_from_wc. Cc: Daniel Vetter Cc: Christian König Suggested-by: Daniel Vetter Signed-off-by: Thomas Hellström Acked-by: Christian König Acked-by: Daniel Vetter --- v7: - Perform a memcpy even if warning with in_interrupt(). Suggested by Christian König. - Fix compilation failure on !X86 (Reported by kernel test robot lkp@intel.com) v8: - Skip kerneldoc for drm_memcpy_init_early() - Export drm_memcpy_from_wc() also for non-x86. --- Documentation/gpu/drm-mm.rst | 2 +- drivers/gpu/drm/drm_cache.c | 148 +++++++++++++++++++++++++++++++++++ drivers/gpu/drm/drm_drv.c | 2 + include/drm/drm_cache.h | 7 ++ 4 files changed, 158 insertions(+), 1 deletion(-) diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst index 21be6deadc12..c66058c5bce7 100644 --- a/Documentation/gpu/drm-mm.rst +++ b/Documentation/gpu/drm-mm.rst @@ -469,7 +469,7 @@ DRM MM Range Allocator Function References .. kernel-doc:: drivers/gpu/drm/drm_mm.c :export: -DRM Cache Handling +DRM Cache Handling and Fast WC memcpy() ================== .. kernel-doc:: drivers/gpu/drm/drm_cache.c diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c index 79a50ef1250f..546599f19a93 100644 --- a/drivers/gpu/drm/drm_cache.c +++ b/drivers/gpu/drm/drm_cache.c @@ -28,6 +28,7 @@ * Authors: Thomas Hellström */ +#include #include #include #include @@ -35,6 +36,9 @@ #include +/* A small bounce buffer that fits on the stack. */ +#define MEMCPY_BOUNCE_SIZE 128 + #if defined(CONFIG_X86) #include @@ -209,3 +213,147 @@ bool drm_need_swiotlb(int dma_bits) return max_iomem > ((u64)1 << dma_bits); } EXPORT_SYMBOL(drm_need_swiotlb); + +static void memcpy_fallback(struct dma_buf_map *dst, + const struct dma_buf_map *src, + unsigned long len) +{ + if (!dst->is_iomem && !src->is_iomem) { + memcpy(dst->vaddr, src->vaddr, len); + } else if (!src->is_iomem) { + dma_buf_map_memcpy_to(dst, src->vaddr, len); + } else if (!dst->is_iomem) { + memcpy_fromio(dst->vaddr, src->vaddr_iomem, len); + } else { + /* + * Bounce size is not performance tuned, but using a + * bounce buffer like this is significantly faster than + * resorting to ioreadxx() + iowritexx(). + */ + char bounce[MEMCPY_BOUNCE_SIZE]; + void __iomem *_src = src->vaddr_iomem; + void __iomem *_dst = dst->vaddr_iomem; + + while (len >= MEMCPY_BOUNCE_SIZE) { + memcpy_fromio(bounce, _src, MEMCPY_BOUNCE_SIZE); + memcpy_toio(_dst, bounce, MEMCPY_BOUNCE_SIZE); + _src += MEMCPY_BOUNCE_SIZE; + _dst += MEMCPY_BOUNCE_SIZE; + len -= MEMCPY_BOUNCE_SIZE; + } + if (len) { + memcpy_fromio(bounce, _src, MEMCPY_BOUNCE_SIZE); + memcpy_toio(_dst, bounce, MEMCPY_BOUNCE_SIZE); + } + } +} + +#ifdef CONFIG_X86 + +static DEFINE_STATIC_KEY_FALSE(has_movntdqa); + +static void __memcpy_ntdqa(void *dst, const void *src, unsigned long len) +{ + kernel_fpu_begin(); + + while (len >= 4) { + asm("movntdqa (%0), %%xmm0\n" + "movntdqa 16(%0), %%xmm1\n" + "movntdqa 32(%0), %%xmm2\n" + "movntdqa 48(%0), %%xmm3\n" + "movaps %%xmm0, (%1)\n" + "movaps %%xmm1, 16(%1)\n" + "movaps %%xmm2, 32(%1)\n" + "movaps %%xmm3, 48(%1)\n" + :: "r" (src), "r" (dst) : "memory"); + src += 64; + dst += 64; + len -= 4; + } + while (len--) { + asm("movntdqa (%0), %%xmm0\n" + "movaps %%xmm0, (%1)\n" + :: "r" (src), "r" (dst) : "memory"); + src += 16; + dst += 16; + } + + kernel_fpu_end(); +} + +/* + * __drm_memcpy_from_wc copies @len bytes from @src to @dst using + * non-temporal instructions where available. Note that all arguments + * (@src, @dst) must be aligned to 16 bytes and @len must be a multiple + * of 16. + */ +static void __drm_memcpy_from_wc(void *dst, const void *src, unsigned long len) +{ + if (unlikely(((unsigned long)dst | (unsigned long)src | len) & 15)) + memcpy(dst, src, len); + else if (likely(len)) + __memcpy_ntdqa(dst, src, len >> 4); +} + +/** + * drm_memcpy_from_wc - Perform the fastest available memcpy from a source + * that may be WC. + * @dst: The destination pointer + * @src: The source pointer + * @len: The size of the area o transfer in bytes + * + * Tries an arch optimized memcpy for prefetching reading out of a WC region, + * and if no such beast is available, falls back to a normal memcpy. + */ +void drm_memcpy_from_wc(struct dma_buf_map *dst, + const struct dma_buf_map *src, + unsigned long len) +{ + if (WARN_ON(in_interrupt())) { + memcpy_fallback(dst, src, len); + return; + } + + if (static_branch_likely(&has_movntdqa)) { + __drm_memcpy_from_wc(dst->is_iomem ? + (void __force *)dst->vaddr_iomem : + dst->vaddr, + src->is_iomem ? + (void const __force *)src->vaddr_iomem : + src->vaddr, + len); + return; + } + + memcpy_fallback(dst, src, len); +} +EXPORT_SYMBOL(drm_memcpy_from_wc); + +/* + * drm_memcpy_init_early - One time initialization of the WC memcpy code + */ +void drm_memcpy_init_early(void) +{ + /* + * Some hypervisors (e.g. KVM) don't support VEX-prefix instructions + * emulation. So don't enable movntdqa in hypervisor guest. + */ + if (static_cpu_has(X86_FEATURE_XMM4_1) && + !boot_cpu_has(X86_FEATURE_HYPERVISOR)) + static_branch_enable(&has_movntdqa); +} +#else +void drm_memcpy_from_wc(struct dma_buf_map *dst, + const struct dma_buf_map *src, + unsigned long len) +{ + WARN_ON(in_interrupt()); + + memcpy_fallback(dst, src, len); +} +EXPORT_SYMBOL(drm_memcpy_from_wc); + +void drm_memcpy_init_early(void) +{ +} +#endif /* CONFIG_X86 */ diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c index 3d8d68a98b95..8804ec7d3215 100644 --- a/drivers/gpu/drm/drm_drv.c +++ b/drivers/gpu/drm/drm_drv.c @@ -35,6 +35,7 @@ #include #include +#include #include #include #include @@ -1041,6 +1042,7 @@ static int __init drm_core_init(void) drm_connector_ida_init(); idr_init(&drm_minors_idr); + drm_memcpy_init_early(); ret = drm_sysfs_init(); if (ret < 0) { diff --git a/include/drm/drm_cache.h b/include/drm/drm_cache.h index e9ad4863d915..cc9de1632dd3 100644 --- a/include/drm/drm_cache.h +++ b/include/drm/drm_cache.h @@ -35,6 +35,8 @@ #include +struct dma_buf_map; + void drm_clflush_pages(struct page *pages[], unsigned long num_pages); void drm_clflush_sg(struct sg_table *st); void drm_clflush_virt_range(void *addr, unsigned long length); @@ -70,4 +72,9 @@ static inline bool drm_arch_can_wc_memory(void) #endif } +void drm_memcpy_init_early(void); + +void drm_memcpy_from_wc(struct dma_buf_map *dst, + const struct dma_buf_map *src, + unsigned long len); #endif -- 2.31.1