From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89402C433B4 for ; Tue, 18 May 2021 08:28:02 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 59F42610E9 for ; Tue, 18 May 2021 08:28:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59F42610E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 683A86EAC6; Tue, 18 May 2021 08:28:00 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8C9596EAC0; Tue, 18 May 2021 08:27:58 +0000 (UTC) IronPort-SDR: ESGkcn6jZwr9uJBCMR/wc9+YLx9/jdgd6heDKBTC8IDDYwxGdbWrO9ap6OF2voOfW8AH8Giy6V qN0OMYYSX3Gw== X-IronPort-AV: E=McAfee;i="6200,9189,9987"; a="197572728" X-IronPort-AV: E=Sophos;i="5.82,309,1613462400"; d="scan'208";a="197572728" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2021 01:27:58 -0700 IronPort-SDR: CYHSjNVrT7WB5jDAwd/GtBEuDiPrNjZN4FWQghuQO0ILOmyVuRhl7pN2ONtZBfGUdogdyVTBhP XjUXU0sVbAtQ== X-IronPort-AV: E=Sophos;i="5.82,309,1613462400"; d="scan'208";a="611892377" Received: from cmutgix-mobl.gar.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.195]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2021 01:27:44 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Date: Tue, 18 May 2021 10:26:56 +0200 Message-Id: <20210518082701.997251-11-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210518082701.997251-1-thomas.hellstrom@linux.intel.com> References: <20210518082701.997251-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 10/15] drm/i915/ttm: Introduce a TTM i915 gem object backend X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" TW9zdCBsb2dpY2FsIHBsYWNlIHRvIGludHJvZHVjZSBUVE0gYnVmZmVyIG9iamVjdHMgaXMgYXMg YW4gaTkxNQpnZW0gb2JqZWN0IGJhY2tlbmQuIFdlIG5lZWQgdG8gYWRkIHNvbWUgb3BzIHRvIGFj Y291bnQgZm9yIGFkZGVkCmZ1bmN0aW9uYWxpdHkgbGlrZSBkZWxheWVkIGRlbGV0ZSBhbmQgTFJV IGxpc3QgbWFuaXB1bGF0aW9uLgoKSW5pdGlhbGx5IHdlIHN1cHBvcnQgb25seSBMTUVNIGFuZCBT WVNURU0gbWVtb3J5LCBidXQgU1lTVEVNCih3aGljaCBpbiB0aGlzIGNhc2UgbWVhbnMgZXZpY3Rl ZCBMTUVNIG9iamVjdHMpIGlzIG5vdAp2aXNpYmxlIHRvIGk5MTUgR0VNIHlldC4gVGhlIHBsYW4g aXMgdG8gbW92ZSB0aGUgaTkxNSBnZW0gc3lzdGVtIHJlZ2lvbgpvdmVyIHRvIHRoZSBUVE0gc3lz dGVtIG1lbW9yeSB0eXBlIGluIHVwY29taW5nIHBhdGNoZXMuCgpXZSBzZXQgdXAgR1BVIGJpbmRp bmdzIGRpcmVjdGx5IGJvdGggZnJvbSBMTUVNIGFuZCBmcm9tIHRoZSBzeXN0ZW0gcmVnaW9uLAph cyB0aGVyZSBpcyBubyBuZWVkIHRvIHVzZSB0aGUgbGVnYWN5IFRUTV9UVCBtZW1vcnkgdHlwZS4g V2UgcmVzZXJ2ZQp0aGF0IGZvciBmdXR1cmUgcG9ydGluZyBvZiBHR1RUIGJpbmRpbmdzIHRvIFRU TS4KClJlbW92ZSB0aGUgb2xkIGxtZW0gYmFja2VuZC4KClNpZ25lZC1vZmYtYnk6IFRob21hcyBI ZWxsc3Ryw7ZtIDx0aG9tYXMuaGVsbHN0cm9tQGxpbnV4LmludGVsLmNvbT4KLS0tCnYyOgotIEJy ZWFrIG91dCBuZWVkZWQgVFRNIGZ1bmN0aW9uYWxpdHkgdG8gYSBzZXBhcmF0ZSBwYXRjaCAoUmVw b3J0ZWQgYnkKQ2hyaXN0aWFuIEvDtm5pZykuCi0gRml4IGFuIHVuaGFuZGxlZCBlcnJvciAoUmVw b3J0ZWQgYnkgTWF0dGhldyBBdWxkIGFuZCBNYWFydGVuIExhbmtob3JzdCkKLSBSZW1vdmUgYSBz dHJheSBsZWZ0b3ZlciBzZ190YWJsZSBhbGxvY2F0aW9uIChSZXBvcnRlZCBieSBNYXR0aGV3IEF1 bGQpCi0gVXNlIHR0bV90dF91bnBvcHVsYXRlKCkgcmF0aGVyIHRoYW4gdHRtX3R0X2Rlc3Ryb3ko KSBpbiB0aGUgcHVyZ2UgcGF0aAogIGFzIHNvbWUgVFRNIGZ1bmN0aW9uYWxpdHkgcmVsaWVzIG9u IGhhdmluZyBhIHR0bV90dCBwcmVzZW50IGZvciAhaXNfaW9tZW0uCi0tLQogZHJpdmVycy9ncHUv ZHJtL2k5MTUvTWFrZWZpbGUgICAgICAgICAgICAgICAgIHwgICAxICsKIGRyaXZlcnMvZ3B1L2Ry bS9pOTE1L2dlbS9pOTE1X2dlbV9sbWVtLmMgICAgICB8ICA4NCAtLS0KIGRyaXZlcnMvZ3B1L2Ry bS9pOTE1L2dlbS9pOTE1X2dlbV9sbWVtLmggICAgICB8ICAgNSAtCiBkcml2ZXJzL2dwdS9kcm0v aTkxNS9nZW0vaTkxNV9nZW1fb2JqZWN0LmMgICAgfCAxMjUgKysrLS0KIGRyaXZlcnMvZ3B1L2Ry bS9pOTE1L2dlbS9pOTE1X2dlbV9vYmplY3QuaCAgICB8ICAgOSArCiAuLi4vZ3B1L2RybS9pOTE1 L2dlbS9pOTE1X2dlbV9vYmplY3RfdHlwZXMuaCAgfCAgMTggKwogZHJpdmVycy9ncHUvZHJtL2k5 MTUvZ2VtL2k5MTVfZ2VtX3JlZ2lvbi5jICAgIHwgICA2ICstCiBkcml2ZXJzL2dwdS9kcm0vaTkx NS9nZW0vaTkxNV9nZW1fdHRtLmMgICAgICAgfCA1MTkgKysrKysrKysrKysrKysrKysrCiBkcml2 ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtLmggICAgICAgfCAgNDggKysKIGRyaXZl cnMvZ3B1L2RybS9pOTE1L2d0L2ludGVsX3JlZ2lvbl9sbWVtLmMgICB8ICAgMyArLQogZHJpdmVy cy9ncHUvZHJtL2k5MTUvaTkxNV9nZW0uYyAgICAgICAgICAgICAgIHwgICA1ICstCiBkcml2ZXJz L2dwdS9kcm0vaTkxNS9pbnRlbF9tZW1vcnlfcmVnaW9uLmMgICAgfCAgIDEgLQogZHJpdmVycy9n cHUvZHJtL2k5MTUvaW50ZWxfbWVtb3J5X3JlZ2lvbi5oICAgIHwgICAxIC0KIGRyaXZlcnMvZ3B1 L2RybS9pOTE1L2ludGVsX3JlZ2lvbl90dG0uYyAgICAgICB8ICAgNSArLQogZHJpdmVycy9ncHUv ZHJtL2k5MTUvaW50ZWxfcmVnaW9uX3R0bS5oICAgICAgIHwgICA3ICstCiAxNSBmaWxlcyBjaGFu Z2VkLCA2OTYgaW5zZXJ0aW9ucygrKSwgMTQxIGRlbGV0aW9ucygtKQogY3JlYXRlIG1vZGUgMTAw NjQ0IGRyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG0uYwogY3JlYXRlIG1vZGUg MTAwNjQ0IGRyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG0uaAoKZGlmZiAtLWdp dCBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L01ha2VmaWxlIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUv TWFrZWZpbGUKaW5kZXggOTU4Y2NjMWVkZmVkLi5lZjBkODg0YTllMmQgMTAwNjQ0Ci0tLSBhL2Ry aXZlcnMvZ3B1L2RybS9pOTE1L01ha2VmaWxlCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L01h a2VmaWxlCkBAIC0xNTUsNiArMTU1LDcgQEAgZ2VtLXkgKz0gXAogCWdlbS9pOTE1X2dlbV9zdG9s ZW4ubyBcCiAJZ2VtL2k5MTVfZ2VtX3Rocm90dGxlLm8gXAogCWdlbS9pOTE1X2dlbV90aWxpbmcu byBcCisJZ2VtL2k5MTVfZ2VtX3R0bS5vIFwKIAlnZW0vaTkxNV9nZW1fdHRtX2JvX3V0aWwubyBc CiAJZ2VtL2k5MTVfZ2VtX3VzZXJwdHIubyBcCiAJZ2VtL2k5MTVfZ2VtX3dhaXQubyBcCmRpZmYg LS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fbG1lbS5jIGIvZHJpdmVy cy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX2xtZW0uYwppbmRleCAzYjRhYTI4YTA3NmQuLjJi OGNkMTVkZTFkOSAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2Vt X2xtZW0uYworKysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fbG1lbS5jCkBA IC00LDc0ICs0LDEwIEBACiAgKi8KIAogI2luY2x1ZGUgImludGVsX21lbW9yeV9yZWdpb24uaCIK LSNpbmNsdWRlICJpbnRlbF9yZWdpb25fdHRtLmgiCiAjaW5jbHVkZSAiZ2VtL2k5MTVfZ2VtX3Jl Z2lvbi5oIgogI2luY2x1ZGUgImdlbS9pOTE1X2dlbV9sbWVtLmgiCiAjaW5jbHVkZSAiaTkxNV9k cnYuaCIKIAotc3RhdGljIHZvaWQgbG1lbV9wdXRfcGFnZXMoc3RydWN0IGRybV9pOTE1X2dlbV9v YmplY3QgKm9iaiwKLQkJCSAgIHN0cnVjdCBzZ190YWJsZSAqcGFnZXMpCi17Ci0JaW50ZWxfcmVn aW9uX3R0bV9ub2RlX2ZyZWUob2JqLT5tbS5yZWdpb24sIG9iai0+bW0uc3RfbW1fbm9kZSk7Ci0J b2JqLT5tbS5kaXJ0eSA9IGZhbHNlOwotCXNnX2ZyZWVfdGFibGUocGFnZXMpOwotCWtmcmVlKHBh Z2VzKTsKLX0KLQotc3RhdGljIGludCBsbWVtX2dldF9wYWdlcyhzdHJ1Y3QgZHJtX2k5MTVfZ2Vt X29iamVjdCAqb2JqKQotewotCXVuc2lnbmVkIGludCBmbGFnczsKLQlzdHJ1Y3Qgc2dfdGFibGUg KnBhZ2VzOwotCi0JZmxhZ3MgPSBJOTE1X0FMTE9DX01JTl9QQUdFX1NJWkU7Ci0JaWYgKG9iai0+ ZmxhZ3MgJiBJOTE1X0JPX0FMTE9DX0NPTlRJR1VPVVMpCi0JCWZsYWdzIHw9IEk5MTVfQUxMT0Nf Q09OVElHVU9VUzsKLQotCW9iai0+bW0uc3RfbW1fbm9kZSA9IGludGVsX3JlZ2lvbl90dG1fbm9k ZV9hbGxvYyhvYmotPm1tLnJlZ2lvbiwKLQkJCQkJCQkgb2JqLT5iYXNlLnNpemUsCi0JCQkJCQkJ IGZsYWdzKTsKLQlpZiAoSVNfRVJSKG9iai0+bW0uc3RfbW1fbm9kZSkpCi0JCXJldHVybiBQVFJf RVJSKG9iai0+bW0uc3RfbW1fbm9kZSk7Ci0KLQkvKiBSYW5nZSBtYW5hZ2VyIGlzIGFsd2F5cyBj b250aWdvdXMgKi8KLQlpZiAob2JqLT5tbS5yZWdpb24tPmlzX3JhbmdlX21hbmFnZXIpCi0JCW9i ai0+ZmxhZ3MgfD0gSTkxNV9CT19BTExPQ19DT05USUdVT1VTOwotCXBhZ2VzID0gaW50ZWxfcmVn aW9uX3R0bV9ub2RlX3RvX3N0KG9iai0+bW0ucmVnaW9uLCBvYmotPm1tLnN0X21tX25vZGUpOwot CWlmIChJU19FUlIocGFnZXMpKSB7Ci0JCWludGVsX3JlZ2lvbl90dG1fbm9kZV9mcmVlKG9iai0+ bW0ucmVnaW9uLCBvYmotPm1tLnN0X21tX25vZGUpOwotCQlyZXR1cm4gUFRSX0VSUihwYWdlcyk7 Ci0JfQotCi0JX19pOTE1X2dlbV9vYmplY3Rfc2V0X3BhZ2VzKG9iaiwgcGFnZXMsIGk5MTVfc2df ZG1hX3NpemVzKHBhZ2VzLT5zZ2wpKTsKLQotCWlmIChvYmotPmZsYWdzICYgSTkxNV9CT19BTExP Q19DUFVfQ0xFQVIpIHsKLQkJdm9pZCBfX2lvbWVtICp2YWRkciA9Ci0JCQlpOTE1X2dlbV9vYmpl Y3RfbG1lbV9pb19tYXAob2JqLCAwLCBvYmotPmJhc2Uuc2l6ZSk7Ci0KLQkJaWYgKCF2YWRkcikg ewotCQkJc3RydWN0IHNnX3RhYmxlICpwYWdlcyA9Ci0JCQkJX19pOTE1X2dlbV9vYmplY3RfdW5z ZXRfcGFnZXMob2JqKTsKLQotCQkJaWYgKCFJU19FUlJfT1JfTlVMTChwYWdlcykpCi0JCQkJbG1l bV9wdXRfcGFnZXMob2JqLCBwYWdlcyk7Ci0JCX0KLQotCQltZW1zZXRfaW8odmFkZHIsIDAsIG9i ai0+YmFzZS5zaXplKTsKLQkJaW9fbWFwcGluZ191bm1hcCh2YWRkcik7Ci0JfQotCi0JcmV0dXJu IDA7Ci19Ci0KLWNvbnN0IHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0X29wcyBpOTE1X2dlbV9s bWVtX29ial9vcHMgPSB7Ci0JLm5hbWUgPSAiaTkxNV9nZW1fb2JqZWN0X2xtZW0iLAotCS5mbGFn cyA9IEk5MTVfR0VNX09CSkVDVF9IQVNfSU9NRU0sCi0KLQkuZ2V0X3BhZ2VzID0gbG1lbV9nZXRf cGFnZXMsCi0JLnB1dF9wYWdlcyA9IGxtZW1fcHV0X3BhZ2VzLAotCS5yZWxlYXNlID0gaTkxNV9n ZW1fb2JqZWN0X3JlbGVhc2VfbWVtb3J5X3JlZ2lvbiwKLX07Ci0KIHZvaWQgX19pb21lbSAqCiBp OTE1X2dlbV9vYmplY3RfbG1lbV9pb19tYXAoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9i aiwKIAkJCSAgICB1bnNpZ25lZCBsb25nIG4sCkBAIC0xMDMsMjMgKzM5LDMgQEAgaTkxNV9nZW1f b2JqZWN0X2NyZWF0ZV9sbWVtKHN0cnVjdCBkcm1faTkxNV9wcml2YXRlICppOTE1LAogCXJldHVy biBpOTE1X2dlbV9vYmplY3RfY3JlYXRlX3JlZ2lvbihpOTE1LT5tbS5yZWdpb25zW0lOVEVMX1JF R0lPTl9MTUVNXSwKIAkJCQkJICAgICBzaXplLCBmbGFncyk7CiB9Ci0KLWludCBfX2k5MTVfZ2Vt X2xtZW1fb2JqZWN0X2luaXQoc3RydWN0IGludGVsX21lbW9yeV9yZWdpb24gKm1lbSwKLQkJCQlz dHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqLAotCQkJCXJlc291cmNlX3NpemVfdCBzaXpl LAotCQkJCXVuc2lnbmVkIGludCBmbGFncykKLXsKLQlzdGF0aWMgc3RydWN0IGxvY2tfY2xhc3Nf a2V5IGxvY2tfY2xhc3M7Ci0Jc3RydWN0IGRybV9pOTE1X3ByaXZhdGUgKmk5MTUgPSBtZW0tPmk5 MTU7Ci0KLQlkcm1fZ2VtX3ByaXZhdGVfb2JqZWN0X2luaXQoJmk5MTUtPmRybSwgJm9iai0+YmFz ZSwgc2l6ZSk7Ci0JaTkxNV9nZW1fb2JqZWN0X2luaXQob2JqLCAmaTkxNV9nZW1fbG1lbV9vYmpf b3BzLCAmbG9ja19jbGFzcywgZmxhZ3MpOwotCi0Jb2JqLT5yZWFkX2RvbWFpbnMgPSBJOTE1X0dF TV9ET01BSU5fV0MgfCBJOTE1X0dFTV9ET01BSU5fR1RUOwotCi0JaTkxNV9nZW1fb2JqZWN0X3Nl dF9jYWNoZV9jb2hlcmVuY3kob2JqLCBJOTE1X0NBQ0hFX05PTkUpOwotCi0JaTkxNV9nZW1fb2Jq ZWN0X2luaXRfbWVtb3J5X3JlZ2lvbihvYmosIG1lbSk7Ci0KLQlyZXR1cm4gMDsKLX0KZGlmZiAt LWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV9sbWVtLmggYi9kcml2ZXJz L2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fbG1lbS5oCmluZGV4IGZhYzZiYzVhNWViYi4uZWE3 NmZkMTFjY2IwIDEwMDY0NAotLS0gYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1f bG1lbS5oCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV9sbWVtLmgKQEAg LTI2LDkgKzI2LDQgQEAgaTkxNV9nZW1fb2JqZWN0X2NyZWF0ZV9sbWVtKHN0cnVjdCBkcm1faTkx NV9wcml2YXRlICppOTE1LAogCQkJICAgIHJlc291cmNlX3NpemVfdCBzaXplLAogCQkJICAgIHVu c2lnbmVkIGludCBmbGFncyk7CiAKLWludCBfX2k5MTVfZ2VtX2xtZW1fb2JqZWN0X2luaXQoc3Ry dWN0IGludGVsX21lbW9yeV9yZWdpb24gKm1lbSwKLQkJCQlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29i amVjdCAqb2JqLAotCQkJCXJlc291cmNlX3NpemVfdCBzaXplLAotCQkJCXVuc2lnbmVkIGludCBm bGFncyk7Ci0KICNlbmRpZiAvKiAhX19JOTE1X0dFTV9MTUVNX0ggKi8KZGlmZiAtLWdpdCBhL2Ry aXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV9vYmplY3QuYyBiL2RyaXZlcnMvZ3B1L2Ry bS9pOTE1L2dlbS9pOTE1X2dlbV9vYmplY3QuYwppbmRleCBjODk1M2UzZjVjNzAuLjg1ODA5OTYx MDdjZSAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX29iamVj dC5jCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV9vYmplY3QuYwpAQCAt MTcyLDcgKzE3Miw3IEBAIHN0YXRpYyB2b2lkIGk5MTVfZ2VtX2Nsb3NlX29iamVjdChzdHJ1Y3Qg ZHJtX2dlbV9vYmplY3QgKmdlbSwgc3RydWN0IGRybV9maWxlICpmCiAJfQogfQogCi1zdGF0aWMg dm9pZCBfX2k5MTVfZ2VtX2ZyZWVfb2JqZWN0X3JjdShzdHJ1Y3QgcmN1X2hlYWQgKmhlYWQpCit2 b2lkIF9faTkxNV9nZW1fZnJlZV9vYmplY3RfcmN1KHN0cnVjdCByY3VfaGVhZCAqaGVhZCkKIHsK IAlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqID0KIAkJY29udGFpbmVyX29mKGhlYWQs IHR5cGVvZigqb2JqKSwgcmN1KTsKQEAgLTIwOCw1OSArMjA4LDY5IEBAIHN0YXRpYyB2b2lkIF9f aTkxNV9nZW1fb2JqZWN0X2ZyZWVfbW1hcHMoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9i aikKIAl9CiB9CiAKLXN0YXRpYyB2b2lkIF9faTkxNV9nZW1fZnJlZV9vYmplY3RzKHN0cnVjdCBk cm1faTkxNV9wcml2YXRlICppOTE1LAotCQkJCSAgICBzdHJ1Y3QgbGxpc3Rfbm9kZSAqZnJlZWQp Cit2b2lkIF9faTkxNV9nZW1fZnJlZV9vYmplY3Qoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3Qg Km9iaikKIHsKLQlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqLCAqb247CisJdHJhY2Vf aTkxNV9nZW1fb2JqZWN0X2Rlc3Ryb3kob2JqKTsKIAotCWxsaXN0X2Zvcl9lYWNoX2VudHJ5X3Nh ZmUob2JqLCBvbiwgZnJlZWQsIGZyZWVkKSB7Ci0JCXRyYWNlX2k5MTVfZ2VtX29iamVjdF9kZXN0 cm95KG9iaik7CisJaWYgKCFsaXN0X2VtcHR5KCZvYmotPnZtYS5saXN0KSkgeworCQlzdHJ1Y3Qg aTkxNV92bWEgKnZtYTsKKworCQkvKgorCQkgKiBOb3RlIHRoYXQgdGhlIHZtYSBrZWVwcyBhbiBv YmplY3QgcmVmZXJlbmNlIHdoaWxlCisJCSAqIGl0IGlzIGFjdGl2ZSwgc28gaXQgKnNob3VsZCog bm90IHNsZWVwIHdoaWxlIHdlCisJCSAqIGRlc3Ryb3kgaXQuIE91ciBkZWJ1ZyBjb2RlIGVycnMg aW5zaXRzIGl0ICptaWdodCouCisJCSAqIEZvciB0aGUgbW9tZW50LCBwbGF5IGFsb25nLgorCQkg Ki8KKwkJc3Bpbl9sb2NrKCZvYmotPnZtYS5sb2NrKTsKKwkJd2hpbGUgKCh2bWEgPSBsaXN0X2Zp cnN0X2VudHJ5X29yX251bGwoJm9iai0+dm1hLmxpc3QsCisJCQkJCQkgICAgICAgc3RydWN0IGk5 MTVfdm1hLAorCQkJCQkJICAgICAgIG9ial9saW5rKSkpIHsKKwkJCUdFTV9CVUdfT04odm1hLT5v YmogIT0gb2JqKTsKKwkJCXNwaW5fdW5sb2NrKCZvYmotPnZtYS5sb2NrKTsKIAotCQlpZiAoIWxp c3RfZW1wdHkoJm9iai0+dm1hLmxpc3QpKSB7Ci0JCQlzdHJ1Y3QgaTkxNV92bWEgKnZtYTsKKwkJ CV9faTkxNV92bWFfcHV0KHZtYSk7CiAKLQkJCS8qCi0JCQkgKiBOb3RlIHRoYXQgdGhlIHZtYSBr ZWVwcyBhbiBvYmplY3QgcmVmZXJlbmNlIHdoaWxlCi0JCQkgKiBpdCBpcyBhY3RpdmUsIHNvIGl0 ICpzaG91bGQqIG5vdCBzbGVlcCB3aGlsZSB3ZQotCQkJICogZGVzdHJveSBpdC4gT3VyIGRlYnVn IGNvZGUgZXJycyBpbnNpdHMgaXQgKm1pZ2h0Ki4KLQkJCSAqIEZvciB0aGUgbW9tZW50LCBwbGF5 IGFsb25nLgotCQkJICovCiAJCQlzcGluX2xvY2soJm9iai0+dm1hLmxvY2spOwotCQkJd2hpbGUg KCh2bWEgPSBsaXN0X2ZpcnN0X2VudHJ5X29yX251bGwoJm9iai0+dm1hLmxpc3QsCi0JCQkJCQkJ ICAgICAgIHN0cnVjdCBpOTE1X3ZtYSwKLQkJCQkJCQkgICAgICAgb2JqX2xpbmspKSkgewotCQkJ CUdFTV9CVUdfT04odm1hLT5vYmogIT0gb2JqKTsKLQkJCQlzcGluX3VubG9jaygmb2JqLT52bWEu bG9jayk7CisJCX0KKwkJc3Bpbl91bmxvY2soJm9iai0+dm1hLmxvY2spOworCX0KIAotCQkJCV9f aTkxNV92bWFfcHV0KHZtYSk7CisJX19pOTE1X2dlbV9vYmplY3RfZnJlZV9tbWFwcyhvYmopOwog Ci0JCQkJc3Bpbl9sb2NrKCZvYmotPnZtYS5sb2NrKTsKLQkJCX0KLQkJCXNwaW5fdW5sb2NrKCZv YmotPnZtYS5sb2NrKTsKLQkJfQorCUdFTV9CVUdfT04oIWxpc3RfZW1wdHkoJm9iai0+bHV0X2xp c3QpKTsKIAotCQlfX2k5MTVfZ2VtX29iamVjdF9mcmVlX21tYXBzKG9iaik7CisJYXRvbWljX3Nl dCgmb2JqLT5tbS5wYWdlc19waW5fY291bnQsIDApOworCV9faTkxNV9nZW1fb2JqZWN0X3B1dF9w YWdlcyhvYmopOworCUdFTV9CVUdfT04oaTkxNV9nZW1fb2JqZWN0X2hhc19wYWdlcyhvYmopKTsK KwliaXRtYXBfZnJlZShvYmotPmJpdF8xNyk7CiAKLQkJR0VNX0JVR19PTighbGlzdF9lbXB0eSgm b2JqLT5sdXRfbGlzdCkpOworCWlmIChvYmotPmJhc2UuaW1wb3J0X2F0dGFjaCkKKwkJZHJtX3By aW1lX2dlbV9kZXN0cm95KCZvYmotPmJhc2UsIE5VTEwpOwogCi0JCWF0b21pY19zZXQoJm9iai0+ bW0ucGFnZXNfcGluX2NvdW50LCAwKTsKLQkJX19pOTE1X2dlbV9vYmplY3RfcHV0X3BhZ2VzKG9i aik7Ci0JCUdFTV9CVUdfT04oaTkxNV9nZW1fb2JqZWN0X2hhc19wYWdlcyhvYmopKTsKLQkJYml0 bWFwX2ZyZWUob2JqLT5iaXRfMTcpOworCWRybV9nZW1fZnJlZV9tbWFwX29mZnNldCgmb2JqLT5i YXNlKTsKIAotCQlpZiAob2JqLT5iYXNlLmltcG9ydF9hdHRhY2gpCi0JCQlkcm1fcHJpbWVfZ2Vt X2Rlc3Ryb3koJm9iai0+YmFzZSwgTlVMTCk7CisJaWYgKG9iai0+b3BzLT5yZWxlYXNlKQorCQlv YmotPm9wcy0+cmVsZWFzZShvYmopOwogCi0JCWRybV9nZW1fZnJlZV9tbWFwX29mZnNldCgmb2Jq LT5iYXNlKTsKKwlpZiAob2JqLT5tbS5uX3BsYWNlbWVudHMgPiAxKQorCQlrZnJlZShvYmotPm1t LnBsYWNlbWVudHMpOwogCi0JCWlmIChvYmotPm9wcy0+cmVsZWFzZSkKLQkJCW9iai0+b3BzLT5y ZWxlYXNlKG9iaik7CisJaWYgKG9iai0+cmVzdl9zaGFyZWRfZnJvbSkKKwkJaTkxNV92bV9yZXN2 X3B1dChvYmotPnJlc3Zfc2hhcmVkX2Zyb20pOworfQogCi0JCWlmIChvYmotPm1tLm5fcGxhY2Vt ZW50cyA+IDEpCi0JCQlrZnJlZShvYmotPm1tLnBsYWNlbWVudHMpOworc3RhdGljIHZvaWQgX19p OTE1X2dlbV9mcmVlX29iamVjdHMoc3RydWN0IGRybV9pOTE1X3ByaXZhdGUgKmk5MTUsCisJCQkJ ICAgIHN0cnVjdCBsbGlzdF9ub2RlICpmcmVlZCkKK3sKKwlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29i amVjdCAqb2JqLCAqb247CiAKLQkJaWYgKG9iai0+cmVzdl9zaGFyZWRfZnJvbSkKLQkJCWk5MTVf dm1fcmVzdl9wdXQob2JqLT5yZXN2X3NoYXJlZF9mcm9tKTsKKwlsbGlzdF9mb3JfZWFjaF9lbnRy eV9zYWZlKG9iaiwgb24sIGZyZWVkLCBmcmVlZCkgeworCQltaWdodF9zbGVlcCgpOworCQlpZiAo b2JqLT5vcHMtPmRlbGF5ZWRfZnJlZSkgeworCQkJb2JqLT5vcHMtPmRlbGF5ZWRfZnJlZShvYmop OworCQkJY29udGludWU7CisJCX0KKwkJX19pOTE1X2dlbV9mcmVlX29iamVjdChvYmopOwogCiAJ CS8qIEJ1dCBrZWVwIHRoZSBwb2ludGVyIGFsaXZlIGZvciBSQ1UtcHJvdGVjdGVkIGxvb2t1cHMg Ki8KIAkJY2FsbF9yY3UoJm9iai0+cmN1LCBfX2k5MTVfZ2VtX2ZyZWVfb2JqZWN0X3JjdSk7CkBA IC0zMTgsNiArMzI4LDcgQEAgc3RhdGljIHZvaWQgaTkxNV9nZW1fZnJlZV9vYmplY3Qoc3RydWN0 IGRybV9nZW1fb2JqZWN0ICpnZW1fb2JqKQogCSAqIHdvcmtlciBhbmQgcGVyZm9ybWluZyBmcmVl cyBkaXJlY3RseSBmcm9tIHN1YnNlcXVlbnQgYWxsb2NhdGlvbnMgZm9yCiAJICogY3J1ZGUgYnV0 IGVmZmVjdGl2ZSBtZW1vcnkgdGhyb3R0bGluZy4KIAkgKi8KKwogCWlmIChsbGlzdF9hZGQoJm9i ai0+ZnJlZWQsICZpOTE1LT5tbS5mcmVlX2xpc3QpKQogCQlxdWV1ZV93b3JrKGk5MTUtPndxLCAm aTkxNS0+bW0uZnJlZV93b3JrKTsKIH0KQEAgLTQxMCw2ICs0MjEsNDIgQEAgaW50IGk5MTVfZ2Vt X29iamVjdF9yZWFkX2Zyb21fcGFnZShzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqLCB1 NjQgb2Zmc2V0LAogCXJldHVybiAwOwogfQogCisvKioKKyAqIGk5MTVfZ2VtX29iamVjdF9ldmlj dGFibGUgLSBXaGV0aGVyIG9iamVjdCBpcyBsaWtlbHkgZXZpY3RhYmxlIGFmdGVyIHVuYmluZC4K KyAqIEBvYmo6IFRoZSBvYmplY3QgdG8gY2hlY2sKKyAqCisgKiBUaGlzIGZ1bmN0aW9uIGNoZWNr cyB3aGV0aGVyIHRoZSBvYmplY3QgaXMgbGlrZWx5IHVudmljdGFibGUgYWZ0ZXIgdW5iaW5kLgor ICogSWYgdGhlIG9iamVjdCBpcyBub3QgbG9ja2VkIHdoZW4gY2hlY2tpbmcsIHRoZSByZXN1bHQg aXMgb25seSBhZHZpc29yeS4KKyAqIElmIHRoZSBvYmplY3QgaXMgbG9ja2VkIHdoZW4gY2hlY2tp bmcsIGFuZCB0aGUgZnVuY3Rpb24gcmV0dXJucyB0cnVlLAorICogdGhlbiBhbiBldmljdGlvbiBz aG91bGQgaW5kZWVkIGJlIHBvc3NpYmxlLiBCdXQgc2luY2UgdW5sb2NrZWQgdm1hCisgKiB1bnBp bm5pbmcgYW5kIHVuYmluZGluZyBpcyBjdXJyZW50bHkgcG9zc2libGUsIHRoZSBvYmplY3QgY2Fu IGFjdHVhbGx5CisgKiBiZWNvbWUgZXZpY3RhYmxlIGV2ZW4gaWYgdGhpcyBmdW5jdGlvbiByZXR1 cm5zIGZhbHNlLgorICoKKyAqIFJldHVybjogdHJ1ZSBpZiB0aGUgb2JqZWN0IG1heSBiZSBldmlj dGFibGUuIEZhbHNlIG90aGVyd2lzZS4KKyAqLworYm9vbCBpOTE1X2dlbV9vYmplY3RfZXZpY3Rh YmxlKHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopCit7CisJc3RydWN0IGk5MTVfdm1h ICp2bWE7CisJaW50IHBpbl9jb3VudCA9IGF0b21pY19yZWFkKCZvYmotPm1tLnBhZ2VzX3Bpbl9j b3VudCk7CisKKwlpZiAoIXBpbl9jb3VudCkKKwkJcmV0dXJuIHRydWU7CisKKwlzcGluX2xvY2so Jm9iai0+dm1hLmxvY2spOworCWxpc3RfZm9yX2VhY2hfZW50cnkodm1hLCAmb2JqLT52bWEubGlz dCwgb2JqX2xpbmspIHsKKwkJaWYgKGk5MTVfdm1hX2lzX3Bpbm5lZCh2bWEpKSB7CisJCQlzcGlu X3VubG9jaygmb2JqLT52bWEubG9jayk7CisJCQlyZXR1cm4gZmFsc2U7CisJCX0KKwkJaWYgKGF0 b21pY19yZWFkKCZ2bWEtPnBhZ2VzX2NvdW50KSkKKwkJCXBpbl9jb3VudC0tOworCX0KKwlzcGlu X3VubG9jaygmb2JqLT52bWEubG9jayk7CisJR0VNX1dBUk5fT04ocGluX2NvdW50IDwgMCk7CisK KwlyZXR1cm4gcGluX2NvdW50ID09IDA7Cit9CisKIHZvaWQgaTkxNV9nZW1faW5pdF9fb2JqZWN0 cyhzdHJ1Y3QgZHJtX2k5MTVfcHJpdmF0ZSAqaTkxNSkKIHsKIAlJTklUX1dPUksoJmk5MTUtPm1t LmZyZWVfd29yaywgX19pOTE1X2dlbV9mcmVlX3dvcmspOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9n cHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX29iamVjdC5oIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUv Z2VtL2k5MTVfZ2VtX29iamVjdC5oCmluZGV4IDJlYmQ3OTUzN2FlYS4uYWU1OTMwZTMwN2Q1IDEw MDY0NAotLS0gYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fb2JqZWN0LmgKKysr IGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX29iamVjdC5oCkBAIC0yMDAsNiAr MjAwLDkgQEAgc3RhdGljIGlubGluZSBib29sIGk5MTVfZ2VtX29iamVjdF90cnlsb2NrKHN0cnVj dCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopCiAKIHN0YXRpYyBpbmxpbmUgdm9pZCBpOTE1X2dl bV9vYmplY3RfdW5sb2NrKHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopCiB7CisJaWYg KG9iai0+b3BzLT5hZGp1c3RfbHJ1KQorCQlvYmotPm9wcy0+YWRqdXN0X2xydShvYmopOworCiAJ ZG1hX3Jlc3ZfdW5sb2NrKG9iai0+YmFzZS5yZXN2KTsKIH0KIApAQCAtNTg3LDYgKzU5MCwxMiBA QCBpbnQgaTkxNV9nZW1fb2JqZWN0X3JlYWRfZnJvbV9wYWdlKHN0cnVjdCBkcm1faTkxNV9nZW1f b2JqZWN0ICpvYmosIHU2NCBvZmZzZXQsCiAKIGJvb2wgaTkxNV9nZW1fb2JqZWN0X2lzX3NobWVt KGNvbnN0IHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopOwogCit2b2lkIF9faTkxNV9n ZW1fZnJlZV9vYmplY3RfcmN1KHN0cnVjdCByY3VfaGVhZCAqaGVhZCk7CisKK3ZvaWQgX19pOTE1 X2dlbV9mcmVlX29iamVjdChzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqKTsKKworYm9v bCBpOTE1X2dlbV9vYmplY3RfZXZpY3RhYmxlKHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpv YmopOworCiAjaWZkZWYgQ09ORklHX01NVV9OT1RJRklFUgogc3RhdGljIGlubGluZSBib29sCiBp OTE1X2dlbV9vYmplY3RfaXNfdXNlcnB0cihzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2Jq KQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX29iamVjdF90 eXBlcy5oIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX29iamVjdF90eXBlcy5o CmluZGV4IDk4ZjY5ZDhmZDM3ZC4uYjM1MDc2NWUxOTM1IDEwMDY0NAotLS0gYS9kcml2ZXJzL2dw dS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fb2JqZWN0X3R5cGVzLmgKKysrIGIvZHJpdmVycy9ncHUv ZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX29iamVjdF90eXBlcy5oCkBAIC02Myw2ICs2MywyMCBAQCBz dHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVjdF9vcHMgewogCQkgICAgICBjb25zdCBzdHJ1Y3QgZHJt X2k5MTVfZ2VtX3B3cml0ZSAqYXJnKTsKIAogCWludCAoKmRtYWJ1Zl9leHBvcnQpKHN0cnVjdCBk cm1faTkxNV9nZW1fb2JqZWN0ICpvYmopOworCisJLyoqCisJICogYWRqdXN0X2xydSAtIG5vdGlm eSB0aGF0IHRoZSBtYWR2aXNlIHZhbHVlIHdhcyB1cGRhdGVkCisJICogQG9iajogVGhlIGdlbSBv YmplY3QKKwkgKgorCSAqIFRoZSBtYWR2aXNlIHZhbHVlIG1heSBoYXZlIGJlZW4gdXBkYXRlZCwg b3Igb2JqZWN0IHdhcyByZWNlbnRseQorCSAqIHJlZmVyZW5jZWQgc28gYWN0IGFjY29yZGluZ2x5 IChQZXJoYXBzIGNoYW5naW5nIGFuIExSVSBsaXN0IGV0YykuCisJICovCisJdm9pZCAoKmFkanVz dF9scnUpKHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopOworCisJLyoqCisJICogZGVs YXllZF9mcmVlIC0gT3ZlcnJpZGUgdGhlIGRlZmF1bHQgZGVsYXllZCBmcmVlIGltcGxlbWVudGF0 aW9uCisJICovCisJdm9pZCAoKmRlbGF5ZWRfZnJlZSkoc3RydWN0IGRybV9pOTE1X2dlbV9vYmpl Y3QgKm9iaik7CiAJdm9pZCAoKnJlbGVhc2UpKHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpv YmopOwogCiAJY29uc3QgY2hhciAqbmFtZTsgLyogZnJpZW5kbHkgbmFtZSBmb3IgZGVidWcsIGUu Zy4gbG9ja2RlcCBjbGFzc2VzICovCkBAIC0zMDcsNiArMzIxLDEwIEBAIHN0cnVjdCBkcm1faTkx NV9nZW1fb2JqZWN0IHsKIAkJYm9vbCBkaXJ0eToxOwogCX0gbW07CiAKKwlzdHJ1Y3QgeworCQlz dHJ1Y3Qgc2dfdGFibGUgKmNhY2hlZF9pb19zdDsKKwl9IHR0bTsKKwogCS8qKiBSZWNvcmQgb2Yg YWRkcmVzcyBiaXQgMTcgb2YgZWFjaCBwYWdlIGF0IGxhc3QgdW5iaW5kLiAqLwogCXVuc2lnbmVk IGxvbmcgKmJpdF8xNzsKIApkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5 MTVfZ2VtX3JlZ2lvbi5jIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ2VtL2k5MTVfZ2VtX3JlZ2lv bi5jCmluZGV4IGYyNWU2NjQ2YzViNy4uZDFmMTg0MDU0MGRkIDEwMDY0NAotLS0gYS9kcml2ZXJz L2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fcmVnaW9uLmMKKysrIGIvZHJpdmVycy9ncHUvZHJt L2k5MTUvZ2VtL2k5MTVfZ2VtX3JlZ2lvbi5jCkBAIC0xOCwxMSArMTgsNyBAQCB2b2lkIGk5MTVf Z2VtX29iamVjdF9pbml0X21lbW9yeV9yZWdpb24oc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3Qg Km9iaiwKIAogCW11dGV4X2xvY2soJm1lbS0+b2JqZWN0cy5sb2NrKTsKIAotCWlmIChvYmotPmZs YWdzICYgSTkxNV9CT19BTExPQ19WT0xBVElMRSkKLQkJbGlzdF9hZGQoJm9iai0+bW0ucmVnaW9u X2xpbmssICZtZW0tPm9iamVjdHMucHVyZ2VhYmxlKTsKLQllbHNlCi0JCWxpc3RfYWRkKCZvYmot Pm1tLnJlZ2lvbl9saW5rLCAmbWVtLT5vYmplY3RzLmxpc3QpOwotCisJbGlzdF9hZGQoJm9iai0+ bW0ucmVnaW9uX2xpbmssICZtZW0tPm9iamVjdHMubGlzdCk7CiAJbXV0ZXhfdW5sb2NrKCZtZW0t Pm9iamVjdHMubG9jayk7CiB9CiAKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dl bS9pOTE1X2dlbV90dG0uYyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG0u YwpuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAwMDAuLjc5MGY1ZWM0NWM0ZAot LS0gL2Rldi9udWxsCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2dlbS9pOTE1X2dlbV90dG0u YwpAQCAtMCwwICsxLDUxOSBAQAorLy8gU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IE1JVAorLyoK KyAqIENvcHlyaWdodCDCqSAyMDIxIEludGVsIENvcnBvcmF0aW9uCisgKi8KKworI2luY2x1ZGUg PGRybS90dG0vdHRtX2JvX2RyaXZlci5oPgorI2luY2x1ZGUgPGRybS90dG0vdHRtX3BsYWNlbWVu dC5oPgorCisjaW5jbHVkZSAiaTkxNV9kcnYuaCIKKyNpbmNsdWRlICJpbnRlbF9tZW1vcnlfcmVn aW9uLmgiCisjaW5jbHVkZSAiaW50ZWxfcmVnaW9uX3R0bS5oIgorCisjaW5jbHVkZSAiZ2VtL2k5 MTVfZ2VtX29iamVjdC5oIgorI2luY2x1ZGUgImdlbS9pOTE1X2dlbV9yZWdpb24uaCIKKyNpbmNs dWRlICJnZW0vaTkxNV9nZW1fdHRtLmgiCisjaW5jbHVkZSAiZ2VtL2k5MTVfZ2VtX3R0bV9ib191 dGlsLmgiCisKKyNkZWZpbmUgSTkxNV9QTF9MTUVNMCBUVE1fUExfUFJJVgorI2RlZmluZSBJOTE1 X1BMX1NZU1RFTSBUVE1fUExfU1lTVEVNCisjZGVmaW5lIEk5MTVfUExfU1RPTEVOIFRUTV9QTF9W UkFNCisjZGVmaW5lIEk5MTVfUExfR0dUVCBUVE1fUExfVFQKKworI2RlZmluZSBJOTE1X1RUTV9Q UklPX1BVUkdFICAgICAwCisjZGVmaW5lIEk5MTVfVFRNX1BSSU9fTk9fUEFHRVMgIDEKKyNkZWZp bmUgSTkxNV9UVE1fUFJJT19IQVNfUEFHRVMgMgorCisvKioKKyAqIHN0cnVjdCBpOTE1X3R0bV90 dCAtIFRUTSBwYWdlIHZlY3RvciB3aXRoIGFkZGl0aW9uYWwgcHJpdmF0ZSBpbmZvcm1hdGlvbgor ICogQHR0bTogVGhlIGJhc2UgVFRNIHBhZ2UgdmVjdG9yLgorICogQGRldjogVGhlIHN0cnVjdCBk ZXZpY2UgdXNlZCBmb3IgZG1hIG1hcHBpbmcgYW5kIHVubWFwcGluZy4KKyAqIEBjYWNoZWRfc3Q6 IFRoZSBjYWNoZWQgc2NhdHRlci1nYXRoZXIgdGFibGUuCisgKgorICogTm90ZSB0aGF0IERNQSBt YXkgYmUgZ29pbmcgb24gcmlnaHQgdXAgdG8gdGhlIHBvaW50IHdoZXJlIHRoZSBwYWdlLQorICog dmVjdG9yIGlzIHVucG9wdWxhdGVkIGluIGRlbGF5ZWQgZGVzdHJveS4gSGVuY2Uga2VlcCB0aGUK KyAqIHNjYXR0ZXItZ2F0aGVyIHRhYmxlIG1hcHBlZCBhbmQgY2FjaGVkIHVwIHRvIHRoYXQgcG9p bnQuIFRoaXMgaXMKKyAqIGRpZmZlcmVudCBmcm9tIHRoZSBjYWNoZWQgZ2VtIG9iamVjdCBpbyBz Y2F0dGVyLWdhdGhlciB0YWJsZSB3aGljaAorICogZG9lc24ndCBoYXZlIGFuIGFzc29jaWF0ZWQg ZG1hIG1hcHBpbmcuCisgKi8KK3N0cnVjdCBpOTE1X3R0bV90dCB7CisJc3RydWN0IHR0bV90dCB0 dG07CisJc3RydWN0IGRldmljZSAqZGV2OworCXN0cnVjdCBzZ190YWJsZSAqY2FjaGVkX3N0Owor fTsKKworc3RhdGljIGNvbnN0IHN0cnVjdCB0dG1fcGxhY2UgbG1lbTBfc3lzX3BsYWNlbWVudF9m bGFnc1tdID0geworCXsKKwkJLmZwZm4gPSAwLAorCQkubHBmbiA9IDAsCisJCS5tZW1fdHlwZSA9 IEk5MTVfUExfTE1FTTAsCisJCS5mbGFncyA9IDAsCisJfSwgeworCQkuZnBmbiA9IDAsCisJCS5s cGZuID0gMCwKKwkJLm1lbV90eXBlID0gSTkxNV9QTF9TWVNURU0sCisJCS5mbGFncyA9IDAsCisJ fQorfTsKKworc3RydWN0IHR0bV9wbGFjZW1lbnQgaTkxNV9sbWVtMF9wbGFjZW1lbnQgPSB7CisJ Lm51bV9wbGFjZW1lbnQgPSAxLAorCS5wbGFjZW1lbnQgPSAmbG1lbTBfc3lzX3BsYWNlbWVudF9m bGFnc1swXSwKKwkubnVtX2J1c3lfcGxhY2VtZW50ID0gMSwKKwkuYnVzeV9wbGFjZW1lbnQgPSAm bG1lbTBfc3lzX3BsYWNlbWVudF9mbGFnc1swXSwKK307CisKK3N0cnVjdCB0dG1fcGxhY2VtZW50 IGk5MTVfbG1lbTBfc3lzX3BsYWNlbWVudCA9IHsKKwkubnVtX3BsYWNlbWVudCA9IDEsCisJLnBs YWNlbWVudCA9ICZsbWVtMF9zeXNfcGxhY2VtZW50X2ZsYWdzWzBdLAorCS5udW1fYnVzeV9wbGFj ZW1lbnQgPSAyLAorCS5idXN5X3BsYWNlbWVudCA9ICZsbWVtMF9zeXNfcGxhY2VtZW50X2ZsYWdz WzBdLAorfTsKKworc3RydWN0IHR0bV9wbGFjZW1lbnQgaTkxNV9zeXNfcGxhY2VtZW50ID0gewor CS5udW1fcGxhY2VtZW50ID0gMSwKKwkucGxhY2VtZW50ID0gJmxtZW0wX3N5c19wbGFjZW1lbnRf ZmxhZ3NbMV0sCisJLm51bV9idXN5X3BsYWNlbWVudCA9IDEsCisJLmJ1c3lfcGxhY2VtZW50ID0g JmxtZW0wX3N5c19wbGFjZW1lbnRfZmxhZ3NbMV0sCit9OworCitzdGF0aWMgdm9pZCBpOTE1X3R0 bV9hZGp1c3RfbHJ1KHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopOworCitzdGF0aWMg c3RydWN0IHR0bV90dCAqaTkxNV90dG1fdHRfY3JlYXRlKHN0cnVjdCB0dG1fYnVmZmVyX29iamVj dCAqYm8sCisJCQkJCSB1aW50MzJfdCBwYWdlX2ZsYWdzKQoreworCXN0cnVjdCB0dG1fcmVzb3Vy Y2VfbWFuYWdlciAqbWFuID0KKwkJdHRtX21hbmFnZXJfdHlwZShiby0+YmRldiwgYm8tPm1lbS5t ZW1fdHlwZSk7CisJc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaiA9IGk5MTVfdHRtX3Rv X2dlbShibyk7CisJc3RydWN0IGk5MTVfdHRtX3R0ICppOTE1X3R0OworCWludCByZXQ7CisKKwlp OTE1X3R0ID0ga3phbGxvYyhzaXplb2YoKmk5MTVfdHQpLCBHRlBfS0VSTkVMKTsKKwlpZiAoIWk5 MTVfdHQpCisJCXJldHVybiBOVUxMOworCisJaWYgKG9iai0+ZmxhZ3MgJiBJOTE1X0JPX0FMTE9D X0NQVV9DTEVBUiAmJgorCSAgICBtYW4tPnVzZV90dCkKKwkJcGFnZV9mbGFncyB8PSBUVE1fUEFH RV9GTEFHX1pFUk9fQUxMT0M7CisKKwlyZXQgPSB0dG1fdHRfaW5pdCgmaTkxNV90dC0+dHRtLCBi bywgcGFnZV9mbGFncywgdHRtX2NhY2hlZCk7CisJaWYgKHJldCkgeworCQlrZnJlZShpOTE1X3R0 KTsKKwkJcmV0dXJuIE5VTEw7CisJfQorCisJaTkxNV90dC0+ZGV2ID0gb2JqLT5iYXNlLmRldi0+ ZGV2OworCisJcmV0dXJuICZpOTE1X3R0LT50dG07Cit9CisKK3N0YXRpYyB2b2lkIGk5MTVfdHRt X3R0X3VucG9wdWxhdGUoc3RydWN0IHR0bV9kZXZpY2UgKmJkZXYsIHN0cnVjdCB0dG1fdHQgKnR0 bSkKK3sKKwlzdHJ1Y3QgaTkxNV90dG1fdHQgKmk5MTVfdHQgPSBjb250YWluZXJfb2YodHRtLCB0 eXBlb2YoKmk5MTVfdHQpLCB0dG0pOworCisJaWYgKGk5MTVfdHQtPmNhY2hlZF9zdCkgeworCQlk bWFfdW5tYXBfc2d0YWJsZShpOTE1X3R0LT5kZXYsIGk5MTVfdHQtPmNhY2hlZF9zdCwKKwkJCQkg IERNQV9CSURJUkVDVElPTkFMLCAwKTsKKwkJc2dfZnJlZV90YWJsZShpOTE1X3R0LT5jYWNoZWRf c3QpOworCQlrZnJlZShpOTE1X3R0LT5jYWNoZWRfc3QpOworCQlpOTE1X3R0LT5jYWNoZWRfc3Qg PSBOVUxMOworCX0KKwl0dG1fcG9vbF9mcmVlKCZiZGV2LT5wb29sLCB0dG0pOworfQorCitzdGF0 aWMgdm9pZCBpOTE1X3R0bV90dF9kZXN0cm95KHN0cnVjdCB0dG1fZGV2aWNlICpiZGV2LCBzdHJ1 Y3QgdHRtX3R0ICp0dG0pCit7CisJc3RydWN0IGk5MTVfdHRtX3R0ICppOTE1X3R0ID0gY29udGFp bmVyX29mKHR0bSwgdHlwZW9mKCppOTE1X3R0KSwgdHRtKTsKKworCXR0bV90dF9kZXN0cm95X2Nv bW1vbihiZGV2LCB0dG0pOworCWtmcmVlKGk5MTVfdHQpOworfQorCitzdGF0aWMgYm9vbCBpOTE1 X3R0bV9ldmljdGlvbl92YWx1YWJsZShzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvLAorCQkJ CSAgICAgICBjb25zdCBzdHJ1Y3QgdHRtX3BsYWNlICpwbGFjZSkKK3sKKwlzdHJ1Y3QgZHJtX2k5 MTVfZ2VtX29iamVjdCAqb2JqID0gaTkxNV90dG1fdG9fZ2VtKGJvKTsKKworCS8qIFdpbGwgZG8g Zm9yIG5vdy4gT3VyIHBpbm5lZCBvYmplY3RzIGFyZSBzdGlsbCBvbiBUVE0ncyBMUlUgbGlzdHMg Ki8KKwlpZiAoIWk5MTVfZ2VtX29iamVjdF9ldmljdGFibGUob2JqKSkKKwkJcmV0dXJuIGZhbHNl OworCisJLyogVGhpcyBpc24ndCB2YWxpZCB3aXRoIGEgYnVkZHkgYWxsb2NhdG9yICovCisJcmV0 dXJuIHR0bV9ib19ldmljdGlvbl92YWx1YWJsZShibywgcGxhY2UpOworfQorCitzdGF0aWMgdm9p ZCBpOTE1X3R0bV9ldmljdF9mbGFncyhzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvLAorCQkJ CSBzdHJ1Y3QgdHRtX3BsYWNlbWVudCAqcGxhY2VtZW50KQoreworCSpwbGFjZW1lbnQgPSBpOTE1 X3N5c19wbGFjZW1lbnQ7Cit9CisKK3N0YXRpYyBpbnQgaTkxNV90dG1fbW92ZV9ub3RpZnkoc3Ry dWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibykKK3sKKwlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVj dCAqb2JqID0gaTkxNV90dG1fdG9fZ2VtKGJvKTsKKwlpbnQgcmV0OworCisJcmV0ID0gaTkxNV9n ZW1fb2JqZWN0X3VuYmluZChvYmosIEk5MTVfR0VNX09CSkVDVF9VTkJJTkRfQUNUSVZFKTsKKwlp ZiAocmV0KQorCQlyZXR1cm4gcmV0OworCisJcmV0ID0gX19pOTE1X2dlbV9vYmplY3RfcHV0X3Bh Z2VzKG9iaik7CisJaWYgKHJldCkKKwkJcmV0dXJuIHJldDsKKworCXJldHVybiAwOworfQorCitz dGF0aWMgdm9pZCBpOTE1X3R0bV9mcmVlX2NhY2hlZF9pb19zdChzdHJ1Y3QgZHJtX2k5MTVfZ2Vt X29iamVjdCAqb2JqKQoreworCWlmIChvYmotPnR0bS5jYWNoZWRfaW9fc3QpIHsKKwkJc2dfZnJl ZV90YWJsZShvYmotPnR0bS5jYWNoZWRfaW9fc3QpOworCQlrZnJlZShvYmotPnR0bS5jYWNoZWRf aW9fc3QpOworCQlvYmotPnR0bS5jYWNoZWRfaW9fc3QgPSBOVUxMOworCX0KK30KKworc3RhdGlj IHZvaWQgaTkxNV90dG1fcHVyZ2Uoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaikKK3sK KwlzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvID0gaTkxNV9nZW1fdG9fdHRtKG9iaik7CisK KwlpZiAob2JqLT5tbS5tYWR2ID09IF9fSTkxNV9NQURWX1BVUkdFRCkKKwkJcmV0dXJuOworCisJ aTkxNV90dG1fZnJlZV9jYWNoZWRfaW9fc3Qob2JqKTsKKworCXR0bV9yZXNvdXJjZV9mcmVlKGJv LCAmYm8tPm1lbSk7CisJaWYgKGJvLT50dG0pIHsKKwkJdHRtX3R0X3VucG9wdWxhdGUoYm8tPmJk ZXYsIGJvLT50dG0pOworCQliby0+dHRtLT5wYWdlX2ZsYWdzIHw9IFRUTV9QQUdFX0ZMQUdfWkVS T19BTExPQzsKKwl9CisKKwlvYmotPm1tLm1hZHYgPSBfX0k5MTVfTUFEVl9QVVJHRUQ7Cit9CisK K3N0YXRpYyB2b2lkIGk5MTVfdHRtX3N3YXBfbm90aWZ5KHN0cnVjdCB0dG1fYnVmZmVyX29iamVj dCAqYm8pCit7CisJc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaiA9IGk5MTVfdHRtX3Rv X2dlbShibyk7CisJaW50IHJldCA9IGk5MTVfdHRtX21vdmVfbm90aWZ5KGJvKTsKKworCUdFTV9X QVJOX09OKHJldCk7CisJR0VNX1dBUk5fT04ob2JqLT50dG0uY2FjaGVkX2lvX3N0KTsKKwlpZiAo IXJldCAmJiBvYmotPm1tLm1hZHYgIT0gSTkxNV9NQURWX1dJTExORUVEKQorCQlpOTE1X3R0bV9w dXJnZShvYmopOworfQorCitzdGF0aWMgdm9pZCBpOTE1X3R0bV9kZWxldGVfbWVtX25vdGlmeShz dHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvKQoreworCXN0cnVjdCBkcm1faTkxNV9nZW1fb2Jq ZWN0ICpvYmogPSBpOTE1X3R0bV90b19nZW0oYm8pOworCisJaWYgKGxpa2VseShvYmopKSB7CisJ CS8qIFRoaXMgcmVsZWFzZXMgYWxsIGdlbSBvYmplY3QgYmluZGluZ3MgdG8gdGhlIGJhY2tlbmQu ICovCisJCV9faTkxNV9nZW1fZnJlZV9vYmplY3Qob2JqKTsKKwl9Cit9CisKK3N0YXRpYyBzdHJ1 Y3QgaW50ZWxfbWVtb3J5X3JlZ2lvbiAqCitpOTE1X3R0bV9yZWdpb24oc3RydWN0IHR0bV9kZXZp Y2UgKmJkZXYsIGludCB0dG1fbWVtX3R5cGUpCit7CisJc3RydWN0IGRybV9pOTE1X3ByaXZhdGUg Kmk5MTUgPSBjb250YWluZXJfb2YoYmRldiwgdHlwZW9mKCppOTE1KSwgYmRldik7CisKKwkvKiBU aGVyZSdzIHNvbWUgcm9vbSBmb3Igb3B0aW1pemF0aW9uIGhlcmUuLi4gKi8KKwlHRU1fQlVHX09O KHR0bV9tZW1fdHlwZSAhPSBJOTE1X1BMX1NZU1RFTSAmJgorCQkgICB0dG1fbWVtX3R5cGUgPCBJ OTE1X1BMX0xNRU0wKTsKKwlpZiAodHRtX21lbV90eXBlID09IEk5MTVfUExfU1lTVEVNKQorCQly ZXR1cm4gaW50ZWxfbWVtb3J5X3JlZ2lvbl9sb29rdXAoaTkxNSwgSU5URUxfTUVNT1JZX1NZU1RF TSwKKwkJCQkJCSAgMCk7CisKKwlyZXR1cm4gaW50ZWxfbWVtb3J5X3JlZ2lvbl9sb29rdXAoaTkx NSwgSU5URUxfTUVNT1JZX0xPQ0FMLAorCQkJCQkgIHR0bV9tZW1fdHlwZSAtIEk5MTVfUExfTE1F TTApOworfQorCitzdGF0aWMgc3RydWN0IHNnX3RhYmxlICppOTE1X3R0bV90dF9nZXRfc3Qoc3Ry dWN0IHR0bV90dCAqdHRtKQoreworCXN0cnVjdCBpOTE1X3R0bV90dCAqaTkxNV90dCA9IGNvbnRh aW5lcl9vZih0dG0sIHR5cGVvZigqaTkxNV90dCksIHR0bSk7CisJc3RydWN0IHNjYXR0ZXJsaXN0 ICpzZzsKKwlzdHJ1Y3Qgc2dfdGFibGUgKnN0OworCWludCByZXQ7CisKKwlpZiAoaTkxNV90dC0+ Y2FjaGVkX3N0KQorCQlyZXR1cm4gaTkxNV90dC0+Y2FjaGVkX3N0OworCisJc3QgPSBremFsbG9j KHNpemVvZigqc3QpLCBHRlBfS0VSTkVMKTsKKwlpZiAoIXN0KQorCQlyZXR1cm4gRVJSX1BUUigt RU5PTUVNKTsKKworCXNnID0gX19zZ19hbGxvY190YWJsZV9mcm9tX3BhZ2VzCisJCShzdCwgdHRt LT5wYWdlcywgdHRtLT5udW1fcGFnZXMsIDAsCisJCSAodW5zaWduZWQgbG9uZyl0dG0tPm51bV9w YWdlcyA8PCBQQUdFX1NISUZULAorCQkgaTkxNV9zZ19zZWdtZW50X3NpemUoKSwgTlVMTCwgMCwg R0ZQX0tFUk5FTCk7CisJaWYgKElTX0VSUihzZykpIHsKKwkJa2ZyZWUoc3QpOworCQlyZXR1cm4g RVJSX0NBU1Qoc2cpOworCX0KKworCXJldCA9IGRtYV9tYXBfc2d0YWJsZShpOTE1X3R0LT5kZXYs IHN0LCBETUFfQklESVJFQ1RJT05BTCwgMCk7CisJaWYgKHJldCkgeworCQlzZ19mcmVlX3RhYmxl KHN0KTsKKwkJa2ZyZWUoc3QpOworCQlyZXR1cm4gRVJSX1BUUihyZXQpOworCX0KKworCWk5MTVf dHQtPmNhY2hlZF9zdCA9IHN0OworCXJldHVybiBzdDsKK30KKworc3RhdGljIHN0cnVjdCBzZ190 YWJsZSAqCitpOTE1X3R0bV9yZXNvdXJjZV9nZXRfc3Qoc3RydWN0IGRybV9pOTE1X2dlbV9vYmpl Y3QgKm9iaiwKKwkJCSBzdHJ1Y3QgdHRtX3Jlc291cmNlICpyZXMpCit7CisJc3RydWN0IHR0bV9i dWZmZXJfb2JqZWN0ICpibyA9IGk5MTVfZ2VtX3RvX3R0bShvYmopOworCXN0cnVjdCB0dG1fcmVz b3VyY2VfbWFuYWdlciAqbWFuID0KKwkJdHRtX21hbmFnZXJfdHlwZShiby0+YmRldiwgcmVzLT5t ZW1fdHlwZSk7CisKKwlpZiAobWFuLT51c2VfdHQpCisJCXJldHVybiBpOTE1X3R0bV90dF9nZXRf c3QoYm8tPnR0bSk7CisKKwlyZXR1cm4gaW50ZWxfcmVnaW9uX3R0bV9ub2RlX3RvX3N0KG9iai0+ bW0ucmVnaW9uLCByZXMtPm1tX25vZGUpOworfQorCitzdGF0aWMgaW50IGk5MTVfdHRtX21vdmUo c3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibywgYm9vbCBldmljdCwKKwkJCSBzdHJ1Y3QgdHRt X29wZXJhdGlvbl9jdHggKmN0eCwKKwkJCSBzdHJ1Y3QgdHRtX3Jlc291cmNlICpuZXdfbWVtLAor CQkJIHN0cnVjdCB0dG1fcGxhY2UgKmhvcCkKK3sKKwlzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVj dCAqb2JqID0gaTkxNV90dG1fdG9fZ2VtKGJvKTsKKwlzdHJ1Y3QgdHRtX3Jlc291cmNlX21hbmFn ZXIgKm5ld19tYW4gPQorCQl0dG1fbWFuYWdlcl90eXBlKGJvLT5iZGV2LCBuZXdfbWVtLT5tZW1f dHlwZSk7CisJc3RydWN0IHR0bV9yZXNvdXJjZV9tYW5hZ2VyICpvbGRfbWFuID0KKwkJdHRtX21h bmFnZXJfdHlwZShiby0+YmRldiwgYm8tPm1lbS5tZW1fdHlwZSk7CisJc3RydWN0IGludGVsX21l bW9yeV9yZWdpb24gKm5ld19yZWcsICpvbGRfcmVnOworCXVuaW9uIHsKKwkJc3RydWN0IGk5MTVf dHRtX2ttYXBfaXRlcl90dCB0dDsKKwkJc3RydWN0IGk5MTVfdHRtX2ttYXBfaXRlcl9pb21hcCBp bzsKKwl9IF9uZXdfaXRlciwgX29sZF9pdGVyOworCXN0cnVjdCBpOTE1X3R0bV9rbWFwX2l0ZXIg Km5ld19pdGVyLCAqb2xkX2l0ZXI7CisJc3RydWN0IHNnX3RhYmxlICpuZXdfc3Q7CisJaW50IHJl dDsKKworCW5ld19yZWcgPSBpOTE1X3R0bV9yZWdpb24oYm8tPmJkZXYsIG5ld19tZW0tPm1lbV90 eXBlKTsKKwlvbGRfcmVnID0gaTkxNV90dG1fcmVnaW9uKGJvLT5iZGV2LCBiby0+bWVtLm1lbV90 eXBlKTsKKwlHRU1fQlVHX09OKCFuZXdfcmVnIHx8ICFvbGRfcmVnKTsKKworCS8qIFN5bmMgZm9y IG5vdy4gV2UgY291bGQgZG8gdGhlIGFjdHVhbCBjb3B5IGFzeW5jLiAqLworCXJldCA9IHR0bV9i b193YWl0X2N0eChibywgY3R4KTsKKwlpZiAocmV0KQorCQlyZXR1cm4gcmV0OworCisJcmV0ID0g aTkxNV90dG1fbW92ZV9ub3RpZnkoYm8pOworCWlmIChyZXQpCisJCXJldHVybiByZXQ7CisKKwlp ZiAob2JqLT5tbS5tYWR2ICE9IEk5MTVfTUFEVl9XSUxMTkVFRCkgeworCQlpOTE1X3R0bV9wdXJn ZShvYmopOworCQl0dG1fcmVzb3VyY2VfZnJlZShibywgbmV3X21lbSk7CisJCXJldHVybiAwOwor CX0KKworCS8qIFBvcHVsYXRlIHR0bSB3aXRoIHBhZ2VzIGlmIG5lZWRlZC4gVHlwaWNhbGx5IHN5 c3RlbSBtZW1vcnkuICovCisJaWYgKG5ld19tYW4tPnVzZV90dCAmJiBiby0+dHRtKSB7CisJCXJl dCA9IHR0bV90dF9wb3B1bGF0ZShiby0+YmRldiwgYm8tPnR0bSwgY3R4KTsKKwkJaWYgKHJldCkK KwkJCXJldHVybiByZXQ7CisJfQorCisJbmV3X3N0ID0gaTkxNV90dG1fcmVzb3VyY2VfZ2V0X3N0 KG9iaiwgbmV3X21lbSk7CisJaWYgKElTX0VSUihuZXdfc3QpKQorCQlyZXR1cm4gUFRSX0VSUihu ZXdfc3QpOworCisJbmV3X2l0ZXIgPSBuZXdfbWFuLT51c2VfdHQgPworCQlpOTE1X3R0bV9rbWFw X2l0ZXJfdHRfaW5pdCgmX25ld19pdGVyLnR0LCBiby0+dHRtKSA6CisJCWk5MTVfdHRtX2ttYXBf aXRlcl9pb21hcF9pbml0KCZfbmV3X2l0ZXIuaW8sICZuZXdfcmVnLT5pb21hcCwKKwkJCQkJICAg ICAgbmV3X3N0LCBuZXdfcmVnLT5yZWdpb24uc3RhcnQpOworCisJb2xkX2l0ZXIgPSBvbGRfbWFu LT51c2VfdHQgPworCQlpOTE1X3R0bV9rbWFwX2l0ZXJfdHRfaW5pdCgmX29sZF9pdGVyLnR0LCBi by0+dHRtKSA6CisJCWk5MTVfdHRtX2ttYXBfaXRlcl9pb21hcF9pbml0KCZfb2xkX2l0ZXIuaW8s ICZvbGRfcmVnLT5pb21hcCwKKwkJCQkJICAgICAgb2JqLT50dG0uY2FjaGVkX2lvX3N0LAorCQkJ CQkgICAgICBvbGRfcmVnLT5yZWdpb24uc3RhcnQpOworCisJaTkxNV90dG1fbW92ZV9tZW1jcHko Ym8sIG5ld19tZW0sIG5ld19pdGVyLCBvbGRfaXRlcik7CisJaTkxNV90dG1fZnJlZV9jYWNoZWRf aW9fc3Qob2JqKTsKKworCWlmICghbmV3X21hbi0+dXNlX3R0KQorCQlvYmotPnR0bS5jYWNoZWRf aW9fc3QgPSBuZXdfc3Q7CisKKwlyZXR1cm4gMDsKK30KKworc3RydWN0IHR0bV9kZXZpY2VfZnVu Y3MgaTkxNV90dG1fYm9fZHJpdmVyID0geworCS50dG1fdHRfY3JlYXRlID0gaTkxNV90dG1fdHRf Y3JlYXRlLAorCS50dG1fdHRfdW5wb3B1bGF0ZSA9IGk5MTVfdHRtX3R0X3VucG9wdWxhdGUsCisJ LnR0bV90dF9kZXN0cm95ID0gaTkxNV90dG1fdHRfZGVzdHJveSwKKwkuZXZpY3Rpb25fdmFsdWFi bGUgPSBpOTE1X3R0bV9ldmljdGlvbl92YWx1YWJsZSwKKwkuZXZpY3RfZmxhZ3MgPSBpOTE1X3R0 bV9ldmljdF9mbGFncywKKwkubW92ZSA9IGk5MTVfdHRtX21vdmUsCisJLnZlcmlmeV9hY2Nlc3Mg PSBOVUxMLAorCS5zd2FwX25vdGlmeSA9IGk5MTVfdHRtX3N3YXBfbm90aWZ5LAorCS5kZWxldGVf bWVtX25vdGlmeSA9IGk5MTVfdHRtX2RlbGV0ZV9tZW1fbm90aWZ5LAorfTsKKworc3RhdGljIGlu dCBpOTE1X3R0bV9nZXRfcGFnZXMoc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaikKK3sK KwlzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJvID0gaTkxNV9nZW1fdG9fdHRtKG9iaik7CisJ c3RydWN0IHR0bV9vcGVyYXRpb25fY3R4IGN0eCA9IHsKKwkJLmludGVycnVwdGlibGUgPSB0cnVl LAorCQkubm9fd2FpdF9ncHUgPSBmYWxzZSwKKwl9OworCXN0cnVjdCBzZ190YWJsZSAqc3Q7CisJ aW50IHJldDsKKworCS8qIFN3YXAgaW4uICovCisJaWYgKGJvLT50dG0gJiYgKGJvLT50dG0tPnBh Z2VfZmxhZ3MgJiBUVE1fUEFHRV9GTEFHX1NXQVBQRUQpKSB7CisJCXJldCA9IHR0bV90dF9wb3B1 bGF0ZShiby0+YmRldiwgYm8tPnR0bSwgJmN0eCk7CisJCWlmIChyZXQpCisJCQlyZXR1cm4gcmV0 OworCX0KKworCS8qIE1vdmUgdG8gdGhlIHJlcXVlc3RlZCBwbGFjZW1lbnQuICovCisJcmV0ID0g dHRtX2JvX3ZhbGlkYXRlKGJvLCAmaTkxNV9sbWVtMF9wbGFjZW1lbnQsICZjdHgpOworCWlmIChy ZXQpCisJCXJldHVybiByZXQgPT0gLUVOT1NQQyA/IC1FTlhJTyA6IHJldDsKKworCS8qIE9iamVj dCBlaXRoZXIgaGFzIGEgcGFnZSB2ZWN0b3Igb3IgaXMgYW4gaW9tZW0gb2JqZWN0ICovCisJc3Qg PSBiby0+dHRtID8gaTkxNV90dG1fdHRfZ2V0X3N0KGJvLT50dG0pIDogb2JqLT50dG0uY2FjaGVk X2lvX3N0OworCWlmIChJU19FUlIoc3QpKQorCQlyZXR1cm4gUFRSX0VSUihzdCk7CisKKwlfX2k5 MTVfZ2VtX29iamVjdF9zZXRfcGFnZXMob2JqLCBzdCwgaTkxNV9zZ19kbWFfc2l6ZXMoc3QtPnNn bCkpOworCisJaTkxNV90dG1fYWRqdXN0X2xydShvYmopOworCisJcmV0dXJuIHJldDsKK30KKwor c3RhdGljIHZvaWQgaTkxNV90dG1fcHV0X3BhZ2VzKHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0 ICpvYmosCisJCQkgICAgICAgc3RydWN0IHNnX3RhYmxlICpzdCkKK3sKKwkvKgorCSAqIFdlJ3Jl IGN1cnJlbnRseSBub3QgY2FsbGVkIGZyb20gYSBzaHJpbmtlciwgc28gcHV0X3BhZ2VzKCkKKwkg KiB0eXBpY2FsbHkgbWVhbnMgdGhlIG9iamVjdCBpcyBhYm91dCB0byBkZXN0cm95ZWQsIG9yIGNh bGxlZAorCSAqIGZyb20gbW92ZV9ub3RpZnkoKS4gU28ganVzdCBhdm9pZCBkb2luZyBtdWNoIGZv ciBub3cuCisJICogSWYgdGhlIG9iamVjdCBpcyBub3QgZGVzdHJveWVkIG5leHQsIFRoZSBUVE0g ZXZpY3Rpb24gbG9naWMKKwkgKiBhbmQgc2hyaW5rZXJzIHdpbGwgbW92ZSBpdCBvdXQgaWYgbmVl ZGVkLgorCSAqLworCisJaTkxNV90dG1fYWRqdXN0X2xydShvYmopOworfQorCitzdGF0aWMgdm9p ZCBpOTE1X3R0bV9hZGp1c3RfbHJ1KHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopCit7 CisJc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICpibyA9IGk5MTVfZ2VtX3RvX3R0bShvYmopOwor CisJLyoKKwkgKiBEb24ndCBtYW5pcHVsYXRlIHRoZSBUVE0gTFJVcyB3aGlsZSBpbiBUVE0gYm8g ZGVzdHJ1Y3Rpb24uCisJICogV2UncmUgY2FsbGVkIHRocm91Z2ggaTkxNV90dG1fZGVsZXRlX21l bV9ub3RpZnkoKS4KKwkgKi8KKwlpZiAoIWtyZWZfcmVhZCgmYm8tPmtyZWYpKQorCQlyZXR1cm47 CisKKwkvKgorCSAqIFB1dCBvbiB0aGUgY29ycmVjdCBMUlUgbGlzdCBkZXBlbmRpbmcgb24gdGhl IE1BRFYgc3RhdHVzCisJICovCisJc3Bpbl9sb2NrKCZiby0+YmRldi0+bHJ1X2xvY2spOworCWlm IChvYmotPm1tLm1hZHYgIT0gSTkxNV9NQURWX1dJTExORUVEKSB7CisJCWJvLT5wcmlvcml0eSA9 IEk5MTVfVFRNX1BSSU9fUFVSR0U7CisJfSBlbHNlIGlmICghaTkxNV9nZW1fb2JqZWN0X2hhc19w YWdlcyhvYmopKSB7CisJCWlmIChiby0+cHJpb3JpdHkgPCBJOTE1X1RUTV9QUklPX0hBU19QQUdF UykKKwkJCWJvLT5wcmlvcml0eSA9IEk5MTVfVFRNX1BSSU9fSEFTX1BBR0VTOworCX0gZWxzZSB7 CisJCWlmIChiby0+cHJpb3JpdHkgPiBJOTE1X1RUTV9QUklPX05PX1BBR0VTKQorCQkJYm8tPnBy aW9yaXR5ID0gSTkxNV9UVE1fUFJJT19OT19QQUdFUzsKKwl9CisKKwl0dG1fYm9fbW92ZV90b19s cnVfdGFpbChibywgJmJvLT5tZW0sIE5VTEwpOworCXNwaW5fdW5sb2NrKCZiby0+YmRldi0+bHJ1 X2xvY2spOworfQorCisvKgorICogVFRNLWJhY2tlZCBnZW0gb2JqZWN0IGRlc3RydWN0aW9uIHJl cXVpcmVzIHNvbWUgY2xhcmlmaWNhdGlvbi4KKyAqIEJhc2ljYWxseSB3ZSBoYXZlIHR3byBwb3Nz aWJpbGl0aWVzIGhlcmUuIFdlIGNhbiBlaXRoZXIgcmVseSBvbiB0aGUKKyAqIGk5MTUgZGVsYXll ZCBkZXN0cnVjdGlvbiBhbmQgcHV0IHRoZSBUVE0gb2JqZWN0IHdoZW4gdGhlIG9iamVjdAorICog aXMgaWRsZS4gVGhpcyB3b3VsZCBiZSBkZXRlY3RlZCBieSBUVE0gd2hpY2ggd291bGQgYnlwYXNz IHRoZQorICogVFRNIGRlbGF5ZWQgZGVzdHJveSBoYW5kbGluZy4gVGhlIG90aGVyIGFwcHJvYWNo IGlzIHRvIHB1dCB0aGUgVFRNCisgKiBvYmplY3QgZWFybHkgYW5kIHJlbHkgb24gdGhlIFRUTSBk ZXN0cm95ZWQgaGFuZGxpbmcsIGFuZCB0aGVuIGZyZWUKKyAqIHRoZSBsZWZ0b3ZlciBwYXJ0cyBv ZiB0aGUgR0VNIG9iamVjdCBvbmNlIFRUTSdzIGRlc3Ryb3llZCBsaXN0IGhhbmRsaW5nIGlzCisg KiBjb21wbGV0ZS4gRm9yIG5vdywgd2UgcmVseSBvbiB0aGUgbGF0dGVyIGZvciB0d28gcmVhc29u czoKKyAqIGEpIFRUTSBjYW4gZXZpY3QgYW4gb2JqZWN0IGV2ZW4gd2hlbiBpdCdzIG9uIHRoZSBk ZWxheWVkIGRlc3Ryb3kgbGlzdCwKKyAqIHdoaWNoIGluIHRoZW9yeSBhbGxvd3MgZm9yIGNvbXBs ZXRlIGV2aWN0aW9uLgorICogYikgVGhlcmUgaXMgd29yayBnb2luZyBvbiBpbiBUVE0gdG8gYWxs b3cgZnJlZWluZyBhbiBvYmplY3QgZXZlbiB3aGVuCisgKiBpdCdzIG5vdCBpZGxlLCBhbmQgdXNp bmcgdGhlIFRUTSBkZXN0cm95ZWQgbGlzdCBoYW5kbGluZyBjb3VsZCBoZWxwIHVzCisgKiBiZW5l Zml0IGZyb20gdGhhdC4KKyAqLworc3RhdGljIHZvaWQgaTkxNV90dG1fZGVsYXllZF9mcmVlKHN0 cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmopCit7CisJdHRtX2JvX3B1dChpOTE1X2dlbV90 b190dG0ob2JqKSk7Cit9CisKK3N0YXRpYyBjb25zdCBzdHJ1Y3QgZHJtX2k5MTVfZ2VtX29iamVj dF9vcHMgaTkxNV9nZW1fdHRtX29ial9vcHMgPSB7CisJLm5hbWUgPSAiaTkxNV9nZW1fb2JqZWN0 X3R0bSIsCisJLmZsYWdzID0gSTkxNV9HRU1fT0JKRUNUX0hBU19JT01FTSwKKworCS5nZXRfcGFn ZXMgPSBpOTE1X3R0bV9nZXRfcGFnZXMsCisJLnB1dF9wYWdlcyA9IGk5MTVfdHRtX3B1dF9wYWdl cywKKwkudHJ1bmNhdGUgPSBpOTE1X3R0bV9wdXJnZSwKKwkuYWRqdXN0X2xydSA9IGk5MTVfdHRt X2FkanVzdF9scnUsCisJLmRlbGF5ZWRfZnJlZSA9IGk5MTVfdHRtX2RlbGF5ZWRfZnJlZSwKK307 CisKK3ZvaWQgaTkxNV90dG1fYm9fZGVzdHJveShzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QgKmJv KQoreworCXN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmogPSBpOTE1X3R0bV90b19nZW0o Ym8pOworCisJaTkxNV9nZW1fb2JqZWN0X3JlbGVhc2VfbWVtb3J5X3JlZ2lvbihvYmopOworCWNh bGxfcmN1KCZvYmotPnJjdSwgX19pOTE1X2dlbV9mcmVlX29iamVjdF9yY3UpOworfQorCisvKioK KyAqIF9faTkxNV9nZW1fdHRtX29iamVjdF9pbml0IC0gSW5pdGlhbGl6ZSBhIHR0bS1iYWNrZWQg aTkxNSBnZW0gb2JqZWN0CisgKiBAbWVtOiBUaGUgaW5pdGlhbCBtZW1vcnkgcmVnaW9uIGZvciB0 aGUgb2JqZWN0LgorICogQG9iajogVGhlIGdlbSBvYmplY3QuCisgKiBAc2l6ZTogT2JqZWN0IHNp emUgaW4gYnl0ZXMuCisgKiBAZmxhZ3M6IGdlbSBvYmplY3QgZmxhZ3MuCisgKgorICogUmV0dXJu OiAwIG9uIHN1Y2Nlc3MsIG5lZ2F0aXZlIGVycm9yIGNvZGUgb24gZmFpbHVyZS4KKyAqLworaW50 IF9faTkxNV9nZW1fdHRtX29iamVjdF9pbml0KHN0cnVjdCBpbnRlbF9tZW1vcnlfcmVnaW9uICpt ZW0sCisJCQkgICAgICAgc3RydWN0IGRybV9pOTE1X2dlbV9vYmplY3QgKm9iaiwKKwkJCSAgICAg ICByZXNvdXJjZV9zaXplX3Qgc2l6ZSwKKwkJCSAgICAgICB1bnNpZ25lZCBpbnQgZmxhZ3MpCit7 CisJc3RhdGljIHN0cnVjdCBsb2NrX2NsYXNzX2tleSBsb2NrX2NsYXNzOworCXN0cnVjdCBkcm1f aTkxNV9wcml2YXRlICppOTE1ID0gbWVtLT5pOTE1OworCXNpemVfdCBhbGlnbm1lbnQgPSAwOwor CWludCByZXQ7CisKKwkvKiBBZGp1c3QgYWxpZ25tZW50IHRvIEdQVS0gYW5kIENQVSBodWdlIHBh Z2Ugc2l6ZXMuICovCisKKwlpZiAobWVtLT5pc19yYW5nZV9tYW5hZ2VyKSB7CisJCWlmIChzaXpl ID49IFNaXzFHKQorCQkJYWxpZ25tZW50ID0gU1pfMUcgPj4gUEFHRV9TSElGVDsKKwkJZWxzZSBp ZiAoc2l6ZSA+PSBTWl8yTSkKKwkJCWFsaWdubWVudCA9IFNaXzJNID4+IFBBR0VfU0hJRlQ7CisJ CWVsc2UgaWYgKHNpemUgPj0gU1pfNjRLKQorCQkJYWxpZ25tZW50ID0gU1pfNjRLID4+IFBBR0Vf U0hJRlQ7CisJfQorCisJZHJtX2dlbV9wcml2YXRlX29iamVjdF9pbml0KCZpOTE1LT5kcm0sICZv YmotPmJhc2UsIHNpemUpOworCWk5MTVfZ2VtX29iamVjdF9pbml0KG9iaiwgJmk5MTVfZ2VtX3R0 bV9vYmpfb3BzLCAmbG9ja19jbGFzcywgZmxhZ3MpOworCWk5MTVfZ2VtX29iamVjdF9pbml0X21l bW9yeV9yZWdpb24ob2JqLCBtZW0pOworCWk5MTVfZ2VtX29iamVjdF9tYWtlX3Vuc2hyaW5rYWJs ZShvYmopOworCW9iai0+cmVhZF9kb21haW5zID0gSTkxNV9HRU1fRE9NQUlOX1dDIHwgSTkxNV9H RU1fRE9NQUlOX0dUVDsKKwlpOTE1X2dlbV9vYmplY3Rfc2V0X2NhY2hlX2NvaGVyZW5jeShvYmos IEk5MTVfQ0FDSEVfTk9ORSk7CisKKwlyZXQgPSB0dG1fYm9faW5pdCgmaTkxNS0+YmRldiwgaTkx NV9nZW1fdG9fdHRtKG9iaiksIHNpemUsCisJCQkgIHR0bV9ib190eXBlX2tlcm5lbCwgJmk5MTVf c3lzX3BsYWNlbWVudCwgYWxpZ25tZW50LAorCQkJICB0cnVlLCBOVUxMLCBOVUxMLCBpOTE1X3R0 bV9ib19kZXN0cm95KTsKKworCS8qIGk5MTUgd2FudHMgLUVOWElPIHdoZW4gb3V0IG9mIG1lbW9y eSByZWdpb24gc3BhY2UuICovCisJcmV0dXJuIChyZXQgPT0gLUVOT1NQQykgPyAtRU5YSU8gOiBy ZXQ7Cit9CmRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRt LmggYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtLmgKbmV3IGZpbGUgbW9k ZSAxMDA2NDQKaW5kZXggMDAwMDAwMDAwMDAwLi5iOGQzZGNiYjUwZGYKLS0tIC9kZXYvbnVsbAor KysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9nZW0vaTkxNV9nZW1fdHRtLmgKQEAgLTAsMCArMSw0 OCBAQAorLyogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6IE1JVCAqLworLyoKKyAqIENvcHlyaWdo dCDCqSAyMDIxIEludGVsIENvcnBvcmF0aW9uCisgKi8KKyNpZm5kZWYgX0k5MTVfR0VNX1RUTV9I XworI2RlZmluZSBfSTkxNV9HRU1fVFRNX0hfCisKKyNpbmNsdWRlICJnZW0vaTkxNV9nZW1fb2Jq ZWN0X3R5cGVzLmgiCisKKy8qKgorICogaTkxNV9nZW1fdG9fdHRtIC0gQ29udmVydCBhIHN0cnVj dCBkcm1faTkxNV9nZW1fb2JqZWN0IHRvIGEKKyAqIHN0cnVjdCB0dG1fYnVmZmVyX29iamVjdC4K KyAqIEBvYmo6IFBvaW50ZXIgdG8gdGhlIGdlbSBvYmplY3QuCisgKgorICogUmV0dXJuOiBQb2lu dGVyIHRvIHRoZSBlbWJlZGRlZCBzdHJ1Y3QgdHRtX2J1ZmZlcl9vYmplY3QuCisgKi8KK3N0YXRp YyBpbmxpbmUgc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0ICoKK2k5MTVfZ2VtX3RvX3R0bShzdHJ1 Y3QgZHJtX2k5MTVfZ2VtX29iamVjdCAqb2JqKQoreworCXJldHVybiAmb2JqLT5fX2RvX25vdF9h Y2Nlc3M7Cit9CisKKy8qCisgKiBpOTE1IHR0bSBnZW0gb2JqZWN0IGRlc3RydWN0b3IuIEludGVy bmFsIHVzZSBvbmx5LgorICovCit2b2lkIGk5MTVfdHRtX2JvX2Rlc3Ryb3koc3RydWN0IHR0bV9i dWZmZXJfb2JqZWN0ICpibyk7CisKKy8qKgorICogaTkxNV90dG1fdG9fZ2VtIC0gQ29udmVydCBh IHN0cnVjdCB0dG1fYnVmZmVyX29iamVjdCB0byBhbiBlbWJlZGRpbmcKKyAqIHN0cnVjdCBkcm1f aTkxNV9nZW1fb2JqZWN0LgorICoKKyAqIFJldHVybjogUG9pbnRlciB0byB0aGUgZW1iZWRkaW5n IHN0cnVjdCB0dG1fYnVmZmVyX29iamVjdCwgb3IgTlVMTAorICogaWYgdGhlIG9iamVjdCB3YXMg bm90IGFuIGk5MTUgdHRtIG9iamVjdC4KKyAqLworc3RhdGljIGlubGluZSBzdHJ1Y3QgZHJtX2k5 MTVfZ2VtX29iamVjdCAqCitpOTE1X3R0bV90b19nZW0oc3RydWN0IHR0bV9idWZmZXJfb2JqZWN0 ICpibykKK3sKKwlpZiAoR0VNX1dBUk5fT04oYm8tPmRlc3Ryb3kgIT0gaTkxNV90dG1fYm9fZGVz dHJveSkpCisJCXJldHVybiBOVUxMOworCisJcmV0dXJuIGNvbnRhaW5lcl9vZihibywgc3RydWN0 IGRybV9pOTE1X2dlbV9vYmplY3QsIF9fZG9fbm90X2FjY2Vzcyk7Cit9CisKK2ludCBfX2k5MTVf Z2VtX3R0bV9vYmplY3RfaW5pdChzdHJ1Y3QgaW50ZWxfbWVtb3J5X3JlZ2lvbiAqbWVtLAorCQkJ ICAgICAgIHN0cnVjdCBkcm1faTkxNV9nZW1fb2JqZWN0ICpvYmosCisJCQkgICAgICAgcmVzb3Vy Y2Vfc2l6ZV90IHNpemUsCisJCQkgICAgICAgdW5zaWduZWQgaW50IGZsYWdzKTsKKyNlbmRpZgpk aWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ3QvaW50ZWxfcmVnaW9uX2xtZW0uYyBi L2RyaXZlcnMvZ3B1L2RybS9pOTE1L2d0L2ludGVsX3JlZ2lvbl9sbWVtLmMKaW5kZXggZjczNjZi MDU0ZjhlLi40YWUxZjcxN2E5NGMgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2d0 L2ludGVsX3JlZ2lvbl9sbWVtLmMKKysrIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvZ3QvaW50ZWxf cmVnaW9uX2xtZW0uYwpAQCAtOSw2ICs5LDcgQEAKICNpbmNsdWRlICJpbnRlbF9yZWdpb25fdHRt LmgiCiAjaW5jbHVkZSAiZ2VtL2k5MTVfZ2VtX2xtZW0uaCIKICNpbmNsdWRlICJnZW0vaTkxNV9n ZW1fcmVnaW9uLmgiCisjaW5jbHVkZSAiZ2VtL2k5MTVfZ2VtX3R0bS5oIgogI2luY2x1ZGUgImlu dGVsX3JlZ2lvbl9sbWVtLmgiCiAKIHN0YXRpYyBpbnQgaW5pdF9mYWtlX2xtZW1fYmFyKHN0cnVj dCBpbnRlbF9tZW1vcnlfcmVnaW9uICptZW0pCkBAIC0xMDcsNyArMTA4LDcgQEAgcmVnaW9uX2xt ZW1faW5pdChzdHJ1Y3QgaW50ZWxfbWVtb3J5X3JlZ2lvbiAqbWVtKQogc3RhdGljIGNvbnN0IHN0 cnVjdCBpbnRlbF9tZW1vcnlfcmVnaW9uX29wcyBpbnRlbF9yZWdpb25fbG1lbV9vcHMgPSB7CiAJ LmluaXQgPSByZWdpb25fbG1lbV9pbml0LAogCS5yZWxlYXNlID0gcmVnaW9uX2xtZW1fcmVsZWFz ZSwKLQkuaW5pdF9vYmplY3QgPSBfX2k5MTVfZ2VtX2xtZW1fb2JqZWN0X2luaXQsCisJLmluaXRf b2JqZWN0ID0gX19pOTE1X2dlbV90dG1fb2JqZWN0X2luaXQsCiB9OwogCiBzdHJ1Y3QgaW50ZWxf bWVtb3J5X3JlZ2lvbiAqCmRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pOTE1X2dl bS5jIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvaTkxNV9nZW0uYwppbmRleCAxODBmNmU5MTA3ZDQu LjM1MDI4M2FiOWE4MyAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvaTkxNV9nZW0u YworKysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pOTE1X2dlbS5jCkBAIC0xMDA1LDggKzEwMDUs MTEgQEAgaTkxNV9nZW1fbWFkdmlzZV9pb2N0bChzdHJ1Y3QgZHJtX2RldmljZSAqZGV2LCB2b2lk ICpkYXRhLAogCQl9CiAJfQogCi0JaWYgKG9iai0+bW0ubWFkdiAhPSBfX0k5MTVfTUFEVl9QVVJH RUQpCisJaWYgKG9iai0+bW0ubWFkdiAhPSBfX0k5MTVfTUFEVl9QVVJHRUQpIHsKIAkJb2JqLT5t bS5tYWR2ID0gYXJncy0+bWFkdjsKKwkJaWYgKG9iai0+b3BzLT5hZGp1c3RfbHJ1KQorCQkJb2Jq LT5vcHMtPmFkanVzdF9scnUob2JqKTsKKwl9CiAKIAlpZiAoaTkxNV9nZW1fb2JqZWN0X2hhc19w YWdlcyhvYmopKSB7CiAJCXN0cnVjdCBsaXN0X2hlYWQgKmxpc3Q7CmRpZmYgLS1naXQgYS9kcml2 ZXJzL2dwdS9kcm0vaTkxNS9pbnRlbF9tZW1vcnlfcmVnaW9uLmMgYi9kcml2ZXJzL2dwdS9kcm0v aTkxNS9pbnRlbF9tZW1vcnlfcmVnaW9uLmMKaW5kZXggNDA5MmNjOTg3Njc5Li5iZDI3ZTg5N2Q0 ZDAgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX21lbW9yeV9yZWdpb24u YworKysgYi9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pbnRlbF9tZW1vcnlfcmVnaW9uLmMKQEAgLTE0 OSw3ICsxNDksNiBAQCBpbnRlbF9tZW1vcnlfcmVnaW9uX2NyZWF0ZShzdHJ1Y3QgZHJtX2k5MTVf cHJpdmF0ZSAqaTkxNSwKIAogCW11dGV4X2luaXQoJm1lbS0+b2JqZWN0cy5sb2NrKTsKIAlJTklU X0xJU1RfSEVBRCgmbWVtLT5vYmplY3RzLmxpc3QpOwotCUlOSVRfTElTVF9IRUFEKCZtZW0tPm9i amVjdHMucHVyZ2VhYmxlKTsKIAlJTklUX0xJU1RfSEVBRCgmbWVtLT5yZXNlcnZlZCk7CiAKIAlt dXRleF9pbml0KCZtZW0tPm1tX2xvY2spOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5 MTUvaW50ZWxfbWVtb3J5X3JlZ2lvbi5oIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50ZWxfbWVt b3J5X3JlZ2lvbi5oCmluZGV4IGU2OWNkZTEzZGFmMi4uN2I1ZmE5N2MwYjU5IDEwMDY0NAotLS0g YS9kcml2ZXJzL2dwdS9kcm0vaTkxNS9pbnRlbF9tZW1vcnlfcmVnaW9uLmgKKysrIGIvZHJpdmVy cy9ncHUvZHJtL2k5MTUvaW50ZWxfbWVtb3J5X3JlZ2lvbi5oCkBAIC0xMDAsNyArMTAwLDYgQEAg c3RydWN0IGludGVsX21lbW9yeV9yZWdpb24gewogCXN0cnVjdCB7CiAJCXN0cnVjdCBtdXRleCBs b2NrOyAvKiBQcm90ZWN0cyBhY2Nlc3MgdG8gb2JqZWN0cyAqLwogCQlzdHJ1Y3QgbGlzdF9oZWFk IGxpc3Q7Ci0JCXN0cnVjdCBsaXN0X2hlYWQgcHVyZ2VhYmxlOwogCX0gb2JqZWN0czsKIAogCXNp emVfdCBjaHVua19zaXplOwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50ZWxf cmVnaW9uX3R0bS5jIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50ZWxfcmVnaW9uX3R0bS5jCmlu ZGV4IDlmYjQ0N2UxOWViZC4uZTQyOWM5YTM4MzcxIDEwMDY0NAotLS0gYS9kcml2ZXJzL2dwdS9k cm0vaTkxNS9pbnRlbF9yZWdpb25fdHRtLmMKKysrIGIvZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50 ZWxfcmVnaW9uX3R0bS5jCkBAIC0xMCw4ICsxMCw3IEBACiAKICNpbmNsdWRlICJpbnRlbF9yZWdp b25fdHRtLmgiCiAKLS8qIEEgWmVyby1pbml0aWFsaXplZCBkcml2ZXIgZm9yIG5vdy4gV2UgZG9u J3QgaGF2ZSBhIFRUTSBiYWNrZW5kIHlldC4gKi8KLXN0YXRpYyBzdHJ1Y3QgdHRtX2RldmljZV9m dW5jcyBpOTE1X3R0bV9ib19kcml2ZXI7CitleHRlcm4gc3RydWN0IHR0bV9kZXZpY2VfZnVuY3Mg aTkxNV90dG1fYm9fZHJpdmVyOwogCiAvKioKICAqIERPQzogVFRNIHN1cHBvcnQgc3RydWN0dXJl CkBAIC0xOTcsNiArMTk2LDcgQEAgc3RydWN0IHNnX3RhYmxlICppbnRlbF9yZWdpb25fdHRtX25v ZGVfdG9fc3Qoc3RydWN0IGludGVsX21lbW9yeV9yZWdpb24gKm1lbSwKIAlyZXR1cm4gaTkxNV9z Z19mcm9tX21tX25vZGUobm9kZSwgbWVtLT5yZWdpb24uc3RhcnQpOwogfQogCisjaWZkZWYgQ09O RklHX0RSTV9JOTE1X1NFTEZURVNUCiAvKioKICAqIGludGVsX3JlZ2lvbl90dG1fbm9kZV9hbGxv YyAtIEFsbG9jYXRlIG1lbW9yeSByZXNvdXJjZXMgZnJvbSBhIHJlZ2lvbgogICogQG1lbTogVGhl IG1lbW9yeSByZWdpb24sCkBAIC0yNDMsMyArMjQzLDQgQEAgdm9pZCAqaW50ZWxfcmVnaW9uX3R0 bV9ub2RlX2FsbG9jKHN0cnVjdCBpbnRlbF9tZW1vcnlfcmVnaW9uICptZW0sCiAJCXJldCA9IC1F TlhJTzsKIAlyZXR1cm4gcmV0ID8gRVJSX1BUUihyZXQpIDogcmVzLm1tX25vZGU7CiB9CisjZW5k aWYKZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX3JlZ2lvbl90dG0uaCBi L2RyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX3JlZ2lvbl90dG0uaAppbmRleCA0MDEyOWY3ZTAz MTcuLjlhNWIwNDM3ZDczZiAxMDA2NDQKLS0tIGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvaW50ZWxf cmVnaW9uX3R0bS5oCisrKyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX3JlZ2lvbl90dG0u aApAQCAtMjEsOSArMjEsMTIgQEAgdm9pZCBpbnRlbF9yZWdpb25fdHRtX2Zpbmkoc3RydWN0IGlu dGVsX21lbW9yeV9yZWdpb24gKm1lbSk7CiBzdHJ1Y3Qgc2dfdGFibGUgKmludGVsX3JlZ2lvbl90 dG1fbm9kZV90b19zdChzdHJ1Y3QgaW50ZWxfbWVtb3J5X3JlZ2lvbiAqbWVtLAogCQkJCQkgICAg IHZvaWQgKm5vZGUpOwogCit2b2lkIGludGVsX3JlZ2lvbl90dG1fbm9kZV9mcmVlKHN0cnVjdCBp bnRlbF9tZW1vcnlfcmVnaW9uICptZW0sCisJCQkJdm9pZCAqbm9kZSk7CisKKyNpZmRlZiBDT05G SUdfRFJNX0k5MTVfU0VMRlRFU1QKIHZvaWQgKmludGVsX3JlZ2lvbl90dG1fbm9kZV9hbGxvYyhz dHJ1Y3QgaW50ZWxfbWVtb3J5X3JlZ2lvbiAqbWVtLAogCQkJCSAgcmVzb3VyY2Vfc2l6ZV90IHNp emUsCiAJCQkJICB1bnNpZ25lZCBpbnQgZmxhZ3MpOwotdm9pZCBpbnRlbF9yZWdpb25fdHRtX25v ZGVfZnJlZShzdHJ1Y3QgaW50ZWxfbWVtb3J5X3JlZ2lvbiAqbWVtLAotCQkJCXZvaWQgKm5vZGUp OworI2VuZGlmCiAjZW5kaWYgLyogX0lOVEVMX1JFR0lPTl9UVE1fSF8gKi8KLS0gCjIuMzEuMQoK X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KSW50ZWwtZ2Z4 IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DFBAC433B4 for ; Tue, 18 May 2021 08:28:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6B1A0610E9 for ; Tue, 18 May 2021 08:28:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B1A0610E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 130786EACF; Tue, 18 May 2021 08:28:02 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8C9596EAC0; Tue, 18 May 2021 08:27:58 +0000 (UTC) IronPort-SDR: ESGkcn6jZwr9uJBCMR/wc9+YLx9/jdgd6heDKBTC8IDDYwxGdbWrO9ap6OF2voOfW8AH8Giy6V qN0OMYYSX3Gw== X-IronPort-AV: E=McAfee;i="6200,9189,9987"; a="197572728" X-IronPort-AV: E=Sophos;i="5.82,309,1613462400"; d="scan'208";a="197572728" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2021 01:27:58 -0700 IronPort-SDR: CYHSjNVrT7WB5jDAwd/GtBEuDiPrNjZN4FWQghuQO0ILOmyVuRhl7pN2ONtZBfGUdogdyVTBhP XjUXU0sVbAtQ== X-IronPort-AV: E=Sophos;i="5.82,309,1613462400"; d="scan'208";a="611892377" Received: from cmutgix-mobl.gar.corp.intel.com (HELO thellst-mobl1.intel.com) ([10.249.254.195]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2021 01:27:44 -0700 From: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 10/15] drm/i915/ttm: Introduce a TTM i915 gem object backend Date: Tue, 18 May 2021 10:26:56 +0200 Message-Id: <20210518082701.997251-11-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210518082701.997251-1-thomas.hellstrom@linux.intel.com> References: <20210518082701.997251-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Most logical place to introduce TTM buffer objects is as an i915 gem object backend. We need to add some ops to account for added functionality like delayed delete and LRU list manipulation. Initially we support only LMEM and SYSTEM memory, but SYSTEM (which in this case means evicted LMEM objects) is not visible to i915 GEM yet. The plan is to move the i915 gem system region over to the TTM system memory type in upcoming patches. We set up GPU bindings directly both from LMEM and from the system region, as there is no need to use the legacy TTM_TT memory type. We reserve that for future porting of GGTT bindings to TTM. Remove the old lmem backend. Signed-off-by: Thomas Hellström --- v2: - Break out needed TTM functionality to a separate patch (Reported by Christian König). - Fix an unhandled error (Reported by Matthew Auld and Maarten Lankhorst) - Remove a stray leftover sg_table allocation (Reported by Matthew Auld) - Use ttm_tt_unpopulate() rather than ttm_tt_destroy() in the purge path as some TTM functionality relies on having a ttm_tt present for !is_iomem. --- drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/gem/i915_gem_lmem.c | 84 --- drivers/gpu/drm/i915/gem/i915_gem_lmem.h | 5 - drivers/gpu/drm/i915/gem/i915_gem_object.c | 125 +++-- drivers/gpu/drm/i915/gem/i915_gem_object.h | 9 + .../gpu/drm/i915/gem/i915_gem_object_types.h | 18 + drivers/gpu/drm/i915/gem/i915_gem_region.c | 6 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 519 ++++++++++++++++++ drivers/gpu/drm/i915/gem/i915_gem_ttm.h | 48 ++ drivers/gpu/drm/i915/gt/intel_region_lmem.c | 3 +- drivers/gpu/drm/i915/i915_gem.c | 5 +- drivers/gpu/drm/i915/intel_memory_region.c | 1 - drivers/gpu/drm/i915/intel_memory_region.h | 1 - drivers/gpu/drm/i915/intel_region_ttm.c | 5 +- drivers/gpu/drm/i915/intel_region_ttm.h | 7 +- 15 files changed, 696 insertions(+), 141 deletions(-) create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm.c create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_ttm.h diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile index 958ccc1edfed..ef0d884a9e2d 100644 --- a/drivers/gpu/drm/i915/Makefile +++ b/drivers/gpu/drm/i915/Makefile @@ -155,6 +155,7 @@ gem-y += \ gem/i915_gem_stolen.o \ gem/i915_gem_throttle.o \ gem/i915_gem_tiling.o \ + gem/i915_gem_ttm.o \ gem/i915_gem_ttm_bo_util.o \ gem/i915_gem_userptr.o \ gem/i915_gem_wait.o \ diff --git a/drivers/gpu/drm/i915/gem/i915_gem_lmem.c b/drivers/gpu/drm/i915/gem/i915_gem_lmem.c index 3b4aa28a076d..2b8cd15de1d9 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_lmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_lmem.c @@ -4,74 +4,10 @@ */ #include "intel_memory_region.h" -#include "intel_region_ttm.h" #include "gem/i915_gem_region.h" #include "gem/i915_gem_lmem.h" #include "i915_drv.h" -static void lmem_put_pages(struct drm_i915_gem_object *obj, - struct sg_table *pages) -{ - intel_region_ttm_node_free(obj->mm.region, obj->mm.st_mm_node); - obj->mm.dirty = false; - sg_free_table(pages); - kfree(pages); -} - -static int lmem_get_pages(struct drm_i915_gem_object *obj) -{ - unsigned int flags; - struct sg_table *pages; - - flags = I915_ALLOC_MIN_PAGE_SIZE; - if (obj->flags & I915_BO_ALLOC_CONTIGUOUS) - flags |= I915_ALLOC_CONTIGUOUS; - - obj->mm.st_mm_node = intel_region_ttm_node_alloc(obj->mm.region, - obj->base.size, - flags); - if (IS_ERR(obj->mm.st_mm_node)) - return PTR_ERR(obj->mm.st_mm_node); - - /* Range manager is always contigous */ - if (obj->mm.region->is_range_manager) - obj->flags |= I915_BO_ALLOC_CONTIGUOUS; - pages = intel_region_ttm_node_to_st(obj->mm.region, obj->mm.st_mm_node); - if (IS_ERR(pages)) { - intel_region_ttm_node_free(obj->mm.region, obj->mm.st_mm_node); - return PTR_ERR(pages); - } - - __i915_gem_object_set_pages(obj, pages, i915_sg_dma_sizes(pages->sgl)); - - if (obj->flags & I915_BO_ALLOC_CPU_CLEAR) { - void __iomem *vaddr = - i915_gem_object_lmem_io_map(obj, 0, obj->base.size); - - if (!vaddr) { - struct sg_table *pages = - __i915_gem_object_unset_pages(obj); - - if (!IS_ERR_OR_NULL(pages)) - lmem_put_pages(obj, pages); - } - - memset_io(vaddr, 0, obj->base.size); - io_mapping_unmap(vaddr); - } - - return 0; -} - -const struct drm_i915_gem_object_ops i915_gem_lmem_obj_ops = { - .name = "i915_gem_object_lmem", - .flags = I915_GEM_OBJECT_HAS_IOMEM, - - .get_pages = lmem_get_pages, - .put_pages = lmem_put_pages, - .release = i915_gem_object_release_memory_region, -}; - void __iomem * i915_gem_object_lmem_io_map(struct drm_i915_gem_object *obj, unsigned long n, @@ -103,23 +39,3 @@ i915_gem_object_create_lmem(struct drm_i915_private *i915, return i915_gem_object_create_region(i915->mm.regions[INTEL_REGION_LMEM], size, flags); } - -int __i915_gem_lmem_object_init(struct intel_memory_region *mem, - struct drm_i915_gem_object *obj, - resource_size_t size, - unsigned int flags) -{ - static struct lock_class_key lock_class; - struct drm_i915_private *i915 = mem->i915; - - drm_gem_private_object_init(&i915->drm, &obj->base, size); - i915_gem_object_init(obj, &i915_gem_lmem_obj_ops, &lock_class, flags); - - obj->read_domains = I915_GEM_DOMAIN_WC | I915_GEM_DOMAIN_GTT; - - i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE); - - i915_gem_object_init_memory_region(obj, mem); - - return 0; -} diff --git a/drivers/gpu/drm/i915/gem/i915_gem_lmem.h b/drivers/gpu/drm/i915/gem/i915_gem_lmem.h index fac6bc5a5ebb..ea76fd11ccb0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_lmem.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_lmem.h @@ -26,9 +26,4 @@ i915_gem_object_create_lmem(struct drm_i915_private *i915, resource_size_t size, unsigned int flags); -int __i915_gem_lmem_object_init(struct intel_memory_region *mem, - struct drm_i915_gem_object *obj, - resource_size_t size, - unsigned int flags); - #endif /* !__I915_GEM_LMEM_H */ diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c index c8953e3f5c70..8580996107ce 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c @@ -172,7 +172,7 @@ static void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *f } } -static void __i915_gem_free_object_rcu(struct rcu_head *head) +void __i915_gem_free_object_rcu(struct rcu_head *head) { struct drm_i915_gem_object *obj = container_of(head, typeof(*obj), rcu); @@ -208,59 +208,69 @@ static void __i915_gem_object_free_mmaps(struct drm_i915_gem_object *obj) } } -static void __i915_gem_free_objects(struct drm_i915_private *i915, - struct llist_node *freed) +void __i915_gem_free_object(struct drm_i915_gem_object *obj) { - struct drm_i915_gem_object *obj, *on; + trace_i915_gem_object_destroy(obj); - llist_for_each_entry_safe(obj, on, freed, freed) { - trace_i915_gem_object_destroy(obj); + if (!list_empty(&obj->vma.list)) { + struct i915_vma *vma; + + /* + * Note that the vma keeps an object reference while + * it is active, so it *should* not sleep while we + * destroy it. Our debug code errs insits it *might*. + * For the moment, play along. + */ + spin_lock(&obj->vma.lock); + while ((vma = list_first_entry_or_null(&obj->vma.list, + struct i915_vma, + obj_link))) { + GEM_BUG_ON(vma->obj != obj); + spin_unlock(&obj->vma.lock); - if (!list_empty(&obj->vma.list)) { - struct i915_vma *vma; + __i915_vma_put(vma); - /* - * Note that the vma keeps an object reference while - * it is active, so it *should* not sleep while we - * destroy it. Our debug code errs insits it *might*. - * For the moment, play along. - */ spin_lock(&obj->vma.lock); - while ((vma = list_first_entry_or_null(&obj->vma.list, - struct i915_vma, - obj_link))) { - GEM_BUG_ON(vma->obj != obj); - spin_unlock(&obj->vma.lock); + } + spin_unlock(&obj->vma.lock); + } - __i915_vma_put(vma); + __i915_gem_object_free_mmaps(obj); - spin_lock(&obj->vma.lock); - } - spin_unlock(&obj->vma.lock); - } + GEM_BUG_ON(!list_empty(&obj->lut_list)); - __i915_gem_object_free_mmaps(obj); + atomic_set(&obj->mm.pages_pin_count, 0); + __i915_gem_object_put_pages(obj); + GEM_BUG_ON(i915_gem_object_has_pages(obj)); + bitmap_free(obj->bit_17); - GEM_BUG_ON(!list_empty(&obj->lut_list)); + if (obj->base.import_attach) + drm_prime_gem_destroy(&obj->base, NULL); - atomic_set(&obj->mm.pages_pin_count, 0); - __i915_gem_object_put_pages(obj); - GEM_BUG_ON(i915_gem_object_has_pages(obj)); - bitmap_free(obj->bit_17); + drm_gem_free_mmap_offset(&obj->base); - if (obj->base.import_attach) - drm_prime_gem_destroy(&obj->base, NULL); + if (obj->ops->release) + obj->ops->release(obj); - drm_gem_free_mmap_offset(&obj->base); + if (obj->mm.n_placements > 1) + kfree(obj->mm.placements); - if (obj->ops->release) - obj->ops->release(obj); + if (obj->resv_shared_from) + i915_vm_resv_put(obj->resv_shared_from); +} - if (obj->mm.n_placements > 1) - kfree(obj->mm.placements); +static void __i915_gem_free_objects(struct drm_i915_private *i915, + struct llist_node *freed) +{ + struct drm_i915_gem_object *obj, *on; - if (obj->resv_shared_from) - i915_vm_resv_put(obj->resv_shared_from); + llist_for_each_entry_safe(obj, on, freed, freed) { + might_sleep(); + if (obj->ops->delayed_free) { + obj->ops->delayed_free(obj); + continue; + } + __i915_gem_free_object(obj); /* But keep the pointer alive for RCU-protected lookups */ call_rcu(&obj->rcu, __i915_gem_free_object_rcu); @@ -318,6 +328,7 @@ static void i915_gem_free_object(struct drm_gem_object *gem_obj) * worker and performing frees directly from subsequent allocations for * crude but effective memory throttling. */ + if (llist_add(&obj->freed, &i915->mm.free_list)) queue_work(i915->wq, &i915->mm.free_work); } @@ -410,6 +421,42 @@ int i915_gem_object_read_from_page(struct drm_i915_gem_object *obj, u64 offset, return 0; } +/** + * i915_gem_object_evictable - Whether object is likely evictable after unbind. + * @obj: The object to check + * + * This function checks whether the object is likely unvictable after unbind. + * If the object is not locked when checking, the result is only advisory. + * If the object is locked when checking, and the function returns true, + * then an eviction should indeed be possible. But since unlocked vma + * unpinning and unbinding is currently possible, the object can actually + * become evictable even if this function returns false. + * + * Return: true if the object may be evictable. False otherwise. + */ +bool i915_gem_object_evictable(struct drm_i915_gem_object *obj) +{ + struct i915_vma *vma; + int pin_count = atomic_read(&obj->mm.pages_pin_count); + + if (!pin_count) + return true; + + spin_lock(&obj->vma.lock); + list_for_each_entry(vma, &obj->vma.list, obj_link) { + if (i915_vma_is_pinned(vma)) { + spin_unlock(&obj->vma.lock); + return false; + } + if (atomic_read(&vma->pages_count)) + pin_count--; + } + spin_unlock(&obj->vma.lock); + GEM_WARN_ON(pin_count < 0); + + return pin_count == 0; +} + void i915_gem_init__objects(struct drm_i915_private *i915) { INIT_WORK(&i915->mm.free_work, __i915_gem_free_work); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h index 2ebd79537aea..ae5930e307d5 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h @@ -200,6 +200,9 @@ static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj) static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj) { + if (obj->ops->adjust_lru) + obj->ops->adjust_lru(obj); + dma_resv_unlock(obj->base.resv); } @@ -587,6 +590,12 @@ int i915_gem_object_read_from_page(struct drm_i915_gem_object *obj, u64 offset, bool i915_gem_object_is_shmem(const struct drm_i915_gem_object *obj); +void __i915_gem_free_object_rcu(struct rcu_head *head); + +void __i915_gem_free_object(struct drm_i915_gem_object *obj); + +bool i915_gem_object_evictable(struct drm_i915_gem_object *obj); + #ifdef CONFIG_MMU_NOTIFIER static inline bool i915_gem_object_is_userptr(struct drm_i915_gem_object *obj) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index 98f69d8fd37d..b350765e1935 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -63,6 +63,20 @@ struct drm_i915_gem_object_ops { const struct drm_i915_gem_pwrite *arg); int (*dmabuf_export)(struct drm_i915_gem_object *obj); + + /** + * adjust_lru - notify that the madvise value was updated + * @obj: The gem object + * + * The madvise value may have been updated, or object was recently + * referenced so act accordingly (Perhaps changing an LRU list etc). + */ + void (*adjust_lru)(struct drm_i915_gem_object *obj); + + /** + * delayed_free - Override the default delayed free implementation + */ + void (*delayed_free)(struct drm_i915_gem_object *obj); void (*release)(struct drm_i915_gem_object *obj); const char *name; /* friendly name for debug, e.g. lockdep classes */ @@ -307,6 +321,10 @@ struct drm_i915_gem_object { bool dirty:1; } mm; + struct { + struct sg_table *cached_io_st; + } ttm; + /** Record of address bit 17 of each page at last unbind. */ unsigned long *bit_17; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_region.c b/drivers/gpu/drm/i915/gem/i915_gem_region.c index f25e6646c5b7..d1f1840540dd 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_region.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_region.c @@ -18,11 +18,7 @@ void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj, mutex_lock(&mem->objects.lock); - if (obj->flags & I915_BO_ALLOC_VOLATILE) - list_add(&obj->mm.region_link, &mem->objects.purgeable); - else - list_add(&obj->mm.region_link, &mem->objects.list); - + list_add(&obj->mm.region_link, &mem->objects.list); mutex_unlock(&mem->objects.lock); } diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c new file mode 100644 index 000000000000..790f5ec45c4d --- /dev/null +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -0,0 +1,519 @@ +// SPDX-License-Identifier: MIT +/* + * Copyright © 2021 Intel Corporation + */ + +#include +#include + +#include "i915_drv.h" +#include "intel_memory_region.h" +#include "intel_region_ttm.h" + +#include "gem/i915_gem_object.h" +#include "gem/i915_gem_region.h" +#include "gem/i915_gem_ttm.h" +#include "gem/i915_gem_ttm_bo_util.h" + +#define I915_PL_LMEM0 TTM_PL_PRIV +#define I915_PL_SYSTEM TTM_PL_SYSTEM +#define I915_PL_STOLEN TTM_PL_VRAM +#define I915_PL_GGTT TTM_PL_TT + +#define I915_TTM_PRIO_PURGE 0 +#define I915_TTM_PRIO_NO_PAGES 1 +#define I915_TTM_PRIO_HAS_PAGES 2 + +/** + * struct i915_ttm_tt - TTM page vector with additional private information + * @ttm: The base TTM page vector. + * @dev: The struct device used for dma mapping and unmapping. + * @cached_st: The cached scatter-gather table. + * + * Note that DMA may be going on right up to the point where the page- + * vector is unpopulated in delayed destroy. Hence keep the + * scatter-gather table mapped and cached up to that point. This is + * different from the cached gem object io scatter-gather table which + * doesn't have an associated dma mapping. + */ +struct i915_ttm_tt { + struct ttm_tt ttm; + struct device *dev; + struct sg_table *cached_st; +}; + +static const struct ttm_place lmem0_sys_placement_flags[] = { + { + .fpfn = 0, + .lpfn = 0, + .mem_type = I915_PL_LMEM0, + .flags = 0, + }, { + .fpfn = 0, + .lpfn = 0, + .mem_type = I915_PL_SYSTEM, + .flags = 0, + } +}; + +struct ttm_placement i915_lmem0_placement = { + .num_placement = 1, + .placement = &lmem0_sys_placement_flags[0], + .num_busy_placement = 1, + .busy_placement = &lmem0_sys_placement_flags[0], +}; + +struct ttm_placement i915_lmem0_sys_placement = { + .num_placement = 1, + .placement = &lmem0_sys_placement_flags[0], + .num_busy_placement = 2, + .busy_placement = &lmem0_sys_placement_flags[0], +}; + +struct ttm_placement i915_sys_placement = { + .num_placement = 1, + .placement = &lmem0_sys_placement_flags[1], + .num_busy_placement = 1, + .busy_placement = &lmem0_sys_placement_flags[1], +}; + +static void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj); + +static struct ttm_tt *i915_ttm_tt_create(struct ttm_buffer_object *bo, + uint32_t page_flags) +{ + struct ttm_resource_manager *man = + ttm_manager_type(bo->bdev, bo->mem.mem_type); + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + struct i915_ttm_tt *i915_tt; + int ret; + + i915_tt = kzalloc(sizeof(*i915_tt), GFP_KERNEL); + if (!i915_tt) + return NULL; + + if (obj->flags & I915_BO_ALLOC_CPU_CLEAR && + man->use_tt) + page_flags |= TTM_PAGE_FLAG_ZERO_ALLOC; + + ret = ttm_tt_init(&i915_tt->ttm, bo, page_flags, ttm_cached); + if (ret) { + kfree(i915_tt); + return NULL; + } + + i915_tt->dev = obj->base.dev->dev; + + return &i915_tt->ttm; +} + +static void i915_ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm) +{ + struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm); + + if (i915_tt->cached_st) { + dma_unmap_sgtable(i915_tt->dev, i915_tt->cached_st, + DMA_BIDIRECTIONAL, 0); + sg_free_table(i915_tt->cached_st); + kfree(i915_tt->cached_st); + i915_tt->cached_st = NULL; + } + ttm_pool_free(&bdev->pool, ttm); +} + +static void i915_ttm_tt_destroy(struct ttm_device *bdev, struct ttm_tt *ttm) +{ + struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm); + + ttm_tt_destroy_common(bdev, ttm); + kfree(i915_tt); +} + +static bool i915_ttm_eviction_valuable(struct ttm_buffer_object *bo, + const struct ttm_place *place) +{ + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + + /* Will do for now. Our pinned objects are still on TTM's LRU lists */ + if (!i915_gem_object_evictable(obj)) + return false; + + /* This isn't valid with a buddy allocator */ + return ttm_bo_eviction_valuable(bo, place); +} + +static void i915_ttm_evict_flags(struct ttm_buffer_object *bo, + struct ttm_placement *placement) +{ + *placement = i915_sys_placement; +} + +static int i915_ttm_move_notify(struct ttm_buffer_object *bo) +{ + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + int ret; + + ret = i915_gem_object_unbind(obj, I915_GEM_OBJECT_UNBIND_ACTIVE); + if (ret) + return ret; + + ret = __i915_gem_object_put_pages(obj); + if (ret) + return ret; + + return 0; +} + +static void i915_ttm_free_cached_io_st(struct drm_i915_gem_object *obj) +{ + if (obj->ttm.cached_io_st) { + sg_free_table(obj->ttm.cached_io_st); + kfree(obj->ttm.cached_io_st); + obj->ttm.cached_io_st = NULL; + } +} + +static void i915_ttm_purge(struct drm_i915_gem_object *obj) +{ + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); + + if (obj->mm.madv == __I915_MADV_PURGED) + return; + + i915_ttm_free_cached_io_st(obj); + + ttm_resource_free(bo, &bo->mem); + if (bo->ttm) { + ttm_tt_unpopulate(bo->bdev, bo->ttm); + bo->ttm->page_flags |= TTM_PAGE_FLAG_ZERO_ALLOC; + } + + obj->mm.madv = __I915_MADV_PURGED; +} + +static void i915_ttm_swap_notify(struct ttm_buffer_object *bo) +{ + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + int ret = i915_ttm_move_notify(bo); + + GEM_WARN_ON(ret); + GEM_WARN_ON(obj->ttm.cached_io_st); + if (!ret && obj->mm.madv != I915_MADV_WILLNEED) + i915_ttm_purge(obj); +} + +static void i915_ttm_delete_mem_notify(struct ttm_buffer_object *bo) +{ + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + + if (likely(obj)) { + /* This releases all gem object bindings to the backend. */ + __i915_gem_free_object(obj); + } +} + +static struct intel_memory_region * +i915_ttm_region(struct ttm_device *bdev, int ttm_mem_type) +{ + struct drm_i915_private *i915 = container_of(bdev, typeof(*i915), bdev); + + /* There's some room for optimization here... */ + GEM_BUG_ON(ttm_mem_type != I915_PL_SYSTEM && + ttm_mem_type < I915_PL_LMEM0); + if (ttm_mem_type == I915_PL_SYSTEM) + return intel_memory_region_lookup(i915, INTEL_MEMORY_SYSTEM, + 0); + + return intel_memory_region_lookup(i915, INTEL_MEMORY_LOCAL, + ttm_mem_type - I915_PL_LMEM0); +} + +static struct sg_table *i915_ttm_tt_get_st(struct ttm_tt *ttm) +{ + struct i915_ttm_tt *i915_tt = container_of(ttm, typeof(*i915_tt), ttm); + struct scatterlist *sg; + struct sg_table *st; + int ret; + + if (i915_tt->cached_st) + return i915_tt->cached_st; + + st = kzalloc(sizeof(*st), GFP_KERNEL); + if (!st) + return ERR_PTR(-ENOMEM); + + sg = __sg_alloc_table_from_pages + (st, ttm->pages, ttm->num_pages, 0, + (unsigned long)ttm->num_pages << PAGE_SHIFT, + i915_sg_segment_size(), NULL, 0, GFP_KERNEL); + if (IS_ERR(sg)) { + kfree(st); + return ERR_CAST(sg); + } + + ret = dma_map_sgtable(i915_tt->dev, st, DMA_BIDIRECTIONAL, 0); + if (ret) { + sg_free_table(st); + kfree(st); + return ERR_PTR(ret); + } + + i915_tt->cached_st = st; + return st; +} + +static struct sg_table * +i915_ttm_resource_get_st(struct drm_i915_gem_object *obj, + struct ttm_resource *res) +{ + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); + struct ttm_resource_manager *man = + ttm_manager_type(bo->bdev, res->mem_type); + + if (man->use_tt) + return i915_ttm_tt_get_st(bo->ttm); + + return intel_region_ttm_node_to_st(obj->mm.region, res->mm_node); +} + +static int i915_ttm_move(struct ttm_buffer_object *bo, bool evict, + struct ttm_operation_ctx *ctx, + struct ttm_resource *new_mem, + struct ttm_place *hop) +{ + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + struct ttm_resource_manager *new_man = + ttm_manager_type(bo->bdev, new_mem->mem_type); + struct ttm_resource_manager *old_man = + ttm_manager_type(bo->bdev, bo->mem.mem_type); + struct intel_memory_region *new_reg, *old_reg; + union { + struct i915_ttm_kmap_iter_tt tt; + struct i915_ttm_kmap_iter_iomap io; + } _new_iter, _old_iter; + struct i915_ttm_kmap_iter *new_iter, *old_iter; + struct sg_table *new_st; + int ret; + + new_reg = i915_ttm_region(bo->bdev, new_mem->mem_type); + old_reg = i915_ttm_region(bo->bdev, bo->mem.mem_type); + GEM_BUG_ON(!new_reg || !old_reg); + + /* Sync for now. We could do the actual copy async. */ + ret = ttm_bo_wait_ctx(bo, ctx); + if (ret) + return ret; + + ret = i915_ttm_move_notify(bo); + if (ret) + return ret; + + if (obj->mm.madv != I915_MADV_WILLNEED) { + i915_ttm_purge(obj); + ttm_resource_free(bo, new_mem); + return 0; + } + + /* Populate ttm with pages if needed. Typically system memory. */ + if (new_man->use_tt && bo->ttm) { + ret = ttm_tt_populate(bo->bdev, bo->ttm, ctx); + if (ret) + return ret; + } + + new_st = i915_ttm_resource_get_st(obj, new_mem); + if (IS_ERR(new_st)) + return PTR_ERR(new_st); + + new_iter = new_man->use_tt ? + i915_ttm_kmap_iter_tt_init(&_new_iter.tt, bo->ttm) : + i915_ttm_kmap_iter_iomap_init(&_new_iter.io, &new_reg->iomap, + new_st, new_reg->region.start); + + old_iter = old_man->use_tt ? + i915_ttm_kmap_iter_tt_init(&_old_iter.tt, bo->ttm) : + i915_ttm_kmap_iter_iomap_init(&_old_iter.io, &old_reg->iomap, + obj->ttm.cached_io_st, + old_reg->region.start); + + i915_ttm_move_memcpy(bo, new_mem, new_iter, old_iter); + i915_ttm_free_cached_io_st(obj); + + if (!new_man->use_tt) + obj->ttm.cached_io_st = new_st; + + return 0; +} + +struct ttm_device_funcs i915_ttm_bo_driver = { + .ttm_tt_create = i915_ttm_tt_create, + .ttm_tt_unpopulate = i915_ttm_tt_unpopulate, + .ttm_tt_destroy = i915_ttm_tt_destroy, + .eviction_valuable = i915_ttm_eviction_valuable, + .evict_flags = i915_ttm_evict_flags, + .move = i915_ttm_move, + .verify_access = NULL, + .swap_notify = i915_ttm_swap_notify, + .delete_mem_notify = i915_ttm_delete_mem_notify, +}; + +static int i915_ttm_get_pages(struct drm_i915_gem_object *obj) +{ + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); + struct ttm_operation_ctx ctx = { + .interruptible = true, + .no_wait_gpu = false, + }; + struct sg_table *st; + int ret; + + /* Swap in. */ + if (bo->ttm && (bo->ttm->page_flags & TTM_PAGE_FLAG_SWAPPED)) { + ret = ttm_tt_populate(bo->bdev, bo->ttm, &ctx); + if (ret) + return ret; + } + + /* Move to the requested placement. */ + ret = ttm_bo_validate(bo, &i915_lmem0_placement, &ctx); + if (ret) + return ret == -ENOSPC ? -ENXIO : ret; + + /* Object either has a page vector or is an iomem object */ + st = bo->ttm ? i915_ttm_tt_get_st(bo->ttm) : obj->ttm.cached_io_st; + if (IS_ERR(st)) + return PTR_ERR(st); + + __i915_gem_object_set_pages(obj, st, i915_sg_dma_sizes(st->sgl)); + + i915_ttm_adjust_lru(obj); + + return ret; +} + +static void i915_ttm_put_pages(struct drm_i915_gem_object *obj, + struct sg_table *st) +{ + /* + * We're currently not called from a shrinker, so put_pages() + * typically means the object is about to destroyed, or called + * from move_notify(). So just avoid doing much for now. + * If the object is not destroyed next, The TTM eviction logic + * and shrinkers will move it out if needed. + */ + + i915_ttm_adjust_lru(obj); +} + +static void i915_ttm_adjust_lru(struct drm_i915_gem_object *obj) +{ + struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); + + /* + * Don't manipulate the TTM LRUs while in TTM bo destruction. + * We're called through i915_ttm_delete_mem_notify(). + */ + if (!kref_read(&bo->kref)) + return; + + /* + * Put on the correct LRU list depending on the MADV status + */ + spin_lock(&bo->bdev->lru_lock); + if (obj->mm.madv != I915_MADV_WILLNEED) { + bo->priority = I915_TTM_PRIO_PURGE; + } else if (!i915_gem_object_has_pages(obj)) { + if (bo->priority < I915_TTM_PRIO_HAS_PAGES) + bo->priority = I915_TTM_PRIO_HAS_PAGES; + } else { + if (bo->priority > I915_TTM_PRIO_NO_PAGES) + bo->priority = I915_TTM_PRIO_NO_PAGES; + } + + ttm_bo_move_to_lru_tail(bo, &bo->mem, NULL); + spin_unlock(&bo->bdev->lru_lock); +} + +/* + * TTM-backed gem object destruction requires some clarification. + * Basically we have two possibilities here. We can either rely on the + * i915 delayed destruction and put the TTM object when the object + * is idle. This would be detected by TTM which would bypass the + * TTM delayed destroy handling. The other approach is to put the TTM + * object early and rely on the TTM destroyed handling, and then free + * the leftover parts of the GEM object once TTM's destroyed list handling is + * complete. For now, we rely on the latter for two reasons: + * a) TTM can evict an object even when it's on the delayed destroy list, + * which in theory allows for complete eviction. + * b) There is work going on in TTM to allow freeing an object even when + * it's not idle, and using the TTM destroyed list handling could help us + * benefit from that. + */ +static void i915_ttm_delayed_free(struct drm_i915_gem_object *obj) +{ + ttm_bo_put(i915_gem_to_ttm(obj)); +} + +static const struct drm_i915_gem_object_ops i915_gem_ttm_obj_ops = { + .name = "i915_gem_object_ttm", + .flags = I915_GEM_OBJECT_HAS_IOMEM, + + .get_pages = i915_ttm_get_pages, + .put_pages = i915_ttm_put_pages, + .truncate = i915_ttm_purge, + .adjust_lru = i915_ttm_adjust_lru, + .delayed_free = i915_ttm_delayed_free, +}; + +void i915_ttm_bo_destroy(struct ttm_buffer_object *bo) +{ + struct drm_i915_gem_object *obj = i915_ttm_to_gem(bo); + + i915_gem_object_release_memory_region(obj); + call_rcu(&obj->rcu, __i915_gem_free_object_rcu); +} + +/** + * __i915_gem_ttm_object_init - Initialize a ttm-backed i915 gem object + * @mem: The initial memory region for the object. + * @obj: The gem object. + * @size: Object size in bytes. + * @flags: gem object flags. + * + * Return: 0 on success, negative error code on failure. + */ +int __i915_gem_ttm_object_init(struct intel_memory_region *mem, + struct drm_i915_gem_object *obj, + resource_size_t size, + unsigned int flags) +{ + static struct lock_class_key lock_class; + struct drm_i915_private *i915 = mem->i915; + size_t alignment = 0; + int ret; + + /* Adjust alignment to GPU- and CPU huge page sizes. */ + + if (mem->is_range_manager) { + if (size >= SZ_1G) + alignment = SZ_1G >> PAGE_SHIFT; + else if (size >= SZ_2M) + alignment = SZ_2M >> PAGE_SHIFT; + else if (size >= SZ_64K) + alignment = SZ_64K >> PAGE_SHIFT; + } + + drm_gem_private_object_init(&i915->drm, &obj->base, size); + i915_gem_object_init(obj, &i915_gem_ttm_obj_ops, &lock_class, flags); + i915_gem_object_init_memory_region(obj, mem); + i915_gem_object_make_unshrinkable(obj); + obj->read_domains = I915_GEM_DOMAIN_WC | I915_GEM_DOMAIN_GTT; + i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE); + + ret = ttm_bo_init(&i915->bdev, i915_gem_to_ttm(obj), size, + ttm_bo_type_kernel, &i915_sys_placement, alignment, + true, NULL, NULL, i915_ttm_bo_destroy); + + /* i915 wants -ENXIO when out of memory region space. */ + return (ret == -ENOSPC) ? -ENXIO : ret; +} diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.h b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h new file mode 100644 index 000000000000..b8d3dcbb50df --- /dev/null +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2021 Intel Corporation + */ +#ifndef _I915_GEM_TTM_H_ +#define _I915_GEM_TTM_H_ + +#include "gem/i915_gem_object_types.h" + +/** + * i915_gem_to_ttm - Convert a struct drm_i915_gem_object to a + * struct ttm_buffer_object. + * @obj: Pointer to the gem object. + * + * Return: Pointer to the embedded struct ttm_buffer_object. + */ +static inline struct ttm_buffer_object * +i915_gem_to_ttm(struct drm_i915_gem_object *obj) +{ + return &obj->__do_not_access; +} + +/* + * i915 ttm gem object destructor. Internal use only. + */ +void i915_ttm_bo_destroy(struct ttm_buffer_object *bo); + +/** + * i915_ttm_to_gem - Convert a struct ttm_buffer_object to an embedding + * struct drm_i915_gem_object. + * + * Return: Pointer to the embedding struct ttm_buffer_object, or NULL + * if the object was not an i915 ttm object. + */ +static inline struct drm_i915_gem_object * +i915_ttm_to_gem(struct ttm_buffer_object *bo) +{ + if (GEM_WARN_ON(bo->destroy != i915_ttm_bo_destroy)) + return NULL; + + return container_of(bo, struct drm_i915_gem_object, __do_not_access); +} + +int __i915_gem_ttm_object_init(struct intel_memory_region *mem, + struct drm_i915_gem_object *obj, + resource_size_t size, + unsigned int flags); +#endif diff --git a/drivers/gpu/drm/i915/gt/intel_region_lmem.c b/drivers/gpu/drm/i915/gt/intel_region_lmem.c index f7366b054f8e..4ae1f717a94c 100644 --- a/drivers/gpu/drm/i915/gt/intel_region_lmem.c +++ b/drivers/gpu/drm/i915/gt/intel_region_lmem.c @@ -9,6 +9,7 @@ #include "intel_region_ttm.h" #include "gem/i915_gem_lmem.h" #include "gem/i915_gem_region.h" +#include "gem/i915_gem_ttm.h" #include "intel_region_lmem.h" static int init_fake_lmem_bar(struct intel_memory_region *mem) @@ -107,7 +108,7 @@ region_lmem_init(struct intel_memory_region *mem) static const struct intel_memory_region_ops intel_region_lmem_ops = { .init = region_lmem_init, .release = region_lmem_release, - .init_object = __i915_gem_lmem_object_init, + .init_object = __i915_gem_ttm_object_init, }; struct intel_memory_region * diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 180f6e9107d4..350283ab9a83 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1005,8 +1005,11 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data, } } - if (obj->mm.madv != __I915_MADV_PURGED) + if (obj->mm.madv != __I915_MADV_PURGED) { obj->mm.madv = args->madv; + if (obj->ops->adjust_lru) + obj->ops->adjust_lru(obj); + } if (i915_gem_object_has_pages(obj)) { struct list_head *list; diff --git a/drivers/gpu/drm/i915/intel_memory_region.c b/drivers/gpu/drm/i915/intel_memory_region.c index 4092cc987679..bd27e897d4d0 100644 --- a/drivers/gpu/drm/i915/intel_memory_region.c +++ b/drivers/gpu/drm/i915/intel_memory_region.c @@ -149,7 +149,6 @@ intel_memory_region_create(struct drm_i915_private *i915, mutex_init(&mem->objects.lock); INIT_LIST_HEAD(&mem->objects.list); - INIT_LIST_HEAD(&mem->objects.purgeable); INIT_LIST_HEAD(&mem->reserved); mutex_init(&mem->mm_lock); diff --git a/drivers/gpu/drm/i915/intel_memory_region.h b/drivers/gpu/drm/i915/intel_memory_region.h index e69cde13daf2..7b5fa97c0b59 100644 --- a/drivers/gpu/drm/i915/intel_memory_region.h +++ b/drivers/gpu/drm/i915/intel_memory_region.h @@ -100,7 +100,6 @@ struct intel_memory_region { struct { struct mutex lock; /* Protects access to objects */ struct list_head list; - struct list_head purgeable; } objects; size_t chunk_size; diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c index 9fb447e19ebd..e429c9a38371 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.c +++ b/drivers/gpu/drm/i915/intel_region_ttm.c @@ -10,8 +10,7 @@ #include "intel_region_ttm.h" -/* A Zero-initialized driver for now. We don't have a TTM backend yet. */ -static struct ttm_device_funcs i915_ttm_bo_driver; +extern struct ttm_device_funcs i915_ttm_bo_driver; /** * DOC: TTM support structure @@ -197,6 +196,7 @@ struct sg_table *intel_region_ttm_node_to_st(struct intel_memory_region *mem, return i915_sg_from_mm_node(node, mem->region.start); } +#ifdef CONFIG_DRM_I915_SELFTEST /** * intel_region_ttm_node_alloc - Allocate memory resources from a region * @mem: The memory region, @@ -243,3 +243,4 @@ void *intel_region_ttm_node_alloc(struct intel_memory_region *mem, ret = -ENXIO; return ret ? ERR_PTR(ret) : res.mm_node; } +#endif diff --git a/drivers/gpu/drm/i915/intel_region_ttm.h b/drivers/gpu/drm/i915/intel_region_ttm.h index 40129f7e0317..9a5b0437d73f 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.h +++ b/drivers/gpu/drm/i915/intel_region_ttm.h @@ -21,9 +21,12 @@ void intel_region_ttm_fini(struct intel_memory_region *mem); struct sg_table *intel_region_ttm_node_to_st(struct intel_memory_region *mem, void *node); +void intel_region_ttm_node_free(struct intel_memory_region *mem, + void *node); + +#ifdef CONFIG_DRM_I915_SELFTEST void *intel_region_ttm_node_alloc(struct intel_memory_region *mem, resource_size_t size, unsigned int flags); -void intel_region_ttm_node_free(struct intel_memory_region *mem, - void *node); +#endif #endif /* _INTEL_REGION_TTM_H_ */ -- 2.31.1