From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CF13C2B9F4 for ; Tue, 22 Jun 2021 11:00:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE83F61351 for ; Tue, 22 Jun 2021 11:00:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE83F61351 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 60C8C6E499; Tue, 22 Jun 2021 11:00:46 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6583E6E499; Tue, 22 Jun 2021 11:00:45 +0000 (UTC) IronPort-SDR: Szqwv/mNpOap/pobqKqvWS5w7rEjGKvSju+JY1OlwYgjbUXfBtSloIhWyIIB8F2ZknndHTyGs2 OPnzAX/pOlpA== X-IronPort-AV: E=McAfee;i="6200,9189,10022"; a="194170832" X-IronPort-AV: E=Sophos;i="5.83,291,1616482800"; d="scan'208";a="194170832" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2021 04:00:43 -0700 IronPort-SDR: 7xTR7/OMbHV//Rxd+Reh5Vj6pOzQdki2c+81IKh3uu6+zHSZt8WaHO6QavXYSQDb6QkODMuO4p RJquB3TLXeIQ== X-IronPort-AV: E=Sophos;i="5.83,291,1616482800"; d="scan'208";a="454217316" Received: from clanggaa-mobl1.ger.corp.intel.com (HELO [10.249.254.95]) ([10.249.254.95]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jun 2021 04:00:41 -0700 Subject: Re: [PATCH v7 3/3] drm/i915/ttm: Use TTM for system memory To: Matthew Auld References: <20210622093418.153400-1-thomas.hellstrom@linux.intel.com> <20210622093418.153400-4-thomas.hellstrom@linux.intel.com> From: =?UTF-8?Q?Thomas_Hellstr=c3=b6m?= Message-ID: Date: Tue, 22 Jun 2021 13:00:38 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel Graphics Development , Matthew Auld , ML dri-devel Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On 6/22/21 12:55 PM, Matthew Auld wrote: > On Tue, 22 Jun 2021 at 10:34, Thomas Hellström > wrote: >> For discrete, use TTM for both cached and WC system memory. That means >> we currently rely on the TTM memory accounting / shrinker. For cached >> system memory we should consider remaining shmem-backed, which can be >> implemented from our ttm_tt_populate callback. We can then also reuse our >> own very elaborate shrinker for that memory. >> >> Signed-off-by: Thomas Hellström >> Reviewed-by: Matthew Auld >> --- >> v2: >> - Fix IS_ERR_OR_NULL() check to IS_ERR() (Reported by Matthew Auld) >> v3: >> - Commit message typo fix >> v6: >> - Fix TODO:s for supporting system memory with TTM. >> - Update the object GEM region after a TTM move if compatible. >> - Add a couple of warnings for shmem on DGFX. >> --- >> drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 3 ++ >> drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 51 +++++++++++++++++----- >> drivers/gpu/drm/i915/i915_drv.h | 3 -- >> drivers/gpu/drm/i915/intel_memory_region.c | 7 ++- >> drivers/gpu/drm/i915/intel_memory_region.h | 8 ++++ >> 5 files changed, 58 insertions(+), 14 deletions(-) >> >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c >> index 7aa1c95c7b7d..3648ae1d6628 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c >> @@ -284,6 +284,7 @@ __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj, >> bool needs_clflush) >> { >> GEM_BUG_ON(obj->mm.madv == __I915_MADV_PURGED); >> + GEM_WARN_ON(IS_DGFX(to_i915(obj->base.dev))); >> >> if (obj->mm.madv == I915_MADV_DONTNEED) >> obj->mm.dirty = false; >> @@ -302,6 +303,7 @@ void i915_gem_object_put_pages_shmem(struct drm_i915_gem_object *obj, struct sg_ >> struct pagevec pvec; >> struct page *page; >> >> + GEM_WARN_ON(IS_DGFX(to_i915(obj->base.dev))); >> __i915_gem_object_release_shmem(obj, pages, true); >> >> i915_gem_gtt_finish_pages(obj, pages); >> @@ -560,6 +562,7 @@ i915_gem_object_create_shmem_from_data(struct drm_i915_private *dev_priv, >> resource_size_t offset; >> int err; >> >> + GEM_WARN_ON(IS_DGFX(dev_priv)); >> obj = i915_gem_object_create_shmem(dev_priv, round_up(size, PAGE_SIZE)); >> if (IS_ERR(obj)) >> return obj; >> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c >> index 966b292d07da..07097f150065 100644 >> --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c >> +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c >> @@ -286,6 +286,25 @@ static void i915_ttm_adjust_gem_after_move(struct drm_i915_gem_object *obj) >> { >> struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); >> unsigned int cache_level; >> + unsigned int i; >> + >> + /* >> + * If object was moved to an allowable region, update the object >> + * region to consider it migrated. Note that if it's currently not >> + * in an allowable region, it's evicted and we don't update the >> + * object region. >> + */ >> + if (intel_region_to_ttm_type(obj->mm.region) != bo->resource->mem_type) { >> + for (i = 0; i < obj->mm.n_placements; ++i) { >> + struct intel_memory_region *mr = obj->mm.placements[i]; >> + >> + if (intel_region_to_ttm_type(mr) == bo->resource->mem_type && >> + mr != obj->mm.region) { >> + intel_memory_region_put(obj->mm.region); >> + obj->mm.region = intel_memory_region_get(mr); > break;? > > i915_gem_object_{init, release}_memory_region? > > There is also the region_link stuff, but I guess we can nuke that? Ah, yes, I'll fix that up. I think we will actually need that for suspend/resume, as the TTM LRU lists aren't sufficient... Thanks for reviewing! /Thomas