From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C1043CC332 for ; Fri, 10 Jul 2026 21:54:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783720493; cv=none; b=fXwCFqttBzJzCpiwlhtHechDRH/OpkYc/tGflmJf7PYU9PDGEeBKOeR+NVgiEEgxhmiSQIFWxGGId+AUpCV/qg9wptWVmaSUXYfpgV/O/egI+Kvnh21i+JjLyTc6fED2cB6HNNqECT8clgxJOxuBZBB+R/+62scZHli2pzR+oeg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783720493; c=relaxed/simple; bh=R87ztW4hRcwi/qx6zKcQyXSRxBMwbHa7YP+KfdMZRa0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=kSRiNDuNvk2tLDUigiIkPGWSehO/WaOYQG1VVzNezAcX+AzjVwcbpdQYaAe+a2p9qxk0YftiV49EeExs3CO+aPUich3RixYiPw6XSm/1BgoPXLqjAU+tsplphK/VlbqdfvCZRLrGtFrzsybhu6JE3RvHwLPM8PH/kILrNGcPLOE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UP5nKdag; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UP5nKdag" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1783720491; x=1815256491; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R87ztW4hRcwi/qx6zKcQyXSRxBMwbHa7YP+KfdMZRa0=; b=UP5nKdagdoUp64/QTpKFaom3CLPv0tyUVO59COjb7hYK7H8zjjuDFCzV GOU4x0pHcKwzfjSmt13Jz4y1nGiVQg5JiPKorqfGMmMlHPYxzpgydpWyk JhDwCkyh4ByWDcXlIuJR5D4h8uQOjjNgXgGZOusAIIAFCirNYZR95o32I L+LHnYWt0+wEV+pKJ9JTdfDPYSO899LhqNFuN0mok+A7Wr85skv9b5zVv mlCDVa+4HTEfDhqto1RAz5xzQrTtbv8ufLWMAnzFpeh9OYuHKzu6233uR mGQ+n7jn2LGLubHChyIS8yFre/uxKf/b9EvHOlUD69NXdYXieFz4YQXno w==; X-CSE-ConnectionGUID: dIwU+Zi+SsKZb3h3Gz1C/Q== X-CSE-MsgGUID: PXVH8tDKSx6qpcRiZmEU0A== X-IronPort-AV: E=McAfee;i="6800,10657,11841"; a="83543065" X-IronPort-AV: E=Sophos;i="6.25,154,1779174000"; d="scan'208";a="83543065" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jul 2026 14:54:49 -0700 X-CSE-ConnectionGUID: qWXALh5xR7Cgit2bFrMWgQ== X-CSE-MsgGUID: Cu6A3+UbRIOxaJX7TgPC1Q== X-ExtLoop1: 1 Received: from gsse-cloud1.jf.intel.com ([10.54.39.91]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jul 2026 14:54:48 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: Carlos Santa , Ryan Neph , Christian Koenig , Huang Rui , Matthew Auld , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , linux-kernel@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Subject: [PATCH v2 04/33] drm/ttm: Introduce ttm_pool_alloc_iter for __ttm_pool_alloc() Date: Fri, 10 Jul 2026 14:54:13 -0700 Message-Id: <20260710215442.2444235-5-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260710215442.2444235-1-matthew.brost@intel.com> References: <20260710215442.2444235-1-matthew.brost@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit __ttm_pool_alloc() walks the page orders in a single loop whose body is about to grow several defragmentation phases (reusing old pages, a move budget, out-of-lock preallocation and fault injection). Left inline these interleave into one deeply nested block of gotos and combinatorial conditionals that is hard to follow and extend. Factor the loop's working state into a struct ttm_pool_alloc_iter and pull the per-order page acquisition and order-fallback into small phase helpers, so the main loop reads as a short sequence of named steps. This is a pure refactor with no functional change; subsequent patches extend the iterator with their defrag phases rather than nesting more logic into the loop. Cc: Carlos Santa Cc: Ryan Neph Cc: Christian Koenig Cc: Huang Rui Cc: Matthew Auld Cc: Maarten Lankhorst Cc: Maxime Ripard Cc: Thomas Zimmermann Cc: David Airlie Cc: Simona Vetter Cc: dri-devel@lists.freedesktop.org Cc: linux-kernel@vger.kernel.org Cc: Thomas Hellström Assisted-by: GitHub_Copilot:claude-opus-4.8 Signed-off-by: Matthew Brost --- drivers/gpu/drm/ttm/ttm_pool.c | 159 ++++++++++++++++++++++----------- 1 file changed, 106 insertions(+), 53 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index 370d991c9882..6826c8ae6fc5 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -791,18 +791,102 @@ static unsigned int ttm_pool_alloc_find_order(unsigned int highest, return min_t(unsigned int, highest, __fls(alloc->remaining_pages)); } +/** + * struct ttm_pool_alloc_iter - Working state for the __ttm_pool_alloc() loop + * + * Bundles the immutable inputs and the mutable per-order state that the + * allocation phase helpers share, so the main loop reads as a short sequence + * of named phases rather than one deeply nested block. + */ +struct ttm_pool_alloc_iter { + /* Immutable for the duration of the allocation. */ + struct ttm_pool *pool; + struct ttm_tt *tt; + const struct ttm_operation_ctx *ctx; + struct ttm_pool_alloc_state *alloc; + struct ttm_pool_tt_restore *restore; + unsigned int beneficial_order; + gfp_t gfp_flags; + + /* Mutated as the loop walks the orders. */ + unsigned int order; + enum ttm_caching page_caching; + bool allow_pools; + struct page *p; +}; + +/* + * Acquire a single page for the current order, leaving it in @it->p (NULL on + * failure). Tries a same-order pool page, then a fresh system allocation. + */ +static void ttm_pool_iter_acquire_page(struct ttm_pool_alloc_iter *it) +{ + struct ttm_pool_type *pt; + + /* First, try to allocate a page from a pool if one exists. */ + it->p = NULL; + pt = ttm_pool_select_type(it->pool, it->page_caching, it->order); + if (!it->p && pt && it->allow_pools) + it->p = ttm_pool_type_take(pt, ttm_pool_nid(it->pool)); + + /* + * If that fails or previously failed, allocate from system. Note that + * this also disallows additional pool allocations using write-back + * cached pools of the same order. + */ + if (!it->p) { + it->page_caching = ttm_cached; + it->allow_pools = false; + it->p = ttm_pool_alloc_page(it->pool, it->gfp_flags, it->order, + it->ctx->beneficial_reclaim_backoff); + } +} + +/* + * The current order could not be satisfied. Lower it and retry, recording a + * sub-optimal backing (so the driver can later re-defrag) when we drop below + * the beneficial order. + * + * Returns 0 to retry the (lowered) order, or a negative errno. + */ +static int ttm_pool_iter_lower_order(struct ttm_pool_alloc_iter *it) +{ + if (!it->order) + return -ENOMEM; + + /* + * Failing to allocate at the device's beneficial order means we are + * about to back this object with a sub-optimal (smaller order) set of + * pages. Record it so the driver can later try to defragment the object + * back to beneficial order. + */ + if (it->beneficial_order && it->order == it->beneficial_order) + it->tt->page_flags |= TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED; + + it->order--; + it->page_caching = it->tt->caching; + it->allow_pools = true; + + return 0; +} + static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, const struct ttm_operation_ctx *ctx, struct ttm_pool_alloc_state *alloc, struct ttm_pool_tt_restore *restore) { - const unsigned int beneficial_order = ttm_pool_beneficial_order(pool); - enum ttm_caching page_caching; + struct ttm_pool_alloc_iter it = { + .pool = pool, + .tt = tt, + .ctx = ctx, + .alloc = alloc, + .restore = restore, + .beneficial_order = ttm_pool_beneficial_order(pool), + .page_caching = tt->caching, + .allow_pools = true, + }; gfp_t gfp_flags = GFP_USER; pgoff_t caching_divide; - unsigned int order; - bool allow_pools; - struct page *p; int r; WARN_ON(!alloc->remaining_pages || ttm_tt_is_populated(tt)); @@ -819,59 +903,28 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, else gfp_flags |= GFP_HIGHUSER; - page_caching = tt->caching; - allow_pools = true; - for (order = ttm_pool_alloc_find_order(MAX_PAGE_ORDER, alloc); + it.gfp_flags = gfp_flags; + + for (it.order = ttm_pool_alloc_find_order(MAX_PAGE_ORDER, alloc); alloc->remaining_pages; - order = ttm_pool_alloc_find_order(order, alloc)) { - struct ttm_pool_type *pt; - - /* First, try to allocate a page from a pool if one exists. */ - p = NULL; - pt = ttm_pool_select_type(pool, page_caching, order); - if (pt && allow_pools) - p = ttm_pool_type_take(pt, ttm_pool_nid(pool)); - - /* - * If that fails or previously failed, allocate from system. - * Note that this also disallows additional pool allocations using - * write-back cached pools of the same order. Consider removing - * that behaviour. - */ - if (!p) { - page_caching = ttm_cached; - allow_pools = false; - p = ttm_pool_alloc_page(pool, gfp_flags, order, - ctx->beneficial_reclaim_backoff); - } - /* If that fails, lower the order if possible and retry. */ - if (!p) { - if (order) { - /* - * Failing to allocate at the device's beneficial - * order means we are about to back this object - * with a sub-optimal (smaller order) set of - * pages. Record it so the driver can later try to - * defragment the object back to beneficial order. - */ - if (beneficial_order && order == beneficial_order) - tt->page_flags |= - TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED; - --order; - page_caching = tt->caching; - allow_pools = true; - continue; - } - r = -ENOMEM; - goto error_free_all; + it.order = ttm_pool_alloc_find_order(it.order, alloc)) { + /* Acquire a page (pool / system) for this order. */ + ttm_pool_iter_acquire_page(&it); + if (!it.p) { + r = ttm_pool_iter_lower_order(&it); + if (r) + goto error_free_all; + continue; } - r = ttm_pool_page_allocated(pool, order, p, page_caching, alloc, - restore); + + r = ttm_pool_page_allocated(pool, it.order, it.p, + it.page_caching, alloc, restore); if (r) goto error_free_page; if (ttm_pool_restore_valid(restore)) { - r = ttm_pool_restore_commit(restore, tt->backup, ctx, alloc); + r = ttm_pool_restore_commit(restore, tt->backup, ctx, + alloc); if (r) goto error_free_all; } @@ -887,7 +940,7 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, return 0; error_free_page: - ttm_pool_free_page(pool, page_caching, order, p, false); + ttm_pool_free_page(pool, it.page_caching, it.order, it.p, false); error_free_all: if (tt->restore) -- 2.34.1