From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0805C44507 for ; Fri, 10 Jul 2026 21:54:50 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2987410E24C; Fri, 10 Jul 2026 21:54:50 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="CLH0vtjs"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2249010E1FB; Fri, 10 Jul 2026 21:54:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1783720489; x=1815256489; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R87ztW4hRcwi/qx6zKcQyXSRxBMwbHa7YP+KfdMZRa0=; b=CLH0vtjscsgUXv6RnbreAWFEOt6p2x8o7WR9djQmLfh9MTwPqyLOkXAn nEKuxnYkzoP6FYe8taXMAu4F0hNTVxs3xigDKUCtRTQr3whHxA8/pwn0y ITa/2MxyeKc8gxSX0SWLWnPKD+M/46R6njYAS7I4xJL4Y8S5SdjlPGoB1 1dgGnapE+Kmi/0HRYnDdpVUMoC5nWen6WYjRisxKhzL7T2a7UtcxswH8d jL8LMZ2rje5hDBYAiDdDexPEUvXvagYgPta0f8FAD1RkCd7/K14bsyaGV 3LGCzWb0Zfd/Ra3bzaJgBmooxRv2JX0CY9xQ2+TBCN1d97tIV9KE59KXv Q==; X-CSE-ConnectionGUID: 1xSZppGDR8Ksxv6odKUgrA== X-CSE-MsgGUID: CdLNwQh/QHGptRTnGDxTgw== X-IronPort-AV: E=McAfee;i="6800,10657,11841"; a="83543070" X-IronPort-AV: E=Sophos;i="6.25,154,1779174000"; d="scan'208";a="83543070" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jul 2026 14:54:49 -0700 X-CSE-ConnectionGUID: qWXALh5xR7Cgit2bFrMWgQ== X-CSE-MsgGUID: Cu6A3+UbRIOxaJX7TgPC1Q== X-ExtLoop1: 1 Received: from gsse-cloud1.jf.intel.com ([10.54.39.91]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jul 2026 14:54:48 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: Carlos Santa , Ryan Neph , Christian Koenig , Huang Rui , Matthew Auld , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , linux-kernel@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Subject: [PATCH v2 04/33] drm/ttm: Introduce ttm_pool_alloc_iter for __ttm_pool_alloc() Date: Fri, 10 Jul 2026 14:54:13 -0700 Message-Id: <20260710215442.2444235-5-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260710215442.2444235-1-matthew.brost@intel.com> References: <20260710215442.2444235-1-matthew.brost@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" __ttm_pool_alloc() walks the page orders in a single loop whose body is about to grow several defragmentation phases (reusing old pages, a move budget, out-of-lock preallocation and fault injection). Left inline these interleave into one deeply nested block of gotos and combinatorial conditionals that is hard to follow and extend. Factor the loop's working state into a struct ttm_pool_alloc_iter and pull the per-order page acquisition and order-fallback into small phase helpers, so the main loop reads as a short sequence of named steps. This is a pure refactor with no functional change; subsequent patches extend the iterator with their defrag phases rather than nesting more logic into the loop. Cc: Carlos Santa Cc: Ryan Neph Cc: Christian Koenig Cc: Huang Rui Cc: Matthew Auld Cc: Maarten Lankhorst Cc: Maxime Ripard Cc: Thomas Zimmermann Cc: David Airlie Cc: Simona Vetter Cc: dri-devel@lists.freedesktop.org Cc: linux-kernel@vger.kernel.org Cc: Thomas Hellström Assisted-by: GitHub_Copilot:claude-opus-4.8 Signed-off-by: Matthew Brost --- drivers/gpu/drm/ttm/ttm_pool.c | 159 ++++++++++++++++++++++----------- 1 file changed, 106 insertions(+), 53 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index 370d991c9882..6826c8ae6fc5 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -791,18 +791,102 @@ static unsigned int ttm_pool_alloc_find_order(unsigned int highest, return min_t(unsigned int, highest, __fls(alloc->remaining_pages)); } +/** + * struct ttm_pool_alloc_iter - Working state for the __ttm_pool_alloc() loop + * + * Bundles the immutable inputs and the mutable per-order state that the + * allocation phase helpers share, so the main loop reads as a short sequence + * of named phases rather than one deeply nested block. + */ +struct ttm_pool_alloc_iter { + /* Immutable for the duration of the allocation. */ + struct ttm_pool *pool; + struct ttm_tt *tt; + const struct ttm_operation_ctx *ctx; + struct ttm_pool_alloc_state *alloc; + struct ttm_pool_tt_restore *restore; + unsigned int beneficial_order; + gfp_t gfp_flags; + + /* Mutated as the loop walks the orders. */ + unsigned int order; + enum ttm_caching page_caching; + bool allow_pools; + struct page *p; +}; + +/* + * Acquire a single page for the current order, leaving it in @it->p (NULL on + * failure). Tries a same-order pool page, then a fresh system allocation. + */ +static void ttm_pool_iter_acquire_page(struct ttm_pool_alloc_iter *it) +{ + struct ttm_pool_type *pt; + + /* First, try to allocate a page from a pool if one exists. */ + it->p = NULL; + pt = ttm_pool_select_type(it->pool, it->page_caching, it->order); + if (!it->p && pt && it->allow_pools) + it->p = ttm_pool_type_take(pt, ttm_pool_nid(it->pool)); + + /* + * If that fails or previously failed, allocate from system. Note that + * this also disallows additional pool allocations using write-back + * cached pools of the same order. + */ + if (!it->p) { + it->page_caching = ttm_cached; + it->allow_pools = false; + it->p = ttm_pool_alloc_page(it->pool, it->gfp_flags, it->order, + it->ctx->beneficial_reclaim_backoff); + } +} + +/* + * The current order could not be satisfied. Lower it and retry, recording a + * sub-optimal backing (so the driver can later re-defrag) when we drop below + * the beneficial order. + * + * Returns 0 to retry the (lowered) order, or a negative errno. + */ +static int ttm_pool_iter_lower_order(struct ttm_pool_alloc_iter *it) +{ + if (!it->order) + return -ENOMEM; + + /* + * Failing to allocate at the device's beneficial order means we are + * about to back this object with a sub-optimal (smaller order) set of + * pages. Record it so the driver can later try to defragment the object + * back to beneficial order. + */ + if (it->beneficial_order && it->order == it->beneficial_order) + it->tt->page_flags |= TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED; + + it->order--; + it->page_caching = it->tt->caching; + it->allow_pools = true; + + return 0; +} + static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, const struct ttm_operation_ctx *ctx, struct ttm_pool_alloc_state *alloc, struct ttm_pool_tt_restore *restore) { - const unsigned int beneficial_order = ttm_pool_beneficial_order(pool); - enum ttm_caching page_caching; + struct ttm_pool_alloc_iter it = { + .pool = pool, + .tt = tt, + .ctx = ctx, + .alloc = alloc, + .restore = restore, + .beneficial_order = ttm_pool_beneficial_order(pool), + .page_caching = tt->caching, + .allow_pools = true, + }; gfp_t gfp_flags = GFP_USER; pgoff_t caching_divide; - unsigned int order; - bool allow_pools; - struct page *p; int r; WARN_ON(!alloc->remaining_pages || ttm_tt_is_populated(tt)); @@ -819,59 +903,28 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, else gfp_flags |= GFP_HIGHUSER; - page_caching = tt->caching; - allow_pools = true; - for (order = ttm_pool_alloc_find_order(MAX_PAGE_ORDER, alloc); + it.gfp_flags = gfp_flags; + + for (it.order = ttm_pool_alloc_find_order(MAX_PAGE_ORDER, alloc); alloc->remaining_pages; - order = ttm_pool_alloc_find_order(order, alloc)) { - struct ttm_pool_type *pt; - - /* First, try to allocate a page from a pool if one exists. */ - p = NULL; - pt = ttm_pool_select_type(pool, page_caching, order); - if (pt && allow_pools) - p = ttm_pool_type_take(pt, ttm_pool_nid(pool)); - - /* - * If that fails or previously failed, allocate from system. - * Note that this also disallows additional pool allocations using - * write-back cached pools of the same order. Consider removing - * that behaviour. - */ - if (!p) { - page_caching = ttm_cached; - allow_pools = false; - p = ttm_pool_alloc_page(pool, gfp_flags, order, - ctx->beneficial_reclaim_backoff); - } - /* If that fails, lower the order if possible and retry. */ - if (!p) { - if (order) { - /* - * Failing to allocate at the device's beneficial - * order means we are about to back this object - * with a sub-optimal (smaller order) set of - * pages. Record it so the driver can later try to - * defragment the object back to beneficial order. - */ - if (beneficial_order && order == beneficial_order) - tt->page_flags |= - TTM_TT_FLAG_BENEFICIAL_ORDER_FAILED; - --order; - page_caching = tt->caching; - allow_pools = true; - continue; - } - r = -ENOMEM; - goto error_free_all; + it.order = ttm_pool_alloc_find_order(it.order, alloc)) { + /* Acquire a page (pool / system) for this order. */ + ttm_pool_iter_acquire_page(&it); + if (!it.p) { + r = ttm_pool_iter_lower_order(&it); + if (r) + goto error_free_all; + continue; } - r = ttm_pool_page_allocated(pool, order, p, page_caching, alloc, - restore); + + r = ttm_pool_page_allocated(pool, it.order, it.p, + it.page_caching, alloc, restore); if (r) goto error_free_page; if (ttm_pool_restore_valid(restore)) { - r = ttm_pool_restore_commit(restore, tt->backup, ctx, alloc); + r = ttm_pool_restore_commit(restore, tt->backup, ctx, + alloc); if (r) goto error_free_all; } @@ -887,7 +940,7 @@ static int __ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, return 0; error_free_page: - ttm_pool_free_page(pool, page_caching, order, p, false); + ttm_pool_free_page(pool, it.page_caching, it.order, it.p, false); error_free_all: if (tt->restore) -- 2.34.1