[PATCH v3 0/5] gpu/buddy: Per-order free and used block scoreboards

[PATCH v3 0/5] gpu/buddy: Per-order free and used block scoreboards

[Francois Dugast]

drm_buddy_print() currently reports per-order free block counts by
walking all rbtrees, which is O(n) in the total number of free blocks
and holds the allocator lock for the duration. On large VRAM heaps with
many small fragments this becomes expensive.

This series replaces the rbtree walk with two lightweight scoreboard
arrays — free_scoreboard and used_scoreboard — indexed by order and
maintained incrementally at the points where block state transitions
occur. The print functions become simple array lookups, and drivers
reading debugfs (/sys/kernel/debug/dri/0/tile0/vram_mm) now get both
free and used counts per order at O(1) cost.

v3: Add __gpu_buddy_undo_splits() helper as suggested by Matt

v2: Add first patch to fix bug reported by Sashiko [1] then update
    following patches accordingly

Francois Dugast (5):
  gpu/buddy: Fix use-after-free in split_block() call sites
  gpu/buddy: Remove redundant condition in alloc_from_freetree() error
    path
  gpu/buddy: Introduce __gpu_buddy_undo_splits() helper
  gpu/buddy: Track per-order free blocks with a scoreboard
  gpu/buddy: Track per-order used blocks with a scoreboard

 drivers/gpu/buddy.c         | 104 ++++++++++++++++++++++--------------
 drivers/gpu/drm/drm_buddy.c |  30 +++++------
 include/linux/gpu_buddy.h   |  15 ++++++
 3 files changed, 90 insertions(+), 59 deletions(-)

-- 
2.43.0

[PATCH v3 1/5] gpu/buddy: Fix use-after-free in split_block() call sites

[Francois Dugast]

When split_block() fails it returns before calling mark_split(), leaving
the block in the FREE state and still linked in the rbtree.  The four
err_undo paths then call __gpu_buddy_free() without first removing the
block from the tree, which leads to two distinct bugs:

 - If the buddy is also free, __gpu_buddy_free() merges the two siblings
   by calling gpu_block_free(mm, block) while block->rb is still linked
   in the tree.  Any subsequent rbtree traversal will follow the now-
   dangling pointer, causing a use-after-free.

 - In alloc_from_freetree(), where there is no buddy guard,
   __gpu_buddy_free() always reaches mark_free() -> rbtree_insert() with
   block still in the tree, corrupting the rbtree.

The same pattern is already used correctly in __force_merge(): call
rbtree_remove() to unlink the block before handing it to
__gpu_buddy_free().  Apply the same fix to all four err_undo sites.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Assisted-by: GitHub Copilot:claude-sonnet-4.6
---
 drivers/gpu/buddy.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
index eb1457376307..dac2027bb64a 100644
--- a/drivers/gpu/buddy.c
+++ b/drivers/gpu/buddy.c
@@ -737,8 +737,10 @@ __alloc_range_bias(struct gpu_buddy *mm,
 	buddy = __get_buddy(block);
 	if (buddy &&
 	    (gpu_buddy_block_is_free(block) &&
-	     gpu_buddy_block_is_free(buddy)))
+	     gpu_buddy_block_is_free(buddy))) {
+		rbtree_remove(mm, block);
 		__gpu_buddy_free(mm, block, false);
+	}
 	return ERR_PTR(err);
 }
 
@@ -847,8 +849,10 @@ alloc_from_freetree(struct gpu_buddy *mm,
 	return block;
 
 err_undo:
-	if (tmp != order)
+	if (tmp != order) {
+		rbtree_remove(mm, block);
 		__gpu_buddy_free(mm, block, false);
+	}
 	return ERR_PTR(err);
 }
 
@@ -968,8 +972,10 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
 	buddy = __get_buddy(block);
 	if (buddy &&
 	    (gpu_buddy_block_is_free(block) &&
-	     gpu_buddy_block_is_free(buddy)))
+	     gpu_buddy_block_is_free(buddy))) {
+		rbtree_remove(mm, block);
 		__gpu_buddy_free(mm, block, false);
+	}
 	return ERR_PTR(err);
 }
 
@@ -1054,8 +1060,10 @@ static int __alloc_range(struct gpu_buddy *mm,
 	buddy = __get_buddy(block);
 	if (buddy &&
 	    (gpu_buddy_block_is_free(block) &&
-	     gpu_buddy_block_is_free(buddy)))
+	     gpu_buddy_block_is_free(buddy))) {
+		rbtree_remove(mm, block);
 		__gpu_buddy_free(mm, block, false);
+	}
 
 err_free:
 	if (err == -ENOSPC && total_allocated_on_err) {
-- 
2.43.0

[PATCH v3 2/5] gpu/buddy: Remove redundant condition in alloc_from_freetree() error path

[Francois Dugast]

The err_undo label in alloc_from_freetree() is only reachable via a
goto from inside the `while (tmp != order)` loop, which means tmp is
guaranteed to differ from order at that point. The surrounding
`if (tmp != order)` guard was therefore always true and can be dropped
without any behavioral change.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
---
 drivers/gpu/buddy.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
index dac2027bb64a..9f6696f47f89 100644
--- a/drivers/gpu/buddy.c
+++ b/drivers/gpu/buddy.c
@@ -849,10 +849,8 @@ alloc_from_freetree(struct gpu_buddy *mm,
 	return block;
 
 err_undo:
-	if (tmp != order) {
-		rbtree_remove(mm, block);
-		__gpu_buddy_free(mm, block, false);
-	}
+	rbtree_remove(mm, block);
+	__gpu_buddy_free(mm, block, false);
 	return ERR_PTR(err);
 }
 
-- 
2.43.0

[PATCH v3 3/5] gpu/buddy: Introduce __gpu_buddy_undo_splits() helper

[Francois Dugast]

The pattern of merging a block back with its buddy on error paths is
duplicated across multiple locations. Extract it into a
__gpu_buddy_undo_splits() helper to avoid repetition and prepare for
future changes.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Suggested-by: Matthew Auld <matthew.auld@intel.com>
---
 drivers/gpu/buddy.c | 43 +++++++++++++++++--------------------------
 1 file changed, 17 insertions(+), 26 deletions(-)

diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
index 9f6696f47f89..8654604b87a4 100644
--- a/drivers/gpu/buddy.c
+++ b/drivers/gpu/buddy.c
@@ -650,6 +650,19 @@ static bool block_incompatible(struct gpu_buddy_block *block, unsigned int flags
 	return needs_clear != gpu_buddy_block_is_clear(block);
 }
 
+static void __gpu_buddy_undo_splits(struct gpu_buddy *mm,
+				    struct gpu_buddy_block *block)
+{
+	struct gpu_buddy_block *buddy = __get_buddy(block);
+
+	if (buddy &&
+	    (gpu_buddy_block_is_free(block) &&
+	     gpu_buddy_block_is_free(buddy))) {
+		rbtree_remove(mm, block);
+		__gpu_buddy_free(mm, block, false);
+	}
+}
+
 static struct gpu_buddy_block *
 __alloc_range_bias(struct gpu_buddy *mm,
 		   u64 start, u64 end,
@@ -659,7 +672,6 @@ __alloc_range_bias(struct gpu_buddy *mm,
 {
 	u64 req_size = mm->chunk_size << order;
 	struct gpu_buddy_block *block;
-	struct gpu_buddy_block *buddy;
 	LIST_HEAD(dfs);
 	int err;
 	int i;
@@ -734,13 +746,7 @@ __alloc_range_bias(struct gpu_buddy *mm,
 	 * bigger is better, so make sure we merge everything back before we
 	 * free the allocated blocks.
 	 */
-	buddy = __get_buddy(block);
-	if (buddy &&
-	    (gpu_buddy_block_is_free(block) &&
-	     gpu_buddy_block_is_free(buddy))) {
-		rbtree_remove(mm, block);
-		__gpu_buddy_free(mm, block, false);
-	}
+	__gpu_buddy_undo_splits(mm, block);
 	return ERR_PTR(err);
 }
 
@@ -849,8 +855,7 @@ alloc_from_freetree(struct gpu_buddy *mm,
 	return block;
 
 err_undo:
-	rbtree_remove(mm, block);
-	__gpu_buddy_free(mm, block, false);
+	__gpu_buddy_undo_splits(mm, block);
 	return ERR_PTR(err);
 }
 
@@ -914,7 +919,6 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
 {
 	struct gpu_buddy_block *block = NULL;
 	unsigned int order, tmp, alignment;
-	struct gpu_buddy_block *buddy;
 	enum gpu_buddy_free_tree tree;
 	unsigned long pages;
 	int err;
@@ -967,13 +971,7 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
 	 * bigger is better, so make sure we merge everything back before we
 	 * free the allocated blocks.
 	 */
-	buddy = __get_buddy(block);
-	if (buddy &&
-	    (gpu_buddy_block_is_free(block) &&
-	     gpu_buddy_block_is_free(buddy))) {
-		rbtree_remove(mm, block);
-		__gpu_buddy_free(mm, block, false);
-	}
+	__gpu_buddy_undo_splits(mm, block);
 	return ERR_PTR(err);
 }
 
@@ -984,7 +982,6 @@ static int __alloc_range(struct gpu_buddy *mm,
 			 u64 *total_allocated_on_err)
 {
 	struct gpu_buddy_block *block;
-	struct gpu_buddy_block *buddy;
 	u64 total_allocated = 0;
 	LIST_HEAD(allocated);
 	u64 end;
@@ -1055,13 +1052,7 @@ static int __alloc_range(struct gpu_buddy *mm,
 	 * bigger is better, so make sure we merge everything back before we
 	 * free the allocated blocks.
 	 */
-	buddy = __get_buddy(block);
-	if (buddy &&
-	    (gpu_buddy_block_is_free(block) &&
-	     gpu_buddy_block_is_free(buddy))) {
-		rbtree_remove(mm, block);
-		__gpu_buddy_free(mm, block, false);
-	}
+	__gpu_buddy_undo_splits(mm, block);
 
 err_free:
 	if (err == -ENOSPC && total_allocated_on_err) {
-- 
2.43.0

[PATCH v3 4/5] gpu/buddy: Track per-order free blocks with a scoreboard

[Francois Dugast]

Reporting per-order free block counts in drm_buddy_print() currently
requires walking all rbtrees, which is O(n) over the total number of
free blocks and holds the allocator lock for the duration. This becomes
expensive on large VRAM heaps with many small free fragments.

Maintain a free_scoreboard[] array indexed by order instead, so that
the count for any order is always available in O(1). The scoreboard is
kept accurate by hooking into the four places where a block's free state
changes: mark_free(), mark_allocated(), mark_split(), and the sites in
__gpu_buddy_free(), __force_merge(), and the four err_undo paths that
call rbtree_remove() directly on free blocks without going through
mark_*().

The print functions are simplified as a result: the rbtree traversal
is replaced by a direct array lookup.

v3: Update after introducing __gpu_buddy_undo_splits() helper

v2: Update after fix for use-after-free in split_block() call sites

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Assisted-by: GitHub Copilot:claude-sonnet-4.6
---
 drivers/gpu/buddy.c         | 36 +++++++++++++++++++++---------------
 drivers/gpu/drm/drm_buddy.c | 16 ++--------------
 include/linux/gpu_buddy.h   |  7 +++++++
 3 files changed, 30 insertions(+), 29 deletions(-)

diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
index 8654604b87a4..de18b63fef0a 100644
--- a/drivers/gpu/buddy.c
+++ b/drivers/gpu/buddy.c
@@ -193,6 +193,8 @@ static void mark_allocated(struct gpu_buddy *mm,
 	block->header &= ~GPU_BUDDY_HEADER_STATE;
 	block->header |= GPU_BUDDY_ALLOCATED;
 
+	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
+
 	rbtree_remove(mm, block);
 }
 
@@ -204,6 +206,8 @@ static void mark_free(struct gpu_buddy *mm,
 	block->header &= ~GPU_BUDDY_HEADER_STATE;
 	block->header |= GPU_BUDDY_FREE;
 
+	mm->free_scoreboard[gpu_buddy_block_order(block)]++;
+
 	tree = get_block_tree(block);
 	rbtree_insert(mm, block, tree);
 }
@@ -214,6 +218,8 @@ static void mark_split(struct gpu_buddy *mm,
 	block->header &= ~GPU_BUDDY_HEADER_STATE;
 	block->header |= GPU_BUDDY_SPLIT;
 
+	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
+
 	rbtree_remove(mm, block);
 }
 
@@ -271,6 +277,7 @@ static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
 		}
 
 		rbtree_remove(mm, buddy);
+		mm->free_scoreboard[gpu_buddy_block_order(buddy)]--;
 		if (force_merge && gpu_buddy_block_is_clear(buddy))
 			mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
 
@@ -335,6 +342,7 @@ static int __force_merge(struct gpu_buddy *mm,
 					iter = rb_prev(iter);
 
 				rbtree_remove(mm, block);
+				mm->free_scoreboard[gpu_buddy_block_order(block)]--;
 				if (gpu_buddy_block_is_clear(block))
 					mm->clear_avail -= gpu_buddy_block_size(mm, block);
 
@@ -384,11 +392,17 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
 
 	BUG_ON(mm->max_order > GPU_BUDDY_MAX_ORDER);
 
+	mm->free_scoreboard = kcalloc(mm->max_order + 1,
+				      sizeof(*mm->free_scoreboard),
+				      GFP_KERNEL);
+	if (!mm->free_scoreboard)
+		return -ENOMEM;
+
 	mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
 				       sizeof(*mm->free_trees),
 				       GFP_KERNEL);
 	if (!mm->free_trees)
-		return -ENOMEM;
+		goto out_free_scoreboard;
 
 	for_each_free_tree(i) {
 		mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
@@ -450,6 +464,8 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
 	while (i--)
 		kfree(mm->free_trees[i]);
 	kfree(mm->free_trees);
+out_free_scoreboard:
+	kfree(mm->free_scoreboard);
 	return -ENOMEM;
 }
 EXPORT_SYMBOL(gpu_buddy_init);
@@ -488,6 +504,7 @@ void gpu_buddy_fini(struct gpu_buddy *mm)
 		kfree(mm->free_trees[i]);
 	kfree(mm->free_trees);
 	kfree(mm->roots);
+	kfree(mm->free_scoreboard);
 }
 EXPORT_SYMBOL(gpu_buddy_fini);
 
@@ -659,6 +676,7 @@ static void __gpu_buddy_undo_splits(struct gpu_buddy *mm,
 	    (gpu_buddy_block_is_free(block) &&
 	     gpu_buddy_block_is_free(buddy))) {
 		rbtree_remove(mm, block);
+		mm->free_scoreboard[gpu_buddy_block_order(block)]--;
 		__gpu_buddy_free(mm, block, false);
 	}
 }
@@ -1487,21 +1505,9 @@ void gpu_buddy_print(struct gpu_buddy *mm)
 		mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
 
 	for (order = mm->max_order; order >= 0; order--) {
-		struct gpu_buddy_block *block, *tmp;
-		struct rb_root *root;
-		u64 count = 0, free;
-		unsigned int tree;
-
-		for_each_free_tree(tree) {
-			root = &mm->free_trees[tree][order];
-
-			rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
-				BUG_ON(!gpu_buddy_block_is_free(block));
-				count++;
-			}
-		}
+		u64 count = mm->free_scoreboard[order];
+		u64 free = count * (mm->chunk_size << order);
 
-		free = count * (mm->chunk_size << order);
 		if (free < SZ_1M)
 			pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
 				order, free >> 10, count);
diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
index faa025498de4..eef995e08a37 100644
--- a/drivers/gpu/drm/drm_buddy.c
+++ b/drivers/gpu/drm/drm_buddy.c
@@ -47,23 +47,11 @@ void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
 		   mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
 
 	for (order = mm->max_order; order >= 0; order--) {
-		struct gpu_buddy_block *block, *tmp;
-		struct rb_root *root;
-		u64 count = 0, free;
-		unsigned int tree;
-
-		for_each_free_tree(tree) {
-			root = &mm->free_trees[tree][order];
-
-			rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
-				BUG_ON(!gpu_buddy_block_is_free(block));
-				count++;
-			}
-		}
+		u64 count = mm->free_scoreboard[order];
+		u64 free = count * (mm->chunk_size << order);
 
 		drm_printf(p, "order-%2d ", order);
 
-		free = count * (mm->chunk_size << order);
 		if (free < SZ_1M)
 			drm_printf(p, "free: %8llu KiB", free >> 10);
 		else
diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
index 71941a039648..a28f7d7637ca 100644
--- a/include/linux/gpu_buddy.h
+++ b/include/linux/gpu_buddy.h
@@ -173,6 +173,13 @@ struct gpu_buddy {
 	 * that fits in the remaining space.
 	 */
 	struct gpu_buddy_block **roots;
+	/*
+	 * Per-order free block scoreboard: free_scoreboard[order] holds the
+	 * number of blocks of that order currently in the free state.
+	 * Incremented in mark_free(), decremented wherever rbtree_remove() is
+	 * called on a free block.
+	 */
+	u64 *free_scoreboard;
 /* public: */
 	unsigned int n_roots;
 	unsigned int max_order;
-- 
2.43.0

[PATCH v3 5/5] gpu/buddy: Track per-order used blocks with a scoreboard

[Francois Dugast]

Extend the scoreboard approach from the previous commit to used blocks,
so drm_buddy_print() can report per-order allocation pressure in O(1).

Unlike free blocks, an allocated block can leave the allocated state
through mark_free() (normal free and gpu_buddy_block_trim()) or be
consumed directly by gpu_block_free() during coalescing. Both sites are
guarded by gpu_buddy_block_is_allocated() and paired with the increment
in mark_allocated().

v2:
- Update after fix for use-after-free in split_block() call sites
- Change goto label to out_free_used_scoreboard for clarity
- Make drm_buddy_print() and gpu_buddy_print() symmetric for used and
  free

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Assisted-by: GitHub Copilot:claude-sonnet-4.6
---
 drivers/gpu/buddy.c         | 39 +++++++++++++++++++++++++++----------
 drivers/gpu/drm/drm_buddy.c | 18 +++++++++++------
 include/linux/gpu_buddy.h   |  8 ++++++++
 3 files changed, 49 insertions(+), 16 deletions(-)

diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
index de18b63fef0a..f81d0d8fde15 100644
--- a/drivers/gpu/buddy.c
+++ b/drivers/gpu/buddy.c
@@ -194,6 +194,7 @@ static void mark_allocated(struct gpu_buddy *mm,
 	block->header |= GPU_BUDDY_ALLOCATED;
 
 	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
+	mm->used_scoreboard[gpu_buddy_block_order(block)]++;
 
 	rbtree_remove(mm, block);
 }
@@ -203,6 +204,9 @@ static void mark_free(struct gpu_buddy *mm,
 {
 	enum gpu_buddy_free_tree tree;
 
+	if (gpu_buddy_block_is_allocated(block))
+		mm->used_scoreboard[gpu_buddy_block_order(block)]--;
+
 	block->header &= ~GPU_BUDDY_HEADER_STATE;
 	block->header |= GPU_BUDDY_FREE;
 
@@ -281,6 +285,9 @@ static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
 		if (force_merge && gpu_buddy_block_is_clear(buddy))
 			mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
 
+		if (gpu_buddy_block_is_allocated(block))
+			mm->used_scoreboard[gpu_buddy_block_order(block)]--;
+
 		gpu_block_free(mm, block);
 		gpu_block_free(mm, buddy);
 
@@ -398,11 +405,17 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
 	if (!mm->free_scoreboard)
 		return -ENOMEM;
 
+	mm->used_scoreboard = kcalloc(mm->max_order + 1,
+				      sizeof(*mm->used_scoreboard),
+				      GFP_KERNEL);
+	if (!mm->used_scoreboard)
+		goto out_free_free_scoreboard;
+
 	mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
 				       sizeof(*mm->free_trees),
 				       GFP_KERNEL);
 	if (!mm->free_trees)
-		goto out_free_scoreboard;
+		goto out_free_used_scoreboard;
 
 	for_each_free_tree(i) {
 		mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
@@ -464,7 +477,9 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
 	while (i--)
 		kfree(mm->free_trees[i]);
 	kfree(mm->free_trees);
-out_free_scoreboard:
+out_free_used_scoreboard:
+	kfree(mm->used_scoreboard);
+out_free_free_scoreboard:
 	kfree(mm->free_scoreboard);
 	return -ENOMEM;
 }
@@ -505,6 +520,7 @@ void gpu_buddy_fini(struct gpu_buddy *mm)
 	kfree(mm->free_trees);
 	kfree(mm->roots);
 	kfree(mm->free_scoreboard);
+	kfree(mm->used_scoreboard);
 }
 EXPORT_SYMBOL(gpu_buddy_fini);
 
@@ -1505,15 +1521,18 @@ void gpu_buddy_print(struct gpu_buddy *mm)
 		mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
 
 	for (order = mm->max_order; order >= 0; order--) {
-		u64 count = mm->free_scoreboard[order];
-		u64 free = count * (mm->chunk_size << order);
-
-		if (free < SZ_1M)
-			pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
-				order, free >> 10, count);
+		u64 free_count = mm->free_scoreboard[order];
+		u64 used_count = mm->used_scoreboard[order];
+		u64 block_size = mm->chunk_size << order;
+		u64 free = free_count * block_size;
+		u64 used = used_count * block_size;
+
+		if (block_size < SZ_1M)
+			pr_info("order-%2d free: %8llu KiB, used: %8llu KiB, free_blocks: %llu, used_blocks: %llu\n",
+				order, free >> 10, used >> 10, free_count, used_count);
 		else
-			pr_info("order-%2d free: %8llu MiB, blocks: %llu\n",
-				order, free >> 20, count);
+			pr_info("order-%2d free: %8llu MiB, used: %8llu MiB, free_blocks: %llu, used_blocks: %llu\n",
+				order, free >> 20, used >> 20, free_count, used_count);
 	}
 }
 EXPORT_SYMBOL(gpu_buddy_print);
diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
index eef995e08a37..1536e59c6fe7 100644
--- a/drivers/gpu/drm/drm_buddy.c
+++ b/drivers/gpu/drm/drm_buddy.c
@@ -47,17 +47,23 @@ void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
 		   mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
 
 	for (order = mm->max_order; order >= 0; order--) {
-		u64 count = mm->free_scoreboard[order];
-		u64 free = count * (mm->chunk_size << order);
+		u64 free_count = mm->free_scoreboard[order];
+		u64 used_count = mm->used_scoreboard[order];
+		u64 block_size = mm->chunk_size << order;
+		u64 free = free_count * block_size;
+		u64 used = used_count * block_size;
 
 		drm_printf(p, "order-%2d ", order);
 
-		if (free < SZ_1M)
-			drm_printf(p, "free: %8llu KiB", free >> 10);
+		if (block_size < SZ_1M)
+			drm_printf(p, "free: %8llu KiB, used: %8llu KiB",
+				   free >> 10, used >> 10);
 		else
-			drm_printf(p, "free: %8llu MiB", free >> 20);
+			drm_printf(p, "free: %8llu MiB, used: %8llu MiB",
+				   free >> 20, used >> 20);
 
-		drm_printf(p, ", blocks: %llu\n", count);
+		drm_printf(p, ", free_blocks: %llu, used_blocks: %llu\n",
+			   free_count, used_count);
 	}
 }
 EXPORT_SYMBOL(drm_buddy_print);
diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
index a28f7d7637ca..e037714563d8 100644
--- a/include/linux/gpu_buddy.h
+++ b/include/linux/gpu_buddy.h
@@ -180,6 +180,14 @@ struct gpu_buddy {
 	 * called on a free block.
 	 */
 	u64 *free_scoreboard;
+	/*
+	 * Per-order used block scoreboard: used_scoreboard[order] holds the
+	 * number of blocks of that order currently in the allocated state.
+	 * Incremented in mark_allocated(), decremented in mark_free() (guarded
+	 * by gpu_buddy_block_is_allocated()) and in __gpu_buddy_free() when an
+	 * allocated block is consumed directly during buddy coalescing.
+	 */
+	u64 *used_scoreboard;
 /* public: */
 	unsigned int n_roots;
 	unsigned int max_order;
-- 
2.43.0

Re: [PATCH v3 1/5] gpu/buddy: Fix use-after-free in split_block() call sites

[Matthew Auld]

On 18/05/2026 15:14, Francois Dugast wrote:
> When split_block() fails it returns before calling mark_split(), leaving
> the block in the FREE state and still linked in the rbtree.  The four
> err_undo paths then call __gpu_buddy_free() without first removing the
> block from the tree, which leads to two distinct bugs:
> 
>   - If the buddy is also free, __gpu_buddy_free() merges the two siblings
>     by calling gpu_block_free(mm, block) while block->rb is still linked
>     in the tree.  Any subsequent rbtree traversal will follow the now-
>     dangling pointer, causing a use-after-free.
> 
>   - In alloc_from_freetree(), where there is no buddy guard,
>     __gpu_buddy_free() always reaches mark_free() -> rbtree_insert() with
>     block still in the tree, corrupting the rbtree.
> 
> The same pattern is already used correctly in __force_merge(): call
> rbtree_remove() to unlink the block before handing it to
> __gpu_buddy_free().  Apply the same fix to all four err_undo sites.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> Assisted-by: GitHub Copilot:claude-sonnet-4.6

Reviewed-by: Matthew Auld <matthew.auld@intel.com>

> ---
>   drivers/gpu/buddy.c | 16 ++++++++++++----
>   1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> index eb1457376307..dac2027bb64a 100644
> --- a/drivers/gpu/buddy.c
> +++ b/drivers/gpu/buddy.c
> @@ -737,8 +737,10 @@ __alloc_range_bias(struct gpu_buddy *mm,
>   	buddy = __get_buddy(block);
>   	if (buddy &&
>   	    (gpu_buddy_block_is_free(block) &&
> -	     gpu_buddy_block_is_free(buddy)))
> +	     gpu_buddy_block_is_free(buddy))) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   	return ERR_PTR(err);
>   }
>   
> @@ -847,8 +849,10 @@ alloc_from_freetree(struct gpu_buddy *mm,
>   	return block;
>   
>   err_undo:
> -	if (tmp != order)
> +	if (tmp != order) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   	return ERR_PTR(err);
>   }
>   
> @@ -968,8 +972,10 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
>   	buddy = __get_buddy(block);
>   	if (buddy &&
>   	    (gpu_buddy_block_is_free(block) &&
> -	     gpu_buddy_block_is_free(buddy)))
> +	     gpu_buddy_block_is_free(buddy))) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   	return ERR_PTR(err);
>   }
>   
> @@ -1054,8 +1060,10 @@ static int __alloc_range(struct gpu_buddy *mm,
>   	buddy = __get_buddy(block);
>   	if (buddy &&
>   	    (gpu_buddy_block_is_free(block) &&
> -	     gpu_buddy_block_is_free(buddy)))
> +	     gpu_buddy_block_is_free(buddy))) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   
>   err_free:
>   	if (err == -ENOSPC && total_allocated_on_err) {

Re: [PATCH v3 1/5] gpu/buddy: Fix use-after-free in split_block() call sites

[Matthew Auld]

On 18/05/2026 15:14, Francois Dugast wrote:
> When split_block() fails it returns before calling mark_split(), leaving
> the block in the FREE state and still linked in the rbtree.  The four
> err_undo paths then call __gpu_buddy_free() without first removing the
> block from the tree, which leads to two distinct bugs:
> 
>   - If the buddy is also free, __gpu_buddy_free() merges the two siblings
>     by calling gpu_block_free(mm, block) while block->rb is still linked
>     in the tree.  Any subsequent rbtree traversal will follow the now-
>     dangling pointer, causing a use-after-free.
> 
>   - In alloc_from_freetree(), where there is no buddy guard,
>     __gpu_buddy_free() always reaches mark_free() -> rbtree_insert() with
>     block still in the tree, corrupting the rbtree.
> 
> The same pattern is already used correctly in __force_merge(): call
> rbtree_remove() to unlink the block before handing it to
> __gpu_buddy_free().  Apply the same fix to all four err_undo sites.
> 
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> Assisted-by: GitHub Copilot:claude-sonnet-4.6
> ---
>   drivers/gpu/buddy.c | 16 ++++++++++++----
>   1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> index eb1457376307..dac2027bb64a 100644
> --- a/drivers/gpu/buddy.c
> +++ b/drivers/gpu/buddy.c
> @@ -737,8 +737,10 @@ __alloc_range_bias(struct gpu_buddy *mm,
>   	buddy = __get_buddy(block);
>   	if (buddy &&
>   	    (gpu_buddy_block_is_free(block) &&
> -	     gpu_buddy_block_is_free(buddy)))
> +	     gpu_buddy_block_is_free(buddy))) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   	return ERR_PTR(err);
>   }
>   
> @@ -847,8 +849,10 @@ alloc_from_freetree(struct gpu_buddy *mm,
>   	return block;
>   
>   err_undo:
> -	if (tmp != order)
> +	if (tmp != order) {
> +		rbtree_remove(mm, block);

Actually, I think this needs the same checking like elsewhere? Say we 
fail on the first split? Nothing was actually split, right?

>   		__gpu_buddy_free(mm, block, false);
> +	}
>   	return ERR_PTR(err);
>   }
>   
> @@ -968,8 +972,10 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
>   	buddy = __get_buddy(block);
>   	if (buddy &&
>   	    (gpu_buddy_block_is_free(block) &&
> -	     gpu_buddy_block_is_free(buddy)))
> +	     gpu_buddy_block_is_free(buddy))) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   	return ERR_PTR(err);
>   }
>   
> @@ -1054,8 +1060,10 @@ static int __alloc_range(struct gpu_buddy *mm,
>   	buddy = __get_buddy(block);
>   	if (buddy &&
>   	    (gpu_buddy_block_is_free(block) &&
> -	     gpu_buddy_block_is_free(buddy)))
> +	     gpu_buddy_block_is_free(buddy))) {
> +		rbtree_remove(mm, block);
>   		__gpu_buddy_free(mm, block, false);
> +	}
>   
>   err_free:
>   	if (err == -ENOSPC && total_allocated_on_err) {

Re: [PATCH v3 3/5] gpu/buddy: Introduce __gpu_buddy_undo_splits() helper

[Matthew Auld]

On 18/05/2026 15:14, Francois Dugast wrote:
> The pattern of merging a block back with its buddy on error paths is
> duplicated across multiple locations. Extract it into a
> __gpu_buddy_undo_splits() helper to avoid repetition and prepare for
> future changes.
> 
> Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> Suggested-by: Matthew Auld <matthew.auld@intel.com>

Reviewed-by: Matthew Auld <matthew.auld@intel.com>

Re: [PATCH v3 4/5] gpu/buddy: Track per-order free blocks with a scoreboard

[Matthew Auld]

On 18/05/2026 15:14, Francois Dugast wrote:
> Reporting per-order free block counts in drm_buddy_print() currently
> requires walking all rbtrees, which is O(n) over the total number of
> free blocks and holds the allocator lock for the duration. This becomes
> expensive on large VRAM heaps with many small free fragments.
> 
> Maintain a free_scoreboard[] array indexed by order instead, so that
> the count for any order is always available in O(1). The scoreboard is
> kept accurate by hooking into the four places where a block's free state
> changes: mark_free(), mark_allocated(), mark_split(), and the sites in
> __gpu_buddy_free(), __force_merge(), and the four err_undo paths that
> call rbtree_remove() directly on free blocks without going through
> mark_*().
> 
> The print functions are simplified as a result: the rbtree traversal
> is replaced by a direct array lookup.
> 
> v3: Update after introducing __gpu_buddy_undo_splits() helper
> 
> v2: Update after fix for use-after-free in split_block() call sites
> 
> Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> Assisted-by: GitHub Copilot:claude-sonnet-4.6

Reviewed-by: Matthew Auld <matthew.auld@intel.com>

> ---
>   drivers/gpu/buddy.c         | 36 +++++++++++++++++++++---------------
>   drivers/gpu/drm/drm_buddy.c | 16 ++--------------
>   include/linux/gpu_buddy.h   |  7 +++++++
>   3 files changed, 30 insertions(+), 29 deletions(-)
> 
> diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> index 8654604b87a4..de18b63fef0a 100644
> --- a/drivers/gpu/buddy.c
> +++ b/drivers/gpu/buddy.c
> @@ -193,6 +193,8 @@ static void mark_allocated(struct gpu_buddy *mm,
>   	block->header &= ~GPU_BUDDY_HEADER_STATE;
>   	block->header |= GPU_BUDDY_ALLOCATED;
>   
> +	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
> +
>   	rbtree_remove(mm, block);
>   }
>   
> @@ -204,6 +206,8 @@ static void mark_free(struct gpu_buddy *mm,
>   	block->header &= ~GPU_BUDDY_HEADER_STATE;
>   	block->header |= GPU_BUDDY_FREE;
>   
> +	mm->free_scoreboard[gpu_buddy_block_order(block)]++;
> +
>   	tree = get_block_tree(block);
>   	rbtree_insert(mm, block, tree);
>   }
> @@ -214,6 +218,8 @@ static void mark_split(struct gpu_buddy *mm,
>   	block->header &= ~GPU_BUDDY_HEADER_STATE;
>   	block->header |= GPU_BUDDY_SPLIT;
>   
> +	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
> +
>   	rbtree_remove(mm, block);
>   }
>   
> @@ -271,6 +277,7 @@ static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
>   		}
>   
>   		rbtree_remove(mm, buddy);
> +		mm->free_scoreboard[gpu_buddy_block_order(buddy)]--;
>   		if (force_merge && gpu_buddy_block_is_clear(buddy))
>   			mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
>   
> @@ -335,6 +342,7 @@ static int __force_merge(struct gpu_buddy *mm,
>   					iter = rb_prev(iter);
>   
>   				rbtree_remove(mm, block);
> +				mm->free_scoreboard[gpu_buddy_block_order(block)]--;
>   				if (gpu_buddy_block_is_clear(block))
>   					mm->clear_avail -= gpu_buddy_block_size(mm, block);
>   
> @@ -384,11 +392,17 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
>   
>   	BUG_ON(mm->max_order > GPU_BUDDY_MAX_ORDER);
>   
> +	mm->free_scoreboard = kcalloc(mm->max_order + 1,
> +				      sizeof(*mm->free_scoreboard),
> +				      GFP_KERNEL);
> +	if (!mm->free_scoreboard)
> +		return -ENOMEM;
> +
>   	mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
>   				       sizeof(*mm->free_trees),
>   				       GFP_KERNEL);
>   	if (!mm->free_trees)
> -		return -ENOMEM;
> +		goto out_free_scoreboard;
>   
>   	for_each_free_tree(i) {
>   		mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
> @@ -450,6 +464,8 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
>   	while (i--)
>   		kfree(mm->free_trees[i]);
>   	kfree(mm->free_trees);
> +out_free_scoreboard:
> +	kfree(mm->free_scoreboard);
>   	return -ENOMEM;
>   }
>   EXPORT_SYMBOL(gpu_buddy_init);
> @@ -488,6 +504,7 @@ void gpu_buddy_fini(struct gpu_buddy *mm)
>   		kfree(mm->free_trees[i]);
>   	kfree(mm->free_trees);
>   	kfree(mm->roots);
> +	kfree(mm->free_scoreboard);
>   }
>   EXPORT_SYMBOL(gpu_buddy_fini);
>   
> @@ -659,6 +676,7 @@ static void __gpu_buddy_undo_splits(struct gpu_buddy *mm,
>   	    (gpu_buddy_block_is_free(block) &&
>   	     gpu_buddy_block_is_free(buddy))) {
>   		rbtree_remove(mm, block);
> +		mm->free_scoreboard[gpu_buddy_block_order(block)]--;
>   		__gpu_buddy_free(mm, block, false);
>   	}
>   }
> @@ -1487,21 +1505,9 @@ void gpu_buddy_print(struct gpu_buddy *mm)
>   		mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
>   
>   	for (order = mm->max_order; order >= 0; order--) {
> -		struct gpu_buddy_block *block, *tmp;
> -		struct rb_root *root;
> -		u64 count = 0, free;
> -		unsigned int tree;
> -
> -		for_each_free_tree(tree) {
> -			root = &mm->free_trees[tree][order];
> -
> -			rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
> -				BUG_ON(!gpu_buddy_block_is_free(block));
> -				count++;
> -			}
> -		}
> +		u64 count = mm->free_scoreboard[order];
> +		u64 free = count * (mm->chunk_size << order);
>   
> -		free = count * (mm->chunk_size << order);
>   		if (free < SZ_1M)
>   			pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
>   				order, free >> 10, count);
> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
> index faa025498de4..eef995e08a37 100644
> --- a/drivers/gpu/drm/drm_buddy.c
> +++ b/drivers/gpu/drm/drm_buddy.c
> @@ -47,23 +47,11 @@ void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
>   		   mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
>   
>   	for (order = mm->max_order; order >= 0; order--) {
> -		struct gpu_buddy_block *block, *tmp;
> -		struct rb_root *root;
> -		u64 count = 0, free;
> -		unsigned int tree;
> -
> -		for_each_free_tree(tree) {
> -			root = &mm->free_trees[tree][order];
> -
> -			rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
> -				BUG_ON(!gpu_buddy_block_is_free(block));
> -				count++;
> -			}
> -		}
> +		u64 count = mm->free_scoreboard[order];
> +		u64 free = count * (mm->chunk_size << order);
>   
>   		drm_printf(p, "order-%2d ", order);
>   
> -		free = count * (mm->chunk_size << order);
>   		if (free < SZ_1M)
>   			drm_printf(p, "free: %8llu KiB", free >> 10);
>   		else
> diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
> index 71941a039648..a28f7d7637ca 100644
> --- a/include/linux/gpu_buddy.h
> +++ b/include/linux/gpu_buddy.h
> @@ -173,6 +173,13 @@ struct gpu_buddy {
>   	 * that fits in the remaining space.
>   	 */
>   	struct gpu_buddy_block **roots;
> +	/*
> +	 * Per-order free block scoreboard: free_scoreboard[order] holds the
> +	 * number of blocks of that order currently in the free state.
> +	 * Incremented in mark_free(), decremented wherever rbtree_remove() is
> +	 * called on a free block.
> +	 */
> +	u64 *free_scoreboard;
>   /* public: */
>   	unsigned int n_roots;
>   	unsigned int max_order;

Re: [PATCH v3 5/5] gpu/buddy: Track per-order used blocks with a scoreboard

[Matthew Auld]

On 18/05/2026 15:14, Francois Dugast wrote:
> Extend the scoreboard approach from the previous commit to used blocks,
> so drm_buddy_print() can report per-order allocation pressure in O(1).
> 
> Unlike free blocks, an allocated block can leave the allocated state
> through mark_free() (normal free and gpu_buddy_block_trim()) or be
> consumed directly by gpu_block_free() during coalescing. Both sites are
> guarded by gpu_buddy_block_is_allocated() and paired with the increment
> in mark_allocated().
> 
> v2:
> - Update after fix for use-after-free in split_block() call sites
> - Change goto label to out_free_used_scoreboard for clarity
> - Make drm_buddy_print() and gpu_buddy_print() symmetric for used and
>    free
> 
> Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> Assisted-by: GitHub Copilot:claude-sonnet-4.6

Could potentially also assert that used_scoreboard is empty at fini(), 
as a quick sanity check that nothing got leaked/missed with the 
accounting. Would also then be checked across the selftests.

Reviewed-by: Matthew Auld <matthew.auld@intel.com>

> ---
>   drivers/gpu/buddy.c         | 39 +++++++++++++++++++++++++++----------
>   drivers/gpu/drm/drm_buddy.c | 18 +++++++++++------
>   include/linux/gpu_buddy.h   |  8 ++++++++
>   3 files changed, 49 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> index de18b63fef0a..f81d0d8fde15 100644
> --- a/drivers/gpu/buddy.c
> +++ b/drivers/gpu/buddy.c
> @@ -194,6 +194,7 @@ static void mark_allocated(struct gpu_buddy *mm,
>   	block->header |= GPU_BUDDY_ALLOCATED;
>   
>   	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
> +	mm->used_scoreboard[gpu_buddy_block_order(block)]++;
>   
>   	rbtree_remove(mm, block);
>   }
> @@ -203,6 +204,9 @@ static void mark_free(struct gpu_buddy *mm,
>   {
>   	enum gpu_buddy_free_tree tree;
>   
> +	if (gpu_buddy_block_is_allocated(block))
> +		mm->used_scoreboard[gpu_buddy_block_order(block)]--;
> +
>   	block->header &= ~GPU_BUDDY_HEADER_STATE;
>   	block->header |= GPU_BUDDY_FREE;
>   
> @@ -281,6 +285,9 @@ static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
>   		if (force_merge && gpu_buddy_block_is_clear(buddy))
>   			mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
>   
> +		if (gpu_buddy_block_is_allocated(block))
> +			mm->used_scoreboard[gpu_buddy_block_order(block)]--;
> +
>   		gpu_block_free(mm, block);
>   		gpu_block_free(mm, buddy);
>   
> @@ -398,11 +405,17 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
>   	if (!mm->free_scoreboard)
>   		return -ENOMEM;
>   
> +	mm->used_scoreboard = kcalloc(mm->max_order + 1,
> +				      sizeof(*mm->used_scoreboard),
> +				      GFP_KERNEL);
> +	if (!mm->used_scoreboard)
> +		goto out_free_free_scoreboard;
> +
>   	mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
>   				       sizeof(*mm->free_trees),
>   				       GFP_KERNEL);
>   	if (!mm->free_trees)
> -		goto out_free_scoreboard;
> +		goto out_free_used_scoreboard;
>   
>   	for_each_free_tree(i) {
>   		mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
> @@ -464,7 +477,9 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
>   	while (i--)
>   		kfree(mm->free_trees[i]);
>   	kfree(mm->free_trees);
> -out_free_scoreboard:
> +out_free_used_scoreboard:
> +	kfree(mm->used_scoreboard);
> +out_free_free_scoreboard:
>   	kfree(mm->free_scoreboard);
>   	return -ENOMEM;
>   }
> @@ -505,6 +520,7 @@ void gpu_buddy_fini(struct gpu_buddy *mm)
>   	kfree(mm->free_trees);
>   	kfree(mm->roots);
>   	kfree(mm->free_scoreboard);
> +	kfree(mm->used_scoreboard);
>   }
>   EXPORT_SYMBOL(gpu_buddy_fini);
>   
> @@ -1505,15 +1521,18 @@ void gpu_buddy_print(struct gpu_buddy *mm)
>   		mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
>   
>   	for (order = mm->max_order; order >= 0; order--) {
> -		u64 count = mm->free_scoreboard[order];
> -		u64 free = count * (mm->chunk_size << order);
> -
> -		if (free < SZ_1M)
> -			pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
> -				order, free >> 10, count);
> +		u64 free_count = mm->free_scoreboard[order];
> +		u64 used_count = mm->used_scoreboard[order];
> +		u64 block_size = mm->chunk_size << order;
> +		u64 free = free_count * block_size;
> +		u64 used = used_count * block_size;
> +
> +		if (block_size < SZ_1M)
> +			pr_info("order-%2d free: %8llu KiB, used: %8llu KiB, free_blocks: %llu, used_blocks: %llu\n",
> +				order, free >> 10, used >> 10, free_count, used_count);
>   		else
> -			pr_info("order-%2d free: %8llu MiB, blocks: %llu\n",
> -				order, free >> 20, count);
> +			pr_info("order-%2d free: %8llu MiB, used: %8llu MiB, free_blocks: %llu, used_blocks: %llu\n",
> +				order, free >> 20, used >> 20, free_count, used_count);
>   	}
>   }
>   EXPORT_SYMBOL(gpu_buddy_print);
> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
> index eef995e08a37..1536e59c6fe7 100644
> --- a/drivers/gpu/drm/drm_buddy.c
> +++ b/drivers/gpu/drm/drm_buddy.c
> @@ -47,17 +47,23 @@ void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
>   		   mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
>   
>   	for (order = mm->max_order; order >= 0; order--) {
> -		u64 count = mm->free_scoreboard[order];
> -		u64 free = count * (mm->chunk_size << order);
> +		u64 free_count = mm->free_scoreboard[order];
> +		u64 used_count = mm->used_scoreboard[order];
> +		u64 block_size = mm->chunk_size << order;
> +		u64 free = free_count * block_size;
> +		u64 used = used_count * block_size;
>   
>   		drm_printf(p, "order-%2d ", order);
>   
> -		if (free < SZ_1M)
> -			drm_printf(p, "free: %8llu KiB", free >> 10);
> +		if (block_size < SZ_1M)
> +			drm_printf(p, "free: %8llu KiB, used: %8llu KiB",
> +				   free >> 10, used >> 10);
>   		else
> -			drm_printf(p, "free: %8llu MiB", free >> 20);
> +			drm_printf(p, "free: %8llu MiB, used: %8llu MiB",
> +				   free >> 20, used >> 20);
>   
> -		drm_printf(p, ", blocks: %llu\n", count);
> +		drm_printf(p, ", free_blocks: %llu, used_blocks: %llu\n",
> +			   free_count, used_count);
>   	}
>   }
>   EXPORT_SYMBOL(drm_buddy_print);
> diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
> index a28f7d7637ca..e037714563d8 100644
> --- a/include/linux/gpu_buddy.h
> +++ b/include/linux/gpu_buddy.h
> @@ -180,6 +180,14 @@ struct gpu_buddy {
>   	 * called on a free block.
>   	 */
>   	u64 *free_scoreboard;
> +	/*
> +	 * Per-order used block scoreboard: used_scoreboard[order] holds the
> +	 * number of blocks of that order currently in the allocated state.
> +	 * Incremented in mark_allocated(), decremented in mark_free() (guarded
> +	 * by gpu_buddy_block_is_allocated()) and in __gpu_buddy_free() when an
> +	 * allocated block is consumed directly during buddy coalescing.
> +	 */
> +	u64 *used_scoreboard;
>   /* public: */
>   	unsigned int n_roots;
>   	unsigned int max_order;

Re: [PATCH v3 5/5] gpu/buddy: Track per-order used blocks with a scoreboard

[Francois Dugast]

On Mon, May 18, 2026 at 05:02:32PM +0100, Matthew Auld wrote:
> On 18/05/2026 15:14, Francois Dugast wrote:
> > Extend the scoreboard approach from the previous commit to used blocks,
> > so drm_buddy_print() can report per-order allocation pressure in O(1).
> > 
> > Unlike free blocks, an allocated block can leave the allocated state
> > through mark_free() (normal free and gpu_buddy_block_trim()) or be
> > consumed directly by gpu_block_free() during coalescing. Both sites are
> > guarded by gpu_buddy_block_is_allocated() and paired with the increment
> > in mark_allocated().
> > 
> > v2:
> > - Update after fix for use-after-free in split_block() call sites
> > - Change goto label to out_free_used_scoreboard for clarity
> > - Make drm_buddy_print() and gpu_buddy_print() symmetric for used and
> >    free
> > 
> > Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> > Assisted-by: GitHub Copilot:claude-sonnet-4.6
> 
> Could potentially also assert that used_scoreboard is empty at fini(), as a
> quick sanity check that nothing got leaked/missed with the accounting. Would
> also then be checked across the selftests.

Good idea, will do.

> 
> Reviewed-by: Matthew Auld <matthew.auld@intel.com>
> 
> > ---
> >   drivers/gpu/buddy.c         | 39 +++++++++++++++++++++++++++----------
> >   drivers/gpu/drm/drm_buddy.c | 18 +++++++++++------
> >   include/linux/gpu_buddy.h   |  8 ++++++++
> >   3 files changed, 49 insertions(+), 16 deletions(-)
> > 
> > diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> > index de18b63fef0a..f81d0d8fde15 100644
> > --- a/drivers/gpu/buddy.c
> > +++ b/drivers/gpu/buddy.c
> > @@ -194,6 +194,7 @@ static void mark_allocated(struct gpu_buddy *mm,
> >   	block->header |= GPU_BUDDY_ALLOCATED;
> >   	mm->free_scoreboard[gpu_buddy_block_order(block)]--;
> > +	mm->used_scoreboard[gpu_buddy_block_order(block)]++;
> >   	rbtree_remove(mm, block);
> >   }
> > @@ -203,6 +204,9 @@ static void mark_free(struct gpu_buddy *mm,
> >   {
> >   	enum gpu_buddy_free_tree tree;
> > +	if (gpu_buddy_block_is_allocated(block))
> > +		mm->used_scoreboard[gpu_buddy_block_order(block)]--;
> > +
> >   	block->header &= ~GPU_BUDDY_HEADER_STATE;
> >   	block->header |= GPU_BUDDY_FREE;
> > @@ -281,6 +285,9 @@ static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
> >   		if (force_merge && gpu_buddy_block_is_clear(buddy))
> >   			mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
> > +		if (gpu_buddy_block_is_allocated(block))
> > +			mm->used_scoreboard[gpu_buddy_block_order(block)]--;
> > +
> >   		gpu_block_free(mm, block);
> >   		gpu_block_free(mm, buddy);
> > @@ -398,11 +405,17 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
> >   	if (!mm->free_scoreboard)
> >   		return -ENOMEM;
> > +	mm->used_scoreboard = kcalloc(mm->max_order + 1,
> > +				      sizeof(*mm->used_scoreboard),
> > +				      GFP_KERNEL);
> > +	if (!mm->used_scoreboard)
> > +		goto out_free_free_scoreboard;
> > +
> >   	mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
> >   				       sizeof(*mm->free_trees),
> >   				       GFP_KERNEL);
> >   	if (!mm->free_trees)
> > -		goto out_free_scoreboard;
> > +		goto out_free_used_scoreboard;
> >   	for_each_free_tree(i) {
> >   		mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
> > @@ -464,7 +477,9 @@ int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
> >   	while (i--)
> >   		kfree(mm->free_trees[i]);
> >   	kfree(mm->free_trees);
> > -out_free_scoreboard:
> > +out_free_used_scoreboard:
> > +	kfree(mm->used_scoreboard);
> > +out_free_free_scoreboard:
> >   	kfree(mm->free_scoreboard);
> >   	return -ENOMEM;
> >   }
> > @@ -505,6 +520,7 @@ void gpu_buddy_fini(struct gpu_buddy *mm)
> >   	kfree(mm->free_trees);
> >   	kfree(mm->roots);
> >   	kfree(mm->free_scoreboard);
> > +	kfree(mm->used_scoreboard);
> >   }
> >   EXPORT_SYMBOL(gpu_buddy_fini);
> > @@ -1505,15 +1521,18 @@ void gpu_buddy_print(struct gpu_buddy *mm)
> >   		mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
> >   	for (order = mm->max_order; order >= 0; order--) {
> > -		u64 count = mm->free_scoreboard[order];
> > -		u64 free = count * (mm->chunk_size << order);
> > -
> > -		if (free < SZ_1M)
> > -			pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
> > -				order, free >> 10, count);
> > +		u64 free_count = mm->free_scoreboard[order];
> > +		u64 used_count = mm->used_scoreboard[order];
> > +		u64 block_size = mm->chunk_size << order;
> > +		u64 free = free_count * block_size;
> > +		u64 used = used_count * block_size;
> > +
> > +		if (block_size < SZ_1M)
> > +			pr_info("order-%2d free: %8llu KiB, used: %8llu KiB, free_blocks: %llu, used_blocks: %llu\n",
> > +				order, free >> 10, used >> 10, free_count, used_count);
> >   		else
> > -			pr_info("order-%2d free: %8llu MiB, blocks: %llu\n",
> > -				order, free >> 20, count);
> > +			pr_info("order-%2d free: %8llu MiB, used: %8llu MiB, free_blocks: %llu, used_blocks: %llu\n",
> > +				order, free >> 20, used >> 20, free_count, used_count);
> >   	}
> >   }
> >   EXPORT_SYMBOL(gpu_buddy_print);
> > diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
> > index eef995e08a37..1536e59c6fe7 100644
> > --- a/drivers/gpu/drm/drm_buddy.c
> > +++ b/drivers/gpu/drm/drm_buddy.c
> > @@ -47,17 +47,23 @@ void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
> >   		   mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
> >   	for (order = mm->max_order; order >= 0; order--) {
> > -		u64 count = mm->free_scoreboard[order];
> > -		u64 free = count * (mm->chunk_size << order);
> > +		u64 free_count = mm->free_scoreboard[order];
> > +		u64 used_count = mm->used_scoreboard[order];
> > +		u64 block_size = mm->chunk_size << order;
> > +		u64 free = free_count * block_size;
> > +		u64 used = used_count * block_size;
> >   		drm_printf(p, "order-%2d ", order);
> > -		if (free < SZ_1M)
> > -			drm_printf(p, "free: %8llu KiB", free >> 10);
> > +		if (block_size < SZ_1M)
> > +			drm_printf(p, "free: %8llu KiB, used: %8llu KiB",
> > +				   free >> 10, used >> 10);
> >   		else
> > -			drm_printf(p, "free: %8llu MiB", free >> 20);
> > +			drm_printf(p, "free: %8llu MiB, used: %8llu MiB",
> > +				   free >> 20, used >> 20);
> > -		drm_printf(p, ", blocks: %llu\n", count);
> > +		drm_printf(p, ", free_blocks: %llu, used_blocks: %llu\n",
> > +			   free_count, used_count);
> >   	}
> >   }
> >   EXPORT_SYMBOL(drm_buddy_print);
> > diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
> > index a28f7d7637ca..e037714563d8 100644
> > --- a/include/linux/gpu_buddy.h
> > +++ b/include/linux/gpu_buddy.h
> > @@ -180,6 +180,14 @@ struct gpu_buddy {
> >   	 * called on a free block.
> >   	 */
> >   	u64 *free_scoreboard;
> > +	/*
> > +	 * Per-order used block scoreboard: used_scoreboard[order] holds the
> > +	 * number of blocks of that order currently in the allocated state.
> > +	 * Incremented in mark_allocated(), decremented in mark_free() (guarded
> > +	 * by gpu_buddy_block_is_allocated()) and in __gpu_buddy_free() when an
> > +	 * allocated block is consumed directly during buddy coalescing.
> > +	 */
> > +	u64 *used_scoreboard;
> >   /* public: */
> >   	unsigned int n_roots;
> >   	unsigned int max_order;
> 

Re: [PATCH v3 1/5] gpu/buddy: Fix use-after-free in split_block() call sites

[Francois Dugast]

On Mon, May 18, 2026 at 04:55:12PM +0100, Matthew Auld wrote:
> On 18/05/2026 15:14, Francois Dugast wrote:
> > When split_block() fails it returns before calling mark_split(), leaving
> > the block in the FREE state and still linked in the rbtree.  The four
> > err_undo paths then call __gpu_buddy_free() without first removing the
> > block from the tree, which leads to two distinct bugs:
> > 
> >   - If the buddy is also free, __gpu_buddy_free() merges the two siblings
> >     by calling gpu_block_free(mm, block) while block->rb is still linked
> >     in the tree.  Any subsequent rbtree traversal will follow the now-
> >     dangling pointer, causing a use-after-free.
> > 
> >   - In alloc_from_freetree(), where there is no buddy guard,
> >     __gpu_buddy_free() always reaches mark_free() -> rbtree_insert() with
> >     block still in the tree, corrupting the rbtree.
> > 
> > The same pattern is already used correctly in __force_merge(): call
> > rbtree_remove() to unlink the block before handing it to
> > __gpu_buddy_free().  Apply the same fix to all four err_undo sites.
> > 
> > Reported-by: Sashiko <sashiko-bot@kernel.org>
> > Signed-off-by: Francois Dugast <francois.dugast@intel.com>
> > Assisted-by: GitHub Copilot:claude-sonnet-4.6
> > ---
> >   drivers/gpu/buddy.c | 16 ++++++++++++----
> >   1 file changed, 12 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> > index eb1457376307..dac2027bb64a 100644
> > --- a/drivers/gpu/buddy.c
> > +++ b/drivers/gpu/buddy.c
> > @@ -737,8 +737,10 @@ __alloc_range_bias(struct gpu_buddy *mm,
> >   	buddy = __get_buddy(block);
> >   	if (buddy &&
> >   	    (gpu_buddy_block_is_free(block) &&
> > -	     gpu_buddy_block_is_free(buddy)))
> > +	     gpu_buddy_block_is_free(buddy))) {
> > +		rbtree_remove(mm, block);
> >   		__gpu_buddy_free(mm, block, false);
> > +	}
> >   	return ERR_PTR(err);
> >   }
> > @@ -847,8 +849,10 @@ alloc_from_freetree(struct gpu_buddy *mm,
> >   	return block;
> >   err_undo:
> > -	if (tmp != order)
> > +	if (tmp != order) {
> > +		rbtree_remove(mm, block);
> 
> Actually, I think this needs the same checking like elsewhere? Say we fail
> on the first split? Nothing was actually split, right?

I think this is unnecessary: for block this is tested above with
BUG_ON(!gpu_buddy_block_is_free(block)). If split_block() fails then it
happens before mark_split() so block remains free. If buddy is not free
then the merge loop is skipped in __gpu_buddy_free() but mark_free() is
called so we do remove + re-insert.

Also, the checks are added with patch #3 and the introduction of
__gpu_buddy_undo_splits().

Francois

> 
> >   		__gpu_buddy_free(mm, block, false);
> > +	}
> >   	return ERR_PTR(err);
> >   }
> > @@ -968,8 +972,10 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
> >   	buddy = __get_buddy(block);
> >   	if (buddy &&
> >   	    (gpu_buddy_block_is_free(block) &&
> > -	     gpu_buddy_block_is_free(buddy)))
> > +	     gpu_buddy_block_is_free(buddy))) {
> > +		rbtree_remove(mm, block);
> >   		__gpu_buddy_free(mm, block, false);
> > +	}
> >   	return ERR_PTR(err);
> >   }
> > @@ -1054,8 +1060,10 @@ static int __alloc_range(struct gpu_buddy *mm,
> >   	buddy = __get_buddy(block);
> >   	if (buddy &&
> >   	    (gpu_buddy_block_is_free(block) &&
> > -	     gpu_buddy_block_is_free(buddy)))
> > +	     gpu_buddy_block_is_free(buddy))) {
> > +		rbtree_remove(mm, block);
> >   		__gpu_buddy_free(mm, block, false);
> > +	}
> >   err_free:
> >   	if (err == -ENOSPC && total_allocated_on_err) {
> 

Re: [PATCH v3 1/5] gpu/buddy: Fix use-after-free in split_block() call sites

[Matthew Auld]

On 20/05/2026 11:58, Francois Dugast wrote:
> On Mon, May 18, 2026 at 04:55:12PM +0100, Matthew Auld wrote:
>> On 18/05/2026 15:14, Francois Dugast wrote:
>>> When split_block() fails it returns before calling mark_split(), leaving
>>> the block in the FREE state and still linked in the rbtree.  The four
>>> err_undo paths then call __gpu_buddy_free() without first removing the
>>> block from the tree, which leads to two distinct bugs:
>>>
>>>    - If the buddy is also free, __gpu_buddy_free() merges the two siblings
>>>      by calling gpu_block_free(mm, block) while block->rb is still linked
>>>      in the tree.  Any subsequent rbtree traversal will follow the now-
>>>      dangling pointer, causing a use-after-free.
>>>
>>>    - In alloc_from_freetree(), where there is no buddy guard,
>>>      __gpu_buddy_free() always reaches mark_free() -> rbtree_insert() with
>>>      block still in the tree, corrupting the rbtree.
>>>
>>> The same pattern is already used correctly in __force_merge(): call
>>> rbtree_remove() to unlink the block before handing it to
>>> __gpu_buddy_free().  Apply the same fix to all four err_undo sites.
>>>
>>> Reported-by: Sashiko <sashiko-bot@kernel.org>
>>> Signed-off-by: Francois Dugast <francois.dugast@intel.com>
>>> Assisted-by: GitHub Copilot:claude-sonnet-4.6
>>> ---
>>>    drivers/gpu/buddy.c | 16 ++++++++++++----
>>>    1 file changed, 12 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
>>> index eb1457376307..dac2027bb64a 100644
>>> --- a/drivers/gpu/buddy.c
>>> +++ b/drivers/gpu/buddy.c
>>> @@ -737,8 +737,10 @@ __alloc_range_bias(struct gpu_buddy *mm,
>>>    	buddy = __get_buddy(block);
>>>    	if (buddy &&
>>>    	    (gpu_buddy_block_is_free(block) &&
>>> -	     gpu_buddy_block_is_free(buddy)))
>>> +	     gpu_buddy_block_is_free(buddy))) {
>>> +		rbtree_remove(mm, block);
>>>    		__gpu_buddy_free(mm, block, false);
>>> +	}
>>>    	return ERR_PTR(err);
>>>    }
>>> @@ -847,8 +849,10 @@ alloc_from_freetree(struct gpu_buddy *mm,
>>>    	return block;
>>>    err_undo:
>>> -	if (tmp != order)
>>> +	if (tmp != order) {
>>> +		rbtree_remove(mm, block);
>>
>> Actually, I think this needs the same checking like elsewhere? Say we fail
>> on the first split? Nothing was actually split, right?
> 
> I think this is unnecessary: for block this is tested above with
> BUG_ON(!gpu_buddy_block_is_free(block)). If split_block() fails then it
> happens before mark_split() so block remains free. If buddy is not free
> then the merge loop is skipped in __gpu_buddy_free() but mark_free() is
> called so we do remove + re-insert.
> 
> Also, the checks are added with patch #3 and the introduction of
> __gpu_buddy_undo_splits().

Right, makes sense.

> 
> Francois
> 
>>
>>>    		__gpu_buddy_free(mm, block, false);
>>> +	}
>>>    	return ERR_PTR(err);
>>>    }
>>> @@ -968,8 +972,10 @@ gpu_buddy_offset_aligned_allocation(struct gpu_buddy *mm,
>>>    	buddy = __get_buddy(block);
>>>    	if (buddy &&
>>>    	    (gpu_buddy_block_is_free(block) &&
>>> -	     gpu_buddy_block_is_free(buddy)))
>>> +	     gpu_buddy_block_is_free(buddy))) {
>>> +		rbtree_remove(mm, block);
>>>    		__gpu_buddy_free(mm, block, false);
>>> +	}
>>>    	return ERR_PTR(err);
>>>    }
>>> @@ -1054,8 +1060,10 @@ static int __alloc_range(struct gpu_buddy *mm,
>>>    	buddy = __get_buddy(block);
>>>    	if (buddy &&
>>>    	    (gpu_buddy_block_is_free(block) &&
>>> -	     gpu_buddy_block_is_free(buddy)))
>>> +	     gpu_buddy_block_is_free(buddy))) {
>>> +		rbtree_remove(mm, block);
>>>    		__gpu_buddy_free(mm, block, false);
>>> +	}
>>>    err_free:
>>>    	if (err == -ENOSPC && total_allocated_on_err) {
>>