* [PATCH] fbdev/ffb: fix corrupted video output on FFB1
From: René Rebe @ 2026-02-05 15:49 UTC (permalink / raw)
To: Helge Deller; +Cc: linux-fbdev
Fix Sun FFB1 corrupted video out [1] by disabling overlay and
initializing window mode to a known state. The issue never appeared on
my FFB2+/vertical nor Elite3D/M6. It could also depend on the PROM
version.
/SUNW,ffb@1e,0: FFB at 000001fc00000000, type 11, DAC pnum[236c] rev[10] manuf_rev[4]
X (II) /dev/fb0: Detected FFB1, Z-buffer, Single-buffered.
X (II) /dev/fb0: BT9068 (PAC1) ramdac detected (with normal cursor control)
X (II) /dev/fb0: Detected Creator/Creator3D
[1] https://www.instagram.com/p/DUTcSmSjSem/
Signed-off-by: René Rebe <rene@exactco.de>
Cc: stable@kernel.org
---
Tested on Sun Ultra 2 w/ FFB1 BT9068 (PAC1) ramdac running T2/Linux.
Defines re-used from xf86-video-sunffb.
---
diff --git a/drivers/video/fbdev/ffb.c b/drivers/video/fbdev/ffb.c
index 34b6abff9493..8d77f102dd82 100644
--- a/drivers/video/fbdev/ffb.c
+++ b/drivers/video/fbdev/ffb.c
@@ -335,6 +335,9 @@ struct ffb_dac {
};
#define FFB_DAC_UCTRL 0x1001 /* User Control */
+#define FFB_DAC_UCTRL_OVENAB 0x00000008 /* Overlay Enable */
+#define FFB_DAC_UCTRL_WMODE 0x00000030 /* Window Mode */
+#define FFB_DAC_UCTRL_WM_COMB 0x00000000 /* Window Mode = Combined */
#define FFB_DAC_UCTRL_MANREV 0x00000f00 /* 4-bit Manufacturing Revision */
#define FFB_DAC_UCTRL_MANREV_SHIFT 8
#define FFB_DAC_TGEN 0x6000 /* Timing Generator */
@@ -425,7 +428,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
{
struct ffb_fbc __iomem *fbc = par->fbc;
struct ffb_dac __iomem *dac = par->dac;
- unsigned long flags;
+ unsigned long flags, uctrl;
spin_lock_irqsave(&par->lock, flags);
FFBWait(par);
@@ -442,7 +445,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
upa_writel(par->bg_cache, &fbc->bg);
FFBWait(par);
- /* Disable cursor. */
+ /* Disable cursor. */
upa_writel(FFB_DAC_CUR_CTRL, &dac->type2);
if (par->flags & FFB_FLAG_INVCURSOR)
upa_writel(0, &dac->value2);
@@ -450,6 +453,15 @@ static void ffb_switch_from_graph(struct ffb_par *par)
upa_writel((FFB_DAC_CUR_CTRL_P0 |
FFB_DAC_CUR_CTRL_P1), &dac->value2);
+ /* Disable overlay and window modes. */
+ upa_writel(FFB_DAC_UCTRL, &dac->type);
+ uctrl = upa_readl(&dac->value);
+ uctrl &= ~FFB_DAC_UCTRL_WMODE;
+ uctrl |= FFB_DAC_UCTRL_WM_COMB;
+ uctrl &= ~FFB_DAC_UCTRL_OVENAB;
+ upa_writel(FFB_DAC_UCTRL, &dac->type);
+ upa_writel(uctrl, &dac->value);
+
spin_unlock_irqrestore(&par->lock, flags);
}
--
René Rebe, ExactCODE GmbH, Berlin, Germany
https://exactco.de • https://t2linux.com • https://patreon.com/renerebe
^ permalink raw reply related
* Re: [PATCH RFC v6 02/26] gpu: Move DRM buddy allocator one level up
From: Dave Airlie @ 2026-02-05 20:55 UTC (permalink / raw)
To: Joel Fernandes
Cc: linux-kernel, Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Alexey Ivanov, Balbir Singh,
Philipp Stanner, Elle Rhumsaa, Daniel Almeida, joel, nouveau,
dri-devel, rust-for-linux, linux-doc, amd-gfx, intel-gfx,
intel-xe, linux-fbdev
In-Reply-To: <20260120204303.3229303-3-joelagnelf@nvidia.com>
On Wed, 21 Jan 2026 at 06:44, Joel Fernandes <joelagnelf@nvidia.com> wrote:
>
> Move the DRM buddy allocator one level up so that it can be used by GPU
> drivers (example, nova-core) that have usecases other than DRM (such as
> VFIO vGPU support). Modify the API, structures and Kconfigs to use
> "gpu_buddy" terminology. Adapt the drivers and tests to use the new API.
>
> The commit cannot be split due to bisectability, however no functional
> change is intended. Verified by running K-UNIT tests and build tested
> various configurations.
>
> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
I suggested this and think it's a good idea.
Reviewed-by: Dave Airlie <airlied@redhat.com>
> ---
> Documentation/gpu/drm-mm.rst | 10 +-
> drivers/gpu/Kconfig | 13 +
> drivers/gpu/Makefile | 2 +
> drivers/gpu/buddy.c | 1310 +++++++++++++++++
> drivers/gpu/drm/Kconfig | 1 +
> drivers/gpu/drm/Kconfig.debug | 4 +-
> drivers/gpu/drm/amd/amdgpu/Kconfig | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 2 +-
> .../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h | 12 +-
> drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 80 +-
> drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h | 20 +-
> drivers/gpu/drm/drm_buddy.c | 1284 +---------------
> drivers/gpu/drm/i915/Kconfig | 1 +
> drivers/gpu/drm/i915/i915_scatterlist.c | 10 +-
> drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 55 +-
> drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 6 +-
> .../drm/i915/selftests/intel_memory_region.c | 20 +-
> drivers/gpu/drm/tests/Makefile | 1 -
> .../gpu/drm/ttm/tests/ttm_bo_validate_test.c | 5 +-
> drivers/gpu/drm/ttm/tests/ttm_mock_manager.c | 18 +-
> drivers/gpu/drm/ttm/tests/ttm_mock_manager.h | 4 +-
> drivers/gpu/drm/xe/Kconfig | 1 +
> drivers/gpu/drm/xe/xe_res_cursor.h | 34 +-
> drivers/gpu/drm/xe/xe_svm.c | 12 +-
> drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 73 +-
> drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h | 4 +-
> drivers/gpu/tests/Makefile | 3 +
> .../gpu_buddy_test.c} | 390 ++---
> drivers/gpu/tests/gpu_random.c | 48 +
> drivers/gpu/tests/gpu_random.h | 28 +
> drivers/video/Kconfig | 2 +
> include/drm/drm_buddy.h | 163 +-
> include/linux/gpu_buddy.h | 177 +++
> 33 files changed, 1995 insertions(+), 1799 deletions(-)
> create mode 100644 drivers/gpu/Kconfig
> create mode 100644 drivers/gpu/buddy.c
> create mode 100644 drivers/gpu/tests/Makefile
> rename drivers/gpu/{drm/tests/drm_buddy_test.c => tests/gpu_buddy_test.c} (68%)
> create mode 100644 drivers/gpu/tests/gpu_random.c
> create mode 100644 drivers/gpu/tests/gpu_random.h
> create mode 100644 include/linux/gpu_buddy.h
>
> diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst
> index d55751cad67c..8e0d31230b29 100644
> --- a/Documentation/gpu/drm-mm.rst
> +++ b/Documentation/gpu/drm-mm.rst
> @@ -509,8 +509,14 @@ DRM GPUVM Function References
> DRM Buddy Allocator
> ===================
>
> -DRM Buddy Function References
> ------------------------------
> +Buddy Allocator Function References (GPU buddy)
> +-----------------------------------------------
> +
> +.. kernel-doc:: drivers/gpu/buddy.c
> + :export:
> +
> +DRM Buddy Specific Logging Function References
> +----------------------------------------------
>
> .. kernel-doc:: drivers/gpu/drm/drm_buddy.c
> :export:
> diff --git a/drivers/gpu/Kconfig b/drivers/gpu/Kconfig
> new file mode 100644
> index 000000000000..22dd29cd50b5
> --- /dev/null
> +++ b/drivers/gpu/Kconfig
> @@ -0,0 +1,13 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +config GPU_BUDDY
> + bool
> + help
> + A page based buddy allocator for GPU memory.
> +
> +config GPU_BUDDY_KUNIT_TEST
> + tristate "KUnit tests for GPU buddy allocator" if !KUNIT_ALL_TESTS
> + depends on GPU_BUDDY && KUNIT
> + default KUNIT_ALL_TESTS
> + help
> + KUnit tests for the GPU buddy allocator.
> diff --git a/drivers/gpu/Makefile b/drivers/gpu/Makefile
> index 36a54d456630..5063caccabdf 100644
> --- a/drivers/gpu/Makefile
> +++ b/drivers/gpu/Makefile
> @@ -6,3 +6,5 @@ obj-y += host1x/ drm/ vga/
> obj-$(CONFIG_IMX_IPUV3_CORE) += ipu-v3/
> obj-$(CONFIG_TRACE_GPU_MEM) += trace/
> obj-$(CONFIG_NOVA_CORE) += nova-core/
> +obj-$(CONFIG_GPU_BUDDY) += buddy.o
> +obj-y += tests/
> diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
> new file mode 100644
> index 000000000000..1347c0436617
> --- /dev/null
> +++ b/drivers/gpu/buddy.c
> @@ -0,0 +1,1310 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2021 Intel Corporation
> + */
> +
> +#include <kunit/test-bug.h>
> +
> +#include <linux/export.h>
> +#include <linux/gpu_buddy.h>
> +#include <linux/kmemleak.h>
> +#include <linux/module.h>
> +#include <linux/sizes.h>
> +
> +static struct kmem_cache *slab_blocks;
> +
> +static struct gpu_buddy_block *gpu_block_alloc(struct gpu_buddy *mm,
> + struct gpu_buddy_block *parent,
> + unsigned int order,
> + u64 offset)
> +{
> + struct gpu_buddy_block *block;
> +
> + BUG_ON(order > GPU_BUDDY_MAX_ORDER);
> +
> + block = kmem_cache_zalloc(slab_blocks, GFP_KERNEL);
> + if (!block)
> + return NULL;
> +
> + block->header = offset;
> + block->header |= order;
> + block->parent = parent;
> +
> + RB_CLEAR_NODE(&block->rb);
> +
> + BUG_ON(block->header & GPU_BUDDY_HEADER_UNUSED);
> + return block;
> +}
> +
> +static void gpu_block_free(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + kmem_cache_free(slab_blocks, block);
> +}
> +
> +static enum gpu_buddy_free_tree
> +get_block_tree(struct gpu_buddy_block *block)
> +{
> + return gpu_buddy_block_is_clear(block) ?
> + GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE;
> +}
> +
> +static struct gpu_buddy_block *
> +rbtree_get_free_block(const struct rb_node *node)
> +{
> + return node ? rb_entry(node, struct gpu_buddy_block, rb) : NULL;
> +}
> +
> +static struct gpu_buddy_block *
> +rbtree_last_free_block(struct rb_root *root)
> +{
> + return rbtree_get_free_block(rb_last(root));
> +}
> +
> +static bool rbtree_is_empty(struct rb_root *root)
> +{
> + return RB_EMPTY_ROOT(root);
> +}
> +
> +static bool gpu_buddy_block_offset_less(const struct gpu_buddy_block *block,
> + const struct gpu_buddy_block *node)
> +{
> + return gpu_buddy_block_offset(block) < gpu_buddy_block_offset(node);
> +}
> +
> +static bool rbtree_block_offset_less(struct rb_node *block,
> + const struct rb_node *node)
> +{
> + return gpu_buddy_block_offset_less(rbtree_get_free_block(block),
> + rbtree_get_free_block(node));
> +}
> +
> +static void rbtree_insert(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block,
> + enum gpu_buddy_free_tree tree)
> +{
> + rb_add(&block->rb,
> + &mm->free_trees[tree][gpu_buddy_block_order(block)],
> + rbtree_block_offset_less);
> +}
> +
> +static void rbtree_remove(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + unsigned int order = gpu_buddy_block_order(block);
> + enum gpu_buddy_free_tree tree;
> + struct rb_root *root;
> +
> + tree = get_block_tree(block);
> + root = &mm->free_trees[tree][order];
> +
> + rb_erase(&block->rb, root);
> + RB_CLEAR_NODE(&block->rb);
> +}
> +
> +static void clear_reset(struct gpu_buddy_block *block)
> +{
> + block->header &= ~GPU_BUDDY_HEADER_CLEAR;
> +}
> +
> +static void mark_cleared(struct gpu_buddy_block *block)
> +{
> + block->header |= GPU_BUDDY_HEADER_CLEAR;
> +}
> +
> +static void mark_allocated(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + block->header &= ~GPU_BUDDY_HEADER_STATE;
> + block->header |= GPU_BUDDY_ALLOCATED;
> +
> + rbtree_remove(mm, block);
> +}
> +
> +static void mark_free(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + enum gpu_buddy_free_tree tree;
> +
> + block->header &= ~GPU_BUDDY_HEADER_STATE;
> + block->header |= GPU_BUDDY_FREE;
> +
> + tree = get_block_tree(block);
> + rbtree_insert(mm, block, tree);
> +}
> +
> +static void mark_split(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + block->header &= ~GPU_BUDDY_HEADER_STATE;
> + block->header |= GPU_BUDDY_SPLIT;
> +
> + rbtree_remove(mm, block);
> +}
> +
> +static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2)
> +{
> + return s1 <= e2 && e1 >= s2;
> +}
> +
> +static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2)
> +{
> + return s1 <= s2 && e1 >= e2;
> +}
> +
> +static struct gpu_buddy_block *
> +__get_buddy(struct gpu_buddy_block *block)
> +{
> + struct gpu_buddy_block *parent;
> +
> + parent = block->parent;
> + if (!parent)
> + return NULL;
> +
> + if (parent->left == block)
> + return parent->right;
> +
> + return parent->left;
> +}
> +
> +static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block,
> + bool force_merge)
> +{
> + struct gpu_buddy_block *parent;
> + unsigned int order;
> +
> + while ((parent = block->parent)) {
> + struct gpu_buddy_block *buddy;
> +
> + buddy = __get_buddy(block);
> +
> + if (!gpu_buddy_block_is_free(buddy))
> + break;
> +
> + if (!force_merge) {
> + /*
> + * Check the block and its buddy clear state and exit
> + * the loop if they both have the dissimilar state.
> + */
> + if (gpu_buddy_block_is_clear(block) !=
> + gpu_buddy_block_is_clear(buddy))
> + break;
> +
> + if (gpu_buddy_block_is_clear(block))
> + mark_cleared(parent);
> + }
> +
> + rbtree_remove(mm, buddy);
> + if (force_merge && gpu_buddy_block_is_clear(buddy))
> + mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
> +
> + gpu_block_free(mm, block);
> + gpu_block_free(mm, buddy);
> +
> + block = parent;
> + }
> +
> + order = gpu_buddy_block_order(block);
> + mark_free(mm, block);
> +
> + return order;
> +}
> +
> +static int __force_merge(struct gpu_buddy *mm,
> + u64 start,
> + u64 end,
> + unsigned int min_order)
> +{
> + unsigned int tree, order;
> + int i;
> +
> + if (!min_order)
> + return -ENOMEM;
> +
> + if (min_order > mm->max_order)
> + return -EINVAL;
> +
> + for_each_free_tree(tree) {
> + for (i = min_order - 1; i >= 0; i--) {
> + struct rb_node *iter = rb_last(&mm->free_trees[tree][i]);
> +
> + while (iter) {
> + struct gpu_buddy_block *block, *buddy;
> + u64 block_start, block_end;
> +
> + block = rbtree_get_free_block(iter);
> + iter = rb_prev(iter);
> +
> + if (!block || !block->parent)
> + continue;
> +
> + block_start = gpu_buddy_block_offset(block);
> + block_end = block_start + gpu_buddy_block_size(mm, block) - 1;
> +
> + if (!contains(start, end, block_start, block_end))
> + continue;
> +
> + buddy = __get_buddy(block);
> + if (!gpu_buddy_block_is_free(buddy))
> + continue;
> +
> + WARN_ON(gpu_buddy_block_is_clear(block) ==
> + gpu_buddy_block_is_clear(buddy));
> +
> + /*
> + * Advance to the next node when the current node is the buddy,
> + * as freeing the block will also remove its buddy from the tree.
> + */
> + if (iter == &buddy->rb)
> + iter = rb_prev(iter);
> +
> + rbtree_remove(mm, block);
> + if (gpu_buddy_block_is_clear(block))
> + mm->clear_avail -= gpu_buddy_block_size(mm, block);
> +
> + order = __gpu_buddy_free(mm, block, true);
> + if (order >= min_order)
> + return 0;
> + }
> + }
> + }
> +
> + return -ENOMEM;
> +}
> +
> +/**
> + * gpu_buddy_init - init memory manager
> + *
> + * @mm: GPU buddy manager to initialize
> + * @size: size in bytes to manage
> + * @chunk_size: minimum page size in bytes for our allocations
> + *
> + * Initializes the memory manager and its resources.
> + *
> + * Returns:
> + * 0 on success, error code on failure.
> + */
> +int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
> +{
> + unsigned int i, j, root_count = 0;
> + u64 offset = 0;
> +
> + if (size < chunk_size)
> + return -EINVAL;
> +
> + if (chunk_size < SZ_4K)
> + return -EINVAL;
> +
> + if (!is_power_of_2(chunk_size))
> + return -EINVAL;
> +
> + size = round_down(size, chunk_size);
> +
> + mm->size = size;
> + mm->avail = size;
> + mm->clear_avail = 0;
> + mm->chunk_size = chunk_size;
> + mm->max_order = ilog2(size) - ilog2(chunk_size);
> +
> + BUG_ON(mm->max_order > GPU_BUDDY_MAX_ORDER);
> +
> + mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
> + sizeof(*mm->free_trees),
> + GFP_KERNEL);
> + if (!mm->free_trees)
> + return -ENOMEM;
> +
> + for_each_free_tree(i) {
> + mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
> + sizeof(struct rb_root),
> + GFP_KERNEL);
> + if (!mm->free_trees[i])
> + goto out_free_tree;
> +
> + for (j = 0; j <= mm->max_order; ++j)
> + mm->free_trees[i][j] = RB_ROOT;
> + }
> +
> + mm->n_roots = hweight64(size);
> +
> + mm->roots = kmalloc_array(mm->n_roots,
> + sizeof(struct gpu_buddy_block *),
> + GFP_KERNEL);
> + if (!mm->roots)
> + goto out_free_tree;
> +
> + /*
> + * Split into power-of-two blocks, in case we are given a size that is
> + * not itself a power-of-two.
> + */
> + do {
> + struct gpu_buddy_block *root;
> + unsigned int order;
> + u64 root_size;
> +
> + order = ilog2(size) - ilog2(chunk_size);
> + root_size = chunk_size << order;
> +
> + root = gpu_block_alloc(mm, NULL, order, offset);
> + if (!root)
> + goto out_free_roots;
> +
> + mark_free(mm, root);
> +
> + BUG_ON(root_count > mm->max_order);
> + BUG_ON(gpu_buddy_block_size(mm, root) < chunk_size);
> +
> + mm->roots[root_count] = root;
> +
> + offset += root_size;
> + size -= root_size;
> + root_count++;
> + } while (size);
> +
> + return 0;
> +
> +out_free_roots:
> + while (root_count--)
> + gpu_block_free(mm, mm->roots[root_count]);
> + kfree(mm->roots);
> +out_free_tree:
> + while (i--)
> + kfree(mm->free_trees[i]);
> + kfree(mm->free_trees);
> + return -ENOMEM;
> +}
> +EXPORT_SYMBOL(gpu_buddy_init);
> +
> +/**
> + * gpu_buddy_fini - tear down the memory manager
> + *
> + * @mm: GPU buddy manager to free
> + *
> + * Cleanup memory manager resources and the freetree
> + */
> +void gpu_buddy_fini(struct gpu_buddy *mm)
> +{
> + u64 root_size, size, start;
> + unsigned int order;
> + int i;
> +
> + size = mm->size;
> +
> + for (i = 0; i < mm->n_roots; ++i) {
> + order = ilog2(size) - ilog2(mm->chunk_size);
> + start = gpu_buddy_block_offset(mm->roots[i]);
> + __force_merge(mm, start, start + size, order);
> +
> + if (WARN_ON(!gpu_buddy_block_is_free(mm->roots[i])))
> + kunit_fail_current_test("buddy_fini() root");
> +
> + gpu_block_free(mm, mm->roots[i]);
> +
> + root_size = mm->chunk_size << order;
> + size -= root_size;
> + }
> +
> + WARN_ON(mm->avail != mm->size);
> +
> + for_each_free_tree(i)
> + kfree(mm->free_trees[i]);
> + kfree(mm->roots);
> +}
> +EXPORT_SYMBOL(gpu_buddy_fini);
> +
> +static int split_block(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + unsigned int block_order = gpu_buddy_block_order(block) - 1;
> + u64 offset = gpu_buddy_block_offset(block);
> +
> + BUG_ON(!gpu_buddy_block_is_free(block));
> + BUG_ON(!gpu_buddy_block_order(block));
> +
> + block->left = gpu_block_alloc(mm, block, block_order, offset);
> + if (!block->left)
> + return -ENOMEM;
> +
> + block->right = gpu_block_alloc(mm, block, block_order,
> + offset + (mm->chunk_size << block_order));
> + if (!block->right) {
> + gpu_block_free(mm, block->left);
> + return -ENOMEM;
> + }
> +
> + mark_split(mm, block);
> +
> + if (gpu_buddy_block_is_clear(block)) {
> + mark_cleared(block->left);
> + mark_cleared(block->right);
> + clear_reset(block);
> + }
> +
> + mark_free(mm, block->left);
> + mark_free(mm, block->right);
> +
> + return 0;
> +}
> +
> +/**
> + * gpu_get_buddy - get buddy address
> + *
> + * @block: GPU buddy block
> + *
> + * Returns the corresponding buddy block for @block, or NULL
> + * if this is a root block and can't be merged further.
> + * Requires some kind of locking to protect against
> + * any concurrent allocate and free operations.
> + */
> +struct gpu_buddy_block *
> +gpu_get_buddy(struct gpu_buddy_block *block)
> +{
> + return __get_buddy(block);
> +}
> +EXPORT_SYMBOL(gpu_get_buddy);
> +
> +/**
> + * gpu_buddy_reset_clear - reset blocks clear state
> + *
> + * @mm: GPU buddy manager
> + * @is_clear: blocks clear state
> + *
> + * Reset the clear state based on @is_clear value for each block
> + * in the freetree.
> + */
> +void gpu_buddy_reset_clear(struct gpu_buddy *mm, bool is_clear)
> +{
> + enum gpu_buddy_free_tree src_tree, dst_tree;
> + u64 root_size, size, start;
> + unsigned int order;
> + int i;
> +
> + size = mm->size;
> + for (i = 0; i < mm->n_roots; ++i) {
> + order = ilog2(size) - ilog2(mm->chunk_size);
> + start = gpu_buddy_block_offset(mm->roots[i]);
> + __force_merge(mm, start, start + size, order);
> +
> + root_size = mm->chunk_size << order;
> + size -= root_size;
> + }
> +
> + src_tree = is_clear ? GPU_BUDDY_DIRTY_TREE : GPU_BUDDY_CLEAR_TREE;
> + dst_tree = is_clear ? GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE;
> +
> + for (i = 0; i <= mm->max_order; ++i) {
> + struct rb_root *root = &mm->free_trees[src_tree][i];
> + struct gpu_buddy_block *block, *tmp;
> +
> + rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
> + rbtree_remove(mm, block);
> + if (is_clear) {
> + mark_cleared(block);
> + mm->clear_avail += gpu_buddy_block_size(mm, block);
> + } else {
> + clear_reset(block);
> + mm->clear_avail -= gpu_buddy_block_size(mm, block);
> + }
> +
> + rbtree_insert(mm, block, dst_tree);
> + }
> + }
> +}
> +EXPORT_SYMBOL(gpu_buddy_reset_clear);
> +
> +/**
> + * gpu_buddy_free_block - free a block
> + *
> + * @mm: GPU buddy manager
> + * @block: block to be freed
> + */
> +void gpu_buddy_free_block(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + BUG_ON(!gpu_buddy_block_is_allocated(block));
> + mm->avail += gpu_buddy_block_size(mm, block);
> + if (gpu_buddy_block_is_clear(block))
> + mm->clear_avail += gpu_buddy_block_size(mm, block);
> +
> + __gpu_buddy_free(mm, block, false);
> +}
> +EXPORT_SYMBOL(gpu_buddy_free_block);
> +
> +static void __gpu_buddy_free_list(struct gpu_buddy *mm,
> + struct list_head *objects,
> + bool mark_clear,
> + bool mark_dirty)
> +{
> + struct gpu_buddy_block *block, *on;
> +
> + WARN_ON(mark_dirty && mark_clear);
> +
> + list_for_each_entry_safe(block, on, objects, link) {
> + if (mark_clear)
> + mark_cleared(block);
> + else if (mark_dirty)
> + clear_reset(block);
> + gpu_buddy_free_block(mm, block);
> + cond_resched();
> + }
> + INIT_LIST_HEAD(objects);
> +}
> +
> +static void gpu_buddy_free_list_internal(struct gpu_buddy *mm,
> + struct list_head *objects)
> +{
> + /*
> + * Don't touch the clear/dirty bit, since allocation is still internal
> + * at this point. For example we might have just failed part of the
> + * allocation.
> + */
> + __gpu_buddy_free_list(mm, objects, false, false);
> +}
> +
> +/**
> + * gpu_buddy_free_list - free blocks
> + *
> + * @mm: GPU buddy manager
> + * @objects: input list head to free blocks
> + * @flags: optional flags like GPU_BUDDY_CLEARED
> + */
> +void gpu_buddy_free_list(struct gpu_buddy *mm,
> + struct list_head *objects,
> + unsigned int flags)
> +{
> + bool mark_clear = flags & GPU_BUDDY_CLEARED;
> +
> + __gpu_buddy_free_list(mm, objects, mark_clear, !mark_clear);
> +}
> +EXPORT_SYMBOL(gpu_buddy_free_list);
> +
> +static bool block_incompatible(struct gpu_buddy_block *block, unsigned int flags)
> +{
> + bool needs_clear = flags & GPU_BUDDY_CLEAR_ALLOCATION;
> +
> + return needs_clear != gpu_buddy_block_is_clear(block);
> +}
> +
> +static struct gpu_buddy_block *
> +__alloc_range_bias(struct gpu_buddy *mm,
> + u64 start, u64 end,
> + unsigned int order,
> + unsigned long flags,
> + bool fallback)
> +{
> + u64 req_size = mm->chunk_size << order;
> + struct gpu_buddy_block *block;
> + struct gpu_buddy_block *buddy;
> + LIST_HEAD(dfs);
> + int err;
> + int i;
> +
> + end = end - 1;
> +
> + for (i = 0; i < mm->n_roots; ++i)
> + list_add_tail(&mm->roots[i]->tmp_link, &dfs);
> +
> + do {
> + u64 block_start;
> + u64 block_end;
> +
> + block = list_first_entry_or_null(&dfs,
> + struct gpu_buddy_block,
> + tmp_link);
> + if (!block)
> + break;
> +
> + list_del(&block->tmp_link);
> +
> + if (gpu_buddy_block_order(block) < order)
> + continue;
> +
> + block_start = gpu_buddy_block_offset(block);
> + block_end = block_start + gpu_buddy_block_size(mm, block) - 1;
> +
> + if (!overlaps(start, end, block_start, block_end))
> + continue;
> +
> + if (gpu_buddy_block_is_allocated(block))
> + continue;
> +
> + if (block_start < start || block_end > end) {
> + u64 adjusted_start = max(block_start, start);
> + u64 adjusted_end = min(block_end, end);
> +
> + if (round_down(adjusted_end + 1, req_size) <=
> + round_up(adjusted_start, req_size))
> + continue;
> + }
> +
> + if (!fallback && block_incompatible(block, flags))
> + continue;
> +
> + if (contains(start, end, block_start, block_end) &&
> + order == gpu_buddy_block_order(block)) {
> + /*
> + * Find the free block within the range.
> + */
> + if (gpu_buddy_block_is_free(block))
> + return block;
> +
> + continue;
> + }
> +
> + if (!gpu_buddy_block_is_split(block)) {
> + err = split_block(mm, block);
> + if (unlikely(err))
> + goto err_undo;
> + }
> +
> + list_add(&block->right->tmp_link, &dfs);
> + list_add(&block->left->tmp_link, &dfs);
> + } while (1);
> +
> + return ERR_PTR(-ENOSPC);
> +
> +err_undo:
> + /*
> + * We really don't want to leave around a bunch of split blocks, since
> + * bigger is better, so make sure we merge everything back before we
> + * free the allocated blocks.
> + */
> + buddy = __get_buddy(block);
> + if (buddy &&
> + (gpu_buddy_block_is_free(block) &&
> + gpu_buddy_block_is_free(buddy)))
> + __gpu_buddy_free(mm, block, false);
> + return ERR_PTR(err);
> +}
> +
> +static struct gpu_buddy_block *
> +__gpu_buddy_alloc_range_bias(struct gpu_buddy *mm,
> + u64 start, u64 end,
> + unsigned int order,
> + unsigned long flags)
> +{
> + struct gpu_buddy_block *block;
> + bool fallback = false;
> +
> + block = __alloc_range_bias(mm, start, end, order,
> + flags, fallback);
> + if (IS_ERR(block))
> + return __alloc_range_bias(mm, start, end, order,
> + flags, !fallback);
> +
> + return block;
> +}
> +
> +static struct gpu_buddy_block *
> +get_maxblock(struct gpu_buddy *mm,
> + unsigned int order,
> + enum gpu_buddy_free_tree tree)
> +{
> + struct gpu_buddy_block *max_block = NULL, *block = NULL;
> + struct rb_root *root;
> + unsigned int i;
> +
> + for (i = order; i <= mm->max_order; ++i) {
> + root = &mm->free_trees[tree][i];
> + block = rbtree_last_free_block(root);
> + if (!block)
> + continue;
> +
> + if (!max_block) {
> + max_block = block;
> + continue;
> + }
> +
> + if (gpu_buddy_block_offset(block) >
> + gpu_buddy_block_offset(max_block)) {
> + max_block = block;
> + }
> + }
> +
> + return max_block;
> +}
> +
> +static struct gpu_buddy_block *
> +alloc_from_freetree(struct gpu_buddy *mm,
> + unsigned int order,
> + unsigned long flags)
> +{
> + struct gpu_buddy_block *block = NULL;
> + struct rb_root *root;
> + enum gpu_buddy_free_tree tree;
> + unsigned int tmp;
> + int err;
> +
> + tree = (flags & GPU_BUDDY_CLEAR_ALLOCATION) ?
> + GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE;
> +
> + if (flags & GPU_BUDDY_TOPDOWN_ALLOCATION) {
> + block = get_maxblock(mm, order, tree);
> + if (block)
> + /* Store the obtained block order */
> + tmp = gpu_buddy_block_order(block);
> + } else {
> + for (tmp = order; tmp <= mm->max_order; ++tmp) {
> + /* Get RB tree root for this order and tree */
> + root = &mm->free_trees[tree][tmp];
> + block = rbtree_last_free_block(root);
> + if (block)
> + break;
> + }
> + }
> +
> + if (!block) {
> + /* Try allocating from the other tree */
> + tree = (tree == GPU_BUDDY_CLEAR_TREE) ?
> + GPU_BUDDY_DIRTY_TREE : GPU_BUDDY_CLEAR_TREE;
> +
> + for (tmp = order; tmp <= mm->max_order; ++tmp) {
> + root = &mm->free_trees[tree][tmp];
> + block = rbtree_last_free_block(root);
> + if (block)
> + break;
> + }
> +
> + if (!block)
> + return ERR_PTR(-ENOSPC);
> + }
> +
> + BUG_ON(!gpu_buddy_block_is_free(block));
> +
> + while (tmp != order) {
> + err = split_block(mm, block);
> + if (unlikely(err))
> + goto err_undo;
> +
> + block = block->right;
> + tmp--;
> + }
> + return block;
> +
> +err_undo:
> + if (tmp != order)
> + __gpu_buddy_free(mm, block, false);
> + return ERR_PTR(err);
> +}
> +
> +static int __alloc_range(struct gpu_buddy *mm,
> + struct list_head *dfs,
> + u64 start, u64 size,
> + struct list_head *blocks,
> + u64 *total_allocated_on_err)
> +{
> + struct gpu_buddy_block *block;
> + struct gpu_buddy_block *buddy;
> + u64 total_allocated = 0;
> + LIST_HEAD(allocated);
> + u64 end;
> + int err;
> +
> + end = start + size - 1;
> +
> + do {
> + u64 block_start;
> + u64 block_end;
> +
> + block = list_first_entry_or_null(dfs,
> + struct gpu_buddy_block,
> + tmp_link);
> + if (!block)
> + break;
> +
> + list_del(&block->tmp_link);
> +
> + block_start = gpu_buddy_block_offset(block);
> + block_end = block_start + gpu_buddy_block_size(mm, block) - 1;
> +
> + if (!overlaps(start, end, block_start, block_end))
> + continue;
> +
> + if (gpu_buddy_block_is_allocated(block)) {
> + err = -ENOSPC;
> + goto err_free;
> + }
> +
> + if (contains(start, end, block_start, block_end)) {
> + if (gpu_buddy_block_is_free(block)) {
> + mark_allocated(mm, block);
> + total_allocated += gpu_buddy_block_size(mm, block);
> + mm->avail -= gpu_buddy_block_size(mm, block);
> + if (gpu_buddy_block_is_clear(block))
> + mm->clear_avail -= gpu_buddy_block_size(mm, block);
> + list_add_tail(&block->link, &allocated);
> + continue;
> + } else if (!mm->clear_avail) {
> + err = -ENOSPC;
> + goto err_free;
> + }
> + }
> +
> + if (!gpu_buddy_block_is_split(block)) {
> + err = split_block(mm, block);
> + if (unlikely(err))
> + goto err_undo;
> + }
> +
> + list_add(&block->right->tmp_link, dfs);
> + list_add(&block->left->tmp_link, dfs);
> + } while (1);
> +
> + if (total_allocated < size) {
> + err = -ENOSPC;
> + goto err_free;
> + }
> +
> + list_splice_tail(&allocated, blocks);
> +
> + return 0;
> +
> +err_undo:
> + /*
> + * We really don't want to leave around a bunch of split blocks, since
> + * bigger is better, so make sure we merge everything back before we
> + * free the allocated blocks.
> + */
> + buddy = __get_buddy(block);
> + if (buddy &&
> + (gpu_buddy_block_is_free(block) &&
> + gpu_buddy_block_is_free(buddy)))
> + __gpu_buddy_free(mm, block, false);
> +
> +err_free:
> + if (err == -ENOSPC && total_allocated_on_err) {
> + list_splice_tail(&allocated, blocks);
> + *total_allocated_on_err = total_allocated;
> + } else {
> + gpu_buddy_free_list_internal(mm, &allocated);
> + }
> +
> + return err;
> +}
> +
> +static int __gpu_buddy_alloc_range(struct gpu_buddy *mm,
> + u64 start,
> + u64 size,
> + u64 *total_allocated_on_err,
> + struct list_head *blocks)
> +{
> + LIST_HEAD(dfs);
> + int i;
> +
> + for (i = 0; i < mm->n_roots; ++i)
> + list_add_tail(&mm->roots[i]->tmp_link, &dfs);
> +
> + return __alloc_range(mm, &dfs, start, size,
> + blocks, total_allocated_on_err);
> +}
> +
> +static int __alloc_contig_try_harder(struct gpu_buddy *mm,
> + u64 size,
> + u64 min_block_size,
> + struct list_head *blocks)
> +{
> + u64 rhs_offset, lhs_offset, lhs_size, filled;
> + struct gpu_buddy_block *block;
> + unsigned int tree, order;
> + LIST_HEAD(blocks_lhs);
> + unsigned long pages;
> + u64 modify_size;
> + int err;
> +
> + modify_size = rounddown_pow_of_two(size);
> + pages = modify_size >> ilog2(mm->chunk_size);
> + order = fls(pages) - 1;
> + if (order == 0)
> + return -ENOSPC;
> +
> + for_each_free_tree(tree) {
> + struct rb_root *root;
> + struct rb_node *iter;
> +
> + root = &mm->free_trees[tree][order];
> + if (rbtree_is_empty(root))
> + continue;
> +
> + iter = rb_last(root);
> + while (iter) {
> + block = rbtree_get_free_block(iter);
> +
> + /* Allocate blocks traversing RHS */
> + rhs_offset = gpu_buddy_block_offset(block);
> + err = __gpu_buddy_alloc_range(mm, rhs_offset, size,
> + &filled, blocks);
> + if (!err || err != -ENOSPC)
> + return err;
> +
> + lhs_size = max((size - filled), min_block_size);
> + if (!IS_ALIGNED(lhs_size, min_block_size))
> + lhs_size = round_up(lhs_size, min_block_size);
> +
> + /* Allocate blocks traversing LHS */
> + lhs_offset = gpu_buddy_block_offset(block) - lhs_size;
> + err = __gpu_buddy_alloc_range(mm, lhs_offset, lhs_size,
> + NULL, &blocks_lhs);
> + if (!err) {
> + list_splice(&blocks_lhs, blocks);
> + return 0;
> + } else if (err != -ENOSPC) {
> + gpu_buddy_free_list_internal(mm, blocks);
> + return err;
> + }
> + /* Free blocks for the next iteration */
> + gpu_buddy_free_list_internal(mm, blocks);
> +
> + iter = rb_prev(iter);
> + }
> + }
> +
> + return -ENOSPC;
> +}
> +
> +/**
> + * gpu_buddy_block_trim - free unused pages
> + *
> + * @mm: GPU buddy manager
> + * @start: start address to begin the trimming.
> + * @new_size: original size requested
> + * @blocks: Input and output list of allocated blocks.
> + * MUST contain single block as input to be trimmed.
> + * On success will contain the newly allocated blocks
> + * making up the @new_size. Blocks always appear in
> + * ascending order
> + *
> + * For contiguous allocation, we round up the size to the nearest
> + * power of two value, drivers consume *actual* size, so remaining
> + * portions are unused and can be optionally freed with this function
> + *
> + * Returns:
> + * 0 on success, error code on failure.
> + */
> +int gpu_buddy_block_trim(struct gpu_buddy *mm,
> + u64 *start,
> + u64 new_size,
> + struct list_head *blocks)
> +{
> + struct gpu_buddy_block *parent;
> + struct gpu_buddy_block *block;
> + u64 block_start, block_end;
> + LIST_HEAD(dfs);
> + u64 new_start;
> + int err;
> +
> + if (!list_is_singular(blocks))
> + return -EINVAL;
> +
> + block = list_first_entry(blocks,
> + struct gpu_buddy_block,
> + link);
> +
> + block_start = gpu_buddy_block_offset(block);
> + block_end = block_start + gpu_buddy_block_size(mm, block);
> +
> + if (WARN_ON(!gpu_buddy_block_is_allocated(block)))
> + return -EINVAL;
> +
> + if (new_size > gpu_buddy_block_size(mm, block))
> + return -EINVAL;
> +
> + if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size))
> + return -EINVAL;
> +
> + if (new_size == gpu_buddy_block_size(mm, block))
> + return 0;
> +
> + new_start = block_start;
> + if (start) {
> + new_start = *start;
> +
> + if (new_start < block_start)
> + return -EINVAL;
> +
> + if (!IS_ALIGNED(new_start, mm->chunk_size))
> + return -EINVAL;
> +
> + if (range_overflows(new_start, new_size, block_end))
> + return -EINVAL;
> + }
> +
> + list_del(&block->link);
> + mark_free(mm, block);
> + mm->avail += gpu_buddy_block_size(mm, block);
> + if (gpu_buddy_block_is_clear(block))
> + mm->clear_avail += gpu_buddy_block_size(mm, block);
> +
> + /* Prevent recursively freeing this node */
> + parent = block->parent;
> + block->parent = NULL;
> +
> + list_add(&block->tmp_link, &dfs);
> + err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL);
> + if (err) {
> + mark_allocated(mm, block);
> + mm->avail -= gpu_buddy_block_size(mm, block);
> + if (gpu_buddy_block_is_clear(block))
> + mm->clear_avail -= gpu_buddy_block_size(mm, block);
> + list_add(&block->link, blocks);
> + }
> +
> + block->parent = parent;
> + return err;
> +}
> +EXPORT_SYMBOL(gpu_buddy_block_trim);
> +
> +static struct gpu_buddy_block *
> +__gpu_buddy_alloc_blocks(struct gpu_buddy *mm,
> + u64 start, u64 end,
> + unsigned int order,
> + unsigned long flags)
> +{
> + if (flags & GPU_BUDDY_RANGE_ALLOCATION)
> + /* Allocate traversing within the range */
> + return __gpu_buddy_alloc_range_bias(mm, start, end,
> + order, flags);
> + else
> + /* Allocate from freetree */
> + return alloc_from_freetree(mm, order, flags);
> +}
> +
> +/**
> + * gpu_buddy_alloc_blocks - allocate power-of-two blocks
> + *
> + * @mm: GPU buddy manager to allocate from
> + * @start: start of the allowed range for this block
> + * @end: end of the allowed range for this block
> + * @size: size of the allocation in bytes
> + * @min_block_size: alignment of the allocation
> + * @blocks: output list head to add allocated blocks
> + * @flags: GPU_BUDDY_*_ALLOCATION flags
> + *
> + * alloc_range_bias() called on range limitations, which traverses
> + * the tree and returns the desired block.
> + *
> + * alloc_from_freetree() called when *no* range restrictions
> + * are enforced, which picks the block from the freetree.
> + *
> + * Returns:
> + * 0 on success, error code on failure.
> + */
> +int gpu_buddy_alloc_blocks(struct gpu_buddy *mm,
> + u64 start, u64 end, u64 size,
> + u64 min_block_size,
> + struct list_head *blocks,
> + unsigned long flags)
> +{
> + struct gpu_buddy_block *block = NULL;
> + u64 original_size, original_min_size;
> + unsigned int min_order, order;
> + LIST_HEAD(allocated);
> + unsigned long pages;
> + int err;
> +
> + if (size < mm->chunk_size)
> + return -EINVAL;
> +
> + if (min_block_size < mm->chunk_size)
> + return -EINVAL;
> +
> + if (!is_power_of_2(min_block_size))
> + return -EINVAL;
> +
> + if (!IS_ALIGNED(start | end | size, mm->chunk_size))
> + return -EINVAL;
> +
> + if (end > mm->size)
> + return -EINVAL;
> +
> + if (range_overflows(start, size, mm->size))
> + return -EINVAL;
> +
> + /* Actual range allocation */
> + if (start + size == end) {
> + if (!IS_ALIGNED(start | end, min_block_size))
> + return -EINVAL;
> +
> + return __gpu_buddy_alloc_range(mm, start, size, NULL, blocks);
> + }
> +
> + original_size = size;
> + original_min_size = min_block_size;
> +
> + /* Roundup the size to power of 2 */
> + if (flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION) {
> + size = roundup_pow_of_two(size);
> + min_block_size = size;
> + /* Align size value to min_block_size */
> + } else if (!IS_ALIGNED(size, min_block_size)) {
> + size = round_up(size, min_block_size);
> + }
> +
> + pages = size >> ilog2(mm->chunk_size);
> + order = fls(pages) - 1;
> + min_order = ilog2(min_block_size) - ilog2(mm->chunk_size);
> +
> + do {
> + order = min(order, (unsigned int)fls(pages) - 1);
> + BUG_ON(order > mm->max_order);
> + BUG_ON(order < min_order);
> +
> + do {
> + block = __gpu_buddy_alloc_blocks(mm, start,
> + end,
> + order,
> + flags);
> + if (!IS_ERR(block))
> + break;
> +
> + if (order-- == min_order) {
> + /* Try allocation through force merge method */
> + if (mm->clear_avail &&
> + !__force_merge(mm, start, end, min_order)) {
> + block = __gpu_buddy_alloc_blocks(mm, start,
> + end,
> + min_order,
> + flags);
> + if (!IS_ERR(block)) {
> + order = min_order;
> + break;
> + }
> + }
> +
> + /*
> + * Try contiguous block allocation through
> + * try harder method.
> + */
> + if (flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION &&
> + !(flags & GPU_BUDDY_RANGE_ALLOCATION))
> + return __alloc_contig_try_harder(mm,
> + original_size,
> + original_min_size,
> + blocks);
> + err = -ENOSPC;
> + goto err_free;
> + }
> + } while (1);
> +
> + mark_allocated(mm, block);
> + mm->avail -= gpu_buddy_block_size(mm, block);
> + if (gpu_buddy_block_is_clear(block))
> + mm->clear_avail -= gpu_buddy_block_size(mm, block);
> + kmemleak_update_trace(block);
> + list_add_tail(&block->link, &allocated);
> +
> + pages -= BIT(order);
> +
> + if (!pages)
> + break;
> + } while (1);
> +
> + /* Trim the allocated block to the required size */
> + if (!(flags & GPU_BUDDY_TRIM_DISABLE) &&
> + original_size != size) {
> + struct list_head *trim_list;
> + LIST_HEAD(temp);
> + u64 trim_size;
> +
> + trim_list = &allocated;
> + trim_size = original_size;
> +
> + if (!list_is_singular(&allocated)) {
> + block = list_last_entry(&allocated, typeof(*block), link);
> + list_move(&block->link, &temp);
> + trim_list = &temp;
> + trim_size = gpu_buddy_block_size(mm, block) -
> + (size - original_size);
> + }
> +
> + gpu_buddy_block_trim(mm,
> + NULL,
> + trim_size,
> + trim_list);
> +
> + if (!list_empty(&temp))
> + list_splice_tail(trim_list, &allocated);
> + }
> +
> + list_splice_tail(&allocated, blocks);
> + return 0;
> +
> +err_free:
> + gpu_buddy_free_list_internal(mm, &allocated);
> + return err;
> +}
> +EXPORT_SYMBOL(gpu_buddy_alloc_blocks);
> +
> +/**
> + * gpu_buddy_block_print - print block information
> + *
> + * @mm: GPU buddy manager
> + * @block: GPU buddy block
> + */
> +void gpu_buddy_block_print(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + u64 start = gpu_buddy_block_offset(block);
> + u64 size = gpu_buddy_block_size(mm, block);
> +
> + pr_info("%#018llx-%#018llx: %llu\n", start, start + size, size);
> +}
> +EXPORT_SYMBOL(gpu_buddy_block_print);
> +
> +/**
> + * gpu_buddy_print - print allocator state
> + *
> + * @mm: GPU buddy manager
> + */
> +void gpu_buddy_print(struct gpu_buddy *mm)
> +{
> + int order;
> +
> + pr_info("chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n",
> + mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
> +
> + for (order = mm->max_order; order >= 0; order--) {
> + struct gpu_buddy_block *block, *tmp;
> + struct rb_root *root;
> + u64 count = 0, free;
> + unsigned int tree;
> +
> + for_each_free_tree(tree) {
> + root = &mm->free_trees[tree][order];
> +
> + rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
> + BUG_ON(!gpu_buddy_block_is_free(block));
> + count++;
> + }
> + }
> +
> + free = count * (mm->chunk_size << order);
> + if (free < SZ_1M)
> + pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
> + order, free >> 10, count);
> + else
> + pr_info("order-%2d free: %8llu MiB, blocks: %llu\n",
> + order, free >> 20, count);
> + }
> +}
> +EXPORT_SYMBOL(gpu_buddy_print);
> +
> +static void gpu_buddy_module_exit(void)
> +{
> + kmem_cache_destroy(slab_blocks);
> +}
> +
> +static int __init gpu_buddy_module_init(void)
> +{
> + slab_blocks = KMEM_CACHE(gpu_buddy_block, 0);
> + if (!slab_blocks)
> + return -ENOMEM;
> +
> + return 0;
> +}
> +
> +module_init(gpu_buddy_module_init);
> +module_exit(gpu_buddy_module_exit);
> +
> +MODULE_DESCRIPTION("GPU Buddy Allocator");
> +MODULE_LICENSE("Dual MIT/GPL");
> diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
> index 7e6bc0b3a589..0475defb37f0 100644
> --- a/drivers/gpu/drm/Kconfig
> +++ b/drivers/gpu/drm/Kconfig
> @@ -220,6 +220,7 @@ config DRM_GPUSVM
> config DRM_BUDDY
> tristate
> depends on DRM
> + select GPU_BUDDY
> help
> A page based buddy allocator
>
> diff --git a/drivers/gpu/drm/Kconfig.debug b/drivers/gpu/drm/Kconfig.debug
> index 05dc43c0b8c5..1f4c408c7920 100644
> --- a/drivers/gpu/drm/Kconfig.debug
> +++ b/drivers/gpu/drm/Kconfig.debug
> @@ -71,6 +71,7 @@ config DRM_KUNIT_TEST
> select DRM_KUNIT_TEST_HELPERS
> select DRM_LIB_RANDOM
> select DRM_SYSFB_HELPER
> + select GPU_BUDDY
> select PRIME_NUMBERS
> default KUNIT_ALL_TESTS
> help
> @@ -88,10 +89,11 @@ config DRM_TTM_KUNIT_TEST
> tristate "KUnit tests for TTM" if !KUNIT_ALL_TESTS
> default n
> depends on DRM && KUNIT && MMU && (UML || COMPILE_TEST)
> - select DRM_TTM
> select DRM_BUDDY
> + select DRM_TTM
> select DRM_EXPORT_FOR_TESTS if m
> select DRM_KUNIT_TEST_HELPERS
> + select GPU_BUDDY
> default KUNIT_ALL_TESTS
> help
> Enables unit tests for TTM, a GPU memory manager subsystem used
> diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig b/drivers/gpu/drm/amd/amdgpu/Kconfig
> index 7f515be5185d..bb131543e1d9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/Kconfig
> +++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
> @@ -23,6 +23,7 @@ config DRM_AMDGPU
> select CRC16
> select BACKLIGHT_CLASS_DEVICE
> select INTERVAL_TREE
> + select GPU_BUDDY
> select DRM_BUDDY
> select DRM_SUBALLOC_HELPER
> select DRM_EXEC
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> index 2a6cf7963dde..e0bd8a68877f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> @@ -5654,7 +5654,7 @@ int amdgpu_ras_add_critical_region(struct amdgpu_device *adev,
> struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
> struct amdgpu_vram_mgr_resource *vres;
> struct ras_critical_region *region;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> int ret = 0;
>
> if (!bo || !bo->tbo.resource)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
> index be2e56ce1355..8908d9e08a30 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
> @@ -55,7 +55,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
> uint64_t start, uint64_t size,
> struct amdgpu_res_cursor *cur)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> struct list_head *head, *next;
> struct drm_mm_node *node;
>
> @@ -71,7 +71,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
> head = &to_amdgpu_vram_mgr_resource(res)->blocks;
>
> block = list_first_entry_or_null(head,
> - struct drm_buddy_block,
> + struct gpu_buddy_block,
> link);
> if (!block)
> goto fallback;
> @@ -81,7 +81,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
>
> next = block->link.next;
> if (next != head)
> - block = list_entry(next, struct drm_buddy_block, link);
> + block = list_entry(next, struct gpu_buddy_block, link);
> }
>
> cur->start = amdgpu_vram_mgr_block_start(block) + start;
> @@ -125,7 +125,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
> */
> static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> struct drm_mm_node *node;
> struct list_head *next;
>
> @@ -146,7 +146,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
> block = cur->node;
>
> next = block->link.next;
> - block = list_entry(next, struct drm_buddy_block, link);
> + block = list_entry(next, struct gpu_buddy_block, link);
>
> cur->node = block;
> cur->start = amdgpu_vram_mgr_block_start(block);
> @@ -175,7 +175,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
> */
> static inline bool amdgpu_res_cleared(struct amdgpu_res_cursor *cur)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
>
> switch (cur->mem_type) {
> case TTM_PL_VRAM:
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> index 9d934c07fa6b..6c06a9c9b13f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> @@ -23,6 +23,8 @@
> */
>
> #include <linux/dma-mapping.h>
> +
> +#include <drm/drm_buddy.h>
> #include <drm/ttm/ttm_range_manager.h>
> #include <drm/drm_drv.h>
>
> @@ -52,15 +54,15 @@ to_amdgpu_device(struct amdgpu_vram_mgr *mgr)
> return container_of(mgr, struct amdgpu_device, mman.vram_mgr);
> }
>
> -static inline struct drm_buddy_block *
> +static inline struct gpu_buddy_block *
> amdgpu_vram_mgr_first_block(struct list_head *list)
> {
> - return list_first_entry_or_null(list, struct drm_buddy_block, link);
> + return list_first_entry_or_null(list, struct gpu_buddy_block, link);
> }
>
> static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> u64 start, size;
>
> block = amdgpu_vram_mgr_first_block(head);
> @@ -71,7 +73,7 @@ static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head)
> start = amdgpu_vram_mgr_block_start(block);
> size = amdgpu_vram_mgr_block_size(block);
>
> - block = list_entry(block->link.next, struct drm_buddy_block, link);
> + block = list_entry(block->link.next, struct gpu_buddy_block, link);
> if (start + size != amdgpu_vram_mgr_block_start(block))
> return false;
> }
> @@ -81,7 +83,7 @@ static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head)
>
> static inline u64 amdgpu_vram_mgr_blocks_size(struct list_head *head)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> u64 size = 0;
>
> list_for_each_entry(block, head, link)
> @@ -254,7 +256,7 @@ const struct attribute_group amdgpu_vram_mgr_attr_group = {
> * Calculate how many bytes of the DRM BUDDY block are inside visible VRAM
> */
> static u64 amdgpu_vram_mgr_vis_size(struct amdgpu_device *adev,
> - struct drm_buddy_block *block)
> + struct gpu_buddy_block *block)
> {
> u64 start = amdgpu_vram_mgr_block_start(block);
> u64 end = start + amdgpu_vram_mgr_block_size(block);
> @@ -279,7 +281,7 @@ u64 amdgpu_vram_mgr_bo_visible_size(struct amdgpu_bo *bo)
> struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
> struct ttm_resource *res = bo->tbo.resource;
> struct amdgpu_vram_mgr_resource *vres = to_amdgpu_vram_mgr_resource(res);
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> u64 usage = 0;
>
> if (amdgpu_gmc_vram_full_visible(&adev->gmc))
> @@ -299,15 +301,15 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man)
> {
> struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
> struct amdgpu_device *adev = to_amdgpu_device(mgr);
> - struct drm_buddy *mm = &mgr->mm;
> + struct gpu_buddy *mm = &mgr->mm;
> struct amdgpu_vram_reservation *rsv, *temp;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> uint64_t vis_usage;
>
> list_for_each_entry_safe(rsv, temp, &mgr->reservations_pending, blocks) {
> - if (drm_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size,
> + if (gpu_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size,
> rsv->size, mm->chunk_size, &rsv->allocated,
> - DRM_BUDDY_RANGE_ALLOCATION))
> + GPU_BUDDY_RANGE_ALLOCATION))
> continue;
>
> block = amdgpu_vram_mgr_first_block(&rsv->allocated);
> @@ -403,7 +405,7 @@ int amdgpu_vram_mgr_query_address_block_info(struct amdgpu_vram_mgr *mgr,
> uint64_t address, struct amdgpu_vram_block_info *info)
> {
> struct amdgpu_vram_mgr_resource *vres;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> u64 start, size;
> int ret = -ENOENT;
>
> @@ -450,8 +452,8 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
> struct amdgpu_vram_mgr_resource *vres;
> u64 size, remaining_size, lpfn, fpfn;
> unsigned int adjust_dcc_size = 0;
> - struct drm_buddy *mm = &mgr->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = &mgr->mm;
> + struct gpu_buddy_block *block;
> unsigned long pages_per_block;
> int r;
>
> @@ -493,17 +495,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
> INIT_LIST_HEAD(&vres->blocks);
>
> if (place->flags & TTM_PL_FLAG_TOPDOWN)
> - vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
> + vres->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
>
> if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS)
> - vres->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
> + vres->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
>
> if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CLEARED)
> - vres->flags |= DRM_BUDDY_CLEAR_ALLOCATION;
> + vres->flags |= GPU_BUDDY_CLEAR_ALLOCATION;
>
> if (fpfn || lpfn != mgr->mm.size)
> /* Allocate blocks in desired range */
> - vres->flags |= DRM_BUDDY_RANGE_ALLOCATION;
> + vres->flags |= GPU_BUDDY_RANGE_ALLOCATION;
>
> if (bo->flags & AMDGPU_GEM_CREATE_GFX12_DCC &&
> adev->gmc.gmc_funcs->get_dcc_alignment)
> @@ -516,7 +518,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
> dcc_size = roundup_pow_of_two(vres->base.size + adjust_dcc_size);
> remaining_size = (u64)dcc_size;
>
> - vres->flags |= DRM_BUDDY_TRIM_DISABLE;
> + vres->flags |= GPU_BUDDY_TRIM_DISABLE;
> }
>
> mutex_lock(&mgr->lock);
> @@ -536,7 +538,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
>
> BUG_ON(min_block_size < mm->chunk_size);
>
> - r = drm_buddy_alloc_blocks(mm, fpfn,
> + r = gpu_buddy_alloc_blocks(mm, fpfn,
> lpfn,
> size,
> min_block_size,
> @@ -545,7 +547,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
>
> if (unlikely(r == -ENOSPC) && pages_per_block == ~0ul &&
> !(place->flags & TTM_PL_FLAG_CONTIGUOUS)) {
> - vres->flags &= ~DRM_BUDDY_CONTIGUOUS_ALLOCATION;
> + vres->flags &= ~GPU_BUDDY_CONTIGUOUS_ALLOCATION;
> pages_per_block = max_t(u32, 2UL << (20UL - PAGE_SHIFT),
> tbo->page_alignment);
>
> @@ -566,7 +568,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
> list_add_tail(&vres->vres_node, &mgr->allocated_vres_list);
>
> if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS && adjust_dcc_size) {
> - struct drm_buddy_block *dcc_block;
> + struct gpu_buddy_block *dcc_block;
> unsigned long dcc_start;
> u64 trim_start;
>
> @@ -576,7 +578,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
> roundup((unsigned long)amdgpu_vram_mgr_block_start(dcc_block),
> adjust_dcc_size);
> trim_start = (u64)dcc_start;
> - drm_buddy_block_trim(mm, &trim_start,
> + gpu_buddy_block_trim(mm, &trim_start,
> (u64)vres->base.size,
> &vres->blocks);
> }
> @@ -614,7 +616,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
> return 0;
>
> error_free_blocks:
> - drm_buddy_free_list(mm, &vres->blocks, 0);
> + gpu_buddy_free_list(mm, &vres->blocks, 0);
> mutex_unlock(&mgr->lock);
> error_fini:
> ttm_resource_fini(man, &vres->base);
> @@ -637,8 +639,8 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man,
> struct amdgpu_vram_mgr_resource *vres = to_amdgpu_vram_mgr_resource(res);
> struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
> struct amdgpu_device *adev = to_amdgpu_device(mgr);
> - struct drm_buddy *mm = &mgr->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = &mgr->mm;
> + struct gpu_buddy_block *block;
> uint64_t vis_usage = 0;
>
> mutex_lock(&mgr->lock);
> @@ -649,7 +651,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man,
> list_for_each_entry(block, &vres->blocks, link)
> vis_usage += amdgpu_vram_mgr_vis_size(adev, block);
>
> - drm_buddy_free_list(mm, &vres->blocks, vres->flags);
> + gpu_buddy_free_list(mm, &vres->blocks, vres->flags);
> amdgpu_vram_mgr_do_reserve(man);
> mutex_unlock(&mgr->lock);
>
> @@ -688,7 +690,7 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
> if (!*sgt)
> return -ENOMEM;
>
> - /* Determine the number of DRM_BUDDY blocks to export */
> + /* Determine the number of GPU_BUDDY blocks to export */
> amdgpu_res_first(res, offset, length, &cursor);
> while (cursor.remaining) {
> num_entries++;
> @@ -704,10 +706,10 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
> sg->length = 0;
>
> /*
> - * Walk down DRM_BUDDY blocks to populate scatterlist nodes
> - * @note: Use iterator api to get first the DRM_BUDDY block
> + * Walk down GPU_BUDDY blocks to populate scatterlist nodes
> + * @note: Use iterator api to get first the GPU_BUDDY block
> * and the number of bytes from it. Access the following
> - * DRM_BUDDY block(s) if more buffer needs to exported
> + * GPU_BUDDY block(s) if more buffer needs to exported
> */
> amdgpu_res_first(res, offset, length, &cursor);
> for_each_sgtable_sg((*sgt), sg, i) {
> @@ -792,10 +794,10 @@ uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr)
> void amdgpu_vram_mgr_clear_reset_blocks(struct amdgpu_device *adev)
> {
> struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
> - struct drm_buddy *mm = &mgr->mm;
> + struct gpu_buddy *mm = &mgr->mm;
>
> mutex_lock(&mgr->lock);
> - drm_buddy_reset_clear(mm, false);
> + gpu_buddy_reset_clear(mm, false);
> mutex_unlock(&mgr->lock);
> }
>
> @@ -815,7 +817,7 @@ static bool amdgpu_vram_mgr_intersects(struct ttm_resource_manager *man,
> size_t size)
> {
> struct amdgpu_vram_mgr_resource *mgr = to_amdgpu_vram_mgr_resource(res);
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
>
> /* Check each drm buddy block individually */
> list_for_each_entry(block, &mgr->blocks, link) {
> @@ -848,7 +850,7 @@ static bool amdgpu_vram_mgr_compatible(struct ttm_resource_manager *man,
> size_t size)
> {
> struct amdgpu_vram_mgr_resource *mgr = to_amdgpu_vram_mgr_resource(res);
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
>
> /* Check each drm buddy block individually */
> list_for_each_entry(block, &mgr->blocks, link) {
> @@ -877,7 +879,7 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man,
> struct drm_printer *printer)
> {
> struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
> - struct drm_buddy *mm = &mgr->mm;
> + struct gpu_buddy *mm = &mgr->mm;
> struct amdgpu_vram_reservation *rsv;
>
> drm_printf(printer, " vis usage:%llu\n",
> @@ -930,7 +932,7 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
> mgr->default_page_size = PAGE_SIZE;
>
> man->func = &amdgpu_vram_mgr_func;
> - err = drm_buddy_init(&mgr->mm, man->size, PAGE_SIZE);
> + err = gpu_buddy_init(&mgr->mm, man->size, PAGE_SIZE);
> if (err)
> return err;
>
> @@ -965,11 +967,11 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
> kfree(rsv);
>
> list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) {
> - drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0);
> + gpu_buddy_free_list(&mgr->mm, &rsv->allocated, 0);
> kfree(rsv);
> }
> if (!adev->gmc.is_app_apu)
> - drm_buddy_fini(&mgr->mm);
> + gpu_buddy_fini(&mgr->mm);
> mutex_unlock(&mgr->lock);
>
> ttm_resource_manager_cleanup(man);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h
> index 5f5fd9a911c2..429a21a2e9b2 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h
> @@ -24,11 +24,11 @@
> #ifndef __AMDGPU_VRAM_MGR_H__
> #define __AMDGPU_VRAM_MGR_H__
>
> -#include <drm/drm_buddy.h>
> +#include <linux/gpu_buddy.h>
>
> struct amdgpu_vram_mgr {
> struct ttm_resource_manager manager;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> /* protects access to buffer objects */
> struct mutex lock;
> struct list_head reservations_pending;
> @@ -57,19 +57,19 @@ struct amdgpu_vram_mgr_resource {
> struct amdgpu_vres_task task;
> };
>
> -static inline u64 amdgpu_vram_mgr_block_start(struct drm_buddy_block *block)
> +static inline u64 amdgpu_vram_mgr_block_start(struct gpu_buddy_block *block)
> {
> - return drm_buddy_block_offset(block);
> + return gpu_buddy_block_offset(block);
> }
>
> -static inline u64 amdgpu_vram_mgr_block_size(struct drm_buddy_block *block)
> +static inline u64 amdgpu_vram_mgr_block_size(struct gpu_buddy_block *block)
> {
> - return (u64)PAGE_SIZE << drm_buddy_block_order(block);
> + return (u64)PAGE_SIZE << gpu_buddy_block_order(block);
> }
>
> -static inline bool amdgpu_vram_mgr_is_cleared(struct drm_buddy_block *block)
> +static inline bool amdgpu_vram_mgr_is_cleared(struct gpu_buddy_block *block)
> {
> - return drm_buddy_block_is_clear(block);
> + return gpu_buddy_block_is_clear(block);
> }
>
> static inline struct amdgpu_vram_mgr_resource *
> @@ -82,8 +82,8 @@ static inline void amdgpu_vram_mgr_set_cleared(struct ttm_resource *res)
> {
> struct amdgpu_vram_mgr_resource *ares = to_amdgpu_vram_mgr_resource(res);
>
> - WARN_ON(ares->flags & DRM_BUDDY_CLEARED);
> - ares->flags |= DRM_BUDDY_CLEARED;
> + WARN_ON(ares->flags & GPU_BUDDY_CLEARED);
> + ares->flags |= GPU_BUDDY_CLEARED;
> }
>
> int amdgpu_vram_mgr_query_address_block_info(struct amdgpu_vram_mgr *mgr,
> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
> index 2f279b46bd2c..188b36054e59 100644
> --- a/drivers/gpu/drm/drm_buddy.c
> +++ b/drivers/gpu/drm/drm_buddy.c
> @@ -3,1262 +3,25 @@
> * Copyright © 2021 Intel Corporation
> */
>
> -#include <kunit/test-bug.h>
> -
> #include <linux/export.h>
> -#include <linux/kmemleak.h>
> #include <linux/module.h>
> #include <linux/sizes.h>
>
> #include <drm/drm_buddy.h>
> #include <drm/drm_print.h>
>
> -enum drm_buddy_free_tree {
> - DRM_BUDDY_CLEAR_TREE = 0,
> - DRM_BUDDY_DIRTY_TREE,
> - DRM_BUDDY_MAX_FREE_TREES,
> -};
> -
> -static struct kmem_cache *slab_blocks;
> -
> -#define for_each_free_tree(tree) \
> - for ((tree) = 0; (tree) < DRM_BUDDY_MAX_FREE_TREES; (tree)++)
> -
> -static struct drm_buddy_block *drm_block_alloc(struct drm_buddy *mm,
> - struct drm_buddy_block *parent,
> - unsigned int order,
> - u64 offset)
> -{
> - struct drm_buddy_block *block;
> -
> - BUG_ON(order > DRM_BUDDY_MAX_ORDER);
> -
> - block = kmem_cache_zalloc(slab_blocks, GFP_KERNEL);
> - if (!block)
> - return NULL;
> -
> - block->header = offset;
> - block->header |= order;
> - block->parent = parent;
> -
> - RB_CLEAR_NODE(&block->rb);
> -
> - BUG_ON(block->header & DRM_BUDDY_HEADER_UNUSED);
> - return block;
> -}
> -
> -static void drm_block_free(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - kmem_cache_free(slab_blocks, block);
> -}
> -
> -static enum drm_buddy_free_tree
> -get_block_tree(struct drm_buddy_block *block)
> -{
> - return drm_buddy_block_is_clear(block) ?
> - DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE;
> -}
> -
> -static struct drm_buddy_block *
> -rbtree_get_free_block(const struct rb_node *node)
> -{
> - return node ? rb_entry(node, struct drm_buddy_block, rb) : NULL;
> -}
> -
> -static struct drm_buddy_block *
> -rbtree_last_free_block(struct rb_root *root)
> -{
> - return rbtree_get_free_block(rb_last(root));
> -}
> -
> -static bool rbtree_is_empty(struct rb_root *root)
> -{
> - return RB_EMPTY_ROOT(root);
> -}
> -
> -static bool drm_buddy_block_offset_less(const struct drm_buddy_block *block,
> - const struct drm_buddy_block *node)
> -{
> - return drm_buddy_block_offset(block) < drm_buddy_block_offset(node);
> -}
> -
> -static bool rbtree_block_offset_less(struct rb_node *block,
> - const struct rb_node *node)
> -{
> - return drm_buddy_block_offset_less(rbtree_get_free_block(block),
> - rbtree_get_free_block(node));
> -}
> -
> -static void rbtree_insert(struct drm_buddy *mm,
> - struct drm_buddy_block *block,
> - enum drm_buddy_free_tree tree)
> -{
> - rb_add(&block->rb,
> - &mm->free_trees[tree][drm_buddy_block_order(block)],
> - rbtree_block_offset_less);
> -}
> -
> -static void rbtree_remove(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - unsigned int order = drm_buddy_block_order(block);
> - enum drm_buddy_free_tree tree;
> - struct rb_root *root;
> -
> - tree = get_block_tree(block);
> - root = &mm->free_trees[tree][order];
> -
> - rb_erase(&block->rb, root);
> - RB_CLEAR_NODE(&block->rb);
> -}
> -
> -static void clear_reset(struct drm_buddy_block *block)
> -{
> - block->header &= ~DRM_BUDDY_HEADER_CLEAR;
> -}
> -
> -static void mark_cleared(struct drm_buddy_block *block)
> -{
> - block->header |= DRM_BUDDY_HEADER_CLEAR;
> -}
> -
> -static void mark_allocated(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - block->header &= ~DRM_BUDDY_HEADER_STATE;
> - block->header |= DRM_BUDDY_ALLOCATED;
> -
> - rbtree_remove(mm, block);
> -}
> -
> -static void mark_free(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - enum drm_buddy_free_tree tree;
> -
> - block->header &= ~DRM_BUDDY_HEADER_STATE;
> - block->header |= DRM_BUDDY_FREE;
> -
> - tree = get_block_tree(block);
> - rbtree_insert(mm, block, tree);
> -}
> -
> -static void mark_split(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - block->header &= ~DRM_BUDDY_HEADER_STATE;
> - block->header |= DRM_BUDDY_SPLIT;
> -
> - rbtree_remove(mm, block);
> -}
> -
> -static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2)
> -{
> - return s1 <= e2 && e1 >= s2;
> -}
> -
> -static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2)
> -{
> - return s1 <= s2 && e1 >= e2;
> -}
> -
> -static struct drm_buddy_block *
> -__get_buddy(struct drm_buddy_block *block)
> -{
> - struct drm_buddy_block *parent;
> -
> - parent = block->parent;
> - if (!parent)
> - return NULL;
> -
> - if (parent->left == block)
> - return parent->right;
> -
> - return parent->left;
> -}
> -
> -static unsigned int __drm_buddy_free(struct drm_buddy *mm,
> - struct drm_buddy_block *block,
> - bool force_merge)
> -{
> - struct drm_buddy_block *parent;
> - unsigned int order;
> -
> - while ((parent = block->parent)) {
> - struct drm_buddy_block *buddy;
> -
> - buddy = __get_buddy(block);
> -
> - if (!drm_buddy_block_is_free(buddy))
> - break;
> -
> - if (!force_merge) {
> - /*
> - * Check the block and its buddy clear state and exit
> - * the loop if they both have the dissimilar state.
> - */
> - if (drm_buddy_block_is_clear(block) !=
> - drm_buddy_block_is_clear(buddy))
> - break;
> -
> - if (drm_buddy_block_is_clear(block))
> - mark_cleared(parent);
> - }
> -
> - rbtree_remove(mm, buddy);
> - if (force_merge && drm_buddy_block_is_clear(buddy))
> - mm->clear_avail -= drm_buddy_block_size(mm, buddy);
> -
> - drm_block_free(mm, block);
> - drm_block_free(mm, buddy);
> -
> - block = parent;
> - }
> -
> - order = drm_buddy_block_order(block);
> - mark_free(mm, block);
> -
> - return order;
> -}
> -
> -static int __force_merge(struct drm_buddy *mm,
> - u64 start,
> - u64 end,
> - unsigned int min_order)
> -{
> - unsigned int tree, order;
> - int i;
> -
> - if (!min_order)
> - return -ENOMEM;
> -
> - if (min_order > mm->max_order)
> - return -EINVAL;
> -
> - for_each_free_tree(tree) {
> - for (i = min_order - 1; i >= 0; i--) {
> - struct rb_node *iter = rb_last(&mm->free_trees[tree][i]);
> -
> - while (iter) {
> - struct drm_buddy_block *block, *buddy;
> - u64 block_start, block_end;
> -
> - block = rbtree_get_free_block(iter);
> - iter = rb_prev(iter);
> -
> - if (!block || !block->parent)
> - continue;
> -
> - block_start = drm_buddy_block_offset(block);
> - block_end = block_start + drm_buddy_block_size(mm, block) - 1;
> -
> - if (!contains(start, end, block_start, block_end))
> - continue;
> -
> - buddy = __get_buddy(block);
> - if (!drm_buddy_block_is_free(buddy))
> - continue;
> -
> - WARN_ON(drm_buddy_block_is_clear(block) ==
> - drm_buddy_block_is_clear(buddy));
> -
> - /*
> - * Advance to the next node when the current node is the buddy,
> - * as freeing the block will also remove its buddy from the tree.
> - */
> - if (iter == &buddy->rb)
> - iter = rb_prev(iter);
> -
> - rbtree_remove(mm, block);
> - if (drm_buddy_block_is_clear(block))
> - mm->clear_avail -= drm_buddy_block_size(mm, block);
> -
> - order = __drm_buddy_free(mm, block, true);
> - if (order >= min_order)
> - return 0;
> - }
> - }
> - }
> -
> - return -ENOMEM;
> -}
> -
> -/**
> - * drm_buddy_init - init memory manager
> - *
> - * @mm: DRM buddy manager to initialize
> - * @size: size in bytes to manage
> - * @chunk_size: minimum page size in bytes for our allocations
> - *
> - * Initializes the memory manager and its resources.
> - *
> - * Returns:
> - * 0 on success, error code on failure.
> - */
> -int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size)
> -{
> - unsigned int i, j, root_count = 0;
> - u64 offset = 0;
> -
> - if (size < chunk_size)
> - return -EINVAL;
> -
> - if (chunk_size < SZ_4K)
> - return -EINVAL;
> -
> - if (!is_power_of_2(chunk_size))
> - return -EINVAL;
> -
> - size = round_down(size, chunk_size);
> -
> - mm->size = size;
> - mm->avail = size;
> - mm->clear_avail = 0;
> - mm->chunk_size = chunk_size;
> - mm->max_order = ilog2(size) - ilog2(chunk_size);
> -
> - BUG_ON(mm->max_order > DRM_BUDDY_MAX_ORDER);
> -
> - mm->free_trees = kmalloc_array(DRM_BUDDY_MAX_FREE_TREES,
> - sizeof(*mm->free_trees),
> - GFP_KERNEL);
> - if (!mm->free_trees)
> - return -ENOMEM;
> -
> - for_each_free_tree(i) {
> - mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
> - sizeof(struct rb_root),
> - GFP_KERNEL);
> - if (!mm->free_trees[i])
> - goto out_free_tree;
> -
> - for (j = 0; j <= mm->max_order; ++j)
> - mm->free_trees[i][j] = RB_ROOT;
> - }
> -
> - mm->n_roots = hweight64(size);
> -
> - mm->roots = kmalloc_array(mm->n_roots,
> - sizeof(struct drm_buddy_block *),
> - GFP_KERNEL);
> - if (!mm->roots)
> - goto out_free_tree;
> -
> - /*
> - * Split into power-of-two blocks, in case we are given a size that is
> - * not itself a power-of-two.
> - */
> - do {
> - struct drm_buddy_block *root;
> - unsigned int order;
> - u64 root_size;
> -
> - order = ilog2(size) - ilog2(chunk_size);
> - root_size = chunk_size << order;
> -
> - root = drm_block_alloc(mm, NULL, order, offset);
> - if (!root)
> - goto out_free_roots;
> -
> - mark_free(mm, root);
> -
> - BUG_ON(root_count > mm->max_order);
> - BUG_ON(drm_buddy_block_size(mm, root) < chunk_size);
> -
> - mm->roots[root_count] = root;
> -
> - offset += root_size;
> - size -= root_size;
> - root_count++;
> - } while (size);
> -
> - return 0;
> -
> -out_free_roots:
> - while (root_count--)
> - drm_block_free(mm, mm->roots[root_count]);
> - kfree(mm->roots);
> -out_free_tree:
> - while (i--)
> - kfree(mm->free_trees[i]);
> - kfree(mm->free_trees);
> - return -ENOMEM;
> -}
> -EXPORT_SYMBOL(drm_buddy_init);
> -
> -/**
> - * drm_buddy_fini - tear down the memory manager
> - *
> - * @mm: DRM buddy manager to free
> - *
> - * Cleanup memory manager resources and the freetree
> - */
> -void drm_buddy_fini(struct drm_buddy *mm)
> -{
> - u64 root_size, size, start;
> - unsigned int order;
> - int i;
> -
> - size = mm->size;
> -
> - for (i = 0; i < mm->n_roots; ++i) {
> - order = ilog2(size) - ilog2(mm->chunk_size);
> - start = drm_buddy_block_offset(mm->roots[i]);
> - __force_merge(mm, start, start + size, order);
> -
> - if (WARN_ON(!drm_buddy_block_is_free(mm->roots[i])))
> - kunit_fail_current_test("buddy_fini() root");
> -
> - drm_block_free(mm, mm->roots[i]);
> -
> - root_size = mm->chunk_size << order;
> - size -= root_size;
> - }
> -
> - WARN_ON(mm->avail != mm->size);
> -
> - for_each_free_tree(i)
> - kfree(mm->free_trees[i]);
> - kfree(mm->roots);
> -}
> -EXPORT_SYMBOL(drm_buddy_fini);
> -
> -static int split_block(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - unsigned int block_order = drm_buddy_block_order(block) - 1;
> - u64 offset = drm_buddy_block_offset(block);
> -
> - BUG_ON(!drm_buddy_block_is_free(block));
> - BUG_ON(!drm_buddy_block_order(block));
> -
> - block->left = drm_block_alloc(mm, block, block_order, offset);
> - if (!block->left)
> - return -ENOMEM;
> -
> - block->right = drm_block_alloc(mm, block, block_order,
> - offset + (mm->chunk_size << block_order));
> - if (!block->right) {
> - drm_block_free(mm, block->left);
> - return -ENOMEM;
> - }
> -
> - mark_split(mm, block);
> -
> - if (drm_buddy_block_is_clear(block)) {
> - mark_cleared(block->left);
> - mark_cleared(block->right);
> - clear_reset(block);
> - }
> -
> - mark_free(mm, block->left);
> - mark_free(mm, block->right);
> -
> - return 0;
> -}
> -
> -/**
> - * drm_get_buddy - get buddy address
> - *
> - * @block: DRM buddy block
> - *
> - * Returns the corresponding buddy block for @block, or NULL
> - * if this is a root block and can't be merged further.
> - * Requires some kind of locking to protect against
> - * any concurrent allocate and free operations.
> - */
> -struct drm_buddy_block *
> -drm_get_buddy(struct drm_buddy_block *block)
> -{
> - return __get_buddy(block);
> -}
> -EXPORT_SYMBOL(drm_get_buddy);
> -
> -/**
> - * drm_buddy_reset_clear - reset blocks clear state
> - *
> - * @mm: DRM buddy manager
> - * @is_clear: blocks clear state
> - *
> - * Reset the clear state based on @is_clear value for each block
> - * in the freetree.
> - */
> -void drm_buddy_reset_clear(struct drm_buddy *mm, bool is_clear)
> -{
> - enum drm_buddy_free_tree src_tree, dst_tree;
> - u64 root_size, size, start;
> - unsigned int order;
> - int i;
> -
> - size = mm->size;
> - for (i = 0; i < mm->n_roots; ++i) {
> - order = ilog2(size) - ilog2(mm->chunk_size);
> - start = drm_buddy_block_offset(mm->roots[i]);
> - __force_merge(mm, start, start + size, order);
> -
> - root_size = mm->chunk_size << order;
> - size -= root_size;
> - }
> -
> - src_tree = is_clear ? DRM_BUDDY_DIRTY_TREE : DRM_BUDDY_CLEAR_TREE;
> - dst_tree = is_clear ? DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE;
> -
> - for (i = 0; i <= mm->max_order; ++i) {
> - struct rb_root *root = &mm->free_trees[src_tree][i];
> - struct drm_buddy_block *block, *tmp;
> -
> - rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
> - rbtree_remove(mm, block);
> - if (is_clear) {
> - mark_cleared(block);
> - mm->clear_avail += drm_buddy_block_size(mm, block);
> - } else {
> - clear_reset(block);
> - mm->clear_avail -= drm_buddy_block_size(mm, block);
> - }
> -
> - rbtree_insert(mm, block, dst_tree);
> - }
> - }
> -}
> -EXPORT_SYMBOL(drm_buddy_reset_clear);
> -
> -/**
> - * drm_buddy_free_block - free a block
> - *
> - * @mm: DRM buddy manager
> - * @block: block to be freed
> - */
> -void drm_buddy_free_block(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - BUG_ON(!drm_buddy_block_is_allocated(block));
> - mm->avail += drm_buddy_block_size(mm, block);
> - if (drm_buddy_block_is_clear(block))
> - mm->clear_avail += drm_buddy_block_size(mm, block);
> -
> - __drm_buddy_free(mm, block, false);
> -}
> -EXPORT_SYMBOL(drm_buddy_free_block);
> -
> -static void __drm_buddy_free_list(struct drm_buddy *mm,
> - struct list_head *objects,
> - bool mark_clear,
> - bool mark_dirty)
> -{
> - struct drm_buddy_block *block, *on;
> -
> - WARN_ON(mark_dirty && mark_clear);
> -
> - list_for_each_entry_safe(block, on, objects, link) {
> - if (mark_clear)
> - mark_cleared(block);
> - else if (mark_dirty)
> - clear_reset(block);
> - drm_buddy_free_block(mm, block);
> - cond_resched();
> - }
> - INIT_LIST_HEAD(objects);
> -}
> -
> -static void drm_buddy_free_list_internal(struct drm_buddy *mm,
> - struct list_head *objects)
> -{
> - /*
> - * Don't touch the clear/dirty bit, since allocation is still internal
> - * at this point. For example we might have just failed part of the
> - * allocation.
> - */
> - __drm_buddy_free_list(mm, objects, false, false);
> -}
> -
> -/**
> - * drm_buddy_free_list - free blocks
> - *
> - * @mm: DRM buddy manager
> - * @objects: input list head to free blocks
> - * @flags: optional flags like DRM_BUDDY_CLEARED
> - */
> -void drm_buddy_free_list(struct drm_buddy *mm,
> - struct list_head *objects,
> - unsigned int flags)
> -{
> - bool mark_clear = flags & DRM_BUDDY_CLEARED;
> -
> - __drm_buddy_free_list(mm, objects, mark_clear, !mark_clear);
> -}
> -EXPORT_SYMBOL(drm_buddy_free_list);
> -
> -static bool block_incompatible(struct drm_buddy_block *block, unsigned int flags)
> -{
> - bool needs_clear = flags & DRM_BUDDY_CLEAR_ALLOCATION;
> -
> - return needs_clear != drm_buddy_block_is_clear(block);
> -}
> -
> -static struct drm_buddy_block *
> -__alloc_range_bias(struct drm_buddy *mm,
> - u64 start, u64 end,
> - unsigned int order,
> - unsigned long flags,
> - bool fallback)
> -{
> - u64 req_size = mm->chunk_size << order;
> - struct drm_buddy_block *block;
> - struct drm_buddy_block *buddy;
> - LIST_HEAD(dfs);
> - int err;
> - int i;
> -
> - end = end - 1;
> -
> - for (i = 0; i < mm->n_roots; ++i)
> - list_add_tail(&mm->roots[i]->tmp_link, &dfs);
> -
> - do {
> - u64 block_start;
> - u64 block_end;
> -
> - block = list_first_entry_or_null(&dfs,
> - struct drm_buddy_block,
> - tmp_link);
> - if (!block)
> - break;
> -
> - list_del(&block->tmp_link);
> -
> - if (drm_buddy_block_order(block) < order)
> - continue;
> -
> - block_start = drm_buddy_block_offset(block);
> - block_end = block_start + drm_buddy_block_size(mm, block) - 1;
> -
> - if (!overlaps(start, end, block_start, block_end))
> - continue;
> -
> - if (drm_buddy_block_is_allocated(block))
> - continue;
> -
> - if (block_start < start || block_end > end) {
> - u64 adjusted_start = max(block_start, start);
> - u64 adjusted_end = min(block_end, end);
> -
> - if (round_down(adjusted_end + 1, req_size) <=
> - round_up(adjusted_start, req_size))
> - continue;
> - }
> -
> - if (!fallback && block_incompatible(block, flags))
> - continue;
> -
> - if (contains(start, end, block_start, block_end) &&
> - order == drm_buddy_block_order(block)) {
> - /*
> - * Find the free block within the range.
> - */
> - if (drm_buddy_block_is_free(block))
> - return block;
> -
> - continue;
> - }
> -
> - if (!drm_buddy_block_is_split(block)) {
> - err = split_block(mm, block);
> - if (unlikely(err))
> - goto err_undo;
> - }
> -
> - list_add(&block->right->tmp_link, &dfs);
> - list_add(&block->left->tmp_link, &dfs);
> - } while (1);
> -
> - return ERR_PTR(-ENOSPC);
> -
> -err_undo:
> - /*
> - * We really don't want to leave around a bunch of split blocks, since
> - * bigger is better, so make sure we merge everything back before we
> - * free the allocated blocks.
> - */
> - buddy = __get_buddy(block);
> - if (buddy &&
> - (drm_buddy_block_is_free(block) &&
> - drm_buddy_block_is_free(buddy)))
> - __drm_buddy_free(mm, block, false);
> - return ERR_PTR(err);
> -}
> -
> -static struct drm_buddy_block *
> -__drm_buddy_alloc_range_bias(struct drm_buddy *mm,
> - u64 start, u64 end,
> - unsigned int order,
> - unsigned long flags)
> -{
> - struct drm_buddy_block *block;
> - bool fallback = false;
> -
> - block = __alloc_range_bias(mm, start, end, order,
> - flags, fallback);
> - if (IS_ERR(block))
> - return __alloc_range_bias(mm, start, end, order,
> - flags, !fallback);
> -
> - return block;
> -}
> -
> -static struct drm_buddy_block *
> -get_maxblock(struct drm_buddy *mm,
> - unsigned int order,
> - enum drm_buddy_free_tree tree)
> -{
> - struct drm_buddy_block *max_block = NULL, *block = NULL;
> - struct rb_root *root;
> - unsigned int i;
> -
> - for (i = order; i <= mm->max_order; ++i) {
> - root = &mm->free_trees[tree][i];
> - block = rbtree_last_free_block(root);
> - if (!block)
> - continue;
> -
> - if (!max_block) {
> - max_block = block;
> - continue;
> - }
> -
> - if (drm_buddy_block_offset(block) >
> - drm_buddy_block_offset(max_block)) {
> - max_block = block;
> - }
> - }
> -
> - return max_block;
> -}
> -
> -static struct drm_buddy_block *
> -alloc_from_freetree(struct drm_buddy *mm,
> - unsigned int order,
> - unsigned long flags)
> -{
> - struct drm_buddy_block *block = NULL;
> - struct rb_root *root;
> - enum drm_buddy_free_tree tree;
> - unsigned int tmp;
> - int err;
> -
> - tree = (flags & DRM_BUDDY_CLEAR_ALLOCATION) ?
> - DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE;
> -
> - if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) {
> - block = get_maxblock(mm, order, tree);
> - if (block)
> - /* Store the obtained block order */
> - tmp = drm_buddy_block_order(block);
> - } else {
> - for (tmp = order; tmp <= mm->max_order; ++tmp) {
> - /* Get RB tree root for this order and tree */
> - root = &mm->free_trees[tree][tmp];
> - block = rbtree_last_free_block(root);
> - if (block)
> - break;
> - }
> - }
> -
> - if (!block) {
> - /* Try allocating from the other tree */
> - tree = (tree == DRM_BUDDY_CLEAR_TREE) ?
> - DRM_BUDDY_DIRTY_TREE : DRM_BUDDY_CLEAR_TREE;
> -
> - for (tmp = order; tmp <= mm->max_order; ++tmp) {
> - root = &mm->free_trees[tree][tmp];
> - block = rbtree_last_free_block(root);
> - if (block)
> - break;
> - }
> -
> - if (!block)
> - return ERR_PTR(-ENOSPC);
> - }
> -
> - BUG_ON(!drm_buddy_block_is_free(block));
> -
> - while (tmp != order) {
> - err = split_block(mm, block);
> - if (unlikely(err))
> - goto err_undo;
> -
> - block = block->right;
> - tmp--;
> - }
> - return block;
> -
> -err_undo:
> - if (tmp != order)
> - __drm_buddy_free(mm, block, false);
> - return ERR_PTR(err);
> -}
> -
> -static int __alloc_range(struct drm_buddy *mm,
> - struct list_head *dfs,
> - u64 start, u64 size,
> - struct list_head *blocks,
> - u64 *total_allocated_on_err)
> -{
> - struct drm_buddy_block *block;
> - struct drm_buddy_block *buddy;
> - u64 total_allocated = 0;
> - LIST_HEAD(allocated);
> - u64 end;
> - int err;
> -
> - end = start + size - 1;
> -
> - do {
> - u64 block_start;
> - u64 block_end;
> -
> - block = list_first_entry_or_null(dfs,
> - struct drm_buddy_block,
> - tmp_link);
> - if (!block)
> - break;
> -
> - list_del(&block->tmp_link);
> -
> - block_start = drm_buddy_block_offset(block);
> - block_end = block_start + drm_buddy_block_size(mm, block) - 1;
> -
> - if (!overlaps(start, end, block_start, block_end))
> - continue;
> -
> - if (drm_buddy_block_is_allocated(block)) {
> - err = -ENOSPC;
> - goto err_free;
> - }
> -
> - if (contains(start, end, block_start, block_end)) {
> - if (drm_buddy_block_is_free(block)) {
> - mark_allocated(mm, block);
> - total_allocated += drm_buddy_block_size(mm, block);
> - mm->avail -= drm_buddy_block_size(mm, block);
> - if (drm_buddy_block_is_clear(block))
> - mm->clear_avail -= drm_buddy_block_size(mm, block);
> - list_add_tail(&block->link, &allocated);
> - continue;
> - } else if (!mm->clear_avail) {
> - err = -ENOSPC;
> - goto err_free;
> - }
> - }
> -
> - if (!drm_buddy_block_is_split(block)) {
> - err = split_block(mm, block);
> - if (unlikely(err))
> - goto err_undo;
> - }
> -
> - list_add(&block->right->tmp_link, dfs);
> - list_add(&block->left->tmp_link, dfs);
> - } while (1);
> -
> - if (total_allocated < size) {
> - err = -ENOSPC;
> - goto err_free;
> - }
> -
> - list_splice_tail(&allocated, blocks);
> -
> - return 0;
> -
> -err_undo:
> - /*
> - * We really don't want to leave around a bunch of split blocks, since
> - * bigger is better, so make sure we merge everything back before we
> - * free the allocated blocks.
> - */
> - buddy = __get_buddy(block);
> - if (buddy &&
> - (drm_buddy_block_is_free(block) &&
> - drm_buddy_block_is_free(buddy)))
> - __drm_buddy_free(mm, block, false);
> -
> -err_free:
> - if (err == -ENOSPC && total_allocated_on_err) {
> - list_splice_tail(&allocated, blocks);
> - *total_allocated_on_err = total_allocated;
> - } else {
> - drm_buddy_free_list_internal(mm, &allocated);
> - }
> -
> - return err;
> -}
> -
> -static int __drm_buddy_alloc_range(struct drm_buddy *mm,
> - u64 start,
> - u64 size,
> - u64 *total_allocated_on_err,
> - struct list_head *blocks)
> -{
> - LIST_HEAD(dfs);
> - int i;
> -
> - for (i = 0; i < mm->n_roots; ++i)
> - list_add_tail(&mm->roots[i]->tmp_link, &dfs);
> -
> - return __alloc_range(mm, &dfs, start, size,
> - blocks, total_allocated_on_err);
> -}
> -
> -static int __alloc_contig_try_harder(struct drm_buddy *mm,
> - u64 size,
> - u64 min_block_size,
> - struct list_head *blocks)
> -{
> - u64 rhs_offset, lhs_offset, lhs_size, filled;
> - struct drm_buddy_block *block;
> - unsigned int tree, order;
> - LIST_HEAD(blocks_lhs);
> - unsigned long pages;
> - u64 modify_size;
> - int err;
> -
> - modify_size = rounddown_pow_of_two(size);
> - pages = modify_size >> ilog2(mm->chunk_size);
> - order = fls(pages) - 1;
> - if (order == 0)
> - return -ENOSPC;
> -
> - for_each_free_tree(tree) {
> - struct rb_root *root;
> - struct rb_node *iter;
> -
> - root = &mm->free_trees[tree][order];
> - if (rbtree_is_empty(root))
> - continue;
> -
> - iter = rb_last(root);
> - while (iter) {
> - block = rbtree_get_free_block(iter);
> -
> - /* Allocate blocks traversing RHS */
> - rhs_offset = drm_buddy_block_offset(block);
> - err = __drm_buddy_alloc_range(mm, rhs_offset, size,
> - &filled, blocks);
> - if (!err || err != -ENOSPC)
> - return err;
> -
> - lhs_size = max((size - filled), min_block_size);
> - if (!IS_ALIGNED(lhs_size, min_block_size))
> - lhs_size = round_up(lhs_size, min_block_size);
> -
> - /* Allocate blocks traversing LHS */
> - lhs_offset = drm_buddy_block_offset(block) - lhs_size;
> - err = __drm_buddy_alloc_range(mm, lhs_offset, lhs_size,
> - NULL, &blocks_lhs);
> - if (!err) {
> - list_splice(&blocks_lhs, blocks);
> - return 0;
> - } else if (err != -ENOSPC) {
> - drm_buddy_free_list_internal(mm, blocks);
> - return err;
> - }
> - /* Free blocks for the next iteration */
> - drm_buddy_free_list_internal(mm, blocks);
> -
> - iter = rb_prev(iter);
> - }
> - }
> -
> - return -ENOSPC;
> -}
> -
> -/**
> - * drm_buddy_block_trim - free unused pages
> - *
> - * @mm: DRM buddy manager
> - * @start: start address to begin the trimming.
> - * @new_size: original size requested
> - * @blocks: Input and output list of allocated blocks.
> - * MUST contain single block as input to be trimmed.
> - * On success will contain the newly allocated blocks
> - * making up the @new_size. Blocks always appear in
> - * ascending order
> - *
> - * For contiguous allocation, we round up the size to the nearest
> - * power of two value, drivers consume *actual* size, so remaining
> - * portions are unused and can be optionally freed with this function
> - *
> - * Returns:
> - * 0 on success, error code on failure.
> - */
> -int drm_buddy_block_trim(struct drm_buddy *mm,
> - u64 *start,
> - u64 new_size,
> - struct list_head *blocks)
> -{
> - struct drm_buddy_block *parent;
> - struct drm_buddy_block *block;
> - u64 block_start, block_end;
> - LIST_HEAD(dfs);
> - u64 new_start;
> - int err;
> -
> - if (!list_is_singular(blocks))
> - return -EINVAL;
> -
> - block = list_first_entry(blocks,
> - struct drm_buddy_block,
> - link);
> -
> - block_start = drm_buddy_block_offset(block);
> - block_end = block_start + drm_buddy_block_size(mm, block);
> -
> - if (WARN_ON(!drm_buddy_block_is_allocated(block)))
> - return -EINVAL;
> -
> - if (new_size > drm_buddy_block_size(mm, block))
> - return -EINVAL;
> -
> - if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size))
> - return -EINVAL;
> -
> - if (new_size == drm_buddy_block_size(mm, block))
> - return 0;
> -
> - new_start = block_start;
> - if (start) {
> - new_start = *start;
> -
> - if (new_start < block_start)
> - return -EINVAL;
> -
> - if (!IS_ALIGNED(new_start, mm->chunk_size))
> - return -EINVAL;
> -
> - if (range_overflows(new_start, new_size, block_end))
> - return -EINVAL;
> - }
> -
> - list_del(&block->link);
> - mark_free(mm, block);
> - mm->avail += drm_buddy_block_size(mm, block);
> - if (drm_buddy_block_is_clear(block))
> - mm->clear_avail += drm_buddy_block_size(mm, block);
> -
> - /* Prevent recursively freeing this node */
> - parent = block->parent;
> - block->parent = NULL;
> -
> - list_add(&block->tmp_link, &dfs);
> - err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL);
> - if (err) {
> - mark_allocated(mm, block);
> - mm->avail -= drm_buddy_block_size(mm, block);
> - if (drm_buddy_block_is_clear(block))
> - mm->clear_avail -= drm_buddy_block_size(mm, block);
> - list_add(&block->link, blocks);
> - }
> -
> - block->parent = parent;
> - return err;
> -}
> -EXPORT_SYMBOL(drm_buddy_block_trim);
> -
> -static struct drm_buddy_block *
> -__drm_buddy_alloc_blocks(struct drm_buddy *mm,
> - u64 start, u64 end,
> - unsigned int order,
> - unsigned long flags)
> -{
> - if (flags & DRM_BUDDY_RANGE_ALLOCATION)
> - /* Allocate traversing within the range */
> - return __drm_buddy_alloc_range_bias(mm, start, end,
> - order, flags);
> - else
> - /* Allocate from freetree */
> - return alloc_from_freetree(mm, order, flags);
> -}
> -
> -/**
> - * drm_buddy_alloc_blocks - allocate power-of-two blocks
> - *
> - * @mm: DRM buddy manager to allocate from
> - * @start: start of the allowed range for this block
> - * @end: end of the allowed range for this block
> - * @size: size of the allocation in bytes
> - * @min_block_size: alignment of the allocation
> - * @blocks: output list head to add allocated blocks
> - * @flags: DRM_BUDDY_*_ALLOCATION flags
> - *
> - * alloc_range_bias() called on range limitations, which traverses
> - * the tree and returns the desired block.
> - *
> - * alloc_from_freetree() called when *no* range restrictions
> - * are enforced, which picks the block from the freetree.
> - *
> - * Returns:
> - * 0 on success, error code on failure.
> - */
> -int drm_buddy_alloc_blocks(struct drm_buddy *mm,
> - u64 start, u64 end, u64 size,
> - u64 min_block_size,
> - struct list_head *blocks,
> - unsigned long flags)
> -{
> - struct drm_buddy_block *block = NULL;
> - u64 original_size, original_min_size;
> - unsigned int min_order, order;
> - LIST_HEAD(allocated);
> - unsigned long pages;
> - int err;
> -
> - if (size < mm->chunk_size)
> - return -EINVAL;
> -
> - if (min_block_size < mm->chunk_size)
> - return -EINVAL;
> -
> - if (!is_power_of_2(min_block_size))
> - return -EINVAL;
> -
> - if (!IS_ALIGNED(start | end | size, mm->chunk_size))
> - return -EINVAL;
> -
> - if (end > mm->size)
> - return -EINVAL;
> -
> - if (range_overflows(start, size, mm->size))
> - return -EINVAL;
> -
> - /* Actual range allocation */
> - if (start + size == end) {
> - if (!IS_ALIGNED(start | end, min_block_size))
> - return -EINVAL;
> -
> - return __drm_buddy_alloc_range(mm, start, size, NULL, blocks);
> - }
> -
> - original_size = size;
> - original_min_size = min_block_size;
> -
> - /* Roundup the size to power of 2 */
> - if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION) {
> - size = roundup_pow_of_two(size);
> - min_block_size = size;
> - /* Align size value to min_block_size */
> - } else if (!IS_ALIGNED(size, min_block_size)) {
> - size = round_up(size, min_block_size);
> - }
> -
> - pages = size >> ilog2(mm->chunk_size);
> - order = fls(pages) - 1;
> - min_order = ilog2(min_block_size) - ilog2(mm->chunk_size);
> -
> - do {
> - order = min(order, (unsigned int)fls(pages) - 1);
> - BUG_ON(order > mm->max_order);
> - BUG_ON(order < min_order);
> -
> - do {
> - block = __drm_buddy_alloc_blocks(mm, start,
> - end,
> - order,
> - flags);
> - if (!IS_ERR(block))
> - break;
> -
> - if (order-- == min_order) {
> - /* Try allocation through force merge method */
> - if (mm->clear_avail &&
> - !__force_merge(mm, start, end, min_order)) {
> - block = __drm_buddy_alloc_blocks(mm, start,
> - end,
> - min_order,
> - flags);
> - if (!IS_ERR(block)) {
> - order = min_order;
> - break;
> - }
> - }
> -
> - /*
> - * Try contiguous block allocation through
> - * try harder method.
> - */
> - if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION &&
> - !(flags & DRM_BUDDY_RANGE_ALLOCATION))
> - return __alloc_contig_try_harder(mm,
> - original_size,
> - original_min_size,
> - blocks);
> - err = -ENOSPC;
> - goto err_free;
> - }
> - } while (1);
> -
> - mark_allocated(mm, block);
> - mm->avail -= drm_buddy_block_size(mm, block);
> - if (drm_buddy_block_is_clear(block))
> - mm->clear_avail -= drm_buddy_block_size(mm, block);
> - kmemleak_update_trace(block);
> - list_add_tail(&block->link, &allocated);
> -
> - pages -= BIT(order);
> -
> - if (!pages)
> - break;
> - } while (1);
> -
> - /* Trim the allocated block to the required size */
> - if (!(flags & DRM_BUDDY_TRIM_DISABLE) &&
> - original_size != size) {
> - struct list_head *trim_list;
> - LIST_HEAD(temp);
> - u64 trim_size;
> -
> - trim_list = &allocated;
> - trim_size = original_size;
> -
> - if (!list_is_singular(&allocated)) {
> - block = list_last_entry(&allocated, typeof(*block), link);
> - list_move(&block->link, &temp);
> - trim_list = &temp;
> - trim_size = drm_buddy_block_size(mm, block) -
> - (size - original_size);
> - }
> -
> - drm_buddy_block_trim(mm,
> - NULL,
> - trim_size,
> - trim_list);
> -
> - if (!list_empty(&temp))
> - list_splice_tail(trim_list, &allocated);
> - }
> -
> - list_splice_tail(&allocated, blocks);
> - return 0;
> -
> -err_free:
> - drm_buddy_free_list_internal(mm, &allocated);
> - return err;
> -}
> -EXPORT_SYMBOL(drm_buddy_alloc_blocks);
> -
> /**
> * drm_buddy_block_print - print block information
> *
> - * @mm: DRM buddy manager
> - * @block: DRM buddy block
> + * @mm: GPU buddy manager
> + * @block: GPU buddy block
> * @p: DRM printer to use
> */
> -void drm_buddy_block_print(struct drm_buddy *mm,
> - struct drm_buddy_block *block,
> +void drm_buddy_block_print(struct gpu_buddy *mm, struct gpu_buddy_block *block,
> struct drm_printer *p)
> {
> - u64 start = drm_buddy_block_offset(block);
> - u64 size = drm_buddy_block_size(mm, block);
> + u64 start = gpu_buddy_block_offset(block);
> + u64 size = gpu_buddy_block_size(mm, block);
>
> drm_printf(p, "%#018llx-%#018llx: %llu\n", start, start + size, size);
> }
> @@ -1267,18 +30,21 @@ EXPORT_SYMBOL(drm_buddy_block_print);
> /**
> * drm_buddy_print - print allocator state
> *
> - * @mm: DRM buddy manager
> + * @mm: GPU buddy manager
> * @p: DRM printer to use
> */
> -void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p)
> +void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
> {
> int order;
>
> - drm_printf(p, "chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n",
> - mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
> + drm_printf(
> + p,
> + "chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n",
> + mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20,
> + mm->clear_avail >> 20);
>
> for (order = mm->max_order; order >= 0; order--) {
> - struct drm_buddy_block *block, *tmp;
> + struct gpu_buddy_block *block, *tmp;
> struct rb_root *root;
> u64 count = 0, free;
> unsigned int tree;
> @@ -1286,8 +52,9 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p)
> for_each_free_tree(tree) {
> root = &mm->free_trees[tree][order];
>
> - rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
> - BUG_ON(!drm_buddy_block_is_free(block));
> + rbtree_postorder_for_each_entry_safe(block, tmp, root,
> + rb) {
> + BUG_ON(!gpu_buddy_block_is_free(block));
> count++;
> }
> }
> @@ -1305,22 +72,5 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p)
> }
> EXPORT_SYMBOL(drm_buddy_print);
>
> -static void drm_buddy_module_exit(void)
> -{
> - kmem_cache_destroy(slab_blocks);
> -}
> -
> -static int __init drm_buddy_module_init(void)
> -{
> - slab_blocks = KMEM_CACHE(drm_buddy_block, 0);
> - if (!slab_blocks)
> - return -ENOMEM;
> -
> - return 0;
> -}
> -
> -module_init(drm_buddy_module_init);
> -module_exit(drm_buddy_module_exit);
> -
> -MODULE_DESCRIPTION("DRM Buddy Allocator");
> +MODULE_DESCRIPTION("DRM-specific GPU Buddy Allocator Print Helpers");
> MODULE_LICENSE("Dual MIT/GPL");
> diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
> index 5e939004b646..859aeca87c19 100644
> --- a/drivers/gpu/drm/i915/Kconfig
> +++ b/drivers/gpu/drm/i915/Kconfig
> @@ -38,6 +38,7 @@ config DRM_I915
> select CEC_CORE if CEC_NOTIFIER
> select VMAP_PFN
> select DRM_TTM
> + select GPU_BUDDY
> select DRM_BUDDY
> select AUXILIARY_BUS
> help
> diff --git a/drivers/gpu/drm/i915/i915_scatterlist.c b/drivers/gpu/drm/i915/i915_scatterlist.c
> index 4d830740946d..6a34dae13769 100644
> --- a/drivers/gpu/drm/i915/i915_scatterlist.c
> +++ b/drivers/gpu/drm/i915/i915_scatterlist.c
> @@ -7,7 +7,7 @@
> #include "i915_scatterlist.h"
> #include "i915_ttm_buddy_manager.h"
>
> -#include <drm/drm_buddy.h>
> +#include <linux/gpu_buddy.h>
> #include <drm/drm_mm.h>
>
> #include <linux/slab.h>
> @@ -167,9 +167,9 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
> struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
> const u64 size = res->size;
> const u32 max_segment = round_down(UINT_MAX, page_alignment);
> - struct drm_buddy *mm = bman_res->mm;
> + struct gpu_buddy *mm = bman_res->mm;
> struct list_head *blocks = &bman_res->blocks;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> struct i915_refct_sgt *rsgt;
> struct scatterlist *sg;
> struct sg_table *st;
> @@ -202,8 +202,8 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
> list_for_each_entry(block, blocks, link) {
> u64 block_size, offset;
>
> - block_size = min_t(u64, size, drm_buddy_block_size(mm, block));
> - offset = drm_buddy_block_offset(block);
> + block_size = min_t(u64, size, gpu_buddy_block_size(mm, block));
> + offset = gpu_buddy_block_offset(block);
>
> while (block_size) {
> u64 len;
> diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
> index d5c6e6605086..f43d7f2771ad 100644
> --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
> +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
> @@ -4,6 +4,7 @@
> */
>
> #include <linux/slab.h>
> +#include <linux/gpu_buddy.h>
>
> #include <drm/drm_buddy.h>
> #include <drm/drm_print.h>
> @@ -16,7 +17,7 @@
>
> struct i915_ttm_buddy_manager {
> struct ttm_resource_manager manager;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> struct list_head reserved;
> struct mutex lock;
> unsigned long visible_size;
> @@ -38,7 +39,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
> {
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
> struct i915_ttm_buddy_resource *bman_res;
> - struct drm_buddy *mm = &bman->mm;
> + struct gpu_buddy *mm = &bman->mm;
> unsigned long n_pages, lpfn;
> u64 min_page_size;
> u64 size;
> @@ -57,13 +58,13 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
> bman_res->mm = mm;
>
> if (place->flags & TTM_PL_FLAG_TOPDOWN)
> - bman_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
> + bman_res->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
>
> if (place->flags & TTM_PL_FLAG_CONTIGUOUS)
> - bman_res->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
> + bman_res->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
>
> if (place->fpfn || lpfn != man->size)
> - bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION;
> + bman_res->flags |= GPU_BUDDY_RANGE_ALLOCATION;
>
> GEM_BUG_ON(!bman_res->base.size);
> size = bman_res->base.size;
> @@ -89,7 +90,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
> goto err_free_res;
> }
>
> - err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
> + err = gpu_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
> (u64)lpfn << PAGE_SHIFT,
> (u64)n_pages << PAGE_SHIFT,
> min_page_size,
> @@ -101,15 +102,15 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
> if (lpfn <= bman->visible_size) {
> bman_res->used_visible_size = PFN_UP(bman_res->base.size);
> } else {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
>
> list_for_each_entry(block, &bman_res->blocks, link) {
> unsigned long start =
> - drm_buddy_block_offset(block) >> PAGE_SHIFT;
> + gpu_buddy_block_offset(block) >> PAGE_SHIFT;
>
> if (start < bman->visible_size) {
> unsigned long end = start +
> - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
> + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
>
> bman_res->used_visible_size +=
> min(end, bman->visible_size) - start;
> @@ -126,7 +127,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
> return 0;
>
> err_free_blocks:
> - drm_buddy_free_list(mm, &bman_res->blocks, 0);
> + gpu_buddy_free_list(mm, &bman_res->blocks, 0);
> mutex_unlock(&bman->lock);
> err_free_res:
> ttm_resource_fini(man, &bman_res->base);
> @@ -141,7 +142,7 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man,
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
>
> mutex_lock(&bman->lock);
> - drm_buddy_free_list(&bman->mm, &bman_res->blocks, 0);
> + gpu_buddy_free_list(&bman->mm, &bman_res->blocks, 0);
> bman->visible_avail += bman_res->used_visible_size;
> mutex_unlock(&bman->lock);
>
> @@ -156,8 +157,8 @@ static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man,
> {
> struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
> - struct drm_buddy *mm = &bman->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = &bman->mm;
> + struct gpu_buddy_block *block;
>
> if (!place->fpfn && !place->lpfn)
> return true;
> @@ -176,9 +177,9 @@ static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man,
> /* Check each drm buddy block individually */
> list_for_each_entry(block, &bman_res->blocks, link) {
> unsigned long fpfn =
> - drm_buddy_block_offset(block) >> PAGE_SHIFT;
> + gpu_buddy_block_offset(block) >> PAGE_SHIFT;
> unsigned long lpfn = fpfn +
> - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
> + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
>
> if (place->fpfn < lpfn && place->lpfn > fpfn)
> return true;
> @@ -194,8 +195,8 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man,
> {
> struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
> - struct drm_buddy *mm = &bman->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = &bman->mm;
> + struct gpu_buddy_block *block;
>
> if (!place->fpfn && !place->lpfn)
> return true;
> @@ -209,9 +210,9 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man,
> /* Check each drm buddy block individually */
> list_for_each_entry(block, &bman_res->blocks, link) {
> unsigned long fpfn =
> - drm_buddy_block_offset(block) >> PAGE_SHIFT;
> + gpu_buddy_block_offset(block) >> PAGE_SHIFT;
> unsigned long lpfn = fpfn +
> - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
> + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
>
> if (fpfn < place->fpfn || lpfn > place->lpfn)
> return false;
> @@ -224,7 +225,7 @@ static void i915_ttm_buddy_man_debug(struct ttm_resource_manager *man,
> struct drm_printer *printer)
> {
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
>
> mutex_lock(&bman->lock);
> drm_printf(printer, "default_page_size: %lluKiB\n",
> @@ -293,7 +294,7 @@ int i915_ttm_buddy_man_init(struct ttm_device *bdev,
> if (!bman)
> return -ENOMEM;
>
> - err = drm_buddy_init(&bman->mm, size, chunk_size);
> + err = gpu_buddy_init(&bman->mm, size, chunk_size);
> if (err)
> goto err_free_bman;
>
> @@ -333,7 +334,7 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type)
> {
> struct ttm_resource_manager *man = ttm_manager_type(bdev, type);
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
> - struct drm_buddy *mm = &bman->mm;
> + struct gpu_buddy *mm = &bman->mm;
> int ret;
>
> ttm_resource_manager_set_used(man, false);
> @@ -345,8 +346,8 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type)
> ttm_set_driver_manager(bdev, type, NULL);
>
> mutex_lock(&bman->lock);
> - drm_buddy_free_list(mm, &bman->reserved, 0);
> - drm_buddy_fini(mm);
> + gpu_buddy_free_list(mm, &bman->reserved, 0);
> + gpu_buddy_fini(mm);
> bman->visible_avail += bman->visible_reserved;
> WARN_ON_ONCE(bman->visible_avail != bman->visible_size);
> mutex_unlock(&bman->lock);
> @@ -371,15 +372,15 @@ int i915_ttm_buddy_man_reserve(struct ttm_resource_manager *man,
> u64 start, u64 size)
> {
> struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
> - struct drm_buddy *mm = &bman->mm;
> + struct gpu_buddy *mm = &bman->mm;
> unsigned long fpfn = start >> PAGE_SHIFT;
> unsigned long flags = 0;
> int ret;
>
> - flags |= DRM_BUDDY_RANGE_ALLOCATION;
> + flags |= GPU_BUDDY_RANGE_ALLOCATION;
>
> mutex_lock(&bman->lock);
> - ret = drm_buddy_alloc_blocks(mm, start,
> + ret = gpu_buddy_alloc_blocks(mm, start,
> start + size,
> size, mm->chunk_size,
> &bman->reserved,
> diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
> index d64620712830..4a92dcf09766 100644
> --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
> +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
> @@ -13,14 +13,14 @@
>
> struct ttm_device;
> struct ttm_resource_manager;
> -struct drm_buddy;
> +struct gpu_buddy;
>
> /**
> * struct i915_ttm_buddy_resource
> *
> * @base: struct ttm_resource base class we extend
> * @blocks: the list of struct i915_buddy_block for this resource/allocation
> - * @flags: DRM_BUDDY_*_ALLOCATION flags
> + * @flags: GPU_BUDDY_*_ALLOCATION flags
> * @used_visible_size: How much of this resource, if any, uses the CPU visible
> * portion, in pages.
> * @mm: the struct i915_buddy_mm for this resource
> @@ -33,7 +33,7 @@ struct i915_ttm_buddy_resource {
> struct list_head blocks;
> unsigned long flags;
> unsigned long used_visible_size;
> - struct drm_buddy *mm;
> + struct gpu_buddy *mm;
> };
>
> /**
> diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
> index 7b856b5090f9..8307390943a2 100644
> --- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c
> +++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
> @@ -6,7 +6,7 @@
> #include <linux/prime_numbers.h>
> #include <linux/sort.h>
>
> -#include <drm/drm_buddy.h>
> +#include <linux/gpu_buddy.h>
>
> #include "../i915_selftest.h"
>
> @@ -371,7 +371,7 @@ static int igt_mock_splintered_region(void *arg)
> struct drm_i915_private *i915 = mem->i915;
> struct i915_ttm_buddy_resource *res;
> struct drm_i915_gem_object *obj;
> - struct drm_buddy *mm;
> + struct gpu_buddy *mm;
> unsigned int expected_order;
> LIST_HEAD(objects);
> u64 size;
> @@ -447,8 +447,8 @@ static int igt_mock_max_segment(void *arg)
> struct drm_i915_private *i915 = mem->i915;
> struct i915_ttm_buddy_resource *res;
> struct drm_i915_gem_object *obj;
> - struct drm_buddy_block *block;
> - struct drm_buddy *mm;
> + struct gpu_buddy_block *block;
> + struct gpu_buddy *mm;
> struct list_head *blocks;
> struct scatterlist *sg;
> I915_RND_STATE(prng);
> @@ -487,8 +487,8 @@ static int igt_mock_max_segment(void *arg)
> mm = res->mm;
> size = 0;
> list_for_each_entry(block, blocks, link) {
> - if (drm_buddy_block_size(mm, block) > size)
> - size = drm_buddy_block_size(mm, block);
> + if (gpu_buddy_block_size(mm, block) > size)
> + size = gpu_buddy_block_size(mm, block);
> }
> if (size < max_segment) {
> pr_err("%s: Failed to create a huge contiguous block [> %u], largest block %lld\n",
> @@ -527,14 +527,14 @@ static u64 igt_object_mappable_total(struct drm_i915_gem_object *obj)
> struct intel_memory_region *mr = obj->mm.region;
> struct i915_ttm_buddy_resource *bman_res =
> to_ttm_buddy_resource(obj->mm.res);
> - struct drm_buddy *mm = bman_res->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = bman_res->mm;
> + struct gpu_buddy_block *block;
> u64 total;
>
> total = 0;
> list_for_each_entry(block, &bman_res->blocks, link) {
> - u64 start = drm_buddy_block_offset(block);
> - u64 end = start + drm_buddy_block_size(mm, block);
> + u64 start = gpu_buddy_block_offset(block);
> + u64 end = start + gpu_buddy_block_size(mm, block);
>
> if (start < resource_size(&mr->io))
> total += min_t(u64, end, resource_size(&mr->io)) - start;
> diff --git a/drivers/gpu/drm/tests/Makefile b/drivers/gpu/drm/tests/Makefile
> index 87d5d5f9332a..d2e2e3d8349a 100644
> --- a/drivers/gpu/drm/tests/Makefile
> +++ b/drivers/gpu/drm/tests/Makefile
> @@ -7,7 +7,6 @@ obj-$(CONFIG_DRM_KUNIT_TEST) += \
> drm_atomic_test.o \
> drm_atomic_state_test.o \
> drm_bridge_test.o \
> - drm_buddy_test.o \
> drm_cmdline_parser_test.o \
> drm_connector_test.o \
> drm_damage_helper_test.o \
> diff --git a/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c b/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c
> index 2eda87882e65..ffa12473077c 100644
> --- a/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c
> +++ b/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c
> @@ -3,6 +3,7 @@
> * Copyright © 2023 Intel Corporation
> */
> #include <linux/delay.h>
> +#include <linux/gpu_buddy.h>
> #include <linux/kthread.h>
>
> #include <drm/ttm/ttm_resource.h>
> @@ -251,7 +252,7 @@ static void ttm_bo_validate_basic(struct kunit *test)
> NULL, &dummy_ttm_bo_destroy);
> KUNIT_EXPECT_EQ(test, err, 0);
>
> - snd_place = ttm_place_kunit_init(test, snd_mem, DRM_BUDDY_TOPDOWN_ALLOCATION);
> + snd_place = ttm_place_kunit_init(test, snd_mem, GPU_BUDDY_TOPDOWN_ALLOCATION);
> snd_placement = ttm_placement_kunit_init(test, snd_place, 1);
>
> err = ttm_bo_validate(bo, snd_placement, &ctx_val);
> @@ -263,7 +264,7 @@ static void ttm_bo_validate_basic(struct kunit *test)
> KUNIT_EXPECT_TRUE(test, ttm_tt_is_populated(bo->ttm));
> KUNIT_EXPECT_EQ(test, bo->resource->mem_type, snd_mem);
> KUNIT_EXPECT_EQ(test, bo->resource->placement,
> - DRM_BUDDY_TOPDOWN_ALLOCATION);
> + GPU_BUDDY_TOPDOWN_ALLOCATION);
>
> ttm_bo_fini(bo);
> ttm_mock_manager_fini(priv->ttm_dev, snd_mem);
> diff --git a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c
> index dd395229e388..294d56d9067e 100644
> --- a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c
> +++ b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c
> @@ -31,7 +31,7 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man,
> {
> struct ttm_mock_manager *manager = to_mock_mgr(man);
> struct ttm_mock_resource *mock_res;
> - struct drm_buddy *mm = &manager->mm;
> + struct gpu_buddy *mm = &manager->mm;
> u64 lpfn, fpfn, alloc_size;
> int err;
>
> @@ -47,14 +47,14 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man,
> INIT_LIST_HEAD(&mock_res->blocks);
>
> if (place->flags & TTM_PL_FLAG_TOPDOWN)
> - mock_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
> + mock_res->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
>
> if (place->flags & TTM_PL_FLAG_CONTIGUOUS)
> - mock_res->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
> + mock_res->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
>
> alloc_size = (uint64_t)mock_res->base.size;
> mutex_lock(&manager->lock);
> - err = drm_buddy_alloc_blocks(mm, fpfn, lpfn, alloc_size,
> + err = gpu_buddy_alloc_blocks(mm, fpfn, lpfn, alloc_size,
> manager->default_page_size,
> &mock_res->blocks,
> mock_res->flags);
> @@ -67,7 +67,7 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man,
> return 0;
>
> error_free_blocks:
> - drm_buddy_free_list(mm, &mock_res->blocks, 0);
> + gpu_buddy_free_list(mm, &mock_res->blocks, 0);
> ttm_resource_fini(man, &mock_res->base);
> mutex_unlock(&manager->lock);
>
> @@ -79,10 +79,10 @@ static void ttm_mock_manager_free(struct ttm_resource_manager *man,
> {
> struct ttm_mock_manager *manager = to_mock_mgr(man);
> struct ttm_mock_resource *mock_res = to_mock_mgr_resource(res);
> - struct drm_buddy *mm = &manager->mm;
> + struct gpu_buddy *mm = &manager->mm;
>
> mutex_lock(&manager->lock);
> - drm_buddy_free_list(mm, &mock_res->blocks, 0);
> + gpu_buddy_free_list(mm, &mock_res->blocks, 0);
> mutex_unlock(&manager->lock);
>
> ttm_resource_fini(man, res);
> @@ -106,7 +106,7 @@ int ttm_mock_manager_init(struct ttm_device *bdev, u32 mem_type, u32 size)
>
> mutex_init(&manager->lock);
>
> - err = drm_buddy_init(&manager->mm, size, PAGE_SIZE);
> + err = gpu_buddy_init(&manager->mm, size, PAGE_SIZE);
>
> if (err) {
> kfree(manager);
> @@ -142,7 +142,7 @@ void ttm_mock_manager_fini(struct ttm_device *bdev, u32 mem_type)
> ttm_resource_manager_set_used(man, false);
>
> mutex_lock(&mock_man->lock);
> - drm_buddy_fini(&mock_man->mm);
> + gpu_buddy_fini(&mock_man->mm);
> mutex_unlock(&mock_man->lock);
>
> ttm_set_driver_manager(bdev, mem_type, NULL);
> diff --git a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h
> index e4c95f86a467..08710756fd8e 100644
> --- a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h
> +++ b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h
> @@ -5,11 +5,11 @@
> #ifndef TTM_MOCK_MANAGER_H
> #define TTM_MOCK_MANAGER_H
>
> -#include <drm/drm_buddy.h>
> +#include <linux/gpu_buddy.h>
>
> struct ttm_mock_manager {
> struct ttm_resource_manager man;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> u64 default_page_size;
> /* protects allocations of mock buffer objects */
> struct mutex lock;
> diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> index 4b288eb3f5b0..982ef754742e 100644
> --- a/drivers/gpu/drm/xe/Kconfig
> +++ b/drivers/gpu/drm/xe/Kconfig
> @@ -11,6 +11,7 @@ config DRM_XE
> # the shmem_readpage() which depends upon tmpfs
> select SHMEM
> select TMPFS
> + select GPU_BUDDY
> select DRM_BUDDY
> select DRM_CLIENT_SELECTION
> select DRM_KMS_HELPER
> diff --git a/drivers/gpu/drm/xe/xe_res_cursor.h b/drivers/gpu/drm/xe/xe_res_cursor.h
> index 4e00008b7081..5f4ab08c0686 100644
> --- a/drivers/gpu/drm/xe/xe_res_cursor.h
> +++ b/drivers/gpu/drm/xe/xe_res_cursor.h
> @@ -58,7 +58,7 @@ struct xe_res_cursor {
> /** @dma_addr: Current element in a struct drm_pagemap_addr array */
> const struct drm_pagemap_addr *dma_addr;
> /** @mm: Buddy allocator for VRAM cursor */
> - struct drm_buddy *mm;
> + struct gpu_buddy *mm;
> /**
> * @dma_start: DMA start address for the current segment.
> * This may be different to @dma_addr.addr since elements in
> @@ -69,7 +69,7 @@ struct xe_res_cursor {
> u64 dma_seg_size;
> };
>
> -static struct drm_buddy *xe_res_get_buddy(struct ttm_resource *res)
> +static struct gpu_buddy *xe_res_get_buddy(struct ttm_resource *res)
> {
> struct ttm_resource_manager *mgr;
>
> @@ -104,30 +104,30 @@ static inline void xe_res_first(struct ttm_resource *res,
> case XE_PL_STOLEN:
> case XE_PL_VRAM0:
> case XE_PL_VRAM1: {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> struct list_head *head, *next;
> - struct drm_buddy *mm = xe_res_get_buddy(res);
> + struct gpu_buddy *mm = xe_res_get_buddy(res);
>
> head = &to_xe_ttm_vram_mgr_resource(res)->blocks;
>
> block = list_first_entry_or_null(head,
> - struct drm_buddy_block,
> + struct gpu_buddy_block,
> link);
> if (!block)
> goto fallback;
>
> - while (start >= drm_buddy_block_size(mm, block)) {
> - start -= drm_buddy_block_size(mm, block);
> + while (start >= gpu_buddy_block_size(mm, block)) {
> + start -= gpu_buddy_block_size(mm, block);
>
> next = block->link.next;
> if (next != head)
> - block = list_entry(next, struct drm_buddy_block,
> + block = list_entry(next, struct gpu_buddy_block,
> link);
> }
>
> cur->mm = mm;
> - cur->start = drm_buddy_block_offset(block) + start;
> - cur->size = min(drm_buddy_block_size(mm, block) - start,
> + cur->start = gpu_buddy_block_offset(block) + start;
> + cur->size = min(gpu_buddy_block_size(mm, block) - start,
> size);
> cur->remaining = size;
> cur->node = block;
> @@ -259,7 +259,7 @@ static inline void xe_res_first_dma(const struct drm_pagemap_addr *dma_addr,
> */
> static inline void xe_res_next(struct xe_res_cursor *cur, u64 size)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> struct list_head *next;
> u64 start;
>
> @@ -295,18 +295,18 @@ static inline void xe_res_next(struct xe_res_cursor *cur, u64 size)
> block = cur->node;
>
> next = block->link.next;
> - block = list_entry(next, struct drm_buddy_block, link);
> + block = list_entry(next, struct gpu_buddy_block, link);
>
>
> - while (start >= drm_buddy_block_size(cur->mm, block)) {
> - start -= drm_buddy_block_size(cur->mm, block);
> + while (start >= gpu_buddy_block_size(cur->mm, block)) {
> + start -= gpu_buddy_block_size(cur->mm, block);
>
> next = block->link.next;
> - block = list_entry(next, struct drm_buddy_block, link);
> + block = list_entry(next, struct gpu_buddy_block, link);
> }
>
> - cur->start = drm_buddy_block_offset(block) + start;
> - cur->size = min(drm_buddy_block_size(cur->mm, block) - start,
> + cur->start = gpu_buddy_block_offset(block) + start;
> + cur->size = min(gpu_buddy_block_size(cur->mm, block) - start,
> cur->remaining);
> cur->node = block;
> break;
> diff --git a/drivers/gpu/drm/xe/xe_svm.c b/drivers/gpu/drm/xe/xe_svm.c
> index f97e0af6a9b0..2b7e266f9bdd 100644
> --- a/drivers/gpu/drm/xe/xe_svm.c
> +++ b/drivers/gpu/drm/xe/xe_svm.c
> @@ -688,7 +688,7 @@ static u64 block_offset_to_pfn(struct xe_vram_region *vr, u64 offset)
> return PHYS_PFN(offset + vr->hpa_base);
> }
>
> -static struct drm_buddy *vram_to_buddy(struct xe_vram_region *vram)
> +static struct gpu_buddy *vram_to_buddy(struct xe_vram_region *vram)
> {
> return &vram->ttm.mm;
> }
> @@ -699,16 +699,16 @@ static int xe_svm_populate_devmem_pfn(struct drm_pagemap_devmem *devmem_allocati
> struct xe_bo *bo = to_xe_bo(devmem_allocation);
> struct ttm_resource *res = bo->ttm.resource;
> struct list_head *blocks = &to_xe_ttm_vram_mgr_resource(res)->blocks;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> int j = 0;
>
> list_for_each_entry(block, blocks, link) {
> struct xe_vram_region *vr = block->private;
> - struct drm_buddy *buddy = vram_to_buddy(vr);
> - u64 block_pfn = block_offset_to_pfn(vr, drm_buddy_block_offset(block));
> + struct gpu_buddy *buddy = vram_to_buddy(vr);
> + u64 block_pfn = block_offset_to_pfn(vr, gpu_buddy_block_offset(block));
> int i;
>
> - for (i = 0; i < drm_buddy_block_size(buddy, block) >> PAGE_SHIFT; ++i)
> + for (i = 0; i < gpu_buddy_block_size(buddy, block) >> PAGE_SHIFT; ++i)
> pfn[j++] = block_pfn + i;
> }
>
> @@ -876,7 +876,7 @@ static int xe_drm_pagemap_populate_mm(struct drm_pagemap *dpagemap,
> struct dma_fence *pre_migrate_fence = NULL;
> struct xe_device *xe = vr->xe;
> struct device *dev = xe->drm.dev;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> struct xe_validation_ctx vctx;
> struct list_head *blocks;
> struct drm_exec exec;
> diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> index 9f70802fce92..8192957261e8 100644
> --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
> @@ -4,8 +4,9 @@
> * Copyright (C) 2021-2022 Red Hat
> */
>
> -#include <drm/drm_managed.h>
> +#include <drm/drm_buddy.h>
> #include <drm/drm_drv.h>
> +#include <drm/drm_managed.h>
>
> #include <drm/ttm/ttm_placement.h>
> #include <drm/ttm/ttm_range_manager.h>
> @@ -17,16 +18,16 @@
> #include "xe_ttm_vram_mgr.h"
> #include "xe_vram_types.h"
>
> -static inline struct drm_buddy_block *
> +static inline struct gpu_buddy_block *
> xe_ttm_vram_mgr_first_block(struct list_head *list)
> {
> - return list_first_entry_or_null(list, struct drm_buddy_block, link);
> + return list_first_entry_or_null(list, struct gpu_buddy_block, link);
> }
>
> -static inline bool xe_is_vram_mgr_blocks_contiguous(struct drm_buddy *mm,
> +static inline bool xe_is_vram_mgr_blocks_contiguous(struct gpu_buddy *mm,
> struct list_head *head)
> {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> u64 start, size;
>
> block = xe_ttm_vram_mgr_first_block(head);
> @@ -34,12 +35,12 @@ static inline bool xe_is_vram_mgr_blocks_contiguous(struct drm_buddy *mm,
> return false;
>
> while (head != block->link.next) {
> - start = drm_buddy_block_offset(block);
> - size = drm_buddy_block_size(mm, block);
> + start = gpu_buddy_block_offset(block);
> + size = gpu_buddy_block_size(mm, block);
>
> - block = list_entry(block->link.next, struct drm_buddy_block,
> + block = list_entry(block->link.next, struct gpu_buddy_block,
> link);
> - if (start + size != drm_buddy_block_offset(block))
> + if (start + size != gpu_buddy_block_offset(block))
> return false;
> }
>
> @@ -53,7 +54,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
> {
> struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
> struct xe_ttm_vram_mgr_resource *vres;
> - struct drm_buddy *mm = &mgr->mm;
> + struct gpu_buddy *mm = &mgr->mm;
> u64 size, min_page_size;
> unsigned long lpfn;
> int err;
> @@ -80,10 +81,10 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
> INIT_LIST_HEAD(&vres->blocks);
>
> if (place->flags & TTM_PL_FLAG_TOPDOWN)
> - vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
> + vres->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
>
> if (place->fpfn || lpfn != man->size >> PAGE_SHIFT)
> - vres->flags |= DRM_BUDDY_RANGE_ALLOCATION;
> + vres->flags |= GPU_BUDDY_RANGE_ALLOCATION;
>
> if (WARN_ON(!vres->base.size)) {
> err = -EINVAL;
> @@ -119,27 +120,27 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
> lpfn = max_t(unsigned long, place->fpfn + (size >> PAGE_SHIFT), lpfn);
> }
>
> - err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
> + err = gpu_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
> (u64)lpfn << PAGE_SHIFT, size,
> min_page_size, &vres->blocks, vres->flags);
> if (err)
> goto error_unlock;
>
> if (place->flags & TTM_PL_FLAG_CONTIGUOUS) {
> - if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks))
> + if (!gpu_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks))
> size = vres->base.size;
> }
>
> if (lpfn <= mgr->visible_size >> PAGE_SHIFT) {
> vres->used_visible_size = size;
> } else {
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
>
> list_for_each_entry(block, &vres->blocks, link) {
> - u64 start = drm_buddy_block_offset(block);
> + u64 start = gpu_buddy_block_offset(block);
>
> if (start < mgr->visible_size) {
> - u64 end = start + drm_buddy_block_size(mm, block);
> + u64 end = start + gpu_buddy_block_size(mm, block);
>
> vres->used_visible_size +=
> min(end, mgr->visible_size) - start;
> @@ -159,11 +160,11 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
> * the object.
> */
> if (vres->base.placement & TTM_PL_FLAG_CONTIGUOUS) {
> - struct drm_buddy_block *block = list_first_entry(&vres->blocks,
> + struct gpu_buddy_block *block = list_first_entry(&vres->blocks,
> typeof(*block),
> link);
>
> - vres->base.start = drm_buddy_block_offset(block) >> PAGE_SHIFT;
> + vres->base.start = gpu_buddy_block_offset(block) >> PAGE_SHIFT;
> } else {
> vres->base.start = XE_BO_INVALID_OFFSET;
> }
> @@ -185,10 +186,10 @@ static void xe_ttm_vram_mgr_del(struct ttm_resource_manager *man,
> struct xe_ttm_vram_mgr_resource *vres =
> to_xe_ttm_vram_mgr_resource(res);
> struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
> - struct drm_buddy *mm = &mgr->mm;
> + struct gpu_buddy *mm = &mgr->mm;
>
> mutex_lock(&mgr->lock);
> - drm_buddy_free_list(mm, &vres->blocks, 0);
> + gpu_buddy_free_list(mm, &vres->blocks, 0);
> mgr->visible_avail += vres->used_visible_size;
> mutex_unlock(&mgr->lock);
>
> @@ -201,7 +202,7 @@ static void xe_ttm_vram_mgr_debug(struct ttm_resource_manager *man,
> struct drm_printer *printer)
> {
> struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
> - struct drm_buddy *mm = &mgr->mm;
> + struct gpu_buddy *mm = &mgr->mm;
>
> mutex_lock(&mgr->lock);
> drm_printf(printer, "default_page_size: %lluKiB\n",
> @@ -224,8 +225,8 @@ static bool xe_ttm_vram_mgr_intersects(struct ttm_resource_manager *man,
> struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
> struct xe_ttm_vram_mgr_resource *vres =
> to_xe_ttm_vram_mgr_resource(res);
> - struct drm_buddy *mm = &mgr->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = &mgr->mm;
> + struct gpu_buddy_block *block;
>
> if (!place->fpfn && !place->lpfn)
> return true;
> @@ -235,9 +236,9 @@ static bool xe_ttm_vram_mgr_intersects(struct ttm_resource_manager *man,
>
> list_for_each_entry(block, &vres->blocks, link) {
> unsigned long fpfn =
> - drm_buddy_block_offset(block) >> PAGE_SHIFT;
> + gpu_buddy_block_offset(block) >> PAGE_SHIFT;
> unsigned long lpfn = fpfn +
> - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
> + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
>
> if (place->fpfn < lpfn && place->lpfn > fpfn)
> return true;
> @@ -254,8 +255,8 @@ static bool xe_ttm_vram_mgr_compatible(struct ttm_resource_manager *man,
> struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
> struct xe_ttm_vram_mgr_resource *vres =
> to_xe_ttm_vram_mgr_resource(res);
> - struct drm_buddy *mm = &mgr->mm;
> - struct drm_buddy_block *block;
> + struct gpu_buddy *mm = &mgr->mm;
> + struct gpu_buddy_block *block;
>
> if (!place->fpfn && !place->lpfn)
> return true;
> @@ -265,9 +266,9 @@ static bool xe_ttm_vram_mgr_compatible(struct ttm_resource_manager *man,
>
> list_for_each_entry(block, &vres->blocks, link) {
> unsigned long fpfn =
> - drm_buddy_block_offset(block) >> PAGE_SHIFT;
> + gpu_buddy_block_offset(block) >> PAGE_SHIFT;
> unsigned long lpfn = fpfn +
> - (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
> + (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
>
> if (fpfn < place->fpfn || lpfn > place->lpfn)
> return false;
> @@ -297,7 +298,7 @@ static void xe_ttm_vram_mgr_fini(struct drm_device *dev, void *arg)
>
> WARN_ON_ONCE(mgr->visible_avail != mgr->visible_size);
>
> - drm_buddy_fini(&mgr->mm);
> + gpu_buddy_fini(&mgr->mm);
>
> ttm_resource_manager_cleanup(&mgr->manager);
>
> @@ -328,7 +329,7 @@ int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr,
> mgr->visible_avail = io_size;
>
> ttm_resource_manager_init(man, &xe->ttm, size);
> - err = drm_buddy_init(&mgr->mm, man->size, default_page_size);
> + err = gpu_buddy_init(&mgr->mm, man->size, default_page_size);
> if (err)
> return err;
>
> @@ -376,7 +377,7 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe,
> if (!*sgt)
> return -ENOMEM;
>
> - /* Determine the number of DRM_BUDDY blocks to export */
> + /* Determine the number of GPU_BUDDY blocks to export */
> xe_res_first(res, offset, length, &cursor);
> while (cursor.remaining) {
> num_entries++;
> @@ -393,10 +394,10 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe,
> sg->length = 0;
>
> /*
> - * Walk down DRM_BUDDY blocks to populate scatterlist nodes
> - * @note: Use iterator api to get first the DRM_BUDDY block
> + * Walk down GPU_BUDDY blocks to populate scatterlist nodes
> + * @note: Use iterator api to get first the GPU_BUDDY block
> * and the number of bytes from it. Access the following
> - * DRM_BUDDY block(s) if more buffer needs to exported
> + * GPU_BUDDY block(s) if more buffer needs to exported
> */
> xe_res_first(res, offset, length, &cursor);
> for_each_sgtable_sg((*sgt), sg, i) {
> diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> index a71e14818ec2..9106da056b49 100644
> --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
> @@ -6,7 +6,7 @@
> #ifndef _XE_TTM_VRAM_MGR_TYPES_H_
> #define _XE_TTM_VRAM_MGR_TYPES_H_
>
> -#include <drm/drm_buddy.h>
> +#include <linux/gpu_buddy.h>
> #include <drm/ttm/ttm_device.h>
>
> /**
> @@ -18,7 +18,7 @@ struct xe_ttm_vram_mgr {
> /** @manager: Base TTM resource manager */
> struct ttm_resource_manager manager;
> /** @mm: DRM buddy allocator which manages the VRAM */
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> /** @visible_size: Proped size of the CPU visible portion */
> u64 visible_size;
> /** @visible_avail: CPU visible portion still unallocated */
> diff --git a/drivers/gpu/tests/Makefile b/drivers/gpu/tests/Makefile
> new file mode 100644
> index 000000000000..31a5ff44cb4e
> --- /dev/null
> +++ b/drivers/gpu/tests/Makefile
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: GPL-2.0
> +
> +obj-$(CONFIG_GPU_BUDDY_KUNIT_TEST) += gpu_buddy_test.o gpu_random.o
> diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/tests/gpu_buddy_test.c
> similarity index 68%
> rename from drivers/gpu/drm/tests/drm_buddy_test.c
> rename to drivers/gpu/tests/gpu_buddy_test.c
> index 5f40b5343bd8..dcd4741a905d 100644
> --- a/drivers/gpu/drm/tests/drm_buddy_test.c
> +++ b/drivers/gpu/tests/gpu_buddy_test.c
> @@ -10,9 +10,9 @@
> #include <linux/sched/signal.h>
> #include <linux/sizes.h>
>
> -#include <drm/drm_buddy.h>
> +#include <linux/gpu_buddy.h>
>
> -#include "../lib/drm_random.h"
> +#include "gpu_random.h"
>
> static unsigned int random_seed;
>
> @@ -21,9 +21,9 @@ static inline u64 get_size(int order, u64 chunk_size)
> return (1 << order) * chunk_size;
> }
>
> -static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> +static void gpu_test_buddy_fragmentation_performance(struct kunit *test)
> {
> - struct drm_buddy_block *block, *tmp;
> + struct gpu_buddy_block *block, *tmp;
> int num_blocks, i, ret, count = 0;
> LIST_HEAD(allocated_blocks);
> unsigned long elapsed_ms;
> @@ -32,7 +32,7 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> LIST_HEAD(clear_list);
> LIST_HEAD(dirty_list);
> LIST_HEAD(free_list);
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> u64 mm_size = SZ_4G;
> ktime_t start, end;
>
> @@ -47,7 +47,7 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> * quickly the allocator can satisfy larger, aligned requests from a pool of
> * highly fragmented space.
> */
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
> "buddy_init failed\n");
>
> num_blocks = mm_size / SZ_64K;
> @@ -55,7 +55,7 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> start = ktime_get();
> /* Allocate with maximum fragmentation - 8K blocks with 64K alignment */
> for (i = 0; i < num_blocks; i++)
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
> &allocated_blocks, 0),
> "buddy_alloc hit an error size=%u\n", SZ_8K);
>
> @@ -68,21 +68,21 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> }
>
> /* Free with different flags to ensure no coalescing */
> - drm_buddy_free_list(&mm, &clear_list, DRM_BUDDY_CLEARED);
> - drm_buddy_free_list(&mm, &dirty_list, 0);
> + gpu_buddy_free_list(&mm, &clear_list, GPU_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &dirty_list, 0);
>
> for (i = 0; i < num_blocks; i++)
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, SZ_64K, SZ_64K,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, SZ_64K, SZ_64K,
> &test_blocks, 0),
> "buddy_alloc hit an error size=%u\n", SZ_64K);
> - drm_buddy_free_list(&mm, &test_blocks, 0);
> + gpu_buddy_free_list(&mm, &test_blocks, 0);
>
> end = ktime_get();
> elapsed_ms = ktime_to_ms(ktime_sub(end, start));
>
> kunit_info(test, "Fragmented allocation took %lu ms\n", elapsed_ms);
>
> - drm_buddy_fini(&mm);
> + gpu_buddy_fini(&mm);
>
> /*
> * Reverse free order under fragmentation
> @@ -96,13 +96,13 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> * deallocation occurs in the opposite order of allocation, exposing the
> * cost difference between a linear freelist scan and an ordered tree lookup.
> */
> - ret = drm_buddy_init(&mm, mm_size, SZ_4K);
> + ret = gpu_buddy_init(&mm, mm_size, SZ_4K);
> KUNIT_ASSERT_EQ(test, ret, 0);
>
> start = ktime_get();
> /* Allocate maximum fragmentation */
> for (i = 0; i < num_blocks; i++)
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
> &allocated_blocks, 0),
> "buddy_alloc hit an error size=%u\n", SZ_8K);
>
> @@ -111,28 +111,28 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
> list_move_tail(&block->link, &free_list);
> count++;
> }
> - drm_buddy_free_list(&mm, &free_list, DRM_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &free_list, GPU_BUDDY_CLEARED);
>
> list_for_each_entry_safe_reverse(block, tmp, &allocated_blocks, link)
> list_move(&block->link, &reverse_list);
> - drm_buddy_free_list(&mm, &reverse_list, DRM_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &reverse_list, GPU_BUDDY_CLEARED);
>
> end = ktime_get();
> elapsed_ms = ktime_to_ms(ktime_sub(end, start));
>
> kunit_info(test, "Reverse-ordered free took %lu ms\n", elapsed_ms);
>
> - drm_buddy_fini(&mm);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> +static void gpu_test_buddy_alloc_range_bias(struct kunit *test)
> {
> u32 mm_size, size, ps, bias_size, bias_start, bias_end, bias_rem;
> - DRM_RND_STATE(prng, random_seed);
> + GPU_RND_STATE(prng, random_seed);
> unsigned int i, count, *order;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> unsigned long flags;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> LIST_HEAD(allocated);
>
> bias_size = SZ_1M;
> @@ -142,11 +142,11 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
>
> kunit_info(test, "mm_size=%u, ps=%u\n", mm_size, ps);
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps),
> "buddy_init failed\n");
>
> count = mm_size / bias_size;
> - order = drm_random_order(count, &prng);
> + order = gpu_random_order(count, &prng);
> KUNIT_EXPECT_TRUE(test, order);
>
> /*
> @@ -166,79 +166,79 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
>
> /* internal round_up too big */
> KUNIT_ASSERT_TRUE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, bias_size + ps, bias_size,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, bias_size, bias_size);
>
> /* size too big */
> KUNIT_ASSERT_TRUE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, bias_size + ps, ps,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, bias_size + ps, ps);
>
> /* bias range too small for size */
> KUNIT_ASSERT_TRUE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start + ps,
> + gpu_buddy_alloc_blocks(&mm, bias_start + ps,
> bias_end, bias_size, ps,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n",
> bias_start + ps, bias_end, bias_size, ps);
>
> /* bias misaligned */
> KUNIT_ASSERT_TRUE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start + ps,
> + gpu_buddy_alloc_blocks(&mm, bias_start + ps,
> bias_end - ps,
> bias_size >> 1, bias_size >> 1,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc h didn't fail with bias(%x-%x), size=%u, ps=%u\n",
> bias_start + ps, bias_end - ps, bias_size >> 1, bias_size >> 1);
>
> /* single big page */
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, bias_size, bias_size,
> &tmp,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc i failed with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, bias_size, bias_size);
> - drm_buddy_free_list(&mm, &tmp, 0);
> + gpu_buddy_free_list(&mm, &tmp, 0);
>
> /* single page with internal round_up */
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, ps, bias_size,
> &tmp,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, ps, bias_size);
> - drm_buddy_free_list(&mm, &tmp, 0);
> + gpu_buddy_free_list(&mm, &tmp, 0);
>
> /* random size within */
> size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
> if (size)
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, size, ps,
> &tmp,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, size, ps);
>
> bias_rem -= size;
> /* too big for current avail */
> KUNIT_ASSERT_TRUE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, bias_rem + ps, ps,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, bias_rem + ps, ps);
>
> @@ -248,10 +248,10 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> size = max(size, ps);
>
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, size, ps,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, size, ps);
> /*
> @@ -259,15 +259,15 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> * unallocated, and ideally not always on the bias
> * boundaries.
> */
> - drm_buddy_free_list(&mm, &tmp, 0);
> + gpu_buddy_free_list(&mm, &tmp, 0);
> } else {
> list_splice_tail(&tmp, &allocated);
> }
> }
>
> kfree(order);
> - drm_buddy_free_list(&mm, &allocated, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &allocated, 0);
> + gpu_buddy_fini(&mm);
>
> /*
> * Something more free-form. Idea is to pick a random starting bias
> @@ -278,7 +278,7 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> * allocated nodes in the middle of the address space.
> */
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps),
> "buddy_init failed\n");
>
> bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps);
> @@ -290,10 +290,10 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> u32 size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
>
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, size, ps,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
> bias_start, bias_end, size, ps);
> bias_rem -= size;
> @@ -319,24 +319,24 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> KUNIT_ASSERT_EQ(test, bias_start, 0);
> KUNIT_ASSERT_EQ(test, bias_end, mm_size);
> KUNIT_ASSERT_TRUE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start, bias_end,
> + gpu_buddy_alloc_blocks(&mm, bias_start, bias_end,
> ps, ps,
> &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc passed with bias(%x-%x), size=%u\n",
> bias_start, bias_end, ps);
>
> - drm_buddy_free_list(&mm, &allocated, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &allocated, 0);
> + gpu_buddy_fini(&mm);
>
> /*
> - * Allocate cleared blocks in the bias range when the DRM buddy's clear avail is
> + * Allocate cleared blocks in the bias range when the GPU buddy's clear avail is
> * zero. This will validate the bias range allocation in scenarios like system boot
> * when no cleared blocks are available and exercise the fallback path too. The resulting
> * blocks should always be dirty.
> */
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps),
> "buddy_init failed\n");
>
> bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps);
> @@ -344,11 +344,11 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> bias_end = max(bias_end, bias_start + ps);
> bias_rem = bias_end - bias_start;
>
> - flags = DRM_BUDDY_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION;
> + flags = GPU_BUDDY_CLEAR_ALLOCATION | GPU_BUDDY_RANGE_ALLOCATION;
> size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
>
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, bias_start,
> + gpu_buddy_alloc_blocks(&mm, bias_start,
> bias_end, size, ps,
> &allocated,
> flags),
> @@ -356,27 +356,27 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
> bias_start, bias_end, size, ps);
>
> list_for_each_entry(block, &allocated, link)
> - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
> + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false);
>
> - drm_buddy_free_list(&mm, &allocated, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &allocated, 0);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_clear(struct kunit *test)
> +static void gpu_test_buddy_alloc_clear(struct kunit *test)
> {
> unsigned long n_pages, total, i = 0;
> const unsigned long ps = SZ_4K;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> const int max_order = 12;
> LIST_HEAD(allocated);
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> unsigned int order;
> u32 mm_size, size;
> LIST_HEAD(dirty);
> LIST_HEAD(clean);
>
> mm_size = SZ_4K << max_order;
> - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
> + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
>
> KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
>
> @@ -389,11 +389,11 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
> * is indeed all dirty pages and vice versa. Free it all again,
> * keeping the dirty/clear status.
> */
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 5 * ps, ps, &allocated,
> - DRM_BUDDY_TOPDOWN_ALLOCATION),
> + GPU_BUDDY_TOPDOWN_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", 5 * ps);
> - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
>
> n_pages = 10;
> do {
> @@ -406,37 +406,37 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
> flags = 0;
> } else {
> list = &clean;
> - flags = DRM_BUDDY_CLEAR_ALLOCATION;
> + flags = GPU_BUDDY_CLEAR_ALLOCATION;
> }
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> ps, ps, list,
> flags),
> "buddy_alloc hit an error size=%lu\n", ps);
> } while (++i < n_pages);
>
> list_for_each_entry(block, &clean, link)
> - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), true);
> + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), true);
>
> list_for_each_entry(block, &dirty, link)
> - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
> + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false);
>
> - drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED);
>
> /*
> * Trying to go over the clear limit for some allocation.
> * The allocation should never fail with reasonable page-size.
> */
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 10 * ps, ps, &clean,
> - DRM_BUDDY_CLEAR_ALLOCATION),
> + GPU_BUDDY_CLEAR_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", 10 * ps);
>
> - drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
> - drm_buddy_free_list(&mm, &dirty, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &dirty, 0);
> + gpu_buddy_fini(&mm);
>
> - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
> + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
>
> /*
> * Create a new mm. Intentionally fragment the address space by creating
> @@ -458,34 +458,34 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
> else
> list = &clean;
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> ps, ps, list, 0),
> "buddy_alloc hit an error size=%lu\n", ps);
> } while (++i < n_pages);
>
> - drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
> - drm_buddy_free_list(&mm, &dirty, 0);
> + gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED);
> + gpu_buddy_free_list(&mm, &dirty, 0);
>
> order = 1;
> do {
> size = SZ_4K << order;
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> size, size, &allocated,
> - DRM_BUDDY_CLEAR_ALLOCATION),
> + GPU_BUDDY_CLEAR_ALLOCATION),
> "buddy_alloc hit an error size=%u\n", size);
> total = 0;
> list_for_each_entry(block, &allocated, link) {
> if (size != mm_size)
> - KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
> - total += drm_buddy_block_size(&mm, block);
> + KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false);
> + total += gpu_buddy_block_size(&mm, block);
> }
> KUNIT_EXPECT_EQ(test, total, size);
>
> - drm_buddy_free_list(&mm, &allocated, 0);
> + gpu_buddy_free_list(&mm, &allocated, 0);
> } while (++order <= max_order);
>
> - drm_buddy_fini(&mm);
> + gpu_buddy_fini(&mm);
>
> /*
> * Create a new mm with a non power-of-two size. Allocate a random size from each
> @@ -494,44 +494,44 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
> */
> mm_size = (SZ_4K << max_order) + (SZ_4K << (max_order - 2));
>
> - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
> + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
> KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
> 4 * ps, ps, &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", 4 * ps);
> - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
> + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
> 2 * ps, ps, &allocated,
> - DRM_BUDDY_CLEAR_ALLOCATION),
> + GPU_BUDDY_CLEAR_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", 2 * ps);
> - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size,
> + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size,
> ps, ps, &allocated,
> - DRM_BUDDY_RANGE_ALLOCATION),
> + GPU_BUDDY_RANGE_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", ps);
> - drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_contiguous(struct kunit *test)
> +static void gpu_test_buddy_alloc_contiguous(struct kunit *test)
> {
> const unsigned long ps = SZ_4K, mm_size = 16 * 3 * SZ_4K;
> unsigned long i, n_pages, total;
> - struct drm_buddy_block *block;
> - struct drm_buddy mm;
> + struct gpu_buddy_block *block;
> + struct gpu_buddy mm;
> LIST_HEAD(left);
> LIST_HEAD(middle);
> LIST_HEAD(right);
> LIST_HEAD(allocated);
>
> - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
> + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
>
> /*
> * Idea is to fragment the address space by alternating block
> * allocations between three different lists; one for left, middle and
> * right. We can then free a list to simulate fragmentation. In
> - * particular we want to exercise the DRM_BUDDY_CONTIGUOUS_ALLOCATION,
> + * particular we want to exercise the GPU_BUDDY_CONTIGUOUS_ALLOCATION,
> * including the try_harder path.
> */
>
> @@ -548,66 +548,66 @@ static void drm_test_buddy_alloc_contiguous(struct kunit *test)
> else
> list = &right;
> KUNIT_ASSERT_FALSE_MSG(test,
> - drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> ps, ps, list, 0),
> "buddy_alloc hit an error size=%lu\n",
> ps);
> } while (++i < n_pages);
>
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 3 * ps, ps, &allocated,
> - DRM_BUDDY_CONTIGUOUS_ALLOCATION),
> + GPU_BUDDY_CONTIGUOUS_ALLOCATION),
> "buddy_alloc didn't error size=%lu\n", 3 * ps);
>
> - drm_buddy_free_list(&mm, &middle, 0);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + gpu_buddy_free_list(&mm, &middle, 0);
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 3 * ps, ps, &allocated,
> - DRM_BUDDY_CONTIGUOUS_ALLOCATION),
> + GPU_BUDDY_CONTIGUOUS_ALLOCATION),
> "buddy_alloc didn't error size=%lu\n", 3 * ps);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 2 * ps, ps, &allocated,
> - DRM_BUDDY_CONTIGUOUS_ALLOCATION),
> + GPU_BUDDY_CONTIGUOUS_ALLOCATION),
> "buddy_alloc didn't error size=%lu\n", 2 * ps);
>
> - drm_buddy_free_list(&mm, &right, 0);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + gpu_buddy_free_list(&mm, &right, 0);
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 3 * ps, ps, &allocated,
> - DRM_BUDDY_CONTIGUOUS_ALLOCATION),
> + GPU_BUDDY_CONTIGUOUS_ALLOCATION),
> "buddy_alloc didn't error size=%lu\n", 3 * ps);
> /*
> * At this point we should have enough contiguous space for 2 blocks,
> * however they are never buddies (since we freed middle and right) so
> * will require the try_harder logic to find them.
> */
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 2 * ps, ps, &allocated,
> - DRM_BUDDY_CONTIGUOUS_ALLOCATION),
> + GPU_BUDDY_CONTIGUOUS_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", 2 * ps);
>
> - drm_buddy_free_list(&mm, &left, 0);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
> + gpu_buddy_free_list(&mm, &left, 0);
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
> 3 * ps, ps, &allocated,
> - DRM_BUDDY_CONTIGUOUS_ALLOCATION),
> + GPU_BUDDY_CONTIGUOUS_ALLOCATION),
> "buddy_alloc hit an error size=%lu\n", 3 * ps);
>
> total = 0;
> list_for_each_entry(block, &allocated, link)
> - total += drm_buddy_block_size(&mm, block);
> + total += gpu_buddy_block_size(&mm, block);
>
> KUNIT_ASSERT_EQ(test, total, ps * 2 + ps * 3);
>
> - drm_buddy_free_list(&mm, &allocated, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &allocated, 0);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_pathological(struct kunit *test)
> +static void gpu_test_buddy_alloc_pathological(struct kunit *test)
> {
> u64 mm_size, size, start = 0;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> const int max_order = 3;
> unsigned long flags = 0;
> int order, top;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> LIST_HEAD(blocks);
> LIST_HEAD(holes);
> LIST_HEAD(tmp);
> @@ -620,7 +620,7 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test)
> */
>
> mm_size = SZ_4K << max_order;
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
> "buddy_init failed\n");
>
> KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
> @@ -630,18 +630,18 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test)
> block = list_first_entry_or_null(&blocks, typeof(*block), link);
> if (block) {
> list_del(&block->link);
> - drm_buddy_free_block(&mm, block);
> + gpu_buddy_free_block(&mm, block);
> }
>
> for (order = top; order--;) {
> size = get_size(order, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start,
> mm_size, size, size,
> &tmp, flags),
> "buddy_alloc hit -ENOMEM with order=%d, top=%d\n",
> order, top);
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_move_tail(&block->link, &blocks);
> @@ -649,45 +649,45 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test)
>
> /* There should be one final page for this sub-allocation */
> size = get_size(0, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc hit -ENOMEM for hole\n");
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_move_tail(&block->link, &holes);
>
> size = get_size(top, mm.chunk_size);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc unexpectedly succeeded at top-order %d/%d, it should be full!",
> top, max_order);
> }
>
> - drm_buddy_free_list(&mm, &holes, 0);
> + gpu_buddy_free_list(&mm, &holes, 0);
>
> /* Nothing larger than blocks of chunk_size now available */
> for (order = 1; order <= max_order; order++) {
> size = get_size(order, mm.chunk_size);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc unexpectedly succeeded at order %d, it should be full!",
> order);
> }
>
> list_splice_tail(&holes, &blocks);
> - drm_buddy_free_list(&mm, &blocks, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &blocks, 0);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
> +static void gpu_test_buddy_alloc_pessimistic(struct kunit *test)
> {
> u64 mm_size, size, start = 0;
> - struct drm_buddy_block *block, *bn;
> + struct gpu_buddy_block *block, *bn;
> const unsigned int max_order = 16;
> unsigned long flags = 0;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> unsigned int order;
> LIST_HEAD(blocks);
> LIST_HEAD(tmp);
> @@ -699,19 +699,19 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
> */
>
> mm_size = SZ_4K << max_order;
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
> "buddy_init failed\n");
>
> KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
>
> for (order = 0; order < max_order; order++) {
> size = get_size(order, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc hit -ENOMEM with order=%d\n",
> order);
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_move_tail(&block->link, &blocks);
> @@ -719,11 +719,11 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
>
> /* And now the last remaining block available */
> size = get_size(0, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc hit -ENOMEM on final alloc\n");
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_move_tail(&block->link, &blocks);
> @@ -731,58 +731,58 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
> /* Should be completely full! */
> for (order = max_order; order--;) {
> size = get_size(order, mm.chunk_size);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc unexpectedly succeeded, it should be full!");
> }
>
> block = list_last_entry(&blocks, typeof(*block), link);
> list_del(&block->link);
> - drm_buddy_free_block(&mm, block);
> + gpu_buddy_free_block(&mm, block);
>
> /* As we free in increasing size, we make available larger blocks */
> order = 1;
> list_for_each_entry_safe(block, bn, &blocks, link) {
> list_del(&block->link);
> - drm_buddy_free_block(&mm, block);
> + gpu_buddy_free_block(&mm, block);
>
> size = get_size(order, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc hit -ENOMEM with order=%d\n",
> order);
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_del(&block->link);
> - drm_buddy_free_block(&mm, block);
> + gpu_buddy_free_block(&mm, block);
> order++;
> }
>
> /* To confirm, now the whole mm should be available */
> size = get_size(max_order, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc (realloc) hit -ENOMEM with order=%d\n",
> max_order);
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_del(&block->link);
> - drm_buddy_free_block(&mm, block);
> - drm_buddy_free_list(&mm, &blocks, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_block(&mm, block);
> + gpu_buddy_free_list(&mm, &blocks, 0);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_optimistic(struct kunit *test)
> +static void gpu_test_buddy_alloc_optimistic(struct kunit *test)
> {
> u64 mm_size, size, start = 0;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> unsigned long flags = 0;
> const int max_order = 16;
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
> LIST_HEAD(blocks);
> LIST_HEAD(tmp);
> int order;
> @@ -794,19 +794,19 @@ static void drm_test_buddy_alloc_optimistic(struct kunit *test)
>
> mm_size = SZ_4K * ((1 << (max_order + 1)) - 1);
>
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
> "buddy_init failed\n");
>
> KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
>
> for (order = 0; order <= max_order; order++) {
> size = get_size(order, mm.chunk_size);
> - KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc hit -ENOMEM with order=%d\n",
> order);
>
> - block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
> KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
>
> list_move_tail(&block->link, &blocks);
> @@ -814,80 +814,80 @@ static void drm_test_buddy_alloc_optimistic(struct kunit *test)
>
> /* Should be completely full! */
> size = get_size(0, mm.chunk_size);
> - KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
> + KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
> size, size, &tmp, flags),
> "buddy_alloc unexpectedly succeeded, it should be full!");
>
> - drm_buddy_free_list(&mm, &blocks, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &blocks, 0);
> + gpu_buddy_fini(&mm);
> }
>
> -static void drm_test_buddy_alloc_limit(struct kunit *test)
> +static void gpu_test_buddy_alloc_limit(struct kunit *test)
> {
> u64 size = U64_MAX, start = 0;
> - struct drm_buddy_block *block;
> + struct gpu_buddy_block *block;
> unsigned long flags = 0;
> LIST_HEAD(allocated);
> - struct drm_buddy mm;
> + struct gpu_buddy mm;
>
> - KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, size, SZ_4K));
> + KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, size, SZ_4K));
>
> - KUNIT_EXPECT_EQ_MSG(test, mm.max_order, DRM_BUDDY_MAX_ORDER,
> + KUNIT_EXPECT_EQ_MSG(test, mm.max_order, GPU_BUDDY_MAX_ORDER,
> "mm.max_order(%d) != %d\n", mm.max_order,
> - DRM_BUDDY_MAX_ORDER);
> + GPU_BUDDY_MAX_ORDER);
>
> size = mm.chunk_size << mm.max_order;
> - KUNIT_EXPECT_FALSE(test, drm_buddy_alloc_blocks(&mm, start, size, size,
> + KUNIT_EXPECT_FALSE(test, gpu_buddy_alloc_blocks(&mm, start, size, size,
> mm.chunk_size, &allocated, flags));
>
> - block = list_first_entry_or_null(&allocated, struct drm_buddy_block, link);
> + block = list_first_entry_or_null(&allocated, struct gpu_buddy_block, link);
> KUNIT_EXPECT_TRUE(test, block);
>
> - KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_order(block), mm.max_order,
> + KUNIT_EXPECT_EQ_MSG(test, gpu_buddy_block_order(block), mm.max_order,
> "block order(%d) != %d\n",
> - drm_buddy_block_order(block), mm.max_order);
> + gpu_buddy_block_order(block), mm.max_order);
>
> - KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_size(&mm, block),
> + KUNIT_EXPECT_EQ_MSG(test, gpu_buddy_block_size(&mm, block),
> BIT_ULL(mm.max_order) * mm.chunk_size,
> "block size(%llu) != %llu\n",
> - drm_buddy_block_size(&mm, block),
> + gpu_buddy_block_size(&mm, block),
> BIT_ULL(mm.max_order) * mm.chunk_size);
>
> - drm_buddy_free_list(&mm, &allocated, 0);
> - drm_buddy_fini(&mm);
> + gpu_buddy_free_list(&mm, &allocated, 0);
> + gpu_buddy_fini(&mm);
> }
>
> -static int drm_buddy_suite_init(struct kunit_suite *suite)
> +static int gpu_buddy_suite_init(struct kunit_suite *suite)
> {
> while (!random_seed)
> random_seed = get_random_u32();
>
> - kunit_info(suite, "Testing DRM buddy manager, with random_seed=0x%x\n",
> + kunit_info(suite, "Testing GPU buddy manager, with random_seed=0x%x\n",
> random_seed);
>
> return 0;
> }
>
> -static struct kunit_case drm_buddy_tests[] = {
> - KUNIT_CASE(drm_test_buddy_alloc_limit),
> - KUNIT_CASE(drm_test_buddy_alloc_optimistic),
> - KUNIT_CASE(drm_test_buddy_alloc_pessimistic),
> - KUNIT_CASE(drm_test_buddy_alloc_pathological),
> - KUNIT_CASE(drm_test_buddy_alloc_contiguous),
> - KUNIT_CASE(drm_test_buddy_alloc_clear),
> - KUNIT_CASE(drm_test_buddy_alloc_range_bias),
> - KUNIT_CASE(drm_test_buddy_fragmentation_performance),
> +static struct kunit_case gpu_buddy_tests[] = {
> + KUNIT_CASE(gpu_test_buddy_alloc_limit),
> + KUNIT_CASE(gpu_test_buddy_alloc_optimistic),
> + KUNIT_CASE(gpu_test_buddy_alloc_pessimistic),
> + KUNIT_CASE(gpu_test_buddy_alloc_pathological),
> + KUNIT_CASE(gpu_test_buddy_alloc_contiguous),
> + KUNIT_CASE(gpu_test_buddy_alloc_clear),
> + KUNIT_CASE(gpu_test_buddy_alloc_range_bias),
> + KUNIT_CASE(gpu_test_buddy_fragmentation_performance),
> {}
> };
>
> -static struct kunit_suite drm_buddy_test_suite = {
> - .name = "drm_buddy",
> - .suite_init = drm_buddy_suite_init,
> - .test_cases = drm_buddy_tests,
> +static struct kunit_suite gpu_buddy_test_suite = {
> + .name = "gpu_buddy",
> + .suite_init = gpu_buddy_suite_init,
> + .test_cases = gpu_buddy_tests,
> };
>
> -kunit_test_suite(drm_buddy_test_suite);
> +kunit_test_suite(gpu_buddy_test_suite);
>
> MODULE_AUTHOR("Intel Corporation");
> -MODULE_DESCRIPTION("Kunit test for drm_buddy functions");
> +MODULE_DESCRIPTION("Kunit test for gpu_buddy functions");
> MODULE_LICENSE("GPL");
> diff --git a/drivers/gpu/tests/gpu_random.c b/drivers/gpu/tests/gpu_random.c
> new file mode 100644
> index 000000000000..54f1f6a3a6c1
> --- /dev/null
> +++ b/drivers/gpu/tests/gpu_random.c
> @@ -0,0 +1,48 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/bitops.h>
> +#include <linux/export.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/random.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +
> +#include "gpu_random.h"
> +
> +u32 gpu_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state)
> +{
> + return upper_32_bits((u64)prandom_u32_state(state) * ep_ro);
> +}
> +EXPORT_SYMBOL(gpu_prandom_u32_max_state);
> +
> +void gpu_random_reorder(unsigned int *order, unsigned int count,
> + struct rnd_state *state)
> +{
> + unsigned int i, j;
> +
> + for (i = 0; i < count; ++i) {
> + BUILD_BUG_ON(sizeof(unsigned int) > sizeof(u32));
> + j = gpu_prandom_u32_max_state(count, state);
> + swap(order[i], order[j]);
> + }
> +}
> +EXPORT_SYMBOL(gpu_random_reorder);
> +
> +unsigned int *gpu_random_order(unsigned int count, struct rnd_state *state)
> +{
> + unsigned int *order, i;
> +
> + order = kmalloc_array(count, sizeof(*order), GFP_KERNEL);
> + if (!order)
> + return order;
> +
> + for (i = 0; i < count; i++)
> + order[i] = i;
> +
> + gpu_random_reorder(order, count, state);
> + return order;
> +}
> +EXPORT_SYMBOL(gpu_random_order);
> +
> +MODULE_DESCRIPTION("GPU Randomization Utilities");
> +MODULE_LICENSE("Dual MIT/GPL");
> diff --git a/drivers/gpu/tests/gpu_random.h b/drivers/gpu/tests/gpu_random.h
> new file mode 100644
> index 000000000000..b68cf3448264
> --- /dev/null
> +++ b/drivers/gpu/tests/gpu_random.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __GPU_RANDOM_H__
> +#define __GPU_RANDOM_H__
> +
> +/* This is a temporary home for a couple of utility functions that should
> + * be transposed to lib/ at the earliest convenience.
> + */
> +
> +#include <linux/prandom.h>
> +
> +#define GPU_RND_STATE_INITIALIZER(seed__) ({ \
> + struct rnd_state state__; \
> + prandom_seed_state(&state__, (seed__)); \
> + state__; \
> +})
> +
> +#define GPU_RND_STATE(name__, seed__) \
> + struct rnd_state name__ = GPU_RND_STATE_INITIALIZER(seed__)
> +
> +unsigned int *gpu_random_order(unsigned int count,
> + struct rnd_state *state);
> +void gpu_random_reorder(unsigned int *order,
> + unsigned int count,
> + struct rnd_state *state);
> +u32 gpu_prandom_u32_max_state(u32 ep_ro,
> + struct rnd_state *state);
> +
> +#endif /* !__GPU_RANDOM_H__ */
> diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig
> index d51777df12d1..6ae1383b0e2e 100644
> --- a/drivers/video/Kconfig
> +++ b/drivers/video/Kconfig
> @@ -37,6 +37,8 @@ source "drivers/char/agp/Kconfig"
>
> source "drivers/gpu/vga/Kconfig"
>
> +source "drivers/gpu/Kconfig"
> +
> source "drivers/gpu/host1x/Kconfig"
> source "drivers/gpu/ipu-v3/Kconfig"
> source "drivers/gpu/nova-core/Kconfig"
> diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h
> index b909fa8f810a..3054369bebff 100644
> --- a/include/drm/drm_buddy.h
> +++ b/include/drm/drm_buddy.h
> @@ -6,166 +6,13 @@
> #ifndef __DRM_BUDDY_H__
> #define __DRM_BUDDY_H__
>
> -#include <linux/bitops.h>
> -#include <linux/list.h>
> -#include <linux/slab.h>
> -#include <linux/sched.h>
> -#include <linux/rbtree.h>
> +#include <linux/gpu_buddy.h>
>
> struct drm_printer;
>
> -#define DRM_BUDDY_RANGE_ALLOCATION BIT(0)
> -#define DRM_BUDDY_TOPDOWN_ALLOCATION BIT(1)
> -#define DRM_BUDDY_CONTIGUOUS_ALLOCATION BIT(2)
> -#define DRM_BUDDY_CLEAR_ALLOCATION BIT(3)
> -#define DRM_BUDDY_CLEARED BIT(4)
> -#define DRM_BUDDY_TRIM_DISABLE BIT(5)
> -
> -struct drm_buddy_block {
> -#define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
> -#define DRM_BUDDY_HEADER_STATE GENMASK_ULL(11, 10)
> -#define DRM_BUDDY_ALLOCATED (1 << 10)
> -#define DRM_BUDDY_FREE (2 << 10)
> -#define DRM_BUDDY_SPLIT (3 << 10)
> -#define DRM_BUDDY_HEADER_CLEAR GENMASK_ULL(9, 9)
> -/* Free to be used, if needed in the future */
> -#define DRM_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6)
> -#define DRM_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0)
> - u64 header;
> -
> - struct drm_buddy_block *left;
> - struct drm_buddy_block *right;
> - struct drm_buddy_block *parent;
> -
> - void *private; /* owned by creator */
> -
> - /*
> - * While the block is allocated by the user through drm_buddy_alloc*,
> - * the user has ownership of the link, for example to maintain within
> - * a list, if so desired. As soon as the block is freed with
> - * drm_buddy_free* ownership is given back to the mm.
> - */
> - union {
> - struct rb_node rb;
> - struct list_head link;
> - };
> -
> - struct list_head tmp_link;
> -};
> -
> -/* Order-zero must be at least SZ_4K */
> -#define DRM_BUDDY_MAX_ORDER (63 - 12)
> -
> -/*
> - * Binary Buddy System.
> - *
> - * Locking should be handled by the user, a simple mutex around
> - * drm_buddy_alloc* and drm_buddy_free* should suffice.
> - */
> -struct drm_buddy {
> - /* Maintain a free list for each order. */
> - struct rb_root **free_trees;
> -
> - /*
> - * Maintain explicit binary tree(s) to track the allocation of the
> - * address space. This gives us a simple way of finding a buddy block
> - * and performing the potentially recursive merge step when freeing a
> - * block. Nodes are either allocated or free, in which case they will
> - * also exist on the respective free list.
> - */
> - struct drm_buddy_block **roots;
> -
> - /*
> - * Anything from here is public, and remains static for the lifetime of
> - * the mm. Everything above is considered do-not-touch.
> - */
> - unsigned int n_roots;
> - unsigned int max_order;
> -
> - /* Must be at least SZ_4K */
> - u64 chunk_size;
> - u64 size;
> - u64 avail;
> - u64 clear_avail;
> -};
> -
> -static inline u64
> -drm_buddy_block_offset(const struct drm_buddy_block *block)
> -{
> - return block->header & DRM_BUDDY_HEADER_OFFSET;
> -}
> -
> -static inline unsigned int
> -drm_buddy_block_order(struct drm_buddy_block *block)
> -{
> - return block->header & DRM_BUDDY_HEADER_ORDER;
> -}
> -
> -static inline unsigned int
> -drm_buddy_block_state(struct drm_buddy_block *block)
> -{
> - return block->header & DRM_BUDDY_HEADER_STATE;
> -}
> -
> -static inline bool
> -drm_buddy_block_is_allocated(struct drm_buddy_block *block)
> -{
> - return drm_buddy_block_state(block) == DRM_BUDDY_ALLOCATED;
> -}
> -
> -static inline bool
> -drm_buddy_block_is_clear(struct drm_buddy_block *block)
> -{
> - return block->header & DRM_BUDDY_HEADER_CLEAR;
> -}
> -
> -static inline bool
> -drm_buddy_block_is_free(struct drm_buddy_block *block)
> -{
> - return drm_buddy_block_state(block) == DRM_BUDDY_FREE;
> -}
> -
> -static inline bool
> -drm_buddy_block_is_split(struct drm_buddy_block *block)
> -{
> - return drm_buddy_block_state(block) == DRM_BUDDY_SPLIT;
> -}
> -
> -static inline u64
> -drm_buddy_block_size(struct drm_buddy *mm,
> - struct drm_buddy_block *block)
> -{
> - return mm->chunk_size << drm_buddy_block_order(block);
> -}
> -
> -int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size);
> -
> -void drm_buddy_fini(struct drm_buddy *mm);
> -
> -struct drm_buddy_block *
> -drm_get_buddy(struct drm_buddy_block *block);
> -
> -int drm_buddy_alloc_blocks(struct drm_buddy *mm,
> - u64 start, u64 end, u64 size,
> - u64 min_page_size,
> - struct list_head *blocks,
> - unsigned long flags);
> -
> -int drm_buddy_block_trim(struct drm_buddy *mm,
> - u64 *start,
> - u64 new_size,
> - struct list_head *blocks);
> -
> -void drm_buddy_reset_clear(struct drm_buddy *mm, bool is_clear);
> -
> -void drm_buddy_free_block(struct drm_buddy *mm, struct drm_buddy_block *block);
> -
> -void drm_buddy_free_list(struct drm_buddy *mm,
> - struct list_head *objects,
> - unsigned int flags);
> -
> -void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p);
> -void drm_buddy_block_print(struct drm_buddy *mm,
> - struct drm_buddy_block *block,
> +/* DRM-specific GPU Buddy Allocator print helpers */
> +void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p);
> +void drm_buddy_block_print(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block,
> struct drm_printer *p);
> #endif
> diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
> new file mode 100644
> index 000000000000..3e4bd11ccb71
> --- /dev/null
> +++ b/include/linux/gpu_buddy.h
> @@ -0,0 +1,177 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2021 Intel Corporation
> + */
> +
> +#ifndef __GPU_BUDDY_H__
> +#define __GPU_BUDDY_H__
> +
> +#include <linux/bitops.h>
> +#include <linux/list.h>
> +#include <linux/rbtree.h>
> +#include <linux/slab.h>
> +#include <linux/sched.h>
> +
> +#define GPU_BUDDY_RANGE_ALLOCATION BIT(0)
> +#define GPU_BUDDY_TOPDOWN_ALLOCATION BIT(1)
> +#define GPU_BUDDY_CONTIGUOUS_ALLOCATION BIT(2)
> +#define GPU_BUDDY_CLEAR_ALLOCATION BIT(3)
> +#define GPU_BUDDY_CLEARED BIT(4)
> +#define GPU_BUDDY_TRIM_DISABLE BIT(5)
> +
> +enum gpu_buddy_free_tree {
> + GPU_BUDDY_CLEAR_TREE = 0,
> + GPU_BUDDY_DIRTY_TREE,
> + GPU_BUDDY_MAX_FREE_TREES,
> +};
> +
> +#define for_each_free_tree(tree) \
> + for ((tree) = 0; (tree) < GPU_BUDDY_MAX_FREE_TREES; (tree)++)
> +
> +struct gpu_buddy_block {
> +#define GPU_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
> +#define GPU_BUDDY_HEADER_STATE GENMASK_ULL(11, 10)
> +#define GPU_BUDDY_ALLOCATED (1 << 10)
> +#define GPU_BUDDY_FREE (2 << 10)
> +#define GPU_BUDDY_SPLIT (3 << 10)
> +#define GPU_BUDDY_HEADER_CLEAR GENMASK_ULL(9, 9)
> +/* Free to be used, if needed in the future */
> +#define GPU_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6)
> +#define GPU_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0)
> + u64 header;
> +
> + struct gpu_buddy_block *left;
> + struct gpu_buddy_block *right;
> + struct gpu_buddy_block *parent;
> +
> + void *private; /* owned by creator */
> +
> + /*
> + * While the block is allocated by the user through gpu_buddy_alloc*,
> + * the user has ownership of the link, for example to maintain within
> + * a list, if so desired. As soon as the block is freed with
> + * gpu_buddy_free* ownership is given back to the mm.
> + */
> + union {
> + struct rb_node rb;
> + struct list_head link;
> + };
> +
> + struct list_head tmp_link;
> +};
> +
> +/* Order-zero must be at least SZ_4K */
> +#define GPU_BUDDY_MAX_ORDER (63 - 12)
> +
> +/*
> + * Binary Buddy System.
> + *
> + * Locking should be handled by the user, a simple mutex around
> + * gpu_buddy_alloc* and gpu_buddy_free* should suffice.
> + */
> +struct gpu_buddy {
> + /* Maintain a free list for each order. */
> + struct rb_root **free_trees;
> +
> + /*
> + * Maintain explicit binary tree(s) to track the allocation of the
> + * address space. This gives us a simple way of finding a buddy block
> + * and performing the potentially recursive merge step when freeing a
> + * block. Nodes are either allocated or free, in which case they will
> + * also exist on the respective free list.
> + */
> + struct gpu_buddy_block **roots;
> +
> + /*
> + * Anything from here is public, and remains static for the lifetime of
> + * the mm. Everything above is considered do-not-touch.
> + */
> + unsigned int n_roots;
> + unsigned int max_order;
> +
> + /* Must be at least SZ_4K */
> + u64 chunk_size;
> + u64 size;
> + u64 avail;
> + u64 clear_avail;
> +};
> +
> +static inline u64
> +gpu_buddy_block_offset(const struct gpu_buddy_block *block)
> +{
> + return block->header & GPU_BUDDY_HEADER_OFFSET;
> +}
> +
> +static inline unsigned int
> +gpu_buddy_block_order(struct gpu_buddy_block *block)
> +{
> + return block->header & GPU_BUDDY_HEADER_ORDER;
> +}
> +
> +static inline unsigned int
> +gpu_buddy_block_state(struct gpu_buddy_block *block)
> +{
> + return block->header & GPU_BUDDY_HEADER_STATE;
> +}
> +
> +static inline bool
> +gpu_buddy_block_is_allocated(struct gpu_buddy_block *block)
> +{
> + return gpu_buddy_block_state(block) == GPU_BUDDY_ALLOCATED;
> +}
> +
> +static inline bool
> +gpu_buddy_block_is_clear(struct gpu_buddy_block *block)
> +{
> + return block->header & GPU_BUDDY_HEADER_CLEAR;
> +}
> +
> +static inline bool
> +gpu_buddy_block_is_free(struct gpu_buddy_block *block)
> +{
> + return gpu_buddy_block_state(block) == GPU_BUDDY_FREE;
> +}
> +
> +static inline bool
> +gpu_buddy_block_is_split(struct gpu_buddy_block *block)
> +{
> + return gpu_buddy_block_state(block) == GPU_BUDDY_SPLIT;
> +}
> +
> +static inline u64
> +gpu_buddy_block_size(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block)
> +{
> + return mm->chunk_size << gpu_buddy_block_order(block);
> +}
> +
> +int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size);
> +
> +void gpu_buddy_fini(struct gpu_buddy *mm);
> +
> +struct gpu_buddy_block *
> +gpu_get_buddy(struct gpu_buddy_block *block);
> +
> +int gpu_buddy_alloc_blocks(struct gpu_buddy *mm,
> + u64 start, u64 end, u64 size,
> + u64 min_page_size,
> + struct list_head *blocks,
> + unsigned long flags);
> +
> +int gpu_buddy_block_trim(struct gpu_buddy *mm,
> + u64 *start,
> + u64 new_size,
> + struct list_head *blocks);
> +
> +void gpu_buddy_reset_clear(struct gpu_buddy *mm, bool is_clear);
> +
> +void gpu_buddy_free_block(struct gpu_buddy *mm, struct gpu_buddy_block *block);
> +
> +void gpu_buddy_free_list(struct gpu_buddy *mm,
> + struct list_head *objects,
> + unsigned int flags);
> +
> +void gpu_buddy_print(struct gpu_buddy *mm);
> +void gpu_buddy_block_print(struct gpu_buddy *mm,
> + struct gpu_buddy_block *block);
> +#endif
> --
> 2.34.1
>
^ permalink raw reply
* [PATCH] staging: sm750fb: rename CamelCase variables in sm750_accel.c
From: Artem Lytkin @ 2026-02-05 23:48 UTC (permalink / raw)
To: Sudip Mukherjee, Teddy Wang, Greg Kroah-Hartman
Cc: linux-fbdev, linux-staging, linux-kernel, Artem Lytkin
Rename CamelCase function parameters and local variables to
snake_case to comply with kernel coding style:
sBase -> src_base dBase -> dst_base
sPitch -> src_pitch dPitch -> dst_pitch
Bpp -> bpp nDirection -> direction
pSrcbuf -> src_buf srcDelta -> src_delta
startBit -> start_bit bytePerPixel -> bytes_per_pixel
fColor -> fg_color bColor -> bg_color
ulBytesPerScan -> bytes_per_scan
ul4BytesPerScan -> bytes_per_scan_aligned
ulBytesRemain -> bytes_remain
ajRemain -> remain_buf
write_dpPort -> write_dp_port
Update the corresponding kernel-doc comments and function
declarations in sm750_accel.h.
Signed-off-by: Artem Lytkin <iprintercanon@gmail.com>
---
drivers/staging/sm750fb/sm750_accel.c | 134 +++++++++++++-------------
drivers/staging/sm750fb/sm750_accel.h | 46 ++++-----
2 files changed, 90 insertions(+), 90 deletions(-)
diff --git a/drivers/staging/sm750fb/sm750_accel.c b/drivers/staging/sm750fb/sm750_accel.c
index 046b9282b..ad055dfb3 100644
--- a/drivers/staging/sm750fb/sm750_accel.c
+++ b/drivers/staging/sm750fb/sm750_accel.c
@@ -27,7 +27,7 @@ static inline u32 read_dpr(struct lynx_accel *accel, int offset)
return readl(accel->dpr_base + offset);
}
-static inline void write_dpPort(struct lynx_accel *accel, u32 data)
+static inline void write_dp_port(struct lynx_accel *accel, u32 data)
{
writel(data, accel->dp_port_base);
}
@@ -85,7 +85,7 @@ void sm750_hw_set2dformat(struct lynx_accel *accel, int fmt)
}
int sm750_hw_fillrect(struct lynx_accel *accel,
- u32 base, u32 pitch, u32 Bpp,
+ u32 base, u32 pitch, u32 bpp,
u32 x, u32 y, u32 width, u32 height,
u32 color, u32 rop)
{
@@ -102,14 +102,14 @@ int sm750_hw_fillrect(struct lynx_accel *accel,
write_dpr(accel, DE_WINDOW_DESTINATION_BASE, base); /* dpr40 */
write_dpr(accel, DE_PITCH,
- ((pitch / Bpp << DE_PITCH_DESTINATION_SHIFT) &
+ ((pitch / bpp << DE_PITCH_DESTINATION_SHIFT) &
DE_PITCH_DESTINATION_MASK) |
- (pitch / Bpp & DE_PITCH_SOURCE_MASK)); /* dpr10 */
+ (pitch / bpp & DE_PITCH_SOURCE_MASK)); /* dpr10 */
write_dpr(accel, DE_WINDOW_WIDTH,
- ((pitch / Bpp << DE_WINDOW_WIDTH_DST_SHIFT) &
+ ((pitch / bpp << DE_WINDOW_WIDTH_DST_SHIFT) &
DE_WINDOW_WIDTH_DST_MASK) |
- (pitch / Bpp & DE_WINDOW_WIDTH_SRC_MASK)); /* dpr44 */
+ (pitch / bpp & DE_WINDOW_WIDTH_SRC_MASK)); /* dpr44 */
write_dpr(accel, DE_FOREGROUND, color); /* DPR14 */
@@ -132,13 +132,13 @@ int sm750_hw_fillrect(struct lynx_accel *accel,
/**
* sm750_hw_copyarea
* @accel: Acceleration device data
- * @sBase: Address of source: offset in frame buffer
- * @sPitch: Pitch value of source surface in BYTE
+ * @src_base: Address of source: offset in frame buffer
+ * @src_pitch: Pitch value of source surface in BYTE
* @sx: Starting x coordinate of source surface
* @sy: Starting y coordinate of source surface
- * @dBase: Address of destination: offset in frame buffer
- * @dPitch: Pitch value of destination surface in BYTE
- * @Bpp: Color depth of destination surface
+ * @dst_base: Address of destination: offset in frame buffer
+ * @dst_pitch: Pitch value of destination surface in BYTE
+ * @bpp: Color depth of destination surface
* @dx: Starting x coordinate of destination surface
* @dy: Starting y coordinate of destination surface
* @width: width of rectangle in pixel value
@@ -146,21 +146,21 @@ int sm750_hw_fillrect(struct lynx_accel *accel,
* @rop2: ROP value
*/
int sm750_hw_copyarea(struct lynx_accel *accel,
- unsigned int sBase, unsigned int sPitch,
+ unsigned int src_base, unsigned int src_pitch,
unsigned int sx, unsigned int sy,
- unsigned int dBase, unsigned int dPitch,
- unsigned int Bpp, unsigned int dx, unsigned int dy,
+ unsigned int dst_base, unsigned int dst_pitch,
+ unsigned int bpp, unsigned int dx, unsigned int dy,
unsigned int width, unsigned int height,
unsigned int rop2)
{
- unsigned int nDirection, de_ctrl;
+ unsigned int direction, de_ctrl;
- nDirection = LEFT_TO_RIGHT;
+ direction = LEFT_TO_RIGHT;
/* Direction of ROP2 operation: 1 = Left to Right, (-1) = Right to Left */
de_ctrl = 0;
/* If source and destination are the same surface, need to check for overlay cases */
- if (sBase == dBase && sPitch == dPitch) {
+ if (src_base == dst_base && src_pitch == dst_pitch) {
/* Determine direction of operation */
if (sy < dy) {
/* +----------+
@@ -173,7 +173,7 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* +----------+
*/
- nDirection = BOTTOM_TO_TOP;
+ direction = BOTTOM_TO_TOP;
} else if (sy > dy) {
/* +----------+
* |D |
@@ -185,7 +185,7 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* +----------+
*/
- nDirection = TOP_TO_BOTTOM;
+ direction = TOP_TO_BOTTOM;
} else {
/* sy == dy */
@@ -198,7 +198,7 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* +------+---+------+
*/
- nDirection = RIGHT_TO_LEFT;
+ direction = RIGHT_TO_LEFT;
} else {
/* sx > dx */
@@ -210,12 +210,12 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* +------+---+------+
*/
- nDirection = LEFT_TO_RIGHT;
+ direction = LEFT_TO_RIGHT;
}
}
}
- if ((nDirection == BOTTOM_TO_TOP) || (nDirection == RIGHT_TO_LEFT)) {
+ if ((direction == BOTTOM_TO_TOP) || (direction == RIGHT_TO_LEFT)) {
sx += width - 1;
sy += height - 1;
dx += width - 1;
@@ -234,14 +234,14 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* It is an address offset (128 bit aligned)
* from the beginning of frame buffer.
*/
- write_dpr(accel, DE_WINDOW_SOURCE_BASE, sBase); /* dpr40 */
+ write_dpr(accel, DE_WINDOW_SOURCE_BASE, src_base); /* dpr40 */
/*
* 2D Destination Base.
* It is an address offset (128 bit aligned)
* from the beginning of frame buffer.
*/
- write_dpr(accel, DE_WINDOW_DESTINATION_BASE, dBase); /* dpr44 */
+ write_dpr(accel, DE_WINDOW_DESTINATION_BASE, dst_base); /* dpr44 */
/*
* Program pitch (distance between the 1st points of two adjacent lines).
@@ -249,9 +249,9 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* pixel values. Need Byte to pixel conversion.
*/
write_dpr(accel, DE_PITCH,
- ((dPitch / Bpp << DE_PITCH_DESTINATION_SHIFT) &
+ ((dst_pitch / bpp << DE_PITCH_DESTINATION_SHIFT) &
DE_PITCH_DESTINATION_MASK) |
- (sPitch / Bpp & DE_PITCH_SOURCE_MASK)); /* dpr10 */
+ (src_pitch / bpp & DE_PITCH_SOURCE_MASK)); /* dpr10 */
/*
* Screen Window width in Pixels.
@@ -259,9 +259,9 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
* for a given point.
*/
write_dpr(accel, DE_WINDOW_WIDTH,
- ((dPitch / Bpp << DE_WINDOW_WIDTH_DST_SHIFT) &
+ ((dst_pitch / bpp << DE_WINDOW_WIDTH_DST_SHIFT) &
DE_WINDOW_WIDTH_DST_MASK) |
- (sPitch / Bpp & DE_WINDOW_WIDTH_SRC_MASK)); /* dpr3c */
+ (src_pitch / bpp & DE_WINDOW_WIDTH_SRC_MASK)); /* dpr3c */
if (accel->de_wait() != 0)
return -1;
@@ -277,7 +277,7 @@ int sm750_hw_copyarea(struct lynx_accel *accel,
(height & DE_DIMENSION_Y_ET_MASK)); /* dpr08 */
de_ctrl = (rop2 & DE_CONTROL_ROP_MASK) | DE_CONTROL_ROP_SELECT |
- ((nDirection == RIGHT_TO_LEFT) ? DE_CONTROL_DIRECTION : 0) |
+ ((direction == RIGHT_TO_LEFT) ? DE_CONTROL_DIRECTION : 0) |
DE_CONTROL_COMMAND_BITBLT | DE_CONTROL_STATUS;
write_dpr(accel, DE_CONTROL, de_ctrl); /* dpr0c */
@@ -299,38 +299,38 @@ static unsigned int de_get_transparency(struct lynx_accel *accel)
/**
* sm750_hw_imageblit
* @accel: Acceleration device data
- * @pSrcbuf: pointer to start of source buffer in system memory
- * @srcDelta: Pitch value (in bytes) of the source buffer, +ive means top down
- * and -ive mean button up
- * @startBit: Mono data can start at any bit in a byte, this value should be
- * 0 to 7
- * @dBase: Address of destination: offset in frame buffer
- * @dPitch: Pitch value of destination surface in BYTE
- * @bytePerPixel: Color depth of destination surface
+ * @src_buf: pointer to start of source buffer in system memory
+ * @src_delta: Pitch value (in bytes) of the source buffer, +ive means top down
+ * and -ive mean button up
+ * @start_bit: Mono data can start at any bit in a byte, this value should be
+ * 0 to 7
+ * @dst_base: Address of destination: offset in frame buffer
+ * @dst_pitch: Pitch value of destination surface in BYTE
+ * @bytes_per_pixel: Color depth of destination surface
* @dx: Starting x coordinate of destination surface
* @dy: Starting y coordinate of destination surface
* @width: width of rectangle in pixel value
* @height: height of rectangle in pixel value
- * @fColor: Foreground color (corresponding to a 1 in the monochrome data
- * @bColor: Background color (corresponding to a 0 in the monochrome data
+ * @fg_color: Foreground color (corresponding to a 1 in the monochrome data
+ * @bg_color: Background color (corresponding to a 0 in the monochrome data
* @rop2: ROP value
*/
-int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
- u32 srcDelta, u32 startBit, u32 dBase, u32 dPitch,
- u32 bytePerPixel, u32 dx, u32 dy, u32 width,
- u32 height, u32 fColor, u32 bColor, u32 rop2)
+int sm750_hw_imageblit(struct lynx_accel *accel, const char *src_buf,
+ u32 src_delta, u32 start_bit, u32 dst_base, u32 dst_pitch,
+ u32 bytes_per_pixel, u32 dx, u32 dy, u32 width,
+ u32 height, u32 fg_color, u32 bg_color, u32 rop2)
{
- unsigned int ulBytesPerScan;
- unsigned int ul4BytesPerScan;
- unsigned int ulBytesRemain;
+ unsigned int bytes_per_scan;
+ unsigned int bytes_per_scan_aligned;
+ unsigned int bytes_remain;
unsigned int de_ctrl = 0;
- unsigned char ajRemain[4];
+ unsigned char remain_buf[4];
int i, j;
- startBit &= 7; /* Just make sure the start bit is within legal range */
- ulBytesPerScan = (width + startBit + 7) / 8;
- ul4BytesPerScan = ulBytesPerScan & ~3;
- ulBytesRemain = ulBytesPerScan & 3;
+ start_bit &= 7; /* Just make sure the start bit is within legal range */
+ bytes_per_scan = (width + start_bit + 7) / 8;
+ bytes_per_scan_aligned = bytes_per_scan & ~3;
+ bytes_remain = bytes_per_scan & 3;
if (accel->de_wait() != 0)
return -1;
@@ -345,7 +345,7 @@ int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
* It is an address offset (128 bit aligned)
* from the beginning of frame buffer.
*/
- write_dpr(accel, DE_WINDOW_DESTINATION_BASE, dBase);
+ write_dpr(accel, DE_WINDOW_DESTINATION_BASE, dst_base);
/*
* Program pitch (distance between the 1st points of two adjacent
@@ -353,9 +353,9 @@ int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
* register uses pixel values. Need Byte to pixel conversion.
*/
write_dpr(accel, DE_PITCH,
- ((dPitch / bytePerPixel << DE_PITCH_DESTINATION_SHIFT) &
+ ((dst_pitch / bytes_per_pixel << DE_PITCH_DESTINATION_SHIFT) &
DE_PITCH_DESTINATION_MASK) |
- (dPitch / bytePerPixel & DE_PITCH_SOURCE_MASK)); /* dpr10 */
+ (dst_pitch / bytes_per_pixel & DE_PITCH_SOURCE_MASK)); /* dpr10 */
/*
* Screen Window width in Pixels.
@@ -363,17 +363,17 @@ int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
* in frame buffer for a given point.
*/
write_dpr(accel, DE_WINDOW_WIDTH,
- ((dPitch / bytePerPixel << DE_WINDOW_WIDTH_DST_SHIFT) &
+ ((dst_pitch / bytes_per_pixel << DE_WINDOW_WIDTH_DST_SHIFT) &
DE_WINDOW_WIDTH_DST_MASK) |
- (dPitch / bytePerPixel & DE_WINDOW_WIDTH_SRC_MASK));
+ (dst_pitch / bytes_per_pixel & DE_WINDOW_WIDTH_SRC_MASK));
/*
* Note: For 2D Source in Host Write, only X_K1_MONO field is needed,
* and Y_K2 field is not used.
- * For mono bitmap, use startBit for X_K1.
+ * For mono bitmap, use start_bit for X_K1.
*/
write_dpr(accel, DE_SOURCE,
- (startBit << DE_SOURCE_X_K1_SHIFT) &
+ (start_bit << DE_SOURCE_X_K1_SHIFT) &
DE_SOURCE_X_K1_MONO_MASK); /* dpr00 */
write_dpr(accel, DE_DESTINATION,
@@ -384,8 +384,8 @@ int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
((width << DE_DIMENSION_X_SHIFT) & DE_DIMENSION_X_MASK) |
(height & DE_DIMENSION_Y_ET_MASK)); /* dpr08 */
- write_dpr(accel, DE_FOREGROUND, fColor);
- write_dpr(accel, DE_BACKGROUND, bColor);
+ write_dpr(accel, DE_FOREGROUND, fg_color);
+ write_dpr(accel, DE_BACKGROUND, bg_color);
de_ctrl = (rop2 & DE_CONTROL_ROP_MASK) |
DE_CONTROL_ROP_SELECT | DE_CONTROL_COMMAND_HOST_WRITE |
@@ -396,16 +396,16 @@ int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
/* Write MONO data (line by line) to 2D Engine data port */
for (i = 0; i < height; i++) {
/* For each line, send the data in chunks of 4 bytes */
- for (j = 0; j < (ul4BytesPerScan / 4); j++)
- write_dpPort(accel, *(unsigned int *)(pSrcbuf + (j * 4)));
+ for (j = 0; j < (bytes_per_scan_aligned / 4); j++)
+ write_dp_port(accel, *(unsigned int *)(src_buf + (j * 4)));
- if (ulBytesRemain) {
- memcpy(ajRemain, pSrcbuf + ul4BytesPerScan,
- ulBytesRemain);
- write_dpPort(accel, *(unsigned int *)ajRemain);
+ if (bytes_remain) {
+ memcpy(remain_buf, src_buf + bytes_per_scan_aligned,
+ bytes_remain);
+ write_dp_port(accel, *(unsigned int *)remain_buf);
}
- pSrcbuf += srcDelta;
+ src_buf += src_delta;
}
return 0;
diff --git a/drivers/staging/sm750fb/sm750_accel.h b/drivers/staging/sm750fb/sm750_accel.h
index 2c79cb730..b0ee6ade3 100644
--- a/drivers/staging/sm750fb/sm750_accel.h
+++ b/drivers/staging/sm750fb/sm750_accel.h
@@ -190,19 +190,19 @@ void sm750_hw_set2dformat(struct lynx_accel *accel, int fmt);
void sm750_hw_de_init(struct lynx_accel *accel);
int sm750_hw_fillrect(struct lynx_accel *accel,
- u32 base, u32 pitch, u32 Bpp,
+ u32 base, u32 pitch, u32 bpp,
u32 x, u32 y, u32 width, u32 height,
u32 color, u32 rop);
/**
* sm750_hm_copyarea
- * @sBase: Address of source: offset in frame buffer
- * @sPitch: Pitch value of source surface in BYTE
+ * @src_base: Address of source: offset in frame buffer
+ * @src_pitch: Pitch value of source surface in BYTE
* @sx: Starting x coordinate of source surface
* @sy: Starting y coordinate of source surface
- * @dBase: Address of destination: offset in frame buffer
- * @dPitch: Pitch value of destination surface in BYTE
- * @Bpp: Color depth of destination surface
+ * @dst_base: Address of destination: offset in frame buffer
+ * @dst_pitch: Pitch value of destination surface in BYTE
+ * @bpp: Color depth of destination surface
* @dx: Starting x coordinate of destination surface
* @dy: Starting y coordinate of destination surface
* @width: width of rectangle in pixel value
@@ -210,34 +210,34 @@ int sm750_hw_fillrect(struct lynx_accel *accel,
* @rop2: ROP value
*/
int sm750_hw_copyarea(struct lynx_accel *accel,
- unsigned int sBase, unsigned int sPitch,
+ unsigned int src_base, unsigned int src_pitch,
unsigned int sx, unsigned int sy,
- unsigned int dBase, unsigned int dPitch,
- unsigned int Bpp, unsigned int dx, unsigned int dy,
+ unsigned int dst_base, unsigned int dst_pitch,
+ unsigned int bpp, unsigned int dx, unsigned int dy,
unsigned int width, unsigned int height,
unsigned int rop2);
/**
* sm750_hw_imageblit
- * @pSrcbuf: pointer to start of source buffer in system memory
- * @srcDelta: Pitch value (in bytes) of the source buffer, +ive means top down
- *>----- and -ive mean button up
- * @startBit: Mono data can start at any bit in a byte, this value should be
- *>----- 0 to 7
- * @dBase: Address of destination: offset in frame buffer
- * @dPitch: Pitch value of destination surface in BYTE
- * @bytePerPixel: Color depth of destination surface
+ * @src_buf: pointer to start of source buffer in system memory
+ * @src_delta: Pitch value (in bytes) of the source buffer, +ive means top down
+ * and -ive mean button up
+ * @start_bit: Mono data can start at any bit in a byte, this value should be
+ * 0 to 7
+ * @dst_base: Address of destination: offset in frame buffer
+ * @dst_pitch: Pitch value of destination surface in BYTE
+ * @bytes_per_pixel: Color depth of destination surface
* @dx: Starting x coordinate of destination surface
* @dy: Starting y coordinate of destination surface
* @width: width of rectangle in pixel value
* @height: height of rectangle in pixel value
- * @fColor: Foreground color (corresponding to a 1 in the monochrome data
- * @bColor: Background color (corresponding to a 0 in the monochrome data
+ * @fg_color: Foreground color (corresponding to a 1 in the monochrome data
+ * @bg_color: Background color (corresponding to a 0 in the monochrome data
* @rop2: ROP value
*/
-int sm750_hw_imageblit(struct lynx_accel *accel, const char *pSrcbuf,
- u32 srcDelta, u32 startBit, u32 dBase, u32 dPitch,
- u32 bytePerPixel, u32 dx, u32 dy, u32 width,
- u32 height, u32 fColor, u32 bColor, u32 rop2);
+int sm750_hw_imageblit(struct lynx_accel *accel, const char *src_buf,
+ u32 src_delta, u32 start_bit, u32 dst_base, u32 dst_pitch,
+ u32 bytes_per_pixel, u32 dx, u32 dy, u32 width,
+ u32 height, u32 fg_color, u32 bg_color, u32 rop2);
#endif
--
2.43.0
^ permalink raw reply related
* Re: [PATCH v5] fbtft: limit dirty rows based on damage range
From: Nam Cao @ 2026-02-06 0:04 UTC (permalink / raw)
To: ChanSoo Shin, andy
Cc: gregkh, dri-devel, linux-fbdev, linux-staging, ChanSoo Shin
In-Reply-To: <20260128203938.962414-1-csshin9928@gmail.com>
Hi ChanSoo,
ChanSoo Shin <csshin9928@gmail.com> writes:
> Instead of marking the entire display as dirty, calculate the start
> and end rows based on the damage offset and length and only mark the
> affected rows dirty. This reduces unnecessary full framebuffer updates
> for partial writes.
>
> Signed-off-by: ChanSoo Shin <csshin9928@gmail.com>
This looks useful and I prefer to see it applied. I understand lack of
testing is the main obstacle. Are you still trying to get it merged? If
not, I can take over.
Nam
^ permalink raw reply
* [PATCH -next] gpu: Move DRM buddy allocator one level up
From: Joel Fernandes @ 2026-02-06 0:34 UTC (permalink / raw)
To: linux-kernel
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Alexey Ivanov, Balbir Singh,
Philipp Stanner, Elle Rhumsaa, Daniel Almeida, joel, nouveau,
dri-devel, rust-for-linux, linux-doc, amd-gfx, intel-gfx,
intel-xe, linux-fbdev, Joel Fernandes
Move the DRM buddy allocator one level up so that it can be used by GPU
drivers (example, nova-core) that have usecases other than DRM (such as
VFIO vGPU support). Modify the API, structures and Kconfigs to use
"gpu_buddy" terminology. Adapt the drivers and tests to use the new API.
The commit cannot be split due to bisectability, however no functional
change is intended. Verified by running K-UNIT tests and build tested
various configurations.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
Rebased on today's linux-next after discussing with Danilo.
Documentation/gpu/drm-mm.rst | 10 +-
drivers/gpu/Kconfig | 13 +
drivers/gpu/Makefile | 2 +
drivers/gpu/buddy.c | 1320 +++++++++++++++++
drivers/gpu/drm/Kconfig | 1 +
drivers/gpu/drm/Kconfig.debug | 4 +-
drivers/gpu/drm/amd/amdgpu/Kconfig | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 2 +-
.../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h | 12 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 80 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h | 20 +-
drivers/gpu/drm/drm_buddy.c | 1294 +---------------
drivers/gpu/drm/i915/Kconfig | 1 +
drivers/gpu/drm/i915/i915_scatterlist.c | 10 +-
drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 55 +-
drivers/gpu/drm/i915/i915_ttm_buddy_manager.h | 6 +-
.../drm/i915/selftests/intel_memory_region.c | 20 +-
drivers/gpu/drm/tests/Makefile | 1 -
.../gpu/drm/ttm/tests/ttm_bo_validate_test.c | 5 +-
drivers/gpu/drm/ttm/tests/ttm_mock_manager.c | 18 +-
drivers/gpu/drm/ttm/tests/ttm_mock_manager.h | 4 +-
drivers/gpu/drm/xe/Kconfig | 1 +
drivers/gpu/drm/xe/xe_res_cursor.h | 34 +-
drivers/gpu/drm/xe/xe_svm.c | 12 +-
drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 73 +-
drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h | 4 +-
drivers/gpu/tests/Makefile | 3 +
.../gpu_buddy_test.c} | 416 +++---
drivers/gpu/tests/gpu_random.c | 48 +
drivers/gpu/tests/gpu_random.h | 28 +
drivers/video/Kconfig | 2 +
include/drm/drm_buddy.h | 163 +-
include/linux/gpu_buddy.h | 177 +++
33 files changed, 2018 insertions(+), 1822 deletions(-)
create mode 100644 drivers/gpu/Kconfig
create mode 100644 drivers/gpu/buddy.c
create mode 100644 drivers/gpu/tests/Makefile
rename drivers/gpu/{drm/tests/drm_buddy_test.c => tests/gpu_buddy_test.c} (67%)
create mode 100644 drivers/gpu/tests/gpu_random.c
create mode 100644 drivers/gpu/tests/gpu_random.h
create mode 100644 include/linux/gpu_buddy.h
diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst
index f22433470c76..32fb506db05b 100644
--- a/Documentation/gpu/drm-mm.rst
+++ b/Documentation/gpu/drm-mm.rst
@@ -526,8 +526,14 @@ DRM GPUVM Function References
DRM Buddy Allocator
===================
-DRM Buddy Function References
------------------------------
+Buddy Allocator Function References (GPU buddy)
+-----------------------------------------------
+
+.. kernel-doc:: drivers/gpu/buddy.c
+ :export:
+
+DRM Buddy Specific Logging Function References
+----------------------------------------------
.. kernel-doc:: drivers/gpu/drm/drm_buddy.c
:export:
diff --git a/drivers/gpu/Kconfig b/drivers/gpu/Kconfig
new file mode 100644
index 000000000000..22dd29cd50b5
--- /dev/null
+++ b/drivers/gpu/Kconfig
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config GPU_BUDDY
+ bool
+ help
+ A page based buddy allocator for GPU memory.
+
+config GPU_BUDDY_KUNIT_TEST
+ tristate "KUnit tests for GPU buddy allocator" if !KUNIT_ALL_TESTS
+ depends on GPU_BUDDY && KUNIT
+ default KUNIT_ALL_TESTS
+ help
+ KUnit tests for the GPU buddy allocator.
diff --git a/drivers/gpu/Makefile b/drivers/gpu/Makefile
index 36a54d456630..5063caccabdf 100644
--- a/drivers/gpu/Makefile
+++ b/drivers/gpu/Makefile
@@ -6,3 +6,5 @@ obj-y += host1x/ drm/ vga/
obj-$(CONFIG_IMX_IPUV3_CORE) += ipu-v3/
obj-$(CONFIG_TRACE_GPU_MEM) += trace/
obj-$(CONFIG_NOVA_CORE) += nova-core/
+obj-$(CONFIG_GPU_BUDDY) += buddy.o
+obj-y += tests/
diff --git a/drivers/gpu/buddy.c b/drivers/gpu/buddy.c
new file mode 100644
index 000000000000..239e62118f8d
--- /dev/null
+++ b/drivers/gpu/buddy.c
@@ -0,0 +1,1320 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#include <kunit/test-bug.h>
+
+#include <linux/export.h>
+#include <linux/gpu_buddy.h>
+#include <linux/kmemleak.h>
+#include <linux/module.h>
+#include <linux/sizes.h>
+
+static struct kmem_cache *slab_blocks;
+
+static struct gpu_buddy_block *gpu_block_alloc(struct gpu_buddy *mm,
+ struct gpu_buddy_block *parent,
+ unsigned int order,
+ u64 offset)
+{
+ struct gpu_buddy_block *block;
+
+ BUG_ON(order > GPU_BUDDY_MAX_ORDER);
+
+ block = kmem_cache_zalloc(slab_blocks, GFP_KERNEL);
+ if (!block)
+ return NULL;
+
+ block->header = offset;
+ block->header |= order;
+ block->parent = parent;
+
+ RB_CLEAR_NODE(&block->rb);
+
+ BUG_ON(block->header & GPU_BUDDY_HEADER_UNUSED);
+ return block;
+}
+
+static void gpu_block_free(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ kmem_cache_free(slab_blocks, block);
+}
+
+static enum gpu_buddy_free_tree
+get_block_tree(struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_is_clear(block) ?
+ GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE;
+}
+
+static struct gpu_buddy_block *
+rbtree_get_free_block(const struct rb_node *node)
+{
+ return node ? rb_entry(node, struct gpu_buddy_block, rb) : NULL;
+}
+
+static struct gpu_buddy_block *
+rbtree_last_free_block(struct rb_root *root)
+{
+ return rbtree_get_free_block(rb_last(root));
+}
+
+static bool rbtree_is_empty(struct rb_root *root)
+{
+ return RB_EMPTY_ROOT(root);
+}
+
+static bool gpu_buddy_block_offset_less(const struct gpu_buddy_block *block,
+ const struct gpu_buddy_block *node)
+{
+ return gpu_buddy_block_offset(block) < gpu_buddy_block_offset(node);
+}
+
+static bool rbtree_block_offset_less(struct rb_node *block,
+ const struct rb_node *node)
+{
+ return gpu_buddy_block_offset_less(rbtree_get_free_block(block),
+ rbtree_get_free_block(node));
+}
+
+static void rbtree_insert(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block,
+ enum gpu_buddy_free_tree tree)
+{
+ rb_add(&block->rb,
+ &mm->free_trees[tree][gpu_buddy_block_order(block)],
+ rbtree_block_offset_less);
+}
+
+static void rbtree_remove(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ unsigned int order = gpu_buddy_block_order(block);
+ enum gpu_buddy_free_tree tree;
+ struct rb_root *root;
+
+ tree = get_block_tree(block);
+ root = &mm->free_trees[tree][order];
+
+ rb_erase(&block->rb, root);
+ RB_CLEAR_NODE(&block->rb);
+}
+
+static void clear_reset(struct gpu_buddy_block *block)
+{
+ block->header &= ~GPU_BUDDY_HEADER_CLEAR;
+}
+
+static void mark_cleared(struct gpu_buddy_block *block)
+{
+ block->header |= GPU_BUDDY_HEADER_CLEAR;
+}
+
+static void mark_allocated(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ block->header &= ~GPU_BUDDY_HEADER_STATE;
+ block->header |= GPU_BUDDY_ALLOCATED;
+
+ rbtree_remove(mm, block);
+}
+
+static void mark_free(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ enum gpu_buddy_free_tree tree;
+
+ block->header &= ~GPU_BUDDY_HEADER_STATE;
+ block->header |= GPU_BUDDY_FREE;
+
+ tree = get_block_tree(block);
+ rbtree_insert(mm, block, tree);
+}
+
+static void mark_split(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ block->header &= ~GPU_BUDDY_HEADER_STATE;
+ block->header |= GPU_BUDDY_SPLIT;
+
+ rbtree_remove(mm, block);
+}
+
+static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2)
+{
+ return s1 <= e2 && e1 >= s2;
+}
+
+static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2)
+{
+ return s1 <= s2 && e1 >= e2;
+}
+
+static struct gpu_buddy_block *
+__get_buddy(struct gpu_buddy_block *block)
+{
+ struct gpu_buddy_block *parent;
+
+ parent = block->parent;
+ if (!parent)
+ return NULL;
+
+ if (parent->left == block)
+ return parent->right;
+
+ return parent->left;
+}
+
+static unsigned int __gpu_buddy_free(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block,
+ bool force_merge)
+{
+ struct gpu_buddy_block *parent;
+ unsigned int order;
+
+ while ((parent = block->parent)) {
+ struct gpu_buddy_block *buddy;
+
+ buddy = __get_buddy(block);
+
+ if (!gpu_buddy_block_is_free(buddy))
+ break;
+
+ if (!force_merge) {
+ /*
+ * Check the block and its buddy clear state and exit
+ * the loop if they both have the dissimilar state.
+ */
+ if (gpu_buddy_block_is_clear(block) !=
+ gpu_buddy_block_is_clear(buddy))
+ break;
+
+ if (gpu_buddy_block_is_clear(block))
+ mark_cleared(parent);
+ }
+
+ rbtree_remove(mm, buddy);
+ if (force_merge && gpu_buddy_block_is_clear(buddy))
+ mm->clear_avail -= gpu_buddy_block_size(mm, buddy);
+
+ gpu_block_free(mm, block);
+ gpu_block_free(mm, buddy);
+
+ block = parent;
+ }
+
+ order = gpu_buddy_block_order(block);
+ mark_free(mm, block);
+
+ return order;
+}
+
+static int __force_merge(struct gpu_buddy *mm,
+ u64 start,
+ u64 end,
+ unsigned int min_order)
+{
+ unsigned int tree, order;
+ int i;
+
+ if (!min_order)
+ return -ENOMEM;
+
+ if (min_order > mm->max_order)
+ return -EINVAL;
+
+ for_each_free_tree(tree) {
+ for (i = min_order - 1; i >= 0; i--) {
+ struct rb_node *iter = rb_last(&mm->free_trees[tree][i]);
+
+ while (iter) {
+ struct gpu_buddy_block *block, *buddy;
+ u64 block_start, block_end;
+
+ block = rbtree_get_free_block(iter);
+ iter = rb_prev(iter);
+
+ if (!block || !block->parent)
+ continue;
+
+ block_start = gpu_buddy_block_offset(block);
+ block_end = block_start + gpu_buddy_block_size(mm, block) - 1;
+
+ if (!contains(start, end, block_start, block_end))
+ continue;
+
+ buddy = __get_buddy(block);
+ if (!gpu_buddy_block_is_free(buddy))
+ continue;
+
+ WARN_ON(gpu_buddy_block_is_clear(block) ==
+ gpu_buddy_block_is_clear(buddy));
+
+ /*
+ * Advance to the next node when the current node is the buddy,
+ * as freeing the block will also remove its buddy from the tree.
+ */
+ if (iter == &buddy->rb)
+ iter = rb_prev(iter);
+
+ rbtree_remove(mm, block);
+ if (gpu_buddy_block_is_clear(block))
+ mm->clear_avail -= gpu_buddy_block_size(mm, block);
+
+ order = __gpu_buddy_free(mm, block, true);
+ if (order >= min_order)
+ return 0;
+ }
+ }
+ }
+
+ return -ENOMEM;
+}
+
+/**
+ * gpu_buddy_init - init memory manager
+ *
+ * @mm: GPU buddy manager to initialize
+ * @size: size in bytes to manage
+ * @chunk_size: minimum page size in bytes for our allocations
+ *
+ * Initializes the memory manager and its resources.
+ *
+ * Returns:
+ * 0 on success, error code on failure.
+ */
+int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size)
+{
+ unsigned int i, j, root_count = 0;
+ u64 offset = 0;
+
+ if (size < chunk_size)
+ return -EINVAL;
+
+ if (chunk_size < SZ_4K)
+ return -EINVAL;
+
+ if (!is_power_of_2(chunk_size))
+ return -EINVAL;
+
+ size = round_down(size, chunk_size);
+
+ mm->size = size;
+ mm->avail = size;
+ mm->clear_avail = 0;
+ mm->chunk_size = chunk_size;
+ mm->max_order = ilog2(size) - ilog2(chunk_size);
+
+ BUG_ON(mm->max_order > GPU_BUDDY_MAX_ORDER);
+
+ mm->free_trees = kmalloc_array(GPU_BUDDY_MAX_FREE_TREES,
+ sizeof(*mm->free_trees),
+ GFP_KERNEL);
+ if (!mm->free_trees)
+ return -ENOMEM;
+
+ for_each_free_tree(i) {
+ mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
+ sizeof(struct rb_root),
+ GFP_KERNEL);
+ if (!mm->free_trees[i])
+ goto out_free_tree;
+
+ for (j = 0; j <= mm->max_order; ++j)
+ mm->free_trees[i][j] = RB_ROOT;
+ }
+
+ mm->n_roots = hweight64(size);
+
+ mm->roots = kmalloc_array(mm->n_roots,
+ sizeof(struct gpu_buddy_block *),
+ GFP_KERNEL);
+ if (!mm->roots)
+ goto out_free_tree;
+
+ /*
+ * Split into power-of-two blocks, in case we are given a size that is
+ * not itself a power-of-two.
+ */
+ do {
+ struct gpu_buddy_block *root;
+ unsigned int order;
+ u64 root_size;
+
+ order = ilog2(size) - ilog2(chunk_size);
+ root_size = chunk_size << order;
+
+ root = gpu_block_alloc(mm, NULL, order, offset);
+ if (!root)
+ goto out_free_roots;
+
+ mark_free(mm, root);
+
+ BUG_ON(root_count > mm->max_order);
+ BUG_ON(gpu_buddy_block_size(mm, root) < chunk_size);
+
+ mm->roots[root_count] = root;
+
+ offset += root_size;
+ size -= root_size;
+ root_count++;
+ } while (size);
+
+ return 0;
+
+out_free_roots:
+ while (root_count--)
+ gpu_block_free(mm, mm->roots[root_count]);
+ kfree(mm->roots);
+out_free_tree:
+ while (i--)
+ kfree(mm->free_trees[i]);
+ kfree(mm->free_trees);
+ return -ENOMEM;
+}
+EXPORT_SYMBOL(gpu_buddy_init);
+
+/**
+ * gpu_buddy_fini - tear down the memory manager
+ *
+ * @mm: GPU buddy manager to free
+ *
+ * Cleanup memory manager resources and the freetree
+ */
+void gpu_buddy_fini(struct gpu_buddy *mm)
+{
+ u64 root_size, size, start;
+ unsigned int order;
+ int i;
+
+ size = mm->size;
+
+ for (i = 0; i < mm->n_roots; ++i) {
+ order = ilog2(size) - ilog2(mm->chunk_size);
+ start = gpu_buddy_block_offset(mm->roots[i]);
+ __force_merge(mm, start, start + size, order);
+
+ if (WARN_ON(!gpu_buddy_block_is_free(mm->roots[i])))
+ kunit_fail_current_test("buddy_fini() root");
+
+ gpu_block_free(mm, mm->roots[i]);
+
+ root_size = mm->chunk_size << order;
+ size -= root_size;
+ }
+
+ WARN_ON(mm->avail != mm->size);
+
+ for_each_free_tree(i)
+ kfree(mm->free_trees[i]);
+ kfree(mm->free_trees);
+ kfree(mm->roots);
+}
+EXPORT_SYMBOL(gpu_buddy_fini);
+
+static int split_block(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ unsigned int block_order = gpu_buddy_block_order(block) - 1;
+ u64 offset = gpu_buddy_block_offset(block);
+
+ BUG_ON(!gpu_buddy_block_is_free(block));
+ BUG_ON(!gpu_buddy_block_order(block));
+
+ block->left = gpu_block_alloc(mm, block, block_order, offset);
+ if (!block->left)
+ return -ENOMEM;
+
+ block->right = gpu_block_alloc(mm, block, block_order,
+ offset + (mm->chunk_size << block_order));
+ if (!block->right) {
+ gpu_block_free(mm, block->left);
+ return -ENOMEM;
+ }
+
+ mark_split(mm, block);
+
+ if (gpu_buddy_block_is_clear(block)) {
+ mark_cleared(block->left);
+ mark_cleared(block->right);
+ clear_reset(block);
+ }
+
+ mark_free(mm, block->left);
+ mark_free(mm, block->right);
+
+ return 0;
+}
+
+/**
+ * gpu_get_buddy - get buddy address
+ *
+ * @block: GPU buddy block
+ *
+ * Returns the corresponding buddy block for @block, or NULL
+ * if this is a root block and can't be merged further.
+ * Requires some kind of locking to protect against
+ * any concurrent allocate and free operations.
+ */
+struct gpu_buddy_block *
+gpu_get_buddy(struct gpu_buddy_block *block)
+{
+ return __get_buddy(block);
+}
+EXPORT_SYMBOL(gpu_get_buddy);
+
+/**
+ * gpu_buddy_reset_clear - reset blocks clear state
+ *
+ * @mm: GPU buddy manager
+ * @is_clear: blocks clear state
+ *
+ * Reset the clear state based on @is_clear value for each block
+ * in the freetree.
+ */
+void gpu_buddy_reset_clear(struct gpu_buddy *mm, bool is_clear)
+{
+ enum gpu_buddy_free_tree src_tree, dst_tree;
+ u64 root_size, size, start;
+ unsigned int order;
+ int i;
+
+ size = mm->size;
+ for (i = 0; i < mm->n_roots; ++i) {
+ order = ilog2(size) - ilog2(mm->chunk_size);
+ start = gpu_buddy_block_offset(mm->roots[i]);
+ __force_merge(mm, start, start + size, order);
+
+ root_size = mm->chunk_size << order;
+ size -= root_size;
+ }
+
+ src_tree = is_clear ? GPU_BUDDY_DIRTY_TREE : GPU_BUDDY_CLEAR_TREE;
+ dst_tree = is_clear ? GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE;
+
+ for (i = 0; i <= mm->max_order; ++i) {
+ struct rb_root *root = &mm->free_trees[src_tree][i];
+ struct gpu_buddy_block *block, *tmp;
+
+ rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
+ rbtree_remove(mm, block);
+ if (is_clear) {
+ mark_cleared(block);
+ mm->clear_avail += gpu_buddy_block_size(mm, block);
+ } else {
+ clear_reset(block);
+ mm->clear_avail -= gpu_buddy_block_size(mm, block);
+ }
+
+ rbtree_insert(mm, block, dst_tree);
+ }
+ }
+}
+EXPORT_SYMBOL(gpu_buddy_reset_clear);
+
+/**
+ * gpu_buddy_free_block - free a block
+ *
+ * @mm: GPU buddy manager
+ * @block: block to be freed
+ */
+void gpu_buddy_free_block(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ BUG_ON(!gpu_buddy_block_is_allocated(block));
+ mm->avail += gpu_buddy_block_size(mm, block);
+ if (gpu_buddy_block_is_clear(block))
+ mm->clear_avail += gpu_buddy_block_size(mm, block);
+
+ __gpu_buddy_free(mm, block, false);
+}
+EXPORT_SYMBOL(gpu_buddy_free_block);
+
+static void __gpu_buddy_free_list(struct gpu_buddy *mm,
+ struct list_head *objects,
+ bool mark_clear,
+ bool mark_dirty)
+{
+ struct gpu_buddy_block *block, *on;
+
+ WARN_ON(mark_dirty && mark_clear);
+
+ list_for_each_entry_safe(block, on, objects, link) {
+ if (mark_clear)
+ mark_cleared(block);
+ else if (mark_dirty)
+ clear_reset(block);
+ gpu_buddy_free_block(mm, block);
+ cond_resched();
+ }
+ INIT_LIST_HEAD(objects);
+}
+
+static void gpu_buddy_free_list_internal(struct gpu_buddy *mm,
+ struct list_head *objects)
+{
+ /*
+ * Don't touch the clear/dirty bit, since allocation is still internal
+ * at this point. For example we might have just failed part of the
+ * allocation.
+ */
+ __gpu_buddy_free_list(mm, objects, false, false);
+}
+
+/**
+ * gpu_buddy_free_list - free blocks
+ *
+ * @mm: GPU buddy manager
+ * @objects: input list head to free blocks
+ * @flags: optional flags like GPU_BUDDY_CLEARED
+ */
+void gpu_buddy_free_list(struct gpu_buddy *mm,
+ struct list_head *objects,
+ unsigned int flags)
+{
+ bool mark_clear = flags & GPU_BUDDY_CLEARED;
+
+ __gpu_buddy_free_list(mm, objects, mark_clear, !mark_clear);
+}
+EXPORT_SYMBOL(gpu_buddy_free_list);
+
+static bool block_incompatible(struct gpu_buddy_block *block, unsigned int flags)
+{
+ bool needs_clear = flags & GPU_BUDDY_CLEAR_ALLOCATION;
+
+ return needs_clear != gpu_buddy_block_is_clear(block);
+}
+
+static struct gpu_buddy_block *
+__alloc_range_bias(struct gpu_buddy *mm,
+ u64 start, u64 end,
+ unsigned int order,
+ unsigned long flags,
+ bool fallback)
+{
+ u64 req_size = mm->chunk_size << order;
+ struct gpu_buddy_block *block;
+ struct gpu_buddy_block *buddy;
+ LIST_HEAD(dfs);
+ int err;
+ int i;
+
+ end = end - 1;
+
+ for (i = 0; i < mm->n_roots; ++i)
+ list_add_tail(&mm->roots[i]->tmp_link, &dfs);
+
+ do {
+ u64 block_start;
+ u64 block_end;
+
+ block = list_first_entry_or_null(&dfs,
+ struct gpu_buddy_block,
+ tmp_link);
+ if (!block)
+ break;
+
+ list_del(&block->tmp_link);
+
+ if (gpu_buddy_block_order(block) < order)
+ continue;
+
+ block_start = gpu_buddy_block_offset(block);
+ block_end = block_start + gpu_buddy_block_size(mm, block) - 1;
+
+ if (!overlaps(start, end, block_start, block_end))
+ continue;
+
+ if (gpu_buddy_block_is_allocated(block))
+ continue;
+
+ if (block_start < start || block_end > end) {
+ u64 adjusted_start = max(block_start, start);
+ u64 adjusted_end = min(block_end, end);
+
+ if (round_down(adjusted_end + 1, req_size) <=
+ round_up(adjusted_start, req_size))
+ continue;
+ }
+
+ if (!fallback && block_incompatible(block, flags))
+ continue;
+
+ if (contains(start, end, block_start, block_end) &&
+ order == gpu_buddy_block_order(block)) {
+ /*
+ * Find the free block within the range.
+ */
+ if (gpu_buddy_block_is_free(block))
+ return block;
+
+ continue;
+ }
+
+ if (!gpu_buddy_block_is_split(block)) {
+ err = split_block(mm, block);
+ if (unlikely(err))
+ goto err_undo;
+ }
+
+ list_add(&block->right->tmp_link, &dfs);
+ list_add(&block->left->tmp_link, &dfs);
+ } while (1);
+
+ return ERR_PTR(-ENOSPC);
+
+err_undo:
+ /*
+ * We really don't want to leave around a bunch of split blocks, since
+ * bigger is better, so make sure we merge everything back before we
+ * free the allocated blocks.
+ */
+ buddy = __get_buddy(block);
+ if (buddy &&
+ (gpu_buddy_block_is_free(block) &&
+ gpu_buddy_block_is_free(buddy)))
+ __gpu_buddy_free(mm, block, false);
+ return ERR_PTR(err);
+}
+
+static struct gpu_buddy_block *
+__gpu_buddy_alloc_range_bias(struct gpu_buddy *mm,
+ u64 start, u64 end,
+ unsigned int order,
+ unsigned long flags)
+{
+ struct gpu_buddy_block *block;
+ bool fallback = false;
+
+ block = __alloc_range_bias(mm, start, end, order,
+ flags, fallback);
+ if (IS_ERR(block))
+ return __alloc_range_bias(mm, start, end, order,
+ flags, !fallback);
+
+ return block;
+}
+
+static struct gpu_buddy_block *
+get_maxblock(struct gpu_buddy *mm,
+ unsigned int order,
+ enum gpu_buddy_free_tree tree)
+{
+ struct gpu_buddy_block *max_block = NULL, *block = NULL;
+ struct rb_root *root;
+ unsigned int i;
+
+ for (i = order; i <= mm->max_order; ++i) {
+ root = &mm->free_trees[tree][i];
+ block = rbtree_last_free_block(root);
+ if (!block)
+ continue;
+
+ if (!max_block) {
+ max_block = block;
+ continue;
+ }
+
+ if (gpu_buddy_block_offset(block) >
+ gpu_buddy_block_offset(max_block)) {
+ max_block = block;
+ }
+ }
+
+ return max_block;
+}
+
+static struct gpu_buddy_block *
+alloc_from_freetree(struct gpu_buddy *mm,
+ unsigned int order,
+ unsigned long flags)
+{
+ struct gpu_buddy_block *block = NULL;
+ struct rb_root *root;
+ enum gpu_buddy_free_tree tree;
+ unsigned int tmp;
+ int err;
+
+ tree = (flags & GPU_BUDDY_CLEAR_ALLOCATION) ?
+ GPU_BUDDY_CLEAR_TREE : GPU_BUDDY_DIRTY_TREE;
+
+ if (flags & GPU_BUDDY_TOPDOWN_ALLOCATION) {
+ block = get_maxblock(mm, order, tree);
+ if (block)
+ /* Store the obtained block order */
+ tmp = gpu_buddy_block_order(block);
+ } else {
+ for (tmp = order; tmp <= mm->max_order; ++tmp) {
+ /* Get RB tree root for this order and tree */
+ root = &mm->free_trees[tree][tmp];
+ block = rbtree_last_free_block(root);
+ if (block)
+ break;
+ }
+ }
+
+ if (!block) {
+ /* Try allocating from the other tree */
+ tree = (tree == GPU_BUDDY_CLEAR_TREE) ?
+ GPU_BUDDY_DIRTY_TREE : GPU_BUDDY_CLEAR_TREE;
+
+ for (tmp = order; tmp <= mm->max_order; ++tmp) {
+ root = &mm->free_trees[tree][tmp];
+ block = rbtree_last_free_block(root);
+ if (block)
+ break;
+ }
+
+ if (!block)
+ return ERR_PTR(-ENOSPC);
+ }
+
+ BUG_ON(!gpu_buddy_block_is_free(block));
+
+ while (tmp != order) {
+ err = split_block(mm, block);
+ if (unlikely(err))
+ goto err_undo;
+
+ block = block->right;
+ tmp--;
+ }
+ return block;
+
+err_undo:
+ if (tmp != order)
+ __gpu_buddy_free(mm, block, false);
+ return ERR_PTR(err);
+}
+
+static int __alloc_range(struct gpu_buddy *mm,
+ struct list_head *dfs,
+ u64 start, u64 size,
+ struct list_head *blocks,
+ u64 *total_allocated_on_err)
+{
+ struct gpu_buddy_block *block;
+ struct gpu_buddy_block *buddy;
+ u64 total_allocated = 0;
+ LIST_HEAD(allocated);
+ u64 end;
+ int err;
+
+ end = start + size - 1;
+
+ do {
+ u64 block_start;
+ u64 block_end;
+
+ block = list_first_entry_or_null(dfs,
+ struct gpu_buddy_block,
+ tmp_link);
+ if (!block)
+ break;
+
+ list_del(&block->tmp_link);
+
+ block_start = gpu_buddy_block_offset(block);
+ block_end = block_start + gpu_buddy_block_size(mm, block) - 1;
+
+ if (!overlaps(start, end, block_start, block_end))
+ continue;
+
+ if (gpu_buddy_block_is_allocated(block)) {
+ err = -ENOSPC;
+ goto err_free;
+ }
+
+ if (contains(start, end, block_start, block_end)) {
+ if (gpu_buddy_block_is_free(block)) {
+ mark_allocated(mm, block);
+ total_allocated += gpu_buddy_block_size(mm, block);
+ mm->avail -= gpu_buddy_block_size(mm, block);
+ if (gpu_buddy_block_is_clear(block))
+ mm->clear_avail -= gpu_buddy_block_size(mm, block);
+ list_add_tail(&block->link, &allocated);
+ continue;
+ } else if (!mm->clear_avail) {
+ err = -ENOSPC;
+ goto err_free;
+ }
+ }
+
+ if (!gpu_buddy_block_is_split(block)) {
+ err = split_block(mm, block);
+ if (unlikely(err))
+ goto err_undo;
+ }
+
+ list_add(&block->right->tmp_link, dfs);
+ list_add(&block->left->tmp_link, dfs);
+ } while (1);
+
+ if (total_allocated < size) {
+ err = -ENOSPC;
+ goto err_free;
+ }
+
+ list_splice_tail(&allocated, blocks);
+
+ return 0;
+
+err_undo:
+ /*
+ * We really don't want to leave around a bunch of split blocks, since
+ * bigger is better, so make sure we merge everything back before we
+ * free the allocated blocks.
+ */
+ buddy = __get_buddy(block);
+ if (buddy &&
+ (gpu_buddy_block_is_free(block) &&
+ gpu_buddy_block_is_free(buddy)))
+ __gpu_buddy_free(mm, block, false);
+
+err_free:
+ if (err == -ENOSPC && total_allocated_on_err) {
+ list_splice_tail(&allocated, blocks);
+ *total_allocated_on_err = total_allocated;
+ } else {
+ gpu_buddy_free_list_internal(mm, &allocated);
+ }
+
+ return err;
+}
+
+static int __gpu_buddy_alloc_range(struct gpu_buddy *mm,
+ u64 start,
+ u64 size,
+ u64 *total_allocated_on_err,
+ struct list_head *blocks)
+{
+ LIST_HEAD(dfs);
+ int i;
+
+ for (i = 0; i < mm->n_roots; ++i)
+ list_add_tail(&mm->roots[i]->tmp_link, &dfs);
+
+ return __alloc_range(mm, &dfs, start, size,
+ blocks, total_allocated_on_err);
+}
+
+static int __alloc_contig_try_harder(struct gpu_buddy *mm,
+ u64 size,
+ u64 min_block_size,
+ struct list_head *blocks)
+{
+ u64 rhs_offset, lhs_offset, lhs_size, filled;
+ struct gpu_buddy_block *block;
+ unsigned int tree, order;
+ LIST_HEAD(blocks_lhs);
+ unsigned long pages;
+ u64 modify_size;
+ int err;
+
+ modify_size = rounddown_pow_of_two(size);
+ pages = modify_size >> ilog2(mm->chunk_size);
+ order = fls(pages) - 1;
+ if (order == 0)
+ return -ENOSPC;
+
+ for_each_free_tree(tree) {
+ struct rb_root *root;
+ struct rb_node *iter;
+
+ root = &mm->free_trees[tree][order];
+ if (rbtree_is_empty(root))
+ continue;
+
+ iter = rb_last(root);
+ while (iter) {
+ block = rbtree_get_free_block(iter);
+
+ /* Allocate blocks traversing RHS */
+ rhs_offset = gpu_buddy_block_offset(block);
+ err = __gpu_buddy_alloc_range(mm, rhs_offset, size,
+ &filled, blocks);
+ if (!err || err != -ENOSPC)
+ return err;
+
+ lhs_size = max((size - filled), min_block_size);
+ if (!IS_ALIGNED(lhs_size, min_block_size))
+ lhs_size = round_up(lhs_size, min_block_size);
+
+ /* Allocate blocks traversing LHS */
+ lhs_offset = gpu_buddy_block_offset(block) - lhs_size;
+ err = __gpu_buddy_alloc_range(mm, lhs_offset, lhs_size,
+ NULL, &blocks_lhs);
+ if (!err) {
+ list_splice(&blocks_lhs, blocks);
+ return 0;
+ } else if (err != -ENOSPC) {
+ gpu_buddy_free_list_internal(mm, blocks);
+ return err;
+ }
+ /* Free blocks for the next iteration */
+ gpu_buddy_free_list_internal(mm, blocks);
+
+ iter = rb_prev(iter);
+ }
+ }
+
+ return -ENOSPC;
+}
+
+/**
+ * gpu_buddy_block_trim - free unused pages
+ *
+ * @mm: GPU buddy manager
+ * @start: start address to begin the trimming.
+ * @new_size: original size requested
+ * @blocks: Input and output list of allocated blocks.
+ * MUST contain single block as input to be trimmed.
+ * On success will contain the newly allocated blocks
+ * making up the @new_size. Blocks always appear in
+ * ascending order
+ *
+ * For contiguous allocation, we round up the size to the nearest
+ * power of two value, drivers consume *actual* size, so remaining
+ * portions are unused and can be optionally freed with this function
+ *
+ * Returns:
+ * 0 on success, error code on failure.
+ */
+int gpu_buddy_block_trim(struct gpu_buddy *mm,
+ u64 *start,
+ u64 new_size,
+ struct list_head *blocks)
+{
+ struct gpu_buddy_block *parent;
+ struct gpu_buddy_block *block;
+ u64 block_start, block_end;
+ LIST_HEAD(dfs);
+ u64 new_start;
+ int err;
+
+ if (!list_is_singular(blocks))
+ return -EINVAL;
+
+ block = list_first_entry(blocks,
+ struct gpu_buddy_block,
+ link);
+
+ block_start = gpu_buddy_block_offset(block);
+ block_end = block_start + gpu_buddy_block_size(mm, block);
+
+ if (WARN_ON(!gpu_buddy_block_is_allocated(block)))
+ return -EINVAL;
+
+ if (new_size > gpu_buddy_block_size(mm, block))
+ return -EINVAL;
+
+ if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size))
+ return -EINVAL;
+
+ if (new_size == gpu_buddy_block_size(mm, block))
+ return 0;
+
+ new_start = block_start;
+ if (start) {
+ new_start = *start;
+
+ if (new_start < block_start)
+ return -EINVAL;
+
+ if (!IS_ALIGNED(new_start, mm->chunk_size))
+ return -EINVAL;
+
+ if (range_overflows(new_start, new_size, block_end))
+ return -EINVAL;
+ }
+
+ list_del(&block->link);
+ mark_free(mm, block);
+ mm->avail += gpu_buddy_block_size(mm, block);
+ if (gpu_buddy_block_is_clear(block))
+ mm->clear_avail += gpu_buddy_block_size(mm, block);
+
+ /* Prevent recursively freeing this node */
+ parent = block->parent;
+ block->parent = NULL;
+
+ list_add(&block->tmp_link, &dfs);
+ err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL);
+ if (err) {
+ mark_allocated(mm, block);
+ mm->avail -= gpu_buddy_block_size(mm, block);
+ if (gpu_buddy_block_is_clear(block))
+ mm->clear_avail -= gpu_buddy_block_size(mm, block);
+ list_add(&block->link, blocks);
+ }
+
+ block->parent = parent;
+ return err;
+}
+EXPORT_SYMBOL(gpu_buddy_block_trim);
+
+static struct gpu_buddy_block *
+__gpu_buddy_alloc_blocks(struct gpu_buddy *mm,
+ u64 start, u64 end,
+ unsigned int order,
+ unsigned long flags)
+{
+ if (flags & GPU_BUDDY_RANGE_ALLOCATION)
+ /* Allocate traversing within the range */
+ return __gpu_buddy_alloc_range_bias(mm, start, end,
+ order, flags);
+ else
+ /* Allocate from freetree */
+ return alloc_from_freetree(mm, order, flags);
+}
+
+/**
+ * gpu_buddy_alloc_blocks - allocate power-of-two blocks
+ *
+ * @mm: GPU buddy manager to allocate from
+ * @start: start of the allowed range for this block
+ * @end: end of the allowed range for this block
+ * @size: size of the allocation in bytes
+ * @min_block_size: alignment of the allocation
+ * @blocks: output list head to add allocated blocks
+ * @flags: GPU_BUDDY_*_ALLOCATION flags
+ *
+ * alloc_range_bias() called on range limitations, which traverses
+ * the tree and returns the desired block.
+ *
+ * alloc_from_freetree() called when *no* range restrictions
+ * are enforced, which picks the block from the freetree.
+ *
+ * Returns:
+ * 0 on success, error code on failure.
+ */
+int gpu_buddy_alloc_blocks(struct gpu_buddy *mm,
+ u64 start, u64 end, u64 size,
+ u64 min_block_size,
+ struct list_head *blocks,
+ unsigned long flags)
+{
+ struct gpu_buddy_block *block = NULL;
+ u64 original_size, original_min_size;
+ unsigned int min_order, order;
+ LIST_HEAD(allocated);
+ unsigned long pages;
+ int err;
+
+ if (size < mm->chunk_size)
+ return -EINVAL;
+
+ if (min_block_size < mm->chunk_size)
+ return -EINVAL;
+
+ if (!is_power_of_2(min_block_size))
+ return -EINVAL;
+
+ if (!IS_ALIGNED(start | end | size, mm->chunk_size))
+ return -EINVAL;
+
+ if (end > mm->size)
+ return -EINVAL;
+
+ if (range_overflows(start, size, mm->size))
+ return -EINVAL;
+
+ /* Actual range allocation */
+ if (start + size == end) {
+ if (!IS_ALIGNED(start | end, min_block_size))
+ return -EINVAL;
+
+ return __gpu_buddy_alloc_range(mm, start, size, NULL, blocks);
+ }
+
+ original_size = size;
+ original_min_size = min_block_size;
+
+ /* Roundup the size to power of 2 */
+ if (flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION) {
+ size = roundup_pow_of_two(size);
+ min_block_size = size;
+ /* Align size value to min_block_size */
+ } else if (!IS_ALIGNED(size, min_block_size)) {
+ size = round_up(size, min_block_size);
+ }
+
+ pages = size >> ilog2(mm->chunk_size);
+ order = fls(pages) - 1;
+ min_order = ilog2(min_block_size) - ilog2(mm->chunk_size);
+
+ if (order > mm->max_order || size > mm->size) {
+ if ((flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION) &&
+ !(flags & GPU_BUDDY_RANGE_ALLOCATION))
+ return __alloc_contig_try_harder(mm, original_size,
+ original_min_size, blocks);
+
+ return -EINVAL;
+ }
+
+ do {
+ order = min(order, (unsigned int)fls(pages) - 1);
+ BUG_ON(order > mm->max_order);
+ BUG_ON(order < min_order);
+
+ do {
+ block = __gpu_buddy_alloc_blocks(mm, start,
+ end,
+ order,
+ flags);
+ if (!IS_ERR(block))
+ break;
+
+ if (order-- == min_order) {
+ /* Try allocation through force merge method */
+ if (mm->clear_avail &&
+ !__force_merge(mm, start, end, min_order)) {
+ block = __gpu_buddy_alloc_blocks(mm, start,
+ end,
+ min_order,
+ flags);
+ if (!IS_ERR(block)) {
+ order = min_order;
+ break;
+ }
+ }
+
+ /*
+ * Try contiguous block allocation through
+ * try harder method.
+ */
+ if (flags & GPU_BUDDY_CONTIGUOUS_ALLOCATION &&
+ !(flags & GPU_BUDDY_RANGE_ALLOCATION))
+ return __alloc_contig_try_harder(mm,
+ original_size,
+ original_min_size,
+ blocks);
+ err = -ENOSPC;
+ goto err_free;
+ }
+ } while (1);
+
+ mark_allocated(mm, block);
+ mm->avail -= gpu_buddy_block_size(mm, block);
+ if (gpu_buddy_block_is_clear(block))
+ mm->clear_avail -= gpu_buddy_block_size(mm, block);
+ kmemleak_update_trace(block);
+ list_add_tail(&block->link, &allocated);
+
+ pages -= BIT(order);
+
+ if (!pages)
+ break;
+ } while (1);
+
+ /* Trim the allocated block to the required size */
+ if (!(flags & GPU_BUDDY_TRIM_DISABLE) &&
+ original_size != size) {
+ struct list_head *trim_list;
+ LIST_HEAD(temp);
+ u64 trim_size;
+
+ trim_list = &allocated;
+ trim_size = original_size;
+
+ if (!list_is_singular(&allocated)) {
+ block = list_last_entry(&allocated, typeof(*block), link);
+ list_move(&block->link, &temp);
+ trim_list = &temp;
+ trim_size = gpu_buddy_block_size(mm, block) -
+ (size - original_size);
+ }
+
+ gpu_buddy_block_trim(mm,
+ NULL,
+ trim_size,
+ trim_list);
+
+ if (!list_empty(&temp))
+ list_splice_tail(trim_list, &allocated);
+ }
+
+ list_splice_tail(&allocated, blocks);
+ return 0;
+
+err_free:
+ gpu_buddy_free_list_internal(mm, &allocated);
+ return err;
+}
+EXPORT_SYMBOL(gpu_buddy_alloc_blocks);
+
+/**
+ * gpu_buddy_block_print - print block information
+ *
+ * @mm: GPU buddy manager
+ * @block: GPU buddy block
+ */
+void gpu_buddy_block_print(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ u64 start = gpu_buddy_block_offset(block);
+ u64 size = gpu_buddy_block_size(mm, block);
+
+ pr_info("%#018llx-%#018llx: %llu\n", start, start + size, size);
+}
+EXPORT_SYMBOL(gpu_buddy_block_print);
+
+/**
+ * gpu_buddy_print - print allocator state
+ *
+ * @mm: GPU buddy manager
+ */
+void gpu_buddy_print(struct gpu_buddy *mm)
+{
+ int order;
+
+ pr_info("chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n",
+ mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
+
+ for (order = mm->max_order; order >= 0; order--) {
+ struct gpu_buddy_block *block, *tmp;
+ struct rb_root *root;
+ u64 count = 0, free;
+ unsigned int tree;
+
+ for_each_free_tree(tree) {
+ root = &mm->free_trees[tree][order];
+
+ rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
+ BUG_ON(!gpu_buddy_block_is_free(block));
+ count++;
+ }
+ }
+
+ free = count * (mm->chunk_size << order);
+ if (free < SZ_1M)
+ pr_info("order-%2d free: %8llu KiB, blocks: %llu\n",
+ order, free >> 10, count);
+ else
+ pr_info("order-%2d free: %8llu MiB, blocks: %llu\n",
+ order, free >> 20, count);
+ }
+}
+EXPORT_SYMBOL(gpu_buddy_print);
+
+static void gpu_buddy_module_exit(void)
+{
+ kmem_cache_destroy(slab_blocks);
+}
+
+static int __init gpu_buddy_module_init(void)
+{
+ slab_blocks = KMEM_CACHE(gpu_buddy_block, 0);
+ if (!slab_blocks)
+ return -ENOMEM;
+
+ return 0;
+}
+
+module_init(gpu_buddy_module_init);
+module_exit(gpu_buddy_module_exit);
+
+MODULE_DESCRIPTION("GPU Buddy Allocator");
+MODULE_LICENSE("Dual MIT/GPL");
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index ac90f49e3a75..d1bf856e0b08 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -220,6 +220,7 @@ config DRM_GPUSVM
config DRM_BUDDY
tristate
depends on DRM
+ select GPU_BUDDY
help
A page based buddy allocator
diff --git a/drivers/gpu/drm/Kconfig.debug b/drivers/gpu/drm/Kconfig.debug
index 05dc43c0b8c5..1f4c408c7920 100644
--- a/drivers/gpu/drm/Kconfig.debug
+++ b/drivers/gpu/drm/Kconfig.debug
@@ -71,6 +71,7 @@ config DRM_KUNIT_TEST
select DRM_KUNIT_TEST_HELPERS
select DRM_LIB_RANDOM
select DRM_SYSFB_HELPER
+ select GPU_BUDDY
select PRIME_NUMBERS
default KUNIT_ALL_TESTS
help
@@ -88,10 +89,11 @@ config DRM_TTM_KUNIT_TEST
tristate "KUnit tests for TTM" if !KUNIT_ALL_TESTS
default n
depends on DRM && KUNIT && MMU && (UML || COMPILE_TEST)
- select DRM_TTM
select DRM_BUDDY
+ select DRM_TTM
select DRM_EXPORT_FOR_TESTS if m
select DRM_KUNIT_TEST_HELPERS
+ select GPU_BUDDY
default KUNIT_ALL_TESTS
help
Enables unit tests for TTM, a GPU memory manager subsystem used
diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 7f515be5185d..bb131543e1d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -23,6 +23,7 @@ config DRM_AMDGPU
select CRC16
select BACKLIGHT_CLASS_DEVICE
select INTERVAL_TREE
+ select GPU_BUDDY
select DRM_BUDDY
select DRM_SUBALLOC_HELPER
select DRM_EXEC
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index f582113d78b7..149f8f942eae 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -5663,7 +5663,7 @@ int amdgpu_ras_add_critical_region(struct amdgpu_device *adev,
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
struct amdgpu_vram_mgr_resource *vres;
struct ras_critical_region *region;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
int ret = 0;
if (!bo || !bo->tbo.resource)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
index be2e56ce1355..8908d9e08a30 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
@@ -55,7 +55,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
uint64_t start, uint64_t size,
struct amdgpu_res_cursor *cur)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
struct list_head *head, *next;
struct drm_mm_node *node;
@@ -71,7 +71,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
head = &to_amdgpu_vram_mgr_resource(res)->blocks;
block = list_first_entry_or_null(head,
- struct drm_buddy_block,
+ struct gpu_buddy_block,
link);
if (!block)
goto fallback;
@@ -81,7 +81,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
next = block->link.next;
if (next != head)
- block = list_entry(next, struct drm_buddy_block, link);
+ block = list_entry(next, struct gpu_buddy_block, link);
}
cur->start = amdgpu_vram_mgr_block_start(block) + start;
@@ -125,7 +125,7 @@ static inline void amdgpu_res_first(struct ttm_resource *res,
*/
static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
struct drm_mm_node *node;
struct list_head *next;
@@ -146,7 +146,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
block = cur->node;
next = block->link.next;
- block = list_entry(next, struct drm_buddy_block, link);
+ block = list_entry(next, struct gpu_buddy_block, link);
cur->node = block;
cur->start = amdgpu_vram_mgr_block_start(block);
@@ -175,7 +175,7 @@ static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
*/
static inline bool amdgpu_res_cleared(struct amdgpu_res_cursor *cur)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
switch (cur->mem_type) {
case TTM_PL_VRAM:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
index 9d934c07fa6b..6c06a9c9b13f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
@@ -23,6 +23,8 @@
*/
#include <linux/dma-mapping.h>
+
+#include <drm/drm_buddy.h>
#include <drm/ttm/ttm_range_manager.h>
#include <drm/drm_drv.h>
@@ -52,15 +54,15 @@ to_amdgpu_device(struct amdgpu_vram_mgr *mgr)
return container_of(mgr, struct amdgpu_device, mman.vram_mgr);
}
-static inline struct drm_buddy_block *
+static inline struct gpu_buddy_block *
amdgpu_vram_mgr_first_block(struct list_head *list)
{
- return list_first_entry_or_null(list, struct drm_buddy_block, link);
+ return list_first_entry_or_null(list, struct gpu_buddy_block, link);
}
static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
u64 start, size;
block = amdgpu_vram_mgr_first_block(head);
@@ -71,7 +73,7 @@ static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head)
start = amdgpu_vram_mgr_block_start(block);
size = amdgpu_vram_mgr_block_size(block);
- block = list_entry(block->link.next, struct drm_buddy_block, link);
+ block = list_entry(block->link.next, struct gpu_buddy_block, link);
if (start + size != amdgpu_vram_mgr_block_start(block))
return false;
}
@@ -81,7 +83,7 @@ static inline bool amdgpu_is_vram_mgr_blocks_contiguous(struct list_head *head)
static inline u64 amdgpu_vram_mgr_blocks_size(struct list_head *head)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
u64 size = 0;
list_for_each_entry(block, head, link)
@@ -254,7 +256,7 @@ const struct attribute_group amdgpu_vram_mgr_attr_group = {
* Calculate how many bytes of the DRM BUDDY block are inside visible VRAM
*/
static u64 amdgpu_vram_mgr_vis_size(struct amdgpu_device *adev,
- struct drm_buddy_block *block)
+ struct gpu_buddy_block *block)
{
u64 start = amdgpu_vram_mgr_block_start(block);
u64 end = start + amdgpu_vram_mgr_block_size(block);
@@ -279,7 +281,7 @@ u64 amdgpu_vram_mgr_bo_visible_size(struct amdgpu_bo *bo)
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
struct ttm_resource *res = bo->tbo.resource;
struct amdgpu_vram_mgr_resource *vres = to_amdgpu_vram_mgr_resource(res);
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
u64 usage = 0;
if (amdgpu_gmc_vram_full_visible(&adev->gmc))
@@ -299,15 +301,15 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man)
{
struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
struct amdgpu_device *adev = to_amdgpu_device(mgr);
- struct drm_buddy *mm = &mgr->mm;
+ struct gpu_buddy *mm = &mgr->mm;
struct amdgpu_vram_reservation *rsv, *temp;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
uint64_t vis_usage;
list_for_each_entry_safe(rsv, temp, &mgr->reservations_pending, blocks) {
- if (drm_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size,
+ if (gpu_buddy_alloc_blocks(mm, rsv->start, rsv->start + rsv->size,
rsv->size, mm->chunk_size, &rsv->allocated,
- DRM_BUDDY_RANGE_ALLOCATION))
+ GPU_BUDDY_RANGE_ALLOCATION))
continue;
block = amdgpu_vram_mgr_first_block(&rsv->allocated);
@@ -403,7 +405,7 @@ int amdgpu_vram_mgr_query_address_block_info(struct amdgpu_vram_mgr *mgr,
uint64_t address, struct amdgpu_vram_block_info *info)
{
struct amdgpu_vram_mgr_resource *vres;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
u64 start, size;
int ret = -ENOENT;
@@ -450,8 +452,8 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
struct amdgpu_vram_mgr_resource *vres;
u64 size, remaining_size, lpfn, fpfn;
unsigned int adjust_dcc_size = 0;
- struct drm_buddy *mm = &mgr->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = &mgr->mm;
+ struct gpu_buddy_block *block;
unsigned long pages_per_block;
int r;
@@ -493,17 +495,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
INIT_LIST_HEAD(&vres->blocks);
if (place->flags & TTM_PL_FLAG_TOPDOWN)
- vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
+ vres->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS)
- vres->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
+ vres->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CLEARED)
- vres->flags |= DRM_BUDDY_CLEAR_ALLOCATION;
+ vres->flags |= GPU_BUDDY_CLEAR_ALLOCATION;
if (fpfn || lpfn != mgr->mm.size)
/* Allocate blocks in desired range */
- vres->flags |= DRM_BUDDY_RANGE_ALLOCATION;
+ vres->flags |= GPU_BUDDY_RANGE_ALLOCATION;
if (bo->flags & AMDGPU_GEM_CREATE_GFX12_DCC &&
adev->gmc.gmc_funcs->get_dcc_alignment)
@@ -516,7 +518,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
dcc_size = roundup_pow_of_two(vres->base.size + adjust_dcc_size);
remaining_size = (u64)dcc_size;
- vres->flags |= DRM_BUDDY_TRIM_DISABLE;
+ vres->flags |= GPU_BUDDY_TRIM_DISABLE;
}
mutex_lock(&mgr->lock);
@@ -536,7 +538,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
BUG_ON(min_block_size < mm->chunk_size);
- r = drm_buddy_alloc_blocks(mm, fpfn,
+ r = gpu_buddy_alloc_blocks(mm, fpfn,
lpfn,
size,
min_block_size,
@@ -545,7 +547,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
if (unlikely(r == -ENOSPC) && pages_per_block == ~0ul &&
!(place->flags & TTM_PL_FLAG_CONTIGUOUS)) {
- vres->flags &= ~DRM_BUDDY_CONTIGUOUS_ALLOCATION;
+ vres->flags &= ~GPU_BUDDY_CONTIGUOUS_ALLOCATION;
pages_per_block = max_t(u32, 2UL << (20UL - PAGE_SHIFT),
tbo->page_alignment);
@@ -566,7 +568,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
list_add_tail(&vres->vres_node, &mgr->allocated_vres_list);
if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS && adjust_dcc_size) {
- struct drm_buddy_block *dcc_block;
+ struct gpu_buddy_block *dcc_block;
unsigned long dcc_start;
u64 trim_start;
@@ -576,7 +578,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
roundup((unsigned long)amdgpu_vram_mgr_block_start(dcc_block),
adjust_dcc_size);
trim_start = (u64)dcc_start;
- drm_buddy_block_trim(mm, &trim_start,
+ gpu_buddy_block_trim(mm, &trim_start,
(u64)vres->base.size,
&vres->blocks);
}
@@ -614,7 +616,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
return 0;
error_free_blocks:
- drm_buddy_free_list(mm, &vres->blocks, 0);
+ gpu_buddy_free_list(mm, &vres->blocks, 0);
mutex_unlock(&mgr->lock);
error_fini:
ttm_resource_fini(man, &vres->base);
@@ -637,8 +639,8 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man,
struct amdgpu_vram_mgr_resource *vres = to_amdgpu_vram_mgr_resource(res);
struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
struct amdgpu_device *adev = to_amdgpu_device(mgr);
- struct drm_buddy *mm = &mgr->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = &mgr->mm;
+ struct gpu_buddy_block *block;
uint64_t vis_usage = 0;
mutex_lock(&mgr->lock);
@@ -649,7 +651,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man,
list_for_each_entry(block, &vres->blocks, link)
vis_usage += amdgpu_vram_mgr_vis_size(adev, block);
- drm_buddy_free_list(mm, &vres->blocks, vres->flags);
+ gpu_buddy_free_list(mm, &vres->blocks, vres->flags);
amdgpu_vram_mgr_do_reserve(man);
mutex_unlock(&mgr->lock);
@@ -688,7 +690,7 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
if (!*sgt)
return -ENOMEM;
- /* Determine the number of DRM_BUDDY blocks to export */
+ /* Determine the number of GPU_BUDDY blocks to export */
amdgpu_res_first(res, offset, length, &cursor);
while (cursor.remaining) {
num_entries++;
@@ -704,10 +706,10 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
sg->length = 0;
/*
- * Walk down DRM_BUDDY blocks to populate scatterlist nodes
- * @note: Use iterator api to get first the DRM_BUDDY block
+ * Walk down GPU_BUDDY blocks to populate scatterlist nodes
+ * @note: Use iterator api to get first the GPU_BUDDY block
* and the number of bytes from it. Access the following
- * DRM_BUDDY block(s) if more buffer needs to exported
+ * GPU_BUDDY block(s) if more buffer needs to exported
*/
amdgpu_res_first(res, offset, length, &cursor);
for_each_sgtable_sg((*sgt), sg, i) {
@@ -792,10 +794,10 @@ uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr)
void amdgpu_vram_mgr_clear_reset_blocks(struct amdgpu_device *adev)
{
struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
- struct drm_buddy *mm = &mgr->mm;
+ struct gpu_buddy *mm = &mgr->mm;
mutex_lock(&mgr->lock);
- drm_buddy_reset_clear(mm, false);
+ gpu_buddy_reset_clear(mm, false);
mutex_unlock(&mgr->lock);
}
@@ -815,7 +817,7 @@ static bool amdgpu_vram_mgr_intersects(struct ttm_resource_manager *man,
size_t size)
{
struct amdgpu_vram_mgr_resource *mgr = to_amdgpu_vram_mgr_resource(res);
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
/* Check each drm buddy block individually */
list_for_each_entry(block, &mgr->blocks, link) {
@@ -848,7 +850,7 @@ static bool amdgpu_vram_mgr_compatible(struct ttm_resource_manager *man,
size_t size)
{
struct amdgpu_vram_mgr_resource *mgr = to_amdgpu_vram_mgr_resource(res);
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
/* Check each drm buddy block individually */
list_for_each_entry(block, &mgr->blocks, link) {
@@ -877,7 +879,7 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man,
struct drm_printer *printer)
{
struct amdgpu_vram_mgr *mgr = to_vram_mgr(man);
- struct drm_buddy *mm = &mgr->mm;
+ struct gpu_buddy *mm = &mgr->mm;
struct amdgpu_vram_reservation *rsv;
drm_printf(printer, " vis usage:%llu\n",
@@ -930,7 +932,7 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
mgr->default_page_size = PAGE_SIZE;
man->func = &amdgpu_vram_mgr_func;
- err = drm_buddy_init(&mgr->mm, man->size, PAGE_SIZE);
+ err = gpu_buddy_init(&mgr->mm, man->size, PAGE_SIZE);
if (err)
return err;
@@ -965,11 +967,11 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
kfree(rsv);
list_for_each_entry_safe(rsv, temp, &mgr->reserved_pages, blocks) {
- drm_buddy_free_list(&mgr->mm, &rsv->allocated, 0);
+ gpu_buddy_free_list(&mgr->mm, &rsv->allocated, 0);
kfree(rsv);
}
if (!adev->gmc.is_app_apu)
- drm_buddy_fini(&mgr->mm);
+ gpu_buddy_fini(&mgr->mm);
mutex_unlock(&mgr->lock);
ttm_resource_manager_cleanup(man);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h
index 5f5fd9a911c2..429a21a2e9b2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.h
@@ -24,11 +24,11 @@
#ifndef __AMDGPU_VRAM_MGR_H__
#define __AMDGPU_VRAM_MGR_H__
-#include <drm/drm_buddy.h>
+#include <linux/gpu_buddy.h>
struct amdgpu_vram_mgr {
struct ttm_resource_manager manager;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
/* protects access to buffer objects */
struct mutex lock;
struct list_head reservations_pending;
@@ -57,19 +57,19 @@ struct amdgpu_vram_mgr_resource {
struct amdgpu_vres_task task;
};
-static inline u64 amdgpu_vram_mgr_block_start(struct drm_buddy_block *block)
+static inline u64 amdgpu_vram_mgr_block_start(struct gpu_buddy_block *block)
{
- return drm_buddy_block_offset(block);
+ return gpu_buddy_block_offset(block);
}
-static inline u64 amdgpu_vram_mgr_block_size(struct drm_buddy_block *block)
+static inline u64 amdgpu_vram_mgr_block_size(struct gpu_buddy_block *block)
{
- return (u64)PAGE_SIZE << drm_buddy_block_order(block);
+ return (u64)PAGE_SIZE << gpu_buddy_block_order(block);
}
-static inline bool amdgpu_vram_mgr_is_cleared(struct drm_buddy_block *block)
+static inline bool amdgpu_vram_mgr_is_cleared(struct gpu_buddy_block *block)
{
- return drm_buddy_block_is_clear(block);
+ return gpu_buddy_block_is_clear(block);
}
static inline struct amdgpu_vram_mgr_resource *
@@ -82,8 +82,8 @@ static inline void amdgpu_vram_mgr_set_cleared(struct ttm_resource *res)
{
struct amdgpu_vram_mgr_resource *ares = to_amdgpu_vram_mgr_resource(res);
- WARN_ON(ares->flags & DRM_BUDDY_CLEARED);
- ares->flags |= DRM_BUDDY_CLEARED;
+ WARN_ON(ares->flags & GPU_BUDDY_CLEARED);
+ ares->flags |= GPU_BUDDY_CLEARED;
}
int amdgpu_vram_mgr_query_address_block_info(struct amdgpu_vram_mgr *mgr,
diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
index fd34d3755f7c..188b36054e59 100644
--- a/drivers/gpu/drm/drm_buddy.c
+++ b/drivers/gpu/drm/drm_buddy.c
@@ -3,1272 +3,25 @@
* Copyright © 2021 Intel Corporation
*/
-#include <kunit/test-bug.h>
-
#include <linux/export.h>
-#include <linux/kmemleak.h>
#include <linux/module.h>
#include <linux/sizes.h>
#include <drm/drm_buddy.h>
#include <drm/drm_print.h>
-enum drm_buddy_free_tree {
- DRM_BUDDY_CLEAR_TREE = 0,
- DRM_BUDDY_DIRTY_TREE,
- DRM_BUDDY_MAX_FREE_TREES,
-};
-
-static struct kmem_cache *slab_blocks;
-
-#define for_each_free_tree(tree) \
- for ((tree) = 0; (tree) < DRM_BUDDY_MAX_FREE_TREES; (tree)++)
-
-static struct drm_buddy_block *drm_block_alloc(struct drm_buddy *mm,
- struct drm_buddy_block *parent,
- unsigned int order,
- u64 offset)
-{
- struct drm_buddy_block *block;
-
- BUG_ON(order > DRM_BUDDY_MAX_ORDER);
-
- block = kmem_cache_zalloc(slab_blocks, GFP_KERNEL);
- if (!block)
- return NULL;
-
- block->header = offset;
- block->header |= order;
- block->parent = parent;
-
- RB_CLEAR_NODE(&block->rb);
-
- BUG_ON(block->header & DRM_BUDDY_HEADER_UNUSED);
- return block;
-}
-
-static void drm_block_free(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- kmem_cache_free(slab_blocks, block);
-}
-
-static enum drm_buddy_free_tree
-get_block_tree(struct drm_buddy_block *block)
-{
- return drm_buddy_block_is_clear(block) ?
- DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE;
-}
-
-static struct drm_buddy_block *
-rbtree_get_free_block(const struct rb_node *node)
-{
- return node ? rb_entry(node, struct drm_buddy_block, rb) : NULL;
-}
-
-static struct drm_buddy_block *
-rbtree_last_free_block(struct rb_root *root)
-{
- return rbtree_get_free_block(rb_last(root));
-}
-
-static bool rbtree_is_empty(struct rb_root *root)
-{
- return RB_EMPTY_ROOT(root);
-}
-
-static bool drm_buddy_block_offset_less(const struct drm_buddy_block *block,
- const struct drm_buddy_block *node)
-{
- return drm_buddy_block_offset(block) < drm_buddy_block_offset(node);
-}
-
-static bool rbtree_block_offset_less(struct rb_node *block,
- const struct rb_node *node)
-{
- return drm_buddy_block_offset_less(rbtree_get_free_block(block),
- rbtree_get_free_block(node));
-}
-
-static void rbtree_insert(struct drm_buddy *mm,
- struct drm_buddy_block *block,
- enum drm_buddy_free_tree tree)
-{
- rb_add(&block->rb,
- &mm->free_trees[tree][drm_buddy_block_order(block)],
- rbtree_block_offset_less);
-}
-
-static void rbtree_remove(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- unsigned int order = drm_buddy_block_order(block);
- enum drm_buddy_free_tree tree;
- struct rb_root *root;
-
- tree = get_block_tree(block);
- root = &mm->free_trees[tree][order];
-
- rb_erase(&block->rb, root);
- RB_CLEAR_NODE(&block->rb);
-}
-
-static void clear_reset(struct drm_buddy_block *block)
-{
- block->header &= ~DRM_BUDDY_HEADER_CLEAR;
-}
-
-static void mark_cleared(struct drm_buddy_block *block)
-{
- block->header |= DRM_BUDDY_HEADER_CLEAR;
-}
-
-static void mark_allocated(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- block->header &= ~DRM_BUDDY_HEADER_STATE;
- block->header |= DRM_BUDDY_ALLOCATED;
-
- rbtree_remove(mm, block);
-}
-
-static void mark_free(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- enum drm_buddy_free_tree tree;
-
- block->header &= ~DRM_BUDDY_HEADER_STATE;
- block->header |= DRM_BUDDY_FREE;
-
- tree = get_block_tree(block);
- rbtree_insert(mm, block, tree);
-}
-
-static void mark_split(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- block->header &= ~DRM_BUDDY_HEADER_STATE;
- block->header |= DRM_BUDDY_SPLIT;
-
- rbtree_remove(mm, block);
-}
-
-static inline bool overlaps(u64 s1, u64 e1, u64 s2, u64 e2)
-{
- return s1 <= e2 && e1 >= s2;
-}
-
-static inline bool contains(u64 s1, u64 e1, u64 s2, u64 e2)
-{
- return s1 <= s2 && e1 >= e2;
-}
-
-static struct drm_buddy_block *
-__get_buddy(struct drm_buddy_block *block)
-{
- struct drm_buddy_block *parent;
-
- parent = block->parent;
- if (!parent)
- return NULL;
-
- if (parent->left == block)
- return parent->right;
-
- return parent->left;
-}
-
-static unsigned int __drm_buddy_free(struct drm_buddy *mm,
- struct drm_buddy_block *block,
- bool force_merge)
-{
- struct drm_buddy_block *parent;
- unsigned int order;
-
- while ((parent = block->parent)) {
- struct drm_buddy_block *buddy;
-
- buddy = __get_buddy(block);
-
- if (!drm_buddy_block_is_free(buddy))
- break;
-
- if (!force_merge) {
- /*
- * Check the block and its buddy clear state and exit
- * the loop if they both have the dissimilar state.
- */
- if (drm_buddy_block_is_clear(block) !=
- drm_buddy_block_is_clear(buddy))
- break;
-
- if (drm_buddy_block_is_clear(block))
- mark_cleared(parent);
- }
-
- rbtree_remove(mm, buddy);
- if (force_merge && drm_buddy_block_is_clear(buddy))
- mm->clear_avail -= drm_buddy_block_size(mm, buddy);
-
- drm_block_free(mm, block);
- drm_block_free(mm, buddy);
-
- block = parent;
- }
-
- order = drm_buddy_block_order(block);
- mark_free(mm, block);
-
- return order;
-}
-
-static int __force_merge(struct drm_buddy *mm,
- u64 start,
- u64 end,
- unsigned int min_order)
-{
- unsigned int tree, order;
- int i;
-
- if (!min_order)
- return -ENOMEM;
-
- if (min_order > mm->max_order)
- return -EINVAL;
-
- for_each_free_tree(tree) {
- for (i = min_order - 1; i >= 0; i--) {
- struct rb_node *iter = rb_last(&mm->free_trees[tree][i]);
-
- while (iter) {
- struct drm_buddy_block *block, *buddy;
- u64 block_start, block_end;
-
- block = rbtree_get_free_block(iter);
- iter = rb_prev(iter);
-
- if (!block || !block->parent)
- continue;
-
- block_start = drm_buddy_block_offset(block);
- block_end = block_start + drm_buddy_block_size(mm, block) - 1;
-
- if (!contains(start, end, block_start, block_end))
- continue;
-
- buddy = __get_buddy(block);
- if (!drm_buddy_block_is_free(buddy))
- continue;
-
- WARN_ON(drm_buddy_block_is_clear(block) ==
- drm_buddy_block_is_clear(buddy));
-
- /*
- * Advance to the next node when the current node is the buddy,
- * as freeing the block will also remove its buddy from the tree.
- */
- if (iter == &buddy->rb)
- iter = rb_prev(iter);
-
- rbtree_remove(mm, block);
- if (drm_buddy_block_is_clear(block))
- mm->clear_avail -= drm_buddy_block_size(mm, block);
-
- order = __drm_buddy_free(mm, block, true);
- if (order >= min_order)
- return 0;
- }
- }
- }
-
- return -ENOMEM;
-}
-
-/**
- * drm_buddy_init - init memory manager
- *
- * @mm: DRM buddy manager to initialize
- * @size: size in bytes to manage
- * @chunk_size: minimum page size in bytes for our allocations
- *
- * Initializes the memory manager and its resources.
- *
- * Returns:
- * 0 on success, error code on failure.
- */
-int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size)
-{
- unsigned int i, j, root_count = 0;
- u64 offset = 0;
-
- if (size < chunk_size)
- return -EINVAL;
-
- if (chunk_size < SZ_4K)
- return -EINVAL;
-
- if (!is_power_of_2(chunk_size))
- return -EINVAL;
-
- size = round_down(size, chunk_size);
-
- mm->size = size;
- mm->avail = size;
- mm->clear_avail = 0;
- mm->chunk_size = chunk_size;
- mm->max_order = ilog2(size) - ilog2(chunk_size);
-
- BUG_ON(mm->max_order > DRM_BUDDY_MAX_ORDER);
-
- mm->free_trees = kmalloc_array(DRM_BUDDY_MAX_FREE_TREES,
- sizeof(*mm->free_trees),
- GFP_KERNEL);
- if (!mm->free_trees)
- return -ENOMEM;
-
- for_each_free_tree(i) {
- mm->free_trees[i] = kmalloc_array(mm->max_order + 1,
- sizeof(struct rb_root),
- GFP_KERNEL);
- if (!mm->free_trees[i])
- goto out_free_tree;
-
- for (j = 0; j <= mm->max_order; ++j)
- mm->free_trees[i][j] = RB_ROOT;
- }
-
- mm->n_roots = hweight64(size);
-
- mm->roots = kmalloc_array(mm->n_roots,
- sizeof(struct drm_buddy_block *),
- GFP_KERNEL);
- if (!mm->roots)
- goto out_free_tree;
-
- /*
- * Split into power-of-two blocks, in case we are given a size that is
- * not itself a power-of-two.
- */
- do {
- struct drm_buddy_block *root;
- unsigned int order;
- u64 root_size;
-
- order = ilog2(size) - ilog2(chunk_size);
- root_size = chunk_size << order;
-
- root = drm_block_alloc(mm, NULL, order, offset);
- if (!root)
- goto out_free_roots;
-
- mark_free(mm, root);
-
- BUG_ON(root_count > mm->max_order);
- BUG_ON(drm_buddy_block_size(mm, root) < chunk_size);
-
- mm->roots[root_count] = root;
-
- offset += root_size;
- size -= root_size;
- root_count++;
- } while (size);
-
- return 0;
-
-out_free_roots:
- while (root_count--)
- drm_block_free(mm, mm->roots[root_count]);
- kfree(mm->roots);
-out_free_tree:
- while (i--)
- kfree(mm->free_trees[i]);
- kfree(mm->free_trees);
- return -ENOMEM;
-}
-EXPORT_SYMBOL(drm_buddy_init);
-
-/**
- * drm_buddy_fini - tear down the memory manager
- *
- * @mm: DRM buddy manager to free
- *
- * Cleanup memory manager resources and the freetree
- */
-void drm_buddy_fini(struct drm_buddy *mm)
-{
- u64 root_size, size, start;
- unsigned int order;
- int i;
-
- size = mm->size;
-
- for (i = 0; i < mm->n_roots; ++i) {
- order = ilog2(size) - ilog2(mm->chunk_size);
- start = drm_buddy_block_offset(mm->roots[i]);
- __force_merge(mm, start, start + size, order);
-
- if (WARN_ON(!drm_buddy_block_is_free(mm->roots[i])))
- kunit_fail_current_test("buddy_fini() root");
-
- drm_block_free(mm, mm->roots[i]);
-
- root_size = mm->chunk_size << order;
- size -= root_size;
- }
-
- WARN_ON(mm->avail != mm->size);
-
- for_each_free_tree(i)
- kfree(mm->free_trees[i]);
- kfree(mm->free_trees);
- kfree(mm->roots);
-}
-EXPORT_SYMBOL(drm_buddy_fini);
-
-static int split_block(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- unsigned int block_order = drm_buddy_block_order(block) - 1;
- u64 offset = drm_buddy_block_offset(block);
-
- BUG_ON(!drm_buddy_block_is_free(block));
- BUG_ON(!drm_buddy_block_order(block));
-
- block->left = drm_block_alloc(mm, block, block_order, offset);
- if (!block->left)
- return -ENOMEM;
-
- block->right = drm_block_alloc(mm, block, block_order,
- offset + (mm->chunk_size << block_order));
- if (!block->right) {
- drm_block_free(mm, block->left);
- return -ENOMEM;
- }
-
- mark_split(mm, block);
-
- if (drm_buddy_block_is_clear(block)) {
- mark_cleared(block->left);
- mark_cleared(block->right);
- clear_reset(block);
- }
-
- mark_free(mm, block->left);
- mark_free(mm, block->right);
-
- return 0;
-}
-
-/**
- * drm_get_buddy - get buddy address
- *
- * @block: DRM buddy block
- *
- * Returns the corresponding buddy block for @block, or NULL
- * if this is a root block and can't be merged further.
- * Requires some kind of locking to protect against
- * any concurrent allocate and free operations.
- */
-struct drm_buddy_block *
-drm_get_buddy(struct drm_buddy_block *block)
-{
- return __get_buddy(block);
-}
-EXPORT_SYMBOL(drm_get_buddy);
-
-/**
- * drm_buddy_reset_clear - reset blocks clear state
- *
- * @mm: DRM buddy manager
- * @is_clear: blocks clear state
- *
- * Reset the clear state based on @is_clear value for each block
- * in the freetree.
- */
-void drm_buddy_reset_clear(struct drm_buddy *mm, bool is_clear)
-{
- enum drm_buddy_free_tree src_tree, dst_tree;
- u64 root_size, size, start;
- unsigned int order;
- int i;
-
- size = mm->size;
- for (i = 0; i < mm->n_roots; ++i) {
- order = ilog2(size) - ilog2(mm->chunk_size);
- start = drm_buddy_block_offset(mm->roots[i]);
- __force_merge(mm, start, start + size, order);
-
- root_size = mm->chunk_size << order;
- size -= root_size;
- }
-
- src_tree = is_clear ? DRM_BUDDY_DIRTY_TREE : DRM_BUDDY_CLEAR_TREE;
- dst_tree = is_clear ? DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE;
-
- for (i = 0; i <= mm->max_order; ++i) {
- struct rb_root *root = &mm->free_trees[src_tree][i];
- struct drm_buddy_block *block, *tmp;
-
- rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
- rbtree_remove(mm, block);
- if (is_clear) {
- mark_cleared(block);
- mm->clear_avail += drm_buddy_block_size(mm, block);
- } else {
- clear_reset(block);
- mm->clear_avail -= drm_buddy_block_size(mm, block);
- }
-
- rbtree_insert(mm, block, dst_tree);
- }
- }
-}
-EXPORT_SYMBOL(drm_buddy_reset_clear);
-
-/**
- * drm_buddy_free_block - free a block
- *
- * @mm: DRM buddy manager
- * @block: block to be freed
- */
-void drm_buddy_free_block(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- BUG_ON(!drm_buddy_block_is_allocated(block));
- mm->avail += drm_buddy_block_size(mm, block);
- if (drm_buddy_block_is_clear(block))
- mm->clear_avail += drm_buddy_block_size(mm, block);
-
- __drm_buddy_free(mm, block, false);
-}
-EXPORT_SYMBOL(drm_buddy_free_block);
-
-static void __drm_buddy_free_list(struct drm_buddy *mm,
- struct list_head *objects,
- bool mark_clear,
- bool mark_dirty)
-{
- struct drm_buddy_block *block, *on;
-
- WARN_ON(mark_dirty && mark_clear);
-
- list_for_each_entry_safe(block, on, objects, link) {
- if (mark_clear)
- mark_cleared(block);
- else if (mark_dirty)
- clear_reset(block);
- drm_buddy_free_block(mm, block);
- cond_resched();
- }
- INIT_LIST_HEAD(objects);
-}
-
-static void drm_buddy_free_list_internal(struct drm_buddy *mm,
- struct list_head *objects)
-{
- /*
- * Don't touch the clear/dirty bit, since allocation is still internal
- * at this point. For example we might have just failed part of the
- * allocation.
- */
- __drm_buddy_free_list(mm, objects, false, false);
-}
-
-/**
- * drm_buddy_free_list - free blocks
- *
- * @mm: DRM buddy manager
- * @objects: input list head to free blocks
- * @flags: optional flags like DRM_BUDDY_CLEARED
- */
-void drm_buddy_free_list(struct drm_buddy *mm,
- struct list_head *objects,
- unsigned int flags)
-{
- bool mark_clear = flags & DRM_BUDDY_CLEARED;
-
- __drm_buddy_free_list(mm, objects, mark_clear, !mark_clear);
-}
-EXPORT_SYMBOL(drm_buddy_free_list);
-
-static bool block_incompatible(struct drm_buddy_block *block, unsigned int flags)
-{
- bool needs_clear = flags & DRM_BUDDY_CLEAR_ALLOCATION;
-
- return needs_clear != drm_buddy_block_is_clear(block);
-}
-
-static struct drm_buddy_block *
-__alloc_range_bias(struct drm_buddy *mm,
- u64 start, u64 end,
- unsigned int order,
- unsigned long flags,
- bool fallback)
-{
- u64 req_size = mm->chunk_size << order;
- struct drm_buddy_block *block;
- struct drm_buddy_block *buddy;
- LIST_HEAD(dfs);
- int err;
- int i;
-
- end = end - 1;
-
- for (i = 0; i < mm->n_roots; ++i)
- list_add_tail(&mm->roots[i]->tmp_link, &dfs);
-
- do {
- u64 block_start;
- u64 block_end;
-
- block = list_first_entry_or_null(&dfs,
- struct drm_buddy_block,
- tmp_link);
- if (!block)
- break;
-
- list_del(&block->tmp_link);
-
- if (drm_buddy_block_order(block) < order)
- continue;
-
- block_start = drm_buddy_block_offset(block);
- block_end = block_start + drm_buddy_block_size(mm, block) - 1;
-
- if (!overlaps(start, end, block_start, block_end))
- continue;
-
- if (drm_buddy_block_is_allocated(block))
- continue;
-
- if (block_start < start || block_end > end) {
- u64 adjusted_start = max(block_start, start);
- u64 adjusted_end = min(block_end, end);
-
- if (round_down(adjusted_end + 1, req_size) <=
- round_up(adjusted_start, req_size))
- continue;
- }
-
- if (!fallback && block_incompatible(block, flags))
- continue;
-
- if (contains(start, end, block_start, block_end) &&
- order == drm_buddy_block_order(block)) {
- /*
- * Find the free block within the range.
- */
- if (drm_buddy_block_is_free(block))
- return block;
-
- continue;
- }
-
- if (!drm_buddy_block_is_split(block)) {
- err = split_block(mm, block);
- if (unlikely(err))
- goto err_undo;
- }
-
- list_add(&block->right->tmp_link, &dfs);
- list_add(&block->left->tmp_link, &dfs);
- } while (1);
-
- return ERR_PTR(-ENOSPC);
-
-err_undo:
- /*
- * We really don't want to leave around a bunch of split blocks, since
- * bigger is better, so make sure we merge everything back before we
- * free the allocated blocks.
- */
- buddy = __get_buddy(block);
- if (buddy &&
- (drm_buddy_block_is_free(block) &&
- drm_buddy_block_is_free(buddy)))
- __drm_buddy_free(mm, block, false);
- return ERR_PTR(err);
-}
-
-static struct drm_buddy_block *
-__drm_buddy_alloc_range_bias(struct drm_buddy *mm,
- u64 start, u64 end,
- unsigned int order,
- unsigned long flags)
-{
- struct drm_buddy_block *block;
- bool fallback = false;
-
- block = __alloc_range_bias(mm, start, end, order,
- flags, fallback);
- if (IS_ERR(block))
- return __alloc_range_bias(mm, start, end, order,
- flags, !fallback);
-
- return block;
-}
-
-static struct drm_buddy_block *
-get_maxblock(struct drm_buddy *mm,
- unsigned int order,
- enum drm_buddy_free_tree tree)
-{
- struct drm_buddy_block *max_block = NULL, *block = NULL;
- struct rb_root *root;
- unsigned int i;
-
- for (i = order; i <= mm->max_order; ++i) {
- root = &mm->free_trees[tree][i];
- block = rbtree_last_free_block(root);
- if (!block)
- continue;
-
- if (!max_block) {
- max_block = block;
- continue;
- }
-
- if (drm_buddy_block_offset(block) >
- drm_buddy_block_offset(max_block)) {
- max_block = block;
- }
- }
-
- return max_block;
-}
-
-static struct drm_buddy_block *
-alloc_from_freetree(struct drm_buddy *mm,
- unsigned int order,
- unsigned long flags)
-{
- struct drm_buddy_block *block = NULL;
- struct rb_root *root;
- enum drm_buddy_free_tree tree;
- unsigned int tmp;
- int err;
-
- tree = (flags & DRM_BUDDY_CLEAR_ALLOCATION) ?
- DRM_BUDDY_CLEAR_TREE : DRM_BUDDY_DIRTY_TREE;
-
- if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) {
- block = get_maxblock(mm, order, tree);
- if (block)
- /* Store the obtained block order */
- tmp = drm_buddy_block_order(block);
- } else {
- for (tmp = order; tmp <= mm->max_order; ++tmp) {
- /* Get RB tree root for this order and tree */
- root = &mm->free_trees[tree][tmp];
- block = rbtree_last_free_block(root);
- if (block)
- break;
- }
- }
-
- if (!block) {
- /* Try allocating from the other tree */
- tree = (tree == DRM_BUDDY_CLEAR_TREE) ?
- DRM_BUDDY_DIRTY_TREE : DRM_BUDDY_CLEAR_TREE;
-
- for (tmp = order; tmp <= mm->max_order; ++tmp) {
- root = &mm->free_trees[tree][tmp];
- block = rbtree_last_free_block(root);
- if (block)
- break;
- }
-
- if (!block)
- return ERR_PTR(-ENOSPC);
- }
-
- BUG_ON(!drm_buddy_block_is_free(block));
-
- while (tmp != order) {
- err = split_block(mm, block);
- if (unlikely(err))
- goto err_undo;
-
- block = block->right;
- tmp--;
- }
- return block;
-
-err_undo:
- if (tmp != order)
- __drm_buddy_free(mm, block, false);
- return ERR_PTR(err);
-}
-
-static int __alloc_range(struct drm_buddy *mm,
- struct list_head *dfs,
- u64 start, u64 size,
- struct list_head *blocks,
- u64 *total_allocated_on_err)
-{
- struct drm_buddy_block *block;
- struct drm_buddy_block *buddy;
- u64 total_allocated = 0;
- LIST_HEAD(allocated);
- u64 end;
- int err;
-
- end = start + size - 1;
-
- do {
- u64 block_start;
- u64 block_end;
-
- block = list_first_entry_or_null(dfs,
- struct drm_buddy_block,
- tmp_link);
- if (!block)
- break;
-
- list_del(&block->tmp_link);
-
- block_start = drm_buddy_block_offset(block);
- block_end = block_start + drm_buddy_block_size(mm, block) - 1;
-
- if (!overlaps(start, end, block_start, block_end))
- continue;
-
- if (drm_buddy_block_is_allocated(block)) {
- err = -ENOSPC;
- goto err_free;
- }
-
- if (contains(start, end, block_start, block_end)) {
- if (drm_buddy_block_is_free(block)) {
- mark_allocated(mm, block);
- total_allocated += drm_buddy_block_size(mm, block);
- mm->avail -= drm_buddy_block_size(mm, block);
- if (drm_buddy_block_is_clear(block))
- mm->clear_avail -= drm_buddy_block_size(mm, block);
- list_add_tail(&block->link, &allocated);
- continue;
- } else if (!mm->clear_avail) {
- err = -ENOSPC;
- goto err_free;
- }
- }
-
- if (!drm_buddy_block_is_split(block)) {
- err = split_block(mm, block);
- if (unlikely(err))
- goto err_undo;
- }
-
- list_add(&block->right->tmp_link, dfs);
- list_add(&block->left->tmp_link, dfs);
- } while (1);
-
- if (total_allocated < size) {
- err = -ENOSPC;
- goto err_free;
- }
-
- list_splice_tail(&allocated, blocks);
-
- return 0;
-
-err_undo:
- /*
- * We really don't want to leave around a bunch of split blocks, since
- * bigger is better, so make sure we merge everything back before we
- * free the allocated blocks.
- */
- buddy = __get_buddy(block);
- if (buddy &&
- (drm_buddy_block_is_free(block) &&
- drm_buddy_block_is_free(buddy)))
- __drm_buddy_free(mm, block, false);
-
-err_free:
- if (err == -ENOSPC && total_allocated_on_err) {
- list_splice_tail(&allocated, blocks);
- *total_allocated_on_err = total_allocated;
- } else {
- drm_buddy_free_list_internal(mm, &allocated);
- }
-
- return err;
-}
-
-static int __drm_buddy_alloc_range(struct drm_buddy *mm,
- u64 start,
- u64 size,
- u64 *total_allocated_on_err,
- struct list_head *blocks)
-{
- LIST_HEAD(dfs);
- int i;
-
- for (i = 0; i < mm->n_roots; ++i)
- list_add_tail(&mm->roots[i]->tmp_link, &dfs);
-
- return __alloc_range(mm, &dfs, start, size,
- blocks, total_allocated_on_err);
-}
-
-static int __alloc_contig_try_harder(struct drm_buddy *mm,
- u64 size,
- u64 min_block_size,
- struct list_head *blocks)
-{
- u64 rhs_offset, lhs_offset, lhs_size, filled;
- struct drm_buddy_block *block;
- unsigned int tree, order;
- LIST_HEAD(blocks_lhs);
- unsigned long pages;
- u64 modify_size;
- int err;
-
- modify_size = rounddown_pow_of_two(size);
- pages = modify_size >> ilog2(mm->chunk_size);
- order = fls(pages) - 1;
- if (order == 0)
- return -ENOSPC;
-
- for_each_free_tree(tree) {
- struct rb_root *root;
- struct rb_node *iter;
-
- root = &mm->free_trees[tree][order];
- if (rbtree_is_empty(root))
- continue;
-
- iter = rb_last(root);
- while (iter) {
- block = rbtree_get_free_block(iter);
-
- /* Allocate blocks traversing RHS */
- rhs_offset = drm_buddy_block_offset(block);
- err = __drm_buddy_alloc_range(mm, rhs_offset, size,
- &filled, blocks);
- if (!err || err != -ENOSPC)
- return err;
-
- lhs_size = max((size - filled), min_block_size);
- if (!IS_ALIGNED(lhs_size, min_block_size))
- lhs_size = round_up(lhs_size, min_block_size);
-
- /* Allocate blocks traversing LHS */
- lhs_offset = drm_buddy_block_offset(block) - lhs_size;
- err = __drm_buddy_alloc_range(mm, lhs_offset, lhs_size,
- NULL, &blocks_lhs);
- if (!err) {
- list_splice(&blocks_lhs, blocks);
- return 0;
- } else if (err != -ENOSPC) {
- drm_buddy_free_list_internal(mm, blocks);
- return err;
- }
- /* Free blocks for the next iteration */
- drm_buddy_free_list_internal(mm, blocks);
-
- iter = rb_prev(iter);
- }
- }
-
- return -ENOSPC;
-}
-
-/**
- * drm_buddy_block_trim - free unused pages
- *
- * @mm: DRM buddy manager
- * @start: start address to begin the trimming.
- * @new_size: original size requested
- * @blocks: Input and output list of allocated blocks.
- * MUST contain single block as input to be trimmed.
- * On success will contain the newly allocated blocks
- * making up the @new_size. Blocks always appear in
- * ascending order
- *
- * For contiguous allocation, we round up the size to the nearest
- * power of two value, drivers consume *actual* size, so remaining
- * portions are unused and can be optionally freed with this function
- *
- * Returns:
- * 0 on success, error code on failure.
- */
-int drm_buddy_block_trim(struct drm_buddy *mm,
- u64 *start,
- u64 new_size,
- struct list_head *blocks)
-{
- struct drm_buddy_block *parent;
- struct drm_buddy_block *block;
- u64 block_start, block_end;
- LIST_HEAD(dfs);
- u64 new_start;
- int err;
-
- if (!list_is_singular(blocks))
- return -EINVAL;
-
- block = list_first_entry(blocks,
- struct drm_buddy_block,
- link);
-
- block_start = drm_buddy_block_offset(block);
- block_end = block_start + drm_buddy_block_size(mm, block);
-
- if (WARN_ON(!drm_buddy_block_is_allocated(block)))
- return -EINVAL;
-
- if (new_size > drm_buddy_block_size(mm, block))
- return -EINVAL;
-
- if (!new_size || !IS_ALIGNED(new_size, mm->chunk_size))
- return -EINVAL;
-
- if (new_size == drm_buddy_block_size(mm, block))
- return 0;
-
- new_start = block_start;
- if (start) {
- new_start = *start;
-
- if (new_start < block_start)
- return -EINVAL;
-
- if (!IS_ALIGNED(new_start, mm->chunk_size))
- return -EINVAL;
-
- if (range_overflows(new_start, new_size, block_end))
- return -EINVAL;
- }
-
- list_del(&block->link);
- mark_free(mm, block);
- mm->avail += drm_buddy_block_size(mm, block);
- if (drm_buddy_block_is_clear(block))
- mm->clear_avail += drm_buddy_block_size(mm, block);
-
- /* Prevent recursively freeing this node */
- parent = block->parent;
- block->parent = NULL;
-
- list_add(&block->tmp_link, &dfs);
- err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL);
- if (err) {
- mark_allocated(mm, block);
- mm->avail -= drm_buddy_block_size(mm, block);
- if (drm_buddy_block_is_clear(block))
- mm->clear_avail -= drm_buddy_block_size(mm, block);
- list_add(&block->link, blocks);
- }
-
- block->parent = parent;
- return err;
-}
-EXPORT_SYMBOL(drm_buddy_block_trim);
-
-static struct drm_buddy_block *
-__drm_buddy_alloc_blocks(struct drm_buddy *mm,
- u64 start, u64 end,
- unsigned int order,
- unsigned long flags)
-{
- if (flags & DRM_BUDDY_RANGE_ALLOCATION)
- /* Allocate traversing within the range */
- return __drm_buddy_alloc_range_bias(mm, start, end,
- order, flags);
- else
- /* Allocate from freetree */
- return alloc_from_freetree(mm, order, flags);
-}
-
-/**
- * drm_buddy_alloc_blocks - allocate power-of-two blocks
- *
- * @mm: DRM buddy manager to allocate from
- * @start: start of the allowed range for this block
- * @end: end of the allowed range for this block
- * @size: size of the allocation in bytes
- * @min_block_size: alignment of the allocation
- * @blocks: output list head to add allocated blocks
- * @flags: DRM_BUDDY_*_ALLOCATION flags
- *
- * alloc_range_bias() called on range limitations, which traverses
- * the tree and returns the desired block.
- *
- * alloc_from_freetree() called when *no* range restrictions
- * are enforced, which picks the block from the freetree.
- *
- * Returns:
- * 0 on success, error code on failure.
- */
-int drm_buddy_alloc_blocks(struct drm_buddy *mm,
- u64 start, u64 end, u64 size,
- u64 min_block_size,
- struct list_head *blocks,
- unsigned long flags)
-{
- struct drm_buddy_block *block = NULL;
- u64 original_size, original_min_size;
- unsigned int min_order, order;
- LIST_HEAD(allocated);
- unsigned long pages;
- int err;
-
- if (size < mm->chunk_size)
- return -EINVAL;
-
- if (min_block_size < mm->chunk_size)
- return -EINVAL;
-
- if (!is_power_of_2(min_block_size))
- return -EINVAL;
-
- if (!IS_ALIGNED(start | end | size, mm->chunk_size))
- return -EINVAL;
-
- if (end > mm->size)
- return -EINVAL;
-
- if (range_overflows(start, size, mm->size))
- return -EINVAL;
-
- /* Actual range allocation */
- if (start + size == end) {
- if (!IS_ALIGNED(start | end, min_block_size))
- return -EINVAL;
-
- return __drm_buddy_alloc_range(mm, start, size, NULL, blocks);
- }
-
- original_size = size;
- original_min_size = min_block_size;
-
- /* Roundup the size to power of 2 */
- if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION) {
- size = roundup_pow_of_two(size);
- min_block_size = size;
- /* Align size value to min_block_size */
- } else if (!IS_ALIGNED(size, min_block_size)) {
- size = round_up(size, min_block_size);
- }
-
- pages = size >> ilog2(mm->chunk_size);
- order = fls(pages) - 1;
- min_order = ilog2(min_block_size) - ilog2(mm->chunk_size);
-
- if (order > mm->max_order || size > mm->size) {
- if ((flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION) &&
- !(flags & DRM_BUDDY_RANGE_ALLOCATION))
- return __alloc_contig_try_harder(mm, original_size,
- original_min_size, blocks);
-
- return -EINVAL;
- }
-
- do {
- order = min(order, (unsigned int)fls(pages) - 1);
- BUG_ON(order > mm->max_order);
- BUG_ON(order < min_order);
-
- do {
- block = __drm_buddy_alloc_blocks(mm, start,
- end,
- order,
- flags);
- if (!IS_ERR(block))
- break;
-
- if (order-- == min_order) {
- /* Try allocation through force merge method */
- if (mm->clear_avail &&
- !__force_merge(mm, start, end, min_order)) {
- block = __drm_buddy_alloc_blocks(mm, start,
- end,
- min_order,
- flags);
- if (!IS_ERR(block)) {
- order = min_order;
- break;
- }
- }
-
- /*
- * Try contiguous block allocation through
- * try harder method.
- */
- if (flags & DRM_BUDDY_CONTIGUOUS_ALLOCATION &&
- !(flags & DRM_BUDDY_RANGE_ALLOCATION))
- return __alloc_contig_try_harder(mm,
- original_size,
- original_min_size,
- blocks);
- err = -ENOSPC;
- goto err_free;
- }
- } while (1);
-
- mark_allocated(mm, block);
- mm->avail -= drm_buddy_block_size(mm, block);
- if (drm_buddy_block_is_clear(block))
- mm->clear_avail -= drm_buddy_block_size(mm, block);
- kmemleak_update_trace(block);
- list_add_tail(&block->link, &allocated);
-
- pages -= BIT(order);
-
- if (!pages)
- break;
- } while (1);
-
- /* Trim the allocated block to the required size */
- if (!(flags & DRM_BUDDY_TRIM_DISABLE) &&
- original_size != size) {
- struct list_head *trim_list;
- LIST_HEAD(temp);
- u64 trim_size;
-
- trim_list = &allocated;
- trim_size = original_size;
-
- if (!list_is_singular(&allocated)) {
- block = list_last_entry(&allocated, typeof(*block), link);
- list_move(&block->link, &temp);
- trim_list = &temp;
- trim_size = drm_buddy_block_size(mm, block) -
- (size - original_size);
- }
-
- drm_buddy_block_trim(mm,
- NULL,
- trim_size,
- trim_list);
-
- if (!list_empty(&temp))
- list_splice_tail(trim_list, &allocated);
- }
-
- list_splice_tail(&allocated, blocks);
- return 0;
-
-err_free:
- drm_buddy_free_list_internal(mm, &allocated);
- return err;
-}
-EXPORT_SYMBOL(drm_buddy_alloc_blocks);
-
/**
* drm_buddy_block_print - print block information
*
- * @mm: DRM buddy manager
- * @block: DRM buddy block
+ * @mm: GPU buddy manager
+ * @block: GPU buddy block
* @p: DRM printer to use
*/
-void drm_buddy_block_print(struct drm_buddy *mm,
- struct drm_buddy_block *block,
+void drm_buddy_block_print(struct gpu_buddy *mm, struct gpu_buddy_block *block,
struct drm_printer *p)
{
- u64 start = drm_buddy_block_offset(block);
- u64 size = drm_buddy_block_size(mm, block);
+ u64 start = gpu_buddy_block_offset(block);
+ u64 size = gpu_buddy_block_size(mm, block);
drm_printf(p, "%#018llx-%#018llx: %llu\n", start, start + size, size);
}
@@ -1277,18 +30,21 @@ EXPORT_SYMBOL(drm_buddy_block_print);
/**
* drm_buddy_print - print allocator state
*
- * @mm: DRM buddy manager
+ * @mm: GPU buddy manager
* @p: DRM printer to use
*/
-void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p)
+void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p)
{
int order;
- drm_printf(p, "chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n",
- mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20, mm->clear_avail >> 20);
+ drm_printf(
+ p,
+ "chunk_size: %lluKiB, total: %lluMiB, free: %lluMiB, clear_free: %lluMiB\n",
+ mm->chunk_size >> 10, mm->size >> 20, mm->avail >> 20,
+ mm->clear_avail >> 20);
for (order = mm->max_order; order >= 0; order--) {
- struct drm_buddy_block *block, *tmp;
+ struct gpu_buddy_block *block, *tmp;
struct rb_root *root;
u64 count = 0, free;
unsigned int tree;
@@ -1296,8 +52,9 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p)
for_each_free_tree(tree) {
root = &mm->free_trees[tree][order];
- rbtree_postorder_for_each_entry_safe(block, tmp, root, rb) {
- BUG_ON(!drm_buddy_block_is_free(block));
+ rbtree_postorder_for_each_entry_safe(block, tmp, root,
+ rb) {
+ BUG_ON(!gpu_buddy_block_is_free(block));
count++;
}
}
@@ -1315,22 +72,5 @@ void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p)
}
EXPORT_SYMBOL(drm_buddy_print);
-static void drm_buddy_module_exit(void)
-{
- kmem_cache_destroy(slab_blocks);
-}
-
-static int __init drm_buddy_module_init(void)
-{
- slab_blocks = KMEM_CACHE(drm_buddy_block, 0);
- if (!slab_blocks)
- return -ENOMEM;
-
- return 0;
-}
-
-module_init(drm_buddy_module_init);
-module_exit(drm_buddy_module_exit);
-
-MODULE_DESCRIPTION("DRM Buddy Allocator");
+MODULE_DESCRIPTION("DRM-specific GPU Buddy Allocator Print Helpers");
MODULE_LICENSE("Dual MIT/GPL");
diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig
index 5e939004b646..859aeca87c19 100644
--- a/drivers/gpu/drm/i915/Kconfig
+++ b/drivers/gpu/drm/i915/Kconfig
@@ -38,6 +38,7 @@ config DRM_I915
select CEC_CORE if CEC_NOTIFIER
select VMAP_PFN
select DRM_TTM
+ select GPU_BUDDY
select DRM_BUDDY
select AUXILIARY_BUS
help
diff --git a/drivers/gpu/drm/i915/i915_scatterlist.c b/drivers/gpu/drm/i915/i915_scatterlist.c
index 4d830740946d..6a34dae13769 100644
--- a/drivers/gpu/drm/i915/i915_scatterlist.c
+++ b/drivers/gpu/drm/i915/i915_scatterlist.c
@@ -7,7 +7,7 @@
#include "i915_scatterlist.h"
#include "i915_ttm_buddy_manager.h"
-#include <drm/drm_buddy.h>
+#include <linux/gpu_buddy.h>
#include <drm/drm_mm.h>
#include <linux/slab.h>
@@ -167,9 +167,9 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
const u64 size = res->size;
const u32 max_segment = round_down(UINT_MAX, page_alignment);
- struct drm_buddy *mm = bman_res->mm;
+ struct gpu_buddy *mm = bman_res->mm;
struct list_head *blocks = &bman_res->blocks;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
struct i915_refct_sgt *rsgt;
struct scatterlist *sg;
struct sg_table *st;
@@ -202,8 +202,8 @@ struct i915_refct_sgt *i915_rsgt_from_buddy_resource(struct ttm_resource *res,
list_for_each_entry(block, blocks, link) {
u64 block_size, offset;
- block_size = min_t(u64, size, drm_buddy_block_size(mm, block));
- offset = drm_buddy_block_offset(block);
+ block_size = min_t(u64, size, gpu_buddy_block_size(mm, block));
+ offset = gpu_buddy_block_offset(block);
while (block_size) {
u64 len;
diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
index d5c6e6605086..f43d7f2771ad 100644
--- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
+++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c
@@ -4,6 +4,7 @@
*/
#include <linux/slab.h>
+#include <linux/gpu_buddy.h>
#include <drm/drm_buddy.h>
#include <drm/drm_print.h>
@@ -16,7 +17,7 @@
struct i915_ttm_buddy_manager {
struct ttm_resource_manager manager;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
struct list_head reserved;
struct mutex lock;
unsigned long visible_size;
@@ -38,7 +39,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
{
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
struct i915_ttm_buddy_resource *bman_res;
- struct drm_buddy *mm = &bman->mm;
+ struct gpu_buddy *mm = &bman->mm;
unsigned long n_pages, lpfn;
u64 min_page_size;
u64 size;
@@ -57,13 +58,13 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
bman_res->mm = mm;
if (place->flags & TTM_PL_FLAG_TOPDOWN)
- bman_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
+ bman_res->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
if (place->flags & TTM_PL_FLAG_CONTIGUOUS)
- bman_res->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
+ bman_res->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
if (place->fpfn || lpfn != man->size)
- bman_res->flags |= DRM_BUDDY_RANGE_ALLOCATION;
+ bman_res->flags |= GPU_BUDDY_RANGE_ALLOCATION;
GEM_BUG_ON(!bman_res->base.size);
size = bman_res->base.size;
@@ -89,7 +90,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
goto err_free_res;
}
- err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
+ err = gpu_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
(u64)lpfn << PAGE_SHIFT,
(u64)n_pages << PAGE_SHIFT,
min_page_size,
@@ -101,15 +102,15 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
if (lpfn <= bman->visible_size) {
bman_res->used_visible_size = PFN_UP(bman_res->base.size);
} else {
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
list_for_each_entry(block, &bman_res->blocks, link) {
unsigned long start =
- drm_buddy_block_offset(block) >> PAGE_SHIFT;
+ gpu_buddy_block_offset(block) >> PAGE_SHIFT;
if (start < bman->visible_size) {
unsigned long end = start +
- (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
+ (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
bman_res->used_visible_size +=
min(end, bman->visible_size) - start;
@@ -126,7 +127,7 @@ static int i915_ttm_buddy_man_alloc(struct ttm_resource_manager *man,
return 0;
err_free_blocks:
- drm_buddy_free_list(mm, &bman_res->blocks, 0);
+ gpu_buddy_free_list(mm, &bman_res->blocks, 0);
mutex_unlock(&bman->lock);
err_free_res:
ttm_resource_fini(man, &bman_res->base);
@@ -141,7 +142,7 @@ static void i915_ttm_buddy_man_free(struct ttm_resource_manager *man,
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
mutex_lock(&bman->lock);
- drm_buddy_free_list(&bman->mm, &bman_res->blocks, 0);
+ gpu_buddy_free_list(&bman->mm, &bman_res->blocks, 0);
bman->visible_avail += bman_res->used_visible_size;
mutex_unlock(&bman->lock);
@@ -156,8 +157,8 @@ static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man,
{
struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
- struct drm_buddy *mm = &bman->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = &bman->mm;
+ struct gpu_buddy_block *block;
if (!place->fpfn && !place->lpfn)
return true;
@@ -176,9 +177,9 @@ static bool i915_ttm_buddy_man_intersects(struct ttm_resource_manager *man,
/* Check each drm buddy block individually */
list_for_each_entry(block, &bman_res->blocks, link) {
unsigned long fpfn =
- drm_buddy_block_offset(block) >> PAGE_SHIFT;
+ gpu_buddy_block_offset(block) >> PAGE_SHIFT;
unsigned long lpfn = fpfn +
- (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
+ (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
if (place->fpfn < lpfn && place->lpfn > fpfn)
return true;
@@ -194,8 +195,8 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man,
{
struct i915_ttm_buddy_resource *bman_res = to_ttm_buddy_resource(res);
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
- struct drm_buddy *mm = &bman->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = &bman->mm;
+ struct gpu_buddy_block *block;
if (!place->fpfn && !place->lpfn)
return true;
@@ -209,9 +210,9 @@ static bool i915_ttm_buddy_man_compatible(struct ttm_resource_manager *man,
/* Check each drm buddy block individually */
list_for_each_entry(block, &bman_res->blocks, link) {
unsigned long fpfn =
- drm_buddy_block_offset(block) >> PAGE_SHIFT;
+ gpu_buddy_block_offset(block) >> PAGE_SHIFT;
unsigned long lpfn = fpfn +
- (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
+ (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
if (fpfn < place->fpfn || lpfn > place->lpfn)
return false;
@@ -224,7 +225,7 @@ static void i915_ttm_buddy_man_debug(struct ttm_resource_manager *man,
struct drm_printer *printer)
{
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
mutex_lock(&bman->lock);
drm_printf(printer, "default_page_size: %lluKiB\n",
@@ -293,7 +294,7 @@ int i915_ttm_buddy_man_init(struct ttm_device *bdev,
if (!bman)
return -ENOMEM;
- err = drm_buddy_init(&bman->mm, size, chunk_size);
+ err = gpu_buddy_init(&bman->mm, size, chunk_size);
if (err)
goto err_free_bman;
@@ -333,7 +334,7 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type)
{
struct ttm_resource_manager *man = ttm_manager_type(bdev, type);
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
- struct drm_buddy *mm = &bman->mm;
+ struct gpu_buddy *mm = &bman->mm;
int ret;
ttm_resource_manager_set_used(man, false);
@@ -345,8 +346,8 @@ int i915_ttm_buddy_man_fini(struct ttm_device *bdev, unsigned int type)
ttm_set_driver_manager(bdev, type, NULL);
mutex_lock(&bman->lock);
- drm_buddy_free_list(mm, &bman->reserved, 0);
- drm_buddy_fini(mm);
+ gpu_buddy_free_list(mm, &bman->reserved, 0);
+ gpu_buddy_fini(mm);
bman->visible_avail += bman->visible_reserved;
WARN_ON_ONCE(bman->visible_avail != bman->visible_size);
mutex_unlock(&bman->lock);
@@ -371,15 +372,15 @@ int i915_ttm_buddy_man_reserve(struct ttm_resource_manager *man,
u64 start, u64 size)
{
struct i915_ttm_buddy_manager *bman = to_buddy_manager(man);
- struct drm_buddy *mm = &bman->mm;
+ struct gpu_buddy *mm = &bman->mm;
unsigned long fpfn = start >> PAGE_SHIFT;
unsigned long flags = 0;
int ret;
- flags |= DRM_BUDDY_RANGE_ALLOCATION;
+ flags |= GPU_BUDDY_RANGE_ALLOCATION;
mutex_lock(&bman->lock);
- ret = drm_buddy_alloc_blocks(mm, start,
+ ret = gpu_buddy_alloc_blocks(mm, start,
start + size,
size, mm->chunk_size,
&bman->reserved,
diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
index d64620712830..4a92dcf09766 100644
--- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
+++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
@@ -13,14 +13,14 @@
struct ttm_device;
struct ttm_resource_manager;
-struct drm_buddy;
+struct gpu_buddy;
/**
* struct i915_ttm_buddy_resource
*
* @base: struct ttm_resource base class we extend
* @blocks: the list of struct i915_buddy_block for this resource/allocation
- * @flags: DRM_BUDDY_*_ALLOCATION flags
+ * @flags: GPU_BUDDY_*_ALLOCATION flags
* @used_visible_size: How much of this resource, if any, uses the CPU visible
* portion, in pages.
* @mm: the struct i915_buddy_mm for this resource
@@ -33,7 +33,7 @@ struct i915_ttm_buddy_resource {
struct list_head blocks;
unsigned long flags;
unsigned long used_visible_size;
- struct drm_buddy *mm;
+ struct gpu_buddy *mm;
};
/**
diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
index 7b856b5090f9..8307390943a2 100644
--- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c
+++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
@@ -6,7 +6,7 @@
#include <linux/prime_numbers.h>
#include <linux/sort.h>
-#include <drm/drm_buddy.h>
+#include <linux/gpu_buddy.h>
#include "../i915_selftest.h"
@@ -371,7 +371,7 @@ static int igt_mock_splintered_region(void *arg)
struct drm_i915_private *i915 = mem->i915;
struct i915_ttm_buddy_resource *res;
struct drm_i915_gem_object *obj;
- struct drm_buddy *mm;
+ struct gpu_buddy *mm;
unsigned int expected_order;
LIST_HEAD(objects);
u64 size;
@@ -447,8 +447,8 @@ static int igt_mock_max_segment(void *arg)
struct drm_i915_private *i915 = mem->i915;
struct i915_ttm_buddy_resource *res;
struct drm_i915_gem_object *obj;
- struct drm_buddy_block *block;
- struct drm_buddy *mm;
+ struct gpu_buddy_block *block;
+ struct gpu_buddy *mm;
struct list_head *blocks;
struct scatterlist *sg;
I915_RND_STATE(prng);
@@ -487,8 +487,8 @@ static int igt_mock_max_segment(void *arg)
mm = res->mm;
size = 0;
list_for_each_entry(block, blocks, link) {
- if (drm_buddy_block_size(mm, block) > size)
- size = drm_buddy_block_size(mm, block);
+ if (gpu_buddy_block_size(mm, block) > size)
+ size = gpu_buddy_block_size(mm, block);
}
if (size < max_segment) {
pr_err("%s: Failed to create a huge contiguous block [> %u], largest block %lld\n",
@@ -527,14 +527,14 @@ static u64 igt_object_mappable_total(struct drm_i915_gem_object *obj)
struct intel_memory_region *mr = obj->mm.region;
struct i915_ttm_buddy_resource *bman_res =
to_ttm_buddy_resource(obj->mm.res);
- struct drm_buddy *mm = bman_res->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = bman_res->mm;
+ struct gpu_buddy_block *block;
u64 total;
total = 0;
list_for_each_entry(block, &bman_res->blocks, link) {
- u64 start = drm_buddy_block_offset(block);
- u64 end = start + drm_buddy_block_size(mm, block);
+ u64 start = gpu_buddy_block_offset(block);
+ u64 end = start + gpu_buddy_block_size(mm, block);
if (start < resource_size(&mr->io))
total += min_t(u64, end, resource_size(&mr->io)) - start;
diff --git a/drivers/gpu/drm/tests/Makefile b/drivers/gpu/drm/tests/Makefile
index 87d5d5f9332a..d2e2e3d8349a 100644
--- a/drivers/gpu/drm/tests/Makefile
+++ b/drivers/gpu/drm/tests/Makefile
@@ -7,7 +7,6 @@ obj-$(CONFIG_DRM_KUNIT_TEST) += \
drm_atomic_test.o \
drm_atomic_state_test.o \
drm_bridge_test.o \
- drm_buddy_test.o \
drm_cmdline_parser_test.o \
drm_connector_test.o \
drm_damage_helper_test.o \
diff --git a/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c b/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c
index 6d95447a989d..b6f90160b0f6 100644
--- a/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c
+++ b/drivers/gpu/drm/ttm/tests/ttm_bo_validate_test.c
@@ -3,6 +3,7 @@
* Copyright © 2023 Intel Corporation
*/
#include <linux/delay.h>
+#include <linux/gpu_buddy.h>
#include <linux/kthread.h>
#include <drm/ttm/ttm_resource.h>
@@ -251,7 +252,7 @@ static void ttm_bo_validate_basic(struct kunit *test)
NULL, &dummy_ttm_bo_destroy);
KUNIT_EXPECT_EQ(test, err, 0);
- snd_place = ttm_place_kunit_init(test, snd_mem, DRM_BUDDY_TOPDOWN_ALLOCATION);
+ snd_place = ttm_place_kunit_init(test, snd_mem, GPU_BUDDY_TOPDOWN_ALLOCATION);
snd_placement = ttm_placement_kunit_init(test, snd_place, 1);
err = ttm_bo_validate(bo, snd_placement, &ctx_val);
@@ -263,7 +264,7 @@ static void ttm_bo_validate_basic(struct kunit *test)
KUNIT_EXPECT_TRUE(test, ttm_tt_is_populated(bo->ttm));
KUNIT_EXPECT_EQ(test, bo->resource->mem_type, snd_mem);
KUNIT_EXPECT_EQ(test, bo->resource->placement,
- DRM_BUDDY_TOPDOWN_ALLOCATION);
+ GPU_BUDDY_TOPDOWN_ALLOCATION);
ttm_bo_fini(bo);
ttm_mock_manager_fini(priv->ttm_dev, snd_mem);
diff --git a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c
index dd395229e388..294d56d9067e 100644
--- a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c
+++ b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.c
@@ -31,7 +31,7 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man,
{
struct ttm_mock_manager *manager = to_mock_mgr(man);
struct ttm_mock_resource *mock_res;
- struct drm_buddy *mm = &manager->mm;
+ struct gpu_buddy *mm = &manager->mm;
u64 lpfn, fpfn, alloc_size;
int err;
@@ -47,14 +47,14 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man,
INIT_LIST_HEAD(&mock_res->blocks);
if (place->flags & TTM_PL_FLAG_TOPDOWN)
- mock_res->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
+ mock_res->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
if (place->flags & TTM_PL_FLAG_CONTIGUOUS)
- mock_res->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
+ mock_res->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
alloc_size = (uint64_t)mock_res->base.size;
mutex_lock(&manager->lock);
- err = drm_buddy_alloc_blocks(mm, fpfn, lpfn, alloc_size,
+ err = gpu_buddy_alloc_blocks(mm, fpfn, lpfn, alloc_size,
manager->default_page_size,
&mock_res->blocks,
mock_res->flags);
@@ -67,7 +67,7 @@ static int ttm_mock_manager_alloc(struct ttm_resource_manager *man,
return 0;
error_free_blocks:
- drm_buddy_free_list(mm, &mock_res->blocks, 0);
+ gpu_buddy_free_list(mm, &mock_res->blocks, 0);
ttm_resource_fini(man, &mock_res->base);
mutex_unlock(&manager->lock);
@@ -79,10 +79,10 @@ static void ttm_mock_manager_free(struct ttm_resource_manager *man,
{
struct ttm_mock_manager *manager = to_mock_mgr(man);
struct ttm_mock_resource *mock_res = to_mock_mgr_resource(res);
- struct drm_buddy *mm = &manager->mm;
+ struct gpu_buddy *mm = &manager->mm;
mutex_lock(&manager->lock);
- drm_buddy_free_list(mm, &mock_res->blocks, 0);
+ gpu_buddy_free_list(mm, &mock_res->blocks, 0);
mutex_unlock(&manager->lock);
ttm_resource_fini(man, res);
@@ -106,7 +106,7 @@ int ttm_mock_manager_init(struct ttm_device *bdev, u32 mem_type, u32 size)
mutex_init(&manager->lock);
- err = drm_buddy_init(&manager->mm, size, PAGE_SIZE);
+ err = gpu_buddy_init(&manager->mm, size, PAGE_SIZE);
if (err) {
kfree(manager);
@@ -142,7 +142,7 @@ void ttm_mock_manager_fini(struct ttm_device *bdev, u32 mem_type)
ttm_resource_manager_set_used(man, false);
mutex_lock(&mock_man->lock);
- drm_buddy_fini(&mock_man->mm);
+ gpu_buddy_fini(&mock_man->mm);
mutex_unlock(&mock_man->lock);
ttm_set_driver_manager(bdev, mem_type, NULL);
diff --git a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h
index e4c95f86a467..08710756fd8e 100644
--- a/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h
+++ b/drivers/gpu/drm/ttm/tests/ttm_mock_manager.h
@@ -5,11 +5,11 @@
#ifndef TTM_MOCK_MANAGER_H
#define TTM_MOCK_MANAGER_H
-#include <drm/drm_buddy.h>
+#include <linux/gpu_buddy.h>
struct ttm_mock_manager {
struct ttm_resource_manager man;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
u64 default_page_size;
/* protects allocations of mock buffer objects */
struct mutex lock;
diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
index 4d7dcaff2b91..789baa0114e4 100644
--- a/drivers/gpu/drm/xe/Kconfig
+++ b/drivers/gpu/drm/xe/Kconfig
@@ -11,6 +11,7 @@ config DRM_XE
# the shmem_readpage() which depends upon tmpfs
select SHMEM
select TMPFS
+ select GPU_BUDDY
select DRM_BUDDY
select DRM_CLIENT_SELECTION
select DRM_KMS_HELPER
diff --git a/drivers/gpu/drm/xe/xe_res_cursor.h b/drivers/gpu/drm/xe/xe_res_cursor.h
index 4e00008b7081..5f4ab08c0686 100644
--- a/drivers/gpu/drm/xe/xe_res_cursor.h
+++ b/drivers/gpu/drm/xe/xe_res_cursor.h
@@ -58,7 +58,7 @@ struct xe_res_cursor {
/** @dma_addr: Current element in a struct drm_pagemap_addr array */
const struct drm_pagemap_addr *dma_addr;
/** @mm: Buddy allocator for VRAM cursor */
- struct drm_buddy *mm;
+ struct gpu_buddy *mm;
/**
* @dma_start: DMA start address for the current segment.
* This may be different to @dma_addr.addr since elements in
@@ -69,7 +69,7 @@ struct xe_res_cursor {
u64 dma_seg_size;
};
-static struct drm_buddy *xe_res_get_buddy(struct ttm_resource *res)
+static struct gpu_buddy *xe_res_get_buddy(struct ttm_resource *res)
{
struct ttm_resource_manager *mgr;
@@ -104,30 +104,30 @@ static inline void xe_res_first(struct ttm_resource *res,
case XE_PL_STOLEN:
case XE_PL_VRAM0:
case XE_PL_VRAM1: {
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
struct list_head *head, *next;
- struct drm_buddy *mm = xe_res_get_buddy(res);
+ struct gpu_buddy *mm = xe_res_get_buddy(res);
head = &to_xe_ttm_vram_mgr_resource(res)->blocks;
block = list_first_entry_or_null(head,
- struct drm_buddy_block,
+ struct gpu_buddy_block,
link);
if (!block)
goto fallback;
- while (start >= drm_buddy_block_size(mm, block)) {
- start -= drm_buddy_block_size(mm, block);
+ while (start >= gpu_buddy_block_size(mm, block)) {
+ start -= gpu_buddy_block_size(mm, block);
next = block->link.next;
if (next != head)
- block = list_entry(next, struct drm_buddy_block,
+ block = list_entry(next, struct gpu_buddy_block,
link);
}
cur->mm = mm;
- cur->start = drm_buddy_block_offset(block) + start;
- cur->size = min(drm_buddy_block_size(mm, block) - start,
+ cur->start = gpu_buddy_block_offset(block) + start;
+ cur->size = min(gpu_buddy_block_size(mm, block) - start,
size);
cur->remaining = size;
cur->node = block;
@@ -259,7 +259,7 @@ static inline void xe_res_first_dma(const struct drm_pagemap_addr *dma_addr,
*/
static inline void xe_res_next(struct xe_res_cursor *cur, u64 size)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
struct list_head *next;
u64 start;
@@ -295,18 +295,18 @@ static inline void xe_res_next(struct xe_res_cursor *cur, u64 size)
block = cur->node;
next = block->link.next;
- block = list_entry(next, struct drm_buddy_block, link);
+ block = list_entry(next, struct gpu_buddy_block, link);
- while (start >= drm_buddy_block_size(cur->mm, block)) {
- start -= drm_buddy_block_size(cur->mm, block);
+ while (start >= gpu_buddy_block_size(cur->mm, block)) {
+ start -= gpu_buddy_block_size(cur->mm, block);
next = block->link.next;
- block = list_entry(next, struct drm_buddy_block, link);
+ block = list_entry(next, struct gpu_buddy_block, link);
}
- cur->start = drm_buddy_block_offset(block) + start;
- cur->size = min(drm_buddy_block_size(cur->mm, block) - start,
+ cur->start = gpu_buddy_block_offset(block) + start;
+ cur->size = min(gpu_buddy_block_size(cur->mm, block) - start,
cur->remaining);
cur->node = block;
break;
diff --git a/drivers/gpu/drm/xe/xe_svm.c b/drivers/gpu/drm/xe/xe_svm.c
index 213f0334518a..cda3bf7e2418 100644
--- a/drivers/gpu/drm/xe/xe_svm.c
+++ b/drivers/gpu/drm/xe/xe_svm.c
@@ -747,7 +747,7 @@ static u64 block_offset_to_pfn(struct drm_pagemap *dpagemap, u64 offset)
return PHYS_PFN(offset + xpagemap->hpa_base);
}
-static struct drm_buddy *vram_to_buddy(struct xe_vram_region *vram)
+static struct gpu_buddy *vram_to_buddy(struct xe_vram_region *vram)
{
return &vram->ttm.mm;
}
@@ -758,17 +758,17 @@ static int xe_svm_populate_devmem_pfn(struct drm_pagemap_devmem *devmem_allocati
struct xe_bo *bo = to_xe_bo(devmem_allocation);
struct ttm_resource *res = bo->ttm.resource;
struct list_head *blocks = &to_xe_ttm_vram_mgr_resource(res)->blocks;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
int j = 0;
list_for_each_entry(block, blocks, link) {
struct xe_vram_region *vr = block->private;
- struct drm_buddy *buddy = vram_to_buddy(vr);
+ struct gpu_buddy *buddy = vram_to_buddy(vr);
u64 block_pfn = block_offset_to_pfn(devmem_allocation->dpagemap,
- drm_buddy_block_offset(block));
+ gpu_buddy_block_offset(block));
int i;
- for (i = 0; i < drm_buddy_block_size(buddy, block) >> PAGE_SHIFT; ++i)
+ for (i = 0; i < gpu_buddy_block_size(buddy, block) >> PAGE_SHIFT; ++i)
pfn[j++] = block_pfn + i;
}
@@ -1033,7 +1033,7 @@ static int xe_drm_pagemap_populate_mm(struct drm_pagemap *dpagemap,
struct dma_fence *pre_migrate_fence = NULL;
struct xe_device *xe = vr->xe;
struct device *dev = xe->drm.dev;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
struct xe_validation_ctx vctx;
struct list_head *blocks;
struct drm_exec exec;
diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
index d6aa61e55f4d..4ebc0fd180d3 100644
--- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
+++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c
@@ -4,8 +4,9 @@
* Copyright (C) 2021-2022 Red Hat
*/
-#include <drm/drm_managed.h>
+#include <drm/drm_buddy.h>
#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
#include <drm/ttm/ttm_placement.h>
#include <drm/ttm/ttm_range_manager.h>
@@ -16,16 +17,16 @@
#include "xe_ttm_vram_mgr.h"
#include "xe_vram_types.h"
-static inline struct drm_buddy_block *
+static inline struct gpu_buddy_block *
xe_ttm_vram_mgr_first_block(struct list_head *list)
{
- return list_first_entry_or_null(list, struct drm_buddy_block, link);
+ return list_first_entry_or_null(list, struct gpu_buddy_block, link);
}
-static inline bool xe_is_vram_mgr_blocks_contiguous(struct drm_buddy *mm,
+static inline bool xe_is_vram_mgr_blocks_contiguous(struct gpu_buddy *mm,
struct list_head *head)
{
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
u64 start, size;
block = xe_ttm_vram_mgr_first_block(head);
@@ -33,12 +34,12 @@ static inline bool xe_is_vram_mgr_blocks_contiguous(struct drm_buddy *mm,
return false;
while (head != block->link.next) {
- start = drm_buddy_block_offset(block);
- size = drm_buddy_block_size(mm, block);
+ start = gpu_buddy_block_offset(block);
+ size = gpu_buddy_block_size(mm, block);
- block = list_entry(block->link.next, struct drm_buddy_block,
+ block = list_entry(block->link.next, struct gpu_buddy_block,
link);
- if (start + size != drm_buddy_block_offset(block))
+ if (start + size != gpu_buddy_block_offset(block))
return false;
}
@@ -52,7 +53,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
{
struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
struct xe_ttm_vram_mgr_resource *vres;
- struct drm_buddy *mm = &mgr->mm;
+ struct gpu_buddy *mm = &mgr->mm;
u64 size, min_page_size;
unsigned long lpfn;
int err;
@@ -79,13 +80,13 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
INIT_LIST_HEAD(&vres->blocks);
if (place->flags & TTM_PL_FLAG_TOPDOWN)
- vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
+ vres->flags |= GPU_BUDDY_TOPDOWN_ALLOCATION;
if (place->flags & TTM_PL_FLAG_CONTIGUOUS)
- vres->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
+ vres->flags |= GPU_BUDDY_CONTIGUOUS_ALLOCATION;
if (place->fpfn || lpfn != man->size >> PAGE_SHIFT)
- vres->flags |= DRM_BUDDY_RANGE_ALLOCATION;
+ vres->flags |= GPU_BUDDY_RANGE_ALLOCATION;
if (WARN_ON(!vres->base.size)) {
err = -EINVAL;
@@ -113,7 +114,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
goto error_unlock;
}
- err = drm_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
+ err = gpu_buddy_alloc_blocks(mm, (u64)place->fpfn << PAGE_SHIFT,
(u64)lpfn << PAGE_SHIFT, size,
min_page_size, &vres->blocks, vres->flags);
if (err)
@@ -122,13 +123,13 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
if (lpfn <= mgr->visible_size >> PAGE_SHIFT) {
vres->used_visible_size = size;
} else {
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
list_for_each_entry(block, &vres->blocks, link) {
- u64 start = drm_buddy_block_offset(block);
+ u64 start = gpu_buddy_block_offset(block);
if (start < mgr->visible_size) {
- u64 end = start + drm_buddy_block_size(mm, block);
+ u64 end = start + gpu_buddy_block_size(mm, block);
vres->used_visible_size +=
min(end, mgr->visible_size) - start;
@@ -148,11 +149,11 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man,
* the object.
*/
if (vres->base.placement & TTM_PL_FLAG_CONTIGUOUS) {
- struct drm_buddy_block *block = list_first_entry(&vres->blocks,
+ struct gpu_buddy_block *block = list_first_entry(&vres->blocks,
typeof(*block),
link);
- vres->base.start = drm_buddy_block_offset(block) >> PAGE_SHIFT;
+ vres->base.start = gpu_buddy_block_offset(block) >> PAGE_SHIFT;
} else {
vres->base.start = XE_BO_INVALID_OFFSET;
}
@@ -174,10 +175,10 @@ static void xe_ttm_vram_mgr_del(struct ttm_resource_manager *man,
struct xe_ttm_vram_mgr_resource *vres =
to_xe_ttm_vram_mgr_resource(res);
struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
- struct drm_buddy *mm = &mgr->mm;
+ struct gpu_buddy *mm = &mgr->mm;
mutex_lock(&mgr->lock);
- drm_buddy_free_list(mm, &vres->blocks, 0);
+ gpu_buddy_free_list(mm, &vres->blocks, 0);
mgr->visible_avail += vres->used_visible_size;
mutex_unlock(&mgr->lock);
@@ -190,7 +191,7 @@ static void xe_ttm_vram_mgr_debug(struct ttm_resource_manager *man,
struct drm_printer *printer)
{
struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
- struct drm_buddy *mm = &mgr->mm;
+ struct gpu_buddy *mm = &mgr->mm;
mutex_lock(&mgr->lock);
drm_printf(printer, "default_page_size: %lluKiB\n",
@@ -213,8 +214,8 @@ static bool xe_ttm_vram_mgr_intersects(struct ttm_resource_manager *man,
struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
struct xe_ttm_vram_mgr_resource *vres =
to_xe_ttm_vram_mgr_resource(res);
- struct drm_buddy *mm = &mgr->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = &mgr->mm;
+ struct gpu_buddy_block *block;
if (!place->fpfn && !place->lpfn)
return true;
@@ -224,9 +225,9 @@ static bool xe_ttm_vram_mgr_intersects(struct ttm_resource_manager *man,
list_for_each_entry(block, &vres->blocks, link) {
unsigned long fpfn =
- drm_buddy_block_offset(block) >> PAGE_SHIFT;
+ gpu_buddy_block_offset(block) >> PAGE_SHIFT;
unsigned long lpfn = fpfn +
- (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
+ (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
if (place->fpfn < lpfn && place->lpfn > fpfn)
return true;
@@ -243,8 +244,8 @@ static bool xe_ttm_vram_mgr_compatible(struct ttm_resource_manager *man,
struct xe_ttm_vram_mgr *mgr = to_xe_ttm_vram_mgr(man);
struct xe_ttm_vram_mgr_resource *vres =
to_xe_ttm_vram_mgr_resource(res);
- struct drm_buddy *mm = &mgr->mm;
- struct drm_buddy_block *block;
+ struct gpu_buddy *mm = &mgr->mm;
+ struct gpu_buddy_block *block;
if (!place->fpfn && !place->lpfn)
return true;
@@ -254,9 +255,9 @@ static bool xe_ttm_vram_mgr_compatible(struct ttm_resource_manager *man,
list_for_each_entry(block, &vres->blocks, link) {
unsigned long fpfn =
- drm_buddy_block_offset(block) >> PAGE_SHIFT;
+ gpu_buddy_block_offset(block) >> PAGE_SHIFT;
unsigned long lpfn = fpfn +
- (drm_buddy_block_size(mm, block) >> PAGE_SHIFT);
+ (gpu_buddy_block_size(mm, block) >> PAGE_SHIFT);
if (fpfn < place->fpfn || lpfn > place->lpfn)
return false;
@@ -286,7 +287,7 @@ static void xe_ttm_vram_mgr_fini(struct drm_device *dev, void *arg)
WARN_ON_ONCE(mgr->visible_avail != mgr->visible_size);
- drm_buddy_fini(&mgr->mm);
+ gpu_buddy_fini(&mgr->mm);
ttm_resource_manager_cleanup(&mgr->manager);
@@ -317,7 +318,7 @@ int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr,
mgr->visible_avail = io_size;
ttm_resource_manager_init(man, &xe->ttm, size);
- err = drm_buddy_init(&mgr->mm, man->size, default_page_size);
+ err = gpu_buddy_init(&mgr->mm, man->size, default_page_size);
if (err)
return err;
@@ -365,7 +366,7 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe,
if (!*sgt)
return -ENOMEM;
- /* Determine the number of DRM_BUDDY blocks to export */
+ /* Determine the number of GPU_BUDDY blocks to export */
xe_res_first(res, offset, length, &cursor);
while (cursor.remaining) {
num_entries++;
@@ -382,10 +383,10 @@ int xe_ttm_vram_mgr_alloc_sgt(struct xe_device *xe,
sg->length = 0;
/*
- * Walk down DRM_BUDDY blocks to populate scatterlist nodes
- * @note: Use iterator api to get first the DRM_BUDDY block
+ * Walk down GPU_BUDDY blocks to populate scatterlist nodes
+ * @note: Use iterator api to get first the GPU_BUDDY block
* and the number of bytes from it. Access the following
- * DRM_BUDDY block(s) if more buffer needs to exported
+ * GPU_BUDDY block(s) if more buffer needs to exported
*/
xe_res_first(res, offset, length, &cursor);
for_each_sgtable_sg((*sgt), sg, i) {
diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
index a71e14818ec2..9106da056b49 100644
--- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
+++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
@@ -6,7 +6,7 @@
#ifndef _XE_TTM_VRAM_MGR_TYPES_H_
#define _XE_TTM_VRAM_MGR_TYPES_H_
-#include <drm/drm_buddy.h>
+#include <linux/gpu_buddy.h>
#include <drm/ttm/ttm_device.h>
/**
@@ -18,7 +18,7 @@ struct xe_ttm_vram_mgr {
/** @manager: Base TTM resource manager */
struct ttm_resource_manager manager;
/** @mm: DRM buddy allocator which manages the VRAM */
- struct drm_buddy mm;
+ struct gpu_buddy mm;
/** @visible_size: Proped size of the CPU visible portion */
u64 visible_size;
/** @visible_avail: CPU visible portion still unallocated */
diff --git a/drivers/gpu/tests/Makefile b/drivers/gpu/tests/Makefile
new file mode 100644
index 000000000000..31a5ff44cb4e
--- /dev/null
+++ b/drivers/gpu/tests/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_GPU_BUDDY_KUNIT_TEST) += gpu_buddy_test.o gpu_random.o
diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/tests/gpu_buddy_test.c
similarity index 67%
rename from drivers/gpu/drm/tests/drm_buddy_test.c
rename to drivers/gpu/tests/gpu_buddy_test.c
index e6f8459c6c54..450e71deed90 100644
--- a/drivers/gpu/drm/tests/drm_buddy_test.c
+++ b/drivers/gpu/tests/gpu_buddy_test.c
@@ -10,9 +10,9 @@
#include <linux/sched/signal.h>
#include <linux/sizes.h>
-#include <drm/drm_buddy.h>
+#include <linux/gpu_buddy.h>
-#include "../lib/drm_random.h"
+#include "gpu_random.h"
static unsigned int random_seed;
@@ -21,9 +21,9 @@ static inline u64 get_size(int order, u64 chunk_size)
return (1 << order) * chunk_size;
}
-static void drm_test_buddy_fragmentation_performance(struct kunit *test)
+static void gpu_test_buddy_fragmentation_performance(struct kunit *test)
{
- struct drm_buddy_block *block, *tmp;
+ struct gpu_buddy_block *block, *tmp;
int num_blocks, i, ret, count = 0;
LIST_HEAD(allocated_blocks);
unsigned long elapsed_ms;
@@ -32,7 +32,7 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
LIST_HEAD(clear_list);
LIST_HEAD(dirty_list);
LIST_HEAD(free_list);
- struct drm_buddy mm;
+ struct gpu_buddy mm;
u64 mm_size = SZ_4G;
ktime_t start, end;
@@ -47,7 +47,7 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
* quickly the allocator can satisfy larger, aligned requests from a pool of
* highly fragmented space.
*/
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
"buddy_init failed\n");
num_blocks = mm_size / SZ_64K;
@@ -55,7 +55,7 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
start = ktime_get();
/* Allocate with maximum fragmentation - 8K blocks with 64K alignment */
for (i = 0; i < num_blocks; i++)
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
&allocated_blocks, 0),
"buddy_alloc hit an error size=%u\n", SZ_8K);
@@ -68,21 +68,21 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
}
/* Free with different flags to ensure no coalescing */
- drm_buddy_free_list(&mm, &clear_list, DRM_BUDDY_CLEARED);
- drm_buddy_free_list(&mm, &dirty_list, 0);
+ gpu_buddy_free_list(&mm, &clear_list, GPU_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &dirty_list, 0);
for (i = 0; i < num_blocks; i++)
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, SZ_64K, SZ_64K,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, SZ_64K, SZ_64K,
&test_blocks, 0),
"buddy_alloc hit an error size=%u\n", SZ_64K);
- drm_buddy_free_list(&mm, &test_blocks, 0);
+ gpu_buddy_free_list(&mm, &test_blocks, 0);
end = ktime_get();
elapsed_ms = ktime_to_ms(ktime_sub(end, start));
kunit_info(test, "Fragmented allocation took %lu ms\n", elapsed_ms);
- drm_buddy_fini(&mm);
+ gpu_buddy_fini(&mm);
/*
* Reverse free order under fragmentation
@@ -96,13 +96,13 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
* deallocation occurs in the opposite order of allocation, exposing the
* cost difference between a linear freelist scan and an ordered tree lookup.
*/
- ret = drm_buddy_init(&mm, mm_size, SZ_4K);
+ ret = gpu_buddy_init(&mm, mm_size, SZ_4K);
KUNIT_ASSERT_EQ(test, ret, 0);
start = ktime_get();
/* Allocate maximum fragmentation */
for (i = 0; i < num_blocks; i++)
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, SZ_8K, SZ_64K,
&allocated_blocks, 0),
"buddy_alloc hit an error size=%u\n", SZ_8K);
@@ -111,28 +111,28 @@ static void drm_test_buddy_fragmentation_performance(struct kunit *test)
list_move_tail(&block->link, &free_list);
count++;
}
- drm_buddy_free_list(&mm, &free_list, DRM_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &free_list, GPU_BUDDY_CLEARED);
list_for_each_entry_safe_reverse(block, tmp, &allocated_blocks, link)
list_move(&block->link, &reverse_list);
- drm_buddy_free_list(&mm, &reverse_list, DRM_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &reverse_list, GPU_BUDDY_CLEARED);
end = ktime_get();
elapsed_ms = ktime_to_ms(ktime_sub(end, start));
kunit_info(test, "Reverse-ordered free took %lu ms\n", elapsed_ms);
- drm_buddy_fini(&mm);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_range_bias(struct kunit *test)
+static void gpu_test_buddy_alloc_range_bias(struct kunit *test)
{
u32 mm_size, size, ps, bias_size, bias_start, bias_end, bias_rem;
- DRM_RND_STATE(prng, random_seed);
+ GPU_RND_STATE(prng, random_seed);
unsigned int i, count, *order;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
unsigned long flags;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
LIST_HEAD(allocated);
bias_size = SZ_1M;
@@ -142,11 +142,11 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
kunit_info(test, "mm_size=%u, ps=%u\n", mm_size, ps);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps),
"buddy_init failed\n");
count = mm_size / bias_size;
- order = drm_random_order(count, &prng);
+ order = gpu_random_order(count, &prng);
KUNIT_EXPECT_TRUE(test, order);
/*
@@ -166,79 +166,79 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
/* internal round_up too big */
KUNIT_ASSERT_TRUE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, bias_size + ps, bias_size,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, bias_size, bias_size);
/* size too big */
KUNIT_ASSERT_TRUE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, bias_size + ps, ps,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, bias_size + ps, ps);
/* bias range too small for size */
KUNIT_ASSERT_TRUE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start + ps,
+ gpu_buddy_alloc_blocks(&mm, bias_start + ps,
bias_end, bias_size, ps,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n",
bias_start + ps, bias_end, bias_size, ps);
/* bias misaligned */
KUNIT_ASSERT_TRUE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start + ps,
+ gpu_buddy_alloc_blocks(&mm, bias_start + ps,
bias_end - ps,
bias_size >> 1, bias_size >> 1,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc h didn't fail with bias(%x-%x), size=%u, ps=%u\n",
bias_start + ps, bias_end - ps, bias_size >> 1, bias_size >> 1);
/* single big page */
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, bias_size, bias_size,
&tmp,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc i failed with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, bias_size, bias_size);
- drm_buddy_free_list(&mm, &tmp, 0);
+ gpu_buddy_free_list(&mm, &tmp, 0);
/* single page with internal round_up */
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, ps, bias_size,
&tmp,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, ps, bias_size);
- drm_buddy_free_list(&mm, &tmp, 0);
+ gpu_buddy_free_list(&mm, &tmp, 0);
/* random size within */
size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
if (size)
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, size, ps,
&tmp,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, size, ps);
bias_rem -= size;
/* too big for current avail */
KUNIT_ASSERT_TRUE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, bias_rem + ps, ps,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc didn't fail with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, bias_rem + ps, ps);
@@ -248,10 +248,10 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
size = max(size, ps);
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, size, ps,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, size, ps);
/*
@@ -259,15 +259,15 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
* unallocated, and ideally not always on the bias
* boundaries.
*/
- drm_buddy_free_list(&mm, &tmp, 0);
+ gpu_buddy_free_list(&mm, &tmp, 0);
} else {
list_splice_tail(&tmp, &allocated);
}
}
kfree(order);
- drm_buddy_free_list(&mm, &allocated, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &allocated, 0);
+ gpu_buddy_fini(&mm);
/*
* Something more free-form. Idea is to pick a random starting bias
@@ -278,7 +278,7 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
* allocated nodes in the middle of the address space.
*/
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps),
"buddy_init failed\n");
bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps);
@@ -290,10 +290,10 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
u32 size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, size, ps,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc failed with bias(%x-%x), size=%u, ps=%u\n",
bias_start, bias_end, size, ps);
bias_rem -= size;
@@ -319,24 +319,24 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
KUNIT_ASSERT_EQ(test, bias_start, 0);
KUNIT_ASSERT_EQ(test, bias_end, mm_size);
KUNIT_ASSERT_TRUE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start, bias_end,
+ gpu_buddy_alloc_blocks(&mm, bias_start, bias_end,
ps, ps,
&allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc passed with bias(%x-%x), size=%u\n",
bias_start, bias_end, ps);
- drm_buddy_free_list(&mm, &allocated, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &allocated, 0);
+ gpu_buddy_fini(&mm);
/*
- * Allocate cleared blocks in the bias range when the DRM buddy's clear avail is
+ * Allocate cleared blocks in the bias range when the GPU buddy's clear avail is
* zero. This will validate the bias range allocation in scenarios like system boot
* when no cleared blocks are available and exercise the fallback path too. The resulting
* blocks should always be dirty.
*/
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, ps),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, ps),
"buddy_init failed\n");
bias_start = round_up(prandom_u32_state(&prng) % (mm_size - ps), ps);
@@ -344,11 +344,11 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
bias_end = max(bias_end, bias_start + ps);
bias_rem = bias_end - bias_start;
- flags = DRM_BUDDY_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION;
+ flags = GPU_BUDDY_CLEAR_ALLOCATION | GPU_BUDDY_RANGE_ALLOCATION;
size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, bias_start,
+ gpu_buddy_alloc_blocks(&mm, bias_start,
bias_end, size, ps,
&allocated,
flags),
@@ -356,27 +356,27 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
bias_start, bias_end, size, ps);
list_for_each_entry(block, &allocated, link)
- KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
+ KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false);
- drm_buddy_free_list(&mm, &allocated, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &allocated, 0);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_clear(struct kunit *test)
+static void gpu_test_buddy_alloc_clear(struct kunit *test)
{
unsigned long n_pages, total, i = 0;
const unsigned long ps = SZ_4K;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
const int max_order = 12;
LIST_HEAD(allocated);
- struct drm_buddy mm;
+ struct gpu_buddy mm;
unsigned int order;
u32 mm_size, size;
LIST_HEAD(dirty);
LIST_HEAD(clean);
mm_size = SZ_4K << max_order;
- KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
+ KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
@@ -389,11 +389,11 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
* is indeed all dirty pages and vice versa. Free it all again,
* keeping the dirty/clear status.
*/
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
5 * ps, ps, &allocated,
- DRM_BUDDY_TOPDOWN_ALLOCATION),
+ GPU_BUDDY_TOPDOWN_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", 5 * ps);
- drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
n_pages = 10;
do {
@@ -406,37 +406,37 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
flags = 0;
} else {
list = &clean;
- flags = DRM_BUDDY_CLEAR_ALLOCATION;
+ flags = GPU_BUDDY_CLEAR_ALLOCATION;
}
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
ps, ps, list,
flags),
"buddy_alloc hit an error size=%lu\n", ps);
} while (++i < n_pages);
list_for_each_entry(block, &clean, link)
- KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), true);
+ KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), true);
list_for_each_entry(block, &dirty, link)
- KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
+ KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false);
- drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED);
/*
* Trying to go over the clear limit for some allocation.
* The allocation should never fail with reasonable page-size.
*/
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
10 * ps, ps, &clean,
- DRM_BUDDY_CLEAR_ALLOCATION),
+ GPU_BUDDY_CLEAR_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", 10 * ps);
- drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
- drm_buddy_free_list(&mm, &dirty, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &dirty, 0);
+ gpu_buddy_fini(&mm);
- KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
+ KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
/*
* Create a new mm. Intentionally fragment the address space by creating
@@ -458,34 +458,34 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
else
list = &clean;
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
ps, ps, list, 0),
"buddy_alloc hit an error size=%lu\n", ps);
} while (++i < n_pages);
- drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
- drm_buddy_free_list(&mm, &dirty, 0);
+ gpu_buddy_free_list(&mm, &clean, GPU_BUDDY_CLEARED);
+ gpu_buddy_free_list(&mm, &dirty, 0);
order = 1;
do {
size = SZ_4K << order;
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
size, size, &allocated,
- DRM_BUDDY_CLEAR_ALLOCATION),
+ GPU_BUDDY_CLEAR_ALLOCATION),
"buddy_alloc hit an error size=%u\n", size);
total = 0;
list_for_each_entry(block, &allocated, link) {
if (size != mm_size)
- KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
- total += drm_buddy_block_size(&mm, block);
+ KUNIT_EXPECT_EQ(test, gpu_buddy_block_is_clear(block), false);
+ total += gpu_buddy_block_size(&mm, block);
}
KUNIT_EXPECT_EQ(test, total, size);
- drm_buddy_free_list(&mm, &allocated, 0);
+ gpu_buddy_free_list(&mm, &allocated, 0);
} while (++order <= max_order);
- drm_buddy_fini(&mm);
+ gpu_buddy_fini(&mm);
/*
* Create a new mm with a non power-of-two size. Allocate a random size from each
@@ -494,44 +494,44 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
*/
mm_size = (SZ_4K << max_order) + (SZ_4K << (max_order - 2));
- KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
+ KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
4 * ps, ps, &allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", 4 * ps);
- drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
+ gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
2 * ps, ps, &allocated,
- DRM_BUDDY_CLEAR_ALLOCATION),
+ GPU_BUDDY_CLEAR_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", 2 * ps);
- drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size,
+ gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size,
ps, ps, &allocated,
- DRM_BUDDY_RANGE_ALLOCATION),
+ GPU_BUDDY_RANGE_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", ps);
- drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &allocated, GPU_BUDDY_CLEARED);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_contiguous(struct kunit *test)
+static void gpu_test_buddy_alloc_contiguous(struct kunit *test)
{
const unsigned long ps = SZ_4K, mm_size = 16 * 3 * SZ_4K;
unsigned long i, n_pages, total;
- struct drm_buddy_block *block;
- struct drm_buddy mm;
+ struct gpu_buddy_block *block;
+ struct gpu_buddy mm;
LIST_HEAD(left);
LIST_HEAD(middle);
LIST_HEAD(right);
LIST_HEAD(allocated);
- KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
+ KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, mm_size, ps));
/*
* Idea is to fragment the address space by alternating block
* allocations between three different lists; one for left, middle and
* right. We can then free a list to simulate fragmentation. In
- * particular we want to exercise the DRM_BUDDY_CONTIGUOUS_ALLOCATION,
+ * particular we want to exercise the GPU_BUDDY_CONTIGUOUS_ALLOCATION,
* including the try_harder path.
*/
@@ -548,66 +548,66 @@ static void drm_test_buddy_alloc_contiguous(struct kunit *test)
else
list = &right;
KUNIT_ASSERT_FALSE_MSG(test,
- drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ gpu_buddy_alloc_blocks(&mm, 0, mm_size,
ps, ps, list, 0),
"buddy_alloc hit an error size=%lu\n",
ps);
} while (++i < n_pages);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
3 * ps, ps, &allocated,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc didn't error size=%lu\n", 3 * ps);
- drm_buddy_free_list(&mm, &middle, 0);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ gpu_buddy_free_list(&mm, &middle, 0);
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
3 * ps, ps, &allocated,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc didn't error size=%lu\n", 3 * ps);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
2 * ps, ps, &allocated,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc didn't error size=%lu\n", 2 * ps);
- drm_buddy_free_list(&mm, &right, 0);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ gpu_buddy_free_list(&mm, &right, 0);
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
3 * ps, ps, &allocated,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc didn't error size=%lu\n", 3 * ps);
/*
* At this point we should have enough contiguous space for 2 blocks,
* however they are never buddies (since we freed middle and right) so
* will require the try_harder logic to find them.
*/
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
2 * ps, ps, &allocated,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", 2 * ps);
- drm_buddy_free_list(&mm, &left, 0);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+ gpu_buddy_free_list(&mm, &left, 0);
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size,
3 * ps, ps, &allocated,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc hit an error size=%lu\n", 3 * ps);
total = 0;
list_for_each_entry(block, &allocated, link)
- total += drm_buddy_block_size(&mm, block);
+ total += gpu_buddy_block_size(&mm, block);
KUNIT_ASSERT_EQ(test, total, ps * 2 + ps * 3);
- drm_buddy_free_list(&mm, &allocated, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &allocated, 0);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_pathological(struct kunit *test)
+static void gpu_test_buddy_alloc_pathological(struct kunit *test)
{
u64 mm_size, size, start = 0;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
const int max_order = 3;
unsigned long flags = 0;
int order, top;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
LIST_HEAD(blocks);
LIST_HEAD(holes);
LIST_HEAD(tmp);
@@ -620,7 +620,7 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test)
*/
mm_size = SZ_4K << max_order;
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
"buddy_init failed\n");
KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
@@ -630,18 +630,18 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test)
block = list_first_entry_or_null(&blocks, typeof(*block), link);
if (block) {
list_del(&block->link);
- drm_buddy_free_block(&mm, block);
+ gpu_buddy_free_block(&mm, block);
}
for (order = top; order--;) {
size = get_size(order, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start,
mm_size, size, size,
&tmp, flags),
"buddy_alloc hit -ENOMEM with order=%d, top=%d\n",
order, top);
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_move_tail(&block->link, &blocks);
@@ -649,45 +649,45 @@ static void drm_test_buddy_alloc_pathological(struct kunit *test)
/* There should be one final page for this sub-allocation */
size = get_size(0, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc hit -ENOMEM for hole\n");
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_move_tail(&block->link, &holes);
size = get_size(top, mm.chunk_size);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc unexpectedly succeeded at top-order %d/%d, it should be full!",
top, max_order);
}
- drm_buddy_free_list(&mm, &holes, 0);
+ gpu_buddy_free_list(&mm, &holes, 0);
/* Nothing larger than blocks of chunk_size now available */
for (order = 1; order <= max_order; order++) {
size = get_size(order, mm.chunk_size);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc unexpectedly succeeded at order %d, it should be full!",
order);
}
list_splice_tail(&holes, &blocks);
- drm_buddy_free_list(&mm, &blocks, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &blocks, 0);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
+static void gpu_test_buddy_alloc_pessimistic(struct kunit *test)
{
u64 mm_size, size, start = 0;
- struct drm_buddy_block *block, *bn;
+ struct gpu_buddy_block *block, *bn;
const unsigned int max_order = 16;
unsigned long flags = 0;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
unsigned int order;
LIST_HEAD(blocks);
LIST_HEAD(tmp);
@@ -699,19 +699,19 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
*/
mm_size = SZ_4K << max_order;
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
"buddy_init failed\n");
KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
for (order = 0; order < max_order; order++) {
size = get_size(order, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc hit -ENOMEM with order=%d\n",
order);
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_move_tail(&block->link, &blocks);
@@ -719,11 +719,11 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
/* And now the last remaining block available */
size = get_size(0, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc hit -ENOMEM on final alloc\n");
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_move_tail(&block->link, &blocks);
@@ -731,58 +731,58 @@ static void drm_test_buddy_alloc_pessimistic(struct kunit *test)
/* Should be completely full! */
for (order = max_order; order--;) {
size = get_size(order, mm.chunk_size);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc unexpectedly succeeded, it should be full!");
}
block = list_last_entry(&blocks, typeof(*block), link);
list_del(&block->link);
- drm_buddy_free_block(&mm, block);
+ gpu_buddy_free_block(&mm, block);
/* As we free in increasing size, we make available larger blocks */
order = 1;
list_for_each_entry_safe(block, bn, &blocks, link) {
list_del(&block->link);
- drm_buddy_free_block(&mm, block);
+ gpu_buddy_free_block(&mm, block);
size = get_size(order, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc hit -ENOMEM with order=%d\n",
order);
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_del(&block->link);
- drm_buddy_free_block(&mm, block);
+ gpu_buddy_free_block(&mm, block);
order++;
}
/* To confirm, now the whole mm should be available */
size = get_size(max_order, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc (realloc) hit -ENOMEM with order=%d\n",
max_order);
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_del(&block->link);
- drm_buddy_free_block(&mm, block);
- drm_buddy_free_list(&mm, &blocks, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_block(&mm, block);
+ gpu_buddy_free_list(&mm, &blocks, 0);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_optimistic(struct kunit *test)
+static void gpu_test_buddy_alloc_optimistic(struct kunit *test)
{
u64 mm_size, size, start = 0;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
unsigned long flags = 0;
const int max_order = 16;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
LIST_HEAD(blocks);
LIST_HEAD(tmp);
int order;
@@ -794,19 +794,19 @@ static void drm_test_buddy_alloc_optimistic(struct kunit *test)
mm_size = SZ_4K * ((1 << (max_order + 1)) - 1);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
"buddy_init failed\n");
KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
for (order = 0; order <= max_order; order++) {
size = get_size(order, mm.chunk_size);
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc hit -ENOMEM with order=%d\n",
order);
- block = list_first_entry_or_null(&tmp, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&tmp, struct gpu_buddy_block, link);
KUNIT_ASSERT_TRUE_MSG(test, block, "alloc_blocks has no blocks\n");
list_move_tail(&block->link, &blocks);
@@ -814,115 +814,115 @@ static void drm_test_buddy_alloc_optimistic(struct kunit *test)
/* Should be completely full! */
size = get_size(0, mm.chunk_size);
- KUNIT_ASSERT_TRUE_MSG(test, drm_buddy_alloc_blocks(&mm, start, mm_size,
+ KUNIT_ASSERT_TRUE_MSG(test, gpu_buddy_alloc_blocks(&mm, start, mm_size,
size, size, &tmp, flags),
"buddy_alloc unexpectedly succeeded, it should be full!");
- drm_buddy_free_list(&mm, &blocks, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &blocks, 0);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_limit(struct kunit *test)
+static void gpu_test_buddy_alloc_limit(struct kunit *test)
{
u64 size = U64_MAX, start = 0;
- struct drm_buddy_block *block;
+ struct gpu_buddy_block *block;
unsigned long flags = 0;
LIST_HEAD(allocated);
- struct drm_buddy mm;
+ struct gpu_buddy mm;
- KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, size, SZ_4K));
+ KUNIT_EXPECT_FALSE(test, gpu_buddy_init(&mm, size, SZ_4K));
- KUNIT_EXPECT_EQ_MSG(test, mm.max_order, DRM_BUDDY_MAX_ORDER,
+ KUNIT_EXPECT_EQ_MSG(test, mm.max_order, GPU_BUDDY_MAX_ORDER,
"mm.max_order(%d) != %d\n", mm.max_order,
- DRM_BUDDY_MAX_ORDER);
+ GPU_BUDDY_MAX_ORDER);
size = mm.chunk_size << mm.max_order;
- KUNIT_EXPECT_FALSE(test, drm_buddy_alloc_blocks(&mm, start, size, size,
+ KUNIT_EXPECT_FALSE(test, gpu_buddy_alloc_blocks(&mm, start, size, size,
mm.chunk_size, &allocated, flags));
- block = list_first_entry_or_null(&allocated, struct drm_buddy_block, link);
+ block = list_first_entry_or_null(&allocated, struct gpu_buddy_block, link);
KUNIT_EXPECT_TRUE(test, block);
- KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_order(block), mm.max_order,
+ KUNIT_EXPECT_EQ_MSG(test, gpu_buddy_block_order(block), mm.max_order,
"block order(%d) != %d\n",
- drm_buddy_block_order(block), mm.max_order);
+ gpu_buddy_block_order(block), mm.max_order);
- KUNIT_EXPECT_EQ_MSG(test, drm_buddy_block_size(&mm, block),
+ KUNIT_EXPECT_EQ_MSG(test, gpu_buddy_block_size(&mm, block),
BIT_ULL(mm.max_order) * mm.chunk_size,
"block size(%llu) != %llu\n",
- drm_buddy_block_size(&mm, block),
+ gpu_buddy_block_size(&mm, block),
BIT_ULL(mm.max_order) * mm.chunk_size);
- drm_buddy_free_list(&mm, &allocated, 0);
- drm_buddy_fini(&mm);
+ gpu_buddy_free_list(&mm, &allocated, 0);
+ gpu_buddy_fini(&mm);
}
-static void drm_test_buddy_alloc_exceeds_max_order(struct kunit *test)
+static void gpu_test_buddy_alloc_exceeds_max_order(struct kunit *test)
{
u64 mm_size = SZ_8G + SZ_2G, size = SZ_8G + SZ_1G, min_block_size = SZ_8G;
- struct drm_buddy mm;
+ struct gpu_buddy mm;
LIST_HEAD(blocks);
int err;
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_init(&mm, mm_size, SZ_4K),
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_init(&mm, mm_size, SZ_4K),
"buddy_init failed\n");
/* CONTIGUOUS allocation should succeed via try_harder fallback */
- KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size, size,
+ KUNIT_ASSERT_FALSE_MSG(test, gpu_buddy_alloc_blocks(&mm, 0, mm_size, size,
SZ_4K, &blocks,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION),
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION),
"buddy_alloc hit an error size=%llu\n", size);
- drm_buddy_free_list(&mm, &blocks, 0);
+ gpu_buddy_free_list(&mm, &blocks, 0);
/* Non-CONTIGUOUS with large min_block_size should return -EINVAL */
- err = drm_buddy_alloc_blocks(&mm, 0, mm_size, size, min_block_size, &blocks, 0);
+ err = gpu_buddy_alloc_blocks(&mm, 0, mm_size, size, min_block_size, &blocks, 0);
KUNIT_EXPECT_EQ(test, err, -EINVAL);
/* Non-CONTIGUOUS + RANGE with large min_block_size should return -EINVAL */
- err = drm_buddy_alloc_blocks(&mm, 0, mm_size, size, min_block_size, &blocks,
- DRM_BUDDY_RANGE_ALLOCATION);
+ err = gpu_buddy_alloc_blocks(&mm, 0, mm_size, size, min_block_size, &blocks,
+ GPU_BUDDY_RANGE_ALLOCATION);
KUNIT_EXPECT_EQ(test, err, -EINVAL);
/* CONTIGUOUS + RANGE should return -EINVAL (no try_harder for RANGE) */
- err = drm_buddy_alloc_blocks(&mm, 0, mm_size, size, SZ_4K, &blocks,
- DRM_BUDDY_CONTIGUOUS_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION);
+ err = gpu_buddy_alloc_blocks(&mm, 0, mm_size, size, SZ_4K, &blocks,
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION | GPU_BUDDY_RANGE_ALLOCATION);
KUNIT_EXPECT_EQ(test, err, -EINVAL);
- drm_buddy_fini(&mm);
+ gpu_buddy_fini(&mm);
}
-static int drm_buddy_suite_init(struct kunit_suite *suite)
+static int gpu_buddy_suite_init(struct kunit_suite *suite)
{
while (!random_seed)
random_seed = get_random_u32();
- kunit_info(suite, "Testing DRM buddy manager, with random_seed=0x%x\n",
+ kunit_info(suite, "Testing GPU buddy manager, with random_seed=0x%x\n",
random_seed);
return 0;
}
-static struct kunit_case drm_buddy_tests[] = {
- KUNIT_CASE(drm_test_buddy_alloc_limit),
- KUNIT_CASE(drm_test_buddy_alloc_optimistic),
- KUNIT_CASE(drm_test_buddy_alloc_pessimistic),
- KUNIT_CASE(drm_test_buddy_alloc_pathological),
- KUNIT_CASE(drm_test_buddy_alloc_contiguous),
- KUNIT_CASE(drm_test_buddy_alloc_clear),
- KUNIT_CASE(drm_test_buddy_alloc_range_bias),
- KUNIT_CASE(drm_test_buddy_fragmentation_performance),
- KUNIT_CASE(drm_test_buddy_alloc_exceeds_max_order),
+static struct kunit_case gpu_buddy_tests[] = {
+ KUNIT_CASE(gpu_test_buddy_alloc_limit),
+ KUNIT_CASE(gpu_test_buddy_alloc_optimistic),
+ KUNIT_CASE(gpu_test_buddy_alloc_pessimistic),
+ KUNIT_CASE(gpu_test_buddy_alloc_pathological),
+ KUNIT_CASE(gpu_test_buddy_alloc_contiguous),
+ KUNIT_CASE(gpu_test_buddy_alloc_clear),
+ KUNIT_CASE(gpu_test_buddy_alloc_range_bias),
+ KUNIT_CASE(gpu_test_buddy_fragmentation_performance),
+ KUNIT_CASE(gpu_test_buddy_alloc_exceeds_max_order),
{}
};
-static struct kunit_suite drm_buddy_test_suite = {
- .name = "drm_buddy",
- .suite_init = drm_buddy_suite_init,
- .test_cases = drm_buddy_tests,
+static struct kunit_suite gpu_buddy_test_suite = {
+ .name = "gpu_buddy",
+ .suite_init = gpu_buddy_suite_init,
+ .test_cases = gpu_buddy_tests,
};
-kunit_test_suite(drm_buddy_test_suite);
+kunit_test_suite(gpu_buddy_test_suite);
MODULE_AUTHOR("Intel Corporation");
-MODULE_DESCRIPTION("Kunit test for drm_buddy functions");
+MODULE_DESCRIPTION("Kunit test for gpu_buddy functions");
MODULE_LICENSE("GPL");
diff --git a/drivers/gpu/tests/gpu_random.c b/drivers/gpu/tests/gpu_random.c
new file mode 100644
index 000000000000..54f1f6a3a6c1
--- /dev/null
+++ b/drivers/gpu/tests/gpu_random.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bitops.h>
+#include <linux/export.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "gpu_random.h"
+
+u32 gpu_prandom_u32_max_state(u32 ep_ro, struct rnd_state *state)
+{
+ return upper_32_bits((u64)prandom_u32_state(state) * ep_ro);
+}
+EXPORT_SYMBOL(gpu_prandom_u32_max_state);
+
+void gpu_random_reorder(unsigned int *order, unsigned int count,
+ struct rnd_state *state)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < count; ++i) {
+ BUILD_BUG_ON(sizeof(unsigned int) > sizeof(u32));
+ j = gpu_prandom_u32_max_state(count, state);
+ swap(order[i], order[j]);
+ }
+}
+EXPORT_SYMBOL(gpu_random_reorder);
+
+unsigned int *gpu_random_order(unsigned int count, struct rnd_state *state)
+{
+ unsigned int *order, i;
+
+ order = kmalloc_array(count, sizeof(*order), GFP_KERNEL);
+ if (!order)
+ return order;
+
+ for (i = 0; i < count; i++)
+ order[i] = i;
+
+ gpu_random_reorder(order, count, state);
+ return order;
+}
+EXPORT_SYMBOL(gpu_random_order);
+
+MODULE_DESCRIPTION("GPU Randomization Utilities");
+MODULE_LICENSE("Dual MIT/GPL");
diff --git a/drivers/gpu/tests/gpu_random.h b/drivers/gpu/tests/gpu_random.h
new file mode 100644
index 000000000000..b68cf3448264
--- /dev/null
+++ b/drivers/gpu/tests/gpu_random.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __GPU_RANDOM_H__
+#define __GPU_RANDOM_H__
+
+/* This is a temporary home for a couple of utility functions that should
+ * be transposed to lib/ at the earliest convenience.
+ */
+
+#include <linux/prandom.h>
+
+#define GPU_RND_STATE_INITIALIZER(seed__) ({ \
+ struct rnd_state state__; \
+ prandom_seed_state(&state__, (seed__)); \
+ state__; \
+})
+
+#define GPU_RND_STATE(name__, seed__) \
+ struct rnd_state name__ = GPU_RND_STATE_INITIALIZER(seed__)
+
+unsigned int *gpu_random_order(unsigned int count,
+ struct rnd_state *state);
+void gpu_random_reorder(unsigned int *order,
+ unsigned int count,
+ struct rnd_state *state);
+u32 gpu_prandom_u32_max_state(u32 ep_ro,
+ struct rnd_state *state);
+
+#endif /* !__GPU_RANDOM_H__ */
diff --git a/drivers/video/Kconfig b/drivers/video/Kconfig
index 9884f003247d..8252a04c3f33 100644
--- a/drivers/video/Kconfig
+++ b/drivers/video/Kconfig
@@ -37,6 +37,8 @@ source "drivers/char/agp/Kconfig"
source "drivers/gpu/vga/Kconfig"
+source "drivers/gpu/Kconfig"
+
source "drivers/gpu/host1x/Kconfig"
source "drivers/gpu/ipu-v3/Kconfig"
source "drivers/gpu/nova-core/Kconfig"
diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h
index b909fa8f810a..3054369bebff 100644
--- a/include/drm/drm_buddy.h
+++ b/include/drm/drm_buddy.h
@@ -6,166 +6,13 @@
#ifndef __DRM_BUDDY_H__
#define __DRM_BUDDY_H__
-#include <linux/bitops.h>
-#include <linux/list.h>
-#include <linux/slab.h>
-#include <linux/sched.h>
-#include <linux/rbtree.h>
+#include <linux/gpu_buddy.h>
struct drm_printer;
-#define DRM_BUDDY_RANGE_ALLOCATION BIT(0)
-#define DRM_BUDDY_TOPDOWN_ALLOCATION BIT(1)
-#define DRM_BUDDY_CONTIGUOUS_ALLOCATION BIT(2)
-#define DRM_BUDDY_CLEAR_ALLOCATION BIT(3)
-#define DRM_BUDDY_CLEARED BIT(4)
-#define DRM_BUDDY_TRIM_DISABLE BIT(5)
-
-struct drm_buddy_block {
-#define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
-#define DRM_BUDDY_HEADER_STATE GENMASK_ULL(11, 10)
-#define DRM_BUDDY_ALLOCATED (1 << 10)
-#define DRM_BUDDY_FREE (2 << 10)
-#define DRM_BUDDY_SPLIT (3 << 10)
-#define DRM_BUDDY_HEADER_CLEAR GENMASK_ULL(9, 9)
-/* Free to be used, if needed in the future */
-#define DRM_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6)
-#define DRM_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0)
- u64 header;
-
- struct drm_buddy_block *left;
- struct drm_buddy_block *right;
- struct drm_buddy_block *parent;
-
- void *private; /* owned by creator */
-
- /*
- * While the block is allocated by the user through drm_buddy_alloc*,
- * the user has ownership of the link, for example to maintain within
- * a list, if so desired. As soon as the block is freed with
- * drm_buddy_free* ownership is given back to the mm.
- */
- union {
- struct rb_node rb;
- struct list_head link;
- };
-
- struct list_head tmp_link;
-};
-
-/* Order-zero must be at least SZ_4K */
-#define DRM_BUDDY_MAX_ORDER (63 - 12)
-
-/*
- * Binary Buddy System.
- *
- * Locking should be handled by the user, a simple mutex around
- * drm_buddy_alloc* and drm_buddy_free* should suffice.
- */
-struct drm_buddy {
- /* Maintain a free list for each order. */
- struct rb_root **free_trees;
-
- /*
- * Maintain explicit binary tree(s) to track the allocation of the
- * address space. This gives us a simple way of finding a buddy block
- * and performing the potentially recursive merge step when freeing a
- * block. Nodes are either allocated or free, in which case they will
- * also exist on the respective free list.
- */
- struct drm_buddy_block **roots;
-
- /*
- * Anything from here is public, and remains static for the lifetime of
- * the mm. Everything above is considered do-not-touch.
- */
- unsigned int n_roots;
- unsigned int max_order;
-
- /* Must be at least SZ_4K */
- u64 chunk_size;
- u64 size;
- u64 avail;
- u64 clear_avail;
-};
-
-static inline u64
-drm_buddy_block_offset(const struct drm_buddy_block *block)
-{
- return block->header & DRM_BUDDY_HEADER_OFFSET;
-}
-
-static inline unsigned int
-drm_buddy_block_order(struct drm_buddy_block *block)
-{
- return block->header & DRM_BUDDY_HEADER_ORDER;
-}
-
-static inline unsigned int
-drm_buddy_block_state(struct drm_buddy_block *block)
-{
- return block->header & DRM_BUDDY_HEADER_STATE;
-}
-
-static inline bool
-drm_buddy_block_is_allocated(struct drm_buddy_block *block)
-{
- return drm_buddy_block_state(block) == DRM_BUDDY_ALLOCATED;
-}
-
-static inline bool
-drm_buddy_block_is_clear(struct drm_buddy_block *block)
-{
- return block->header & DRM_BUDDY_HEADER_CLEAR;
-}
-
-static inline bool
-drm_buddy_block_is_free(struct drm_buddy_block *block)
-{
- return drm_buddy_block_state(block) == DRM_BUDDY_FREE;
-}
-
-static inline bool
-drm_buddy_block_is_split(struct drm_buddy_block *block)
-{
- return drm_buddy_block_state(block) == DRM_BUDDY_SPLIT;
-}
-
-static inline u64
-drm_buddy_block_size(struct drm_buddy *mm,
- struct drm_buddy_block *block)
-{
- return mm->chunk_size << drm_buddy_block_order(block);
-}
-
-int drm_buddy_init(struct drm_buddy *mm, u64 size, u64 chunk_size);
-
-void drm_buddy_fini(struct drm_buddy *mm);
-
-struct drm_buddy_block *
-drm_get_buddy(struct drm_buddy_block *block);
-
-int drm_buddy_alloc_blocks(struct drm_buddy *mm,
- u64 start, u64 end, u64 size,
- u64 min_page_size,
- struct list_head *blocks,
- unsigned long flags);
-
-int drm_buddy_block_trim(struct drm_buddy *mm,
- u64 *start,
- u64 new_size,
- struct list_head *blocks);
-
-void drm_buddy_reset_clear(struct drm_buddy *mm, bool is_clear);
-
-void drm_buddy_free_block(struct drm_buddy *mm, struct drm_buddy_block *block);
-
-void drm_buddy_free_list(struct drm_buddy *mm,
- struct list_head *objects,
- unsigned int flags);
-
-void drm_buddy_print(struct drm_buddy *mm, struct drm_printer *p);
-void drm_buddy_block_print(struct drm_buddy *mm,
- struct drm_buddy_block *block,
+/* DRM-specific GPU Buddy Allocator print helpers */
+void drm_buddy_print(struct gpu_buddy *mm, struct drm_printer *p);
+void drm_buddy_block_print(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block,
struct drm_printer *p);
#endif
diff --git a/include/linux/gpu_buddy.h b/include/linux/gpu_buddy.h
new file mode 100644
index 000000000000..3e4bd11ccb71
--- /dev/null
+++ b/include/linux/gpu_buddy.h
@@ -0,0 +1,177 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+#ifndef __GPU_BUDDY_H__
+#define __GPU_BUDDY_H__
+
+#include <linux/bitops.h>
+#include <linux/list.h>
+#include <linux/rbtree.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+
+#define GPU_BUDDY_RANGE_ALLOCATION BIT(0)
+#define GPU_BUDDY_TOPDOWN_ALLOCATION BIT(1)
+#define GPU_BUDDY_CONTIGUOUS_ALLOCATION BIT(2)
+#define GPU_BUDDY_CLEAR_ALLOCATION BIT(3)
+#define GPU_BUDDY_CLEARED BIT(4)
+#define GPU_BUDDY_TRIM_DISABLE BIT(5)
+
+enum gpu_buddy_free_tree {
+ GPU_BUDDY_CLEAR_TREE = 0,
+ GPU_BUDDY_DIRTY_TREE,
+ GPU_BUDDY_MAX_FREE_TREES,
+};
+
+#define for_each_free_tree(tree) \
+ for ((tree) = 0; (tree) < GPU_BUDDY_MAX_FREE_TREES; (tree)++)
+
+struct gpu_buddy_block {
+#define GPU_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
+#define GPU_BUDDY_HEADER_STATE GENMASK_ULL(11, 10)
+#define GPU_BUDDY_ALLOCATED (1 << 10)
+#define GPU_BUDDY_FREE (2 << 10)
+#define GPU_BUDDY_SPLIT (3 << 10)
+#define GPU_BUDDY_HEADER_CLEAR GENMASK_ULL(9, 9)
+/* Free to be used, if needed in the future */
+#define GPU_BUDDY_HEADER_UNUSED GENMASK_ULL(8, 6)
+#define GPU_BUDDY_HEADER_ORDER GENMASK_ULL(5, 0)
+ u64 header;
+
+ struct gpu_buddy_block *left;
+ struct gpu_buddy_block *right;
+ struct gpu_buddy_block *parent;
+
+ void *private; /* owned by creator */
+
+ /*
+ * While the block is allocated by the user through gpu_buddy_alloc*,
+ * the user has ownership of the link, for example to maintain within
+ * a list, if so desired. As soon as the block is freed with
+ * gpu_buddy_free* ownership is given back to the mm.
+ */
+ union {
+ struct rb_node rb;
+ struct list_head link;
+ };
+
+ struct list_head tmp_link;
+};
+
+/* Order-zero must be at least SZ_4K */
+#define GPU_BUDDY_MAX_ORDER (63 - 12)
+
+/*
+ * Binary Buddy System.
+ *
+ * Locking should be handled by the user, a simple mutex around
+ * gpu_buddy_alloc* and gpu_buddy_free* should suffice.
+ */
+struct gpu_buddy {
+ /* Maintain a free list for each order. */
+ struct rb_root **free_trees;
+
+ /*
+ * Maintain explicit binary tree(s) to track the allocation of the
+ * address space. This gives us a simple way of finding a buddy block
+ * and performing the potentially recursive merge step when freeing a
+ * block. Nodes are either allocated or free, in which case they will
+ * also exist on the respective free list.
+ */
+ struct gpu_buddy_block **roots;
+
+ /*
+ * Anything from here is public, and remains static for the lifetime of
+ * the mm. Everything above is considered do-not-touch.
+ */
+ unsigned int n_roots;
+ unsigned int max_order;
+
+ /* Must be at least SZ_4K */
+ u64 chunk_size;
+ u64 size;
+ u64 avail;
+ u64 clear_avail;
+};
+
+static inline u64
+gpu_buddy_block_offset(const struct gpu_buddy_block *block)
+{
+ return block->header & GPU_BUDDY_HEADER_OFFSET;
+}
+
+static inline unsigned int
+gpu_buddy_block_order(struct gpu_buddy_block *block)
+{
+ return block->header & GPU_BUDDY_HEADER_ORDER;
+}
+
+static inline unsigned int
+gpu_buddy_block_state(struct gpu_buddy_block *block)
+{
+ return block->header & GPU_BUDDY_HEADER_STATE;
+}
+
+static inline bool
+gpu_buddy_block_is_allocated(struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_state(block) == GPU_BUDDY_ALLOCATED;
+}
+
+static inline bool
+gpu_buddy_block_is_clear(struct gpu_buddy_block *block)
+{
+ return block->header & GPU_BUDDY_HEADER_CLEAR;
+}
+
+static inline bool
+gpu_buddy_block_is_free(struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_state(block) == GPU_BUDDY_FREE;
+}
+
+static inline bool
+gpu_buddy_block_is_split(struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_state(block) == GPU_BUDDY_SPLIT;
+}
+
+static inline u64
+gpu_buddy_block_size(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ return mm->chunk_size << gpu_buddy_block_order(block);
+}
+
+int gpu_buddy_init(struct gpu_buddy *mm, u64 size, u64 chunk_size);
+
+void gpu_buddy_fini(struct gpu_buddy *mm);
+
+struct gpu_buddy_block *
+gpu_get_buddy(struct gpu_buddy_block *block);
+
+int gpu_buddy_alloc_blocks(struct gpu_buddy *mm,
+ u64 start, u64 end, u64 size,
+ u64 min_page_size,
+ struct list_head *blocks,
+ unsigned long flags);
+
+int gpu_buddy_block_trim(struct gpu_buddy *mm,
+ u64 *start,
+ u64 new_size,
+ struct list_head *blocks);
+
+void gpu_buddy_reset_clear(struct gpu_buddy *mm, bool is_clear);
+
+void gpu_buddy_free_block(struct gpu_buddy *mm, struct gpu_buddy_block *block);
+
+void gpu_buddy_free_list(struct gpu_buddy *mm,
+ struct list_head *objects,
+ unsigned int flags);
+
+void gpu_buddy_print(struct gpu_buddy *mm);
+void gpu_buddy_block_print(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block);
+#endif
--
2.34.1
^ permalink raw reply related
* [PATCH -next v7 0/2] rust: Add CList and GPU buddy allocator bindings
From: Joel Fernandes @ 2026-02-06 0:41 UTC (permalink / raw)
To: linux-kernel
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Balbir Singh, Philipp Stanner,
Elle Rhumsaa, Daniel Almeida, joel, nouveau, dri-devel,
rust-for-linux, linux-doc, amd-gfx, intel-gfx, intel-xe,
linux-fbdev, Joel Fernandes
This series provides Rust bindings infrastructure needed for nova-core
memory management:
1. CList module for interfacing with C circular linked lists, required
for iterating over GPU buddy allocator blocks.
2. GPU buddy allocator bindings for physical memory management in
nova-core.
These patches are based on the RFC v6 nova-core memory management series:
https://lore.kernel.org/all/20260120204303.3229303-1-joelagnelf@nvidia.com/
Main changes from v6 are based on suggestions by Gary and Dave.
This patch requires the DRM buddy code movement patch as a prerequisite:
https://lore.kernel.org/all/20260206003451.1914130-1-joelagnelf@nvidia.com/
Rebased on linux-next commit 9845cf73f7db ("Add linux-next specific files
for 20260205").
Joel Fernandes (2):
rust: clist: Add support to interface with C linked lists
rust: gpu: Add GPU buddy allocator bindings
MAINTAINERS | 7 +
drivers/gpu/Kconfig | 7 +
rust/bindings/bindings_helper.h | 11 +
rust/helpers/gpu.c | 23 ++
rust/helpers/helpers.c | 2 +
rust/helpers/list.c | 21 ++
rust/kernel/clist.rs | 315 +++++++++++++++++++
rust/kernel/gpu/buddy.rs | 530 ++++++++++++++++++++++++++++++++
rust/kernel/gpu/mod.rs | 5 +
rust/kernel/lib.rs | 4 +
10 files changed, 925 insertions(+)
create mode 100644 rust/helpers/gpu.c
create mode 100644 rust/helpers/list.c
create mode 100644 rust/kernel/clist.rs
create mode 100644 rust/kernel/gpu/buddy.rs
create mode 100644 rust/kernel/gpu/mod.rs
--
2.34.1
^ permalink raw reply
* [PATCH -next v7 1/2] rust: clist: Add support to interface with C linked lists
From: Joel Fernandes @ 2026-02-06 0:41 UTC (permalink / raw)
To: linux-kernel
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Balbir Singh, Philipp Stanner,
Elle Rhumsaa, Daniel Almeida, joel, nouveau, dri-devel,
rust-for-linux, linux-doc, amd-gfx, intel-gfx, intel-xe,
linux-fbdev, Joel Fernandes
In-Reply-To: <20260206004110.1914814-1-joelagnelf@nvidia.com>
Add a new module `clist` for working with C's doubly circular linked
lists. Provide low-level iteration over list nodes.
Typed iteration over actual items is provided with a `clist_create`
macro to assist in creation of the `CList` type.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
MAINTAINERS | 7 +
drivers/gpu/Kconfig | 7 +
rust/helpers/helpers.c | 1 +
rust/helpers/list.c | 21 +++
rust/kernel/clist.rs | 315 +++++++++++++++++++++++++++++++++++++++++
rust/kernel/lib.rs | 2 +
6 files changed, 353 insertions(+)
create mode 100644 rust/helpers/list.c
create mode 100644 rust/kernel/clist.rs
diff --git a/MAINTAINERS b/MAINTAINERS
index 900fc00b73e6..310bb479260c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -23204,6 +23204,13 @@ S: Maintained
T: git https://github.com/Rust-for-Linux/linux.git rust-analyzer-next
F: scripts/generate_rust_analyzer.py
+RUST TO C LIST INTERFACES
+M: Joel Fernandes <joelagnelf@nvidia.com>
+M: Alexandre Courbot <acourbot@nvidia.com>
+L: rust-for-linux@vger.kernel.org
+S: Maintained
+F: rust/kernel/clist.rs
+
RXRPC SOCKETS (AF_RXRPC)
M: David Howells <dhowells@redhat.com>
M: Marc Dionne <marc.dionne@auristor.com>
diff --git a/drivers/gpu/Kconfig b/drivers/gpu/Kconfig
index 22dd29cd50b5..2c3dec070645 100644
--- a/drivers/gpu/Kconfig
+++ b/drivers/gpu/Kconfig
@@ -1,7 +1,14 @@
# SPDX-License-Identifier: GPL-2.0
+config RUST_CLIST
+ bool
+ depends on RUST
+ help
+ Rust abstraction for interfacing with C linked lists.
+
config GPU_BUDDY
bool
+ select RUST_CLIST if RUST
help
A page based buddy allocator for GPU memory.
diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c
index a3c42e51f00a..724fcb8240ac 100644
--- a/rust/helpers/helpers.c
+++ b/rust/helpers/helpers.c
@@ -35,6 +35,7 @@
#include "io.c"
#include "jump_label.c"
#include "kunit.c"
+#include "list.c"
#include "maple_tree.c"
#include "mm.c"
#include "mutex.c"
diff --git a/rust/helpers/list.c b/rust/helpers/list.c
new file mode 100644
index 000000000000..3390b154fa36
--- /dev/null
+++ b/rust/helpers/list.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Helpers for C Circular doubly linked list implementation.
+ */
+
+#include <linux/list.h>
+
+#ifndef __rust_helper
+#define __rust_helper
+#endif
+
+__rust_helper void rust_helper_INIT_LIST_HEAD(struct list_head *list)
+{
+ INIT_LIST_HEAD(list);
+}
+
+__rust_helper void rust_helper_list_add_tail(struct list_head *new, struct list_head *head)
+{
+ list_add_tail(new, head);
+}
diff --git a/rust/kernel/clist.rs b/rust/kernel/clist.rs
new file mode 100644
index 000000000000..1f6d4db13c1d
--- /dev/null
+++ b/rust/kernel/clist.rs
@@ -0,0 +1,315 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! A C doubly circular intrusive linked list interface for rust code.
+//!
+//! # Examples
+//!
+//! ```
+//! use kernel::{
+//! bindings,
+//! clist_create,
+//! types::Opaque, //
+//! };
+//! # // Create test list with values (0, 10, 20) - normally done by C code but it is
+//! # // emulated here for doctests using the C bindings.
+//! # use core::mem::MaybeUninit;
+//! #
+//! # /// C struct with embedded `list_head` (typically will be allocated by C code).
+//! # #[repr(C)]
+//! # pub(crate) struct SampleItemC {
+//! # pub value: i32,
+//! # pub link: bindings::list_head,
+//! # }
+//! #
+//! # let mut head = MaybeUninit::<bindings::list_head>::uninit();
+//! #
+//! # let head = head.as_mut_ptr();
+//! # // SAFETY: head and all the items are test objects allocated in this scope.
+//! # unsafe { bindings::INIT_LIST_HEAD(head) };
+//! #
+//! # let mut items = [
+//! # MaybeUninit::<SampleItemC>::uninit(),
+//! # MaybeUninit::<SampleItemC>::uninit(),
+//! # MaybeUninit::<SampleItemC>::uninit(),
+//! # ];
+//! #
+//! # for (i, item) in items.iter_mut().enumerate() {
+//! # let ptr = item.as_mut_ptr();
+//! # // SAFETY: pointers are to allocated test objects with a list_head field.
+//! # unsafe {
+//! # (*ptr).value = i as i32 * 10;
+//! # // addr_of_mut!() computes address of link directly as link is uninitialized.
+//! # bindings::INIT_LIST_HEAD(core::ptr::addr_of_mut!((*ptr).link));
+//! # bindings::list_add_tail(&mut (*ptr).link, head);
+//! # }
+//! # }
+//!
+//! // Rust wrapper for the C struct.
+//! // The list item struct in this example is defined in C code as:
+//! // struct SampleItemC {
+//! // int value;
+//! // struct list_head link;
+//! // };
+//! //
+//! #[repr(transparent)]
+//! pub(crate) struct Item(Opaque<SampleItemC>);
+//!
+//! impl Item {
+//! pub(crate) fn value(&self) -> i32 {
+//! // SAFETY: [`Item`] has same layout as [`SampleItemC`].
+//! unsafe { (*self.0.get()).value }
+//! }
+//! }
+//!
+//! // Create typed [`CList`] from sentinel head.
+//! // SAFETY: head is valid, items are [`SampleItemC`] with embedded `link` field.
+//! let list = unsafe { clist_create!(head, Item, SampleItemC, link) };
+//!
+//! // Iterate directly over typed items.
+//! let mut found_0 = false;
+//! let mut found_10 = false;
+//! let mut found_20 = false;
+//!
+//! for item in list.iter() {
+//! let val = item.value();
+//! if val == 0 { found_0 = true; }
+//! if val == 10 { found_10 = true; }
+//! if val == 20 { found_20 = true; }
+//! }
+//!
+//! assert!(found_0 && found_10 && found_20);
+//! ```
+
+use core::{
+ iter::FusedIterator,
+ marker::PhantomData, //
+};
+
+use crate::{
+ bindings,
+ types::Opaque, //
+};
+
+use pin_init::PinInit;
+
+/// Wraps a `list_head` object for use in intrusive linked lists.
+///
+/// # Invariants
+///
+/// - [`CListHead`] represents an allocated and valid `list_head` structure.
+/// - Once a [`CListHead`] is created in Rust, it will not be modified by non-Rust code.
+/// - All `list_head` for individual items are not modified for the lifetime of [`CListHead`].
+#[repr(transparent)]
+pub(crate) struct CListHead(Opaque<bindings::list_head>);
+
+impl CListHead {
+ /// Create a `&CListHead` reference from a raw `list_head` pointer.
+ ///
+ /// # Safety
+ ///
+ /// - `ptr` must be a valid pointer to an allocated and initialized `list_head` structure.
+ /// - `ptr` must remain valid and unmodified for the lifetime `'a`.
+ #[inline]
+ pub(crate) unsafe fn from_raw<'a>(ptr: *mut bindings::list_head) -> &'a Self {
+ // SAFETY:
+ // - [`CListHead`] has same layout as `list_head`.
+ // - `ptr` is valid and unmodified for 'a.
+ unsafe { &*ptr.cast() }
+ }
+
+ /// Get the raw `list_head` pointer.
+ #[inline]
+ pub(crate) fn as_raw(&self) -> *mut bindings::list_head {
+ self.0.get()
+ }
+
+ /// Get the next [`CListHead`] in the list.
+ #[inline]
+ pub(crate) fn next(&self) -> &Self {
+ let raw = self.as_raw();
+ // SAFETY:
+ // - `self.as_raw()` is valid per type invariants.
+ // - The `next` pointer is guaranteed to be non-NULL.
+ unsafe { Self::from_raw((*raw).next) }
+ }
+
+ /// Check if this node is linked in a list (not isolated).
+ #[inline]
+ pub(crate) fn is_linked(&self) -> bool {
+ let raw = self.as_raw();
+ // SAFETY: self.as_raw() is valid per type invariants.
+ unsafe { (*raw).next != raw && (*raw).prev != raw }
+ }
+
+ /// Pin-initializer that initializes the list head.
+ pub(crate) fn new() -> impl PinInit<Self> {
+ // SAFETY: `INIT_LIST_HEAD` initializes `slot` to a valid empty list.
+ unsafe {
+ pin_init::pin_init_from_closure(move |slot: *mut Self| {
+ bindings::INIT_LIST_HEAD(slot.cast());
+ Ok(())
+ })
+ }
+ }
+}
+
+// SAFETY: [`CListHead`] can be sent to any thread.
+unsafe impl Send for CListHead {}
+
+// SAFETY: [`CListHead`] can be shared among threads as it is not modified
+// by non-Rust code per type invariants.
+unsafe impl Sync for CListHead {}
+
+impl PartialEq for CListHead {
+ fn eq(&self, other: &Self) -> bool {
+ core::ptr::eq(self, other)
+ }
+}
+
+impl Eq for CListHead {}
+
+/// Low-level iterator over `list_head` nodes.
+///
+/// An iterator used to iterate over a C intrusive linked list (`list_head`). Caller has to
+/// perform conversion of returned [`CListHead`] to an item (using `container_of` macro or similar).
+///
+/// # Invariants
+///
+/// [`CListHeadIter`] is iterating over an allocated, initialized and valid list.
+struct CListHeadIter<'a> {
+ /// Current position in the list.
+ current: &'a CListHead,
+ /// The sentinel head (used to detect end of iteration).
+ sentinel: &'a CListHead,
+}
+
+impl<'a> Iterator for CListHeadIter<'a> {
+ type Item = &'a CListHead;
+
+ #[inline]
+ fn next(&mut self) -> Option<Self::Item> {
+ // Check if we've reached the sentinel (end of list).
+ if core::ptr::eq(self.current, self.sentinel) {
+ return None;
+ }
+
+ let item = self.current;
+ self.current = item.next();
+ Some(item)
+ }
+}
+
+impl<'a> FusedIterator for CListHeadIter<'a> {}
+
+/// A typed C linked list with a sentinel head.
+///
+/// A sentinel head represents the entire linked list and can be used for
+/// iteration over items of type `T`, it is not associated with a specific item.
+///
+/// The const generic `OFFSET` specifies the byte offset of the `list_head` field within
+/// the struct that `T` wraps.
+///
+/// # Invariants
+///
+/// - The [`CListHead`] is an allocated and valid sentinel C `list_head` structure.
+/// - `OFFSET` is the byte offset of the `list_head` field within the struct that `T` wraps.
+/// - All the list's `list_head` nodes are allocated and have valid next/prev pointers.
+#[repr(transparent)]
+pub(crate) struct CList<T, const OFFSET: usize>(CListHead, PhantomData<T>);
+
+impl<T, const OFFSET: usize> CList<T, OFFSET> {
+ /// Create a typed [`CList`] reference from a raw sentinel `list_head` pointer.
+ ///
+ /// # Safety
+ ///
+ /// - `ptr` must be a valid pointer to an allocated and initialized `list_head` structure
+ /// representing a list sentinel.
+ /// - `ptr` must remain valid and unmodified for the lifetime `'a`.
+ /// - The list must contain items where the `list_head` field is at byte offset `OFFSET`.
+ /// - `T` must be `#[repr(transparent)]` over the C struct.
+ #[inline]
+ pub(crate) unsafe fn from_raw<'a>(ptr: *mut bindings::list_head) -> &'a Self {
+ // SAFETY:
+ // - [`CList`] has same layout as [`CListHead`] due to repr(transparent).
+ // - Caller guarantees `ptr` is a valid, sentinel `list_head` object.
+ unsafe { &*ptr.cast() }
+ }
+
+ /// Check if the list is empty.
+ #[inline]
+ #[expect(dead_code)]
+ pub(crate) fn is_empty(&self) -> bool {
+ !self.0.is_linked()
+ }
+
+ /// Create an iterator over typed items.
+ #[inline]
+ pub(crate) fn iter(&self) -> CListIter<'_, T, OFFSET> {
+ let head = &self.0;
+ CListIter {
+ head_iter: CListHeadIter {
+ current: head.next(),
+ sentinel: head,
+ },
+ _phantom: PhantomData,
+ }
+ }
+}
+
+/// High-level iterator over typed list items.
+pub(crate) struct CListIter<'a, T, const OFFSET: usize> {
+ head_iter: CListHeadIter<'a>,
+ _phantom: PhantomData<&'a T>,
+}
+
+impl<'a, T, const OFFSET: usize> Iterator for CListIter<'a, T, OFFSET> {
+ type Item = &'a T;
+
+ fn next(&mut self) -> Option<Self::Item> {
+ let head = self.head_iter.next()?;
+
+ // Convert to item using OFFSET.
+ // SAFETY: `item_ptr` calculation from `OFFSET` (calculated using offset_of!)
+ // is valid per invariants.
+ Some(unsafe { &*head.as_raw().byte_sub(OFFSET).cast::<T>() })
+ }
+}
+
+impl<'a, T, const OFFSET: usize> FusedIterator for CListIter<'a, T, OFFSET> {}
+
+/// Create a C doubly-circular linked list interface `CList` from a raw `list_head` pointer.
+///
+/// This macro creates a `CList<T, OFFSET>` that can iterate over items of type `$rust_type`
+/// linked via the `$field` field in the underlying C struct `$c_type`.
+///
+/// # Arguments
+///
+/// - `$head`: Raw pointer to the sentinel `list_head` object (`*mut bindings::list_head`).
+/// - `$rust_type`: Each item's rust wrapper type.
+/// - `$c_type`: Each item's C struct type that contains the embedded `list_head`.
+/// - `$field`: The name of the `list_head` field within the C struct.
+///
+/// # Safety
+///
+/// This is an unsafe macro. The caller must ensure:
+///
+/// - `$head` is a valid, initialized sentinel `list_head` pointing to a list that remains
+/// unmodified for the lifetime of the rust `CList`.
+/// - The list contains items of type `$c_type` linked via an embedded `$field`.
+/// - `$rust_type` is `#[repr(transparent)]` over `$c_type` or has compatible layout.
+///
+/// # Examples
+///
+/// Refer to the examples in this module's documentation.
+#[macro_export]
+macro_rules! clist_create {
+ ($head:expr, $rust_type:ty, $c_type:ty, $($field:tt).+) => {{
+ // Compile-time check that field path is a list_head.
+ let _: fn(*const $c_type) -> *const $crate::bindings::list_head =
+ |p| &raw const (*p).$($field).+;
+
+ // Calculate offset and create `CList`.
+ const OFFSET: usize = ::core::mem::offset_of!($c_type, $($field).+);
+ $crate::clist::CList::<$rust_type, OFFSET>::from_raw($head)
+ }};
+}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 3da92f18f4ee..8439c30f40b5 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -75,6 +75,8 @@
pub mod bug;
#[doc(hidden)]
pub mod build_assert;
+#[cfg(CONFIG_RUST_CLIST)]
+pub(crate) mod clist;
pub mod clk;
#[cfg(CONFIG_CONFIGFS_FS)]
pub mod configfs;
--
2.34.1
^ permalink raw reply related
* [PATCH -next v7 2/2] rust: gpu: Add GPU buddy allocator bindings
From: Joel Fernandes @ 2026-02-06 0:41 UTC (permalink / raw)
To: linux-kernel
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Balbir Singh, Philipp Stanner,
Elle Rhumsaa, Daniel Almeida, joel, nouveau, dri-devel,
rust-for-linux, linux-doc, amd-gfx, intel-gfx, intel-xe,
linux-fbdev, Joel Fernandes
In-Reply-To: <20260206004110.1914814-1-joelagnelf@nvidia.com>
Add safe Rust abstractions over the Linux kernel's GPU buddy
allocator for physical memory management. The GPU buddy allocator
implements a binary buddy system useful for GPU physical memory
allocation. nova-core will use it for physical memory allocation.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
rust/bindings/bindings_helper.h | 11 +
rust/helpers/gpu.c | 23 ++
rust/helpers/helpers.c | 1 +
rust/kernel/gpu/buddy.rs | 530 ++++++++++++++++++++++++++++++++
rust/kernel/gpu/mod.rs | 5 +
rust/kernel/lib.rs | 2 +
6 files changed, 572 insertions(+)
create mode 100644 rust/helpers/gpu.c
create mode 100644 rust/kernel/gpu/buddy.rs
create mode 100644 rust/kernel/gpu/mod.rs
diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h
index 083cc44aa952..dbb765a9fdbd 100644
--- a/rust/bindings/bindings_helper.h
+++ b/rust/bindings/bindings_helper.h
@@ -29,6 +29,7 @@
#include <linux/hrtimer_types.h>
#include <linux/acpi.h>
+#include <linux/gpu_buddy.h>
#include <drm/drm_device.h>
#include <drm/drm_drv.h>
#include <drm/drm_file.h>
@@ -146,6 +147,16 @@ const vm_flags_t RUST_CONST_HELPER_VM_MIXEDMAP = VM_MIXEDMAP;
const vm_flags_t RUST_CONST_HELPER_VM_HUGEPAGE = VM_HUGEPAGE;
const vm_flags_t RUST_CONST_HELPER_VM_NOHUGEPAGE = VM_NOHUGEPAGE;
+#if IS_ENABLED(CONFIG_GPU_BUDDY)
+const unsigned long RUST_CONST_HELPER_GPU_BUDDY_RANGE_ALLOCATION = GPU_BUDDY_RANGE_ALLOCATION;
+const unsigned long RUST_CONST_HELPER_GPU_BUDDY_TOPDOWN_ALLOCATION = GPU_BUDDY_TOPDOWN_ALLOCATION;
+const unsigned long RUST_CONST_HELPER_GPU_BUDDY_CONTIGUOUS_ALLOCATION =
+ GPU_BUDDY_CONTIGUOUS_ALLOCATION;
+const unsigned long RUST_CONST_HELPER_GPU_BUDDY_CLEAR_ALLOCATION = GPU_BUDDY_CLEAR_ALLOCATION;
+const unsigned long RUST_CONST_HELPER_GPU_BUDDY_CLEARED = GPU_BUDDY_CLEARED;
+const unsigned long RUST_CONST_HELPER_GPU_BUDDY_TRIM_DISABLE = GPU_BUDDY_TRIM_DISABLE;
+#endif
+
#if IS_ENABLED(CONFIG_ANDROID_BINDER_IPC_RUST)
#include "../../drivers/android/binder/rust_binder.h"
#include "../../drivers/android/binder/rust_binder_events.h"
diff --git a/rust/helpers/gpu.c b/rust/helpers/gpu.c
new file mode 100644
index 000000000000..38b1a4e6bef8
--- /dev/null
+++ b/rust/helpers/gpu.c
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/gpu_buddy.h>
+
+#ifdef CONFIG_GPU_BUDDY
+
+__rust_helper u64 rust_helper_gpu_buddy_block_offset(const struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_offset(block);
+}
+
+__rust_helper unsigned int rust_helper_gpu_buddy_block_order(struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_order(block);
+}
+
+__rust_helper u64 rust_helper_gpu_buddy_block_size(struct gpu_buddy *mm,
+ struct gpu_buddy_block *block)
+{
+ return gpu_buddy_block_size(mm, block);
+}
+
+#endif /* CONFIG_GPU_BUDDY */
diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c
index 724fcb8240ac..a53929ce52a3 100644
--- a/rust/helpers/helpers.c
+++ b/rust/helpers/helpers.c
@@ -32,6 +32,7 @@
#include "err.c"
#include "irq.c"
#include "fs.c"
+#include "gpu.c"
#include "io.c"
#include "jump_label.c"
#include "kunit.c"
diff --git a/rust/kernel/gpu/buddy.rs b/rust/kernel/gpu/buddy.rs
new file mode 100644
index 000000000000..00290ce53aeb
--- /dev/null
+++ b/rust/kernel/gpu/buddy.rs
@@ -0,0 +1,530 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! GPU buddy allocator bindings.
+//!
+//! C header: [`include/linux/gpu_buddy.h`](srctree/include/linux/gpu_buddy.h)
+//!
+//! This module provides Rust abstractions over the Linux kernel's GPU buddy
+//! allocator, which implements a binary buddy memory allocator.
+//!
+//! The buddy allocator manages a contiguous address space and allocates blocks
+//! in power-of-two sizes, useful for GPU physical memory management.
+//!
+//! # Examples
+//!
+//! ```
+//! use kernel::{
+//! gpu::buddy::{BuddyFlags, GpuBuddy, GpuBuddyAllocParams, GpuBuddyParams},
+//! prelude::*,
+//! sizes::*, //
+//! };
+//!
+//! // Create a 1GB buddy allocator with 4KB minimum chunk size.
+//! let buddy = GpuBuddy::new(&GpuBuddyParams {
+//! base_offset_bytes: 0,
+//! physical_memory_size_bytes: SZ_1G as u64,
+//! chunk_size_bytes: SZ_4K as u64,
+//! })?;
+//!
+//! // Verify initial state.
+//! assert_eq!(buddy.size(), SZ_1G as u64);
+//! assert_eq!(buddy.chunk_size(), SZ_4K as u64);
+//! let initial_free = buddy.free_memory_bytes();
+//!
+//! // Base allocation params - mutated between calls for field overrides.
+//! let mut params = GpuBuddyAllocParams {
+//! start_range_address: 0,
+//! end_range_address: 0, // Entire range.
+//! size_bytes: SZ_16M as u64,
+//! min_block_size_bytes: SZ_16M as u64,
+//! buddy_flags: BuddyFlags::try_new(BuddyFlags::RANGE_ALLOCATION)?,
+//! };
+//!
+//! // Test top-down allocation (allocates from highest addresses).
+//! params.buddy_flags = BuddyFlags::try_new(BuddyFlags::TOPDOWN_ALLOCATION)?;
+//! let topdown = buddy.alloc_blocks(¶ms)?;
+//! assert_eq!(buddy.free_memory_bytes(), initial_free - SZ_16M as u64);
+//!
+//! for block in topdown.iter() {
+//! assert_eq!(block.offset(), (SZ_1G - SZ_16M) as u64);
+//! assert_eq!(block.order(), 12); // 2^12 pages
+//! assert_eq!(block.size(), SZ_16M as u64);
+//! }
+//! drop(topdown);
+//! assert_eq!(buddy.free_memory_bytes(), initial_free);
+//!
+//! // Allocate 16MB - should result in a single 16MB block at offset 0.
+//! params.buddy_flags = BuddyFlags::try_new(BuddyFlags::RANGE_ALLOCATION)?;
+//! let allocated = buddy.alloc_blocks(¶ms)?;
+//! assert_eq!(buddy.free_memory_bytes(), initial_free - SZ_16M as u64);
+//!
+//! for block in allocated.iter() {
+//! assert_eq!(block.offset(), 0);
+//! assert_eq!(block.order(), 12); // 2^12 pages
+//! assert_eq!(block.size(), SZ_16M as u64);
+//! }
+//! drop(allocated);
+//! assert_eq!(buddy.free_memory_bytes(), initial_free);
+//!
+//! // Test non-contiguous allocation with fragmented memory.
+//! // Create fragmentation by allocating 4MB blocks at [0,4M) and [8M,12M).
+//! params.end_range_address = SZ_4M as u64;
+//! params.size_bytes = SZ_4M as u64;
+//! params.min_block_size_bytes = SZ_4M as u64;
+//! let frag1 = buddy.alloc_blocks(¶ms)?;
+//! assert_eq!(buddy.free_memory_bytes(), initial_free - SZ_4M as u64);
+//!
+//! params.start_range_address = SZ_8M as u64;
+//! params.end_range_address = (SZ_8M + SZ_4M) as u64;
+//! let frag2 = buddy.alloc_blocks(¶ms)?;
+//! assert_eq!(buddy.free_memory_bytes(), initial_free - SZ_8M as u64);
+//!
+//! // Allocate 8MB without CONTIGUOUS - should return 2 blocks from the holes.
+//! params.start_range_address = 0;
+//! params.end_range_address = SZ_16M as u64;
+//! params.size_bytes = SZ_8M as u64;
+//! let fragmented = buddy.alloc_blocks(¶ms)?;
+//! assert_eq!(buddy.free_memory_bytes(), initial_free - (SZ_16M) as u64);
+//!
+//! let (mut count, mut total) = (0u32, 0u64);
+//! for block in fragmented.iter() {
+//! // The 8MB allocation should return 2 blocks, each 4MB.
+//! assert_eq!(block.size(), SZ_4M as u64);
+//! total += block.size();
+//! count += 1;
+//! }
+//! assert_eq!(total, SZ_8M as u64);
+//! assert_eq!(count, 2);
+//! drop(fragmented);
+//! drop(frag2);
+//! drop(frag1);
+//! assert_eq!(buddy.free_memory_bytes(), initial_free);
+//!
+//! // Test CONTIGUOUS failure when only fragmented space available.
+//! // Create a small buddy allocator with only 16MB of memory.
+//! let small = GpuBuddy::new(&GpuBuddyParams {
+//! base_offset_bytes: 0,
+//! physical_memory_size_bytes: SZ_16M as u64,
+//! chunk_size_bytes: SZ_4K as u64,
+//! })?;
+//!
+//! // Allocate 4MB blocks at [0,4M) and [8M,12M) to create fragmented memory.
+//! params.start_range_address = 0;
+//! params.end_range_address = SZ_4M as u64;
+//! params.size_bytes = SZ_4M as u64;
+//! let hole1 = small.alloc_blocks(¶ms)?;
+//!
+//! params.start_range_address = SZ_8M as u64;
+//! params.end_range_address = (SZ_8M + SZ_4M) as u64;
+//! let hole2 = small.alloc_blocks(¶ms)?;
+//!
+//! // 8MB contiguous should fail - only two non-contiguous 4MB holes exist.
+//! params.start_range_address = 0;
+//! params.end_range_address = 0;
+//! params.size_bytes = SZ_8M as u64;
+//! params.buddy_flags = BuddyFlags::try_new(BuddyFlags::CONTIGUOUS_ALLOCATION)?;
+//! let result = small.alloc_blocks(¶ms);
+//! assert!(result.is_err());
+//! drop(hole2);
+//! drop(hole1);
+//!
+//! # Ok::<(), Error>(())
+//! ```
+
+use crate::{
+ bindings,
+ clist::CListHead,
+ clist_create,
+ error::to_result,
+ new_mutex,
+ prelude::*,
+ sync::{
+ lock::mutex::MutexGuard,
+ Arc,
+ Mutex, //
+ },
+ types::Opaque,
+};
+
+/// Flags for GPU buddy allocator operations.
+///
+/// These flags control the allocation behavior of the buddy allocator.
+#[derive(Clone, Copy, Default, PartialEq, Eq)]
+pub struct BuddyFlags(usize);
+
+impl BuddyFlags {
+ /// Range-based allocation from start to end addresses.
+ pub const RANGE_ALLOCATION: usize = bindings::GPU_BUDDY_RANGE_ALLOCATION;
+
+ /// Allocate from top of address space downward.
+ pub const TOPDOWN_ALLOCATION: usize = bindings::GPU_BUDDY_TOPDOWN_ALLOCATION;
+
+ /// Allocate physically contiguous blocks.
+ pub const CONTIGUOUS_ALLOCATION: usize = bindings::GPU_BUDDY_CONTIGUOUS_ALLOCATION;
+
+ /// Request allocation from the cleared (zeroed) memory. The zero'ing is not
+ /// done by the allocator, but by the caller before freeing old blocks.
+ pub const CLEAR_ALLOCATION: usize = bindings::GPU_BUDDY_CLEAR_ALLOCATION;
+
+ /// Disable trimming of partially used blocks.
+ pub const TRIM_DISABLE: usize = bindings::GPU_BUDDY_TRIM_DISABLE;
+
+ /// Mark blocks as cleared (zeroed) when freeing. When set during free,
+ /// indicates that the caller has already zeroed the memory.
+ pub const CLEARED: usize = bindings::GPU_BUDDY_CLEARED;
+
+ /// Create [`BuddyFlags`] from a raw value with validation.
+ ///
+ /// Use `|` operator to combine flags if needed, before calling this method.
+ pub fn try_new(flags: usize) -> Result<Self> {
+ // Flags must not exceed u32::MAX to satisfy the GPU buddy allocator C API.
+ if flags > u32::MAX as usize {
+ return Err(EINVAL);
+ }
+
+ // `TOPDOWN_ALLOCATION` only works without `RANGE_ALLOCATION`. When both are
+ // set, `TOPDOWN_ALLOCATION` is silently ignored by the allocator. Reject this.
+ if (flags & Self::RANGE_ALLOCATION) != 0 && (flags & Self::TOPDOWN_ALLOCATION) != 0 {
+ return Err(EINVAL);
+ }
+
+ Ok(Self(flags))
+ }
+
+ /// Get raw value of the flags.
+ pub(crate) fn as_raw(self) -> usize {
+ self.0
+ }
+}
+
+/// Parameters for creating a GPU buddy allocator.
+pub struct GpuBuddyParams {
+ /// Base offset in bytes where the managed memory region starts.
+ /// Allocations will be offset by this value.
+ pub base_offset_bytes: u64,
+ /// Total physical memory size managed by the allocator in bytes.
+ pub physical_memory_size_bytes: u64,
+ /// Minimum allocation unit / chunk size in bytes, must be >= 4KB.
+ pub chunk_size_bytes: u64,
+}
+
+/// Parameters for allocating blocks from a GPU buddy allocator.
+pub struct GpuBuddyAllocParams {
+ /// Start of allocation range in bytes. Use 0 for beginning.
+ pub start_range_address: u64,
+ /// End of allocation range in bytes. Use 0 for entire range.
+ pub end_range_address: u64,
+ /// Total size to allocate in bytes.
+ pub size_bytes: u64,
+ /// Minimum block size for fragmented allocations in bytes.
+ pub min_block_size_bytes: u64,
+ /// Buddy allocator behavior flags.
+ pub buddy_flags: BuddyFlags,
+}
+
+/// Inner structure holding the actual buddy allocator.
+///
+/// # Synchronization
+///
+/// The C `gpu_buddy` API requires synchronization (see `include/linux/gpu_buddy.h`).
+/// The internal [`GpuBuddyGuard`] ensures that the lock is held for all
+/// allocator and free operations, preventing races between concurrent allocations
+/// and the freeing that occurs when [`AllocatedBlocks`] is dropped.
+///
+/// # Invariants
+///
+/// The inner [`Opaque`] contains a valid, initialized buddy allocator.
+#[pin_data(PinnedDrop)]
+struct GpuBuddyInner {
+ #[pin]
+ inner: Opaque<bindings::gpu_buddy>,
+ #[pin]
+ lock: Mutex<()>,
+ /// Base offset for all allocations (does not change after init).
+ base_offset: u64,
+ /// Cached chunk size (does not change after init).
+ chunk_size: u64,
+ /// Cached total size (does not change after init).
+ size: u64,
+}
+
+impl GpuBuddyInner {
+ /// Create a pin-initializer for the buddy allocator.
+ fn new(params: &GpuBuddyParams) -> impl PinInit<Self, Error> {
+ let base_offset = params.base_offset_bytes;
+ let size = params.physical_memory_size_bytes;
+ let chunk_size = params.chunk_size_bytes;
+
+ try_pin_init!(Self {
+ inner <- Opaque::try_ffi_init(|ptr| {
+ // SAFETY: ptr points to valid uninitialized memory from the pin-init
+ // infrastructure. gpu_buddy_init will initialize the structure.
+ to_result(unsafe { bindings::gpu_buddy_init(ptr, size, chunk_size) })
+ }),
+ lock <- new_mutex!(()),
+ base_offset: base_offset,
+ chunk_size: chunk_size,
+ size: size,
+ })
+ }
+
+ /// Lock the mutex and return a guard for accessing the allocator.
+ fn lock(&self) -> GpuBuddyGuard<'_> {
+ GpuBuddyGuard {
+ inner: self,
+ _guard: self.lock.lock(),
+ }
+ }
+}
+
+#[pinned_drop]
+impl PinnedDrop for GpuBuddyInner {
+ fn drop(self: Pin<&mut Self>) {
+ let guard = self.lock();
+
+ // SAFETY: guard provides exclusive access to the allocator.
+ unsafe {
+ bindings::gpu_buddy_fini(guard.as_raw());
+ }
+ }
+}
+
+// SAFETY: [`GpuBuddyInner`] can be sent between threads.
+unsafe impl Send for GpuBuddyInner {}
+
+// SAFETY: [`GpuBuddyInner`] is `Sync` because the internal [`GpuBuddyGuard`]
+// serializes all access to the C allocator, preventing data races.
+unsafe impl Sync for GpuBuddyInner {}
+
+/// Guard that proves the lock is held, enabling access to the allocator.
+///
+/// # Invariants
+///
+/// The inner `_guard` holds the lock for the duration of this guard's lifetime.
+pub(crate) struct GpuBuddyGuard<'a> {
+ inner: &'a GpuBuddyInner,
+ _guard: MutexGuard<'a, ()>,
+}
+
+impl GpuBuddyGuard<'_> {
+ /// Get a raw pointer to the underlying C `gpu_buddy` structure.
+ fn as_raw(&self) -> *mut bindings::gpu_buddy {
+ self.inner.inner.get()
+ }
+}
+
+/// GPU buddy allocator instance.
+///
+/// This structure wraps the C `gpu_buddy` allocator using reference counting.
+/// The allocator is automatically cleaned up when all references are dropped.
+///
+/// # Invariants
+///
+/// The inner [`Arc`] points to a valid, initialized GPU buddy allocator.
+pub struct GpuBuddy(Arc<GpuBuddyInner>);
+
+impl GpuBuddy {
+ /// Create a new buddy allocator.
+ ///
+ /// Creates a buddy allocator that manages a contiguous address space of the given
+ /// size, with the specified minimum allocation unit (chunk_size must be at least 4KB).
+ pub fn new(params: &GpuBuddyParams) -> Result<Self> {
+ Ok(Self(Arc::pin_init(
+ GpuBuddyInner::new(params),
+ GFP_KERNEL,
+ )?))
+ }
+
+ /// Get the base offset for allocations.
+ pub fn base_offset(&self) -> u64 {
+ self.0.base_offset
+ }
+
+ /// Get the chunk size (minimum allocation unit).
+ pub fn chunk_size(&self) -> u64 {
+ self.0.chunk_size
+ }
+
+ /// Get the total managed size.
+ pub fn size(&self) -> u64 {
+ self.0.size
+ }
+
+ /// Get the available (free) memory in bytes.
+ pub fn free_memory_bytes(&self) -> u64 {
+ let guard = self.0.lock();
+ // SAFETY: guard provides exclusive access to the allocator.
+ unsafe { (*guard.as_raw()).avail }
+ }
+
+ /// Allocate blocks from the buddy allocator.
+ ///
+ /// Returns an [`Arc<AllocatedBlocks>`] structure that owns the allocated blocks
+ /// and automatically frees them when all references are dropped.
+ ///
+ /// Takes `&self` instead of `&mut self` because the internal [`Mutex`] provides
+ /// synchronization - no external `&mut` exclusivity needed.
+ pub fn alloc_blocks(&self, params: &GpuBuddyAllocParams) -> Result<Arc<AllocatedBlocks>> {
+ let buddy_arc = Arc::clone(&self.0);
+
+ // Create pin-initializer that initializes list and allocates blocks.
+ let init = try_pin_init!(AllocatedBlocks {
+ buddy: Arc::clone(&buddy_arc),
+ list <- CListHead::new(),
+ flags: params.buddy_flags,
+ _: {
+ // Lock while allocating to serialize with concurrent frees.
+ let guard = buddy.lock();
+
+ // SAFETY: `guard` provides exclusive access to the buddy allocator.
+ to_result(unsafe {
+ bindings::gpu_buddy_alloc_blocks(
+ guard.as_raw(),
+ params.start_range_address,
+ params.end_range_address,
+ params.size_bytes,
+ params.min_block_size_bytes,
+ list.as_raw(),
+ params.buddy_flags.as_raw(),
+ )
+ })?
+ }
+ });
+
+ Arc::pin_init(init, GFP_KERNEL)
+ }
+}
+
+/// Allocated blocks from the buddy allocator with automatic cleanup.
+///
+/// This structure owns a list of allocated blocks and ensures they are
+/// automatically freed when dropped. Use `iter()` to iterate over all
+/// allocated [`Block`] structures.
+///
+/// # Invariants
+///
+/// - `list` is an initialized, valid list head containing allocated blocks.
+/// - `buddy` references a valid [`GpuBuddyInner`].
+#[pin_data(PinnedDrop)]
+pub struct AllocatedBlocks {
+ #[pin]
+ list: CListHead,
+ buddy: Arc<GpuBuddyInner>,
+ flags: BuddyFlags,
+}
+
+impl AllocatedBlocks {
+ /// Check if the block list is empty.
+ pub fn is_empty(&self) -> bool {
+ // An empty list head points to itself.
+ !self.list.is_linked()
+ }
+
+ /// Iterate over allocated blocks.
+ ///
+ /// Returns an iterator yielding [`AllocatedBlock`] references. The blocks
+ /// are only valid for the duration of the borrow of `self`.
+ pub fn iter(&self) -> impl Iterator<Item = AllocatedBlock<'_>> + '_ {
+ // SAFETY: list contains gpu_buddy_block items linked via __bindgen_anon_1.link.
+ let clist = unsafe {
+ clist_create!(
+ self.list.as_raw(),
+ Block,
+ bindings::gpu_buddy_block,
+ __bindgen_anon_1.link
+ )
+ };
+
+ clist
+ .iter()
+ .map(|block| AllocatedBlock { block, alloc: self })
+ }
+}
+
+#[pinned_drop]
+impl PinnedDrop for AllocatedBlocks {
+ fn drop(self: Pin<&mut Self>) {
+ let guard = self.buddy.lock();
+
+ // SAFETY:
+ // - list is valid per the type's invariants.
+ // - guard provides exclusive access to the allocator.
+ // CAST: BuddyFlags were validated to fit in u32 at construction.
+ unsafe {
+ bindings::gpu_buddy_free_list(
+ guard.as_raw(),
+ self.list.as_raw(),
+ self.flags.as_raw() as u32,
+ );
+ }
+ }
+}
+
+/// A GPU buddy block.
+///
+/// Transparent wrapper over C `gpu_buddy_block` structure. This type is returned
+/// as references from [`CListIter`] during iteration over [`AllocatedBlocks`].
+///
+/// # Invariants
+///
+/// The inner [`Opaque`] contains a valid, allocated `gpu_buddy_block`.
+#[repr(transparent)]
+pub struct Block(Opaque<bindings::gpu_buddy_block>);
+
+impl Block {
+ /// Get a raw pointer to the underlying C block.
+ fn as_raw(&self) -> *mut bindings::gpu_buddy_block {
+ self.0.get()
+ }
+
+ /// Get the block's offset in the address space.
+ pub(crate) fn offset(&self) -> u64 {
+ // SAFETY: self.as_raw() is valid per the type's invariants.
+ unsafe { bindings::gpu_buddy_block_offset(self.as_raw()) }
+ }
+
+ /// Get the block order.
+ pub(crate) fn order(&self) -> u32 {
+ // SAFETY: self.as_raw() is valid per the type's invariants.
+ unsafe { bindings::gpu_buddy_block_order(self.as_raw()) }
+ }
+}
+
+// SAFETY: `Block` is a transparent wrapper over `gpu_buddy_block` which is not
+// modified after allocation. It can be safely sent between threads.
+unsafe impl Send for Block {}
+
+// SAFETY: `Block` is a transparent wrapper over `gpu_buddy_block` which is not
+// modified after allocation. It can be safely shared among threads.
+unsafe impl Sync for Block {}
+
+/// An allocated block with access to the allocation list.
+///
+/// # Invariants
+///
+/// - `block` is a valid reference to an allocated [`Block`].
+/// - `alloc` is a valid reference to the [`AllocatedBlocks`] that owns this block.
+pub struct AllocatedBlock<'a> {
+ block: &'a Block,
+ alloc: &'a AllocatedBlocks,
+}
+
+impl AllocatedBlock<'_> {
+ /// Get the block's offset in the address space.
+ ///
+ /// Returns the absolute offset including the allocator's base offset.
+ /// This is the actual address to use for accessing the allocated memory.
+ pub fn offset(&self) -> u64 {
+ self.alloc.buddy.base_offset + self.block.offset()
+ }
+
+ /// Get the block order (size = chunk_size << order).
+ pub fn order(&self) -> u32 {
+ self.block.order()
+ }
+
+ /// Get the block's size in bytes.
+ pub fn size(&self) -> u64 {
+ self.alloc.buddy.chunk_size << self.block.order()
+ }
+}
diff --git a/rust/kernel/gpu/mod.rs b/rust/kernel/gpu/mod.rs
new file mode 100644
index 000000000000..8f25e6367edc
--- /dev/null
+++ b/rust/kernel/gpu/mod.rs
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! GPU subsystem abstractions.
+
+pub mod buddy;
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 8439c30f40b5..e30faa5ca8f4 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -99,6 +99,8 @@
pub mod firmware;
pub mod fmt;
pub mod fs;
+#[cfg(CONFIG_GPU_BUDDY)]
+pub mod gpu;
#[cfg(CONFIG_I2C = "y")]
pub mod i2c;
pub mod id_pool;
--
2.34.1
^ permalink raw reply related
* Re: [PATCH RFC v6 02/26] gpu: Move DRM buddy allocator one level up
From: Joel Fernandes @ 2026-02-06 1:04 UTC (permalink / raw)
To: Dave Airlie
Cc: linux-kernel, Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Alexey Ivanov, Balbir Singh,
Philipp Stanner, Elle Rhumsaa, Daniel Almeida, joel, nouveau,
dri-devel, rust-for-linux, linux-doc, amd-gfx, intel-gfx,
intel-xe, linux-fbdev
In-Reply-To: <CAPM=9tz5iXoFQ3+4hPFW+tZCL2zWe0WJ07-oFkP8TNVL_J_SSg@mail.gmail.com>
On 2/5/2026 3:55 PM, Dave Airlie wrote:
> On Wed, 21 Jan 2026 at 06:44, Joel Fernandes <joelagnelf@nvidia.com> wrote:
>>
>> Move the DRM buddy allocator one level up so that it can be used by GPU
>> drivers (example, nova-core) that have usecases other than DRM (such as
>> VFIO vGPU support). Modify the API, structures and Kconfigs to use
>> "gpu_buddy" terminology. Adapt the drivers and tests to use the new API.
>>
>> The commit cannot be split due to bisectability, however no functional
>> change is intended. Verified by running K-UNIT tests and build tested
>> various configurations.
>>
>> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
>
> I suggested this and think it's a good idea.
>
> Reviewed-by: Dave Airlie <airlied@redhat.com>
Thanks, Dave!
--
Joel Fernandes
^ permalink raw reply
* Re: [PATCH -next] gpu: Move DRM buddy allocator one level up
From: Joel Fernandes @ 2026-02-06 1:06 UTC (permalink / raw)
To: linux-kernel
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Alexey Ivanov, Balbir Singh,
Philipp Stanner, Elle Rhumsaa, Daniel Almeida, joel, nouveau,
dri-devel, rust-for-linux, linux-doc, amd-gfx, intel-gfx,
intel-xe, linux-fbdev
In-Reply-To: <20260206003451.1914130-1-joelagnelf@nvidia.com>
On 2/5/2026 7:34 PM, Joel Fernandes wrote:
> Move the DRM buddy allocator one level up so that it can be used by GPU
> drivers (example, nova-core) that have usecases other than DRM (such as
> VFIO vGPU support). Modify the API, structures and Kconfigs to use
> "gpu_buddy" terminology. Adapt the drivers and tests to use the new API.
>
> The commit cannot be split due to bisectability, however no functional
> change is intended. Verified by running K-UNIT tests and build tested
> various configurations.
>
> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
I forgot to add Dave Airlie's review tag here from his other email. When you
apply, could you add it?
Or, let me know if I should resend it. Thanks.
--
Joel Fernandes
^ permalink raw reply
* Re: [PATCH RFC v6 02/26] gpu: Move DRM buddy allocator one level up
From: Dave Airlie @ 2026-02-06 1:07 UTC (permalink / raw)
To: Joel Fernandes
Cc: linux-kernel, Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Alexey Ivanov, Balbir Singh,
Philipp Stanner, Elle Rhumsaa, Daniel Almeida, joel, nouveau,
dri-devel, rust-for-linux, linux-doc, amd-gfx, intel-gfx,
intel-xe, linux-fbdev
In-Reply-To: <44542f0b-be3d-4e82-aacf-0bf19ab69954@nvidia.com>
On Fri, 6 Feb 2026 at 11:04, Joel Fernandes <joelagnelf@nvidia.com> wrote:
>
>
>
> On 2/5/2026 3:55 PM, Dave Airlie wrote:
> > On Wed, 21 Jan 2026 at 06:44, Joel Fernandes <joelagnelf@nvidia.com> wrote:
> >>
> >> Move the DRM buddy allocator one level up so that it can be used by GPU
> >> drivers (example, nova-core) that have usecases other than DRM (such as
> >> VFIO vGPU support). Modify the API, structures and Kconfigs to use
> >> "gpu_buddy" terminology. Adapt the drivers and tests to use the new API.
> >>
> >> The commit cannot be split due to bisectability, however no functional
> >> change is intended. Verified by running K-UNIT tests and build tested
> >> various configurations.
> >>
> >> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
> >
> > I suggested this and think it's a good idea.
> >
> > Reviewed-by: Dave Airlie <airlied@redhat.com>
> Thanks, Dave!
I'm going to apply this to drm-misc-next today but I'll move some of it around.
Dave.
^ permalink raw reply
* Re: [PATCH -next] gpu: Move DRM buddy allocator one level up
From: Dave Airlie @ 2026-02-06 1:48 UTC (permalink / raw)
To: Joel Fernandes
Cc: linux-kernel, Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Alexey Ivanov, Balbir Singh,
Philipp Stanner, Elle Rhumsaa, Daniel Almeida, joel, nouveau,
dri-devel, rust-for-linux, linux-doc, amd-gfx, intel-gfx,
intel-xe, linux-fbdev
In-Reply-To: <06ff773a-06bf-4d60-bd0a-75a0359ce41c@nvidia.com>
On Fri, 6 Feb 2026 at 11:06, Joel Fernandes <joelagnelf@nvidia.com> wrote:
>
>
>
> On 2/5/2026 7:34 PM, Joel Fernandes wrote:
> > Move the DRM buddy allocator one level up so that it can be used by GPU
> > drivers (example, nova-core) that have usecases other than DRM (such as
> > VFIO vGPU support). Modify the API, structures and Kconfigs to use
> > "gpu_buddy" terminology. Adapt the drivers and tests to use the new API.
> >
> > The commit cannot be split due to bisectability, however no functional
> > change is intended. Verified by running K-UNIT tests and build tested
> > various configurations.
> >
> > Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
>
> I forgot to add Dave Airlie's review tag here from his other email. When you
> apply, could you add it?
>
> Or, let me know if I should resend it. Thanks.
>
This one was a bit messy, so I've taken this into drm-misc-next now,
it doesn't quite end up in the same places as yours, but it looks the
same, and I updated MAINTAINERS at the end.
Now you can just care about the rust side of it.
Regards,
Dave.
^ permalink raw reply
* Re: [PATCH] fbdev/ffb: fix corrupted video output on FFB1
From: Helge Deller @ 2026-02-06 10:12 UTC (permalink / raw)
To: René Rebe; +Cc: linux-fbdev
In-Reply-To: <20260205.164958.765506119384437798.rene@exactco.de>
Hello Rene,
On 2/5/26 16:49, René Rebe wrote:
> Fix Sun FFB1 corrupted video out [1] by disabling overlay and
> initializing window mode to a known state. The issue never appeared on
> my FFB2+/vertical nor Elite3D/M6. It could also depend on the PROM
> version.
>
> /SUNW,ffb@1e,0: FFB at 000001fc00000000, type 11, DAC pnum[236c] rev[10] manuf_rev[4]
> X (II) /dev/fb0: Detected FFB1, Z-buffer, Single-buffered.
> X (II) /dev/fb0: BT9068 (PAC1) ramdac detected (with normal cursor control)
> X (II) /dev/fb0: Detected Creator/Creator3D
>
> [1] https://www.instagram.com/p/DUTcSmSjSem/
The patch itself seems ok.
But could you please link to a freely accessible website instead of Instagram?
I and many others don't have (and don't want) an instagram account, so
the link is not very useful.
Helge
> Signed-off-by: René Rebe <rene@exactco.de>
> Cc: stable@kernel.org
> ---
> Tested on Sun Ultra 2 w/ FFB1 BT9068 (PAC1) ramdac running T2/Linux.
> Defines re-used from xf86-video-sunffb.
> ---
> diff --git a/drivers/video/fbdev/ffb.c b/drivers/video/fbdev/ffb.c
> index 34b6abff9493..8d77f102dd82 100644
> --- a/drivers/video/fbdev/ffb.c
> +++ b/drivers/video/fbdev/ffb.c
> @@ -335,6 +335,9 @@ struct ffb_dac {
> };
>
> #define FFB_DAC_UCTRL 0x1001 /* User Control */
> +#define FFB_DAC_UCTRL_OVENAB 0x00000008 /* Overlay Enable */
> +#define FFB_DAC_UCTRL_WMODE 0x00000030 /* Window Mode */
> +#define FFB_DAC_UCTRL_WM_COMB 0x00000000 /* Window Mode = Combined */
> #define FFB_DAC_UCTRL_MANREV 0x00000f00 /* 4-bit Manufacturing Revision */
> #define FFB_DAC_UCTRL_MANREV_SHIFT 8
> #define FFB_DAC_TGEN 0x6000 /* Timing Generator */
> @@ -425,7 +428,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
> {
> struct ffb_fbc __iomem *fbc = par->fbc;
> struct ffb_dac __iomem *dac = par->dac;
> - unsigned long flags;
> + unsigned long flags, uctrl;
>
> spin_lock_irqsave(&par->lock, flags);
> FFBWait(par);
> @@ -442,7 +445,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
> upa_writel(par->bg_cache, &fbc->bg);
> FFBWait(par);
>
> - /* Disable cursor. */
> + /* Disable cursor. */
> upa_writel(FFB_DAC_CUR_CTRL, &dac->type2);
> if (par->flags & FFB_FLAG_INVCURSOR)
> upa_writel(0, &dac->value2);
> @@ -450,6 +453,15 @@ static void ffb_switch_from_graph(struct ffb_par *par)
> upa_writel((FFB_DAC_CUR_CTRL_P0 |
> FFB_DAC_CUR_CTRL_P1), &dac->value2);
>
> + /* Disable overlay and window modes. */
> + upa_writel(FFB_DAC_UCTRL, &dac->type);
> + uctrl = upa_readl(&dac->value);
> + uctrl &= ~FFB_DAC_UCTRL_WMODE;
> + uctrl |= FFB_DAC_UCTRL_WM_COMB;
> + uctrl &= ~FFB_DAC_UCTRL_OVENAB;
> + upa_writel(FFB_DAC_UCTRL, &dac->type);
> + upa_writel(uctrl, &dac->value);
> +
> spin_unlock_irqrestore(&par->lock, flags);
> }
>
>
^ permalink raw reply
* Re: [PATCH] fbdev/ffb: fix corrupted video output on FFB1
From: René Rebe @ 2026-02-06 10:30 UTC (permalink / raw)
To: Helge Deller; +Cc: linux-fbdev
In-Reply-To: <685bb544-1976-45ad-a6bd-f298e2b05cb0@gmx.de>
Hi,
> On 6. Feb 2026, at 11:12, Helge Deller <deller@gmx.de> wrote:
>
> Hello Rene,
>
> On 2/5/26 16:49, René Rebe wrote:
>> Fix Sun FFB1 corrupted video out [1] by disabling overlay and
>> initializing window mode to a known state. The issue never appeared on
>> my FFB2+/vertical nor Elite3D/M6. It could also depend on the PROM
>> version.
>> /SUNW,ffb@1e,0: FFB at 000001fc00000000, type 11, DAC pnum[236c] rev[10] manuf_rev[4]
>> X (II) /dev/fb0: Detected FFB1, Z-buffer, Single-buffered.
>> X (II) /dev/fb0: BT9068 (PAC1) ramdac detected (with normal cursor control)
>> X (II) /dev/fb0: Detected Creator/Creator3D
>> [1] https://www.instagram.com/p/DUTcSmSjSem/
>
> The patch itself seems ok.
> But could you please link to a freely accessible website instead of Instagram?
> I and many others don't have (and don't want) an instagram account, so
> the link is not very useful.
Well it is more free than accessing Linux Foundation training material ;-)
It was only for illustration purposes if someone wanted to see or follow random
daily development fun and not that important. Should I resend it w/o it?
Thanks!
René
> Helge
>
>> Signed-off-by: René Rebe <rene@exactco.de>
>> Cc: stable@kernel.org
>> ---
>> Tested on Sun Ultra 2 w/ FFB1 BT9068 (PAC1) ramdac running T2/Linux.
>> Defines re-used from xf86-video-sunffb.
>> ---
>> diff --git a/drivers/video/fbdev/ffb.c b/drivers/video/fbdev/ffb.c
>> index 34b6abff9493..8d77f102dd82 100644
>> --- a/drivers/video/fbdev/ffb.c
>> +++ b/drivers/video/fbdev/ffb.c
>> @@ -335,6 +335,9 @@ struct ffb_dac {
>> };
>> #define FFB_DAC_UCTRL 0x1001 /* User Control */
>> +#define FFB_DAC_UCTRL_OVENAB 0x00000008 /* Overlay Enable */
>> +#define FFB_DAC_UCTRL_WMODE 0x00000030 /* Window Mode */
>> +#define FFB_DAC_UCTRL_WM_COMB 0x00000000 /* Window Mode = Combined */
>> #define FFB_DAC_UCTRL_MANREV 0x00000f00 /* 4-bit Manufacturing Revision */
>> #define FFB_DAC_UCTRL_MANREV_SHIFT 8
>> #define FFB_DAC_TGEN 0x6000 /* Timing Generator */
>> @@ -425,7 +428,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
>> {
>> struct ffb_fbc __iomem *fbc = par->fbc;
>> struct ffb_dac __iomem *dac = par->dac;
>> - unsigned long flags;
>> + unsigned long flags, uctrl;
>> spin_lock_irqsave(&par->lock, flags);
>> FFBWait(par);
>> @@ -442,7 +445,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
>> upa_writel(par->bg_cache, &fbc->bg);
>> FFBWait(par);
>> - /* Disable cursor. */
>> + /* Disable cursor. */
>> upa_writel(FFB_DAC_CUR_CTRL, &dac->type2);
>> if (par->flags & FFB_FLAG_INVCURSOR)
>> upa_writel(0, &dac->value2);
>> @@ -450,6 +453,15 @@ static void ffb_switch_from_graph(struct ffb_par *par)
>> upa_writel((FFB_DAC_CUR_CTRL_P0 |
>> FFB_DAC_CUR_CTRL_P1), &dac->value2);
>> + /* Disable overlay and window modes. */
>> + upa_writel(FFB_DAC_UCTRL, &dac->type);
>> + uctrl = upa_readl(&dac->value);
>> + uctrl &= ~FFB_DAC_UCTRL_WMODE;
>> + uctrl |= FFB_DAC_UCTRL_WM_COMB;
>> + uctrl &= ~FFB_DAC_UCTRL_OVENAB;
>> + upa_writel(FFB_DAC_UCTRL, &dac->type);
>> + upa_writel(uctrl, &dac->value);
>> +
>> spin_unlock_irqrestore(&par->lock, flags);
>> }
>>
>
--
https://exactco.de • https://t2linux.com • https://patreon.com/renerebe
^ permalink raw reply
* Re: [PATCH v5] fbtft: limit dirty rows based on damage range
From: Andy Shevchenko @ 2026-02-06 10:36 UTC (permalink / raw)
To: Nam Cao; +Cc: ChanSoo Shin, andy, gregkh, dri-devel, linux-fbdev, linux-staging
In-Reply-To: <87y0l668ki.fsf@yellow.woof>
On Fri, Feb 06, 2026 at 01:04:45AM +0100, Nam Cao wrote:
> ChanSoo Shin <csshin9928@gmail.com> writes:
> > Instead of marking the entire display as dirty, calculate the start
> > and end rows based on the damage offset and length and only mark the
> > affected rows dirty. This reduces unnecessary full framebuffer updates
> > for partial writes.
>
> This looks useful and I prefer to see it applied. I understand lack of
> testing is the main obstacle.
Correct.
> Are you still trying to get it merged? If not, I can take over.
Do you have an actual HW to test? If not, won't be applied either.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* Re: [PATCH] fbdev/ffb: fix corrupted video output on FFB1
From: René Rebe @ 2026-02-06 10:40 UTC (permalink / raw)
To: Helge Deller; +Cc: linux-fbdev
In-Reply-To: <CC6760AE-15B4-45FC-B796-577AAE728ADD@exactco.de>
Hello again,
> On 6. Feb 2026, at 11:30, René Rebe <rene@exactco.de> wrote:
>
> Hi,
>
>> On 6. Feb 2026, at 11:12, Helge Deller <deller@gmx.de> wrote:
>>
>> Hello Rene,
>>
>> On 2/5/26 16:49, René Rebe wrote:
>>> Fix Sun FFB1 corrupted video out [1] by disabling overlay and
>>> initializing window mode to a known state. The issue never appeared on
>>> my FFB2+/vertical nor Elite3D/M6. It could also depend on the PROM
>>> version.
>>> /SUNW,ffb@1e,0: FFB at 000001fc00000000, type 11, DAC pnum[236c] rev[10] manuf_rev[4]
>>> X (II) /dev/fb0: Detected FFB1, Z-buffer, Single-buffered.
>>> X (II) /dev/fb0: BT9068 (PAC1) ramdac detected (with normal cursor control)
>>> X (II) /dev/fb0: Detected Creator/Creator3D
>>> [1] https://www.instagram.com/p/DUTcSmSjSem/
>>
>> The patch itself seems ok.
>> But could you please link to a freely accessible website instead of Instagram?
>> I and many others don't have (and don't want) an instagram account, so
>> the link is not very useful.
>
> Well it is more free than accessing Linux Foundation training material ;-)
> It was only for illustration purposes if someone wanted to see or follow random
> daily development fun and not that important. Should I resend it w/o it?
While I try to influence next gen to get into OpenSource and low level programming,
how about Mastodon links?
https://chaos.social/@ReneRebe/116023241660154102
Do you want to edit the patch or should I send a V2?
René
> Thanks!
> René
>
>> Helge
>>
>>> Signed-off-by: René Rebe <rene@exactco.de>
>>> Cc: stable@kernel.org
>>> ---
>>> Tested on Sun Ultra 2 w/ FFB1 BT9068 (PAC1) ramdac running T2/Linux.
>>> Defines re-used from xf86-video-sunffb.
>>> ---
>>> diff --git a/drivers/video/fbdev/ffb.c b/drivers/video/fbdev/ffb.c
>>> index 34b6abff9493..8d77f102dd82 100644
>>> --- a/drivers/video/fbdev/ffb.c
>>> +++ b/drivers/video/fbdev/ffb.c
>>> @@ -335,6 +335,9 @@ struct ffb_dac {
>>> };
>>> #define FFB_DAC_UCTRL 0x1001 /* User Control */
>>> +#define FFB_DAC_UCTRL_OVENAB 0x00000008 /* Overlay Enable */
>>> +#define FFB_DAC_UCTRL_WMODE 0x00000030 /* Window Mode */
>>> +#define FFB_DAC_UCTRL_WM_COMB 0x00000000 /* Window Mode = Combined */
>>> #define FFB_DAC_UCTRL_MANREV 0x00000f00 /* 4-bit Manufacturing Revision */
>>> #define FFB_DAC_UCTRL_MANREV_SHIFT 8
>>> #define FFB_DAC_TGEN 0x6000 /* Timing Generator */
>>> @@ -425,7 +428,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
>>> {
>>> struct ffb_fbc __iomem *fbc = par->fbc;
>>> struct ffb_dac __iomem *dac = par->dac;
>>> - unsigned long flags;
>>> + unsigned long flags, uctrl;
>>> spin_lock_irqsave(&par->lock, flags);
>>> FFBWait(par);
>>> @@ -442,7 +445,7 @@ static void ffb_switch_from_graph(struct ffb_par *par)
>>> upa_writel(par->bg_cache, &fbc->bg);
>>> FFBWait(par);
>>> - /* Disable cursor. */
>>> + /* Disable cursor. */
>>> upa_writel(FFB_DAC_CUR_CTRL, &dac->type2);
>>> if (par->flags & FFB_FLAG_INVCURSOR)
>>> upa_writel(0, &dac->value2);
>>> @@ -450,6 +453,15 @@ static void ffb_switch_from_graph(struct ffb_par *par)
>>> upa_writel((FFB_DAC_CUR_CTRL_P0 |
>>> FFB_DAC_CUR_CTRL_P1), &dac->value2);
>>> + /* Disable overlay and window modes. */
>>> + upa_writel(FFB_DAC_UCTRL, &dac->type);
>>> + uctrl = upa_readl(&dac->value);
>>> + uctrl &= ~FFB_DAC_UCTRL_WMODE;
>>> + uctrl |= FFB_DAC_UCTRL_WM_COMB;
>>> + uctrl &= ~FFB_DAC_UCTRL_OVENAB;
>>> + upa_writel(FFB_DAC_UCTRL, &dac->type);
>>> + upa_writel(uctrl, &dac->value);
>>> +
>>> spin_unlock_irqrestore(&par->lock, flags);
>>> }
>>>
>>
>
> --
> https://exactco.de • https://t2linux.com • https://patreon.com/renerebe
>
--
https://exactco.de • https://t2linux.com • https://patreon.com/renerebe
^ permalink raw reply
* Re: [PATCH] fbdev/ffb: fix corrupted video output on FFB1
From: Helge Deller @ 2026-02-06 10:41 UTC (permalink / raw)
To: René Rebe; +Cc: linux-fbdev
In-Reply-To: <CC6760AE-15B4-45FC-B796-577AAE728ADD@exactco.de>
On 2/6/26 11:30, René Rebe wrote:
>> On 6. Feb 2026, at 11:12, Helge Deller <deller@gmx.de> wrote:
>>
>> Hello Rene,
>>
>> On 2/5/26 16:49, René Rebe wrote:
>>> Fix Sun FFB1 corrupted video out [1] by disabling overlay and
>>> initializing window mode to a known state. The issue never appeared on
>>> my FFB2+/vertical nor Elite3D/M6. It could also depend on the PROM
>>> version.
>>> /SUNW,ffb@1e,0: FFB at 000001fc00000000, type 11, DAC pnum[236c] rev[10] manuf_rev[4]
>>> X (II) /dev/fb0: Detected FFB1, Z-buffer, Single-buffered.
>>> X (II) /dev/fb0: BT9068 (PAC1) ramdac detected (with normal cursor control)
>>> X (II) /dev/fb0: Detected Creator/Creator3D
>>> [1] https://www.instagram.com/p/DUTcSmSjSem/
>>
>> The patch itself seems ok.
>> But could you please link to a freely accessible website instead of Instagram?
>> I and many others don't have (and don't want) an instagram account, so
>> the link is not very useful.
>
> Well it is more free than accessing Linux Foundation training material ;-)
> It was only for illustration purposes if someone wanted to see or follow random
> daily development fun and not that important. Should I resend it w/o it?
No need to resend.
I asked, because I don't know what's visible there, and maybe you
had another link which could have been used instead.
It seems that's not the case, so I'll take the patch as-is.
applied to fbdev git tree.
Thanks!
Helge
^ permalink raw reply
* Re: [PATCH v5] fbtft: limit dirty rows based on damage range
From: Nam Cao @ 2026-02-06 10:51 UTC (permalink / raw)
To: Andy Shevchenko
Cc: ChanSoo Shin, andy, gregkh, dri-devel, linux-fbdev, linux-staging
In-Reply-To: <aYXEHajVeKvIMmL7@smile.fi.intel.com>
Andy Shevchenko <andriy.shevchenko@intel.com> writes:
> Do you have an actual HW to test? If not, won't be applied either.
Yes, a pair with SPI interface.
Nam
^ permalink raw reply
* Re: [PATCH] video: of_display_timing: Fix device node reference leak in of_get_display_timings()
From: Helge Deller @ 2026-02-06 10:52 UTC (permalink / raw)
To: Felix Gu, Thierry Reding, Laurent Pinchart, Philipp Zabel,
Stephen Warren, Steffen Trumtrar
Cc: linux-fbdev, dri-devel, linux-kernel
In-Reply-To: <20260131-of_display_timging-v1-1-f4f8d9f87c3c@gmail.com>
On 1/31/26 13:48, Felix Gu wrote:
> Use for_each_child_of_node_scoped instead of for_each_child_of_node
> to ensure automatic of_node_put on early exit paths, preventing
> device node reference leak.
>
> Fixes: cc3f414cf2e4 ("video: add of helper for display timings/videomode")
> Signed-off-by: Felix Gu <ustc.gu@gmail.com>
> ---
> drivers/video/of_display_timing.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
applied.
Thanks!
Helge
^ permalink raw reply
* Re: [PATCH v1 1/3] fbdev: au1100fb: Mark several local functions as static
From: Helge Deller @ 2026-02-06 11:06 UTC (permalink / raw)
To: Uwe Kleine-König; +Cc: Chen Ni, linux-fbdev, dri-devel
In-Reply-To: <ceb08e29f6a07075b5ca63e4ed30c5051fddcdfd.1770196161.git.u.kleine-koenig@baylibre.com>
Hi Uwe,
On 2/4/26 10:15, Uwe Kleine-König wrote:
> This fixes several (fatal) compiler warnings à la
>
> drivers/video/fbdev/au1100fb.c:530:6: error: no previous prototype for ‘au1100fb_drv_remove’ [-Werror=missing-prototypes]
> 523 | void au1100fb_drv_remove(struct platform_device *dev)
> | ^~~~~~~~~~~~~~~~~~~
>
> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
I've applied patches #1 and #3 of this series to the fbdev git tree.
Patch #2 needs fixing, as it breaks build on s390x.
Helge
^ permalink raw reply
* Re: [PATCH v1 1/3] fbdev: au1100fb: Mark several local functions as static
From: Helge Deller @ 2026-02-06 11:22 UTC (permalink / raw)
To: Uwe Kleine-König; +Cc: Chen Ni, linux-fbdev, dri-devel
In-Reply-To: <cf0433eb-e22d-498b-93a2-04b461aaa6e2@gmx.de>
On 2/6/26 12:06, Helge Deller wrote:
> Hi Uwe,
>
> On 2/4/26 10:15, Uwe Kleine-König wrote:
>> This fixes several (fatal) compiler warnings à la
>>
>> drivers/video/fbdev/au1100fb.c:530:6: error: no previous prototype for ‘au1100fb_drv_remove’ [-Werror=missing-prototypes]
>> 523 | void au1100fb_drv_remove(struct platform_device *dev)
>> | ^~~~~~~~~~~~~~~~~~~
>>
>> Signed-off-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
>
> I've applied patches #1 and #3 of this series to the fbdev git tree.
>
> Patch #2 needs fixing, as it breaks build on s390x.
I fixed up patch #2 manually for now by excluding s390x as test platform.
If you have a better patch, I'm happy to take it.
Thanks!
Helge
^ permalink raw reply
* Re: [PATCH v5] fbtft: limit dirty rows based on damage range
From: Andy Shevchenko @ 2026-02-06 11:35 UTC (permalink / raw)
To: Nam Cao; +Cc: ChanSoo Shin, andy, gregkh, dri-devel, linux-fbdev, linux-staging
In-Reply-To: <87ms1mb0vy.fsf@yellow.woof>
On Fri, Feb 06, 2026 at 05:51:45PM +0700, Nam Cao wrote:
> Andy Shevchenko <andriy.shevchenko@intel.com> writes:
> > Do you have an actual HW to test? If not, won't be applied either.
>
> Yes, a pair with SPI interface.
Cool! Would be nice to have tested contribution, indeed.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* Re: [PATCH v1 1/3] fbdev: au1100fb: Mark several local functions as static
From: Uwe Kleine-König @ 2026-02-06 14:15 UTC (permalink / raw)
To: Helge Deller; +Cc: Chen Ni, linux-fbdev, dri-devel
In-Reply-To: <28affc45-7f41-40a5-82bf-308a6e34d956@gmx.de>
[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]
Hello Helge,
On Fri, Feb 06, 2026 at 12:22:39PM +0100, Helge Deller wrote:
> On 2/6/26 12:06, Helge Deller wrote:
> > On 2/4/26 10:15, Uwe Kleine-König wrote:
> > > This fixes several (fatal) compiler warnings à la
> > >
> > > drivers/video/fbdev/au1100fb.c:530:6: error: no previous prototype for ‘au1100fb_drv_remove’ [-Werror=missing-prototypes]
> > > 523 | void au1100fb_drv_remove(struct platform_device *dev)
> > > | ^~~~~~~~~~~~~~~~~~~
> > >
> > > Signed-off-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
> >
> > I've applied patches #1 and #3 of this series to the fbdev git tree.
> >
> > Patch #2 needs fixing, as it breaks build on s390x.
>
> I fixed up patch #2 manually for now by excluding s390x as test platform.
> If you have a better patch, I'm happy to take it.
I didn't look at that yet, but I suspect that the failure isn't s390x
specific and I think it's better to not relax the dependencies yet.
Best regards
Uwe
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* Re: [PATCH -next v7 1/2] rust: clist: Add support to interface with C linked lists
From: Gary Guo @ 2026-02-06 15:25 UTC (permalink / raw)
To: Joel Fernandes, linux-kernel
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, Jonathan Corbet, Alex Deucher,
Christian König, Jani Nikula, Joonas Lahtinen, Rodrigo Vivi,
Tvrtko Ursulin, Huang Rui, Matthew Auld, Matthew Brost,
Lucas De Marchi, Thomas Hellström, Helge Deller,
Danilo Krummrich, Alice Ryhl, Miguel Ojeda, Alex Gaynor,
Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin,
Andreas Hindborg, Trevor Gross, John Hubbard, Alistair Popple,
Timur Tabi, Edwin Peer, Alexandre Courbot, Andrea Righi,
Andy Ritger, Zhi Wang, Balbir Singh, Philipp Stanner,
Elle Rhumsaa, Daniel Almeida, joel, nouveau, dri-devel,
rust-for-linux, linux-doc, amd-gfx, intel-gfx, intel-xe,
linux-fbdev
In-Reply-To: <20260206004110.1914814-2-joelagnelf@nvidia.com>
On Fri Feb 6, 2026 at 12:41 AM GMT, Joel Fernandes wrote:
> Add a new module `clist` for working with C's doubly circular linked
> lists. Provide low-level iteration over list nodes.
>
> Typed iteration over actual items is provided with a `clist_create`
> macro to assist in creation of the `CList` type.
>
> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
> ---
> MAINTAINERS | 7 +
> drivers/gpu/Kconfig | 7 +
> rust/helpers/helpers.c | 1 +
> rust/helpers/list.c | 21 +++
> rust/kernel/clist.rs | 315 +++++++++++++++++++++++++++++++++++++++++
> rust/kernel/lib.rs | 2 +
> 6 files changed, 353 insertions(+)
> create mode 100644 rust/helpers/list.c
> create mode 100644 rust/kernel/clist.rs
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 900fc00b73e6..310bb479260c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -23204,6 +23204,13 @@ S: Maintained
> T: git https://github.com/Rust-for-Linux/linux.git rust-analyzer-next
> F: scripts/generate_rust_analyzer.py
>
> +RUST TO C LIST INTERFACES
> +M: Joel Fernandes <joelagnelf@nvidia.com>
> +M: Alexandre Courbot <acourbot@nvidia.com>
> +L: rust-for-linux@vger.kernel.org
> +S: Maintained
> +F: rust/kernel/clist.rs
I still think we should try to work on a more powerful list infra that works for
both C and Rust, but I reckon that is a longer term effort, and shouldn't
prevent a simpler version from getting in and be used by abstractions that need
it. So
Acked-by: Gary Guo <gary@garyguo.net>
Some nits below:
> +
> RXRPC SOCKETS (AF_RXRPC)
> M: David Howells <dhowells@redhat.com>
> M: Marc Dionne <marc.dionne@auristor.com>
> diff --git a/drivers/gpu/Kconfig b/drivers/gpu/Kconfig
> index 22dd29cd50b5..2c3dec070645 100644
> --- a/drivers/gpu/Kconfig
> +++ b/drivers/gpu/Kconfig
> @@ -1,7 +1,14 @@
> # SPDX-License-Identifier: GPL-2.0
>
> +config RUST_CLIST
> + bool
> + depends on RUST
> + help
> + Rust abstraction for interfacing with C linked lists.
I am not sure if we need extra config entry. This is fully generic so shouldn't
generate any code unless there is an user.
> +
> config GPU_BUDDY
> bool
> + select RUST_CLIST if RUST
> help
> A page based buddy allocator for GPU memory.
>
> diff --git a/rust/helpers/helpers.c b/rust/helpers/helpers.c
> index a3c42e51f00a..724fcb8240ac 100644
> --- a/rust/helpers/helpers.c
> +++ b/rust/helpers/helpers.c
> @@ -35,6 +35,7 @@
> #include "io.c"
> #include "jump_label.c"
> #include "kunit.c"
> +#include "list.c"
> #include "maple_tree.c"
> #include "mm.c"
> #include "mutex.c"
> diff --git a/rust/helpers/list.c b/rust/helpers/list.c
> new file mode 100644
> index 000000000000..3390b154fa36
> --- /dev/null
> +++ b/rust/helpers/list.c
> @@ -0,0 +1,21 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +/*
> + * Helpers for C Circular doubly linked list implementation.
> + */
> +
> +#include <linux/list.h>
> +
> +#ifndef __rust_helper
> +#define __rust_helper
> +#endif
This shouldn't be needed.
> +
> +__rust_helper void rust_helper_INIT_LIST_HEAD(struct list_head *list)
> +{
> + INIT_LIST_HEAD(list);
> +}
> +
> +__rust_helper void rust_helper_list_add_tail(struct list_head *new, struct list_head *head)
> +{
> + list_add_tail(new, head);
> +}
> diff --git a/rust/kernel/clist.rs b/rust/kernel/clist.rs
> new file mode 100644
> index 000000000000..1f6d4db13c1d
> --- /dev/null
> +++ b/rust/kernel/clist.rs
> @@ -0,0 +1,315 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! A C doubly circular intrusive linked list interface for rust code.
> +//!
> +//! # Examples
> +//!
> +//! ```
> +//! use kernel::{
> +//! bindings,
> +//! clist_create,
> +//! types::Opaque, //
> +//! };
> +//! # // Create test list with values (0, 10, 20) - normally done by C code but it is
> +//! # // emulated here for doctests using the C bindings.
> +//! # use core::mem::MaybeUninit;
> +//! #
> +//! # /// C struct with embedded `list_head` (typically will be allocated by C code).
> +//! # #[repr(C)]
> +//! # pub(crate) struct SampleItemC {
> +//! # pub value: i32,
> +//! # pub link: bindings::list_head,
> +//! # }
> +//! #
> +//! # let mut head = MaybeUninit::<bindings::list_head>::uninit();
> +//! #
> +//! # let head = head.as_mut_ptr();
> +//! # // SAFETY: head and all the items are test objects allocated in this scope.
> +//! # unsafe { bindings::INIT_LIST_HEAD(head) };
> +//! #
> +//! # let mut items = [
> +//! # MaybeUninit::<SampleItemC>::uninit(),
> +//! # MaybeUninit::<SampleItemC>::uninit(),
> +//! # MaybeUninit::<SampleItemC>::uninit(),
> +//! # ];
> +//! #
> +//! # for (i, item) in items.iter_mut().enumerate() {
> +//! # let ptr = item.as_mut_ptr();
> +//! # // SAFETY: pointers are to allocated test objects with a list_head field.
> +//! # unsafe {
> +//! # (*ptr).value = i as i32 * 10;
> +//! # // addr_of_mut!() computes address of link directly as link is uninitialized.
> +//! # bindings::INIT_LIST_HEAD(core::ptr::addr_of_mut!((*ptr).link));
> +//! # bindings::list_add_tail(&mut (*ptr).link, head);
> +//! # }
> +//! # }
> +//!
> +//! // Rust wrapper for the C struct.
> +//! // The list item struct in this example is defined in C code as:
> +//! // struct SampleItemC {
> +//! // int value;
> +//! // struct list_head link;
> +//! // };
> +//! //
> +//! #[repr(transparent)]
> +//! pub(crate) struct Item(Opaque<SampleItemC>);
> +//!
> +//! impl Item {
> +//! pub(crate) fn value(&self) -> i32 {
> +//! // SAFETY: [`Item`] has same layout as [`SampleItemC`].
> +//! unsafe { (*self.0.get()).value }
> +//! }
> +//! }
> +//!
> +//! // Create typed [`CList`] from sentinel head.
> +//! // SAFETY: head is valid, items are [`SampleItemC`] with embedded `link` field.
> +//! let list = unsafe { clist_create!(head, Item, SampleItemC, link) };
> +//!
> +//! // Iterate directly over typed items.
> +//! let mut found_0 = false;
> +//! let mut found_10 = false;
> +//! let mut found_20 = false;
> +//!
> +//! for item in list.iter() {
> +//! let val = item.value();
> +//! if val == 0 { found_0 = true; }
> +//! if val == 10 { found_10 = true; }
> +//! if val == 20 { found_20 = true; }
> +//! }
> +//!
> +//! assert!(found_0 && found_10 && found_20);
> +//! ```
> +
> +use core::{
> + iter::FusedIterator,
> + marker::PhantomData, //
> +};
> +
> +use crate::{
> + bindings,
> + types::Opaque, //
> +};
> +
> +use pin_init::PinInit;
> +
> +/// Wraps a `list_head` object for use in intrusive linked lists.
> +///
> +/// # Invariants
> +///
> +/// - [`CListHead`] represents an allocated and valid `list_head` structure.
> +/// - Once a [`CListHead`] is created in Rust, it will not be modified by non-Rust code.
> +/// - All `list_head` for individual items are not modified for the lifetime of [`CListHead`].
> +#[repr(transparent)]
> +pub(crate) struct CListHead(Opaque<bindings::list_head>);
> +
> +impl CListHead {
> + /// Create a `&CListHead` reference from a raw `list_head` pointer.
> + ///
> + /// # Safety
> + ///
> + /// - `ptr` must be a valid pointer to an allocated and initialized `list_head` structure.
> + /// - `ptr` must remain valid and unmodified for the lifetime `'a`.
> + #[inline]
> + pub(crate) unsafe fn from_raw<'a>(ptr: *mut bindings::list_head) -> &'a Self {
> + // SAFETY:
> + // - [`CListHead`] has same layout as `list_head`.
> + // - `ptr` is valid and unmodified for 'a.
> + unsafe { &*ptr.cast() }
> + }
> +
> + /// Get the raw `list_head` pointer.
> + #[inline]
> + pub(crate) fn as_raw(&self) -> *mut bindings::list_head {
> + self.0.get()
> + }
> +
> + /// Get the next [`CListHead`] in the list.
> + #[inline]
> + pub(crate) fn next(&self) -> &Self {
> + let raw = self.as_raw();
> + // SAFETY:
> + // - `self.as_raw()` is valid per type invariants.
> + // - The `next` pointer is guaranteed to be non-NULL.
> + unsafe { Self::from_raw((*raw).next) }
> + }
> +
> + /// Check if this node is linked in a list (not isolated).
> + #[inline]
> + pub(crate) fn is_linked(&self) -> bool {
> + let raw = self.as_raw();
> + // SAFETY: self.as_raw() is valid per type invariants.
> + unsafe { (*raw).next != raw && (*raw).prev != raw }
> + }
> +
> + /// Pin-initializer that initializes the list head.
> + pub(crate) fn new() -> impl PinInit<Self> {
> + // SAFETY: `INIT_LIST_HEAD` initializes `slot` to a valid empty list.
> + unsafe {
> + pin_init::pin_init_from_closure(move |slot: *mut Self| {
pin_init::ffi_init should be used for this.
> + bindings::INIT_LIST_HEAD(slot.cast());
> + Ok(())
> + })
> + }
> + }
> +}
> +
> +// SAFETY: [`CListHead`] can be sent to any thread.
> +unsafe impl Send for CListHead {}
> +
> +// SAFETY: [`CListHead`] can be shared among threads as it is not modified
> +// by non-Rust code per type invariants.
> +unsafe impl Sync for CListHead {}
> +
> +impl PartialEq for CListHead {
#[inline]
> + fn eq(&self, other: &Self) -> bool {
> + core::ptr::eq(self, other)
> + }
> +}
> +
> +impl Eq for CListHead {}
^ permalink raw reply
* Re: [PATCH -next v7 1/2] rust: clist: Add support to interface with C linked lists
From: Danilo Krummrich @ 2026-02-06 15:53 UTC (permalink / raw)
To: Gary Guo
Cc: Joel Fernandes, linux-kernel, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Jonathan Corbet,
Alex Deucher, Christian König, Jani Nikula, Joonas Lahtinen,
Rodrigo Vivi, Tvrtko Ursulin, Huang Rui, Matthew Auld,
Matthew Brost, Lucas De Marchi, Thomas Hellström,
Helge Deller, Alice Ryhl, Miguel Ojeda, Alex Gaynor, Boqun Feng,
Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Trevor Gross, John Hubbard, Alistair Popple, Timur Tabi,
Edwin Peer, Alexandre Courbot, Andrea Righi, Andy Ritger,
Zhi Wang, Balbir Singh, Philipp Stanner, Elle Rhumsaa,
Daniel Almeida, joel, nouveau, dri-devel, rust-for-linux,
linux-doc, amd-gfx, intel-gfx, intel-xe, linux-fbdev
In-Reply-To: <DG7ZF1UT98RQ.3F42J3ULGV2OC@garyguo.net>
On Fri Feb 6, 2026 at 4:25 PM CET, Gary Guo wrote:
> On Fri Feb 6, 2026 at 12:41 AM GMT, Joel Fernandes wrote:
>> diff --git a/drivers/gpu/Kconfig b/drivers/gpu/Kconfig
>> index 22dd29cd50b5..2c3dec070645 100644
>> --- a/drivers/gpu/Kconfig
>> +++ b/drivers/gpu/Kconfig
>> @@ -1,7 +1,14 @@
>> # SPDX-License-Identifier: GPL-2.0
>>
>> +config RUST_CLIST
>> + bool
>> + depends on RUST
>> + help
>> + Rust abstraction for interfacing with C linked lists.
>
> I am not sure if we need extra config entry. This is fully generic so shouldn't
> generate any code unless there is an user.
I also don't think we need a Kconfig for this.
In any case, it shouln't be in drivers/gpu/Kconfig.
>> +
>> config GPU_BUDDY
>> bool
>> + select RUST_CLIST if RUST
If we will have a Kconfig for this, this belongs in the GPU buddy patch.
>> help
>> A page based buddy allocator for GPU memory.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox