* [PATCH v7 01/10] drm/gem: Consider GEM object reclaimable if shrinking fails
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 02/10] drm/panthor: Remove unused panthor_vm_op_ctx::map::new_vma field Boris Brezillon
` (9 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
If the object wasn't moved to a different LRU after the shrink callback
returns, it means the buffer is still reclaimable. Update the remaining
counter to reflect that.
v2:
- Collect R-b
v3:
- Collect R-b
v4:
- No changes
v5:
- No changes
v6:
- No changes
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
drivers/gpu/drm/drm_gem.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 0377a5fd402d..3a309aeaaee5 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1700,6 +1700,16 @@ drm_gem_lru_scan(struct drm_gem_lru *lru,
*/
WARN_ON(obj->lru == &still_in_lru);
WARN_ON(obj->lru == lru);
+ } else if (obj->lru == &still_in_lru) {
+ /*
+ * If the object wasn't moved and wasn't shrunk either,
+ * it's still remaining as reclaimable. Note that
+ * obj->lru is supposed to be checked with the LRU lock
+ * held for an accurate result, but we don't care about
+ * accuracy here. Worst thing that could happen is an
+ * extra scan.
+ */
+ *remaining += obj->size >> PAGE_SHIFT;
}
dma_resv_unlock(obj->resv);
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 02/10] drm/panthor: Remove unused panthor_vm_op_ctx::map::new_vma field
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 01/10] drm/gem: Consider GEM object reclaimable if shrinking fails Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-02 14:31 ` Liviu Dudau
2026-04-01 13:48 ` [PATCH v7 03/10] drm/panthor: Move panthor_gems_debugfs_init() to panthor_gem.c Boris Brezillon
` (8 subsequent siblings)
10 siblings, 1 reply; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
This field is never used, drop it.
v7:
- New patch
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
drivers/gpu/drm/panthor/panthor_mmu.c | 5 -----
1 file changed, 5 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
index f8c41e36afa4..ab67286e3a6e 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.c
+++ b/drivers/gpu/drm/panthor/panthor_mmu.c
@@ -197,11 +197,6 @@ struct panthor_vm_op_ctx {
* to allocate in ::run_job().
*/
struct sg_table *sgt;
-
- /**
- * @map.new_vma: The new VMA object that will be inserted to the VA tree.
- */
- struct panthor_vma *new_vma;
} map;
};
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v7 02/10] drm/panthor: Remove unused panthor_vm_op_ctx::map::new_vma field
2026-04-01 13:48 ` [PATCH v7 02/10] drm/panthor: Remove unused panthor_vm_op_ctx::map::new_vma field Boris Brezillon
@ 2026-04-02 14:31 ` Liviu Dudau
0 siblings, 0 replies; 14+ messages in thread
From: Liviu Dudau @ 2026-04-02 14:31 UTC (permalink / raw)
To: Boris Brezillon
Cc: Steven Price, Adrián Larumbe, dri-devel, David Airlie,
Simona Vetter, Akash Goel, Rob Clark, Sean Paul, Konrad Dybcio,
Akhil P Oommen, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
On Wed, Apr 01, 2026 at 03:48:46PM +0200, Boris Brezillon wrote:
> This field is never used, drop it.
>
> v7:
> - New patch
>
> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Best regards,
Liviu
> ---
> drivers/gpu/drm/panthor/panthor_mmu.c | 5 -----
> 1 file changed, 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
> index f8c41e36afa4..ab67286e3a6e 100644
> --- a/drivers/gpu/drm/panthor/panthor_mmu.c
> +++ b/drivers/gpu/drm/panthor/panthor_mmu.c
> @@ -197,11 +197,6 @@ struct panthor_vm_op_ctx {
> * to allocate in ::run_job().
> */
> struct sg_table *sgt;
> -
> - /**
> - * @map.new_vma: The new VMA object that will be inserted to the VA tree.
> - */
> - struct panthor_vma *new_vma;
> } map;
> };
>
> --
> 2.53.0
>
--
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v7 03/10] drm/panthor: Move panthor_gems_debugfs_init() to panthor_gem.c
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 01/10] drm/gem: Consider GEM object reclaimable if shrinking fails Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 02/10] drm/panthor: Remove unused panthor_vm_op_ctx::map::new_vma field Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 04/10] drm/panthor: Group panthor_kernel_bo_xxx() helpers Boris Brezillon
` (7 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
There's no reason for panthor_drv to know about panthor_gem.c internals,
so let's move the GEM debugfs init logic to panthor_gem.c.
v2:
- Collect R-bs
v3:
-No changes
v4:
- No changes
v5:
- No changes
v6:
- No changes
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
drivers/gpu/drm/panthor/panthor_drv.c | 26 +-----------------------
drivers/gpu/drm/panthor/panthor_gem.c | 29 +++++++++++++++++++++++++--
drivers/gpu/drm/panthor/panthor_gem.h | 3 +--
3 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_drv.c b/drivers/gpu/drm/panthor/panthor_drv.c
index 87d27c3c1456..b1339d4db0a9 100644
--- a/drivers/gpu/drm/panthor/panthor_drv.c
+++ b/drivers/gpu/drm/panthor/panthor_drv.c
@@ -1756,34 +1756,10 @@ static const struct file_operations panthor_drm_driver_fops = {
};
#ifdef CONFIG_DEBUG_FS
-static int panthor_gems_show(struct seq_file *m, void *data)
-{
- struct drm_info_node *node = m->private;
- struct drm_device *dev = node->minor->dev;
- struct panthor_device *ptdev = container_of(dev, struct panthor_device, base);
-
- panthor_gem_debugfs_print_bos(ptdev, m);
-
- return 0;
-}
-
-static struct drm_info_list panthor_debugfs_list[] = {
- {"gems", panthor_gems_show, 0, NULL},
-};
-
-static int panthor_gems_debugfs_init(struct drm_minor *minor)
-{
- drm_debugfs_create_files(panthor_debugfs_list,
- ARRAY_SIZE(panthor_debugfs_list),
- minor->debugfs_root, minor);
-
- return 0;
-}
-
static void panthor_debugfs_init(struct drm_minor *minor)
{
panthor_mmu_debugfs_init(minor);
- panthor_gems_debugfs_init(minor);
+ panthor_gem_debugfs_init(minor);
}
#endif
diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
index 6d14b0269574..c7b8b84a8f8b 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.c
+++ b/drivers/gpu/drm/panthor/panthor_gem.c
@@ -9,6 +9,8 @@
#include <linux/err.h>
#include <linux/slab.h>
+#include <drm/drm_debugfs.h>
+#include <drm/drm_file.h>
#include <drm/drm_print.h>
#include <drm/panthor_drm.h>
@@ -683,8 +685,8 @@ static void panthor_gem_debugfs_bo_print(struct panthor_gem_object *bo,
totals->reclaimable += resident_size;
}
-void panthor_gem_debugfs_print_bos(struct panthor_device *ptdev,
- struct seq_file *m)
+static void panthor_gem_debugfs_print_bos(struct panthor_device *ptdev,
+ struct seq_file *m)
{
struct gem_size_totals totals = {0};
struct panthor_gem_object *bo;
@@ -704,4 +706,27 @@ void panthor_gem_debugfs_print_bos(struct panthor_device *ptdev,
seq_printf(m, "Total size: %zd, Total resident: %zd, Total reclaimable: %zd\n",
totals.size, totals.resident, totals.reclaimable);
}
+
+static int panthor_gem_show_bos(struct seq_file *m, void *data)
+{
+ struct drm_info_node *node = m->private;
+ struct drm_device *dev = node->minor->dev;
+ struct panthor_device *ptdev =
+ container_of(dev, struct panthor_device, base);
+
+ panthor_gem_debugfs_print_bos(ptdev, m);
+
+ return 0;
+}
+
+static struct drm_info_list panthor_gem_debugfs_list[] = {
+ { "gems", panthor_gem_show_bos, 0, NULL },
+};
+
+void panthor_gem_debugfs_init(struct drm_minor *minor)
+{
+ drm_debugfs_create_files(panthor_gem_debugfs_list,
+ ARRAY_SIZE(panthor_gem_debugfs_list),
+ minor->debugfs_root, minor);
+}
#endif
diff --git a/drivers/gpu/drm/panthor/panthor_gem.h b/drivers/gpu/drm/panthor/panthor_gem.h
index 22519c570b5a..94b2d17cf032 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.h
+++ b/drivers/gpu/drm/panthor/panthor_gem.h
@@ -203,8 +203,7 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
void panthor_kernel_bo_destroy(struct panthor_kernel_bo *bo);
#ifdef CONFIG_DEBUG_FS
-void panthor_gem_debugfs_print_bos(struct panthor_device *pfdev,
- struct seq_file *m);
+void panthor_gem_debugfs_init(struct drm_minor *minor);
#endif
#endif /* __PANTHOR_GEM_H__ */
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 04/10] drm/panthor: Group panthor_kernel_bo_xxx() helpers
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (2 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 03/10] drm/panthor: Move panthor_gems_debugfs_init() to panthor_gem.c Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 05/10] drm/panthor: Don't call drm_gpuvm_bo_extobj_add() if the object is private Boris Brezillon
` (6 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
Move all panthor_kernel_bo_xxx() helpers at the end of the file, just
before the debugfs init logic. This will make further panthor_gem.c
refactoring more readable.
v2:
- Collect R-bs
v3:
- No changes
v4:
- No changes
v5:
- No changes
v6:
- No changes
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
drivers/gpu/drm/panthor/panthor_gem.c | 212 +++++++++++++-------------
1 file changed, 106 insertions(+), 106 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
index c7b8b84a8f8b..5065f99c9bc4 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.c
+++ b/drivers/gpu/drm/panthor/panthor_gem.c
@@ -132,112 +132,6 @@ static void panthor_gem_free_object(struct drm_gem_object *obj)
drm_gem_object_put(vm_root_gem);
}
-/**
- * panthor_kernel_bo_destroy() - Destroy a kernel buffer object
- * @bo: Kernel buffer object to destroy. If NULL or an ERR_PTR(), the destruction
- * is skipped.
- */
-void panthor_kernel_bo_destroy(struct panthor_kernel_bo *bo)
-{
- struct panthor_vm *vm;
-
- if (IS_ERR_OR_NULL(bo))
- return;
-
- vm = bo->vm;
- panthor_kernel_bo_vunmap(bo);
-
- drm_WARN_ON(bo->obj->dev,
- to_panthor_bo(bo->obj)->exclusive_vm_root_gem != panthor_vm_root_gem(vm));
- panthor_vm_unmap_range(vm, bo->va_node.start, bo->va_node.size);
- panthor_vm_free_va(vm, &bo->va_node);
- drm_gem_object_put(bo->obj);
- panthor_vm_put(vm);
- kfree(bo);
-}
-
-/**
- * panthor_kernel_bo_create() - Create and map a GEM object to a VM
- * @ptdev: Device.
- * @vm: VM to map the GEM to. If NULL, the kernel object is not GPU mapped.
- * @size: Size of the buffer object.
- * @bo_flags: Combination of drm_panthor_bo_flags flags.
- * @vm_map_flags: Combination of drm_panthor_vm_bind_op_flags (only those
- * that are related to map operations).
- * @gpu_va: GPU address assigned when mapping to the VM.
- * If gpu_va == PANTHOR_VM_KERNEL_AUTO_VA, the virtual address will be
- * automatically allocated.
- * @name: Descriptive label of the BO's contents
- *
- * Return: A valid pointer in case of success, an ERR_PTR() otherwise.
- */
-struct panthor_kernel_bo *
-panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
- size_t size, u32 bo_flags, u32 vm_map_flags,
- u64 gpu_va, const char *name)
-{
- struct drm_gem_shmem_object *obj;
- struct panthor_kernel_bo *kbo;
- struct panthor_gem_object *bo;
- u32 debug_flags = PANTHOR_DEBUGFS_GEM_USAGE_FLAG_KERNEL;
- int ret;
-
- if (drm_WARN_ON(&ptdev->base, !vm))
- return ERR_PTR(-EINVAL);
-
- kbo = kzalloc_obj(*kbo);
- if (!kbo)
- return ERR_PTR(-ENOMEM);
-
- obj = drm_gem_shmem_create(&ptdev->base, size);
- if (IS_ERR(obj)) {
- ret = PTR_ERR(obj);
- goto err_free_bo;
- }
-
- bo = to_panthor_bo(&obj->base);
- kbo->obj = &obj->base;
- bo->flags = bo_flags;
- bo->base.map_wc = should_map_wc(bo, vm);
- bo->exclusive_vm_root_gem = panthor_vm_root_gem(vm);
- drm_gem_object_get(bo->exclusive_vm_root_gem);
- bo->base.base.resv = bo->exclusive_vm_root_gem->resv;
-
- if (vm == panthor_fw_vm(ptdev))
- debug_flags |= PANTHOR_DEBUGFS_GEM_USAGE_FLAG_FW_MAPPED;
-
- panthor_gem_kernel_bo_set_label(kbo, name);
- panthor_gem_debugfs_set_usage_flags(to_panthor_bo(kbo->obj), debug_flags);
-
- /* The system and GPU MMU page size might differ, which becomes a
- * problem for FW sections that need to be mapped at explicit address
- * since our PAGE_SIZE alignment might cover a VA range that's
- * expected to be used for another section.
- * Make sure we never map more than we need.
- */
- size = ALIGN(size, panthor_vm_page_size(vm));
- ret = panthor_vm_alloc_va(vm, gpu_va, size, &kbo->va_node);
- if (ret)
- goto err_put_obj;
-
- ret = panthor_vm_map_bo_range(vm, bo, 0, size, kbo->va_node.start, vm_map_flags);
- if (ret)
- goto err_free_va;
-
- kbo->vm = panthor_vm_get(vm);
- return kbo;
-
-err_free_va:
- panthor_vm_free_va(vm, &kbo->va_node);
-
-err_put_obj:
- drm_gem_object_put(&obj->base);
-
-err_free_bo:
- kfree(kbo);
- return ERR_PTR(ret);
-}
-
static struct sg_table *
panthor_gem_prime_map_dma_buf(struct dma_buf_attachment *attach,
enum dma_data_direction dir)
@@ -603,6 +497,112 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
return 0;
}
+/**
+ * panthor_kernel_bo_destroy() - Destroy a kernel buffer object
+ * @bo: Kernel buffer object to destroy. If NULL or an ERR_PTR(), the destruction
+ * is skipped.
+ */
+void panthor_kernel_bo_destroy(struct panthor_kernel_bo *bo)
+{
+ struct panthor_vm *vm;
+
+ if (IS_ERR_OR_NULL(bo))
+ return;
+
+ vm = bo->vm;
+ panthor_kernel_bo_vunmap(bo);
+
+ drm_WARN_ON(bo->obj->dev,
+ to_panthor_bo(bo->obj)->exclusive_vm_root_gem != panthor_vm_root_gem(vm));
+ panthor_vm_unmap_range(vm, bo->va_node.start, bo->va_node.size);
+ panthor_vm_free_va(vm, &bo->va_node);
+ drm_gem_object_put(bo->obj);
+ panthor_vm_put(vm);
+ kfree(bo);
+}
+
+/**
+ * panthor_kernel_bo_create() - Create and map a GEM object to a VM
+ * @ptdev: Device.
+ * @vm: VM to map the GEM to. If NULL, the kernel object is not GPU mapped.
+ * @size: Size of the buffer object.
+ * @bo_flags: Combination of drm_panthor_bo_flags flags.
+ * @vm_map_flags: Combination of drm_panthor_vm_bind_op_flags (only those
+ * that are related to map operations).
+ * @gpu_va: GPU address assigned when mapping to the VM.
+ * If gpu_va == PANTHOR_VM_KERNEL_AUTO_VA, the virtual address will be
+ * automatically allocated.
+ * @name: Descriptive label of the BO's contents
+ *
+ * Return: A valid pointer in case of success, an ERR_PTR() otherwise.
+ */
+struct panthor_kernel_bo *
+panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
+ size_t size, u32 bo_flags, u32 vm_map_flags,
+ u64 gpu_va, const char *name)
+{
+ struct drm_gem_shmem_object *obj;
+ struct panthor_kernel_bo *kbo;
+ struct panthor_gem_object *bo;
+ u32 debug_flags = PANTHOR_DEBUGFS_GEM_USAGE_FLAG_KERNEL;
+ int ret;
+
+ if (drm_WARN_ON(&ptdev->base, !vm))
+ return ERR_PTR(-EINVAL);
+
+ kbo = kzalloc_obj(*kbo);
+ if (!kbo)
+ return ERR_PTR(-ENOMEM);
+
+ obj = drm_gem_shmem_create(&ptdev->base, size);
+ if (IS_ERR(obj)) {
+ ret = PTR_ERR(obj);
+ goto err_free_bo;
+ }
+
+ bo = to_panthor_bo(&obj->base);
+ kbo->obj = &obj->base;
+ bo->flags = bo_flags;
+ bo->base.map_wc = should_map_wc(bo, vm);
+ bo->exclusive_vm_root_gem = panthor_vm_root_gem(vm);
+ drm_gem_object_get(bo->exclusive_vm_root_gem);
+ bo->base.base.resv = bo->exclusive_vm_root_gem->resv;
+
+ if (vm == panthor_fw_vm(ptdev))
+ debug_flags |= PANTHOR_DEBUGFS_GEM_USAGE_FLAG_FW_MAPPED;
+
+ panthor_gem_kernel_bo_set_label(kbo, name);
+ panthor_gem_debugfs_set_usage_flags(to_panthor_bo(kbo->obj), debug_flags);
+
+ /* The system and GPU MMU page size might differ, which becomes a
+ * problem for FW sections that need to be mapped at explicit address
+ * since our PAGE_SIZE alignment might cover a VA range that's
+ * expected to be used for another section.
+ * Make sure we never map more than we need.
+ */
+ size = ALIGN(size, panthor_vm_page_size(vm));
+ ret = panthor_vm_alloc_va(vm, gpu_va, size, &kbo->va_node);
+ if (ret)
+ goto err_put_obj;
+
+ ret = panthor_vm_map_bo_range(vm, bo, 0, size, kbo->va_node.start, vm_map_flags);
+ if (ret)
+ goto err_free_va;
+
+ kbo->vm = panthor_vm_get(vm);
+ return kbo;
+
+err_free_va:
+ panthor_vm_free_va(vm, &kbo->va_node);
+
+err_put_obj:
+ drm_gem_object_put(&obj->base);
+
+err_free_bo:
+ kfree(kbo);
+ return ERR_PTR(ret);
+}
+
#ifdef CONFIG_DEBUG_FS
struct gem_size_totals {
size_t size;
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 05/10] drm/panthor: Don't call drm_gpuvm_bo_extobj_add() if the object is private
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (3 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 04/10] drm/panthor: Group panthor_kernel_bo_xxx() helpers Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 06/10] drm/panthor: Part ways with drm_gem_shmem_object Boris Brezillon
` (5 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
drm_gpuvm_bo_extobj_add() is a NOP if the object is private, but it
forces us to take/release the VM resv lock, so let's do that only when
we know the object can be shared.
v3:
- New commit
v4:
- Collect R-bs
v5:
- No changes
v6:
- No changes
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Steven Price <steven.price@arm.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
---
drivers/gpu/drm/panthor/panthor_mmu.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
index ab67286e3a6e..ac11c8c6a863 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.c
+++ b/drivers/gpu/drm/panthor/panthor_mmu.c
@@ -1278,9 +1278,11 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
}
/* Insert BO into the extobj list last, when we know nothing can fail. */
- dma_resv_lock(panthor_vm_resv(vm), NULL);
- drm_gpuvm_bo_extobj_add(op_ctx->map.vm_bo);
- dma_resv_unlock(panthor_vm_resv(vm));
+ if (bo->base.base.resv != panthor_vm_resv(vm)) {
+ dma_resv_lock(panthor_vm_resv(vm), NULL);
+ drm_gpuvm_bo_extobj_add(op_ctx->map.vm_bo);
+ dma_resv_unlock(panthor_vm_resv(vm));
+ }
return 0;
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 06/10] drm/panthor: Part ways with drm_gem_shmem_object
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (4 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 05/10] drm/panthor: Don't call drm_gpuvm_bo_extobj_add() if the object is private Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 07/10] drm/panthor: Lazily allocate pages on mmap() Boris Brezillon
` (4 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
While drm_gem_shmem_object does most of the job we need it to do, the
way sub-resources (pages, sgt, vmap) are handled and their lifetimes
gets in the way of BO reclaim. There has been attempts to address
that [1], but in the meantime, new gem_shmem users were introduced
(accel drivers), and some of them manually free some of these resources.
This makes things harder to control/sanitize/validate.
Thomas Zimmerman is not a huge fan of enforcing lifetimes of sub-resources
and forcing gem_shmem users to go through new gem_shmem helpers when they
need manual control of some sort, and I believe this is a dead end if
we don't force users to follow some stricter rules through carefully
designed helpers, because there will always be one user doing crazy things
with gem_shmem_object internals, which ends up tripping out the common
helpers when they are called.
The consensus we reached was that we would be better off forking
gem_shmem in panthor. So here we are, parting ways with gem_shmem. The
current transition tries to minimize the changes, but there are still
some aspects that are different, the main one being that we no longer
have a pages_use_count, and pages stays around until the GEM object is
destroyed (or when evicted once we've added a shrinker). The sgt also
no longer retains pages. This is losely based on how msm does things by
the way.
If there's any interest in sharing code (probably with msm, since the
panthor shrinker is going to be losely based on the msm implementation),
we can always change gears and do that once we have everything
working/merged.
[1]https://patchwork.kernel.org/project/dri-devel/patch/20240105184624.508603-1-dmitry.osipenko@collabora.com/
v2:
- Fix refcounting
- Add a _locked suffix to a bunch of functions expecting the resv lock
to be held
- Take the lock before releasing resources in panthor_gem_free_object()
v3:
- Use ERR_CAST() to fix an ERR-ptr deref
- Add missing resv_[un]lock() around a panthor_gem_backing_unpin_locked()
call
v4:
- Fix an error path in panthor_gem_vmap_get_locked()
- Don't leave bo->base.pages with an ERR_PTR()
- Make panthor_gem_{pin,unpin}[_locked]() more consistent
- Don't fail in panthor_gem_dev_map_get_sgt_locked() if the pages are not
allocated
v5:
- Add missing static specifier on our vm_ops
v6:
- Fix huge_fault handling
- s/uint32_t/u32/
- s/drm_dbg_kms/drm_dbg_driver/
- Collect R-bs
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Steven Price <steven.price@arm.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
---
drivers/gpu/drm/panthor/Kconfig | 1 -
drivers/gpu/drm/panthor/panthor_drv.c | 7 +-
drivers/gpu/drm/panthor/panthor_fw.c | 16 +-
drivers/gpu/drm/panthor/panthor_gem.c | 706 ++++++++++++++++++++----
drivers/gpu/drm/panthor/panthor_gem.h | 62 ++-
drivers/gpu/drm/panthor/panthor_mmu.c | 48 +-
drivers/gpu/drm/panthor/panthor_sched.c | 9 +-
7 files changed, 675 insertions(+), 174 deletions(-)
diff --git a/drivers/gpu/drm/panthor/Kconfig b/drivers/gpu/drm/panthor/Kconfig
index 55b40ad07f3b..911e7f4810c3 100644
--- a/drivers/gpu/drm/panthor/Kconfig
+++ b/drivers/gpu/drm/panthor/Kconfig
@@ -8,7 +8,6 @@ config DRM_PANTHOR
depends on MMU
select DEVFREQ_GOV_SIMPLE_ONDEMAND
select DRM_EXEC
- select DRM_GEM_SHMEM_HELPER
select DRM_GPUVM
select DRM_SCHED
select IOMMU_IO_PGTABLE_LPAE
diff --git a/drivers/gpu/drm/panthor/panthor_drv.c b/drivers/gpu/drm/panthor/panthor_drv.c
index b1339d4db0a9..73fc983dc9b4 100644
--- a/drivers/gpu/drm/panthor/panthor_drv.c
+++ b/drivers/gpu/drm/panthor/panthor_drv.c
@@ -21,6 +21,7 @@
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
#include <drm/drm_exec.h>
+#include <drm/drm_file.h>
#include <drm/drm_ioctl.h>
#include <drm/drm_print.h>
#include <drm/drm_syncobj.h>
@@ -1578,7 +1579,7 @@ static int panthor_ioctl_bo_query_info(struct drm_device *ddev, void *data,
args->create_flags = bo->flags;
args->extra_flags = 0;
- if (drm_gem_is_imported(&bo->base.base))
+ if (drm_gem_is_imported(&bo->base))
args->extra_flags |= DRM_PANTHOR_BO_IS_IMPORTED;
drm_gem_object_put(obj);
@@ -1793,8 +1794,7 @@ static const struct drm_driver panthor_drm_driver = {
.major = 1,
.minor = 8,
- .gem_create_object = panthor_gem_create_object,
- .gem_prime_import_sg_table = drm_gem_shmem_prime_import_sg_table,
+ .gem_prime_import_sg_table = panthor_gem_prime_import_sg_table,
.gem_prime_import = panthor_gem_prime_import,
#ifdef CONFIG_DEBUG_FS
.debugfs_init = panthor_debugfs_init,
@@ -1944,3 +1944,4 @@ module_exit(panthor_exit);
MODULE_AUTHOR("Panthor Project Developers");
MODULE_DESCRIPTION("Panthor DRM Driver");
MODULE_LICENSE("Dual MIT/GPL");
+MODULE_IMPORT_NS("DMA_BUF");
diff --git a/drivers/gpu/drm/panthor/panthor_fw.c b/drivers/gpu/drm/panthor/panthor_fw.c
index 8886002e1d31..be0da5b1f3ab 100644
--- a/drivers/gpu/drm/panthor/panthor_fw.c
+++ b/drivers/gpu/drm/panthor/panthor_fw.c
@@ -628,7 +628,6 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev,
u32 cache_mode = hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_CACHE_MODE_MASK;
struct panthor_gem_object *bo;
u32 vm_map_flags = 0;
- struct sg_table *sgt;
u64 va = hdr.va.start;
if (!(hdr.flags & CSF_FW_BINARY_IFACE_ENTRY_WR))
@@ -666,11 +665,12 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev,
panthor_fw_init_section_mem(ptdev, section);
bo = to_panthor_bo(section->mem->obj);
- sgt = drm_gem_shmem_get_pages_sgt(&bo->base);
- if (IS_ERR(sgt))
- return PTR_ERR(sgt);
- dma_sync_sgtable_for_device(ptdev->base.dev, sgt, DMA_TO_DEVICE);
+ /* An sgt should have been requested when the kernel BO was GPU-mapped. */
+ if (drm_WARN_ON_ONCE(&ptdev->base, !bo->dmap.sgt))
+ return -EINVAL;
+
+ dma_sync_sgtable_for_device(ptdev->base.dev, bo->dmap.sgt, DMA_TO_DEVICE);
}
if (hdr.va.start == CSF_MCU_SHARED_REGION_START)
@@ -730,8 +730,10 @@ panthor_reload_fw_sections(struct panthor_device *ptdev, bool full_reload)
continue;
panthor_fw_init_section_mem(ptdev, section);
- sgt = drm_gem_shmem_get_pages_sgt(&to_panthor_bo(section->mem->obj)->base);
- if (!drm_WARN_ON(&ptdev->base, IS_ERR_OR_NULL(sgt)))
+
+ /* An sgt should have been requested when the kernel BO was GPU-mapped. */
+ sgt = to_panthor_bo(section->mem->obj)->dmap.sgt;
+ if (!drm_WARN_ON_ONCE(&ptdev->base, !sgt))
dma_sync_sgtable_for_device(ptdev->base.dev, sgt, DMA_TO_DEVICE);
}
}
diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
index 5065f99c9bc4..0723a0dc0127 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.c
+++ b/drivers/gpu/drm/panthor/panthor_gem.c
@@ -8,9 +8,11 @@
#include <linux/dma-mapping.h>
#include <linux/err.h>
#include <linux/slab.h>
+#include <linux/vmalloc.h>
#include <drm/drm_debugfs.h>
#include <drm/drm_file.h>
+#include <drm/drm_prime.h>
#include <drm/drm_print.h>
#include <drm/panthor_drm.h>
@@ -44,7 +46,7 @@ static void panthor_gem_debugfs_bo_init(struct panthor_gem_object *bo)
static void panthor_gem_debugfs_bo_add(struct panthor_gem_object *bo)
{
- struct panthor_device *ptdev = container_of(bo->base.base.dev,
+ struct panthor_device *ptdev = container_of(bo->base.dev,
struct panthor_device, base);
bo->debugfs.creator.tgid = current->tgid;
@@ -57,7 +59,7 @@ static void panthor_gem_debugfs_bo_add(struct panthor_gem_object *bo)
static void panthor_gem_debugfs_bo_rm(struct panthor_gem_object *bo)
{
- struct panthor_device *ptdev = container_of(bo->base.base.dev,
+ struct panthor_device *ptdev = container_of(bo->base.dev,
struct panthor_device, base);
if (list_empty(&bo->debugfs.node))
@@ -80,9 +82,9 @@ static void panthor_gem_debugfs_bo_init(struct panthor_gem_object *bo) {}
#endif
static bool
-should_map_wc(struct panthor_gem_object *bo, struct panthor_vm *exclusive_vm)
+should_map_wc(struct panthor_gem_object *bo)
{
- struct panthor_device *ptdev = container_of(bo->base.base.dev, struct panthor_device, base);
+ struct panthor_device *ptdev = container_of(bo->base.dev, struct panthor_device, base);
/* We can't do uncached mappings if the device is coherent,
* because the zeroing done by the shmem layer at page allocation
@@ -112,6 +114,210 @@ should_map_wc(struct panthor_gem_object *bo, struct panthor_vm *exclusive_vm)
return true;
}
+static void
+panthor_gem_backing_cleanup_locked(struct panthor_gem_object *bo)
+{
+ dma_resv_assert_held(bo->base.resv);
+
+ if (!bo->backing.pages)
+ return;
+
+ drm_gem_put_pages(&bo->base, bo->backing.pages, true, false);
+ bo->backing.pages = NULL;
+}
+
+static int
+panthor_gem_backing_get_pages_locked(struct panthor_gem_object *bo)
+{
+ struct page **pages;
+
+ dma_resv_assert_held(bo->base.resv);
+
+ if (bo->backing.pages)
+ return 0;
+
+ pages = drm_gem_get_pages(&bo->base);
+ if (IS_ERR(pages)) {
+ drm_dbg_driver(bo->base.dev, "Failed to get pages (%pe)\n", pages);
+ return PTR_ERR(pages);
+ }
+
+ bo->backing.pages = pages;
+ return 0;
+}
+
+static int panthor_gem_backing_pin_locked(struct panthor_gem_object *bo)
+{
+ int ret;
+
+ dma_resv_assert_held(bo->base.resv);
+ drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base));
+
+ if (refcount_inc_not_zero(&bo->backing.pin_count))
+ return 0;
+
+ ret = panthor_gem_backing_get_pages_locked(bo);
+ if (!ret)
+ refcount_set(&bo->backing.pin_count, 1);
+
+ return ret;
+}
+
+static void panthor_gem_backing_unpin_locked(struct panthor_gem_object *bo)
+{
+ dma_resv_assert_held(bo->base.resv);
+ drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base));
+
+ if (refcount_dec_and_test(&bo->backing.pin_count)) {
+ /* We don't release anything when pin_count drops to zero.
+ * Pages stay there until an explicit cleanup is requested.
+ */
+ }
+}
+
+static void
+panthor_gem_dev_map_cleanup_locked(struct panthor_gem_object *bo)
+{
+ dma_resv_assert_held(bo->base.resv);
+
+ if (!bo->dmap.sgt)
+ return;
+
+ dma_unmap_sgtable(drm_dev_dma_dev(bo->base.dev), bo->dmap.sgt, DMA_BIDIRECTIONAL, 0);
+ sg_free_table(bo->dmap.sgt);
+ kfree(bo->dmap.sgt);
+ bo->dmap.sgt = NULL;
+}
+
+static struct sg_table *
+panthor_gem_dev_map_get_sgt_locked(struct panthor_gem_object *bo)
+{
+ struct sg_table *sgt;
+ int ret;
+
+ dma_resv_assert_held(bo->base.resv);
+
+ if (bo->dmap.sgt)
+ return bo->dmap.sgt;
+
+ /* Pages stay around after they've been allocated. At least that stands
+ * until we add a shrinker.
+ */
+ ret = panthor_gem_backing_get_pages_locked(bo);
+ if (ret)
+ return ERR_PTR(ret);
+
+ sgt = drm_prime_pages_to_sg(bo->base.dev, bo->backing.pages,
+ bo->base.size >> PAGE_SHIFT);
+ if (IS_ERR(sgt))
+ return sgt;
+
+ /* Map the pages for use by the h/w. */
+ ret = dma_map_sgtable(drm_dev_dma_dev(bo->base.dev), sgt, DMA_BIDIRECTIONAL, 0);
+ if (ret)
+ goto err_free_sgt;
+
+ bo->dmap.sgt = sgt;
+ return sgt;
+
+err_free_sgt:
+ sg_free_table(sgt);
+ kfree(sgt);
+ return ERR_PTR(ret);
+}
+
+struct sg_table *
+panthor_gem_get_dev_sgt(struct panthor_gem_object *bo)
+{
+ struct sg_table *sgt;
+
+ dma_resv_lock(bo->base.resv, NULL);
+ sgt = panthor_gem_dev_map_get_sgt_locked(bo);
+ dma_resv_unlock(bo->base.resv);
+
+ return sgt;
+}
+
+static void
+panthor_gem_vmap_cleanup_locked(struct panthor_gem_object *bo)
+{
+ if (!bo->cmap.vaddr)
+ return;
+
+ vunmap(bo->cmap.vaddr);
+ bo->cmap.vaddr = NULL;
+ panthor_gem_backing_unpin_locked(bo);
+}
+
+static int
+panthor_gem_prep_for_cpu_map_locked(struct panthor_gem_object *bo)
+{
+ if (should_map_wc(bo)) {
+ struct sg_table *sgt;
+
+ sgt = panthor_gem_dev_map_get_sgt_locked(bo);
+ if (IS_ERR(sgt))
+ return PTR_ERR(sgt);
+ }
+
+ return 0;
+}
+
+static void *
+panthor_gem_vmap_get_locked(struct panthor_gem_object *bo)
+{
+ pgprot_t prot = PAGE_KERNEL;
+ void *vaddr;
+ int ret;
+
+ dma_resv_assert_held(bo->base.resv);
+
+ if (drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base)))
+ return ERR_PTR(-EINVAL);
+
+ if (refcount_inc_not_zero(&bo->cmap.vaddr_use_count)) {
+ drm_WARN_ON_ONCE(bo->base.dev, !bo->cmap.vaddr);
+ return bo->cmap.vaddr;
+ }
+
+ ret = panthor_gem_backing_pin_locked(bo);
+ if (ret)
+ return ERR_PTR(ret);
+
+ ret = panthor_gem_prep_for_cpu_map_locked(bo);
+ if (ret)
+ goto err_unpin;
+
+ if (should_map_wc(bo))
+ prot = pgprot_writecombine(prot);
+
+ vaddr = vmap(bo->backing.pages, bo->base.size >> PAGE_SHIFT, VM_MAP, prot);
+ if (!vaddr) {
+ ret = -ENOMEM;
+ goto err_unpin;
+ }
+
+ bo->cmap.vaddr = vaddr;
+ refcount_set(&bo->cmap.vaddr_use_count, 1);
+ return vaddr;
+
+err_unpin:
+ panthor_gem_backing_unpin_locked(bo);
+ return ERR_PTR(ret);
+}
+
+static void
+panthor_gem_vmap_put_locked(struct panthor_gem_object *bo)
+{
+ dma_resv_assert_held(bo->base.resv);
+
+ if (drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base)))
+ return;
+
+ if (refcount_dec_and_test(&bo->cmap.vaddr_use_count))
+ panthor_gem_vmap_cleanup_locked(bo);
+}
+
static void panthor_gem_free_object(struct drm_gem_object *obj)
{
struct panthor_gem_object *bo = to_panthor_bo(obj);
@@ -127,8 +333,19 @@ static void panthor_gem_free_object(struct drm_gem_object *obj)
mutex_destroy(&bo->label.lock);
- drm_gem_free_mmap_offset(&bo->base.base);
- drm_gem_shmem_free(&bo->base);
+ if (drm_gem_is_imported(obj)) {
+ drm_prime_gem_destroy(obj, bo->dmap.sgt);
+ } else {
+ dma_resv_lock(obj->resv, NULL);
+ panthor_gem_vmap_cleanup_locked(bo);
+ panthor_gem_dev_map_cleanup_locked(bo);
+ panthor_gem_backing_cleanup_locked(bo);
+ dma_resv_unlock(obj->resv);
+ }
+
+ drm_gem_object_release(obj);
+
+ kfree(bo);
drm_gem_object_put(vm_root_gem);
}
@@ -159,15 +376,15 @@ panthor_gem_prime_begin_cpu_access(struct dma_buf *dma_buf,
{
struct drm_gem_object *obj = dma_buf->priv;
struct drm_device *dev = obj->dev;
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
struct dma_buf_attachment *attach;
dma_resv_lock(obj->resv, NULL);
- if (shmem->sgt)
- dma_sync_sgtable_for_cpu(dev->dev, shmem->sgt, dir);
+ if (bo->dmap.sgt)
+ dma_sync_sgtable_for_cpu(drm_dev_dma_dev(dev), bo->dmap.sgt, dir);
- if (shmem->vaddr)
- invalidate_kernel_vmap_range(shmem->vaddr, shmem->base.size);
+ if (bo->cmap.vaddr)
+ invalidate_kernel_vmap_range(bo->cmap.vaddr, bo->base.size);
list_for_each_entry(attach, &dma_buf->attachments, node) {
struct sg_table *sgt = attach->priv;
@@ -186,7 +403,7 @@ panthor_gem_prime_end_cpu_access(struct dma_buf *dma_buf,
{
struct drm_gem_object *obj = dma_buf->priv;
struct drm_device *dev = obj->dev;
- struct drm_gem_shmem_object *shmem = to_drm_gem_shmem_obj(obj);
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
struct dma_buf_attachment *attach;
dma_resv_lock(obj->resv, NULL);
@@ -197,11 +414,11 @@ panthor_gem_prime_end_cpu_access(struct dma_buf *dma_buf,
dma_sync_sgtable_for_device(attach->dev, sgt, dir);
}
- if (shmem->vaddr)
- flush_kernel_vmap_range(shmem->vaddr, shmem->base.size);
+ if (bo->cmap.vaddr)
+ flush_kernel_vmap_range(bo->cmap.vaddr, bo->base.size);
- if (shmem->sgt)
- dma_sync_sgtable_for_device(dev->dev, shmem->sgt, dir);
+ if (bo->dmap.sgt)
+ dma_sync_sgtable_for_device(drm_dev_dma_dev(dev), bo->dmap.sgt, dir);
dma_resv_unlock(obj->resv);
return 0;
@@ -258,53 +475,345 @@ panthor_gem_prime_import(struct drm_device *dev,
return drm_gem_prime_import(dev, dma_buf);
}
+static void panthor_gem_print_info(struct drm_printer *p, unsigned int indent,
+ const struct drm_gem_object *obj)
+{
+ const struct panthor_gem_object *bo = to_panthor_bo(obj);
+
+ if (drm_gem_is_imported(&bo->base))
+ return;
+
+ drm_printf_indent(p, indent, "resident=%s\n", str_true_false(bo->backing.pages));
+ drm_printf_indent(p, indent, "pages_pin_count=%u\n", refcount_read(&bo->backing.pin_count));
+ drm_printf_indent(p, indent, "vmap_use_count=%u\n",
+ refcount_read(&bo->cmap.vaddr_use_count));
+ drm_printf_indent(p, indent, "vaddr=%p\n", bo->cmap.vaddr);
+}
+
+static int panthor_gem_pin_locked(struct drm_gem_object *obj)
+{
+ if (!drm_gem_is_imported(obj))
+ return panthor_gem_backing_pin_locked(to_panthor_bo(obj));
+
+ return 0;
+}
+
+static void panthor_gem_unpin_locked(struct drm_gem_object *obj)
+{
+ if (!drm_gem_is_imported(obj))
+ panthor_gem_backing_unpin_locked(to_panthor_bo(obj));
+}
+
+int panthor_gem_pin(struct panthor_gem_object *bo)
+{
+ int ret = 0;
+
+ if (drm_gem_is_imported(&bo->base))
+ return 0;
+
+ if (refcount_inc_not_zero(&bo->backing.pin_count))
+ return 0;
+
+ dma_resv_lock(bo->base.resv, NULL);
+ ret = panthor_gem_backing_pin_locked(bo);
+ dma_resv_unlock(bo->base.resv);
+
+ return ret;
+}
+
+void panthor_gem_unpin(struct panthor_gem_object *bo)
+{
+ if (drm_gem_is_imported(&bo->base))
+ return;
+
+ if (refcount_dec_not_one(&bo->backing.pin_count))
+ return;
+
+ dma_resv_lock(bo->base.resv, NULL);
+ panthor_gem_backing_unpin_locked(bo);
+ dma_resv_unlock(bo->base.resv);
+}
+
+static struct sg_table *panthor_gem_get_sg_table(struct drm_gem_object *obj)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
+
+ drm_WARN_ON_ONCE(obj->dev, drm_gem_is_imported(obj));
+ drm_WARN_ON_ONCE(obj->dev, !bo->backing.pages);
+ drm_WARN_ON_ONCE(obj->dev, !refcount_read(&bo->backing.pin_count));
+
+ return drm_prime_pages_to_sg(obj->dev, bo->backing.pages, obj->size >> PAGE_SHIFT);
+}
+
+static int panthor_gem_vmap_locked(struct drm_gem_object *obj,
+ struct iosys_map *map)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
+ void *vaddr;
+
+ dma_resv_assert_held(obj->resv);
+
+ if (drm_gem_is_imported(obj))
+ return dma_buf_vmap(obj->import_attach->dmabuf, map);
+
+ vaddr = panthor_gem_vmap_get_locked(bo);
+ if (IS_ERR(vaddr))
+ return PTR_ERR(vaddr);
+
+ iosys_map_set_vaddr(map, vaddr);
+ return 0;
+}
+
+static void panthor_gem_vunmap_locked(struct drm_gem_object *obj,
+ struct iosys_map *map)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
+
+ dma_resv_assert_held(obj->resv);
+
+ if (drm_gem_is_imported(obj)) {
+ dma_buf_vunmap(obj->import_attach->dmabuf, map);
+ } else {
+ drm_WARN_ON_ONCE(obj->dev, bo->cmap.vaddr != map->vaddr);
+ panthor_gem_vmap_put_locked(bo);
+ }
+}
+
+static int panthor_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
+ int ret;
+
+ if (drm_gem_is_imported(obj)) {
+ /* Reset both vm_ops and vm_private_data, so we don't end up with
+ * vm_ops pointing to our implementation if the dma-buf backend
+ * doesn't set those fields.
+ */
+ vma->vm_private_data = NULL;
+ vma->vm_ops = NULL;
+
+ ret = dma_buf_mmap(obj->dma_buf, vma, 0);
+
+ /* Drop the reference drm_gem_mmap_obj() acquired.*/
+ if (!ret)
+ drm_gem_object_put(obj);
+
+ return ret;
+ }
+
+ if (is_cow_mapping(vma->vm_flags))
+ return -EINVAL;
+
+ dma_resv_lock(obj->resv, NULL);
+ ret = panthor_gem_backing_get_pages_locked(bo);
+ if (!ret)
+ ret = panthor_gem_prep_for_cpu_map_locked(bo);
+ dma_resv_unlock(obj->resv);
+
+ if (ret)
+ return ret;
+
+ vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
+ vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
+ if (should_map_wc(bo))
+ vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+
+ return 0;
+}
+
static enum drm_gem_object_status panthor_gem_status(struct drm_gem_object *obj)
{
struct panthor_gem_object *bo = to_panthor_bo(obj);
enum drm_gem_object_status res = 0;
- if (drm_gem_is_imported(&bo->base.base) || bo->base.pages)
+ if (drm_gem_is_imported(&bo->base) || bo->backing.pages)
res |= DRM_GEM_OBJECT_RESIDENT;
return res;
}
-static const struct drm_gem_object_funcs panthor_gem_funcs = {
- .free = panthor_gem_free_object,
- .print_info = drm_gem_shmem_object_print_info,
- .pin = drm_gem_shmem_object_pin,
- .unpin = drm_gem_shmem_object_unpin,
- .get_sg_table = drm_gem_shmem_object_get_sg_table,
- .vmap = drm_gem_shmem_object_vmap,
- .vunmap = drm_gem_shmem_object_vunmap,
- .mmap = drm_gem_shmem_object_mmap,
- .status = panthor_gem_status,
- .export = panthor_gem_prime_export,
- .vm_ops = &drm_gem_shmem_vm_ops,
+static vm_fault_t insert_page(struct vm_fault *vmf, unsigned int order, struct page *page)
+{
+ if (!order) {
+ return vmf_insert_pfn(vmf->vma, vmf->address, page_to_pfn(page));
+#ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP
+ } else if (order == PMD_ORDER) {
+ unsigned long pfn = page_to_pfn(page);
+ unsigned long paddr = pfn << PAGE_SHIFT;
+ bool aligned = (vmf->address & ~PMD_MASK) == (paddr & ~PMD_MASK);
+
+ if (aligned &&
+ folio_test_pmd_mappable(page_folio(page))) {
+ pfn &= PMD_MASK >> PAGE_SHIFT;
+ return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE);
+ }
+#endif
+ }
+
+ return VM_FAULT_FALLBACK;
+}
+
+static vm_fault_t panthor_gem_any_fault(struct vm_fault *vmf, unsigned int order)
+{
+ struct vm_area_struct *vma = vmf->vma;
+ struct drm_gem_object *obj = vma->vm_private_data;
+ struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
+ loff_t num_pages = obj->size >> PAGE_SHIFT;
+ vm_fault_t ret;
+ pgoff_t page_offset;
+
+ if (order && order != PMD_ORDER)
+ return VM_FAULT_FALLBACK;
+
+ /* Offset to faulty address in the VMA. */
+ page_offset = vmf->pgoff - vma->vm_pgoff;
+
+ dma_resv_lock(bo->base.resv, NULL);
+
+ if (page_offset >= num_pages ||
+ drm_WARN_ON_ONCE(obj->dev, !bo->backing.pages)) {
+ ret = VM_FAULT_SIGBUS;
+ goto out;
+ }
+
+ ret = insert_page(vmf, order, bo->backing.pages[page_offset]);
+
+ out:
+ dma_resv_unlock(bo->base.resv);
+
+ return ret;
+}
+
+static vm_fault_t panthor_gem_fault(struct vm_fault *vmf)
+{
+ return panthor_gem_any_fault(vmf, 0);
+}
+
+static void panthor_gem_vm_open(struct vm_area_struct *vma)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
+
+ drm_WARN_ON(bo->base.dev, drm_gem_is_imported(&bo->base));
+
+ dma_resv_lock(bo->base.resv, NULL);
+
+ /* We should have already pinned the pages when the buffer was first
+ * mmap'd, vm_open() just grabs an additional reference for the new
+ * mm the vma is getting copied into (ie. on fork()).
+ */
+ drm_WARN_ON_ONCE(bo->base.dev, !bo->backing.pages);
+
+ dma_resv_unlock(bo->base.resv);
+
+ drm_gem_vm_open(vma);
+}
+
+static const struct vm_operations_struct panthor_gem_vm_ops = {
+ .fault = panthor_gem_fault,
+#ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP
+ .huge_fault = panthor_gem_any_fault,
+#endif
+ .open = panthor_gem_vm_open,
+ .close = drm_gem_vm_close,
};
-/**
- * panthor_gem_create_object - Implementation of driver->gem_create_object.
- * @ddev: DRM device
- * @size: Size in bytes of the memory the object will reference
- *
- * This lets the GEM helpers allocate object structs for us, and keep
- * our BO stats correct.
- */
-struct drm_gem_object *panthor_gem_create_object(struct drm_device *ddev, size_t size)
-{
- struct panthor_gem_object *obj;
+static const struct drm_gem_object_funcs panthor_gem_funcs = {
+ .free = panthor_gem_free_object,
+ .print_info = panthor_gem_print_info,
+ .pin = panthor_gem_pin_locked,
+ .unpin = panthor_gem_unpin_locked,
+ .get_sg_table = panthor_gem_get_sg_table,
+ .vmap = panthor_gem_vmap_locked,
+ .vunmap = panthor_gem_vunmap_locked,
+ .mmap = panthor_gem_mmap,
+ .status = panthor_gem_status,
+ .export = panthor_gem_prime_export,
+ .vm_ops = &panthor_gem_vm_ops,
+};
- obj = kzalloc_obj(*obj);
- if (!obj)
+static struct panthor_gem_object *
+panthor_gem_alloc_object(u32 flags)
+{
+ struct panthor_gem_object *bo;
+
+ bo = kzalloc_obj(*bo);
+ if (!bo)
return ERR_PTR(-ENOMEM);
- obj->base.base.funcs = &panthor_gem_funcs;
- mutex_init(&obj->label.lock);
+ bo->base.funcs = &panthor_gem_funcs;
+ bo->flags = flags;
+ mutex_init(&bo->label.lock);
+ panthor_gem_debugfs_bo_init(bo);
+ return bo;
+}
- panthor_gem_debugfs_bo_init(obj);
+static struct panthor_gem_object *
+panthor_gem_create(struct drm_device *dev, size_t size, uint32_t flags,
+ struct panthor_vm *exclusive_vm, u32 usage_flags)
+{
+ struct panthor_gem_object *bo;
+ int ret;
- return &obj->base.base;
+ bo = panthor_gem_alloc_object(flags);
+ if (IS_ERR(bo))
+ return bo;
+
+ size = PAGE_ALIGN(size);
+ ret = drm_gem_object_init(dev, &bo->base, size);
+ if (ret)
+ goto err_put;
+
+ /* Our buffers are kept pinned, so allocating them
+ * from the MOVABLE zone is a really bad idea, and
+ * conflicts with CMA. See comments above new_inode()
+ * why this is required _and_ expected if you're
+ * going to pin these pages.
+ */
+ mapping_set_gfp_mask(bo->base.filp->f_mapping,
+ GFP_HIGHUSER | __GFP_RETRY_MAYFAIL | __GFP_NOWARN);
+
+ ret = drm_gem_create_mmap_offset(&bo->base);
+ if (ret)
+ goto err_put;
+
+ if (exclusive_vm) {
+ bo->exclusive_vm_root_gem = panthor_vm_root_gem(exclusive_vm);
+ drm_gem_object_get(bo->exclusive_vm_root_gem);
+ bo->base.resv = bo->exclusive_vm_root_gem->resv;
+ }
+
+ panthor_gem_debugfs_set_usage_flags(bo, usage_flags);
+ return bo;
+
+err_put:
+ drm_gem_object_put(&bo->base);
+ return ERR_PTR(ret);
+}
+
+struct drm_gem_object *
+panthor_gem_prime_import_sg_table(struct drm_device *dev,
+ struct dma_buf_attachment *attach,
+ struct sg_table *sgt)
+{
+ struct panthor_gem_object *bo;
+ int ret;
+
+ bo = panthor_gem_alloc_object(0);
+ if (IS_ERR(bo))
+ return ERR_CAST(bo);
+
+ drm_gem_private_object_init(dev, &bo->base, attach->dmabuf->size);
+
+ ret = drm_gem_create_mmap_offset(&bo->base);
+ if (ret)
+ goto err_put;
+
+ bo->dmap.sgt = sgt;
+ return &bo->base;
+
+err_put:
+ drm_gem_object_put(&bo->base);
+ return ERR_PTR(ret);
}
/**
@@ -325,54 +834,22 @@ panthor_gem_create_with_handle(struct drm_file *file,
u64 *size, u32 flags, u32 *handle)
{
int ret;
- struct drm_gem_shmem_object *shmem;
struct panthor_gem_object *bo;
- shmem = drm_gem_shmem_create(ddev, *size);
- if (IS_ERR(shmem))
- return PTR_ERR(shmem);
-
- bo = to_panthor_bo(&shmem->base);
- bo->flags = flags;
- bo->base.map_wc = should_map_wc(bo, exclusive_vm);
-
- if (exclusive_vm) {
- bo->exclusive_vm_root_gem = panthor_vm_root_gem(exclusive_vm);
- drm_gem_object_get(bo->exclusive_vm_root_gem);
- bo->base.base.resv = bo->exclusive_vm_root_gem->resv;
- }
-
- panthor_gem_debugfs_set_usage_flags(bo, 0);
-
- /* If this is a write-combine mapping, we query the sgt to force a CPU
- * cache flush (dma_map_sgtable() is called when the sgt is created).
- * This ensures the zero-ing is visible to any uncached mapping created
- * by vmap/mmap.
- * FIXME: Ideally this should be done when pages are allocated, not at
- * BO creation time.
- */
- if (shmem->map_wc) {
- struct sg_table *sgt;
-
- sgt = drm_gem_shmem_get_pages_sgt(shmem);
- if (IS_ERR(sgt)) {
- ret = PTR_ERR(sgt);
- goto out_put_gem;
- }
- }
+ bo = panthor_gem_create(ddev, *size, flags, exclusive_vm, 0);
+ if (IS_ERR(bo))
+ return PTR_ERR(bo);
/*
* Allocate an id of idr table where the obj is registered
* and handle has the id what user can see.
*/
- ret = drm_gem_handle_create(file, &shmem->base, handle);
+ ret = drm_gem_handle_create(file, &bo->base, handle);
if (!ret)
- *size = bo->base.base.size;
+ *size = bo->base.size;
-out_put_gem:
/* drop reference from allocate - handle holds it now. */
- drm_gem_object_put(&shmem->base);
-
+ drm_gem_object_put(&bo->base);
return ret;
}
@@ -417,18 +894,17 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
u64 offset, u64 size)
{
struct panthor_gem_object *bo = to_panthor_bo(obj);
- struct drm_gem_shmem_object *shmem = &bo->base;
- const struct drm_device *dev = shmem->base.dev;
+ struct device *dma_dev = drm_dev_dma_dev(bo->base.dev);
struct sg_table *sgt;
struct scatterlist *sgl;
unsigned int count;
/* Make sure the range is in bounds. */
- if (offset + size < offset || offset + size > shmem->base.size)
+ if (offset + size < offset || offset + size > bo->base.size)
return -EINVAL;
/* Disallow CPU-cache maintenance on imported buffers. */
- if (drm_gem_is_imported(&shmem->base))
+ if (drm_gem_is_imported(&bo->base))
return -EINVAL;
switch (type) {
@@ -441,14 +917,14 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
}
/* Don't bother if it's WC-mapped */
- if (shmem->map_wc)
+ if (should_map_wc(bo))
return 0;
/* Nothing to do if the size is zero. */
if (size == 0)
return 0;
- sgt = drm_gem_shmem_get_pages_sgt(shmem);
+ sgt = panthor_gem_get_dev_sgt(bo);
if (IS_ERR(sgt))
return PTR_ERR(sgt);
@@ -489,9 +965,9 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
*
* for the flush+invalidate case.
*/
- dma_sync_single_for_device(dev->dev, paddr, len, DMA_TO_DEVICE);
+ dma_sync_single_for_device(dma_dev, paddr, len, DMA_TO_DEVICE);
if (type == DRM_PANTHOR_BO_SYNC_CPU_CACHE_FLUSH_AND_INVALIDATE)
- dma_sync_single_for_cpu(dev->dev, paddr, len, DMA_FROM_DEVICE);
+ dma_sync_single_for_cpu(dma_dev, paddr, len, DMA_FROM_DEVICE);
}
return 0;
@@ -541,7 +1017,6 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
size_t size, u32 bo_flags, u32 vm_map_flags,
u64 gpu_va, const char *name)
{
- struct drm_gem_shmem_object *obj;
struct panthor_kernel_bo *kbo;
struct panthor_gem_object *bo;
u32 debug_flags = PANTHOR_DEBUGFS_GEM_USAGE_FLAG_KERNEL;
@@ -554,25 +1029,18 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
if (!kbo)
return ERR_PTR(-ENOMEM);
- obj = drm_gem_shmem_create(&ptdev->base, size);
- if (IS_ERR(obj)) {
- ret = PTR_ERR(obj);
- goto err_free_bo;
- }
-
- bo = to_panthor_bo(&obj->base);
- kbo->obj = &obj->base;
- bo->flags = bo_flags;
- bo->base.map_wc = should_map_wc(bo, vm);
- bo->exclusive_vm_root_gem = panthor_vm_root_gem(vm);
- drm_gem_object_get(bo->exclusive_vm_root_gem);
- bo->base.base.resv = bo->exclusive_vm_root_gem->resv;
-
if (vm == panthor_fw_vm(ptdev))
debug_flags |= PANTHOR_DEBUGFS_GEM_USAGE_FLAG_FW_MAPPED;
+ bo = panthor_gem_create(&ptdev->base, size, bo_flags, vm, debug_flags);
+ if (IS_ERR(bo)) {
+ ret = PTR_ERR(bo);
+ goto err_free_kbo;
+ }
+
+ kbo->obj = &bo->base;
+
panthor_gem_kernel_bo_set_label(kbo, name);
- panthor_gem_debugfs_set_usage_flags(to_panthor_bo(kbo->obj), debug_flags);
/* The system and GPU MMU page size might differ, which becomes a
* problem for FW sections that need to be mapped at explicit address
@@ -596,9 +1064,9 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
panthor_vm_free_va(vm, &kbo->va_node);
err_put_obj:
- drm_gem_object_put(&obj->base);
+ drm_gem_object_put(&bo->base);
-err_free_bo:
+err_free_kbo:
kfree(kbo);
return ERR_PTR(ret);
}
@@ -646,7 +1114,7 @@ static void panthor_gem_debugfs_bo_print(struct panthor_gem_object *bo,
struct seq_file *m,
struct gem_size_totals *totals)
{
- unsigned int refcount = kref_read(&bo->base.base.refcount);
+ unsigned int refcount = kref_read(&bo->base.refcount);
char creator_info[32] = {};
size_t resident_size;
u32 gem_usage_flags = bo->debugfs.flags;
@@ -656,21 +1124,21 @@ static void panthor_gem_debugfs_bo_print(struct panthor_gem_object *bo,
if (!refcount)
return;
- resident_size = bo->base.pages ? bo->base.base.size : 0;
+ resident_size = bo->backing.pages ? bo->base.size : 0;
snprintf(creator_info, sizeof(creator_info),
"%s/%d", bo->debugfs.creator.process_name, bo->debugfs.creator.tgid);
seq_printf(m, "%-32s%-16d%-16d%-16zd%-16zd0x%-16lx",
creator_info,
- bo->base.base.name,
+ bo->base.name,
refcount,
- bo->base.base.size,
+ bo->base.size,
resident_size,
- drm_vma_node_start(&bo->base.base.vma_node));
+ drm_vma_node_start(&bo->base.vma_node));
- if (drm_gem_is_imported(&bo->base.base))
+ if (drm_gem_is_imported(&bo->base))
gem_state_flags |= PANTHOR_DEBUGFS_GEM_STATE_FLAG_IMPORTED;
- if (bo->base.base.dma_buf)
+ if (bo->base.dma_buf)
gem_state_flags |= PANTHOR_DEBUGFS_GEM_STATE_FLAG_EXPORTED;
seq_printf(m, "0x%-8x 0x%-10x", gem_state_flags, gem_usage_flags);
@@ -679,10 +1147,8 @@ static void panthor_gem_debugfs_bo_print(struct panthor_gem_object *bo,
seq_printf(m, "%s\n", bo->label.str ? : "");
}
- totals->size += bo->base.base.size;
+ totals->size += bo->base.size;
totals->resident += resident_size;
- if (bo->base.madv > 0)
- totals->reclaimable += resident_size;
}
static void panthor_gem_debugfs_print_bos(struct panthor_device *ptdev,
diff --git a/drivers/gpu/drm/panthor/panthor_gem.h b/drivers/gpu/drm/panthor/panthor_gem.h
index 94b2d17cf032..b66478c9590c 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.h
+++ b/drivers/gpu/drm/panthor/panthor_gem.h
@@ -5,7 +5,7 @@
#ifndef __PANTHOR_GEM_H__
#define __PANTHOR_GEM_H__
-#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_gem.h>
#include <drm/drm_mm.h>
#include <linux/iosys-map.h>
@@ -60,12 +60,51 @@ struct panthor_gem_debugfs {
u32 flags;
};
+/**
+ * struct panthor_gem_backing - GEM memory backing related data
+ */
+struct panthor_gem_backing {
+ /** @pages: Pages requested with drm_gem_get_pages() */
+ struct page **pages;
+
+ /** @pin_count: Number of active pin requests on this GEM */
+ refcount_t pin_count;
+};
+
+/**
+ * struct panthor_gem_cpu_map - GEM CPU mapping related data
+ */
+struct panthor_gem_cpu_map {
+ /** @vaddr: Address returned by vmap() */
+ void *vaddr;
+
+ /** @vaddr_use_count: Number of active vmap() requests on this GEM */
+ refcount_t vaddr_use_count;
+};
+
+/**
+ * struct panthor_gem_dev_map - GEM device mapping related data
+ */
+struct panthor_gem_dev_map {
+ /** @sgt: Device mapped sg_table for this GEM */
+ struct sg_table *sgt;
+};
+
/**
* struct panthor_gem_object - Driver specific GEM object.
*/
struct panthor_gem_object {
- /** @base: Inherit from drm_gem_shmem_object. */
- struct drm_gem_shmem_object base;
+ /** @base: Inherit from drm_gem_object. */
+ struct drm_gem_object base;
+
+ /** @backing: Memory backing state */
+ struct panthor_gem_backing backing;
+
+ /** @cmap: CPU mapping state */
+ struct panthor_gem_cpu_map cmap;
+
+ /** @dmap: Device mapping state */
+ struct panthor_gem_dev_map dmap;
/**
* @exclusive_vm_root_gem: Root GEM of the exclusive VM this GEM object
@@ -130,22 +169,25 @@ struct panthor_kernel_bo {
void *kmap;
};
-static inline
-struct panthor_gem_object *to_panthor_bo(struct drm_gem_object *obj)
-{
- return container_of(to_drm_gem_shmem_obj(obj), struct panthor_gem_object, base);
-}
+#define to_panthor_bo(obj) container_of_const(obj, struct panthor_gem_object, base)
void panthor_gem_init(struct panthor_device *ptdev);
-struct drm_gem_object *panthor_gem_create_object(struct drm_device *ddev, size_t size);
-
+struct drm_gem_object *
+panthor_gem_prime_import_sg_table(struct drm_device *dev,
+ struct dma_buf_attachment *attach,
+ struct sg_table *sgt);
int
panthor_gem_create_with_handle(struct drm_file *file,
struct drm_device *ddev,
struct panthor_vm *exclusive_vm,
u64 *size, u32 flags, uint32_t *handle);
+struct sg_table *
+panthor_gem_get_dev_sgt(struct panthor_gem_object *bo);
+int panthor_gem_pin(struct panthor_gem_object *bo);
+void panthor_gem_unpin(struct panthor_gem_object *bo);
+
void panthor_gem_bo_set_label(struct drm_gem_object *obj, const char *label);
void panthor_gem_kernel_bo_set_label(struct panthor_kernel_bo *bo, const char *label);
int panthor_gem_sync(struct drm_gem_object *obj,
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
index ac11c8c6a863..1f42ff505883 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.c
+++ b/drivers/gpu/drm/panthor/panthor_mmu.c
@@ -5,6 +5,7 @@
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
#include <drm/drm_exec.h>
+#include <drm/drm_file.h>
#include <drm/drm_gpuvm.h>
#include <drm/drm_managed.h>
#include <drm/drm_print.h>
@@ -1078,8 +1079,7 @@ static void panthor_vm_bo_free(struct drm_gpuvm_bo *vm_bo)
{
struct panthor_gem_object *bo = to_panthor_bo(vm_bo->obj);
- if (!drm_gem_is_imported(&bo->base.base))
- drm_gem_shmem_unpin(&bo->base);
+ panthor_gem_unpin(bo);
kfree(vm_bo);
}
@@ -1201,7 +1201,7 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
return -EINVAL;
/* Make sure the VA and size are in-bounds. */
- if (size > bo->base.base.size || offset > bo->base.base.size - size)
+ if (size > bo->base.size || offset > bo->base.size - size)
return -EINVAL;
/* If the BO has an exclusive VM attached, it can't be mapped to other VMs. */
@@ -1218,39 +1218,30 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
if (ret)
goto err_cleanup;
- if (!drm_gem_is_imported(&bo->base.base)) {
- /* Pre-reserve the BO pages, so the map operation doesn't have to
- * allocate. This pin is dropped in panthor_vm_bo_free(), so
- * once we have successfully called drm_gpuvm_bo_create(),
- * GPUVM will take care of dropping the pin for us.
- */
- ret = drm_gem_shmem_pin(&bo->base);
- if (ret)
- goto err_cleanup;
- }
+ /* Pre-reserve the BO pages, so the map operation doesn't have to
+ * allocate.
+ */
+ ret = panthor_gem_pin(bo);
+ if (ret)
+ goto err_cleanup;
- sgt = drm_gem_shmem_get_pages_sgt(&bo->base);
+ sgt = panthor_gem_get_dev_sgt(bo);
if (IS_ERR(sgt)) {
- if (!drm_gem_is_imported(&bo->base.base))
- drm_gem_shmem_unpin(&bo->base);
-
+ panthor_gem_unpin(bo);
ret = PTR_ERR(sgt);
goto err_cleanup;
}
op_ctx->map.sgt = sgt;
- preallocated_vm_bo = drm_gpuvm_bo_create(&vm->base, &bo->base.base);
+ preallocated_vm_bo = drm_gpuvm_bo_create(&vm->base, &bo->base);
if (!preallocated_vm_bo) {
- if (!drm_gem_is_imported(&bo->base.base))
- drm_gem_shmem_unpin(&bo->base);
-
+ panthor_gem_unpin(bo);
ret = -ENOMEM;
goto err_cleanup;
}
op_ctx->map.vm_bo = drm_gpuvm_bo_obtain_prealloc(preallocated_vm_bo);
-
op_ctx->map.bo_offset = offset;
/* L1, L2 and L3 page tables.
@@ -1278,7 +1269,7 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
}
/* Insert BO into the extobj list last, when we know nothing can fail. */
- if (bo->base.base.resv != panthor_vm_resv(vm)) {
+ if (bo->base.resv != panthor_vm_resv(vm)) {
dma_resv_lock(panthor_vm_resv(vm), NULL);
drm_gpuvm_bo_extobj_add(op_ctx->map.vm_bo);
dma_resv_unlock(panthor_vm_resv(vm));
@@ -2049,9 +2040,9 @@ static void panthor_vma_link(struct panthor_vm *vm,
{
struct panthor_gem_object *bo = to_panthor_bo(vma->base.gem.obj);
- mutex_lock(&bo->base.base.gpuva.lock);
+ mutex_lock(&bo->base.gpuva.lock);
drm_gpuva_link(&vma->base, vm_bo);
- mutex_unlock(&bo->base.base.gpuva.lock);
+ mutex_unlock(&bo->base.gpuva.lock);
}
static void panthor_vma_unlink(struct panthor_vma *vma)
@@ -2103,11 +2094,12 @@ static int panthor_gpuva_sm_step_map(struct drm_gpuva_op *op, void *priv)
static bool
iova_mapped_as_huge_page(struct drm_gpuva_op_map *op, u64 addr)
{
+ struct panthor_gem_object *bo = to_panthor_bo(op->gem.obj);
const struct page *pg;
pgoff_t bo_offset;
bo_offset = addr - op->va.addr + op->gem.offset;
- pg = to_panthor_bo(op->gem.obj)->base.pages[bo_offset >> PAGE_SHIFT];
+ pg = bo->backing.pages[bo_offset >> PAGE_SHIFT];
return folio_size(page_folio(pg)) >= SZ_2M;
}
@@ -2176,7 +2168,7 @@ static int panthor_gpuva_sm_step_remap(struct drm_gpuva_op *op,
u64 size = op->remap.prev->va.addr + op->remap.prev->va.range - unmap_start;
ret = panthor_vm_map_pages(vm, unmap_start, flags_to_prot(unmap_vma->flags),
- bo->base.sgt, offset, size);
+ bo->dmap.sgt, offset, size);
if (ret)
return ret;
@@ -2190,7 +2182,7 @@ static int panthor_gpuva_sm_step_remap(struct drm_gpuva_op *op,
u64 size = unmap_start + unmap_range - op->remap.next->va.addr;
ret = panthor_vm_map_pages(vm, addr, flags_to_prot(unmap_vma->flags),
- bo->base.sgt, op->remap.next->gem.offset, size);
+ bo->dmap.sgt, op->remap.next->gem.offset, size);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/panthor/panthor_sched.c b/drivers/gpu/drm/panthor/panthor_sched.c
index 2fe04d0f0e3a..97581943a138 100644
--- a/drivers/gpu/drm/panthor/panthor_sched.c
+++ b/drivers/gpu/drm/panthor/panthor_sched.c
@@ -3,7 +3,7 @@
#include <drm/drm_drv.h>
#include <drm/drm_exec.h>
-#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_file.h>
#include <drm/drm_managed.h>
#include <drm/drm_print.h>
#include <drm/gpu_scheduler.h>
@@ -871,8 +871,7 @@ panthor_queue_get_syncwait_obj(struct panthor_group *group, struct panthor_queue
int ret;
if (queue->syncwait.kmap) {
- bo = container_of(queue->syncwait.obj,
- struct panthor_gem_object, base.base);
+ bo = to_panthor_bo(queue->syncwait.obj);
goto out_sync;
}
@@ -882,7 +881,7 @@ panthor_queue_get_syncwait_obj(struct panthor_group *group, struct panthor_queue
if (drm_WARN_ON(&ptdev->base, IS_ERR_OR_NULL(bo)))
goto err_put_syncwait_obj;
- queue->syncwait.obj = &bo->base.base;
+ queue->syncwait.obj = &bo->base;
ret = drm_gem_vmap(queue->syncwait.obj, &map);
if (drm_WARN_ON(&ptdev->base, ret))
goto err_put_syncwait_obj;
@@ -896,7 +895,7 @@ panthor_queue_get_syncwait_obj(struct panthor_group *group, struct panthor_queue
* panthor_gem_sync() is a NOP if map_wc=true, so no need to check
* it here.
*/
- panthor_gem_sync(&bo->base.base,
+ panthor_gem_sync(&bo->base,
DRM_PANTHOR_BO_SYNC_CPU_CACHE_FLUSH_AND_INVALIDATE,
queue->syncwait.offset,
queue->syncwait.sync64 ?
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 07/10] drm/panthor: Lazily allocate pages on mmap()
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (5 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 06/10] drm/panthor: Part ways with drm_gem_shmem_object Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 08/10] drm/panthor: Split panthor_vm_prepare_map_op_ctx() to prepare for reclaim Boris Brezillon
` (3 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
Defer pages allocation until their first access.
v2:
- Don't deal with FAULT_FLAG_INTERRUPTIBLE
- Make sure bo->backing.pages is never an ERR_PTR()
- Drop a useless vm_fault_t local var
- Fix comment in panthor_gem_fault()
v3:
- Collect R-bs
v4:
- No changes
v5:
- No changes
v6:
- Fix huge_fault handling
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Steven Price <steven.price@arm.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
---
drivers/gpu/drm/panthor/panthor_gem.c | 118 ++++++++++++++++----------
1 file changed, 75 insertions(+), 43 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
index 0723a0dc0127..d9c3a71da29b 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.c
+++ b/drivers/gpu/drm/panthor/panthor_gem.c
@@ -604,15 +604,6 @@ static int panthor_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *v
if (is_cow_mapping(vma->vm_flags))
return -EINVAL;
- dma_resv_lock(obj->resv, NULL);
- ret = panthor_gem_backing_get_pages_locked(bo);
- if (!ret)
- ret = panthor_gem_prep_for_cpu_map_locked(bo);
- dma_resv_unlock(obj->resv);
-
- if (ret)
- return ret;
-
vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
if (should_map_wc(bo))
@@ -653,35 +644,95 @@ static vm_fault_t insert_page(struct vm_fault *vmf, unsigned int order, struct p
return VM_FAULT_FALLBACK;
}
+static vm_fault_t nonblocking_page_setup(struct vm_fault *vmf,
+ unsigned int order,
+ pgoff_t page_offset)
+{
+ struct vm_area_struct *vma = vmf->vma;
+ struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
+ vm_fault_t ret;
+
+ if (!dma_resv_trylock(bo->base.resv))
+ return VM_FAULT_RETRY;
+
+ if (bo->backing.pages)
+ ret = insert_page(vmf, order, bo->backing.pages[page_offset]);
+ else
+ ret = VM_FAULT_RETRY;
+
+ dma_resv_unlock(bo->base.resv);
+ return ret;
+}
+
+static vm_fault_t blocking_page_setup(struct vm_fault *vmf, unsigned int order,
+ struct panthor_gem_object *bo,
+ pgoff_t page_offset, bool mmap_lock_held)
+{
+ vm_fault_t ret;
+ int err;
+
+ err = dma_resv_lock_interruptible(bo->base.resv, NULL);
+ if (err)
+ return mmap_lock_held ? VM_FAULT_NOPAGE : VM_FAULT_RETRY;
+
+ err = panthor_gem_backing_get_pages_locked(bo);
+ if (!err)
+ err = panthor_gem_prep_for_cpu_map_locked(bo);
+
+ if (err) {
+ ret = mmap_lock_held ? VM_FAULT_SIGBUS : VM_FAULT_RETRY;
+ } else {
+ struct page *page = bo->backing.pages[page_offset];
+
+ if (mmap_lock_held)
+ ret = insert_page(vmf, order, page);
+ else
+ ret = VM_FAULT_RETRY;
+ }
+
+ dma_resv_unlock(bo->base.resv);
+
+ return ret;
+}
+
static vm_fault_t panthor_gem_any_fault(struct vm_fault *vmf, unsigned int order)
{
struct vm_area_struct *vma = vmf->vma;
- struct drm_gem_object *obj = vma->vm_private_data;
struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
- loff_t num_pages = obj->size >> PAGE_SHIFT;
- vm_fault_t ret;
+ loff_t num_pages = bo->base.size >> PAGE_SHIFT;
pgoff_t page_offset;
+ vm_fault_t ret;
if (order && order != PMD_ORDER)
return VM_FAULT_FALLBACK;
/* Offset to faulty address in the VMA. */
page_offset = vmf->pgoff - vma->vm_pgoff;
+ if (page_offset >= num_pages)
+ return VM_FAULT_SIGBUS;
- dma_resv_lock(bo->base.resv, NULL);
+ ret = nonblocking_page_setup(vmf, order, page_offset);
+ if (ret != VM_FAULT_RETRY)
+ return ret;
- if (page_offset >= num_pages ||
- drm_WARN_ON_ONCE(obj->dev, !bo->backing.pages)) {
- ret = VM_FAULT_SIGBUS;
- goto out;
+ /* Check if we're allowed to retry. */
+ if (fault_flag_allow_retry_first(vmf->flags)) {
+ /* If we're allowed to retry but not wait here, return
+ * immediately, the wait will be done when the fault
+ * handler is called again, with the mmap_lock held.
+ */
+ if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT)
+ return VM_FAULT_RETRY;
+
+ /* Wait with the mmap lock released, if we're allowed to. */
+ drm_gem_object_get(&bo->base);
+ mmap_read_unlock(vmf->vma->vm_mm);
+ ret = blocking_page_setup(vmf, order, bo, page_offset, false);
+ drm_gem_object_put(&bo->base);
+ return ret;
}
- ret = insert_page(vmf, order, bo->backing.pages[page_offset]);
-
- out:
- dma_resv_unlock(bo->base.resv);
-
- return ret;
+ return blocking_page_setup(vmf, order, bo, page_offset, true);
}
static vm_fault_t panthor_gem_fault(struct vm_fault *vmf)
@@ -689,31 +740,12 @@ static vm_fault_t panthor_gem_fault(struct vm_fault *vmf)
return panthor_gem_any_fault(vmf, 0);
}
-static void panthor_gem_vm_open(struct vm_area_struct *vma)
-{
- struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
-
- drm_WARN_ON(bo->base.dev, drm_gem_is_imported(&bo->base));
-
- dma_resv_lock(bo->base.resv, NULL);
-
- /* We should have already pinned the pages when the buffer was first
- * mmap'd, vm_open() just grabs an additional reference for the new
- * mm the vma is getting copied into (ie. on fork()).
- */
- drm_WARN_ON_ONCE(bo->base.dev, !bo->backing.pages);
-
- dma_resv_unlock(bo->base.resv);
-
- drm_gem_vm_open(vma);
-}
-
static const struct vm_operations_struct panthor_gem_vm_ops = {
.fault = panthor_gem_fault,
#ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP
.huge_fault = panthor_gem_any_fault,
#endif
- .open = panthor_gem_vm_open,
+ .open = drm_gem_vm_open,
.close = drm_gem_vm_close,
};
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 08/10] drm/panthor: Split panthor_vm_prepare_map_op_ctx() to prepare for reclaim
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (6 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 07/10] drm/panthor: Lazily allocate pages on mmap() Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-01 13:48 ` [PATCH v7 09/10] drm/panthor: Track the number of mmap on a BO Boris Brezillon
` (2 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
We're gonna need just the page table reservation logic when we restore
evicted BO mappings, so let's prepare for that by extracting the
op_ctx init and page table pre-allocation into separate helpers.
v2:
- Collect R-bs
v3:
- No changes
v4:
- No changes
v5:
- No changes
v6:
- No changes
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
drivers/gpu/drm/panthor/panthor_mmu.c | 68 ++++++++++++++++-----------
1 file changed, 41 insertions(+), 27 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
index 1f42ff505883..b78f694ca92b 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.c
+++ b/drivers/gpu/drm/panthor/panthor_mmu.c
@@ -1175,6 +1175,44 @@ panthor_vm_op_ctx_prealloc_vmas(struct panthor_vm_op_ctx *op_ctx)
return 0;
}
+static void panthor_vm_init_op_ctx(struct panthor_vm_op_ctx *op_ctx,
+ u64 size, u64 va, u32 flags)
+{
+ memset(op_ctx, 0, sizeof(*op_ctx));
+ op_ctx->flags = flags;
+ op_ctx->va.range = size;
+ op_ctx->va.addr = va;
+}
+
+static int panthor_vm_op_ctx_prealloc_pts(struct panthor_vm_op_ctx *op_ctx)
+{
+ u64 size = op_ctx->va.range;
+ u64 va = op_ctx->va.addr;
+ int ret;
+
+ /* L1, L2 and L3 page tables.
+ * We could optimize L3 allocation by iterating over the sgt and merging
+ * 2M contiguous blocks, but it's simpler to over-provision and return
+ * the pages if they're not used.
+ */
+ u64 pt_count = ((ALIGN(va + size, 1ull << 39) - ALIGN_DOWN(va, 1ull << 39)) >> 39) +
+ ((ALIGN(va + size, 1ull << 30) - ALIGN_DOWN(va, 1ull << 30)) >> 30) +
+ ((ALIGN(va + size, 1ull << 21) - ALIGN_DOWN(va, 1ull << 21)) >> 21);
+
+ op_ctx->rsvd_page_tables.pages = kzalloc_objs(*op_ctx->rsvd_page_tables.pages,
+ pt_count);
+ if (!op_ctx->rsvd_page_tables.pages)
+ return -ENOMEM;
+
+ ret = kmem_cache_alloc_bulk(pt_cache, GFP_KERNEL, pt_count,
+ op_ctx->rsvd_page_tables.pages);
+ op_ctx->rsvd_page_tables.count = ret;
+ if (ret != pt_count)
+ return -ENOMEM;
+
+ return 0;
+}
+
#define PANTHOR_VM_BIND_OP_MAP_FLAGS \
(DRM_PANTHOR_VM_BIND_OP_MAP_READONLY | \
DRM_PANTHOR_VM_BIND_OP_MAP_NOEXEC | \
@@ -1190,7 +1228,6 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
{
struct drm_gpuvm_bo *preallocated_vm_bo;
struct sg_table *sgt = NULL;
- u64 pt_count;
int ret;
if (!bo)
@@ -1209,10 +1246,7 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
bo->exclusive_vm_root_gem != panthor_vm_root_gem(vm))
return -EINVAL;
- memset(op_ctx, 0, sizeof(*op_ctx));
- op_ctx->flags = flags;
- op_ctx->va.range = size;
- op_ctx->va.addr = va;
+ panthor_vm_init_op_ctx(op_ctx, size, va, flags);
ret = panthor_vm_op_ctx_prealloc_vmas(op_ctx);
if (ret)
@@ -1244,29 +1278,9 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
op_ctx->map.vm_bo = drm_gpuvm_bo_obtain_prealloc(preallocated_vm_bo);
op_ctx->map.bo_offset = offset;
- /* L1, L2 and L3 page tables.
- * We could optimize L3 allocation by iterating over the sgt and merging
- * 2M contiguous blocks, but it's simpler to over-provision and return
- * the pages if they're not used.
- */
- pt_count = ((ALIGN(va + size, 1ull << 39) - ALIGN_DOWN(va, 1ull << 39)) >> 39) +
- ((ALIGN(va + size, 1ull << 30) - ALIGN_DOWN(va, 1ull << 30)) >> 30) +
- ((ALIGN(va + size, 1ull << 21) - ALIGN_DOWN(va, 1ull << 21)) >> 21);
-
- op_ctx->rsvd_page_tables.pages = kzalloc_objs(*op_ctx->rsvd_page_tables.pages,
- pt_count);
- if (!op_ctx->rsvd_page_tables.pages) {
- ret = -ENOMEM;
+ ret = panthor_vm_op_ctx_prealloc_pts(op_ctx);
+ if (ret)
goto err_cleanup;
- }
-
- ret = kmem_cache_alloc_bulk(pt_cache, GFP_KERNEL, pt_count,
- op_ctx->rsvd_page_tables.pages);
- op_ctx->rsvd_page_tables.count = ret;
- if (ret != pt_count) {
- ret = -ENOMEM;
- goto err_cleanup;
- }
/* Insert BO into the extobj list last, when we know nothing can fail. */
if (bo->base.resv != panthor_vm_resv(vm)) {
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v7 09/10] drm/panthor: Track the number of mmap on a BO
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (7 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 08/10] drm/panthor: Split panthor_vm_prepare_map_op_ctx() to prepare for reclaim Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-02 14:33 ` Liviu Dudau
2026-04-01 13:48 ` [PATCH v7 10/10] drm/panthor: Add a GEM shrinker Boris Brezillon
2026-04-03 11:54 ` [PATCH v7 00/10] " Boris Brezillon
10 siblings, 1 reply; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
This will be used to order things by reclaimability.
v2:
- Fix refcounting
v3:
- Fix refcounting (again)
v4:
- Collect R-b
v5:
- Collect R-b
v6:
- Warn on is_imported() (not supposed to happen since mmap is
redirected to the dmabuf layer) instead of adding a conditional
in panthor_gem_vm_open()
v7:
- No changes
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
drivers/gpu/drm/panthor/panthor_gem.c | 42 +++++++++++++++++++++++++--
drivers/gpu/drm/panthor/panthor_gem.h | 3 ++
2 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
index d9c3a71da29b..eb139759beda 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.c
+++ b/drivers/gpu/drm/panthor/panthor_gem.c
@@ -488,6 +488,7 @@ static void panthor_gem_print_info(struct drm_printer *p, unsigned int indent,
drm_printf_indent(p, indent, "vmap_use_count=%u\n",
refcount_read(&bo->cmap.vaddr_use_count));
drm_printf_indent(p, indent, "vaddr=%p\n", bo->cmap.vaddr);
+ drm_printf_indent(p, indent, "mmap_count=%u\n", refcount_read(&bo->cmap.mmap_count));
}
static int panthor_gem_pin_locked(struct drm_gem_object *obj)
@@ -604,6 +605,13 @@ static int panthor_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *v
if (is_cow_mapping(vma->vm_flags))
return -EINVAL;
+ if (!refcount_inc_not_zero(&bo->cmap.mmap_count)) {
+ dma_resv_lock(obj->resv, NULL);
+ if (!refcount_inc_not_zero(&bo->cmap.mmap_count))
+ refcount_set(&bo->cmap.mmap_count, 1);
+ dma_resv_unlock(obj->resv);
+ }
+
vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
if (should_map_wc(bo))
@@ -740,13 +748,43 @@ static vm_fault_t panthor_gem_fault(struct vm_fault *vmf)
return panthor_gem_any_fault(vmf, 0);
}
+static void panthor_gem_vm_open(struct vm_area_struct *vma)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
+
+ drm_WARN_ON(bo->base.dev, drm_gem_is_imported(&bo->base));
+ drm_WARN_ON(bo->base.dev, !refcount_inc_not_zero(&bo->cmap.mmap_count));
+
+ drm_gem_vm_open(vma);
+}
+
+static void panthor_gem_vm_close(struct vm_area_struct *vma)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
+
+ if (drm_gem_is_imported(&bo->base))
+ goto out;
+
+ if (refcount_dec_not_one(&bo->cmap.mmap_count))
+ goto out;
+
+ dma_resv_lock(bo->base.resv, NULL);
+ if (refcount_dec_and_test(&bo->cmap.mmap_count)) {
+ /* Nothing to do, pages are reclaimed lazily. */
+ }
+ dma_resv_unlock(bo->base.resv);
+
+out:
+ drm_gem_object_put(&bo->base);
+}
+
static const struct vm_operations_struct panthor_gem_vm_ops = {
.fault = panthor_gem_fault,
#ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP
.huge_fault = panthor_gem_any_fault,
#endif
- .open = drm_gem_vm_open,
- .close = drm_gem_vm_close,
+ .open = panthor_gem_vm_open,
+ .close = panthor_gem_vm_close,
};
static const struct drm_gem_object_funcs panthor_gem_funcs = {
diff --git a/drivers/gpu/drm/panthor/panthor_gem.h b/drivers/gpu/drm/panthor/panthor_gem.h
index b66478c9590c..c0a18dca732c 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.h
+++ b/drivers/gpu/drm/panthor/panthor_gem.h
@@ -80,6 +80,9 @@ struct panthor_gem_cpu_map {
/** @vaddr_use_count: Number of active vmap() requests on this GEM */
refcount_t vaddr_use_count;
+
+ /** @mmap_count: Number of active mmap() requests on this GEM */
+ refcount_t mmap_count;
};
/**
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v7 09/10] drm/panthor: Track the number of mmap on a BO
2026-04-01 13:48 ` [PATCH v7 09/10] drm/panthor: Track the number of mmap on a BO Boris Brezillon
@ 2026-04-02 14:33 ` Liviu Dudau
0 siblings, 0 replies; 14+ messages in thread
From: Liviu Dudau @ 2026-04-02 14:33 UTC (permalink / raw)
To: Boris Brezillon
Cc: Steven Price, Adrián Larumbe, dri-devel, David Airlie,
Simona Vetter, Akash Goel, Rob Clark, Sean Paul, Konrad Dybcio,
Akhil P Oommen, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
On Wed, Apr 01, 2026 at 03:48:53PM +0200, Boris Brezillon wrote:
> This will be used to order things by reclaimability.
>
> v2:
> - Fix refcounting
>
> v3:
> - Fix refcounting (again)
>
> v4:
> - Collect R-b
>
> v5:
> - Collect R-b
>
> v6:
> - Warn on is_imported() (not supposed to happen since mmap is
> redirected to the dmabuf layer) instead of adding a conditional
> in panthor_gem_vm_open()
>
> v7:
> - No changes
>
> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> Reviewed-by: Steven Price <steven.price@arm.com>
Reviewed-by: Liviu Dudau <liviu.dudau@arm.com>
Best regards,
Liviu
> ---
> drivers/gpu/drm/panthor/panthor_gem.c | 42 +++++++++++++++++++++++++--
> drivers/gpu/drm/panthor/panthor_gem.h | 3 ++
> 2 files changed, 43 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
> index d9c3a71da29b..eb139759beda 100644
> --- a/drivers/gpu/drm/panthor/panthor_gem.c
> +++ b/drivers/gpu/drm/panthor/panthor_gem.c
> @@ -488,6 +488,7 @@ static void panthor_gem_print_info(struct drm_printer *p, unsigned int indent,
> drm_printf_indent(p, indent, "vmap_use_count=%u\n",
> refcount_read(&bo->cmap.vaddr_use_count));
> drm_printf_indent(p, indent, "vaddr=%p\n", bo->cmap.vaddr);
> + drm_printf_indent(p, indent, "mmap_count=%u\n", refcount_read(&bo->cmap.mmap_count));
> }
>
> static int panthor_gem_pin_locked(struct drm_gem_object *obj)
> @@ -604,6 +605,13 @@ static int panthor_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *v
> if (is_cow_mapping(vma->vm_flags))
> return -EINVAL;
>
> + if (!refcount_inc_not_zero(&bo->cmap.mmap_count)) {
> + dma_resv_lock(obj->resv, NULL);
> + if (!refcount_inc_not_zero(&bo->cmap.mmap_count))
> + refcount_set(&bo->cmap.mmap_count, 1);
> + dma_resv_unlock(obj->resv);
> + }
> +
> vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
> vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
> if (should_map_wc(bo))
> @@ -740,13 +748,43 @@ static vm_fault_t panthor_gem_fault(struct vm_fault *vmf)
> return panthor_gem_any_fault(vmf, 0);
> }
>
> +static void panthor_gem_vm_open(struct vm_area_struct *vma)
> +{
> + struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
> +
> + drm_WARN_ON(bo->base.dev, drm_gem_is_imported(&bo->base));
> + drm_WARN_ON(bo->base.dev, !refcount_inc_not_zero(&bo->cmap.mmap_count));
> +
> + drm_gem_vm_open(vma);
> +}
> +
> +static void panthor_gem_vm_close(struct vm_area_struct *vma)
> +{
> + struct panthor_gem_object *bo = to_panthor_bo(vma->vm_private_data);
> +
> + if (drm_gem_is_imported(&bo->base))
> + goto out;
> +
> + if (refcount_dec_not_one(&bo->cmap.mmap_count))
> + goto out;
> +
> + dma_resv_lock(bo->base.resv, NULL);
> + if (refcount_dec_and_test(&bo->cmap.mmap_count)) {
> + /* Nothing to do, pages are reclaimed lazily. */
> + }
> + dma_resv_unlock(bo->base.resv);
> +
> +out:
> + drm_gem_object_put(&bo->base);
> +}
> +
> static const struct vm_operations_struct panthor_gem_vm_ops = {
> .fault = panthor_gem_fault,
> #ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP
> .huge_fault = panthor_gem_any_fault,
> #endif
> - .open = drm_gem_vm_open,
> - .close = drm_gem_vm_close,
> + .open = panthor_gem_vm_open,
> + .close = panthor_gem_vm_close,
> };
>
> static const struct drm_gem_object_funcs panthor_gem_funcs = {
> diff --git a/drivers/gpu/drm/panthor/panthor_gem.h b/drivers/gpu/drm/panthor/panthor_gem.h
> index b66478c9590c..c0a18dca732c 100644
> --- a/drivers/gpu/drm/panthor/panthor_gem.h
> +++ b/drivers/gpu/drm/panthor/panthor_gem.h
> @@ -80,6 +80,9 @@ struct panthor_gem_cpu_map {
>
> /** @vaddr_use_count: Number of active vmap() requests on this GEM */
> refcount_t vaddr_use_count;
> +
> + /** @mmap_count: Number of active mmap() requests on this GEM */
> + refcount_t mmap_count;
> };
>
> /**
> --
> 2.53.0
>
--
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v7 10/10] drm/panthor: Add a GEM shrinker
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (8 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 09/10] drm/panthor: Track the number of mmap on a BO Boris Brezillon
@ 2026-04-01 13:48 ` Boris Brezillon
2026-04-03 11:54 ` [PATCH v7 00/10] " Boris Brezillon
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-01 13:48 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
From: Akash Goel <akash.goel@arm.com>
This implementation is losely based on the MSM shrinker, and it's
relying on the drm_gpuvm eviction/validation infrastructure.
Right now we only support swapout/eviction, but we could add an extra
flag to specify when buffer content doesn't need to be preserved to
avoid the swapout/swapin dance.
Locking is a bit of a nightmare, but using _trylock() all the way in
the reclaim path seems to make lockdep happy. And yes, we might be
missing opportunities to reclaim when the system is under heavy GPU
load/heavy memory pressure/heavy GPU VM activity, but that's better
than no reclaim at all.
v2:
- Move gpu_mapped_shared next to the mmapped LRU
- Add a bunch of missing is_[vm_bo,vma]_evicted() tests
- Only test mmap_count to check if a BO is mmaped
- Remove stale comment about shrinker not being a thing
- Allow pin_count to be non-zero in panthor_gem_swapin_locked()
- Fix panthor_gem_sync() to check for BO residency before doing the CPU sync
- Fix the value returned by panthor_gem_shrinker_count() in case some
memory has been released
- Check drmm_mutex_init() ret code
- Explicitly mention that PANTHOR_GEM_UNRECLAIMABLE is the initial state
of all BOs
v3:
- Make panthor_gem_try_evict() static
- Collect {A,R}-bs
v4:
- Update the reclaim_state in panthor_gem_mmap()
- Don't reclaim GPU-mapped BOs if can_block() returns false
- Skip evicited vm_bos in panthor_vm_update_bo_reclaim_lru_locked() to
avoid spurious WARN_ON()s
- Explain why we have to do this
select_evicted_vma/repopulate_evicted_vma dance
v5:
- Properly report the reclaimable size in panthor_gem_debugfs_print_bos()
- Check panthor_vm_lock_region() errors in
panthor_vm_evict_bo_mappings_locked()
- Fix lock order inversion (dma_resv_wait_timeout() inside gpuva.lock)
v6:
- Don't remap if the unmapped VMA is evicted.
- Drop a stale comment in panthor_gem_dev_map_get_sgt_locked()
- s/PANTHOR_GEM_GPU_MAPPED_PRIVATE/PANTHOR_GEM_GPU_MAPPED_SINGLE_VM/
- s/PANTHOR_GEM_GPU_MAPPED_SHARED/PANTHOR_GEM_GPU_MAPPED_MULTI_VM/
- Just count the number of vm_bo to determine the reclaim state in
is_gpu_mapped()
- Drop a redundant panthor_gem_backing_get_pages_locked() call in
panthor_gem_swapin_locked()
- Add more comments to panthor_vm_evict_bo_mappings_locked() to
convince Claude it's actually safe.
- Add a comment in panthor_gem_shrinker_count() to mention the race
and hopefully clear Claude's concerns.
- Rework the "is-still-in-list" check we have in
panthor_mmu_reclaim_priv_bos() to address Claude's concerns
- Don't call panthor_vm_unlock_region() if panthor_vm_lock_region()
fails in remap_evicted_vma(). Was harmless, but confusing, as
pointed out by Claude
- Fix can_block() to allow blocking on KSWAPD_RECLAIM-only in a kswapd
context
- Fix a lockdep warning when the last ref on a BO is released in the
shrinker path, where the resv lock can't be acquired other than with
a try_lock()
v7:
- Skip drm_gpuvm_bo_deferred_cleanup() when repopulating VMAs to fix a
deadlock
Signed-off-by: Akash Goel <akash.goel@arm.com>
Co-developed-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Acked-by: Liviu Dudau <liviu.dudau@arm.com>
Reviewed-by: Steven Price <steven.price@arm.com>
---
drivers/gpu/drm/panthor/panthor_device.c | 11 +-
drivers/gpu/drm/panthor/panthor_device.h | 73 ++++
drivers/gpu/drm/panthor/panthor_gem.c | 508 ++++++++++++++++++++++-
drivers/gpu/drm/panthor/panthor_gem.h | 68 +++
drivers/gpu/drm/panthor/panthor_mmu.c | 397 +++++++++++++++++-
drivers/gpu/drm/panthor/panthor_mmu.h | 8 +
6 files changed, 1027 insertions(+), 38 deletions(-)
diff --git a/drivers/gpu/drm/panthor/panthor_device.c b/drivers/gpu/drm/panthor/panthor_device.c
index 54fbb1aa07c5..bc62a498a8a8 100644
--- a/drivers/gpu/drm/panthor/panthor_device.c
+++ b/drivers/gpu/drm/panthor/panthor_device.c
@@ -2,6 +2,7 @@
/* Copyright 2018 Marty E. Plummer <hanetzer@startmail.com> */
/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
/* Copyright 2023 Collabora ltd. */
+/* Copyright 2025 ARM Limited. All rights reserved. */
#include <linux/clk.h>
#include <linux/mm.h>
@@ -122,6 +123,7 @@ void panthor_device_unplug(struct panthor_device *ptdev)
panthor_sched_unplug(ptdev);
panthor_fw_unplug(ptdev);
panthor_mmu_unplug(ptdev);
+ panthor_gem_shrinker_unplug(ptdev);
panthor_gpu_unplug(ptdev);
panthor_pwr_unplug(ptdev);
@@ -291,10 +293,14 @@ int panthor_device_init(struct panthor_device *ptdev)
if (ret)
goto err_unplug_gpu;
- ret = panthor_mmu_init(ptdev);
+ ret = panthor_gem_shrinker_init(ptdev);
if (ret)
goto err_unplug_gpu;
+ ret = panthor_mmu_init(ptdev);
+ if (ret)
+ goto err_unplug_shrinker;
+
ret = panthor_fw_init(ptdev);
if (ret)
goto err_unplug_mmu;
@@ -326,6 +332,9 @@ int panthor_device_init(struct panthor_device *ptdev)
err_unplug_mmu:
panthor_mmu_unplug(ptdev);
+err_unplug_shrinker:
+ panthor_gem_shrinker_unplug(ptdev);
+
err_unplug_gpu:
panthor_gpu_unplug(ptdev);
diff --git a/drivers/gpu/drm/panthor/panthor_device.h b/drivers/gpu/drm/panthor/panthor_device.h
index b6696f73a536..5cba272f9b4d 100644
--- a/drivers/gpu/drm/panthor/panthor_device.h
+++ b/drivers/gpu/drm/panthor/panthor_device.h
@@ -14,6 +14,7 @@
#include <linux/spinlock.h>
#include <drm/drm_device.h>
+#include <drm/drm_gem.h>
#include <drm/drm_mm.h>
#include <drm/gpu_scheduler.h>
#include <drm/panthor_drm.h>
@@ -178,6 +179,78 @@ struct panthor_device {
/** @devfreq: Device frequency scaling management data. */
struct panthor_devfreq *devfreq;
+ /** @reclaim: Reclaim related stuff */
+ struct {
+ /** @reclaim.shrinker: Shrinker instance */
+ struct shrinker *shrinker;
+
+ /** @reclaim.lock: Lock protecting all LRUs */
+ struct mutex lock;
+
+ /**
+ * @reclaim.unused: BOs with unused pages
+ *
+ * Basically all buffers that got mmapped, vmapped or GPU mapped and
+ * then unmapped. There should be no contention on these buffers,
+ * making them ideal to reclaim.
+ */
+ struct drm_gem_lru unused;
+
+ /**
+ * @reclaim.mmapped: mmap()-ed buffers
+ *
+ * Those are relatively easy to reclaim since we don't need user
+ * agreement, we can simply teardown the mapping and let it fault on
+ * the next access.
+ */
+ struct drm_gem_lru mmapped;
+
+ /**
+ * @reclaim.gpu_mapped_shared: shared BO LRU list
+ *
+ * That's the most tricky BO type to reclaim, because it involves
+ * tearing down all mappings in all VMs where this BO is mapped,
+ * which increases the risk of contention and thus decreases the
+ * likeliness of success.
+ */
+ struct drm_gem_lru gpu_mapped_shared;
+
+ /**
+ * @reclaim.vms: VM LRU list
+ *
+ * VMs that have reclaimable BOs only mapped to a single VM are placed
+ * in this LRU. Reclaiming such BOs implies waiting for VM idleness
+ * (no in-flight GPU jobs targeting this VM), meaning we can't reclaim
+ * those if we're in a context where we can't block/sleep.
+ */
+ struct list_head vms;
+
+ /**
+ * @reclaim.gpu_mapped_count: Global counter of pages that are GPU mapped
+ *
+ * Allows us to get the number of reclaimable pages without walking
+ * the vms and gpu_mapped_shared LRUs.
+ */
+ long gpu_mapped_count;
+
+ /**
+ * @reclaim.retry_count: Number of times we ran the shrinker without being
+ * able to reclaim stuff
+ *
+ * Used to stop scanning GEMs when too many attempts were made
+ * without progress.
+ */
+ atomic_t retry_count;
+
+#ifdef CONFIG_DEBUG_FS
+ /**
+ * @reclaim.nr_pages_reclaimed_on_last_scan: Number of pages reclaimed on the last
+ * shrinker scan
+ */
+ unsigned long nr_pages_reclaimed_on_last_scan;
+#endif
+ } reclaim;
+
/** @unplug: Device unplug related fields. */
struct {
/** @lock: Lock used to serialize unplug operations. */
diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c
index eb139759beda..69cef05b6ef7 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.c
+++ b/drivers/gpu/drm/panthor/panthor_gem.c
@@ -2,8 +2,10 @@
/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
/* Copyright 2023 Collabora ltd. */
/* Copyright 2025 Amazon.com, Inc. or its affiliates */
+/* Copyright 2025 ARM Limited. All rights reserved. */
#include <linux/cleanup.h>
+#include <linux/debugfs.h>
#include <linux/dma-buf.h>
#include <linux/dma-mapping.h>
#include <linux/err.h>
@@ -12,6 +14,8 @@
#include <drm/drm_debugfs.h>
#include <drm/drm_file.h>
+#include <drm/drm_gpuvm.h>
+#include <drm/drm_managed.h>
#include <drm/drm_prime.h>
#include <drm/drm_print.h>
#include <drm/panthor_drm.h>
@@ -114,10 +118,116 @@ should_map_wc(struct panthor_gem_object *bo)
return true;
}
+static bool is_gpu_mapped(struct panthor_gem_object *bo,
+ enum panthor_gem_reclaim_state *state)
+{
+ struct drm_gpuvm_bo *vm_bo;
+ u32 vm_count = 0;
+
+ drm_gem_for_each_gpuvm_bo(vm_bo, &bo->base) {
+ /* Skip evicted GPU mappings. */
+ if (vm_bo->evicted)
+ continue;
+
+ if (vm_count++) {
+ *state = PANTHOR_GEM_GPU_MAPPED_MULTI_VM;
+ break;
+ }
+
+ *state = PANTHOR_GEM_GPU_MAPPED_SINGLE_VM;
+ }
+
+ return vm_count > 0;
+}
+
+static enum panthor_gem_reclaim_state
+panthor_gem_evaluate_reclaim_state_locked(struct panthor_gem_object *bo)
+{
+ enum panthor_gem_reclaim_state gpu_mapped_state;
+
+ dma_resv_assert_held(bo->base.resv);
+ lockdep_assert_held(&bo->base.gpuva.lock);
+
+ /* If pages have not been allocated, there's nothing to reclaim. */
+ if (!bo->backing.pages)
+ return PANTHOR_GEM_UNRECLAIMABLE;
+
+ /* If memory is pinned, we prevent reclaim. */
+ if (refcount_read(&bo->backing.pin_count))
+ return PANTHOR_GEM_UNRECLAIMABLE;
+
+ if (is_gpu_mapped(bo, &gpu_mapped_state))
+ return gpu_mapped_state;
+
+ if (refcount_read(&bo->cmap.mmap_count))
+ return PANTHOR_GEM_MMAPPED;
+
+ return PANTHOR_GEM_UNUSED;
+}
+
+void panthor_gem_update_reclaim_state_locked(struct panthor_gem_object *bo,
+ enum panthor_gem_reclaim_state *old_statep)
+{
+ struct panthor_device *ptdev = container_of(bo->base.dev, struct panthor_device, base);
+ enum panthor_gem_reclaim_state old_state = bo->reclaim_state;
+ enum panthor_gem_reclaim_state new_state;
+ bool was_gpu_mapped, is_gpu_mapped;
+
+ if (old_statep)
+ *old_statep = old_state;
+
+ new_state = panthor_gem_evaluate_reclaim_state_locked(bo);
+ if (new_state == old_state)
+ return;
+
+ was_gpu_mapped = old_state == PANTHOR_GEM_GPU_MAPPED_MULTI_VM ||
+ old_state == PANTHOR_GEM_GPU_MAPPED_SINGLE_VM;
+ is_gpu_mapped = new_state == PANTHOR_GEM_GPU_MAPPED_MULTI_VM ||
+ new_state == PANTHOR_GEM_GPU_MAPPED_SINGLE_VM;
+
+ if (is_gpu_mapped && !was_gpu_mapped)
+ ptdev->reclaim.gpu_mapped_count += bo->base.size >> PAGE_SHIFT;
+ else if (!is_gpu_mapped && was_gpu_mapped)
+ ptdev->reclaim.gpu_mapped_count -= bo->base.size >> PAGE_SHIFT;
+
+ switch (new_state) {
+ case PANTHOR_GEM_UNUSED:
+ drm_gem_lru_move_tail(&ptdev->reclaim.unused, &bo->base);
+ break;
+ case PANTHOR_GEM_MMAPPED:
+ drm_gem_lru_move_tail(&ptdev->reclaim.mmapped, &bo->base);
+ break;
+ case PANTHOR_GEM_GPU_MAPPED_SINGLE_VM:
+ panthor_vm_update_bo_reclaim_lru_locked(bo);
+ break;
+ case PANTHOR_GEM_GPU_MAPPED_MULTI_VM:
+ drm_gem_lru_move_tail(&ptdev->reclaim.gpu_mapped_shared, &bo->base);
+ break;
+ case PANTHOR_GEM_UNRECLAIMABLE:
+ drm_gem_lru_remove(&bo->base);
+ break;
+ default:
+ drm_WARN(&ptdev->base, true, "invalid GEM reclaim state (%d)\n", new_state);
+ break;
+ }
+
+ bo->reclaim_state = new_state;
+}
+
+static void
+bo_assert_locked_or_gone(struct panthor_gem_object *bo)
+{
+ /* If the refcount is zero, the BO is being freed, and we
+ * allow the lock to not be held in that particular case.
+ */
+ if (kref_read(&bo->base.refcount))
+ dma_resv_assert_held(bo->base.resv);
+}
+
static void
panthor_gem_backing_cleanup_locked(struct panthor_gem_object *bo)
{
- dma_resv_assert_held(bo->base.resv);
+ bo_assert_locked_or_gone(bo);
if (!bo->backing.pages)
return;
@@ -157,28 +267,35 @@ static int panthor_gem_backing_pin_locked(struct panthor_gem_object *bo)
return 0;
ret = panthor_gem_backing_get_pages_locked(bo);
- if (!ret)
+ if (!ret) {
refcount_set(&bo->backing.pin_count, 1);
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
+ }
return ret;
}
static void panthor_gem_backing_unpin_locked(struct panthor_gem_object *bo)
{
- dma_resv_assert_held(bo->base.resv);
+ bo_assert_locked_or_gone(bo);
drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base));
if (refcount_dec_and_test(&bo->backing.pin_count)) {
/* We don't release anything when pin_count drops to zero.
* Pages stay there until an explicit cleanup is requested.
*/
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
}
}
static void
panthor_gem_dev_map_cleanup_locked(struct panthor_gem_object *bo)
{
- dma_resv_assert_held(bo->base.resv);
+ bo_assert_locked_or_gone(bo);
if (!bo->dmap.sgt)
return;
@@ -200,9 +317,6 @@ panthor_gem_dev_map_get_sgt_locked(struct panthor_gem_object *bo)
if (bo->dmap.sgt)
return bo->dmap.sgt;
- /* Pages stay around after they've been allocated. At least that stands
- * until we add a shrinker.
- */
ret = panthor_gem_backing_get_pages_locked(bo);
if (ret)
return ERR_PTR(ret);
@@ -336,11 +450,19 @@ static void panthor_gem_free_object(struct drm_gem_object *obj)
if (drm_gem_is_imported(obj)) {
drm_prime_gem_destroy(obj, bo->dmap.sgt);
} else {
- dma_resv_lock(obj->resv, NULL);
+ /* The last ref on the GEM object can be released
+ * by the shrinker, which can't block on the resv
+ * lock acquisition. In practice, even if we were
+ * taking the lock, it wouldn't block because we're
+ * the last piece of code having a visibility on
+ * this GEM, but lockdep can't see that, so we've
+ * just tought the _cleanup_locked() helpers about
+ * this "being freed" exception, and we call those
+ * without the lock held here.
+ */
panthor_gem_vmap_cleanup_locked(bo);
panthor_gem_dev_map_cleanup_locked(bo);
panthor_gem_backing_cleanup_locked(bo);
- dma_resv_unlock(obj->resv);
}
drm_gem_object_release(obj);
@@ -535,6 +657,41 @@ void panthor_gem_unpin(struct panthor_gem_object *bo)
dma_resv_unlock(bo->base.resv);
}
+int panthor_gem_swapin_locked(struct panthor_gem_object *bo)
+{
+ struct sg_table *sgt;
+
+ dma_resv_assert_held(bo->base.resv);
+
+ if (drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base)))
+ return -EINVAL;
+
+ sgt = panthor_gem_dev_map_get_sgt_locked(bo);
+ if (IS_ERR(sgt))
+ return PTR_ERR(sgt);
+
+ return 0;
+}
+
+static void panthor_gem_evict_locked(struct panthor_gem_object *bo)
+{
+ dma_resv_assert_held(bo->base.resv);
+ lockdep_assert_held(&bo->base.gpuva.lock);
+
+ if (drm_WARN_ON_ONCE(bo->base.dev, drm_gem_is_imported(&bo->base)))
+ return;
+
+ if (drm_WARN_ON_ONCE(bo->base.dev, refcount_read(&bo->backing.pin_count)))
+ return;
+
+ if (drm_WARN_ON_ONCE(bo->base.dev, !bo->backing.pages))
+ return;
+
+ panthor_gem_dev_map_cleanup_locked(bo);
+ panthor_gem_backing_cleanup_locked(bo);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+}
+
static struct sg_table *panthor_gem_get_sg_table(struct drm_gem_object *obj)
{
struct panthor_gem_object *bo = to_panthor_bo(obj);
@@ -607,8 +764,12 @@ static int panthor_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *v
if (!refcount_inc_not_zero(&bo->cmap.mmap_count)) {
dma_resv_lock(obj->resv, NULL);
- if (!refcount_inc_not_zero(&bo->cmap.mmap_count))
+ if (!refcount_inc_not_zero(&bo->cmap.mmap_count)) {
refcount_set(&bo->cmap.mmap_count, 1);
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
+ }
dma_resv_unlock(obj->resv);
}
@@ -692,6 +853,10 @@ static vm_fault_t blocking_page_setup(struct vm_fault *vmf, unsigned int order,
} else {
struct page *page = bo->backing.pages[page_offset];
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
+
if (mmap_lock_held)
ret = insert_page(vmf, order, page);
else
@@ -734,7 +899,12 @@ static vm_fault_t panthor_gem_any_fault(struct vm_fault *vmf, unsigned int order
/* Wait with the mmap lock released, if we're allowed to. */
drm_gem_object_get(&bo->base);
- mmap_read_unlock(vmf->vma->vm_mm);
+
+ if (vmf->flags & FAULT_FLAG_VMA_LOCK)
+ vma_end_read(vmf->vma);
+ else
+ mmap_read_unlock(vmf->vma->vm_mm);
+
ret = blocking_page_setup(vmf, order, bo, page_offset, false);
drm_gem_object_put(&bo->base);
return ret;
@@ -770,7 +940,9 @@ static void panthor_gem_vm_close(struct vm_area_struct *vma)
dma_resv_lock(bo->base.resv, NULL);
if (refcount_dec_and_test(&bo->cmap.mmap_count)) {
- /* Nothing to do, pages are reclaimed lazily. */
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
}
dma_resv_unlock(bo->base.resv);
@@ -810,6 +982,7 @@ panthor_gem_alloc_object(u32 flags)
if (!bo)
return ERR_PTR(-ENOMEM);
+ bo->reclaim_state = PANTHOR_GEM_UNRECLAIMABLE;
bo->base.funcs = &panthor_gem_funcs;
bo->flags = flags;
mutex_init(&bo->label.lock);
@@ -968,6 +1141,7 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
struct sg_table *sgt;
struct scatterlist *sgl;
unsigned int count;
+ int ret;
/* Make sure the range is in bounds. */
if (offset + size < offset || offset + size > bo->base.size)
@@ -994,9 +1168,21 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
if (size == 0)
return 0;
- sgt = panthor_gem_get_dev_sgt(bo);
- if (IS_ERR(sgt))
- return PTR_ERR(sgt);
+ ret = dma_resv_lock_interruptible(bo->base.resv, NULL);
+ if (ret)
+ return ret;
+
+ /* If there's no pages, there's no point pulling those back, bail out early. */
+ if (!bo->backing.pages) {
+ ret = 0;
+ goto out_unlock;
+ }
+
+ sgt = panthor_gem_dev_map_get_sgt_locked(bo);
+ if (IS_ERR(sgt)) {
+ ret = PTR_ERR(sgt);
+ goto out_unlock;
+ }
for_each_sgtable_dma_sg(sgt, sgl, count) {
if (size == 0)
@@ -1040,7 +1226,11 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
dma_sync_single_for_cpu(dma_dev, paddr, len, DMA_FROM_DEVICE);
}
- return 0;
+ ret = 0;
+
+out_unlock:
+ dma_resv_unlock(bo->base.resv);
+ return ret;
}
/**
@@ -1050,11 +1240,13 @@ panthor_gem_sync(struct drm_gem_object *obj, u32 type,
*/
void panthor_kernel_bo_destroy(struct panthor_kernel_bo *bo)
{
+ struct panthor_device *ptdev;
struct panthor_vm *vm;
if (IS_ERR_OR_NULL(bo))
return;
+ ptdev = container_of(bo->obj->dev, struct panthor_device, base);
vm = bo->vm;
panthor_kernel_bo_vunmap(bo);
@@ -1062,6 +1254,8 @@ void panthor_kernel_bo_destroy(struct panthor_kernel_bo *bo)
to_panthor_bo(bo->obj)->exclusive_vm_root_gem != panthor_vm_root_gem(vm));
panthor_vm_unmap_range(vm, bo->va_node.start, bo->va_node.size);
panthor_vm_free_va(vm, &bo->va_node);
+ if (vm == panthor_fw_vm(ptdev))
+ panthor_gem_unpin(to_panthor_bo(bo->obj));
drm_gem_object_put(bo->obj);
panthor_vm_put(vm);
kfree(bo);
@@ -1110,6 +1304,12 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
kbo->obj = &bo->base;
+ if (vm == panthor_fw_vm(ptdev)) {
+ ret = panthor_gem_pin(bo);
+ if (ret)
+ goto err_put_obj;
+ }
+
panthor_gem_kernel_bo_set_label(kbo, name);
/* The system and GPU MMU page size might differ, which becomes a
@@ -1121,7 +1321,7 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
size = ALIGN(size, panthor_vm_page_size(vm));
ret = panthor_vm_alloc_va(vm, gpu_va, size, &kbo->va_node);
if (ret)
- goto err_put_obj;
+ goto err_unpin;
ret = panthor_vm_map_bo_range(vm, bo, 0, size, kbo->va_node.start, vm_map_flags);
if (ret)
@@ -1133,6 +1333,10 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
err_free_va:
panthor_vm_free_va(vm, &kbo->va_node);
+err_unpin:
+ if (vm == panthor_fw_vm(ptdev))
+ panthor_gem_unpin(bo);
+
err_put_obj:
drm_gem_object_put(&bo->base);
@@ -1141,6 +1345,241 @@ panthor_kernel_bo_create(struct panthor_device *ptdev, struct panthor_vm *vm,
return ERR_PTR(ret);
}
+static bool can_swap(void)
+{
+ return get_nr_swap_pages() > 0;
+}
+
+static bool can_block(struct shrink_control *sc)
+{
+ /* If direct reclaim is allowed, we can always block.
+ * If kswapd reclaim is allowed, we can block, but only if we're called
+ * by the kswapd thread.
+ */
+ return (sc->gfp_mask & __GFP_DIRECT_RECLAIM) ||
+ ((sc->gfp_mask & __GFP_KSWAPD_RECLAIM) && current_is_kswapd());
+}
+
+static unsigned long
+panthor_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc)
+{
+ struct panthor_device *ptdev = shrinker->private_data;
+ unsigned long count;
+
+ /* We currently don't have a flag to tell when the content of a
+ * BO can be discarded.
+ */
+ if (!can_swap())
+ return 0;
+
+ /* This is racy, but that's okay because the returned count is just a
+ * hint. That's also what MSM is doing (no atomic var, it's relying on
+ * the fact unsigned long access is usually atomic), so if it's good
+ * enough for them, it's good enough for us too.
+ */
+ count = ptdev->reclaim.unused.count;
+ count += ptdev->reclaim.mmapped.count;
+
+ if (can_block(sc))
+ count += ptdev->reclaim.gpu_mapped_count;
+
+ return count ? count : SHRINK_EMPTY;
+}
+
+static bool panthor_gem_try_evict_no_resv_wait(struct drm_gem_object *obj,
+ struct ww_acquire_ctx *ticket)
+{
+ /*
+ * Track last locked entry for unwinding locks in error and
+ * success paths
+ */
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
+ struct drm_gpuvm_bo *vm_bo, *last_locked = NULL;
+ enum panthor_gem_reclaim_state old_state;
+ int ret = 0;
+
+ /* To avoid potential lock ordering issue between bo_gpuva and
+ * mapping->i_mmap_rwsem, unmap the pages from CPU side before
+ * acquring the bo_gpuva lock. As the bo_resv lock is held, CPU
+ * page fault handler won't be able to map in the pages whilst
+ * eviction is in progress.
+ */
+ drm_vma_node_unmap(&bo->base.vma_node, bo->base.dev->anon_inode->i_mapping);
+
+ /* We take this lock when walking the list to prevent
+ * insertion/deletion.
+ */
+ /* We can only trylock in that path, because
+ * - allocation might happen while some of these locks are held
+ * - lock ordering is different in other paths
+ * vm_resv -> bo_resv -> bo_gpuva
+ * vs
+ * bo_resv -> bo_gpuva -> vm_resv
+ *
+ * If we fail to lock that's fine, we back off and will get
+ * back to it later.
+ */
+ if (!mutex_trylock(&bo->base.gpuva.lock))
+ return false;
+
+ drm_gem_for_each_gpuvm_bo(vm_bo, obj) {
+ struct dma_resv *resv = drm_gpuvm_resv(vm_bo->vm);
+
+ if (resv == obj->resv)
+ continue;
+
+ if (!dma_resv_trylock(resv)) {
+ ret = -EDEADLK;
+ goto out_unlock;
+ }
+
+ last_locked = vm_bo;
+ }
+
+ /* Update the state before trying to evict the buffer, if the state was
+ * updated to something that's harder to reclaim (higher value in the
+ * enum), skip it (will be processed when the relevant LRU is).
+ */
+ panthor_gem_update_reclaim_state_locked(bo, &old_state);
+ if (old_state < bo->reclaim_state) {
+ ret = -EAGAIN;
+ goto out_unlock;
+ }
+
+ /* Couldn't teardown the GPU mappings? Skip. */
+ ret = panthor_vm_evict_bo_mappings_locked(bo);
+ if (ret)
+ goto out_unlock;
+
+ /* If everything went fine, evict the object. */
+ panthor_gem_evict_locked(bo);
+
+out_unlock:
+ if (last_locked) {
+ drm_gem_for_each_gpuvm_bo(vm_bo, obj) {
+ struct dma_resv *resv = drm_gpuvm_resv(vm_bo->vm);
+
+ if (resv == obj->resv)
+ continue;
+
+ dma_resv_unlock(resv);
+
+ if (last_locked == vm_bo)
+ break;
+ }
+ }
+ mutex_unlock(&bo->base.gpuva.lock);
+
+ return ret == 0;
+}
+
+static bool panthor_gem_try_evict(struct drm_gem_object *obj,
+ struct ww_acquire_ctx *ticket)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(obj);
+
+ /* Wait was too long, skip. */
+ if (dma_resv_wait_timeout(obj->resv, DMA_RESV_USAGE_BOOKKEEP, false, 10) <= 0)
+ return false;
+
+ return panthor_gem_try_evict_no_resv_wait(&bo->base, ticket);
+}
+
+static unsigned long
+panthor_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
+{
+ struct panthor_device *ptdev = shrinker->private_data;
+ unsigned long remaining = 0;
+ unsigned long freed = 0;
+
+ if (!can_swap())
+ goto out;
+
+ freed += drm_gem_lru_scan(&ptdev->reclaim.unused,
+ sc->nr_to_scan - freed, &remaining,
+ panthor_gem_try_evict_no_resv_wait, NULL);
+ if (freed >= sc->nr_to_scan)
+ goto out;
+
+ freed += drm_gem_lru_scan(&ptdev->reclaim.mmapped,
+ sc->nr_to_scan - freed, &remaining,
+ panthor_gem_try_evict_no_resv_wait, NULL);
+ if (freed >= sc->nr_to_scan)
+ goto out;
+
+ if (!can_block(sc))
+ goto out;
+
+ freed += panthor_mmu_reclaim_priv_bos(ptdev, sc->nr_to_scan - freed,
+ &remaining, panthor_gem_try_evict);
+ if (freed >= sc->nr_to_scan)
+ goto out;
+
+ freed += drm_gem_lru_scan(&ptdev->reclaim.gpu_mapped_shared,
+ sc->nr_to_scan - freed, &remaining,
+ panthor_gem_try_evict, NULL);
+
+out:
+#ifdef CONFIG_DEBUG_FS
+ /* This is racy, but that's okay, because this is just debugfs
+ * reporting and doesn't need to be accurate.
+ */
+ ptdev->reclaim.nr_pages_reclaimed_on_last_scan = freed;
+#endif
+
+ /* If there are things to reclaim, try a couple times before giving up. */
+ if (!freed && remaining > 0 &&
+ atomic_inc_return(&ptdev->reclaim.retry_count) < 2)
+ return 0;
+
+ atomic_set(&ptdev->reclaim.retry_count, 0);
+
+ if (freed)
+ return freed;
+
+ /* There's nothing left to reclaim, or the resources are contended. Give up now. */
+ return SHRINK_STOP;
+}
+
+int panthor_gem_shrinker_init(struct panthor_device *ptdev)
+{
+ struct shrinker *shrinker;
+ int ret;
+
+ ret = drmm_mutex_init(&ptdev->base, &ptdev->reclaim.lock);
+ if (ret)
+ return ret;
+
+ INIT_LIST_HEAD(&ptdev->reclaim.vms);
+ drm_gem_lru_init(&ptdev->reclaim.unused, &ptdev->reclaim.lock);
+ drm_gem_lru_init(&ptdev->reclaim.mmapped, &ptdev->reclaim.lock);
+ drm_gem_lru_init(&ptdev->reclaim.gpu_mapped_shared, &ptdev->reclaim.lock);
+ ptdev->reclaim.gpu_mapped_count = 0;
+
+ /* Teach lockdep about lock ordering wrt. shrinker: */
+ fs_reclaim_acquire(GFP_KERNEL);
+ might_lock(&ptdev->reclaim.lock);
+ fs_reclaim_release(GFP_KERNEL);
+
+ shrinker = shrinker_alloc(0, "drm-panthor-gem");
+ if (!shrinker)
+ return -ENOMEM;
+
+ shrinker->count_objects = panthor_gem_shrinker_count;
+ shrinker->scan_objects = panthor_gem_shrinker_scan;
+ shrinker->private_data = ptdev;
+ ptdev->reclaim.shrinker = shrinker;
+
+ shrinker_register(shrinker);
+ return 0;
+}
+
+void panthor_gem_shrinker_unplug(struct panthor_device *ptdev)
+{
+ if (ptdev->reclaim.shrinker)
+ shrinker_free(ptdev->reclaim.shrinker);
+}
+
#ifdef CONFIG_DEBUG_FS
struct gem_size_totals {
size_t size;
@@ -1184,6 +1623,7 @@ static void panthor_gem_debugfs_bo_print(struct panthor_gem_object *bo,
struct seq_file *m,
struct gem_size_totals *totals)
{
+ enum panthor_gem_reclaim_state reclaim_state = bo->reclaim_state;
unsigned int refcount = kref_read(&bo->base.refcount);
char creator_info[32] = {};
size_t resident_size;
@@ -1219,6 +1659,8 @@ static void panthor_gem_debugfs_bo_print(struct panthor_gem_object *bo,
totals->size += bo->base.size;
totals->resident += resident_size;
+ if (reclaim_state != PANTHOR_GEM_UNRECLAIMABLE)
+ totals->reclaimable += resident_size;
}
static void panthor_gem_debugfs_print_bos(struct panthor_device *ptdev,
@@ -1259,10 +1701,42 @@ static struct drm_info_list panthor_gem_debugfs_list[] = {
{ "gems", panthor_gem_show_bos, 0, NULL },
};
+static int shrink_get(void *data, u64 *val)
+{
+ struct panthor_device *ptdev =
+ container_of(data, struct panthor_device, base);
+
+ *val = ptdev->reclaim.nr_pages_reclaimed_on_last_scan;
+ return 0;
+}
+
+static int shrink_set(void *data, u64 val)
+{
+ struct panthor_device *ptdev =
+ container_of(data, struct panthor_device, base);
+ struct shrink_control sc = {
+ .gfp_mask = GFP_KERNEL,
+ .nr_to_scan = val,
+ };
+
+ fs_reclaim_acquire(GFP_KERNEL);
+ if (ptdev->reclaim.shrinker)
+ panthor_gem_shrinker_scan(ptdev->reclaim.shrinker, &sc);
+ fs_reclaim_release(GFP_KERNEL);
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(panthor_gem_debugfs_shrink_fops,
+ shrink_get, shrink_set,
+ "0x%08llx\n");
+
void panthor_gem_debugfs_init(struct drm_minor *minor)
{
drm_debugfs_create_files(panthor_gem_debugfs_list,
ARRAY_SIZE(panthor_gem_debugfs_list),
minor->debugfs_root, minor);
+ debugfs_create_file("shrink", 0600, minor->debugfs_root,
+ minor->dev, &panthor_gem_debugfs_shrink_fops);
}
#endif
diff --git a/drivers/gpu/drm/panthor/panthor_gem.h b/drivers/gpu/drm/panthor/panthor_gem.h
index c0a18dca732c..ae0491d0b121 100644
--- a/drivers/gpu/drm/panthor/panthor_gem.h
+++ b/drivers/gpu/drm/panthor/panthor_gem.h
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 or MIT */
/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
/* Copyright 2023 Collabora ltd. */
+/* Copyright 2025 ARM Limited. All rights reserved. */
#ifndef __PANTHOR_GEM_H__
#define __PANTHOR_GEM_H__
@@ -93,6 +94,65 @@ struct panthor_gem_dev_map {
struct sg_table *sgt;
};
+/**
+ * enum panthor_gem_reclaim_state - Reclaim state of a GEM object
+ *
+ * This is defined in descending reclaimability order and some part
+ * of the code depends on that.
+ */
+enum panthor_gem_reclaim_state {
+ /**
+ * @PANTHOR_GEM_UNUSED: GEM is currently unused
+ *
+ * This can happen when the GEM was previously vmap-ed, mmap-ed,
+ * and/or GPU mapped and got unmapped. Because pages are lazily
+ * returned to the shmem layer, we want to keep a list of such
+ * BOs, because they should be fairly easy to reclaim (no need
+ * to wait for GPU to be done, and no need to tear down user
+ * mappings either).
+ */
+ PANTHOR_GEM_UNUSED,
+
+ /**
+ * @PANTHOR_GEM_MMAPPED: GEM is currently mmap-ed
+ *
+ * When a GEM has pages allocated and the mmap_count is > 0, the
+ * GEM is placed in the mmapped list. This comes right after
+ * unused because we can relatively easily tear down user mappings.
+ */
+ PANTHOR_GEM_MMAPPED,
+
+ /**
+ * @PANTHOR_GEM_GPU_MAPPED_SINGLE_VM: GEM is GPU mapped to only one VM
+ *
+ * When a GEM is mapped to a single VM, reclaim requests have more
+ * chances to succeed, because we only need to synchronize against
+ * a single GPU context. This is more annoying than reclaiming
+ * mmap-ed pages still, because we have to wait for in-flight jobs
+ * to land, and we might not be able to acquire all necessary locks
+ * at reclaim time either.
+ */
+ PANTHOR_GEM_GPU_MAPPED_SINGLE_VM,
+
+ /**
+ * @PANTHOR_GEM_GPU_MAPPED_MULTI_VM: GEM is GPU mapped to multiple VMs
+ *
+ * Like PANTHOR_GEM_GPU_MAPPED_SINGLE_VM, but the synchronization across
+ * VMs makes such BOs harder to reclaim.
+ */
+ PANTHOR_GEM_GPU_MAPPED_MULTI_VM,
+
+ /**
+ * @PANTHOR_GEM_UNRECLAIMABLE: GEM can't be reclaimed
+ *
+ * Happens when the GEM memory is pinned. It's also the state all GEM
+ * objects start in, because no memory is allocated until explicitly
+ * requested by a CPU or GPU map, meaning there's nothing to reclaim
+ * until such an allocation happens.
+ */
+ PANTHOR_GEM_UNRECLAIMABLE,
+};
+
/**
* struct panthor_gem_object - Driver specific GEM object.
*/
@@ -109,6 +169,9 @@ struct panthor_gem_object {
/** @dmap: Device mapping state */
struct panthor_gem_dev_map dmap;
+ /** @reclaim_state: Cached reclaim state */
+ enum panthor_gem_reclaim_state reclaim_state;
+
/**
* @exclusive_vm_root_gem: Root GEM of the exclusive VM this GEM object
* is attached to.
@@ -190,6 +253,11 @@ struct sg_table *
panthor_gem_get_dev_sgt(struct panthor_gem_object *bo);
int panthor_gem_pin(struct panthor_gem_object *bo);
void panthor_gem_unpin(struct panthor_gem_object *bo);
+int panthor_gem_swapin_locked(struct panthor_gem_object *bo);
+void panthor_gem_update_reclaim_state_locked(struct panthor_gem_object *bo,
+ enum panthor_gem_reclaim_state *old_state);
+int panthor_gem_shrinker_init(struct panthor_device *ptdev);
+void panthor_gem_shrinker_unplug(struct panthor_device *ptdev);
void panthor_gem_bo_set_label(struct drm_gem_object *obj, const char *label);
void panthor_gem_kernel_bo_set_label(struct panthor_kernel_bo *bo, const char *label);
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
index b78f694ca92b..fa8b31df85c9 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.c
+++ b/drivers/gpu/drm/panthor/panthor_mmu.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0 or MIT
/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
/* Copyright 2023 Collabora ltd. */
+/* Copyright 2025 ARM Limited. All rights reserved. */
#include <drm/drm_debugfs.h>
#include <drm/drm_drv.h>
@@ -131,6 +132,9 @@ struct panthor_vma {
* Only map related flags are accepted.
*/
u32 flags;
+
+ /** @evicted: True if the VMA has been evicted. */
+ bool evicted;
};
/**
@@ -198,6 +202,9 @@ struct panthor_vm_op_ctx {
* to allocate in ::run_job().
*/
struct sg_table *sgt;
+
+ /** @map.bo: the BO being mapped. */
+ struct panthor_gem_object *bo;
} map;
};
@@ -380,6 +387,18 @@ struct panthor_vm {
/** @locked_region.size: Size of the locked region. */
u64 size;
} locked_region;
+
+ /** @reclaim: Fields related to BO reclaim. */
+ struct {
+ /** @reclaim.lru: LRU of BOs that are only mapped to this VM. */
+ struct drm_gem_lru lru;
+
+ /**
+ * @reclaim.lru_node: Node used to insert the VM in
+ * panthor_device::reclaim::vms.
+ */
+ struct list_head lru_node;
+ } reclaim;
};
/**
@@ -684,6 +703,16 @@ int panthor_vm_active(struct panthor_vm *vm)
if (refcount_inc_not_zero(&vm->as.active_cnt))
goto out_dev_exit;
+ /* As soon as active is called, we place the VM at the end of the VM LRU.
+ * If something fails after that, the only downside is that this VM that
+ * never became active in the first place will be reclaimed last, but
+ * that's an acceptable trade-off.
+ */
+ mutex_lock(&ptdev->reclaim.lock);
+ if (vm->reclaim.lru.count)
+ list_move_tail(&vm->reclaim.lru_node, &ptdev->reclaim.vms);
+ mutex_unlock(&ptdev->reclaim.lock);
+
/* Make sure we don't race with lock/unlock_region() calls
* happening around VM bind operations.
*/
@@ -1079,7 +1108,15 @@ static void panthor_vm_bo_free(struct drm_gpuvm_bo *vm_bo)
{
struct panthor_gem_object *bo = to_panthor_bo(vm_bo->obj);
- panthor_gem_unpin(bo);
+ /* We couldn't call this when we unlinked, because the resv lock can't
+ * be taken in the dma signalling path, so call it now.
+ */
+ dma_resv_lock(bo->base.resv, NULL);
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
+ dma_resv_unlock(bo->base.resv);
+
kfree(vm_bo);
}
@@ -1088,6 +1125,16 @@ static void panthor_vm_cleanup_op_ctx(struct panthor_vm_op_ctx *op_ctx,
{
u32 remaining_pt_count = op_ctx->rsvd_page_tables.count -
op_ctx->rsvd_page_tables.ptr;
+ u32 op_type = op_ctx->flags & DRM_PANTHOR_VM_BIND_OP_TYPE_MASK;
+
+ /* If this is a map operation and no BO is attached, we're being called
+ * from vm_bo_validate() and we can't acquire the VM lock because it's
+ * already held. In that case, we just skip the deferred vm_bo cleanup,
+ * which is fine, because the vm_bo validation is not calling
+ * drm_gpuvm_bo_put_deferred().
+ */
+ bool skip_deferred_cleanup = op_type == DRM_PANTHOR_VM_BIND_OP_TYPE_MAP &&
+ !op_ctx->map.bo;
if (remaining_pt_count) {
kmem_cache_free_bulk(pt_cache, remaining_pt_count,
@@ -1100,10 +1147,16 @@ static void panthor_vm_cleanup_op_ctx(struct panthor_vm_op_ctx *op_ctx,
if (op_ctx->map.vm_bo)
drm_gpuvm_bo_put_deferred(op_ctx->map.vm_bo);
+ if (op_ctx->map.bo) {
+ panthor_gem_unpin(op_ctx->map.bo);
+ drm_gem_object_put(&op_ctx->map.bo->base);
+ }
+
for (u32 i = 0; i < ARRAY_SIZE(op_ctx->preallocated_vmas); i++)
kfree(op_ctx->preallocated_vmas[i]);
- drm_gpuvm_bo_deferred_cleanup(&vm->base);
+ if (!skip_deferred_cleanup)
+ drm_gpuvm_bo_deferred_cleanup(&vm->base);
}
static void
@@ -1259,18 +1312,17 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
if (ret)
goto err_cleanup;
+ drm_gem_object_get(&bo->base);
+ op_ctx->map.bo = bo;
+
sgt = panthor_gem_get_dev_sgt(bo);
if (IS_ERR(sgt)) {
- panthor_gem_unpin(bo);
ret = PTR_ERR(sgt);
goto err_cleanup;
}
- op_ctx->map.sgt = sgt;
-
preallocated_vm_bo = drm_gpuvm_bo_create(&vm->base, &bo->base);
if (!preallocated_vm_bo) {
- panthor_gem_unpin(bo);
ret = -ENOMEM;
goto err_cleanup;
}
@@ -1289,6 +1341,13 @@ static int panthor_vm_prepare_map_op_ctx(struct panthor_vm_op_ctx *op_ctx,
dma_resv_unlock(panthor_vm_resv(vm));
}
+ /* And finally update the BO state. */
+ dma_resv_lock(bo->base.resv, NULL);
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
+ dma_resv_unlock(bo->base.resv);
+
return 0;
err_cleanup:
@@ -1876,6 +1935,10 @@ static void panthor_vm_free(struct drm_gpuvm *gpuvm)
struct panthor_vm *vm = container_of(gpuvm, struct panthor_vm, base);
struct panthor_device *ptdev = vm->ptdev;
+ mutex_lock(&ptdev->reclaim.lock);
+ list_del_init(&vm->reclaim.lru_node);
+ mutex_unlock(&ptdev->reclaim.lock);
+
mutex_lock(&vm->heaps.lock);
if (drm_WARN_ON(&ptdev->base, vm->heaps.pool))
panthor_heap_pool_destroy(vm->heaps.pool);
@@ -2089,7 +2152,7 @@ static int panthor_gpuva_sm_step_map(struct drm_gpuva_op *op, void *priv)
panthor_vma_init(vma, op_ctx->flags & PANTHOR_VM_MAP_FLAGS);
ret = panthor_vm_map_pages(vm, op->map.va.addr, flags_to_prot(vma->flags),
- op_ctx->map.sgt, op->map.gem.offset,
+ op_ctx->map.bo->dmap.sgt, op->map.gem.offset,
op->map.va.range);
if (ret) {
panthor_vm_op_ctx_return_vma(op_ctx, vma);
@@ -2173,21 +2236,27 @@ static int panthor_gpuva_sm_step_remap(struct drm_gpuva_op *op,
* atomicity. panthor_vm_lock_region() bails out early if the new region
* is already part of the locked region, so no need to do this check here.
*/
- panthor_vm_lock_region(vm, unmap_start, unmap_range);
- panthor_vm_unmap_pages(vm, unmap_start, unmap_range);
+ if (!unmap_vma->evicted) {
+ panthor_vm_lock_region(vm, unmap_start, unmap_range);
+ panthor_vm_unmap_pages(vm, unmap_start, unmap_range);
+ }
if (op->remap.prev) {
struct panthor_gem_object *bo = to_panthor_bo(op->remap.prev->gem.obj);
u64 offset = op->remap.prev->gem.offset + unmap_start - op->remap.prev->va.addr;
u64 size = op->remap.prev->va.addr + op->remap.prev->va.range - unmap_start;
- ret = panthor_vm_map_pages(vm, unmap_start, flags_to_prot(unmap_vma->flags),
- bo->dmap.sgt, offset, size);
- if (ret)
- return ret;
+ if (!unmap_vma->evicted) {
+ ret = panthor_vm_map_pages(vm, unmap_start,
+ flags_to_prot(unmap_vma->flags),
+ bo->dmap.sgt, offset, size);
+ if (ret)
+ return ret;
+ }
prev_vma = panthor_vm_op_ctx_get_vma(op_ctx);
panthor_vma_init(prev_vma, unmap_vma->flags);
+ prev_vma->evicted = unmap_vma->evicted;
}
if (op->remap.next) {
@@ -2195,13 +2264,17 @@ static int panthor_gpuva_sm_step_remap(struct drm_gpuva_op *op,
u64 addr = op->remap.next->va.addr;
u64 size = unmap_start + unmap_range - op->remap.next->va.addr;
- ret = panthor_vm_map_pages(vm, addr, flags_to_prot(unmap_vma->flags),
- bo->dmap.sgt, op->remap.next->gem.offset, size);
- if (ret)
- return ret;
+ if (!unmap_vma->evicted) {
+ ret = panthor_vm_map_pages(vm, addr, flags_to_prot(unmap_vma->flags),
+ bo->dmap.sgt, op->remap.next->gem.offset,
+ size);
+ if (ret)
+ return ret;
+ }
next_vma = panthor_vm_op_ctx_get_vma(op_ctx);
panthor_vma_init(next_vma, unmap_vma->flags);
+ next_vma->evicted = unmap_vma->evicted;
}
drm_gpuva_remap(prev_vma ? &prev_vma->base : NULL,
@@ -2231,19 +2304,230 @@ static int panthor_gpuva_sm_step_unmap(struct drm_gpuva_op *op,
struct panthor_vma *unmap_vma = container_of(op->unmap.va, struct panthor_vma, base);
struct panthor_vm *vm = priv;
- panthor_vm_unmap_pages(vm, unmap_vma->base.va.addr,
- unmap_vma->base.va.range);
+ if (!unmap_vma->evicted) {
+ panthor_vm_unmap_pages(vm, unmap_vma->base.va.addr,
+ unmap_vma->base.va.range);
+ }
+
drm_gpuva_unmap(&op->unmap);
panthor_vma_unlink(unmap_vma);
return 0;
}
+void panthor_vm_update_bo_reclaim_lru_locked(struct panthor_gem_object *bo)
+{
+ struct panthor_device *ptdev = container_of(bo->base.dev, struct panthor_device, base);
+ struct panthor_vm *vm = NULL;
+ struct drm_gpuvm_bo *vm_bo;
+
+ dma_resv_assert_held(bo->base.resv);
+ lockdep_assert_held(&bo->base.gpuva.lock);
+
+ drm_gem_for_each_gpuvm_bo(vm_bo, &bo->base) {
+ if (vm_bo->evicted)
+ continue;
+
+ /* We're only supposed to have one non-evicted vm_bo in the list if we get
+ * there.
+ */
+ drm_WARN_ON(&ptdev->base, vm);
+ vm = container_of(vm_bo->vm, struct panthor_vm, base);
+
+ mutex_lock(&ptdev->reclaim.lock);
+ drm_gem_lru_move_tail_locked(&vm->reclaim.lru, &bo->base);
+ if (list_empty(&vm->reclaim.lru_node))
+ list_move(&vm->reclaim.lru_node, &ptdev->reclaim.vms);
+ mutex_unlock(&ptdev->reclaim.lock);
+ }
+}
+
+int panthor_vm_evict_bo_mappings_locked(struct panthor_gem_object *bo)
+{
+ struct drm_gpuvm_bo *vm_bo;
+ int ret = 0;
+
+ drm_gem_for_each_gpuvm_bo(vm_bo, &bo->base) {
+ struct panthor_vm *vm = container_of(vm_bo->vm, struct panthor_vm, base);
+ struct drm_gpuva *va;
+
+ /* Skip already evicted GPU mappings. */
+ if (vm_bo->evicted)
+ continue;
+
+ if (!mutex_trylock(&vm->op_lock))
+ return -EDEADLK;
+
+ drm_gpuvm_bo_evict(vm_bo, true);
+ drm_gpuvm_bo_for_each_va(va, vm_bo) {
+ struct panthor_vma *vma = container_of(va, struct panthor_vma, base);
+
+ if (vma->evicted)
+ continue;
+
+ /* If something fail in the middle of a VM_BO eviction, the VM_BO
+ * is considered fully evicted, but some of its VMAs might still be
+ * active. That's okay because the pages won't be released if this
+ * function returns an error.
+ *
+ * On the next job targeting this VM, the partially evicted VM_BO
+ * will be validated, causing all its evicted VMAs to be repopulated
+ * before the job runs. So no GPU fault expected.
+ */
+ ret = panthor_vm_lock_region(vm, va->va.addr, va->va.range);
+ if (ret)
+ break;
+
+ panthor_vm_unmap_pages(vm, va->va.addr, va->va.range);
+ panthor_vm_unlock_region(vm);
+ vma->evicted = true;
+ }
+
+ mutex_unlock(&vm->op_lock);
+
+ if (ret)
+ break;
+ }
+
+ return ret;
+}
+
+static struct panthor_vma *select_evicted_vma(struct drm_gpuvm_bo *vm_bo,
+ struct panthor_vm_op_ctx *op_ctx)
+{
+ struct panthor_vm *vm = container_of(vm_bo->vm, struct panthor_vm, base);
+ struct panthor_vma *first_evicted_vma = NULL;
+ struct drm_gpuva *va;
+
+ /* Take op_lock to protect against va insertion/removal. */
+ mutex_lock(&vm->op_lock);
+ drm_gpuvm_bo_for_each_va(va, vm_bo) {
+ struct panthor_vma *vma = container_of(va, struct panthor_vma, base);
+
+ if (vma->evicted) {
+ first_evicted_vma = vma;
+ panthor_vm_init_op_ctx(op_ctx, va->va.range, va->va.addr, vma->flags);
+ op_ctx->map.bo_offset = va->gem.offset;
+ break;
+ }
+ }
+ mutex_unlock(&vm->op_lock);
+
+ return first_evicted_vma;
+}
+
+static int remap_evicted_vma(struct drm_gpuvm_bo *vm_bo,
+ struct panthor_vma *evicted_vma,
+ struct panthor_vm_op_ctx *op_ctx)
+{
+ struct panthor_vm *vm = container_of(vm_bo->vm, struct panthor_vm, base);
+ struct panthor_gem_object *bo = to_panthor_bo(vm_bo->obj);
+ struct drm_gpuva *va;
+ bool found = false;
+ int ret;
+
+ ret = panthor_vm_op_ctx_prealloc_pts(op_ctx);
+ if (ret)
+ goto out_cleanup;
+
+ /* Take op_lock to protect against va insertion/removal. Note that the
+ * evicted_vma selection was done with the same lock held, but we had
+ * to release it so we can allocate PTs, because this very same lock
+ * is taken in a DMA-signalling path.
+ */
+ mutex_lock(&vm->op_lock);
+ drm_gpuvm_bo_for_each_va(va, vm_bo) {
+ struct panthor_vma *vma = container_of(va, struct panthor_vma, base);
+
+ if (vma != evicted_vma)
+ continue;
+
+ /* Because we had to release the lock between the evicted_vma selection
+ * and its repopulation, we can't rely solely on pointer equality (the
+ * VMA might have been freed and a new one allocated at the same address).
+ * If the evicted bit is still set, we're sure it's our VMA, because
+ * population/eviction is serialized with the BO resv lock.
+ */
+ if (vma->evicted)
+ found = true;
+
+ break;
+ }
+
+ if (found) {
+ vm->op_ctx = op_ctx;
+ ret = panthor_vm_lock_region(vm, evicted_vma->base.va.addr,
+ evicted_vma->base.va.range);
+ if (!ret) {
+ ret = panthor_vm_map_pages(vm, evicted_vma->base.va.addr,
+ flags_to_prot(evicted_vma->flags),
+ bo->dmap.sgt,
+ evicted_vma->base.gem.offset,
+ evicted_vma->base.va.range);
+ if (!ret)
+ evicted_vma->evicted = false;
+
+ panthor_vm_unlock_region(vm);
+ }
+
+ vm->op_ctx = NULL;
+ }
+
+ mutex_unlock(&vm->op_lock);
+
+out_cleanup:
+ panthor_vm_cleanup_op_ctx(op_ctx, vm);
+ return ret;
+}
+
+static int panthor_vm_restore_vmas(struct drm_gpuvm_bo *vm_bo)
+{
+ struct panthor_vm *vm = container_of(vm_bo->vm, struct panthor_vm, base);
+ struct panthor_gem_object *bo = to_panthor_bo(vm_bo->obj);
+ struct panthor_vm_op_ctx op_ctx;
+
+ if (drm_WARN_ON_ONCE(&vm->ptdev->base, !bo->dmap.sgt))
+ return -EINVAL;
+
+ for (struct panthor_vma *vma = select_evicted_vma(vm_bo, &op_ctx);
+ vma; vma = select_evicted_vma(vm_bo, &op_ctx)) {
+ int ret;
+
+ ret = remap_evicted_vma(vm_bo, vma, &op_ctx);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int panthor_vm_bo_validate(struct drm_gpuvm_bo *vm_bo,
+ struct drm_exec *exec)
+{
+ struct panthor_gem_object *bo = to_panthor_bo(vm_bo->obj);
+ int ret;
+
+ ret = panthor_gem_swapin_locked(bo);
+ if (ret)
+ return ret;
+
+ ret = panthor_vm_restore_vmas(vm_bo);
+ if (ret)
+ return ret;
+
+ drm_gpuvm_bo_evict(vm_bo, false);
+ mutex_lock(&bo->base.gpuva.lock);
+ panthor_gem_update_reclaim_state_locked(bo, NULL);
+ mutex_unlock(&bo->base.gpuva.lock);
+ return 0;
+}
+
static const struct drm_gpuvm_ops panthor_gpuvm_ops = {
.vm_free = panthor_vm_free,
.vm_bo_free = panthor_vm_bo_free,
.sm_step_map = panthor_gpuva_sm_step_map,
.sm_step_remap = panthor_gpuva_sm_step_remap,
.sm_step_unmap = panthor_gpuva_sm_step_unmap,
+ .vm_bo_validate = panthor_vm_bo_validate,
};
/**
@@ -2458,6 +2742,8 @@ panthor_vm_create(struct panthor_device *ptdev, bool for_mcu,
vm->kernel_auto_va.start = auto_kernel_va_start;
vm->kernel_auto_va.end = vm->kernel_auto_va.start + auto_kernel_va_size - 1;
+ drm_gem_lru_init(&vm->reclaim.lru, &ptdev->reclaim.lock);
+ INIT_LIST_HEAD(&vm->reclaim.lru_node);
INIT_LIST_HEAD(&vm->node);
INIT_LIST_HEAD(&vm->as.lru_node);
vm->as.id = -1;
@@ -2805,7 +3091,78 @@ int panthor_vm_prepare_mapped_bos_resvs(struct drm_exec *exec, struct panthor_vm
if (ret)
return ret;
- return drm_gpuvm_prepare_objects(&vm->base, exec, slot_count);
+ ret = drm_gpuvm_prepare_objects(&vm->base, exec, slot_count);
+ if (ret)
+ return ret;
+
+ return drm_gpuvm_validate(&vm->base, exec);
+}
+
+unsigned long
+panthor_mmu_reclaim_priv_bos(struct panthor_device *ptdev,
+ unsigned int nr_to_scan, unsigned long *remaining,
+ bool (*shrink)(struct drm_gem_object *,
+ struct ww_acquire_ctx *))
+{
+ unsigned long freed = 0;
+ LIST_HEAD(remaining_vms);
+ LIST_HEAD(vms);
+
+ mutex_lock(&ptdev->reclaim.lock);
+ list_splice_init(&ptdev->reclaim.vms, &vms);
+
+ while (freed < nr_to_scan) {
+ struct panthor_vm *vm;
+
+ vm = list_first_entry_or_null(&vms, typeof(*vm),
+ reclaim.lru_node);
+ if (!vm)
+ break;
+
+ if (!kref_get_unless_zero(&vm->base.kref)) {
+ list_del_init(&vm->reclaim.lru_node);
+ continue;
+ }
+
+ mutex_unlock(&ptdev->reclaim.lock);
+
+ freed += drm_gem_lru_scan(&vm->reclaim.lru, nr_to_scan - freed,
+ remaining, shrink, NULL);
+
+ mutex_lock(&ptdev->reclaim.lock);
+
+ /* If the VM is still in the temporary list, remove it so we
+ * can proceed with the next VM.
+ */
+ if (vm == list_first_entry_or_null(&vms, typeof(*vm), reclaim.lru_node)) {
+ list_del_init(&vm->reclaim.lru_node);
+
+ /* Keep the VM around if there are still things to
+ * reclaim, so we can preserve the LRU order when
+ * re-inserting in ptdev->reclaim.vms at the end.
+ */
+ if (vm->reclaim.lru.count > 0)
+ list_add_tail(&vm->reclaim.lru_node, &remaining_vms);
+ }
+
+ mutex_unlock(&ptdev->reclaim.lock);
+
+ panthor_vm_put(vm);
+
+ mutex_lock(&ptdev->reclaim.lock);
+ }
+
+ /* Re-insert VMs with remaining data to reclaim at the beginning of
+ * the LRU. Note that any activeness change on the VM that happened
+ * while we were reclaiming would have moved the VM out of our
+ * temporary [remaining_]vms list, meaning anything we re-insert here
+ * preserves the LRU order.
+ */
+ list_splice_tail(&vms, &remaining_vms);
+ list_splice(&remaining_vms, &ptdev->reclaim.vms);
+ mutex_unlock(&ptdev->reclaim.lock);
+
+ return freed;
}
/**
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.h b/drivers/gpu/drm/panthor/panthor_mmu.h
index 0e268fdfdb2f..3522fbbce369 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.h
+++ b/drivers/gpu/drm/panthor/panthor_mmu.h
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 or MIT */
/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
/* Copyright 2023 Collabora ltd. */
+/* Copyright 2025 ARM Limited. All rights reserved. */
#ifndef __PANTHOR_MMU_H__
#define __PANTHOR_MMU_H__
@@ -46,6 +47,13 @@ struct panthor_vm *panthor_vm_create(struct panthor_device *ptdev, bool for_mcu,
u64 kernel_auto_va_start,
u64 kernel_auto_va_size);
+void panthor_vm_update_bo_reclaim_lru_locked(struct panthor_gem_object *bo);
+int panthor_vm_evict_bo_mappings_locked(struct panthor_gem_object *bo);
+unsigned long
+panthor_mmu_reclaim_priv_bos(struct panthor_device *ptdev,
+ unsigned int nr_to_scan, unsigned long *remaining,
+ bool (*shrink)(struct drm_gem_object *,
+ struct ww_acquire_ctx *));
int panthor_vm_prepare_mapped_bos_resvs(struct drm_exec *exec,
struct panthor_vm *vm,
u32 slot_count);
--
2.53.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH v7 00/10] drm/panthor: Add a GEM shrinker
2026-04-01 13:48 [PATCH v7 00/10] drm/panthor: Add a GEM shrinker Boris Brezillon
` (9 preceding siblings ...)
2026-04-01 13:48 ` [PATCH v7 10/10] drm/panthor: Add a GEM shrinker Boris Brezillon
@ 2026-04-03 11:54 ` Boris Brezillon
10 siblings, 0 replies; 14+ messages in thread
From: Boris Brezillon @ 2026-04-03 11:54 UTC (permalink / raw)
To: Boris Brezillon, Steven Price, Liviu Dudau, Adrián Larumbe
Cc: dri-devel, David Airlie, Simona Vetter, Akash Goel, Rob Clark,
Sean Paul, Konrad Dybcio, Akhil P Oommen, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, Dmitry Osipenko, Chris Diamand,
Danilo Krummrich, Matthew Brost, Thomas Hellström,
Alice Ryhl, Chia-I Wu, kernel
On Wed, 1 Apr 2026 15:48:44 +0200
Boris Brezillon <boris.brezillon@collabora.com> wrote:
> Hello,
>
> This is an attempt at adding a GEM shrinker to panthor so the system
> can finally reclaim GPU memory.
>
> This implementation is losely based on the MSM shrinker (which is why
> I added the MSM maintainers in Cc), and it's relying on the drm_gpuvm
> eviction/validation infrastructure.
>
> I've only done very basic IGT-based [1] and chromium-based (opening
> a lot of tabs on Aquarium until the system starts reclaiming+swapping
> out GPU buffers) testing, but I'm posting this early so I can get
> preliminary feedback on the implementation. If someone knows about
> better tools/ways to test the shrinker, please let me know.
>
> A few words about some design/implementation choices:
> - No MADVISE support because I want to see if we can live with just
> transparent reclaim
> - We considered basing this implementation on the generic shrinker work
> started by Dmitry [2], but
> 1. with the activeness/idleness tracking happening at the VM
> granularity, having per-BO LRUs would caused a lot of
> list_move()s that are not really needed (the VM as a whole
> become active/idle, we can track individual BOs)
> 2. Thomas Zimmermann recently suggested that we should have our
> own GEM implementation instead of trying to add this extra reclaim
> complexity to gem-shmem. There are some plans to create a
> gem-uma (Unified Memory Architecture) lib that would do more
> than gem-shmem but in a way that doesn't force all its users
> to pay the overhead (size overhead of the gem object, mostly)
> for features they don't use. Patch "Part ways with
> drm_gem_shmem_object" is showing what this component-based lib
> API could look like if it were to be extracted
> - At the moment we only support swapout, but we could add an
> extra flag to specify when buffer content doesn't need to be
> preserved to avoid the swapout/swapin dance. First candidate for
> this DISCARD_ON_RECLAIM flag would probably be the tiler heap chunks.
> - Reclaim uses _try_lock() all the way because of the various lock order
> inversions between the reclaim path and submission paths. That means
> we don't try very hard to reclaim hot GPU buffers, but the locking is
> such a mess that I don't really see a better option to be honest.
>
>
> Changes in v2:
> - No fundamental changes in this v2, since the feedback I got were more
> focused on bugs than the overall approach. Check the changelog in each
> patch for more details.
>
> Changes in v3:
> - Mostly fixes (see the changelog in each patch)
>
> Changes in v4:
> - Only fixes (see the changelog in each patch)
>
> Changes in v5:
> - Only fixes (see the changelog in each patch)
>
> Changes in v6:
> - Fix huge fault handling and various other things reported by Adrian
> and the review bot (see the changelog in each patch for more)
>
> Changes in v7:
> - Fix a deadlock reported by lockdep
> - Add a commit to remove an unused field
>
> Regards,
>
> Boris
>
> [1]https://gitlab.freedesktop.org/bbrezillon/igt-gpu-tools/-/commit/fc76934a5579767d2aabe787d40e38a17c3f4ea4
> [2]https://lkml.org/lkml/2024/1/5/665
>
> Akash Goel (1):
> drm/panthor: Add a GEM shrinker
>
> Boris Brezillon (9):
> drm/gem: Consider GEM object reclaimable if shrinking fails
> drm/panthor: Remove unused panthor_vm_op_ctx::map::new_vma field
> drm/panthor: Move panthor_gems_debugfs_init() to panthor_gem.c
> drm/panthor: Group panthor_kernel_bo_xxx() helpers
> drm/panthor: Don't call drm_gpuvm_bo_extobj_add() if the object is
> private
> drm/panthor: Part ways with drm_gem_shmem_object
> drm/panthor: Lazily allocate pages on mmap()
> drm/panthor: Split panthor_vm_prepare_map_op_ctx() to prepare for
> reclaim
> drm/panthor: Track the number of mmap on a BO
Queued to drm-misc-next.
>
> drivers/gpu/drm/drm_gem.c | 10 +
> drivers/gpu/drm/panthor/Kconfig | 1 -
> drivers/gpu/drm/panthor/panthor_device.c | 11 +-
> drivers/gpu/drm/panthor/panthor_device.h | 73 ++
> drivers/gpu/drm/panthor/panthor_drv.c | 33 +-
> drivers/gpu/drm/panthor/panthor_fw.c | 16 +-
> drivers/gpu/drm/panthor/panthor_gem.c | 1453 ++++++++++++++++++----
> drivers/gpu/drm/panthor/panthor_gem.h | 136 +-
> drivers/gpu/drm/panthor/panthor_mmu.c | 512 ++++++--
> drivers/gpu/drm/panthor/panthor_mmu.h | 8 +
> drivers/gpu/drm/panthor/panthor_sched.c | 9 +-
> 11 files changed, 1923 insertions(+), 339 deletions(-)
>
^ permalink raw reply [flat|nested] 14+ messages in thread