* [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync
@ 2026-11-17 12:50 Natalie Vock
2026-11-17 12:50 ` [PATCH v2 01/10] drm/amdgpu: Remove unused parameter from amdgpu_vm_pte_fragment Natalie Vock
` (9 more replies)
0 siblings, 10 replies; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
Hi,
this is v2 of my attempt at improving sparse binding perf.
The motivator is that many games have come to heavily use sparse
bindings, and they often rely on being able to overlap frame work on
gfx/compute queues with sparse binding work on sdma. If PRT mappings
are synced to every other submissions like they are right now, heavy
sparse binding load can stall the gfx/compute submissions for quite a
long while, leading to very noticeable stutters and hitches.
Another problem is that each sparse binding update requires a separate
ioctl, so the CPU will also spend significant time constantly switching
between user and kernel mode to execute the binding changes one at a
time.
To remedy this, this patchset introduces a "VM update context" structure.
This context encapsulates some state that is carried around different VM
update functions, in patch 1 this is the mappings that were freed along
the way. amdgpu_vm_clear_freed then only clears the mappings that were
actually freed in the update operation, which is a requirement for being
able to explicitly sync that unmap.
In later patches, the update context is expanded to hold a set of
pending mapping changes, so that a future ioctl that changes many
mappings at a time has a place to keep pending changes while every
change is validated, before applying everything atomically if things
look okay.
Best,
Natalie
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
Changes in v2:
- Added pending change structures to update context in order to batch changes
- Link to v1: https://patch.msgid.link/20260529-prt-explicit-sync-v1-0-9f1c60955cab@gmx.de
---
Natalie Vock (10):
drm/amdgpu: Remove unused parameter from amdgpu_vm_pte_fragment
drm/amdgpu: Add and use vm_update_ctx struct
drm/amdgpu: Make VM maps pending per update context
drm/amdgpu: Make removed/cleared mappings pending per update context
drm/amdgpu: Track dirty bo_va structures in update ctx
drm/amdgpu: Add function to roll back a VM update ctx
drm/amdgpu: Move some VA op validation to a helper
drm/amdgpu: Only reject conflicting mappings if they are not cleared
drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl
drm/amdgpu: Bump minor version
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 33 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 31 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c | 21 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 477 ++++++++++++++--
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 3 +
drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c | 9 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 10 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 660 +++++++++++++++--------
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 100 +++-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 8 +-
include/uapi/drm/amdgpu_drm.h | 46 ++
15 files changed, 1099 insertions(+), 311 deletions(-)
---
base-commit: 65b7b0d2cf8c3cba99d599493ac96f3ab32c0a06
change-id: 20260529-prt-explicit-sync-3d74b9dd420c
Best regards,
--
Natalie Vock <nat@pixelcluster.dev>
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v2 01/10] drm/amdgpu: Remove unused parameter from amdgpu_vm_pte_fragment
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 12:50 ` [PATCH v2 02/10] drm/amdgpu: Add and use vm_update_ctx struct Natalie Vock
` (8 subsequent siblings)
9 siblings, 0 replies; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
Trivial cleanup. No functional change.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index c73eb416c0e26..4eae6a831b4ee 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -745,14 +745,13 @@ static void amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
* @params: see amdgpu_vm_update_params definition
* @start: first PTE to handle
* @end: last PTE to handle
- * @flags: hw mapping flags
* @frag: resulting fragment size
* @frag_end: end of this fragment
*
* Returns the first possible fragment for the start and end address.
*/
static void amdgpu_vm_pte_fragment(struct amdgpu_vm_update_params *params,
- uint64_t start, uint64_t end, uint64_t flags,
+ uint64_t start, uint64_t end,
unsigned int *frag, uint64_t *frag_end)
{
/**
@@ -825,8 +824,7 @@ int amdgpu_vm_ptes_update(struct amdgpu_vm_update_params *params,
int r;
/* figure out the initial fragment */
- amdgpu_vm_pte_fragment(params, frag_start, end, flags, &frag,
- &frag_end);
+ amdgpu_vm_pte_fragment(params, frag_start, end, &frag, &frag_end);
/* walk over the address space and update the PTs */
amdgpu_vm_pt_start(adev, params->vm, start, &cursor);
@@ -932,7 +930,7 @@ int amdgpu_vm_ptes_update(struct amdgpu_vm_update_params *params,
if (frag_start >= frag_end) {
/* figure out the next fragment */
amdgpu_vm_pte_fragment(params, frag_start, end,
- flags, &frag, &frag_end);
+ &frag, &frag_end);
if (frag < shift)
break;
}
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 02/10] drm/amdgpu: Add and use vm_update_ctx struct
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
2026-11-17 12:50 ` [PATCH v2 01/10] drm/amdgpu: Remove unused parameter from amdgpu_vm_pte_fragment Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 13:26 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 03/10] drm/amdgpu: Make VM maps pending per update context Natalie Vock
` (7 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
This adds a "context" struct to the functions responsible for mapping
page tables (and committing the maps to the GPU page-tables).
The core idea here is to record the changes associated with specific VM
mapping operations (most importantly, which mappings were freed by each
map change). We need this to properly implement explicit sync for unmap
operations, which can only be performed for unmaps of either
always-valid or PRT mappings.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 30 ++-
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 31 ++-
drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c | 18 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 53 ++---
drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c | 8 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 10 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 248 +++++++++++++++--------
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 59 +++++-
10 files changed, 317 insertions(+), 144 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index f060b206c255f..08ab754e56d3d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1286,22 +1286,27 @@ static int unmap_bo_from_gpuvm(struct kgd_mem *mem,
struct amdgpu_bo_va *bo_va = entry->bo_va;
struct amdgpu_device *adev = entry->adev;
struct amdgpu_vm *vm = bo_va->base.vm;
+ struct amdgpu_vm_update_ctx update_ctx;
if (bo_va->queue_refcount) {
pr_debug("bo_va->queue_refcount %d\n", bo_va->queue_refcount);
return -EBUSY;
}
- (void)amdgpu_vm_bo_unmap(adev, bo_va, entry->va);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+
+ (void)amdgpu_vm_bo_unmap(&update_ctx, bo_va, entry->va);
/* VM entity stopped if process killed, don't clear freed pt bo */
if (!amdgpu_vm_ready(vm))
- return 0;
+ goto out;
- (void)amdgpu_vm_clear_freed(adev, vm, &bo_va->last_pt_update);
+ (void)amdgpu_vm_clear_freed(&update_ctx, &bo_va->last_pt_update);
(void)amdgpu_sync_fence(sync, bo_va->last_pt_update, GFP_KERNEL);
+out:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
return 0;
}
@@ -1311,6 +1316,7 @@ static int update_gpuvm_pte(struct kgd_mem *mem,
{
struct amdgpu_bo_va *bo_va = entry->bo_va;
struct amdgpu_device *adev = entry->adev;
+ struct amdgpu_vm_update_ctx update_ctx;
int ret;
ret = kfd_mem_dmamap_attachment(mem, entry);
@@ -1318,7 +1324,10 @@ static int update_gpuvm_pte(struct kgd_mem *mem,
return ret;
/* Update the page tables */
- ret = amdgpu_vm_bo_update(adev, bo_va, false);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, bo_va->base.vm);
+ ret = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+
if (ret) {
pr_err("amdgpu_vm_bo_update failed\n");
return ret;
@@ -1332,12 +1341,17 @@ static int map_bo_to_gpuvm(struct kgd_mem *mem,
struct amdgpu_sync *sync,
bool no_update_pte)
{
+ struct amdgpu_bo_va *bo_va = entry->bo_va;
+ struct amdgpu_device *adev = entry->adev;
+ struct amdgpu_vm_update_ctx update_ctx;
int ret;
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, bo_va->base.vm);
/* Set virtual address for the allocation */
- ret = amdgpu_vm_bo_map(entry->adev, entry->bo_va, entry->va, 0,
+ ret = amdgpu_vm_bo_map(&update_ctx, bo_va, entry->va, 0,
amdgpu_bo_size(entry->bo_va->base.bo),
entry->pte_flags);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
if (ret) {
pr_err("Failed to map VA 0x%llx in vm. ret %d\n",
entry->va, ret);
@@ -2982,6 +2996,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence __rcu *
struct amdgpu_sync sync_obj;
unsigned long failed_size = 0;
unsigned long total_size = 0;
+ struct amdgpu_vm_update_ctx update_ctx;
struct drm_exec exec;
int ret;
@@ -3091,7 +3106,10 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence __rcu *
struct amdgpu_fpriv *fpriv =
container_of(peer_vm, struct amdgpu_fpriv, vm);
- ret = amdgpu_vm_bo_update(adev, fpriv->prt_va, false);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, peer_vm);
+ ret = amdgpu_vm_bo_update(&update_ctx, fpriv->prt_va, false);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+
if (ret) {
dev_dbg(adev->dev,
"Memory eviction: handle PRT moved failed, pid %8d. Try again.\n",
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 71e2b83faa12b..d1f5ec9a3ef0f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1126,6 +1126,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
struct amdgpu_vm *vm = &fpriv->vm;
struct amdgpu_bo_list_entry *e;
struct amdgpu_bo_va *bo_va;
+ struct amdgpu_vm_update_ctx update_ctx;
unsigned int i;
int r;
@@ -1147,34 +1148,36 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
if (!amdgpu_vm_ready(vm))
return -EINVAL;
- r = amdgpu_vm_clear_freed(adev, vm, NULL);
+ r = amdgpu_vm_delayed_free(adev, vm, NULL);
if (r)
return r;
- r = amdgpu_vm_bo_update(adev, fpriv->prt_va, false);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+
+ r = amdgpu_vm_bo_update(&update_ctx, fpriv->prt_va, false);
if (r)
- return r;
+ goto err_fini_ctx;
r = amdgpu_sync_fence(&p->sync, fpriv->prt_va->last_pt_update,
GFP_KERNEL);
if (r)
- return r;
+ goto err_fini_ctx;
if (fpriv->csa_va) {
bo_va = fpriv->csa_va;
- r = amdgpu_vm_bo_update(adev, bo_va, false);
+ r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
if (r)
- return r;
+ goto err_fini_ctx;
r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update,
GFP_KERNEL);
if (r)
- return r;
+ goto err_fini_ctx;
}
if (fpriv->seq64_va) {
bo_va = fpriv->seq64_va;
- r = amdgpu_vm_bo_update(adev, bo_va, false);
+ r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
if (r) {
DRM_ERROR("failed to do vm_bo_update on userq sem\n");
return r;
@@ -1196,16 +1199,18 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
if (bo_va == NULL)
continue;
- r = amdgpu_vm_bo_update(adev, bo_va, false);
+ r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
if (r)
- return r;
+ goto err_fini_ctx;
r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update,
GFP_KERNEL);
if (r)
- return r;
+ goto err_fini_ctx;
}
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+
r = amdgpu_vm_handle_moved(adev, vm, drm_exec_ticket(&p->exec));
if (r)
return r;
@@ -1228,6 +1233,10 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
}
return 0;
+
+err_fini_ctx:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+ return r;
}
static int amdgpu_cs_sync_rings(struct amdgpu_cs_parser *p)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
index 11d70fe4f75a9..f33dfdb6efe2f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
@@ -66,6 +66,7 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
struct amdgpu_bo *bo, struct amdgpu_bo_va **bo_va,
uint64_t csa_addr, uint32_t size)
{
+ struct amdgpu_vm_update_ctx update_ctx;
struct drm_exec exec;
int r;
@@ -88,7 +89,9 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
goto error;
}
- r = amdgpu_vm_bo_map(adev, *bo_va, csa_addr, 0, size,
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+
+ r = amdgpu_vm_bo_map(&update_ctx, *bo_va, csa_addr, 0, size,
AMDGPU_PTE_READABLE | AMDGPU_PTE_WRITEABLE |
AMDGPU_PTE_EXECUTABLE);
@@ -96,9 +99,11 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
drm_err(adev_to_drm(adev),
"failed to do bo_map on static CSA, err=%d\n", r);
amdgpu_vm_bo_del(adev, *bo_va);
- goto error;
+ goto error_ctx;
}
+error_ctx:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
error:
drm_exec_fini(&exec);
return r;
@@ -108,6 +113,7 @@ int amdgpu_unmap_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
struct amdgpu_bo *bo, struct amdgpu_bo_va *bo_va,
uint64_t csa_addr)
{
+ struct amdgpu_vm_update_ctx update_ctx;
struct drm_exec exec;
int r;
@@ -124,15 +130,19 @@ int amdgpu_unmap_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
}
}
- r = amdgpu_vm_bo_unmap(adev, bo_va, csa_addr);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+
+ r = amdgpu_vm_bo_unmap(&update_ctx, bo_va, csa_addr);
if (r) {
drm_err(adev_to_drm(adev),
"failed to do bo_unmap on static CSA, err=%d\n", r);
- goto error;
+ goto error_ctx;
}
amdgpu_vm_bo_del(adev, bo_va);
+error_ctx:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
error:
drm_exec_fini(&exec);
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index 7b85152d6872c..cecb2e95b776c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -564,7 +564,7 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach)
/* Reserve fences for two SDMA page table updates */
r = dma_resv_reserve_fences(resv, 2);
if (!r)
- r = amdgpu_vm_clear_freed(adev, vm, NULL);
+ r = amdgpu_vm_delayed_free(adev, vm, NULL);
/* Don't pass 'ticket' to amdgpu_vm_handle_moved: we want the clear=true
* path to be used otherwise we might update the PT of another process
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index af8f96c8a2b59..55bdf5070837a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -51,6 +51,7 @@
static int
amdgpu_gem_add_input_fence(struct drm_file *filp,
+ struct amdgpu_vm_update_ctx *update_ctx,
uint64_t syncobj_handles_array,
uint32_t num_syncobj_handles)
{
@@ -77,9 +78,10 @@ amdgpu_gem_add_input_fence(struct drm_file *filp,
if (ret)
goto free_memdup;
- dma_fence_wait(fence, false);
+ ret = amdgpu_sync_fence(&update_ctx->sync, fence, GFP_KERNEL);
+ if (ret)
+ goto free_memdup;
- /* TODO: optimize async handling */
dma_fence_put(fence);
}
@@ -361,12 +363,13 @@ static void amdgpu_gem_object_close(struct drm_gem_object *obj,
if (!bo_va)
goto out_unlock;
+
amdgpu_vm_bo_del(adev, bo_va);
amdgpu_vm_bo_update_shared(bo);
if (!amdgpu_vm_ready(vm))
goto out_unlock;
- r = amdgpu_vm_clear_freed(adev, vm, &fence);
+ r = amdgpu_vm_delayed_free(adev, vm, &fence);
if (unlikely(r < 0) && !drm_dev_is_unplugged(adev_to_drm(adev)))
dev_err(adev->dev, "failed to clear page "
"tables on GEM object close (%ld)\n", r);
@@ -808,8 +811,7 @@ int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void *data,
* otherwise stub fence in case of error.
*/
static struct dma_fence *
-amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
- struct amdgpu_vm *vm,
+amdgpu_gem_va_update_vm(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint32_t operation)
{
@@ -817,10 +819,9 @@ amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
int r = 0;
/* If the VM is not ready return only a stub. */
- if (!amdgpu_vm_ready(vm))
+ if (!amdgpu_vm_ready(ctx->vm))
return dma_fence_get_stub();
-
/*
* First clean up any freed mappings in the VM.
*
@@ -828,20 +829,20 @@ amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
* schedules GPU work. If nothing needs clearing, @fence can remain as
* the original vm->last_update.
*/
- r = amdgpu_vm_clear_freed(adev, vm, &vm->last_update);
+ r = amdgpu_vm_clear_freed(ctx, &ctx->vm->last_update);
if (r)
goto error;
/* For MAP/REPLACE we also need to update the BO mappings. */
if (operation == AMDGPU_VA_OP_MAP ||
operation == AMDGPU_VA_OP_REPLACE) {
- r = amdgpu_vm_bo_update(adev, bo_va, false);
+ r = amdgpu_vm_bo_update(ctx, bo_va, false);
if (r)
goto error;
}
/* Always update PDEs after we touched the mappings. */
- r = amdgpu_vm_update_pdes(adev, vm, false);
+ r = amdgpu_vm_update_pdes(ctx->adev, ctx->vm, false);
error:
if (r && r != -ERESTARTSYS)
@@ -849,28 +850,28 @@ amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
if ((operation == AMDGPU_VA_OP_MAP ||
operation == AMDGPU_VA_OP_REPLACE) &&
- !amdgpu_vm_is_bo_always_valid(vm, bo_va->base.bo)) {
+ !amdgpu_vm_is_bo_always_valid(ctx->vm, bo_va->base.bo)) {
/*
* For MAP/REPLACE of non per-VM BOs we need to sync to both the
* bo_va->last_pt_update and vm->last_update or otherwise we
* potentially miss the PDE updates or TLB invalidation.
*/
- fence = dma_fence_unwrap_merge(vm->last_update,
- vm->last_tlb_inv,
+ fence = dma_fence_unwrap_merge(ctx->vm->last_update,
+ ctx->vm->last_tlb_inv,
bo_va->last_pt_update);
if (!fence)
dma_fence_wait(bo_va->last_pt_update, false);
} else {
- fence = dma_fence_unwrap_merge(vm->last_update,
- vm->last_tlb_inv);
+ fence = dma_fence_unwrap_merge(ctx->vm->last_update,
+ ctx->vm->last_tlb_inv);
}
if (!fence) {
/* As fallback in OOM situations */
- dma_fence_wait(vm->last_update, false);
- dma_fence_wait(vm->last_tlb_inv, false);
+ dma_fence_wait(ctx->vm->last_update, false);
+ dma_fence_wait(ctx->vm->last_tlb_inv, false);
fence = dma_fence_get_stub();
}
@@ -895,6 +896,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
struct amdgpu_bo_va *bo_va;
struct drm_syncobj *timeline_syncobj = NULL;
struct dma_fence_chain *timeline_chain = NULL;
+ struct amdgpu_vm_update_ctx update_ctx;
struct drm_exec exec;
uint64_t vm_size, tmp;
int r = 0;
@@ -967,8 +969,10 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
if (r)
goto error_put_gobj;
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, &fpriv->vm);
+
/* Add input syncobj fences (if any) for synchronization. */
- r = amdgpu_gem_add_input_fence(filp,
+ r = amdgpu_gem_add_input_fence(filp, &update_ctx,
args->input_fence_syncobj_handles,
args->num_syncobj_handles);
if (r)
@@ -1018,21 +1022,21 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
switch (args->operation) {
case AMDGPU_VA_OP_MAP:
- r = amdgpu_vm_bo_map(adev, bo_va, args->va_address,
+ r = amdgpu_vm_bo_map(&update_ctx, bo_va, args->va_address,
args->offset_in_bo, args->map_size,
args->flags);
break;
case AMDGPU_VA_OP_UNMAP:
- r = amdgpu_vm_bo_unmap(adev, bo_va, args->va_address);
+ r = amdgpu_vm_bo_unmap(&update_ctx, bo_va, args->va_address);
break;
case AMDGPU_VA_OP_CLEAR:
- r = amdgpu_vm_bo_clear_mappings(adev, &fpriv->vm,
- args->va_address,
+ r = amdgpu_vm_bo_clear_mappings(&update_ctx, args->va_address,
args->map_size);
break;
case AMDGPU_VA_OP_REPLACE:
- r = amdgpu_vm_bo_replace_map(adev, bo_va, args->va_address,
+ r = amdgpu_vm_bo_replace_map(&update_ctx, bo_va,
+ args->va_address,
args->offset_in_bo, args->map_size,
args->flags);
break;
@@ -1049,7 +1053,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
(!adev->debug_vm || timeline_syncobj)) {
struct dma_fence *fence;
- fence = amdgpu_gem_va_update_vm(adev, &fpriv->vm, bo_va,
+ fence = amdgpu_gem_va_update_vm(&update_ctx, bo_va,
args->operation);
if (timeline_syncobj) {
if (!args->vm_timeline_point) {
@@ -1070,6 +1074,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
}
error:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
dma_fence_chain_free(timeline_chain);
if (timeline_syncobj)
drm_syncobj_put(timeline_syncobj);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c
index fbf9ec7d467b4..63068be7ae4fd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c
@@ -67,6 +67,7 @@ static inline u64 amdgpu_seq64_get_va_base(struct amdgpu_device *adev)
int amdgpu_seq64_map(struct amdgpu_device *adev, struct amdgpu_vm *vm,
struct amdgpu_bo_va **bo_va)
{
+ struct amdgpu_vm_update_ctx update_ctx;
struct amdgpu_bo *bo;
struct drm_exec exec;
u64 seq64_addr;
@@ -94,15 +95,18 @@ int amdgpu_seq64_map(struct amdgpu_device *adev, struct amdgpu_vm *vm,
seq64_addr = amdgpu_seq64_get_va_base(adev) & AMDGPU_GMC_HOLE_MASK;
- r = amdgpu_vm_bo_map(adev, *bo_va, seq64_addr, 0,
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+ r = amdgpu_vm_bo_map(&update_ctx, *bo_va, seq64_addr, 0,
AMDGPU_VA_RESERVED_SEQ64_SIZE,
AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_MTYPE_UC);
if (r) {
DRM_ERROR("failed to do bo_map on userq sem, err=%d\n", r);
amdgpu_vm_bo_del(adev, *bo_va);
- goto error;
+ goto err_ctx;
}
+err_ctx:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
error:
drm_exec_fini(&exec);
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
index 11f022c56667a..b1bfd00dd2935 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ualink.c
@@ -3077,7 +3077,7 @@ static void amdgpu_ualink_invalidate_import_mappings(struct amdgpu_bo *bo)
* reserved by amdgpu_vm_lock_pd() above (it reserves 2 +
* num_fences on the same VM root PD dma_resv).
*/
- r = amdgpu_vm_clear_freed(adev, vm, NULL);
+ r = amdgpu_vm_delayed_free(adev, vm, NULL);
if (!r)
r = amdgpu_vm_handle_moved(adev, vm, &exec.ticket);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
index ed6f3b67e8720..4d4b1b6da1ff5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c
@@ -994,6 +994,7 @@ amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec,
struct amdgpu_vm *vm)
{
struct ttm_operation_ctx ctx = { false, false };
+ struct amdgpu_vm_update_ctx update_ctx;
struct amdgpu_bo_va *bo_va;
struct amdgpu_bo *bo, *tmp;
int ret;
@@ -1018,8 +1019,10 @@ amdgpu_userq_bo_validate(struct amdgpu_device *adev, struct drm_exec *exec,
if (ret)
return ret;
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
/* This moves the bo_va to the idle list */
- ret = amdgpu_vm_bo_update(adev, bo_va, false);
+ ret = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
if (ret)
return ret;
@@ -1038,6 +1041,7 @@ amdgpu_userq_vm_validate_and_restore_queue(struct amdgpu_userq_mgr *uq_mgr)
bool invalidated = false, new_addition = false;
struct ttm_operation_ctx ctx = { true, false };
struct amdgpu_device *adev = uq_mgr->adev;
+ struct amdgpu_vm_update_ctx update_ctx;
struct amdgpu_hmm_range *range;
struct amdgpu_vm *vm = &fpriv->vm;
unsigned long key, tmp_key;
@@ -1140,7 +1144,9 @@ amdgpu_userq_vm_validate_and_restore_queue(struct amdgpu_userq_mgr *uq_mgr)
* explicitly here (as the CS path does) so sparse mappings survive a
* VRAM-lost reset.
*/
- ret = amdgpu_vm_bo_update(adev, fpriv->prt_va, false);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+ ret = amdgpu_vm_bo_update(&update_ctx, fpriv->prt_va, false);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
if (ret)
goto unlock_all;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 94b1cfe05e3e0..90f27feb84257 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1319,37 +1319,31 @@ void amdgpu_vm_get_memory(struct amdgpu_vm *vm,
* Returns:
* 0 for success, -EINVAL for failure.
*/
-int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va,
- bool clear)
+int amdgpu_vm_bo_update(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_bo_va *bo_va, bool clear)
{
struct amdgpu_bo *bo = bo_va->base.bo;
- struct amdgpu_vm *vm = bo_va->base.vm;
+ struct amdgpu_vm *vm = ctx->vm;
struct amdgpu_bo_va_mapping *mapping;
struct dma_fence **last_update;
dma_addr_t *pages_addr = NULL;
struct ttm_resource *mem;
- struct amdgpu_sync sync;
bool flush_tlb = clear;
uint64_t vram_base;
uint64_t flags;
bool uncached;
int r;
- amdgpu_sync_create(&sync);
if (clear) {
mem = NULL;
- /* Implicitly sync to command submissions in the same VM before
- * unmapping.
- */
- r = amdgpu_sync_resv(adev, &sync, vm->root.bo->tbo.base.resv,
- AMDGPU_SYNC_EQ_OWNER, vm);
+ r = amdgpu_vm_update_ctx_ensure_unmap_synced(ctx);
if (r)
- goto error_free;
+ goto error;
if (bo) {
- r = amdgpu_sync_kfd(&sync, bo->tbo.base.resv);
+ r = amdgpu_sync_kfd(&ctx->sync, bo->tbo.base.resv);
if (r)
- goto error_free;
+ goto error;
}
} else if (!bo) {
mem = NULL;
@@ -1374,16 +1368,16 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va,
pages_addr = bo->tbo.ttm->dma_address;
/* Implicitly sync to moving fences before mapping anything */
- r = amdgpu_sync_resv(adev, &sync, bo->tbo.base.resv,
+ r = amdgpu_sync_resv(ctx->adev, &ctx->sync, bo->tbo.base.resv,
AMDGPU_SYNC_EXPLICIT, vm);
if (r)
- goto error_free;
+ goto error;
}
if (bo) {
struct amdgpu_device *bo_adev;
- flags = amdgpu_ttm_tt_pte_flags(adev, bo->tbo.ttm, mem);
+ flags = amdgpu_ttm_tt_pte_flags(ctx->adev, bo->tbo.ttm, mem);
if (amdgpu_bo_encrypted(bo))
flags |= AMDGPU_PTE_TMZ;
@@ -1422,18 +1416,18 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va,
update_flags &= ~AMDGPU_PTE_WRITEABLE;
/* Apply ASIC specific mapping flags */
- amdgpu_gmc_get_vm_pte(adev, vm, bo, mapping->flags,
+ amdgpu_gmc_get_vm_pte(ctx->adev, vm, bo, mapping->flags,
&update_flags);
trace_amdgpu_vm_bo_update(mapping);
- r = amdgpu_vm_update_range(adev, vm, false, false, flush_tlb,
- !uncached, &sync, mapping->start,
+ r = amdgpu_vm_update_range(ctx->adev, vm, false, false, flush_tlb,
+ !uncached, &ctx->sync, mapping->start,
mapping->last, update_flags,
mapping->offset, vram_base, mem,
pages_addr, last_update);
if (r)
- goto error_free;
+ goto error;
}
/* If the BO is not in its preferred location add it back to
@@ -1466,8 +1460,7 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va,
trace_amdgpu_vm_bo_mapping(mapping);
}
-error_free:
- amdgpu_sync_free(&sync);
+error:
return r;
}
@@ -1596,55 +1589,103 @@ static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)
}
}
+/**
+ * amdgpu_vm_update_ctx_ensure_unmap_synced - VM update sync helper
+ * @ctx: context for updating the VM
+ *
+ * Ensures that ctx->sync will synchronize to everything that's necessary
+ * to unmap freed pages associated with the context.
+ */
+int amdgpu_vm_update_ctx_ensure_unmap_synced(struct amdgpu_vm_update_ctx *ctx)
+{
+ int r;
+
+ if (ctx->unmap_synced)
+ return 0;
+
+ /*
+ * Implicitly sync to command submissions in the same VM before
+ * unmapping, unless we unmap with explicit sync.
+ */
+ r = amdgpu_sync_resv(ctx->adev, &ctx->sync,
+ ctx->vm->root.bo->tbo.base.resv,
+ AMDGPU_SYNC_EQ_OWNER, ctx->vm);
+ if (r)
+ return r;
+
+ ctx->unmap_synced = true;
+
+ return 0;
+}
+
+/**
+ * amdgpu_vm_update_ctx_init - create a context for VM update operations
+ * @ctx: The new context to create
+ * @adev: The device associated with the VM update
+ * @vm: The VM being updated
+ *
+ * Initializes the context to an empty state.
+ */
+void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_device *adev, struct amdgpu_vm *vm)
+{
+ memset(ctx, 0, sizeof(*ctx));
+ ctx->adev = adev;
+ ctx->vm = vm;
+ INIT_LIST_HEAD(&ctx->freed);
+ amdgpu_sync_create(&ctx->sync);
+}
+
+/**
+ * amdgpu_vm_update_ctx_fini - finish a VM update operation
+ * @ctx: The context of the update
+ *
+ * Adds any leftover mappings needing to be cleared to the VM's delayed free list,
+ * then frees up resources used by the context.
+ */
+void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
+{
+ list_splice(&ctx->freed, &ctx->vm->delayed_freed);
+ amdgpu_sync_free(&ctx->sync);
+}
+
/**
* amdgpu_vm_clear_freed - clear freed BOs in the PT
*
- * @adev: amdgpu_device pointer
- * @vm: requested vm
+ * @ctx: Context for VM updates
* @fence: optional resulting fence (unchanged if no work needed to be done
* or if an error occurred)
*
- * Make sure all freed BOs are cleared in the PT.
+ * Make sure all BOs freed by VM updates in the context are cleared in the PT.
* PTs have to be reserved and mutex must be locked!
*
* Returns:
* 0 for success.
*
*/
-int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
- struct amdgpu_vm *vm,
+int amdgpu_vm_clear_freed(struct amdgpu_vm_update_ctx *ctx,
struct dma_fence **fence)
{
struct amdgpu_bo_va_mapping *mapping;
struct dma_fence *f = NULL;
- struct amdgpu_sync sync;
int r;
- if (list_empty(&vm->freed))
- return 0;
-
- /*
- * Implicitly sync to command submissions in the same VM before
- * unmapping.
- */
- amdgpu_sync_create(&sync);
- r = amdgpu_sync_resv(adev, &sync, vm->root.bo->tbo.base.resv,
- AMDGPU_SYNC_EQ_OWNER, vm);
+ r = amdgpu_vm_update_ctx_ensure_unmap_synced(ctx);
if (r)
- goto error_free;
+ return r;
- while (!list_empty(&vm->freed)) {
- mapping = list_first_entry(&vm->freed,
- struct amdgpu_bo_va_mapping, list);
+ while (!list_empty(&ctx->freed)) {
+ mapping = list_first_entry(&ctx->freed,
+ struct amdgpu_bo_va_mapping, list);
list_del(&mapping->list);
- r = amdgpu_vm_update_range(adev, vm, false, false, true, false,
- &sync, mapping->start, mapping->last,
+ r = amdgpu_vm_update_range(ctx->adev, ctx->vm, false, false, true, false,
+ &ctx->sync, mapping->start, mapping->last,
0, 0, 0, NULL, NULL, &f);
- amdgpu_vm_free_mapping(adev, vm, mapping, f);
+ amdgpu_vm_free_mapping(ctx->adev, ctx->vm, mapping, f);
if (r) {
dma_fence_put(f);
- goto error_free;
+ return r;
}
}
@@ -1655,12 +1696,40 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
dma_fence_put(f);
}
-error_free:
- amdgpu_sync_free(&sync);
return r;
}
+/**
+ * amdgpu_vm_delayed_free - execute delayed PT clearing
+ *
+ * @adev: Device to use for freeing
+ * @vm: VM whose BOs should be freed
+ * @fence: optional resulting fence (unchanged if no work needed to be done
+ * or if an error occurred)
+ *
+ * Go over the list of BOs whose PT clearing was delayed and clear all of them.
+ * PTs have to be reserved and mutex must be locked!
+ *
+ * Returns:
+ * 0 for success.
+ *
+ */
+int amdgpu_vm_delayed_free(struct amdgpu_device *adev, struct amdgpu_vm *vm,
+ struct dma_fence **fence)
+{
+ struct amdgpu_vm_update_ctx ctx;
+ int r;
+
+ amdgpu_vm_update_ctx_init(&ctx, adev, vm);
+
+ list_splice_init(&vm->delayed_freed, &ctx.freed);
+ r = amdgpu_vm_clear_freed(&ctx, fence);
+
+ amdgpu_vm_update_ctx_fini(&ctx);
+ return r;
+}
+
/**
* amdgpu_vm_handle_moved - handle moved BOs in the PT
*
@@ -1679,18 +1748,22 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
struct amdgpu_vm *vm,
struct ww_acquire_ctx *ticket)
{
+ struct amdgpu_vm_update_ctx update_ctx;
struct amdgpu_bo_va *bo_va, *tmp;
struct dma_resv *resv;
struct amdgpu_bo *bo;
bool clear, unlock;
int r;
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+
list_for_each_entry_safe(bo_va, tmp, &vm->always_valid.needs_update,
base.vm_status) {
/* Per VM BOs never need to bo cleared in the page tables */
- r = amdgpu_vm_bo_update(adev, bo_va, false);
+ r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
+
if (r)
- return r;
+ goto error;
}
spin_lock(&vm->individual_lock);
@@ -1716,7 +1789,7 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
unlock = false;
}
- r = amdgpu_vm_bo_update(adev, bo_va, clear);
+ r = amdgpu_vm_bo_update(&update_ctx, bo_va, clear);
if (unlock)
dma_resv_unlock(resv);
@@ -1736,6 +1809,9 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
}
spin_unlock(&vm->individual_lock);
+error:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+
return 0;
}
@@ -1889,7 +1965,7 @@ static int amdgpu_vm_verify_parameters(struct amdgpu_device *adev,
/**
* amdgpu_vm_bo_map - map bo inside a vm
*
- * @adev: amdgpu_device pointer
+ * @ctx: VM update context
* @bo_va: bo_va to store the address
* @saddr: where to map the BO
* @offset: requested offset in the BO
@@ -1903,7 +1979,7 @@ static int amdgpu_vm_verify_parameters(struct amdgpu_device *adev,
*
* Object has to be reserved and unreserved outside!
*/
-int amdgpu_vm_bo_map(struct amdgpu_device *adev,
+int amdgpu_vm_bo_map(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint64_t saddr, uint64_t offset,
uint64_t size, uint32_t flags)
@@ -1914,7 +1990,7 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
uint64_t eaddr;
int r;
- r = amdgpu_vm_verify_parameters(adev, bo, saddr, offset, size);
+ r = amdgpu_vm_verify_parameters(ctx->adev, bo, saddr, offset, size);
if (r)
return r;
@@ -1924,7 +2000,8 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
tmp = amdgpu_vm_it_iter_first(&vm->va, saddr, eaddr);
if (tmp) {
/* bo and tmp overlap, invalid addr */
- dev_err(adev->dev, "bo %p va 0x%010Lx-0x%010Lx conflict with "
+ dev_err(ctx->adev->dev,
+ "bo %p va 0x%010Lx-0x%010Lx conflict with "
"0x%010Lx-0x%010Lx\n", bo, saddr, eaddr,
tmp->start, tmp->last + 1);
return -EINVAL;
@@ -1939,7 +2016,7 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
mapping->offset = offset;
mapping->flags = flags;
- amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
+ amdgpu_vm_bo_insert_map(ctx->adev, bo_va, mapping);
return 0;
}
@@ -1947,7 +2024,7 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
/**
* amdgpu_vm_bo_replace_map - map bo inside a vm, replacing existing mappings
*
- * @adev: amdgpu_device pointer
+ * @ctx: VM update context
* @bo_va: bo_va to store the address
* @saddr: where to map the BO
* @offset: requested offset in the BO
@@ -1962,7 +2039,7 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
*
* Object has to be reserved and unreserved outside!
*/
-int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
+int amdgpu_vm_bo_replace_map(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint64_t saddr, uint64_t offset,
uint64_t size, uint32_t flags)
@@ -1972,7 +2049,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
uint64_t eaddr;
int r;
- r = amdgpu_vm_verify_parameters(adev, bo, saddr, offset, size);
+ r = amdgpu_vm_verify_parameters(ctx->adev, bo, saddr, offset, size);
if (r)
return r;
@@ -1981,7 +2058,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
if (!mapping)
return -ENOMEM;
- r = amdgpu_vm_bo_clear_mappings(adev, bo_va->base.vm, saddr, size);
+ r = amdgpu_vm_bo_clear_mappings(ctx, saddr, size);
if (r) {
kfree(mapping);
return r;
@@ -1995,7 +2072,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
mapping->offset = offset;
mapping->flags = flags;
- amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
+ amdgpu_vm_bo_insert_map(ctx->adev, bo_va, mapping);
return 0;
}
@@ -2003,7 +2080,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
/**
* amdgpu_vm_bo_unmap - remove bo mapping from vm
*
- * @adev: amdgpu_device pointer
+ * @ctx: VM update context
* @bo_va: bo_va to remove the address from
* @saddr: where to the BO is mapped
*
@@ -2014,12 +2091,12 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
*
* Object has to be reserved and unreserved outside!
*/
-int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
+int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint64_t saddr)
{
struct amdgpu_bo_va_mapping *mapping;
- struct amdgpu_vm *vm = bo_va->base.vm;
+ struct amdgpu_vm *vm = ctx->vm;
bool valid = true;
saddr /= AMDGPU_GPU_PAGE_SIZE;
@@ -2046,7 +2123,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
* from user space.
*/
if (unlikely(bo_va->userq_va_mapped))
- amdgpu_userq_gem_va_unmap_validate(adev, mapping);
+ amdgpu_userq_gem_va_unmap_validate(ctx->adev, mapping);
list_del(&mapping->list);
amdgpu_vm_it_remove(mapping, &vm->va);
@@ -2054,9 +2131,9 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
trace_amdgpu_vm_bo_unmap(bo_va, mapping);
if (valid)
- list_add(&mapping->list, &vm->freed);
+ list_add(&mapping->list, &ctx->freed);
else
- amdgpu_vm_free_mapping(adev, vm, mapping,
+ amdgpu_vm_free_mapping(ctx->adev, vm, mapping,
bo_va->last_pt_update);
return 0;
@@ -2065,8 +2142,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
/**
* amdgpu_vm_bo_clear_mappings - remove all mappings in a specific range
*
- * @adev: amdgpu_device pointer
- * @vm: VM structure to use
+ * @ctx: VM update context
* @saddr: start of the range
* @size: size of the range
*
@@ -2075,8 +2151,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
* Returns:
* 0 for success, error for failure.
*/
-int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
- struct amdgpu_vm *vm,
+int amdgpu_vm_bo_clear_mappings(struct amdgpu_vm_update_ctx *ctx,
uint64_t saddr, uint64_t size)
{
struct amdgpu_bo_va_mapping *before, *after, *tmp, *next;
@@ -2084,7 +2159,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
uint64_t eaddr;
int r;
- r = amdgpu_vm_verify_parameters(adev, NULL, saddr, 0, size);
+ r = amdgpu_vm_verify_parameters(ctx->adev, NULL, saddr, 0, size);
if (r)
return r;
@@ -2105,7 +2180,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
INIT_LIST_HEAD(&after->list);
/* Now gather all removed mappings */
- tmp = amdgpu_vm_it_iter_first(&vm->va, saddr, eaddr);
+ tmp = amdgpu_vm_it_iter_first(&ctx->vm->va, saddr, eaddr);
while (tmp) {
/* Remember mapping split at the start */
if (tmp->start < saddr) {
@@ -2136,7 +2211,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
/* And free them up */
list_for_each_entry_safe(tmp, next, &removed, list) {
- amdgpu_vm_it_remove(tmp, &vm->va);
+ amdgpu_vm_it_remove(tmp, &ctx->vm->va);
list_del(&tmp->list);
if (tmp->start < saddr)
@@ -2145,7 +2220,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
tmp->last = eaddr;
tmp->bo_va = NULL;
- list_add(&tmp->list, &vm->freed);
+ list_add(&tmp->list, &ctx->freed);
trace_amdgpu_vm_bo_unmap(NULL, tmp);
}
@@ -2153,11 +2228,11 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
if (!list_empty(&before->list)) {
struct amdgpu_bo *bo = before->bo_va->base.bo;
- amdgpu_vm_it_insert(before, &vm->va);
+ amdgpu_vm_it_insert(before, &ctx->vm->va);
if (before->flags & AMDGPU_VM_PAGE_PRT)
- amdgpu_vm_prt_get(adev);
+ amdgpu_vm_prt_get(ctx->adev);
- if (amdgpu_vm_is_bo_always_valid(vm, bo) &&
+ if (amdgpu_vm_is_bo_always_valid(ctx->vm, bo) &&
!before->bo_va->base.moved)
amdgpu_vm_bo_needs_update(&before->bo_va->base);
} else {
@@ -2168,11 +2243,11 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
if (!list_empty(&after->list)) {
struct amdgpu_bo *bo = after->bo_va->base.bo;
- amdgpu_vm_it_insert(after, &vm->va);
+ amdgpu_vm_it_insert(after, &ctx->vm->va);
if (after->flags & AMDGPU_VM_PAGE_PRT)
- amdgpu_vm_prt_get(adev);
+ amdgpu_vm_prt_get(ctx->adev);
- if (amdgpu_vm_is_bo_always_valid(vm, bo) &&
+ if (amdgpu_vm_is_bo_always_valid(ctx->vm, bo) &&
!after->bo_va->base.moved)
amdgpu_vm_bo_needs_update(&after->bo_va->base);
} else {
@@ -2248,6 +2323,7 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev,
struct amdgpu_bo *bo = bo_va->base.bo;
struct amdgpu_vm *vm = bo_va->base.vm;
struct amdgpu_vm_bo_base **base;
+ struct amdgpu_vm_update_ctx update_ctx;
dma_resv_assert_held(vm->root.bo->tbo.base.resv);
@@ -2271,12 +2347,14 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev,
list_del(&bo_va->base.vm_status);
spin_unlock(&vm->individual_lock);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
+
list_for_each_entry_safe(mapping, next, &bo_va->valids, list) {
list_del(&mapping->list);
amdgpu_vm_it_remove(mapping, &vm->va);
mapping->bo_va = NULL;
trace_amdgpu_vm_bo_unmap(bo_va, mapping);
- list_add(&mapping->list, &vm->freed);
+ list_add(&mapping->list, &update_ctx.freed);
}
list_for_each_entry_safe(mapping, next, &bo_va->invalids, list) {
list_del(&mapping->list);
@@ -2285,6 +2363,8 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev,
bo_va->last_pt_update);
}
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+
dma_fence_put(bo_va->last_pt_update);
if (bo && bo_va->is_xgmi)
@@ -2656,7 +2736,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm,
amdgpu_vm_bo_status_init(&vm->always_valid);
spin_lock_init(&vm->individual_lock);
amdgpu_vm_bo_status_init(&vm->individual);
- INIT_LIST_HEAD(&vm->freed);
+ INIT_LIST_HEAD(&vm->delayed_freed);
INIT_KFIFO(vm->faults);
spin_lock_init(&vm->stats_lock);
@@ -2883,7 +2963,7 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)
dma_fence_put(vm->last_seq_flush);
dma_fence_put(vm->last_tlb_inv);
- list_for_each_entry_safe(mapping, tmp, &vm->freed, list) {
+ list_for_each_entry_safe(mapping, tmp, &vm->delayed_freed, list) {
if (mapping->flags & AMDGPU_VM_PAGE_PRT && prt_fini_needed) {
amdgpu_vm_prt_fini(adev, vm);
prt_fini_needed = false;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index f6285a9d9e989..8445599d9539a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -264,6 +264,39 @@ struct amdgpu_task_info {
struct kref refcount;
};
+/**
+ * struct amdgpu_vm_update_ctx
+ *
+ * Encapsulates various state that is carried around during VM updates.
+ */
+struct amdgpu_vm_update_ctx {
+ /**
+ * @adev: amdgpu device the update is done on
+ */
+ struct amdgpu_device *adev;
+ /**
+ * @vm: target VM of the update
+ */
+ struct amdgpu_vm *vm;
+
+ /**
+ * @freed: A list of mappings that are freed and pending PT/PD updates during
+ * this update operation.
+ */
+ struct list_head freed;
+
+ /**
+ * @sync: Container for everything to sync to before committing PT/PD updates.
+ */
+ struct amdgpu_sync sync;
+
+ /**
+ * @unmap_synced: Whether the sync is strong enough to guarantee a safe unmap,
+ * i.e. all previous submissions in the VM are waited on.
+ */
+ bool unmap_synced;
+};
+
struct amdgpu_vm_fault_info {
/* fault address */
uint64_t addr;
@@ -318,7 +351,7 @@ struct amdgpu_vm {
* This list contains amdgpu_bo_va_mapping objects which have been freed
* but not updated in the PTs
*/
- struct list_head freed;
+ struct list_head delayed_freed;
/* contains the page directory */
struct amdgpu_vm_bo_base root;
@@ -453,9 +486,10 @@ void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
bool *need_pipe_sync, bool *emit_gds_needed);
int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
struct amdgpu_vm *vm, bool immediate);
-int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
- struct amdgpu_vm *vm,
+int amdgpu_vm_clear_freed(struct amdgpu_vm_update_ctx *ctx,
struct dma_fence **fence);
+int amdgpu_vm_delayed_free(struct amdgpu_device *adev, struct amdgpu_vm *vm,
+ struct dma_fence **fence);
int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
struct amdgpu_vm *vm,
struct ww_acquire_ctx *ticket);
@@ -472,7 +506,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
uint64_t offset, uint64_t vram_base,
struct ttm_resource *res, dma_addr_t *pages_addr,
struct dma_fence **fence);
-int amdgpu_vm_bo_update(struct amdgpu_device *adev,
+int amdgpu_vm_bo_update(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
bool clear);
bool amdgpu_vm_evictable(struct amdgpu_bo *bo);
@@ -488,20 +522,21 @@ struct amdgpu_bo_va *amdgpu_vm_bo_find(struct amdgpu_vm *vm,
struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
struct amdgpu_vm *vm,
struct amdgpu_bo *bo);
-int amdgpu_vm_bo_map(struct amdgpu_device *adev,
+
+int amdgpu_vm_bo_map(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint64_t addr, uint64_t offset,
uint64_t size, uint32_t flags);
-int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
+int amdgpu_vm_bo_replace_map(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint64_t addr, uint64_t offset,
uint64_t size, uint32_t flags);
-int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
+int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va *bo_va,
uint64_t addr);
-int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
- struct amdgpu_vm *vm,
+int amdgpu_vm_bo_clear_mappings(struct amdgpu_vm_update_ctx *ctx,
uint64_t saddr, uint64_t size);
+
struct amdgpu_bo_va_mapping *amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm *vm,
uint64_t addr);
void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket);
@@ -545,6 +580,12 @@ int amdgpu_vm_pt_map_tables(struct amdgpu_device *adev, struct amdgpu_vm *vm);
bool amdgpu_vm_is_bo_always_valid(struct amdgpu_vm *vm, struct amdgpu_bo *bo);
+void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_device *adev,
+ struct amdgpu_vm *vm);
+void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx);
+int amdgpu_vm_update_ctx_ensure_unmap_synced(struct amdgpu_vm_update_ctx *ctx);
+
/**
* amdgpu_vm_tlb_seq - return tlb flush sequence number
* @vm: the amdgpu_vm structure to query
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 03/10] drm/amdgpu: Make VM maps pending per update context
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
2026-11-17 12:50 ` [PATCH v2 01/10] drm/amdgpu: Remove unused parameter from amdgpu_vm_pte_fragment Natalie Vock
2026-11-17 12:50 ` [PATCH v2 02/10] drm/amdgpu: Add and use vm_update_ctx struct Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 13:43 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 04/10] drm/amdgpu: Make removed/cleared mappings " Natalie Vock
` (6 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
For later being able to discard pending changes when an error is
encountered afterwards.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 5 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 124 +++++++++++++++++------
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 11 ++
7 files changed, 113 insertions(+), 33 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index 08ab754e56d3d..b20b84d0a54bb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -1351,13 +1351,16 @@ static int map_bo_to_gpuvm(struct kgd_mem *mem,
ret = amdgpu_vm_bo_map(&update_ctx, bo_va, entry->va, 0,
amdgpu_bo_size(entry->bo_va->base.bo),
entry->pte_flags);
- amdgpu_vm_update_ctx_fini(&update_ctx);
if (ret) {
pr_err("Failed to map VA 0x%llx in vm. ret %d\n",
entry->va, ret);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
return ret;
}
+ amdgpu_vm_update_ctx_commit(&update_ctx);
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+
if (no_update_pte)
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
index f33dfdb6efe2f..8ec6b46e3f25a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
@@ -22,6 +22,7 @@
* * Author: Monk.liu@amd.com
*/
+#include "amdgpu_vm.h"
#include <drm/drm_exec.h>
#include "amdgpu.h"
@@ -94,6 +95,7 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
r = amdgpu_vm_bo_map(&update_ctx, *bo_va, csa_addr, 0, size,
AMDGPU_PTE_READABLE | AMDGPU_PTE_WRITEABLE |
AMDGPU_PTE_EXECUTABLE);
+ amdgpu_vm_update_ctx_commit(&update_ctx);
if (r) {
drm_err(adev_to_drm(adev),
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 55bdf5070837a..2816d0851bf01 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -1044,6 +1044,8 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
break;
}
+ amdgpu_vm_update_ctx_commit(&update_ctx);
+
/*
* Once the VA operation is done, update the VM and obtain the fence
* that represents the last relevant update for this mapping. This
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index 2e6a8e6f577a7..e0090dc10682f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -64,6 +64,7 @@ struct amdgpu_bo_param {
struct amdgpu_bo_va_mapping {
struct amdgpu_bo_va *bo_va;
struct list_head list;
+ struct list_head pending_list;
struct rb_node rb;
uint64_t start;
uint64_t last;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c
index 63068be7ae4fd..84a14a5cc0be2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_seq64.c
@@ -104,6 +104,7 @@ int amdgpu_seq64_map(struct amdgpu_device *adev, struct amdgpu_vm *vm,
amdgpu_vm_bo_del(adev, *bo_va);
goto err_ctx;
}
+ amdgpu_vm_update_ctx_commit(&update_ctx);
err_ctx:
amdgpu_vm_update_ctx_fini(&update_ctx);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 90f27feb84257..97cf04da6a025 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1589,6 +1589,73 @@ static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)
}
}
+/**
+ * amdgpu_vm_bo_commit_insert - insert a new mapping
+ *
+ * @adev: amdgpu_device pointer
+ * @bo_va: bo_va to store the address
+ * @mapping: the mapping to insert
+ *
+ * Insert a new mapping into all structures.
+ */
+static void amdgpu_vm_bo_commit_insert(struct amdgpu_device *adev,
+ struct amdgpu_bo_va *bo_va,
+ struct amdgpu_bo_va_mapping *mapping)
+{
+ struct amdgpu_vm *vm = bo_va->base.vm;
+ struct amdgpu_bo *bo = bo_va->base.bo;
+
+ mapping->bo_va = bo_va;
+ list_add(&mapping->list, &bo_va->invalids);
+ amdgpu_vm_it_insert(mapping, &vm->va);
+
+ if (mapping->flags & AMDGPU_VM_PAGE_PRT)
+ amdgpu_vm_prt_get(adev);
+
+ if (bo && !bo_va->base.moved)
+ amdgpu_vm_bo_needs_update(&bo_va->base);
+
+ trace_amdgpu_vm_bo_map(bo_va, mapping);
+}
+
+void amdgpu_vm_update_ctx_commit(struct amdgpu_vm_update_ctx *ctx)
+{
+ struct amdgpu_bo_va_mapping *mapping, *next;
+
+ list_for_each_entry_safe(mapping, next, &ctx->new, pending_list) {
+ list_del_init(&mapping->pending_list);
+ amdgpu_vm_bo_commit_insert(ctx->adev, mapping->bo_va, mapping);
+ }
+}
+
+static int
+amdgpu_vm_update_ctx_add_new_mapping(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_bo_va *bo_va,
+ struct amdgpu_bo_va_mapping *mapping)
+{
+ struct interval_tree_node *pending;
+
+ if (interval_tree_iter_first(&ctx->pending_maps, mapping->start,
+ mapping->last))
+ return -EINVAL;
+
+ pending = kzalloc_obj(*pending);
+ if (!pending)
+ return -ENOMEM;
+
+ if (interval_tree_iter_first(&ctx->pending_maps, mapping->start,
+ mapping->last))
+ return -EINVAL;
+
+ pending->start = mapping->start;
+ pending->last = mapping->last;
+ interval_tree_insert(pending, &ctx->pending_maps);
+
+ mapping->bo_va = bo_va;
+ list_add(&mapping->pending_list, &ctx->new);
+ return 0;
+}
+
/**
* amdgpu_vm_update_ctx_ensure_unmap_synced - VM update sync helper
* @ctx: context for updating the VM
@@ -1632,6 +1699,9 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
memset(ctx, 0, sizeof(*ctx));
ctx->adev = adev;
ctx->vm = vm;
+
+ ctx->pending_maps = RB_ROOT_CACHED;
+ INIT_LIST_HEAD(&ctx->new);
INIT_LIST_HEAD(&ctx->freed);
amdgpu_sync_create(&ctx->sync);
}
@@ -1645,8 +1715,17 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
*/
void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
{
+ struct interval_tree_node *pending, *next_pending;
+
list_splice(&ctx->freed, &ctx->vm->delayed_freed);
amdgpu_sync_free(&ctx->sync);
+
+ rbtree_postorder_for_each_entry_safe(pending, next_pending, &ctx->pending_maps.rb_root, rb)
+ kfree(pending);
+
+ ctx->pending_maps = RB_ROOT_CACHED;
+
+ WARN_ON(!list_empty(&ctx->new));
}
/**
@@ -1901,36 +1980,6 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
return bo_va;
}
-
-/**
- * amdgpu_vm_bo_insert_map - insert a new mapping
- *
- * @adev: amdgpu_device pointer
- * @bo_va: bo_va to store the address
- * @mapping: the mapping to insert
- *
- * Insert a new mapping into all structures.
- */
-static void amdgpu_vm_bo_insert_map(struct amdgpu_device *adev,
- struct amdgpu_bo_va *bo_va,
- struct amdgpu_bo_va_mapping *mapping)
-{
- struct amdgpu_vm *vm = bo_va->base.vm;
-
- mapping->bo_va = bo_va;
- list_add(&mapping->list, &bo_va->invalids);
- amdgpu_vm_it_insert(mapping, &vm->va);
-
- if (mapping->flags & AMDGPU_VM_PAGE_PRT)
- amdgpu_vm_prt_get(adev);
-
- /* PRT mappings have no BO and are kept off the state lists */
- if (bo_va->base.bo && !bo_va->base.moved)
- amdgpu_vm_bo_needs_update(&bo_va->base);
-
- trace_amdgpu_vm_bo_map(bo_va, mapping);
-}
-
/* Validate operation parameters to prevent potential abuse */
static int amdgpu_vm_verify_parameters(struct amdgpu_device *adev,
struct amdgpu_bo *bo,
@@ -2015,8 +2064,14 @@ int amdgpu_vm_bo_map(struct amdgpu_vm_update_ctx *ctx,
mapping->last = eaddr;
mapping->offset = offset;
mapping->flags = flags;
+ mapping->bo_va = bo_va;
+ INIT_LIST_HEAD(&mapping->pending_list);
- amdgpu_vm_bo_insert_map(ctx->adev, bo_va, mapping);
+ r = amdgpu_vm_update_ctx_add_new_mapping(ctx, bo_va, mapping);
+ if (r) {
+ kfree(mapping);
+ return r;
+ }
return 0;
}
@@ -2071,8 +2126,13 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_vm_update_ctx *ctx,
mapping->last = eaddr;
mapping->offset = offset;
mapping->flags = flags;
+ INIT_LIST_HEAD(&mapping->pending_list);
- amdgpu_vm_bo_insert_map(ctx->adev, bo_va, mapping);
+ r = amdgpu_vm_update_ctx_add_new_mapping(ctx, bo_va, mapping);
+ if (r) {
+ kfree(mapping);
+ return r;
+ }
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index 8445599d9539a..d35494440cb5a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -279,6 +279,16 @@ struct amdgpu_vm_update_ctx {
*/
struct amdgpu_vm *vm;
+ /**
+ * @mapped_intervals: Interval tree of VA ranges mapped by this update operation.
+ */
+ struct rb_root_cached pending_maps;
+
+ /**
+ * @new: Pending new mappings not yet applied to bookkeeping structures.
+ */
+ struct list_head new;
+
/**
* @freed: A list of mappings that are freed and pending PT/PD updates during
* this update operation.
@@ -583,6 +593,7 @@ bool amdgpu_vm_is_bo_always_valid(struct amdgpu_vm *vm, struct amdgpu_bo *bo);
void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_device *adev,
struct amdgpu_vm *vm);
+void amdgpu_vm_update_ctx_commit(struct amdgpu_vm_update_ctx *ctx);
void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx);
int amdgpu_vm_update_ctx_ensure_unmap_synced(struct amdgpu_vm_update_ctx *ctx);
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 04/10] drm/amdgpu: Make removed/cleared mappings pending per update context
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (2 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 03/10] drm/amdgpu: Make VM maps pending per update context Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 14:05 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 05/10] drm/amdgpu: Track dirty bo_va structures in update ctx Natalie Vock
` (5 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
This completes the effort to make all mapping changes pending to be able
to easily revert to the previous state in case an error is encountered
after unmapping.
In certain cases, it is possible to skip syncing with every submission
in the whole VM before unmapping pagetables. The important condition is
that no memory must be released to the system while the GPU still has a
chance to access it. This is true for VM-always-valid buffers since the
resv is shared with the VM's root PD and thus there will always be
synchronization before unmapping, but also for PRT pages which do not
point to any real memory location at all.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 7 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 305 ++++++++++++++++++++------------
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 24 +++
4 files changed, 220 insertions(+), 117 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
index 8ec6b46e3f25a..2331c6af6ce46 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c
@@ -140,6 +140,7 @@ int amdgpu_unmap_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm,
"failed to do bo_unmap on static CSA, err=%d\n", r);
goto error_ctx;
}
+ amdgpu_vm_update_ctx_commit(&update_ctx);
amdgpu_vm_bo_del(adev, bo_va);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 2816d0851bf01..39c72bfb73e71 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -1044,7 +1044,12 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
break;
}
- amdgpu_vm_update_ctx_commit(&update_ctx);
+ if (!r) {
+ r = amdgpu_vm_update_ctx_execute_replace(&update_ctx);
+ if (r)
+ goto error;
+ amdgpu_vm_update_ctx_commit(&update_ctx);
+ }
/*
* Once the VA operation is done, update the VM and obtain the fence
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 97cf04da6a025..1e2d45979efdb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1618,10 +1618,74 @@ static void amdgpu_vm_bo_commit_insert(struct amdgpu_device *adev,
trace_amdgpu_vm_bo_map(bo_va, mapping);
}
+static void amdgpu_vm_bo_commit_remove(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_bo_va_mapping *mapping)
+{
+ struct amdgpu_bo_va_mapping *entry;
+ struct amdgpu_bo_va *bo_va = mapping->bo_va;
+ bool valid;
+
+ entry = list_next_entry(mapping, list);
+ while (!list_entry_is_head(entry, &bo_va->valids, list) &&
+ !list_entry_is_head(entry, &bo_va->invalids, list)) {
+ /*
+ * If we go through the entire list just to end up back where
+ * we started without encountering either the valids or invalids
+ * head, that means the mapping is in neither list and probably
+ * doesn't even belong to the bo_va at all, which is a bug somewhere
+ * else in the bookkeeping.
+ */
+ if (unlikely(entry == mapping)) {
+ drm_err(adev_to_drm(ctx->adev),
+ "Could not find to-be-unmapped mapping in BO mapping lists!\n");
+ return;
+ }
+ entry = list_next_entry(entry, list);
+ }
+ valid = list_entry_is_head(entry, &bo_va->valids, list);
+
+ /* It's unlikely to happen that the mapping userq hasn't been idled
+ * during user requests GEM unmap IOCTL except for forcing the unmap
+ * from user space.
+ */
+ if (unlikely(bo_va->userq_va_mapped))
+ amdgpu_userq_gem_va_unmap_validate(ctx->adev, mapping);
+
+ list_del(&mapping->list);
+ amdgpu_vm_it_remove(mapping, &bo_va->base.vm->va);
+ mapping->bo_va = NULL;
+ trace_amdgpu_vm_bo_unmap(bo_va, mapping);
+
+ if (valid) {
+ /*
+ * BOs that aren't VM_ALWAYS_VALID may have their physical
+ * memory immediately freed when the unmap job finishes, and
+ * we can't trust that userspace won't still try accessing that
+ * memory, therefore we can't sync explicitly. This doesn't
+ * apply to PRT mappings because there's no physical memory for
+ * these at all.
+ */
+ ctx->explicit_sync_unmap &=
+ amdgpu_vm_is_bo_always_valid(ctx->vm, bo_va->base.bo) ||
+ mapping->flags & AMDGPU_VM_PAGE_PRT;
+ list_add(&mapping->list, &ctx->freed);
+ } else {
+ amdgpu_vm_free_mapping(ctx->adev, ctx->vm, mapping,
+ bo_va->last_pt_update);
+ }
+}
+
void amdgpu_vm_update_ctx_commit(struct amdgpu_vm_update_ctx *ctx)
{
struct amdgpu_bo_va_mapping *mapping, *next;
+ /* Users that clear/replace mappings should call amdgpu_vm_update_ctx_execute_replace() */
+ WARN_ON(!list_empty(&ctx->cleared));
+
+ list_for_each_entry_safe(mapping, next, &ctx->deleted, pending_list) {
+ list_del_init(&mapping->pending_list);
+ amdgpu_vm_bo_commit_remove(ctx, mapping);
+ }
list_for_each_entry_safe(mapping, next, &ctx->new, pending_list) {
list_del_init(&mapping->pending_list);
amdgpu_vm_bo_commit_insert(ctx->adev, mapping->bo_va, mapping);
@@ -1652,10 +1716,103 @@ amdgpu_vm_update_ctx_add_new_mapping(struct amdgpu_vm_update_ctx *ctx,
interval_tree_insert(pending, &ctx->pending_maps);
mapping->bo_va = bo_va;
+ WARN_ON(!list_empty(&mapping->pending_list));
list_add(&mapping->pending_list, &ctx->new);
return 0;
}
+static void
+amdgpu_vm_update_ctx_add_replaced_mapping(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_bo_va_mapping *mapping)
+{
+ if (list_empty(&mapping->pending_list))
+ list_add(&mapping->pending_list, &ctx->cleared);
+}
+
+static void
+amdgpu_vm_update_ctx_add_freed_mapping(struct amdgpu_vm_update_ctx *ctx,
+ struct amdgpu_bo_va_mapping *mapping)
+{
+ WARN_ON(!list_empty(&mapping->pending_list));
+ list_add(&mapping->pending_list, &ctx->deleted);
+}
+
+int amdgpu_vm_update_ctx_execute_replace(struct amdgpu_vm_update_ctx *ctx)
+{
+ LIST_HEAD(new_maps);
+ struct interval_tree_node *map;
+ struct amdgpu_bo_va_mapping *cleared, *next, *gap;
+ uint64_t cur_end;
+ int r;
+
+ list_for_each_entry_safe(cleared, next, &ctx->cleared, pending_list) {
+ cur_end = cleared->start;
+
+ map = interval_tree_iter_first(&ctx->cleared_ranges, cleared->start, cleared->last);
+ while (map && map->start <= cleared->last) {
+ if (map->start <= cur_end) {
+ cur_end = map->last + 1;
+ map = interval_tree_iter_next(map, cleared->start, cleared->last);
+ continue;
+ }
+
+ gap = kmalloc_obj(*gap);
+ if (!gap) {
+ r = -ENOMEM;
+ goto error;
+ }
+
+ gap->bo_va = cleared->bo_va;
+ gap->start = cur_end;
+ gap->last = map->start - 1;
+ gap->offset = cleared->offset;
+ gap->offset += (gap->start - cleared->start) << AMDGPU_GPU_PAGE_SHIFT;
+ gap->flags = cleared->flags;
+ list_add(&gap->pending_list, &new_maps);
+
+ cur_end = map->last + 1;
+ map = interval_tree_iter_next(map, cleared->start, cleared->last);
+ }
+
+ if (cur_end <= cleared->last) {
+ gap = kmalloc_obj(*gap);
+ if (!gap) {
+ r = -ENOMEM;
+ goto error;
+ }
+
+ gap->bo_va = cleared->bo_va;
+ gap->start = cur_end;
+ gap->last = cleared->last;
+ gap->offset = cleared->offset;
+ gap->offset += (gap->start - cleared->start) << AMDGPU_GPU_PAGE_SHIFT;
+ gap->flags = cleared->flags;
+ list_add(&gap->pending_list, &new_maps);
+ }
+
+ list_del_init(&cleared->pending_list);
+ amdgpu_vm_update_ctx_add_freed_mapping(ctx, cleared);
+ }
+
+ list_for_each_entry_safe(gap, next, &new_maps, pending_list) {
+ list_del_init(&gap->pending_list);
+ r = amdgpu_vm_update_ctx_add_new_mapping(ctx, gap->bo_va, gap);
+ if (r) {
+ kfree(gap);
+ goto error;
+ }
+ }
+
+ return 0;
+
+error:
+ list_for_each_entry_safe(gap, next, &new_maps, pending_list) {
+ list_del(&gap->pending_list);
+ kfree(gap);
+ }
+ return r;
+}
+
/**
* amdgpu_vm_update_ctx_ensure_unmap_synced - VM update sync helper
* @ctx: context for updating the VM
@@ -1676,7 +1833,9 @@ int amdgpu_vm_update_ctx_ensure_unmap_synced(struct amdgpu_vm_update_ctx *ctx)
*/
r = amdgpu_sync_resv(ctx->adev, &ctx->sync,
ctx->vm->root.bo->tbo.base.resv,
- AMDGPU_SYNC_EQ_OWNER, ctx->vm);
+ ctx->explicit_sync_unmap ? AMDGPU_SYNC_EXPLICIT :
+ AMDGPU_SYNC_EQ_OWNER,
+ ctx->vm);
if (r)
return r;
@@ -1701,8 +1860,11 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
ctx->vm = vm;
ctx->pending_maps = RB_ROOT_CACHED;
+ ctx->cleared_ranges = RB_ROOT_CACHED;
INIT_LIST_HEAD(&ctx->new);
+ INIT_LIST_HEAD(&ctx->deleted);
INIT_LIST_HEAD(&ctx->freed);
+ INIT_LIST_HEAD(&ctx->cleared);
amdgpu_sync_create(&ctx->sync);
}
@@ -1715,17 +1877,23 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
*/
void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
{
- struct interval_tree_node *pending, *next_pending;
+ struct interval_tree_node *node, *next_node;
list_splice(&ctx->freed, &ctx->vm->delayed_freed);
amdgpu_sync_free(&ctx->sync);
- rbtree_postorder_for_each_entry_safe(pending, next_pending, &ctx->pending_maps.rb_root, rb)
- kfree(pending);
+ rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->pending_maps.rb_root, rb) {
+ kfree(node);
+ }
+ rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->cleared_ranges.rb_root, rb) {
+ kfree(node);
+ }
ctx->pending_maps = RB_ROOT_CACHED;
+ ctx->cleared_ranges = RB_ROOT_CACHED;
WARN_ON(!list_empty(&ctx->new));
+ WARN_ON(!list_empty(&ctx->deleted));
}
/**
@@ -1749,6 +1917,9 @@ int amdgpu_vm_clear_freed(struct amdgpu_vm_update_ctx *ctx,
struct dma_fence *f = NULL;
int r;
+ if (list_empty(&ctx->freed))
+ return 0;
+
r = amdgpu_vm_update_ctx_ensure_unmap_synced(ctx);
if (r)
return r;
@@ -2156,8 +2327,6 @@ int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
uint64_t saddr)
{
struct amdgpu_bo_va_mapping *mapping;
- struct amdgpu_vm *vm = ctx->vm;
- bool valid = true;
saddr /= AMDGPU_GPU_PAGE_SIZE;
@@ -2167,8 +2336,6 @@ int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
}
if (&mapping->list == &bo_va->valids) {
- valid = false;
-
list_for_each_entry(mapping, &bo_va->invalids, list) {
if (mapping->start == saddr)
break;
@@ -2178,24 +2345,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
return -ENOENT;
}
- /* It's unlikely to happen that the mapping userq hasn't been idled
- * during user requests GEM unmap IOCTL except for forcing the unmap
- * from user space.
- */
- if (unlikely(bo_va->userq_va_mapped))
- amdgpu_userq_gem_va_unmap_validate(ctx->adev, mapping);
-
- list_del(&mapping->list);
- amdgpu_vm_it_remove(mapping, &vm->va);
- mapping->bo_va = NULL;
- trace_amdgpu_vm_bo_unmap(bo_va, mapping);
-
- if (valid)
- list_add(&mapping->list, &ctx->freed);
- else
- amdgpu_vm_free_mapping(ctx->adev, vm, mapping,
- bo_va->last_pt_update);
-
+ amdgpu_vm_update_ctx_add_freed_mapping(ctx, mapping);
return 0;
}
@@ -2214,8 +2364,8 @@ int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
int amdgpu_vm_bo_clear_mappings(struct amdgpu_vm_update_ctx *ctx,
uint64_t saddr, uint64_t size)
{
- struct amdgpu_bo_va_mapping *before, *after, *tmp, *next;
- LIST_HEAD(removed);
+ struct interval_tree_node *cleared_range;
+ struct amdgpu_bo_va_mapping *tmp;
uint64_t eaddr;
int r;
@@ -2223,96 +2373,23 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_vm_update_ctx *ctx,
if (r)
return r;
- saddr /= AMDGPU_GPU_PAGE_SIZE;
- eaddr = saddr + (size - 1) / AMDGPU_GPU_PAGE_SIZE;
-
- /* Allocate all the needed memory */
- before = kzalloc_obj(*before);
- if (!before)
+ cleared_range = kmalloc_obj(*cleared_range);
+ if (!cleared_range)
return -ENOMEM;
- INIT_LIST_HEAD(&before->list);
- after = kzalloc_obj(*after);
- if (!after) {
- kfree(before);
- return -ENOMEM;
- }
- INIT_LIST_HEAD(&after->list);
+ saddr /= AMDGPU_GPU_PAGE_SIZE;
+ eaddr = saddr + (size - 1) / AMDGPU_GPU_PAGE_SIZE;
/* Now gather all removed mappings */
tmp = amdgpu_vm_it_iter_first(&ctx->vm->va, saddr, eaddr);
while (tmp) {
- /* Remember mapping split at the start */
- if (tmp->start < saddr) {
- before->start = tmp->start;
- before->last = saddr - 1;
- before->offset = tmp->offset;
- before->flags = tmp->flags;
- before->bo_va = tmp->bo_va;
- list_add(&before->list, &tmp->bo_va->invalids);
- }
-
- /* Remember mapping split at the end */
- if (tmp->last > eaddr) {
- after->start = eaddr + 1;
- after->last = tmp->last;
- after->offset = tmp->offset;
- after->offset += (after->start - tmp->start) << AMDGPU_GPU_PAGE_SHIFT;
- after->flags = tmp->flags;
- after->bo_va = tmp->bo_va;
- list_add(&after->list, &tmp->bo_va->invalids);
- }
-
- list_del(&tmp->list);
- list_add(&tmp->list, &removed);
-
+ amdgpu_vm_update_ctx_add_replaced_mapping(ctx, tmp);
tmp = amdgpu_vm_it_iter_next(tmp, saddr, eaddr);
}
- /* And free them up */
- list_for_each_entry_safe(tmp, next, &removed, list) {
- amdgpu_vm_it_remove(tmp, &ctx->vm->va);
- list_del(&tmp->list);
-
- if (tmp->start < saddr)
- tmp->start = saddr;
- if (tmp->last > eaddr)
- tmp->last = eaddr;
-
- tmp->bo_va = NULL;
- list_add(&tmp->list, &ctx->freed);
- trace_amdgpu_vm_bo_unmap(NULL, tmp);
- }
-
- /* Insert partial mapping before the range */
- if (!list_empty(&before->list)) {
- struct amdgpu_bo *bo = before->bo_va->base.bo;
-
- amdgpu_vm_it_insert(before, &ctx->vm->va);
- if (before->flags & AMDGPU_VM_PAGE_PRT)
- amdgpu_vm_prt_get(ctx->adev);
-
- if (amdgpu_vm_is_bo_always_valid(ctx->vm, bo) &&
- !before->bo_va->base.moved)
- amdgpu_vm_bo_needs_update(&before->bo_va->base);
- } else {
- kfree(before);
- }
-
- /* Insert partial mapping after the range */
- if (!list_empty(&after->list)) {
- struct amdgpu_bo *bo = after->bo_va->base.bo;
-
- amdgpu_vm_it_insert(after, &ctx->vm->va);
- if (after->flags & AMDGPU_VM_PAGE_PRT)
- amdgpu_vm_prt_get(ctx->adev);
-
- if (amdgpu_vm_is_bo_always_valid(ctx->vm, bo) &&
- !after->bo_va->base.moved)
- amdgpu_vm_bo_needs_update(&after->bo_va->base);
- } else {
- kfree(after);
- }
+ cleared_range->start = saddr;
+ cleared_range->last = eaddr;
+ interval_tree_insert(cleared_range, &ctx->cleared_ranges);
return 0;
}
@@ -2409,13 +2486,8 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev,
amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
- list_for_each_entry_safe(mapping, next, &bo_va->valids, list) {
- list_del(&mapping->list);
- amdgpu_vm_it_remove(mapping, &vm->va);
- mapping->bo_va = NULL;
- trace_amdgpu_vm_bo_unmap(bo_va, mapping);
- list_add(&mapping->list, &update_ctx.freed);
- }
+ list_for_each_entry_safe(mapping, next, &bo_va->valids, list)
+ amdgpu_vm_update_ctx_add_freed_mapping(&update_ctx, mapping);
list_for_each_entry_safe(mapping, next, &bo_va->invalids, list) {
list_del(&mapping->list);
amdgpu_vm_it_remove(mapping, &vm->va);
@@ -2423,6 +2495,7 @@ void amdgpu_vm_bo_del(struct amdgpu_device *adev,
bo_va->last_pt_update);
}
+ amdgpu_vm_update_ctx_commit(&update_ctx);
amdgpu_vm_update_ctx_fini(&update_ctx);
dma_fence_put(bo_va->last_pt_update);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index d35494440cb5a..5657aa63e9491 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -284,11 +284,27 @@ struct amdgpu_vm_update_ctx {
*/
struct rb_root_cached pending_maps;
+ /**
+ * @cleared_ranges: Interval tree of VA ranges cleared by this update operation.
+ */
+ struct rb_root_cached cleared_ranges;
+
/**
* @new: Pending new mappings not yet applied to bookkeeping structures.
*/
struct list_head new;
+ /**
+ * @deleted: Pending freed mappings not yet applied to bookkeeping structures.
+ */
+ struct list_head deleted;
+
+ /**
+ * @replaced: Pending mappings that have been cleared, potentially partially,
+ * and are not yet applied to bookkeeping structures.
+ */
+ struct list_head cleared;
+
/**
* @freed: A list of mappings that are freed and pending PT/PD updates during
* this update operation.
@@ -305,6 +321,13 @@ struct amdgpu_vm_update_ctx {
* i.e. all previous submissions in the VM are waited on.
*/
bool unmap_synced;
+
+ /**
+ * @explicit_sync_unmap: Whether it's allowed to sync explicitly (i.e. only wait
+ * on explicit dependencies instead of every submission that may have used the BO)
+ * before committing unmaps.
+ */
+ bool explicit_sync_unmap;
};
struct amdgpu_vm_fault_info {
@@ -593,6 +616,7 @@ bool amdgpu_vm_is_bo_always_valid(struct amdgpu_vm *vm, struct amdgpu_bo *bo);
void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_device *adev,
struct amdgpu_vm *vm);
+int amdgpu_vm_update_ctx_execute_replace(struct amdgpu_vm_update_ctx *ctx);
void amdgpu_vm_update_ctx_commit(struct amdgpu_vm_update_ctx *ctx);
void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx);
int amdgpu_vm_update_ctx_ensure_unmap_synced(struct amdgpu_vm_update_ctx *ctx);
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 05/10] drm/amdgpu: Track dirty bo_va structures in update ctx
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (3 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 04/10] drm/amdgpu: Make removed/cleared mappings " Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 14:28 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 06/10] drm/amdgpu: Add function to roll back a VM " Natalie Vock
` (4 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
So we can later update all changed BOs without needing to do tracking
separately.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 16 ++++++++++++++--
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 5 +++++
3 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index e0090dc10682f..068c57ed537fd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -80,6 +80,8 @@ struct amdgpu_bo_va {
/* protected by bo being reserved */
unsigned ref_count;
+ struct list_head update_link;
+
/* all other members protected by the VM PD being reserved */
struct dma_fence *last_pt_update;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 1e2d45979efdb..330c288053352 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1718,6 +1718,8 @@ amdgpu_vm_update_ctx_add_new_mapping(struct amdgpu_vm_update_ctx *ctx,
mapping->bo_va = bo_va;
WARN_ON(!list_empty(&mapping->pending_list));
list_add(&mapping->pending_list, &ctx->new);
+ if (list_empty(&mapping->bo_va->update_link))
+ list_add(&mapping->bo_va->update_link, &ctx->dirty_bo_vas);
return 0;
}
@@ -1733,8 +1735,10 @@ static void
amdgpu_vm_update_ctx_add_freed_mapping(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_bo_va_mapping *mapping)
{
- WARN_ON(!list_empty(&mapping->pending_list));
- list_add(&mapping->pending_list, &ctx->deleted);
+ if (list_empty(&mapping->bo_va->update_link))
+ list_add(&mapping->bo_va->update_link, &ctx->dirty_bo_vas);
+ if (list_empty(&mapping->pending_list))
+ list_add(&mapping->pending_list, &ctx->deleted);
}
int amdgpu_vm_update_ctx_execute_replace(struct amdgpu_vm_update_ctx *ctx)
@@ -1861,6 +1865,7 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
ctx->pending_maps = RB_ROOT_CACHED;
ctx->cleared_ranges = RB_ROOT_CACHED;
+ INIT_LIST_HEAD(&ctx->dirty_bo_vas);
INIT_LIST_HEAD(&ctx->new);
INIT_LIST_HEAD(&ctx->deleted);
INIT_LIST_HEAD(&ctx->freed);
@@ -1878,10 +1883,16 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
{
struct interval_tree_node *node, *next_node;
+ struct amdgpu_bo_va *bo_va, *next_bo_va;
list_splice(&ctx->freed, &ctx->vm->delayed_freed);
amdgpu_sync_free(&ctx->sync);
+ list_for_each_entry_safe(bo_va, next_bo_va, &ctx->dirty_bo_vas,
+ update_link) {
+ list_del_init(&bo_va->update_link);
+ }
+
rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->pending_maps.rb_root, rb) {
kfree(node);
}
@@ -2135,6 +2146,7 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
bo_va->ref_count = 1;
bo_va->last_pt_update = dma_fence_get_stub();
+ INIT_LIST_HEAD(&bo_va->update_link);
INIT_LIST_HEAD(&bo_va->valids);
INIT_LIST_HEAD(&bo_va->invalids);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index 5657aa63e9491..7a5f558970b5c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -305,6 +305,11 @@ struct amdgpu_vm_update_ctx {
*/
struct list_head cleared;
+ /**
+ * @dirty_bo_vas: amdgpu_bo_va structures touched by current pending mapping changes.
+ */
+ struct list_head dirty_bo_vas;
+
/**
* @freed: A list of mappings that are freed and pending PT/PD updates during
* this update operation.
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 06/10] drm/amdgpu: Add function to roll back a VM update ctx
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (4 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 05/10] drm/amdgpu: Track dirty bo_va structures in update ctx Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 14:40 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 07/10] drm/amdgpu: Move some VA op validation to a helper Natalie Vock
` (3 subsequent siblings)
9 siblings, 1 reply; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
Used for discarding pending operations and retrying. This is needed when
we do lots of mapping operations at once and drm_exec tells us to drop
everything and retry to avoid deadlocks.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 41 +++++++++++++++++++++-------------
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 1 +
2 files changed, 27 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 330c288053352..ac4a1ed9b1196 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1874,37 +1874,48 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
}
/**
- * amdgpu_vm_update_ctx_fini - finish a VM update operation
- * @ctx: The context of the update
+ * amdgpu_vm_update_ctx_rollback - clear pending VM updates
+ * @ctx: The context to roll back
*
- * Adds any leftover mappings needing to be cleared to the VM's delayed free list,
- * then frees up resources used by the context.
+ * Cleans up any pending mapping changes, but does not reset
+ * anything else (like attached syncobjs).
*/
-void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
+void amdgpu_vm_update_ctx_rollback(struct amdgpu_vm_update_ctx *ctx)
{
+ struct amdgpu_bo_va_mapping *mapping, *next;
struct interval_tree_node *node, *next_node;
struct amdgpu_bo_va *bo_va, *next_bo_va;
- list_splice(&ctx->freed, &ctx->vm->delayed_freed);
- amdgpu_sync_free(&ctx->sync);
+ list_for_each_entry_safe(mapping, next, &ctx->new, pending_list)
+ list_del_init(&mapping->pending_list);
+ list_for_each_entry_safe(mapping, next, &ctx->deleted, pending_list)
+ list_del_init(&mapping->pending_list);
list_for_each_entry_safe(bo_va, next_bo_va, &ctx->dirty_bo_vas,
- update_link) {
+ update_link)
list_del_init(&bo_va->update_link);
- }
- rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->pending_maps.rb_root, rb) {
+ rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->pending_maps.rb_root, rb)
kfree(node);
- }
- rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->cleared_ranges.rb_root, rb) {
+ rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->cleared_ranges.rb_root, rb)
kfree(node);
- }
ctx->pending_maps = RB_ROOT_CACHED;
ctx->cleared_ranges = RB_ROOT_CACHED;
+}
- WARN_ON(!list_empty(&ctx->new));
- WARN_ON(!list_empty(&ctx->deleted));
+/**
+ * amdgpu_vm_update_ctx_fini - finish a VM update operation
+ * @ctx: The context of the update
+ *
+ * Adds any leftover mappings needing to be cleared to the VM's delayed free list,
+ * then frees up resources used by the context.
+ */
+void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
+{
+ amdgpu_vm_update_ctx_rollback(ctx);
+ list_splice(&ctx->freed, &ctx->vm->delayed_freed);
+ amdgpu_sync_free(&ctx->sync);
}
/**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index 7a5f558970b5c..8d4f59c286c2a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -623,6 +623,7 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
struct amdgpu_vm *vm);
int amdgpu_vm_update_ctx_execute_replace(struct amdgpu_vm_update_ctx *ctx);
void amdgpu_vm_update_ctx_commit(struct amdgpu_vm_update_ctx *ctx);
+void amdgpu_vm_update_ctx_rollback(struct amdgpu_vm_update_ctx *ctx);
void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx);
int amdgpu_vm_update_ctx_ensure_unmap_synced(struct amdgpu_vm_update_ctx *ctx);
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 07/10] drm/amdgpu: Move some VA op validation to a helper
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (5 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 06/10] drm/amdgpu: Add function to roll back a VM " Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 12:50 ` [PATCH v2 08/10] drm/amdgpu: Only reject conflicting mappings if they are not cleared Natalie Vock
` (2 subsequent siblings)
9 siblings, 0 replies; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
To be reused in a new VA ioctl in a subsequent patch.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 71 +++++++++++++++++++--------------
1 file changed, 40 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 39c72bfb73e71..fb7abb095b8ec 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -878,9 +878,11 @@ amdgpu_gem_va_update_vm(struct amdgpu_vm_update_ctx *ctx,
return fence;
}
-int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
- struct drm_file *filp)
+static int amdgpu_gem_va_validate_args(struct amdgpu_device *adev, uint64_t va,
+ uint64_t size, uint32_t flags)
{
+ uint64_t vm_size, map_end;
+
const uint32_t valid_flags = AMDGPU_VM_DELAY_UPDATE |
AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_PAGE_WRITEABLE |
AMDGPU_VM_PAGE_EXECUTABLE | AMDGPU_VM_MTYPE_MASK |
@@ -888,53 +890,60 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
const uint32_t prt_flags = AMDGPU_VM_DELAY_UPDATE |
AMDGPU_VM_PAGE_PRT;
- struct drm_amdgpu_gem_va *args = data;
- struct drm_gem_object *gobj;
- struct amdgpu_device *adev = drm_to_adev(dev);
- struct amdgpu_fpriv *fpriv = filp->driver_priv;
- struct amdgpu_bo *abo;
- struct amdgpu_bo_va *bo_va;
- struct drm_syncobj *timeline_syncobj = NULL;
- struct dma_fence_chain *timeline_chain = NULL;
- struct amdgpu_vm_update_ctx update_ctx;
- struct drm_exec exec;
- uint64_t vm_size, tmp;
- int r = 0;
-
- /* Validate virtual address range against reserved regions. */
- if (args->va_address < AMDGPU_VA_RESERVED_BOTTOM) {
- dev_dbg(dev->dev,
+ if (va < AMDGPU_VA_RESERVED_BOTTOM) {
+ dev_dbg(adev_to_drm(adev)->dev,
"va_address 0x%llx is in reserved area 0x%llx\n",
- args->va_address, AMDGPU_VA_RESERVED_BOTTOM);
+ va, AMDGPU_VA_RESERVED_BOTTOM);
return -EINVAL;
}
- if (args->va_address >= AMDGPU_GMC_HOLE_START &&
- args->va_address < AMDGPU_GMC_HOLE_END) {
- dev_dbg(dev->dev,
+ if (va >= AMDGPU_GMC_HOLE_START && va < AMDGPU_GMC_HOLE_END) {
+ dev_dbg(adev_to_drm(adev)->dev,
"va_address 0x%llx is in VA hole 0x%llx-0x%llx\n",
- args->va_address, AMDGPU_GMC_HOLE_START,
+ va, AMDGPU_GMC_HOLE_START,
AMDGPU_GMC_HOLE_END);
return -EINVAL;
}
- args->va_address &= AMDGPU_GMC_HOLE_MASK;
-
vm_size = adev->vm_manager.max_pfn * AMDGPU_GPU_PAGE_SIZE;
vm_size -= AMDGPU_VA_RESERVED_TOP;
- if (check_add_overflow(args->va_address, args->map_size, &tmp) || tmp > vm_size) {
- dev_dbg(dev->dev,
+ if (check_add_overflow(va & AMDGPU_GMC_HOLE_MASK, size, &map_end) || map_end > vm_size) {
+ dev_dbg(adev_to_drm(adev)->dev,
"va_address 0x%llx is in top reserved area 0x%llx\n",
- args->va_address + args->map_size, vm_size);
+ va + size, vm_size);
return -EINVAL;
}
- if ((args->flags & ~valid_flags) && (args->flags & ~prt_flags)) {
- dev_dbg(dev->dev, "invalid flags combination 0x%08X\n",
- args->flags);
+ if ((flags & ~valid_flags) && (flags & ~prt_flags)) {
+ dev_dbg(adev_to_drm(adev)->dev, "invalid flags combination 0x%08X\n",
+ flags);
return -EINVAL;
}
+ return 0;
+}
+
+int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp)
+{
+ struct drm_amdgpu_gem_va *args = data;
+ struct drm_gem_object *gobj;
+ struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_fpriv *fpriv = filp->driver_priv;
+ struct amdgpu_bo *abo;
+ struct amdgpu_bo_va *bo_va;
+ struct drm_syncobj *timeline_syncobj = NULL;
+ struct dma_fence_chain *timeline_chain = NULL;
+ struct amdgpu_vm_update_ctx update_ctx;
+ struct drm_exec exec;
+ int r = 0;
+
+ r = amdgpu_gem_va_validate_args(adev, args->va_address, args->map_size, args->flags);
+ if (r)
+ return r;
+
+ args->va_address &= AMDGPU_GMC_HOLE_MASK;
+
/* Validate operation type. */
switch (args->operation) {
case AMDGPU_VA_OP_MAP:
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 08/10] drm/amdgpu: Only reject conflicting mappings if they are not cleared
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (6 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 07/10] drm/amdgpu: Move some VA op validation to a helper Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 15:06 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl Natalie Vock
2026-11-17 12:50 ` [PATCH v2 10/10] drm/amdgpu: Bump minor version Natalie Vock
9 siblings, 1 reply; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
Cleared mappings will not immediately disappear from the VM interval
tree, instead they will be in a pending list. For the REPLACE_MAPPING
op to work correctly, skip rejecting these conflicts as they will be
properly resolved later.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index ac4a1ed9b1196..491f4b50f7602 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2241,7 +2241,7 @@ int amdgpu_vm_bo_map(struct amdgpu_vm_update_ctx *ctx,
eaddr = saddr + (size - 1) / AMDGPU_GPU_PAGE_SIZE;
tmp = amdgpu_vm_it_iter_first(&vm->va, saddr, eaddr);
- if (tmp) {
+ if (tmp && list_empty(&tmp->pending_list)) {
/* bo and tmp overlap, invalid addr */
dev_err(ctx->adev->dev,
"bo %p va 0x%010Lx-0x%010Lx conflict with "
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (7 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 08/10] drm/amdgpu: Only reject conflicting mappings if they are not cleared Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
2026-11-17 15:34 ` sashiko-bot
2026-11-17 22:46 ` Dave Airlie
2026-11-17 12:50 ` [PATCH v2 10/10] drm/amdgpu: Bump minor version Natalie Vock
9 siblings, 2 replies; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
This new ioctl is a modernized version of AMDGPU_GEM_VA, with the
main changes being support for many operations batched in one ioctl,
and a better sync model that allows passing multiple syncobjs for
both wait and signal. Timeline syncobj are now properly supported in
both wait and signal cases.
One breaking change compared to AMDGPU_GEM_VA is that
synchronization is now explicit (where possible), meaning that
previous submissions are not necessarily waited on.
Using this ioctl allows applications that hammer lots of sparse
operations, particularly some games running under Proton, to see
significant reductions in stuttering.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 3 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 348 +++++++++++++++++++++++++++++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h | 2 +
include/uapi/drm/amdgpu_drm.h | 46 +++++
4 files changed, 396 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 72b3710462824..188c8ec234c02 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -3124,7 +3124,8 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = {
DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_WAIT, amdgpu_userq_wait_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(AMDGPU_GEM_LIST_HANDLES, amdgpu_gem_list_handles_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(AMDGPU_PROC_OPTIONS, amdgpu_proc_options_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
- DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW)
+ DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(AMDGPU_GEM_VA2, amdgpu_gem_va2_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
};
static const struct drm_driver amdgpu_kms_driver = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index fb7abb095b8ec..0d77103b0a973 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -53,10 +53,13 @@ static int
amdgpu_gem_add_input_fence(struct drm_file *filp,
struct amdgpu_vm_update_ctx *update_ctx,
uint64_t syncobj_handles_array,
+ uint64_t syncobj_points_array,
uint32_t num_syncobj_handles)
{
struct dma_fence *fence;
uint32_t *syncobj_handles;
+ uint64_t *syncobj_points;
+ uint64_t point;
int ret, i;
if (!num_syncobj_handles)
@@ -67,6 +70,17 @@ amdgpu_gem_add_input_fence(struct drm_file *filp,
if (IS_ERR(syncobj_handles))
return PTR_ERR(syncobj_handles);
+ if (syncobj_points_array) {
+ syncobj_points = memdup_user(u64_to_user_ptr(syncobj_points_array),
+ size_mul(sizeof(uint64_t), num_syncobj_handles));
+ if (IS_ERR(syncobj_points)) {
+ kfree(syncobj_handles);
+ return PTR_ERR(syncobj_handles);
+ }
+ } else {
+ syncobj_points = NULL;
+ }
+
for (i = 0; i < num_syncobj_handles; i++) {
if (!syncobj_handles[i]) {
@@ -74,7 +88,9 @@ amdgpu_gem_add_input_fence(struct drm_file *filp,
goto free_memdup;
}
- ret = drm_syncobj_find_fence(filp, syncobj_handles[i], 0, 0, &fence);
+ point = syncobj_points ? syncobj_points[i] : 0;
+
+ ret = drm_syncobj_find_fence(filp, syncobj_handles[i], point, 0, &fence);
if (ret)
goto free_memdup;
@@ -87,9 +103,111 @@ amdgpu_gem_add_input_fence(struct drm_file *filp,
free_memdup:
kfree(syncobj_handles);
+ kfree(syncobj_points);
return ret;
}
+struct amdgpu_gem_timeline_out {
+ struct drm_syncobj **syncobjs;
+ struct dma_fence_chain **chains;
+ uint64_t *timeline_points;
+};
+
+static int amdgpu_gem_prepare_timeline_nodes(
+ struct drm_file *filp, uint64_t syncobj_handles_array,
+ uint64_t syncobj_points_array, uint32_t num_syncobjs,
+ struct amdgpu_gem_timeline_out *out)
+{
+ uint32_t *syncobj_handles;
+ uint32_t i;
+ int r;
+
+ syncobj_handles = memdup_user(u64_to_user_ptr(syncobj_handles_array),
+ size_mul(sizeof(uint32_t), num_syncobjs));
+ if (IS_ERR(syncobj_handles))
+ return PTR_ERR(syncobj_handles);
+
+ out->timeline_points =
+ memdup_user(u64_to_user_ptr(syncobj_points_array),
+ size_mul(sizeof(uint64_t), num_syncobjs));
+ if (IS_ERR(out->timeline_points)) {
+ r = PTR_ERR(out->timeline_points);
+ goto err_free_handles;
+ }
+
+ out->chains = kzalloc_objs(*out->chains, num_syncobjs);
+ if (!out->chains) {
+ r = -ENOMEM;
+ goto err_free_points;
+ }
+
+ out->syncobjs = kzalloc_objs(*out->syncobjs, num_syncobjs);
+ if (!out->syncobjs) {
+ r = -ENOMEM;
+ goto err_free_chains;
+ }
+
+ for (i = 0; i < num_syncobjs; i++) {
+ out->syncobjs[i] = drm_syncobj_find(filp, syncobj_handles[i]);
+ if (!out->syncobjs[i]) {
+ r = -ENOENT;
+ goto err_loop_rollback;
+ }
+
+ if (!out->timeline_points[i])
+ continue;
+
+ out->chains[i] = dma_fence_chain_alloc();
+ if (!out->chains[i]) {
+ r = -ENOMEM;
+ goto err_loop_rollback;
+ }
+
+ continue;
+err_loop_rollback:
+ for (;;) {
+ if (out->syncobjs[i])
+ drm_syncobj_put(out->syncobjs[i]);
+ if (out->chains[i])
+ dma_fence_chain_free(out->chains[i]);
+ if (i-- == 0)
+ goto err_free_sync;
+ }
+ }
+
+ return 0;
+
+err_free_sync:
+ kfree(out->syncobjs);
+ out->syncobjs = NULL;
+err_free_chains:
+ kfree(out->chains);
+ out->chains = NULL;
+err_free_points:
+ kfree(out->timeline_points);
+ out->timeline_points = NULL;
+err_free_handles:
+ kfree(syncobj_handles);
+ return r;
+}
+
+static void amdgpu_gem_timeline_out_fini(struct amdgpu_gem_timeline_out *tl_out,
+ uint32_t num_syncobjs)
+{
+ uint32_t i;
+
+ for (i = 0; i < num_syncobjs; i++) {
+ if (tl_out->syncobjs)
+ drm_syncobj_put(tl_out->syncobjs[i]);
+ if (tl_out->chains)
+ dma_fence_chain_free(tl_out->chains[i]);
+ }
+
+ kfree(tl_out->timeline_points);
+ kfree(tl_out->syncobjs);
+ kfree(tl_out->chains);
+}
+
static int
amdgpu_gem_update_timeline_node(struct drm_file *filp,
uint32_t syncobj_handle,
@@ -982,7 +1100,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
/* Add input syncobj fences (if any) for synchronization. */
r = amdgpu_gem_add_input_fence(filp, &update_ctx,
- args->input_fence_syncobj_handles,
+ args->input_fence_syncobj_handles, 0,
args->num_syncobj_handles);
if (r)
goto error_put_gobj;
@@ -1100,6 +1218,232 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
return r;
}
+int amdgpu_gem_va2_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp)
+{
+ struct drm_amdgpu_gem_va2 *args = data;
+ struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_fpriv *fpriv = filp->driver_priv;
+ struct drm_amdgpu_gem_va_mapping *mappings;
+ struct amdgpu_gem_timeline_out tl_out;
+ struct amdgpu_vm_update_ctx update_ctx;
+ struct drm_gem_object *gobj;
+ struct amdgpu_bo_va *bo_va;
+ struct drm_exec exec;
+ bool delay = false;
+ u32 i;
+ int r;
+
+ mappings = vmemdup_array_user(u64_to_user_ptr(args->mappings),
+ args->num_mappings,
+ sizeof(struct drm_amdgpu_gem_va_mapping));
+ if (IS_ERR(mappings))
+ return PTR_ERR(mappings);
+
+ r = amdgpu_gem_prepare_timeline_nodes(filp, args->syncobjs_out,
+ args->timeline_points_out,
+ args->num_syncobjs_out, &tl_out);
+ if (r)
+ goto out_free_mappings;
+
+ drm_exec_init(&exec,
+ DRM_EXEC_INTERRUPTIBLE_WAIT | DRM_EXEC_IGNORE_DUPLICATES,
+ 0);
+ amdgpu_vm_update_ctx_init(&update_ctx, adev, &fpriv->vm);
+ update_ctx.explicit_sync_unmap = true;
+
+ r = amdgpu_gem_add_input_fence(filp, &update_ctx, args->syncobjs_in,
+ args->timeline_points_in,
+ args->num_syncobjs_in);
+ if (r)
+ goto out;
+
+ drm_exec_until_all_locked(&exec)
+ {
+ r = amdgpu_vm_lock_pd(&fpriv->vm, &exec, 2);
+ drm_exec_retry_on_contention(&exec);
+ if (unlikely(r))
+ goto out;
+
+ for (i = 0; i < args->num_mappings; ++i) {
+ r = amdgpu_gem_va_validate_args(adev, mappings[i].va,
+ mappings[i].map_size,
+ mappings[i].flags);
+ if (r)
+ goto out;
+
+ mappings[i].va &= AMDGPU_GMC_HOLE_MASK;
+
+ if (mappings[i].flags & AMDGPU_VM_DELAY_UPDATE) {
+ /* Can't delay if we're supposed to signal fences */
+ if (args->num_syncobjs_out) {
+ r = -EINVAL;
+ goto out;
+ }
+ /* If one mapping is delayed, all mappings must be delayed */
+ if (i && !delay) {
+ r = -EINVAL;
+ goto out;
+ }
+ } else if (delay) {
+ /* If one mapping is not delayed, no mappings may be delayed */
+ r = -EINVAL;
+ goto out;
+ }
+
+ if ((mappings[i].operation != AMDGPU_VA_OP_CLEAR) &&
+ !(mappings[i].flags & AMDGPU_VM_PAGE_PRT)) {
+ gobj = drm_gem_object_lookup(
+ filp, mappings[i].handle);
+ if (gobj == NULL) {
+ r = -ENOENT;
+ goto out;
+ }
+ bo_va = amdgpu_vm_bo_find(
+ &fpriv->vm, gem_to_amdgpu_bo(gobj));
+ if (!bo_va) {
+ drm_gem_object_put(gobj);
+ r = -ENOENT;
+ goto out;
+ }
+ } else {
+ gobj = NULL;
+ bo_va = mappings[i].operation !=
+ AMDGPU_VA_OP_CLEAR ?
+ fpriv->prt_va :
+ NULL;
+ }
+
+ if (gobj) {
+ r = drm_exec_lock_obj(&exec, gobj);
+ if (drm_exec_is_contended(&exec))
+ amdgpu_vm_update_ctx_rollback(
+ &update_ctx);
+ drm_exec_retry_on_contention(&exec);
+ if (unlikely(r))
+ goto out;
+ }
+
+ switch (mappings[i].operation) {
+ case AMDGPU_VA_OP_MAP:
+ r = amdgpu_vm_bo_map(&update_ctx, bo_va,
+ mappings[i].va,
+ mappings[i].offset_in_bo,
+ mappings[i].map_size,
+ mappings[i].flags);
+ break;
+ case AMDGPU_VA_OP_UNMAP:
+ r = amdgpu_vm_bo_unmap(&update_ctx, bo_va,
+ mappings[i].flags);
+ break;
+
+ case AMDGPU_VA_OP_CLEAR:
+ r = amdgpu_vm_bo_clear_mappings(
+ &update_ctx, mappings[i].va,
+ mappings[i].map_size);
+ break;
+ case AMDGPU_VA_OP_REPLACE:
+ r = amdgpu_vm_bo_replace_map(
+ &update_ctx, bo_va, mappings[i].va,
+ mappings[i].offset_in_bo,
+ mappings[i].map_size,
+ mappings[i].flags);
+ break;
+ default:
+ r = -EINVAL;
+ break;
+ }
+
+ /* It's fine to put our reference here, exec still holds one */
+ drm_gem_object_put(gobj);
+
+ if (r)
+ goto out;
+ }
+ }
+
+ r = amdgpu_vm_update_ctx_execute_replace(&update_ctx);
+ if (r)
+ goto out;
+
+ amdgpu_vm_update_ctx_commit(&update_ctx);
+
+ if (!delay && (!adev->debug_vm || args->num_syncobjs_out)) {
+ struct dma_fence *fence, *merged;
+
+ if (!amdgpu_vm_ready(&fpriv->vm)) {
+ fence = dma_fence_get_stub();
+ goto insert_fence;
+ }
+
+ r = amdgpu_vm_clear_freed(&update_ctx, &fpriv->vm.last_update);
+ if (r) {
+ drm_err(adev_to_drm(adev),
+ "Could not clear freed PT entries: %d\n", r);
+ goto insert_fence;
+ }
+
+ fence = dma_fence_get(fpriv->vm.last_update);
+
+ list_for_each_entry(bo_va, &update_ctx.dirty_bo_vas,
+ update_link) {
+ r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
+ if (r) {
+ drm_err(adev_to_drm(adev),
+ "Could not update BO PT entries: %d\n", r);
+ goto insert_fence;
+ }
+ }
+
+ list_for_each_entry(bo_va, &update_ctx.dirty_bo_vas,
+ update_link) {
+ if (!amdgpu_vm_is_bo_always_valid(&fpriv->vm,
+ bo_va->base.bo)) {
+ dma_fence_put(bo_va->last_pt_update);
+ bo_va->last_pt_update = dma_fence_get(fence);
+ }
+ }
+
+ /* Always update PDEs after we touched the mappings. */
+ r = amdgpu_vm_update_pdes(adev, &fpriv->vm, false);
+ if (r)
+ goto insert_fence;
+
+ merged = dma_fence_unwrap_merge(fence, fpriv->vm.last_update);
+ if (!merged) {
+ /* As fallback in OOM situations */
+ dma_fence_wait(fpriv->vm.last_update, false);
+ dma_fence_wait(fence, false);
+ merged = dma_fence_get_stub();
+ }
+ fence = merged;
+
+insert_fence:
+ for (i = 0; i < args->num_syncobjs_out; ++i) {
+ if (!tl_out.timeline_points[i]) {
+ /* Replace the existing fence when no point is given. */
+ drm_syncobj_replace_fence(tl_out.syncobjs[i],
+ fence);
+ } else {
+ /* Attach the last-update fence at a specific point. */
+ drm_syncobj_add_point(
+ tl_out.syncobjs[i], tl_out.chains[i],
+ fence, tl_out.timeline_points[i]);
+ tl_out.chains[i] = NULL;
+ }
+ }
+ dma_fence_put(fence);
+ }
+
+out:
+ amdgpu_vm_update_ctx_fini(&update_ctx);
+ drm_exec_fini(&exec);
+ amdgpu_gem_timeline_out_fini(&tl_out, args->num_syncobjs_out);
+out_free_mappings:
+ kvfree(mappings);
+ return r;
+}
+
int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp)
{
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
index b558336bc4c6c..38fafc978c834 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
@@ -65,6 +65,8 @@ int amdgpu_gem_wait_idle_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp);
int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp);
+int amdgpu_gem_va2_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp);
int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
struct drm_file *filp);
int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 29009f6aabde9..c9e52bfa7d028 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -60,6 +60,7 @@ extern "C" {
#define DRM_AMDGPU_GEM_LIST_HANDLES 0x19
#define DRM_AMDGPU_PROC_OPTIONS 0x1A
#define DRM_AMDGPU_UALINK_HANDLE 0x1B
+#define DRM_AMDGPU_GEM_VA2 0x1C
#define DRM_IOCTL_AMDGPU_GEM_CREATE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create)
#define DRM_IOCTL_AMDGPU_GEM_MMAP DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap)
@@ -83,6 +84,7 @@ extern "C" {
#define DRM_IOCTL_AMDGPU_GEM_LIST_HANDLES DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_LIST_HANDLES, struct drm_amdgpu_gem_list_handles)
#define DRM_IOCTL_AMDGPU_PROC_OPTIONS DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_PROC_OPTIONS, struct drm_amdgpu_proc_options)
#define DRM_IOCTL_AMDGPU_UALINK_HANDLE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_UALINK_HANDLE, union drm_amdgpu_ualink_handle)
+#define DRM_IOCTL_AMDGPU_GEM_VA2 DRM_IOW(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_VA2, struct drm_amdgpu_gem_va)
/**
* DOC: memory domains
@@ -945,6 +947,50 @@ struct drm_amdgpu_gem_va {
__u64 input_fence_syncobj_handles;
};
+struct drm_amdgpu_gem_va_mapping {
+ /** GEM object handle */
+ __u32 handle;
+ /** AMDGPU_VA_OP_* */
+ __u32 operation;
+ /** AMDGPU_VM_PAGE_* */
+ __u32 flags;
+ __u32 _pad;
+
+ /** virtual address to assign . Must be correctly aligned.*/
+ __u64 va;
+ /** Specify offset inside of BO to assign. Must be correctly aligned.*/
+ __u64 offset_in_bo;
+ /** Specify mapping size. Must be correctly aligned. */
+ __u64 map_size;
+};
+
+struct drm_amdgpu_gem_va2 {
+ /** Number of mapping ops in @mappings */
+ __u32 num_mappings;
+ __u32 _pad;
+ /** Pointer to array of struct drm_amdgpu_gem_va_mapping */
+ __u64 mappings;
+
+ /** Number of elements in @syncobjs_in and @timeline_points_in */
+ __u32 num_syncobjs_in;
+ /** Number of elements in @syncobjs_out and @timeline_points_out */
+ __u32 num_syncobjs_out;
+ /** Array of u32 syncobj handles to wait for before mapping */
+ __u64 syncobjs_in;
+ /**
+ * Array of timeline points, one for each object in @syncobjs_in.
+ * If the syncobj is binary, its corrensponding timeline point must be 0.
+ */
+ __u64 timeline_points_in;
+ /** Array of u32 syncobj handles to signal after mapping */
+ __u64 syncobjs_out;
+ /**
+ * Array of timeline points, one for each object in @syncobjs_out.
+ * If the syncobj is binary, its corrensponding timeline point must be 0.
+ */
+ __u64 timeline_points_out;
+};
+
#define AMDGPU_HW_IP_GFX 0
#define AMDGPU_HW_IP_COMPUTE 1
#define AMDGPU_HW_IP_DMA 2
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v2 10/10] drm/amdgpu: Bump minor version
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
` (8 preceding siblings ...)
2026-11-17 12:50 ` [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl Natalie Vock
@ 2026-11-17 12:50 ` Natalie Vock
9 siblings, 0 replies; 19+ messages in thread
From: Natalie Vock @ 2026-11-17 12:50 UTC (permalink / raw)
To: Felix Kuehling, Alex Deucher, Christian König, David Airlie,
Simona Vetter, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, Timur Kristóf
Cc: amd-gfx, dri-devel, linux-kernel, Natalie Vock
To signal availability of the new AMDGPU_GEM_VA2 ioctl to userspace.
Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 188c8ec234c02..5b0c373102068 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -129,9 +129,10 @@
* - 3.64.0 - Userq IP support query
* - 3.65.0 - Add userq syncobj timeline signaling support
* - 3.66.0 - Allow jpeg type 4 packets and VCE SEI command
+ * - 3.67.0 - Add AMDGPU_GEM_VA2 ioctl
*/
#define KMS_DRIVER_MAJOR 3
-#define KMS_DRIVER_MINOR 66
+#define KMS_DRIVER_MINOR 67
#define KMS_DRIVER_PATCHLEVEL 0
/*
--
2.56.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v2 02/10] drm/amdgpu: Add and use vm_update_ctx struct
2026-11-17 12:50 ` [PATCH v2 02/10] drm/amdgpu: Add and use vm_update_ctx struct Natalie Vock
@ 2026-11-17 13:26 ` sashiko-bot
0 siblings, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 13:26 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 7 potential
issues to consider:
- [High] drm/amdgpu: fence and sync leaks in
amdgpu_gem_add_input_fence() error paths
- [High] drm/amdgpu: stale vm->delayed_freed entries overwrite new PTEs
in amdgpu_gem_va_update_vm()
- [Medium] drm/amdgpu: swallowed error and sync leak in
amdgpu_vm_handle_moved()
- [Medium] drm/amdgpu: update_ctx.sync leak on seq64_va error in
amdgpu_cs_vm_handling()
- [Medium] drm/amdgpu: input fences dropped in amdgpu_gem_va_ioctl()
without PT range update
- [Medium] drm/amdgpu: cross-BO fence accumulation in
amdgpu_vm_bo_update()
- [Low] drm/amdgpu: outdated kernel-doc comments for
amdgpu_vm_bo_update() and vm_update_ctx
--
commit 613a377be8b434a7253a6f919cefd9f59fe0c3de
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Add and use vm_update_ctx struct
This commit introduces struct amdgpu_vm_update_ctx to track freed mappings
and synchronization state across VM mapping and page table update
operations, and moves deferred unmaps onto vm->delayed_freed.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 71e2b83faa12b..d1f5ec9a3ef0f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -1126,6 +1126,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
> struct amdgpu_vm *vm = &fpriv->vm;
> struct amdgpu_bo_list_entry *e;
> struct amdgpu_bo_va *bo_va;
> + struct amdgpu_vm_update_ctx update_ctx;
> unsigned int i;
> int r;
>
> @@ -1147,34 +1148,36 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p)
> if (!amdgpu_vm_ready(vm))
> return -EINVAL;
>
> - r = amdgpu_vm_clear_freed(adev, vm, NULL);
> + r = amdgpu_vm_delayed_free(adev, vm, NULL);
> if (r)
> return r;
>
> - r = amdgpu_vm_bo_update(adev, fpriv->prt_va, false);
> + amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
> +
> + r = amdgpu_vm_bo_update(&update_ctx, fpriv->prt_va, false);
> if (r)
> - return r;
> + goto err_fini_ctx;
>
> r = amdgpu_sync_fence(&p->sync, fpriv->prt_va->last_pt_update,
> GFP_KERNEL);
> if (r)
> - return r;
> + goto err_fini_ctx;
>
> if (fpriv->csa_va) {
> bo_va = fpriv->csa_va;
> - r = amdgpu_vm_bo_update(adev, bo_va, false);
> + r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
> if (r)
> - return r;
> + goto err_fini_ctx;
>
> r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update,
> GFP_KERNEL);
> if (r)
> - return r;
> + goto err_fini_ctx;
> }
>
> if (fpriv->seq64_va) {
> bo_va = fpriv->seq64_va;
> - r = amdgpu_vm_bo_update(adev, bo_va, false);
> + r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
> if (r) {
> DRM_ERROR("failed to do vm_bo_update on userq sem\n");
> return r;
> }
>
> r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update,
> GFP_KERNEL);
> if (r)
> return r;
> }
[Severity: Medium]
Do the two error returns in the fpriv->seq64_va block leak
update_ctx.sync? Both paths return r directly instead of jumping to
err_fini_ctx to call amdgpu_vm_update_ctx_fini(&update_ctx), leaking any
amdgpu_sync_entry allocations and dma_fence references added during the
earlier csa_va or seq64_va updates.
[ ... ]
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index c29df93790c53..e4492c44ce1cb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -51,6 +51,7 @@
>
> static int
> amdgpu_gem_add_input_fence(struct drm_file *filp,
> + struct amdgpu_vm_update_ctx *update_ctx,
> uint64_t syncobj_handles_array,
> uint32_t num_syncobj_handles)
> {
> @@ -77,9 +78,10 @@ amdgpu_gem_add_input_fence(struct drm_file *filp,
> if (ret)
> goto free_memdup;
>
> - dma_fence_wait(fence, false);
> + ret = amdgpu_sync_fence(&update_ctx->sync, fence, GFP_KERNEL);
> + if (ret)
> + goto free_memdup;
>
> - /* TODO: optimize async handling */
> dma_fence_put(fence);
> }
[Severity: High]
Does jumping to free_memdup when amdgpu_sync_fence() fails leak the fence
reference acquired by drm_syncobj_find_fence()?
[ ... ]
> @@ -808,8 +811,7 @@ int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void *data,
> * otherwise stub fence in case of error.
> */
> static struct dma_fence *
> -amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
> - struct amdgpu_vm *vm,
> +amdgpu_gem_va_update_vm(struct amdgpu_vm_update_ctx *ctx,
> struct amdgpu_bo_va *bo_va,
> uint32_t operation)
> {
> @@ -817,10 +819,9 @@ amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
> int r = 0;
>
> /* If the VM is not ready return only a stub. */
> - if (!amdgpu_vm_ready(vm))
> + if (!amdgpu_vm_ready(ctx->vm))
> return dma_fence_get_stub();
>
> -
> /*
> * First clean up any freed mappings in the VM.
> *
> @@ -828,20 +829,20 @@ amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
> * schedules GPU work. If nothing needs clearing, @fence can remain as
> * the original vm->last_update.
> */
> - r = amdgpu_vm_clear_freed(adev, vm, &vm->last_update);
> + r = amdgpu_vm_clear_freed(ctx, &ctx->vm->last_update);
> if (r)
> goto error;
>
> /* For MAP/REPLACE we also need to update the BO mappings. */
> if (operation == AMDGPU_VA_OP_MAP ||
> operation == AMDGPU_VA_OP_REPLACE) {
> - r = amdgpu_vm_bo_update(adev, bo_va, false);
> + r = amdgpu_vm_bo_update(ctx, bo_va, false);
> if (r)
> goto error;
> }
[Severity: High]
Can stale entries on ctx->vm->delayed_freed overwrite newly mapped PTEs?
When an unmap is deferred (for example via AMDGPU_VM_DELAY_UPDATE),
amdgpu_vm_bo_unmap() removes the mapping from vm->va and
amdgpu_vm_update_ctx_fini() splices it onto vm->delayed_freed. If
userspace then maps a BO at the same VA without AMDGPU_VM_DELAY_UPDATE,
amdgpu_gem_va_update_vm() only clears ctx->freed from the current ioctl and
writes the new BO's valid PTEs while the old unmap remains on
vm->delayed_freed. A later amdgpu_vm_delayed_free() in
amdgpu_cs_vm_handling() or amdgpu_gem_object_close() will then zero those
active PTEs and cause GPU page faults.
In addition, amdgpu_userq_vm_validate_and_restore_queue() and KFD VM
updates never call amdgpu_vm_delayed_free(), leaving mappings on
vm->delayed_freed uncleared in those paths.
[ ... ]
> @@ -967,8 +969,10 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
> if (r)
> goto error_put_gobj;
>
> + amdgpu_vm_update_ctx_init(&update_ctx, adev, &fpriv->vm);
> +
> /* Add input syncobj fences (if any) for synchronization. */
> - r = amdgpu_gem_add_input_fence(filp,
> + r = amdgpu_gem_add_input_fence(filp, &update_ctx,
> args->input_fence_syncobj_handles,
> args->num_syncobj_handles);
> if (r)
> goto error_put_gobj;
[ ... ]
> @@ -1049,7 +1053,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
> (!adev->debug_vm || timeline_syncobj)) {
> struct dma_fence *fence;
>
> - fence = amdgpu_gem_va_update_vm(adev, &fpriv->vm, bo_va,
> + fence = amdgpu_gem_va_update_vm(&update_ctx, bo_va,
> args->operation);
> if (timeline_syncobj) {
> if (!args->vm_timeline_point) {
> @@ -1070,6 +1074,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
> }
>
> error:
> + amdgpu_vm_update_ctx_fini(&update_ctx);
> dma_fence_chain_free(timeline_chain);
> if (timeline_syncobj)
> drm_syncobj_put(timeline_syncobj);
> drm_exec_fini(&exec);
> error_put_gobj:
> drm_gem_object_put(gobj);
> return r;
> }
[Severity: High]
Does jumping to error_put_gobj when amdgpu_gem_add_input_fence() fails
leak update_ctx.sync? If amdgpu_gem_add_input_fence() adds a fence to
update_ctx.sync on the first handle and then fails on a later handle,
error_put_gobj skips amdgpu_vm_update_ctx_fini(&update_ctx) and leaks the
accumulated amdgpu_sync_entry allocations and dma_fence references.
[Severity: Medium]
Are input fences in update_ctx.sync silently dropped when
amdgpu_vm_update_range() does not run?
Previously, amdgpu_gem_add_input_fence() waited on each input fence with
dma_fence_wait(). Now the fences are queued in update_ctx.sync, which is
only consumed by amdgpu_vm_update_range(). When AMDGPU_VM_DELAY_UPDATE is
set, or when an operation such as AMDGPU_VA_OP_CLEAR on an unmapped range
does not emit a PT range update, amdgpu_vm_update_ctx_fini() frees
update_ctx.sync without waiting on the input fences or chaining them into
timeline_syncobj.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 94b1cfe05e3e0..90f27feb84257 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1319,37 +1319,31 @@ void amdgpu_vm_get_memory(struct amdgpu_vm *vm,
> * Returns:
> * 0 for success, -EINVAL for failure.
> */
> -int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va,
> - bool clear)
> +int amdgpu_vm_bo_update(struct amdgpu_vm_update_ctx *ctx,
> + struct amdgpu_bo_va *bo_va, bool clear)
> {
> struct amdgpu_bo *bo = bo_va->base.bo;
> - struct amdgpu_vm *vm = bo_va->base.vm;
> + struct amdgpu_vm *vm = ctx->vm;
> struct amdgpu_bo_va_mapping *mapping;
> struct dma_fence **last_update;
> dma_addr_t *pages_addr = NULL;
> struct ttm_resource *mem;
> - struct amdgpu_sync sync;
> bool flush_tlb = clear;
> uint64_t vram_base;
> uint64_t flags;
> bool uncached;
> int r;
>
> - amdgpu_sync_create(&sync);
> if (clear) {
> mem = NULL;
>
> - /* Implicitly sync to command submissions in the same VM before
> - * unmapping.
> - */
> - r = amdgpu_sync_resv(adev, &sync, vm->root.bo->tbo.base.resv,
> - AMDGPU_SYNC_EQ_OWNER, vm);
> + r = amdgpu_vm_update_ctx_ensure_unmap_synced(ctx);
> if (r)
> - goto error_free;
> + goto error;
> if (bo) {
> - r = amdgpu_sync_kfd(&sync, bo->tbo.base.resv);
> + r = amdgpu_sync_kfd(&ctx->sync, bo->tbo.base.resv);
> if (r)
> - goto error_free;
> + goto error;
> }
> } else if (!bo) {
> mem = NULL;
> @@ -1374,16 +1368,16 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va,
> pages_addr = bo->tbo.ttm->dma_address;
>
> /* Implicitly sync to moving fences before mapping anything */
> - r = amdgpu_sync_resv(adev, &sync, bo->tbo.base.resv,
> + r = amdgpu_sync_resv(ctx->adev, &ctx->sync, bo->tbo.base.resv,
> AMDGPU_SYNC_EXPLICIT, vm);
> if (r)
> - goto error_free;
> + goto error;
> }
[Severity: Medium]
Does sharing ctx->sync across multiple amdgpu_vm_bo_update() calls cause
later BO updates to wait on unrelated fences?
amdgpu_vm_bo_update() adds per-BO moving fences, VM-wide
AMDGPU_SYNC_EQ_OWNER fences, and KFD eviction fences to ctx->sync before
checking whether bo_va->invalids is empty, and never clears per-BO fences
between calls. When amdgpu_cs_vm_handling() or amdgpu_vm_handle_moved()
reuses one update_ctx across a loop of BOs, every subsequent PT update
waits on all earlier BOs' moving fences (even for BOs whose bo_va->invalids
was empty), and an earlier clear=true BO forces later clear=false updates
in amdgpu_vm_handle_moved() to wait on prior VM submissions and enable
signaling on KFD eviction fences.
[ ... ]
> @@ -1679,18 +1748,22 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
> struct amdgpu_vm *vm,
> struct ww_acquire_ctx *ticket)
> {
> + struct amdgpu_vm_update_ctx update_ctx;
> struct amdgpu_bo_va *bo_va, *tmp;
> struct dma_resv *resv;
> struct amdgpu_bo *bo;
> bool clear, unlock;
> int r;
>
> + amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
> +
> list_for_each_entry_safe(bo_va, tmp, &vm->always_valid.needs_update,
> base.vm_status) {
> /* Per VM BOs never need to bo cleared in the page tables */
> - r = amdgpu_vm_bo_update(adev, bo_va, false);
> + r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
> +
> if (r)
> - return r;
> + goto error;
> }
>
> spin_lock(&vm->individual_lock);
> @@ -1716,7 +1789,7 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
> unlock = false;
> }
>
> - r = amdgpu_vm_bo_update(adev, bo_va, clear);
> + r = amdgpu_vm_bo_update(&update_ctx, bo_va, clear);
>
> if (unlock)
> dma_resv_unlock(resv);
> if (r)
> return r;
>
> @@ -1736,6 +1809,9 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
> }
> spin_unlock(&vm->individual_lock);
>
> +error:
> + amdgpu_vm_update_ctx_fini(&update_ctx);
> +
> return 0;
> }
[Severity: Medium]
Does amdgpu_vm_handle_moved() swallow errors from the always_valid loop and
leak update_ctx.sync on errors in the individual loop?
When amdgpu_vm_bo_update() fails on an entry in
vm->always_valid.needs_update, jumping to error returns 0 instead of r, so
callers such as amdgpu_cs_vm_handling() proceed with stale page tables.
Conversely, when amdgpu_vm_bo_update() fails in the
vm->individual.needs_update loop, returning r directly skips
amdgpu_vm_update_ctx_fini(&update_ctx) and leaks update_ctx.sync.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> index f6285a9d9e989..8445599d9539a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> @@ -264,6 +264,39 @@ struct amdgpu_task_info {
> struct kref refcount;
> };
>
> +/**
> + * struct amdgpu_vm_update_ctx
> + *
> + * Encapsulates various state that is carried around during VM updates.
> + */
[Severity: Low]
This isn't a bug, but scripts/kernel-doc expects a short description on the
struct amdgpu_vm_update_ctx line, and the kernel-doc comments on
amdgpu_vm_bo_update() and amdgpu_gem_va_update_vm() still list @adev and
@vm instead of @ctx.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=2
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 03/10] drm/amdgpu: Make VM maps pending per update context
2026-11-17 12:50 ` [PATCH v2 03/10] drm/amdgpu: Make VM maps pending per update context Natalie Vock
@ 2026-11-17 13:43 ` sashiko-bot
0 siblings, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 13:43 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Medium] drm/amdgpu: 64-bit PFN truncation in
amdgpu_vm_update_ctx_add_new_mapping()
- [Low] drm/amdgpu: duplicate interval check in
amdgpu_vm_update_ctx_add_new_mapping()
- [Low] drm/amdgpu: fix pending_maps kernel-doc in struct
amdgpu_vm_update_ctx
--
commit 471887082bbdcd3f5636c1bb4feae875d96ebdb8
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Make VM maps pending per update context
This commit stages new VM mappings in struct amdgpu_vm_update_ctx and
inserts them into the VM bookkeeping structures when
amdgpu_vm_update_ctx_commit() is called.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 90f27feb84257..97cf04da6a025 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1589,6 +1589,73 @@ static void amdgpu_vm_prt_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)
[ ... ]
> +static int
> +amdgpu_vm_update_ctx_add_new_mapping(struct amdgpu_vm_update_ctx *ctx,
> + struct amdgpu_bo_va *bo_va,
> + struct amdgpu_bo_va_mapping *mapping)
> +{
> + struct interval_tree_node *pending;
> +
> + if (interval_tree_iter_first(&ctx->pending_maps, mapping->start,
> + mapping->last))
> + return -EINVAL;
> +
> + pending = kzalloc_obj(*pending);
> + if (!pending)
> + return -ENOMEM;
> +
> + if (interval_tree_iter_first(&ctx->pending_maps, mapping->start,
> + mapping->last))
> + return -EINVAL;
[Severity: Low]
Is this second interval_tree_iter_first() check redundant, and would it
leak pending if ever taken? ctx->pending_maps is not modified since the
identical check before kzalloc_obj(), and returning -EINVAL here skips
kfree(pending).
> +
> + pending->start = mapping->start;
> + pending->last = mapping->last;
> + interval_tree_insert(pending, &ctx->pending_maps);
[Severity: Medium]
Can using struct interval_tree_node here truncate 64-bit GPU page frame
numbers on 32-bit architectures?
mapping->start and mapping->last are uint64_t PFNs (which is why amdgpu_vm.c
defines amdgpu_vm_it with uint64_t via INTERVAL_TREE_DEFINE), whereas
struct interval_tree_node and interval_tree_iter_first() use unsigned long.
On 32-bit builds, GPU virtual addresses at or above 16 TiB (PFN >= 1ULL << 32)
have their upper 32 bits truncated when querying and inserting into
ctx->pending_maps, which can cause false -EINVAL conflicts, missed overlaps,
or inverted intervals where pending->start > pending->last.
While a mapping is pending on ctx->new prior to amdgpu_vm_bo_commit_insert(),
its embedded rb and uint64_t start/last/__subtree_last fields are unused, so
inserting mapping directly into ctx->pending_maps via the amdgpu_vm_it
helpers would also avoid allocating pending.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> index 8445599d9539a..d35494440cb5a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> @@ -279,6 +279,16 @@ struct amdgpu_vm_update_ctx {
> */
> struct amdgpu_vm *vm;
>
> + /**
> + * @mapped_intervals: Interval tree of VA ranges mapped by this update operation.
> + */
> + struct rb_root_cached pending_maps;
[Severity: Low]
The kernel-doc tag @mapped_intervals does not match the struct member name
pending_maps, which triggers a scripts/kernel-doc warning.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=3
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 04/10] drm/amdgpu: Make removed/cleared mappings pending per update context
2026-11-17 12:50 ` [PATCH v2 04/10] drm/amdgpu: Make removed/cleared mappings " Natalie Vock
@ 2026-11-17 14:05 ` sashiko-bot
0 siblings, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 14:05 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 6 potential
issues to consider:
- [High] drm/amdgpu: missing amdgpu_vm_update_ctx_commit() in
unmap_bo_from_gpuvm()
- [High] drm/amdgpu: unclamped cleared mapping unmaps surviving gaps in
clear_freed()
- [Medium] drm/amdgpu: uncleaned pending lists in
amdgpu_vm_update_ctx_fini() on error
- [Medium] drm/amdgpu: cur_end moves backward on nested ranges in
execute_replace()
- [Medium] drm/amdgpu: 32-bit PFN truncation in cleared_ranges
interval_tree_node
- [Low] drm/amdgpu: mismatched @replaced kernel-doc tag for cleared
list_head
--
commit 2c940ee3e7377ed3b2aa248bd851e7573259f51f
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Make removed/cleared mappings pending per update context
This commit defers VM mapping removals and range clears into lists and
interval trees on struct amdgpu_vm_update_ctx until the context is
committed, and allows explicit synchronization when unmapping always-valid
BOs or PRT mappings.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 97cf04da6a025..1e2d45979efdb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
[ ... ]
> @@ -1652,10 +1716,103 @@ amdgpu_vm_update_ctx_add_new_mapping(struct amdgpu_vm_update_ctx *ctx,
> interval_tree_insert(pending, &ctx->pending_maps);
>
> mapping->bo_va = bo_va;
> + WARN_ON(!list_empty(&mapping->pending_list));
> list_add(&mapping->pending_list, &ctx->new);
> return 0;
> }
>
> +static void
> +amdgpu_vm_update_ctx_add_replaced_mapping(struct amdgpu_vm_update_ctx *ctx,
> + struct amdgpu_bo_va_mapping *mapping)
> +{
> + if (list_empty(&mapping->pending_list))
> + list_add(&mapping->pending_list, &ctx->cleared);
> +}
> +
> +static void
> +amdgpu_vm_update_ctx_add_freed_mapping(struct amdgpu_vm_update_ctx *ctx,
> + struct amdgpu_bo_va_mapping *mapping)
> +{
> + WARN_ON(!list_empty(&mapping->pending_list));
> + list_add(&mapping->pending_list, &ctx->deleted);
> +}
> +
> +int amdgpu_vm_update_ctx_execute_replace(struct amdgpu_vm_update_ctx *ctx)
> +{
> + LIST_HEAD(new_maps);
> + struct interval_tree_node *map;
> + struct amdgpu_bo_va_mapping *cleared, *next, *gap;
> + uint64_t cur_end;
> + int r;
> +
> + list_for_each_entry_safe(cleared, next, &ctx->cleared, pending_list) {
> + cur_end = cleared->start;
> +
> + map = interval_tree_iter_first(&ctx->cleared_ranges, cleared->start, cleared->last);
> + while (map && map->start <= cleared->last) {
> + if (map->start <= cur_end) {
> + cur_end = map->last + 1;
> + map = interval_tree_iter_next(map, cleared->start, cleared->last);
> + continue;
> + }
[Severity: Medium]
Can cur_end move backward here when ctx->cleared_ranges contains nested or
overlapping intervals? interval_tree_iter_first() and
interval_tree_iter_next() walk intervals in ascending order of map->start,
so if an earlier interval extends past a later one (for example, [0x10,
0x20] followed by [0x12, 0x15]), the second iteration sees map->start <=
cur_end and rewinds cur_end from 0x21 back to 0x16, causing the already
cleared subrange [0x16, 0x20] to be treated as an uncleared gap and
re-mapped (or rejected as a duplicate in
amdgpu_vm_update_ctx_add_new_mapping()). Using max(cur_end, map->last + 1)
would prevent cur_end from decreasing.
> +
> + gap = kmalloc_obj(*gap);
> + if (!gap) {
> + r = -ENOMEM;
> + goto error;
> + }
> +
> + gap->bo_va = cleared->bo_va;
> + gap->start = cur_end;
> + gap->last = map->start - 1;
> + gap->offset = cleared->offset;
> + gap->offset += (gap->start - cleared->start) << AMDGPU_GPU_PAGE_SHIFT;
> + gap->flags = cleared->flags;
> + list_add(&gap->pending_list, &new_maps);
> +
> + cur_end = map->last + 1;
> + map = interval_tree_iter_next(map, cleared->start, cleared->last);
> + }
> +
> + if (cur_end <= cleared->last) {
> + gap = kmalloc_obj(*gap);
> + if (!gap) {
> + r = -ENOMEM;
> + goto error;
> + }
> +
> + gap->bo_va = cleared->bo_va;
> + gap->start = cur_end;
> + gap->last = cleared->last;
> + gap->offset = cleared->offset;
> + gap->offset += (gap->start - cleared->start) << AMDGPU_GPU_PAGE_SHIFT;
> + gap->flags = cleared->flags;
> + list_add(&gap->pending_list, &new_maps);
> + }
> +
> + list_del_init(&cleared->pending_list);
> + amdgpu_vm_update_ctx_add_freed_mapping(ctx, cleared);
> + }
[Severity: High]
Does queueing the unclamped cleared mapping onto ctx->deleted cause
amdgpu_vm_clear_freed() to wipe the hardware PTEs for the surviving gap
subranges too?
Previously, amdgpu_vm_bo_clear_mappings() clamped tmp->start and tmp->last
to [saddr, eaddr] before adding tmp to ctx->freed, so
amdgpu_vm_clear_freed() only cleared PTEs in the unmapped range. With this
change, cleared keeps its full [cleared->start, cleared->last] span when
amdgpu_vm_bo_commit_remove() moves it to ctx->freed, while the surviving
gap mappings are added to cleared->bo_va->invalids. In
amdgpu_gem_va_update_vm(), amdgpu_vm_clear_freed() then zeroes the hardware
PTEs across the entire original mapping, and the surviving gaps are never
re-populated in the page tables for AMDGPU_VA_OP_CLEAR (which never calls
amdgpu_vm_bo_update()) or for AMDGPU_VA_OP_REPLACE when the split mapping
belongs to another BO or fpriv->prt_va:
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c:amdgpu_gem_va_update_vm() {
...
r = amdgpu_vm_clear_freed(ctx, &ctx->vm->last_update);
if (r)
goto error;
/* For MAP/REPLACE we also need to update the BO mappings. */
if (operation == AMDGPU_VA_OP_MAP ||
operation == AMDGPU_VA_OP_REPLACE) {
r = amdgpu_vm_bo_update(ctx, bo_va, false);
if (r)
goto error;
}
...
}
Even when replacing a subrange with the same bo_va, amdgpu_vm_clear_freed()
clears the full original range and flushes the TLB before
amdgpu_vm_bo_update() restores the surviving subranges, which can fault
concurrent GPU work accessing the unmodified parts of the mapping.
[ ... ]
> @@ -1715,17 +1877,23 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
> */
> void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
> {
> - struct interval_tree_node *pending, *next_pending;
> + struct interval_tree_node *node, *next_node;
>
> list_splice(&ctx->freed, &ctx->vm->delayed_freed);
> amdgpu_sync_free(&ctx->sync);
>
> - rbtree_postorder_for_each_entry_safe(pending, next_pending, &ctx->pending_maps.rb_root, rb)
> - kfree(pending);
> + rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->pending_maps.rb_root, rb) {
> + kfree(node);
> + }
> + rbtree_postorder_for_each_entry_safe(node, next_node, &ctx->cleared_ranges.rb_root, rb) {
> + kfree(node);
> + }
>
> ctx->pending_maps = RB_ROOT_CACHED;
> + ctx->cleared_ranges = RB_ROOT_CACHED;
>
> WARN_ON(!list_empty(&ctx->new));
> + WARN_ON(!list_empty(&ctx->deleted));
> }
[Severity: Medium]
If an error occurs before amdgpu_vm_update_ctx_commit() is called—for
example, when amdgpu_vm_update_ctx_add_new_mapping() fails in
amdgpu_vm_bo_replace_map() after amdgpu_vm_bo_clear_mappings() has already
populated ctx->cleared, or when an allocation or mapping insertion fails
midway through amdgpu_vm_update_ctx_execute_replace()—does
amdgpu_vm_update_ctx_fini() leak ctx->new mappings and leave live mappings
linked into the stack-allocated ctx->deleted and ctx->cleared lists?
In this commit, jumping to error in amdgpu_gem_va_ioctl() hits the WARN_ON()
checks, leaks any struct amdgpu_bo_va_mapping objects already moved to
ctx->new, and leaves existing mappings with mapping->pending_list pointing
into the popped update_ctx stack frame. Note that commit "drm/amdgpu: Add
function to roll back a VM update ctx" later in the series adds
amdgpu_vm_update_ctx_rollback(), but that helper still only unlinks
ctx->new entries without calling kfree(mapping) and does not unlink entries
on ctx->cleared at all.
[ ... ]
> @@ -2178,24 +2345,7 @@ int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
> return -ENOENT;
> }
>
> - /* It's unlikely to happen that the mapping userq hasn't been idled
> - * during user requests GEM unmap IOCTL except for forcing the unmap
> - * from user space.
> - */
> - if (unlikely(bo_va->userq_va_mapped))
> - amdgpu_userq_gem_va_unmap_validate(ctx->adev, mapping);
> -
> - list_del(&mapping->list);
> - amdgpu_vm_it_remove(mapping, &vm->va);
> - mapping->bo_va = NULL;
> - trace_amdgpu_vm_bo_unmap(bo_va, mapping);
> -
> - if (valid)
> - list_add(&mapping->list, &ctx->freed);
> - else
> - amdgpu_vm_free_mapping(ctx->adev, vm, mapping,
> - bo_va->last_pt_update);
> -
> + amdgpu_vm_update_ctx_add_freed_mapping(ctx, mapping);
> return 0;
> }
[Severity: High]
Does unmap_bo_from_gpuvm() also need to call amdgpu_vm_update_ctx_commit()
after amdgpu_vm_bo_unmap()?
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c:unmap_bo_from_gpuvm() {
struct amdgpu_bo_va *bo_va = entry->bo_va;
struct amdgpu_vm *vm = peer_vm;
struct amdgpu_vm_update_ctx update_ctx;
amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
(void)amdgpu_vm_bo_unmap(&update_ctx, bo_va, entry->va);
/* VM entity stopped if process killed, don't clear freed pt bo */
if (!amdgpu_vm_ready(vm))
goto out;
(void)amdgpu_vm_clear_freed(&update_ctx, &bo_va->last_pt_update);
(void)amdgpu_sync_fence(sync, bo_va->last_pt_update, GFP_KERNEL);
out:
amdgpu_vm_update_ctx_fini(&update_ctx);
return 0;
}
Because amdgpu_vm_bo_unmap() now only queues the mapping onto
update_ctx.deleted until amdgpu_vm_update_ctx_commit() runs,
unmap_bo_from_gpuvm() passes an empty update_ctx.freed list to
amdgpu_vm_clear_freed() and never removes the mapping from bo_va or vm->va.
Every KFD unmap leaves the GPU page tables mapped, triggers
WARN_ON(!list_empty(&ctx->deleted)) in amdgpu_vm_update_ctx_fini(), and
leaves mapping->pending_list pointing at the stack-allocated update_ctx (or
rolls the unmap back once commit "drm/amdgpu: Add function to roll back a
VM update ctx" is applied).
>
> @@ -2214,8 +2364,8 @@ int amdgpu_vm_bo_unmap(struct amdgpu_vm_update_ctx *ctx,
> int amdgpu_vm_bo_clear_mappings(struct amdgpu_vm_update_ctx *ctx,
> uint64_t saddr, uint64_t size)
> {
> - struct amdgpu_bo_va_mapping *before, *after, *tmp, *next;
> - LIST_HEAD(removed);
> + struct interval_tree_node *cleared_range;
> + struct amdgpu_bo_va_mapping *tmp;
> uint64_t eaddr;
> int r;
>
> @@ -2223,96 +2373,23 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_vm_update_ctx *ctx,
> if (r)
> return r;
>
> - saddr /= AMDGPU_GPU_PAGE_SIZE;
> - eaddr = saddr + (size - 1) / AMDGPU_GPU_PAGE_SIZE;
> -
> - /* Allocate all the needed memory */
> - before = kzalloc_obj(*before);
> - if (!before)
> + cleared_range = kmalloc_obj(*cleared_range);
> + if (!cleared_range)
> return -ENOMEM;
> - INIT_LIST_HEAD(&before->list);
>
> - after = kzalloc_obj(*after);
> - if (!after) {
> - kfree(before);
> - return -ENOMEM;
> - }
> - INIT_LIST_HEAD(&after->list);
> + saddr /= AMDGPU_GPU_PAGE_SIZE;
> + eaddr = saddr + (size - 1) / AMDGPU_GPU_PAGE_SIZE;
>
> /* Now gather all removed mappings */
> tmp = amdgpu_vm_it_iter_first(&ctx->vm->va, saddr, eaddr);
> while (tmp) {
> - /* Remember mapping split at the start */
> - if (tmp->start < saddr) {
> - before->start = tmp->start;
> - before->last = saddr - 1;
> - before->offset = tmp->offset;
> - before->flags = tmp->flags;
> - before->bo_va = tmp->bo_va;
> - list_add(&before->list, &tmp->bo_va->invalids);
> - }
> -
> - /* Remember mapping split at the end */
> - if (tmp->last > eaddr) {
> - after->start = eaddr + 1;
> - after->last = tmp->last;
> - after->offset = tmp->offset;
> - after->offset += (after->start - tmp->start) << AMDGPU_GPU_PAGE_SHIFT;
> - after->flags = tmp->flags;
> - after->bo_va = tmp->bo_va;
> - list_add(&after->list, &tmp->bo_va->invalids);
> - }
> -
> - list_del(&tmp->list);
> - list_add(&tmp->list, &removed);
> -
> + amdgpu_vm_update_ctx_add_replaced_mapping(ctx, tmp);
> tmp = amdgpu_vm_it_iter_next(tmp, saddr, eaddr);
> }
>
> - /* And free them up */
> - list_for_each_entry_safe(tmp, next, &removed, list) {
> - amdgpu_vm_it_remove(tmp, &ctx->vm->va);
> - list_del(&tmp->list);
> -
> - if (tmp->start < saddr)
> - tmp->start = saddr;
> - if (tmp->last > eaddr)
> - tmp->last = eaddr;
> -
> - tmp->bo_va = NULL;
> - list_add(&tmp->list, &ctx->freed);
> - trace_amdgpu_vm_bo_unmap(NULL, tmp);
> - }
> -
> - /* Insert partial mapping before the range */
> - if (!list_empty(&before->list)) {
> - struct amdgpu_bo *bo = before->bo_va->base.bo;
> -
> - amdgpu_vm_it_insert(before, &ctx->vm->va);
> - if (before->flags & AMDGPU_VM_PAGE_PRT)
> - amdgpu_vm_prt_get(ctx->adev);
> -
> - if (amdgpu_vm_is_bo_always_valid(ctx->vm, bo) &&
> - !before->bo_va->base.moved)
> - amdgpu_vm_bo_needs_update(&before->bo_va->base);
> - } else {
> - kfree(before);
> - }
> -
> - /* Insert partial mapping after the range */
> - if (!list_empty(&after->list)) {
> - struct amdgpu_bo *bo = after->bo_va->base.bo;
> -
> - amdgpu_vm_it_insert(after, &ctx->vm->va);
> - if (after->flags & AMDGPU_VM_PAGE_PRT)
> - amdgpu_vm_prt_get(ctx->adev);
> -
> - if (amdgpu_vm_is_bo_always_valid(ctx->vm, bo) &&
> - !after->bo_va->base.moved)
> - amdgpu_vm_bo_needs_update(&after->bo_va->base);
> - } else {
> - kfree(after);
> - }
> + cleared_range->start = saddr;
> + cleared_range->last = eaddr;
> + interval_tree_insert(cleared_range, &ctx->cleared_ranges);
[Severity: Medium]
On 32-bit architectures, does storing 64-bit GPU page frame numbers into
struct interval_tree_node truncate saddr and eaddr?
struct interval_tree_node uses unsigned long for start and last, whereas GPU
PFNs are uint64_t (up to 36 bits on 48-bit GPU VA spaces, which is why
amdgpu_vm.c defines its own uint64_t amdgpu_vm_it interval tree). When a
mapping at or above 16 TiB (PFN >= 1ULL << 32) is cleared on a 32-bit build,
cleared_range->start and cleared_range->last are truncated to 32 bits; in
amdgpu_vm_update_ctx_execute_replace(), cur_end = map->last + 1 then drops
below the 64-bit cleared->start, causing (gap->start - cleared->start) <<
AMDGPU_GPU_PAGE_SHIFT to underflow and produce an out-of-bounds gap->offset.
[ ... ]
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> index d35494440cb5a..5657aa63e9491 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
> @@ -284,11 +284,27 @@ struct amdgpu_vm_update_ctx {
> */
> struct rb_root_cached pending_maps;
>
> + /**
> + * @cleared_ranges: Interval tree of VA ranges cleared by this update operation.
> + */
> + struct rb_root_cached cleared_ranges;
> +
> /**
> * @new: Pending new mappings not yet applied to bookkeeping structures.
> */
> struct list_head new;
>
> + /**
> + * @deleted: Pending freed mappings not yet applied to bookkeeping structures.
> + */
> + struct list_head deleted;
> +
> + /**
> + * @replaced: Pending mappings that have been cleared, potentially partially,
> + * and are not yet applied to bookkeeping structures.
> + */
> + struct list_head cleared;
[Severity: Low]
This kernel-doc comment uses @replaced instead of @cleared to document
struct list_head cleared, which triggers a kernel-doc warning on W=1 builds.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=4
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 05/10] drm/amdgpu: Track dirty bo_va structures in update ctx
2026-11-17 12:50 ` [PATCH v2 05/10] drm/amdgpu: Track dirty bo_va structures in update ctx Natalie Vock
@ 2026-11-17 14:28 ` sashiko-bot
0 siblings, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 14:28 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] drm/amdgpu: UAF on bo_va in amdgpu_vm_update_ctx_fini() after
amdgpu_vm_bo_del()
- [High] drm/amdgpu: skipped delete in
amdgpu_vm_update_ctx_add_freed_mapping()
- [Low] drm/amdgpu: update_link placed under wrong lock comment in
amdgpu_bo_va
--
commit 3f1753904c640653510160bf642992a7a55fbc3d
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Track dirty bo_va structures in update ctx
This commit adds an update_link list head to struct amdgpu_bo_va and tracks
modified bo_va structures in a dirty_bo_vas list on struct amdgpu_vm_update_ctx.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> index e0090dc10682f..068c57ed537fd 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> @@ -80,6 +80,8 @@ struct amdgpu_bo_va {
> /* protected by bo being reserved */
> unsigned ref_count;
>
> + struct list_head update_link;
> +
> /* all other members protected by the VM PD being reserved */
> struct dma_fence *last_pt_update;
>
[Severity: Low]
Should update_link be placed below the "all other members protected by the VM
PD being reserved" comment? It is modified by the update_ctx helpers while
only the VM PD reservation lock is held, including when the individual BO is
NULL (such as fpriv->prt_va) or not reserved (when clearing overlapping
mappings of other BOs in the VM).
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 1e2d45979efdb..330c288053352 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
[ ... ]
> @@ -1733,8 +1735,10 @@ static void
> amdgpu_vm_update_ctx_add_freed_mapping(struct amdgpu_vm_update_ctx *ctx,
> struct amdgpu_bo_va_mapping *mapping)
> {
> - WARN_ON(!list_empty(&mapping->pending_list));
> - list_add(&mapping->pending_list, &ctx->deleted);
> + if (list_empty(&mapping->bo_va->update_link))
> + list_add(&mapping->bo_va->update_link, &ctx->dirty_bo_vas);
> + if (list_empty(&mapping->pending_list))
> + list_add(&mapping->pending_list, &ctx->deleted);
> }
[Severity: High]
Can skipping list_add(&mapping->pending_list, &ctx->deleted) when
mapping->pending_list is non-empty leave freed mappings in vm->va?
Within one update_ctx, amdgpu_vm_bo_clear_mappings() and
amdgpu_vm_bo_replace_map() put overlapping mappings onto ctx->cleared via
mapping->pending_list; if amdgpu_vm_bo_unmap() is then called on that mapping
in the same context, amdgpu_vm_update_ctx_add_freed_mapping() leaves it on
ctx->cleared instead of moving it to ctx->deleted, so
amdgpu_vm_update_ctx_execute_replace() re-inserts any non-cleared remainder.
Furthermore, if an update_ctx is aborted before
amdgpu_vm_update_ctx_execute_replace() finishes processing ctx->cleared (for
example, if amdgpu_vm_bo_replace_map() fails in
amdgpu_vm_update_ctx_add_new_mapping()), neither amdgpu_vm_update_ctx_fini()
here nor amdgpu_vm_update_ctx_rollback() in commit "drm/amdgpu: Add function to
roll back a VM update ctx" unlinks entries from ctx->cleared. When the BO is
later removed in amdgpu_vm_bo_del(), amdgpu_vm_update_ctx_add_freed_mapping()
silently skips adding the mapping to update_ctx.deleted and kfree(bo_va) frees
bo_va while the mapping remains in vm->va with a dangling mapping->bo_va
pointer, causing a use-after-free on subsequent vm->va traversals.
> int amdgpu_vm_update_ctx_execute_replace(struct amdgpu_vm_update_ctx *ctx)
[ ... ]
> @@ -1878,10 +1883,16 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
> void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
> {
> struct interval_tree_node *node, *next_node;
> + struct amdgpu_bo_va *bo_va, *next_bo_va;
>
> list_splice(&ctx->freed, &ctx->vm->delayed_freed);
> amdgpu_sync_free(&ctx->sync);
>
> + list_for_each_entry_safe(bo_va, next_bo_va, &ctx->dirty_bo_vas,
> + update_link) {
> + list_del_init(&bo_va->update_link);
> + }
[Severity: High]
Does this cause a use-after-free on bo_va when a caller unmaps and deletes a
bo_va before finalizing update_ctx?
In amdgpu_unmap_static_csa(), amdgpu_vm_bo_unmap() adds bo_va->update_link to
update_ctx.dirty_bo_vas, and amdgpu_vm_bo_del() then frees bo_va before
amdgpu_vm_update_ctx_fini() runs:
drivers/gpu/drm/amd/amdgpu/amdgpu_csa.c:amdgpu_unmap_static_csa() {
...
r = amdgpu_vm_bo_unmap(&update_ctx, bo_va, csa_addr);
if (r) {
drm_err(adev_to_drm(adev),
"failed to do bo_unmap on static CSA, err=%d\n", r);
goto error_ctx;
}
amdgpu_vm_update_ctx_commit(&update_ctx);
amdgpu_vm_bo_del(adev, bo_va);
error_ctx:
amdgpu_vm_update_ctx_fini(&update_ctx);
...
}
Because amdgpu_vm_bo_del() calls kfree(bo_va) without unlinking
bo_va->update_link from the outer update_ctx.dirty_bo_vas,
amdgpu_vm_update_ctx_fini() executes list_del_init(&bo_va->update_link) on the
freed bo_va. Unlinking bo_va->update_link in amdgpu_vm_bo_del() before
kfree(bo_va) would avoid this.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=5
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 06/10] drm/amdgpu: Add function to roll back a VM update ctx
2026-11-17 12:50 ` [PATCH v2 06/10] drm/amdgpu: Add function to roll back a VM " Natalie Vock
@ 2026-11-17 14:40 ` sashiko-bot
0 siblings, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 14:40 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] drm/amdgpu: leak of ctx->new mappings in
amdgpu_vm_update_ctx_rollback()
- [High] drm/amdgpu: missing ctx->cleared unlink in
amdgpu_vm_update_ctx_rollback()
--
commit b40894c15667a95794d5db4dfea6f22963fb9ec5
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Add function to roll back a VM update ctx
This commit introduces amdgpu_vm_update_ctx_rollback() to discard pending
mapping changes in a VM update context and calls it from
amdgpu_vm_update_ctx_fini().
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 330c288053352..ac4a1ed9b1196 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1874,37 +1874,48 @@ void amdgpu_vm_update_ctx_init(struct amdgpu_vm_update_ctx *ctx,
> }
>
> /**
> - * amdgpu_vm_update_ctx_fini - finish a VM update operation
> - * @ctx: The context of the update
> + * amdgpu_vm_update_ctx_rollback - clear pending VM updates
> + * @ctx: The context to roll back
> *
> - * Adds any leftover mappings needing to be cleared to the VM's delayed free list,
> - * then frees up resources used by the context.
> + * Cleans up any pending mapping changes, but does not reset
> + * anything else (like attached syncobjs).
> */
> -void amdgpu_vm_update_ctx_fini(struct amdgpu_vm_update_ctx *ctx)
> +void amdgpu_vm_update_ctx_rollback(struct amdgpu_vm_update_ctx *ctx)
> {
> + struct amdgpu_bo_va_mapping *mapping, *next;
> struct interval_tree_node *node, *next_node;
> struct amdgpu_bo_va *bo_va, *next_bo_va;
>
> - list_splice(&ctx->freed, &ctx->vm->delayed_freed);
> - amdgpu_sync_free(&ctx->sync);
> + list_for_each_entry_safe(mapping, next, &ctx->new, pending_list)
> + list_del_init(&mapping->pending_list);
[Severity: High]
Does this leak each struct amdgpu_bo_va_mapping on ctx->new?
Unlike mappings on ctx->deleted, newly allocated mappings added by
amdgpu_vm_update_ctx_add_new_mapping() are not inserted into bo_va->invalids
or vm->va until amdgpu_vm_update_ctx_commit() runs. If a rollback or an
error before commit hits this loop, unlinking mapping->pending_list without
calling kfree(mapping) drops the only reference to those allocations.
> + list_for_each_entry_safe(mapping, next, &ctx->deleted, pending_list)
> + list_del_init(&mapping->pending_list);
[Severity: High]
Should amdgpu_vm_update_ctx_rollback() also unlink mappings on
ctx->cleared?
amdgpu_vm_bo_clear_mappings() links overlapping live mappings from vm->va
onto ctx->cleared via mapping->pending_list. If a replace or clear
operation rolls back or fails before amdgpu_vm_update_ctx_execute_replace()
drains ctx->cleared, those mappings stay linked to ctx->cleared while
ctx->cleared_ranges is reset. If the context is then destroyed,
list_empty(&mapping->pending_list) remains false on the live mappings in
vm->va, causing later unmaps and amdgpu_vm_bo_del() to skip removing them
from vm->va before freeing bo_va, which leaves dangling pointers and leads
to a use-after-free.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=6
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 08/10] drm/amdgpu: Only reject conflicting mappings if they are not cleared
2026-11-17 12:50 ` [PATCH v2 08/10] drm/amdgpu: Only reject conflicting mappings if they are not cleared Natalie Vock
@ 2026-11-17 15:06 ` sashiko-bot
0 siblings, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 15:06 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [Critical] drm/amdgpu: amdgpu_vm_update_ctx_rollback() fails to unlink
ctx->cleared
- [High] drm/amdgpu: missing interval tree iteration in
amdgpu_vm_bo_map()
- [High] drm/amdgpu: amdgpu_vm_bo_map() ignores conflicts on partially
cleared ranges
- [High] drm/amdgpu: amdgpu_vm_update_ctx_rollback() leaks ctx->new
mappings
- [High] drm/amdgpu: missing amdgpu_vm_update_ctx_commit() in
unmap_bo_from_gpuvm()
--
commit 4bdb539ca2c3dc59efa5a70eaf09b4e87c351cb1
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Only reject conflicting mappings if they are not cleared
This commit updates amdgpu_vm_bo_map() to only reject an overlapping
mapping in the VM interval tree when its pending_list is empty.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index ac4a1ed9b1196..491f4b50f7602 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -2241,7 +2241,7 @@ int amdgpu_vm_bo_map(struct amdgpu_vm_update_ctx *ctx,
> eaddr = saddr + (size - 1) / AMDGPU_GPU_PAGE_SIZE;
>
> tmp = amdgpu_vm_it_iter_first(&vm->va, saddr, eaddr);
> - if (tmp) {
> + if (tmp && list_empty(&tmp->pending_list)) {
> /* bo and tmp overlap, invalid addr */
> dev_err(ctx->adev->dev,
> "bo %p va 0x%010Lx-0x%010Lx conflict with "
[Severity: High]
Can amdgpu_vm_bo_map() miss a conflict when [saddr, eaddr] overlaps more
than one mapping in vm->va?
If the first node returned by amdgpu_vm_it_iter_first() has a non-empty
pending_list (because it was unmapped or cleared earlier in the same
update context) and a subsequent overlapping node in [saddr, eaddr] is
still active with an empty pending_list, amdgpu_vm_bo_map() never walks
the remaining nodes via amdgpu_vm_it_iter_next().
amdgpu_vm_update_ctx_commit() then inserts the new mapping into vm->va on
top of the active mapping.
Should this loop with amdgpu_vm_it_iter_next(tmp, saddr, eaddr) over all
overlapping nodes in [saddr, eaddr] and reject the map if any node has an
empty pending_list?
[Severity: High]
Does checking list_empty(&tmp->pending_list) also allow mapping over the
uncleared portion of a partially cleared mapping?
When amdgpu_vm_bo_clear_mappings() partially clears an existing mapping
tmp, it adds the entire unsplit mapping to ctx->cleared via
tmp->pending_list while recording only the cleared sub-range in
ctx->cleared_ranges:
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:amdgpu_vm_bo_clear_mappings() {
...
tmp = amdgpu_vm_it_iter_first(&ctx->vm->va, saddr, eaddr);
while (tmp) {
amdgpu_vm_update_ctx_add_replaced_mapping(ctx, tmp);
tmp = amdgpu_vm_it_iter_next(tmp, saddr, eaddr);
}
cleared_range->start = saddr;
cleared_range->last = eaddr;
interval_tree_insert(cleared_range, &ctx->cleared_ranges);
...
}
Because list_empty(&tmp->pending_list) is false across all of
[tmp->start, tmp->last], a subsequent amdgpu_vm_bo_map() overlapping the
still-active part of tmp is not rejected here. If a later clear in the
same update context covers that remaining range,
amdgpu_vm_update_ctx_execute_replace() produces no gap mapping to collide
with ctx->pending_maps, so the invalid map succeeds without ever reporting
a conflict.
[Severity: Critical]
What happens to tmp->pending_list when an update context is rolled back
after staging cleared mappings on ctx->cleared?
In commit "drm/amdgpu: Add function to roll back a VM update ctx",
amdgpu_vm_update_ctx_rollback() unlinks ctx->new and ctx->deleted, but
does not unlink ctx->cleared:
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:amdgpu_vm_update_ctx_rollback() {
...
list_for_each_entry_safe(mapping, next, &ctx->new, pending_list)
list_del_init(&mapping->pending_list);
list_for_each_entry_safe(mapping, next, &ctx->deleted, pending_list)
list_del_init(&mapping->pending_list);
...
}
If an operation rolls back after amdgpu_vm_bo_clear_mappings() or
amdgpu_vm_bo_replace_map() has placed a live mapping onto ctx->cleared,
that mapping stays in vm->va with mapping->pending_list pointing into the
expired stack-allocated update_ctx.cleared head. Because
list_empty(&mapping->pending_list) then remains false, this check in
amdgpu_vm_bo_map() treats the live mapping as cleared and allows
conflicting mappings over it. In addition,
amdgpu_vm_update_ctx_add_freed_mapping() skips any mapping whose
pending_list is not empty, so amdgpu_vm_bo_del() frees bo_va while leaving
its mapping in vm->va with a dangling bo_va pointer, leading to a
use-after-free on later VM interval tree walks.
[Severity: High]
Does rolling back a context after amdgpu_vm_bo_map() queues a new mapping
leak the mapping allocation?
amdgpu_vm_bo_map() allocates mapping with kmalloc_obj(*mapping) and
queues it on ctx->new via amdgpu_vm_update_ctx_add_new_mapping(), only
inserting it into vm->va and bo_va->invalids when
amdgpu_vm_update_ctx_commit() runs. When a later step fails (such as
amdgpu_vm_update_ctx_execute_replace() after mapping over a partially
cleared range) or hits lock contention, amdgpu_vm_update_ctx_rollback()
calls list_del_init(&mapping->pending_list) on each entry in ctx->new
without freeing mapping.
[Severity: High]
In commit "drm/amdgpu: Make removed/cleared mappings pending per update
context", does unmap_bo_from_gpuvm() also miss a call to
amdgpu_vm_update_ctx_commit() before clearing freed mappings?
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c:unmap_bo_from_gpuvm() {
...
amdgpu_vm_update_ctx_init(&update_ctx, adev, vm);
(void)amdgpu_vm_bo_unmap(&update_ctx, bo_va, entry->va);
/* VM entity stopped if process killed, don't clear freed pt bo */
if (!amdgpu_vm_ready(vm))
goto out;
(void)amdgpu_vm_clear_freed(&update_ctx, &bo_va->last_pt_update);
(void)amdgpu_sync_fence(sync, bo_va->last_pt_update, GFP_KERNEL);
out:
amdgpu_vm_update_ctx_fini(&update_ctx);
return 0;
}
Because amdgpu_vm_bo_unmap() only stages the mapping on
update_ctx.deleted until amdgpu_vm_update_ctx_commit() moves it to
update_ctx.freed and removes it from vm->va, amdgpu_vm_clear_freed() sees
an empty freed list and amdgpu_vm_update_ctx_fini() rolls the unmap back,
leaving the mapping active in vm->va.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=8
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl
2026-11-17 12:50 ` [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl Natalie Vock
@ 2026-11-17 15:34 ` sashiko-bot
2026-11-17 22:46 ` Dave Airlie
1 sibling, 0 replies; 19+ messages in thread
From: sashiko-bot @ 2026-11-17 15:34 UTC (permalink / raw)
To: Natalie Vock; +Cc: amd-gfx, dri-devel
Thank you for your contribution! Sashiko AI review found 15 potential
issues to consider:
- [High] drm/amdgpu: wrong argument to amdgpu_vm_bo_unmap() in
amdgpu_gem_va2_ioctl()
- [High] drm/amdgpu: syncobj_handles leak in
amdgpu_gem_prepare_timeline_nodes()
- [High] drm/amdgpu: uninitialized fence pointer in
amdgpu_gem_va2_ioctl() error path
- [High] drm/amdgpu: lost bo_va->last_pt_update and TLB fences in
amdgpu_gem_va2_ioctl()
- [High] drm/amdgpu: wrong struct in DRM_IOCTL_AMDGPU_GEM_VA2 macro
definition
- [High] drm/amdgpu: missing runtime PM acquisition in
amdgpu_gem_va2_ioctl()
- [High] drm/amdgpu: unlocked BO update on CLEAR/REPLACE in
amdgpu_gem_va2_ioctl()
- [High] drm/amdgpu: unlocked amdgpu_vm_bo_find() call in
amdgpu_gem_va2_ioctl()
- [High] drm/amdgpu: gobj ref leak on lock contention/error in
amdgpu_gem_va2_ioctl()
- [High] drm/amdgpu: memory leak and list corruption in
amdgpu_vm_update_ctx_rollback()
- [High] drm/amdgpu: fence ref leak when merging fences in
amdgpu_gem_va2_ioctl()
- [Medium] drm/amdgpu: delay never set to true in amdgpu_gem_va2_ioctl()
- [Medium] drm/amdgpu: wrong pointer in PTR_ERR() in
amdgpu_gem_add_input_fence()
- [Medium] drm/amdgpu: gobj ref leak on drm_exec_lock_obj() contention
or error
- [Medium] drm/amdgpu: in-place VA masking in amdgpu_gem_va2_ioctl()
breaks retry
--
commit dddcd1a96bab3eebca95b4fd6d51bf06141d06e1
Author: Natalie Vock <nat@pixelcluster.dev>
drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl
This commit adds the DRM_IOCTL_AMDGPU_GEM_VA2 ioctl to batch multiple VM
mapping operations in a single call with multiple input and output binary or
timeline syncobjs.
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index 7eed5c360be20..6d6a6017d4787 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -53,10 +53,13 @@ static int
> amdgpu_gem_add_input_fence(struct drm_file *filp,
> struct amdgpu_vm_update_ctx *update_ctx,
> uint64_t syncobj_handles_array,
> + uint64_t syncobj_points_array,
> uint32_t num_syncobj_handles)
> {
> struct dma_fence *fence;
> uint32_t *syncobj_handles;
> + uint64_t *syncobj_points;
> + uint64_t point;
> int ret, i;
>
> if (!num_syncobj_handles)
> @@ -67,6 +70,17 @@ amdgpu_gem_add_input_fence(struct drm_file *filp,
> if (IS_ERR(syncobj_handles))
> return PTR_ERR(syncobj_handles);
>
> + if (syncobj_points_array) {
> + syncobj_points = memdup_user(u64_to_user_ptr(syncobj_points_array),
> + size_mul(sizeof(uint64_t), num_syncobj_handles));
> + if (IS_ERR(syncobj_points)) {
> + kfree(syncobj_handles);
> + return PTR_ERR(syncobj_handles);
> + }
[Severity: Medium]
Should this return PTR_ERR(syncobj_points) instead of
PTR_ERR(syncobj_handles)? syncobj_handles is a valid kernel heap pointer
here, so returning PTR_ERR(syncobj_handles) leaks its lower 32 bits as the
ioctl return value (or returns 0 if 4 GiB-aligned).
[ ... ]
> +static int amdgpu_gem_prepare_timeline_nodes(
> + struct drm_file *filp, uint64_t syncobj_handles_array,
> + uint64_t syncobj_points_array, uint32_t num_syncobjs,
> + struct amdgpu_gem_timeline_out *out)
> +{
> + uint32_t *syncobj_handles;
> + uint32_t i;
> + int r;
> +
> + syncobj_handles = memdup_user(u64_to_user_ptr(syncobj_handles_array),
> + size_mul(sizeof(uint32_t), num_syncobjs));
> + if (IS_ERR(syncobj_handles))
> + return PTR_ERR(syncobj_handles);
[ ... ]
> + for (i = 0; i < num_syncobjs; i++) {
> + out->syncobjs[i] = drm_syncobj_find(filp, syncobj_handles[i]);
> + if (!out->syncobjs[i]) {
> + r = -ENOENT;
> + goto err_loop_rollback;
> + }
> +
> + if (!out->timeline_points[i])
> + continue;
> +
> + out->chains[i] = dma_fence_chain_alloc();
> + if (!out->chains[i]) {
> + r = -ENOMEM;
> + goto err_loop_rollback;
> + }
> +
> + continue;
> +err_loop_rollback:
> + for (;;) {
> + if (out->syncobjs[i])
> + drm_syncobj_put(out->syncobjs[i]);
> + if (out->chains[i])
> + dma_fence_chain_free(out->chains[i]);
> + if (i-- == 0)
> + goto err_free_sync;
> + }
> + }
> +
> + return 0;
[Severity: High]
Does the success path here leak syncobj_handles? kfree(syncobj_handles) is
only called on the error path at err_free_handles, so every call with
num_syncobjs > 0 leaks the memdup_user() buffer.
[ ... ]
> @@ -1100,6 +1218,232 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
> return r;
> }
>
> +int amdgpu_gem_va2_ioctl(struct drm_device *dev, void *data,
> + struct drm_file *filp)
> +{
> + struct drm_amdgpu_gem_va2 *args = data;
> + struct amdgpu_device *adev = drm_to_adev(dev);
> + struct amdgpu_fpriv *fpriv = filp->driver_priv;
> + struct drm_amdgpu_gem_va_mapping *mappings;
> + struct amdgpu_gem_timeline_out tl_out;
> + struct amdgpu_vm_update_ctx update_ctx;
> + struct drm_gem_object *gobj;
> + struct amdgpu_bo_va *bo_va;
> + struct drm_exec exec;
> + bool delay = false;
> + u32 i;
> + int r;
> +
> + mappings = vmemdup_array_user(u64_to_user_ptr(args->mappings),
> + args->num_mappings,
> + sizeof(struct drm_amdgpu_gem_va_mapping));
> + if (IS_ERR(mappings))
> + return PTR_ERR(mappings);
> +
> + r = amdgpu_gem_prepare_timeline_nodes(filp, args->syncobjs_out,
> + args->timeline_points_out,
> + args->num_syncobjs_out, &tl_out);
> + if (r)
> + goto out_free_mappings;
> +
> + drm_exec_init(&exec,
> + DRM_EXEC_INTERRUPTIBLE_WAIT | DRM_EXEC_IGNORE_DUPLICATES,
> + 0);
> + amdgpu_vm_update_ctx_init(&update_ctx, adev, &fpriv->vm);
[Severity: High]
Does amdgpu_gem_va2_ioctl() need to acquire a runtime PM reference via
PM_RUNTIME_ACQUIRE_IF_ENABLED_AUTOSUSPEND(), like amdgpu_gem_va_ioctl() does?
Without it, non-delayed VM updates can write to VRAM, flush HDP via MMIO, or
submit SDMA jobs while the GPU is runtime-suspended.
> + update_ctx.explicit_sync_unmap = true;
> +
> + r = amdgpu_gem_add_input_fence(filp, &update_ctx, args->syncobjs_in,
> + args->timeline_points_in,
> + args->num_syncobjs_in);
> + if (r)
> + goto out;
> +
> + drm_exec_until_all_locked(&exec)
> + {
> + r = amdgpu_vm_lock_pd(&fpriv->vm, &exec, 2);
> + drm_exec_retry_on_contention(&exec);
> + if (unlikely(r))
> + goto out;
> +
> + for (i = 0; i < args->num_mappings; ++i) {
> + r = amdgpu_gem_va_validate_args(adev, mappings[i].va,
> + mappings[i].map_size,
> + mappings[i].flags);
> + if (r)
> + goto out;
> +
> + mappings[i].va &= AMDGPU_GMC_HOLE_MASK;
[Severity: Medium]
Can masking mappings[i].va in place break retries on lock contention? For
an address in the upper canonical range (>= AMDGPU_GMC_HOLE_END), masking it
with AMDGPU_GMC_HOLE_MASK places it inside [AMDGPU_GMC_HOLE_START,
AMDGPU_GMC_HOLE_END), so if drm_exec_until_all_locked() retries the loop,
amdgpu_gem_va_validate_args() will reject mappings[i].va with -EINVAL.
> +
> + if (mappings[i].flags & AMDGPU_VM_DELAY_UPDATE) {
> + /* Can't delay if we're supposed to signal fences */
> + if (args->num_syncobjs_out) {
> + r = -EINVAL;
> + goto out;
> + }
> + /* If one mapping is delayed, all mappings must be delayed */
> + if (i && !delay) {
> + r = -EINVAL;
> + goto out;
> + }
> + } else if (delay) {
> + /* If one mapping is not delayed, no mappings may be delayed */
> + r = -EINVAL;
> + goto out;
> + }
[Severity: Medium]
Is delay ever set to true when AMDGPU_VM_DELAY_UPDATE is set? Because delay
stays false, any batch with two or more delayed mappings fails with -EINVAL
at if (i && !delay), a batch where only mappings[0] is delayed is not
rejected, and page table updates are still executed immediately.
> +
> + if ((mappings[i].operation != AMDGPU_VA_OP_CLEAR) &&
> + !(mappings[i].flags & AMDGPU_VM_PAGE_PRT)) {
> + gobj = drm_gem_object_lookup(
> + filp, mappings[i].handle);
> + if (gobj == NULL) {
> + r = -ENOENT;
> + goto out;
> + }
> + bo_va = amdgpu_vm_bo_find(
> + &fpriv->vm, gem_to_amdgpu_bo(gobj));
> + if (!bo_va) {
> + drm_gem_object_put(gobj);
> + r = -ENOENT;
> + goto out;
> + }
[Severity: High]
Is it safe to call amdgpu_vm_bo_find() before locking gobj with
drm_exec_lock_obj()? amdgpu_vm_bo_find() walks bo->vm_bo, which is
protected by bo->tbo.base.resv; on a shared BO, another DRM file can
concurrently modify bo->vm_bo and free its amdgpu_bo_va in
amdgpu_vm_bo_del().
> + } else {
> + gobj = NULL;
> + bo_va = mappings[i].operation !=
> + AMDGPU_VA_OP_CLEAR ?
> + fpriv->prt_va :
> + NULL;
> + }
> +
> + if (gobj) {
> + r = drm_exec_lock_obj(&exec, gobj);
> + if (drm_exec_is_contended(&exec))
> + amdgpu_vm_update_ctx_rollback(
> + &update_ctx);
[Severity: High]
When amdgpu_vm_update_ctx_rollback() runs here on lock contention (or via
amdgpu_vm_update_ctx_fini() on error), does it leak ctx->new mappings and
corrupt ctx->cleared?
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c:amdgpu_vm_update_ctx_rollback() {
...
list_for_each_entry_safe(mapping, next, &ctx->new, pending_list)
list_del_init(&mapping->pending_list);
list_for_each_entry_safe(mapping, next, &ctx->deleted, pending_list)
list_del_init(&mapping->pending_list);
...
}
Entries on ctx->new were allocated by amdgpu_vm_bo_map() or
amdgpu_vm_bo_replace_map() and are unlinked without kfree(mapping).
In addition, ctx->cleared is never unlinked at all, leaving live mappings in
fpriv->vm.va with mapping->pending_list pointing into update_ctx.cleared on
the stack.
> + drm_exec_retry_on_contention(&exec);
[Severity: Medium]
Does drm_exec_retry_on_contention(&exec) leak the gobj reference from
drm_gem_object_lookup()? drm_exec_lock_obj() takes its own reference for
exec->contended on -EDEADLK, and jumping back to the start of the loop
skips drm_gem_object_put(gobj) below.
> + if (unlikely(r))
> + goto out;
> + }
[Severity: High]
Does jumping to out when drm_exec_lock_obj() returns a non-contention error
(such as -EINTR or -ENOMEM) also leak the gobj lookup reference?
drm_exec_lock_obj() does not record gobj in exec->objects on failure, and
goto out bypasses drm_gem_object_put(gobj).
> +
> + switch (mappings[i].operation) {
> + case AMDGPU_VA_OP_MAP:
> + r = amdgpu_vm_bo_map(&update_ctx, bo_va,
> + mappings[i].va,
> + mappings[i].offset_in_bo,
> + mappings[i].map_size,
> + mappings[i].flags);
> + break;
> + case AMDGPU_VA_OP_UNMAP:
> + r = amdgpu_vm_bo_unmap(&update_ctx, bo_va,
> + mappings[i].flags);
> + break;
[Severity: High]
Should amdgpu_vm_bo_unmap() be passed mappings[i].va instead of
mappings[i].flags? Passing flags as the virtual address makes
AMDGPU_VA_OP_UNMAP look up page 0 and fail with -ENOENT.
[ ... ]
> + if (!delay && (!adev->debug_vm || args->num_syncobjs_out)) {
> + struct dma_fence *fence, *merged;
> +
> + if (!amdgpu_vm_ready(&fpriv->vm)) {
> + fence = dma_fence_get_stub();
> + goto insert_fence;
> + }
> +
> + r = amdgpu_vm_clear_freed(&update_ctx, &fpriv->vm.last_update);
> + if (r) {
> + drm_err(adev_to_drm(adev),
> + "Could not clear freed PT entries: %d\n", r);
> + goto insert_fence;
> + }
[Severity: High]
If amdgpu_vm_clear_freed() fails, does goto insert_fence use an
uninitialized fence pointer? fence is only initialized afterwards, so
insert_fence will pass stack garbage to drm_syncobj_replace_fence(),
drm_syncobj_add_point(), and dma_fence_put().
> +
> + fence = dma_fence_get(fpriv->vm.last_update);
> +
> + list_for_each_entry(bo_va, &update_ctx.dirty_bo_vas,
> + update_link) {
> + r = amdgpu_vm_bo_update(&update_ctx, bo_va, false);
> + if (r) {
> + drm_err(adev_to_drm(adev),
> + "Could not update BO PT entries: %d\n", r);
> + goto insert_fence;
> + }
> + }
[Severity: High]
Can this loop call amdgpu_vm_bo_update() on BOs whose dma_resv is not
locked?
AMDGPU_VA_OP_CLEAR locks no BO, and AMDGPU_VA_OP_REPLACE only locks the new
BO from mappings[i].handle. However, amdgpu_vm_update_ctx_execute_replace()
adds the bo_va of every existing mapping that was cleared or split to
update_ctx.dirty_bo_vas, so amdgpu_vm_bo_update() then accesses
bo->tbo.resource and calls amdgpu_sync_resv() on those affected BOs without
holding their reservation locks.
> +
> + list_for_each_entry(bo_va, &update_ctx.dirty_bo_vas,
> + update_link) {
> + if (!amdgpu_vm_is_bo_always_valid(&fpriv->vm,
> + bo_va->base.bo)) {
> + dma_fence_put(bo_va->last_pt_update);
> + bo_va->last_pt_update = dma_fence_get(fence);
> + }
> + }
[Severity: High]
Does this discard the page table update fences just generated by
amdgpu_vm_bo_update()?
For non-always-valid BOs (and PRT mappings where bo is NULL),
amdgpu_vm_bo_update() stores the new PT update fence in
bo_va->last_pt_update rather than fpriv->vm.last_update. Overwriting
bo_va->last_pt_update with the pre-update fence without merging
bo_va->last_pt_update (or fpriv->vm.last_tlb_inv) into fence drops those PT
update and TLB invalidation fences from both bo_va->last_pt_update and the
signaled output syncobjs.
> +
> + /* Always update PDEs after we touched the mappings. */
> + r = amdgpu_vm_update_pdes(adev, &fpriv->vm, false);
> + if (r)
> + goto insert_fence;
> +
> + merged = dma_fence_unwrap_merge(fence, fpriv->vm.last_update);
> + if (!merged) {
> + /* As fallback in OOM situations */
> + dma_fence_wait(fpriv->vm.last_update, false);
> + dma_fence_wait(fence, false);
> + merged = dma_fence_get_stub();
> + }
> + fence = merged;
[Severity: High]
Does assigning fence = merged without calling dma_fence_put(fence) leak the
fence reference acquired before the amdgpu_vm_bo_update() loop?
> diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
> index 29009f6aabde9..c9e52bfa7d028 100644
> --- a/include/uapi/drm/amdgpu_drm.h
> +++ b/include/uapi/drm/amdgpu_drm.h
> @@ -83,6 +84,7 @@ extern "C" {
> #define DRM_IOCTL_AMDGPU_GEM_LIST_HANDLES DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_LIST_HANDLES, struct drm_amdgpu_gem_list_handles)
> #define DRM_IOCTL_AMDGPU_PROC_OPTIONS DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_PROC_OPTIONS, struct drm_amdgpu_proc_options)
> #define DRM_IOCTL_AMDGPU_UALINK_HANDLE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_UALINK_HANDLE, union drm_amdgpu_ualink_handle)
> +#define DRM_IOCTL_AMDGPU_GEM_VA2 DRM_IOW(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_VA2, struct drm_amdgpu_gem_va)
[Severity: High]
Should DRM_IOCTL_AMDGPU_GEM_VA2 use struct drm_amdgpu_gem_va2 instead of
struct drm_amdgpu_gem_va? struct drm_amdgpu_gem_va is 64 bytes whereas
struct drm_amdgpu_gem_va2 is 56 bytes, so drm_ioctl() will copy 8 bytes past
the end of the user struct.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261117-prt-explicit-sync-v2-0-bb3851cb6a0c@pixelcluster.dev?part=9
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl
2026-11-17 12:50 ` [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl Natalie Vock
2026-11-17 15:34 ` sashiko-bot
@ 2026-11-17 22:46 ` Dave Airlie
1 sibling, 0 replies; 19+ messages in thread
From: Dave Airlie @ 2026-11-17 22:46 UTC (permalink / raw)
To: Natalie Vock
Cc: Felix Kuehling, Alex Deucher, Christian König, Simona Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Timur Kristóf, amd-gfx, dri-devel, linux-kernel
On Tue, 17 Nov 2026 at 22:50, Natalie Vock <nat@pixelcluster.dev> wrote:
>
> This new ioctl is a modernized version of AMDGPU_GEM_VA, with the
> main changes being support for many operations batched in one ioctl,
> and a better sync model that allows passing multiple syncobjs for
> both wait and signal. Timeline syncobj are now properly supported in
> both wait and signal cases.
>
> One breaking change compared to AMDGPU_GEM_VA is that
> synchronization is now explicit (where possible), meaning that
> previous submissions are not necessarily waited on.
>
> Using this ioctl allows applications that hammer lots of sparse
> operations, particularly some games running under Proton, to see
> significant reductions in stuttering.
>
> Signed-off-by: Natalie Vock <nat@pixelcluster.dev>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 3 +-
> drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 348 +++++++++++++++++++++++++++++++-
> drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h | 2 +
> include/uapi/drm/amdgpu_drm.h | 46 +++++
> 4 files changed, 396 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index 72b3710462824..188c8ec234c02 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -3124,7 +3124,8 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = {
> DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_WAIT, amdgpu_userq_wait_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_LIST_HANDLES, amdgpu_gem_list_handles_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> DRM_IOCTL_DEF_DRV(AMDGPU_PROC_OPTIONS, amdgpu_proc_options_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> - DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW)
> + DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> + DRM_IOCTL_DEF_DRV(AMDGPU_GEM_VA2, amdgpu_gem_va2_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> };
Can we just call it VM_BIND for consistency with xe and nouveau?
Ideally of course we'd use gpuvm for amd, but even without that it
would be good to at least have somewhat consistent naming just so
grepping find things.
Dave.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-11-17 22:47 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-17 12:50 [PATCH v2 00/10] drm/amdgpu: Rework VM update ops for batching and explicit sync Natalie Vock
2026-11-17 12:50 ` [PATCH v2 01/10] drm/amdgpu: Remove unused parameter from amdgpu_vm_pte_fragment Natalie Vock
2026-11-17 12:50 ` [PATCH v2 02/10] drm/amdgpu: Add and use vm_update_ctx struct Natalie Vock
2026-11-17 13:26 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 03/10] drm/amdgpu: Make VM maps pending per update context Natalie Vock
2026-11-17 13:43 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 04/10] drm/amdgpu: Make removed/cleared mappings " Natalie Vock
2026-11-17 14:05 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 05/10] drm/amdgpu: Track dirty bo_va structures in update ctx Natalie Vock
2026-11-17 14:28 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 06/10] drm/amdgpu: Add function to roll back a VM " Natalie Vock
2026-11-17 14:40 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 07/10] drm/amdgpu: Move some VA op validation to a helper Natalie Vock
2026-11-17 12:50 ` [PATCH v2 08/10] drm/amdgpu: Only reject conflicting mappings if they are not cleared Natalie Vock
2026-11-17 15:06 ` sashiko-bot
2026-11-17 12:50 ` [PATCH v2 09/10] drm/amdgpu: Add new AMDGPU_GEM_VA2 ioctl Natalie Vock
2026-11-17 15:34 ` sashiko-bot
2026-11-17 22:46 ` Dave Airlie
2026-11-17 12:50 ` [PATCH v2 10/10] drm/amdgpu: Bump minor version Natalie Vock
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox