* [RFC] drm/amdgpu: handling PTE backend update errors
@ 2026-11-10 10:29 Mu, Zihao
2026-11-10 12:26 ` Christian König
0 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-10 10:29 UTC (permalink / raw)
To: amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Koenig, Christian,
Thadeu Lima de Souza Cascardo
From: zihaomu <zmu@amd.com>
Hello Alex, Christian and Cascardo,
I would like guidance on propagating PTE backend update errors and how
that should interact with the no-fail unmap contract.
At drm-amdgpu-next commit 5349075eac14, the void helper
amdgpu_vm_pte_update_flags() calls update_funcs->update() but drops its
int return value. The CPU backend can report an interrupted fence wait;
the SDMA backend can report a dependency or replacement-job allocation
failure. The PTE walker can then advance past unwritten entries, invoke
the backend again, and return success for an incomplete mapping.
The illustrative diff below returns that error and stops the walker.
It is not a complete fix for merging: propagating failures also needs
correct finalization of partially written PTEs, pending SDMA work, TLB
handling and the stack-local tlb_flush_waitlist.
The local RFC has two cleanup prerequisites adapted from Cascardo's
September series [1]: handle missing/empty SDMA jobs, and finish partial
range updates while preserving the original error. The ignored backend
return was also raised by Sashiko in that discussion [2]. I am not
claiming the earlier cleanup work or the observation as new.
I have excluded the earlier mapping-retention proposal, which Christian
explicitly NAKed [3]. amdgpu_vm_clear_freed() remains unchanged, and a
KFD caller still discards its result. The remaining contract questions
are therefore important before proposing this for merging:
1. Should errors during clear/unmap be prevented or completed inside
the backend before the PTE walker can propagate failures?
2. Is there a current map/unmap split or successor series that this
work should be based on?
On the native base above, isolated models using extracted functions
passed 5 PTE, 17 range/finalization and 2 SDMA ownership scenarios,
including baseline failure and deliberately broken controls, with no
ASan/UBSan reports. GCC 13.3.0 compiled amdgpu_vm.o, amdgpu_vm_pt.o and
amdgpu_vm_sdma.o with CONFIG_DRM_AMDGPU=m. These are modeled errors and
object builds, not live kernel fault injection or hardware validation.
There is no claim that this resolves the original GPU queue stalls.
For background, the earlier vendor-tree PR is [4]. Its four commits
are not the proposed native submission, and the independent CPU fence
scan optimization is outside this discussion.
[1] https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg635900.html
[2] https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg635904.html
[3] https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg638593.html
[4] https://github.com/ROCm/amdgpu/pull/243
Signed-off-by: zihaomu <zmu@amd.com>
---
Illustrative error-propagation diff only; cleanup prerequisites and
agreement on the unmap failure contract are still required.
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index c73eb416c0e2..d9cfced93081 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -701,12 +701,12 @@ static void amdgpu_vm_pte_update_noretry_flags(struct amdgpu_device *adev,
*
* Make sure to set the right flags for the PTEs at the desired level.
*/
-static void amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
- struct amdgpu_bo_vm *pt,
- unsigned int level,
- uint64_t pe, uint64_t addr,
- unsigned int count, uint32_t incr,
- uint64_t flags)
+static int amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
+ struct amdgpu_bo_vm *pt,
+ unsigned int level,
+ u64 pe, u64 addr,
+ unsigned int count, u32 incr,
+ u64 flags)
{
struct amdgpu_device *adev = params->adev;
@@ -735,8 +735,8 @@ static void amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
if (level == AMDGPU_VM_PTB)
amdgpu_vm_pte_update_noretry_flags(adev, &flags);
- params->vm->update_funcs->update(params, pt, pe, addr, count, incr,
- flags);
+ return params->vm->update_funcs->update(params, pt, pe, addr, count, incr,
+ flags);
}
/**
@@ -812,7 +812,7 @@ static void amdgpu_vm_pte_fragment(struct amdgpu_vm_update_params *params,
* Update the page tables in the range @start - @end.
*
* Returns:
- * 0 for success, -EINVAL for failure.
+ * 0 for success, negative errno for failure.
*/
int amdgpu_vm_ptes_update(struct amdgpu_vm_update_params *params,
uint64_t start, uint64_t end,
@@ -921,9 +921,11 @@ int amdgpu_vm_ptes_update(struct amdgpu_vm_update_params *params,
trace_amdgpu_vm_update_ptes(params, frag_start, upd_end,
min(nptes, 32u), dst, incr,
upd_flags);
- amdgpu_vm_pte_update_flags(params, to_amdgpu_bo_vm(pt),
- cursor.level, pe_start, dst,
- nptes, incr, upd_flags);
+ r = amdgpu_vm_pte_update_flags(params, to_amdgpu_bo_vm(pt),
+ cursor.level, pe_start, dst,
+ nptes, incr, upd_flags);
+ if (r)
+ return r;
pe_start += nptes * 8;
dst += nptes * incr;
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC] drm/amdgpu: handling PTE backend update errors
2026-11-10 10:29 [RFC] drm/amdgpu: handling PTE backend update errors Mu, Zihao
@ 2026-11-10 12:26 ` Christian König
2026-11-11 5:39 ` [RFC PATCH] drm/amdgpu: make CPU page-table update waits uninterruptible Mu, Zihao
0 siblings, 1 reply; 25+ messages in thread
From: Christian König @ 2026-11-10 12:26 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/10/26 11:29, Mu, Zihao wrote:
> From: zihaomu <zmu@amd.com>
>
> Hello Alex, Christian and Cascardo,
>
> I would like guidance on propagating PTE backend update errors and how
> that should interact with the no-fail unmap contract.
>
> At drm-amdgpu-next commit 5349075eac14, the void helper
> amdgpu_vm_pte_update_flags() calls update_funcs->update() but drops its
> int return value. The CPU backend can report an interrupted fence wait;
> the SDMA backend can report a dependency or replacement-job allocation
> failure. The PTE walker can then advance past unwritten entries, invoke
> the backend again, and return success for an incomplete mapping.
Yeah we need to remove those error codes. PTE updates generally can't fail.
Regards,
Christian.
>
> The illustrative diff below returns that error and stops the walker.
> It is not a complete fix for merging: propagating failures also needs
> correct finalization of partially written PTEs, pending SDMA work, TLB
> handling and the stack-local tlb_flush_waitlist.
>
> The local RFC has two cleanup prerequisites adapted from Cascardo's
> September series [1]: handle missing/empty SDMA jobs, and finish partial
> range updates while preserving the original error. The ignored backend
> return was also raised by Sashiko in that discussion [2]. I am not
> claiming the earlier cleanup work or the observation as new.
>
> I have excluded the earlier mapping-retention proposal, which Christian
> explicitly NAKed [3]. amdgpu_vm_clear_freed() remains unchanged, and a
> KFD caller still discards its result. The remaining contract questions
> are therefore important before proposing this for merging:
>
> 1. Should errors during clear/unmap be prevented or completed inside
> the backend before the PTE walker can propagate failures?
> 2. Is there a current map/unmap split or successor series that this
> work should be based on?
>
> On the native base above, isolated models using extracted functions
> passed 5 PTE, 17 range/finalization and 2 SDMA ownership scenarios,
> including baseline failure and deliberately broken controls, with no
> ASan/UBSan reports. GCC 13.3.0 compiled amdgpu_vm.o, amdgpu_vm_pt.o and
> amdgpu_vm_sdma.o with CONFIG_DRM_AMDGPU=m. These are modeled errors and
> object builds, not live kernel fault injection or hardware validation.
> There is no claim that this resolves the original GPU queue stalls.
>
> For background, the earlier vendor-tree PR is [4]. Its four commits
> are not the proposed native submission, and the independent CPU fence
> scan optimization is outside this discussion.
>
> [1] https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg635900.html
> [2] https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg635904.html
> [3] https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg638593.html
> [4] https://github.com/ROCm/amdgpu/pull/243
>
> Signed-off-by: zihaomu <zmu@amd.com>
> ---
> Illustrative error-propagation diff only; cleanup prerequisites and
> agreement on the unmap failure contract are still required.
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> index c73eb416c0e2..d9cfced93081 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> @@ -701,12 +701,12 @@ static void amdgpu_vm_pte_update_noretry_flags(struct amdgpu_device *adev,
> *
> * Make sure to set the right flags for the PTEs at the desired level.
> */
> -static void amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
> - struct amdgpu_bo_vm *pt,
> - unsigned int level,
> - uint64_t pe, uint64_t addr,
> - unsigned int count, uint32_t incr,
> - uint64_t flags)
> +static int amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
> + struct amdgpu_bo_vm *pt,
> + unsigned int level,
> + u64 pe, u64 addr,
> + unsigned int count, u32 incr,
> + u64 flags)
> {
> struct amdgpu_device *adev = params->adev;
>
> @@ -735,8 +735,8 @@ static void amdgpu_vm_pte_update_flags(struct amdgpu_vm_update_params *params,
> if (level == AMDGPU_VM_PTB)
> amdgpu_vm_pte_update_noretry_flags(adev, &flags);
>
> - params->vm->update_funcs->update(params, pt, pe, addr, count, incr,
> - flags);
> + return params->vm->update_funcs->update(params, pt, pe, addr, count, incr,
> + flags);
> }
>
> /**
> @@ -812,7 +812,7 @@ static void amdgpu_vm_pte_fragment(struct amdgpu_vm_update_params *params,
> * Update the page tables in the range @start - @end.
> *
> * Returns:
> - * 0 for success, -EINVAL for failure.
> + * 0 for success, negative errno for failure.
> */
> int amdgpu_vm_ptes_update(struct amdgpu_vm_update_params *params,
> uint64_t start, uint64_t end,
> @@ -921,9 +921,11 @@ int amdgpu_vm_ptes_update(struct amdgpu_vm_update_params *params,
> trace_amdgpu_vm_update_ptes(params, frag_start, upd_end,
> min(nptes, 32u), dst, incr,
> upd_flags);
> - amdgpu_vm_pte_update_flags(params, to_amdgpu_bo_vm(pt),
> - cursor.level, pe_start, dst,
> - nptes, incr, upd_flags);
> + r = amdgpu_vm_pte_update_flags(params, to_amdgpu_bo_vm(pt),
> + cursor.level, pe_start, dst,
> + nptes, incr, upd_flags);
> + if (r)
> + return r;
>
> pe_start += nptes * 8;
> dst += nptes * incr;
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH] drm/amdgpu: make CPU page-table update waits uninterruptible
2026-11-10 12:26 ` Christian König
@ 2026-11-11 5:39 ` Mu, Zihao
2026-11-11 9:06 ` Christian König
0 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 5:39 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: zihaomu <zmu@amd.com>
The CPU update callback waits for KERNEL fences before writing a batch
of page-table entries. A pending signal can interrupt that wait and
return -ERESTARTSYS without writing the entries. The PTE walker ignores
the callback's result and advances past that batch.
Page-table updates must complete instead of abandoning writes because
of a signal. Make the dependency wait uninterruptible and remove the
interruption error path from the CPU update callback. The dependencies
are still waited on before any entry is written.
Keep the shared update callback type, the preparation phase, and the
SDMA backend unchanged. The CPU callback still returns zero through the
existing interface.
Link: https://lore.kernel.org/r/4edd67f7-381a-4446-9509-552739c8f016@amd.com
Signed-off-by: zihaomu <zmu@amd.com>
---
Hi Christian,
> Yeah we need to remove those error codes. PTE updates generally can't fail.
Thanks for clarifying. I have replaced the propagation proposal with
this focused first step for the CPU update callback. It waits for the
KERNEL dependencies without allowing a signal to abandon the PTE writes.
This is based on drm-amdgpu-next at a6836fcbb5b2. The previous walker
error-propagation diff and its cleanup prerequisites are not included.
This is not a complete conversion of all VM operations to a no-fail
interface. The CPU preparation wait remains interruptible before the
update phase. The shared callback type and SDMA backend are unchanged;
SDMA dependency registration and replacement-job allocations can still
fail. I have not hidden those failures behind a void callback or added
an unbounded allocation retry loop.
For the remaining SDMA work, is there a reserved-resource or map/unmap
split series I should build on? Job, scheduler-fence and IB allocations
need a forward-progress design before the remaining errors can be
removed safely.
Validation of this patch:
- 28 modeled dependency scenarios and 384 parameter combinations, using
the native CPU update and reservation traversal function bodies with
simulated fence/storage primitives. Both the original omitted-write
behavior and a deliberately broken swallow-the-error control were
detected. ASan/UBSan reported no errors.
- amdgpu_vm_cpu.o compiled with GCC 13.3.0 and CONFIG_DRM_AMDGPU=m in a
fresh private build directory. Strict checkpatch: 0 errors/warnings/checks.
The PTE write/flags/commit code is unchanged. This has not been linked
into a full module or loaded on hardware, and it does not establish a
fix for the original multi-second queue stalls. The independent CPU
fence-scan performance proposal is not included here.
Regards,
Zihao
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
index d03b9bbe20c..dc1cebbc0d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
@@ -78,12 +78,10 @@ static int amdgpu_vm_cpu_update(struct amdgpu_vm_update_params *p,
{
unsigned int i;
uint64_t value;
- long r;
- r = dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
- true, MAX_SCHEDULE_TIMEOUT);
- if (r < 0)
- return r;
+ /* Page-table updates must not skip writes because of a signal. */
+ dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
+ false, MAX_SCHEDULE_TIMEOUT);
pe += (unsigned long)amdgpu_bo_kptr(&vmbo->bo);
base-commit: a6836fcbb5b22968019256025321d655b78bc49c
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC PATCH] drm/amdgpu: make CPU page-table update waits uninterruptible
2026-11-11 5:39 ` [RFC PATCH] drm/amdgpu: make CPU page-table update waits uninterruptible Mu, Zihao
@ 2026-11-11 9:06 ` Christian König
2026-11-11 11:55 ` [RFC PATCH v2 0/6] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
` (6 more replies)
0 siblings, 7 replies; 25+ messages in thread
From: Christian König @ 2026-11-11 9:06 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/11/26 06:39, Mu, Zihao wrote:
> From: zihaomu <zmu@amd.com>
>
> The CPU update callback waits for KERNEL fences before writing a batch
> of page-table entries. A pending signal can interrupt that wait and
> return -ERESTARTSYS without writing the entries. The PTE walker ignores
> the callback's result and advances past that batch.
>
> Page-table updates must complete instead of abandoning writes because
> of a signal. Make the dependency wait uninterruptible and remove the
> interruption error path from the CPU update callback. The dependencies
> are still waited on before any entry is written.
>
> Keep the shared update callback type, the preparation phase, and the
> SDMA backend unchanged. The CPU callback still returns zero through the
> existing interface.
>
> Link: https://lore.kernel.org/r/4edd67f7-381a-4446-9509-552739c8f016@amd.com
> Signed-off-by: zihaomu <zmu@amd.com>
The Signed-off-by line should be your full legal name, e.g. similar to a signature. At least of hand zihaomu doesn't looks correct.
> ---
> Hi Christian,
>
>> Yeah we need to remove those error codes. PTE updates generally can't fail.
>
> Thanks for clarifying. I have replaced the propagation proposal with
> this focused first step for the CPU update callback. It waits for the
> KERNEL dependencies without allowing a signal to abandon the PTE writes.
>
> This is based on drm-amdgpu-next at a6836fcbb5b2. The previous walker
> error-propagation diff and its cleanup prerequisites are not included.
>
> This is not a complete conversion of all VM operations to a no-fail
> interface. The CPU preparation wait remains interruptible before the
> update phase. The shared callback type and SDMA backend are unchanged;
> SDMA dependency registration and replacement-job allocations can still
> fail. I have not hidden those failures behind a void callback or added
> an unbounded allocation retry loop.
>
> For the remaining SDMA work, is there a reserved-resource or map/unmap
> split series I should build on? Job, scheduler-fence and IB allocations
> need a forward-progress design before the remaining errors can be
> removed safely.
>
> Validation of this patch:
> - 28 modeled dependency scenarios and 384 parameter combinations, using
> the native CPU update and reservation traversal function bodies with
> simulated fence/storage primitives. Both the original omitted-write
> behavior and a deliberately broken swallow-the-error control were
> detected. ASan/UBSan reported no errors.
> - amdgpu_vm_cpu.o compiled with GCC 13.3.0 and CONFIG_DRM_AMDGPU=m in a
> fresh private build directory. Strict checkpatch: 0 errors/warnings/checks.
>
> The PTE write/flags/commit code is unchanged. This has not been linked
> into a full module or loaded on hardware, and it does not establish a
> fix for the original multi-second queue stalls. The independent CPU
> fence-scan performance proposal is not included here.
>
> Regards,
> Zihao
>
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> index d03b9bbe20c..dc1cebbc0d9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> @@ -78,12 +78,10 @@ static int amdgpu_vm_cpu_update(struct amdgpu_vm_update_params *p,
> {
> unsigned int i;
> uint64_t value;
> - long r;
>
> - r = dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
> - true, MAX_SCHEDULE_TIMEOUT);
> - if (r < 0)
> - return r;
> + /* Page-table updates must not skip writes because of a signal. */
> + dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
> + false, MAX_SCHEDULE_TIMEOUT);
The problem is rather that we shouldn't call dma_resv_wait_timeout() here in the first place.
That should be done in amdgpu_vm_cpu_prepare() instead but currently can't be done there because we don't call commit/prepare around page table allocation.
So we need to fix that first and then fix this bug here.
Regards,
Christian.
>
> pe += (unsigned long)amdgpu_bo_kptr(&vmbo->bo);
>
>
> base-commit: a6836fcbb5b22968019256025321d655b78bc49c
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH v2 0/6] drm/amdgpu: prepare updates across page-table allocation
2026-11-11 9:06 ` Christian König
@ 2026-11-11 11:55 ` Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 1/6] drm/amdgpu: handle missing or empty SDMA VM update jobs Mu, Zihao
` (5 subsequent siblings)
6 siblings, 0 replies; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:55 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
Hi Christian, Alex and Cascardo,
This v2 replaces my posted interrupt-flag-only RFC. I first bracket
page-table allocation with commit/prepare, then move the CPU KERNEL
dependency wait into prepare, as requested. My author and sign-off name
is corrected to Zihao Mu.
The six patches are based on drm-amdgpu-next a6836fcbb5b2:
1. Handle missing/empty SDMA jobs and consume each job once.
2. Track internal SDMA completion independently of the public output
fence, including immediate submissions, IB rollover and nested clears.
3. Finalize partial range updates on error, preserving the first error
and using the preceding internal completion for TLB ordering.
4. Unwind failed page-table initialization through the VM entry teardown;
finish the clear's current work before returning its original error.
5. Commit before allocation and re-prepare after clearing the new table,
preserving the original sync even when the old phase had an empty job.
6. Wait for shared root KERNEL dependencies in CPU prepare, including
NULL-sync calls; remove the reservation wait from CPU update.
Patches 1 and 3 adapt Cascardo's earlier cleanup proposals, retaining his
authorship and original sign-off. I am not claiming that work as new.
The internal completion prerequisite comes before partial-error range
finalization, so an earlier submitted IB is already tracked if allocating
its replacement fails. A clear completion is also retained if the outer
prepare fails. The public !immediate output condition and DONT_PIPELINE
policy are unchanged. PDE error handling is outside this series.
The KERNEL wait keeps its original interruptible policy. Interruption now
fails prepare before that phase starts writing; previous phases have
been committed and receive partial-range finalization. Allocation drops
the eviction lock and can introduce new move dependencies, which is why
the next phase must prepare again. The CPU write/flags/commit tail is
unchanged.
This does not reintroduce the old PTE-walker error-propagation proposal,
mapping retention or the independent no-KERNEL scan fastpath. SDMA
allocations can still fail. This is not a complete conversion of all
mapping/unmapping operations to a no-fail interface.
Local validation:
- All six intermediate source states compiled amdgpu_vm.o,
amdgpu_vm_pt.o, amdgpu_vm_sdma.o and amdgpu_vm_cpu.o with GCC 13.3.0
and CONFIG_DRM_AMDGPU=m.
- Each intermediate state passed its applicable extracted-function
checks. The final state passed 106 successful range/backend cases,
149 failure cases and 2 successful PDE cases. SDMA fences remain
pending at submit; the native TLB callback and worker bodies run with
both worker-first and GPU-first simulated completion schedules.
- Two older source states reproduce the missing completion dependency;
four deliberate mutations detect missing nested completion, prepare,
sync and allocation boundaries.
- 960 CPU phase/parameter cases preserve PTE values/flags, with two
KERNEL-wait interruption cases, one explicit-sync interruption case
and two detected missing-wait controls. A 64-batch single-phase model
calls the reservation wait once instead of 64 times; this is a call
count, not a timing result. ASan/UBSan reported no errors.
The GPU, scheduler, fence primitives, BO/TTM lifetime and PTE/PDE walker
orchestration in those checks are simulated. They do not prove kernel
concurrency or hardware ordering. No full module link, driver load, live
fault injection or hardware A/B has been done. This does not establish
a fix for the original multi-second GPU queue stalls.
Please review the phase boundaries and internal completion ownership,
especially immediate SDMA submissions and errors after a nested clear.
Changes from the posted RFC:
- Establish allocation boundaries before moving the CPU wait.
- Remove the wait from update instead of making that wait uninterruptible.
- Add narrowly scoped job, completion and initialization prerequisites.
- Correct my full author/sign-off name and retain prior-work attribution.
Earlier cleanup proposals:
https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-1-d6d835591e0f@igalia.com/
https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-4-d6d835591e0f@igalia.com/
Latest review:
https://lore.kernel.org/r/938026a1-3b02-40b4-98e3-a0a22f045f82@amd.com
Regards,
Zihao
base-commit: a6836fcbb5b22968019256025321d655b78bc49c
--
2.43.0
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH v2 1/6] drm/amdgpu: handle missing or empty SDMA VM update jobs
2026-11-11 9:06 ` Christian König
2026-11-11 11:55 ` [RFC PATCH v2 0/6] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
@ 2026-11-11 11:55 ` Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 2/6] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
` (4 subsequent siblings)
6 siblings, 0 replies; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:55 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
A failed PTE walk can leave an allocated SDMA job with no commands, or
no current job after submitting a full IB and failing to allocate its
replacement. A final commit of partial work must handle both states.
Return without submitting when there is no job, and free an empty job
instead of emitting an empty IB. Clear the job pointer after freeing
or submitting it so that a later final commit cannot consume it twice.
Based on Cascardo's earlier NULL/empty-job patch. Retain the current
int-returning commit callback and scheduler layout.
[zmu: Clear the job pointer after submission and retain the current API.]
Link: https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-1-d6d835591e0f@igalia.com/
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index d3a5e848741..ade6c7aa15c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -91,6 +91,7 @@ static int amdgpu_vm_sdma_prepare(struct amdgpu_vm_update_params *p,
if (r) {
p->num_dw_left = 0;
amdgpu_job_free(p->job);
+ p->job = NULL;
}
return r;
}
@@ -107,14 +108,23 @@ static int amdgpu_vm_sdma_prepare(struct amdgpu_vm_update_params *p,
static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
struct dma_fence **fence)
{
- struct amdgpu_ib *ib = p->job->ibs;
+ struct amdgpu_ib *ib;
struct amdgpu_ring *ring;
struct dma_fence *f;
+ if (!p->job)
+ return 0;
+
+ ib = p->job->ibs;
+ if (!ib->length_dw) {
+ amdgpu_job_free(p->job);
+ p->job = NULL;
+ return 0;
+ }
+
ring = container_of(p->vm->delayed.rq->sched, struct amdgpu_ring,
sched);
- WARN_ON(ib->length_dw == 0);
amdgpu_ring_pad_ib(ring, ib);
if (p->needs_flush)
@@ -122,6 +132,7 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
WARN_ON(ib->length_dw > p->num_dw_left);
f = amdgpu_job_submit(p->job);
+ p->job = NULL;
if (p->unlocked) {
struct dma_fence *tmp = dma_fence_get(f);
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v2 2/6] drm/amdgpu: track internal completion of SDMA VM updates
2026-11-11 9:06 ` Christian König
2026-11-11 11:55 ` [RFC PATCH v2 0/6] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 1/6] drm/amdgpu: handle missing or empty SDMA VM update jobs Mu, Zihao
@ 2026-11-11 11:55 ` Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 3/6] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
` (3 subsequent siblings)
6 siblings, 0 replies; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:55 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
The public output fence is intentionally not updated for immediate VM
updates. It can also be omitted on intermediate IB submissions. Neither
case removes the need to order a later TLB invalidation after the SDMA
commands have completed.
Keep an owned reference to the latest submission in the update context,
including immediate and rollover submissions. Use that internal fence
for range TLB handling and release it when each context is destroyed.
Leave the public output-fence condition and DONT_PIPELINE policy alone.
Return a nested table clear completion to its outer context so an early
exit can also wait for that clear before finalizing the range.
This establishes completion tracking before adding partial-error range
finalization, so that finalization never depends on a later patch to
recover an already submitted IB's fence.
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 10 +++++++---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 8 +++++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 12 +++++++++---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 ++
4 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 94b1cfe05e3..1e41f159ced 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -658,7 +658,9 @@ int amdgpu_vm_validate(struct amdgpu_device *adev, struct amdgpu_vm *vm,
bo_base->moved = true;
if (vm->generation != new_vm_generation) {
- r = amdgpu_vm_pt_clear(adev, vm, to_amdgpu_bo_vm(bo_base->bo), false);
+ r = amdgpu_vm_pt_clear(adev, vm,
+ to_amdgpu_bo_vm(bo_base->bo),
+ false, NULL);
if (r)
return r;
}
@@ -1081,6 +1083,7 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
amdgpu_vm_bo_idle(entry);
error:
+ dma_fence_put(params.last_update);
drm_dev_exit(idx);
return r;
}
@@ -1286,7 +1289,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
goto error_free;
if (params.needs_flush) {
- amdgpu_vm_tlb_flush(¶ms, *fence, tlb_cb);
+ amdgpu_vm_tlb_flush(¶ms, params.last_update, tlb_cb);
tlb_cb = NULL;
}
@@ -1294,6 +1297,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
error_free:
kfree(tlb_cb);
+ dma_fence_put(params.last_update);
amdgpu_vm_end_critical(¶ms);
drm_dev_exit(idx);
return r;
@@ -2710,7 +2714,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm,
if (r)
goto error_free_root;
- r = amdgpu_vm_pt_clear(adev, vm, root, false);
+ r = amdgpu_vm_pt_clear(adev, vm, root, false, NULL);
if (r)
goto error_free_root;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
index 8ebb0b03329..71fc1fffaa6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
@@ -77,6 +77,11 @@ struct amdgpu_vm_update_params {
*/
struct amdgpu_job *job;
+ /**
+ * @last_update: owned reference to the last SDMA completion fence
+ */
+ struct dma_fence *last_update;
+
/**
* @num_dw_left: number of dw left for the IB
*/
@@ -116,7 +121,8 @@ struct amdgpu_vm_update_funcs {
};
int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
- struct amdgpu_bo_vm *vmbo, bool immediate);
+ struct amdgpu_bo_vm *vmbo, bool immediate,
+ struct dma_fence **last_update);
int amdgpu_vm_pt_create(struct amdgpu_device *adev, struct amdgpu_vm *vm,
int level, bool immediate, struct amdgpu_bo_vm **vmbo,
int32_t xcp_id);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index c73eb416c0e..9955be111ce 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -353,6 +353,7 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev,
* @vm: VM to clear BO from
* @vmbo: BO to clear
* @immediate: use an immediate update
+ * @last_update: optional output for the internal clear completion fence
*
* Root PD needs to be reserved when calling this.
*
@@ -360,7 +361,8 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev,
* 0 on success, errno otherwise.
*/
int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
- struct amdgpu_bo_vm *vmbo, bool immediate)
+ struct amdgpu_bo_vm *vmbo, bool immediate,
+ struct dma_fence **last_update)
{
unsigned int level = adev->vm_manager.root_level;
struct ttm_operation_ctx ctx = { true, false };
@@ -390,11 +392,11 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
if (!drm_dev_enter(adev_to_drm(adev), &idx))
return -ENODEV;
+ memset(¶ms, 0, sizeof(params));
r = vm->update_funcs->map_table(vmbo);
if (r)
goto exit;
- memset(¶ms, 0, sizeof(params));
params.adev = adev;
params.vm = vm;
params.immediate = immediate;
@@ -429,6 +431,9 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
r = vm->update_funcs->commit(¶ms, &vm->last_update);
exit:
+ if (last_update && params.last_update)
+ swap(*last_update, params.last_update);
+ dma_fence_put(params.last_update);
drm_dev_exit(idx);
return r;
}
@@ -543,7 +548,8 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
*/
pt_bo->parent = amdgpu_bo_ref(cursor->parent->bo);
amdgpu_vm_bo_base_init(entry, p->vm, pt_bo);
- r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate);
+ r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate,
+ &p->last_update);
if (r)
goto error_unpin;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index ade6c7aa15c..02f3271d96a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -133,6 +133,8 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
WARN_ON(ib->length_dw > p->num_dw_left);
f = amdgpu_job_submit(p->job);
p->job = NULL;
+ dma_fence_put(p->last_update);
+ p->last_update = dma_fence_get(f);
if (p->unlocked) {
struct dma_fence *tmp = dma_fence_get(f);
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v2 3/6] drm/amdgpu: finalize partial VM range updates on error
2026-11-11 9:06 ` Christian König
` (2 preceding siblings ...)
2026-11-11 11:55 ` [RFC PATCH v2 2/6] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
@ 2026-11-11 11:55 ` Mu, Zihao
2026-11-11 12:44 ` Thadeu Lima de Souza Cascardo
2026-11-11 11:55 ` [RFC PATCH v2 4/6] drm/amdgpu: unwind failed page-table initialization Mu, Zihao
` (2 subsequent siblings)
6 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:55 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
A PTE walk can fail after earlier batches changed page tables or queued
SDMA commands. Its current early exit bypasses the final commit, TLB
handling and draining of the stack-local tlb_flush_waitlist.
Stop the range walk on failure, then finish the existing commit, TLB
and page-table cleanup sequence. Preserve the first update error so
that successful finalization cannot turn an incomplete mapping into
success. Preparation failures keep their existing early exit.
The preceding internal-completion tracking keeps the latest submitted
IB available to TLB finalization, including immediate updates and failed
replacement-job allocations. Finalization does not rely on a caller
output fence being present or updated.
[zmu: Adapt to the int-returning commit API and tracked completion.]
Link: https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-4-d6d835591e0f@igalia.com/
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 1e41f159ced..f76459f8312 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1179,7 +1179,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
struct amdgpu_vm_tlb_seq_struct *tlb_cb;
struct amdgpu_vm_update_params params;
struct amdgpu_res_cursor cursor;
- int r, idx;
+ int r, commit_r, idx;
if (!drm_dev_enter(adev_to_drm(adev), &idx))
return -ENODEV;
@@ -1278,14 +1278,17 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
tmp = start + num_entries;
r = amdgpu_vm_ptes_update(¶ms, start, tmp, addr, flags);
if (r)
- goto error_free;
+ break;
amdgpu_res_next(&cursor, num_entries * AMDGPU_GPU_PAGE_SIZE);
start = tmp;
}
- r = vm->update_funcs->commit(¶ms, fence);
- if (r)
+ /* Finish partial work without hiding the original update error. */
+ commit_r = vm->update_funcs->commit(¶ms, fence);
+ if (!r)
+ r = commit_r;
+ if (commit_r)
goto error_free;
if (params.needs_flush) {
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v2 4/6] drm/amdgpu: unwind failed page-table initialization
2026-11-11 9:06 ` Christian König
` (3 preceding siblings ...)
2026-11-11 11:55 ` [RFC PATCH v2 3/6] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
@ 2026-11-11 11:55 ` Mu, Zihao
2026-11-11 13:10 ` Christian König
2026-11-11 11:55 ` [RFC PATCH v2 5/6] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
2026-11-11 11:56 ` [RFC PATCH v2 6/6] drm/amdgpu: wait for page-table moves in CPU prepare Mu, Zihao
6 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:55 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
A new table is linked to the VM before it is cleared. If clearing fails,
the allocation path only unpins and drops the BO reference, leaving the
entry, VM status list and accounting referring to that BO.
Unwind the initialized entry through amdgpu_vm_pt_free() instead. Finish
any partial SDMA clear before returning its original error, so no current
job is abandoned. The tracked completion and reservation fences retain
the submitted work while TTM handles the BO release.
Keep this initialization-error fix separate from the update-phase
boundary changes. PDE iteration and its error policy are not changed.
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index 9955be111ce..bdd06b7718c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -372,7 +372,7 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
struct amdgpu_bo *bo = &vmbo->bo;
uint64_t value = 0, flags = 0;
uint64_t addr;
- int r, idx;
+ int r, commit_r, idx;
/* Figure out our place in the hierarchy */
if (ancestor->parent) {
@@ -426,10 +426,9 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
r = vm->update_funcs->update(¶ms, vmbo, addr, 0, entries,
value, flags);
- if (r)
- goto exit;
-
- r = vm->update_funcs->commit(¶ms, &vm->last_update);
+ commit_r = vm->update_funcs->commit(¶ms, &vm->last_update);
+ if (!r)
+ r = commit_r;
exit:
if (last_update && params.last_update)
swap(*last_update, params.last_update);
@@ -507,6 +506,8 @@ int amdgpu_vm_pt_create(struct amdgpu_device *adev, struct amdgpu_vm *vm,
return 0;
}
+static void amdgpu_vm_pt_free(struct amdgpu_vm_bo_base *entry);
+
/**
* amdgpu_vm_pt_alloc - Allocate a specific page table
*
@@ -550,8 +551,10 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
amdgpu_vm_bo_base_init(entry, p->vm, pt_bo);
r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate,
&p->last_update);
- if (r)
- goto error_unpin;
+ if (r) {
+ amdgpu_vm_pt_free(entry);
+ return r;
+ }
return 0;
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v2 5/6] drm/amdgpu: restart VM updates around page-table allocation
2026-11-11 9:06 ` Christian König
` (4 preceding siblings ...)
2026-11-11 11:55 ` [RFC PATCH v2 4/6] drm/amdgpu: unwind failed page-table initialization Mu, Zihao
@ 2026-11-11 11:55 ` Mu, Zihao
2026-11-11 12:56 ` Thadeu Lima de Souza Cascardo
2026-11-11 11:56 ` [RFC PATCH v2 6/6] drm/amdgpu: wait for page-table moves in CPU prepare Mu, Zihao
6 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:55 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
Page-table allocation drops the VM eviction lock and can introduce new
move dependencies. End the current update phase before that allocation,
then prepare a new phase after creating and clearing the table.
Preserve the original sync across each prepare. An empty SDMA phase may
be freed before allocation, so its job cannot be relied on to carry
those dependencies into the resumed phase.
Transfer the nested clear's internal completion to the outer context as
well. If a subsequent prepare fails, range finalization still orders
the TLB work after the already submitted clear, including immediate
submissions that do not expose a public output fence.
Leave a successfully cleared table installed when prepare fails, and
keep the existing fast return for already allocated tables. The earlier
cleanup patches handle failed initialization and partial range work.
No PDE iteration or output-fence policy is changed here.
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 5 +++++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 8 +++++++-
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index f76459f8312..7d85e5e74c5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1206,6 +1206,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
params.vm = vm;
params.immediate = immediate;
params.pages_addr = pages_addr;
+ params.sync = sync;
params.unlocked = unlocked;
params.needs_flush = flush_tlb;
params.override_pte = allow_override && adev->gmc.override_pte;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
index 71fc1fffaa6..df1aaa00603 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
@@ -82,6 +82,11 @@ struct amdgpu_vm_update_params {
*/
struct dma_fence *last_update;
+ /**
+ * @sync: original dependencies to preserve across prepare calls
+ */
+ struct amdgpu_sync *sync;
+
/**
* @num_dw_left: number of dw left for the IB
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index bdd06b7718c..e42af004dc0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -531,6 +531,11 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
if (entry->bo)
return 0;
+ /* Allocation may introduce new move dependencies. End this phase. */
+ r = p->vm->update_funcs->commit(p, NULL);
+ if (r)
+ return r;
+
amdgpu_vm_end_critical(p);
r = amdgpu_vm_pt_create(p->adev, p->vm, cursor->level, p->immediate,
&pt, p->vm->root.bo->xcp_id);
@@ -556,7 +561,8 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
return r;
}
- return 0;
+ return p->vm->update_funcs->prepare(p, p->sync,
+ AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE);
error_unpin:
if (p->vm->is_npa)
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v2 6/6] drm/amdgpu: wait for page-table moves in CPU prepare
2026-11-11 9:06 ` Christian König
` (5 preceding siblings ...)
2026-11-11 11:55 ` [RFC PATCH v2 5/6] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
@ 2026-11-11 11:56 ` Mu, Zihao
2026-11-11 13:14 ` Christian König
6 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-11 11:56 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
The CPU backend repeats reservation traversal for every PTE batch. An
interrupted wait can also skip a batch whose result the walker ignores.
Wait for the shared root reservation's KERNEL dependencies in prepare,
including calls without an explicit sync. Allocation now commits and
restarts the phase, so dependencies added while the eviction lock was
dropped are covered before the resumed phase writes entries.
Preserve the existing interruptible wait policy. An interruption now
fails prepare before that phase writes any entries, while prior phases
have been committed and receive normal partial-range finalization.
Keep explicit sync handling and the CPU write/flags/commit tail intact.
The update callback now only writes entries and returns zero through
the existing interface. This does not claim that all VM preparation or
SDMA allocation operations have become non-failing.
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
index d03b9bbe20c..89e76e18d9e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
@@ -52,10 +52,19 @@ static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p,
struct amdgpu_sync *sync,
u64 k_job_id)
{
- if (!sync)
- return 0;
+ struct dma_resv *resv = p->vm->root.bo->tbo.base.resv;
+ long r;
+
+ if (sync) {
+ r = amdgpu_sync_wait(sync, true);
+ if (r)
+ return r;
+ }
- return amdgpu_sync_wait(sync, true);
+ /* Allocation restarts prepare before any further page-table writes. */
+ r = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_KERNEL,
+ true, MAX_SCHEDULE_TIMEOUT);
+ return r < 0 ? r : 0;
}
/**
@@ -78,12 +87,6 @@ static int amdgpu_vm_cpu_update(struct amdgpu_vm_update_params *p,
{
unsigned int i;
uint64_t value;
- long r;
-
- r = dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
- true, MAX_SCHEDULE_TIMEOUT);
- if (r < 0)
- return r;
pe += (unsigned long)amdgpu_bo_kptr(&vmbo->bo);
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v2 3/6] drm/amdgpu: finalize partial VM range updates on error
2026-11-11 11:55 ` [RFC PATCH v2 3/6] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
@ 2026-11-11 12:44 ` Thadeu Lima de Souza Cascardo
0 siblings, 0 replies; 25+ messages in thread
From: Thadeu Lima de Souza Cascardo @ 2026-11-11 12:44 UTC (permalink / raw)
To: Mu, Zihao
Cc: Koenig, Christian, amd-gfx@lists.freedesktop.org,
Deucher, Alexander
On Wed, Nov 11, 2026 at 11:55:51AM +0000, Mu, Zihao wrote:
> From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
>
> A PTE walk can fail after earlier batches changed page tables or queued
> SDMA commands. Its current early exit bypasses the final commit, TLB
> handling and draining of the stack-local tlb_flush_waitlist.
>
> Stop the range walk on failure, then finish the existing commit, TLB
> and page-table cleanup sequence. Preserve the first update error so
> that successful finalization cannot turn an incomplete mapping into
> success. Preparation failures keep their existing early exit.
>
> The preceding internal-completion tracking keeps the latest submitted
> IB available to TLB finalization, including immediate updates and failed
> replacement-job allocations. Finalization does not rely on a caller
> output fence being present or updated.
>
> [zmu: Adapt to the int-returning commit API and tracked completion.]
> Link: https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-4-d6d835591e0f@igalia.com/
> Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
> Signed-off-by: Zihao Mu <zmu@amd.com>
If you apply this patch [1], you don't need to adjust for the int API, and
it makes both this and the next patch cleaner.
Is there any reason not to apply it?
I am sending another cleanup in response to one of the patches for
comments, if they look fine, I can send it properly.
Thanks.
Cascardo.
[1] https://lore.kernel.org/amd-gfx/20260924-amdgpu_vm_update_funcs_commit_void-v1-1-9ad7a8283920@igalia.com/
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 1e41f159ced..f76459f8312 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1179,7 +1179,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> struct amdgpu_vm_tlb_seq_struct *tlb_cb;
> struct amdgpu_vm_update_params params;
> struct amdgpu_res_cursor cursor;
> - int r, idx;
> + int r, commit_r, idx;
>
> if (!drm_dev_enter(adev_to_drm(adev), &idx))
> return -ENODEV;
> @@ -1278,14 +1278,17 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> tmp = start + num_entries;
> r = amdgpu_vm_ptes_update(¶ms, start, tmp, addr, flags);
> if (r)
> - goto error_free;
> + break;
>
> amdgpu_res_next(&cursor, num_entries * AMDGPU_GPU_PAGE_SIZE);
> start = tmp;
> }
>
> - r = vm->update_funcs->commit(¶ms, fence);
> - if (r)
> + /* Finish partial work without hiding the original update error. */
> + commit_r = vm->update_funcs->commit(¶ms, fence);
> + if (!r)
> + r = commit_r;
> + if (commit_r)
> goto error_free;
>
> if (params.needs_flush) {
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v2 5/6] drm/amdgpu: restart VM updates around page-table allocation
2026-11-11 11:55 ` [RFC PATCH v2 5/6] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
@ 2026-11-11 12:56 ` Thadeu Lima de Souza Cascardo
0 siblings, 0 replies; 25+ messages in thread
From: Thadeu Lima de Souza Cascardo @ 2026-11-11 12:56 UTC (permalink / raw)
To: Mu, Zihao
Cc: Koenig, Christian, amd-gfx@lists.freedesktop.org,
Deucher, Alexander
On Wed, Nov 11, 2026 at 11:55:58AM +0000, Mu, Zihao wrote:
> From: Zihao Mu <zmu@amd.com>
>
> Page-table allocation drops the VM eviction lock and can introduce new
> move dependencies. End the current update phase before that allocation,
> then prepare a new phase after creating and clearing the table.
>
> Preserve the original sync across each prepare. An empty SDMA phase may
> be freed before allocation, so its job cannot be relied on to carry
> those dependencies into the resumed phase.
>
> Transfer the nested clear's internal completion to the outer context as
> well. If a subsequent prepare fails, range finalization still orders
> the TLB work after the already submitted clear, including immediate
> submissions that do not expose a public output fence.
>
> Leave a successfully cleared table installed when prepare fails, and
> keep the existing fast return for already allocated tables. The earlier
> cleanup patches handle failed initialization and partial range work.
> No PDE iteration or output-fence policy is changed here.
>
> Signed-off-by: Zihao Mu <zmu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 5 +++++
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 8 +++++++-
> 3 files changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index f76459f8312..7d85e5e74c5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1206,6 +1206,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> params.vm = vm;
> params.immediate = immediate;
> params.pages_addr = pages_addr;
> + params.sync = sync;
> params.unlocked = unlocked;
> params.needs_flush = flush_tlb;
> params.override_pte = allow_override && adev->gmc.override_pte;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> index 71fc1fffaa6..df1aaa00603 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> @@ -82,6 +82,11 @@ struct amdgpu_vm_update_params {
> */
> struct dma_fence *last_update;
>
> + /**
> + * @sync: original dependencies to preserve across prepare calls
> + */
> + struct amdgpu_sync *sync;
> +
> /**
> * @num_dw_left: number of dw left for the IB
> */
Similarly, we might want to use the same job ID, so here is my proposal,
whcih should go before this patch (and then prepare needs to drop its job
id parameter).
From 05a61399d6bcddb3eb75786e25e88221e70afdfd Mon Sep 17 00:00:00 2001
From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Date: Tue, 10 Nov 2026 20:16:17 -0300
Subject: [PATCH] drm/amdgpu: pass job id as part of amdgpu_vm_update_params
Jobs may be allocated not only during prepare step, but also during the
update step. Also, in case we might want to call prepare again after a
commit when dropping the lock, we want to use the same original job ID.
So make it part of the update params instead of a parameter for prepare
only.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++----
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 4 +---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 7 ++++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 4 ++--
4 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 94b1cfe05e3e0..2e95f182db838 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1054,9 +1054,9 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
params.adev = adev;
params.vm = vm;
params.immediate = immediate;
+ params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES;
- r = vm->update_funcs->prepare(¶ms, NULL,
- AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES);
+ r = vm->update_funcs->prepare(¶ms, NULL);
if (r)
goto error;
@@ -1207,6 +1207,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
params.needs_flush = flush_tlb;
params.override_pte = allow_override && adev->gmc.override_pte;
INIT_LIST_HEAD(¶ms.tlb_flush_waitlist);
+ params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE;
r = amdgpu_vm_begin_critical(¶ms);
if (r)
@@ -1220,8 +1221,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
dma_fence_put(tmp);
}
- r = vm->update_funcs->prepare(¶ms, sync,
- AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE);
+ r = vm->update_funcs->prepare(¶ms, sync);
if (r)
goto error_free;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
index d03b9bbe20c7c..dd62fef0ca164 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
@@ -43,14 +43,12 @@ static int amdgpu_vm_cpu_map_table(struct amdgpu_bo_vm *table)
*
* @p: see amdgpu_vm_update_params definition
* @sync: sync obj with fences to wait on
- * @k_job_id: the id for tracing/debug purposes
*
* Returns:
* Negativ errno, 0 for success.
*/
static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p,
- struct amdgpu_sync *sync,
- u64 k_job_id)
+ struct amdgpu_sync *sync)
{
if (!sync)
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
index 8ebb0b033291e..c6e4469bebef7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
@@ -102,12 +102,17 @@ struct amdgpu_vm_update_params {
* @tlb_flush_waitlist: temporary storage for BOs until tlb_flush
*/
struct list_head tlb_flush_waitlist;
+
+ /**
+ * @k_job_id: job id to use for this update.
+ */
+ u64 k_job_id;
};
struct amdgpu_vm_update_funcs {
int (*map_table)(struct amdgpu_bo_vm *bo);
int (*prepare)(struct amdgpu_vm_update_params *p,
- struct amdgpu_sync *sync, u64 k_job_id);
+ struct amdgpu_sync *sync);
int (*update)(struct amdgpu_vm_update_params *p,
struct amdgpu_bo_vm *bo, uint64_t pe, uint64_t addr,
unsigned count, uint32_t incr, uint64_t flags);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index c73eb416c0e26..856ea8bedb64b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -398,9 +398,9 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
params.adev = adev;
params.vm = vm;
params.immediate = immediate;
+ params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR;
- r = vm->update_funcs->prepare(¶ms, NULL,
- AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR);
+ r = vm->update_funcs->prepare(¶ms, NULL);
if (r)
goto exit;
--
2.47.3
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> index bdd06b7718c..e42af004dc0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> @@ -531,6 +531,11 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
> if (entry->bo)
> return 0;
>
> + /* Allocation may introduce new move dependencies. End this phase. */
> + r = p->vm->update_funcs->commit(p, NULL);
> + if (r)
> + return r;
> +
> amdgpu_vm_end_critical(p);
> r = amdgpu_vm_pt_create(p->adev, p->vm, cursor->level, p->immediate,
> &pt, p->vm->root.bo->xcp_id);
> @@ -556,7 +561,8 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
> return r;
> }
>
> - return 0;
> + return p->vm->update_funcs->prepare(p, p->sync,
> + AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE);
>
> error_unpin:
> if (p->vm->is_npa)
> --
> 2.43.0
>
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v2 4/6] drm/amdgpu: unwind failed page-table initialization
2026-11-11 11:55 ` [RFC PATCH v2 4/6] drm/amdgpu: unwind failed page-table initialization Mu, Zihao
@ 2026-11-11 13:10 ` Christian König
0 siblings, 0 replies; 25+ messages in thread
From: Christian König @ 2026-11-11 13:10 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/11/26 12:55, Mu, Zihao wrote:
> From: Zihao Mu <zmu@amd.com>
>
> A new table is linked to the VM before it is cleared. If clearing fails,
> the allocation path only unpins and drops the BO reference, leaving the
> entry, VM status list and accounting referring to that BO.
>
> Unwind the initialized entry through amdgpu_vm_pt_free() instead. Finish
> any partial SDMA clear before returning its original error, so no current
> job is abandoned. The tracked completion and reservation fences retain
> the submitted work while TTM handles the BO release.
>
> Keep this initialization-error fix separate from the update-phase
> boundary changes. PDE iteration and its error policy are not changed.
James already stumbled over that problem and the patch to fix that is already reviewed and should show up in amd-staging-drm-next soon.
Regards,
Christian.
>
> Signed-off-by: Zihao Mu <zmu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 17 ++++++++++-------
> 1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> index 9955be111ce..bdd06b7718c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> @@ -372,7 +372,7 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> struct amdgpu_bo *bo = &vmbo->bo;
> uint64_t value = 0, flags = 0;
> uint64_t addr;
> - int r, idx;
> + int r, commit_r, idx;
>
> /* Figure out our place in the hierarchy */
> if (ancestor->parent) {
> @@ -426,10 +426,9 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>
> r = vm->update_funcs->update(¶ms, vmbo, addr, 0, entries,
> value, flags);
> - if (r)
> - goto exit;
> -
> - r = vm->update_funcs->commit(¶ms, &vm->last_update);
> + commit_r = vm->update_funcs->commit(¶ms, &vm->last_update);
> + if (!r)
> + r = commit_r;
> exit:
> if (last_update && params.last_update)
> swap(*last_update, params.last_update);
> @@ -507,6 +506,8 @@ int amdgpu_vm_pt_create(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> return 0;
> }
>
> +static void amdgpu_vm_pt_free(struct amdgpu_vm_bo_base *entry);
> +
> /**
> * amdgpu_vm_pt_alloc - Allocate a specific page table
> *
> @@ -550,8 +551,10 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
> amdgpu_vm_bo_base_init(entry, p->vm, pt_bo);
> r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate,
> &p->last_update);
> - if (r)
> - goto error_unpin;
> + if (r) {
> + amdgpu_vm_pt_free(entry);
> + return r;
> + }
>
> return 0;
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v2 6/6] drm/amdgpu: wait for page-table moves in CPU prepare
2026-11-11 11:56 ` [RFC PATCH v2 6/6] drm/amdgpu: wait for page-table moves in CPU prepare Mu, Zihao
@ 2026-11-11 13:14 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 0/5] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
` (5 more replies)
0 siblings, 6 replies; 25+ messages in thread
From: Christian König @ 2026-11-11 13:14 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/11/26 12:56, Mu, Zihao wrote:
> From: Zihao Mu <zmu@amd.com>
>
> The CPU backend repeats reservation traversal for every PTE batch. An
> interrupted wait can also skip a batch whose result the walker ignores.
That irrelevant.
> Wait for the shared root reservation's KERNEL dependencies in prepare,
> including calls without an explicit sync. Allocation now commits and
> restarts the phase, so dependencies added while the eviction lock was
> dropped are covered before the resumed phase writes entries.
>
> Preserve the existing interruptible wait policy.
Clear NAK to that. As said before unmap operations triggered by MMU notifiers *must* complete and can't be interrupted.
We could add an interruptible flag to the update paramaters but that is most likely overkill.
> An interruption now
> fails prepare before that phase writes any entries, while prior phases
> have been committed and receive normal partial-range finalization.
> Keep explicit sync handling and the CPU write/flags/commit tail intact.
>
> The update callback now only writes entries and returns zero through
> the existing interface. This does not claim that all VM preparation or
> SDMA allocation operations have become non-failing.
>
> Signed-off-by: Zihao Mu <zmu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 21 ++++++++++++---------
> 1 file changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> index d03b9bbe20c..89e76e18d9e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> @@ -52,10 +52,19 @@ static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p,
> struct amdgpu_sync *sync,
> u64 k_job_id)
> {
> - if (!sync)
> - return 0;
> + struct dma_resv *resv = p->vm->root.bo->tbo.base.resv;
> + long r;
> +
> + if (sync) {
> + r = amdgpu_sync_wait(sync, true);
> + if (r)
> + return r;
> + }
>
> - return amdgpu_sync_wait(sync, true);
> + /* Allocation restarts prepare before any further page-table writes. */
> + r = dma_resv_wait_timeout(resv, DMA_RESV_USAGE_KERNEL,
> + true, MAX_SCHEDULE_TIMEOUT);
> + return r < 0 ? r : 0;
That check is unnecessary, MAX_SCHEDULE_TIMEOUT blocks forever.
Regards,
Christian.
> }
>
> /**
> @@ -78,12 +87,6 @@ static int amdgpu_vm_cpu_update(struct amdgpu_vm_update_params *p,
> {
> unsigned int i;
> uint64_t value;
> - long r;
> -
> - r = dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
> - true, MAX_SCHEDULE_TIMEOUT);
> - if (r < 0)
> - return r;
>
> pe += (unsigned long)amdgpu_bo_kptr(&vmbo->bo);
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH v3 0/5] drm/amdgpu: prepare updates across page-table allocation
2026-11-11 13:14 ` Christian König
@ 2026-11-17 11:39 ` Mu, Zihao
2026-11-17 11:39 ` [RFC PATCH v3 1/5] drm/amdgpu: pass job id as part of amdgpu_vm_update_params Mu, Zihao
` (4 subsequent siblings)
5 siblings, 0 replies; 25+ messages in thread
From: Mu, Zihao @ 2026-11-17 11:39 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
Hello Christian, Alex and Cascardo,
This v3 waits uninterruptibly for explicit sync and root KERNEL move
fences in CPU prepare. Allocation commits the old phase and prepares
another phase before writes resume; CPU update no longer scans the
reservation for every PTE batch.
Changes since v2:
- Use intr=false for both CPU waits. Drop the result check for the
infinite reservation wait, following the MMU-notifier unmap contract.
- Reuse Cascardo's void-commit cleanup [1] and the existing PT teardown
fix [2] as prerequisites. Remove our duplicate teardown submission.
- Adapt Cascardo's context job-ID proposal [3], including SDMA prepare,
initial allocation and IB rollover, while preserving his authorship.
- Keep internal SDMA completion tracking before partial-range cleanup.
Fold nested-clear finalization into the allocation-phase change.
- Explain the same-entity finished-fence ordering and owned-reference
transfers in the completion-tracking commit description.
Why one internal completion is enough for a range:
A range prefix, its nested clear and its resumed phase all preserve
immediate and select the same VM entity. That entity keeps its engine
while work is unfinished, and SDMA submissions execute in queue order.
amdgpu_job_submit() returns a finished fence, so the latest such fence
covers those submissions. This relies on no cross-entity global order.
Each context owns its last_update reference. Submission replaces it;
missing/empty commits retain it. Nested clear transfers its reference
with a swap and puts the displaced reference. TLB finalization takes
its own reference before the context releases its private reference.
The optional public output and DONT_PIPELINE policy stay unchanged.
Dependency and application order:
The native base is identified below. The November 17 public refs still
lack the three prerequisites carried in the source backup:
A. Cascardo's missing/empty SDMA job handling (posted as v2 patch 1/6).
B. Cascardo's void-commit cleanup [1].
C. The existing publicly posted PT teardown fix [2].
These are native adaptations with original authors and sign-offs. I have
not identified the exact queued reviewed PT-fix commit, so I have not
inferred a Reviewed-by from equivalent public code. The prerequisites
are not resubmitted as new patches in this five-patch series.
The complete stack is backed up here (source backup, not a pull request):
https://github.com/zihaomu/amdgpu/tree/fix/vm-prepare-uninterruptible-v3-polished
To reproduce from a disposable checkout with the native base available:
git fetch https://github.com/zihaomu/amdgpu.git \
fix/vm-prepare-uninterruptible-v3-polished
vm_prereq_tip=adc41aaa1cb5e3103cad42ed9180d68fc384ffc9
git switch --detach "$vm_prereq_tip"
git am /path/to/v3/outgoing/v3-000[1-5]-*.patch
That checkout contains prerequisites A-C only. Do not apply them again if
they are already present; port onto the maintainer's queued versions when
the corresponding commits become available. The cover is not a patch.
The five patches are context job ID, internal completion tracking,
partial-range finalization, allocation-phase restart, and CPU prepare
waits, in that order. Original sync and job ID survive allocation.
Validation:
- GCC 13.3.0 compiled the four relevant VM objects at all eight commit
states, with CONFIG_DRM_AMDGPU=m. Applicable offline checks also ran
at each of the five outgoing source states.
- The lifecycle fixture executes extracted native walker, cursor, PTE
flags, free-list, backend, explicit-sync and TLB-worker bodies. On the
final state it checks 96 range cases, 54 allocation failures, four
free-list paths and two PDE paths. Unlocked walks do not allocate.
- Pending-signal checks include 16 CPU unmap and four CPU allocation
restart cases, plus corresponding SDMA cases in the scheduler model.
Separate CPU/reservation checks cover 960 normal and 960 signal
combinations, eight signal/unmap cases and two missing-wait controls.
- Nine broken controls are detected, including removed SDMA KERNEL
dependencies, allocation in unlocked walks, lost original sync,
lost clear completion and a completion from the wrong entity.
- Nine direct backend/scheduler unit cases are counted separately.
Two explicit contract controls confirm the unchanged native walker
still ignores a backend update error; they are not successful unmaps.
- Strict checkpatch passes for the five patches and cover. ASCII EML
extraction, default git am and per-commit full-tree application match
the exact sources and authors. ASan/UBSan reported no runtime errors.
The model is bounded to a two-level PT fixture. Hardware encoding,
BO/TTM, scheduler, GPU, list and fence primitives are simulated; this
is not live fault injection or a kernel concurrency proof. Operation
counts are not measured performance gains. SDMA allocation failures,
the complete no-fail unmap conversion and the original queue-stall
cause remain unresolved. The independent fast-path proposal is excluded.
Please review the revised wait contract and phase/fence ordering.
I can rebase onto the queued prerequisite commits when they are known.
[1]
https://lore.kernel.org/amd-gfx/20260924-amdgpu_vm_update_funcs_commit_void-v1-1-9ad7a8283920@igalia.com/
[2]
https://lore.kernel.org/amd-gfx/20261109090517.2905697-1-Jesse.Zhang@amd.com/
[3]
https://lore.kernel.org/amd-gfx/avRl4ajLaCSa8AeG@quatroqueijos.cascardo.eti.br/
Thanks,
Zihao
---
base-commit: a6836fcbb5b22968019256025321d655b78bc49c
prerequisite-patch-id: 8b58a9b82460d152ba8dbbd77cacb733c73221bd
prerequisite-patch-id: c6ccf9fd4659ee004f8af666adc985bfe4a05de4
prerequisite-patch-id: 6eec44dc54d47d900a5172c4aed842e66e8ce868
--
2.43.0
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH v3 1/5] drm/amdgpu: pass job id as part of amdgpu_vm_update_params
2026-11-11 13:14 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 0/5] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
@ 2026-11-17 11:39 ` Mu, Zihao
2026-11-17 14:56 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 2/5] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
` (3 subsequent siblings)
5 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-17 11:39 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Keep the original identifier in the update context so replacement jobs
and phases restarted around allocation use the same tracing identity.
Initialize the ID for range, PDE and table-clear contexts. Use it in
SDMA's initial and rollover allocations, and remove prepare's separate
job-ID argument from both backends and the callback type.
[zmu: Complete the SDMA prepare/alloc_job/rollover changes omitted from
the inline proposal, and adapt to the current context layout.]
Link: https://lore.kernel.org/amd-gfx/avRl4ajLaCSa8AeG@quatroqueijos.cascardo.eti.br/
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++----
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 4 +---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 7 ++++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 4 ++--
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 12 +++++-------
5 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 172af3b8f1a..7199cec1f95 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1055,8 +1055,8 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
params.vm = vm;
params.immediate = immediate;
- r = vm->update_funcs->prepare(¶ms, NULL,
- AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES);
+ params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES;
+ r = vm->update_funcs->prepare(¶ms, NULL);
if (r)
goto error;
@@ -1218,8 +1218,8 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
dma_fence_put(tmp);
}
- r = vm->update_funcs->prepare(¶ms, sync,
- AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE);
+ params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE;
+ r = vm->update_funcs->prepare(¶ms, sync);
if (r)
goto error_free;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
index 8c7afd34783..7521c5be207 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
@@ -43,14 +43,12 @@ static int amdgpu_vm_cpu_map_table(struct amdgpu_bo_vm *table)
*
* @p: see amdgpu_vm_update_params definition
* @sync: sync obj with fences to wait on
- * @k_job_id: the id for tracing/debug purposes
*
* Returns:
* Negativ errno, 0 for success.
*/
static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p,
- struct amdgpu_sync *sync,
- u64 k_job_id)
+ struct amdgpu_sync *sync)
{
if (!sync)
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
index 0b7cc3a1701..ea52be053d8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
@@ -77,6 +77,11 @@ struct amdgpu_vm_update_params {
*/
struct amdgpu_job *job;
+ /**
+ * @k_job_id: original job identifier for every submission in this context
+ */
+ u64 k_job_id;
+
/**
* @num_dw_left: number of dw left for the IB
*/
@@ -107,7 +112,7 @@ struct amdgpu_vm_update_params {
struct amdgpu_vm_update_funcs {
int (*map_table)(struct amdgpu_bo_vm *bo);
int (*prepare)(struct amdgpu_vm_update_params *p,
- struct amdgpu_sync *sync, u64 k_job_id);
+ struct amdgpu_sync *sync);
int (*update)(struct amdgpu_vm_update_params *p,
struct amdgpu_bo_vm *bo, uint64_t pe, uint64_t addr,
unsigned count, uint32_t incr, uint64_t flags);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index f205adfd59e..0ff2d06bf53 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -399,8 +399,8 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
params.vm = vm;
params.immediate = immediate;
- r = vm->update_funcs->prepare(¶ms, NULL,
- AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR);
+ params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR;
+ r = vm->update_funcs->prepare(¶ms, NULL);
if (r)
goto exit;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index e7180a6661a..38ca2d78b38 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -41,7 +41,7 @@ static int amdgpu_vm_sdma_map_table(struct amdgpu_bo_vm *table)
/* Allocate a new job for @count PTE updates */
static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
- unsigned int count, u64 k_job_id)
+ unsigned int count)
{
enum amdgpu_ib_pool_type pool = p->immediate ? AMDGPU_IB_POOL_IMMEDIATE
: AMDGPU_IB_POOL_DELAYED;
@@ -57,7 +57,7 @@ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
ndw = min(ndw, AMDGPU_VM_SDMA_MAX_NUM_DW);
r = amdgpu_job_alloc_with_ib(p->adev, entity, AMDGPU_FENCE_OWNER_VM,
- ndw * 4, pool, k_job_id, &p->job);
+ ndw * 4, pool, p->k_job_id, &p->job);
if (r)
return r;
@@ -70,17 +70,16 @@ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
*
* @p: see amdgpu_vm_update_params definition
* @sync: amdgpu_sync object with fences to wait for
- * @k_job_id: identifier of the job, for tracing purpose
*
* Returns:
* Negativ errno, 0 for success.
*/
static int amdgpu_vm_sdma_prepare(struct amdgpu_vm_update_params *p,
- struct amdgpu_sync *sync, u64 k_job_id)
+ struct amdgpu_sync *sync)
{
int r;
- r = amdgpu_vm_sdma_alloc_job(p, 0, k_job_id);
+ r = amdgpu_vm_sdma_alloc_job(p, 0);
if (r)
return r;
@@ -256,8 +255,7 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
if (ndw < 32) {
amdgpu_vm_sdma_commit(p, NULL);
- r = amdgpu_vm_sdma_alloc_job(p, count,
- AMDGPU_KERNEL_JOB_ID_VM_UPDATE);
+ r = amdgpu_vm_sdma_alloc_job(p, count);
if (r)
return r;
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v3 2/5] drm/amdgpu: track internal completion of SDMA VM updates
2026-11-11 13:14 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 0/5] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
2026-11-17 11:39 ` [RFC PATCH v3 1/5] drm/amdgpu: pass job id as part of amdgpu_vm_update_params Mu, Zihao
@ 2026-11-17 11:39 ` Mu, Zihao
2026-11-18 8:21 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 3/5] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
` (2 subsequent siblings)
5 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-17 11:39 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
Retain the latest SDMA finished fence independently of the optional
public output. A prefix can be submitted before PT allocation fails,
leaving no final job to replace the caller's older output fence. TLB
finalization must still wait for that submitted prefix.
Every phase of a range and its nested PT clear preserves immediate,
so sdma_alloc_job() selects the same VM scheduler entity throughout.
The entity keeps its engine while work is unfinished, and submissions
execute in order there. amdgpu_job_submit() returns the scheduler's
finished fence. Thus the last submitted finished fence covers the
prefix, clear and resumed phase; no cross-entity ordering is assumed.
The context owns last_update. On submission, put the old reference and
get the new finished fence. Missing/empty commits keep the previous
completion. A nested clear transfers its reference to the outer context
by swapping outputs, then puts the displaced reference at exit. Range,
PDE and standalone-clear owners put their private reference at teardown;
TLB finalization obtains its own reference before that put.
Keep the public !immediate output and DONT_PIPELINE policy unchanged.
Track completion before adding partial-range finalization, so the
latter can order its TLB work even after replacement allocation fails.
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 10 +++++++---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 8 +++++++-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 12 +++++++++---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 ++
4 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 7199cec1f95..3b4df22c5ed 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -658,7 +658,9 @@ int amdgpu_vm_validate(struct amdgpu_device *adev, struct amdgpu_vm *vm,
bo_base->moved = true;
if (vm->generation != new_vm_generation) {
- r = amdgpu_vm_pt_clear(adev, vm, to_amdgpu_bo_vm(bo_base->bo), false);
+ r = amdgpu_vm_pt_clear(adev, vm,
+ to_amdgpu_bo_vm(bo_base->bo),
+ false, NULL);
if (r)
return r;
}
@@ -1079,6 +1081,7 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
amdgpu_vm_bo_idle(entry);
error:
+ dma_fence_put(params.last_update);
drm_dev_exit(idx);
return r;
}
@@ -1282,7 +1285,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
vm->update_funcs->commit(¶ms, fence);
if (params.needs_flush) {
- amdgpu_vm_tlb_flush(¶ms, *fence, tlb_cb);
+ amdgpu_vm_tlb_flush(¶ms, params.last_update, tlb_cb);
tlb_cb = NULL;
}
@@ -1290,6 +1293,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
error_free:
kfree(tlb_cb);
+ dma_fence_put(params.last_update);
amdgpu_vm_end_critical(¶ms);
drm_dev_exit(idx);
return r;
@@ -2706,7 +2710,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm,
if (r)
goto error_free_root;
- r = amdgpu_vm_pt_clear(adev, vm, root, false);
+ r = amdgpu_vm_pt_clear(adev, vm, root, false, NULL);
if (r)
goto error_free_root;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
index ea52be053d8..9bd71f4d5b9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
@@ -82,6 +82,11 @@ struct amdgpu_vm_update_params {
*/
u64 k_job_id;
+ /**
+ * @last_update: owned reference to the last SDMA completion fence
+ */
+ struct dma_fence *last_update;
+
/**
* @num_dw_left: number of dw left for the IB
*/
@@ -121,7 +126,8 @@ struct amdgpu_vm_update_funcs {
};
int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
- struct amdgpu_bo_vm *vmbo, bool immediate);
+ struct amdgpu_bo_vm *vmbo, bool immediate,
+ struct dma_fence **last_update);
int amdgpu_vm_pt_create(struct amdgpu_device *adev, struct amdgpu_vm *vm,
int level, bool immediate, struct amdgpu_bo_vm **vmbo,
int32_t xcp_id);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index 0ff2d06bf53..7b7a576d95f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -353,6 +353,7 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev,
* @vm: VM to clear BO from
* @vmbo: BO to clear
* @immediate: use an immediate update
+ * @last_update: optional output for the internal clear completion fence
*
* Root PD needs to be reserved when calling this.
*
@@ -360,7 +361,8 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev,
* 0 on success, errno otherwise.
*/
int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
- struct amdgpu_bo_vm *vmbo, bool immediate)
+ struct amdgpu_bo_vm *vmbo, bool immediate,
+ struct dma_fence **last_update)
{
unsigned int level = adev->vm_manager.root_level;
struct ttm_operation_ctx ctx = { true, false };
@@ -390,11 +392,11 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
if (!drm_dev_enter(adev_to_drm(adev), &idx))
return -ENODEV;
+ memset(¶ms, 0, sizeof(params));
r = vm->update_funcs->map_table(vmbo);
if (r)
goto exit;
- memset(¶ms, 0, sizeof(params));
params.adev = adev;
params.vm = vm;
params.immediate = immediate;
@@ -429,6 +431,9 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
vm->update_funcs->commit(¶ms, &vm->last_update);
exit:
+ if (last_update && params.last_update)
+ swap(*last_update, params.last_update);
+ dma_fence_put(params.last_update);
drm_dev_exit(idx);
return r;
}
@@ -545,7 +550,8 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
*/
pt_bo->parent = amdgpu_bo_ref(cursor->parent->bo);
amdgpu_vm_bo_base_init(entry, p->vm, pt_bo);
- r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate);
+ r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate,
+ &p->last_update);
if (r) {
/* The entry points to the PT now, take it out again */
amdgpu_vm_pt_free(entry);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
index 38ca2d78b38..d2d9a16a7ef 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
@@ -130,6 +130,8 @@ static void amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
WARN_ON(ib->length_dw > p->num_dw_left);
f = amdgpu_job_submit(p->job);
p->job = NULL;
+ dma_fence_put(p->last_update);
+ p->last_update = dma_fence_get(f);
if (p->unlocked) {
struct dma_fence *tmp = dma_fence_get(f);
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v3 3/5] drm/amdgpu: finalize partial VM range updates on error
2026-11-11 13:14 ` Christian König
` (2 preceding siblings ...)
2026-11-17 11:39 ` [RFC PATCH v3 2/5] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
@ 2026-11-17 11:39 ` Mu, Zihao
2026-11-18 8:34 ` Christian König
2026-11-17 11:40 ` [RFC PATCH v3 4/5] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
2026-11-17 11:40 ` [RFC PATCH v3 5/5] drm/amdgpu: wait uninterruptibly for page-table moves in CPU prepare Mu, Zihao
5 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-17 11:39 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Stop walking on an update error, then commit existing work and complete
the TLB and temporary page-table-list cleanup. The void commit callback
does not overwrite the original update result.
The preceding internal completion tracking orders finalization after
the latest submitted IB, including immediate and rollover submissions.
[zmu: Rebase on commit-void and the internal completion prerequisite.]
Link: https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-4-d6d835591e0f@igalia.com/
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 3b4df22c5ed..3f3c5a82fc1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1276,7 +1276,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
tmp = start + num_entries;
r = amdgpu_vm_ptes_update(¶ms, start, tmp, addr, flags);
if (r)
- goto error_free;
+ break;
amdgpu_res_next(&cursor, num_entries * AMDGPU_GPU_PAGE_SIZE);
start = tmp;
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v3 4/5] drm/amdgpu: restart VM updates around page-table allocation
2026-11-11 13:14 ` Christian König
` (3 preceding siblings ...)
2026-11-17 11:39 ` [RFC PATCH v3 3/5] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
@ 2026-11-17 11:40 ` Mu, Zihao
2026-11-18 8:39 ` Christian König
2026-11-17 11:40 ` [RFC PATCH v3 5/5] drm/amdgpu: wait uninterruptibly for page-table moves in CPU prepare Mu, Zihao
5 siblings, 1 reply; 25+ messages in thread
From: Mu, Zihao @ 2026-11-17 11:40 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
Commit the current phase before dropping the eviction lock to allocate
a page table. After clearing the new table, prepare another phase with
the original sync and the context's original job ID.
The eviction lock prevents table movement during a prepared phase.
pt_alloc() drops that lock for allocation, so it must end the phase
first. pt_clear() validates the new table before its own prepare; the
outer prepare then covers move dependencies before resumed writes.
All these submissions preserve immediate and use the same entity.
Finish the nested clear's current job even if its update fails, keeping
the original error. An empty job is freed by commit; a submitted clear
completion is already transferred to the range context for finalization.
Keep an initialized table installed if the outer prepare fails. Failed
initialization uses the existing teardown prerequisite, rather than
introducing another copy of that fix here. Existing tables still return
without ending or restarting the current phase.
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 5 +++++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 8 ++++----
3 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 3f3c5a82fc1..df65e02bfa2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1204,6 +1204,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
params.vm = vm;
params.immediate = immediate;
params.pages_addr = pages_addr;
+ params.sync = sync;
params.unlocked = unlocked;
params.needs_flush = flush_tlb;
params.override_pte = allow_override && adev->gmc.override_pte;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
index 9bd71f4d5b9..cdf14f69376 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
@@ -87,6 +87,11 @@ struct amdgpu_vm_update_params {
*/
struct dma_fence *last_update;
+ /**
+ * @sync: original dependencies to preserve across prepare calls
+ */
+ struct amdgpu_sync *sync;
+
/**
* @num_dw_left: number of dw left for the IB
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
index 7b7a576d95f..1a2352c8c8c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
@@ -426,9 +426,6 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
r = vm->update_funcs->update(¶ms, vmbo, addr, 0, entries,
value, flags);
- if (r)
- goto exit;
-
vm->update_funcs->commit(¶ms, &vm->last_update);
exit:
if (last_update && params.last_update)
@@ -532,6 +529,9 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
if (entry->bo)
return 0;
+ /* Allocation can add move dependencies while the lock is dropped. */
+ p->vm->update_funcs->commit(p, NULL);
+
amdgpu_vm_end_critical(p);
r = amdgpu_vm_pt_create(p->adev, p->vm, cursor->level, p->immediate,
&pt, p->vm->root.bo->xcp_id);
@@ -558,7 +558,7 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
return r;
}
- return 0;
+ return p->vm->update_funcs->prepare(p, p->sync);
error_unpin:
if (p->vm->is_npa)
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH v3 5/5] drm/amdgpu: wait uninterruptibly for page-table moves in CPU prepare
2026-11-11 13:14 ` Christian König
` (4 preceding siblings ...)
2026-11-17 11:40 ` [RFC PATCH v3 4/5] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
@ 2026-11-17 11:40 ` Mu, Zihao
5 siblings, 0 replies; 25+ messages in thread
From: Mu, Zihao @ 2026-11-17 11:40 UTC (permalink / raw)
To: Koenig, Christian, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
From: Zihao Mu <zmu@amd.com>
Wait for the shared root KERNEL move dependencies in prepare, including
NULL-sync calls, and remove the per-batch wait from the CPU update.
Allocation now commits and prepares again before further writes.
Use uninterruptible waits for both explicit sync and KERNEL dependencies.
An MMU-notifier unmap must complete even if a signal is pending; moving
an interruptible wait into prepare does not meet that requirement.
The infinite, uninterruptible reservation wait needs no error branch.
Keep the CPU write/flags body and public callback types otherwise
unchanged. SDMA resource-allocation failures remain outside this CPU
signal-handling fix; no allocation retry or no-fail claim is added.
Link: https://lore.kernel.org/amd-gfx/bf90d631-c63e-4df0-92ad-213bd65ef0ba@amd.com/
Signed-off-by: Zihao Mu <zmu@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
index 7521c5be207..b6d774751cc 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
@@ -50,10 +50,19 @@ static int amdgpu_vm_cpu_map_table(struct amdgpu_bo_vm *table)
static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p,
struct amdgpu_sync *sync)
{
- if (!sync)
- return 0;
+ struct dma_resv *resv = p->vm->root.bo->tbo.base.resv;
+ int r;
- return amdgpu_sync_wait(sync, true);
+ if (sync) {
+ r = amdgpu_sync_wait(sync, false);
+ if (r)
+ return r;
+ }
+
+ /* MMU-notifier unmaps must finish, including when a signal is pending. */
+ dma_resv_wait_timeout(resv, DMA_RESV_USAGE_KERNEL,
+ false, MAX_SCHEDULE_TIMEOUT);
+ return 0;
}
/**
@@ -76,12 +85,6 @@ static int amdgpu_vm_cpu_update(struct amdgpu_vm_update_params *p,
{
unsigned int i;
uint64_t value;
- long r;
-
- r = dma_resv_wait_timeout(vmbo->bo.tbo.base.resv, DMA_RESV_USAGE_KERNEL,
- true, MAX_SCHEDULE_TIMEOUT);
- if (r < 0)
- return r;
pe += (unsigned long)amdgpu_bo_kptr(&vmbo->bo);
--
2.43.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v3 1/5] drm/amdgpu: pass job id as part of amdgpu_vm_update_params
2026-11-17 11:39 ` [RFC PATCH v3 1/5] drm/amdgpu: pass job id as part of amdgpu_vm_update_params Mu, Zihao
@ 2026-11-17 14:56 ` Christian König
0 siblings, 0 replies; 25+ messages in thread
From: Christian König @ 2026-11-17 14:56 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/17/26 12:39, Mu, Zihao wrote:
> From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
>
> Keep the original identifier in the update context so replacement jobs
> and phases restarted around allocation use the same tracing identity.
>
> Initialize the ID for range, PDE and table-clear contexts. Use it in
> SDMA's initial and rollover allocations, and remove prepare's separate
> job-ID argument from both backends and the callback type.
>
> [zmu: Complete the SDMA prepare/alloc_job/rollover changes omitted from
> the inline proposal, and adapt to the current context layout.]
> Link: https://lore.kernel.org/amd-gfx/avRl4ajLaCSa8AeG@quatroqueijos.cascardo.eti.br/
> Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
> Signed-off-by: Zihao Mu <zmu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++----
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c | 4 +---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 7 ++++++-
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 4 ++--
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 12 +++++-------
> 5 files changed, 18 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 172af3b8f1a..7199cec1f95 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1055,8 +1055,8 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
> params.vm = vm;
> params.immediate = immediate;
>
> - r = vm->update_funcs->prepare(¶ms, NULL,
> - AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES);
> + params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_UPDATE_PDES;
Seeting that up should probably be grouped together with setting the other params fields.
Same for other places, apart from that nit pick looks good to me.
Regards,
Christian.
> + r = vm->update_funcs->prepare(¶ms, NULL);
> if (r)
> goto error;
>
> @@ -1218,8 +1218,8 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> dma_fence_put(tmp);
> }
>
> - r = vm->update_funcs->prepare(¶ms, sync,
> - AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE);
> + params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_UPDATE_RANGE;
> + r = vm->update_funcs->prepare(¶ms, sync);
> if (r)
> goto error_free;
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> index 8c7afd34783..7521c5be207 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c
> @@ -43,14 +43,12 @@ static int amdgpu_vm_cpu_map_table(struct amdgpu_bo_vm *table)
> *
> * @p: see amdgpu_vm_update_params definition
> * @sync: sync obj with fences to wait on
> - * @k_job_id: the id for tracing/debug purposes
> *
> * Returns:
> * Negativ errno, 0 for success.
> */
> static int amdgpu_vm_cpu_prepare(struct amdgpu_vm_update_params *p,
> - struct amdgpu_sync *sync,
> - u64 k_job_id)
> + struct amdgpu_sync *sync)
> {
> if (!sync)
> return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> index 0b7cc3a1701..ea52be053d8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> @@ -77,6 +77,11 @@ struct amdgpu_vm_update_params {
> */
> struct amdgpu_job *job;
>
> + /**
> + * @k_job_id: original job identifier for every submission in this context
> + */
> + u64 k_job_id;
> +
> /**
> * @num_dw_left: number of dw left for the IB
> */
> @@ -107,7 +112,7 @@ struct amdgpu_vm_update_params {
> struct amdgpu_vm_update_funcs {
> int (*map_table)(struct amdgpu_bo_vm *bo);
> int (*prepare)(struct amdgpu_vm_update_params *p,
> - struct amdgpu_sync *sync, u64 k_job_id);
> + struct amdgpu_sync *sync);
> int (*update)(struct amdgpu_vm_update_params *p,
> struct amdgpu_bo_vm *bo, uint64_t pe, uint64_t addr,
> unsigned count, uint32_t incr, uint64_t flags);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> index f205adfd59e..0ff2d06bf53 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> @@ -399,8 +399,8 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> params.vm = vm;
> params.immediate = immediate;
>
> - r = vm->update_funcs->prepare(¶ms, NULL,
> - AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR);
> + params.k_job_id = AMDGPU_KERNEL_JOB_ID_VM_PT_CLEAR;
> + r = vm->update_funcs->prepare(¶ms, NULL);
> if (r)
> goto exit;
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
> index e7180a6661a..38ca2d78b38 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
> @@ -41,7 +41,7 @@ static int amdgpu_vm_sdma_map_table(struct amdgpu_bo_vm *table)
>
> /* Allocate a new job for @count PTE updates */
> static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
> - unsigned int count, u64 k_job_id)
> + unsigned int count)
> {
> enum amdgpu_ib_pool_type pool = p->immediate ? AMDGPU_IB_POOL_IMMEDIATE
> : AMDGPU_IB_POOL_DELAYED;
> @@ -57,7 +57,7 @@ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
> ndw = min(ndw, AMDGPU_VM_SDMA_MAX_NUM_DW);
>
> r = amdgpu_job_alloc_with_ib(p->adev, entity, AMDGPU_FENCE_OWNER_VM,
> - ndw * 4, pool, k_job_id, &p->job);
> + ndw * 4, pool, p->k_job_id, &p->job);
> if (r)
> return r;
>
> @@ -70,17 +70,16 @@ static int amdgpu_vm_sdma_alloc_job(struct amdgpu_vm_update_params *p,
> *
> * @p: see amdgpu_vm_update_params definition
> * @sync: amdgpu_sync object with fences to wait for
> - * @k_job_id: identifier of the job, for tracing purpose
> *
> * Returns:
> * Negativ errno, 0 for success.
> */
> static int amdgpu_vm_sdma_prepare(struct amdgpu_vm_update_params *p,
> - struct amdgpu_sync *sync, u64 k_job_id)
> + struct amdgpu_sync *sync)
> {
> int r;
>
> - r = amdgpu_vm_sdma_alloc_job(p, 0, k_job_id);
> + r = amdgpu_vm_sdma_alloc_job(p, 0);
> if (r)
> return r;
>
> @@ -256,8 +255,7 @@ static int amdgpu_vm_sdma_update(struct amdgpu_vm_update_params *p,
> if (ndw < 32) {
> amdgpu_vm_sdma_commit(p, NULL);
>
> - r = amdgpu_vm_sdma_alloc_job(p, count,
> - AMDGPU_KERNEL_JOB_ID_VM_UPDATE);
> + r = amdgpu_vm_sdma_alloc_job(p, count);
> if (r)
> return r;
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v3 2/5] drm/amdgpu: track internal completion of SDMA VM updates
2026-11-17 11:39 ` [RFC PATCH v3 2/5] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
@ 2026-11-18 8:21 ` Christian König
0 siblings, 0 replies; 25+ messages in thread
From: Christian König @ 2026-11-18 8:21 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/17/26 12:39, Mu, Zihao wrote:
> From: Zihao Mu <zmu@amd.com>
>
> Retain the latest SDMA finished fence independently of the optional
> public output. A prefix can be submitted before PT allocation fails,
> leaving no final job to replace the caller's older output fence. TLB
> finalization must still wait for that submitted prefix.
>
> Every phase of a range and its nested PT clear preserves immediate,
> so sdma_alloc_job() selects the same VM scheduler entity throughout.
> The entity keeps its engine while work is unfinished, and submissions
> execute in order there. amdgpu_job_submit() returns the scheduler's
> finished fence. Thus the last submitted finished fence covers the
> prefix, clear and resumed phase; no cross-entity ordering is assumed.
>
> The context owns last_update. On submission, put the old reference and
> get the new finished fence. Missing/empty commits keep the previous
> completion. A nested clear transfers its reference to the outer context
> by swapping outputs, then puts the displaced reference at exit. Range,
> PDE and standalone-clear owners put their private reference at teardown;
> TLB finalization obtains its own reference before that put.
>
> Keep the public !immediate output and DONT_PIPELINE policy unchanged.
> Track completion before adding partial-range finalization, so the
> latter can order its TLB work even after replacement allocation fails.
>
> Signed-off-by: Zihao Mu <zmu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 10 +++++++---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 8 +++++++-
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 12 +++++++++---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 ++
> 4 files changed, 25 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 7199cec1f95..3b4df22c5ed 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -658,7 +658,9 @@ int amdgpu_vm_validate(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> bo_base->moved = true;
>
> if (vm->generation != new_vm_generation) {
> - r = amdgpu_vm_pt_clear(adev, vm, to_amdgpu_bo_vm(bo_base->bo), false);
> + r = amdgpu_vm_pt_clear(adev, vm,
> + to_amdgpu_bo_vm(bo_base->bo),
> + false, NULL);
> if (r)
> return r;
> }
> @@ -1079,6 +1081,7 @@ int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
> amdgpu_vm_bo_idle(entry);
>
> error:
> + dma_fence_put(params.last_update);
> drm_dev_exit(idx);
> return r;
> }
> @@ -1282,7 +1285,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> vm->update_funcs->commit(¶ms, fence);
>
> if (params.needs_flush) {
> - amdgpu_vm_tlb_flush(¶ms, *fence, tlb_cb);
> + amdgpu_vm_tlb_flush(¶ms, params.last_update, tlb_cb);
> tlb_cb = NULL;
> }
>
> @@ -1290,6 +1293,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>
> error_free:
> kfree(tlb_cb);
> + dma_fence_put(params.last_update);
> amdgpu_vm_end_critical(¶ms);
> drm_dev_exit(idx);
> return r;
> @@ -2706,7 +2710,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> if (r)
> goto error_free_root;
>
> - r = amdgpu_vm_pt_clear(adev, vm, root, false);
> + r = amdgpu_vm_pt_clear(adev, vm, root, false, NULL);
> if (r)
> goto error_free_root;
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> index ea52be053d8..9bd71f4d5b9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> @@ -82,6 +82,11 @@ struct amdgpu_vm_update_params {
> */
> u64 k_job_id;
>
> + /**
> + * @last_update: owned reference to the last SDMA completion fence
> + */
> + struct dma_fence *last_update;
> +
> /**
> * @num_dw_left: number of dw left for the IB
> */
> @@ -121,7 +126,8 @@ struct amdgpu_vm_update_funcs {
> };
>
> int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> - struct amdgpu_bo_vm *vmbo, bool immediate);
> + struct amdgpu_bo_vm *vmbo, bool immediate,
> + struct dma_fence **last_update);
> int amdgpu_vm_pt_create(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> int level, bool immediate, struct amdgpu_bo_vm **vmbo,
> int32_t xcp_id);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> index 0ff2d06bf53..7b7a576d95f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> @@ -353,6 +353,7 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev,
> * @vm: VM to clear BO from
> * @vmbo: BO to clear
> * @immediate: use an immediate update
> + * @last_update: optional output for the internal clear completion fence
> *
> * Root PD needs to be reserved when calling this.
> *
> @@ -360,7 +361,8 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev,
> * 0 on success, errno otherwise.
> */
> int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> - struct amdgpu_bo_vm *vmbo, bool immediate)
> + struct amdgpu_bo_vm *vmbo, bool immediate,
> + struct dma_fence **last_update)
> {
> unsigned int level = adev->vm_manager.root_level;
> struct ttm_operation_ctx ctx = { true, false };
> @@ -390,11 +392,11 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> if (!drm_dev_enter(adev_to_drm(adev), &idx))
> return -ENODEV;
>
> + memset(¶ms, 0, sizeof(params));
> r = vm->update_funcs->map_table(vmbo);
> if (r)
> goto exit;
>
> - memset(¶ms, 0, sizeof(params));
> params.adev = adev;
> params.vm = vm;
> params.immediate = immediate;
> @@ -429,6 +431,9 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>
> vm->update_funcs->commit(¶ms, &vm->last_update);
> exit:
> + if (last_update && params.last_update)
> + swap(*last_update, params.last_update);
> + dma_fence_put(params.last_update);
> drm_dev_exit(idx);
> return r;
> }
> @@ -545,7 +550,8 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
> */
> pt_bo->parent = amdgpu_bo_ref(cursor->parent->bo);
> amdgpu_vm_bo_base_init(entry, p->vm, pt_bo);
> - r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate);
> + r = amdgpu_vm_pt_clear(p->adev, p->vm, pt, p->immediate,
> + &p->last_update);
The code you work on here is outdated, I'm about to remove immediate updates from amdgpu_vm_pt_clear() and only give the parameters to that function as well.
Apart from that the patch looks good to me.
Regards,
Christian.
> if (r) {
> /* The entry points to the PT now,
So what take it out again */
> amdgpu_vm_pt_free(entry);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
> index 38ca2d78b38..d2d9a16a7ef 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c
> @@ -130,6 +130,8 @@ static void amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
> WARN_ON(ib->length_dw > p->num_dw_left);
> f = amdgpu_job_submit(p->job);
> p->job = NULL;
> + dma_fence_put(p->last_update);
> + p->last_update = dma_fence_get(f);
>
> if (p->unlocked) {
> struct dma_fence *tmp = dma_fence_get(f);
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v3 3/5] drm/amdgpu: finalize partial VM range updates on error
2026-11-17 11:39 ` [RFC PATCH v3 3/5] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
@ 2026-11-18 8:34 ` Christian König
0 siblings, 0 replies; 25+ messages in thread
From: Christian König @ 2026-11-18 8:34 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/17/26 12:39, Mu, Zihao wrote:
> From: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
>
> Stop walking on an update error, then commit existing work and complete
> the TLB and temporary page-table-list cleanup. The void commit callback
> does not overwrite the original update result.
>
> The preceding internal completion tracking orders finalization after
> the latest submitted IB, including immediate and rollover submissions.
>
> [zmu: Rebase on commit-void and the internal completion prerequisite.]
> Link: https://lore.kernel.org/amd-gfx/20260914-amdgpu_vm_fixes-v1-4-d6d835591e0f@igalia.com/
> Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
> Signed-off-by: Zihao Mu <zmu@amd.com>
Yeah once more clear NAK to that!
First of all we can't flush the TLB nor free the page tables because we don't know if they are still used or not.
What we need to do first is to make sure we commit the SDMA work before allocating new page tables.
It would even be better to move the allocation of page tables outside of the loop which updates the PTEs. But we need a better data structure for the page addresses to do something like that.
Regards,
Christian.
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 3b4df22c5ed..3f3c5a82fc1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1276,7 +1276,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> tmp = start + num_entries;
> r = amdgpu_vm_ptes_update(¶ms, start, tmp, addr, flags);
> if (r)
> - goto error_free;
> + break;
>
> amdgpu_res_next(&cursor, num_entries * AMDGPU_GPU_PAGE_SIZE);
> start = tmp;
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH v3 4/5] drm/amdgpu: restart VM updates around page-table allocation
2026-11-17 11:40 ` [RFC PATCH v3 4/5] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
@ 2026-11-18 8:39 ` Christian König
0 siblings, 0 replies; 25+ messages in thread
From: Christian König @ 2026-11-18 8:39 UTC (permalink / raw)
To: Mu, Zihao, amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander, Thadeu Lima de Souza Cascardo
On 11/17/26 12:40, Mu, Zihao wrote:
> From: Zihao Mu <zmu@amd.com>
>
> Commit the current phase before dropping the eviction lock to allocate
> a page table. After clearing the new table, prepare another phase with
> the original sync and the context's original job ID.
>
> The eviction lock prevents table movement during a prepared phase.
> pt_alloc() drops that lock for allocation, so it must end the phase
> first. pt_clear() validates the new table before its own prepare; the
> outer prepare then covers move dependencies before resumed writes.
> All these submissions preserve immediate and use the same entity.
>
> Finish the nested clear's current job even if its update fails, keeping
> the original error. An empty job is freed by commit; a submitted clear
> completion is already transferred to the range context for finalization.
>
> Keep an initialized table installed if the outer prepare fails. Failed
> initialization uses the existing teardown prerequisite, rather than
> introducing another copy of that fix here. Existing tables still return
> without ending or restarting the current phase.
>
> Signed-off-by: Zihao Mu <zmu@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h | 5 +++++
> drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c | 8 ++++----
> 3 files changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 3f3c5a82fc1..df65e02bfa2 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -1204,6 +1204,7 @@ int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm,
> params.vm = vm;
> params.immediate = immediate;
> params.pages_addr = pages_addr;
> + params.sync = sync;
> params.unlocked = unlocked;
> params.needs_flush = flush_tlb;
> params.override_pte = allow_override && adev->gmc.override_pte;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> index 9bd71f4d5b9..cdf14f69376 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_internal.h
> @@ -87,6 +87,11 @@ struct amdgpu_vm_update_params {
> */
> struct dma_fence *last_update;
>
> + /**
> + * @sync: original dependencies to preserve across prepare calls
> + */
> + struct amdgpu_sync *sync;
> +
That is nonsense. Only the first prepare call needs the dependencies.
> /**
> * @num_dw_left: number of dw left for the IB
> */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> index 7b7a576d95f..1a2352c8c8c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.c
> @@ -426,9 +426,6 @@ int amdgpu_vm_pt_clear(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>
> r = vm->update_funcs->update(¶ms, vmbo, addr, 0, entries,
> value, flags);
> - if (r)
> - goto exit;
> -
> vm->update_funcs->commit(¶ms, &vm->last_update);
> exit:
> if (last_update && params.last_update)
> @@ -532,6 +529,9 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
> if (entry->bo)
> return 0;
>
> + /* Allocation can add move dependencies while the lock is dropped. */
No it doesn't. What can be is that we have more page tables clears but those execute sequenctially with the PTE updates.
> + p->vm->update_funcs->commit(p, NULL);
We need Thadeu's patch to ignore commits with empty IBs for this to work correctly.
Additional to that the returned fence parameter should be dropped from the commit function.
Regards,
Christian.
> +
> amdgpu_vm_end_critical(p);
> r = amdgpu_vm_pt_create(p->adev, p->vm, cursor->level, p->immediate,
> &pt, p->vm->root.bo->xcp_id);
> @@ -558,7 +558,7 @@ static int amdgpu_vm_pt_alloc(struct amdgpu_vm_update_params *p,
> return r;
> }
>
> - return 0;
> + return p->vm->update_funcs->prepare(p, p->sync);
>
> error_unpin:
> if (p->vm->is_npa)
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2026-11-18 8:39 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-10 10:29 [RFC] drm/amdgpu: handling PTE backend update errors Mu, Zihao
2026-11-10 12:26 ` Christian König
2026-11-11 5:39 ` [RFC PATCH] drm/amdgpu: make CPU page-table update waits uninterruptible Mu, Zihao
2026-11-11 9:06 ` Christian König
2026-11-11 11:55 ` [RFC PATCH v2 0/6] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 1/6] drm/amdgpu: handle missing or empty SDMA VM update jobs Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 2/6] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
2026-11-11 11:55 ` [RFC PATCH v2 3/6] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
2026-11-11 12:44 ` Thadeu Lima de Souza Cascardo
2026-11-11 11:55 ` [RFC PATCH v2 4/6] drm/amdgpu: unwind failed page-table initialization Mu, Zihao
2026-11-11 13:10 ` Christian König
2026-11-11 11:55 ` [RFC PATCH v2 5/6] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
2026-11-11 12:56 ` Thadeu Lima de Souza Cascardo
2026-11-11 11:56 ` [RFC PATCH v2 6/6] drm/amdgpu: wait for page-table moves in CPU prepare Mu, Zihao
2026-11-11 13:14 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 0/5] drm/amdgpu: prepare updates across page-table allocation Mu, Zihao
2026-11-17 11:39 ` [RFC PATCH v3 1/5] drm/amdgpu: pass job id as part of amdgpu_vm_update_params Mu, Zihao
2026-11-17 14:56 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 2/5] drm/amdgpu: track internal completion of SDMA VM updates Mu, Zihao
2026-11-18 8:21 ` Christian König
2026-11-17 11:39 ` [RFC PATCH v3 3/5] drm/amdgpu: finalize partial VM range updates on error Mu, Zihao
2026-11-18 8:34 ` Christian König
2026-11-17 11:40 ` [RFC PATCH v3 4/5] drm/amdgpu: restart VM updates around page-table allocation Mu, Zihao
2026-11-18 8:39 ` Christian König
2026-11-17 11:40 ` [RFC PATCH v3 5/5] drm/amdgpu: wait uninterruptibly for page-table moves in CPU prepare Mu, Zihao
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.