From: "Kuehling, Felix" <felix.kuehling@amd.com>
To: "Honglei Huang" <honghuan@amd.com>,
"Christian König" <christian.koenig@amd.com>,
"Huang Rui" <ray.huang@amd.com>
Cc: "Xiaogang Chen" <xiaogang.chen@amd.com>,
"Oak Zeng" <Oak.Zeng@amd.com>,
"Jenny Liu" <Jenny-Jing.Liu@amd.com>,
"Zhu Lingshan" <lingshan.zhu@amd.com>,
"Honglei Huang" <honglei1.huang@amd.com>,
"Junhua Shen" <Junhua.Shen@amd.com>, "Yiru Ma" <yiru.ma@amd.com>,
"Philip Yang" <Philip.Yang@amd.com>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Simona Vetter" <simona@ffwll.ch>,
"Matthew Brost" <matthew.brost@intel.com>,
"Rodrigo Vivi" <rodrigo.vivi@intel.com>,
"Thomas Hellström" <thomas.hellstrom@linux.intel.com>,
"Danilo Krummrich" <dakr@kernel.org>,
"Alice Ryhl" <aliceryhl@google.com>,
amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
"Timur Kristóf" <timur.kristof@gmail.com>
Subject: Re: [PATCH v10 02/22] drm/amdgpu: add SVM core header and VM integration
Date: Fri, 20 Nov 2026 18:31:11 -0500 [thread overview]
Message-ID: <61bec126-d855-4b14-a8bf-80cfdcea66fd@amd.com> (raw)
In-Reply-To: <1bf6fd84-9ea5-49fd-b906-7b2c9692c018@amd.com>
On 2026-11-16 00:18, Honglei Huang wrote:
> On 11/10/2026 10:46 PM, Kuehling, Felix wrote:
>
> ...
>
> >>> + * @checkpoint_ts: IH ring timestamp captured on unmap, used to drop
> >>> stale
> >>> + * faults.
> >>> + * @default_granularity: Default range granularity.
> >>> + * @xnack_enabled: Whether retry (XNACK) faults are enabled.
> >>> + * @invalidate_ranges: Callback of the MMU notifier invalidate
> event.
> >>> + */
> >>> +struct amdgpu_svm {
> >>> + struct drm_gpusvm gpusvm;
> >>> + struct kref refcount;
> >>> + struct amdgpu_device *adev;
> >>> + struct amdgpu_vm *vm;
> >>> + struct amdgpu_svm_attr_tree *attr_tree;
> >>> + struct rw_semaphore svm_lock;
> >>> + spinlock_t work_lock;
> >>> + struct amdgpu_svm_gc gc;
> >>> + atomic_t exiting;
> >>> + uint64_t checkpoint_ts;
> >> By now we have figured out that TS in faults are not useable for fault
> >> handling.
> >>
> >> Background is that they reset from time to time because of GFXOFF and
> >> may vary depending on the engine who sends the fault.
> >>
> >> Adding Timur from Valve on CC, he send out a patch to completely
> >> remove the existing timestamp handling from the fault filter.
> >>
> >> We need to come up with a different approach here.
> >
> > At least for the fault filter that shouldn't be a big problem. We don't
> > need the SW fault filter on any GPU that has a CAM. That should be any
> > GPU that supports retry faults starting with Vega20. AFAIK the only
> GPUs
> > that still need the SW fault filter are Vega10 and Raven. If the
> > timestamps work on those, we're fine. If there are any newer GPUs left
> > that don't enable the CAM for whatever reason, let's fix that instead.
> >
> > Regards,
> > Felix
>
> Hi Christian and Felix,
>
> I totally remoed the ts check in amdgpu drm svm fault, following is
> diff, is that looks valid for you?
>
> diff:
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
> index d95ac3651..15d8da7ea 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.c
> @@ -31,7 +31,6 @@
> #include "amdgpu.h"
> #include "amdgpu_vm.h"
> #include "amdgpu_gmc.h"
> -#include "amdgpu_ih.h"
> #include "amdgpu_trace.h"
>
> #include <drm/drm_exec.h>
> @@ -358,18 +357,16 @@ static int fault_map_range(struct amdgpu_svm *svm,
> * @adev: The amdgpu device.
> * @pasid: PASID of the faulting context.
> * @fault_page: Faulting page index.
> - * @ts: IH ring timestamp of the fault.
> * @write_fault: True if the fault was caused by a write.
> *
> * Entry point from the VM fault path. Look up the SVM context, drop the
> - * fault if the context is gone/exiting or the mm is dying, filter stale
> - * retry faults using the checkpoint timestamp, then resolve the
> covering
> + * fault if the context is gone/exiting or the mm is dying, then
> resolve the covering
> * attribute range (creating a default one if needed) and map it.
> *
> * Return: 0 on success or a dropped fault, negative error code on
> failure.
> */
> int amdgpu_svm_handle_fault(struct amdgpu_device *adev, uint32_t pasid,
> - uint64_t fault_page, uint64_t ts,
> + uint64_t fault_page,
> bool write_fault)
> {
> struct amdgpu_svm *svm;
> @@ -378,7 +375,6 @@ int amdgpu_svm_handle_fault(struct amdgpu_device
> *adev, uint32_t pasid,
> unsigned long attr_start_page, attr_last_page;
> uint64_t fault_addr = fault_page << PAGE_SHIFT;
> struct mm_struct *mm = NULL;
> - uint64_t ckpt;
> int ret;
>
> if (amdgpu_in_reset(adev))
> @@ -414,21 +410,6 @@ int amdgpu_svm_handle_fault(struct amdgpu_device
> *adev, uint32_t pasid,
> goto out_put;
> }
>
> - ckpt = READ_ONCE(svm->checkpoint_ts);
> - if (ckpt != 0) {
> - if (amdgpu_ih_ts_after_or_equal(ts, ckpt)) {
> - pr_debug(
> - "draining stale retry fault, drop fault 0x%llx
> ts=%llu checkpoint=%llu\n",
> - fault_addr, ts, ckpt);
> - amdgpu_gmc_filter_faults_remove(
> - adev, fault_page, pasid);
> - ret = 0;
> - goto out_put;
> - } else {
> - WRITE_ONCE(svm->checkpoint_ts, 0);
> - }
> - }
You can't just remove this. The reason this is here is to drain stale
page faults for ranges that were unmapped. If you remove this, those
stale faults will cause correct applications to crash.
You'll need to replace this with some other method. E.g. a sequence
number that doesn't wrap when he IH ring wraps around.
Regards,
Felix
> -
> amdgpu_svm_lock(svm);
>
> mutex_lock(&svm->attr_tree->lock);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.h
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.h
> index c169a9660..15c3915bd 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_fault.h
> @@ -33,7 +33,7 @@ struct amdgpu_svm_attr_range;
> struct amdgpu_svm_attrs;
>
> int amdgpu_svm_handle_fault(struct amdgpu_device *adev, uint32_t pasid,
> - uint64_t fault_page, uint64_t ts,
> + uint64_t fault_page,
> bool write_fault);
>
> #endif /* __AMDGPU_SVM_FAULT_H__ */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
> index 627b61c7e..f552725f9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_svm_range.c
> @@ -758,39 +758,6 @@ void amdgpu_svm_range_unqueue(struct amdgpu_svm
> *svm,
> drm_gpusvm_range_put(&range->base);
> }
>
> -/**
> - * amdgpu_svm_capture_checkpoint_ts() - Record the IH ring write
> pointer time
> - * @svm: The SVM context.
> - *
> - * Capture the timestamp of the most recent interrupt handler write
> pointer
> - * into @svm->checkpoint_ts. Retry faults whose timestamp predates this
> - * checkpoint are stale and are dropped by the fault handler.
> - */
> -void amdgpu_svm_capture_checkpoint_ts(struct amdgpu_svm *svm)
> -{
> - struct amdgpu_device *adev = svm->adev;
> - struct amdgpu_ih_ring *ih;
> - uint32_t checkpoint_wptr;
> -
> - if (!adev->irq.retry_cam_enabled && adev->irq.ih1.ring_size) {
> - ih = &adev->irq.ih1;
> - checkpoint_wptr = amdgpu_ih_get_wptr(adev, ih);
> - if (ih->rptr != checkpoint_wptr) {
> - WRITE_ONCE(svm->checkpoint_ts,
> - amdgpu_ih_decode_iv_ts(adev, ih,
> - checkpoint_wptr, -1));
> - return;
> - }
> - }
> -
> - ih = &adev->irq.ih_soft;
> - checkpoint_wptr = amdgpu_ih_get_wptr(adev, ih);
> - if (ih->rptr != checkpoint_wptr)
> - WRITE_ONCE(svm->checkpoint_ts,
> - amdgpu_ih_decode_iv_ts(adev, ih,
> - checkpoint_wptr, -1));
> -}
> -
> /**
> * amdgpu_svm_range_invalidate() - MMU notifier invalidate_ranges
> callback
> * @svm: The SVM context.
> @@ -802,8 +769,7 @@ void amdgpu_svm_capture_checkpoint_ts(struct
> amdgpu_svm *svm)
> *
> * Dispatch the two phase notifier handling across every range in
> * [@adj_start, @adj_end): run the begin phase for all ranges, then
> - * the end phase. On an unmap event, also capture a checkpoint timestamp
> - * so stale retry faults can be filtered.
> + * the end phase.
> */
> void amdgpu_svm_range_invalidate(struct amdgpu_svm *svm,
> struct drm_gpusvm_notifier *notifier,
> @@ -814,9 +780,6 @@ void amdgpu_svm_range_invalidate(struct amdgpu_svm
> *svm,
> struct drm_gpusvm_range *r;
> bool needs_flush = false;
>
> - if (mmu_range->event == MMU_NOTIFY_UNMAP)
> - amdgpu_svm_capture_checkpoint_ts(svm);
> -
> r = first;
> drm_gpusvm_for_each_range(r, notifier, adj_start, adj_end)
> needs_flush |= amdgpu_svm_range_notifier_event_begin(svm, r,
>
> Regards,
> Honglei
>
> >
> >
> >>
> >>> + u8 default_granularity;
> >>> + bool xnack_enabled;
> >> It's probably ok to have that here for now, but that will become a VM
> >> property in the future I think.
> >>
> >>> + void (*invalidate_ranges)(struct amdgpu_svm *svm,
> >>> + struct drm_gpusvm_notifier *notifier,
> >>> + const struct mmu_notifier_range *mmu_range,
> >>> + struct drm_gpusvm_range *first,
> >>> + uint64_t adj_start, uint64_t adj_end);
> >> That looks a bit questionable. Why is that a pointer inside the
> >> amdgpu_svm structure?
> >>
> >> Apart from those questions that looks rather reasonable.
> >>
> >> Regards,
> >> Christian.
> ...
next prev parent reply other threads:[~2026-11-20 23:31 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-21 2:58 [PATCH v10 00/22] drm/amdgpu: AMDGPU SVM support based on DRM (Phase 1: single GPU, XNACK on) Huang Rui
2026-10-21 2:58 ` [PATCH v10 01/22] drm/amdgpu: add SVM ioctl UAPI definitions Huang Rui
2026-11-09 12:26 ` Christian König
2026-10-21 2:58 ` [PATCH v10 02/22] drm/amdgpu: add SVM core header and VM integration Huang Rui
2026-11-09 12:33 ` Christian König
2026-11-10 2:51 ` Huang, Honglei
2026-11-10 14:46 ` Kuehling, Felix
2026-11-16 5:18 ` Honglei Huang
2026-11-20 23:31 ` Kuehling, Felix [this message]
2026-11-16 5:36 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 03/22] drm/amdgpu: implement SVM attribute tree and helper functions Huang Rui
2026-11-09 12:41 ` Christian König
2026-11-09 16:54 ` Aveline Noir
2026-11-10 2:55 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 04/22] drm/amdgpu: implement SVM attribute set/get/clear operations Huang Rui
2026-11-09 12:52 ` Christian König
2026-11-10 6:23 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 05/22] drm/amdgpu: add SVM range types and work queue interface Huang Rui
2026-11-09 13:48 ` Christian König
2026-11-10 6:44 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 06/22] drm/amdgpu/gmc: add get_svm_pte_flags callback Huang Rui
2026-11-09 13:52 ` Christian König
2026-11-10 6:57 ` Huang, Honglei
2026-11-10 10:01 ` Christian König
2026-11-10 10:24 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 07/22] drm/amdgpu: implement SVM range GPU mapping core Huang Rui
2026-11-09 13:55 ` Christian König
2026-11-10 7:01 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 08/22] drm/amdgpu: implement SVM range notifier and GC helpers Huang Rui
2026-11-09 17:12 ` Aveline Noir
2026-11-10 8:30 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 09/22] drm/amdgpu: add SVM notifier invalidate callback and checkpoint Huang Rui
2026-10-21 2:58 ` [PATCH v10 10/22] drm/amdgpu: implement SVM initialization and lifecycle Huang Rui
2026-11-09 16:59 ` Aveline Noir
2026-11-10 8:48 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 11/22] drm/amdgpu: add SVM ioctl entry and fault handler module Huang Rui
2026-11-09 16:47 ` Aveline Noir
2026-11-10 8:59 ` Huang, Honglei
2026-10-21 2:58 ` [PATCH v10 12/22] drm/amdgpu: integrate SVM into build system and VM fault path Huang Rui
2026-10-21 2:58 ` [PATCH v10 13/22] drm/amdgpu: make VM eviction lock read/write to match notifier lock Huang Rui
2026-10-21 2:58 ` [PATCH v10 14/22] drm/amdgpu: recheck range validity after page table allocation Huang Rui
2026-10-21 2:58 ` [PATCH v10 15/22] drm/amdgpu: use drm_gpusvm notifier lock for VM eviction Huang Rui
2026-10-21 2:58 ` [PATCH v10 16/22] drm/pagemap: fix page undercount for compound src entries Huang Rui
2026-10-21 2:58 ` [PATCH v10 17/22] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Huang Rui
2026-10-21 2:58 ` [PATCH v10 18/22] drm/amdgpu: implement drm_pagemap SDMA migration callbacks Huang Rui
2026-10-21 2:58 ` [PATCH v10 19/22] drm/amdgpu: implement synchronous TTM eviction for SVM BOs Huang Rui
2026-10-21 2:58 ` [PATCH v10 20/22] drm/amdgpu: hook up ZONE_DEVICE registration in device init and reset Huang Rui
2026-10-21 2:59 ` [PATCH v10 21/22] drm/amdgpu: add SVM range migration helpers for drm_pagemap Huang Rui
2026-10-21 2:59 ` [PATCH v10 22/22] drm/amdgpu: integrate VRAM migration into SVM fault and prefetch paths Huang Rui
2026-11-11 15:08 ` [PATCH v10 00/22] drm/amdgpu: AMDGPU SVM support based on DRM (Phase 1: single GPU, XNACK on) Huang, Honglei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=61bec126-d855-4b14-a8bf-80cfdcea66fd@amd.com \
--to=felix.kuehling@amd.com \
--cc=Jenny-Jing.Liu@amd.com \
--cc=Junhua.Shen@amd.com \
--cc=Oak.Zeng@amd.com \
--cc=Philip.Yang@amd.com \
--cc=alexander.deucher@amd.com \
--cc=aliceryhl@google.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
--cc=dakr@kernel.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=honghuan@amd.com \
--cc=honglei1.huang@amd.com \
--cc=lingshan.zhu@amd.com \
--cc=matthew.brost@intel.com \
--cc=ray.huang@amd.com \
--cc=rodrigo.vivi@intel.com \
--cc=simona@ffwll.ch \
--cc=thomas.hellstrom@linux.intel.com \
--cc=timur.kristof@gmail.com \
--cc=xiaogang.chen@amd.com \
--cc=yiru.ma@amd.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.