From: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
To: intel-xe@lists.freedesktop.org, shekhar.chauhan@intel.com,
harish.chegondi@intel.com
Cc: matthew.brost@intel.com
Subject: [PATCH v9 4/7] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown
Date: Fri, 20 Nov 2026 18:01:50 -0800 [thread overview]
Message-ID: <20261121020145.147384-13-umesh.nerlige.ramappa@intel.com> (raw)
In-Reply-To: <20261121020145.147384-9-umesh.nerlige.ramappa@intel.com>
Config batches submitted with output syncs and context LRI batches are
not waited for. If one is still in flight when the stream is closed, it
can program the OA registers after they were restored.
Both go to the same exec queue, so also record the LRI batch fence in
stream->last_fence, and wait for it before the HW teardown, bounded by
the job timeout of the exec queue. Also wait for the LRI batch that
disables the context save/restore of the OA counters during the HW
teardown, which covers the stream init error path as well.
A config batch with an input sync that doesn't signal sits in the
scheduler, where no TDR times it out, and still runs once the sync
signals, possibly reprogramming the OA config of a later stream. Don't
kill the exec queue when the wait times out to prevent that: the TDR of
a killed kernel exec queue resets the GT and can wedge the device.
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Reviewed-by: Shekhar Chauhan <shekhar.chauhan@intel.com>
---
v2: Kill the exec queue if the wait times out (Sashiko)
v3: Drop the exec queue kill, it can reset the GT and wedge the device
when the batch is on the kernel exec queue. Wait for the context LRI
batch of the HW teardown too (Sashiko)
---
drivers/gpu/drm/xe/xe_oa.c | 46 +++++++++++++++++++++++++++++++-------
1 file changed, 38 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
index 7677f3f2c65a..7241cf18b325 100644
--- a/drivers/gpu/drm/xe/xe_oa.c
+++ b/drivers/gpu/drm/xe/xe_oa.c
@@ -741,6 +741,12 @@ static void xe_oa_free_configs(struct xe_oa_stream *stream)
dma_fence_put(stream->last_fence);
}
+static void xe_oa_update_last_fence(struct xe_oa_stream *stream, struct dma_fence *fence)
+{
+ dma_fence_put(stream->last_fence);
+ stream->last_fence = dma_fence_get(fence);
+}
+
static int xe_oa_load_with_lri(struct xe_oa_stream *stream, struct xe_oa_reg *reg_lri, u32 count)
{
struct dma_fence *fence;
@@ -761,6 +767,7 @@ static int xe_oa_load_with_lri(struct xe_oa_stream *stream, struct xe_oa_reg *re
goto free_bb;
}
xe_bb_free(bb, fence);
+ xe_oa_update_last_fence(stream, fence);
dma_fence_put(fence);
return 0;
@@ -848,6 +855,28 @@ static u32 oag_configure_mmio_trigger(const struct xe_oa_stream *stream, bool en
return REG_MASKED_FIELD_ENABLE(OAG_OA_DEBUG_DISABLE_MMIO_TRG);
}
+/*
+ * Wait for the last OA batch of the stream, so that it can't program the HW after the HW
+ * teardown.
+ */
+static void xe_oa_stream_drain(struct xe_oa_stream *stream)
+{
+ struct xe_exec_queue *q = stream->exec_q ?: stream->k_exec_q;
+ long timeout;
+
+ if (!stream->last_fence || dma_fence_is_signaled(stream->last_fence))
+ return;
+
+ /*
+ * A batch waiting for an input sync that doesn't signal isn't covered by a TDR and still
+ * runs once the sync signals, possibly reprogramming the OA config. Don't kill the exec
+ * queue to prevent that, the TDR of a killed kernel exec queue resets the GT.
+ */
+ timeout = msecs_to_jiffies(q->sched_props.job_timeout_ms) + HZ;
+ if (!dma_fence_wait_timeout(stream->last_fence, false, timeout))
+ drm_err(&stream->oa->xe->drm, "Timed out waiting for OA batch\n");
+}
+
static void xe_oa_disable_metric_set(struct xe_oa_stream *stream)
{
struct xe_mmio *mmio = &stream->gt->mmio;
@@ -864,9 +893,14 @@ static void xe_oa_disable_metric_set(struct xe_oa_stream *stream)
xe_mmio_write32(mmio, __oa_regs(stream)->oa_debug,
oag_configure_mmio_trigger(stream, false));
- /* disable the context save/restore or OAR counters */
- if (stream->exec_q)
+ /*
+ * disable the context save/restore or OAR counters, and wait for that batch like for
+ * the ones before it
+ */
+ if (stream->exec_q) {
xe_oa_configure_oa_context(stream, false);
+ xe_oa_stream_drain(stream);
+ }
/* Make sure we disable noa to save power. */
if (GT_VER(stream->gt) < 35)
@@ -896,6 +930,8 @@ static void xe_oa_stream_destroy(struct xe_oa_stream *stream)
mutex_destroy(&stream->stream_lock);
+ xe_oa_stream_drain(stream);
+
if (stream->sample)
xe_reg_dewhitelist_oa_regs(stream->gt);
@@ -1002,12 +1038,6 @@ xe_oa_alloc_config_buffer(struct xe_oa_stream *stream, struct xe_oa_config *oa_c
return oa_bo;
}
-static void xe_oa_update_last_fence(struct xe_oa_stream *stream, struct dma_fence *fence)
-{
- dma_fence_put(stream->last_fence);
- stream->last_fence = dma_fence_get(fence);
-}
-
static void xe_oa_fence_work_fn(struct work_struct *w)
{
struct xe_oa_fence *ofence = container_of(w, typeof(*ofence), work.work);
--
2.55.0
next prev parent reply other threads:[~2026-11-21 2:02 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-21 2:01 [PATCH v9 0/7] Cleanup OA stream on PCI unbind/remove Umesh Nerlige Ramappa
2026-11-21 2:01 ` [PATCH v9 1/7] drm/xe/oa: Ignore a HW tail pointer outside the OA buffer Umesh Nerlige Ramappa
2026-11-21 2:01 ` [PATCH v9 2/7] drm/xe/oa: Use the drm device address_space for the OA stream fd Umesh Nerlige Ramappa
2026-11-21 2:01 ` [PATCH v9 3/7] drm/xe: Reject vm_ops->access on BO mappings of an unplugged device Umesh Nerlige Ramappa
2026-11-21 2:01 ` Umesh Nerlige Ramappa [this message]
2026-11-21 2:11 ` [PATCH v9 4/7] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown sashiko-bot
2026-11-21 2:01 ` [PATCH v9 5/7] drm/xe/oa: Fence OA ioctls and mmap against device unplug and wedging Umesh Nerlige Ramappa
2026-11-21 2:01 ` [PATCH v9 6/7] drm/xe/oa: Free the OA buffer and the stream after the stream destroy Umesh Nerlige Ramappa
2026-11-21 2:01 ` [PATCH v9 7/7] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset Umesh Nerlige Ramappa
2026-11-21 2:37 ` sashiko-bot
2026-11-21 2:09 ` ✗ CI.checkpatch: warning for Cleanup OA stream on PCI unbind/remove (rev9) Patchwork
2026-11-21 2:11 ` ✓ CI.KUnit: success " Patchwork
2026-11-21 2:48 ` ✗ Xe.CI.BAT: failure " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121020145.147384-13-umesh.nerlige.ramappa@intel.com \
--to=umesh.nerlige.ramappa@intel.com \
--cc=harish.chegondi@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=matthew.brost@intel.com \
--cc=shekhar.chauhan@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox