From: sashiko-bot@kernel.org
To: "Umesh Nerlige Ramappa" <umesh.nerlige.ramappa@intel.com>,
intel-xe@lists.freedesktop.org
Subject: Re: [PATCH v7 7/7] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
Date: Wed, 18 Nov 2026 03:51:02 +0000 [thread overview]
Message-ID: <sashiko-outbox-220052@kernel.org> (raw)
In-Reply-To: <20261118025140.3782678-16-umesh.nerlige.ramappa@intel.com>
Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [High] drm/xe/oa: out-of-bounds vmap read in
xe_oa_buffer_check_unlocked() on reset
- [Medium] drm/xe/oa: HW access and missing drain in xe_oa_stream_init()
error path
- [Medium] drm/xe/oa: drm_dev_is_unplugged() race in xe_oa_release()
skips ttm.wq flush
- [Medium] drm/xe/oa: OA buffer BO destroy can outlive flush_workqueue()
on unplug
- [Medium] drm/xe/oa: unnecessary fence wait in xe_oa_quiesce_oa_unit()
when wedged
--
commit 3beacae376d78af14b851e582701fe6ee70c6d4c
Author: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
This commit quiesces active OA streams on driver unbind and PCI reset by
marking streams dead, releasing device-bound resources, and rejecting
further operations on the stream fd until it is closed.
> diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
> index c7ccf273ced75..07ec4255cf089 100644
> --- a/drivers/gpu/drm/xe/xe_oa.c
> +++ b/drivers/gpu/drm/xe/xe_oa.c
> @@ -261,13 +261,18 @@ static bool mert_wa_14026633728(struct xe_oa_stream *s)
>
> static bool xe_oa_buffer_check_unlocked(struct xe_oa_stream *stream)
> {
> - u32 gtt_offset = xe_bo_ggtt_addr(stream->oa_buffer.bo);
> - u32 tail, hw_tail, partial_report_size, available;
> + u32 gtt_offset, tail, hw_tail, partial_report_size, available;
> int report_size = stream->oa_buffer.format->size;
> unsigned long flags;
>
> spin_lock_irqsave(&stream->oa_buffer.ptr_lock, flags);
>
> + if (stream->dead) {
> + spin_unlock_irqrestore(&stream->oa_buffer.ptr_lock, flags);
> + return false;
> + }
> +
> + gtt_offset = xe_bo_ggtt_addr(stream->oa_buffer.bo);
> hw_tail = xe_oa_hw_tail_read(stream);
> hw_tail -= gtt_offset;
[Severity: High]
Can xe_oa_buffer_check_unlocked() read stream->oa_buffer.bo->vmap out of
bounds when the device is wedged or resetting before xe_oa_fini() runs?
stream->dead is only set in xe_oa_quiesce_oa_unit() during xe_oa_fini().
During a PCI slot reset (between prepare_device_for_reset() and
devres_release_group() in xe_pci_error_slot_reset()), surprise hot-unplug,
or runtime wedge, stream->dead is still false while the 200 Hz
poll_check_timer and xe_oa_read() continue calling
xe_oa_buffer_check_unlocked().
When xe_oa_hw_tail_read() returns 0xffffffff on a disabled PCI device or 0
after a secondary bus reset, subtracting gtt_offset wraps the u32 hw_tail
to a value >= stream->oa_buffer.circ_size. Because xe_oa_circ_diff() does
not clamp or wrap values >= circ_size, the backward walk passes that tail
offset into oa_report_id(stream, tail) and indexes stream->oa_buffer.bo's
vmap past the end of the buffer.
[ ... ]
> @@ -1129,12 +1152,28 @@ static int xe_oa_emit_oa_config(struct xe_oa_stream *stream, struct xe_oa_config
> if (err == -ENOENT)
> xe_oa_config_cb(fence, &ofence->cb);
>
> - /* If nothing needs to be signaled we wait synchronously */
> + /*
> + * If nothing needs to be signaled we wait synchronously unless the device
> + * is unplugged/wedged.
> + */
> if (!num_signal) {
> - dma_fence_wait(&ofence->base, false);
> + while (!dma_fence_wait_timeout(&ofence->base, false, HZ / 10)) {
> + if (drm_dev_is_unplugged(>_to_xe(stream->gt)->drm)) {
> + ret = -ENODEV;
> + break;
> + }
> + if (xe_wedged(gt_to_xe(stream->gt))) {
> + ret = -ECANCELED;
> + break;
> + }
> + }
> dma_fence_put(&ofence->base);
> }
>
> + /* The callers clean up the syncs on error */
> + if (ret)
> + return ret;
[Severity: Medium]
When xe_oa_emit_oa_config() returns early here during xe_oa_stream_init()
-> xe_oa_enable_metric_set(), does the error unwind path in
xe_oa_stream_init() mishandle the in-flight batch and wedged state?
drivers/gpu/drm/xe/xe_oa.c:xe_oa_stream_init() {
...
err_put_k_exec_q:
xe_oa_disable_metric_set(stream);
xe_exec_queue_put(stream->k_exec_q);
err_free_oa_buf:
xe_oa_free_oa_buffer(stream);
err_fw_put:
xe_force_wake_put(gt_to_fw(gt), stream->fw_ref);
...
}
Because stream->oa_unit->exclusive_stream has not been set yet,
stream->dead is false. If the wait aborted because the device wedged,
err_put_k_exec_q still performs MMIO in xe_oa_disable_metric_set() and
xe_force_wake_put(), and when stream->exec_q is set it submits a context
restore batch and waits in xe_oa_stream_drain() on the wedged device.
If the wait aborted because the device was unplugged, when stream->exec_q
is set xe_oa_disable_metric_set() sees !stream->dead and blocks in
xe_oa_stream_drain() inside the drm_dev_enter() section of xe_oa_ioctl();
when stream->exec_q is NULL, nothing drains stream->last_fence before
xe_oa_disable_metric_set(), xe_force_wake_put(), and xe_pm_runtime_put()
run.
[ ... ]
> @@ -1753,17 +1811,37 @@ static void xe_oa_destroy_locked(struct xe_oa_stream *stream)
> kfree(stream);
> }
>
> +static void xe_oa_quiesce_oa_unit(struct xe_oa_unit *u, bool hw_accessible);
> +
> static int xe_oa_release(struct inode *inode, struct file *file)
> {
> struct xe_oa_stream *stream = file->private_data;
> struct xe_gt *gt = stream->gt;
> + struct xe_device *xe = gt_to_xe(gt);
> + int idx;
>
> mutex_lock(>->oa.gt_lock);
> - xe_oa_destroy_locked(stream);
> + if (drm_dev_enter(&xe->drm, &idx)) {
> + xe_oa_destroy_locked(stream);
> + drm_dev_exit(idx);
> + } else {
> + if (!stream->dead)
> + xe_oa_quiesce_oa_unit(stream->oa_unit, !xe_wedged(xe));
> +
> + xe_oa_destroy_locked(stream);
> + }
> mutex_unlock(>->oa.gt_lock);
>
> + /*
> + * Freeing the OA buffer may defer the BO destroy to the TTM workqueue. Once the device
> + * is unplugged, this may be the last reference on the drm device, so let the destroy
> + * finish before drm_dev_put() releases the GGTT.
> + */
> + if (drm_dev_is_unplugged(&xe->drm))
> + flush_workqueue(xe->ttm.wq);
> +
> /* Release the reference the OA stream kept on the driver */
> - drm_dev_put(>_to_xe(gt)->drm);
> + drm_dev_put(&xe->drm);
>
> return 0;
> }
[Severity: Medium]
Can xe_oa_release() race with drm_dev_unplug() after drm_dev_exit() and
skip flush_workqueue(xe->ttm.wq) before dropping the last drm_device
reference?
CPU0 CPU1
xe_oa_release()
drm_dev_enter(&xe->drm, &idx)
xe_oa_destroy_locked(stream)
...
queue_work(xe->ttm.wq, ...)
drm_dev_exit(idx)
if (drm_dev_is_unplugged(&xe->drm))
--> false
xe_pci_remove()
xe_device_remove()
drm_dev_unplug(&xe->drm)
...
devm_drm_dev_init_release()
drm_dev_put(&xe->drm)
drm_dev_put(&xe->drm)
drm_dev_release()
ggtt_fini_early() <-- runs before ttm.wq destroys BO
Because ggtt_fini_early() is registered via drmm after xe_device_destroy()
(which calls ttm_device_fini(&xe->ttm)), dropping the last drm_device
reference without flushing xe->ttm.wq tears down ggtt->mm while
ttm_bo_delayed_delete() -> xe_ggtt_remove_bo() is still queued.
[ ... ]
> @@ -2633,6 +2734,97 @@ static void xe_oa_unregister(void *arg)
> oa->metrics_kobj = NULL;
> }
>
> +static void xe_oa_quiesce_oa_unit(struct xe_oa_unit *u, bool hw_accessible)
> +{
> + struct xe_oa_stream *stream = u->exclusive_stream;
> + unsigned long flags;
> +
> + if (!stream)
> + return;
> +
> + mutex_lock(&stream->stream_lock);
> +
> + spin_lock_irqsave(&stream->oa_buffer.ptr_lock, flags);
> + WRITE_ONCE(stream->dead, true);
> + spin_unlock_irqrestore(&stream->oa_buffer.ptr_lock, flags);
> +
> + if (stream->enabled && hw_accessible)
> + xe_oa_disable_locked(stream);
> +
> + stream->enabled = false;
> +
> + if (stream->sample)
> + hrtimer_cancel(&stream->poll_check_timer);
> +
> + wake_up_all(&stream->poll_wq);
> +
> + xe_oa_stream_drain(stream);
> +
> + /*
> + * Restore the HW state while the stream still holds its runtime PM ref and exec_q.
> + * The device is not reset on unbind, so this can't be left to a later reset.
> + */
> + if (hw_accessible)
> + xe_oa_stream_hw_teardown(stream);
[Severity: Medium]
Should xe_oa_stream_drain(stream) be skipped when !hw_accessible?
When the device is wedged, xe_oa_stream_hw_teardown() is skipped and
free_oa_config_bo() hands stream->last_fence to xe_bb_free(), which does
not require the fence to be signaled first. However, stream->last_fence
can remain unsignaled on a wedged device (for example when
xe_oa_emit_oa_config() broke out of its wait with -ECANCELED, or when an
OA batch is waiting on an unsignaled input dependency), causing
xe_oa_stream_drain() to stall uninterruptibly for job_timeout_ms + HZ and
log a timeout error.
> +
> + /*
> + * Keep the OA buffer pinned until xe_oa_release(). An mmap holds a reference on the
> + * BO, so unpinning it here leaves it on the TTM LRU, where the shrinker may take the
> + * last reference and free the BO after the GGTT and the shrinker itself are gone.
> + * drm_dev_unplug() has zapped the userspace mappings and refaults get a dummy page.
> + * Drop the DMA mapping now, while pinned: the fd may be released, and the BO
> + * unpinned, while xe_bo_pinned_fini() DMA-unmaps it without the BO lock.
> + */
> + xe_bo_dma_unmap_pinned(stream->oa_buffer.bo);
[Severity: Medium]
Can the final reference on stream->oa_buffer.bo be dropped after
xe_oa_release() flushes xe->ttm.wq?
xe_bo_dma_unmap_pinned() leaves bo->pinned_link on
xe->pinned.early.kernel_bo_present, where xe_bo_pinned_fini() ->
xe_bo_apply_to_pinned() still visits it and takes a temporary reference on
the BO outside xe->pinned.lock:
CPU0 (devres_release_all) CPU1 (close OA fd)
xe_bo_pinned_fini()
xe_bo_apply_to_pinned()
xe_bo_get(bo)
spin_unlock(&xe->pinned.lock)
xe_oa_release()
xe_oa_destroy_locked()
xe_bo_unpin_map_no_vm(bo)
xe_bo_unpin(bo)
xe_bo_put(bo) <-- ref > 0
flush_workqueue(xe->ttm.wq)
xe_bo_dma_unmap_pinned(bo)
xe_bo_put(bo)
ttm_bo_release()
queue_work(xe->ttm.wq, ...)
...
devm_drm_dev_init_release()
drm_dev_put(&xe->drm) drm_dev_put(&xe->drm)
ggtt_fini_early() <-- runs before ttm.wq destroys BO
Because CPU0 holds a reference on bo while CPU1 runs xe_bo_put(bo) and
flush_workqueue(xe->ttm.wq), CPU0's xe_bo_put(bo) queues delayed BO
destruction onto xe->ttm.wq after the flush, allowing ggtt_fini_early()
and xe_shrinker_fini() to tear down the GGTT and shrinker before the BO is
destroyed.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118025140.3782678-9-umesh.nerlige.ramappa@intel.com?part=7
next prev parent reply other threads:[~2026-11-18 3:51 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 2:51 [PATCH v7 0/7] Cleanup OA stream on PCI unbind/remove Umesh Nerlige Ramappa
2026-11-18 2:51 ` [PATCH v7 1/7] drm/xe/oa: Use the drm device address_space for the OA stream fd Umesh Nerlige Ramappa
2026-11-18 3:14 ` sashiko-bot
2026-11-18 2:51 ` [PATCH v7 2/7] drm/xe/oa: Split OA stream destroy into HW teardown and resource release Umesh Nerlige Ramappa
2026-11-18 2:51 ` [PATCH v7 3/7] drm/xe/oa: Take the runtime PM reference in xe_oa_destroy_locked() Umesh Nerlige Ramappa
2026-11-18 2:51 ` [PATCH v7 4/7] drm/xe/oa: Skip OA HW teardown on a wedged device Umesh Nerlige Ramappa
2026-11-18 3:25 ` sashiko-bot
2026-11-18 2:51 ` [PATCH v7 5/7] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown Umesh Nerlige Ramappa
2026-11-18 2:51 ` [PATCH v7 6/7] drm/xe: Reject vm_ops->access on BO mappings of an unplugged device Umesh Nerlige Ramappa
2026-11-18 3:05 ` sashiko-bot
2026-11-18 2:51 ` [PATCH v7 7/7] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset Umesh Nerlige Ramappa
2026-11-18 3:51 ` sashiko-bot [this message]
2026-11-18 3:01 ` ✓ CI.KUnit: success for Cleanup OA stream on PCI unbind/remove (rev7) Patchwork
2026-11-18 3:37 ` ✓ Xe.CI.BAT: " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-220052@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=intel-xe@lists.freedesktop.org \
--cc=sashiko-reviews@lists.linux.dev \
--cc=umesh.nerlige.ramappa@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox