From: Shekhar Chauhan <shekhar.chauhan@intel.com>
To: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>,
<intel-xe@lists.freedesktop.org>
Subject: Re: [PATCH v8 8/8] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
Date: Fri, 20 Nov 2026 13:01:04 +0530 [thread overview]
Message-ID: <93e5b6be-6087-43a2-b8c8-8607cbcc7d71@intel.com> (raw)
In-Reply-To: <sashiko-outbox-222279@kernel.org>
On 11/19/2026 6:41, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 5 potential
> issues to consider:
> - [Medium] drm/xe/oa: drm_dev_is_unplugged() race in xe_oa_release()
> skips ttm.wq flush
> - [Medium] drm/xe/oa: OA buffer BO references can outlive
> flush_workqueue() on unplug
> - [Medium] drm/xe/oa: missing or blocking drain in xe_oa_stream_init()
> error path
> - [Medium] drm/xe/oa: NULL sg_table deref on resume after
> xe_bo_dma_unmap_pinned()
> - [Medium] drm/xe/oa: unconditional drain and batch submit on wedged
> device in destroy
> --
>
> commit 2b379416db1422398b8cce664304d4acea228c64
> Author: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>
> drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
>
> This commit quiesces active OA streams on driver unbind or PCI reset by
> marking them dead and tearing down their hardware and device resources
> while keeping the stream structure and pinned OA buffer allocated until
> the stream file descriptor is closed.
I'm unsure on the case reported by Sashiko. Please have a look.
-shekhar
>
>> diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
>> index 223419347eb7b..0b51bc818ef47 100644
>> --- a/drivers/gpu/drm/xe/xe_oa.c
>> +++ b/drivers/gpu/drm/xe/xe_oa.c
> [ ... ]
>
>> @@ -937,6 +959,13 @@ static void xe_oa_stream_destroy(struct xe_oa_stream *stream)
>> xe_oa_disable_metric_set(stream);
>> xe_exec_queue_put(stream->k_exec_q);
>>
>> + /*
>> + * The OA buffer is freed on release of the stream fd, an mmap may still hold a
>> + * reference on it. Drop its DMA mapping now, while pinned: the fd may be released, and
>> + * the BO unpinned, while xe_bo_pinned_fini() DMA-unmaps it without the BO lock.
>> + */
>> + xe_bo_dma_unmap_pinned(stream->oa_buffer.bo);
>> +
>> xe_force_wake_put(gt_to_fw(gt), stream->fw_ref);
>> xe_pm_runtime_put(stream->oa->xe);
> [Severity: Medium]
> Can calling xe_bo_dma_unmap_pinned() here while stream->oa_buffer.bo stays
> pinned on xe->pinned.early.kernel_bo_present cause a NULL pointer
> dereference if the device suspends and resumes before
> xe_oa_free_oa_buffer() unpins the BO?
>
> CPU0 CPU1
> xe_oa_release()
> xe_oa_destroy_locked()
> xe_oa_stream_destroy()
> xe_bo_dma_unmap_pinned(bo)
> xe_tt->sg = NULL
> xe_pm_runtime_put()
> xe_pm_runtime_put()
> xe_pm_suspend()
> xe_bo_evict_all()
> xe_pm_resume()
> xe_bo_restore_early()
> xe_bo_restore_and_map_ggtt()
> xe_ggtt_map_bo()
> xe_res_first_sg(NULL, ...)
> <-- NULL pointer dereference
> xe_oa_free_oa_buffer()
> xe_bo_unpin_map_no_vm(bo)
>
> Non-MERT OA buffers are created with XE_BO_FLAG_SYSTEM | XE_BO_FLAG_GGTT
> and remain linked on xe->pinned.early.kernel_bo_present until
> xe_bo_unpin() removes bo->pinned_link. Once xe_oa_destroy_locked() drops
> the last runtime PM reference, a system or D3cold runtime suspend/resume
> cycle will evict and restore the pinned BO via xe_ggtt_map_bo(), passing
> xe_bo_sg(bo) == NULL to xe_res_first_sg().
>
> [ ... ]
>
>> @@ -1754,19 +1786,35 @@ static long xe_oa_ioctl(struct file *file,
>> return ret;
>> }
>>
>> +/*
>> + * Disable the stream, restore the HW state and release everything but the OA buffer and the
>> + * stream struct, the same whether the stream fd is closed or the device goes away, and
>> + * whether or not the HW still responds: all HW waits are bounded. Afterwards the stream is
>> + * dead and any further read, poll, mmap or ioctl on the stream fd fails with -ENODEV.
>> + */
>> static void xe_oa_destroy_locked(struct xe_oa_stream *stream)
>> {
>> + unsigned long flags;
>> +
>> xe_pm_runtime_get(gt_to_xe(stream->gt));
>> + mutex_lock(&stream->stream_lock);
>> +
>> + spin_lock_irqsave(&stream->oa_buffer.ptr_lock, flags);
>> + WRITE_ONCE(stream->dead, true);
>> + spin_unlock_irqrestore(&stream->oa_buffer.ptr_lock, flags);
>>
>> if (stream->enabled)
>> xe_oa_disable_locked(stream);
>>
>> + wake_up_all(&stream->poll_wq);
>> +
>> xe_oa_stream_destroy(stream);
> [Severity: Medium]
> When the device is wedged, does xe_oa_stream_destroy() stall for multiple
> seconds and log a timeout error?
>
> xe_oa_stream_destroy() unconditionally calls xe_oa_stream_drain() and
> xe_oa_disable_metric_set() without checking xe_wedged(). If
> stream->last_fence is unsignaled when the device wedges (for example when
> a config batch is waiting on an input fence or after xe_oa_emit_oa_config()
> breaks out of its wait loop on wedge), xe_oa_stream_drain() sleeps
> uninterruptibly for job_timeout_ms + HZ and prints "Timed out waiting for
> OA batch". Then, when stream->exec_q is set, xe_oa_disable_metric_set()
> calls xe_oa_configure_oa_context(stream, false) to submit a new batch to
> the already-wedged queue (which guc_exec_queue_run_job() does not submit
> to hardware) and waits again in xe_oa_stream_drain() until TDR or timeout.
>
> [Severity: Medium]
> Similarly, in xe_oa_stream_init(), what happens when
> xe_oa_enable_metric_set() -> xe_oa_emit_oa_config() cuts its synchronous
> wait short on device unplug or wedge (introduced in commit "drm/xe/oa:
> Fence OA ioctls and mmap against device unplug and wedging")?
>
> drivers/gpu/drm/xe/xe_oa.c:xe_oa_stream_init() {
> ...
> err_put_k_exec_q:
> xe_oa_disable_metric_set(stream);
> xe_exec_queue_put(stream->k_exec_q);
> err_free_oa_buf:
> xe_oa_free_oa_buffer(stream);
> err_fw_put:
> xe_force_wake_put(gt_to_fw(gt), stream->fw_ref);
> xe_pm_runtime_put(stream->oa->xe);
> xe_oa_free_configs(stream);
> ...
> }
>
> At that point the config batch has already been submitted to
> stream->k_exec_q and recorded in stream->last_fence. Unlike
> xe_oa_stream_destroy(), err_put_k_exec_q does not call
> xe_oa_stream_drain() before xe_oa_disable_metric_set(). When
> stream->exec_q is NULL, the in-flight config batch on stream->k_exec_q is
> never drained before OA registers, forcewake, and runtime PM are released.
> When stream->exec_q is non-NULL, xe_oa_disable_metric_set() queues a
> context-restore batch behind the unsignaled config batch and blocks in
> xe_oa_stream_drain() for job_timeout_ms + HZ while still inside the
> drm_dev_enter() section of xe_observation_ioctl(), blocking
> drm_dev_unplug().
>
>>
>> if (stream->exec_q)
>> xe_exec_queue_put(stream->exec_q);
>>
>> drm_syncobj_put(stream->ufence_syncobj);
>> + mutex_unlock(&stream->stream_lock);
>> xe_pm_runtime_put(gt_to_xe(stream->gt));
>> }
>>
>> @@ -1775,14 +1823,24 @@ static int xe_oa_release(struct inode *inode, struct file *file)
>> struct xe_oa_stream *stream = file->private_data;
>> struct xe_gt *gt = stream->gt;
>>
>> + /* gt_lock serializes against xe_oa_fini(), which destroys the stream on device removal */
>> mutex_lock(>->oa.gt_lock);
>> - xe_oa_destroy_locked(stream);
>> + if (!stream->dead)
>> + xe_oa_destroy_locked(stream);
>> mutex_unlock(>->oa.gt_lock);
>>
>> xe_oa_free_oa_buffer(stream);
>> mutex_destroy(&stream->stream_lock);
>> kfree(stream);
>>
>> + /*
>> + * Freeing the OA buffer may defer the BO destroy to the TTM workqueue. Once the device
>> + * is unplugged, this may be the last reference on the drm device, so let the destroy
>> + * finish before drm_dev_put() releases the GGTT.
>> + */
>> + if (drm_dev_is_unplugged(>_to_xe(gt)->drm))
>> + flush_workqueue(gt_to_xe(gt)->ttm.wq);
>> +
>> /* Release the reference the OA stream kept on the driver */
>> drm_dev_put(>_to_xe(gt)->drm);
> [Severity: Medium]
> Can xe_oa_release() race with drm_dev_unplug() and skip flushing
> xe->ttm.wq before dropping the final drm_device reference?
>
> CPU0 CPU1
> xe_oa_release()
> xe_oa_free_oa_buffer(stream)
> ...
> queue_work(xe->ttm.wq,
> &bo->delayed_delete)
> drm_dev_is_unplugged(&xe->drm)
> --> false
> xe_device_remove()
> drm_dev_unplug(&xe->drm)
> ...
> drm_dev_put(&xe->drm)
> drm_dev_put(&xe->drm)
> drm_managed_release()
> ggtt_fini_early()
> drm_mm_takedown(&ggtt->mm)
> ttm_bo_delayed_delete()
> xe_ttm_bo_destroy()
> xe_ggtt_remove_bo(ggtt, bo)
> <-- use-after-free on ggtt
>
> drm_dev_is_unplugged() drops its SRCU read lock immediately, and nothing
> during device removal flushes xe->ttm.wq for an already-deleted BO. Because
> ggtt_fini_early() is registered with drmm after xe_device_destroy()
> (which calls ttm_device_fini() to drain xe->ttm.wq), the final
> drm_dev_put() tears down and frees tile->mem.ggtt while
> bo->delayed_delete is still pending on xe->ttm.wq.
>
> [Severity: Medium]
> Even after the device is already unplugged, can a temporary reference on
> stream->oa_buffer.bo outlive xe_oa_free_oa_buffer() and cause
> flush_workqueue(xe->ttm.wq) to run before bo->delayed_delete is queued?
>
> CPU0 CPU1
> xe_device_remove()
> ...
> xe_bo_pinned_fini()
> xe_bo_apply_to_pinned()
> xe_bo_get(bo)
> spin_unlock(&xe->pinned.lock)
> xe_oa_release()
> xe_oa_free_oa_buffer()
> xe_bo_unpin_map_no_vm(bo)
> xe_bo_put(bo)
> flush_workqueue(xe->ttm.wq)
> xe_bo_dma_unmap_pinned(bo)
> xe_bo_put(bo)
> queue_work(xe->ttm.wq,
> &bo->delayed_delete)
> drm_dev_put(&xe->drm)
> drm_dev_put(&xe->drm)
> drm_managed_release()
> ggtt_fini_early()
> <-- ggtt freed before
> bo->delayed_delete runs
>
> Because xe_oa_stream_destroy() leaves stream->oa_buffer.bo on
> xe->pinned.early.kernel_bo_present, xe_bo_pinned_fini() still visits it
> and holds a reference outside xe->pinned.lock. Similarly, in
> xe_bo_unpin_map_no_vm(), xe_bo_unpin() puts the BO back on the TTM LRU
> and unlocks dma-resv before xe_bo_put(bo), allowing xe_shrinker_scan() to
> grab a temporary BO reference. In either case, xe_oa_release() flushes
> xe->ttm.wq before the last BO reference is dropped.
>
--
Shekhar Chauhan
Linux Graphics Software Engineer
Linux Platform Enabling
Intel Corporation
next prev parent reply other threads:[~2026-11-20 7:31 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 0:27 [PATCH v8 0/8] Cleanup OA stream on PCI unbind/remove Umesh Nerlige Ramappa
2026-11-19 0:27 ` [PATCH v8 1/8] drm/xe/oa: Ignore a HW tail pointer outside the OA buffer Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-20 20:14 ` Umesh Nerlige Ramappa
2026-11-20 7:21 ` Shekhar Chauhan
2026-11-20 20:07 ` Umesh Nerlige Ramappa
2026-11-19 0:27 ` [PATCH v8 2/8] drm/xe/oa: Use the drm device address_space for the OA stream fd Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-20 7:23 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 3/8] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown Umesh Nerlige Ramappa
2026-11-20 7:24 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 4/8] drm/xe: Reject vm_ops->access on BO mappings of an unplugged device Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-19 22:36 ` Matthew Brost
2026-11-19 0:27 ` [PATCH v8 5/8] drm/xe/oa: Fence OA ioctls and mmap against device unplug and wedging Umesh Nerlige Ramappa
2026-11-20 7:25 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 6/8] drm/xe/oa: Take the runtime PM reference in xe_oa_destroy_locked() Umesh Nerlige Ramappa
2026-11-20 7:27 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 7/8] drm/xe/oa: Free the OA buffer and the stream after the stream destroy Umesh Nerlige Ramappa
2026-11-19 0:48 ` sashiko-bot
2026-11-20 7:29 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 8/8] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset Umesh Nerlige Ramappa
2026-11-19 1:11 ` sashiko-bot
2026-11-20 7:31 ` Shekhar Chauhan [this message]
2026-11-19 6:05 ` ✓ CI.KUnit: success for Cleanup OA stream on PCI unbind/remove (rev8) Patchwork
2026-11-19 6:46 ` ✓ Xe.CI.BAT: " Patchwork
2026-11-20 9:31 ` ✗ Xe.CI.FULL: failure " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=93e5b6be-6087-43a2-b8c8-8607cbcc7d71@intel.com \
--to=shekhar.chauhan@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=umesh.nerlige.ramappa@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.