From: Shekhar Chauhan <shekhar.chauhan@intel.com>
To: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>,
<intel-xe@lists.freedesktop.org>
Subject: Re: [PATCH v8 8/8] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
Date: Fri, 20 Nov 2026 13:01:04 +0530 [thread overview]
Message-ID: <93e5b6be-6087-43a2-b8c8-8607cbcc7d71@intel.com> (raw)
In-Reply-To: <sashiko-outbox-222279@kernel.org>
On 11/19/2026 6:41, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 5 potential
> issues to consider:
> - [Medium] drm/xe/oa: drm_dev_is_unplugged() race in xe_oa_release()
> skips ttm.wq flush
> - [Medium] drm/xe/oa: OA buffer BO references can outlive
> flush_workqueue() on unplug
> - [Medium] drm/xe/oa: missing or blocking drain in xe_oa_stream_init()
> error path
> - [Medium] drm/xe/oa: NULL sg_table deref on resume after
> xe_bo_dma_unmap_pinned()
> - [Medium] drm/xe/oa: unconditional drain and batch submit on wedged
> device in destroy
> --
>
> commit 2b379416db1422398b8cce664304d4acea228c64
> Author: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
>
> drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
>
> This commit quiesces active OA streams on driver unbind or PCI reset by
> marking them dead and tearing down their hardware and device resources
> while keeping the stream structure and pinned OA buffer allocated until
> the stream file descriptor is closed.
I'm unsure on the case reported by Sashiko. Please have a look.
-shekhar
>
>> diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
>> index 223419347eb7b..0b51bc818ef47 100644
>> --- a/drivers/gpu/drm/xe/xe_oa.c
>> +++ b/drivers/gpu/drm/xe/xe_oa.c
> [ ... ]
>
>> @@ -937,6 +959,13 @@ static void xe_oa_stream_destroy(struct xe_oa_stream *stream)
>> xe_oa_disable_metric_set(stream);
>> xe_exec_queue_put(stream->k_exec_q);
>>
>> + /*
>> + * The OA buffer is freed on release of the stream fd, an mmap may still hold a
>> + * reference on it. Drop its DMA mapping now, while pinned: the fd may be released, and
>> + * the BO unpinned, while xe_bo_pinned_fini() DMA-unmaps it without the BO lock.
>> + */
>> + xe_bo_dma_unmap_pinned(stream->oa_buffer.bo);
>> +
>> xe_force_wake_put(gt_to_fw(gt), stream->fw_ref);
>> xe_pm_runtime_put(stream->oa->xe);
> [Severity: Medium]
> Can calling xe_bo_dma_unmap_pinned() here while stream->oa_buffer.bo stays
> pinned on xe->pinned.early.kernel_bo_present cause a NULL pointer
> dereference if the device suspends and resumes before
> xe_oa_free_oa_buffer() unpins the BO?
>
> CPU0 CPU1
> xe_oa_release()
> xe_oa_destroy_locked()
> xe_oa_stream_destroy()
> xe_bo_dma_unmap_pinned(bo)
> xe_tt->sg = NULL
> xe_pm_runtime_put()
> xe_pm_runtime_put()
> xe_pm_suspend()
> xe_bo_evict_all()
> xe_pm_resume()
> xe_bo_restore_early()
> xe_bo_restore_and_map_ggtt()
> xe_ggtt_map_bo()
> xe_res_first_sg(NULL, ...)
> <-- NULL pointer dereference
> xe_oa_free_oa_buffer()
> xe_bo_unpin_map_no_vm(bo)
>
> Non-MERT OA buffers are created with XE_BO_FLAG_SYSTEM | XE_BO_FLAG_GGTT
> and remain linked on xe->pinned.early.kernel_bo_present until
> xe_bo_unpin() removes bo->pinned_link. Once xe_oa_destroy_locked() drops
> the last runtime PM reference, a system or D3cold runtime suspend/resume
> cycle will evict and restore the pinned BO via xe_ggtt_map_bo(), passing
> xe_bo_sg(bo) == NULL to xe_res_first_sg().
>
> [ ... ]
>
>> @@ -1754,19 +1786,35 @@ static long xe_oa_ioctl(struct file *file,
>> return ret;
>> }
>>
>> +/*
>> + * Disable the stream, restore the HW state and release everything but the OA buffer and the
>> + * stream struct, the same whether the stream fd is closed or the device goes away, and
>> + * whether or not the HW still responds: all HW waits are bounded. Afterwards the stream is
>> + * dead and any further read, poll, mmap or ioctl on the stream fd fails with -ENODEV.
>> + */
>> static void xe_oa_destroy_locked(struct xe_oa_stream *stream)
>> {
>> + unsigned long flags;
>> +
>> xe_pm_runtime_get(gt_to_xe(stream->gt));
>> + mutex_lock(&stream->stream_lock);
>> +
>> + spin_lock_irqsave(&stream->oa_buffer.ptr_lock, flags);
>> + WRITE_ONCE(stream->dead, true);
>> + spin_unlock_irqrestore(&stream->oa_buffer.ptr_lock, flags);
>>
>> if (stream->enabled)
>> xe_oa_disable_locked(stream);
>>
>> + wake_up_all(&stream->poll_wq);
>> +
>> xe_oa_stream_destroy(stream);
> [Severity: Medium]
> When the device is wedged, does xe_oa_stream_destroy() stall for multiple
> seconds and log a timeout error?
>
> xe_oa_stream_destroy() unconditionally calls xe_oa_stream_drain() and
> xe_oa_disable_metric_set() without checking xe_wedged(). If
> stream->last_fence is unsignaled when the device wedges (for example when
> a config batch is waiting on an input fence or after xe_oa_emit_oa_config()
> breaks out of its wait loop on wedge), xe_oa_stream_drain() sleeps
> uninterruptibly for job_timeout_ms + HZ and prints "Timed out waiting for
> OA batch". Then, when stream->exec_q is set, xe_oa_disable_metric_set()
> calls xe_oa_configure_oa_context(stream, false) to submit a new batch to
> the already-wedged queue (which guc_exec_queue_run_job() does not submit
> to hardware) and waits again in xe_oa_stream_drain() until TDR or timeout.
>
> [Severity: Medium]
> Similarly, in xe_oa_stream_init(), what happens when
> xe_oa_enable_metric_set() -> xe_oa_emit_oa_config() cuts its synchronous
> wait short on device unplug or wedge (introduced in commit "drm/xe/oa:
> Fence OA ioctls and mmap against device unplug and wedging")?
>
> drivers/gpu/drm/xe/xe_oa.c:xe_oa_stream_init() {
> ...
> err_put_k_exec_q:
> xe_oa_disable_metric_set(stream);
> xe_exec_queue_put(stream->k_exec_q);
> err_free_oa_buf:
> xe_oa_free_oa_buffer(stream);
> err_fw_put:
> xe_force_wake_put(gt_to_fw(gt), stream->fw_ref);
> xe_pm_runtime_put(stream->oa->xe);
> xe_oa_free_configs(stream);
> ...
> }
>
> At that point the config batch has already been submitted to
> stream->k_exec_q and recorded in stream->last_fence. Unlike
> xe_oa_stream_destroy(), err_put_k_exec_q does not call
> xe_oa_stream_drain() before xe_oa_disable_metric_set(). When
> stream->exec_q is NULL, the in-flight config batch on stream->k_exec_q is
> never drained before OA registers, forcewake, and runtime PM are released.
> When stream->exec_q is non-NULL, xe_oa_disable_metric_set() queues a
> context-restore batch behind the unsignaled config batch and blocks in
> xe_oa_stream_drain() for job_timeout_ms + HZ while still inside the
> drm_dev_enter() section of xe_observation_ioctl(), blocking
> drm_dev_unplug().
>
>>
>> if (stream->exec_q)
>> xe_exec_queue_put(stream->exec_q);
>>
>> drm_syncobj_put(stream->ufence_syncobj);
>> + mutex_unlock(&stream->stream_lock);
>> xe_pm_runtime_put(gt_to_xe(stream->gt));
>> }
>>
>> @@ -1775,14 +1823,24 @@ static int xe_oa_release(struct inode *inode, struct file *file)
>> struct xe_oa_stream *stream = file->private_data;
>> struct xe_gt *gt = stream->gt;
>>
>> + /* gt_lock serializes against xe_oa_fini(), which destroys the stream on device removal */
>> mutex_lock(>->oa.gt_lock);
>> - xe_oa_destroy_locked(stream);
>> + if (!stream->dead)
>> + xe_oa_destroy_locked(stream);
>> mutex_unlock(>->oa.gt_lock);
>>
>> xe_oa_free_oa_buffer(stream);
>> mutex_destroy(&stream->stream_lock);
>> kfree(stream);
>>
>> + /*
>> + * Freeing the OA buffer may defer the BO destroy to the TTM workqueue. Once the device
>> + * is unplugged, this may be the last reference on the drm device, so let the destroy
>> + * finish before drm_dev_put() releases the GGTT.
>> + */
>> + if (drm_dev_is_unplugged(>_to_xe(gt)->drm))
>> + flush_workqueue(gt_to_xe(gt)->ttm.wq);
>> +
>> /* Release the reference the OA stream kept on the driver */
>> drm_dev_put(>_to_xe(gt)->drm);
> [Severity: Medium]
> Can xe_oa_release() race with drm_dev_unplug() and skip flushing
> xe->ttm.wq before dropping the final drm_device reference?
>
> CPU0 CPU1
> xe_oa_release()
> xe_oa_free_oa_buffer(stream)
> ...
> queue_work(xe->ttm.wq,
> &bo->delayed_delete)
> drm_dev_is_unplugged(&xe->drm)
> --> false
> xe_device_remove()
> drm_dev_unplug(&xe->drm)
> ...
> drm_dev_put(&xe->drm)
> drm_dev_put(&xe->drm)
> drm_managed_release()
> ggtt_fini_early()
> drm_mm_takedown(&ggtt->mm)
> ttm_bo_delayed_delete()
> xe_ttm_bo_destroy()
> xe_ggtt_remove_bo(ggtt, bo)
> <-- use-after-free on ggtt
>
> drm_dev_is_unplugged() drops its SRCU read lock immediately, and nothing
> during device removal flushes xe->ttm.wq for an already-deleted BO. Because
> ggtt_fini_early() is registered with drmm after xe_device_destroy()
> (which calls ttm_device_fini() to drain xe->ttm.wq), the final
> drm_dev_put() tears down and frees tile->mem.ggtt while
> bo->delayed_delete is still pending on xe->ttm.wq.
>
> [Severity: Medium]
> Even after the device is already unplugged, can a temporary reference on
> stream->oa_buffer.bo outlive xe_oa_free_oa_buffer() and cause
> flush_workqueue(xe->ttm.wq) to run before bo->delayed_delete is queued?
>
> CPU0 CPU1
> xe_device_remove()
> ...
> xe_bo_pinned_fini()
> xe_bo_apply_to_pinned()
> xe_bo_get(bo)
> spin_unlock(&xe->pinned.lock)
> xe_oa_release()
> xe_oa_free_oa_buffer()
> xe_bo_unpin_map_no_vm(bo)
> xe_bo_put(bo)
> flush_workqueue(xe->ttm.wq)
> xe_bo_dma_unmap_pinned(bo)
> xe_bo_put(bo)
> queue_work(xe->ttm.wq,
> &bo->delayed_delete)
> drm_dev_put(&xe->drm)
> drm_dev_put(&xe->drm)
> drm_managed_release()
> ggtt_fini_early()
> <-- ggtt freed before
> bo->delayed_delete runs
>
> Because xe_oa_stream_destroy() leaves stream->oa_buffer.bo on
> xe->pinned.early.kernel_bo_present, xe_bo_pinned_fini() still visits it
> and holds a reference outside xe->pinned.lock. Similarly, in
> xe_bo_unpin_map_no_vm(), xe_bo_unpin() puts the BO back on the TTM LRU
> and unlocks dma-resv before xe_bo_put(bo), allowing xe_shrinker_scan() to
> grab a temporary BO reference. In either case, xe_oa_release() flushes
> xe->ttm.wq before the last BO reference is dropped.
>
--
Shekhar Chauhan
Linux Graphics Software Engineer
Linux Platform Enabling
Intel Corporation
next prev parent reply other threads:[~2026-11-20 7:31 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 0:27 [PATCH v8 0/8] Cleanup OA stream on PCI unbind/remove Umesh Nerlige Ramappa
2026-11-19 0:27 ` [PATCH v8 1/8] drm/xe/oa: Ignore a HW tail pointer outside the OA buffer Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-20 7:21 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 2/8] drm/xe/oa: Use the drm device address_space for the OA stream fd Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-20 7:23 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 3/8] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown Umesh Nerlige Ramappa
2026-11-20 7:24 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 4/8] drm/xe: Reject vm_ops->access on BO mappings of an unplugged device Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-19 22:36 ` Matthew Brost
2026-11-19 0:27 ` [PATCH v8 5/8] drm/xe/oa: Fence OA ioctls and mmap against device unplug and wedging Umesh Nerlige Ramappa
2026-11-20 7:25 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 6/8] drm/xe/oa: Take the runtime PM reference in xe_oa_destroy_locked() Umesh Nerlige Ramappa
2026-11-20 7:27 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 7/8] drm/xe/oa: Free the OA buffer and the stream after the stream destroy Umesh Nerlige Ramappa
2026-11-19 0:48 ` sashiko-bot
2026-11-20 7:29 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 8/8] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset Umesh Nerlige Ramappa
2026-11-19 1:11 ` sashiko-bot
2026-11-20 7:31 ` Shekhar Chauhan [this message]
2026-11-19 6:05 ` ✓ CI.KUnit: success for Cleanup OA stream on PCI unbind/remove (rev8) Patchwork
2026-11-19 6:46 ` ✓ Xe.CI.BAT: " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=93e5b6be-6087-43a2-b8c8-8607cbcc7d71@intel.com \
--to=shekhar.chauhan@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=umesh.nerlige.ramappa@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox