All of lore.kernel.org
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Umesh Nerlige Ramappa" <umesh.nerlige.ramappa@intel.com>,
	intel-xe@lists.freedesktop.org
Subject: Re: [PATCH v9 7/7] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset
Date: Sat, 21 Nov 2026 02:37:29 +0000	[thread overview]
Message-ID: <sashiko-outbox-226548@kernel.org> (raw)
In-Reply-To: <20261121020145.147384-16-umesh.nerlige.ramappa@intel.com>

Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [Medium] drm/xe/oa: xe_oa_release() drops gt_lock before
  xe_oa_free_oa_buffer()
- [Medium] drm/xe/oa: drm_dev_is_unplugged() race in xe_oa_release()
  skips ttm.wq flush
- [Medium] drm/xe/oa: OA buffer BO references can outlive
  flush_workqueue() on unplug
- [Medium] drm/xe/oa: unlocked stream->dead check for runtime PM in
  xe_oa_release()
- [Medium] drm/xe/oa: xe_oa_disable_metric_set() submits batch on wedged
  device
--

commit ee62062a6a8afa8ac686e03248886c0c4f01f369
Author: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>

drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset

This commit quiesces active OA streams on driver unbind and PCI slot reset
by marking streams dead, tearing down hardware and device-bound state in
xe_oa_fini(), and deferring OA buffer and stream struct deallocation until
xe_oa_release().

> diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
> index e719cd92b271b..e8228a244cc2e 100644
> --- a/drivers/gpu/drm/xe/xe_oa.c
> +++ b/drivers/gpu/drm/xe/xe_oa.c

[ ... ]

> @@ -865,7 +887,9 @@ static void xe_oa_stream_drain(struct xe_oa_stream *stream)
>  	struct xe_exec_queue *q = stream->exec_q ?: stream->k_exec_q;
>  	long timeout;
>  
> -	if (!stream->last_fence || dma_fence_is_signaled(stream->last_fence))
> +	/* Nothing in the ring runs anymore on a wedged device */
> +	if (xe_wedged(stream->oa->xe) || !stream->last_fence ||
> +	    dma_fence_is_signaled(stream->last_fence))
>  		return;
>  
>  	/*

[Severity: Medium]
When a stream with stream->exec_q is destroyed on a wedged device, does
xe_oa_disable_metric_set() still submit a context-restore batch whose
fence will never signal?

drivers/gpu/drm/xe/xe_oa.c:xe_oa_disable_metric_set() {
    ...
	if (stream->exec_q) {
		xe_oa_configure_oa_context(stream, false);
		xe_oa_stream_drain(stream);
	}
    ...
}

xe_oa_configure_oa_context() pushes a new job onto the already-wedged
stream->exec_q and replaces stream->last_fence with the new unsignaled job
fence, which xe_oa_stream_drain() skips waiting for and
xe_oa_free_configs() attaches to every oa_bo->bb suballocation in
kernel_bb_pool.  On a long-running queue where guc_exec_queue_kill()
already ran at wedge time, work_tdr is never scheduled for the new job,
leaking the job, stream->exec_q and its drm_device reference.

Guarding the xe_oa_configure_oa_context() call in
xe_oa_disable_metric_set() with !xe_wedged(stream->oa->xe) would avoid
submitting a batch to a wedged device.

[ ... ]

> @@ -1754,33 +1781,67 @@ static long xe_oa_ioctl(struct file *file,
>  	return ret;
>  }
>  
> +/*
> + * Disable the stream, restore the HW state and release everything but the OA buffer and the
> + * stream struct, the same whether the stream fd is closed or the device goes away, and
> + * whether or not the HW still responds: all HW waits are bounded. Afterwards the stream is
> + * dead and any further read, poll, mmap or ioctl on the stream fd fails with -ENODEV.
> + */
>  static void xe_oa_destroy_locked(struct xe_oa_stream *stream)
>  {
> +	unsigned long flags;
> +
> +	mutex_lock(&stream->stream_lock);
> +
> +	spin_lock_irqsave(&stream->oa_buffer.ptr_lock, flags);
> +	WRITE_ONCE(stream->dead, true);
> +	spin_unlock_irqrestore(&stream->oa_buffer.ptr_lock, flags);
> +
>  	if (stream->enabled)
>  		xe_oa_disable_locked(stream);
>  
> +	wake_up_all(&stream->poll_wq);
> +
>  	xe_oa_stream_destroy(stream);
>  
>  	if (stream->exec_q)
>  		xe_exec_queue_put(stream->exec_q);
>  
>  	drm_syncobj_put(stream->ufence_syncobj);
> +	mutex_unlock(&stream->stream_lock);
>  }
>  
>  static int xe_oa_release(struct inode *inode, struct file *file)
>  {
>  	struct xe_oa_stream *stream = file->private_data;
>  	struct xe_gt *gt = stream->gt;
> +	/*
> +	 * Don't wake a removed device for a stream xe_oa_fini() destroyed. If xe_oa_fini()
> +	 * marks the stream dead only after this, the device is still bound while it does.
> +	 */
> +	bool rpm = !READ_ONCE(stream->dead);
>  
> -	xe_pm_runtime_get(gt_to_xe(gt));
> +	if (rpm)
> +		xe_pm_runtime_get(gt_to_xe(gt));

[Severity: Medium]
Can sampling stream->dead before acquiring gt->oa.gt_lock still wake or
touch runtime PM on an unbound device?

CPU0                              CPU1
xe_oa_release()
  rpm = !READ_ONCE(stream->dead);
  /* rpm == true */
                                  xe_device_remove()
                                    ...
                                    xe_oa_fini()
                                      xe_oa_quiesce_gt()
                                        mutex_lock(&gt->oa.gt_lock)
                                        xe_oa_destroy_locked(stream)
                                          stream->dead = true
                                        mutex_unlock(&gt->oa.gt_lock)
                                    /* driver unbind completes */
  xe_pm_runtime_get(...)  <-- runtime PM on unbound device
  ...
  xe_pm_runtime_put(...)

Nothing in xe_oa_fini() or devres_release_all() waits for an in-flight
xe_oa_release() that has not yet acquired gt->oa.gt_lock.

> +	/* gt_lock serializes against xe_oa_fini(), which destroys the stream on device removal */
>  	mutex_lock(&gt->oa.gt_lock);
> -	xe_oa_destroy_locked(stream);
> +	if (!stream->dead)
> +		xe_oa_destroy_locked(stream);
>  	mutex_unlock(&gt->oa.gt_lock);
>  
>  	xe_oa_free_oa_buffer(stream);

[Severity: Medium]
Can dropping gt->oa.gt_lock before xe_oa_free_oa_buffer() let
xe_bo_pinned_fini() race with unpinning stream->oa_buffer.bo?

CPU0                              CPU1
xe_oa_release()
  mutex_lock(&gt->oa.gt_lock)
  xe_oa_destroy_locked(stream)
    u->exclusive_stream = NULL
  mutex_unlock(&gt->oa.gt_lock)
                                  xe_oa_quiesce_gt()
                                    mutex_lock(&gt->oa.gt_lock)
                                    /* exclusive_stream is NULL, skips
                                       xe_bo_dma_unmap_pinned() */
                                    mutex_unlock(&gt->oa.gt_lock)
                                  xe_bo_pinned_fini()
                                    xe_bo_apply_to_pinned()
                                      xe_bo_dma_unmap_pinned(bo)
  xe_oa_free_oa_buffer(stream)
    xe_bo_unpin_map_no_vm(bo)  <-- races on xe_tt->sg without bo lock

Once xe_bo_unpin_map_no_vm() unpins and unlocks the BO while
xe_bo_apply_to_pinned() still holds its temporary BO reference, a
concurrent shrinker unmap (xe_tt_unmap_sg()) and xe_bo_dma_unmap_pinned()
can both unmap and free xe_tt->sg.  Keeping xe_oa_free_oa_buffer() inside
gt->oa.gt_lock (or DMA-unmapping before clearing exclusive_stream) would
close this window.

>  	mutex_destroy(&stream->stream_lock);
>  	kfree(stream);
> -	xe_pm_runtime_put(gt_to_xe(gt));
> +	if (rpm)
> +		xe_pm_runtime_put(gt_to_xe(gt));
> +
> +	/*
> +	 * Freeing the OA buffer may defer the BO destroy to the TTM workqueue. Once the device
> +	 * is unplugged, this may be the last reference on the drm device, so let the destroy
> +	 * finish before drm_dev_put() releases the GGTT.
> +	 */
> +	if (drm_dev_is_unplugged(&gt_to_xe(gt)->drm))
> +		flush_workqueue(gt_to_xe(gt)->ttm.wq);
>  
>  	/* Release the reference the OA stream kept on the driver */
>  	drm_dev_put(&gt_to_xe(gt)->drm);

[Severity: Medium]
Can drm_dev_is_unplugged() return false right before drm_dev_unplug() runs
on driver unbind, skipping flush_workqueue() when bo->delayed_delete is
already queued on xe->ttm.wq?

CPU0                              CPU1
xe_oa_release()
  xe_oa_free_oa_buffer(stream)
    /* queues bo->delayed_delete
       on xe->ttm.wq */
  drm_dev_is_unplugged(...)
    /* returns false */
                                  xe_device_remove()
                                    drm_dev_unplug(&xe->drm)
                                    ...
                                    /* devm drops its drm_dev_put() */
  drm_dev_put(&xe->drm)
    drm_managed_release()
      ggtt_fini_early()
                                  ttm_bo_delayed_delete()
                                    xe_ttm_bo_destroy()
                                      xe_ggtt_remove_bo()  <-- UAF on ggtt

ggtt_fini_early() is registered with drmm after xe_device_destroy(), so on
the final drm_dev_put() it tears down ggtt->mm and frees tile->mem.ggtt
before xe_device_destroy() calls ttm_device_fini() to drain xe->ttm.wq.

[ ... ]

> @@ -2673,6 +2740,49 @@ static void xe_oa_unregister(void *arg)
>  	oa->metrics_kobj = NULL;
>  }
>  
> +static void xe_oa_quiesce_gt(struct xe_gt *gt)
> +{
> +	int i;
> +
> +	mutex_lock(&gt->oa.gt_lock);
> +	for (i = 0; i < gt->oa.num_oa_units; i++) {
> +		struct xe_oa_stream *stream = gt->oa.oa_unit[i].exclusive_stream;
> +
> +		if (!stream)
> +			continue;
> +
> +		xe_oa_destroy_locked(stream);
> +
> +		/*
> +		 * The OA buffer stays pinned until the stream fd is released, an mmap may still
> +		 * hold a reference on it. Drop its DMA mapping now: the fd may be released, and
> +		 * the BO unpinned, while xe_bo_pinned_fini() DMA-unmaps it without the BO lock.
> +		 */
> +		xe_bo_dma_unmap_pinned(stream->oa_buffer.bo);

[Severity: Medium]
Even after the device is unplugged, can a temporary reference on
stream->oa_buffer.bo outlive xe_oa_release()'s flush_workqueue(xe->ttm.wq)
so that bo->delayed_delete is queued only after the flush?

xe_bo_dma_unmap_pinned() leaves bo->pinned_link on
xe->pinned.early.kernel_bo_present, where xe_bo_pinned_fini() ->
xe_bo_apply_to_pinned() grabs a temporary reference via xe_bo_get(bo) and
drops xe->pinned.lock before calling xe_bo_put(bo):

CPU0                              CPU1
xe_bo_pinned_fini()
  xe_bo_apply_to_pinned()
    xe_bo_get(bo)
    spin_unlock(&xe->pinned.lock)
                                  xe_oa_release()
                                    xe_oa_free_oa_buffer(stream)
                                      xe_bo_unpin(bo)
                                      xe_bo_put(bo) /* refcount > 0 */
                                    flush_workqueue(xe->ttm.wq)
    xe_bo_dma_unmap_pinned(bo)
    xe_bo_put(bo)
      /* queues bo->delayed_delete
         after flush_workqueue() */
                                    drm_dev_put(&xe->drm)
                                      ggtt_fini_early()  <-- UAF in destroy

The same can happen if memory reclaim (xe_shrinker_scan()) grabs a
temporary reference on the BO between xe_bo_unpin() + xe_bo_unlock() and
xe_bo_put() in xe_bo_unpin_map_no_vm().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261121020145.147384-9-umesh.nerlige.ramappa@intel.com?part=7

  reply	other threads:[~2026-11-21  2:37 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-21  2:01 [PATCH v9 0/7] Cleanup OA stream on PCI unbind/remove Umesh Nerlige Ramappa
2026-11-21  2:01 ` [PATCH v9 1/7] drm/xe/oa: Ignore a HW tail pointer outside the OA buffer Umesh Nerlige Ramappa
2026-11-21  2:01 ` [PATCH v9 2/7] drm/xe/oa: Use the drm device address_space for the OA stream fd Umesh Nerlige Ramappa
2026-11-21  2:01 ` [PATCH v9 3/7] drm/xe: Reject vm_ops->access on BO mappings of an unplugged device Umesh Nerlige Ramappa
2026-11-21  2:01 ` [PATCH v9 4/7] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown Umesh Nerlige Ramappa
2026-11-21  2:11   ` sashiko-bot
2026-11-21  2:01 ` [PATCH v9 5/7] drm/xe/oa: Fence OA ioctls and mmap against device unplug and wedging Umesh Nerlige Ramappa
2026-11-21  2:01 ` [PATCH v9 6/7] drm/xe/oa: Free the OA buffer and the stream after the stream destroy Umesh Nerlige Ramappa
2026-11-21  2:01 ` [PATCH v9 7/7] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset Umesh Nerlige Ramappa
2026-11-21  2:37   ` sashiko-bot [this message]
2026-11-21  2:09 ` ✗ CI.checkpatch: warning for Cleanup OA stream on PCI unbind/remove (rev9) Patchwork
2026-11-21  2:11 ` ✓ CI.KUnit: success " Patchwork

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-226548@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=intel-xe@lists.freedesktop.org \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=umesh.nerlige.ramappa@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.