From: Shekhar Chauhan <shekhar.chauhan@intel.com>
To: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>,
<intel-xe@lists.freedesktop.org>, <harish.chegondi@intel.com>
Cc: <matthew.brost@intel.com>
Subject: Re: [PATCH v8 5/8] drm/xe/oa: Fence OA ioctls and mmap against device unplug and wedging
Date: Fri, 20 Nov 2026 12:55:15 +0530 [thread overview]
Message-ID: <82c8bbc5-bdb3-44e5-bf3c-c1906e666016@intel.com> (raw)
In-Reply-To: <20261119002737.3899877-15-umesh.nerlige.ramappa@intel.com>
On 11/19/2026 5:57, Umesh Nerlige Ramappa wrote:
> drm_ioctl() only checks for unplug on entry, so an OA ioctl that
> started before drm_dev_unplug() can still open a stream, or access
> oa->xe and the metrics idr, while the device is removed. The ioctls and
> the mmap of the OA stream fd don't go through drm_ioctl() at all, and
> neither through the wedged check of xe_drm_ioctl().
>
> Run the OA ioctls of the drm fd and the ioctls and mmap of the OA stream
> fd in a drm_dev_enter() section, so that they fail with -ENODEV once the
> device is unplugged, and drm_dev_unplug() waits for the ones in flight.
> Fail the stream fd ioctls with -ECANCELED on a wedged device, like
> xe_drm_ioctl() does.
>
> A config without output syncs waits synchronously for its batch, while
> holding stream_lock or gt_lock and now in a drm_dev_enter() section. If
> the batch waits on an input fence that never signals, or never completes
> on a wedged device, that would block drm_dev_unplug() or the stream
> teardown. Stop waiting once the device is unplugged or wedged, and fail
> the config.
>
> A stream fd can still outlive the device, the following patch handles
> that.
>
> Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
LGTM,
Reviewed-by: Shekhar Chauhan <shekhar.chauhan@intel.com>
> ---
> v8: Split out of "drm/xe/oa: Quiesce OA streams on driver unbind and PCI
> reset", which carried these changes since v4
> ---
> drivers/gpu/drm/xe/xe_oa.c | 75 +++++++++++++++++++++++------
> drivers/gpu/drm/xe/xe_observation.c | 20 +++++++-
> 2 files changed, 78 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
> index 1c2f57735e0d..72f173fb23cd 100644
> --- a/drivers/gpu/drm/xe/xe_oa.c
> +++ b/drivers/gpu/drm/xe/xe_oa.c
> @@ -43,6 +43,7 @@
> #include "xe_sriov.h"
> #include "xe_sync.h"
> #include "xe_wa.h"
> +#include "xe_wedge.h"
>
> #define DEFAULT_POLL_FREQUENCY_HZ 200
> #define DEFAULT_POLL_PERIOD_NS (NSEC_PER_SEC / DEFAULT_POLL_FREQUENCY_HZ)
> @@ -1082,6 +1083,7 @@ static int xe_oa_emit_oa_config(struct xe_oa_stream *stream, struct xe_oa_config
> struct xe_oa_fence *ofence;
> int i, err, num_signal = 0;
> struct dma_fence *fence;
> + int ret = 0;
>
> ofence = kzalloc_obj(*ofence);
> if (!ofence) {
> @@ -1125,12 +1127,28 @@ static int xe_oa_emit_oa_config(struct xe_oa_stream *stream, struct xe_oa_config
> if (err == -ENOENT)
> xe_oa_config_cb(fence, &ofence->cb);
>
> - /* If nothing needs to be signaled we wait synchronously */
> + /*
> + * If nothing needs to be signaled we wait synchronously unless the device
> + * is unplugged/wedged.
> + */
> if (!num_signal) {
> - dma_fence_wait(&ofence->base, false);
> + while (!dma_fence_wait_timeout(&ofence->base, false, HZ / 10)) {
> + if (drm_dev_is_unplugged(>_to_xe(stream->gt)->drm)) {
> + ret = -ENODEV;
> + break;
> + }
> + if (xe_wedged(gt_to_xe(stream->gt))) {
> + ret = -ECANCELED;
> + break;
> + }
> + }
> dma_fence_put(&ofence->base);
> }
>
> + /* The callers clean up the syncs on error */
> + if (ret)
> + return ret;
> +
> /* Done with syncs */
> for (i = 0; i < stream->num_syncs; i++)
> xe_sync_entry_cleanup(&stream->syncs[i]);
> @@ -1722,11 +1740,20 @@ static long xe_oa_ioctl(struct file *file,
> unsigned long arg)
> {
> struct xe_oa_stream *stream = file->private_data;
> - long ret;
> + struct xe_device *xe = gt_to_xe(stream->gt);
> + long ret = -ECANCELED;
> + int idx;
>
> - mutex_lock(&stream->stream_lock);
> - ret = xe_oa_ioctl_locked(stream, cmd, arg);
> - mutex_unlock(&stream->stream_lock);
> + if (!drm_dev_enter(&xe->drm, &idx))
> + return -ENODEV;
> +
> + if (!xe_wedged(xe)) {
> + mutex_lock(&stream->stream_lock);
> + ret = xe_oa_ioctl_locked(stream, cmd, arg);
> + mutex_unlock(&stream->stream_lock);
> + }
> +
> + drm_dev_exit(idx);
>
> return ret;
> }
> @@ -1765,27 +1792,41 @@ static int xe_oa_release(struct inode *inode, struct file *file)
> static int xe_oa_mmap(struct file *file, struct vm_area_struct *vma)
> {
> struct xe_oa_stream *stream = file->private_data;
> + struct xe_device *xe = gt_to_xe(stream->gt);
> int ret = xe_observation_paranoid_check();
> - struct xe_bo *bo = stream->oa_buffer.bo;
> + struct xe_bo *bo;
> + int idx;
>
> if (ret) {
> - drm_dbg(&stream->oa->xe->drm, "Insufficient privilege to map OA buffer\n");
> + drm_dbg(&xe->drm, "Insufficient privilege to map OA buffer\n");
> return ret;
> }
>
> + /*
> + * drm_dev_unplug() zaps the OA buffer mmaps and waits for this section to finish, so
> + * no mmap of the OA buffer can be set up after that. Can't use stream_lock here, it is
> + * held across user copies which fault on mmap_lock, held by the caller.
> + */
> + if (!drm_dev_enter(&xe->drm, &idx))
> + return -ENODEV;
> +
> /*
> * Userspace always maps the OA buffer at offset 0, the mapping is moved to the OA
> * buffer's own range in the drm device's address_space below
> */
> if (vma->vm_pgoff) {
> - drm_dbg(&stream->oa->xe->drm, "Wrong mmap offset, must be 0\n");
> - return -EINVAL;
> + drm_dbg(&xe->drm, "Wrong mmap offset, must be 0\n");
> + ret = -EINVAL;
> + goto exit;
> }
>
> + bo = stream->oa_buffer.bo;
> +
> /* Can mmap the entire OA buffer or nothing (no partial OA buffer mmaps) */
> if (vma->vm_end - vma->vm_start != xe_bo_size(bo)) {
> - drm_dbg(&stream->oa->xe->drm, "Wrong mmap size, must be OA buffer size\n");
> - return -EINVAL;
> + drm_dbg(&xe->drm, "Wrong mmap size, must be OA buffer size\n");
> + ret = -EINVAL;
> + goto exit;
> }
>
> /*
> @@ -1793,15 +1834,19 @@ static int xe_oa_mmap(struct file *file, struct vm_area_struct *vma)
> * VM_MAYSHARE, don't copy the vma on fork
> */
> if (vma->vm_flags & (VM_WRITE | VM_EXEC | VM_SHARED | VM_MAYSHARE)) {
> - drm_dbg(&stream->oa->xe->drm, "mmap must be read only\n");
> - return -EINVAL;
> + drm_dbg(&xe->drm, "mmap must be read only\n");
> + ret = -EINVAL;
> + goto exit;
> }
> vm_flags_mod(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP | VM_DONTCOPY,
> VM_MAYWRITE | VM_MAYEXEC);
>
> vma->vm_pgoff = drm_vma_node_start(&bo->ttm.base.vma_node);
>
> - return drm_gem_mmap_obj(&bo->ttm.base, xe_bo_size(bo), vma);
> + ret = drm_gem_mmap_obj(&bo->ttm.base, xe_bo_size(bo), vma);
> +exit:
> + drm_dev_exit(idx);
> + return ret;
> }
>
> static const struct file_operations xe_oa_fops = {
> diff --git a/drivers/gpu/drm/xe/xe_observation.c b/drivers/gpu/drm/xe/xe_observation.c
> index 39e05b9131a7..9e23ade51dc1 100644
> --- a/drivers/gpu/drm/xe/xe_observation.c
> +++ b/drivers/gpu/drm/xe/xe_observation.c
> @@ -7,6 +7,7 @@
> #include <linux/perf_event.h>
> #include <linux/sysctl.h>
>
> +#include <drm/drm_drv.h>
> #include <uapi/drm/xe_drm.h>
>
> #include "xe_eu_stall.h"
> @@ -35,8 +36,8 @@ int xe_observation_paranoid_check(void)
> return perf_allow_cpu();
> }
>
> -static int xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg,
> - struct drm_file *file)
> +static int __xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg,
> + struct drm_file *file)
> {
> switch (arg->observation_op) {
> case DRM_XE_OBSERVATION_OP_STREAM_OPEN:
> @@ -50,6 +51,21 @@ static int xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *
> }
> }
>
> +static int xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg,
> + struct drm_file *file)
> +{
> + int idx, ret;
> +
> + /* Hold off drm_dev_unplug() while the OA ioctls run */
> + if (!drm_dev_enter(dev, &idx))
> + return -ENODEV;
> +
> + ret = __xe_oa_ioctl(dev, arg, file);
> +
> + drm_dev_exit(idx);
> + return ret;
> +}
> +
> static int xe_eu_stall_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg,
> struct drm_file *file)
> {
--
Shekhar Chauhan
Linux Graphics Software Engineer
Linux Platform Enabling
Intel Corporation
next prev parent reply other threads:[~2026-11-20 7:25 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 0:27 [PATCH v8 0/8] Cleanup OA stream on PCI unbind/remove Umesh Nerlige Ramappa
2026-11-19 0:27 ` [PATCH v8 1/8] drm/xe/oa: Ignore a HW tail pointer outside the OA buffer Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-20 7:21 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 2/8] drm/xe/oa: Use the drm device address_space for the OA stream fd Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-20 7:23 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 3/8] drm/xe/oa: Wait for in-flight OA batches before stream HW teardown Umesh Nerlige Ramappa
2026-11-20 7:24 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 4/8] drm/xe: Reject vm_ops->access on BO mappings of an unplugged device Umesh Nerlige Ramappa
2026-11-19 0:39 ` sashiko-bot
2026-11-19 22:36 ` Matthew Brost
2026-11-19 0:27 ` [PATCH v8 5/8] drm/xe/oa: Fence OA ioctls and mmap against device unplug and wedging Umesh Nerlige Ramappa
2026-11-20 7:25 ` Shekhar Chauhan [this message]
2026-11-19 0:27 ` [PATCH v8 6/8] drm/xe/oa: Take the runtime PM reference in xe_oa_destroy_locked() Umesh Nerlige Ramappa
2026-11-20 7:27 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 7/8] drm/xe/oa: Free the OA buffer and the stream after the stream destroy Umesh Nerlige Ramappa
2026-11-19 0:48 ` sashiko-bot
2026-11-20 7:29 ` Shekhar Chauhan
2026-11-19 0:27 ` [PATCH v8 8/8] drm/xe/oa: Quiesce OA streams on driver unbind and PCI reset Umesh Nerlige Ramappa
2026-11-19 1:11 ` sashiko-bot
2026-11-20 7:31 ` Shekhar Chauhan
2026-11-19 6:05 ` ✓ CI.KUnit: success for Cleanup OA stream on PCI unbind/remove (rev8) Patchwork
2026-11-19 6:46 ` ✓ Xe.CI.BAT: " Patchwork
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=82c8bbc5-bdb3-44e5-bf3c-c1906e666016@intel.com \
--to=shekhar.chauhan@intel.com \
--cc=harish.chegondi@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=matthew.brost@intel.com \
--cc=umesh.nerlige.ramappa@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox