[PATCH 00/20] Intel Xe GPU Debug Support (eudebug) v11

[PATCH 00/20] Intel Xe GPU Debug Support (eudebug) v11

[Mika Kuoppala]

Hi,

This is the v11 patch series for Intel Xe GPU debug support (eudebug).

This is eudebug core only. The EU page fault block (v10 patches
20-27) is not part of current series and will be sent when we have
reached agreement about this series.

One TTM specific core patch is first in the series.

Series should address all but one (see Open questions) sashiko
findings on v10:
https://sashiko.dev/#/patchset/20260903145952.848051-1-mika.kuoppala%40linux.intel.com

Additionally we did multiple local Sashiko runs with claude-opus-5.5 and
gemini-4-argon to get further comments on reworks. All non code
change issues were addressed by more documentation and comments.

We addressed over 170 review findings. Most came from sashiko of
which 13 of them high severity and about 80 medium.

Claude Opus 5.5 was used as an AI coding assistant.

# Major changes from v10

All per connection state is now guarded by a single mutex.
It replaces multiple smaller locks and eliminates all use
of READ_ONCE/WRITE_ONCE access. This addresses most of sashiko
UAF concerns.

Discovery no longer runs on a workqueue. The connect ioctl
takes the target's ioctl_lock for write and pushes
all discovery events into the kfifo before returing.
This avoids the issues discussed in patch 4 in previous
series.

eudebug is now mutually exclusive with OA in addition to SR-IOV.
These interfere with eachothers in HW.

# Open questions

In patch 14, the userptr vm access path still resolves the
addresses through a task_struct's current mm rather than the mm
at bind time. See the XXX FIXME comment in the patch.

# Supported HW

The series has been test with IGT on the following platforms:

- Lunarlake (LNL)
- Battlemage (BMG)
- Pantherlake (PTL)

The code for this submission can be found at:
https://gitlab.freedesktop.org/miku/kernel/-/tree/eudebug-v11

Tests:
https://gitlab.freedesktop.org/DominikPiatkowski-Intel/igt-gpu-tools/-/tree/eudebug-dev-next

Userspace:
https://github.com/intel/compute-runtime/
(behind compile-time default disabled flags
NEO_ENABLE_XE_EU_DEBUG_SUPPORT and NEO_USE_XE_EU_DEBUG_EXP_UPSTREAM)

GDB: https://sourceware.org/pipermail/gdb-patches/2026-August/229340.html

This series continues from the following previous submissions:
v1: https://lists.freedesktop.org/archives/intel-xe/2024-July/043605.html
v2: https://lists.freedesktop.org/archives/intel-xe/2024-October/052260.html
v3: https://lists.freedesktop.org/archives/intel-xe/2024-December/061476.html
v4: https://lists.freedesktop.org/archives/intel-xe/2025-August/091645.html
v5: https://lists.freedesktop.org/archives/intel-xe/2025-October/097859.html
v6: https://lists.freedesktop.org/archives/intel-xe/2025-December/106405.html
v7: https://lists.freedesktop.org/archives/intel-xe/2026-February/115750.html
v8: https://lists.freedesktop.org/archives/intel-xe/2026-April/124147.html
v9: https://lists.freedesktop.org/archives/intel-xe/2026-June/128834.html
v10: https://lists.freedesktop.org/archives/intel-xe/2026-September/139020.html


Christoph Manszewski (5):
  drm/xe: Implement SR-IOV and eudebug exclusivity
  drm/xe: Introduce ADD_DEBUG_DATA and REMOVE_DEBUG_DATA vm bind ops
  drm/xe/eudebug: Introduce vm bind and vm bind debug data events
  drm/xe/eudebug_test: Introduce eudebug live tests
  drm/xe: Add xe_client_debugfs and introduce debug_data file

Dominik Grzegorzek (5):
  drm/xe: Add EUDEBUG_ENABLE exec queue property
  drm/xe/eudebug: Introduce exec_queue events
  drm/xe/eudebug: Add hw enablement
  drm/xe/eudebug: Introduce EU control interface
  drm/xe/eudebug: Introduce per device attention scan worker

Mika Kuoppala (10):
  drm/ttm: Add ttm_bo_access_locked()
  drm/xe/eudebug: Introduce eudebug interface
  drm/xe/eudebug: Add documentation
  drm/xe/eudebug: Add connection establishment documentation
  drm/xe/eudebug: Introduce discovery for resources
  drm/xe/eudebug: Mark guc contexts as debuggable
  drm/xe/eudebug: Add ufence events with acks
  drm/xe/eudebug: Add vm open/pread/pwrite
  drm/xe/eudebug: Add userptr vm pread/pwrite
  drm/xe/eudebug: Track debuggable exec queues

 .../ABI/testing/sysfs-driver-intel-xe-eudebug |   61 +
 Documentation/gpu/driver-uapi.rst             |    2 +
 Documentation/gpu/xe/index.rst                |    1 +
 Documentation/gpu/xe/xe_eudebug.rst           |  100 +
 .../userspace-api/ioctl/ioctl-number.rst      |    1 +
 MAINTAINERS                                   |    2 +
 drivers/gpu/drm/ttm/ttm_bo_vm.c               |   61 +-
 drivers/gpu/drm/xe/Kconfig                    |   10 +
 drivers/gpu/drm/xe/Makefile                   |    7 +-
 drivers/gpu/drm/xe/abi/guc_actions_abi.h      |    5 +
 drivers/gpu/drm/xe/abi/guc_klvs_abi.h         |    1 +
 drivers/gpu/drm/xe/regs/xe_engine_regs.h      |    1 +
 drivers/gpu/drm/xe/regs/xe_gt_regs.h          |   41 +
 drivers/gpu/drm/xe/tests/xe_eudebug_test.c    |  241 ++
 drivers/gpu/drm/xe/tests/xe_live_test_mod.c   |    5 +
 drivers/gpu/drm/xe/xe_client_debugfs.c        |  134 +
 drivers/gpu/drm/xe/xe_client_debugfs.h        |   17 +
 drivers/gpu/drm/xe/xe_debug_data.c            |  388 ++
 drivers/gpu/drm/xe/xe_debug_data.h            |   22 +
 drivers/gpu/drm/xe/xe_debug_data_types.h      |   26 +
 drivers/gpu/drm/xe/xe_device.c                |   36 +-
 drivers/gpu/drm/xe/xe_device.h                |   53 +
 drivers/gpu/drm/xe/xe_device_types.h          |   75 +
 drivers/gpu/drm/xe/xe_eudebug.c               | 3196 +++++++++++++++++
 drivers/gpu/drm/xe/xe_eudebug.h               |  114 +
 drivers/gpu/drm/xe/xe_eudebug_hw.c            |  806 +++++
 drivers/gpu/drm/xe/xe_eudebug_hw.h            |   30 +
 drivers/gpu/drm/xe/xe_eudebug_types.h         |  212 ++
 drivers/gpu/drm/xe/xe_eudebug_vm.c            |  662 ++++
 drivers/gpu/drm/xe/xe_eudebug_vm.h            |   13 +
 drivers/gpu/drm/xe/xe_exec_queue.c            |  114 +-
 drivers/gpu/drm/xe/xe_exec_queue.h            |    2 +
 drivers/gpu/drm/xe/xe_exec_queue_types.h      |   15 +
 drivers/gpu/drm/xe/xe_gt_debug.c              |  228 ++
 drivers/gpu/drm/xe/xe_gt_debug.h              |   31 +
 drivers/gpu/drm/xe/xe_guc.c                   |   15 +
 drivers/gpu/drm/xe/xe_guc.h                   |    6 +
 drivers/gpu/drm/xe/xe_guc_ads.c               |    5 +
 drivers/gpu/drm/xe/xe_guc_submit.c            |   34 +
 drivers/gpu/drm/xe/xe_hw_engine.h             |   14 +
 drivers/gpu/drm/xe/xe_oa.c                    |   61 +
 drivers/gpu/drm/xe/xe_oa.h                    |    1 +
 drivers/gpu/drm/xe/xe_reg_sr.c                |   19 +-
 drivers/gpu/drm/xe/xe_reg_sr.h                |    4 +-
 drivers/gpu/drm/xe/xe_reg_whitelist.c         |    2 +-
 drivers/gpu/drm/xe/xe_rtp.c                   |    2 +-
 drivers/gpu/drm/xe/xe_sync.c                  |   47 +-
 drivers/gpu/drm/xe/xe_sync.h                  |    3 +
 drivers/gpu/drm/xe/xe_sync_types.h            |   29 +-
 drivers/gpu/drm/xe/xe_userptr.c               |    4 +
 drivers/gpu/drm/xe/xe_userptr.h               |   34 +
 drivers/gpu/drm/xe/xe_vm.c                    |  245 +-
 drivers/gpu/drm/xe/xe_vm_types.h              |   31 +
 drivers/gpu/drm/xe/xe_wa_oob.rules            |    4 +
 include/drm/ttm/ttm_bo.h                      |    2 +
 include/uapi/drm/xe_drm.h                     |  133 +-
 include/uapi/drm/xe_drm_eudebug.h             |  523 +++
 57 files changed, 7870 insertions(+), 61 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
 create mode 100644 Documentation/gpu/xe/xe_eudebug.rst
 create mode 100644 drivers/gpu/drm/xe/tests/xe_eudebug_test.c
 create mode 100644 drivers/gpu/drm/xe/xe_client_debugfs.c
 create mode 100644 drivers/gpu/drm/xe/xe_client_debugfs.h
 create mode 100644 drivers/gpu/drm/xe/xe_debug_data.c
 create mode 100644 drivers/gpu/drm/xe/xe_debug_data.h
 create mode 100644 drivers/gpu/drm/xe/xe_debug_data_types.h
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_hw.c
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_hw.h
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_vm.c
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_vm.h
 create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c
 create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.h
 create mode 100644 include/uapi/drm/xe_drm_eudebug.h

-- 
2.53.0

[PATCH 01/20] drm/ttm: Add ttm_bo_access_locked()

[Mika Kuoppala]

ttm_bo_access() reserves the buffer object itself, so a caller that
already holds the reservation cannot use it. Split the body out as
ttm_bo_access_locked(), which asserts the reservation is held, and leave
ttm_bo_access() as reserve, access, unreserve.

The first user, eudebug vm access, has to wait for the move fences and
access the object under one reservation.

Cc: Felix Kuehling <Felix.Kuehling@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Michel Dänzer <michel.daenzer@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 drivers/gpu/drm/ttm/ttm_bo_vm.c | 61 +++++++++++++++++++++++----------
 include/drm/ttm/ttm_bo.h        |  2 ++
 2 files changed, 44 insertions(+), 19 deletions(-)

diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 872bf444b1f0..9a252bce9091 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -405,6 +405,47 @@ static int ttm_bo_vm_access_kmap(struct ttm_buffer_object *bo,
 	return len;
 }
 
+/**
+ * ttm_bo_access_locked - Helper to access a reserved buffer object
+ *
+ * @bo: ttm buffer object, reserved by the caller
+ * @offset: access offset into buffer object
+ * @buf: pointer to caller memory to read into or write from
+ * @len: length of access
+ * @write: write access
+ *
+ * Same as ttm_bo_access(), for a caller that already holds the reservation,
+ * for instance to wait for fences before the access without the placement
+ * changing in between.
+ *
+ * Returns:
+ * @len if successful, negative error code on failure.
+ */
+int ttm_bo_access_locked(struct ttm_buffer_object *bo, unsigned long offset,
+			 void *buf, int len, int write)
+{
+	dma_resv_assert_held(bo->base.resv);
+
+	if (len < 1 || (offset + len) > bo->base.size)
+		return -EIO;
+
+	if (!bo->resource)
+		return -ENODATA;
+
+	switch (bo->resource->mem_type) {
+	case TTM_PL_SYSTEM:
+		fallthrough;
+	case TTM_PL_TT:
+		return ttm_bo_vm_access_kmap(bo, offset, buf, len, write);
+	default:
+		if (bo->bdev->funcs->access_memory)
+			return bo->bdev->funcs->access_memory
+				(bo, offset, buf, len, write);
+		return -EIO;
+	}
+}
+EXPORT_SYMBOL(ttm_bo_access_locked);
+
 /**
  * ttm_bo_access - Helper to access a buffer object
  *
@@ -433,26 +474,8 @@ int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset,
 	if (ret)
 		return ret;
 
-	if (!bo->resource) {
-		ret = -ENODATA;
-		goto unlock;
-	}
-
-	switch (bo->resource->mem_type) {
-	case TTM_PL_SYSTEM:
-		fallthrough;
-	case TTM_PL_TT:
-		ret = ttm_bo_vm_access_kmap(bo, offset, buf, len, write);
-		break;
-	default:
-		if (bo->bdev->funcs->access_memory)
-			ret = bo->bdev->funcs->access_memory
-				(bo, offset, buf, len, write);
-		else
-			ret = -EIO;
-	}
+	ret = ttm_bo_access_locked(bo, offset, buf, len, write);
 
-unlock:
 	ttm_bo_unreserve(bo);
 
 	return ret;
diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h
index 32791c4db2a9..3b08baa33ffd 100644
--- a/include/drm/ttm/ttm_bo.h
+++ b/include/drm/ttm/ttm_bo.h
@@ -441,6 +441,8 @@ s64 ttm_bo_evict_cgroup(struct ttm_device *bdev,
 			struct dmem_cgroup_pool_state *limit_pool,
 			s64 target_bytes,
 			struct ttm_operation_ctx *ctx);
+int ttm_bo_access_locked(struct ttm_buffer_object *bo, unsigned long offset,
+			 void *buf, int len, int write);
 int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset,
 		  void *buf, int len, int write);
 vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo,
-- 
2.53.0

[PATCH 02/20] drm/xe/eudebug: Introduce eudebug interface

[Mika Kuoppala]

Add the eudebug interface to the Xe driver, enabling
user-space debuggers (e.g., GDB) to track and interact with GPU resources
of a DRM client. Debuggers can inspect or modify these resources,
for example, to locate ISA/ELF sections and install breakpoints in a
shader's instruction stream.

A debugger opens a connection to the Xe driver via a DRM ioctl, specifying
the target DRM client's file descriptor. This returns an anonymous file
descriptor for the connection, which can be used to listen for resource
creation/destruction events. The same file descriptor can also be used to
receive hardware state change events and control execution flow by
interrupting EU threads on the GPU (in follow-up patches).

Introduce the eudebug connection and event queuing,
adding VM create/destroy events as a baseline. Additional events and
hardware control for full debugger operation are needed and will be
introduced in follow-up patches.

The resource tracking components are inspired by Maciej Patelczyk's work on
resource handling for i915. Chris Wilson suggested a two-way mapping
approach, which simplifies using the resource map as definitive
bookkeeping for resources relayed to the debugger during the discovery
phase (in a follow-up patch).

v2: - Kconfig support (Matthew)
    - ptraced access control (Lucas)
    - pass expected event length to user (Zbigniew)
    - only track long running VMs
    - checkpatch (Tilak)
    - include order (Andrzej)
    - 32bit fixes (Andrzej)
    - cleaner get_task_struct
    - remove xa_array and use clients.list for tracking (Mika)
v3: - adapt to removal of clients.lock (Mika)
    - create_event cleanup (Christoph)
v4: - add proper header guards (Christoph)
    - better read_event fault handling (Christoph, Mika)
    - simplify attach (Mika)
    - connect using target file descriptors
    - avoid event->seqno after queue as it can UAF (Mika)
    - use drmm for eudebug_fini (Maciej)
    - squash dynamic enable
v6: - drm->authenticated is overzealous for render (Mika)
v7: - struct member documentation (Mika)
    - enforce seqno mbz (Mika)
v8: - head->seqno fix (Mika)
    - resource alloc and removal cleanup (Mika)
    - s/wait_interruptible_timeout/wait_timeout (Mika)
    - use fd_install in connect (Mika)
v9: - fix xef vs debugger race (Sashiko)
    - assign d->xe early (Sashiko)
    - simplify locking (Mika)
    - fix pending->len outside lock (Sashiko)
    - remove version from connect to address fd_install leak (Sashiko)
    - create vm by id to avoid use after free (Sashiko)
    - use xe_eudebug_detach on connection error path (Sashiko)
    - don't sleep on kzalloc as we can disconnect (Sashiko)
    - check if enabled on connection (Sashiko)
    - preallocated event fifo for lockless allocs (Sashiko, Mika)
v10:
    - wakeup after detach, take account of occupied (Sashiko)
    - GFP_ACCOUNT for fifo (Sashiko)
    - set O_CLOEXEC on connection fd (Sashiko)
    - docstrings (Maciej)
    - use guard and __kfree (Mika)
    - init_early (Maciej)
    - remove unused struct members
    - no need to send destroy if vm was not announced (Claude)
    - avoid dereference of xef on eu_print (Claude)
    - don't silently truncate target fd (Claude)
    - preallocate resource handles
    - zero as target fd (Claude)
    - hold drm_device (Claude)
v11: - fix eudebug=n build (Claude)
     - document seqno as timeline (Sashiko)
v12: - drain nothing on detach, say so in the uapi (Sashiko)
     - drop resource handles on detach to avoid uaf (Sashiko)
     - ioctl naming, sysfs abi (Claude)
v13: - create enable_eudebug last, so init cannot undo an enable
       (review-prompts)
v14: - one mutex for all debugger state, no lockless accesses (Mika)
v15: - __free() variables defined where they are assigned (Sashiko)
     - poll in one lock section, no EPOLLIN once detached (Sashiko)
     - connect in the uapi ioctl list (Sashiko)
     - copy the event out of @events.pending directly, no bounce
       allocation per read (Sashiko)

Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Cc: Andi Shyti <andi.shyti@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>
Cc: Andrzej Hajda <andrzej.hajda@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
---
 .../ABI/testing/sysfs-driver-intel-xe-eudebug |   36 +
 Documentation/gpu/driver-uapi.rst             |    2 +
 .../userspace-api/ioctl/ioctl-number.rst      |    1 +
 MAINTAINERS                                   |    2 +
 drivers/gpu/drm/xe/Kconfig                    |   10 +
 drivers/gpu/drm/xe/Makefile                   |    3 +
 drivers/gpu/drm/xe/xe_device.c                |    8 +
 drivers/gpu/drm/xe/xe_device_types.h          |   29 +
 drivers/gpu/drm/xe/xe_eudebug.c               | 1212 +++++++++++++++++
 drivers/gpu/drm/xe/xe_eudebug.h               |   68 +
 drivers/gpu/drm/xe/xe_eudebug_types.h         |  168 +++
 drivers/gpu/drm/xe/xe_vm.c                    |   15 +-
 include/uapi/drm/xe_drm.h                     |   24 +
 include/uapi/drm/xe_drm_eudebug.h             |  131 ++
 14 files changed, 1707 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug.c
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug.h
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_types.h
 create mode 100644 include/uapi/drm/xe_drm_eudebug.h

diff --git a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
new file mode 100644
index 000000000000..26aff5116b8a
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
@@ -0,0 +1,36 @@
+What:		/sys/bus/pci/drivers/xe/.../enable_eudebug
+Date:		August 2026
+KernelVersion:	7.4
+Contact:	intel-xe@lists.freedesktop.org
+Description:	RW. Controls whether the EU debugger interface is armed for
+		this device.
+
+		Reads back 1 when the interface is enabled and 0 when it is
+		not. Writing a boolean value ("0"/"1", "n"/"y", "off"/"on")
+		enables or disables it. The interface starts out disabled.
+
+		While disabled, DRM_IOCTL_XE_EUDEBUG_CONNECT fails with
+		-EOPNOTSUPP.
+
+		Writing 0 while any debugger connection is still attached
+		fails with -EBUSY.
+
+		Once enabled, connecting needs no privilege beyond access to
+		the render node: a client may attach to any DRM file it holds
+		a descriptor for, including its own. Any such client can
+		therefore keep the interface from being disabled for as long
+		as it keeps its connection open. Disabling does not revoke
+		existing connections; to disable, the processes holding them
+		have to close them or be terminated. A connection shows up as
+		an "anon_inode:[xe_eudebug]" entry under /proc/<pid>/fd.
+
+		Each connection allocates about 4 MiB of kernel memory for
+		its event queue, held until the connection is closed. The
+		number of connections is not limited beyond the file
+		descriptor limit; the memory is charged to the memory cgroup
+		of the process that connected.
+
+		This attribute is only present when the driver is built with
+		CONFIG_DRM_XE_EUDEBUG and the interface was set up at probe
+		time. If probe could not set it up, the attribute is absent
+		and the connect ioctl fails with -EOPNOTSUPP.
diff --git a/Documentation/gpu/driver-uapi.rst b/Documentation/gpu/driver-uapi.rst
index 627fc68c7a21..1fbcac949da9 100644
--- a/Documentation/gpu/driver-uapi.rst
+++ b/Documentation/gpu/driver-uapi.rst
@@ -30,6 +30,8 @@ drm/xe uAPI
 
 .. kernel-doc:: include/uapi/drm/xe_drm.h
 
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+
 drm/asahi uAPI
 ================
 
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index dbe462040999..8c2113628efe 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -261,6 +261,7 @@ Code  Seq#    Include File                                             Comments
 'i'   80-8F  linux/i8k.h
 'i'   90-9F  `linux/iio/*.h`                                           IIO
 'j'   00-3F  linux/joystick.h
+'j'   40-4F  uapi/drm/xe_drm_eudebug.h                                 Intel Xe EU debugger connection
 'k'   00-0F  linux/spi/spidev.h                                        conflict!
 'k'   00-05  video/kyro.h                                              conflict!
 'k'   10-17  linux/hsi/hsi_char.h                                      HSI character device
diff --git a/MAINTAINERS b/MAINTAINERS
index 1c3aec082b0a..ec06cbef5c4e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -13397,11 +13397,13 @@ Q:	http://patchwork.freedesktop.org/project/intel-xe/
 B:	https://gitlab.freedesktop.org/drm/xe/kernel/-/issues
 C:	irc://irc.oftc.net/xe
 T:	git https://gitlab.freedesktop.org/drm/xe/kernel.git
+F:	Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
 F:	Documentation/ABI/testing/sysfs-driver-intel-xe-hwmon
 F:	Documentation/gpu/xe/
 F:	drivers/gpu/drm/xe/
 F:	include/drm/intel/
 F:	include/uapi/drm/xe_drm.h
+F:	include/uapi/drm/xe_drm_eudebug.h
 
 INTEL ELKHART LAKE PSE I/O DRIVER
 M:	Raag Jadav <raag.jadav@intel.com>
diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
index 8e57473b6fb2..b703bcb17574 100644
--- a/drivers/gpu/drm/xe/Kconfig
+++ b/drivers/gpu/drm/xe/Kconfig
@@ -143,6 +143,16 @@ config DRM_XE_FORCE_PROBE
 
 	  Use "!*" to block the probe of the driver for all known devices.
 
+config DRM_XE_EUDEBUG
+	bool "Enable gdb debugger support (eudebug)"
+	depends on DRM_XE
+	default n
+	help
+	  Choose this option if you want to add support for a debugger (gdb)
+	  to attach to a process using Xe and debug its gpu/gpgpu programs.
+	  With debugger support, Xe provides an interface for a debugger to
+	  track, inspect and modify the resources of that process.
+
 menu "drm/Xe Debugging"
 depends on DRM_XE
 depends on EXPERT
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index 0392ba58dd65..d42620e10c26 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -163,6 +163,9 @@ xe-$(CONFIG_I2C)	+= xe_i2c.o \
 xe-$(CONFIG_DRM_XE_GPUSVM) += xe_svm.o
 xe-$(CONFIG_DRM_GPUSVM) += xe_userptr.o
 
+# debugging shaders with gdb (eudebug) support
+xe-$(CONFIG_DRM_XE_EUDEBUG) += xe_eudebug.o
+
 # graphics hardware monitoring (HWMON) support
 xe-$(CONFIG_HWMON) += xe_hwmon.o
 
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 273e2c45fea9..ed28abc4f37f 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -34,6 +34,7 @@
 #include "xe_dma_buf.h"
 #include "xe_drm_client.h"
 #include "xe_drv.h"
+#include "xe_eudebug.h"
 #include "xe_exec.h"
 #include "xe_exec_queue.h"
 #include "xe_force_wake.h"
@@ -181,6 +182,8 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file)
 
 	guard(xe_pm_runtime)(xe);
 
+	xe_eudebug_file_close(xef);
+
 	/*
 	 * No need for exec_queue.lock here as there is no contention for it
 	 * when FD is closing as IOCTLs presumably can't be modifying the
@@ -231,6 +234,7 @@ static const struct drm_ioctl_desc xe_ioctls[] = {
 			  DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(XE_VM_GET_PROPERTY, xe_vm_get_property_ioctl,
 			  DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(XE_EUDEBUG_CONNECT, xe_eudebug_connect_ioctl, DRM_RENDER_ALLOW),
 };
 
 static long xe_drm_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
@@ -1064,6 +1068,8 @@ int xe_device_probe(struct xe_device *xe)
 	if (err)
 		return err;
 
+	xe_eudebug_init_early(xe);
+
 	err = drm_dev_register(&xe->drm, 0);
 	if (err)
 		return err;
@@ -1101,6 +1107,8 @@ int xe_device_probe(struct xe_device *xe)
 	if (err)
 		goto err_unregister_display;
 
+	xe_eudebug_init(xe);
+
 	detect_preproduction_hw(xe);
 
 	err = devm_add_action_or_reset(xe->drm.dev, xe_device_sanitize, xe);
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index d99881c6e1d5..000bf91bc5f0 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -15,6 +15,7 @@
 #include "xe_cper_types.h"
 #include "xe_devcoredump_types.h"
 #include "xe_drm_ras_types.h"
+#include "xe_eudebug_types.h"
 #include "xe_heci_gsc.h"
 #include "xe_late_bind_fw_types.h"
 #include "xe_oa_types.h"
@@ -621,6 +622,26 @@ struct xe_device {
 	atomic_t g2g_test_count;
 #endif
 
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	/** @eudebug: eudebug globals for the device */
+	struct {
+		/** @eudebug.session_count: session counter to track connections */
+		u64 session_count;
+
+		/**
+		 * @eudebug.cap_state: eudebug capability state, protected by
+		 * @eudebug.lock
+		 */
+		enum xe_eudebug_cap_state cap_state;
+
+		/** @eudebug.attached: number of debuggers currently attached */
+		unsigned int attached;
+
+		/** @eudebug.lock: protects the eudebug state of this device */
+		struct mutex lock;
+	} eudebug;
+#endif
+
 	/* private: */
 
 #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY)
@@ -708,6 +729,14 @@ struct xe_file {
 		/** @mmio_gem.pci_barrier: MMIO GEM object for PCI barrier mmap. */
 		struct xe_mmio_gem *pci_barrier;
 	} mmio_gem;
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	/** @eudebug: struct to hold eudebug connection specifics */
+	struct {
+		/** @eudebug.debugger: the debugger connection into this xe_file */
+		struct xe_eudebug *debugger;
+	} eudebug;
+#endif
 };
 
 #endif
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
new file mode 100644
index 000000000000..0a57d99fc5e8
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -0,0 +1,1212 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/cleanup.h>
+#include <linux/poll.h>
+#include <linux/uaccess.h>
+
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
+#include <uapi/drm/xe_drm.h>
+
+#include "xe_assert.h"
+#include "xe_device.h"
+#include "xe_eudebug.h"
+#include "xe_eudebug_types.h"
+#include "xe_macros.h"
+#include "xe_vm.h"
+
+#define cast_event(T, event) container_of((event), typeof(*(T)), base)
+
+static const struct rhashtable_params rhash_res = {
+	.head_offset = offsetof(struct xe_eudebug_handle, rh_head),
+	.key_len = sizeof_field(struct xe_eudebug_handle, key),
+	.key_offset = offsetof(struct xe_eudebug_handle, key),
+	.automatic_shrinking = true,
+};
+
+static struct xe_eudebug_resource *
+resource_from_type(struct xe_eudebug *d, int t)
+{
+	return &d->target.res[t];
+}
+
+static int
+xe_eudebug_resources_init(struct xe_eudebug *d)
+{
+	int ret;
+	int i;
+
+	ret = 0;
+	for (i = 0; i < XE_EUDEBUG_RES_TYPE_COUNT; i++) {
+		struct xe_eudebug_resource *r = resource_from_type(d, i);
+
+		xa_init_flags(&r->xa, XA_FLAGS_ALLOC1);
+		ret = rhashtable_init(&r->rh, &rhash_res);
+
+		if (ret) {
+			xa_destroy(&r->xa);
+			break;
+		}
+	}
+
+	if (!ret)
+		return 0;
+
+	while (i--) {
+		struct xe_eudebug_resource *r = resource_from_type(d, i);
+
+		xa_destroy(&r->xa);
+		rhashtable_destroy(&r->rh);
+	}
+
+	return ret;
+}
+
+/*
+ * Drop every handle, keeping the maps themselves usable.
+ *
+ * The maps hold raw pointers to the target's resources, and the reference
+ * that keeps those alive belongs to the target, not to us. Once the target
+ * is detached it is free to destroy them, so the handles go in the same
+ * critical section that detaches us. A lookup then either sees a live
+ * resource or finds us detached.
+ */
+static void
+xe_eudebug_resources_clear(struct xe_eudebug *d)
+{
+	unsigned long j;
+	int err;
+	int i;
+
+	lockdep_assert_held(&d->lock);
+
+	for (i = 0; i < XE_EUDEBUG_RES_TYPE_COUNT; i++) {
+		struct xe_eudebug_resource *r = resource_from_type(d, i);
+		struct xe_eudebug_handle *h;
+
+		xa_for_each(&r->xa, j, h) {
+			struct xe_eudebug_handle *t;
+
+			err = rhashtable_remove_fast(&r->rh,
+						     &h->rh_head,
+						     rhash_res);
+			xe_eudebug_assert(d, !err);
+			t = xa_erase(&r->xa, h->id);
+			if (XE_WARN_ON(!t))
+				continue;
+
+			xe_eudebug_assert(d, t == h);
+			kfree(t);
+		}
+	}
+}
+
+static void
+xe_eudebug_resources_destroy(struct xe_eudebug *d)
+{
+	int i;
+
+	scoped_guard(mutex, &d->lock)
+		xe_eudebug_resources_clear(d);
+
+	for (i = 0; i < XE_EUDEBUG_RES_TYPE_COUNT; i++) {
+		struct xe_eudebug_resource *r = resource_from_type(d, i);
+
+		rhashtable_destroy(&r->rh);
+		xe_eudebug_assert(d, xa_empty(&r->xa));
+		xa_destroy(&r->xa);
+	}
+}
+
+static bool __xe_eudebug_is_detached(struct xe_eudebug *d)
+{
+	lockdep_assert_held(&d->lock);
+
+	return !d->target.xef;
+}
+
+static bool xe_eudebug_is_detached(struct xe_eudebug *d)
+{
+	guard(mutex)(&d->lock);
+
+	return __xe_eudebug_is_detached(d);
+}
+
+static void xe_eudebug_free(struct kref *ref)
+{
+	struct xe_eudebug *d = container_of(ref, typeof(*d), ref);
+
+	xe_assert(d->xe, xe_eudebug_is_detached(d));
+
+	xe_eudebug_resources_destroy(d);
+
+	kvfree(d->events.fifo_buf);
+	kvfree(d->events.staging);
+	kvfree(d->events.pending);
+	mutex_destroy(&d->lock);
+	kfree(d);
+}
+
+static void xe_eudebug_put(struct xe_eudebug *d)
+{
+	kref_put(&d->ref, xe_eudebug_free);
+}
+
+static bool xe_eudebug_detach(struct xe_eudebug *d,
+			      const int err)
+{
+	struct xe_file *target = NULL;
+
+	XE_WARN_ON(err > 0);
+
+	mutex_lock(&d->xe->eudebug.lock);
+
+	/*
+	 * The target may start tearing its resources down the moment it has
+	 * been detached, and the fd close path does exactly that. Drop the
+	 * handles in the same critical section, so nothing can be resolved
+	 * afterwards.
+	 */
+	scoped_guard(mutex, &d->lock) {
+		if (d->target.xef) {
+			target = d->target.xef;
+			d->target.err = err;
+			d->target.xef = NULL;
+			xe_eudebug_resources_clear(d);
+		}
+	}
+
+	if (target) {
+		XE_WARN_ON(target->eudebug.debugger != d);
+		target->eudebug.debugger = NULL;
+
+		d->xe->eudebug.attached--;
+
+		eu_dbg(d, "session %llu detached with %d", d->session, err);
+	}
+	mutex_unlock(&d->xe->eudebug.lock);
+
+	wake_up_all(&d->events.write_done);
+
+	if (target) {
+		xe_eudebug_put(d);
+		xe_file_put(target);
+	}
+
+	return !!target;
+}
+
+#define xe_eudebug_disconnect(_d, _err) ({ \
+	struct xe_eudebug *__disc_d = (_d); \
+	int __disc_err = (_err); \
+	if (xe_eudebug_detach(__disc_d, __disc_err)) \
+		eu_dbg(__disc_d, "Session disconnected, err = %d (%s:%d)", \
+		       __disc_err, __func__, __LINE__); \
+})
+
+static int event_fifo_pending(struct xe_eudebug *d,
+			      struct drm_xe_eudebug_event **pending)
+{
+	struct drm_xe_eudebug_event *e = d->events.pending;
+	unsigned int len, copied;
+
+	lockdep_assert_held(&d->lock);
+
+	*pending = NULL;
+
+	if (__xe_eudebug_is_detached(d))
+		return -ENOTCONN;
+
+	if (d->events.pending_occupied) {
+		*pending = e;
+		return 0;
+	}
+
+	if (kfifo_out_peek(&d->events.fifo, e, sizeof(*e)) < sizeof(*e))
+		return -ENOENT;
+
+	len = e->len;
+	if (len <= sizeof(*e) || len > XE_EUDEBUG_EVENT_MAX_SIZE) {
+		eu_dbg(d, "bad event len %u", len);
+		return -EIO;
+	}
+
+	copied = kfifo_out(&d->events.fifo, e, len);
+	if (copied != len) {
+		eu_dbg(d, "fifo inconsistency");
+		return -EIO;
+	}
+
+	d->events.pending_occupied = true;
+	*pending = e;
+
+	return 0;
+}
+
+/* Read and written under xe->eudebug.lock, see @eudebug.cap_state */
+static void set_cap_state(struct xe_device *xe, enum xe_eudebug_cap_state state)
+{
+	lockdep_assert_held(&xe->eudebug.lock);
+
+	xe->eudebug.cap_state = state;
+}
+
+static bool __xe_eudebug_is_enabled(struct xe_device *xe)
+{
+	lockdep_assert_held(&xe->eudebug.lock);
+
+	return xe->eudebug.cap_state == XE_EUDEBUG_CAP_ENABLED;
+}
+
+bool xe_eudebug_is_enabled(struct xe_device *xe)
+{
+	guard(mutex)(&xe->eudebug.lock);
+
+	return __xe_eudebug_is_enabled(xe);
+}
+
+static struct xe_eudebug *
+xe_eudebug_get(struct xe_file *xef)
+{
+	struct xe_device *xe = xef->xe;
+	struct xe_eudebug *d;
+
+	mutex_lock(&xe->eudebug.lock);
+	d = xef->eudebug.debugger;
+	if (d && !kref_get_unless_zero(&d->ref))
+		d = NULL;
+	mutex_unlock(&xe->eudebug.lock);
+
+	if (d && xe_eudebug_is_detached(d)) {
+		xe_eudebug_put(d);
+		d = NULL;
+	}
+
+	return d;
+}
+
+static int xe_eudebug_queue_event(struct xe_eudebug *d,
+				  struct drm_xe_eudebug_event *event)
+{
+	unsigned int copied;
+
+	lockdep_assert_held(&d->lock);
+
+	xe_eudebug_assert(d, event->len > sizeof(struct drm_xe_eudebug_event));
+	xe_eudebug_assert(d, event->type);
+	xe_eudebug_assert(d, event->type != DRM_XE_EUDEBUG_EVENT_READ);
+	xe_eudebug_assert(d, event->len <= XE_EUDEBUG_EVENT_MAX_SIZE);
+	xe_eudebug_assert(d, event == d->events.staging);
+
+	if (__xe_eudebug_is_detached(d))
+		return -ENOTCONN;
+
+	if (kfifo_avail(&d->events.fifo) < event->len)
+		return -ENOSPC;
+
+	copied = kfifo_in(&d->events.fifo, event, event->len);
+	if (XE_WARN_ON(copied != event->len))
+		return -EIO;
+
+	wake_up_all(&d->events.write_done);
+
+	return 0;
+}
+
+static struct xe_eudebug_handle *
+alloc_handle(const u64 key)
+{
+	struct xe_eudebug_handle *h;
+
+	h = kzalloc_obj(*h, GFP_KERNEL);
+	if (!h)
+		return NULL;
+
+	h->key = key;
+
+	return h;
+}
+
+static struct xe_eudebug_handle *
+__find_handle(struct xe_eudebug_resource *r,
+	      const u64 key)
+{
+	struct xe_eudebug_handle *h;
+
+	h = rhashtable_lookup_fast(&r->rh,
+				   &key,
+				   rhash_res);
+	return h;
+}
+
+/*
+ * Adding a handle is split around d->lock: handle_prepare() allocates the
+ * handle and reserves its id, __handle_insert() publishes it and stamps
+ * @seqno under the lock, and handle_abort() undoes a prepare that did not
+ * get inserted. _xe_eudebug_add_handle() does all three. A caller with
+ * more to do in the same critical section calls them itself.
+ */
+static struct xe_eudebug_handle *
+handle_prepare(struct xe_eudebug *d, int type, void *p)
+{
+	struct xe_eudebug_resource *r;
+	struct xe_eudebug_handle *h;
+	int err;
+
+	if (XE_WARN_ON(!p))
+		return ERR_PTR(-EINVAL);
+
+	h = alloc_handle((uintptr_t)p);
+	if (!h)
+		return ERR_PTR(-ENOMEM);
+
+	r = resource_from_type(d, type);
+
+	/*
+	 * Reserve the id, and the nodes backing it, before taking the lock.
+	 * A reserved entry reads back as NULL, so nothing can observe the id
+	 * until __handle_insert() stores the handle, and that store is
+	 * guaranteed to find the slot already there and so never has to
+	 * allocate.
+	 *
+	 * Allocate cyclically so a freed id is not handed out again while
+	 * the DESTROY event that freed it may still be on its way to the
+	 * fifo. A debugger applying events in read order would otherwise see
+	 * a CREATE for an id it still considers live.
+	 */
+	err = xa_alloc_cyclic(&r->xa, &h->id, XA_ZERO_ENTRY, xa_limit_31b,
+			      &r->next_id, GFP_KERNEL);
+	if (err < 0) {
+		kfree(h);
+		return ERR_PTR(err);
+	}
+
+	return h;
+}
+
+static void handle_abort(struct xe_eudebug *d, int type,
+			 struct xe_eudebug_handle *h)
+{
+	xa_erase(&resource_from_type(d, type)->xa, h->id);
+	kfree(h);
+}
+
+/* Return: the handle id, or a negative error after which the caller aborts */
+static int __handle_insert(struct xe_eudebug *d, int type,
+			   struct xe_eudebug_handle *h, u64 *seqno)
+{
+	struct xe_eudebug_resource *r = resource_from_type(d, type);
+	int err;
+
+	lockdep_assert_held(&d->lock);
+
+	if (__find_handle(r, h->key))
+		return -EEXIST;
+
+	/*
+	 * A detach marks us detached and clears the map in one critical
+	 * section, so a handle either goes in before it and is dropped by it,
+	 * or not at all.
+	 */
+	if (__xe_eudebug_is_detached(d))
+		return -ENOTCONN;
+
+	err = rhashtable_insert_fast(&r->rh, &h->rh_head, rhash_res);
+	if (err)
+		return err;
+
+	xa_store(&r->xa, h->id, h, GFP_ATOMIC);
+
+	/*
+	 * Stamped at generation, not at queueing, so concurrent producers can
+	 * take N and N+1 and then queue them the other way round. That is by
+	 * design: the number is the debugger's timeline, not the delivery
+	 * order. See drm_xe_eudebug_event.seqno.
+	 */
+	if (seqno)
+		*seqno = atomic64_inc_return(&d->events.seqno);
+
+	xe_eudebug_assert(d, h->id);
+
+	return h->id;
+}
+
+static int _xe_eudebug_add_handle(struct xe_eudebug *d,
+				  int type,
+				  void *p,
+				  u64 *seqno)
+{
+	struct xe_eudebug_handle *h;
+	int id;
+
+	h = handle_prepare(d, type, p);
+	if (IS_ERR(h))
+		return PTR_ERR(h);
+
+	scoped_guard(mutex, &d->lock)
+		id = __handle_insert(d, type, h, seqno);
+
+	if (id < 0)
+		handle_abort(d, type, h);
+
+	return id;
+}
+
+static int xe_eudebug_add_handle(struct xe_eudebug *d,
+				 int type,
+				 void *p,
+				 u64 *seqno)
+{
+	int ret;
+
+	ret = _xe_eudebug_add_handle(d, type, p, seqno);
+
+	eu_dbg(d, "handle type %d handle %p added: %d\n", type, p, ret);
+
+	return ret;
+}
+
+/* Return: the removed handle's id, or a negative error */
+static int __xe_eudebug_remove_handle(struct xe_eudebug *d, int type, void *p,
+				      u64 *seqno)
+{
+	const u64 key = (uintptr_t)p;
+	struct xe_eudebug_resource *r;
+	struct xe_eudebug_handle *h, *xa_h;
+	int ret;
+
+	lockdep_assert_held(&d->lock);
+
+	if (XE_WARN_ON(!key))
+		return -EINVAL;
+
+	r = resource_from_type(d, type);
+
+	h = __find_handle(r, key);
+	if (!h)
+		return -ENOENT;
+
+	xa_h = xa_load(&r->xa, h->id);
+	if (XE_WARN_ON(!xa_h || xa_h != h))
+		return -EIO;
+
+	ret = rhashtable_remove_fast(&r->rh,
+				     &h->rh_head,
+				     rhash_res);
+	if (XE_WARN_ON(ret))
+		return -EIO;
+
+	xa_h = xa_erase(&r->xa, h->id);
+	if (XE_WARN_ON(xa_h != h))
+		return -EIO;
+
+	ret = h->id;
+	if (seqno)
+		*seqno = atomic64_inc_return(&d->events.seqno);
+
+	kfree(h);
+
+	return ret;
+}
+
+static int _xe_eudebug_remove_handle(struct xe_eudebug *d, int type, void *p,
+				     u64 *seqno)
+{
+	guard(mutex)(&d->lock);
+
+	return __xe_eudebug_remove_handle(d, type, p, seqno);
+}
+
+static int xe_eudebug_remove_handle(struct xe_eudebug *d, int type, void *p,
+				    u64 *seqno)
+{
+	int ret;
+
+	ret = _xe_eudebug_remove_handle(d, type, p, seqno);
+
+	eu_dbg(d, "handle type %d handle %p removed: %d\n", type, p, ret);
+
+	return ret;
+}
+
+static struct drm_xe_eudebug_event *
+xe_eudebug_prepare_event(struct xe_eudebug *d, u16 type, u64 seqno, u16 flags,
+			 u32 len)
+{
+	const u16 known_flags =
+		DRM_XE_EUDEBUG_EVENT_CREATE |
+		DRM_XE_EUDEBUG_EVENT_DESTROY;
+	struct drm_xe_eudebug_event *event = d->events.staging;
+
+	lockdep_assert_held(&d->lock);
+
+	xe_eudebug_assert(d, type <= XE_EUDEBUG_MAX_EVENT_TYPE);
+	xe_eudebug_assert(d, !(~known_flags & flags));
+	xe_eudebug_assert(d, len > sizeof(*event));
+	xe_eudebug_assert(d, len <= XE_EUDEBUG_EVENT_MAX_SIZE);
+
+	memset(event, 0, len);
+
+	event->len = len;
+	event->type = type;
+	event->flags = flags;
+	event->seqno = seqno;
+
+	return event;
+}
+
+static int send_vm_event(struct xe_eudebug *d, u32 flags,
+			 const u64 vm_handle,
+			 const u64 seqno)
+{
+	struct drm_xe_eudebug_event *event;
+	struct drm_xe_eudebug_event_vm *e;
+	int err;
+
+	scoped_guard(mutex, &d->lock) {
+		event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_VM,
+						 seqno, flags, sizeof(*e));
+		e = cast_event(e, event);
+		e->vm_handle = vm_handle;
+
+		err = xe_eudebug_queue_event(d, event);
+	}
+
+	return err;
+}
+
+static int vm_create_event(struct xe_eudebug *d, struct xe_vm *vm)
+{
+	int vm_id;
+	u64 seqno;
+	int ret;
+
+	vm_id = xe_eudebug_add_handle(d, XE_EUDEBUG_RES_TYPE_VM, vm, &seqno);
+	if (vm_id < 0)
+		return vm_id;
+
+	ret = send_vm_event(d, DRM_XE_EUDEBUG_EVENT_CREATE, vm_id, seqno);
+	if (ret)
+		eu_dbg(d, "send_vm_event create error %d\n", ret);
+
+	return ret;
+}
+
+static int vm_destroy_event(struct xe_eudebug *d, struct xe_vm *vm)
+{
+	int vm_id;
+	u64 seqno;
+	int ret;
+
+	vm_id = xe_eudebug_remove_handle(d, XE_EUDEBUG_RES_TYPE_VM, vm, &seqno);
+	if (vm_id < 0)
+		return vm_id;
+
+	ret = send_vm_event(d, DRM_XE_EUDEBUG_EVENT_DESTROY, vm_id, seqno);
+	if (ret)
+		eu_dbg(d, "send_vm_event destroy error %d\n", ret);
+
+	return ret;
+}
+
+void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm)
+{
+	struct xe_eudebug *d;
+	int err;
+
+	/* Debuggable queues run on fault mode vms only, see exec_queue_set_eudebug() */
+	if (!xe_vm_in_fault_mode(vm))
+		return;
+
+	d = xe_eudebug_get(xef);
+	if (!d)
+		return;
+
+	err = vm_create_event(d, vm);
+	if (err)
+		xe_eudebug_disconnect(d, err);
+
+	xe_eudebug_put(d);
+}
+
+void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)
+{
+	struct xe_eudebug *d;
+	int err;
+
+	if (!xe_vm_in_fault_mode(vm))
+		return;
+
+	d = xe_eudebug_get(xef);
+	if (!d)
+		return;
+
+	/*
+	 * A vm we never handed out to the debugger needs no destroy event.
+	 * Only a genuine bookkeeping inconsistency should drop the session.
+	 */
+	err = vm_destroy_event(d, vm);
+	if (err && err != -ENOENT)
+		xe_eudebug_disconnect(d, err);
+
+	xe_eudebug_put(d);
+}
+
+static int add_debugger(struct xe_device *xe, struct xe_eudebug *d,
+			struct drm_file *target)
+{
+	struct xe_file *xef = target->driver_priv;
+
+	guard(mutex)(&xe->eudebug.lock);
+
+	if (!__xe_eudebug_is_enabled(xe))
+		return -EOPNOTSUPP;
+
+	if (xef->eudebug.debugger)
+		return -EBUSY;
+
+	scoped_guard(mutex, &d->lock)
+		d->target.xef = xe_file_get(xef);
+	d->target.pid = xef->pid;
+	kref_get(&d->ref);
+	xef->eudebug.debugger = d;
+
+	xe->eudebug.attached++;
+
+	do {
+		d->session = ++xe->eudebug.session_count;
+	} while (!d->session);
+
+	return 0;
+}
+
+static int
+xe_eudebug_attach(struct xe_device *xe, struct drm_file *parent_file,
+		  struct xe_eudebug *d, u64 target_fd)
+{
+	struct drm_file *drm_file;
+	struct xe_file *target_xef;
+	int ret;
+
+	if (XE_IOCTL_DBG(xe, target_fd > INT_MAX))
+		return -EBADF;
+
+	struct file *file __free(fput) = fget(target_fd);
+	if (XE_IOCTL_DBG(xe, !file))
+		return -EBADF;
+
+	drm_file = file->private_data;
+	if (XE_IOCTL_DBG(xe, !drm_file))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, parent_file->filp->f_op != file->f_op))
+		return -EINVAL;
+
+	target_xef = drm_file->driver_priv;
+	if (XE_IOCTL_DBG(xe, !target_xef))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, xe != target_xef->xe))
+		return -EINVAL;
+
+	ret = add_debugger(xe, d, drm_file);
+	if (XE_IOCTL_DBG(xe, ret))
+		return ret;
+
+	eu_dbg(d, "session %llu attached to %s", d->session,
+	       parent_file == drm_file ? "self" : "remote");
+
+	return 0;
+}
+
+static int xe_eudebug_release(struct inode *inode, struct file *file)
+{
+	struct xe_eudebug *d = file->private_data;
+	struct drm_device *drm = &d->xe->drm;
+
+	xe_eudebug_disconnect(d, 0);
+	xe_eudebug_put(d);
+
+	drm_dev_put(drm);
+
+	return 0;
+}
+
+static bool __event_fifo_has_events(struct xe_eudebug *d)
+{
+	lockdep_assert_held(&d->lock);
+
+	if (d->events.pending_occupied)
+		return true;
+
+	return kfifo_len(&d->events.fifo) > sizeof(struct drm_xe_eudebug_event);
+}
+
+static bool event_fifo_has_events(struct xe_eudebug *d)
+{
+	guard(mutex)(&d->lock);
+
+	/* Allow all waiters to proceed to check their state */
+	return __xe_eudebug_is_detached(d) || __event_fifo_has_events(d);
+}
+
+/*
+ * wait_event_*() evaluates its condition with the task already marked as
+ * sleeping, where the lock cannot be taken. wait_woken() lets us check
+ * the condition while running, without losing a wakeup in between.
+ *
+ * Return: as wait_event_interruptible_timeout()
+ */
+static long wait_for_events(struct xe_eudebug *d, long timeout)
+{
+	DEFINE_WAIT_FUNC(wait, woken_wake_function);
+
+	add_wait_queue(&d->events.write_done, &wait);
+	while (!event_fifo_has_events(d)) {
+		if (signal_pending(current)) {
+			timeout = -ERESTARTSYS;
+			break;
+		}
+
+		if (!timeout)
+			break;
+
+		timeout = wait_woken(&wait, TASK_INTERRUPTIBLE, timeout);
+	}
+	remove_wait_queue(&d->events.write_done, &wait);
+
+	return timeout;
+}
+
+static __poll_t xe_eudebug_poll(struct file *file, poll_table *wait)
+{
+	struct xe_eudebug * const d = file->private_data;
+	__poll_t ret = 0;
+
+	poll_wait(file, &d->events.write_done, wait);
+
+	guard(mutex)(&d->lock);
+
+	if (__xe_eudebug_is_detached(d)) {
+		/* Queued events went with the detach, READ_EVENT only fails */
+		ret = EPOLLHUP;
+		if (d->target.err)
+			ret |= EPOLLERR;
+	} else if (__event_fifo_has_events(d)) {
+		ret = EPOLLIN;
+	}
+
+	return ret;
+}
+
+static void xe_eudebug_reader_clear(struct xe_eudebug *d)
+{
+	if (!d)
+		return;
+
+	guard(mutex)(&d->lock);
+	d->events.reader_active = false;
+}
+
+DEFINE_FREE(reader_active, struct xe_eudebug *, xe_eudebug_reader_clear(_T))
+
+static long xe_eudebug_read_event(struct xe_eudebug *d,
+				  const u64 arg,
+				  const bool wait)
+{
+	struct xe_device *xe = d->xe;
+	struct drm_xe_eudebug_event __user * const user_orig =
+		u64_to_user_ptr(arg);
+	struct drm_xe_eudebug_event user_event;
+	struct drm_xe_eudebug_event *pending;
+	long ret = 0;
+	int pending_len = 0;
+	int fifo_ret;
+
+	if (XE_IOCTL_DBG(xe, copy_from_user(&user_event, user_orig, sizeof(user_event))))
+		return -EFAULT;
+
+	if (XE_IOCTL_DBG(xe, user_event.type != DRM_XE_EUDEBUG_EVENT_READ))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_event.len < sizeof(*user_orig)))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_event.flags))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_event.seqno))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_event.reserved))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, xe_eudebug_is_detached(d)))
+		return -ENOTCONN;
+
+	scoped_guard(mutex, &d->lock) {
+		if (d->events.reader_active)
+			return -EBUSY;
+
+		d->events.reader_active = true;
+	}
+
+	struct xe_eudebug *reader __free(reader_active) = d;
+
+	/* XXX: define wait time in connect arguments ? */
+	if (wait) {
+		ret = wait_for_events(d, secs_to_jiffies(5));
+
+		if (XE_IOCTL_DBG(xe, ret < 0))
+			return ret;
+	}
+
+	mutex_lock(&d->lock);
+	fifo_ret = event_fifo_pending(d, &pending);
+	if (fifo_ret == 0) {
+		if (user_event.len < pending->len) {
+			pending_len = pending->len;
+			ret = -EMSGSIZE;
+		} else if (!access_ok(user_orig, pending->len)) {
+			ret = -EFAULT;
+		} else {
+			eu_dbg(d, "event read: type=%u, flags=0x%x, seqno=%llu",
+			       pending->type, pending->flags, pending->seqno);
+			ret = 0;
+		}
+	} else if (fifo_ret == -ENOENT) {
+		ret = wait ? -ETIMEDOUT : -EAGAIN;
+	} else {
+		ret = fifo_ret;		/* -ENOTCONN or -EIO */
+	}
+	mutex_unlock(&d->lock);
+
+	/* disconnect takes xe->eudebug.lock, so do it only after dropping ours */
+	if (fifo_ret == -EIO)
+		xe_eudebug_disconnect(d, -EIO);
+
+	if (ret == -EMSGSIZE) {
+		if (XE_IOCTL_DBG(xe, put_user(pending_len, &user_orig->len)))
+			ret = -EFAULT;
+	}
+
+	/*
+	 * Copied without d->lock, so that faulting on the user pointer does not
+	 * happen under it. Nothing else touches @events.pending meanwhile: only
+	 * event_fifo_pending() writes it, only the reader marked by
+	 * @events.reader_active calls that, and the event stays occupied until
+	 * the copy has succeeded.
+	 */
+	if (!ret && __copy_to_user(user_orig, pending, pending->len))
+		ret = -EFAULT;
+
+	if (!ret) {
+		mutex_lock(&d->lock);
+		d->events.pending_occupied = false;
+		mutex_unlock(&d->lock);
+	}
+	if (ret)
+		eu_dbg(d, "event read=%ld", ret);
+
+	return ret;
+}
+
+/**
+ * xe_eudebug_ioctl - Issue a command to eudebug interface
+ *
+ * @file : eudebug file (returned from connect)
+ * @cmd : cmd
+ * @arg : arguments depending on cmd
+ *
+ * Issue a eudebug command
+ *
+ * Return: 0 on success, negative error code on failure. -ENOTTY for an
+ * unknown command.
+ */
+static long xe_eudebug_ioctl(struct file *file,
+			     unsigned int cmd,
+			     unsigned long arg)
+{
+	struct xe_eudebug * const d = file->private_data;
+	long ret;
+
+	switch (cmd) {
+	case DRM_XE_EUDEBUG_IOCTL_READ_EVENT:
+		ret = xe_eudebug_read_event(d, arg,
+					    !(file->f_flags & O_NONBLOCK));
+		break;
+	default:
+		ret = -ENOTTY;
+	}
+
+	return ret;
+}
+
+static const struct file_operations fops = {
+	.owner		= THIS_MODULE,
+	.release	= xe_eudebug_release,
+	.poll		= xe_eudebug_poll,
+	.unlocked_ioctl	= xe_eudebug_ioctl,
+	.compat_ioctl	= compat_ptr_ioctl,
+};
+
+/*
+ * Tell lockdep up front where the lock sits, so that a violation is
+ * reported on the first run rather than once both paths have been seen.
+ * Reclaim never takes it, so allocating under it is fine.
+ */
+static void xe_eudebug_lock_prime(struct xe_eudebug *d)
+{
+	if (!IS_ENABLED(CONFIG_LOCKDEP))
+		return;
+
+	mutex_lock(&d->lock);
+	fs_reclaim_acquire(GFP_KERNEL);
+	fs_reclaim_release(GFP_KERNEL);
+	mutex_unlock(&d->lock);
+}
+
+static int
+xe_eudebug_connect(struct xe_device *xe,
+		   struct drm_file *drm_file,
+		   struct drm_xe_eudebug_connect *param)
+{
+	const u64 known_open_flags = 0;
+	struct xe_eudebug *d;
+	struct file *file;
+	int fd, err;
+
+	if (XE_IOCTL_DBG(xe, param->extensions))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, param->flags & ~known_open_flags))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, param->reserved))
+		return -EINVAL;
+
+	if (!xe_eudebug_is_enabled(xe))
+		return -EOPNOTSUPP;
+
+	d = kzalloc_obj(*d, GFP_KERNEL | __GFP_ACCOUNT);
+	if (XE_IOCTL_DBG(xe, !d))
+		return -ENOMEM;
+
+	d->xe = xe;
+
+	kref_init(&d->ref);
+	mutex_init(&d->lock);
+	xe_eudebug_lock_prime(d);
+	init_waitqueue_head(&d->events.write_done);
+
+	err = xe_eudebug_resources_init(d);
+	if (XE_IOCTL_DBG(xe, err)) {
+		mutex_destroy(&d->lock);
+		kfree(d);
+		return err;
+	}
+
+	d->events.pending = kvzalloc(XE_EUDEBUG_EVENT_MAX_SIZE,
+				     GFP_KERNEL | __GFP_ACCOUNT);
+	if (!d->events.pending) {
+		err = -ENOMEM;
+		goto err_put;
+	}
+
+	d->events.staging = kvzalloc(XE_EUDEBUG_EVENT_MAX_SIZE,
+				     GFP_KERNEL | __GFP_ACCOUNT);
+	if (!d->events.staging) {
+		err = -ENOMEM;
+		goto err_put;
+	}
+
+	d->events.fifo_buf = kvmalloc(XE_EUDEBUG_FIFO_SIZE,
+				      GFP_KERNEL | __GFP_ACCOUNT);
+	if (!d->events.fifo_buf) {
+		err = -ENOMEM;
+		goto err_put;
+	}
+
+	err = kfifo_init(&d->events.fifo, d->events.fifo_buf, XE_EUDEBUG_FIFO_SIZE);
+	if (XE_IOCTL_DBG(xe, err))
+		goto err_put;
+
+	err = xe_eudebug_attach(xe, drm_file, d, param->fd);
+	if (XE_IOCTL_DBG(xe, err))
+		goto err_put;
+
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (fd < 0) {
+		err = fd;
+		goto err_detach;
+	}
+
+	file = anon_inode_getfile("[xe_eudebug]", &fops, d, 0);
+	if (IS_ERR(file)) {
+		err = PTR_ERR(file);
+		goto err_fd;
+	}
+
+	eu_dbg(d, "connected session %llu", d->session);
+
+	drm_dev_get(&xe->drm);
+
+	fd_install(fd, file);
+
+	return fd;
+
+err_fd:
+	put_unused_fd(fd);
+err_detach:
+	xe_eudebug_detach(d, err);
+err_put:
+	xe_eudebug_put(d);
+
+	return err;
+}
+
+void xe_eudebug_file_close(struct xe_file *xef)
+{
+	struct xe_eudebug *d;
+
+	d = xe_eudebug_get(xef);
+	if (d) {
+		xe_eudebug_detach(d, 0);
+		xe_eudebug_put(d);
+	}
+}
+
+static int xe_eudebug_enable(struct xe_device *xe, bool enable)
+{
+	guard(mutex)(&xe->eudebug.lock);
+
+	if (xe->eudebug.cap_state == XE_EUDEBUG_CAP_NOT_SUPPORTED)
+		return -EPERM;
+
+	if (!enable && xe->eudebug.attached)
+		return -EBUSY;
+
+	if (enable == __xe_eudebug_is_enabled(xe))
+		return 0;
+
+	set_cap_state(xe, enable ?
+		      XE_EUDEBUG_CAP_ENABLED : XE_EUDEBUG_CAP_DISABLED);
+
+	return 0;
+}
+
+static ssize_t enable_eudebug_show(struct device *dev,
+				   struct device_attribute *attr, char *buf)
+{
+	struct xe_device *xe = pdev_to_xe_device(to_pci_dev(dev));
+
+	return sysfs_emit(buf, "%u\n", xe_eudebug_is_enabled(xe));
+}
+
+static ssize_t enable_eudebug_store(struct device *dev,
+				    struct device_attribute *attr,
+				    const char *buf, size_t count)
+{
+	struct xe_device *xe = pdev_to_xe_device(to_pci_dev(dev));
+	bool enable;
+	int ret;
+
+	ret = kstrtobool(buf, &enable);
+	if (ret)
+		return ret;
+
+	ret = xe_eudebug_enable(xe, enable);
+	if (ret)
+		return ret;
+
+	return count;
+}
+
+static DEVICE_ATTR_RW(enable_eudebug);
+
+static void xe_eudebug_sysfs_fini(void *arg)
+{
+	struct xe_device *xe = arg;
+	struct drm_device *dev = &xe->drm;
+
+	sysfs_remove_file(&dev->dev->kobj,
+			  &dev_attr_enable_eudebug.attr);
+}
+
+static void xe_eudebug_lock_fini(struct drm_device *dev, void *__unused)
+{
+	mutex_destroy(&to_xe_device(dev)->eudebug.lock);
+}
+
+void xe_eudebug_init_early(struct xe_device *xe)
+{
+	/*
+	 * Not drmm_mutex_init(), which can fail and leave the lock destroyed:
+	 * the state is read under it from here on. Nothing can reach the
+	 * device yet, so the state itself is set without it. Without the
+	 * release action the lock is merely never destroyed, which only
+	 * costs the mutex debug checks.
+	 */
+	mutex_init(&xe->eudebug.lock);
+	if (drmm_add_action(&xe->drm, xe_eudebug_lock_fini, NULL))
+		drm_dbg(&xe->drm, "eudebug lock will not be destroyed\n");
+
+	xe->eudebug.cap_state = XE_EUDEBUG_CAP_DISABLED;
+}
+
+void xe_eudebug_init(struct xe_device *xe)
+{
+	struct drm_device *dev = &xe->drm;
+	int err;
+
+	/* early init failed */
+	if (xe->eudebug.cap_state == XE_EUDEBUG_CAP_NOT_SUPPORTED)
+		return;
+
+	err = devm_add_action_or_reset(dev->dev, xe_eudebug_sysfs_fini, xe);
+	if (err)
+		goto out_err;
+
+	/*
+	 * Last: once the attribute exists it can enable us, and nothing may
+	 * fail and overwrite cap_state after that. Removing a file that was
+	 * never created is a no-op, so the remove action can go first.
+	 */
+	err = sysfs_create_file(&dev->dev->kobj,
+				&dev_attr_enable_eudebug.attr);
+	if (err)
+		goto out_err;
+
+	return;
+
+out_err:
+	drm_warn(&xe->drm, "eudebug disabled, init fail: %d\n", err);
+
+	scoped_guard(mutex, &xe->eudebug.lock)
+		set_cap_state(xe, XE_EUDEBUG_CAP_NOT_SUPPORTED);
+}
+
+/**
+ * xe_eudebug_connect_ioctl - Connect to eudebug interface
+ * @dev : DRM device
+ * @data : ioctl data, (struct drm_xe_eudebug_connect)
+ * @file : DRM file
+ *
+ * Connect to the eudebug interface.
+ *
+ * Return: eudebug filedesriptor on success, negative error code on failure.
+ *
+ */
+int xe_eudebug_connect_ioctl(struct drm_device *dev,
+			     void *data,
+			     struct drm_file *file)
+{
+	struct xe_device *xe = to_xe_device(dev);
+	struct drm_xe_eudebug_connect * const param = data;
+
+	return xe_eudebug_connect(xe, file, param);
+}
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
new file mode 100644
index 000000000000..91e604c39814
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#ifndef _XE_EUDEBUG_H_
+#define _XE_EUDEBUG_H_
+
+#include <linux/errno.h>
+#include <linux/types.h>
+
+struct drm_device;
+struct drm_file;
+struct xe_device;
+struct xe_file;
+struct xe_vm;
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+
+#define XE_EUDEBUG_DBG_STR  "eudbg: %llu:%lld (%d/%d) -> (%d): "
+
+#define __eu_print(d, func, fmt, ...) \
+	do { \
+		struct xe_eudebug *__pd = (d); \
+		func(&__pd->xe->drm, XE_EUDEBUG_DBG_STR fmt, \
+		     __pd->session, \
+		     atomic64_read(&__pd->events.seqno), \
+		     current->pid, \
+		     task_tgid_nr(current), \
+		     __pd->target.pid, \
+		     ##__VA_ARGS__); \
+	} while (0)
+
+#define eu_dbg(d, fmt, ...)   __eu_print(d, drm_dbg, fmt, ##__VA_ARGS__)
+
+#define xe_eudebug_assert(d, ...) xe_assert((d)->xe, ##__VA_ARGS__)
+
+int xe_eudebug_connect_ioctl(struct drm_device *dev,
+			     void *data,
+			     struct drm_file *file);
+
+void xe_eudebug_init(struct xe_device *xe);
+void xe_eudebug_init_early(struct xe_device *xe);
+bool xe_eudebug_is_enabled(struct xe_device *xe);
+
+void xe_eudebug_file_close(struct xe_file *xef);
+
+void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm);
+void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm);
+
+#else
+
+static inline int xe_eudebug_connect_ioctl(struct drm_device *dev,
+					   void *data,
+					   struct drm_file *file) { return -EOPNOTSUPP; }
+
+static inline void xe_eudebug_init(struct xe_device *xe) { }
+static inline void xe_eudebug_init_early(struct xe_device *xe) { }
+static inline bool xe_eudebug_is_enabled(struct xe_device *xe) { return false; }
+
+static inline void xe_eudebug_file_close(struct xe_file *xef) { }
+
+static inline void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm) { }
+static inline void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm) { }
+
+#endif /* CONFIG_DRM_XE_EUDEBUG */
+
+#endif /* _XE_EUDEBUG_H_ */
diff --git a/drivers/gpu/drm/xe/xe_eudebug_types.h b/drivers/gpu/drm/xe/xe_eudebug_types.h
new file mode 100644
index 000000000000..598abb249195
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug_types.h
@@ -0,0 +1,168 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#ifndef _XE_EUDEBUG_TYPES_H_
+#define _XE_EUDEBUG_TYPES_H_
+
+#include <linux/kfifo.h>
+#include <linux/kref.h>
+#include <linux/mutex.h>
+#include <linux/rhashtable.h>
+#include <linux/sizes.h>
+#include <linux/wait.h>
+#include <linux/xarray.h>
+
+#include <uapi/drm/xe_drm_eudebug.h>
+
+struct xe_device;
+
+/**
+ * enum xe_eudebug_cap_state - eudebug capability state
+ *
+ * @XE_EUDEBUG_CAP_NOT_SUPPORTED: eudebug feature support off
+ * @XE_EUDEBUG_CAP_DISABLED: eudebug feature supported but disabled
+ * @XE_EUDEBUG_CAP_ENABLED: eudebug enabled
+ */
+enum xe_eudebug_cap_state {
+	XE_EUDEBUG_CAP_NOT_SUPPORTED = 0,
+	XE_EUDEBUG_CAP_DISABLED,
+	XE_EUDEBUG_CAP_ENABLED,
+};
+
+#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_VM
+
+/**
+ * struct xe_eudebug_handle - eudebug resource handle
+ */
+struct xe_eudebug_handle {
+	/** @key: key value in rhashtable <key:id> */
+	u64 key;
+
+	/** @id: opaque handle id for xarray <id:key> */
+	u32 id;
+
+	/** @rh_head: rhashtable head */
+	struct rhash_head rh_head;
+};
+
+/**
+ * struct xe_eudebug_resource - Resource map for one resource
+ */
+struct xe_eudebug_resource {
+	/** @xa: xarrays for <id->key> */
+	struct xarray xa;
+
+	/** @next_id: next id to try in @xa, protected by the @xa lock */
+	u32 next_id;
+
+	/** @rh: rhashtable for <key->id> */
+	struct rhashtable rh;
+};
+
+#define XE_EUDEBUG_RES_TYPE_VM		0
+#define XE_EUDEBUG_RES_TYPE_COUNT	(XE_EUDEBUG_RES_TYPE_VM + 1)
+
+/**
+ * struct xe_eudebug - Top level struct for eudebug: the connection
+ */
+struct xe_eudebug {
+	/** @ref: kref counter for this struct */
+	struct kref ref;
+
+	/**
+	 * @lock: guards all the mutable state of this struct: the target
+	 * link and error, the resource maps, the event queue with its
+	 * buffers and reader, the ack tree and the async disconnect error.
+	 *
+	 * Nests inside the vm reservation of a vm bind and inside
+	 * xe->eudebug.lock, and outside the user fence and xe_file eudebug
+	 * spinlocks. Never touch user memory, disconnect, or drop what may
+	 * be the last reference to this struct or to a user fence while
+	 * holding it.
+	 */
+	struct mutex lock;
+
+	/** @target: debug target specifics */
+	struct {
+		/**
+		 * @target.xef: the target xe_file that we are debugging, NULL
+		 * once detached. Written under both xe->eudebug.lock and
+		 * @lock, so either is enough to read it.
+		 */
+		struct xe_file *xef;
+
+		/** @target.pid: pid of target */
+		pid_t pid;
+
+		/** @target.err: error code on disconnect */
+		int err;
+
+		/** @target.res: resource maps for all types */
+		struct xe_eudebug_resource res[XE_EUDEBUG_RES_TYPE_COUNT];
+	} target;
+
+	/** @xe: the parent device we are serving */
+	struct xe_device *xe;
+
+	/** @session: session number for this connection (for logs) */
+	u64 session;
+
+	/** @events: kfifo queue of to-be-delivered events */
+	struct {
+#define XE_EUDEBUG_EVENT_MAX_SIZE SZ_64K
+		/**
+		 * @events.pending: pending event bounce buffer, preallocated.
+		 * Owned by the reader marked by @events.reader_active, which
+		 * copies it out without @lock while @events.pending_occupied is
+		 * set.
+		 */
+		struct drm_xe_eudebug_event *pending;
+
+		/**
+		 * @events.pending_occupied: @events.pending holds an event
+		 * that was popped from the fifo but not yet delivered
+		 */
+		bool pending_occupied;
+
+		/** @events.reader_active: a read is in progress, only one at a time */
+		bool reader_active;
+
+		/** @events.staging: write side staging buffer, preallocated */
+		struct drm_xe_eudebug_event *staging;
+
+		/** @events.fifo: queue of events pending */
+		struct kfifo fifo;
+
+		/** @events.fifo_buf: memory for the fifo */
+		void *fifo_buf;
+#define XE_EUDEBUG_FIFO_SIZE SZ_4M
+
+		/** @events.write_done: waitqueue for signalling write to fifo */
+		wait_queue_head_t write_done;
+
+		/**
+		 * @events.seqno: seqno counter to stamp events for fifo
+		 *
+		 * Stamped when an event is generated, which is not the same
+		 * moment it reaches the fifo, so concurrent producers can
+		 * take N and N+1 and then queue them the other way round.
+		 *
+		 * That is deliberate. The number is the client timeline the
+		 * debugger reconstructs, not a delivery order. Making the two
+		 * agree would mean holding @lock across event generation,
+		 * serialising the producers and losing the very ordering the
+		 * debugger is after.
+		 *
+		 * See drm_xe_eudebug_event.seqno for what this promises
+		 * userspace.
+		 *
+		 * Atomic, so that it can also be read without @lock, as
+		 * __eu_print() does for every debug message.
+		 */
+		atomic64_t seqno;
+	} events;
+};
+
+#endif /* _XE_EUDEBUG_TYPES_H_ */
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index dd855876f571..b9b851adc9bf 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -28,6 +28,7 @@
 #include "xe_cpu_bind.h"
 #include "xe_device.h"
 #include "xe_drm_client.h"
+#include "xe_eudebug.h"
 #include "xe_exec_queue.h"
 #include "xe_gt.h"
 #include "xe_migrate.h"
@@ -2226,10 +2227,18 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data,
 	args->reserved[0] = xe_bo_main_addr(vm->pt_root[0]->bo, XE_PAGE_SIZE);
 #endif
 
+	/*
+	 * Announce to the debugger before the id is published, as from that
+	 * point on a concurrent destroy can race us to the resource map.
+	 */
+	xe_eudebug_vm_create(xef, vm);
+
 	/* user id alloc must always be last in ioctl to prevent UAF */
 	err = xa_alloc(&xef->vm.xa, &id, vm, xa_limit_32b, GFP_KERNEL);
-	if (err)
+	if (err) {
+		xe_eudebug_vm_destroy(xef, vm);
 		goto err_close_and_put;
+	}
 
 	args->vm_id = id;
 
@@ -2264,8 +2273,10 @@ int xe_vm_destroy_ioctl(struct drm_device *dev, void *data,
 		xa_erase(&xef->vm.xa, args->vm_id);
 	mutex_unlock(&xef->vm.lock);
 
-	if (!err)
+	if (!err) {
+		xe_eudebug_vm_destroy(xef, vm);
 		xe_vm_close_and_put(vm);
+	}
 
 	return err;
 }
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index 578f0c1eb43e..942a7fe81f4b 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -85,6 +85,7 @@ extern "C" {
  *  - &DRM_IOCTL_XE_VM_QUERY_MEM_RANGE_ATTRS
  *  - &DRM_IOCTL_XE_EXEC_QUEUE_SET_PROPERTY
  *  - &DRM_IOCTL_XE_VM_GET_PROPERTY
+ *  - &DRM_IOCTL_XE_EUDEBUG_CONNECT
  */
 
 /*
@@ -110,6 +111,7 @@ extern "C" {
 #define DRM_XE_VM_QUERY_MEM_RANGE_ATTRS	0x0d
 #define DRM_XE_EXEC_QUEUE_SET_PROPERTY	0x0e
 #define DRM_XE_VM_GET_PROPERTY		0x0f
+#define DRM_XE_EUDEBUG_CONNECT		0x10
 
 /* Must be kept compact -- no holes */
 
@@ -129,6 +131,7 @@ extern "C" {
 #define DRM_IOCTL_XE_VM_QUERY_MEM_RANGE_ATTRS	DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_VM_QUERY_MEM_RANGE_ATTRS, struct drm_xe_vm_query_mem_range_attr)
 #define DRM_IOCTL_XE_EXEC_QUEUE_SET_PROPERTY	DRM_IOW(DRM_COMMAND_BASE + DRM_XE_EXEC_QUEUE_SET_PROPERTY, struct drm_xe_exec_queue_set_property)
 #define DRM_IOCTL_XE_VM_GET_PROPERTY		DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_VM_GET_PROPERTY, struct drm_xe_vm_get_property)
+#define DRM_IOCTL_XE_EUDEBUG_CONNECT		DRM_IOW(DRM_COMMAND_BASE + DRM_XE_EUDEBUG_CONNECT, struct drm_xe_eudebug_connect)
 
 /**
  * DOC: Xe IOCTL Extensions
@@ -2634,6 +2637,27 @@ enum drm_xe_ras_error_component {
 	[DRM_XE_RAS_ERR_COMP_FABRIC] = "fabric",			\
 }
 
+/**
+ * struct drm_xe_eudebug_connect - Input of &DRM_IOCTL_XE_EUDEBUG_CONNECT
+ *
+ * This structure is used to connect to an eudebug interface of target drm file.
+ */
+struct drm_xe_eudebug_connect {
+	/** @extensions: Pointer to the first extension struct, if any */
+	__u64 extensions;
+
+	/** @fd: Debug target DRM client fd */
+	__u64 fd;
+
+	/** @flags: Flags, MBZ */
+	__u64 flags;
+
+	/** @reserved: MBZ */
+	__u64 reserved;
+};
+
+#include "xe_drm_eudebug.h"
+
 #if defined(__cplusplus)
 }
 #endif
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
new file mode 100644
index 000000000000..63f4a6142a1a
--- /dev/null
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _UAPI_XE_DRM_EUDEBUG_H_
+#define _UAPI_XE_DRM_EUDEBUG_H_
+
+#include "drm.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/**
+ * DOC: DRM_XE_EUDEBUG_IOCTL_READ_EVENT
+ *
+ * Receive one event from the connection returned by
+ * &DRM_IOCTL_XE_EUDEBUG_CONNECT. The argument is a pointer to a
+ * &struct drm_xe_eudebug_event filled in as described there.
+ *
+ * A connection starts out blocking, and then waits up to five seconds for
+ * an event to arrive; O_NONBLOCK can be set on it with fcntl(). Only one
+ * reader at a time is allowed on a connection.
+ *
+ * Return: 0 on success. Negative error code on failure:
+ *
+ * - -EMSGSIZE if the pending event is larger than the supplied len. len is
+ *   updated with the size needed and the event stays queued.
+ * - -ETIMEDOUT if the blocking wait expired with no event.
+ * - -EAGAIN if O_NONBLOCK was set and no event was queued.
+ * - -EBUSY if another thread is already reading on this connection.
+ * - -ENOTCONN if the debug target is gone.
+ *
+ * Detaching discards whatever is still queued: the events describe resources
+ * of a client that no longer exists, and the handles in them can no longer be
+ * resolved. A debugger that gets -ENOTCONN, or EPOLLHUP from poll(), should
+ * treat the event stream as having ended at an unspecified point rather than
+ * assume it saw everything the target did.
+ */
+#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT		_IOWR('j', 0x40, struct drm_xe_eudebug_event)
+
+/**
+ * struct drm_xe_eudebug_event - Base type of event delivered by xe_eudebug.
+ *
+ * Base event for xe_eudebug interface.
+ *
+ * For receiving events :c:member:`drm_xe_eudebug_event.type` has to
+ * be DRM_XE_EUDEBUG_EVENT_READ. On return, this is set to the type
+ * of event received. :c:member:`drm_xe_eudebug_event.len` has to be
+ * set to maximum size that can be received. On return, len will be set
+ * to the event size. If the pending event was larger than this size,
+ * -EMSGSIZE is returned instead of 0 and the caller should retry with a larger
+ * allocated receive length.
+ *
+ * :c:member:`drm_xe_eudebug_event.seqno` can be used to form a timeline
+ * as event delivery order does not guarantee event creation
+ * order. Must be zero in the read request.
+ *
+ * :c:member:`drm_xe_eudebug_event.flags` will indicate if a resource was
+ * created or destroyed. Must be zero in the read request.
+ *
+ * %DRM_XE_EUDEBUG_EVENT_NONE and %DRM_XE_EUDEBUG_EVENT_READ are never
+ * delivered: the first is unused and the second is the request type for
+ * the read ioctl.
+ *
+ * :c:member:`drm_xe_eudebug_event.len` is authoritative for the event that
+ * was read, and events may grow fields in later kernels. Step to the next
+ * event and skip a type this debugger does not know by @len rather than by
+ * the size of the matching struct in the header it was built against.
+ */
+struct drm_xe_eudebug_event {
+	/** @len: Length */
+	__u32 len;
+
+	/** @type: Type */
+	__u16 type;
+#define DRM_XE_EUDEBUG_EVENT_NONE		0
+#define DRM_XE_EUDEBUG_EVENT_READ		1
+#define DRM_XE_EUDEBUG_EVENT_VM			2
+
+	/** @flags: Flags */
+	__u16 flags;
+#define DRM_XE_EUDEBUG_EVENT_CREATE		(1 << 0)
+#define DRM_XE_EUDEBUG_EVENT_DESTROY		(1 << 1)
+
+	/**
+	 * @seqno: Sequence number, for the debugger to construct a timeline.
+	 *
+	 * Allocated per connection and monotonically increasing, taken at the
+	 * point the event is generated rather than when it is queued.
+	 *
+	 * Event generation is asynchronous: threads in the target produce
+	 * events concurrently, so events are not guaranteed to arrive in
+	 * seqno order. The numbering may also contain gaps, as a number can
+	 * be allocated for an event that is never queued. Neither a lower
+	 * seqno arriving after a higher one nor a gap means an event was
+	 * lost; if an event cannot be queued the connection is disconnected
+	 * instead of silently dropping it.
+	 *
+	 * Use this to order and correlate the events that have been read,
+	 * which is what timeline means here. Do not use it as a delivery
+	 * order guarantee.
+	 */
+	__u64 seqno;
+
+	/** @reserved: Reserved field, must be zero. */
+	__u64 reserved;
+};
+
+/**
+ * struct drm_xe_eudebug_event_vm - VM event
+ *
+ * VM event is delivered when vm is created or destroyed. Only vms created
+ * with %DRM_XE_VM_CREATE_FLAG_FAULT_MODE, the ones debuggable exec queues
+ * can run on, are reported; other vms of the target are not visible to the
+ * debugger.
+ */
+struct drm_xe_eudebug_event_vm {
+	/** @base: base event */
+	struct drm_xe_eudebug_event base;
+
+	/** @vm_handle: unique handle for vm */
+	__u64 vm_handle;
+};
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif /* _UAPI_XE_DRM_EUDEBUG_H_ */
-- 
2.53.0

[PATCH 03/20] drm/xe/eudebug: Add documentation

[Mika Kuoppala]

Add core documentation for connection setup and events.
Follow-up patches can then incrementally add their workings.

Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/index.rst      |  1 +
 Documentation/gpu/xe/xe_eudebug.rst | 56 +++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_eudebug.c     | 19 ++++++++++
 3 files changed, 76 insertions(+)
 create mode 100644 Documentation/gpu/xe/xe_eudebug.rst

diff --git a/Documentation/gpu/xe/index.rst b/Documentation/gpu/xe/index.rst
index 0247a255f7e6..8d4abdfe71a7 100644
--- a/Documentation/gpu/xe/index.rst
+++ b/Documentation/gpu/xe/index.rst
@@ -36,3 +36,4 @@ The display, or :ref:`drm-kms`, support for drm/xe is provided by
    xe_configfs
    xe_gt_stats
    xe_sigid
+   xe_eudebug
diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
new file mode 100644
index 000000000000..a73f8bd9a8e9
--- /dev/null
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -0,0 +1,56 @@
+.. SPDX-License-Identifier: (GPL-2.0+ OR MIT)
+
+===================================
+Xe EU Debug (eudebug)
+===================================
+
+.. The uapi header is also included in full by driver-uapi.rst, so keep
+   the structs pulled in here out of the global C namespace to avoid
+   duplicate declaration warnings.
+
+.. c:namespace:: xe_eudebug
+
+Overview
+========
+
+The EU Debug (eudebug) subsystem provides an interface for debugging
+Execution Unit (EU) workloads on Intel Xe GPUs, independently from
+CPU process debugging. This enables tools like GDB (via gdbserver-intelgt)
+to debug GPU compute workloads through the Level-Zero Debug API.
+In practice, this allows SYCL/DPC++ workloads submitted by the
+compute-runtime to be debugged interactively.
+
+
+Connection Establishment
+========================
+
+.. kernel-doc:: drivers/gpu/drm/xe/xe_eudebug.c
+   :doc: Connection Establishment
+
+Events
+======
+
+.. kernel-doc:: drivers/gpu/drm/xe/xe_eudebug.c
+   :doc: Events
+
+Reading Events
+--------------
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :doc: DRM_XE_EUDEBUG_IOCTL_READ_EVENT
+
+Base Event Header
+-----------------
+
+The debugger receives events with the DRM_XE_EUDEBUG_IOCTL_READ_EVENT
+ioctl on the eudebug file descriptor. Every event has
+:c:type:`drm_xe_eudebug_event` as a header.
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event
+
+Resource Event Types
+--------------------
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event_vm
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 0a57d99fc5e8..61572c006d22 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -19,6 +19,25 @@
 #include "xe_macros.h"
 #include "xe_vm.h"
 
+/**
+ * DOC: Connection Establishment
+ *
+ * To debug a target DRM client, the debugger must first establish
+ * a connection using :c:type:`drm_xe_eudebug_connect`.
+ *
+ */
+
+/**
+ * DOC: Events
+ *
+ * Resource creation/destruction and hardware state changes are
+ * delivered as events to the debugger.
+ *
+ * To read an event, ioctl DRM_XE_EUDEBUG_IOCTL_READ_EVENT command
+ * can be issued with prefilled :c:type:`drm_xe_eudebug_event` as argument.
+ *
+ */
+
 #define cast_event(T, event) container_of((event), typeof(*(T)), base)
 
 static const struct rhashtable_params rhash_res = {
-- 
2.53.0

[PATCH 04/20] drm/xe/eudebug: Add connection establishment documentation

[Mika Kuoppala]

Add documentation for connecting to a target DRM Xe client
for debugging.

v2: improved Pipe and Fork documentation (Sashiko)
v10: qualify the ptrace_may_access() parity claim per fd
     acquisition method (Claude)

Cc: Christian König <christian.koenig@amd.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/xe_eudebug.rst |  11 +++
 drivers/gpu/drm/xe/xe_eudebug.c     | 104 +++++++++++++++++++++++++++-
 2 files changed, 114 insertions(+), 1 deletion(-)

diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index a73f8bd9a8e9..9c6070784e84 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -27,6 +27,17 @@ Connection Establishment
 .. kernel-doc:: drivers/gpu/drm/xe/xe_eudebug.c
    :doc: Connection Establishment
 
+File Descriptor Acquisition Methods
+-----------------------------------
+
+.. kernel-doc:: drivers/gpu/drm/xe/xe_eudebug.c
+   :doc: File Descriptor Acquisition Methods
+
+Security Model
+--------------
+.. kernel-doc:: drivers/gpu/drm/xe/xe_eudebug.c
+   :doc: Security Model
+
 Events
 ======
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 61572c006d22..d18ba28f43b1 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -23,8 +23,110 @@
  * DOC: Connection Establishment
  *
  * To debug a target DRM client, the debugger must first establish
- * a connection using :c:type:`drm_xe_eudebug_connect`.
+ * a connection with the DRM_IOCTL_XE_EUDEBUG_CONNECT ioctl, issued on a
+ * DRM file descriptor of its own for the same device and taking a
+ * :c:type:`drm_xe_eudebug_connect`. On success the ioctl returns a new
+ * file descriptor, the eudebug file descriptor, on which the debugger
+ * reads events and issues the eudebug ioctls. Closing it ends the
+ * connection.
  *
+ * The interface has to be enabled for the device first, by writing 1 to
+ * its enable_eudebug sysfs attribute; until then the connect ioctl fails
+ * with -EOPNOTSUPP. See Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug.
+ *
+ * The debug target's DRM client file descriptor is passed into the
+ * connect ioctl via :c:member:`drm_xe_eudebug_connect.fd`.
+ * This is the fd that the target process obtained from open('/dev/dri/cardX').
+ *
+ * This file descriptor must be a valid DRM client fd in the debugger's
+ * (calling) process context. For debugging an Xe DRM client in
+ * another process, pidfd_getfd() can be used to acquire a duplicate
+ * of the file descriptor from the target process. See
+ * :ref:`File Descriptor Acquisition Methods <fd_acquisition_methods>` for
+ * details on how to obtain the target's fd.
+ *
+ */
+
+/**
+ * DOC: Security Model
+ *
+ * If you are inside the same process, you can connect to your own DRM client
+ * by simply passing its fd to drm_xe_eudebug_connect.
+ *
+ * For a remote process, possession of that process's DRM client fd is
+ * the capability: the connect ioctl adds no credential check of its own.
+ * What sets the bar is therefore how that fd was obtained.
+ *
+ * Taking an fd from a process that did not offer it goes through
+ * pidfd_getfd(), and the kernel enforces credentials there.
+ * __pidfd_fget() requires ptrace_may_access() with
+ * PTRACE_MODE_ATTACH_REALCREDS, under the target's
+ * signal->exec_update_lock. That is the check ptrace(PTRACE_ATTACH)
+ * makes, and the one process_vm_readv() makes under the same lock in
+ * mm_access(), so for this path the same ptrace_may_access() rules apply
+ * as in CPU process debugging with gdb. Most of the File Descriptor
+ * Acquisition Methods end up here, including those that only use procfs
+ * to discover which fd number to ask for.
+ *
+ * Two methods do not involve ptrace_may_access(), because nothing is
+ * being taken. With SCM_RIGHTS the target sends its own fd over a unix
+ * socket, which is the target consenting. With fork() inheritance the fd
+ * belonged to the debugger before the fork and the child merely
+ * inherited a copy of it.
+ *
+ * See :ref:`File Descriptor Acquisition Methods <fd_acquisition_methods>`.
+ */
+
+/**
+ * DOC: File Descriptor Acquisition Methods
+ * .. _fd_acquisition_methods:
+ *
+ * There are multiple ways to get the target DRM client fd for
+ * another process:
+ *
+ * Unix domain socket
+ *     The debugger can receive the DRM client fd from the target via a Unix
+ *     domain socket using SCM_RIGHTS ancillary data. This is the standard
+ *     mechanism for passing file descriptors between processes, but requires
+ *     coordination to establish the socket connection.
+ *
+ * Pipe
+ *     The target sends the fd number through a pipe, and the debugger
+ *     duplicates it via pidfd_getfd(). This requires coordination between
+ *     the processes to establish the pipe and synchronize the transfer.
+ *
+ * Fork
+ *     If the debugger spawns the target process via fork(), a DRM client
+ *     fd can be shared across the fork boundary — either inherited by the
+ *     child from the debugger, or acquired from the child by the debugger
+ *     via pidfd_getfd() after the child opens the device. This is useful
+ *     when the debugger launches the target directly, similar to how gdb
+ *     spawns inferiors.
+ *
+ * Ptrace
+ *     The debugger can attach to the target process using ptrace. It can
+ *     intercept the return value of the target's open() or drmOpen() call
+ *     by inspecting registers at syscall exit to catch the fd when it is
+ *     created. Alternatively, if the target has already opened the device,
+ *     the debugger can read the target's memory using PTRACE_PEEKDATA to
+ *     locate the stored fd value in a known variable or data structure.
+ *     Once the fd number is known, pidfd_getfd() can be used to acquire a
+ *     duplicate in the debugger's process context.
+ *
+ * Procfs
+ *     The debugger can look through /proc/<targetpid>/fd for file descriptors
+ *     and inspect /proc/<targetpid>/fdinfo to see which of those
+ *     are for an Xe DRM client, then duplicate that fd number with
+ *     pidfd_getfd().
+ *
+ * Brute force
+ *     The debugger can duplicate each fd listed in /proc/<targetpid>/fd
+ *     with pidfd_getfd() and try to connect with it to see if it succeeds.
+ *
+ * In both procfs methods the fd has to be taken with pidfd_getfd(). Opening
+ * /proc/<targetpid>/fd/<N> instead opens the device node anew and yields a
+ * new, empty DRM client of the debugger's own, which connect accepts but
+ * which is not the target.
  */
 
 /**
-- 
2.53.0

[PATCH 05/20] drm/xe: Implement SR-IOV and eudebug exclusivity

[Mika Kuoppala]

From: Christoph Manszewski <christoph.manszewski@intel.com>

EU debug functionality relies on access to specific mmio registers.
Since VFs don't have access to those registers and in order to avoid
interference with VFs, make SR-IOV and eudebug functionality exclusive.
I.e. don't allow enabling eudebug in VF mode and don't allow enabling
eudebug when any VFs are enabled. Likewise, don't allow enabling
VFs when eudebug is enabled.

v2: initialize lock early (Sashiko)
v3: say why eudebug was refused (Claude)
v4: avoid racing with work (Sashiko)
v5: - don't log per VF probe (Sashiko)
v6: - cap_state under eudebug.lock (Mika)
v7: - poll stop comment: a PF waits the last round out (Sashiko)
v8: - move right after the interface documentation, the vm access, EU
      control, attention worker and live tests patches now cover their
      own parts (Mika)

Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 .../ABI/testing/sysfs-driver-intel-xe-eudebug |  4 ++++
 drivers/gpu/drm/xe/xe_eudebug.c               | 21 +++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
index 26aff5116b8a..884a791789ad 100644
--- a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
+++ b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
@@ -34,3 +34,7 @@ Description:	RW. Controls whether the EU debugger interface is armed for
 		CONFIG_DRM_XE_EUDEBUG and the interface was set up at probe
 		time. If probe could not set it up, the attribute is absent
 		and the connect ioctl fails with -EOPNOTSUPP.
+
+		On an SR-IOV VF the attribute is absent. On a PF, writing 1
+		while VFs are enabled fails with -EBUSY, and while the
+		interface is enabled enabling VFs fails with -EPERM.
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index d18ba28f43b1..ad8e1c02f20b 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -17,6 +17,7 @@
 #include "xe_eudebug.h"
 #include "xe_eudebug_types.h"
 #include "xe_macros.h"
+#include "xe_sriov_pf.h"
 #include "xe_vm.h"
 
 /**
@@ -1203,6 +1204,8 @@ void xe_eudebug_file_close(struct xe_file *xef)
 
 static int xe_eudebug_enable(struct xe_device *xe, bool enable)
 {
+	int ret;
+
 	guard(mutex)(&xe->eudebug.lock);
 
 	if (xe->eudebug.cap_state == XE_EUDEBUG_CAP_NOT_SUPPORTED)
@@ -1214,9 +1217,21 @@ static int xe_eudebug_enable(struct xe_device *xe, bool enable)
 	if (enable == __xe_eudebug_is_enabled(xe))
 		return 0;
 
+	/* Lock VFs out first, so that a refusal leaves nothing to take back */
+	if (enable && IS_SRIOV_PF(xe)) {
+		ret = xe_sriov_pf_lockdown(xe);
+		if (ret) {
+			drm_dbg(&xe->drm, "Can't enable eudebug: VFs are enabled\n");
+			return ret;
+		}
+	}
+
 	set_cap_state(xe, enable ?
 		      XE_EUDEBUG_CAP_ENABLED : XE_EUDEBUG_CAP_DISABLED);
 
+	if (!enable && IS_SRIOV_PF(xe))
+		xe_sriov_pf_end_lockdown(xe);
+
 	return 0;
 }
 
@@ -1288,6 +1303,11 @@ void xe_eudebug_init(struct xe_device *xe)
 	if (xe->eudebug.cap_state == XE_EUDEBUG_CAP_NOT_SUPPORTED)
 		return;
 
+	if (IS_SRIOV_VF(xe)) {
+		drm_dbg(&xe->drm, "eudebug not available in SR-IOV VF mode\n");
+		goto out_not_supported;
+	}
+
 	err = devm_add_action_or_reset(dev->dev, xe_eudebug_sysfs_fini, xe);
 	if (err)
 		goto out_err;
@@ -1307,6 +1327,7 @@ void xe_eudebug_init(struct xe_device *xe)
 out_err:
 	drm_warn(&xe->drm, "eudebug disabled, init fail: %d\n", err);
 
+out_not_supported:
 	scoped_guard(mutex, &xe->eudebug.lock)
 		set_cap_state(xe, XE_EUDEBUG_CAP_NOT_SUPPORTED);
 }
-- 
2.53.0

[PATCH 06/20] drm/xe/eudebug: Introduce discovery for resources

[Mika Kuoppala]

A debugger connection can occur after a client has created
and destroyed an arbitrary number of resources. To support
this, we need to relay all currently existing resources to
the debugger. The client is held on selected ioctls until
this discovery process, run in the connect ioctl, is complete.

Based on discovery work by Maciej Patelczyk for the i915 driver.

v2: - use rw_semaphore to block DRM ioctls during discovery (Matthew)
    - only lock according to ioctl at play (Dominik)
v4: - s/discovery_lock/ioctl_lock
    - change lock to be per xe_file as is connections
v5: - helper to get eudebug instance during discovery (Maciej)
v6: - complete_all even if no client (Sashiko)
    - don't flush on detach (Sashiko)
v7: - guard close with ioctl lock (Sashiko)
v8: - discovery lock comment fix (Maciej)
v9: - disconnect on failed discovery (Sashiko)
    - avoid close race (Sashiko)
v10: - unordered discovery workqueue (Sashiko)
v11: - -EBUSY only from the ioctls that wait for discovery, keep
       -ENOTTY (review-prompts)
v12: - discover in the connect ioctl, no worker, killable gate (Mika)
v13: - exec queue uapi, helpers and exports moved to their users (Sashiko)
     - scoped ioctl_lock in attach, no goto next to __free() (Sashiko)

Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Co-developed-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Acked-by: Matthew Brost <matthew.brost@intel.com> #locking
---
 drivers/gpu/drm/xe/xe_device.c       | 24 +++++++-
 drivers/gpu/drm/xe/xe_device.h       | 53 ++++++++++++++++
 drivers/gpu/drm/xe/xe_device_types.h | 12 ++++
 drivers/gpu/drm/xe/xe_eudebug.c      | 90 +++++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_eudebug.h      |  1 +
 include/uapi/drm/xe_drm.h            |  8 +++
 6 files changed, 183 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index ed28abc4f37f..d11429a69325 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -117,6 +117,10 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)
 
 	mutex_init(&xef->mmio_gem.lock);
 
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	init_rwsem(&xef->eudebug.ioctl_lock);
+#endif
+
 	file->driver_priv = xef;
 	kref_init(&xef->refcount);
 
@@ -248,8 +252,16 @@ static long xe_drm_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 
 	ACQUIRE(xe_pm_runtime_ioctl, pm)(xe);
 	ret = ACQUIRE_ERR(xe_pm_runtime_ioctl, &pm);
-	if (ret >= 0)
+	if (ret >= 0) {
+		int lock = xe_eudebug_discovery_lock(file, cmd);
+
+		if (lock < 0)
+			return lock;
+
 		ret = drm_ioctl(file, cmd, arg);
+		if (lock)
+			xe_eudebug_discovery_unlock(file, cmd);
+	}
 
 	return ret;
 }
@@ -266,8 +278,16 @@ static long xe_drm_compat_ioctl(struct file *file, unsigned int cmd, unsigned lo
 
 	ACQUIRE(xe_pm_runtime_ioctl, pm)(xe);
 	ret = ACQUIRE_ERR(xe_pm_runtime_ioctl, &pm);
-	if (ret >= 0)
+	if (ret >= 0) {
+		int lock = xe_eudebug_discovery_lock(file, cmd);
+
+		if (lock < 0)
+			return lock;
+
 		ret = drm_compat_ioctl(file, cmd, arg);
+		if (lock)
+			xe_eudebug_discovery_unlock(file, cmd);
+	}
 
 	return ret;
 }
diff --git a/drivers/gpu/drm/xe/xe_device.h b/drivers/gpu/drm/xe/xe_device.h
index 6df60edcb8f8..8409db81e99d 100644
--- a/drivers/gpu/drm/xe/xe_device.h
+++ b/drivers/gpu/drm/xe/xe_device.h
@@ -7,6 +7,7 @@
 #define _XE_DEVICE_H_
 
 #include <drm/drm_util.h>
+#include <drm/drm_ioctl.h>
 
 #include "xe_device_types.h"
 #include "xe_gt_types.h"
@@ -293,4 +294,56 @@ void xe_device_exit(void);
 #define LNL_FLUSH_WORK(wrk__) \
 	flush_work(wrk__)
 
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+static inline bool xe_eudebug_needs_ioctl_lock(const unsigned int cmd)
+{
+	const unsigned int xe_cmd = DRM_IOCTL_NR(cmd) - DRM_COMMAND_BASE;
+
+	switch (xe_cmd) {
+	case DRM_XE_VM_CREATE:
+	case DRM_XE_VM_DESTROY:
+	case DRM_XE_VM_BIND:
+	case DRM_XE_EXEC_QUEUE_CREATE:
+	case DRM_XE_EXEC_QUEUE_DESTROY:
+		return true;
+	}
+
+	return false;
+}
+
+/*
+ * A connect waiting for the write side holds new readers off, and a client
+ * killed meanwhile must not go on to create resources once let in: exec
+ * queue creation, for one, cannot start its scheduler's rescuer thread
+ * with a fatal signal pending.
+ *
+ * Return: 1 if the lock was taken, 0 if @cmd does not need it, -EINTR on a
+ * fatal signal.
+ */
+static inline int xe_eudebug_discovery_lock(struct file *file, unsigned int cmd)
+{
+	struct drm_file *file_priv = file->private_data;
+	struct xe_file *xef = file_priv->driver_priv;
+
+	if (!xe_eudebug_needs_ioctl_lock(cmd))
+		return 0;
+
+	if (down_read_killable(&xef->eudebug.ioctl_lock))
+		return -EINTR;
+
+	return 1;
+}
+
+static inline void xe_eudebug_discovery_unlock(struct file *file, unsigned int cmd)
+{
+	struct drm_file *file_priv = file->private_data;
+	struct xe_file *xef = file_priv->driver_priv;
+
+	up_read(&xef->eudebug.ioctl_lock);
+}
+#else
+static inline int xe_eudebug_discovery_lock(struct file *file, unsigned int cmd) { return 0; }
+static inline void xe_eudebug_discovery_unlock(struct file *file, unsigned int cmd) { }
+#endif /* CONFIG_DRM_XE_EUDEBUG */
+
 #endif
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 000bf91bc5f0..6375a74414d3 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -735,6 +735,18 @@ struct xe_file {
 	struct {
 		/** @eudebug.debugger: the debugger connection into this xe_file */
 		struct xe_eudebug *debugger;
+
+		/**
+		 * @eudebug.ioctl_lock: excludes the ioctls that mutate the
+		 * eudebug visible resources of this client, so that a writer
+		 * sees a stable set of vms and exec queues. Taken for read
+		 * around the whole ioctl by xe_eudebug_discovery_lock(), for
+		 * the ioctls named in xe_eudebug_needs_ioctl_lock(), and for
+		 * write by the connect ioctl, across discovery, and by file
+		 * close. Held across an entire ioctl, so it nests outside
+		 * every lock those ioctls take.
+		 */
+		struct rw_semaphore ioctl_lock;
 	} eudebug;
 #endif
 };
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index ad8e1c02f20b..e0ef34588d3d 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -46,6 +46,14 @@
  * :ref:`File Descriptor Acquisition Methods <fd_acquisition_methods>` for
  * details on how to obtain the target's fd.
  *
+ * All existing resources that the target Xe DRM client already has are
+ * sent as events, as if they had been created right after the
+ * connection. This is referred to as discovery. The events are queued
+ * before the connect ioctl returns, so every eudebug ioctl can be used
+ * on the connection right away. To have a stable set of resources, the
+ * connect waits for the target's resource ioctls in flight to finish,
+ * and holds off new ones until discovery is done.
+ *
  */
 
 /**
@@ -392,7 +400,7 @@ bool xe_eudebug_is_enabled(struct xe_device *xe)
 }
 
 static struct xe_eudebug *
-xe_eudebug_get(struct xe_file *xef)
+__xe_eudebug_get(struct xe_file *xef)
 {
 	struct xe_device *xe = xef->xe;
 	struct xe_eudebug *d;
@@ -411,6 +419,19 @@ xe_eudebug_get(struct xe_file *xef)
 	return d;
 }
 
+/*
+ * The connect ioctl publishes the debugger and runs discovery under the
+ * target's ioctl_lock held for write, so a caller holding it for read
+ * only ever sees a debugger that discovery is done with.
+ */
+static struct xe_eudebug *
+xe_eudebug_get(struct xe_file *xef)
+{
+	lockdep_assert_held(&xef->eudebug.ioctl_lock);
+
+	return __xe_eudebug_get(xef);
+}
+
 static int xe_eudebug_queue_event(struct xe_eudebug *d,
 				  struct drm_xe_eudebug_event *event)
 {
@@ -778,6 +799,48 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)
 	xe_eudebug_put(d);
 }
 
+/*
+ * All the relevant ioctls that manipulate the xe_file resources eudebug
+ * cares about are gated behind down_read(&xef->eudebug.ioctl_lock). The
+ * caller holds it for write, which excludes all those ioctls and lets us
+ * peek into the resource lists without taking their individual locks.
+ */
+static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
+{
+	struct xe_vm *vm;
+	unsigned long i;
+	unsigned int vm_count = 0;
+	int err = 0;
+
+	lockdep_assert_held_write(&xef->eudebug.ioctl_lock);
+
+	eu_dbg(d, "Discovery start for %llu", d->session);
+
+	/* vm lifetime guaranteed by the ioctl_lock */
+	xa_for_each(&xef->vm.xa, i, vm) {
+		/* Only fault mode vms are announced, see xe_eudebug_vm_create() */
+		if (!xe_vm_in_fault_mode(vm))
+			continue;
+
+		err = vm_create_event(d, vm);
+		if (err)
+			goto out;
+		vm_count++;
+	}
+
+out:
+	if (vm_count)
+		eu_dbg(d, "Discovery found %u vms",
+		       vm_count);
+
+	eu_dbg(d, "Discovery end for %llu: %d", d->session, err);
+
+	if (err < 0)
+		xe_eudebug_disconnect(d, err);
+
+	return err;
+}
+
 static int add_debugger(struct xe_device *xe, struct xe_eudebug *d,
 			struct drm_file *target)
 {
@@ -835,6 +898,22 @@ xe_eudebug_attach(struct xe_device *xe, struct drm_file *parent_file,
 	if (XE_IOCTL_DBG(xe, xe != target_xef->xe))
 		return -EINVAL;
 
+	/*
+	 * Keep the target's resource ioctls out from before the debugger is
+	 * published until discovery has queued what already exists, so that
+	 * each resource is reported exactly once: either here, or by the
+	 * ioctl that creates it. Discovery relies on this to walk the
+	 * resource lists unlocked, see discover_client(). The file reference
+	 * keeps target_xef alive until we are done with it.
+	 *
+	 * This waits for the gated ioctls already in flight, which can stall
+	 * in their user copies. Killable, so that the debugger can give up.
+	 */
+	ACQUIRE(rwsem_write_kill, ioctl_lock)(&target_xef->eudebug.ioctl_lock);
+	ret = ACQUIRE_ERR(rwsem_write_kill, &ioctl_lock);
+	if (ret)
+		return ret;
+
 	ret = add_debugger(xe, d, drm_file);
 	if (XE_IOCTL_DBG(xe, ret))
 		return ret;
@@ -842,7 +921,8 @@ xe_eudebug_attach(struct xe_device *xe, struct drm_file *parent_file,
 	eu_dbg(d, "session %llu attached to %s", d->session,
 	       parent_file == drm_file ? "self" : "remote");
 
-	return 0;
+	/* On failure, discovery has disconnected us already */
+	return discover_client(d, target_xef);
 }
 
 static int xe_eudebug_release(struct inode *inode, struct file *file)
@@ -1195,11 +1275,15 @@ void xe_eudebug_file_close(struct xe_file *xef)
 {
 	struct xe_eudebug *d;
 
-	d = xe_eudebug_get(xef);
+	down_write(&xef->eudebug.ioctl_lock);
+
+	d = __xe_eudebug_get(xef);
 	if (d) {
 		xe_eudebug_detach(d, 0);
 		xe_eudebug_put(d);
 	}
+
+	up_write(&xef->eudebug.ioctl_lock);
 }
 
 static int xe_eudebug_enable(struct xe_device *xe, bool enable)
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index 91e604c39814..413a45129193 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -14,6 +14,7 @@ struct drm_file;
 struct xe_device;
 struct xe_file;
 struct xe_vm;
+struct xe_eudebug;
 
 #if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
 
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index 942a7fe81f4b..e84b11324f08 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -2641,6 +2641,14 @@ enum drm_xe_ras_error_component {
  * struct drm_xe_eudebug_connect - Input of &DRM_IOCTL_XE_EUDEBUG_CONNECT
  *
  * This structure is used to connect to an eudebug interface of target drm file.
+ *
+ * On connect, the resources the target already has are first relayed to
+ * the debugger as events. This is called discovery. The events are
+ * queued before the ioctl returns. If they do not fit the event queue,
+ * the ioctl fails with -ENOSPC.
+ *
+ * The ioctl waits for the target's resource ioctls in flight to finish,
+ * and fails with -EINTR if a fatal signal arrives meanwhile.
  */
 struct drm_xe_eudebug_connect {
 	/** @extensions: Pointer to the first extension struct, if any */
-- 
2.53.0

[PATCH 07/20] drm/xe: Add EUDEBUG_ENABLE exec queue property

[Mika Kuoppala]

From: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

Introduce an immutable eudebug property for exec_queues,
using a flags value to enable eudebug-specific features. For now, the
engine LRC uses this flag to enable the runalone hardware feature.
Runalone ensures that only one hardware engine in a group
[rcs0, ccs0-3] is active on a tile.

v2: - check CONFIG_DRM_XE_EUDEBUG and LR mode (Matthew)
    - disable preempt (Dominik)
    - lrc_create remove from engine init
v3: - remove redundant runalone setup (Sashiko)
v4: - deny if pxp is on (Sashiko)
    - deny if secondary mq (Sashiko)
    - deny if parallel (Sashiko)
    - really deny if pxp (Claude)
v5: - fix the multiqueue (Sashiko)

Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 .../ABI/testing/sysfs-driver-intel-xe-eudebug | 10 +++
 drivers/gpu/drm/xe/xe_exec_queue.c            | 81 ++++++++++++++++++-
 drivers/gpu/drm/xe/xe_exec_queue.h            |  2 +
 drivers/gpu/drm/xe/xe_exec_queue_types.h      |  7 ++
 drivers/gpu/drm/xe/xe_hw_engine.h             |  9 +++
 include/uapi/drm/xe_drm.h                     | 12 +++
 6 files changed, 120 insertions(+), 1 deletion(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
index 884a791789ad..50ab7db1f811 100644
--- a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
+++ b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
@@ -24,6 +24,16 @@ Description:	RW. Controls whether the EU debugger interface is armed for
 		have to close them or be terminated. A connection shows up as
 		an "anon_inode:[xe_eudebug]" entry under /proc/<pid>/fd.
 
+		Once enabled, any client with access to the render node can
+		also create debuggable exec queues
+		(DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG), whether or not a
+		debugger is attached. While such a queue executes, the other
+		render and compute engines of its tile are kept idle
+		(runalone), and as its preemption timeout is infinite it is
+		not reset if it does not yield the engine when preempted.
+		Work from other clients on those engines can therefore be
+		held off for as long as such a queue keeps running.
+
 		Each connection allocates about 4 MiB of kernel memory for
 		its event queue, held until the connection is closed. The
 		number of connections is not limited beyond the file
diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c
index 7ade915c59a8..e65bda492e58 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue.c
+++ b/drivers/gpu/drm/xe/xe_exec_queue.c
@@ -31,6 +31,7 @@
 #include "xe_trace.h"
 #include "xe_vm.h"
 #include "xe_pxp.h"
+#include "xe_eudebug.h"
 
 /**
  * DOC: Execution Queue
@@ -416,6 +417,9 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q, u32 exec_queue_flags)
 	if (q->flags & EXEC_QUEUE_FLAG_DISABLE_STATE_CACHE_PERF_FIX)
 		flags |= XE_LRC_DISABLE_STATE_CACHE_PERF_FIX;
 
+	if (q->eudebug_flags & EXEC_QUEUE_EUDEBUG_FLAG_ENABLE)
+		flags |= XE_LRC_CREATE_RUNALONE;
+
 	err = q->ops->init(q);
 	if (err)
 		return err;
@@ -1050,6 +1054,57 @@ static int exec_queue_set_state_cache_perf_fix(struct xe_device *xe, struct xe_e
 	return 0;
 }
 
+static int exec_queue_set_eudebug(struct xe_device *xe, struct xe_exec_queue *q,
+				  u64 value)
+{
+	const u64 known_flags = DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE;
+
+	if (XE_IOCTL_DBG(xe, !xe_engine_supports_eudebug(q->class)))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, (value & ~known_flags)))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, !IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)))
+		return -EOPNOTSUPP;
+
+	/*
+	 * The preemption timeout below is infinite, which is only safe where
+	 * no dma-fence waits for the queue to be preempted: a fault mode vm
+	 * has no preempt fences.
+	 */
+	if (XE_IOCTL_DBG(xe, !xe_exec_queue_is_lr(q) ||
+			 !xe_vm_in_fault_mode(q->vm)))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, xe_exec_queue_is_parallel(q)))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe,
+			 !(value & DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE)))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, !xe_eudebug_is_enabled(xe)))
+		return -EPERM;
+
+	q->eudebug_flags = EXEC_QUEUE_EUDEBUG_FLAG_ENABLE;
+	q->sched_props.preempt_timeout_us = 0;
+
+	return 0;
+}
+
+/**
+ * xe_exec_queue_is_debuggable() - Whether an exec_queue is debuggable
+ * @q: The exec_queue
+ *
+ * Return: True if @q was created with %DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG,
+ * false otherwise.
+ */
+bool xe_exec_queue_is_debuggable(struct xe_exec_queue *q)
+{
+	return q->eudebug_flags & EXEC_QUEUE_EUDEBUG_FLAG_ENABLE;
+}
+
 typedef int (*xe_exec_queue_set_property_fn)(struct xe_device *xe,
 					     struct xe_exec_queue *q,
 					     u64 value);
@@ -1064,6 +1119,7 @@ static const xe_exec_queue_set_property_fn exec_queue_set_property_funcs[] = {
 							exec_queue_set_multi_queue_priority,
 	[DRM_XE_EXEC_QUEUE_SET_DISABLE_STATE_CACHE_PERF_FIX] =
 							exec_queue_set_state_cache_perf_fix,
+	[DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG] = exec_queue_set_eudebug,
 };
 
 /**
@@ -1136,6 +1192,28 @@ static int exec_queue_user_ext_check(struct xe_exec_queue *q, u64 properties)
 			 (properties & BIT_ULL(DRM_XE_EXEC_QUEUE_SET_PROPERTY_PXP_TYPE))))
 		return -EINVAL;
 
+	/*
+	 * A debugger can inspect and modify the EU state and memory of a
+	 * debuggable queue, so it must not be combined with protected content.
+	 * Test the property mask rather than the queue state, as the extensions
+	 * are applied in the order userspace chained them and either one may be
+	 * processed first.
+	 */
+	if (XE_IOCTL_DBG(xe, (properties & BIT_ULL(DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG)) &&
+			 (properties & BIT_ULL(DRM_XE_EXEC_QUEUE_SET_PROPERTY_PXP_TYPE))))
+		return -EINVAL;
+
+	/*
+	 * EUDEBUG turns on runalone, which lets a single engine of the group
+	 * execute at a time, while a multi-queue group is there to run several
+	 * in parallel. Reject the combination. Testing the property mask
+	 * covers the primary as well, and does not depend on the order in
+	 * which userspace chained the extensions.
+	 */
+	if (XE_IOCTL_DBG(xe, (properties & BIT_ULL(DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG)) &&
+			 (properties & BIT_ULL(DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_GROUP))))
+		return -EINVAL;
+
 	return 0;
 }
 
@@ -1171,7 +1249,8 @@ static int exec_queue_user_ext_set_property(struct xe_device *xe,
 			 ext.property != DRM_XE_EXEC_QUEUE_SET_HANG_REPLAY_STATE &&
 			 ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_GROUP &&
 			 ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE_PRIORITY &&
-			 ext.property != DRM_XE_EXEC_QUEUE_SET_DISABLE_STATE_CACHE_PERF_FIX))
+			 ext.property != DRM_XE_EXEC_QUEUE_SET_DISABLE_STATE_CACHE_PERF_FIX &&
+			 ext.property != DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG))
 		return -EINVAL;
 
 	idx = array_index_nospec(ext.property, ARRAY_SIZE(exec_queue_set_property_funcs));
diff --git a/drivers/gpu/drm/xe/xe_exec_queue.h b/drivers/gpu/drm/xe/xe_exec_queue.h
index e8963f85cabd..95a0d84b9590 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue.h
+++ b/drivers/gpu/drm/xe/xe_exec_queue.h
@@ -162,4 +162,6 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch);
 struct xe_lrc *xe_exec_queue_lrc(struct xe_exec_queue *q);
 struct xe_lrc *xe_exec_queue_get_lrc(struct xe_exec_queue *q, u16 idx);
 
+bool xe_exec_queue_is_debuggable(struct xe_exec_queue *q);
+
 #endif
diff --git a/drivers/gpu/drm/xe/xe_exec_queue_types.h b/drivers/gpu/drm/xe/xe_exec_queue_types.h
index 48c05398b016..32893341287d 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue_types.h
+++ b/drivers/gpu/drm/xe/xe_exec_queue_types.h
@@ -155,6 +155,13 @@ struct xe_exec_queue {
 	/** @ban_reason: Bitmask of ban reasons (DRM_XE_EXEC_QUEUE_BAN_REASON_*) */
 	atomic_t ban_reason;
 
+	/**
+	 * @eudebug_flags: immutable eudebug flags for this exec queue.
+	 * Set up with DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG.
+	 */
+#define EXEC_QUEUE_EUDEBUG_FLAG_ENABLE		BIT(0)
+	unsigned long eudebug_flags;
+
 	union {
 		/** @execlist: execlist backend specific state for exec queue */
 		struct xe_execlist_exec_queue *execlist;
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h
index c3ee37f8cfc0..532d69677981 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.h
+++ b/drivers/gpu/drm/xe/xe_hw_engine.h
@@ -78,4 +78,13 @@ enum xe_force_wake_domains xe_hw_engine_to_fw_domain(struct xe_hw_engine *hwe);
 
 u32 xe_hw_engine_mmio_read32(struct xe_hw_engine *hwe, struct xe_reg reg);
 
+static inline bool xe_engine_supports_eudebug(const enum xe_engine_class ec)
+{
+	if (ec == XE_ENGINE_CLASS_COMPUTE ||
+	    ec == XE_ENGINE_CLASS_RENDER)
+		return true;
+
+	return false;
+}
+
 #endif
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index e84b11324f08..1a157bf1589c 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1396,6 +1396,16 @@ struct drm_xe_vm_get_property {
  *  - %DRM_XE_EXEC_QUEUE_SET_DISABLE_STATE_CACHE_PERF_FIX - Set the queue to
  *    enable render color cache keying on BTP+BTI instead of just BTI
  *    (only valid for render queues).
+ *  - %DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG - Make the queue debuggable
+ *    by an EU debugger. The queue runs in runalone mode, one engine at
+ *    a time, with a preemption timeout of 0. Fails with -EINVAL unless
+ *    the value is exactly %DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE and the
+ *    queue is a width 1 render or compute queue on a VM created with
+ *    %DRM_XE_VM_CREATE_FLAG_FAULT_MODE,
+ *    not combined with %DRM_XE_EXEC_QUEUE_SET_PROPERTY_PXP_TYPE or
+ *    %DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_GROUP. Fails with
+ *    -EOPNOTSUPP if the kernel is built without CONFIG_DRM_XE_EUDEBUG,
+ *    and with -EPERM if eudebug is not enabled through sysfs.
  *
  * The example below shows how to use &struct drm_xe_exec_queue_create to create
  * a simple exec_queue (no parallel submission) of class
@@ -1443,6 +1453,8 @@ struct drm_xe_exec_queue_create {
 #define     DRM_XE_MULTI_GROUP_CREATE				(1ull << 63)
 #define   DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE_PRIORITY	5
 #define   DRM_XE_EXEC_QUEUE_SET_DISABLE_STATE_CACHE_PERF_FIX	6
+#define   DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG		7
+#define     DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE		(1 << 0)
 	/** @extensions: Pointer to the first extension struct, if any */
 	__u64 extensions;
 
-- 
2.53.0

[PATCH 08/20] drm/xe/eudebug: Introduce exec_queue events

[Mika Kuoppala]

From: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

Add events to inform the debugger about the creation and destruction of
exec_queues. Use user engine class types instead of the internal
xe_engine_class enum in exec_queue events. During discovery, only advertise
exec_queues with render or compute class, excluding others.

v2: - Only track long running queues
    - Checkpatch (Tilak)
v3: __counted_by added
v4: - use helpers for filtering engines (Mika)
v5: - fix lrc handles memcpy size bug (Mika)
v6: - lookup for create event (Sashiko)
    - defer engine class lookup (Sashiko)
    - avoid uninitialized read on adding handle (Sashiko)
    - too strict warn on q->width check (Sashiko)
    - create eudebug eq event early (Claude)
    - dont drop session if vm is already gone (Claude)
v7: - lookups under the debugger lock (Mika)

Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/xe_eudebug.rst   |   3 +
 drivers/gpu/drm/xe/xe_eudebug.c       | 282 +++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_eudebug.h       |   7 +
 drivers/gpu/drm/xe/xe_eudebug_types.h |   6 +-
 drivers/gpu/drm/xe/xe_exec_queue.c    |   8 +-
 include/uapi/drm/xe_drm_eudebug.h     |  31 +++
 6 files changed, 330 insertions(+), 7 deletions(-)

diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index 9c6070784e84..f3152a13e250 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -65,3 +65,6 @@ Resource Event Types
 
 .. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
    :identifiers: drm_xe_eudebug_event_vm
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event_exec_queue
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index e0ef34588d3d..0f8b0452f9ba 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -16,6 +16,8 @@
 #include "xe_device.h"
 #include "xe_eudebug.h"
 #include "xe_eudebug_types.h"
+#include "xe_exec_queue.h"
+#include "xe_hw_engine.h"
 #include "xe_macros.h"
 #include "xe_sriov_pf.h"
 #include "xe_vm.h"
@@ -486,6 +488,28 @@ __find_handle(struct xe_eudebug_resource *r,
 	return h;
 }
 
+static int find_handle(struct xe_eudebug *d,
+		       const int type,
+		       const void *p)
+{
+	const u64 key = (uintptr_t)p;
+	struct xe_eudebug_resource *r;
+	struct xe_eudebug_handle *h;
+	int id;
+
+	if (XE_WARN_ON(!key))
+		return -EINVAL;
+
+	r = resource_from_type(d, type);
+
+	mutex_lock(&d->lock);
+	h = __find_handle(r, key);
+	id = h ? h->id : -ENOENT;
+	mutex_unlock(&d->lock);
+
+	return id;
+}
+
 /*
  * Adding a handle is split around d->lock: handle_prepare() allocates the
  * handle and reserves its id, __handle_insert() publishes it and stamps
@@ -799,6 +823,244 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)
 	xe_eudebug_put(d);
 }
 
+static const u16 xe_to_user_engine_class[] = {
+	[XE_ENGINE_CLASS_RENDER] = DRM_XE_ENGINE_CLASS_RENDER,
+	[XE_ENGINE_CLASS_COPY] = DRM_XE_ENGINE_CLASS_COPY,
+	[XE_ENGINE_CLASS_VIDEO_DECODE] = DRM_XE_ENGINE_CLASS_VIDEO_DECODE,
+	[XE_ENGINE_CLASS_VIDEO_ENHANCE] = DRM_XE_ENGINE_CLASS_VIDEO_ENHANCE,
+	[XE_ENGINE_CLASS_COMPUTE] = DRM_XE_ENGINE_CLASS_COMPUTE,
+};
+
+static int send_exec_queue_event(struct xe_eudebug *d, u32 flags,
+				 u64 vm_handle, u64 exec_queue_handle,
+				 enum xe_engine_class class,
+				 u32 width, u64 *lrc_handles, u64 seqno)
+{
+	struct drm_xe_eudebug_event *event;
+	struct drm_xe_eudebug_event_exec_queue *e;
+	const u32 sz = struct_size(e, lrc_handle, width);
+	u32 xe_engine_class;
+	int err;
+
+	if (!xe_engine_supports_eudebug(class))
+		return -EINVAL;
+
+	xe_engine_class = xe_to_user_engine_class[class];
+
+	mutex_lock(&d->lock);
+	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE,
+					 seqno, flags, sz);
+	e = cast_event(e, event);
+
+	e->vm_handle = vm_handle;
+	e->exec_queue_handle = exec_queue_handle;
+	e->engine_class = xe_engine_class;
+	e->width = width;
+
+	memcpy(e->lrc_handle, lrc_handles, width * sizeof(u64));
+
+	err = xe_eudebug_queue_event(d, event);
+	mutex_unlock(&d->lock);
+
+	return err;
+}
+
+/*
+ * A debuggable queue needs only an LR vm, and a fault mode vm can be
+ * destroyed while its queues live on: xe_vm_destroy_ioctl() only refuses
+ * a vm with preempt fence mode queues. The debugger has then already seen
+ * the vm go, so report such a queue against vm handle 0 rather than
+ * treating the missing handle as a diverged resource map.
+ *
+ * Looked up in the critical section that stamps the queue event's seqno.
+ * A vm destroy removes the handle and stamps its own seqno under the same
+ * lock, so the handle is gone exactly when the vm destroy came first in
+ * seqno order, and a debugger replaying the events never sees a queue
+ * event on a vm it has dropped, or vm 0 for one it has not.
+ */
+static int __exec_queue_vm_handle(struct xe_eudebug *d, struct xe_exec_queue *q)
+{
+	struct xe_eudebug_handle *h;
+
+	lockdep_assert_held(&d->lock);
+
+	h = __find_handle(resource_from_type(d, XE_EUDEBUG_RES_TYPE_VM),
+			  (uintptr_t)q->vm);
+
+	return h ? h->id : 0;
+}
+
+static int exec_queue_create_event(struct xe_eudebug *d,
+				   struct xe_file *xef, struct xe_exec_queue *q)
+{
+	struct xe_eudebug_handle *h;
+	int h_vm, h_queue;
+	u64 h_lrc[XE_HW_ENGINE_MAX_INSTANCE], seqno;
+	int i;
+	int ret;
+
+	if (!xe_exec_queue_is_debuggable(q))
+		return 0;
+
+	if (XE_WARN_ON(q->width > XE_HW_ENGINE_MAX_INSTANCE))
+		return -EINVAL;
+
+	ret = 0;
+	for (i = 0; i < q->width; i++) {
+		ret = _xe_eudebug_add_handle(d,
+					     XE_EUDEBUG_RES_TYPE_LRC,
+					     q->lrc[i],
+					     NULL);
+
+		if (ret < 0)
+			break;
+
+		XE_WARN_ON(!ret);
+
+		h_lrc[i] = ret;
+	}
+
+	if (ret < 0)
+		return ret;
+
+	h = handle_prepare(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE, q);
+	if (IS_ERR(h))
+		return PTR_ERR(h);
+
+	scoped_guard(mutex, &d->lock) {
+		h_queue = __handle_insert(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE, h,
+					  &seqno);
+		h_vm = __exec_queue_vm_handle(d, q);
+	}
+
+	eu_dbg(d, "exec queue %p added: %d, vm %d\n", q, h_queue, h_vm);
+
+	if (h_queue < 0) {
+		handle_abort(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE, h);
+		return h_queue;
+	}
+
+	/* No need to cleanup for added handles on error as if we fail
+	 * we disconnect
+	 */
+
+	ret = send_exec_queue_event(d, DRM_XE_EUDEBUG_EVENT_CREATE,
+				    h_vm, h_queue, q->class,
+				    q->width, h_lrc, seqno);
+
+	if (ret)
+		eu_dbg(d, "send_exec_queue_event create error %d", ret);
+
+	return ret;
+}
+
+static int exec_queue_destroy_event(struct xe_eudebug *d,
+				    struct xe_file *xef,
+				    struct xe_exec_queue *q)
+{
+	int h_vm, h_queue;
+	u64 h_lrc[XE_HW_ENGINE_MAX_INSTANCE], seqno;
+	int i;
+	int ret, err = 0;
+
+	if (!xe_exec_queue_is_debuggable(q))
+		return 0;
+
+	/*
+	 * A queue we never handed out to the debugger needs no destroy event.
+	 * The queue handle is added last on create, so its presence implies
+	 * the lrc handles are there too. Past this point a missing handle
+	 * means our resource map and the client have diverged, and that has
+	 * to drop the session. The exception is a concurrent detach, which
+	 * empties the maps without the ioctl_lock, but only after marking us
+	 * detached.
+	 */
+	if (find_handle(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE, q) == -ENOENT)
+		return 0;
+
+	if (XE_WARN_ON(q->width > XE_HW_ENGINE_MAX_INSTANCE))
+		return -EINVAL;
+
+	for (i = 0; i < q->width; i++) {
+		ret = _xe_eudebug_remove_handle(d,
+						XE_EUDEBUG_RES_TYPE_LRC,
+						q->lrc[i],
+						NULL);
+		if (ret < 0) {
+			XE_WARN_ON(!xe_eudebug_is_detached(d));
+			if (!err)
+				err = ret;
+			h_lrc[i] = 0;
+			continue;
+		}
+
+		XE_WARN_ON(!ret);
+		h_lrc[i] = ret;
+	}
+
+	if (err)
+		return err;
+
+	scoped_guard(mutex, &d->lock) {
+		h_queue = __xe_eudebug_remove_handle(d,
+						     XE_EUDEBUG_RES_TYPE_EXEC_QUEUE,
+						     q, &seqno);
+		h_vm = __exec_queue_vm_handle(d, q);
+	}
+
+	eu_dbg(d, "exec queue %p removed: %d, vm %d\n", q, h_queue, h_vm);
+
+	if (h_queue < 0)
+		return h_queue;
+
+	ret = send_exec_queue_event(d, DRM_XE_EUDEBUG_EVENT_DESTROY,
+				    h_vm, h_queue, q->class,
+				    q->width, h_lrc, seqno);
+
+	if (ret)
+		eu_dbg(d, "send_exec_queue_event destroy error %d\n", ret);
+
+	return ret;
+}
+
+void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q)
+{
+	struct xe_eudebug *d;
+	int err;
+
+	if (!xe_engine_supports_eudebug(q->class))
+		return;
+
+	d = xe_eudebug_get(xef);
+	if (!d)
+		return;
+
+	err = exec_queue_create_event(d, xef, q);
+	if (err)
+		xe_eudebug_disconnect(d, err);
+
+	xe_eudebug_put(d);
+}
+
+void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q)
+{
+	struct xe_eudebug *d;
+	int err;
+
+	if (!xe_engine_supports_eudebug(q->class))
+		return;
+
+	d = xe_eudebug_get(xef);
+	if (!d)
+		return;
+
+	err = exec_queue_destroy_event(d, xef, q);
+	if (err)
+		xe_eudebug_disconnect(d, err);
+
+	xe_eudebug_put(d);
+}
+
 /*
  * All the relevant ioctls that manipulate the xe_file resources eudebug
  * cares about are gated behind down_read(&xef->eudebug.ioctl_lock). The
@@ -807,9 +1069,10 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)
  */
 static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
 {
+	struct xe_exec_queue *q;
 	struct xe_vm *vm;
 	unsigned long i;
-	unsigned int vm_count = 0;
+	unsigned int vm_count = 0, eq_count = 0;
 	int err = 0;
 
 	lockdep_assert_held_write(&xef->eudebug.ioctl_lock);
@@ -828,10 +1091,21 @@ static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
 		vm_count++;
 	}
 
+	xa_for_each(&xef->exec_queue.xa, i, q) {
+		if (!xe_engine_supports_eudebug(q->class))
+			continue;
+
+		err = exec_queue_create_event(d, xef, q);
+		if (err)
+			goto out;
+
+		eq_count++;
+	}
+
 out:
-	if (vm_count)
-		eu_dbg(d, "Discovery found %u vms",
-		       vm_count);
+	if (vm_count || eq_count)
+		eu_dbg(d, "Discovery found %u vms, %u exec_queues",
+		       vm_count, eq_count);
 
 	eu_dbg(d, "Discovery end for %llu: %d", d->session, err);
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index 413a45129193..ec07c567356d 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -14,6 +14,7 @@ struct drm_file;
 struct xe_device;
 struct xe_file;
 struct xe_vm;
+struct xe_exec_queue;
 struct xe_eudebug;
 
 #if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
@@ -49,6 +50,9 @@ void xe_eudebug_file_close(struct xe_file *xef);
 void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm);
 void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm);
 
+void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q);
+void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q);
+
 #else
 
 static inline int xe_eudebug_connect_ioctl(struct drm_device *dev,
@@ -64,6 +68,9 @@ static inline void xe_eudebug_file_close(struct xe_file *xef) { }
 static inline void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm) { }
 static inline void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm) { }
 
+static inline void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q) { }
+static inline void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q) { }
+
 #endif /* CONFIG_DRM_XE_EUDEBUG */
 
 #endif /* _XE_EUDEBUG_H_ */
diff --git a/drivers/gpu/drm/xe/xe_eudebug_types.h b/drivers/gpu/drm/xe/xe_eudebug_types.h
index 598abb249195..0cb1b3b678e4 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_types.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_types.h
@@ -31,7 +31,7 @@ enum xe_eudebug_cap_state {
 	XE_EUDEBUG_CAP_ENABLED,
 };
 
-#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_VM
+#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE
 
 /**
  * struct xe_eudebug_handle - eudebug resource handle
@@ -62,7 +62,9 @@ struct xe_eudebug_resource {
 };
 
 #define XE_EUDEBUG_RES_TYPE_VM		0
-#define XE_EUDEBUG_RES_TYPE_COUNT	(XE_EUDEBUG_RES_TYPE_VM + 1)
+#define XE_EUDEBUG_RES_TYPE_EXEC_QUEUE  1
+#define XE_EUDEBUG_RES_TYPE_LRC		2
+#define XE_EUDEBUG_RES_TYPE_COUNT	(XE_EUDEBUG_RES_TYPE_LRC + 1)
 
 /**
  * struct xe_eudebug - Top level struct for eudebug: the connection
diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c
index e65bda492e58..3eeb8a108b40 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue.c
+++ b/drivers/gpu/drm/xe/xe_exec_queue.c
@@ -1548,10 +1548,14 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,
 	if (eci[0].engine_class != DRM_XE_ENGINE_CLASS_VM_BIND)
 		xe_vm_add_exec_queue(vm, q);
 
+	xe_eudebug_exec_queue_create(xef, q);
+
 	/* user id alloc must always be last in ioctl to prevent UAF */
 	err = xa_alloc(&xef->exec_queue.xa, &id, q, xa_limit_32b, GFP_KERNEL);
-	if (err)
+	if (err) {
+		xe_eudebug_exec_queue_destroy(xef, q);
 		goto del_hw_engine_group;
+	}
 
 	args->exec_queue_id = id;
 
@@ -1736,6 +1740,8 @@ int xe_exec_queue_destroy_ioctl(struct drm_device *dev, void *data,
 	if (q->vm && q->hwe->hw_engine_group)
 		xe_hw_engine_group_del_exec_queue(q->hwe->hw_engine_group, q);
 
+	xe_eudebug_exec_queue_destroy(xef, q);
+
 	xe_exec_queue_kill(q);
 
 	trace_xe_exec_queue_close(q);
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index 63f4a6142a1a..12e905441f1d 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -78,6 +78,7 @@ struct drm_xe_eudebug_event {
 #define DRM_XE_EUDEBUG_EVENT_NONE		0
 #define DRM_XE_EUDEBUG_EVENT_READ		1
 #define DRM_XE_EUDEBUG_EVENT_VM			2
+#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE		3
 
 	/** @flags: Flags */
 	__u16 flags;
@@ -124,6 +125,36 @@ struct drm_xe_eudebug_event_vm {
 	__u64 vm_handle;
 };
 
+/**
+ * struct drm_xe_eudebug_event_exec_queue - Exec Queue resource event
+ *
+ * Resource creation/destruction event for an Exec Queue
+ */
+struct drm_xe_eudebug_event_exec_queue {
+	/** @base: base event */
+	struct drm_xe_eudebug_event base;
+
+	/**
+	 * @vm_handle: the vm handle this exec queue belongs to, or 0 if the
+	 * vm was destroyed before this event. A vm can be destroyed while
+	 * its exec queues live on, so a destroy event can carry 0 where the
+	 * create event carried the vm handle.
+	 */
+	__u64 vm_handle;
+
+	/** @exec_queue_handle: unique handle for this exec queue */
+	__u64 exec_queue_handle;
+
+	/** @engine_class: engine class, a %DRM_XE_ENGINE_CLASS_* value */
+	__u32 engine_class;
+
+	/** @width: width of exec queue, how many lrcs (handles) it has */
+	__u32 width;
+
+	/** @lrc_handle: array of lrc handles */
+	__u64 lrc_handle[] __counted_by(width);
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.53.0

[PATCH 09/20] drm/xe/eudebug: Mark guc contexts as debuggable

[Mika Kuoppala]

Inform the GuC which contexts are debuggable, as their
handling differs from ordinary contexts.

v2: void return, use xe_gt_dbg, no need for lrc (Matt)
v3: add the workaround enabling (Daniele)
v4: version needed to 70.49.4
v5: bail out early before registering eq (Daniele)
v6: export the guc action for future (Mika)
v7: remove redundant xe_gt_info (Sashiko)

Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Lucas De Marchi <lucas.demarchi@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Jan Sokolowski <jan.sokolowski@intel.com>
Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
---
 drivers/gpu/drm/xe/abi/guc_actions_abi.h |  5 ++++
 drivers/gpu/drm/xe/abi/guc_klvs_abi.h    |  1 +
 drivers/gpu/drm/xe/xe_exec_queue.c       |  5 ++++
 drivers/gpu/drm/xe/xe_guc.c              | 15 +++++++++++
 drivers/gpu/drm/xe/xe_guc.h              |  6 +++++
 drivers/gpu/drm/xe/xe_guc_ads.c          |  5 ++++
 drivers/gpu/drm/xe/xe_guc_submit.c       | 34 ++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_wa_oob.rules       |  2 ++
 include/uapi/drm/xe_drm.h                |  5 ++--
 9 files changed, 76 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/xe/abi/guc_actions_abi.h b/drivers/gpu/drm/xe/abi/guc_actions_abi.h
index f5c9b37038d4..48ba41fd8ccb 100644
--- a/drivers/gpu/drm/xe/abi/guc_actions_abi.h
+++ b/drivers/gpu/drm/xe/abi/guc_actions_abi.h
@@ -162,6 +162,7 @@ enum xe_guc_action {
 	XE_GUC_ACTION_NOTIFY_FLUSH_LOG_BUFFER_TO_FILE = 0x8003,
 	XE_GUC_ACTION_NOTIFY_CRASH_DUMP_POSTED = 0x8004,
 	XE_GUC_ACTION_NOTIFY_EXCEPTION = 0x8005,
+	XE_GUC_ACTION_EU_KERNEL_DEBUG = 0x8006,
 	XE_GUC_ACTION_TEST_G2G_SEND = 0xF001,
 	XE_GUC_ACTION_TEST_G2G_RECV = 0xF002,
 	XE_GUC_ACTION_LIMIT
@@ -285,4 +286,8 @@ enum xe_guc_g2g_type {
 /* invalid type for XE_GUC_ACTION_NOTIFY_MEMORY_CAT_ERROR */
 #define XE_GUC_CAT_ERR_TYPE_INVALID 0xdeadbeef
 
+enum xe_guc_eu_kernel_debug_request_type {
+	XE_GUC_EU_KERNEL_DEBUG_ENABLE = 0x3,
+};
+
 #endif
diff --git a/drivers/gpu/drm/xe/abi/guc_klvs_abi.h b/drivers/gpu/drm/xe/abi/guc_klvs_abi.h
index 4bcdc1a96785..147a1d0a1472 100644
--- a/drivers/gpu/drm/xe/abi/guc_klvs_abi.h
+++ b/drivers/gpu/drm/xe/abi/guc_klvs_abi.h
@@ -522,6 +522,7 @@ enum xe_guc_klv_ids {
 	GUC_WA_KLV_WAKE_POWER_DOMAINS_FOR_OUTBOUND_MMIO					= 0x900a,
 	GUC_WA_KLV_RESET_BB_STACK_PTR_ON_VF_SWITCH					= 0x900b,
 	GUC_WA_KLV_RESTORE_UNSAVED_MEDIA_CONTROL_REG					= 0x900c,
+	GUC_WA_KLV_RESET_DEP_ENGINES_ON_DEBUG_CTX_SWITCH				= 0x900d,
 	GUC_WA_KLV_CLR_CS_INDIRECT_RING_STATE_IF_IDLE_AT_CTX_REG			= 0x900e,
 	GUC_WA_KLV_REMAP_RANGED_TLB_INV							= 0x900f,
 	GUC_WA_KLV_IGNORE_MMIO_READ_SEM_TOKEN_64					= 0x9010,
diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c
index 3eeb8a108b40..870abbf08d2e 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue.c
+++ b/drivers/gpu/drm/xe/xe_exec_queue.c
@@ -21,6 +21,7 @@
 #include "xe_gt.h"
 #include "xe_gt_sriov_pf.h"
 #include "xe_gt_sriov_vf.h"
+#include "xe_guc.h"
 #include "xe_hw_engine_class_sysfs.h"
 #include "xe_hw_engine_group.h"
 #include "xe_irq.h"
@@ -1058,6 +1059,7 @@ static int exec_queue_set_eudebug(struct xe_device *xe, struct xe_exec_queue *q,
 				  u64 value)
 {
 	const u64 known_flags = DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE;
+	struct xe_guc *guc = &q->gt->uc.guc;
 
 	if (XE_IOCTL_DBG(xe, !xe_engine_supports_eudebug(q->class)))
 		return -EINVAL;
@@ -1068,6 +1070,9 @@ static int exec_queue_set_eudebug(struct xe_device *xe, struct xe_exec_queue *q,
 	if (XE_IOCTL_DBG(xe, !IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)))
 		return -EOPNOTSUPP;
 
+	if (XE_IOCTL_DBG(xe, !xe_guc_has_debug_contexts(guc)))
+		return -EOPNOTSUPP;
+
 	/*
 	 * The preemption timeout below is infinite, which is only safe where
 	 * no dma-fence waits for the queue to be preempted: a fault mode vm
diff --git a/drivers/gpu/drm/xe/xe_guc.c b/drivers/gpu/drm/xe/xe_guc.c
index 300126e6624f..8b9af0c483b5 100644
--- a/drivers/gpu/drm/xe/xe_guc.c
+++ b/drivers/gpu/drm/xe/xe_guc.c
@@ -823,6 +823,13 @@ int xe_guc_init(struct xe_guc *guc)
 		return 0;
 	}
 
+	if (IS_ENABLED(CONFIG_DRM_XE_EUDEBUG) && XE_GT_WA(gt, 14022766366) &&
+	    !xe_guc_has_debug_contexts(guc))
+		xe_gt_info(gt, "eudebug needs GuC version %u.%u.%u or greater\n",
+			   XE_GUC_VER_DEBUG_CONTEXTS_MAJOR,
+			   XE_GUC_VER_DEBUG_CONTEXTS_MINOR,
+			   XE_GUC_VER_DEBUG_CONTEXTS_PATCH);
+
 	ret = xe_guc_log_init(&guc->log);
 	if (ret)
 		goto out;
@@ -1959,6 +1966,14 @@ u16 xe_hwe_guc_logical_instance(struct xe_hw_engine *hwe)
 	return hwe->logical_instance;
 }
 
+bool xe_guc_has_debug_contexts(struct xe_guc *guc)
+{
+	return GUC_FIRMWARE_VER_AT_LEAST(guc,
+					 XE_GUC_VER_DEBUG_CONTEXTS_MAJOR,
+					 XE_GUC_VER_DEBUG_CONTEXTS_MINOR,
+					 XE_GUC_VER_DEBUG_CONTEXTS_PATCH);
+}
+
 #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
 #include "tests/xe_guc_g2g_test.c"
 #endif
diff --git a/drivers/gpu/drm/xe/xe_guc.h b/drivers/gpu/drm/xe/xe_guc.h
index 543ba4e5981e..79d76970995d 100644
--- a/drivers/gpu/drm/xe/xe_guc.h
+++ b/drivers/gpu/drm/xe/xe_guc.h
@@ -29,6 +29,11 @@
 #define GUC_FIRMWARE_VER_AT_LEAST(guc, ver...) \
 	xe_guc_fw_version_at_least((guc), MAKE_GUC_VER_ARGS(ver))
 
+/* Lowest GuC version that knows how to handle debuggable contexts */
+#define XE_GUC_VER_DEBUG_CONTEXTS_MAJOR	70
+#define XE_GUC_VER_DEBUG_CONTEXTS_MINOR	49
+#define XE_GUC_VER_DEBUG_CONTEXTS_PATCH	4
+
 struct drm_printer;
 struct xe_gt;
 
@@ -66,6 +71,7 @@ int xe_guc_start(struct xe_guc *guc);
 void xe_guc_wedge_declare(struct xe_guc *guc);
 bool xe_guc_using_main_gamctrl_queues(struct xe_guc *guc);
 bool xe_guc_has_paging_engine(struct xe_guc *guc);
+bool xe_guc_has_debug_contexts(struct xe_guc *guc);
 
 #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
 int xe_guc_g2g_test_notification(struct xe_guc *guc, u32 *payload, u32 len);
diff --git a/drivers/gpu/drm/xe/xe_guc_ads.c b/drivers/gpu/drm/xe/xe_guc_ads.c
index c3ce3c8c905a..0b1bdb40230b 100644
--- a/drivers/gpu/drm/xe/xe_guc_ads.c
+++ b/drivers/gpu/drm/xe/xe_guc_ads.c
@@ -402,6 +402,11 @@ static void guc_waklv_init(struct xe_guc_ads *ads)
 				 GUC_FEATURE_KLV_DISABLE_MULTI_QUEUE);
 	}
 
+	if (IS_ENABLED(CONFIG_DRM_XE_EUDEBUG) && XE_GT_WA(gt, 14022766366) &&
+	    xe_guc_has_debug_contexts(&gt->uc.guc))
+		guc_waklv_enable(ads, NULL, 0, &offset, &remain,
+				 GUC_WA_KLV_RESET_DEP_ENGINES_ON_DEBUG_CTX_SWITCH);
+
 	size = guc_ads_waklv_size(ads) - remain;
 	if (!size)
 		return;
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index b14b37b767f5..61a7d731f7d1 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -1023,6 +1023,37 @@ static u32 xe_hwe_guc_logical_to_submit_mask(struct xe_hw_engine *hwe, u32 logic
 	return logical_mask;
 }
 
+static int guc_action_eu_kernel_debug(struct xe_guc *guc, u32 id, u32 cmd)
+{
+	const u32 action[] = {
+		XE_GUC_ACTION_EU_KERNEL_DEBUG,
+		id,
+		cmd,
+		0, /* reserved */
+	};
+
+	return xe_guc_ct_send(&guc->ct, action,
+			      ARRAY_SIZE(action), 0, 0);
+}
+
+static void set_eu_kernel_debug(struct xe_exec_queue *q)
+{
+	struct xe_guc *guc = exec_queue_to_guc(q);
+	struct xe_gt *gt = guc_to_gt(guc);
+	int ret;
+
+	ret = guc_action_eu_kernel_debug(guc, q->guc->id,
+					 XE_GUC_EU_KERNEL_DEBUG_ENABLE);
+
+	if (ret)
+		xe_gt_err_ratelimited(gt,
+				      "GuC ctx=%u debug enabling failed with %d\n",
+				      q->guc->id, ret);
+	else
+		xe_gt_dbg(gt,
+			  "GuC ctx=%u debug enable requested\n", q->guc->id);
+}
+
 static void register_exec_queue(struct xe_exec_queue *q, int ctx_type)
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
@@ -1084,6 +1115,9 @@ static void register_exec_queue(struct xe_exec_queue *q, int ctx_type)
 
 	if (xe_exec_queue_is_multi_queue_secondary(q))
 		guc_exec_queue_send_cgp_sync(q, CGP_SYNC_REGISTRATION);
+
+	if (xe_exec_queue_is_debuggable(q))
+		set_eu_kernel_debug(q);
 }
 
 static u32 wq_space_until_wrap(struct xe_exec_queue *q)
diff --git a/drivers/gpu/drm/xe/xe_wa_oob.rules b/drivers/gpu/drm/xe/xe_wa_oob.rules
index 61aa50a63764..da774e17cb69 100644
--- a/drivers/gpu/drm/xe/xe_wa_oob.rules
+++ b/drivers/gpu/drm/xe/xe_wa_oob.rules
@@ -78,3 +78,5 @@
 		MEDIA_VERSION_RANGE(1301, 3503), FUNC(xe_rtp_match_not_sriov_vf)
 16030224309	GRAPHICS_VERSION_RANGE(3510, 3511)
 14028709524	GRAPHICS_VERSION(3510)
+14022766366	GRAPHICS_VERSION_RANGE(2001, 2004)
+		GRAPHICS_VERSION_RANGE(3000, 3005)
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index 1a157bf1589c..759e76b7ac52 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -1404,8 +1404,9 @@ struct drm_xe_vm_get_property {
  *    %DRM_XE_VM_CREATE_FLAG_FAULT_MODE,
  *    not combined with %DRM_XE_EXEC_QUEUE_SET_PROPERTY_PXP_TYPE or
  *    %DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_GROUP. Fails with
- *    -EOPNOTSUPP if the kernel is built without CONFIG_DRM_XE_EUDEBUG,
- *    and with -EPERM if eudebug is not enabled through sysfs.
+ *    -EOPNOTSUPP if the kernel is built without CONFIG_DRM_XE_EUDEBUG
+ *    or the GuC does not support debug contexts, and with -EPERM if
+ *    eudebug is not enabled through sysfs.
  *
  * The example below shows how to use &struct drm_xe_exec_queue_create to create
  * a simple exec_queue (no parallel submission) of class
-- 
2.53.0

[PATCH 10/20] drm/xe: Introduce ADD_DEBUG_DATA and REMOVE_DEBUG_DATA vm bind ops

[Mika Kuoppala]

From: Christoph Manszewski <christoph.manszewski@intel.com>

Make it possible to add and remove per vm debug data, which can be used
to annotate vm ranges (using pseudopaths) or to associate them with
a file which can carry arbitrary debug data (e.g. binary instruction to
code line mapping). The debug data is kept separate from the vmas. Each
address can be associated with only one debug data entry i.e. debug data
entries cannot overlap. Each entry is atomic, so the creation address
and range have to be passed in order to remove it.

For debug data manipulation only the 'op' and 'extensions' fields from
'struct drm_xe_vm_bind_op' are used. All required parameters are passed
through 'struct drm_xe_vm_bind_op_ext_debug_data' and a valid instance
should be present in the extension chain pointed to by the 'extensions'
field.

Debug data will be accessible through the eudebug event interface,
introduced in the following patch. An alternative way to access debug data
using debugfs, without relying on eudebug, will be proposed as a follow-up
to the eudebug series.

v2: enforce empty path on unmap (Joonas, Mika)
v3: - fix overflow, deny zero ranges, null str for empty check (Sashiko)
    - fix TOCTOUs and list handling issues (Sashiko)
v4: - check full path for zero, overflow check (Sashiko)
    - don't unwind failed extensions, use op commit (Sashiko)
    - restore removes by relink, remove fallible alloc (Sashiko)

Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 drivers/gpu/drm/xe/Makefile              |   1 +
 drivers/gpu/drm/xe/xe_debug_data.c       | 388 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_debug_data.h       |  22 ++
 drivers/gpu/drm/xe/xe_debug_data_types.h |  26 ++
 drivers/gpu/drm/xe/xe_vm.c               | 226 ++++++++++++-
 drivers/gpu/drm/xe/xe_vm_types.h         |  31 ++
 include/uapi/drm/xe_drm.h                |  88 ++++-
 7 files changed, 777 insertions(+), 5 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_debug_data.c
 create mode 100644 drivers/gpu/drm/xe/xe_debug_data.h
 create mode 100644 drivers/gpu/drm/xe/xe_debug_data_types.h

diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index d42620e10c26..aee129a48040 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -36,6 +36,7 @@ xe-y += xe_bb.o \
 	xe_bo.o \
 	xe_bo_evict.o \
 	xe_cpu_bind.o \
+	xe_debug_data.o \
 	xe_dep_scheduler.o \
 	xe_devcoredump.o \
 	xe_device.o \
diff --git a/drivers/gpu/drm/xe/xe_debug_data.c b/drivers/gpu/drm/xe/xe_debug_data.c
new file mode 100644
index 000000000000..f01b28ff496d
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_debug_data.c
@@ -0,0 +1,388 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2025 Intel Corporation
+ */
+
+#include "xe_debug_data.h"
+#include "xe_debug_data_types.h"
+#include "xe_vm.h"
+
+const char *xe_debug_data_pseudo_path_to_string(u64 pseudopath)
+{
+	switch (pseudopath) {
+	case DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_MODULE_AREA:
+		return "[module_area]";
+	case DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SBA_AREA:
+		return "[sba_area]";
+	case DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SIP_AREA:
+		return "[sip_area]";
+	default:
+		return "[unknown]";
+	}
+}
+
+static const char *
+debug_data_path_str(u64 flags, u64 pseudopath, const char *pathname)
+{
+	return (flags & DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO) ?
+		xe_debug_data_pseudo_path_to_string(pseudopath) : pathname;
+}
+
+static bool
+debug_data_overlaps(const struct drm_xe_vm_bind_op_ext_debug_data *a,
+		    const struct xe_debug_data *b)
+{
+	u64 a_end, b_end;
+
+	/*
+	 * Ranges that wrap u64 are rejected by debug_data_ext_validate()
+	 * before we get here, so neither side should ever overflow. Be
+	 * defensive anyway and report overlap, which makes callers reject
+	 * the bind rather than act on a bogus range.
+	 */
+	if (check_add_overflow(a->addr, a->range, &a_end) ||
+	    check_add_overflow(b->addr, b->range, &b_end))
+		return true;
+
+	return a->addr < b_end && b->addr < a_end;
+}
+
+static bool
+debug_data_matches(const struct drm_xe_vm_bind_op_ext_debug_data *a,
+		   const struct xe_debug_data *b)
+{
+	return a->addr == b->addr && a->range == b->range;
+}
+
+static bool
+debug_data_is_empty(const struct drm_xe_vm_bind_op_ext_debug_data *ext)
+{
+	return !ext->flags && !ext->offset && !ext->reserved &&
+		mem_is_zero(ext->pathname, sizeof(ext->pathname));
+}
+
+/*
+ * Validate a debug data extension copied from userspace.
+ *
+ * This is the single source of truth for what the extension is allowed
+ * to contain. It is called from the check phase and again from the
+ * commit phase (on a freshly-copied snapshot) so that the kernel never
+ * trusts the result of an earlier copy_from_user(): see Sashiko's TOCTOU
+ * comment on v1.
+ */
+static int
+debug_data_ext_validate(struct xe_device *xe, u32 operation,
+			const struct drm_xe_vm_bind_op_ext_debug_data *ext)
+{
+	u64 end;
+
+	if (XE_IOCTL_DBG(xe, operation != DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA &&
+			 operation != DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, ext->reserved))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, check_add_overflow(ext->addr, ext->range, &end)))
+		return -EINVAL;
+
+	/* Remove takes only addr+range; everything else must be zero. */
+	if (operation == DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA) {
+		if (XE_IOCTL_DBG(xe, !debug_data_is_empty(ext)))
+			return -EINVAL;
+		if (XE_IOCTL_DBG(xe, !ext->range))
+			return -EINVAL;
+		return 0;
+	}
+
+	/* Add. */
+	if (XE_IOCTL_DBG(xe, !ext->range))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, ext->flags & ~DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO))
+		return -EINVAL;
+
+	if (ext->flags & DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO) {
+		if (XE_IOCTL_DBG(xe, ext->offset))
+			return -EINVAL;
+		if (XE_IOCTL_DBG(xe,
+				 ext->pseudopath < DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_MODULE_AREA ||
+				 ext->pseudopath > DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SIP_AREA))
+			return -EINVAL;
+		if (XE_IOCTL_DBG(xe, !mem_is_zero(ext->pathname + sizeof(ext->pseudopath),
+						  sizeof(ext->pathname) -
+						  sizeof(ext->pseudopath))))
+			return -EINVAL;
+	} else {
+		if (XE_IOCTL_DBG(xe, !ext->pathname[0]))
+			return -EINVAL;
+		if (XE_IOCTL_DBG(xe, strnlen(ext->pathname, sizeof(ext->pathname)) >=
+				 sizeof(ext->pathname)))
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+int xe_debug_data_check_extension(struct xe_vm *vm, u32 operation, u64 extension)
+{
+	const void __user * const address = u64_to_user_ptr(extension);
+	struct drm_xe_vm_bind_op_ext_debug_data *ext;
+	int ret;
+
+	ext = kzalloc_obj(*ext, GFP_KERNEL);
+	if (!ext)
+		return -ENOMEM;
+
+	if (copy_from_user(ext, address, sizeof(*ext))) {
+		ret = -EFAULT;
+		goto out;
+	}
+
+	ret = debug_data_ext_validate(vm->xe, operation, ext);
+	/*
+	 * Intentionally no list walk here: any check we make against
+	 * vm->debug_data.list outside the commit critical section can be
+	 * invalidated before commit runs (concurrent ioctls). The commit
+	 * phase re-validates and re-checks under the list lock; this
+	 * function exists only for early rejection of syntactically
+	 * malformed input.
+	 */
+out:
+	kfree(ext);
+	return ret;
+}
+
+static int xe_debug_data_add(struct xe_vm *vm, struct xe_vma_op *vma_op,
+			     struct drm_xe_vm_bind_op_ext_debug_data *ext)
+{
+	const bool pseudo = ext->flags & DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO;
+	/* validate() has checked that the path is NUL terminated */
+	const size_t path_len = pseudo ? 0 : strlen(ext->pathname);
+	struct xe_debug_data *dd, *iter;
+	int ret = 0;
+
+	vm_dbg(&vm->xe->drm,
+	       "ADD_DEBUG_DATA: addr=0x%016llx, range=0x%016llx, offset=0x%08llx, flags=0x%016llx, path=%s\n",
+	       ext->addr, ext->range, ext->offset, ext->flags,
+	       debug_data_path_str(ext->flags, ext->pseudopath, ext->pathname));
+
+	/* The pathname would otherwise make every entry a kmalloc-8k one */
+	dd = kzalloc_flex(*dd, pathname, path_len + 1, GFP_KERNEL_ACCOUNT);
+	if (!dd)
+		return -ENOMEM;
+
+	dd->addr = ext->addr;
+	dd->range = ext->range;
+	dd->flags = ext->flags;
+	dd->offset = ext->offset;
+
+	if (pseudo)
+		dd->pseudopath = ext->pseudopath;
+	else
+		memcpy(dd->pathname, ext->pathname, path_len);
+
+	/*
+	 * Overlap-check-and-insert atomically under the list lock.
+	 * Doing the check in a separate critical section would race
+	 * with concurrent adds on the same vm.
+	 */
+	mutex_lock(&vm->debug_data.lock);
+	list_for_each_entry(iter, &vm->debug_data.list, link) {
+		if (debug_data_matches(ext, iter)) {
+			ret = -EEXIST;
+			goto err_unlock;
+		}
+		if (debug_data_overlaps(ext, iter)) {
+			ret = -EINVAL;
+			goto err_unlock;
+		}
+	}
+	list_add_tail(&dd->link, &vm->debug_data.list);
+	mutex_unlock(&vm->debug_data.lock);
+
+	/*
+	 * Borrowed reference for the unwind path and for the bind event.
+	 * The list owns it from here, so this must not be freed through
+	 * the op.
+	 */
+	vma_op->modify_debug_data.added = dd;
+
+	vma_op->flags |= XE_VMA_OP_COMMITTED;
+
+	return 0;
+
+err_unlock:
+	mutex_unlock(&vm->debug_data.lock);
+	kfree(dd);
+	return ret;
+}
+
+static int xe_debug_data_remove(struct xe_vm *vm, struct xe_vma_op *vma_op,
+				struct drm_xe_vm_bind_op_ext_debug_data *ext)
+{
+	struct xe_debug_data *dd, *found = NULL;
+
+	vm_dbg(&vm->xe->drm,
+	       "REMOVE_DEBUG_DATA: addr=0x%016llx, range=0x%016llx\n",
+	       ext->addr, ext->range);
+
+	mutex_lock(&vm->debug_data.lock);
+	list_for_each_entry(dd, &vm->debug_data.list, link) {
+		if (debug_data_matches(ext, dd)) {
+			list_del(&dd->link);
+			found = dd;
+			break;
+		}
+	}
+	mutex_unlock(&vm->debug_data.lock);
+
+	if (!found) {
+		drm_dbg(&vm->xe->drm,
+			"Debug data to remove not found: addr=0x%llx range=0x%llx\n",
+			ext->addr, ext->range);
+		return -ENOENT;
+	}
+
+	vma_op->modify_debug_data.removed = found;
+	vma_op->flags |= XE_VMA_OP_COMMITTED;
+
+	return 0;
+}
+
+int xe_debug_data_process_extension(struct xe_vm *vm, struct drm_gpuva_ops *ops,
+				    u32 operation, u64 extension)
+{
+	const void __user * const address = u64_to_user_ptr(extension);
+	struct drm_xe_vm_bind_op_ext_debug_data *ext;
+	struct xe_vma_op *vma_op;
+	struct drm_gpuva_op *op;
+	int ret;
+
+	ext = kzalloc_obj(*ext, GFP_KERNEL);
+	if (!ext)
+		return -ENOMEM;
+
+	if (copy_from_user(ext, address, sizeof(*ext))) {
+		ret = -EFAULT;
+		goto out;
+	}
+
+	/*
+	 * Re-validate the freshly-copied extension. The check phase
+	 * cannot be trusted: userspace may have mutated the backing
+	 * memory between check and commit (TOCTOU). All decisions below
+	 * are made against this kernel-local copy.
+	 */
+	ret = debug_data_ext_validate(vm->xe, operation, ext);
+	if (ret)
+		goto out;
+
+	op = drm_gpuva_first_op(ops);
+	if (XE_WARN_ON(op != drm_gpuva_last_op(ops))) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	vma_op = gpuva_op_to_vma_op(op);
+
+	if (vma_op->subop == XE_VMA_SUBOP_ADD_DEBUG_DATA)
+		ret = xe_debug_data_add(vm, vma_op, ext);
+	else if (vma_op->subop == XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+		ret = xe_debug_data_remove(vm, vma_op, ext);
+	else
+		ret = -EINVAL;
+
+out:
+	kfree(ext);
+	return ret;
+}
+
+static int xe_debug_data_op_unwind_add(struct xe_vm *vm, struct xe_vma_op *vma_op)
+{
+	const struct xe_debug_data *op_data = vma_op->modify_debug_data.added;
+	struct xe_debug_data *dd, *found = NULL;
+
+	if (!op_data)
+		return 0;
+
+	vm_dbg(&vm->xe->drm,
+	       "Reverting debug data add: addr=0x%016llx, range=0x%016llx, offset=0x%08llx, flags=0x%016llx, path=%s\n",
+	       op_data->addr, op_data->range, op_data->offset, op_data->flags,
+	       debug_data_path_str(op_data->flags, op_data->pseudopath, op_data->pathname));
+
+	mutex_lock(&vm->debug_data.lock);
+	list_for_each_entry(dd, &vm->debug_data.list, link) {
+		if (dd->addr == op_data->addr && dd->range == op_data->range) {
+			list_del(&dd->link);
+			found = dd;
+			break;
+		}
+	}
+	mutex_unlock(&vm->debug_data.lock);
+
+	if (!found) {
+		/*
+		 * Internal invariant: we are unwinding an add that
+		 * previously succeeded, so the entry must be on the
+		 * list unless somebody else removed it behind our
+		 * back. There is no such "somebody else" in the
+		 * current locking design.
+		 */
+		drm_warn(&vm->xe->drm,
+			 "unwind add: entry missing addr=0x%llx range=0x%llx\n",
+			 op_data->addr, op_data->range);
+		return -ENOENT;
+	}
+
+	kfree(found);
+	vma_op->modify_debug_data.added = NULL;
+
+	return 0;
+}
+
+static int xe_debug_data_op_unwind_remove(struct xe_vm *vm, struct xe_vma_op *vma_op)
+{
+	struct xe_debug_data *dd = vma_op->modify_debug_data.removed;
+
+	if (!dd)
+		return 0;
+
+	vm_dbg(&vm->xe->drm,
+	       "Reverting debug data remove: addr=0x%016llx, range=0x%016llx, offset=0x%08llx, flags=0x%016llx, path=%s\n",
+	       dd->addr, dd->range, dd->offset, dd->flags,
+	       debug_data_path_str(dd->flags, dd->pseudopath, dd->pathname));
+
+	mutex_lock(&vm->debug_data.lock);
+	list_add_tail(&dd->link, &vm->debug_data.list);
+	mutex_unlock(&vm->debug_data.lock);
+
+	vma_op->modify_debug_data.removed = NULL;
+
+	return 0;
+}
+
+int xe_debug_data_op_unwind(struct xe_vm *vm, struct xe_vma_op *vma_op)
+{
+	switch (vma_op->subop) {
+	case XE_VMA_SUBOP_ADD_DEBUG_DATA:
+		return xe_debug_data_op_unwind_add(vm, vma_op);
+	case XE_VMA_SUBOP_REMOVE_DEBUG_DATA:
+		return xe_debug_data_op_unwind_remove(vm, vma_op);
+	default:
+		drm_err(&vm->xe->drm, "Invalid debug data subop %d\n", vma_op->subop);
+		return -EINVAL;
+	}
+}
+
+void xe_debug_data_destroy(struct xe_vm *vm)
+{
+	struct xe_debug_data *dd, *tmp;
+
+	mutex_lock(&vm->debug_data.lock);
+	list_for_each_entry_safe(dd, tmp, &vm->debug_data.list, link) {
+		list_del(&dd->link);
+		kfree(dd);
+	}
+	mutex_unlock(&vm->debug_data.lock);
+}
diff --git a/drivers/gpu/drm/xe/xe_debug_data.h b/drivers/gpu/drm/xe/xe_debug_data.h
new file mode 100644
index 000000000000..0c0089a427a0
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_debug_data.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2025 Intel Corporation
+ */
+
+#ifndef _XE_DEBUG_DATA_H_
+#define _XE_DEBUG_DATA_H_
+
+#include <linux/types.h>
+
+struct drm_gpuva_ops;
+struct xe_vm;
+struct xe_vma_op;
+
+const char *xe_debug_data_pseudo_path_to_string(u64 pseudopath);
+int xe_debug_data_check_extension(struct xe_vm *vm, u32 operation, u64 extension);
+int xe_debug_data_process_extension(struct xe_vm *vm, struct drm_gpuva_ops *ops, u32 operation,
+				    u64 extension);
+int xe_debug_data_op_unwind(struct xe_vm *vm, struct xe_vma_op *vma_op);
+void xe_debug_data_destroy(struct xe_vm *vm);
+
+#endif /* _XE_DEBUG_DATA_H_ */
diff --git a/drivers/gpu/drm/xe/xe_debug_data_types.h b/drivers/gpu/drm/xe/xe_debug_data_types.h
new file mode 100644
index 000000000000..b495510b41d6
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_debug_data_types.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#ifndef _XE_DEBUG_DATA_TYPES_H_
+#define _XE_DEBUG_DATA_TYPES_H_
+
+#include <linux/list.h>
+#include <linux/stddef.h>
+#include <linux/types.h>
+
+struct xe_debug_data {
+	struct list_head link;
+	u64 addr;
+	u64 range;
+	u64 flags;
+	u64 offset;
+	union {
+		u64 pseudopath;
+		/* NUL terminated, allocated to the length of the path */
+		DECLARE_FLEX_ARRAY(char, pathname);
+	};
+};
+
+#endif /* _XE_DEBUG_DATA_TYPES_H_ */
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index b9b851adc9bf..138ccdf87eb2 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -26,6 +26,7 @@
 #include "xe_assert.h"
 #include "xe_bo.h"
 #include "xe_cpu_bind.h"
+#include "xe_debug_data.h"
 #include "xe_device.h"
 #include "xe_drm_client.h"
 #include "xe_eudebug.h"
@@ -1457,10 +1458,29 @@ static struct drm_gpuva_op *xe_vm_op_alloc(void)
 	return &op->base;
 }
 
+static void xe_vm_op_free(struct drm_gpuva_op *__op)
+{
+	struct xe_vma_op *op = gpuva_op_to_vma_op(__op);
+
+	/*
+	 * Only REMOVE_DEBUG_DATA ops carry a detached list object here.
+	 * modify_debug_data overlaps map/unmap/prefetch in the union, so
+	 * it must not be touched for any other op type. NULL-safe: set
+	 * only on a committed remove that was not unwound.
+	 */
+	if (__op->op == DRM_GPUVA_OP_DRIVER &&
+	    (op->subop == XE_VMA_SUBOP_ADD_DEBUG_DATA ||
+	     op->subop == XE_VMA_SUBOP_REMOVE_DEBUG_DATA))
+		kfree(op->modify_debug_data.removed);
+
+	kfree(op);
+}
+
 static void xe_vm_free(struct drm_gpuvm *gpuvm);
 
 static const struct drm_gpuvm_ops gpuvm_ops = {
 	.op_alloc = xe_vm_op_alloc,
+	.op_free = xe_vm_op_free,
 	.vm_bo_validate = xe_gpuvm_validate,
 	.vm_free = xe_vm_free,
 };
@@ -1780,6 +1800,9 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags, struct xe_file *xef)
 	xe_vm_init_prove_locking(xe, vm);
 	xe_range_fence_tree_init(&vm->rftree);
 
+	INIT_LIST_HEAD(&vm->debug_data.list);
+	mutex_init(&vm->debug_data.lock);
+
 	vm->pt_ops = &xelp_pt_ops;
 
 	/*
@@ -1920,6 +1943,7 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags, struct xe_file *xef)
 		xe_svm_close(vm);
 	xe_svm_fini(vm);
 err_no_resv:
+	mutex_destroy(&vm->debug_data.lock);
 	mutex_destroy(&vm->snap_mutex);
 	xe_range_fence_tree_fini(&vm->rftree);
 	ttm_lru_bulk_move_fini(&xe->ttm, &vm->lru_bulk_move);
@@ -2077,6 +2101,8 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 
 	xe_range_fence_tree_fini(&vm->rftree);
 
+	xe_debug_data_destroy(vm);
+
 	xe_vm_put(vm);
 }
 
@@ -2094,6 +2120,7 @@ static void vm_destroy_work_func(struct work_struct *w)
 	if (xe_vm_in_preempt_fence_mode(vm))
 		flush_work(&vm->preempt.rebind_work);
 
+	mutex_destroy(&vm->debug_data.lock);
 	mutex_destroy(&vm->snap_mutex);
 
 	if (vm->flags & XE_VM_FLAG_LR_MODE)
@@ -2418,6 +2445,7 @@ static void prep_vma_destroy(struct xe_vm *vm, struct xe_vma *vma,
 #if IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM)
 static void print_op(struct xe_device *xe, struct drm_gpuva_op *op)
 {
+	struct xe_vma_op *vma_op;
 	struct xe_vma *vma;
 
 	switch (op->op) {
@@ -2452,6 +2480,12 @@ static void print_op(struct xe_device *xe, struct drm_gpuva_op *op)
 		vm_dbg(&xe->drm, "PREFETCH: addr=0x%016llx, range=0x%016llx",
 		       (ULL)xe_vma_start(vma), (ULL)xe_vma_size(vma));
 		break;
+	case DRM_GPUVA_OP_DRIVER:
+		vma_op = gpuva_op_to_vma_op(op);
+		if (vma_op->subop != XE_VMA_SUBOP_ADD_DEBUG_DATA &&
+		    vma_op->subop != XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+			drm_warn(&xe->drm, "Unexpected vma sub op: %d", vma_op->subop);
+		break;
 	default:
 		drm_warn(&xe->drm, "NOT POSSIBLE\n");
 	}
@@ -2502,6 +2536,7 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_vma_ops *vops,
 	struct drm_gpuva_ops *ops;
 	struct drm_gpuva_op *__op;
 	struct drm_gpuvm_bo *vm_bo;
+	struct xe_vma_op *vma_op;
 	u64 range_start = addr;
 	u64 range_end = addr + range;
 	int err;
@@ -2558,6 +2593,24 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_vma_ops *vops,
 		drm_gpuvm_bo_put(vm_bo);
 		xe_bo_unlock(bo);
 		break;
+	case DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA:
+	case DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA:
+		ops = kzalloc(sizeof(*ops), GFP_KERNEL);
+		if (!ops)
+			return ERR_PTR(-ENOMEM);
+
+		INIT_LIST_HEAD(&ops->list);
+		vma_op = kzalloc_obj(*vma_op, GFP_KERNEL);
+		if (!vma_op) {
+			kfree(ops);
+			return ERR_PTR(-ENOMEM);
+		}
+
+		vma_op->base.op = DRM_GPUVA_OP_DRIVER;
+		vma_op->subop = operation == DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA ?
+			XE_VMA_SUBOP_ADD_DEBUG_DATA : XE_VMA_SUBOP_REMOVE_DEBUG_DATA;
+		list_add_tail(&vma_op->base.entry, &ops->list);
+		break;
 	default:
 		drm_warn(&vm->xe->drm, "NOT POSSIBLE\n");
 		ops = ERR_PTR(-EINVAL);
@@ -2869,6 +2922,11 @@ static int xe_vma_op_commit(struct xe_vm *vm, struct xe_vma_op *op)
 	case DRM_GPUVA_OP_PREFETCH:
 		op->flags |= XE_VMA_OP_COMMITTED;
 		break;
+	case DRM_GPUVA_OP_DRIVER:
+		if (op->subop != XE_VMA_SUBOP_ADD_DEBUG_DATA &&
+		    op->subop != XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+			drm_warn(&vm->xe->drm, "Unexpected vma sub op: %d", op->subop);
+		break;
 	default:
 		drm_warn(&vm->xe->drm, "NOT POSSIBLE\n");
 	}
@@ -3071,6 +3129,11 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct drm_gpuva_ops *ops,
 				xe_vma_ops_incr_pt_update_ops(vops, op->tile_mask, 1);
 
 			break;
+		case DRM_GPUVA_OP_DRIVER:
+			if (op->subop != XE_VMA_SUBOP_ADD_DEBUG_DATA &&
+			    op->subop != XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+				drm_warn(&vm->xe->drm, "Unexpected vma sub op: %d", op->subop);
+			break;
 		default:
 			drm_warn(&vm->xe->drm, "NOT POSSIBLE\n");
 		}
@@ -3150,6 +3213,13 @@ static void xe_vma_op_unwind(struct xe_vm *vm, struct xe_vma_op *op,
 	case DRM_GPUVA_OP_PREFETCH:
 		/* Nothing to do */
 		break;
+	case DRM_GPUVA_OP_DRIVER:
+		if (op->subop != XE_VMA_SUBOP_ADD_DEBUG_DATA &&
+		    op->subop != XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+			drm_warn(&vm->xe->drm, "Unexpected vma sub op: %d", op->subop);
+		else if (post_commit)
+			xe_debug_data_op_unwind(vm, op);
+		break;
 	default:
 		drm_warn(&vm->xe->drm, "NOT POSSIBLE\n");
 	}
@@ -3505,6 +3575,11 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,
 		}
 		break;
 	}
+	case DRM_GPUVA_OP_DRIVER:
+		if (op->subop != XE_VMA_SUBOP_ADD_DEBUG_DATA &&
+		    op->subop != XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+			drm_warn(&vm->xe->drm, "Unexpected vma sub op: %d", op->subop);
+		break;
 	default:
 		drm_warn(&vm->xe->drm, "NOT POSSIBLE\n");
 	}
@@ -3704,6 +3779,11 @@ static void op_add_ufence(struct xe_vm *vm, struct xe_vma_op *op,
 	case DRM_GPUVA_OP_PREFETCH:
 		vma_add_ufence(gpuva_to_vma(op->base.prefetch.va), ufence);
 		break;
+	case DRM_GPUVA_OP_DRIVER:
+		if (op->subop != XE_VMA_SUBOP_ADD_DEBUG_DATA &&
+		    op->subop != XE_VMA_SUBOP_REMOVE_DEBUG_DATA)
+			drm_warn(&vm->xe->drm, "Unexpected vma sub op: %d", op->subop);
+		break;
 	default:
 		drm_warn(&vm->xe->drm, "NOT POSSIBLE\n");
 	}
@@ -3791,6 +3871,89 @@ ALLOW_ERROR_INJECTION(vm_bind_ioctl_ops_execute, ERRNO);
 #define XE_64K_PAGE_MASK 0xffffull
 #define ALL_DRM_XE_SYNCS_FLAGS (DRM_XE_SYNCS_FLAG_WAIT_FOR_OP)
 
+#define MAX_USER_EXTENSIONS	16
+
+typedef int (*xe_vm_bind_user_extension_check_fn)(struct xe_vm *vm, u32 operation, u64 extension);
+
+typedef int (*xe_vm_bind_user_extension_process_fn)(struct xe_vm *vm, struct drm_gpuva_ops *ops,
+						    u32 operation, u64 extension);
+
+static const xe_vm_bind_user_extension_check_fn vm_bind_extension_check_funcs[] = {
+	[DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA] = xe_debug_data_check_extension,
+};
+
+static const xe_vm_bind_user_extension_process_fn vm_bind_extension_process_funcs[] = {
+	[DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA] = xe_debug_data_process_extension,
+};
+
+static int __vm_bind_op_user_extensions(struct xe_vm *vm, struct drm_gpuva_ops *ops,
+					u32 operation, u64 extensions)
+{
+	struct xe_device *xe = vm->xe;
+	struct drm_xe_user_extension ext;
+	int debug_data_count = 0;
+	int ext_count = 0;
+	int err;
+	u32 idx;
+
+	while (extensions) {
+		u64 __user *address = u64_to_user_ptr(extensions);
+
+		if (XE_IOCTL_DBG(xe, ++ext_count >= MAX_USER_EXTENSIONS))
+			return -E2BIG;
+
+		err = copy_from_user(&ext, address, sizeof(ext));
+		if (XE_IOCTL_DBG(xe, err))
+			return -EFAULT;
+
+		if (XE_IOCTL_DBG(xe, operation != DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA &&
+				 operation != DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA &&
+				 ext.name == DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA) ||
+		    XE_IOCTL_DBG(xe, ext.name == DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA &&
+				 ++debug_data_count > 1))
+			return -EINVAL;
+
+		if (XE_IOCTL_DBG(xe, ext.pad) ||
+		    XE_IOCTL_DBG(xe, ext.name > DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA))
+			return -EINVAL;
+
+		if (!ops) {
+			idx = array_index_nospec(ext.name,
+						 ARRAY_SIZE(vm_bind_extension_check_funcs));
+			err = vm_bind_extension_check_funcs[idx](vm, operation, extensions);
+		} else {
+			idx = array_index_nospec(ext.name,
+						 ARRAY_SIZE(vm_bind_extension_process_funcs));
+			err = vm_bind_extension_process_funcs[idx](vm, ops, operation,
+								   extensions);
+		}
+
+		if (XE_IOCTL_DBG(xe, err))
+			return err;
+
+		extensions = ext.next_extension;
+	}
+
+	/* A debug data op is described by its extension, so it must have one */
+	if (XE_IOCTL_DBG(xe, (operation == DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA ||
+			      operation == DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA) &&
+			 !debug_data_count))
+		return -EINVAL;
+
+	return 0;
+}
+
+static int vm_bind_ioctl_check_user_extensions(struct xe_vm *vm, u32 operation, u64 extensions)
+{
+	return __vm_bind_op_user_extensions(vm, NULL, operation, extensions);
+}
+
+static int vm_bind_ioctl_process_user_extensions(struct xe_vm *vm, struct drm_gpuva_ops *ops,
+						 u32 operation, u64 extensions)
+{
+	return __vm_bind_op_user_extensions(vm, ops, operation, extensions);
+}
+
 static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm,
 				    struct drm_xe_vm_bind *args,
 				    struct drm_xe_vm_bind_op **bind_ops)
@@ -3842,6 +4005,7 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm,
 			DRM_XE_VM_BIND_FLAG_CPU_ADDR_MIRROR;
 		bool is_decompress = flags & DRM_XE_VM_BIND_FLAG_DECOMPRESS;
 		u16 pat_index = (*bind_ops)[i].pat_index;
+		u64 extensions = (*bind_ops)[i].extensions;
 		u16 coh_mode;
 		bool comp_en;
 
@@ -3872,7 +4036,7 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm,
 			goto free_bind_ops;
 		}
 
-		if (XE_IOCTL_DBG(xe, op > DRM_XE_VM_BIND_OP_PREFETCH) ||
+		if (XE_IOCTL_DBG(xe, op > DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA) ||
 		    XE_IOCTL_DBG(xe, flags & ~SUPPORTED_FLAGS) ||
 		    XE_IOCTL_DBG(xe, obj && (is_null || is_cpu_addr_mirror)) ||
 		    XE_IOCTL_DBG(xe, obj_offset && (is_null ||
@@ -3924,7 +4088,9 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm,
 		    XE_IOCTL_DBG(xe, addr & ~PAGE_MASK) ||
 		    XE_IOCTL_DBG(xe, range & ~PAGE_MASK) ||
 		    XE_IOCTL_DBG(xe, !range &&
-				 op != DRM_XE_VM_BIND_OP_UNMAP_ALL)) {
+				 op != DRM_XE_VM_BIND_OP_UNMAP_ALL &&
+				 op != DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA &&
+				 op != DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA)) {
 			err = -EINVAL;
 			goto free_bind_ops;
 		}
@@ -3935,6 +4101,25 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe, struct xe_vm *vm,
 			err = -EOPNOTSUPP;
 			goto free_bind_ops;
 		}
+
+		/*
+		 * A debug data op is described entirely by its extension.
+		 * Require the rest of the op to be zero, so that it can be
+		 * given a meaning later without breaking callers.
+		 */
+		if (XE_IOCTL_DBG(xe, (op == DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA ||
+				      op == DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA) &&
+				 (obj || obj_offset || addr || range || flags ||
+				  (*bind_ops)[i].pad || (*bind_ops)[i].pad2 ||
+				  !mem_is_zero((*bind_ops)[i].reserved,
+					       sizeof((*bind_ops)[i].reserved))))) {
+			err = -EINVAL;
+			goto free_bind_ops;
+		}
+
+		err = vm_bind_ioctl_check_user_extensions(vm, op, extensions);
+		if (err)
+			goto free_bind_ops;
 	}
 
 	return 0;
@@ -3969,6 +4154,19 @@ static int vm_bind_ioctl_signal_fences(struct xe_vm *vm,
 	return err;
 }
 
+static bool vm_bind_ops_have_debug_data(const struct drm_xe_vm_bind_op *bind_ops,
+					u32 num_binds)
+{
+	u32 i;
+
+	for (i = 0; i < num_binds; i++)
+		if (bind_ops[i].op == DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA ||
+		    bind_ops[i].op == DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA)
+			return true;
+
+	return false;
+}
+
 static void xe_vma_ops_init(struct xe_vma_ops *vops, struct xe_vm *vm,
 			    struct xe_exec_queue *q,
 			    struct xe_sync_entry *syncs, u32 num_syncs)
@@ -4198,6 +4396,22 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		goto free_syncs;
 	}
 
+	/*
+	 * Debug data ops take effect as the ioctl processes them, so they
+	 * cannot wait for in-fences. A bind of only debug data ops also
+	 * ends in vm_bind_ioctl_signal_fences(), which would reject the
+	 * in-fences only after the debug data change can no longer be
+	 * unwound.
+	 */
+	if (num_syncs && vm_bind_ops_have_debug_data(bind_ops, args->num_binds)) {
+		for (i = 0; i < num_syncs; i++) {
+			if (XE_IOCTL_DBG(xe, syncs[i].fence)) {
+				err = -EOPNOTSUPP;
+				goto free_syncs;
+			}
+		}
+	}
+
 	if (!args->num_binds) {
 		err = -ENODATA;
 		goto free_syncs;
@@ -4214,11 +4428,17 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		u64 obj_offset = bind_ops[i].obj_offset;
 		u32 prefetch_region = bind_ops[i].prefetch_mem_region_instance;
 		u16 pat_index = bind_ops[i].pat_index;
+		u64 extensions = bind_ops[i].extensions;
 
 		ops[i] = vm_bind_ioctl_ops_create(vm, &vops, bos[i], obj_offset,
 						  addr, range, op, flags,
 						  prefetch_region, pat_index);
-		if (IS_ERR(ops[i])) {
+
+		if (!IS_ERR(ops[i]) && extensions) {
+			err = vm_bind_ioctl_process_user_extensions(vm, ops[i], op, extensions);
+			if (err)
+				goto unwind_ops;
+		} else if (IS_ERR(ops[i])) {
 			err = PTR_ERR(ops[i]);
 			ops[i] = NULL;
 			goto unwind_ops;
diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
index fb9305124679..361e26ee2b46 100644
--- a/drivers/gpu/drm/xe/xe_vm_types.h
+++ b/drivers/gpu/drm/xe/xe_vm_types.h
@@ -15,6 +15,7 @@
 #include <linux/mmu_notifier.h>
 #include <linux/scatterlist.h>
 
+#include "xe_debug_data_types.h"
 #include "xe_device_types.h"
 #include "xe_pt_types.h"
 #include "xe_range_fence.h"
@@ -423,6 +424,15 @@ struct xe_vm {
 	bool batch_invalidate_tlb;
 	/** @xef: Xe file handle for tracking this VM's drm client */
 	struct xe_file *xef;
+
+	/** @debug_data: track debug_data mapped to vm */
+	struct {
+		/** @list: list of debug data items for this vm */
+		struct list_head list;
+
+		/** @lock: guards access to the debug data list */
+		struct mutex lock;
+	} debug_data;
 };
 
 /** struct xe_vma_op_map - VMA map operation */
@@ -496,6 +506,21 @@ struct xe_vma_op_prefetch_range {
 	struct drm_pagemap *dpagemap;
 };
 
+/** struct xe_vma_op_modify_debug_data - debug data altering operation */
+struct xe_vma_op_modify_debug_data {
+	/**
+	 * @added: entry published on vm->debug_data.list, owned by the
+	 * list and only borrowed here. Cleared once unwind frees it.
+	 */
+	struct xe_debug_data *added;
+
+	/**
+	 * @removed: detached from the list. Relinked if the ioctl is unwound,
+	 * freed with the op otherwise.
+	 */
+	struct xe_debug_data *removed;
+};
+
 /** enum xe_vma_op_flags - flags for VMA operation */
 enum xe_vma_op_flags {
 	/** @XE_VMA_OP_COMMITTED: VMA operation committed */
@@ -512,6 +537,10 @@ enum xe_vma_subop {
 	XE_VMA_SUBOP_MAP_RANGE,
 	/** @XE_VMA_SUBOP_UNMAP_RANGE: Unmap range */
 	XE_VMA_SUBOP_UNMAP_RANGE,
+	/** @XE_VMA_SUBOP_ADD_DEBUG_DATA: Add debug data to vm */
+	XE_VMA_SUBOP_ADD_DEBUG_DATA,
+	/** @XE_VMA_SUBOP_REMOVE_DEBUG_DATA: Remove debug data from vm */
+	XE_VMA_SUBOP_REMOVE_DEBUG_DATA,
 };
 
 /** struct xe_vma_op - VMA operation */
@@ -540,6 +569,8 @@ struct xe_vma_op {
 		struct xe_vma_op_unmap_range unmap_range;
 		/** @prefetch_range: VMA prefetch range operation specific data */
 		struct xe_vma_op_prefetch_range prefetch_range;
+		/** @modify_debug_data: debug_data operation specific data */
+		struct xe_vma_op_modify_debug_data modify_debug_data;
 	};
 };
 
diff --git a/include/uapi/drm/xe_drm.h b/include/uapi/drm/xe_drm.h
index 759e76b7ac52..750141aa734d 100644
--- a/include/uapi/drm/xe_drm.h
+++ b/include/uapi/drm/xe_drm.h
@@ -6,6 +6,8 @@
 #ifndef _UAPI_XE_DRM_H_
 #define _UAPI_XE_DRM_H_
 
+#include <linux/limits.h>
+
 #include "drm.h"
 
 #if defined(__cplusplus)
@@ -1021,6 +1023,73 @@ struct drm_xe_vm_destroy {
 	__u64 reserved[2];
 };
 
+/**
+ * struct drm_xe_vm_bind_op_ext_debug_data - debug data extension struct for
+ * :c:type:`drm_xe_vm_bind_op`
+ *
+ * The GPU VM can be annotated by issuing a bind operation with the
+ * :c:member:`drm_xe_vm_bind_op.op` set to %DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA or
+ * %DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA. Each such operation has to provide a
+ * :c:type:`drm_xe_vm_bind_op_ext_debug_data` extension, which describes the
+ * debug data to add or remove.
+ *
+ * This extension can either point to a file that contains relevant debug data
+ * or annotate the VM range with a pseudopath by setting the
+ * %DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO flag and providing one of the supported
+ * pseudopath values:
+ *
+ *  - %DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_MODULE_AREA
+ *  - %DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SBA_AREA
+ *  - %DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SIP_AREA
+ *
+ * Debug data operations take effect as the ioctl processes them, they are not
+ * ordered against earlier work on the bind exec queue. A
+ * &DRM_IOCTL_XE_VM_BIND carrying a debug data operation therefore does not
+ * support in-fences: any &struct drm_xe_sync without %DRM_XE_SYNC_FLAG_SIGNAL
+ * fails the ioctl with -EOPNOTSUPP.
+ */
+struct drm_xe_vm_bind_op_ext_debug_data {
+	/** @base: Base user extension */
+	struct drm_xe_user_extension base;
+
+	/** @addr: Address of the debug data mapping */
+	__u64 addr;
+
+	/** @range: Range of the debug data mapping */
+	__u64 range;
+
+#define DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO (1 << 0)
+	/** @flags: Debug data flags */
+	__u64 flags;
+
+	/**
+	 * @offset: Offset into the debug data file, MBZ when
+	 * %DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO is set
+	 */
+	__u64 offset;
+
+	/** @reserved: Reserved */
+	__u64 reserved;
+
+	union {
+#define DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_MODULE_AREA	0x1
+#define DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SBA_AREA	0x2
+#define DRM_XE_VM_BIND_DEBUG_DATA_PSEUDO_SIP_AREA	0x3
+		/**
+		 * @pseudopath: Pseudopath used when
+		 * %DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO is set, the rest
+		 * of the union MBZ then
+		 */
+		__u64 pseudopath;
+
+		/**
+		 * @pathname: Path to the debug data file, NUL terminated and
+		 * not empty
+		 */
+		char pathname[PATH_MAX];
+	};
+};
+
 /**
  * struct drm_xe_vm_bind_op - run bind operations
  *
@@ -1030,6 +1099,8 @@ struct drm_xe_vm_destroy {
  *  - %DRM_XE_VM_BIND_OP_MAP_USERPTR
  *  - %DRM_XE_VM_BIND_OP_UNMAP_ALL
  *  - %DRM_XE_VM_BIND_OP_PREFETCH
+ *  - %DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA
+ *  - %DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA
  *
  * and the @flags can be:
  *  - %DRM_XE_VM_BIND_FLAG_READONLY - Setup the page tables as read-only
@@ -1081,7 +1152,14 @@ struct drm_xe_vm_destroy {
  *    the memory region advised by madvise.
  */
 struct drm_xe_vm_bind_op {
-	/** @extensions: Pointer to the first extension struct, if any */
+#define DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA	0
+	/**
+	 * @extensions: Pointer to the first extension struct, if any. The
+	 * only extension is %DRM_XE_VM_BIND_OP_EXTENSION_DEBUG_DATA, which is
+	 * required by, and only valid for, %DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA
+	 * and %DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA. Any other extension, or
+	 * an extension on another operation, is rejected.
+	 */
 	__u64 extensions;
 
 	/**
@@ -1135,7 +1213,11 @@ struct drm_xe_vm_bind_op {
 	 * attributes likely do not apply.  Simply leaving as zero is one
 	 * option (still a valid pat_index). Same applies to
 	 * DRM_XE_VM_BIND_FLAG_CPU_ADDR_MIRROR bindings as for such mapping
-	 * there is no actual memory being mapped.
+	 * there is no actual memory being mapped, and to the
+	 * %DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA and
+	 * %DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA operations, which annotate a
+	 * range rather than map anything: the value is ignored for them, so
+	 * zero is fine.
 	 */
 	__u16 pat_index;
 
@@ -1174,6 +1256,8 @@ struct drm_xe_vm_bind_op {
 #define DRM_XE_VM_BIND_OP_MAP_USERPTR	0x2
 #define DRM_XE_VM_BIND_OP_UNMAP_ALL	0x3
 #define DRM_XE_VM_BIND_OP_PREFETCH	0x4
+#define DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA	0x5
+#define DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA	0x6
 	/** @op: Bind operation to perform */
 	__u32 op;
 
-- 
2.53.0

[PATCH 11/20] drm/xe/eudebug: Introduce vm bind and vm bind debug data events

[Mika Kuoppala]

From: Christoph Manszewski <christoph.manszewski@intel.com>

Add events to track the bind ioctl and associated debug data add and
remove operations, as a single bind can involve multiple operations
and may fail mid-process.

Add a bind event to signal to the debugger when a bind operation is
executed. Further add debug data add and remove operations so the
debugger can keep track of regions where they reside. The bind event
is important as we will want to include the ufence event later in the
series and tie it to this bind.

Only deliver bind+operations to the debugger if the vm bind
op execution chain succeeds.

v2: - avoid sending bind if there are no operations (Mika)
    - documentation (Christoph, Mika)
v3: - use removed field dd for removing
v4: - fix potential null deref (Sashiko)
v5: - skip non lr vms in discovery (Sashiko)
v6: - look up the debugger under a spinlock of its own instead of
      eudebug.lock, which the vm bind cannot take under the reservation
      once eudebug enable checks for OA streams (lockdep) (Mika)
v7: - async disconnect under the debugger lock, wq moved here (Mika)
v8: - document the async disconnect window (Sashiko)

Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Co-developed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/xe_eudebug.rst   |   6 +
 drivers/gpu/drm/xe/xe_device.c        |   1 +
 drivers/gpu/drm/xe/xe_device_types.h  |  11 +
 drivers/gpu/drm/xe/xe_eudebug.c       | 344 +++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_eudebug.h       |   7 +
 drivers/gpu/drm/xe/xe_eudebug_types.h |  12 +-
 drivers/gpu/drm/xe/xe_vm.c            |   4 +
 include/uapi/drm/xe_drm_eudebug.h     |  98 +++++++-
 8 files changed, 470 insertions(+), 13 deletions(-)

diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index f3152a13e250..71a8441e04cf 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -68,3 +68,9 @@ Resource Event Types
 
 .. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
    :identifiers: drm_xe_eudebug_event_exec_queue
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event_vm_bind
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event_vm_bind_op_debug_data
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index d11429a69325..6968ae9cc300 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -119,6 +119,7 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)
 
 #if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
 	init_rwsem(&xef->eudebug.ioctl_lock);
+	spin_lock_init(&xef->eudebug.lock);
 #endif
 
 	file->driver_priv = xef;
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 6375a74414d3..72747cb6b4f6 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -639,6 +639,9 @@ struct xe_device {
 
 		/** @eudebug.lock: protects the eudebug state of this device */
 		struct mutex lock;
+
+		/** @eudebug.wq: used for the async session disconnect */
+		struct workqueue_struct *wq;
 	} eudebug;
 #endif
 
@@ -736,6 +739,14 @@ struct xe_file {
 		/** @eudebug.debugger: the debugger connection into this xe_file */
 		struct xe_eudebug *debugger;
 
+		/**
+		 * @eudebug.lock: protects @eudebug.debugger. Writers hold
+		 * xe->eudebug.lock as well, lookups take only this, so that
+		 * they nest inside any lock the caller holds, the vm
+		 * reservation of a vm bind included.
+		 */
+		spinlock_t lock;
+
 		/**
 		 * @eudebug.ioctl_lock: excludes the ioctls that mutate the
 		 * eudebug visible resources of this client, so that a writer
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 0f8b0452f9ba..c6d08b27042d 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -5,6 +5,7 @@
 
 #include <linux/anon_inodes.h>
 #include <linux/cleanup.h>
+#include <linux/dma-resv.h>
 #include <linux/poll.h>
 #include <linux/uaccess.h>
 
@@ -13,6 +14,7 @@
 #include <uapi/drm/xe_drm.h>
 
 #include "xe_assert.h"
+#include "xe_debug_data_types.h"
 #include "xe_device.h"
 #include "xe_eudebug.h"
 #include "xe_eudebug_types.h"
@@ -314,7 +316,8 @@ static bool xe_eudebug_detach(struct xe_eudebug *d,
 
 	if (target) {
 		XE_WARN_ON(target->eudebug.debugger != d);
-		target->eudebug.debugger = NULL;
+		scoped_guard(spinlock, &target->eudebug.lock)
+			target->eudebug.debugger = NULL;
 
 		d->xe->eudebug.attached--;
 
@@ -340,6 +343,44 @@ static bool xe_eudebug_detach(struct xe_eudebug *d,
 		       __disc_err, __func__, __LINE__); \
 })
 
+static void disconnect_work_fn(struct work_struct *work)
+{
+	struct xe_eudebug *d = container_of(work, typeof(*d), disconnect_work);
+	int err;
+
+	scoped_guard(mutex, &d->lock)
+		err = d->disconnect_err;
+
+	xe_eudebug_disconnect(d, err);
+	xe_eudebug_put(d);
+}
+
+/*
+ * xe_eudebug_detach() takes xe->eudebug.lock, which must not be taken
+ * under a vm reservation, see @eudebug.lock of struct xe_file. A producer
+ * that holds one, the vm bind, hands the detach to a worker instead. The
+ * first error is the one that broke the session, so only that is kept.
+ *
+ * Until the worker runs, the session still reads as attached: the
+ * debugger can read what was queued, and producers can still queue. That
+ * leaves nothing behind, the debugger discards the session on -ENOTCONN
+ * anyway.
+ */
+static void xe_eudebug_disconnect_async(struct xe_eudebug *d, int err)
+{
+	XE_WARN_ON(err >= 0);
+
+	scoped_guard(mutex, &d->lock) {
+		if (d->disconnect_err)
+			return;
+
+		d->disconnect_err = err;
+	}
+
+	kref_get(&d->ref);
+	queue_work(d->xe->eudebug.wq, &d->disconnect_work);
+}
+
 static int event_fifo_pending(struct xe_eudebug *d,
 			      struct drm_xe_eudebug_event **pending)
 {
@@ -404,14 +445,13 @@ bool xe_eudebug_is_enabled(struct xe_device *xe)
 static struct xe_eudebug *
 __xe_eudebug_get(struct xe_file *xef)
 {
-	struct xe_device *xe = xef->xe;
 	struct xe_eudebug *d;
 
-	mutex_lock(&xe->eudebug.lock);
+	spin_lock(&xef->eudebug.lock);
 	d = xef->eudebug.debugger;
 	if (d && !kref_get_unless_zero(&d->ref))
 		d = NULL;
-	mutex_unlock(&xe->eudebug.lock);
+	spin_unlock(&xef->eudebug.lock);
 
 	if (d && xe_eudebug_is_detached(d)) {
 		xe_eudebug_put(d);
@@ -706,7 +746,8 @@ xe_eudebug_prepare_event(struct xe_eudebug *d, u16 type, u64 seqno, u16 flags,
 {
 	const u16 known_flags =
 		DRM_XE_EUDEBUG_EVENT_CREATE |
-		DRM_XE_EUDEBUG_EVENT_DESTROY;
+		DRM_XE_EUDEBUG_EVENT_DESTROY |
+		DRM_XE_EUDEBUG_EVENT_STATE_CHANGE;
 	struct drm_xe_eudebug_event *event = d->events.staging;
 
 	lockdep_assert_held(&d->lock);
@@ -1061,6 +1102,252 @@ void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q)
 	xe_eudebug_put(d);
 }
 
+static int send_vm_bind_event(struct xe_eudebug *d,
+			      u64 vm_handle,
+			      u32 bind_flags,
+			      u32 num_ops, u64 seqno)
+{
+	struct drm_xe_eudebug_event_vm_bind *e;
+	struct drm_xe_eudebug_event *event;
+	const u32 sz = sizeof(*e);
+	const u32 base_flags = DRM_XE_EUDEBUG_EVENT_STATE_CHANGE;
+	int err;
+
+	mutex_lock(&d->lock);
+	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_VM_BIND,
+					 seqno, base_flags, sz);
+	e = cast_event(e, event);
+
+	e->vm_handle = vm_handle;
+	e->flags = bind_flags;
+	e->num_bind_ops = num_ops;
+
+	err = xe_eudebug_queue_event(d, event);
+	mutex_unlock(&d->lock);
+
+	return err;
+}
+
+static int vm_bind_event(struct xe_eudebug *d,
+			 struct xe_vm *vm,
+			 u32 flags,
+			 u32 num_ops,
+			 u64 *seqno)
+{
+	int h_vm;
+
+	/*
+	 * Stamped at generation, see @events.seqno in xe_eudebug_types.h,
+	 * and before the vm lookup: a vm handle still found here is destroyed
+	 * with a later seqno than this bind. One already gone drops the bind
+	 * with -ENOENT, see xe_eudebug_vm_bind_execute().
+	 */
+	*seqno = atomic64_inc_return(&d->events.seqno);
+
+	h_vm = find_handle(d, XE_EUDEBUG_RES_TYPE_VM, vm);
+	if (h_vm < 0)
+		return h_vm;
+
+	return send_vm_bind_event(d, h_vm, flags, num_ops, *seqno);
+}
+
+static int vm_bind_op_event(struct xe_eudebug *d,
+			    const u32 flags,
+			    const u64 bind_ref_seqno,
+			    const struct xe_debug_data *debug_data)
+{
+	struct drm_xe_eudebug_event_vm_bind_op_debug_data *e;
+	struct drm_xe_eudebug_event *event;
+	const bool pseudo = debug_data->flags & DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO;
+	/* NUL included, so that the debugger can read the tail as a string */
+	const size_t path_sz = pseudo ? 0 : strlen(debug_data->pathname) + 1;
+	const u32 sz = round_up(struct_size(e, pathname, path_sz), 8);
+	u64 seqno;
+	int err;
+
+	/* A path of the size the bind extension accepts has to fit an event */
+	BUILD_BUG_ON(sizeof(*e) +
+		     sizeof_field(struct drm_xe_vm_bind_op_ext_debug_data, pathname) >
+		     XE_EUDEBUG_EVENT_MAX_SIZE);
+
+	seqno = atomic64_inc_return(&d->events.seqno);
+
+	mutex_lock(&d->lock);
+	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_DEBUG_DATA,
+					 seqno, flags, sz);
+	e = cast_event(e, event);
+
+	e->vm_bind_ref_seqno = bind_ref_seqno;
+	e->addr = debug_data->addr;
+	e->range = debug_data->range;
+	e->flags = debug_data->flags;
+	e->offset = debug_data->offset;
+
+	/*
+	 * The event is sized to the path, and xe_eudebug_prepare_event() has
+	 * zeroed it, so the terminator and any alignment padding are there
+	 * without writing them.
+	 */
+	if (pseudo)
+		e->pseudopath = debug_data->pseudopath;
+	else
+		memcpy(e->pathname, debug_data->pathname, path_sz - 1);
+
+	err = xe_eudebug_queue_event(d, event);
+	mutex_unlock(&d->lock);
+
+	return err;
+}
+
+/*
+ * The debug data entry a bind op adds or removes, and the event flag for
+ * it, or NULL if the op is not a debug data op. vm_bind_ioctl_check_args()
+ * requires the extension, so a debug data op always has its entry here.
+ * Counting and sending both go through this, so that num_bind_ops always
+ * matches the op events that follow.
+ */
+static const struct xe_debug_data *bind_op_debug_data(struct xe_vma_op *op,
+						      u32 *flags)
+{
+	if (op->base.op != DRM_GPUVA_OP_DRIVER)
+		return NULL;
+
+	if (op->subop == XE_VMA_SUBOP_ADD_DEBUG_DATA) {
+		*flags = DRM_XE_EUDEBUG_EVENT_CREATE;
+		return op->modify_debug_data.added;
+	}
+
+	if (op->subop == XE_VMA_SUBOP_REMOVE_DEBUG_DATA) {
+		*flags = DRM_XE_EUDEBUG_EVENT_DESTROY;
+		return op->modify_debug_data.removed;
+	}
+
+	return NULL;
+}
+
+void xe_eudebug_vm_bind_execute(struct xe_vm *vm,
+				struct xe_vma_ops *ops)
+{
+	struct xe_eudebug *d;
+	struct xe_vma_op *op;
+	u64 bind_seqno = 0;
+	u32 num_ops;
+	int err;
+
+	if (!xe_vm_in_fault_mode(vm))
+		return;
+
+	d = xe_eudebug_get(vm->xef);
+	if (!d)
+		return;
+
+	num_ops = 0;
+	list_for_each_entry(op, &ops->list, link) {
+		u32 flags;
+
+		if (bind_op_debug_data(op, &flags))
+			num_ops++;
+	}
+
+	lockdep_assert_held(&vm->lock);
+
+	if (!num_ops) {
+		xe_eudebug_put(d);
+		return;
+	}
+
+	err = vm_bind_event(d, vm, 0,
+			    num_ops, &bind_seqno);
+	if (err)
+		goto out_err;
+
+	list_for_each_entry(op, &ops->list, link) {
+		const struct xe_debug_data *dd;
+		u32 flags;
+
+		dd = bind_op_debug_data(op, &flags);
+		if (!dd)
+			continue;
+
+		err = vm_bind_op_event(d, flags, bind_seqno, dd);
+		if (err)
+			goto out_err;
+	}
+
+out_err:
+	/*
+	 * -ENOENT is vm_bind_event() not finding the vm: a racing
+	 * DRM_XE_VM_DESTROY has already removed it and told the debugger, so
+	 * there is nothing left to report the bind against.
+	 */
+	if (err && err != -ENOENT)
+		xe_eudebug_disconnect_async(d, err);
+
+	xe_eudebug_put(d);
+}
+
+static int vm_discover_binds(struct xe_eudebug *d, struct xe_vm *vm,
+			     unsigned int *ops_count)
+{
+	struct xe_debug_data *dd;
+	struct list_head *pos;
+	unsigned int count;
+	u64 ref_seqno;
+	int err;
+
+	/*
+	 * Only fault mode vms are announced to the debugger, so only those
+	 * have a handle to reference here. Nothing stops a client adding
+	 * debug data to another vm, and without this the bind event below
+	 * would fail to resolve the vm and take down the whole discovery.
+	 * Matches xe_eudebug_vm_bind_execute().
+	 */
+	if (!xe_vm_in_fault_mode(vm))
+		return 0;
+
+	/*
+	 * Walked without vm->debug_data.lock. The caller holds the target's
+	 * eudebug.ioctl_lock for write, and everything that adds to or
+	 * removes from this list runs under DRM_XE_VM_BIND, which takes that
+	 * same rwsem for read, so the list cannot change under us here.
+	 */
+	if (list_empty(&vm->debug_data.list))
+		return 0;
+
+	count = 0;
+	list_for_each(pos, &vm->debug_data.list)
+		count++;
+
+	ref_seqno = 0;
+	err = vm_bind_event(d, vm, 0, count, &ref_seqno);
+	if (err) {
+		eu_dbg(d, "vm_bind_event error %d\n", err);
+		goto out_err;
+	}
+
+	list_for_each_entry(dd, &vm->debug_data.list, link) {
+		err = vm_bind_op_event(d, DRM_XE_EUDEBUG_EVENT_CREATE, ref_seqno, dd);
+		if (err) {
+			eu_dbg(d, "vm_bind_op_event error %d\n", err);
+			goto out_err;
+		}
+	}
+
+	*ops_count += count;
+
+	return 0;
+
+out_err:
+	/*
+	 * The event helpers leave the disconnect to their caller. Do it before
+	 * discovery lets the target's ioctls in again, as the other discovery
+	 * events do.
+	 */
+	xe_eudebug_disconnect(d, err);
+
+	return err;
+}
+
 /*
  * All the relevant ioctls that manipulate the xe_file resources eudebug
  * cares about are gated behind down_read(&xef->eudebug.ioctl_lock). The
@@ -1072,7 +1359,7 @@ static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
 	struct xe_exec_queue *q;
 	struct xe_vm *vm;
 	unsigned long i;
-	unsigned int vm_count = 0, eq_count = 0;
+	unsigned int vm_count = 0, eq_count = 0, ops_count = 0;
 	int err = 0;
 
 	lockdep_assert_held_write(&xef->eudebug.ioctl_lock);
@@ -1089,6 +1376,10 @@ static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
 		if (err)
 			goto out;
 		vm_count++;
+
+		err = vm_discover_binds(d, vm, &ops_count);
+		if (err)
+			goto out;
 	}
 
 	xa_for_each(&xef->exec_queue.xa, i, q) {
@@ -1103,9 +1394,9 @@ static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
 	}
 
 out:
-	if (vm_count || eq_count)
-		eu_dbg(d, "Discovery found %u vms, %u exec_queues",
-		       vm_count, eq_count);
+	if (vm_count || eq_count || ops_count)
+		eu_dbg(d, "Discovery found %u vms, %u exec_queues, %u bind_ops",
+		       vm_count, eq_count, ops_count);
 
 	eu_dbg(d, "Discovery end for %llu: %d", d->session, err);
 
@@ -1132,7 +1423,8 @@ static int add_debugger(struct xe_device *xe, struct xe_eudebug *d,
 		d->target.xef = xe_file_get(xef);
 	d->target.pid = xef->pid;
 	kref_get(&d->ref);
-	xef->eudebug.debugger = d;
+	scoped_guard(spinlock, &xef->eudebug.lock)
+		xef->eudebug.debugger = d;
 
 	xe->eudebug.attached++;
 
@@ -1204,6 +1496,14 @@ static int xe_eudebug_release(struct inode *inode, struct file *file)
 	struct xe_eudebug *d = file->private_data;
 	struct drm_device *drm = &d->xe->drm;
 
+	/*
+	 * Let a pending disconnect record the error that broke the session.
+	 * One queued after this flush finds the session already detached; it
+	 * holds its own reference on d, and the final drm_dev_put() drains
+	 * eudebug.wq before eudebug.lock is destroyed or xe is freed.
+	 */
+	flush_work(&d->disconnect_work);
+
 	xe_eudebug_disconnect(d, 0);
 	xe_eudebug_put(d);
 
@@ -1433,17 +1733,25 @@ static const struct file_operations fops = {
 /*
  * Tell lockdep up front where the lock sits, so that a violation is
  * reported on the first run rather than once both paths have been seen.
- * Reclaim never takes it, so allocating under it is fine.
+ * It is taken under the vm reservation of a vm bind, and mmap_lock ranks
+ * above dma_resv, so touching user memory under it is caught. Reclaim
+ * never takes it, so allocating under it is fine.
  */
 static void xe_eudebug_lock_prime(struct xe_eudebug *d)
 {
+	struct dma_resv resv;
+
 	if (!IS_ENABLED(CONFIG_LOCKDEP))
 		return;
 
+	dma_resv_init(&resv);
+	dma_resv_lock(&resv, NULL);
 	mutex_lock(&d->lock);
 	fs_reclaim_acquire(GFP_KERNEL);
 	fs_reclaim_release(GFP_KERNEL);
 	mutex_unlock(&d->lock);
+	dma_resv_unlock(&resv);
+	dma_resv_fini(&resv);
 }
 
 static int
@@ -1479,6 +1787,8 @@ xe_eudebug_connect(struct xe_device *xe,
 	xe_eudebug_lock_prime(d);
 	init_waitqueue_head(&d->events.write_done);
 
+	INIT_WORK(&d->disconnect_work, disconnect_work_fn);
+
 	err = xe_eudebug_resources_init(d);
 	if (XE_IOCTL_DBG(xe, err)) {
 		mutex_destroy(&d->lock);
@@ -1655,6 +1965,7 @@ void xe_eudebug_init_early(struct xe_device *xe)
 void xe_eudebug_init(struct xe_device *xe)
 {
 	struct drm_device *dev = &xe->drm;
+	struct workqueue_struct *wq;
 	int err;
 
 	/* early init failed */
@@ -1666,6 +1977,17 @@ void xe_eudebug_init(struct xe_device *xe)
 		goto out_not_supported;
 	}
 
+	wq = alloc_workqueue("xe-eudebug", WQ_UNBOUND, 0);
+	if (!wq) {
+		err = -ENOMEM;
+		goto out_err;
+	}
+
+	err = drmm_add_action_or_reset(dev, __drmm_workqueue_release, wq);
+	if (err)
+		goto out_err;
+	xe->eudebug.wq = wq;
+
 	err = devm_add_action_or_reset(dev->dev, xe_eudebug_sysfs_fini, xe);
 	if (err)
 		goto out_err;
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index ec07c567356d..c85f86322c89 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -11,10 +11,14 @@
 
 struct drm_device;
 struct drm_file;
+struct xe_debug_data;
 struct xe_device;
 struct xe_file;
 struct xe_vm;
 struct xe_exec_queue;
+struct xe_vma;
+struct xe_vma_ops;
+struct xe_user_fence;
 struct xe_eudebug;
 
 #if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
@@ -53,6 +57,8 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm);
 void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q);
 void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q);
 
+void xe_eudebug_vm_bind_execute(struct xe_vm *vm, struct xe_vma_ops *ops);
+
 #else
 
 static inline int xe_eudebug_connect_ioctl(struct drm_device *dev,
@@ -71,6 +77,7 @@ static inline void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)
 static inline void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q) { }
 static inline void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q) { }
 
+static inline void xe_eudebug_vm_bind_execute(struct xe_vm *vm, struct xe_vma_ops *ops) { }
 #endif /* CONFIG_DRM_XE_EUDEBUG */
 
 #endif /* _XE_EUDEBUG_H_ */
diff --git a/drivers/gpu/drm/xe/xe_eudebug_types.h b/drivers/gpu/drm/xe/xe_eudebug_types.h
index 0cb1b3b678e4..008fa4bdb316 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_types.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_types.h
@@ -12,6 +12,7 @@
 #include <linux/rhashtable.h>
 #include <linux/sizes.h>
 #include <linux/wait.h>
+#include <linux/workqueue.h>
 #include <linux/xarray.h>
 
 #include <uapi/drm/xe_drm_eudebug.h>
@@ -31,7 +32,7 @@ enum xe_eudebug_cap_state {
 	XE_EUDEBUG_CAP_ENABLED,
 };
 
-#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE
+#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_DEBUG_DATA
 
 /**
  * struct xe_eudebug_handle - eudebug resource handle
@@ -111,6 +112,15 @@ struct xe_eudebug {
 	/** @session: session number for this connection (for logs) */
 	u64 session;
 
+	/**
+	 * @disconnect_work: detaches the session for a producer that cannot
+	 * take xe->eudebug.lock, see xe_eudebug_disconnect_async()
+	 */
+	struct work_struct disconnect_work;
+
+	/** @disconnect_err: error @disconnect_work detaches with, first one wins */
+	int disconnect_err;
+
 	/** @events: kfifo queue of to-be-delivered events */
 	struct {
 #define XE_EUDEBUG_EVENT_MAX_SIZE SZ_64K
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 138ccdf87eb2..86929c6ad3dc 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -3839,6 +3839,10 @@ static struct dma_fence *vm_bind_ioctl_ops_execute(struct xe_vm *vm,
 		xe_vm_set_validation_exec(vm, &exec);
 		fence = ops_execute(vm, vops);
 		xe_vm_set_validation_exec(vm, NULL);
+
+		if (!IS_ERR(fence) || PTR_ERR(fence) == -ENODATA)
+			xe_eudebug_vm_bind_execute(vm, vops);
+
 		if (IS_ERR(fence)) {
 			if (PTR_ERR(fence) == -ENODATA)
 				vm_bind_ioctl_ops_fini(vm, vops, NULL);
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index 12e905441f1d..8d0b424be20d 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -58,7 +58,8 @@ extern "C" {
  * order. Must be zero in the read request.
  *
  * :c:member:`drm_xe_eudebug_event.flags` will indicate if a resource was
- * created or destroyed. Must be zero in the read request.
+ * created, destroyed, or if its state changed. Must be zero in the read
+ * request.
  *
  * %DRM_XE_EUDEBUG_EVENT_NONE and %DRM_XE_EUDEBUG_EVENT_READ are never
  * delivered: the first is unused and the second is the request type for
@@ -79,11 +80,14 @@ struct drm_xe_eudebug_event {
 #define DRM_XE_EUDEBUG_EVENT_READ		1
 #define DRM_XE_EUDEBUG_EVENT_VM			2
 #define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE		3
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND		4
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_DEBUG_DATA	5
 
 	/** @flags: Flags */
 	__u16 flags;
 #define DRM_XE_EUDEBUG_EVENT_CREATE		(1 << 0)
 #define DRM_XE_EUDEBUG_EVENT_DESTROY		(1 << 1)
+#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE	(1 << 2)
 
 	/**
 	 * @seqno: Sequence number, for the debugger to construct a timeline.
@@ -155,6 +159,98 @@ struct drm_xe_eudebug_event_exec_queue {
 	__u64 lrc_handle[] __counted_by(width);
 };
 
+/**
+ * struct drm_xe_eudebug_event_vm_bind - VM Bind Event
+ *
+ * When the client (debuggee) calls the vm_bind_ioctl with the
+ * DRM_XE_VM_BIND_OP_[ADD|REMOVE]_DEBUG_DATA operation, the following event
+ * sequence will be created (for the debugger)::
+ *
+ *  ┌───────────────────────┐
+ *  │  EVENT_VM_BIND        ├──────────────────┬─┐
+ *  └───────────────────────┘                  │ │
+ *      ┌──────────────────────────────────┐   │ │
+ *      │ EVENT_VM_BIND_OP_DEBUG_DATA #1   ├───┘ │
+ *      └──────────────────────────────────┘     │
+ *                      ...                      │
+ *      ┌──────────────────────────────────┐     │
+ *      │ EVENT_VM_BIND_OP_DEBUG_DATA #n   ├─────┘
+ *      └──────────────────────────────────┘
+ *
+ * Discovery replays the debug data a vm already has the same way: one
+ * VM_BIND for the vm, with @flags zero, followed by an
+ * EVENT_VM_BIND_OP_DEBUG_DATA with %DRM_XE_EUDEBUG_EVENT_CREATE for each
+ * entry, as if a single bind had added them all.
+ *
+ * All the events below VM_BIND will reference the VM_BIND
+ * they associate with, by field .vm_bind_ref_seqno.
+ */
+struct drm_xe_eudebug_event_vm_bind {
+	/** @base: Base event */
+	struct drm_xe_eudebug_event base;
+
+	/** @vm_handle: VM handle for this bind */
+	__u64 vm_handle;
+
+	/** @flags: Bind specific flags, none are defined yet so always zero */
+	__u32 flags;
+
+	/** @num_bind_ops: How many [ADD|REMOVE]_DEBUG_DATA operations this bind has */
+	__u32 num_bind_ops;
+};
+
+/**
+ * struct drm_xe_eudebug_event_vm_bind_op_debug_data - VM Bind Op Debug Data Event
+ *
+ * When the target drm client issues a vm bind, each operation of type
+ * %DRM_XE_VM_BIND_OP_ADD_DEBUG_DATA or %DRM_XE_VM_BIND_OP_REMOVE_DEBUG_DATA
+ * will generate an event of this type. For reference see
+ * :c:type:`drm_xe_vm_bind_op_ext_debug_data`.
+ */
+struct drm_xe_eudebug_event_vm_bind_op_debug_data {
+	/** @base: Base event */
+	struct drm_xe_eudebug_event base;
+
+	/** @vm_bind_ref_seqno: Parent :c:member:`drm_xe_eudebug_event_vm_bind.base.seqno` */
+	__u64 vm_bind_ref_seqno;
+
+	/** @addr: Address of the debug data mapping */
+	__u64 addr;
+
+	/** @range: Range of the debug data mapping */
+	__u64 range;
+
+	/** @flags: Debug data flags */
+	__u64 flags;
+
+	/** @offset: Offset into the debug data file */
+	__u64 offset;
+
+	/** @reserved: Reserved, always zero */
+	__u64 reserved;
+
+	/**
+	 * @pseudopath: Pseudopath when
+	 * %DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO is set in @flags, zero
+	 * otherwise
+	 */
+	__u64 pseudopath;
+
+	/**
+	 * @pathname: Path to the debug data file
+	 *
+	 * Present only when %DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO is not
+	 * set in @flags, and covered by
+	 * :c:member:`drm_xe_eudebug_event.len` rather than by a fixed size:
+	 * the event is as long as the path it carries. The path is NUL
+	 * terminated and the terminator is included in @len, which may then
+	 * be padded with further zeroes to keep the event size aligned, so
+	 * read it as a string rather than taking the whole tail of the
+	 * event as the name.
+	 */
+	char pathname[];
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.53.0

[PATCH 12/20] drm/xe/eudebug: Add ufence events with acks

[Mika Kuoppala]

When the vma is in place, the debugger needs to intercept before
userspace proceeds with the workload, for example to install
a breakpoint in an EU shader.

If the ufence is part of a bind sequence, attach the debugger in
xe_user_fence. When the ufence signal is about to be delivered,
check if this ufence needs to be tracked by the debugger.
If so, stall the delivery of the ufence signal until the
debugger has acked the ufence (event), with the ack ioctl.

v2: - return err instead of 0 to guarantee signalling (Dominik)
    - checkpatch (Tilak)
    - Kconfig (Mika, Andrzej)
    - use lock instead of cmpxchg (Mika)
v4: - improve ref handling and no ufences nodebug binds
v5: - remove overzealous warn_on on bind_ref_seqno (Christoph)
    - remove superfluous signalled (Mika)
    - fix double free on bind sequence (Mika)
    - Don't fill op fields if no debugger (Maciej)
v6: - rework to align with xe_eudebug_bind_execute()
v7: - fix setting signalled before debugger acks (Jan)
v8: - explicit debugger set and clears (Mika)
    - take reference for tracking check (Mika)
    - fix leak of ref in handle_ack (Mika)
v9: - enforce mbz on user_arg.type (Sashiko)
v10: - avoid double signal if not tracked (Sashiko)
     - clear bind ref seqno if no debugger (Sashiko)
     - add reserved field
     - avoid referencing ack after dropping lock (Claude)
     - clear bind ref if tracking fails (Claude)
     - use GFP_KERNEL for ack allocation
     - don't starve discovery wq (Claude)
v11: - clear bind ref if detached late (Sashiko)
v12: - drop the debugger ref when not tracking (review-prompts)
v13: - ack tree under the debugger lock (Mika)
v14: - ack and its event added under one lock (Sashiko)

Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/xe_eudebug.rst   |   9 +
 drivers/gpu/drm/xe/xe_eudebug.c       | 391 +++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_eudebug.h       |   9 +
 drivers/gpu/drm/xe/xe_eudebug_types.h |   9 +-
 drivers/gpu/drm/xe/xe_sync.c          |  47 ++--
 drivers/gpu/drm/xe/xe_sync.h          |   3 +
 drivers/gpu/drm/xe/xe_sync_types.h    |  29 +-
 include/uapi/drm/xe_drm_eudebug.h     | 117 +++++++-
 8 files changed, 573 insertions(+), 41 deletions(-)

diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index 71a8441e04cf..fbee568d44be 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -74,3 +74,12 @@ Resource Event Types
 
 .. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
    :identifiers: drm_xe_eudebug_event_vm_bind_op_debug_data
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event_vm_bind_ufence
+
+Acknowledging Events
+--------------------
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_ack
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index c6d08b27042d..934dc116120e 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -22,6 +22,7 @@
 #include "xe_hw_engine.h"
 #include "xe_macros.h"
 #include "xe_sriov_pf.h"
+#include "xe_sync.h"
 #include "xe_vm.h"
 
 /**
@@ -290,10 +291,130 @@ static void xe_eudebug_put(struct xe_eudebug *d)
 	kref_put(&d->ref, xe_eudebug_free);
 }
 
+struct xe_eudebug_ack {
+	struct rb_node rb_node;
+	u64 seqno;
+	u64 ts_insert;
+	struct xe_user_fence *ufence;
+};
+
+#define fetch_ack(x) rb_entry(x, struct xe_eudebug_ack, rb_node)
+
+static int compare_ack(const u64 a, const u64 b)
+{
+	if (a < b)
+		return -1;
+	else if (a > b)
+		return 1;
+
+	return 0;
+}
+
+static int ack_insert_cmp(struct rb_node * const node,
+			  const struct rb_node * const p)
+{
+	return compare_ack(fetch_ack(node)->seqno,
+			   fetch_ack(p)->seqno);
+}
+
+static int ack_lookup_cmp(const void * const key,
+			  const struct rb_node * const node)
+{
+	return compare_ack(*(const u64 *)key,
+			   fetch_ack(node)->seqno);
+}
+
+static struct xe_eudebug_ack *remove_ack(struct xe_eudebug *d, u64 seqno)
+{
+	struct rb_root * const root = &d->acks.tree;
+	struct rb_node *node;
+
+	scoped_guard(mutex, &d->lock) {
+		node = rb_find(&seqno, root, ack_lookup_cmp);
+		if (node)
+			rb_erase(node, root);
+	}
+
+	if (!node)
+		return NULL;
+
+	return rb_entry_safe(node, struct xe_eudebug_ack, rb_node);
+}
+
+static void ufence_signal_worker(struct work_struct *w)
+{
+	struct xe_user_fence * const ufence =
+		container_of(w, struct xe_user_fence, eudebug.worker);
+
+	xe_sync_ufence_signal(ufence);
+
+	xe_sync_ufence_put(ufence);
+}
+
+/*
+ * Not on a workqueue of our own: a user_fence_worker() on xe->ordered_wq
+ * can still get here through a disconnect while the device is being
+ * released. xe_device_destroy() drains ordered_wq only after the drmm
+ * actions of xe_eudebug_init() have run, and destroys xe->unordered_wq
+ * after it.
+ */
+static void kick_ufence_worker(struct xe_user_fence *f)
+{
+	queue_work(f->xe->unordered_wq, &f->eudebug.worker);
+}
+
+static void handle_ack(struct xe_eudebug *d, struct xe_eudebug_ack *ack,
+		       bool on_disconnect)
+{
+	struct xe_user_fence *f = ack->ufence;
+	struct xe_eudebug *debugger = NULL;
+	u64 signalled_by;
+	bool signal = false;
+
+	spin_lock(&f->eudebug.lock);
+	if (!f->eudebug.signalled_seqno) {
+		f->eudebug.signalled_seqno = ack->seqno;
+		f->eudebug.bind_ref_seqno = 0;
+		debugger = f->eudebug.debugger;
+		f->eudebug.debugger = NULL;
+		signal = true;
+	}
+	signalled_by = f->eudebug.signalled_seqno;
+	spin_unlock(&f->eudebug.lock);
+
+	if (signal)
+		kick_ufence_worker(f);
+	else
+		xe_sync_ufence_put(f);
+
+	eu_dbg(d, "ACK: seqno=%llu: signalled by %llu (%s) (held %lluus)",
+	       ack->seqno, signalled_by,
+	       on_disconnect ? "disconnect" : "debugger",
+	       ktime_us_delta(ktime_get(), ack->ts_insert));
+
+	kfree(ack);
+
+	if (debugger)
+		xe_eudebug_put(debugger);
+}
+
+/*
+ * handle_ack() can drop a user fence reference, so the tree is taken out
+ * under the lock and released after it is dropped.
+ */
+static void release_acks(struct xe_eudebug *d, struct rb_root *root)
+{
+	struct xe_eudebug_ack *ack, *n;
+
+	rbtree_postorder_for_each_entry_safe(ack, n, root, rb_node)
+		handle_ack(d, ack, true);
+}
+
 static bool xe_eudebug_detach(struct xe_eudebug *d,
 			      const int err)
 {
 	struct xe_file *target = NULL;
+	struct rb_root acks;
 
 	XE_WARN_ON(err > 0);
 
@@ -304,6 +425,9 @@ static bool xe_eudebug_detach(struct xe_eudebug *d,
 	 * been detached, and the fd close path does exactly that. Drop the
 	 * handles in the same critical section, so nothing can be resolved
 	 * afterwards.
+	 *
+	 * Take the acks out even if already detached: an ack inserted after
+	 * the detach is released by the disconnect its failed event causes.
 	 */
 	scoped_guard(mutex, &d->lock) {
 		if (d->target.xef) {
@@ -312,6 +436,9 @@ static bool xe_eudebug_detach(struct xe_eudebug *d,
 			d->target.xef = NULL;
 			xe_eudebug_resources_clear(d);
 		}
+
+		acks = d->acks.tree;
+		d->acks.tree = RB_ROOT;
 	}
 
 	if (target) {
@@ -327,6 +454,8 @@ static bool xe_eudebug_detach(struct xe_eudebug *d,
 
 	wake_up_all(&d->events.write_done);
 
+	release_acks(d, &acks);
+
 	if (target) {
 		xe_eudebug_put(d);
 		xe_file_put(target);
@@ -363,8 +492,9 @@ static void disconnect_work_fn(struct work_struct *work)
  *
  * Until the worker runs, the session still reads as attached: the
  * debugger can read what was queued, and producers can still queue. That
- * leaves nothing behind, the debugger discards the session on -ENOTCONN
- * anyway.
+ * leaves nothing behind. The debugger discards the session on -ENOTCONN
+ * anyway, and a bind that fails here never hands its user fence over to
+ * wait for an ack.
  */
 static void xe_eudebug_disconnect_async(struct xe_eudebug *d, int err)
 {
@@ -747,7 +877,8 @@ xe_eudebug_prepare_event(struct xe_eudebug *d, u16 type, u64 seqno, u16 flags,
 	const u16 known_flags =
 		DRM_XE_EUDEBUG_EVENT_CREATE |
 		DRM_XE_EUDEBUG_EVENT_DESTROY |
-		DRM_XE_EUDEBUG_EVENT_STATE_CHANGE;
+		DRM_XE_EUDEBUG_EVENT_STATE_CHANGE |
+		DRM_XE_EUDEBUG_EVENT_NEED_ACK;
 	struct drm_xe_eudebug_event *event = d->events.staging;
 
 	lockdep_assert_held(&d->lock);
@@ -1199,6 +1330,191 @@ static int vm_bind_op_event(struct xe_eudebug *d,
 	return err;
 }
 
+void xe_eudebug_ufence_init(struct xe_user_fence *ufence)
+{
+	spin_lock_init(&ufence->eudebug.lock);
+	ufence->eudebug.debugger = NULL;
+	ufence->eudebug.bind_ref_seqno = 0;
+	ufence->eudebug.signalled_seqno = 0;
+	INIT_WORK(&ufence->eudebug.worker, ufence_signal_worker);
+}
+
+void xe_eudebug_ufence_fini(struct xe_user_fence *ufence)
+{
+	struct xe_eudebug *d;
+
+	/*
+	 * A fence can die still holding its bind reference. A bind of debug
+	 * data ops only has no page table work, and its fences are signalled
+	 * by xe_vm_bind_ioctl() after xe_eudebug_vm_bind_execute() attached
+	 * this one. In-fences are rejected up front for such a bind, so only
+	 * an allocation failure in xe_sync_in_fence_get() can make that
+	 * signalling fail, and then the fence is never signalled and nothing
+	 * will ever track or ack it. There is no event to send for a fence
+	 * that never signals, so just drop the reference.
+	 *
+	 * This is the release of the last fence reference, so nothing else
+	 * can be looking at the fence any more.
+	 */
+	d = ufence->eudebug.debugger;
+	if (!d)
+		return;
+
+	ufence->eudebug.debugger = NULL;
+
+	xe_eudebug_put(d);
+}
+
+static int track_ufence(struct xe_eudebug *d,
+			struct xe_user_fence *ufence,
+			u64 bind_ref_seqno,
+			bool *tracked)
+{
+	struct drm_xe_eudebug_event *event;
+	struct drm_xe_eudebug_event_vm_bind_ufence *e;
+	const u32 sz = sizeof(*e);
+	const u32 flags = DRM_XE_EUDEBUG_EVENT_CREATE |
+		DRM_XE_EUDEBUG_EVENT_NEED_ACK;
+	struct xe_eudebug_ack *ack;
+	struct rb_node *old;
+	u64 seqno;
+	int ret;
+
+	*tracked = false;
+
+	if (XE_WARN_ON(!bind_ref_seqno))
+		return -EINVAL;
+
+	seqno = atomic64_inc_return(&d->events.seqno);
+
+	/*
+	 * No __GFP_ACCOUNT: this runs in the fence worker, where it would not
+	 * be charged to the debugger or the target anyway.
+	 */
+	ack = kzalloc_obj(*ack, GFP_KERNEL);
+	if (!ack) {
+		eu_dbg(d, "ack allocation fail");
+		return -ENOMEM;
+	}
+
+	ack->seqno = seqno;
+	ack->ts_insert = ktime_get();
+	ack->ufence = ufence;
+
+	__xe_sync_ufence_get(ufence);
+
+	/*
+	 * The ack and the event asking for it go in under one lock, so the
+	 * debugger cannot ack a fence it has not been told about. Once the
+	 * lock is dropped the ack may already be gone, so it is not touched
+	 * after that.
+	 */
+	mutex_lock(&d->lock);
+	old = rb_find_add(&ack->rb_node, &d->acks.tree, ack_insert_cmp);
+	if (old) {
+		mutex_unlock(&d->lock);
+		xe_sync_ufence_put(ufence);
+		kfree(ack);
+		return -EEXIST;
+	}
+
+	*tracked = true;
+
+	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE,
+					 seqno, flags, sz);
+
+	e = cast_event(e, event);
+	e->vm_bind_ref_seqno = bind_ref_seqno;
+
+	ret = xe_eudebug_queue_event(d, event);
+	mutex_unlock(&d->lock);
+
+	return ret;
+}
+
+/*
+ * Signalling without an ack: the fence will not be tracked, so it must not
+ * keep the debugger either. Drop the reference taken in
+ * xe_eudebug_vm_bind_execute() now, as handle_ack() does for a tracked
+ * fence, instead of leaving it to the fence destructor, which can be as
+ * late as the vma the fence is attached to going away.
+ */
+static void ufence_drop_debugger(struct xe_user_fence *ufence)
+{
+	struct xe_eudebug *d;
+
+	scoped_guard(spinlock, &ufence->eudebug.lock) {
+		d = ufence->eudebug.debugger;
+		ufence->eudebug.debugger = NULL;
+		ufence->eudebug.bind_ref_seqno = 0;
+		ufence->eudebug.signalled_seqno = 0;
+	}
+
+	if (d)
+		xe_eudebug_put(d);
+}
+
+/**
+ * xe_eudebug_ufence_track - Track the ufence for eudebug
+ * @ufence : user fence that might be applicable to tracking
+ *
+ * If this user fence was part of bind sequence, we need
+ * to track it so that we can hold the client signalling on behalf
+ * of debugger and thus deliver event to debugger.
+ *
+ * Return: true debugger will track, false debugger not interested
+ *
+ */
+bool xe_eudebug_ufence_track(struct xe_user_fence *ufence)
+{
+	struct xe_eudebug *d;
+	u64 bind_ref_seqno;
+	bool tracked = false;
+	int ret;
+
+	spin_lock(&ufence->eudebug.lock);
+	d = ufence->eudebug.debugger;
+	if (d && !kref_get_unless_zero(&d->ref))
+		d = NULL;
+
+	if (!d) {
+		ufence->eudebug.bind_ref_seqno = 0;
+		ufence->eudebug.signalled_seqno = 0;
+	}
+	/*
+	 * Taken now: once the fence is in the ack tree, an ack or a
+	 * disconnect clears this.
+	 */
+	bind_ref_seqno = ufence->eudebug.bind_ref_seqno;
+	spin_unlock(&ufence->eudebug.lock);
+
+	if (!d)
+		return false;
+
+	if (xe_eudebug_is_detached(d)) {
+		/* Nothing will ack this fence any more */
+		ufence_drop_debugger(ufence);
+		xe_eudebug_put(d);
+		return false;
+	}
+
+	ret = track_ufence(d, ufence, bind_ref_seqno, &tracked);
+	if (ret) {
+		/*
+		 * If we failed before the ack made it into the tree, the
+		 * disconnect below has nothing to release for this fence.
+		 */
+		if (!tracked)
+			ufence_drop_debugger(ufence);
+
+		xe_eudebug_disconnect(d, ret);
+	}
+
+	xe_eudebug_put(d);
+
+	return tracked;
+}
+
 /*
  * The debug data entry a bind op adds or removes, and the event flag for
  * it, or NULL if the op is not a debug data op. vm_bind_ioctl_check_args()
@@ -1228,8 +1544,10 @@ static const struct xe_debug_data *bind_op_debug_data(struct xe_vma_op *op,
 void xe_eudebug_vm_bind_execute(struct xe_vm *vm,
 				struct xe_vma_ops *ops)
 {
+	struct xe_user_fence *ufence = NULL;
 	struct xe_eudebug *d;
 	struct xe_vma_op *op;
+	unsigned int i;
 	u64 bind_seqno = 0;
 	u32 num_ops;
 	int err;
@@ -1241,6 +1559,15 @@ void xe_eudebug_vm_bind_execute(struct xe_vm *vm,
 	if (!d)
 		return;
 
+	for (i = 0; i < ops->num_syncs; i++) {
+		struct xe_sync_entry *se = &ops->syncs[i];
+
+		if (xe_sync_is_ufence(se)) {
+			xe_assert(vm->xe, !ufence);
+			ufence = se->ufence;
+		}
+	}
+
 	num_ops = 0;
 	list_for_each_entry(op, &ops->list, link) {
 		u32 flags;
@@ -1256,7 +1583,8 @@ void xe_eudebug_vm_bind_execute(struct xe_vm *vm,
 		return;
 	}
 
-	err = vm_bind_event(d, vm, 0,
+	err = vm_bind_event(d, vm,
+			    ufence ? DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE : 0,
 			    num_ops, &bind_seqno);
 	if (err)
 		goto out_err;
@@ -1274,6 +1602,19 @@ void xe_eudebug_vm_bind_execute(struct xe_vm *vm,
 			goto out_err;
 	}
 
+	/*
+	 * Safe after the events: the bind's out-syncs, this user fence among
+	 * them, are attached to its fence only in vm_bind_ioctl_ops_fini() or
+	 * vm_bind_ioctl_signal_fences(), after this returns.
+	 */
+	if (ufence) {
+		spin_lock(&ufence->eudebug.lock);
+		kref_get(&d->ref);
+		ufence->eudebug.debugger = d;
+		ufence->eudebug.bind_ref_seqno = bind_seqno;
+		spin_unlock(&ufence->eudebug.lock);
+	}
+
 out_err:
 	/*
 	 * -ENOENT is vm_bind_event() not finding the vm: a racing
@@ -1691,6 +2032,42 @@ static long xe_eudebug_read_event(struct xe_eudebug *d,
 	return ret;
 }
 
+static long
+xe_eudebug_ack_event_ioctl(struct xe_eudebug *d,
+			   const u64 arg)
+{
+	struct drm_xe_eudebug_ack __user * const user_ptr =
+		u64_to_user_ptr(arg);
+	struct drm_xe_eudebug_ack user_arg;
+	struct xe_eudebug_ack *ack;
+	struct xe_device *xe = d->xe;
+
+	if (XE_IOCTL_DBG(xe, copy_from_user(&user_arg,
+					    user_ptr,
+					    sizeof(user_arg))))
+		return -EFAULT;
+
+	if (XE_IOCTL_DBG(xe, user_arg.type))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_arg.flags))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_arg.reserved))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, xe_eudebug_is_detached(d)))
+		return -ENOTCONN;
+
+	ack = remove_ack(d, user_arg.seqno);
+	if (XE_IOCTL_DBG(xe, !ack))
+		return -EINVAL;
+
+	handle_ack(d, ack, false);
+
+	return 0;
+}
+
 /**
  * xe_eudebug_ioctl - Issue a command to eudebug interface
  *
@@ -1715,6 +2092,10 @@ static long xe_eudebug_ioctl(struct file *file,
 		ret = xe_eudebug_read_event(d, arg,
 					    !(file->f_flags & O_NONBLOCK));
 		break;
+	case DRM_XE_EUDEBUG_IOCTL_ACK_EVENT:
+		ret = xe_eudebug_ack_event_ioctl(d, arg);
+		eu_dbg(d, "ioctl cmd=EVENT_ACK ret=%ld\n", ret);
+		break;
 	default:
 		ret = -ENOTTY;
 	}
@@ -1789,6 +2170,8 @@ xe_eudebug_connect(struct xe_device *xe,
 
 	INIT_WORK(&d->disconnect_work, disconnect_work_fn);
 
+	d->acks.tree = RB_ROOT;
+
 	err = xe_eudebug_resources_init(d);
 	if (XE_IOCTL_DBG(xe, err)) {
 		mutex_destroy(&d->lock);
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index c85f86322c89..fc46d80a0b60 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -59,6 +59,10 @@ void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q)
 
 void xe_eudebug_vm_bind_execute(struct xe_vm *vm, struct xe_vma_ops *ops);
 
+void xe_eudebug_ufence_init(struct xe_user_fence *ufence);
+void xe_eudebug_ufence_fini(struct xe_user_fence *ufence);
+bool xe_eudebug_ufence_track(struct xe_user_fence *ufence);
+
 #else
 
 static inline int xe_eudebug_connect_ioctl(struct drm_device *dev,
@@ -78,6 +82,11 @@ static inline void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_e
 static inline void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q) { }
 
 static inline void xe_eudebug_vm_bind_execute(struct xe_vm *vm, struct xe_vma_ops *ops) { }
+
+static inline void xe_eudebug_ufence_init(struct xe_user_fence *ufence) { }
+static inline void xe_eudebug_ufence_fini(struct xe_user_fence *ufence) { }
+static inline bool xe_eudebug_ufence_track(struct xe_user_fence *ufence) { return false; }
+
 #endif /* CONFIG_DRM_XE_EUDEBUG */
 
 #endif /* _XE_EUDEBUG_H_ */
diff --git a/drivers/gpu/drm/xe/xe_eudebug_types.h b/drivers/gpu/drm/xe/xe_eudebug_types.h
index 008fa4bdb316..a8a440c62c4f 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_types.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_types.h
@@ -9,6 +9,7 @@
 #include <linux/kfifo.h>
 #include <linux/kref.h>
 #include <linux/mutex.h>
+#include <linux/rbtree.h>
 #include <linux/rhashtable.h>
 #include <linux/sizes.h>
 #include <linux/wait.h>
@@ -32,7 +33,7 @@ enum xe_eudebug_cap_state {
 	XE_EUDEBUG_CAP_ENABLED,
 };
 
-#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_DEBUG_DATA
+#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE
 
 /**
  * struct xe_eudebug_handle - eudebug resource handle
@@ -175,6 +176,12 @@ struct xe_eudebug {
 		 */
 		atomic64_t seqno;
 	} events;
+
+	/** @acks: user fence acks tracked by this debugger */
+	struct {
+		/** @acks.tree: pending acks by seqnos */
+		struct rb_root tree;
+	} acks;
 };
 
 #endif /* _XE_EUDEBUG_TYPES_H_ */
diff --git a/drivers/gpu/drm/xe/xe_sync.c b/drivers/gpu/drm/xe/xe_sync.c
index 48a38d64a42a..3812420a5b97 100644
--- a/drivers/gpu/drm/xe/xe_sync.c
+++ b/drivers/gpu/drm/xe/xe_sync.c
@@ -17,27 +17,20 @@
 #include <uapi/drm/xe_drm.h>
 
 #include "xe_device.h"
+#include "xe_eudebug.h"
 #include "xe_exec_queue.h"
 #include "xe_macros.h"
 #include "xe_sched_job_types.h"
 
-struct xe_user_fence {
-	struct xe_device *xe;
-	struct kref refcount;
-	struct dma_fence_cb cb;
-	struct work_struct worker;
-	struct mm_struct *mm;
-	u64 __user *addr;
-	u64 value;
-	int signalled;
-};
-
 static void user_fence_destroy(struct kref *kref)
 {
 	struct xe_user_fence *ufence = container_of(kref, struct xe_user_fence,
 						 refcount);
 
 	mmdrop(ufence->mm);
+
+	xe_eudebug_ufence_fini(ufence);
+
 	kfree(ufence);
 }
 
@@ -51,7 +44,8 @@ static void user_fence_put(struct xe_user_fence *ufence)
 	kref_put(&ufence->refcount, user_fence_destroy);
 }
 
-static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr,
+static struct xe_user_fence *user_fence_create(struct xe_device *xe,
+					       u64 addr,
 					       u64 value)
 {
 	struct xe_user_fence *ufence;
@@ -72,14 +66,23 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr,
 	ufence->mm = current->mm;
 	mmgrab(ufence->mm);
 
+	xe_eudebug_ufence_init(ufence);
+
 	return ufence;
 }
 
-static void user_fence_worker(struct work_struct *w)
+void xe_sync_ufence_signal(struct xe_user_fence *ufence)
 {
-	struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker);
+	/*
+	 * Each ufence has one signaller: user_fence_worker() for one the
+	 * debugger does not track, and ufence_signal_worker() for one it
+	 * does, which handle_ack() queues only once. The xchg() makes
+	 * signalling idempotent regardless. Waiters are woken only after the
+	 * state update, so the UMD can reuse the ufence without hitting -EBUSY.
+	 */
+	if (xchg(&ufence->signalled, 1))
+		return;
 
-	WRITE_ONCE(ufence->signalled, 1);
 	if (mmget_not_zero(ufence->mm)) {
 		kthread_use_mm(ufence->mm);
 		if (copy_to_user(ufence->addr, &ufence->value, sizeof(ufence->value)))
@@ -90,11 +93,17 @@ static void user_fence_worker(struct work_struct *w)
 		drm_dbg(&ufence->xe->drm, "mmget_not_zero() failed, ufence wasn't signaled\n");
 	}
 
-	/*
-	 * Wake up waiters only after updating the ufence state, allowing the UMD
-	 * to safely reuse the same ufence without encountering -EBUSY errors.
-	 */
 	wake_up_all(&ufence->xe->ufence_wq);
+}
+
+static void user_fence_worker(struct work_struct *w)
+{
+	struct xe_user_fence *ufence = container_of(w, struct xe_user_fence, worker);
+
+	/* Lets see if debugger wants to track this */
+	if (!xe_eudebug_ufence_track(ufence))
+		xe_sync_ufence_signal(ufence);
+
 	user_fence_put(ufence);
 }
 
diff --git a/drivers/gpu/drm/xe/xe_sync.h b/drivers/gpu/drm/xe/xe_sync.h
index 6b949194acff..457480656794 100644
--- a/drivers/gpu/drm/xe/xe_sync.h
+++ b/drivers/gpu/drm/xe/xe_sync.h
@@ -8,7 +8,9 @@
 
 #include "xe_sync_types.h"
 
+struct dma_fence;
 struct drm_syncobj;
+struct drm_xe_sync;
 struct xe_device;
 struct xe_exec_queue;
 struct xe_file;
@@ -45,5 +47,6 @@ struct xe_user_fence *__xe_sync_ufence_get(struct xe_user_fence *ufence);
 struct xe_user_fence *xe_sync_ufence_get(struct xe_sync_entry *sync);
 void xe_sync_ufence_put(struct xe_user_fence *ufence);
 int xe_sync_ufence_get_status(struct xe_user_fence *ufence);
+void xe_sync_ufence_signal(struct xe_user_fence *ufence);
 
 #endif
diff --git a/drivers/gpu/drm/xe/xe_sync_types.h b/drivers/gpu/drm/xe/xe_sync_types.h
index b88f1833e28c..aa5fff7b89c2 100644
--- a/drivers/gpu/drm/xe/xe_sync_types.h
+++ b/drivers/gpu/drm/xe/xe_sync_types.h
@@ -6,13 +6,32 @@
 #ifndef _XE_SYNC_TYPES_H_
 #define _XE_SYNC_TYPES_H_
 
+#include <linux/dma-fence-array.h>
+#include <linux/kref.h>
+#include <linux/spinlock.h>
 #include <linux/types.h>
 
-struct drm_syncobj;
-struct dma_fence;
-struct dma_fence_chain;
-struct drm_xe_sync;
-struct user_fence;
+struct xe_user_fence {
+	struct xe_device *xe;
+	struct kref refcount;
+	struct dma_fence_cb cb;
+	struct work_struct worker;
+	struct mm_struct *mm;
+	u64 __user *addr;
+	u64 value;
+	int signalled;
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	struct {
+		/** @lock: guards access to eudebug struct members */
+		spinlock_t lock;
+		struct xe_eudebug *debugger;
+		u64 bind_ref_seqno;
+		u64 signalled_seqno;
+		struct work_struct worker;
+	} eudebug;
+#endif
+};
 
 struct xe_sync_entry {
 	struct drm_syncobj *syncobj;
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index 8d0b424be20d..f5ac97b5ebf9 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -39,6 +39,7 @@ extern "C" {
  * assume it saw everything the target did.
  */
 #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT		_IOWR('j', 0x40, struct drm_xe_eudebug_event)
+#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT		_IOW('j', 0x41, struct drm_xe_eudebug_ack)
 
 /**
  * struct drm_xe_eudebug_event - Base type of event delivered by xe_eudebug.
@@ -61,6 +62,10 @@ extern "C" {
  * created, destroyed, or if its state changed. Must be zero in the read
  * request.
  *
+ * If DRM_XE_EUDEBUG_EVENT_NEED_ACK is set, xe_eudebug
+ * will hold the said resource until it is acked by userspace
+ * using the acking ioctl with the seqno of the said event.
+ *
  * %DRM_XE_EUDEBUG_EVENT_NONE and %DRM_XE_EUDEBUG_EVENT_READ are never
  * delivered: the first is unused and the second is the request type for
  * the read ioctl.
@@ -82,12 +87,14 @@ struct drm_xe_eudebug_event {
 #define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE		3
 #define DRM_XE_EUDEBUG_EVENT_VM_BIND		4
 #define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_DEBUG_DATA	5
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE	6
 
 	/** @flags: Flags */
 	__u16 flags;
 #define DRM_XE_EUDEBUG_EVENT_CREATE		(1 << 0)
 #define DRM_XE_EUDEBUG_EVENT_DESTROY		(1 << 1)
 #define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE	(1 << 2)
+#define DRM_XE_EUDEBUG_EVENT_NEED_ACK		(1 << 3)
 
 	/**
 	 * @seqno: Sequence number, for the debugger to construct a timeline.
@@ -104,8 +111,10 @@ struct drm_xe_eudebug_event {
 	 * instead of silently dropping it.
 	 *
 	 * Use this to order and correlate the events that have been read,
-	 * which is what timeline means here. Do not use it as a delivery
-	 * order guarantee.
+	 * which is what timeline means here, and to refer back to an earlier
+	 * event: %DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE and
+	 * &drm_xe_eudebug_ack.seqno identify the event they belong to this
+	 * way. Do not use it as a delivery order guarantee.
 	 */
 	__u64 seqno;
 
@@ -167,15 +176,19 @@ struct drm_xe_eudebug_event_exec_queue {
  * sequence will be created (for the debugger)::
  *
  *  ┌───────────────────────┐
- *  │  EVENT_VM_BIND        ├──────────────────┬─┐
- *  └───────────────────────┘                  │ │
- *      ┌──────────────────────────────────┐   │ │
- *      │ EVENT_VM_BIND_OP_DEBUG_DATA #1   ├───┘ │
- *      └──────────────────────────────────┘     │
- *                      ...                      │
- *      ┌──────────────────────────────────┐     │
- *      │ EVENT_VM_BIND_OP_DEBUG_DATA #n   ├─────┘
- *      └──────────────────────────────────┘
+ *  │  EVENT_VM_BIND        ├──────────────────┬─┬┄┐
+ *  └───────────────────────┘                  │ │ ┊
+ *      ┌──────────────────────────────────┐   │ │ ┊
+ *      │ EVENT_VM_BIND_OP_DEBUG_DATA #1   ├───┘ │ ┊
+ *      └──────────────────────────────────┘     │ ┊
+ *                      ...                      │ ┊
+ *      ┌──────────────────────────────────┐     │ ┊
+ *      │ EVENT_VM_BIND_OP_DEBUG_DATA #n   ├─────┘ ┊
+ *      └──────────────────────────────────┘       ┊
+ *                                                 ┊
+ *      ┌┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┐       ┊
+ *      ┊ EVENT_UFENCE                     ├┄┄┄┄┄┄┄┘
+ *      └┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┘
  *
  * Discovery replays the debug data a vm already has the same way: one
  * VM_BIND for the vm, with @flags zero, followed by an
@@ -184,6 +197,30 @@ struct drm_xe_eudebug_event_exec_queue {
  *
  * All the events below VM_BIND will reference the VM_BIND
  * they associate with, by field .vm_bind_ref_seqno.
+ *
+ * EVENT_UFENCE only follows a VM_BIND with
+ * %DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE set in @flags, which is set
+ * when the client attached a sync of type %DRM_XE_SYNC_TYPE_USER_FENCE to
+ * its vm_bind_ioctl(). The user fence of a bind without debug data
+ * operations, which sends no VM_BIND, is neither reported nor held.
+ * EVENT_UFENCE is not sent, even with the flag set, if the
+ * vm_bind_ioctl() fails after the ops were executed and the fence is
+ * never signalled.
+ *
+ * When EVENT_UFENCE is sent by the driver, all the operations of the
+ * original vm_bind_ioctl() have completed, not only its debug data ones.
+ * Memory in a debug data [addr, range] is then present if this bind or an
+ * earlier one mapped it. Accessing it before the related EVENT_UFENCE
+ * leads to undefined results, as bind operations are asynchronous and the
+ * backing storage might not be there yet when VM_BIND is received.
+ *
+ * Client's UFENCE sync will be held by the driver: client's
+ * drm_xe_wait_ufence will not complete and the value of the ufence
+ * won't appear until ufence is acked by the debugger process calling
+ * DRM_XE_EUDEBUG_IOCTL_ACK_EVENT with the event_ufence.base.seqno.
+ * This will signal the fence, .value will update and the wait will
+ * complete allowing the client to continue.
+ *
  */
 struct drm_xe_eudebug_event_vm_bind {
 	/** @base: Base event */
@@ -192,8 +229,14 @@ struct drm_xe_eudebug_event_vm_bind {
 	/** @vm_handle: VM handle for this bind */
 	__u64 vm_handle;
 
-	/** @flags: Bind specific flags, none are defined yet so always zero */
+	/**
+	 * @flags: Bind specific flags
+	 *
+	 * %DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE: the bind has a user
+	 * fence attached, so EVENT_UFENCE may follow, see above.
+	 */
 	__u32 flags;
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE (1 << 0)
 
 	/** @num_bind_ops: How many [ADD|REMOVE]_DEBUG_DATA operations this bind has */
 	__u32 num_bind_ops;
@@ -251,6 +294,56 @@ struct drm_xe_eudebug_event_vm_bind_op_debug_data {
 	char pathname[];
 };
 
+/**
+ * struct drm_xe_eudebug_event_vm_bind_ufence - User Fence Event
+ *
+ * Delivered when the operations of a vm bind reported as a
+ * %DRM_XE_EUDEBUG_EVENT_VM_BIND with
+ * %DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE have completed, before the
+ * client's user fence for that bind is signalled. See
+ * :c:type:`drm_xe_eudebug_event_vm_bind`. This event will have
+ * %DRM_XE_EUDEBUG_EVENT_CREATE and %DRM_XE_EUDEBUG_EVENT_NEED_ACK set in
+ * :c:member:`drm_xe_eudebug_event.flags` and upon receiving this event
+ * you need to ack it with DRM_XE_EUDEBUG_IOCTL_ACK_EVENT, see
+ * :c:type:`drm_xe_eudebug_ack`.
+ */
+struct drm_xe_eudebug_event_vm_bind_ufence {
+	/** @base: Base event */
+	struct drm_xe_eudebug_event base;
+
+	/** @vm_bind_ref_seqno: Parent :c:member:`drm_xe_eudebug_event_vm_bind.base.seqno` */
+	__u64 vm_bind_ref_seqno;
+};
+
+/**
+ * struct drm_xe_eudebug_ack - Deliver ack for an event
+ *
+ * If event base.flags has DRM_XE_EUDEBUG_EVENT_NEED_ACK set,
+ * then the associated resource processing is held for client and
+ * thus held for the debugger. In order to release the client,
+ * ack needs to be delivered with DRM_XE_EUDEBUG_IOCTL_ACK_EVENT.
+ * Each such event is acked once. When the connection is disconnected,
+ * including by the debugger closing it, everything still held is
+ * released as if acked.
+ *
+ * The ioctl returns -EINVAL if @seqno is not that of an event waiting for an
+ * ack, which includes one already acked, and -ENOTCONN once the
+ * connection has been disconnected.
+ */
+struct drm_xe_eudebug_ack {
+	/** @type: Type, must be zero */
+	__u32 type;
+
+	/** @flags: Flags, must be zero */
+	__u32 flags;
+
+	/** @seqno: Seqno of event that is to be acked */
+	__u64 seqno;
+
+	/** @reserved: Reserved field, must be zero. */
+	__u64 reserved;
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.53.0

[PATCH 13/20] drm/xe/eudebug: Add vm open/pread/pwrite

[Mika Kuoppala]

The debugger needs access to the client's vm to read and write, for
example to inspect ISA/ELF and set up breakpoints.

Add an ioctl to open the target vm with the debugger client and
vm_handle, and hook up pread/pwrite support.

Open will take a timeout argument so that standard fsync
can be used for explicit flushing between cpu/gpu for
the target vm.

Implement this for bo-backed storage. userptr will
be done in the following patch.

v2: - checkpatch (Maciej)
    - 32bit fixes (Andrzej)
    - bo_vmap (Mika)
    - fix vm leak if can't allocate k_buffer (Mika)
    - assert vm write held for vma (Matthew)
v3: - fw ref, ttm_bo_access
    - timeout boundary check (Dominik)
    - don't try to copy to user on zero bytes (Mika)
v4: - offset as unsigned long (Thomas)
    - check XE_VMA_DESTROYED
v5: - drm_dev_put before releasing debugger (Mika)
v6: - stop flushing on first error (Mika)
v7: - deny access on debugger disconnect (Andrzej)
v8: - correct offset calc in ttm_bo_access (Sashiko)
    - overzealous warn removed (Sashiko)
    - check param.extensions (Sashiko)
    - avoid hitting assert (Sashiko)
    - fix detach race on open (Claude)
    - build assert on ioctl cmd (Claude)
    - reject >u32 on vm lookup (Claude)
    - allow smaller vmallocs before fail
v9: - take runtime pm (Sashiko)
v10: - bound the fsync timeout (Sashiko)
v11: - no -EBUSY on discovery, lookup needs no eudebug.lock (Mika)
v12: - reject imported dma-bufs with -EINVAL (Sashiko)

Cc: Andrzej Hajda <andrzej.hajda@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/xe_eudebug.rst  |   6 +
 drivers/gpu/drm/xe/Makefile          |   2 +-
 drivers/gpu/drm/xe/regs/xe_gt_regs.h |  23 +
 drivers/gpu/drm/xe/xe_device_types.h |   7 +
 drivers/gpu/drm/xe/xe_eudebug.c      |  74 +++-
 drivers/gpu/drm/xe/xe_eudebug.h      |  11 +
 drivers/gpu/drm/xe/xe_eudebug_vm.c   | 610 +++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_eudebug_vm.h   |  13 +
 drivers/gpu/drm/xe/xe_hw_engine.h    |   5 +
 include/uapi/drm/xe_drm_eudebug.h    |  69 ++-
 10 files changed, 810 insertions(+), 10 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_vm.c
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_vm.h

diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index fbee568d44be..bf5d1f0ce0a0 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -83,3 +83,9 @@ Acknowledging Events
 
 .. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
    :identifiers: drm_xe_eudebug_ack
+
+VM Access
+=========
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_vm_open
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index aee129a48040..f24ef4261dcb 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -165,7 +165,7 @@ xe-$(CONFIG_DRM_XE_GPUSVM) += xe_svm.o
 xe-$(CONFIG_DRM_GPUSVM) += xe_userptr.o
 
 # debugging shaders with gdb (eudebug) support
-xe-$(CONFIG_DRM_XE_EUDEBUG) += xe_eudebug.o
+xe-$(CONFIG_DRM_XE_EUDEBUG) += xe_eudebug.o xe_eudebug_vm.o
 
 # graphics hardware monitoring (HWMON) support
 xe-$(CONFIG_HWMON) += xe_hwmon.o
diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
index 06d45b1f2218..cd972a03a792 100644
--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
@@ -657,6 +657,29 @@
 #define   CCS_MODE_CSLICE(cslice, ccs) \
 	((ccs) << ((cslice) * CCS_MODE_CSLICE_WIDTH))
 
+#define RCU_ASYNC_FLUSH				XE_REG(0x149fc)
+#define   RCU_ASYNC_FLUSH_IN_PROGRESS	REG_BIT(31)
+#define   RCU_ASYNC_FLUSH_ENGINE_ID_SHIFT	28
+#define   RCU_ASYNC_FLUSH_AMFS		REG_BIT(8)
+#define   RCU_ASYNC_FLUSH_PREFETCH	REG_BIT(7)
+#define   RCU_ASYNC_FLUSH_DATA_PORT	REG_BIT(6)
+#define   RCU_ASYNC_FLUSH_DATA_CACHE	REG_BIT(5)
+#define   RCU_ASYNC_FLUSH_HDC_PIPELINE	REG_BIT(4)
+#define   RCU_ASYNC_INVALIDATE_HDC_PIPELINE REG_BIT(3)
+#define   RCU_ASYNC_INVALIDATE_CONSTANT_CACHE REG_BIT(2)
+#define   RCU_ASYNC_INVALIDATE_TEXTURE_CACHE REG_BIT(1)
+#define   RCU_ASYNC_INVALIDATE_INSTRUCTION_CACHE REG_BIT(0)
+#define   RCU_ASYNC_FLUSH_AND_INVALIDATE_ALL ( \
+	RCU_ASYNC_FLUSH_AMFS | \
+	RCU_ASYNC_FLUSH_PREFETCH | \
+	RCU_ASYNC_FLUSH_DATA_PORT | \
+	RCU_ASYNC_FLUSH_DATA_CACHE | \
+	RCU_ASYNC_FLUSH_HDC_PIPELINE | \
+	RCU_ASYNC_INVALIDATE_HDC_PIPELINE | \
+	RCU_ASYNC_INVALIDATE_CONSTANT_CACHE | \
+	RCU_ASYNC_INVALIDATE_TEXTURE_CACHE | \
+	RCU_ASYNC_INVALIDATE_INSTRUCTION_CACHE)
+
 #define FORCEWAKE_ACK_GT			XE_REG(0x130044)
 
 /* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 72747cb6b4f6..73b576547193 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -640,6 +640,13 @@ struct xe_device {
 		/** @eudebug.lock: protects the eudebug state of this device */
 		struct mutex lock;
 
+		/**
+		 * @eudebug.flush_lock: serializes the vm fsync() cache flushes,
+		 * which share RCU_ASYNC_FLUSH and save and restore the engine
+		 * idle messaging around it
+		 */
+		struct mutex flush_lock;
+
 		/** @eudebug.wq: used for the async session disconnect */
 		struct workqueue_struct *wq;
 	} eudebug;
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 934dc116120e..e31418970113 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -18,6 +18,7 @@
 #include "xe_device.h"
 #include "xe_eudebug.h"
 #include "xe_eudebug_types.h"
+#include "xe_eudebug_vm.h"
 #include "xe_exec_queue.h"
 #include "xe_hw_engine.h"
 #include "xe_macros.h"
@@ -264,7 +265,7 @@ static bool __xe_eudebug_is_detached(struct xe_eudebug *d)
 	return !d->target.xef;
 }
 
-static bool xe_eudebug_is_detached(struct xe_eudebug *d)
+bool xe_eudebug_is_detached(struct xe_eudebug *d)
 {
 	guard(mutex)(&d->lock);
 
@@ -286,7 +287,7 @@ static void xe_eudebug_free(struct kref *ref)
 	kfree(d);
 }
 
-static void xe_eudebug_put(struct xe_eudebug *d)
+void xe_eudebug_put(struct xe_eudebug *d)
 {
 	kref_put(&d->ref, xe_eudebug_free);
 }
@@ -870,6 +871,31 @@ static int xe_eudebug_remove_handle(struct xe_eudebug *d, int type, void *p,
 	return ret;
 }
 
+static void *find_resource__unlocked(struct xe_eudebug_resource *r,
+				     u32 id)
+{
+	struct xe_eudebug_handle *h;
+
+	h = xa_load(&r->xa, id);
+
+	return h ? (void *)(uintptr_t)h->key : NULL;
+}
+
+struct xe_vm *xe_eudebug_vm_get(struct xe_eudebug *d, u32 id)
+{
+	struct xe_eudebug_resource *r =
+		resource_from_type(d, XE_EUDEBUG_RES_TYPE_VM);
+	struct xe_vm *vm;
+
+	mutex_lock(&d->lock);
+	vm = find_resource__unlocked(r, id);
+	if (vm)
+		xe_vm_get(vm);
+	mutex_unlock(&d->lock);
+
+	return vm;
+}
+
 static struct drm_xe_eudebug_event *
 xe_eudebug_prepare_event(struct xe_eudebug *d, u16 type, u64 seqno, u16 flags,
 			 u32 len)
@@ -2096,6 +2122,10 @@ static long xe_eudebug_ioctl(struct file *file,
 		ret = xe_eudebug_ack_event_ioctl(d, arg);
 		eu_dbg(d, "ioctl cmd=EVENT_ACK ret=%ld\n", ret);
 		break;
+	case DRM_XE_EUDEBUG_IOCTL_VM_OPEN:
+		ret = xe_eudebug_vm_open_ioctl(d, arg);
+		eu_dbg(d, "ioctl cmd=VM_OPEN ret=%ld\n", ret);
+		break;
 	default:
 		ret = -ENOTTY;
 	}
@@ -2253,7 +2283,8 @@ void xe_eudebug_file_close(struct xe_file *xef)
 	up_write(&xef->eudebug.ioctl_lock);
 }
 
-static int xe_eudebug_enable(struct xe_device *xe, bool enable)
+static int __xe_eudebug_enable(struct xe_device *xe, bool enable,
+			       bool *end_lockdown)
 {
 	int ret;
 
@@ -2280,8 +2311,37 @@ static int xe_eudebug_enable(struct xe_device *xe, bool enable)
 	set_cap_state(xe, enable ?
 		      XE_EUDEBUG_CAP_ENABLED : XE_EUDEBUG_CAP_DISABLED);
 
-	if (!enable && IS_SRIOV_PF(xe))
-		xe_sriov_pf_end_lockdown(xe);
+	/*
+	 * The lockdown must outlive any vm fsync() cache flush still writing
+	 * gt registers. Waiting those out under eudebug.lock would stall
+	 * everything else behind their polls, so hand the release to the
+	 * caller, which runs after we drop it.
+	 */
+	if (!enable)
+		*end_lockdown = IS_SRIOV_PF(xe);
+
+	return 0;
+}
+
+static int xe_eudebug_enable(struct xe_device *xe, bool enable)
+{
+	bool end_lockdown = false;
+	int ret;
+
+	ret = __xe_eudebug_enable(xe, enable, &end_lockdown);
+	if (ret || !end_lockdown)
+		return ret;
+
+	/*
+	 * A debugger detached before the disable may still be in a vm
+	 * fsync() cache flush, writing gt registers. A flush only starts
+	 * under flush_lock, after checking for the detach, so taking it once
+	 * waits out the last one.
+	 */
+	mutex_lock(&xe->eudebug.flush_lock);
+	mutex_unlock(&xe->eudebug.flush_lock);
+
+	xe_sriov_pf_end_lockdown(xe);
 
 	return 0;
 }
@@ -2375,6 +2435,10 @@ void xe_eudebug_init(struct xe_device *xe)
 	if (err)
 		goto out_err;
 
+	err = drmm_mutex_init(dev, &xe->eudebug.flush_lock);
+	if (err)
+		goto out_err;
+
 	/*
 	 * Last: once the attribute exists it can enable us, and nothing may
 	 * fail and overwrite cap_state after that. Removing a file that was
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index fc46d80a0b60..dc92cdbbf75e 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -37,10 +37,15 @@ struct xe_eudebug;
 		     ##__VA_ARGS__); \
 	} while (0)
 
+#define eu_warn(d, fmt, ...)  __eu_print(d, drm_warn, fmt, ##__VA_ARGS__)
 #define eu_dbg(d, fmt, ...)   __eu_print(d, drm_dbg, fmt, ##__VA_ARGS__)
 
 #define xe_eudebug_assert(d, ...) xe_assert((d)->xe, ##__VA_ARGS__)
 
+#define xe_eudebug_for_each_hw_engine(__hwe, __gt, __id) \
+	for_each_hw_engine(__hwe, __gt, __id)	       \
+		for_each_if(xe_hw_engine_has_eudebug(__hwe))
+
 int xe_eudebug_connect_ioctl(struct drm_device *dev,
 			     void *data,
 			     struct drm_file *file);
@@ -54,6 +59,12 @@ void xe_eudebug_file_close(struct xe_file *xef);
 void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm);
 void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm);
 
+bool xe_eudebug_is_detached(struct xe_eudebug *d);
+
+void xe_eudebug_put(struct xe_eudebug *d);
+
+struct xe_vm *xe_eudebug_vm_get(struct xe_eudebug *d, u32 vm_id);
+
 void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q);
 void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q);
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug_vm.c b/drivers/gpu/drm/xe/xe_eudebug_vm.c
new file mode 100644
index 000000000000..385fcc7e8f5c
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug_vm.c
@@ -0,0 +1,610 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#include "xe_eudebug_vm.h"
+
+#include <linux/anon_inodes.h>
+#include <linux/dma-resv.h>
+#include <linux/fs.h>
+#include <linux/string_choices.h>
+#include <linux/vmalloc.h>
+
+#include <drm/drm_drv.h>
+
+#include "xe_bo.h"
+#include "xe_device.h"
+#include "xe_eudebug.h"
+#include "xe_eudebug_types.h"
+#include "xe_force_wake.h"
+#include "xe_gt.h"
+#include "xe_mmio.h"
+#include "xe_pm.h"
+#include "xe_vm.h"
+#include "xe_wedge.h"
+
+#include "regs/xe_gt_regs.h"
+#include "regs/xe_engine_regs.h"
+
+static int xe_eudebug_vma_access(struct xe_vma *vma,
+				 const u64 offset_in_vma,
+				 void *buf, const size_t len, bool write)
+{
+	struct xe_bo *bo;
+	u64 bytes;
+
+	lockdep_assert_held_write(&xe_vma_vm(vma)->lock);
+
+	if (offset_in_vma >= xe_vma_size(vma))
+		return -EINVAL;
+
+	if (vma->gpuva.flags & XE_VMA_DESTROYED)
+		return -EINVAL;
+
+	bytes = min_t(u64, len, xe_vma_size(vma) - offset_in_vma);
+	if (!bytes)
+		return 0;
+
+	bo = xe_bo_get(xe_vma_bo(vma));
+	if (bo) {
+		const u64 bo_offset = xe_vma_bo_offset(vma) + offset_in_vma;
+		int ret;
+		long lret;
+
+		/*
+		 * An imported dma-buf comes with dma addresses only. Its
+		 * page array stays empty, and ttm_bo_access_locked() would
+		 * kmap a NULL page.
+		 */
+		if (bo->ttm.type == ttm_bo_type_sg) {
+			xe_bo_put(bo);
+			return -EINVAL;
+		}
+
+		/*
+		 * A bo that has just been moved into place is cleared, or
+		 * copied, by a dma engine, and that work is published only as
+		 * a DMA_RESV_USAGE_KERNEL fence on its dma_resv. Nothing below
+		 * ttm_bo_access_locked() waits for it, so without this the
+		 * debugger reads whatever the pages held before the clear
+		 * landed. Those are recycled pages, and vram is a device wide
+		 * pool, so their previous owner need not be the target being
+		 * debugged.
+		 *
+		 * dma-resv.h states the rule plainly for this usage class:
+		 * drivers always must wait for these fences before accessing
+		 * the resource.
+		 *
+		 * Wait and access under one reservation. vm->lock keeps this
+		 * vm's rebinds out, but not a cpu fault migration or an
+		 * eviction, and either can install a new placement with a new
+		 * kernel fence the moment the reservation is dropped.
+		 */
+		ret = xe_bo_lock(bo, true);
+		if (ret) {
+			xe_bo_put(bo);
+			return ret;
+		}
+
+		if (xe_bo_madv_is_dontneed(bo) || xe_bo_is_purged(bo)) {
+			ret = -EINVAL;
+			goto out_unlock;
+		}
+
+		lret = dma_resv_wait_timeout(bo->ttm.base.resv,
+					     DMA_RESV_USAGE_KERNEL, true,
+					     MAX_SCHEDULE_TIMEOUT);
+		if (lret < 0)
+			ret = lret;
+		else
+			ret = ttm_bo_access_locked(&bo->ttm, bo_offset, buf,
+						   bytes, write);
+
+out_unlock:
+		xe_bo_unlock(bo);
+		xe_bo_put(bo);
+
+		return ret;
+	}
+
+	return -EINVAL;
+}
+
+static int xe_eudebug_vm_access(struct xe_vm *vm, const u64 offset,
+				void *buf, const size_t len, bool write)
+{
+	struct xe_vma *vma;
+	int ret;
+
+	down_write(&vm->lock);
+
+	if (offset >= vm->size || len > vm->size - offset) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	vma = xe_vm_find_overlapping_vma(vm, offset, len);
+	if (vma) {
+		u64 vma_local_offset;
+
+		/*
+		 * xe_vm_find_overlapping_vma() may return a VMA that starts
+		 * above @offset, when @offset lies in a hole but
+		 * [offset, offset+len) reaches into the VMA. Reject anything
+		 * that doesn't actually cover @offset itself.
+		 */
+		if (offset < xe_vma_start(vma) ||
+		    offset >= (xe_vma_start(vma) + xe_vma_size(vma))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		/* Translate VM-space offset to VMA-local offset. */
+		vma_local_offset = offset - xe_vma_start(vma);
+		ret = xe_eudebug_vma_access(vma, vma_local_offset, buf, len, write);
+	} else {
+		ret = -EINVAL;
+	}
+
+out:
+	up_write(&vm->lock);
+
+	return ret;
+}
+
+struct vm_file {
+	struct xe_eudebug *debugger;
+	struct xe_vm *vm;
+	u64 flags;
+	u64 vm_handle;
+	u32 timeout_us;
+};
+
+static ssize_t __vm_read_write(struct xe_vm *vm,
+			       void *bb,
+			       char __user *r_buffer,
+			       const char __user *w_buffer,
+			       const u64 offset,
+			       size_t len,
+			       const bool write)
+{
+	ssize_t ret;
+
+	if (!len)
+		return 0;
+
+	if (write) {
+		ret = copy_from_user(bb, w_buffer, len);
+		if (ret)
+			return -EFAULT;
+
+		ret = xe_eudebug_vm_access(vm, offset, bb, len, true);
+		if (ret <= 0)
+			return ret;
+
+		len = ret;
+	} else {
+		ret = xe_eudebug_vm_access(vm, offset, bb, len, false);
+		if (ret <= 0)
+			return ret;
+
+		len = ret;
+
+		ret = copy_to_user(r_buffer, bb, len);
+		if (ret)
+			return -EFAULT;
+	}
+
+	return len;
+}
+
+static ssize_t __xe_eudebug_vm_access(struct file *file,
+				      char __user *r_buffer,
+				      const char __user *w_buffer,
+				      size_t count, loff_t *__pos)
+{
+	struct vm_file *vmf = file->private_data;
+	struct xe_eudebug * const d = vmf->debugger;
+	struct xe_device * const xe = d->xe;
+	const bool write = !!w_buffer;
+	struct xe_vm *vm;
+	ssize_t copied = 0;
+	ssize_t bytes_left;
+	ssize_t ret;
+	unsigned long alloc_len;
+	loff_t pos = *__pos;
+	void *k_buffer;
+
+	if (XE_IOCTL_DBG(d->xe, xe_eudebug_is_detached(d)))
+		return -ENOTCONN;
+
+	if (XE_IOCTL_DBG(xe, write && r_buffer))
+		return -EINVAL;
+
+	vm = xe_eudebug_vm_get(d, vmf->vm_handle);
+	if (XE_IOCTL_DBG(xe, !vm))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, vm != vmf->vm)) {
+		eu_warn(d, "vm_access(%s): vm handle mismatch vm_handle=%llu, flags=0x%llx, pos=%llu, count=%zu\n",
+			str_write_read(write),
+			vmf->vm_handle, vmf->flags, pos, count);
+		xe_vm_put(vm);
+		return -EINVAL;
+	}
+
+	if (!count) {
+		xe_vm_put(vm);
+		return 0;
+	}
+
+	if (pos >= vm->size) {
+		xe_vm_put(vm);
+		return write ? -ENOSPC : 0;
+	}
+
+	count = min_t(u64, count, vm->size - pos);
+	bytes_left = count;
+
+	alloc_len = min_t(unsigned long, ALIGN(count, PAGE_SIZE), 64 * SZ_1M);
+	do  {
+		k_buffer = __vmalloc(alloc_len, GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
+		if (k_buffer)
+			break;
+
+		alloc_len >>= 1;
+	} while (alloc_len >= PAGE_SIZE);
+
+	if (XE_IOCTL_DBG(xe, !k_buffer)) {
+		xe_vm_put(vm);
+		return -ENOMEM;
+	}
+
+	do {
+		const ssize_t len = min_t(ssize_t, bytes_left, alloc_len);
+
+		ret = __vm_read_write(vm, k_buffer,
+				      write ? NULL : r_buffer + copied,
+				      write ? w_buffer + copied : NULL,
+				      (u64)(pos + copied),
+				      len,
+				      write);
+		if (ret <= 0)
+			break;
+
+		bytes_left -= ret;
+		copied += ret;
+	} while (bytes_left > 0);
+
+	vfree(k_buffer);
+	xe_vm_put(vm);
+
+	if (XE_WARN_ON(copied < 0))
+		copied = 0;
+
+	*__pos += copied;
+
+	return copied ?: ret;
+}
+
+/*
+ * The vm fd holds a drm_device reference, which keeps struct xe_device
+ * around but not the hardware: after an unbind the mmio mapping is gone.
+ * As with xe_bo_cpu_fault(), refuse to touch the device once it has been
+ * unplugged or wedged.
+ */
+static int vm_file_enter(struct xe_device *xe, int *idx)
+{
+	if (xe_wedged(xe))
+		return -ECANCELED;
+
+	if (!drm_dev_enter(&xe->drm, idx))
+		return -ENODEV;
+
+	return 0;
+}
+
+static ssize_t xe_eudebug_vm_read(struct file *file,
+				  char __user *buffer,
+				  size_t count, loff_t *pos)
+{
+	struct vm_file *vmf = file->private_data;
+	struct xe_device *xe = vmf->debugger->xe;
+	ssize_t ret;
+	int idx;
+
+	ret = vm_file_enter(xe, &idx);
+	if (ret)
+		return ret;
+
+	/*
+	 * These are plain file operations, not DRM ioctls, so nothing above
+	 * us is holding a runtime PM reference. The access reaches VRAM and
+	 * MMIO, which is not allowed while the device is suspended.
+	 */
+	xe_pm_runtime_get(xe);
+	ret = __xe_eudebug_vm_access(file, buffer, NULL, count, pos);
+	xe_pm_runtime_put(xe);
+
+	drm_dev_exit(idx);
+
+	return ret;
+}
+
+static ssize_t xe_eudebug_vm_write(struct file *file,
+				   const char __user *buffer,
+				   size_t count, loff_t *pos)
+{
+	struct vm_file *vmf = file->private_data;
+	struct xe_device *xe = vmf->debugger->xe;
+	ssize_t ret;
+	int idx;
+
+	ret = vm_file_enter(xe, &idx);
+	if (ret)
+		return ret;
+
+	xe_pm_runtime_get(xe);
+	ret = __xe_eudebug_vm_access(file, NULL, buffer, count, pos);
+	xe_pm_runtime_put(xe);
+
+	drm_dev_exit(idx);
+
+	return ret;
+}
+
+static int engine_rcu_flush(struct xe_eudebug *d,
+			    struct xe_hw_engine *hwe,
+			    u32 timeout_us)
+{
+	const struct xe_reg psmi_addr = RING_PSMI_CTL(hwe->mmio_base);
+	struct xe_gt *gt = hwe->gt;
+	unsigned int fw_gt, fw_ref;
+	u32 mask = RCU_ASYNC_FLUSH_AND_INVALIDATE_ALL;
+	u32 psmi_ctrl;
+	u32 id;
+	int ret;
+
+	if (hwe->class == XE_ENGINE_CLASS_RENDER)
+		id = 0;
+	else if (hwe->class == XE_ENGINE_CLASS_COMPUTE)
+		id = hwe->instance + 1;
+	else
+		return -EINVAL;
+
+	mask |= id << RCU_ASYNC_FLUSH_ENGINE_ID_SHIFT;
+
+	/*
+	 * RCU_ASYNC_FLUSH is a gt global register in the GT forcewake
+	 * domain, while RING_PSMI_CTL belongs to the engine's own domain.
+	 * Holding only the latter lets the flush request be dropped and the
+	 * status poll read back 0, so fsync() would report a flush that
+	 * never happened.
+	 */
+	fw_gt = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+	if (!fw_gt)
+		return -ETIMEDOUT;
+
+	fw_ref = xe_force_wake_get(gt_to_fw(gt), hwe->domain);
+	if (!fw_ref) {
+		xe_force_wake_put(gt_to_fw(gt), fw_gt);
+		return -ETIMEDOUT;
+	}
+
+	/*
+	 * Serialize flushes. They share RCU_ASYNC_FLUSH, and a concurrent
+	 * one would restore RING_PSMI_CTL under our feet. This is not
+	 * eudebug.lock, as the polls below can take a long while and
+	 * nothing else needs to wait for them.
+	 */
+	mutex_lock(&d->xe->eudebug.flush_lock);
+
+	/*
+	 * A disable on a PF takes this lock, once every debugger is
+	 * detached, before it lets VFs be enabled. Look again under it, so
+	 * that no flush touches the gt after that.
+	 */
+	if (xe_eudebug_is_detached(d)) {
+		ret = -ENOTCONN;
+		goto out_unlock;
+	}
+
+	psmi_ctrl = xe_mmio_read32(&gt->mmio, psmi_addr);
+	if (!(psmi_ctrl & IDLE_MSG_DISABLE))
+		xe_mmio_write32(&gt->mmio, psmi_addr,
+				REG_MASKED_FIELD_ENABLE(IDLE_MSG_DISABLE));
+
+	/* XXX: Timeout is per operation but in here we flush previous */
+	ret = xe_mmio_wait32(&gt->mmio, RCU_ASYNC_FLUSH,
+			     RCU_ASYNC_FLUSH_IN_PROGRESS, 0,
+			     timeout_us, NULL, false);
+	if (ret)
+		goto out;
+
+	xe_mmio_write32(&gt->mmio, RCU_ASYNC_FLUSH, mask);
+
+	ret = xe_mmio_wait32(&gt->mmio, RCU_ASYNC_FLUSH,
+			     RCU_ASYNC_FLUSH_IN_PROGRESS, 0,
+			     timeout_us, NULL, false);
+out:
+	if (!(psmi_ctrl & IDLE_MSG_DISABLE))
+		xe_mmio_write32(&gt->mmio, psmi_addr,
+				REG_MASKED_FIELD_DISABLE(IDLE_MSG_DISABLE));
+
+out_unlock:
+	mutex_unlock(&d->xe->eudebug.flush_lock);
+	xe_force_wake_put(gt_to_fw(gt), fw_ref);
+	xe_force_wake_put(gt_to_fw(gt), fw_gt);
+
+	return ret;
+}
+
+static int xe_eudebug_vm_fsync(struct file *file, loff_t start, loff_t end, int datasync)
+{
+	struct vm_file *vmf = file->private_data;
+	struct xe_eudebug *d = vmf->debugger;
+	struct xe_gt *gt;
+	int gt_id;
+	int ret = -EINVAL;
+	int idx, err;
+
+	eu_dbg(d, "vm_fsync: vm_handle=%llu, flags=0x%llx, start=%llu, end=%llu datasync=%d\n",
+	       vmf->vm_handle, vmf->flags, start, end, datasync);
+
+	if (XE_IOCTL_DBG(d->xe, xe_eudebug_is_detached(d)))
+		return -ENOTCONN;
+
+	err = vm_file_enter(d->xe, &idx);
+	if (err)
+		return err;
+
+	/* Not an ioctl, so take our own runtime PM ref before touching MMIO */
+	xe_pm_runtime_get(d->xe);
+
+	for_each_gt(gt, d->xe, gt_id) {
+		struct xe_hw_engine *hwe;
+		enum xe_hw_engine_id id;
+
+		/* XXX: vm open per engine? */
+		xe_eudebug_for_each_hw_engine(hwe, gt, id) {
+			ret = engine_rcu_flush(d, hwe, vmf->timeout_us);
+			if (ret)
+				goto out;
+		}
+	}
+out:
+	xe_pm_runtime_put(d->xe);
+	drm_dev_exit(idx);
+
+	return ret;
+}
+
+static int xe_eudebug_vm_release(struct inode *inode, struct file *file)
+{
+	struct vm_file *vmf = file->private_data;
+	struct xe_eudebug *d = vmf->debugger;
+	struct xe_device *xe = d->xe;
+
+	eu_dbg(d, "vm_release: vm_handle=%llu, flags=0x%llx\n",
+	       vmf->vm_handle, vmf->flags);
+
+	xe_vm_put(vmf->vm);
+	xe_eudebug_put(d);
+	drm_dev_put(&xe->drm);
+
+	kfree(vmf);
+
+	return 0;
+}
+
+/* Ceiling for fsync, see xe_eudebug_vm_open_ioctl() */
+#define XE_EUDEBUG_VM_SYNC_MAX_TIMEOUT_NS (10ull * NSEC_PER_SEC)
+
+static const struct file_operations vm_fops = {
+	.owner   = THIS_MODULE,
+	.llseek  = generic_file_llseek,
+	.read    = xe_eudebug_vm_read,
+	.write   = xe_eudebug_vm_write,
+	.fsync   = xe_eudebug_vm_fsync,
+	.release = xe_eudebug_vm_release,
+};
+
+long xe_eudebug_vm_open_ioctl(struct xe_eudebug *d, unsigned long arg)
+{
+	struct drm_xe_eudebug_vm_open param;
+	struct xe_device * const xe = d->xe;
+	struct vm_file *vmf = NULL;
+	struct xe_vm *vm;
+	struct file *file;
+	long ret = 0;
+	int fd;
+
+	BUILD_BUG_ON(_IOC_SIZE(DRM_XE_EUDEBUG_IOCTL_VM_OPEN) != sizeof(param));
+	BUILD_BUG_ON(!(_IOC_DIR(DRM_XE_EUDEBUG_IOCTL_VM_OPEN) & _IOC_WRITE));
+
+	if (XE_IOCTL_DBG(xe, copy_from_user(&param, (void __user *)arg, sizeof(param))))
+		return -EFAULT;
+
+	if (XE_IOCTL_DBG(xe, param.flags))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, param.extensions))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, param.vm_handle > U32_MAX))
+		return -EINVAL;
+
+	/*
+	 * The flush waits on mmio while holding eudebug.flush_lock, so an
+	 * unbounded timeout here would stall every other fsync() on the
+	 * device. The hardware flush is a microsecond scale operation; the
+	 * cap is generous rather than tight.
+	 */
+	if (XE_IOCTL_DBG(xe, param.timeout_ns > XE_EUDEBUG_VM_SYNC_MAX_TIMEOUT_NS))
+		return -EINVAL;
+
+	/*
+	 * The map keys vms by pointer without holding a reference. While
+	 * attached, a destroy removes the handle ahead of the final put, and
+	 * a detach empties the map in the same critical section that detaches
+	 * us, so the lookup never resolves a stale handle.
+	 */
+	vm = xe_eudebug_vm_get(d, param.vm_handle);
+	if (XE_IOCTL_DBG(xe, !vm))
+		return xe_eudebug_is_detached(d) ? -ENOTCONN : -EINVAL;
+
+	vmf = kzalloc_obj(*vmf, GFP_KERNEL);
+	if (XE_IOCTL_DBG(xe, !vmf)) {
+		ret = -ENOMEM;
+		goto out_vm_put;
+	}
+
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (XE_IOCTL_DBG(xe, fd < 0)) {
+		ret = fd;
+		goto out_free;
+	}
+
+	kref_get(&d->ref);
+	vmf->debugger = d;
+	vmf->vm = vm;
+	vmf->flags = param.flags;
+	vmf->vm_handle = param.vm_handle;
+	/*
+	 * 0 would make the flush sample its status once and give up, which
+	 * is not what leaving the timeout unset reads as, so it selects the
+	 * maximum. Round up, so that a short timeout does not truncate to 0.
+	 */
+	vmf->timeout_us = DIV_ROUND_UP_ULL(param.timeout_ns ?: XE_EUDEBUG_VM_SYNC_MAX_TIMEOUT_NS,
+					   NSEC_PER_USEC);
+
+	file = anon_inode_getfile("[xe_eudebug.vm]", &vm_fops, vmf, O_RDWR);
+	if (IS_ERR(file)) {
+		ret = PTR_ERR(file);
+		XE_IOCTL_DBG(xe, ret);
+		goto out_fd_put;
+	}
+
+	eu_dbg(d, "vm_open: handle=%llu, flags=0x%llx, fd=%d\n",
+	       vmf->vm_handle, vmf->flags, fd);
+
+	drm_dev_get(&xe->drm);
+
+	file->f_mode |= FMODE_PREAD | FMODE_PWRITE |
+		FMODE_READ | FMODE_WRITE | FMODE_LSEEK;
+
+	fd_install(fd, file);
+
+	return fd;
+
+out_fd_put:
+	put_unused_fd(fd);
+	xe_eudebug_put(d);
+out_free:
+	kfree(vmf);
+out_vm_put:
+	xe_vm_put(vm);
+
+	return ret;
+}
diff --git a/drivers/gpu/drm/xe/xe_eudebug_vm.h b/drivers/gpu/drm/xe/xe_eudebug_vm.h
new file mode 100644
index 000000000000..dfd43b93588f
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug_vm.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#ifndef _XE_EUDEBUG_VM_H_
+#define _XE_EUDEBUG_VM_H_
+
+struct xe_eudebug;
+
+long xe_eudebug_vm_open_ioctl(struct xe_eudebug *d, unsigned long arg);
+
+#endif /* _XE_EUDEBUG_VM_H_ */
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h
index 532d69677981..7319ac0bb91e 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.h
+++ b/drivers/gpu/drm/xe/xe_hw_engine.h
@@ -87,4 +87,9 @@ static inline bool xe_engine_supports_eudebug(const enum xe_engine_class ec)
 	return false;
 }
 
+static inline bool xe_hw_engine_has_eudebug(const struct xe_hw_engine *hwe)
+{
+	return xe_engine_supports_eudebug(hwe->class);
+}
+
 #endif
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index f5ac97b5ebf9..448821c32d29 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -40,6 +40,7 @@ extern "C" {
  */
 #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT		_IOWR('j', 0x40, struct drm_xe_eudebug_event)
 #define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT		_IOW('j', 0x41, struct drm_xe_eudebug_ack)
+#define DRM_XE_EUDEBUG_IOCTL_VM_OPEN		_IOW('j', 0x42, struct drm_xe_eudebug_vm_open)
 
 /**
  * struct drm_xe_eudebug_event - Base type of event delivered by xe_eudebug.
@@ -209,10 +210,11 @@ struct drm_xe_eudebug_event_exec_queue {
  *
  * When EVENT_UFENCE is sent by the driver, all the operations of the
  * original vm_bind_ioctl() have completed, not only its debug data ones.
- * Memory in a debug data [addr, range] is then present if this bind or an
- * earlier one mapped it. Accessing it before the related EVENT_UFENCE
- * leads to undefined results, as bind operations are asynchronous and the
- * backing storage might not be there yet when VM_BIND is received.
+ * Memory in a debug data [addr, range] is then present, and accessible
+ * through DRM_XE_EUDEBUG_IOCTL_VM_OPEN, if this bind or an earlier one
+ * mapped it. Accessing it before the related EVENT_UFENCE leads to
+ * undefined results, as bind operations are asynchronous and the backing
+ * storage might not be there yet when VM_BIND is received.
  *
  * Client's UFENCE sync will be held by the driver: client's
  * drm_xe_wait_ufence will not complete and the value of the ufence
@@ -344,6 +346,65 @@ struct drm_xe_eudebug_ack {
 	__u64 reserved;
 };
 
+/**
+ * struct drm_xe_eudebug_vm_open - Open a target vm
+ *
+ * Open target VM for reading and writing with DRM_XE_EUDEBUG_IOCTL_VM_OPEN.
+ *
+ * File descriptor is returned which can be used with pread and pwrite
+ * to inspect and modify the target VM.
+ *
+ * Multiple operations can be synced with calling fsync(fd). The
+ * fsync will time out if the VM can't be guaranteed to be in sync, see
+ * @timeout_ns. Caller should re-read the state with pread again.
+ *
+ * pread() and pwrite() take a vm address as the offset. Like accesses
+ * through ptrace, a transfer carries on across adjacent mappings. It
+ * comes back short where it reaches an address no mapping covers, or
+ * where an access fails after some bytes were already transferred.
+ * Errors worth telling apart:
+ *
+ * - -EINVAL if no mapping of this vm covers the offset, or the vm handle
+ *   has gone. Retrying does not help until the target binds the range.
+ *   Mappings whose backing store the CPU cannot reach, such as imported
+ *   dma-bufs, fail with -EINVAL as well, as do buffer objects the target
+ *   marked purgeable (DONTNEED) or the kernel already purged.
+ * - -EINTR if the wait for the backing store to be ready was interrupted.
+ * - -ENOTCONN if the debug target is gone.
+ * - -ENOSPC from pwrite() at an offset at or past the end of the vm.
+ *   pread() returns 0 there.
+ * - -ENOMEM, -EFAULT, -EIO or -ENODATA if a bounce buffer could not be
+ *   allocated, the user buffer is bad, or the backing store could not
+ *   be accessed.
+ * - -ECANCELED if the device is wedged, and -ENODEV if it is gone.
+ * - -ETIMEDOUT from fsync() if the flush did not complete within
+ *   @timeout_ns. The vm may be out of sync, so pread() again.
+ *
+ */
+struct drm_xe_eudebug_vm_open {
+	/** @extensions: Reserved for future extensions, must be zero */
+	__u64 extensions;
+
+	/** @vm_handle: handle of vm to be accessed */
+	__u64 vm_handle;
+
+	/** @flags: flags, must be zero */
+	__u64 flags;
+
+	/**
+	 * @timeout_ns: Timeout value in nanoseconds
+	 *
+	 * Bounds each hardware wait of an fsync() on the returned fd.
+	 * fsync() flushes, and waits on, every engine that supports
+	 * debugging in turn, so one call can take a multiple of this value.
+	 * The flush waits on hardware while holding a device wide lock, so
+	 * the value is capped: anything above 10 seconds is rejected with
+	 * -EINVAL. 0 selects the cap. The value is rounded up to
+	 * microseconds.
+	 */
+	__u64 timeout_ns;
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.53.0

[PATCH 14/20] drm/xe/eudebug: Add userptr vm pread/pwrite

[Mika Kuoppala]

Implement debugger vm access for userptrs.

When bind is done, take a ref to the current task so that
we know from which vm the address was bound. Then during
debugger pread/pwrite we use this target task as
parameter to access the debuggee vm with access_process_vm().

This is based on suggestions from Simona, Thomas and Joonas.

v2: need to add offset into vma (Dominik)
v3: move code into xe_userptr.c (Mika)
v4: check that task mm has not changed (Claude)

Cc: Simona Vetter <simona@ffwll.ch>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Andrzej Hajda <andrzej.hajda@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 drivers/gpu/drm/xe/xe_eudebug_vm.c | 52 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_userptr.c    |  4 +++
 drivers/gpu/drm/xe/xe_userptr.h    | 34 +++++++++++++++++++
 include/uapi/drm/xe_drm_eudebug.h  |  2 ++
 4 files changed, 92 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_eudebug_vm.c b/drivers/gpu/drm/xe/xe_eudebug_vm.c
index 385fcc7e8f5c..e18792c87ca8 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_vm.c
+++ b/drivers/gpu/drm/xe/xe_eudebug_vm.c
@@ -8,6 +8,7 @@
 #include <linux/anon_inodes.h>
 #include <linux/dma-resv.h>
 #include <linux/fs.h>
+#include <linux/sched/mm.h>
 #include <linux/string_choices.h>
 #include <linux/vmalloc.h>
 
@@ -106,6 +107,57 @@ static int xe_eudebug_vma_access(struct xe_vma *vma,
 		xe_bo_put(bo);
 
 		return ret;
+	} else if (xe_vma_is_userptr(vma)) {
+		struct xe_userptr *userptr = &to_userptr_vma(vma)->userptr;
+		struct mm_struct *mm;
+		int ret;
+
+		if (XE_WARN_ON(!userptr->eudebug.task))
+			return -EINVAL;
+
+		/*
+		 * XXX FIXME: access_remote_vm() on userptr->notifier.mm would
+		 * fit better, as the notifier already holds a reference on the
+		 * mm and no task reference would be needed at all. It is not
+		 * exported though, so go through the task with
+		 * access_process_vm(), which is.
+		 *
+		 * Going through the task means the address space is resolved
+		 * from it on every access, and the task outlives its mm: after
+		 * an execve the same task carries a different address space,
+		 * and after the thread exits it carries none at all even
+		 * though the process and this mapping are still alive.
+		 *
+		 * Confirm the mm the userptr was bound against is still the
+		 * one the task holds, so that a replaced address space is
+		 * refused instead of handing the debugger bytes from an
+		 * unrelated program. access_process_vm() resolves the mm
+		 * again on its own, so this narrows that window rather than
+		 * closing it. Keying the access on the mm instead, with
+		 * get_user_pages_remote() on userptr->notifier.mm, would
+		 * remove the task from the picture entirely.
+		 */
+		mm = get_task_mm(userptr->eudebug.task);
+		if (!mm)
+			return -ESRCH;
+
+		ret = mm == userptr->notifier.mm ? 0 : -ESRCH;
+		mmput(mm);
+		if (ret)
+			return ret;
+
+		/*
+		 * access_process_vm() reports a failure to reach even the
+		 * first byte as 0, not as an errno. Turn that into -EIO, as
+		 * /proc/pid/mem does, so that a read does not look like the
+		 * end of the vm and a write does not return 0.
+		 */
+		ret = access_process_vm(userptr->eudebug.task,
+					xe_vma_userptr(vma) + offset_in_vma,
+					buf, bytes,
+					write ? FOLL_WRITE : 0);
+
+		return ret ?: -EIO;
 	}
 
 	return -EINVAL;
diff --git a/drivers/gpu/drm/xe/xe_userptr.c b/drivers/gpu/drm/xe/xe_userptr.c
index 9c1dac0fce6f..87ca94a81a50 100644
--- a/drivers/gpu/drm/xe/xe_userptr.c
+++ b/drivers/gpu/drm/xe/xe_userptr.c
@@ -421,6 +421,8 @@ int xe_userptr_setup(struct xe_userptr_vma *uvma, unsigned long start,
 	if (err)
 		return err;
 
+	xe_eudebug_track_userptr_task(userptr);
+
 	return 0;
 }
 
@@ -429,6 +431,8 @@ void xe_userptr_remove(struct xe_userptr_vma *uvma)
 	struct xe_vm *vm = xe_vma_vm(&uvma->vma);
 	struct xe_userptr *userptr = &uvma->userptr;
 
+	xe_eudebug_untrack_userptr_task(userptr);
+
 	drm_gpusvm_free_pages(&vm->svm.gpusvm, &uvma->userptr.pages,
 			      xe_vma_size(&uvma->vma) >> PAGE_SHIFT);
 
diff --git a/drivers/gpu/drm/xe/xe_userptr.h b/drivers/gpu/drm/xe/xe_userptr.h
index 2a3cd1b5efbb..d6e1c46f56d6 100644
--- a/drivers/gpu/drm/xe/xe_userptr.h
+++ b/drivers/gpu/drm/xe/xe_userptr.h
@@ -9,6 +9,7 @@
 #include <linux/list.h>
 #include <linux/mutex.h>
 #include <linux/notifier.h>
+#include <linux/sched/task.h>
 #include <linux/scatterlist.h>
 #include <linux/spinlock.h>
 
@@ -95,6 +96,17 @@ struct xe_userptr {
 #if IS_ENABLED(CONFIG_DRM_XE_USERPTR_INVAL_INJECT)
 	u32 divisor;
 #endif
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	/** @eudebug: eudebug state of the userptr */
+	struct {
+		/**
+		 * @eudebug.task: task that bound the userptr, see
+		 * xe_eudebug_vma_access()
+		 */
+		struct task_struct *task;
+	} eudebug;
+#endif
 };
 
 #if IS_ENABLED(CONFIG_DRM_GPUSVM)
@@ -133,4 +145,26 @@ static inline void xe_vma_userptr_force_invalidate(struct xe_userptr_vma *uvma)
 {
 }
 #endif
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+static inline void xe_eudebug_track_userptr_task(struct xe_userptr *userptr)
+{
+	/* See xe_eudebug_vma_access() for why this is a task, not the mm */
+	userptr->eudebug.task = get_task_struct(current);
+}
+
+static inline void xe_eudebug_untrack_userptr_task(struct xe_userptr *userptr)
+{
+	put_task_struct(userptr->eudebug.task);
+}
+#else
+static inline void xe_eudebug_track_userptr_task(struct xe_userptr *userptr)
+{
+}
+
+static inline void xe_eudebug_untrack_userptr_task(struct xe_userptr *userptr)
+{
+}
+#endif /* CONFIG_DRM_XE_EUDEBUG */
+
 #endif
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index 448821c32d29..4bb01a3c4fe4 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -371,6 +371,8 @@ struct drm_xe_eudebug_ack {
  *   marked purgeable (DONTNEED) or the kernel already purged.
  * - -EINTR if the wait for the backing store to be ready was interrupted.
  * - -ENOTCONN if the debug target is gone.
+ * - -ESRCH if the mapping is a userptr whose address space can no longer
+ *   be reached.
  * - -ENOSPC from pwrite() at an offset at or past the end of the vm.
  *   pread() returns 0 there.
  * - -ENOMEM, -EFAULT, -EIO or -ENODATA if a bounce buffer could not be
-- 
2.53.0

[PATCH 15/20] drm/xe/eudebug: Add hw enablement

[Mika Kuoppala]

From: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

In order to turn on debug capabilities (i.e. breakpoints), TD_CTL
and some other registers need to be programmed. Implement eudebug
mode enabling, including eudebug related workarounds.

v2: Move workarounds to xe_wa_oob. Use reg_sr directly instead of
xe_rtp as it suits better for dynamic manipulation of those registers we
do later in the series.
v3: get rid of undefining XE_MCR_REG (Mika)
v4: return if dss_mask is all 0.
v5: remove unused GLOBAL_DEBUG_ENABLE (Sashiko)
v6: - don't leak fw ref, check for zero dss_mask (Sashiko)
    - hold runtime for hw access (Sashiko)
    - make iter size check byte granularity for safety (Claude)
    - simplify xe_reg_sr_add with overwrite (Claude)
    - check that bitmap size is aligned to hw (4) (Claude)
v7: - assert attention bitmap size granularity (Sashiko)
    - overwrite fix (Claude)
v8: - drop duplicate STALL_DOP_GATING_DISABLE (Sashiko)
v9: - refuse eudebug on graphics versions without known debug
      register layouts (review-prompts)
v10: - cap_state under eudebug.lock (Mika)
v11: - no empty #else in xe_eudebug_hw.h (Sashiko)

Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
---
 drivers/gpu/drm/xe/Makefile           |   2 +-
 drivers/gpu/drm/xe/regs/xe_gt_regs.h  |  18 ++
 drivers/gpu/drm/xe/xe_eudebug.c       |   9 +-
 drivers/gpu/drm/xe/xe_eudebug_hw.c    |  71 ++++++++
 drivers/gpu/drm/xe/xe_eudebug_hw.h    |  21 +++
 drivers/gpu/drm/xe/xe_gt_debug.c      | 228 ++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_gt_debug.h      |  31 ++++
 drivers/gpu/drm/xe/xe_oa.c            |   1 +
 drivers/gpu/drm/xe/xe_reg_sr.c        |  19 ++-
 drivers/gpu/drm/xe/xe_reg_sr.h        |   4 +-
 drivers/gpu/drm/xe/xe_reg_whitelist.c |   2 +-
 drivers/gpu/drm/xe/xe_rtp.c           |   2 +-
 drivers/gpu/drm/xe/xe_wa_oob.rules    |   2 +
 13 files changed, 402 insertions(+), 8 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_hw.c
 create mode 100644 drivers/gpu/drm/xe/xe_eudebug_hw.h
 create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.c
 create mode 100644 drivers/gpu/drm/xe/xe_gt_debug.h

diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index f24ef4261dcb..356f44aac308 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -165,7 +165,7 @@ xe-$(CONFIG_DRM_XE_GPUSVM) += xe_svm.o
 xe-$(CONFIG_DRM_GPUSVM) += xe_userptr.o
 
 # debugging shaders with gdb (eudebug) support
-xe-$(CONFIG_DRM_XE_EUDEBUG) += xe_eudebug.o xe_eudebug_vm.o
+xe-$(CONFIG_DRM_XE_EUDEBUG) += xe_eudebug.o xe_eudebug_vm.o xe_eudebug_hw.o xe_gt_debug.o
 
 # graphics hardware monitoring (HWMON) support
 xe-$(CONFIG_HWMON) += xe_hwmon.o
diff --git a/drivers/gpu/drm/xe/regs/xe_gt_regs.h b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
index cd972a03a792..765a5c076695 100644
--- a/drivers/gpu/drm/xe/regs/xe_gt_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_gt_regs.h
@@ -553,10 +553,20 @@
 #define   DG2_DISABLE_ROUND_ENABLE_ALLOW_FOR_SSLA	REG_BIT(15)
 #define   CLEAR_OPTIMIZATION_DISABLE			REG_BIT(6)
 
+#define TD_CTL					XE_REG_MCR(0xe400)
+#define   TD_CTL_FEH_AND_FEE_ENABLE		REG_BIT(7) /* forced halt and exception */
+#define   TD_CTL_FORCE_EXTERNAL_HALT		REG_BIT(6)
+#define   TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE	REG_BIT(4)
+#define   TD_CTL_FORCE_EXCEPTION		REG_BIT(3)
+#define   TD_CTL_BREAKPOINT_ENABLE		REG_BIT(2)
+#define   TD_CTL_GLOBAL_DEBUG_ENABLE		REG_BIT(0) /* XeHP */
+
 #define CACHE_MODE_SS				XE_REG_MCR(0xe420, XE_REG_OPTION_MASKED)
 #define   DISABLE_ECC				REG_BIT(5)
 #define   ENABLE_PREFETCH_INTO_IC		REG_BIT(3)
 
+#define EU_ATT(reg, row)			XE_REG_MCR((reg ? 0xe478 : 0xe470) + (row) * 4)
+
 #define ROW_CHICKEN4				XE_REG_MCR(0xe48c, XE_REG_OPTION_MASKED)
 #define   DISABLE_GRF_CLEAR			REG_BIT(13)
 #define   XEHP_DIS_BBL_SYSPIPE			REG_BIT(11)
@@ -566,6 +576,8 @@
 #define   THREAD_EX_ARB_MODE			REG_GENMASK(3, 2)
 #define   THREAD_EX_ARB_MODE_RR_AFTER_DEP	REG_FIELD_PREP(THREAD_EX_ARB_MODE, 0x2)
 
+#define EU_ATT_CLR(reg, row)			XE_REG_MCR((reg ? 0xe698 : 0xe490) + (row) * 4)
+
 #define ROW_CHICKEN3				XE_REG_MCR(0xe49c, XE_REG_OPTION_MASKED)
 #define   XE2_EUPEND_CHK_FLUSH_DIS		REG_BIT(14)
 #define   DIS_EU_GRF_POISON_TO_LSC		REG_BIT(13)
@@ -589,6 +601,7 @@
 #define   DISABLE_READ_SUPPRESSION		REG_BIT(15)
 #define   DISABLE_EARLY_READ			REG_BIT(14)
 #define   ENABLE_LARGE_GRF_MODE			REG_BIT(12)
+#define   XEHPC_DISABLE_BTB			REG_BIT(11)
 #define   PUSH_CONST_DEREF_HOLD_DIS		REG_BIT(8)
 #define   DISABLE_TDL_SVHS_GATING		REG_BIT(1)
 #define   DISABLE_DOP_GATING			REG_BIT(0)
@@ -680,6 +693,11 @@
 	RCU_ASYNC_INVALIDATE_TEXTURE_CACHE | \
 	RCU_ASYNC_INVALIDATE_INSTRUCTION_CACHE)
 
+#define RCU_DEBUG_1				XE_REG(0x14a00)
+#define   RCU_DEBUG_1_ENGINE_STATUS		REG_GENMASK(2, 0)
+#define   RCU_DEBUG_1_RUNALONE_ACTIVE		REG_BIT(2)
+#define   RCU_DEBUG_1_CONTEXT_ACTIVE		REG_BIT(0)
+
 #define FORCEWAKE_ACK_GT			XE_REG(0x130044)
 
 /* Applicable for all FORCEWAKE_DOMAIN and FORCEWAKE_ACK_DOMAIN regs */
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index e31418970113..bf4d3ac4e092 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -2402,7 +2402,14 @@ void xe_eudebug_init_early(struct xe_device *xe)
 	if (drmm_add_action(&xe->drm, xe_eudebug_lock_fini, NULL))
 		drm_dbg(&xe->drm, "eudebug lock will not be destroyed\n");
 
-	xe->eudebug.cap_state = XE_EUDEBUG_CAP_DISABLED;
+	if (GRAPHICS_VER(xe) >= 35) {
+		/* debug register layouts are only known up to Xe3 */
+		drm_dbg(&xe->drm, "eudebug not supported on graphics version %u\n",
+			GRAPHICS_VER(xe));
+		xe->eudebug.cap_state = XE_EUDEBUG_CAP_NOT_SUPPORTED;
+	} else {
+		xe->eudebug.cap_state = XE_EUDEBUG_CAP_DISABLED;
+	}
 }
 
 void xe_eudebug_init(struct xe_device *xe)
diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.c b/drivers/gpu/drm/xe/xe_eudebug_hw.c
new file mode 100644
index 000000000000..7388facfe0a5
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug_hw.c
@@ -0,0 +1,71 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#include "xe_eudebug_hw.h"
+
+#include <linux/delay.h>
+#include <linux/pm_runtime.h>
+#include <generated/xe_wa_oob.h>
+
+#include "regs/xe_gt_regs.h"
+#include "regs/xe_engine_regs.h"
+
+#include "xe_eudebug.h"
+#include "xe_eudebug_types.h"
+#include "xe_exec_queue.h"
+#include "xe_exec_queue_types.h"
+#include "xe_force_wake.h"
+#include "xe_gt.h"
+#include "xe_gt_debug.h"
+#include "xe_gt_mcr.h"
+#include "xe_hw_engine.h"
+#include "xe_lrc.h"
+#include "xe_macros.h"
+#include "xe_mmio.h"
+#include "xe_reg_sr.h"
+#include "xe_rtp.h"
+#include "xe_wa.h"
+
+static void add_sr_entry(struct xe_hw_engine *hwe,
+			 struct xe_reg_mcr mcr_reg,
+			 u32 mask, bool enable)
+{
+	const struct xe_reg_sr_entry sr_entry = {
+		.reg = mcr_reg.__reg,
+		.clr_bits = mask,
+		.set_bits = enable ? mask : 0,
+		.read_mask = mask,
+	};
+
+	xe_reg_sr_add(&hwe->reg_sr, &sr_entry, hwe->gt, true);
+}
+
+void xe_eudebug_init_hw_engine(struct xe_hw_engine *hwe, bool enable)
+{
+	struct xe_gt *gt = hwe->gt;
+	struct xe_device *xe = gt_to_xe(gt);
+
+	if (!xe_rtp_match_first_render_or_compute(xe, gt, hwe))
+		return;
+
+	if (XE_GT_WA(gt, 18022722726))
+		add_sr_entry(hwe, ROW_CHICKEN,
+			     STALL_DOP_GATING_DISABLE, enable);
+
+	if (XE_GT_WA(gt, 14015474168))
+		add_sr_entry(hwe, ROW_CHICKEN2,
+			     XEHPC_DISABLE_BTB,
+			     enable);
+
+	add_sr_entry(hwe, TD_CTL,
+		     TD_CTL_BREAKPOINT_ENABLE |
+		     TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE |
+		     TD_CTL_FEH_AND_FEE_ENABLE,
+		     enable);
+
+	if (xe->info.graphics_verx100 >= 1250)
+		add_sr_entry(hwe, TD_CTL,
+			     TD_CTL_GLOBAL_DEBUG_ENABLE, enable);
+}
diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.h b/drivers/gpu/drm/xe/xe_eudebug_hw.h
new file mode 100644
index 000000000000..447e83cd4e38
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_eudebug_hw.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023-2025 Intel Corporation
+ */
+
+#ifndef _XE_EUDEBUG_HW_H_
+#define _XE_EUDEBUG_HW_H_
+
+#include <linux/types.h>
+
+struct xe_eudebug;
+struct xe_hw_engine;
+struct xe_gt;
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+
+void xe_eudebug_init_hw_engine(struct xe_hw_engine *hwe, bool enable);
+
+#endif /* CONFIG_DRM_XE_EUDEBUG */
+
+#endif /* _XE_EUDEBUG_HW_H_ */
diff --git a/drivers/gpu/drm/xe/xe_gt_debug.c b/drivers/gpu/drm/xe/xe_gt_debug.c
new file mode 100644
index 000000000000..a2f660777949
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_gt_debug.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include "regs/xe_gt_regs.h"
+#include "xe_device.h"
+#include "xe_force_wake.h"
+#include "xe_gt.h"
+#include "xe_gt_topology.h"
+#include "xe_gt_debug.h"
+#include "xe_gt_mcr.h"
+#include "xe_pm.h"
+#include "xe_macros.h"
+
+unsigned int xe_gt_eu_att_regs(struct xe_gt *gt)
+{
+	return (GRAPHICS_VERx100(gt_to_xe(gt)) >= 3000) ? 2u : 1u;
+}
+
+int xe_gt_foreach_dss_group_instance(struct xe_gt *gt,
+				     int (*fn)(struct xe_gt *gt,
+					       void *data,
+					       u16 group,
+					       u16 instance,
+					       bool present),
+				     void *data)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	xe_dss_mask_t dss_mask;
+	unsigned int dss, fw_gt, fw_render;
+	u16 group, instance;
+	int ret = 0;
+
+	/* EUs, and the render forcewake domain, exist on the main GT only */
+	if (!xe_gt_is_main_type(gt))
+		return -EINVAL;
+
+	if (!xe_pm_runtime_get_if_active(xe))
+		return -EIO;
+
+	/*
+	 * The EU attention registers sit in the render forcewake domain.
+	 * xe_force_wake_get() wakes one domain per call, so take GT and
+	 * render separately.
+	 */
+	fw_gt = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+	fw_render = xe_force_wake_get(gt_to_fw(gt), XE_FW_RENDER);
+	if (!fw_gt || !fw_render) {
+		ret = -ETIMEDOUT;
+		goto out_fw;
+	}
+
+	bitmap_or(dss_mask, gt->fuse_topo.g_dss_mask, gt->fuse_topo.c_dss_mask,
+		  XE_MAX_DSS_FUSE_BITS);
+
+	/* If by any chance dss_mask is all 0s */
+	if (!bitmap_weight(dss_mask, XE_MAX_DSS_FUSE_BITS)) {
+		ret = -EINVAL;
+		goto out_fw;
+	}
+
+	/*
+	 * Note: This removes terminating zeros when last dss is fused out!
+	 * In order bitmask to be exactly the same as on with i915 we would
+	 * need to figure out max dss for given platform, most probably by
+	 * querying hwconfig
+	 */
+
+	for (dss = 0;
+	     dss <= find_last_bit(dss_mask, XE_MAX_DSS_FUSE_BITS);
+	     dss++) {
+		xe_gt_mcr_get_dss_steering(gt, dss, &group, &instance);
+
+		ret = fn(gt, data, group, instance, test_bit(dss, dss_mask));
+		if (ret)
+			break;
+	}
+
+out_fw:
+	xe_force_wake_put(gt_to_fw(gt), fw_render);
+	xe_force_wake_put(gt_to_fw(gt), fw_gt);
+	xe_pm_runtime_put(xe);
+
+	return ret;
+}
+
+static int read_first_attention_mcr(struct xe_gt *gt, void *data,
+				    u16 group, u16 instance, bool present)
+{
+	unsigned int reg, row;
+
+	if (!present)
+		return 0;
+
+	for (reg = 0; reg < xe_gt_eu_att_regs(gt); reg++) {
+		for (row = 0; row < XE_GT_EU_ATT_ROWS; row++) {
+			u32 val;
+
+			val = xe_gt_mcr_unicast_read(gt, EU_ATT(reg, row), group, instance);
+
+			if (val)
+				return 1;
+		}
+	}
+
+	return 0;
+}
+
+#define MAX_EUS_PER_ROW 4u
+#define MAX_THREADS 8u
+
+/**
+ * xe_gt_eu_attention_bitmap_size - query size of the attention bitmask
+ *
+ * @gt: pointer to struct xe_gt
+ *
+ * The bitmask is u32 sized hardware register content, one bit per thread,
+ * in native endianness. It is a byte array rather than a kernel bitmap:
+ * it crosses to userspace as one, and its size is expressed in bytes.
+ *
+ * Return: size in bytes, always a multiple of sizeof(u32).
+ */
+int xe_gt_eu_attention_bitmap_size(struct xe_gt *gt)
+{
+	xe_dss_mask_t dss_mask;
+
+	/*
+	 * The readers move through the buffer one u32 register at a time, so
+	 * every dss/register/row has to contribute a whole number of them.
+	 * Everything else in the size below is integer multiplication, which
+	 * makes the total u32 aligned by construction rather than by luck.
+	 */
+	BUILD_BUG_ON(MAX_THREADS * MAX_EUS_PER_ROW / 8 != sizeof(u32));
+
+	bitmap_or(dss_mask, gt->fuse_topo.c_dss_mask,
+		  gt->fuse_topo.g_dss_mask, XE_MAX_DSS_FUSE_BITS);
+
+	if (XE_WARN_ON(bitmap_empty(dss_mask, XE_MAX_DSS_FUSE_BITS)))
+		return 0;
+
+	return (find_last_bit(dss_mask, XE_MAX_DSS_FUSE_BITS) + 1) *
+		XE_GT_EU_ATT_ROWS * xe_gt_eu_att_regs(gt) * MAX_THREADS *
+		MAX_EUS_PER_ROW / 8;
+}
+
+struct attn_read_iter {
+	struct xe_gt *gt;
+	unsigned int i;
+	unsigned int size;
+	u8 *bits;
+};
+
+static int read_eu_attentions_mcr(struct xe_gt *gt, void *data,
+				  u16 group, u16 instance, bool present)
+{
+	struct attn_read_iter * const iter = data;
+	unsigned int reg, row;
+
+	for (reg = 0; reg < xe_gt_eu_att_regs(gt); reg++) {
+		for (row = 0; row < XE_GT_EU_ATT_ROWS; row++) {
+			u32 val;
+
+			if (iter->i + sizeof(val) > iter->size)
+				return 0;
+
+			if (present)
+				val = xe_gt_mcr_unicast_read(gt, EU_ATT(reg, row), group, instance);
+			else
+				val = 0;
+
+			memcpy(&iter->bits[iter->i], &val, sizeof(val));
+			iter->i += sizeof(val);
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * xe_gt_eu_attention_bitmap - query host attention
+ *
+ * @gt: pointer to struct xe_gt
+ * @bits: buffer to fill with the attention bits, one u32 per dss, register
+ *	and row, in the order xe_gt_foreach_dss_group_instance() walks them
+ * @bitmap_size: size of @bits in bytes, a multiple of sizeof(u32)
+ *
+ * Return: 0 on success, negative otherwise.
+ */
+int xe_gt_eu_attention_bitmap(struct xe_gt *gt, u8 *bits,
+			      unsigned int bitmap_size)
+{
+	struct attn_read_iter iter = {
+		.gt = gt,
+		.i = 0,
+		.size = bitmap_size,
+		.bits = bits
+	};
+
+	/* One u32 of attention state per dss, register and row */
+	BUILD_BUG_ON(MAX_THREADS * MAX_EUS_PER_ROW / 8 != sizeof(u32));
+
+	if (XE_WARN_ON(!IS_ALIGNED(bitmap_size, sizeof(u32))))
+		return -EINVAL;
+
+	return xe_gt_foreach_dss_group_instance(gt, read_eu_attentions_mcr, &iter);
+}
+
+/**
+ * xe_gt_eu_threads_needing_attention - Query host attention
+ *
+ * @gt: pointer to struct xe_gt
+ *
+ * Return: 1 if threads waiting host attention, 0 otherwise.
+ */
+int xe_gt_eu_threads_needing_attention(struct xe_gt *gt)
+{
+	int err;
+
+	err = xe_gt_foreach_dss_group_instance(gt, read_first_attention_mcr, NULL);
+
+	/*
+	 * Negative is a suspended device, a forcewake timeout, a GT without
+	 * EUs or an empty dss mask. Report no attention, the next scan looks
+	 * again.
+	 */
+	return err < 0 ? 0 : err;
+}
diff --git a/drivers/gpu/drm/xe/xe_gt_debug.h b/drivers/gpu/drm/xe/xe_gt_debug.h
new file mode 100644
index 000000000000..d5a13e8667c4
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_gt_debug.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_GT_DEBUG_H_
+#define _XE_GT_DEBUG_H_
+
+#include <linux/bits.h>
+#include <linux/math.h>
+
+struct xe_gt;
+
+#define XE_GT_EU_ATT_ROWS 2u
+
+unsigned int xe_gt_eu_att_regs(struct xe_gt *gt);
+
+int xe_gt_eu_threads_needing_attention(struct xe_gt *gt);
+int xe_gt_foreach_dss_group_instance(struct xe_gt *gt,
+				     int (*fn)(struct xe_gt *gt,
+					       void *data,
+					       u16 group,
+					       u16 instance,
+					       bool present),
+				     void *data);
+
+int xe_gt_eu_attention_bitmap_size(struct xe_gt *gt);
+int xe_gt_eu_attention_bitmap(struct xe_gt *gt, u8 *bits,
+			      unsigned int bitmap_size);
+
+#endif
diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
index 838802482ada..fe91abaccb14 100644
--- a/drivers/gpu/drm/xe/xe_oa.c
+++ b/drivers/gpu/drm/xe/xe_oa.c
@@ -27,6 +27,7 @@
 #include "xe_bb.h"
 #include "xe_bo.h"
 #include "xe_device.h"
+#include "xe_eudebug.h"
 #include "xe_exec_queue.h"
 #include "xe_force_wake.h"
 #include "xe_gt.h"
diff --git a/drivers/gpu/drm/xe/xe_reg_sr.c b/drivers/gpu/drm/xe/xe_reg_sr.c
index eab84a528d12..ca943956602f 100644
--- a/drivers/gpu/drm/xe/xe_reg_sr.c
+++ b/drivers/gpu/drm/xe/xe_reg_sr.c
@@ -105,7 +105,8 @@ static struct xe_reg sanitize_mcr(struct xe_reg_sr *sr,
 
 int xe_reg_sr_add(struct xe_reg_sr *sr,
 		  const struct xe_reg_sr_entry *e,
-		  struct xe_gt *gt)
+		  struct xe_gt *gt,
+		  bool overwrite)
 {
 	unsigned long idx = e->reg.addr;
 	struct xe_reg_sr_entry *pentry = xa_load(&sr->xa, idx);
@@ -115,15 +116,27 @@ int xe_reg_sr_add(struct xe_reg_sr *sr,
 	reg = sanitize_mcr(sr, e, gt);
 
 	if (pentry) {
-		if (!compatible_entries(pentry, e)) {
+		if (!overwrite && !compatible_entries(pentry, e)) {
 			ret = -EINVAL;
 			goto fail;
 		}
 
 		pentry->clr_bits |= e->clr_bits;
-		pentry->set_bits |= e->set_bits;
 		pentry->read_mask |= e->read_mask;
 
+		/*
+		 * Overwrite replaces the bits this entry owns instead of
+		 * accumulating them, so a caller can lower bits it raised
+		 * earlier. Same result as the plain or for a caller whose
+		 * set_bits and clr_bits are the same mask, but it also holds
+		 * when set_bits is a subset of clr_bits.
+		 */
+		if (overwrite)
+			pentry->set_bits = (pentry->set_bits & ~e->clr_bits) |
+					   e->set_bits;
+		else
+			pentry->set_bits |= e->set_bits;
+
 		return 0;
 	}
 
diff --git a/drivers/gpu/drm/xe/xe_reg_sr.h b/drivers/gpu/drm/xe/xe_reg_sr.h
index d26cf4713383..8231640edb0f 100644
--- a/drivers/gpu/drm/xe/xe_reg_sr.h
+++ b/drivers/gpu/drm/xe/xe_reg_sr.h
@@ -6,6 +6,8 @@
 #ifndef _XE_REG_SR_H_
 #define _XE_REG_SR_H_
 
+#include <linux/types.h>
+
 /*
  * Reg save/restore bookkeeping
  */
@@ -28,7 +30,7 @@ void xe_reg_sr_lrc_check(struct xe_reg_sr *sr,
 			 struct drm_printer *p);
 
 int xe_reg_sr_add(struct xe_reg_sr *sr, const struct xe_reg_sr_entry *e,
-		  struct xe_gt *gt);
+		  struct xe_gt *gt, bool overwrite);
 void xe_reg_sr_apply_mmio(struct xe_reg_sr *sr, struct xe_gt *gt);
 void xe_reg_sr_apply_whitelist(struct xe_hw_engine *hwe);
 
diff --git a/drivers/gpu/drm/xe/xe_reg_whitelist.c b/drivers/gpu/drm/xe/xe_reg_whitelist.c
index cab1b578ca0e..744844f59bd5 100644
--- a/drivers/gpu/drm/xe/xe_reg_whitelist.c
+++ b/drivers/gpu/drm/xe/xe_reg_whitelist.c
@@ -193,7 +193,7 @@ static int whitelist_apply_to_hwe(struct xe_hw_engine *hwe, struct xe_reg_sr *in
 		}
 
 		xe_reg_whitelist_print_entry(&p, 0, reg, entry);
-		xe_reg_sr_add(out, &hwe_entry, hwe->gt);
+		xe_reg_sr_add(out, &hwe_entry, hwe->gt, false);
 
 		slot++;
 	}
diff --git a/drivers/gpu/drm/xe/xe_rtp.c b/drivers/gpu/drm/xe/xe_rtp.c
index 6a8d6ea68f25..56641a72f33f 100644
--- a/drivers/gpu/drm/xe/xe_rtp.c
+++ b/drivers/gpu/drm/xe/xe_rtp.c
@@ -244,7 +244,7 @@ static void rtp_add_sr_entry(const struct xe_rtp_action *action,
 
 	sr_entry.reg.addr += mmio_base;
 
-	xe_reg_sr_add(sr, &sr_entry, gt);
+	xe_reg_sr_add(sr, &sr_entry, gt, false);
 }
 
 static bool rtp_process_one_sr(const struct xe_rtp_entry_sr *entry,
diff --git a/drivers/gpu/drm/xe/xe_wa_oob.rules b/drivers/gpu/drm/xe/xe_wa_oob.rules
index da774e17cb69..6e1bd16e3329 100644
--- a/drivers/gpu/drm/xe/xe_wa_oob.rules
+++ b/drivers/gpu/drm/xe/xe_wa_oob.rules
@@ -80,3 +80,5 @@
 14028709524	GRAPHICS_VERSION(3510)
 14022766366	GRAPHICS_VERSION_RANGE(2001, 2004)
 		GRAPHICS_VERSION_RANGE(3000, 3005)
+18022722726	GRAPHICS_VERSION_RANGE(1250, 1274)
+14015474168	PLATFORM(PVC)
-- 
2.53.0

[PATCH 16/20] drm/xe/eudebug: Track debuggable exec queues

[Mika Kuoppala]

Attention handling, added with the EU control interface later in the
series, has to find the exec queue that raised attention. Track the
debuggable exec queues in a device wide list instead of walking the
exec queues of every client with a debugger attached.

v2: - enabled check under eudebug.lock, document return (Mika)

Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 .../ABI/testing/sysfs-driver-intel-xe-eudebug |  8 ++-
 drivers/gpu/drm/xe/xe_device_types.h          |  3 +
 drivers/gpu/drm/xe/xe_eudebug.c               | 66 +++++++++++++++++++
 drivers/gpu/drm/xe/xe_eudebug.h               |  5 ++
 drivers/gpu/drm/xe/xe_exec_queue.c            | 20 ++++++
 drivers/gpu/drm/xe/xe_exec_queue_types.h      |  8 +++
 6 files changed, 107 insertions(+), 3 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
index 50ab7db1f811..237dea4c6699 100644
--- a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
+++ b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
@@ -12,8 +12,8 @@ Description:	RW. Controls whether the EU debugger interface is armed for
 		While disabled, DRM_IOCTL_XE_EUDEBUG_CONNECT fails with
 		-EOPNOTSUPP.
 
-		Writing 0 while any debugger connection is still attached
-		fails with -EBUSY.
+		Writing 0 while any debugger connection is still attached,
+		or any debuggable exec queue still exists, fails with -EBUSY.
 
 		Once enabled, connecting needs no privilege beyond access to
 		the render node: a client may attach to any DRM file it holds
@@ -32,7 +32,9 @@ Description:	RW. Controls whether the EU debugger interface is armed for
 		(runalone), and as its preemption timeout is infinite it is
 		not reset if it does not yield the engine when preempted.
 		Work from other clients on those engines can therefore be
-		held off for as long as such a queue keeps running.
+		held off for as long as such a queue keeps running. Such a
+		queue also keeps the interface from being disabled until it
+		is destroyed.
 
 		Each connection allocates about 4 MiB of kernel memory for
 		its event queue, held until the connection is closed. The
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 73b576547193..354b258f4ca6 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -637,6 +637,9 @@ struct xe_device {
 		/** @eudebug.attached: number of debuggers currently attached */
 		unsigned int attached;
 
+		/** @eudebug.targets: exec queues that are debuggable */
+		struct list_head targets;
+
 		/** @eudebug.lock: protects the eudebug state of this device */
 		struct mutex lock;
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index bf4d3ac4e092..03ac45653172 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -2296,6 +2296,13 @@ static int __xe_eudebug_enable(struct xe_device *xe, bool enable,
 	if (!enable && xe->eudebug.attached)
 		return -EBUSY;
 
+	/*
+	 * A debuggable queue keeps its runalone and debug state for as long
+	 * as it lives, so do not drop the interface from under it.
+	 */
+	if (!enable && !list_empty(&xe->eudebug.targets))
+		return -EBUSY;
+
 	if (enable == __xe_eudebug_is_enabled(xe))
 		return 0;
 
@@ -2391,6 +2398,8 @@ static void xe_eudebug_lock_fini(struct drm_device *dev, void *__unused)
 
 void xe_eudebug_init_early(struct xe_device *xe)
 {
+	INIT_LIST_HEAD(&xe->eudebug.targets);
+
 	/*
 	 * Not drmm_mutex_init(), which can fail and leave the lock destroyed:
 	 * the state is read under it from here on. Nothing can reach the
@@ -2466,6 +2475,63 @@ void xe_eudebug_init(struct xe_device *xe)
 		set_cap_state(xe, XE_EUDEBUG_CAP_NOT_SUPPORTED);
 }
 
+/**
+ * xe_eudebug_track_exec_queue - Make an exec queue findable on attention
+ * @q: The exec queue
+ *
+ * Adds a debuggable exec queue into the device wide target list, so that
+ * the attention handling can resolve the hardware context that raised
+ * attention back to the exec queue owning it. Membership does not depend
+ * on a debugger being attached to the owning client.
+ *
+ * Must be called only after the queue is fully initialized, as the lookup
+ * dereferences the lrc array of every queue it walks.
+ *
+ * Return: 0 on success, -EPERM if eudebug got disabled since the queue was
+ * made debuggable.
+ */
+int xe_eudebug_track_exec_queue(struct xe_exec_queue *q)
+{
+	struct xe_device *xe = gt_to_xe(q->gt);
+
+	if (!xe_exec_queue_is_debuggable(q))
+		return 0;
+
+	guard(mutex)(&xe->eudebug.lock);
+
+	/*
+	 * The property checked the enabled state, but a disable may have run
+	 * since. That refuses while queues are tracked, so look again under
+	 * the lock it checks under, or this one would slip in behind it.
+	 */
+	if (!__xe_eudebug_is_enabled(xe))
+		return -EPERM;
+
+	if (list_empty(&q->eudebug.target_link))
+		list_add_tail(&q->eudebug.target_link, &xe->eudebug.targets);
+
+	return 0;
+}
+
+/**
+ * xe_eudebug_untrack_exec_queue - Remove an exec queue from the target list
+ * @q: The exec queue
+ *
+ * Reverses xe_eudebug_track_exec_queue(). Safe to call on a queue that was
+ * never tracked, and safe to repeat.
+ */
+void xe_eudebug_untrack_exec_queue(struct xe_exec_queue *q)
+{
+	struct xe_device *xe = gt_to_xe(q->gt);
+
+	if (!xe_exec_queue_is_debuggable(q))
+		return;
+
+	guard(mutex)(&xe->eudebug.lock);
+
+	list_del_init(&q->eudebug.target_link);
+}
+
 /**
  * xe_eudebug_connect_ioctl - Connect to eudebug interface
  * @dev : DRM device
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index dc92cdbbf75e..ce61537a596e 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -54,6 +54,9 @@ void xe_eudebug_init(struct xe_device *xe);
 void xe_eudebug_init_early(struct xe_device *xe);
 bool xe_eudebug_is_enabled(struct xe_device *xe);
 
+int xe_eudebug_track_exec_queue(struct xe_exec_queue *q);
+void xe_eudebug_untrack_exec_queue(struct xe_exec_queue *q);
+
 void xe_eudebug_file_close(struct xe_file *xef);
 
 void xe_eudebug_vm_create(struct xe_file *xef, struct xe_vm *vm);
@@ -83,6 +86,8 @@ static inline int xe_eudebug_connect_ioctl(struct drm_device *dev,
 static inline void xe_eudebug_init(struct xe_device *xe) { }
 static inline void xe_eudebug_init_early(struct xe_device *xe) { }
 static inline bool xe_eudebug_is_enabled(struct xe_device *xe) { return false; }
+static inline int xe_eudebug_track_exec_queue(struct xe_exec_queue *q) { return 0; }
+static inline void xe_eudebug_untrack_exec_queue(struct xe_exec_queue *q) { }
 
 static inline void xe_eudebug_file_close(struct xe_file *xef) { }
 
diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c
index 870abbf08d2e..5f39b9cf06c0 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue.c
+++ b/drivers/gpu/drm/xe/xe_exec_queue.c
@@ -232,6 +232,9 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe,
 	INIT_LIST_HEAD(&q->vm_exec_queue_link);
 	INIT_LIST_HEAD(&q->hw_engine_group_link);
 	INIT_LIST_HEAD(&q->pxp.link);
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	INIT_LIST_HEAD(&q->eudebug.target_link);
+#endif
 	spin_lock_init(&q->multi_queue.lock);
 	spin_lock_init(&q->lrc_lookup_lock);
 	q->multi_queue.priority = XE_MULTI_QUEUE_PRIORITY_NORMAL;
@@ -514,6 +517,17 @@ struct xe_exec_queue *xe_exec_queue_create(struct xe_device *xe, struct xe_vm *v
 			goto err_post_init;
 	}
 
+	/*
+	 * Same reasoning as for the PXP list above: the attention lookup walks
+	 * xe->eudebug.targets and dereferences q->lrc[], which is only filled
+	 * in by __xe_exec_queue_init(), so the queue must not be published
+	 * there while it is half-initialized, nor ahead of anything that can
+	 * still fail, as the error paths free the queue bypassing its kref.
+	 */
+	err = xe_eudebug_track_exec_queue(q);
+	if (err)
+		goto err_post_init;
+
 	/*
 	 * Publish the LRC BO back-pointers last: past this point the queue can
 	 * only be destroyed through xe_exec_queue_destroy(), so a concurrent
@@ -684,6 +698,12 @@ void xe_exec_queue_fini(struct xe_exec_queue *q)
 	if (q->xef && atomic_dec_and_test(&q->xef->exec_queue.pending_removal))
 		wake_up_var(&q->xef->exec_queue.pending_removal);
 
+	/*
+	 * Not untracked at kill: a killed queue can still sit on the engine
+	 * with halted threads, and the attention lookup has to resolve it.
+	 */
+	xe_eudebug_untrack_exec_queue(q);
+
 	__xe_exec_queue_fini(q);
 	__xe_exec_queue_free(q);
 }
diff --git a/drivers/gpu/drm/xe/xe_exec_queue_types.h b/drivers/gpu/drm/xe/xe_exec_queue_types.h
index 32893341287d..4e987f73cd1a 100644
--- a/drivers/gpu/drm/xe/xe_exec_queue_types.h
+++ b/drivers/gpu/drm/xe/xe_exec_queue_types.h
@@ -284,6 +284,14 @@ struct xe_exec_queue {
 	u64 tlb_flush_seqno;
 	/** @hw_engine_group_link: link into exec queues in the same hw engine group */
 	struct list_head hw_engine_group_link;
+
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+	/** @eudebug: eudebug specifics */
+	struct {
+		/** @eudebug.target_link: link into xe_device.eudebug.targets */
+		struct list_head target_link;
+	} eudebug;
+#endif
 	/**
 	 * @lrc_lookup_lock: Lock for protecting lrc array access. Only used when
 	 * running in parallel to queue creation is possible.
-- 
2.53.0

[PATCH 17/20] drm/xe/eudebug: Introduce EU control interface

[Mika Kuoppala]

From: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

Introduce EU control functionality, which allows the EU debugger
to interrupt, resume, and inform about the current state of
EU threads during execution. Provide an abstraction layer,
so that in the future the GuC will only need to provide appropriate
callbacks.

Based on an implementation created by the authors and others in the
i915 driver.

v2: - checkpatch (Maciej)
    - lrc index off by one fix (Mika)
    - checkpatch (Tilak)
    - 32bit fixes (Andrzej, Mika)
    - find_resource_get for client (Mika)
v3: - fw ref (Mika)
    - attention register naming
v4: - fused off handling (Dominik)
    - squash xe3 parts and ptl attentions (Mika)
v5: - s/ioctl_lock/exec_queue.lock to avoid wrong lock order (Mika)
v6: - require seqno MBZ (Mika)
v7: - error return consistency (Mika)
v8: - consolidate boilerplate and lrc sanity check (Mika)
v9: - keep FEE when setting FEH, user_arg reserved (Sashiko)
v10: - take runtime pm for hw access (Sashiko)
     - gate sriov (Sashiko)
     - more careful logging (Sashiko)
     - proper teardown if no runtime active
     - don't warn on wrong context (Claude)
     - documentation fixes (Claude)
     - byte size iter size check for safety (Claude)
v11: - interrupt on the engine the queue runs on (Sashiko)
v12: - don't reset media gts on enable (review-prompts)
v13: - no -EBUSY on discovery, OA reads cap_state under its gt_lock (Mika)

Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 .../ABI/testing/sysfs-driver-intel-xe-eudebug |   9 +
 Documentation/gpu/xe/xe_eudebug.rst           |   6 +
 drivers/gpu/drm/xe/regs/xe_engine_regs.h      |   1 +
 drivers/gpu/drm/xe/xe_device_types.h          |   5 +-
 drivers/gpu/drm/xe/xe_eudebug.c               | 162 +++-
 drivers/gpu/drm/xe/xe_eudebug.h               |   5 +
 drivers/gpu/drm/xe/xe_eudebug_hw.c            | 735 ++++++++++++++++++
 drivers/gpu/drm/xe/xe_eudebug_hw.h            |   7 +
 drivers/gpu/drm/xe/xe_eudebug_types.h         |  25 +
 drivers/gpu/drm/xe/xe_oa.c                    |  60 ++
 drivers/gpu/drm/xe/xe_oa.h                    |   1 +
 include/uapi/drm/xe_drm_eudebug.h             |  68 ++
 12 files changed, 1081 insertions(+), 3 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
index 237dea4c6699..fec47a40144a 100644
--- a/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
+++ b/Documentation/ABI/testing/sysfs-driver-intel-xe-eudebug
@@ -15,6 +15,15 @@ Description:	RW. Controls whether the EU debugger interface is armed for
 		Writing 0 while any debugger connection is still attached,
 		or any debuggable exec queue still exists, fails with -EBUSY.
 
+		Changing the setting resets each gt that has render or
+		compute engines, so that the debug registers take effect.
+		Exec queues with jobs in flight on that gt are banned by the
+		reset.
+
+		Writing 1 while an OA stream is open fails with -EBUSY, and
+		while the interface is enabled opening an OA stream fails
+		with -EBUSY.
+
 		Once enabled, connecting needs no privilege beyond access to
 		the render node: a client may attach to any DRM file it holds
 		a descriptor for, including its own. Any such client can
diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index bf5d1f0ce0a0..b5aed372f4d7 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -89,3 +89,9 @@ VM Access
 
 .. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
    :identifiers: drm_xe_eudebug_vm_open
+
+EU/HW Control
+=============
+
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_eu_control
diff --git a/drivers/gpu/drm/xe/regs/xe_engine_regs.h b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
index aaec9d02b986..9cee40460de9 100644
--- a/drivers/gpu/drm/xe/regs/xe_engine_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
@@ -171,6 +171,7 @@
 #define   IDLE_DELAY				REG_GENMASK(20, 0)
 
 #define RING_CURRENT_LRCA(base)			XE_REG((base) + 0x240)
+#define   CURRENT_LRCA_VALID			REG_BIT(0)
 
 #define RING_CONTEXT_CONTROL(base)		XE_REG((base) + 0x244, XE_REG_OPTION_MASKED)
 #define	  CTX_CTRL_PXP_ENABLE			REG_BIT(10)
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 354b258f4ca6..6be8ea04f8f6 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -629,8 +629,9 @@ struct xe_device {
 		u64 session_count;
 
 		/**
-		 * @eudebug.cap_state: eudebug capability state, protected by
-		 * @eudebug.lock
+		 * @eudebug.cap_state: eudebug capability state. Written
+		 * under @eudebug.lock and the OA gt_lock of every gt, read
+		 * under either, see set_cap_state()
 		 */
 		enum xe_eudebug_cap_state cap_state;
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 03ac45653172..4fab8304046b 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -17,11 +17,15 @@
 #include "xe_debug_data_types.h"
 #include "xe_device.h"
 #include "xe_eudebug.h"
+#include "xe_eudebug_hw.h"
 #include "xe_eudebug_types.h"
 #include "xe_eudebug_vm.h"
 #include "xe_exec_queue.h"
+#include "xe_gt.h"
 #include "xe_hw_engine.h"
 #include "xe_macros.h"
+#include "xe_oa.h"
+#include "xe_pm.h"
 #include "xe_sriov_pf.h"
 #include "xe_sync.h"
 #include "xe_vm.h"
@@ -551,12 +555,32 @@ static int event_fifo_pending(struct xe_eudebug *d,
 	return 0;
 }
 
-/* Read and written under xe->eudebug.lock, see @eudebug.cap_state */
+/*
+ * xe_oa_stream_open_ioctl_locked() checks the state under the OA gt_lock,
+ * where it cannot take xe->eudebug.lock: __xe_eudebug_enable() calls into
+ * OA with that held. So write it under both, and either is enough to read
+ * it.
+ */
 static void set_cap_state(struct xe_device *xe, enum xe_eudebug_cap_state state)
 {
+	struct xe_gt *gt;
+	u8 id;
+
 	lockdep_assert_held(&xe->eudebug.lock);
 
+	/* Without OA the gt_locks are not even initialized */
+	if (!xe->oa.xe) {
+		xe->eudebug.cap_state = state;
+		return;
+	}
+
+	for_each_gt(gt, xe, id)
+		mutex_lock_nested(&gt->oa.gt_lock, id);
+
 	xe->eudebug.cap_state = state;
+
+	for_each_gt(gt, xe, id)
+		mutex_unlock(&gt->oa.gt_lock);
 }
 
 static bool __xe_eudebug_is_enabled(struct xe_device *xe)
@@ -573,6 +597,19 @@ bool xe_eudebug_is_enabled(struct xe_device *xe)
 	return __xe_eudebug_is_enabled(xe);
 }
 
+/**
+ * xe_eudebug_is_enabled_oa - xe_eudebug_is_enabled() for OA
+ * @gt: the gt whose OA gt_lock the caller holds
+ *
+ * Return: true if eudebug is enabled, see set_cap_state()
+ */
+bool xe_eudebug_is_enabled_oa(struct xe_gt *gt)
+{
+	lockdep_assert_held(&gt->oa.gt_lock);
+
+	return gt_to_xe(gt)->eudebug.cap_state == XE_EUDEBUG_CAP_ENABLED;
+}
+
 static struct xe_eudebug *
 __xe_eudebug_get(struct xe_file *xef)
 {
@@ -896,6 +933,42 @@ struct xe_vm *xe_eudebug_vm_get(struct xe_eudebug *d, u32 id)
 	return vm;
 }
 
+/*
+ * The map keys queues by pointer without holding a reference. While
+ * attached, a destroy removes the handle ahead of the final put, and a
+ * detach clears the map in the same critical section that detaches us, so
+ * a handle found under the lock is never stale.
+ */
+struct xe_exec_queue *xe_eudebug_exec_queue_get(struct xe_eudebug *d, u32 id)
+{
+	struct xe_eudebug_resource *r =
+		resource_from_type(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE);
+	struct xe_exec_queue *eq;
+
+	guard(mutex)(&d->lock);
+
+	if (__xe_eudebug_is_detached(d))
+		return ERR_PTR(-ENOTCONN);
+
+	eq = find_resource__unlocked(r, id);
+	if (eq)
+		xe_exec_queue_get(eq);
+
+	return eq ?: ERR_PTR(-EINVAL);
+}
+
+struct xe_lrc *xe_eudebug_find_lrc(struct xe_eudebug *d, u32 id)
+{
+	struct xe_eudebug_resource *r = resource_from_type(d, XE_EUDEBUG_RES_TYPE_LRC);
+	struct xe_lrc *lrc;
+
+	mutex_lock(&d->lock);
+	lrc = find_resource__unlocked(r, id);
+	mutex_unlock(&d->lock);
+
+	return lrc;
+}
+
 static struct drm_xe_eudebug_event *
 xe_eudebug_prepare_event(struct xe_eudebug *d, u16 type, u64 seqno, u16 flags,
 			 u32 len)
@@ -2126,6 +2199,10 @@ static long xe_eudebug_ioctl(struct file *file,
 		ret = xe_eudebug_vm_open_ioctl(d, arg);
 		eu_dbg(d, "ioctl cmd=VM_OPEN ret=%ld\n", ret);
 		break;
+	case DRM_XE_EUDEBUG_IOCTL_EU_CONTROL:
+		ret = xe_eudebug_eu_control(d, arg);
+		eu_dbg(d, "ioctl cmd=EU_CONTROL ret=%ld\n", ret);
+		break;
 	default:
 		ret = -ENOTTY;
 	}
@@ -2250,6 +2327,8 @@ xe_eudebug_connect(struct xe_device *xe,
 		goto err_fd;
 	}
 
+	xe_eudebug_hw_init(d);
+
 	eu_dbg(d, "connected session %llu", d->session);
 
 	drm_dev_get(&xe->drm);
@@ -2283,10 +2362,61 @@ void xe_eudebug_file_close(struct xe_file *xef)
 	up_write(&xef->eudebug.ioctl_lock);
 }
 
+struct eudebug_hw_update {
+	struct work_struct work;
+	struct xe_gt *gt;
+	bool enable;
+};
+
+static void eudebug_hw_update_work(struct work_struct *w)
+{
+	struct eudebug_hw_update *u = container_of(w, typeof(*u), work);
+	struct xe_gt *gt = u->gt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(gt->hw_engines); i++) {
+		if (!(gt->info.engine_mask & BIT(i)))
+			continue;
+
+		xe_eudebug_init_hw_engine(&gt->hw_engines[i], u->enable);
+	}
+
+	/*
+	 * If a reset is already pending this does not queue another one,
+	 * but that reset is queued behind us and applies the new state.
+	 */
+	xe_gt_reset_async(gt);
+}
+
+/*
+ * The new state only reaches the hardware through a reset applying
+ * hwe->reg_sr. Update the lists from gt->ordered_wq, which is where
+ * gt_reset_worker() runs, so that no reset can be walking them meanwhile,
+ * and so that a reset already in flight, which would make
+ * xe_gt_reset_async() a no-op, has either finished before the update or
+ * runs after it.
+ */
+static void eudebug_hw_update(struct xe_gt *gt, bool enable)
+{
+	struct eudebug_hw_update u = {
+		.gt = gt,
+		.enable = enable,
+	};
+
+	INIT_WORK_ONSTACK(&u.work, eudebug_hw_update_work);
+	queue_work(gt->ordered_wq, &u.work);
+	flush_work(&u.work);
+	destroy_work_on_stack(&u.work);
+
+	flush_work(&gt->reset.worker);
+}
+
 static int __xe_eudebug_enable(struct xe_device *xe, bool enable,
 			       bool *end_lockdown)
 {
+	struct xe_gt *gt;
 	int ret;
+	u8 id;
 
 	guard(mutex)(&xe->eudebug.lock);
 
@@ -2315,6 +2445,36 @@ static int __xe_eudebug_enable(struct xe_device *xe, bool enable,
 		}
 	}
 
+	/*
+	 * eudebug and OA are mutually exclusive: both own EU state, see
+	 * xe_oa_stream_open_ioctl_locked(). Publish the new state before
+	 * looking for streams, and take it back if one is found. An OA open
+	 * racing this is then either seen here, because it took the gt_lock
+	 * first and published its stream, or sees eudebug enabled under that
+	 * same lock and is refused.
+	 */
+	if (enable) {
+		set_cap_state(xe, XE_EUDEBUG_CAP_ENABLED);
+		if (xe_oa_stream_active(xe)) {
+			set_cap_state(xe, XE_EUDEBUG_CAP_DISABLED);
+			/* Nothing was started that the lockdown has to outlive */
+			if (IS_SRIOV_PF(xe))
+				xe_sriov_pf_end_lockdown(xe);
+			drm_dbg(&xe->drm, "Can't enable eudebug: OA stream is open\n");
+			return -EBUSY;
+		}
+	}
+
+	xe_pm_runtime_get(xe);
+	for_each_gt(gt, xe, id) {
+		/* No engine there to program, so no reason to reset it */
+		if (xe_gt_is_media_type(gt))
+			continue;
+
+		eudebug_hw_update(gt, enable);
+	}
+	xe_pm_runtime_put(xe);
+
 	set_cap_state(xe, enable ?
 		      XE_EUDEBUG_CAP_ENABLED : XE_EUDEBUG_CAP_DISABLED);
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index ce61537a596e..11a40bc33e6e 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -14,6 +14,7 @@ struct drm_file;
 struct xe_debug_data;
 struct xe_device;
 struct xe_file;
+struct xe_gt;
 struct xe_vm;
 struct xe_exec_queue;
 struct xe_vma;
@@ -53,6 +54,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,
 void xe_eudebug_init(struct xe_device *xe);
 void xe_eudebug_init_early(struct xe_device *xe);
 bool xe_eudebug_is_enabled(struct xe_device *xe);
+bool xe_eudebug_is_enabled_oa(struct xe_gt *gt);
 
 int xe_eudebug_track_exec_queue(struct xe_exec_queue *q);
 void xe_eudebug_untrack_exec_queue(struct xe_exec_queue *q);
@@ -70,6 +72,8 @@ struct xe_vm *xe_eudebug_vm_get(struct xe_eudebug *d, u32 vm_id);
 
 void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q);
 void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q);
+struct xe_exec_queue *xe_eudebug_exec_queue_get(struct xe_eudebug *d, u32 id);
+struct xe_lrc *xe_eudebug_find_lrc(struct xe_eudebug *d, u32 id);
 
 void xe_eudebug_vm_bind_execute(struct xe_vm *vm, struct xe_vma_ops *ops);
 
@@ -86,6 +90,7 @@ static inline int xe_eudebug_connect_ioctl(struct drm_device *dev,
 static inline void xe_eudebug_init(struct xe_device *xe) { }
 static inline void xe_eudebug_init_early(struct xe_device *xe) { }
 static inline bool xe_eudebug_is_enabled(struct xe_device *xe) { return false; }
+static inline bool xe_eudebug_is_enabled_oa(struct xe_gt *gt) { return false; }
 static inline int xe_eudebug_track_exec_queue(struct xe_exec_queue *q) { return 0; }
 static inline void xe_eudebug_untrack_exec_queue(struct xe_exec_queue *q) { }
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.c b/drivers/gpu/drm/xe/xe_eudebug_hw.c
index 7388facfe0a5..3c025f279488 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_hw.c
+++ b/drivers/gpu/drm/xe/xe_eudebug_hw.c
@@ -24,8 +24,10 @@
 #include "xe_lrc.h"
 #include "xe_macros.h"
 #include "xe_mmio.h"
+#include "xe_pm.h"
 #include "xe_reg_sr.h"
 #include "xe_rtp.h"
+#include "xe_vm.h"
 #include "xe_wa.h"
 
 static void add_sr_entry(struct xe_hw_engine *hwe,
@@ -69,3 +71,736 @@ void xe_eudebug_init_hw_engine(struct xe_hw_engine *hwe, bool enable)
 		add_sr_entry(hwe, TD_CTL,
 			     TD_CTL_GLOBAL_DEBUG_ENABLE, enable);
 }
+
+static int read_hw_lrca_fw(struct xe_hw_engine *hwe, u32 *lrc_hw)
+{
+	u32 lrc_reg;
+
+	lrc_reg = xe_hw_engine_mmio_read32(hwe, RING_CURRENT_LRCA(0));
+
+	if (!(lrc_reg & CURRENT_LRCA_VALID))
+		return -ENOENT;
+
+	*lrc_hw = lrc_reg & GENMASK(31, 12);
+
+	return 0;
+}
+
+static int read_hw_lrca(struct xe_hw_engine *hwe, u32 *lrc_hw)
+{
+	unsigned int fw_ref;
+	int ret;
+
+	fw_ref = xe_force_wake_get(gt_to_fw(hwe->gt), hwe->domain);
+	if (!fw_ref)
+		return -ETIMEDOUT;
+
+	ret = read_hw_lrca_fw(hwe, lrc_hw);
+
+	xe_force_wake_put(gt_to_fw(hwe->gt), fw_ref);
+
+	return ret;
+}
+
+static bool lrca_equals(u32 a, u32 b)
+{
+	return (a & GENMASK(31, 12)) == (b & GENMASK(31, 12));
+}
+
+static int match_exec_queue_lrca(struct xe_exec_queue *q, u32 lrc_hw)
+{
+	int i;
+
+	for (i = 0; i < q->width; i++)
+		if (lrca_equals(lower_32_bits(xe_lrc_descriptor(q->lrc[i])), lrc_hw))
+			return i;
+
+	return -1;
+}
+
+static int rcu_debug1_engine_index(const struct xe_hw_engine * const hwe)
+{
+	if (hwe->class == XE_ENGINE_CLASS_RENDER) {
+		XE_WARN_ON(hwe->instance);
+		return 0;
+	}
+
+	XE_WARN_ON(hwe->instance > 3);
+
+	return hwe->instance + 1;
+}
+
+static u32 engine_status_xe1(const struct xe_hw_engine * const hwe,
+			     u32 rcu_debug1)
+{
+	const unsigned int first = 7;
+	const unsigned int incr = 3;
+	const unsigned int i = rcu_debug1_engine_index(hwe);
+	const unsigned int shift = first + (i * incr);
+
+	return (rcu_debug1 >> shift) & RCU_DEBUG_1_ENGINE_STATUS;
+}
+
+static u32 engine_status_xe2(const struct xe_hw_engine * const hwe,
+			     u32 rcu_debug1)
+{
+	const unsigned int first = 7;
+	const unsigned int incr = 4;
+	const unsigned int i = rcu_debug1_engine_index(hwe);
+	const unsigned int shift = first + (i * incr);
+
+	return (rcu_debug1 >> shift) & RCU_DEBUG_1_ENGINE_STATUS;
+}
+
+static u32 engine_status_xe3(const struct xe_hw_engine * const hwe,
+			     u32 rcu_debug1)
+{
+	const unsigned int first = 6;
+	const unsigned int incr = 4;
+	const unsigned int i = rcu_debug1_engine_index(hwe);
+	const unsigned int shift = first + (i * incr);
+
+	return (rcu_debug1 >> shift) & RCU_DEBUG_1_ENGINE_STATUS;
+}
+
+static u32 engine_status(const struct xe_hw_engine * const hwe,
+			 u32 rcu_debug1)
+{
+	u32 status = 0;
+
+	if (GRAPHICS_VER(gt_to_xe(hwe->gt)) < 20)
+		status = engine_status_xe1(hwe, rcu_debug1);
+	else if (GRAPHICS_VER(gt_to_xe(hwe->gt)) < 30)
+		status = engine_status_xe2(hwe, rcu_debug1);
+	else if (GRAPHICS_VER(gt_to_xe(hwe->gt)) < 35)
+		status = engine_status_xe3(hwe, rcu_debug1);
+	else
+		XE_WARN_ON(GRAPHICS_VER(gt_to_xe(hwe->gt)));
+
+	return status;
+}
+
+static bool engine_has_runalone_set(const struct xe_hw_engine * const hwe,
+				    u32 rcu_debug1)
+{
+	return engine_status(hwe, rcu_debug1) & RCU_DEBUG_1_RUNALONE_ACTIVE;
+}
+
+static bool engine_has_context_set(const struct xe_hw_engine * const hwe,
+				   u32 rcu_debug1)
+{
+	return engine_status(hwe, rcu_debug1) & RCU_DEBUG_1_CONTEXT_ACTIVE;
+}
+
+static struct xe_hw_engine *get_runalone_active_hw_engine(struct xe_gt *gt)
+{
+	struct xe_hw_engine *hwe, *first = NULL;
+	unsigned int num_active, id, fw_gt, fw_render;
+	u32 val;
+
+	/* No render/compute engine, and no render domain, elsewhere */
+	if (!xe_gt_is_main_type(gt))
+		return ERR_PTR(-ENOENT);
+
+	/*
+	 * RCU_DEBUG_1 sits in the render forcewake range on gen12 and dg2.
+	 * xe_force_wake_get() wakes one domain per call, so take both.
+	 */
+	fw_gt = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+	fw_render = xe_force_wake_get(gt_to_fw(gt), XE_FW_RENDER);
+	if (!fw_gt || !fw_render) {
+		xe_force_wake_put(gt_to_fw(gt), fw_render);
+		xe_force_wake_put(gt_to_fw(gt), fw_gt);
+		drm_dbg(&gt_to_xe(gt)->drm, "eudbg: runalone failed to get force wake\n");
+		return ERR_PTR(-ETIMEDOUT);
+	}
+
+	val = xe_mmio_read32(&gt->mmio, RCU_DEBUG_1);
+	xe_force_wake_put(gt_to_fw(gt), fw_render);
+	xe_force_wake_put(gt_to_fw(gt), fw_gt);
+
+	drm_dbg(&gt_to_xe(gt)->drm, "eudbg: runalone RCU_DEBUG_1 = 0x%08x\n", val);
+
+	/*
+	 * An all-ones read means the mmio access did not land - the device is
+	 * gone or mid-reset. Decoding it would flag every engine as runalone
+	 * and active and spuriously report several active engines, so bail.
+	 */
+	if (val == ~0u)
+		return ERR_PTR(-EIO);
+
+	num_active = 0;
+	xe_eudebug_for_each_hw_engine(hwe, gt, id) {
+		bool runalone, ctx;
+
+		runalone = engine_has_runalone_set(hwe, val);
+		ctx = engine_has_context_set(hwe, val);
+
+		drm_dbg(&gt_to_xe(gt)->drm, "eudbg: engine %s: runalone=%s, context=%s\n",
+			hwe->name, runalone ? "active" : "inactive",
+			ctx ? "active" : "inactive");
+
+		/*
+		 * On discrete parts the context status can read idle while
+		 * the context has raised attention, so do not rely on the
+		 * active bit there.
+		 */
+		if (IS_DGFX(gt_to_xe(gt)))
+			ctx = true;
+
+		if (runalone && ctx) {
+			num_active++;
+
+			drm_dbg(&gt_to_xe(gt)->drm, "eudbg: runalone engine %s %s\n",
+				hwe->name, first ? "ignored" : "selected");
+			if (!first)
+				first = hwe;
+		}
+	}
+
+	if (num_active > 1)
+		drm_err(&gt_to_xe(gt)->drm, "eudbg: %u runalone engines active!\n",
+			num_active);
+
+	if (!first)
+		return ERR_PTR(-ENOENT);
+
+	return first;
+}
+
+static struct xe_exec_queue *active_hwe_to_exec_queue(struct xe_hw_engine *hwe,
+						      int *lrc_idx)
+{
+	struct xe_device *xe = gt_to_xe(hwe->gt);
+	struct xe_gt *gt = hwe->gt;
+	struct xe_exec_queue *q, *found = NULL;
+	int idx, err;
+	u32 lrc_hw;
+
+	err = read_hw_lrca(hwe, &lrc_hw);
+	if (err)
+		return ERR_PTR(err);
+
+	mutex_lock(&xe->eudebug.lock);
+	list_for_each_entry(q, &xe->eudebug.targets, eudebug.target_link) {
+		if (q->gt != gt)
+			continue;
+
+		if (q->class != hwe->class)
+			continue;
+
+		idx = match_exec_queue_lrca(q, lrc_hw);
+		if (idx < 0)
+			continue;
+
+		found = xe_exec_queue_get_unless_zero(q);
+		if (!found)
+			continue;
+
+		if (lrc_idx)
+			*lrc_idx = idx;
+
+		break;
+	}
+	mutex_unlock(&xe->eudebug.lock);
+
+	if (!found)
+		return ERR_PTR(-ENOENT);
+
+	/*
+	 * The context can switch out while we walk the target list, so
+	 * both of these are ordinary outcomes rather than driver bugs.
+	 */
+	if (read_hw_lrca(hwe, &lrc_hw)) {
+		xe_exec_queue_put(found);
+		return ERR_PTR(-ENOENT);
+	}
+
+	if (match_exec_queue_lrca(found, lrc_hw) < 0) {
+		xe_exec_queue_put(found);
+		return ERR_PTR(-ENOENT);
+	}
+
+	return found;
+}
+
+struct xe_exec_queue *
+xe_gt_runalone_active_queue_get(struct xe_gt *gt, struct xe_hw_engine **active_hwe,
+				int *lrc_idx)
+{
+	struct xe_hw_engine *active;
+
+	active = get_runalone_active_hw_engine(gt);
+	if (IS_ERR(active))
+		return ERR_CAST(active);
+
+	if (active_hwe)
+		*active_hwe = active;
+
+	return active_hwe_to_exec_queue(active, lrc_idx);
+}
+
+static int check_lrc_sanity(const struct xe_exec_queue *q,
+			    const struct xe_lrc *lrc,
+			    const int lrc_idx)
+{
+	if (lrc_idx >= q->width)
+		return -EINVAL;
+
+	if (q->lrc[lrc_idx] != lrc)
+		return -EINVAL;
+
+	return 0;
+}
+
+/*
+ * The active context was sampled before the bitmask was copied in from
+ * userspace and before eudebug.lock was taken, and the hardware may have
+ * switched since. EU_ATT and EU_ATT_CLR are gt wide, so check again just
+ * before touching them, as xe_eu_control_interrupt_all() does for TD_CTL,
+ * rather than report or clear attention of some other context.
+ */
+static int check_lrc_active(struct xe_hw_engine *hwe, struct xe_lrc *lrc)
+{
+	u32 lrc_hw;
+	int err;
+
+	err = read_hw_lrca(hwe, &lrc_hw);
+	if (err)
+		return err;
+
+	if (!lrca_equals(lower_32_bits(xe_lrc_descriptor(lrc)), lrc_hw))
+		return -EBUSY;
+
+	return 0;
+}
+
+static int do_eu_control(struct xe_eudebug *d,
+			 const struct drm_xe_eudebug_eu_control * const arg,
+			 struct drm_xe_eudebug_eu_control __user * const user_ptr)
+{
+	void __user * const bitmask_ptr = u64_to_user_ptr(arg->bitmask_ptr);
+	struct xe_device *xe = d->xe;
+	struct xe_exec_queue *q, *active;
+	struct xe_hw_engine *active_hwe;
+	struct xe_lrc *lrc;
+	unsigned int hw_attn_size, attn_size;
+	u8 *bits = NULL;
+	u64 seqno;
+	int lrc_idx;
+	int ret;
+
+	if (xe_eudebug_is_detached(d))
+		return -ENOTCONN;
+
+	/* Accept only hardware reg granularity mask */
+	if (XE_IOCTL_DBG(xe, !IS_ALIGNED(arg->bitmask_size, sizeof(u32))))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, arg->seqno))
+		return -EINVAL;
+
+	/* Handles are u32 internally, do not let the upper half alias them */
+	if (XE_IOCTL_DBG(xe, arg->exec_queue_handle > U32_MAX ||
+			 arg->lrc_handle > U32_MAX))
+		return -EINVAL;
+
+	q = xe_eudebug_exec_queue_get(d, arg->exec_queue_handle);
+	if (XE_IOCTL_DBG(xe, IS_ERR(q)))
+		return PTR_ERR(q);
+
+	/* Run control is for fault mode vms only, see exec_queue_set_eudebug() */
+	if (XE_IOCTL_DBG(xe, !xe_exec_queue_is_debuggable(q) ||
+			 !xe_vm_in_fault_mode(q->vm))) {
+		ret = -EINVAL;
+		goto queue_put;
+	}
+
+	lrc = xe_eudebug_find_lrc(d, arg->lrc_handle);
+	if (XE_IOCTL_DBG(xe, !lrc)) {
+		ret = -EINVAL;
+		goto queue_put;
+	}
+
+	if (!xe_pm_runtime_get_if_active(xe)) {
+		ret = -EIO;
+		goto queue_put;
+	}
+
+	active = xe_gt_runalone_active_queue_get(q->gt, &active_hwe, &lrc_idx);
+	if (XE_IOCTL_DBG(xe, IS_ERR(active))) {
+		ret = PTR_ERR(active);
+		goto out_pm;
+	}
+
+	if (XE_IOCTL_DBG(xe, q != active)) {
+		ret = -EBUSY;
+		goto active_put;
+	}
+
+	ret = check_lrc_sanity(q, lrc, lrc_idx);
+	if (XE_IOCTL_DBG(xe, ret))
+		goto active_put;
+
+	hw_attn_size = xe_gt_eu_attention_bitmap_size(q->gt);
+	attn_size = arg->bitmask_size;
+
+	if (attn_size > hw_attn_size)
+		attn_size = hw_attn_size;
+
+	/*
+	 * RESUME clears attention, so a short mask cannot be applied and then
+	 * grown and retried like the others: the retry would clear the
+	 * leading threads again, possibly after they hit their next
+	 * breakpoint. Report the size and refuse without touching anything.
+	 */
+	if (XE_IOCTL_DBG(xe, arg->cmd == DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME &&
+			 attn_size < hw_attn_size)) {
+		ret = -EINVAL;
+		if (put_user(hw_attn_size, &user_ptr->bitmask_size))
+			ret = -EFAULT;
+		goto active_put;
+	}
+
+	/*
+	 * Keep the zero size case without a buffer: memdup_user() would
+	 * hand back a ZERO_SIZE_PTR, and the copy back to userspace below
+	 * expects NULL when there is nothing to copy.
+	 */
+	if (attn_size > 0) {
+		bits = memdup_user(bitmask_ptr, attn_size);
+		if (IS_ERR(bits)) {
+			ret = PTR_ERR(bits);
+			bits = NULL;
+			goto active_put;
+		}
+	}
+
+	mutex_lock(&xe->eudebug.lock);
+
+	/*
+	 * Check again under the lock: memdup_user() can stall for as long as
+	 * userspace likes, and once detached, eudebug can be disabled and an
+	 * OA stream opened. The detach is published under this lock.
+	 */
+	if (xe_eudebug_is_detached(d)) {
+		ret = -ENOTCONN;
+		goto out_unlock;
+	}
+
+	ret = -EINVAL;
+	switch (arg->cmd) {
+	case DRM_XE_EUDEBUG_EU_CONTROL_CMD_INTERRUPT_ALL:
+		/* Make sure we dont promise anything but interrupting all */
+		if (!attn_size)
+			ret = d->ops->interrupt_all(d, active, active_hwe, lrc);
+		else
+			ret = -EINVAL;
+		break;
+	case DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED:
+		ret = check_lrc_active(active_hwe, lrc);
+		if (!ret)
+			ret = d->ops->stopped(d, active, lrc, bits, attn_size);
+		break;
+	case DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME:
+		ret = check_lrc_active(active_hwe, lrc);
+		if (!ret)
+			ret = d->ops->resume(d, active, lrc, bits, attn_size);
+		break;
+	default:
+		break;
+	}
+
+	if (ret == 0)
+		seqno = atomic64_inc_return(&d->events.seqno);
+
+out_unlock:
+	mutex_unlock(&xe->eudebug.lock);
+
+	if (ret)
+		goto out_free;
+
+	if (put_user(seqno, &user_ptr->seqno)) {
+		ret = -EFAULT;
+		goto out_free;
+	}
+
+	/*
+	 * Only STOPPED fills the bitmask. RESUME reads it, and has resumed
+	 * the threads by now, so a buffer it cannot write back to must not
+	 * turn that into an error.
+	 */
+	if (arg->cmd == DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED &&
+	    copy_to_user(bitmask_ptr, bits, attn_size)) {
+		ret = -EFAULT;
+		goto out_free;
+	}
+
+	if (hw_attn_size != arg->bitmask_size)
+		if (put_user(hw_attn_size, &user_ptr->bitmask_size))
+			ret = -EFAULT;
+
+out_free:
+	kfree(bits);
+active_put:
+	xe_exec_queue_put(active);
+out_pm:
+	xe_pm_runtime_put(xe);
+queue_put:
+	xe_exec_queue_put(q);
+
+	return ret;
+}
+
+static int xe_eu_control_interrupt_all(struct xe_eudebug *d,
+				       struct xe_exec_queue *active,
+				       struct xe_hw_engine *hwe,
+				       struct xe_lrc *lrc)
+{
+	unsigned int fw_ref = 0;
+	u32 lrc_hw, lrc_hw_post;
+	u32 td_ctl;
+	int err = -EINVAL;
+
+	fw_ref = xe_force_wake_get(gt_to_fw(active->gt), hwe->domain);
+	if (!fw_ref) {
+		err = -ETIMEDOUT;
+		goto out;
+	}
+
+	/* Additional check just before issuing MMIO writes */
+	err = read_hw_lrca_fw(hwe, &lrc_hw);
+	if (err)
+		goto out;
+
+	if (!lrca_equals(lower_32_bits(xe_lrc_descriptor(lrc)), lrc_hw)) {
+		err = -EBUSY;
+		goto out;
+	}
+
+	td_ctl = xe_gt_mcr_unicast_read_any(active->gt, TD_CTL);
+
+	/* Halt on next thread dispatch */
+	if (!(td_ctl & TD_CTL_FORCE_EXTERNAL_HALT))
+		xe_gt_mcr_multicast_write(active->gt, TD_CTL,
+					  td_ctl | TD_CTL_FORCE_EXTERNAL_HALT);
+	else
+		eu_warn(d, "TD_CTL force external halt bit already set!\n");
+
+	/*
+	 * The sleep is needed because some interrupts are ignored
+	 * by the HW, hence we allow the HW some time to acknowledge
+	 * that.
+	 */
+	usleep_range(100, 110);
+
+	/* Halt regardless of thread dependencies */
+	if (!(td_ctl & TD_CTL_FORCE_EXCEPTION))
+		xe_gt_mcr_multicast_write(active->gt, TD_CTL,
+					  td_ctl | TD_CTL_FORCE_EXCEPTION |
+					  TD_CTL_FORCE_EXTERNAL_HALT);
+	else
+		eu_warn(d, "TD_CTL force exception bit already set!\n");
+
+	usleep_range(100, 110);
+
+	xe_gt_mcr_multicast_write(active->gt, TD_CTL, td_ctl &
+				  ~(TD_CTL_FORCE_EXTERNAL_HALT | TD_CTL_FORCE_EXCEPTION));
+
+	/*
+	 * In case of stopping wrong ctx emit warning.
+	 * Nothing else we can do for now.
+	 */
+	if (!read_hw_lrca_fw(hwe, &lrc_hw_post) &&
+	    !lrca_equals(lrc_hw, lrc_hw_post))
+		eu_warn(d, "interrupted wrong context 0x%08x, wanted 0x%08x\n",
+			lrc_hw_post, lrc_hw);
+
+out:
+	if (fw_ref)
+		xe_force_wake_put(gt_to_fw(active->gt), fw_ref);
+
+	return err;
+}
+
+struct ss_iter {
+	struct xe_eudebug *debugger;
+	unsigned int i;
+
+	unsigned int size;
+	u8 *bits;
+};
+
+/*
+ * check_attn_mcr() and clear_attn_mcr() run under
+ * xe_gt_foreach_dss_group_instance(), which holds the GT and render forcewake
+ * that EU_ATT and EU_ATT_CLR need.
+ */
+static int check_attn_mcr(struct xe_gt *gt, void *data,
+			  u16 group, u16 instance, bool present)
+{
+	struct ss_iter *iter = data;
+	struct xe_eudebug *d = iter->debugger;
+	unsigned int reg, row;
+
+	for (reg = 0; reg < xe_gt_eu_att_regs(gt); reg++) {
+		for (row = 0; row < XE_GT_EU_ATT_ROWS; row++) {
+			u32 val, cur = 0;
+
+			if (iter->i + sizeof(val) > iter->size)
+				return 0;
+
+			memcpy(&val, &iter->bits[iter->i], sizeof(val));
+			iter->i += sizeof(val);
+
+			if (present)
+				cur = xe_gt_mcr_unicast_read(gt, EU_ATT(reg, row), group, instance);
+
+			if ((val | cur) != cur) {
+				eu_dbg(d,
+				       "WRONG CLEAR (%u:%u:%u:%u) EU_ATT_CLR: 0x%08x; EU_ATT: 0x%08x\n",
+				       group, instance, reg, row, val, cur);
+				return -EINVAL;
+			}
+		}
+	}
+
+	return 0;
+}
+
+static int clear_attn_mcr(struct xe_gt *gt, void *data,
+			  u16 group, u16 instance, bool present)
+{
+	struct ss_iter *iter = data;
+	struct xe_eudebug *d = iter->debugger;
+	unsigned int reg, row;
+
+	for (reg = 0; reg < xe_gt_eu_att_regs(gt); reg++) {
+		for (row = 0; row < XE_GT_EU_ATT_ROWS; row++) {
+			u32 val;
+
+			if (iter->i + sizeof(val) > iter->size)
+				return 0;
+
+			memcpy(&val, &iter->bits[iter->i], sizeof(val));
+			iter->i += sizeof(val);
+
+			if (!val)
+				continue;
+
+			if (present) {
+				xe_gt_mcr_unicast_write(gt, EU_ATT_CLR(reg, row), val,
+							group, instance);
+
+				eu_dbg(d,
+				       "EU_ATT_CLR: (%u:%u:%u:%u): 0x%08x\n",
+				       group, instance, reg, row, val);
+			} else {
+				eu_dbg(d,
+				       "EU_ATT_CLR: (%u:%u:%u:%u): 0x%08x to fused off dss\n",
+				       group, instance, reg, row, val);
+			}
+		}
+	}
+
+	return 0;
+}
+
+static int xe_eu_control_resume(struct xe_eudebug *d,
+				struct xe_exec_queue *active,
+				struct xe_lrc *lrc,
+				u8 *bits, unsigned int bitmask_size)
+{
+	struct ss_iter iter = {
+		.debugger = d,
+		.i = 0,
+		.size = bitmask_size,
+		.bits = bits
+	};
+
+	/*
+	 * hsdes: 18021122357
+	 * We need to avoid clearing attention bits that are not set
+	 * in order to avoid the EOT hang on PVC.
+	 */
+	if (GRAPHICS_VERx100(d->xe) == 1260) {
+		int err;
+
+		err = xe_gt_foreach_dss_group_instance(active->gt, check_attn_mcr, &iter);
+		if (err)
+			return err;
+
+		iter.i = 0;
+	}
+
+	return xe_gt_foreach_dss_group_instance(active->gt, clear_attn_mcr, &iter);
+}
+
+static int xe_eu_control_stopped(struct xe_eudebug *d,
+				 struct xe_exec_queue *active,
+				 struct xe_lrc *lrc,
+				 u8 *bits, unsigned int bitmask_size)
+{
+	return xe_gt_eu_attention_bitmap(active->gt, bits, bitmask_size);
+}
+
+static const struct xe_eudebug_eu_control_ops eu_control = {
+	.interrupt_all = xe_eu_control_interrupt_all,
+	.stopped = xe_eu_control_stopped,
+	.resume = xe_eu_control_resume,
+};
+
+void xe_eudebug_hw_init(struct xe_eudebug *d)
+{
+	d->ops = &eu_control;
+}
+
+long xe_eudebug_eu_control(struct xe_eudebug *d, const u64 arg)
+{
+	struct drm_xe_eudebug_eu_control __user * const user_ptr =
+		u64_to_user_ptr(arg);
+	struct drm_xe_eudebug_eu_control user_arg;
+	struct xe_device *xe = d->xe;
+	int ret;
+
+	BUILD_BUG_ON(!(_IOC_DIR(DRM_XE_EUDEBUG_IOCTL_EU_CONTROL) & _IOC_WRITE));
+	BUILD_BUG_ON(!(_IOC_DIR(DRM_XE_EUDEBUG_IOCTL_EU_CONTROL) & _IOC_READ));
+	BUILD_BUG_ON(_IOC_SIZE(DRM_XE_EUDEBUG_IOCTL_EU_CONTROL) != sizeof(user_arg));
+
+	if (copy_from_user(&user_arg,
+			   user_ptr,
+			   sizeof(user_arg)))
+		return -EFAULT;
+
+	if (XE_IOCTL_DBG(xe, user_arg.flags))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, user_arg.reserved))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe,
+			 user_arg.cmd != DRM_XE_EUDEBUG_EU_CONTROL_CMD_INTERRUPT_ALL &&
+			 user_arg.cmd != DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED &&
+			 user_arg.cmd != DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME))
+		return -EINVAL;
+
+	if (XE_IOCTL_DBG(xe, IS_SRIOV_VF(xe)))
+		return -EPERM;
+
+	if (!access_ok(u64_to_user_ptr(user_arg.bitmask_ptr), user_arg.bitmask_size))
+		return -EFAULT;
+
+	eu_dbg(d,
+	       "eu_control: cmd=%u, flags=0x%x, exec_queue_handle=%llu, bitmask_size=%u\n",
+	       user_arg.cmd, user_arg.flags, user_arg.exec_queue_handle,
+	       user_arg.bitmask_size);
+
+	ret = do_eu_control(d, &user_arg, user_ptr);
+
+	eu_dbg(d,
+	       "eu_control: cmd=%u, flags=0x%x, exec_queue_handle=%llu, bitmask_size=%u ret=%d\n",
+	       user_arg.cmd, user_arg.flags, user_arg.exec_queue_handle,
+	       user_arg.bitmask_size, ret);
+
+	return ret;
+}
diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.h b/drivers/gpu/drm/xe/xe_eudebug_hw.h
index 447e83cd4e38..8c9d8fea1189 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_hw.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_hw.h
@@ -14,8 +14,15 @@ struct xe_gt;
 
 #if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
 
+void xe_eudebug_hw_init(struct xe_eudebug *d);
 void xe_eudebug_init_hw_engine(struct xe_hw_engine *hwe, bool enable);
 
+long xe_eudebug_eu_control(struct xe_eudebug *d, const u64 arg);
+
+struct xe_exec_queue *xe_gt_runalone_active_queue_get(struct xe_gt *gt,
+						      struct xe_hw_engine **active_hwe,
+						      int *lrc_idx);
+
 #endif /* CONFIG_DRM_XE_EUDEBUG */
 
 #endif /* _XE_EUDEBUG_HW_H_ */
diff --git a/drivers/gpu/drm/xe/xe_eudebug_types.h b/drivers/gpu/drm/xe/xe_eudebug_types.h
index a8a440c62c4f..9cbcadecb2a7 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_types.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_types.h
@@ -19,6 +19,10 @@
 #include <uapi/drm/xe_drm_eudebug.h>
 
 struct xe_device;
+struct xe_eudebug;
+struct xe_hw_engine;
+struct xe_exec_queue;
+struct xe_lrc;
 
 /**
  * enum xe_eudebug_cap_state - eudebug capability state
@@ -68,6 +72,24 @@ struct xe_eudebug_resource {
 #define XE_EUDEBUG_RES_TYPE_LRC		2
 #define XE_EUDEBUG_RES_TYPE_COUNT	(XE_EUDEBUG_RES_TYPE_LRC + 1)
 
+/**
+ * struct xe_eudebug_eu_control_ops - interface for eu thread
+ * state control backend
+ */
+struct xe_eudebug_eu_control_ops {
+	/** @interrupt_all: interrupts workload active on given hwe */
+	int (*interrupt_all)(struct xe_eudebug *e, struct xe_exec_queue *q,
+			     struct xe_hw_engine *hwe, struct xe_lrc *lrc);
+
+	/** @resume: resumes threads reflected by bitmask active on given hwe */
+	int (*resume)(struct xe_eudebug *e, struct xe_exec_queue *q,
+		      struct xe_lrc *lrc, u8 *bitmap, unsigned int bitmap_size);
+
+	/** @stopped: returns bitmap reflecting threads which signal attention */
+	int (*stopped)(struct xe_eudebug *e, struct xe_exec_queue *q,
+		       struct xe_lrc *lrc, u8 *bitmap, unsigned int bitmap_size);
+};
+
 /**
  * struct xe_eudebug - Top level struct for eudebug: the connection
  */
@@ -182,6 +204,9 @@ struct xe_eudebug {
 		/** @acks.tree: pending acks by seqnos */
 		struct rb_root tree;
 	} acks;
+
+	/** @ops: operations for eu_control */
+	const struct xe_eudebug_eu_control_ops *ops;
 };
 
 #endif /* _XE_EUDEBUG_TYPES_H_ */
diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
index fe91abaccb14..52b3f342344a 100644
--- a/drivers/gpu/drm/xe/xe_oa.c
+++ b/drivers/gpu/drm/xe/xe_oa.c
@@ -1865,6 +1865,40 @@ static int xe_oa_stream_init(struct xe_oa_stream *stream,
 	return ret;
 }
 
+/**
+ * xe_oa_stream_active() - Whether any OA unit has a stream open
+ * @xe: the &xe_device
+ *
+ * Return: true if a stream is open on any OA unit of any gt.
+ */
+bool xe_oa_stream_active(struct xe_device *xe)
+{
+	struct xe_gt *gt;
+	u8 id;
+
+	/* Without OA the gt_locks are not even initialized */
+	if (!xe->oa.xe)
+		return false;
+
+	for_each_gt(gt, xe, id) {
+		unsigned int i;
+		bool active = false;
+
+		mutex_lock(&gt->oa.gt_lock);
+		for (i = 0; i < gt->oa.num_oa_units; i++)
+			if (gt->oa.oa_unit[i].exclusive_stream) {
+				active = true;
+				break;
+			}
+		mutex_unlock(&gt->oa.gt_lock);
+
+		if (active)
+			return true;
+	}
+
+	return false;
+}
+
 static int xe_oa_stream_open_ioctl_locked(struct xe_oa *oa,
 					  struct xe_oa_open_param *param)
 {
@@ -1880,6 +1914,21 @@ static int xe_oa_stream_open_ioctl_locked(struct xe_oa *oa,
 		goto exit;
 	}
 
+	/*
+	 * eudebug and OA both change EU and engine state that the other
+	 * relies on, such as the debug registers and workarounds eudebug
+	 * keeps in hwe->reg_sr and the settings OA changes and restores
+	 * around a stream. They do not have to run at the same time, so
+	 * refuse rather than interleave. xe_eudebug_enable() checks the
+	 * other direction with xe_oa_stream_active(), under the same
+	 * gt_lock held here.
+	 */
+	if (xe_eudebug_is_enabled_oa(param->hwe->gt)) {
+		drm_dbg(&oa->xe->drm, "OA not available while eudebug is enabled\n");
+		ret = -EBUSY;
+		goto exit;
+	}
+
 	ret = drm_syncobj_create(&ufence_syncobj, DRM_SYNCOBJ_CREATE_SIGNALED,
 				 NULL);
 	if (ret)
@@ -2090,6 +2139,17 @@ int xe_oa_stream_open_ioctl(struct drm_device *dev, u64 data, struct drm_file *f
 			ret = -EOPNOTSUPP;
 			goto err_exec_q;
 		}
+
+		/*
+		 * A debuggable queue runs with a preemption timeout of 0 so
+		 * that a debugger can hold its threads halted, and a stream
+		 * may change and restore engine settings around it. Keep
+		 * debuggable queues out of OA.
+		 */
+		if (XE_IOCTL_DBG(oa->xe, xe_exec_queue_is_debuggable(param.exec_q))) {
+			ret = -EBUSY;
+			goto err_exec_q;
+		}
 	}
 
 	/*
diff --git a/drivers/gpu/drm/xe/xe_oa.h b/drivers/gpu/drm/xe/xe_oa.h
index e510826f9efc..e40a6c418788 100644
--- a/drivers/gpu/drm/xe/xe_oa.h
+++ b/drivers/gpu/drm/xe/xe_oa.h
@@ -17,6 +17,7 @@ struct xe_hw_engine;
 int xe_oa_init(struct xe_device *xe);
 int xe_oa_register(struct xe_device *xe);
 int xe_oa_stream_open_ioctl(struct drm_device *dev, u64 data, struct drm_file *file);
+bool xe_oa_stream_active(struct xe_device *xe);
 int xe_oa_add_config_ioctl(struct drm_device *dev, u64 data, struct drm_file *file);
 int xe_oa_remove_config_ioctl(struct drm_device *dev, u64 data, struct drm_file *file);
 u32 xe_oa_timestamp_frequency(struct xe_gt *gt);
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index 4bb01a3c4fe4..fd498480df84 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -41,6 +41,7 @@ extern "C" {
 #define DRM_XE_EUDEBUG_IOCTL_READ_EVENT		_IOWR('j', 0x40, struct drm_xe_eudebug_event)
 #define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT		_IOW('j', 0x41, struct drm_xe_eudebug_ack)
 #define DRM_XE_EUDEBUG_IOCTL_VM_OPEN		_IOW('j', 0x42, struct drm_xe_eudebug_vm_open)
+#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL		_IOWR('j', 0x43, struct drm_xe_eudebug_eu_control)
 
 /**
  * struct drm_xe_eudebug_event - Base type of event delivered by xe_eudebug.
@@ -407,6 +408,73 @@ struct drm_xe_eudebug_vm_open {
 	__u64 timeout_ns;
 };
 
+/**
+ * struct drm_xe_eudebug_eu_control - Control EU states
+ *
+ * Issue commands to execution units in hardware.
+ *
+ * With DRM_XE_EUDEBUG_IOCTL_EU_CONTROL debugger can
+ * interrupt all threads on execution units, query thread
+ * state and resume execution.
+ *
+ * :c:member:`drm_xe_eudebug_eu_control.seqno`
+ * will be updated to the timeline point when
+ * the command was issued.
+ *
+ * :c:member:`drm_xe_eudebug_eu_control.cmd` can
+ * be following:
+ *
+ * *DRM_XE_EUDEBUG_EU_CONTROL_CMD_INTERRUPT_ALL*
+ *  will instruct hardware to stop all threads on EUs
+ *  for exec_queue:lrc. This command takes no bitmask, so
+ *  :c:member:`drm_xe_eudebug_eu_control.bitmask_size`
+ *  must be set to zero.
+ *
+ * *DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED*
+ *  returns the bitmask for threads that are
+ *  in so called attention state.
+ *
+ * *DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME*
+ *  resumes the threads whose bit is set in the bitmask.
+ *
+ */
+struct drm_xe_eudebug_eu_control {
+	/** @cmd: Command for execution units */
+#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_INTERRUPT_ALL	0
+#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED		1
+#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME		2
+	__u32 cmd;
+
+	/** @flags: Flags, must be set to zero */
+	__u32 flags;
+
+	/** @seqno: Seqno, must be set to zero */
+	__u64 seqno;
+
+	/** @exec_queue_handle: Exec queue handle for the command */
+	__u64 exec_queue_handle;
+
+	/** @lrc_handle: LRC handle for the command */
+	__u64 lrc_handle;
+
+	/** @reserved: Reserved field, must be set to zero */
+	__u32 reserved;
+
+	/**
+	 * @bitmask_size: Bitmask size in bytes
+	 *
+	 * Only the leading bytes that fit are used. If this differs from
+	 * the size the hardware bitmask needs, it is overwritten with that
+	 * size on return, so a short buffer can be grown and the command
+	 * retried. RESUME does not take a short buffer: it fails with
+	 * -EINVAL and resumes nothing, but still reports the size.
+	 */
+	__u32 bitmask_size;
+
+	/** @bitmask_ptr: Bitmask pointer, each bit is one thread */
+	__u64 bitmask_ptr;
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.53.0

[PATCH 18/20] drm/xe/eudebug: Introduce per device attention scan worker

[Mika Kuoppala]

From: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

Scan for EU debugging attention bits periodically to detect if some EU
thread has entered the system routine (SIP) due to EU thread exception.

Make the scanning interval 50 times slower when there are no
debuggable exec queues and no debugger connection open. Send an
attention event whenever we see attention with a debugger present.

Based on work by the authors and others who worked on attentions in
i915.

v2: - use xa_array for files
    - null ptr deref fix for non-debugged context (Dominik)
    - checkpatch (Tilak)
    - use discovery_lock during list traversal
v3: - engine status per gen improvements, force_wake ref
    - __counted_by (Mika)
v4: - attention register naming (Dominik)
v5: - free event on error (Mika)
v6: - annotate data race on extending the poll interval (Mika)
v7: - sysfs race fix, return error instead of XA_WARN_ON (Sashiko)
    - attention_poll_stop to async to avoid deadlock (lockdep)
v8: - don't send if there are no attentions
    - stop before cancel (Claude)
    - use a device workqueue instead of system_dfl_wq (Maciej, Claude)
v9: - use mem_is_zero for the attention bitmap (Sashiko)
v10: - assert attention bitmask alignment (Sashiko)
     - static align the attention bitmap
v11: - adapt to runalone queue get signature (Sashiko)
v12: - resolve the attention owner once, bail out early without a
       debugger and don't log it as an error (Mika)
     - dont disconnect on destroy race (Claude)
v13: - poll state under eudebug.lock, no discovery special case (Mika)
v14: - stamp attention seqno before the handle lookups, document why an
       in-flight poll round outliving a disable is harmless (Sashiko)

Signed-off-by: Dominik Grzegorzek <dominik.grzegorzek@intel.com>
Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 Documentation/gpu/xe/xe_eudebug.rst   |   3 +
 drivers/gpu/drm/xe/xe_device_types.h  |  12 +
 drivers/gpu/drm/xe/xe_eudebug.c       | 492 +++++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_eudebug.h       |   1 +
 drivers/gpu/drm/xe/xe_eudebug_hw.c    |  16 +-
 drivers/gpu/drm/xe/xe_eudebug_hw.h    |   2 +
 drivers/gpu/drm/xe/xe_eudebug_types.h |   2 +-
 include/uapi/drm/xe_drm_eudebug.h     |  41 +++
 8 files changed, 553 insertions(+), 16 deletions(-)

diff --git a/Documentation/gpu/xe/xe_eudebug.rst b/Documentation/gpu/xe/xe_eudebug.rst
index b5aed372f4d7..5cc681617d60 100644
--- a/Documentation/gpu/xe/xe_eudebug.rst
+++ b/Documentation/gpu/xe/xe_eudebug.rst
@@ -78,6 +78,9 @@ Resource Event Types
 .. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
    :identifiers: drm_xe_eudebug_event_vm_bind_ufence
 
+.. kernel-doc:: include/uapi/drm/xe_drm_eudebug.h
+   :identifiers: drm_xe_eudebug_event_eu_attention
+
 Acknowledging Events
 --------------------
 
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 6be8ea04f8f6..2cc5472d2708 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -653,6 +653,18 @@ struct xe_device {
 
 		/** @eudebug.wq: used for the async session disconnect */
 		struct workqueue_struct *wq;
+
+		/** @eudebug.attention_wq: used for the attention poll work */
+		struct workqueue_struct *attention_wq;
+
+		/** @eudebug.attention_dwork: attention poll work */
+		struct delayed_work attention_dwork;
+
+		/**
+		 * @eudebug.send_attentions: handle attentions or ignore,
+		 * protected by @eudebug.lock
+		 */
+		bool send_attentions;
 	} eudebug;
 #endif
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 4fab8304046b..9b1f2f97b603 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -7,6 +7,8 @@
 #include <linux/cleanup.h>
 #include <linux/dma-resv.h>
 #include <linux/poll.h>
+#include <linux/sched/mm.h>
+#include <linux/string.h>
 #include <linux/uaccess.h>
 
 #include <drm/drm_drv.h>
@@ -22,6 +24,8 @@
 #include "xe_eudebug_vm.h"
 #include "xe_exec_queue.h"
 #include "xe_gt.h"
+#include "xe_gt_debug.h"
+#include "xe_guc_submit.h"
 #include "xe_hw_engine.h"
 #include "xe_macros.h"
 #include "xe_oa.h"
@@ -642,6 +646,17 @@ xe_eudebug_get(struct xe_file *xef)
 	return __xe_eudebug_get(xef);
 }
 
+/*
+ * Without the ioctl_lock, the debugger can be seen while discovery is
+ * still relaying the target's resources, so a resource may not have a
+ * handle yet.
+ */
+struct xe_eudebug *
+xe_eudebug_get_nolock(struct xe_file *xef)
+{
+	return __xe_eudebug_get(xef);
+}
+
 static int xe_eudebug_queue_event(struct xe_eudebug *d,
 				  struct drm_xe_eudebug_event *event)
 {
@@ -2218,6 +2233,362 @@ static const struct file_operations fops = {
 	.compat_ioctl	= compat_ptr_ioctl,
 };
 
+/*
+ * The uapi tells userspace the attention bitmask sits 8 byte aligned within
+ * the event, so it may be scanned as u32 or u64 words. That holds today
+ * because the preceding members are u64, but a field added ahead of the
+ * array could quietly break it, so pin it down here. Only the offset can
+ * be: plain __u64 members are 4 byte aligned on i386.
+ */
+static_assert(offsetof(struct drm_xe_eudebug_event_eu_attention, bitmask) %
+	      sizeof(__u64) == 0);
+
+static int send_attention_event(struct xe_eudebug *d, struct xe_exec_queue *q,
+				int lrc_idx, void *bitmap, unsigned int size,
+				u64 seqno)
+{
+	struct drm_xe_eudebug_event_eu_attention *e;
+	struct drm_xe_eudebug_event *event;
+	const u32 sz = struct_size(e, bitmask, size);
+	int h_queue, h_lrc;
+	int ret;
+
+	if (lrc_idx < 0 || lrc_idx >= q->width)
+		return -EINVAL;
+
+	h_queue = find_handle(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE, q);
+	if (h_queue < 0)
+		return h_queue;
+
+	h_lrc = find_handle(d, XE_EUDEBUG_RES_TYPE_LRC, q->lrc[lrc_idx]);
+	if (h_lrc < 0)
+		return h_lrc;
+
+	mutex_lock(&d->lock);
+	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_EU_ATTENTION,
+					 seqno, DRM_XE_EUDEBUG_EVENT_STATE_CHANGE,
+					 sz);
+
+	e = cast_event(e, event);
+	e->exec_queue_handle = h_queue;
+	e->lrc_handle = h_lrc;
+	e->bitmask_size = size;
+
+	memcpy(e->bitmask, bitmap, size);
+	ret = xe_eudebug_queue_event(d, event);
+	mutex_unlock(&d->lock);
+
+	return ret;
+}
+
+static int relay_gt_attention(struct xe_eudebug *d, struct xe_exec_queue *q,
+			      struct xe_hw_engine *hwe, int lrc_idx)
+{
+	const u32 size = xe_gt_eu_attention_bitmap_size(q->gt);
+	void *bitmask;
+	u64 seqno;
+	int ret;
+
+	/* If we fail at this time, assume we manage to send eventually */
+	bitmask = kvzalloc(size, GFP_KERNEL);
+	if (!bitmask)
+		return 0;
+
+	/*
+	 * Attention is state, so the event is stamped where that state is
+	 * observed: as the bits are read. That is also before the handle
+	 * lookups in send_attention_event(), so a handle still found there is
+	 * destroyed with a later seqno. A round that ends up sending nothing
+	 * leaves a gap in the numbering, which the uapi allows.
+	 */
+	seqno = atomic64_inc_return(&d->events.seqno);
+
+	ret = xe_gt_eu_attention_bitmap(q->gt, bitmask, size);
+
+	/*
+	 * EU_ATT is gt wide, and q was looked up before reading it. Only if
+	 * q is still the active context after the read are the bits its
+	 * own. Otherwise drop the round, rather than relay attention of some
+	 * other context under q's handles.
+	 */
+	if (!ret) {
+		ret = xe_eudebug_check_lrc_active(hwe, q->lrc[lrc_idx]);
+		if (ret == -EBUSY || ret == -ENOENT) {
+			eu_dbg(d, "context switched while reading attention, skipped");
+			ret = 0;
+			goto out;
+		}
+	}
+
+	if (!ret && !mem_is_zero(bitmask, size)) {
+		ret = send_attention_event(d, q, lrc_idx, bitmask, size, seqno);
+
+		if (ret == -ENOENT) {
+			eu_dbg(d, "attention on a queue without handles, skipped");
+			ret = 0;
+		}
+
+		/*
+		 * Unlike the other events, attention is state, and it is sent
+		 * again every round for as long as it lasts. A full fifo
+		 * already holds unread copies of it, so drop this one rather
+		 * than the debugger, and with it the target.
+		 */
+		if (ret == -ENOSPC) {
+			eu_dbg(d, "event fifo full, attention skipped");
+			ret = 0;
+		}
+
+		if (ret && !xe_eudebug_is_detached(d))
+			xe_eudebug_disconnect(d, ret);
+	}
+
+out:
+	kvfree(bitmask);
+
+	return ret;
+}
+
+static void handle_attention_fail(struct xe_gt *gt, int gt_id, int ret)
+{
+	/*
+	 * The owner of the attention cannot be resolved, but that covers two
+	 * different things. The context may have switched out between reading
+	 * the bits and looking it up, for instance after a debugger resumed
+	 * its threads, and then there is nothing left to service. Or the
+	 * threads are still waiting, in a workload that is not in debug mode,
+	 * and nobody is ever going to answer them.
+	 *
+	 * The error tells the two apart no better than the bits do: a
+	 * workload not in debug mode is not runalone either, so it fails the
+	 * same lookup. Read the attention again instead. If it is gone, the
+	 * scan raced something that serviced it; if it is still there, it is
+	 * stuck and the gt has to be reset to get the engine back.
+	 */
+	if (!xe_gt_eu_threads_needing_attention(gt))
+		return;
+
+	/* TODO: error capture */
+	drm_err_ratelimited(&gt_to_xe(gt)->drm,
+			    "gt:%d unable to handle eu attention ret = %d\n",
+			    gt_id, ret);
+
+	xe_gt_reset_async(gt);
+}
+
+/*
+ * Attention on a debuggable queue that nobody is going to service: its
+ * client has no debugger, it was never attached or it detached while
+ * threads were halted, or the queue is already dead. The gt still has to
+ * be reset to get the engine back. But userspace did this on its own, so
+ * it is not an error, and the reset logs itself already.
+ *
+ * TODO: recover orphaned threads without a full gt reset.
+ */
+static void handle_attention_no_debugger(struct xe_gt *gt, int gt_id)
+{
+	/* Same race as in handle_attention_fail() */
+	if (!xe_gt_eu_threads_needing_attention(gt))
+		return;
+
+	drm_dbg(&gt_to_xe(gt)->drm,
+		"gt:%d eu attention with no debugger attached, resetting\n",
+		gt_id);
+
+	xe_gt_reset_async(gt);
+}
+
+/*
+ * Forcewake not acking, or a read that did not land because the device is
+ * suspending or mid-reset, says nothing about the attention itself. A gt
+ * reset over it would throw away threads a debugger is holding, so look
+ * again next round instead.
+ */
+static bool attention_error_is_transient(int err)
+{
+	return err == -ETIMEDOUT || err == -EIO;
+}
+
+/*
+ * Resolve the queue that raised attention on @gt and the debugger of its
+ * client, once, and relay the attention to it. The debugger being absent
+ * is decided here, before anything is sent, and not inferred afterwards
+ * from how sending failed.
+ */
+static void handle_gt_attention(struct xe_gt *gt, int gt_id)
+{
+	struct xe_exec_queue *q;
+	struct xe_hw_engine *hwe;
+	struct xe_eudebug *d;
+	int ret, lrc_idx;
+
+	q = xe_gt_runalone_active_queue_get(gt, &hwe, &lrc_idx);
+	if (IS_ERR(q)) {
+		if (!attention_error_is_transient(PTR_ERR(q)))
+			handle_attention_fail(gt, gt_id, PTR_ERR(q));
+		return;
+	}
+
+	/*
+	 * A killed, banned or reset queue can stay resident with halted
+	 * threads. It is going away and nothing can be resumed on it, so do
+	 * not relay its attention to a debugger, just get the engine back.
+	 */
+	if (q->ops->reset_status(q)) {
+		xe_exec_queue_put(q);
+		handle_attention_no_debugger(gt, gt_id);
+		return;
+	}
+
+	d = xe_eudebug_get_nolock(q->vm->xef);
+	if (!d) {
+		xe_exec_queue_put(q);
+		handle_attention_no_debugger(gt, gt_id);
+		return;
+	}
+
+	/*
+	 * A queue discovery has not reached yet has no handle, and
+	 * relay_gt_attention() skips it until the next round.
+	 */
+	ret = relay_gt_attention(d, q, hwe, lrc_idx);
+
+	/* Detached under us: sending failing is a consequence, not a cause */
+	if (ret && xe_eudebug_is_detached(d)) {
+		xe_eudebug_put(d);
+		xe_exec_queue_put(q);
+		handle_attention_no_debugger(gt, gt_id);
+		return;
+	}
+
+	xe_eudebug_put(d);
+	xe_exec_queue_put(q);
+
+	if (ret && !attention_error_is_transient(ret))
+		handle_attention_fail(gt, gt_id, ret);
+}
+
+/*
+ * A gt reset is in flight from the moment xe_gt_reset_async() marks GuC
+ * submission stopped until the restart at the end of gt_reset_worker()
+ * starts it again. The attention poll takes forcewake and does steered
+ * reads on every dss, none of which may race a reset of any gt on the
+ * device, so it has to stay out for that whole window.
+ */
+static bool attention_poll_reset_in_flight(struct xe_device *xe)
+{
+	struct xe_gt *gt;
+	u8 id;
+
+	if (xe_device_is_in_reset(xe))
+		return true;
+
+	for_each_gt(gt, xe, id) {
+		if (xe_guc_read_stopped(&gt->uc.guc))
+			return true;
+
+		if (work_busy(&gt->reset.worker))
+			return true;
+	}
+
+	return false;
+}
+
+static void attention_poll_work(struct work_struct *work)
+{
+	struct xe_device *xe = container_of(work, typeof(*xe),
+					    eudebug.attention_dwork.work);
+	const unsigned int poll_interval_ms = 100;
+	long delay = msecs_to_jiffies(poll_interval_ms);
+	bool polled = false;
+	struct xe_gt *gt;
+	u8 gt_id;
+
+	scoped_guard(mutex, &xe->eudebug.lock) {
+		if (!xe->eudebug.send_attentions)
+			return;
+	}
+
+	/* Skip this round and look again at the poll interval */
+	if (attention_poll_reset_in_flight(xe))
+		goto requeue;
+
+	if (xe_pm_runtime_get_if_active(xe)) {
+		for_each_gt(gt, xe, gt_id) {
+			if (gt->info.type != XE_GT_TYPE_MAIN)
+				continue;
+
+			if (!xe_gt_eu_threads_needing_attention(gt))
+				continue;
+
+			handle_gt_attention(gt, gt_id);
+		}
+
+		xe_pm_runtime_put(xe);
+	}
+
+	polled = true;
+
+requeue:
+	mutex_lock(&xe->eudebug.lock);
+
+	/*
+	 * Poll fast while the device carries any debuggable exec queue,
+	 * whether or not a debugger is attached to its client. Attention has
+	 * to be noticed with low latency to be worth anything, and for an
+	 * offline capture there is no debugger to key off at all.
+	 *
+	 * Poll fast for an attached debugger too, even with no debuggable
+	 * queue: attention from a workload that is not in debug mode has to
+	 * be spotted and the gt reset, and a debugger is the one waiting for
+	 * that to happen.
+	 *
+	 * Decided under the lock that queues and debuggers are added under,
+	 * so one that arrives afterwards wakes the poll, see
+	 * xe_eudebug_track_exec_queue().
+	 */
+	if (polled && list_empty(&xe->eudebug.targets) && !xe->eudebug.attached)
+		delay = round_jiffies_up_relative(5 * HZ);
+
+	if (xe->eudebug.send_attentions)
+		queue_delayed_work(xe->eudebug.attention_wq,
+				   &xe->eudebug.attention_dwork, delay);
+
+	mutex_unlock(&xe->eudebug.lock);
+}
+
+/*
+ * The cancel is deliberately the async one. This runs under
+ * xe->eudebug.lock and attention_poll_work() takes that same lock, so
+ * waiting for the worker here would deadlock.
+ *
+ * Clearing send_attentions is therefore what actually stops the poll: the
+ * worker checks it under the lock before each round and before rearming,
+ * so the cancel only has to take out a queued run.
+ *
+ * A round already past that check can still finish after the disable has
+ * returned. Disabling requires that no debugger is attached and no
+ * debuggable exec queue exists, so the round has nothing to relay, and
+ * the most it can do is queue a gt reset, which eudebug_hw_update() does
+ * anyway. Only a PF has to wait it out, before VFs may be enabled, and
+ * xe_eudebug_enable() does that once the lock is dropped.
+ */
+static void xe_eudebug_attention_poll_stop(struct xe_device *xe)
+{
+	lockdep_assert_held(&xe->eudebug.lock);
+
+	xe->eudebug.send_attentions = false;
+	cancel_delayed_work(&xe->eudebug.attention_dwork);
+}
+
+static void xe_eudebug_attention_poll_start(struct xe_device *xe)
+{
+	lockdep_assert_held(&xe->eudebug.lock);
+
+	xe->eudebug.send_attentions = true;
+	mod_delayed_work(xe->eudebug.attention_wq, &xe->eudebug.attention_dwork, 0);
+}
+
 /*
  * Tell lockdep up front where the lock sits, so that a violation is
  * reported on the first run rather than once both paths have been seen.
@@ -2329,6 +2700,18 @@ xe_eudebug_connect(struct xe_device *xe,
 
 	xe_eudebug_hw_init(d);
 
+	/*
+	 * add_debugger() dropped eudebug.lock, and the target closing its
+	 * fd since then detaches us and lets a disable run to completion,
+	 * lockdown release included, or the device got unbound. Starting the
+	 * poll regardless would leave it reading gt registers with eudebug
+	 * off and VFs allowed, or after they are unmapped.
+	 */
+	scoped_guard(mutex, &xe->eudebug.lock) {
+		if (__xe_eudebug_is_enabled(xe))
+			xe_eudebug_attention_poll_start(xe);
+	}
+
 	eu_dbg(d, "connected session %llu", d->session);
 
 	drm_dev_get(&xe->drm);
@@ -2478,14 +2861,19 @@ static int __xe_eudebug_enable(struct xe_device *xe, bool enable,
 	set_cap_state(xe, enable ?
 		      XE_EUDEBUG_CAP_ENABLED : XE_EUDEBUG_CAP_DISABLED);
 
-	/*
-	 * The lockdown must outlive any vm fsync() cache flush still writing
-	 * gt registers. Waiting those out under eudebug.lock would stall
-	 * everything else behind their polls, so hand the release to the
-	 * caller, which runs after we drop it.
-	 */
-	if (!enable)
+	if (enable) {
+		xe_eudebug_attention_poll_start(xe);
+	} else {
+		xe_eudebug_attention_poll_stop(xe);
+		/*
+		 * The lockdown must outlive the attention worker: it reads GT
+		 * mmio, and VFs must not be enabled while it does. Stopping it
+		 * for good needs a synchronous cancel, which cannot happen
+		 * under eudebug.lock because the worker takes that same lock.
+		 * Hand the release to the caller, which runs after we drop it.
+		 */
 		*end_lockdown = IS_SRIOV_PF(xe);
+	}
 
 	return 0;
 }
@@ -2499,6 +2887,13 @@ static int xe_eudebug_enable(struct xe_device *xe, bool enable)
 	if (ret || !end_lockdown)
 		return ret;
 
+	/*
+	 * eudebug.lock is dropped by now, so the worker can finish and this
+	 * cannot deadlock against it. Only once it is truly stopped is it
+	 * safe to let VFs be enabled again.
+	 */
+	cancel_delayed_work_sync(&xe->eudebug.attention_dwork);
+
 	/*
 	 * A debugger detached before the disable may still be in a vm
 	 * fsync() cache flush, writing gt registers. A flush only starts
@@ -2508,6 +2903,17 @@ static int xe_eudebug_enable(struct xe_device *xe, bool enable)
 	mutex_lock(&xe->eudebug.flush_lock);
 	mutex_unlock(&xe->eudebug.flush_lock);
 
+	/*
+	 * An enable may have got in between dropping eudebug.lock and the
+	 * cancel, and the cancel then took out the poll it queued. That
+	 * enable holds a lockdown of its own, so only the poll needs putting
+	 * back.
+	 */
+	scoped_guard(mutex, &xe->eudebug.lock) {
+		if (__xe_eudebug_is_enabled(xe))
+			xe_eudebug_attention_poll_start(xe);
+	}
+
 	xe_sriov_pf_end_lockdown(xe);
 
 	return 0;
@@ -2551,6 +2957,41 @@ static void xe_eudebug_sysfs_fini(void *arg)
 			  &dev_attr_enable_eudebug.attr);
 }
 
+/*
+ * xe_eudebug_fini() only runs at the final drm_dev_put(), which an open
+ * file, or a debugger connection, can hold off long past unbind. devm
+ * teardown unmaps the registers the poll reads well before that, and the
+ * device is left runtime active, so stop the poll at unbind.
+ *
+ * devm actions unwind in reverse, so the sysfs knob is gone by now. But
+ * xe ioctls do not hold off unplug, and a connect already past its entry
+ * check can still start the poll. It only does so under eudebug.lock with
+ * eudebug enabled, so mark it disabled under that lock first.
+ */
+static void xe_eudebug_attention_fini(void *arg)
+{
+	struct xe_device *xe = arg;
+
+	scoped_guard(mutex, &xe->eudebug.lock) {
+		set_cap_state(xe, XE_EUDEBUG_CAP_DISABLED);
+		xe_eudebug_attention_poll_stop(xe);
+	}
+
+	cancel_delayed_work_sync(&xe->eudebug.attention_dwork);
+}
+
+static void xe_eudebug_fini(struct drm_device *dev, void *__unused)
+{
+	struct xe_device *xe = to_xe_device(dev);
+
+	xe_assert(xe, !xe->eudebug.attached);
+	xe_assert(xe, list_empty(&xe->eudebug.targets));
+
+	scoped_guard(mutex, &xe->eudebug.lock)
+		xe_eudebug_attention_poll_stop(xe);
+	cancel_delayed_work_sync(&xe->eudebug.attention_dwork);
+}
+
 static void xe_eudebug_lock_fini(struct drm_device *dev, void *__unused)
 {
 	mutex_destroy(&to_xe_device(dev)->eudebug.lock);
@@ -2559,6 +3000,7 @@ static void xe_eudebug_lock_fini(struct drm_device *dev, void *__unused)
 void xe_eudebug_init_early(struct xe_device *xe)
 {
 	INIT_LIST_HEAD(&xe->eudebug.targets);
+	INIT_DELAYED_WORK(&xe->eudebug.attention_dwork, attention_poll_work);
 
 	/*
 	 * Not drmm_mutex_init(), which can fail and leave the lock destroyed:
@@ -2607,6 +3049,25 @@ void xe_eudebug_init(struct xe_device *xe)
 		goto out_err;
 	xe->eudebug.wq = wq;
 
+	/*
+	 * Freezable, so that system suspend drains the poll before any gt
+	 * is suspended, and holds it off until everything has resumed.
+	 */
+	wq = drmm_alloc_ordered_workqueue(dev, "xe-eudebug-attn", WQ_FREEZABLE);
+	if (IS_ERR(wq)) {
+		err = PTR_ERR(wq);
+		goto out_err;
+	}
+	xe->eudebug.attention_wq = wq;
+
+	err = drmm_add_action_or_reset(&xe->drm, xe_eudebug_fini, NULL);
+	if (err)
+		goto out_err;
+
+	err = devm_add_action_or_reset(dev->dev, xe_eudebug_attention_fini, xe);
+	if (err)
+		goto out_err;
+
 	err = devm_add_action_or_reset(dev->dev, xe_eudebug_sysfs_fini, xe);
 	if (err)
 		goto out_err;
@@ -2670,6 +3131,23 @@ int xe_eudebug_track_exec_queue(struct xe_exec_queue *q)
 	if (list_empty(&q->eudebug.target_link))
 		list_add_tail(&q->eudebug.target_link, &xe->eudebug.targets);
 
+	/*
+	 * Attention polling falls back to a lazy interval while the device
+	 * carries nothing debuggable. A queue that arrives during such a
+	 * sleep would have to wait it out before any attention of its own
+	 * is noticed, which is long enough for the job timeout to kill the
+	 * workload first. Wake the poll instead.
+	 *
+	 * attention_poll_work() picks its delay and rearms under this lock,
+	 * so the wakeup always lands after that decision. A stopped poll
+	 * returns at its own guard.
+	 *
+	 * The worker takes eudebug.lock itself, but mod_delayed_work() only
+	 * queues it, so doing this under the lock cannot deadlock.
+	 */
+	mod_delayed_work(xe->eudebug.attention_wq,
+			 &xe->eudebug.attention_dwork, 0);
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug.h b/drivers/gpu/drm/xe/xe_eudebug.h
index 11a40bc33e6e..766295e0b112 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.h
+++ b/drivers/gpu/drm/xe/xe_eudebug.h
@@ -66,6 +66,7 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm);
 
 bool xe_eudebug_is_detached(struct xe_eudebug *d);
 
+struct xe_eudebug *xe_eudebug_get_nolock(struct xe_file *xef);
 void xe_eudebug_put(struct xe_eudebug *d);
 
 struct xe_vm *xe_eudebug_vm_get(struct xe_eudebug *d, u32 vm_id);
diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.c b/drivers/gpu/drm/xe/xe_eudebug_hw.c
index 3c025f279488..f5fd50f58838 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_hw.c
+++ b/drivers/gpu/drm/xe/xe_eudebug_hw.c
@@ -354,13 +354,13 @@ static int check_lrc_sanity(const struct xe_exec_queue *q,
 }
 
 /*
- * The active context was sampled before the bitmask was copied in from
- * userspace and before eudebug.lock was taken, and the hardware may have
- * switched since. EU_ATT and EU_ATT_CLR are gt wide, so check again just
- * before touching them, as xe_eu_control_interrupt_all() does for TD_CTL,
- * rather than report or clear attention of some other context.
+ * The active context is sampled once, and the hardware may switch away
+ * from it any time after. EU_ATT and EU_ATT_CLR are gt wide, so check
+ * again right before clearing them, or right after reading them, as
+ * xe_eu_control_interrupt_all() does for TD_CTL, rather than report or
+ * clear attention of some other context.
  */
-static int check_lrc_active(struct xe_hw_engine *hwe, struct xe_lrc *lrc)
+int xe_eudebug_check_lrc_active(struct xe_hw_engine *hwe, struct xe_lrc *lrc)
 {
 	u32 lrc_hw;
 	int err;
@@ -498,12 +498,12 @@ static int do_eu_control(struct xe_eudebug *d,
 			ret = -EINVAL;
 		break;
 	case DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED:
-		ret = check_lrc_active(active_hwe, lrc);
+		ret = xe_eudebug_check_lrc_active(active_hwe, lrc);
 		if (!ret)
 			ret = d->ops->stopped(d, active, lrc, bits, attn_size);
 		break;
 	case DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME:
-		ret = check_lrc_active(active_hwe, lrc);
+		ret = xe_eudebug_check_lrc_active(active_hwe, lrc);
 		if (!ret)
 			ret = d->ops->resume(d, active, lrc, bits, attn_size);
 		break;
diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.h b/drivers/gpu/drm/xe/xe_eudebug_hw.h
index 8c9d8fea1189..ba8e7cc93bff 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_hw.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_hw.h
@@ -11,6 +11,7 @@
 struct xe_eudebug;
 struct xe_hw_engine;
 struct xe_gt;
+struct xe_lrc;
 
 #if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
 
@@ -22,6 +23,7 @@ long xe_eudebug_eu_control(struct xe_eudebug *d, const u64 arg);
 struct xe_exec_queue *xe_gt_runalone_active_queue_get(struct xe_gt *gt,
 						      struct xe_hw_engine **active_hwe,
 						      int *lrc_idx);
+int xe_eudebug_check_lrc_active(struct xe_hw_engine *hwe, struct xe_lrc *lrc);
 
 #endif /* CONFIG_DRM_XE_EUDEBUG */
 
diff --git a/drivers/gpu/drm/xe/xe_eudebug_types.h b/drivers/gpu/drm/xe/xe_eudebug_types.h
index 9cbcadecb2a7..1b379cc52ba7 100644
--- a/drivers/gpu/drm/xe/xe_eudebug_types.h
+++ b/drivers/gpu/drm/xe/xe_eudebug_types.h
@@ -37,7 +37,7 @@ enum xe_eudebug_cap_state {
 	XE_EUDEBUG_CAP_ENABLED,
 };
 
-#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE
+#define XE_EUDEBUG_MAX_EVENT_TYPE DRM_XE_EUDEBUG_EVENT_EU_ATTENTION
 
 /**
  * struct xe_eudebug_handle - eudebug resource handle
diff --git a/include/uapi/drm/xe_drm_eudebug.h b/include/uapi/drm/xe_drm_eudebug.h
index fd498480df84..92fa96c35c3e 100644
--- a/include/uapi/drm/xe_drm_eudebug.h
+++ b/include/uapi/drm/xe_drm_eudebug.h
@@ -90,6 +90,7 @@ struct drm_xe_eudebug_event {
 #define DRM_XE_EUDEBUG_EVENT_VM_BIND		4
 #define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_DEBUG_DATA	5
 #define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE	6
+#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION	7
 
 	/** @flags: Flags */
 	__u16 flags;
@@ -475,6 +476,46 @@ struct drm_xe_eudebug_eu_control {
 	__u64 bitmask_ptr;
 };
 
+/**
+ * struct drm_xe_eudebug_event_eu_attention - EU Attention Event
+ *
+ * Whenever there is any thread in halted/attentions state, this
+ * event will be delivered. The event will be delivered periodically
+ * until there are no attentions detected.
+ *
+ */
+struct drm_xe_eudebug_event_eu_attention {
+	/** @base: base event */
+	struct drm_xe_eudebug_event base;
+
+	/** @exec_queue_handle: Exec queue handle for the attentions */
+	__u64 exec_queue_handle;
+
+	/** @lrc_handle: LRC handle for the attentions */
+	__u64 lrc_handle;
+
+	/**
+	 * @flags: Reserved for future use, reads as zero
+	 *
+	 * Event wide flags are carried in
+	 * :c:member:`drm_xe_eudebug_event_eu_attention.base`.
+	 */
+	__u32 flags;
+
+	/** @bitmask_size: Bitmask size in bytes for bitmask[] */
+	__u32 bitmask_size;
+
+	/**
+	 * @bitmask: Attention bits, one per thread
+	 *
+	 * @bitmask_size is always a multiple of 4, and the array is 8 byte
+	 * aligned within the event, so it can be scanned as u32 or u64
+	 * words instead of byte at a time. The content is hardware register
+	 * state in native endianness.
+	 */
+	__u8 bitmask[];
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.53.0

[PATCH 19/20] drm/xe/eudebug_test: Introduce eudebug live tests

[Mika Kuoppala]

From: Christoph Manszewski <christoph.manszewski@intel.com>

Introduce a kunit test for eudebug. For now it checks the dynamic
application of WAs.

v2: adapt to removal of call_for_each_device (Mika)
v3: s/FW_RENDER/FORCEWAKE_ALL (Mika)
v4: enforce kunit rules and naming (Michal)
v5: restore hw state if enable error (Sashiko)
    drop XE_MCR_REG def (Claude)
    fail if hw state can't be restored (Claude)
v6: read cap_state under eudebug.lock (Mika)
v7: - check all forcewake domains, VF comment to the SR-IOV patch (Sashiko)

Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 drivers/gpu/drm/xe/tests/xe_eudebug_test.c  | 241 ++++++++++++++++++++
 drivers/gpu/drm/xe/tests/xe_live_test_mod.c |   5 +
 drivers/gpu/drm/xe/xe_eudebug.c             |   4 +
 3 files changed, 250 insertions(+)
 create mode 100644 drivers/gpu/drm/xe/tests/xe_eudebug_test.c

diff --git a/drivers/gpu/drm/xe/tests/xe_eudebug_test.c b/drivers/gpu/drm/xe/tests/xe_eudebug_test.c
new file mode 100644
index 000000000000..a05fe07cd3ba
--- /dev/null
+++ b/drivers/gpu/drm/xe/tests/xe_eudebug_test.c
@@ -0,0 +1,241 @@
+// SPDX-License-Identifier: GPL-2.0 AND MIT
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#include <kunit/visibility.h>
+#include <linux/string_choices.h>
+
+#include "regs/xe_gt_regs.h"
+#include "xe_force_wake.h"
+#include "xe_gt_mcr.h"
+#include "xe_mmio.h"
+#include "tests/xe_kunit_helpers.h"
+#include "tests/xe_pci_test.h"
+#include "tests/xe_test.h"
+
+static const char *reg_to_str(struct xe_reg reg)
+{
+	if (reg.raw == TD_CTL.__reg.raw)
+		return "TD_CTL";
+	else if (reg.raw == ROW_CHICKEN.__reg.raw)
+		return "ROW_CHICKEN";
+	else if (reg.raw == ROW_CHICKEN2.__reg.raw)
+		return "ROW_CHICKEN2";
+	else if (reg.raw == ROW_CHICKEN3.__reg.raw)
+		return "ROW_CHICKEN3";
+	else
+		return "UNKNOWN REG";
+}
+
+static u32 get_reg_mask(struct xe_device *xe, struct xe_reg reg, struct kunit *test)
+{
+	u32 val = 0;
+
+	if (reg.raw == TD_CTL.__reg.raw) {
+		val = TD_CTL_BREAKPOINT_ENABLE |
+		      TD_CTL_FORCE_THREAD_BREAKPOINT_ENABLE |
+		      TD_CTL_FEH_AND_FEE_ENABLE;
+		if (GRAPHICS_VERx100(xe) >= 1250)
+			val |= TD_CTL_GLOBAL_DEBUG_ENABLE;
+	} else if (reg.raw == ROW_CHICKEN.__reg.raw) {
+		val = STALL_DOP_GATING_DISABLE;
+	} else if (reg.raw == ROW_CHICKEN2.__reg.raw) {
+		val = XEHPC_DISABLE_BTB;
+	} else if (reg.raw == ROW_CHICKEN3.__reg.raw) {
+		val = XE2_EUPEND_CHK_FLUSH_DIS;
+	} else {
+		KUNIT_FAIL(test, "Invalid register selection: 0x%x\n", reg.raw);
+	}
+
+	return val;
+}
+
+static u32 get_reg_expected(struct xe_device *xe, struct xe_reg reg,
+			    bool enable_eudebug, struct kunit *test)
+{
+	const u32 reg_mask = get_reg_mask(xe, reg, test);
+	u32 reg_bits = 0;
+
+	/*
+	 * XXX ROW_CHICKEN3 is expected set either way, and it is the odd one
+	 * out here: xe_eudebug_init_hw_engine() programs ROW_CHICKEN,
+	 * ROW_CHICKEN2 and TD_CTL, but never ROW_CHICKEN3.
+	 * XE2_EUPEND_CHK_FLUSH_DIS is set by WA 14020338487 in xe_wa.c on all
+	 * Xe2 render and compute engines, with no reference to eudebug, so as
+	 * written this arm asserts that an unrelated workaround is applied.
+	 *
+	 * Pending a check against the hardware documentation on whether
+	 * eupend flush checking has to stay disabled for eudebug.
+	 */
+	if (enable_eudebug || reg.raw == ROW_CHICKEN3.__reg.raw)
+		reg_bits = reg_mask;
+	else
+		reg_bits = 0;
+
+	return reg_bits;
+}
+
+static void check_reg(struct xe_gt *gt, bool enable_eudebug,
+		      struct xe_reg reg, struct kunit *test)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	const u32 reg_bits_expected = get_reg_expected(xe, reg, enable_eudebug, test);
+	const u32 reg_mask = get_reg_mask(xe, reg, test);
+	u32 reg_bits = 0;
+
+	if (reg.mcr)
+		reg_bits = xe_gt_mcr_unicast_read_any(gt, (struct xe_reg_mcr){.__reg = reg});
+	else
+		reg_bits = xe_mmio_read32(&gt->mmio, reg);
+
+	reg_bits &= reg_mask;
+
+	KUNIT_EXPECT_EQ_MSG(test, reg_bits_expected, reg_bits,
+			    "Invalid bits set for %s\n", reg_to_str(reg));
+}
+
+static void __check_regs(struct xe_gt *gt, bool enable_eudebug, struct kunit *test)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+
+	if (GRAPHICS_VERx100(xe) >= 1200)
+		check_reg(gt, enable_eudebug, TD_CTL.__reg, test);
+
+	if (GRAPHICS_VERx100(xe) >= 1250 && GRAPHICS_VERx100(xe) <= 1274)
+		check_reg(gt, enable_eudebug, ROW_CHICKEN.__reg, test);
+
+	if (xe->info.platform == XE_PVC)
+		check_reg(gt, enable_eudebug, ROW_CHICKEN2.__reg, test);
+
+	/* The range of Wa_14020338487, which sets the bit checked */
+	if (GRAPHICS_VERx100(xe) >= 2001 && GRAPHICS_VERx100(xe) <= 2004)
+		check_reg(gt, enable_eudebug, ROW_CHICKEN3.__reg, test);
+}
+
+static void check_regs(struct xe_device *xe, bool enable_eudebug, struct kunit *test)
+{
+	struct xe_gt *gt;
+	unsigned int fw_ref;
+	u8 id;
+
+	kunit_printk(KERN_DEBUG, test, "Check regs for eudebug %s\n",
+		     str_enabled_disabled(enable_eudebug));
+
+	for_each_gt(gt, xe, id) {
+		if (xe_gt_is_media_type(gt))
+			continue;
+
+		/* XXX: Figure out per platform proper domain */
+		fw_ref = xe_force_wake_get(gt_to_fw(gt), XE_FORCEWAKE_ALL);
+		if (!xe_force_wake_ref_has_domain(fw_ref, XE_FORCEWAKE_ALL)) {
+			/* Aborting does not return, so put the partial ref first */
+			xe_force_wake_put(gt_to_fw(gt), fw_ref);
+			KUNIT_FAIL_AND_ABORT(test, "Forcewake failed.\n");
+		}
+
+		__check_regs(gt, enable_eudebug, test);
+
+		xe_force_wake_put(gt_to_fw(gt), fw_ref);
+	}
+}
+
+struct eudebug_state_restore {
+	struct xe_device *xe;
+	struct kunit *test;
+	bool enabled;
+};
+
+static void restore_eudebug_state(void *arg)
+{
+	struct eudebug_state_restore *r = arg;
+
+	KUNIT_EXPECT_EQ_MSG(r->test, 0, xe_eudebug_enable(r->xe, r->enabled),
+			    "Failed to restore eudebug to %s\n",
+			    str_enabled_disabled(r->enabled));
+}
+
+static void toggle_reg(struct kunit *test)
+{
+	struct xe_device *xe = test->priv;
+	const bool enable_eudebug = xe_eudebug_is_enabled(xe);
+	struct eudebug_state_restore *restore;
+	bool supported;
+	int ret;
+
+	scoped_guard(mutex, &xe->eudebug.lock)
+		supported = xe->eudebug.cap_state != XE_EUDEBUG_CAP_NOT_SUPPORTED;
+
+	/* Covers an SR-IOV VF too, xe_eudebug_init() marks one unsupported */
+	if (!supported)
+		kunit_skip(test, "eudebug not supported\n");
+
+	/*
+	 * An open OA stream keeps eudebug from being enabled, and sets bits
+	 * of its own in the registers checked below.
+	 */
+	if (!enable_eudebug && xe_oa_stream_active(xe))
+		kunit_skip(test, "OA stream open\n");
+
+	kunit_printk(KERN_DEBUG, test, "Test eudebug WAs for graphics version: %u\n",
+		     GRAPHICS_VERx100(xe));
+
+	check_regs(xe, enable_eudebug, test);
+
+	/* Nothing to restore until the first toggle below */
+	restore = kunit_kzalloc(test, sizeof(*restore), GFP_KERNEL);
+	KUNIT_ASSERT_NOT_NULL(test, restore);
+	restore->xe = xe;
+	restore->test = test;
+	restore->enabled = enable_eudebug;
+	KUNIT_ASSERT_EQ(test, 0,
+			kunit_add_action_or_reset(test, restore_eudebug_state,
+						  restore));
+
+	ret = xe_eudebug_enable(xe, !enable_eudebug);
+	if (ret == -EBUSY)
+		kunit_skip(test, "eudebug toggle busy\n");
+	KUNIT_ASSERT_EQ(test, 0, ret);
+
+	check_regs(xe, !enable_eudebug, test);
+
+	ret = xe_eudebug_enable(xe, enable_eudebug);
+	KUNIT_ASSERT_EQ(test, 0, ret);
+	check_regs(xe, enable_eudebug, test);
+}
+
+static bool xe_eudebug_has_regs(struct xe_device *xe)
+{
+	return GRAPHICS_VERx100(xe) >= 1200;
+}
+
+static int xe_eudebug_test_init(struct kunit *test)
+{
+	struct xe_device *xe;
+	int ret;
+
+	ret = xe_kunit_helper_xe_device_live_test_init(test);
+	if (ret)
+		return ret;
+
+	xe = test->priv;
+	kunit_info(test, "running on graphics version: %u\n", GRAPHICS_VERx100(xe));
+
+	if (!xe_eudebug_has_regs(xe))
+		kunit_skip(test, "hardware not supported: 0x%x\n", GRAPHICS_VERx100(xe));
+
+	return 0;
+}
+
+static struct kunit_case xe_eudebug_tests[] = {
+	KUNIT_CASE_PARAM(toggle_reg, xe_pci_live_device_gen_param),
+	{}
+};
+
+VISIBLE_IF_KUNIT
+struct kunit_suite xe_eudebug_test_suite = {
+	.name = "xe_eudebug",
+	.test_cases = xe_eudebug_tests,
+	.init = xe_eudebug_test_init,
+};
+EXPORT_SYMBOL_IF_KUNIT(xe_eudebug_test_suite);
diff --git a/drivers/gpu/drm/xe/tests/xe_live_test_mod.c b/drivers/gpu/drm/xe/tests/xe_live_test_mod.c
index 87cd7db20e5f..6d92aee38509 100644
--- a/drivers/gpu/drm/xe/tests/xe_live_test_mod.c
+++ b/drivers/gpu/drm/xe/tests/xe_live_test_mod.c
@@ -25,6 +25,11 @@ kunit_test_suite(xe_guc_g2g_test_suite);
 kunit_test_suite(xe_bo_page_size_alloc_suite);
 #endif
 
+#if IS_ENABLED(CONFIG_DRM_XE_EUDEBUG)
+extern struct kunit_suite xe_eudebug_test_suite;
+kunit_test_suite(xe_eudebug_test_suite);
+#endif
+
 MODULE_AUTHOR("Intel Corporation");
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("xe live kunit tests");
diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
index 9b1f2f97b603..c8ad1a62cc6d 100644
--- a/drivers/gpu/drm/xe/xe_eudebug.c
+++ b/drivers/gpu/drm/xe/xe_eudebug.c
@@ -3190,3 +3190,7 @@ int xe_eudebug_connect_ioctl(struct drm_device *dev,
 
 	return xe_eudebug_connect(xe, file, param);
 }
+
+#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST)
+#include "tests/xe_eudebug_test.c"
+#endif
-- 
2.53.0

[PATCH 20/20] drm/xe: Add xe_client_debugfs and introduce debug_data file

[Mika Kuoppala]

From: Christoph Manszewski <christoph.manszewski@intel.com>

Create a debug_data file for each xe file/client. It lists all mapped
debug data and mimics '/proc/pid/maps'.
Each line represents a single mapping and has the following format:

  <vm id> <begin>-<end> <flags> <offset>	<pathname>

v2: trim len in debug_data_read() (Sashiko)
v3: - fix return len, avoid rewind (Sashiko)
    - use guard, kdoc, cleanups (Michal)
    - drop client ref after seq_file is done (Claude)

Cc: Sunil Khatri <sunil.khatri@amd.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Signed-off-by: Christoph Manszewski <christoph.manszewski@intel.com>
Signed-off-by: Maciej Patelczyk <maciej.patelczyk@intel.com>
Signed-off-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
---
 drivers/gpu/drm/xe/Makefile            |   3 +-
 drivers/gpu/drm/xe/xe_client_debugfs.c | 134 +++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_client_debugfs.h |  17 ++++
 drivers/gpu/drm/xe/xe_device.c         |   3 +
 4 files changed, 156 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/xe/xe_client_debugfs.c
 create mode 100644 drivers/gpu/drm/xe/xe_client_debugfs.h

diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index 356f44aac308..dface1936da3 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -366,7 +366,8 @@ ifeq ($(CONFIG_DRM_FBDEV_EMULATION),y)
 endif
 
 ifeq ($(CONFIG_DEBUG_FS),y)
-	xe-y += xe_debugfs.o \
+	xe-y += xe_client_debugfs.o \
+		xe_debugfs.o \
 		xe_gt_debugfs.o \
 		xe_gt_sriov_vf_debugfs.o \
 		xe_gt_stats.o \
diff --git a/drivers/gpu/drm/xe/xe_client_debugfs.c b/drivers/gpu/drm/xe/xe_client_debugfs.c
new file mode 100644
index 000000000000..ca0f83acf8af
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_client_debugfs.c
@@ -0,0 +1,134 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
+
+#include "xe_client_debugfs.h"
+#include "xe_debug_data.h"
+#include "xe_debug_data_types.h"
+#include "xe_device.h"
+#include "xe_device_types.h"
+#include "xe_vm.h"
+#include "xe_vm_types.h"
+
+static void debug_data_show_vm(struct seq_file *m, unsigned long vm_id,
+			       struct xe_vm *vm)
+{
+	struct xe_debug_data *dd;
+
+	guard(mutex)(&vm->debug_data.lock);
+
+	list_for_each_entry(dd, &vm->debug_data.list, link) {
+		/* Format: <vm id> <begin>-<end> <flags> <offset>\t<pathname> */
+		seq_printf(m, "%lu 0x%llx-0x%llx 0x%llx 0x%llx\t",
+			   vm_id,
+			   dd->addr,
+			   dd->addr + dd->range,
+			   dd->flags,
+			   dd->offset);
+
+		if (dd->flags & DRM_XE_VM_BIND_DEBUG_DATA_FLAG_PSEUDO)
+			seq_puts(m, xe_debug_data_pseudo_path_to_string(dd->pseudopath));
+		else
+			/*
+			 * The pathname originates from userspace; escape it the
+			 * same way seq_path() does for /proc/pid/maps so that a
+			 * newline cannot break the line oriented format.
+			 */
+			seq_escape(m, dd->pathname, "\n");
+
+		seq_putc(m, '\n');
+	}
+}
+
+static int debug_data_show(struct seq_file *m, void *data)
+{
+	struct xe_file *xef = m->private;
+	unsigned long vm_id = 0;
+	struct xe_vm *vm;
+
+	/*
+	 * xef->vm.lock only covers the lookup and reference, so hold it just
+	 * long enough to pin the next VM. A closed VM has an empty list.
+	 *
+	 * xe_file_close() puts the VMs without erasing them from xef->vm.xa,
+	 * but we never see those stale entries: drm_file_free() removes this
+	 * file before ->postclose, and the debugfs proxy waits out readers
+	 * in flight and fails later reads on an fd that is still open.
+	 */
+	for (;;) {
+		mutex_lock(&xef->vm.lock);
+		vm = xa_find(&xef->vm.xa, &vm_id, ULONG_MAX, XA_PRESENT);
+		if (vm)
+			xe_vm_get(vm);
+		mutex_unlock(&xef->vm.lock);
+
+		if (!vm)
+			break;
+
+		debug_data_show_vm(m, vm_id, vm);
+		xe_vm_put(vm);
+		vm_id++;
+	}
+
+	return 0;
+}
+
+static int debug_data_open(struct inode *inode, struct file *file)
+{
+	struct xe_file *xef = inode->i_private;
+	int ret;
+
+	ret = single_open(file, debug_data_show, xef);
+	if (ret)
+		return ret;
+
+	/* Keeps xef alive until release, see debug_data_show() for the VMs */
+	xe_file_get(xef);
+
+	return 0;
+}
+
+static int debug_data_release(struct inode *inode, struct file *file)
+{
+	struct seq_file *m = file->private_data;
+	struct xe_file *xef = m->private;
+	int ret;
+
+	ret = single_release(inode, file);
+	xe_file_put(xef);
+
+	return ret;
+}
+
+static const struct file_operations debug_data_fops = {
+	.owner = THIS_MODULE,
+	.open = debug_data_open,
+	.read = seq_read,
+	.llseek = seq_lseek,
+	.release = debug_data_release,
+};
+
+/**
+ * xe_client_debugfs_register() - Register the per-client debugfs files
+ * @xef: The xe file to register the files for
+ *
+ * Create the debugfs entries exposed under the DRM client directory of @xef.
+ * Called from xe_file_open(), after DRM has created the client directory.
+ *
+ * The files are removed by drm_file_free() before ->postclose runs, so they
+ * do not need to be torn down by the driver. Nothing is created if DRM failed
+ * to create the client directory, as a NULL parent would place the file in the
+ * debugfs root where it would outlive @xef.
+ */
+void xe_client_debugfs_register(struct xe_file *xef)
+{
+	if (IS_ERR_OR_NULL(xef->drm->debugfs_client))
+		return;
+
+	debugfs_create_file("debug_data", 0444, xef->drm->debugfs_client, xef,
+			    &debug_data_fops);
+}
diff --git a/drivers/gpu/drm/xe/xe_client_debugfs.h b/drivers/gpu/drm/xe/xe_client_debugfs.h
new file mode 100644
index 000000000000..8b2e3d230ef8
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_client_debugfs.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2026 Intel Corporation
+ */
+
+#ifndef _XE_CLIENT_DEBUGFS_H_
+#define _XE_CLIENT_DEBUGFS_H_
+
+struct xe_file;
+
+#ifdef CONFIG_DEBUG_FS
+void xe_client_debugfs_register(struct xe_file *xef);
+#else
+static inline void xe_client_debugfs_register(struct xe_file *xef) { }
+#endif
+
+#endif
diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
index 6968ae9cc300..a50147b4fc34 100644
--- a/drivers/gpu/drm/xe/xe_device.c
+++ b/drivers/gpu/drm/xe/xe_device.c
@@ -25,6 +25,7 @@
 #include "regs/xe_regs.h"
 #include "xe_bo.h"
 #include "xe_bo_evict.h"
+#include "xe_client_debugfs.h"
 #include "xe_configfs.h"
 #include "xe_debugfs.h"
 #include "xe_cpu_bind.h"
@@ -132,6 +133,8 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file)
 		put_task_struct(task);
 	}
 
+	xe_client_debugfs_register(xef);
+
 	return 0;
 }
 
-- 
2.53.0

Re: [PATCH 11/20] drm/xe/eudebug: Introduce vm bind and vm bind debug data events

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] drm/xe/eudebug: missed bind events in xe_vm_bind_ioctl() on
  prefetch -ENODATA
--

commit cd195b3b0e2ac13dc8ba630c0ab3fbbd171bde93
Author: Christoph Manszewski <c.manszewski@gmail.com>

drm/xe/eudebug: Introduce vm bind and vm bind debug data events

This commit adds eudebug VM bind and debug data operation events to notify
an attached debugger when VM bind debug data operations are executed or
discovered.

> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index bfd5a76e953fb..0b6645c86e9a8 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -3810,6 +3810,10 @@ static struct dma_fence *vm_bind_ioctl_ops_execute(struct xe_vm *vm,
>  		xe_vm_set_validation_exec(vm, &exec);
>  		fence = ops_execute(vm, vops);
>  		xe_vm_set_validation_exec(vm, NULL);
> +
> +		if (!IS_ERR(fence) || PTR_ERR(fence) == -ENODATA)
> +			xe_eudebug_vm_bind_execute(vm, vops);
> +
>  		if (IS_ERR(fence)) {
>  			if (PTR_ERR(fence) == -ENODATA)
>  				vm_bind_ioctl_ops_fini(vm, vops, NULL);

[Severity: Medium]
Can a multi-operation VM bind skip xe_eudebug_vm_bind_execute() while still
leaving debug data changes committed?

In xe_vm_bind_ioctl(), debug data add and remove operations are committed to
vm->debug_data.list during vm_bind_ioctl_process_user_extensions().  If the
same bind array also includes an SVM prefetch operation and
vm_bind_ioctl_ops_prefetch_ranges() returns -ENODATA (for example, when
prefetch_thread_func() hits a VRAM allocation failure or
xe_svm_range_get_pages() returns -EFAULT), xe_vm_bind_ioctl() jumps to
unwind_ops without calling vm_bind_ioctl_ops_execute():

drivers/gpu/drm/xe/xe_vm.c:xe_vm_bind_ioctl() {
    ...
	err = vm_bind_ioctl_ops_prefetch_ranges(vm, &vops);
	if (err)
		goto unwind_ops;

	fence = vm_bind_ioctl_ops_execute(vm, &vops);
    ...
unwind_ops:
	if (err && err != -ENODATA)
		vm_bind_ioctl_ops_unwind(vm, ops, args->num_binds);
	xe_vma_ops_fini(&vops);
    ...
free_syncs:
	if (err == -ENODATA)
		err = vm_bind_ioctl_signal_fences(vm, q, syncs, num_syncs);
    ...
}

Because -ENODATA skips vm_bind_ioctl_ops_unwind(), signals the out-fences,
and returns 0 to userspace, the debug data changes stay on
vm->debug_data.list without delivering the bind and debug data events to the
debugger.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109135549.4128280-1-mika.kuoppala@linux.intel.com?part=11

Re: [PATCH 08/20] drm/xe/eudebug: Introduce exec_queue events

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] drm/xe/eudebug: discover_client() counts non-debuggable queues
  in eq_count
--

commit 926a26106604e590716ee8ab1001dab4ed5f0566
Author: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

drm/xe/eudebug: Introduce exec_queue events

This commit adds eudebug resource creation and destruction events for
debuggable render and compute exec queues, both during live ioctls and
when discovering an existing client's resources.

> diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
> index e0ef34588d3d4..0f8b0452f9bae 100644
> --- a/drivers/gpu/drm/xe/xe_eudebug.c
> +++ b/drivers/gpu/drm/xe/xe_eudebug.c
[ ... ]
> +void xe_eudebug_exec_queue_create(struct xe_file *xef, struct xe_exec_queue *q)
> +{
> +	struct xe_eudebug *d;
> +	int err;
> +
> +	if (!xe_engine_supports_eudebug(q->class))
> +		return;
> +
> +	d = xe_eudebug_get(xef);
> +	if (!d)
> +		return;
> +
> +	err = exec_queue_create_event(d, xef, q);
> +	if (err)
> +		xe_eudebug_disconnect(d, err);
> +
> +	xe_eudebug_put(d);
> +}
> +
> +void xe_eudebug_exec_queue_destroy(struct xe_file *xef, struct xe_exec_queue *q)
> +{
> +	struct xe_eudebug *d;
> +	int err;
> +
> +	if (!xe_engine_supports_eudebug(q->class))
> +		return;
> +
> +	d = xe_eudebug_get(xef);
> +	if (!d)
> +		return;
> +
> +	err = exec_queue_destroy_event(d, xef, q);
> +	if (err)
> +		xe_eudebug_disconnect(d, err);
> +
> +	xe_eudebug_put(d);
> +}
> +
>  /*
>   * All the relevant ioctls that manipulate the xe_file resources eudebug
>   * cares about are gated behind down_read(&xef->eudebug.ioctl_lock). The
> @@ -807,9 +1069,10 @@ void xe_eudebug_vm_destroy(struct xe_file *xef, struct xe_vm *vm)
>   */
>  static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
>  {
> +	struct xe_exec_queue *q;
>  	struct xe_vm *vm;
>  	unsigned long i;
> -	unsigned int vm_count = 0;
> +	unsigned int vm_count = 0, eq_count = 0;
>  	int err = 0;
>  
>  	lockdep_assert_held_write(&xef->eudebug.ioctl_lock);
> @@ -828,10 +1091,21 @@ static int discover_client(struct xe_eudebug *d, struct xe_file *xef)
>  		vm_count++;
>  	}
>  
> +	xa_for_each(&xef->exec_queue.xa, i, q) {
> +		if (!xe_engine_supports_eudebug(q->class))
> +			continue;
> +
> +		err = exec_queue_create_event(d, xef, q);
> +		if (err)
> +			goto out;
> +
> +		eq_count++;
> +	}
> +
>  out:
> -	if (vm_count)
> -		eu_dbg(d, "Discovery found %u vms",
> -		       vm_count);
> +	if (vm_count || eq_count)
> +		eu_dbg(d, "Discovery found %u vms, %u exec_queues",
> +		       vm_count, eq_count);

[Severity: Low]
Does discover_client() count non-debuggable render and compute queues in
eq_count?

exec_queue_create_event() returns 0 without emitting an event when
!xe_exec_queue_is_debuggable(q), so any render or compute queue created
without DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG still increments eq_count.
Checking xe_exec_queue_is_debuggable(q) in discover_client(), as well as
at the top of xe_eudebug_exec_queue_create() and
xe_eudebug_exec_queue_destroy(), would also avoid taking and dropping the
debugger reference for non-debuggable queues.  Finally, the kernel-doc for
struct drm_xe_eudebug_event_exec_queue in include/uapi/drm/xe_drm_eudebug.h
could mention that only queues with DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG
are reported, matching struct drm_xe_eudebug_event_vm.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109135549.4128280-1-mika.kuoppala@linux.intel.com?part=8

Re: [PATCH 13/20] drm/xe/eudebug: Add vm open/pread/pwrite

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] drm/xe/eudebug: unlocked vm->size reads in
  __xe_eudebug_vm_access()
- [Low] drm/xe/eudebug: detach race in __xe_eudebug_vm_access() returns
  -EINVAL
--

commit a3a16f65fbfea0ccc25672bcf54427e90a80cace
Author: Mika Kuoppala <mika.kuoppala@linux.intel.com>

drm/xe/eudebug: Add vm open/pread/pwrite

This commit adds the DRM_XE_EUDEBUG_IOCTL_VM_OPEN ioctl and file operations
for reading, writing, and syncing a target client's BO-backed VM mappings.

> diff --git a/drivers/gpu/drm/xe/xe_eudebug_vm.c b/drivers/gpu/drm/xe/xe_eudebug_vm.c
> new file mode 100644
> index 0000000000000..385fcc7e8f5c2
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_eudebug_vm.c

[ ... ]

> +static ssize_t __xe_eudebug_vm_access(struct file *file,
> +				      char __user *r_buffer,
> +				      const char __user *w_buffer,
> +				      size_t count, loff_t *__pos)
> +{
> +	struct vm_file *vmf = file->private_data;
> +	struct xe_eudebug * const d = vmf->debugger;
> +	struct xe_device * const xe = d->xe;
> +	const bool write = !!w_buffer;
> +	struct xe_vm *vm;
> +	ssize_t copied = 0;
> +	ssize_t bytes_left;
> +	ssize_t ret;
> +	unsigned long alloc_len;
> +	loff_t pos = *__pos;
> +	void *k_buffer;
> +
> +	if (XE_IOCTL_DBG(d->xe, xe_eudebug_is_detached(d)))
> +		return -ENOTCONN;
> +
> +	if (XE_IOCTL_DBG(xe, write && r_buffer))
> +		return -EINVAL;
> +
> +	vm = xe_eudebug_vm_get(d, vmf->vm_handle);
> +	if (XE_IOCTL_DBG(xe, !vm))
> +		return -EINVAL;

[Severity: Low]
Can __xe_eudebug_vm_access() return -EINVAL instead of -ENOTCONN if the
target detaches between xe_eudebug_is_detached() and xe_eudebug_vm_get()?

CPU0                              CPU1
__xe_eudebug_vm_access()
  xe_eudebug_is_detached(d)
    -> false
                                  xe_eudebug_detach()
                                    d->target.xef = NULL
                                    xe_eudebug_resources_clear(d)
  xe_eudebug_vm_get(d, ...)
    -> NULL
  return -EINVAL;  <-- expected -ENOTCONN

Checking xe_eudebug_is_detached(d) when !vm, as xe_eudebug_vm_open_ioctl()
does, would return -ENOTCONN as documented in the UAPI header.

> +
> +	if (XE_IOCTL_DBG(xe, vm != vmf->vm)) {
> +		eu_warn(d, "vm_access(%s): vm handle mismatch vm_handle=%llu, flags=0x%llx, pos=%llu, count=%zu\n",
> +			str_write_read(write),
> +			vmf->vm_handle, vmf->flags, pos, count);
> +		xe_vm_put(vm);
> +		return -EINVAL;
> +	}
> +
> +	if (!count) {
> +		xe_vm_put(vm);
> +		return 0;
> +	}
> +
> +	if (pos >= vm->size) {
> +		xe_vm_put(vm);
> +		return write ? -ENOSPC : 0;
> +	}
> +
> +	count = min_t(u64, count, vm->size - pos);
> +	bytes_left = count;
> +
> +	alloc_len = min_t(unsigned long, ALIGN(count, PAGE_SIZE), 64 * SZ_1M);
> +	do  {
> +		k_buffer = __vmalloc(alloc_len, GFP_KERNEL_ACCOUNT | __GFP_NOWARN);

[Severity: Medium]
Can reading vm->size twice without holding vm->lock race with xe_vm_close()
zeroing vm->size?

drivers/gpu/drm/xe/xe_vm.c:xe_vm_close() {
	down_write(&vm->lock);
	xe_vm_lock(vm, false);

	if (xe_vm_in_fault_mode(vm))
		xe_svm_notifier_lock(vm);

	vm->size = 0;
    ...
}

CPU0                              CPU1
__xe_eudebug_vm_access()
  vm = xe_eudebug_vm_get(...)
                                  xe_vm_close(vm)
                                    down_write(&vm->lock)
  if (pos >= vm->size)
    -> false (pos == 0)
                                    vm->size = 0
                                    up_write(&vm->lock)
  count = min_t(u64, count,
                vm->size - pos)
    -> 0
  alloc_len = 0
  __vmalloc(0, ...)               <-- hits WARN_ON_ONCE(!size)

If pos > 0 when vm->size becomes 0 between the two reads, vm->size - pos
underflows u64; if vm->size is already 0 at the first read, pread() returns
0 (EOF) and pwrite() returns -ENOSPC instead of -EINVAL for a closed VM.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109135549.4128280-1-mika.kuoppala@linux.intel.com?part=13

Re: [PATCH 12/20] drm/xe/eudebug: Add ufence events with acks

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] drm/xe/eudebug: use of destroyed eudebug.lock in
  xe_eudebug_detach()
- [Low] drm/xe/eudebug: TOCTOU race in xe_eudebug_ack_event_ioctl() on
  detach
--

commit b38d3ecda4b8b7ae85aedf3b0af41c39cf303abb
Author: Mika Kuoppala <mika.kuoppala@linux.intel.com>

drm/xe/eudebug: Add ufence events with acks

This commit adds DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE events and the
DRM_XE_EUDEBUG_IOCTL_ACK_EVENT ioctl so that a debugger can stall user
fence signalling on a VM bind with debug data until the event is
acknowledged.

> diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
> index c6d08b27042d2..934dc116120e6 100644
> --- a/drivers/gpu/drm/xe/xe_eudebug.c
> +++ b/drivers/gpu/drm/xe/xe_eudebug.c

[ ... ]

> @@ -1199,6 +1330,191 @@ static int vm_bind_op_event(struct xe_eudebug *d,
>  	return err;
>  }

[ ... ]

> +static int track_ufence(struct xe_eudebug *d,
> +			struct xe_user_fence *ufence,
> +			u64 bind_ref_seqno,
> +			bool *tracked)
> +{

[ ... ]

> +	/*
> +	 * The ack and the event asking for it go in under one lock, so the
> +	 * debugger cannot ack a fence it has not been told about. Once the
> +	 * lock is dropped the ack may already be gone, so it is not touched
> +	 * after that.
> +	 */
> +	mutex_lock(&d->lock);
> +	old = rb_find_add(&ack->rb_node, &d->acks.tree, ack_insert_cmp);
> +	if (old) {
> +		mutex_unlock(&d->lock);
> +		xe_sync_ufence_put(ufence);
> +		kfree(ack);
> +		return -EEXIST;
> +	}
> +
> +	*tracked = true;
> +
> +	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE,
> +					 seqno, flags, sz);
> +
> +	e = cast_event(e, event);
> +	e->vm_bind_ref_seqno = bind_ref_seqno;
> +
> +	ret = xe_eudebug_queue_event(d, event);
> +	mutex_unlock(&d->lock);
> +
> +	return ret;
> +}

[ ... ]

> +bool xe_eudebug_ufence_track(struct xe_user_fence *ufence)
> +{
> +	struct xe_eudebug *d;
> +	u64 bind_ref_seqno;
> +	bool tracked = false;
> +	int ret;

[ ... ]

> +	if (xe_eudebug_is_detached(d)) {
> +		/* Nothing will ack this fence any more */
> +		ufence_drop_debugger(ufence);
> +		xe_eudebug_put(d);
> +		return false;
> +	}
> +
> +	ret = track_ufence(d, ufence, bind_ref_seqno, &tracked);
> +	if (ret) {
> +		/*
> +		 * If we failed before the ack made it into the tree, the
> +		 * disconnect below has nothing to release for this fence.
> +		 */
> +		if (!tracked)
> +			ufence_drop_debugger(ufence);
> +
> +		xe_eudebug_disconnect(d, ret);
> +	}
> +
> +	xe_eudebug_put(d);
> +
> +	return tracked;
> +}

[Severity: Medium]
Can xe_eudebug_ufence_track() lock a destroyed d->xe->eudebug.lock in
xe_eudebug_detach()?

As noted above kick_ufence_worker(), user_fence_worker() runs on
xe->ordered_wq, which xe_device_destroy() drains only after the drmm
actions registered in xe_device_probe() have already run.  One of those
actions is xe_eudebug_lock_fini(), which destroys xe->eudebug.lock:

drivers/gpu/drm/xe/xe_eudebug.c:xe_eudebug_lock_fini() {
	mutex_destroy(&to_xe_device(dev)->eudebug.lock);
}

A user fence holds a reference to struct xe_eudebug in
ufence->eudebug.debugger, but not a drm_device reference:

CPU0 (user_fence_worker)          CPU1
xe_eudebug_ufence_track()
  xe_eudebug_is_detached(d)
    -> false
                                  xe_eudebug_release()
                                    xe_eudebug_disconnect(d, 0)
                                    drm_dev_put(&d->xe->drm)
                                      ...
                                      xe_eudebug_lock_fini()
                                        mutex_destroy(&xe->eudebug.lock)
                                      xe_device_destroy()
                                        destroy_workqueue(xe->ordered_wq)
  track_ufence()
    xe_eudebug_queue_event()
      -> -ENOTCONN
  xe_eudebug_disconnect(d, ret)
    xe_eudebug_detach()
      mutex_lock(&d->xe->eudebug.lock)  <-- destroyed mutex

With lock debugging enabled, mutex_lock() warns on the destroyed mutex.
If track_ufence() checked __xe_eudebug_is_detached(d) under d->lock
before inserting ack into d->acks.tree, a racing detach would not need a
second xe_eudebug_disconnect() call.

[ ... ]

> @@ -1691,6 +2032,42 @@ static long xe_eudebug_read_event(struct xe_eudebug *d,
>  	return ret;
>  }
>  
> +static long
> +xe_eudebug_ack_event_ioctl(struct xe_eudebug *d,
> +			   const u64 arg)
> +{

[ ... ]

> +	if (XE_IOCTL_DBG(xe, xe_eudebug_is_detached(d)))
> +		return -ENOTCONN;
> +
> +	ack = remove_ack(d, user_arg.seqno);
> +	if (XE_IOCTL_DBG(xe, !ack))
> +		return -EINVAL;
> +
> +	handle_ack(d, ack, false);
> +
> +	return 0;
> +}

[Severity: Low]
Can xe_eudebug_ack_event_ioctl() return -EINVAL instead of -ENOTCONN when
racing with xe_eudebug_detach()?

xe_eudebug_is_detached() and remove_ack() take d->lock in separate
critical sections.  If xe_eudebug_detach() runs in between, it clears
d->target.xef and empties d->acks.tree under d->lock, so remove_ack()
finds nothing and the ioctl returns -EINVAL for a valid seqno instead of
the documented -ENOTCONN.  Checking __xe_eudebug_is_detached(d) inside
remove_ack() under d->lock would avoid the race.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109135549.4128280-1-mika.kuoppala@linux.intel.com?part=12

Re: [PATCH 18/20] drm/xe/eudebug: Introduce per device attention scan worker

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] drm/xe/eudebug: attention seqno race with discover_client()
  queue create
--

commit 9b3fd09a4bd67b0862bac3d3c0bc36809193f44f
Author: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

drm/xe/eudebug: Introduce per device attention scan worker

This commit introduces a periodic per-device delayed worker to scan for EU
debugging attention bits and send DRM_XE_EUDEBUG_EVENT_EU_ATTENTION events
when an attached debugger is present, or reset the GT if no debugger will
service the halted threads.

> diff --git a/drivers/gpu/drm/xe/xe_eudebug.c b/drivers/gpu/drm/xe/xe_eudebug.c
> index 4fab8304046b3..9b1f2f97b603b 100644
> --- a/drivers/gpu/drm/xe/xe_eudebug.c
> +++ b/drivers/gpu/drm/xe/xe_eudebug.c
[ ... ]
> @@ -2218,6 +2233,362 @@ static const struct file_operations fops = {
>  	.compat_ioctl	= compat_ptr_ioctl,
>  };
>  
> +/*
> + * The uapi tells userspace the attention bitmask sits 8 byte aligned within
> + * the event, so it may be scanned as u32 or u64 words. That holds today
> + * because the preceding members are u64, but a field added ahead of the
> + * array could quietly break it, so pin it down here. Only the offset can
> + * be: plain __u64 members are 4 byte aligned on i386.
> + */
> +static_assert(offsetof(struct drm_xe_eudebug_event_eu_attention, bitmask) %
> +	      sizeof(__u64) == 0);
> +
> +static int send_attention_event(struct xe_eudebug *d, struct xe_exec_queue *q,
> +				int lrc_idx, void *bitmap, unsigned int size,
> +				u64 seqno)
> +{
> +	struct drm_xe_eudebug_event_eu_attention *e;
> +	struct drm_xe_eudebug_event *event;
> +	const u32 sz = struct_size(e, bitmask, size);
> +	int h_queue, h_lrc;
> +	int ret;
> +
> +	if (lrc_idx < 0 || lrc_idx >= q->width)
> +		return -EINVAL;
> +
> +	h_queue = find_handle(d, XE_EUDEBUG_RES_TYPE_EXEC_QUEUE, q);
> +	if (h_queue < 0)
> +		return h_queue;
> +
> +	h_lrc = find_handle(d, XE_EUDEBUG_RES_TYPE_LRC, q->lrc[lrc_idx]);
> +	if (h_lrc < 0)
> +		return h_lrc;
> +
> +	mutex_lock(&d->lock);
> +	event = xe_eudebug_prepare_event(d, DRM_XE_EUDEBUG_EVENT_EU_ATTENTION,
> +					 seqno, DRM_XE_EUDEBUG_EVENT_STATE_CHANGE,
> +					 sz);
> +
> +	e = cast_event(e, event);
> +	e->exec_queue_handle = h_queue;
> +	e->lrc_handle = h_lrc;
> +	e->bitmask_size = size;
> +
> +	memcpy(e->bitmask, bitmap, size);
> +	ret = xe_eudebug_queue_event(d, event);
> +	mutex_unlock(&d->lock);
> +
> +	return ret;
> +}
> +
> +static int relay_gt_attention(struct xe_eudebug *d, struct xe_exec_queue *q,
> +			      struct xe_hw_engine *hwe, int lrc_idx)
> +{
> +	const u32 size = xe_gt_eu_attention_bitmap_size(q->gt);
> +	void *bitmask;
> +	u64 seqno;
> +	int ret;
> +
> +	/* If we fail at this time, assume we manage to send eventually */
> +	bitmask = kvzalloc(size, GFP_KERNEL);
> +	if (!bitmask)
> +		return 0;
> +
> +	/*
> +	 * Attention is state, so the event is stamped where that state is
> +	 * observed: as the bits are read. That is also before the handle
> +	 * lookups in send_attention_event(), so a handle still found there is
> +	 * destroyed with a later seqno. A round that ends up sending nothing
> +	 * leaves a gap in the numbering, which the uapi allows.
> +	 */
> +	seqno = atomic64_inc_return(&d->events.seqno);
> +
> +	ret = xe_gt_eu_attention_bitmap(q->gt, bitmask, size);

[Severity: Medium]
Can relay_gt_attention() send a DRM_XE_EUDEBUG_EVENT_EU_ATTENTION event
with a smaller seqno than the queue's DRM_XE_EUDEBUG_EVENT_CREATE event
when racing with discover_client()?

In xe_eudebug_attach(), add_debugger() publishes xef->eudebug.debugger
before discover_client() runs, and handle_gt_attention() looks up the
debugger via xe_eudebug_get_nolock() without holding xef->eudebug.ioctl_lock:

CPU0 (attention_wq)               CPU1 (connect ioctl)
handle_gt_attention()             xe_eudebug_attach()
  d = xe_eudebug_get_nolock(...)    discover_client()
  relay_gt_attention()                ...
    seqno = atomic64_inc_return()       exec_queue_create_event()
      --> S1                              _xe_eudebug_add_handle(LRC)
    xe_gt_eu_attention_bitmap()           mutex_lock(&d->lock)
                                          __handle_insert(EXEC_QUEUE)
                                            --> seqno = S2 (S2 > S1)
                                          mutex_unlock(&d->lock)
    send_attention_event()
      find_handle(EXEC_QUEUE)
      find_handle(LRC)
      mutex_lock(&d->lock)
      xe_eudebug_queue_event(S1)
      mutex_unlock(&d->lock)
                                          send_exec_queue_event(S2)

The attention event for q is stamped with seqno S1 < S2 and can also be
queued into d->events.fifo ahead of the EXEC_QUEUE CREATE event because
exec_queue_create_event() drops d->lock between __handle_insert() and
send_exec_queue_event(), so userspace sees an attention event for handles
that have not been created yet.

Looking up h_queue and h_lrc before stamping seqno in relay_gt_attention()
(and re-checking that the handles still exist before queueing, or holding
xef->eudebug.ioctl_lock) would ensure discovery has already assigned the
CREATE seqno first.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109135549.4128280-1-mika.kuoppala@linux.intel.com?part=18

Re: [PATCH 17/20] drm/xe/eudebug: Introduce EU control interface

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] drm/xe/eudebug: missing wedged and unplug guards in
  do_eu_control()
It also noticed 1 related pre-existing issue this patch does not
introduce:
- [High] drm/xe/oa: unbalanced de-whitelisting in
  xe_oa_stream_open_ioctl_locked() [1]
--

commit c12aefb410d72e1670863e0bab5fd107ad1622e4
Author: Dominik Grzegorzek <dominik.grzegorzek@intel.com>

drm/xe/eudebug: Introduce EU control interface

This commit adds the DRM_XE_EUDEBUG_IOCTL_EU_CONTROL ioctl to let an EU
debugger interrupt, query, and resume EU threads on debuggable execution
queues, and makes eudebug and OA streams mutually exclusive.

> diff --git a/drivers/gpu/drm/xe/xe_eudebug_hw.c b/drivers/gpu/drm/xe/xe_eudebug_hw.c
> index 7388facfe0a5d..3c025f279488f 100644
> --- a/drivers/gpu/drm/xe/xe_eudebug_hw.c
> +++ b/drivers/gpu/drm/xe/xe_eudebug_hw.c
[ ... ]
> +static int do_eu_control(struct xe_eudebug *d,
> +			 const struct drm_xe_eudebug_eu_control * const arg,
> +			 struct drm_xe_eudebug_eu_control __user * const user_ptr)
> +{
[ ... ]
> +	lrc = xe_eudebug_find_lrc(d, arg->lrc_handle);
> +	if (XE_IOCTL_DBG(xe, !lrc)) {
> +		ret = -EINVAL;
> +		goto queue_put;
> +	}
> +
> +	if (!xe_pm_runtime_get_if_active(xe)) {
> +		ret = -EIO;
> +		goto queue_put;
> +	}
> +
> +	active = xe_gt_runalone_active_queue_get(q->gt, &active_hwe, &lrc_idx);
> +	if (XE_IOCTL_DBG(xe, IS_ERR(active))) {
> +		ret = PTR_ERR(active);
> +		goto out_pm;
> +	}

[Severity: Medium]
Can do_eu_control() race with PCI device unbind or run on a wedged device
to dereference a NULL hwe->gt or access unmapped MMIO registers?

Unlike DRM character device ioctls, DRM_XE_EUDEBUG_IOCTL_EU_CONTROL is
dispatched directly on the anonymous [xe_eudebug] file, which holds a
drm_dev_get() reference that keeps struct xe_device alive after PCI
unbind.  xe_pm_runtime_get_if_active(xe) does not stop callers during or
after unbind because xe_pm_runtime_fini() calls pm_runtime_get_sync() and
pm_runtime_forbid(), leaving runtime PM active while devres tears down
the device:

CPU0                                  CPU1
do_eu_control()
  xe_pm_runtime_get_if_active(xe)
  ...
  memdup_user() / usleep_range()
                                      xe_pci_remove()
                                        drm_dev_unplug(&xe->drm)
                                        ...
                                        devres_release_all()
                                          hw_engine_fini()
                                            hwe->gt = NULL
                                          pcim_iomap_release()
  check_lrc_active(active_hwe, lrc)
    read_hw_lrca(hwe, &lrc_hw)
      gt_to_fw(hwe->gt)  <-- NULL pointer dereference

If hwe->gt is instead cleared during get_runalone_active_hw_engine(),
engine_status() dereferences hwe->gt as well, and after mmio_fini() clears
xe->mmio.regs, MMIO accesses trigger device warnings.  Sibling anonymous
fd callbacks in drivers/gpu/drm/xe/xe_eudebug_vm.c guard against both
cases via vm_file_enter(), which checks xe_wedged(xe) and wraps hardware
accesses in drm_dev_enter() / drm_dev_exit().

> diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
> index fe91abaccb14a..52b3f342344a3 100644
> --- a/drivers/gpu/drm/xe/xe_oa.c
> +++ b/drivers/gpu/drm/xe/xe_oa.c
[ ... ]
> @@ -1880,6 +1914,21 @@ static int xe_oa_stream_open_ioctl_locked(struct xe_oa *oa,
>  		goto exit;
>  	}
>  
> +	/*
> +	 * eudebug and OA both change EU and engine state that the other
> +	 * relies on, such as the debug registers and workarounds eudebug
> +	 * keeps in hwe->reg_sr and the settings OA changes and restores
> +	 * around a stream. They do not have to run at the same time, so
> +	 * refuse rather than interleave. xe_eudebug_enable() checks the
> +	 * other direction with xe_oa_stream_active(), under the same
> +	 * gt_lock held here.
> +	 */
> +	if (xe_eudebug_is_enabled_oa(param->hwe->gt)) {
> +		drm_dbg(&oa->xe->drm, "OA not available while eudebug is enabled\n");
> +		ret = -EBUSY;
> +		goto exit;
> +	}

[Severity: High]
This problem wasn't introduced by this patch, but can
xe_oa_stream_open_ioctl_locked() cause an unbalanced
xe_reg_dewhitelist_oa_regs() call [1] when opening a stream with
DRM_XE_OA_PROPERTY_SAMPLE_OA fails?

xe_oa_stream_init() sets stream->sample = param->sample, but
xe_reg_whitelist_oa_regs() is only called after xe_oa_enable_locked() and
anon_inode_getfd() both succeed:

drivers/gpu/drm/xe/xe_oa.c:xe_oa_stream_open_ioctl_locked() {
    ...
	if (!param->disabled) {
		ret = xe_oa_enable_locked(stream);
		if (ret)
			goto err_destroy;
	}

	stream_fd = anon_inode_getfd("[xe_oa]", &xe_oa_fops, stream, 0);
	if (stream_fd < 0) {
		ret = stream_fd;
		goto err_disable;
	}

	if (stream->sample)
		xe_reg_whitelist_oa_regs(stream->gt);
    ...
err_disable:
	if (!param->disabled)
		xe_oa_disable_locked(stream);
err_destroy:
	xe_oa_stream_destroy(stream);
    ...
}

If anon_inode_getfd() fails (for example with -EMFILE) or
xe_oa_enable_locked() fails, xe_oa_stream_destroy() sees stream->sample
set and calls xe_reg_dewhitelist_oa_regs(stream->gt) without a matching
xe_reg_whitelist_oa_regs().  That triggers xe_assert() and underflows
gt->oa.whitelist_count from 0 to U32_MAX (or drops another active OA
unit's whitelist reference on the same GT).

[1] https://sashiko.dev/#/bug/linux-44e618e3-ff9f-4811-9ea1-86af519616cc

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109135549.4128280-1-mika.kuoppala@linux.intel.com?part=17