[igt-dev] [PATCH i-g-t 0/3] [RFC] benchmarks/gem_wsim: added basic xe support

[igt-dev] [PATCH i-g-t 0/3] [RFC] benchmarks/gem_wsim: added basic xe support

[Marcin Bernatowicz]

Added basic xe support with few examples.
Single binary handles both i915 and Xe devices,
but workload definitions differs between i915 and xe.
Xe does not use context abstraction, introduces new VM and Exec Queue
steps and BATCH step references exec queue.
For more details see wsim/README.
Some functionality is still missing: working sets,
load balancing (need some input if/how to do it in Xe - exec queues
width?).

The tool is handy for scheduling tests, we find it useful to verify vGPU
profiles defining different execution quantum/preemption timeout settings.

There is also some rationale for the tool in following thread:
https://lore.kernel.org/dri-devel/a443495f-5d1b-52e1-9b2f-80167deb6d57@linux.intel.com/

With this patch it should be possible to run following on xe device:

gem_wsim -w benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim -c 36 -r 600

Best with drm debug logs disabled:

echo 0 > /sys/module/drm/parameters/debug

lib/xe_spin: fixed duration xe_spin capability - is already 
  under review https://patchwork.freedesktop.org/series/122624/

Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>

Marcin Bernatowicz (3):
  lib/xe_spin: fixed duration xe_spin capability
  lib/igt_device_scan: Xe get integrated/discrete card functions
  [RFC] benchmarks/gem_wsim: added basic xe support

 benchmarks/gem_wsim.c                         | 842 ++++++++++++++----
 benchmarks/wsim/README                        |  87 +-
 benchmarks/wsim/xe_cloud-gaming-60fps.wsim    |  25 +
 benchmarks/wsim/xe_example.wsim               |  28 +
 benchmarks/wsim/xe_example01.wsim             |  19 +
 benchmarks/wsim/xe_example_fence.wsim         |  23 +
 .../wsim/xe_media_load_balance_fhd26u7.wsim   |  63 ++
 lib/igt_device_scan.c                         |  34 +-
 lib/igt_device_scan.h                         |   2 +
 lib/xe/xe_spin.c                              | 121 ++-
 lib/xe/xe_spin.h                              |  27 +-
 tests/xe/xe_dma_buf_sync.c                    |   6 +-
 tests/xe/xe_exec_balancer.c                   |   9 +-
 tests/xe/xe_exec_reset.c                      |  24 +-
 tests/xe/xe_exec_threads.c                    |   7 +-
 tests/xe/xe_vm.c                              |   9 +-
 16 files changed, 1094 insertions(+), 232 deletions(-)
 create mode 100644 benchmarks/wsim/xe_cloud-gaming-60fps.wsim
 create mode 100644 benchmarks/wsim/xe_example.wsim
 create mode 100644 benchmarks/wsim/xe_example01.wsim
 create mode 100644 benchmarks/wsim/xe_example_fence.wsim
 create mode 100644 benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim

-- 
2.30.2

[igt-dev] [PATCH i-g-t 1/3] lib/xe_spin: fixed duration xe_spin capability

[Marcin Bernatowicz]

Introduced struct xe_spin_opts for xe_spin initialization,
adjusted tests to new xe_spin_init signature.

Extended spinner with fixed duration capability. It allows
to prepare fixed duration (ex. 10ms) workloads and take workloads/second
measurements, a handy utility for scheduling tests.

v2: - added asserts in div64_u64_round_up, duration_to_ctx_ticks,
      simplified loop_addr (Zbyszek)
    - added xe_spin_init_opts macro (Zbyszek)
    - corrected patch title (Kamil)

Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
---
 lib/xe/xe_spin.c            | 121 ++++++++++++++++++++++++++++++------
 lib/xe/xe_spin.h            |  27 +++++++-
 tests/xe/xe_dma_buf_sync.c  |   6 +-
 tests/xe/xe_exec_balancer.c |   9 ++-
 tests/xe/xe_exec_reset.c    |  24 ++++---
 tests/xe/xe_exec_threads.c  |   7 ++-
 tests/xe/xe_vm.c            |   9 +--
 7 files changed, 158 insertions(+), 45 deletions(-)

diff --git a/lib/xe/xe_spin.c b/lib/xe/xe_spin.c
index 7113972ee..dcc4aeea1 100644
--- a/lib/xe/xe_spin.c
+++ b/lib/xe/xe_spin.c
@@ -16,41 +16,130 @@
 #include "xe_ioctl.h"
 #include "xe_spin.h"
 
+static uint32_t read_timestamp_frequency(int fd, int gt_id)
+{
+	struct xe_device *dev = xe_device_get(fd);
+
+	igt_assert(dev && dev->gts && dev->gts->num_gt);
+	igt_assert(gt_id >= 0 && gt_id <= dev->gts->num_gt);
+
+	return dev->gts->gts[gt_id].clock_freq;
+}
+
+static uint64_t div64_u64_round_up(const uint64_t x, const uint64_t y)
+{
+	igt_assert(y > 0);
+
+	return (x + y - 1) / y;
+}
+
+/**
+ * duration_to_ctx_ticks:
+ * @fd: opened device
+ * @gt_id: tile id
+ * @duration_ns: duration in nanoseconds to be converted to context timestamp ticks
+ * @return: duration converted to context timestamp ticks.
+ */
+uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t duration_ns)
+{
+	uint32_t f = read_timestamp_frequency(fd, gt_id);
+	uint64_t ctx_ticks = div64_u64_round_up(duration_ns * f, NSEC_PER_SEC);
+
+	igt_assert_lt_u64(ctx_ticks, XE_SPIN_MAX_CTX_TICKS);
+
+	return ctx_ticks;
+}
+
+#define MI_SRM_CS_MMIO				(1 << 19)
+#define MI_LRI_CS_MMIO				(1 << 19)
+#define MI_LRR_DST_CS_MMIO			(1 << 19)
+#define MI_LRR_SRC_CS_MMIO			(1 << 18)
+#define CTX_TIMESTAMP 0x3a8;
+#define CS_GPR(x) (0x600 + 8 * (x))
+enum { START_TS, NOW_TS };
+
 /**
  * xe_spin_init:
  * @spin: pointer to mapped bo in which spinner code will be written
- * @addr: offset of spinner within vm
- * @preempt: allow spinner to be preempted or not
+ * @opts: pointer to spinner initialization options
  */
-void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt)
+void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts)
 {
-	uint64_t batch_offset = (char *)&spin->batch - (char *)spin;
-	uint64_t batch_addr = addr + batch_offset;
-	uint64_t start_offset = (char *)&spin->start - (char *)spin;
-	uint64_t start_addr = addr + start_offset;
-	uint64_t end_offset = (char *)&spin->end - (char *)spin;
-	uint64_t end_addr = addr + end_offset;
+	uint64_t loop_addr;
+	uint64_t start_addr = opts->addr + offsetof(struct xe_spin, start);
+	uint64_t end_addr = opts->addr + offsetof(struct xe_spin, end);
+	uint64_t ticks_delta_addr = opts->addr + offsetof(struct xe_spin, ticks_delta);
+	uint64_t pad_addr = opts->addr + offsetof(struct xe_spin, pad);
 	int b = 0;
 
 	spin->start = 0;
 	spin->end = 0xffffffff;
+	spin->ticks_delta = 0;
+
+	if (opts->ctx_ticks) {
+		/* store start timestamp */
+		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
+		spin->batch[b++] = CS_GPR(START_TS) + 4;
+		spin->batch[b++] = 0;
+		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
+		spin->batch[b++] = CTX_TIMESTAMP;
+		spin->batch[b++] = CS_GPR(START_TS);
+	}
+
+	loop_addr = opts->addr + b * sizeof(uint32_t);
 
 	spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
 	spin->batch[b++] = start_addr;
 	spin->batch[b++] = start_addr >> 32;
 	spin->batch[b++] = 0xc0ffee;
 
-	if (preempt)
+	if (opts->preempt)
 		spin->batch[b++] = (0x5 << 23);
 
+	if (opts->ctx_ticks) {
+		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
+		spin->batch[b++] = CS_GPR(NOW_TS) + 4;
+		spin->batch[b++] = 0;
+		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
+		spin->batch[b++] = CTX_TIMESTAMP;
+		spin->batch[b++] = CS_GPR(NOW_TS);
+
+		/* delta = now - start; inverted to match COND_BBE */
+		spin->batch[b++] = MI_MATH(4);
+		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCA, MI_MATH_REG(NOW_TS));
+		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCB, MI_MATH_REG(START_TS));
+		spin->batch[b++] = MI_MATH_SUB;
+		spin->batch[b++] = MI_MATH_STOREINV(MI_MATH_REG(NOW_TS), MI_MATH_REG_ACCU);
+
+		/* Save delta for reading by COND_BBE */
+		spin->batch[b++] = MI_STORE_REGISTER_MEM | MI_SRM_CS_MMIO | 2;
+		spin->batch[b++] = CS_GPR(NOW_TS);
+		spin->batch[b++] = ticks_delta_addr;
+		spin->batch[b++] = ticks_delta_addr >> 32;
+
+		/* Delay between SRM and COND_BBE to post the writes */
+		for (int n = 0; n < 8; n++) {
+			spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
+			spin->batch[b++] = pad_addr;
+			spin->batch[b++] = pad_addr >> 32;
+			spin->batch[b++] = 0xc0ffee;
+		}
+
+		/* Break if delta [time elapsed] > ns */
+		spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
+		spin->batch[b++] = ~(opts->ctx_ticks);
+		spin->batch[b++] = ticks_delta_addr;
+		spin->batch[b++] = ticks_delta_addr >> 32;
+	}
+
 	spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
 	spin->batch[b++] = 0;
 	spin->batch[b++] = end_addr;
 	spin->batch[b++] = end_addr >> 32;
 
 	spin->batch[b++] = MI_BATCH_BUFFER_START | 1 << 8 | 1;
-	spin->batch[b++] = batch_addr;
-	spin->batch[b++] = batch_addr >> 32;
+	spin->batch[b++] = loop_addr;
+	spin->batch[b++] = loop_addr >> 32;
 
 	igt_assert(b <= ARRAY_SIZE(spin->batch));
 }
@@ -133,11 +222,7 @@ xe_spin_create(int fd, const struct igt_spin_factory *opt)
 	addr = intel_allocator_alloc_with_strategy(ahnd, spin->handle, bo_size, 0, ALLOC_STRATEGY_LOW_TO_HIGH);
 	xe_vm_bind_sync(fd, spin->vm, spin->handle, 0, addr, bo_size);
 
-	if (!(opt->flags & IGT_SPIN_NO_PREEMPTION))
-		xe_spin_init(xe_spin, addr, true);
-	else
-		xe_spin_init(xe_spin, addr, false);
-
+	xe_spin_init_opts(xe_spin, .addr = addr, .preempt = !(opt->flags & IGT_SPIN_NO_PREEMPTION));
 	exec.exec_queue_id = spin->engine;
 	exec.address = addr;
 	sync.handle = spin->syncobj;
@@ -219,7 +304,7 @@ void xe_cork_init(int fd, struct drm_xe_engine_class_instance *hwe,
 	exec_queue = xe_exec_queue_create(fd, vm, hwe, 0);
 	syncobj = syncobj_create(fd, 0);
 
-	xe_spin_init(spin, addr, true);
+	xe_spin_init_opts(spin, .addr = addr, .preempt = true);
 	exec.exec_queue_id = exec_queue;
 	exec.address = addr;
 	sync.handle = syncobj;
diff --git a/lib/xe/xe_spin.h b/lib/xe/xe_spin.h
index c84db175d..f1abc1102 100644
--- a/lib/xe/xe_spin.h
+++ b/lib/xe/xe_spin.h
@@ -15,15 +15,38 @@
 #include "xe_query.h"
 #include "lib/igt_dummyload.h"
 
+#define XE_SPIN_MAX_CTX_TICKS UINT32_MAX - 1000
+
+/** struct xe_spin_opts
+ *
+ * @addr: offset of spinner within vm
+ * @preempt: allow spinner to be preempted or not
+ * @ctx_ticks: number of ticks after which spinner is stopped, applied if > 0
+ *
+ * Used to initialize struct xe_spin spinner behavior.
+ */
+struct xe_spin_opts {
+	uint64_t addr;
+	bool preempt;
+	uint32_t ctx_ticks;
+};
+
 /* Mapped GPU object */
 struct xe_spin {
-	uint32_t batch[16];
+	uint32_t batch[128];
 	uint64_t pad;
 	uint32_t start;
 	uint32_t end;
+	uint32_t ticks_delta;
 };
+
 igt_spin_t *xe_spin_create(int fd, const struct igt_spin_factory *opt);
-void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt);
+uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t ns);
+void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts);
+
+#define xe_spin_init_opts(fd, ...) \
+	xe_spin_init(fd, &((struct xe_spin_opts){__VA_ARGS__}))
+
 bool xe_spin_started(struct xe_spin *spin);
 void xe_spin_sync_wait(int fd, struct igt_spin *spin);
 void xe_spin_wait_started(struct xe_spin *spin);
diff --git a/tests/xe/xe_dma_buf_sync.c b/tests/xe/xe_dma_buf_sync.c
index 29d675154..627f4c1e5 100644
--- a/tests/xe/xe_dma_buf_sync.c
+++ b/tests/xe/xe_dma_buf_sync.c
@@ -144,7 +144,6 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
 		uint64_t sdi_offset = (char *)&data[i]->data - (char *)data[i];
 		uint64_t sdi_addr = addr + sdi_offset;
 		uint64_t spin_offset = (char *)&data[i]->spin - (char *)data[i];
-		uint64_t spin_addr = addr + spin_offset;
 		struct drm_xe_sync sync[2] = {
 			{ .flags = DRM_XE_SYNC_SYNCOBJ, },
 			{ .flags = DRM_XE_SYNC_SYNCOBJ | DRM_XE_SYNC_SIGNAL, },
@@ -153,14 +152,15 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
 			.num_batch_buffer = 1,
 			.syncs = to_user_pointer(sync),
 		};
+		struct xe_spin_opts spin_opts = { .addr = addr + spin_offset, .preempt = true };
 		uint32_t syncobj;
 		int b = 0;
 		int sync_fd;
 
 		/* Write spinner on FD[0] */
-		xe_spin_init(&data[i]->spin, spin_addr, true);
+		xe_spin_init(&data[i]->spin, &spin_opts);
 		exec.exec_queue_id = exec_queue[0];
-		exec.address = spin_addr;
+		exec.address = spin_opts.addr;
 		xe_exec(fd[0], &exec);
 
 		/* Export prime BO as sync file and veify business */
diff --git a/tests/xe/xe_exec_balancer.c b/tests/xe/xe_exec_balancer.c
index f364a4b7a..d7d8dd8fb 100644
--- a/tests/xe/xe_exec_balancer.c
+++ b/tests/xe/xe_exec_balancer.c
@@ -52,6 +52,7 @@ static void test_all_active(int fd, int gt, int class)
 	struct {
 		struct xe_spin spin;
 	} *data;
+	struct xe_spin_opts spin_opts = { .preempt = false };
 	struct drm_xe_engine_class_instance *hwe;
 	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
 	int i, num_placements = 0;
@@ -90,16 +91,14 @@ static void test_all_active(int fd, int gt, int class)
 	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
 
 	for (i = 0; i < num_placements; i++) {
-		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
-		uint64_t spin_addr = addr + spin_offset;
-
-		xe_spin_init(&data[i].spin, spin_addr, false);
+		spin_opts.addr = addr + (char *)&data[i].spin - (char *)data;
+		xe_spin_init(&data[i].spin, &spin_opts);
 		sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
 		sync[1].flags |= DRM_XE_SYNC_SIGNAL;
 		sync[1].handle = syncobjs[i];
 
 		exec.exec_queue_id = exec_queues[i];
-		exec.address = spin_addr;
+		exec.address = spin_opts.addr;
 		xe_exec(fd, &exec);
 		xe_spin_wait_started(&data[i].spin);
 	}
diff --git a/tests/xe/xe_exec_reset.c b/tests/xe/xe_exec_reset.c
index a2d33baf1..be6bbada6 100644
--- a/tests/xe/xe_exec_reset.c
+++ b/tests/xe/xe_exec_reset.c
@@ -44,6 +44,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
 	size_t bo_size;
 	uint32_t bo = 0;
 	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts = { .addr = addr, .preempt = false };
 
 	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
 	bo_size = sizeof(*spin);
@@ -60,7 +61,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
 	sync[0].handle = syncobj_create(fd, 0);
 	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
 
-	xe_spin_init(spin, addr, false);
+	xe_spin_init(spin, &spin_opts);
 
 	sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
 	sync[1].flags |= DRM_XE_SYNC_SIGNAL;
@@ -165,6 +166,7 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
 		uint64_t pad;
 		uint32_t data;
 	} *data;
+	struct xe_spin_opts spin_opts = { .preempt = false };
 	struct drm_xe_engine_class_instance *hwe;
 	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
 	int i, j, b, num_placements = 0, bad_batches = 1;
@@ -236,7 +238,6 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
 		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
 		uint64_t batch_addr = base_addr + batch_offset;
 		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
-		uint64_t spin_addr = base_addr + spin_offset;
 		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
 		uint64_t sdi_addr = base_addr + sdi_offset;
 		uint64_t exec_addr;
@@ -247,8 +248,9 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
 			batches[j] = batch_addr;
 
 		if (i < bad_batches) {
-			xe_spin_init(&data[i].spin, spin_addr, false);
-			exec_addr = spin_addr;
+			spin_opts.addr = base_addr + spin_offset;
+			xe_spin_init(&data[i].spin, &spin_opts);
+			exec_addr = spin_opts.addr;
 		} else {
 			b = 0;
 			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
@@ -368,6 +370,7 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
 		uint64_t pad;
 		uint32_t data;
 	} *data;
+	struct xe_spin_opts spin_opts = { .preempt = false };
 	int i, b;
 
 	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
@@ -417,15 +420,15 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
 		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
 		uint64_t batch_addr = base_addr + batch_offset;
 		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
-		uint64_t spin_addr = base_addr + spin_offset;
 		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
 		uint64_t sdi_addr = base_addr + sdi_offset;
 		uint64_t exec_addr;
 		int e = i % n_exec_queues;
 
 		if (!i) {
-			xe_spin_init(&data[i].spin, spin_addr, false);
-			exec_addr = spin_addr;
+			spin_opts.addr = base_addr + spin_offset;
+			xe_spin_init(&data[i].spin, &spin_opts);
+			exec_addr = spin_opts.addr;
 		} else {
 			b = 0;
 			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
@@ -539,6 +542,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
 		uint64_t exec_sync;
 		uint32_t data;
 	} *data;
+	struct xe_spin_opts spin_opts = { .preempt = false };
 	int i, b;
 
 	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
@@ -593,15 +597,15 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
 		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
 		uint64_t batch_addr = base_addr + batch_offset;
 		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
-		uint64_t spin_addr = base_addr + spin_offset;
 		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
 		uint64_t sdi_addr = base_addr + sdi_offset;
 		uint64_t exec_addr;
 		int e = i % n_exec_queues;
 
 		if (!i) {
-			xe_spin_init(&data[i].spin, spin_addr, false);
-			exec_addr = spin_addr;
+			spin_opts.addr = base_addr + spin_offset;
+			xe_spin_init(&data[i].spin, &spin_opts);
+			exec_addr = spin_opts.addr;
 		} else {
 			b = 0;
 			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
diff --git a/tests/xe/xe_exec_threads.c b/tests/xe/xe_exec_threads.c
index e64c1639a..ff4ebc280 100644
--- a/tests/xe/xe_exec_threads.c
+++ b/tests/xe/xe_exec_threads.c
@@ -486,6 +486,7 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
 		uint64_t pad;
 		uint32_t data;
 	} *data;
+	struct xe_spin_opts spin_opts = { .preempt = false };
 	int i, j, b, hang_exec_queue = n_exec_queues / 2;
 	bool owns_vm = false, owns_fd = false;
 
@@ -562,15 +563,15 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
 		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
 		uint64_t batch_addr = addr + batch_offset;
 		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
-		uint64_t spin_addr = addr + spin_offset;
 		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
 		uint64_t sdi_addr = addr + sdi_offset;
 		uint64_t exec_addr;
 		int e = i % n_exec_queues;
 
 		if (flags & HANG && e == hang_exec_queue && i == e) {
-			xe_spin_init(&data[i].spin, spin_addr, false);
-			exec_addr = spin_addr;
+			spin_opts.addr = addr + spin_offset;
+			xe_spin_init(&data[i].spin, &spin_opts);
+			exec_addr = spin_opts.addr;
 		} else {
 			b = 0;
 			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
diff --git a/tests/xe/xe_vm.c b/tests/xe/xe_vm.c
index e42c04e33..87604a407 100644
--- a/tests/xe/xe_vm.c
+++ b/tests/xe/xe_vm.c
@@ -727,6 +727,7 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
 		uint64_t pad;
 		uint32_t data;
 	} *data;
+	struct xe_spin_opts spin_opts = { .preempt = true };
 	int i, b;
 
 	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
@@ -755,14 +756,14 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
 		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
 		uint64_t sdi_addr = addr + sdi_offset;
 		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
-		uint64_t spin_addr = addr + spin_offset;
 		int e = i;
 
 		if (i == 0) {
-			/* Cork 1st exec_queue with a spinner */
-			xe_spin_init(&data[i].spin, spin_addr, true);
+			/* Cork 1st engine with a spinner */
+			spin_opts.addr = addr + spin_offset;
+			xe_spin_init(&data[i].spin, &spin_opts);
 			exec.exec_queue_id = exec_queues[e];
-			exec.address = spin_addr;
+			exec.address = spin_opts.addr;
 			sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
 			sync[1].flags |= DRM_XE_SYNC_SIGNAL;
 			sync[1].handle = syncobjs[e];
-- 
2.30.2

Re: [igt-dev] [PATCH i-g-t 1/3] lib/xe_spin: fixed duration xe_spin capability

[Zbigniew Kempczyński]

On Fri, Aug 25, 2023 at 01:19:11PM +0000, Marcin Bernatowicz wrote:
> Introduced struct xe_spin_opts for xe_spin initialization,
> adjusted tests to new xe_spin_init signature.
> 
> Extended spinner with fixed duration capability. It allows
> to prepare fixed duration (ex. 10ms) workloads and take workloads/second
> measurements, a handy utility for scheduling tests.
> 
> v2: - added asserts in div64_u64_round_up, duration_to_ctx_ticks,
>       simplified loop_addr (Zbyszek)
>     - added xe_spin_init_opts macro (Zbyszek)
>     - corrected patch title (Kamil)
> 
> Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>

I've seen this code before, and after refactoring looks ok for me:

Acked-by: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>

--
Zbigniew

> ---
>  lib/xe/xe_spin.c            | 121 ++++++++++++++++++++++++++++++------
>  lib/xe/xe_spin.h            |  27 +++++++-
>  tests/xe/xe_dma_buf_sync.c  |   6 +-
>  tests/xe/xe_exec_balancer.c |   9 ++-
>  tests/xe/xe_exec_reset.c    |  24 ++++---
>  tests/xe/xe_exec_threads.c  |   7 ++-
>  tests/xe/xe_vm.c            |   9 +--
>  7 files changed, 158 insertions(+), 45 deletions(-)
> 
> diff --git a/lib/xe/xe_spin.c b/lib/xe/xe_spin.c
> index 7113972ee..dcc4aeea1 100644
> --- a/lib/xe/xe_spin.c
> +++ b/lib/xe/xe_spin.c
> @@ -16,41 +16,130 @@
>  #include "xe_ioctl.h"
>  #include "xe_spin.h"
>  
> +static uint32_t read_timestamp_frequency(int fd, int gt_id)
> +{
> +	struct xe_device *dev = xe_device_get(fd);
> +
> +	igt_assert(dev && dev->gts && dev->gts->num_gt);
> +	igt_assert(gt_id >= 0 && gt_id <= dev->gts->num_gt);
> +
> +	return dev->gts->gts[gt_id].clock_freq;
> +}
> +
> +static uint64_t div64_u64_round_up(const uint64_t x, const uint64_t y)
> +{
> +	igt_assert(y > 0);
> +
> +	return (x + y - 1) / y;
> +}
> +
> +/**
> + * duration_to_ctx_ticks:
> + * @fd: opened device
> + * @gt_id: tile id
> + * @duration_ns: duration in nanoseconds to be converted to context timestamp ticks
> + * @return: duration converted to context timestamp ticks.
> + */
> +uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t duration_ns)
> +{
> +	uint32_t f = read_timestamp_frequency(fd, gt_id);
> +	uint64_t ctx_ticks = div64_u64_round_up(duration_ns * f, NSEC_PER_SEC);
> +
> +	igt_assert_lt_u64(ctx_ticks, XE_SPIN_MAX_CTX_TICKS);
> +
> +	return ctx_ticks;
> +}
> +
> +#define MI_SRM_CS_MMIO				(1 << 19)
> +#define MI_LRI_CS_MMIO				(1 << 19)
> +#define MI_LRR_DST_CS_MMIO			(1 << 19)
> +#define MI_LRR_SRC_CS_MMIO			(1 << 18)
> +#define CTX_TIMESTAMP 0x3a8;
> +#define CS_GPR(x) (0x600 + 8 * (x))
> +enum { START_TS, NOW_TS };
> +
>  /**
>   * xe_spin_init:
>   * @spin: pointer to mapped bo in which spinner code will be written
> - * @addr: offset of spinner within vm
> - * @preempt: allow spinner to be preempted or not
> + * @opts: pointer to spinner initialization options
>   */
> -void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt)
> +void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts)
>  {
> -	uint64_t batch_offset = (char *)&spin->batch - (char *)spin;
> -	uint64_t batch_addr = addr + batch_offset;
> -	uint64_t start_offset = (char *)&spin->start - (char *)spin;
> -	uint64_t start_addr = addr + start_offset;
> -	uint64_t end_offset = (char *)&spin->end - (char *)spin;
> -	uint64_t end_addr = addr + end_offset;
> +	uint64_t loop_addr;
> +	uint64_t start_addr = opts->addr + offsetof(struct xe_spin, start);
> +	uint64_t end_addr = opts->addr + offsetof(struct xe_spin, end);
> +	uint64_t ticks_delta_addr = opts->addr + offsetof(struct xe_spin, ticks_delta);
> +	uint64_t pad_addr = opts->addr + offsetof(struct xe_spin, pad);
>  	int b = 0;
>  
>  	spin->start = 0;
>  	spin->end = 0xffffffff;
> +	spin->ticks_delta = 0;
> +
> +	if (opts->ctx_ticks) {
> +		/* store start timestamp */
> +		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
> +		spin->batch[b++] = CS_GPR(START_TS) + 4;
> +		spin->batch[b++] = 0;
> +		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
> +		spin->batch[b++] = CTX_TIMESTAMP;
> +		spin->batch[b++] = CS_GPR(START_TS);
> +	}
> +
> +	loop_addr = opts->addr + b * sizeof(uint32_t);
>  
>  	spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>  	spin->batch[b++] = start_addr;
>  	spin->batch[b++] = start_addr >> 32;
>  	spin->batch[b++] = 0xc0ffee;
>  
> -	if (preempt)
> +	if (opts->preempt)
>  		spin->batch[b++] = (0x5 << 23);
>  
> +	if (opts->ctx_ticks) {
> +		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
> +		spin->batch[b++] = CS_GPR(NOW_TS) + 4;
> +		spin->batch[b++] = 0;
> +		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
> +		spin->batch[b++] = CTX_TIMESTAMP;
> +		spin->batch[b++] = CS_GPR(NOW_TS);
> +
> +		/* delta = now - start; inverted to match COND_BBE */
> +		spin->batch[b++] = MI_MATH(4);
> +		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCA, MI_MATH_REG(NOW_TS));
> +		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCB, MI_MATH_REG(START_TS));
> +		spin->batch[b++] = MI_MATH_SUB;
> +		spin->batch[b++] = MI_MATH_STOREINV(MI_MATH_REG(NOW_TS), MI_MATH_REG_ACCU);
> +
> +		/* Save delta for reading by COND_BBE */
> +		spin->batch[b++] = MI_STORE_REGISTER_MEM | MI_SRM_CS_MMIO | 2;
> +		spin->batch[b++] = CS_GPR(NOW_TS);
> +		spin->batch[b++] = ticks_delta_addr;
> +		spin->batch[b++] = ticks_delta_addr >> 32;
> +
> +		/* Delay between SRM and COND_BBE to post the writes */
> +		for (int n = 0; n < 8; n++) {
> +			spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> +			spin->batch[b++] = pad_addr;
> +			spin->batch[b++] = pad_addr >> 32;
> +			spin->batch[b++] = 0xc0ffee;
> +		}
> +
> +		/* Break if delta [time elapsed] > ns */
> +		spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
> +		spin->batch[b++] = ~(opts->ctx_ticks);
> +		spin->batch[b++] = ticks_delta_addr;
> +		spin->batch[b++] = ticks_delta_addr >> 32;
> +	}
> +
>  	spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
>  	spin->batch[b++] = 0;
>  	spin->batch[b++] = end_addr;
>  	spin->batch[b++] = end_addr >> 32;
>  
>  	spin->batch[b++] = MI_BATCH_BUFFER_START | 1 << 8 | 1;
> -	spin->batch[b++] = batch_addr;
> -	spin->batch[b++] = batch_addr >> 32;
> +	spin->batch[b++] = loop_addr;
> +	spin->batch[b++] = loop_addr >> 32;
>  
>  	igt_assert(b <= ARRAY_SIZE(spin->batch));
>  }
> @@ -133,11 +222,7 @@ xe_spin_create(int fd, const struct igt_spin_factory *opt)
>  	addr = intel_allocator_alloc_with_strategy(ahnd, spin->handle, bo_size, 0, ALLOC_STRATEGY_LOW_TO_HIGH);
>  	xe_vm_bind_sync(fd, spin->vm, spin->handle, 0, addr, bo_size);
>  
> -	if (!(opt->flags & IGT_SPIN_NO_PREEMPTION))
> -		xe_spin_init(xe_spin, addr, true);
> -	else
> -		xe_spin_init(xe_spin, addr, false);
> -
> +	xe_spin_init_opts(xe_spin, .addr = addr, .preempt = !(opt->flags & IGT_SPIN_NO_PREEMPTION));
>  	exec.exec_queue_id = spin->engine;
>  	exec.address = addr;
>  	sync.handle = spin->syncobj;
> @@ -219,7 +304,7 @@ void xe_cork_init(int fd, struct drm_xe_engine_class_instance *hwe,
>  	exec_queue = xe_exec_queue_create(fd, vm, hwe, 0);
>  	syncobj = syncobj_create(fd, 0);
>  
> -	xe_spin_init(spin, addr, true);
> +	xe_spin_init_opts(spin, .addr = addr, .preempt = true);
>  	exec.exec_queue_id = exec_queue;
>  	exec.address = addr;
>  	sync.handle = syncobj;
> diff --git a/lib/xe/xe_spin.h b/lib/xe/xe_spin.h
> index c84db175d..f1abc1102 100644
> --- a/lib/xe/xe_spin.h
> +++ b/lib/xe/xe_spin.h
> @@ -15,15 +15,38 @@
>  #include "xe_query.h"
>  #include "lib/igt_dummyload.h"
>  
> +#define XE_SPIN_MAX_CTX_TICKS UINT32_MAX - 1000
> +
> +/** struct xe_spin_opts
> + *
> + * @addr: offset of spinner within vm
> + * @preempt: allow spinner to be preempted or not
> + * @ctx_ticks: number of ticks after which spinner is stopped, applied if > 0
> + *
> + * Used to initialize struct xe_spin spinner behavior.
> + */
> +struct xe_spin_opts {
> +	uint64_t addr;
> +	bool preempt;
> +	uint32_t ctx_ticks;
> +};
> +
>  /* Mapped GPU object */
>  struct xe_spin {
> -	uint32_t batch[16];
> +	uint32_t batch[128];
>  	uint64_t pad;
>  	uint32_t start;
>  	uint32_t end;
> +	uint32_t ticks_delta;
>  };
> +
>  igt_spin_t *xe_spin_create(int fd, const struct igt_spin_factory *opt);
> -void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt);
> +uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t ns);
> +void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts);
> +
> +#define xe_spin_init_opts(fd, ...) \
> +	xe_spin_init(fd, &((struct xe_spin_opts){__VA_ARGS__}))
> +
>  bool xe_spin_started(struct xe_spin *spin);
>  void xe_spin_sync_wait(int fd, struct igt_spin *spin);
>  void xe_spin_wait_started(struct xe_spin *spin);
> diff --git a/tests/xe/xe_dma_buf_sync.c b/tests/xe/xe_dma_buf_sync.c
> index 29d675154..627f4c1e5 100644
> --- a/tests/xe/xe_dma_buf_sync.c
> +++ b/tests/xe/xe_dma_buf_sync.c
> @@ -144,7 +144,6 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
>  		uint64_t sdi_offset = (char *)&data[i]->data - (char *)data[i];
>  		uint64_t sdi_addr = addr + sdi_offset;
>  		uint64_t spin_offset = (char *)&data[i]->spin - (char *)data[i];
> -		uint64_t spin_addr = addr + spin_offset;
>  		struct drm_xe_sync sync[2] = {
>  			{ .flags = DRM_XE_SYNC_SYNCOBJ, },
>  			{ .flags = DRM_XE_SYNC_SYNCOBJ | DRM_XE_SYNC_SIGNAL, },
> @@ -153,14 +152,15 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
>  			.num_batch_buffer = 1,
>  			.syncs = to_user_pointer(sync),
>  		};
> +		struct xe_spin_opts spin_opts = { .addr = addr + spin_offset, .preempt = true };
>  		uint32_t syncobj;
>  		int b = 0;
>  		int sync_fd;
>  
>  		/* Write spinner on FD[0] */
> -		xe_spin_init(&data[i]->spin, spin_addr, true);
> +		xe_spin_init(&data[i]->spin, &spin_opts);
>  		exec.exec_queue_id = exec_queue[0];
> -		exec.address = spin_addr;
> +		exec.address = spin_opts.addr;
>  		xe_exec(fd[0], &exec);
>  
>  		/* Export prime BO as sync file and veify business */
> diff --git a/tests/xe/xe_exec_balancer.c b/tests/xe/xe_exec_balancer.c
> index f364a4b7a..d7d8dd8fb 100644
> --- a/tests/xe/xe_exec_balancer.c
> +++ b/tests/xe/xe_exec_balancer.c
> @@ -52,6 +52,7 @@ static void test_all_active(int fd, int gt, int class)
>  	struct {
>  		struct xe_spin spin;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	struct drm_xe_engine_class_instance *hwe;
>  	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>  	int i, num_placements = 0;
> @@ -90,16 +91,14 @@ static void test_all_active(int fd, int gt, int class)
>  	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>  
>  	for (i = 0; i < num_placements; i++) {
> -		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = addr + spin_offset;
> -
> -		xe_spin_init(&data[i].spin, spin_addr, false);
> +		spin_opts.addr = addr + (char *)&data[i].spin - (char *)data;
> +		xe_spin_init(&data[i].spin, &spin_opts);
>  		sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>  		sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>  		sync[1].handle = syncobjs[i];
>  
>  		exec.exec_queue_id = exec_queues[i];
> -		exec.address = spin_addr;
> +		exec.address = spin_opts.addr;
>  		xe_exec(fd, &exec);
>  		xe_spin_wait_started(&data[i].spin);
>  	}
> diff --git a/tests/xe/xe_exec_reset.c b/tests/xe/xe_exec_reset.c
> index a2d33baf1..be6bbada6 100644
> --- a/tests/xe/xe_exec_reset.c
> +++ b/tests/xe/xe_exec_reset.c
> @@ -44,6 +44,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
>  	size_t bo_size;
>  	uint32_t bo = 0;
>  	struct xe_spin *spin;
> +	struct xe_spin_opts spin_opts = { .addr = addr, .preempt = false };
>  
>  	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
>  	bo_size = sizeof(*spin);
> @@ -60,7 +61,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
>  	sync[0].handle = syncobj_create(fd, 0);
>  	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>  
> -	xe_spin_init(spin, addr, false);
> +	xe_spin_init(spin, &spin_opts);
>  
>  	sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>  	sync[1].flags |= DRM_XE_SYNC_SIGNAL;
> @@ -165,6 +166,7 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	struct drm_xe_engine_class_instance *hwe;
>  	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>  	int i, j, b, num_placements = 0, bad_batches = 1;
> @@ -236,7 +238,6 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = base_addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = base_addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = base_addr + sdi_offset;
>  		uint64_t exec_addr;
> @@ -247,8 +248,9 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>  			batches[j] = batch_addr;
>  
>  		if (i < bad_batches) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = base_addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> @@ -368,6 +370,7 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	int i, b;
>  
>  	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
> @@ -417,15 +420,15 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = base_addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = base_addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = base_addr + sdi_offset;
>  		uint64_t exec_addr;
>  		int e = i % n_exec_queues;
>  
>  		if (!i) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = base_addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> @@ -539,6 +542,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t exec_sync;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	int i, b;
>  
>  	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
> @@ -593,15 +597,15 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = base_addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = base_addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = base_addr + sdi_offset;
>  		uint64_t exec_addr;
>  		int e = i % n_exec_queues;
>  
>  		if (!i) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = base_addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> diff --git a/tests/xe/xe_exec_threads.c b/tests/xe/xe_exec_threads.c
> index e64c1639a..ff4ebc280 100644
> --- a/tests/xe/xe_exec_threads.c
> +++ b/tests/xe/xe_exec_threads.c
> @@ -486,6 +486,7 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	int i, j, b, hang_exec_queue = n_exec_queues / 2;
>  	bool owns_vm = false, owns_fd = false;
>  
> @@ -562,15 +563,15 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = addr + sdi_offset;
>  		uint64_t exec_addr;
>  		int e = i % n_exec_queues;
>  
>  		if (flags & HANG && e == hang_exec_queue && i == e) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> diff --git a/tests/xe/xe_vm.c b/tests/xe/xe_vm.c
> index e42c04e33..87604a407 100644
> --- a/tests/xe/xe_vm.c
> +++ b/tests/xe/xe_vm.c
> @@ -727,6 +727,7 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = true };
>  	int i, b;
>  
>  	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
> @@ -755,14 +756,14 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = addr + sdi_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = addr + spin_offset;
>  		int e = i;
>  
>  		if (i == 0) {
> -			/* Cork 1st exec_queue with a spinner */
> -			xe_spin_init(&data[i].spin, spin_addr, true);
> +			/* Cork 1st engine with a spinner */
> +			spin_opts.addr = addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
>  			exec.exec_queue_id = exec_queues[e];
> -			exec.address = spin_addr;
> +			exec.address = spin_opts.addr;
>  			sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>  			sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>  			sync[1].handle = syncobjs[e];
> -- 
> 2.30.2
> 

Re: [igt-dev] [PATCH i-g-t 1/3] lib/xe_spin: fixed duration xe_spin capability

[Kamil Konieczny]

Hi Marcin,

On 2023-08-25 at 13:19:11 +0000, Marcin Bernatowicz wrote:
> Introduced struct xe_spin_opts for xe_spin initialization,
> adjusted tests to new xe_spin_init signature.
> 
> Extended spinner with fixed duration capability. It allows
> to prepare fixed duration (ex. 10ms) workloads and take workloads/second
> measurements, a handy utility for scheduling tests.
> 
> v2: - added asserts in div64_u64_round_up, duration_to_ctx_ticks,
>       simplified loop_addr (Zbyszek)
>     - added xe_spin_init_opts macro (Zbyszek)
>     - corrected patch title (Kamil)
> 
> Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
> ---
>  lib/xe/xe_spin.c            | 121 ++++++++++++++++++++++++++++++------
>  lib/xe/xe_spin.h            |  27 +++++++-
>  tests/xe/xe_dma_buf_sync.c  |   6 +-
>  tests/xe/xe_exec_balancer.c |   9 ++-
>  tests/xe/xe_exec_reset.c    |  24 ++++---
>  tests/xe/xe_exec_threads.c  |   7 ++-
>  tests/xe/xe_vm.c            |   9 +--
>  7 files changed, 158 insertions(+), 45 deletions(-)
> 
> diff --git a/lib/xe/xe_spin.c b/lib/xe/xe_spin.c
> index 7113972ee..dcc4aeea1 100644
> --- a/lib/xe/xe_spin.c
> +++ b/lib/xe/xe_spin.c
> @@ -16,41 +16,130 @@
>  #include "xe_ioctl.h"
>  #include "xe_spin.h"
>  
> +static uint32_t read_timestamp_frequency(int fd, int gt_id)
> +{
> +	struct xe_device *dev = xe_device_get(fd);
> +
> +	igt_assert(dev && dev->gts && dev->gts->num_gt);
> +	igt_assert(gt_id >= 0 && gt_id <= dev->gts->num_gt);
> +
> +	return dev->gts->gts[gt_id].clock_freq;
> +}
> +
> +static uint64_t div64_u64_round_up(const uint64_t x, const uint64_t y)
> +{
> +	igt_assert(y > 0);
> +
> +	return (x + y - 1) / y;
----------- ^^^^^^^^^
This may overflow to value < y.

> +}
> +
> +/**
> + * duration_to_ctx_ticks:
> + * @fd: opened device
> + * @gt_id: tile id
> + * @duration_ns: duration in nanoseconds to be converted to context timestamp ticks
> + * @return: duration converted to context timestamp ticks.
> + */
> +uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t duration_ns)
> +{
> +	uint32_t f = read_timestamp_frequency(fd, gt_id);
> +	uint64_t ctx_ticks = div64_u64_round_up(duration_ns * f, NSEC_PER_SEC);
> +
> +	igt_assert_lt_u64(ctx_ticks, XE_SPIN_MAX_CTX_TICKS);
> +
> +	return ctx_ticks;
> +}
> +
> +#define MI_SRM_CS_MMIO				(1 << 19)
> +#define MI_LRI_CS_MMIO				(1 << 19)
> +#define MI_LRR_DST_CS_MMIO			(1 << 19)
> +#define MI_LRR_SRC_CS_MMIO			(1 << 18)
> +#define CTX_TIMESTAMP 0x3a8;
> +#define CS_GPR(x) (0x600 + 8 * (x))

Put newline here.

> +enum { START_TS, NOW_TS };
> +
>  /**
>   * xe_spin_init:
>   * @spin: pointer to mapped bo in which spinner code will be written
> - * @addr: offset of spinner within vm
> - * @preempt: allow spinner to be preempted or not
> + * @opts: pointer to spinner initialization options
>   */
> -void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt)
> +void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts)
>  {
> -	uint64_t batch_offset = (char *)&spin->batch - (char *)spin;
> -	uint64_t batch_addr = addr + batch_offset;
> -	uint64_t start_offset = (char *)&spin->start - (char *)spin;
> -	uint64_t start_addr = addr + start_offset;
> -	uint64_t end_offset = (char *)&spin->end - (char *)spin;
> -	uint64_t end_addr = addr + end_offset;
> +	uint64_t loop_addr;
> +	uint64_t start_addr = opts->addr + offsetof(struct xe_spin, start);
> +	uint64_t end_addr = opts->addr + offsetof(struct xe_spin, end);
> +	uint64_t ticks_delta_addr = opts->addr + offsetof(struct xe_spin, ticks_delta);
> +	uint64_t pad_addr = opts->addr + offsetof(struct xe_spin, pad);
>  	int b = 0;
>  
>  	spin->start = 0;
>  	spin->end = 0xffffffff;
> +	spin->ticks_delta = 0;
> +
> +	if (opts->ctx_ticks) {
> +		/* store start timestamp */
> +		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
> +		spin->batch[b++] = CS_GPR(START_TS) + 4;
> +		spin->batch[b++] = 0;
> +		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
> +		spin->batch[b++] = CTX_TIMESTAMP;
> +		spin->batch[b++] = CS_GPR(START_TS);
> +	}
> +
> +	loop_addr = opts->addr + b * sizeof(uint32_t);
>  
>  	spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>  	spin->batch[b++] = start_addr;
>  	spin->batch[b++] = start_addr >> 32;
>  	spin->batch[b++] = 0xc0ffee;
>  
> -	if (preempt)
> +	if (opts->preempt)
>  		spin->batch[b++] = (0x5 << 23);
>  
> +	if (opts->ctx_ticks) {
> +		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
> +		spin->batch[b++] = CS_GPR(NOW_TS) + 4;
> +		spin->batch[b++] = 0;
> +		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
> +		spin->batch[b++] = CTX_TIMESTAMP;
> +		spin->batch[b++] = CS_GPR(NOW_TS);
> +
> +		/* delta = now - start; inverted to match COND_BBE */
> +		spin->batch[b++] = MI_MATH(4);
> +		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCA, MI_MATH_REG(NOW_TS));
> +		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCB, MI_MATH_REG(START_TS));
> +		spin->batch[b++] = MI_MATH_SUB;
> +		spin->batch[b++] = MI_MATH_STOREINV(MI_MATH_REG(NOW_TS), MI_MATH_REG_ACCU);
> +
> +		/* Save delta for reading by COND_BBE */
> +		spin->batch[b++] = MI_STORE_REGISTER_MEM | MI_SRM_CS_MMIO | 2;
> +		spin->batch[b++] = CS_GPR(NOW_TS);
> +		spin->batch[b++] = ticks_delta_addr;
> +		spin->batch[b++] = ticks_delta_addr >> 32;
> +
> +		/* Delay between SRM and COND_BBE to post the writes */
> +		for (int n = 0; n < 8; n++) {
> +			spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> +			spin->batch[b++] = pad_addr;
> +			spin->batch[b++] = pad_addr >> 32;
> +			spin->batch[b++] = 0xc0ffee;
> +		}
> +
> +		/* Break if delta [time elapsed] > ns */
> +		spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
> +		spin->batch[b++] = ~(opts->ctx_ticks);
> +		spin->batch[b++] = ticks_delta_addr;
> +		spin->batch[b++] = ticks_delta_addr >> 32;
> +	}
> +
>  	spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
>  	spin->batch[b++] = 0;
>  	spin->batch[b++] = end_addr;
>  	spin->batch[b++] = end_addr >> 32;
>  
>  	spin->batch[b++] = MI_BATCH_BUFFER_START | 1 << 8 | 1;
> -	spin->batch[b++] = batch_addr;
> -	spin->batch[b++] = batch_addr >> 32;
> +	spin->batch[b++] = loop_addr;
> +	spin->batch[b++] = loop_addr >> 32;
>  
>  	igt_assert(b <= ARRAY_SIZE(spin->batch));
>  }
> @@ -133,11 +222,7 @@ xe_spin_create(int fd, const struct igt_spin_factory *opt)
>  	addr = intel_allocator_alloc_with_strategy(ahnd, spin->handle, bo_size, 0, ALLOC_STRATEGY_LOW_TO_HIGH);
>  	xe_vm_bind_sync(fd, spin->vm, spin->handle, 0, addr, bo_size);
>  
> -	if (!(opt->flags & IGT_SPIN_NO_PREEMPTION))
> -		xe_spin_init(xe_spin, addr, true);
> -	else
> -		xe_spin_init(xe_spin, addr, false);
> -
> +	xe_spin_init_opts(xe_spin, .addr = addr, .preempt = !(opt->flags & IGT_SPIN_NO_PREEMPTION));
>  	exec.exec_queue_id = spin->engine;
>  	exec.address = addr;
>  	sync.handle = spin->syncobj;
> @@ -219,7 +304,7 @@ void xe_cork_init(int fd, struct drm_xe_engine_class_instance *hwe,
>  	exec_queue = xe_exec_queue_create(fd, vm, hwe, 0);
>  	syncobj = syncobj_create(fd, 0);
>  
> -	xe_spin_init(spin, addr, true);
> +	xe_spin_init_opts(spin, .addr = addr, .preempt = true);
>  	exec.exec_queue_id = exec_queue;
>  	exec.address = addr;
>  	sync.handle = syncobj;
> diff --git a/lib/xe/xe_spin.h b/lib/xe/xe_spin.h
> index c84db175d..f1abc1102 100644
> --- a/lib/xe/xe_spin.h
> +++ b/lib/xe/xe_spin.h
> @@ -15,15 +15,38 @@
>  #include "xe_query.h"
>  #include "lib/igt_dummyload.h"
>  
> +#define XE_SPIN_MAX_CTX_TICKS UINT32_MAX - 1000
> +
> +/** struct xe_spin_opts
> + *
> + * @addr: offset of spinner within vm
> + * @preempt: allow spinner to be preempted or not
> + * @ctx_ticks: number of ticks after which spinner is stopped, applied if > 0
> + *
> + * Used to initialize struct xe_spin spinner behavior.
> + */
> +struct xe_spin_opts {
> +	uint64_t addr;
> +	bool preempt;
> +	uint32_t ctx_ticks;
> +};
> +
>  /* Mapped GPU object */
>  struct xe_spin {
> -	uint32_t batch[16];
> +	uint32_t batch[128];
>  	uint64_t pad;
>  	uint32_t start;
>  	uint32_t end;
> +	uint32_t ticks_delta;
>  };
> +
>  igt_spin_t *xe_spin_create(int fd, const struct igt_spin_factory *opt);
> -void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt);
> +uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t ns);
> +void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts);
> +
> +#define xe_spin_init_opts(fd, ...) \
> +	xe_spin_init(fd, &((struct xe_spin_opts){__VA_ARGS__}))
> +
>  bool xe_spin_started(struct xe_spin *spin);
>  void xe_spin_sync_wait(int fd, struct igt_spin *spin);
>  void xe_spin_wait_started(struct xe_spin *spin);
> diff --git a/tests/xe/xe_dma_buf_sync.c b/tests/xe/xe_dma_buf_sync.c
> index 29d675154..627f4c1e5 100644
> --- a/tests/xe/xe_dma_buf_sync.c
> +++ b/tests/xe/xe_dma_buf_sync.c
> @@ -144,7 +144,6 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
>  		uint64_t sdi_offset = (char *)&data[i]->data - (char *)data[i];
>  		uint64_t sdi_addr = addr + sdi_offset;
>  		uint64_t spin_offset = (char *)&data[i]->spin - (char *)data[i];
> -		uint64_t spin_addr = addr + spin_offset;
>  		struct drm_xe_sync sync[2] = {
>  			{ .flags = DRM_XE_SYNC_SYNCOBJ, },
>  			{ .flags = DRM_XE_SYNC_SYNCOBJ | DRM_XE_SYNC_SIGNAL, },
> @@ -153,14 +152,15 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
>  			.num_batch_buffer = 1,
>  			.syncs = to_user_pointer(sync),
>  		};
> +		struct xe_spin_opts spin_opts = { .addr = addr + spin_offset, .preempt = true };
>  		uint32_t syncobj;
>  		int b = 0;
>  		int sync_fd;
>  
>  		/* Write spinner on FD[0] */
> -		xe_spin_init(&data[i]->spin, spin_addr, true);
> +		xe_spin_init(&data[i]->spin, &spin_opts);
>  		exec.exec_queue_id = exec_queue[0];
> -		exec.address = spin_addr;
> +		exec.address = spin_opts.addr;
>  		xe_exec(fd[0], &exec);
>  
>  		/* Export prime BO as sync file and veify business */
> diff --git a/tests/xe/xe_exec_balancer.c b/tests/xe/xe_exec_balancer.c
> index f364a4b7a..d7d8dd8fb 100644
> --- a/tests/xe/xe_exec_balancer.c
> +++ b/tests/xe/xe_exec_balancer.c
> @@ -52,6 +52,7 @@ static void test_all_active(int fd, int gt, int class)
>  	struct {
>  		struct xe_spin spin;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	struct drm_xe_engine_class_instance *hwe;
>  	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>  	int i, num_placements = 0;
> @@ -90,16 +91,14 @@ static void test_all_active(int fd, int gt, int class)
>  	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>  
>  	for (i = 0; i < num_placements; i++) {
> -		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = addr + spin_offset;
> -
> -		xe_spin_init(&data[i].spin, spin_addr, false);
> +		spin_opts.addr = addr + (char *)&data[i].spin - (char *)data;
> +		xe_spin_init(&data[i].spin, &spin_opts);
>  		sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>  		sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>  		sync[1].handle = syncobjs[i];
>  
>  		exec.exec_queue_id = exec_queues[i];
> -		exec.address = spin_addr;
> +		exec.address = spin_opts.addr;
>  		xe_exec(fd, &exec);
>  		xe_spin_wait_started(&data[i].spin);
>  	}
> diff --git a/tests/xe/xe_exec_reset.c b/tests/xe/xe_exec_reset.c
> index a2d33baf1..be6bbada6 100644
> --- a/tests/xe/xe_exec_reset.c
> +++ b/tests/xe/xe_exec_reset.c
> @@ -44,6 +44,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
>  	size_t bo_size;
>  	uint32_t bo = 0;
>  	struct xe_spin *spin;
> +	struct xe_spin_opts spin_opts = { .addr = addr, .preempt = false };
>  
>  	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
>  	bo_size = sizeof(*spin);
> @@ -60,7 +61,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
>  	sync[0].handle = syncobj_create(fd, 0);
>  	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>  
> -	xe_spin_init(spin, addr, false);
> +	xe_spin_init(spin, &spin_opts);
>  
>  	sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>  	sync[1].flags |= DRM_XE_SYNC_SIGNAL;
> @@ -165,6 +166,7 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	struct drm_xe_engine_class_instance *hwe;
>  	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>  	int i, j, b, num_placements = 0, bad_batches = 1;
> @@ -236,7 +238,6 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = base_addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = base_addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = base_addr + sdi_offset;
>  		uint64_t exec_addr;
> @@ -247,8 +248,9 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>  			batches[j] = batch_addr;
>  
>  		if (i < bad_batches) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = base_addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> @@ -368,6 +370,7 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	int i, b;
>  
>  	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
> @@ -417,15 +420,15 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = base_addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = base_addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = base_addr + sdi_offset;
>  		uint64_t exec_addr;
>  		int e = i % n_exec_queues;
>  
>  		if (!i) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = base_addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> @@ -539,6 +542,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t exec_sync;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	int i, b;
>  
>  	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
> @@ -593,15 +597,15 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = base_addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = base_addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = base_addr + sdi_offset;
>  		uint64_t exec_addr;
>  		int e = i % n_exec_queues;
>  
>  		if (!i) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = base_addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> diff --git a/tests/xe/xe_exec_threads.c b/tests/xe/xe_exec_threads.c
> index e64c1639a..ff4ebc280 100644
> --- a/tests/xe/xe_exec_threads.c
> +++ b/tests/xe/xe_exec_threads.c
> @@ -486,6 +486,7 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = false };
>  	int i, j, b, hang_exec_queue = n_exec_queues / 2;
>  	bool owns_vm = false, owns_fd = false;
>  
> @@ -562,15 +563,15 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>  		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>  		uint64_t batch_addr = addr + batch_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = addr + spin_offset;
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = addr + sdi_offset;
>  		uint64_t exec_addr;
>  		int e = i % n_exec_queues;
>  
>  		if (flags & HANG && e == hang_exec_queue && i == e) {
> -			xe_spin_init(&data[i].spin, spin_addr, false);
> -			exec_addr = spin_addr;
> +			spin_opts.addr = addr + spin_offset;
> +			xe_spin_init(&data[i].spin, &spin_opts);
> +			exec_addr = spin_opts.addr;
>  		} else {
>  			b = 0;
>  			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> diff --git a/tests/xe/xe_vm.c b/tests/xe/xe_vm.c
> index e42c04e33..87604a407 100644
> --- a/tests/xe/xe_vm.c
> +++ b/tests/xe/xe_vm.c
> @@ -727,6 +727,7 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
>  		uint64_t pad;
>  		uint32_t data;
>  	} *data;
> +	struct xe_spin_opts spin_opts = { .preempt = true };
>  	int i, b;
>  
>  	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
> @@ -755,14 +756,14 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
>  		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>  		uint64_t sdi_addr = addr + sdi_offset;
>  		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
> -		uint64_t spin_addr = addr + spin_offset;
>  		int e = i;
>  
>  		if (i == 0) {
> -			/* Cork 1st exec_queue with a spinner */
----------------------- ^

> -			xe_spin_init(&data[i].spin, spin_addr, true);
> +			/* Cork 1st engine with a spinner */
----------------------- ^
Do not change this.

> +			spin_opts.addr = addr + spin_offset;
----------- ^
You introduced new spin_opts helper structure, imho it is worth
effort to split this patch into two, first with spin_opts changes,
then your addition to make spins with duration.

Regards,
Kamil

> +			xe_spin_init(&data[i].spin, &spin_opts);
>  			exec.exec_queue_id = exec_queues[e];
> -			exec.address = spin_addr;
> +			exec.address = spin_opts.addr;
>  			sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>  			sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>  			sync[1].handle = syncobjs[e];
> -- 
> 2.30.2
> 

Re: [igt-dev] [PATCH i-g-t 1/3] lib/xe_spin: fixed duration xe_spin capability

[Bernatowicz, Marcin]

Hi Kamil,

On 8/30/2023 1:53 PM, Kamil Konieczny wrote:
> Hi Marcin,
> 
> On 2023-08-25 at 13:19:11 +0000, Marcin Bernatowicz wrote:
>> Introduced struct xe_spin_opts for xe_spin initialization,
>> adjusted tests to new xe_spin_init signature.
>>
>> Extended spinner with fixed duration capability. It allows
>> to prepare fixed duration (ex. 10ms) workloads and take workloads/second
>> measurements, a handy utility for scheduling tests.
>>
>> v2: - added asserts in div64_u64_round_up, duration_to_ctx_ticks,
>>        simplified loop_addr (Zbyszek)
>>      - added xe_spin_init_opts macro (Zbyszek)
>>      - corrected patch title (Kamil)
>>
>> Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
>> ---
>>   lib/xe/xe_spin.c            | 121 ++++++++++++++++++++++++++++++------
>>   lib/xe/xe_spin.h            |  27 +++++++-
>>   tests/xe/xe_dma_buf_sync.c  |   6 +-
>>   tests/xe/xe_exec_balancer.c |   9 ++-
>>   tests/xe/xe_exec_reset.c    |  24 ++++---
>>   tests/xe/xe_exec_threads.c  |   7 ++-
>>   tests/xe/xe_vm.c            |   9 +--
>>   7 files changed, 158 insertions(+), 45 deletions(-)
>>
>> diff --git a/lib/xe/xe_spin.c b/lib/xe/xe_spin.c
>> index 7113972ee..dcc4aeea1 100644
>> --- a/lib/xe/xe_spin.c
>> +++ b/lib/xe/xe_spin.c
>> @@ -16,41 +16,130 @@
>>   #include "xe_ioctl.h"
>>   #include "xe_spin.h"
>>   
>> +static uint32_t read_timestamp_frequency(int fd, int gt_id)
>> +{
>> +	struct xe_device *dev = xe_device_get(fd);
>> +
>> +	igt_assert(dev && dev->gts && dev->gts->num_gt);
>> +	igt_assert(gt_id >= 0 && gt_id <= dev->gts->num_gt);
>> +
>> +	return dev->gts->gts[gt_id].clock_freq;
>> +}
>> +
>> +static uint64_t div64_u64_round_up(const uint64_t x, const uint64_t y)
>> +{
>> +	igt_assert(y > 0);
>> +
>> +	return (x + y - 1) / y;
> ----------- ^^^^^^^^^
> This may overflow to value < y.
> 
>> +}
>> +
>> +/**
>> + * duration_to_ctx_ticks:
>> + * @fd: opened device
>> + * @gt_id: tile id
>> + * @duration_ns: duration in nanoseconds to be converted to context timestamp ticks
>> + * @return: duration converted to context timestamp ticks.
>> + */
>> +uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t duration_ns)
>> +{
>> +	uint32_t f = read_timestamp_frequency(fd, gt_id);
>> +	uint64_t ctx_ticks = div64_u64_round_up(duration_ns * f, NSEC_PER_SEC);
>> +
>> +	igt_assert_lt_u64(ctx_ticks, XE_SPIN_MAX_CTX_TICKS);
>> +
>> +	return ctx_ticks;
>> +}
>> +
>> +#define MI_SRM_CS_MMIO				(1 << 19)
>> +#define MI_LRI_CS_MMIO				(1 << 19)
>> +#define MI_LRR_DST_CS_MMIO			(1 << 19)
>> +#define MI_LRR_SRC_CS_MMIO			(1 << 18)
>> +#define CTX_TIMESTAMP 0x3a8;
>> +#define CS_GPR(x) (0x600 + 8 * (x))
> 
> Put newline here.
> 
>> +enum { START_TS, NOW_TS };
>> +
>>   /**
>>    * xe_spin_init:
>>    * @spin: pointer to mapped bo in which spinner code will be written
>> - * @addr: offset of spinner within vm
>> - * @preempt: allow spinner to be preempted or not
>> + * @opts: pointer to spinner initialization options
>>    */
>> -void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt)
>> +void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts)
>>   {
>> -	uint64_t batch_offset = (char *)&spin->batch - (char *)spin;
>> -	uint64_t batch_addr = addr + batch_offset;
>> -	uint64_t start_offset = (char *)&spin->start - (char *)spin;
>> -	uint64_t start_addr = addr + start_offset;
>> -	uint64_t end_offset = (char *)&spin->end - (char *)spin;
>> -	uint64_t end_addr = addr + end_offset;
>> +	uint64_t loop_addr;
>> +	uint64_t start_addr = opts->addr + offsetof(struct xe_spin, start);
>> +	uint64_t end_addr = opts->addr + offsetof(struct xe_spin, end);
>> +	uint64_t ticks_delta_addr = opts->addr + offsetof(struct xe_spin, ticks_delta);
>> +	uint64_t pad_addr = opts->addr + offsetof(struct xe_spin, pad);
>>   	int b = 0;
>>   
>>   	spin->start = 0;
>>   	spin->end = 0xffffffff;
>> +	spin->ticks_delta = 0;
>> +
>> +	if (opts->ctx_ticks) {
>> +		/* store start timestamp */
>> +		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
>> +		spin->batch[b++] = CS_GPR(START_TS) + 4;
>> +		spin->batch[b++] = 0;
>> +		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
>> +		spin->batch[b++] = CTX_TIMESTAMP;
>> +		spin->batch[b++] = CS_GPR(START_TS);
>> +	}
>> +
>> +	loop_addr = opts->addr + b * sizeof(uint32_t);
>>   
>>   	spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>>   	spin->batch[b++] = start_addr;
>>   	spin->batch[b++] = start_addr >> 32;
>>   	spin->batch[b++] = 0xc0ffee;
>>   
>> -	if (preempt)
>> +	if (opts->preempt)
>>   		spin->batch[b++] = (0x5 << 23);
>>   
>> +	if (opts->ctx_ticks) {
>> +		spin->batch[b++] = MI_LOAD_REGISTER_IMM(1) | MI_LRI_CS_MMIO;
>> +		spin->batch[b++] = CS_GPR(NOW_TS) + 4;
>> +		spin->batch[b++] = 0;
>> +		spin->batch[b++] = MI_LOAD_REGISTER_REG | MI_LRR_DST_CS_MMIO | MI_LRR_SRC_CS_MMIO;
>> +		spin->batch[b++] = CTX_TIMESTAMP;
>> +		spin->batch[b++] = CS_GPR(NOW_TS);
>> +
>> +		/* delta = now - start; inverted to match COND_BBE */
>> +		spin->batch[b++] = MI_MATH(4);
>> +		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCA, MI_MATH_REG(NOW_TS));
>> +		spin->batch[b++] = MI_MATH_LOAD(MI_MATH_REG_SRCB, MI_MATH_REG(START_TS));
>> +		spin->batch[b++] = MI_MATH_SUB;
>> +		spin->batch[b++] = MI_MATH_STOREINV(MI_MATH_REG(NOW_TS), MI_MATH_REG_ACCU);
>> +
>> +		/* Save delta for reading by COND_BBE */
>> +		spin->batch[b++] = MI_STORE_REGISTER_MEM | MI_SRM_CS_MMIO | 2;
>> +		spin->batch[b++] = CS_GPR(NOW_TS);
>> +		spin->batch[b++] = ticks_delta_addr;
>> +		spin->batch[b++] = ticks_delta_addr >> 32;
>> +
>> +		/* Delay between SRM and COND_BBE to post the writes */
>> +		for (int n = 0; n < 8; n++) {
>> +			spin->batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>> +			spin->batch[b++] = pad_addr;
>> +			spin->batch[b++] = pad_addr >> 32;
>> +			spin->batch[b++] = 0xc0ffee;
>> +		}
>> +
>> +		/* Break if delta [time elapsed] > ns */
>> +		spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
>> +		spin->batch[b++] = ~(opts->ctx_ticks);
>> +		spin->batch[b++] = ticks_delta_addr;
>> +		spin->batch[b++] = ticks_delta_addr >> 32;
>> +	}
>> +
>>   	spin->batch[b++] = MI_COND_BATCH_BUFFER_END | MI_DO_COMPARE | 2;
>>   	spin->batch[b++] = 0;
>>   	spin->batch[b++] = end_addr;
>>   	spin->batch[b++] = end_addr >> 32;
>>   
>>   	spin->batch[b++] = MI_BATCH_BUFFER_START | 1 << 8 | 1;
>> -	spin->batch[b++] = batch_addr;
>> -	spin->batch[b++] = batch_addr >> 32;
>> +	spin->batch[b++] = loop_addr;
>> +	spin->batch[b++] = loop_addr >> 32;
>>   
>>   	igt_assert(b <= ARRAY_SIZE(spin->batch));
>>   }
>> @@ -133,11 +222,7 @@ xe_spin_create(int fd, const struct igt_spin_factory *opt)
>>   	addr = intel_allocator_alloc_with_strategy(ahnd, spin->handle, bo_size, 0, ALLOC_STRATEGY_LOW_TO_HIGH);
>>   	xe_vm_bind_sync(fd, spin->vm, spin->handle, 0, addr, bo_size);
>>   
>> -	if (!(opt->flags & IGT_SPIN_NO_PREEMPTION))
>> -		xe_spin_init(xe_spin, addr, true);
>> -	else
>> -		xe_spin_init(xe_spin, addr, false);
>> -
>> +	xe_spin_init_opts(xe_spin, .addr = addr, .preempt = !(opt->flags & IGT_SPIN_NO_PREEMPTION));
>>   	exec.exec_queue_id = spin->engine;
>>   	exec.address = addr;
>>   	sync.handle = spin->syncobj;
>> @@ -219,7 +304,7 @@ void xe_cork_init(int fd, struct drm_xe_engine_class_instance *hwe,
>>   	exec_queue = xe_exec_queue_create(fd, vm, hwe, 0);
>>   	syncobj = syncobj_create(fd, 0);
>>   
>> -	xe_spin_init(spin, addr, true);
>> +	xe_spin_init_opts(spin, .addr = addr, .preempt = true);
>>   	exec.exec_queue_id = exec_queue;
>>   	exec.address = addr;
>>   	sync.handle = syncobj;
>> diff --git a/lib/xe/xe_spin.h b/lib/xe/xe_spin.h
>> index c84db175d..f1abc1102 100644
>> --- a/lib/xe/xe_spin.h
>> +++ b/lib/xe/xe_spin.h
>> @@ -15,15 +15,38 @@
>>   #include "xe_query.h"
>>   #include "lib/igt_dummyload.h"
>>   
>> +#define XE_SPIN_MAX_CTX_TICKS UINT32_MAX - 1000
>> +
>> +/** struct xe_spin_opts
>> + *
>> + * @addr: offset of spinner within vm
>> + * @preempt: allow spinner to be preempted or not
>> + * @ctx_ticks: number of ticks after which spinner is stopped, applied if > 0
>> + *
>> + * Used to initialize struct xe_spin spinner behavior.
>> + */
>> +struct xe_spin_opts {
>> +	uint64_t addr;
>> +	bool preempt;
>> +	uint32_t ctx_ticks;
>> +};
>> +
>>   /* Mapped GPU object */
>>   struct xe_spin {
>> -	uint32_t batch[16];
>> +	uint32_t batch[128];
>>   	uint64_t pad;
>>   	uint32_t start;
>>   	uint32_t end;
>> +	uint32_t ticks_delta;
>>   };
>> +
>>   igt_spin_t *xe_spin_create(int fd, const struct igt_spin_factory *opt);
>> -void xe_spin_init(struct xe_spin *spin, uint64_t addr, bool preempt);
>> +uint32_t duration_to_ctx_ticks(int fd, int gt_id, uint64_t ns);
>> +void xe_spin_init(struct xe_spin *spin, struct xe_spin_opts *opts);
>> +
>> +#define xe_spin_init_opts(fd, ...) \
>> +	xe_spin_init(fd, &((struct xe_spin_opts){__VA_ARGS__}))
>> +
>>   bool xe_spin_started(struct xe_spin *spin);
>>   void xe_spin_sync_wait(int fd, struct igt_spin *spin);
>>   void xe_spin_wait_started(struct xe_spin *spin);
>> diff --git a/tests/xe/xe_dma_buf_sync.c b/tests/xe/xe_dma_buf_sync.c
>> index 29d675154..627f4c1e5 100644
>> --- a/tests/xe/xe_dma_buf_sync.c
>> +++ b/tests/xe/xe_dma_buf_sync.c
>> @@ -144,7 +144,6 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
>>   		uint64_t sdi_offset = (char *)&data[i]->data - (char *)data[i];
>>   		uint64_t sdi_addr = addr + sdi_offset;
>>   		uint64_t spin_offset = (char *)&data[i]->spin - (char *)data[i];
>> -		uint64_t spin_addr = addr + spin_offset;
>>   		struct drm_xe_sync sync[2] = {
>>   			{ .flags = DRM_XE_SYNC_SYNCOBJ, },
>>   			{ .flags = DRM_XE_SYNC_SYNCOBJ | DRM_XE_SYNC_SIGNAL, },
>> @@ -153,14 +152,15 @@ test_export_dma_buf(struct drm_xe_engine_class_instance *hwe0,
>>   			.num_batch_buffer = 1,
>>   			.syncs = to_user_pointer(sync),
>>   		};
>> +		struct xe_spin_opts spin_opts = { .addr = addr + spin_offset, .preempt = true };
>>   		uint32_t syncobj;
>>   		int b = 0;
>>   		int sync_fd;
>>   
>>   		/* Write spinner on FD[0] */
>> -		xe_spin_init(&data[i]->spin, spin_addr, true);
>> +		xe_spin_init(&data[i]->spin, &spin_opts);
>>   		exec.exec_queue_id = exec_queue[0];
>> -		exec.address = spin_addr;
>> +		exec.address = spin_opts.addr;
>>   		xe_exec(fd[0], &exec);
>>   
>>   		/* Export prime BO as sync file and veify business */
>> diff --git a/tests/xe/xe_exec_balancer.c b/tests/xe/xe_exec_balancer.c
>> index f364a4b7a..d7d8dd8fb 100644
>> --- a/tests/xe/xe_exec_balancer.c
>> +++ b/tests/xe/xe_exec_balancer.c
>> @@ -52,6 +52,7 @@ static void test_all_active(int fd, int gt, int class)
>>   	struct {
>>   		struct xe_spin spin;
>>   	} *data;
>> +	struct xe_spin_opts spin_opts = { .preempt = false };
>>   	struct drm_xe_engine_class_instance *hwe;
>>   	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>>   	int i, num_placements = 0;
>> @@ -90,16 +91,14 @@ static void test_all_active(int fd, int gt, int class)
>>   	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>>   
>>   	for (i = 0; i < num_placements; i++) {
>> -		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
>> -		uint64_t spin_addr = addr + spin_offset;
>> -
>> -		xe_spin_init(&data[i].spin, spin_addr, false);
>> +		spin_opts.addr = addr + (char *)&data[i].spin - (char *)data;
>> +		xe_spin_init(&data[i].spin, &spin_opts);
>>   		sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>>   		sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>>   		sync[1].handle = syncobjs[i];
>>   
>>   		exec.exec_queue_id = exec_queues[i];
>> -		exec.address = spin_addr;
>> +		exec.address = spin_opts.addr;
>>   		xe_exec(fd, &exec);
>>   		xe_spin_wait_started(&data[i].spin);
>>   	}
>> diff --git a/tests/xe/xe_exec_reset.c b/tests/xe/xe_exec_reset.c
>> index a2d33baf1..be6bbada6 100644
>> --- a/tests/xe/xe_exec_reset.c
>> +++ b/tests/xe/xe_exec_reset.c
>> @@ -44,6 +44,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
>>   	size_t bo_size;
>>   	uint32_t bo = 0;
>>   	struct xe_spin *spin;
>> +	struct xe_spin_opts spin_opts = { .addr = addr, .preempt = false };
>>   
>>   	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
>>   	bo_size = sizeof(*spin);
>> @@ -60,7 +61,7 @@ static void test_spin(int fd, struct drm_xe_engine_class_instance *eci)
>>   	sync[0].handle = syncobj_create(fd, 0);
>>   	xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>>   
>> -	xe_spin_init(spin, addr, false);
>> +	xe_spin_init(spin, &spin_opts);
>>   
>>   	sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>>   	sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>> @@ -165,6 +166,7 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>>   		uint64_t pad;
>>   		uint32_t data;
>>   	} *data;
>> +	struct xe_spin_opts spin_opts = { .preempt = false };
>>   	struct drm_xe_engine_class_instance *hwe;
>>   	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
>>   	int i, j, b, num_placements = 0, bad_batches = 1;
>> @@ -236,7 +238,6 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>>   		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>>   		uint64_t batch_addr = base_addr + batch_offset;
>>   		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
>> -		uint64_t spin_addr = base_addr + spin_offset;
>>   		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>>   		uint64_t sdi_addr = base_addr + sdi_offset;
>>   		uint64_t exec_addr;
>> @@ -247,8 +248,9 @@ test_balancer(int fd, int gt, int class, int n_exec_queues, int n_execs,
>>   			batches[j] = batch_addr;
>>   
>>   		if (i < bad_batches) {
>> -			xe_spin_init(&data[i].spin, spin_addr, false);
>> -			exec_addr = spin_addr;
>> +			spin_opts.addr = base_addr + spin_offset;
>> +			xe_spin_init(&data[i].spin, &spin_opts);
>> +			exec_addr = spin_opts.addr;
>>   		} else {
>>   			b = 0;
>>   			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>> @@ -368,6 +370,7 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
>>   		uint64_t pad;
>>   		uint32_t data;
>>   	} *data;
>> +	struct xe_spin_opts spin_opts = { .preempt = false };
>>   	int i, b;
>>   
>>   	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
>> @@ -417,15 +420,15 @@ test_legacy_mode(int fd, struct drm_xe_engine_class_instance *eci,
>>   		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>>   		uint64_t batch_addr = base_addr + batch_offset;
>>   		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
>> -		uint64_t spin_addr = base_addr + spin_offset;
>>   		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>>   		uint64_t sdi_addr = base_addr + sdi_offset;
>>   		uint64_t exec_addr;
>>   		int e = i % n_exec_queues;
>>   
>>   		if (!i) {
>> -			xe_spin_init(&data[i].spin, spin_addr, false);
>> -			exec_addr = spin_addr;
>> +			spin_opts.addr = base_addr + spin_offset;
>> +			xe_spin_init(&data[i].spin, &spin_opts);
>> +			exec_addr = spin_opts.addr;
>>   		} else {
>>   			b = 0;
>>   			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>> @@ -539,6 +542,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>>   		uint64_t exec_sync;
>>   		uint32_t data;
>>   	} *data;
>> +	struct xe_spin_opts spin_opts = { .preempt = false };
>>   	int i, b;
>>   
>>   	igt_assert(n_exec_queues <= MAX_N_EXECQUEUES);
>> @@ -593,15 +597,15 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>>   		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>>   		uint64_t batch_addr = base_addr + batch_offset;
>>   		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
>> -		uint64_t spin_addr = base_addr + spin_offset;
>>   		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>>   		uint64_t sdi_addr = base_addr + sdi_offset;
>>   		uint64_t exec_addr;
>>   		int e = i % n_exec_queues;
>>   
>>   		if (!i) {
>> -			xe_spin_init(&data[i].spin, spin_addr, false);
>> -			exec_addr = spin_addr;
>> +			spin_opts.addr = base_addr + spin_offset;
>> +			xe_spin_init(&data[i].spin, &spin_opts);
>> +			exec_addr = spin_opts.addr;
>>   		} else {
>>   			b = 0;
>>   			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>> diff --git a/tests/xe/xe_exec_threads.c b/tests/xe/xe_exec_threads.c
>> index e64c1639a..ff4ebc280 100644
>> --- a/tests/xe/xe_exec_threads.c
>> +++ b/tests/xe/xe_exec_threads.c
>> @@ -486,6 +486,7 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>>   		uint64_t pad;
>>   		uint32_t data;
>>   	} *data;
>> +	struct xe_spin_opts spin_opts = { .preempt = false };
>>   	int i, j, b, hang_exec_queue = n_exec_queues / 2;
>>   	bool owns_vm = false, owns_fd = false;
>>   
>> @@ -562,15 +563,15 @@ test_legacy_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>>   		uint64_t batch_offset = (char *)&data[i].batch - (char *)data;
>>   		uint64_t batch_addr = addr + batch_offset;
>>   		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
>> -		uint64_t spin_addr = addr + spin_offset;
>>   		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>>   		uint64_t sdi_addr = addr + sdi_offset;
>>   		uint64_t exec_addr;
>>   		int e = i % n_exec_queues;
>>   
>>   		if (flags & HANG && e == hang_exec_queue && i == e) {
>> -			xe_spin_init(&data[i].spin, spin_addr, false);
>> -			exec_addr = spin_addr;
>> +			spin_opts.addr = addr + spin_offset;
>> +			xe_spin_init(&data[i].spin, &spin_opts);
>> +			exec_addr = spin_opts.addr;
>>   		} else {
>>   			b = 0;
>>   			data[i].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
>> diff --git a/tests/xe/xe_vm.c b/tests/xe/xe_vm.c
>> index e42c04e33..87604a407 100644
>> --- a/tests/xe/xe_vm.c
>> +++ b/tests/xe/xe_vm.c
>> @@ -727,6 +727,7 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
>>   		uint64_t pad;
>>   		uint32_t data;
>>   	} *data;
>> +	struct xe_spin_opts spin_opts = { .preempt = true };
>>   	int i, b;
>>   
>>   	vm = xe_vm_create(fd, DRM_XE_VM_CREATE_ASYNC_BIND_OPS, 0);
>> @@ -755,14 +756,14 @@ test_bind_execqueues_independent(int fd, struct drm_xe_engine_class_instance *ec
>>   		uint64_t sdi_offset = (char *)&data[i].data - (char *)data;
>>   		uint64_t sdi_addr = addr + sdi_offset;
>>   		uint64_t spin_offset = (char *)&data[i].spin - (char *)data;
>> -		uint64_t spin_addr = addr + spin_offset;
>>   		int e = i;
>>   
>>   		if (i == 0) {
>> -			/* Cork 1st exec_queue with a spinner */
> ----------------------- ^
> 
>> -			xe_spin_init(&data[i].spin, spin_addr, true);
>> +			/* Cork 1st engine with a spinner */
> ----------------------- ^
> Do not change this.
> 
>> +			spin_opts.addr = addr + spin_offset;
> ----------- ^
> You introduced new spin_opts helper structure, imho it is worth
> effort to split this patch into two, first with spin_opts changes,
> then your addition to make spins with duration.
> 
> Regards,
> Kamil

I agree with the comments, I prefer to continue and send version 3 to 
https://patchwork.freedesktop.org/series/122624/ ?

The main focus of this RFC are gem_wsim changes.

Thanks,
Marcin

> 
>> +			xe_spin_init(&data[i].spin, &spin_opts);
>>   			exec.exec_queue_id = exec_queues[e];
>> -			exec.address = spin_addr;
>> +			exec.address = spin_opts.addr;
>>   			sync[0].flags &= ~DRM_XE_SYNC_SIGNAL;
>>   			sync[1].flags |= DRM_XE_SYNC_SIGNAL;
>>   			sync[1].handle = syncobjs[e];
>> -- 
>> 2.30.2
>>

[igt-dev] [PATCH i-g-t 2/3] lib/igt_device_scan: Xe get integrated/discrete card functions

[Marcin Bernatowicz]

Xe functions to get integrated/discrete card.

Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
---
 lib/igt_device_scan.c | 34 +++++++++++++++++++++++++---------
 lib/igt_device_scan.h |  2 ++
 2 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/lib/igt_device_scan.c b/lib/igt_device_scan.c
index ae69ed09f..f8b8ca281 100644
--- a/lib/igt_device_scan.c
+++ b/lib/igt_device_scan.c
@@ -769,25 +769,27 @@ __copy_dev_to_card(struct igt_device *dev, struct igt_device_card *card)
  * Iterate over all igt_devices array and find first discrete/integrated card.
  * card->pci_slot_name will be updated only if a card is found.
  */
-static bool __find_first_i915_card(struct igt_device_card *card, bool discrete)
+static bool __find_first_intel_card(struct igt_device_card *card, bool want_discrete,
+				    const char *drv_name)
 {
 	struct igt_device *dev;
-	int cmp;
+	int is_integrated;
 
+	igt_assert(drv_name);
 	memset(card, 0, sizeof(*card));
 
 	igt_list_for_each_entry(dev, &igt_devs.all, link) {
 
-		if (!is_pci_subsystem(dev) || strcmp(dev->driver, "i915"))
+		if (!is_pci_subsystem(dev) || strcmp(dev->driver, drv_name))
 			continue;
 
-		cmp = strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
-			      PCI_SLOT_NAME_SIZE);
+		is_integrated = !strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
+					 PCI_SLOT_NAME_SIZE);
 
-		if (discrete && cmp) {
+		if (want_discrete && !is_integrated) {
 			__copy_dev_to_card(dev, card);
 			return true;
-		} else if (!discrete && !cmp) {
+		} else if (!want_discrete && is_integrated) {
 			__copy_dev_to_card(dev, card);
 			return true;
 		}
@@ -800,14 +802,28 @@ bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card)
 {
 	igt_assert(card);
 
-	return __find_first_i915_card(card, true);
+	return __find_first_intel_card(card, true, "i915");
+}
+
+bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card)
+{
+	igt_assert(card);
+
+	return __find_first_intel_card(card, true, "xe");
 }
 
 bool igt_device_find_integrated_card(struct igt_device_card *card)
 {
 	igt_assert(card);
 
-	return __find_first_i915_card(card, false);
+	return __find_first_intel_card(card, false, "i915");
+}
+
+bool igt_device_find_xe_integrated_card(struct igt_device_card *card)
+{
+	igt_assert(card);
+
+	return __find_first_intel_card(card, false, "xe");
 }
 
 static struct igt_device *igt_device_from_syspath(const char *syspath)
diff --git a/lib/igt_device_scan.h b/lib/igt_device_scan.h
index e6b0f1b90..b8f6a843d 100644
--- a/lib/igt_device_scan.h
+++ b/lib/igt_device_scan.h
@@ -87,6 +87,8 @@ bool igt_device_card_match_pci(const char *filter,
 	struct igt_device_card *card);
 bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card);
 bool igt_device_find_integrated_card(struct igt_device_card *card);
+bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card);
+bool igt_device_find_xe_integrated_card(struct igt_device_card *card);
 char *igt_device_get_pretty_name(struct igt_device_card *card, bool numeric);
 int igt_open_card(struct igt_device_card *card);
 int igt_open_render(struct igt_device_card *card);
-- 
2.30.2

Re: [igt-dev] [PATCH i-g-t 2/3] lib/igt_device_scan: Xe get integrated/discrete card functions

[Kamil Konieczny]

Hi Marcin,

On 2023-08-25 at 13:19:12 +0000, Marcin Bernatowicz wrote:
> Xe functions to get integrated/discrete card.
> 
> Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
> ---
>  lib/igt_device_scan.c | 34 +++++++++++++++++++++++++---------
>  lib/igt_device_scan.h |  2 ++
>  2 files changed, 27 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/igt_device_scan.c b/lib/igt_device_scan.c
> index ae69ed09f..f8b8ca281 100644
> --- a/lib/igt_device_scan.c
> +++ b/lib/igt_device_scan.c
> @@ -769,25 +769,27 @@ __copy_dev_to_card(struct igt_device *dev, struct igt_device_card *card)
>   * Iterate over all igt_devices array and find first discrete/integrated card.
>   * card->pci_slot_name will be updated only if a card is found.
>   */
> -static bool __find_first_i915_card(struct igt_device_card *card, bool discrete)
> +static bool __find_first_intel_card(struct igt_device_card *card, bool want_discrete,
> +				    const char *drv_name)
>  {
>  	struct igt_device *dev;
> -	int cmp;
> +	int is_integrated;
>  
> +	igt_assert(drv_name);
>  	memset(card, 0, sizeof(*card));
>  
>  	igt_list_for_each_entry(dev, &igt_devs.all, link) {
>  
> -		if (!is_pci_subsystem(dev) || strcmp(dev->driver, "i915"))
> +		if (!is_pci_subsystem(dev) || strcmp(dev->driver, drv_name))
--------------------------------------------------------- ^^^^^^^^

May you extend this to cover also "intel" card here?
I mean both cases, i915 or xe.

>  			continue;
>  
> -		cmp = strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
> -			      PCI_SLOT_NAME_SIZE);
> +		is_integrated = !strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
> +					 PCI_SLOT_NAME_SIZE);
>  
> -		if (discrete && cmp) {
> +		if (want_discrete && !is_integrated) {
>  			__copy_dev_to_card(dev, card);
>  			return true;
> -		} else if (!discrete && !cmp) {
> +		} else if (!want_discrete && is_integrated) {
>  			__copy_dev_to_card(dev, card);
>  			return true;
>  		}
> @@ -800,14 +802,28 @@ bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card)
>  {
>  	igt_assert(card);
>  
> -	return __find_first_i915_card(card, true);
> +	return __find_first_intel_card(card, true, "i915");
> +}
> +

Document all new public lib functions.

> +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card)
> +{
> +	igt_assert(card);
> +
> +	return __find_first_intel_card(card, true, "xe");
>  }
>  
>  bool igt_device_find_integrated_card(struct igt_device_card *card)
>  {
>  	igt_assert(card);
>  
> -	return __find_first_i915_card(card, false);
> +	return __find_first_intel_card(card, false, "i915");
> +}
> +

Same here.

Regards,
Kamil

> +bool igt_device_find_xe_integrated_card(struct igt_device_card *card)
> +{
> +	igt_assert(card);
> +
> +	return __find_first_intel_card(card, false, "xe");
>  }
>  
>  static struct igt_device *igt_device_from_syspath(const char *syspath)
> diff --git a/lib/igt_device_scan.h b/lib/igt_device_scan.h
> index e6b0f1b90..b8f6a843d 100644
> --- a/lib/igt_device_scan.h
> +++ b/lib/igt_device_scan.h
> @@ -87,6 +87,8 @@ bool igt_device_card_match_pci(const char *filter,
>  	struct igt_device_card *card);
>  bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card);
>  bool igt_device_find_integrated_card(struct igt_device_card *card);
> +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card);
> +bool igt_device_find_xe_integrated_card(struct igt_device_card *card);
>  char *igt_device_get_pretty_name(struct igt_device_card *card, bool numeric);
>  int igt_open_card(struct igt_device_card *card);
>  int igt_open_render(struct igt_device_card *card);
> -- 
> 2.30.2
> 

Re: [igt-dev] [PATCH i-g-t 2/3] lib/igt_device_scan: Xe get integrated/discrete card functions

[Zbigniew Kempczyński]

On Mon, Aug 28, 2023 at 07:05:34PM +0200, Kamil Konieczny wrote:
> Hi Marcin,
> 
> On 2023-08-25 at 13:19:12 +0000, Marcin Bernatowicz wrote:
> > Xe functions to get integrated/discrete card.
> > 
> > Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
> > ---
> >  lib/igt_device_scan.c | 34 +++++++++++++++++++++++++---------
> >  lib/igt_device_scan.h |  2 ++
> >  2 files changed, 27 insertions(+), 9 deletions(-)
> > 
> > diff --git a/lib/igt_device_scan.c b/lib/igt_device_scan.c
> > index ae69ed09f..f8b8ca281 100644
> > --- a/lib/igt_device_scan.c
> > +++ b/lib/igt_device_scan.c
> > @@ -769,25 +769,27 @@ __copy_dev_to_card(struct igt_device *dev, struct igt_device_card *card)
> >   * Iterate over all igt_devices array and find first discrete/integrated card.
> >   * card->pci_slot_name will be updated only if a card is found.
> >   */
> > -static bool __find_first_i915_card(struct igt_device_card *card, bool discrete)
> > +static bool __find_first_intel_card(struct igt_device_card *card, bool want_discrete,
> > +				    const char *drv_name)

As drv_name occurs here function likely should be called
__first_first_card_by_driver_name() or sth.

--
Zbigniew

> >  {
> >  	struct igt_device *dev;
> > -	int cmp;
> > +	int is_integrated;
> >  
> > +	igt_assert(drv_name);
> >  	memset(card, 0, sizeof(*card));
> >  
> >  	igt_list_for_each_entry(dev, &igt_devs.all, link) {
> >  
> > -		if (!is_pci_subsystem(dev) || strcmp(dev->driver, "i915"))
> > +		if (!is_pci_subsystem(dev) || strcmp(dev->driver, drv_name))
> --------------------------------------------------------- ^^^^^^^^
> 
> May you extend this to cover also "intel" card here?
> I mean both cases, i915 or xe.
> 
> >  			continue;
> >  
> > -		cmp = strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
> > -			      PCI_SLOT_NAME_SIZE);
> > +		is_integrated = !strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
> > +					 PCI_SLOT_NAME_SIZE);
> >  
> > -		if (discrete && cmp) {
> > +		if (want_discrete && !is_integrated) {
> >  			__copy_dev_to_card(dev, card);
> >  			return true;
> > -		} else if (!discrete && !cmp) {
> > +		} else if (!want_discrete && is_integrated) {
> >  			__copy_dev_to_card(dev, card);
> >  			return true;
> >  		}
> > @@ -800,14 +802,28 @@ bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card)
> >  {
> >  	igt_assert(card);
> >  
> > -	return __find_first_i915_card(card, true);
> > +	return __find_first_intel_card(card, true, "i915");
> > +}
> > +
> 
> Document all new public lib functions.
> 
> > +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card)
> > +{
> > +	igt_assert(card);
> > +
> > +	return __find_first_intel_card(card, true, "xe");
> >  }
> >  
> >  bool igt_device_find_integrated_card(struct igt_device_card *card)
> >  {
> >  	igt_assert(card);
> >  
> > -	return __find_first_i915_card(card, false);
> > +	return __find_first_intel_card(card, false, "i915");
> > +}
> > +
> 
> Same here.
> 
> Regards,
> Kamil
> 
> > +bool igt_device_find_xe_integrated_card(struct igt_device_card *card)
> > +{
> > +	igt_assert(card);
> > +
> > +	return __find_first_intel_card(card, false, "xe");
> >  }
> >  
> >  static struct igt_device *igt_device_from_syspath(const char *syspath)
> > diff --git a/lib/igt_device_scan.h b/lib/igt_device_scan.h
> > index e6b0f1b90..b8f6a843d 100644
> > --- a/lib/igt_device_scan.h
> > +++ b/lib/igt_device_scan.h
> > @@ -87,6 +87,8 @@ bool igt_device_card_match_pci(const char *filter,
> >  	struct igt_device_card *card);
> >  bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card);
> >  bool igt_device_find_integrated_card(struct igt_device_card *card);
> > +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card);
> > +bool igt_device_find_xe_integrated_card(struct igt_device_card *card);
> >  char *igt_device_get_pretty_name(struct igt_device_card *card, bool numeric);
> >  int igt_open_card(struct igt_device_card *card);
> >  int igt_open_render(struct igt_device_card *card);
> > -- 
> > 2.30.2
> > 

Re: [igt-dev] [PATCH i-g-t 2/3] lib/igt_device_scan: Xe get integrated/discrete card functions

[Kamil Konieczny]

Hi Zbigniew,

On 2023-08-30 at 12:03:30 +0200, Zbigniew Kempczyński wrote:
> On Mon, Aug 28, 2023 at 07:05:34PM +0200, Kamil Konieczny wrote:
> > Hi Marcin,
> > 
> > On 2023-08-25 at 13:19:12 +0000, Marcin Bernatowicz wrote:
> > > Xe functions to get integrated/discrete card.
> > > 
> > > Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
> > > ---
> > >  lib/igt_device_scan.c | 34 +++++++++++++++++++++++++---------
> > >  lib/igt_device_scan.h |  2 ++
> > >  2 files changed, 27 insertions(+), 9 deletions(-)
> > > 
> > > diff --git a/lib/igt_device_scan.c b/lib/igt_device_scan.c
> > > index ae69ed09f..f8b8ca281 100644
> > > --- a/lib/igt_device_scan.c
> > > +++ b/lib/igt_device_scan.c
> > > @@ -769,25 +769,27 @@ __copy_dev_to_card(struct igt_device *dev, struct igt_device_card *card)
> > >   * Iterate over all igt_devices array and find first discrete/integrated card.
> > >   * card->pci_slot_name will be updated only if a card is found.
> > >   */
> > > -static bool __find_first_i915_card(struct igt_device_card *card, bool discrete)
> > > +static bool __find_first_intel_card(struct igt_device_card *card, bool want_discrete,
> > > +				    const char *drv_name)
> 
> As drv_name occurs here function likely should be called
> __first_first_card_by_driver_name() or sth.

or even longer:

__first_first_intel_card_by_driver_name()

btw allow drv_name be NULL, in that case accept i915 or xe.

Regrds,
Kamil

> 
> --
> Zbigniew
> 
> > >  {
> > >  	struct igt_device *dev;
> > > -	int cmp;
> > > +	int is_integrated;
> > >  
> > > +	igt_assert(drv_name);
> > >  	memset(card, 0, sizeof(*card));
> > >  
> > >  	igt_list_for_each_entry(dev, &igt_devs.all, link) {
> > >  
> > > -		if (!is_pci_subsystem(dev) || strcmp(dev->driver, "i915"))
> > > +		if (!is_pci_subsystem(dev) || strcmp(dev->driver, drv_name))
> > --------------------------------------------------------- ^^^^^^^^
> > 
> > May you extend this to cover also "intel" card here?
> > I mean both cases, i915 or xe.
> > 
> > >  			continue;
> > >  
> > > -		cmp = strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
> > > -			      PCI_SLOT_NAME_SIZE);
> > > +		is_integrated = !strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
> > > +					 PCI_SLOT_NAME_SIZE);
> > >  
> > > -		if (discrete && cmp) {
> > > +		if (want_discrete && !is_integrated) {
> > >  			__copy_dev_to_card(dev, card);
> > >  			return true;
> > > -		} else if (!discrete && !cmp) {
> > > +		} else if (!want_discrete && is_integrated) {
> > >  			__copy_dev_to_card(dev, card);
> > >  			return true;
> > >  		}
> > > @@ -800,14 +802,28 @@ bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card)
> > >  {
> > >  	igt_assert(card);
> > >  
> > > -	return __find_first_i915_card(card, true);
> > > +	return __find_first_intel_card(card, true, "i915");
> > > +}
> > > +
> > 
> > Document all new public lib functions.
> > 
> > > +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card)
> > > +{
> > > +	igt_assert(card);
> > > +
> > > +	return __find_first_intel_card(card, true, "xe");
> > >  }
> > >  
> > >  bool igt_device_find_integrated_card(struct igt_device_card *card)
> > >  {
> > >  	igt_assert(card);
> > >  
> > > -	return __find_first_i915_card(card, false);
> > > +	return __find_first_intel_card(card, false, "i915");
> > > +}
> > > +
> > 
> > Same here.
> > 
> > Regards,
> > Kamil
> > 
> > > +bool igt_device_find_xe_integrated_card(struct igt_device_card *card)
> > > +{
> > > +	igt_assert(card);
> > > +
> > > +	return __find_first_intel_card(card, false, "xe");
> > >  }
> > >  
> > >  static struct igt_device *igt_device_from_syspath(const char *syspath)
> > > diff --git a/lib/igt_device_scan.h b/lib/igt_device_scan.h
> > > index e6b0f1b90..b8f6a843d 100644
> > > --- a/lib/igt_device_scan.h
> > > +++ b/lib/igt_device_scan.h
> > > @@ -87,6 +87,8 @@ bool igt_device_card_match_pci(const char *filter,
> > >  	struct igt_device_card *card);
> > >  bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card);
> > >  bool igt_device_find_integrated_card(struct igt_device_card *card);
> > > +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card);
> > > +bool igt_device_find_xe_integrated_card(struct igt_device_card *card);
> > >  char *igt_device_get_pretty_name(struct igt_device_card *card, bool numeric);
> > >  int igt_open_card(struct igt_device_card *card);
> > >  int igt_open_render(struct igt_device_card *card);
> > > -- 
> > > 2.30.2
> > > 

Re: [igt-dev] [PATCH i-g-t 2/3] lib/igt_device_scan: Xe get integrated/discrete card functions

[Bernatowicz, Marcin]

On 8/28/2023 7:05 PM, Kamil Konieczny wrote:
> Hi Marcin,
> 
> On 2023-08-25 at 13:19:12 +0000, Marcin Bernatowicz wrote:
>> Xe functions to get integrated/discrete card.
>>
>> Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
>> ---
>>   lib/igt_device_scan.c | 34 +++++++++++++++++++++++++---------
>>   lib/igt_device_scan.h |  2 ++
>>   2 files changed, 27 insertions(+), 9 deletions(-)
>>
>> diff --git a/lib/igt_device_scan.c b/lib/igt_device_scan.c
>> index ae69ed09f..f8b8ca281 100644
>> --- a/lib/igt_device_scan.c
>> +++ b/lib/igt_device_scan.c
>> @@ -769,25 +769,27 @@ __copy_dev_to_card(struct igt_device *dev, struct igt_device_card *card)
>>    * Iterate over all igt_devices array and find first discrete/integrated card.
>>    * card->pci_slot_name will be updated only if a card is found.
>>    */
>> -static bool __find_first_i915_card(struct igt_device_card *card, bool discrete)
>> +static bool __find_first_intel_card(struct igt_device_card *card, bool want_discrete,
>> +				    const char *drv_name)
>>   {
>>   	struct igt_device *dev;
>> -	int cmp;
>> +	int is_integrated;
>>   
>> +	igt_assert(drv_name);
>>   	memset(card, 0, sizeof(*card));
>>   
>>   	igt_list_for_each_entry(dev, &igt_devs.all, link) {
>>   
>> -		if (!is_pci_subsystem(dev) || strcmp(dev->driver, "i915"))
>> +		if (!is_pci_subsystem(dev) || strcmp(dev->driver, drv_name))
> --------------------------------------------------------- ^^^^^^^^
> 
> May you extend this to cover also "intel" card here?
> I mean both cases, i915 or xe.
> 

I didn't want to complicate the code too much, but isn't it
__find_first_intel_card(...,"i915") || __find_first_intel_card(...,"xe")
?


>>   			continue;
>>   
>> -		cmp = strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
>> -			      PCI_SLOT_NAME_SIZE);
>> +		is_integrated = !strncmp(dev->pci_slot_name, INTEGRATED_I915_GPU_PCI_ID,
>> +					 PCI_SLOT_NAME_SIZE);
>>   
>> -		if (discrete && cmp) {
>> +		if (want_discrete && !is_integrated) {
>>   			__copy_dev_to_card(dev, card);
>>   			return true;
>> -		} else if (!discrete && !cmp) {
>> +		} else if (!want_discrete && is_integrated) {
>>   			__copy_dev_to_card(dev, card);
>>   			return true;
>>   		}
>> @@ -800,14 +802,28 @@ bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card)
>>   {
>>   	igt_assert(card);
>>   
>> -	return __find_first_i915_card(card, true);
>> +	return __find_first_intel_card(card, true, "i915");
>> +}
>> +
> 
> Document all new public lib functions.
> 
>> +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card)
>> +{
>> +	igt_assert(card);
>> +
>> +	return __find_first_intel_card(card, true, "xe");
>>   }
>>   
>>   bool igt_device_find_integrated_card(struct igt_device_card *card)
>>   {
>>   	igt_assert(card);
>>   
>> -	return __find_first_i915_card(card, false);
>> +	return __find_first_intel_card(card, false, "i915");
>> +}
>> +
> 
> Same here.
> 
> Regards,
> Kamil
> 
>> +bool igt_device_find_xe_integrated_card(struct igt_device_card *card)
>> +{
>> +	igt_assert(card);
>> +
>> +	return __find_first_intel_card(card, false, "xe");
>>   }
>>   
>>   static struct igt_device *igt_device_from_syspath(const char *syspath)
>> diff --git a/lib/igt_device_scan.h b/lib/igt_device_scan.h
>> index e6b0f1b90..b8f6a843d 100644
>> --- a/lib/igt_device_scan.h
>> +++ b/lib/igt_device_scan.h
>> @@ -87,6 +87,8 @@ bool igt_device_card_match_pci(const char *filter,
>>   	struct igt_device_card *card);
>>   bool igt_device_find_first_i915_discrete_card(struct igt_device_card *card);
>>   bool igt_device_find_integrated_card(struct igt_device_card *card);
>> +bool igt_device_find_first_xe_discrete_card(struct igt_device_card *card);
>> +bool igt_device_find_xe_integrated_card(struct igt_device_card *card);
>>   char *igt_device_get_pretty_name(struct igt_device_card *card, bool numeric);
>>   int igt_open_card(struct igt_device_card *card);
>>   int igt_open_render(struct igt_device_card *card);
>> -- 
>> 2.30.2
>>

[igt-dev] [PATCH i-g-t 3/3] [RFC] benchmarks/gem_wsim: added basic xe support

[Marcin Bernatowicz]

Added basic xe support with few examples.
Single binary handles both i915 and Xe devices,
but workload definitions differs between i915 and xe.
Xe does not use context abstraction, introduces new VM and Exec Queue
steps and BATCH step references exec queue.
For more details see wsim/README.
Some functionality is still missing: working sets,
load balancing (need some input if/how to do it in Xe - exec queues
width?).

The tool is handy for scheduling tests, we find it useful to verify vGPU
profiles defining different execution quantum/preemption timeout settings.

There is also some rationale for the tool in following thread:
https://lore.kernel.org/dri-devel/a443495f-5d1b-52e1-9b2f-80167deb6d57@linux.intel.com/

With this patch it should be possible to run following on xe device:

gem_wsim -w benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim -c 36 -r 600

Best with drm debug logs disabled:

echo 0 > /sys/module/drm/parameters/debug

Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
---
 benchmarks/gem_wsim.c                         | 842 ++++++++++++++----
 benchmarks/wsim/README                        |  87 +-
 benchmarks/wsim/xe_cloud-gaming-60fps.wsim    |  25 +
 benchmarks/wsim/xe_example.wsim               |  28 +
 benchmarks/wsim/xe_example01.wsim             |  19 +
 benchmarks/wsim/xe_example_fence.wsim         |  23 +
 .../wsim/xe_media_load_balance_fhd26u7.wsim   |  63 ++
 7 files changed, 909 insertions(+), 178 deletions(-)
 create mode 100644 benchmarks/wsim/xe_cloud-gaming-60fps.wsim
 create mode 100644 benchmarks/wsim/xe_example.wsim
 create mode 100644 benchmarks/wsim/xe_example01.wsim
 create mode 100644 benchmarks/wsim/xe_example_fence.wsim
 create mode 100644 benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim

diff --git a/benchmarks/gem_wsim.c b/benchmarks/gem_wsim.c
index 7b5e62a3b..a9dcb7e9f 100644
--- a/benchmarks/gem_wsim.c
+++ b/benchmarks/gem_wsim.c
@@ -42,6 +42,7 @@
 #include <limits.h>
 #include <pthread.h>
 #include <math.h>
+#include <ctype.h>
 
 #include "drm.h"
 #include "drmtest.h"
@@ -60,6 +61,12 @@
 #include "i915/gem_engine_topology.h"
 #include "i915/gem_mman.h"
 
+#include "igt_syncobj.h"
+#include "intel_allocator.h"
+#include "xe_drm.h"
+#include "xe/xe_ioctl.h"
+#include "xe/xe_spin.h"
+
 enum intel_engine_id {
 	DEFAULT,
 	RCS,
@@ -73,6 +80,7 @@ enum intel_engine_id {
 
 struct duration {
 	unsigned int min, max;
+	bool unbound_duration;
 };
 
 enum w_type
@@ -93,6 +101,9 @@ enum w_type
 	TERMINATE,
 	SSEU,
 	WORKINGSET,
+	VM,
+	EXEC_QUEUE,
+	SKIP,
 };
 
 struct dep_entry {
@@ -108,6 +119,10 @@ struct deps
 	struct dep_entry *list;
 };
 
+#define for_each_dep(__dep, __deps) \
+	for (int __i = 0; __i < __deps.nr && \
+	     (__dep = &__deps.list[__i]); ++__i)
+
 struct w_arg {
 	char *filename;
 	char *desc;
@@ -144,18 +159,18 @@ struct w_step
 	enum w_type type;
 	unsigned int context;
 	unsigned int engine;
+	unsigned int eq_idx;
 	struct duration duration;
-	bool unbound_duration;
 	struct deps data_deps;
 	struct deps fence_deps;
 	int emit_fence;
+
 	union {
 		int sync;
 		int delay;
 		int period;
 		int target;
 		int throttle;
-		int fence_signal;
 		int priority;
 		struct {
 			unsigned int engine_map_count;
@@ -168,21 +183,50 @@ struct w_step
 		};
 		int sseu;
 		struct working_set working_set;
+		struct vm *vm;
+		struct exec_queue *eq;
 	};
 
 	/* Implementation details */
 	unsigned int idx;
 	struct igt_list_head rq_link;
+
 	unsigned int request;
 	unsigned int preempt_us;
 
 	struct drm_i915_gem_execbuffer2 eb;
 	struct drm_i915_gem_exec_object2 *obj;
 	struct drm_i915_gem_relocation_entry reloc[3];
+
+	struct drm_xe_exec exec;
+	size_t bb_size;
+	struct xe_spin *spin;
+	struct drm_xe_sync *syncs;
+
 	uint32_t bb_handle;
 	uint32_t *bb_duration;
 };
 
+struct vm {
+	uint32_t id;
+	bool compute_mode;
+	uint64_t ahnd;
+};
+
+struct exec_queue {
+	uint32_t id;
+	uint32_t vm_idx; /* index in workload.vm_list */
+	struct drm_xe_engine_class_instance hwe;
+	bool compute_mode; /* vm should also be in compute mode */
+	/* timeout applied when compute_mode == false*/
+	uint32_t job_timeout_ms;
+	/* todo: preempt, timeslice and other props */
+
+	/* for qd_throttle */
+	unsigned int nrequest;
+	struct igt_list_head requests;
+};
+
 struct ctx {
 	uint32_t id;
 	int priority;
@@ -218,7 +262,12 @@ struct workload
 	unsigned int nr_ctxs;
 	struct ctx *ctx_list;
 
-	struct working_set **working_sets; /* array indexed by set id */
+	unsigned int nr_vms;
+	struct vm *vm_list;
+	unsigned int nr_eqs;
+	struct exec_queue *eq_list;
+
+	struct working_set **working_sets;
 	int max_working_set_id;
 
 	int sync_timeline;
@@ -228,18 +277,49 @@ struct workload
 	unsigned int nrequest[NUM_ENGINES];
 };
 
+#define for_each_exec_queue(__eq, __wrk) \
+	for (int __i = 0; __i < (__wrk)->nr_eqs && \
+	     (__eq = &(__wrk)->eq_list[__i]); ++__i)
+
+#define for_each_vm(__vm, __wrk) \
+	for (int __i = 0; __i < (__wrk)->nr_vms && \
+	     (__vm = &(__wrk)->vm_list[__i]); ++__i)
+
 static unsigned int master_prng;
 
 static int verbose = 1;
-static int fd;
+static int fd = -1;
 static struct drm_i915_gem_context_param_sseu device_sseu = {
 	.slice_mask = -1 /* Force read on first use. */
 };
 
+static bool is_xe;
+
 #define SYNCEDCLIENTS	(1<<1)
 #define DEPSYNC		(1<<2)
 #define SSEU		(1<<3)
 
+static void __attribute__((format(printf, 1, 2)))
+wsim_err(const char *fmt, ...)
+{
+	va_list ap;
+
+	if (!verbose)
+		return;
+
+	va_start(ap, fmt);
+	vfprintf(stderr, fmt, ap);
+	va_end(ap);
+}
+
+#define check_arg(cond, fmt, ...) \
+{ \
+	if (cond) { \
+		wsim_err(fmt, __VA_ARGS__); \
+		return NULL; \
+	} \
+}
+
 static const char *ring_str_map[NUM_ENGINES] = {
 	[DEFAULT] = "DEFAULT",
 	[RCS] = "RCS",
@@ -250,6 +330,14 @@ static const char *ring_str_map[NUM_ENGINES] = {
 	[VECS] = "VECS",
 };
 
+static void w_sync(int fd_, struct w_step *w)
+{
+	if (is_xe)
+		igt_assert(syncobj_wait(fd_, &w->syncs[0].handle, 1, INT64_MAX, 0, NULL));
+	else
+		gem_sync(fd_, w->obj[0].handle);
+}
+
 static int read_timestamp_frequency(int i915)
 {
 	int value = 0;
@@ -351,15 +439,23 @@ parse_dependency(unsigned int nr_steps, struct w_step *w, char *str)
 		if (entry.target > 0 || ((int)nr_steps + entry.target) < 0)
 			return -1;
 
-		add_dep(&w->data_deps, entry);
+		/* only fence deps in xe, let f-1 <==> -1 */
+		if (is_xe)
+			add_dep(&w->fence_deps, entry);
+		else
+			add_dep(&w->data_deps, entry);
 
 		break;
 	case 's':
-		submit_fence = true;
+		/* no submit fence in xe ? */
+		if (!is_xe)
+			submit_fence = true;
 		/* Fall-through. */
 	case 'f':
-		/* Multiple fences not yet supported. */
-		igt_assert_eq(w->fence_deps.nr, 0);
+		/* xe supports multiple fences */
+		if (!is_xe)
+			/* Multiple fences not yet supported. */
+			igt_assert_eq(w->fence_deps.nr, 0);
 
 		entry.target = atoi(++str);
 		if (entry.target > 0 || ((int)nr_steps + entry.target) < 0)
@@ -429,25 +525,120 @@ out:
 	return ret;
 }
 
-static void __attribute__((format(printf, 1, 2)))
-wsim_err(const char *fmt, ...)
+static long __duration(long dur, double scale)
 {
-	va_list ap;
+	return round(scale * dur);
+}
 
-	if (!verbose)
-		return;
+static int
+parse_duration(unsigned int nr_steps, struct duration *dur, double scale_dur, char *_desc)
+{
+	char *sep = NULL;
+	long int tmpl;
 
-	va_start(ap, fmt);
-	vfprintf(stderr, fmt, ap);
-	va_end(ap);
+	if (_desc[0] == '*') {
+		if (intel_gen(intel_get_drm_devid(fd)) < 8) {
+			wsim_err("Infinite batch at step %u needs Gen8+!\n", nr_steps);
+			return -1;
+		}
+		dur->unbound_duration = true;
+	} else {
+		tmpl = strtol(_desc, &sep, 10);
+		if (tmpl <= 0 || tmpl == LONG_MIN || tmpl == LONG_MAX) {
+			return -1;
+		}
+		dur->min = __duration(tmpl, scale_dur);
+
+		if (sep && *sep == '-') {
+			tmpl = strtol(sep + 1, NULL, 10);
+			if (tmpl <= 0 || __duration(tmpl, scale_dur) <= dur->min ||
+			    tmpl == LONG_MIN || tmpl == LONG_MAX) {
+				return -1;
+			}
+			dur->max = __duration(tmpl, scale_dur);
+		} else {
+			dur->max = dur->min;
+		}
+	}
+
+	return 0;
 }
 
-#define check_arg(cond, fmt, ...) \
-{ \
-	if (cond) { \
-		wsim_err(fmt, __VA_ARGS__); \
-		return NULL; \
-	} \
+/* v.compute_mode - 0 | 1 */
+static int
+parse_vm(unsigned int nr_steps, struct w_step *w, char *_desc)
+{
+	struct vm _vm = {};
+	char *field, *ctx = NULL;
+
+	/* skip v. part */
+	igt_assert(_desc && _desc[0] == 'v' && _desc[1] == '.');
+
+	if ((field = strtok_r(_desc + 2, ".", &ctx)))
+		_vm.compute_mode = (atoi(field) == 1);
+
+	w->vm = malloc(sizeof(_vm));
+	*w->vm = _vm;
+
+	return 0;
+}
+
+/* e.vm_idx.class.instance.compute_mode<0|1>.job_timeout_ms
+
+   class - int - corresponding to RCS, BCS, VCS, VECS, CCS
+   instance - int  -1 = virtual, >=0 instance id
+*/
+static int
+parse_exec_queue(unsigned int nr_steps, struct w_step *w, char *_desc)
+{
+	struct exec_queue eq = {};
+	int id;
+	char *field, *ctx = NULL;
+
+	/* skip e. part */
+	igt_assert(_desc && _desc[0] == 'e' && _desc[1] == '.');
+
+	/* vm_idx */
+	if ((field = strtok_r(_desc + 2, ".", &ctx)))
+		id = atoi(field);
+
+	if (id < 0) {
+		wsim_err("Invalid vm index at step %u!\n", nr_steps);
+		return -1;
+	}
+	eq.vm_idx = id;
+
+	/* class */
+	if ((field = strtok_r(0, ".", &ctx)))
+		id = atoi(field);
+
+	if (id < 0 || id > 255) {
+		wsim_err("Invalid engine class at step %u!\n", nr_steps);
+		return -1;
+	}
+	eq.hwe.engine_class = id;
+
+	/* instance -1 - virtual, >= 0 - instance id */
+	if ((field = strtok_r(0, ".", &ctx)))
+		id = atoi(field);
+
+	if (id < -1 || id > 255) {
+		wsim_err("Invalid engine instance at step %u!\n", nr_steps);
+		return -1;
+	}
+	eq.hwe.engine_instance = id;
+
+	if ((field = strtok_r(0, ".", &ctx)))
+		eq.compute_mode = (atoi(field) == 1);
+
+	/* 0 - default, > 0 timeout */
+	if ((field = strtok_r(0, ".", &ctx)))
+		eq.job_timeout_ms = atoi(field);
+
+	w->eq = malloc(sizeof(eq));
+	*w->eq = eq;
+
+	return 0;
 }
 
 static int str_to_engine(const char *str)
@@ -855,11 +1046,6 @@ static uint64_t engine_list_mask(const char *_str)
 static unsigned long
 allocate_working_set(struct workload *wrk, struct working_set *set);
 
-static long __duration(long dur, double scale)
-{
-	return round(scale * dur);
-}
-
 #define int_field(_STEP_, _FIELD_, _COND_, _ERR_) \
 	if ((field = strtok_r(fstart, ".", &fctx))) { \
 		tmp = atoi(field); \
@@ -895,14 +1081,42 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 		if ((field = strtok_r(fstart, ".", &fctx))) {
 			fstart = NULL;
 
-			if (!strcmp(field, "d")) {
+			/* line starting with # is a comment */
+			if (field[0] == '#') {
+				step.type = SKIP;
+				goto add_step;
+			}
+
+			if (!strcmp(field, "v")) {
+				tmp = parse_vm(nr_steps, &step, _token);
+				check_arg(tmp < 0, "Invalid vm at step %u!\n", nr_steps);
+				step.type = VM;
+				goto add_step;
+			} else if (!strcmp(field, "e")) {
+				tmp = parse_exec_queue(nr_steps, &step, _token);
+				check_arg(tmp < 0, "Invalid exec queue at step %u!\n", nr_steps);
+				step.type = EXEC_QUEUE;
+				goto add_step;
+			} else if (!strcmp(field, "d")) {
 				int_field(DELAY, delay, tmp <= 0,
 					  "Invalid delay at step %u!\n");
 			} else if (!strcmp(field, "p")) {
-				int_field(PERIOD, period, tmp <= 0,
-					  "Invalid period at step %u!\n");
+				/* not using int_field macro to handle scale_dur */
+				if ((field = strtok_r(fstart, ".", &fctx))) {
+					tmp = atoi(field);
+					check_arg(tmp <= 0, "Invalid period at step %u!\n", nr_steps);
+					step.type = PERIOD;
+					step.period = __duration(tmp, scale_dur);
+					goto add_step;
+				}
 			} else if (!strcmp(field, "P")) {
 				unsigned int nr = 0;
+
+				if (is_xe) {
+					step.type = SKIP;
+					goto add_step;
+				}
+
 				while ((field = strtok_r(fstart, ".", &fctx))) {
 					tmp = atoi(field);
 					check_arg(nr == 0 && tmp <= 0,
@@ -928,6 +1142,11 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 					  "Invalid sync target at step %u!\n");
 			} else if (!strcmp(field, "S")) {
 				unsigned int nr = 0;
+				if (is_xe) {
+					step.type = SKIP;
+					goto add_step;
+				}
+
 				while ((field = strtok_r(fstart, ".", &fctx))) {
 					tmp = atoi(field);
 					check_arg(tmp <= 0 && nr == 0,
@@ -964,6 +1183,10 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 				goto add_step;
 			} else if (!strcmp(field, "M")) {
 				unsigned int nr = 0;
+				if (is_xe) {
+					step.type = SKIP;
+					goto add_step;
+				}
 				while ((field = strtok_r(fstart, ".", &fctx))) {
 					tmp = atoi(field);
 					check_arg(nr == 0 && tmp <= 0,
@@ -996,7 +1219,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 				unsigned int nr = 0;
 				while ((field = strtok_r(fstart, ".", &fctx))) {
 					tmp = atoi(field);
-					check_arg(nr == 0 && tmp <= 0,
+					check_arg(nr == 0 && (is_xe ? tmp < 0 : tmp <= 0),
 						  "Invalid context at step %u!\n",
 						  nr_steps);
 					check_arg(nr == 1 && tmp < 0,
@@ -1018,6 +1241,10 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 				goto add_step;
 			} else if (!strcmp(field, "B")) {
 				unsigned int nr = 0;
+				if (is_xe) {
+					step.type = SKIP;
+					goto add_step;
+				}
 				while ((field = strtok_r(fstart, ".", &fctx))) {
 					tmp = atoi(field);
 					check_arg(nr == 0 && tmp <= 0,
@@ -1037,6 +1264,10 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 				goto add_step;
 			} else if (!strcmp(field, "b")) {
 				unsigned int nr = 0;
+				if (is_xe) {
+					step.type = SKIP;
+					goto add_step;
+				}
 				while ((field = strtok_r(fstart, ".", &fctx))) {
 					check_arg(nr > 2,
 						  "Invalid bond format at step %u!\n",
@@ -1101,19 +1332,22 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 			}
 
 			tmp = atoi(field);
-			check_arg(tmp < 0, "Invalid ctx id at step %u!\n",
+			check_arg(tmp < 0, "Invalid %s id at step %u!\n",
+				  (is_xe ? "exec queue" : "ctx"),
 				  nr_steps);
 			step.context = tmp;
+			step.eq_idx = tmp;
 
 			valid++;
 		}
 
-		if ((field = strtok_r(fstart, ".", &fctx))) {
+		/* engine desc in BATCH type is i915 specific */
+		if (!is_xe && (field = strtok_r(fstart, ".", &fctx))) {
 			fstart = NULL;
 
 			i = str_to_engine(field);
 			check_arg(i < 0,
-				  "Invalid engine id at step %u!\n", nr_steps);
+				"Invalid engine id at step %u!\n", nr_steps);
 
 			valid++;
 
@@ -1121,38 +1355,11 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 		}
 
 		if ((field = strtok_r(fstart, ".", &fctx))) {
-			char *sep = NULL;
-			long int tmpl;
-
 			fstart = NULL;
 
-			if (field[0] == '*') {
-				check_arg(intel_gen(intel_get_drm_devid(fd)) < 8,
-					  "Infinite batch at step %u needs Gen8+!\n",
-					  nr_steps);
-				step.unbound_duration = true;
-			} else {
-				tmpl = strtol(field, &sep, 10);
-				check_arg(tmpl <= 0 || tmpl == LONG_MIN ||
-					  tmpl == LONG_MAX,
-					  "Invalid duration at step %u!\n",
-					  nr_steps);
-				step.duration.min = __duration(tmpl, scale_dur);
-
-				if (sep && *sep == '-') {
-					tmpl = strtol(sep + 1, NULL, 10);
-					check_arg(tmpl <= 0 ||
-						tmpl <= step.duration.min ||
-						tmpl == LONG_MIN ||
-						tmpl == LONG_MAX,
-						"Invalid duration range at step %u!\n",
-						nr_steps);
-					step.duration.max = __duration(tmpl,
-								       scale_dur);
-				} else {
-					step.duration.max = step.duration.min;
-				}
-			}
+			tmp = parse_duration(nr_steps, &step.duration, scale_dur, field);
+			check_arg(tmp < 0,
+				  "Invalid duration at step %u!\n", nr_steps);
 
 			valid++;
 		}
@@ -1170,7 +1377,8 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 		if ((field = strtok_r(fstart, ".", &fctx))) {
 			fstart = NULL;
 
-			check_arg(strlen(field) != 1 ||
+			check_arg(!strlen(field) ||
+				  (strlen(field) > 1 && !isspace(field[1]) && field[1] != '#') ||
 				  (field[0] != '0' && field[0] != '1'),
 				  "Invalid wait boolean at step %u!\n",
 				  nr_steps);
@@ -1179,23 +1387,28 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
 			valid++;
 		}
 
-		check_arg(valid != 5, "Invalid record at step %u!\n", nr_steps);
+		check_arg(valid != (is_xe ? 4 : 5), "Invalid record at step %u!\n", nr_steps);
 
 		step.type = BATCH;
 
 add_step:
-		if (step.type == DELAY)
-			step.delay = __duration(step.delay, scale_time);
+		if (step.type == SKIP) {
+			if (verbose > 3)
+				printf("skipped STEP: %s\n", _token);
+		} else {
+			if (step.type == DELAY)
+				step.delay = __duration(step.delay, scale_time);
 
-		step.idx = nr_steps++;
-		step.request = -1;
-		steps = realloc(steps, sizeof(step) * nr_steps);
-		igt_assert(steps);
+			step.idx = nr_steps++;
+			step.request = -1;
+			steps = realloc(steps, sizeof(step) * nr_steps);
+			igt_assert(steps);
 
-		memcpy(&steps[nr_steps - 1], &step, sizeof(step));
+			memcpy(&steps[nr_steps - 1], &step, sizeof(step));
+		}
 
 		free(token);
-	}
+	} // while ((_token = strtok_r(tstart, ",", &tctx))) {
 
 	if (app_w) {
 		steps = realloc(steps, sizeof(step) *
@@ -1211,7 +1424,7 @@ add_step:
 		nr_steps += app_w->nr_steps;
 	}
 
-	wrk = malloc(sizeof(*wrk));
+	wrk = calloc(1, sizeof(*wrk));
 	igt_assert(wrk);
 
 	wrk->nr_steps = nr_steps;
@@ -1370,6 +1583,24 @@ __get_ctx(struct workload *wrk, const struct w_step *w)
 	return &wrk->ctx_list[w->context];
 }
 
+static struct exec_queue *
+get_eq(struct workload *wrk, const struct w_step *w)
+{
+	igt_assert(w->eq_idx < wrk->nr_eqs);
+
+	return &wrk->eq_list[w->eq_idx];
+}
+
+static struct vm *
+get_vm(struct workload *wrk, const struct w_step *w)
+{
+	uint32_t vm_idx = get_eq(wrk, w)->vm_idx;
+
+	igt_assert(vm_idx < wrk->nr_vms);
+
+	return &wrk->vm_list[vm_idx];
+}
+
 static uint32_t mmio_base(int i915, enum intel_engine_id engine, int gen)
 {
 	const char *name;
@@ -1554,7 +1785,7 @@ static uint32_t alloc_bo(int i915, unsigned long size)
 }
 
 static void
-alloc_step_batch(struct workload *wrk, struct w_step *w)
+i915_alloc_step_batch(struct workload *wrk, struct w_step *w)
 {
 	enum intel_engine_id engine = w->engine;
 	unsigned int j = 0;
@@ -1622,6 +1853,68 @@ alloc_step_batch(struct workload *wrk, struct w_step *w)
 #endif
 }
 
+static void
+xe_alloc_step_batch(struct workload *wrk, struct w_step *w)
+{
+	struct vm *vm = get_vm(wrk, w);
+	struct exec_queue *eq = get_eq(wrk, w);
+	struct dep_entry *dep;
+	int i;
+
+	w->bb_size = ALIGN(sizeof(*w->spin) + xe_cs_prefetch_size(fd), xe_get_default_alignment(fd));
+	w->bb_handle = xe_bo_create(fd, 0, vm->id, w->bb_size);
+	w->spin = xe_bo_map(fd, w->bb_handle, w->bb_size);
+	w->exec.address = intel_allocator_alloc_with_strategy(vm->ahnd, w->bb_handle, w->bb_size,
+							0, ALLOC_STRATEGY_LOW_TO_HIGH);
+	xe_vm_bind_sync(fd, vm->id, w->bb_handle, 0, w->exec.address, w->bb_size);
+	xe_spin_init_opts(w->spin, .addr = w->exec.address,
+				   .preempt = (w->preempt_us > 0),
+				   .ctx_ticks = duration_to_ctx_ticks(fd, eq->hwe.gt_id,
+								1000 * get_duration(wrk, w)));
+	w->exec.exec_queue_id = eq->id;
+	w->exec.num_batch_buffer = 1;
+	/* always at least one out fence */
+	w->exec.num_syncs = 1;
+	/* count syncs */
+	igt_assert_eq(0, w->data_deps.nr);
+	for_each_dep(dep, w->fence_deps) {
+		int dep_idx = w->idx + dep->target;
+
+		igt_assert(dep_idx >= 0 && dep_idx < w->idx);
+		igt_assert(wrk->steps[dep_idx].type == SW_FENCE ||
+			   wrk->steps[dep_idx].type == BATCH);
+
+		w->exec.num_syncs++;
+	}
+	w->syncs = calloc(w->exec.num_syncs, sizeof(*w->syncs));
+	/* fill syncs */
+	i = 0;
+	/* out fence */
+	w->syncs[i].handle = syncobj_create(fd, 0);
+	w->syncs[i++].flags = DRM_XE_SYNC_SYNCOBJ | DRM_XE_SYNC_SIGNAL;
+	/* in fence(s) */
+	for_each_dep(dep, w->fence_deps) {
+		int dep_idx = w->idx + dep->target;
+
+		igt_assert(wrk->steps[dep_idx].type == SW_FENCE ||
+			   wrk->steps[dep_idx].type == BATCH);
+		igt_assert(wrk->steps[dep_idx].syncs && wrk->steps[dep_idx].syncs[0].handle);
+
+		w->syncs[i].handle = wrk->steps[dep_idx].syncs[0].handle;
+		w->syncs[i++].flags = DRM_XE_SYNC_SYNCOBJ;
+	}
+	w->exec.syncs = to_user_pointer(w->syncs);
+}
+
+static void
+alloc_step_batch(struct workload *wrk, struct w_step *w)
+{
+	if (is_xe)
+		xe_alloc_step_batch(wrk, w);
+	else
+		i915_alloc_step_batch(wrk, w);
+}
+
 static bool set_priority(uint32_t ctx_id, int prio)
 {
 	struct drm_i915_gem_context_param param = {
@@ -1848,20 +2141,77 @@ static void measure_active_set(struct workload *wrk)
 
 #define alloca0(sz) ({ size_t sz__ = (sz); memset(alloca(sz__), 0, sz__); })
 
-static int prepare_workload(unsigned int id, struct workload *wrk)
+static int xe_prepare_vms_eqs(unsigned int id, struct workload *wrk)
+{
+	struct w_step *w;
+	int i, j;
+
+	/* Create vms - should be done before exec queues */
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		if (w->type != VM)
+			continue;
+		wrk->nr_vms++;
+	}
+	igt_assert(wrk->nr_vms);
+	wrk->vm_list = calloc(wrk->nr_vms, sizeof(struct vm));
+
+	for (j = 0 /*vm_idx*/, i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		struct vm *vm_;
+
+		if (w->type != VM)
+			continue;
+		vm_ = &wrk->vm_list[j];
+		*vm_ = *w->vm;
+		vm_->id = xe_vm_create(fd, 0 /*flags*/, 0 /*ext*/);
+		vm_->ahnd = intel_allocator_open(fd, vm_->id, INTEL_ALLOCATOR_RELOC);
+		j++;
+	}
+
+	/* Create exec queues */
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		if (w->type != EXEC_QUEUE)
+			continue;
+		wrk->nr_eqs++;
+	}
+	igt_assert(wrk->nr_eqs);
+	wrk->eq_list = calloc(wrk->nr_eqs, sizeof(struct exec_queue));
+
+	for (j = 0 /*eq_idx*/, i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		struct exec_queue *eq;
+		struct vm *vm_;
+
+		if (w->type != EXEC_QUEUE)
+			continue;
+		eq = &(wrk->eq_list[j]);
+		*eq = *w->eq;
+		vm_ = get_vm(wrk, w);
+		igt_assert(vm_);
+		igt_assert(eq->hwe.engine_instance >= 0);
+		eq->id = xe_exec_queue_create(fd, vm_->id, &eq->hwe, 0 /*ext*/);
+		/* init request list */
+		IGT_INIT_LIST_HEAD(&eq->requests);
+		eq->nrequest = 0;
+		j++;
+	}
+
+	/* create syncobjs for SW_FENCE */
+	for (j = 0, i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++)
+		if (w->type == SW_FENCE) {
+			w->syncs = calloc(1, sizeof(struct drm_xe_sync));
+			w->syncs[0].handle = syncobj_create(fd, 0);
+			w->syncs[0].flags = DRM_XE_SYNC_SYNCOBJ;
+		}
+
+	return 0;
+}
+
+static int i915_prepare_ctxs(unsigned int id, struct workload *wrk)
 {
-	struct working_set **sets;
-	unsigned long total = 0;
 	uint32_t share_vm = 0;
 	int max_ctx = -1;
 	struct w_step *w;
 	int i, j;
 
-	wrk->id = id;
-	wrk->bb_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
-	wrk->bo_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
-	wrk->run = true;
-
 	/*
 	 * Pre-scan workload steps to allocate context list storage.
 	 */
@@ -2050,6 +2400,25 @@ static int prepare_workload(unsigned int id, struct workload *wrk)
 	if (share_vm)
 		vm_destroy(fd, share_vm);
 
+	return 0;
+}
+
+static int prepare_workload(unsigned int id, struct workload *wrk)
+{
+	struct w_step *w;
+	int i, j;
+
+	wrk->id = id;
+	wrk->bb_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
+	wrk->bo_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
+	wrk->run = true;
+
+	if (is_xe) {
+		xe_prepare_vms_eqs(id, wrk);
+	} else {
+		i915_prepare_ctxs(id, wrk);
+	}
+
 	/* Record default preemption. */
 	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
 		if (w->type == BATCH)
@@ -2070,75 +2439,89 @@ static int prepare_workload(unsigned int id, struct workload *wrk)
 		for (j = i + 1; j < wrk->nr_steps; j++) {
 			w2 = &wrk->steps[j];
 
-			if (w2->context != w->context)
-				continue;
-			else if (w2->type == PREEMPTION)
-				break;
-			else if (w2->type != BATCH)
-				continue;
+			if (is_xe) {
+				if (w2->eq_idx != w->eq_idx)
+					continue;
+				else if (w2->type == PREEMPTION)
+					break;
+				else if (w2->type != BATCH)
+					continue;
+			} else {
+				if (w2->context != w->context)
+					continue;
+				else if (w2->type == PREEMPTION)
+					break;
+				else if (w2->type != BATCH)
+					continue;
+			}
 
 			w2->preempt_us = w->period;
 		}
 	}
 
-	/*
-	 * Scan for SSEU control steps.
-	 */
-	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
-		if (w->type == SSEU) {
-			get_device_sseu();
-			break;
+	if (!is_xe) {
+		struct working_set **sets;
+		unsigned long total = 0;
+
+		/*
+		* Scan for SSEU control steps.
+		*/
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			if (w->type == SSEU) {
+				get_device_sseu();
+				break;
+			}
 		}
-	}
 
-	/*
-	 * Allocate working sets.
-	 */
-	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
-		if (w->type == WORKINGSET && !w->working_set.shared)
-			total += allocate_working_set(wrk, &w->working_set);
-	}
+		/*
+		* Allocate working sets.
+		*/
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			if (w->type == WORKINGSET && !w->working_set.shared)
+				total += allocate_working_set(wrk, &w->working_set);
+		}
 
-	if (verbose > 2)
-		printf("%u: %lu bytes in working sets.\n", wrk->id, total);
+		if (verbose > 2)
+			printf("%u: %lu bytes in working sets.\n", wrk->id, total);
 
-	/*
-	 * Map of working set ids.
-	 */
-	wrk->max_working_set_id = -1;
-	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
-		if (w->type == WORKINGSET &&
-		    w->working_set.id > wrk->max_working_set_id)
-			wrk->max_working_set_id = w->working_set.id;
-	}
+		/*
+		* Map of working set ids.
+		*/
+		wrk->max_working_set_id = -1;
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			if (w->type == WORKINGSET &&
+			w->working_set.id > wrk->max_working_set_id)
+				wrk->max_working_set_id = w->working_set.id;
+		}
 
-	sets = wrk->working_sets;
-	wrk->working_sets = calloc(wrk->max_working_set_id + 1,
-				   sizeof(*wrk->working_sets));
-	igt_assert(wrk->working_sets);
+		sets = wrk->working_sets;
+		wrk->working_sets = calloc(wrk->max_working_set_id + 1,
+					sizeof(*wrk->working_sets));
+		igt_assert(wrk->working_sets);
 
-	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
-		struct working_set *set;
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			struct working_set *set;
 
-		if (w->type != WORKINGSET)
-			continue;
+			if (w->type != WORKINGSET)
+				continue;
 
-		if (!w->working_set.shared) {
-			set = &w->working_set;
-		} else {
-			igt_assert(sets);
+			if (!w->working_set.shared) {
+				set = &w->working_set;
+			} else {
+				igt_assert(sets);
 
-			set = sets[w->working_set.id];
-			igt_assert(set->shared);
-			igt_assert(set->sizes);
+				set = sets[w->working_set.id];
+				igt_assert(set->shared);
+				igt_assert(set->sizes);
+			}
+
+			wrk->working_sets[w->working_set.id] = set;
 		}
 
-		wrk->working_sets[w->working_set.id] = set;
+		if (sets)
+			free(sets);
 	}
 
-	if (sets)
-		free(sets);
-
 	/*
 	 * Allocate batch buffers.
 	 */
@@ -2149,7 +2532,9 @@ static int prepare_workload(unsigned int id, struct workload *wrk)
 		alloc_step_batch(wrk, w);
 	}
 
-	measure_active_set(wrk);
+	if (!is_xe) {
+		measure_active_set(wrk);
+	}
 
 	return 0;
 }
@@ -2172,7 +2557,7 @@ update_bb_start(struct workload *wrk, struct w_step *w)
 
 	/* ticks is inverted for MI_DO_COMPARE (less-than comparison) */
 	ticks = 0;
-	if (!w->unbound_duration)
+	if (!w->duration.unbound_duration)
 		ticks = ~ns_to_ctx_ticks(1000 * get_duration(wrk, w));
 
 	*w->bb_duration = ticks;
@@ -2193,7 +2578,32 @@ static void w_sync_to(struct workload *wrk, struct w_step *w, int target)
 	igt_assert(target < wrk->nr_steps);
 	igt_assert(wrk->steps[target].type == BATCH);
 
-	gem_sync(fd, wrk->steps[target].obj[0].handle);
+	w_sync(fd, &wrk->steps[target]);
+}
+
+static void do_xe_exec(struct workload *wrk, struct w_step *w)
+{
+	struct exec_queue *eq = get_eq(wrk, w);
+
+	igt_assert(w->emit_fence <= 0);
+	if (w->emit_fence == -1)
+		syncobj_reset(fd, &w->syncs[0].handle, 1);
+
+	/* update duration if random */
+	if (w->duration.max != w->duration.min)
+		xe_spin_init_opts(w->spin, .addr = w->exec.address,
+					   .preempt = (w->preempt_us > 0),
+					   .ctx_ticks = duration_to_ctx_ticks(fd, eq->hwe.gt_id,
+								1000LL * get_duration(wrk, w)));
+	xe_exec(fd, &w->exec);
+
+	/* for qd_throttle */
+	if (w->rq_link.prev != NULL || w->rq_link.next != NULL) {
+		igt_list_del(&w->rq_link);
+		eq->nrequest--;
+	}
+	igt_list_add_tail(&w->rq_link, &eq->requests);
+	eq->nrequest++;
 }
 
 static void
@@ -2252,7 +2662,7 @@ static void sync_deps(struct workload *wrk, struct w_step *w)
 		igt_assert(dep_idx >= 0 && dep_idx < w->idx);
 		igt_assert(wrk->steps[dep_idx].type == BATCH);
 
-		gem_sync(fd, wrk->steps[dep_idx].obj[0].handle);
+		w_sync(fd, &wrk->steps[dep_idx]);
 	}
 }
 
@@ -2280,6 +2690,8 @@ static void *run_workload(void *data)
 			enum intel_engine_id engine = w->engine;
 			int do_sleep = 0;
 
+			igt_assert(w->type != SKIP);
+
 			if (w->type == DELAY) {
 				do_sleep = w->delay;
 			} else if (w->type == PERIOD) {
@@ -2306,7 +2718,7 @@ static void *run_workload(void *data)
 
 				igt_assert(s_idx >= 0 && s_idx < i);
 				igt_assert(wrk->steps[s_idx].type == BATCH);
-				gem_sync(fd, wrk->steps[s_idx].obj[0].handle);
+				w_sync(fd, &wrk->steps[s_idx]);
 				continue;
 			} else if (w->type == THROTTLE) {
 				throttle = w->throttle;
@@ -2320,6 +2732,9 @@ static void *run_workload(void *data)
 					sw_sync_timeline_create_fence(wrk->sync_timeline,
 								      cur_seqno + w->idx);
 				igt_assert(w->emit_fence > 0);
+				if (is_xe)
+					/* Convert sync file to syncobj */
+					syncobj_import_sync_file(fd, w->syncs[0].handle, w->emit_fence);
 				continue;
 			} else if (w->type == SW_FENCE_SIGNAL) {
 				int tgt = w->idx + w->target;
@@ -2349,9 +2764,12 @@ static void *run_workload(void *data)
 
 				igt_assert(t_idx >= 0 && t_idx < i);
 				igt_assert(wrk->steps[t_idx].type == BATCH);
-				igt_assert(wrk->steps[t_idx].unbound_duration);
+				igt_assert(wrk->steps[t_idx].duration.unbound_duration);
 
-				*wrk->steps[t_idx].bb_duration = 0xffffffff;
+				if (is_xe)
+					xe_spin_end(wrk->steps[t_idx].spin);
+				else
+					*wrk->steps[t_idx].bb_duration = 0xffffffff;
 				__sync_synchronize();
 				continue;
 			} else if (w->type == SSEU) {
@@ -2365,7 +2783,9 @@ static void *run_workload(void *data)
 				   w->type == ENGINE_MAP ||
 				   w->type == LOAD_BALANCE ||
 				   w->type == BOND ||
-				   w->type == WORKINGSET) {
+				   w->type == WORKINGSET ||
+				   w->type == VM ||
+				   w->type == EXEC_QUEUE) {
 				   /* No action for these at execution time. */
 				continue;
 			}
@@ -2383,34 +2803,54 @@ static void *run_workload(void *data)
 			if (throttle > 0)
 				w_sync_to(wrk, w, i - throttle);
 
-			do_eb(wrk, w, engine);
+			if (is_xe)
+				do_xe_exec(wrk, w);
+			else {
+				do_eb(wrk, w, engine);
 
-			if (w->request != -1) {
-				igt_list_del(&w->rq_link);
-				wrk->nrequest[w->request]--;
+				if (w->request != -1) {
+					igt_list_del(&w->rq_link);
+					wrk->nrequest[w->request]--;
+				}
+				w->request = engine;
+				igt_list_add_tail(&w->rq_link, &wrk->requests[engine]);
+				wrk->nrequest[engine]++;
 			}
-			w->request = engine;
-			igt_list_add_tail(&w->rq_link, &wrk->requests[engine]);
-			wrk->nrequest[engine]++;
 
 			if (!wrk->run)
 				break;
 
 			if (w->sync)
-				gem_sync(fd, w->obj[0].handle);
+				w_sync(fd, w);
 
 			if (qd_throttle > 0) {
-				while (wrk->nrequest[engine] > qd_throttle) {
-					struct w_step *s;
+				if (is_xe) {
+					struct exec_queue *eq = get_eq(wrk, w);
 
-					s = igt_list_first_entry(&wrk->requests[engine],
-								 s, rq_link);
+					while (eq->nrequest > qd_throttle) {
+						struct w_step *s;
 
-					gem_sync(fd, s->obj[0].handle);
+						s = igt_list_first_entry(&eq->requests, s, rq_link);
 
-					s->request = -1;
-					igt_list_del(&s->rq_link);
-					wrk->nrequest[engine]--;
+						w_sync(fd, s);
+
+						igt_list_del(&s->rq_link);
+						eq->nrequest--;
+					}
+				} else {
+					while (wrk->nrequest[engine] > qd_throttle) {
+						struct w_step *s;
+
+						s = igt_list_first_entry(&wrk->requests[engine],
+									s, rq_link);
+
+						w_sync(fd, s);
+						// gem_sync(fd, s->obj[0].handle);
+
+						s->request = -1;
+						igt_list_del(&s->rq_link);
+						wrk->nrequest[engine]--;
+					}
 				}
 			}
 		}
@@ -2427,18 +2867,51 @@ static void *run_workload(void *data)
 		for (i = 0, w = wrk->steps; wrk->run && (i < wrk->nr_steps);
 		     i++, w++) {
 			if (w->emit_fence > 0) {
-				close(w->emit_fence);
-				w->emit_fence = -1;
+				if (is_xe) {
+					igt_assert(w->type == SW_FENCE);
+					close(w->emit_fence);
+					w->emit_fence = -1;
+					syncobj_reset(fd, &w->syncs[0].handle, 1);
+				} else {
+					close(w->emit_fence);
+					w->emit_fence = -1;
+				}
 			}
 		}
-	}
+	} // main loop
 
-	for (i = 0; i < NUM_ENGINES; i++) {
-		if (!wrk->nrequest[i])
-			continue;
+	if (is_xe) {
+		struct exec_queue *eq;
 
-		w = igt_list_last_entry(&wrk->requests[i], w, rq_link);
-		gem_sync(fd, w->obj[0].handle);
+		for_each_exec_queue(eq, wrk) {
+			if (eq->nrequest) {
+				w = igt_list_last_entry(&eq->requests, w, rq_link);
+				w_sync(fd, w);
+			}
+		}
+
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			if (w->type == BATCH) {
+				w_sync(fd, w);
+				syncobj_destroy(fd, w->syncs[0].handle);
+				free(w->syncs);
+				xe_vm_unbind_sync(fd, get_vm(wrk, w)->id, 0, w->exec.address, w->bb_size);
+				gem_munmap(w->spin, w->bb_size);
+				gem_close(fd, w->bb_handle);
+			} else if (w->type == SW_FENCE) {
+				syncobj_destroy(fd, w->syncs[0].handle);
+				free(w->syncs);
+			}
+		}
+	}
+	else {
+		for (i = 0; i < NUM_ENGINES; i++) {
+			if (!wrk->nrequest[i])
+				continue;
+
+			w = igt_list_last_entry(&wrk->requests[i], w, rq_link);
+			w_sync(fd, w);
+		}
 	}
 
 	clock_gettime(CLOCK_MONOTONIC, &t_end);
@@ -2460,6 +2933,21 @@ static void *run_workload(void *data)
 
 static void fini_workload(struct workload *wrk)
 {
+	if (is_xe) {
+		struct exec_queue *eq;
+		struct vm *vm_;
+
+		for_each_exec_queue(eq, wrk)
+			xe_exec_queue_destroy(fd, eq->id);
+		free(wrk->eq_list);
+		wrk->nr_eqs = 0;
+		for_each_vm(vm_, wrk) {
+			put_ahnd(vm_->ahnd);
+			xe_vm_destroy(fd, vm_->id);
+		}
+		free(wrk->vm_list);
+		wrk->nr_vms = 0;
+	}
 	free(wrk->steps);
 	free(wrk);
 }
@@ -2519,6 +3007,13 @@ static char *load_workload_descriptor(char *filename)
 	close(infd);
 
 	for (i = 0; i < len; i++) {
+		/* '#' starts comment till end of line */
+		if (buf[i] == '#')
+			/* replace ',' in comments to not break parsing */
+			while (++i < len && buf[i] != '\n')
+				if (buf[i] == ',')
+					buf[i] = ';';
+
 		if (buf[i] == '\n')
 			buf[i] = ',';
 	}
@@ -2562,7 +3057,7 @@ int main(int argc, char **argv)
 	int prio = 0;
 	double t;
 	int i, c, ret;
-	char *drm_dev;
+	char *drm_dev = NULL;
 
 	master_prng = time(NULL);
 
@@ -2660,8 +3155,12 @@ int main(int argc, char **argv)
 		ret = igt_device_find_first_i915_discrete_card(&card);
 		if (!ret)
 			ret = igt_device_find_integrated_card(&card);
+		if (!ret)
+			ret = igt_device_find_first_xe_discrete_card(&card);
+		if (!ret)
+			ret = igt_device_find_xe_integrated_card(&card);
 		if (!ret) {
-			wsim_err("No device filter specified and no i915 devices found!\n");
+			wsim_err("No device filter specified and no intel devices found!\n");
 			return EXIT_FAILURE;
 		}
 	}
@@ -2676,6 +3175,7 @@ int main(int argc, char **argv)
 	}
 
 	fd = open(drm_dev, O_RDWR);
+
 	if (fd < 0) {
 		wsim_err("Failed to open '%s'! (%s)\n",
 			 drm_dev, strerror(errno));
@@ -2684,6 +3184,10 @@ int main(int argc, char **argv)
 	if (verbose > 1)
 		printf("Using device %s\n", drm_dev);
 
+	is_xe = is_xe_device(fd);
+	if (is_xe)
+		xe_device_get(fd);
+
 	if (!nr_w_args) {
 		wsim_err("No workload descriptor(s)!\n");
 		goto err;
diff --git a/benchmarks/wsim/README b/benchmarks/wsim/README
index 8c71f2fe6..ddfefff47 100644
--- a/benchmarks/wsim/README
+++ b/benchmarks/wsim/README
@@ -1,6 +1,9 @@
 Workload descriptor format
 ==========================
 
+Lines starting with '#' are treated as comments (do not create work step).
+
+# i915
 ctx.engine.duration_us.dependency.wait,...
 <uint>.<str>.<uint>[-<uint>]|*.<int <= 0>[/<int <= 0>][...].<0|1>,...
 B.<uint>
@@ -11,6 +14,23 @@ b.<uint>.<str>[|<str>].<str>
 w|W.<uint>.<str>[/<str>]...
 f
 
+# xe
+Xe does not use context abstraction and adds additional work step types
+for VM (v.) and exec queue (e.) creation.
+Each v. and e. step creates array entry (in workload's VM and Exec Queue arrays).
+Batch step references the exec queue on which it is to be executed.
+Exec queue reference (eq_idx) is the index (0-based) in workload's exec queue array.
+VM reference (vm_idx) is the index (0-based) in workload's VM array.
+
+v.compute_mode
+v.<0|1>
+e.vm_idx.class.instance.compute_mode.job_timeout_ms,...
+e.<uint>.<uint 0=RCS,1=BCS,2=VCS,3=VECS,4=CCS>.<int>.<0|1>.<uint>,...
+eq_idx.duration_us.dependency.wait,...
+<uint>.<uint>[-<uint>]|*.<int <= 0>[/<int <= 0>][...].<0|1>,...
+d|p|s|t|q|a|T.<int>,...
+f
+
 For duration a range can be given from which a random value will be picked
 before every submit. Since this and seqno management requires CPU access to
 objects, care needs to be taken in order to ensure the submit queue is deep
@@ -27,21 +47,22 @@ Additional workload steps are also supported:
  'q' - Throttle to n max queue depth.
  'f' - Create a sync fence.
  'a' - Advance the previously created sync fence.
- 'B' - Turn on context load balancing.
- 'b' - Set up engine bonds.
- 'M' - Set up engine map.
- 'P' - Context priority.
- 'S' - Context SSEU configuration.
+ 'B' - Turn on context load balancing. (i915 only)
+ 'b' - Set up engine bonds. (i915 only)
+ 'M' - Set up engine map. (i915 only)
+ 'P' - Context priority. (i915 only)
+ 'S' - Context SSEU configuration. (i915 only)
  'T' - Terminate an infinite batch.
- 'w' - Working set. (See Working sets section.)
- 'W' - Shared working set.
- 'X' - Context preemption control.
+ 'w' - Working set. (See Working sets section.) (i915 only)
+ 'W' - Shared working set. (i915 only)
+ 'X' - Context preemption control. (i915 only)
 
 Engine ids: DEFAULT, RCS, BCS, VCS, VCS1, VCS2, VECS
 
 Example (leading spaces must not be present in the actual file):
 ----------------------------------------------------------------
 
+# i915
   1.VCS1.3000.0.1
   1.RCS.500-1000.-1.0
   1.RCS.3700.0.0
@@ -51,6 +72,25 @@ Example (leading spaces must not be present in the actual file):
   1.VCS2.600.-1.1
   p.16000
 
+# xe equivalent
+  #VM: v.compute_mode
+  v.0
+  #EXEC_QUEUE: e.vm_idx.class.intance.compute_mode.job_timeout_ms
+  e.0.2.0.0.0 # VCS1
+  e.0.0.0.0.0 # RCS
+  e.0.2.1.0.0 # VCS2
+  e.0.0.0.0.0 # second RCS exec queue
+  #BATCH: eq_idx.duration.dependency.wait
+  0.3000.0.1       # 1.VCS1.3000.0.1
+  1.500-1000.-1.0  # 1.RCS.500-1000.-1.0
+  3.3700.0.0       # 1.RCS.3700.0.0
+  1.1000.-2.1      # 1.RCS.1000.-2.0
+  2.2300.-2.0      # 1.VCS2.2300.-2.0
+  3.4700.-1.0      # 1.RCS.4700.-1.0
+  2.600.-1.1       # 1.VCS2.600.-1.1
+  p.16000
+
+
 The above workload described in human language works like this:
 
   1.   A batch is sent to the VCS1 engine which will be executing for 3ms on the
@@ -76,16 +116,30 @@ Multiple dependencies can be given separated by forward slashes.
 
 Example:
 
+# i915
   1.VCS1.3000.0.1
   1.RCS.3700.0.0
   1.VCS2.2300.-1/-2.0
 
+# xe
+  v.0
+  e.0.2.0.0.0
+  e.0.0.0.0.0
+  e.0.2.1.0.0.0
+  0.3000.0.1
+  1.3700.0.0
+  2.2300.-1/-2.0
+
 I this case the last step has a data dependency on both first and second steps.
 
 Batch durations can also be specified as infinite by using the '*' in the
 duration field. Such batches must be ended by the terminate command ('T')
 otherwise they will cause a GPU hang to be reported.
 
+Note: On Xe Batch dependencies are expressed with syncobjects,
+so there is no difference between f-1 and -1
+ex. 1.1000.-2.0 is same as 1.1000.f-2.0.
+
 Sync (fd) fences
 ----------------
 
@@ -114,6 +168,7 @@ VCS1 and VCS2 batches will have a sync fence dependency on the RCS batch.
 
 Example:
 
+# i915
   1.RCS.500-1000.0.0
   f
   2.VCS1.3000.f-1.0
@@ -123,13 +178,27 @@ Example:
   s.-4
   s.-4
 
+# xe equivalent
+  v.0
+  e.0.0.0.0.0    # RCS
+  e.0.2.0.0.0    # VCS1
+  e.0.2.1.0.0    # VCS2
+  0.500-1000.0.0
+  f
+  1.3000.f-1.0
+  2.3000.f-2.0
+  0.500-1000.0.1
+  a.-4
+  s.-4
+  s.-4
+
 VCS1 and VCS2 batches have an input sync fence dependecy on the standalone fence
 created at the second step. They are submitted ahead of time while still not
 runnable. When the second RCS batch completes the standalone fence is signaled
 which allows the two VCS batches to be executed. Finally we wait until the both
 VCS batches have completed before starting the (optional) next iteration.
 
-Submit fences
+Submit fences (i915 only?)
 -------------
 
 Submit fences are a type of input fence which are signalled when the originating
diff --git a/benchmarks/wsim/xe_cloud-gaming-60fps.wsim b/benchmarks/wsim/xe_cloud-gaming-60fps.wsim
new file mode 100644
index 000000000..9fdf15e27
--- /dev/null
+++ b/benchmarks/wsim/xe_cloud-gaming-60fps.wsim
@@ -0,0 +1,25 @@
+#w.1.10n8m
+#w.2.3n16m
+#1.RCS.500-1500.r1-0-4/w2-0.0
+#1.RCS.500-1500.r1-5-9/w2-1.0
+#1.RCS.500-1500.r2-0-1/w2-2.0
+#M.2.VCS
+#B.2
+#3.RCS.500-1500.r2-2.0
+#2.DEFAULT.2000-4000.-1.0
+#4.VCS1.250-750.-1.1
+#p.16667
+#
+#xe
+v.0
+e.0.0.0.0.0 # 1.RCS.500-1500.r1-0-4/w2-0.0
+e.0.2.0.0.0 # 2.DEFAULT.2000-4000.-1.0
+e.0.0.0.0.0 # 3.RCS.500-1500.r2-2.0
+e.0.2.1.0.0 # 4.VCS1.250-750.-1.1
+0.500-1500.0.0
+0.500-1500.0.0
+0.500-1500.0.0
+2.500-1500.-2.0 # #3.RCS.500-1500.r2-2.0
+1.2000-4000.-1.0
+3.250-750.-1.1
+p.16667
diff --git a/benchmarks/wsim/xe_example.wsim b/benchmarks/wsim/xe_example.wsim
new file mode 100644
index 000000000..3fa620932
--- /dev/null
+++ b/benchmarks/wsim/xe_example.wsim
@@ -0,0 +1,28 @@
+#i915
+#1.VCS1.3000.0.1
+#1.RCS.500-1000.-1.0
+#1.RCS.3700.0.0
+#1.RCS.1000.-2.0
+#1.VCS2.2300.-2.0
+#1.RCS.4700.-1.0
+#1.VCS2.600.-1.1
+#p.16000
+#
+#xe
+#
+#VM: v.compute_mode
+v.0
+#EXEC_QUEUE: e.vm_idx.class.intance.compute_mode.job_timeout_ms
+e.0.2.0.0.0 # VCS1
+e.0.0.0.0.0 # RCS
+e.0.2.1.0.0 # VCS2
+e.0.0.0.0.0 # second RCS exec_queue
+#BATCH: eq_idx.duration.dependency.wait
+0.3000.0.1       # 1.VCS1.3000.0.1
+1.500-1000.-1.0  # 1.RCS.500-1000.-1.0
+3.3700.0.0       # 1.RCS.3700.0.0
+1.1000.-2.1      # 1.RCS.1000.-2.0
+2.2300.-2.0      # 1.VCS2.2300.-2.0
+3.4700.-1.0      # 1.RCS.4700.-1.0
+2.600.-1.1       # 1.VCS2.600.-1.1
+p.16000
diff --git a/benchmarks/wsim/xe_example01.wsim b/benchmarks/wsim/xe_example01.wsim
new file mode 100644
index 000000000..496905371
--- /dev/null
+++ b/benchmarks/wsim/xe_example01.wsim
@@ -0,0 +1,19 @@
+#VM: v.compute_mode
+v.0
+#EXEC_QUEUE: e.vm_idx.class.intance.compute_mode.job_timeout_ms
+e.0.0.0.0.0
+e.0.2.0.0.0
+e.0.1.0.0.0
+#BATCH: eq_idx.duration.dependency.wait
+# B1 - 10ms batch on BCS0
+2.10000.0.0
+# B2 - 10ms batch on RCS0; waits on B1
+0.10000.0.0
+# B3 - 10ms batch on VECS0; waits on B2
+1.10000.0.0
+# B4 - 10ms batch on BCS0
+2.10000.0.0
+# B5 - 10ms batch on RCS0; waits on B4
+0.10000.-1.0
+# B6 - 10ms batch on VECS0; waits on B5; wait on batch fence out
+1.10000.-1.1
diff --git a/benchmarks/wsim/xe_example_fence.wsim b/benchmarks/wsim/xe_example_fence.wsim
new file mode 100644
index 000000000..4f810d64e
--- /dev/null
+++ b/benchmarks/wsim/xe_example_fence.wsim
@@ -0,0 +1,23 @@
+#i915
+#1.RCS.500-1000.0.0
+#f
+#2.VCS1.3000.f-1.0
+#2.VCS2.3000.f-2.0
+#1.RCS.500-1000.0.1
+#a.-4
+#s.-4
+#s.-4
+#
+#xe
+v.0
+e.0.0.0.0.0
+e.0.2.0.0.0
+e.0.2.1.0.0
+0.500-1000.0.0
+f
+1.3000.f-1.0
+2.3000.f-2.0
+0.500-1000.0.1
+a.-4
+s.-4
+s.-4
diff --git a/benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim b/benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim
new file mode 100644
index 000000000..2214914eb
--- /dev/null
+++ b/benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim
@@ -0,0 +1,63 @@
+# https://lore.kernel.org/dri-devel/a443495f-5d1b-52e1-9b2f-80167deb6d57@linux.intel.com/
+#i915
+#M.3.VCS
+#B.3
+#1.VCS1.1200-1800.0.0
+#1.VCS1.1900-2100.0.0
+#2.RCS.1500-2000.-1.0
+#3.VCS.1400-1800.-1.1
+#1.VCS1.1900-2100.-1.0
+#2.RCS.1500-2000.-1.0
+#3.VCS.1400-1800.-1.1
+#1.VCS1.1900-2100.-1.0
+#2.RCS.200-400.-1.0
+#2.RCS.1500-2000.0.0
+#3.VCS.1400-1800.-1.1
+#1.VCS1.1900-2100.-1.0
+#2.RCS.1500-2000.-1.0
+#3.VCS.1400-1800.-1.1
+#1.VCS1.1900-2100.-1.0
+#2.RCS.200-400.-1.0
+#2.RCS.1500-2000.0.0
+#3.VCS.1400-1800.-1.1
+#1.VCS1.1900-2100.-1.0
+#2.RCS.1500-2000.-1.0
+#3.VCS.1400-1800.-1.1
+#1.VCS1.1900-2100.-1.0
+#2.RCS.1500-2000.-1.0
+#2.RCS.1500-2000.0.0
+#3.VCS.1400-1800.-1.1
+#
+#xe
+#
+#M.3.VCS ??
+#B.3     ??
+v.0
+e.0.2.0.0.0 # 1.VCS1
+e.0.0.0.0.0 # 2.RCS
+e.0.2.1.0.0 # 3.VCS - no load balancing yet always VCS2
+0.1200-1800.0.0
+0.1900-2100.0.0
+1.1500-2000.-1.0
+2.1400-1800.-1.1
+0.1900-2100.-1.0
+1.1500-2000.-1.0
+2.1400-1800.-1.1
+0.1900-2100.-1.0
+1.200-400.-1.0
+1.1500-2000.0.0
+2.1400-1800.-1.1
+0.1900-2100.-1.0
+1.1500-2000.-1.0
+2.1400-1800.-1.1
+0.1900-2100.-1.0
+1.200-400.-1.0
+1.1500-2000.0.0
+2.1400-1800.-1.1
+0.1900-2100.-1.0
+1.1500-2000.-1.0
+2.1400-1800.-1.1
+0.1900-2100.-1.0
+1.1500-2000.-1.0
+1.1500-2000.0.0
+2.1400-1800.-1.1
-- 
2.30.2

Re: [igt-dev] [PATCH i-g-t 3/3] [RFC] benchmarks/gem_wsim: added basic xe support

[Kamil Konieczny]

Hi Marcin,

I only looked over your code, see some nits below.

On 2023-08-25 at 13:19:13 +0000, Marcin Bernatowicz wrote:
> Added basic xe support with few examples.
> Single binary handles both i915 and Xe devices,
> but workload definitions differs between i915 and xe.
> Xe does not use context abstraction, introduces new VM and Exec Queue
> steps and BATCH step references exec queue.
> For more details see wsim/README.
> Some functionality is still missing: working sets,
> load balancing (need some input if/how to do it in Xe - exec queues
> width?).
> 
> The tool is handy for scheduling tests, we find it useful to verify vGPU
> profiles defining different execution quantum/preemption timeout settings.
> 
> There is also some rationale for the tool in following thread:
> https://lore.kernel.org/dri-devel/a443495f-5d1b-52e1-9b2f-80167deb6d57@linux.intel.com/
> 
> With this patch it should be possible to run following on xe device:
> 
> gem_wsim -w benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim -c 36 -r 600
> 
> Best with drm debug logs disabled:
> 
> echo 0 > /sys/module/drm/parameters/debug
> 
> Signed-off-by: Marcin Bernatowicz <marcin.bernatowicz@linux.intel.com>
> ---
>  benchmarks/gem_wsim.c                         | 842 ++++++++++++++----
>  benchmarks/wsim/README                        |  87 +-
>  benchmarks/wsim/xe_cloud-gaming-60fps.wsim    |  25 +
>  benchmarks/wsim/xe_example.wsim               |  28 +
>  benchmarks/wsim/xe_example01.wsim             |  19 +
>  benchmarks/wsim/xe_example_fence.wsim         |  23 +
>  .../wsim/xe_media_load_balance_fhd26u7.wsim   |  63 ++
>  7 files changed, 909 insertions(+), 178 deletions(-)
>  create mode 100644 benchmarks/wsim/xe_cloud-gaming-60fps.wsim
>  create mode 100644 benchmarks/wsim/xe_example.wsim
>  create mode 100644 benchmarks/wsim/xe_example01.wsim
>  create mode 100644 benchmarks/wsim/xe_example_fence.wsim
>  create mode 100644 benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim
> 
> diff --git a/benchmarks/gem_wsim.c b/benchmarks/gem_wsim.c
> index 7b5e62a3b..a9dcb7e9f 100644
> --- a/benchmarks/gem_wsim.c
> +++ b/benchmarks/gem_wsim.c
> @@ -42,6 +42,7 @@
>  #include <limits.h>
>  #include <pthread.h>
>  #include <math.h>
> +#include <ctype.h>
------------ ^
Put this in alphabetical order, maybe some separate cleanup?

>  
>  #include "drm.h"
>  #include "drmtest.h"
> @@ -60,6 +61,12 @@
>  #include "i915/gem_engine_topology.h"
>  #include "i915/gem_mman.h"
>  
> +#include "igt_syncobj.h"
> +#include "intel_allocator.h"
> +#include "xe_drm.h"
> +#include "xe/xe_ioctl.h"
> +#include "xe/xe_spin.h"
> +
>  enum intel_engine_id {
>  	DEFAULT,
>  	RCS,
> @@ -73,6 +80,7 @@ enum intel_engine_id {
>  
>  struct duration {
>  	unsigned int min, max;
> +	bool unbound_duration;
-------- ^
imho this should go in separate patch.

>  };
>  
>  enum w_type
> @@ -93,6 +101,9 @@ enum w_type
>  	TERMINATE,
>  	SSEU,
>  	WORKINGSET,
> +	VM,
> +	EXEC_QUEUE,
> +	SKIP,

imho this SKIP should also go in separate patch.

>  };
>  
>  struct dep_entry {
> @@ -108,6 +119,10 @@ struct deps
>  	struct dep_entry *list;
>  };
>  
> +#define for_each_dep(__dep, __deps) \
> +	for (int __i = 0; __i < __deps.nr && \
> +	     (__dep = &__deps.list[__i]); ++__i)
> +
>  struct w_arg {
>  	char *filename;
>  	char *desc;
> @@ -144,18 +159,18 @@ struct w_step
>  	enum w_type type;
>  	unsigned int context;
>  	unsigned int engine;
> +	unsigned int eq_idx;
>  	struct duration duration;
> -	bool unbound_duration;
>  	struct deps data_deps;
>  	struct deps fence_deps;
>  	int emit_fence;
> +

Looks like cleanup.

>  	union {
>  		int sync;
>  		int delay;
>  		int period;
>  		int target;
>  		int throttle;
> -		int fence_signal;
>  		int priority;
>  		struct {
>  			unsigned int engine_map_count;
> @@ -168,21 +183,50 @@ struct w_step
>  		};
>  		int sseu;
>  		struct working_set working_set;
> +		struct vm *vm;
> +		struct exec_queue *eq;
>  	};
>  
>  	/* Implementation details */
>  	unsigned int idx;
>  	struct igt_list_head rq_link;
> +
>  	unsigned int request;
>  	unsigned int preempt_us;
>  
>  	struct drm_i915_gem_execbuffer2 eb;
>  	struct drm_i915_gem_exec_object2 *obj;
>  	struct drm_i915_gem_relocation_entry reloc[3];
> +
> +	struct drm_xe_exec exec;
> +	size_t bb_size;
> +	struct xe_spin *spin;
> +	struct drm_xe_sync *syncs;
> +
>  	uint32_t bb_handle;
>  	uint32_t *bb_duration;
>  };
>  
> +struct vm {
> +	uint32_t id;
> +	bool compute_mode;
> +	uint64_t ahnd;
> +};
> +
> +struct exec_queue {
> +	uint32_t id;
> +	uint32_t vm_idx; /* index in workload.vm_list */
> +	struct drm_xe_engine_class_instance hwe;
> +	bool compute_mode; /* vm should also be in compute mode */
> +	/* timeout applied when compute_mode == false*/
> +	uint32_t job_timeout_ms;
> +	/* todo: preempt, timeslice and other props */
> +
> +	/* for qd_throttle */
> +	unsigned int nrequest;
> +	struct igt_list_head requests;
> +};
> +
>  struct ctx {
>  	uint32_t id;
>  	int priority;
> @@ -218,7 +262,12 @@ struct workload
>  	unsigned int nr_ctxs;
>  	struct ctx *ctx_list;
>  
> -	struct working_set **working_sets; /* array indexed by set id */
> +	unsigned int nr_vms;
> +	struct vm *vm_list;
> +	unsigned int nr_eqs;
> +	struct exec_queue *eq_list;
> +
> +	struct working_set **working_sets;
>  	int max_working_set_id;
>  
>  	int sync_timeline;
> @@ -228,18 +277,49 @@ struct workload
>  	unsigned int nrequest[NUM_ENGINES];
>  };
>  
> +#define for_each_exec_queue(__eq, __wrk) \
> +	for (int __i = 0; __i < (__wrk)->nr_eqs && \
> +	     (__eq = &(__wrk)->eq_list[__i]); ++__i)
> +
> +#define for_each_vm(__vm, __wrk) \
> +	for (int __i = 0; __i < (__wrk)->nr_vms && \
> +	     (__vm = &(__wrk)->vm_list[__i]); ++__i)
> +
>  static unsigned int master_prng;
>  
>  static int verbose = 1;
> -static int fd;
> +static int fd = -1;

Cleanup.

>  static struct drm_i915_gem_context_param_sseu device_sseu = {
>  	.slice_mask = -1 /* Force read on first use. */
>  };
>  
> +static bool is_xe;
> +
>  #define SYNCEDCLIENTS	(1<<1)
>  #define DEPSYNC		(1<<2)
>  #define SSEU		(1<<3)
>  
> +static void __attribute__((format(printf, 1, 2)))
> +wsim_err(const char *fmt, ...)
> +{
> +	va_list ap;
> +
> +	if (!verbose)
> +		return;
> +
> +	va_start(ap, fmt);
> +	vfprintf(stderr, fmt, ap);
> +	va_end(ap);
> +}
> +
> +#define check_arg(cond, fmt, ...) \
> +{ \
> +	if (cond) { \
> +		wsim_err(fmt, __VA_ARGS__); \
> +		return NULL; \
> +	} \
> +}
> +
>  static const char *ring_str_map[NUM_ENGINES] = {
>  	[DEFAULT] = "DEFAULT",
>  	[RCS] = "RCS",
> @@ -250,6 +330,14 @@ static const char *ring_str_map[NUM_ENGINES] = {
>  	[VECS] = "VECS",
>  };
>  
> +static void w_sync(int fd_, struct w_step *w)
> +{
> +	if (is_xe)
> +		igt_assert(syncobj_wait(fd_, &w->syncs[0].handle, 1, INT64_MAX, 0, NULL));
> +	else
> +		gem_sync(fd_, w->obj[0].handle);
> +}
> +
>  static int read_timestamp_frequency(int i915)
>  {
>  	int value = 0;
> @@ -351,15 +439,23 @@ parse_dependency(unsigned int nr_steps, struct w_step *w, char *str)
>  		if (entry.target > 0 || ((int)nr_steps + entry.target) < 0)
>  			return -1;
>  
> -		add_dep(&w->data_deps, entry);
> +		/* only fence deps in xe, let f-1 <==> -1 */
> +		if (is_xe)
> +			add_dep(&w->fence_deps, entry);
> +		else
> +			add_dep(&w->data_deps, entry);
>  
>  		break;
>  	case 's':
> -		submit_fence = true;
> +		/* no submit fence in xe ? */
> +		if (!is_xe)
> +			submit_fence = true;
>  		/* Fall-through. */
>  	case 'f':
> -		/* Multiple fences not yet supported. */
> -		igt_assert_eq(w->fence_deps.nr, 0);
> +		/* xe supports multiple fences */
> +		if (!is_xe)
> +			/* Multiple fences not yet supported. */
> +			igt_assert_eq(w->fence_deps.nr, 0);
>  
>  		entry.target = atoi(++str);
>  		if (entry.target > 0 || ((int)nr_steps + entry.target) < 0)
> @@ -429,25 +525,120 @@ out:
>  	return ret;
>  }
>  
> -static void __attribute__((format(printf, 1, 2)))
> -wsim_err(const char *fmt, ...)
> +static long __duration(long dur, double scale)
>  {
> -	va_list ap;
> +	return round(scale * dur);
> +}
>  
> -	if (!verbose)
> -		return;
> +static int
> +parse_duration(unsigned int nr_steps, struct duration *dur, double scale_dur, char *_desc)
> +{
> +	char *sep = NULL;
> +	long int tmpl;
>  
> -	va_start(ap, fmt);
> -	vfprintf(stderr, fmt, ap);
> -	va_end(ap);
> +	if (_desc[0] == '*') {
> +		if (intel_gen(intel_get_drm_devid(fd)) < 8) {
> +			wsim_err("Infinite batch at step %u needs Gen8+!\n", nr_steps);
> +			return -1;
> +		}
> +		dur->unbound_duration = true;
> +	} else {
> +		tmpl = strtol(_desc, &sep, 10);
> +		if (tmpl <= 0 || tmpl == LONG_MIN || tmpl == LONG_MAX) {
> +			return -1;
> +		}
> +		dur->min = __duration(tmpl, scale_dur);
> +
> +		if (sep && *sep == '-') {
> +			tmpl = strtol(sep + 1, NULL, 10);
> +			if (tmpl <= 0 || __duration(tmpl, scale_dur) <= dur->min ||
> +			    tmpl == LONG_MIN || tmpl == LONG_MAX) {
> +				return -1;
> +			}
> +			dur->max = __duration(tmpl, scale_dur);
> +		} else {
> +			dur->max = dur->min;
> +		}
> +	}
> +
> +	return 0;
>  }
>  
> -#define check_arg(cond, fmt, ...) \
> -{ \
> -	if (cond) { \
> -		wsim_err(fmt, __VA_ARGS__); \
> -		return NULL; \
> -	} \
> +/* v.compute_mode - 0 | 1 */
> +static int
> +parse_vm(unsigned int nr_steps, struct w_step *w, char *_desc)
> +{
> +	struct vm _vm = {};
> +	char *field, *ctx = NULL;
> +
> +	/* skip v. part */
> +	igt_assert(_desc && _desc[0] == 'v' && _desc[1] == '.');
> +
> +	if ((field = strtok_r(_desc + 2, ".", &ctx)))
> +		_vm.compute_mode = (atoi(field) == 1);
> +
> +	w->vm = malloc(sizeof(_vm));
> +	*w->vm = _vm;
> +
> +	return 0;
> +}
> +
> +/* e.vm_idx.class.instance.compute_mode<0|1>.job_timeout_ms
> +
> +   class - int - corresponding to RCS, BCS, VCS, VECS, CCS
> +   instance - int  -1 = virtual, >=0 instance id
> +*/
> +static int
> +parse_exec_queue(unsigned int nr_steps, struct w_step *w, char *_desc)
> +{
> +	struct exec_queue eq = {};
> +	int id;
> +	char *field, *ctx = NULL;
> +
> +	/* skip e. part */
> +	igt_assert(_desc && _desc[0] == 'e' && _desc[1] == '.');
> +
> +	/* vm_idx */
> +	if ((field = strtok_r(_desc + 2, ".", &ctx)))
> +		id = atoi(field);
> +
> +	if (id < 0) {
> +		wsim_err("Invalid vm index at step %u!\n", nr_steps);
> +		return -1;
> +	}
> +	eq.vm_idx = id;
> +
> +	/* class */
> +	if ((field = strtok_r(0, ".", &ctx)))
> +		id = atoi(field);
> +
> +	if (id < 0 || id > 255) {
> +		wsim_err("Invalid engine class at step %u!\n", nr_steps);
> +		return -1;
> +	}
> +	eq.hwe.engine_class = id;
> +
> +	/* instance -1 - virtual, >= 0 - instance id */
> +	if ((field = strtok_r(0, ".", &ctx)))
> +		id = atoi(field);
> +
> +	if (id < -1 || id > 255) {
> +		wsim_err("Invalid engine instance at step %u!\n", nr_steps);
> +		return -1;
> +	}
> +	eq.hwe.engine_instance = id;
> +
> +	if ((field = strtok_r(0, ".", &ctx)))
> +		eq.compute_mode = (atoi(field) == 1);
> +
> +	/* 0 - default, > 0 timeout */
> +	if ((field = strtok_r(0, ".", &ctx)))
> +		eq.job_timeout_ms = atoi(field);
> +
> +	w->eq = malloc(sizeof(eq));
> +	*w->eq = eq;
> +
> +	return 0;
>  }
>  
>  static int str_to_engine(const char *str)
> @@ -855,11 +1046,6 @@ static uint64_t engine_list_mask(const char *_str)
>  static unsigned long
>  allocate_working_set(struct workload *wrk, struct working_set *set);
>  
> -static long __duration(long dur, double scale)
> -{
> -	return round(scale * dur);
> -}
> -
>  #define int_field(_STEP_, _FIELD_, _COND_, _ERR_) \
>  	if ((field = strtok_r(fstart, ".", &fctx))) { \
>  		tmp = atoi(field); \
> @@ -895,14 +1081,42 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  		if ((field = strtok_r(fstart, ".", &fctx))) {
>  			fstart = NULL;
>  
> -			if (!strcmp(field, "d")) {
> +			/* line starting with # is a comment */
> +			if (field[0] == '#') {
> +				step.type = SKIP;
> +				goto add_step;
> +			}
> +
> +			if (!strcmp(field, "v")) {
> +				tmp = parse_vm(nr_steps, &step, _token);
> +				check_arg(tmp < 0, "Invalid vm at step %u!\n", nr_steps);
> +				step.type = VM;
> +				goto add_step;
> +			} else if (!strcmp(field, "e")) {
> +				tmp = parse_exec_queue(nr_steps, &step, _token);
> +				check_arg(tmp < 0, "Invalid exec queue at step %u!\n", nr_steps);
> +				step.type = EXEC_QUEUE;
> +				goto add_step;
> +			} else if (!strcmp(field, "d")) {
>  				int_field(DELAY, delay, tmp <= 0,
>  					  "Invalid delay at step %u!\n");
>  			} else if (!strcmp(field, "p")) {
> -				int_field(PERIOD, period, tmp <= 0,
> -					  "Invalid period at step %u!\n");
> +				/* not using int_field macro to handle scale_dur */
> +				if ((field = strtok_r(fstart, ".", &fctx))) {
> +					tmp = atoi(field);
> +					check_arg(tmp <= 0, "Invalid period at step %u!\n", nr_steps);
> +					step.type = PERIOD;
> +					step.period = __duration(tmp, scale_dur);
> +					goto add_step;
> +				}
>  			} else if (!strcmp(field, "P")) {
>  				unsigned int nr = 0;
> +
> +				if (is_xe) {
> +					step.type = SKIP;
> +					goto add_step;
> +				}
> +
>  				while ((field = strtok_r(fstart, ".", &fctx))) {
>  					tmp = atoi(field);
>  					check_arg(nr == 0 && tmp <= 0,
> @@ -928,6 +1142,11 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  					  "Invalid sync target at step %u!\n");
>  			} else if (!strcmp(field, "S")) {
>  				unsigned int nr = 0;
> +				if (is_xe) {
> +					step.type = SKIP;
> +					goto add_step;
> +				}
> +
>  				while ((field = strtok_r(fstart, ".", &fctx))) {
>  					tmp = atoi(field);
>  					check_arg(tmp <= 0 && nr == 0,
> @@ -964,6 +1183,10 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  				goto add_step;
>  			} else if (!strcmp(field, "M")) {
>  				unsigned int nr = 0;
> +				if (is_xe) {
> +					step.type = SKIP;
> +					goto add_step;
> +				}
>  				while ((field = strtok_r(fstart, ".", &fctx))) {
>  					tmp = atoi(field);
>  					check_arg(nr == 0 && tmp <= 0,
> @@ -996,7 +1219,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  				unsigned int nr = 0;
>  				while ((field = strtok_r(fstart, ".", &fctx))) {
>  					tmp = atoi(field);
> -					check_arg(nr == 0 && tmp <= 0,
> +					check_arg(nr == 0 && (is_xe ? tmp < 0 : tmp <= 0),
>  						  "Invalid context at step %u!\n",
>  						  nr_steps);
>  					check_arg(nr == 1 && tmp < 0,
> @@ -1018,6 +1241,10 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  				goto add_step;
>  			} else if (!strcmp(field, "B")) {
>  				unsigned int nr = 0;
> +				if (is_xe) {
> +					step.type = SKIP;
> +					goto add_step;
> +				}
>  				while ((field = strtok_r(fstart, ".", &fctx))) {
>  					tmp = atoi(field);
>  					check_arg(nr == 0 && tmp <= 0,
> @@ -1037,6 +1264,10 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  				goto add_step;
>  			} else if (!strcmp(field, "b")) {
>  				unsigned int nr = 0;
> +				if (is_xe) {
> +					step.type = SKIP;
> +					goto add_step;
> +				}
>  				while ((field = strtok_r(fstart, ".", &fctx))) {
>  					check_arg(nr > 2,
>  						  "Invalid bond format at step %u!\n",
> @@ -1101,19 +1332,22 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  			}
>  
>  			tmp = atoi(field);
> -			check_arg(tmp < 0, "Invalid ctx id at step %u!\n",
> +			check_arg(tmp < 0, "Invalid %s id at step %u!\n",
> +				  (is_xe ? "exec queue" : "ctx"),
>  				  nr_steps);
>  			step.context = tmp;
> +			step.eq_idx = tmp;
>  
>  			valid++;
>  		}
>  
> -		if ((field = strtok_r(fstart, ".", &fctx))) {
> +		/* engine desc in BATCH type is i915 specific */
> +		if (!is_xe && (field = strtok_r(fstart, ".", &fctx))) {
>  			fstart = NULL;
>  
>  			i = str_to_engine(field);
>  			check_arg(i < 0,
> -				  "Invalid engine id at step %u!\n", nr_steps);
> +				"Invalid engine id at step %u!\n", nr_steps);
>  
>  			valid++;
>  
> @@ -1121,38 +1355,11 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  		}
>  
>  		if ((field = strtok_r(fstart, ".", &fctx))) {
> -			char *sep = NULL;
> -			long int tmpl;
> -
>  			fstart = NULL;
>  
> -			if (field[0] == '*') {
> -				check_arg(intel_gen(intel_get_drm_devid(fd)) < 8,
> -					  "Infinite batch at step %u needs Gen8+!\n",
> -					  nr_steps);
> -				step.unbound_duration = true;
> -			} else {
> -				tmpl = strtol(field, &sep, 10);
> -				check_arg(tmpl <= 0 || tmpl == LONG_MIN ||
> -					  tmpl == LONG_MAX,
> -					  "Invalid duration at step %u!\n",
> -					  nr_steps);
> -				step.duration.min = __duration(tmpl, scale_dur);
> -
> -				if (sep && *sep == '-') {
> -					tmpl = strtol(sep + 1, NULL, 10);
> -					check_arg(tmpl <= 0 ||
> -						tmpl <= step.duration.min ||
> -						tmpl == LONG_MIN ||
> -						tmpl == LONG_MAX,
> -						"Invalid duration range at step %u!\n",
> -						nr_steps);
> -					step.duration.max = __duration(tmpl,
> -								       scale_dur);
> -				} else {
> -					step.duration.max = step.duration.min;
> -				}
> -			}
> +			tmp = parse_duration(nr_steps, &step.duration, scale_dur, field);
> +			check_arg(tmp < 0,
> +				  "Invalid duration at step %u!\n", nr_steps);
>  
>  			valid++;
>  		}
> @@ -1170,7 +1377,8 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  		if ((field = strtok_r(fstart, ".", &fctx))) {
>  			fstart = NULL;
>  
> -			check_arg(strlen(field) != 1 ||
> +			check_arg(!strlen(field) ||
> +				  (strlen(field) > 1 && !isspace(field[1]) && field[1] != '#') ||
>  				  (field[0] != '0' && field[0] != '1'),
>  				  "Invalid wait boolean at step %u!\n",
>  				  nr_steps);
> @@ -1179,23 +1387,28 @@ parse_workload(struct w_arg *arg, unsigned int flags, double scale_dur,
>  			valid++;
>  		}
>  
> -		check_arg(valid != 5, "Invalid record at step %u!\n", nr_steps);
> +		check_arg(valid != (is_xe ? 4 : 5), "Invalid record at step %u!\n", nr_steps);
>  
>  		step.type = BATCH;
>  
>  add_step:
> -		if (step.type == DELAY)
> -			step.delay = __duration(step.delay, scale_time);
> +		if (step.type == SKIP) {
> +			if (verbose > 3)
> +				printf("skipped STEP: %s\n", _token);
> +		} else {
> +			if (step.type == DELAY)
> +				step.delay = __duration(step.delay, scale_time);
>  
> -		step.idx = nr_steps++;
> -		step.request = -1;
> -		steps = realloc(steps, sizeof(step) * nr_steps);
> -		igt_assert(steps);
> +			step.idx = nr_steps++;
> +			step.request = -1;
> +			steps = realloc(steps, sizeof(step) * nr_steps);
> +			igt_assert(steps);
>  
> -		memcpy(&steps[nr_steps - 1], &step, sizeof(step));
> +			memcpy(&steps[nr_steps - 1], &step, sizeof(step));
> +		}
>  
>  		free(token);
> -	}
> +	} // while ((_token = strtok_r(tstart, ",", &tctx))) {
>  
>  	if (app_w) {
>  		steps = realloc(steps, sizeof(step) *
> @@ -1211,7 +1424,7 @@ add_step:
>  		nr_steps += app_w->nr_steps;
>  	}
>  
> -	wrk = malloc(sizeof(*wrk));
> +	wrk = calloc(1, sizeof(*wrk));
>  	igt_assert(wrk);
>  
>  	wrk->nr_steps = nr_steps;
> @@ -1370,6 +1583,24 @@ __get_ctx(struct workload *wrk, const struct w_step *w)
>  	return &wrk->ctx_list[w->context];
>  }
>  
> +static struct exec_queue *
> +get_eq(struct workload *wrk, const struct w_step *w)
> +{
> +	igt_assert(w->eq_idx < wrk->nr_eqs);
> +
> +	return &wrk->eq_list[w->eq_idx];
> +}
> +
> +static struct vm *
> +get_vm(struct workload *wrk, const struct w_step *w)
> +{
> +	uint32_t vm_idx = get_eq(wrk, w)->vm_idx;
> +
> +	igt_assert(vm_idx < wrk->nr_vms);
> +
> +	return &wrk->vm_list[vm_idx];
> +}
> +
>  static uint32_t mmio_base(int i915, enum intel_engine_id engine, int gen)
>  {
>  	const char *name;
> @@ -1554,7 +1785,7 @@ static uint32_t alloc_bo(int i915, unsigned long size)
>  }
>  
>  static void
> -alloc_step_batch(struct workload *wrk, struct w_step *w)
> +i915_alloc_step_batch(struct workload *wrk, struct w_step *w)
>  {
>  	enum intel_engine_id engine = w->engine;
>  	unsigned int j = 0;
> @@ -1622,6 +1853,68 @@ alloc_step_batch(struct workload *wrk, struct w_step *w)
>  #endif
>  }
>  
> +static void
> +xe_alloc_step_batch(struct workload *wrk, struct w_step *w)
> +{
> +	struct vm *vm = get_vm(wrk, w);
> +	struct exec_queue *eq = get_eq(wrk, w);
> +	struct dep_entry *dep;
> +	int i;
> +
> +	w->bb_size = ALIGN(sizeof(*w->spin) + xe_cs_prefetch_size(fd), xe_get_default_alignment(fd));
> +	w->bb_handle = xe_bo_create(fd, 0, vm->id, w->bb_size);
> +	w->spin = xe_bo_map(fd, w->bb_handle, w->bb_size);
> +	w->exec.address = intel_allocator_alloc_with_strategy(vm->ahnd, w->bb_handle, w->bb_size,
> +							0, ALLOC_STRATEGY_LOW_TO_HIGH);
> +	xe_vm_bind_sync(fd, vm->id, w->bb_handle, 0, w->exec.address, w->bb_size);
> +	xe_spin_init_opts(w->spin, .addr = w->exec.address,
> +				   .preempt = (w->preempt_us > 0),
> +				   .ctx_ticks = duration_to_ctx_ticks(fd, eq->hwe.gt_id,
> +								1000 * get_duration(wrk, w)));
> +	w->exec.exec_queue_id = eq->id;
> +	w->exec.num_batch_buffer = 1;
> +	/* always at least one out fence */
> +	w->exec.num_syncs = 1;
> +	/* count syncs */
> +	igt_assert_eq(0, w->data_deps.nr);
> +	for_each_dep(dep, w->fence_deps) {
> +		int dep_idx = w->idx + dep->target;
> +
> +		igt_assert(dep_idx >= 0 && dep_idx < w->idx);
> +		igt_assert(wrk->steps[dep_idx].type == SW_FENCE ||
> +			   wrk->steps[dep_idx].type == BATCH);
> +
> +		w->exec.num_syncs++;
> +	}
> +	w->syncs = calloc(w->exec.num_syncs, sizeof(*w->syncs));
> +	/* fill syncs */
> +	i = 0;
> +	/* out fence */
> +	w->syncs[i].handle = syncobj_create(fd, 0);
> +	w->syncs[i++].flags = DRM_XE_SYNC_SYNCOBJ | DRM_XE_SYNC_SIGNAL;
> +	/* in fence(s) */
> +	for_each_dep(dep, w->fence_deps) {
> +		int dep_idx = w->idx + dep->target;
> +
> +		igt_assert(wrk->steps[dep_idx].type == SW_FENCE ||
> +			   wrk->steps[dep_idx].type == BATCH);
> +		igt_assert(wrk->steps[dep_idx].syncs && wrk->steps[dep_idx].syncs[0].handle);
> +
> +		w->syncs[i].handle = wrk->steps[dep_idx].syncs[0].handle;
> +		w->syncs[i++].flags = DRM_XE_SYNC_SYNCOBJ;
> +	}
> +	w->exec.syncs = to_user_pointer(w->syncs);
> +}
> +
> +static void
> +alloc_step_batch(struct workload *wrk, struct w_step *w)
> +{
> +	if (is_xe)
> +		xe_alloc_step_batch(wrk, w);
> +	else
> +		i915_alloc_step_batch(wrk, w);
> +}
> +
>  static bool set_priority(uint32_t ctx_id, int prio)
>  {
>  	struct drm_i915_gem_context_param param = {
> @@ -1848,20 +2141,77 @@ static void measure_active_set(struct workload *wrk)
>  
>  #define alloca0(sz) ({ size_t sz__ = (sz); memset(alloca(sz__), 0, sz__); })
>  
> -static int prepare_workload(unsigned int id, struct workload *wrk)
> +static int xe_prepare_vms_eqs(unsigned int id, struct workload *wrk)
> +{
> +	struct w_step *w;
> +	int i, j;
> +
> +	/* Create vms - should be done before exec queues */
> +	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +		if (w->type != VM)
> +			continue;
> +		wrk->nr_vms++;
> +	}
> +	igt_assert(wrk->nr_vms);
> +	wrk->vm_list = calloc(wrk->nr_vms, sizeof(struct vm));
> +
> +	for (j = 0 /*vm_idx*/, i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +		struct vm *vm_;
> +
> +		if (w->type != VM)
> +			continue;
> +		vm_ = &wrk->vm_list[j];
> +		*vm_ = *w->vm;
> +		vm_->id = xe_vm_create(fd, 0 /*flags*/, 0 /*ext*/);
> +		vm_->ahnd = intel_allocator_open(fd, vm_->id, INTEL_ALLOCATOR_RELOC);
> +		j++;
> +	}
> +
> +	/* Create exec queues */
> +	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +		if (w->type != EXEC_QUEUE)
> +			continue;
> +		wrk->nr_eqs++;
> +	}
> +	igt_assert(wrk->nr_eqs);
> +	wrk->eq_list = calloc(wrk->nr_eqs, sizeof(struct exec_queue));
> +
> +	for (j = 0 /*eq_idx*/, i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +		struct exec_queue *eq;
> +		struct vm *vm_;
> +
> +		if (w->type != EXEC_QUEUE)
> +			continue;
> +		eq = &(wrk->eq_list[j]);
> +		*eq = *w->eq;
> +		vm_ = get_vm(wrk, w);
> +		igt_assert(vm_);
> +		igt_assert(eq->hwe.engine_instance >= 0);
> +		eq->id = xe_exec_queue_create(fd, vm_->id, &eq->hwe, 0 /*ext*/);
> +		/* init request list */
> +		IGT_INIT_LIST_HEAD(&eq->requests);
> +		eq->nrequest = 0;
> +		j++;
> +	}
> +
> +	/* create syncobjs for SW_FENCE */
> +	for (j = 0, i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++)
> +		if (w->type == SW_FENCE) {
> +			w->syncs = calloc(1, sizeof(struct drm_xe_sync));
> +			w->syncs[0].handle = syncobj_create(fd, 0);
> +			w->syncs[0].flags = DRM_XE_SYNC_SYNCOBJ;
> +		}
> +
> +	return 0;
> +}
> +
> +static int i915_prepare_ctxs(unsigned int id, struct workload *wrk)
>  {
> -	struct working_set **sets;
> -	unsigned long total = 0;
>  	uint32_t share_vm = 0;
>  	int max_ctx = -1;
>  	struct w_step *w;
>  	int i, j;
>  
> -	wrk->id = id;
> -	wrk->bb_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
> -	wrk->bo_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
> -	wrk->run = true;
> -
>  	/*
>  	 * Pre-scan workload steps to allocate context list storage.
>  	 */
> @@ -2050,6 +2400,25 @@ static int prepare_workload(unsigned int id, struct workload *wrk)
>  	if (share_vm)
>  		vm_destroy(fd, share_vm);
>  
> +	return 0;
> +}
> +
> +static int prepare_workload(unsigned int id, struct workload *wrk)
> +{
> +	struct w_step *w;
> +	int i, j;
> +
> +	wrk->id = id;
> +	wrk->bb_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
> +	wrk->bo_prng = (wrk->flags & SYNCEDCLIENTS) ? master_prng : rand();
> +	wrk->run = true;
> +
> +	if (is_xe) {
-------------- ^
No need for braces in if-else with signle statements.
Consider using checkpatch.pl

> +		xe_prepare_vms_eqs(id, wrk);
> +	} else {
--- ^ ---- ^
> +		i915_prepare_ctxs(id, wrk);
> +	}
--- ^

> +
>  	/* Record default preemption. */
>  	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
>  		if (w->type == BATCH)
> @@ -2070,75 +2439,89 @@ static int prepare_workload(unsigned int id, struct workload *wrk)
>  		for (j = i + 1; j < wrk->nr_steps; j++) {
>  			w2 = &wrk->steps[j];
>  
> -			if (w2->context != w->context)
> -				continue;
> -			else if (w2->type == PREEMPTION)
> -				break;
> -			else if (w2->type != BATCH)
> -				continue;
> +			if (is_xe) {
> +				if (w2->eq_idx != w->eq_idx)
> +					continue;
> +				else if (w2->type == PREEMPTION)
--------------- ^
No need for 'else' after continue/break

> +					break;
> +				else if (w2->type != BATCH)
> +					continue;
> +			} else {
> +				if (w2->context != w->context)
> +					continue;
> +				else if (w2->type == PREEMPTION)
> +					break;
> +				else if (w2->type != BATCH)
> +					continue;
> +			}
>  
>  			w2->preempt_us = w->period;
>  		}
>  	}
>  
> -	/*
> -	 * Scan for SSEU control steps.
> -	 */
> -	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> -		if (w->type == SSEU) {
> -			get_device_sseu();
> -			break;
> +	if (!is_xe) {
> +		struct working_set **sets;
> +		unsigned long total = 0;
> +
> +		/*
> +		* Scan for SSEU control steps.
> +		*/
> +		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +			if (w->type == SSEU) {
> +				get_device_sseu();
> +				break;
> +			}
>  		}
> -	}
>  
> -	/*
> -	 * Allocate working sets.
> -	 */
> -	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> -		if (w->type == WORKINGSET && !w->working_set.shared)
> -			total += allocate_working_set(wrk, &w->working_set);
> -	}
> +		/*
> +		* Allocate working sets.
> +		*/
> +		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +			if (w->type == WORKINGSET && !w->working_set.shared)
> +				total += allocate_working_set(wrk, &w->working_set);
> +		}
>  
> -	if (verbose > 2)
> -		printf("%u: %lu bytes in working sets.\n", wrk->id, total);
> +		if (verbose > 2)
> +			printf("%u: %lu bytes in working sets.\n", wrk->id, total);
>  
> -	/*
> -	 * Map of working set ids.
> -	 */
> -	wrk->max_working_set_id = -1;
> -	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> -		if (w->type == WORKINGSET &&
> -		    w->working_set.id > wrk->max_working_set_id)
> -			wrk->max_working_set_id = w->working_set.id;
> -	}
> +		/*
> +		* Map of working set ids.
> +		*/
> +		wrk->max_working_set_id = -1;
> +		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +			if (w->type == WORKINGSET &&
> +			w->working_set.id > wrk->max_working_set_id)
> +				wrk->max_working_set_id = w->working_set.id;
> +		}
>  
> -	sets = wrk->working_sets;
> -	wrk->working_sets = calloc(wrk->max_working_set_id + 1,
> -				   sizeof(*wrk->working_sets));
> -	igt_assert(wrk->working_sets);
> +		sets = wrk->working_sets;
> +		wrk->working_sets = calloc(wrk->max_working_set_id + 1,
> +					sizeof(*wrk->working_sets));
> +		igt_assert(wrk->working_sets);
>  
> -	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> -		struct working_set *set;
> +		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +			struct working_set *set;
>  
> -		if (w->type != WORKINGSET)
> -			continue;
> +			if (w->type != WORKINGSET)
> +				continue;
>  
> -		if (!w->working_set.shared) {
> -			set = &w->working_set;
> -		} else {
> -			igt_assert(sets);
> +			if (!w->working_set.shared) {
> +				set = &w->working_set;
> +			} else {
> +				igt_assert(sets);
>  
> -			set = sets[w->working_set.id];
> -			igt_assert(set->shared);
> -			igt_assert(set->sizes);
> +				set = sets[w->working_set.id];
> +				igt_assert(set->shared);
> +				igt_assert(set->sizes);
> +			}
> +
> +			wrk->working_sets[w->working_set.id] = set;
>  		}
>  
> -		wrk->working_sets[w->working_set.id] = set;
> +		if (sets)
> +			free(sets);
>  	}
>  
> -	if (sets)
> -		free(sets);
> -
>  	/*
>  	 * Allocate batch buffers.
>  	 */
> @@ -2149,7 +2532,9 @@ static int prepare_workload(unsigned int id, struct workload *wrk)
>  		alloc_step_batch(wrk, w);
>  	}
>  
> -	measure_active_set(wrk);
> +	if (!is_xe) {
> +		measure_active_set(wrk);
> +	}
>  
>  	return 0;
>  }
> @@ -2172,7 +2557,7 @@ update_bb_start(struct workload *wrk, struct w_step *w)
>  
>  	/* ticks is inverted for MI_DO_COMPARE (less-than comparison) */
>  	ticks = 0;
> -	if (!w->unbound_duration)
> +	if (!w->duration.unbound_duration)
>  		ticks = ~ns_to_ctx_ticks(1000 * get_duration(wrk, w));
>  
>  	*w->bb_duration = ticks;
> @@ -2193,7 +2578,32 @@ static void w_sync_to(struct workload *wrk, struct w_step *w, int target)
>  	igt_assert(target < wrk->nr_steps);
>  	igt_assert(wrk->steps[target].type == BATCH);
>  
> -	gem_sync(fd, wrk->steps[target].obj[0].handle);
> +	w_sync(fd, &wrk->steps[target]);
> +}
> +
> +static void do_xe_exec(struct workload *wrk, struct w_step *w)
> +{
> +	struct exec_queue *eq = get_eq(wrk, w);
> +
> +	igt_assert(w->emit_fence <= 0);
> +	if (w->emit_fence == -1)
> +		syncobj_reset(fd, &w->syncs[0].handle, 1);
> +
> +	/* update duration if random */
> +	if (w->duration.max != w->duration.min)
> +		xe_spin_init_opts(w->spin, .addr = w->exec.address,
> +					   .preempt = (w->preempt_us > 0),
> +					   .ctx_ticks = duration_to_ctx_ticks(fd, eq->hwe.gt_id,
> +								1000LL * get_duration(wrk, w)));
> +	xe_exec(fd, &w->exec);
> +
> +	/* for qd_throttle */
> +	if (w->rq_link.prev != NULL || w->rq_link.next != NULL) {
> +		igt_list_del(&w->rq_link);
> +		eq->nrequest--;
> +	}
> +	igt_list_add_tail(&w->rq_link, &eq->requests);
> +	eq->nrequest++;
>  }
>  
>  static void
> @@ -2252,7 +2662,7 @@ static void sync_deps(struct workload *wrk, struct w_step *w)
>  		igt_assert(dep_idx >= 0 && dep_idx < w->idx);
>  		igt_assert(wrk->steps[dep_idx].type == BATCH);
>  
> -		gem_sync(fd, wrk->steps[dep_idx].obj[0].handle);
> +		w_sync(fd, &wrk->steps[dep_idx]);
>  	}
>  }
>  
> @@ -2280,6 +2690,8 @@ static void *run_workload(void *data)
>  			enum intel_engine_id engine = w->engine;
>  			int do_sleep = 0;
>  
> +			igt_assert(w->type != SKIP);

Why you assert on SKIP? imho better continue as comments are
treated as SKIPS in descriptions of runs.

Regards,
Kamil

> +
>  			if (w->type == DELAY) {
>  				do_sleep = w->delay;
>  			} else if (w->type == PERIOD) {
> @@ -2306,7 +2718,7 @@ static void *run_workload(void *data)
>  
>  				igt_assert(s_idx >= 0 && s_idx < i);
>  				igt_assert(wrk->steps[s_idx].type == BATCH);
> -				gem_sync(fd, wrk->steps[s_idx].obj[0].handle);
> +				w_sync(fd, &wrk->steps[s_idx]);
>  				continue;
>  			} else if (w->type == THROTTLE) {
>  				throttle = w->throttle;
> @@ -2320,6 +2732,9 @@ static void *run_workload(void *data)
>  					sw_sync_timeline_create_fence(wrk->sync_timeline,
>  								      cur_seqno + w->idx);
>  				igt_assert(w->emit_fence > 0);
> +				if (is_xe)
> +					/* Convert sync file to syncobj */
> +					syncobj_import_sync_file(fd, w->syncs[0].handle, w->emit_fence);
>  				continue;
>  			} else if (w->type == SW_FENCE_SIGNAL) {
>  				int tgt = w->idx + w->target;
> @@ -2349,9 +2764,12 @@ static void *run_workload(void *data)
>  
>  				igt_assert(t_idx >= 0 && t_idx < i);
>  				igt_assert(wrk->steps[t_idx].type == BATCH);
> -				igt_assert(wrk->steps[t_idx].unbound_duration);
> +				igt_assert(wrk->steps[t_idx].duration.unbound_duration);
>  
> -				*wrk->steps[t_idx].bb_duration = 0xffffffff;
> +				if (is_xe)
> +					xe_spin_end(wrk->steps[t_idx].spin);
> +				else
> +					*wrk->steps[t_idx].bb_duration = 0xffffffff;
>  				__sync_synchronize();
>  				continue;
>  			} else if (w->type == SSEU) {
> @@ -2365,7 +2783,9 @@ static void *run_workload(void *data)
>  				   w->type == ENGINE_MAP ||
>  				   w->type == LOAD_BALANCE ||
>  				   w->type == BOND ||
> -				   w->type == WORKINGSET) {
> +				   w->type == WORKINGSET ||
> +				   w->type == VM ||
> +				   w->type == EXEC_QUEUE) {
>  				   /* No action for these at execution time. */
>  				continue;
>  			}
> @@ -2383,34 +2803,54 @@ static void *run_workload(void *data)
>  			if (throttle > 0)
>  				w_sync_to(wrk, w, i - throttle);
>  
> -			do_eb(wrk, w, engine);
> +			if (is_xe)
> +				do_xe_exec(wrk, w);
> +			else {
> +				do_eb(wrk, w, engine);
>  
> -			if (w->request != -1) {
> -				igt_list_del(&w->rq_link);
> -				wrk->nrequest[w->request]--;
> +				if (w->request != -1) {
> +					igt_list_del(&w->rq_link);
> +					wrk->nrequest[w->request]--;
> +				}
> +				w->request = engine;
> +				igt_list_add_tail(&w->rq_link, &wrk->requests[engine]);
> +				wrk->nrequest[engine]++;
>  			}
> -			w->request = engine;
> -			igt_list_add_tail(&w->rq_link, &wrk->requests[engine]);
> -			wrk->nrequest[engine]++;
>  
>  			if (!wrk->run)
>  				break;
>  
>  			if (w->sync)
> -				gem_sync(fd, w->obj[0].handle);
> +				w_sync(fd, w);
>  
>  			if (qd_throttle > 0) {
> -				while (wrk->nrequest[engine] > qd_throttle) {
> -					struct w_step *s;
> +				if (is_xe) {
> +					struct exec_queue *eq = get_eq(wrk, w);
>  
> -					s = igt_list_first_entry(&wrk->requests[engine],
> -								 s, rq_link);
> +					while (eq->nrequest > qd_throttle) {
> +						struct w_step *s;
>  
> -					gem_sync(fd, s->obj[0].handle);
> +						s = igt_list_first_entry(&eq->requests, s, rq_link);
>  
> -					s->request = -1;
> -					igt_list_del(&s->rq_link);
> -					wrk->nrequest[engine]--;
> +						w_sync(fd, s);
> +
> +						igt_list_del(&s->rq_link);
> +						eq->nrequest--;
> +					}
> +				} else {
> +					while (wrk->nrequest[engine] > qd_throttle) {
> +						struct w_step *s;
> +
> +						s = igt_list_first_entry(&wrk->requests[engine],
> +									s, rq_link);
> +
> +						w_sync(fd, s);
> +						// gem_sync(fd, s->obj[0].handle);
> +
> +						s->request = -1;
> +						igt_list_del(&s->rq_link);
> +						wrk->nrequest[engine]--;
> +					}
>  				}
>  			}
>  		}
> @@ -2427,18 +2867,51 @@ static void *run_workload(void *data)
>  		for (i = 0, w = wrk->steps; wrk->run && (i < wrk->nr_steps);
>  		     i++, w++) {
>  			if (w->emit_fence > 0) {
> -				close(w->emit_fence);
> -				w->emit_fence = -1;
> +				if (is_xe) {
> +					igt_assert(w->type == SW_FENCE);
> +					close(w->emit_fence);
> +					w->emit_fence = -1;
> +					syncobj_reset(fd, &w->syncs[0].handle, 1);
> +				} else {
> +					close(w->emit_fence);
> +					w->emit_fence = -1;
> +				}
>  			}
>  		}
> -	}
> +	} // main loop
>  
> -	for (i = 0; i < NUM_ENGINES; i++) {
> -		if (!wrk->nrequest[i])
> -			continue;
> +	if (is_xe) {
> +		struct exec_queue *eq;
>  
> -		w = igt_list_last_entry(&wrk->requests[i], w, rq_link);
> -		gem_sync(fd, w->obj[0].handle);
> +		for_each_exec_queue(eq, wrk) {
> +			if (eq->nrequest) {
> +				w = igt_list_last_entry(&eq->requests, w, rq_link);
> +				w_sync(fd, w);
> +			}
> +		}
> +
> +		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
> +			if (w->type == BATCH) {
> +				w_sync(fd, w);
> +				syncobj_destroy(fd, w->syncs[0].handle);
> +				free(w->syncs);
> +				xe_vm_unbind_sync(fd, get_vm(wrk, w)->id, 0, w->exec.address, w->bb_size);
> +				gem_munmap(w->spin, w->bb_size);
> +				gem_close(fd, w->bb_handle);
> +			} else if (w->type == SW_FENCE) {
> +				syncobj_destroy(fd, w->syncs[0].handle);
> +				free(w->syncs);
> +			}
> +		}
> +	}
> +	else {
> +		for (i = 0; i < NUM_ENGINES; i++) {
> +			if (!wrk->nrequest[i])
> +				continue;
> +
> +			w = igt_list_last_entry(&wrk->requests[i], w, rq_link);
> +			w_sync(fd, w);
> +		}
>  	}
>  
>  	clock_gettime(CLOCK_MONOTONIC, &t_end);
> @@ -2460,6 +2933,21 @@ static void *run_workload(void *data)
>  
>  static void fini_workload(struct workload *wrk)
>  {
> +	if (is_xe) {
> +		struct exec_queue *eq;
> +		struct vm *vm_;
> +
> +		for_each_exec_queue(eq, wrk)
> +			xe_exec_queue_destroy(fd, eq->id);
> +		free(wrk->eq_list);
> +		wrk->nr_eqs = 0;
> +		for_each_vm(vm_, wrk) {
> +			put_ahnd(vm_->ahnd);
> +			xe_vm_destroy(fd, vm_->id);
> +		}
> +		free(wrk->vm_list);
> +		wrk->nr_vms = 0;
> +	}
>  	free(wrk->steps);
>  	free(wrk);
>  }
> @@ -2519,6 +3007,13 @@ static char *load_workload_descriptor(char *filename)
>  	close(infd);
>  
>  	for (i = 0; i < len; i++) {
> +		/* '#' starts comment till end of line */
> +		if (buf[i] == '#')
> +			/* replace ',' in comments to not break parsing */
> +			while (++i < len && buf[i] != '\n')
> +				if (buf[i] == ',')
> +					buf[i] = ';';
> +
>  		if (buf[i] == '\n')
>  			buf[i] = ',';
>  	}
> @@ -2562,7 +3057,7 @@ int main(int argc, char **argv)
>  	int prio = 0;
>  	double t;
>  	int i, c, ret;
> -	char *drm_dev;
> +	char *drm_dev = NULL;
>  
>  	master_prng = time(NULL);
>  
> @@ -2660,8 +3155,12 @@ int main(int argc, char **argv)
>  		ret = igt_device_find_first_i915_discrete_card(&card);
>  		if (!ret)
>  			ret = igt_device_find_integrated_card(&card);
> +		if (!ret)
> +			ret = igt_device_find_first_xe_discrete_card(&card);
> +		if (!ret)
> +			ret = igt_device_find_xe_integrated_card(&card);
>  		if (!ret) {
> -			wsim_err("No device filter specified and no i915 devices found!\n");
> +			wsim_err("No device filter specified and no intel devices found!\n");
>  			return EXIT_FAILURE;
>  		}
>  	}
> @@ -2676,6 +3175,7 @@ int main(int argc, char **argv)
>  	}
>  
>  	fd = open(drm_dev, O_RDWR);
> +
>  	if (fd < 0) {
>  		wsim_err("Failed to open '%s'! (%s)\n",
>  			 drm_dev, strerror(errno));
> @@ -2684,6 +3184,10 @@ int main(int argc, char **argv)
>  	if (verbose > 1)
>  		printf("Using device %s\n", drm_dev);
>  
> +	is_xe = is_xe_device(fd);
> +	if (is_xe)
> +		xe_device_get(fd);
> +
>  	if (!nr_w_args) {
>  		wsim_err("No workload descriptor(s)!\n");
>  		goto err;
> diff --git a/benchmarks/wsim/README b/benchmarks/wsim/README
> index 8c71f2fe6..ddfefff47 100644
> --- a/benchmarks/wsim/README
> +++ b/benchmarks/wsim/README
> @@ -1,6 +1,9 @@
>  Workload descriptor format
>  ==========================
>  
> +Lines starting with '#' are treated as comments (do not create work step).
> +
> +# i915
>  ctx.engine.duration_us.dependency.wait,...
>  <uint>.<str>.<uint>[-<uint>]|*.<int <= 0>[/<int <= 0>][...].<0|1>,...
>  B.<uint>
> @@ -11,6 +14,23 @@ b.<uint>.<str>[|<str>].<str>
>  w|W.<uint>.<str>[/<str>]...
>  f
>  
> +# xe
> +Xe does not use context abstraction and adds additional work step types
> +for VM (v.) and exec queue (e.) creation.
> +Each v. and e. step creates array entry (in workload's VM and Exec Queue arrays).
> +Batch step references the exec queue on which it is to be executed.
> +Exec queue reference (eq_idx) is the index (0-based) in workload's exec queue array.
> +VM reference (vm_idx) is the index (0-based) in workload's VM array.
> +
> +v.compute_mode
> +v.<0|1>
> +e.vm_idx.class.instance.compute_mode.job_timeout_ms,...
> +e.<uint>.<uint 0=RCS,1=BCS,2=VCS,3=VECS,4=CCS>.<int>.<0|1>.<uint>,...
> +eq_idx.duration_us.dependency.wait,...
> +<uint>.<uint>[-<uint>]|*.<int <= 0>[/<int <= 0>][...].<0|1>,...
> +d|p|s|t|q|a|T.<int>,...
> +f
> +
>  For duration a range can be given from which a random value will be picked
>  before every submit. Since this and seqno management requires CPU access to
>  objects, care needs to be taken in order to ensure the submit queue is deep
> @@ -27,21 +47,22 @@ Additional workload steps are also supported:
>   'q' - Throttle to n max queue depth.
>   'f' - Create a sync fence.
>   'a' - Advance the previously created sync fence.
> - 'B' - Turn on context load balancing.
> - 'b' - Set up engine bonds.
> - 'M' - Set up engine map.
> - 'P' - Context priority.
> - 'S' - Context SSEU configuration.
> + 'B' - Turn on context load balancing. (i915 only)
> + 'b' - Set up engine bonds. (i915 only)
> + 'M' - Set up engine map. (i915 only)
> + 'P' - Context priority. (i915 only)
> + 'S' - Context SSEU configuration. (i915 only)
>   'T' - Terminate an infinite batch.
> - 'w' - Working set. (See Working sets section.)
> - 'W' - Shared working set.
> - 'X' - Context preemption control.
> + 'w' - Working set. (See Working sets section.) (i915 only)
> + 'W' - Shared working set. (i915 only)
> + 'X' - Context preemption control. (i915 only)
>  
>  Engine ids: DEFAULT, RCS, BCS, VCS, VCS1, VCS2, VECS
>  
>  Example (leading spaces must not be present in the actual file):
>  ----------------------------------------------------------------
>  
> +# i915
>    1.VCS1.3000.0.1
>    1.RCS.500-1000.-1.0
>    1.RCS.3700.0.0
> @@ -51,6 +72,25 @@ Example (leading spaces must not be present in the actual file):
>    1.VCS2.600.-1.1
>    p.16000
>  
> +# xe equivalent
> +  #VM: v.compute_mode
> +  v.0
> +  #EXEC_QUEUE: e.vm_idx.class.intance.compute_mode.job_timeout_ms
> +  e.0.2.0.0.0 # VCS1
> +  e.0.0.0.0.0 # RCS
> +  e.0.2.1.0.0 # VCS2
> +  e.0.0.0.0.0 # second RCS exec queue
> +  #BATCH: eq_idx.duration.dependency.wait
> +  0.3000.0.1       # 1.VCS1.3000.0.1
> +  1.500-1000.-1.0  # 1.RCS.500-1000.-1.0
> +  3.3700.0.0       # 1.RCS.3700.0.0
> +  1.1000.-2.1      # 1.RCS.1000.-2.0
> +  2.2300.-2.0      # 1.VCS2.2300.-2.0
> +  3.4700.-1.0      # 1.RCS.4700.-1.0
> +  2.600.-1.1       # 1.VCS2.600.-1.1
> +  p.16000
> +
> +
>  The above workload described in human language works like this:
>  
>    1.   A batch is sent to the VCS1 engine which will be executing for 3ms on the
> @@ -76,16 +116,30 @@ Multiple dependencies can be given separated by forward slashes.
>  
>  Example:
>  
> +# i915
>    1.VCS1.3000.0.1
>    1.RCS.3700.0.0
>    1.VCS2.2300.-1/-2.0
>  
> +# xe
> +  v.0
> +  e.0.2.0.0.0
> +  e.0.0.0.0.0
> +  e.0.2.1.0.0.0
> +  0.3000.0.1
> +  1.3700.0.0
> +  2.2300.-1/-2.0
> +
>  I this case the last step has a data dependency on both first and second steps.
>  
>  Batch durations can also be specified as infinite by using the '*' in the
>  duration field. Such batches must be ended by the terminate command ('T')
>  otherwise they will cause a GPU hang to be reported.
>  
> +Note: On Xe Batch dependencies are expressed with syncobjects,
> +so there is no difference between f-1 and -1
> +ex. 1.1000.-2.0 is same as 1.1000.f-2.0.
> +
>  Sync (fd) fences
>  ----------------
>  
> @@ -114,6 +168,7 @@ VCS1 and VCS2 batches will have a sync fence dependency on the RCS batch.
>  
>  Example:
>  
> +# i915
>    1.RCS.500-1000.0.0
>    f
>    2.VCS1.3000.f-1.0
> @@ -123,13 +178,27 @@ Example:
>    s.-4
>    s.-4
>  
> +# xe equivalent
> +  v.0
> +  e.0.0.0.0.0    # RCS
> +  e.0.2.0.0.0    # VCS1
> +  e.0.2.1.0.0    # VCS2
> +  0.500-1000.0.0
> +  f
> +  1.3000.f-1.0
> +  2.3000.f-2.0
> +  0.500-1000.0.1
> +  a.-4
> +  s.-4
> +  s.-4
> +
>  VCS1 and VCS2 batches have an input sync fence dependecy on the standalone fence
>  created at the second step. They are submitted ahead of time while still not
>  runnable. When the second RCS batch completes the standalone fence is signaled
>  which allows the two VCS batches to be executed. Finally we wait until the both
>  VCS batches have completed before starting the (optional) next iteration.
>  
> -Submit fences
> +Submit fences (i915 only?)
>  -------------
>  
>  Submit fences are a type of input fence which are signalled when the originating
> diff --git a/benchmarks/wsim/xe_cloud-gaming-60fps.wsim b/benchmarks/wsim/xe_cloud-gaming-60fps.wsim
> new file mode 100644
> index 000000000..9fdf15e27
> --- /dev/null
> +++ b/benchmarks/wsim/xe_cloud-gaming-60fps.wsim
> @@ -0,0 +1,25 @@
> +#w.1.10n8m
> +#w.2.3n16m
> +#1.RCS.500-1500.r1-0-4/w2-0.0
> +#1.RCS.500-1500.r1-5-9/w2-1.0
> +#1.RCS.500-1500.r2-0-1/w2-2.0
> +#M.2.VCS
> +#B.2
> +#3.RCS.500-1500.r2-2.0
> +#2.DEFAULT.2000-4000.-1.0
> +#4.VCS1.250-750.-1.1
> +#p.16667
> +#
> +#xe
> +v.0
> +e.0.0.0.0.0 # 1.RCS.500-1500.r1-0-4/w2-0.0
> +e.0.2.0.0.0 # 2.DEFAULT.2000-4000.-1.0
> +e.0.0.0.0.0 # 3.RCS.500-1500.r2-2.0
> +e.0.2.1.0.0 # 4.VCS1.250-750.-1.1
> +0.500-1500.0.0
> +0.500-1500.0.0
> +0.500-1500.0.0
> +2.500-1500.-2.0 # #3.RCS.500-1500.r2-2.0
> +1.2000-4000.-1.0
> +3.250-750.-1.1
> +p.16667
> diff --git a/benchmarks/wsim/xe_example.wsim b/benchmarks/wsim/xe_example.wsim
> new file mode 100644
> index 000000000..3fa620932
> --- /dev/null
> +++ b/benchmarks/wsim/xe_example.wsim
> @@ -0,0 +1,28 @@
> +#i915
> +#1.VCS1.3000.0.1
> +#1.RCS.500-1000.-1.0
> +#1.RCS.3700.0.0
> +#1.RCS.1000.-2.0
> +#1.VCS2.2300.-2.0
> +#1.RCS.4700.-1.0
> +#1.VCS2.600.-1.1
> +#p.16000
> +#
> +#xe
> +#
> +#VM: v.compute_mode
> +v.0
> +#EXEC_QUEUE: e.vm_idx.class.intance.compute_mode.job_timeout_ms
> +e.0.2.0.0.0 # VCS1
> +e.0.0.0.0.0 # RCS
> +e.0.2.1.0.0 # VCS2
> +e.0.0.0.0.0 # second RCS exec_queue
> +#BATCH: eq_idx.duration.dependency.wait
> +0.3000.0.1       # 1.VCS1.3000.0.1
> +1.500-1000.-1.0  # 1.RCS.500-1000.-1.0
> +3.3700.0.0       # 1.RCS.3700.0.0
> +1.1000.-2.1      # 1.RCS.1000.-2.0
> +2.2300.-2.0      # 1.VCS2.2300.-2.0
> +3.4700.-1.0      # 1.RCS.4700.-1.0
> +2.600.-1.1       # 1.VCS2.600.-1.1
> +p.16000
> diff --git a/benchmarks/wsim/xe_example01.wsim b/benchmarks/wsim/xe_example01.wsim
> new file mode 100644
> index 000000000..496905371
> --- /dev/null
> +++ b/benchmarks/wsim/xe_example01.wsim
> @@ -0,0 +1,19 @@
> +#VM: v.compute_mode
> +v.0
> +#EXEC_QUEUE: e.vm_idx.class.intance.compute_mode.job_timeout_ms
> +e.0.0.0.0.0
> +e.0.2.0.0.0
> +e.0.1.0.0.0
> +#BATCH: eq_idx.duration.dependency.wait
> +# B1 - 10ms batch on BCS0
> +2.10000.0.0
> +# B2 - 10ms batch on RCS0; waits on B1
> +0.10000.0.0
> +# B3 - 10ms batch on VECS0; waits on B2
> +1.10000.0.0
> +# B4 - 10ms batch on BCS0
> +2.10000.0.0
> +# B5 - 10ms batch on RCS0; waits on B4
> +0.10000.-1.0
> +# B6 - 10ms batch on VECS0; waits on B5; wait on batch fence out
> +1.10000.-1.1
> diff --git a/benchmarks/wsim/xe_example_fence.wsim b/benchmarks/wsim/xe_example_fence.wsim
> new file mode 100644
> index 000000000..4f810d64e
> --- /dev/null
> +++ b/benchmarks/wsim/xe_example_fence.wsim
> @@ -0,0 +1,23 @@
> +#i915
> +#1.RCS.500-1000.0.0
> +#f
> +#2.VCS1.3000.f-1.0
> +#2.VCS2.3000.f-2.0
> +#1.RCS.500-1000.0.1
> +#a.-4
> +#s.-4
> +#s.-4
> +#
> +#xe
> +v.0
> +e.0.0.0.0.0
> +e.0.2.0.0.0
> +e.0.2.1.0.0
> +0.500-1000.0.0
> +f
> +1.3000.f-1.0
> +2.3000.f-2.0
> +0.500-1000.0.1
> +a.-4
> +s.-4
> +s.-4
> diff --git a/benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim b/benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim
> new file mode 100644
> index 000000000..2214914eb
> --- /dev/null
> +++ b/benchmarks/wsim/xe_media_load_balance_fhd26u7.wsim
> @@ -0,0 +1,63 @@
> +# https://lore.kernel.org/dri-devel/a443495f-5d1b-52e1-9b2f-80167deb6d57@linux.intel.com/
> +#i915
> +#M.3.VCS
> +#B.3
> +#1.VCS1.1200-1800.0.0
> +#1.VCS1.1900-2100.0.0
> +#2.RCS.1500-2000.-1.0
> +#3.VCS.1400-1800.-1.1
> +#1.VCS1.1900-2100.-1.0
> +#2.RCS.1500-2000.-1.0
> +#3.VCS.1400-1800.-1.1
> +#1.VCS1.1900-2100.-1.0
> +#2.RCS.200-400.-1.0
> +#2.RCS.1500-2000.0.0
> +#3.VCS.1400-1800.-1.1
> +#1.VCS1.1900-2100.-1.0
> +#2.RCS.1500-2000.-1.0
> +#3.VCS.1400-1800.-1.1
> +#1.VCS1.1900-2100.-1.0
> +#2.RCS.200-400.-1.0
> +#2.RCS.1500-2000.0.0
> +#3.VCS.1400-1800.-1.1
> +#1.VCS1.1900-2100.-1.0
> +#2.RCS.1500-2000.-1.0
> +#3.VCS.1400-1800.-1.1
> +#1.VCS1.1900-2100.-1.0
> +#2.RCS.1500-2000.-1.0
> +#2.RCS.1500-2000.0.0
> +#3.VCS.1400-1800.-1.1
> +#
> +#xe
> +#
> +#M.3.VCS ??
> +#B.3     ??
> +v.0
> +e.0.2.0.0.0 # 1.VCS1
> +e.0.0.0.0.0 # 2.RCS
> +e.0.2.1.0.0 # 3.VCS - no load balancing yet always VCS2
> +0.1200-1800.0.0
> +0.1900-2100.0.0
> +1.1500-2000.-1.0
> +2.1400-1800.-1.1
> +0.1900-2100.-1.0
> +1.1500-2000.-1.0
> +2.1400-1800.-1.1
> +0.1900-2100.-1.0
> +1.200-400.-1.0
> +1.1500-2000.0.0
> +2.1400-1800.-1.1
> +0.1900-2100.-1.0
> +1.1500-2000.-1.0
> +2.1400-1800.-1.1
> +0.1900-2100.-1.0
> +1.200-400.-1.0
> +1.1500-2000.0.0
> +2.1400-1800.-1.1
> +0.1900-2100.-1.0
> +1.1500-2000.-1.0
> +2.1400-1800.-1.1
> +0.1900-2100.-1.0
> +1.1500-2000.-1.0
> +1.1500-2000.0.0
> +2.1400-1800.-1.1
> -- 
> 2.30.2
> 

[igt-dev] ✗ GitLab.Pipeline: warning for benchmarks/gem_wsim: added basic xe support

[Patchwork]

== Series Details ==

Series: benchmarks/gem_wsim: added basic xe support
URL   : https://patchwork.freedesktop.org/series/122920/
State : warning

== Summary ==

Pipeline status: FAILED.

see https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/pipelines/970708 for the overview.

build-containers:build-debian-arm64 has failed (https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/48051972):
  time="2023-08-25T14:46:31Z" level=fatal msg="Error determining repository tags: Get https://registry.freedesktop.org/v2/gfx-ci/igt-ci-tags/build-debian-arm64/tags/list?last=commit-3a9f3015d8f18845de584931c1cf62da39ace1a6&n=100: dial tcp 147.75.198.156:443: i/o timeout" 
  Building!
  STEP 1: FROM debian:buster
  Getting image source signatures
  Copying blob sha256:d6b7393fb4f375905c31c483d81ce2a2905f88aba8cb198874da2b54035bc41d
  Copying config sha256:de08540e8ff0e470ff7956df4bed403725a5f45c186e9bf495da5344ff8fbe84
  Writing manifest to image destination
  Storing signatures
  STEP 2: RUN apt-get update
  error running container: error creating container for [/bin/sh -c apt-get update]: time="2023-08-25T14:46:36Z" level=warning msg="signal: killed"
  time="2023-08-25T14:46:36Z" level=error msg="container_linux.go:346: starting container process caused \"process_linux.go:297: applying cgroup configuration for process caused \\\"mountpoint for cgroup not found\\\"\"\n"
  container_linux.go:346: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"mountpoint for cgroup not found\""
  : exit status 1
  Error: error building at STEP "RUN apt-get update": error while running runtime: exit status 1
  section_end:1692974796:step_script
  section_start:1692974796:cleanup_file_variables
  Cleaning up project directory and file based variables
  section_end:1692974797:cleanup_file_variables
  ERROR: Job failed: exit code 1

== Logs ==

For more details see: https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/pipelines/970708

[igt-dev] ✓ CI.xeBAT: success for benchmarks/gem_wsim: added basic xe support

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]

== Series Details ==

Series: benchmarks/gem_wsim: added basic xe support
URL   : https://patchwork.freedesktop.org/series/122920/
State : success

== Summary ==

CI Bug Log - changes from XEIGT_7453_BAT -> XEIGTPW_9663_BAT
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  

Participating hosts (3 -> 3)
------------------------------

  No changes in participating hosts

Known issues
------------

  Here are the changes found in XEIGTPW_9663_BAT that come from known issues:

### IGT changes ###

#### Possible fixes ####

  * igt@kms_pipe_crc_basic@compare-crc-sanitycheck-nv12:
    - bat-adlp-7:         [FAIL][1] ([Intel XE#400]) -> [PASS][2] +1 similar issue
   [1]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7453/bat-adlp-7/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-nv12.html
   [2]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_9663/bat-adlp-7/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-nv12.html

  
  [Intel XE#400]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/400


Build changes
-------------

  * IGT: IGT_7453 -> IGTPW_9663

  IGTPW_9663: 9663
  IGT_7453: 7453
  xe-340-c77796cf84361b4716839141f2e48de2bf7f4bd5: c77796cf84361b4716839141f2e48de2bf7f4bd5



[-- Attachment #2: Type: text/html, Size: 1728 bytes --]

[igt-dev] ✓ Fi.CI.BAT: success for benchmarks/gem_wsim: added basic xe support

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 2169 bytes --]

== Series Details ==

Series: benchmarks/gem_wsim: added basic xe support
URL   : https://patchwork.freedesktop.org/series/122920/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_13568 -> IGTPW_9663
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/index.html

Participating hosts (40 -> 39)
------------------------------

  Missing    (1): fi-snb-2520m 

Known issues
------------

  Here are the changes found in IGTPW_9663 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@kms_psr@primary_page_flip:
    - bat-rplp-1:         NOTRUN -> [ABORT][1] ([i915#8442] / [i915#8668] / [i915#8860])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/bat-rplp-1/igt@kms_psr@primary_page_flip.html

  
#### Possible fixes ####

  * igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1:
    - bat-rplp-1:         [ABORT][2] ([i915#8442] / [i915#8668]) -> [PASS][3]
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/bat-rplp-1/igt@kms_pipe_crc_basic@read-crc-frame-sequence@pipe-d-edp-1.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#8442]: https://gitlab.freedesktop.org/drm/intel/issues/8442
  [i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
  [i915#8860]: https://gitlab.freedesktop.org/drm/intel/issues/8860
  [i915#8879]: https://gitlab.freedesktop.org/drm/intel/issues/8879


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7453 -> IGTPW_9663

  CI-20190529: 20190529
  CI_DRM_13568: f8b90de70ee92dbdebc6f2078e2edd12756d7a63 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_9663: 9663
  IGT_7453: 7453

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/index.html

[-- Attachment #2: Type: text/html, Size: 2781 bytes --]

[igt-dev] ✓ Fi.CI.IGT: success for benchmarks/gem_wsim: added basic xe support

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 80452 bytes --]

== Series Details ==

Series: benchmarks/gem_wsim: added basic xe support
URL   : https://patchwork.freedesktop.org/series/122920/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_13568_full -> IGTPW_9663_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/index.html

Participating hosts (9 -> 9)
------------------------------

  No changes in participating hosts

New tests
---------

  New tests have been introduced between CI_DRM_13568_full and IGTPW_9663_full:

### New IGT tests (35) ###

  * igt@kms_flip@absolute-wf_vblank-interruptible@a-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@absolute-wf_vblank-interruptible@b-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@absolute-wf_vblank-interruptible@c-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@basic-flip-vs-dpms@a-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@basic-flip-vs-dpms@b-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@basic-flip-vs-dpms@c-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@bo-too-big-interruptible@a-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@bo-too-big-interruptible@b-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@bo-too-big-interruptible@c-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@bo-too-big@a-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@bo-too-big@b-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@bo-too-big@c-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@flip-vs-blocking-wf-vblank@a-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@flip-vs-blocking-wf-vblank@b-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@flip-vs-blocking-wf-vblank@c-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@flip-vs-wf_vblank-interruptible@a-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@flip-vs-wf_vblank-interruptible@b-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_flip@flip-vs-wf_vblank-interruptible@c-hdmi-a3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@bad-vsync-start@hdmi-a-3-pipe-a:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@bad-vsync-start@hdmi-a-3-pipe-b:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@bad-vsync-start@hdmi-a-3-pipe-c:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@bad-vsync-start@hdmi-a-3-pipe-d:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@int-max-clock@hdmi-a-3-pipe-a:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@int-max-clock@hdmi-a-3-pipe-b:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@int-max-clock@hdmi-a-3-pipe-c:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@int-max-clock@hdmi-a-3-pipe-d:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-hdisplay@hdmi-a-3-pipe-a:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-hdisplay@hdmi-a-3-pipe-b:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-hdisplay@hdmi-a-3-pipe-c:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-hdisplay@hdmi-a-3-pipe-d:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-vdisplay@hdmi-a-3-pipe-a:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-vdisplay@hdmi-a-3-pipe-b:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-vdisplay@hdmi-a-3-pipe-c:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_invalid_mode@zero-vdisplay@hdmi-a-3-pipe-d:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  * igt@kms_rmfb@close-fd@pipe-c-hdmi-a-3:
    - Statuses : 1 pass(s)
    - Exec time: [0.0] s

  

Known issues
------------

  Here are the changes found in IGTPW_9663_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@api_intel_bb@blit-reloc-purge-cache:
    - shard-dg2:          NOTRUN -> [SKIP][1] ([i915#8411]) +1 similar issue
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@api_intel_bb@blit-reloc-purge-cache.html
    - shard-rkl:          NOTRUN -> [SKIP][2] ([i915#8411]) +1 similar issue
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-6/igt@api_intel_bb@blit-reloc-purge-cache.html

  * igt@device_reset@unbind-cold-reset-rebind:
    - shard-mtlp:         NOTRUN -> [SKIP][3] ([i915#7701])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@device_reset@unbind-cold-reset-rebind.html

  * igt@drm_fdinfo@most-busy-idle-check-all@rcs0:
    - shard-rkl:          NOTRUN -> [FAIL][4] ([i915#7742])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@drm_fdinfo@most-busy-idle-check-all@rcs0.html

  * igt@drm_fdinfo@most-busy-idle-check-all@vecs1:
    - shard-dg2:          NOTRUN -> [SKIP][5] ([i915#8414]) +9 similar issues
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@drm_fdinfo@most-busy-idle-check-all@vecs1.html

  * igt@drm_fdinfo@virtual-busy-all:
    - shard-mtlp:         NOTRUN -> [SKIP][6] ([i915#8414])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@drm_fdinfo@virtual-busy-all.html

  * igt@drm_mm@drm_mm_test:
    - shard-snb:          NOTRUN -> [SKIP][7] ([fdo#109271] / [i915#8661])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-snb5/igt@drm_mm@drm_mm_test.html
    - shard-dg2:          NOTRUN -> [SKIP][8] ([i915#8661])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@drm_mm@drm_mm_test.html

  * igt@feature_discovery@psr1:
    - shard-dg2:          NOTRUN -> [SKIP][9] ([i915#658]) +5 similar issues
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@feature_discovery@psr1.html
    - shard-rkl:          NOTRUN -> [SKIP][10] ([i915#658]) +1 similar issue
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-6/igt@feature_discovery@psr1.html

  * igt@gem_caching@writes:
    - shard-mtlp:         NOTRUN -> [SKIP][11] ([i915#4873]) +1 similar issue
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@gem_caching@writes.html

  * igt@gem_close_race@multigpu-basic-threads:
    - shard-mtlp:         NOTRUN -> [SKIP][12] ([i915#7697])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@gem_close_race@multigpu-basic-threads.html

  * igt@gem_create@create-ext-set-pat:
    - shard-dg2:          NOTRUN -> [SKIP][13] ([i915#8562])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@gem_create@create-ext-set-pat.html

  * igt@gem_ctx_exec@basic-nohangcheck:
    - shard-mtlp:         [PASS][14] -> [FAIL][15] ([i915#6121])
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-4/igt@gem_ctx_exec@basic-nohangcheck.html
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@gem_ctx_exec@basic-nohangcheck.html

  * igt@gem_ctx_persistence@heartbeat-hostile:
    - shard-dg2:          NOTRUN -> [SKIP][16] ([i915#8555])
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-3/igt@gem_ctx_persistence@heartbeat-hostile.html

  * igt@gem_ctx_persistence@legacy-engines-queued:
    - shard-snb:          NOTRUN -> [SKIP][17] ([fdo#109271] / [i915#1099]) +3 similar issues
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-snb6/igt@gem_ctx_persistence@legacy-engines-queued.html

  * igt@gem_eio@in-flight-10ms:
    - shard-mtlp:         NOTRUN -> [ABORT][18] ([i915#8503])
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@gem_eio@in-flight-10ms.html

  * igt@gem_eio@reset-stress:
    - shard-dg2:          [PASS][19] -> [FAIL][20] ([i915#5784])
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-8/igt@gem_eio@reset-stress.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@gem_eio@reset-stress.html

  * igt@gem_exec_balancer@bonded-dual:
    - shard-mtlp:         NOTRUN -> [SKIP][21] ([i915#4771])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@gem_exec_balancer@bonded-dual.html

  * igt@gem_exec_balancer@bonded-pair:
    - shard-dg2:          NOTRUN -> [SKIP][22] ([i915#4771])
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@gem_exec_balancer@bonded-pair.html

  * igt@gem_exec_balancer@sliced:
    - shard-dg2:          NOTRUN -> [SKIP][23] ([i915#4812])
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@gem_exec_balancer@sliced.html

  * igt@gem_exec_capture@capture-invisible@smem0:
    - shard-mtlp:         NOTRUN -> [SKIP][24] ([i915#6334])
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@gem_exec_capture@capture-invisible@smem0.html

  * igt@gem_exec_capture@capture-recoverable:
    - shard-rkl:          NOTRUN -> [SKIP][25] ([i915#6344])
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-4/igt@gem_exec_capture@capture-recoverable.html

  * igt@gem_exec_fair@basic-deadline:
    - shard-glk:          [PASS][26] -> [FAIL][27] ([i915#2846])
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-glk4/igt@gem_exec_fair@basic-deadline.html
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-glk3/igt@gem_exec_fair@basic-deadline.html

  * igt@gem_exec_fair@basic-pace:
    - shard-dg2:          NOTRUN -> [SKIP][28] ([i915#3539])
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@gem_exec_fair@basic-pace.html

  * igt@gem_exec_fair@basic-pace@vcs0:
    - shard-rkl:          [PASS][29] -> [FAIL][30] ([i915#2842])
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-2/igt@gem_exec_fair@basic-pace@vcs0.html
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@gem_exec_fair@basic-pace@vcs0.html

  * igt@gem_exec_fair@basic-sync:
    - shard-mtlp:         NOTRUN -> [SKIP][31] ([i915#4473] / [i915#4771])
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@gem_exec_fair@basic-sync.html

  * igt@gem_exec_fence@concurrent:
    - shard-mtlp:         NOTRUN -> [SKIP][32] ([i915#4812])
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@gem_exec_fence@concurrent.html

  * igt@gem_exec_flush@basic-uc-pro-default:
    - shard-dg2:          NOTRUN -> [SKIP][33] ([i915#3539] / [i915#4852]) +5 similar issues
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@gem_exec_flush@basic-uc-pro-default.html

  * igt@gem_exec_reloc@basic-write-cpu-noreloc:
    - shard-mtlp:         NOTRUN -> [SKIP][34] ([i915#3281]) +1 similar issue
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@gem_exec_reloc@basic-write-cpu-noreloc.html

  * igt@gem_exec_reloc@basic-write-wc-noreloc:
    - shard-dg2:          NOTRUN -> [SKIP][35] ([i915#3281]) +4 similar issues
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@gem_exec_reloc@basic-write-wc-noreloc.html

  * igt@gem_exec_schedule@noreorder-priority@rcs0:
    - shard-mtlp:         NOTRUN -> [DMESG-FAIL][36] ([i915#9121])
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@gem_exec_schedule@noreorder-priority@rcs0.html

  * igt@gem_fence_thrash@bo-write-verify-y:
    - shard-mtlp:         NOTRUN -> [SKIP][37] ([i915#4860])
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@gem_fence_thrash@bo-write-verify-y.html

  * igt@gem_fenced_exec_thrash@2-spare-fences:
    - shard-dg2:          NOTRUN -> [SKIP][38] ([i915#4860]) +4 similar issues
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-1/igt@gem_fenced_exec_thrash@2-spare-fences.html

  * igt@gem_huc_copy@huc-copy:
    - shard-rkl:          NOTRUN -> [SKIP][39] ([i915#2190])
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@gem_huc_copy@huc-copy.html

  * igt@gem_lmem_swapping@heavy-random:
    - shard-mtlp:         NOTRUN -> [SKIP][40] ([i915#4613]) +1 similar issue
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@gem_lmem_swapping@heavy-random.html

  * igt@gem_lmem_swapping@massive-random:
    - shard-rkl:          NOTRUN -> [SKIP][41] ([i915#4613])
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-4/igt@gem_lmem_swapping@massive-random.html
    - shard-tglu:         NOTRUN -> [SKIP][42] ([i915#4613])
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-9/igt@gem_lmem_swapping@massive-random.html

  * igt@gem_lmem_swapping@smem-oom@lmem0:
    - shard-dg2:          NOTRUN -> [TIMEOUT][43] ([i915#5493])
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@gem_lmem_swapping@smem-oom@lmem0.html

  * igt@gem_mmap@big-bo:
    - shard-dg2:          NOTRUN -> [SKIP][44] ([i915#4083]) +4 similar issues
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-3/igt@gem_mmap@big-bo.html

  * igt@gem_mmap_gtt@cpuset-basic-small-copy:
    - shard-dg1:          NOTRUN -> [SKIP][45] ([i915#4077])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@gem_mmap_gtt@cpuset-basic-small-copy.html

  * igt@gem_mmap_gtt@cpuset-medium-copy-odd:
    - shard-mtlp:         NOTRUN -> [SKIP][46] ([i915#4077]) +6 similar issues
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@gem_mmap_gtt@cpuset-medium-copy-odd.html

  * igt@gem_mmap_gtt@zero-extend:
    - shard-dg2:          NOTRUN -> [SKIP][47] ([i915#4077]) +10 similar issues
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@gem_mmap_gtt@zero-extend.html

  * igt@gem_mmap_wc@fault-concurrent:
    - shard-mtlp:         NOTRUN -> [SKIP][48] ([i915#4083]) +1 similar issue
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@gem_mmap_wc@fault-concurrent.html

  * igt@gem_partial_pwrite_pread@write-display:
    - shard-rkl:          NOTRUN -> [SKIP][49] ([i915#3282]) +1 similar issue
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@gem_partial_pwrite_pread@write-display.html

  * igt@gem_pread@snoop:
    - shard-dg1:          NOTRUN -> [SKIP][50] ([i915#3282])
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-17/igt@gem_pread@snoop.html
    - shard-mtlp:         NOTRUN -> [SKIP][51] ([i915#3282]) +1 similar issue
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@gem_pread@snoop.html

  * igt@gem_pxp@create-regular-context-2:
    - shard-dg2:          NOTRUN -> [SKIP][52] ([i915#4270]) +1 similar issue
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@gem_pxp@create-regular-context-2.html

  * igt@gem_pxp@fail-invalid-protected-context:
    - shard-mtlp:         NOTRUN -> [SKIP][53] ([i915#4270])
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@gem_pxp@fail-invalid-protected-context.html

  * igt@gem_readwrite@beyond-eob:
    - shard-dg2:          NOTRUN -> [SKIP][54] ([i915#3282]) +2 similar issues
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@gem_readwrite@beyond-eob.html

  * igt@gem_render_copy@y-tiled-mc-ccs-to-y-tiled-ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][55] ([i915#8428]) +4 similar issues
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@gem_render_copy@y-tiled-mc-ccs-to-y-tiled-ccs.html

  * igt@gem_set_tiling_vs_blt@tiled-to-untiled:
    - shard-dg2:          NOTRUN -> [SKIP][56] ([i915#4079]) +1 similar issue
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-10/igt@gem_set_tiling_vs_blt@tiled-to-untiled.html

  * igt@gem_softpin@evict-snoop-interruptible:
    - shard-tglu:         NOTRUN -> [SKIP][57] ([fdo#109312])
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-6/igt@gem_softpin@evict-snoop-interruptible.html

  * igt@gem_userptr_blits@map-fixed-invalidate-overlap:
    - shard-dg2:          NOTRUN -> [SKIP][58] ([i915#3297] / [i915#4880])
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@gem_userptr_blits@map-fixed-invalidate-overlap.html

  * igt@gem_userptr_blits@readonly-unsync:
    - shard-tglu:         NOTRUN -> [SKIP][59] ([i915#3297])
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-7/igt@gem_userptr_blits@readonly-unsync.html

  * igt@gem_userptr_blits@unsync-unmap-after-close:
    - shard-dg2:          NOTRUN -> [SKIP][60] ([i915#3297]) +1 similar issue
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@gem_userptr_blits@unsync-unmap-after-close.html
    - shard-rkl:          NOTRUN -> [SKIP][61] ([i915#3297]) +1 similar issue
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@gem_userptr_blits@unsync-unmap-after-close.html

  * igt@gen7_exec_parse@basic-offset:
    - shard-dg2:          NOTRUN -> [SKIP][62] ([fdo#109289]) +3 similar issues
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@gen7_exec_parse@basic-offset.html
    - shard-rkl:          NOTRUN -> [SKIP][63] ([fdo#109289])
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@gen7_exec_parse@basic-offset.html

  * igt@gen9_exec_parse@allowed-single:
    - shard-apl:          [PASS][64] -> [ABORT][65] ([i915#5566])
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-apl3/igt@gen9_exec_parse@allowed-single.html
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-apl4/igt@gen9_exec_parse@allowed-single.html

  * igt@gen9_exec_parse@batch-zero-length:
    - shard-mtlp:         NOTRUN -> [SKIP][66] ([i915#2856]) +2 similar issues
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@gen9_exec_parse@batch-zero-length.html

  * igt@gen9_exec_parse@bb-chained:
    - shard-dg1:          NOTRUN -> [SKIP][67] ([i915#2527])
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@gen9_exec_parse@bb-chained.html

  * igt@gen9_exec_parse@bb-start-param:
    - shard-dg2:          NOTRUN -> [SKIP][68] ([i915#2856]) +3 similar issues
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@gen9_exec_parse@bb-start-param.html

  * igt@i915_hangman@detector@vcs0:
    - shard-mtlp:         [PASS][69] -> [FAIL][70] ([i915#8456]) +2 similar issues
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-3/igt@i915_hangman@detector@vcs0.html
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@i915_hangman@detector@vcs0.html

  * igt@i915_module_load@reload-with-fault-injection:
    - shard-mtlp:         NOTRUN -> [ABORT][71] ([i915#8489] / [i915#8668])
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@i915_module_load@reload-with-fault-injection.html

  * igt@i915_pm_backlight@basic-brightness:
    - shard-tglu:         NOTRUN -> [SKIP][72] ([i915#3546] / [i915#7561])
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-3/igt@i915_pm_backlight@basic-brightness.html

  * igt@i915_pm_backlight@fade-with-suspend:
    - shard-dg2:          NOTRUN -> [SKIP][73] ([i915#5354] / [i915#7561])
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@i915_pm_backlight@fade-with-suspend.html
    - shard-mtlp:         NOTRUN -> [INCOMPLETE][74] ([fdo#103375] / [i915#6311])
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@i915_pm_backlight@fade-with-suspend.html

  * igt@i915_pm_dc@dc6-dpms:
    - shard-dg2:          NOTRUN -> [SKIP][75] ([i915#5978])
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@i915_pm_dc@dc6-dpms.html

  * igt@i915_pm_dc@dc9-dpms:
    - shard-apl:          [PASS][76] -> [SKIP][77] ([fdo#109271])
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-apl4/igt@i915_pm_dc@dc9-dpms.html
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-apl6/igt@i915_pm_dc@dc9-dpms.html

  * igt@i915_pm_freq_mult@media-freq@gt0:
    - shard-rkl:          NOTRUN -> [SKIP][78] ([i915#6590])
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-2/igt@i915_pm_freq_mult@media-freq@gt0.html

  * igt@i915_pm_rc6_residency@media-rc6-accuracy:
    - shard-mtlp:         NOTRUN -> [SKIP][79] ([fdo#109289]) +1 similar issue
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@i915_pm_rc6_residency@media-rc6-accuracy.html

  * igt@i915_pm_rpm@modeset-non-lpsp:
    - shard-dg2:          [PASS][80] -> [SKIP][81] ([i915#1397])
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-1/igt@i915_pm_rpm@modeset-non-lpsp.html
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@i915_pm_rpm@modeset-non-lpsp.html

  * igt@i915_pm_rpm@modeset-non-lpsp-stress:
    - shard-rkl:          [PASS][82] -> [SKIP][83] ([i915#1397]) +1 similar issue
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-4/igt@i915_pm_rpm@modeset-non-lpsp-stress.html
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@i915_pm_rpm@modeset-non-lpsp-stress.html
    - shard-mtlp:         NOTRUN -> [SKIP][84] ([i915#1397])
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@i915_pm_rpm@modeset-non-lpsp-stress.html

  * igt@i915_pm_sseu@full-enable:
    - shard-dg2:          NOTRUN -> [SKIP][85] ([i915#4387])
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@i915_pm_sseu@full-enable.html

  * igt@i915_selftest@live@migrate:
    - shard-mtlp:         NOTRUN -> [DMESG-FAIL][86] ([i915#7699])
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@i915_selftest@live@migrate.html

  * igt@i915_suspend@fence-restore-tiled2untiled:
    - shard-snb:          NOTRUN -> [DMESG-WARN][87] ([i915#8841]) +6 similar issues
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-snb1/igt@i915_suspend@fence-restore-tiled2untiled.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-a-edp-1-4-rc_ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][88] ([i915#8502]) +11 similar issues
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-a-edp-1-4-rc_ccs.html

  * igt@kms_async_flips@async-flip-with-page-flip-events@pipe-d-hdmi-a-1-y-rc_ccs:
    - shard-dg1:          NOTRUN -> [SKIP][89] ([i915#8502]) +7 similar issues
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@kms_async_flips@async-flip-with-page-flip-events@pipe-d-hdmi-a-1-y-rc_ccs.html

  * igt@kms_atomic_transition@plane-all-modeset-transition-internal-panels:
    - shard-dg2:          NOTRUN -> [SKIP][90] ([i915#1769] / [i915#3555])
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-3/igt@kms_atomic_transition@plane-all-modeset-transition-internal-panels.html

  * igt@kms_big_fb@4-tiled-32bpp-rotate-270:
    - shard-rkl:          NOTRUN -> [SKIP][91] ([i915#5286])
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_big_fb@4-tiled-32bpp-rotate-270.html

  * igt@kms_big_fb@4-tiled-64bpp-rotate-180:
    - shard-mtlp:         [PASS][92] -> [FAIL][93] ([i915#5138])
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-7/igt@kms_big_fb@4-tiled-64bpp-rotate-180.html
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_big_fb@4-tiled-64bpp-rotate-180.html

  * igt@kms_big_fb@4-tiled-8bpp-rotate-90:
    - shard-mtlp:         NOTRUN -> [SKIP][94] ([fdo#111614])
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@kms_big_fb@4-tiled-8bpp-rotate-90.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-async-flip:
    - shard-tglu:         NOTRUN -> [SKIP][95] ([fdo#111615] / [i915#5286])
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-7/igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-async-flip.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-180-hflip:
    - shard-mtlp:         NOTRUN -> [FAIL][96] ([i915#5138])
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-180-hflip.html

  * igt@kms_big_fb@x-tiled-32bpp-rotate-270:
    - shard-dg2:          NOTRUN -> [SKIP][97] ([fdo#111614]) +5 similar issues
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@kms_big_fb@x-tiled-32bpp-rotate-270.html
    - shard-rkl:          NOTRUN -> [SKIP][98] ([fdo#111614] / [i915#3638])
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-2/igt@kms_big_fb@x-tiled-32bpp-rotate-270.html

  * igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-180-hflip-async-flip:
    - shard-rkl:          [PASS][99] -> [FAIL][100] ([i915#3743])
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-1/igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-180-hflip-async-flip.html
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_big_fb@x-tiled-max-hw-stride-32bpp-rotate-180-hflip-async-flip.html

  * igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-async-flip:
    - shard-mtlp:         [PASS][101] -> [FAIL][102] ([i915#3743])
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-3/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-async-flip.html
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-180-async-flip.html

  * igt@kms_big_fb@y-tiled-addfb-size-offset-overflow:
    - shard-mtlp:         NOTRUN -> [SKIP][103] ([i915#6187])
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@kms_big_fb@y-tiled-addfb-size-offset-overflow.html

  * igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-0-hflip:
    - shard-mtlp:         NOTRUN -> [SKIP][104] ([fdo#111615]) +2 similar issues
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@kms_big_fb@y-tiled-max-hw-stride-32bpp-rotate-0-hflip.html

  * igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip:
    - shard-dg2:          NOTRUN -> [SKIP][105] ([i915#5190]) +7 similar issues
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip.html

  * igt@kms_big_fb@yf-tiled-32bpp-rotate-90:
    - shard-dg2:          NOTRUN -> [SKIP][106] ([i915#4538] / [i915#5190]) +4 similar issues
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_big_fb@yf-tiled-32bpp-rotate-90.html

  * igt@kms_big_fb@yf-tiled-max-hw-stride-32bpp-rotate-180:
    - shard-rkl:          NOTRUN -> [SKIP][107] ([fdo#110723]) +2 similar issues
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-6/igt@kms_big_fb@yf-tiled-max-hw-stride-32bpp-rotate-180.html
    - shard-tglu:         NOTRUN -> [SKIP][108] ([fdo#111615])
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-9/igt@kms_big_fb@yf-tiled-max-hw-stride-32bpp-rotate-180.html

  * igt@kms_ccs@pipe-a-bad-rotation-90-4_tiled_dg2_rc_ccs:
    - shard-tglu:         NOTRUN -> [SKIP][109] ([i915#3689] / [i915#5354] / [i915#6095]) +2 similar issues
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-9/igt@kms_ccs@pipe-a-bad-rotation-90-4_tiled_dg2_rc_ccs.html

  * igt@kms_ccs@pipe-a-crc-sprite-planes-basic-4_tiled_dg2_mc_ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][110] ([i915#6095]) +17 similar issues
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@kms_ccs@pipe-a-crc-sprite-planes-basic-4_tiled_dg2_mc_ccs.html

  * igt@kms_ccs@pipe-b-bad-aux-stride-y_tiled_ccs:
    - shard-dg1:          NOTRUN -> [SKIP][111] ([i915#3689] / [i915#5354] / [i915#6095])
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@kms_ccs@pipe-b-bad-aux-stride-y_tiled_ccs.html

  * igt@kms_ccs@pipe-b-bad-aux-stride-yf_tiled_ccs:
    - shard-rkl:          NOTRUN -> [SKIP][112] ([i915#3734] / [i915#5354] / [i915#6095]) +1 similar issue
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-2/igt@kms_ccs@pipe-b-bad-aux-stride-yf_tiled_ccs.html

  * igt@kms_ccs@pipe-b-bad-pixel-format-4_tiled_dg2_rc_ccs:
    - shard-rkl:          NOTRUN -> [SKIP][113] ([i915#5354] / [i915#6095]) +6 similar issues
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@kms_ccs@pipe-b-bad-pixel-format-4_tiled_dg2_rc_ccs.html

  * igt@kms_ccs@pipe-b-bad-pixel-format-y_tiled_gen12_mc_ccs:
    - shard-mtlp:         NOTRUN -> [SKIP][114] ([i915#3886] / [i915#6095]) +5 similar issues
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@kms_ccs@pipe-b-bad-pixel-format-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-b-crc-primary-rotation-180-yf_tiled_ccs:
    - shard-tglu:         NOTRUN -> [SKIP][115] ([fdo#111615] / [i915#3689] / [i915#5354] / [i915#6095])
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-6/igt@kms_ccs@pipe-b-crc-primary-rotation-180-yf_tiled_ccs.html

  * igt@kms_ccs@pipe-c-crc-primary-basic-y_tiled_gen12_mc_ccs:
    - shard-tglu:         NOTRUN -> [SKIP][116] ([i915#3689] / [i915#3886] / [i915#5354] / [i915#6095])
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-8/igt@kms_ccs@pipe-c-crc-primary-basic-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs:
    - shard-dg2:          NOTRUN -> [SKIP][117] ([i915#3689] / [i915#5354]) +11 similar issues
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs.html

  * igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs_cc:
    - shard-dg2:          NOTRUN -> [SKIP][118] ([i915#3689] / [i915#3886] / [i915#5354]) +1 similar issue
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-3/igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_rc_ccs_cc.html

  * igt@kms_ccs@pipe-d-bad-aux-stride-4_tiled_mtl_rc_ccs_cc:
    - shard-tglu:         NOTRUN -> [SKIP][119] ([i915#5354] / [i915#6095]) +7 similar issues
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-7/igt@kms_ccs@pipe-d-bad-aux-stride-4_tiled_mtl_rc_ccs_cc.html

  * igt@kms_ccs@pipe-d-bad-pixel-format-4_tiled_dg2_mc_ccs:
    - shard-rkl:          NOTRUN -> [SKIP][120] ([i915#5354]) +9 similar issues
   [120]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-2/igt@kms_ccs@pipe-d-bad-pixel-format-4_tiled_dg2_mc_ccs.html

  * igt@kms_cdclk@plane-scaling@pipe-c-edp-1:
    - shard-mtlp:         NOTRUN -> [SKIP][121] ([i915#4087]) +3 similar issues
   [121]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@kms_cdclk@plane-scaling@pipe-c-edp-1.html

  * igt@kms_chamelium_color@ctm-green-to-red:
    - shard-dg2:          NOTRUN -> [SKIP][122] ([fdo#111827])
   [122]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@kms_chamelium_color@ctm-green-to-red.html
    - shard-rkl:          NOTRUN -> [SKIP][123] ([fdo#111827])
   [123]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_chamelium_color@ctm-green-to-red.html

  * igt@kms_chamelium_hpd@common-hpd-after-suspend:
    - shard-tglu:         NOTRUN -> [SKIP][124] ([i915#7828])
   [124]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-2/igt@kms_chamelium_hpd@common-hpd-after-suspend.html

  * igt@kms_chamelium_hpd@dp-hpd-fast:
    - shard-mtlp:         NOTRUN -> [SKIP][125] ([i915#7828]) +1 similar issue
   [125]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@kms_chamelium_hpd@dp-hpd-fast.html

  * igt@kms_chamelium_hpd@hdmi-hpd-with-enabled-mode:
    - shard-dg2:          NOTRUN -> [SKIP][126] ([i915#7828]) +7 similar issues
   [126]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@kms_chamelium_hpd@hdmi-hpd-with-enabled-mode.html
    - shard-rkl:          NOTRUN -> [SKIP][127] ([i915#7828]) +3 similar issues
   [127]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-6/igt@kms_chamelium_hpd@hdmi-hpd-with-enabled-mode.html

  * igt@kms_content_protection@atomic-dpms:
    - shard-dg2:          NOTRUN -> [SKIP][128] ([i915#7118]) +1 similar issue
   [128]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@kms_content_protection@atomic-dpms.html

  * igt@kms_content_protection@dp-mst-type-1:
    - shard-dg2:          NOTRUN -> [SKIP][129] ([i915#3299])
   [129]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@kms_content_protection@dp-mst-type-1.html

  * igt@kms_content_protection@mei_interface:
    - shard-mtlp:         NOTRUN -> [SKIP][130] ([i915#8063])
   [130]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_content_protection@mei_interface.html

  * igt@kms_content_protection@srm@pipe-a-dp-4:
    - shard-dg2:          NOTRUN -> [TIMEOUT][131] ([i915#7173])
   [131]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@kms_content_protection@srm@pipe-a-dp-4.html

  * igt@kms_content_protection@type1:
    - shard-dg2:          NOTRUN -> [SKIP][132] ([i915#7118] / [i915#7162])
   [132]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@kms_content_protection@type1.html

  * igt@kms_cursor_crc@cursor-offscreen-512x170:
    - shard-dg2:          NOTRUN -> [SKIP][133] ([i915#3359]) +2 similar issues
   [133]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_cursor_crc@cursor-offscreen-512x170.html

  * igt@kms_cursor_crc@cursor-onscreen-32x10:
    - shard-mtlp:         NOTRUN -> [SKIP][134] ([i915#8814]) +3 similar issues
   [134]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@kms_cursor_crc@cursor-onscreen-32x10.html

  * igt@kms_cursor_crc@cursor-random-max-size:
    - shard-tglu:         NOTRUN -> [SKIP][135] ([i915#3555])
   [135]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-6/igt@kms_cursor_crc@cursor-random-max-size.html

  * igt@kms_cursor_crc@cursor-sliding-512x512:
    - shard-mtlp:         NOTRUN -> [SKIP][136] ([i915#3359])
   [136]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@kms_cursor_crc@cursor-sliding-512x512.html

  * igt@kms_cursor_legacy@2x-flip-vs-cursor-atomic:
    - shard-dg1:          NOTRUN -> [SKIP][137] ([fdo#111767] / [fdo#111825])
   [137]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@kms_cursor_legacy@2x-flip-vs-cursor-atomic.html
    - shard-mtlp:         NOTRUN -> [SKIP][138] ([fdo#111767] / [i915#3546])
   [138]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_cursor_legacy@2x-flip-vs-cursor-atomic.html

  * igt@kms_cursor_legacy@cursora-vs-flipb-atomic:
    - shard-mtlp:         NOTRUN -> [SKIP][139] ([i915#3546]) +3 similar issues
   [139]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@kms_cursor_legacy@cursora-vs-flipb-atomic.html

  * igt@kms_cursor_legacy@cursorb-vs-flipb-atomic:
    - shard-dg2:          NOTRUN -> [SKIP][140] ([fdo#109274] / [i915#5354]) +3 similar issues
   [140]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@kms_cursor_legacy@cursorb-vs-flipb-atomic.html
    - shard-rkl:          NOTRUN -> [SKIP][141] ([fdo#111825]) +4 similar issues
   [141]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_cursor_legacy@cursorb-vs-flipb-atomic.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions:
    - shard-apl:          [PASS][142] -> [FAIL][143] ([i915#2346])
   [142]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-apl7/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html
   [143]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-apl4/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
    - shard-glk:          [PASS][144] -> [FAIL][145] ([i915#2346])
   [144]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-glk9/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
   [145]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-glk9/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html

  * igt@kms_cursor_legacy@short-busy-flip-before-cursor-toggle:
    - shard-dg2:          NOTRUN -> [SKIP][146] ([i915#4103] / [i915#4213]) +1 similar issue
   [146]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_cursor_legacy@short-busy-flip-before-cursor-toggle.html

  * igt@kms_dsc@dsc-with-formats:
    - shard-mtlp:         NOTRUN -> [SKIP][147] ([i915#3840])
   [147]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@kms_dsc@dsc-with-formats.html

  * igt@kms_fbcon_fbt@fbc-suspend:
    - shard-dg2:          [PASS][148] -> [FAIL][149] ([fdo#103375]) +1 similar issue
   [148]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-10/igt@kms_fbcon_fbt@fbc-suspend.html
   [149]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@kms_fbcon_fbt@fbc-suspend.html

  * igt@kms_flip@2x-absolute-wf_vblank-interruptible:
    - shard-tglu:         NOTRUN -> [SKIP][150] ([fdo#109274] / [i915#3637])
   [150]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-9/igt@kms_flip@2x-absolute-wf_vblank-interruptible.html

  * igt@kms_flip@2x-flip-vs-blocking-wf-vblank:
    - shard-mtlp:         NOTRUN -> [SKIP][151] ([fdo#111767] / [i915#3637])
   [151]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_flip@2x-flip-vs-blocking-wf-vblank.html

  * igt@kms_flip@2x-flip-vs-panning-vs-hang:
    - shard-dg2:          NOTRUN -> [SKIP][152] ([fdo#109274]) +6 similar issues
   [152]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@kms_flip@2x-flip-vs-panning-vs-hang.html

  * igt@kms_flip@2x-flip-vs-rmfb-interruptible:
    - shard-dg2:          NOTRUN -> [SKIP][153] ([fdo#109274] / [fdo#111767])
   [153]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@kms_flip@2x-flip-vs-rmfb-interruptible.html
    - shard-snb:          NOTRUN -> [SKIP][154] ([fdo#109271] / [fdo#111767])
   [154]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-snb6/igt@kms_flip@2x-flip-vs-rmfb-interruptible.html

  * igt@kms_flip@2x-plain-flip-ts-check:
    - shard-mtlp:         NOTRUN -> [SKIP][155] ([i915#3637])
   [155]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@kms_flip@2x-plain-flip-ts-check.html

  * igt@kms_flip@flip-vs-fences-interruptible:
    - shard-dg2:          NOTRUN -> [SKIP][156] ([i915#8381])
   [156]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@kms_flip@flip-vs-fences-interruptible.html

  * igt@kms_flip_scaled_crc@flip-32bpp-yftileccs-to-64bpp-yftile-downscaling@pipe-a-default-mode:
    - shard-mtlp:         NOTRUN -> [SKIP][157] ([i915#2672])
   [157]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@kms_flip_scaled_crc@flip-32bpp-yftileccs-to-64bpp-yftile-downscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-upscaling@pipe-a-valid-mode:
    - shard-dg2:          NOTRUN -> [SKIP][158] ([i915#2672]) +4 similar issues
   [158]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-upscaling@pipe-a-valid-mode.html
    - shard-rkl:          NOTRUN -> [SKIP][159] ([i915#2672])
   [159]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-upscaling@pipe-a-valid-mode.html

  * igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-wc:
    - shard-dg2:          NOTRUN -> [SKIP][160] ([i915#8708]) +9 similar issues
   [160]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-wc.html

  * igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-indfb-plflip-blt:
    - shard-dg2:          NOTRUN -> [SKIP][161] ([i915#5354]) +40 similar issues
   [161]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-indfb-plflip-blt.html
    - shard-rkl:          NOTRUN -> [SKIP][162] ([fdo#111825] / [i915#1825]) +14 similar issues
   [162]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-indfb-plflip-blt.html

  * igt@kms_frontbuffer_tracking@fbcpsr-1p-rte:
    - shard-rkl:          NOTRUN -> [SKIP][163] ([i915#3023]) +6 similar issues
   [163]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@kms_frontbuffer_tracking@fbcpsr-1p-rte.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-indfb-draw-mmap-gtt:
    - shard-mtlp:         NOTRUN -> [SKIP][164] ([i915#8708]) +4 similar issues
   [164]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-indfb-draw-mmap-gtt.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-shrfb-draw-mmap-wc:
    - shard-tglu:         NOTRUN -> [SKIP][165] ([fdo#109280]) +4 similar issues
   [165]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-7/igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-pri-shrfb-draw-mmap-wc.html

  * igt@kms_frontbuffer_tracking@fbcpsr-tiling-4:
    - shard-dg1:          NOTRUN -> [SKIP][166] ([i915#5439])
   [166]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-13/igt@kms_frontbuffer_tracking@fbcpsr-tiling-4.html

  * igt@kms_frontbuffer_tracking@fbcpsr-tiling-y:
    - shard-mtlp:         NOTRUN -> [SKIP][167] ([i915#5460])
   [167]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@kms_frontbuffer_tracking@fbcpsr-tiling-y.html

  * igt@kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-blt:
    - shard-dg2:          NOTRUN -> [SKIP][168] ([i915#3458]) +10 similar issues
   [168]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@kms_frontbuffer_tracking@psr-1p-primscrn-spr-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@psr-2p-primscrn-cur-indfb-draw-pwrite:
    - shard-mtlp:         NOTRUN -> [SKIP][169] ([i915#1825]) +10 similar issues
   [169]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@kms_frontbuffer_tracking@psr-2p-primscrn-cur-indfb-draw-pwrite.html

  * igt@kms_frontbuffer_tracking@psr-suspend:
    - shard-mtlp:         [PASS][170] -> [FAIL][171] ([fdo#103375])
   [170]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-4/igt@kms_frontbuffer_tracking@psr-suspend.html
   [171]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@kms_frontbuffer_tracking@psr-suspend.html

  * igt@kms_hdr@invalid-metadata-sizes:
    - shard-dg2:          NOTRUN -> [SKIP][172] ([i915#3555] / [i915#8228])
   [172]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-10/igt@kms_hdr@invalid-metadata-sizes.html

  * igt@kms_hdr@static-swap:
    - shard-mtlp:         NOTRUN -> [SKIP][173] ([i915#8228])
   [173]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@kms_hdr@static-swap.html

  * igt@kms_panel_fitting@legacy:
    - shard-dg2:          NOTRUN -> [SKIP][174] ([i915#6301])
   [174]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@kms_panel_fitting@legacy.html

  * igt@kms_pipe_crc_basic@suspend-read-crc@pipe-b-dp-4:
    - shard-dg2:          NOTRUN -> [FAIL][175] ([fdo#103375])
   [175]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-b-dp-4.html

  * igt@kms_plane@pixel-format-source-clamping@pipe-b-planes:
    - shard-mtlp:         [PASS][176] -> [FAIL][177] ([i915#1623])
   [176]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-7/igt@kms_plane@pixel-format-source-clamping@pipe-b-planes.html
   [177]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_plane@pixel-format-source-clamping@pipe-b-planes.html

  * igt@kms_plane_lowres@tiling-y:
    - shard-mtlp:         NOTRUN -> [SKIP][178] ([i915#8821])
   [178]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@kms_plane_lowres@tiling-y.html

  * igt@kms_plane_lowres@tiling-yf:
    - shard-dg2:          NOTRUN -> [SKIP][179] ([i915#3555] / [i915#8821])
   [179]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_plane_lowres@tiling-yf.html
    - shard-rkl:          NOTRUN -> [SKIP][180] ([i915#3555]) +2 similar issues
   [180]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-6/igt@kms_plane_lowres@tiling-yf.html

  * igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-b-dp-2:
    - shard-dg2:          NOTRUN -> [SKIP][181] ([i915#5176]) +7 similar issues
   [181]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-b-dp-2.html

  * igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-d-edp-1:
    - shard-mtlp:         NOTRUN -> [SKIP][182] ([i915#5176]) +5 similar issues
   [182]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-d-edp-1.html

  * igt@kms_plane_scaling@plane-upscale-with-rotation-20x20@pipe-a-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [SKIP][183] ([i915#5176]) +7 similar issues
   [183]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_plane_scaling@plane-upscale-with-rotation-20x20@pipe-a-hdmi-a-1.html

  * igt@kms_plane_scaling@plane-upscale-with-rotation-factor-0-25@pipe-b-hdmi-a-4:
    - shard-dg1:          NOTRUN -> [SKIP][184] ([i915#5176]) +11 similar issues
   [184]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-14/igt@kms_plane_scaling@plane-upscale-with-rotation-factor-0-25@pipe-b-hdmi-a-4.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-unity-scaling@pipe-c-hdmi-a-1:
    - shard-dg1:          NOTRUN -> [SKIP][185] ([i915#5235]) +11 similar issues
   [185]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@kms_plane_scaling@planes-downscale-factor-0-25-unity-scaling@pipe-c-hdmi-a-1.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-b-hdmi-a-1:
    - shard-rkl:          NOTRUN -> [SKIP][186] ([i915#5235]) +1 similar issue
   [186]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-b-hdmi-a-1.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-25@pipe-d-hdmi-a-3:
    - shard-dg2:          NOTRUN -> [SKIP][187] ([i915#5235]) +19 similar issues
   [187]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@kms_plane_scaling@planes-downscale-factor-0-25@pipe-d-hdmi-a-3.html

  * igt@kms_plane_scaling@planes-downscale-factor-0-5-unity-scaling@pipe-b-vga-1:
    - shard-snb:          NOTRUN -> [SKIP][188] ([fdo#109271]) +249 similar issues
   [188]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-snb2/igt@kms_plane_scaling@planes-downscale-factor-0-5-unity-scaling@pipe-b-vga-1.html

  * igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-c-edp-1:
    - shard-mtlp:         NOTRUN -> [SKIP][189] ([i915#5235]) +7 similar issues
   [189]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_plane_scaling@planes-upscale-factor-0-25-downscale-factor-0-5@pipe-c-edp-1.html

  * igt@kms_psr2_su@page_flip-xrgb8888:
    - shard-mtlp:         NOTRUN -> [SKIP][190] ([i915#4348]) +1 similar issue
   [190]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_psr2_su@page_flip-xrgb8888.html

  * igt@kms_psr@psr2_cursor_mmap_gtt:
    - shard-rkl:          NOTRUN -> [SKIP][191] ([i915#1072]) +2 similar issues
   [191]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@kms_psr@psr2_cursor_mmap_gtt.html

  * igt@kms_psr@psr2_sprite_plane_onoff:
    - shard-tglu:         NOTRUN -> [SKIP][192] ([fdo#110189]) +5 similar issues
   [192]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-3/igt@kms_psr@psr2_sprite_plane_onoff.html

  * igt@kms_psr@sprite_blt:
    - shard-dg2:          NOTRUN -> [SKIP][193] ([i915#1072]) +6 similar issues
   [193]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-2/igt@kms_psr@sprite_blt.html

  * igt@kms_psr_stress_test@invalidate-primary-flip-overlay:
    - shard-dg2:          NOTRUN -> [SKIP][194] ([i915#5461] / [i915#658])
   [194]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_psr_stress_test@invalidate-primary-flip-overlay.html

  * igt@kms_rotation_crc@bad-pixel-format:
    - shard-mtlp:         NOTRUN -> [SKIP][195] ([i915#4235])
   [195]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@kms_rotation_crc@bad-pixel-format.html

  * igt@kms_rotation_crc@primary-yf-tiled-reflect-x-270:
    - shard-rkl:          NOTRUN -> [SKIP][196] ([fdo#111615] / [i915#5289])
   [196]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-1/igt@kms_rotation_crc@primary-yf-tiled-reflect-x-270.html

  * igt@kms_selftest@drm_plane:
    - shard-mtlp:         NOTRUN -> [SKIP][197] ([i915#8661])
   [197]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@kms_selftest@drm_plane.html

  * igt@kms_setmode@invalid-clone-single-crtc:
    - shard-rkl:          NOTRUN -> [SKIP][198] ([i915#3555] / [i915#4098])
   [198]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-4/igt@kms_setmode@invalid-clone-single-crtc.html

  * igt@kms_sysfs_edid_timing:
    - shard-dg2:          NOTRUN -> [FAIL][199] ([IGT#2])
   [199]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-3/igt@kms_sysfs_edid_timing.html

  * igt@kms_tv_load_detect@load-detect:
    - shard-mtlp:         NOTRUN -> [SKIP][200] ([fdo#109309])
   [200]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_tv_load_detect@load-detect.html

  * igt@kms_universal_plane@universal-plane-pipe-c-sanity:
    - shard-rkl:          NOTRUN -> [SKIP][201] ([i915#4070] / [i915#6768])
   [201]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-4/igt@kms_universal_plane@universal-plane-pipe-c-sanity.html

  * igt@kms_vblank@pipe-d-query-idle-hang:
    - shard-rkl:          NOTRUN -> [SKIP][202] ([i915#4070] / [i915#533] / [i915#6768])
   [202]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-4/igt@kms_vblank@pipe-d-query-idle-hang.html

  * igt@kms_vrr@flipline:
    - shard-dg2:          NOTRUN -> [SKIP][203] ([i915#3555]) +1 similar issue
   [203]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-1/igt@kms_vrr@flipline.html

  * igt@perf@enable-disable@0-rcs0:
    - shard-dg2:          [PASS][204] -> [FAIL][205] ([i915#8724])
   [204]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-12/igt@perf@enable-disable@0-rcs0.html
   [205]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@perf@enable-disable@0-rcs0.html

  * igt@perf@gen8-unprivileged-single-ctx-counters:
    - shard-dg2:          NOTRUN -> [SKIP][206] ([i915#2436])
   [206]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@perf@gen8-unprivileged-single-ctx-counters.html

  * igt@perf@mi-rpc:
    - shard-mtlp:         NOTRUN -> [SKIP][207] ([i915#2434])
   [207]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-5/igt@perf@mi-rpc.html

  * igt@perf_pmu@frequency@gt0:
    - shard-dg2:          NOTRUN -> [FAIL][208] ([i915#6806])
   [208]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@perf_pmu@frequency@gt0.html

  * igt@perf_pmu@rc6@other-idle-gt0:
    - shard-dg2:          NOTRUN -> [SKIP][209] ([i915#8516])
   [209]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@perf_pmu@rc6@other-idle-gt0.html

  * igt@prime_mmap@test_aperture_limit@test_aperture_limit-smem:
    - shard-dg2:          NOTRUN -> [INCOMPLETE][210] ([i915#5493])
   [210]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-1/igt@prime_mmap@test_aperture_limit@test_aperture_limit-smem.html

  * igt@v3d/v3d_perfmon@get-values-valid-perfmon:
    - shard-mtlp:         NOTRUN -> [SKIP][211] ([i915#2575]) +4 similar issues
   [211]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@v3d/v3d_perfmon@get-values-valid-perfmon.html

  * igt@v3d/v3d_submit_cl@single-in-sync:
    - shard-rkl:          NOTRUN -> [SKIP][212] ([fdo#109315]) +2 similar issues
   [212]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@v3d/v3d_submit_cl@single-in-sync.html

  * igt@v3d/v3d_submit_cl@single-out-sync:
    - shard-tglu:         NOTRUN -> [SKIP][213] ([fdo#109315] / [i915#2575]) +2 similar issues
   [213]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-6/igt@v3d/v3d_submit_cl@single-out-sync.html

  * igt@v3d/v3d_submit_cl@valid-multisync-submission:
    - shard-dg2:          NOTRUN -> [SKIP][214] ([i915#2575]) +5 similar issues
   [214]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@v3d/v3d_submit_cl@valid-multisync-submission.html

  * igt@vc4/vc4_mmap@mmap-bo:
    - shard-dg2:          NOTRUN -> [SKIP][215] ([i915#7711]) +5 similar issues
   [215]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-3/igt@vc4/vc4_mmap@mmap-bo.html
    - shard-rkl:          NOTRUN -> [SKIP][216] ([i915#7711]) +3 similar issues
   [216]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@vc4/vc4_mmap@mmap-bo.html

  * igt@vc4/vc4_perfmon@destroy-valid-perfmon:
    - shard-dg1:          NOTRUN -> [SKIP][217] ([i915#7711])
   [217]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-14/igt@vc4/vc4_perfmon@destroy-valid-perfmon.html

  * igt@vc4/vc4_perfmon@get-values-valid-perfmon:
    - shard-tglu:         NOTRUN -> [SKIP][218] ([i915#2575])
   [218]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-7/igt@vc4/vc4_perfmon@get-values-valid-perfmon.html

  * igt@vc4/vc4_purgeable_bo@mark-purgeable-twice:
    - shard-mtlp:         NOTRUN -> [SKIP][219] ([i915#7711]) +2 similar issues
   [219]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@vc4/vc4_purgeable_bo@mark-purgeable-twice.html

  
#### Possible fixes ####

  * igt@gem_ctx_persistence@engines-hostile@vcs0:
    - shard-mtlp:         [FAIL][220] ([i915#2410]) -> [PASS][221] +2 similar issues
   [220]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-5/igt@gem_ctx_persistence@engines-hostile@vcs0.html
   [221]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-3/igt@gem_ctx_persistence@engines-hostile@vcs0.html

  * igt@gem_eio@in-flight-suspend:
    - shard-mtlp:         [FAIL][222] ([i915#6121] / [i915#7052]) -> [PASS][223]
   [222]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-5/igt@gem_eio@in-flight-suspend.html
   [223]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@gem_eio@in-flight-suspend.html

  * igt@gem_eio@kms:
    - shard-dg2:          [INCOMPLETE][224] ([i915#7892]) -> [PASS][225]
   [224]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-2/igt@gem_eio@kms.html
   [225]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@gem_eio@kms.html

  * igt@gem_exec_fair@basic-flow@rcs0:
    - shard-tglu:         [FAIL][226] ([i915#2842]) -> [PASS][227]
   [226]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-tglu-2/igt@gem_exec_fair@basic-flow@rcs0.html
   [227]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-9/igt@gem_exec_fair@basic-flow@rcs0.html

  * igt@gem_exec_fair@basic-pace-share@rcs0:
    - shard-glk:          [FAIL][228] ([i915#2842]) -> [PASS][229]
   [228]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-glk2/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [229]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-glk4/igt@gem_exec_fair@basic-pace-share@rcs0.html

  * igt@gem_exec_fair@basic-pace-solo@rcs0:
    - shard-rkl:          [FAIL][230] ([i915#2842]) -> [PASS][231]
   [230]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-4/igt@gem_exec_fair@basic-pace-solo@rcs0.html
   [231]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@gem_exec_fair@basic-pace-solo@rcs0.html

  * igt@gem_exec_schedule@preemptive-hang@vcs0:
    - shard-mtlp:         [FAIL][232] ([i915#9051]) -> [PASS][233]
   [232]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-7/igt@gem_exec_schedule@preemptive-hang@vcs0.html
   [233]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@gem_exec_schedule@preemptive-hang@vcs0.html

  * igt@gem_exec_suspend@basic-s0@smem:
    - shard-dg2:          [INCOMPLETE][234] ([i915#6311]) -> [PASS][235]
   [234]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-6/igt@gem_exec_suspend@basic-s0@smem.html
   [235]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-11/igt@gem_exec_suspend@basic-s0@smem.html

  * igt@i915_hangman@engine-engine-hang@vcs0:
    - shard-mtlp:         [FAIL][236] ([i915#7069]) -> [PASS][237]
   [236]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-1/igt@i915_hangman@engine-engine-hang@vcs0.html
   [237]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@i915_hangman@engine-engine-hang@vcs0.html

  * igt@i915_module_load@reload-with-fault-injection:
    - shard-dg2:          [DMESG-WARN][238] ([i915#7061] / [i915#8617]) -> [PASS][239]
   [238]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-12/igt@i915_module_load@reload-with-fault-injection.html
   [239]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-1/igt@i915_module_load@reload-with-fault-injection.html

  * igt@i915_pm_rc6_residency@rc6-fence:
    - shard-tglu:         [WARN][240] ([i915#2681]) -> [PASS][241]
   [240]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-tglu-5/igt@i915_pm_rc6_residency@rc6-fence.html
   [241]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-tglu-10/igt@i915_pm_rc6_residency@rc6-fence.html

  * igt@i915_pm_rc6_residency@rc6-idle@rcs0:
    - shard-dg1:          [FAIL][242] ([i915#3591]) -> [PASS][243] +1 similar issue
   [242]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg1-14/igt@i915_pm_rc6_residency@rc6-idle@rcs0.html
   [243]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-18/igt@i915_pm_rc6_residency@rc6-idle@rcs0.html

  * igt@i915_pm_rpm@dpms-lpsp:
    - shard-rkl:          [SKIP][244] ([i915#1397]) -> [PASS][245] +1 similar issue
   [244]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-1/igt@i915_pm_rpm@dpms-lpsp.html
   [245]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@i915_pm_rpm@dpms-lpsp.html

  * igt@i915_pm_rpm@dpms-mode-unset-lpsp:
    - shard-dg2:          [SKIP][246] ([i915#1397]) -> [PASS][247]
   [246]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-8/igt@i915_pm_rpm@dpms-mode-unset-lpsp.html
   [247]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-12/igt@i915_pm_rpm@dpms-mode-unset-lpsp.html

  * igt@i915_pm_rpm@dpms-non-lpsp:
    - shard-dg1:          [SKIP][248] ([i915#1397]) -> [PASS][249]
   [248]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg1-19/igt@i915_pm_rpm@dpms-non-lpsp.html
   [249]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-16/igt@i915_pm_rpm@dpms-non-lpsp.html

  * igt@i915_selftest@live@requests:
    - shard-mtlp:         [ABORT][250] ([i915#7920] / [i915#7982]) -> [PASS][251]
   [250]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-6/igt@i915_selftest@live@requests.html
   [251]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@i915_selftest@live@requests.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip:
    - shard-mtlp:         [FAIL][252] ([i915#3743]) -> [PASS][253] +1 similar issue
   [252]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-2/igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html
   [253]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_big_fb@4-tiled-max-hw-stride-32bpp-rotate-0-hflip-async-flip.html

  * igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip:
    - shard-mtlp:         [FAIL][254] ([i915#5138]) -> [PASS][255]
   [254]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-4/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip.html
   [255]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-1/igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-0-hflip.html

  * igt@kms_fbcon_fbt@fbc-suspend:
    - shard-mtlp:         [FAIL][256] ([fdo#103375]) -> [PASS][257] +2 similar issues
   [256]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-5/igt@kms_fbcon_fbt@fbc-suspend.html
   [257]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-7/igt@kms_fbcon_fbt@fbc-suspend.html

  * igt@kms_flip@flip-vs-suspend-interruptible@b-hdmi-a3:
    - shard-dg2:          [FAIL][258] ([fdo#103375]) -> [PASS][259]
   [258]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-5/igt@kms_flip@flip-vs-suspend-interruptible@b-hdmi-a3.html
   [259]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-6/igt@kms_flip@flip-vs-suspend-interruptible@b-hdmi-a3.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-shrfb-pgflip-blt:
    - shard-dg2:          [FAIL][260] ([i915#6880]) -> [PASS][261] +1 similar issue
   [260]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-11/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-shrfb-pgflip-blt.html
   [261]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-5/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-shrfb-pgflip-blt.html

  * igt@kms_vblank@pipe-a-ts-continuation-dpms-suspend:
    - shard-rkl:          [INCOMPLETE][262] -> [PASS][263]
   [262]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-6/igt@kms_vblank@pipe-a-ts-continuation-dpms-suspend.html
   [263]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-7/igt@kms_vblank@pipe-a-ts-continuation-dpms-suspend.html

  * igt@perf@non-zero-reason@0-rcs0:
    - shard-dg2:          [FAIL][264] ([i915#7484]) -> [PASS][265]
   [264]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg2-5/igt@perf@non-zero-reason@0-rcs0.html
   [265]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg2-8/igt@perf@non-zero-reason@0-rcs0.html

  * igt@perf_pmu@all-busy-idle-check-all:
    - shard-mtlp:         [FAIL][266] ([i915#5234]) -> [PASS][267]
   [266]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-7/igt@perf_pmu@all-busy-idle-check-all.html
   [267]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@perf_pmu@all-busy-idle-check-all.html

  * igt@perf_pmu@rc6@runtime-pm-long-gt1:
    - shard-mtlp:         [SKIP][268] ([i915#8537]) -> [PASS][269] +2 similar issues
   [268]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-4/igt@perf_pmu@rc6@runtime-pm-long-gt1.html
   [269]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@perf_pmu@rc6@runtime-pm-long-gt1.html

  * igt@perf_pmu@render-node-busy-idle@ccs0:
    - shard-mtlp:         [FAIL][270] ([i915#4349]) -> [PASS][271]
   [270]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-3/igt@perf_pmu@render-node-busy-idle@ccs0.html
   [271]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-2/igt@perf_pmu@render-node-busy-idle@ccs0.html

  * igt@sysfs_heartbeat_interval@precise@vecs0:
    - shard-mtlp:         [FAIL][272] ([i915#8332]) -> [PASS][273]
   [272]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-7/igt@sysfs_heartbeat_interval@precise@vecs0.html
   [273]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-8/igt@sysfs_heartbeat_interval@precise@vecs0.html

  * igt@sysfs_preempt_timeout@timeout@vecs0:
    - shard-mtlp:         [ABORT][274] ([i915#8521] / [i915#8865]) -> [PASS][275]
   [274]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-2/igt@sysfs_preempt_timeout@timeout@vecs0.html
   [275]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@sysfs_preempt_timeout@timeout@vecs0.html

  
#### Warnings ####

  * igt@gem_exec_suspend@basic-s0@smem:
    - shard-snb:          [DMESG-WARN][276] ([i915#8841]) -> [DMESG-FAIL][277] ([fdo#103375])
   [276]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-snb1/igt@gem_exec_suspend@basic-s0@smem.html
   [277]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-snb7/igt@gem_exec_suspend@basic-s0@smem.html

  * igt@gem_lmem_swapping@smem-oom@lmem0:
    - shard-dg1:          [TIMEOUT][278] ([i915#5493]) -> [DMESG-WARN][279] ([i915#4936] / [i915#5493])
   [278]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg1-17/igt@gem_lmem_swapping@smem-oom@lmem0.html
   [279]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-13/igt@gem_lmem_swapping@smem-oom@lmem0.html

  * igt@kms_async_flips@crc@pipe-a-edp-1:
    - shard-mtlp:         [DMESG-FAIL][280] ([i915#8561]) -> [DMESG-FAIL][281] ([i915#1982] / [i915#8561])
   [280]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-8/igt@kms_async_flips@crc@pipe-a-edp-1.html
   [281]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-6/igt@kms_async_flips@crc@pipe-a-edp-1.html

  * igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
    - shard-mtlp:         [DMESG-FAIL][282] ([i915#1982] / [i915#2017] / [i915#5954]) -> [DMESG-FAIL][283] ([i915#2017] / [i915#5954])
   [282]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-mtlp-4/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
   [283]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-mtlp-4/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html

  * igt@kms_fbcon_fbt@psr:
    - shard-rkl:          [SKIP][284] ([fdo#110189] / [i915#3955]) -> [SKIP][285] ([i915#3955])
   [284]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-2/igt@kms_fbcon_fbt@psr.html
   [285]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-6/igt@kms_fbcon_fbt@psr.html

  * igt@kms_force_connector_basic@force-load-detect:
    - shard-rkl:          [SKIP][286] ([fdo#109285]) -> [SKIP][287] ([fdo#109285] / [i915#4098])
   [286]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-7/igt@kms_force_connector_basic@force-load-detect.html
   [287]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-2/igt@kms_force_connector_basic@force-load-detect.html

  * igt@kms_multipipe_modeset@basic-max-pipe-crc-check:
    - shard-rkl:          [SKIP][288] ([i915#4816]) -> [SKIP][289] ([i915#4070] / [i915#4816])
   [288]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-rkl-4/igt@kms_multipipe_modeset@basic-max-pipe-crc-check.html
   [289]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-rkl-2/igt@kms_multipipe_modeset@basic-max-pipe-crc-check.html

  * igt@kms_psr@primary_mmap_gtt:
    - shard-dg1:          [SKIP][290] ([i915#1072] / [i915#4078]) -> [SKIP][291] ([i915#1072])
   [290]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg1-18/igt@kms_psr@primary_mmap_gtt.html
   [291]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-19/igt@kms_psr@primary_mmap_gtt.html

  * igt@kms_psr@primary_page_flip:
    - shard-dg1:          [SKIP][292] ([i915#1072]) -> [SKIP][293] ([i915#1072] / [i915#4078])
   [292]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13568/shard-dg1-15/igt@kms_psr@primary_page_flip.html
   [293]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/shard-dg1-16/igt@kms_psr@primary_page_flip.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [IGT#2]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/issues/2
  [fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109309]: https://bugs.freedesktop.org/show_bug.cgi?id=109309
  [fdo#109312]: https://bugs.freedesktop.org/show_bug.cgi?id=109312
  [fdo#109315]: https://bugs.freedesktop.org/show_bug.cgi?id=109315
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#110723]: https://bugs.freedesktop.org/show_bug.cgi?id=110723
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111767]: https://bugs.freedesktop.org/show_bug.cgi?id=111767
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
  [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
  [i915#1623]: https://gitlab.freedesktop.org/drm/intel/issues/1623
  [i915#1769]: https://gitlab.freedesktop.org/drm/intel/issues/1769
  [i915#1825]: https://gitlab.freedesktop.org/drm/intel/issues/1825
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#2017]: https://gitlab.freedesktop.org/drm/intel/issues/2017
  [i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2410]: https://gitlab.freedesktop.org/drm/intel/issues/2410
  [i915#2434]: https://gitlab.freedesktop.org/drm/intel/issues/2434
  [i915#2436]: https://gitlab.freedesktop.org/drm/intel/issues/2436
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2575]: https://gitlab.freedesktop.org/drm/intel/issues/2575
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2681]: https://gitlab.freedesktop.org/drm/intel/issues/2681
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2846]: https://gitlab.freedesktop.org/drm/intel/issues/2846
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#3023]: https://gitlab.freedesktop.org/drm/intel/issues/3023
  [i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3299]: https://gitlab.freedesktop.org/drm/intel/issues/3299
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3458]: https://gitlab.freedesktop.org/drm/intel/issues/3458
  [i915#3539]: https://gitlab.freedesktop.org/drm/intel/issues/3539
  [i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3591]: https://gitlab.freedesktop.org/drm/intel/issues/3591
  [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
  [i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3734]: https://gitlab.freedesktop.org/drm/intel/issues/3734
  [i915#3743]: https://gitlab.freedesktop.org/drm/intel/issues/3743
  [i915#3840]: https://gitlab.freedesktop.org/drm/intel/issues/3840
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
  [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
  [i915#4077]: https://gitlab.freedesktop.org/drm/intel/issues/4077
  [i915#4078]: https://gitlab.freedesktop.org/drm/intel/issues/4078
  [i915#4079]: https://gitlab.freedesktop.org/drm/intel/issues/4079
  [i915#4083]: https://gitlab.freedesktop.org/drm/intel/issues/4083
  [i915#4087]: https://gitlab.freedesktop.org/drm/intel/issues/4087
  [i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103
  [i915#4213]: https://gitlab.freedesktop.org/drm/intel/issues/4213
  [i915#4235]: https://gitlab.freedesktop.org/drm/intel/issues/4235
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4348]: https://gitlab.freedesktop.org/drm/intel/issues/4348
  [i915#4349]: https://gitlab.freedesktop.org/drm/intel/issues/4349
  [i915#4387]: https://gitlab.freedesktop.org/drm/intel/issues/4387
  [i915#4473]: https://gitlab.freedesktop.org/drm/intel/issues/4473
  [i915#4538]: https://gitlab.freedesktop.org/drm/intel/issues/4538
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4771]: https://gitlab.freedesktop.org/drm/intel/issues/4771
  [i915#4812]: https://gitlab.freedesktop.org/drm/intel/issues/4812
  [i915#4816]: https://gitlab.freedesktop.org/drm/intel/issues/4816
  [i915#4852]: https://gitlab.freedesktop.org/drm/intel/issues/4852
  [i915#4860]: https://gitlab.freedesktop.org/drm/intel/issues/4860
  [i915#4873]: https://gitlab.freedesktop.org/drm/intel/issues/4873
  [i915#4880]: https://gitlab.freedesktop.org/drm/intel/issues/4880
  [i915#4936]: https://gitlab.freedesktop.org/drm/intel/issues/4936
  [i915#5138]: https://gitlab.freedesktop.org/drm/intel/issues/5138
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5190]: https://gitlab.freedesktop.org/drm/intel/issues/5190
  [i915#5234]: https://gitlab.freedesktop.org/drm/intel/issues/5234
  [i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
  [i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
  [i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
  [i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
  [i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
  [i915#5439]: https://gitlab.freedesktop.org/drm/intel/issues/5439
  [i915#5460]: https://gitlab.freedesktop.org/drm/intel/issues/5460
  [i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
  [i915#5493]: https://gitlab.freedesktop.org/drm/intel/issues/5493
  [i915#5566]: https://gitlab.freedesktop.org/drm/intel/issues/5566
  [i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
  [i915#5954]: https://gitlab.freedesktop.org/drm/intel/issues/5954
  [i915#5978]: https://gitlab.freedesktop.org/drm/intel/issues/5978
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6121]: https://gitlab.freedesktop.org/drm/intel/issues/6121
  [i915#6187]: https://gitlab.freedesktop.org/drm/intel/issues/6187
  [i915#6301]: https://gitlab.freedesktop.org/drm/intel/issues/6301
  [i915#6311]: https://gitlab.freedesktop.org/drm/intel/issues/6311
  [i915#6334]: https://gitlab.freedesktop.org/drm/intel/issues/6334
  [i915#6344]: https://gitlab.freedesktop.org/drm/intel/issues/6344
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#6590]: https://gitlab.freedesktop.org/drm/intel/issues/6590
  [i915#6768]: https://gitlab.freedesktop.org/drm/intel/issues/6768
  [i915#6806]: https://gitlab.freedesktop.org/drm/intel/issues/6806
  [i915#6880]: https://gitlab.freedesktop.org/drm/intel/issues/6880
  [i915#7052]: https://gitlab.freedesktop.org/drm/intel/issues/7052
  [i915#7061]: https://gitlab.freedesktop.org/drm/intel/issues/7061
  [i915#7069]: https://gitlab.freedesktop.org/drm/intel/issues/7069
  [i915#7118]: https://gitlab.freedesktop.org/drm/intel/issues/7118
  [i915#7162]: https://gitlab.freedesktop.org/drm/intel/issues/7162
  [i915#7173]: https://gitlab.freedesktop.org/drm/intel/issues/7173
  [i915#7484]: https://gitlab.freedesktop.org/drm/intel/issues/7484
  [i915#7561]: https://gitlab.freedesktop.org/drm/intel/issues/7561
  [i915#7697]: https://gitlab.freedesktop.org/drm/intel/issues/7697
  [i915#7699]: https://gitlab.freedesktop.org/drm/intel/issues/7699
  [i915#7701]: https://gitlab.freedesktop.org/drm/intel/issues/7701
  [i915#7711]: https://gitlab.freedesktop.org/drm/intel/issues/7711
  [i915#7742]: https://gitlab.freedesktop.org/drm/intel/issues/7742
  [i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
  [i915#7892]: https://gitlab.freedesktop.org/drm/intel/issues/7892
  [i915#7920]: https://gitlab.freedesktop.org/drm/intel/issues/7920
  [i915#7982]: https://gitlab.freedesktop.org/drm/intel/issues/7982
  [i915#8063]: https://gitlab.freedesktop.org/drm/intel/issues/8063
  [i915#8228]: https://gitlab.freedesktop.org/drm/intel/issues/8228
  [i915#8332]: https://gitlab.freedesktop.org/drm/intel/issues/8332
  [i915#8381]: https://gitlab.freedesktop.org/drm/intel/issues/8381
  [i915#8411]: https://gitlab.freedesktop.org/drm/intel/issues/8411
  [i915#8414]: https://gitlab.freedesktop.org/drm/intel/issues/8414
  [i915#8428]: https://gitlab.freedesktop.org/drm/intel/issues/8428
  [i915#8456]: https://gitlab.freedesktop.org/drm/intel/issues/8456
  [i915#8489]: https://gitlab.freedesktop.org/drm/intel/issues/8489
  [i915#8502]: https://gitlab.freedesktop.org/drm/intel/issues/8502
  [i915#8503]: https://gitlab.freedesktop.org/drm/intel/issues/8503
  [i915#8516]: https://gitlab.freedesktop.org/drm/intel/issues/8516
  [i915#8521]: https://gitlab.freedesktop.org/drm/intel/issues/8521
  [i915#8537]: https://gitlab.freedesktop.org/drm/intel/issues/8537
  [i915#8555]: https://gitlab.freedesktop.org/drm/intel/issues/8555
  [i915#8561]: https://gitlab.freedesktop.org/drm/intel/issues/8561
  [i915#8562]: https://gitlab.freedesktop.org/drm/intel/issues/8562
  [i915#8617]: https://gitlab.freedesktop.org/drm/intel/issues/8617
  [i915#8661]: https://gitlab.freedesktop.org/drm/intel/issues/8661
  [i915#8668]: https://gitlab.freedesktop.org/drm/intel/issues/8668
  [i915#8708]: https://gitlab.freedesktop.org/drm/intel/issues/8708
  [i915#8724]: https://gitlab.freedesktop.org/drm/intel/issues/8724
  [i915#8814]: https://gitlab.freedesktop.org/drm/intel/issues/8814
  [i915#8821]: https://gitlab.freedesktop.org/drm/intel/issues/8821
  [i915#8841]: https://gitlab.freedesktop.org/drm/intel/issues/8841
  [i915#8865]: https://gitlab.freedesktop.org/drm/intel/issues/8865
  [i915#9051]: https://gitlab.freedesktop.org/drm/intel/issues/9051
  [i915#9053]: https://gitlab.freedesktop.org/drm/intel/issues/9053
  [i915#9121]: https://gitlab.freedesktop.org/drm/intel/issues/9121


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7453 -> IGTPW_9663
  * Piglit: piglit_4509 -> None

  CI-20190529: 20190529
  CI_DRM_13568: f8b90de70ee92dbdebc6f2078e2edd12756d7a63 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_9663: 9663
  IGT_7453: 7453
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9663/index.html

[-- Attachment #2: Type: text/html, Size: 97969 bytes --]