* [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
@ 2023-12-08 4:18 Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Bommu Krishnaiah @ 2023-12-08 4:18 UTC (permalink / raw)
To: igt-dev; +Cc: Bommu Krishnaiah
remove the num_engines/instances members from drm_xe_wait_user_fence structure
and add a exec_queue_id member
invalid-exec_queue-wait subtest to excess behaviour when exec_queue reset happen
about test
Skipping the GPU mapping(vm_bind) for object, so that exec_queue
reset will happen and xe_wait_ufence will end return EIO not ETIME
v3: fixed build failures
NOTE: Corresponding driver changes is,
kmd commit: drm/xe/uapi: Return correct error code for xe_wait_user_fence_ioctl
I am able to see exec_queue reset was happened and xe_wait_user_fence_ioctl returned EIO
test result
root@DUT7075PVC:/home/gta# LD_LIBRARY_PATH=/home/gta/ ./xe_waitfence --r invalid-exec_queue-wait
IGT-Version: 1.28-g3c0162fc4 (x86_64) (Linux: 6.6.0-rc3-xe x86_64)
Opened device: /dev/dri/card0
Starting subtest: invalid-exec_queue-wait
Subtest invalid-exec_queue-wait: SUCCESS (0.993s)
dmesg logs
[ 807.680378] [IGT] xe_waitfence: executing
[ 807.699796] [drm:drm_stub_open [drm]]
[ 807.704536] xe 0000:51:00.0: [drm:drm_open_helper [drm]] comm="xe_waitfence", pid=2952, minor=0
[ 807.715155] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 807.727328] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 807.739580] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.751518] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.763550] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.775525] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.787556] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.799494] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.811531] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.823476] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.835577] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 807.847921] [IGT] xe_waitfence: starting subtest invalid-exec_queue-wait
[ 807.855528] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_VM_CREATE
[ 807.891346] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: Applying GT save-restore MMIOs
[ 807.901602] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: REG[0x9424] = 0x7ffffffc
-----------------
-----------------
[ 808.560967] xe REG[0x4500-0x45ff]: deny rw access
[ 808.566292] xe REG[0x1e3a8-0x1e3af]: allow read access
[ 808.572161] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: Applying ccs3 save-restore MMIOs
[ 808.582462] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: REG[0x260c4] = 0x3f7e0104
[ 808.592096] xe 0000:51:00.0: [drm:xe_reg_sr_apply_whitelist [xe]] Whitelisting ccs3 registers
[ 808.601962] xe REG[0x4400-0x45ff]: deny rw access
[ 808.607281] xe REG[0x4500-0x45ff]: deny rw access
[ 808.612608] xe REG[0x263a8-0x263af]: allow read access
[ 808.618477] xe 0000:51:00.0: [drm] GT0: resumed
[ 808.626283] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_EXEC_QUEUE_CREATE
[ 808.638765] krishna xe_exec_queue_create_ioctl
[ 808.645592] krishna args->exec_queue_id = 1
[ 808.650328] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_GEM_CREATE
[ 808.662621] xe 0000:51:00.0: [drm:xe_migrate_clear [xe]] Pass 0, size: 262144
[ 808.672889] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_GEM_MMAP_OFFSET
[ 808.685733] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_EXEC
[ 808.696900] krishna args->exec_queue_id = 1
[ 808.702700] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_WAIT_USER_FENCE
[ 808.704620] xe 0000:51:00.0: [drm:pf_queue_work_func [xe]]
ASID: 1048575
VFID: 0
PDATA: 0x00a3
Faulted Address: 0x00000000001a0000
FaultType: 0
AccessType: 0
FaultLevel: 4
EngineClass: 3
EngineInstance: 0
[ 808.750685] xe 0000:51:00.0: [drm:pf_queue_work_func [xe]] Fault response: Unsuccessful -22
[ 808.760519] xe 0000:51:00.0: [drm:xe_guc_exec_queue_memory_cat_error_handler [xe]] Engine memory cat error: guc_id=2
[ 808.773237] xe 0000:51:00.0: [drm] exec gueue reset detected
[ 808.773965] xe 0000:51:00.0: [drm] Timedout job: seqno=4294967169, guc_id=2, flags=0x8
[ 808.779632] xe 0000:51:00.0: [drm:xe_wait_user_fence_ioctl [xe]] Ioctl argument check failed at drivers/gpu/drm/xe/xe_wait_user_fence.c:174: err < 0
[ 808.789655] xe 0000:51:00.0: [drm] Xe device coredump has been created
[ 808.803796] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence", pid=2952, ret=-5
[ 808.811133] xe 0000:51:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data
[ 808.811220] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_EXEC_QUEUE_DESTROY
[ 808.823605] xe 0000:51:00.0: [drm] Engine reset: guc_id=2
[ 808.829862] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_GEM_CLOSE
[ 808.843312] xe 0000:51:00.0: [drm:guc_exec_queue_timedout_job [xe]] Timedout signaled job: seqno=4294967169, guc_id=2, flags=0x9
[ 808.848707] [IGT] xe_waitfence: finished subtest invalid-exec_queue-wait, SUCCESS
[ 808.882255] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 808.894404] xe 0000:51:00.0: [drm:drm_file_free.part.0 [drm]] comm="xe_waitfence", pid=2952, dev=0xe200, open_count=1
[ 808.907374] xe 0000:51:00.0: [drm:drm_lastclose [drm]]
[ 808.913594] xe 0000:51:00.0: [drm:drm_lastclose [drm]] driver lastclose completed
Bommu Krishnaiah (2):
drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence
structure
drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset
happen
include/drm-uapi/xe_drm.h | 28 ++++----
lib/xe/xe_ioctl.c | 27 +++-----
lib/xe/xe_ioctl.h | 9 +--
tests/intel/xe_evict.c | 4 +-
tests/intel/xe_exec_balancer.c | 15 ++--
tests/intel/xe_exec_compute_mode.c | 18 ++---
tests/intel/xe_exec_fault_mode.c | 19 ++---
tests/intel/xe_exec_reset.c | 6 +-
tests/intel/xe_exec_threads.c | 15 ++--
tests/intel/xe_waitfence.c | 107 +++++++++++++++++++++++++----
10 files changed, 161 insertions(+), 87 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
@ 2023-12-08 4:18 ` Bommu Krishnaiah
2023-12-08 5:57 ` Rodrigo Vivi
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
` (2 subsequent siblings)
3 siblings, 1 reply; 14+ messages in thread
From: Bommu Krishnaiah @ 2023-12-08 4:18 UTC (permalink / raw)
To: igt-dev; +Cc: Bommu Krishnaiah, Rodrigo Vivi
remove the num_engines/instances members from drm_xe_wait_user_fence structure
and add a exec_queue_id member
Right now this is only checking if the engine list is sane and nothing
else. In the end every operation with this IOCTL is a soft check.
So, let's formalize that and only use this IOCTL to wait on the fence.
exec_queue_id member will help to user space to get proper error code
from kernel while in exec_queue reset
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
---
include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
lib/xe/xe_ioctl.c | 27 +++++++++++----------------
lib/xe/xe_ioctl.h | 9 +++------
tests/intel/xe_evict.c | 4 ++--
tests/intel/xe_exec_balancer.c | 15 ++++++++-------
tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
tests/intel/xe_exec_reset.c | 6 +++---
tests/intel/xe_exec_threads.c | 15 ++++++++-------
tests/intel/xe_waitfence.c | 25 ++++++++++---------------
10 files changed, 79 insertions(+), 87 deletions(-)
diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
index 590f7b7af..fd06e4920 100644
--- a/include/drm-uapi/xe_drm.h
+++ b/include/drm-uapi/xe_drm.h
@@ -129,7 +129,6 @@ struct xe_user_extension {
* It is returned as part of the @drm_xe_engine, but it also is used as
* the input of engine selection for both @drm_xe_exec_queue_create and
* @drm_xe_query_engine_cycles
- *
*/
struct drm_xe_engine_class_instance {
#define DRM_XE_ENGINE_CLASS_RENDER 0
@@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
*/
#define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
#define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
+ /** @engine_class: engine class id */
__u16 engine_class;
-
+ /** @engine_instance: engine instance id */
__u16 engine_instance;
+ /** @gt_id: Unique ID of this GT within the PCI Device */
__u16 gt_id;
/** @pad: MBZ */
__u16 pad;
@@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
*
* Note: For userptr and externally imported dma-buf the kernel expects
* either 1WAY or 2WAY for the @pat_index.
+ *
+ * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD restrictions
+ * on the @pat_index. For such mappings there is no actual memory being
+ * mapped (the address in the PTE is invalid), so the various PAT memory
+ * attributes likely do not apply. Simply leaving as zero is one
+ * option (still a valid pat_index).
*/
__u16 pat_index;
@@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
/** @op: wait operation (type of comparison) */
__u16 op;
-#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait on VM bind */
-#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
+#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
/** @flags: wait flags */
__u16 flags;
@@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
*/
__s64 timeout;
- /**
- * @num_engines: number of engine instances to wait on, must be zero
- * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
- */
- __u64 num_engines;
+ /** @exec_queue_id: exec_queue_id returned from xe_exec_queue_create_ioctl */
+ __u32 exec_queue_id;
- /**
- * @instances: user pointer to array of drm_xe_engine_class_instance to
- * wait on, must be NULL when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
- */
- __u64 instances;
+ /** @pad2: MBZ */
+ __u32 pad2;
/** @reserved: Reserved */
__u64 reserved[2];
diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c
index b29ca40ad..0a1ddc86b 100644
--- a/lib/xe/xe_ioctl.c
+++ b/lib/xe/xe_ioctl.c
@@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
* @fd: xe device fd
* @addr: address of value to compare
* @value: expected value (equal) in @address
- * @eci: engine class instance
+ * @exec_queue: exec_queue id
* @timeout: pointer to time to wait in nanoseconds
*
* Function compares @value with memory pointed by @addr until they are equal.
@@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
* signalled. Returns 0 on success, -errno of ioctl on error.
*/
int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t *timeout)
+ uint32_t exec_queue, int64_t *timeout)
{
struct drm_xe_wait_user_fence wait = {
.addr = to_user_pointer(addr),
.op = DRM_XE_UFENCE_WAIT_OP_EQ,
- .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
+ .flags = 0,
.value = value,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
- .num_engines = eci ? 1 :0,
- .instances = eci ? to_user_pointer(eci) : 0,
+ .exec_queue_id = exec_queue,
};
igt_assert(timeout);
@@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* @fd: xe device fd
* @addr: address of value to compare
* @value: expected value (equal) in @address
- * @eci: engine class instance
+ * @exec_queue: exec_queue id
* @timeout: time to wait in nanoseconds
*
* Function compares @value with memory pointed by @addr until they are equal.
@@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* Returns elapsed time in nanoseconds if user fence was signalled.
*/
int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout)
+ uint32_t exec_queue, int64_t timeout)
{
- igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
+ igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue, &timeout), 0);
return timeout;
}
@@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* @fd: xe device fd
* @addr: address of value to compare
* @value: expected value (equal) in @address
- * @eci: engine class instance
+ * @exec_queue: exec_queue id
* @timeout: absolute time when wait expire
*
* Function compares @value with memory pointed by @addr until they are equal.
@@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* Returns elapsed time in nanoseconds if user fence was signalled.
*/
int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout)
+ uint32_t exec_queue, int64_t timeout)
{
struct drm_xe_wait_user_fence wait = {
.addr = to_user_pointer(addr),
.op = DRM_XE_UFENCE_WAIT_OP_EQ,
- .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP | DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
+ .flags = !exec_queue ? DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
.value = value,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = timeout,
- .num_engines = eci ? 1 : 0,
- .instances = eci ? to_user_pointer(eci) : 0,
+ .exec_queue_id = exec_queue,
};
struct timespec ts;
diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h
index bd660bd27..9c0b4b9bc 100644
--- a/lib/xe/xe_ioctl.h
+++ b/lib/xe/xe_ioctl.h
@@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue, uint64_t addr,
struct drm_xe_sync *sync, uint32_t num_syncs);
void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr);
int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t *timeout);
+ uint32_t exec_queue, int64_t *timeout);
int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout);
+ uint32_t exec_queue, int64_t timeout);
int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout);
+ uint32_t exec_queue, int64_t timeout);
void xe_force_gt_reset(int fd, int gt);
#endif /* XE_IOCTL_H */
diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
index 89dc46fae..a37c2d97a 100644
--- a/tests/intel/xe_evict.c
+++ b/tests/intel/xe_evict.c
@@ -317,7 +317,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
}
#define TWENTY_SEC MS_TO_NS(20000)
xe_wait_ufence(fd, &data[i].vm_sync, USER_FENCE_VALUE,
- NULL, TWENTY_SEC);
+ bind_exec_queues[0], TWENTY_SEC);
}
sync[0].addr = addr + (char *)&data[i].exec_sync -
(char *)data;
@@ -352,7 +352,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
data = xe_bo_map(fd, __bo,
ALIGN(sizeof(*data) * n_execs, 0x1000));
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, TWENTY_SEC);
+ exec_queues[i], TWENTY_SEC);
igt_assert_eq(data[i].data, 0xc0ffee);
}
munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000));
diff --git a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
index 79ff65e89..2448d49bc 100644
--- a/tests/intel/xe_exec_balancer.c
+++ b/tests/intel/xe_exec_balancer.c
@@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
bo_size, sync, 1);
#define ONE_SEC MS_TO_NS(1000)
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
if (flags & REBIND && i + 1 != n_execs) {
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ exec_queues[e], ONE_SEC);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, NULL,
0);
@@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ 0, ONE_SEC);
data[0].vm_sync = 0;
}
@@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
* an invalidate.
*/
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL, ONE_SEC);
+ USER_FENCE_VALUE, exec_queues[e],
+ ONE_SEC);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
/*
@@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
for (i = j; i < n_execs; i++)
- xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
- ONE_SEC);
+ xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
+ exec_queues[i], ONE_SEC);
/* Wait for all execs to complete */
if (flags & INVALIDATE)
@@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
i < n_execs; i++)
diff --git a/tests/intel/xe_exec_compute_mode.c b/tests/intel/xe_exec_compute_mode.c
index 7d3004d65..749a9b634 100644
--- a/tests/intel/xe_exec_compute_mode.c
+++ b/tests/intel/xe_exec_compute_mode.c
@@ -171,8 +171,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC : ONE_SEC;
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], fence_timeout);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -198,7 +198,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
if (flags & REBIND && i + 1 != n_execs) {
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, fence_timeout);
+ exec_queues[e], fence_timeout);
xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
addr, bo_size, NULL, 0);
@@ -214,7 +214,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, fence_timeout);
+ bind_exec_queues[e], fence_timeout);
data[0].vm_sync = 0;
}
@@ -227,7 +227,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
* an invalidate.
*/
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL,
+ USER_FENCE_VALUE, exec_queues[e],
fence_timeout);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
@@ -257,8 +257,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
j = flags & INVALIDATE ? n_execs - 1 : 0;
for (i = j; i < n_execs; i++)
- xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
+ exec_queues[i], fence_timeout);
/* Wait for all execs to complete */
if (flags & INVALIDATE)
@@ -267,8 +267,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], fence_timeout);
for (i = j; i < n_execs; i++)
igt_assert_eq(data[i].data, 0xc0ffee);
diff --git a/tests/intel/xe_exec_fault_mode.c b/tests/intel/xe_exec_fault_mode.c
index ee7cbb604..d76b3a056 100644
--- a/tests/intel/xe_exec_fault_mode.c
+++ b/tests/intel/xe_exec_fault_mode.c
@@ -195,15 +195,16 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
}
#define ONE_SEC MS_TO_NS(1000)
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], ONE_SEC);
data[0].vm_sync = 0;
if (flags & PREFETCH) {
/* Should move to system memory */
xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0, addr,
bo_size, sync, 1, 0);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
- ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], ONE_SEC);
data[0].vm_sync = 0;
}
@@ -230,7 +231,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
if (flags & REBIND && i + 1 != n_execs) {
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ exec_queues[e], ONE_SEC);
xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
addr, bo_size, NULL, 0);
@@ -246,7 +247,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ bind_exec_queues[e], ONE_SEC);
data[0].vm_sync = 0;
}
@@ -259,7 +260,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
* an invalidate.
*/
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL, ONE_SEC);
+ USER_FENCE_VALUE, exec_queues[e],
+ ONE_SEC);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
/*
@@ -290,13 +292,14 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
j = flags & INVALIDATE ? n_execs - 1 : 0;
for (i = j; i < n_execs; i++)
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL, ONE_SEC);
+ USER_FENCE_VALUE, exec_queues[i], ONE_SEC);
}
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], ONE_SEC);
if (!(flags & INVALID_FAULT)) {
for (i = j; i < n_execs; i++)
diff --git a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
index 094b34896..398c90af5 100644
--- a/tests/intel/xe_exec_reset.c
+++ b/tests/intel/xe_exec_reset.c
@@ -564,7 +564,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
#define THREE_SEC MS_TO_NS(3000)
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -621,7 +621,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
int err;
err = __xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, &timeout);
+ exec_queues[i], &timeout);
if (flags & GT_RESET)
/* exec races with reset: may timeout or complete */
igt_assert(err == -ETIME || !err);
@@ -631,7 +631,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
if (!(flags & GT_RESET)) {
for (i = 1; i < n_execs; i++)
diff --git a/tests/intel/xe_exec_threads.c b/tests/intel/xe_exec_threads.c
index fcb926698..7985240c9 100644
--- a/tests/intel/xe_exec_threads.c
+++ b/tests/intel/xe_exec_threads.c
@@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
for (j = i - 0x20; j <= i; ++j)
xe_wait_ufence(fd, &data[j].exec_sync,
USER_FENCE_VALUE,
- NULL, fence_timeout);
+ exec_queues[e], fence_timeout);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
NULL, 0);
@@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, fence_timeout);
+ 0, fence_timeout);
data[0].vm_sync = 0;
}
@@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
xe_wait_ufence(fd, &data[j].exec_sync,
USER_FENCE_VALUE,
- NULL, fence_timeout);
+ exec_queues[e],
+ fence_timeout);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
/*
@@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
j = flags & INVALIDATE ?
(flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
for (i = j; i < n_execs; i++)
- xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
+ exec_queues[e], fence_timeout);
/* Wait for all execs to complete */
if (flags & INVALIDATE)
@@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
for (i = j; i < n_execs; i++)
igt_assert_eq(data[i].data, 0xc0ffee);
diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
index 3be987954..4e94403a3 100644
--- a/tests/intel/xe_waitfence.c
+++ b/tests/intel/xe_waitfence.c
@@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm, uint32_t bo, uint64_t offset,
}
static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout)
+ uint32_t exec_queue, int64_t timeout)
{
struct drm_xe_wait_user_fence wait = {
.addr = to_user_pointer(addr),
.op = DRM_XE_UFENCE_WAIT_OP_EQ,
- .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
+ .flags = !exec_queue ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
.value = value,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = timeout,
- .num_engines = eci ? 1 : 0,
- .instances = eci ? to_user_pointer(eci) : 0,
+ .exec_queue_id = exec_queue,
};
struct timespec ts;
- igt_assert(eci);
igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait), 0);
igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
@@ -82,7 +79,7 @@ enum waittype {
static void
waitfence(int fd, enum waittype wt)
{
- struct drm_xe_engine *engine = NULL;
+ uint32_t exec_queue;
struct timespec ts;
int64_t current, signalled;
uint32_t bo_1;
@@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
if (wt == RELTIME) {
- timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL, MS_TO_NS(10));
+ timeout = xe_wait_ufence(fd, &wait_fence, 7, 0, MS_TO_NS(10));
igt_debug("wait type: RELTIME - timeout: %ld, timeout left: %ld\n",
MS_TO_NS(10), timeout);
} else if (wt == ENGINE) {
- engine = xe_engine(fd, 1);
+ exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
clock_gettime(CLOCK_MONOTONIC, &ts);
current = ts.tv_sec * 1e9 + ts.tv_nsec;
timeout = current + MS_TO_NS(10);
- signalled = wait_with_eci_abstime(fd, &wait_fence, 7, &engine->instance, timeout);
+ signalled = wait_with_eci_abstime(fd, &wait_fence, 7, exec_queue, timeout);
igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
", signalled: %" PRId64
", elapsed: %" PRId64 "\n",
@@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
clock_gettime(CLOCK_MONOTONIC, &ts);
current = ts.tv_sec * 1e9 + ts.tv_nsec;
timeout = current + MS_TO_NS(10);
- signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, NULL, timeout);
+ signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0, timeout);
igt_debug("wait type: ABSTIME - timeout: %" PRId64
", signalled: %" PRId64
", elapsed: %" PRId64 "\n",
@@ -191,8 +188,7 @@ invalid_ops(int fd)
.value = 1,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = 1,
- .num_engines = 0,
- .instances = 0,
+ .exec_queue_id = 0,
};
uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
@@ -216,8 +212,7 @@ invalid_engine(int fd)
.value = 1,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = -1,
- .num_engines = 1,
- .instances = 0,
+ .exec_queue_id = 0,
};
uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
@ 2023-12-08 4:18 ` Bommu Krishnaiah
2023-12-08 6:00 ` Rodrigo Vivi
2023-12-08 5:48 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5) Patchwork
2023-12-08 13:16 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6) Patchwork
3 siblings, 1 reply; 14+ messages in thread
From: Bommu Krishnaiah @ 2023-12-08 4:18 UTC (permalink / raw)
To: igt-dev; +Cc: Bommu Krishnaiah, Rodrigo Vivi
Skipping the GPU mapping(vm_bind) for object, so that exec_queue
reset will happen and xe_wait_ufence will end return EIO not ETIME
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
---
tests/intel/xe_waitfence.c | 82 ++++++++++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
index 4e94403a3..bd95fa1f8 100644
--- a/tests/intel/xe_waitfence.c
+++ b/tests/intel/xe_waitfence.c
@@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
*
* SUBTEST: invalid-engine
* Description: Check query with invalid engine info returns expected error code
+ *
+ * SUBTEST: invalid-exec_queue-wait
+ * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
*/
static void
@@ -224,6 +227,82 @@ invalid_engine(int fd)
do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EFAULT);
}
+static void
+invalid_exec_queue_wait(int fd)
+{
+ uint32_t bo, b;
+ uint64_t batch_offset;
+ uint64_t batch_addr;
+ uint64_t sdi_offset;
+ uint64_t sdi_addr;
+ uint64_t addr = 0x1a0000;
+
+ struct {
+ uint32_t batch[16];
+ uint64_t pad;
+ uint64_t vm_sync;
+ uint64_t exec_sync;
+ uint32_t data;
+ } *data;
+
+#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
+ struct drm_xe_sync sync[1] = {
+ { .flags = DRM_XE_SYNC_TYPE_USER_FENCE | DRM_XE_SYNC_FLAG_SIGNAL,
+ .timeline_value = USER_FENCE_VALUE },
+ };
+
+ struct drm_xe_exec exec = {
+ .num_batch_buffer = 1,
+ .num_syncs = 1,
+ .syncs = to_user_pointer(sync),
+ };
+
+ uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
+ uint32_t exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
+ struct drm_xe_wait_user_fence1 wait = {
+ .op = DRM_XE_UFENCE_WAIT_OP_EQ,
+ .flags = 0,
+ .value = 0xc0ffee,
+ .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
+ .timeout = -1,
+ .exec_queue_id = exec_queue,
+ };
+
+ bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
+ data = xe_bo_map(fd, bo, 0x40000);
+
+ batch_offset = (char *)&data[0].batch - (char *)data;
+ batch_addr = addr + batch_offset;
+ sdi_offset = (char *)&data[0].data - (char *)data;
+ sdi_addr = addr + sdi_offset;
+
+ b = 0;
+ data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
+ data[0].batch[b++] = sdi_addr;
+ data[0].batch[b++] = sdi_addr >> 32;
+ data[0].batch[b++] = 0xc0ffee;
+ data[0].batch[b++] = MI_BATCH_BUFFER_END;
+ igt_assert(b <= ARRAY_SIZE(data[0].batch));
+
+ wait.addr = to_user_pointer(&data[0].exec_sync);
+ exec.exec_queue_id = exec_queue;
+ exec.address = batch_addr;
+
+ xe_exec(fd, &exec);
+
+ /**
+ * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
+ * reset will happen and xe_wait_ufence will end return EIO not ETIME
+ */
+ do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
+
+ xe_exec_queue_destroy(fd, exec_queue);
+
+ if (bo) {
+ munmap(data, 0x40000);
+ gem_close(fd, bo);
+ }
+}
igt_main
{
@@ -250,6 +329,9 @@ igt_main
igt_subtest("invalid-engine")
invalid_engine(fd);
+ igt_subtest("invalid-exec_queue-wait")
+ invalid_exec_queue_wait(fd);
+
igt_fixture
drm_close_driver(fd);
}
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5)
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
@ 2023-12-08 5:48 ` Patchwork
2023-12-08 13:16 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6) Patchwork
3 siblings, 0 replies; 14+ messages in thread
From: Patchwork @ 2023-12-08 5:48 UTC (permalink / raw)
To: Bommu Krishnaiah; +Cc: igt-dev
== Series Details ==
Series: RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5)
URL : https://patchwork.freedesktop.org/series/127364/
State : failure
== Summary ==
IGT patchset build failed on latest successful build
431c2d2dd5828b25fcbe1c82afbac865f4771aee tests/intel/xe_create: create-big-vram subtest
Tail of build.log:
[613/1662] Compiling C object 'tests/59830eb@@i915_pm_rpm@exe/intel_i915_pm_rpm.c.o'.
[614/1662] Compiling C object 'tests/59830eb@@msm_mapping@exe/msm_msm_mapping.c.o'.
[615/1662] Compiling C object 'tests/59830eb@@xe_ccs@exe/intel_xe_ccs.c.o'.
[616/1662] Compiling C object 'tests/59830eb@@gem_sync@exe/intel_gem_sync.c.o'.
[617/1662] Compiling C object 'tests/59830eb@@msm_submitoverhead@exe/msm_msm_submitoverhead.c.o'.
[618/1662] Compiling C object 'tests/59830eb@@xe_pm_residency@exe/intel_xe_pm_residency.c.o'.
[619/1662] Compiling C object 'tests/59830eb@@xe_perf_pmu@exe/intel_xe_perf_pmu.c.o'.
[620/1662] Compiling C object 'tests/59830eb@@xe_spin_batch@exe/intel_xe_spin_batch.c.o'.
[621/1662] Compiling C object 'tests/59830eb@@xe_prime_self_import@exe/intel_xe_prime_self_import.c.o'.
[622/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_create_bo@exe/v3d_create_bo.c.o'.
[623/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_get_bo_offset@exe/v3d_get_bo_offset.c.o'.
[624/1662] Compiling C object 'tests/59830eb@@kms_psr@exe/intel_kms_psr.c.o'.
[625/1662] Compiling C object 'tests/59830eb@@msm_submit@exe/msm_msm_submit.c.o'.
[626/1662] Compiling C object 'tests/59830eb@@msm_recovery@exe/msm_msm_recovery.c.o'.
[627/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_get_param@exe/v3d_get_param.c.o'.
[628/1662] Compiling C object 'tests/59830eb@@xe_sysfs_scheduler@exe/intel_xe_sysfs_scheduler.c.o'.
[629/1662] Compiling C object 'tests/59830eb@@kms_chamelium_frames@exe/chamelium_kms_chamelium_helper.c.o'.
[630/1662] Compiling C object 'tests/59830eb@@kms_chamelium_audio@exe/chamelium_kms_chamelium_helper.c.o'.
[631/1662] Compiling C object 'tests/59830eb@@msm_shrink@exe/msm_msm_shrink.c.o'.
[632/1662] Compiling C object 'tests/59830eb@@kms_chamelium_hpd@exe/chamelium_kms_chamelium_helper.c.o'.
[633/1662] Compiling C object 'tests/59830eb@@gem_softpin@exe/intel_gem_softpin.c.o'.
[634/1662] Compiling C object 'tests/59830eb@@xe_pm@exe/intel_xe_pm.c.o'.
[635/1662] Compiling C object 'tests/59830eb@@kms_chamelium_color@exe/kms_color_helper.c.o'.
[636/1662] Compiling C object 'tests/59830eb@@kms_chamelium_edid@exe/chamelium_kms_chamelium_helper.c.o'.
[637/1662] Compiling C object 'tests/59830eb@@xe_query@exe/intel_xe_query.c.o'.
[638/1662] Compiling C object 'tests/59830eb@@xe_exec_balancer@exe/intel_xe_exec_balancer.c.o'.
[639/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_job_submission@exe/v3d_job_submission.c.o'.
[640/1662] Compiling C object 'tests/59830eb@@xe_pat@exe/intel_xe_pat.c.o'.
[641/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_submit_csd@exe/v3d_submit_csd.c.o'.
[642/1662] Compiling C object 'tests/59830eb@@xe_exec_reset@exe/intel_xe_exec_reset.c.o'.
[643/1662] Compiling C object 'tests/59830eb@@xe_evict@exe/intel_xe_evict.c.o'.
[644/1662] Compiling C object 'tests/59830eb@@kms_psr2_sf@exe/intel_kms_psr2_sf.c.o'.
[645/1662] Compiling C object 'tests/59830eb@@kms_chamelium_audio@exe/chamelium_kms_chamelium_audio.c.o'.
[646/1662] Compiling C object 'tests/59830eb@@i915_query@exe/intel_i915_query.c.o'.
[647/1662] Compiling C object 'tests/59830eb@@kms_chamelium_edid@exe/chamelium_kms_chamelium_edid.c.o'.
[648/1662] Compiling C object 'tests/59830eb@@xe_intel_bb@exe/intel_xe_intel_bb.c.o'.
[649/1662] Compiling C object 'tests/59830eb@@kms_chamelium_color@exe/chamelium_kms_chamelium_color.c.o'.
[650/1662] Compiling C object 'tests/59830eb@@gem_exec_balancer@exe/intel_gem_exec_balancer.c.o'.
[651/1662] Generating i915-perf-equations with a custom command.
[652/1662] Compiling C object 'tests/59830eb@@kms_chamelium_frames@exe/chamelium_kms_chamelium_frames.c.o'.
[653/1662] Compiling C object 'tests/59830eb@@kms_pm_rpm@exe/intel_kms_pm_rpm.c.o'.
[654/1662] Compiling C object 'tests/59830eb@@kms_chamelium_hpd@exe/chamelium_kms_chamelium_hpd.c.o'.
[655/1662] Compiling C object 'tests/59830eb@@gem_exec_fence@exe/intel_gem_exec_fence.c.o'.
[656/1662] Compiling C object 'tests/59830eb@@xe_vm@exe/intel_xe_vm.c.o'.
[657/1662] Compiling C object 'tests/59830eb@@kms_frontbuffer_tracking@exe/intel_kms_frontbuffer_tracking.c.o'.
[658/1662] Compiling C object 'tests/59830eb@@perf_pmu@exe/intel_perf_pmu.c.o'.
[659/1662] Compiling C object 'tests/59830eb@@gem_exec_schedule@exe/intel_gem_exec_schedule.c.o'.
[660/1662] Compiling C object 'tests/59830eb@@perf@exe/intel_perf.c.o'.
[661/1662] Linking target lib/libigt.so.0.
ninja: build stopped: subcommand failed.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
@ 2023-12-08 5:57 ` Rodrigo Vivi
2023-12-08 13:05 ` Matthew Brost
0 siblings, 1 reply; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-08 5:57 UTC (permalink / raw)
To: Bommu Krishnaiah, Matthew Brost; +Cc: igt-dev
On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> remove the num_engines/instances members from drm_xe_wait_user_fence structure
> and add a exec_queue_id member
>
> Right now this is only checking if the engine list is sane and nothing
> else. In the end every operation with this IOCTL is a soft check.
> So, let's formalize that and only use this IOCTL to wait on the fence.
>
> exec_queue_id member will help to user space to get proper error code
> from kernel while in exec_queue reset
>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Francois Dugast <francois.dugast@intel.com>
> ---
> include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> lib/xe/xe_ioctl.h | 9 +++------
> tests/intel/xe_evict.c | 4 ++--
> tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> tests/intel/xe_exec_reset.c | 6 +++---
> tests/intel/xe_exec_threads.c | 15 ++++++++-------
> tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> 10 files changed, 79 insertions(+), 87 deletions(-)
>
> diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> index 590f7b7af..fd06e4920 100644
> --- a/include/drm-uapi/xe_drm.h
> +++ b/include/drm-uapi/xe_drm.h
> @@ -129,7 +129,6 @@ struct xe_user_extension {
> * It is returned as part of the @drm_xe_engine, but it also is used as
> * the input of engine selection for both @drm_xe_exec_queue_create and
> * @drm_xe_query_engine_cycles
> - *
> */
> struct drm_xe_engine_class_instance {
> #define DRM_XE_ENGINE_CLASS_RENDER 0
> @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> */
> #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> + /** @engine_class: engine class id */
> __u16 engine_class;
> -
> + /** @engine_instance: engine instance id */
> __u16 engine_instance;
> + /** @gt_id: Unique ID of this GT within the PCI Device */
> __u16 gt_id;
> /** @pad: MBZ */
> __u16 pad;
> @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> *
> * Note: For userptr and externally imported dma-buf the kernel expects
> * either 1WAY or 2WAY for the @pat_index.
> + *
> + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD restrictions
> + * on the @pat_index. For such mappings there is no actual memory being
> + * mapped (the address in the PTE is invalid), so the various PAT memory
> + * attributes likely do not apply. Simply leaving as zero is one
> + * option (still a valid pat_index).
> */
> __u16 pat_index;
>
> @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> /** @op: wait operation (type of comparison) */
> __u16 op;
>
> -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait on VM bind */
> -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> /** @flags: wait flags */
> __u16 flags;
>
> @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> */
> __s64 timeout;
>
> - /**
> - * @num_engines: number of engine instances to wait on, must be zero
> - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> - */
> - __u64 num_engines;
> + /** @exec_queue_id: exec_queue_id returned from xe_exec_queue_create_ioctl */
> + __u32 exec_queue_id;
>
> - /**
> - * @instances: user pointer to array of drm_xe_engine_class_instance to
> - * wait on, must be NULL when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> - */
> - __u64 instances;
> + /** @pad2: MBZ */
> + __u32 pad2;
>
> /** @reserved: Reserved */
> __u64 reserved[2];
> diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c
> index b29ca40ad..0a1ddc86b 100644
> --- a/lib/xe/xe_ioctl.c
> +++ b/lib/xe/xe_ioctl.c
> @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> * @fd: xe device fd
> * @addr: address of value to compare
> * @value: expected value (equal) in @address
> - * @eci: engine class instance
> + * @exec_queue: exec_queue id
> * @timeout: pointer to time to wait in nanoseconds
> *
> * Function compares @value with memory pointed by @addr until they are equal.
> @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> * signalled. Returns 0 on success, -errno of ioctl on error.
> */
> int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t *timeout)
> + uint32_t exec_queue, int64_t *timeout)
> {
> struct drm_xe_wait_user_fence wait = {
> .addr = to_user_pointer(addr),
> .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> + .flags = 0,
> .value = value,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> - .num_engines = eci ? 1 :0,
> - .instances = eci ? to_user_pointer(eci) : 0,
> + .exec_queue_id = exec_queue,
> };
>
> igt_assert(timeout);
> @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * @fd: xe device fd
> * @addr: address of value to compare
> * @value: expected value (equal) in @address
> - * @eci: engine class instance
> + * @exec_queue: exec_queue id
> * @timeout: time to wait in nanoseconds
> *
> * Function compares @value with memory pointed by @addr until they are equal.
> @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * Returns elapsed time in nanoseconds if user fence was signalled.
> */
> int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout)
> + uint32_t exec_queue, int64_t timeout)
> {
> - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue, &timeout), 0);
> return timeout;
> }
>
> @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * @fd: xe device fd
> * @addr: address of value to compare
> * @value: expected value (equal) in @address
> - * @eci: engine class instance
> + * @exec_queue: exec_queue id
> * @timeout: absolute time when wait expire
> *
> * Function compares @value with memory pointed by @addr until they are equal.
> @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * Returns elapsed time in nanoseconds if user fence was signalled.
> */
> int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout)
> + uint32_t exec_queue, int64_t timeout)
> {
> struct drm_xe_wait_user_fence wait = {
> .addr = to_user_pointer(addr),
> .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP | DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> + .flags = !exec_queue ? DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
To be on the safe side we should probably change the signature of this function
to explicitly receive this flag instead of relying on the invalid exec_queue_id number.
Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP was used here...
Matt, thoughts?
Everything else in the patch looks good to me.
> .value = value,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = timeout,
> - .num_engines = eci ? 1 : 0,
> - .instances = eci ? to_user_pointer(eci) : 0,
> + .exec_queue_id = exec_queue,
> };
> struct timespec ts;
>
> diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h
> index bd660bd27..9c0b4b9bc 100644
> --- a/lib/xe/xe_ioctl.h
> +++ b/lib/xe/xe_ioctl.h
> @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue, uint64_t addr,
> struct drm_xe_sync *sync, uint32_t num_syncs);
> void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr);
> int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t *timeout);
> + uint32_t exec_queue, int64_t *timeout);
> int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout);
> + uint32_t exec_queue, int64_t timeout);
> int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout);
> + uint32_t exec_queue, int64_t timeout);
> void xe_force_gt_reset(int fd, int gt);
>
> #endif /* XE_IOCTL_H */
> diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
> index 89dc46fae..a37c2d97a 100644
> --- a/tests/intel/xe_evict.c
> +++ b/tests/intel/xe_evict.c
> @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> }
> #define TWENTY_SEC MS_TO_NS(20000)
> xe_wait_ufence(fd, &data[i].vm_sync, USER_FENCE_VALUE,
> - NULL, TWENTY_SEC);
> + bind_exec_queues[0], TWENTY_SEC);
> }
> sync[0].addr = addr + (char *)&data[i].exec_sync -
> (char *)data;
> @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> data = xe_bo_map(fd, __bo,
> ALIGN(sizeof(*data) * n_execs, 0x1000));
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, TWENTY_SEC);
> + exec_queues[i], TWENTY_SEC);
> igt_assert_eq(data[i].data, 0xc0ffee);
> }
> munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000));
> diff --git a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> index 79ff65e89..2448d49bc 100644
> --- a/tests/intel/xe_exec_balancer.c
> +++ b/tests/intel/xe_exec_balancer.c
> @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> bo_size, sync, 1);
>
> #define ONE_SEC MS_TO_NS(1000)
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
>
> if (flags & REBIND && i + 1 != n_execs) {
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + exec_queues[e], ONE_SEC);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, NULL,
> 0);
>
> @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + 0, ONE_SEC);
> data[0].vm_sync = 0;
> }
>
> @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> * an invalidate.
> */
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL, ONE_SEC);
> + USER_FENCE_VALUE, exec_queues[e],
> + ONE_SEC);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> /*
> @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
>
> j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> for (i = j; i < n_execs; i++)
> - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> - ONE_SEC);
> + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> + exec_queues[i], ONE_SEC);
>
> /* Wait for all execs to complete */
> if (flags & INVALIDATE)
> @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
>
> for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> i < n_execs; i++)
> diff --git a/tests/intel/xe_exec_compute_mode.c b/tests/intel/xe_exec_compute_mode.c
> index 7d3004d65..749a9b634 100644
> --- a/tests/intel/xe_exec_compute_mode.c
> +++ b/tests/intel/xe_exec_compute_mode.c
> @@ -171,8 +171,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC : ONE_SEC;
>
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], fence_timeout);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -198,7 +198,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> if (flags & REBIND && i + 1 != n_execs) {
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + exec_queues[e], fence_timeout);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> addr, bo_size, NULL, 0);
>
> @@ -214,7 +214,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + bind_exec_queues[e], fence_timeout);
> data[0].vm_sync = 0;
> }
>
> @@ -227,7 +227,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> * an invalidate.
> */
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL,
> + USER_FENCE_VALUE, exec_queues[e],
> fence_timeout);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> @@ -257,8 +257,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> j = flags & INVALIDATE ? n_execs - 1 : 0;
> for (i = j; i < n_execs; i++)
> - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> + exec_queues[i], fence_timeout);
>
> /* Wait for all execs to complete */
> if (flags & INVALIDATE)
> @@ -267,8 +267,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], fence_timeout);
>
> for (i = j; i < n_execs; i++)
> igt_assert_eq(data[i].data, 0xc0ffee);
> diff --git a/tests/intel/xe_exec_fault_mode.c b/tests/intel/xe_exec_fault_mode.c
> index ee7cbb604..d76b3a056 100644
> --- a/tests/intel/xe_exec_fault_mode.c
> +++ b/tests/intel/xe_exec_fault_mode.c
> @@ -195,15 +195,16 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> }
>
> #define ONE_SEC MS_TO_NS(1000)
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], ONE_SEC);
> data[0].vm_sync = 0;
>
> if (flags & PREFETCH) {
> /* Should move to system memory */
> xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0, addr,
> bo_size, sync, 1, 0);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> - ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], ONE_SEC);
> data[0].vm_sync = 0;
> }
>
> @@ -230,7 +231,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> if (flags & REBIND && i + 1 != n_execs) {
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + exec_queues[e], ONE_SEC);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> addr, bo_size, NULL, 0);
>
> @@ -246,7 +247,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + bind_exec_queues[e], ONE_SEC);
> data[0].vm_sync = 0;
> }
>
> @@ -259,7 +260,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> * an invalidate.
> */
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL, ONE_SEC);
> + USER_FENCE_VALUE, exec_queues[e],
> + ONE_SEC);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> /*
> @@ -290,13 +292,14 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> j = flags & INVALIDATE ? n_execs - 1 : 0;
> for (i = j; i < n_execs; i++)
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL, ONE_SEC);
> + USER_FENCE_VALUE, exec_queues[i], ONE_SEC);
> }
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], ONE_SEC);
>
> if (!(flags & INVALID_FAULT)) {
> for (i = j; i < n_execs; i++)
> diff --git a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
> index 094b34896..398c90af5 100644
> --- a/tests/intel/xe_exec_reset.c
> +++ b/tests/intel/xe_exec_reset.c
> @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>
> #define THREE_SEC MS_TO_NS(3000)
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> int err;
>
> err = __xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, &timeout);
> + exec_queues[i], &timeout);
> if (flags & GT_RESET)
> /* exec races with reset: may timeout or complete */
> igt_assert(err == -ETIME || !err);
> @@ -631,7 +631,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
>
> if (!(flags & GT_RESET)) {
> for (i = 1; i < n_execs; i++)
> diff --git a/tests/intel/xe_exec_threads.c b/tests/intel/xe_exec_threads.c
> index fcb926698..7985240c9 100644
> --- a/tests/intel/xe_exec_threads.c
> +++ b/tests/intel/xe_exec_threads.c
> @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>
> fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
>
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> for (j = i - 0x20; j <= i; ++j)
> xe_wait_ufence(fd, &data[j].exec_sync,
> USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + exec_queues[e], fence_timeout);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> NULL, 0);
>
> @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + 0, fence_timeout);
> data[0].vm_sync = 0;
> }
>
> @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> xe_wait_ufence(fd, &data[j].exec_sync,
> USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + exec_queues[e],
> + fence_timeout);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> /*
> @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> j = flags & INVALIDATE ?
> (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> for (i = j; i < n_execs; i++)
> - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> + exec_queues[e], fence_timeout);
>
> /* Wait for all execs to complete */
> if (flags & INVALIDATE)
> @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
>
> for (i = j; i < n_execs; i++)
> igt_assert_eq(data[i].data, 0xc0ffee);
> diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> index 3be987954..4e94403a3 100644
> --- a/tests/intel/xe_waitfence.c
> +++ b/tests/intel/xe_waitfence.c
> @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm, uint32_t bo, uint64_t offset,
> }
>
> static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout)
> + uint32_t exec_queue, int64_t timeout)
> {
> struct drm_xe_wait_user_fence wait = {
> .addr = to_user_pointer(addr),
> .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> + .flags = !exec_queue ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> .value = value,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = timeout,
> - .num_engines = eci ? 1 : 0,
> - .instances = eci ? to_user_pointer(eci) : 0,
> + .exec_queue_id = exec_queue,
> };
> struct timespec ts;
>
> - igt_assert(eci);
> igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait), 0);
> igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
>
> @@ -82,7 +79,7 @@ enum waittype {
> static void
> waitfence(int fd, enum waittype wt)
> {
> - struct drm_xe_engine *engine = NULL;
> + uint32_t exec_queue;
> struct timespec ts;
> int64_t current, signalled;
> uint32_t bo_1;
> @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
>
> if (wt == RELTIME) {
> - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL, MS_TO_NS(10));
> + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0, MS_TO_NS(10));
> igt_debug("wait type: RELTIME - timeout: %ld, timeout left: %ld\n",
> MS_TO_NS(10), timeout);
> } else if (wt == ENGINE) {
> - engine = xe_engine(fd, 1);
> + exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
> clock_gettime(CLOCK_MONOTONIC, &ts);
> current = ts.tv_sec * 1e9 + ts.tv_nsec;
> timeout = current + MS_TO_NS(10);
> - signalled = wait_with_eci_abstime(fd, &wait_fence, 7, &engine->instance, timeout);
> + signalled = wait_with_eci_abstime(fd, &wait_fence, 7, exec_queue, timeout);
> igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> ", signalled: %" PRId64
> ", elapsed: %" PRId64 "\n",
> @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> clock_gettime(CLOCK_MONOTONIC, &ts);
> current = ts.tv_sec * 1e9 + ts.tv_nsec;
> timeout = current + MS_TO_NS(10);
> - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, NULL, timeout);
> + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0, timeout);
> igt_debug("wait type: ABSTIME - timeout: %" PRId64
> ", signalled: %" PRId64
> ", elapsed: %" PRId64 "\n",
> @@ -191,8 +188,7 @@ invalid_ops(int fd)
> .value = 1,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = 1,
> - .num_engines = 0,
> - .instances = 0,
> + .exec_queue_id = 0,
> };
>
> uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> @@ -216,8 +212,7 @@ invalid_engine(int fd)
> .value = 1,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = -1,
> - .num_engines = 1,
> - .instances = 0,
> + .exec_queue_id = 0,
> };
>
> uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
@ 2023-12-08 6:00 ` Rodrigo Vivi
2023-12-08 12:59 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-08 6:00 UTC (permalink / raw)
To: Bommu Krishnaiah; +Cc: igt-dev
On Fri, Dec 08, 2023 at 09:48:25AM +0530, Bommu Krishnaiah wrote:
> Skipping the GPU mapping(vm_bind) for object, so that exec_queue
'skipping' sounds like you are not waiting at all...
> reset will happen and xe_wait_ufence will end return EIO not ETIME
>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Francois Dugast <francois.dugast@intel.com>
> ---
> tests/intel/xe_waitfence.c | 82 ++++++++++++++++++++++++++++++++++++++
> 1 file changed, 82 insertions(+)
>
> diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> index 4e94403a3..bd95fa1f8 100644
> --- a/tests/intel/xe_waitfence.c
> +++ b/tests/intel/xe_waitfence.c
> @@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
> *
> * SUBTEST: invalid-engine
> * Description: Check query with invalid engine info returns expected error code
> + *
> + * SUBTEST: invalid-exec_queue-wait
'invalid wait' sounded to me like a negative test with invalid timeout.
So I believe we need to rephrase some stuff here, but the test looks right.
> + * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
> */
>
> static void
> @@ -224,6 +227,82 @@ invalid_engine(int fd)
> do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EFAULT);
> }
>
> +static void
> +invalid_exec_queue_wait(int fd)
> +{
> + uint32_t bo, b;
> + uint64_t batch_offset;
> + uint64_t batch_addr;
> + uint64_t sdi_offset;
> + uint64_t sdi_addr;
> + uint64_t addr = 0x1a0000;
> +
> + struct {
> + uint32_t batch[16];
> + uint64_t pad;
> + uint64_t vm_sync;
> + uint64_t exec_sync;
> + uint32_t data;
> + } *data;
> +
> +#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
> + struct drm_xe_sync sync[1] = {
> + { .flags = DRM_XE_SYNC_TYPE_USER_FENCE | DRM_XE_SYNC_FLAG_SIGNAL,
> + .timeline_value = USER_FENCE_VALUE },
> + };
> +
> + struct drm_xe_exec exec = {
> + .num_batch_buffer = 1,
> + .num_syncs = 1,
> + .syncs = to_user_pointer(sync),
> + };
> +
> + uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> + uint32_t exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
> + struct drm_xe_wait_user_fence1 wait = {
> + .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> + .flags = 0,
> + .value = 0xc0ffee,
> + .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> + .timeout = -1,
> + .exec_queue_id = exec_queue,
> + };
> +
> + bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
> + data = xe_bo_map(fd, bo, 0x40000);
> +
> + batch_offset = (char *)&data[0].batch - (char *)data;
> + batch_addr = addr + batch_offset;
> + sdi_offset = (char *)&data[0].data - (char *)data;
> + sdi_addr = addr + sdi_offset;
> +
> + b = 0;
> + data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> + data[0].batch[b++] = sdi_addr;
> + data[0].batch[b++] = sdi_addr >> 32;
> + data[0].batch[b++] = 0xc0ffee;
> + data[0].batch[b++] = MI_BATCH_BUFFER_END;
> + igt_assert(b <= ARRAY_SIZE(data[0].batch));
> +
> + wait.addr = to_user_pointer(&data[0].exec_sync);
> + exec.exec_queue_id = exec_queue;
> + exec.address = batch_addr;
> +
> + xe_exec(fd, &exec);
> +
> + /**
> + * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
> + * reset will happen and xe_wait_ufence will end return EIO not ETIME
> + */
> + do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
> +
> + xe_exec_queue_destroy(fd, exec_queue);
> +
> + if (bo) {
> + munmap(data, 0x40000);
> + gem_close(fd, bo);
> + }
> +}
>
> igt_main
> {
> @@ -250,6 +329,9 @@ igt_main
> igt_subtest("invalid-engine")
> invalid_engine(fd);
>
> + igt_subtest("invalid-exec_queue-wait")
> + invalid_exec_queue_wait(fd);
> +
> igt_fixture
> drm_close_driver(fd);
> }
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 6:00 ` Rodrigo Vivi
@ 2023-12-08 12:59 ` Bommu, Krishnaiah
2023-12-08 13:48 ` Rodrigo Vivi
0 siblings, 1 reply; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-08 12:59 UTC (permalink / raw)
To: Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Friday, December 8, 2023 11:31 AM
> To: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: igt-dev@lists.freedesktop.org
> Subject: Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl
> when exec_queue reset happen
>
> On Fri, Dec 08, 2023 at 09:48:25AM +0530, Bommu Krishnaiah wrote:
> > Skipping the GPU mapping(vm_bind) for object, so that exec_queue
>
> 'skipping' sounds like you are not waiting at all...
>
> > reset will happen and xe_wait_ufence will end return EIO not ETIME
> >
> > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Cc: Francois Dugast <francois.dugast@intel.com>
> > ---
> > tests/intel/xe_waitfence.c | 82
> > ++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 82 insertions(+)
> >
> > diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > index 4e94403a3..bd95fa1f8 100644
> > --- a/tests/intel/xe_waitfence.c
> > +++ b/tests/intel/xe_waitfence.c
> > @@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
> > *
> > * SUBTEST: invalid-engine
> > * Description: Check query with invalid engine info returns expected
> > error code
> > + *
> > + * SUBTEST: invalid-exec_queue-wait
>
> 'invalid wait' sounded to me like a negative test with invalid timeout.
>
> So I believe we need to rephrase some stuff here, but the test looks right.
>
Commit message:
drm-uapi/xe: Don't wait on user_fence during exec queue reset
don't wait till timeout on user fence when exec_queue reset is detected
and return return proper error code
root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools# git diff
diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
index bd95fa1f8..9a2c854c1 100644
--- a/tests/intel/xe_waitfence.c
+++ b/tests/intel/xe_waitfence.c
@@ -150,8 +150,8 @@ waitfence(int fd, enum waittype wt)
* SUBTEST: invalid-engine
* Description: Check query with invalid engine info returns expected error code
*
- * SUBTEST: invalid-exec_queue-wait
- * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
+ * SUBTEST: exec_queue-reset-wait
+ * Description: Don't wait till timeout on user fence when exec_queue reset is detected and return return proper error
*/
static void
@@ -228,7 +228,7 @@ invalid_engine(int fd)
}
static void
-invalid_exec_queue_wait(int fd)
+exec_queue_reset_wait(int fd)
{
uint32_t bo, b;
uint64_t batch_offset;
@@ -291,8 +291,8 @@ invalid_exec_queue_wait(int fd)
xe_exec(fd, &exec);
/**
- * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
- * reset will happen and xe_wait_ufence will end return EIO not ETIME
+ * Don't do the GPU mapping(vm_bind) for object, so that exec_queue
+ * reset will happen and xe_wait_ufence will return EIO not ETIME
*/
do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
@@ -329,8 +329,8 @@ igt_main
igt_subtest("invalid-engine")
invalid_engine(fd);
- igt_subtest("invalid-exec_queue-wait")
- invalid_exec_queue_wait(fd);
+ igt_subtest("exec_queue-reset-wait")
+ exec_queue_reset_wait(fd);
igt_fixture
drm_close_driver(fd);
root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
This is what I can think of now, please suggest anything need to be changed
Regards,
Krishna
> > + * Description: Check xe_wait_ufence will return expected error code
> > + while exec_queue reset happen
> > */
> >
> > static void
> > @@ -224,6 +227,82 @@ invalid_engine(int fd)
> > do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait,
> EFAULT); }
> >
> > +static void
> > +invalid_exec_queue_wait(int fd)
> > +{
> > + uint32_t bo, b;
> > + uint64_t batch_offset;
> > + uint64_t batch_addr;
> > + uint64_t sdi_offset;
> > + uint64_t sdi_addr;
> > + uint64_t addr = 0x1a0000;
> > +
> > + struct {
> > + uint32_t batch[16];
> > + uint64_t pad;
> > + uint64_t vm_sync;
> > + uint64_t exec_sync;
> > + uint32_t data;
> > + } *data;
> > +
> > +#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
> > + struct drm_xe_sync sync[1] = {
> > + { .flags = DRM_XE_SYNC_TYPE_USER_FENCE |
> DRM_XE_SYNC_FLAG_SIGNAL,
> > + .timeline_value = USER_FENCE_VALUE },
> > + };
> > +
> > + struct drm_xe_exec exec = {
> > + .num_batch_buffer = 1,
> > + .num_syncs = 1,
> > + .syncs = to_user_pointer(sync),
> > + };
> > +
> > + uint32_t vm = xe_vm_create(fd,
> DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > + uint32_t exec_queue = xe_exec_queue_create_class(fd, vm,
> DRM_XE_ENGINE_CLASS_COPY);
> > + struct drm_xe_wait_user_fence1 wait = {
> > + .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > + .flags = 0,
> > + .value = 0xc0ffee,
> > + .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > + .timeout = -1,
> > + .exec_queue_id = exec_queue,
> > + };
> > +
> > + bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
> > + data = xe_bo_map(fd, bo, 0x40000);
> > +
> > + batch_offset = (char *)&data[0].batch - (char *)data;
> > + batch_addr = addr + batch_offset;
> > + sdi_offset = (char *)&data[0].data - (char *)data;
> > + sdi_addr = addr + sdi_offset;
> > +
> > + b = 0;
> > + data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> > + data[0].batch[b++] = sdi_addr;
> > + data[0].batch[b++] = sdi_addr >> 32;
> > + data[0].batch[b++] = 0xc0ffee;
> > + data[0].batch[b++] = MI_BATCH_BUFFER_END;
> > + igt_assert(b <= ARRAY_SIZE(data[0].batch));
> > +
> > + wait.addr = to_user_pointer(&data[0].exec_sync);
> > + exec.exec_queue_id = exec_queue;
> > + exec.address = batch_addr;
> > +
> > + xe_exec(fd, &exec);
> > +
> > + /**
> > + * Skipping the GPU mapping(vm_bind) for object, so that
> exec_queue
> > + * reset will happen and xe_wait_ufence will end return EIO not
> ETIME
> > + */
> > + do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
> > +
> > + xe_exec_queue_destroy(fd, exec_queue);
> > +
> > + if (bo) {
> > + munmap(data, 0x40000);
> > + gem_close(fd, bo);
> > + }
> > +}
> >
> > igt_main
> > {
> > @@ -250,6 +329,9 @@ igt_main
> > igt_subtest("invalid-engine")
> > invalid_engine(fd);
> >
> > + igt_subtest("invalid-exec_queue-wait")
> > + invalid_exec_queue_wait(fd);
> > +
> > igt_fixture
> > drm_close_driver(fd);
> > }
> > --
> > 2.25.1
> >
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 5:57 ` Rodrigo Vivi
@ 2023-12-08 13:05 ` Matthew Brost
2023-12-11 5:36 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Matthew Brost @ 2023-12-08 13:05 UTC (permalink / raw)
To: Rodrigo Vivi; +Cc: igt-dev, Bommu Krishnaiah
On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > remove the num_engines/instances members from drm_xe_wait_user_fence structure
> > and add a exec_queue_id member
> >
> > Right now this is only checking if the engine list is sane and nothing
> > else. In the end every operation with this IOCTL is a soft check.
> > So, let's formalize that and only use this IOCTL to wait on the fence.
> >
> > exec_queue_id member will help to user space to get proper error code
> > from kernel while in exec_queue reset
> >
> > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Cc: Francois Dugast <francois.dugast@intel.com>
> > ---
> > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > lib/xe/xe_ioctl.h | 9 +++------
> > tests/intel/xe_evict.c | 4 ++--
> > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > tests/intel/xe_exec_reset.c | 6 +++---
> > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > 10 files changed, 79 insertions(+), 87 deletions(-)
> >
> > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > index 590f7b7af..fd06e4920 100644
> > --- a/include/drm-uapi/xe_drm.h
> > +++ b/include/drm-uapi/xe_drm.h
> > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > * It is returned as part of the @drm_xe_engine, but it also is used as
> > * the input of engine selection for both @drm_xe_exec_queue_create and
> > * @drm_xe_query_engine_cycles
> > - *
> > */
> > struct drm_xe_engine_class_instance {
> > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > */
> > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > + /** @engine_class: engine class id */
> > __u16 engine_class;
> > -
> > + /** @engine_instance: engine instance id */
> > __u16 engine_instance;
> > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > __u16 gt_id;
> > /** @pad: MBZ */
> > __u16 pad;
> > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > *
> > * Note: For userptr and externally imported dma-buf the kernel expects
> > * either 1WAY or 2WAY for the @pat_index.
> > + *
> > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD restrictions
> > + * on the @pat_index. For such mappings there is no actual memory being
> > + * mapped (the address in the PTE is invalid), so the various PAT memory
> > + * attributes likely do not apply. Simply leaving as zero is one
> > + * option (still a valid pat_index).
> > */
> > __u16 pat_index;
> >
> > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > /** @op: wait operation (type of comparison) */
> > __u16 op;
> >
> > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait on VM bind */
> > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > /** @flags: wait flags */
> > __u16 flags;
> >
> > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > */
> > __s64 timeout;
> >
> > - /**
> > - * @num_engines: number of engine instances to wait on, must be zero
> > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > - */
> > - __u64 num_engines;
> > + /** @exec_queue_id: exec_queue_id returned from xe_exec_queue_create_ioctl */
> > + __u32 exec_queue_id;
> >
> > - /**
> > - * @instances: user pointer to array of drm_xe_engine_class_instance to
> > - * wait on, must be NULL when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > - */
> > - __u64 instances;
> > + /** @pad2: MBZ */
> > + __u32 pad2;
> >
> > /** @reserved: Reserved */
> > __u64 reserved[2];
> > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c
> > index b29ca40ad..0a1ddc86b 100644
> > --- a/lib/xe/xe_ioctl.c
> > +++ b/lib/xe/xe_ioctl.c
> > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> > * @fd: xe device fd
> > * @addr: address of value to compare
> > * @value: expected value (equal) in @address
> > - * @eci: engine class instance
> > + * @exec_queue: exec_queue id
> > * @timeout: pointer to time to wait in nanoseconds
> > *
> > * Function compares @value with memory pointed by @addr until they are equal.
> > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> > * signalled. Returns 0 on success, -errno of ioctl on error.
> > */
> > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t *timeout)
> > + uint32_t exec_queue, int64_t *timeout)
> > {
> > struct drm_xe_wait_user_fence wait = {
> > .addr = to_user_pointer(addr),
> > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > + .flags = 0,
> > .value = value,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > - .num_engines = eci ? 1 :0,
> > - .instances = eci ? to_user_pointer(eci) : 0,
> > + .exec_queue_id = exec_queue,
> > };
> >
> > igt_assert(timeout);
> > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * @fd: xe device fd
> > * @addr: address of value to compare
> > * @value: expected value (equal) in @address
> > - * @eci: engine class instance
> > + * @exec_queue: exec_queue id
> > * @timeout: time to wait in nanoseconds
> > *
> > * Function compares @value with memory pointed by @addr until they are equal.
> > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * Returns elapsed time in nanoseconds if user fence was signalled.
> > */
> > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout)
> > + uint32_t exec_queue, int64_t timeout)
> > {
> > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue, &timeout), 0);
> > return timeout;
> > }
> >
> > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * @fd: xe device fd
> > * @addr: address of value to compare
> > * @value: expected value (equal) in @address
> > - * @eci: engine class instance
> > + * @exec_queue: exec_queue id
> > * @timeout: absolute time when wait expire
> > *
> > * Function compares @value with memory pointed by @addr until they are equal.
> > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * Returns elapsed time in nanoseconds if user fence was signalled.
> > */
> > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout)
> > + uint32_t exec_queue, int64_t timeout)
> > {
> > struct drm_xe_wait_user_fence wait = {
> > .addr = to_user_pointer(addr),
> > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP | DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > + .flags = !exec_queue ? DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
>
> To be on the safe side we should probably change the signature of this function
> to explicitly receive this flag instead of relying on the invalid exec_queue_id number.
>
> Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP was used here...
>
> Matt, thoughts?
>
Agree with everything here, !exec_queue really has nothing to do with
setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So yes, please
change this function to accpt a flags field.
Everything else LGTM.
Matt
> Everything else in the patch looks good to me.
>
> > .value = value,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = timeout,
> > - .num_engines = eci ? 1 : 0,
> > - .instances = eci ? to_user_pointer(eci) : 0,
> > + .exec_queue_id = exec_queue,
> > };
> > struct timespec ts;
> >
> > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h
> > index bd660bd27..9c0b4b9bc 100644
> > --- a/lib/xe/xe_ioctl.h
> > +++ b/lib/xe/xe_ioctl.h
> > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue, uint64_t addr,
> > struct drm_xe_sync *sync, uint32_t num_syncs);
> > void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr);
> > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t *timeout);
> > + uint32_t exec_queue, int64_t *timeout);
> > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout);
> > + uint32_t exec_queue, int64_t timeout);
> > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout);
> > + uint32_t exec_queue, int64_t timeout);
> > void xe_force_gt_reset(int fd, int gt);
> >
> > #endif /* XE_IOCTL_H */
> > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
> > index 89dc46fae..a37c2d97a 100644
> > --- a/tests/intel/xe_evict.c
> > +++ b/tests/intel/xe_evict.c
> > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> > }
> > #define TWENTY_SEC MS_TO_NS(20000)
> > xe_wait_ufence(fd, &data[i].vm_sync, USER_FENCE_VALUE,
> > - NULL, TWENTY_SEC);
> > + bind_exec_queues[0], TWENTY_SEC);
> > }
> > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > (char *)data;
> > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> > data = xe_bo_map(fd, __bo,
> > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, TWENTY_SEC);
> > + exec_queues[i], TWENTY_SEC);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > }
> > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000));
> > diff --git a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > index 79ff65e89..2448d49bc 100644
> > --- a/tests/intel/xe_exec_balancer.c
> > +++ b/tests/intel/xe_exec_balancer.c
> > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> > bo_size, sync, 1);
> >
> > #define ONE_SEC MS_TO_NS(1000)
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> >
> > if (flags & REBIND && i + 1 != n_execs) {
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + exec_queues[e], ONE_SEC);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, NULL,
> > 0);
> >
> > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + 0, ONE_SEC);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> > * an invalidate.
> > */
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > + USER_FENCE_VALUE, exec_queues[e],
> > + ONE_SEC);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > /*
> > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> >
> > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > for (i = j; i < n_execs; i++)
> > - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> > - ONE_SEC);
> > + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > + exec_queues[i], ONE_SEC);
> >
> > /* Wait for all execs to complete */
> > if (flags & INVALIDATE)
> > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
> >
> > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > i < n_execs; i++)
> > diff --git a/tests/intel/xe_exec_compute_mode.c b/tests/intel/xe_exec_compute_mode.c
> > index 7d3004d65..749a9b634 100644
> > --- a/tests/intel/xe_exec_compute_mode.c
> > +++ b/tests/intel/xe_exec_compute_mode.c
> > @@ -171,8 +171,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC : ONE_SEC;
> >
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], fence_timeout);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -198,7 +198,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > if (flags & REBIND && i + 1 != n_execs) {
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + exec_queues[e], fence_timeout);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> > addr, bo_size, NULL, 0);
> >
> > @@ -214,7 +214,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + bind_exec_queues[e], fence_timeout);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -227,7 +227,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > * an invalidate.
> > */
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL,
> > + USER_FENCE_VALUE, exec_queues[e],
> > fence_timeout);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > @@ -257,8 +257,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > for (i = j; i < n_execs; i++)
> > - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > + exec_queues[i], fence_timeout);
> >
> > /* Wait for all execs to complete */
> > if (flags & INVALIDATE)
> > @@ -267,8 +267,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> > sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], fence_timeout);
> >
> > for (i = j; i < n_execs; i++)
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > diff --git a/tests/intel/xe_exec_fault_mode.c b/tests/intel/xe_exec_fault_mode.c
> > index ee7cbb604..d76b3a056 100644
> > --- a/tests/intel/xe_exec_fault_mode.c
> > +++ b/tests/intel/xe_exec_fault_mode.c
> > @@ -195,15 +195,16 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > }
> >
> > #define ONE_SEC MS_TO_NS(1000)
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], ONE_SEC);
> > data[0].vm_sync = 0;
> >
> > if (flags & PREFETCH) {
> > /* Should move to system memory */
> > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0, addr,
> > bo_size, sync, 1, 0);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > - ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], ONE_SEC);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -230,7 +231,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > if (flags & REBIND && i + 1 != n_execs) {
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + exec_queues[e], ONE_SEC);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> > addr, bo_size, NULL, 0);
> >
> > @@ -246,7 +247,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + bind_exec_queues[e], ONE_SEC);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -259,7 +260,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > * an invalidate.
> > */
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > + USER_FENCE_VALUE, exec_queues[e],
> > + ONE_SEC);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > /*
> > @@ -290,13 +292,14 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > for (i = j; i < n_execs; i++)
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > + USER_FENCE_VALUE, exec_queues[i], ONE_SEC);
> > }
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> > sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], ONE_SEC);
> >
> > if (!(flags & INVALID_FAULT)) {
> > for (i = j; i < n_execs; i++)
> > diff --git a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
> > index 094b34896..398c90af5 100644
> > --- a/tests/intel/xe_exec_reset.c
> > +++ b/tests/intel/xe_exec_reset.c
> > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> >
> > #define THREE_SEC MS_TO_NS(3000)
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> > int err;
> >
> > err = __xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, &timeout);
> > + exec_queues[i], &timeout);
> > if (flags & GT_RESET)
> > /* exec races with reset: may timeout or complete */
> > igt_assert(err == -ETIME || !err);
> > @@ -631,7 +631,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
> >
> > if (!(flags & GT_RESET)) {
> > for (i = 1; i < n_execs; i++)
> > diff --git a/tests/intel/xe_exec_threads.c b/tests/intel/xe_exec_threads.c
> > index fcb926698..7985240c9 100644
> > --- a/tests/intel/xe_exec_threads.c
> > +++ b/tests/intel/xe_exec_threads.c
> > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> >
> > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
> >
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > for (j = i - 0x20; j <= i; ++j)
> > xe_wait_ufence(fd, &data[j].exec_sync,
> > USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + exec_queues[e], fence_timeout);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > NULL, 0);
> >
> > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + 0, fence_timeout);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > xe_wait_ufence(fd, &data[j].exec_sync,
> > USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + exec_queues[e],
> > + fence_timeout);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > /*
> > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > j = flags & INVALIDATE ?
> > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > for (i = j; i < n_execs; i++)
> > - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > + exec_queues[e], fence_timeout);
> >
> > /* Wait for all execs to complete */
> > if (flags & INVALIDATE)
> > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
> >
> > for (i = j; i < n_execs; i++)
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > index 3be987954..4e94403a3 100644
> > --- a/tests/intel/xe_waitfence.c
> > +++ b/tests/intel/xe_waitfence.c
> > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm, uint32_t bo, uint64_t offset,
> > }
> >
> > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout)
> > + uint32_t exec_queue, int64_t timeout)
> > {
> > struct drm_xe_wait_user_fence wait = {
> > .addr = to_user_pointer(addr),
> > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > + .flags = !exec_queue ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > .value = value,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = timeout,
> > - .num_engines = eci ? 1 : 0,
> > - .instances = eci ? to_user_pointer(eci) : 0,
> > + .exec_queue_id = exec_queue,
> > };
> > struct timespec ts;
> >
> > - igt_assert(eci);
> > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait), 0);
> > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> >
> > @@ -82,7 +79,7 @@ enum waittype {
> > static void
> > waitfence(int fd, enum waittype wt)
> > {
> > - struct drm_xe_engine *engine = NULL;
> > + uint32_t exec_queue;
> > struct timespec ts;
> > int64_t current, signalled;
> > uint32_t bo_1;
> > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> >
> > if (wt == RELTIME) {
> > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL, MS_TO_NS(10));
> > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0, MS_TO_NS(10));
> > igt_debug("wait type: RELTIME - timeout: %ld, timeout left: %ld\n",
> > MS_TO_NS(10), timeout);
> > } else if (wt == ENGINE) {
> > - engine = xe_engine(fd, 1);
> > + exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
> > clock_gettime(CLOCK_MONOTONIC, &ts);
> > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > timeout = current + MS_TO_NS(10);
> > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7, &engine->instance, timeout);
> > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7, exec_queue, timeout);
> > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > ", signalled: %" PRId64
> > ", elapsed: %" PRId64 "\n",
> > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > clock_gettime(CLOCK_MONOTONIC, &ts);
> > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > timeout = current + MS_TO_NS(10);
> > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, NULL, timeout);
> > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0, timeout);
> > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > ", signalled: %" PRId64
> > ", elapsed: %" PRId64 "\n",
> > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > .value = 1,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = 1,
> > - .num_engines = 0,
> > - .instances = 0,
> > + .exec_queue_id = 0,
> > };
> >
> > uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > @@ -216,8 +212,7 @@ invalid_engine(int fd)
> > .value = 1,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = -1,
> > - .num_engines = 1,
> > - .instances = 0,
> > + .exec_queue_id = 0,
> > };
> >
> > uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > --
> > 2.25.1
> >
^ permalink raw reply [flat|nested] 14+ messages in thread
* ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6)
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
` (2 preceding siblings ...)
2023-12-08 5:48 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5) Patchwork
@ 2023-12-08 13:16 ` Patchwork
3 siblings, 0 replies; 14+ messages in thread
From: Patchwork @ 2023-12-08 13:16 UTC (permalink / raw)
To: Bommu, Krishnaiah; +Cc: igt-dev
== Series Details ==
Series: RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6)
URL : https://patchwork.freedesktop.org/series/127364/
State : failure
== Summary ==
Applying: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
Applying: drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
Using index info to reconstruct a base tree...
M tests/intel/xe_waitfence.c
Patch failed at 0002 drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 12:59 ` Bommu, Krishnaiah
@ 2023-12-08 13:48 ` Rodrigo Vivi
0 siblings, 0 replies; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-08 13:48 UTC (permalink / raw)
To: Bommu, Krishnaiah; +Cc: igt-dev@lists.freedesktop.org
On Fri, Dec 08, 2023 at 07:59:34AM -0500, Bommu, Krishnaiah wrote:
>
>
> > -----Original Message-----
> > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Sent: Friday, December 8, 2023 11:31 AM
> > To: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>
> > Cc: igt-dev@lists.freedesktop.org
> > Subject: Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl
> > when exec_queue reset happen
> >
> > On Fri, Dec 08, 2023 at 09:48:25AM +0530, Bommu Krishnaiah wrote:
> > > Skipping the GPU mapping(vm_bind) for object, so that exec_queue
> >
> > 'skipping' sounds like you are not waiting at all...
> >
> > > reset will happen and xe_wait_ufence will end return EIO not ETIME
> > >
> > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > ---
> > > tests/intel/xe_waitfence.c | 82
> > > ++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 82 insertions(+)
> > >
> > > diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > > index 4e94403a3..bd95fa1f8 100644
> > > --- a/tests/intel/xe_waitfence.c
> > > +++ b/tests/intel/xe_waitfence.c
> > > @@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
> > > *
> > > * SUBTEST: invalid-engine
> > > * Description: Check query with invalid engine info returns expected
> > > error code
> > > + *
> > > + * SUBTEST: invalid-exec_queue-wait
> >
> > 'invalid wait' sounded to me like a negative test with invalid timeout.
> >
> > So I believe we need to rephrase some stuff here, but the test looks right.
> >
>
> Commit message:
> drm-uapi/xe: Don't wait on user_fence during exec queue reset
>
> don't wait till timeout on user fence when exec_queue reset is detected
> and return return proper error code
>
>
> root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools# git diff
> diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> index bd95fa1f8..9a2c854c1 100644
> --- a/tests/intel/xe_waitfence.c
> +++ b/tests/intel/xe_waitfence.c
> @@ -150,8 +150,8 @@ waitfence(int fd, enum waittype wt)
> * SUBTEST: invalid-engine
> * Description: Check query with invalid engine info returns expected error code
> *
> - * SUBTEST: invalid-exec_queue-wait
> - * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
> + * SUBTEST: exec_queue-reset-wait
> + * Description: Don't wait till timeout on user fence when exec_queue reset is detected and return return proper error
> */
>
> static void
> @@ -228,7 +228,7 @@ invalid_engine(int fd)
> }
>
> static void
> -invalid_exec_queue_wait(int fd)
> +exec_queue_reset_wait(int fd)
> {
> uint32_t bo, b;
> uint64_t batch_offset;
> @@ -291,8 +291,8 @@ invalid_exec_queue_wait(int fd)
> xe_exec(fd, &exec);
>
> /**
> - * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
> - * reset will happen and xe_wait_ufence will end return EIO not ETIME
> + * Don't do the GPU mapping(vm_bind) for object, so that exec_queue
> + * reset will happen and xe_wait_ufence will return EIO not ETIME
> */
> do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
>
> @@ -329,8 +329,8 @@ igt_main
> igt_subtest("invalid-engine")
> invalid_engine(fd);
>
> - igt_subtest("invalid-exec_queue-wait")
> - invalid_exec_queue_wait(fd);
> + igt_subtest("exec_queue-reset-wait")
> + exec_queue_reset_wait(fd);
>
> igt_fixture
> drm_close_driver(fd);
> root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
> root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
>
> This is what I can think of now, please suggest anything need to be changed
yeap, this works for me:
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
>
> Regards,
> Krishna
>
> > > + * Description: Check xe_wait_ufence will return expected error code
> > > + while exec_queue reset happen
> > > */
> > >
> > > static void
> > > @@ -224,6 +227,82 @@ invalid_engine(int fd)
> > > do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait,
> > EFAULT); }
> > >
> > > +static void
> > > +invalid_exec_queue_wait(int fd)
> > > +{
> > > + uint32_t bo, b;
> > > + uint64_t batch_offset;
> > > + uint64_t batch_addr;
> > > + uint64_t sdi_offset;
> > > + uint64_t sdi_addr;
> > > + uint64_t addr = 0x1a0000;
> > > +
> > > + struct {
> > > + uint32_t batch[16];
> > > + uint64_t pad;
> > > + uint64_t vm_sync;
> > > + uint64_t exec_sync;
> > > + uint32_t data;
> > > + } *data;
> > > +
> > > +#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
> > > + struct drm_xe_sync sync[1] = {
> > > + { .flags = DRM_XE_SYNC_TYPE_USER_FENCE |
> > DRM_XE_SYNC_FLAG_SIGNAL,
> > > + .timeline_value = USER_FENCE_VALUE },
> > > + };
> > > +
> > > + struct drm_xe_exec exec = {
> > > + .num_batch_buffer = 1,
> > > + .num_syncs = 1,
> > > + .syncs = to_user_pointer(sync),
> > > + };
> > > +
> > > + uint32_t vm = xe_vm_create(fd,
> > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > + uint32_t exec_queue = xe_exec_queue_create_class(fd, vm,
> > DRM_XE_ENGINE_CLASS_COPY);
> > > + struct drm_xe_wait_user_fence1 wait = {
> > > + .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > + .flags = 0,
> > > + .value = 0xc0ffee,
> > > + .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > + .timeout = -1,
> > > + .exec_queue_id = exec_queue,
> > > + };
> > > +
> > > + bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
> > > + data = xe_bo_map(fd, bo, 0x40000);
> > > +
> > > + batch_offset = (char *)&data[0].batch - (char *)data;
> > > + batch_addr = addr + batch_offset;
> > > + sdi_offset = (char *)&data[0].data - (char *)data;
> > > + sdi_addr = addr + sdi_offset;
> > > +
> > > + b = 0;
> > > + data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> > > + data[0].batch[b++] = sdi_addr;
> > > + data[0].batch[b++] = sdi_addr >> 32;
> > > + data[0].batch[b++] = 0xc0ffee;
> > > + data[0].batch[b++] = MI_BATCH_BUFFER_END;
> > > + igt_assert(b <= ARRAY_SIZE(data[0].batch));
> > > +
> > > + wait.addr = to_user_pointer(&data[0].exec_sync);
> > > + exec.exec_queue_id = exec_queue;
> > > + exec.address = batch_addr;
> > > +
> > > + xe_exec(fd, &exec);
> > > +
> > > + /**
> > > + * Skipping the GPU mapping(vm_bind) for object, so that
> > exec_queue
> > > + * reset will happen and xe_wait_ufence will end return EIO not
> > ETIME
> > > + */
> > > + do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
> > > +
> > > + xe_exec_queue_destroy(fd, exec_queue);
> > > +
> > > + if (bo) {
> > > + munmap(data, 0x40000);
> > > + gem_close(fd, bo);
> > > + }
> > > +}
> > >
> > > igt_main
> > > {
> > > @@ -250,6 +329,9 @@ igt_main
> > > igt_subtest("invalid-engine")
> > > invalid_engine(fd);
> > >
> > > + igt_subtest("invalid-exec_queue-wait")
> > > + invalid_exec_queue_wait(fd);
> > > +
> > > igt_fixture
> > > drm_close_driver(fd);
> > > }
> > > --
> > > 2.25.1
> > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 13:05 ` Matthew Brost
@ 2023-12-11 5:36 ` Bommu, Krishnaiah
2023-12-11 7:03 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-11 5:36 UTC (permalink / raw)
To: Brost, Matthew, Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Brost, Matthew <matthew.brost@intel.com>
> Sent: Friday, December 8, 2023 6:35 PM
> To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> dev@lists.freedesktop.org
> Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> drm_xe_wait_user_fence structure
>
> On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > remove the num_engines/instances members from
> drm_xe_wait_user_fence
> > > structure and add a exec_queue_id member
> > >
> > > Right now this is only checking if the engine list is sane and
> > > nothing else. In the end every operation with this IOCTL is a soft check.
> > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > >
> > > exec_queue_id member will help to user space to get proper error
> > > code from kernel while in exec_queue reset
> > >
> > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > ---
> > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > lib/xe/xe_ioctl.h | 9 +++------
> > > tests/intel/xe_evict.c | 4 ++--
> > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > tests/intel/xe_exec_reset.c | 6 +++---
> > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > >
> > > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > > index 590f7b7af..fd06e4920 100644
> > > --- a/include/drm-uapi/xe_drm.h
> > > +++ b/include/drm-uapi/xe_drm.h
> > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > * It is returned as part of the @drm_xe_engine, but it also is used as
> > > * the input of engine selection for both @drm_xe_exec_queue_create
> and
> > > * @drm_xe_query_engine_cycles
> > > - *
> > > */
> > > struct drm_xe_engine_class_instance {
> > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > */
> > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > + /** @engine_class: engine class id */
> > > __u16 engine_class;
> > > -
> > > + /** @engine_instance: engine instance id */
> > > __u16 engine_instance;
> > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > __u16 gt_id;
> > > /** @pad: MBZ */
> > > __u16 pad;
> > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > *
> > > * Note: For userptr and externally imported dma-buf the kernel
> expects
> > > * either 1WAY or 2WAY for the @pat_index.
> > > + *
> > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD
> restrictions
> > > + * on the @pat_index. For such mappings there is no actual memory
> being
> > > + * mapped (the address in the PTE is invalid), so the various PAT
> memory
> > > + * attributes likely do not apply. Simply leaving as zero is one
> > > + * option (still a valid pat_index).
> > > */
> > > __u16 pat_index;
> > >
> > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > /** @op: wait operation (type of comparison) */
> > > __u16 op;
> > >
> > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait
> on VM bind */
> > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > /** @flags: wait flags */
> > > __u16 flags;
> > >
> > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > */
> > > __s64 timeout;
> > >
> > > - /**
> > > - * @num_engines: number of engine instances to wait on, must be
> zero
> > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > - */
> > > - __u64 num_engines;
> > > + /** @exec_queue_id: exec_queue_id returned from
> xe_exec_queue_create_ioctl */
> > > + __u32 exec_queue_id;
> > >
> > > - /**
> > > - * @instances: user pointer to array of
> drm_xe_engine_class_instance to
> > > - * wait on, must be NULL when
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > - */
> > > - __u64 instances;
> > > + /** @pad2: MBZ */
> > > + __u32 pad2;
> > >
> > > /** @reserved: Reserved */
> > > __u64 reserved[2];
> > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > b29ca40ad..0a1ddc86b 100644
> > > --- a/lib/xe/xe_ioctl.c
> > > +++ b/lib/xe/xe_ioctl.c
> > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue,
> uint64_t addr)
> > > * @fd: xe device fd
> > > * @addr: address of value to compare
> > > * @value: expected value (equal) in @address
> > > - * @eci: engine class instance
> > > + * @exec_queue: exec_queue id
> > > * @timeout: pointer to time to wait in nanoseconds
> > > *
> > > * Function compares @value with memory pointed by @addr until they
> are equal.
> > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> exec_queue, uint64_t addr)
> > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > */
> > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t *timeout)
> > > + uint32_t exec_queue, int64_t *timeout)
> > > {
> > > struct drm_xe_wait_user_fence wait = {
> > > .addr = to_user_pointer(addr),
> > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > + .flags = 0,
> > > .value = value,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > - .num_engines = eci ? 1 :0,
> > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > + .exec_queue_id = exec_queue,
> > > };
> > >
> > > igt_assert(timeout);
> > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * @fd: xe device fd
> > > * @addr: address of value to compare
> > > * @value: expected value (equal) in @address
> > > - * @eci: engine class instance
> > > + * @exec_queue: exec_queue id
> > > * @timeout: time to wait in nanoseconds
> > > *
> > > * Function compares @value with memory pointed by @addr until they
> are equal.
> > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > */
> > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout)
> > > + uint32_t exec_queue, int64_t timeout)
> > > {
> > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue,
> > > +&timeout), 0);
> > > return timeout;
> > > }
> > >
> > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * @fd: xe device fd
> > > * @addr: address of value to compare
> > > * @value: expected value (equal) in @address
> > > - * @eci: engine class instance
> > > + * @exec_queue: exec_queue id
> > > * @timeout: absolute time when wait expire
> > > *
> > > * Function compares @value with memory pointed by @addr until they
> are equal.
> > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > */
> > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout)
> > > + uint32_t exec_queue, int64_t timeout)
> > > {
> > > struct drm_xe_wait_user_fence wait = {
> > > .addr = to_user_pointer(addr),
> > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > + .flags = !exec_queue ?
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> >
> > To be on the safe side we should probably change the signature of this
> > function to explicitly receive this flag instead of relying on the invalid
> exec_queue_id number.
> >
> > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP was
> used here...
> >
> > Matt, thoughts?
> >
>
> Agree with everything here, !exec_queue really has nothing to do with
> setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So yes,
> please change this function to accpt a flags field.
>
> Everything else LGTM.
>
> Matt
I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not thing to do with exec_queue, I will change this function to accept flag, but DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is related to "no_engines" I removed from kernel side also https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
Matt please let me know your insights on DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
Regards,
Krishna.
>
> > Everything else in the patch looks good to me.
> >
> > > .value = value,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = timeout,
> > > - .num_engines = eci ? 1 : 0,
> > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > + .exec_queue_id = exec_queue,
> > > };
> > > struct timespec ts;
> > >
> > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > bd660bd27..9c0b4b9bc 100644
> > > --- a/lib/xe/xe_ioctl.h
> > > +++ b/lib/xe/xe_ioctl.h
> > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue,
> uint64_t addr,
> > > struct drm_xe_sync *sync, uint32_t num_syncs); void
> > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t *timeout);
> > > + uint32_t exec_queue, int64_t *timeout);
> > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout);
> > > + uint32_t exec_queue, int64_t timeout);
> > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout);
> > > + uint32_t exec_queue, int64_t timeout);
> > > void xe_force_gt_reset(int fd, int gt);
> > >
> > > #endif /* XE_IOCTL_H */
> > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c index
> > > 89dc46fae..a37c2d97a 100644
> > > --- a/tests/intel/xe_evict.c
> > > +++ b/tests/intel/xe_evict.c
> > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > }
> > > #define TWENTY_SEC MS_TO_NS(20000)
> > > xe_wait_ufence(fd, &data[i].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, TWENTY_SEC);
> > > + bind_exec_queues[0], TWENTY_SEC);
> > > }
> > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > (char *)data;
> > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > data = xe_bo_map(fd, __bo,
> > > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, TWENTY_SEC);
> > > + exec_queues[i], TWENTY_SEC);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > }
> > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff --git
> > > a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > > index 79ff65e89..2448d49bc 100644
> > > --- a/tests/intel/xe_exec_balancer.c
> > > +++ b/tests/intel/xe_exec_balancer.c
> > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> n_exec_queues, int n_execs,
> > > bo_size, sync, 1);
> > >
> > > #define ONE_SEC MS_TO_NS(1000)
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +ONE_SEC);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > >
> > > if (flags & REBIND && i + 1 != n_execs) {
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + exec_queues[e], ONE_SEC);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> NULL,
> > > 0);
> > >
> > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> n_exec_queues, int n_execs,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + 0, ONE_SEC);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> n_exec_queues, int n_execs,
> > > * an invalidate.
> > > */
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + USER_FENCE_VALUE,
> exec_queues[e],
> > > + ONE_SEC);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > /*
> > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > >
> > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > for (i = j; i < n_execs; i++)
> > > - xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE, NULL,
> > > - ONE_SEC);
> > > + xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > + exec_queues[i], ONE_SEC);
> > >
> > > /* Wait for all execs to complete */
> > > if (flags & INVALIDATE)
> > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +ONE_SEC);
> > >
> > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > i < n_execs; i++)
> > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > b/tests/intel/xe_exec_compute_mode.c
> > > index 7d3004d65..749a9b634 100644
> > > --- a/tests/intel/xe_exec_compute_mode.c
> > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> ONE_SEC;
> > >
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], fence_timeout);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -198,7 +198,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > if (flags & REBIND && i + 1 != n_execs) {
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + exec_queues[e], fence_timeout);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> 0,
> > > addr, bo_size, NULL, 0);
> > >
> > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + bind_exec_queues[e], fence_timeout);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > * an invalidate.
> > > */
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL,
> > > + USER_FENCE_VALUE,
> exec_queues[e],
> > > fence_timeout);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > @@ -257,8 +257,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > for (i = j; i < n_execs; i++)
> > > - xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > + exec_queues[i], fence_timeout);
> > >
> > > /* Wait for all execs to complete */
> > > if (flags & INVALIDATE)
> > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> bo_size,
> > > sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], fence_timeout);
> > >
> > > for (i = j; i < n_execs; i++)
> > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > a/tests/intel/xe_exec_fault_mode.c
> > > b/tests/intel/xe_exec_fault_mode.c
> > > index ee7cbb604..d76b3a056 100644
> > > --- a/tests/intel/xe_exec_fault_mode.c
> > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > }
> > >
> > > #define ONE_SEC MS_TO_NS(1000)
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], ONE_SEC);
> > > data[0].vm_sync = 0;
> > >
> > > if (flags & PREFETCH) {
> > > /* Should move to system memory */
> > > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0,
> addr,
> > > bo_size, sync, 1, 0);
> > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > - ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > + bind_exec_queues[0], ONE_SEC);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > if (flags & REBIND && i + 1 != n_execs) {
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + exec_queues[e], ONE_SEC);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> 0,
> > > addr, bo_size, NULL, 0);
> > >
> > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + bind_exec_queues[e], ONE_SEC);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > * an invalidate.
> > > */
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + USER_FENCE_VALUE,
> exec_queues[e],
> > > + ONE_SEC);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > /*
> > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > for (i = j; i < n_execs; i++)
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > > + USER_FENCE_VALUE, exec_queues[i],
> ONE_SEC);
> > > }
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> bo_size,
> > > sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], ONE_SEC);
> > >
> > > if (!(flags & INVALID_FAULT)) {
> > > for (i = j; i < n_execs; i++)
> > > diff --git a/tests/intel/xe_exec_reset.c
> > > b/tests/intel/xe_exec_reset.c index 094b34896..398c90af5 100644
> > > --- a/tests/intel/xe_exec_reset.c
> > > +++ b/tests/intel/xe_exec_reset.c
> > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > >
> > > #define THREE_SEC MS_TO_NS(3000)
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> THREE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +THREE_SEC);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > int err;
> > >
> > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, &timeout);
> > > + exec_queues[i], &timeout);
> > > if (flags & GT_RESET)
> > > /* exec races with reset: may timeout or complete */
> > > igt_assert(err == -ETIME || !err); @@ -631,7 +631,7
> @@
> > > test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> THREE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +THREE_SEC);
> > >
> > > if (!(flags & GT_RESET)) {
> > > for (i = 1; i < n_execs; i++)
> > > diff --git a/tests/intel/xe_exec_threads.c
> > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9 100644
> > > --- a/tests/intel/xe_exec_threads.c
> > > +++ b/tests/intel/xe_exec_threads.c
> > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> > > addr, uint64_t userptr,
> > >
> > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
> > >
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +fence_timeout);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > for (j = i - 0x20; j <= i; ++j)
> > > xe_wait_ufence(fd, &data[j].exec_sync,
> > > USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + exec_queues[e], fence_timeout);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > NULL, 0);
> > >
> > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + 0, fence_timeout);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > > xe_wait_ufence(fd,
> &data[j].exec_sync,
> > > USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + exec_queues[e],
> > > + fence_timeout);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > /*
> > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > j = flags & INVALIDATE ?
> > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > for (i = j; i < n_execs; i++)
> > > - xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > + exec_queues[e], fence_timeout);
> > >
> > > /* Wait for all execs to complete */
> > > if (flags & INVALIDATE)
> > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> > > addr, uint64_t userptr,
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +fence_timeout);
> > >
> > > for (i = j; i < n_execs; i++)
> > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c index
> > > 3be987954..4e94403a3 100644
> > > --- a/tests/intel/xe_waitfence.c
> > > +++ b/tests/intel/xe_waitfence.c
> > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > uint32_t bo, uint64_t offset, }
> > >
> > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t
> value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout)
> > > + uint32_t exec_queue, int64_t timeout)
> > > {
> > > struct drm_xe_wait_user_fence wait = {
> > > .addr = to_user_pointer(addr),
> > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > + .flags = !exec_queue ? 0 :
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > .value = value,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = timeout,
> > > - .num_engines = eci ? 1 : 0,
> > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > + .exec_queue_id = exec_queue,
> > > };
> > > struct timespec ts;
> > >
> > > - igt_assert(eci);
> > > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE,
> &wait), 0);
> > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > >
> > > @@ -82,7 +79,7 @@ enum waittype {
> > > static void
> > > waitfence(int fd, enum waittype wt) {
> > > - struct drm_xe_engine *engine = NULL;
> > > + uint32_t exec_queue;
> > > struct timespec ts;
> > > int64_t current, signalled;
> > > uint32_t bo_1;
> > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > >
> > > if (wt == RELTIME) {
> > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> MS_TO_NS(10));
> > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> MS_TO_NS(10));
> > > igt_debug("wait type: RELTIME - timeout: %ld, timeout left:
> %ld\n",
> > > MS_TO_NS(10), timeout);
> > > } else if (wt == ENGINE) {
> > > - engine = xe_engine(fd, 1);
> > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > +DRM_XE_ENGINE_CLASS_COPY);
> > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > timeout = current + MS_TO_NS(10);
> > > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> &engine->instance, timeout);
> > > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> exec_queue,
> > > +timeout);
> > > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > > ", signalled: %" PRId64
> > > ", elapsed: %" PRId64 "\n",
> > > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > timeout = current + MS_TO_NS(10);
> > > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7,
> NULL, timeout);
> > > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0,
> > > +timeout);
> > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > ", signalled: %" PRId64
> > > ", elapsed: %" PRId64 "\n",
> > > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > > .value = 1,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = 1,
> > > - .num_engines = 0,
> > > - .instances = 0,
> > > + .exec_queue_id = 0,
> > > };
> > >
> > > uint32_t vm = xe_vm_create(fd,
> > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8 +212,7
> @@ invalid_engine(int fd)
> > > .value = 1,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = -1,
> > > - .num_engines = 1,
> > > - .instances = 0,
> > > + .exec_queue_id = 0,
> > > };
> > >
> > > uint32_t vm = xe_vm_create(fd,
> > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > --
> > > 2.25.1
> > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-11 5:36 ` Bommu, Krishnaiah
@ 2023-12-11 7:03 ` Bommu, Krishnaiah
2023-12-11 18:58 ` Rodrigo Vivi
0 siblings, 1 reply; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-11 7:03 UTC (permalink / raw)
To: Brost, Matthew, Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Bommu, Krishnaiah
> Sent: Monday, December 11, 2023 11:06 AM
> To: Brost, Matthew <matthew.brost@intel.com>; Vivi, Rodrigo
> <rodrigo.vivi@intel.com>
> Cc: igt-dev@lists.freedesktop.org
> Subject: RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> drm_xe_wait_user_fence structure
>
>
>
> > -----Original Message-----
> > From: Brost, Matthew <matthew.brost@intel.com>
> > Sent: Friday, December 8, 2023 6:35 PM
> > To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> > dev@lists.freedesktop.org
> > Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > drm_xe_wait_user_fence structure
> >
> > On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > > remove the num_engines/instances members from
> > drm_xe_wait_user_fence
> > > > structure and add a exec_queue_id member
> > > >
> > > > Right now this is only checking if the engine list is sane and
> > > > nothing else. In the end every operation with this IOCTL is a soft check.
> > > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > > >
> > > > exec_queue_id member will help to user space to get proper error
> > > > code from kernel while in exec_queue reset
> > > >
> > > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > > ---
> > > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > > lib/xe/xe_ioctl.h | 9 +++------
> > > > tests/intel/xe_evict.c | 4 ++--
> > > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > > tests/intel/xe_exec_reset.c | 6 +++---
> > > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > > >
> > > > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > > > index 590f7b7af..fd06e4920 100644
> > > > --- a/include/drm-uapi/xe_drm.h
> > > > +++ b/include/drm-uapi/xe_drm.h
> > > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > > * It is returned as part of the @drm_xe_engine, but it also is used as
> > > > * the input of engine selection for both
> > > > @drm_xe_exec_queue_create
> > and
> > > > * @drm_xe_query_engine_cycles
> > > > - *
> > > > */
> > > > struct drm_xe_engine_class_instance {
> > > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > > */
> > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > > + /** @engine_class: engine class id */
> > > > __u16 engine_class;
> > > > -
> > > > + /** @engine_instance: engine instance id */
> > > > __u16 engine_instance;
> > > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > > __u16 gt_id;
> > > > /** @pad: MBZ */
> > > > __u16 pad;
> > > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > > *
> > > > * Note: For userptr and externally imported dma-buf the kernel
> > expects
> > > > * either 1WAY or 2WAY for the @pat_index.
> > > > + *
> > > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD
> > restrictions
> > > > + * on the @pat_index. For such mappings there is no actual
> > > > +memory
> > being
> > > > + * mapped (the address in the PTE is invalid), so the various
> > > > +PAT
> > memory
> > > > + * attributes likely do not apply. Simply leaving as zero is one
> > > > + * option (still a valid pat_index).
> > > > */
> > > > __u16 pat_index;
> > > >
> > > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > > /** @op: wait operation (type of comparison) */
> > > > __u16 op;
> > > >
> > > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /*
> e.g. Wait
> > on VM bind */
> > > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > > /** @flags: wait flags */
> > > > __u16 flags;
> > > >
> > > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > > */
> > > > __s64 timeout;
> > > >
> > > > - /**
> > > > - * @num_engines: number of engine instances to wait on, must be
> > zero
> > > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > - */
> > > > - __u64 num_engines;
> > > > + /** @exec_queue_id: exec_queue_id returned from
> > xe_exec_queue_create_ioctl */
> > > > + __u32 exec_queue_id;
> > > >
> > > > - /**
> > > > - * @instances: user pointer to array of
> > drm_xe_engine_class_instance to
> > > > - * wait on, must be NULL when
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > - */
> > > > - __u64 instances;
> > > > + /** @pad2: MBZ */
> > > > + __u32 pad2;
> > > >
> > > > /** @reserved: Reserved */
> > > > __u64 reserved[2];
> > > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > > b29ca40ad..0a1ddc86b 100644
> > > > --- a/lib/xe/xe_ioctl.c
> > > > +++ b/lib/xe/xe_ioctl.c
> > > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue,
> > uint64_t addr)
> > > > * @fd: xe device fd
> > > > * @addr: address of value to compare
> > > > * @value: expected value (equal) in @address
> > > > - * @eci: engine class instance
> > > > + * @exec_queue: exec_queue id
> > > > * @timeout: pointer to time to wait in nanoseconds
> > > > *
> > > > * Function compares @value with memory pointed by @addr until
> > > > they
> > are equal.
> > > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> > exec_queue, uint64_t addr)
> > > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > > */
> > > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t *timeout)
> > > > + uint32_t exec_queue, int64_t *timeout)
> > > > {
> > > > struct drm_xe_wait_user_fence wait = {
> > > > .addr = to_user_pointer(addr),
> > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > > + .flags = 0,
> > > > .value = value,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > - .num_engines = eci ? 1 :0,
> > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > + .exec_queue_id = exec_queue,
> > > > };
> > > >
> > > > igt_assert(timeout);
> > > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > uint64_t value,
> > > > * @fd: xe device fd
> > > > * @addr: address of value to compare
> > > > * @value: expected value (equal) in @address
> > > > - * @eci: engine class instance
> > > > + * @exec_queue: exec_queue id
> > > > * @timeout: time to wait in nanoseconds
> > > > *
> > > > * Function compares @value with memory pointed by @addr until
> > > > they
> > are equal.
> > > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > uint64_t value,
> > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > */
> > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout)
> > > > + uint32_t exec_queue, int64_t timeout)
> > > > {
> > > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue,
> > > > +&timeout), 0);
> > > > return timeout;
> > > > }
> > > >
> > > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> > uint64_t value,
> > > > * @fd: xe device fd
> > > > * @addr: address of value to compare
> > > > * @value: expected value (equal) in @address
> > > > - * @eci: engine class instance
> > > > + * @exec_queue: exec_queue id
> > > > * @timeout: absolute time when wait expire
> > > > *
> > > > * Function compares @value with memory pointed by @addr until
> > > > they
> > are equal.
> > > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > *addr,
> > uint64_t value,
> > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > */
> > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout)
> > > > + uint32_t exec_queue, int64_t timeout)
> > > > {
> > > > struct drm_xe_wait_user_fence wait = {
> > > > .addr = to_user_pointer(addr),
> > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > + .flags = !exec_queue ?
> > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > >
> > > To be on the safe side we should probably change the signature of
> > > this function to explicitly receive this flag instead of relying on
> > > the invalid
> > exec_queue_id number.
> > >
> > > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP
> was
> > used here...
> > >
> > > Matt, thoughts?
> > >
> >
> > Agree with everything here, !exec_queue really has nothing to do with
> > setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So
> yes,
> > please change this function to accpt a flags field.
> >
> > Everything else LGTM.
> >
> > Matt
>
> I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not thing
> to do with exec_queue, I will change this function to accept flag, but
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is related to
> "no_engines" I removed from kernel side also
> https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
>
> Matt please let me know your insights on
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
>
> Regards,
> Krishna.
> >
> > > Everything else in the patch looks good to me.
> > >
> > > > .value = value,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = timeout,
> > > > - .num_engines = eci ? 1 : 0,
> > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > + .exec_queue_id = exec_queue,
> > > > };
> > > > struct timespec ts;
> > > >
> > > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > > bd660bd27..9c0b4b9bc 100644
> > > > --- a/lib/xe/xe_ioctl.h
> > > > +++ b/lib/xe/xe_ioctl.h
> > > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue,
> > uint64_t addr,
> > > > struct drm_xe_sync *sync, uint32_t num_syncs); void
> > > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t *timeout);
> > > > + uint32_t exec_queue, int64_t *timeout);
> > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout);
> > > > + uint32_t exec_queue, int64_t timeout);
> > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout);
> > > > + uint32_t exec_queue, int64_t timeout);
> > > > void xe_force_gt_reset(int fd, int gt);
> > > >
> > > > #endif /* XE_IOCTL_H */
> > > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c index
> > > > 89dc46fae..a37c2d97a 100644
> > > > --- a/tests/intel/xe_evict.c
> > > > +++ b/tests/intel/xe_evict.c
> > > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > }
> > > > #define TWENTY_SEC MS_TO_NS(20000)
> > > > xe_wait_ufence(fd, &data[i].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, TWENTY_SEC);
> > > > + bind_exec_queues[0], TWENTY_SEC);
> > > > }
> > > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > > (char *)data;
> > > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > data = xe_bo_map(fd, __bo,
> > > > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, TWENTY_SEC);
> > > > + exec_queues[i], TWENTY_SEC);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > }
> > > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff --git
> > > > a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > > > index 79ff65e89..2448d49bc 100644
> > > > --- a/tests/intel/xe_exec_balancer.c
> > > > +++ b/tests/intel/xe_exec_balancer.c
> > > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> > n_exec_queues, int n_execs,
> > > > bo_size, sync, 1);
> > > >
> > > > #define ONE_SEC MS_TO_NS(1000)
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > >
> > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + exec_queues[e], ONE_SEC);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > NULL,
> > > > 0);
> > > >
> > > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> > n_exec_queues, int n_execs,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + 0, ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> > n_exec_queues, int n_execs,
> > > > * an invalidate.
> > > > */
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + USER_FENCE_VALUE,
> > exec_queues[e],
> > > > + ONE_SEC);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) {
> > > > /*
> > > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > >
> > > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > + exec_queues[i], ONE_SEC);
Need to change exec_queues[i] index to "i % n_exec_queues", need to change other files also, I will change
exec_queues[i] => exec_queues[i % n_exec_queues]
Regards,
Krishna
> > > >
> > > > /* Wait for all execs to complete */
> > > > if (flags & INVALIDATE)
> > > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +ONE_SEC);
> > > >
> > > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > > i < n_execs; i++)
> > > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > > b/tests/intel/xe_exec_compute_mode.c
> > > > index 7d3004d65..749a9b634 100644
> > > > --- a/tests/intel/xe_exec_compute_mode.c
> > > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > >
> > > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> > ONE_SEC;
> > > >
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], fence_timeout);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -198,7 +198,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > >
> > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + exec_queues[e], fence_timeout);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > 0,
> > > > addr, bo_size, NULL, 0);
> > > >
> > > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + bind_exec_queues[e], fence_timeout);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > * an invalidate.
> > > > */
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL,
> > > > + USER_FENCE_VALUE,
> > exec_queues[e],
> > > > fence_timeout);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) { @@ -257,8 +257,8 @@
> > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > >
> > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > + exec_queues[i], fence_timeout);
> > > >
> > > > /* Wait for all execs to complete */
> > > > if (flags & INVALIDATE)
> > > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > bo_size,
> > > > sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], fence_timeout);
> > > >
> > > > for (i = j; i < n_execs; i++)
> > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > a/tests/intel/xe_exec_fault_mode.c
> > > > b/tests/intel/xe_exec_fault_mode.c
> > > > index ee7cbb604..d76b3a056 100644
> > > > --- a/tests/intel/xe_exec_fault_mode.c
> > > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > }
> > > >
> > > > #define ONE_SEC MS_TO_NS(1000)
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > >
> > > > if (flags & PREFETCH) {
> > > > /* Should move to system memory */
> > > > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0,
> > addr,
> > > > bo_size, sync, 1, 0);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > >
> > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + exec_queues[e], ONE_SEC);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > 0,
> > > > addr, bo_size, NULL, 0);
> > > >
> > > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + bind_exec_queues[e], ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > * an invalidate.
> > > > */
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + USER_FENCE_VALUE,
> > exec_queues[e],
> > > > + ONE_SEC);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) {
> > > > /*
> > > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > > > + USER_FENCE_VALUE, exec_queues[i],
> > ONE_SEC);
> > > > }
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > bo_size,
> > > > sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], ONE_SEC);
> > > >
> > > > if (!(flags & INVALID_FAULT)) {
> > > > for (i = j; i < n_execs; i++)
> > > > diff --git a/tests/intel/xe_exec_reset.c
> > > > b/tests/intel/xe_exec_reset.c index 094b34896..398c90af5 100644
> > > > --- a/tests/intel/xe_exec_reset.c
> > > > +++ b/tests/intel/xe_exec_reset.c
> > > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > > >
> > > > #define THREE_SEC MS_TO_NS(3000)
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > THREE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +THREE_SEC);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > int err;
> > > >
> > > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, &timeout);
> > > > + exec_queues[i], &timeout);
> > > > if (flags & GT_RESET)
> > > > /* exec races with reset: may timeout or complete */
> > > > igt_assert(err == -ETIME || !err); @@ -631,7 +631,7
> > @@
> > > > test_compute_mode(int fd, struct drm_xe_engine_class_instance
> > > > *eci,
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > THREE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +THREE_SEC);
> > > >
> > > > if (!(flags & GT_RESET)) {
> > > > for (i = 1; i < n_execs; i++)
> > > > diff --git a/tests/intel/xe_exec_threads.c
> > > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9 100644
> > > > --- a/tests/intel/xe_exec_threads.c
> > > > +++ b/tests/intel/xe_exec_threads.c
> > > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t addr, uint64_t userptr,
> > > >
> > > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC :
> > > > THREE_SEC;
> > > >
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +fence_timeout);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > for (j = i - 0x20; j <= i; ++j)
> > > > xe_wait_ufence(fd, &data[j].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + exec_queues[e], fence_timeout);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > > NULL, 0);
> > > >
> > > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + 0, fence_timeout);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > > > xe_wait_ufence(fd,
> > &data[j].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + exec_queues[e],
> > > > + fence_timeout);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) {
> > > > /*
> > > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > j = flags & INVALIDATE ?
> > > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > + exec_queues[e], fence_timeout);
> > > >
> > > > /* Wait for all execs to complete */
> > > > if (flags & INVALIDATE)
> > > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t addr, uint64_t userptr,
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +fence_timeout);
> > > >
> > > > for (i = j; i < n_execs; i++)
> > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c index
> > > > 3be987954..4e94403a3 100644
> > > > --- a/tests/intel/xe_waitfence.c
> > > > +++ b/tests/intel/xe_waitfence.c
> > > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > > uint32_t bo, uint64_t offset, }
> > > >
> > > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr,
> > > > uint64_t
> > value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout)
> > > > + uint32_t exec_queue, int64_t timeout)
> > > > {
> > > > struct drm_xe_wait_user_fence wait = {
> > > > .addr = to_user_pointer(addr),
> > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > + .flags = !exec_queue ? 0 :
> > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > .value = value,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = timeout,
> > > > - .num_engines = eci ? 1 : 0,
> > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > + .exec_queue_id = exec_queue,
> > > > };
> > > > struct timespec ts;
> > > >
> > > > - igt_assert(eci);
> > > > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE,
> > &wait), 0);
> > > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > > >
> > > > @@ -82,7 +79,7 @@ enum waittype {
> > > > static void
> > > > waitfence(int fd, enum waittype wt) {
> > > > - struct drm_xe_engine *engine = NULL;
> > > > + uint32_t exec_queue;
> > > > struct timespec ts;
> > > > int64_t current, signalled;
> > > > uint32_t bo_1;
> > > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > > >
> > > > if (wt == RELTIME) {
> > > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> > MS_TO_NS(10));
> > > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> > MS_TO_NS(10));
> > > > igt_debug("wait type: RELTIME - timeout: %ld, timeout left:
> > %ld\n",
> > > > MS_TO_NS(10), timeout);
> > > > } else if (wt == ENGINE) {
> > > > - engine = xe_engine(fd, 1);
> > > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > > +DRM_XE_ENGINE_CLASS_COPY);
> > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > timeout = current + MS_TO_NS(10);
> > > > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > &engine->instance, timeout);
> > > > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > exec_queue,
> > > > +timeout);
> > > > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > > > ", signalled: %" PRId64
> > > > ", elapsed: %" PRId64 "\n",
> > > > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > timeout = current + MS_TO_NS(10);
> > > > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7,
> > NULL, timeout);
> > > > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0,
> > > > +timeout);
> > > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > > ", signalled: %" PRId64
> > > > ", elapsed: %" PRId64 "\n",
> > > > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > > > .value = 1,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = 1,
> > > > - .num_engines = 0,
> > > > - .instances = 0,
> > > > + .exec_queue_id = 0,
> > > > };
> > > >
> > > > uint32_t vm = xe_vm_create(fd,
> > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8 +212,7
> > @@ invalid_engine(int fd)
> > > > .value = 1,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = -1,
> > > > - .num_engines = 1,
> > > > - .instances = 0,
> > > > + .exec_queue_id = 0,
> > > > };
> > > >
> > > > uint32_t vm = xe_vm_create(fd,
> > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > > --
> > > > 2.25.1
> > > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-11 7:03 ` Bommu, Krishnaiah
@ 2023-12-11 18:58 ` Rodrigo Vivi
2023-12-12 4:23 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-11 18:58 UTC (permalink / raw)
To: Bommu, Krishnaiah; +Cc: igt-dev@lists.freedesktop.org
On Mon, Dec 11, 2023 at 07:03:47AM +0000, Bommu, Krishnaiah wrote:
>
>
> > -----Original Message-----
> > From: Bommu, Krishnaiah
> > Sent: Monday, December 11, 2023 11:06 AM
> > To: Brost, Matthew <matthew.brost@intel.com>; Vivi, Rodrigo
> > <rodrigo.vivi@intel.com>
> > Cc: igt-dev@lists.freedesktop.org
> > Subject: RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > drm_xe_wait_user_fence structure
> >
> >
> >
> > > -----Original Message-----
> > > From: Brost, Matthew <matthew.brost@intel.com>
> > > Sent: Friday, December 8, 2023 6:35 PM
> > > To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> > > dev@lists.freedesktop.org
> > > Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > > drm_xe_wait_user_fence structure
> > >
> > > On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > > > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > > > remove the num_engines/instances members from
> > > drm_xe_wait_user_fence
> > > > > structure and add a exec_queue_id member
> > > > >
> > > > > Right now this is only checking if the engine list is sane and
> > > > > nothing else. In the end every operation with this IOCTL is a soft check.
> > > > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > > > >
> > > > > exec_queue_id member will help to user space to get proper error
> > > > > code from kernel while in exec_queue reset
> > > > >
> > > > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > > > ---
> > > > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > > > lib/xe/xe_ioctl.h | 9 +++------
> > > > > tests/intel/xe_evict.c | 4 ++--
> > > > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > > > tests/intel/xe_exec_reset.c | 6 +++---
> > > > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > > > >
> > > > > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > > > > index 590f7b7af..fd06e4920 100644
> > > > > --- a/include/drm-uapi/xe_drm.h
> > > > > +++ b/include/drm-uapi/xe_drm.h
> > > > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > > > * It is returned as part of the @drm_xe_engine, but it also is used as
> > > > > * the input of engine selection for both
> > > > > @drm_xe_exec_queue_create
> > > and
> > > > > * @drm_xe_query_engine_cycles
> > > > > - *
> > > > > */
> > > > > struct drm_xe_engine_class_instance {
> > > > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > > > */
> > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > > > + /** @engine_class: engine class id */
> > > > > __u16 engine_class;
> > > > > -
> > > > > + /** @engine_instance: engine instance id */
> > > > > __u16 engine_instance;
> > > > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > > > __u16 gt_id;
> > > > > /** @pad: MBZ */
> > > > > __u16 pad;
> > > > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > > > *
> > > > > * Note: For userptr and externally imported dma-buf the kernel
> > > expects
> > > > > * either 1WAY or 2WAY for the @pat_index.
> > > > > + *
> > > > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD
> > > restrictions
> > > > > + * on the @pat_index. For such mappings there is no actual
> > > > > +memory
> > > being
> > > > > + * mapped (the address in the PTE is invalid), so the various
> > > > > +PAT
> > > memory
> > > > > + * attributes likely do not apply. Simply leaving as zero is one
> > > > > + * option (still a valid pat_index).
> > > > > */
> > > > > __u16 pat_index;
> > > > >
> > > > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > > > /** @op: wait operation (type of comparison) */
> > > > > __u16 op;
> > > > >
> > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /*
> > e.g. Wait
> > > on VM bind */
> > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > > > /** @flags: wait flags */
> > > > > __u16 flags;
> > > > >
> > > > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > > > */
> > > > > __s64 timeout;
> > > > >
> > > > > - /**
> > > > > - * @num_engines: number of engine instances to wait on, must be
> > > zero
> > > > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > - */
> > > > > - __u64 num_engines;
> > > > > + /** @exec_queue_id: exec_queue_id returned from
> > > xe_exec_queue_create_ioctl */
> > > > > + __u32 exec_queue_id;
> > > > >
> > > > > - /**
> > > > > - * @instances: user pointer to array of
> > > drm_xe_engine_class_instance to
> > > > > - * wait on, must be NULL when
> > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > - */
> > > > > - __u64 instances;
> > > > > + /** @pad2: MBZ */
> > > > > + __u32 pad2;
> > > > >
> > > > > /** @reserved: Reserved */
> > > > > __u64 reserved[2];
> > > > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > > > b29ca40ad..0a1ddc86b 100644
> > > > > --- a/lib/xe/xe_ioctl.c
> > > > > +++ b/lib/xe/xe_ioctl.c
> > > > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue,
> > > uint64_t addr)
> > > > > * @fd: xe device fd
> > > > > * @addr: address of value to compare
> > > > > * @value: expected value (equal) in @address
> > > > > - * @eci: engine class instance
> > > > > + * @exec_queue: exec_queue id
> > > > > * @timeout: pointer to time to wait in nanoseconds
> > > > > *
> > > > > * Function compares @value with memory pointed by @addr until
> > > > > they
> > > are equal.
> > > > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> > > exec_queue, uint64_t addr)
> > > > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > > > */
> > > > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t *timeout)
> > > > > + uint32_t exec_queue, int64_t *timeout)
> > > > > {
> > > > > struct drm_xe_wait_user_fence wait = {
> > > > > .addr = to_user_pointer(addr),
> > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > > > + .flags = 0,
> > > > > .value = value,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > - .num_engines = eci ? 1 :0,
> > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > + .exec_queue_id = exec_queue,
> > > > > };
> > > > >
> > > > > igt_assert(timeout);
> > > > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > > uint64_t value,
> > > > > * @fd: xe device fd
> > > > > * @addr: address of value to compare
> > > > > * @value: expected value (equal) in @address
> > > > > - * @eci: engine class instance
> > > > > + * @exec_queue: exec_queue id
> > > > > * @timeout: time to wait in nanoseconds
> > > > > *
> > > > > * Function compares @value with memory pointed by @addr until
> > > > > they
> > > are equal.
> > > > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > > uint64_t value,
> > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > */
> > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout)
> > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > {
> > > > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > > > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue,
> > > > > +&timeout), 0);
> > > > > return timeout;
> > > > > }
> > > > >
> > > > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> > > uint64_t value,
> > > > > * @fd: xe device fd
> > > > > * @addr: address of value to compare
> > > > > * @value: expected value (equal) in @address
> > > > > - * @eci: engine class instance
> > > > > + * @exec_queue: exec_queue id
> > > > > * @timeout: absolute time when wait expire
> > > > > *
> > > > > * Function compares @value with memory pointed by @addr until
> > > > > they
> > > are equal.
> > > > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > > *addr,
> > > uint64_t value,
> > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > */
> > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> > value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout)
> > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > {
> > > > > struct drm_xe_wait_user_fence wait = {
> > > > > .addr = to_user_pointer(addr),
> > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > > + .flags = !exec_queue ?
> > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > >
> > > > To be on the safe side we should probably change the signature of
> > > > this function to explicitly receive this flag instead of relying on
> > > > the invalid
> > > exec_queue_id number.
> > > >
> > > > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP
> > was
> > > used here...
> > > >
> > > > Matt, thoughts?
> > > >
> > >
> > > Agree with everything here, !exec_queue really has nothing to do with
> > > setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So
> > yes,
> > > please change this function to accpt a flags field.
> > >
> > > Everything else LGTM.
> > >
> > > Matt
> >
> > I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not thing
> > to do with exec_queue, I will change this function to accept flag, but
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is related to
> > "no_engines" I removed from kernel side also
> > https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
> >
> > Matt please let me know your insights on
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
> >
> > Regards,
> > Krishna.
> > >
> > > > Everything else in the patch looks good to me.
> > > >
> > > > > .value = value,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = timeout,
> > > > > - .num_engines = eci ? 1 : 0,
> > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > + .exec_queue_id = exec_queue,
> > > > > };
> > > > > struct timespec ts;
> > > > >
> > > > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > > > bd660bd27..9c0b4b9bc 100644
> > > > > --- a/lib/xe/xe_ioctl.h
> > > > > +++ b/lib/xe/xe_ioctl.h
> > > > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue,
> > > uint64_t addr,
> > > > > struct drm_xe_sync *sync, uint32_t num_syncs); void
> > > > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t *timeout);
> > > > > + uint32_t exec_queue, int64_t *timeout);
> > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout);
> > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> > value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout);
> > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > void xe_force_gt_reset(int fd, int gt);
> > > > >
> > > > > #endif /* XE_IOCTL_H */
> > > > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c index
> > > > > 89dc46fae..a37c2d97a 100644
> > > > > --- a/tests/intel/xe_evict.c
> > > > > +++ b/tests/intel/xe_evict.c
> > > > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > }
> > > > > #define TWENTY_SEC MS_TO_NS(20000)
> > > > > xe_wait_ufence(fd, &data[i].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, TWENTY_SEC);
> > > > > + bind_exec_queues[0], TWENTY_SEC);
> > > > > }
> > > > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > > > (char *)data;
> > > > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > data = xe_bo_map(fd, __bo,
> > > > > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, TWENTY_SEC);
> > > > > + exec_queues[i], TWENTY_SEC);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > }
> > > > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff --git
> > > > > a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > > > > index 79ff65e89..2448d49bc 100644
> > > > > --- a/tests/intel/xe_exec_balancer.c
> > > > > +++ b/tests/intel/xe_exec_balancer.c
> > > > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > > > > bo_size, sync, 1);
> > > > >
> > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int
> > > > > n_exec_queues, int n_execs,
> > > > >
> > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + exec_queues[e], ONE_SEC);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > NULL,
> > > > > 0);
> > > > >
> > > > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + 0, ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > > > > * an invalidate.
> > > > > */
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + USER_FENCE_VALUE,
> > > exec_queues[e],
> > > > > + ONE_SEC);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) {
> > > > > /*
> > > > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > > > n_exec_queues, int n_execs,
> > > > >
> > > > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > + exec_queues[i], ONE_SEC);
>
> Need to change exec_queues[i] index to "i % n_exec_queues", need to change other files also, I will change
> exec_queues[i] => exec_queues[i % n_exec_queues]
why?
>
> Regards,
> Krishna
>
> > > > >
> > > > > /* Wait for all execs to complete */
> > > > > if (flags & INVALIDATE)
> > > > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > > > n_exec_queues, int n_execs,
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +ONE_SEC);
> > > > >
> > > > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > > > i < n_execs; i++)
> > > > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > > > b/tests/intel/xe_exec_compute_mode.c
> > > > > index 7d3004d65..749a9b634 100644
> > > > > --- a/tests/intel/xe_exec_compute_mode.c
> > > > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > > > drm_xe_engine_class_instance *eci,
> > > > >
> > > > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> > > ONE_SEC;
> > > > >
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -198,7 +198,7 @@ test_exec(int fd, struct
> > > > > drm_xe_engine_class_instance *eci,
> > > > >
> > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + exec_queues[e], fence_timeout);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > > 0,
> > > > > addr, bo_size, NULL, 0);
> > > > >
> > > > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + bind_exec_queues[e], fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > * an invalidate.
> > > > > */
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL,
> > > > > + USER_FENCE_VALUE,
> > > exec_queues[e],
> > > > > fence_timeout);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) { @@ -257,8 +257,8 @@
> > > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > > >
> > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > + exec_queues[i], fence_timeout);
> > > > >
> > > > > /* Wait for all execs to complete */
> > > > > if (flags & INVALIDATE)
> > > > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > bo_size,
> > > > > sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], fence_timeout);
> > > > >
> > > > > for (i = j; i < n_execs; i++)
> > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > a/tests/intel/xe_exec_fault_mode.c
> > > > > b/tests/intel/xe_exec_fault_mode.c
> > > > > index ee7cbb604..d76b3a056 100644
> > > > > --- a/tests/intel/xe_exec_fault_mode.c
> > > > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > }
> > > > >
> > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > if (flags & PREFETCH) {
> > > > > /* Should move to system memory */
> > > > > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0,
> > > addr,
> > > > > bo_size, sync, 1, 0);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > > > drm_xe_engine_class_instance *eci,
> > > > >
> > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + exec_queues[e], ONE_SEC);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > > 0,
> > > > > addr, bo_size, NULL, 0);
> > > > >
> > > > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + bind_exec_queues[e], ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > * an invalidate.
> > > > > */
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + USER_FENCE_VALUE,
> > > exec_queues[e],
> > > > > + ONE_SEC);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) {
> > > > > /*
> > > > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > > > > + USER_FENCE_VALUE, exec_queues[i],
> > > ONE_SEC);
> > > > > }
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > bo_size,
> > > > > sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], ONE_SEC);
> > > > >
> > > > > if (!(flags & INVALID_FAULT)) {
> > > > > for (i = j; i < n_execs; i++)
> > > > > diff --git a/tests/intel/xe_exec_reset.c
> > > > > b/tests/intel/xe_exec_reset.c index 094b34896..398c90af5 100644
> > > > > --- a/tests/intel/xe_exec_reset.c
> > > > > +++ b/tests/intel/xe_exec_reset.c
> > > > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > > > >
> > > > > #define THREE_SEC MS_TO_NS(3000)
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > THREE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +THREE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > int err;
> > > > >
> > > > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, &timeout);
> > > > > + exec_queues[i], &timeout);
> > > > > if (flags & GT_RESET)
> > > > > /* exec races with reset: may timeout or complete */
> > > > > igt_assert(err == -ETIME || !err); @@ -631,7 +631,7
> > > @@
> > > > > test_compute_mode(int fd, struct drm_xe_engine_class_instance
> > > > > *eci,
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > THREE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +THREE_SEC);
> > > > >
> > > > > if (!(flags & GT_RESET)) {
> > > > > for (i = 1; i < n_execs; i++)
> > > > > diff --git a/tests/intel/xe_exec_threads.c
> > > > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9 100644
> > > > > --- a/tests/intel/xe_exec_threads.c
> > > > > +++ b/tests/intel/xe_exec_threads.c
> > > > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t addr, uint64_t userptr,
> > > > >
> > > > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC :
> > > > > THREE_SEC;
> > > > >
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > for (j = i - 0x20; j <= i; ++j)
> > > > > xe_wait_ufence(fd, &data[j].exec_sync,
> > > > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + exec_queues[e], fence_timeout);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > > > NULL, 0);
> > > > >
> > > > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + 0, fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > > > > xe_wait_ufence(fd,
> > > &data[j].exec_sync,
> > > > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + exec_queues[e],
> > > > > + fence_timeout);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) {
> > > > > /*
> > > > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > j = flags & INVALIDATE ?
> > > > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > + exec_queues[e], fence_timeout);
> > > > >
> > > > > /* Wait for all execs to complete */
> > > > > if (flags & INVALIDATE)
> > > > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t addr, uint64_t userptr,
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +fence_timeout);
> > > > >
> > > > > for (i = j; i < n_execs; i++)
> > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c index
> > > > > 3be987954..4e94403a3 100644
> > > > > --- a/tests/intel/xe_waitfence.c
> > > > > +++ b/tests/intel/xe_waitfence.c
> > > > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > > > uint32_t bo, uint64_t offset, }
> > > > >
> > > > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr,
> > > > > uint64_t
> > > value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout)
> > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > {
> > > > > struct drm_xe_wait_user_fence wait = {
> > > > > .addr = to_user_pointer(addr),
> > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > + .flags = !exec_queue ? 0 :
> > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > .value = value,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = timeout,
> > > > > - .num_engines = eci ? 1 : 0,
> > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > + .exec_queue_id = exec_queue,
> > > > > };
> > > > > struct timespec ts;
> > > > >
> > > > > - igt_assert(eci);
> > > > > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE,
> > > &wait), 0);
> > > > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > > > >
> > > > > @@ -82,7 +79,7 @@ enum waittype {
> > > > > static void
> > > > > waitfence(int fd, enum waittype wt) {
> > > > > - struct drm_xe_engine *engine = NULL;
> > > > > + uint32_t exec_queue;
> > > > > struct timespec ts;
> > > > > int64_t current, signalled;
> > > > > uint32_t bo_1;
> > > > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > > > >
> > > > > if (wt == RELTIME) {
> > > > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> > > MS_TO_NS(10));
> > > > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> > > MS_TO_NS(10));
> > > > > igt_debug("wait type: RELTIME - timeout: %ld, timeout left:
> > > %ld\n",
> > > > > MS_TO_NS(10), timeout);
> > > > > } else if (wt == ENGINE) {
> > > > > - engine = xe_engine(fd, 1);
> > > > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > > > +DRM_XE_ENGINE_CLASS_COPY);
> > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > timeout = current + MS_TO_NS(10);
> > > > > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > > &engine->instance, timeout);
> > > > > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > > exec_queue,
> > > > > +timeout);
> > > > > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > > > > ", signalled: %" PRId64
> > > > > ", elapsed: %" PRId64 "\n",
> > > > > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > timeout = current + MS_TO_NS(10);
> > > > > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7,
> > > NULL, timeout);
> > > > > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0,
> > > > > +timeout);
> > > > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > > > ", signalled: %" PRId64
> > > > > ", elapsed: %" PRId64 "\n",
> > > > > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > > > > .value = 1,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = 1,
> > > > > - .num_engines = 0,
> > > > > - .instances = 0,
> > > > > + .exec_queue_id = 0,
> > > > > };
> > > > >
> > > > > uint32_t vm = xe_vm_create(fd,
> > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8 +212,7
> > > @@ invalid_engine(int fd)
> > > > > .value = 1,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = -1,
> > > > > - .num_engines = 1,
> > > > > - .instances = 0,
> > > > > + .exec_queue_id = 0,
> > > > > };
> > > > >
> > > > > uint32_t vm = xe_vm_create(fd,
> > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > > > --
> > > > > 2.25.1
> > > > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-11 18:58 ` Rodrigo Vivi
@ 2023-12-12 4:23 ` Bommu, Krishnaiah
0 siblings, 0 replies; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-12 4:23 UTC (permalink / raw)
To: Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Tuesday, December 12, 2023 12:29 AM
> To: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: Brost, Matthew <matthew.brost@intel.com>; igt-
> dev@lists.freedesktop.org
> Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> drm_xe_wait_user_fence structure
>
> On Mon, Dec 11, 2023 at 07:03:47AM +0000, Bommu, Krishnaiah wrote:
> >
> >
> > > -----Original Message-----
> > > From: Bommu, Krishnaiah
> > > Sent: Monday, December 11, 2023 11:06 AM
> > > To: Brost, Matthew <matthew.brost@intel.com>; Vivi, Rodrigo
> > > <rodrigo.vivi@intel.com>
> > > Cc: igt-dev@lists.freedesktop.org
> > > Subject: RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > > drm_xe_wait_user_fence structure
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Brost, Matthew <matthew.brost@intel.com>
> > > > Sent: Friday, December 8, 2023 6:35 PM
> > > > To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > > Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> > > > dev@lists.freedesktop.org
> > > > Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member
> > > > to drm_xe_wait_user_fence structure
> > > >
> > > > On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > > > > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > > > > remove the num_engines/instances members from
> > > > drm_xe_wait_user_fence
> > > > > > structure and add a exec_queue_id member
> > > > > >
> > > > > > Right now this is only checking if the engine list is sane and
> > > > > > nothing else. In the end every operation with this IOCTL is a soft
> check.
> > > > > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > > > > >
> > > > > > exec_queue_id member will help to user space to get proper
> > > > > > error code from kernel while in exec_queue reset
> > > > > >
> > > > > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > > > > ---
> > > > > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > > > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > > > > lib/xe/xe_ioctl.h | 9 +++------
> > > > > > tests/intel/xe_evict.c | 4 ++--
> > > > > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > > > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > > > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > > > > tests/intel/xe_exec_reset.c | 6 +++---
> > > > > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > > > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > > > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > > > > >
> > > > > > diff --git a/include/drm-uapi/xe_drm.h
> > > > > > b/include/drm-uapi/xe_drm.h index 590f7b7af..fd06e4920 100644
> > > > > > --- a/include/drm-uapi/xe_drm.h
> > > > > > +++ b/include/drm-uapi/xe_drm.h
> > > > > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > > > > * It is returned as part of the @drm_xe_engine, but it also is used
> as
> > > > > > * the input of engine selection for both
> > > > > > @drm_xe_exec_queue_create
> > > > and
> > > > > > * @drm_xe_query_engine_cycles
> > > > > > - *
> > > > > > */
> > > > > > struct drm_xe_engine_class_instance {
> > > > > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > > > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > > > > */
> > > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > > > > + /** @engine_class: engine class id */
> > > > > > __u16 engine_class;
> > > > > > -
> > > > > > + /** @engine_instance: engine instance id */
> > > > > > __u16 engine_instance;
> > > > > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > > > > __u16 gt_id;
> > > > > > /** @pad: MBZ */
> > > > > > __u16 pad;
> > > > > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > > > > *
> > > > > > * Note: For userptr and externally imported dma-buf the
> > > > > > kernel
> > > > expects
> > > > > > * either 1WAY or 2WAY for the @pat_index.
> > > > > > + *
> > > > > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no
> KMD
> > > > restrictions
> > > > > > + * on the @pat_index. For such mappings there is no actual
> > > > > > +memory
> > > > being
> > > > > > + * mapped (the address in the PTE is invalid), so the
> > > > > > +various PAT
> > > > memory
> > > > > > + * attributes likely do not apply. Simply leaving as zero is one
> > > > > > + * option (still a valid pat_index).
> > > > > > */
> > > > > > __u16 pat_index;
> > > > > >
> > > > > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > > > > /** @op: wait operation (type of comparison) */
> > > > > > __u16 op;
> > > > > >
> > > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /*
> > > e.g. Wait
> > > > on VM bind */
> > > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > > > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > > > > /** @flags: wait flags */
> > > > > > __u16 flags;
> > > > > >
> > > > > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > > > > */
> > > > > > __s64 timeout;
> > > > > >
> > > > > > - /**
> > > > > > - * @num_engines: number of engine instances to wait on,
> must be
> > > > zero
> > > > > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > > - */
> > > > > > - __u64 num_engines;
> > > > > > + /** @exec_queue_id: exec_queue_id returned from
> > > > xe_exec_queue_create_ioctl */
> > > > > > + __u32 exec_queue_id;
> > > > > >
> > > > > > - /**
> > > > > > - * @instances: user pointer to array of
> > > > drm_xe_engine_class_instance to
> > > > > > - * wait on, must be NULL when
> > > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > > - */
> > > > > > - __u64 instances;
> > > > > > + /** @pad2: MBZ */
> > > > > > + __u32 pad2;
> > > > > >
> > > > > > /** @reserved: Reserved */
> > > > > > __u64 reserved[2];
> > > > > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > > > > b29ca40ad..0a1ddc86b 100644
> > > > > > --- a/lib/xe/xe_ioctl.c
> > > > > > +++ b/lib/xe/xe_ioctl.c
> > > > > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t
> > > > > > exec_queue,
> > > > uint64_t addr)
> > > > > > * @fd: xe device fd
> > > > > > * @addr: address of value to compare
> > > > > > * @value: expected value (equal) in @address
> > > > > > - * @eci: engine class instance
> > > > > > + * @exec_queue: exec_queue id
> > > > > > * @timeout: pointer to time to wait in nanoseconds
> > > > > > *
> > > > > > * Function compares @value with memory pointed by @addr
> > > > > > until they
> > > > are equal.
> > > > > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> > > > exec_queue, uint64_t addr)
> > > > > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > > > > */
> > > > > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t *timeout)
> > > > > > + uint32_t exec_queue, int64_t *timeout)
> > > > > > {
> > > > > > struct drm_xe_wait_user_fence wait = {
> > > > > > .addr = to_user_pointer(addr),
> > > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > > - .flags = !eci ?
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > > > > + .flags = 0,
> > > > > > .value = value,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > - .num_engines = eci ? 1 :0,
> > > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > > + .exec_queue_id = exec_queue,
> > > > > > };
> > > > > >
> > > > > > igt_assert(timeout);
> > > > > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * @fd: xe device fd
> > > > > > * @addr: address of value to compare
> > > > > > * @value: expected value (equal) in @address
> > > > > > - * @eci: engine class instance
> > > > > > + * @exec_queue: exec_queue id
> > > > > > * @timeout: time to wait in nanoseconds
> > > > > > *
> > > > > > * Function compares @value with memory pointed by @addr
> > > > > > until they
> > > > are equal.
> > > > > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > > */
> > > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t timeout)
> > > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > > {
> > > > > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci,
> &timeout), 0);
> > > > > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value,
> exec_queue,
> > > > > > +&timeout), 0);
> > > > > > return timeout;
> > > > > > }
> > > > > >
> > > > > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * @fd: xe device fd
> > > > > > * @addr: address of value to compare
> > > > > > * @value: expected value (equal) in @address
> > > > > > - * @eci: engine class instance
> > > > > > + * @exec_queue: exec_queue id
> > > > > > * @timeout: absolute time when wait expire
> > > > > > *
> > > > > > * Function compares @value with memory pointed by @addr
> > > > > > until they
> > > > are equal.
> > > > > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > > */
> > > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr,
> > > > > > uint64_t
> > > value,
> > > > > > - struct drm_xe_engine_class_instance
> *eci,
> > > > > > - int64_t timeout)
> > > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > > {
> > > > > > struct drm_xe_wait_user_fence wait = {
> > > > > > .addr = to_user_pointer(addr),
> > > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > > - .flags = !eci ?
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> > > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > > > + .flags = !exec_queue ?
> > > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > >
> > > > > To be on the safe side we should probably change the signature
> > > > > of this function to explicitly receive this flag instead of
> > > > > relying on the invalid
> > > > exec_queue_id number.
> > > > >
> > > > > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP
> > > was
> > > > used here...
> > > > >
> > > > > Matt, thoughts?
> > > > >
> > > >
> > > > Agree with everything here, !exec_queue really has nothing to do
> > > > with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> > > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So
> > > yes,
> > > > please change this function to accpt a flags field.
> > > >
> > > > Everything else LGTM.
> > > >
> > > > Matt
> > >
> > > I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not
> thing
> > > to do with exec_queue, I will change this function to accept flag,
> > > but DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is
> > > related to "no_engines" I removed from kernel side also
> > >
> https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
> > >
> > > Matt please let me know your insights on
> > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
> > >
> > > Regards,
> > > Krishna.
> > > >
> > > > > Everything else in the patch looks good to me.
> > > > >
> > > > > > .value = value,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = timeout,
> > > > > > - .num_engines = eci ? 1 : 0,
> > > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > > + .exec_queue_id = exec_queue,
> > > > > > };
> > > > > > struct timespec ts;
> > > > > >
> > > > > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > > > > bd660bd27..9c0b4b9bc 100644
> > > > > > --- a/lib/xe/xe_ioctl.h
> > > > > > +++ b/lib/xe/xe_ioctl.h
> > > > > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t
> > > > > > exec_queue,
> > > > uint64_t addr,
> > > > > > struct drm_xe_sync *sync, uint32_t num_syncs);
> void
> > > > > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > > > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t *timeout);
> > > > > > + uint32_t exec_queue, int64_t *timeout);
> > > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t timeout);
> > > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr,
> > > > > > uint64_t
> > > value,
> > > > > > - struct drm_xe_engine_class_instance
> *eci,
> > > > > > - int64_t timeout);
> > > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > > void xe_force_gt_reset(int fd, int gt);
> > > > > >
> > > > > > #endif /* XE_IOCTL_H */
> > > > > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
> > > > > > index 89dc46fae..a37c2d97a 100644
> > > > > > --- a/tests/intel/xe_evict.c
> > > > > > +++ b/tests/intel/xe_evict.c
> > > > > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > }
> > > > > > #define TWENTY_SEC MS_TO_NS(20000)
> > > > > > xe_wait_ufence(fd, &data[i].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, TWENTY_SEC);
> > > > > > + bind_exec_queues[0],
> TWENTY_SEC);
> > > > > > }
> > > > > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > > > > (char *)data;
> > > > > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > data = xe_bo_map(fd, __bo,
> > > > > > ALIGN(sizeof(*data) * n_execs,
> 0x1000));
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, TWENTY_SEC);
> > > > > > + exec_queues[i], TWENTY_SEC);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > }
> > > > > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff
> > > > > > --git a/tests/intel/xe_exec_balancer.c
> > > > > > b/tests/intel/xe_exec_balancer.c index 79ff65e89..2448d49bc
> > > > > > 100644
> > > > > > --- a/tests/intel/xe_exec_balancer.c
> > > > > > +++ b/tests/intel/xe_exec_balancer.c
> > > > > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > > > > bo_size, sync, 1);
> > > > > >
> > > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -514,7 +514,7 @@
> > > > > > test_cm(int fd, int gt, int class, int n_exec_queues, int
> > > > > > n_execs,
> > > > > >
> > > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + exec_queues[e], ONE_SEC);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr,
> bo_size,
> > > > NULL,
> > > > > > 0);
> > > > > >
> > > > > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + 0, ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > > > > * an invalidate.
> > > > > > */
> > > > > > xe_wait_ufence(fd,
> &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE,
> NULL,
> > > > ONE_SEC);
> > > > > > + USER_FENCE_VALUE,
> > > > exec_queues[e],
> > > > > > + ONE_SEC);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) {
> > > > > > /*
> > > > > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > > > > n_exec_queues, int n_execs,
> > > > > >
> > > > > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + exec_queues[i], ONE_SEC);
> >
> > Need to change exec_queues[i] index to "i % n_exec_queues", need to
> > change other files also, I will change exec_queues[i] => exec_queues[i
> > % n_exec_queues]
>
> why?
We need to use same exec_queue, which is user for xe_exec, with current change test is failing with exec_queue error
From test
int e = i % n_exec_queues;
------------
exec.exec_queue_id = exec_queues[e];
exec.address = exec_addr;
xe_exec(fd, &exec);
with index change tests are passing
Regards,
Krishna.
>
> >
> > Regards,
> > Krishna
> >
> > > > > >
> > > > > > /* Wait for all execs to complete */
> > > > > > if (flags & INVALIDATE)
> > > > > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > > > > n_exec_queues, int n_execs,
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +ONE_SEC);
> > > > > >
> > > > > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > > > > i < n_execs; i++)
> > > > > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > > > > b/tests/intel/xe_exec_compute_mode.c
> > > > > > index 7d3004d65..749a9b634 100644
> > > > > > --- a/tests/intel/xe_exec_compute_mode.c
> > > > > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > > > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > > > > drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> > > > ONE_SEC;
> > > > > >
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -198,7 +198,7 @@
> > > > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + exec_queues[e], fence_timeout);
> > > > > > xe_vm_unbind_async(fd, vm,
> bind_exec_queues[e],
> > > > 0,
> > > > > > addr, bo_size, NULL, 0);
> > > > > >
> > > > > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + bind_exec_queues[e],
> fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > * an invalidate.
> > > > > > */
> > > > > > xe_wait_ufence(fd,
> &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE,
> NULL,
> > > > > > + USER_FENCE_VALUE,
> > > > exec_queues[e],
> > > > > > fence_timeout);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) { @@ -257,8 +257,8
> @@
> > > > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + exec_queues[i], fence_timeout);
> > > > > >
> > > > > > /* Wait for all execs to complete */
> > > > > > if (flags & INVALIDATE)
> > > > > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > > bo_size,
> > > > > > sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], fence_timeout);
> > > > > >
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > > a/tests/intel/xe_exec_fault_mode.c
> > > > > > b/tests/intel/xe_exec_fault_mode.c
> > > > > > index ee7cbb604..d76b3a056 100644
> > > > > > --- a/tests/intel/xe_exec_fault_mode.c
> > > > > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > > > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > }
> > > > > >
> > > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > if (flags & PREFETCH) {
> > > > > > /* Should move to system memory */
> > > > > > xe_vm_prefetch_async(fd, vm,
> bind_exec_queues[0], 0,
> > > > addr,
> > > > > > bo_size, sync, 1, 0);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > > > > drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + exec_queues[e], ONE_SEC);
> > > > > > xe_vm_unbind_async(fd, vm,
> bind_exec_queues[e],
> > > > 0,
> > > > > > addr, bo_size, NULL, 0);
> > > > > >
> > > > > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + bind_exec_queues[e], ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > * an invalidate.
> > > > > > */
> > > > > > xe_wait_ufence(fd,
> &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE,
> NULL,
> > > > ONE_SEC);
> > > > > > + USER_FENCE_VALUE,
> > > > exec_queues[e],
> > > > > > + ONE_SEC);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) {
> > > > > > /*
> > > > > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > > > > + USER_FENCE_VALUE,
> exec_queues[i],
> > > > ONE_SEC);
> > > > > > }
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > > bo_size,
> > > > > > sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > >
> > > > > > if (!(flags & INVALID_FAULT)) {
> > > > > > for (i = j; i < n_execs; i++) diff --git
> > > > > > a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
> > > > > > index 094b34896..398c90af5 100644
> > > > > > --- a/tests/intel/xe_exec_reset.c
> > > > > > +++ b/tests/intel/xe_exec_reset.c
> > > > > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > > > > >
> > > > > > #define THREE_SEC MS_TO_NS(3000)
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > THREE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +THREE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -621,7 +621,7 @@
> > > > > > test_compute_mode(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > int err;
> > > > > >
> > > > > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, &timeout);
> > > > > > + exec_queues[i], &timeout);
> > > > > > if (flags & GT_RESET)
> > > > > > /* exec races with reset: may timeout or
> complete */
> > > > > > igt_assert(err == -ETIME || !err); @@ -631,7
> +631,7
> > > > @@
> > > > > > test_compute_mode(int fd, struct drm_xe_engine_class_instance
> > > > > > *eci,
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > THREE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +THREE_SEC);
> > > > > >
> > > > > > if (!(flags & GT_RESET)) {
> > > > > > for (i = 1; i < n_execs; i++) diff --git
> > > > > > a/tests/intel/xe_exec_threads.c
> > > > > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9
> > > > > > 100644
> > > > > > --- a/tests/intel/xe_exec_threads.c
> > > > > > +++ b/tests/intel/xe_exec_threads.c
> > > > > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t addr, uint64_t userptr,
> > > > > >
> > > > > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC :
> > > > > > THREE_SEC;
> > > > > >
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -359,7 +359,7 @@
> > > > > > test_compute_mode(int fd, uint32_t vm, uint64_t
> > > > addr, uint64_t userptr,
> > > > > > for (j = i - 0x20; j <= i; ++j)
> > > > > > xe_wait_ufence(fd,
> &data[j].exec_sync,
> > > > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + exec_queues[e],
> fence_timeout);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr,
> bo_size,
> > > > > > NULL, 0);
> > > > > >
> > > > > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t
> > > > addr, uint64_t userptr,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + 0, fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t
> > > > addr, uint64_t userptr,
> > > > > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i;
> ++j)
> > > > > > xe_wait_ufence(fd,
> > > > &data[j].exec_sync,
> > > > > >
> USER_FENCE_VALUE,
> > > > > > - NULL,
> fence_timeout);
> > > > > > + exec_queues[e],
> > > > > > + fence_timeout);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) {
> > > > > > /*
> > > > > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t
> > > > addr, uint64_t userptr,
> > > > > > j = flags & INVALIDATE ?
> > > > > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + exec_queues[e], fence_timeout);
> > > > > >
> > > > > > /* Wait for all execs to complete */
> > > > > > if (flags & INVALIDATE)
> > > > > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t addr, uint64_t userptr,
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +fence_timeout);
> > > > > >
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > > > > > index
> > > > > > 3be987954..4e94403a3 100644
> > > > > > --- a/tests/intel/xe_waitfence.c
> > > > > > +++ b/tests/intel/xe_waitfence.c
> > > > > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > > > > uint32_t bo, uint64_t offset, }
> > > > > >
> > > > > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr,
> > > > > > uint64_t
> > > > value,
> > > > > > - struct
> drm_xe_engine_class_instance *eci,
> > > > > > - int64_t timeout)
> > > > > > + uint32_t exec_queue, int64_t
> timeout)
> > > > > > {
> > > > > > struct drm_xe_wait_user_fence wait = {
> > > > > > .addr = to_user_pointer(addr),
> > > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > > - .flags = !eci ? 0 :
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > > + .flags = !exec_queue ? 0 :
> > > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > > .value = value,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = timeout,
> > > > > > - .num_engines = eci ? 1 : 0,
> > > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > > + .exec_queue_id = exec_queue,
> > > > > > };
> > > > > > struct timespec ts;
> > > > > >
> > > > > > - igt_assert(eci);
> > > > > > igt_assert_eq(igt_ioctl(fd,
> DRM_IOCTL_XE_WAIT_USER_FENCE,
> > > > &wait), 0);
> > > > > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > > > > >
> > > > > > @@ -82,7 +79,7 @@ enum waittype { static void waitfence(int
> > > > > > fd, enum waittype wt) {
> > > > > > - struct drm_xe_engine *engine = NULL;
> > > > > > + uint32_t exec_queue;
> > > > > > struct timespec ts;
> > > > > > int64_t current, signalled;
> > > > > > uint32_t bo_1;
> > > > > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > > > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > > > > >
> > > > > > if (wt == RELTIME) {
> > > > > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> > > > MS_TO_NS(10));
> > > > > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> > > > MS_TO_NS(10));
> > > > > > igt_debug("wait type: RELTIME - timeout: %ld,
> timeout left:
> > > > %ld\n",
> > > > > > MS_TO_NS(10), timeout);
> > > > > > } else if (wt == ENGINE) {
> > > > > > - engine = xe_engine(fd, 1);
> > > > > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > > > > +DRM_XE_ENGINE_CLASS_COPY);
> > > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > > timeout = current + MS_TO_NS(10);
> > > > > > - signalled = wait_with_eci_abstime(fd, &wait_fence,
> 7,
> > > > &engine->instance, timeout);
> > > > > > + signalled = wait_with_eci_abstime(fd, &wait_fence,
> 7,
> > > > exec_queue,
> > > > > > +timeout);
> > > > > > igt_debug("wait type: ENGINE ABSTIME - timeout:
> %" PRId64
> > > > > > ", signalled: %" PRId64
> > > > > > ", elapsed: %" PRId64 "\n", @@ -128,7
> +125,7 @@
> > > > > > waitfence(int fd, enum waittype wt)
> > > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > > timeout = current + MS_TO_NS(10);
> > > > > > - signalled = xe_wait_ufence_abstime(fd,
> &wait_fence, 7,
> > > > NULL, timeout);
> > > > > > + signalled = xe_wait_ufence_abstime(fd,
> &wait_fence, 7, 0,
> > > > > > +timeout);
> > > > > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > > > > ", signalled: %" PRId64
> > > > > > ", elapsed: %" PRId64 "\n", @@ -191,8
> +188,7 @@
> > > > > > invalid_ops(int fd)
> > > > > > .value = 1,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = 1,
> > > > > > - .num_engines = 0,
> > > > > > - .instances = 0,
> > > > > > + .exec_queue_id = 0,
> > > > > > };
> > > > > >
> > > > > > uint32_t vm = xe_vm_create(fd,
> > > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8
> +212,7
> > > > @@ invalid_engine(int fd)
> > > > > > .value = 1,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = -1,
> > > > > > - .num_engines = 1,
> > > > > > - .instances = 0,
> > > > > > + .exec_queue_id = 0,
> > > > > > };
> > > > > >
> > > > > > uint32_t vm = xe_vm_create(fd,
> > > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > > > > --
> > > > > > 2.25.1
> > > > > >
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2023-12-12 4:23 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
2023-12-08 5:57 ` Rodrigo Vivi
2023-12-08 13:05 ` Matthew Brost
2023-12-11 5:36 ` Bommu, Krishnaiah
2023-12-11 7:03 ` Bommu, Krishnaiah
2023-12-11 18:58 ` Rodrigo Vivi
2023-12-12 4:23 ` Bommu, Krishnaiah
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
2023-12-08 6:00 ` Rodrigo Vivi
2023-12-08 12:59 ` Bommu, Krishnaiah
2023-12-08 13:48 ` Rodrigo Vivi
2023-12-08 5:48 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5) Patchwork
2023-12-08 13:16 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6) Patchwork
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.