* [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
@ 2023-12-08 4:18 Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Bommu Krishnaiah @ 2023-12-08 4:18 UTC (permalink / raw)
To: igt-dev; +Cc: Bommu Krishnaiah
remove the num_engines/instances members from drm_xe_wait_user_fence structure
and add a exec_queue_id member
invalid-exec_queue-wait subtest to excess behaviour when exec_queue reset happen
about test
Skipping the GPU mapping(vm_bind) for object, so that exec_queue
reset will happen and xe_wait_ufence will end return EIO not ETIME
v3: fixed build failures
NOTE: Corresponding driver changes is,
kmd commit: drm/xe/uapi: Return correct error code for xe_wait_user_fence_ioctl
I am able to see exec_queue reset was happened and xe_wait_user_fence_ioctl returned EIO
test result
root@DUT7075PVC:/home/gta# LD_LIBRARY_PATH=/home/gta/ ./xe_waitfence --r invalid-exec_queue-wait
IGT-Version: 1.28-g3c0162fc4 (x86_64) (Linux: 6.6.0-rc3-xe x86_64)
Opened device: /dev/dri/card0
Starting subtest: invalid-exec_queue-wait
Subtest invalid-exec_queue-wait: SUCCESS (0.993s)
dmesg logs
[ 807.680378] [IGT] xe_waitfence: executing
[ 807.699796] [drm:drm_stub_open [drm]]
[ 807.704536] xe 0000:51:00.0: [drm:drm_open_helper [drm]] comm="xe_waitfence", pid=2952, minor=0
[ 807.715155] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 807.727328] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 807.739580] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.751518] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.763550] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.775525] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.787556] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.799494] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.811531] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.823476] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_DEVICE_QUERY
[ 807.835577] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 807.847921] [IGT] xe_waitfence: starting subtest invalid-exec_queue-wait
[ 807.855528] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_VM_CREATE
[ 807.891346] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: Applying GT save-restore MMIOs
[ 807.901602] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: REG[0x9424] = 0x7ffffffc
-----------------
-----------------
[ 808.560967] xe REG[0x4500-0x45ff]: deny rw access
[ 808.566292] xe REG[0x1e3a8-0x1e3af]: allow read access
[ 808.572161] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: Applying ccs3 save-restore MMIOs
[ 808.582462] xe 0000:51:00.0: [drm:xe_reg_sr_apply_mmio [xe]] GT0: REG[0x260c4] = 0x3f7e0104
[ 808.592096] xe 0000:51:00.0: [drm:xe_reg_sr_apply_whitelist [xe]] Whitelisting ccs3 registers
[ 808.601962] xe REG[0x4400-0x45ff]: deny rw access
[ 808.607281] xe REG[0x4500-0x45ff]: deny rw access
[ 808.612608] xe REG[0x263a8-0x263af]: allow read access
[ 808.618477] xe 0000:51:00.0: [drm] GT0: resumed
[ 808.626283] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_EXEC_QUEUE_CREATE
[ 808.638765] krishna xe_exec_queue_create_ioctl
[ 808.645592] krishna args->exec_queue_id = 1
[ 808.650328] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_GEM_CREATE
[ 808.662621] xe 0000:51:00.0: [drm:xe_migrate_clear [xe]] Pass 0, size: 262144
[ 808.672889] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_GEM_MMAP_OFFSET
[ 808.685733] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_EXEC
[ 808.696900] krishna args->exec_queue_id = 1
[ 808.702700] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_WAIT_USER_FENCE
[ 808.704620] xe 0000:51:00.0: [drm:pf_queue_work_func [xe]]
ASID: 1048575
VFID: 0
PDATA: 0x00a3
Faulted Address: 0x00000000001a0000
FaultType: 0
AccessType: 0
FaultLevel: 4
EngineClass: 3
EngineInstance: 0
[ 808.750685] xe 0000:51:00.0: [drm:pf_queue_work_func [xe]] Fault response: Unsuccessful -22
[ 808.760519] xe 0000:51:00.0: [drm:xe_guc_exec_queue_memory_cat_error_handler [xe]] Engine memory cat error: guc_id=2
[ 808.773237] xe 0000:51:00.0: [drm] exec gueue reset detected
[ 808.773965] xe 0000:51:00.0: [drm] Timedout job: seqno=4294967169, guc_id=2, flags=0x8
[ 808.779632] xe 0000:51:00.0: [drm:xe_wait_user_fence_ioctl [xe]] Ioctl argument check failed at drivers/gpu/drm/xe/xe_wait_user_fence.c:174: err < 0
[ 808.789655] xe 0000:51:00.0: [drm] Xe device coredump has been created
[ 808.803796] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence", pid=2952, ret=-5
[ 808.811133] xe 0000:51:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data
[ 808.811220] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, XE_EXEC_QUEUE_DESTROY
[ 808.823605] xe 0000:51:00.0: [drm] Engine reset: guc_id=2
[ 808.829862] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_GEM_CLOSE
[ 808.843312] xe 0000:51:00.0: [drm:guc_exec_queue_timedout_job [xe]] Timedout signaled job: seqno=4294967169, guc_id=2, flags=0x9
[ 808.848707] [IGT] xe_waitfence: finished subtest invalid-exec_queue-wait, SUCCESS
[ 808.882255] xe 0000:51:00.0: [drm:drm_ioctl [drm]] comm="xe_waitfence" pid=2952, dev=0xe200, auth=1, DRM_IOCTL_VERSION
[ 808.894404] xe 0000:51:00.0: [drm:drm_file_free.part.0 [drm]] comm="xe_waitfence", pid=2952, dev=0xe200, open_count=1
[ 808.907374] xe 0000:51:00.0: [drm:drm_lastclose [drm]]
[ 808.913594] xe 0000:51:00.0: [drm:drm_lastclose [drm]] driver lastclose completed
Bommu Krishnaiah (2):
drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence
structure
drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset
happen
include/drm-uapi/xe_drm.h | 28 ++++----
lib/xe/xe_ioctl.c | 27 +++-----
lib/xe/xe_ioctl.h | 9 +--
tests/intel/xe_evict.c | 4 +-
tests/intel/xe_exec_balancer.c | 15 ++--
tests/intel/xe_exec_compute_mode.c | 18 ++---
tests/intel/xe_exec_fault_mode.c | 19 ++---
tests/intel/xe_exec_reset.c | 6 +-
tests/intel/xe_exec_threads.c | 15 ++--
tests/intel/xe_waitfence.c | 107 +++++++++++++++++++++++++----
10 files changed, 161 insertions(+), 87 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
@ 2023-12-08 4:18 ` Bommu Krishnaiah
2023-12-08 5:57 ` Rodrigo Vivi
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
` (2 subsequent siblings)
3 siblings, 1 reply; 14+ messages in thread
From: Bommu Krishnaiah @ 2023-12-08 4:18 UTC (permalink / raw)
To: igt-dev; +Cc: Bommu Krishnaiah, Rodrigo Vivi
remove the num_engines/instances members from drm_xe_wait_user_fence structure
and add a exec_queue_id member
Right now this is only checking if the engine list is sane and nothing
else. In the end every operation with this IOCTL is a soft check.
So, let's formalize that and only use this IOCTL to wait on the fence.
exec_queue_id member will help to user space to get proper error code
from kernel while in exec_queue reset
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
---
include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
lib/xe/xe_ioctl.c | 27 +++++++++++----------------
lib/xe/xe_ioctl.h | 9 +++------
tests/intel/xe_evict.c | 4 ++--
tests/intel/xe_exec_balancer.c | 15 ++++++++-------
tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
tests/intel/xe_exec_reset.c | 6 +++---
tests/intel/xe_exec_threads.c | 15 ++++++++-------
tests/intel/xe_waitfence.c | 25 ++++++++++---------------
10 files changed, 79 insertions(+), 87 deletions(-)
diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
index 590f7b7af..fd06e4920 100644
--- a/include/drm-uapi/xe_drm.h
+++ b/include/drm-uapi/xe_drm.h
@@ -129,7 +129,6 @@ struct xe_user_extension {
* It is returned as part of the @drm_xe_engine, but it also is used as
* the input of engine selection for both @drm_xe_exec_queue_create and
* @drm_xe_query_engine_cycles
- *
*/
struct drm_xe_engine_class_instance {
#define DRM_XE_ENGINE_CLASS_RENDER 0
@@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
*/
#define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
#define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
+ /** @engine_class: engine class id */
__u16 engine_class;
-
+ /** @engine_instance: engine instance id */
__u16 engine_instance;
+ /** @gt_id: Unique ID of this GT within the PCI Device */
__u16 gt_id;
/** @pad: MBZ */
__u16 pad;
@@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
*
* Note: For userptr and externally imported dma-buf the kernel expects
* either 1WAY or 2WAY for the @pat_index.
+ *
+ * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD restrictions
+ * on the @pat_index. For such mappings there is no actual memory being
+ * mapped (the address in the PTE is invalid), so the various PAT memory
+ * attributes likely do not apply. Simply leaving as zero is one
+ * option (still a valid pat_index).
*/
__u16 pat_index;
@@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
/** @op: wait operation (type of comparison) */
__u16 op;
-#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait on VM bind */
-#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
+#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
/** @flags: wait flags */
__u16 flags;
@@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
*/
__s64 timeout;
- /**
- * @num_engines: number of engine instances to wait on, must be zero
- * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
- */
- __u64 num_engines;
+ /** @exec_queue_id: exec_queue_id returned from xe_exec_queue_create_ioctl */
+ __u32 exec_queue_id;
- /**
- * @instances: user pointer to array of drm_xe_engine_class_instance to
- * wait on, must be NULL when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
- */
- __u64 instances;
+ /** @pad2: MBZ */
+ __u32 pad2;
/** @reserved: Reserved */
__u64 reserved[2];
diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c
index b29ca40ad..0a1ddc86b 100644
--- a/lib/xe/xe_ioctl.c
+++ b/lib/xe/xe_ioctl.c
@@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
* @fd: xe device fd
* @addr: address of value to compare
* @value: expected value (equal) in @address
- * @eci: engine class instance
+ * @exec_queue: exec_queue id
* @timeout: pointer to time to wait in nanoseconds
*
* Function compares @value with memory pointed by @addr until they are equal.
@@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
* signalled. Returns 0 on success, -errno of ioctl on error.
*/
int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t *timeout)
+ uint32_t exec_queue, int64_t *timeout)
{
struct drm_xe_wait_user_fence wait = {
.addr = to_user_pointer(addr),
.op = DRM_XE_UFENCE_WAIT_OP_EQ,
- .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
+ .flags = 0,
.value = value,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
- .num_engines = eci ? 1 :0,
- .instances = eci ? to_user_pointer(eci) : 0,
+ .exec_queue_id = exec_queue,
};
igt_assert(timeout);
@@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* @fd: xe device fd
* @addr: address of value to compare
* @value: expected value (equal) in @address
- * @eci: engine class instance
+ * @exec_queue: exec_queue id
* @timeout: time to wait in nanoseconds
*
* Function compares @value with memory pointed by @addr until they are equal.
@@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* Returns elapsed time in nanoseconds if user fence was signalled.
*/
int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout)
+ uint32_t exec_queue, int64_t timeout)
{
- igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
+ igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue, &timeout), 0);
return timeout;
}
@@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* @fd: xe device fd
* @addr: address of value to compare
* @value: expected value (equal) in @address
- * @eci: engine class instance
+ * @exec_queue: exec_queue id
* @timeout: absolute time when wait expire
*
* Function compares @value with memory pointed by @addr until they are equal.
@@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
* Returns elapsed time in nanoseconds if user fence was signalled.
*/
int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout)
+ uint32_t exec_queue, int64_t timeout)
{
struct drm_xe_wait_user_fence wait = {
.addr = to_user_pointer(addr),
.op = DRM_XE_UFENCE_WAIT_OP_EQ,
- .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP | DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
+ .flags = !exec_queue ? DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
.value = value,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = timeout,
- .num_engines = eci ? 1 : 0,
- .instances = eci ? to_user_pointer(eci) : 0,
+ .exec_queue_id = exec_queue,
};
struct timespec ts;
diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h
index bd660bd27..9c0b4b9bc 100644
--- a/lib/xe/xe_ioctl.h
+++ b/lib/xe/xe_ioctl.h
@@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue, uint64_t addr,
struct drm_xe_sync *sync, uint32_t num_syncs);
void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr);
int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t *timeout);
+ uint32_t exec_queue, int64_t *timeout);
int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout);
+ uint32_t exec_queue, int64_t timeout);
int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout);
+ uint32_t exec_queue, int64_t timeout);
void xe_force_gt_reset(int fd, int gt);
#endif /* XE_IOCTL_H */
diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
index 89dc46fae..a37c2d97a 100644
--- a/tests/intel/xe_evict.c
+++ b/tests/intel/xe_evict.c
@@ -317,7 +317,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
}
#define TWENTY_SEC MS_TO_NS(20000)
xe_wait_ufence(fd, &data[i].vm_sync, USER_FENCE_VALUE,
- NULL, TWENTY_SEC);
+ bind_exec_queues[0], TWENTY_SEC);
}
sync[0].addr = addr + (char *)&data[i].exec_sync -
(char *)data;
@@ -352,7 +352,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
data = xe_bo_map(fd, __bo,
ALIGN(sizeof(*data) * n_execs, 0x1000));
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, TWENTY_SEC);
+ exec_queues[i], TWENTY_SEC);
igt_assert_eq(data[i].data, 0xc0ffee);
}
munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000));
diff --git a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
index 79ff65e89..2448d49bc 100644
--- a/tests/intel/xe_exec_balancer.c
+++ b/tests/intel/xe_exec_balancer.c
@@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
bo_size, sync, 1);
#define ONE_SEC MS_TO_NS(1000)
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
if (flags & REBIND && i + 1 != n_execs) {
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ exec_queues[e], ONE_SEC);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, NULL,
0);
@@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ 0, ONE_SEC);
data[0].vm_sync = 0;
}
@@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
* an invalidate.
*/
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL, ONE_SEC);
+ USER_FENCE_VALUE, exec_queues[e],
+ ONE_SEC);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
/*
@@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
for (i = j; i < n_execs; i++)
- xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
- ONE_SEC);
+ xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
+ exec_queues[i], ONE_SEC);
/* Wait for all execs to complete */
if (flags & INVALIDATE)
@@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
i < n_execs; i++)
diff --git a/tests/intel/xe_exec_compute_mode.c b/tests/intel/xe_exec_compute_mode.c
index 7d3004d65..749a9b634 100644
--- a/tests/intel/xe_exec_compute_mode.c
+++ b/tests/intel/xe_exec_compute_mode.c
@@ -171,8 +171,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC : ONE_SEC;
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], fence_timeout);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -198,7 +198,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
if (flags & REBIND && i + 1 != n_execs) {
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, fence_timeout);
+ exec_queues[e], fence_timeout);
xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
addr, bo_size, NULL, 0);
@@ -214,7 +214,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, fence_timeout);
+ bind_exec_queues[e], fence_timeout);
data[0].vm_sync = 0;
}
@@ -227,7 +227,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
* an invalidate.
*/
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL,
+ USER_FENCE_VALUE, exec_queues[e],
fence_timeout);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
@@ -257,8 +257,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
j = flags & INVALIDATE ? n_execs - 1 : 0;
for (i = j; i < n_execs; i++)
- xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
+ exec_queues[i], fence_timeout);
/* Wait for all execs to complete */
if (flags & INVALIDATE)
@@ -267,8 +267,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], fence_timeout);
for (i = j; i < n_execs; i++)
igt_assert_eq(data[i].data, 0xc0ffee);
diff --git a/tests/intel/xe_exec_fault_mode.c b/tests/intel/xe_exec_fault_mode.c
index ee7cbb604..d76b3a056 100644
--- a/tests/intel/xe_exec_fault_mode.c
+++ b/tests/intel/xe_exec_fault_mode.c
@@ -195,15 +195,16 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
}
#define ONE_SEC MS_TO_NS(1000)
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], ONE_SEC);
data[0].vm_sync = 0;
if (flags & PREFETCH) {
/* Should move to system memory */
xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0, addr,
bo_size, sync, 1, 0);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
- ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], ONE_SEC);
data[0].vm_sync = 0;
}
@@ -230,7 +231,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
if (flags & REBIND && i + 1 != n_execs) {
xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ exec_queues[e], ONE_SEC);
xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
addr, bo_size, NULL, 0);
@@ -246,7 +247,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, ONE_SEC);
+ bind_exec_queues[e], ONE_SEC);
data[0].vm_sync = 0;
}
@@ -259,7 +260,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
* an invalidate.
*/
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL, ONE_SEC);
+ USER_FENCE_VALUE, exec_queues[e],
+ ONE_SEC);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
/*
@@ -290,13 +292,14 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
j = flags & INVALIDATE ? n_execs - 1 : 0;
for (i = j; i < n_execs; i++)
xe_wait_ufence(fd, &data[i].exec_sync,
- USER_FENCE_VALUE, NULL, ONE_SEC);
+ USER_FENCE_VALUE, exec_queues[i], ONE_SEC);
}
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
+ bind_exec_queues[0], ONE_SEC);
if (!(flags & INVALID_FAULT)) {
for (i = j; i < n_execs; i++)
diff --git a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
index 094b34896..398c90af5 100644
--- a/tests/intel/xe_exec_reset.c
+++ b/tests/intel/xe_exec_reset.c
@@ -564,7 +564,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
#define THREE_SEC MS_TO_NS(3000)
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -621,7 +621,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
int err;
err = __xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
- NULL, &timeout);
+ exec_queues[i], &timeout);
if (flags & GT_RESET)
/* exec races with reset: may timeout or complete */
igt_assert(err == -ETIME || !err);
@@ -631,7 +631,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
if (!(flags & GT_RESET)) {
for (i = 1; i < n_execs; i++)
diff --git a/tests/intel/xe_exec_threads.c b/tests/intel/xe_exec_threads.c
index fcb926698..7985240c9 100644
--- a/tests/intel/xe_exec_threads.c
+++ b/tests/intel/xe_exec_threads.c
@@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
data[0].vm_sync = 0;
for (i = 0; i < n_execs; i++) {
@@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
for (j = i - 0x20; j <= i; ++j)
xe_wait_ufence(fd, &data[j].exec_sync,
USER_FENCE_VALUE,
- NULL, fence_timeout);
+ exec_queues[e], fence_timeout);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
NULL, 0);
@@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
addr, bo_size, sync,
1);
xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
- NULL, fence_timeout);
+ 0, fence_timeout);
data[0].vm_sync = 0;
}
@@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
xe_wait_ufence(fd, &data[j].exec_sync,
USER_FENCE_VALUE,
- NULL, fence_timeout);
+ exec_queues[e],
+ fence_timeout);
igt_assert_eq(data[i].data, 0xc0ffee);
} else if (i * 2 != n_execs) {
/*
@@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
j = flags & INVALIDATE ?
(flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
for (i = j; i < n_execs; i++)
- xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
- fence_timeout);
+ xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
+ exec_queues[e], fence_timeout);
/* Wait for all execs to complete */
if (flags & INVALIDATE)
@@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
sync[0].addr = to_user_pointer(&data[0].vm_sync);
xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
- xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
+ xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
for (i = j; i < n_execs; i++)
igt_assert_eq(data[i].data, 0xc0ffee);
diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
index 3be987954..4e94403a3 100644
--- a/tests/intel/xe_waitfence.c
+++ b/tests/intel/xe_waitfence.c
@@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm, uint32_t bo, uint64_t offset,
}
static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t value,
- struct drm_xe_engine_class_instance *eci,
- int64_t timeout)
+ uint32_t exec_queue, int64_t timeout)
{
struct drm_xe_wait_user_fence wait = {
.addr = to_user_pointer(addr),
.op = DRM_XE_UFENCE_WAIT_OP_EQ,
- .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
+ .flags = !exec_queue ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
.value = value,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = timeout,
- .num_engines = eci ? 1 : 0,
- .instances = eci ? to_user_pointer(eci) : 0,
+ .exec_queue_id = exec_queue,
};
struct timespec ts;
- igt_assert(eci);
igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait), 0);
igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
@@ -82,7 +79,7 @@ enum waittype {
static void
waitfence(int fd, enum waittype wt)
{
- struct drm_xe_engine *engine = NULL;
+ uint32_t exec_queue;
struct timespec ts;
int64_t current, signalled;
uint32_t bo_1;
@@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
if (wt == RELTIME) {
- timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL, MS_TO_NS(10));
+ timeout = xe_wait_ufence(fd, &wait_fence, 7, 0, MS_TO_NS(10));
igt_debug("wait type: RELTIME - timeout: %ld, timeout left: %ld\n",
MS_TO_NS(10), timeout);
} else if (wt == ENGINE) {
- engine = xe_engine(fd, 1);
+ exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
clock_gettime(CLOCK_MONOTONIC, &ts);
current = ts.tv_sec * 1e9 + ts.tv_nsec;
timeout = current + MS_TO_NS(10);
- signalled = wait_with_eci_abstime(fd, &wait_fence, 7, &engine->instance, timeout);
+ signalled = wait_with_eci_abstime(fd, &wait_fence, 7, exec_queue, timeout);
igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
", signalled: %" PRId64
", elapsed: %" PRId64 "\n",
@@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
clock_gettime(CLOCK_MONOTONIC, &ts);
current = ts.tv_sec * 1e9 + ts.tv_nsec;
timeout = current + MS_TO_NS(10);
- signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, NULL, timeout);
+ signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0, timeout);
igt_debug("wait type: ABSTIME - timeout: %" PRId64
", signalled: %" PRId64
", elapsed: %" PRId64 "\n",
@@ -191,8 +188,7 @@ invalid_ops(int fd)
.value = 1,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = 1,
- .num_engines = 0,
- .instances = 0,
+ .exec_queue_id = 0,
};
uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
@@ -216,8 +212,7 @@ invalid_engine(int fd)
.value = 1,
.mask = DRM_XE_UFENCE_WAIT_MASK_U64,
.timeout = -1,
- .num_engines = 1,
- .instances = 0,
+ .exec_queue_id = 0,
};
uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
@ 2023-12-08 4:18 ` Bommu Krishnaiah
2023-12-08 6:00 ` Rodrigo Vivi
2023-12-08 5:48 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5) Patchwork
2023-12-08 13:16 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6) Patchwork
3 siblings, 1 reply; 14+ messages in thread
From: Bommu Krishnaiah @ 2023-12-08 4:18 UTC (permalink / raw)
To: igt-dev; +Cc: Bommu Krishnaiah, Rodrigo Vivi
Skipping the GPU mapping(vm_bind) for object, so that exec_queue
reset will happen and xe_wait_ufence will end return EIO not ETIME
Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Francois Dugast <francois.dugast@intel.com>
---
tests/intel/xe_waitfence.c | 82 ++++++++++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
index 4e94403a3..bd95fa1f8 100644
--- a/tests/intel/xe_waitfence.c
+++ b/tests/intel/xe_waitfence.c
@@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
*
* SUBTEST: invalid-engine
* Description: Check query with invalid engine info returns expected error code
+ *
+ * SUBTEST: invalid-exec_queue-wait
+ * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
*/
static void
@@ -224,6 +227,82 @@ invalid_engine(int fd)
do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EFAULT);
}
+static void
+invalid_exec_queue_wait(int fd)
+{
+ uint32_t bo, b;
+ uint64_t batch_offset;
+ uint64_t batch_addr;
+ uint64_t sdi_offset;
+ uint64_t sdi_addr;
+ uint64_t addr = 0x1a0000;
+
+ struct {
+ uint32_t batch[16];
+ uint64_t pad;
+ uint64_t vm_sync;
+ uint64_t exec_sync;
+ uint32_t data;
+ } *data;
+
+#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
+ struct drm_xe_sync sync[1] = {
+ { .flags = DRM_XE_SYNC_TYPE_USER_FENCE | DRM_XE_SYNC_FLAG_SIGNAL,
+ .timeline_value = USER_FENCE_VALUE },
+ };
+
+ struct drm_xe_exec exec = {
+ .num_batch_buffer = 1,
+ .num_syncs = 1,
+ .syncs = to_user_pointer(sync),
+ };
+
+ uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
+ uint32_t exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
+ struct drm_xe_wait_user_fence1 wait = {
+ .op = DRM_XE_UFENCE_WAIT_OP_EQ,
+ .flags = 0,
+ .value = 0xc0ffee,
+ .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
+ .timeout = -1,
+ .exec_queue_id = exec_queue,
+ };
+
+ bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
+ data = xe_bo_map(fd, bo, 0x40000);
+
+ batch_offset = (char *)&data[0].batch - (char *)data;
+ batch_addr = addr + batch_offset;
+ sdi_offset = (char *)&data[0].data - (char *)data;
+ sdi_addr = addr + sdi_offset;
+
+ b = 0;
+ data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
+ data[0].batch[b++] = sdi_addr;
+ data[0].batch[b++] = sdi_addr >> 32;
+ data[0].batch[b++] = 0xc0ffee;
+ data[0].batch[b++] = MI_BATCH_BUFFER_END;
+ igt_assert(b <= ARRAY_SIZE(data[0].batch));
+
+ wait.addr = to_user_pointer(&data[0].exec_sync);
+ exec.exec_queue_id = exec_queue;
+ exec.address = batch_addr;
+
+ xe_exec(fd, &exec);
+
+ /**
+ * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
+ * reset will happen and xe_wait_ufence will end return EIO not ETIME
+ */
+ do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
+
+ xe_exec_queue_destroy(fd, exec_queue);
+
+ if (bo) {
+ munmap(data, 0x40000);
+ gem_close(fd, bo);
+ }
+}
igt_main
{
@@ -250,6 +329,9 @@ igt_main
igt_subtest("invalid-engine")
invalid_engine(fd);
+ igt_subtest("invalid-exec_queue-wait")
+ invalid_exec_queue_wait(fd);
+
igt_fixture
drm_close_driver(fd);
}
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5)
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
@ 2023-12-08 5:48 ` Patchwork
2023-12-08 13:16 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6) Patchwork
3 siblings, 0 replies; 14+ messages in thread
From: Patchwork @ 2023-12-08 5:48 UTC (permalink / raw)
To: Bommu Krishnaiah; +Cc: igt-dev
== Series Details ==
Series: RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5)
URL : https://patchwork.freedesktop.org/series/127364/
State : failure
== Summary ==
IGT patchset build failed on latest successful build
431c2d2dd5828b25fcbe1c82afbac865f4771aee tests/intel/xe_create: create-big-vram subtest
Tail of build.log:
[613/1662] Compiling C object 'tests/59830eb@@i915_pm_rpm@exe/intel_i915_pm_rpm.c.o'.
[614/1662] Compiling C object 'tests/59830eb@@msm_mapping@exe/msm_msm_mapping.c.o'.
[615/1662] Compiling C object 'tests/59830eb@@xe_ccs@exe/intel_xe_ccs.c.o'.
[616/1662] Compiling C object 'tests/59830eb@@gem_sync@exe/intel_gem_sync.c.o'.
[617/1662] Compiling C object 'tests/59830eb@@msm_submitoverhead@exe/msm_msm_submitoverhead.c.o'.
[618/1662] Compiling C object 'tests/59830eb@@xe_pm_residency@exe/intel_xe_pm_residency.c.o'.
[619/1662] Compiling C object 'tests/59830eb@@xe_perf_pmu@exe/intel_xe_perf_pmu.c.o'.
[620/1662] Compiling C object 'tests/59830eb@@xe_spin_batch@exe/intel_xe_spin_batch.c.o'.
[621/1662] Compiling C object 'tests/59830eb@@xe_prime_self_import@exe/intel_xe_prime_self_import.c.o'.
[622/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_create_bo@exe/v3d_create_bo.c.o'.
[623/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_get_bo_offset@exe/v3d_get_bo_offset.c.o'.
[624/1662] Compiling C object 'tests/59830eb@@kms_psr@exe/intel_kms_psr.c.o'.
[625/1662] Compiling C object 'tests/59830eb@@msm_submit@exe/msm_msm_submit.c.o'.
[626/1662] Compiling C object 'tests/59830eb@@msm_recovery@exe/msm_msm_recovery.c.o'.
[627/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_get_param@exe/v3d_get_param.c.o'.
[628/1662] Compiling C object 'tests/59830eb@@xe_sysfs_scheduler@exe/intel_xe_sysfs_scheduler.c.o'.
[629/1662] Compiling C object 'tests/59830eb@@kms_chamelium_frames@exe/chamelium_kms_chamelium_helper.c.o'.
[630/1662] Compiling C object 'tests/59830eb@@kms_chamelium_audio@exe/chamelium_kms_chamelium_helper.c.o'.
[631/1662] Compiling C object 'tests/59830eb@@msm_shrink@exe/msm_msm_shrink.c.o'.
[632/1662] Compiling C object 'tests/59830eb@@kms_chamelium_hpd@exe/chamelium_kms_chamelium_helper.c.o'.
[633/1662] Compiling C object 'tests/59830eb@@gem_softpin@exe/intel_gem_softpin.c.o'.
[634/1662] Compiling C object 'tests/59830eb@@xe_pm@exe/intel_xe_pm.c.o'.
[635/1662] Compiling C object 'tests/59830eb@@kms_chamelium_color@exe/kms_color_helper.c.o'.
[636/1662] Compiling C object 'tests/59830eb@@kms_chamelium_edid@exe/chamelium_kms_chamelium_helper.c.o'.
[637/1662] Compiling C object 'tests/59830eb@@xe_query@exe/intel_xe_query.c.o'.
[638/1662] Compiling C object 'tests/59830eb@@xe_exec_balancer@exe/intel_xe_exec_balancer.c.o'.
[639/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_job_submission@exe/v3d_job_submission.c.o'.
[640/1662] Compiling C object 'tests/59830eb@@xe_pat@exe/intel_xe_pat.c.o'.
[641/1662] Compiling C object 'tests/v3d/cad21b8@@v3d_submit_csd@exe/v3d_submit_csd.c.o'.
[642/1662] Compiling C object 'tests/59830eb@@xe_exec_reset@exe/intel_xe_exec_reset.c.o'.
[643/1662] Compiling C object 'tests/59830eb@@xe_evict@exe/intel_xe_evict.c.o'.
[644/1662] Compiling C object 'tests/59830eb@@kms_psr2_sf@exe/intel_kms_psr2_sf.c.o'.
[645/1662] Compiling C object 'tests/59830eb@@kms_chamelium_audio@exe/chamelium_kms_chamelium_audio.c.o'.
[646/1662] Compiling C object 'tests/59830eb@@i915_query@exe/intel_i915_query.c.o'.
[647/1662] Compiling C object 'tests/59830eb@@kms_chamelium_edid@exe/chamelium_kms_chamelium_edid.c.o'.
[648/1662] Compiling C object 'tests/59830eb@@xe_intel_bb@exe/intel_xe_intel_bb.c.o'.
[649/1662] Compiling C object 'tests/59830eb@@kms_chamelium_color@exe/chamelium_kms_chamelium_color.c.o'.
[650/1662] Compiling C object 'tests/59830eb@@gem_exec_balancer@exe/intel_gem_exec_balancer.c.o'.
[651/1662] Generating i915-perf-equations with a custom command.
[652/1662] Compiling C object 'tests/59830eb@@kms_chamelium_frames@exe/chamelium_kms_chamelium_frames.c.o'.
[653/1662] Compiling C object 'tests/59830eb@@kms_pm_rpm@exe/intel_kms_pm_rpm.c.o'.
[654/1662] Compiling C object 'tests/59830eb@@kms_chamelium_hpd@exe/chamelium_kms_chamelium_hpd.c.o'.
[655/1662] Compiling C object 'tests/59830eb@@gem_exec_fence@exe/intel_gem_exec_fence.c.o'.
[656/1662] Compiling C object 'tests/59830eb@@xe_vm@exe/intel_xe_vm.c.o'.
[657/1662] Compiling C object 'tests/59830eb@@kms_frontbuffer_tracking@exe/intel_kms_frontbuffer_tracking.c.o'.
[658/1662] Compiling C object 'tests/59830eb@@perf_pmu@exe/intel_perf_pmu.c.o'.
[659/1662] Compiling C object 'tests/59830eb@@gem_exec_schedule@exe/intel_gem_exec_schedule.c.o'.
[660/1662] Compiling C object 'tests/59830eb@@perf@exe/intel_perf.c.o'.
[661/1662] Linking target lib/libigt.so.0.
ninja: build stopped: subcommand failed.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
@ 2023-12-08 5:57 ` Rodrigo Vivi
2023-12-08 13:05 ` Matthew Brost
0 siblings, 1 reply; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-08 5:57 UTC (permalink / raw)
To: Bommu Krishnaiah, Matthew Brost; +Cc: igt-dev
On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> remove the num_engines/instances members from drm_xe_wait_user_fence structure
> and add a exec_queue_id member
>
> Right now this is only checking if the engine list is sane and nothing
> else. In the end every operation with this IOCTL is a soft check.
> So, let's formalize that and only use this IOCTL to wait on the fence.
>
> exec_queue_id member will help to user space to get proper error code
> from kernel while in exec_queue reset
>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Francois Dugast <francois.dugast@intel.com>
> ---
> include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> lib/xe/xe_ioctl.h | 9 +++------
> tests/intel/xe_evict.c | 4 ++--
> tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> tests/intel/xe_exec_reset.c | 6 +++---
> tests/intel/xe_exec_threads.c | 15 ++++++++-------
> tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> 10 files changed, 79 insertions(+), 87 deletions(-)
>
> diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> index 590f7b7af..fd06e4920 100644
> --- a/include/drm-uapi/xe_drm.h
> +++ b/include/drm-uapi/xe_drm.h
> @@ -129,7 +129,6 @@ struct xe_user_extension {
> * It is returned as part of the @drm_xe_engine, but it also is used as
> * the input of engine selection for both @drm_xe_exec_queue_create and
> * @drm_xe_query_engine_cycles
> - *
> */
> struct drm_xe_engine_class_instance {
> #define DRM_XE_ENGINE_CLASS_RENDER 0
> @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> */
> #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> + /** @engine_class: engine class id */
> __u16 engine_class;
> -
> + /** @engine_instance: engine instance id */
> __u16 engine_instance;
> + /** @gt_id: Unique ID of this GT within the PCI Device */
> __u16 gt_id;
> /** @pad: MBZ */
> __u16 pad;
> @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> *
> * Note: For userptr and externally imported dma-buf the kernel expects
> * either 1WAY or 2WAY for the @pat_index.
> + *
> + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD restrictions
> + * on the @pat_index. For such mappings there is no actual memory being
> + * mapped (the address in the PTE is invalid), so the various PAT memory
> + * attributes likely do not apply. Simply leaving as zero is one
> + * option (still a valid pat_index).
> */
> __u16 pat_index;
>
> @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> /** @op: wait operation (type of comparison) */
> __u16 op;
>
> -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait on VM bind */
> -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> /** @flags: wait flags */
> __u16 flags;
>
> @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> */
> __s64 timeout;
>
> - /**
> - * @num_engines: number of engine instances to wait on, must be zero
> - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> - */
> - __u64 num_engines;
> + /** @exec_queue_id: exec_queue_id returned from xe_exec_queue_create_ioctl */
> + __u32 exec_queue_id;
>
> - /**
> - * @instances: user pointer to array of drm_xe_engine_class_instance to
> - * wait on, must be NULL when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> - */
> - __u64 instances;
> + /** @pad2: MBZ */
> + __u32 pad2;
>
> /** @reserved: Reserved */
> __u64 reserved[2];
> diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c
> index b29ca40ad..0a1ddc86b 100644
> --- a/lib/xe/xe_ioctl.c
> +++ b/lib/xe/xe_ioctl.c
> @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> * @fd: xe device fd
> * @addr: address of value to compare
> * @value: expected value (equal) in @address
> - * @eci: engine class instance
> + * @exec_queue: exec_queue id
> * @timeout: pointer to time to wait in nanoseconds
> *
> * Function compares @value with memory pointed by @addr until they are equal.
> @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> * signalled. Returns 0 on success, -errno of ioctl on error.
> */
> int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t *timeout)
> + uint32_t exec_queue, int64_t *timeout)
> {
> struct drm_xe_wait_user_fence wait = {
> .addr = to_user_pointer(addr),
> .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> + .flags = 0,
> .value = value,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> - .num_engines = eci ? 1 :0,
> - .instances = eci ? to_user_pointer(eci) : 0,
> + .exec_queue_id = exec_queue,
> };
>
> igt_assert(timeout);
> @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * @fd: xe device fd
> * @addr: address of value to compare
> * @value: expected value (equal) in @address
> - * @eci: engine class instance
> + * @exec_queue: exec_queue id
> * @timeout: time to wait in nanoseconds
> *
> * Function compares @value with memory pointed by @addr until they are equal.
> @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * Returns elapsed time in nanoseconds if user fence was signalled.
> */
> int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout)
> + uint32_t exec_queue, int64_t timeout)
> {
> - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue, &timeout), 0);
> return timeout;
> }
>
> @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * @fd: xe device fd
> * @addr: address of value to compare
> * @value: expected value (equal) in @address
> - * @eci: engine class instance
> + * @exec_queue: exec_queue id
> * @timeout: absolute time when wait expire
> *
> * Function compares @value with memory pointed by @addr until they are equal.
> @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> * Returns elapsed time in nanoseconds if user fence was signalled.
> */
> int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout)
> + uint32_t exec_queue, int64_t timeout)
> {
> struct drm_xe_wait_user_fence wait = {
> .addr = to_user_pointer(addr),
> .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP | DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> + .flags = !exec_queue ? DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
To be on the safe side we should probably change the signature of this function
to explicitly receive this flag instead of relying on the invalid exec_queue_id number.
Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP was used here...
Matt, thoughts?
Everything else in the patch looks good to me.
> .value = value,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = timeout,
> - .num_engines = eci ? 1 : 0,
> - .instances = eci ? to_user_pointer(eci) : 0,
> + .exec_queue_id = exec_queue,
> };
> struct timespec ts;
>
> diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h
> index bd660bd27..9c0b4b9bc 100644
> --- a/lib/xe/xe_ioctl.h
> +++ b/lib/xe/xe_ioctl.h
> @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue, uint64_t addr,
> struct drm_xe_sync *sync, uint32_t num_syncs);
> void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr);
> int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t *timeout);
> + uint32_t exec_queue, int64_t *timeout);
> int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout);
> + uint32_t exec_queue, int64_t timeout);
> int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout);
> + uint32_t exec_queue, int64_t timeout);
> void xe_force_gt_reset(int fd, int gt);
>
> #endif /* XE_IOCTL_H */
> diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
> index 89dc46fae..a37c2d97a 100644
> --- a/tests/intel/xe_evict.c
> +++ b/tests/intel/xe_evict.c
> @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> }
> #define TWENTY_SEC MS_TO_NS(20000)
> xe_wait_ufence(fd, &data[i].vm_sync, USER_FENCE_VALUE,
> - NULL, TWENTY_SEC);
> + bind_exec_queues[0], TWENTY_SEC);
> }
> sync[0].addr = addr + (char *)&data[i].exec_sync -
> (char *)data;
> @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> data = xe_bo_map(fd, __bo,
> ALIGN(sizeof(*data) * n_execs, 0x1000));
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, TWENTY_SEC);
> + exec_queues[i], TWENTY_SEC);
> igt_assert_eq(data[i].data, 0xc0ffee);
> }
> munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000));
> diff --git a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> index 79ff65e89..2448d49bc 100644
> --- a/tests/intel/xe_exec_balancer.c
> +++ b/tests/intel/xe_exec_balancer.c
> @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> bo_size, sync, 1);
>
> #define ONE_SEC MS_TO_NS(1000)
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
>
> if (flags & REBIND && i + 1 != n_execs) {
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + exec_queues[e], ONE_SEC);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, NULL,
> 0);
>
> @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + 0, ONE_SEC);
> data[0].vm_sync = 0;
> }
>
> @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> * an invalidate.
> */
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL, ONE_SEC);
> + USER_FENCE_VALUE, exec_queues[e],
> + ONE_SEC);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> /*
> @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
>
> j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> for (i = j; i < n_execs; i++)
> - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> - ONE_SEC);
> + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> + exec_queues[i], ONE_SEC);
>
> /* Wait for all execs to complete */
> if (flags & INVALIDATE)
> @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
>
> for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> i < n_execs; i++)
> diff --git a/tests/intel/xe_exec_compute_mode.c b/tests/intel/xe_exec_compute_mode.c
> index 7d3004d65..749a9b634 100644
> --- a/tests/intel/xe_exec_compute_mode.c
> +++ b/tests/intel/xe_exec_compute_mode.c
> @@ -171,8 +171,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC : ONE_SEC;
>
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], fence_timeout);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -198,7 +198,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> if (flags & REBIND && i + 1 != n_execs) {
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + exec_queues[e], fence_timeout);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> addr, bo_size, NULL, 0);
>
> @@ -214,7 +214,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + bind_exec_queues[e], fence_timeout);
> data[0].vm_sync = 0;
> }
>
> @@ -227,7 +227,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> * an invalidate.
> */
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL,
> + USER_FENCE_VALUE, exec_queues[e],
> fence_timeout);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> @@ -257,8 +257,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> j = flags & INVALIDATE ? n_execs - 1 : 0;
> for (i = j; i < n_execs; i++)
> - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> + exec_queues[i], fence_timeout);
>
> /* Wait for all execs to complete */
> if (flags & INVALIDATE)
> @@ -267,8 +267,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], fence_timeout);
>
> for (i = j; i < n_execs; i++)
> igt_assert_eq(data[i].data, 0xc0ffee);
> diff --git a/tests/intel/xe_exec_fault_mode.c b/tests/intel/xe_exec_fault_mode.c
> index ee7cbb604..d76b3a056 100644
> --- a/tests/intel/xe_exec_fault_mode.c
> +++ b/tests/intel/xe_exec_fault_mode.c
> @@ -195,15 +195,16 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> }
>
> #define ONE_SEC MS_TO_NS(1000)
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], ONE_SEC);
> data[0].vm_sync = 0;
>
> if (flags & PREFETCH) {
> /* Should move to system memory */
> xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0, addr,
> bo_size, sync, 1, 0);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> - ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], ONE_SEC);
> data[0].vm_sync = 0;
> }
>
> @@ -230,7 +231,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
>
> if (flags & REBIND && i + 1 != n_execs) {
> xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + exec_queues[e], ONE_SEC);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> addr, bo_size, NULL, 0);
>
> @@ -246,7 +247,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, ONE_SEC);
> + bind_exec_queues[e], ONE_SEC);
> data[0].vm_sync = 0;
> }
>
> @@ -259,7 +260,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> * an invalidate.
> */
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL, ONE_SEC);
> + USER_FENCE_VALUE, exec_queues[e],
> + ONE_SEC);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> /*
> @@ -290,13 +292,14 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> j = flags & INVALIDATE ? n_execs - 1 : 0;
> for (i = j; i < n_execs; i++)
> xe_wait_ufence(fd, &data[i].exec_sync,
> - USER_FENCE_VALUE, NULL, ONE_SEC);
> + USER_FENCE_VALUE, exec_queues[i], ONE_SEC);
> }
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> + bind_exec_queues[0], ONE_SEC);
>
> if (!(flags & INVALID_FAULT)) {
> for (i = j; i < n_execs; i++)
> diff --git a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
> index 094b34896..398c90af5 100644
> --- a/tests/intel/xe_exec_reset.c
> +++ b/tests/intel/xe_exec_reset.c
> @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
>
> #define THREE_SEC MS_TO_NS(3000)
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> int err;
>
> err = __xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> - NULL, &timeout);
> + exec_queues[i], &timeout);
> if (flags & GT_RESET)
> /* exec races with reset: may timeout or complete */
> igt_assert(err == -ETIME || !err);
> @@ -631,7 +631,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
>
> if (!(flags & GT_RESET)) {
> for (i = 1; i < n_execs; i++)
> diff --git a/tests/intel/xe_exec_threads.c b/tests/intel/xe_exec_threads.c
> index fcb926698..7985240c9 100644
> --- a/tests/intel/xe_exec_threads.c
> +++ b/tests/intel/xe_exec_threads.c
> @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>
> fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
>
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
> data[0].vm_sync = 0;
>
> for (i = 0; i < n_execs; i++) {
> @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> for (j = i - 0x20; j <= i; ++j)
> xe_wait_ufence(fd, &data[j].exec_sync,
> USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + exec_queues[e], fence_timeout);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> NULL, 0);
>
> @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> addr, bo_size, sync,
> 1);
> xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + 0, fence_timeout);
> data[0].vm_sync = 0;
> }
>
> @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> xe_wait_ufence(fd, &data[j].exec_sync,
> USER_FENCE_VALUE,
> - NULL, fence_timeout);
> + exec_queues[e],
> + fence_timeout);
> igt_assert_eq(data[i].data, 0xc0ffee);
> } else if (i * 2 != n_execs) {
> /*
> @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> j = flags & INVALIDATE ?
> (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> for (i = j; i < n_execs; i++)
> - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> - fence_timeout);
> + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> + exec_queues[e], fence_timeout);
>
> /* Wait for all execs to complete */
> if (flags & INVALIDATE)
> @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
>
> sync[0].addr = to_user_pointer(&data[0].vm_sync);
> xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
>
> for (i = j; i < n_execs; i++)
> igt_assert_eq(data[i].data, 0xc0ffee);
> diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> index 3be987954..4e94403a3 100644
> --- a/tests/intel/xe_waitfence.c
> +++ b/tests/intel/xe_waitfence.c
> @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm, uint32_t bo, uint64_t offset,
> }
>
> static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t value,
> - struct drm_xe_engine_class_instance *eci,
> - int64_t timeout)
> + uint32_t exec_queue, int64_t timeout)
> {
> struct drm_xe_wait_user_fence wait = {
> .addr = to_user_pointer(addr),
> .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> + .flags = !exec_queue ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> .value = value,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = timeout,
> - .num_engines = eci ? 1 : 0,
> - .instances = eci ? to_user_pointer(eci) : 0,
> + .exec_queue_id = exec_queue,
> };
> struct timespec ts;
>
> - igt_assert(eci);
> igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait), 0);
> igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
>
> @@ -82,7 +79,7 @@ enum waittype {
> static void
> waitfence(int fd, enum waittype wt)
> {
> - struct drm_xe_engine *engine = NULL;
> + uint32_t exec_queue;
> struct timespec ts;
> int64_t current, signalled;
> uint32_t bo_1;
> @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
>
> if (wt == RELTIME) {
> - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL, MS_TO_NS(10));
> + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0, MS_TO_NS(10));
> igt_debug("wait type: RELTIME - timeout: %ld, timeout left: %ld\n",
> MS_TO_NS(10), timeout);
> } else if (wt == ENGINE) {
> - engine = xe_engine(fd, 1);
> + exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
> clock_gettime(CLOCK_MONOTONIC, &ts);
> current = ts.tv_sec * 1e9 + ts.tv_nsec;
> timeout = current + MS_TO_NS(10);
> - signalled = wait_with_eci_abstime(fd, &wait_fence, 7, &engine->instance, timeout);
> + signalled = wait_with_eci_abstime(fd, &wait_fence, 7, exec_queue, timeout);
> igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> ", signalled: %" PRId64
> ", elapsed: %" PRId64 "\n",
> @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> clock_gettime(CLOCK_MONOTONIC, &ts);
> current = ts.tv_sec * 1e9 + ts.tv_nsec;
> timeout = current + MS_TO_NS(10);
> - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, NULL, timeout);
> + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0, timeout);
> igt_debug("wait type: ABSTIME - timeout: %" PRId64
> ", signalled: %" PRId64
> ", elapsed: %" PRId64 "\n",
> @@ -191,8 +188,7 @@ invalid_ops(int fd)
> .value = 1,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = 1,
> - .num_engines = 0,
> - .instances = 0,
> + .exec_queue_id = 0,
> };
>
> uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> @@ -216,8 +212,7 @@ invalid_engine(int fd)
> .value = 1,
> .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> .timeout = -1,
> - .num_engines = 1,
> - .instances = 0,
> + .exec_queue_id = 0,
> };
>
> uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
@ 2023-12-08 6:00 ` Rodrigo Vivi
2023-12-08 12:59 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-08 6:00 UTC (permalink / raw)
To: Bommu Krishnaiah; +Cc: igt-dev
On Fri, Dec 08, 2023 at 09:48:25AM +0530, Bommu Krishnaiah wrote:
> Skipping the GPU mapping(vm_bind) for object, so that exec_queue
'skipping' sounds like you are not waiting at all...
> reset will happen and xe_wait_ufence will end return EIO not ETIME
>
> Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Cc: Francois Dugast <francois.dugast@intel.com>
> ---
> tests/intel/xe_waitfence.c | 82 ++++++++++++++++++++++++++++++++++++++
> 1 file changed, 82 insertions(+)
>
> diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> index 4e94403a3..bd95fa1f8 100644
> --- a/tests/intel/xe_waitfence.c
> +++ b/tests/intel/xe_waitfence.c
> @@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
> *
> * SUBTEST: invalid-engine
> * Description: Check query with invalid engine info returns expected error code
> + *
> + * SUBTEST: invalid-exec_queue-wait
'invalid wait' sounded to me like a negative test with invalid timeout.
So I believe we need to rephrase some stuff here, but the test looks right.
> + * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
> */
>
> static void
> @@ -224,6 +227,82 @@ invalid_engine(int fd)
> do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EFAULT);
> }
>
> +static void
> +invalid_exec_queue_wait(int fd)
> +{
> + uint32_t bo, b;
> + uint64_t batch_offset;
> + uint64_t batch_addr;
> + uint64_t sdi_offset;
> + uint64_t sdi_addr;
> + uint64_t addr = 0x1a0000;
> +
> + struct {
> + uint32_t batch[16];
> + uint64_t pad;
> + uint64_t vm_sync;
> + uint64_t exec_sync;
> + uint32_t data;
> + } *data;
> +
> +#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
> + struct drm_xe_sync sync[1] = {
> + { .flags = DRM_XE_SYNC_TYPE_USER_FENCE | DRM_XE_SYNC_FLAG_SIGNAL,
> + .timeline_value = USER_FENCE_VALUE },
> + };
> +
> + struct drm_xe_exec exec = {
> + .num_batch_buffer = 1,
> + .num_syncs = 1,
> + .syncs = to_user_pointer(sync),
> + };
> +
> + uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> + uint32_t exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
> + struct drm_xe_wait_user_fence1 wait = {
> + .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> + .flags = 0,
> + .value = 0xc0ffee,
> + .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> + .timeout = -1,
> + .exec_queue_id = exec_queue,
> + };
> +
> + bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
> + data = xe_bo_map(fd, bo, 0x40000);
> +
> + batch_offset = (char *)&data[0].batch - (char *)data;
> + batch_addr = addr + batch_offset;
> + sdi_offset = (char *)&data[0].data - (char *)data;
> + sdi_addr = addr + sdi_offset;
> +
> + b = 0;
> + data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> + data[0].batch[b++] = sdi_addr;
> + data[0].batch[b++] = sdi_addr >> 32;
> + data[0].batch[b++] = 0xc0ffee;
> + data[0].batch[b++] = MI_BATCH_BUFFER_END;
> + igt_assert(b <= ARRAY_SIZE(data[0].batch));
> +
> + wait.addr = to_user_pointer(&data[0].exec_sync);
> + exec.exec_queue_id = exec_queue;
> + exec.address = batch_addr;
> +
> + xe_exec(fd, &exec);
> +
> + /**
> + * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
> + * reset will happen and xe_wait_ufence will end return EIO not ETIME
> + */
> + do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
> +
> + xe_exec_queue_destroy(fd, exec_queue);
> +
> + if (bo) {
> + munmap(data, 0x40000);
> + gem_close(fd, bo);
> + }
> +}
>
> igt_main
> {
> @@ -250,6 +329,9 @@ igt_main
> igt_subtest("invalid-engine")
> invalid_engine(fd);
>
> + igt_subtest("invalid-exec_queue-wait")
> + invalid_exec_queue_wait(fd);
> +
> igt_fixture
> drm_close_driver(fd);
> }
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 6:00 ` Rodrigo Vivi
@ 2023-12-08 12:59 ` Bommu, Krishnaiah
2023-12-08 13:48 ` Rodrigo Vivi
0 siblings, 1 reply; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-08 12:59 UTC (permalink / raw)
To: Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Friday, December 8, 2023 11:31 AM
> To: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: igt-dev@lists.freedesktop.org
> Subject: Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl
> when exec_queue reset happen
>
> On Fri, Dec 08, 2023 at 09:48:25AM +0530, Bommu Krishnaiah wrote:
> > Skipping the GPU mapping(vm_bind) for object, so that exec_queue
>
> 'skipping' sounds like you are not waiting at all...
>
> > reset will happen and xe_wait_ufence will end return EIO not ETIME
> >
> > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Cc: Francois Dugast <francois.dugast@intel.com>
> > ---
> > tests/intel/xe_waitfence.c | 82
> > ++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 82 insertions(+)
> >
> > diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > index 4e94403a3..bd95fa1f8 100644
> > --- a/tests/intel/xe_waitfence.c
> > +++ b/tests/intel/xe_waitfence.c
> > @@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
> > *
> > * SUBTEST: invalid-engine
> > * Description: Check query with invalid engine info returns expected
> > error code
> > + *
> > + * SUBTEST: invalid-exec_queue-wait
>
> 'invalid wait' sounded to me like a negative test with invalid timeout.
>
> So I believe we need to rephrase some stuff here, but the test looks right.
>
Commit message:
drm-uapi/xe: Don't wait on user_fence during exec queue reset
don't wait till timeout on user fence when exec_queue reset is detected
and return return proper error code
root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools# git diff
diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
index bd95fa1f8..9a2c854c1 100644
--- a/tests/intel/xe_waitfence.c
+++ b/tests/intel/xe_waitfence.c
@@ -150,8 +150,8 @@ waitfence(int fd, enum waittype wt)
* SUBTEST: invalid-engine
* Description: Check query with invalid engine info returns expected error code
*
- * SUBTEST: invalid-exec_queue-wait
- * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
+ * SUBTEST: exec_queue-reset-wait
+ * Description: Don't wait till timeout on user fence when exec_queue reset is detected and return return proper error
*/
static void
@@ -228,7 +228,7 @@ invalid_engine(int fd)
}
static void
-invalid_exec_queue_wait(int fd)
+exec_queue_reset_wait(int fd)
{
uint32_t bo, b;
uint64_t batch_offset;
@@ -291,8 +291,8 @@ invalid_exec_queue_wait(int fd)
xe_exec(fd, &exec);
/**
- * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
- * reset will happen and xe_wait_ufence will end return EIO not ETIME
+ * Don't do the GPU mapping(vm_bind) for object, so that exec_queue
+ * reset will happen and xe_wait_ufence will return EIO not ETIME
*/
do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
@@ -329,8 +329,8 @@ igt_main
igt_subtest("invalid-engine")
invalid_engine(fd);
- igt_subtest("invalid-exec_queue-wait")
- invalid_exec_queue_wait(fd);
+ igt_subtest("exec_queue-reset-wait")
+ exec_queue_reset_wait(fd);
igt_fixture
drm_close_driver(fd);
root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
This is what I can think of now, please suggest anything need to be changed
Regards,
Krishna
> > + * Description: Check xe_wait_ufence will return expected error code
> > + while exec_queue reset happen
> > */
> >
> > static void
> > @@ -224,6 +227,82 @@ invalid_engine(int fd)
> > do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait,
> EFAULT); }
> >
> > +static void
> > +invalid_exec_queue_wait(int fd)
> > +{
> > + uint32_t bo, b;
> > + uint64_t batch_offset;
> > + uint64_t batch_addr;
> > + uint64_t sdi_offset;
> > + uint64_t sdi_addr;
> > + uint64_t addr = 0x1a0000;
> > +
> > + struct {
> > + uint32_t batch[16];
> > + uint64_t pad;
> > + uint64_t vm_sync;
> > + uint64_t exec_sync;
> > + uint32_t data;
> > + } *data;
> > +
> > +#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
> > + struct drm_xe_sync sync[1] = {
> > + { .flags = DRM_XE_SYNC_TYPE_USER_FENCE |
> DRM_XE_SYNC_FLAG_SIGNAL,
> > + .timeline_value = USER_FENCE_VALUE },
> > + };
> > +
> > + struct drm_xe_exec exec = {
> > + .num_batch_buffer = 1,
> > + .num_syncs = 1,
> > + .syncs = to_user_pointer(sync),
> > + };
> > +
> > + uint32_t vm = xe_vm_create(fd,
> DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > + uint32_t exec_queue = xe_exec_queue_create_class(fd, vm,
> DRM_XE_ENGINE_CLASS_COPY);
> > + struct drm_xe_wait_user_fence1 wait = {
> > + .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > + .flags = 0,
> > + .value = 0xc0ffee,
> > + .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > + .timeout = -1,
> > + .exec_queue_id = exec_queue,
> > + };
> > +
> > + bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
> > + data = xe_bo_map(fd, bo, 0x40000);
> > +
> > + batch_offset = (char *)&data[0].batch - (char *)data;
> > + batch_addr = addr + batch_offset;
> > + sdi_offset = (char *)&data[0].data - (char *)data;
> > + sdi_addr = addr + sdi_offset;
> > +
> > + b = 0;
> > + data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> > + data[0].batch[b++] = sdi_addr;
> > + data[0].batch[b++] = sdi_addr >> 32;
> > + data[0].batch[b++] = 0xc0ffee;
> > + data[0].batch[b++] = MI_BATCH_BUFFER_END;
> > + igt_assert(b <= ARRAY_SIZE(data[0].batch));
> > +
> > + wait.addr = to_user_pointer(&data[0].exec_sync);
> > + exec.exec_queue_id = exec_queue;
> > + exec.address = batch_addr;
> > +
> > + xe_exec(fd, &exec);
> > +
> > + /**
> > + * Skipping the GPU mapping(vm_bind) for object, so that
> exec_queue
> > + * reset will happen and xe_wait_ufence will end return EIO not
> ETIME
> > + */
> > + do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
> > +
> > + xe_exec_queue_destroy(fd, exec_queue);
> > +
> > + if (bo) {
> > + munmap(data, 0x40000);
> > + gem_close(fd, bo);
> > + }
> > +}
> >
> > igt_main
> > {
> > @@ -250,6 +329,9 @@ igt_main
> > igt_subtest("invalid-engine")
> > invalid_engine(fd);
> >
> > + igt_subtest("invalid-exec_queue-wait")
> > + invalid_exec_queue_wait(fd);
> > +
> > igt_fixture
> > drm_close_driver(fd);
> > }
> > --
> > 2.25.1
> >
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 5:57 ` Rodrigo Vivi
@ 2023-12-08 13:05 ` Matthew Brost
2023-12-11 5:36 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Matthew Brost @ 2023-12-08 13:05 UTC (permalink / raw)
To: Rodrigo Vivi; +Cc: igt-dev, Bommu Krishnaiah
On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > remove the num_engines/instances members from drm_xe_wait_user_fence structure
> > and add a exec_queue_id member
> >
> > Right now this is only checking if the engine list is sane and nothing
> > else. In the end every operation with this IOCTL is a soft check.
> > So, let's formalize that and only use this IOCTL to wait on the fence.
> >
> > exec_queue_id member will help to user space to get proper error code
> > from kernel while in exec_queue reset
> >
> > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > Cc: Francois Dugast <francois.dugast@intel.com>
> > ---
> > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > lib/xe/xe_ioctl.h | 9 +++------
> > tests/intel/xe_evict.c | 4 ++--
> > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > tests/intel/xe_exec_reset.c | 6 +++---
> > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > 10 files changed, 79 insertions(+), 87 deletions(-)
> >
> > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > index 590f7b7af..fd06e4920 100644
> > --- a/include/drm-uapi/xe_drm.h
> > +++ b/include/drm-uapi/xe_drm.h
> > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > * It is returned as part of the @drm_xe_engine, but it also is used as
> > * the input of engine selection for both @drm_xe_exec_queue_create and
> > * @drm_xe_query_engine_cycles
> > - *
> > */
> > struct drm_xe_engine_class_instance {
> > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > */
> > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > + /** @engine_class: engine class id */
> > __u16 engine_class;
> > -
> > + /** @engine_instance: engine instance id */
> > __u16 engine_instance;
> > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > __u16 gt_id;
> > /** @pad: MBZ */
> > __u16 pad;
> > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > *
> > * Note: For userptr and externally imported dma-buf the kernel expects
> > * either 1WAY or 2WAY for the @pat_index.
> > + *
> > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD restrictions
> > + * on the @pat_index. For such mappings there is no actual memory being
> > + * mapped (the address in the PTE is invalid), so the various PAT memory
> > + * attributes likely do not apply. Simply leaving as zero is one
> > + * option (still a valid pat_index).
> > */
> > __u16 pat_index;
> >
> > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > /** @op: wait operation (type of comparison) */
> > __u16 op;
> >
> > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait on VM bind */
> > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > /** @flags: wait flags */
> > __u16 flags;
> >
> > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > */
> > __s64 timeout;
> >
> > - /**
> > - * @num_engines: number of engine instances to wait on, must be zero
> > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > - */
> > - __u64 num_engines;
> > + /** @exec_queue_id: exec_queue_id returned from xe_exec_queue_create_ioctl */
> > + __u32 exec_queue_id;
> >
> > - /**
> > - * @instances: user pointer to array of drm_xe_engine_class_instance to
> > - * wait on, must be NULL when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > - */
> > - __u64 instances;
> > + /** @pad2: MBZ */
> > + __u32 pad2;
> >
> > /** @reserved: Reserved */
> > __u64 reserved[2];
> > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c
> > index b29ca40ad..0a1ddc86b 100644
> > --- a/lib/xe/xe_ioctl.c
> > +++ b/lib/xe/xe_ioctl.c
> > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> > * @fd: xe device fd
> > * @addr: address of value to compare
> > * @value: expected value (equal) in @address
> > - * @eci: engine class instance
> > + * @exec_queue: exec_queue id
> > * @timeout: pointer to time to wait in nanoseconds
> > *
> > * Function compares @value with memory pointed by @addr until they are equal.
> > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr)
> > * signalled. Returns 0 on success, -errno of ioctl on error.
> > */
> > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t *timeout)
> > + uint32_t exec_queue, int64_t *timeout)
> > {
> > struct drm_xe_wait_user_fence wait = {
> > .addr = to_user_pointer(addr),
> > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > + .flags = 0,
> > .value = value,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > - .num_engines = eci ? 1 :0,
> > - .instances = eci ? to_user_pointer(eci) : 0,
> > + .exec_queue_id = exec_queue,
> > };
> >
> > igt_assert(timeout);
> > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * @fd: xe device fd
> > * @addr: address of value to compare
> > * @value: expected value (equal) in @address
> > - * @eci: engine class instance
> > + * @exec_queue: exec_queue id
> > * @timeout: time to wait in nanoseconds
> > *
> > * Function compares @value with memory pointed by @addr until they are equal.
> > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * Returns elapsed time in nanoseconds if user fence was signalled.
> > */
> > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout)
> > + uint32_t exec_queue, int64_t timeout)
> > {
> > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue, &timeout), 0);
> > return timeout;
> > }
> >
> > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * @fd: xe device fd
> > * @addr: address of value to compare
> > * @value: expected value (equal) in @address
> > - * @eci: engine class instance
> > + * @exec_queue: exec_queue id
> > * @timeout: absolute time when wait expire
> > *
> > * Function compares @value with memory pointed by @addr until they are equal.
> > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > * Returns elapsed time in nanoseconds if user fence was signalled.
> > */
> > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout)
> > + uint32_t exec_queue, int64_t timeout)
> > {
> > struct drm_xe_wait_user_fence wait = {
> > .addr = to_user_pointer(addr),
> > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP | DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > + .flags = !exec_queue ? DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
>
> To be on the safe side we should probably change the signature of this function
> to explicitly receive this flag instead of relying on the invalid exec_queue_id number.
>
> Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP was used here...
>
> Matt, thoughts?
>
Agree with everything here, !exec_queue really has nothing to do with
setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So yes, please
change this function to accpt a flags field.
Everything else LGTM.
Matt
> Everything else in the patch looks good to me.
>
> > .value = value,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = timeout,
> > - .num_engines = eci ? 1 : 0,
> > - .instances = eci ? to_user_pointer(eci) : 0,
> > + .exec_queue_id = exec_queue,
> > };
> > struct timespec ts;
> >
> > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h
> > index bd660bd27..9c0b4b9bc 100644
> > --- a/lib/xe/xe_ioctl.h
> > +++ b/lib/xe/xe_ioctl.h
> > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue, uint64_t addr,
> > struct drm_xe_sync *sync, uint32_t num_syncs);
> > void xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr);
> > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t *timeout);
> > + uint32_t exec_queue, int64_t *timeout);
> > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout);
> > + uint32_t exec_queue, int64_t timeout);
> > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout);
> > + uint32_t exec_queue, int64_t timeout);
> > void xe_force_gt_reset(int fd, int gt);
> >
> > #endif /* XE_IOCTL_H */
> > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
> > index 89dc46fae..a37c2d97a 100644
> > --- a/tests/intel/xe_evict.c
> > +++ b/tests/intel/xe_evict.c
> > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> > }
> > #define TWENTY_SEC MS_TO_NS(20000)
> > xe_wait_ufence(fd, &data[i].vm_sync, USER_FENCE_VALUE,
> > - NULL, TWENTY_SEC);
> > + bind_exec_queues[0], TWENTY_SEC);
> > }
> > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > (char *)data;
> > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct drm_xe_engine_class_instance *eci,
> > data = xe_bo_map(fd, __bo,
> > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, TWENTY_SEC);
> > + exec_queues[i], TWENTY_SEC);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > }
> > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000));
> > diff --git a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > index 79ff65e89..2448d49bc 100644
> > --- a/tests/intel/xe_exec_balancer.c
> > +++ b/tests/intel/xe_exec_balancer.c
> > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> > bo_size, sync, 1);
> >
> > #define ONE_SEC MS_TO_NS(1000)
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> >
> > if (flags & REBIND && i + 1 != n_execs) {
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + exec_queues[e], ONE_SEC);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, NULL,
> > 0);
> >
> > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + 0, ONE_SEC);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> > * an invalidate.
> > */
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > + USER_FENCE_VALUE, exec_queues[e],
> > + ONE_SEC);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > /*
> > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> >
> > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > for (i = j; i < n_execs; i++)
> > - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> > - ONE_SEC);
> > + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > + exec_queues[i], ONE_SEC);
> >
> > /* Wait for all execs to complete */
> > if (flags & INVALIDATE)
> > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int n_exec_queues, int n_execs,
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, ONE_SEC);
> >
> > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > i < n_execs; i++)
> > diff --git a/tests/intel/xe_exec_compute_mode.c b/tests/intel/xe_exec_compute_mode.c
> > index 7d3004d65..749a9b634 100644
> > --- a/tests/intel/xe_exec_compute_mode.c
> > +++ b/tests/intel/xe_exec_compute_mode.c
> > @@ -171,8 +171,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC : ONE_SEC;
> >
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], fence_timeout);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -198,7 +198,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > if (flags & REBIND && i + 1 != n_execs) {
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + exec_queues[e], fence_timeout);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> > addr, bo_size, NULL, 0);
> >
> > @@ -214,7 +214,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + bind_exec_queues[e], fence_timeout);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -227,7 +227,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > * an invalidate.
> > */
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL,
> > + USER_FENCE_VALUE, exec_queues[e],
> > fence_timeout);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > @@ -257,8 +257,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > for (i = j; i < n_execs; i++)
> > - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > + exec_queues[i], fence_timeout);
> >
> > /* Wait for all execs to complete */
> > if (flags & INVALIDATE)
> > @@ -267,8 +267,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> > sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], fence_timeout);
> >
> > for (i = j; i < n_execs; i++)
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > diff --git a/tests/intel/xe_exec_fault_mode.c b/tests/intel/xe_exec_fault_mode.c
> > index ee7cbb604..d76b3a056 100644
> > --- a/tests/intel/xe_exec_fault_mode.c
> > +++ b/tests/intel/xe_exec_fault_mode.c
> > @@ -195,15 +195,16 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > }
> >
> > #define ONE_SEC MS_TO_NS(1000)
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], ONE_SEC);
> > data[0].vm_sync = 0;
> >
> > if (flags & PREFETCH) {
> > /* Should move to system memory */
> > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0, addr,
> > bo_size, sync, 1, 0);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > - ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], ONE_SEC);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -230,7 +231,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > if (flags & REBIND && i + 1 != n_execs) {
> > xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + exec_queues[e], ONE_SEC);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[e], 0,
> > addr, bo_size, NULL, 0);
> >
> > @@ -246,7 +247,7 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, ONE_SEC);
> > + bind_exec_queues[e], ONE_SEC);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -259,7 +260,8 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > * an invalidate.
> > */
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > + USER_FENCE_VALUE, exec_queues[e],
> > + ONE_SEC);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > /*
> > @@ -290,13 +292,14 @@ test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > for (i = j; i < n_execs; i++)
> > xe_wait_ufence(fd, &data[i].exec_sync,
> > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > + USER_FENCE_VALUE, exec_queues[i], ONE_SEC);
> > }
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr, bo_size,
> > sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, ONE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > + bind_exec_queues[0], ONE_SEC);
> >
> > if (!(flags & INVALID_FAULT)) {
> > for (i = j; i < n_execs; i++)
> > diff --git a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
> > index 094b34896..398c90af5 100644
> > --- a/tests/intel/xe_exec_reset.c
> > +++ b/tests/intel/xe_exec_reset.c
> > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> >
> > #define THREE_SEC MS_TO_NS(3000)
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> > int err;
> >
> > err = __xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > - NULL, &timeout);
> > + exec_queues[i], &timeout);
> > if (flags & GT_RESET)
> > /* exec races with reset: may timeout or complete */
> > igt_assert(err == -ETIME || !err);
> > @@ -631,7 +631,7 @@ test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, THREE_SEC);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, THREE_SEC);
> >
> > if (!(flags & GT_RESET)) {
> > for (i = 1; i < n_execs; i++)
> > diff --git a/tests/intel/xe_exec_threads.c b/tests/intel/xe_exec_threads.c
> > index fcb926698..7985240c9 100644
> > --- a/tests/intel/xe_exec_threads.c
> > +++ b/tests/intel/xe_exec_threads.c
> > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> >
> > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
> >
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
> > data[0].vm_sync = 0;
> >
> > for (i = 0; i < n_execs; i++) {
> > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > for (j = i - 0x20; j <= i; ++j)
> > xe_wait_ufence(fd, &data[j].exec_sync,
> > USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + exec_queues[e], fence_timeout);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > NULL, 0);
> >
> > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > addr, bo_size, sync,
> > 1);
> > xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + 0, fence_timeout);
> > data[0].vm_sync = 0;
> > }
> >
> > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > xe_wait_ufence(fd, &data[j].exec_sync,
> > USER_FENCE_VALUE,
> > - NULL, fence_timeout);
> > + exec_queues[e],
> > + fence_timeout);
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > } else if (i * 2 != n_execs) {
> > /*
> > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> > j = flags & INVALIDATE ?
> > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > for (i = j; i < n_execs; i++)
> > - xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE, NULL,
> > - fence_timeout);
> > + xe_wait_ufence(fd, &data[i].exec_sync, USER_FENCE_VALUE,
> > + exec_queues[e], fence_timeout);
> >
> > /* Wait for all execs to complete */
> > if (flags & INVALIDATE)
> > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t addr, uint64_t userptr,
> >
> > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL, fence_timeout);
> > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0, fence_timeout);
> >
> > for (i = j; i < n_execs; i++)
> > igt_assert_eq(data[i].data, 0xc0ffee);
> > diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > index 3be987954..4e94403a3 100644
> > --- a/tests/intel/xe_waitfence.c
> > +++ b/tests/intel/xe_waitfence.c
> > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm, uint32_t bo, uint64_t offset,
> > }
> >
> > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t value,
> > - struct drm_xe_engine_class_instance *eci,
> > - int64_t timeout)
> > + uint32_t exec_queue, int64_t timeout)
> > {
> > struct drm_xe_wait_user_fence wait = {
> > .addr = to_user_pointer(addr),
> > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > + .flags = !exec_queue ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > .value = value,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = timeout,
> > - .num_engines = eci ? 1 : 0,
> > - .instances = eci ? to_user_pointer(eci) : 0,
> > + .exec_queue_id = exec_queue,
> > };
> > struct timespec ts;
> >
> > - igt_assert(eci);
> > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait), 0);
> > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> >
> > @@ -82,7 +79,7 @@ enum waittype {
> > static void
> > waitfence(int fd, enum waittype wt)
> > {
> > - struct drm_xe_engine *engine = NULL;
> > + uint32_t exec_queue;
> > struct timespec ts;
> > int64_t current, signalled;
> > uint32_t bo_1;
> > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> >
> > if (wt == RELTIME) {
> > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL, MS_TO_NS(10));
> > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0, MS_TO_NS(10));
> > igt_debug("wait type: RELTIME - timeout: %ld, timeout left: %ld\n",
> > MS_TO_NS(10), timeout);
> > } else if (wt == ENGINE) {
> > - engine = xe_engine(fd, 1);
> > + exec_queue = xe_exec_queue_create_class(fd, vm, DRM_XE_ENGINE_CLASS_COPY);
> > clock_gettime(CLOCK_MONOTONIC, &ts);
> > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > timeout = current + MS_TO_NS(10);
> > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7, &engine->instance, timeout);
> > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7, exec_queue, timeout);
> > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > ", signalled: %" PRId64
> > ", elapsed: %" PRId64 "\n",
> > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > clock_gettime(CLOCK_MONOTONIC, &ts);
> > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > timeout = current + MS_TO_NS(10);
> > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, NULL, timeout);
> > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0, timeout);
> > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > ", signalled: %" PRId64
> > ", elapsed: %" PRId64 "\n",
> > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > .value = 1,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = 1,
> > - .num_engines = 0,
> > - .instances = 0,
> > + .exec_queue_id = 0,
> > };
> >
> > uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > @@ -216,8 +212,7 @@ invalid_engine(int fd)
> > .value = 1,
> > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > .timeout = -1,
> > - .num_engines = 1,
> > - .instances = 0,
> > + .exec_queue_id = 0,
> > };
> >
> > uint32_t vm = xe_vm_create(fd, DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > --
> > 2.25.1
> >
^ permalink raw reply [flat|nested] 14+ messages in thread
* ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6)
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
` (2 preceding siblings ...)
2023-12-08 5:48 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5) Patchwork
@ 2023-12-08 13:16 ` Patchwork
3 siblings, 0 replies; 14+ messages in thread
From: Patchwork @ 2023-12-08 13:16 UTC (permalink / raw)
To: Bommu, Krishnaiah; +Cc: igt-dev
== Series Details ==
Series: RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6)
URL : https://patchwork.freedesktop.org/series/127364/
State : failure
== Summary ==
Applying: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
Applying: drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
Using index info to reconstruct a base tree...
M tests/intel/xe_waitfence.c
Patch failed at 0002 drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen
2023-12-08 12:59 ` Bommu, Krishnaiah
@ 2023-12-08 13:48 ` Rodrigo Vivi
0 siblings, 0 replies; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-08 13:48 UTC (permalink / raw)
To: Bommu, Krishnaiah; +Cc: igt-dev@lists.freedesktop.org
On Fri, Dec 08, 2023 at 07:59:34AM -0500, Bommu, Krishnaiah wrote:
>
>
> > -----Original Message-----
> > From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Sent: Friday, December 8, 2023 11:31 AM
> > To: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>
> > Cc: igt-dev@lists.freedesktop.org
> > Subject: Re: [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl
> > when exec_queue reset happen
> >
> > On Fri, Dec 08, 2023 at 09:48:25AM +0530, Bommu Krishnaiah wrote:
> > > Skipping the GPU mapping(vm_bind) for object, so that exec_queue
> >
> > 'skipping' sounds like you are not waiting at all...
> >
> > > reset will happen and xe_wait_ufence will end return EIO not ETIME
> > >
> > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > ---
> > > tests/intel/xe_waitfence.c | 82
> > > ++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 82 insertions(+)
> > >
> > > diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > > index 4e94403a3..bd95fa1f8 100644
> > > --- a/tests/intel/xe_waitfence.c
> > > +++ b/tests/intel/xe_waitfence.c
> > > @@ -149,6 +149,9 @@ waitfence(int fd, enum waittype wt)
> > > *
> > > * SUBTEST: invalid-engine
> > > * Description: Check query with invalid engine info returns expected
> > > error code
> > > + *
> > > + * SUBTEST: invalid-exec_queue-wait
> >
> > 'invalid wait' sounded to me like a negative test with invalid timeout.
> >
> > So I believe we need to rephrase some stuff here, but the test looks right.
> >
>
> Commit message:
> drm-uapi/xe: Don't wait on user_fence during exec queue reset
>
> don't wait till timeout on user fence when exec_queue reset is detected
> and return return proper error code
>
>
> root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools# git diff
> diff --git a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> index bd95fa1f8..9a2c854c1 100644
> --- a/tests/intel/xe_waitfence.c
> +++ b/tests/intel/xe_waitfence.c
> @@ -150,8 +150,8 @@ waitfence(int fd, enum waittype wt)
> * SUBTEST: invalid-engine
> * Description: Check query with invalid engine info returns expected error code
> *
> - * SUBTEST: invalid-exec_queue-wait
> - * Description: Check xe_wait_ufence will return expected error code while exec_queue reset happen
> + * SUBTEST: exec_queue-reset-wait
> + * Description: Don't wait till timeout on user fence when exec_queue reset is detected and return return proper error
> */
>
> static void
> @@ -228,7 +228,7 @@ invalid_engine(int fd)
> }
>
> static void
> -invalid_exec_queue_wait(int fd)
> +exec_queue_reset_wait(int fd)
> {
> uint32_t bo, b;
> uint64_t batch_offset;
> @@ -291,8 +291,8 @@ invalid_exec_queue_wait(int fd)
> xe_exec(fd, &exec);
>
> /**
> - * Skipping the GPU mapping(vm_bind) for object, so that exec_queue
> - * reset will happen and xe_wait_ufence will end return EIO not ETIME
> + * Don't do the GPU mapping(vm_bind) for object, so that exec_queue
> + * reset will happen and xe_wait_ufence will return EIO not ETIME
> */
> do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
>
> @@ -329,8 +329,8 @@ igt_main
> igt_subtest("invalid-engine")
> invalid_engine(fd);
>
> - igt_subtest("invalid-exec_queue-wait")
> - invalid_exec_queue_wait(fd);
> + igt_subtest("exec_queue-reset-wait")
> + exec_queue_reset_wait(fd);
>
> igt_fixture
> drm_close_driver(fd);
> root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
> root@kbommu-desk:/home/kbommu/xe_public/igt-gpu-tools#
>
> This is what I can think of now, please suggest anything need to be changed
yeap, this works for me:
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
>
> Regards,
> Krishna
>
> > > + * Description: Check xe_wait_ufence will return expected error code
> > > + while exec_queue reset happen
> > > */
> > >
> > > static void
> > > @@ -224,6 +227,82 @@ invalid_engine(int fd)
> > > do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait,
> > EFAULT); }
> > >
> > > +static void
> > > +invalid_exec_queue_wait(int fd)
> > > +{
> > > + uint32_t bo, b;
> > > + uint64_t batch_offset;
> > > + uint64_t batch_addr;
> > > + uint64_t sdi_offset;
> > > + uint64_t sdi_addr;
> > > + uint64_t addr = 0x1a0000;
> > > +
> > > + struct {
> > > + uint32_t batch[16];
> > > + uint64_t pad;
> > > + uint64_t vm_sync;
> > > + uint64_t exec_sync;
> > > + uint32_t data;
> > > + } *data;
> > > +
> > > +#define USER_FENCE_VALUE 0xdeadbeefdeadbeefull
> > > + struct drm_xe_sync sync[1] = {
> > > + { .flags = DRM_XE_SYNC_TYPE_USER_FENCE |
> > DRM_XE_SYNC_FLAG_SIGNAL,
> > > + .timeline_value = USER_FENCE_VALUE },
> > > + };
> > > +
> > > + struct drm_xe_exec exec = {
> > > + .num_batch_buffer = 1,
> > > + .num_syncs = 1,
> > > + .syncs = to_user_pointer(sync),
> > > + };
> > > +
> > > + uint32_t vm = xe_vm_create(fd,
> > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > + uint32_t exec_queue = xe_exec_queue_create_class(fd, vm,
> > DRM_XE_ENGINE_CLASS_COPY);
> > > + struct drm_xe_wait_user_fence1 wait = {
> > > + .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > + .flags = 0,
> > > + .value = 0xc0ffee,
> > > + .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > + .timeout = -1,
> > > + .exec_queue_id = exec_queue,
> > > + };
> > > +
> > > + bo = xe_bo_create(fd, vm, 0x40000, vram_if_possible(fd, 0), 0);
> > > + data = xe_bo_map(fd, bo, 0x40000);
> > > +
> > > + batch_offset = (char *)&data[0].batch - (char *)data;
> > > + batch_addr = addr + batch_offset;
> > > + sdi_offset = (char *)&data[0].data - (char *)data;
> > > + sdi_addr = addr + sdi_offset;
> > > +
> > > + b = 0;
> > > + data[0].batch[b++] = MI_STORE_DWORD_IMM_GEN4;
> > > + data[0].batch[b++] = sdi_addr;
> > > + data[0].batch[b++] = sdi_addr >> 32;
> > > + data[0].batch[b++] = 0xc0ffee;
> > > + data[0].batch[b++] = MI_BATCH_BUFFER_END;
> > > + igt_assert(b <= ARRAY_SIZE(data[0].batch));
> > > +
> > > + wait.addr = to_user_pointer(&data[0].exec_sync);
> > > + exec.exec_queue_id = exec_queue;
> > > + exec.address = batch_addr;
> > > +
> > > + xe_exec(fd, &exec);
> > > +
> > > + /**
> > > + * Skipping the GPU mapping(vm_bind) for object, so that
> > exec_queue
> > > + * reset will happen and xe_wait_ufence will end return EIO not
> > ETIME
> > > + */
> > > + do_ioctl_err(fd, DRM_IOCTL_XE_WAIT_USER_FENCE, &wait, EIO);
> > > +
> > > + xe_exec_queue_destroy(fd, exec_queue);
> > > +
> > > + if (bo) {
> > > + munmap(data, 0x40000);
> > > + gem_close(fd, bo);
> > > + }
> > > +}
> > >
> > > igt_main
> > > {
> > > @@ -250,6 +329,9 @@ igt_main
> > > igt_subtest("invalid-engine")
> > > invalid_engine(fd);
> > >
> > > + igt_subtest("invalid-exec_queue-wait")
> > > + invalid_exec_queue_wait(fd);
> > > +
> > > igt_fixture
> > > drm_close_driver(fd);
> > > }
> > > --
> > > 2.25.1
> > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-08 13:05 ` Matthew Brost
@ 2023-12-11 5:36 ` Bommu, Krishnaiah
2023-12-11 7:03 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-11 5:36 UTC (permalink / raw)
To: Brost, Matthew, Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Brost, Matthew <matthew.brost@intel.com>
> Sent: Friday, December 8, 2023 6:35 PM
> To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> dev@lists.freedesktop.org
> Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> drm_xe_wait_user_fence structure
>
> On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > remove the num_engines/instances members from
> drm_xe_wait_user_fence
> > > structure and add a exec_queue_id member
> > >
> > > Right now this is only checking if the engine list is sane and
> > > nothing else. In the end every operation with this IOCTL is a soft check.
> > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > >
> > > exec_queue_id member will help to user space to get proper error
> > > code from kernel while in exec_queue reset
> > >
> > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > ---
> > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > lib/xe/xe_ioctl.h | 9 +++------
> > > tests/intel/xe_evict.c | 4 ++--
> > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > tests/intel/xe_exec_reset.c | 6 +++---
> > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > >
> > > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > > index 590f7b7af..fd06e4920 100644
> > > --- a/include/drm-uapi/xe_drm.h
> > > +++ b/include/drm-uapi/xe_drm.h
> > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > * It is returned as part of the @drm_xe_engine, but it also is used as
> > > * the input of engine selection for both @drm_xe_exec_queue_create
> and
> > > * @drm_xe_query_engine_cycles
> > > - *
> > > */
> > > struct drm_xe_engine_class_instance {
> > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > */
> > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > + /** @engine_class: engine class id */
> > > __u16 engine_class;
> > > -
> > > + /** @engine_instance: engine instance id */
> > > __u16 engine_instance;
> > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > __u16 gt_id;
> > > /** @pad: MBZ */
> > > __u16 pad;
> > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > *
> > > * Note: For userptr and externally imported dma-buf the kernel
> expects
> > > * either 1WAY or 2WAY for the @pat_index.
> > > + *
> > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD
> restrictions
> > > + * on the @pat_index. For such mappings there is no actual memory
> being
> > > + * mapped (the address in the PTE is invalid), so the various PAT
> memory
> > > + * attributes likely do not apply. Simply leaving as zero is one
> > > + * option (still a valid pat_index).
> > > */
> > > __u16 pat_index;
> > >
> > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > /** @op: wait operation (type of comparison) */
> > > __u16 op;
> > >
> > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /* e.g. Wait
> on VM bind */
> > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > /** @flags: wait flags */
> > > __u16 flags;
> > >
> > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > */
> > > __s64 timeout;
> > >
> > > - /**
> > > - * @num_engines: number of engine instances to wait on, must be
> zero
> > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > - */
> > > - __u64 num_engines;
> > > + /** @exec_queue_id: exec_queue_id returned from
> xe_exec_queue_create_ioctl */
> > > + __u32 exec_queue_id;
> > >
> > > - /**
> > > - * @instances: user pointer to array of
> drm_xe_engine_class_instance to
> > > - * wait on, must be NULL when
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > - */
> > > - __u64 instances;
> > > + /** @pad2: MBZ */
> > > + __u32 pad2;
> > >
> > > /** @reserved: Reserved */
> > > __u64 reserved[2];
> > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > b29ca40ad..0a1ddc86b 100644
> > > --- a/lib/xe/xe_ioctl.c
> > > +++ b/lib/xe/xe_ioctl.c
> > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue,
> uint64_t addr)
> > > * @fd: xe device fd
> > > * @addr: address of value to compare
> > > * @value: expected value (equal) in @address
> > > - * @eci: engine class instance
> > > + * @exec_queue: exec_queue id
> > > * @timeout: pointer to time to wait in nanoseconds
> > > *
> > > * Function compares @value with memory pointed by @addr until they
> are equal.
> > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> exec_queue, uint64_t addr)
> > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > */
> > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t *timeout)
> > > + uint32_t exec_queue, int64_t *timeout)
> > > {
> > > struct drm_xe_wait_user_fence wait = {
> > > .addr = to_user_pointer(addr),
> > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > + .flags = 0,
> > > .value = value,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > - .num_engines = eci ? 1 :0,
> > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > + .exec_queue_id = exec_queue,
> > > };
> > >
> > > igt_assert(timeout);
> > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * @fd: xe device fd
> > > * @addr: address of value to compare
> > > * @value: expected value (equal) in @address
> > > - * @eci: engine class instance
> > > + * @exec_queue: exec_queue id
> > > * @timeout: time to wait in nanoseconds
> > > *
> > > * Function compares @value with memory pointed by @addr until they
> are equal.
> > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > */
> > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout)
> > > + uint32_t exec_queue, int64_t timeout)
> > > {
> > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue,
> > > +&timeout), 0);
> > > return timeout;
> > > }
> > >
> > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * @fd: xe device fd
> > > * @addr: address of value to compare
> > > * @value: expected value (equal) in @address
> > > - * @eci: engine class instance
> > > + * @exec_queue: exec_queue id
> > > * @timeout: absolute time when wait expire
> > > *
> > > * Function compares @value with memory pointed by @addr until they
> are equal.
> > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> uint64_t value,
> > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > */
> > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout)
> > > + uint32_t exec_queue, int64_t timeout)
> > > {
> > > struct drm_xe_wait_user_fence wait = {
> > > .addr = to_user_pointer(addr),
> > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > + .flags = !exec_queue ?
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> >
> > To be on the safe side we should probably change the signature of this
> > function to explicitly receive this flag instead of relying on the invalid
> exec_queue_id number.
> >
> > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP was
> used here...
> >
> > Matt, thoughts?
> >
>
> Agree with everything here, !exec_queue really has nothing to do with
> setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So yes,
> please change this function to accpt a flags field.
>
> Everything else LGTM.
>
> Matt
I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not thing to do with exec_queue, I will change this function to accept flag, but DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is related to "no_engines" I removed from kernel side also https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
Matt please let me know your insights on DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
Regards,
Krishna.
>
> > Everything else in the patch looks good to me.
> >
> > > .value = value,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = timeout,
> > > - .num_engines = eci ? 1 : 0,
> > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > + .exec_queue_id = exec_queue,
> > > };
> > > struct timespec ts;
> > >
> > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > bd660bd27..9c0b4b9bc 100644
> > > --- a/lib/xe/xe_ioctl.h
> > > +++ b/lib/xe/xe_ioctl.h
> > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue,
> uint64_t addr,
> > > struct drm_xe_sync *sync, uint32_t num_syncs); void
> > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t *timeout);
> > > + uint32_t exec_queue, int64_t *timeout);
> > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout);
> > > + uint32_t exec_queue, int64_t timeout);
> > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout);
> > > + uint32_t exec_queue, int64_t timeout);
> > > void xe_force_gt_reset(int fd, int gt);
> > >
> > > #endif /* XE_IOCTL_H */
> > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c index
> > > 89dc46fae..a37c2d97a 100644
> > > --- a/tests/intel/xe_evict.c
> > > +++ b/tests/intel/xe_evict.c
> > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > }
> > > #define TWENTY_SEC MS_TO_NS(20000)
> > > xe_wait_ufence(fd, &data[i].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, TWENTY_SEC);
> > > + bind_exec_queues[0], TWENTY_SEC);
> > > }
> > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > (char *)data;
> > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > data = xe_bo_map(fd, __bo,
> > > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, TWENTY_SEC);
> > > + exec_queues[i], TWENTY_SEC);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > }
> > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff --git
> > > a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > > index 79ff65e89..2448d49bc 100644
> > > --- a/tests/intel/xe_exec_balancer.c
> > > +++ b/tests/intel/xe_exec_balancer.c
> > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> n_exec_queues, int n_execs,
> > > bo_size, sync, 1);
> > >
> > > #define ONE_SEC MS_TO_NS(1000)
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +ONE_SEC);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > >
> > > if (flags & REBIND && i + 1 != n_execs) {
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + exec_queues[e], ONE_SEC);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> NULL,
> > > 0);
> > >
> > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> n_exec_queues, int n_execs,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + 0, ONE_SEC);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> n_exec_queues, int n_execs,
> > > * an invalidate.
> > > */
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + USER_FENCE_VALUE,
> exec_queues[e],
> > > + ONE_SEC);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > /*
> > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > >
> > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > for (i = j; i < n_execs; i++)
> > > - xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE, NULL,
> > > - ONE_SEC);
> > > + xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > + exec_queues[i], ONE_SEC);
> > >
> > > /* Wait for all execs to complete */
> > > if (flags & INVALIDATE)
> > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +ONE_SEC);
> > >
> > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > i < n_execs; i++)
> > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > b/tests/intel/xe_exec_compute_mode.c
> > > index 7d3004d65..749a9b634 100644
> > > --- a/tests/intel/xe_exec_compute_mode.c
> > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> ONE_SEC;
> > >
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], fence_timeout);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -198,7 +198,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > if (flags & REBIND && i + 1 != n_execs) {
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + exec_queues[e], fence_timeout);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> 0,
> > > addr, bo_size, NULL, 0);
> > >
> > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + bind_exec_queues[e], fence_timeout);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > * an invalidate.
> > > */
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL,
> > > + USER_FENCE_VALUE,
> exec_queues[e],
> > > fence_timeout);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > @@ -257,8 +257,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > for (i = j; i < n_execs; i++)
> > > - xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > + exec_queues[i], fence_timeout);
> > >
> > > /* Wait for all execs to complete */
> > > if (flags & INVALIDATE)
> > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> bo_size,
> > > sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], fence_timeout);
> > >
> > > for (i = j; i < n_execs; i++)
> > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > a/tests/intel/xe_exec_fault_mode.c
> > > b/tests/intel/xe_exec_fault_mode.c
> > > index ee7cbb604..d76b3a056 100644
> > > --- a/tests/intel/xe_exec_fault_mode.c
> > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > }
> > >
> > > #define ONE_SEC MS_TO_NS(1000)
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], ONE_SEC);
> > > data[0].vm_sync = 0;
> > >
> > > if (flags & PREFETCH) {
> > > /* Should move to system memory */
> > > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0,
> addr,
> > > bo_size, sync, 1, 0);
> > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > - ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > + bind_exec_queues[0], ONE_SEC);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > >
> > > if (flags & REBIND && i + 1 != n_execs) {
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + exec_queues[e], ONE_SEC);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> 0,
> > > addr, bo_size, NULL, 0);
> > >
> > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, ONE_SEC);
> > > + bind_exec_queues[e], ONE_SEC);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > * an invalidate.
> > > */
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + USER_FENCE_VALUE,
> exec_queues[e],
> > > + ONE_SEC);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > /*
> > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > for (i = j; i < n_execs; i++)
> > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > > + USER_FENCE_VALUE, exec_queues[i],
> ONE_SEC);
> > > }
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> bo_size,
> > > sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > + bind_exec_queues[0], ONE_SEC);
> > >
> > > if (!(flags & INVALID_FAULT)) {
> > > for (i = j; i < n_execs; i++)
> > > diff --git a/tests/intel/xe_exec_reset.c
> > > b/tests/intel/xe_exec_reset.c index 094b34896..398c90af5 100644
> > > --- a/tests/intel/xe_exec_reset.c
> > > +++ b/tests/intel/xe_exec_reset.c
> > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > >
> > > #define THREE_SEC MS_TO_NS(3000)
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> THREE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +THREE_SEC);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct
> drm_xe_engine_class_instance *eci,
> > > int err;
> > >
> > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > - NULL, &timeout);
> > > + exec_queues[i], &timeout);
> > > if (flags & GT_RESET)
> > > /* exec races with reset: may timeout or complete */
> > > igt_assert(err == -ETIME || !err); @@ -631,7 +631,7
> @@
> > > test_compute_mode(int fd, struct drm_xe_engine_class_instance *eci,
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> THREE_SEC);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +THREE_SEC);
> > >
> > > if (!(flags & GT_RESET)) {
> > > for (i = 1; i < n_execs; i++)
> > > diff --git a/tests/intel/xe_exec_threads.c
> > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9 100644
> > > --- a/tests/intel/xe_exec_threads.c
> > > +++ b/tests/intel/xe_exec_threads.c
> > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> > > addr, uint64_t userptr,
> > >
> > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC : THREE_SEC;
> > >
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +fence_timeout);
> > > data[0].vm_sync = 0;
> > >
> > > for (i = 0; i < n_execs; i++) {
> > > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > for (j = i - 0x20; j <= i; ++j)
> > > xe_wait_ufence(fd, &data[j].exec_sync,
> > > USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + exec_queues[e], fence_timeout);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > NULL, 0);
> > >
> > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > addr, bo_size, sync,
> > > 1);
> > > xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + 0, fence_timeout);
> > > data[0].vm_sync = 0;
> > > }
> > >
> > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > > xe_wait_ufence(fd,
> &data[j].exec_sync,
> > > USER_FENCE_VALUE,
> > > - NULL, fence_timeout);
> > > + exec_queues[e],
> > > + fence_timeout);
> > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > } else if (i * 2 != n_execs) {
> > > /*
> > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> addr, uint64_t userptr,
> > > j = flags & INVALIDATE ?
> > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > for (i = j; i < n_execs; i++)
> > > - xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE, NULL,
> > > - fence_timeout);
> > > + xe_wait_ufence(fd, &data[i].exec_sync,
> USER_FENCE_VALUE,
> > > + exec_queues[e], fence_timeout);
> > >
> > > /* Wait for all execs to complete */
> > > if (flags & INVALIDATE)
> > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm, uint64_t
> > > addr, uint64_t userptr,
> > >
> > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> fence_timeout);
> > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > +fence_timeout);
> > >
> > > for (i = j; i < n_execs; i++)
> > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c index
> > > 3be987954..4e94403a3 100644
> > > --- a/tests/intel/xe_waitfence.c
> > > +++ b/tests/intel/xe_waitfence.c
> > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > uint32_t bo, uint64_t offset, }
> > >
> > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr, uint64_t
> value,
> > > - struct drm_xe_engine_class_instance *eci,
> > > - int64_t timeout)
> > > + uint32_t exec_queue, int64_t timeout)
> > > {
> > > struct drm_xe_wait_user_fence wait = {
> > > .addr = to_user_pointer(addr),
> > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > + .flags = !exec_queue ? 0 :
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > .value = value,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = timeout,
> > > - .num_engines = eci ? 1 : 0,
> > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > + .exec_queue_id = exec_queue,
> > > };
> > > struct timespec ts;
> > >
> > > - igt_assert(eci);
> > > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE,
> &wait), 0);
> > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > >
> > > @@ -82,7 +79,7 @@ enum waittype {
> > > static void
> > > waitfence(int fd, enum waittype wt) {
> > > - struct drm_xe_engine *engine = NULL;
> > > + uint32_t exec_queue;
> > > struct timespec ts;
> > > int64_t current, signalled;
> > > uint32_t bo_1;
> > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > >
> > > if (wt == RELTIME) {
> > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> MS_TO_NS(10));
> > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> MS_TO_NS(10));
> > > igt_debug("wait type: RELTIME - timeout: %ld, timeout left:
> %ld\n",
> > > MS_TO_NS(10), timeout);
> > > } else if (wt == ENGINE) {
> > > - engine = xe_engine(fd, 1);
> > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > +DRM_XE_ENGINE_CLASS_COPY);
> > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > timeout = current + MS_TO_NS(10);
> > > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> &engine->instance, timeout);
> > > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> exec_queue,
> > > +timeout);
> > > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > > ", signalled: %" PRId64
> > > ", elapsed: %" PRId64 "\n",
> > > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > timeout = current + MS_TO_NS(10);
> > > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7,
> NULL, timeout);
> > > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0,
> > > +timeout);
> > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > ", signalled: %" PRId64
> > > ", elapsed: %" PRId64 "\n",
> > > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > > .value = 1,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = 1,
> > > - .num_engines = 0,
> > > - .instances = 0,
> > > + .exec_queue_id = 0,
> > > };
> > >
> > > uint32_t vm = xe_vm_create(fd,
> > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8 +212,7
> @@ invalid_engine(int fd)
> > > .value = 1,
> > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > .timeout = -1,
> > > - .num_engines = 1,
> > > - .instances = 0,
> > > + .exec_queue_id = 0,
> > > };
> > >
> > > uint32_t vm = xe_vm_create(fd,
> > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > --
> > > 2.25.1
> > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-11 5:36 ` Bommu, Krishnaiah
@ 2023-12-11 7:03 ` Bommu, Krishnaiah
2023-12-11 18:58 ` Rodrigo Vivi
0 siblings, 1 reply; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-11 7:03 UTC (permalink / raw)
To: Brost, Matthew, Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Bommu, Krishnaiah
> Sent: Monday, December 11, 2023 11:06 AM
> To: Brost, Matthew <matthew.brost@intel.com>; Vivi, Rodrigo
> <rodrigo.vivi@intel.com>
> Cc: igt-dev@lists.freedesktop.org
> Subject: RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> drm_xe_wait_user_fence structure
>
>
>
> > -----Original Message-----
> > From: Brost, Matthew <matthew.brost@intel.com>
> > Sent: Friday, December 8, 2023 6:35 PM
> > To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> > dev@lists.freedesktop.org
> > Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > drm_xe_wait_user_fence structure
> >
> > On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > > remove the num_engines/instances members from
> > drm_xe_wait_user_fence
> > > > structure and add a exec_queue_id member
> > > >
> > > > Right now this is only checking if the engine list is sane and
> > > > nothing else. In the end every operation with this IOCTL is a soft check.
> > > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > > >
> > > > exec_queue_id member will help to user space to get proper error
> > > > code from kernel while in exec_queue reset
> > > >
> > > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > > ---
> > > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > > lib/xe/xe_ioctl.h | 9 +++------
> > > > tests/intel/xe_evict.c | 4 ++--
> > > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > > tests/intel/xe_exec_reset.c | 6 +++---
> > > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > > >
> > > > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > > > index 590f7b7af..fd06e4920 100644
> > > > --- a/include/drm-uapi/xe_drm.h
> > > > +++ b/include/drm-uapi/xe_drm.h
> > > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > > * It is returned as part of the @drm_xe_engine, but it also is used as
> > > > * the input of engine selection for both
> > > > @drm_xe_exec_queue_create
> > and
> > > > * @drm_xe_query_engine_cycles
> > > > - *
> > > > */
> > > > struct drm_xe_engine_class_instance {
> > > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > > */
> > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > > + /** @engine_class: engine class id */
> > > > __u16 engine_class;
> > > > -
> > > > + /** @engine_instance: engine instance id */
> > > > __u16 engine_instance;
> > > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > > __u16 gt_id;
> > > > /** @pad: MBZ */
> > > > __u16 pad;
> > > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > > *
> > > > * Note: For userptr and externally imported dma-buf the kernel
> > expects
> > > > * either 1WAY or 2WAY for the @pat_index.
> > > > + *
> > > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD
> > restrictions
> > > > + * on the @pat_index. For such mappings there is no actual
> > > > +memory
> > being
> > > > + * mapped (the address in the PTE is invalid), so the various
> > > > +PAT
> > memory
> > > > + * attributes likely do not apply. Simply leaving as zero is one
> > > > + * option (still a valid pat_index).
> > > > */
> > > > __u16 pat_index;
> > > >
> > > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > > /** @op: wait operation (type of comparison) */
> > > > __u16 op;
> > > >
> > > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /*
> e.g. Wait
> > on VM bind */
> > > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > > /** @flags: wait flags */
> > > > __u16 flags;
> > > >
> > > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > > */
> > > > __s64 timeout;
> > > >
> > > > - /**
> > > > - * @num_engines: number of engine instances to wait on, must be
> > zero
> > > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > - */
> > > > - __u64 num_engines;
> > > > + /** @exec_queue_id: exec_queue_id returned from
> > xe_exec_queue_create_ioctl */
> > > > + __u32 exec_queue_id;
> > > >
> > > > - /**
> > > > - * @instances: user pointer to array of
> > drm_xe_engine_class_instance to
> > > > - * wait on, must be NULL when
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > - */
> > > > - __u64 instances;
> > > > + /** @pad2: MBZ */
> > > > + __u32 pad2;
> > > >
> > > > /** @reserved: Reserved */
> > > > __u64 reserved[2];
> > > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > > b29ca40ad..0a1ddc86b 100644
> > > > --- a/lib/xe/xe_ioctl.c
> > > > +++ b/lib/xe/xe_ioctl.c
> > > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue,
> > uint64_t addr)
> > > > * @fd: xe device fd
> > > > * @addr: address of value to compare
> > > > * @value: expected value (equal) in @address
> > > > - * @eci: engine class instance
> > > > + * @exec_queue: exec_queue id
> > > > * @timeout: pointer to time to wait in nanoseconds
> > > > *
> > > > * Function compares @value with memory pointed by @addr until
> > > > they
> > are equal.
> > > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> > exec_queue, uint64_t addr)
> > > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > > */
> > > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t *timeout)
> > > > + uint32_t exec_queue, int64_t *timeout)
> > > > {
> > > > struct drm_xe_wait_user_fence wait = {
> > > > .addr = to_user_pointer(addr),
> > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > > + .flags = 0,
> > > > .value = value,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > - .num_engines = eci ? 1 :0,
> > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > + .exec_queue_id = exec_queue,
> > > > };
> > > >
> > > > igt_assert(timeout);
> > > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > uint64_t value,
> > > > * @fd: xe device fd
> > > > * @addr: address of value to compare
> > > > * @value: expected value (equal) in @address
> > > > - * @eci: engine class instance
> > > > + * @exec_queue: exec_queue id
> > > > * @timeout: time to wait in nanoseconds
> > > > *
> > > > * Function compares @value with memory pointed by @addr until
> > > > they
> > are equal.
> > > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > uint64_t value,
> > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > */
> > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout)
> > > > + uint32_t exec_queue, int64_t timeout)
> > > > {
> > > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue,
> > > > +&timeout), 0);
> > > > return timeout;
> > > > }
> > > >
> > > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> > uint64_t value,
> > > > * @fd: xe device fd
> > > > * @addr: address of value to compare
> > > > * @value: expected value (equal) in @address
> > > > - * @eci: engine class instance
> > > > + * @exec_queue: exec_queue id
> > > > * @timeout: absolute time when wait expire
> > > > *
> > > > * Function compares @value with memory pointed by @addr until
> > > > they
> > are equal.
> > > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > *addr,
> > uint64_t value,
> > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > */
> > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout)
> > > > + uint32_t exec_queue, int64_t timeout)
> > > > {
> > > > struct drm_xe_wait_user_fence wait = {
> > > > .addr = to_user_pointer(addr),
> > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > + .flags = !exec_queue ?
> > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > >
> > > To be on the safe side we should probably change the signature of
> > > this function to explicitly receive this flag instead of relying on
> > > the invalid
> > exec_queue_id number.
> > >
> > > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP
> was
> > used here...
> > >
> > > Matt, thoughts?
> > >
> >
> > Agree with everything here, !exec_queue really has nothing to do with
> > setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So
> yes,
> > please change this function to accpt a flags field.
> >
> > Everything else LGTM.
> >
> > Matt
>
> I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not thing
> to do with exec_queue, I will change this function to accept flag, but
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is related to
> "no_engines" I removed from kernel side also
> https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
>
> Matt please let me know your insights on
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
>
> Regards,
> Krishna.
> >
> > > Everything else in the patch looks good to me.
> > >
> > > > .value = value,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = timeout,
> > > > - .num_engines = eci ? 1 : 0,
> > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > + .exec_queue_id = exec_queue,
> > > > };
> > > > struct timespec ts;
> > > >
> > > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > > bd660bd27..9c0b4b9bc 100644
> > > > --- a/lib/xe/xe_ioctl.h
> > > > +++ b/lib/xe/xe_ioctl.h
> > > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue,
> > uint64_t addr,
> > > > struct drm_xe_sync *sync, uint32_t num_syncs); void
> > > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t *timeout);
> > > > + uint32_t exec_queue, int64_t *timeout);
> > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout);
> > > > + uint32_t exec_queue, int64_t timeout);
> > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout);
> > > > + uint32_t exec_queue, int64_t timeout);
> > > > void xe_force_gt_reset(int fd, int gt);
> > > >
> > > > #endif /* XE_IOCTL_H */
> > > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c index
> > > > 89dc46fae..a37c2d97a 100644
> > > > --- a/tests/intel/xe_evict.c
> > > > +++ b/tests/intel/xe_evict.c
> > > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > }
> > > > #define TWENTY_SEC MS_TO_NS(20000)
> > > > xe_wait_ufence(fd, &data[i].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, TWENTY_SEC);
> > > > + bind_exec_queues[0], TWENTY_SEC);
> > > > }
> > > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > > (char *)data;
> > > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > data = xe_bo_map(fd, __bo,
> > > > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, TWENTY_SEC);
> > > > + exec_queues[i], TWENTY_SEC);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > }
> > > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff --git
> > > > a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > > > index 79ff65e89..2448d49bc 100644
> > > > --- a/tests/intel/xe_exec_balancer.c
> > > > +++ b/tests/intel/xe_exec_balancer.c
> > > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> > n_exec_queues, int n_execs,
> > > > bo_size, sync, 1);
> > > >
> > > > #define ONE_SEC MS_TO_NS(1000)
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > >
> > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + exec_queues[e], ONE_SEC);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > NULL,
> > > > 0);
> > > >
> > > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> > n_exec_queues, int n_execs,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + 0, ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> > n_exec_queues, int n_execs,
> > > > * an invalidate.
> > > > */
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + USER_FENCE_VALUE,
> > exec_queues[e],
> > > > + ONE_SEC);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) {
> > > > /*
> > > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > >
> > > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > + exec_queues[i], ONE_SEC);
Need to change exec_queues[i] index to "i % n_exec_queues", need to change other files also, I will change
exec_queues[i] => exec_queues[i % n_exec_queues]
Regards,
Krishna
> > > >
> > > > /* Wait for all execs to complete */
> > > > if (flags & INVALIDATE)
> > > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +ONE_SEC);
> > > >
> > > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > > i < n_execs; i++)
> > > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > > b/tests/intel/xe_exec_compute_mode.c
> > > > index 7d3004d65..749a9b634 100644
> > > > --- a/tests/intel/xe_exec_compute_mode.c
> > > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > >
> > > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> > ONE_SEC;
> > > >
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], fence_timeout);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -198,7 +198,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > >
> > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + exec_queues[e], fence_timeout);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > 0,
> > > > addr, bo_size, NULL, 0);
> > > >
> > > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + bind_exec_queues[e], fence_timeout);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > * an invalidate.
> > > > */
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL,
> > > > + USER_FENCE_VALUE,
> > exec_queues[e],
> > > > fence_timeout);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) { @@ -257,8 +257,8 @@
> > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > >
> > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > + exec_queues[i], fence_timeout);
> > > >
> > > > /* Wait for all execs to complete */
> > > > if (flags & INVALIDATE)
> > > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > bo_size,
> > > > sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], fence_timeout);
> > > >
> > > > for (i = j; i < n_execs; i++)
> > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > a/tests/intel/xe_exec_fault_mode.c
> > > > b/tests/intel/xe_exec_fault_mode.c
> > > > index ee7cbb604..d76b3a056 100644
> > > > --- a/tests/intel/xe_exec_fault_mode.c
> > > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > }
> > > >
> > > > #define ONE_SEC MS_TO_NS(1000)
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > >
> > > > if (flags & PREFETCH) {
> > > > /* Should move to system memory */
> > > > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0,
> > addr,
> > > > bo_size, sync, 1, 0);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > >
> > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + exec_queues[e], ONE_SEC);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > 0,
> > > > addr, bo_size, NULL, 0);
> > > >
> > > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, ONE_SEC);
> > > > + bind_exec_queues[e], ONE_SEC);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > * an invalidate.
> > > > */
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + USER_FENCE_VALUE,
> > exec_queues[e],
> > > > + ONE_SEC);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) {
> > > > /*
> > > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > > > + USER_FENCE_VALUE, exec_queues[i],
> > ONE_SEC);
> > > > }
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > bo_size,
> > > > sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > ONE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > + bind_exec_queues[0], ONE_SEC);
> > > >
> > > > if (!(flags & INVALID_FAULT)) {
> > > > for (i = j; i < n_execs; i++)
> > > > diff --git a/tests/intel/xe_exec_reset.c
> > > > b/tests/intel/xe_exec_reset.c index 094b34896..398c90af5 100644
> > > > --- a/tests/intel/xe_exec_reset.c
> > > > +++ b/tests/intel/xe_exec_reset.c
> > > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > > >
> > > > #define THREE_SEC MS_TO_NS(3000)
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > THREE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +THREE_SEC);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct
> > drm_xe_engine_class_instance *eci,
> > > > int err;
> > > >
> > > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, &timeout);
> > > > + exec_queues[i], &timeout);
> > > > if (flags & GT_RESET)
> > > > /* exec races with reset: may timeout or complete */
> > > > igt_assert(err == -ETIME || !err); @@ -631,7 +631,7
> > @@
> > > > test_compute_mode(int fd, struct drm_xe_engine_class_instance
> > > > *eci,
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > THREE_SEC);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +THREE_SEC);
> > > >
> > > > if (!(flags & GT_RESET)) {
> > > > for (i = 1; i < n_execs; i++)
> > > > diff --git a/tests/intel/xe_exec_threads.c
> > > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9 100644
> > > > --- a/tests/intel/xe_exec_threads.c
> > > > +++ b/tests/intel/xe_exec_threads.c
> > > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t addr, uint64_t userptr,
> > > >
> > > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC :
> > > > THREE_SEC;
> > > >
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +fence_timeout);
> > > > data[0].vm_sync = 0;
> > > >
> > > > for (i = 0; i < n_execs; i++) {
> > > > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > for (j = i - 0x20; j <= i; ++j)
> > > > xe_wait_ufence(fd, &data[j].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + exec_queues[e], fence_timeout);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > > NULL, 0);
> > > >
> > > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > addr, bo_size, sync,
> > > > 1);
> > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + 0, fence_timeout);
> > > > data[0].vm_sync = 0;
> > > > }
> > > >
> > > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > > > xe_wait_ufence(fd,
> > &data[j].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > - NULL, fence_timeout);
> > > > + exec_queues[e],
> > > > + fence_timeout);
> > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > } else if (i * 2 != n_execs) {
> > > > /*
> > > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t
> > addr, uint64_t userptr,
> > > > j = flags & INVALIDATE ?
> > > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > > for (i = j; i < n_execs; i++)
> > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE, NULL,
> > > > - fence_timeout);
> > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > USER_FENCE_VALUE,
> > > > + exec_queues[e], fence_timeout);
> > > >
> > > > /* Wait for all execs to complete */
> > > > if (flags & INVALIDATE)
> > > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > uint64_t addr, uint64_t userptr,
> > > >
> > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > fence_timeout);
> > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > +fence_timeout);
> > > >
> > > > for (i = j; i < n_execs; i++)
> > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c index
> > > > 3be987954..4e94403a3 100644
> > > > --- a/tests/intel/xe_waitfence.c
> > > > +++ b/tests/intel/xe_waitfence.c
> > > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > > uint32_t bo, uint64_t offset, }
> > > >
> > > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr,
> > > > uint64_t
> > value,
> > > > - struct drm_xe_engine_class_instance *eci,
> > > > - int64_t timeout)
> > > > + uint32_t exec_queue, int64_t timeout)
> > > > {
> > > > struct drm_xe_wait_user_fence wait = {
> > > > .addr = to_user_pointer(addr),
> > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > + .flags = !exec_queue ? 0 :
> > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > .value = value,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = timeout,
> > > > - .num_engines = eci ? 1 : 0,
> > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > + .exec_queue_id = exec_queue,
> > > > };
> > > > struct timespec ts;
> > > >
> > > > - igt_assert(eci);
> > > > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE,
> > &wait), 0);
> > > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > > >
> > > > @@ -82,7 +79,7 @@ enum waittype {
> > > > static void
> > > > waitfence(int fd, enum waittype wt) {
> > > > - struct drm_xe_engine *engine = NULL;
> > > > + uint32_t exec_queue;
> > > > struct timespec ts;
> > > > int64_t current, signalled;
> > > > uint32_t bo_1;
> > > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > > >
> > > > if (wt == RELTIME) {
> > > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> > MS_TO_NS(10));
> > > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> > MS_TO_NS(10));
> > > > igt_debug("wait type: RELTIME - timeout: %ld, timeout left:
> > %ld\n",
> > > > MS_TO_NS(10), timeout);
> > > > } else if (wt == ENGINE) {
> > > > - engine = xe_engine(fd, 1);
> > > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > > +DRM_XE_ENGINE_CLASS_COPY);
> > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > timeout = current + MS_TO_NS(10);
> > > > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > &engine->instance, timeout);
> > > > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > exec_queue,
> > > > +timeout);
> > > > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > > > ", signalled: %" PRId64
> > > > ", elapsed: %" PRId64 "\n",
> > > > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > timeout = current + MS_TO_NS(10);
> > > > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7,
> > NULL, timeout);
> > > > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0,
> > > > +timeout);
> > > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > > ", signalled: %" PRId64
> > > > ", elapsed: %" PRId64 "\n",
> > > > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > > > .value = 1,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = 1,
> > > > - .num_engines = 0,
> > > > - .instances = 0,
> > > > + .exec_queue_id = 0,
> > > > };
> > > >
> > > > uint32_t vm = xe_vm_create(fd,
> > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8 +212,7
> > @@ invalid_engine(int fd)
> > > > .value = 1,
> > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > .timeout = -1,
> > > > - .num_engines = 1,
> > > > - .instances = 0,
> > > > + .exec_queue_id = 0,
> > > > };
> > > >
> > > > uint32_t vm = xe_vm_create(fd,
> > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > > --
> > > > 2.25.1
> > > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-11 7:03 ` Bommu, Krishnaiah
@ 2023-12-11 18:58 ` Rodrigo Vivi
2023-12-12 4:23 ` Bommu, Krishnaiah
0 siblings, 1 reply; 14+ messages in thread
From: Rodrigo Vivi @ 2023-12-11 18:58 UTC (permalink / raw)
To: Bommu, Krishnaiah; +Cc: igt-dev@lists.freedesktop.org
On Mon, Dec 11, 2023 at 07:03:47AM +0000, Bommu, Krishnaiah wrote:
>
>
> > -----Original Message-----
> > From: Bommu, Krishnaiah
> > Sent: Monday, December 11, 2023 11:06 AM
> > To: Brost, Matthew <matthew.brost@intel.com>; Vivi, Rodrigo
> > <rodrigo.vivi@intel.com>
> > Cc: igt-dev@lists.freedesktop.org
> > Subject: RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > drm_xe_wait_user_fence structure
> >
> >
> >
> > > -----Original Message-----
> > > From: Brost, Matthew <matthew.brost@intel.com>
> > > Sent: Friday, December 8, 2023 6:35 PM
> > > To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> > > dev@lists.freedesktop.org
> > > Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > > drm_xe_wait_user_fence structure
> > >
> > > On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > > > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > > > remove the num_engines/instances members from
> > > drm_xe_wait_user_fence
> > > > > structure and add a exec_queue_id member
> > > > >
> > > > > Right now this is only checking if the engine list is sane and
> > > > > nothing else. In the end every operation with this IOCTL is a soft check.
> > > > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > > > >
> > > > > exec_queue_id member will help to user space to get proper error
> > > > > code from kernel while in exec_queue reset
> > > > >
> > > > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > > > ---
> > > > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > > > lib/xe/xe_ioctl.h | 9 +++------
> > > > > tests/intel/xe_evict.c | 4 ++--
> > > > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > > > tests/intel/xe_exec_reset.c | 6 +++---
> > > > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > > > >
> > > > > diff --git a/include/drm-uapi/xe_drm.h b/include/drm-uapi/xe_drm.h
> > > > > index 590f7b7af..fd06e4920 100644
> > > > > --- a/include/drm-uapi/xe_drm.h
> > > > > +++ b/include/drm-uapi/xe_drm.h
> > > > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > > > * It is returned as part of the @drm_xe_engine, but it also is used as
> > > > > * the input of engine selection for both
> > > > > @drm_xe_exec_queue_create
> > > and
> > > > > * @drm_xe_query_engine_cycles
> > > > > - *
> > > > > */
> > > > > struct drm_xe_engine_class_instance {
> > > > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > > > */
> > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > > > + /** @engine_class: engine class id */
> > > > > __u16 engine_class;
> > > > > -
> > > > > + /** @engine_instance: engine instance id */
> > > > > __u16 engine_instance;
> > > > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > > > __u16 gt_id;
> > > > > /** @pad: MBZ */
> > > > > __u16 pad;
> > > > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > > > *
> > > > > * Note: For userptr and externally imported dma-buf the kernel
> > > expects
> > > > > * either 1WAY or 2WAY for the @pat_index.
> > > > > + *
> > > > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no KMD
> > > restrictions
> > > > > + * on the @pat_index. For such mappings there is no actual
> > > > > +memory
> > > being
> > > > > + * mapped (the address in the PTE is invalid), so the various
> > > > > +PAT
> > > memory
> > > > > + * attributes likely do not apply. Simply leaving as zero is one
> > > > > + * option (still a valid pat_index).
> > > > > */
> > > > > __u16 pat_index;
> > > > >
> > > > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > > > /** @op: wait operation (type of comparison) */
> > > > > __u16 op;
> > > > >
> > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /*
> > e.g. Wait
> > > on VM bind */
> > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > > > /** @flags: wait flags */
> > > > > __u16 flags;
> > > > >
> > > > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > > > */
> > > > > __s64 timeout;
> > > > >
> > > > > - /**
> > > > > - * @num_engines: number of engine instances to wait on, must be
> > > zero
> > > > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > - */
> > > > > - __u64 num_engines;
> > > > > + /** @exec_queue_id: exec_queue_id returned from
> > > xe_exec_queue_create_ioctl */
> > > > > + __u32 exec_queue_id;
> > > > >
> > > > > - /**
> > > > > - * @instances: user pointer to array of
> > > drm_xe_engine_class_instance to
> > > > > - * wait on, must be NULL when
> > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > - */
> > > > > - __u64 instances;
> > > > > + /** @pad2: MBZ */
> > > > > + __u32 pad2;
> > > > >
> > > > > /** @reserved: Reserved */
> > > > > __u64 reserved[2];
> > > > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > > > b29ca40ad..0a1ddc86b 100644
> > > > > --- a/lib/xe/xe_ioctl.c
> > > > > +++ b/lib/xe/xe_ioctl.c
> > > > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t exec_queue,
> > > uint64_t addr)
> > > > > * @fd: xe device fd
> > > > > * @addr: address of value to compare
> > > > > * @value: expected value (equal) in @address
> > > > > - * @eci: engine class instance
> > > > > + * @exec_queue: exec_queue id
> > > > > * @timeout: pointer to time to wait in nanoseconds
> > > > > *
> > > > > * Function compares @value with memory pointed by @addr until
> > > > > they
> > > are equal.
> > > > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> > > exec_queue, uint64_t addr)
> > > > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > > > */
> > > > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t *timeout)
> > > > > + uint32_t exec_queue, int64_t *timeout)
> > > > > {
> > > > > struct drm_xe_wait_user_fence wait = {
> > > > > .addr = to_user_pointer(addr),
> > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > > > + .flags = 0,
> > > > > .value = value,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > - .num_engines = eci ? 1 :0,
> > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > + .exec_queue_id = exec_queue,
> > > > > };
> > > > >
> > > > > igt_assert(timeout);
> > > > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > > uint64_t value,
> > > > > * @fd: xe device fd
> > > > > * @addr: address of value to compare
> > > > > * @value: expected value (equal) in @address
> > > > > - * @eci: engine class instance
> > > > > + * @exec_queue: exec_queue id
> > > > > * @timeout: time to wait in nanoseconds
> > > > > *
> > > > > * Function compares @value with memory pointed by @addr until
> > > > > they
> > > are equal.
> > > > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t *addr,
> > > uint64_t value,
> > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > */
> > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout)
> > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > {
> > > > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci, &timeout), 0);
> > > > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value, exec_queue,
> > > > > +&timeout), 0);
> > > > > return timeout;
> > > > > }
> > > > >
> > > > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t *addr,
> > > uint64_t value,
> > > > > * @fd: xe device fd
> > > > > * @addr: address of value to compare
> > > > > * @value: expected value (equal) in @address
> > > > > - * @eci: engine class instance
> > > > > + * @exec_queue: exec_queue id
> > > > > * @timeout: absolute time when wait expire
> > > > > *
> > > > > * Function compares @value with memory pointed by @addr until
> > > > > they
> > > are equal.
> > > > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > > *addr,
> > > uint64_t value,
> > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > */
> > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> > value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout)
> > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > {
> > > > > struct drm_xe_wait_user_fence wait = {
> > > > > .addr = to_user_pointer(addr),
> > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > - .flags = !eci ? DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > > + .flags = !exec_queue ?
> > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > >
> > > > To be on the safe side we should probably change the signature of
> > > > this function to explicitly receive this flag instead of relying on
> > > > the invalid
> > > exec_queue_id number.
> > > >
> > > > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP
> > was
> > > used here...
> > > >
> > > > Matt, thoughts?
> > > >
> > >
> > > Agree with everything here, !exec_queue really has nothing to do with
> > > setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So
> > yes,
> > > please change this function to accpt a flags field.
> > >
> > > Everything else LGTM.
> > >
> > > Matt
> >
> > I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not thing
> > to do with exec_queue, I will change this function to accept flag, but
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is related to
> > "no_engines" I removed from kernel side also
> > https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
> >
> > Matt please let me know your insights on
> > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
> >
> > Regards,
> > Krishna.
> > >
> > > > Everything else in the patch looks good to me.
> > > >
> > > > > .value = value,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = timeout,
> > > > > - .num_engines = eci ? 1 : 0,
> > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > + .exec_queue_id = exec_queue,
> > > > > };
> > > > > struct timespec ts;
> > > > >
> > > > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > > > bd660bd27..9c0b4b9bc 100644
> > > > > --- a/lib/xe/xe_ioctl.h
> > > > > +++ b/lib/xe/xe_ioctl.h
> > > > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t exec_queue,
> > > uint64_t addr,
> > > > > struct drm_xe_sync *sync, uint32_t num_syncs); void
> > > > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t *timeout);
> > > > > + uint32_t exec_queue, int64_t *timeout);
> > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout);
> > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr, uint64_t
> > value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout);
> > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > void xe_force_gt_reset(int fd, int gt);
> > > > >
> > > > > #endif /* XE_IOCTL_H */
> > > > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c index
> > > > > 89dc46fae..a37c2d97a 100644
> > > > > --- a/tests/intel/xe_evict.c
> > > > > +++ b/tests/intel/xe_evict.c
> > > > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > }
> > > > > #define TWENTY_SEC MS_TO_NS(20000)
> > > > > xe_wait_ufence(fd, &data[i].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, TWENTY_SEC);
> > > > > + bind_exec_queues[0], TWENTY_SEC);
> > > > > }
> > > > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > > > (char *)data;
> > > > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > data = xe_bo_map(fd, __bo,
> > > > > ALIGN(sizeof(*data) * n_execs, 0x1000));
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, TWENTY_SEC);
> > > > > + exec_queues[i], TWENTY_SEC);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > }
> > > > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff --git
> > > > > a/tests/intel/xe_exec_balancer.c b/tests/intel/xe_exec_balancer.c
> > > > > index 79ff65e89..2448d49bc 100644
> > > > > --- a/tests/intel/xe_exec_balancer.c
> > > > > +++ b/tests/intel/xe_exec_balancer.c
> > > > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > > > > bo_size, sync, 1);
> > > > >
> > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -514,7 +514,7 @@ test_cm(int fd, int gt, int class, int
> > > > > n_exec_queues, int n_execs,
> > > > >
> > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + exec_queues[e], ONE_SEC);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > NULL,
> > > > > 0);
> > > > >
> > > > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + 0, ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> > > n_exec_queues, int n_execs,
> > > > > * an invalidate.
> > > > > */
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + USER_FENCE_VALUE,
> > > exec_queues[e],
> > > > > + ONE_SEC);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) {
> > > > > /*
> > > > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > > > n_exec_queues, int n_execs,
> > > > >
> > > > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > + exec_queues[i], ONE_SEC);
>
> Need to change exec_queues[i] index to "i % n_exec_queues", need to change other files also, I will change
> exec_queues[i] => exec_queues[i % n_exec_queues]
why?
>
> Regards,
> Krishna
>
> > > > >
> > > > > /* Wait for all execs to complete */
> > > > > if (flags & INVALIDATE)
> > > > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > > > n_exec_queues, int n_execs,
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +ONE_SEC);
> > > > >
> > > > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > > > i < n_execs; i++)
> > > > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > > > b/tests/intel/xe_exec_compute_mode.c
> > > > > index 7d3004d65..749a9b634 100644
> > > > > --- a/tests/intel/xe_exec_compute_mode.c
> > > > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > > > drm_xe_engine_class_instance *eci,
> > > > >
> > > > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> > > ONE_SEC;
> > > > >
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -198,7 +198,7 @@ test_exec(int fd, struct
> > > > > drm_xe_engine_class_instance *eci,
> > > > >
> > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + exec_queues[e], fence_timeout);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > > 0,
> > > > > addr, bo_size, NULL, 0);
> > > > >
> > > > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + bind_exec_queues[e], fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > * an invalidate.
> > > > > */
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL,
> > > > > + USER_FENCE_VALUE,
> > > exec_queues[e],
> > > > > fence_timeout);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) { @@ -257,8 +257,8 @@
> > > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > > >
> > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > + exec_queues[i], fence_timeout);
> > > > >
> > > > > /* Wait for all execs to complete */
> > > > > if (flags & INVALIDATE)
> > > > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > bo_size,
> > > > > sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], fence_timeout);
> > > > >
> > > > > for (i = j; i < n_execs; i++)
> > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > a/tests/intel/xe_exec_fault_mode.c
> > > > > b/tests/intel/xe_exec_fault_mode.c
> > > > > index ee7cbb604..d76b3a056 100644
> > > > > --- a/tests/intel/xe_exec_fault_mode.c
> > > > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > }
> > > > >
> > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > if (flags & PREFETCH) {
> > > > > /* Should move to system memory */
> > > > > xe_vm_prefetch_async(fd, vm, bind_exec_queues[0], 0,
> > > addr,
> > > > > bo_size, sync, 1, 0);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > > > drm_xe_engine_class_instance *eci,
> > > > >
> > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + exec_queues[e], ONE_SEC);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[e],
> > > 0,
> > > > > addr, bo_size, NULL, 0);
> > > > >
> > > > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, ONE_SEC);
> > > > > + bind_exec_queues[e], ONE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > * an invalidate.
> > > > > */
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + USER_FENCE_VALUE,
> > > exec_queues[e],
> > > > > + ONE_SEC);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) {
> > > > > /*
> > > > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > - USER_FENCE_VALUE, NULL, ONE_SEC);
> > > > > + USER_FENCE_VALUE, exec_queues[i],
> > > ONE_SEC);
> > > > > }
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > bo_size,
> > > > > sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > ONE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE,
> > > > > + bind_exec_queues[0], ONE_SEC);
> > > > >
> > > > > if (!(flags & INVALID_FAULT)) {
> > > > > for (i = j; i < n_execs; i++)
> > > > > diff --git a/tests/intel/xe_exec_reset.c
> > > > > b/tests/intel/xe_exec_reset.c index 094b34896..398c90af5 100644
> > > > > --- a/tests/intel/xe_exec_reset.c
> > > > > +++ b/tests/intel/xe_exec_reset.c
> > > > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > > > >
> > > > > #define THREE_SEC MS_TO_NS(3000)
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > THREE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +THREE_SEC);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -621,7 +621,7 @@ test_compute_mode(int fd, struct
> > > drm_xe_engine_class_instance *eci,
> > > > > int err;
> > > > >
> > > > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, &timeout);
> > > > > + exec_queues[i], &timeout);
> > > > > if (flags & GT_RESET)
> > > > > /* exec races with reset: may timeout or complete */
> > > > > igt_assert(err == -ETIME || !err); @@ -631,7 +631,7
> > > @@
> > > > > test_compute_mode(int fd, struct drm_xe_engine_class_instance
> > > > > *eci,
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > THREE_SEC);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +THREE_SEC);
> > > > >
> > > > > if (!(flags & GT_RESET)) {
> > > > > for (i = 1; i < n_execs; i++)
> > > > > diff --git a/tests/intel/xe_exec_threads.c
> > > > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9 100644
> > > > > --- a/tests/intel/xe_exec_threads.c
> > > > > +++ b/tests/intel/xe_exec_threads.c
> > > > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t addr, uint64_t userptr,
> > > > >
> > > > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC :
> > > > > THREE_SEC;
> > > > >
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > >
> > > > > for (i = 0; i < n_execs; i++) {
> > > > > @@ -359,7 +359,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > for (j = i - 0x20; j <= i; ++j)
> > > > > xe_wait_ufence(fd, &data[j].exec_sync,
> > > > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + exec_queues[e], fence_timeout);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size,
> > > > > NULL, 0);
> > > > >
> > > > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > addr, bo_size, sync,
> > > > > 1);
> > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + 0, fence_timeout);
> > > > > data[0].vm_sync = 0;
> > > > > }
> > > > >
> > > > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i; ++j)
> > > > > xe_wait_ufence(fd,
> > > &data[j].exec_sync,
> > > > > USER_FENCE_VALUE,
> > > > > - NULL, fence_timeout);
> > > > > + exec_queues[e],
> > > > > + fence_timeout);
> > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > } else if (i * 2 != n_execs) {
> > > > > /*
> > > > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t
> > > addr, uint64_t userptr,
> > > > > j = flags & INVALIDATE ?
> > > > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > > > for (i = j; i < n_execs; i++)
> > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE, NULL,
> > > > > - fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > USER_FENCE_VALUE,
> > > > > + exec_queues[e], fence_timeout);
> > > > >
> > > > > /* Wait for all execs to complete */
> > > > > if (flags & INVALIDATE)
> > > > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > uint64_t addr, uint64_t userptr,
> > > > >
> > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > - xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, NULL,
> > > fence_timeout);
> > > > > + xe_wait_ufence(fd, &data[0].vm_sync, USER_FENCE_VALUE, 0,
> > > > > +fence_timeout);
> > > > >
> > > > > for (i = j; i < n_execs; i++)
> > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c index
> > > > > 3be987954..4e94403a3 100644
> > > > > --- a/tests/intel/xe_waitfence.c
> > > > > +++ b/tests/intel/xe_waitfence.c
> > > > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > > > uint32_t bo, uint64_t offset, }
> > > > >
> > > > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr,
> > > > > uint64_t
> > > value,
> > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > - int64_t timeout)
> > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > {
> > > > > struct drm_xe_wait_user_fence wait = {
> > > > > .addr = to_user_pointer(addr),
> > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > - .flags = !eci ? 0 : DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > + .flags = !exec_queue ? 0 :
> > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > .value = value,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = timeout,
> > > > > - .num_engines = eci ? 1 : 0,
> > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > + .exec_queue_id = exec_queue,
> > > > > };
> > > > > struct timespec ts;
> > > > >
> > > > > - igt_assert(eci);
> > > > > igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_WAIT_USER_FENCE,
> > > &wait), 0);
> > > > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > > > >
> > > > > @@ -82,7 +79,7 @@ enum waittype {
> > > > > static void
> > > > > waitfence(int fd, enum waittype wt) {
> > > > > - struct drm_xe_engine *engine = NULL;
> > > > > + uint32_t exec_queue;
> > > > > struct timespec ts;
> > > > > int64_t current, signalled;
> > > > > uint32_t bo_1;
> > > > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > > > >
> > > > > if (wt == RELTIME) {
> > > > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> > > MS_TO_NS(10));
> > > > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> > > MS_TO_NS(10));
> > > > > igt_debug("wait type: RELTIME - timeout: %ld, timeout left:
> > > %ld\n",
> > > > > MS_TO_NS(10), timeout);
> > > > > } else if (wt == ENGINE) {
> > > > > - engine = xe_engine(fd, 1);
> > > > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > > > +DRM_XE_ENGINE_CLASS_COPY);
> > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > timeout = current + MS_TO_NS(10);
> > > > > - signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > > &engine->instance, timeout);
> > > > > + signalled = wait_with_eci_abstime(fd, &wait_fence, 7,
> > > exec_queue,
> > > > > +timeout);
> > > > > igt_debug("wait type: ENGINE ABSTIME - timeout: %" PRId64
> > > > > ", signalled: %" PRId64
> > > > > ", elapsed: %" PRId64 "\n",
> > > > > @@ -128,7 +125,7 @@ waitfence(int fd, enum waittype wt)
> > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > timeout = current + MS_TO_NS(10);
> > > > > - signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7,
> > > NULL, timeout);
> > > > > + signalled = xe_wait_ufence_abstime(fd, &wait_fence, 7, 0,
> > > > > +timeout);
> > > > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > > > ", signalled: %" PRId64
> > > > > ", elapsed: %" PRId64 "\n",
> > > > > @@ -191,8 +188,7 @@ invalid_ops(int fd)
> > > > > .value = 1,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = 1,
> > > > > - .num_engines = 0,
> > > > > - .instances = 0,
> > > > > + .exec_queue_id = 0,
> > > > > };
> > > > >
> > > > > uint32_t vm = xe_vm_create(fd,
> > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8 +212,7
> > > @@ invalid_engine(int fd)
> > > > > .value = 1,
> > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > .timeout = -1,
> > > > > - .num_engines = 1,
> > > > > - .instances = 0,
> > > > > + .exec_queue_id = 0,
> > > > > };
> > > > >
> > > > > uint32_t vm = xe_vm_create(fd,
> > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > > > --
> > > > > 2.25.1
> > > > >
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure
2023-12-11 18:58 ` Rodrigo Vivi
@ 2023-12-12 4:23 ` Bommu, Krishnaiah
0 siblings, 0 replies; 14+ messages in thread
From: Bommu, Krishnaiah @ 2023-12-12 4:23 UTC (permalink / raw)
To: Vivi, Rodrigo; +Cc: igt-dev@lists.freedesktop.org
> -----Original Message-----
> From: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> Sent: Tuesday, December 12, 2023 12:29 AM
> To: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>
> Cc: Brost, Matthew <matthew.brost@intel.com>; igt-
> dev@lists.freedesktop.org
> Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> drm_xe_wait_user_fence structure
>
> On Mon, Dec 11, 2023 at 07:03:47AM +0000, Bommu, Krishnaiah wrote:
> >
> >
> > > -----Original Message-----
> > > From: Bommu, Krishnaiah
> > > Sent: Monday, December 11, 2023 11:06 AM
> > > To: Brost, Matthew <matthew.brost@intel.com>; Vivi, Rodrigo
> > > <rodrigo.vivi@intel.com>
> > > Cc: igt-dev@lists.freedesktop.org
> > > Subject: RE: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member to
> > > drm_xe_wait_user_fence structure
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Brost, Matthew <matthew.brost@intel.com>
> > > > Sent: Friday, December 8, 2023 6:35 PM
> > > > To: Vivi, Rodrigo <rodrigo.vivi@intel.com>
> > > > Cc: Bommu, Krishnaiah <krishnaiah.bommu@intel.com>; igt-
> > > > dev@lists.freedesktop.org
> > > > Subject: Re: [PATCH v4 1/2] drm-uapi/xe: add exec_queue_id member
> > > > to drm_xe_wait_user_fence structure
> > > >
> > > > On Fri, Dec 08, 2023 at 12:57:56AM -0500, Rodrigo Vivi wrote:
> > > > > On Fri, Dec 08, 2023 at 09:48:24AM +0530, Bommu Krishnaiah wrote:
> > > > > > remove the num_engines/instances members from
> > > > drm_xe_wait_user_fence
> > > > > > structure and add a exec_queue_id member
> > > > > >
> > > > > > Right now this is only checking if the engine list is sane and
> > > > > > nothing else. In the end every operation with this IOCTL is a soft
> check.
> > > > > > So, let's formalize that and only use this IOCTL to wait on the fence.
> > > > > >
> > > > > > exec_queue_id member will help to user space to get proper
> > > > > > error code from kernel while in exec_queue reset
> > > > > >
> > > > > > Signed-off-by: Bommu Krishnaiah <krishnaiah.bommu@intel.com>
> > > > > > Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > > > Cc: Francois Dugast <francois.dugast@intel.com>
> > > > > > ---
> > > > > > include/drm-uapi/xe_drm.h | 28 ++++++++++++++--------------
> > > > > > lib/xe/xe_ioctl.c | 27 +++++++++++----------------
> > > > > > lib/xe/xe_ioctl.h | 9 +++------
> > > > > > tests/intel/xe_evict.c | 4 ++--
> > > > > > tests/intel/xe_exec_balancer.c | 15 ++++++++-------
> > > > > > tests/intel/xe_exec_compute_mode.c | 18 +++++++++---------
> > > > > > tests/intel/xe_exec_fault_mode.c | 19 +++++++++++--------
> > > > > > tests/intel/xe_exec_reset.c | 6 +++---
> > > > > > tests/intel/xe_exec_threads.c | 15 ++++++++-------
> > > > > > tests/intel/xe_waitfence.c | 25 ++++++++++---------------
> > > > > > 10 files changed, 79 insertions(+), 87 deletions(-)
> > > > > >
> > > > > > diff --git a/include/drm-uapi/xe_drm.h
> > > > > > b/include/drm-uapi/xe_drm.h index 590f7b7af..fd06e4920 100644
> > > > > > --- a/include/drm-uapi/xe_drm.h
> > > > > > +++ b/include/drm-uapi/xe_drm.h
> > > > > > @@ -129,7 +129,6 @@ struct xe_user_extension {
> > > > > > * It is returned as part of the @drm_xe_engine, but it also is used
> as
> > > > > > * the input of engine selection for both
> > > > > > @drm_xe_exec_queue_create
> > > > and
> > > > > > * @drm_xe_query_engine_cycles
> > > > > > - *
> > > > > > */
> > > > > > struct drm_xe_engine_class_instance {
> > > > > > #define DRM_XE_ENGINE_CLASS_RENDER 0
> > > > > > @@ -143,9 +142,11 @@ struct drm_xe_engine_class_instance {
> > > > > > */
> > > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5
> > > > > > #define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6
> > > > > > + /** @engine_class: engine class id */
> > > > > > __u16 engine_class;
> > > > > > -
> > > > > > + /** @engine_instance: engine instance id */
> > > > > > __u16 engine_instance;
> > > > > > + /** @gt_id: Unique ID of this GT within the PCI Device */
> > > > > > __u16 gt_id;
> > > > > > /** @pad: MBZ */
> > > > > > __u16 pad;
> > > > > > @@ -736,6 +737,12 @@ struct drm_xe_vm_bind_op {
> > > > > > *
> > > > > > * Note: For userptr and externally imported dma-buf the
> > > > > > kernel
> > > > expects
> > > > > > * either 1WAY or 2WAY for the @pat_index.
> > > > > > + *
> > > > > > + * For DRM_XE_VM_BIND_FLAG_NULL bindings there are no
> KMD
> > > > restrictions
> > > > > > + * on the @pat_index. For such mappings there is no actual
> > > > > > +memory
> > > > being
> > > > > > + * mapped (the address in the PTE is invalid), so the
> > > > > > +various PAT
> > > > memory
> > > > > > + * attributes likely do not apply. Simply leaving as zero is one
> > > > > > + * option (still a valid pat_index).
> > > > > > */
> > > > > > __u16 pat_index;
> > > > > >
> > > > > > @@ -1024,8 +1031,7 @@ struct drm_xe_wait_user_fence {
> > > > > > /** @op: wait operation (type of comparison) */
> > > > > > __u16 op;
> > > > > >
> > > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP (1 << 0) /*
> > > e.g. Wait
> > > > on VM bind */
> > > > > > -#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 1)
> > > > > > +#define DRM_XE_UFENCE_WAIT_FLAG_ABSTIME (1 << 0)
> > > > > > /** @flags: wait flags */
> > > > > > __u16 flags;
> > > > > >
> > > > > > @@ -1058,17 +1064,11 @@ struct drm_xe_wait_user_fence {
> > > > > > */
> > > > > > __s64 timeout;
> > > > > >
> > > > > > - /**
> > > > > > - * @num_engines: number of engine instances to wait on,
> must be
> > > > zero
> > > > > > - * when DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > > - */
> > > > > > - __u64 num_engines;
> > > > > > + /** @exec_queue_id: exec_queue_id returned from
> > > > xe_exec_queue_create_ioctl */
> > > > > > + __u32 exec_queue_id;
> > > > > >
> > > > > > - /**
> > > > > > - * @instances: user pointer to array of
> > > > drm_xe_engine_class_instance to
> > > > > > - * wait on, must be NULL when
> > > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP set
> > > > > > - */
> > > > > > - __u64 instances;
> > > > > > + /** @pad2: MBZ */
> > > > > > + __u32 pad2;
> > > > > >
> > > > > > /** @reserved: Reserved */
> > > > > > __u64 reserved[2];
> > > > > > diff --git a/lib/xe/xe_ioctl.c b/lib/xe/xe_ioctl.c index
> > > > > > b29ca40ad..0a1ddc86b 100644
> > > > > > --- a/lib/xe/xe_ioctl.c
> > > > > > +++ b/lib/xe/xe_ioctl.c
> > > > > > @@ -462,7 +462,7 @@ void xe_exec_wait(int fd, uint32_t
> > > > > > exec_queue,
> > > > uint64_t addr)
> > > > > > * @fd: xe device fd
> > > > > > * @addr: address of value to compare
> > > > > > * @value: expected value (equal) in @address
> > > > > > - * @eci: engine class instance
> > > > > > + * @exec_queue: exec_queue id
> > > > > > * @timeout: pointer to time to wait in nanoseconds
> > > > > > *
> > > > > > * Function compares @value with memory pointed by @addr
> > > > > > until they
> > > > are equal.
> > > > > > @@ -471,17 +471,15 @@ void xe_exec_wait(int fd, uint32_t
> > > > exec_queue, uint64_t addr)
> > > > > > * signalled. Returns 0 on success, -errno of ioctl on error.
> > > > > > */
> > > > > > int __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t *timeout)
> > > > > > + uint32_t exec_queue, int64_t *timeout)
> > > > > > {
> > > > > > struct drm_xe_wait_user_fence wait = {
> > > > > > .addr = to_user_pointer(addr),
> > > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > > - .flags = !eci ?
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP : 0,
> > > > > > + .flags = 0,
> > > > > > .value = value,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > - .num_engines = eci ? 1 :0,
> > > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > > + .exec_queue_id = exec_queue,
> > > > > > };
> > > > > >
> > > > > > igt_assert(timeout);
> > > > > > @@ -499,7 +497,7 @@ int __xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * @fd: xe device fd
> > > > > > * @addr: address of value to compare
> > > > > > * @value: expected value (equal) in @address
> > > > > > - * @eci: engine class instance
> > > > > > + * @exec_queue: exec_queue id
> > > > > > * @timeout: time to wait in nanoseconds
> > > > > > *
> > > > > > * Function compares @value with memory pointed by @addr
> > > > > > until they
> > > > are equal.
> > > > > > @@ -508,10 +506,9 @@ int __xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > > */
> > > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t timeout)
> > > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > > {
> > > > > > - igt_assert_eq(__xe_wait_ufence(fd, addr, value, eci,
> &timeout), 0);
> > > > > > + igt_assert_eq(__xe_wait_ufence(fd, addr, value,
> exec_queue,
> > > > > > +&timeout), 0);
> > > > > > return timeout;
> > > > > > }
> > > > > >
> > > > > > @@ -520,7 +517,7 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * @fd: xe device fd
> > > > > > * @addr: address of value to compare
> > > > > > * @value: expected value (equal) in @address
> > > > > > - * @eci: engine class instance
> > > > > > + * @exec_queue: exec_queue id
> > > > > > * @timeout: absolute time when wait expire
> > > > > > *
> > > > > > * Function compares @value with memory pointed by @addr
> > > > > > until they
> > > > are equal.
> > > > > > @@ -529,18 +526,16 @@ int64_t xe_wait_ufence(int fd, uint64_t
> > > > > > *addr,
> > > > uint64_t value,
> > > > > > * Returns elapsed time in nanoseconds if user fence was signalled.
> > > > > > */
> > > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr,
> > > > > > uint64_t
> > > value,
> > > > > > - struct drm_xe_engine_class_instance
> *eci,
> > > > > > - int64_t timeout)
> > > > > > + uint32_t exec_queue, int64_t timeout)
> > > > > > {
> > > > > > struct drm_xe_wait_user_fence wait = {
> > > > > > .addr = to_user_pointer(addr),
> > > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > > - .flags = !eci ?
> DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP |
> > > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > > > + .flags = !exec_queue ?
> > > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME : 0,
> > > > >
> > > > > To be on the safe side we should probably change the signature
> > > > > of this function to explicitly receive this flag instead of
> > > > > relying on the invalid
> > > > exec_queue_id number.
> > > > >
> > > > > Also it is strange to me that DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP
> > > was
> > > > used here...
> > > > >
> > > > > Matt, thoughts?
> > > > >
> > > >
> > > > Agree with everything here, !exec_queue really has nothing to do
> > > > with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME. Setting
> > > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP when eci did make sense. So
> > > yes,
> > > > please change this function to accpt a flags field.
> > > >
> > > > Everything else LGTM.
> > > >
> > > > Matt
> > >
> > > I agree with setting DRM_XE_UFENCE_WAIT_FLAG_ABSTIME it is not
> thing
> > > to do with exec_queue, I will change this function to accept flag,
> > > but DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP is not required since it is
> > > related to "no_engines" I removed from kernel side also
> > >
> https://patchwork.freedesktop.org/patch/571236/?series=127365&rev=5
> > >
> > > Matt please let me know your insights on
> > > DRM_XE_UFENCE_WAIT_FLAG_SOFT_OP flag
> > >
> > > Regards,
> > > Krishna.
> > > >
> > > > > Everything else in the patch looks good to me.
> > > > >
> > > > > > .value = value,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = timeout,
> > > > > > - .num_engines = eci ? 1 : 0,
> > > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > > + .exec_queue_id = exec_queue,
> > > > > > };
> > > > > > struct timespec ts;
> > > > > >
> > > > > > diff --git a/lib/xe/xe_ioctl.h b/lib/xe/xe_ioctl.h index
> > > > > > bd660bd27..9c0b4b9bc 100644
> > > > > > --- a/lib/xe/xe_ioctl.h
> > > > > > +++ b/lib/xe/xe_ioctl.h
> > > > > > @@ -89,14 +89,11 @@ void xe_exec_sync(int fd, uint32_t
> > > > > > exec_queue,
> > > > uint64_t addr,
> > > > > > struct drm_xe_sync *sync, uint32_t num_syncs);
> void
> > > > > > xe_exec_wait(int fd, uint32_t exec_queue, uint64_t addr); int
> > > > > > __xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t *timeout);
> > > > > > + uint32_t exec_queue, int64_t *timeout);
> > > > > > int64_t xe_wait_ufence(int fd, uint64_t *addr, uint64_t value,
> > > > > > - struct drm_xe_engine_class_instance *eci,
> > > > > > - int64_t timeout);
> > > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > > int64_t xe_wait_ufence_abstime(int fd, uint64_t *addr,
> > > > > > uint64_t
> > > value,
> > > > > > - struct drm_xe_engine_class_instance
> *eci,
> > > > > > - int64_t timeout);
> > > > > > + uint32_t exec_queue, int64_t timeout);
> > > > > > void xe_force_gt_reset(int fd, int gt);
> > > > > >
> > > > > > #endif /* XE_IOCTL_H */
> > > > > > diff --git a/tests/intel/xe_evict.c b/tests/intel/xe_evict.c
> > > > > > index 89dc46fae..a37c2d97a 100644
> > > > > > --- a/tests/intel/xe_evict.c
> > > > > > +++ b/tests/intel/xe_evict.c
> > > > > > @@ -317,7 +317,7 @@ test_evict_cm(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > }
> > > > > > #define TWENTY_SEC MS_TO_NS(20000)
> > > > > > xe_wait_ufence(fd, &data[i].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, TWENTY_SEC);
> > > > > > + bind_exec_queues[0],
> TWENTY_SEC);
> > > > > > }
> > > > > > sync[0].addr = addr + (char *)&data[i].exec_sync -
> > > > > > (char *)data;
> > > > > > @@ -352,7 +352,7 @@ test_evict_cm(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > data = xe_bo_map(fd, __bo,
> > > > > > ALIGN(sizeof(*data) * n_execs,
> 0x1000));
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, TWENTY_SEC);
> > > > > > + exec_queues[i], TWENTY_SEC);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > }
> > > > > > munmap(data, ALIGN(sizeof(*data) * n_execs, 0x1000)); diff
> > > > > > --git a/tests/intel/xe_exec_balancer.c
> > > > > > b/tests/intel/xe_exec_balancer.c index 79ff65e89..2448d49bc
> > > > > > 100644
> > > > > > --- a/tests/intel/xe_exec_balancer.c
> > > > > > +++ b/tests/intel/xe_exec_balancer.c
> > > > > > @@ -483,7 +483,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > > > > bo_size, sync, 1);
> > > > > >
> > > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -514,7 +514,7 @@
> > > > > > test_cm(int fd, int gt, int class, int n_exec_queues, int
> > > > > > n_execs,
> > > > > >
> > > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + exec_queues[e], ONE_SEC);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr,
> bo_size,
> > > > NULL,
> > > > > > 0);
> > > > > >
> > > > > > @@ -529,7 +529,7 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + 0, ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -542,7 +542,8 @@ test_cm(int fd, int gt, int class, int
> > > > n_exec_queues, int n_execs,
> > > > > > * an invalidate.
> > > > > > */
> > > > > > xe_wait_ufence(fd,
> &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE,
> NULL,
> > > > ONE_SEC);
> > > > > > + USER_FENCE_VALUE,
> > > > exec_queues[e],
> > > > > > + ONE_SEC);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) {
> > > > > > /*
> > > > > > @@ -571,8 +572,8 @@ test_cm(int fd, int gt, int class, int
> > > > > > n_exec_queues, int n_execs,
> > > > > >
> > > > > > j = flags & INVALIDATE && n_execs ? n_execs - 1 : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + exec_queues[i], ONE_SEC);
> >
> > Need to change exec_queues[i] index to "i % n_exec_queues", need to
> > change other files also, I will change exec_queues[i] => exec_queues[i
> > % n_exec_queues]
>
> why?
We need to use same exec_queue, which is user for xe_exec, with current change test is failing with exec_queue error
From test
int e = i % n_exec_queues;
------------
exec.exec_queue_id = exec_queues[e];
exec.address = exec_addr;
xe_exec(fd, &exec);
with index change tests are passing
Regards,
Krishna.
>
> >
> > Regards,
> > Krishna
> >
> > > > > >
> > > > > > /* Wait for all execs to complete */
> > > > > > if (flags & INVALIDATE)
> > > > > > @@ -580,7 +581,7 @@ test_cm(int fd, int gt, int class, int
> > > > > > n_exec_queues, int n_execs,
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +ONE_SEC);
> > > > > >
> > > > > > for (i = (flags & INVALIDATE && n_execs) ? n_execs - 1 : 0;
> > > > > > i < n_execs; i++)
> > > > > > diff --git a/tests/intel/xe_exec_compute_mode.c
> > > > > > b/tests/intel/xe_exec_compute_mode.c
> > > > > > index 7d3004d65..749a9b634 100644
> > > > > > --- a/tests/intel/xe_exec_compute_mode.c
> > > > > > +++ b/tests/intel/xe_exec_compute_mode.c
> > > > > > @@ -171,8 +171,8 @@ test_exec(int fd, struct
> > > > > > drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > fence_timeout = igt_run_in_simulation() ? HUNDRED_SEC :
> > > > ONE_SEC;
> > > > > >
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -198,7 +198,7 @@
> > > > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + exec_queues[e], fence_timeout);
> > > > > > xe_vm_unbind_async(fd, vm,
> bind_exec_queues[e],
> > > > 0,
> > > > > > addr, bo_size, NULL, 0);
> > > > > >
> > > > > > @@ -214,7 +214,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + bind_exec_queues[e],
> fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -227,7 +227,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > * an invalidate.
> > > > > > */
> > > > > > xe_wait_ufence(fd,
> &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE,
> NULL,
> > > > > > + USER_FENCE_VALUE,
> > > > exec_queues[e],
> > > > > > fence_timeout);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) { @@ -257,8 +257,8
> @@
> > > > > > test_exec(int fd, struct drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + exec_queues[i], fence_timeout);
> > > > > >
> > > > > > /* Wait for all execs to complete */
> > > > > > if (flags & INVALIDATE)
> > > > > > @@ -267,8 +267,8 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > > bo_size,
> > > > > > sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], fence_timeout);
> > > > > >
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > > a/tests/intel/xe_exec_fault_mode.c
> > > > > > b/tests/intel/xe_exec_fault_mode.c
> > > > > > index ee7cbb604..d76b3a056 100644
> > > > > > --- a/tests/intel/xe_exec_fault_mode.c
> > > > > > +++ b/tests/intel/xe_exec_fault_mode.c
> > > > > > @@ -195,15 +195,16 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > }
> > > > > >
> > > > > > #define ONE_SEC MS_TO_NS(1000)
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > if (flags & PREFETCH) {
> > > > > > /* Should move to system memory */
> > > > > > xe_vm_prefetch_async(fd, vm,
> bind_exec_queues[0], 0,
> > > > addr,
> > > > > > bo_size, sync, 1, 0);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -230,7 +231,7 @@ test_exec(int fd, struct
> > > > > > drm_xe_engine_class_instance *eci,
> > > > > >
> > > > > > if (flags & REBIND && i + 1 != n_execs) {
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + exec_queues[e], ONE_SEC);
> > > > > > xe_vm_unbind_async(fd, vm,
> bind_exec_queues[e],
> > > > 0,
> > > > > > addr, bo_size, NULL, 0);
> > > > > >
> > > > > > @@ -246,7 +247,7 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, ONE_SEC);
> > > > > > + bind_exec_queues[e], ONE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -259,7 +260,8 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > * an invalidate.
> > > > > > */
> > > > > > xe_wait_ufence(fd,
> &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE,
> NULL,
> > > > ONE_SEC);
> > > > > > + USER_FENCE_VALUE,
> > > > exec_queues[e],
> > > > > > + ONE_SEC);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) {
> > > > > > /*
> > > > > > @@ -290,13 +292,14 @@ test_exec(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > j = flags & INVALIDATE ? n_execs - 1 : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > xe_wait_ufence(fd, &data[i].exec_sync,
> > > > > > - USER_FENCE_VALUE, NULL,
> ONE_SEC);
> > > > > > + USER_FENCE_VALUE,
> exec_queues[i],
> > > > ONE_SEC);
> > > > > > }
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, bind_exec_queues[0], 0, addr,
> > > > bo_size,
> > > > > > sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > ONE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE,
> > > > > > + bind_exec_queues[0], ONE_SEC);
> > > > > >
> > > > > > if (!(flags & INVALID_FAULT)) {
> > > > > > for (i = j; i < n_execs; i++) diff --git
> > > > > > a/tests/intel/xe_exec_reset.c b/tests/intel/xe_exec_reset.c
> > > > > > index 094b34896..398c90af5 100644
> > > > > > --- a/tests/intel/xe_exec_reset.c
> > > > > > +++ b/tests/intel/xe_exec_reset.c
> > > > > > @@ -564,7 +564,7 @@ test_compute_mode(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > xe_vm_bind_async(fd, vm, 0, bo, 0, addr, bo_size, sync, 1);
> > > > > >
> > > > > > #define THREE_SEC MS_TO_NS(3000)
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > THREE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +THREE_SEC);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -621,7 +621,7 @@
> > > > > > test_compute_mode(int fd, struct
> > > > drm_xe_engine_class_instance *eci,
> > > > > > int err;
> > > > > >
> > > > > > err = __xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, &timeout);
> > > > > > + exec_queues[i], &timeout);
> > > > > > if (flags & GT_RESET)
> > > > > > /* exec races with reset: may timeout or
> complete */
> > > > > > igt_assert(err == -ETIME || !err); @@ -631,7
> +631,7
> > > > @@
> > > > > > test_compute_mode(int fd, struct drm_xe_engine_class_instance
> > > > > > *eci,
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > THREE_SEC);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +THREE_SEC);
> > > > > >
> > > > > > if (!(flags & GT_RESET)) {
> > > > > > for (i = 1; i < n_execs; i++) diff --git
> > > > > > a/tests/intel/xe_exec_threads.c
> > > > > > b/tests/intel/xe_exec_threads.c index fcb926698..7985240c9
> > > > > > 100644
> > > > > > --- a/tests/intel/xe_exec_threads.c
> > > > > > +++ b/tests/intel/xe_exec_threads.c
> > > > > > @@ -331,7 +331,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t addr, uint64_t userptr,
> > > > > >
> > > > > > fence_timeout = igt_run_in_simulation() ? THIRTY_SEC :
> > > > > > THREE_SEC;
> > > > > >
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > >
> > > > > > for (i = 0; i < n_execs; i++) { @@ -359,7 +359,7 @@
> > > > > > test_compute_mode(int fd, uint32_t vm, uint64_t
> > > > addr, uint64_t userptr,
> > > > > > for (j = i - 0x20; j <= i; ++j)
> > > > > > xe_wait_ufence(fd,
> &data[j].exec_sync,
> > > > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + exec_queues[e],
> fence_timeout);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr,
> bo_size,
> > > > > > NULL, 0);
> > > > > >
> > > > > > @@ -374,7 +374,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t
> > > > addr, uint64_t userptr,
> > > > > > addr,
> bo_size, sync,
> > > > > > 1);
> > > > > > xe_wait_ufence(fd, &data[0].vm_sync,
> > > > USER_FENCE_VALUE,
> > > > > > - NULL, fence_timeout);
> > > > > > + 0, fence_timeout);
> > > > > > data[0].vm_sync = 0;
> > > > > > }
> > > > > >
> > > > > > @@ -389,7 +389,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t
> > > > addr, uint64_t userptr,
> > > > > > for (j = i == 0x20 ? 0 : i - 0x1f; j <= i;
> ++j)
> > > > > > xe_wait_ufence(fd,
> > > > &data[j].exec_sync,
> > > > > >
> USER_FENCE_VALUE,
> > > > > > - NULL,
> fence_timeout);
> > > > > > + exec_queues[e],
> > > > > > + fence_timeout);
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee);
> > > > > > } else if (i * 2 != n_execs) {
> > > > > > /*
> > > > > > @@ -421,8 +422,8 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t
> > > > addr, uint64_t userptr,
> > > > > > j = flags & INVALIDATE ?
> > > > > > (flags & RACE ? n_execs / 2 + 1 : n_execs - 1) : 0;
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > - xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE, NULL,
> > > > > > - fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[i].exec_sync,
> > > > USER_FENCE_VALUE,
> > > > > > + exec_queues[e], fence_timeout);
> > > > > >
> > > > > > /* Wait for all execs to complete */
> > > > > > if (flags & INVALIDATE)
> > > > > > @@ -430,7 +431,7 @@ test_compute_mode(int fd, uint32_t vm,
> > > > > > uint64_t addr, uint64_t userptr,
> > > > > >
> > > > > > sync[0].addr = to_user_pointer(&data[0].vm_sync);
> > > > > > xe_vm_unbind_async(fd, vm, 0, 0, addr, bo_size, sync, 1);
> > > > > > - xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, NULL,
> > > > fence_timeout);
> > > > > > + xe_wait_ufence(fd, &data[0].vm_sync,
> USER_FENCE_VALUE, 0,
> > > > > > +fence_timeout);
> > > > > >
> > > > > > for (i = j; i < n_execs; i++)
> > > > > > igt_assert_eq(data[i].data, 0xc0ffee); diff --git
> > > > > > a/tests/intel/xe_waitfence.c b/tests/intel/xe_waitfence.c
> > > > > > index
> > > > > > 3be987954..4e94403a3 100644
> > > > > > --- a/tests/intel/xe_waitfence.c
> > > > > > +++ b/tests/intel/xe_waitfence.c
> > > > > > @@ -37,22 +37,19 @@ static void do_bind(int fd, uint32_t vm,
> > > > > > uint32_t bo, uint64_t offset, }
> > > > > >
> > > > > > static int64_t wait_with_eci_abstime(int fd, uint64_t *addr,
> > > > > > uint64_t
> > > > value,
> > > > > > - struct
> drm_xe_engine_class_instance *eci,
> > > > > > - int64_t timeout)
> > > > > > + uint32_t exec_queue, int64_t
> timeout)
> > > > > > {
> > > > > > struct drm_xe_wait_user_fence wait = {
> > > > > > .addr = to_user_pointer(addr),
> > > > > > .op = DRM_XE_UFENCE_WAIT_OP_EQ,
> > > > > > - .flags = !eci ? 0 :
> DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > > + .flags = !exec_queue ? 0 :
> > > > DRM_XE_UFENCE_WAIT_FLAG_ABSTIME,
> > > > > > .value = value,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = timeout,
> > > > > > - .num_engines = eci ? 1 : 0,
> > > > > > - .instances = eci ? to_user_pointer(eci) : 0,
> > > > > > + .exec_queue_id = exec_queue,
> > > > > > };
> > > > > > struct timespec ts;
> > > > > >
> > > > > > - igt_assert(eci);
> > > > > > igt_assert_eq(igt_ioctl(fd,
> DRM_IOCTL_XE_WAIT_USER_FENCE,
> > > > &wait), 0);
> > > > > > igt_assert_eq(clock_gettime(CLOCK_MONOTONIC, &ts), 0);
> > > > > >
> > > > > > @@ -82,7 +79,7 @@ enum waittype { static void waitfence(int
> > > > > > fd, enum waittype wt) {
> > > > > > - struct drm_xe_engine *engine = NULL;
> > > > > > + uint32_t exec_queue;
> > > > > > struct timespec ts;
> > > > > > int64_t current, signalled;
> > > > > > uint32_t bo_1;
> > > > > > @@ -111,15 +108,15 @@ waitfence(int fd, enum waittype wt)
> > > > > > do_bind(fd, vm, bo_7, 0, 0xeffff0000, 0x10000, 7);
> > > > > >
> > > > > > if (wt == RELTIME) {
> > > > > > - timeout = xe_wait_ufence(fd, &wait_fence, 7, NULL,
> > > > MS_TO_NS(10));
> > > > > > + timeout = xe_wait_ufence(fd, &wait_fence, 7, 0,
> > > > MS_TO_NS(10));
> > > > > > igt_debug("wait type: RELTIME - timeout: %ld,
> timeout left:
> > > > %ld\n",
> > > > > > MS_TO_NS(10), timeout);
> > > > > > } else if (wt == ENGINE) {
> > > > > > - engine = xe_engine(fd, 1);
> > > > > > + exec_queue = xe_exec_queue_create_class(fd, vm,
> > > > > > +DRM_XE_ENGINE_CLASS_COPY);
> > > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > > timeout = current + MS_TO_NS(10);
> > > > > > - signalled = wait_with_eci_abstime(fd, &wait_fence,
> 7,
> > > > &engine->instance, timeout);
> > > > > > + signalled = wait_with_eci_abstime(fd, &wait_fence,
> 7,
> > > > exec_queue,
> > > > > > +timeout);
> > > > > > igt_debug("wait type: ENGINE ABSTIME - timeout:
> %" PRId64
> > > > > > ", signalled: %" PRId64
> > > > > > ", elapsed: %" PRId64 "\n", @@ -128,7
> +125,7 @@
> > > > > > waitfence(int fd, enum waittype wt)
> > > > > > clock_gettime(CLOCK_MONOTONIC, &ts);
> > > > > > current = ts.tv_sec * 1e9 + ts.tv_nsec;
> > > > > > timeout = current + MS_TO_NS(10);
> > > > > > - signalled = xe_wait_ufence_abstime(fd,
> &wait_fence, 7,
> > > > NULL, timeout);
> > > > > > + signalled = xe_wait_ufence_abstime(fd,
> &wait_fence, 7, 0,
> > > > > > +timeout);
> > > > > > igt_debug("wait type: ABSTIME - timeout: %" PRId64
> > > > > > ", signalled: %" PRId64
> > > > > > ", elapsed: %" PRId64 "\n", @@ -191,8
> +188,7 @@
> > > > > > invalid_ops(int fd)
> > > > > > .value = 1,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = 1,
> > > > > > - .num_engines = 0,
> > > > > > - .instances = 0,
> > > > > > + .exec_queue_id = 0,
> > > > > > };
> > > > > >
> > > > > > uint32_t vm = xe_vm_create(fd,
> > > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0); @@ -216,8
> +212,7
> > > > @@ invalid_engine(int fd)
> > > > > > .value = 1,
> > > > > > .mask = DRM_XE_UFENCE_WAIT_MASK_U64,
> > > > > > .timeout = -1,
> > > > > > - .num_engines = 1,
> > > > > > - .instances = 0,
> > > > > > + .exec_queue_id = 0,
> > > > > > };
> > > > > >
> > > > > > uint32_t vm = xe_vm_create(fd,
> > > > > > DRM_XE_VM_CREATE_FLAG_ASYNC_DEFAULT, 0);
> > > > > > --
> > > > > > 2.25.1
> > > > > >
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2023-12-12 4:23 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-12-08 4:18 [PATCH v4 0/2] RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure Bommu Krishnaiah
2023-12-08 4:18 ` [PATCH v4 1/2] " Bommu Krishnaiah
2023-12-08 5:57 ` Rodrigo Vivi
2023-12-08 13:05 ` Matthew Brost
2023-12-11 5:36 ` Bommu, Krishnaiah
2023-12-11 7:03 ` Bommu, Krishnaiah
2023-12-11 18:58 ` Rodrigo Vivi
2023-12-12 4:23 ` Bommu, Krishnaiah
2023-12-08 4:18 ` [PATCH v4 2/2] drm-uapi/xe: Skip xe_wait_user_fence_ioctl when exec_queue reset happen Bommu Krishnaiah
2023-12-08 6:00 ` Rodrigo Vivi
2023-12-08 12:59 ` Bommu, Krishnaiah
2023-12-08 13:48 ` Rodrigo Vivi
2023-12-08 5:48 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev5) Patchwork
2023-12-08 13:16 ` ✗ Fi.CI.BUILD: failure for RFC: drm-uapi/xe: add exec_queue_id member to drm_xe_wait_user_fence structure (rev6) Patchwork
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox