* [PATCH 0/5] Convert multiple bind ops to 1 job
@ 2024-04-29 16:47 Matthew Brost
2024-04-29 16:47 ` [PATCH 1/5] drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue Matthew Brost
` (7 more replies)
0 siblings, 8 replies; 9+ messages in thread
From: Matthew Brost @ 2024-04-29 16:47 UTC (permalink / raw)
To: intel-xe; +Cc: Matthew Brost
Continuation of merging parts of [1]. Patch #2 in this series is quite
large but unsure how to split the patch without breaking functionality.
Matt
[1] https://patchwork.freedesktop.org/series/125608/
Matthew Brost (5):
drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue
drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
drm/xe: Convert multiple bind ops into single job
drm/xe: Update VM trace events
drm/xe: Update PT layer with better error handling
drivers/gpu/drm/xe/xe_bo_types.h | 2 +
drivers/gpu/drm/xe/xe_migrate.c | 307 ++++----
drivers/gpu/drm/xe/xe_migrate.h | 34 +-
drivers/gpu/drm/xe/xe_pt.c | 1251 ++++++++++++++++++++----------
drivers/gpu/drm/xe/xe_pt.h | 14 +-
drivers/gpu/drm/xe/xe_pt_types.h | 48 ++
drivers/gpu/drm/xe/xe_trace.h | 10 +-
drivers/gpu/drm/xe/xe_vm.c | 623 +++++----------
drivers/gpu/drm/xe/xe_vm.h | 2 +
drivers/gpu/drm/xe/xe_vm_types.h | 39 +-
10 files changed, 1274 insertions(+), 1056 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/5] drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
@ 2024-04-29 16:47 ` Matthew Brost
2024-04-29 16:47 ` [PATCH 2/5] drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops Matthew Brost
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Matthew Brost @ 2024-04-29 16:47 UTC (permalink / raw)
To: intel-xe; +Cc: Matthew Brost
Engine is old nomenclature, replace with exec queue.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/xe/xe_migrate.c | 9 ++++-----
drivers/gpu/drm/xe/xe_migrate.h | 2 +-
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c
index 9f6e9b7f11c8..87e0460fefd7 100644
--- a/drivers/gpu/drm/xe/xe_migrate.c
+++ b/drivers/gpu/drm/xe/xe_migrate.c
@@ -85,15 +85,14 @@ struct xe_migrate {
#define MAX_PTE_PER_SDI 0x1FE
/**
- * xe_tile_migrate_engine() - Get this tile's migrate engine.
+ * xe_tile_migrate_exec_queue() - Get this tile's migrate exec queue.
* @tile: The tile.
*
- * Returns the default migrate engine of this tile.
- * TODO: Perhaps this function is slightly misplaced, and even unneeded?
+ * Returns the default migrate exec queue of this tile.
*
- * Return: The default migrate engine
+ * Return: The default migrate exec queue
*/
-struct xe_exec_queue *xe_tile_migrate_engine(struct xe_tile *tile)
+struct xe_exec_queue *xe_tile_migrate_exec_queue(struct xe_tile *tile)
{
return tile->migrate->q;
}
diff --git a/drivers/gpu/drm/xe/xe_migrate.h b/drivers/gpu/drm/xe/xe_migrate.h
index 951f19318ea4..a5bcaafe4a99 100644
--- a/drivers/gpu/drm/xe/xe_migrate.h
+++ b/drivers/gpu/drm/xe/xe_migrate.h
@@ -106,5 +106,5 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
void xe_migrate_wait(struct xe_migrate *m);
-struct xe_exec_queue *xe_tile_migrate_engine(struct xe_tile *tile);
+struct xe_exec_queue *xe_tile_migrate_exec_queue(struct xe_tile *tile);
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/5] drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
2024-04-29 16:47 ` [PATCH 1/5] drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue Matthew Brost
@ 2024-04-29 16:47 ` Matthew Brost
2024-04-29 16:47 ` [PATCH 3/5] drm/xe: Convert multiple bind ops into single job Matthew Brost
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Matthew Brost @ 2024-04-29 16:47 UTC (permalink / raw)
To: intel-xe; +Cc: Matthew Brost, Oak Zeng, Thomas Hellström
Each xe_vma_op resolves to 0-3 pt_ops. Add storage for the pt_ops to
xe_vma_ops which is dynamically allocated based the number and types of
xe_vma_op in the xe_vma_ops list. Allocation only implemented in this
patch.
This will help with converting xe_vma_ops (multiple xe_vma_op) in a
atomic update unit.
Cc: Oak Zeng <oak.zeng@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/xe/xe_pt_types.h | 12 ++++++
drivers/gpu/drm/xe/xe_vm.c | 66 +++++++++++++++++++++++++++++++-
drivers/gpu/drm/xe/xe_vm_types.h | 8 ++++
3 files changed, 84 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_pt_types.h b/drivers/gpu/drm/xe/xe_pt_types.h
index cee70cb0f014..2093150f461e 100644
--- a/drivers/gpu/drm/xe/xe_pt_types.h
+++ b/drivers/gpu/drm/xe/xe_pt_types.h
@@ -74,4 +74,16 @@ struct xe_vm_pgtable_update {
u32 flags;
};
+/** struct xe_vm_pgtable_update_op - Page table update operation */
+struct xe_vm_pgtable_update_op {
+ /** @entries: entries to update for this operation */
+ struct xe_vm_pgtable_update entries[XE_VM_MAX_LEVEL * 2 + 1];
+ /** @num_entries: number of entries for this update operation */
+ u32 num_entries;
+ /** @bind: is a bind */
+ bool bind;
+ /** @rebind: is a rebind */
+ bool rebind;
+};
+
#endif
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index dfd31b346021..b60ae743d2e6 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -712,6 +712,42 @@ int xe_vm_userptr_check_repin(struct xe_vm *vm)
list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN;
}
+static int xe_vma_ops_alloc(struct xe_vma_ops *vops)
+{
+ int i;
+
+ for (i = 0; i < XE_MAX_TILES_PER_DEVICE; ++i) {
+ if (!vops->pt_update_ops[i].num_ops)
+ continue;
+
+ vops->pt_update_ops[i].ops =
+ kmalloc_array(vops->pt_update_ops[i].num_ops,
+ sizeof(*vops->pt_update_ops[i].ops),
+ GFP_KERNEL);
+ if (!vops->pt_update_ops[i].ops)
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void xe_vma_ops_fini(struct xe_vma_ops *vops)
+{
+ int i;
+
+ for (i = 0; i < XE_MAX_TILES_PER_DEVICE; ++i)
+ kfree(vops->pt_update_ops[i].ops);
+}
+
+static void xe_vma_ops_incr_pt_update_ops(struct xe_vma_ops *vops, u8 tile_mask)
+{
+ int i;
+
+ for (i = 0; i < XE_MAX_TILES_PER_DEVICE; ++i)
+ if (BIT(i) & tile_mask)
+ ++vops->pt_update_ops[i].num_ops;
+}
+
static void xe_vm_populate_rebind(struct xe_vma_op *op, struct xe_vma *vma,
u8 tile_mask)
{
@@ -739,6 +775,7 @@ static int xe_vm_ops_add_rebind(struct xe_vma_ops *vops, struct xe_vma *vma,
xe_vm_populate_rebind(op, vma, tile_mask);
list_add_tail(&op->link, &vops->list);
+ xe_vma_ops_incr_pt_update_ops(vops, tile_mask);
return 0;
}
@@ -779,6 +816,10 @@ int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)
goto free_ops;
}
+ err = xe_vma_ops_alloc(&vops);
+ if (err)
+ goto free_ops;
+
fence = ops_execute(vm, &vops);
if (IS_ERR(fence)) {
err = PTR_ERR(fence);
@@ -793,6 +834,7 @@ int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)
list_del(&op->link);
kfree(op);
}
+ xe_vma_ops_fini(&vops);
return err;
}
@@ -814,12 +856,20 @@ struct dma_fence *xe_vma_rebind(struct xe_vm *vm, struct xe_vma *vma, u8 tile_ma
if (err)
return ERR_PTR(err);
+ err = xe_vma_ops_alloc(&vops);
+ if (err) {
+ fence = ERR_PTR(err);
+ goto free_ops;
+ }
+
fence = ops_execute(vm, &vops);
+free_ops:
list_for_each_entry_safe(op, next_op, &vops.list, link) {
list_del(&op->link);
kfree(op);
}
+ xe_vma_ops_fini(&vops);
return fence;
}
@@ -2276,7 +2326,6 @@ static int xe_vma_op_commit(struct xe_vm *vm, struct xe_vma_op *op)
return err;
}
-
static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
struct drm_gpuva_ops *ops,
struct xe_sync_entry *syncs, u32 num_syncs,
@@ -2328,6 +2377,9 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
return PTR_ERR(vma);
op->map.vma = vma;
+ if (op->map.immediate || !xe_vm_in_fault_mode(vm))
+ xe_vma_ops_incr_pt_update_ops(vops,
+ op->tile_mask);
break;
}
case DRM_GPUVA_OP_REMAP:
@@ -2372,6 +2424,8 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
vm_dbg(&xe->drm, "REMAP:SKIP_PREV: addr=0x%016llx, range=0x%016llx",
(ULL)op->remap.start,
(ULL)op->remap.range);
+ } else {
+ xe_vma_ops_incr_pt_update_ops(vops, op->tile_mask);
}
}
@@ -2408,13 +2462,16 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
vm_dbg(&xe->drm, "REMAP:SKIP_NEXT: addr=0x%016llx, range=0x%016llx",
(ULL)op->remap.start,
(ULL)op->remap.range);
+ } else {
+ xe_vma_ops_incr_pt_update_ops(vops, op->tile_mask);
}
}
+ xe_vma_ops_incr_pt_update_ops(vops, op->tile_mask);
break;
}
case DRM_GPUVA_OP_UNMAP:
case DRM_GPUVA_OP_PREFETCH:
- /* Nothing to do */
+ xe_vma_ops_incr_pt_update_ops(vops, op->tile_mask);
break;
default:
drm_warn(&vm->xe->drm, "NOT POSSIBLE");
@@ -3252,11 +3309,16 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
goto unwind_ops;
}
+ err = xe_vma_ops_alloc(&vops);
+ if (err)
+ goto unwind_ops;
+
err = vm_bind_ioctl_ops_execute(vm, &vops);
unwind_ops:
if (err && err != -ENODATA)
vm_bind_ioctl_ops_unwind(vm, ops, args->num_binds);
+ xe_vma_ops_fini(&vops);
for (i = args->num_binds - 1; i >= 0; --i)
if (ops[i])
drm_gpuva_ops_free(&vm->gpuvm, ops[i]);
diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
index ce1a63a5e3e7..211c88801182 100644
--- a/drivers/gpu/drm/xe/xe_vm_types.h
+++ b/drivers/gpu/drm/xe/xe_vm_types.h
@@ -21,6 +21,7 @@ struct xe_bo;
struct xe_sync_entry;
struct xe_user_fence;
struct xe_vm;
+struct xe_vm_pgtable_update_op;
#define XE_VMA_READ_ONLY DRM_GPUVA_USERBITS
#define XE_VMA_DESTROYED (DRM_GPUVA_USERBITS << 1)
@@ -368,6 +369,13 @@ struct xe_vma_ops {
struct xe_sync_entry *syncs;
/** @num_syncs: number of syncs */
u32 num_syncs;
+ /** @pt_update_ops: page table update operations */
+ struct {
+ /** @ops: operations */
+ struct xe_vm_pgtable_update_op *ops;
+ /** @num_ops: number of operations */
+ u32 num_ops;
+ } pt_update_ops[XE_MAX_TILES_PER_DEVICE];
};
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 3/5] drm/xe: Convert multiple bind ops into single job
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
2024-04-29 16:47 ` [PATCH 1/5] drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue Matthew Brost
2024-04-29 16:47 ` [PATCH 2/5] drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops Matthew Brost
@ 2024-04-29 16:47 ` Matthew Brost
2024-04-29 16:47 ` [PATCH 4/5] drm/xe: Update VM trace events Matthew Brost
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Matthew Brost @ 2024-04-29 16:47 UTC (permalink / raw)
To: intel-xe; +Cc: Matthew Brost, Oak Zeng, Thomas Hellström
This aligns with the uAPI of an array of binds or single bind that
results in multiple GPUVA ops to be considered a single atomic
operations.
The implemenation is roughly:
- xe_vma_ops is a list of xe_vma_op (GPUVA op)
- each xe_vma_op resolves to 0-3 PT ops
- xe_vma_ops creates a single job
- if at any point during binding a failure occurs, xe_vma_ops contains
the information necessary unwind the PT and VMA (GPUVA) state
Cc: Oak Zeng <oak.zeng@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/xe/xe_bo_types.h | 2 +
drivers/gpu/drm/xe/xe_migrate.c | 298 ++++----
drivers/gpu/drm/xe/xe_migrate.h | 32 +-
drivers/gpu/drm/xe/xe_pt.c | 1083 +++++++++++++++++++-----------
drivers/gpu/drm/xe/xe_pt.h | 14 +-
drivers/gpu/drm/xe/xe_pt_types.h | 36 +
drivers/gpu/drm/xe/xe_vm.c | 519 ++++----------
drivers/gpu/drm/xe/xe_vm.h | 2 +
drivers/gpu/drm/xe/xe_vm_types.h | 43 +-
9 files changed, 1008 insertions(+), 1021 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_bo_types.h b/drivers/gpu/drm/xe/xe_bo_types.h
index 86422e113d39..02d68873558a 100644
--- a/drivers/gpu/drm/xe/xe_bo_types.h
+++ b/drivers/gpu/drm/xe/xe_bo_types.h
@@ -58,6 +58,8 @@ struct xe_bo {
#endif
/** @freed: List node for delayed put. */
struct llist_node freed;
+ /** @update_index: Update index if PT BO */
+ int update_index;
/** @created: Whether the bo has passed initial creation */
bool created;
diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c
index 87e0460fefd7..23e04ac84eae 100644
--- a/drivers/gpu/drm/xe/xe_migrate.c
+++ b/drivers/gpu/drm/xe/xe_migrate.c
@@ -1103,6 +1103,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,
}
static void write_pgtable(struct xe_tile *tile, struct xe_bb *bb, u64 ppgtt_ofs,
+ const struct xe_vm_pgtable_update_op *pt_op,
const struct xe_vm_pgtable_update *update,
struct xe_migrate_pt_update *pt_update)
{
@@ -1137,8 +1138,12 @@ static void write_pgtable(struct xe_tile *tile, struct xe_bb *bb, u64 ppgtt_ofs,
bb->cs[bb->len++] = MI_STORE_DATA_IMM | MI_SDI_NUM_QW(chunk);
bb->cs[bb->len++] = lower_32_bits(addr);
bb->cs[bb->len++] = upper_32_bits(addr);
- ops->populate(pt_update, tile, NULL, bb->cs + bb->len, ofs, chunk,
- update);
+ if (pt_op->bind)
+ ops->populate(pt_update, tile, NULL, bb->cs + bb->len,
+ ofs, chunk, update);
+ else
+ ops->clear(pt_update, tile, NULL, bb->cs + bb->len,
+ ofs, chunk, update);
bb->len += chunk * 2;
ofs += chunk;
@@ -1163,114 +1168,58 @@ struct migrate_test_params {
static struct dma_fence *
xe_migrate_update_pgtables_cpu(struct xe_migrate *m,
- struct xe_vm *vm, struct xe_bo *bo,
- const struct xe_vm_pgtable_update *updates,
- u32 num_updates, bool wait_vm,
struct xe_migrate_pt_update *pt_update)
{
XE_TEST_DECLARE(struct migrate_test_params *test =
to_migrate_test_params
(xe_cur_kunit_priv(XE_TEST_LIVE_MIGRATE));)
const struct xe_migrate_pt_update_ops *ops = pt_update->ops;
- struct dma_fence *fence;
+ struct xe_vm *vm = pt_update->vops->vm;
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &pt_update->vops->pt_update_ops[pt_update->tile_id];
int err;
- u32 i;
+ u32 j, i;
if (XE_TEST_ONLY(test && test->force_gpu))
return ERR_PTR(-ETIME);
- if (bo && !dma_resv_test_signaled(bo->ttm.base.resv,
- DMA_RESV_USAGE_KERNEL))
- return ERR_PTR(-ETIME);
-
- if (wait_vm && !dma_resv_test_signaled(xe_vm_resv(vm),
- DMA_RESV_USAGE_BOOKKEEP))
- return ERR_PTR(-ETIME);
-
if (ops->pre_commit) {
pt_update->job = NULL;
err = ops->pre_commit(pt_update);
if (err)
return ERR_PTR(err);
}
- for (i = 0; i < num_updates; i++) {
- const struct xe_vm_pgtable_update *update = &updates[i];
-
- ops->populate(pt_update, m->tile, &update->pt_bo->vmap, NULL,
- update->ofs, update->qwords, update);
- }
-
- if (vm) {
- trace_xe_vm_cpu_bind(vm);
- xe_device_wmb(vm->xe);
- }
-
- fence = dma_fence_get_stub();
-
- return fence;
-}
-
-static bool no_in_syncs(struct xe_vm *vm, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs)
-{
- struct dma_fence *fence;
- int i;
- for (i = 0; i < num_syncs; i++) {
- fence = syncs[i].fence;
-
- if (fence && !test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
- &fence->flags))
- return false;
- }
- if (q) {
- fence = xe_exec_queue_last_fence_get(q, vm);
- if (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) {
- dma_fence_put(fence);
- return false;
+ for (j = 0; j < pt_update_ops->num_ops; ++j) {
+ const struct xe_vm_pgtable_update_op *pt_op =
+ &pt_update_ops->ops[j];
+
+ for (i = 0; i < pt_op->num_entries; i++) {
+ const struct xe_vm_pgtable_update *update =
+ &pt_op->entries[i];
+
+ if (pt_op->bind)
+ ops->populate(pt_update, m->tile,
+ &update->pt_bo->vmap, NULL,
+ update->ofs, update->qwords,
+ update);
+ else
+ ops->clear(pt_update, m->tile,
+ &update->pt_bo->vmap, NULL,
+ update->ofs, update->qwords, update);
}
- dma_fence_put(fence);
}
- return true;
+ trace_xe_vm_cpu_bind(vm);
+ xe_device_wmb(vm->xe);
+
+ return dma_fence_get_stub();
}
-/**
- * xe_migrate_update_pgtables() - Pipelined page-table update
- * @m: The migrate context.
- * @vm: The vm we'll be updating.
- * @bo: The bo whose dma-resv we will await before updating, or NULL if userptr.
- * @q: The exec queue to be used for the update or NULL if the default
- * migration engine is to be used.
- * @updates: An array of update descriptors.
- * @num_updates: Number of descriptors in @updates.
- * @syncs: Array of xe_sync_entry to await before updating. Note that waits
- * will block the engine timeline.
- * @num_syncs: Number of entries in @syncs.
- * @pt_update: Pointer to a struct xe_migrate_pt_update, which contains
- * pointers to callback functions and, if subclassed, private arguments to
- * those.
- *
- * Perform a pipelined page-table update. The update descriptors are typically
- * built under the same lock critical section as a call to this function. If
- * using the default engine for the updates, they will be performed in the
- * order they grab the job_mutex. If different engines are used, external
- * synchronization is needed for overlapping updates to maintain page-table
- * consistency. Note that the meaing of "overlapping" is that the updates
- * touch the same page-table, which might be a higher-level page-directory.
- * If no pipelining is needed, then updates may be performed by the cpu.
- *
- * Return: A dma_fence that, when signaled, indicates the update completion.
- */
-struct dma_fence *
-xe_migrate_update_pgtables(struct xe_migrate *m,
- struct xe_vm *vm,
- struct xe_bo *bo,
- struct xe_exec_queue *q,
- const struct xe_vm_pgtable_update *updates,
- u32 num_updates,
- struct xe_sync_entry *syncs, u32 num_syncs,
- struct xe_migrate_pt_update *pt_update)
+static struct dma_fence *
+__xe_migrate_update_pgtables(struct xe_migrate *m,
+ struct xe_migrate_pt_update *pt_update,
+ struct xe_vm_pgtable_update_ops *pt_update_ops)
{
const struct xe_migrate_pt_update_ops *ops = pt_update->ops;
struct xe_tile *tile = m->tile;
@@ -1279,59 +1228,45 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
struct xe_sched_job *job;
struct dma_fence *fence;
struct drm_suballoc *sa_bo = NULL;
- struct xe_vma *vma = pt_update->vma;
struct xe_bb *bb;
- u32 i, batch_size, ppgtt_ofs, update_idx, page_ofs = 0;
+ u32 i, j, batch_size = 0, ppgtt_ofs, update_idx, page_ofs = 0;
+ u32 num_updates = 0, current_update = 0;
u64 addr;
int err = 0;
- bool usm = !q && xe->info.has_usm;
- bool first_munmap_rebind = vma &&
- vma->gpuva.flags & XE_VMA_FIRST_REBIND;
- struct xe_exec_queue *q_override = !q ? m->q : q;
- u16 pat_index = xe->pat.idx[XE_CACHE_WB];
+ bool is_migrate = pt_update_ops->q == m->q;
+ bool usm = is_migrate && xe->info.has_usm;
- /* Use the CPU if no in syncs and engine is idle */
- if (no_in_syncs(vm, q, syncs, num_syncs) && xe_exec_queue_is_idle(q_override)) {
- fence = xe_migrate_update_pgtables_cpu(m, vm, bo, updates,
- num_updates,
- first_munmap_rebind,
- pt_update);
- if (!IS_ERR(fence) || fence == ERR_PTR(-EAGAIN))
- return fence;
+ for (i = 0; i < pt_update_ops->num_ops; ++i) {
+ struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[i];
+ struct xe_vm_pgtable_update *updates = pt_op->entries;
+
+ num_updates += pt_op->num_entries;
+ for (j = 0; j < pt_op->num_entries; ++j) {
+ u32 num_cmds = DIV_ROUND_UP(updates[j].qwords, 0x1ff);
+
+ /* align noop + MI_STORE_DATA_IMM cmd prefix */
+ batch_size += 4 * num_cmds + updates[j].qwords * 2;
+ }
}
/* fixed + PTE entries */
if (IS_DGFX(xe))
- batch_size = 2;
+ batch_size += 2;
else
- batch_size = 6 + num_updates * 2;
+ batch_size += 6 + num_updates * 2;
- for (i = 0; i < num_updates; i++) {
- u32 num_cmds = DIV_ROUND_UP(updates[i].qwords, MAX_PTE_PER_SDI);
-
- /* align noop + MI_STORE_DATA_IMM cmd prefix */
- batch_size += 4 * num_cmds + updates[i].qwords * 2;
- }
-
- /*
- * XXX: Create temp bo to copy from, if batch_size becomes too big?
- *
- * Worst case: Sum(2 * (each lower level page size) + (top level page size))
- * Should be reasonably bound..
- */
- xe_tile_assert(tile, batch_size < SZ_128K);
-
- bb = xe_bb_new(gt, batch_size, !q && xe->info.has_usm);
+ bb = xe_bb_new(gt, batch_size, usm);
if (IS_ERR(bb))
return ERR_CAST(bb);
/* For sysmem PTE's, need to map them in our hole.. */
if (!IS_DGFX(xe)) {
ppgtt_ofs = NUM_KERNEL_PDE - 1;
- if (q) {
- xe_tile_assert(tile, num_updates <= NUM_VMUSA_WRITES_PER_UNIT);
+ if (!is_migrate) {
+ u32 num_units = DIV_ROUND_UP(num_updates,
+ NUM_VMUSA_WRITES_PER_UNIT);
- sa_bo = drm_suballoc_new(&m->vm_update_sa, 1,
+ sa_bo = drm_suballoc_new(&m->vm_update_sa, num_units,
GFP_KERNEL, true, 0);
if (IS_ERR(sa_bo)) {
err = PTR_ERR(sa_bo);
@@ -1351,14 +1286,26 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
bb->cs[bb->len++] = ppgtt_ofs * XE_PAGE_SIZE + page_ofs;
bb->cs[bb->len++] = 0; /* upper_32_bits */
- for (i = 0; i < num_updates; i++) {
- struct xe_bo *pt_bo = updates[i].pt_bo;
+ for (i = 0; i < pt_update_ops->num_ops; ++i) {
+ struct xe_vm_pgtable_update_op *pt_op =
+ &pt_update_ops->ops[i];
+ struct xe_vm_pgtable_update *updates = pt_op->entries;
- xe_tile_assert(tile, pt_bo->size == SZ_4K);
+ for (j = 0; j < pt_op->num_entries; ++j, ++current_update) {
+ struct xe_vm *vm = pt_update->vops->vm;
+ struct xe_bo *pt_bo = updates[j].pt_bo;
- addr = vm->pt_ops->pte_encode_bo(pt_bo, 0, pat_index, 0);
- bb->cs[bb->len++] = lower_32_bits(addr);
- bb->cs[bb->len++] = upper_32_bits(addr);
+ xe_tile_assert(tile, pt_bo->size == SZ_4K);
+
+ /* Map a PT at most once */
+ if (pt_bo->update_index < 0)
+ pt_bo->update_index = current_update;
+
+ addr = vm->pt_ops->pte_encode_bo(pt_bo, 0,
+ XE_CACHE_WB, 0);
+ bb->cs[bb->len++] = lower_32_bits(addr);
+ bb->cs[bb->len++] = upper_32_bits(addr);
+ }
}
bb->cs[bb->len++] = MI_BATCH_BUFFER_END;
@@ -1366,22 +1313,39 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
addr = xe_migrate_vm_addr(ppgtt_ofs, 0) +
(page_ofs / sizeof(u64)) * XE_PAGE_SIZE;
- for (i = 0; i < num_updates; i++)
- write_pgtable(tile, bb, addr + i * XE_PAGE_SIZE,
- &updates[i], pt_update);
+ for (i = 0; i < pt_update_ops->num_ops; ++i) {
+ struct xe_vm_pgtable_update_op *pt_op =
+ &pt_update_ops->ops[i];
+ struct xe_vm_pgtable_update *updates = pt_op->entries;
+
+ for (j = 0; j < pt_op->num_entries; ++j) {
+ struct xe_bo *pt_bo = updates[j].pt_bo;
+
+ write_pgtable(tile, bb, addr +
+ pt_bo->update_index * XE_PAGE_SIZE,
+ pt_op, &updates[j], pt_update);
+ }
+ }
} else {
/* phys pages, no preamble required */
bb->cs[bb->len++] = MI_BATCH_BUFFER_END;
update_idx = bb->len;
- for (i = 0; i < num_updates; i++)
- write_pgtable(tile, bb, 0, &updates[i], pt_update);
+ for (i = 0; i < pt_update_ops->num_ops; ++i) {
+ struct xe_vm_pgtable_update_op *pt_op =
+ &pt_update_ops->ops[i];
+ struct xe_vm_pgtable_update *updates = pt_op->entries;
+
+ for (j = 0; j < pt_op->num_entries; ++j)
+ write_pgtable(tile, bb, 0, pt_op, &updates[j],
+ pt_update);
+ }
}
- if (!q)
+ if (is_migrate)
mutex_lock(&m->job_mutex);
- job = xe_bb_create_migration_job(q ?: m->q, bb,
+ job = xe_bb_create_migration_job(pt_update_ops->q, bb,
xe_migrate_batch_base(m, usm),
update_idx);
if (IS_ERR(job)) {
@@ -1389,32 +1353,6 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
goto err_bb;
}
- /* Wait on BO move */
- if (bo) {
- err = job_add_deps(job, bo->ttm.base.resv,
- DMA_RESV_USAGE_KERNEL);
- if (err)
- goto err_job;
- }
-
- /*
- * Munmap style VM unbind, need to wait for all jobs to be complete /
- * trigger preempts before moving forward
- */
- if (first_munmap_rebind) {
- err = job_add_deps(job, xe_vm_resv(vm),
- DMA_RESV_USAGE_BOOKKEEP);
- if (err)
- goto err_job;
- }
-
- err = xe_sched_job_last_fence_add_dep(job, vm);
- for (i = 0; !err && i < num_syncs; i++)
- err = xe_sync_entry_add_deps(&syncs[i], job);
-
- if (err)
- goto err_job;
-
if (ops->pre_commit) {
pt_update->job = job;
err = ops->pre_commit(pt_update);
@@ -1425,7 +1363,7 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
fence = dma_fence_get(&job->drm.s_fence->finished);
xe_sched_job_push(job);
- if (!q)
+ if (is_migrate)
mutex_unlock(&m->job_mutex);
xe_bb_free(bb, fence);
@@ -1436,7 +1374,7 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
err_job:
xe_sched_job_put(job);
err_bb:
- if (!q)
+ if (is_migrate)
mutex_unlock(&m->job_mutex);
xe_bb_free(bb, NULL);
err:
@@ -1444,6 +1382,38 @@ xe_migrate_update_pgtables(struct xe_migrate *m,
return ERR_PTR(err);
}
+/**
+ * xe_migrate_update_pgtables() - Pipelined page-table update
+ * @m: The migrate context.
+ * @pt_update: PT update arguments
+ *
+ * Perform a pipelined page-table update. The update descriptors are typically
+ * built under the same lock critical section as a call to this function. If
+ * using the default engine for the updates, they will be performed in the
+ * order they grab the job_mutex. If different engines are used, external
+ * synchronization is needed for overlapping updates to maintain page-table
+ * consistency. Note that the meaing of "overlapping" is that the updates
+ * touch the same page-table, which might be a higher-level page-directory.
+ * If no pipelining is needed, then updates may be performed by the cpu.
+ *
+ * Return: A dma_fence that, when signaled, indicates the update completion.
+ */
+struct dma_fence *
+xe_migrate_update_pgtables(struct xe_migrate *m,
+ struct xe_migrate_pt_update *pt_update)
+
+{
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &pt_update->vops->pt_update_ops[pt_update->tile_id];
+ struct dma_fence *fence;
+
+ fence = xe_migrate_update_pgtables_cpu(m, pt_update);
+ if (!IS_ERR(fence))
+ return fence;
+
+ return __xe_migrate_update_pgtables(m, pt_update, pt_update_ops);
+}
+
/**
* xe_migrate_wait() - Complete all operations using the xe_migrate context
* @m: Migrate context to wait for.
diff --git a/drivers/gpu/drm/xe/xe_migrate.h b/drivers/gpu/drm/xe/xe_migrate.h
index a5bcaafe4a99..453e0ecf5034 100644
--- a/drivers/gpu/drm/xe/xe_migrate.h
+++ b/drivers/gpu/drm/xe/xe_migrate.h
@@ -47,6 +47,24 @@ struct xe_migrate_pt_update_ops {
struct xe_tile *tile, struct iosys_map *map,
void *pos, u32 ofs, u32 num_qwords,
const struct xe_vm_pgtable_update *update);
+ /**
+ * @clear: Clear a command buffer or page-table with ptes.
+ * @pt_update: Embeddable callback argument.
+ * @tile: The tile for the current operation.
+ * @map: struct iosys_map into the memory to be populated.
+ * @pos: If @map is NULL, map into the memory to be populated.
+ * @ofs: qword offset into @map, unused if @map is NULL.
+ * @num_qwords: Number of qwords to write.
+ * @update: Information about the PTEs to be inserted.
+ *
+ * This interface is intended to be used as a callback into the
+ * page-table system to populate command buffers or shared
+ * page-tables with PTEs.
+ */
+ void (*clear)(struct xe_migrate_pt_update *pt_update,
+ struct xe_tile *tile, struct iosys_map *map,
+ void *pos, u32 ofs, u32 num_qwords,
+ const struct xe_vm_pgtable_update *update);
/**
* @pre_commit: Callback to be called just before arming the
@@ -67,14 +85,10 @@ struct xe_migrate_pt_update_ops {
struct xe_migrate_pt_update {
/** @ops: Pointer to the struct xe_migrate_pt_update_ops callbacks */
const struct xe_migrate_pt_update_ops *ops;
- /** @vma: The vma we're updating the pagetable for. */
- struct xe_vma *vma;
+ /** @vops: VMA operations */
+ struct xe_vma_ops *vops;
/** @job: The job if a GPU page-table update. NULL otherwise */
struct xe_sched_job *job;
- /** @start: Start of update for the range fence */
- u64 start;
- /** @last: Last of update for the range fence */
- u64 last;
/** @tile_id: Tile ID of the update */
u8 tile_id;
};
@@ -96,12 +110,6 @@ struct xe_vm *xe_migrate_get_vm(struct xe_migrate *m);
struct dma_fence *
xe_migrate_update_pgtables(struct xe_migrate *m,
- struct xe_vm *vm,
- struct xe_bo *bo,
- struct xe_exec_queue *q,
- const struct xe_vm_pgtable_update *updates,
- u32 num_updates,
- struct xe_sync_entry *syncs, u32 num_syncs,
struct xe_migrate_pt_update *pt_update);
void xe_migrate_wait(struct xe_migrate *m);
diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c
index 8d3765d3351e..78e97c43e526 100644
--- a/drivers/gpu/drm/xe/xe_pt.c
+++ b/drivers/gpu/drm/xe/xe_pt.c
@@ -9,12 +9,14 @@
#include "xe_bo.h"
#include "xe_device.h"
#include "xe_drm_client.h"
+#include "xe_exec_queue.h"
#include "xe_gt.h"
#include "xe_gt_tlb_invalidation.h"
#include "xe_migrate.h"
#include "xe_pt_types.h"
#include "xe_pt_walk.h"
#include "xe_res_cursor.h"
+#include "xe_sync.h"
#include "xe_trace.h"
#include "xe_ttm_stolen_mgr.h"
#include "xe_vm.h"
@@ -325,6 +327,7 @@ xe_pt_new_shared(struct xe_walk_update *wupd, struct xe_pt *parent,
entry->pt = parent;
entry->flags = 0;
entry->qwords = 0;
+ entry->pt_bo->update_index = -1;
if (alloc_entries) {
entry->pt_entries = kmalloc_array(XE_PDES,
@@ -832,9 +835,7 @@ static void xe_pt_commit_locks_assert(struct xe_vma *vma)
lockdep_assert_held(&vm->lock);
- if (xe_vma_is_userptr(vma))
- lockdep_assert_held_read(&vm->userptr.notifier_lock);
- else if (!xe_vma_is_null(vma))
+ if (!xe_vma_is_userptr(vma) && !xe_vma_is_null(vma))
dma_resv_assert_held(xe_vma_bo(vma)->ttm.base.resv);
xe_vm_assert_held(vm);
@@ -856,10 +857,8 @@ static void xe_pt_commit_bind(struct xe_vma *vma,
if (!rebind)
pt->num_live += entries[i].qwords;
- if (!pt->level) {
- kfree(entries[i].pt_entries);
+ if (!pt->level)
continue;
- }
pt_dir = as_xe_pt_dir(pt);
for (j = 0; j < entries[i].qwords; j++) {
@@ -872,10 +871,18 @@ static void xe_pt_commit_bind(struct xe_vma *vma,
pt_dir->children[j_] = &newpte->base;
}
- kfree(entries[i].pt_entries);
}
}
+static void xe_pt_free_bind(struct xe_vm_pgtable_update *entries,
+ u32 num_entries)
+{
+ u32 i;
+
+ for (i = 0; i < num_entries; i++)
+ kfree(entries[i].pt_entries);
+}
+
static int
xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
struct xe_vm_pgtable_update *entries, u32 *num_entries)
@@ -894,12 +901,13 @@ xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
static void xe_vm_dbg_print_entries(struct xe_device *xe,
const struct xe_vm_pgtable_update *entries,
- unsigned int num_entries)
+ unsigned int num_entries, bool bind)
#if (IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM))
{
unsigned int i;
- vm_dbg(&xe->drm, "%u entries to update\n", num_entries);
+ vm_dbg(&xe->drm, "%s: %u entries to update\n", bind ? "bind" : "unbind",
+ num_entries);
for (i = 0; i < num_entries; i++) {
const struct xe_vm_pgtable_update *entry = &entries[i];
struct xe_pt *xe_pt = entry->pt;
@@ -920,66 +928,122 @@ static void xe_vm_dbg_print_entries(struct xe_device *xe,
{}
#endif
-#ifdef CONFIG_DRM_XE_USERPTR_INVAL_INJECT
+static int job_add_deps(struct xe_sched_job *job, struct dma_resv *resv,
+ enum dma_resv_usage usage)
+{
+ return drm_sched_job_add_resv_dependencies(&job->drm, resv, usage);
+}
-static int xe_pt_userptr_inject_eagain(struct xe_userptr_vma *uvma)
+static bool no_in_syncs(struct xe_sync_entry *syncs, u32 num_syncs)
{
- u32 divisor = uvma->userptr.divisor ? uvma->userptr.divisor : 2;
- static u32 count;
+ int i;
- if (count++ % divisor == divisor - 1) {
- struct xe_vm *vm = xe_vma_vm(&uvma->vma);
+ for (i = 0; i < num_syncs; i++) {
+ struct dma_fence *fence = syncs[i].fence;
- uvma->userptr.divisor = divisor << 1;
- spin_lock(&vm->userptr.invalidated_lock);
- list_move_tail(&uvma->userptr.invalidate_link,
- &vm->userptr.invalidated);
- spin_unlock(&vm->userptr.invalidated_lock);
- return true;
+ if (fence && !test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
+ &fence->flags))
+ return false;
}
- return false;
+ return true;
}
-#else
-
-static bool xe_pt_userptr_inject_eagain(struct xe_userptr_vma *uvma)
+static int vma_add_deps(struct xe_vma *vma, struct xe_sched_job *job)
{
- return false;
+ struct xe_bo *bo = xe_vma_bo(vma);
+
+ xe_bo_assert_held(bo);
+
+ if (bo && !bo->vm) {
+ if (!job) {
+ if (!dma_resv_test_signaled(bo->ttm.base.resv,
+ DMA_RESV_USAGE_KERNEL))
+ return -ETIME;
+ } else {
+ return job_add_deps(job, bo->ttm.base.resv,
+ DMA_RESV_USAGE_KERNEL);
+ }
+ }
+
+ return 0;
}
-#endif
+static int op_add_deps(struct xe_vm *vm, struct xe_vma_op *op,
+ struct xe_sched_job *job)
+{
+ int err = 0;
-/**
- * struct xe_pt_migrate_pt_update - Callback argument for pre-commit callbacks
- * @base: Base we derive from.
- * @bind: Whether this is a bind or an unbind operation. A bind operation
- * makes the pre-commit callback error with -EAGAIN if it detects a
- * pending invalidation.
- * @locked: Whether the pre-commit callback locked the userptr notifier lock
- * and it needs unlocking.
- */
-struct xe_pt_migrate_pt_update {
- struct xe_migrate_pt_update base;
- bool bind;
- bool locked;
-};
+ switch (op->base.op) {
+ case DRM_GPUVA_OP_MAP:
+ if (!op->map.immediate && xe_vm_in_fault_mode(vm))
+ break;
+
+ err = vma_add_deps(op->map.vma, job);
+ break;
+ case DRM_GPUVA_OP_REMAP:
+ if (op->remap.prev)
+ err = vma_add_deps(op->remap.prev, job);
+ if (!err && op->remap.next)
+ err = vma_add_deps(op->remap.next, job);
+ break;
+ case DRM_GPUVA_OP_UNMAP:
+ break;
+ case DRM_GPUVA_OP_PREFETCH:
+ err = vma_add_deps(gpuva_to_vma(op->base.prefetch.va), job);
+ break;
+ default:
+ drm_warn(&vm->xe->drm, "NOT POSSIBLE");
+ }
+
+ return err;
+}
-/*
- * This function adds the needed dependencies to a page-table update job
- * to make sure racing jobs for separate bind engines don't race writing
- * to the same page-table range, wreaking havoc. Initially use a single
- * fence for the entire VM. An optimization would use smaller granularity.
- */
static int xe_pt_vm_dependencies(struct xe_sched_job *job,
- struct xe_range_fence_tree *rftree,
- u64 start, u64 last)
+ struct xe_vm *vm,
+ struct xe_vma_ops *vops,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_range_fence_tree *rftree)
{
struct xe_range_fence *rtfence;
struct dma_fence *fence;
- int err;
+ struct xe_vma_op *op;
+ int err = 0, i;
+
+ xe_vm_assert_held(vm);
+
+ if (!job && !no_in_syncs(vops->syncs, vops->num_syncs))
+ return -ETIME;
+
+ if (!job && !xe_exec_queue_is_idle(pt_update_ops->q))
+ return -ETIME;
+
+ if (pt_update_ops->wait_vm_bookkeep) {
+ if (!job) {
+ if (!dma_resv_test_signaled(xe_vm_resv(vm),
+ DMA_RESV_USAGE_BOOKKEEP))
+ return -ETIME;
+ } else {
+ err = job_add_deps(job, xe_vm_resv(vm),
+ DMA_RESV_USAGE_BOOKKEEP);
+ if (err)
+ return err;
+ }
+ } else if (pt_update_ops->wait_vm_kernel) {
+ if (!job) {
+ if (!dma_resv_test_signaled(xe_vm_resv(vm),
+ DMA_RESV_USAGE_KERNEL))
+ return -ETIME;
+ } else {
+ err = job_add_deps(job, xe_vm_resv(vm),
+ DMA_RESV_USAGE_KERNEL);
+ if (err)
+ return err;
+ }
+ }
- rtfence = xe_range_fence_tree_first(rftree, start, last);
+ rtfence = xe_range_fence_tree_first(rftree, pt_update_ops->start,
+ pt_update_ops->last);
while (rtfence) {
fence = rtfence->fence;
@@ -997,80 +1061,166 @@ static int xe_pt_vm_dependencies(struct xe_sched_job *job,
return err;
}
- rtfence = xe_range_fence_tree_next(rtfence, start, last);
+ rtfence = xe_range_fence_tree_next(rtfence,
+ pt_update_ops->start,
+ pt_update_ops->last);
}
- return 0;
+ list_for_each_entry(op, &vops->list, link) {
+ err = op_add_deps(vm, op, job);
+ if (err)
+ return err;
+ }
+
+ for (i = 0; job && !err && i < vops->num_syncs; i++)
+ err = xe_sync_entry_add_deps(&vops->syncs[i], job);
+
+ return err;
}
static int xe_pt_pre_commit(struct xe_migrate_pt_update *pt_update)
{
- struct xe_range_fence_tree *rftree =
- &xe_vma_vm(pt_update->vma)->rftree[pt_update->tile_id];
+ struct xe_vma_ops *vops = pt_update->vops;
+ struct xe_vm *vm = vops->vm;
+ struct xe_range_fence_tree *rftree = &vm->rftree[pt_update->tile_id];
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &vops->pt_update_ops[pt_update->tile_id];
+
+ return xe_pt_vm_dependencies(pt_update->job, vm, pt_update->vops,
+ pt_update_ops, rftree);
+}
+
+#ifdef CONFIG_DRM_XE_USERPTR_INVAL_INJECT
+
+static bool xe_pt_userptr_inject_eagain(struct xe_userptr_vma *uvma)
+{
+ u32 divisor = uvma->userptr.divisor ? uvma->userptr.divisor : 2;
+ static u32 count;
+
+ if (count++ % divisor == divisor - 1) {
+ uvma->userptr.divisor = divisor << 1;
+ return true;
+ }
- return xe_pt_vm_dependencies(pt_update->job, rftree,
- pt_update->start, pt_update->last);
+ return false;
}
-static int xe_pt_userptr_pre_commit(struct xe_migrate_pt_update *pt_update)
+#else
+
+static bool xe_pt_userptr_inject_eagain(struct xe_userptr_vma *uvma)
{
- struct xe_pt_migrate_pt_update *userptr_update =
- container_of(pt_update, typeof(*userptr_update), base);
- struct xe_userptr_vma *uvma = to_userptr_vma(pt_update->vma);
- unsigned long notifier_seq = uvma->userptr.notifier_seq;
- struct xe_vm *vm = xe_vma_vm(&uvma->vma);
- int err = xe_pt_vm_dependencies(pt_update->job,
- &vm->rftree[pt_update->tile_id],
- pt_update->start,
- pt_update->last);
+ return false;
+}
- if (err)
- return err;
+#endif
- userptr_update->locked = false;
+static int vma_check_userptr(struct xe_vm *vm, struct xe_vma *vma,
+ struct xe_vm_pgtable_update_ops *pt_update)
+{
+ struct xe_userptr_vma *uvma;
+ unsigned long notifier_seq;
- /*
- * Wait until nobody is running the invalidation notifier, and
- * since we're exiting the loop holding the notifier lock,
- * nobody can proceed invalidating either.
- *
- * Note that we don't update the vma->userptr.notifier_seq since
- * we don't update the userptr pages.
- */
- do {
- down_read(&vm->userptr.notifier_lock);
- if (!mmu_interval_read_retry(&uvma->userptr.notifier,
- notifier_seq))
- break;
+ lockdep_assert_held_read(&vm->userptr.notifier_lock);
- up_read(&vm->userptr.notifier_lock);
+ if (!xe_vma_is_userptr(vma))
+ return 0;
- if (userptr_update->bind)
- return -EAGAIN;
+ uvma = to_userptr_vma(vma);
+ notifier_seq = uvma->userptr.notifier_seq;
- notifier_seq = mmu_interval_read_begin(&uvma->userptr.notifier);
- } while (true);
+ if (uvma->userptr.initial_bind && !xe_vm_in_fault_mode(vm))
+ return 0;
- /* Inject errors to test_whether they are handled correctly */
- if (userptr_update->bind && xe_pt_userptr_inject_eagain(uvma)) {
- up_read(&vm->userptr.notifier_lock);
+ if (!mmu_interval_read_retry(&uvma->userptr.notifier,
+ notifier_seq) &&
+ !xe_pt_userptr_inject_eagain(uvma))
+ return 0;
+
+ if (xe_vm_in_fault_mode(vm)) {
return -EAGAIN;
- }
+ } else {
+ spin_lock(&vm->userptr.invalidated_lock);
+ list_move_tail(&uvma->userptr.invalidate_link,
+ &vm->userptr.invalidated);
+ spin_unlock(&vm->userptr.invalidated_lock);
- userptr_update->locked = true;
+ if (xe_vm_in_preempt_fence_mode(vm)) {
+ struct dma_resv_iter cursor;
+ struct dma_fence *fence;
+ long err;
+
+ dma_resv_iter_begin(&cursor, xe_vm_resv(vm),
+ DMA_RESV_USAGE_BOOKKEEP);
+ dma_resv_for_each_fence_unlocked(&cursor, fence)
+ dma_fence_enable_sw_signaling(fence);
+ dma_resv_iter_end(&cursor);
+
+ err = dma_resv_wait_timeout(xe_vm_resv(vm),
+ DMA_RESV_USAGE_BOOKKEEP,
+ false, MAX_SCHEDULE_TIMEOUT);
+ XE_WARN_ON(err <= 0);
+ }
+ }
return 0;
}
-static const struct xe_migrate_pt_update_ops bind_ops = {
- .populate = xe_vm_populate_pgtable,
- .pre_commit = xe_pt_pre_commit,
-};
+static int op_check_userptr(struct xe_vm *vm, struct xe_vma_op *op,
+ struct xe_vm_pgtable_update_ops *pt_update)
+{
+ int err = 0;
-static const struct xe_migrate_pt_update_ops userptr_bind_ops = {
- .populate = xe_vm_populate_pgtable,
- .pre_commit = xe_pt_userptr_pre_commit,
-};
+ lockdep_assert_held_read(&vm->userptr.notifier_lock);
+
+ switch (op->base.op) {
+ case DRM_GPUVA_OP_MAP:
+ if (!op->map.immediate && xe_vm_in_fault_mode(vm))
+ break;
+
+ err = vma_check_userptr(vm, op->map.vma, pt_update);
+ break;
+ case DRM_GPUVA_OP_REMAP:
+ if (op->remap.prev)
+ err = vma_check_userptr(vm, op->remap.prev, pt_update);
+ if (!err && op->remap.next)
+ err = vma_check_userptr(vm, op->remap.next, pt_update);
+ break;
+ case DRM_GPUVA_OP_UNMAP:
+ break;
+ case DRM_GPUVA_OP_PREFETCH:
+ err = vma_check_userptr(vm, gpuva_to_vma(op->base.prefetch.va),
+ pt_update);
+ break;
+ default:
+ drm_warn(&vm->xe->drm, "NOT POSSIBLE");
+ }
+
+ return err;
+}
+
+static int xe_pt_userptr_pre_commit(struct xe_migrate_pt_update *pt_update)
+{
+ struct xe_vm *vm = pt_update->vops->vm;
+ struct xe_vma_ops *vops = pt_update->vops;
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &vops->pt_update_ops[pt_update->tile_id];
+ struct xe_vma_op *op;
+ int err;
+
+ err = xe_pt_pre_commit(pt_update);
+ if (err)
+ return err;
+
+ down_read(&vm->userptr.notifier_lock);
+
+ list_for_each_entry(op, &vops->list, link) {
+ err = op_check_userptr(vm, op, pt_update_ops);
+ if (err)
+ break;
+ }
+
+ return err;
+}
struct invalidation_fence {
struct xe_gt_tlb_invalidation_fence base;
@@ -1166,190 +1316,6 @@ static int invalidation_fence_init(struct xe_gt *gt,
return ret && ret != -ENOENT ? ret : 0;
}
-static void xe_pt_calc_rfence_interval(struct xe_vma *vma,
- struct xe_pt_migrate_pt_update *update,
- struct xe_vm_pgtable_update *entries,
- u32 num_entries)
-{
- int i, level = 0;
-
- for (i = 0; i < num_entries; i++) {
- const struct xe_vm_pgtable_update *entry = &entries[i];
-
- if (entry->pt->level > level)
- level = entry->pt->level;
- }
-
- /* Greedy (non-optimal) calculation but simple */
- update->base.start = ALIGN_DOWN(xe_vma_start(vma),
- 0x1ull << xe_pt_shift(level));
- update->base.last = ALIGN(xe_vma_end(vma),
- 0x1ull << xe_pt_shift(level)) - 1;
-}
-
-/**
- * __xe_pt_bind_vma() - Build and connect a page-table tree for the vma
- * address range.
- * @tile: The tile to bind for.
- * @vma: The vma to bind.
- * @q: The exec_queue with which to do pipelined page-table updates.
- * @syncs: Entries to sync on before binding the built tree to the live vm tree.
- * @num_syncs: Number of @sync entries.
- * @rebind: Whether we're rebinding this vma to the same address range without
- * an unbind in-between.
- *
- * This function builds a page-table tree (see xe_pt_stage_bind() for more
- * information on page-table building), and the xe_vm_pgtable_update entries
- * abstracting the operations needed to attach it to the main vm tree. It
- * then takes the relevant locks and updates the metadata side of the main
- * vm tree and submits the operations for pipelined attachment of the
- * gpu page-table to the vm main tree, (which can be done either by the
- * cpu and the GPU).
- *
- * Return: A valid dma-fence representing the pipelined attachment operation
- * on success, an error pointer on error.
- */
-struct dma_fence *
-__xe_pt_bind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs,
- bool rebind)
-{
- struct xe_vm_pgtable_update entries[XE_VM_MAX_LEVEL * 2 + 1];
- struct xe_pt_migrate_pt_update bind_pt_update = {
- .base = {
- .ops = xe_vma_is_userptr(vma) ? &userptr_bind_ops : &bind_ops,
- .vma = vma,
- .tile_id = tile->id,
- },
- .bind = true,
- };
- struct xe_vm *vm = xe_vma_vm(vma);
- u32 num_entries;
- struct dma_fence *fence;
- struct invalidation_fence *ifence = NULL;
- struct xe_range_fence *rfence;
- int err;
-
- bind_pt_update.locked = false;
- xe_bo_assert_held(xe_vma_bo(vma));
- xe_vm_assert_held(vm);
-
- vm_dbg(&xe_vma_vm(vma)->xe->drm,
- "Preparing bind, with range [%llx...%llx) engine %p.\n",
- xe_vma_start(vma), xe_vma_end(vma), q);
-
- err = xe_pt_prepare_bind(tile, vma, entries, &num_entries);
- if (err)
- goto err;
-
- err = dma_resv_reserve_fences(xe_vm_resv(vm), 1);
- if (!err && !xe_vma_has_no_bo(vma) && !xe_vma_bo(vma)->vm)
- err = dma_resv_reserve_fences(xe_vma_bo(vma)->ttm.base.resv, 1);
- if (err)
- goto err;
-
- xe_tile_assert(tile, num_entries <= ARRAY_SIZE(entries));
-
- xe_vm_dbg_print_entries(tile_to_xe(tile), entries, num_entries);
- xe_pt_calc_rfence_interval(vma, &bind_pt_update, entries,
- num_entries);
-
- /*
- * If rebind, we have to invalidate TLB on !LR vms to invalidate
- * cached PTEs point to freed memory. on LR vms this is done
- * automatically when the context is re-enabled by the rebind worker,
- * or in fault mode it was invalidated on PTE zapping.
- *
- * If !rebind, and scratch enabled VMs, there is a chance the scratch
- * PTE is already cached in the TLB so it needs to be invalidated.
- * on !LR VMs this is done in the ring ops preceding a batch, but on
- * non-faulting LR, in particular on user-space batch buffer chaining,
- * it needs to be done here.
- */
- if ((!rebind && xe_vm_has_scratch(vm) && xe_vm_in_preempt_fence_mode(vm))) {
- ifence = kzalloc(sizeof(*ifence), GFP_KERNEL);
- if (!ifence)
- return ERR_PTR(-ENOMEM);
- } else if (rebind && !xe_vm_in_lr_mode(vm)) {
- /* We bump also if batch_invalidate_tlb is true */
- vm->tlb_flush_seqno++;
- }
-
- rfence = kzalloc(sizeof(*rfence), GFP_KERNEL);
- if (!rfence) {
- kfree(ifence);
- return ERR_PTR(-ENOMEM);
- }
-
- fence = xe_migrate_update_pgtables(tile->migrate,
- vm, xe_vma_bo(vma), q,
- entries, num_entries,
- syncs, num_syncs,
- &bind_pt_update.base);
- if (!IS_ERR(fence)) {
- bool last_munmap_rebind = vma->gpuva.flags & XE_VMA_LAST_REBIND;
- LLIST_HEAD(deferred);
- int err;
-
- err = xe_range_fence_insert(&vm->rftree[tile->id], rfence,
- &xe_range_fence_kfree_ops,
- bind_pt_update.base.start,
- bind_pt_update.base.last, fence);
- if (err)
- dma_fence_wait(fence, false);
-
- /* TLB invalidation must be done before signaling rebind */
- if (ifence) {
- int err = invalidation_fence_init(tile->primary_gt,
- ifence, fence,
- xe_vma_start(vma),
- xe_vma_end(vma),
- xe_vma_vm(vma)->usm.asid);
- if (err) {
- dma_fence_put(fence);
- kfree(ifence);
- return ERR_PTR(err);
- }
- fence = &ifence->base.base;
- }
-
- /* add shared fence now for pagetable delayed destroy */
- dma_resv_add_fence(xe_vm_resv(vm), fence, rebind ||
- last_munmap_rebind ?
- DMA_RESV_USAGE_KERNEL :
- DMA_RESV_USAGE_BOOKKEEP);
-
- if (!xe_vma_has_no_bo(vma) && !xe_vma_bo(vma)->vm)
- dma_resv_add_fence(xe_vma_bo(vma)->ttm.base.resv, fence,
- DMA_RESV_USAGE_BOOKKEEP);
- xe_pt_commit_bind(vma, entries, num_entries, rebind,
- bind_pt_update.locked ? &deferred : NULL);
-
- /* This vma is live (again?) now */
- vma->tile_present |= BIT(tile->id);
-
- if (bind_pt_update.locked) {
- to_userptr_vma(vma)->userptr.initial_bind = true;
- up_read(&vm->userptr.notifier_lock);
- xe_bo_put_commit(&deferred);
- }
- if (!rebind && last_munmap_rebind &&
- xe_vm_in_preempt_fence_mode(vm))
- xe_vm_queue_rebind_worker(vm);
- } else {
- kfree(rfence);
- kfree(ifence);
- if (bind_pt_update.locked)
- up_read(&vm->userptr.notifier_lock);
- xe_pt_abort_bind(vma, entries, num_entries);
- }
-
- return fence;
-
-err:
- return ERR_PTR(err);
-}
-
struct xe_pt_stage_unbind_walk {
/** @base: The pagewalk base-class. */
struct xe_pt_walk base;
@@ -1500,8 +1466,8 @@ xe_migrate_clear_pgtable_callback(struct xe_migrate_pt_update *pt_update,
void *ptr, u32 qword_ofs, u32 num_qwords,
const struct xe_vm_pgtable_update *update)
{
- struct xe_vma *vma = pt_update->vma;
- u64 empty = __xe_pt_empty_pte(tile, xe_vma_vm(vma), update->pt->level);
+ struct xe_vm *vm = pt_update->vops->vm;
+ u64 empty = __xe_pt_empty_pte(tile, vm, update->pt->level);
int i;
if (map && map->is_iomem)
@@ -1545,151 +1511,464 @@ xe_pt_commit_unbind(struct xe_vma *vma,
}
}
-static const struct xe_migrate_pt_update_ops unbind_ops = {
- .populate = xe_migrate_clear_pgtable_callback,
+static void
+xe_pt_update_ops_rfence_interval(struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma *vma)
+{
+ u32 current_op = pt_update_ops->current_op;
+ struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[current_op];
+ int i, level = 0;
+ u64 start, last;
+
+ for (i = 0; i < pt_op->num_entries; i++) {
+ const struct xe_vm_pgtable_update *entry = &pt_op->entries[i];
+
+ if (entry->pt->level > level)
+ level = entry->pt->level;
+ }
+
+ /* Greedy (non-optimal) calculation but simple */
+ start = ALIGN_DOWN(xe_vma_start(vma), 0x1ull << xe_pt_shift(level));
+ last = ALIGN(xe_vma_end(vma), 0x1ull << xe_pt_shift(level)) - 1;
+
+ if (start < pt_update_ops->start)
+ pt_update_ops->start = start;
+ if (last > pt_update_ops->last)
+ pt_update_ops->last = last;
+}
+
+static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma *vma)
+{
+ u32 current_op = pt_update_ops->current_op;
+ struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[current_op];
+ struct llist_head *deferred = &pt_update_ops->deferred;
+ int err;
+
+ xe_bo_assert_held(xe_vma_bo(vma));
+
+ vm_dbg(&xe_vma_vm(vma)->xe->drm,
+ "Preparing bind, with range [%llx...%llx)\n",
+ xe_vma_start(vma), xe_vma_end(vma) - 1);
+
+ pt_op->vma = NULL;
+ pt_op->bind = true;
+ pt_op->rebind = BIT(tile->id) & vma->tile_present;
+
+ err = xe_pt_prepare_bind(tile, vma, pt_op->entries,
+ &pt_op->num_entries);
+ if (!err) {
+ xe_tile_assert(tile, pt_op->num_entries <=
+ ARRAY_SIZE(pt_op->entries));
+ xe_vm_dbg_print_entries(tile_to_xe(tile), pt_op->entries,
+ pt_op->num_entries, true);
+
+ xe_pt_update_ops_rfence_interval(pt_update_ops, vma);
+ ++pt_update_ops->current_op;
+ pt_update_ops->needs_userptr_lock |= xe_vma_is_userptr(vma);
+
+ /*
+ * If rebind, we have to invalidate TLB on !LR vms to invalidate
+ * cached PTEs point to freed memory. on LR vms this is done
+ * automatically when the context is re-enabled by the rebind
+ * worker, or in fault mode it was invalidated on PTE zapping.
+ *
+ * If !rebind, and scratch enabled VMs, there is a chance the
+ * scratch PTE is already cached in the TLB so it needs to be
+ * invalidated. on !LR VMs this is done in the ring ops
+ * preceding a batch, but on non-faulting LR, in particular on
+ * user-space batch buffer chaining, it needs to be done here.
+ */
+ pt_update_ops->needs_invalidation |=
+ (pt_op->rebind && !xe_vm_in_lr_mode(vm) &&
+ !vm->batch_invalidate_tlb) ||
+ (!pt_op->rebind && vm->scratch_pt[tile->id] &&
+ xe_vm_in_preempt_fence_mode(vm));
+
+ /* FIXME: Don't commit right away */
+ vma->tile_staged |= BIT(tile->id);
+ pt_op->vma = vma;
+ xe_pt_commit_bind(vma, pt_op->entries, pt_op->num_entries,
+ pt_op->rebind, deferred);
+ }
+
+ return err;
+}
+
+static int unbind_op_prepare(struct xe_tile *tile,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma *vma)
+{
+ u32 current_op = pt_update_ops->current_op;
+ struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[current_op];
+ struct llist_head *deferred = &pt_update_ops->deferred;
+
+ if (!((vma->tile_present | vma->tile_staged) & BIT(tile->id)))
+ return 0;
+
+ xe_bo_assert_held(xe_vma_bo(vma));
+
+ vm_dbg(&xe_vma_vm(vma)->xe->drm,
+ "Preparing unbind, with range [%llx...%llx)\n",
+ xe_vma_start(vma), xe_vma_end(vma) - 1);
+
+ /*
+ * Wait for invalidation to complete. Can corrupt internal page table
+ * state if an invalidation is running while preparing an unbind.
+ */
+ if (xe_vma_is_userptr(vma) && xe_vm_in_fault_mode(xe_vma_vm(vma)))
+ mmu_interval_read_begin(&to_userptr_vma(vma)->userptr.notifier);
+
+ pt_op->vma = vma;
+ pt_op->bind = false;
+ pt_op->rebind = false;
+
+ pt_op->num_entries = xe_pt_stage_unbind(tile, vma, pt_op->entries);
+
+ xe_vm_dbg_print_entries(tile_to_xe(tile), pt_op->entries,
+ pt_op->num_entries, false);
+ xe_pt_update_ops_rfence_interval(pt_update_ops, vma);
+ ++pt_update_ops->current_op;
+ pt_update_ops->needs_userptr_lock |= xe_vma_is_userptr(vma);
+ pt_update_ops->needs_invalidation = true;
+
+ /* FIXME: Don't commit right away */
+ xe_pt_commit_unbind(vma, pt_op->entries, pt_op->num_entries,
+ deferred);
+
+ return 0;
+}
+
+static int op_prepare(struct xe_vm *vm,
+ struct xe_tile *tile,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma_op *op)
+{
+ int err = 0;
+
+ xe_vm_assert_held(vm);
+
+ switch (op->base.op) {
+ case DRM_GPUVA_OP_MAP:
+ if (!op->map.immediate && xe_vm_in_fault_mode(vm))
+ break;
+
+ err = bind_op_prepare(vm, tile, pt_update_ops, op->map.vma);
+ pt_update_ops->wait_vm_kernel = true;
+ break;
+ case DRM_GPUVA_OP_REMAP:
+ err = unbind_op_prepare(tile, pt_update_ops,
+ gpuva_to_vma(op->base.remap.unmap->va));
+
+ if (!err && op->remap.prev) {
+ err = bind_op_prepare(vm, tile, pt_update_ops,
+ op->remap.prev);
+ pt_update_ops->wait_vm_bookkeep = true;
+ }
+ if (!err && op->remap.next) {
+ err = bind_op_prepare(vm, tile, pt_update_ops,
+ op->remap.next);
+ pt_update_ops->wait_vm_bookkeep = true;
+ }
+ break;
+ case DRM_GPUVA_OP_UNMAP:
+ err = unbind_op_prepare(tile, pt_update_ops,
+ gpuva_to_vma(op->base.unmap.va));
+ break;
+ case DRM_GPUVA_OP_PREFETCH:
+ err = bind_op_prepare(vm, tile, pt_update_ops,
+ gpuva_to_vma(op->base.prefetch.va));
+ pt_update_ops->wait_vm_kernel = true;
+ break;
+ default:
+ drm_warn(&vm->xe->drm, "NOT POSSIBLE");
+ }
+
+ return err;
+}
+
+static void
+xe_pt_update_ops_init(struct xe_vm_pgtable_update_ops *pt_update_ops)
+{
+ init_llist_head(&pt_update_ops->deferred);
+ pt_update_ops->start = ~0x0ull;
+ pt_update_ops->last = 0x0ull;
+}
+
+/**
+ * xe_pt_update_ops_prepare() - Prepare PT update operations
+ * @tile: Tile of PT update operations
+ * @vops: VMA operationa
+ *
+ * Prepare PT update operations which includes updating internal PT state,
+ * allocate memory for page tables, populate page table being pruned in, and
+ * create PT update operations for leaf insertion / removal.
+ *
+ * Return: 0 on success, negative error code on error.
+ */
+int xe_pt_update_ops_prepare(struct xe_tile *tile, struct xe_vma_ops *vops)
+{
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &vops->pt_update_ops[tile->id];
+ struct xe_vma_op *op;
+ int err;
+
+ lockdep_assert_held(&vops->vm->lock);
+ xe_vm_assert_held(vops->vm);
+
+ xe_pt_update_ops_init(pt_update_ops);
+
+ list_for_each_entry(op, &vops->list, link) {
+ err = op_prepare(vops->vm, tile, pt_update_ops, op);
+
+ if (err)
+ return err;
+ }
+
+ xe_tile_assert(tile, pt_update_ops->current_op <=
+ pt_update_ops->num_ops);
+
+ return 0;
+}
+
+static void bind_op_commit(struct xe_vm *vm, struct xe_tile *tile,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma *vma, struct dma_fence *fence)
+{
+ if (!xe_vma_has_no_bo(vma) && !xe_vma_bo(vma)->vm)
+ dma_resv_add_fence(xe_vma_bo(vma)->ttm.base.resv, fence,
+ pt_update_ops->wait_vm_bookkeep ?
+ DMA_RESV_USAGE_KERNEL :
+ DMA_RESV_USAGE_BOOKKEEP);
+ vma->tile_present |= BIT(tile->id);
+ vma->tile_staged &= ~BIT(tile->id);
+ if (xe_vma_is_userptr(vma)) {
+ lockdep_assert_held_read(&vm->userptr.notifier_lock);
+ to_userptr_vma(vma)->userptr.initial_bind = true;
+ }
+
+ /*
+ * Kick rebind worker if this bind triggers preempt fences and not in
+ * the rebind worker
+ */
+ if (pt_update_ops->wait_vm_bookkeep &&
+ xe_vm_in_preempt_fence_mode(vm) &&
+ !current->mm)
+ xe_vm_queue_rebind_worker(vm);
+}
+
+static void unbind_op_commit(struct xe_vm *vm, struct xe_tile *tile,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma *vma, struct dma_fence *fence)
+{
+ if (!xe_vma_has_no_bo(vma) && !xe_vma_bo(vma)->vm)
+ dma_resv_add_fence(xe_vma_bo(vma)->ttm.base.resv, fence,
+ pt_update_ops->wait_vm_bookkeep ?
+ DMA_RESV_USAGE_KERNEL :
+ DMA_RESV_USAGE_BOOKKEEP);
+ vma->tile_present &= ~BIT(tile->id);
+ if (!vma->tile_present) {
+ list_del_init(&vma->combined_links.rebind);
+ if (xe_vma_is_userptr(vma)) {
+ lockdep_assert_held_read(&vm->userptr.notifier_lock);
+
+ spin_lock(&vm->userptr.invalidated_lock);
+ list_del_init(&to_userptr_vma(vma)->userptr.invalidate_link);
+ spin_unlock(&vm->userptr.invalidated_lock);
+ }
+ }
+}
+
+static void op_commit(struct xe_vm *vm,
+ struct xe_tile *tile,
+ struct xe_vm_pgtable_update_ops *pt_update_ops,
+ struct xe_vma_op *op, struct dma_fence *fence)
+{
+ xe_vm_assert_held(vm);
+
+ switch (op->base.op) {
+ case DRM_GPUVA_OP_MAP:
+ if (!op->map.immediate && xe_vm_in_fault_mode(vm))
+ break;
+
+ bind_op_commit(vm, tile, pt_update_ops, op->map.vma, fence);
+ break;
+ case DRM_GPUVA_OP_REMAP:
+ unbind_op_commit(vm, tile, pt_update_ops,
+ gpuva_to_vma(op->base.remap.unmap->va), fence);
+
+ if (op->remap.prev)
+ bind_op_commit(vm, tile, pt_update_ops, op->remap.prev,
+ fence);
+ if (op->remap.next)
+ bind_op_commit(vm, tile, pt_update_ops, op->remap.next,
+ fence);
+ break;
+ case DRM_GPUVA_OP_UNMAP:
+ unbind_op_commit(vm, tile, pt_update_ops,
+ gpuva_to_vma(op->base.unmap.va), fence);
+ break;
+ case DRM_GPUVA_OP_PREFETCH:
+ bind_op_commit(vm, tile, pt_update_ops,
+ gpuva_to_vma(op->base.prefetch.va), fence);
+ break;
+ default:
+ drm_warn(&vm->xe->drm, "NOT POSSIBLE");
+ }
+}
+
+static const struct xe_migrate_pt_update_ops migrate_ops = {
+ .populate = xe_vm_populate_pgtable,
+ .clear = xe_migrate_clear_pgtable_callback,
.pre_commit = xe_pt_pre_commit,
};
-static const struct xe_migrate_pt_update_ops userptr_unbind_ops = {
- .populate = xe_migrate_clear_pgtable_callback,
+static const struct xe_migrate_pt_update_ops userptr_migrate_ops = {
+ .populate = xe_vm_populate_pgtable,
+ .clear = xe_migrate_clear_pgtable_callback,
.pre_commit = xe_pt_userptr_pre_commit,
};
/**
- * __xe_pt_unbind_vma() - Disconnect and free a page-table tree for the vma
- * address range.
- * @tile: The tile to unbind for.
- * @vma: The vma to unbind.
- * @q: The exec_queue with which to do pipelined page-table updates.
- * @syncs: Entries to sync on before disconnecting the tree to be destroyed.
- * @num_syncs: Number of @sync entries.
+ * xe_pt_update_ops_run() - Run PT update operations
+ * @tile: Tile of PT update operations
+ * @vops: VMA operationa
*
- * This function builds a the xe_vm_pgtable_update entries abstracting the
- * operations needed to detach the page-table tree to be destroyed from the
- * man vm tree.
- * It then takes the relevant locks and submits the operations for
- * pipelined detachment of the gpu page-table from the vm main tree,
- * (which can be done either by the cpu and the GPU), Finally it frees the
- * detached page-table tree.
- *
- * Return: A valid dma-fence representing the pipelined detachment operation
- * on success, an error pointer on error.
+ * Run PT update operations which includes committing internal PT state changes,
+ * creating job for PT update operations for leaf insertion / removal, and
+ * installing job fence in various places.
+ *
+ * Return: fence on success, negative ERR_PTR on error.
*/
struct dma_fence *
-__xe_pt_unbind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs)
+xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops)
{
- struct xe_vm_pgtable_update entries[XE_VM_MAX_LEVEL * 2 + 1];
- struct xe_pt_migrate_pt_update unbind_pt_update = {
- .base = {
- .ops = xe_vma_is_userptr(vma) ? &userptr_unbind_ops :
- &unbind_ops,
- .vma = vma,
- .tile_id = tile->id,
- },
- };
- struct xe_vm *vm = xe_vma_vm(vma);
- u32 num_entries;
- struct dma_fence *fence = NULL;
- struct invalidation_fence *ifence;
+ struct xe_vm *vm = vops->vm;
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &vops->pt_update_ops[tile->id];
+ struct dma_fence *fence;
+ struct invalidation_fence *ifence = NULL;
struct xe_range_fence *rfence;
- int err;
-
- LLIST_HEAD(deferred);
+ struct xe_vma_op *op;
+ int err = 0;
+ struct xe_migrate_pt_update update = {
+ .ops = pt_update_ops->needs_userptr_lock ?
+ &userptr_migrate_ops :
+ &migrate_ops,
+ .vops = vops,
+ .tile_id = tile->id
+ };
- xe_bo_assert_held(xe_vma_bo(vma));
+ lockdep_assert_held(&vm->lock);
xe_vm_assert_held(vm);
- vm_dbg(&xe_vma_vm(vma)->xe->drm,
- "Preparing unbind, with range [%llx...%llx) engine %p.\n",
- xe_vma_start(vma), xe_vma_end(vma), q);
-
- num_entries = xe_pt_stage_unbind(tile, vma, entries);
- xe_tile_assert(tile, num_entries <= ARRAY_SIZE(entries));
-
- xe_vm_dbg_print_entries(tile_to_xe(tile), entries, num_entries);
- xe_pt_calc_rfence_interval(vma, &unbind_pt_update, entries,
- num_entries);
+ if (!pt_update_ops->current_op) {
+ xe_tile_assert(tile, xe_vm_in_fault_mode(vm));
- err = dma_resv_reserve_fences(xe_vm_resv(vm), 1);
- if (!err && !xe_vma_has_no_bo(vma) && !xe_vma_bo(vma)->vm)
- err = dma_resv_reserve_fences(xe_vma_bo(vma)->ttm.base.resv, 1);
- if (err)
- return ERR_PTR(err);
+ return dma_fence_get_stub();
+ }
- ifence = kzalloc(sizeof(*ifence), GFP_KERNEL);
- if (!ifence)
- return ERR_PTR(-ENOMEM);
+ if (pt_update_ops->needs_invalidation) {
+ ifence = kzalloc(sizeof(*ifence), GFP_KERNEL);
+ if (!ifence)
+ return ERR_PTR(-ENOMEM);
+ }
rfence = kzalloc(sizeof(*rfence), GFP_KERNEL);
if (!rfence) {
- kfree(ifence);
- return ERR_PTR(-ENOMEM);
+ err = -ENOMEM;
+ goto free_ifence;
}
- /*
- * Even if we were already evicted and unbind to destroy, we need to
- * clear again here. The eviction may have updated pagetables at a
- * lower level, because it needs to be more conservative.
- */
- fence = xe_migrate_update_pgtables(tile->migrate,
- vm, NULL, q ? q :
- vm->q[tile->id],
- entries, num_entries,
- syncs, num_syncs,
- &unbind_pt_update.base);
- if (!IS_ERR(fence)) {
- int err;
-
- err = xe_range_fence_insert(&vm->rftree[tile->id], rfence,
- &xe_range_fence_kfree_ops,
- unbind_pt_update.base.start,
- unbind_pt_update.base.last, fence);
- if (err)
- dma_fence_wait(fence, false);
+ fence = xe_migrate_update_pgtables(tile->migrate, &update);
+ if (IS_ERR(fence)) {
+ err = PTR_ERR(fence);
+ goto free_rfence;
+ }
- /* TLB invalidation must be done before signaling unbind */
+ err = xe_range_fence_insert(&vm->rftree[tile->id], rfence,
+ &xe_range_fence_kfree_ops,
+ pt_update_ops->start,
+ pt_update_ops->last, fence);
+ if (err)
+ dma_fence_wait(fence, false);
+
+ /* tlb invalidation must be done before signaling rebind */
+ if (ifence) {
err = invalidation_fence_init(tile->primary_gt, ifence, fence,
- xe_vma_start(vma),
- xe_vma_end(vma),
- xe_vma_vm(vma)->usm.asid);
- if (err) {
- dma_fence_put(fence);
- kfree(ifence);
- return ERR_PTR(err);
- }
+ pt_update_ops->start,
+ pt_update_ops->last,
+ vm->usm.asid);
+ if (err)
+ goto put_fence;
fence = &ifence->base.base;
-
- /* add shared fence now for pagetable delayed destroy */
- dma_resv_add_fence(xe_vm_resv(vm), fence,
- DMA_RESV_USAGE_BOOKKEEP);
-
- /* This fence will be installed by caller when doing eviction */
- if (!xe_vma_has_no_bo(vma) && !xe_vma_bo(vma)->vm)
- dma_resv_add_fence(xe_vma_bo(vma)->ttm.base.resv, fence,
- DMA_RESV_USAGE_BOOKKEEP);
- xe_pt_commit_unbind(vma, entries, num_entries,
- unbind_pt_update.locked ? &deferred : NULL);
- vma->tile_present &= ~BIT(tile->id);
- } else {
- kfree(rfence);
- kfree(ifence);
}
- if (!vma->tile_present)
- list_del_init(&vma->combined_links.rebind);
+ dma_resv_add_fence(xe_vm_resv(vm), fence,
+ pt_update_ops->wait_vm_bookkeep ?
+ DMA_RESV_USAGE_KERNEL :
+ DMA_RESV_USAGE_BOOKKEEP);
- if (unbind_pt_update.locked) {
- xe_tile_assert(tile, xe_vma_is_userptr(vma));
+ list_for_each_entry(op, &vops->list, link)
+ op_commit(vops->vm, tile, pt_update_ops, op, fence);
- if (!vma->tile_present) {
- spin_lock(&vm->userptr.invalidated_lock);
- list_del_init(&to_userptr_vma(vma)->userptr.invalidate_link);
- spin_unlock(&vm->userptr.invalidated_lock);
- }
+ if (pt_update_ops->needs_userptr_lock)
up_read(&vm->userptr.notifier_lock);
- xe_bo_put_commit(&deferred);
- }
return fence;
+
+put_fence:
+ if (pt_update_ops->needs_userptr_lock)
+ up_read(&vm->userptr.notifier_lock);
+ dma_fence_put(fence);
+free_rfence:
+ kfree(rfence);
+free_ifence:
+ kfree(ifence);
+
+ return ERR_PTR(err);
+}
+
+/**
+ * xe_pt_update_ops_fini() - Finish PT update operations
+ * @tile: Tile of PT update operations
+ * @vops: VMA operations
+ *
+ * Finish PT update operations by committing to destroy page table memory
+ */
+void xe_pt_update_ops_fini(struct xe_tile *tile, struct xe_vma_ops *vops)
+{
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &vops->pt_update_ops[tile->id];
+ int i;
+
+ lockdep_assert_held(&vops->vm->lock);
+ xe_vm_assert_held(vops->vm);
+
+ /* FIXME: Not 100% correct */
+ for (i = 0; i < pt_update_ops->num_ops; ++i) {
+ struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[i];
+
+ if (pt_op->bind)
+ xe_pt_free_bind(pt_op->entries, pt_op->num_entries);
+ }
+ xe_bo_put_commit(&vops->pt_update_ops[tile->id].deferred);
}
+
+/**
+ * xe_pt_update_ops_abort() - Abort PT update operations
+ * @tile: Tile of PT update operations
+ * @vops: VMA operationa
+ *
+ * Abort PT update operations by unwinding internal PT state
+ */
+void xe_pt_update_ops_abort(struct xe_tile *tile, struct xe_vma_ops *vops)
+{
+ lockdep_assert_held(&vops->vm->lock);
+ xe_vm_assert_held(vops->vm);
+
+ /* FIXME: Just kill VM for now + cleanup PTs */
+ xe_bo_put_commit(&vops->pt_update_ops[tile->id].deferred);
+ xe_vm_kill(vops->vm, false);
+ }
diff --git a/drivers/gpu/drm/xe/xe_pt.h b/drivers/gpu/drm/xe/xe_pt.h
index 71a4fbfcff43..9ab386431cad 100644
--- a/drivers/gpu/drm/xe/xe_pt.h
+++ b/drivers/gpu/drm/xe/xe_pt.h
@@ -17,6 +17,7 @@ struct xe_sync_entry;
struct xe_tile;
struct xe_vm;
struct xe_vma;
+struct xe_vma_ops;
/* Largest huge pte is currently 1GiB. May become device dependent. */
#define MAX_HUGEPTE_LEVEL 2
@@ -34,14 +35,11 @@ void xe_pt_populate_empty(struct xe_tile *tile, struct xe_vm *vm,
void xe_pt_destroy(struct xe_pt *pt, u32 flags, struct llist_head *deferred);
-struct dma_fence *
-__xe_pt_bind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs,
- bool rebind);
-
-struct dma_fence *
-__xe_pt_unbind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs);
+int xe_pt_update_ops_prepare(struct xe_tile *tile, struct xe_vma_ops *vops);
+struct dma_fence *xe_pt_update_ops_run(struct xe_tile *tile,
+ struct xe_vma_ops *vops);
+void xe_pt_update_ops_fini(struct xe_tile *tile, struct xe_vma_ops *vops);
+void xe_pt_update_ops_abort(struct xe_tile *tile, struct xe_vma_ops *vops);
bool xe_pt_zap_ptes(struct xe_tile *tile, struct xe_vma *vma);
diff --git a/drivers/gpu/drm/xe/xe_pt_types.h b/drivers/gpu/drm/xe/xe_pt_types.h
index 2093150f461e..384cc04de719 100644
--- a/drivers/gpu/drm/xe/xe_pt_types.h
+++ b/drivers/gpu/drm/xe/xe_pt_types.h
@@ -78,6 +78,8 @@ struct xe_vm_pgtable_update {
struct xe_vm_pgtable_update_op {
/** @entries: entries to update for this operation */
struct xe_vm_pgtable_update entries[XE_VM_MAX_LEVEL * 2 + 1];
+ /** @vma: VMA for operation, operation not valid if NULL */
+ struct xe_vma *vma;
/** @num_entries: number of entries for this update operation */
u32 num_entries;
/** @bind: is a bind */
@@ -86,4 +88,38 @@ struct xe_vm_pgtable_update_op {
bool rebind;
};
+/** struct xe_vm_pgtable_update_ops: page table update operations */
+struct xe_vm_pgtable_update_ops {
+ /** @ops: operations */
+ struct xe_vm_pgtable_update_op *ops;
+ /** @deferred: deferred list to destroy PT entries */
+ struct llist_head deferred;
+ /** @q: exec queue for PT operations */
+ struct xe_exec_queue *q;
+ /** @start: start address of ops */
+ u64 start;
+ /** @last: last address of ops */
+ u64 last;
+ /** @num_ops: number of operations */
+ u32 num_ops;
+ /** @current_op: current operations */
+ u32 current_op;
+ /** @needs_userptr_lock: Needs userptr lock */
+ bool needs_userptr_lock;
+ /** @needs_invalidation: Needs invalidation */
+ bool needs_invalidation;
+ /**
+ * @wait_vm_bookkeep: PT operations need to wait until VM is idle
+ * (bookkeep dma-resv slots are idle) and stage all future VM activity
+ * behind these operations (install PT operations into VM kernel
+ * dma-resv slot).
+ */
+ bool wait_vm_bookkeep;
+ /**
+ * @wait_vm_kernel: PT operations need to wait until VM kernel dma-resv
+ * slots are idle.
+ */
+ bool wait_vm_kernel;
+};
+
#endif
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index b60ae743d2e6..aef0ed5f9571 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -315,7 +315,7 @@ int __xe_vm_userptr_needs_repin(struct xe_vm *vm)
#define XE_VM_REBIND_RETRY_TIMEOUT_MS 1000
-static void xe_vm_kill(struct xe_vm *vm, bool unlocked)
+void xe_vm_kill(struct xe_vm *vm, bool unlocked)
{
struct xe_exec_queue *q;
@@ -792,7 +792,7 @@ int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)
struct xe_vma *vma, *next;
struct xe_vma_ops vops;
struct xe_vma_op *op, *next_op;
- int err;
+ int err, i;
lockdep_assert_held(&vm->lock);
if ((xe_vm_in_lr_mode(vm) && !rebind_worker) ||
@@ -800,6 +800,8 @@ int xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)
return 0;
xe_vma_ops_init(&vops, vm, NULL, NULL, 0);
+ for (i = 0; i < XE_MAX_TILES_PER_DEVICE; ++i)
+ vops.pt_update_ops[i].wait_vm_bookkeep = true;
xe_vm_assert_held(vm);
list_for_each_entry(vma, &vm->rebind_list, combined_links.rebind) {
@@ -844,6 +846,8 @@ struct dma_fence *xe_vma_rebind(struct xe_vm *vm, struct xe_vma *vma, u8 tile_ma
struct dma_fence *fence = NULL;
struct xe_vma_ops vops;
struct xe_vma_op *op, *next_op;
+ struct xe_tile *tile;
+ u8 id;
int err;
lockdep_assert_held(&vm->lock);
@@ -851,6 +855,11 @@ struct dma_fence *xe_vma_rebind(struct xe_vm *vm, struct xe_vma *vma, u8 tile_ma
xe_assert(vm->xe, xe_vm_in_fault_mode(vm));
xe_vma_ops_init(&vops, vm, NULL, NULL, 0);
+ for_each_tile(tile, vm->xe, id) {
+ vops.pt_update_ops[id].wait_vm_bookkeep = true;
+ vops.pt_update_ops[tile->id].q =
+ xe_tile_migrate_exec_queue(tile);
+ }
err = xe_vm_ops_add_rebind(&vops, vma, tile_mask);
if (err)
@@ -1685,147 +1694,6 @@ to_wait_exec_queue(struct xe_vm *vm, struct xe_exec_queue *q)
return q ? q : vm->q[0];
}
-static struct dma_fence *
-xe_vm_unbind_vma(struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs,
- bool first_op, bool last_op)
-{
- struct xe_vm *vm = xe_vma_vm(vma);
- struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
- struct xe_tile *tile;
- struct dma_fence *fence = NULL;
- struct dma_fence **fences = NULL;
- struct dma_fence_array *cf = NULL;
- int cur_fence = 0;
- int number_tiles = hweight8(vma->tile_present);
- int err;
- u8 id;
-
- trace_xe_vma_unbind(vma);
-
- if (number_tiles > 1) {
- fences = kmalloc_array(number_tiles, sizeof(*fences),
- GFP_KERNEL);
- if (!fences)
- return ERR_PTR(-ENOMEM);
- }
-
- for_each_tile(tile, vm->xe, id) {
- if (!(vma->tile_present & BIT(id)))
- goto next;
-
- fence = __xe_pt_unbind_vma(tile, vma, q ? q : vm->q[id],
- first_op ? syncs : NULL,
- first_op ? num_syncs : 0);
- if (IS_ERR(fence)) {
- err = PTR_ERR(fence);
- goto err_fences;
- }
-
- if (fences)
- fences[cur_fence++] = fence;
-
-next:
- if (q && vm->pt_root[id] && !list_empty(&q->multi_gt_list))
- q = list_next_entry(q, multi_gt_list);
- }
-
- if (fences) {
- cf = dma_fence_array_create(number_tiles, fences,
- vm->composite_fence_ctx,
- vm->composite_fence_seqno++,
- false);
- if (!cf) {
- --vm->composite_fence_seqno;
- err = -ENOMEM;
- goto err_fences;
- }
- }
-
- fence = cf ? &cf->base : !fence ?
- xe_exec_queue_last_fence_get(wait_exec_queue, vm) : fence;
-
- return fence;
-
-err_fences:
- if (fences) {
- while (cur_fence)
- dma_fence_put(fences[--cur_fence]);
- kfree(fences);
- }
-
- return ERR_PTR(err);
-}
-
-static struct dma_fence *
-xe_vm_bind_vma(struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_sync_entry *syncs, u32 num_syncs,
- u8 tile_mask, bool first_op, bool last_op)
-{
- struct xe_tile *tile;
- struct dma_fence *fence;
- struct dma_fence **fences = NULL;
- struct dma_fence_array *cf = NULL;
- struct xe_vm *vm = xe_vma_vm(vma);
- int cur_fence = 0;
- int number_tiles = hweight8(tile_mask);
- int err;
- u8 id;
-
- trace_xe_vma_bind(vma);
-
- if (number_tiles > 1) {
- fences = kmalloc_array(number_tiles, sizeof(*fences),
- GFP_KERNEL);
- if (!fences)
- return ERR_PTR(-ENOMEM);
- }
-
- for_each_tile(tile, vm->xe, id) {
- if (!(tile_mask & BIT(id)))
- goto next;
-
- fence = __xe_pt_bind_vma(tile, vma, q ? q : vm->q[id],
- first_op ? syncs : NULL,
- first_op ? num_syncs : 0,
- vma->tile_present & BIT(id));
- if (IS_ERR(fence)) {
- err = PTR_ERR(fence);
- goto err_fences;
- }
-
- if (fences)
- fences[cur_fence++] = fence;
-
-next:
- if (q && vm->pt_root[id] && !list_empty(&q->multi_gt_list))
- q = list_next_entry(q, multi_gt_list);
- }
-
- if (fences) {
- cf = dma_fence_array_create(number_tiles, fences,
- vm->composite_fence_ctx,
- vm->composite_fence_seqno++,
- false);
- if (!cf) {
- --vm->composite_fence_seqno;
- err = -ENOMEM;
- goto err_fences;
- }
- }
-
- return cf ? &cf->base : fence;
-
-err_fences:
- if (fences) {
- while (cur_fence)
- dma_fence_put(fences[--cur_fence]);
- kfree(fences);
- }
-
- return ERR_PTR(err);
-}
-
static struct xe_user_fence *
find_ufence_get(struct xe_sync_entry *syncs, u32 num_syncs)
{
@@ -1841,48 +1709,6 @@ find_ufence_get(struct xe_sync_entry *syncs, u32 num_syncs)
return NULL;
}
-static struct dma_fence *
-xe_vm_bind(struct xe_vm *vm, struct xe_vma *vma, struct xe_exec_queue *q,
- struct xe_bo *bo, struct xe_sync_entry *syncs, u32 num_syncs,
- u8 tile_mask, bool immediate, bool first_op, bool last_op)
-{
- struct dma_fence *fence;
- struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
-
- xe_vm_assert_held(vm);
- xe_bo_assert_held(bo);
-
- if (immediate) {
- fence = xe_vm_bind_vma(vma, q, syncs, num_syncs, tile_mask,
- first_op, last_op);
- if (IS_ERR(fence))
- return fence;
- } else {
- xe_assert(vm->xe, xe_vm_in_fault_mode(vm));
-
- fence = xe_exec_queue_last_fence_get(wait_exec_queue, vm);
- }
-
- return fence;
-}
-
-static struct dma_fence *
-xe_vm_unbind(struct xe_vm *vm, struct xe_vma *vma,
- struct xe_exec_queue *q, struct xe_sync_entry *syncs,
- u32 num_syncs, bool first_op, bool last_op)
-{
- struct dma_fence *fence;
-
- xe_vm_assert_held(vm);
- xe_bo_assert_held(xe_vma_bo(vma));
-
- fence = xe_vm_unbind_vma(vma, q, syncs, num_syncs, first_op, last_op);
- if (IS_ERR(fence))
- return fence;
-
- return fence;
-}
-
#define ALL_DRM_XE_VM_CREATE_FLAGS (DRM_XE_VM_CREATE_FLAG_SCRATCH_PAGE | \
DRM_XE_VM_CREATE_FLAG_LR_MODE | \
DRM_XE_VM_CREATE_FLAG_FAULT_MODE)
@@ -2023,21 +1849,6 @@ static const u32 region_to_mem_type[] = {
XE_PL_VRAM1,
};
-static struct dma_fence *
-xe_vm_prefetch(struct xe_vm *vm, struct xe_vma *vma,
- struct xe_exec_queue *q, struct xe_sync_entry *syncs,
- u32 num_syncs, bool first_op, bool last_op)
-{
- struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, q);
-
- if (vma->tile_mask != (vma->tile_present & ~vma->tile_invalidated)) {
- return xe_vm_bind(vm, vma, q, xe_vma_bo(vma), syncs, num_syncs,
- vma->tile_mask, true, first_op, last_op);
- } else {
- return xe_exec_queue_last_fence_get(wait_exec_queue, vm);
- }
-}
-
static void prep_vma_destroy(struct xe_vm *vm, struct xe_vma *vma,
bool post_commit)
{
@@ -2326,13 +2137,10 @@ static int xe_vma_op_commit(struct xe_vm *vm, struct xe_vma_op *op)
return err;
}
-static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
- struct drm_gpuva_ops *ops,
- struct xe_sync_entry *syncs, u32 num_syncs,
- struct xe_vma_ops *vops, bool last)
+static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct drm_gpuva_ops *ops,
+ struct xe_vma_ops *vops)
{
struct xe_device *xe = vm->xe;
- struct xe_vma_op *last_op = NULL;
struct drm_gpuva_op *__op;
struct xe_tile *tile;
u8 id, tile_mask = 0;
@@ -2346,19 +2154,10 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
drm_gpuva_for_each_op(__op, ops) {
struct xe_vma_op *op = gpuva_op_to_vma_op(__op);
struct xe_vma *vma;
- bool first = list_empty(&vops->list);
unsigned int flags = 0;
INIT_LIST_HEAD(&op->link);
list_add_tail(&op->link, &vops->list);
-
- if (first) {
- op->flags |= XE_VMA_OP_FIRST;
- op->num_syncs = num_syncs;
- op->syncs = syncs;
- }
-
- op->q = q;
op->tile_mask = tile_mask;
switch (op->base.op) {
@@ -2471,197 +2270,21 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,
}
case DRM_GPUVA_OP_UNMAP:
case DRM_GPUVA_OP_PREFETCH:
+ /* FIXME: Need to skip some prefetch ops */
xe_vma_ops_incr_pt_update_ops(vops, op->tile_mask);
break;
default:
drm_warn(&vm->xe->drm, "NOT POSSIBLE");
}
- last_op = op;
-
err = xe_vma_op_commit(vm, op);
if (err)
return err;
}
- /* FIXME: Unhandled corner case */
- XE_WARN_ON(!last_op && last && !list_empty(&vops->list));
-
- if (!last_op)
- return 0;
-
- if (last) {
- last_op->flags |= XE_VMA_OP_LAST;
- last_op->num_syncs = num_syncs;
- last_op->syncs = syncs;
- }
-
return 0;
}
-static struct dma_fence *op_execute(struct xe_vm *vm, struct xe_vma *vma,
- struct xe_vma_op *op)
-{
- struct dma_fence *fence = NULL;
-
- lockdep_assert_held(&vm->lock);
-
- xe_vm_assert_held(vm);
- xe_bo_assert_held(xe_vma_bo(vma));
-
- switch (op->base.op) {
- case DRM_GPUVA_OP_MAP:
- fence = xe_vm_bind(vm, vma, op->q, xe_vma_bo(vma),
- op->syncs, op->num_syncs,
- op->tile_mask,
- op->map.immediate || !xe_vm_in_fault_mode(vm),
- op->flags & XE_VMA_OP_FIRST,
- op->flags & XE_VMA_OP_LAST);
- break;
- case DRM_GPUVA_OP_REMAP:
- {
- bool prev = !!op->remap.prev;
- bool next = !!op->remap.next;
-
- if (!op->remap.unmap_done) {
- if (prev || next)
- vma->gpuva.flags |= XE_VMA_FIRST_REBIND;
- fence = xe_vm_unbind(vm, vma, op->q, op->syncs,
- op->num_syncs,
- op->flags & XE_VMA_OP_FIRST,
- op->flags & XE_VMA_OP_LAST &&
- !prev && !next);
- if (IS_ERR(fence))
- break;
- op->remap.unmap_done = true;
- }
-
- if (prev) {
- op->remap.prev->gpuva.flags |= XE_VMA_LAST_REBIND;
- dma_fence_put(fence);
- fence = xe_vm_bind(vm, op->remap.prev, op->q,
- xe_vma_bo(op->remap.prev), op->syncs,
- op->num_syncs,
- op->remap.prev->tile_mask, true,
- false,
- op->flags & XE_VMA_OP_LAST && !next);
- op->remap.prev->gpuva.flags &= ~XE_VMA_LAST_REBIND;
- if (IS_ERR(fence))
- break;
- op->remap.prev = NULL;
- }
-
- if (next) {
- op->remap.next->gpuva.flags |= XE_VMA_LAST_REBIND;
- dma_fence_put(fence);
- fence = xe_vm_bind(vm, op->remap.next, op->q,
- xe_vma_bo(op->remap.next),
- op->syncs, op->num_syncs,
- op->remap.next->tile_mask, true,
- false, op->flags & XE_VMA_OP_LAST);
- op->remap.next->gpuva.flags &= ~XE_VMA_LAST_REBIND;
- if (IS_ERR(fence))
- break;
- op->remap.next = NULL;
- }
-
- break;
- }
- case DRM_GPUVA_OP_UNMAP:
- fence = xe_vm_unbind(vm, vma, op->q, op->syncs,
- op->num_syncs, op->flags & XE_VMA_OP_FIRST,
- op->flags & XE_VMA_OP_LAST);
- break;
- case DRM_GPUVA_OP_PREFETCH:
- fence = xe_vm_prefetch(vm, vma, op->q, op->syncs, op->num_syncs,
- op->flags & XE_VMA_OP_FIRST,
- op->flags & XE_VMA_OP_LAST);
- break;
- default:
- drm_warn(&vm->xe->drm, "NOT POSSIBLE");
- }
-
- if (IS_ERR(fence))
- trace_xe_vma_fail(vma);
-
- return fence;
-}
-
-static struct dma_fence *
-__xe_vma_op_execute(struct xe_vm *vm, struct xe_vma *vma,
- struct xe_vma_op *op)
-{
- struct dma_fence *fence;
- int err;
-
-retry_userptr:
- fence = op_execute(vm, vma, op);
- if (IS_ERR(fence) && PTR_ERR(fence) == -EAGAIN) {
- lockdep_assert_held_write(&vm->lock);
-
- if (op->base.op == DRM_GPUVA_OP_REMAP) {
- if (!op->remap.unmap_done)
- vma = gpuva_to_vma(op->base.remap.unmap->va);
- else if (op->remap.prev)
- vma = op->remap.prev;
- else
- vma = op->remap.next;
- }
-
- if (xe_vma_is_userptr(vma)) {
- err = xe_vma_userptr_pin_pages(to_userptr_vma(vma));
- if (!err)
- goto retry_userptr;
-
- fence = ERR_PTR(err);
- trace_xe_vma_fail(vma);
- }
- }
-
- return fence;
-}
-
-static struct dma_fence *
-xe_vma_op_execute(struct xe_vm *vm, struct xe_vma_op *op)
-{
- struct dma_fence *fence = ERR_PTR(-ENOMEM);
-
- lockdep_assert_held(&vm->lock);
-
- switch (op->base.op) {
- case DRM_GPUVA_OP_MAP:
- fence = __xe_vma_op_execute(vm, op->map.vma, op);
- break;
- case DRM_GPUVA_OP_REMAP:
- {
- struct xe_vma *vma;
-
- if (!op->remap.unmap_done)
- vma = gpuva_to_vma(op->base.remap.unmap->va);
- else if (op->remap.prev)
- vma = op->remap.prev;
- else
- vma = op->remap.next;
-
- fence = __xe_vma_op_execute(vm, vma, op);
- break;
- }
- case DRM_GPUVA_OP_UNMAP:
- fence = __xe_vma_op_execute(vm, gpuva_to_vma(op->base.unmap.va),
- op);
- break;
- case DRM_GPUVA_OP_PREFETCH:
- fence = __xe_vma_op_execute(vm,
- gpuva_to_vma(op->base.prefetch.va),
- op);
- break;
- default:
- drm_warn(&vm->xe->drm, "NOT POSSIBLE");
- }
-
- return fence;
-}
-
static void xe_vma_op_unwind(struct xe_vm *vm, struct xe_vma_op *op,
bool post_commit, bool prev_post_commit,
bool next_post_commit)
@@ -2847,23 +2470,110 @@ static int vm_bind_ioctl_ops_lock_and_prep(struct drm_exec *exec,
return 0;
}
+static int vm_ops_setup_tile_args(struct xe_vm *vm, struct xe_vma_ops *vops)
+{
+ struct xe_exec_queue *q = vops->q;
+ struct xe_tile *tile;
+ int number_tiles = 0;
+ u8 id;
+
+ for_each_tile(tile, vm->xe, id) {
+ if (vops->pt_update_ops[id].num_ops)
+ ++number_tiles;
+
+ if (vops->pt_update_ops[id].q)
+ continue;
+
+ if (q) {
+ vops->pt_update_ops[id].q = q;
+ if (vm->pt_root[id] && !list_empty(&q->multi_gt_list))
+ q = list_next_entry(q, multi_gt_list);
+ } else {
+ vops->pt_update_ops[id].q = vm->q[id];
+ }
+ }
+
+ return number_tiles;
+}
+
static struct dma_fence *ops_execute(struct xe_vm *vm,
struct xe_vma_ops *vops)
{
- struct xe_vma_op *op, *next;
+ struct xe_tile *tile;
struct dma_fence *fence = NULL;
+ struct dma_fence **fences = NULL;
+ struct dma_fence_array *cf = NULL;
+ int number_tiles = 0, current_fence = 0, err;
+ u8 id;
- list_for_each_entry_safe(op, next, &vops->list, link) {
- dma_fence_put(fence);
- fence = xe_vma_op_execute(vm, op);
- if (IS_ERR(fence)) {
- drm_warn(&vm->xe->drm, "VM op(%d) failed with %ld",
- op->base.op, PTR_ERR(fence));
- fence = ERR_PTR(-ENOSPC);
- break;
+ number_tiles = vm_ops_setup_tile_args(vm, vops);
+ if (number_tiles == 0)
+ return ERR_PTR(-ENODATA);
+
+ if (number_tiles > 1) {
+ fences = kmalloc_array(number_tiles, sizeof(*fences),
+ GFP_KERNEL);
+ if (!fences)
+ return ERR_PTR(-ENOMEM);
+ }
+
+ for_each_tile(tile, vm->xe, id) {
+ if (!vops->pt_update_ops[id].num_ops)
+ continue;
+
+ err = xe_pt_update_ops_prepare(tile, vops);
+ if (err) {
+ fence = ERR_PTR(err);
+ goto err_out;
}
}
+ for_each_tile(tile, vm->xe, id) {
+ if (!vops->pt_update_ops[id].num_ops)
+ continue;
+
+ fence = xe_pt_update_ops_run(tile, vops);
+ if (IS_ERR(fence))
+ goto err_out;
+
+ if (fences)
+ fences[current_fence++] = fence;
+ }
+
+ if (fences) {
+ cf = dma_fence_array_create(number_tiles, fences,
+ vm->composite_fence_ctx,
+ vm->composite_fence_seqno++,
+ false);
+ if (!cf) {
+ --vm->composite_fence_seqno;
+ fence = ERR_PTR(-ENOMEM);
+ goto err_out;
+ }
+ fence = &cf->base;
+ }
+
+ for_each_tile(tile, vm->xe, id) {
+ if (!vops->pt_update_ops[id].num_ops)
+ continue;
+
+ xe_pt_update_ops_fini(tile, vops);
+ }
+
+ return fence;
+
+err_out:
+ for_each_tile(tile, vm->xe, id) {
+ if (!vops->pt_update_ops[id].num_ops)
+ continue;
+
+ xe_pt_update_ops_abort(tile, vops);
+ }
+ while (current_fence)
+ dma_fence_put(fences[--current_fence]);
+ kfree(fences);
+ kfree(cf);
+
return fence;
}
@@ -2944,12 +2654,10 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,
fence = ops_execute(vm, vops);
if (IS_ERR(fence)) {
err = PTR_ERR(fence);
- /* FIXME: Killing VM rather than proper error handling */
- xe_vm_kill(vm, false);
goto unlock;
- } else {
- vm_bind_ioctl_ops_fini(vm, vops, fence);
}
+
+ vm_bind_ioctl_ops_fini(vm, vops, fence);
}
unlock:
@@ -3297,8 +3005,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
goto unwind_ops;
}
- err = vm_bind_ioctl_ops_parse(vm, q, ops[i], syncs, num_syncs,
- &vops, i == args->num_binds - 1);
+ err = vm_bind_ioctl_ops_parse(vm, ops[i], &vops);
if (err)
goto unwind_ops;
}
diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h
index 204a4ff63f88..24e2eb106430 100644
--- a/drivers/gpu/drm/xe/xe_vm.h
+++ b/drivers/gpu/drm/xe/xe_vm.h
@@ -260,6 +260,8 @@ static inline struct dma_resv *xe_vm_resv(struct xe_vm *vm)
return drm_gpuvm_resv(&vm->gpuvm);
}
+void xe_vm_kill(struct xe_vm *vm, bool unlocked);
+
/**
* xe_vm_assert_held(vm) - Assert that the vm's reservation object is held.
* @vm: The vm
diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
index 211c88801182..39bfa259c7be 100644
--- a/drivers/gpu/drm/xe/xe_vm_types.h
+++ b/drivers/gpu/drm/xe/xe_vm_types.h
@@ -26,14 +26,12 @@ struct xe_vm_pgtable_update_op;
#define XE_VMA_READ_ONLY DRM_GPUVA_USERBITS
#define XE_VMA_DESTROYED (DRM_GPUVA_USERBITS << 1)
#define XE_VMA_ATOMIC_PTE_BIT (DRM_GPUVA_USERBITS << 2)
-#define XE_VMA_FIRST_REBIND (DRM_GPUVA_USERBITS << 3)
-#define XE_VMA_LAST_REBIND (DRM_GPUVA_USERBITS << 4)
-#define XE_VMA_PTE_4K (DRM_GPUVA_USERBITS << 5)
-#define XE_VMA_PTE_2M (DRM_GPUVA_USERBITS << 6)
-#define XE_VMA_PTE_1G (DRM_GPUVA_USERBITS << 7)
-#define XE_VMA_PTE_64K (DRM_GPUVA_USERBITS << 8)
-#define XE_VMA_PTE_COMPACT (DRM_GPUVA_USERBITS << 9)
-#define XE_VMA_DUMPABLE (DRM_GPUVA_USERBITS << 10)
+#define XE_VMA_PTE_4K (DRM_GPUVA_USERBITS << 3)
+#define XE_VMA_PTE_2M (DRM_GPUVA_USERBITS << 4)
+#define XE_VMA_PTE_1G (DRM_GPUVA_USERBITS << 5)
+#define XE_VMA_PTE_64K (DRM_GPUVA_USERBITS << 6)
+#define XE_VMA_PTE_COMPACT (DRM_GPUVA_USERBITS << 7)
+#define XE_VMA_DUMPABLE (DRM_GPUVA_USERBITS << 8)
/** struct xe_userptr - User pointer */
struct xe_userptr {
@@ -100,6 +98,9 @@ struct xe_vma {
*/
u8 tile_present;
+ /** @tile_staged: bind is staged for this VMA */
+ u8 tile_staged;
+
/**
* @pat_index: The pat index to use when encoding the PTEs for this vma.
*/
@@ -315,29 +316,18 @@ struct xe_vma_op_prefetch {
/** enum xe_vma_op_flags - flags for VMA operation */
enum xe_vma_op_flags {
- /** @XE_VMA_OP_FIRST: first VMA operation for a set of syncs */
- XE_VMA_OP_FIRST = BIT(0),
- /** @XE_VMA_OP_LAST: last VMA operation for a set of syncs */
- XE_VMA_OP_LAST = BIT(1),
/** @XE_VMA_OP_COMMITTED: VMA operation committed */
- XE_VMA_OP_COMMITTED = BIT(2),
+ XE_VMA_OP_COMMITTED = BIT(0),
/** @XE_VMA_OP_PREV_COMMITTED: Previous VMA operation committed */
- XE_VMA_OP_PREV_COMMITTED = BIT(3),
+ XE_VMA_OP_PREV_COMMITTED = BIT(1),
/** @XE_VMA_OP_NEXT_COMMITTED: Next VMA operation committed */
- XE_VMA_OP_NEXT_COMMITTED = BIT(4),
+ XE_VMA_OP_NEXT_COMMITTED = BIT(2),
};
/** struct xe_vma_op - VMA operation */
struct xe_vma_op {
/** @base: GPUVA base operation */
struct drm_gpuva_op base;
- /** @q: exec queue for this operation */
- struct xe_exec_queue *q;
- /**
- * @syncs: syncs for this operation, only used on first and last
- * operation
- */
- struct xe_sync_entry *syncs;
/** @num_syncs: number of syncs */
u32 num_syncs;
/** @link: async operation link */
@@ -363,19 +353,14 @@ struct xe_vma_ops {
struct list_head list;
/** @vm: VM */
struct xe_vm *vm;
- /** @q: exec queue these operations */
+ /** @q: exec queue for VMA operations */
struct xe_exec_queue *q;
/** @syncs: syncs these operation */
struct xe_sync_entry *syncs;
/** @num_syncs: number of syncs */
u32 num_syncs;
/** @pt_update_ops: page table update operations */
- struct {
- /** @ops: operations */
- struct xe_vm_pgtable_update_op *ops;
- /** @num_ops: number of operations */
- u32 num_ops;
- } pt_update_ops[XE_MAX_TILES_PER_DEVICE];
+ struct xe_vm_pgtable_update_ops pt_update_ops[XE_MAX_TILES_PER_DEVICE];
};
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 4/5] drm/xe: Update VM trace events
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
` (2 preceding siblings ...)
2024-04-29 16:47 ` [PATCH 3/5] drm/xe: Convert multiple bind ops into single job Matthew Brost
@ 2024-04-29 16:47 ` Matthew Brost
2024-04-29 16:47 ` [PATCH 5/5] drm/xe: Update PT layer with better error handling Matthew Brost
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Matthew Brost @ 2024-04-29 16:47 UTC (permalink / raw)
To: intel-xe; +Cc: Matthew Brost, Oak Zeng, Thomas Hellström
The trace events have changed moving to a single job per VM bind IOCTL,
update the trace events align with old behavior as much as possible.
Cc: Oak Zeng <oak.zeng@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/xe/xe_trace.h | 10 ++++-----
drivers/gpu/drm/xe/xe_vm.c | 42 +++++++++++++++++++++++++++++++++--
2 files changed, 45 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_trace.h b/drivers/gpu/drm/xe/xe_trace.h
index 2d56cfc09e42..f18ae1c63031 100644
--- a/drivers/gpu/drm/xe/xe_trace.h
+++ b/drivers/gpu/drm/xe/xe_trace.h
@@ -423,11 +423,6 @@ DEFINE_EVENT(xe_vma, xe_vma_acc,
TP_ARGS(vma)
);
-DEFINE_EVENT(xe_vma, xe_vma_fail,
- TP_PROTO(struct xe_vma *vma),
- TP_ARGS(vma)
-);
-
DEFINE_EVENT(xe_vma, xe_vma_bind,
TP_PROTO(struct xe_vma *vma),
TP_ARGS(vma)
@@ -541,6 +536,11 @@ DEFINE_EVENT(xe_vm, xe_vm_rebind_worker_exit,
TP_ARGS(vm)
);
+DEFINE_EVENT(xe_vm, xe_vm_ops_fail,
+ TP_PROTO(struct xe_vm *vm),
+ TP_ARGS(vm)
+);
+
/* GuC */
DECLARE_EVENT_CLASS(xe_guc_ct_flow_control,
TP_PROTO(u32 _head, u32 _tail, u32 size, u32 space, u32 len),
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index aef0ed5f9571..bb9712586002 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -2470,6 +2470,38 @@ static int vm_bind_ioctl_ops_lock_and_prep(struct drm_exec *exec,
return 0;
}
+static void op_trace(struct xe_vma_op *op)
+{
+ switch (op->base.op) {
+ case DRM_GPUVA_OP_MAP:
+ trace_xe_vma_bind(op->map.vma);
+ break;
+ case DRM_GPUVA_OP_REMAP:
+ trace_xe_vma_unbind(gpuva_to_vma(op->base.remap.unmap->va));
+ if (op->remap.prev)
+ trace_xe_vma_bind(op->remap.prev);
+ if (op->remap.next)
+ trace_xe_vma_bind(op->remap.next);
+ break;
+ case DRM_GPUVA_OP_UNMAP:
+ trace_xe_vma_unbind(gpuva_to_vma(op->base.unmap.va));
+ break;
+ case DRM_GPUVA_OP_PREFETCH:
+ trace_xe_vma_bind(gpuva_to_vma(op->base.prefetch.va));
+ break;
+ default:
+ XE_WARN_ON("NOT POSSIBLE");
+ }
+}
+
+static void trace_xe_vm_ops_execute(struct xe_vma_ops *vops)
+{
+ struct xe_vma_op *op;
+
+ list_for_each_entry(op, &vops->list, link)
+ op_trace(op);
+}
+
static int vm_ops_setup_tile_args(struct xe_vm *vm, struct xe_vma_ops *vops)
{
struct xe_exec_queue *q = vops->q;
@@ -2513,8 +2545,10 @@ static struct dma_fence *ops_execute(struct xe_vm *vm,
if (number_tiles > 1) {
fences = kmalloc_array(number_tiles, sizeof(*fences),
GFP_KERNEL);
- if (!fences)
- return ERR_PTR(-ENOMEM);
+ if (!fences) {
+ fence = ERR_PTR(-ENOMEM);
+ goto err_trace;
+ }
}
for_each_tile(tile, vm->xe, id) {
@@ -2528,6 +2562,8 @@ static struct dma_fence *ops_execute(struct xe_vm *vm,
}
}
+ trace_xe_vm_ops_execute(vops);
+
for_each_tile(tile, vm->xe, id) {
if (!vops->pt_update_ops[id].num_ops)
continue;
@@ -2574,6 +2610,8 @@ static struct dma_fence *ops_execute(struct xe_vm *vm,
kfree(fences);
kfree(cf);
+err_trace:
+ trace_xe_vm_ops_fail(vm);
return fence;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 5/5] drm/xe: Update PT layer with better error handling
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
` (3 preceding siblings ...)
2024-04-29 16:47 ` [PATCH 4/5] drm/xe: Update VM trace events Matthew Brost
@ 2024-04-29 16:47 ` Matthew Brost
2024-04-29 16:52 ` ✓ CI.Patch_applied: success for Convert multiple bind ops to 1 job Patchwork
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Matthew Brost @ 2024-04-29 16:47 UTC (permalink / raw)
To: intel-xe; +Cc: Matthew Brost
Update PT layer so if a memory allocation for a PTE fails the error can
be propagated to the user without requiring to be killed.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/xe/xe_pt.c | 208 ++++++++++++++++++++++++++++---------
1 file changed, 160 insertions(+), 48 deletions(-)
diff --git a/drivers/gpu/drm/xe/xe_pt.c b/drivers/gpu/drm/xe/xe_pt.c
index 78e97c43e526..339d00fcf87e 100644
--- a/drivers/gpu/drm/xe/xe_pt.c
+++ b/drivers/gpu/drm/xe/xe_pt.c
@@ -813,19 +813,27 @@ xe_vm_populate_pgtable(struct xe_migrate_pt_update *pt_update, struct xe_tile *t
}
}
-static void xe_pt_abort_bind(struct xe_vma *vma,
- struct xe_vm_pgtable_update *entries,
- u32 num_entries)
+static void xe_pt_cancel_bind(struct xe_vma *vma,
+ struct xe_vm_pgtable_update *entries,
+ u32 num_entries)
{
u32 i, j;
for (i = 0; i < num_entries; i++) {
- if (!entries[i].pt_entries)
+ struct xe_pt *pt = entries[i].pt;
+
+ if (!pt)
continue;
- for (j = 0; j < entries[i].qwords; j++)
- xe_pt_destroy(entries[i].pt_entries[j].pt, xe_vma_vm(vma)->flags, NULL);
+ if (pt->level) {
+ for (j = 0; j < entries[i].qwords; j++)
+ xe_pt_destroy(entries[i].pt_entries[j].pt,
+ xe_vma_vm(vma)->flags, NULL);
+ }
+
kfree(entries[i].pt_entries);
+ entries[i].pt_entries = NULL;
+ entries[i].qwords = 0;
}
}
@@ -841,10 +849,61 @@ static void xe_pt_commit_locks_assert(struct xe_vma *vma)
xe_vm_assert_held(vm);
}
-static void xe_pt_commit_bind(struct xe_vma *vma,
- struct xe_vm_pgtable_update *entries,
- u32 num_entries, bool rebind,
- struct llist_head *deferred)
+static void xe_pt_commit(struct xe_vma *vma,
+ struct xe_vm_pgtable_update *entries,
+ u32 num_entries, struct llist_head *deferred)
+{
+ u32 i, j;
+
+ xe_pt_commit_locks_assert(vma);
+
+ for (i = 0; i < num_entries; i++) {
+ struct xe_pt *pt = entries[i].pt;
+
+ if (!pt->level)
+ continue;
+
+ for (j = 0; j < entries[i].qwords; j++) {
+ struct xe_pt *oldpte = entries[i].pt_entries[j].pt;
+
+ xe_pt_destroy(oldpte, xe_vma_vm(vma)->flags, deferred);
+ }
+ }
+}
+
+static void xe_pt_abort_bind(struct xe_vma *vma,
+ struct xe_vm_pgtable_update *entries,
+ u32 num_entries, bool rebind)
+{
+ int i, j;
+
+ xe_pt_commit_locks_assert(vma);
+
+ for (i = num_entries - 1; i >= 0; --i) {
+ struct xe_pt *pt = entries[i].pt;
+ struct xe_pt_dir *pt_dir;
+
+ if (!rebind)
+ pt->num_live -= entries[i].qwords;
+
+ if (!pt->level)
+ continue;
+
+ pt_dir = as_xe_pt_dir(pt);
+ for (j = 0; j < entries[i].qwords; j++) {
+ u32 j_ = j + entries[i].ofs;
+ struct xe_pt *newpte = xe_pt_entry(pt_dir, j_);
+ struct xe_pt *oldpte = entries[i].pt_entries[j].pt;
+
+ pt_dir->children[j_] = oldpte ? &oldpte->base : 0;
+ xe_pt_destroy(newpte, xe_vma_vm(vma)->flags, NULL);
+ }
+ }
+}
+
+static void xe_pt_commit_prepare_bind(struct xe_vma *vma,
+ struct xe_vm_pgtable_update *entries,
+ u32 num_entries, bool rebind)
{
u32 i, j;
@@ -864,12 +923,13 @@ static void xe_pt_commit_bind(struct xe_vma *vma,
for (j = 0; j < entries[i].qwords; j++) {
u32 j_ = j + entries[i].ofs;
struct xe_pt *newpte = entries[i].pt_entries[j].pt;
+ struct xe_pt *oldpte = NULL;
if (xe_pt_entry(pt_dir, j_))
- xe_pt_destroy(xe_pt_entry(pt_dir, j_),
- xe_vma_vm(vma)->flags, deferred);
+ oldpte = xe_pt_entry(pt_dir, j_);
pt_dir->children[j_] = &newpte->base;
+ entries[i].pt_entries[j].pt = oldpte;
}
}
}
@@ -893,8 +953,6 @@ xe_pt_prepare_bind(struct xe_tile *tile, struct xe_vma *vma,
err = xe_pt_stage_bind(tile, vma, entries, num_entries);
if (!err)
xe_tile_assert(tile, *num_entries);
- else /* abort! */
- xe_pt_abort_bind(vma, entries, *num_entries);
return err;
}
@@ -1413,7 +1471,7 @@ xe_pt_stage_unbind_post_descend(struct xe_ptw *parent, pgoff_t offset,
&end_offset))
return 0;
- (void)xe_pt_new_shared(&xe_walk->wupd, xe_child, offset, false);
+ (void)xe_pt_new_shared(&xe_walk->wupd, xe_child, offset, true);
xe_walk->wupd.updates[level].update->qwords = end_offset - offset;
return 0;
@@ -1481,32 +1539,57 @@ xe_migrate_clear_pgtable_callback(struct xe_migrate_pt_update *pt_update,
memset64(ptr, empty, num_qwords);
}
+static void xe_pt_abort_unbind(struct xe_vma *vma,
+ struct xe_vm_pgtable_update *entries,
+ u32 num_entries)
+{
+ int j, i;
+
+ xe_pt_commit_locks_assert(vma);
+
+ for (j = num_entries - 1; j >= 0; --j) {
+ struct xe_vm_pgtable_update *entry = &entries[j];
+ struct xe_pt *pt = entry->pt;
+ struct xe_pt_dir *pt_dir = as_xe_pt_dir(pt);
+
+ pt->num_live += entry->qwords;
+
+ if (!pt->level)
+ continue;
+
+ for (i = entry->ofs; i < entry->ofs + entry->qwords; i++)
+ pt_dir->children[i] =
+ entries[j].pt_entries[i - entry->ofs].pt ?
+ &entries[j].pt_entries[i - entry->ofs].pt->base : 0;
+ }
+}
+
static void
-xe_pt_commit_unbind(struct xe_vma *vma,
- struct xe_vm_pgtable_update *entries, u32 num_entries,
- struct llist_head *deferred)
+xe_pt_commit_prepare_unbind(struct xe_vma *vma,
+ struct xe_vm_pgtable_update *entries,
+ u32 num_entries)
{
- u32 j;
+ int j, i;
xe_pt_commit_locks_assert(vma);
for (j = 0; j < num_entries; ++j) {
struct xe_vm_pgtable_update *entry = &entries[j];
struct xe_pt *pt = entry->pt;
+ struct xe_pt_dir *pt_dir;
pt->num_live -= entry->qwords;
- if (pt->level) {
- struct xe_pt_dir *pt_dir = as_xe_pt_dir(pt);
- u32 i;
-
- for (i = entry->ofs; i < entry->ofs + entry->qwords;
- i++) {
- if (xe_pt_entry(pt_dir, i))
- xe_pt_destroy(xe_pt_entry(pt_dir, i),
- xe_vma_vm(vma)->flags, deferred);
+ if (!pt->level)
+ continue;
- pt_dir->children[i] = NULL;
- }
+ pt_dir = as_xe_pt_dir(pt);
+ for (i = entry->ofs; i < entry->ofs + entry->qwords; i++) {
+ if (xe_pt_entry(pt_dir, i))
+ entries[j].pt_entries[i - entry->ofs].pt =
+ xe_pt_entry(pt_dir, i);
+ else
+ entries[j].pt_entries[i - entry->ofs].pt = NULL;
+ pt_dir->children[i] = NULL;
}
}
}
@@ -1543,7 +1626,6 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
{
u32 current_op = pt_update_ops->current_op;
struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[current_op];
- struct llist_head *deferred = &pt_update_ops->deferred;
int err;
xe_bo_assert_held(xe_vma_bo(vma));
@@ -1586,11 +1668,12 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile,
(!pt_op->rebind && vm->scratch_pt[tile->id] &&
xe_vm_in_preempt_fence_mode(vm));
- /* FIXME: Don't commit right away */
vma->tile_staged |= BIT(tile->id);
pt_op->vma = vma;
- xe_pt_commit_bind(vma, pt_op->entries, pt_op->num_entries,
- pt_op->rebind, deferred);
+ xe_pt_commit_prepare_bind(vma, pt_op->entries,
+ pt_op->num_entries, pt_op->rebind);
+ } else {
+ xe_pt_cancel_bind(vma, pt_op->entries, pt_op->num_entries);
}
return err;
@@ -1602,7 +1685,6 @@ static int unbind_op_prepare(struct xe_tile *tile,
{
u32 current_op = pt_update_ops->current_op;
struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[current_op];
- struct llist_head *deferred = &pt_update_ops->deferred;
if (!((vma->tile_present | vma->tile_staged) & BIT(tile->id)))
return 0;
@@ -1633,9 +1715,7 @@ static int unbind_op_prepare(struct xe_tile *tile,
pt_update_ops->needs_userptr_lock |= xe_vma_is_userptr(vma);
pt_update_ops->needs_invalidation = true;
- /* FIXME: Don't commit right away */
- xe_pt_commit_unbind(vma, pt_op->entries, pt_op->num_entries,
- deferred);
+ xe_pt_commit_prepare_unbind(vma, pt_op->entries, pt_op->num_entries);
return 0;
}
@@ -1851,7 +1931,7 @@ xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops)
struct invalidation_fence *ifence = NULL;
struct xe_range_fence *rfence;
struct xe_vma_op *op;
- int err = 0;
+ int err = 0, i;
struct xe_migrate_pt_update update = {
.ops = pt_update_ops->needs_userptr_lock ?
&userptr_migrate_ops :
@@ -1871,8 +1951,10 @@ xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops)
if (pt_update_ops->needs_invalidation) {
ifence = kzalloc(sizeof(*ifence), GFP_KERNEL);
- if (!ifence)
- return ERR_PTR(-ENOMEM);
+ if (!ifence) {
+ err = -ENOMEM;
+ goto kill_vm_tile1;
+ }
}
rfence = kzalloc(sizeof(*rfence), GFP_KERNEL);
@@ -1887,6 +1969,15 @@ xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops)
goto free_rfence;
}
+ /* Point of no return - VM killed if failure after this */
+ for (i = 0; i < pt_update_ops->current_op; ++i) {
+ struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[i];
+
+ xe_pt_commit(pt_op->vma, pt_op->entries,
+ pt_op->num_entries, &pt_update_ops->deferred);
+ pt_op->vma = NULL; /* skip in xe_pt_update_ops_abort */
+ }
+
err = xe_range_fence_insert(&vm->rftree[tile->id], rfence,
&xe_range_fence_kfree_ops,
pt_update_ops->start,
@@ -1922,10 +2013,15 @@ xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops)
if (pt_update_ops->needs_userptr_lock)
up_read(&vm->userptr.notifier_lock);
dma_fence_put(fence);
+ if (!tile->id)
+ xe_vm_kill(vops->vm, false);
free_rfence:
kfree(rfence);
free_ifence:
kfree(ifence);
+kill_vm_tile1:
+ if (tile->id)
+ xe_vm_kill(vops->vm, false);
return ERR_PTR(err);
}
@@ -1946,12 +2042,10 @@ void xe_pt_update_ops_fini(struct xe_tile *tile, struct xe_vma_ops *vops)
lockdep_assert_held(&vops->vm->lock);
xe_vm_assert_held(vops->vm);
- /* FIXME: Not 100% correct */
- for (i = 0; i < pt_update_ops->num_ops; ++i) {
+ for (i = 0; i < pt_update_ops->current_op; ++i) {
struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[i];
- if (pt_op->bind)
- xe_pt_free_bind(pt_op->entries, pt_op->num_entries);
+ xe_pt_free_bind(pt_op->entries, pt_op->num_entries);
}
xe_bo_put_commit(&vops->pt_update_ops[tile->id].deferred);
}
@@ -1965,10 +2059,28 @@ void xe_pt_update_ops_fini(struct xe_tile *tile, struct xe_vma_ops *vops)
*/
void xe_pt_update_ops_abort(struct xe_tile *tile, struct xe_vma_ops *vops)
{
+ struct xe_vm_pgtable_update_ops *pt_update_ops =
+ &vops->pt_update_ops[tile->id];
+ int i;
+
lockdep_assert_held(&vops->vm->lock);
xe_vm_assert_held(vops->vm);
- /* FIXME: Just kill VM for now + cleanup PTs */
+ for (i = pt_update_ops->num_ops - 1; i >= 0; --i) {
+ struct xe_vm_pgtable_update_op *pt_op =
+ &pt_update_ops->ops[i];
+
+ if (!pt_op->vma || i >= pt_update_ops->current_op)
+ continue;
+
+ if (pt_op->bind)
+ xe_pt_abort_bind(pt_op->vma, pt_op->entries,
+ pt_op->num_entries,
+ pt_op->rebind);
+ else
+ xe_pt_abort_unbind(pt_op->vma, pt_op->entries,
+ pt_op->num_entries);
+ }
+
xe_bo_put_commit(&vops->pt_update_ops[tile->id].deferred);
- xe_vm_kill(vops->vm, false);
- }
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* ✓ CI.Patch_applied: success for Convert multiple bind ops to 1 job
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
` (4 preceding siblings ...)
2024-04-29 16:47 ` [PATCH 5/5] drm/xe: Update PT layer with better error handling Matthew Brost
@ 2024-04-29 16:52 ` Patchwork
2024-04-29 16:53 ` ✗ CI.checkpatch: warning " Patchwork
2024-04-29 16:54 ` ✓ CI.KUnit: success " Patchwork
7 siblings, 0 replies; 9+ messages in thread
From: Patchwork @ 2024-04-29 16:52 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-xe
== Series Details ==
Series: Convert multiple bind ops to 1 job
URL : https://patchwork.freedesktop.org/series/133034/
State : success
== Summary ==
=== Applying kernel patches on branch 'drm-tip' with base: ===
Base commit: acf512bb5ff4 drm-tip: 2024y-04m-29d-15h-11m-22s UTC integration manifest
=== git am output follows ===
Applying: drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue
Applying: drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
Applying: drm/xe: Convert multiple bind ops into single job
Applying: drm/xe: Update VM trace events
Applying: drm/xe: Update PT layer with better error handling
^ permalink raw reply [flat|nested] 9+ messages in thread
* ✗ CI.checkpatch: warning for Convert multiple bind ops to 1 job
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
` (5 preceding siblings ...)
2024-04-29 16:52 ` ✓ CI.Patch_applied: success for Convert multiple bind ops to 1 job Patchwork
@ 2024-04-29 16:53 ` Patchwork
2024-04-29 16:54 ` ✓ CI.KUnit: success " Patchwork
7 siblings, 0 replies; 9+ messages in thread
From: Patchwork @ 2024-04-29 16:53 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-xe
== Series Details ==
Series: Convert multiple bind ops to 1 job
URL : https://patchwork.freedesktop.org/series/133034/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt
Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master
0daf0be5bb95eb0a0e42275e00a0e42d8d8fd543
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 20a4e151b919c37f9d45e82108101adee605704e
Author: Matthew Brost <matthew.brost@intel.com>
Date: Mon Apr 29 09:47:13 2024 -0700
drm/xe: Update PT layer with better error handling
Update PT layer so if a memory allocation for a PTE fails the error can
be propagated to the user without requiring to be killed.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
+ /mt/dim checkpatch acf512bb5ff4d2e8ed5ef67444b6c1f92a222d77 drm-intel
d7f72028c671 drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue
f04b0237f65d drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
ed98bce36b51 drm/xe: Convert multiple bind ops into single job
-:13: WARNING:TYPO_SPELLING: 'implemenation' may be misspelled - perhaps 'implementation'?
#13:
The implemenation is roughly:
^^^^^^^^^^^^^
-:878: WARNING:SUSPECT_CODE_INDENT: suspect code indent for conditional statements (8, 15)
#878: FILE: drivers/gpu/drm/xe/xe_pt.c:1131:
+ if (uvma->userptr.initial_bind && !xe_vm_in_fault_mode(vm))
+ return 0;
-:879: ERROR:CODE_INDENT: code indent should use tabs where possible
#879: FILE: drivers/gpu/drm/xe/xe_pt.c:1132:
+ return 0;$
-:879: WARNING:LEADING_SPACE: please, no spaces at the start of a line
#879: FILE: drivers/gpu/drm/xe/xe_pt.c:1132:
+ return 0;$
-:1541: WARNING:BLOCK_COMMENT_STYLE: Block comments should align the * on each line
#1541: FILE: drivers/gpu/drm/xe/xe_pt.c:1841:
+ * installing job fence in various places.
+ *
-:1542: WARNING:BLOCK_COMMENT_STYLE: Block comments should align the * on each line
#1542: FILE: drivers/gpu/drm/xe/xe_pt.c:1842:
+ *
+ * Return: fence on success, negative ERR_PTR on error.
-:1768: WARNING:LEADING_SPACE: please, no spaces at the start of a line
#1768: FILE: drivers/gpu/drm/xe/xe_pt.c:1974:
+ }$
total: 1 errors, 6 warnings, 0 checks, 2499 lines checked
fa08171fd5c3 drm/xe: Update VM trace events
20a4e151b919 drm/xe: Update PT layer with better error handling
^ permalink raw reply [flat|nested] 9+ messages in thread
* ✓ CI.KUnit: success for Convert multiple bind ops to 1 job
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
` (6 preceding siblings ...)
2024-04-29 16:53 ` ✗ CI.checkpatch: warning " Patchwork
@ 2024-04-29 16:54 ` Patchwork
7 siblings, 0 replies; 9+ messages in thread
From: Patchwork @ 2024-04-29 16:54 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-xe
== Series Details ==
Series: Convert multiple bind ops to 1 job
URL : https://patchwork.freedesktop.org/series/133034/
State : success
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
[16:53:19] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[16:53:23] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
../arch/x86/um/user-offsets.c:17:6: warning: no previous prototype for ‘foo’ [-Wmissing-prototypes]
17 | void foo(void)
| ^~~
In file included from ../arch/um/kernel/asm-offsets.c:1:
../arch/x86/um/shared/sysdep/kernel-offsets.h:9:6: warning: no previous prototype for ‘foo’ [-Wmissing-prototypes]
9 | void foo(void)
| ^~~
../arch/x86/um/bugs_64.c:9:6: warning: no previous prototype for ‘arch_check_bugs’ [-Wmissing-prototypes]
9 | void arch_check_bugs(void)
| ^~~~~~~~~~~~~~~
../arch/x86/um/bugs_64.c:13:6: warning: no previous prototype for ‘arch_examine_signal’ [-Wmissing-prototypes]
13 | void arch_examine_signal(int sig, struct uml_pt_regs *regs)
| ^~~~~~~~~~~~~~~~~~~
../arch/x86/um/fault.c:18:5: warning: no previous prototype for ‘arch_fixup’ [-Wmissing-prototypes]
18 | int arch_fixup(unsigned long address, struct uml_pt_regs *regs)
| ^~~~~~~~~~
../arch/x86/um/os-Linux/registers.c:146:15: warning: no previous prototype for ‘get_thread_reg’ [-Wmissing-prototypes]
146 | unsigned long get_thread_reg(int reg, jmp_buf *buf)
| ^~~~~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:16:5: warning: no previous prototype for ‘__vdso_clock_gettime’ [-Wmissing-prototypes]
16 | int __vdso_clock_gettime(clockid_t clock, struct __kernel_old_timespec *ts)
| ^~~~~~~~~~~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:30:5: warning: no previous prototype for ‘__vdso_gettimeofday’ [-Wmissing-prototypes]
30 | int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
| ^~~~~~~~~~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:44:21: warning: no previous prototype for ‘__vdso_time’ [-Wmissing-prototypes]
44 | __kernel_old_time_t __vdso_time(__kernel_old_time_t *t)
| ^~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:57:1: warning: no previous prototype for ‘__vdso_getcpu’ [-Wmissing-prototypes]
57 | __vdso_getcpu(unsigned *cpu, unsigned *node, struct getcpu_cache *unused)
| ^~~~~~~~~~~~~
../arch/um/os-Linux/skas/process.c:107:6: warning: no previous prototype for ‘wait_stub_done’ [-Wmissing-prototypes]
107 | void wait_stub_done(int pid)
| ^~~~~~~~~~~~~~
../arch/um/os-Linux/skas/process.c:683:6: warning: no previous prototype for ‘__switch_mm’ [-Wmissing-prototypes]
683 | void __switch_mm(struct mm_id *mm_idp)
| ^~~~~~~~~~~
../arch/x86/um/os-Linux/mcontext.c:7:6: warning: no previous prototype for ‘get_regs_from_mc’ [-Wmissing-prototypes]
7 | void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/skas/process.c:36:12: warning: no previous prototype for ‘start_uml’ [-Wmissing-prototypes]
36 | int __init start_uml(void)
| ^~~~~~~~~
../arch/um/kernel/skas/mmu.c:17:5: warning: no previous prototype for ‘init_new_context’ [-Wmissing-prototypes]
17 | int init_new_context(struct task_struct *task, struct mm_struct *mm)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/skas/mmu.c:60:6: warning: no previous prototype for ‘destroy_context’ [-Wmissing-prototypes]
60 | void destroy_context(struct mm_struct *mm)
| ^~~~~~~~~~~~~~~
../arch/x86/um/ptrace_64.c:111:5: warning: no previous prototype for ‘poke_user’ [-Wmissing-prototypes]
111 | int poke_user(struct task_struct *child, long addr, long data)
| ^~~~~~~~~
../arch/x86/um/ptrace_64.c:171:5: warning: no previous prototype for ‘peek_user’ [-Wmissing-prototypes]
171 | int peek_user(struct task_struct *child, long addr, long data)
| ^~~~~~~~~
../arch/um/os-Linux/main.c:187:7: warning: no previous prototype for ‘__wrap_malloc’ [-Wmissing-prototypes]
187 | void *__wrap_malloc(int size)
| ^~~~~~~~~~~~~
../arch/um/os-Linux/main.c:208:7: warning: no previous prototype for ‘__wrap_calloc’ [-Wmissing-prototypes]
208 | void *__wrap_calloc(int n, int size)
| ^~~~~~~~~~~~~
../arch/um/os-Linux/main.c:222:6: warning: no previous prototype for ‘__wrap_free’ [-Wmissing-prototypes]
222 | void __wrap_free(void *ptr)
| ^~~~~~~~~~~
../arch/um/os-Linux/mem.c:28:6: warning: no previous prototype for ‘kasan_map_memory’ [-Wmissing-prototypes]
28 | void kasan_map_memory(void *start, size_t len)
| ^~~~~~~~~~~~~~~~
../arch/um/os-Linux/mem.c:212:13: warning: no previous prototype for ‘check_tmpexec’ [-Wmissing-prototypes]
212 | void __init check_tmpexec(void)
| ^~~~~~~~~~~~~
../arch/x86/um/signal.c:560:6: warning: no previous prototype for ‘sys_rt_sigreturn’ [-Wmissing-prototypes]
560 | long sys_rt_sigreturn(void)
| ^~~~~~~~~~~~~~~~
../arch/um/os-Linux/signal.c:75:6: warning: no previous prototype for ‘sig_handler’ [-Wmissing-prototypes]
75 | void sig_handler(int sig, struct siginfo *si, mcontext_t *mc)
| ^~~~~~~~~~~
../arch/um/os-Linux/signal.c:111:6: warning: no previous prototype for ‘timer_alarm_handler’ [-Wmissing-prototypes]
111 | void timer_alarm_handler(int sig, struct siginfo *unused_si, mcontext_t *mc)
| ^~~~~~~~~~~~~~~~~~~
../arch/um/os-Linux/start_up.c:301:12: warning: no previous prototype for ‘parse_iomem’ [-Wmissing-prototypes]
301 | int __init parse_iomem(char *str, int *add)
| ^~~~~~~~~~~
../arch/um/kernel/mem.c:202:8: warning: no previous prototype for ‘pgd_alloc’ [-Wmissing-prototypes]
202 | pgd_t *pgd_alloc(struct mm_struct *mm)
| ^~~~~~~~~
../arch/um/kernel/mem.c:215:7: warning: no previous prototype for ‘uml_kmalloc’ [-Wmissing-prototypes]
215 | void *uml_kmalloc(int size, int flags)
| ^~~~~~~~~~~
../arch/um/kernel/process.c:51:5: warning: no previous prototype for ‘pid_to_processor_id’ [-Wmissing-prototypes]
51 | int pid_to_processor_id(int pid)
| ^~~~~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:87:7: warning: no previous prototype for ‘__switch_to’ [-Wmissing-prototypes]
87 | void *__switch_to(struct task_struct *from, struct task_struct *to)
| ^~~~~~~~~~~
../arch/um/kernel/process.c:140:6: warning: no previous prototype for ‘fork_handler’ [-Wmissing-prototypes]
140 | void fork_handler(void)
| ^~~~~~~~~~~~
../arch/um/kernel/process.c:217:6: warning: no previous prototype for ‘arch_cpu_idle’ [-Wmissing-prototypes]
217 | void arch_cpu_idle(void)
| ^~~~~~~~~~~~~
../arch/um/kernel/process.c:253:5: warning: no previous prototype for ‘copy_to_user_proc’ [-Wmissing-prototypes]
253 | int copy_to_user_proc(void __user *to, void *from, int size)
| ^~~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:263:5: warning: no previous prototype for ‘clear_user_proc’ [-Wmissing-prototypes]
263 | int clear_user_proc(void __user *buf, int size)
| ^~~~~~~~~~~~~~~
../arch/um/kernel/process.c:271:6: warning: no previous prototype for ‘set_using_sysemu’ [-Wmissing-prototypes]
271 | void set_using_sysemu(int value)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:278:5: warning: no previous prototype for ‘get_using_sysemu’ [-Wmissing-prototypes]
278 | int get_using_sysemu(void)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:316:12: warning: no previous prototype for ‘make_proc_sysemu’ [-Wmissing-prototypes]
316 | int __init make_proc_sysemu(void)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:348:15: warning: no previous prototype for ‘arch_align_stack’ [-Wmissing-prototypes]
348 | unsigned long arch_align_stack(unsigned long sp)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:45:6: warning: no previous prototype for ‘machine_restart’ [-Wmissing-prototypes]
45 | void machine_restart(char * __unused)
| ^~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:51:6: warning: no previous prototype for ‘machine_power_off’ [-Wmissing-prototypes]
51 | void machine_power_off(void)
| ^~~~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:57:6: warning: no previous prototype for ‘machine_halt’ [-Wmissing-prototypes]
57 | void machine_halt(void)
| ^~~~~~~~~~~~
../arch/x86/um/syscalls_64.c:48:6: warning: no previous prototype for ‘arch_switch_to’ [-Wmissing-prototypes]
48 | void arch_switch_to(struct task_struct *to)
| ^~~~~~~~~~~~~~
../arch/um/kernel/tlb.c:579:6: warning: no previous prototype for ‘flush_tlb_mm_range’ [-Wmissing-prototypes]
579 | void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
| ^~~~~~~~~~~~~~~~~~
../arch/um/kernel/tlb.c:594:6: warning: no previous prototype for ‘force_flush_all’ [-Wmissing-prototypes]
594 | void force_flush_all(void)
| ^~~~~~~~~~~~~~~
../arch/um/kernel/um_arch.c:408:19: warning: no previous prototype for ‘read_initrd’ [-Wmissing-prototypes]
408 | int __init __weak read_initrd(void)
| ^~~~~~~~~~~
../arch/um/kernel/um_arch.c:461:7: warning: no previous prototype for ‘text_poke’ [-Wmissing-prototypes]
461 | void *text_poke(void *addr, const void *opcode, size_t len)
| ^~~~~~~~~
../arch/um/kernel/um_arch.c:473:6: warning: no previous prototype for ‘text_poke_sync’ [-Wmissing-prototypes]
473 | void text_poke_sync(void)
| ^~~~~~~~~~~~~~
../arch/um/kernel/kmsg_dump.c:60:12: warning: no previous prototype for ‘kmsg_dumper_stdout_init’ [-Wmissing-prototypes]
60 | int __init kmsg_dumper_stdout_init(void)
| ^~~~~~~~~~~~~~~~~~~~~~~
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
156 | u64 ioread64_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
163 | u64 ioread64_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
170 | u64 ioread64be_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
178 | u64 ioread64be_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
stty: 'standard input': Inappropriate ioctl for device
[16:53:49] Starting KUnit Kernel (1/1)...
[16:53:49] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[16:53:49] =================== guc_dbm (7 subtests) ===================
[16:53:49] [PASSED] test_empty
[16:53:49] [PASSED] test_default
[16:53:49] ======================== test_size ========================
[16:53:49] [PASSED] 4
[16:53:49] [PASSED] 8
[16:53:49] [PASSED] 32
[16:53:49] [PASSED] 256
[16:53:49] ==================== [PASSED] test_size ====================
[16:53:49] ======================= test_reuse ========================
[16:53:49] [PASSED] 4
[16:53:49] [PASSED] 8
[16:53:49] [PASSED] 32
[16:53:49] [PASSED] 256
[16:53:49] =================== [PASSED] test_reuse ====================
[16:53:49] =================== test_range_overlap ====================
[16:53:49] [PASSED] 4
[16:53:49] [PASSED] 8
[16:53:49] [PASSED] 32
[16:53:49] [PASSED] 256
[16:53:49] =============== [PASSED] test_range_overlap ================
[16:53:49] =================== test_range_compact ====================
[16:53:49] [PASSED] 4
[16:53:49] [PASSED] 8
[16:53:49] [PASSED] 32
[16:53:49] [PASSED] 256
[16:53:49] =============== [PASSED] test_range_compact ================
[16:53:49] ==================== test_range_spare =====================
[16:53:49] [PASSED] 4
[16:53:49] [PASSED] 8
[16:53:49] [PASSED] 32
[16:53:49] [PASSED] 256
[16:53:49] ================ [PASSED] test_range_spare =================
[16:53:49] ===================== [PASSED] guc_dbm =====================
[16:53:49] =================== guc_idm (6 subtests) ===================
[16:53:49] [PASSED] bad_init
[16:53:49] [PASSED] no_init
[16:53:49] [PASSED] init_fini
[16:53:49] [PASSED] check_used
[16:53:49] [PASSED] check_quota
[16:53:49] [PASSED] check_all
[16:53:49] ===================== [PASSED] guc_idm =====================
[16:53:49] ================== no_relay (3 subtests) ===================
[16:53:49] [PASSED] xe_drops_guc2pf_if_not_ready
[16:53:49] [PASSED] xe_drops_guc2vf_if_not_ready
[16:53:49] [PASSED] xe_rejects_send_if_not_ready
[16:53:49] ==================== [PASSED] no_relay =====================
[16:53:49] ================== pf_relay (14 subtests) ==================
[16:53:49] [PASSED] pf_rejects_guc2pf_too_short
[16:53:49] [PASSED] pf_rejects_guc2pf_too_long
[16:53:49] [PASSED] pf_rejects_guc2pf_no_payload
[16:53:49] [PASSED] pf_fails_no_payload
[16:53:49] [PASSED] pf_fails_bad_origin
[16:53:49] [PASSED] pf_fails_bad_type
[16:53:49] [PASSED] pf_txn_reports_error
[16:53:49] [PASSED] pf_txn_sends_pf2guc
[16:53:49] [PASSED] pf_sends_pf2guc
[16:53:49] [SKIPPED] pf_loopback_nop
[16:53:49] [SKIPPED] pf_loopback_echo
[16:53:49] [SKIPPED] pf_loopback_fail
[16:53:49] [SKIPPED] pf_loopback_busy
[16:53:49] [SKIPPED] pf_loopback_retry
[16:53:49] ==================== [PASSED] pf_relay =====================
[16:53:49] ================== vf_relay (3 subtests) ===================
[16:53:49] [PASSED] vf_rejects_guc2vf_too_short
[16:53:49] [PASSED] vf_rejects_guc2vf_too_long
[16:53:49] [PASSED] vf_rejects_guc2vf_no_payload
[16:53:49] ==================== [PASSED] vf_relay =====================
[16:53:49] ================= pf_service (11 subtests) =================
[16:53:49] [PASSED] pf_negotiate_any
[16:53:49] [PASSED] pf_negotiate_base_match
[16:53:49] [PASSED] pf_negotiate_base_newer
[16:53:49] [PASSED] pf_negotiate_base_next
[16:53:49] [SKIPPED] pf_negotiate_base_older
[16:53:49] [PASSED] pf_negotiate_base_prev
[16:53:49] [PASSED] pf_negotiate_latest_match
[16:53:49] [PASSED] pf_negotiate_latest_newer
[16:53:49] [PASSED] pf_negotiate_latest_next
[16:53:49] [SKIPPED] pf_negotiate_latest_older
[16:53:49] [SKIPPED] pf_negotiate_latest_prev
[16:53:49] =================== [PASSED] pf_service ====================
[16:53:49] ===================== lmtt (1 subtest) =====================
[16:53:49] ======================== test_ops =========================
[16:53:49] [PASSED] 2-level
[16:53:49] [PASSED] multi-level
[16:53:49] ==================== [PASSED] test_ops =====================
[16:53:49] ====================== [PASSED] lmtt =======================
[16:53:49] ==================== xe_bo (2 subtests) ====================
[16:53:49] [SKIPPED] xe_ccs_migrate_kunit
[16:53:49] [SKIPPED] xe_bo_evict_kunit
[16:53:49] ===================== [SKIPPED] xe_bo ======================
[16:53:49] ================== xe_dma_buf (1 subtest) ==================
[16:53:49] [SKIPPED] xe_dma_buf_kunit
[16:53:49] =================== [SKIPPED] xe_dma_buf ===================
[16:53:49] ================== xe_migrate (1 subtest) ==================
[16:53:49] [SKIPPED] xe_migrate_sanity_kunit
[16:53:49] =================== [SKIPPED] xe_migrate ===================
[16:53:49] =================== xe_mocs (2 subtests) ===================
[16:53:49] [SKIPPED] xe_live_mocs_kernel_kunit
[16:53:49] [SKIPPED] xe_live_mocs_reset_kunit
[16:53:49] ==================== [SKIPPED] xe_mocs =====================
[16:53:49] =================== xe_pci (2 subtests) ====================
[16:53:49] [PASSED] xe_gmdid_graphics_ip
[16:53:49] [PASSED] xe_gmdid_media_ip
[16:53:49] ===================== [PASSED] xe_pci ======================
[16:53:49] ==================== xe_rtp (1 subtest) ====================
[16:53:49] ================== xe_rtp_process_tests ===================
[16:53:49] [PASSED] coalesce-same-reg
[16:53:49] [PASSED] no-match-no-add
[16:53:49] [PASSED] no-match-no-add-multiple-rules
[16:53:49] [PASSED] two-regs-two-entries
[16:53:49] [PASSED] clr-one-set-other
[16:53:49] [PASSED] set-field
[16:53:49] [PASSED] conflict-duplicate
[16:53:49] [PASSED] conflict-not-disjoint
[16:53:49] [PASSED] conflict-reg-type
[16:53:49] ============== [PASSED] xe_rtp_process_tests ===============
[16:53:49] ===================== [PASSED] xe_rtp ======================
[16:53:49] ==================== xe_wa (1 subtest) =====================
[16:53:49] ======================== xe_wa_gt =========================
[16:53:49] [PASSED] TIGERLAKE (B0)
[16:53:49] [PASSED] DG1 (A0)
[16:53:49] [PASSED] DG1 (B0)
[16:53:49] [PASSED] ALDERLAKE_S (A0)
[16:53:49] [PASSED] ALDERLAKE_S (B0)
[16:53:49] [PASSED] ALDERLAKE_S (C0)
[16:53:49] [PASSED] ALDERLAKE_S (D0)
[16:53:49] [PASSED] ALDERLAKE_P (A0)
[16:53:49] [PASSED] ALDERLAKE_P (B0)
[16:53:49] [PASSED] ALDERLAKE_P (C0)
[16:53:49] [PASSED] ALDERLAKE_S_RPLS (D0)
[16:53:49] [PASSED] ALDERLAKE_P_RPLU (E0)
[16:53:49] [PASSED] DG2_G10 (C0)
[16:53:49] [PASSED] DG2_G11 (B1)
[16:53:49] [PASSED] DG2_G12 (A1)
[16:53:49] [PASSED] METEORLAKE (g:A0, m:A0)
[16:53:49] [PASSED] METEORLAKE (g:A0, m:A0)
[16:53:49] [PASSED] METEORLAKE (g:A0, m:A0)
[16:53:49] [PASSED] LUNARLAKE (g:A0, m:A0)
[16:53:49] [PASSED] LUNARLAKE (g:B0, m:A0)
[16:53:49] ==================== [PASSED] xe_wa_gt =====================
[16:53:49] ====================== [PASSED] xe_wa ======================
[16:53:49] ============================================================
[16:53:49] Testing complete. Ran 98 tests: passed: 84, skipped: 14
[16:53:49] Elapsed time: 29.586s total, 4.255s configuring, 25.110s building, 0.190s running
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig
[16:53:49] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[16:53:51] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
In file included from ../arch/um/kernel/asm-offsets.c:1:
../arch/x86/um/shared/sysdep/kernel-offsets.h:9:6: warning: no previous prototype for ‘foo’ [-Wmissing-prototypes]
9 | void foo(void)
| ^~~
../arch/x86/um/ptrace_64.c:111:5: warning: no previous prototype for ‘poke_user’ [-Wmissing-prototypes]
111 | int poke_user(struct task_struct *child, long addr, long data)
| ^~~~~~~~~
../arch/x86/um/ptrace_64.c:171:5: warning: no previous prototype for ‘peek_user’ [-Wmissing-prototypes]
171 | int peek_user(struct task_struct *child, long addr, long data)
| ^~~~~~~~~
../arch/x86/um/signal.c:560:6: warning: no previous prototype for ‘sys_rt_sigreturn’ [-Wmissing-prototypes]
560 | long sys_rt_sigreturn(void)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/mem.c:202:8: warning: no previous prototype for ‘pgd_alloc’ [-Wmissing-prototypes]
202 | pgd_t *pgd_alloc(struct mm_struct *mm)
| ^~~~~~~~~
../arch/um/kernel/mem.c:215:7: warning: no previous prototype for ‘uml_kmalloc’ [-Wmissing-prototypes]
215 | void *uml_kmalloc(int size, int flags)
| ^~~~~~~~~~~
../arch/um/kernel/process.c:51:5: warning: no previous prototype for ‘pid_to_processor_id’ [-Wmissing-prototypes]
51 | int pid_to_processor_id(int pid)
| ^~~~~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:87:7: warning: no previous prototype for ‘__switch_to’ [-Wmissing-prototypes]
87 | void *__switch_to(struct task_struct *from, struct task_struct *to)
| ^~~~~~~~~~~
../arch/um/kernel/process.c:140:6: warning: no previous prototype for ‘fork_handler’ [-Wmissing-prototypes]
140 | void fork_handler(void)
| ^~~~~~~~~~~~
../arch/um/kernel/process.c:217:6: warning: no previous prototype for ‘arch_cpu_idle’ [-Wmissing-prototypes]
217 | void arch_cpu_idle(void)
| ^~~~~~~~~~~~~
../arch/um/kernel/process.c:253:5: warning: no previous prototype for ‘copy_to_user_proc’ [-Wmissing-prototypes]
253 | int copy_to_user_proc(void __user *to, void *from, int size)
| ^~~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:263:5: warning: no previous prototype for ‘clear_user_proc’ [-Wmissing-prototypes]
263 | int clear_user_proc(void __user *buf, int size)
| ^~~~~~~~~~~~~~~
../arch/um/kernel/process.c:271:6: warning: no previous prototype for ‘set_using_sysemu’ [-Wmissing-prototypes]
271 | void set_using_sysemu(int value)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:278:5: warning: no previous prototype for ‘get_using_sysemu’ [-Wmissing-prototypes]
278 | int get_using_sysemu(void)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:316:12: warning: no previous prototype for ‘make_proc_sysemu’ [-Wmissing-prototypes]
316 | int __init make_proc_sysemu(void)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:348:15: warning: no previous prototype for ‘arch_align_stack’ [-Wmissing-prototypes]
348 | unsigned long arch_align_stack(unsigned long sp)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:45:6: warning: no previous prototype for ‘machine_restart’ [-Wmissing-prototypes]
45 | void machine_restart(char * __unused)
| ^~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:51:6: warning: no previous prototype for ‘machine_power_off’ [-Wmissing-prototypes]
51 | void machine_power_off(void)
| ^~~~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:57:6: warning: no previous prototype for ‘machine_halt’ [-Wmissing-prototypes]
57 | void machine_halt(void)
| ^~~~~~~~~~~~
../arch/x86/um/syscalls_64.c:48:6: warning: no previous prototype for ‘arch_switch_to’ [-Wmissing-prototypes]
48 | void arch_switch_to(struct task_struct *to)
| ^~~~~~~~~~~~~~
../arch/um/kernel/tlb.c:579:6: warning: no previous prototype for ‘flush_tlb_mm_range’ [-Wmissing-prototypes]
579 | void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
| ^~~~~~~~~~~~~~~~~~
../arch/um/kernel/tlb.c:594:6: warning: no previous prototype for ‘force_flush_all’ [-Wmissing-prototypes]
594 | void force_flush_all(void)
| ^~~~~~~~~~~~~~~
../arch/um/kernel/um_arch.c:408:19: warning: no previous prototype for ‘read_initrd’ [-Wmissing-prototypes]
408 | int __init __weak read_initrd(void)
| ^~~~~~~~~~~
../arch/um/kernel/um_arch.c:461:7: warning: no previous prototype for ‘text_poke’ [-Wmissing-prototypes]
461 | void *text_poke(void *addr, const void *opcode, size_t len)
| ^~~~~~~~~
../arch/um/kernel/um_arch.c:473:6: warning: no previous prototype for ‘text_poke_sync’ [-Wmissing-prototypes]
473 | void text_poke_sync(void)
| ^~~~~~~~~~~~~~
../arch/um/kernel/kmsg_dump.c:60:12: warning: no previous prototype for ‘kmsg_dumper_stdout_init’ [-Wmissing-prototypes]
60 | int __init kmsg_dumper_stdout_init(void)
| ^~~~~~~~~~~~~~~~~~~~~~~
../arch/um/kernel/skas/process.c:36:12: warning: no previous prototype for ‘start_uml’ [-Wmissing-prototypes]
36 | int __init start_uml(void)
| ^~~~~~~~~
../arch/um/kernel/skas/mmu.c:17:5: warning: no previous prototype for ‘init_new_context’ [-Wmissing-prototypes]
17 | int init_new_context(struct task_struct *task, struct mm_struct *mm)
| ^~~~~~~~~~~~~~~~
../arch/um/kernel/skas/mmu.c:60:6: warning: no previous prototype for ‘destroy_context’ [-Wmissing-prototypes]
60 | void destroy_context(struct mm_struct *mm)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
156 | u64 ioread64_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
163 | u64 ioread64_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
170 | u64 ioread64be_lo_hi(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
178 | u64 ioread64be_hi_lo(const void __iomem *addr)
| ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
| ^~~~~~~~~~~~~~~~~
[16:54:12] Starting KUnit Kernel (1/1)...
[16:54:12] ============================================================
Running tests with:
$ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt
[16:54:12] ============ drm_test_pick_cmdline (2 subtests) ============
[16:54:12] [PASSED] drm_test_pick_cmdline_res_1920_1080_60
[16:54:12] =============== drm_test_pick_cmdline_named ===============
[16:54:12] [PASSED] NTSC
[16:54:12] [PASSED] NTSC-J
[16:54:12] [PASSED] PAL
[16:54:12] [PASSED] PAL-M
[16:54:12] =========== [PASSED] drm_test_pick_cmdline_named ===========
[16:54:12] ============== [PASSED] drm_test_pick_cmdline ==============
[16:54:12] ================== drm_buddy (7 subtests) ==================
[16:54:12] [PASSED] drm_test_buddy_alloc_limit
[16:54:12] [PASSED] drm_test_buddy_alloc_optimistic
[16:54:12] [PASSED] drm_test_buddy_alloc_pessimistic
[16:54:12] [PASSED] drm_test_buddy_alloc_pathological
[16:54:12] [PASSED] drm_test_buddy_alloc_contiguous
[16:54:12] [PASSED] drm_test_buddy_alloc_clear
[16:54:12] [PASSED] drm_test_buddy_alloc_range_bias
[16:54:12] ==================== [PASSED] drm_buddy ====================
[16:54:12] ============= drm_cmdline_parser (40 subtests) =============
[16:54:12] [PASSED] drm_test_cmdline_force_d_only
[16:54:12] [PASSED] drm_test_cmdline_force_D_only_dvi
[16:54:12] [PASSED] drm_test_cmdline_force_D_only_hdmi
[16:54:12] [PASSED] drm_test_cmdline_force_D_only_not_digital
[16:54:12] [PASSED] drm_test_cmdline_force_e_only
[16:54:12] [PASSED] drm_test_cmdline_res
[16:54:12] [PASSED] drm_test_cmdline_res_vesa
[16:54:12] [PASSED] drm_test_cmdline_res_vesa_rblank
[16:54:12] [PASSED] drm_test_cmdline_res_rblank
[16:54:12] [PASSED] drm_test_cmdline_res_bpp
[16:54:12] [PASSED] drm_test_cmdline_res_refresh
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_margins
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_force_off
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_analog
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_digital
[16:54:12] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced_margins_force_on
[16:54:12] [PASSED] drm_test_cmdline_res_margins_force_on
[16:54:12] [PASSED] drm_test_cmdline_res_vesa_margins
[16:54:12] [PASSED] drm_test_cmdline_name
[16:54:12] [PASSED] drm_test_cmdline_name_bpp
[16:54:12] [PASSED] drm_test_cmdline_name_option
[16:54:12] [PASSED] drm_test_cmdline_name_bpp_option
[16:54:12] [PASSED] drm_test_cmdline_rotate_0
[16:54:12] [PASSED] drm_test_cmdline_rotate_90
[16:54:12] [PASSED] drm_test_cmdline_rotate_180
[16:54:12] [PASSED] drm_test_cmdline_rotate_270
[16:54:12] [PASSED] drm_test_cmdline_hmirror
[16:54:12] [PASSED] drm_test_cmdline_vmirror
[16:54:12] [PASSED] drm_test_cmdline_margin_options
[16:54:12] [PASSED] drm_test_cmdline_multiple_options
[16:54:12] [PASSED] drm_test_cmdline_bpp_extra_and_option
[16:54:12] [PASSED] drm_test_cmdline_extra_and_option
[16:54:12] [PASSED] drm_test_cmdline_freestanding_options
[16:54:12] [PASSED] drm_test_cmdline_freestanding_force_e_and_options
[16:54:12] [PASSED] drm_test_cmdline_panel_orientation
[16:54:12] ================ drm_test_cmdline_invalid =================
[16:54:12] [PASSED] margin_only
[16:54:12] [PASSED] interlace_only
[16:54:12] [PASSED] res_missing_x
[16:54:12] [PASSED] res_missing_y
[16:54:12] [PASSED] res_bad_y
[16:54:12] [PASSED] res_missing_y_bpp
[16:54:12] [PASSED] res_bad_bpp
[16:54:12] [PASSED] res_bad_refresh
[16:54:12] [PASSED] res_bpp_refresh_force_on_off
[16:54:12] [PASSED] res_invalid_mode
[16:54:12] [PASSED] res_bpp_wrong_place_mode
[16:54:12] [PASSED] name_bpp_refresh
[16:54:12] [PASSED] name_refresh
[16:54:12] [PASSED] name_refresh_wrong_mode
[16:54:12] [PASSED] name_refresh_invalid_mode
[16:54:12] [PASSED] rotate_multiple
[16:54:12] [PASSED] rotate_invalid_val
[16:54:12] [PASSED] rotate_truncated
[16:54:12] [PASSED] invalid_option
[16:54:12] [PASSED] invalid_tv_option
[16:54:12] [PASSED] truncated_tv_option
[16:54:12] ============ [PASSED] drm_test_cmdline_invalid =============
[16:54:12] =============== drm_test_cmdline_tv_options ===============
[16:54:12] [PASSED] NTSC
[16:54:12] [PASSED] NTSC_443
[16:54:12] [PASSED] NTSC_J
[16:54:12] [PASSED] PAL
[16:54:12] [PASSED] PAL_M
[16:54:12] [PASSED] PAL_N
[16:54:12] [PASSED] SECAM
[16:54:12] =========== [PASSED] drm_test_cmdline_tv_options ===========
[16:54:12] =============== [PASSED] drm_cmdline_parser ================
[16:54:12] ============= drmm_connector_init (3 subtests) =============
[16:54:12] [PASSED] drm_test_drmm_connector_init
[16:54:12] [PASSED] drm_test_drmm_connector_init_null_ddc
[16:54:12] ========= drm_test_drmm_connector_init_type_valid =========
[16:54:12] [PASSED] Unknown
[16:54:12] [PASSED] VGA
[16:54:12] [PASSED] DVI-I
[16:54:12] [PASSED] DVI-D
[16:54:12] [PASSED] DVI-A
[16:54:12] [PASSED] Composite
[16:54:12] [PASSED] SVIDEO
[16:54:12] [PASSED] LVDS
[16:54:12] [PASSED] Component
[16:54:12] [PASSED] DIN
[16:54:12] [PASSED] DP
[16:54:12] [PASSED] HDMI-A
[16:54:12] [PASSED] HDMI-B
[16:54:12] [PASSED] TV
[16:54:12] [PASSED] eDP
[16:54:12] [PASSED] Virtual
[16:54:12] [PASSED] DSI
[16:54:12] [PASSED] DPI
[16:54:12] [PASSED] Writeback
[16:54:12] [PASSED] SPI
[16:54:12] [PASSED] USB
[16:54:12] ===== [PASSED] drm_test_drmm_connector_init_type_valid =====
[16:54:12] =============== [PASSED] drmm_connector_init ===============
[16:54:12] ========== drm_get_tv_mode_from_name (2 subtests) ==========
[16:54:12] ========== drm_test_get_tv_mode_from_name_valid ===========
[16:54:12] [PASSED] NTSC
[16:54:12] [PASSED] NTSC-443
[16:54:12] [PASSED] NTSC-J
[16:54:12] [PASSED] PAL
[16:54:12] [PASSED] PAL-M
[16:54:12] [PASSED] PAL-N
[16:54:12] [PASSED] SECAM
[16:54:12] ====== [PASSED] drm_test_get_tv_mode_from_name_valid =======
[16:54:12] [PASSED] drm_test_get_tv_mode_from_name_truncated
[16:54:12] ============ [PASSED] drm_get_tv_mode_from_name ============
[16:54:12] ============= drm_damage_helper (21 subtests) ==============
[16:54:12] [PASSED] drm_test_damage_iter_no_damage
[16:54:12] [PASSED] drm_test_damage_iter_no_damage_fractional_src
[16:54:12] [PASSED] drm_test_damage_iter_no_damage_src_moved
[16:54:12] [PASSED] drm_test_damage_iter_no_damage_fractional_src_moved
[16:54:12] [PASSED] drm_test_damage_iter_no_damage_not_visible
[16:54:12] [PASSED] drm_test_damage_iter_no_damage_no_crtc
[16:54:12] [PASSED] drm_test_damage_iter_no_damage_no_fb
[16:54:12] [PASSED] drm_test_damage_iter_simple_damage
[16:54:12] [PASSED] drm_test_damage_iter_single_damage
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_intersect_src
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_outside_src
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_fractional_src
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_intersect_fractional_src
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_outside_fractional_src
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_src_moved
[16:54:12] [PASSED] drm_test_damage_iter_single_damage_fractional_src_moved
[16:54:12] [PASSED] drm_test_damage_iter_damage
[16:54:12] [PASSED] drm_test_damage_iter_damage_one_intersect
[16:54:12] [PASSED] drm_test_damage_iter_damage_one_outside
[16:54:12] [PASSED] drm_test_damage_iter_damage_src_moved
[16:54:12] [PASSED] drm_test_damage_iter_damage_not_visible
[16:54:12] ================ [PASSED] drm_damage_helper ================
[16:54:12] ============== drm_dp_mst_helper (3 subtests) ==============
[16:54:12] ============== drm_test_dp_mst_calc_pbn_mode ==============
[16:54:12] [PASSED] Clock 154000 BPP 30 DSC disabled
[16:54:12] [PASSED] Clock 234000 BPP 30 DSC disabled
[16:54:12] [PASSED] Clock 297000 BPP 24 DSC disabled
[16:54:12] [PASSED] Clock 332880 BPP 24 DSC enabled
[16:54:12] [PASSED] Clock 324540 BPP 24 DSC enabled
[16:54:12] ========== [PASSED] drm_test_dp_mst_calc_pbn_mode ==========
[16:54:12] ============== drm_test_dp_mst_calc_pbn_div ===============
[16:54:12] [PASSED] Link rate 2000000 lane count 4
[16:54:12] [PASSED] Link rate 2000000 lane count 2
[16:54:12] [PASSED] Link rate 2000000 lane count 1
[16:54:12] [PASSED] Link rate 1350000 lane count 4
[16:54:12] [PASSED] Link rate 1350000 lane count 2
[16:54:12] [PASSED] Link rate 1350000 lane count 1
[16:54:12] [PASSED] Link rate 1000000 lane count 4
[16:54:12] [PASSED] Link rate 1000000 lane count 2
[16:54:12] [PASSED] Link rate 1000000 lane count 1
[16:54:12] [PASSED] Link rate 810000 lane count 4
[16:54:12] [PASSED] Link rate 810000 lane count 2
[16:54:12] [PASSED] Link rate 810000 lane count 1
[16:54:12] [PASSED] Link rate 540000 lane count 4
[16:54:12] [PASSED] Link rate 540000 lane count 2
[16:54:12] [PASSED] Link rate 540000 lane count 1
[16:54:12] [PASSED] Link rate 270000 lane count 4
[16:54:12] [PASSED] Link rate 270000 lane count 2
[16:54:12] [PASSED] Link rate 270000 lane count 1
[16:54:12] [PASSED] Link rate 162000 lane count 4
[16:54:12] [PASSED] Link rate 162000 lane count 2
[16:54:12] [PASSED] Link rate 162000 lane count 1
[16:54:12] ========== [PASSED] drm_test_dp_mst_calc_pbn_div ===========
[16:54:12] ========= drm_test_dp_mst_sideband_msg_req_decode =========
[16:54:12] [PASSED] DP_ENUM_PATH_RESOURCES with port number
[16:54:12] [PASSED] DP_POWER_UP_PHY with port number
[16:54:12] [PASSED] DP_POWER_DOWN_PHY with port number
[16:54:12] [PASSED] DP_ALLOCATE_PAYLOAD with SDP stream sinks
[16:54:12] [PASSED] DP_ALLOCATE_PAYLOAD with port number
[16:54:12] [PASSED] DP_ALLOCATE_PAYLOAD with VCPI
[16:54:12] [PASSED] DP_ALLOCATE_PAYLOAD with PBN
[16:54:12] [PASSED] DP_QUERY_PAYLOAD with port number
[16:54:12] [PASSED] DP_QUERY_PAYLOAD with VCPI
[16:54:12] [PASSED] DP_REMOTE_DPCD_READ with port number
[16:54:12] [PASSED] DP_REMOTE_DPCD_READ with DPCD address
[16:54:12] [PASSED] DP_REMOTE_DPCD_READ with max number of bytes
[16:54:12] [PASSED] DP_REMOTE_DPCD_WRITE with port number
[16:54:12] [PASSED] DP_REMOTE_DPCD_WRITE with DPCD address
[16:54:12] [PASSED] DP_REMOTE_DPCD_WRITE with data array
[16:54:12] [PASSED] DP_REMOTE_I2C_READ with port number
[16:54:12] [PASSED] DP_REMOTE_I2C_READ with I2C device ID
[16:54:12] [PASSED] DP_REMOTE_I2C_READ with transactions array
[16:54:12] [PASSED] DP_REMOTE_I2C_WRITE with port number
[16:54:12] [PASSED] DP_REMOTE_I2C_WRITE with I2C device ID
[16:54:12] [PASSED] DP_REMOTE_I2C_WRITE with data array
[16:54:12] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream ID
[16:54:12] [PASSED] DP_QUERY_STREAM_ENC_STATUS with client ID
[16:54:12] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream event
[16:54:12] [PASSED] DP_QUERY_STREAM_ENC_STATUS with valid stream event
[16:54:12] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream behavior
[16:54:12] [PASSED] DP_QUERY_STREAM_ENC_STATUS with a valid stream behavior
[16:54:12] ===== [PASSED] drm_test_dp_mst_sideband_msg_req_decode =====
[16:54:12] ================ [PASSED] drm_dp_mst_helper ================
[16:54:12] ================== drm_exec (7 subtests) ===================
[16:54:12] [PASSED] sanitycheck
[16:54:12] [PASSED] test_lock
[16:54:12] [PASSED] test_lock_unlock
[16:54:12] [PASSED] test_duplicates
[16:54:12] [PASSED] test_prepare
[16:54:12] [PASSED] test_prepare_array
[16:54:12] [PASSED] test_multiple_loops
[16:54:12] ==================== [PASSED] drm_exec =====================
[16:54:12] =========== drm_format_helper_test (17 subtests) ===========
[16:54:12] ============== drm_test_fb_xrgb8888_to_gray8 ==============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ========== [PASSED] drm_test_fb_xrgb8888_to_gray8 ==========
[16:54:12] ============= drm_test_fb_xrgb8888_to_rgb332 ==============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb332 ==========
[16:54:12] ============= drm_test_fb_xrgb8888_to_rgb565 ==============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb565 ==========
[16:54:12] ============ drm_test_fb_xrgb8888_to_xrgb1555 =============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======== [PASSED] drm_test_fb_xrgb8888_to_xrgb1555 =========
[16:54:12] ============ drm_test_fb_xrgb8888_to_argb1555 =============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======== [PASSED] drm_test_fb_xrgb8888_to_argb1555 =========
[16:54:12] ============ drm_test_fb_xrgb8888_to_rgba5551 =============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======== [PASSED] drm_test_fb_xrgb8888_to_rgba5551 =========
[16:54:12] ============= drm_test_fb_xrgb8888_to_rgb888 ==============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb888 ==========
[16:54:12] ============ drm_test_fb_xrgb8888_to_argb8888 =============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======== [PASSED] drm_test_fb_xrgb8888_to_argb8888 =========
[16:54:12] =========== drm_test_fb_xrgb8888_to_xrgb2101010 ===========
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======= [PASSED] drm_test_fb_xrgb8888_to_xrgb2101010 =======
[16:54:12] =========== drm_test_fb_xrgb8888_to_argb2101010 ===========
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======= [PASSED] drm_test_fb_xrgb8888_to_argb2101010 =======
[16:54:12] ============== drm_test_fb_xrgb8888_to_mono ===============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ========== [PASSED] drm_test_fb_xrgb8888_to_mono ===========
[16:54:12] ==================== drm_test_fb_swab =====================
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ================ [PASSED] drm_test_fb_swab =================
[16:54:12] ============ drm_test_fb_xrgb8888_to_xbgr8888 =============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======== [PASSED] drm_test_fb_xrgb8888_to_xbgr8888 =========
[16:54:12] ============ drm_test_fb_xrgb8888_to_abgr8888 =============
[16:54:12] [PASSED] single_pixel_source_buffer
[16:54:12] [PASSED] single_pixel_clip_rectangle
[16:54:12] [PASSED] well_known_colors
[16:54:12] [PASSED] destination_pitch
[16:54:12] ======== [PASSED] drm_test_fb_xrgb8888_to_abgr8888 =========
[16:54:12] ================= drm_test_fb_clip_offset =================
[16:54:12] [PASSED] pass through
[16:54:12] [PASSED] horizontal offset
[16:54:12] [PASSED] vertical offset
[16:54:12] [PASSED] horizontal and vertical offset
[16:54:12] [PASSED] horizontal offset (custom pitch)
[16:54:12] [PASSED] vertical offset (custom pitch)
[16:54:12] [PASSED] horizontal and vertical offset (custom pitch)
[16:54:12] ============= [PASSED] drm_test_fb_clip_offset =============
[16:54:12] ============== drm_test_fb_build_fourcc_list ==============
[16:54:12] [PASSED] no native formats
[16:54:12] [PASSED] XRGB8888 as native format
[16:54:12] [PASSED] remove duplicates
[16:54:12] [PASSED] convert alpha formats
[16:54:12] [PASSED] random formats
[16:54:12] ========== [PASSED] drm_test_fb_build_fourcc_list ==========
[16:54:12] =================== drm_test_fb_memcpy ====================
[16:54:12] [PASSED] single_pixel_source_buffer: XR24 little-endian (0x34325258)
[16:54:12] [PASSED] single_pixel_source_buffer: XRA8 little-endian (0x38415258)
[16:54:12] [PASSED] single_pixel_source_buffer: YU24 little-endian (0x34325559)
[16:54:12] [PASSED] single_pixel_clip_rectangle: XB24 little-endian (0x34324258)
[16:54:12] [PASSED] single_pixel_clip_rectangle: XRA8 little-endian (0x38415258)
[16:54:12] [PASSED] single_pixel_clip_rectangle: YU24 little-endian (0x34325559)
[16:54:12] [PASSED] well_known_colors: XB24 little-endian (0x34324258)
[16:54:12] [PASSED] well_known_colors: XRA8 little-endian (0x38415258)
[16:54:12] [PASSED] well_known_colors: YU24 little-endian (0x34325559)
[16:54:12] [PASSED] destination_pitch: XB24 little-endian (0x34324258)
[16:54:12] [PASSED] destination_pitch: XRA8 little-endian (0x38415258)
[16:54:12] [PASSED] destination_pitch: YU24 little-endian (0x34325559)
[16:54:12] =============== [PASSED] drm_test_fb_memcpy ================
[16:54:12] ============= [PASSED] drm_format_helper_test ==============
[16:54:12] ================= drm_format (18 subtests) =================
[16:54:12] [PASSED] drm_test_format_block_width_invalid
[16:54:12] [PASSED] drm_test_format_block_width_one_plane
[16:54:12] [PASSED] drm_test_format_block_width_two_plane
[16:54:12] [PASSED] drm_test_format_block_width_three_plane
[16:54:12] [PASSED] drm_test_format_block_width_tiled
[16:54:12] [PASSED] drm_test_format_block_height_invalid
[16:54:12] [PASSED] drm_test_format_block_height_one_plane
[16:54:12] [PASSED] drm_test_format_block_height_two_plane
[16:54:12] [PASSED] drm_test_format_block_height_three_plane
[16:54:12] [PASSED] drm_test_format_block_height_tiled
[16:54:12] [PASSED] drm_test_format_min_pitch_invalid
[16:54:12] [PASSED] drm_test_format_min_pitch_one_plane_8bpp
[16:54:12] [PASSED] drm_test_format_min_pitch_one_plane_16bpp
[16:54:12] [PASSED] drm_test_format_min_pitch_one_plane_24bpp
[16:54:12] [PASSED] drm_test_format_min_pitch_one_plane_32bpp
[16:54:12] [PASSED] drm_test_format_min_pitch_two_plane
[16:54:12] [PASSED] drm_test_format_min_pitch_three_plane_8bpp
[16:54:12] [PASSED] drm_test_format_min_pitch_tiled
[16:54:12] =================== [PASSED] drm_format ====================
[16:54:12] =============== drm_framebuffer (1 subtest) ================
[16:54:12] =============== drm_test_framebuffer_create ===============
[16:54:12] [PASSED] ABGR8888 normal sizes
[16:54:12] [PASSED] ABGR8888 max sizes
[16:54:12] [PASSED] ABGR8888 pitch greater than min required
[16:54:12] [PASSED] ABGR8888 pitch less than min required
[16:54:12] [PASSED] ABGR8888 Invalid width
[16:54:12] [PASSED] ABGR8888 Invalid buffer handle
[16:54:12] [PASSED] No pixel format
[16:54:12] [PASSED] ABGR8888 Width 0
[16:54:12] [PASSED] ABGR8888 Height 0
[16:54:12] [PASSED] ABGR8888 Out of bound height * pitch combination
[16:54:12] [PASSED] ABGR8888 Large buffer offset
[16:54:12] [PASSED] ABGR8888 Set DRM_MODE_FB_MODIFIERS without modifiers
[16:54:12] [PASSED] ABGR8888 Valid buffer modifier
[16:54:12] [PASSED] ABGR8888 Invalid buffer modifier(DRM_FORMAT_MOD_SAMSUNG_64_32_TILE)
[16:54:12] [PASSED] ABGR8888 Extra pitches without DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] ABGR8888 Extra pitches with DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] NV12 Normal sizes
[16:54:12] [PASSED] NV12 Max sizes
[16:54:12] [PASSED] NV12 Invalid pitch
[16:54:12] [PASSED] NV12 Invalid modifier/missing DRM_MODE_FB_MODIFIERS flag
[16:54:12] [PASSED] NV12 different modifier per-plane
[16:54:12] [PASSED] NV12 with DRM_FORMAT_MOD_SAMSUNG_64_32_TILE
[16:54:12] [PASSED] NV12 Valid modifiers without DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] NV12 Modifier for inexistent plane
[16:54:12] [PASSED] NV12 Handle for inexistent plane
[16:54:12] [PASSED] NV12 Handle for inexistent plane without DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] YVU420 DRM_MODE_FB_MODIFIERS set without modifier
[16:54:12] [PASSED] YVU420 Normal sizes
[16:54:12] [PASSED] YVU420 Max sizes
[16:54:12] [PASSED] YVU420 Invalid pitch
[16:54:12] [PASSED] YVU420 Different pitches
[16:54:12] [PASSED] YVU420 Different buffer offsets/pitches
[16:54:12] [PASSED] YVU420 Modifier set just for plane 0, without DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] YVU420 Modifier set just for planes 0, 1, without DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] YVU420 Modifier set just for plane 0, 1, with DRM_MODE_FB_MODIFIERS
[16:54:12] [PASSED] YVU420 Valid modifier
[16:54:12] [PASSED] YVU420 Different modifiers per plane
[16:54:12] [PASSED] YVU420 Modifier for inexistent plane
[16:54:12] [PASSED] X0L2 Normal sizes
[16:54:12] [PASSED] X0L2 Max sizes
[16:54:12] [PASSED] X0L2 Invalid pitch
[16:54:12] [PASSED] X0L2 Pitch greater than minimum required
[16:54:12] [PASSED] X0L2 Handle for inexistent plane
[16:54:12] [PASSED] X0L2 Offset for inexistent plane, without DRM_MODE_FB_MODIFIERS set
[16:54:12] [PASSED] X0L2 Modifier without DRM_MODE_FB_MODIFIERS set
[16:54:12] [PASSED] X0L2 Valid modifier
[16:54:12] [PASSED] X0L2 Modifier for inexistent plane
[16:54:12] =========== [PASSED] drm_test_framebuffer_create ===========
[16:54:12] ================= [PASSED] drm_framebuffer =================
[16:54:12] ================ drm_gem_shmem (8 subtests) ================
[16:54:12] [PASSED] drm_gem_shmem_test_obj_create
[16:54:12] [PASSED] drm_gem_shmem_test_obj_create_private
[16:54:12] [PASSED] drm_gem_shmem_test_pin_pages
[16:54:12] [PASSED] drm_gem_shmem_test_vmap
[16:54:12] [PASSED] drm_gem_shmem_test_get_pages_sgt
[16:54:12] [PASSED] drm_gem_shmem_test_get_sg_table
[16:54:12] [PASSED] drm_gem_shmem_test_madvise
[16:54:12] [PASSED] drm_gem_shmem_test_purge
[16:54:12] ================== [PASSED] drm_gem_shmem ==================
[16:54:12] ================= drm_managed (2 subtests) =================
[16:54:12] [PASSED] drm_test_managed_release_action
[16:54:12] [PASSED] drm_test_managed_run_action
[16:54:12] =================== [PASSED] drm_managed ===================
[16:54:12] =================== drm_mm (6 subtests) ====================
[16:54:12] [PASSED] drm_test_mm_init
[16:54:12] [PASSED] drm_test_mm_debug
[16:54:12] [PASSED] drm_test_mm_align32
[16:54:12] [PASSED] drm_test_mm_align64
[16:54:12] [PASSED] drm_test_mm_lowest
[16:54:12] [PASSED] drm_test_mm_highest
[16:54:12] ===================== [PASSED] drm_mm ======================
[16:54:12] ============= drm_modes_analog_tv (4 subtests) =============
[16:54:12] [PASSED] drm_test_modes_analog_tv_ntsc_480i
[16:54:12] [PASSED] drm_test_modes_analog_tv_ntsc_480i_inlined
[16:54:12] [PASSED] drm_test_modes_analog_tv_pal_576i
[16:54:12] [PASSED] drm_test_modes_analog_tv_pal_576i_inlined
[16:54:12] =============== [PASSED] drm_modes_analog_tv ===============
[16:54:12] ============== drm_plane_helper (2 subtests) ===============
[16:54:12] =============== drm_test_check_plane_state ================
[16:54:12] [PASSED] clipping_simple
[16:54:12] [PASSED] clipping_rotate_reflect
[16:54:12] [PASSED] positioning_simple
[16:54:12] [PASSED] upscaling
[16:54:12] [PASSED] downscaling
[16:54:12] [PASSED] rounding1
[16:54:12] [PASSED] rounding2
[16:54:12] [PASSED] rounding3
[16:54:12] [PASSED] rounding4
[16:54:12] =========== [PASSED] drm_test_check_plane_state ============
[16:54:12] =========== drm_test_check_invalid_plane_state ============
[16:54:12] [PASSED] positioning_invalid
[16:54:12] [PASSED] upscaling_invalid
[16:54:12] [PASSED] downscaling_invalid
[16:54:12] ======= [PASSED] drm_test_check_invalid_plane_state ========
[16:54:12] ================ [PASSED] drm_plane_helper =================
[16:54:12] ====== drm_connector_helper_tv_get_modes (1 subtest) =======
[16:54:12] ====== drm_test_connector_helper_tv_get_modes_check =======
[16:54:12] [PASSED] None
[16:54:12] [PASSED] PAL
[16:54:12] [PASSED] NTSC
[16:54:12] [PASSED] Both, NTSC Default
[16:54:12] [PASSED] Both, PAL Default
[16:54:12] [PASSED] Both, NTSC Default, with PAL on command-line
[16:54:12] [PASSED] Both, PAL Default, with NTSC on command-line
[16:54:12] == [PASSED] drm_test_connector_helper_tv_get_modes_check ===
[16:54:12] ======== [PASSED] drm_connector_helper_tv_get_modes ========
[16:54:12] ================== drm_rect (9 subtests) ===================
[16:54:12] [PASSED] drm_test_rect_clip_scaled_div_by_zero
[16:54:12] [PASSED] drm_test_rect_clip_scaled_not_clipped
[16:54:12] [PASSED] drm_test_rect_clip_scaled_clipped
stty: 'standard input': Inappropriate ioctl for device
[16:54:12] [PASSED] drm_test_rect_clip_scaled_signed_vs_unsigned
[16:54:12] ================= drm_test_rect_intersect =================
[16:54:12] [PASSED] top-left x bottom-right: 2x2+1+1 x 2x2+0+0
[16:54:12] [PASSED] top-right x bottom-left: 2x2+0+0 x 2x2+1-1
[16:54:12] [PASSED] bottom-left x top-right: 2x2+1-1 x 2x2+0+0
[16:54:12] [PASSED] bottom-right x top-left: 2x2+0+0 x 2x2+1+1
[16:54:12] [PASSED] right x left: 2x1+0+0 x 3x1+1+0
[16:54:12] [PASSED] left x right: 3x1+1+0 x 2x1+0+0
[16:54:12] [PASSED] up x bottom: 1x2+0+0 x 1x3+0-1
[16:54:12] [PASSED] bottom x up: 1x3+0-1 x 1x2+0+0
[16:54:12] [PASSED] touching corner: 1x1+0+0 x 2x2+1+1
[16:54:12] [PASSED] touching side: 1x1+0+0 x 1x1+1+0
[16:54:12] [PASSED] equal rects: 2x2+0+0 x 2x2+0+0
[16:54:12] [PASSED] inside another: 2x2+0+0 x 1x1+1+1
[16:54:12] [PASSED] far away: 1x1+0+0 x 1x1+3+6
[16:54:12] [PASSED] points intersecting: 0x0+5+10 x 0x0+5+10
[16:54:12] [PASSED] points not intersecting: 0x0+0+0 x 0x0+5+10
[16:54:12] ============= [PASSED] drm_test_rect_intersect =============
[16:54:12] ================ drm_test_rect_calc_hscale ================
[16:54:12] [PASSED] normal use
[16:54:12] [PASSED] out of max range
[16:54:12] [PASSED] out of min range
[16:54:12] [PASSED] zero dst
[16:54:12] [PASSED] negative src
[16:54:12] [PASSED] negative dst
[16:54:12] ============ [PASSED] drm_test_rect_calc_hscale ============
[16:54:12] ================ drm_test_rect_calc_vscale ================
[16:54:12] [PASSED] normal use
[16:54:12] [PASSED] out of max range
[16:54:12] [PASSED] out of min range
[16:54:12] [PASSED] zero dst
[16:54:12] [PASSED] negative src
[16:54:12] [PASSED] negative dst
[16:54:12] ============ [PASSED] drm_test_rect_calc_vscale ============
[16:54:12] ================== drm_test_rect_rotate ===================
[16:54:12] [PASSED] reflect-x
[16:54:12] [PASSED] reflect-y
[16:54:12] [PASSED] rotate-0
[16:54:12] [PASSED] rotate-90
[16:54:12] [PASSED] rotate-180
[16:54:12] [PASSED] rotate-270
[16:54:12] ============== [PASSED] drm_test_rect_rotate ===============
[16:54:12] ================ drm_test_rect_rotate_inv =================
[16:54:12] [PASSED] reflect-x
[16:54:12] [PASSED] reflect-y
[16:54:12] [PASSED] rotate-0
[16:54:12] [PASSED] rotate-90
[16:54:12] [PASSED] rotate-180
[16:54:12] [PASSED] rotate-270
[16:54:12] ============ [PASSED] drm_test_rect_rotate_inv =============
[16:54:12] ==================== [PASSED] drm_rect =====================
[16:54:12] ============================================================
[16:54:12] Testing complete. Ran 417 tests: passed: 417
[16:54:13] Elapsed time: 23.707s total, 1.703s configuring, 21.835s building, 0.151s running
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-04-29 16:54 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-29 16:47 [PATCH 0/5] Convert multiple bind ops to 1 job Matthew Brost
2024-04-29 16:47 ` [PATCH 1/5] drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue Matthew Brost
2024-04-29 16:47 ` [PATCH 2/5] drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops Matthew Brost
2024-04-29 16:47 ` [PATCH 3/5] drm/xe: Convert multiple bind ops into single job Matthew Brost
2024-04-29 16:47 ` [PATCH 4/5] drm/xe: Update VM trace events Matthew Brost
2024-04-29 16:47 ` [PATCH 5/5] drm/xe: Update PT layer with better error handling Matthew Brost
2024-04-29 16:52 ` ✓ CI.Patch_applied: success for Convert multiple bind ops to 1 job Patchwork
2024-04-29 16:53 ` ✗ CI.checkpatch: warning " Patchwork
2024-04-29 16:54 ` ✓ CI.KUnit: success " Patchwork
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox