[PATCH v4 0/7] Tiny steps towards amdgpu checkpoint and restore

[PATCH v4 0/7] Tiny steps towards amdgpu checkpoint and restore

[Tvrtko Ursulin]

I am sending this early with the main goal of getting a feel on how it will be
received etc.

Cover letter is written mostly for people who already know what checkpoint and
restore is, so I will just restate the high level goal - To be able to
checkpoint and restore purely a rendernode process (so no kfd in the picture).

And for people not too familiar what it is, probably best to first read about it
here: https://criu.org/Main_Page.

There are three pieces of work here:

1.
This kernel series which adds new uapi to amdgpu.

2.
A new IGT test case which helped me find what doesn't work and to verify what I
added.

https://cgit.freedesktop.org/~tursulin/intel-gpu-tools/log/?h=amd-criu

The IGT starts out adding some very basic tests (first commit) which are laid
out in order of increasing complexity.

Following commits then add more tests and by the end, with all these series
combined, they all pass apart from the forking subtest. But that one I am
leaving out of scope for now.

(And to be clear, before this work, none of the tests cases can pass.)

3.
Changes to the amdgpu CRIU plugin which use the above uapi, among other changes.

https://github.com/tursulin/criu/pull/new/amdgpu-render-node-rfc

As a picture is worth a thousand words the best I can do is some from terminal
pastes showing it all in action.

Terminal 1:

$ sudo ~/build-holo/tests/amdgpu/amd_criu --r busy-client-content
IGT-Version: 2.3-ge37a85b91 (x86_64) (Linux: 7.1.0-rc2-cfs x86_64)
Using IGT_SRANDOM=1779805687 for randomisation
Opened device: /dev/dri/renderD128
Starting subtest: busy-client-content
Start checkpointing within 10 seconds...

Now switch to terminal 2:

$ sudo /usr/local/sbin/criu dump -t `pgrep amd_criu | head -1` \
  -L /usr/local/lib/criu/ -vvv -o criu.log -j --link-remap --tcp-established \
  --file-locks --ext-unix-sk

Back to terminal 1:

...
Killed # This is normal - CRIU dump has saved and terminated the process

Back to terminal 2, lets restore it:

$ sudo /usr/local/sbin/criu restore  -L /usr/local/lib/criu/ -vvv \
  -o restore.log --shell-job --link-remap --tcp-established --file-locks 
  --ext-unix-sk
Subtest busy-client-content: SUCCESS (10.739s)

And that is it. Client which was busy looping submitting a SDMA_NOP IB was
successfuly checkpointed and restored to completion. It both ran for the
remainder of the indented duration and we checked buffer content was as
expected at the end.

There is definitely more to do. Play more with exported buffers, syncobjs,
fences, buffer objects lists, but for now, as said, I am looking for some early
feedback etc.

v2:
 * Dropped the reserved VM ID query.
 * Made userptr address an explicit union in the bo list entry uapi.
 * Changed copy buffer from synchronous to providing a fence to userspace.
 * Rebased for removal of ctx->guilty.

v3:
 * Reworked for XArray context handling which landed in the meantime.
 * Implemented feedback from Christian:
   * Copy buffer:
     * Use syncobj timeline instead of sync_file.
     * Add offset and size support.
     * Use DMA_RESV_USAGE_BOOKKEEP.
   * Rework userptr detection and fetching of flags (new patches in series).

v4:
 * Catchup with upstream amdgpu changes.

Tvrtko Ursulin (7):
  drm/amdgpu: Check correct field in amdgpu_ttm_tt_is_userptr()
  drm/amdgpu: Move amdgpu_ttm_tt_is_userptr() under conditional
    compilation
  drm/amdgpu: Extend amdgpu_ttm_tt_get_userptr() to return flags
  drm/amdgpu: Extend listing of buffer handles with the userptr object
    flag
  drm/amdgpu: Add a new ioctl for listing client contexts
  drm/amdgpu: Add context handle renaming operation
  drm/amdgpu: Add driver managed buffer copy

 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c  | 143 ++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c  |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c  | 146 ++++++++++++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h  |   4 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c  |  46 +++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h  |  16 ++-
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c |   4 +-
 include/uapi/drm/amdgpu_drm.h            |  59 ++++++++-
 8 files changed, 386 insertions(+), 36 deletions(-)

-- 
2.56.0

[PATCH v4 1/7] drm/amdgpu: Check correct field in amdgpu_ttm_tt_is_userptr()

[Tvrtko Ursulin]

Presence of a task is a better indicator whether something is a userptr
than the address of an userspace VMA.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Suggested-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 8f0720a77520..8acb12da6e8b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -1416,7 +1416,7 @@ bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
 {
 	struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
 
-	if (gtt == NULL || !gtt->userptr)
+	if (gtt == NULL || !gtt->usertask)
 		return false;
 
 	return true;
-- 
2.56.0

[PATCH v4 2/7] drm/amdgpu: Move amdgpu_ttm_tt_is_userptr() under conditional compilation

[Tvrtko Ursulin]

If userptr support is not compiled in there is no point to emit an out of
line function and respective calls which always returns false. Move the
declaration and implementation to the the already present conditional blocks.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Cc: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 26 ++++++++++++-------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h |  7 ++++++-
 2 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 8acb12da6e8b..cebeabd0ee72 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -802,6 +802,19 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
 	return r;
 }
 
+/*
+ * amdgpu_ttm_tt_is_userptr - Have the pages backing by userptr?
+ */
+bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
+{
+	struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
+
+	if (gtt == NULL || !gtt->usertask)
+		return false;
+
+	return true;
+}
+
 #endif
 
 /*
@@ -1409,19 +1422,6 @@ bool amdgpu_ttm_tt_affect_userptr(struct ttm_tt *ttm, unsigned long start,
 	return true;
 }
 
-/*
- * amdgpu_ttm_tt_is_userptr - Have the pages backing by userptr?
- */
-bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
-{
-	struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
-
-	if (gtt == NULL || !gtt->usertask)
-		return false;
-
-	return true;
-}
-
 /*
  * amdgpu_ttm_tt_is_readonly - Is the ttm_tt object read only?
  */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index adb6cfae5052..8d14d8a5c25c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -213,12 +213,18 @@ uint64_t amdgpu_ttm_domain_start(struct amdgpu_device *adev, uint32_t type);
 #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
 int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
 				 struct amdgpu_hmm_range *range);
+bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm);
 #else
 static inline int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
 					       struct amdgpu_hmm_range *range)
 {
 	return -EPERM;
 }
+
+static inline bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
+{
+	return false;
+}
 #endif
 
 /**
@@ -253,7 +259,6 @@ bool amdgpu_ttm_tt_affect_userptr(struct ttm_tt *ttm, unsigned long start,
 				  unsigned long end, unsigned long *userptr);
 bool amdgpu_ttm_tt_userptr_invalidated(struct ttm_tt *ttm,
 				       int *last_invalidated);
-bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm);
 bool amdgpu_ttm_tt_is_readonly(struct ttm_tt *ttm);
 uint64_t amdgpu_ttm_tt_pde_flags(struct ttm_tt *ttm, struct ttm_resource *mem);
 uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
-- 
2.56.0

[PATCH v4 3/7] drm/amdgpu: Extend amdgpu_ttm_tt_get_userptr() to return flags

[Tvrtko Ursulin]

Extend amdgpu_ttm_tt_get_userptr() to also return userptr creation flags
so those can be queried from a subsequent patch.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
Suggested-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c  | 6 ++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h  | 2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 4 +++-
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index cebeabd0ee72..c1b972258e93 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -1326,10 +1326,11 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev,
  * task
  *
  * @tbo: The ttm_buffer_object that contains the userptr
- * @user_addr:  The returned value
+ * @user_addr: Returned userptr address
+ * @user_flags: Returned userptr flags
  */
 int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo,
-			      uint64_t *user_addr)
+			      uint64_t *user_addr, uint32_t *user_flags)
 {
 	struct amdgpu_ttm_tt *gtt;
 
@@ -1338,6 +1339,7 @@ int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo,
 
 	gtt = (void *)tbo->ttm;
 	*user_addr = gtt->userptr;
+	*user_flags = gtt->userflags;
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index 8d14d8a5c25c..6967b5a06d20 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -250,7 +250,7 @@ static inline u64 amdgpu_gtt_node_to_byte_offset(const struct drm_mm_node *gtt_n
 
 void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct amdgpu_hmm_range *range);
 int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo,
-			      uint64_t *user_addr);
+			      uint64_t *user_addr, uint32_t *user_flags);
 int amdgpu_ttm_tt_set_userptr(struct ttm_buffer_object *bo,
 			      uint64_t addr, uint32_t flags);
 bool amdgpu_ttm_tt_has_userptr(struct ttm_tt *ttm);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index c9db79b86868..715d119fea36 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -2115,6 +2115,7 @@ static int criu_checkpoint_bos(struct kfd_process *p,
 			struct kfd_criu_bo_bucket *bo_bucket;
 			struct kfd_criu_bo_priv_data *bo_priv;
 			int i, dev_idx = 0;
+			u32 user_flags;
 
 			kgd_mem = (struct kgd_mem *)mem;
 			dumper_bo = kgd_mem->bo;
@@ -2137,7 +2138,8 @@ static int criu_checkpoint_bos(struct kfd_process *p,
 
 			if (bo_bucket->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_USERPTR) {
 				ret = amdgpu_ttm_tt_get_userptr(&dumper_bo->tbo,
-								&bo_priv->user_addr);
+								&bo_priv->user_addr,
+								&user_flags);
 				if (ret) {
 					pr_err("Failed to obtain user address for user-pointer bo\n");
 					goto exit;
-- 
2.56.0

[PATCH v4 4/7] drm/amdgpu: Extend listing of buffer handles with the userptr object flag

[Tvrtko Ursulin]

Add a new flag (AMDGPU_GEM_LIST_HANDLES_FLAG_IS_USERPTR) to the
DRM_IOCTL_AMDGPU_GEM_LIST_CONTEXTS ioctl.

This is required to allow for checkpoint and restore of userptr objects.

The existing struct drm_amdgpu_gem_list_handles_entry is used with the
userptr creation flags places in alloc_flags, while the alignment field
contains the userspace address.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 21 +++++++++++++++------
 include/uapi/drm/amdgpu_drm.h           | 14 ++++++++++----
 2 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 385be1e0944c..e3d3e205a4fa 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -1267,14 +1267,23 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 
 		bo_entry = &bo_entries[bo_index];
 
-		bo_entry->size = amdgpu_bo_size(bo);
-		bo_entry->alloc_flags = bo->flags & AMDGPU_GEM_CREATE_SETTABLE_MASK;
-		bo_entry->preferred_domains = bo->preferred_domains;
 		bo_entry->gem_handle = id;
-		bo_entry->alignment = (u64)bo->tbo.page_alignment << PAGE_SHIFT;
+		bo_entry->size = amdgpu_bo_size(bo);
+		if (amdgpu_ttm_tt_is_userptr(bo->tbo.ttm)) {
+			u32 flags;
 
-		if (bo->tbo.base.import_attach)
-			bo_entry->flags |= AMDGPU_GEM_LIST_HANDLES_FLAG_IS_IMPORT;
+			bo_entry->flags = AMDGPU_GEM_LIST_HANDLES_FLAG_IS_USERPTR;
+			amdgpu_ttm_tt_get_userptr(&bo->tbo, &bo_entry->userptr,
+						  &flags);
+			bo_entry->alloc_flags = flags;
+		} else {
+			bo_entry->alloc_flags = bo->flags & AMDGPU_GEM_CREATE_SETTABLE_MASK;
+			bo_entry->preferred_domains = bo->preferred_domains;
+			bo_entry->alignment = (u64)bo->tbo.page_alignment << PAGE_SHIFT;
+
+			if (bo->tbo.base.import_attach)
+				bo_entry->flags = AMDGPU_GEM_LIST_HANDLES_FLAG_IS_IMPORT;
+		}
 
 		bo_index += 1;
 	}
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 29009f6aabde..7620d6acfa0a 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -851,6 +851,7 @@ struct drm_amdgpu_gem_op {
 };
 
 #define AMDGPU_GEM_LIST_HANDLES_FLAG_IS_IMPORT	(1 << 0)
+#define AMDGPU_GEM_LIST_HANDLES_FLAG_IS_USERPTR	(1 << 1)
 
 struct drm_amdgpu_gem_list_handles {
 	/* User pointer to array of drm_amdgpu_gem_bo_info_entry */
@@ -866,7 +867,7 @@ struct drm_amdgpu_gem_list_handles_entry {
 	/* gem handle of buffer object */
 	__u32 gem_handle;
 
-	/* Currently just one flag: IS_IMPORT */
+	/* AMDGPU_GEM_LIST_HANDLES_FLAG_* */
 	__u32 flags;
 
 	/* Size of bo */
@@ -875,11 +876,16 @@ struct drm_amdgpu_gem_list_handles_entry {
 	/* Preferred domains for GEM_CREATE */
 	__u64 preferred_domains;
 
-	/* GEM_CREATE flags for re-creation of buffer */
+	/* GEM_CREATE flags for re-creation of buffer or drm_amdgpu_gem_userptr.flags */
 	__u64 alloc_flags;
 
-	/* physical start_addr alignment in bytes for some HW requirements */
-	__u64 alignment;
+	union {
+		/* physical start_addr alignment in bytes for some HW requirements */
+		__u64 alignment;
+
+		/* drm_amdgpu_gem_userptr.addr for userptr objects */
+		__u64 userptr;
+	};
 };
 
 #define AMDGPU_VA_OP_MAP			1
-- 
2.56.0

[PATCH v4 5/7] drm/amdgpu: Add a new ioctl for listing client contexts

[Tvrtko Ursulin]

Similar in principle to DRM_IOCTL_AMDGPU_GEM_LIST_HANDLES, we need a new
ioctl to list any contexts created by the client.

Lets add it as DRM_IOCTL_AMDGPU_GEM_LIST_CONTEXTS, and the respective uapi
data structures.

Structures contain all information required to re-create the context,
modulo the handle id replication, which will be handled in a following
patch.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 102 ++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h |   2 +
 include/uapi/drm/amdgpu_drm.h           |  31 +++++++
 4 files changed, 137 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
index 2f0349233695..5eb156d72db9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
@@ -723,6 +723,108 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
 	return r;
 }
 
+/**
+ * amdgpu_gem_list_contexts_ioctl - get information about clients contexts
+ *
+ * @dev: drm device pointer
+ * @data: drm_amdgpu_gem_list_handles
+ * @filp: drm file pointer
+ *
+ * Returns:
+ * 0 for success, -errno for errors.
+ */
+int amdgpu_gem_list_contexts_ioctl(struct drm_device *dev, void *data,
+				   struct drm_file *filp)
+{
+	struct drm_amdgpu_gem_list_contexts *args = data;
+	struct drm_amdgpu_gem_list_contexts_entry *contexts;
+	struct amdgpu_fpriv *fpriv = filp->driver_priv;
+	struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
+	struct amdgpu_device *adev = mgr->adev;
+	unsigned long id, idx = 0, num = 0;
+	struct amdgpu_ctx *ctx;
+	int ret = 0;
+
+	if (args->padding)
+		return -EINVAL;
+
+	xa_for_each(&mgr->ctx_handles, id, ctx)
+		num++;
+
+	if (num == 0 || args->num_contexts < num) {
+		args->num_contexts = num;
+		return 0;
+	}
+
+	contexts = kvzalloc_objs(*contexts, num);
+	if (!contexts)
+		return -ENOMEM;
+
+	xa_lock(&mgr->ctx_handles);
+	xa_for_each(&mgr->ctx_handles, id, ctx) {
+		struct drm_amdgpu_gem_list_contexts_entry *context;
+		enum amd_dpm_forced_level level;
+
+		if (idx >= num) {
+			ret = -EAGAIN;
+			break;
+		}
+
+		kref_get(&ctx->refcount);
+		xa_unlock(&mgr->ctx_handles);
+
+		context = &contexts[idx];
+
+		context->handle = id;
+		context->init_priority = ctx->init_priority;
+		context->override_priority = ctx->override_priority;
+
+		mutex_lock(&adev->pm.stable_pstate_ctx_lock);
+		if (ctx == adev->pm.stable_pstate_ctx) {
+			level = amdgpu_dpm_get_performance_level(adev);
+
+			switch (level) {
+			case AMD_DPM_FORCED_LEVEL_AUTO:
+				context->pstate_flags = AMDGPU_CTX_STABLE_PSTATE_NONE;
+				break;
+			case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD:
+				context->pstate_flags = AMDGPU_CTX_STABLE_PSTATE_STANDARD;
+				break;
+			case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK:
+				context->pstate_flags = AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK;
+				break;
+			case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK:
+				context->pstate_flags = AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK;
+				break;
+			case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK:
+				context->pstate_flags = AMDGPU_CTX_STABLE_PSTATE_PEAK;
+				break;
+			default:
+				ret = -EIO;
+				break;
+			};
+
+		}
+		mutex_unlock(&adev->pm.stable_pstate_ctx_lock);
+
+		idx++;
+		amdgpu_ctx_put(ctx);
+		xa_lock(&mgr->ctx_handles);
+	}
+	xa_unlock(&mgr->ctx_handles);
+
+	args->num_contexts = idx;
+
+	if (!ret)
+		if (copy_to_user(u64_to_user_ptr(args->contexts), contexts,
+				 num * sizeof(*contexts)))
+			ret = -EFAULT;
+
+	kvfree(contexts);
+
+	return ret;
+}
+
 struct amdgpu_ctx *amdgpu_ctx_get(struct amdgpu_fpriv *fpriv, uint32_t id)
 {
 	struct amdgpu_ctx *ctx;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index ab6f9131ac11..79d57659d748 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -3139,7 +3139,8 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = {
 	DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_WAIT, amdgpu_userq_wait_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(AMDGPU_GEM_LIST_HANDLES, amdgpu_gem_list_handles_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(AMDGPU_PROC_OPTIONS, amdgpu_proc_options_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
-	DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW)
+	DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(AMDGPU_GEM_LIST_CONTEXTS, amdgpu_gem_list_contexts_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
 };
 
 static const struct drm_driver amdgpu_kms_driver = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
index b558336bc4c6..0e17d9fc665f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
@@ -69,6 +69,8 @@ int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data,
 			struct drm_file *filp);
 int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 				  struct drm_file *filp);
+int amdgpu_gem_list_contexts_ioctl(struct drm_device *dev, void *data,
+				   struct drm_file *filp);
 
 int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void *data,
 				struct drm_file *filp);
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 7620d6acfa0a..7f7aae3c2498 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -60,6 +60,7 @@ extern "C" {
 #define DRM_AMDGPU_GEM_LIST_HANDLES	0x19
 #define DRM_AMDGPU_PROC_OPTIONS		0x1A
 #define DRM_AMDGPU_UALINK_HANDLE	0x1B
+#define DRM_AMDGPU_GEM_LIST_CONTEXTS	0x1C
 
 #define DRM_IOCTL_AMDGPU_GEM_CREATE	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create)
 #define DRM_IOCTL_AMDGPU_GEM_MMAP	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap)
@@ -83,6 +84,7 @@ extern "C" {
 #define DRM_IOCTL_AMDGPU_GEM_LIST_HANDLES DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_LIST_HANDLES, struct drm_amdgpu_gem_list_handles)
 #define DRM_IOCTL_AMDGPU_PROC_OPTIONS	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_PROC_OPTIONS, struct drm_amdgpu_proc_options)
 #define DRM_IOCTL_AMDGPU_UALINK_HANDLE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_UALINK_HANDLE, union drm_amdgpu_ualink_handle)
+#define DRM_IOCTL_AMDGPU_GEM_LIST_CONTEXTS DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_LIST_CONTEXTS, struct drm_amdgpu_gem_list_contexts)
 
 /**
  * DOC: memory domains
@@ -888,6 +890,35 @@ struct drm_amdgpu_gem_list_handles_entry {
 	};
 };
 
+struct drm_amdgpu_gem_list_contexts {
+	/* User pointer to array of drm_amdgpu_gem_list_contexts_entry */
+	__u64 contexts;
+
+	/* Size of the contexts buffer / Number of contexts in the client (if larger than size of buffer, must retry) */
+	__u32 num_contexts;
+
+	__u32 padding;
+};
+
+struct drm_amdgpu_gem_list_contexts_entry {
+	/* gem context handle */
+	__u32 handle;
+
+	/* AMDGPU_GEM_LIST_CONTEXTS_FLAG_* */
+	__u32 flags;
+
+	/* context initial priority */
+	__s32 init_priority;
+
+	/* context override priority */
+	__s32 override_priority;
+
+	/* pstate flags */
+	__u32 pstate_flags;
+
+	__u32 padding;
+};
+
 #define AMDGPU_VA_OP_MAP			1
 #define AMDGPU_VA_OP_UNMAP			2
 #define AMDGPU_VA_OP_CLEAR			3
-- 
2.56.0

[PATCH v4 6/7] drm/amdgpu: Add context handle renaming operation

[Tvrtko Ursulin]

Identical in principle to DRM_IOCTL_GEM_CHANGE_HANDLE, we need a method
for renaming the driver specific context handles.

We add a new operation (AMDGPU_CTX_OP_CHANGE_HANDLE) to the
DRM_IOCTL_AMDGPU_CTX ioctl.

Source handle is given in the existing input handle field, while the
destination handle needs to be passed in the existing flags field.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c | 41 +++++++++++++++++++++++++
 include/uapi/drm/amdgpu_drm.h           |  1 +
 2 files changed, 42 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
index 5eb156d72db9..d47ccbe7ab52 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.c
@@ -518,6 +518,44 @@ static int amdgpu_ctx_free(struct amdgpu_fpriv *fpriv, uint32_t id)
 	return ctx ? 0 : -EINVAL;
 }
 
+static int amdgpu_ctx_change_handle(struct amdgpu_fpriv *fpriv,
+				    uint32_t old, uint32_t id)
+{
+	struct amdgpu_ctx_mgr *mgr = &fpriv->ctx_mgr;
+	XA_STATE(xas, &mgr->ctx_handles, id);
+	struct amdgpu_ctx *ctx;
+	int r;
+
+	if (!old || !id || old == id)
+		return -EINVAL;
+
+	do {
+		xas_lock(&xas);
+
+		ctx = xa_load(&mgr->ctx_handles, old);
+		if (!ctx) {
+			r = -EINVAL;
+			goto unlock;
+		}
+
+		if (xas_load(&xas)) {
+			r = -EBUSY;
+			goto unlock;
+		}
+
+		xas_store(&xas, ctx);
+		r = xas_error(&xas);
+		if (r)
+			goto unlock;
+
+		__xa_erase(&mgr->ctx_handles, old);
+unlock:
+		xas_unlock(&xas);
+	} while (xas_nomem(&xas, GFP_KERNEL));
+
+	return r;
+}
+
 static int amdgpu_ctx_query(struct amdgpu_device *adev,
 			    struct amdgpu_fpriv *fpriv, uint32_t id,
 			    union drm_amdgpu_ctx_out *out)
@@ -716,6 +754,9 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data,
 			return -EINVAL;
 		r = amdgpu_ctx_stable_pstate(adev, fpriv, id, true, &stable_pstate);
 		break;
+	case AMDGPU_CTX_OP_CHANGE_HANDLE:
+		r = amdgpu_ctx_change_handle(fpriv, id, args->in.flags);
+		break;
 	default:
 		return -EINVAL;
 	}
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 7f7aae3c2498..ac0e7811c389 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -261,6 +261,7 @@ union drm_amdgpu_bo_list {
 #define AMDGPU_CTX_OP_QUERY_STATE2	4
 #define AMDGPU_CTX_OP_GET_STABLE_PSTATE	5
 #define AMDGPU_CTX_OP_SET_STABLE_PSTATE	6
+#define AMDGPU_CTX_OP_CHANGE_HANDLE	7
 
 /* GPU reset status */
 #define AMDGPU_CTX_NO_RESET		0
-- 
2.56.0

[PATCH v4 7/7] drm/amdgpu: Add driver managed buffer copy

[Tvrtko Ursulin]

Saving and restoring buffer object content poses a challenge for the
checkpoint and restore process for at least two reasons.

For example not all objects can be exported as dma-buf to enable copying
from a separate client context, neither can any objects be easily copied
from the same context since the injected CRIU code is unaware of the GPU
virtual memory free and allocated ranges.

Lets bypass both problems by simply exposing access to the already present
fast kernel copy via a new DRM_IOCTL_AMDGPU_GEM_COPY_BUFFER ioctl.

By giving the kernel simply the source and destination handles it is able
to copy them without the need for objects to be mapped, or shared with a
separate client.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c |   1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 125 ++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c |  14 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h |   7 ++
 include/uapi/drm/amdgpu_drm.h           |  13 +++
 6 files changed, 155 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 79d57659d748..c78e8c048c63 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -3141,6 +3141,7 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = {
 	DRM_IOCTL_DEF_DRV(AMDGPU_PROC_OPTIONS, amdgpu_proc_options_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(AMDGPU_UALINK_HANDLE, amdgpu_gem_ualink_handle_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(AMDGPU_GEM_LIST_CONTEXTS, amdgpu_gem_list_contexts_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(AMDGPU_GEM_COPY_BUFFER, amdgpu_gem_copy_buffer_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
 };
 
 static const struct drm_driver amdgpu_kms_driver = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index e3d3e205a4fa..7ad14bdfa292 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -30,6 +30,7 @@
 #include <linux/pagemap.h>
 #include <linux/pci.h>
 #include <linux/dma-buf.h>
+#include <linux/dma-fence-chain.h>
 #include <linux/dma-fence-unwrap.h>
 #include <linux/uaccess.h>
 #include <linux/pm_runtime.h>
@@ -1300,6 +1301,130 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 	return ret;
 }
 
+/**
+ * amdgpu_gem_copy_buffer_ioctl - copy buffer object content
+ *
+ * @dev: drm device pointer
+ * @data: drm_amdgpu_gem_copy_buffer
+ * @filp: drm file pointer
+ *
+ * Returns:
+ * 0 for success, -errno for errors.
+ */
+int amdgpu_gem_copy_buffer_ioctl(struct drm_device *dev, void *data,
+				 struct drm_file *filp)
+{
+	struct amdgpu_copy_mem src_mem = {}, dst_mem = {};
+	struct drm_amdgpu_gem_copy_buffer *args = data;
+	struct amdgpu_device *adev = drm_to_adev(dev);
+	struct drm_gem_object *src_gobj, *dst_gobj;
+	struct amdgpu_bo *src_bo, *dst_bo;
+	struct dma_fence *fence = NULL;
+	struct dma_fence_chain *chain;
+	struct drm_syncobj *syncobj;
+	struct drm_exec exec;
+	unsigned int e;
+	u64 end;
+	int r;
+
+	if (args->mbz)
+		return -EINVAL;
+
+	if (!args->syncobj_handle || !args->timeline_point)
+		return -EINVAL;
+
+	src_gobj = drm_gem_object_lookup(filp, args->src_handle);
+	if (!src_gobj)
+		return -ENOENT;
+
+	dst_gobj = drm_gem_object_lookup(filp, args->dst_handle);
+	if (!dst_gobj) {
+		r = -ENOENT;
+		goto err_dst;
+	}
+
+	syncobj = drm_syncobj_find(filp, args->syncobj_handle);
+	if (!syncobj) {
+		r = -EINVAL;
+		goto err_syncobj;
+	}
+
+	chain = dma_fence_chain_alloc();
+	if (!chain) {
+		r = -ENOMEM;
+		goto err_chain;
+	}
+
+	src_bo = gem_to_amdgpu_bo(src_gobj);
+	dst_bo = gem_to_amdgpu_bo(dst_gobj);
+
+	if (check_add_overflow(args->src_offset, args->copy_size, &end) ||
+	    end > amdgpu_bo_size(src_bo)) {
+		r = -E2BIG;
+		goto err_sizes;
+	}
+
+	if (check_add_overflow(args->dst_offset, args->copy_size, &end) ||
+	    end > amdgpu_bo_size(dst_bo)) {
+		r = -E2BIG;
+		goto err_sizes;
+	}
+
+	drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
+
+	drm_exec_until_all_locked(&exec) {
+		r = drm_exec_prepare_obj(&exec, &src_bo->tbo.base, 1);
+		drm_exec_retry_on_contention(&exec);
+		if (r)
+			goto err_unlock;
+
+		r = drm_exec_prepare_obj(&exec, &dst_bo->tbo.base, 1);
+		drm_exec_retry_on_contention(&exec);
+		if (r)
+			goto err_unlock;
+	}
+
+	src_mem.bo = &src_bo->tbo;
+	src_mem.mem = src_bo->tbo.resource;
+	src_mem.offset = args->src_offset;
+	dst_mem.bo = &dst_bo->tbo;
+	dst_mem.mem = dst_bo->tbo.resource;
+	dst_mem.offset = args->dst_offset;
+
+	e = atomic_inc_return(&adev->mman.next_move_entity) %
+			      adev->mman.num_move_entities;
+
+	r = amdgpu_ttm_copy_mem_to_mem(adev, &adev->mman.move_entities[e],
+				       &src_mem, &dst_mem,
+				       args->copy_size,
+				       amdgpu_bo_encrypted(src_bo),
+				       dst_bo->tbo.base.resv, &fence);
+	if (r) {
+		goto err_unlock;
+	} else if (!fence) {
+		r = -ENXIO;
+		goto err_unlock;
+	}
+
+	dma_resv_add_fence(src_bo->tbo.base.resv, fence, DMA_RESV_USAGE_BOOKKEEP);
+	dma_resv_add_fence(dst_bo->tbo.base.resv, fence, DMA_RESV_USAGE_BOOKKEEP);
+	drm_syncobj_add_point(syncobj, chain, fence, args->timeline_point);
+	dma_fence_put(fence);
+
+err_unlock:
+	drm_exec_fini(&exec);
+err_sizes:
+	if (r)
+		dma_fence_chain_free(chain);
+err_chain:
+	drm_syncobj_put(syncobj);
+err_syncobj:
+	drm_gem_object_put(dst_gobj);
+err_dst:
+	drm_gem_object_put(src_gobj);
+	return r;
+}
+
 static unsigned int amdgpu_gem_align_pitch(struct amdgpu_device *adev,
 					   unsigned int width,
 					   unsigned int cpp,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
index 0e17d9fc665f..64529680fdad 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.h
@@ -71,6 +71,8 @@ int amdgpu_gem_list_handles_ioctl(struct drm_device *dev, void *data,
 				  struct drm_file *filp);
 int amdgpu_gem_list_contexts_ioctl(struct drm_device *dev, void *data,
 				   struct drm_file *filp);
+int amdgpu_gem_copy_buffer_ioctl(struct drm_device *dev, void *data,
+				 struct drm_file *filp);
 
 int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void *data,
 				struct drm_file *filp);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index c1b972258e93..0254a006e2f1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -308,13 +308,13 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity,
  *
  */
 __attribute__((nonnull))
-static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
-				      struct amdgpu_ttm_buffer_entity *entity,
-				      const struct amdgpu_copy_mem *src,
-				      const struct amdgpu_copy_mem *dst,
-				      uint64_t size, bool tmz,
-				      struct dma_resv *resv,
-				      struct dma_fence **f)
+int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
+			       struct amdgpu_ttm_buffer_entity *entity,
+			       const struct amdgpu_copy_mem *src,
+			       const struct amdgpu_copy_mem *dst,
+			       uint64_t size, bool tmz,
+			       struct dma_resv *resv,
+			       struct dma_fence **f)
 {
 	struct amdgpu_res_cursor src_mm, dst_mm;
 	struct dma_fence *fence = NULL;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index 6967b5a06d20..0bdf28a00398 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -276,4 +276,11 @@ void amdgpu_ttm_mmio_remap_free_sgt(struct device *dev,
 				    enum dma_data_direction dir,
 				    struct sg_table *sgt);
 
+int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
+			       struct amdgpu_ttm_buffer_entity *entity,
+			       const struct amdgpu_copy_mem *src,
+			       const struct amdgpu_copy_mem *dst,
+			       uint64_t size, bool tmz,
+			       struct dma_resv *resv,
+			       struct dma_fence **f);
 #endif
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index ac0e7811c389..8de748f37e2f 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -61,6 +61,7 @@ extern "C" {
 #define DRM_AMDGPU_PROC_OPTIONS		0x1A
 #define DRM_AMDGPU_UALINK_HANDLE	0x1B
 #define DRM_AMDGPU_GEM_LIST_CONTEXTS	0x1C
+#define DRM_AMDGPU_GEM_COPY_BUFFER	0x1D
 
 #define DRM_IOCTL_AMDGPU_GEM_CREATE	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create)
 #define DRM_IOCTL_AMDGPU_GEM_MMAP	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap)
@@ -85,6 +86,7 @@ extern "C" {
 #define DRM_IOCTL_AMDGPU_PROC_OPTIONS	DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_PROC_OPTIONS, struct drm_amdgpu_proc_options)
 #define DRM_IOCTL_AMDGPU_UALINK_HANDLE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_UALINK_HANDLE, union drm_amdgpu_ualink_handle)
 #define DRM_IOCTL_AMDGPU_GEM_LIST_CONTEXTS DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_LIST_CONTEXTS, struct drm_amdgpu_gem_list_contexts)
+#define DRM_IOCTL_AMDGPU_GEM_COPY_BUFFER DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_COPY_BUFFER, struct drm_amdgpu_gem_copy_buffer)
 
 /**
  * DOC: memory domains
@@ -216,6 +218,17 @@ union drm_amdgpu_gem_create {
 	struct drm_amdgpu_gem_create_out	out;
 };
 
+struct drm_amdgpu_gem_copy_buffer {
+	__u32	src_handle;
+	__u32	dst_handle;
+	__u64	src_offset;
+	__u64	dst_offset;
+	__u64	copy_size;
+	__u32	syncobj_handle;
+	__u32   mbz;
+	__u64   timeline_point;
+};
+
 /** Opcode to create new residency list.  */
 #define AMDGPU_BO_LIST_OP_CREATE	0
 /** Opcode to destroy previously created residency list */
-- 
2.56.0

Re: [PATCH v4 1/7] drm/amdgpu: Check correct field in amdgpu_ttm_tt_is_userptr()

[Christian König]

On 10/19/26 15:01, Tvrtko Ursulin wrote:
> Presence of a task is a better indicator whether something is a userptr
> than the address of an userspace VMA.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
> Suggested-by: Christian König <christian.koenig@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index 8f0720a77520..8acb12da6e8b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1416,7 +1416,7 @@ bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
>  {
>  	struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
>  
> -	if (gtt == NULL || !gtt->userptr)
> +	if (gtt == NULL || !gtt->usertask)
>  		return false;
>  
>  	return true;

Re: [PATCH v4 2/7] drm/amdgpu: Move amdgpu_ttm_tt_is_userptr() under conditional compilation

[Christian König]

On 10/19/26 15:01, Tvrtko Ursulin wrote:
> If userptr support is not compiled in there is no point to emit an out of
> line function and respective calls which always returns false. Move the
> declaration and implementation to the the already present conditional blocks.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
> Cc: Christian König <christian.koenig@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 26 ++++++++++++-------------
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h |  7 ++++++-
>  2 files changed, 19 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index 8acb12da6e8b..cebeabd0ee72 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -802,6 +802,19 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
>  	return r;
>  }
>  
> +/*
> + * amdgpu_ttm_tt_is_userptr - Have the pages backing by userptr?
> + */
> +bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
> +{
> +	struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
> +
> +	if (gtt == NULL || !gtt->usertask)
> +		return false;
> +
> +	return true;
> +}
> +
>  #endif
>  
>  /*
> @@ -1409,19 +1422,6 @@ bool amdgpu_ttm_tt_affect_userptr(struct ttm_tt *ttm, unsigned long start,
>  	return true;
>  }
>  
> -/*
> - * amdgpu_ttm_tt_is_userptr - Have the pages backing by userptr?
> - */
> -bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
> -{
> -	struct amdgpu_ttm_tt *gtt = ttm_to_amdgpu_ttm_tt(ttm);
> -
> -	if (gtt == NULL || !gtt->usertask)
> -		return false;
> -
> -	return true;
> -}
> -
>  /*
>   * amdgpu_ttm_tt_is_readonly - Is the ttm_tt object read only?
>   */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> index adb6cfae5052..8d14d8a5c25c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> @@ -213,12 +213,18 @@ uint64_t amdgpu_ttm_domain_start(struct amdgpu_device *adev, uint32_t type);
>  #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR)
>  int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
>  				 struct amdgpu_hmm_range *range);
> +bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm);
>  #else
>  static inline int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo,
>  					       struct amdgpu_hmm_range *range)
>  {
>  	return -EPERM;
>  }
> +
> +static inline bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm)
> +{
> +	return false;
> +}
>  #endif
>  
>  /**
> @@ -253,7 +259,6 @@ bool amdgpu_ttm_tt_affect_userptr(struct ttm_tt *ttm, unsigned long start,
>  				  unsigned long end, unsigned long *userptr);
>  bool amdgpu_ttm_tt_userptr_invalidated(struct ttm_tt *ttm,
>  				       int *last_invalidated);
> -bool amdgpu_ttm_tt_is_userptr(struct ttm_tt *ttm);
>  bool amdgpu_ttm_tt_is_readonly(struct ttm_tt *ttm);
>  uint64_t amdgpu_ttm_tt_pde_flags(struct ttm_tt *ttm, struct ttm_resource *mem);
>  uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,