* [PATCH v9 1/5] drm/amdgpu: Implement a new userqueue fence driver
@ 2024-04-29 6:43 Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 2/5] drm/amdgpu: Add mqd support for the fence address Arunpravin Paneer Selvam
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Arunpravin Paneer Selvam @ 2024-04-29 6:43 UTC (permalink / raw)
To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam
Developed a userqueue fence driver for the userqueue process shared
BO synchronization.
Create a dma fence having write pointer as the seqno and allocate a
seq64 memory for each user queue process and feed this memory address
into the firmware/hardware, thus the firmware writes the read pointer
into the given address when the process completes it execution.
Compare wptr and rptr, if rptr >= wptr, signal the fences for the waiting
process to consume the buffers.
v2: Worked on review comments from Christian for the following
modifications
- Add wptr as sequence number into the fence
- Add a reference count for the fence driver
- Add dma_fence_put below the list_del as it might
frees the userq fence.
- Trim unnecessary code in interrupt handler.
- Check dma fence signaled state in dma fence creation
function for a potential problem of hardware completing
the job processing beforehand.
- Add necessary locks.
- Create a list and process all the unsignaled fences.
- clean up fences in destroy function.
- implement .signaled callback function
v3: Worked on review comments from Christian
- Modify naming convention for reference counted objects
- Fix fence driver reference drop issue
- Drop amdgpu_userq_fence_driver_process() function return value
v4: Worked on review comments from Christian
- Moved fence driver allocation into amdgpu_userq_fence_driver_alloc()
- Added detail doc mentioning the differences b/w
two spinlocks declared.
v5: Worked on review comments from Christian
- Check before upcast and remove local variable
- Add error handling in fence_drv alloc function.
- Move rptr read fn outside of the loop and remove WARN_ON in
destroy function.
v6:
- clear the seq64 memory in user fence driver(Christian)
- fix for the wptr va bo mapping(Christian)
- move the fence_drv xa entry erase code from the interrupt handler
into user fence destroy function
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Suggested-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/amd/amdgpu/Makefile | 3 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 +
.../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 257 ++++++++++++++++++
.../gpu/drm/amd/amdgpu/amdgpu_userq_fence.h | 69 +++++
drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 12 +-
.../gpu/drm/amd/include/amdgpu_userqueue.h | 5 +-
6 files changed, 347 insertions(+), 5 deletions(-)
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index a640bfa468ad..f8dd808bd71c 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -80,7 +80,8 @@ amdgpu-y += amdgpu_device.o amdgpu_doorbell_mgr.o amdgpu_kms.o \
amdgpu_umc.o smu_v11_0_i2c.o amdgpu_fru_eeprom.o amdgpu_rap.o \
amdgpu_fw_attestation.o amdgpu_securedisplay.o \
amdgpu_eeprom.o amdgpu_mca.o amdgpu_psp_ta.o amdgpu_lsdma.o \
- amdgpu_ring_mux.o amdgpu_xcp.o amdgpu_seq64.o amdgpu_aca.o
+ amdgpu_ring_mux.o amdgpu_xcp.o amdgpu_seq64.o amdgpu_aca.o \
+ amdgpu_userq_fence.o
amdgpu-$(CONFIG_PROC_FS) += amdgpu_fdinfo.o
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index acee1c279abb..844f7b5f90db 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -51,6 +51,7 @@
#include "amdgpu_sched.h"
#include "amdgpu_xgmi.h"
#include "amdgpu_userqueue.h"
+#include "amdgpu_userq_fence.h"
#include "../amdxcp/amdgpu_xcp_drv.h"
/*
@@ -3012,6 +3013,10 @@ static int __init amdgpu_init(void)
if (r)
goto error_fence;
+ r = amdgpu_userq_fence_slab_init();
+ if (r)
+ goto error_fence;
+
DRM_INFO("amdgpu kernel modesetting enabled.\n");
amdgpu_register_atpx_handler();
amdgpu_acpi_detect();
@@ -3037,6 +3042,7 @@ static void __exit amdgpu_exit(void)
amdgpu_acpi_release();
amdgpu_sync_fini();
amdgpu_fence_slab_fini();
+ amdgpu_userq_fence_slab_fini();
mmu_notifier_synchronize();
amdgpu_xcp_drv_release();
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
new file mode 100644
index 000000000000..f7baea2c67ab
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
@@ -0,0 +1,257 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2023 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/kref.h>
+#include <linux/slab.h>
+
+#include <drm/drm_syncobj.h>
+
+#include "amdgpu.h"
+#include "amdgpu_userq_fence.h"
+
+static const struct dma_fence_ops amdgpu_userq_fence_ops;
+static struct kmem_cache *amdgpu_userq_fence_slab;
+
+int amdgpu_userq_fence_slab_init(void)
+{
+ amdgpu_userq_fence_slab = kmem_cache_create("amdgpu_userq_fence",
+ sizeof(struct amdgpu_userq_fence),
+ 0,
+ SLAB_HWCACHE_ALIGN,
+ NULL);
+ if (!amdgpu_userq_fence_slab)
+ return -ENOMEM;
+
+ return 0;
+}
+
+void amdgpu_userq_fence_slab_fini(void)
+{
+ rcu_barrier();
+ kmem_cache_destroy(amdgpu_userq_fence_slab);
+}
+
+static inline struct amdgpu_userq_fence *to_amdgpu_userq_fence(struct dma_fence *f)
+{
+ if (!f || f->ops != &amdgpu_userq_fence_ops)
+ return NULL;
+
+ return container_of(f, struct amdgpu_userq_fence, base);
+}
+
+static u64 amdgpu_userq_fence_read(struct amdgpu_userq_fence_driver *fence_drv)
+{
+ return le64_to_cpu(*fence_drv->cpu_addr);
+}
+
+int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
+ struct amdgpu_usermode_queue *userq)
+{
+ struct amdgpu_userq_fence_driver *fence_drv;
+ int r;
+
+ fence_drv = kzalloc(sizeof(*fence_drv), GFP_KERNEL);
+ if (!fence_drv) {
+ DRM_ERROR("Failed to allocate memory for fence driver\n");
+ return -ENOMEM;
+ }
+
+ /* Acquire seq64 memory */
+ r = amdgpu_seq64_alloc(adev, &fence_drv->gpu_addr,
+ &fence_drv->cpu_addr);
+ if (r) {
+ kfree(fence_drv);
+ return -ENOMEM;
+ }
+
+ memset(fence_drv->cpu_addr, 0, sizeof(u64));
+
+ kref_init(&fence_drv->refcount);
+ INIT_LIST_HEAD(&fence_drv->fences);
+ spin_lock_init(&fence_drv->fence_list_lock);
+
+ fence_drv->adev = adev;
+ fence_drv->context = dma_fence_context_alloc(1);
+ get_task_comm(fence_drv->timeline_name, current);
+
+ userq->fence_drv = fence_drv;
+
+ return 0;
+}
+
+void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv)
+{
+ struct amdgpu_userq_fence *userq_fence, *tmp;
+ struct dma_fence *fence;
+ u64 rptr;
+
+ if (!fence_drv)
+ return;
+
+ rptr = amdgpu_userq_fence_read(fence_drv);
+
+ spin_lock(&fence_drv->fence_list_lock);
+ list_for_each_entry_safe(userq_fence, tmp, &fence_drv->fences, link) {
+ fence = &userq_fence->base;
+
+ if (rptr >= fence->seqno) {
+ dma_fence_signal(fence);
+ list_del(&userq_fence->link);
+
+ dma_fence_put(fence);
+ } else {
+ break;
+ }
+ }
+ spin_unlock(&fence_drv->fence_list_lock);
+}
+
+void amdgpu_userq_fence_driver_destroy(struct kref *ref)
+{
+ struct amdgpu_userq_fence_driver *fence_drv = container_of(ref,
+ struct amdgpu_userq_fence_driver,
+ refcount);
+ struct amdgpu_device *adev = fence_drv->adev;
+ struct amdgpu_userq_fence *fence, *tmp;
+ struct dma_fence *f;
+
+ spin_lock(&fence_drv->fence_list_lock);
+ list_for_each_entry_safe(fence, tmp, &fence_drv->fences, link) {
+ f = &fence->base;
+
+ if (!dma_fence_is_signaled(f)) {
+ dma_fence_set_error(f, -ECANCELED);
+ dma_fence_signal(f);
+ }
+
+ list_del(&fence->link);
+ dma_fence_put(f);
+ }
+ spin_unlock(&fence_drv->fence_list_lock);
+
+ /* Free seq64 memory */
+ amdgpu_seq64_free(adev, fence_drv->gpu_addr);
+ kfree(fence_drv);
+}
+
+void amdgpu_userq_fence_driver_get(struct amdgpu_userq_fence_driver *fence_drv)
+{
+ kref_get(&fence_drv->refcount);
+}
+
+void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv)
+{
+ kref_put(&fence_drv->refcount, amdgpu_userq_fence_driver_destroy);
+}
+
+int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq,
+ u64 seq, struct dma_fence **f)
+{
+ struct amdgpu_userq_fence_driver *fence_drv;
+ struct amdgpu_userq_fence *userq_fence;
+ struct dma_fence *fence;
+
+ fence_drv = userq->fence_drv;
+ if (!fence_drv)
+ return -EINVAL;
+
+ userq_fence = kmem_cache_alloc(amdgpu_userq_fence_slab, GFP_ATOMIC);
+ if (!userq_fence)
+ return -ENOMEM;
+
+ spin_lock_init(&userq_fence->lock);
+ INIT_LIST_HEAD(&userq_fence->link);
+ fence = &userq_fence->base;
+ userq_fence->fence_drv = fence_drv;
+
+ dma_fence_init(fence, &amdgpu_userq_fence_ops, &userq_fence->lock,
+ fence_drv->context, seq);
+
+ amdgpu_userq_fence_driver_get(fence_drv);
+ dma_fence_get(fence);
+
+ spin_lock(&fence_drv->fence_list_lock);
+ /* Check if hardware has already processed the job */
+ if (!dma_fence_is_signaled(fence))
+ list_add_tail(&userq_fence->link, &fence_drv->fences);
+ else
+ dma_fence_put(fence);
+
+ spin_unlock(&fence_drv->fence_list_lock);
+
+ *f = fence;
+
+ return 0;
+}
+
+static const char *amdgpu_userq_fence_get_driver_name(struct dma_fence *f)
+{
+ return "amdgpu_userqueue_fence";
+}
+
+static const char *amdgpu_userq_fence_get_timeline_name(struct dma_fence *f)
+{
+ struct amdgpu_userq_fence *fence = to_amdgpu_userq_fence(f);
+
+ return fence->fence_drv->timeline_name;
+}
+
+static bool amdgpu_userq_fence_signaled(struct dma_fence *f)
+{
+ struct amdgpu_userq_fence *fence = to_amdgpu_userq_fence(f);
+ struct amdgpu_userq_fence_driver *fence_drv = fence->fence_drv;
+ u64 rptr, wptr;
+
+ rptr = amdgpu_userq_fence_read(fence_drv);
+ wptr = fence->base.seqno;
+
+ if (rptr >= wptr)
+ return true;
+
+ return false;
+}
+
+static void amdgpu_userq_fence_free(struct rcu_head *rcu)
+{
+ struct dma_fence *fence = container_of(rcu, struct dma_fence, rcu);
+ struct amdgpu_userq_fence *userq_fence = to_amdgpu_userq_fence(fence);
+ struct amdgpu_userq_fence_driver *fence_drv = userq_fence->fence_drv;
+
+ /* Release the fence driver reference */
+ amdgpu_userq_fence_driver_put(fence_drv);
+ kmem_cache_free(amdgpu_userq_fence_slab, userq_fence);
+}
+
+static void amdgpu_userq_fence_release(struct dma_fence *f)
+{
+ call_rcu(&f->rcu, amdgpu_userq_fence_free);
+}
+
+static const struct dma_fence_ops amdgpu_userq_fence_ops = {
+ .use_64bit_seqno = true,
+ .get_driver_name = amdgpu_userq_fence_get_driver_name,
+ .get_timeline_name = amdgpu_userq_fence_get_timeline_name,
+ .signaled = amdgpu_userq_fence_signaled,
+ .release = amdgpu_userq_fence_release,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
new file mode 100644
index 000000000000..c3e04cdbb9e7
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
@@ -0,0 +1,69 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright 2023 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_USERQ_FENCE_H__
+#define __AMDGPU_USERQ_FENCE_H__
+
+#include <linux/types.h>
+
+#include "amdgpu_userqueue.h"
+
+struct amdgpu_userq_fence {
+ struct dma_fence base;
+ /*
+ * This lock is necessary to synchronize the
+ * userqueue dma fence operations.
+ */
+ spinlock_t lock;
+ struct list_head link;
+ struct amdgpu_userq_fence_driver *fence_drv;
+};
+
+struct amdgpu_userq_fence_driver {
+ struct kref refcount;
+ u64 gpu_addr;
+ u64 *cpu_addr;
+ u64 context;
+ /*
+ * This lock is necesaary to synchronize the access
+ * to the fences list by the fence driver.
+ */
+ spinlock_t fence_list_lock;
+ struct list_head fences;
+ struct amdgpu_device *adev;
+ char timeline_name[TASK_COMM_LEN];
+};
+
+int amdgpu_userq_fence_slab_init(void);
+void amdgpu_userq_fence_slab_fini(void);
+int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq,
+ u64 seq, struct dma_fence **f);
+void amdgpu_userq_fence_driver_get(struct amdgpu_userq_fence_driver *fence_drv);
+void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv);
+int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
+ struct amdgpu_usermode_queue *userq);
+void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
+void amdgpu_userq_fence_driver_destroy(struct kref *ref);
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
index 98cd07a42623..73d225aa3e1e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
@@ -26,6 +26,7 @@
#include "amdgpu.h"
#include "amdgpu_vm.h"
#include "amdgpu_userqueue.h"
+#include "amdgpu_userq_fence.h"
static void
amdgpu_userqueue_cleanup(struct amdgpu_userq_mgr *uq_mgr,
@@ -36,6 +37,7 @@ amdgpu_userqueue_cleanup(struct amdgpu_userq_mgr *uq_mgr,
const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type];
uq_funcs->mqd_destroy(uq_mgr, queue);
+ amdgpu_userq_fence_driver_put(queue->fence_drv);
idr_remove(&uq_mgr->userq_idr, queue_id);
kfree(queue);
}
@@ -55,7 +57,7 @@ amdgpu_userqueue_fence_get_driver_name(struct dma_fence *fence)
static const char *
amdgpu_userqueue_fence_get_timeline_name(struct dma_fence *f)
{
- struct amdgpu_userq_fence *ef = container_of(f, struct amdgpu_userq_fence, base);
+ struct amdgpu_userq_ev_fence *ef = container_of(f, struct amdgpu_userq_ev_fence, base);
return ef->timeline_name;
}
@@ -69,7 +71,7 @@ static const struct dma_fence_ops amdgpu_userqueue_eviction_fence_ops = {
static void
amdgpu_userqueue_init_eviction_fence(struct amdgpu_userq_mgr *uq_mgr)
{
- struct amdgpu_userq_fence *fence = &uq_mgr->eviction_fence;
+ struct amdgpu_userq_ev_fence *fence = &uq_mgr->eviction_fence;
atomic_t seq = ATOMIC_INIT(0);
spin_lock_init(&fence->lock);
@@ -403,6 +405,12 @@ amdgpu_userqueue_create(struct drm_file *filp, union drm_amdgpu_userq *args)
}
queue->doorbell_index = index;
+ r = amdgpu_userq_fence_driver_alloc(adev, queue);
+ if (r) {
+ DRM_ERROR("Failed to alloc fence driver\n");
+ goto unlock;
+ }
+
r = uq_funcs->mqd_create(uq_mgr, &args->in, queue);
if (r) {
DRM_ERROR("Failed to create Queue\n");
diff --git a/drivers/gpu/drm/amd/include/amdgpu_userqueue.h b/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
index 5ba18374bfbb..d3738f645adc 100644
--- a/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
+++ b/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
@@ -48,6 +48,7 @@ struct amdgpu_usermode_queue {
struct amdgpu_userq_obj db_obj;
struct amdgpu_userq_obj fw_obj;
struct amdgpu_userq_obj wptr_obj;
+ struct amdgpu_userq_fence_driver *fence_drv;
};
struct amdgpu_userq_funcs {
@@ -58,7 +59,7 @@ struct amdgpu_userq_funcs {
struct amdgpu_usermode_queue *uq);
};
-struct amdgpu_userq_fence {
+struct amdgpu_userq_ev_fence {
u64 fence_ctx;
atomic_t seq;
spinlock_t lock;
@@ -71,7 +72,7 @@ struct amdgpu_userq_mgr {
struct idr userq_idr;
struct mutex userq_mutex;
struct amdgpu_device *adev;
- struct amdgpu_userq_fence eviction_fence;
+ struct amdgpu_userq_ev_fence eviction_fence;
};
int amdgpu_userq_ioctl(struct drm_device *dev, void *data, struct drm_file *filp);
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v9 2/5] drm/amdgpu: Add mqd support for the fence address
2024-04-29 6:43 [PATCH v9 1/5] drm/amdgpu: Implement a new userqueue fence driver Arunpravin Paneer Selvam
@ 2024-04-29 6:43 ` Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 3/5] drm/amdgpu: UAPI headers for userqueue Secure semaphore Arunpravin Paneer Selvam
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Arunpravin Paneer Selvam @ 2024-04-29 6:43 UTC (permalink / raw)
To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam
- Add a field in struct v11_gfx_mqd for userqueue
fence address.
- Assign fence gpu VA address to the userqueue mqd
fence address fields.
v2: Remove the mask and replace with lower_32_bits (Christian)
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c | 11 +++++++++++
drivers/gpu/drm/amd/include/v11_structs.h | 4 ++--
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c
index 6369a26fb07e..d8592cfef534 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0_userqueue.c
@@ -25,6 +25,7 @@
#include "v11_structs.h"
#include "mes_v11_0.h"
#include "amdgpu_userqueue.h"
+#include "amdgpu_userq_fence.h"
#define AMDGPU_USERQ_PROC_CTX_SZ PAGE_SIZE
#define AMDGPU_USERQ_GANG_CTX_SZ PAGE_SIZE
@@ -204,6 +205,14 @@ static int mes_v11_0_userq_create_ctx_space(struct amdgpu_userq_mgr *uq_mgr,
return 0;
}
+static void mes_v11_0_userq_set_fence_space(struct amdgpu_usermode_queue *queue)
+{
+ struct v11_gfx_mqd *mqd = queue->mqd.cpu_ptr;
+
+ mqd->fenceaddress_lo = lower_32_bits(queue->fence_drv->gpu_addr);
+ mqd->fenceaddress_hi = upper_32_bits(queue->fence_drv->gpu_addr);
+}
+
static int mes_v11_0_userq_mqd_create(struct amdgpu_userq_mgr *uq_mgr,
struct drm_amdgpu_userq_in *args_in,
struct amdgpu_usermode_queue *queue)
@@ -274,6 +283,8 @@ static int mes_v11_0_userq_mqd_create(struct amdgpu_userq_mgr *uq_mgr,
goto free_mqd;
}
+ mes_v11_0_userq_set_fence_space(queue);
+
/* FW expects WPTR BOs to be mapped into GART */
r = mes_v11_0_create_wptr_mapping(uq_mgr, queue, userq_props->wptr_gpu_addr);
if (r) {
diff --git a/drivers/gpu/drm/amd/include/v11_structs.h b/drivers/gpu/drm/amd/include/v11_structs.h
index f8008270f813..797ce6a1e56e 100644
--- a/drivers/gpu/drm/amd/include/v11_structs.h
+++ b/drivers/gpu/drm/amd/include/v11_structs.h
@@ -535,8 +535,8 @@ struct v11_gfx_mqd {
uint32_t reserved_507; // offset: 507 (0x1FB)
uint32_t reserved_508; // offset: 508 (0x1FC)
uint32_t reserved_509; // offset: 509 (0x1FD)
- uint32_t reserved_510; // offset: 510 (0x1FE)
- uint32_t reserved_511; // offset: 511 (0x1FF)
+ uint32_t fenceaddress_lo; // offset: 510 (0x1FE)
+ uint32_t fenceaddress_hi; // offset: 511 (0x1FF)
};
struct v11_sdma_mqd {
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v9 3/5] drm/amdgpu: UAPI headers for userqueue Secure semaphore
2024-04-29 6:43 [PATCH v9 1/5] drm/amdgpu: Implement a new userqueue fence driver Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 2/5] drm/amdgpu: Add mqd support for the fence address Arunpravin Paneer Selvam
@ 2024-04-29 6:43 ` Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 4/5] drm/amdgpu: Implement userqueue signal/wait IOCTL Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 5/5] drm/amdgpu: Enable userq fence interrupt support Arunpravin Paneer Selvam
3 siblings, 0 replies; 7+ messages in thread
From: Arunpravin Paneer Selvam @ 2024-04-29 6:43 UTC (permalink / raw)
To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam
Add UAPI header support for userqueue Secure semaphore
v2: Worked on review comments from Christian for the following
modifications
- Add bo handles, bo flags and padding fields.
- Include value/va in a combined array.
v3: Worked on review comments from Christian
- Add num_fences field to obtain the number of objects required
to allocate memory for userq_fence_info.
- Replace obj_handle name with syncobj_handle.
- Replace point name with syncobj_point.
- Replace count_handles name with num_syncobj_handles.
- Fix structure padding related issues.
v4: Worked on review comments from Christian
- Modify the bo flags description.
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
---
include/uapi/drm/amdgpu_drm.h | 115 ++++++++++++++++++++++++++++++++++
1 file changed, 115 insertions(+)
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index def6874a588c..1217ae30a8d8 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -55,6 +55,8 @@ extern "C" {
#define DRM_AMDGPU_FENCE_TO_HANDLE 0x14
#define DRM_AMDGPU_SCHED 0x15
#define DRM_AMDGPU_USERQ 0x16
+#define DRM_AMDGPU_USERQ_SIGNAL 0x17
+#define DRM_AMDGPU_USERQ_WAIT 0x18
#define DRM_IOCTL_AMDGPU_GEM_CREATE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create)
#define DRM_IOCTL_AMDGPU_GEM_MMAP DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap)
@@ -73,6 +75,8 @@ extern "C" {
#define DRM_IOCTL_AMDGPU_FENCE_TO_HANDLE DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_FENCE_TO_HANDLE, union drm_amdgpu_fence_to_handle)
#define DRM_IOCTL_AMDGPU_SCHED DRM_IOW(DRM_COMMAND_BASE + DRM_AMDGPU_SCHED, union drm_amdgpu_sched)
#define DRM_IOCTL_AMDGPU_USERQ DRM_IOW(DRM_COMMAND_BASE + DRM_AMDGPU_USERQ, union drm_amdgpu_userq)
+#define DRM_IOCTL_AMDGPU_USERQ_SIGNAL DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_USERQ_SIGNAL, struct drm_amdgpu_userq_signal)
+#define DRM_IOCTL_AMDGPU_USERQ_WAIT DRM_IOWR(DRM_COMMAND_BASE + DRM_AMDGPU_USERQ_WAIT, struct drm_amdgpu_userq_wait)
/**
* DOC: memory domains
@@ -431,6 +435,117 @@ union drm_amdgpu_userq {
struct drm_amdgpu_userq_out out;
};
+/* dma_resv usage flag */
+#define AMDGPU_USERQ_BO_WRITE 1
+
+/* userq signal/wait ioctl */
+struct drm_amdgpu_userq_signal {
+ /**
+ * @queue_id: Queue handle used by the userq fence creation function
+ * to retrieve the WPTR.
+ */
+ __u32 queue_id;
+ /**
+ * @flags: flags to indicate special function for userq fence creation.
+ * Unused for now.
+ */
+ __u32 flags;
+ /**
+ * @syncobj_handles_array: An array of syncobj handles used by the userq fence
+ * creation IOCTL to install the created dma_fence object which can be
+ * utilized by userspace to explicitly synchronize GPU commands.
+ */
+ __u64 syncobj_handles_array;
+ /**
+ * @num_syncobj_handles: A count that represents the number of syncobj handles in
+ * @syncobj_handles_array.
+ */
+ __u64 num_syncobj_handles;
+ /**
+ * @syncobj_point: A given point on the timeline to be signaled.
+ * Unused for now.
+ */
+ __u64 syncobj_point;
+ /**
+ * @bo_handles_array: An array of GEM BO handles used by the userq fence creation
+ * IOCTL to install the created dma_fence object which can be utilized by
+ * userspace to synchronize the BO usage between user processes.
+ */
+ __u64 bo_handles_array;
+ /**
+ * @num_bo_handles: A count that represents the number of GEM BO handles in
+ * @bo_handles_array.
+ */
+ __u32 num_bo_handles;
+ /**
+ * @bo_flags: flags to indicate BOs synchronize for READ or WRITE
+ */
+ __u32 bo_flags;
+};
+
+struct drm_amdgpu_userq_fence_info {
+ /**
+ * @va: A gpu address allocated for each queue which stores the
+ * read pointer (RPTR) value.
+ */
+ __u64 va;
+ /**
+ * @value: A 64 bit value represents the write pointer (WPTR) of the
+ * queue commands which compared with the RPTR value to signal the
+ * fences.
+ */
+ __u64 value;
+};
+
+struct drm_amdgpu_userq_wait {
+ /**
+ * @waitq_id: Queue handle used to retrieve the queue information to store
+ * the fence driver references in the wait user queue structure.
+ */
+ __u32 waitq_id;
+ /**
+ * @flags: flags to specify special function for userq wait information.
+ * Unused for now.
+ */
+ __u32 flags;
+ /**
+ * @bo_wait_flags: flags to define the BOs for READ or WRITE to store the
+ * matching fence wait info pair in @userq_fence_info.
+ */
+ __u32 bo_wait_flags;
+ __u32 pad;
+ /**
+ * @syncobj_handles_array: An array of syncobj handles defined to get the
+ * fence wait information of every syncobj handles in the array.
+ */
+ __u64 syncobj_handles_array;
+ /**
+ * @bo_handles_array: An array of GEM BO handles defined to fetch the fence
+ * wait information of every BO handles in the array.
+ */
+ __u64 bo_handles_array;
+ /**
+ * @num_syncobj_handles: A count that represents the number of syncobj handles in
+ * @syncobj_handles_array.
+ */
+ __u32 num_syncobj_handles;
+ /**
+ * @num_bo_handles: A count that represents the number of GEM BO handles in
+ * @bo_handles_array.
+ */
+ __u32 num_bo_handles;
+ /**
+ * @userq_fence_info: An array of fence information (va and value) pair of each
+ * objects stored in @syncobj_handles_array and @bo_handles_array.
+ */
+ __u64 userq_fence_info;
+ /**
+ * @num_fences: A count that represents the number of actual fences installed in
+ * each syncobj and bo handles.
+ */
+ __u64 num_fences;
+};
+
/* vm ioctl */
#define AMDGPU_VM_OP_RESERVE_VMID 1
#define AMDGPU_VM_OP_UNRESERVE_VMID 2
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v9 4/5] drm/amdgpu: Implement userqueue signal/wait IOCTL
2024-04-29 6:43 [PATCH v9 1/5] drm/amdgpu: Implement a new userqueue fence driver Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 2/5] drm/amdgpu: Add mqd support for the fence address Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 3/5] drm/amdgpu: UAPI headers for userqueue Secure semaphore Arunpravin Paneer Selvam
@ 2024-04-29 6:43 ` Arunpravin Paneer Selvam
2024-04-30 8:40 ` Christian König
2024-04-29 6:43 ` [PATCH v9 5/5] drm/amdgpu: Enable userq fence interrupt support Arunpravin Paneer Selvam
3 siblings, 1 reply; 7+ messages in thread
From: Arunpravin Paneer Selvam @ 2024-04-29 6:43 UTC (permalink / raw)
To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam
This patch introduces new IOCTL for userqueue secure semaphore.
The signal IOCTL called from userspace application creates a drm
syncobj and array of bo GEM handles and passed in as parameter to
the driver to install the fence into it.
The wait IOCTL gets an array of drm syncobjs, finds the fences
attached to the drm syncobjs and obtain the array of
memory_address/fence_value combintion which are returned to
userspace.
v2: (Christian)
- Install fence into GEM BO object.
- Lock all BO's using the dma resv subsystem
- Reorder the sequence in signal IOCTL function.
- Get write pointer from the shadow wptr
- use userq_fence to fetch the va/value in wait IOCTL.
v3: (Christian)
- Use drm_exec helper for the proper BO drm reserve and avoid BO
lock/unlock issues.
- fence/fence driver reference count logic for signal/wait IOCTLs.
v4: (Christian)
- Fixed the drm_exec calling sequence
- use dma_resv_for_each_fence_unlock if BO's are not locked
- Modified the fence_info array storing logic.
v5: (Christian)
- Keep fence_drv until wait queue execution.
- Add dma_fence_wait for other fences.
- Lock BO's using drm_exec as the number of fences in them could
change.
- Install signaled fences as well into BO/Syncobj.
- Move Syncobj fence installation code after the drm_exec_prepare_array.
- Directly add dma_resv_usage_rw(args->bo_flags....
- remove unnecessary dma_fence_put.
v6: (Christian)
- Add xarray stuff to store the fence_drv
- Implement a function to iterate over the xarray and drop
the fence_drv references.
- Add drm_exec_until_all_locked() wrapper
- Add a check that if we haven't exceeded the user allocated num_fences
before adding dma_fence to the fences array.
v7: (Christian)
- Use memdup_user() for kmalloc_array + copy_from_user
- Move the fence_drv references from the xarray into the newly created fence
and drop the fence_drv references when we signal this fence.
- Move this locking of BOs before the "if (!wait_info->num_fences)",
this way you need this code block only once.
- Merge the error handling code and the cleanup + return 0 code.
- Initializing the xa should probably be done in the userq code.
- Remove the userq back pointer stored in fence_drv.
- Pass xarray as parameter in amdgpu_userq_walk_and_drop_fence_drv()
v8: (Christian)
- Move fence_drv references must come before adding the fence to the list.
- Use xa_lock_irqsave_nested for nested spinlock operations.
- userq_mgr should be per fpriv and not one per device.
- Restructure the interrupt process code for the early exit of the loop.
- The reference acquired in the syncobj fence replace code needs to be
kept around.
- Modify the dma_fence acquire placement in wait IOCTL.
- Move USERQ_BO_WRITE flag to UAPI header file.
- drop the fence drv reference after telling the hw to stop accessing it.
- Add multi sync object support to userq signal IOCTL.
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Suggested-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 +
.../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 454 +++++++++++++++++-
.../gpu/drm/amd/amdgpu/amdgpu_userq_fence.h | 5 +
drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 29 +-
.../gpu/drm/amd/include/amdgpu_userqueue.h | 1 +
5 files changed, 484 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 844f7b5f90db..5892a4c1a92e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -2918,6 +2918,8 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = {
DRM_IOCTL_DEF_DRV(AMDGPU_GEM_OP, amdgpu_gem_op_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(AMDGPU_GEM_USERPTR, amdgpu_gem_userptr_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(AMDGPU_USERQ, amdgpu_userq_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_SIGNAL, amdgpu_userq_signal_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
+ DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_WAIT, amdgpu_userq_wait_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
};
static const struct drm_driver amdgpu_kms_driver = {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
index f7baea2c67ab..6fb75cc1d20c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
@@ -25,6 +25,7 @@
#include <linux/kref.h>
#include <linux/slab.h>
+#include <drm/drm_exec.h>
#include <drm/drm_syncobj.h>
#include "amdgpu.h"
@@ -102,8 +103,11 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv)
{
+ struct amdgpu_userq_fence_driver *stored_fence_drv;
struct amdgpu_userq_fence *userq_fence, *tmp;
+ unsigned long index, flags;
struct dma_fence *fence;
+ struct xarray *xa;
u64 rptr;
if (!fence_drv)
@@ -114,15 +118,25 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d
spin_lock(&fence_drv->fence_list_lock);
list_for_each_entry_safe(userq_fence, tmp, &fence_drv->fences, link) {
fence = &userq_fence->base;
+ xa = &userq_fence->fence_drv_xa;
- if (rptr >= fence->seqno) {
- dma_fence_signal(fence);
- list_del(&userq_fence->link);
-
- dma_fence_put(fence);
- } else {
+ if (rptr < fence->seqno)
break;
+
+ dma_fence_signal(fence);
+ /*
+ * Walk over the fence_drv xarray and drop the old wait ioctl
+ * fence_drv references.
+ */
+ xa_lock_irqsave_nested(xa, flags, SINGLE_DEPTH_NESTING);
+ xa_for_each(xa, index, stored_fence_drv) {
+ __xa_erase(xa, index);
+ amdgpu_userq_fence_driver_put(stored_fence_drv);
}
+ xa_unlock_irqrestore(xa, flags);
+
+ list_del(&userq_fence->link);
+ dma_fence_put(fence);
}
spin_unlock(&fence_drv->fence_list_lock);
}
@@ -132,8 +146,11 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
struct amdgpu_userq_fence_driver *fence_drv = container_of(ref,
struct amdgpu_userq_fence_driver,
refcount);
+ struct amdgpu_userq_fence_driver *xa_fence_drv;
struct amdgpu_device *adev = fence_drv->adev;
struct amdgpu_userq_fence *fence, *tmp;
+ struct xarray *xa = &adev->userq_xa;
+ unsigned long index;
struct dma_fence *f;
spin_lock(&fence_drv->fence_list_lock);
@@ -150,6 +167,12 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
}
spin_unlock(&fence_drv->fence_list_lock);
+ xa_lock(xa);
+ xa_for_each(xa, index, xa_fence_drv)
+ if (xa_fence_drv == fence_drv)
+ __xa_erase(xa, index);
+ xa_unlock(xa);
+
/* Free seq64 memory */
amdgpu_seq64_free(adev, fence_drv->gpu_addr);
kfree(fence_drv);
@@ -188,9 +211,37 @@ int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq,
dma_fence_init(fence, &amdgpu_userq_fence_ops, &userq_fence->lock,
fence_drv->context, seq);
+ xa_init_flags(&userq_fence->fence_drv_xa, XA_FLAGS_LOCK_IRQ);
+
amdgpu_userq_fence_driver_get(fence_drv);
dma_fence_get(fence);
+ if (!xa_empty(&userq->uq_fence_drv_xa)) {
+ struct amdgpu_userq_fence_driver *stored_fence_drv;
+ unsigned long flags, index;
+
+ /*
+ * Move fence_drv references of old signal IOCTL calls to the
+ * newly created userq fence xarray.
+ */
+ xa_lock(&userq->uq_fence_drv_xa);
+ xa_for_each(&userq->uq_fence_drv_xa, index, stored_fence_drv) {
+ /* Skip if the queues are different */
+ if (stored_fence_drv != fence_drv)
+ continue;
+
+ xa_lock_irqsave_nested(&userq_fence->fence_drv_xa, flags,
+ SINGLE_DEPTH_NESTING);
+ __xa_store(&userq_fence->fence_drv_xa, index,
+ stored_fence_drv, GFP_KERNEL);
+ xa_unlock_irqrestore(&userq_fence->fence_drv_xa, flags);
+
+ /* Erase fence_drv reference entry from userq xarray */
+ __xa_erase(&userq->uq_fence_drv_xa, index);
+ }
+ xa_unlock(&userq->uq_fence_drv_xa);
+ }
+
spin_lock(&fence_drv->fence_list_lock);
/* Check if hardware has already processed the job */
if (!dma_fence_is_signaled(fence))
@@ -240,6 +291,8 @@ static void amdgpu_userq_fence_free(struct rcu_head *rcu)
/* Release the fence driver reference */
amdgpu_userq_fence_driver_put(fence_drv);
+
+ xa_destroy(&userq_fence->fence_drv_xa);
kmem_cache_free(amdgpu_userq_fence_slab, userq_fence);
}
@@ -255,3 +308,392 @@ static const struct dma_fence_ops amdgpu_userq_fence_ops = {
.signaled = amdgpu_userq_fence_signaled,
.release = amdgpu_userq_fence_release,
};
+
+/**
+ * amdgpu_userq_fence_read_wptr - Read the userq wptr value
+ *
+ * @filp: drm file private data structure
+ * @queue: user mode queue structure pointer
+ * @wptr: write pointer value
+ *
+ * Read the wptr value from userq's MQD. The userq signal IOCTL
+ * creates a dma_fence for the shared buffers that expects the
+ * RPTR value written to seq64 memory >= WPTR.
+ *
+ * Returns wptr value on success, error on failure.
+ */
+static int amdgpu_userq_fence_read_wptr(struct drm_file *filp,
+ struct amdgpu_usermode_queue *queue,
+ u64 *wptr)
+{
+ struct amdgpu_fpriv *fpriv = filp->driver_priv;
+ struct amdgpu_bo_va_mapping *mapping;
+ struct amdgpu_vm *vm = &fpriv->vm;
+ struct amdgpu_bo *bo;
+ u64 addr, *ptr;
+ int r;
+
+ addr = queue->userq_prop->wptr_gpu_addr;
+ addr &= AMDGPU_GMC_HOLE_MASK;
+
+ mapping = amdgpu_vm_bo_lookup_mapping(vm, addr >> PAGE_SHIFT);
+ if (!mapping)
+ return -EINVAL;
+
+ bo = mapping->bo_va->base.bo;
+ r = amdgpu_bo_reserve(bo, true);
+ if (r) {
+ DRM_ERROR("Failed to reserve userqueue wptr bo");
+ return r;
+ }
+
+ r = amdgpu_bo_kmap(bo, (void **)&ptr);
+ if (r) {
+ DRM_ERROR("Failed mapping the userqueue wptr bo");
+ goto map_error;
+ }
+
+ *wptr = le64_to_cpu(*ptr);
+
+ amdgpu_bo_kunmap(bo);
+ amdgpu_bo_unreserve(bo);
+
+ return 0;
+
+map_error:
+ amdgpu_bo_unreserve(bo);
+ return r;
+}
+
+int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp)
+{
+ struct amdgpu_fpriv *fpriv = filp->driver_priv;
+ struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr;
+ struct drm_amdgpu_userq_signal *args = data;
+ struct amdgpu_usermode_queue *queue;
+ struct drm_gem_object **gobj = NULL;
+ struct drm_syncobj **syncobj = NULL;
+ u32 *syncobj_handles, num_syncobj_handles;
+ u32 *bo_handles, num_bo_handles;
+ struct dma_fence *fence;
+ struct drm_exec exec;
+ int r, i, entry;
+ u64 wptr;
+
+ /* Array of syncobj handles */
+ num_syncobj_handles = args->num_syncobj_handles;
+ syncobj_handles = memdup_user(u64_to_user_ptr(args->syncobj_handles_array),
+ sizeof(u32) * num_syncobj_handles);
+ if (IS_ERR(syncobj_handles))
+ return PTR_ERR(syncobj_handles);
+
+ /* Array of syncobj object handles */
+ syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), GFP_KERNEL);
+ if (!syncobj) {
+ r = -ENOMEM;
+ goto free_syncobj_handles;
+ }
+
+ for (entry = 0; entry < num_syncobj_handles; entry++) {
+ syncobj[entry] = drm_syncobj_find(filp, syncobj_handles[entry]);
+ if (!syncobj[entry]) {
+ r = -ENOENT;
+ goto free_syncobj_handles;
+ }
+ }
+
+ /* Array of bo handles */
+ num_bo_handles = args->num_bo_handles;
+ bo_handles = memdup_user(u64_to_user_ptr(args->bo_handles_array),
+ sizeof(u32) * num_bo_handles);
+ if (IS_ERR(bo_handles))
+ goto free_syncobj_handles;
+
+ /* Array of GEM object handles */
+ gobj = kmalloc_array(num_bo_handles, sizeof(*gobj), GFP_KERNEL);
+ if (!gobj) {
+ r = -ENOMEM;
+ goto free_bo_handles;
+ }
+
+ for (entry = 0; entry < num_bo_handles; entry++) {
+ gobj[entry] = drm_gem_object_lookup(filp, bo_handles[entry]);
+ if (!gobj[entry]) {
+ r = -ENOENT;
+ goto put_gobj;
+ }
+ }
+
+ drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
+ drm_exec_until_all_locked(&exec) {
+ r = drm_exec_prepare_array(&exec, gobj, num_bo_handles, 1);
+ drm_exec_retry_on_contention(&exec);
+ if (r)
+ goto exec_fini;
+ }
+
+ /*Retrieve the user queue */
+ queue = idr_find(&userq_mgr->userq_idr, args->queue_id);
+ if (!queue) {
+ r = -ENOENT;
+ goto exec_fini;
+ }
+
+ r = amdgpu_userq_fence_read_wptr(filp, queue, &wptr);
+ if (r)
+ goto exec_fini;
+
+ /* Create a new fence */
+ r = amdgpu_userq_fence_create(queue, wptr, &fence);
+ if (r)
+ goto exec_fini;
+
+ for (i = 0; i < num_bo_handles; i++)
+ dma_resv_add_fence(gobj[i]->resv, fence,
+ dma_resv_usage_rw(args->bo_flags &
+ AMDGPU_USERQ_BO_WRITE));
+
+ /* Add the created fence to syncobj/BO's */
+ for (i = 0; i < num_syncobj_handles; i++)
+ drm_syncobj_replace_fence(syncobj[i], fence);
+
+ /* drop the reference acquired in fence creation function */
+ dma_fence_put(fence);
+
+exec_fini:
+ drm_exec_fini(&exec);
+put_gobj:
+ while (entry-- > 0)
+ drm_gem_object_put(gobj[entry]);
+ kfree(gobj);
+free_bo_handles:
+ kfree(bo_handles);
+free_syncobj_handles:
+ while (i-- > 0)
+ drm_syncobj_put(syncobj[i]);
+ kfree(syncobj_handles);
+
+ return r;
+}
+
+int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp)
+{
+ struct amdgpu_fpriv *fpriv = filp->driver_priv;
+ struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr;
+ struct drm_amdgpu_userq_fence_info *fence_info = NULL;
+ struct drm_amdgpu_userq_wait *wait_info = data;
+ struct amdgpu_usermode_queue *waitq;
+ u32 *syncobj_handles, *bo_handles;
+ struct dma_fence **fences = NULL;
+ u32 num_syncobj, num_bo_handles;
+ struct drm_gem_object **gobj;
+ struct drm_exec exec;
+ int r, i, entry, cnt;
+ u64 num_fences = 0;
+
+ num_bo_handles = wait_info->num_bo_handles;
+ bo_handles = memdup_user(u64_to_user_ptr(wait_info->bo_handles_array),
+ sizeof(u32) * num_bo_handles);
+ if (IS_ERR(bo_handles))
+ return PTR_ERR(bo_handles);
+
+ num_syncobj = wait_info->num_syncobj_handles;
+ syncobj_handles = memdup_user(u64_to_user_ptr(wait_info->syncobj_handles_array),
+ sizeof(u32) * num_syncobj);
+ if (IS_ERR(syncobj_handles)) {
+ r = PTR_ERR(syncobj_handles);
+ goto free_bo_handles;
+ }
+
+ /* Array of GEM object handles */
+ gobj = kmalloc_array(num_bo_handles, sizeof(*gobj), GFP_KERNEL);
+ if (!gobj) {
+ r = -ENOMEM;
+ goto free_syncobj_handles;
+ }
+
+ for (entry = 0; entry < num_bo_handles; entry++) {
+ gobj[entry] = drm_gem_object_lookup(filp, bo_handles[entry]);
+ if (!gobj[entry]) {
+ r = -ENOENT;
+ goto put_gobj;
+ }
+ }
+
+ drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
+ drm_exec_until_all_locked(&exec) {
+ r = drm_exec_prepare_array(&exec, gobj, num_bo_handles, 0);
+ drm_exec_retry_on_contention(&exec);
+ if (r) {
+ drm_exec_fini(&exec);
+ goto put_gobj;
+ }
+ }
+
+ if (!wait_info->num_fences) {
+ /* Count syncobj's fence */
+ for (i = 0; i < num_syncobj; i++) {
+ struct dma_fence *fence;
+
+ r = drm_syncobj_find_fence(filp, syncobj_handles[i],
+ 0, 0, &fence);
+ dma_fence_put(fence);
+
+ if (r || !fence)
+ continue;
+
+ num_fences++;
+ }
+
+ /* Count GEM objects fence */
+ for (i = 0; i < num_bo_handles; i++) {
+ struct dma_resv_iter resv_cursor;
+ struct dma_fence *fence;
+
+ dma_resv_for_each_fence(&resv_cursor, gobj[i]->resv,
+ dma_resv_usage_rw(wait_info->bo_wait_flags &
+ AMDGPU_USERQ_BO_WRITE), fence)
+ num_fences++;
+ }
+
+ /*
+ * Passing num_fences = 0 means that userspace doesn't want to
+ * retrieve userq_fence_info. If num_fences = 0 we skip filling
+ * userq_fence_info and return the actual number of fences on
+ * args->num_fences.
+ */
+ wait_info->num_fences = num_fences;
+ } else {
+ /* Array of fence info */
+ fence_info = kmalloc_array(wait_info->num_fences, sizeof(*fence_info), GFP_KERNEL);
+ if (!fence_info) {
+ r = -ENOMEM;
+ goto exec_fini;
+ }
+
+ /* Array of fences */
+ fences = kmalloc_array(wait_info->num_fences, sizeof(*fences), GFP_KERNEL);
+ if (!fences) {
+ r = -ENOMEM;
+ goto free_fence_info;
+ }
+
+ /* Retrieve GEM objects fence */
+ for (i = 0; i < num_bo_handles; i++) {
+ struct dma_resv_iter resv_cursor;
+ struct dma_fence *fence;
+
+ dma_resv_for_each_fence(&resv_cursor, gobj[i]->resv,
+ dma_resv_usage_rw(wait_info->bo_wait_flags &
+ AMDGPU_USERQ_BO_WRITE), fence) {
+ if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) {
+ r = -EINVAL;
+ goto free_fences;
+ }
+
+ fences[num_fences++] = fence;
+ dma_fence_get(fence);
+ }
+ }
+
+ /* Retrieve syncobj's fence */
+ for (i = 0; i < num_syncobj; i++) {
+ struct dma_fence *fence;
+
+ r = drm_syncobj_find_fence(filp, syncobj_handles[i],
+ 0, 0, &fence);
+ if (r || !fence)
+ continue;
+
+ if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) {
+ r = -EINVAL;
+ goto free_fences;
+ }
+
+ fences[num_fences++] = fence;
+ }
+
+ waitq = idr_find(&userq_mgr->userq_idr, wait_info->waitq_id);
+ if (!waitq)
+ goto free_fences;
+
+ for (i = 0, cnt = 0; i < wait_info->num_fences; i++) {
+ struct amdgpu_userq_fence_driver *fence_drv;
+ struct amdgpu_userq_fence *userq_fence;
+ u32 index;
+
+ userq_fence = to_amdgpu_userq_fence(fences[i]);
+ if (!userq_fence) {
+ /*
+ * Just waiting on other driver fences should
+ * be good for now
+ */
+ dma_fence_wait(fences[i], false);
+ dma_fence_put(fences[i]);
+
+ continue;
+ }
+
+ fence_drv = userq_fence->fence_drv;
+ /*
+ * We need to make sure the user queue release their reference
+ * to the fence drivers at some point before queue destruction.
+ * Otherwise, we would gather those references until we don't
+ * have any more space left and crash.
+ */
+ r = xa_alloc(&waitq->uq_fence_drv_xa, &index,
+ fence_drv, xa_limit_32b, GFP_KERNEL);
+ if (r)
+ goto free_fences;
+
+ amdgpu_userq_fence_driver_get(fence_drv);
+
+ /* Store drm syncobj's gpu va address and value */
+ fence_info[cnt].va = fence_drv->gpu_addr;
+ fence_info[cnt].value = fences[i]->seqno;
+
+ dma_fence_put(fences[i]);
+ /* Increment the actual userq fence count */
+ cnt++;
+ }
+
+ wait_info->num_fences = cnt;
+ /* Copy userq fence info to user space */
+ if (copy_to_user(u64_to_user_ptr(wait_info->userq_fence_info),
+ fence_info, wait_info->num_fences * sizeof(*fence_info))) {
+ r = -EFAULT;
+ goto free_fences;
+ }
+
+ kfree(fences);
+ kfree(fence_info);
+ }
+
+ drm_exec_fini(&exec);
+ for (i = 0; i < num_bo_handles; i++)
+ drm_gem_object_put(gobj[i]);
+
+ kfree(syncobj_handles);
+ kfree(bo_handles);
+
+ return 0;
+
+free_fences:
+ kfree(fences);
+free_fence_info:
+ kfree(fence_info);
+exec_fini:
+ drm_exec_fini(&exec);
+put_gobj:
+ while (entry-- > 0)
+ drm_gem_object_put(gobj[entry]);
+ kfree(gobj);
+free_syncobj_handles:
+ kfree(syncobj_handles);
+free_bo_handles:
+ kfree(bo_handles);
+
+ return r;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
index c3e04cdbb9e7..1b75900fc782 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
@@ -37,6 +37,7 @@ struct amdgpu_userq_fence {
*/
spinlock_t lock;
struct list_head link;
+ struct xarray fence_drv_xa;
struct amdgpu_userq_fence_driver *fence_drv;
};
@@ -65,5 +66,9 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
struct amdgpu_usermode_queue *userq);
void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
void amdgpu_userq_fence_driver_destroy(struct kref *ref);
+int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp);
+int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data,
+ struct drm_file *filp);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
index 73d225aa3e1e..404c39073661 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
@@ -28,6 +28,32 @@
#include "amdgpu_userqueue.h"
#include "amdgpu_userq_fence.h"
+static void amdgpu_userq_walk_and_drop_fence_drv(struct xarray *xa)
+{
+ struct amdgpu_userq_fence_driver *fence_drv;
+ unsigned long index;
+
+ if (xa_empty(xa))
+ return;
+
+ xa_lock(xa);
+ xa_for_each(xa, index, fence_drv) {
+ __xa_erase(xa, index);
+ amdgpu_userq_fence_driver_put(fence_drv);
+ }
+
+ xa_unlock(xa);
+}
+
+static void
+amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq)
+{
+ amdgpu_userq_walk_and_drop_fence_drv(&userq->uq_fence_drv_xa);
+ xa_destroy(&userq->uq_fence_drv_xa);
+ /* Drop the fence_drv reference held by user queue */
+ amdgpu_userq_fence_driver_put(userq->fence_drv);
+}
+
static void
amdgpu_userqueue_cleanup(struct amdgpu_userq_mgr *uq_mgr,
struct amdgpu_usermode_queue *queue,
@@ -37,7 +63,7 @@ amdgpu_userqueue_cleanup(struct amdgpu_userq_mgr *uq_mgr,
const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type];
uq_funcs->mqd_destroy(uq_mgr, queue);
- amdgpu_userq_fence_driver_put(queue->fence_drv);
+ amdgpu_userq_fence_driver_free(queue);
idr_remove(&uq_mgr->userq_idr, queue_id);
kfree(queue);
}
@@ -405,6 +431,7 @@ amdgpu_userqueue_create(struct drm_file *filp, union drm_amdgpu_userq *args)
}
queue->doorbell_index = index;
+ xa_init_flags(&queue->uq_fence_drv_xa, XA_FLAGS_ALLOC);
r = amdgpu_userq_fence_driver_alloc(adev, queue);
if (r) {
DRM_ERROR("Failed to alloc fence driver\n");
diff --git a/drivers/gpu/drm/amd/include/amdgpu_userqueue.h b/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
index d3738f645adc..27e1617b234f 100644
--- a/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
+++ b/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
@@ -48,6 +48,7 @@ struct amdgpu_usermode_queue {
struct amdgpu_userq_obj db_obj;
struct amdgpu_userq_obj fw_obj;
struct amdgpu_userq_obj wptr_obj;
+ struct xarray uq_fence_drv_xa;
struct amdgpu_userq_fence_driver *fence_drv;
};
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v9 5/5] drm/amdgpu: Enable userq fence interrupt support
2024-04-29 6:43 [PATCH v9 1/5] drm/amdgpu: Implement a new userqueue fence driver Arunpravin Paneer Selvam
` (2 preceding siblings ...)
2024-04-29 6:43 ` [PATCH v9 4/5] drm/amdgpu: Implement userqueue signal/wait IOCTL Arunpravin Paneer Selvam
@ 2024-04-29 6:43 ` Arunpravin Paneer Selvam
2024-04-30 8:52 ` Christian König
3 siblings, 1 reply; 7+ messages in thread
From: Arunpravin Paneer Selvam @ 2024-04-29 6:43 UTC (permalink / raw)
To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam
Add support to handle the userqueue protected fence signal hardware
interrupt.
Create a xarray which maps the doorbell index to the fence driver address.
This would help to retrieve the fence driver information when an userq fence
interrupt is triggered. Firmware sends the doorbell offset value and
this info is compared with the queue's mqd doorbell offset value.
If they are same, we process the userq fence interrupt.
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 5 ++--
.../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 6 +++++
drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 25 ++++++++++---------
drivers/gpu/drm/amd/amdgpu/mes_v10_1.c | 5 ----
drivers/gpu/drm/amd/amdgpu/mes_v11_0.c | 7 ------
6 files changed, 23 insertions(+), 27 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 4ca14b02668b..2d5ef2e74c71 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1043,6 +1043,8 @@ struct amdgpu_device {
struct amdgpu_mqd mqds[AMDGPU_HW_IP_NUM];
const struct amdgpu_userq_funcs *userq_funcs[AMDGPU_HW_IP_NUM];
+ struct xarray userq_xa;
+
/* df */
struct amdgpu_df df;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 7753a2e64d41..fd919105a181 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3982,6 +3982,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
spin_lock_init(&adev->audio_endpt_idx_lock);
spin_lock_init(&adev->mm_stats.lock);
+ xa_init_flags(&adev->userq_xa, XA_FLAGS_LOCK_IRQ);
+
INIT_LIST_HEAD(&adev->shadow_list);
mutex_init(&adev->shadow_list_lock);
@@ -4719,9 +4721,6 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
}
adev->in_suspend = false;
- if (adev->enable_mes)
- amdgpu_mes_self_test(adev);
-
if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DEV_D0))
DRM_WARN("smart shift update failed\n");
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
index 6fb75cc1d20c..614953b0fc19 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
@@ -70,6 +70,7 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
struct amdgpu_usermode_queue *userq)
{
struct amdgpu_userq_fence_driver *fence_drv;
+ unsigned long flags;
int r;
fence_drv = kzalloc(sizeof(*fence_drv), GFP_KERNEL);
@@ -96,6 +97,11 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
fence_drv->context = dma_fence_context_alloc(1);
get_task_comm(fence_drv->timeline_name, current);
+ xa_lock_irqsave(&adev->userq_xa, flags);
+ __xa_store(&adev->userq_xa, userq->doorbell_index,
+ fence_drv, GFP_KERNEL);
+ xa_unlock_irqrestore(&adev->userq_xa, flags);
+
userq->fence_drv = fence_drv;
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index a786e25432ae..d6cdca0a652f 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -49,6 +49,7 @@
#include "gfx_v11_0_3.h"
#include "nbio_v4_3.h"
#include "mes_v11_0.h"
+#include "amdgpu_userq_fence.h"
#define GFX11_NUM_GFX_RINGS 1
#define GFX11_MEC_HPD_SIZE 2048
@@ -5939,25 +5940,25 @@ static int gfx_v11_0_eop_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry)
{
- int i;
+ u32 doorbell_offset = entry->src_data[0];
u8 me_id, pipe_id, queue_id;
struct amdgpu_ring *ring;
- uint32_t mes_queue_id = entry->src_data[0];
+ int i;
DRM_DEBUG("IH: CP EOP\n");
- if (adev->enable_mes && (mes_queue_id & AMDGPU_FENCE_MES_QUEUE_FLAG)) {
- struct amdgpu_mes_queue *queue;
+ if (adev->enable_mes && doorbell_offset) {
+ struct amdgpu_userq_fence_driver *fence_drv = NULL;
+ struct xarray *xa = &adev->userq_xa;
+ unsigned long index, flags;
- mes_queue_id &= AMDGPU_FENCE_MES_QUEUE_ID_MASK;
+ xa_lock_irqsave(xa, flags);
+ xa_for_each(xa, index, fence_drv)
+ if (doorbell_offset == index)
+ break;
+ xa_unlock_irqrestore(xa, flags);
- spin_lock(&adev->mes.queue_id_lock);
- queue = idr_find(&adev->mes.queue_id_idr, mes_queue_id);
- if (queue) {
- DRM_DEBUG("process mes queue id = %d\n", mes_queue_id);
- amdgpu_fence_process(queue->ring);
- }
- spin_unlock(&adev->mes.queue_id_lock);
+ amdgpu_userq_fence_driver_process(fence_drv);
} else {
me_id = (entry->ring_id & 0x0c) >> 2;
pipe_id = (entry->ring_id & 0x03) >> 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
index 4d1121d1a1e7..faa489c75fea 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
@@ -1163,11 +1163,6 @@ static int mes_v10_0_early_init(void *handle)
static int mes_v10_0_late_init(void *handle)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- if (!amdgpu_in_reset(adev))
- amdgpu_mes_self_test(adev);
-
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index feb7fa2c304c..0051e2d6af86 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -1276,13 +1276,6 @@ static int mes_v11_0_early_init(void *handle)
static int mes_v11_0_late_init(void *handle)
{
- struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-
- /* it's only intended for use in mes_self_test case, not for s0ix and reset */
- if (!amdgpu_in_reset(adev) && !adev->in_s0ix && !adev->in_suspend &&
- (amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(11, 0, 3)))
- amdgpu_mes_self_test(adev);
-
return 0;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v9 4/5] drm/amdgpu: Implement userqueue signal/wait IOCTL
2024-04-29 6:43 ` [PATCH v9 4/5] drm/amdgpu: Implement userqueue signal/wait IOCTL Arunpravin Paneer Selvam
@ 2024-04-30 8:40 ` Christian König
0 siblings, 0 replies; 7+ messages in thread
From: Christian König @ 2024-04-30 8:40 UTC (permalink / raw)
To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher
Am 29.04.24 um 08:43 schrieb Arunpravin Paneer Selvam:
> This patch introduces new IOCTL for userqueue secure semaphore.
>
> The signal IOCTL called from userspace application creates a drm
> syncobj and array of bo GEM handles and passed in as parameter to
> the driver to install the fence into it.
>
> The wait IOCTL gets an array of drm syncobjs, finds the fences
> attached to the drm syncobjs and obtain the array of
> memory_address/fence_value combintion which are returned to
> userspace.
>
> v2: (Christian)
> - Install fence into GEM BO object.
> - Lock all BO's using the dma resv subsystem
> - Reorder the sequence in signal IOCTL function.
> - Get write pointer from the shadow wptr
> - use userq_fence to fetch the va/value in wait IOCTL.
>
> v3: (Christian)
> - Use drm_exec helper for the proper BO drm reserve and avoid BO
> lock/unlock issues.
> - fence/fence driver reference count logic for signal/wait IOCTLs.
>
> v4: (Christian)
> - Fixed the drm_exec calling sequence
> - use dma_resv_for_each_fence_unlock if BO's are not locked
> - Modified the fence_info array storing logic.
>
> v5: (Christian)
> - Keep fence_drv until wait queue execution.
> - Add dma_fence_wait for other fences.
> - Lock BO's using drm_exec as the number of fences in them could
> change.
> - Install signaled fences as well into BO/Syncobj.
> - Move Syncobj fence installation code after the drm_exec_prepare_array.
> - Directly add dma_resv_usage_rw(args->bo_flags....
> - remove unnecessary dma_fence_put.
>
> v6: (Christian)
> - Add xarray stuff to store the fence_drv
> - Implement a function to iterate over the xarray and drop
> the fence_drv references.
> - Add drm_exec_until_all_locked() wrapper
> - Add a check that if we haven't exceeded the user allocated num_fences
> before adding dma_fence to the fences array.
>
> v7: (Christian)
> - Use memdup_user() for kmalloc_array + copy_from_user
> - Move the fence_drv references from the xarray into the newly created fence
> and drop the fence_drv references when we signal this fence.
> - Move this locking of BOs before the "if (!wait_info->num_fences)",
> this way you need this code block only once.
> - Merge the error handling code and the cleanup + return 0 code.
> - Initializing the xa should probably be done in the userq code.
> - Remove the userq back pointer stored in fence_drv.
> - Pass xarray as parameter in amdgpu_userq_walk_and_drop_fence_drv()
>
> v8: (Christian)
> - Move fence_drv references must come before adding the fence to the list.
> - Use xa_lock_irqsave_nested for nested spinlock operations.
> - userq_mgr should be per fpriv and not one per device.
> - Restructure the interrupt process code for the early exit of the loop.
> - The reference acquired in the syncobj fence replace code needs to be
> kept around.
> - Modify the dma_fence acquire placement in wait IOCTL.
> - Move USERQ_BO_WRITE flag to UAPI header file.
> - drop the fence drv reference after telling the hw to stop accessing it.
> - Add multi sync object support to userq signal IOCTL.
>
> Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
> Suggested-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 +
> .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 454 +++++++++++++++++-
> .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.h | 5 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 29 +-
> .../gpu/drm/amd/include/amdgpu_userqueue.h | 1 +
> 5 files changed, 484 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index 844f7b5f90db..5892a4c1a92e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -2918,6 +2918,8 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = {
> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_OP, amdgpu_gem_op_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_USERPTR, amdgpu_gem_userptr_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> DRM_IOCTL_DEF_DRV(AMDGPU_USERQ, amdgpu_userq_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> + DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_SIGNAL, amdgpu_userq_signal_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> + DRM_IOCTL_DEF_DRV(AMDGPU_USERQ_WAIT, amdgpu_userq_wait_ioctl, DRM_AUTH|DRM_RENDER_ALLOW),
> };
>
> static const struct drm_driver amdgpu_kms_driver = {
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
> index f7baea2c67ab..6fb75cc1d20c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
> @@ -25,6 +25,7 @@
> #include <linux/kref.h>
> #include <linux/slab.h>
>
> +#include <drm/drm_exec.h>
> #include <drm/drm_syncobj.h>
>
> #include "amdgpu.h"
> @@ -102,8 +103,11 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
>
> void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv)
> {
> + struct amdgpu_userq_fence_driver *stored_fence_drv;
> struct amdgpu_userq_fence *userq_fence, *tmp;
> + unsigned long index, flags;
> struct dma_fence *fence;
> + struct xarray *xa;
> u64 rptr;
>
> if (!fence_drv)
> @@ -114,15 +118,25 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d
> spin_lock(&fence_drv->fence_list_lock);
> list_for_each_entry_safe(userq_fence, tmp, &fence_drv->fences, link) {
> fence = &userq_fence->base;
> + xa = &userq_fence->fence_drv_xa;
>
> - if (rptr >= fence->seqno) {
> - dma_fence_signal(fence);
> - list_del(&userq_fence->link);
> -
> - dma_fence_put(fence);
> - } else {
> + if (rptr < fence->seqno)
> break;
> +
> + dma_fence_signal(fence);
> + /*
> + * Walk over the fence_drv xarray and drop the old wait ioctl
> + * fence_drv references.
> + */
> + xa_lock_irqsave_nested(xa, flags, SINGLE_DEPTH_NESTING);
> + xa_for_each(xa, index, stored_fence_drv) {
> + __xa_erase(xa, index);
> + amdgpu_userq_fence_driver_put(stored_fence_drv);
> }
> + xa_unlock_irqrestore(xa, flags);
> +
> + list_del(&userq_fence->link);
> + dma_fence_put(fence);
> }
> spin_unlock(&fence_drv->fence_list_lock);
> }
> @@ -132,8 +146,11 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
> struct amdgpu_userq_fence_driver *fence_drv = container_of(ref,
> struct amdgpu_userq_fence_driver,
> refcount);
> + struct amdgpu_userq_fence_driver *xa_fence_drv;
> struct amdgpu_device *adev = fence_drv->adev;
> struct amdgpu_userq_fence *fence, *tmp;
> + struct xarray *xa = &adev->userq_xa;
> + unsigned long index;
> struct dma_fence *f;
>
> spin_lock(&fence_drv->fence_list_lock);
> @@ -150,6 +167,12 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
> }
> spin_unlock(&fence_drv->fence_list_lock);
>
> + xa_lock(xa);
> + xa_for_each(xa, index, xa_fence_drv)
> + if (xa_fence_drv == fence_drv)
> + __xa_erase(xa, index);
> + xa_unlock(xa);
> +
> /* Free seq64 memory */
> amdgpu_seq64_free(adev, fence_drv->gpu_addr);
> kfree(fence_drv);
> @@ -188,9 +211,37 @@ int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq,
> dma_fence_init(fence, &amdgpu_userq_fence_ops, &userq_fence->lock,
> fence_drv->context, seq);
>
> + xa_init_flags(&userq_fence->fence_drv_xa, XA_FLAGS_LOCK_IRQ);
> +
> amdgpu_userq_fence_driver_get(fence_drv);
> dma_fence_get(fence);
>
> + if (!xa_empty(&userq->uq_fence_drv_xa)) {
> + struct amdgpu_userq_fence_driver *stored_fence_drv;
> + unsigned long flags, index;
> +
> + /*
> + * Move fence_drv references of old signal IOCTL calls to the
> + * newly created userq fence xarray.
> + */
> + xa_lock(&userq->uq_fence_drv_xa);
> + xa_for_each(&userq->uq_fence_drv_xa, index, stored_fence_drv) {
> + /* Skip if the queues are different */
> + if (stored_fence_drv != fence_drv)
> + continue;
Hui? That looks inverted to me.
We need to store all the references to other fences drivers and *not*
ourself.
But this should probably be checked before adding the fence driver to
the XA instead.
> +
> + xa_lock_irqsave_nested(&userq_fence->fence_drv_xa, flags,
> + SINGLE_DEPTH_NESTING);
> + __xa_store(&userq_fence->fence_drv_xa, index,
> + stored_fence_drv, GFP_KERNEL);
That won't work like this. The problem is that __xa_store() can only
drop it's own lock, but that one is nested with the one in uq_fence_drv_xa.
So you need to iterate over uq_fence_drv_xa without holding a lock or
come up with some other idea to store the fence driver references, e.g.
a kvmalloced() array for example.
Since the we don't want to modify userq_fence->fence_drv_xa any more
that should work out fine.
Apart from that the patch looks like it should work.
Regards,
Christian.
> + xa_unlock_irqrestore(&userq_fence->fence_drv_xa, flags);
> +
> + /* Erase fence_drv reference entry from userq xarray */
> + __xa_erase(&userq->uq_fence_drv_xa, index);
> + }
> + xa_unlock(&userq->uq_fence_drv_xa);
> + }
> +
> spin_lock(&fence_drv->fence_list_lock);
> /* Check if hardware has already processed the job */
> if (!dma_fence_is_signaled(fence))
> @@ -240,6 +291,8 @@ static void amdgpu_userq_fence_free(struct rcu_head *rcu)
>
> /* Release the fence driver reference */
> amdgpu_userq_fence_driver_put(fence_drv);
> +
> + xa_destroy(&userq_fence->fence_drv_xa);
> kmem_cache_free(amdgpu_userq_fence_slab, userq_fence);
> }
>
> @@ -255,3 +308,392 @@ static const struct dma_fence_ops amdgpu_userq_fence_ops = {
> .signaled = amdgpu_userq_fence_signaled,
> .release = amdgpu_userq_fence_release,
> };
> +
> +/**
> + * amdgpu_userq_fence_read_wptr - Read the userq wptr value
> + *
> + * @filp: drm file private data structure
> + * @queue: user mode queue structure pointer
> + * @wptr: write pointer value
> + *
> + * Read the wptr value from userq's MQD. The userq signal IOCTL
> + * creates a dma_fence for the shared buffers that expects the
> + * RPTR value written to seq64 memory >= WPTR.
> + *
> + * Returns wptr value on success, error on failure.
> + */
> +static int amdgpu_userq_fence_read_wptr(struct drm_file *filp,
> + struct amdgpu_usermode_queue *queue,
> + u64 *wptr)
> +{
> + struct amdgpu_fpriv *fpriv = filp->driver_priv;
> + struct amdgpu_bo_va_mapping *mapping;
> + struct amdgpu_vm *vm = &fpriv->vm;
> + struct amdgpu_bo *bo;
> + u64 addr, *ptr;
> + int r;
> +
> + addr = queue->userq_prop->wptr_gpu_addr;
> + addr &= AMDGPU_GMC_HOLE_MASK;
> +
> + mapping = amdgpu_vm_bo_lookup_mapping(vm, addr >> PAGE_SHIFT);
> + if (!mapping)
> + return -EINVAL;
> +
> + bo = mapping->bo_va->base.bo;
> + r = amdgpu_bo_reserve(bo, true);
> + if (r) {
> + DRM_ERROR("Failed to reserve userqueue wptr bo");
> + return r;
> + }
> +
> + r = amdgpu_bo_kmap(bo, (void **)&ptr);
> + if (r) {
> + DRM_ERROR("Failed mapping the userqueue wptr bo");
> + goto map_error;
> + }
> +
> + *wptr = le64_to_cpu(*ptr);
> +
> + amdgpu_bo_kunmap(bo);
> + amdgpu_bo_unreserve(bo);
> +
> + return 0;
> +
> +map_error:
> + amdgpu_bo_unreserve(bo);
> + return r;
> +}
> +
> +int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
> + struct drm_file *filp)
> +{
> + struct amdgpu_fpriv *fpriv = filp->driver_priv;
> + struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr;
> + struct drm_amdgpu_userq_signal *args = data;
> + struct amdgpu_usermode_queue *queue;
> + struct drm_gem_object **gobj = NULL;
> + struct drm_syncobj **syncobj = NULL;
> + u32 *syncobj_handles, num_syncobj_handles;
> + u32 *bo_handles, num_bo_handles;
> + struct dma_fence *fence;
> + struct drm_exec exec;
> + int r, i, entry;
> + u64 wptr;
> +
> + /* Array of syncobj handles */
> + num_syncobj_handles = args->num_syncobj_handles;
> + syncobj_handles = memdup_user(u64_to_user_ptr(args->syncobj_handles_array),
> + sizeof(u32) * num_syncobj_handles);
> + if (IS_ERR(syncobj_handles))
> + return PTR_ERR(syncobj_handles);
> +
> + /* Array of syncobj object handles */
> + syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), GFP_KERNEL);
> + if (!syncobj) {
> + r = -ENOMEM;
> + goto free_syncobj_handles;
> + }
> +
> + for (entry = 0; entry < num_syncobj_handles; entry++) {
> + syncobj[entry] = drm_syncobj_find(filp, syncobj_handles[entry]);
> + if (!syncobj[entry]) {
> + r = -ENOENT;
> + goto free_syncobj_handles;
> + }
> + }
> +
> + /* Array of bo handles */
> + num_bo_handles = args->num_bo_handles;
> + bo_handles = memdup_user(u64_to_user_ptr(args->bo_handles_array),
> + sizeof(u32) * num_bo_handles);
> + if (IS_ERR(bo_handles))
> + goto free_syncobj_handles;
> +
> + /* Array of GEM object handles */
> + gobj = kmalloc_array(num_bo_handles, sizeof(*gobj), GFP_KERNEL);
> + if (!gobj) {
> + r = -ENOMEM;
> + goto free_bo_handles;
> + }
> +
> + for (entry = 0; entry < num_bo_handles; entry++) {
> + gobj[entry] = drm_gem_object_lookup(filp, bo_handles[entry]);
> + if (!gobj[entry]) {
> + r = -ENOENT;
> + goto put_gobj;
> + }
> + }
> +
> + drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
> + drm_exec_until_all_locked(&exec) {
> + r = drm_exec_prepare_array(&exec, gobj, num_bo_handles, 1);
> + drm_exec_retry_on_contention(&exec);
> + if (r)
> + goto exec_fini;
> + }
> +
> + /*Retrieve the user queue */
> + queue = idr_find(&userq_mgr->userq_idr, args->queue_id);
> + if (!queue) {
> + r = -ENOENT;
> + goto exec_fini;
> + }
> +
> + r = amdgpu_userq_fence_read_wptr(filp, queue, &wptr);
> + if (r)
> + goto exec_fini;
> +
> + /* Create a new fence */
> + r = amdgpu_userq_fence_create(queue, wptr, &fence);
> + if (r)
> + goto exec_fini;
> +
> + for (i = 0; i < num_bo_handles; i++)
> + dma_resv_add_fence(gobj[i]->resv, fence,
> + dma_resv_usage_rw(args->bo_flags &
> + AMDGPU_USERQ_BO_WRITE));
> +
> + /* Add the created fence to syncobj/BO's */
> + for (i = 0; i < num_syncobj_handles; i++)
> + drm_syncobj_replace_fence(syncobj[i], fence);
> +
> + /* drop the reference acquired in fence creation function */
> + dma_fence_put(fence);
> +
> +exec_fini:
> + drm_exec_fini(&exec);
> +put_gobj:
> + while (entry-- > 0)
> + drm_gem_object_put(gobj[entry]);
> + kfree(gobj);
> +free_bo_handles:
> + kfree(bo_handles);
> +free_syncobj_handles:
> + while (i-- > 0)
> + drm_syncobj_put(syncobj[i]);
> + kfree(syncobj_handles);
> +
> + return r;
> +}
> +
> +int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data,
> + struct drm_file *filp)
> +{
> + struct amdgpu_fpriv *fpriv = filp->driver_priv;
> + struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr;
> + struct drm_amdgpu_userq_fence_info *fence_info = NULL;
> + struct drm_amdgpu_userq_wait *wait_info = data;
> + struct amdgpu_usermode_queue *waitq;
> + u32 *syncobj_handles, *bo_handles;
> + struct dma_fence **fences = NULL;
> + u32 num_syncobj, num_bo_handles;
> + struct drm_gem_object **gobj;
> + struct drm_exec exec;
> + int r, i, entry, cnt;
> + u64 num_fences = 0;
> +
> + num_bo_handles = wait_info->num_bo_handles;
> + bo_handles = memdup_user(u64_to_user_ptr(wait_info->bo_handles_array),
> + sizeof(u32) * num_bo_handles);
> + if (IS_ERR(bo_handles))
> + return PTR_ERR(bo_handles);
> +
> + num_syncobj = wait_info->num_syncobj_handles;
> + syncobj_handles = memdup_user(u64_to_user_ptr(wait_info->syncobj_handles_array),
> + sizeof(u32) * num_syncobj);
> + if (IS_ERR(syncobj_handles)) {
> + r = PTR_ERR(syncobj_handles);
> + goto free_bo_handles;
> + }
> +
> + /* Array of GEM object handles */
> + gobj = kmalloc_array(num_bo_handles, sizeof(*gobj), GFP_KERNEL);
> + if (!gobj) {
> + r = -ENOMEM;
> + goto free_syncobj_handles;
> + }
> +
> + for (entry = 0; entry < num_bo_handles; entry++) {
> + gobj[entry] = drm_gem_object_lookup(filp, bo_handles[entry]);
> + if (!gobj[entry]) {
> + r = -ENOENT;
> + goto put_gobj;
> + }
> + }
> +
> + drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, 0);
> + drm_exec_until_all_locked(&exec) {
> + r = drm_exec_prepare_array(&exec, gobj, num_bo_handles, 0);
> + drm_exec_retry_on_contention(&exec);
> + if (r) {
> + drm_exec_fini(&exec);
> + goto put_gobj;
> + }
> + }
> +
> + if (!wait_info->num_fences) {
> + /* Count syncobj's fence */
> + for (i = 0; i < num_syncobj; i++) {
> + struct dma_fence *fence;
> +
> + r = drm_syncobj_find_fence(filp, syncobj_handles[i],
> + 0, 0, &fence);
> + dma_fence_put(fence);
> +
> + if (r || !fence)
> + continue;
> +
> + num_fences++;
> + }
> +
> + /* Count GEM objects fence */
> + for (i = 0; i < num_bo_handles; i++) {
> + struct dma_resv_iter resv_cursor;
> + struct dma_fence *fence;
> +
> + dma_resv_for_each_fence(&resv_cursor, gobj[i]->resv,
> + dma_resv_usage_rw(wait_info->bo_wait_flags &
> + AMDGPU_USERQ_BO_WRITE), fence)
> + num_fences++;
> + }
> +
> + /*
> + * Passing num_fences = 0 means that userspace doesn't want to
> + * retrieve userq_fence_info. If num_fences = 0 we skip filling
> + * userq_fence_info and return the actual number of fences on
> + * args->num_fences.
> + */
> + wait_info->num_fences = num_fences;
> + } else {
> + /* Array of fence info */
> + fence_info = kmalloc_array(wait_info->num_fences, sizeof(*fence_info), GFP_KERNEL);
> + if (!fence_info) {
> + r = -ENOMEM;
> + goto exec_fini;
> + }
> +
> + /* Array of fences */
> + fences = kmalloc_array(wait_info->num_fences, sizeof(*fences), GFP_KERNEL);
> + if (!fences) {
> + r = -ENOMEM;
> + goto free_fence_info;
> + }
> +
> + /* Retrieve GEM objects fence */
> + for (i = 0; i < num_bo_handles; i++) {
> + struct dma_resv_iter resv_cursor;
> + struct dma_fence *fence;
> +
> + dma_resv_for_each_fence(&resv_cursor, gobj[i]->resv,
> + dma_resv_usage_rw(wait_info->bo_wait_flags &
> + AMDGPU_USERQ_BO_WRITE), fence) {
> + if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) {
> + r = -EINVAL;
> + goto free_fences;
> + }
> +
> + fences[num_fences++] = fence;
> + dma_fence_get(fence);
> + }
> + }
> +
> + /* Retrieve syncobj's fence */
> + for (i = 0; i < num_syncobj; i++) {
> + struct dma_fence *fence;
> +
> + r = drm_syncobj_find_fence(filp, syncobj_handles[i],
> + 0, 0, &fence);
> + if (r || !fence)
> + continue;
> +
> + if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) {
> + r = -EINVAL;
> + goto free_fences;
> + }
> +
> + fences[num_fences++] = fence;
> + }
> +
> + waitq = idr_find(&userq_mgr->userq_idr, wait_info->waitq_id);
> + if (!waitq)
> + goto free_fences;
> +
> + for (i = 0, cnt = 0; i < wait_info->num_fences; i++) {
> + struct amdgpu_userq_fence_driver *fence_drv;
> + struct amdgpu_userq_fence *userq_fence;
> + u32 index;
> +
> + userq_fence = to_amdgpu_userq_fence(fences[i]);
> + if (!userq_fence) {
> + /*
> + * Just waiting on other driver fences should
> + * be good for now
> + */
> + dma_fence_wait(fences[i], false);
> + dma_fence_put(fences[i]);
> +
> + continue;
> + }
> +
> + fence_drv = userq_fence->fence_drv;
> + /*
> + * We need to make sure the user queue release their reference
> + * to the fence drivers at some point before queue destruction.
> + * Otherwise, we would gather those references until we don't
> + * have any more space left and crash.
> + */
> + r = xa_alloc(&waitq->uq_fence_drv_xa, &index,
> + fence_drv, xa_limit_32b, GFP_KERNEL);
> + if (r)
> + goto free_fences;
> +
> + amdgpu_userq_fence_driver_get(fence_drv);
> +
> + /* Store drm syncobj's gpu va address and value */
> + fence_info[cnt].va = fence_drv->gpu_addr;
> + fence_info[cnt].value = fences[i]->seqno;
> +
> + dma_fence_put(fences[i]);
> + /* Increment the actual userq fence count */
> + cnt++;
> + }
> +
> + wait_info->num_fences = cnt;
> + /* Copy userq fence info to user space */
> + if (copy_to_user(u64_to_user_ptr(wait_info->userq_fence_info),
> + fence_info, wait_info->num_fences * sizeof(*fence_info))) {
> + r = -EFAULT;
> + goto free_fences;
> + }
> +
> + kfree(fences);
> + kfree(fence_info);
> + }
> +
> + drm_exec_fini(&exec);
> + for (i = 0; i < num_bo_handles; i++)
> + drm_gem_object_put(gobj[i]);
> +
> + kfree(syncobj_handles);
> + kfree(bo_handles);
> +
> + return 0;
> +
> +free_fences:
> + kfree(fences);
> +free_fence_info:
> + kfree(fence_info);
> +exec_fini:
> + drm_exec_fini(&exec);
> +put_gobj:
> + while (entry-- > 0)
> + drm_gem_object_put(gobj[entry]);
> + kfree(gobj);
> +free_syncobj_handles:
> + kfree(syncobj_handles);
> +free_bo_handles:
> + kfree(bo_handles);
> +
> + return r;
> +}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
> index c3e04cdbb9e7..1b75900fc782 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.h
> @@ -37,6 +37,7 @@ struct amdgpu_userq_fence {
> */
> spinlock_t lock;
> struct list_head link;
> + struct xarray fence_drv_xa;
> struct amdgpu_userq_fence_driver *fence_drv;
> };
>
> @@ -65,5 +66,9 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
> struct amdgpu_usermode_queue *userq);
> void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
> void amdgpu_userq_fence_driver_destroy(struct kref *ref);
> +int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
> + struct drm_file *filp);
> +int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data,
> + struct drm_file *filp);
>
> #endif
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
> index 73d225aa3e1e..404c39073661 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c
> @@ -28,6 +28,32 @@
> #include "amdgpu_userqueue.h"
> #include "amdgpu_userq_fence.h"
>
> +static void amdgpu_userq_walk_and_drop_fence_drv(struct xarray *xa)
> +{
> + struct amdgpu_userq_fence_driver *fence_drv;
> + unsigned long index;
> +
> + if (xa_empty(xa))
> + return;
> +
> + xa_lock(xa);
> + xa_for_each(xa, index, fence_drv) {
> + __xa_erase(xa, index);
> + amdgpu_userq_fence_driver_put(fence_drv);
> + }
> +
> + xa_unlock(xa);
> +}
> +
> +static void
> +amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq)
> +{
> + amdgpu_userq_walk_and_drop_fence_drv(&userq->uq_fence_drv_xa);
> + xa_destroy(&userq->uq_fence_drv_xa);
> + /* Drop the fence_drv reference held by user queue */
> + amdgpu_userq_fence_driver_put(userq->fence_drv);
> +}
> +
> static void
> amdgpu_userqueue_cleanup(struct amdgpu_userq_mgr *uq_mgr,
> struct amdgpu_usermode_queue *queue,
> @@ -37,7 +63,7 @@ amdgpu_userqueue_cleanup(struct amdgpu_userq_mgr *uq_mgr,
> const struct amdgpu_userq_funcs *uq_funcs = adev->userq_funcs[queue->queue_type];
>
> uq_funcs->mqd_destroy(uq_mgr, queue);
> - amdgpu_userq_fence_driver_put(queue->fence_drv);
> + amdgpu_userq_fence_driver_free(queue);
> idr_remove(&uq_mgr->userq_idr, queue_id);
> kfree(queue);
> }
> @@ -405,6 +431,7 @@ amdgpu_userqueue_create(struct drm_file *filp, union drm_amdgpu_userq *args)
> }
> queue->doorbell_index = index;
>
> + xa_init_flags(&queue->uq_fence_drv_xa, XA_FLAGS_ALLOC);
> r = amdgpu_userq_fence_driver_alloc(adev, queue);
> if (r) {
> DRM_ERROR("Failed to alloc fence driver\n");
> diff --git a/drivers/gpu/drm/amd/include/amdgpu_userqueue.h b/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
> index d3738f645adc..27e1617b234f 100644
> --- a/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
> +++ b/drivers/gpu/drm/amd/include/amdgpu_userqueue.h
> @@ -48,6 +48,7 @@ struct amdgpu_usermode_queue {
> struct amdgpu_userq_obj db_obj;
> struct amdgpu_userq_obj fw_obj;
> struct amdgpu_userq_obj wptr_obj;
> + struct xarray uq_fence_drv_xa;
> struct amdgpu_userq_fence_driver *fence_drv;
> };
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v9 5/5] drm/amdgpu: Enable userq fence interrupt support
2024-04-29 6:43 ` [PATCH v9 5/5] drm/amdgpu: Enable userq fence interrupt support Arunpravin Paneer Selvam
@ 2024-04-30 8:52 ` Christian König
0 siblings, 0 replies; 7+ messages in thread
From: Christian König @ 2024-04-30 8:52 UTC (permalink / raw)
To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher
Am 29.04.24 um 08:43 schrieb Arunpravin Paneer Selvam:
> Add support to handle the userqueue protected fence signal hardware
> interrupt.
>
> Create a xarray which maps the doorbell index to the fence driver address.
> This would help to retrieve the fence driver information when an userq fence
> interrupt is triggered. Firmware sends the doorbell offset value and
> this info is compared with the queue's mqd doorbell offset value.
> If they are same, we process the userq fence interrupt.
>
> Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 5 ++--
> .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 6 +++++
> drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 25 ++++++++++---------
> drivers/gpu/drm/amd/amdgpu/mes_v10_1.c | 5 ----
> drivers/gpu/drm/amd/amdgpu/mes_v11_0.c | 7 ------
> 6 files changed, 23 insertions(+), 27 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 4ca14b02668b..2d5ef2e74c71 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1043,6 +1043,8 @@ struct amdgpu_device {
> struct amdgpu_mqd mqds[AMDGPU_HW_IP_NUM];
> const struct amdgpu_userq_funcs *userq_funcs[AMDGPU_HW_IP_NUM];
>
> + struct xarray userq_xa;
> +
> /* df */
> struct amdgpu_df df;
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 7753a2e64d41..fd919105a181 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3982,6 +3982,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
> spin_lock_init(&adev->audio_endpt_idx_lock);
> spin_lock_init(&adev->mm_stats.lock);
>
> + xa_init_flags(&adev->userq_xa, XA_FLAGS_LOCK_IRQ);
> +
> INIT_LIST_HEAD(&adev->shadow_list);
> mutex_init(&adev->shadow_list_lock);
>
> @@ -4719,9 +4721,6 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
> }
> adev->in_suspend = false;
>
> - if (adev->enable_mes)
> - amdgpu_mes_self_test(adev);
> -
Since this patch here breaks the in kernel MES self test you should
probably create a follow up patch to completely remove it.
(Or even better remove it before applying this patch).
> if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DEV_D0))
> DRM_WARN("smart shift update failed\n");
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
> index 6fb75cc1d20c..614953b0fc19 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
> @@ -70,6 +70,7 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
> struct amdgpu_usermode_queue *userq)
> {
> struct amdgpu_userq_fence_driver *fence_drv;
> + unsigned long flags;
> int r;
>
> fence_drv = kzalloc(sizeof(*fence_drv), GFP_KERNEL);
> @@ -96,6 +97,11 @@ int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
> fence_drv->context = dma_fence_context_alloc(1);
> get_task_comm(fence_drv->timeline_name, current);
>
> + xa_lock_irqsave(&adev->userq_xa, flags);
> + __xa_store(&adev->userq_xa, userq->doorbell_index,
> + fence_drv, GFP_KERNEL);
> + xa_unlock_irqrestore(&adev->userq_xa, flags);
> +
> userq->fence_drv = fence_drv;
>
> return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> index a786e25432ae..d6cdca0a652f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
> @@ -49,6 +49,7 @@
> #include "gfx_v11_0_3.h"
> #include "nbio_v4_3.h"
> #include "mes_v11_0.h"
> +#include "amdgpu_userq_fence.h"
>
> #define GFX11_NUM_GFX_RINGS 1
> #define GFX11_MEC_HPD_SIZE 2048
> @@ -5939,25 +5940,25 @@ static int gfx_v11_0_eop_irq(struct amdgpu_device *adev,
> struct amdgpu_irq_src *source,
> struct amdgpu_iv_entry *entry)
> {
> - int i;
> + u32 doorbell_offset = entry->src_data[0];
> u8 me_id, pipe_id, queue_id;
> struct amdgpu_ring *ring;
> - uint32_t mes_queue_id = entry->src_data[0];
> + int i;
>
> DRM_DEBUG("IH: CP EOP\n");
>
> - if (adev->enable_mes && (mes_queue_id & AMDGPU_FENCE_MES_QUEUE_FLAG)) {
> - struct amdgpu_mes_queue *queue;
> + if (adev->enable_mes && doorbell_offset) {
> + struct amdgpu_userq_fence_driver *fence_drv = NULL;
> + struct xarray *xa = &adev->userq_xa;
> + unsigned long index, flags;
>
> - mes_queue_id &= AMDGPU_FENCE_MES_QUEUE_ID_MASK;
> + xa_lock_irqsave(xa, flags);
> + xa_for_each(xa, index, fence_drv)
> + if (doorbell_offset == index)
> + break;
The array is indexed by the doorbell offset, so instead of going over
all the entries you can just use xa_load().
> + xa_unlock_irqrestore(xa, flags);
>
> - spin_lock(&adev->mes.queue_id_lock);
> - queue = idr_find(&adev->mes.queue_id_idr, mes_queue_id);
> - if (queue) {
> - DRM_DEBUG("process mes queue id = %d\n", mes_queue_id);
> - amdgpu_fence_process(queue->ring);
> - }
> - spin_unlock(&adev->mes.queue_id_lock);
> + amdgpu_userq_fence_driver_process(fence_drv);
You need to make sure that the fence_drv isn't freed until this completes.
The easiest way would be to keep the xa lock until that is done.
> } else {
> me_id = (entry->ring_id & 0x0c) >> 2;
> pipe_id = (entry->ring_id & 0x03) >> 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
> index 4d1121d1a1e7..faa489c75fea 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
> @@ -1163,11 +1163,6 @@ static int mes_v10_0_early_init(void *handle)
>
> static int mes_v10_0_late_init(void *handle)
> {
> - struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> -
> - if (!amdgpu_in_reset(adev))
> - amdgpu_mes_self_test(adev);
> -
> return 0;
> }
You can completely drop the function. The callback is ignored when NULL.
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> index feb7fa2c304c..0051e2d6af86 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> @@ -1276,13 +1276,6 @@ static int mes_v11_0_early_init(void *handle)
>
> static int mes_v11_0_late_init(void *handle)
> {
> - struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> -
> - /* it's only intended for use in mes_self_test case, not for s0ix and reset */
> - if (!amdgpu_in_reset(adev) && !adev->in_s0ix && !adev->in_suspend &&
> - (amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(11, 0, 3)))
> - amdgpu_mes_self_test(adev);
> -
> return 0;
> }
Same here.
Regards,
Christian.
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-04-30 8:52 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-29 6:43 [PATCH v9 1/5] drm/amdgpu: Implement a new userqueue fence driver Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 2/5] drm/amdgpu: Add mqd support for the fence address Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 3/5] drm/amdgpu: UAPI headers for userqueue Secure semaphore Arunpravin Paneer Selvam
2024-04-29 6:43 ` [PATCH v9 4/5] drm/amdgpu: Implement userqueue signal/wait IOCTL Arunpravin Paneer Selvam
2024-04-30 8:40 ` Christian König
2024-04-29 6:43 ` [PATCH v9 5/5] drm/amdgpu: Enable userq fence interrupt support Arunpravin Paneer Selvam
2024-04-30 8:52 ` Christian König
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.