* [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence
@ 2025-04-09 5:48 Arunpravin Paneer Selvam
2025-04-09 5:48 ` [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked Arunpravin Paneer Selvam
` (4 more replies)
0 siblings, 5 replies; 11+ messages in thread
From: Arunpravin Paneer Selvam @ 2025-04-09 5:48 UTC (permalink / raw)
To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam
Fix lockdep warnings.
[ +0.000637] ================================
[ +0.000004] WARNING: inconsistent lock state
[ +0.000004] 6.12.0+ #18 Tainted: G W OE
[ +0.000004] --------------------------------
[ +0.000004] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
[ +0.000004] Xwayland/1952 [HC0[0]:SC0[0]:HE1:SE1] takes:
[ +0.000005] ffff8884636f4740 (&fence_drv->fence_list_lock){?...}-{2:2}, at: amdgpu_userq_fence_driver_destroy+0xb8/0x540 [amdgpu]
[ +0.000208] {IN-HARDIRQ-W} state was registered at:
[ +0.000004] lock_acquire.part.0+0x116/0x360
[ +0.000005] lock_acquire+0x7c/0xc0
[ +0.000005] _raw_spin_lock+0x2f/0x60
[ +0.000005] amdgpu_userq_fence_driver_process+0x75/0x400 [amdgpu]
[ +0.000185] gfx_v12_0_eop_irq+0x29f/0x420 [amdgpu]
[ +0.000210] amdgpu_irq_dispatch+0x2a4/0x7b0 [amdgpu]
[ +0.000191] amdgpu_ih_process+0x1e1/0x3d0 [amdgpu]
[ +0.000185] amdgpu_irq_handler+0x28/0xc0 [amdgpu]
[ +0.000183] __handle_irq_event_percpu+0x1bb/0x590
[ +0.000005] handle_irq_event+0xab/0x1d0
[ +0.000005] handle_edge_irq+0x1fd/0xc10
[ +0.000005] __common_interrupt+0x83/0x190
[ +0.000004] common_interrupt+0xb1/0xe0
[ +0.000005] asm_common_interrupt+0x27/0x40
[ +0.000004] cpuidle_enter_state+0x2ba/0x530
[ +0.000005] cpuidle_enter+0x4f/0xb0
[ +0.000006] call_cpuidle+0x46/0xd0
[ +0.000005] do_idle+0x367/0x430
[ +0.000004] cpu_startup_entry+0x58/0x70
[ +0.000005] start_secondary+0x224/0x2b0
[ +0.000005] common_startup_64+0x13e/0x141
[ +0.000005] irq event stamp: 88271
[ +0.000004] hardirqs last enabled at (88271): [<ffffffffad9ca7a1>] _raw_spin_unlock_irqrestore+0x51/0x80
[ +0.000005] hardirqs last disabled at (88270): [<ffffffffad9ca424>] _raw_spin_lock_irqsave+0x74/0x80
[ +0.000005] softirqs last enabled at (87858): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0
[ +0.000005] softirqs last disabled at (87849): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0
[ +0.000005]
other info that might help us debug this:
[ +0.000004] Possible unsafe locking scenario:
[ +0.000003] CPU0
[ +0.000004] ----
[ +0.000003] lock(&fence_drv->fence_list_lock);
Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
index a4953d668972..24d19b920100 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c
@@ -159,10 +159,11 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
struct amdgpu_device *adev = fence_drv->adev;
struct amdgpu_userq_fence *fence, *tmp;
struct xarray *xa = &adev->userq_xa;
+ unsigned long fence_list_flags;
unsigned long index, flags;
struct dma_fence *f;
- spin_lock(&fence_drv->fence_list_lock);
+ spin_lock_irqsave(&fence_drv->fence_list_lock, fence_list_flags);
list_for_each_entry_safe(fence, tmp, &fence_drv->fences, link) {
f = &fence->base;
@@ -174,7 +175,7 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref)
list_del(&fence->link);
dma_fence_put(f);
}
- spin_unlock(&fence_drv->fence_list_lock);
+ spin_unlock_irqrestore(&fence_drv->fence_list_lock, fence_list_flags);
xa_lock_irqsave(xa, flags);
xa_for_each(xa, index, xa_fence_drv)
--
2.43.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked 2025-04-09 5:48 [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Arunpravin Paneer Selvam @ 2025-04-09 5:48 ` Arunpravin Paneer Selvam 2025-05-02 12:35 ` Christian König 2025-04-09 5:48 ` [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings Arunpravin Paneer Selvam ` (3 subsequent siblings) 4 siblings, 1 reply; 11+ messages in thread From: Arunpravin Paneer Selvam @ 2025-04-09 5:48 UTC (permalink / raw) To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam Add lock before accessing dma_fence_is_signaled_locked. Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> --- drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c index 24d19b920100..d5b35b5df527 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c @@ -259,11 +259,13 @@ static int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq, /* Check if hardware has already processed the job */ spin_lock_irqsave(&fence_drv->fence_list_lock, flags); + spin_lock_nested(fence->lock, SINGLE_DEPTH_NESTING); if (!dma_fence_is_signaled_locked(fence)) list_add_tail(&userq_fence->link, &fence_drv->fences); else dma_fence_put(fence); + spin_unlock(fence->lock); spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); *f = fence; -- 2.43.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked 2025-04-09 5:48 ` [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked Arunpravin Paneer Selvam @ 2025-05-02 12:35 ` Christian König 0 siblings, 0 replies; 11+ messages in thread From: Christian König @ 2025-05-02 12:35 UTC (permalink / raw) To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher On 4/9/25 07:48, Arunpravin Paneer Selvam wrote: > Add lock before accessing dma_fence_is_signaled_locked. > > Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > index 24d19b920100..d5b35b5df527 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > @@ -259,11 +259,13 @@ static int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq, > > /* Check if hardware has already processed the job */ > spin_lock_irqsave(&fence_drv->fence_list_lock, flags); > + spin_lock_nested(fence->lock, SINGLE_DEPTH_NESTING); That is clear incorrect use of spin_lock_nested(). Why does a normal spinlock() doesn't work? Regards, Christian. > if (!dma_fence_is_signaled_locked(fence)) > list_add_tail(&userq_fence->link, &fence_drv->fences); > else > dma_fence_put(fence); > > + spin_unlock(fence->lock); > spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags); > > *f = fence; ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings 2025-04-09 5:48 [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Arunpravin Paneer Selvam 2025-04-09 5:48 ` [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked Arunpravin Paneer Selvam @ 2025-04-09 5:48 ` Arunpravin Paneer Selvam 2025-05-02 12:42 ` Christian König 2025-04-09 5:48 ` [PATCH 4/5] drm/amdgpu/userq: Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) Arunpravin Paneer Selvam ` (2 subsequent siblings) 4 siblings, 1 reply; 11+ messages in thread From: Arunpravin Paneer Selvam @ 2025-04-09 5:48 UTC (permalink / raw) To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam The ttm_bo_pin and ttm_bo_unpin warnings are resolved by moving the doorbell bo reserve up before pin/unpin. WARNING: CPU: 11 PID: 1818 at drivers/gpu/drm/ttm/ttm_bo.c:592 ttm_bo_pin+0x1f6/0x270 [ttm] [ +0.000277] CPU: 11 UID: 1000 PID: 1818 Comm: Xwayland Tainted: G W 6.12.0+ #15 [ +0.000006] Tainted: [W]=WARN [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 3072 12/20/2024 [ +0.000004] RIP: 0010:ttm_bo_pin+0x1f6/0x270 [ttm] [ +0.000005] RSP: 0018:ffff88846ca879d0 EFLAGS: 00010246 [ +0.000007] RAX: 0000000000000000 RBX: ffff88810b7ca848 RCX: 0000000000000000 [ +0.000004] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 [ +0.000005] RBP: ffff88846ca879e8 R08: 0000000000000000 R09: 0000000000000000 [ +0.000004] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88810b7ca848 [ +0.000004] R13: ffff88846c666250 R14: 1ffff1108d950f44 R15: ffff88846ca87aa0 [ +0.000005] FS: 00007c45ff436d00(0000) GS:ffff888409580000(0000) knlGS:0000000000000000 [ +0.000004] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.000005] CR2: 00005b0c142a60e0 CR3: 000000012ce5a000 CR4: 0000000000f50ef0 [ +0.000004] PKRU: 55555554 [ +0.000004] Call Trace: [ +0.000004] <TASK> [ +0.000005] ? show_regs+0x6c/0x80 [ +0.000007] ? __warn+0xd2/0x2d0 [ +0.000007] ? ttm_bo_pin+0x1f6/0x270 [ttm] [ +0.000031] ? report_bug+0x282/0x2f0 [ +0.000012] ? handle_bug+0x6e/0xc0 [ +0.000007] ? exc_invalid_op+0x18/0x50 [ +0.000007] ? asm_exc_invalid_op+0x1b/0x20 [ +0.000017] ? ttm_bo_pin+0x1f6/0x270 [ttm] [ +0.000014] amdgpu_bo_pin+0x365/0x9d0 [amdgpu] [ +0.000191] ? __pfx_amdgpu_bo_pin+0x10/0x10 [amdgpu] [ +0.000185] ? drm_gem_object_lookup+0x81/0xc0 [ +0.000008] ? kasan_save_alloc_info+0x37/0x60 [ +0.000007] ? __kasan_kmalloc+0xc3/0xd0 [ +0.000013] amdgpu_userqueue_get_doorbell_index+0xee/0x5f0 [amdgpu] [ +0.000209] amdgpu_userq_ioctl+0x6b4/0xd40 [amdgpu] [ +0.000193] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000211] ? lock_acquire+0x7c/0xc0 [ +0.000006] ? drm_dev_enter+0x51/0x190 [ +0.000015] drm_ioctl_kernel+0x18b/0x330 [ +0.000007] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000190] ? __pfx_drm_ioctl_kernel+0x10/0x10 [ +0.000005] ? lock_acquire+0x7c/0xc0 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? __kasan_check_write+0x14/0x30 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000011] drm_ioctl+0x589/0xd00 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000194] ? __pfx_drm_ioctl+0x10/0x10 [ +0.000006] ? __pm_runtime_resume+0x80/0x110 [ +0.000021] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? trace_hardirqs_on+0x53/0x60 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] [ +0.000185] __x64_sys_ioctl+0x13a/0x1c0 [ +0.000010] x64_sys_call+0x11ad/0x25f0 [ +0.000007] do_syscall_64+0x91/0x180 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? irqentry_exit+0x77/0xb0 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? exc_page_fault+0x93/0x150 [ +0.000009] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ +0.000005] RIP: 0033:0x7c45ff924ded [ +0.000005] RSP: 002b:00007ffff7167810 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ +0.000008] RAX: ffffffffffffffda RBX: 00000000c0486456 RCX: 00007c45ff924ded [ +0.000004] RDX: 00007ffff7167870 RSI: 00000000c0486456 RDI: 000000000000000b [ +0.000004] RBP: 00007ffff7167860 R08: ffff800100000000 R09: 0000000000010000 [ +0.000005] R10: 00007ffff7167950 R11: 0000000000000246 R12: 00005b0c2a51bc48 [ +0.000004] R13: 000000000000000b R14: 0000000000000000 R15: 00007ffff7167950 [ +0.000022] </TASK> [ +0.000004] irq event stamp: 80693 [ +0.000004] hardirqs last enabled at (80699): [<ffffffff86a693a9>] __up_console_sem+0x79/0xa0 [ +0.000005] hardirqs last disabled at (80704): [<ffffffff86a6938e>] __up_console_sem+0x5e/0xa0 [ +0.000005] softirqs last enabled at (80390): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000005] softirqs last disabled at (80385): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000006] ---[ end trace 0000000000000000 ]--- ------------------------------------------------------------------------------------------------------ [ +0.000006] WARNING: CPU: 10 PID: 1818 at drivers/gpu/drm/ttm/ttm_bo.c:611 ttm_bo_unpin+0x21f/0x2c0 [ttm] [ +0.000280] CPU: 10 UID: 1000 PID: 1818 Comm: Xwayland Tainted: G W 6.12.0+ #15 [ +0.000006] Tainted: [W]=WARN [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 3072 12/20/2024 [ +0.000004] RIP: 0010:ttm_bo_unpin+0x21f/0x2c0 [ttm] [ +0.000005] RSP: 0018:ffff88846ca87888 EFLAGS: 00010246 [ +0.000007] RAX: 0000000000000000 RBX: ffff88810b7ca848 RCX: 0000000000000000 [ +0.000005] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 [ +0.000004] RBP: ffff88846ca878a0 R08: 0000000000000000 R09: 0000000000000000 [ +0.000004] R10: 0000000000000000 R11: 0000000000000000 R12: ffff888164e90050 [ +0.000005] R13: ffff88846c666200 R14: 0000000000000001 R15: ffff888168402d28 [ +0.000004] FS: 00007c45ff436d00(0000) GS:ffff888409500000(0000) knlGS:0000000000000000 [ +0.000005] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.000004] CR2: 00007c45f7373b20 CR3: 000000012ce5a000 CR4: 0000000000f50ef0 [ +0.000005] PKRU: 55555554 [ +0.000004] Call Trace: [ +0.000004] <TASK> [ +0.000005] ? show_regs+0x6c/0x80 [ +0.000008] ? __warn+0xd2/0x2d0 [ +0.000007] ? ttm_bo_unpin+0x21f/0x2c0 [ttm] [ +0.000012] ? report_bug+0x282/0x2f0 [ +0.000013] ? handle_bug+0x6e/0xc0 [ +0.000006] ? exc_invalid_op+0x18/0x50 [ +0.000008] ? asm_exc_invalid_op+0x1b/0x20 [ +0.000017] ? ttm_bo_unpin+0x21f/0x2c0 [ttm] [ +0.000011] ? ttm_bo_unpin+0x217/0x2c0 [ttm] [ +0.000011] amdgpu_bo_unpin+0x45/0x250 [amdgpu] [ +0.000216] amdgpu_userq_ioctl+0x2c3/0xd40 [amdgpu] [ +0.000226] ? drm_dev_exit+0x2d/0x60 [ +0.000010] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000201] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? lock_acquire+0x7c/0xc0 [ +0.000006] ? drm_dev_enter+0x51/0x190 [ +0.000015] drm_ioctl_kernel+0x18b/0x330 [ +0.000007] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000188] ? __pfx_drm_ioctl_kernel+0x10/0x10 [ +0.000006] ? lock_acquire+0x7c/0xc0 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? __kasan_check_write+0x14/0x30 [ +0.000006] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000010] drm_ioctl+0x589/0xd00 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000211] ? __pfx_drm_ioctl+0x10/0x10 [ +0.000006] ? __pm_runtime_resume+0x80/0x110 [ +0.000020] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? trace_hardirqs_on+0x53/0x60 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] [ +0.000186] __x64_sys_ioctl+0x13a/0x1c0 [ +0.000010] x64_sys_call+0x11ad/0x25f0 [ +0.000007] do_syscall_64+0x91/0x180 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000010] ? __pfx___rseq_handle_notify_resume+0x10/0x10 [ +0.000005] ? __pfx_blkcg_maybe_throttle_current+0x10/0x10 [ +0.000013] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000011] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000010] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? irqentry_exit_to_user_mode+0x8b/0x260 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? irqentry_exit+0x77/0xb0 [ +0.000004] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? exc_page_fault+0x93/0x150 [ +0.000010] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ +0.000005] RIP: 0033:0x7c45ff924ded [ +0.000005] RSP: 002b:00007ffff7168790 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ +0.000008] RAX: ffffffffffffffda RBX: 00000000c0486456 RCX: 00007c45ff924ded [ +0.000005] RDX: 00007ffff71687f0 RSI: 00000000c0486456 RDI: 000000000000000b [ +0.000004] RBP: 00007ffff71687e0 R08: 00005b0c2a49b010 R09: 0000000000000007 [ +0.000004] R10: 00005b0c2a4d7140 R11: 0000000000000246 R12: 000000000000000b [ +0.000004] R13: 00007c45ff19e5cc R14: 00005b0c2a51c538 R15: 00005b0c2a51bbd8 [ +0.000022] </TASK> [ +0.000005] irq event stamp: 87419 [ +0.000004] hardirqs last enabled at (87425): [<ffffffff86a693a9>] __up_console_sem+0x79/0xa0 [ +0.000005] hardirqs last disabled at (87430): [<ffffffff86a6938e>] __up_console_sem+0x5e/0xa0 [ +0.000005] softirqs last enabled at (87058): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000006] softirqs last disabled at (87053): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000005] ---[ end trace 0000000000000000 ]--- Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> --- drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c index beae931152a3..7427e080b389 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c @@ -208,14 +208,14 @@ amdgpu_userqueue_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, db_obj->obj = amdgpu_bo_ref(gem_to_amdgpu_bo(gobj)); drm_gem_object_put(gobj); - /* Pin the BO before generating the index, unpin in queue destroy */ - r = amdgpu_bo_pin(db_obj->obj, AMDGPU_GEM_DOMAIN_DOORBELL); + r = amdgpu_bo_reserve(db_obj->obj, true); if (r) { DRM_ERROR("[Usermode queues] Failed to pin doorbell object\n"); goto unref_bo; } - r = amdgpu_bo_reserve(db_obj->obj, true); + /* Pin the BO before generating the index, unpin in queue destroy */ + r = amdgpu_bo_pin(db_obj->obj, AMDGPU_GEM_DOMAIN_DOORBELL); if (r) { DRM_ERROR("[Usermode queues] Failed to pin doorbell object\n"); goto unpin_bo; @@ -264,6 +264,7 @@ amdgpu_userqueue_destroy(struct drm_file *filp, int queue_id) struct amdgpu_fpriv *fpriv = filp->driver_priv; struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr; struct amdgpu_usermode_queue *queue; + int r; cancel_delayed_work(&uq_mgr->resume_work); mutex_lock(&uq_mgr->userq_mutex); @@ -275,7 +276,11 @@ amdgpu_userqueue_destroy(struct drm_file *filp, int queue_id) return -EINVAL; } - amdgpu_bo_unpin(queue->db_obj.obj); + r = amdgpu_bo_reserve(queue->db_obj.obj, true); + if (!r) { + amdgpu_bo_unpin(queue->db_obj.obj); + amdgpu_bo_unreserve(queue->db_obj.obj); + } amdgpu_bo_unref(&queue->db_obj.obj); amdgpu_userqueue_cleanup(uq_mgr, queue, queue_id); mutex_unlock(&uq_mgr->userq_mutex); -- 2.43.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings 2025-04-09 5:48 ` [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings Arunpravin Paneer Selvam @ 2025-05-02 12:42 ` Christian König 2025-05-12 1:36 ` Zhang, Jesse(Jie) 0 siblings, 1 reply; 11+ messages in thread From: Christian König @ 2025-05-02 12:42 UTC (permalink / raw) To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher On 4/9/25 07:48, Arunpravin Paneer Selvam wrote: > The ttm_bo_pin and ttm_bo_unpin warnings are resolved by moving the > doorbell bo reserve up before pin/unpin. > > WARNING: CPU: 11 PID: 1818 at drivers/gpu/drm/ttm/ttm_bo.c:592 ttm_bo_pin+0x1f6/0x270 [ttm] > [ +0.000277] CPU: 11 UID: 1000 PID: 1818 Comm: Xwayland Tainted: G W 6.12.0+ #15 > [ +0.000006] Tainted: [W]=WARN > [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 3072 12/20/2024 > [ +0.000004] RIP: 0010:ttm_bo_pin+0x1f6/0x270 [ttm] > [ +0.000005] RSP: 0018:ffff88846ca879d0 EFLAGS: 00010246 > [ +0.000007] RAX: 0000000000000000 RBX: ffff88810b7ca848 RCX: 0000000000000000 > [ +0.000004] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 > [ +0.000005] RBP: ffff88846ca879e8 R08: 0000000000000000 R09: 0000000000000000 > [ +0.000004] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88810b7ca848 > [ +0.000004] R13: ffff88846c666250 R14: 1ffff1108d950f44 R15: ffff88846ca87aa0 > [ +0.000005] FS: 00007c45ff436d00(0000) GS:ffff888409580000(0000) knlGS:0000000000000000 > [ +0.000004] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ +0.000005] CR2: 00005b0c142a60e0 CR3: 000000012ce5a000 CR4: 0000000000f50ef0 > [ +0.000004] PKRU: 55555554 > [ +0.000004] Call Trace: > [ +0.000004] <TASK> > [ +0.000005] ? show_regs+0x6c/0x80 > [ +0.000007] ? __warn+0xd2/0x2d0 > [ +0.000007] ? ttm_bo_pin+0x1f6/0x270 [ttm] > [ +0.000031] ? report_bug+0x282/0x2f0 > [ +0.000012] ? handle_bug+0x6e/0xc0 > [ +0.000007] ? exc_invalid_op+0x18/0x50 > [ +0.000007] ? asm_exc_invalid_op+0x1b/0x20 > [ +0.000017] ? ttm_bo_pin+0x1f6/0x270 [ttm] > [ +0.000014] amdgpu_bo_pin+0x365/0x9d0 [amdgpu] > [ +0.000191] ? __pfx_amdgpu_bo_pin+0x10/0x10 [amdgpu] > [ +0.000185] ? drm_gem_object_lookup+0x81/0xc0 > [ +0.000008] ? kasan_save_alloc_info+0x37/0x60 > [ +0.000007] ? __kasan_kmalloc+0xc3/0xd0 > [ +0.000013] amdgpu_userqueue_get_doorbell_index+0xee/0x5f0 [amdgpu] > [ +0.000209] amdgpu_userq_ioctl+0x6b4/0xd40 [amdgpu] > [ +0.000193] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] > [ +0.000211] ? lock_acquire+0x7c/0xc0 > [ +0.000006] ? drm_dev_enter+0x51/0x190 > [ +0.000015] drm_ioctl_kernel+0x18b/0x330 > [ +0.000007] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] > [ +0.000190] ? __pfx_drm_ioctl_kernel+0x10/0x10 > [ +0.000005] ? lock_acquire+0x7c/0xc0 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? __kasan_check_write+0x14/0x30 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000011] drm_ioctl+0x589/0xd00 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] > [ +0.000194] ? __pfx_drm_ioctl+0x10/0x10 > [ +0.000006] ? __pm_runtime_resume+0x80/0x110 > [ +0.000021] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? trace_hardirqs_on+0x53/0x60 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 > [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] > [ +0.000185] __x64_sys_ioctl+0x13a/0x1c0 > [ +0.000010] x64_sys_call+0x11ad/0x25f0 > [ +0.000007] do_syscall_64+0x91/0x180 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? irqentry_exit+0x77/0xb0 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? exc_page_fault+0x93/0x150 > [ +0.000009] entry_SYSCALL_64_after_hwframe+0x76/0x7e > [ +0.000005] RIP: 0033:0x7c45ff924ded > [ +0.000005] RSP: 002b:00007ffff7167810 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > [ +0.000008] RAX: ffffffffffffffda RBX: 00000000c0486456 RCX: 00007c45ff924ded > [ +0.000004] RDX: 00007ffff7167870 RSI: 00000000c0486456 RDI: 000000000000000b > [ +0.000004] RBP: 00007ffff7167860 R08: ffff800100000000 R09: 0000000000010000 > [ +0.000005] R10: 00007ffff7167950 R11: 0000000000000246 R12: 00005b0c2a51bc48 > [ +0.000004] R13: 000000000000000b R14: 0000000000000000 R15: 00007ffff7167950 > [ +0.000022] </TASK> > [ +0.000004] irq event stamp: 80693 > [ +0.000004] hardirqs last enabled at (80699): [<ffffffff86a693a9>] __up_console_sem+0x79/0xa0 > [ +0.000005] hardirqs last disabled at (80704): [<ffffffff86a6938e>] __up_console_sem+0x5e/0xa0 > [ +0.000005] softirqs last enabled at (80390): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000005] softirqs last disabled at (80385): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000006] ---[ end trace 0000000000000000 ]--- > ------------------------------------------------------------------------------------------------------ > > [ +0.000006] WARNING: CPU: 10 PID: 1818 at drivers/gpu/drm/ttm/ttm_bo.c:611 ttm_bo_unpin+0x21f/0x2c0 [ttm] > [ +0.000280] CPU: 10 UID: 1000 PID: 1818 Comm: Xwayland Tainted: G W 6.12.0+ #15 > [ +0.000006] Tainted: [W]=WARN > [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 3072 12/20/2024 > [ +0.000004] RIP: 0010:ttm_bo_unpin+0x21f/0x2c0 [ttm] > [ +0.000005] RSP: 0018:ffff88846ca87888 EFLAGS: 00010246 > [ +0.000007] RAX: 0000000000000000 RBX: ffff88810b7ca848 RCX: 0000000000000000 > [ +0.000005] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 > [ +0.000004] RBP: ffff88846ca878a0 R08: 0000000000000000 R09: 0000000000000000 > [ +0.000004] R10: 0000000000000000 R11: 0000000000000000 R12: ffff888164e90050 > [ +0.000005] R13: ffff88846c666200 R14: 0000000000000001 R15: ffff888168402d28 > [ +0.000004] FS: 00007c45ff436d00(0000) GS:ffff888409500000(0000) knlGS:0000000000000000 > [ +0.000005] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ +0.000004] CR2: 00007c45f7373b20 CR3: 000000012ce5a000 CR4: 0000000000f50ef0 > [ +0.000005] PKRU: 55555554 > [ +0.000004] Call Trace: > [ +0.000004] <TASK> > [ +0.000005] ? show_regs+0x6c/0x80 > [ +0.000008] ? __warn+0xd2/0x2d0 > [ +0.000007] ? ttm_bo_unpin+0x21f/0x2c0 [ttm] > [ +0.000012] ? report_bug+0x282/0x2f0 > [ +0.000013] ? handle_bug+0x6e/0xc0 > [ +0.000006] ? exc_invalid_op+0x18/0x50 > [ +0.000008] ? asm_exc_invalid_op+0x1b/0x20 > [ +0.000017] ? ttm_bo_unpin+0x21f/0x2c0 [ttm] > [ +0.000011] ? ttm_bo_unpin+0x217/0x2c0 [ttm] > [ +0.000011] amdgpu_bo_unpin+0x45/0x250 [amdgpu] > [ +0.000216] amdgpu_userq_ioctl+0x2c3/0xd40 [amdgpu] > [ +0.000226] ? drm_dev_exit+0x2d/0x60 > [ +0.000010] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] > [ +0.000201] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? lock_acquire+0x7c/0xc0 > [ +0.000006] ? drm_dev_enter+0x51/0x190 > [ +0.000015] drm_ioctl_kernel+0x18b/0x330 > [ +0.000007] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] > [ +0.000188] ? __pfx_drm_ioctl_kernel+0x10/0x10 > [ +0.000006] ? lock_acquire+0x7c/0xc0 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? __kasan_check_write+0x14/0x30 > [ +0.000006] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] drm_ioctl+0x589/0xd00 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] > [ +0.000211] ? __pfx_drm_ioctl+0x10/0x10 > [ +0.000006] ? __pm_runtime_resume+0x80/0x110 > [ +0.000020] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? trace_hardirqs_on+0x53/0x60 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 > [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] > [ +0.000186] __x64_sys_ioctl+0x13a/0x1c0 > [ +0.000010] x64_sys_call+0x11ad/0x25f0 > [ +0.000007] do_syscall_64+0x91/0x180 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] ? __pfx___rseq_handle_notify_resume+0x10/0x10 > [ +0.000005] ? __pfx_blkcg_maybe_throttle_current+0x10/0x10 > [ +0.000013] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 > [ +0.000011] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? irqentry_exit_to_user_mode+0x8b/0x260 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? irqentry_exit+0x77/0xb0 > [ +0.000004] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? exc_page_fault+0x93/0x150 > [ +0.000010] entry_SYSCALL_64_after_hwframe+0x76/0x7e > [ +0.000005] RIP: 0033:0x7c45ff924ded > [ +0.000005] RSP: 002b:00007ffff7168790 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > [ +0.000008] RAX: ffffffffffffffda RBX: 00000000c0486456 RCX: 00007c45ff924ded > [ +0.000005] RDX: 00007ffff71687f0 RSI: 00000000c0486456 RDI: 000000000000000b > [ +0.000004] RBP: 00007ffff71687e0 R08: 00005b0c2a49b010 R09: 0000000000000007 > [ +0.000004] R10: 00005b0c2a4d7140 R11: 0000000000000246 R12: 000000000000000b > [ +0.000004] R13: 00007c45ff19e5cc R14: 00005b0c2a51c538 R15: 00005b0c2a51bbd8 > [ +0.000022] </TASK> > [ +0.000005] irq event stamp: 87419 > [ +0.000004] hardirqs last enabled at (87425): [<ffffffff86a693a9>] __up_console_sem+0x79/0xa0 > [ +0.000005] hardirqs last disabled at (87430): [<ffffffff86a6938e>] __up_console_sem+0x5e/0xa0 > [ +0.000005] softirqs last enabled at (87058): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000006] softirqs last disabled at (87053): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000005] ---[ end trace 0000000000000000 ]--- > > Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> The patch itself is Reviewed-by: Christian König <christian.koenig@amd.com> But somebody should work on removing the pinning alltogether. We now have the eviction fence for that instead. Regards, Christian. > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 13 +++++++++---- > 1 file changed, 9 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > index beae931152a3..7427e080b389 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > @@ -208,14 +208,14 @@ amdgpu_userqueue_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, > db_obj->obj = amdgpu_bo_ref(gem_to_amdgpu_bo(gobj)); > drm_gem_object_put(gobj); > > - /* Pin the BO before generating the index, unpin in queue destroy */ > - r = amdgpu_bo_pin(db_obj->obj, AMDGPU_GEM_DOMAIN_DOORBELL); > + r = amdgpu_bo_reserve(db_obj->obj, true); > if (r) { > DRM_ERROR("[Usermode queues] Failed to pin doorbell object\n"); > goto unref_bo; > } > > - r = amdgpu_bo_reserve(db_obj->obj, true); > + /* Pin the BO before generating the index, unpin in queue destroy */ > + r = amdgpu_bo_pin(db_obj->obj, AMDGPU_GEM_DOMAIN_DOORBELL); > if (r) { > DRM_ERROR("[Usermode queues] Failed to pin doorbell object\n"); > goto unpin_bo; > @@ -264,6 +264,7 @@ amdgpu_userqueue_destroy(struct drm_file *filp, int queue_id) > struct amdgpu_fpriv *fpriv = filp->driver_priv; > struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr; > struct amdgpu_usermode_queue *queue; > + int r; > > cancel_delayed_work(&uq_mgr->resume_work); > mutex_lock(&uq_mgr->userq_mutex); > @@ -275,7 +276,11 @@ amdgpu_userqueue_destroy(struct drm_file *filp, int queue_id) > return -EINVAL; > } > > - amdgpu_bo_unpin(queue->db_obj.obj); > + r = amdgpu_bo_reserve(queue->db_obj.obj, true); > + if (!r) { > + amdgpu_bo_unpin(queue->db_obj.obj); > + amdgpu_bo_unreserve(queue->db_obj.obj); > + } > amdgpu_bo_unref(&queue->db_obj.obj); > amdgpu_userqueue_cleanup(uq_mgr, queue, queue_id); > mutex_unlock(&uq_mgr->userq_mutex); ^ permalink raw reply [flat|nested] 11+ messages in thread
* RE: [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings 2025-05-02 12:42 ` Christian König @ 2025-05-12 1:36 ` Zhang, Jesse(Jie) 0 siblings, 0 replies; 11+ messages in thread From: Zhang, Jesse(Jie) @ 2025-05-12 1:36 UTC (permalink / raw) To: Koenig, Christian, Paneer Selvam, Arunpravin, amd-gfx@lists.freedesktop.org Cc: Deucher, Alexander [AMD Official Use Only - AMD Internal Distribution Only] Reviewed-by: Jesse Zhang <Jesse.Zhang@amd.com> -----Original Message----- From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Christian König Sent: Friday, May 2, 2025 8:43 PM To: Paneer Selvam, Arunpravin <Arunpravin.PaneerSelvam@amd.com>; amd-gfx@lists.freedesktop.org Cc: Deucher, Alexander <Alexander.Deucher@amd.com> Subject: Re: [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings On 4/9/25 07:48, Arunpravin Paneer Selvam wrote: > The ttm_bo_pin and ttm_bo_unpin warnings are resolved by moving the > doorbell bo reserve up before pin/unpin. > > WARNING: CPU: 11 PID: 1818 at drivers/gpu/drm/ttm/ttm_bo.c:592 ttm_bo_pin+0x1f6/0x270 [ttm] > [ +0.000277] CPU: 11 UID: 1000 PID: 1818 Comm: Xwayland Tainted: G W 6.12.0+ #15 > [ +0.000006] Tainted: [W]=WARN > [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING > B650-PLUS, BIOS 3072 12/20/2024 [ +0.000004] RIP: > 0010:ttm_bo_pin+0x1f6/0x270 [ttm] [ +0.000005] RSP: > 0018:ffff88846ca879d0 EFLAGS: 00010246 [ +0.000007] RAX: > 0000000000000000 RBX: ffff88810b7ca848 RCX: 0000000000000000 [ > +0.000004] RDX: 0000000000000000 RSI: 0000000000000000 RDI: > 0000000000000000 [ +0.000005] RBP: ffff88846ca879e8 R08: > 0000000000000000 R09: 0000000000000000 [ +0.000004] R10: > 0000000000000000 R11: 0000000000000000 R12: ffff88810b7ca848 [ > +0.000004] R13: ffff88846c666250 R14: 1ffff1108d950f44 R15: > ffff88846ca87aa0 [ +0.000005] FS: 00007c45ff436d00(0000) > GS:ffff888409580000(0000) knlGS:0000000000000000 [ +0.000004] CS: > 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.000005] CR2: 00005b0c142a60e0 CR3: 000000012ce5a000 CR4: 0000000000f50ef0 [ +0.000004] PKRU: 55555554 [ +0.000004] Call Trace: > [ +0.000004] <TASK> > [ +0.000005] ? show_regs+0x6c/0x80 > [ +0.000007] ? __warn+0xd2/0x2d0 > [ +0.000007] ? ttm_bo_pin+0x1f6/0x270 [ttm] [ +0.000031] ? > report_bug+0x282/0x2f0 [ +0.000012] ? handle_bug+0x6e/0xc0 [ > +0.000007] ? exc_invalid_op+0x18/0x50 [ +0.000007] ? > asm_exc_invalid_op+0x1b/0x20 [ +0.000017] ? ttm_bo_pin+0x1f6/0x270 > [ttm] [ +0.000014] amdgpu_bo_pin+0x365/0x9d0 [amdgpu] [ +0.000191] > ? __pfx_amdgpu_bo_pin+0x10/0x10 [amdgpu] [ +0.000185] ? > drm_gem_object_lookup+0x81/0xc0 [ +0.000008] ? > kasan_save_alloc_info+0x37/0x60 [ +0.000007] ? > __kasan_kmalloc+0xc3/0xd0 [ +0.000013] > amdgpu_userqueue_get_doorbell_index+0xee/0x5f0 [amdgpu] [ +0.000209] > amdgpu_userq_ioctl+0x6b4/0xd40 [amdgpu] [ +0.000193] ? > __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000211] ? > lock_acquire+0x7c/0xc0 [ +0.000006] ? drm_dev_enter+0x51/0x190 [ > +0.000015] drm_ioctl_kernel+0x18b/0x330 [ +0.000007] ? > __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000190] ? > __pfx_drm_ioctl_kernel+0x10/0x10 [ +0.000005] ? > lock_acquire+0x7c/0xc0 [ +0.000009] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? __kasan_check_write+0x14/0x30 [ +0.000005] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000011] drm_ioctl+0x589/0xd00 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ > +0.000194] ? __pfx_drm_ioctl+0x10/0x10 [ +0.000006] ? > __pm_runtime_resume+0x80/0x110 [ +0.000021] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? trace_hardirqs_on+0x53/0x60 [ +0.000005] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 > [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] [ +0.000185] > __x64_sys_ioctl+0x13a/0x1c0 [ +0.000010] x64_sys_call+0x11ad/0x25f0 > [ +0.000007] do_syscall_64+0x91/0x180 [ +0.000007] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? irqentry_exit+0x77/0xb0 [ +0.000005] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? exc_page_fault+0x93/0x150 [ +0.000009] > entry_SYSCALL_64_after_hwframe+0x76/0x7e > [ +0.000005] RIP: 0033:0x7c45ff924ded [ +0.000005] RSP: > 002b:00007ffff7167810 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ > +0.000008] RAX: ffffffffffffffda RBX: 00000000c0486456 RCX: > 00007c45ff924ded [ +0.000004] RDX: 00007ffff7167870 RSI: > 00000000c0486456 RDI: 000000000000000b [ +0.000004] RBP: > 00007ffff7167860 R08: ffff800100000000 R09: 0000000000010000 [ > +0.000005] R10: 00007ffff7167950 R11: 0000000000000246 R12: > 00005b0c2a51bc48 [ +0.000004] R13: 000000000000000b R14: > 0000000000000000 R15: 00007ffff7167950 [ +0.000022] </TASK> [ > +0.000004] irq event stamp: 80693 [ +0.000004] hardirqs last enabled > at (80699): [<ffffffff86a693a9>] __up_console_sem+0x79/0xa0 [ > +0.000005] hardirqs last disabled at (80704): [<ffffffff86a6938e>] > __up_console_sem+0x5e/0xa0 [ +0.000005] softirqs last enabled at > (80390): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000005] > softirqs last disabled at (80385): [<ffffffff8687377e>] > __irq_exit_rcu+0x17e/0x1d0 [ +0.000006] ---[ end trace > 0000000000000000 ]--- > ---------------------------------------------------------------------- > -------------------------------- > > [ +0.000006] WARNING: CPU: 10 PID: 1818 at drivers/gpu/drm/ttm/ttm_bo.c:611 ttm_bo_unpin+0x21f/0x2c0 [ttm] > [ +0.000280] CPU: 10 UID: 1000 PID: 1818 Comm: Xwayland Tainted: G W 6.12.0+ #15 > [ +0.000006] Tainted: [W]=WARN > [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING > B650-PLUS, BIOS 3072 12/20/2024 [ +0.000004] RIP: > 0010:ttm_bo_unpin+0x21f/0x2c0 [ttm] [ +0.000005] RSP: > 0018:ffff88846ca87888 EFLAGS: 00010246 [ +0.000007] RAX: > 0000000000000000 RBX: ffff88810b7ca848 RCX: 0000000000000000 [ > +0.000005] RDX: 0000000000000000 RSI: 0000000000000000 RDI: > 0000000000000000 [ +0.000004] RBP: ffff88846ca878a0 R08: > 0000000000000000 R09: 0000000000000000 [ +0.000004] R10: > 0000000000000000 R11: 0000000000000000 R12: ffff888164e90050 [ > +0.000005] R13: ffff88846c666200 R14: 0000000000000001 R15: > ffff888168402d28 [ +0.000004] FS: 00007c45ff436d00(0000) > GS:ffff888409500000(0000) knlGS:0000000000000000 [ +0.000005] CS: > 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.000004] CR2: 00007c45f7373b20 CR3: 000000012ce5a000 CR4: 0000000000f50ef0 [ +0.000005] PKRU: 55555554 [ +0.000004] Call Trace: > [ +0.000004] <TASK> > [ +0.000005] ? show_regs+0x6c/0x80 > [ +0.000008] ? __warn+0xd2/0x2d0 > [ +0.000007] ? ttm_bo_unpin+0x21f/0x2c0 [ttm] [ +0.000012] ? > report_bug+0x282/0x2f0 [ +0.000013] ? handle_bug+0x6e/0xc0 [ > +0.000006] ? exc_invalid_op+0x18/0x50 [ +0.000008] ? > asm_exc_invalid_op+0x1b/0x20 [ +0.000017] ? ttm_bo_unpin+0x21f/0x2c0 > [ttm] [ +0.000011] ? ttm_bo_unpin+0x217/0x2c0 [ttm] [ +0.000011] > amdgpu_bo_unpin+0x45/0x250 [amdgpu] [ +0.000216] > amdgpu_userq_ioctl+0x2c3/0xd40 [amdgpu] [ +0.000226] ? > drm_dev_exit+0x2d/0x60 [ +0.000010] ? > __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ +0.000201] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? lock_acquire+0x7c/0xc0 [ +0.000006] ? > drm_dev_enter+0x51/0x190 [ +0.000015] drm_ioctl_kernel+0x18b/0x330 [ > +0.000007] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ > +0.000188] ? __pfx_drm_ioctl_kernel+0x10/0x10 [ +0.000006] ? > lock_acquire+0x7c/0xc0 [ +0.000008] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? __kasan_check_write+0x14/0x30 [ +0.000006] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] drm_ioctl+0x589/0xd00 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __pfx_amdgpu_userq_ioctl+0x10/0x10 [amdgpu] [ > +0.000211] ? __pfx_drm_ioctl+0x10/0x10 [ +0.000006] ? > __pm_runtime_resume+0x80/0x110 [ +0.000020] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? trace_hardirqs_on+0x53/0x60 [ +0.000005] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 > [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] [ +0.000186] > __x64_sys_ioctl+0x13a/0x1c0 [ +0.000010] x64_sys_call+0x11ad/0x25f0 > [ +0.000007] do_syscall_64+0x91/0x180 [ +0.000007] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000007] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] ? __pfx___rseq_handle_notify_resume+0x10/0x10 > [ +0.000005] ? __pfx_blkcg_maybe_throttle_current+0x10/0x10 > [ +0.000013] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000007] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000011] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? irqentry_exit_to_user_mode+0x8b/0x260 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? irqentry_exit+0x77/0xb0 [ +0.000004] ? > srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? exc_page_fault+0x93/0x150 [ +0.000010] > entry_SYSCALL_64_after_hwframe+0x76/0x7e > [ +0.000005] RIP: 0033:0x7c45ff924ded [ +0.000005] RSP: > 002b:00007ffff7168790 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ > +0.000008] RAX: ffffffffffffffda RBX: 00000000c0486456 RCX: > 00007c45ff924ded [ +0.000005] RDX: 00007ffff71687f0 RSI: > 00000000c0486456 RDI: 000000000000000b [ +0.000004] RBP: > 00007ffff71687e0 R08: 00005b0c2a49b010 R09: 0000000000000007 [ > +0.000004] R10: 00005b0c2a4d7140 R11: 0000000000000246 R12: > 000000000000000b [ +0.000004] R13: 00007c45ff19e5cc R14: > 00005b0c2a51c538 R15: 00005b0c2a51bbd8 [ +0.000022] </TASK> [ > +0.000005] irq event stamp: 87419 [ +0.000004] hardirqs last enabled > at (87425): [<ffffffff86a693a9>] __up_console_sem+0x79/0xa0 [ > +0.000005] hardirqs last disabled at (87430): [<ffffffff86a6938e>] > __up_console_sem+0x5e/0xa0 [ +0.000005] softirqs last enabled at > (87058): [<ffffffff8687377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000006] > softirqs last disabled at (87053): [<ffffffff8687377e>] > __irq_exit_rcu+0x17e/0x1d0 [ +0.000005] ---[ end trace > 0000000000000000 ]--- > > Signed-off-by: Arunpravin Paneer Selvam > <Arunpravin.PaneerSelvam@amd.com> The patch itself is Reviewed-by: Christian König <christian.koenig@amd.com> But somebody should work on removing the pinning alltogether. We now have the eviction fence for that instead. Regards, Christian. > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c | 13 +++++++++---- > 1 file changed, 9 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > index beae931152a3..7427e080b389 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userqueue.c > @@ -208,14 +208,14 @@ amdgpu_userqueue_get_doorbell_index(struct amdgpu_userq_mgr *uq_mgr, > db_obj->obj = amdgpu_bo_ref(gem_to_amdgpu_bo(gobj)); > drm_gem_object_put(gobj); > > - /* Pin the BO before generating the index, unpin in queue destroy */ > - r = amdgpu_bo_pin(db_obj->obj, AMDGPU_GEM_DOMAIN_DOORBELL); > + r = amdgpu_bo_reserve(db_obj->obj, true); > if (r) { > DRM_ERROR("[Usermode queues] Failed to pin doorbell object\n"); > goto unref_bo; > } > > - r = amdgpu_bo_reserve(db_obj->obj, true); > + /* Pin the BO before generating the index, unpin in queue destroy */ > + r = amdgpu_bo_pin(db_obj->obj, AMDGPU_GEM_DOMAIN_DOORBELL); > if (r) { > DRM_ERROR("[Usermode queues] Failed to pin doorbell object\n"); > goto unpin_bo; > @@ -264,6 +264,7 @@ amdgpu_userqueue_destroy(struct drm_file *filp, int queue_id) > struct amdgpu_fpriv *fpriv = filp->driver_priv; > struct amdgpu_userq_mgr *uq_mgr = &fpriv->userq_mgr; > struct amdgpu_usermode_queue *queue; > + int r; > > cancel_delayed_work(&uq_mgr->resume_work); > mutex_lock(&uq_mgr->userq_mutex); > @@ -275,7 +276,11 @@ amdgpu_userqueue_destroy(struct drm_file *filp, int queue_id) > return -EINVAL; > } > > - amdgpu_bo_unpin(queue->db_obj.obj); > + r = amdgpu_bo_reserve(queue->db_obj.obj, true); > + if (!r) { > + amdgpu_bo_unpin(queue->db_obj.obj); > + amdgpu_bo_unreserve(queue->db_obj.obj); > + } > amdgpu_bo_unref(&queue->db_obj.obj); > amdgpu_userqueue_cleanup(uq_mgr, queue, queue_id); > mutex_unlock(&uq_mgr->userq_mutex); ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 4/5] drm/amdgpu/userq: Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) 2025-04-09 5:48 [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Arunpravin Paneer Selvam 2025-04-09 5:48 ` [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked Arunpravin Paneer Selvam 2025-04-09 5:48 ` [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings Arunpravin Paneer Selvam @ 2025-04-09 5:48 ` Arunpravin Paneer Selvam 2025-05-02 13:02 ` Christian König 2025-04-09 5:48 ` [PATCH 5/5] drm/amdgpu: Acquire BO locks only during BO processing Arunpravin Paneer Selvam 2025-05-02 12:30 ` [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Christian König 4 siblings, 1 reply; 11+ messages in thread From: Arunpravin Paneer Selvam @ 2025-04-09 5:48 UTC (permalink / raw) To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) warning logs. Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> --- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index cd6eb7a3bc58..1d7e2b85f45a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -1413,16 +1413,16 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv) mutex_init(&fpriv->bo_list_lock); idr_init_base(&fpriv->bo_list_handles, 1); + r = amdgpu_userq_mgr_init(&fpriv->userq_mgr, adev); + if (r) + DRM_WARN("Can't setup usermode queues, use legacy workload submission only\n"); + r = amdgpu_eviction_fence_init(&fpriv->evf_mgr); if (r) goto error_vm; amdgpu_ctx_mgr_init(&fpriv->ctx_mgr, adev); - r = amdgpu_userq_mgr_init(&fpriv->userq_mgr, adev); - if (r) - DRM_WARN("Can't setup usermode queues, use legacy workload submission only\n"); - file_priv->driver_priv = fpriv; goto out_suspend; -- 2.43.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 4/5] drm/amdgpu/userq: Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) 2025-04-09 5:48 ` [PATCH 4/5] drm/amdgpu/userq: Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) Arunpravin Paneer Selvam @ 2025-05-02 13:02 ` Christian König 0 siblings, 0 replies; 11+ messages in thread From: Christian König @ 2025-05-02 13:02 UTC (permalink / raw) To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher On 4/9/25 07:48, Arunpravin Paneer Selvam wrote: > Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) warning logs. > > Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> Acked-by: Christian König <christian.koenig@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > index cd6eb7a3bc58..1d7e2b85f45a 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c > @@ -1413,16 +1413,16 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv) > mutex_init(&fpriv->bo_list_lock); > idr_init_base(&fpriv->bo_list_handles, 1); > > + r = amdgpu_userq_mgr_init(&fpriv->userq_mgr, adev); > + if (r) > + DRM_WARN("Can't setup usermode queues, use legacy workload submission only\n"); > + > r = amdgpu_eviction_fence_init(&fpriv->evf_mgr); > if (r) > goto error_vm; > > amdgpu_ctx_mgr_init(&fpriv->ctx_mgr, adev); > > - r = amdgpu_userq_mgr_init(&fpriv->userq_mgr, adev); > - if (r) > - DRM_WARN("Can't setup usermode queues, use legacy workload submission only\n"); > - > file_priv->driver_priv = fpriv; > goto out_suspend; > ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 5/5] drm/amdgpu: Acquire BO locks only during BO processing 2025-04-09 5:48 [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Arunpravin Paneer Selvam ` (2 preceding siblings ...) 2025-04-09 5:48 ` [PATCH 4/5] drm/amdgpu/userq: Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) Arunpravin Paneer Selvam @ 2025-04-09 5:48 ` Arunpravin Paneer Selvam 2025-05-02 13:07 ` Christian König 2025-05-02 12:30 ` [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Christian König 4 siblings, 1 reply; 11+ messages in thread From: Arunpravin Paneer Selvam @ 2025-04-09 5:48 UTC (permalink / raw) To: amd-gfx; +Cc: christian.koenig, alexander.deucher, Arunpravin Paneer Selvam Fix lockdep warnings. [ +0.000024] WARNING: CPU: 10 PID: 1909 at drivers/gpu/drm/drm_syncobj.c:456 drm_syncobj_find_fence+0x58c/0x6e0 [ +0.000519] CPU: 10 UID: 1000 PID: 1909 Comm: gnome-shel:cs0 Tainted: G W OE 6.12.0+ #18 [ +0.000008] Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 3072 12/20/2024 [ +0.000004] RIP: 0010:drm_syncobj_find_fence+0x58c/0x6e0 [ +0.000006] RSP: 0018:ffff88846d9ef680 EFLAGS: 00010202 [ +0.000008] RAX: 0000000000000000 RBX: 0000000000001388 RCX: 0000000000000001 [ +0.000004] RDX: 1ffff1108f5ad1da RSI: 0000000000000001 RDI: ffff88847ad68ed0 [ +0.000005] RBP: ffff88846d9ef770 R08: 0000000000000000 R09: 0000000000000000 [ +0.000004] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88847ad68000 [ +0.000004] R13: 0000000000000002 R14: ffff888149353d00 R15: 000000000000000f [ +0.000005] FS: 00007269977fe6c0(0000) GS:ffff888409500000(0000) knlGS:0000000000000000 [ +0.000005] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ +0.000004] CR2: 0000643a866b50d0 CR3: 0000000469eb2000 CR4: 0000000000f50ef0 [ +0.000005] PKRU: 55555554 [ +0.000004] Call Trace: [ +0.000004] <TASK> [ +0.000005] ? show_regs+0x6c/0x80 [ +0.000010] ? __warn+0xd2/0x2d0 [ +0.000008] ? drm_syncobj_find_fence+0x58c/0x6e0 [ +0.000009] ? report_bug+0x282/0x2f0 [ +0.000014] ? handle_bug+0x6e/0xc0 [ +0.000008] ? exc_invalid_op+0x18/0x50 [ +0.000007] ? asm_exc_invalid_op+0x1b/0x20 [ +0.000020] ? drm_syncobj_find_fence+0x58c/0x6e0 [ +0.000012] ? __pfx_drm_syncobj_find_fence+0x10/0x10 [ +0.000012] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? lock_is_held_type+0xa3/0x130 [ +0.000016] amdgpu_userq_wait_ioctl+0x92d/0x2200 [amdgpu] [ +0.000257] ? amdgpu_userq_wait_ioctl+0x92d/0x2200 [amdgpu] [ +0.000180] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? __lock_acquire+0x1b19/0x69c0 [ +0.000022] ? __pfx_amdgpu_userq_wait_ioctl+0x10/0x10 [amdgpu] [ +0.000179] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? __kasan_check_read+0x11/0x20 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? mark_lock+0xfd/0x17c0 [ +0.000012] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? lock_acquire.part.0+0x116/0x360 [ +0.000006] ? drm_dev_enter+0x51/0x190 [ +0.000008] ? __pfx___lock_acquire+0x10/0x10 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? find_held_lock+0x36/0x140 [ +0.000021] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? lock_acquire+0x7c/0xc0 [ +0.000005] ? drm_dev_enter+0x51/0x190 [ +0.000027] drm_ioctl_kernel+0x18b/0x330 [ +0.000008] ? __pfx_amdgpu_userq_wait_ioctl+0x10/0x10 [amdgpu] [ +0.000181] ? __pfx_drm_ioctl_kernel+0x10/0x10 [ +0.000005] ? lock_acquire+0x7c/0xc0 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? __kasan_check_write+0x14/0x30 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000010] drm_ioctl+0x589/0xd00 [ +0.000006] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000006] ? __pfx_amdgpu_userq_wait_ioctl+0x10/0x10 [amdgpu] [ +0.000187] ? __pfx_drm_ioctl+0x10/0x10 [ +0.000007] ? __pm_runtime_resume+0x80/0x110 [ +0.000021] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? trace_hardirqs_on+0x53/0x60 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] [ +0.000182] __x64_sys_ioctl+0x13a/0x1c0 [ +0.000012] x64_sys_call+0x11ad/0x25f0 [ +0.000007] do_syscall_64+0x91/0x180 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000009] ? __pfx___do_sys_prctl+0x10/0x10 [ +0.000012] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? do_syscall_64+0x9d/0x180 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? irqentry_exit_to_user_mode+0x8b/0x260 [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? irqentry_exit+0x77/0xb0 [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 [ +0.000005] ? exc_page_fault+0x93/0x150 [ +0.000009] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ +0.000005] RIP: 0033:0x7269b0324ded [ +0.000006] Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 [ +0.000005] RSP: 002b:00007269977fc9b0 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ +0.000008] RAX: ffffffffffffffda RBX: 00007269977fcb00 RCX: 00007269b0324ded [ +0.000004] RDX: 00007269977fcb00 RSI: 00000000c0406458 RDI: 000000000000000d [ +0.000004] RBP: 00007269977fca00 R08: 00007269977fcc1c R09: 0000000000000000 [ +0.000005] R10: 000000000000000f R11: 0000000000000246 R12: 000000000000000d [ +0.000004] R13: 00005bce7c309a50 R14: 00007269977fca30 R15: 0000000000000000 [ +0.000021] </TASK> [ +0.000005] irq event stamp: 1359 [ +0.000004] hardirqs last enabled at (1365): [<ffffffffaa8693a9>] __up_console_sem+0x79/0xa0 [ +0.000007] hardirqs last disabled at (1370): [<ffffffffaa86938e>] __up_console_sem+0x5e/0xa0 [ +0.000005] softirqs last enabled at (756): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000006] softirqs last disabled at (749): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0 [ +0.000005] ---[ end trace 0000000000000000 ]--- Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> --- .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 116 +++++++++++------- 1 file changed, 69 insertions(+), 47 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c index d5b35b5df527..c5de39a8ff98 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c @@ -395,6 +395,43 @@ static void amdgpu_userq_fence_cleanup(struct dma_fence *fence) dma_fence_put(fence); } +static int amdgpu_userq_exec_lock(struct drm_exec *exec, u32 flags, + struct drm_gem_object **rgobj, + struct drm_gem_object **wgobj, + u32 num_read_handles, + u32 num_write_handles, + unsigned int num_fences) +{ + int r; + + if (!exec | !rgobj | !wgobj) + return -EINVAL; + + drm_exec_init(exec, flags, + (num_read_handles + num_write_handles)); + + /* Lock all BOs with retry handling */ + drm_exec_until_all_locked(exec) { + r = drm_exec_prepare_array(exec, rgobj, num_read_handles, num_fences); + drm_exec_retry_on_contention(exec); + if (r) + drm_exec_fini(exec); + + r = drm_exec_prepare_array(exec, wgobj, num_write_handles, num_fences); + drm_exec_retry_on_contention(exec); + if (r) + drm_exec_fini(exec); + } + + return r; +} + +static void amdgpu_userq_exec_unlock(struct drm_exec *exec) +{ + if (exec) + drm_exec_fini(exec); +} + int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) { @@ -511,24 +548,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, queue->last_fence = dma_fence_get(fence); mutex_unlock(&userq_mgr->userq_mutex); - drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, - (num_read_bo_handles + num_write_bo_handles)); - - /* Lock all BOs with retry handling */ - drm_exec_until_all_locked(&exec) { - r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1); - drm_exec_retry_on_contention(&exec); - if (r) { - amdgpu_userq_fence_cleanup(fence); - goto exec_fini; - } - - r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1); - drm_exec_retry_on_contention(&exec); - if (r) { - amdgpu_userq_fence_cleanup(fence); - goto exec_fini; - } + r = amdgpu_userq_exec_lock(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, + gobj_read, gobj_write, + num_read_bo_handles, + num_write_bo_handles, + 1); + if (r) { + amdgpu_userq_fence_cleanup(fence); + goto put_gobj_write; } for (i = 0; i < num_read_bo_handles; i++) { @@ -546,6 +573,7 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, dma_resv_add_fence(gobj_write[i]->resv, fence, DMA_RESV_USAGE_WRITE); } + amdgpu_userq_exec_unlock(&exec); /* Add the created fence to syncobj/BO's */ for (i = 0; i < num_syncobj_handles; i++) @@ -554,8 +582,6 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, /* drop the reference acquired in fence creation function */ dma_fence_put(fence); -exec_fini: - drm_exec_fini(&exec); put_gobj_write: while (wentry-- > 0) drm_gem_object_put(gobj_write[wentry]); @@ -666,26 +692,6 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, } } - drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, - (num_read_bo_handles + num_write_bo_handles)); - - /* Lock all BOs with retry handling */ - drm_exec_until_all_locked(&exec) { - r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1); - drm_exec_retry_on_contention(&exec); - if (r) { - drm_exec_fini(&exec); - goto put_gobj_write; - } - - r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1); - drm_exec_retry_on_contention(&exec); - if (r) { - drm_exec_fini(&exec); - goto put_gobj_write; - } - } - if (!wait_info->num_fences) { if (num_points) { struct dma_fence_unwrap iter; @@ -698,7 +704,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT, &fence); if (r) - goto exec_fini; + goto put_gobj_write; dma_fence_unwrap_for_each(f, &iter, fence) num_fences++; @@ -716,12 +722,20 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT, &fence); if (r) - goto exec_fini; + goto put_gobj_write; num_fences++; dma_fence_put(fence); } + r = amdgpu_userq_exec_lock(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, + gobj_read, gobj_write, + num_read_bo_handles, + num_write_bo_handles, + 1); + if (r) + goto put_gobj_write; + /* Count GEM objects fence */ for (i = 0; i < num_read_bo_handles; i++) { struct dma_resv_iter resv_cursor; @@ -740,7 +754,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, DMA_RESV_USAGE_WRITE, fence) num_fences++; } - + amdgpu_userq_exec_unlock(&exec); /* * Passing num_fences = 0 means that userspace doesn't want to * retrieve userq_fence_info. If num_fences = 0 we skip filling @@ -753,7 +767,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, fence_info = kmalloc_array(wait_info->num_fences, sizeof(*fence_info), GFP_KERNEL); if (!fence_info) { r = -ENOMEM; - goto exec_fini; + goto put_gobj_write; } /* Array of fences */ @@ -763,6 +777,14 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, goto free_fence_info; } + r = amdgpu_userq_exec_lock(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, + gobj_read, gobj_write, + num_read_bo_handles, + num_write_bo_handles, + 1); + if (r) + goto free_fences; + /* Retrieve GEM read objects fence */ for (i = 0; i < num_read_bo_handles; i++) { struct dma_resv_iter resv_cursor; @@ -772,6 +794,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, DMA_RESV_USAGE_READ, fence) { if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { r = -EINVAL; + amdgpu_userq_exec_unlock(&exec); goto free_fences; } @@ -789,6 +812,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, DMA_RESV_USAGE_WRITE, fence) { if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { r = -EINVAL; + amdgpu_userq_exec_unlock(&exec); goto free_fences; } @@ -796,6 +820,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, dma_fence_get(fence); } } + amdgpu_userq_exec_unlock(&exec); if (num_points) { struct dma_fence_unwrap iter; @@ -901,7 +926,6 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, kfree(fence_info); } - drm_exec_fini(&exec); for (i = 0; i < num_read_bo_handles; i++) drm_gem_object_put(gobj_read[i]); kfree(gobj_read); @@ -924,8 +948,6 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, kfree(fences); free_fence_info: kfree(fence_info); -exec_fini: - drm_exec_fini(&exec); put_gobj_write: while (wentry-- > 0) drm_gem_object_put(gobj_write[wentry]); -- 2.43.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 5/5] drm/amdgpu: Acquire BO locks only during BO processing 2025-04-09 5:48 ` [PATCH 5/5] drm/amdgpu: Acquire BO locks only during BO processing Arunpravin Paneer Selvam @ 2025-05-02 13:07 ` Christian König 0 siblings, 0 replies; 11+ messages in thread From: Christian König @ 2025-05-02 13:07 UTC (permalink / raw) To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher On 4/9/25 07:48, Arunpravin Paneer Selvam wrote: > Fix lockdep warnings. > > [ +0.000024] WARNING: CPU: 10 PID: 1909 at drivers/gpu/drm/drm_syncobj.c:456 drm_syncobj_find_fence+0x58c/0x6e0 > [ +0.000519] CPU: 10 UID: 1000 PID: 1909 Comm: gnome-shel:cs0 Tainted: G W OE 6.12.0+ #18 > [ +0.000008] Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE > [ +0.000004] Hardware name: ASUS System Product Name/TUF GAMING B650-PLUS, BIOS 3072 12/20/2024 > [ +0.000004] RIP: 0010:drm_syncobj_find_fence+0x58c/0x6e0 > [ +0.000006] RSP: 0018:ffff88846d9ef680 EFLAGS: 00010202 > [ +0.000008] RAX: 0000000000000000 RBX: 0000000000001388 RCX: 0000000000000001 > [ +0.000004] RDX: 1ffff1108f5ad1da RSI: 0000000000000001 RDI: ffff88847ad68ed0 > [ +0.000005] RBP: ffff88846d9ef770 R08: 0000000000000000 R09: 0000000000000000 > [ +0.000004] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88847ad68000 > [ +0.000004] R13: 0000000000000002 R14: ffff888149353d00 R15: 000000000000000f > [ +0.000005] FS: 00007269977fe6c0(0000) GS:ffff888409500000(0000) knlGS:0000000000000000 > [ +0.000005] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ +0.000004] CR2: 0000643a866b50d0 CR3: 0000000469eb2000 CR4: 0000000000f50ef0 > [ +0.000005] PKRU: 55555554 > [ +0.000004] Call Trace: > [ +0.000004] <TASK> > [ +0.000005] ? show_regs+0x6c/0x80 > [ +0.000010] ? __warn+0xd2/0x2d0 > [ +0.000008] ? drm_syncobj_find_fence+0x58c/0x6e0 > [ +0.000009] ? report_bug+0x282/0x2f0 > [ +0.000014] ? handle_bug+0x6e/0xc0 > [ +0.000008] ? exc_invalid_op+0x18/0x50 > [ +0.000007] ? asm_exc_invalid_op+0x1b/0x20 > [ +0.000020] ? drm_syncobj_find_fence+0x58c/0x6e0 > [ +0.000012] ? __pfx_drm_syncobj_find_fence+0x10/0x10 > [ +0.000012] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? lock_is_held_type+0xa3/0x130 > [ +0.000016] amdgpu_userq_wait_ioctl+0x92d/0x2200 [amdgpu] > [ +0.000257] ? amdgpu_userq_wait_ioctl+0x92d/0x2200 [amdgpu] > [ +0.000180] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __lock_acquire+0x1b19/0x69c0 > [ +0.000022] ? __pfx_amdgpu_userq_wait_ioctl+0x10/0x10 [amdgpu] > [ +0.000179] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __kasan_check_read+0x11/0x20 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? mark_lock+0xfd/0x17c0 > [ +0.000012] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? lock_acquire.part.0+0x116/0x360 > [ +0.000006] ? drm_dev_enter+0x51/0x190 > [ +0.000008] ? __pfx___lock_acquire+0x10/0x10 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? find_held_lock+0x36/0x140 > [ +0.000021] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? lock_acquire+0x7c/0xc0 > [ +0.000005] ? drm_dev_enter+0x51/0x190 > [ +0.000027] drm_ioctl_kernel+0x18b/0x330 > [ +0.000008] ? __pfx_amdgpu_userq_wait_ioctl+0x10/0x10 [amdgpu] > [ +0.000181] ? __pfx_drm_ioctl_kernel+0x10/0x10 > [ +0.000005] ? lock_acquire+0x7c/0xc0 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? __kasan_check_write+0x14/0x30 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000010] drm_ioctl+0x589/0xd00 > [ +0.000006] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000006] ? __pfx_amdgpu_userq_wait_ioctl+0x10/0x10 [amdgpu] > [ +0.000187] ? __pfx_drm_ioctl+0x10/0x10 > [ +0.000007] ? __pm_runtime_resume+0x80/0x110 > [ +0.000021] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? trace_hardirqs_on+0x53/0x60 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? _raw_spin_unlock_irqrestore+0x51/0x80 > [ +0.000013] amdgpu_drm_ioctl+0xd2/0x1c0 [amdgpu] > [ +0.000182] __x64_sys_ioctl+0x13a/0x1c0 > [ +0.000012] x64_sys_call+0x11ad/0x25f0 > [ +0.000007] do_syscall_64+0x91/0x180 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 > [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 > [ +0.000009] ? __pfx___do_sys_prctl+0x10/0x10 > [ +0.000012] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? syscall_exit_to_user_mode+0x95/0x260 > [ +0.000008] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? do_syscall_64+0x9d/0x180 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000009] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? irqentry_exit_to_user_mode+0x8b/0x260 > [ +0.000007] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? irqentry_exit+0x77/0xb0 > [ +0.000005] ? srso_alias_return_thunk+0x5/0xfbef5 > [ +0.000005] ? exc_page_fault+0x93/0x150 > [ +0.000009] entry_SYSCALL_64_after_hwframe+0x76/0x7e > [ +0.000005] RIP: 0033:0x7269b0324ded > [ +0.000006] Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 > [ +0.000005] RSP: 002b:00007269977fc9b0 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 > [ +0.000008] RAX: ffffffffffffffda RBX: 00007269977fcb00 RCX: 00007269b0324ded > [ +0.000004] RDX: 00007269977fcb00 RSI: 00000000c0406458 RDI: 000000000000000d > [ +0.000004] RBP: 00007269977fca00 R08: 00007269977fcc1c R09: 0000000000000000 > [ +0.000005] R10: 000000000000000f R11: 0000000000000246 R12: 000000000000000d > [ +0.000004] R13: 00005bce7c309a50 R14: 00007269977fca30 R15: 0000000000000000 > [ +0.000021] </TASK> > [ +0.000005] irq event stamp: 1359 > [ +0.000004] hardirqs last enabled at (1365): [<ffffffffaa8693a9>] __up_console_sem+0x79/0xa0 > [ +0.000007] hardirqs last disabled at (1370): [<ffffffffaa86938e>] __up_console_sem+0x5e/0xa0 > [ +0.000005] softirqs last enabled at (756): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000006] softirqs last disabled at (749): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000005] ---[ end trace 0000000000000000 ]--- > > Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> > --- > .../gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 116 +++++++++++------- > 1 file changed, 69 insertions(+), 47 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > index d5b35b5df527..c5de39a8ff98 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > @@ -395,6 +395,43 @@ static void amdgpu_userq_fence_cleanup(struct dma_fence *fence) > dma_fence_put(fence); > } > > +static int amdgpu_userq_exec_lock(struct drm_exec *exec, u32 flags, > + struct drm_gem_object **rgobj, > + struct drm_gem_object **wgobj, > + u32 num_read_handles, > + u32 num_write_handles, > + unsigned int num_fences) num_fences should always be 1 if I'm not completely mistaken. > +{ > + int r; > + > + if (!exec | !rgobj | !wgobj) > + return -EINVAL; > + > + drm_exec_init(exec, flags, > + (num_read_handles + num_write_handles)); > + > + /* Lock all BOs with retry handling */ > + drm_exec_until_all_locked(exec) { > + r = drm_exec_prepare_array(exec, rgobj, num_read_handles, num_fences); > + drm_exec_retry_on_contention(exec); > + if (r) > + drm_exec_fini(exec); Don't call drm_exec_fini() here. Rather just return the error. > + > + r = drm_exec_prepare_array(exec, wgobj, num_write_handles, num_fences); > + drm_exec_retry_on_contention(exec); > + if (r) > + drm_exec_fini(exec); Same here. > + } > + > + return r; > +} > + > +static void amdgpu_userq_exec_unlock(struct drm_exec *exec) > +{ > + if (exec) > + drm_exec_fini(exec); > +} > + That looks superflous. Just call drm_exec_fini() in the caller. Regards, Christian. > int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, > struct drm_file *filp) > { > @@ -511,24 +548,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, > queue->last_fence = dma_fence_get(fence); > mutex_unlock(&userq_mgr->userq_mutex); > > - drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, > - (num_read_bo_handles + num_write_bo_handles)); > - > - /* Lock all BOs with retry handling */ > - drm_exec_until_all_locked(&exec) { > - r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1); > - drm_exec_retry_on_contention(&exec); > - if (r) { > - amdgpu_userq_fence_cleanup(fence); > - goto exec_fini; > - } > - > - r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1); > - drm_exec_retry_on_contention(&exec); > - if (r) { > - amdgpu_userq_fence_cleanup(fence); > - goto exec_fini; > - } > + r = amdgpu_userq_exec_lock(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, > + gobj_read, gobj_write, > + num_read_bo_handles, > + num_write_bo_handles, > + 1); > + if (r) { > + amdgpu_userq_fence_cleanup(fence); > + goto put_gobj_write; > } > > for (i = 0; i < num_read_bo_handles; i++) { > @@ -546,6 +573,7 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, > dma_resv_add_fence(gobj_write[i]->resv, fence, > DMA_RESV_USAGE_WRITE); > } > + amdgpu_userq_exec_unlock(&exec); > > /* Add the created fence to syncobj/BO's */ > for (i = 0; i < num_syncobj_handles; i++) > @@ -554,8 +582,6 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data, > /* drop the reference acquired in fence creation function */ > dma_fence_put(fence); > > -exec_fini: > - drm_exec_fini(&exec); > put_gobj_write: > while (wentry-- > 0) > drm_gem_object_put(gobj_write[wentry]); > @@ -666,26 +692,6 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > } > } > > - drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, > - (num_read_bo_handles + num_write_bo_handles)); > - > - /* Lock all BOs with retry handling */ > - drm_exec_until_all_locked(&exec) { > - r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1); > - drm_exec_retry_on_contention(&exec); > - if (r) { > - drm_exec_fini(&exec); > - goto put_gobj_write; > - } > - > - r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1); > - drm_exec_retry_on_contention(&exec); > - if (r) { > - drm_exec_fini(&exec); > - goto put_gobj_write; > - } > - } > - > if (!wait_info->num_fences) { > if (num_points) { > struct dma_fence_unwrap iter; > @@ -698,7 +704,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT, > &fence); > if (r) > - goto exec_fini; > + goto put_gobj_write; > > dma_fence_unwrap_for_each(f, &iter, fence) > num_fences++; > @@ -716,12 +722,20 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT, > &fence); > if (r) > - goto exec_fini; > + goto put_gobj_write; > > num_fences++; > dma_fence_put(fence); > } > > + r = amdgpu_userq_exec_lock(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, > + gobj_read, gobj_write, > + num_read_bo_handles, > + num_write_bo_handles, > + 1); > + if (r) > + goto put_gobj_write; > + > /* Count GEM objects fence */ > for (i = 0; i < num_read_bo_handles; i++) { > struct dma_resv_iter resv_cursor; > @@ -740,7 +754,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > DMA_RESV_USAGE_WRITE, fence) > num_fences++; > } > - > + amdgpu_userq_exec_unlock(&exec); > /* > * Passing num_fences = 0 means that userspace doesn't want to > * retrieve userq_fence_info. If num_fences = 0 we skip filling > @@ -753,7 +767,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > fence_info = kmalloc_array(wait_info->num_fences, sizeof(*fence_info), GFP_KERNEL); > if (!fence_info) { > r = -ENOMEM; > - goto exec_fini; > + goto put_gobj_write; > } > > /* Array of fences */ > @@ -763,6 +777,14 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > goto free_fence_info; > } > > + r = amdgpu_userq_exec_lock(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT, > + gobj_read, gobj_write, > + num_read_bo_handles, > + num_write_bo_handles, > + 1); > + if (r) > + goto free_fences; > + > /* Retrieve GEM read objects fence */ > for (i = 0; i < num_read_bo_handles; i++) { > struct dma_resv_iter resv_cursor; > @@ -772,6 +794,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > DMA_RESV_USAGE_READ, fence) { > if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { > r = -EINVAL; > + amdgpu_userq_exec_unlock(&exec); > goto free_fences; > } > > @@ -789,6 +812,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > DMA_RESV_USAGE_WRITE, fence) { > if (WARN_ON_ONCE(num_fences >= wait_info->num_fences)) { > r = -EINVAL; > + amdgpu_userq_exec_unlock(&exec); > goto free_fences; > } > > @@ -796,6 +820,7 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > dma_fence_get(fence); > } > } > + amdgpu_userq_exec_unlock(&exec); > > if (num_points) { > struct dma_fence_unwrap iter; > @@ -901,7 +926,6 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > kfree(fence_info); > } > > - drm_exec_fini(&exec); > for (i = 0; i < num_read_bo_handles; i++) > drm_gem_object_put(gobj_read[i]); > kfree(gobj_read); > @@ -924,8 +948,6 @@ int amdgpu_userq_wait_ioctl(struct drm_device *dev, void *data, > kfree(fences); > free_fence_info: > kfree(fence_info); > -exec_fini: > - drm_exec_fini(&exec); > put_gobj_write: > while (wentry-- > 0) > drm_gem_object_put(gobj_write[wentry]); ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence 2025-04-09 5:48 [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Arunpravin Paneer Selvam ` (3 preceding siblings ...) 2025-04-09 5:48 ` [PATCH 5/5] drm/amdgpu: Acquire BO locks only during BO processing Arunpravin Paneer Selvam @ 2025-05-02 12:30 ` Christian König 4 siblings, 0 replies; 11+ messages in thread From: Christian König @ 2025-05-02 12:30 UTC (permalink / raw) To: Arunpravin Paneer Selvam, amd-gfx; +Cc: alexander.deucher On 4/9/25 07:48, Arunpravin Paneer Selvam wrote: > Fix lockdep warnings. > > [ +0.000637] ================================ > [ +0.000004] WARNING: inconsistent lock state > [ +0.000004] 6.12.0+ #18 Tainted: G W OE > [ +0.000004] -------------------------------- > [ +0.000004] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage. > [ +0.000004] Xwayland/1952 [HC0[0]:SC0[0]:HE1:SE1] takes: > [ +0.000005] ffff8884636f4740 (&fence_drv->fence_list_lock){?...}-{2:2}, at: amdgpu_userq_fence_driver_destroy+0xb8/0x540 [amdgpu] > [ +0.000208] {IN-HARDIRQ-W} state was registered at: > [ +0.000004] lock_acquire.part.0+0x116/0x360 > [ +0.000005] lock_acquire+0x7c/0xc0 > [ +0.000005] _raw_spin_lock+0x2f/0x60 > [ +0.000005] amdgpu_userq_fence_driver_process+0x75/0x400 [amdgpu] > [ +0.000185] gfx_v12_0_eop_irq+0x29f/0x420 [amdgpu] > [ +0.000210] amdgpu_irq_dispatch+0x2a4/0x7b0 [amdgpu] > [ +0.000191] amdgpu_ih_process+0x1e1/0x3d0 [amdgpu] > [ +0.000185] amdgpu_irq_handler+0x28/0xc0 [amdgpu] > [ +0.000183] __handle_irq_event_percpu+0x1bb/0x590 > [ +0.000005] handle_irq_event+0xab/0x1d0 > [ +0.000005] handle_edge_irq+0x1fd/0xc10 > [ +0.000005] __common_interrupt+0x83/0x190 > [ +0.000004] common_interrupt+0xb1/0xe0 > [ +0.000005] asm_common_interrupt+0x27/0x40 > [ +0.000004] cpuidle_enter_state+0x2ba/0x530 > [ +0.000005] cpuidle_enter+0x4f/0xb0 > [ +0.000006] call_cpuidle+0x46/0xd0 > [ +0.000005] do_idle+0x367/0x430 > [ +0.000004] cpu_startup_entry+0x58/0x70 > [ +0.000005] start_secondary+0x224/0x2b0 > [ +0.000005] common_startup_64+0x13e/0x141 > [ +0.000005] irq event stamp: 88271 > [ +0.000004] hardirqs last enabled at (88271): [<ffffffffad9ca7a1>] _raw_spin_unlock_irqrestore+0x51/0x80 > [ +0.000005] hardirqs last disabled at (88270): [<ffffffffad9ca424>] _raw_spin_lock_irqsave+0x74/0x80 > [ +0.000005] softirqs last enabled at (87858): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000005] softirqs last disabled at (87849): [<ffffffffaa67377e>] __irq_exit_rcu+0x17e/0x1d0 > [ +0.000005] > other info that might help us debug this: > [ +0.000004] Possible unsafe locking scenario: > > [ +0.000003] CPU0 > [ +0.000004] ---- > [ +0.000003] lock(&fence_drv->fence_list_lock); > > Signed-off-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > index a4953d668972..24d19b920100 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c > @@ -159,10 +159,11 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref) > struct amdgpu_device *adev = fence_drv->adev; > struct amdgpu_userq_fence *fence, *tmp; > struct xarray *xa = &adev->userq_xa; > + unsigned long fence_list_flags; Drop that. > unsigned long index, flags; > struct dma_fence *f; > > - spin_lock(&fence_drv->fence_list_lock); > + spin_lock_irqsave(&fence_drv->fence_list_lock, fence_list_flags); And just use flags here. xa_lock_irqsave() also uses the flags parameter to save the interrupt flags. With that done the patch is Reviewed-by: Christian König <christian.koenig@amd.com> Regards, Christian. > list_for_each_entry_safe(fence, tmp, &fence_drv->fences, link) { > f = &fence->base; > > @@ -174,7 +175,7 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref) > list_del(&fence->link); > dma_fence_put(f); > } > - spin_unlock(&fence_drv->fence_list_lock); > + spin_unlock_irqrestore(&fence_drv->fence_list_lock, fence_list_flags); > > xa_lock_irqsave(xa, flags); > xa_for_each(xa, index, xa_fence_drv) ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2025-05-12 1:36 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-04-09 5:48 [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Arunpravin Paneer Selvam 2025-04-09 5:48 ` [PATCH 2/5] drm/amdgpu/userq: Add lock before accessing dma_fence_is_signaled_locked Arunpravin Paneer Selvam 2025-05-02 12:35 ` Christian König 2025-04-09 5:48 ` [PATCH 3/5] drm/amdgpu: Fix userq ttm_bo_pin and ttm_bo_unpin lockdep warnings Arunpravin Paneer Selvam 2025-05-02 12:42 ` Christian König 2025-05-12 1:36 ` Zhang, Jesse(Jie) 2025-04-09 5:48 ` [PATCH 4/5] drm/amdgpu/userq: Fix DEBUG_LOCKS_WARN_ON(lock->magic != lock) Arunpravin Paneer Selvam 2025-05-02 13:02 ` Christian König 2025-04-09 5:48 ` [PATCH 5/5] drm/amdgpu: Acquire BO locks only during BO processing Arunpravin Paneer Selvam 2025-05-02 13:07 ` Christian König 2025-05-02 12:30 ` [PATCH 1/5] drm/amdgpu/userq: Fix lock contention in userq fence Christian König
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.