* [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency
[not found] ` <1559616755-13116-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
@ 2019-06-04 2:52 ` Zeng, Oak
0 siblings, 0 replies; 8+ messages in thread
From: Zeng, Oak @ 2019-06-04 2:52 UTC (permalink / raw)
To: amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Kuehling, Felix, Zeng, Oak
The idea to break the circular lock dependency is to move allocate_mqd
out of dqm lock protection. See callstack #1 below.
[ 59.510149] [drm] Initialized amdgpu 3.30.0 20150101 for 0000:04:00.0 on minor 0
[ 513.604034] ======================================================
[ 513.604205] WARNING: possible circular locking dependency detected
[ 513.604375] 4.18.0-kfd-root #2 Tainted: G W
[ 513.604530] ------------------------------------------------------
[ 513.604699] kswapd0/611 is trying to acquire lock:
[ 513.604840] 00000000d254022e (&dqm->lock_hidden){+.+.}, at: evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.605150]
but task is already holding lock:
[ 513.605307] 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
[ 513.605540]
which lock already depends on the new lock.
[ 513.605747]
the existing dependency chain (in reverse order) is:
[ 513.605944]
-> #4 (&anon_vma->rwsem){++++}:
[ 513.606106] __vma_adjust+0x147/0x7f0
[ 513.606231] __split_vma+0x179/0x190
[ 513.606353] mprotect_fixup+0x217/0x260
[ 513.606553] do_mprotect_pkey+0x211/0x380
[ 513.606752] __x64_sys_mprotect+0x1b/0x20
[ 513.606954] do_syscall_64+0x50/0x1a0
[ 513.607149] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.607380]
-> #3 (&mapping->i_mmap_rwsem){++++}:
[ 513.607678] rmap_walk_file+0x1f0/0x280
[ 513.607887] page_referenced+0xdd/0x180
[ 513.608081] shrink_page_list+0x853/0xcb0
[ 513.608279] shrink_inactive_list+0x33b/0x700
[ 513.608483] shrink_node_memcg+0x37a/0x7f0
[ 513.608682] shrink_node+0xd8/0x490
[ 513.608869] balance_pgdat+0x18b/0x3b0
[ 513.609062] kswapd+0x203/0x5c0
[ 513.609241] kthread+0x100/0x140
[ 513.609420] ret_from_fork+0x24/0x30
[ 513.609607]
-> #2 (fs_reclaim){+.+.}:
[ 513.609883] kmem_cache_alloc_trace+0x34/0x2e0
[ 513.610093] reservation_object_reserve_shared+0x139/0x300
[ 513.610326] ttm_bo_init_reserved+0x291/0x480 [ttm]
[ 513.610567] amdgpu_bo_do_create+0x1d2/0x650 [amdgpu]
[ 513.610811] amdgpu_bo_create+0x40/0x1f0 [amdgpu]
[ 513.611041] amdgpu_bo_create_reserved+0x249/0x2d0 [amdgpu]
[ 513.611290] amdgpu_bo_create_kernel+0x12/0x70 [amdgpu]
[ 513.611584] amdgpu_ttm_init+0x2cb/0x560 [amdgpu]
[ 513.611823] gmc_v9_0_sw_init+0x400/0x750 [amdgpu]
[ 513.612491] amdgpu_device_init+0x14eb/0x1990 [amdgpu]
[ 513.612730] amdgpu_driver_load_kms+0x78/0x290 [amdgpu]
[ 513.612958] drm_dev_register+0x111/0x1a0
[ 513.613171] amdgpu_pci_probe+0x11c/0x1e0 [amdgpu]
[ 513.613389] local_pci_probe+0x3f/0x90
[ 513.613581] pci_device_probe+0x102/0x1c0
[ 513.613779] driver_probe_device+0x2a7/0x480
[ 513.613984] __driver_attach+0x10a/0x110
[ 513.614179] bus_for_each_dev+0x67/0xc0
[ 513.614372] bus_add_driver+0x1eb/0x260
[ 513.614565] driver_register+0x5b/0xe0
[ 513.614756] do_one_initcall+0xac/0x357
[ 513.614952] do_init_module+0x5b/0x213
[ 513.615145] load_module+0x2542/0x2d30
[ 513.615337] __do_sys_finit_module+0xd2/0x100
[ 513.615541] do_syscall_64+0x50/0x1a0
[ 513.615731] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.615963]
-> #1 (reservation_ww_class_mutex){+.+.}:
[ 513.616293] amdgpu_amdkfd_alloc_gtt_mem+0xcf/0x2c0 [amdgpu]
[ 513.616554] init_mqd+0x223/0x260 [amdgpu]
[ 513.616779] create_queue_nocpsch+0x4d9/0x600 [amdgpu]
[ 513.617031] pqm_create_queue+0x37c/0x520 [amdgpu]
[ 513.617270] kfd_ioctl_create_queue+0x2f9/0x650 [amdgpu]
[ 513.617522] kfd_ioctl+0x202/0x350 [amdgpu]
[ 513.617724] do_vfs_ioctl+0x9f/0x6c0
[ 513.617914] ksys_ioctl+0x66/0x70
[ 513.618095] __x64_sys_ioctl+0x16/0x20
[ 513.618286] do_syscall_64+0x50/0x1a0
[ 513.618476] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.618695]
-> #0 (&dqm->lock_hidden){+.+.}:
[ 513.618984] __mutex_lock+0x98/0x970
[ 513.619197] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.619459] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
[ 513.619710] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
[ 513.620103] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
[ 513.620363] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
[ 513.620614] __mmu_notifier_invalidate_range_start+0x70/0xb0
[ 513.620851] try_to_unmap_one+0x7fc/0x8f0
[ 513.621049] rmap_walk_anon+0x121/0x290
[ 513.621242] try_to_unmap+0x93/0xf0
[ 513.621428] shrink_page_list+0x606/0xcb0
[ 513.621625] shrink_inactive_list+0x33b/0x700
[ 513.621835] shrink_node_memcg+0x37a/0x7f0
[ 513.622034] shrink_node+0xd8/0x490
[ 513.622219] balance_pgdat+0x18b/0x3b0
[ 513.622410] kswapd+0x203/0x5c0
[ 513.622589] kthread+0x100/0x140
[ 513.622769] ret_from_fork+0x24/0x30
[ 513.622957]
other info that might help us debug this:
[ 513.623354] Chain exists of:
&dqm->lock_hidden --> &mapping->i_mmap_rwsem --> &anon_vma->rwsem
[ 513.623900] Possible unsafe locking scenario:
[ 513.624189] CPU0 CPU1
[ 513.624397] ---- ----
[ 513.624594] lock(&anon_vma->rwsem);
[ 513.624771] lock(&mapping->i_mmap_rwsem);
[ 513.625020] lock(&anon_vma->rwsem);
[ 513.625253] lock(&dqm->lock_hidden);
[ 513.625433]
*** DEADLOCK ***
[ 513.625783] 3 locks held by kswapd0/611:
[ 513.625967] #0: 00000000f14edf84 (fs_reclaim){+.+.}, at: __fs_reclaim_acquire+0x5/0x30
[ 513.626309] #1: 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
[ 513.626671] #2: 0000000067b5cd12 (srcu){....}, at: __mmu_notifier_invalidate_range_start+0x5/0xb0
[ 513.627037]
stack backtrace:
[ 513.627292] CPU: 0 PID: 611 Comm: kswapd0 Tainted: G W 4.18.0-kfd-root #2
[ 513.627632] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 513.627990] Call Trace:
[ 513.628143] dump_stack+0x7c/0xbb
[ 513.628315] print_circular_bug.isra.37+0x21b/0x228
[ 513.628581] __lock_acquire+0xf7d/0x1470
[ 513.628782] ? unwind_next_frame+0x6c/0x4f0
[ 513.628974] ? lock_acquire+0xec/0x1e0
[ 513.629154] lock_acquire+0xec/0x1e0
[ 513.629357] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.629587] __mutex_lock+0x98/0x970
[ 513.629790] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630047] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630309] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630562] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630816] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
[ 513.631057] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
[ 513.631288] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
[ 513.631536] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
[ 513.632076] __mmu_notifier_invalidate_range_start+0x70/0xb0
[ 513.632299] try_to_unmap_one+0x7fc/0x8f0
[ 513.632487] ? page_lock_anon_vma_read+0x68/0x250
[ 513.632690] rmap_walk_anon+0x121/0x290
[ 513.632875] try_to_unmap+0x93/0xf0
[ 513.633050] ? page_remove_rmap+0x330/0x330
[ 513.633239] ? rcu_read_unlock+0x60/0x60
[ 513.633422] ? page_get_anon_vma+0x160/0x160
[ 513.633613] shrink_page_list+0x606/0xcb0
[ 513.633800] shrink_inactive_list+0x33b/0x700
[ 513.633997] shrink_node_memcg+0x37a/0x7f0
[ 513.634186] ? shrink_node+0xd8/0x490
[ 513.634363] shrink_node+0xd8/0x490
[ 513.634537] balance_pgdat+0x18b/0x3b0
[ 513.634718] kswapd+0x203/0x5c0
[ 513.634887] ? wait_woken+0xb0/0xb0
[ 513.635062] kthread+0x100/0x140
[ 513.635231] ? balance_pgdat+0x3b0/0x3b0
[ 513.635414] ? kthread_delayed_work_timer_fn+0x80/0x80
[ 513.635626] ret_from_fork+0x24/0x30
[ 513.636042] Evicting PASID 32768 queues
[ 513.936236] Restoring PASID 32768 queues
[ 524.708912] Evicting PASID 32768 queues
[ 524.999875] Restoring PASID 32768 queues
Change-Id: I334c8c9329be12e468ea7aabc878842ec003bd8e
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
---
.../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index d811f63..2cd47e3 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -274,6 +274,12 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
print_queue(q);
+ mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
+ q->properties.type)];
+ q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
+ if (!q->mqd_mem_obj)
+ return -ENOMEM;
+
dqm_lock(dqm);
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
@@ -301,8 +307,6 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
- mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
- q->properties.type)];
if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE) {
retval = allocate_hqd(dqm, q);
if (retval)
@@ -321,13 +325,10 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
if (retval)
goto out_deallocate_hqd;
- q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
- if (!q->mqd_mem_obj)
- goto out_deallocate_doorbell;
retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
&q->gart_mqd_addr, &q->properties);
if (retval)
- goto out_uninit_mqd;
+ goto out_deallocate_doorbell;
if (q->properties.is_active) {
@@ -338,7 +339,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe,
q->queue, &q->properties, current->mm);
if (retval)
- goto out_uninit_mqd;
+ goto out_deallocate_doorbell;
}
list_add(&q->list, &qpd->queues_list);
@@ -358,10 +359,9 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
dqm->total_queue_count++;
pr_debug("Total of %d queues are accountable so far\n",
dqm->total_queue_count);
- goto out_unlock;
+ dqm_unlock(dqm);
+ return retval;
-out_uninit_mqd:
- mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
out_deallocate_doorbell:
deallocate_doorbell(qpd, q);
out_deallocate_hqd:
@@ -375,6 +375,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
deallocate_vmid(dqm, qpd, q);
out_unlock:
dqm_unlock(dqm);
+ mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
return retval;
}
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency
[not found] ` <1559750793-16608-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
@ 2019-06-05 16:06 ` Zeng, Oak
[not found] ` <1559750793-16608-5-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
0 siblings, 1 reply; 8+ messages in thread
From: Zeng, Oak @ 2019-06-05 16:06 UTC (permalink / raw)
To: amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Kuehling, Felix, Freehill, Chris, Zeng, Oak,
Liu, Alex
The idea to break the circular lock dependency is to move allocate_mqd
out of dqm lock protection. See callstack #1 below.
[ 59.510149] [drm] Initialized amdgpu 3.30.0 20150101 for 0000:04:00.0 on minor 0
[ 513.604034] ======================================================
[ 513.604205] WARNING: possible circular locking dependency detected
[ 513.604375] 4.18.0-kfd-root #2 Tainted: G W
[ 513.604530] ------------------------------------------------------
[ 513.604699] kswapd0/611 is trying to acquire lock:
[ 513.604840] 00000000d254022e (&dqm->lock_hidden){+.+.}, at: evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.605150]
but task is already holding lock:
[ 513.605307] 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
[ 513.605540]
which lock already depends on the new lock.
[ 513.605747]
the existing dependency chain (in reverse order) is:
[ 513.605944]
-> #4 (&anon_vma->rwsem){++++}:
[ 513.606106] __vma_adjust+0x147/0x7f0
[ 513.606231] __split_vma+0x179/0x190
[ 513.606353] mprotect_fixup+0x217/0x260
[ 513.606553] do_mprotect_pkey+0x211/0x380
[ 513.606752] __x64_sys_mprotect+0x1b/0x20
[ 513.606954] do_syscall_64+0x50/0x1a0
[ 513.607149] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.607380]
-> #3 (&mapping->i_mmap_rwsem){++++}:
[ 513.607678] rmap_walk_file+0x1f0/0x280
[ 513.607887] page_referenced+0xdd/0x180
[ 513.608081] shrink_page_list+0x853/0xcb0
[ 513.608279] shrink_inactive_list+0x33b/0x700
[ 513.608483] shrink_node_memcg+0x37a/0x7f0
[ 513.608682] shrink_node+0xd8/0x490
[ 513.608869] balance_pgdat+0x18b/0x3b0
[ 513.609062] kswapd+0x203/0x5c0
[ 513.609241] kthread+0x100/0x140
[ 513.609420] ret_from_fork+0x24/0x30
[ 513.609607]
-> #2 (fs_reclaim){+.+.}:
[ 513.609883] kmem_cache_alloc_trace+0x34/0x2e0
[ 513.610093] reservation_object_reserve_shared+0x139/0x300
[ 513.610326] ttm_bo_init_reserved+0x291/0x480 [ttm]
[ 513.610567] amdgpu_bo_do_create+0x1d2/0x650 [amdgpu]
[ 513.610811] amdgpu_bo_create+0x40/0x1f0 [amdgpu]
[ 513.611041] amdgpu_bo_create_reserved+0x249/0x2d0 [amdgpu]
[ 513.611290] amdgpu_bo_create_kernel+0x12/0x70 [amdgpu]
[ 513.611584] amdgpu_ttm_init+0x2cb/0x560 [amdgpu]
[ 513.611823] gmc_v9_0_sw_init+0x400/0x750 [amdgpu]
[ 513.612491] amdgpu_device_init+0x14eb/0x1990 [amdgpu]
[ 513.612730] amdgpu_driver_load_kms+0x78/0x290 [amdgpu]
[ 513.612958] drm_dev_register+0x111/0x1a0
[ 513.613171] amdgpu_pci_probe+0x11c/0x1e0 [amdgpu]
[ 513.613389] local_pci_probe+0x3f/0x90
[ 513.613581] pci_device_probe+0x102/0x1c0
[ 513.613779] driver_probe_device+0x2a7/0x480
[ 513.613984] __driver_attach+0x10a/0x110
[ 513.614179] bus_for_each_dev+0x67/0xc0
[ 513.614372] bus_add_driver+0x1eb/0x260
[ 513.614565] driver_register+0x5b/0xe0
[ 513.614756] do_one_initcall+0xac/0x357
[ 513.614952] do_init_module+0x5b/0x213
[ 513.615145] load_module+0x2542/0x2d30
[ 513.615337] __do_sys_finit_module+0xd2/0x100
[ 513.615541] do_syscall_64+0x50/0x1a0
[ 513.615731] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.615963]
-> #1 (reservation_ww_class_mutex){+.+.}:
[ 513.616293] amdgpu_amdkfd_alloc_gtt_mem+0xcf/0x2c0 [amdgpu]
[ 513.616554] init_mqd+0x223/0x260 [amdgpu]
[ 513.616779] create_queue_nocpsch+0x4d9/0x600 [amdgpu]
[ 513.617031] pqm_create_queue+0x37c/0x520 [amdgpu]
[ 513.617270] kfd_ioctl_create_queue+0x2f9/0x650 [amdgpu]
[ 513.617522] kfd_ioctl+0x202/0x350 [amdgpu]
[ 513.617724] do_vfs_ioctl+0x9f/0x6c0
[ 513.617914] ksys_ioctl+0x66/0x70
[ 513.618095] __x64_sys_ioctl+0x16/0x20
[ 513.618286] do_syscall_64+0x50/0x1a0
[ 513.618476] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.618695]
-> #0 (&dqm->lock_hidden){+.+.}:
[ 513.618984] __mutex_lock+0x98/0x970
[ 513.619197] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.619459] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
[ 513.619710] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
[ 513.620103] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
[ 513.620363] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
[ 513.620614] __mmu_notifier_invalidate_range_start+0x70/0xb0
[ 513.620851] try_to_unmap_one+0x7fc/0x8f0
[ 513.621049] rmap_walk_anon+0x121/0x290
[ 513.621242] try_to_unmap+0x93/0xf0
[ 513.621428] shrink_page_list+0x606/0xcb0
[ 513.621625] shrink_inactive_list+0x33b/0x700
[ 513.621835] shrink_node_memcg+0x37a/0x7f0
[ 513.622034] shrink_node+0xd8/0x490
[ 513.622219] balance_pgdat+0x18b/0x3b0
[ 513.622410] kswapd+0x203/0x5c0
[ 513.622589] kthread+0x100/0x140
[ 513.622769] ret_from_fork+0x24/0x30
[ 513.622957]
other info that might help us debug this:
[ 513.623354] Chain exists of:
&dqm->lock_hidden --> &mapping->i_mmap_rwsem --> &anon_vma->rwsem
[ 513.623900] Possible unsafe locking scenario:
[ 513.624189] CPU0 CPU1
[ 513.624397] ---- ----
[ 513.624594] lock(&anon_vma->rwsem);
[ 513.624771] lock(&mapping->i_mmap_rwsem);
[ 513.625020] lock(&anon_vma->rwsem);
[ 513.625253] lock(&dqm->lock_hidden);
[ 513.625433]
*** DEADLOCK ***
[ 513.625783] 3 locks held by kswapd0/611:
[ 513.625967] #0: 00000000f14edf84 (fs_reclaim){+.+.}, at: __fs_reclaim_acquire+0x5/0x30
[ 513.626309] #1: 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
[ 513.626671] #2: 0000000067b5cd12 (srcu){....}, at: __mmu_notifier_invalidate_range_start+0x5/0xb0
[ 513.627037]
stack backtrace:
[ 513.627292] CPU: 0 PID: 611 Comm: kswapd0 Tainted: G W 4.18.0-kfd-root #2
[ 513.627632] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 513.627990] Call Trace:
[ 513.628143] dump_stack+0x7c/0xbb
[ 513.628315] print_circular_bug.isra.37+0x21b/0x228
[ 513.628581] __lock_acquire+0xf7d/0x1470
[ 513.628782] ? unwind_next_frame+0x6c/0x4f0
[ 513.628974] ? lock_acquire+0xec/0x1e0
[ 513.629154] lock_acquire+0xec/0x1e0
[ 513.629357] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.629587] __mutex_lock+0x98/0x970
[ 513.629790] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630047] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630309] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630562] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630816] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
[ 513.631057] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
[ 513.631288] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
[ 513.631536] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
[ 513.632076] __mmu_notifier_invalidate_range_start+0x70/0xb0
[ 513.632299] try_to_unmap_one+0x7fc/0x8f0
[ 513.632487] ? page_lock_anon_vma_read+0x68/0x250
[ 513.632690] rmap_walk_anon+0x121/0x290
[ 513.632875] try_to_unmap+0x93/0xf0
[ 513.633050] ? page_remove_rmap+0x330/0x330
[ 513.633239] ? rcu_read_unlock+0x60/0x60
[ 513.633422] ? page_get_anon_vma+0x160/0x160
[ 513.633613] shrink_page_list+0x606/0xcb0
[ 513.633800] shrink_inactive_list+0x33b/0x700
[ 513.633997] shrink_node_memcg+0x37a/0x7f0
[ 513.634186] ? shrink_node+0xd8/0x490
[ 513.634363] shrink_node+0xd8/0x490
[ 513.634537] balance_pgdat+0x18b/0x3b0
[ 513.634718] kswapd+0x203/0x5c0
[ 513.634887] ? wait_woken+0xb0/0xb0
[ 513.635062] kthread+0x100/0x140
[ 513.635231] ? balance_pgdat+0x3b0/0x3b0
[ 513.635414] ? kthread_delayed_work_timer_fn+0x80/0x80
[ 513.635626] ret_from_fork+0x24/0x30
[ 513.636042] Evicting PASID 32768 queues
[ 513.936236] Restoring PASID 32768 queues
[ 524.708912] Evicting PASID 32768 queues
[ 524.999875] Restoring PASID 32768 queues
Change-Id: I334c8c9329be12e468ea7aabc878842ec003bd8e
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
---
.../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 21 +++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 787b5be..6b1a2ee 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -274,6 +274,12 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
print_queue(q);
+ mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
+ q->properties.type)];
+ q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
+ if (!q->mqd_mem_obj)
+ return -ENOMEM;
+
dqm_lock(dqm);
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
@@ -299,8 +305,6 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
- mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
- q->properties.type)];
if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE) {
retval = allocate_hqd(dqm, q);
if (retval)
@@ -319,13 +323,10 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
if (retval)
goto out_deallocate_hqd;
- q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
- if (!q->mqd_mem_obj)
- goto out_deallocate_doorbell;
retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
&q->gart_mqd_addr, &q->properties);
if (retval)
- goto out_uninit_mqd;
+ goto out_deallocate_doorbell;
if (q->properties.is_active) {
@@ -336,7 +337,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe,
q->queue, &q->properties, current->mm);
if (retval)
- goto out_uninit_mqd;
+ goto out_deallocate_doorbell;
}
list_add(&q->list, &qpd->queues_list);
@@ -356,10 +357,9 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
dqm->total_queue_count++;
pr_debug("Total of %d queues are accountable so far\n",
dqm->total_queue_count);
- goto out_unlock;
+ dqm_unlock(dqm);
+ return retval;
-out_uninit_mqd:
- mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
out_deallocate_doorbell:
deallocate_doorbell(qpd, q);
out_deallocate_hqd:
@@ -373,6 +373,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
deallocate_vmid(dqm, qpd, q);
out_unlock:
dqm_unlock(dqm);
+ mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
return retval;
}
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency
[not found] ` <1559750793-16608-5-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
@ 2019-06-05 22:20 ` Kuehling, Felix
0 siblings, 0 replies; 8+ messages in thread
From: Kuehling, Felix @ 2019-06-05 22:20 UTC (permalink / raw)
To: Zeng, Oak,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Freehill, Chris, Liu, Alex
This patch looks good to me, but it'll probably change a little if you
implement my suggestions for patch 4.
Regards,
Felix
On 2019-06-05 12:06 p.m., Zeng, Oak wrote:
> The idea to break the circular lock dependency is to move allocate_mqd
> out of dqm lock protection. See callstack #1 below.
>
> [ 59.510149] [drm] Initialized amdgpu 3.30.0 20150101 for 0000:04:00.0 on minor 0
>
> [ 513.604034] ======================================================
> [ 513.604205] WARNING: possible circular locking dependency detected
> [ 513.604375] 4.18.0-kfd-root #2 Tainted: G W
> [ 513.604530] ------------------------------------------------------
> [ 513.604699] kswapd0/611 is trying to acquire lock:
> [ 513.604840] 00000000d254022e (&dqm->lock_hidden){+.+.}, at: evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.605150]
> but task is already holding lock:
> [ 513.605307] 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
> [ 513.605540]
> which lock already depends on the new lock.
>
> [ 513.605747]
> the existing dependency chain (in reverse order) is:
> [ 513.605944]
> -> #4 (&anon_vma->rwsem){++++}:
> [ 513.606106] __vma_adjust+0x147/0x7f0
> [ 513.606231] __split_vma+0x179/0x190
> [ 513.606353] mprotect_fixup+0x217/0x260
> [ 513.606553] do_mprotect_pkey+0x211/0x380
> [ 513.606752] __x64_sys_mprotect+0x1b/0x20
> [ 513.606954] do_syscall_64+0x50/0x1a0
> [ 513.607149] entry_SYSCALL_64_after_hwframe+0x49/0xbe
> [ 513.607380]
> -> #3 (&mapping->i_mmap_rwsem){++++}:
> [ 513.607678] rmap_walk_file+0x1f0/0x280
> [ 513.607887] page_referenced+0xdd/0x180
> [ 513.608081] shrink_page_list+0x853/0xcb0
> [ 513.608279] shrink_inactive_list+0x33b/0x700
> [ 513.608483] shrink_node_memcg+0x37a/0x7f0
> [ 513.608682] shrink_node+0xd8/0x490
> [ 513.608869] balance_pgdat+0x18b/0x3b0
> [ 513.609062] kswapd+0x203/0x5c0
> [ 513.609241] kthread+0x100/0x140
> [ 513.609420] ret_from_fork+0x24/0x30
> [ 513.609607]
> -> #2 (fs_reclaim){+.+.}:
> [ 513.609883] kmem_cache_alloc_trace+0x34/0x2e0
> [ 513.610093] reservation_object_reserve_shared+0x139/0x300
> [ 513.610326] ttm_bo_init_reserved+0x291/0x480 [ttm]
> [ 513.610567] amdgpu_bo_do_create+0x1d2/0x650 [amdgpu]
> [ 513.610811] amdgpu_bo_create+0x40/0x1f0 [amdgpu]
> [ 513.611041] amdgpu_bo_create_reserved+0x249/0x2d0 [amdgpu]
> [ 513.611290] amdgpu_bo_create_kernel+0x12/0x70 [amdgpu]
> [ 513.611584] amdgpu_ttm_init+0x2cb/0x560 [amdgpu]
> [ 513.611823] gmc_v9_0_sw_init+0x400/0x750 [amdgpu]
> [ 513.612491] amdgpu_device_init+0x14eb/0x1990 [amdgpu]
> [ 513.612730] amdgpu_driver_load_kms+0x78/0x290 [amdgpu]
> [ 513.612958] drm_dev_register+0x111/0x1a0
> [ 513.613171] amdgpu_pci_probe+0x11c/0x1e0 [amdgpu]
> [ 513.613389] local_pci_probe+0x3f/0x90
> [ 513.613581] pci_device_probe+0x102/0x1c0
> [ 513.613779] driver_probe_device+0x2a7/0x480
> [ 513.613984] __driver_attach+0x10a/0x110
> [ 513.614179] bus_for_each_dev+0x67/0xc0
> [ 513.614372] bus_add_driver+0x1eb/0x260
> [ 513.614565] driver_register+0x5b/0xe0
> [ 513.614756] do_one_initcall+0xac/0x357
> [ 513.614952] do_init_module+0x5b/0x213
> [ 513.615145] load_module+0x2542/0x2d30
> [ 513.615337] __do_sys_finit_module+0xd2/0x100
> [ 513.615541] do_syscall_64+0x50/0x1a0
> [ 513.615731] entry_SYSCALL_64_after_hwframe+0x49/0xbe
> [ 513.615963]
> -> #1 (reservation_ww_class_mutex){+.+.}:
> [ 513.616293] amdgpu_amdkfd_alloc_gtt_mem+0xcf/0x2c0 [amdgpu]
> [ 513.616554] init_mqd+0x223/0x260 [amdgpu]
> [ 513.616779] create_queue_nocpsch+0x4d9/0x600 [amdgpu]
> [ 513.617031] pqm_create_queue+0x37c/0x520 [amdgpu]
> [ 513.617270] kfd_ioctl_create_queue+0x2f9/0x650 [amdgpu]
> [ 513.617522] kfd_ioctl+0x202/0x350 [amdgpu]
> [ 513.617724] do_vfs_ioctl+0x9f/0x6c0
> [ 513.617914] ksys_ioctl+0x66/0x70
> [ 513.618095] __x64_sys_ioctl+0x16/0x20
> [ 513.618286] do_syscall_64+0x50/0x1a0
> [ 513.618476] entry_SYSCALL_64_after_hwframe+0x49/0xbe
> [ 513.618695]
> -> #0 (&dqm->lock_hidden){+.+.}:
> [ 513.618984] __mutex_lock+0x98/0x970
> [ 513.619197] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.619459] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
> [ 513.619710] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
> [ 513.620103] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
> [ 513.620363] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
> [ 513.620614] __mmu_notifier_invalidate_range_start+0x70/0xb0
> [ 513.620851] try_to_unmap_one+0x7fc/0x8f0
> [ 513.621049] rmap_walk_anon+0x121/0x290
> [ 513.621242] try_to_unmap+0x93/0xf0
> [ 513.621428] shrink_page_list+0x606/0xcb0
> [ 513.621625] shrink_inactive_list+0x33b/0x700
> [ 513.621835] shrink_node_memcg+0x37a/0x7f0
> [ 513.622034] shrink_node+0xd8/0x490
> [ 513.622219] balance_pgdat+0x18b/0x3b0
> [ 513.622410] kswapd+0x203/0x5c0
> [ 513.622589] kthread+0x100/0x140
> [ 513.622769] ret_from_fork+0x24/0x30
> [ 513.622957]
> other info that might help us debug this:
>
> [ 513.623354] Chain exists of:
> &dqm->lock_hidden --> &mapping->i_mmap_rwsem --> &anon_vma->rwsem
>
> [ 513.623900] Possible unsafe locking scenario:
>
> [ 513.624189] CPU0 CPU1
> [ 513.624397] ---- ----
> [ 513.624594] lock(&anon_vma->rwsem);
> [ 513.624771] lock(&mapping->i_mmap_rwsem);
> [ 513.625020] lock(&anon_vma->rwsem);
> [ 513.625253] lock(&dqm->lock_hidden);
> [ 513.625433]
> *** DEADLOCK ***
>
> [ 513.625783] 3 locks held by kswapd0/611:
> [ 513.625967] #0: 00000000f14edf84 (fs_reclaim){+.+.}, at: __fs_reclaim_acquire+0x5/0x30
> [ 513.626309] #1: 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
> [ 513.626671] #2: 0000000067b5cd12 (srcu){....}, at: __mmu_notifier_invalidate_range_start+0x5/0xb0
> [ 513.627037]
> stack backtrace:
> [ 513.627292] CPU: 0 PID: 611 Comm: kswapd0 Tainted: G W 4.18.0-kfd-root #2
> [ 513.627632] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
> [ 513.627990] Call Trace:
> [ 513.628143] dump_stack+0x7c/0xbb
> [ 513.628315] print_circular_bug.isra.37+0x21b/0x228
> [ 513.628581] __lock_acquire+0xf7d/0x1470
> [ 513.628782] ? unwind_next_frame+0x6c/0x4f0
> [ 513.628974] ? lock_acquire+0xec/0x1e0
> [ 513.629154] lock_acquire+0xec/0x1e0
> [ 513.629357] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.629587] __mutex_lock+0x98/0x970
> [ 513.629790] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.630047] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.630309] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.630562] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
> [ 513.630816] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
> [ 513.631057] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
> [ 513.631288] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
> [ 513.631536] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
> [ 513.632076] __mmu_notifier_invalidate_range_start+0x70/0xb0
> [ 513.632299] try_to_unmap_one+0x7fc/0x8f0
> [ 513.632487] ? page_lock_anon_vma_read+0x68/0x250
> [ 513.632690] rmap_walk_anon+0x121/0x290
> [ 513.632875] try_to_unmap+0x93/0xf0
> [ 513.633050] ? page_remove_rmap+0x330/0x330
> [ 513.633239] ? rcu_read_unlock+0x60/0x60
> [ 513.633422] ? page_get_anon_vma+0x160/0x160
> [ 513.633613] shrink_page_list+0x606/0xcb0
> [ 513.633800] shrink_inactive_list+0x33b/0x700
> [ 513.633997] shrink_node_memcg+0x37a/0x7f0
> [ 513.634186] ? shrink_node+0xd8/0x490
> [ 513.634363] shrink_node+0xd8/0x490
> [ 513.634537] balance_pgdat+0x18b/0x3b0
> [ 513.634718] kswapd+0x203/0x5c0
> [ 513.634887] ? wait_woken+0xb0/0xb0
> [ 513.635062] kthread+0x100/0x140
> [ 513.635231] ? balance_pgdat+0x3b0/0x3b0
> [ 513.635414] ? kthread_delayed_work_timer_fn+0x80/0x80
> [ 513.635626] ret_from_fork+0x24/0x30
> [ 513.636042] Evicting PASID 32768 queues
> [ 513.936236] Restoring PASID 32768 queues
> [ 524.708912] Evicting PASID 32768 queues
> [ 524.999875] Restoring PASID 32768 queues
>
> Change-Id: I334c8c9329be12e468ea7aabc878842ec003bd8e
> Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
> ---
> .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 21 +++++++++++----------
> 1 file changed, 11 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> index 787b5be..6b1a2ee 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> @@ -274,6 +274,12 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
>
> print_queue(q);
>
> + mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
> + q->properties.type)];
> + q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
> + if (!q->mqd_mem_obj)
> + return -ENOMEM;
> +
> dqm_lock(dqm);
>
> if (dqm->total_queue_count >= max_num_of_queues_per_device) {
> @@ -299,8 +305,6 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> q->properties.tba_addr = qpd->tba_addr;
> q->properties.tma_addr = qpd->tma_addr;
>
> - mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
> - q->properties.type)];
> if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE) {
> retval = allocate_hqd(dqm, q);
> if (retval)
> @@ -319,13 +323,10 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> if (retval)
> goto out_deallocate_hqd;
>
> - q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
> - if (!q->mqd_mem_obj)
> - goto out_deallocate_doorbell;
> retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
> &q->gart_mqd_addr, &q->properties);
> if (retval)
> - goto out_uninit_mqd;
> + goto out_deallocate_doorbell;
>
> if (q->properties.is_active) {
>
> @@ -336,7 +337,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe,
> q->queue, &q->properties, current->mm);
> if (retval)
> - goto out_uninit_mqd;
> + goto out_deallocate_doorbell;
> }
>
> list_add(&q->list, &qpd->queues_list);
> @@ -356,10 +357,9 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> dqm->total_queue_count++;
> pr_debug("Total of %d queues are accountable so far\n",
> dqm->total_queue_count);
> - goto out_unlock;
> + dqm_unlock(dqm);
> + return retval;
>
> -out_uninit_mqd:
> - mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> out_deallocate_doorbell:
> deallocate_doorbell(qpd, q);
> out_deallocate_hqd:
> @@ -373,6 +373,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> deallocate_vmid(dqm, qpd, q);
> out_unlock:
> dqm_unlock(dqm);
> + mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> return retval;
> }
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 4/6] drm/amdkfd: Separate mqd allocation and initialization
@ 2019-06-06 18:25 Zeng, Oak
[not found] ` <1559845507-3052-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
0 siblings, 1 reply; 8+ messages in thread
From: Zeng, Oak @ 2019-06-06 18:25 UTC (permalink / raw)
To: amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Kuehling, Felix, Freehill, Chris, Zeng, Oak,
Liu, Alex
Introduce a new mqd allocation interface and split the original
init_mqd function into two functions: allocate_mqd and init_mqd.
Also renamed uninit_mqd to free_mqd. This is preparation work to
fix a circular lock dependency.
Change-Id: I26e53ee1abcdd688ad11d35b433da77e3fa1bee7
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
---
.../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 34 ++++-----
drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 16 ++--
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c | 4 +-
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h | 18 +++--
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c | 78 +++++++------------
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 78 +++++++------------
drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c | 88 ++++++++--------------
7 files changed, 124 insertions(+), 192 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 3c042eb..10d4f4f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -319,11 +319,11 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
if (retval)
goto out_deallocate_hqd;
- retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
- &q->gart_mqd_addr, &q->properties);
- if (retval)
+ q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
+ if (!q->mqd_mem_obj)
goto out_deallocate_doorbell;
-
+ mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
+ &q->gart_mqd_addr, &q->properties);
if (q->properties.is_active) {
if (WARN(q->process->mm != current->mm,
@@ -333,7 +333,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe,
q->queue, &q->properties, current->mm);
if (retval)
- goto out_uninit_mqd;
+ goto out_free_mqd;
}
list_add(&q->list, &qpd->queues_list);
@@ -355,8 +355,8 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
dqm->total_queue_count);
goto out_unlock;
-out_uninit_mqd:
- mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+out_free_mqd:
+ mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
out_deallocate_doorbell:
deallocate_doorbell(qpd, q);
out_deallocate_hqd:
@@ -450,7 +450,7 @@ static int destroy_queue_nocpsch_locked(struct device_queue_manager *dqm,
if (retval == -ETIME)
qpd->reset_wavefronts = true;
- mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+ mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
list_del(&q->list);
if (list_empty(&qpd->queues_list)) {
@@ -527,7 +527,7 @@ static int update_queue(struct device_queue_manager *dqm, struct queue *q)
}
}
- retval = mqd_mgr->update_mqd(mqd_mgr, q->mqd, &q->properties);
+ mqd_mgr->update_mqd(mqd_mgr, q->mqd, &q->properties);
/*
* check active state vs. the previous state and modify
@@ -1160,11 +1160,11 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
- retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
- &q->gart_mqd_addr, &q->properties);
- if (retval)
+ q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
+ if (!q->mqd_mem_obj)
goto out_deallocate_doorbell;
-
+ mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
+ &q->gart_mqd_addr, &q->properties);
dqm_lock(dqm);
list_add(&q->list, &qpd->queues_list);
@@ -1373,8 +1373,8 @@ static int destroy_queue_cpsch(struct device_queue_manager *dqm,
dqm_unlock(dqm);
- /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
- mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+ /* Do free_mqd after dqm_unlock(dqm) to avoid circular locking */
+ mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
return retval;
@@ -1615,14 +1615,14 @@ static int process_termination_cpsch(struct device_queue_manager *dqm,
kfd_dec_compute_active(dqm->dev);
/* Lastly, free mqd resources.
- * Do uninit_mqd() after dqm_unlock to avoid circular locking.
+ * Do free_mqd() after dqm_unlock to avoid circular locking.
*/
list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
q->properties.type)];
list_del(&q->list);
qpd->queue_count--;
- mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
+ mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
}
return retval;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
index 1cc03b3..229500c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
@@ -132,13 +132,14 @@ static bool initialize(struct kernel_queue *kq, struct kfd_dev *dev,
kq->queue->device = dev;
kq->queue->process = kfd_get_process(current);
- retval = kq->mqd_mgr->init_mqd(kq->mqd_mgr, &kq->queue->mqd,
- &kq->queue->mqd_mem_obj,
+ kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
+ &kq->queue->properties);
+ if (!kq->queue->mqd_mem_obj)
+ goto err_allocate_mqd;
+ kq->mqd_mgr->init_mqd(kq->mqd_mgr, &kq->queue->mqd,
+ kq->queue->mqd_mem_obj,
&kq->queue->gart_mqd_addr,
&kq->queue->properties);
- if (retval != 0)
- goto err_init_mqd;
-
/* assign HIQ to HQD */
if (type == KFD_QUEUE_TYPE_HIQ) {
pr_debug("Assigning hiq to hqd\n");
@@ -164,7 +165,8 @@ static bool initialize(struct kernel_queue *kq, struct kfd_dev *dev,
return true;
err_alloc_fence:
-err_init_mqd:
+ kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
+err_allocate_mqd:
uninit_queue(kq->queue);
err_init_queue:
kfd_gtt_sa_free(dev, kq->wptr_mem);
@@ -193,7 +195,7 @@ static void uninitialize(struct kernel_queue *kq)
else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
- kq->mqd_mgr->uninit_mqd(kq->mqd_mgr, kq->queue->mqd,
+ kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
kq->queue->mqd_mem_obj);
kfd_gtt_sa_free(kq->dev, kq->rptr_mem);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
index cc04b362..d6cf391 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
@@ -45,7 +45,7 @@ int pipe_priority_map[] = {
KFD_PIPE_PRIORITY_CS_HIGH
};
-struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev)
+struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev, struct queue_properties *q)
{
struct kfd_mem_obj *mqd_mem_obj = NULL;
@@ -86,7 +86,7 @@ struct kfd_mem_obj *allocate_sdma_mqd(struct kfd_dev *dev,
return mqd_mem_obj;
}
-void uninit_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
+void free_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
struct kfd_mem_obj *mqd_mem_obj)
{
WARN_ON(!mqd_mem_obj->gtt_mem);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
index 66b8c67..550b61e 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
@@ -39,7 +39,7 @@
* @destroy_mqd: Destroys the HQD slot and by that preempt the relevant queue.
* Used only for no cp scheduling.
*
- * @uninit_mqd: Releases the mqd buffer from local gpu memory.
+ * @free_mqd: Releases the mqd buffer from local gpu memory.
*
* @is_occupied: Checks if the relevant HQD slot is occupied.
*
@@ -64,8 +64,11 @@
*/
extern int pipe_priority_map[];
struct mqd_manager {
- int (*init_mqd)(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+ struct kfd_mem_obj* (*allocate_mqd)(struct kfd_dev *kfd,
+ struct queue_properties *q);
+
+ void (*init_mqd)(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q);
int (*load_mqd)(struct mqd_manager *mm, void *mqd,
@@ -73,7 +76,7 @@ struct mqd_manager {
struct queue_properties *p,
struct mm_struct *mms);
- int (*update_mqd)(struct mqd_manager *mm, void *mqd,
+ void (*update_mqd)(struct mqd_manager *mm, void *mqd,
struct queue_properties *q);
int (*destroy_mqd)(struct mqd_manager *mm, void *mqd,
@@ -81,7 +84,7 @@ struct mqd_manager {
unsigned int timeout, uint32_t pipe_id,
uint32_t queue_id);
- void (*uninit_mqd)(struct mqd_manager *mm, void *mqd,
+ void (*free_mqd)(struct mqd_manager *mm, void *mqd,
struct kfd_mem_obj *mqd_mem_obj);
bool (*is_occupied)(struct mqd_manager *mm, void *mqd,
@@ -102,11 +105,12 @@ struct mqd_manager {
uint32_t mqd_size;
};
-struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev);
+struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev,
+ struct queue_properties *q);
struct kfd_mem_obj *allocate_sdma_mqd(struct kfd_dev *dev,
struct queue_properties *q);
-void uninit_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
+void free_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
struct kfd_mem_obj *mqd_mem_obj);
void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
index e911438..28876ac 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
@@ -77,9 +77,6 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
{
struct kfd_mem_obj *mqd_mem_obj;
- if (q->type == KFD_QUEUE_TYPE_HIQ)
- return allocate_hiq_mqd(kfd);
-
if (kfd_gtt_sa_allocate(kfd, sizeof(struct cik_mqd),
&mqd_mem_obj))
return NULL;
@@ -87,21 +84,15 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
return mqd_mem_obj;
}
-static int init_mqd(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
uint64_t addr;
struct cik_mqd *m;
- int retval;
- struct kfd_dev *kfd = mm->dev;
-
- *mqd_mem_obj = allocate_mqd(kfd, q);
- if (!*mqd_mem_obj)
- return -ENOMEM;
- m = (struct cik_mqd *) (*mqd_mem_obj)->cpu_ptr;
- addr = (*mqd_mem_obj)->gpu_addr;
+ m = (struct cik_mqd *) mqd_mem_obj->cpu_ptr;
+ addr = mqd_mem_obj->gpu_addr;
memset(m, 0, ALIGN(sizeof(struct cik_mqd), 256));
@@ -144,37 +135,27 @@ static int init_mqd(struct mqd_manager *mm, void **mqd,
*mqd = m;
if (gart_addr)
*gart_addr = addr;
- retval = mm->update_mqd(mm, m, q);
-
- return retval;
+ mm->update_mqd(mm, m, q);
}
-static int init_mqd_sdma(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd_sdma(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
- int retval;
struct cik_sdma_rlc_registers *m;
- struct kfd_dev *dev = mm->dev;
- *mqd_mem_obj = allocate_sdma_mqd(dev, q);
- if (!*mqd_mem_obj)
- return -ENOMEM;
-
- m = (struct cik_sdma_rlc_registers *) (*mqd_mem_obj)->cpu_ptr;
+ m = (struct cik_sdma_rlc_registers *) mqd_mem_obj->cpu_ptr;
memset(m, 0, sizeof(struct cik_sdma_rlc_registers));
*mqd = m;
if (gart_addr)
- *gart_addr = (*mqd_mem_obj)->gpu_addr;
-
- retval = mm->update_mqd(mm, m, q);
+ *gart_addr = mqd_mem_obj->gpu_addr;
- return retval;
+ mm->update_mqd(mm, m, q);
}
-static void uninit_mqd(struct mqd_manager *mm, void *mqd,
+static void free_mqd(struct mqd_manager *mm, void *mqd,
struct kfd_mem_obj *mqd_mem_obj)
{
kfd_gtt_sa_free(mm->dev, mqd_mem_obj);
@@ -203,7 +184,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
mms);
}
-static int __update_mqd(struct mqd_manager *mm, void *mqd,
+static void __update_mqd(struct mqd_manager *mm, void *mqd,
struct queue_properties *q, unsigned int atc_bit)
{
struct cik_mqd *m;
@@ -237,23 +218,21 @@ static int __update_mqd(struct mqd_manager *mm, void *mqd,
set_priority(m, q);
q->is_active = QUEUE_IS_ACTIVE(*q);
-
- return 0;
}
-static int update_mqd(struct mqd_manager *mm, void *mqd,
+static void update_mqd(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
- return __update_mqd(mm, mqd, q, 1);
+ __update_mqd(mm, mqd, q, 1);
}
-static int update_mqd_hawaii(struct mqd_manager *mm, void *mqd,
+static void update_mqd_hawaii(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
- return __update_mqd(mm, mqd, q, 0);
+ __update_mqd(mm, mqd, q, 0);
}
-static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
+static void update_mqd_sdma(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct cik_sdma_rlc_registers *m;
@@ -278,8 +257,6 @@ static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
m->sdma_queue_id = q->sdma_queue_id;
q->is_active = QUEUE_IS_ACTIVE(*q);
-
- return 0;
}
static int destroy_mqd(struct mqd_manager *mm, void *mqd,
@@ -326,14 +303,14 @@ static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd,
* queues but with different initial values.
*/
-static int init_mqd_hiq(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd_hiq(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
- return init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
+ init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
}
-static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
+static void update_mqd_hiq(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct cik_mqd *m;
@@ -360,7 +337,6 @@ static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
q->is_active = QUEUE_IS_ACTIVE(*q);
set_priority(m, q);
- return 0;
}
#if defined(CONFIG_DEBUG_FS)
@@ -399,8 +375,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
switch (type) {
case KFD_MQD_TYPE_CP:
case KFD_MQD_TYPE_COMPUTE:
+ mqd->allocate_mqd = allocate_mqd;
mqd->init_mqd = init_mqd;
- mqd->uninit_mqd = uninit_mqd;
+ mqd->free_mqd = free_mqd;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd;
mqd->destroy_mqd = destroy_mqd;
@@ -411,8 +388,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_HIQ:
+ mqd->allocate_mqd = allocate_hiq_mqd;
mqd->init_mqd = init_mqd_hiq;
- mqd->uninit_mqd = uninit_mqd_hiq_sdma;
+ mqd->free_mqd = free_mqd_hiq_sdma;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd_hiq;
mqd->destroy_mqd = destroy_mqd;
@@ -423,8 +401,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_DIQ:
+ mqd->allocate_mqd = allocate_hiq_mqd;
mqd->init_mqd = init_mqd_hiq;
- mqd->uninit_mqd = uninit_mqd;
+ mqd->free_mqd = free_mqd;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd_hiq;
mqd->destroy_mqd = destroy_mqd;
@@ -435,8 +414,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_SDMA:
+ mqd->allocate_mqd = allocate_sdma_mqd;
mqd->init_mqd = init_mqd_sdma;
- mqd->uninit_mqd = uninit_mqd_hiq_sdma;
+ mqd->free_mqd = free_mqd_hiq_sdma;
mqd->load_mqd = load_mqd_sdma;
mqd->update_mqd = update_mqd_sdma;
mqd->destroy_mqd = destroy_mqd_sdma;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
index 818944b..0c58f91 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
@@ -79,9 +79,6 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
int retval;
struct kfd_mem_obj *mqd_mem_obj = NULL;
- if (q->type == KFD_QUEUE_TYPE_HIQ)
- return allocate_hiq_mqd(kfd);
-
/* From V9, for CWSR, the control stack is located on the next page
* boundary after the mqd, we will use the gtt allocation function
* instead of sub-allocation function.
@@ -110,21 +107,15 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
}
-static int init_mqd(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
- int retval;
uint64_t addr;
struct v9_mqd *m;
- struct kfd_dev *kfd = mm->dev;
-
- *mqd_mem_obj = allocate_mqd(kfd, q);
- if (!*mqd_mem_obj)
- return -ENOMEM;
- m = (struct v9_mqd *) (*mqd_mem_obj)->cpu_ptr;
- addr = (*mqd_mem_obj)->gpu_addr;
+ m = (struct v9_mqd *) mqd_mem_obj->cpu_ptr;
+ addr = mqd_mem_obj->gpu_addr;
memset(m, 0, sizeof(struct v9_mqd));
@@ -173,9 +164,7 @@ static int init_mqd(struct mqd_manager *mm, void **mqd,
*mqd = m;
if (gart_addr)
*gart_addr = addr;
- retval = mm->update_mqd(mm, m, q);
-
- return retval;
+ mm->update_mqd(mm, m, q);
}
static int load_mqd(struct mqd_manager *mm, void *mqd,
@@ -190,7 +179,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
wptr_shift, 0, mms);
}
-static int update_mqd(struct mqd_manager *mm, void *mqd,
+static void update_mqd(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct v9_mqd *m;
@@ -252,8 +241,6 @@ static int update_mqd(struct mqd_manager *mm, void *mqd,
set_priority(m, q);
q->is_active = QUEUE_IS_ACTIVE(*q);
-
- return 0;
}
@@ -267,7 +254,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
pipe_id, queue_id);
}
-static void uninit_mqd(struct mqd_manager *mm, void *mqd,
+static void free_mqd(struct mqd_manager *mm, void *mqd,
struct kfd_mem_obj *mqd_mem_obj)
{
struct kfd_dev *kfd = mm->dev;
@@ -311,62 +298,47 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,
return 0;
}
-static int init_mqd_hiq(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd_hiq(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
struct v9_mqd *m;
- int retval = init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
- if (retval != 0)
- return retval;
+ init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
m = get_mqd(*mqd);
m->cp_hqd_pq_control |= 1 << CP_HQD_PQ_CONTROL__PRIV_STATE__SHIFT |
1 << CP_HQD_PQ_CONTROL__KMD_QUEUE__SHIFT;
-
- return retval;
}
-static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
+static void update_mqd_hiq(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct v9_mqd *m;
- int retval = update_mqd(mm, mqd, q);
- if (retval != 0)
- return retval;
+ update_mqd(mm, mqd, q);
/* TODO: what's the point? update_mqd already does this. */
m = get_mqd(mqd);
m->cp_hqd_vmid = q->vmid;
- return retval;
}
-static int init_mqd_sdma(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd_sdma(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
- int retval;
struct v9_sdma_mqd *m;
- struct kfd_dev *dev = mm->dev;
-
- *mqd_mem_obj = allocate_sdma_mqd(dev, q);
- if (!*mqd_mem_obj)
- return -ENOMEM;
- m = (struct v9_sdma_mqd *) (*mqd_mem_obj)->cpu_ptr;
+ m = (struct v9_sdma_mqd *) mqd_mem_obj->cpu_ptr;
memset(m, 0, sizeof(struct v9_sdma_mqd));
*mqd = m;
if (gart_addr)
- *gart_addr = (*mqd_mem_obj)->gpu_addr;
+ *gart_addr = mqd_mem_obj->gpu_addr;
- retval = mm->update_mqd(mm, m, q);
-
- return retval;
+ mm->update_mqd(mm, m, q);
}
static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
@@ -380,7 +352,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
#define SDMA_RLC_DUMMY_DEFAULT 0xf
-static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
+static void update_mqd_sdma(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct v9_sdma_mqd *m;
@@ -404,8 +376,6 @@ static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
m->sdmax_rlcx_dummy_reg = SDMA_RLC_DUMMY_DEFAULT;
q->is_active = QUEUE_IS_ACTIVE(*q);
-
- return 0;
}
/*
@@ -462,8 +432,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
switch (type) {
case KFD_MQD_TYPE_CP:
case KFD_MQD_TYPE_COMPUTE:
+ mqd->allocate_mqd = allocate_mqd;
mqd->init_mqd = init_mqd;
- mqd->uninit_mqd = uninit_mqd;
+ mqd->free_mqd = free_mqd;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd;
mqd->destroy_mqd = destroy_mqd;
@@ -475,8 +446,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_HIQ:
+ mqd->allocate_mqd = allocate_hiq_mqd;
mqd->init_mqd = init_mqd_hiq;
- mqd->uninit_mqd = uninit_mqd_hiq_sdma;
+ mqd->free_mqd = free_mqd_hiq_sdma;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd_hiq;
mqd->destroy_mqd = destroy_mqd;
@@ -487,8 +459,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_DIQ:
+ mqd->allocate_mqd = allocate_hiq_mqd;
mqd->init_mqd = init_mqd_hiq;
- mqd->uninit_mqd = uninit_mqd;
+ mqd->free_mqd = free_mqd;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd_hiq;
mqd->destroy_mqd = destroy_mqd;
@@ -499,8 +472,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_SDMA:
+ mqd->allocate_mqd = allocate_sdma_mqd;
mqd->init_mqd = init_mqd_sdma;
- mqd->uninit_mqd = uninit_mqd_hiq_sdma;
+ mqd->free_mqd = free_mqd_hiq_sdma;
mqd->load_mqd = load_mqd_sdma;
mqd->update_mqd = update_mqd_sdma;
mqd->destroy_mqd = destroy_mqd_sdma;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
index 00e6a59..7d144f5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
@@ -80,9 +80,6 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
{
struct kfd_mem_obj *mqd_mem_obj;
- if (q->type == KFD_QUEUE_TYPE_HIQ)
- return allocate_hiq_mqd(kfd);
-
if (kfd_gtt_sa_allocate(kfd, sizeof(struct vi_mqd),
&mqd_mem_obj))
return NULL;
@@ -90,21 +87,15 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
return mqd_mem_obj;
}
-static int init_mqd(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
- int retval;
uint64_t addr;
struct vi_mqd *m;
- struct kfd_dev *kfd = mm->dev;
-
- *mqd_mem_obj = allocate_mqd(kfd, q);
- if (!*mqd_mem_obj)
- return -ENOMEM;
- m = (struct vi_mqd *) (*mqd_mem_obj)->cpu_ptr;
- addr = (*mqd_mem_obj)->gpu_addr;
+ m = (struct vi_mqd *) mqd_mem_obj->cpu_ptr;
+ addr = mqd_mem_obj->gpu_addr;
memset(m, 0, sizeof(struct vi_mqd));
@@ -159,9 +150,7 @@ static int init_mqd(struct mqd_manager *mm, void **mqd,
*mqd = m;
if (gart_addr)
*gart_addr = addr;
- retval = mm->update_mqd(mm, m, q);
-
- return retval;
+ mm->update_mqd(mm, m, q);
}
static int load_mqd(struct mqd_manager *mm, void *mqd,
@@ -177,7 +166,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
wptr_shift, wptr_mask, mms);
}
-static int __update_mqd(struct mqd_manager *mm, void *mqd,
+static void __update_mqd(struct mqd_manager *mm, void *mqd,
struct queue_properties *q, unsigned int mtype,
unsigned int atc_bit)
{
@@ -245,21 +234,19 @@ static int __update_mqd(struct mqd_manager *mm, void *mqd,
set_priority(m, q);
q->is_active = QUEUE_IS_ACTIVE(*q);
-
- return 0;
}
-static int update_mqd(struct mqd_manager *mm, void *mqd,
+static void update_mqd(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
- return __update_mqd(mm, mqd, q, MTYPE_CC, 1);
+ __update_mqd(mm, mqd, q, MTYPE_CC, 1);
}
-static int update_mqd_tonga(struct mqd_manager *mm, void *mqd,
+static void update_mqd_tonga(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
- return __update_mqd(mm, mqd, q, MTYPE_UC, 0);
+ __update_mqd(mm, mqd, q, MTYPE_UC, 0);
}
static int destroy_mqd(struct mqd_manager *mm, void *mqd,
@@ -272,7 +259,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
pipe_id, queue_id);
}
-static void uninit_mqd(struct mqd_manager *mm, void *mqd,
+static void free_mqd(struct mqd_manager *mm, void *mqd,
struct kfd_mem_obj *mqd_mem_obj)
{
kfd_gtt_sa_free(mm->dev, mqd_mem_obj);
@@ -309,61 +296,44 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,
return 0;
}
-static int init_mqd_hiq(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd_hiq(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
struct vi_mqd *m;
- int retval = init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
-
- if (retval != 0)
- return retval;
+ init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
m = get_mqd(*mqd);
m->cp_hqd_pq_control |= 1 << CP_HQD_PQ_CONTROL__PRIV_STATE__SHIFT |
1 << CP_HQD_PQ_CONTROL__KMD_QUEUE__SHIFT;
-
- return retval;
}
-static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
+static void update_mqd_hiq(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct vi_mqd *m;
- int retval = __update_mqd(mm, mqd, q, MTYPE_UC, 0);
-
- if (retval != 0)
- return retval;
+ __update_mqd(mm, mqd, q, MTYPE_UC, 0);
m = get_mqd(mqd);
m->cp_hqd_vmid = q->vmid;
- return retval;
}
-static int init_mqd_sdma(struct mqd_manager *mm, void **mqd,
- struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
+static void init_mqd_sdma(struct mqd_manager *mm, void **mqd,
+ struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
struct queue_properties *q)
{
- int retval;
struct vi_sdma_mqd *m;
- struct kfd_dev *dev = mm->dev;
-
- *mqd_mem_obj = allocate_sdma_mqd(dev, q);
- if (!*mqd_mem_obj)
- return -ENOMEM;
- m = (struct vi_sdma_mqd *) (*mqd_mem_obj)->cpu_ptr;
+ m = (struct vi_sdma_mqd *) mqd_mem_obj->cpu_ptr;
memset(m, 0, sizeof(struct vi_sdma_mqd));
*mqd = m;
if (gart_addr)
- *gart_addr = (*mqd_mem_obj)->gpu_addr;
+ *gart_addr = mqd_mem_obj->gpu_addr;
- retval = mm->update_mqd(mm, m, q);
-
- return retval;
+ mm->update_mqd(mm, m, q);
}
static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
@@ -375,7 +345,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
mms);
}
-static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
+static void update_mqd_sdma(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
struct vi_sdma_mqd *m;
@@ -400,8 +370,6 @@ static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
m->sdma_queue_id = q->sdma_queue_id;
q->is_active = QUEUE_IS_ACTIVE(*q);
-
- return 0;
}
/*
@@ -458,8 +426,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
switch (type) {
case KFD_MQD_TYPE_CP:
case KFD_MQD_TYPE_COMPUTE:
+ mqd->allocate_mqd = allocate_mqd;
mqd->init_mqd = init_mqd;
- mqd->uninit_mqd = uninit_mqd;
+ mqd->free_mqd = free_mqd;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd;
mqd->destroy_mqd = destroy_mqd;
@@ -471,8 +440,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_HIQ:
+ mqd->allocate_mqd = allocate_hiq_mqd;
mqd->init_mqd = init_mqd_hiq;
- mqd->uninit_mqd = uninit_mqd_hiq_sdma;
+ mqd->free_mqd = free_mqd_hiq_sdma;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd_hiq;
mqd->destroy_mqd = destroy_mqd;
@@ -483,8 +453,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_DIQ:
+ mqd->allocate_mqd = allocate_hiq_mqd;
mqd->init_mqd = init_mqd_hiq;
- mqd->uninit_mqd = uninit_mqd;
+ mqd->free_mqd = free_mqd;
mqd->load_mqd = load_mqd;
mqd->update_mqd = update_mqd_hiq;
mqd->destroy_mqd = destroy_mqd;
@@ -495,8 +466,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
#endif
break;
case KFD_MQD_TYPE_SDMA:
+ mqd->allocate_mqd = allocate_sdma_mqd;
mqd->init_mqd = init_mqd_sdma;
- mqd->uninit_mqd = uninit_mqd_hiq_sdma;
+ mqd->free_mqd = free_mqd_hiq_sdma;
mqd->load_mqd = load_mqd_sdma;
mqd->update_mqd = update_mqd_sdma;
mqd->destroy_mqd = destroy_mqd_sdma;
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency
[not found] ` <1559845507-3052-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
@ 2019-06-06 18:25 ` Zeng, Oak
2019-06-06 18:25 ` [PATCH 6/6] drm/amdkfd: Fix sdma queue allocate race condition Zeng, Oak
2019-06-06 20:13 ` [PATCH 4/6] drm/amdkfd: Separate mqd allocation and initialization Kuehling, Felix
2 siblings, 0 replies; 8+ messages in thread
From: Zeng, Oak @ 2019-06-06 18:25 UTC (permalink / raw)
To: amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Kuehling, Felix, Freehill, Chris, Zeng, Oak,
Liu, Alex
The idea to break the circular lock dependency is to move allocate_mqd
out of dqm lock protection. See callstack #1 below.
[ 59.510149] [drm] Initialized amdgpu 3.30.0 20150101 for 0000:04:00.0 on minor 0
[ 513.604034] ======================================================
[ 513.604205] WARNING: possible circular locking dependency detected
[ 513.604375] 4.18.0-kfd-root #2 Tainted: G W
[ 513.604530] ------------------------------------------------------
[ 513.604699] kswapd0/611 is trying to acquire lock:
[ 513.604840] 00000000d254022e (&dqm->lock_hidden){+.+.}, at: evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.605150]
but task is already holding lock:
[ 513.605307] 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
[ 513.605540]
which lock already depends on the new lock.
[ 513.605747]
the existing dependency chain (in reverse order) is:
[ 513.605944]
-> #4 (&anon_vma->rwsem){++++}:
[ 513.606106] __vma_adjust+0x147/0x7f0
[ 513.606231] __split_vma+0x179/0x190
[ 513.606353] mprotect_fixup+0x217/0x260
[ 513.606553] do_mprotect_pkey+0x211/0x380
[ 513.606752] __x64_sys_mprotect+0x1b/0x20
[ 513.606954] do_syscall_64+0x50/0x1a0
[ 513.607149] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.607380]
-> #3 (&mapping->i_mmap_rwsem){++++}:
[ 513.607678] rmap_walk_file+0x1f0/0x280
[ 513.607887] page_referenced+0xdd/0x180
[ 513.608081] shrink_page_list+0x853/0xcb0
[ 513.608279] shrink_inactive_list+0x33b/0x700
[ 513.608483] shrink_node_memcg+0x37a/0x7f0
[ 513.608682] shrink_node+0xd8/0x490
[ 513.608869] balance_pgdat+0x18b/0x3b0
[ 513.609062] kswapd+0x203/0x5c0
[ 513.609241] kthread+0x100/0x140
[ 513.609420] ret_from_fork+0x24/0x30
[ 513.609607]
-> #2 (fs_reclaim){+.+.}:
[ 513.609883] kmem_cache_alloc_trace+0x34/0x2e0
[ 513.610093] reservation_object_reserve_shared+0x139/0x300
[ 513.610326] ttm_bo_init_reserved+0x291/0x480 [ttm]
[ 513.610567] amdgpu_bo_do_create+0x1d2/0x650 [amdgpu]
[ 513.610811] amdgpu_bo_create+0x40/0x1f0 [amdgpu]
[ 513.611041] amdgpu_bo_create_reserved+0x249/0x2d0 [amdgpu]
[ 513.611290] amdgpu_bo_create_kernel+0x12/0x70 [amdgpu]
[ 513.611584] amdgpu_ttm_init+0x2cb/0x560 [amdgpu]
[ 513.611823] gmc_v9_0_sw_init+0x400/0x750 [amdgpu]
[ 513.612491] amdgpu_device_init+0x14eb/0x1990 [amdgpu]
[ 513.612730] amdgpu_driver_load_kms+0x78/0x290 [amdgpu]
[ 513.612958] drm_dev_register+0x111/0x1a0
[ 513.613171] amdgpu_pci_probe+0x11c/0x1e0 [amdgpu]
[ 513.613389] local_pci_probe+0x3f/0x90
[ 513.613581] pci_device_probe+0x102/0x1c0
[ 513.613779] driver_probe_device+0x2a7/0x480
[ 513.613984] __driver_attach+0x10a/0x110
[ 513.614179] bus_for_each_dev+0x67/0xc0
[ 513.614372] bus_add_driver+0x1eb/0x260
[ 513.614565] driver_register+0x5b/0xe0
[ 513.614756] do_one_initcall+0xac/0x357
[ 513.614952] do_init_module+0x5b/0x213
[ 513.615145] load_module+0x2542/0x2d30
[ 513.615337] __do_sys_finit_module+0xd2/0x100
[ 513.615541] do_syscall_64+0x50/0x1a0
[ 513.615731] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.615963]
-> #1 (reservation_ww_class_mutex){+.+.}:
[ 513.616293] amdgpu_amdkfd_alloc_gtt_mem+0xcf/0x2c0 [amdgpu]
[ 513.616554] init_mqd+0x223/0x260 [amdgpu]
[ 513.616779] create_queue_nocpsch+0x4d9/0x600 [amdgpu]
[ 513.617031] pqm_create_queue+0x37c/0x520 [amdgpu]
[ 513.617270] kfd_ioctl_create_queue+0x2f9/0x650 [amdgpu]
[ 513.617522] kfd_ioctl+0x202/0x350 [amdgpu]
[ 513.617724] do_vfs_ioctl+0x9f/0x6c0
[ 513.617914] ksys_ioctl+0x66/0x70
[ 513.618095] __x64_sys_ioctl+0x16/0x20
[ 513.618286] do_syscall_64+0x50/0x1a0
[ 513.618476] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 513.618695]
-> #0 (&dqm->lock_hidden){+.+.}:
[ 513.618984] __mutex_lock+0x98/0x970
[ 513.619197] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.619459] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
[ 513.619710] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
[ 513.620103] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
[ 513.620363] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
[ 513.620614] __mmu_notifier_invalidate_range_start+0x70/0xb0
[ 513.620851] try_to_unmap_one+0x7fc/0x8f0
[ 513.621049] rmap_walk_anon+0x121/0x290
[ 513.621242] try_to_unmap+0x93/0xf0
[ 513.621428] shrink_page_list+0x606/0xcb0
[ 513.621625] shrink_inactive_list+0x33b/0x700
[ 513.621835] shrink_node_memcg+0x37a/0x7f0
[ 513.622034] shrink_node+0xd8/0x490
[ 513.622219] balance_pgdat+0x18b/0x3b0
[ 513.622410] kswapd+0x203/0x5c0
[ 513.622589] kthread+0x100/0x140
[ 513.622769] ret_from_fork+0x24/0x30
[ 513.622957]
other info that might help us debug this:
[ 513.623354] Chain exists of:
&dqm->lock_hidden --> &mapping->i_mmap_rwsem --> &anon_vma->rwsem
[ 513.623900] Possible unsafe locking scenario:
[ 513.624189] CPU0 CPU1
[ 513.624397] ---- ----
[ 513.624594] lock(&anon_vma->rwsem);
[ 513.624771] lock(&mapping->i_mmap_rwsem);
[ 513.625020] lock(&anon_vma->rwsem);
[ 513.625253] lock(&dqm->lock_hidden);
[ 513.625433]
*** DEADLOCK ***
[ 513.625783] 3 locks held by kswapd0/611:
[ 513.625967] #0: 00000000f14edf84 (fs_reclaim){+.+.}, at: __fs_reclaim_acquire+0x5/0x30
[ 513.626309] #1: 00000000961547fc (&anon_vma->rwsem){++++}, at: page_lock_anon_vma_read+0xe4/0x250
[ 513.626671] #2: 0000000067b5cd12 (srcu){....}, at: __mmu_notifier_invalidate_range_start+0x5/0xb0
[ 513.627037]
stack backtrace:
[ 513.627292] CPU: 0 PID: 611 Comm: kswapd0 Tainted: G W 4.18.0-kfd-root #2
[ 513.627632] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 513.627990] Call Trace:
[ 513.628143] dump_stack+0x7c/0xbb
[ 513.628315] print_circular_bug.isra.37+0x21b/0x228
[ 513.628581] __lock_acquire+0xf7d/0x1470
[ 513.628782] ? unwind_next_frame+0x6c/0x4f0
[ 513.628974] ? lock_acquire+0xec/0x1e0
[ 513.629154] lock_acquire+0xec/0x1e0
[ 513.629357] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.629587] __mutex_lock+0x98/0x970
[ 513.629790] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630047] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630309] ? evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630562] evict_process_queues_nocpsch+0x26/0x140 [amdgpu]
[ 513.630816] kfd_process_evict_queues+0x3b/0xb0 [amdgpu]
[ 513.631057] kgd2kfd_quiesce_mm+0x1c/0x40 [amdgpu]
[ 513.631288] amdgpu_amdkfd_evict_userptr+0x38/0x70 [amdgpu]
[ 513.631536] amdgpu_mn_invalidate_range_start_hsa+0xa6/0xc0 [amdgpu]
[ 513.632076] __mmu_notifier_invalidate_range_start+0x70/0xb0
[ 513.632299] try_to_unmap_one+0x7fc/0x8f0
[ 513.632487] ? page_lock_anon_vma_read+0x68/0x250
[ 513.632690] rmap_walk_anon+0x121/0x290
[ 513.632875] try_to_unmap+0x93/0xf0
[ 513.633050] ? page_remove_rmap+0x330/0x330
[ 513.633239] ? rcu_read_unlock+0x60/0x60
[ 513.633422] ? page_get_anon_vma+0x160/0x160
[ 513.633613] shrink_page_list+0x606/0xcb0
[ 513.633800] shrink_inactive_list+0x33b/0x700
[ 513.633997] shrink_node_memcg+0x37a/0x7f0
[ 513.634186] ? shrink_node+0xd8/0x490
[ 513.634363] shrink_node+0xd8/0x490
[ 513.634537] balance_pgdat+0x18b/0x3b0
[ 513.634718] kswapd+0x203/0x5c0
[ 513.634887] ? wait_woken+0xb0/0xb0
[ 513.635062] kthread+0x100/0x140
[ 513.635231] ? balance_pgdat+0x3b0/0x3b0
[ 513.635414] ? kthread_delayed_work_timer_fn+0x80/0x80
[ 513.635626] ret_from_fork+0x24/0x30
[ 513.636042] Evicting PASID 32768 queues
[ 513.936236] Restoring PASID 32768 queues
[ 524.708912] Evicting PASID 32768 queues
[ 524.999875] Restoring PASID 32768 queues
Change-Id: I334c8c9329be12e468ea7aabc878842ec003bd8e
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
---
drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 10d4f4f..166636c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -274,6 +274,12 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
print_queue(q);
+ mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
+ q->properties.type)];
+ q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
+ if (!q->mqd_mem_obj)
+ return -ENOMEM;
+
dqm_lock(dqm);
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
@@ -299,8 +305,6 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
- mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
- q->properties.type)];
if (q->properties.type == KFD_QUEUE_TYPE_COMPUTE) {
retval = allocate_hqd(dqm, q);
if (retval)
@@ -319,9 +323,6 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
if (retval)
goto out_deallocate_hqd;
- q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
- if (!q->mqd_mem_obj)
- goto out_deallocate_doorbell;
mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
&q->gart_mqd_addr, &q->properties);
if (q->properties.is_active) {
@@ -333,7 +334,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe,
q->queue, &q->properties, current->mm);
if (retval)
- goto out_free_mqd;
+ goto out_deallocate_doorbell;
}
list_add(&q->list, &qpd->queues_list);
@@ -353,10 +354,9 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
dqm->total_queue_count++;
pr_debug("Total of %d queues are accountable so far\n",
dqm->total_queue_count);
- goto out_unlock;
+ dqm_unlock(dqm);
+ return retval;
-out_free_mqd:
- mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
out_deallocate_doorbell:
deallocate_doorbell(qpd, q);
out_deallocate_hqd:
@@ -370,6 +370,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
deallocate_vmid(dqm, qpd, q);
out_unlock:
dqm_unlock(dqm);
+ mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
return retval;
}
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 6/6] drm/amdkfd: Fix sdma queue allocate race condition
[not found] ` <1559845507-3052-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-06 18:25 ` [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency Zeng, Oak
@ 2019-06-06 18:25 ` Zeng, Oak
[not found] ` <1559845507-3052-3-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-06 20:13 ` [PATCH 4/6] drm/amdkfd: Separate mqd allocation and initialization Kuehling, Felix
2 siblings, 1 reply; 8+ messages in thread
From: Zeng, Oak @ 2019-06-06 18:25 UTC (permalink / raw)
To: amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Kuehling, Felix, Freehill, Chris, Zeng, Oak,
Liu, Alex
SDMA queue allocation requires the dqm lock at it modify
the global dqm members. Move up the dqm_lock so sdma
queue allocation is enclosed in the critical section. Move
mqd allocation out of critical section to avoid circular
lock dependency.
Change-Id: I96abd42eae6e77c82a5ba1b8e600af3efe8d791d
Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
---
.../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 24 +++++++++++-----------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 166636c..cd259b8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1133,23 +1133,27 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
if (dqm->total_queue_count >= max_num_of_queues_per_device) {
pr_warn("Can't create new usermode queue because %d queues were already created\n",
dqm->total_queue_count);
- retval = -EPERM;
- goto out;
+ return -EPERM;
}
+ mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
+ q->properties.type)];
+ q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
+ if (!q->mqd_mem_obj)
+ return -ENOMEM;
+
+ dqm_lock(dqm);
if (q->properties.type == KFD_QUEUE_TYPE_SDMA ||
q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {
retval = allocate_sdma_queue(dqm, q);
if (retval)
- goto out;
+ goto out_unlock;
}
retval = allocate_doorbell(qpd, q);
if (retval)
goto out_deallocate_sdma_queue;
- mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
- q->properties.type)];
/*
* Eviction state logic: mark all queues as evicted, even ones
* not currently active. Restoring inactive queues later only
@@ -1161,12 +1165,8 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
q->properties.tba_addr = qpd->tba_addr;
q->properties.tma_addr = qpd->tma_addr;
- q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
- if (!q->mqd_mem_obj)
- goto out_deallocate_doorbell;
mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
&q->gart_mqd_addr, &q->properties);
- dqm_lock(dqm);
list_add(&q->list, &qpd->queues_list);
qpd->queue_count++;
@@ -1192,13 +1192,13 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
dqm_unlock(dqm);
return retval;
-out_deallocate_doorbell:
- deallocate_doorbell(qpd, q);
out_deallocate_sdma_queue:
if (q->properties.type == KFD_QUEUE_TYPE_SDMA ||
q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
deallocate_sdma_queue(dqm, q);
-out:
+out_unlock:
+ dqm_unlock(dqm);
+ mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
return retval;
}
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 4/6] drm/amdkfd: Separate mqd allocation and initialization
[not found] ` <1559845507-3052-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-06 18:25 ` [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency Zeng, Oak
2019-06-06 18:25 ` [PATCH 6/6] drm/amdkfd: Fix sdma queue allocate race condition Zeng, Oak
@ 2019-06-06 20:13 ` Kuehling, Felix
2 siblings, 0 replies; 8+ messages in thread
From: Kuehling, Felix @ 2019-06-06 20:13 UTC (permalink / raw)
To: Zeng, Oak,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Freehill, Chris, Liu, Alex
On 2019-06-06 2:25 p.m., Zeng, Oak wrote:
> Introduce a new mqd allocation interface and split the original
> init_mqd function into two functions: allocate_mqd and init_mqd.
> Also renamed uninit_mqd to free_mqd. This is preparation work to
> fix a circular lock dependency.
>
> Change-Id: I26e53ee1abcdd688ad11d35b433da77e3fa1bee7
> Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
> ---
> .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 34 ++++-----
> drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c | 16 ++--
> drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c | 4 +-
> drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h | 18 +++--
> drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c | 78 +++++++------------
> drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c | 78 +++++++------------
> drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c | 88 ++++++++--------------
> 7 files changed, 124 insertions(+), 192 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> index 3c042eb..10d4f4f 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> @@ -319,11 +319,11 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> if (retval)
> goto out_deallocate_hqd;
>
> - retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
> - &q->gart_mqd_addr, &q->properties);
> - if (retval)
> + q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
> + if (!q->mqd_mem_obj)
> goto out_deallocate_doorbell;
You need to set retval = -ENOMEM here.
> -
> + mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
> + &q->gart_mqd_addr, &q->properties);
> if (q->properties.is_active) {
>
> if (WARN(q->process->mm != current->mm,
> @@ -333,7 +333,7 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe,
> q->queue, &q->properties, current->mm);
> if (retval)
> - goto out_uninit_mqd;
> + goto out_free_mqd;
> }
>
> list_add(&q->list, &qpd->queues_list);
> @@ -355,8 +355,8 @@ static int create_queue_nocpsch(struct device_queue_manager *dqm,
> dqm->total_queue_count);
> goto out_unlock;
>
> -out_uninit_mqd:
> - mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> +out_free_mqd:
> + mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> out_deallocate_doorbell:
> deallocate_doorbell(qpd, q);
> out_deallocate_hqd:
> @@ -450,7 +450,7 @@ static int destroy_queue_nocpsch_locked(struct device_queue_manager *dqm,
> if (retval == -ETIME)
> qpd->reset_wavefronts = true;
>
> - mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> + mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
>
> list_del(&q->list);
> if (list_empty(&qpd->queues_list)) {
> @@ -527,7 +527,7 @@ static int update_queue(struct device_queue_manager *dqm, struct queue *q)
> }
> }
>
> - retval = mqd_mgr->update_mqd(mqd_mgr, q->mqd, &q->properties);
> + mqd_mgr->update_mqd(mqd_mgr, q->mqd, &q->properties);
I think there is a chance now that retval will remain uninitialized in
this function (in the non-HWS case if a queue is inactive). You should
initialized it to 0 at the start of the function.
Check the other functions as well. Check for compiler warnings. Most of
these types of problems should be caught by the compiler.
>
> /*
> * check active state vs. the previous state and modify
> @@ -1160,11 +1160,11 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
> dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
> q->properties.tba_addr = qpd->tba_addr;
> q->properties.tma_addr = qpd->tma_addr;
> - retval = mqd_mgr->init_mqd(mqd_mgr, &q->mqd, &q->mqd_mem_obj,
> - &q->gart_mqd_addr, &q->properties);
> - if (retval)
> + q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
> + if (!q->mqd_mem_obj)
> goto out_deallocate_doorbell;
retval = -ENOMEM;
> -
> + mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
> + &q->gart_mqd_addr, &q->properties);
> dqm_lock(dqm);
>
> list_add(&q->list, &qpd->queues_list);
> @@ -1373,8 +1373,8 @@ static int destroy_queue_cpsch(struct device_queue_manager *dqm,
>
> dqm_unlock(dqm);
>
> - /* Do uninit_mqd after dqm_unlock(dqm) to avoid circular locking */
> - mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> + /* Do free_mqd after dqm_unlock(dqm) to avoid circular locking */
> + mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
>
> return retval;
>
> @@ -1615,14 +1615,14 @@ static int process_termination_cpsch(struct device_queue_manager *dqm,
> kfd_dec_compute_active(dqm->dev);
>
> /* Lastly, free mqd resources.
> - * Do uninit_mqd() after dqm_unlock to avoid circular locking.
> + * Do free_mqd() after dqm_unlock to avoid circular locking.
> */
> list_for_each_entry_safe(q, next, &qpd->queues_list, list) {
> mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
> q->properties.type)];
> list_del(&q->list);
> qpd->queue_count--;
> - mqd_mgr->uninit_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> + mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> }
>
> return retval;
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> index 1cc03b3..229500c 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c
> @@ -132,13 +132,14 @@ static bool initialize(struct kernel_queue *kq, struct kfd_dev *dev,
> kq->queue->device = dev;
> kq->queue->process = kfd_get_process(current);
>
> - retval = kq->mqd_mgr->init_mqd(kq->mqd_mgr, &kq->queue->mqd,
> - &kq->queue->mqd_mem_obj,
> + kq->queue->mqd_mem_obj = kq->mqd_mgr->allocate_mqd(kq->mqd_mgr->dev,
> + &kq->queue->properties);
> + if (!kq->queue->mqd_mem_obj)
> + goto err_allocate_mqd;
retval = -ENOMEM;
Regards,
Felix
> + kq->mqd_mgr->init_mqd(kq->mqd_mgr, &kq->queue->mqd,
> + kq->queue->mqd_mem_obj,
> &kq->queue->gart_mqd_addr,
> &kq->queue->properties);
> - if (retval != 0)
> - goto err_init_mqd;
> -
> /* assign HIQ to HQD */
> if (type == KFD_QUEUE_TYPE_HIQ) {
> pr_debug("Assigning hiq to hqd\n");
> @@ -164,7 +165,8 @@ static bool initialize(struct kernel_queue *kq, struct kfd_dev *dev,
>
> return true;
> err_alloc_fence:
> -err_init_mqd:
> + kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd, kq->queue->mqd_mem_obj);
> +err_allocate_mqd:
> uninit_queue(kq->queue);
> err_init_queue:
> kfd_gtt_sa_free(dev, kq->wptr_mem);
> @@ -193,7 +195,7 @@ static void uninitialize(struct kernel_queue *kq)
> else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
> kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
>
> - kq->mqd_mgr->uninit_mqd(kq->mqd_mgr, kq->queue->mqd,
> + kq->mqd_mgr->free_mqd(kq->mqd_mgr, kq->queue->mqd,
> kq->queue->mqd_mem_obj);
>
> kfd_gtt_sa_free(kq->dev, kq->rptr_mem);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
> index cc04b362..d6cf391 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
> @@ -45,7 +45,7 @@ int pipe_priority_map[] = {
> KFD_PIPE_PRIORITY_CS_HIGH
> };
>
> -struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev)
> +struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev, struct queue_properties *q)
> {
> struct kfd_mem_obj *mqd_mem_obj = NULL;
>
> @@ -86,7 +86,7 @@ struct kfd_mem_obj *allocate_sdma_mqd(struct kfd_dev *dev,
> return mqd_mem_obj;
> }
>
> -void uninit_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
> +void free_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
> struct kfd_mem_obj *mqd_mem_obj)
> {
> WARN_ON(!mqd_mem_obj->gtt_mem);
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
> index 66b8c67..550b61e 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
> @@ -39,7 +39,7 @@
> * @destroy_mqd: Destroys the HQD slot and by that preempt the relevant queue.
> * Used only for no cp scheduling.
> *
> - * @uninit_mqd: Releases the mqd buffer from local gpu memory.
> + * @free_mqd: Releases the mqd buffer from local gpu memory.
> *
> * @is_occupied: Checks if the relevant HQD slot is occupied.
> *
> @@ -64,8 +64,11 @@
> */
> extern int pipe_priority_map[];
> struct mqd_manager {
> - int (*init_mqd)(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> + struct kfd_mem_obj* (*allocate_mqd)(struct kfd_dev *kfd,
> + struct queue_properties *q);
> +
> + void (*init_mqd)(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q);
>
> int (*load_mqd)(struct mqd_manager *mm, void *mqd,
> @@ -73,7 +76,7 @@ struct mqd_manager {
> struct queue_properties *p,
> struct mm_struct *mms);
>
> - int (*update_mqd)(struct mqd_manager *mm, void *mqd,
> + void (*update_mqd)(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q);
>
> int (*destroy_mqd)(struct mqd_manager *mm, void *mqd,
> @@ -81,7 +84,7 @@ struct mqd_manager {
> unsigned int timeout, uint32_t pipe_id,
> uint32_t queue_id);
>
> - void (*uninit_mqd)(struct mqd_manager *mm, void *mqd,
> + void (*free_mqd)(struct mqd_manager *mm, void *mqd,
> struct kfd_mem_obj *mqd_mem_obj);
>
> bool (*is_occupied)(struct mqd_manager *mm, void *mqd,
> @@ -102,11 +105,12 @@ struct mqd_manager {
> uint32_t mqd_size;
> };
>
> -struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev);
> +struct kfd_mem_obj *allocate_hiq_mqd(struct kfd_dev *dev,
> + struct queue_properties *q);
>
> struct kfd_mem_obj *allocate_sdma_mqd(struct kfd_dev *dev,
> struct queue_properties *q);
> -void uninit_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
> +void free_mqd_hiq_sdma(struct mqd_manager *mm, void *mqd,
> struct kfd_mem_obj *mqd_mem_obj);
>
> void mqd_symmetrically_map_cu_mask(struct mqd_manager *mm,
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
> index e911438..28876ac 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
> @@ -77,9 +77,6 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
> {
> struct kfd_mem_obj *mqd_mem_obj;
>
> - if (q->type == KFD_QUEUE_TYPE_HIQ)
> - return allocate_hiq_mqd(kfd);
> -
> if (kfd_gtt_sa_allocate(kfd, sizeof(struct cik_mqd),
> &mqd_mem_obj))
> return NULL;
> @@ -87,21 +84,15 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
> return mqd_mem_obj;
> }
>
> -static int init_mqd(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> uint64_t addr;
> struct cik_mqd *m;
> - int retval;
> - struct kfd_dev *kfd = mm->dev;
> -
> - *mqd_mem_obj = allocate_mqd(kfd, q);
> - if (!*mqd_mem_obj)
> - return -ENOMEM;
>
> - m = (struct cik_mqd *) (*mqd_mem_obj)->cpu_ptr;
> - addr = (*mqd_mem_obj)->gpu_addr;
> + m = (struct cik_mqd *) mqd_mem_obj->cpu_ptr;
> + addr = mqd_mem_obj->gpu_addr;
>
> memset(m, 0, ALIGN(sizeof(struct cik_mqd), 256));
>
> @@ -144,37 +135,27 @@ static int init_mqd(struct mqd_manager *mm, void **mqd,
> *mqd = m;
> if (gart_addr)
> *gart_addr = addr;
> - retval = mm->update_mqd(mm, m, q);
> -
> - return retval;
> + mm->update_mqd(mm, m, q);
> }
>
> -static int init_mqd_sdma(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd_sdma(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> - int retval;
> struct cik_sdma_rlc_registers *m;
> - struct kfd_dev *dev = mm->dev;
>
> - *mqd_mem_obj = allocate_sdma_mqd(dev, q);
> - if (!*mqd_mem_obj)
> - return -ENOMEM;
> -
> - m = (struct cik_sdma_rlc_registers *) (*mqd_mem_obj)->cpu_ptr;
> + m = (struct cik_sdma_rlc_registers *) mqd_mem_obj->cpu_ptr;
>
> memset(m, 0, sizeof(struct cik_sdma_rlc_registers));
>
> *mqd = m;
> if (gart_addr)
> - *gart_addr = (*mqd_mem_obj)->gpu_addr;
> -
> - retval = mm->update_mqd(mm, m, q);
> + *gart_addr = mqd_mem_obj->gpu_addr;
>
> - return retval;
> + mm->update_mqd(mm, m, q);
> }
>
> -static void uninit_mqd(struct mqd_manager *mm, void *mqd,
> +static void free_mqd(struct mqd_manager *mm, void *mqd,
> struct kfd_mem_obj *mqd_mem_obj)
> {
> kfd_gtt_sa_free(mm->dev, mqd_mem_obj);
> @@ -203,7 +184,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
> mms);
> }
>
> -static int __update_mqd(struct mqd_manager *mm, void *mqd,
> +static void __update_mqd(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q, unsigned int atc_bit)
> {
> struct cik_mqd *m;
> @@ -237,23 +218,21 @@ static int __update_mqd(struct mqd_manager *mm, void *mqd,
> set_priority(m, q);
>
> q->is_active = QUEUE_IS_ACTIVE(*q);
> -
> - return 0;
> }
>
> -static int update_mqd(struct mqd_manager *mm, void *mqd,
> +static void update_mqd(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> - return __update_mqd(mm, mqd, q, 1);
> + __update_mqd(mm, mqd, q, 1);
> }
>
> -static int update_mqd_hawaii(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_hawaii(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> - return __update_mqd(mm, mqd, q, 0);
> + __update_mqd(mm, mqd, q, 0);
> }
>
> -static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct cik_sdma_rlc_registers *m;
> @@ -278,8 +257,6 @@ static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> m->sdma_queue_id = q->sdma_queue_id;
>
> q->is_active = QUEUE_IS_ACTIVE(*q);
> -
> - return 0;
> }
>
> static int destroy_mqd(struct mqd_manager *mm, void *mqd,
> @@ -326,14 +303,14 @@ static bool is_occupied_sdma(struct mqd_manager *mm, void *mqd,
> * queues but with different initial values.
> */
>
> -static int init_mqd_hiq(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd_hiq(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> - return init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
> + init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
> }
>
> -static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct cik_mqd *m;
> @@ -360,7 +337,6 @@ static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> q->is_active = QUEUE_IS_ACTIVE(*q);
>
> set_priority(m, q);
> - return 0;
> }
>
> #if defined(CONFIG_DEBUG_FS)
> @@ -399,8 +375,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
> switch (type) {
> case KFD_MQD_TYPE_CP:
> case KFD_MQD_TYPE_COMPUTE:
> + mqd->allocate_mqd = allocate_mqd;
> mqd->init_mqd = init_mqd;
> - mqd->uninit_mqd = uninit_mqd;
> + mqd->free_mqd = free_mqd;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd;
> mqd->destroy_mqd = destroy_mqd;
> @@ -411,8 +388,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_HIQ:
> + mqd->allocate_mqd = allocate_hiq_mqd;
> mqd->init_mqd = init_mqd_hiq;
> - mqd->uninit_mqd = uninit_mqd_hiq_sdma;
> + mqd->free_mqd = free_mqd_hiq_sdma;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd_hiq;
> mqd->destroy_mqd = destroy_mqd;
> @@ -423,8 +401,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_DIQ:
> + mqd->allocate_mqd = allocate_hiq_mqd;
> mqd->init_mqd = init_mqd_hiq;
> - mqd->uninit_mqd = uninit_mqd;
> + mqd->free_mqd = free_mqd;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd_hiq;
> mqd->destroy_mqd = destroy_mqd;
> @@ -435,8 +414,9 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_SDMA:
> + mqd->allocate_mqd = allocate_sdma_mqd;
> mqd->init_mqd = init_mqd_sdma;
> - mqd->uninit_mqd = uninit_mqd_hiq_sdma;
> + mqd->free_mqd = free_mqd_hiq_sdma;
> mqd->load_mqd = load_mqd_sdma;
> mqd->update_mqd = update_mqd_sdma;
> mqd->destroy_mqd = destroy_mqd_sdma;
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> index 818944b..0c58f91 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
> @@ -79,9 +79,6 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
> int retval;
> struct kfd_mem_obj *mqd_mem_obj = NULL;
>
> - if (q->type == KFD_QUEUE_TYPE_HIQ)
> - return allocate_hiq_mqd(kfd);
> -
> /* From V9, for CWSR, the control stack is located on the next page
> * boundary after the mqd, we will use the gtt allocation function
> * instead of sub-allocation function.
> @@ -110,21 +107,15 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
>
> }
>
> -static int init_mqd(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> - int retval;
> uint64_t addr;
> struct v9_mqd *m;
> - struct kfd_dev *kfd = mm->dev;
> -
> - *mqd_mem_obj = allocate_mqd(kfd, q);
> - if (!*mqd_mem_obj)
> - return -ENOMEM;
>
> - m = (struct v9_mqd *) (*mqd_mem_obj)->cpu_ptr;
> - addr = (*mqd_mem_obj)->gpu_addr;
> + m = (struct v9_mqd *) mqd_mem_obj->cpu_ptr;
> + addr = mqd_mem_obj->gpu_addr;
>
> memset(m, 0, sizeof(struct v9_mqd));
>
> @@ -173,9 +164,7 @@ static int init_mqd(struct mqd_manager *mm, void **mqd,
> *mqd = m;
> if (gart_addr)
> *gart_addr = addr;
> - retval = mm->update_mqd(mm, m, q);
> -
> - return retval;
> + mm->update_mqd(mm, m, q);
> }
>
> static int load_mqd(struct mqd_manager *mm, void *mqd,
> @@ -190,7 +179,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
> wptr_shift, 0, mms);
> }
>
> -static int update_mqd(struct mqd_manager *mm, void *mqd,
> +static void update_mqd(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct v9_mqd *m;
> @@ -252,8 +241,6 @@ static int update_mqd(struct mqd_manager *mm, void *mqd,
> set_priority(m, q);
>
> q->is_active = QUEUE_IS_ACTIVE(*q);
> -
> - return 0;
> }
>
>
> @@ -267,7 +254,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
> pipe_id, queue_id);
> }
>
> -static void uninit_mqd(struct mqd_manager *mm, void *mqd,
> +static void free_mqd(struct mqd_manager *mm, void *mqd,
> struct kfd_mem_obj *mqd_mem_obj)
> {
> struct kfd_dev *kfd = mm->dev;
> @@ -311,62 +298,47 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,
> return 0;
> }
>
> -static int init_mqd_hiq(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd_hiq(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> struct v9_mqd *m;
> - int retval = init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
>
> - if (retval != 0)
> - return retval;
> + init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
>
> m = get_mqd(*mqd);
>
> m->cp_hqd_pq_control |= 1 << CP_HQD_PQ_CONTROL__PRIV_STATE__SHIFT |
> 1 << CP_HQD_PQ_CONTROL__KMD_QUEUE__SHIFT;
> -
> - return retval;
> }
>
> -static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct v9_mqd *m;
> - int retval = update_mqd(mm, mqd, q);
>
> - if (retval != 0)
> - return retval;
> + update_mqd(mm, mqd, q);
>
> /* TODO: what's the point? update_mqd already does this. */
> m = get_mqd(mqd);
> m->cp_hqd_vmid = q->vmid;
> - return retval;
> }
>
> -static int init_mqd_sdma(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd_sdma(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> - int retval;
> struct v9_sdma_mqd *m;
> - struct kfd_dev *dev = mm->dev;
> -
> - *mqd_mem_obj = allocate_sdma_mqd(dev, q);
> - if (!*mqd_mem_obj)
> - return -ENOMEM;
>
> - m = (struct v9_sdma_mqd *) (*mqd_mem_obj)->cpu_ptr;
> + m = (struct v9_sdma_mqd *) mqd_mem_obj->cpu_ptr;
>
> memset(m, 0, sizeof(struct v9_sdma_mqd));
>
> *mqd = m;
> if (gart_addr)
> - *gart_addr = (*mqd_mem_obj)->gpu_addr;
> + *gart_addr = mqd_mem_obj->gpu_addr;
>
> - retval = mm->update_mqd(mm, m, q);
> -
> - return retval;
> + mm->update_mqd(mm, m, q);
> }
>
> static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
> @@ -380,7 +352,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
>
> #define SDMA_RLC_DUMMY_DEFAULT 0xf
>
> -static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct v9_sdma_mqd *m;
> @@ -404,8 +376,6 @@ static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> m->sdmax_rlcx_dummy_reg = SDMA_RLC_DUMMY_DEFAULT;
>
> q->is_active = QUEUE_IS_ACTIVE(*q);
> -
> - return 0;
> }
>
> /*
> @@ -462,8 +432,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
> switch (type) {
> case KFD_MQD_TYPE_CP:
> case KFD_MQD_TYPE_COMPUTE:
> + mqd->allocate_mqd = allocate_mqd;
> mqd->init_mqd = init_mqd;
> - mqd->uninit_mqd = uninit_mqd;
> + mqd->free_mqd = free_mqd;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd;
> mqd->destroy_mqd = destroy_mqd;
> @@ -475,8 +446,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_HIQ:
> + mqd->allocate_mqd = allocate_hiq_mqd;
> mqd->init_mqd = init_mqd_hiq;
> - mqd->uninit_mqd = uninit_mqd_hiq_sdma;
> + mqd->free_mqd = free_mqd_hiq_sdma;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd_hiq;
> mqd->destroy_mqd = destroy_mqd;
> @@ -487,8 +459,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_DIQ:
> + mqd->allocate_mqd = allocate_hiq_mqd;
> mqd->init_mqd = init_mqd_hiq;
> - mqd->uninit_mqd = uninit_mqd;
> + mqd->free_mqd = free_mqd;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd_hiq;
> mqd->destroy_mqd = destroy_mqd;
> @@ -499,8 +472,9 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_SDMA:
> + mqd->allocate_mqd = allocate_sdma_mqd;
> mqd->init_mqd = init_mqd_sdma;
> - mqd->uninit_mqd = uninit_mqd_hiq_sdma;
> + mqd->free_mqd = free_mqd_hiq_sdma;
> mqd->load_mqd = load_mqd_sdma;
> mqd->update_mqd = update_mqd_sdma;
> mqd->destroy_mqd = destroy_mqd_sdma;
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> index 00e6a59..7d144f5 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
> @@ -80,9 +80,6 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
> {
> struct kfd_mem_obj *mqd_mem_obj;
>
> - if (q->type == KFD_QUEUE_TYPE_HIQ)
> - return allocate_hiq_mqd(kfd);
> -
> if (kfd_gtt_sa_allocate(kfd, sizeof(struct vi_mqd),
> &mqd_mem_obj))
> return NULL;
> @@ -90,21 +87,15 @@ static struct kfd_mem_obj *allocate_mqd(struct kfd_dev *kfd,
> return mqd_mem_obj;
> }
>
> -static int init_mqd(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> - int retval;
> uint64_t addr;
> struct vi_mqd *m;
> - struct kfd_dev *kfd = mm->dev;
> -
> - *mqd_mem_obj = allocate_mqd(kfd, q);
> - if (!*mqd_mem_obj)
> - return -ENOMEM;
>
> - m = (struct vi_mqd *) (*mqd_mem_obj)->cpu_ptr;
> - addr = (*mqd_mem_obj)->gpu_addr;
> + m = (struct vi_mqd *) mqd_mem_obj->cpu_ptr;
> + addr = mqd_mem_obj->gpu_addr;
>
> memset(m, 0, sizeof(struct vi_mqd));
>
> @@ -159,9 +150,7 @@ static int init_mqd(struct mqd_manager *mm, void **mqd,
> *mqd = m;
> if (gart_addr)
> *gart_addr = addr;
> - retval = mm->update_mqd(mm, m, q);
> -
> - return retval;
> + mm->update_mqd(mm, m, q);
> }
>
> static int load_mqd(struct mqd_manager *mm, void *mqd,
> @@ -177,7 +166,7 @@ static int load_mqd(struct mqd_manager *mm, void *mqd,
> wptr_shift, wptr_mask, mms);
> }
>
> -static int __update_mqd(struct mqd_manager *mm, void *mqd,
> +static void __update_mqd(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q, unsigned int mtype,
> unsigned int atc_bit)
> {
> @@ -245,21 +234,19 @@ static int __update_mqd(struct mqd_manager *mm, void *mqd,
> set_priority(m, q);
>
> q->is_active = QUEUE_IS_ACTIVE(*q);
> -
> - return 0;
> }
>
>
> -static int update_mqd(struct mqd_manager *mm, void *mqd,
> +static void update_mqd(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> - return __update_mqd(mm, mqd, q, MTYPE_CC, 1);
> + __update_mqd(mm, mqd, q, MTYPE_CC, 1);
> }
>
> -static int update_mqd_tonga(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_tonga(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> - return __update_mqd(mm, mqd, q, MTYPE_UC, 0);
> + __update_mqd(mm, mqd, q, MTYPE_UC, 0);
> }
>
> static int destroy_mqd(struct mqd_manager *mm, void *mqd,
> @@ -272,7 +259,7 @@ static int destroy_mqd(struct mqd_manager *mm, void *mqd,
> pipe_id, queue_id);
> }
>
> -static void uninit_mqd(struct mqd_manager *mm, void *mqd,
> +static void free_mqd(struct mqd_manager *mm, void *mqd,
> struct kfd_mem_obj *mqd_mem_obj)
> {
> kfd_gtt_sa_free(mm->dev, mqd_mem_obj);
> @@ -309,61 +296,44 @@ static int get_wave_state(struct mqd_manager *mm, void *mqd,
> return 0;
> }
>
> -static int init_mqd_hiq(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd_hiq(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> struct vi_mqd *m;
> - int retval = init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
> -
> - if (retval != 0)
> - return retval;
> + init_mqd(mm, mqd, mqd_mem_obj, gart_addr, q);
>
> m = get_mqd(*mqd);
>
> m->cp_hqd_pq_control |= 1 << CP_HQD_PQ_CONTROL__PRIV_STATE__SHIFT |
> 1 << CP_HQD_PQ_CONTROL__KMD_QUEUE__SHIFT;
> -
> - return retval;
> }
>
> -static int update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_hiq(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct vi_mqd *m;
> - int retval = __update_mqd(mm, mqd, q, MTYPE_UC, 0);
> -
> - if (retval != 0)
> - return retval;
> + __update_mqd(mm, mqd, q, MTYPE_UC, 0);
>
> m = get_mqd(mqd);
> m->cp_hqd_vmid = q->vmid;
> - return retval;
> }
>
> -static int init_mqd_sdma(struct mqd_manager *mm, void **mqd,
> - struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr,
> +static void init_mqd_sdma(struct mqd_manager *mm, void **mqd,
> + struct kfd_mem_obj *mqd_mem_obj, uint64_t *gart_addr,
> struct queue_properties *q)
> {
> - int retval;
> struct vi_sdma_mqd *m;
> - struct kfd_dev *dev = mm->dev;
> -
> - *mqd_mem_obj = allocate_sdma_mqd(dev, q);
> - if (!*mqd_mem_obj)
> - return -ENOMEM;
>
> - m = (struct vi_sdma_mqd *) (*mqd_mem_obj)->cpu_ptr;
> + m = (struct vi_sdma_mqd *) mqd_mem_obj->cpu_ptr;
>
> memset(m, 0, sizeof(struct vi_sdma_mqd));
>
> *mqd = m;
> if (gart_addr)
> - *gart_addr = (*mqd_mem_obj)->gpu_addr;
> + *gart_addr = mqd_mem_obj->gpu_addr;
>
> - retval = mm->update_mqd(mm, m, q);
> -
> - return retval;
> + mm->update_mqd(mm, m, q);
> }
>
> static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
> @@ -375,7 +345,7 @@ static int load_mqd_sdma(struct mqd_manager *mm, void *mqd,
> mms);
> }
>
> -static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> +static void update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> struct queue_properties *q)
> {
> struct vi_sdma_mqd *m;
> @@ -400,8 +370,6 @@ static int update_mqd_sdma(struct mqd_manager *mm, void *mqd,
> m->sdma_queue_id = q->sdma_queue_id;
>
> q->is_active = QUEUE_IS_ACTIVE(*q);
> -
> - return 0;
> }
>
> /*
> @@ -458,8 +426,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
> switch (type) {
> case KFD_MQD_TYPE_CP:
> case KFD_MQD_TYPE_COMPUTE:
> + mqd->allocate_mqd = allocate_mqd;
> mqd->init_mqd = init_mqd;
> - mqd->uninit_mqd = uninit_mqd;
> + mqd->free_mqd = free_mqd;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd;
> mqd->destroy_mqd = destroy_mqd;
> @@ -471,8 +440,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_HIQ:
> + mqd->allocate_mqd = allocate_hiq_mqd;
> mqd->init_mqd = init_mqd_hiq;
> - mqd->uninit_mqd = uninit_mqd_hiq_sdma;
> + mqd->free_mqd = free_mqd_hiq_sdma;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd_hiq;
> mqd->destroy_mqd = destroy_mqd;
> @@ -483,8 +453,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_DIQ:
> + mqd->allocate_mqd = allocate_hiq_mqd;
> mqd->init_mqd = init_mqd_hiq;
> - mqd->uninit_mqd = uninit_mqd;
> + mqd->free_mqd = free_mqd;
> mqd->load_mqd = load_mqd;
> mqd->update_mqd = update_mqd_hiq;
> mqd->destroy_mqd = destroy_mqd;
> @@ -495,8 +466,9 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
> #endif
> break;
> case KFD_MQD_TYPE_SDMA:
> + mqd->allocate_mqd = allocate_sdma_mqd;
> mqd->init_mqd = init_mqd_sdma;
> - mqd->uninit_mqd = uninit_mqd_hiq_sdma;
> + mqd->free_mqd = free_mqd_hiq_sdma;
> mqd->load_mqd = load_mqd_sdma;
> mqd->update_mqd = update_mqd_sdma;
> mqd->destroy_mqd = destroy_mqd_sdma;
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 6/6] drm/amdkfd: Fix sdma queue allocate race condition
[not found] ` <1559845507-3052-3-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
@ 2019-06-06 20:17 ` Kuehling, Felix
0 siblings, 0 replies; 8+ messages in thread
From: Kuehling, Felix @ 2019-06-06 20:17 UTC (permalink / raw)
To: Zeng, Oak,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
Cc: Zhao, Yong, Freehill, Chris, Liu, Alex
Patches 5 and 6 are Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
On 2019-06-06 2:25 p.m., Zeng, Oak wrote:
> SDMA queue allocation requires the dqm lock at it modify
> the global dqm members. Move up the dqm_lock so sdma
> queue allocation is enclosed in the critical section. Move
> mqd allocation out of critical section to avoid circular
> lock dependency.
>
> Change-Id: I96abd42eae6e77c82a5ba1b8e600af3efe8d791d
> Signed-off-by: Oak Zeng <Oak.Zeng@amd.com>
> ---
> .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 24 +++++++++++-----------
> 1 file changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> index 166636c..cd259b8 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
> @@ -1133,23 +1133,27 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
> if (dqm->total_queue_count >= max_num_of_queues_per_device) {
> pr_warn("Can't create new usermode queue because %d queues were already created\n",
> dqm->total_queue_count);
> - retval = -EPERM;
> - goto out;
> + return -EPERM;
> }
>
> + mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
> + q->properties.type)];
> + q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
> + if (!q->mqd_mem_obj)
> + return -ENOMEM;
> +
> + dqm_lock(dqm);
> if (q->properties.type == KFD_QUEUE_TYPE_SDMA ||
> q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI) {
> retval = allocate_sdma_queue(dqm, q);
> if (retval)
> - goto out;
> + goto out_unlock;
> }
>
> retval = allocate_doorbell(qpd, q);
> if (retval)
> goto out_deallocate_sdma_queue;
>
> - mqd_mgr = dqm->mqd_mgrs[get_mqd_type_from_queue_type(
> - q->properties.type)];
> /*
> * Eviction state logic: mark all queues as evicted, even ones
> * not currently active. Restoring inactive queues later only
> @@ -1161,12 +1165,8 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
> dqm->asic_ops.init_sdma_vm(dqm, q, qpd);
> q->properties.tba_addr = qpd->tba_addr;
> q->properties.tma_addr = qpd->tma_addr;
> - q->mqd_mem_obj = mqd_mgr->allocate_mqd(mqd_mgr->dev, &q->properties);
> - if (!q->mqd_mem_obj)
> - goto out_deallocate_doorbell;
> mqd_mgr->init_mqd(mqd_mgr, &q->mqd, q->mqd_mem_obj,
> &q->gart_mqd_addr, &q->properties);
> - dqm_lock(dqm);
>
> list_add(&q->list, &qpd->queues_list);
> qpd->queue_count++;
> @@ -1192,13 +1192,13 @@ static int create_queue_cpsch(struct device_queue_manager *dqm, struct queue *q,
> dqm_unlock(dqm);
> return retval;
>
> -out_deallocate_doorbell:
> - deallocate_doorbell(qpd, q);
> out_deallocate_sdma_queue:
> if (q->properties.type == KFD_QUEUE_TYPE_SDMA ||
> q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
> deallocate_sdma_queue(dqm, q);
> -out:
> +out_unlock:
> + dqm_unlock(dqm);
> + mqd_mgr->free_mqd(mqd_mgr, q->mqd, q->mqd_mem_obj);
> return retval;
> }
>
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2019-06-06 20:17 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-06-06 18:25 [PATCH 4/6] drm/amdkfd: Separate mqd allocation and initialization Zeng, Oak
[not found] ` <1559845507-3052-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-06 18:25 ` [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency Zeng, Oak
2019-06-06 18:25 ` [PATCH 6/6] drm/amdkfd: Fix sdma queue allocate race condition Zeng, Oak
[not found] ` <1559845507-3052-3-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-06 20:17 ` Kuehling, Felix
2019-06-06 20:13 ` [PATCH 4/6] drm/amdkfd: Separate mqd allocation and initialization Kuehling, Felix
-- strict thread matches above, loose matches on Subject: below --
2019-06-05 16:06 [PATCH 1/6] drm/amdkfd: Only initialize sdma vm for sdma queues Zeng, Oak
[not found] ` <1559750793-16608-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-05 16:06 ` [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency Zeng, Oak
[not found] ` <1559750793-16608-5-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-05 22:20 ` Kuehling, Felix
2019-06-04 2:52 [PATCH 1/6] drm/amdkfd: Only initialize sdma vm for sdma queues Zeng, Oak
[not found] ` <1559616755-13116-1-git-send-email-Oak.Zeng-5C7GfCeVMHo@public.gmane.org>
2019-06-04 2:52 ` [PATCH 5/6] drm/amdkfd: Fix a circular lock dependency Zeng, Oak
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox