From mboxrd@z Thu Jan 1 00:00:00 1970
From: bugzilla-daemon@freedesktop.org
Subject: [Bug 109692] deadlock occurs during GPU reset
Date: Thu, 21 Feb 2019 17:23:00 +0000
Message-ID:
References:
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="===============0351976424=="
Return-path:
Received: from culpepper.freedesktop.org (culpepper.freedesktop.org
[IPv6:2610:10:20:722:a800:ff:fe98:4b55])
by gabe.freedesktop.org (Postfix) with ESMTP id AD2F08919C
for ; Thu, 21 Feb 2019 17:23:00 +0000 (UTC)
In-Reply-To:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Errors-To: dri-devel-bounces@lists.freedesktop.org
Sender: "dri-devel"
To: dri-devel@lists.freedesktop.org
List-Id: dri-devel@lists.freedesktop.org
--===============0351976424==
Content-Type: multipart/alternative; boundary="15507697800.dcAeaCa.8678"
Content-Transfer-Encoding: 7bit
--15507697800.dcAeaCa.8678
Date: Thu, 21 Feb 2019 17:23:00 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://bugs.freedesktop.org/
Auto-Submitted: auto-generated
https://bugs.freedesktop.org/show_bug.cgi?id=3D109692
Andrey Grodzovsky changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |christian.koenig@amd.com
--- Comment #9 from Andrey Grodzovsky ---
Adding Christian to consult on solution to the problem:
The deadlock happens because we lock fence->lock from dma_fence_signal foll=
owed
by sched->job_list_lock from drm_sched_process_job on one hand while on the
other do reverse order locking in drm_sched_stop, locking first
sched->job_list_lock and then fence->lock when trying to remove the cb from
dma_fence_remove_callback.
I see 2 possible solutions -=20
1) Change ring_mirror_list to lock-less list (e.g. RCU list), problem is I
don't think there is a readily available implementation of doubly linked
lockless list, I didn't do to much search though...=20
2) Go back to removing a job from ring_mirror_list in drm_sched_job_finish=
-
as I see it for this to work we need to add a wait_queue (signal) to
drm_sched_job which will signal AFTER job is removed from ring_mirror_list =
in
drm_sched_job_finish, then in drm_sched_stop when iterating over
ring_mirror_list you build a new list of all jobs in progress (for them
dma_fence_remove_callback returns false) and then you wait on all theirs
wait_queues - then you are sure they removed themselves from ring_mirror_li=
st
and you can proceed.=20
Specific to AMDGPU - To insure the jobs are not being freed in
drm_sched_job_finish while we process them in drm_sched_stop it's not enoug=
h to
wait to do cancel_delayed_work_sync(&sched->work_tdr) for this scheduler b=
ut
also for all the other schedulers in the device and even in the hive for XG=
MI
use case.
--=20
You are receiving this mail because:
You are the assignee for the bug.=
--15507697800.dcAeaCa.8678
Date: Thu, 21 Feb 2019 17:23:00 +0000
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://bugs.freedesktop.org/
Auto-Submitted: auto-generated
Andrey Grodzovsky
changed
bug 10969=
2
| What |
Removed |
Added |
| CC |
|
christian.koenig@amd.com
|
Commen=
t # 9
on bug 10969=
2
from Andrey Grodzovsky
Adding Christian to consult on solution to the problem:
The deadlock happens because we lock fence->lock from dma_fence_signal f=
ollowed
by sched->job_list_lock from drm_sched_process_job on one hand while on =
the
other do reverse order locking in drm_sched_stop, locking first
sched->job_list_lock and then fence->lock when trying to remove the c=
b from
dma_fence_remove_callback.
I see 2 possible solutions -=20
1) Change ring_mirror_list to lock-less list (e.g. RCU list), problem is I
don't think there is a readily available implementation of doubly linked
lockless list, I didn't do to much search though...=20
2) Go back to removing a job from ring_mirror_list in drm_sched_job_finish=
-
as I see it for this to work we need to add a wait_queue (signal) to
drm_sched_job which will signal AFTER job is removed from ring_mirror_list =
in
drm_sched_job_finish, then in drm_sched_stop when iterating over
ring_mirror_list you build a new list of all jobs in progress (for them
dma_fence_remove_callback returns false) and then you wait on all theirs
wait_queues - then you are sure they removed themselves from ring_mirror_li=
st
and you can proceed.=20
Specific to AMDGPU - To insure the jobs are not being freed in
drm_sched_job_finish while we process them in drm_sched_stop it's not enoug=
h to
wait to do cancel_delayed_work_sync(&sched->work_tdr) for this sche=
duler but
also for all the other schedulers in the device and even in the hive for XG=
MI
use case.
You are receiving this mail because:
- You are the assignee for the bug.
=
--15507697800.dcAeaCa.8678--
--===============0351976424==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs
IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz
dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs
--===============0351976424==--