From mboxrd@z Thu Jan 1 00:00:00 1970 From: bugzilla-daemon@freedesktop.org Subject: [Bug 100712] ring 0 stalled after bytes_moved_threshold reached - Cap Verde - HD 7770 Date: Wed, 19 Apr 2017 12:03:32 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1801126957==" Return-path: Received: from culpepper.freedesktop.org (culpepper.freedesktop.org [IPv6:2610:10:20:722:a800:ff:fe98:4b55]) by gabe.freedesktop.org (Postfix) with ESMTP id 9458D6E079 for ; Wed, 19 Apr 2017 12:03:32 +0000 (UTC) In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: dri-devel@lists.freedesktop.org List-Id: dri-devel@lists.freedesktop.org --===============1801126957== Content-Type: multipart/alternative; boundary="14926034120.9cCB.9771"; charset="UTF-8" --14926034120.9cCB.9771 Date: Wed, 19 Apr 2017 12:03:32 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated https://bugs.freedesktop.org/show_bug.cgi?id=3D100712 --- Comment #5 from Julien Isorce --- (In reply to Michel D=C3=A4nzer from comment #4) > (In reply to Julien Isorce from comment #0) > > In kernel radeon_object.c::radeon_bo_list_validate, once "bytes_moved > > > bytes_moved_threshold" is reached (this is the case for 850 bo in the s= ame > > list_for_each_entry loop), I can see that radeon_ib_schedule emits a fe= nce > > that it takes more than the radeon.lockup_timeout to be signaled. >=20 > radeon_ib_schedule is called for submitting the command stream from > userspace, not for any BO moves directly, right? >=20 > How did you determine that this hang is directly related to bytes_moved / > bytes_moved_threshold? Maybe it's only indirectly related, e.g. due to the > threshold preventing a BO from being moved to VRAM despite userspace's > preference. >=20 I added a trace and the fence that is not signaled on time is always the one emited by radeon_ib_schedule after that the bytes_moved_threshold is reache= d. But you are right it could be only indirectly related. Here is the sequence I have: ioctl_radeon_cs radeon_bo_list_validate bytes_moved > bytes_moved_threshold(=3D1024*1024ull) 800 bo are not moved from gtt to vram because of that. radeon_cs_ib_vm_chunk radeon_ib_schedule(rdev, &parser->ib, NULL, true); radeon_fence_emit on ring 0 r600_mmio_hdp_flush /ioctl_radeon_cs Then anything calling ttm_bo_wait will block more than the radeon.lockup_timeout because the above fence is not signaled on time. Could it be that something is not flushed properly ? (ref: https://patchwork.kernel.org/patch/5807141/ ? tlb_flush ?)=20 Are you saying that some bos are required to be moved from gtt to vram in o= rder for this fence to be signaled ? As you can see above it happens when vram_usage >=3D half_vram so radeon_bo_get_threshold_for_moves returns 1024*1024, which explains why onl= y 1 or 2 bos can be moved from gtt to vram in that case and why all others are forced to stay in gtt. In the same run of radeon_bo_list_validate there are many calls to ttm_bo_validate with both domain and current_domain as VRAM, this is the ca= se for around 400 bo. Maybe this cause delay for this fence to be signaled, providing vram usage is high too. >=20 > > Also it seems the fence is signaled by swapper after more than 10 secon= ds > > but it is too late. I requires to reduce the "15" param above to 4 to s= ee > > that. >=20 > How does "swapper" (what is that exactly?) signal the fence? My wording was wrong sorry, I should have said "the first entity noticing t= hat the fence is signaled" by calling radeon_fence_activity. swapper is the name for process 0 (idle). I change drm logging to print process name and id: (current->comm, current->pid) >=20 > It might be worth looking into why this happens, though. If domain =3D=3D > current_domain =3D=3D RADEON_GEM_DOMAIN_VRAM, I wouldn't expect ttm_bo_va= lidate > to trigger a blit. I will check though I think I get just confused by a previous trace. --=20 You are receiving this mail because: You are the assignee for the bug.= --14926034120.9cCB.9771 Date: Wed, 19 Apr 2017 12:03:32 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.freedesktop.org/ Auto-Submitted: auto-generated

Commen= t # 5 on bug 10071= 2 from Julien Isorce
(In reply to Michel D=C3=A4nzer from comment #4)
> (In reply to Julien Isorce from comment #0)
> > In kernel radeon_object.c::radeon_bo_list_validate, once "by=
tes_moved >
> > bytes_moved_threshold" is reached (this is the case for 850 =
bo in the same
> > list_for_each_entry loop), I can see that radeon_ib_schedule emit=
s a fence
> > that it takes more than the radeon.lockup_timeout to be signaled.
>=20
> radeon_ib_schedule is called for submitting the command stream from
> userspace, not for any BO moves directly, right?
>=20
> How did you determine that this hang is directly related to bytes_move=
d /
> bytes_moved_threshold? Maybe it's only indirectly related, e.g. due to=
 the
> threshold preventing a BO from being moved to VRAM despite userspace's
> preference.
> 

I added a trace and the fence that is not signaled on time is always the one
emited by radeon_ib_schedule after that the bytes_moved_threshold is reache=
d.
But you are right it could be only indirectly related.

Here is the sequence I have:

ioctl_radeon_cs
  radeon_bo_list_validate
    bytes_moved > bytes_moved_threshold(=3D1024*1024ull)
    800 bo are not moved from gtt to vram because of that.
  radeon_cs_ib_vm_chunk
    radeon_ib_schedule(rdev, &parser->ib, NULL, true);
      radeon_fence_emit on ring 0
      r600_mmio_hdp_flush
/ioctl_radeon_cs

Then anything calling ttm_bo_wait will block more than the
radeon.lockup_timeout because the above fence is not signaled on time.
Could it be that something is not flushed properly ? (ref:
https://patchwork.k=
ernel.org/patch/5807141/ ? tlb_flush ?)=20

Are you saying that some bos are required to be moved from gtt to vram in o=
rder
for this fence to be signaled ?

As you can see above it happens when vram_usage >=3D half_vram so
radeon_bo_get_threshold_for_moves returns 1024*1024, which explains why onl=
y 1
or 2 bos can be moved from gtt to vram in that case and why all others are
forced to stay in gtt.

In the same run of radeon_bo_list_validate there are many calls to
ttm_bo_validate with both domain and current_domain as VRAM, this is the ca=
se
for around 400 bo. Maybe this cause delay for this fence to be signaled,
providing vram usage is high too.

>=20
> > Also it seems the fence is signaled by swapper after more than 10=
 seconds
> > but it is too late. I requires to reduce the "15" param=
 above to 4 to see
> > that.
>=20
> How does "swapper" (what is that exactly?) signal the fence?=


My wording was wrong sorry, I should have said "the first entity notic=
ing that
the fence is signaled" by calling radeon_fence_activity. swapper is th=
e name
for process 0 (idle). I change drm logging to print process name and id:
(current->comm, current->pid)

>=20
> It might be worth looking into why this happens, though. If domain =3D=
=3D
> current_domain =3D=3D RADEON_GEM_DOMAIN_VRAM, I wouldn't expect ttm_bo=
_validate
> to trigger a blit.

I will check though I think I get just confused by a previous trace.


You are receiving this mail because:
  • You are the assignee for the bug.
= --14926034120.9cCB.9771-- --===============1801126957== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== --===============1801126957==--