| Bug ID | 96271 |
|---|---|
| Summary | TF2: GPU lockup on HD 7950 |
| Product | Mesa |
| Version | git |
| Hardware | x86-64 (AMD64) |
| OS | Linux (All) |
| Status | NEW |
| Severity | normal |
| Priority | medium |
| Component | Drivers/Gallium/radeonsi |
| Assignee | dri-devel@lists.freedesktop.org |
| Reporter | hi+freedesktop@stephenliang.pw |
| QA Contact | dri-devel@lists.freedesktop.org |
Created attachment 124165 [details] dmesg on kernel 4.7 I'm getting a GPU lockup on Team Fortress 2 which seems to be related to bug #80419 since my dmesg is the same as comment #44 but this is occurring on Team Fortress 2 rather than XCOM. If I run native TF2 with GALLIUM_DDEBUG=3D800, I cannot reproduce the bug b= ut the frame rate drops significantly. With GALLIUM_DDEBUG=3D"800 noflush&quo= t;, I was able to reproduce the hang. This leads me to believe that the issue is related h= ow high the FPS is running.=20 My kernel is: Linux localhost.localdomain 4.4.9-300.fc23.x86_64 I've also tried 4.6 and 4.7, all of which hangs. My ATI driver is: 7.7.99-3.20160524git040a7b8.fc23 (latest commit in git for radeon) I've also tried 7.6.1-3.20160215gitd41fccc.fc23 (the latest stable from fed= ora 23), all of which hangs. My mesa version is: 11.3.0-0.37.git357495b.fc23 I've also tried 11.1.0-2.20151218.fc23 (the latest stable from fedora 23), = all of which hangs. I've attached the GALLIUM_DDEBUG output and dmesg. One curious thing is tha= t on kernel 4.7, the kernel detects a hang and is able to reset the GPU resultin= g in a temporary freeze that eventually resumes. However, the freeze will come a= gain after a short while (and resumes). As a data point, I don't get any lockups while playing Skyrim on Wine.
Created attachment 124166 [details]<=
/span>
output from GALLIUM_DDEBUG=3D"800 noflush" steam steam://rungamei=
d/440
One more thing, I did run an apitrace and replayed the apitrac= e to see if that reproduces the hang. The replay didn't hang my machine initially but after running it a 2-3 times, it was able to hang the machine. I would attach the apitrace but it is 2-3 GB and the hang is several hundreds of thousands of frames in (and it's not reproducible every time during replay, just sometim= es).
If the trace reproduces the hang at least occasionally, it may= still be useful. Can you upload it to Google Drive or a similar service?
Here's the trace: https://drive.google.com/file/d/0B9JbOVmvb9ugd3ZlbGJRUVFDd= mc/view?usp=3Dsharing SHA Sum: 7a368bb2a95dd939df604e7c8f239953e2957ec9 The hang occurs near the end after the team selection screen while walking = out. If the GPU doesn't hang, then you'll see it stutter and then eventually recover. This particular trace was with a GPU soft reset so it recovered.=20 I'll see if I can get another one without GPU soft reset.
I just ran the apitrace 6 times in a row and it finally hung o= n the 6th try, so it's fairly difficult to get apitrace to hang the GPU but it eventually doe= s do it.=20 On this run, I'm using: Kernel: 4.4.9-300.fc23.x86_64 with radeon.lockup_timeout=3D0 xorg-x11-drv-ati: 7.6.1-3.20160215gitd41fccc.fc23 strace on apitrace: strace: Process 5913 attached wait4(5914, strace on glretrace: strace: Process 5914 attached futex(0xff92ec, FUTEX_WAIT_PRIVATE, 5359, NULL
Created attachment 124197 [details]
Four different dmesg outputs during a hang with GPU soft reset enabled
I posted the previous comment too soon. The hang while using apitrace hangs
everything except the mouse cursor. I'm unable to switch to text mode (ctrl=
+f1)
and there is nothing new in dmesg other than the bootup messages (I was abl=
e to
get this via SSH).
As an aside, I've also attached 4 different GPU hang messages from dmesg fr=
om
various different hangs while I was playing TF2.
Does the issue go away if you set any of these options? R600_DEBUG=3Dnoce,nohyperz
Created attachment 124225 [details]
Kernel log, dmesg, gallium debug logs during GPU hang and system crash
No good but I've got quite a bit of information! I tried with this command:
R600_DEBUG=3Dnoce,nohyperz GALLIUM_DDEBUG=3D"800 noflush" steam
steam://rungameid/440
The GPU recovered and I was able to alt-tab to gnome terminal and grab the
dmesg. See "dmesg_after_alt_tab.txt".
I then continued to play the game and it hung again. This time, the entire
screen froze with only the mouse being able to move. Luckily, this time I w=
as
able to drop to TTY4, and grab a dmesg. See "dmesg_after_hard_hang.txt=
", there
is certainly a lot more output here with quite a bit of call traces which I=
've
never seen before.
Now, I decided to restart X since the mouse still moved so I wondered if th=
is
is an X issue. So I ran "sudo service gdm restart" and it crashed=
the system
with a calltrace displayed on screen on text mode. On restart, I checked the
kernel logs and I can see a lot of call traces, so it seems whatever happen=
ed
ended up tainting the kernel requiring the system to need a hard reset. See
"kernel_log_during_gdm_restart.txt".
Finally, since I ran this with GALLIUM_DDEBUG=3D"800 noflush", I =
was able to
collect another gallium debug that goes along with this. I really hope this
information helps capture the state of the system at the time of crash. Ple=
ase
let me know if you want me to try anything else.
Hi Stephen, if you still have the same machine/setup maybe wor= th to try the apitrace 1) that I mentioned here https://bugs.freedesktop.org/show_b= ug.cgi?id=3D100712#c7 . I wonder if it is the same issue. About "dmesg_after_hard_hang.txt" / "kernel_log_during_gdm_r= estart.txt" all these exact same backtraces are happening because the ring 0(GFX) is still stuck after gpu reset. So acceleration is marked as disabled. But then mesa clear some buffers being cached before reset and it hits WARN_ON in radeon_ttm_bo_destroy, see https://cgit.freedesktop.org= /~agd5f/linux/tree/drivers/gpu/drm/radeon/radeon_object.c?h=3Damd-staging-4= .9#n72. I recently submitted a patch to change it to a WARN_ON_ONCE. But the real problem is the ring 0 stalled in the first place. For me I fou= nd that setting R600_DEBUG=3Dnowc workarounds the problem. Also I have not found any full dmesg log in the attachments, especially inf= os generated at startup.
This old commit should fix the lockups: https://cgit.freedesktop.org/mesa/mesa/commit= /?id=3D6dc96de303290e8d1fc294da478c4f370be98dea
| What | Removed | Added |
|---|---|---|
| Resolution | --- | FIXED |
| Status | NEW | RESOLVED |
Assuming this was fixed as per comment 10 and closing. Please reopen if this is not the case.