| Bug ID | 89785 |
|---|---|
| Summary | GPU Fault 147 and Ring Stalls and Tests Fail in Pillars of Eternity |
| Product | Mesa |
| Version | 10.5 |
| Hardware | x86-64 (AMD64) |
| OS | Linux (All) |
| Status | NEW |
| Severity | normal |
| Priority | medium |
| Component | Drivers/Gallium/radeonsi |
| Assignee | dri-devel@lists.freedesktop.org |
| Reporter | matt.scheirer@gmail.com |
| QA Contact | dri-devel@lists.freedesktop.org |
Created attachment 114656 [details]
Kernel log of GPU Hang
On my r9 290 GPU, the newly released Pillars of Eternity causes a GPU fault and
kernel panic on 10.5.1. Works fine with an Intel part. Requires a REISUB to
reset the system every time.
Attached is the kernel log of the crashing behavior - it is reproducible every
time I run the game as soon as it hits a loading screen. Kernel is 3.19.2.
> radeon 0000:01:00.0: VM_CONTEXT1_=
PROTECTION_FAULT_ADDR 0x01000000
> radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C0C8001
> VM fault (0x01, vmid 6) at page 16777216, read from 'TC6' (0x54433600)=
(200)
Looks like a shader tries to read from virtual address 0x1000000000, which =
is
probably not what was intended, so the shader code might be incorrect.
Please run the game with the environment variable R600_DEBUG=3Dvs,gs,ps ,
redirect its stderr output to a file and attach that file here after the ha=
ng.
The game is capturing stderr and won't give me any shader output, the syntax I was using was R600_DEBUG=vs,gs,ps ./PillarsOfEternity 2> ~/poe-crash-log.txt, also tried &> log and only got its Unity output. Other Gallium tunables like gallium_hud work, so it seems just to be internal stderr capture in Mono or Unity. Not well versed in either enough to know if there is a way to circumvent them, though, and Google isn't helping. Any other options? It seems really bizarre that this game works on Intel, Nvidia (blob), and OSX drivers if it has broken GLSL shaders.
Created attachment 114675 [details] player.log R9 270X for crash after char creation The STDERR output is redirected to ~/.config/unity3d/.config/unity3d/Obsidian Entertainment/Pillars of Eternity/player.log I'm experiencing crashes in Pillars of Eternity as well with an R9 270X card. Sometimes they happen on the loading screen, sometimes directly after character creation. Attached is the gzipped player.log file of my latest crash (right after character creation, using R600_DEBUG=vs,gs,ps). While sometimes I can REISUB after the screen turns black, most of the time (and also with the crash to which the attached file corresponds) the only way to go is a hard reset. For this reason I'm also not sure if the relevant information was written to disc, or if the system froze before that... I'm using the shim linked below, as Pillars of Eternity fails to detect the VRAM size, but the crashes also happen without this workaround. https://github.com/dscharrer/void/blob/master/hacks/glamdmeminfo.c (Description: https://forums.obsidian.net/topic/71852-linux-crashes-after-character-creation/)
Here is the bug: s_mov_b64 vcc, s[6:7] ; Store pointer for s_load_dword* in vcc <snip> v_cmpx_le_f32_e32 vcc, 0, v6 ; Kill instructions v_cmpx_le_f32_e32 vcc, 0, v6 v_cmpx_le_f32_e32 vcc, 0, v6 v_cmpx_le_f32_e32 vcc, 0, v6 <snip> s_load_dwordx8 s[8:15], vcc, 0x48 ; vcc no longer contains a pointer. The lowering of kill instructions needs to save and restore VCC if it is live. As a work around, you can try this branch: http://cgit.freedesktop.org/~tstellar/llvm/log/?h=sched-perf-Mar-27-2015 or take the top two commits and apply it to llvm master. I think this will avoid the bug.
While I cannot get portage to compile llvm from git (neither from master, nor from your branch) without getting an error (https://forums.gentoo.org/viewtopic-t-1013718.html), I've tried to apply the commits "R600/SI: Disable register pressure tracking in the scheduler" (11a8738d716fa9f67da9fce2892c30352780e89b) and "XXX: Schedmodul" (b08f2e3be487f399b04363972bfd5dfc05652673) to llvm-3.6. The patches apply and the resulting code compiles and installs fine, but the game still crashes at the same spot.
Created attachment 114688 [details] player.log R9 270X loading screen crash with patches from ~tstellar This is the Player.log when using llvm-3.6 with the two patches 11a8738d716fa9f67da9fce2892c30352780e89b and b08f2e3be487f399b04363972bfd5dfc05652673 from http://cgit.freedesktop.org/~tstellar/llvm/log/?h=sched-perf-Mar-27-2015. This time the crash happened on the loading screen, before character creation.
| What | Removed | Added |
|---|---|---|
| CC | tilman@code-monkey.de |
I uploaded an apitrace that lets me trigger the bug to http://files.code-monkey.de/PillarsOfEternity.trace .
(In reply to Tilman Sauerbeck from comment #7) > I uploaded an apitrace that lets me trigger the bug to > http://files.code-monkey.de/PillarsOfEternity.trace . Please ignore this one; it only caused a GPU fault with drivers that didn't have the fix for bug #88301. With a proper driver (and LLVM trunk r227583; 3.6.0 is known to be "bad"), I can finish character creation and click through the first dialog in the game. Haven't tried doing anything else yet.
I'm terribly sorry. Please also ignore my previous player.log files, as they were recorded with llvm-3.6.0. I now finally managed to get a working installation of llvm git master and mesa 10.5.2 (with a few bolts and nuts from mesa git master), and while the game is stuttering slightly, it indeed doesn't crash with the current git version of llvm, at least not in the parts I've tested up to now.
Created attachment 114780 [details] [review] Fix for bug described in commet 4 This patch should fix the problem I described in comment 4, can you test it?
I can confirm, patch from the last comment fixes this issue on my Radeon HD 7750.
Comment on att= achment 114780 [details] [r= eview] Fix for bug described in commet 4 Review of attachment 114780= [details] [review]: ----------------------------------------------------------------- I think it would be better if SI_KILL could be fixed not to clobber VCC in = the first place, e.g. by using the 64-bit encoding of the v_cmp(x)_lt_f32[0] instruction. But failing that, this looks good. [0] BTW, it's currently using v_cmpx_le_f32, but I think that's incorrect, = as the comment says "Clear this thread from the exec mask if the operand = is negative".
(In reply to Alex B from comment #11) > I can confirm, patch from the last comment fixes this issue on my Radeon HD > 7750. Same here. Pillars of Eternity seems to work now on my system (llvm-3.6.0 with the patch and mesa-10-5.2, Radeon R9 270X).
I built mesa-git and llvm-svn and tested quite a bit of the game, after an hour of play nothing crashes. I then tried to do everything in game that would generate new shaders (casted all the spells, traveled to several zones, changed equipment, etc) to see if anything triggered the bug, and it seems to be resolved on my end. Here is the player.log of around five minutes of gameplay containing the debug output from llvm. Its about 20MB big, so I threw it on Google Drive. Framerates are good, no graphical glitches, and no crashes. https://drive.google.com/file/d/0ByAFDLY1V3W2T1ZUYUREZHFjRkE/view?usp=sharing So this is fixed on my end upstream.
| What | Removed | Added |
|---|---|---|
| Status | NEW | RESOLVED |
| Resolution | --- | FIXED |
At least, three people said: "the issue is fixed" an= d I can confirm that "Pillars of Eternity" works like a charm on my end. I think it's = enough for closing this very old issue (almost 2 years ago).