| Bug ID | 87278 |
|---|---|
| Summary | Packet0 not allowed and GPU fault detected errors with Serious Engine games |
| Product | Mesa |
| Version | git |
| Hardware | Other |
| OS | All |
| Status | NEW |
| Severity | normal |
| Priority | medium |
| Component | Drivers/Gallium/radeonsi |
| Assignee | dri-devel@lists.freedesktop.org |
| Reporter | daniel@constexpr.org |
Created attachment 110808 [details] dmesg output with the GPU fault errors filtered out Running Serious Sam 3 or The Talos Principle spams dmesg with thousands of these errors: [ 6001.212237] radeon 0000:01:00.0: GPU fault detected: 147 0x02528801 [ 6001.212243] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF02192 [ 6001.212246] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x12088001 [ 6001.212249] VM fault (0x01, vmid 9) at page 267395474, read from TC (136) There are also a few "Packet0 not allowed" errors (followed by a hex dump): [15446.473341] radeon 0000:01:00.0: Packet0 not allowed! So far it's only these errors in dmesg - I haven't observed any actual rendering issues, crashes, GPU lockups because of this. I have only attached a filtered kernel log with the GPU fault errors removed - the full log is available at http://constexpr.org/tmp/serious-dmesg.log (140 MiB). Both of these games use the Serious Engine 3.5 (Serious Sam 3) or 4 (The Talos Principle). This is also reproducible with The Talos Principle Public Test which as of now is still available as a free download on Steam. Kernel: 3.18.0-gentoo GPU: Radeon HD 7950 Driver: radeonsi, Mesa 10.5.0-devel (git-ff96537) This might be related to bug 84500 - however those spurious Packet0 have been gone for a while now with updated Mesa - now I got them again but only while running Serious Engine games.
Created attachment 110809 [details]
standard output from The Talos Principle and Serious Sam 3
The standard output has this repeated a few times:
radeon: The kernel rejected CS, see dmesg for more information.
| What | Removed | Added |
|---|---|---|
| CC | haagch@frickel.club |
This also happens on a HD 7970M (pitcairn). 00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09) 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Wimbledon XT [Radeon HD 7970M] (rev ff) On latest mesa git master and both with linux 3.18 and drm-next-3.19. Setting the game to lowest settings doesn't seem to help. When closely looking at the ground the vm faults seem to stop, when looking a little bit in the distance, they start happening again. [47303.471209] radeon 0000:01:00.0: GPU fault detected: 147 0x0fe2c401 [47303.471211] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF03E4E [47303.471212] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x020C4001 [47303.471214] VM fault (0x01, vmid 1) at page 267402830, read from TC (196) [47303.487684] radeon 0000:01:00.0: GPU fault detected: 147 0x09c2c801 [47303.487689] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF03E4E [47303.487691] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x020C8001 [47303.487694] VM fault (0x01, vmid 1) at page 267402830, read from TC (200) [47303.487696] radeon 0000:01:00.0: GPU fault detected: 147 0x09c28401 [47303.487698] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FFFFFFF [47303.487699] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x020C4001 [47303.487701] VM fault (0x01, vmid 1) at page 268435455, read from TC (196) [47303.504293] radeon 0000:01:00.0: GPU fault detected: 147 0x09c24801 [47303.504298] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF03E4E [47303.504300] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x02048001 [47303.504302] VM fault (0x01, vmid 1) at page 267402830, read from TC (72)
Does the environment variable R600_DEBUG=3Dnodma avoid this pr= oblem?
(In reply to Michel D=C3=A4nzer from comment #3) > Does the environment variable R600_DEBUG=3Dnodma= avoid this problem? No, R600_DEBUG=3Dnodma does not help.
(In reply to Daniel Scharrer from comment #0) > Created attachment 110808 [details] > dmesg output with the GPU fault errors filtered out > > Running Serious Sam 3 or The Talos Principle spams dmesg with thousands of > these errors: > > [ 6001.212237] radeon 0000:01:00.0: GPU fault detected: 147 0x02528801 > [ 6001.212243] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR > 0x0FF02192 > [ 6001.212246] radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS > 0x12088001 > [ 6001.212249] VM fault (0x01, vmid 9) at page 267395474, read from TC (136) > > There are also a few "Packet0 not allowed" errors (followed by a hex dump): > > [15446.473341] radeon 0000:01:00.0: Packet0 not allowed! > > So far it's only these errors in dmesg - I haven't observed any actual > rendering issues, crashes, GPU lockups because of this. > > I have only attached a filtered kernel log with the GPU fault errors removed > - the full log is available at http://constexpr.org/tmp/serious-dmesg.log > (140 MiB). > > Both of these games use the Serious Engine 3.5 (Serious Sam 3) or 4 (The > Talos Principle). This is also reproducible with The Talos Principle Public > Test which as of now is still available as a free download on Steam. > > Kernel: 3.18.0-gentoo > GPU: Radeon HD 7950 > Driver: radeonsi, Mesa 10.5.0-devel (git-ff96537) > > This might be related to bug 84500 - however those spurious Packet0 have > been gone for a while now with updated Mesa - now I got them again but only > while running Serious Engine games. I haven't had a look at the log when launching SS3, but for sure it crashes in no time. It crashes in no time once in a game. It could be related to your bug. However, I think the VM and the Packet 0 are different bugs. I'll have a look in the logs if I get something similar.
| What | Removed | Added |
|---|---|---|
| CC | tstellar@gmail.com |
(In reply to Christoph Haag from comment #2) > [47303.471211] radeon 0000:01:00.0: VM_CONTEXT= 1_PROTECTION_FAULT_ADDR=20=20 > 0x0FF03E4E The fact that bits 32-39 of the faulting addresses are FF indicates incorre= ct shader code generation resulting in those address bits being clobbered. Can you generate an apitrace which reproduces the Packet0 error and/or GPUVM faults?
(In reply to Michel D=C3=A4nzer from comment #3) > Does the environment variable R600_DEBUG=3Dnodma= avoid this problem? No change here either. But now that you mention it: IIRC SS3 did sometimes = lock up the GPU before http://cgit.freedesktop.org/mesa/mesa/commit/?= id=3Dae4536b4f71cbe76230ea7edc7eb4d6041e651b4 (In reply to Michel D=C3=A4nzer from comment #6) > Can you generate an apitrace which reproduces th= e Packet0 error and/or GPUVM > faults? Here is one with VM faults from starting The Talos Principle Public Test, j= ust up to the main menu: http://constexpr.org/= tmp/Talos_Demo.trace (149 MiB) Sometimes there is also a Packet0 error at the end. Didn't get it while recording, got it 2/3 times while replaying.
| What | Removed | Added |
|---|---|---|
| CC | tstellar@gmail.com | maraeo@gmail.com |
(In reply to Daniel Scharrer from comment #7) > Here is one with VM faults from starting The Tal= os Principle Public Test, > just up to the main menu: >=20 > http://constexpr= .org/tmp/Talos_Demo.trace (149 MiB) Thanks. The VM faults generated by this apitrace turned out to be a Mesa regression. I bisected it: 5e0fbe1b631d883eb0e033938a534a259c8d95fd is the first bad commit commit 5e0fbe1b631d883eb0e033938a534a259c8d95fd Author: Marek Ol=C5=A1=C3=A1k <marek.olsak@amd.com> Date: Sat Oct 4 20:41:03 2014 +0200 radeonsi: remove vs.ucps_enabled from the shader key Written CLIPDIST outputs are simply disabled in PA_CL_VS_OUT_CNTL.
Note that I'm only getting the VM faults with my Cape Verde ca= rd, not with my Kaveri. Seems to be SI specific. I haven't been able to reproduce the Packet0 error with this apitrace.
Created attachment 11= 1036 [details] [review] possible fix for VM faults Does this patch fix the VM faults?
(In reply to Marek Ol=C5=A1=C3=A1k from comment #10) > Created attachment 111036 [details] [review= ] [review] > possible fix for VM faults >=20 > Does this patch fix the VM faults? I don't think it does for me. I still get radeon 0000:01:00.0: GPU fault detected: 147 0x0342c401 radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF0081A radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x020C4001 VM fault (0x01, vmid 1) at page 267388954, read from TC (196) radeon 0000:01:00.0: GPU fault detected: 147 0x03424401 radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF0081A radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x02044001 VM fault (0x01, vmid 1) at page 267388954, read from TC (68) radeon 0000:01:00.0: GPU fault detected: 147 0x03428401 radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF0081A radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x02084001 VM fault (0x01, vmid 1) at page 267388954, read from TC (132) radeon 0000:01:00.0: GPU fault detected: 147 0x03420401 radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF0081A radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x02004001 VM fault (0x01, vmid 1) at page 267388954, read from TC (4) radeon 0000:01:00.0: GPU fault detected: 147 0x03428401 radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x0FF0081A radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x02084001 VM fault (0x01, vmid 1) at page 267388954, read from TC (132)
Sorry, it has been a couple of weeks, but I confirm I have the same problem with my R9 270X. I'm using latest mesa, drm, ddx from git repositories on a 3.19-rc1 kernel. I'll be attaching my log. I'll also test the patch.
Created attachment 111430 [details]
VM faults and Packet0 error when quitting the current game
on R9 270X with today's latest mesa, drm, ddx from git repositories and kernel
3.19-rc1
(In reply to Alexandre Demers from comment #13) > Created attachment 111430 [details] > VM faults and Packet0 error when quitting the current game > > on R9 270X with today's latest mesa, drm, ddx from git repositories and > kernel 3.19-rc1 Patch tested and I get the same error as before, as Christoph.
same issue, dota2 and with no Packet0 message. kernel 3.18 llvm git mesa git r9-270x/radeonsi it happens fairly frequently (or every time i pick 'morphling') :( i haven't had this problem with 6570 (r600g)... ... Jan 05 06:25:08 -- kernel: switching to power state: Jan 05 06:25:08 -- kernel: ui class: performance Jan 05 06:25:08 -- kernel: internal class: none Jan 05 06:25:08 -- kernel: caps: Jan 05 06:25:08 -- kernel: uvd vclk: 0 dclk: 0 Jan 05 06:25:08 -- kernel: power level 0 sclk: 30000 mclk: 15000 vddc: 875 vddci: 850 pcie gen: 1 Jan 05 06:25:08 -- kernel: power level 1 sclk: 45000 mclk: 140000 vddc: 950 vddci: 1025 pcie gen: 1 Jan 05 06:25:08 -- kernel: power level 2 sclk: 105000 mclk: 140000 vddc: 1163 vddci: 1025 pcie gen: 1 Jan 05 06:25:08 -- kernel: power level 3 sclk: 112000 mclk: 140000 vddc: 1206 vddci: 1025 pcie gen: 1 Jan 05 06:25:08 -- kernel: status: c r Jan 05 06:25:40 -- kernel: IPVS: Creating netns size=2056 id=21 Jan 05 06:25:40 -- kernel: IPVS: ftp: loaded support on port[0] = 21 Jan 05 06:28:59 -- kernel: radeon 0000:01:00.0: GPU fault detected: 147 0x00044401 Jan 05 06:28:59 -- kernel: radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x01000000 Jan 05 06:28:59 -- kernel: radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x04044001 Jan 05 06:28:59 -- kernel: VM fault (0x01, vmid 2) at page 16777216, read from TC (68) Jan 05 06:28:59 -- kernel: radeon 0000:01:00.0: GPU fault detected: 147 0x00044401 Jan 05 06:28:59 -- kernel: radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x01000000 Jan 05 06:28:59 -- kernel: radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x04044001 Jan 05 06:28:59 -- kernel: VM fault (0x01, vmid 2) at page 16777216, read from TC (68) Jan 05 06:29:10 -- kernel: radeon 0000:01:00.0: ring 0 stalled for more than 10293msec Jan 05 06:29:10 -- kernel: radeon 0000:01:00.0: GPU lockup (current fence id 0x00000000002165c7 last fence id 0x00000000002165e9 on ring 0) Jan 05 06:29:10 -- kernel: radeon 0000:01:00.0: failed to get a new IB (-35) Jan 05 06:29:10 -- kernel: [drm:radeon_cs_ib_fill] *ERROR* Failed to get ib ! Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: failed to get a new IB (-35) Jan 05 06:29:11 -- kernel: [drm:radeon_cs_ib_fill] *ERROR* Failed to get ib ! Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: Saved 1355 dwords of commands on ring 0. Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GPU softreset: 0x0000004D Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_STATUS = 0xF7D24028 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_STATUS_SE0 = 0xEFC00000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_STATUS_SE1 = 0xEFC00000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: SRBM_STATUS = 0x200000C0 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: SRBM_STATUS2 = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_008674_CP_STALLED_STAT1 = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_008678_CP_STALLED_STAT2 = 0x40000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_00867C_CP_BUSY_STAT = 0x00008006 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_008680_CP_STAT = 0x80228647 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_00D034_DMA_STATUS_REG = 0x44483106 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_00D834_DMA_STATUS_REG = 0x44C83D57 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_SOFT_RESET=0x0000DDFF Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: SRBM_SOFT_RESET=0x00100100 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_STATUS = 0x00003028 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_STATUS_SE0 = 0x00000006 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GRBM_STATUS_SE1 = 0x00000006 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: SRBM_STATUS = 0x200000C0 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: SRBM_STATUS2 = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_008674_CP_STALLED_STAT1 = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_008678_CP_STALLED_STAT2 = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_00867C_CP_BUSY_STAT = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_008680_CP_STAT = 0x00000000 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_00D034_DMA_STATUS_REG = 0x44C83D57 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: R_00D834_DMA_STATUS_REG = 0x44C83D57 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: GPU reset succeeded, trying to resume Jan 05 06:29:11 -- kernel: [drm] probing gen 2 caps for device 8086:2e31 = 2212501/0 Jan 05 06:29:11 -- kernel: [drm] PCIE GART of 1024M enabled (table at 0x0000000000277000). Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: WB enabled Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: fence driver on ring 0 use gpu addr 0x0000000100000c00 and cpu addr 0xffff8800db08fc00 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: fence driver on ring 1 use gpu addr 0x0000000100000c04 and cpu addr 0xffff8800db08fc04 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: fence driver on ring 2 use gpu addr 0x0000000100000c08 and cpu addr 0xffff8800db08fc08 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: fence driver on ring 3 use gpu addr 0x0000000100000c0c and cpu addr 0xffff8800db08fc0c Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: fence driver on ring 4 use gpu addr 0x0000000100000c10 and cpu addr 0xffff8800db08fc10 Jan 05 06:29:11 -- kernel: radeon 0000:01:00.0: fence driver on ring 5 use gpu addr 0x0000000000075a18 and cpu addr 0xffffc900048b5a18 Jan 05 06:29:11 -- kernel: [drm] ring test on 0 succeeded in 3 usecs Jan 05 06:29:11 -- kernel: [drm] ring test on 1 succeeded in 1 usecs Jan 05 06:29:11 -- kernel: [drm] ring test on 2 succeeded in 1 usecs Jan 05 06:29:11 -- kernel: [drm] ring test on 3 succeeded in 6 usecs Jan 05 06:29:11 -- kernel: [drm] ring test on 4 succeeded in 6 usecs Jan 05 06:29:11 -- kernel: [drm] ring test on 5 succeeded in 2 usecs Jan 05 06:29:11 -- kernel: [drm] UVD initialized successfully. Jan 05 06:29:11 -- kernel: switching from power state: Jan 05 06:29:11 -- kernel: ui class: none Jan 05 06:29:11 -- kernel: internal class: boot Jan 05 06:29:11 -- kernel: caps: Jan 05 06:29:11 -- kernel: uvd vclk: 0 dclk: 0 Jan 05 06:29:11 -- kernel: power level 0 sclk: 15000 mclk: 15000 vddc: 950 vddci: 950 pcie gen: 1 Jan 05 06:29:11 -- kernel: status: c b ...
Created attachment 111751 [details]
sorry i should've attached it
This seems to be fixed with current Mesa git. Can you confirm?=
(In reply to Marek Ol=C5=A1=C3=A1k from comment #17) > This seems to be fixed with current Mesa git. Ca= n you confirm? I played for a while and the problems were gone for me. The performance is still very bad and sometimes there is some graphics/text= ure corruption flickering. But this specific issue here seems to be fixed.
(In reply to Marek Ol=C5=A1=C3=A1k from comment #17) > This seems to be fixed with current Mesa git. Ca= n you confirm? Indeed, no GPU faults and no Packet 0 observered. SS3 doesn't crash the who= le desktop anymore. Do you have any idea what was pushed that may have fixed the bug we were seeing.
I can confirm that the GPU fault errors are gone, but still get Packet0 errors (both in game and in the apitrace from Comment 7). Also, there were still GPU fault errors in The Talos Principle and demo (but not the apitrace) until I also updated LLVM.
(In reply to Daniel Scharrer from comment #20) > I can confirm that the GPU fault errors are gone, but still get Packet0 > errors (both in game and in the apitrace from Comment 7). > > Also, there were still GPU fault errors in The Talos Principle and demo (but > not the apitrace) until I also updated LLVM. Good point about LLVM, because I'm also using yesterday's svn LLVM code.