Reproducible "PGRAPH TLB flush timeout" hang on NV96

Reproducible "PGRAPH TLB flush timeout" hang on NV96

[Maarten Maathuis]

Hi everyone,

I was wondering what to do to dig into this problem further.The kernel
is several weeks old (nouveau tree), it's at commit
000463f13fba6b2f94a5bfcb0d615751ae9c34a0.

As you can see from the mesages below the problem is reproducable to
the point of getting exactly the same error.

Nov 25 17:57:51 madman kernel: [548360.773743] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH TLB flush idle timeout fail: 0x019ffe03
0x00145009 0x0000102d 0x0034db43
Nov 25 19:34:09 madman kernel: [ 5623.952688] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH TLB flush idle timeout fail: 0x019ffe03
0x00145009 0x0000102d 0x0034db43
Nov 25 22:54:47 madman kernel: [ 7854.679007] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH TLB flush idle timeout fail: 0x019ffe03
0x00145009 0x0000102d 0x0034db43
Nov 25 23:04:33 madman kernel: [  540.388597] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH TLB flush idle timeout fail: 0x019ffe03
0x00145009 0x0000102d 0x0034db43

I also updated the kernel to latest because it has the improved parsing:

Nov 25 23:42:57 madman kernel: [  169.014553] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH_STATUS  : 0x019ffe03 BUSY DISPATCH VFETCH
CCACHE_UNK4 STRMOUT_GSCHED_UNK5 UNK14XX UNK24XX_CSCHED UNK1CXX CLIPID
ZCULL ENG2D UNK34XX TPRAST TPROP MP ROP
Nov 25 23:42:57 madman kernel: [  169.014564] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH_VSTATUS0: 0x00145009 VFETCH CCACHE
Nov 25 23:42:57 madman kernel: [  169.014569] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH_VSTATUS1: 0x0000102d MP
Nov 25 23:42:57 madman kernel: [  169.014573] nouveau E[
PGRAPH][0000:01:00.0] PGRAPH_VSTATUS2: 0x0034db43 ENG2D ROP

Do you have any ideas to find out what is causing this?

Sincerely,

Maarten.

-- 
Far away from the primal instinct, the song seems to fade away, the
river get wider between your thoughts and the things we do and say.

Re: Reproducible "PGRAPH TLB flush timeout" hang on NV96

[Marcin Slusarz]

On Sun, Nov 25, 2012 at 11:50:50PM +0100, Maarten Maathuis wrote:
> Hi everyone,
> 
> I was wondering what to do to dig into this problem further.The kernel
> is several weeks old (nouveau tree), it's at commit
> 000463f13fba6b2f94a5bfcb0d615751ae9c34a0.
> 
> As you can see from the mesages below the problem is reproducable to
> the point of getting exactly the same error.

You missed the most important information - how do you reproduce it?
:)

Marcin

Re: Reproducible "PGRAPH TLB flush timeout" hang on NV96

[Maarten Maathuis]

On Mon, Nov 26, 2012 at 7:04 PM, Marcin Slusarz
<marcin.slusarz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Sun, Nov 25, 2012 at 11:50:50PM +0100, Maarten Maathuis wrote:
>> Hi everyone,
>>
>> I was wondering what to do to dig into this problem further.The kernel
>> is several weeks old (nouveau tree), it's at commit
>> 000463f13fba6b2f94a5bfcb0d615751ae9c34a0.
>>
>> As you can see from the mesages below the problem is reproducable to
>> the point of getting exactly the same error.
>
> You missed the most important information - how do you reproduce it?
> :)
>
> Marcin

A specific savegame for a game called freeorion which causes the
problem within one turn typically, if you are interested in it i can
dig it up.

-- 
Far away from the primal instinct, the song seems to fade away, the
river get wider between your thoughts and the things we do and say.

Re: Reproducible "PGRAPH TLB flush timeout" hang on NV96

[Marcin Slusarz]

On Mon, Nov 26, 2012 at 08:16:17PM +0100, Maarten Maathuis wrote:
> On Mon, Nov 26, 2012 at 7:04 PM, Marcin Slusarz
> <marcin.slusarz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > On Sun, Nov 25, 2012 at 11:50:50PM +0100, Maarten Maathuis wrote:
> >> Hi everyone,
> >>
> >> I was wondering what to do to dig into this problem further.The kernel
> >> is several weeks old (nouveau tree), it's at commit
> >> 000463f13fba6b2f94a5bfcb0d615751ae9c34a0.
> >>
> >> As you can see from the mesages below the problem is reproducable to
> >> the point of getting exactly the same error.
> >
> > You missed the most important information - how do you reproduce it?
> > :)
> >
> > Marcin
> 
> A specific savegame for a game called freeorion which causes the
> problem within one turn typically, if you are interested in it i can
> dig it up.

Apitrace log could be useful - if you can reproduce hang with retrace.

Marcin

Re: Reproducible "PGRAPH TLB flush timeout" hang on NV96

[Maarten Maathuis]

On Mon, Nov 26, 2012 at 8:28 PM, Marcin Slusarz
<marcin.slusarz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Mon, Nov 26, 2012 at 08:16:17PM +0100, Maarten Maathuis wrote:
>> On Mon, Nov 26, 2012 at 7:04 PM, Marcin Slusarz
>> <marcin.slusarz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> > On Sun, Nov 25, 2012 at 11:50:50PM +0100, Maarten Maathuis wrote:
>> >> Hi everyone,
>> >>
>> >> I was wondering what to do to dig into this problem further.The kernel
>> >> is several weeks old (nouveau tree), it's at commit
>> >> 000463f13fba6b2f94a5bfcb0d615751ae9c34a0.
>> >>
>> >> As you can see from the mesages below the problem is reproducable to
>> >> the point of getting exactly the same error.
>> >
>> > You missed the most important information - how do you reproduce it?
>> > :)
>> >
>> > Marcin
>>
>> A specific savegame for a game called freeorion which causes the
>> problem within one turn typically, if you are interested in it i can
>> dig it up.
>
> Apitrace log could be useful - if you can reproduce hang with retrace.
>
> Marcin

I can't even get it to run with apitrace (tried even with git master):

apitrace: redirecting dlopen("libGL.so.1", 0x102)
nv50_screen_get_param:181 - unknown PIPE_CAP 76
apitrace: warning: caught signal 11
apitrace: ignoring exception while tracing

For some reason i can't catch it with gdb, the process always exits
and then i cannot get a backtrace.

-- 
Far away from the primal instinct, the song seems to fade away, the
river get wider between your thoughts and the things we do and say.