i915 freakout with latest 3.7 git

i915 freakout with latest 3.7 git

[Heinz Diehl]

Hi,

with latest linus-3.7 git from today, after some time, my machine gets
more and more unresponsible, fanspeed increases, and that's what I see in the logs:

Dec  3 18:08:10 wildsau kernel: [35092.535757] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung
Dec  3 18:08:10 wildsau kernel: [35092.535768] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state
Dec  3 18:08:12 wildsau kernel: [35094.050918] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung
Dec  3 18:08:12 wildsau kernel: [35094.051081] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged!
Dec  3 18:08:12 wildsau kernel: [35094.051086] [drm:i915_reset] *ERROR* Failed to reset chip.

I have never seen that before, up to 3.6.7.

Don't know what information would be important for you, but will
provide anything you'll ask me to.

Thanks,
Heinz.

Re: i915 freakout with latest 3.7 git

[devendra.aaru]

Add more CC's

On Mon, Dec 3, 2012 at 12:39 PM, Heinz Diehl <htd@fancy-poultry.org> wrote:
> Hi,
>
> with latest linus-3.7 git from today, after some time, my machine gets
> more and more unresponsible, fanspeed increases, and that's what I see in the logs:
>
> Dec  3 18:08:10 wildsau kernel: [35092.535757] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung
> Dec  3 18:08:10 wildsau kernel: [35092.535768] [drm] capturing error event; look for more information in /debug/dri/0/i915_error_state
> Dec  3 18:08:12 wildsau kernel: [35094.050918] [drm:i915_hangcheck_hung] *ERROR* Hangcheck timer elapsed... GPU hung
> Dec  3 18:08:12 wildsau kernel: [35094.051081] [drm:i915_reset] *ERROR* GPU hanging too fast, declaring wedged!
> Dec  3 18:08:12 wildsau kernel: [35094.051086] [drm:i915_reset] *ERROR* Failed to reset chip.
>
> I have never seen that before, up to 3.6.7.
>
> Don't know what information would be important for you, but will
> provide anything you'll ask me to.
>
> Thanks,
> Heinz.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 03.12.2012, devendra.aaru wrote: 

> Add more CC's

Thanks!

This is a real showstopper for me, it occurs in every session now. 
Booting with "i915.i915_enable_rc6=0" doesn't help either..

Re: i915 freakout with latest 3.7 git

[Dave Airlie]

On Tue, Dec 4, 2012 at 6:36 PM, Heinz Diehl <htd@fancy-poultry.org> wrote:
> On 03.12.2012, devendra.aaru wrote:
>
>> Add more CC's
>
> Thanks!
>
> This is a real showstopper for me, it occurs in every session now.
> Booting with "i915.i915_enable_rc6=0" doesn't help

https://bugs.freedesktop.org/show_bug.cgi?id=55984

intel guys are as lost as anyone.

Dave.

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Tue, Dec 4, 2012 at 10:27 AM, Dave Airlie <airlied@gmail.com> wrote:
> On Tue, Dec 4, 2012 at 6:36 PM, Heinz Diehl <htd@fancy-poultry.org> wrote:
>> On 03.12.2012, devendra.aaru wrote:
>>
>>> Add more CC's
>>
>> Thanks!
>>
>> This is a real showstopper for me, it occurs in every session now.
>> Booting with "i915.i915_enable_rc6=0" doesn't help
>
> https://bugs.freedesktop.org/show_bug.cgi?id=55984
>
> intel guys are as lost as anyone.

Yeah, if anyone can somewhat reliably reproduce this (you need to
disable rc6 on ilk to not hit another issue which seems much easier to
hit) and bisect it, this would be _very_ much appreciated - we've
pretty much tested all possible "disable stuff" and "revert random
patch" we could thing of, and we can't reproduce these hangs no matter
how hard we bang our heads against this. Atm we're trying to come up
with ways to dump more debug information, but with no clue whatsoever
what's going on that's slow-going.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Tue, Dec 4, 2012 at 10:27 AM, Dave Airlie <airlied@gmail.com> wrote:
> On Tue, Dec 4, 2012 at 6:36 PM, Heinz Diehl <htd@fancy-poultry.org> wrote:
>> On 03.12.2012, devendra.aaru wrote:
>>
>>> Add more CC's
>>
>> Thanks!
>>
>> This is a real showstopper for me, it occurs in every session now.
>> Booting with "i915.i915_enable_rc6=0" doesn't help
>
> https://bugs.freedesktop.org/show_bug.cgi?id=55984
>
> intel guys are as lost as anyone.

Yeah, if anyone can somewhat reliably reproduce this (you need to
disable rc6 on ilk to not hit another issue which seems much easier to
hit) and bisect it, this would be _very_ much appreciated - we've
pretty much tested all possible "disable stuff" and "revert random
patch" we could thing of, and we can't reproduce these hangs no matter
how hard we bang our heads against this. Atm we're trying to come up
with ways to dump more debug information, but with no clue whatsoever
what's going on that's slow-going.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 04.12.2012, Daniel Vetter wrote: 

> Yeah, if anyone can somewhat reliably reproduce this

Ok, I see. So the beginning would be to reliably reproduce the the
hang. I have encountered it in any possbile situasjon, both when
watching videos on Youtube and right after booting the machine and
doing absolutely nothing.

I'll try around a little bit and see if I can find something that
triggers this hang. 

Btw: which kernel is known to be the "last good one"?

> (you need to disable rc6 on ilk to not hit another issue which seems much easier to
> hit) 

Ilk? If this stands for "Ironlake": I'm on Sandybridge.

> and bisect it, this would be _very_ much appreciated - we've
> pretty much tested all possible "disable stuff" and "revert random
> patch" we could thing of, and we can't reproduce these hangs no matter
> how hard we bang our heads against this.

Bisecting will be a pain without being able to reproduce
the hang reliably.

> Atm we're trying to come up with ways to dump more debug
> information, >but with no clue whatsoever what's going on that's slow-going.

Is there anything at the moment I can do to help you to get a grip on
this problem? My machine is a Core i5-420M laptop with 4GB RAM (Asus
U45-JC). 

Heinz

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 04.12.2012, Daniel Vetter wrote: 

> Yeah, if anyone can somewhat reliably reproduce this

Ok, I see. So the beginning would be to reliably reproduce the the
hang. I have encountered it in any possbile situasjon, both when
watching videos on Youtube and right after booting the machine and
doing absolutely nothing.

I'll try around a little bit and see if I can find something that
triggers this hang. 

Btw: which kernel is known to be the "last good one"?

> (you need to disable rc6 on ilk to not hit another issue which seems much easier to
> hit) 

Ilk? If this stands for "Ironlake": I'm on Sandybridge.

> and bisect it, this would be _very_ much appreciated - we've
> pretty much tested all possible "disable stuff" and "revert random
> patch" we could thing of, and we can't reproduce these hangs no matter
> how hard we bang our heads against this.

Bisecting will be a pain without being able to reproduce
the hang reliably.

> Atm we're trying to come up with ways to dump more debug
> information, >but with no clue whatsoever what's going on that's slow-going.

Is there anything at the moment I can do to help you to get a grip on
this problem? My machine is a Core i5-420M laptop with 4GB RAM (Asus
U45-JC). 

Heinz

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Tue, Dec 04, 2012 at 01:35:22PM +0100, Heinz Diehl wrote:
> On 04.12.2012, Daniel Vetter wrote: 
> 
> > Yeah, if anyone can somewhat reliably reproduce this
> 
> Ok, I see. So the beginning would be to reliably reproduce the the
> hang. I have encountered it in any possbile situasjon, both when
> watching videos on Youtube and right after booting the machine and
> doing absolutely nothing.
> 
> I'll try around a little bit and see if I can find something that
> triggers this hang. 
> 
> Btw: which kernel is known to be the "last good one"?

If it's the ilk one we only know that 3.6.x series seems to be solid, and
something in 3.7-rc (probably before -rc1) broke stuff. So not too useful.

> > (you need to disable rc6 on ilk to not hit another issue which seems much easier to
> > hit) 
> 
> Ilk? If this stands for "Ironlake": I'm on Sandybridge.

Hm, then it could very well be something different, so I think we need to
track this one as a separate bug. Can you please file a new one on
bugs.freedesktop.org against DRI -> DRM (Intel) and attach dmesg when
booting with drm.debug=0xe (just so we know what's in your box) plus the
i915_error_state from debugfs once the gpu is hung (if you can get at that
file, reboot kills it).

Thanks, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Lekensteyn]

On Tuesday 04 December 2012 13:35:22 Heinz Diehl wrote:
> Btw: which kernel is known to be the "last good one"?
As mentioned in the linked bug [1], I bisected it to:

commit 504c7267a1e84b157cbd7e9c1b805e1bc0c2c846
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date:   Thu Aug 23 13:12:52 2012 +0100

    drm/i915: Use cpu relocations if the object is in the GTT but not mappable

> > (you need to disable rc6 on ilk to not hit another issue which seems much
> > easier to hit)
> 
> Ilk? If this stands for "Ironlake": I'm on Sandybridge.
> 
> > ...
> 
> Bisecting will be a pain without being able to reproduce
> the hang reliably.
> 
> > Atm we're trying to come up with ways to dump more debug
> > information, >but with no clue whatsoever what's going on that's
> > slow-going.
> Is there anything at the moment I can do to help you to get a grip on
> this problem? My machine is a Core i5-420M laptop with 4GB RAM (Asus
> U45-JC).
i5-420M is not SB, but ILK. i5-2xxx is SB. I have a i5-460M myself. 
i915.i915_enable_rc6=0 worked for me, if it does not work for you, then you 
probably hit another bug.

Peter


 [1]: https://bugs.freedesktop.org/show_bug.cgi?id=55984#c9

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Tue, Dec 4, 2012 at 9:41 PM, Lekensteyn <lekensteyn@gmail.com> wrote:
> On Tuesday 04 December 2012 13:35:22 Heinz Diehl wrote:
>> Btw: which kernel is known to be the "last good one"?
> As mentioned in the linked bug [1], I bisected it to:
>
> commit 504c7267a1e84b157cbd7e9c1b805e1bc0c2c846
> Author: Chris Wilson <chris@chris-wilson.co.uk>
> Date:   Thu Aug 23 13:12:52 2012 +0100
>
>     drm/i915: Use cpu relocations if the object is in the GTT but not mappable

Iirc your issue goes away with rc6=0, the residual bugs we still have
all still happen with rc6 disable, so probably something else. Hence
also why we're asking everyone who can still reproduce to try a
bisect, since with rc6 disabled we've can't reproduce the hang any
more (beforehand we could reproduce it on 3 different ilk machines).
The important part is to not enable rc6 (on ironlake at least) when
bisecting.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 04.12.2012, Lekensteyn wrote: 

> As mentioned in the linked bug [1], I bisected it to:
> 
> commit 504c7267a1e84b157cbd7e9c1b805e1bc0c2c846
> Author: Chris Wilson <chris@chris-wilson.co.uk>
> Date:   Thu Aug 23 13:12:52 2012 +0100
> 
>     drm/i915: Use cpu relocations if the object is in the GTT but not mappable

Ok, but in comment 11 in the same thread you mention that reverting
this patch didn't fix the issue for you:

"Reverting that commit on top of 3.7-rc4 did not fix the hang issue."

> i5-420M is not SB, but ILK. i5-2xxx is SB. I have a i5-460M myself. 

Yes, you're right, my bad! Don't know what I was thinking as I wrote that. I
don't have any i5-420M either, but an i5-450M. It was clearly not my
day..

[htd@wildsau ~]$ cat /proc/cpuinfo | grep model
model	     	 : 37
model name	 : Intel(R) Core(TM) i5 CPU       M 450  @ 2.40GHz
[....]

> i915.i915_enable_rc6=0 worked for me, if it does not work for you, then you 
> probably hit another bug.

I have now i915.i915_enable_rc6=0 in grub.cfg and disabled the XFCE
compositor. Now I'm trying to hit the bug again...

Heinz

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 04.12.2012, Daniel Vetter wrote: 

> The important part is to not enable rc6 (on ironlake at least) when
> bisecting.

A shot in the dark: could it be that all the machines wich encounter
this hang have nvidia's optimus? Mine has. Could that somehow be
related? (I'm by no means a programmer or a kernel hacker..).

Re: i915 freakout with latest 3.7 git

[Lekensteyn]

On Tuesday 04 December 2012 22:08:45 Heinz Diehl wrote:
> Ok, but in comment 11 in the same thread you mention that reverting
> this patch didn't fix the issue for you:
> 
> "Reverting that commit on top of 3.7-rc4 did not fix the hang issue."
The bisected commit was from between rc2 and rc3:
$ git describe 504c7267a1e84b157cbd7e9c1b805e1bc0c2c846
v3.6-rc2-88-g504c726
The fact that reverting that commit does not help implies that some commits 
thereafter also expose the bug.

> > i915.i915_enable_rc6=0 worked for me, if it does not work for you, then
> > you  probably hit another bug.
> 
> I have now i915.i915_enable_rc6=0 in grub.cfg and disabled the XFCE
> compositor. Now I'm trying to hit the bug again...
Do you have a reliable reproduce method? As you can see in the linked bug it 
was caused by relatively low memory pressure combined with high I/O (caching? 
delays? Who knows).

> A shot in the dark: could it be that all the machines wich encounter
> this hang have nvidia's optimus? Mine has. Could that somehow be
> related? (I'm by no means a programmer or a kernel hacker..).
It is unlikely that Optimus has anything to do with this.

Peter

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Tue, Dec 4, 2012 at 11:09 PM, Lekensteyn <lekensteyn@gmail.com> wrote:
> On Tuesday 04 December 2012 22:08:45 Heinz Diehl wrote:
>> Ok, but in comment 11 in the same thread you mention that reverting
>> this patch didn't fix the issue for you:
>>
>> "Reverting that commit on top of 3.7-rc4 did not fix the hang issue."
> The bisected commit was from between rc2 and rc3:
> $ git describe 504c7267a1e84b157cbd7e9c1b805e1bc0c2c846
> v3.6-rc2-88-g504c726

This just means that after -rc2 there are 88 patches until 504c72.
This doesn't mean at all that this patch is included in -rc3 - git
history is non-linear! In fact this commit is only part of the 3.7-rc1
release, so if you just update Linus' tree it will have shown up
somewhere between the 3.6 and 3.7-rc1 tag being pushed out.

> The fact that reverting that commit does not help implies that some commits
> thereafter also expose the bug.

Well, enabling rc6 was merge before the offending commit but still
works around at least a class of bugs. So it's very likely that we're
just hunting down different strawmens ...

>> > i915.i915_enable_rc6=0 worked for me, if it does not work for you, then
>> > you  probably hit another bug.
>>
>> I have now i915.i915_enable_rc6=0 in grub.cfg and disabled the XFCE
>> compositor. Now I'm trying to hit the bug again...
> Do you have a reliable reproduce method? As you can see in the linked bug it
> was caused by relatively low memory pressure combined with high I/O (caching?
> delays? Who knows).

Nope, we could only reproduce quickly with rc6 enabled :(
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 05.12.2012, Lekensteyn wrote:

> > I have now i915.i915_enable_rc6=0 in grub.cfg and disabled the XFCE
> > compositor. Now I'm trying to hit the bug again...

> Do you have a reliable reproduce method? As you can see in the linked bug it
> was caused by relatively low memory pressure combined with high I/O (caching?
> delays? Who knows).

No, unfortunately not. I will do my very best to find out how to
trigger it. For	  now, I'm trying with a script which produces
max. I/O. Will also try by replaying a lot of high resolution videos and
similar.

> It is unlikely that Optimus has anything to do with this.

Ok.

Heinz

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 05.12.2012, Daniel Vetter wrote:

> Nope, we could only reproduce quickly with rc6 enabled :(

Could reproduce it today this way:

 dd if=/dev/zero of=deleteme bs=1M count=50000

while watching several HD videos on Youtube. Just tried	once, so I'm
not shure if this will work all the way. Will try again now.

My "i915_error_state" is here:

http://www.fritha.org/i915/error-01.tar.bz2

Heinz

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 04.12.2012, Daniel Vetter wrote: 

> Yeah, if anyone can somewhat reliably reproduce this

While writing a big file with dd and watching high resolution videos
on youtube, I've managed to reproduce the hang. Unfortunately, it
doesn't occur within seconds. Some playing around is neccessary, and
it takes between 30 sec. and 20 min.

> > Btw: which kernel is known to be the "last good one"?

> If it's the ilk one we only know that 3.6.x series seems to be solid, and
> something in 3.7-rc (probably before -rc1) broke stuff. So not too useful.

I tried 3.6.9 several times over a few hours and could not trigger the
hang, which clearly adds evidence to this statement. I don't want to
scream out too loud, but 3.6.9 seems not to be affected. Will try
some more hours to get a 3.6.9 box to hang, though.. Just in case..

Heinz

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 06.12.2012, Heinz Diehl wrote: 

[....]

Here are some more error-logs, inkl. dmesg after booting with drm
debug options turned on:

http://www.fritha.org/i915/gpu-hang.tar.bz2

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 06.12.2012, Heinz Diehl wrote: 

[....]

Ok, the last one for today. After extensive testing with heavy load
and I/O while watching HD videos, I can almost safely conclude with 
the following:

1.) The hang does *never* occur with 3.6.9 vanilla 

2.) The hang does *always* occur with 3.7-rc8+ / latest git

3.) The hang doesn't occur with 3.7/latest git when 

 Driver "Intel"
 Options "NoAccel" "True" 

in Xorg.xonf is set (with all the drawbacks this introduces). Maybe
this rings a bell for someone..

In all cases, the machine is booted with "i915.i915_enable_rc6=0".

Please contact me if you think I can help to debug this further.

Thanks,
Heinz.

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Fri, Dec 7, 2012 at 9:44 PM, Heinz Diehl <htd-iEI8Y0CNJBYdnm+yROfE0A@public.gmane.org> wrote:
> On 07.12.2012, Daniel Vetter wrote:
>> > I think I can reliably reproduce the hang on my machine now. I have to
>> > try some HD-videos on Youtube while writing a big file with dd. The
>> > hang often occurs withing max. 5 min.
>
>> That sounds pretty awesome: Just to check, is this already with rc6
>> disable? Also, which gpu chip?
>
> This is with latest 3.7-git and i915.i915_enable_rc6=0.
> Attached is a logfile/dmesg after booting with debug options on which
> hopefully shows you the gpu chip.
>
>> Sure, always glad to help excellent bug reporters along. My usual
>> kernel bisect howto is:
>> http://www.reactivated.net/weblog/archives/2006/01/using-git-bisect-to-find-buggy-kernel-patches/
>> It seems to server rather well thus far.

ilk with rc6 disabled, and the two hangs you've attached both die on
the MI_FLUSH in between a 3D primitive and a 2D blit, like all the
other non-rc6 hangs we've seen thus far that indicate that 3.7
regressed. This A looks _very_ good. I'm adding lists again so that
people are updated and can check whether I've analyzed the
error_states correctly. For reference I've uploaded your dmesg and
error_states at

http://people.freedesktop.org/~danvet/stuff/gpu-hang.tar.bz2

> Thanks, I've read it and think that will be pretty easy (technically).
> Am I right to download Linus' tree first, and so compile an 3.7.0-rc1,
> and if I can reproduce the bug with it, it should be a little bit of a
> shorter way to get the offending patch bisected?
>
>> Good luck with the exams!
>
> Thanks! :-)
>
>> Yeah, something in 3.7 seems to have blown up - we have a few reports
>> all claiming that 3.6 is solid, while 3.7 is not :(
>
> I'll try my very best to detect the offending patch. So stay tuned ;-)

Yeah, this would be very good information to move forward with this bug.

Thanks a lot for your hard work in helping with reproducing this bug.

Yours, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Chris Wilson]

On Fri, 7 Dec 2012 22:08:13 +0100, Daniel Vetter <daniel-/w4YWyX8dFk@public.gmane.org> wrote:
> ilk with rc6 disabled, and the two hangs you've attached both die on
> the MI_FLUSH in between a 3D primitive and a 2D blit, like all the
> other non-rc6 hangs we've seen thus far that indicate that 3.7
> regressed. This A looks _very_ good. I'm adding lists again so that
> people are updated and can check whether I've analyzed the
> error_states correctly. For reference I've uploaded your dmesg and
> error_states at

The error states do disappear into a black hole during the execution of
a 3DPRIMITIVE. The similarity between the two appear that the WM kernel
loaded for the 3DPRIMITIVE both appear to be recently bound, and were
the last kernels to be bound in the batch. Coincidence? Maybe, the
INSTDONE in both cases is again the same highly unusual condition
suggesting that the EU died.  However, both error-states also suggest
that a fresh surface was uploaded for the same 3DPRIMITIVE - but I'm
having to guess since the error-state doesn't include the auxiliary
state for me to check. One thing you can try is SNA, which packs its
batches differently with the advantage that more auxiliary state is
included in the error-state. It also packs all the kernels into a
single buffer which will reduce the frequency at which it is paged
out/in. So if you can reproduce with SNA (use Option "AccelMethod"
"SNA" in a device section of your xorg.conf snippet) I expect the
error-state to be quite different and hopefully shed some more light on
the issue.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 07.12.2012, Daniel Vetter wrote: 

[....]

I did a "git bisect" betweeb 3.6 and 3.7-rc8 and ended up with
this. Unfortunately, git can't revert this patch on top of master, sp
I have not been able to test if a revert will cure the problem.

After reading on the net that Peter (Lekensteyn) already ended up with
bisecting the same patch and it didn't work for him reverting it on
top of 3-7-rc4, I'm somewhat clueless..

What else can I do to help finding the cause?

Heinz


[root@wildsau linux-git]# git bisect good
6c085a728cf000ac1865d66f8c9b52935558b328 is the first bad commit
commit 6c085a728cf000ac1865d66f8c9b52935558b328
Author: Chris Wilson <chris-Y6uKTt2uX1cEflXRtASbqLVCufUGDwFn@public.gmane.org>
Date:   Mon Aug 20 11:40:46 2012 +0200

    drm/i915: Track unbound pages
    
    When dealing with a working set larger than the GATT, or even the
    mappable aperture when touching through the GTT, we end up with
    evicting
    objects only to rebind them at a new offset again later. Moving an
    object into and out of the GTT requires clflushing the pages, thus
    causing a double-clflush penalty for rebinding.
    
    To avoid having to clflush on rebinding, we can track the pages as
    they
    are evicted from the GTT and only relinquish those pages on memory
    pressure.
    
    As usual, if it were not for the handling of out-of-memory
    condition and
    having to manually shrink our own bo caches, it would be a net
    reduction
    of code. Alas.
    
    Note: The patch also contains a few changes to the last-hope
    evict_everything logic in i916_gem_execbuffer.c - we no longer try
    to
    only evict the purgeable stuff in a first try (since that's
    superflous
    and only helps in OOM corner-cases, not fragmented-gtt trashing
    situations).
    
    Also, the extraction of the get_pages retry loop from bind_to_gtt
    (and
    other callsites) to get_pages should imo have been a separate
    patch.
    
    v2: Ditch the newly added put_pages (for unbound objects only) in
    i915_gem_reset. A quick irc discussion hasn't revealed any
    important
    reason for this, so if we need this, I'd like to have a git
    blame'able
    explanation for it.
    
    v3: Undo the s/drm_malloc_ab/kmalloc/ in get_pages that Chris
    noticed.
    
    Signed-off-by: Chris Wilson <chris-Y6uKTt2uX1cEflXRtASbqLVCufUGDwFn@public.gmane.org>
    [danvet: Split out code movements and rant a bit in the commit
    message
    with a few Notes. Done v2]
    Signed-off-by: Daniel Vetter <daniel.vetter-/w4YWyX8dFk@public.gmane.org>

:040000 040000 c4f02e0d05a570d0baf9d2f19a6c276c06a55142
df93a56308637e3840353c3c9425ec96c3422dcc M	drivers
[root@wildsau linux-git]# 

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 08.12.2012, Chris Wilson wrote: 

> One thing you can try is SNA, which packs its
> batches differently with the advantage that more auxiliary state is
> included in the error-state. It also packs all the kernels into a
> single buffer which will reduce the frequency at which it is paged
> out/in. So if you can reproduce with SNA (use Option "AccelMethod"
> "SNA" in a device section of your xorg.conf snippet) I expect the
> error-state to be quite different and hopefully shed some more light on
> the issue.

I tried this with latest 3.7-rc8 git, but no matter how hard I try, I
can't get the gpu to hang (with i915.915_enable_rc6=0). Will use this 
as my default kernel the next few days and see if the hang occurs by
chance.

Heinz

Re: i915 freakout with latest 3.7 git

[Daniel Vetter]

On Sat, Dec 08, 2012 at 02:06:48PM +0100, Heinz Diehl wrote:
> On 07.12.2012, Daniel Vetter wrote: 
> 
> [....]
> 
> I did a "git bisect" betweeb 3.6 and 3.7-rc8 and ended up with
> this. Unfortunately, git can't revert this patch on top of master, sp
> I have not been able to test if a revert will cure the problem.
> 
> After reading on the net that Peter (Lekensteyn) already ended up with
> bisecting the same patch and it didn't work for him reverting it on
> top of 3-7-rc4, I'm somewhat clueless..
> 
> What else can I do to help finding the cause?

Can you please test the patch at

https://bugs.freedesktop.org/attachment.cgi?id=70111

That one should disable all effects of the unbound tracking, since a
revert of the below commit conflicts.

Thanks, Daniel
> 
> Heinz
> 
> 
> [root@wildsau linux-git]# git bisect good
> 6c085a728cf000ac1865d66f8c9b52935558b328 is the first bad commit
> commit 6c085a728cf000ac1865d66f8c9b52935558b328
> Author: Chris Wilson <chris-Y6uKTt2uX1cEflXRtASbqLVCufUGDwFn@public.gmane.org>
> Date:   Mon Aug 20 11:40:46 2012 +0200
> 
>     drm/i915: Track unbound pages
>     
>     When dealing with a working set larger than the GATT, or even the
>     mappable aperture when touching through the GTT, we end up with
>     evicting
>     objects only to rebind them at a new offset again later. Moving an
>     object into and out of the GTT requires clflushing the pages, thus
>     causing a double-clflush penalty for rebinding.
>     
>     To avoid having to clflush on rebinding, we can track the pages as
>     they
>     are evicted from the GTT and only relinquish those pages on memory
>     pressure.
>     
>     As usual, if it were not for the handling of out-of-memory
>     condition and
>     having to manually shrink our own bo caches, it would be a net
>     reduction
>     of code. Alas.
>     
>     Note: The patch also contains a few changes to the last-hope
>     evict_everything logic in i916_gem_execbuffer.c - we no longer try
>     to
>     only evict the purgeable stuff in a first try (since that's
>     superflous
>     and only helps in OOM corner-cases, not fragmented-gtt trashing
>     situations).
>     
>     Also, the extraction of the get_pages retry loop from bind_to_gtt
>     (and
>     other callsites) to get_pages should imo have been a separate
>     patch.
>     
>     v2: Ditch the newly added put_pages (for unbound objects only) in
>     i915_gem_reset. A quick irc discussion hasn't revealed any
>     important
>     reason for this, so if we need this, I'd like to have a git
>     blame'able
>     explanation for it.
>     
>     v3: Undo the s/drm_malloc_ab/kmalloc/ in get_pages that Chris
>     noticed.
>     
>     Signed-off-by: Chris Wilson <chris-Y6uKTt2uX1cEflXRtASbqLVCufUGDwFn@public.gmane.org>
>     [danvet: Split out code movements and rant a bit in the commit
>     message
>     with a few Notes. Done v2]
>     Signed-off-by: Daniel Vetter <daniel.vetter-/w4YWyX8dFk@public.gmane.org>
> 
> :040000 040000 c4f02e0d05a570d0baf9d2f19a6c276c06a55142
> df93a56308637e3840353c3c9425ec96c3422dcc M	drivers
> [root@wildsau linux-git]# 
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: i915 freakout with latest 3.7 git

[Chris Wilson]

On Sat, 8 Dec 2012 15:30:53 +0100, Heinz Diehl <htd-iEI8Y0CNJBYdnm+yROfE0A@public.gmane.org> wrote:
> On 08.12.2012, Chris Wilson wrote: 
> 
> > One thing you can try is SNA, which packs its
> > batches differently with the advantage that more auxiliary state is
> > included in the error-state. It also packs all the kernels into a
> > single buffer which will reduce the frequency at which it is paged
> > out/in. So if you can reproduce with SNA (use Option "AccelMethod"
> > "SNA" in a device section of your xorg.conf snippet) I expect the
> > error-state to be quite different and hopefully shed some more light on
> > the issue.
> 
> I tried this with latest 3.7-rc8 git, but no matter how hard I try, I
> can't get the gpu to hang (with i915.915_enable_rc6=0). Will use this 
> as my default kernel the next few days and see if the hang occurs by
> chance.

Can you confirm one thing: are you able to reproduce the hangs at all on
3.7-rc8, using your original setup?
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 11.12.2012, Chris Wilson wrote: 

> Can you confirm one thing: are you able to reproduce the hangs at all on
> 3.7-rc8, using your original setup?

I can reproduce the hang with both 3.7-rc8 and 3.7 final inkl. latest
Linus-git. All with i915.i915_enable_rc6=0.

Heinz

Re: i915 freakout with latest 3.7 git

[Heinz Diehl]

On 11.12.2012, Daniel Vetter wrote: 

> Can you please test the patch at
> 
> https://bugs.freedesktop.org/attachment.cgi?id=70111
> 
> That one should disable all effects of the unbound tracking, since a
> revert of the below commit conflicts.

I applied this patch to Linus' git from today. "Boom" after about 1
min.

The errorstate file is here:

http://www.fritha.org/i915/errorstate3.tar.bz2

Heinz