[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 1543 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

            Bug ID: 96588
           Summary: [regression] [amdgpu] Errors scheduling IBs
           Product: DRI
           Version: DRI git
          Hardware: Other
                OS: All
            Status: NEW
          Severity: normal
          Priority: medium
         Component: DRM/AMDgpu
          Assignee: dri-devel@lists.freedesktop.org
          Reporter: mike@fireburn.co.uk

Hi

I've been seeing these errors in my kernel logs:

amdgpu 0000:01:00.0: couldn't schedule ib
[drm:amdgpu_job_run] *ERROR* Error scheduling IBs (-22)
[drm:amd_sched_main] *ERROR* Failed to run job!

I've bisected it down to:

a7c77c7fe5f659428e73d77aa4a8ac80b638daf3 is the first bad commit
commit a7c77c7fe5f659428e73d77aa4a8ac80b638daf3
Author: Christian König <christian.koenig@amd.com>
Date:   Wed Jun 15 13:44:05 2016 +0200

    drm/amdgpu: pipeline evictions as well

    This boosts Xonotic from 38fps to 47fps when artificially limiting VRAM to
    256MB for testing. It should improve all CPU bound rendering situations
    where we have a lot of swapping to/from VRAM.

    Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
    Signed-off-by: Christian König <christian.koenig@amd.com>
    Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

:040000 040000 2fdfd546d7175759ed5f09bdec209f71d084ab1e
6b0cdbc8f42f4e3873e3bbdcc440336856073883 M      drivers

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 3076 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 564 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Mike Lothian <mike@fireburn.co.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mike@fireburn.co.uk

--- Comment #1 from Mike Lothian <mike@fireburn.co.uk> ---
Created attachment 124602
  --> https://bugs.freedesktop.org/attachment.cgi?id=124602&action=edit
journalctl output

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1997 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 240 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #2 from Mike Lothian <mike@fireburn.co.uk> ---
Reverting that commit makes the errors go away

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1009 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 310 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #3 from Christian König <deathsimple@vodafone.de> ---
Could be some kind of race condition, please provide the output of "journalctl
--dmesg -o short-monotonic".

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1097 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 592 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Mike Lothian <mike@fireburn.co.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #124602|0                           |1
        is obsolete|                            |

--- Comment #4 from Mike Lothian <mike@fireburn.co.uk> ---
Created attachment 124607
  --> https://bugs.freedesktop.org/attachment.cgi?id=124607&action=edit
demsg output

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1988 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 538 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #5 from Christian König <deathsimple@vodafone.de> ---
Ok, clearly not a race condition but something is wrong here the cause the
driver tries to initialize the ring buffers multiple times.

Maybe something is causing a GPU reset, but as far as I remember those should
still be turned of by default.

Please provide a journalctl output from a boot with the patch in question
reverted.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1315 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 411 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #6 from Mike Lothian <mike@fireburn.co.uk> ---
I should probably have specified this is a prime laptop with dynpm 

The card initialises each time it's needed, it always does this during boot and
again when X starts, and each time I load a game with DRI_PRIME=1

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1180 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 453 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #7 from Christian König <deathsimple@vodafone.de> ---
Ah, enlightenment! Thanks that was the info I was missing.

We probably just forget to wait for all evictions before we turn of the GPU
resulting in the still running jobs to produce this error message.

Give me a second to hack together a patch.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1230 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 638 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Christian König <deathsimple@vodafone.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |deathsimple@vodafone.de

--- Comment #8 from Christian König <deathsimple@vodafone.de> ---
Created attachment 124616
  --> https://bugs.freedesktop.org/attachment.cgi?id=124616&action=edit
Possible fix

Please test the attached patch it should fix the issue.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2162 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 240 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #9 from Mike Lothian <mike@fireburn.co.uk> ---
I'll test this tonight when I get home, thanks

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1009 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 611 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Mike Lothian <mike@fireburn.co.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #124607|0                           |1
        is obsolete|                            |

--- Comment #10 from Mike Lothian <mike@fireburn.co.uk> ---
Created attachment 124623
  --> https://bugs.freedesktop.org/attachment.cgi?id=124623&action=edit
dmesg

Still seems to happen

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1994 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 645 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Christian König <deathsimple@vodafone.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |alexdeucher@gmail.com

--- Comment #11 from Christian König <deathsimple@vodafone.de> ---
I'm running out of ideas. Does that have any other negative results except for
the error message?

Alex any idea what else could cause an eviction during switching of the dGPU?

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1969 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 683 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #12 from Alex Deucher <alexdeucher@gmail.com> ---
(In reply to Christian König from comment #11)
> I'm running out of ideas. Does that have any other negative results except
> for the error message?
> 
> Alex any idea what else could cause an eviction during switching of the dGPU?

powering up/down the dGPU should hit the same code as resume/suspend.  Are you
seeing similar issues with suspend and resume?  Maybe the scheduler isn't
getting stopped properly on suspend?  We recently fixed something like this for
gpu reset.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1535 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 642 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Mike Lothian <mike@fireburn.co.uk> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #124623|0                           |1
        is obsolete|                            |

--- Comment #13 from Mike Lothian <mike@fireburn.co.uk> ---
Created attachment 124672
  --> https://bugs.freedesktop.org/attachment.cgi?id=124672&action=edit
dmesg

It seems to spam the logs more when I fire up a game

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2025 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 568 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #14 from Christian König <deathsimple@vodafone.de> ---
Created attachment 124708
  --> https://bugs.freedesktop.org/attachment.cgi?id=124708&action=edit
Additional fix.

Alex suspend/resume idea was the right approach.

I was able to reproduce the issue and so find a pretty fundamental bug in one
of my recent patches.

Please see the additional fix, together with the first patch it should resolve
the issue.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1553 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 288 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

--- Comment #15 from Mike Lothian <mike@fireburn.co.uk> ---
The patch already seems to have landed in drm-next-4.8-wip and it does indeed
seem to fix it

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1058 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 96588] [regression] [amdgpu] Errors scheduling IBs

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 650 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=96588

Vedran Miletić <vedran@miletic.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #16 from Vedran Miletić <vedran@miletic.net> ---
Marking as resolved per comment #15 and
https://cgit.freedesktop.org/~agd5f/linux/commit/?h=drm-next-4.8-wip&id=ce774d0254ed05ff6d3e3ce2c598aa4f79d45c3c

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2308 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel