[Bug 102006] gstreamer vaapih264enc segfault

[Bug 102006] gstreamer vaapih264enc segfault

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 1028 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=102006

            Bug ID: 102006
           Summary: gstreamer vaapih264enc segfault
           Product: DRI
           Version: XOrg git
          Hardware: x86-64 (AMD64)
                OS: Linux (All)
            Status: NEW
          Severity: normal
          Priority: medium
         Component: DRM/Radeon
          Assignee: dri-devel@lists.freedesktop.org
          Reporter: rataj28@gmail.com

Created attachment 133180
  --> https://bugs.freedesktop.org/attachment.cgi?id=133180&action=edit
proposed patch by Julien Isorce

Based on following bug:

https://bugzilla.gnome.org/show_bug.cgi?id=785085

In Mesa the picture_id is a frame number which is in the range 0-31 here:

https://cgit.freedesktop.org/mesa/mesa/tree/src/gallium/state_trackers/va/picture.c#n430

In gst vaapi it is used as surface handles and that can be larger than 31 thus
causing SEGFAULT.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2714 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 102006] gstreamer vaapih264enc segfault

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 258 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=102006

--- Comment #1 from Julien Isorce <julien.isorce@gmail.com> ---
Also see https://bugzilla.gnome.org/show_bug.cgi?id=785085

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1074 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 102006] gstreamer vaapih264enc segfault

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 1534 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=102006

--- Comment #2 from Julien Isorce <julien.isorce@gmail.com> ---
Comment on attachment 133180
  --> https://bugs.freedesktop.org/attachment.cgi?id=133180
proposed patch by Julien Isorce

commit 91d93aa62162f98d6377e5c796b63faa263f2c18
Author: Julien Isorce <julien.isorce@gmail.com>
Date:   Tue Jul 25 15:31:28 2017 +0100

    st/va: change frame_idx from array to hash table

    The picture_id was assumed to be a frame number so in 0-31.
    But the vaapi client gstreamer-vaapi uses the surfaces handles
    as identifier which are unsigned int.

    This bug can happen when using a lot of vaapi surfaces within
    the same process. Indeed Mesa/st/va increments a counter for the
    surface ID: mesa/util/u_handle_table.c::handle_table_add which
    starts from 0 and incremented by 1 at each call.
    So creating more than 32 surfaces was a problem.

    The following bug contains a test that reproduces the problem
    by running a couple of vaapih264enc in the same process. The
    above also explains why there was no pb when running them in
    separated processes.

    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=102006
    Signed-off-by: Julien Isorce <jisorce@oblong.com>
    Tested-by: Tomas Rataj <rataj28@gmail.com>
    Acked-by: Christian König <christian.koenig@amd.com>
    Reviewed-and-tested-by: Boyuan Zhang <Boyuan.Zhang@amd.com>

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 2958 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[Bug 102006] gstreamer vaapih264enc segfault

[bugzilla-daemon]

[-- Attachment #1.1: Type: text/plain, Size: 434 bytes --]

https://bugs.freedesktop.org/show_bug.cgi?id=102006

Julien Isorce <julien.isorce@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

-- 
You are receiving this mail because:
You are the assignee for the bug.

[-- Attachment #1.2: Type: text/html, Size: 1248 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel