QtMultimedia on i.MX6

QtMultimedia on i.MX6

[Thomas Senyk]

Hi,

I've started to investigate to see the QtMultimedia-gstreamer integration on 
the i.MX6 could be optimized.

I'm writing this mail to sync with others who might already done some work or 
investigations on this matter, to get hints where to start and what's worth to 
investigate.



State right now:

The HW-decoders are used (according to gstreamer log).
That's a very good start/out of the box experience! Good job..freescale I 
guess?

There is still a lot of cpu load:
   - 720p: smooth playback but~18% complete cpu load => ~70% of one core
   - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one core 
+ some additional cpu load

The obvious reason for this cpu load is some sort of memcopy and/or 
conversion.
  ... gstreamer doesn't render into gpu-memory directly.
  ... or (if you want it the other way around) the gstreamer (/cpu) memory 
isn't mapped into gpu-memory (which is possibly on the imx6)




Started to do some digging:

One of the first things I tried was to compile gst-plugin-gl (which is part of 
the meta-fsl-arm layer via a .bbappend).
The current recipe is driver-version-1.1.0 based and is not building out of 
the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz (based 
on 4.0.0) building.**

So, after I got it deployed I hoped for magic to happen and everything is 
perfect now ... no big surprise: it's not :) nothing changed, possibly the 
gst-pipline build by qtmultimedia is the ame.



The reason for this mail / some questions:

So I though, I write this mail before I waste a lot of time.
What does people think is the best way forward?
Try to convince QtMultimedia to use glsink?
Try to use a fakesink/memory-based-sink and try to map the cpu memory into gpu 
memory via vivante APIs?

Does anyone have c/c++ code I could try?
Ideally for direct framebuffer/EGL use (I'm not using X11)
I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
 .. but it's crashing rather early and I'm not sure the if the time is well 
spend fix this code? Anyone used it before?
Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?


Greet
Thomas

** some more info in gst-plugin-gl:
It's 'detected' on the board:
root@nitrogen-1:~# gst-inspect |grep gl
opengl:  glupload: OpenGL upload
opengl:  gldownload: OpenGL video maker
opengl:  glimagesink: OpenGL video sink
opengl:  glfiltercube: OpenGL cube filter
opengl:  gleffects: Gstreamer OpenGL Effects

... if someone is interested on how I got it building, let me know. If I got 
something useful out of it (read: "it's tested") I'll start a discussion on 
how to upstream it properly into meta-fsl-arm (by the way: I find the current 
setup of how to bbappend and it's patch looks like very weird)

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote:
> Hi,
> 
> I've started to investigate to see the QtMultimedia-gstreamer integration on
> the i.MX6 could be optimized.
> 
> I'm writing this mail to sync with others who might already done some work
> or investigations on this matter, to get hints where to start and what's
> worth to investigate.
> 
> 
> 
> State right now:
> 
> The HW-decoders are used (according to gstreamer log).
> That's a very good start/out of the box experience! Good job..freescale I
> guess?
> 
> There is still a lot of cpu load:
>    - 720p: smooth playback but~18% complete cpu load => ~70% of one core
>    - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one
> core + some additional cpu load
> 
> The obvious reason for this cpu load is some sort of memcopy and/or
> conversion.
>   ... gstreamer doesn't render into gpu-memory directly.
>   ... or (if you want it the other way around) the gstreamer (/cpu) memory
> isn't mapped into gpu-memory (which is possibly on the imx6)
> 
> 
> 
> 
> Started to do some digging:
> 
> One of the first things I tried was to compile gst-plugin-gl (which is part
> of the meta-fsl-arm layer via a .bbappend).
> The current recipe is driver-version-1.1.0 based and is not building out of
> the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz
> (based on 4.0.0) building.**
> 
> So, after I got it deployed I hoped for magic to happen and everything is
> perfect now ... no big surprise: it's not :) nothing changed, possibly the
> gst-pipline build by qtmultimedia is the ame.
> 
> 
> 
> The reason for this mail / some questions:
> 
> So I though, I write this mail before I waste a lot of time.
> What does people think is the best way forward?
> Try to convince QtMultimedia to use glsink?
> Try to use a fakesink/memory-based-sink and try to map the cpu memory into
> gpu memory via vivante APIs?
> 
> Does anyone have c/c++ code I could try?
> Ideally for direct framebuffer/EGL use (I'm not using X11)
> I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
>  .. but it's crashing rather early and I'm not sure the if the time is well
> spend fix this code? Anyone used it before?
> Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?
> 
> 
> Greet
> Thomas

Ok ... what is it with me and sending mails 1minute to early all the time ...
After I pressed "send", I had a very simple and obvious idea .. why not just 
simply try glimagesink instead of mfw_v4lsink:

gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux ! 
vpudec !  glimagesink

 ... long shot, but:
It's actually working!
It just opens /dev/fb0 and renders the video into it.

1080p with 0%(!!) cpu load!

I think that pretty much answers all questions I had ;)
The way forward: patch QtMultimedia to use glimagesink
 ... I hope it's going to be trivial to just give him a active FBO or texture.


Sorry to everybody who read my way to long mail ;)

> 
> ** some more info in gst-plugin-gl:
> It's 'detected' on the board:
> root@nitrogen-1:~# gst-inspect |grep gl
> opengl:  glupload: OpenGL upload
> opengl:  gldownload: OpenGL video maker
> opengl:  glimagesink: OpenGL video sink
> opengl:  glfiltercube: OpenGL cube filter
> opengl:  gleffects: Gstreamer OpenGL Effects
> 
> ... if someone is interested on how I got it building, let me know. If I got
> something useful out of it (read: "it's tested") I'll start a discussion on
> how to upstream it properly into meta-fsl-arm (by the way: I find the
> current setup of how to bbappend and it's patch looks like very weird)

Re: QtMultimedia on i.MX6

[Otavio Salvador]

On Thu, Jul 4, 2013 at 12:14 PM, Thomas Senyk
<thomas.senyk@pelagicore.com> wrote:
> On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote:
>> Hi,
>>
>> I've started to investigate to see the QtMultimedia-gstreamer integration on
>> the i.MX6 could be optimized.
>>
>> I'm writing this mail to sync with others who might already done some work
>> or investigations on this matter, to get hints where to start and what's
>> worth to investigate.
>>
>>
>>
>> State right now:
>>
>> The HW-decoders are used (according to gstreamer log).
>> That's a very good start/out of the box experience! Good job..freescale I
>> guess?
>>
>> There is still a lot of cpu load:
>>    - 720p: smooth playback but~18% complete cpu load => ~70% of one core
>>    - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one
>> core + some additional cpu load
>>
>> The obvious reason for this cpu load is some sort of memcopy and/or
>> conversion.
>>   ... gstreamer doesn't render into gpu-memory directly.
>>   ... or (if you want it the other way around) the gstreamer (/cpu) memory
>> isn't mapped into gpu-memory (which is possibly on the imx6)
>>
>>
>>
>>
>> Started to do some digging:
>>
>> One of the first things I tried was to compile gst-plugin-gl (which is part
>> of the meta-fsl-arm layer via a .bbappend).
>> The current recipe is driver-version-1.1.0 based and is not building out of
>> the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz
>> (based on 4.0.0) building.**
>>
>> So, after I got it deployed I hoped for magic to happen and everything is
>> perfect now ... no big surprise: it's not :) nothing changed, possibly the
>> gst-pipline build by qtmultimedia is the ame.
>>
>>
>>
>> The reason for this mail / some questions:
>>
>> So I though, I write this mail before I waste a lot of time.
>> What does people think is the best way forward?
>> Try to convince QtMultimedia to use glsink?
>> Try to use a fakesink/memory-based-sink and try to map the cpu memory into
>> gpu memory via vivante APIs?
>>
>> Does anyone have c/c++ code I could try?
>> Ideally for direct framebuffer/EGL use (I'm not using X11)
>> I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
>>  .. but it's crashing rather early and I'm not sure the if the time is well
>> spend fix this code? Anyone used it before?
>> Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?
>>
>>
>> Greet
>> Thomas
>
> Ok ... what is it with me and sending mails 1minute to early all the time ...
> After I pressed "send", I had a very simple and obvious idea .. why not just
> simply try glimagesink instead of mfw_v4lsink:
>
> gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux !
> vpudec !  glimagesink
>
>  ... long shot, but:
> It's actually working!
> It just opens /dev/fb0 and renders the video into it.
>
> 1080p with 0%(!!) cpu load!
>
> I think that pretty much answers all questions I had ;)
> The way forward: patch QtMultimedia to use glimagesink
>  ... I hope it's going to be trivial to just give him a active FBO or texture.
>
>
> Sorry to everybody who read my way to long mail ;)

It was very good read. By the way, Rogerio (add in Cc) started to try
to get the gst-plugin-gl code more manageble in bitbucket
(https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/all)
but it was not finished. Maybe you could help on this and we could try
to merge the Vivante backend upstream? This would allow us to drop the
bbappend completely.

>> ** some more info in gst-plugin-gl:
>> It's 'detected' on the board:
>> root@nitrogen-1:~# gst-inspect |grep gl
>> opengl:  glupload: OpenGL upload
>> opengl:  gldownload: OpenGL video maker
>> opengl:  glimagesink: OpenGL video sink
>> opengl:  glfiltercube: OpenGL cube filter
>> opengl:  gleffects: Gstreamer OpenGL Effects
>>
>> ... if someone is interested on how I got it building, let me know. If I got
>> something useful out of it (read: "it's tested") I'll start a discussion on
>> how to upstream it properly into meta-fsl-arm (by the way: I find the
>> current setup of how to bbappend and it's patch looks like very weird)
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale



--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: QtMultimedia on i.MX6

[Sébastien Taylor]

How would one gain access (read-only) to this repository?

On 2013-07-04, at 3:42 PM, Otavio Salvador <otavio@ossystems.com.br> wrote:

> On Thu, Jul 4, 2013 at 12:14 PM, Thomas Senyk
> <thomas.senyk@pelagicore.com> wrote:
>> On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote:
>>> Hi,
>>> 
>>> I've started to investigate to see the QtMultimedia-gstreamer integration on
>>> the i.MX6 could be optimized.
>>> 
>>> I'm writing this mail to sync with others who might already done some work
>>> or investigations on this matter, to get hints where to start and what's
>>> worth to investigate.
>>> 
>>> 
>>> 
>>> State right now:
>>> 
>>> The HW-decoders are used (according to gstreamer log).
>>> That's a very good start/out of the box experience! Good job..freescale I
>>> guess?
>>> 
>>> There is still a lot of cpu load:
>>>   - 720p: smooth playback but~18% complete cpu load => ~70% of one core
>>>   - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one
>>> core + some additional cpu load
>>> 
>>> The obvious reason for this cpu load is some sort of memcopy and/or
>>> conversion.
>>>  ... gstreamer doesn't render into gpu-memory directly.
>>>  ... or (if you want it the other way around) the gstreamer (/cpu) memory
>>> isn't mapped into gpu-memory (which is possibly on the imx6)
>>> 
>>> 
>>> 
>>> 
>>> Started to do some digging:
>>> 
>>> One of the first things I tried was to compile gst-plugin-gl (which is part
>>> of the meta-fsl-arm layer via a .bbappend).
>>> The current recipe is driver-version-1.1.0 based and is not building out of
>>> the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz
>>> (based on 4.0.0) building.**
>>> 
>>> So, after I got it deployed I hoped for magic to happen and everything is
>>> perfect now ... no big surprise: it's not :) nothing changed, possibly the
>>> gst-pipline build by qtmultimedia is the ame.
>>> 
>>> 
>>> 
>>> The reason for this mail / some questions:
>>> 
>>> So I though, I write this mail before I waste a lot of time.
>>> What does people think is the best way forward?
>>> Try to convince QtMultimedia to use glsink?
>>> Try to use a fakesink/memory-based-sink and try to map the cpu memory into
>>> gpu memory via vivante APIs?
>>> 
>>> Does anyone have c/c++ code I could try?
>>> Ideally for direct framebuffer/EGL use (I'm not using X11)
>>> I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
>>> .. but it's crashing rather early and I'm not sure the if the time is well
>>> spend fix this code? Anyone used it before?
>>> Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?
>>> 
>>> 
>>> Greet
>>> Thomas
>> 
>> Ok ... what is it with me and sending mails 1minute to early all the time ...
>> After I pressed "send", I had a very simple and obvious idea .. why not just
>> simply try glimagesink instead of mfw_v4lsink:
>> 
>> gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux !
>> vpudec !  glimagesink
>> 
>> ... long shot, but:
>> It's actually working!
>> It just opens /dev/fb0 and renders the video into it.
>> 
>> 1080p with 0%(!!) cpu load!
>> 
>> I think that pretty much answers all questions I had ;)
>> The way forward: patch QtMultimedia to use glimagesink
>> ... I hope it's going to be trivial to just give him a active FBO or texture.
>> 
>> 
>> Sorry to everybody who read my way to long mail ;)
> 
> It was very good read. By the way, Rogerio (add in Cc) started to try
> to get the gst-plugin-gl code more manageble in bitbucket
> (https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/all)
> but it was not finished. Maybe you could help on this and we could try
> to merge the Vivante backend upstream? This would allow us to drop the
> bbappend completely.
> 
>>> ** some more info in gst-plugin-gl:
>>> It's 'detected' on the board:
>>> root@nitrogen-1:~# gst-inspect |grep gl
>>> opengl:  glupload: OpenGL upload
>>> opengl:  gldownload: OpenGL video maker
>>> opengl:  glimagesink: OpenGL video sink
>>> opengl:  glfiltercube: OpenGL cube filter
>>> opengl:  gleffects: Gstreamer OpenGL Effects
>>> 
>>> ... if someone is interested on how I got it building, let me know. If I got
>>> something useful out of it (read: "it's tested") I'll start a discussion on
>>> how to upstream it properly into meta-fsl-arm (by the way: I find the
>>> current setup of how to bbappend and it's patch looks like very weird)
>> _______________________________________________
>> meta-freescale mailing list
>> meta-freescale@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-freescale
> 
> 
> 
> --
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://projetos.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Re: QtMultimedia on i.MX6

[Otavio Salvador]

On Thu, Jul 4, 2013 at 6:53 PM, Sébastien Taylor <me@staylor.ca> wrote:
> How would one gain access (read-only) to this repository?

I think it is not public by mistake. Rogerio, can you make this (and
gstreamer/common) public? I think it has nothing private there.

--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: QtMultimedia on i.MX6

[Rogerio Nunes]

Repository is public now.
I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0 this
afternoon.

BTW, I still need to update the gst-plugins-gl recipe in the
meta-fsl-arm layer to point to this repo. It's cleaner than the patch
we currently have in the recipe.

Regards.

On Thu, Jul 4, 2013 at 6:03 PM, Otavio Salvador <otavio@ossystems.com.br> wrote:
> On Thu, Jul 4, 2013 at 6:53 PM, Sébastien Taylor <me@staylor.ca> wrote:
>> How would one gain access (read-only) to this repository?
>
> I think it is not public by mistake. Rogerio, can you make this (and
> gstreamer/common) public? I think it has nothing private there.
>
> --
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://projetos.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> Repository is public now.
> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0 this
> afternoon.
> 
> BTW, I still need to update the gst-plugins-gl recipe in the
> meta-fsl-arm layer to point to this repo. It's cleaner than the patch
> we currently have in the recipe.
> 

Sounds wonderful!
If something is ready to test, let me know.

fyi:
I had one 'source' problem with 4.0.0: deprecated glib functions.
I hacked common/m4/gst-glib2.m4:
-    GLIB_EXTRA_CFLAGS="$GLIB_EXTRA_CFLAGS -Wno-deprecated-declarations"
+    GLIB_EXTRA_CFLAGS="$GLIB_EXTRA_CFLAGS -DG_DISABLE_DEPRECATED"

to get it running ... guess that's the wrong way of doing it, but it helped :) 

> Regards.
> 
> On Thu, Jul 4, 2013 at 6:03 PM, Otavio Salvador <otavio@ossystems.com.br> 
wrote:
> > On Thu, Jul 4, 2013 at 6:53 PM, Sébastien Taylor <me@staylor.ca> wrote:
> >> How would one gain access (read-only) to this repository?
> > 
> > I think it is not public by mistake. Rogerio, can you make this (and
> > gstreamer/common) public? I think it has nothing private there.
> > 
> > --
> > Otavio Salvador                             O.S. Systems
> > http://www.ossystems.com.br        http://projetos.ossystems.com.br
> > Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
> > _______________________________________________
> > meta-freescale mailing list
> > meta-freescale@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/meta-freescale
> 
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale

Re: QtMultimedia on i.MX6

[Otavio Salvador]

On Tue, Jul 9, 2013 at 5:14 AM, Thomas Senyk
<thomas.senyk@pelagicore.com> wrote:
> On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
>> Repository is public now.
>> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0 this
>> afternoon.
>>
>> BTW, I still need to update the gst-plugins-gl recipe in the
>> meta-fsl-arm layer to point to this repo. It's cleaner than the patch
>> we currently have in the recipe.
>>
>
> Sounds wonderful!
> If something is ready to test, let me know.
>
> fyi:
> I had one 'source' problem with 4.0.0: deprecated glib functions.
> I hacked common/m4/gst-glib2.m4:
> -    GLIB_EXTRA_CFLAGS="$GLIB_EXTRA_CFLAGS -Wno-deprecated-declarations"
> +    GLIB_EXTRA_CFLAGS="$GLIB_EXTRA_CFLAGS -DG_DISABLE_DEPRECATED"
>
> to get it running ... guess that's the wrong way of doing it, but it helped :)

Not really; this seems to be the right fix for Yocto (but not for upstream) ;)

--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: QtMultimedia on i.MX6

[Rogerio Nunes]

[-- Attachment #1: Type: text/plain, Size: 2664 bytes --]

On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <thomas.senyk@pelagicore.com>
wrote:
> On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
>> Repository is public now.
>> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0 this
>> afternoon.
>>
>> BTW, I still need to update the gst-plugins-gl recipe in the
>> meta-fsl-arm layer to point to this repo. It's cleaner than the patch
>> we currently have in the recipe.
>>
>
> Sounds wonderful!
> If something is ready to test, let me know.
>

Very small change from previous BSP to 4.0.0 in the gst-plugins-gl package:
https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf217a90e160fd7e4c45f02a23da974130?at=fsl-0.10
I've just sent a patch to the list (awaiting approval as it was big...) to
update meta-fsl-arm.

In the bitbucket repo, branch fsl-0.10 has commits that reflect "as is"
packages from the BSP.
My idea now is to work on branch 0.10 to generate a set of patches that we
can upstream.
I would appreciate any help.

> fyi:
> I had one 'source' problem with 4.0.0: deprecated glib functions.
> I hacked common/m4/gst-glib2.m4:
> -    GLIB_EXTRA_CFLAGS="$GLIB_EXTRA_CFLAGS -Wno-deprecated-declarations"
> +    GLIB_EXTRA_CFLAGS="$GLIB_EXTRA_CFLAGS -DG_DISABLE_DEPRECATED"
>
> to get it running ... guess that's the wrong way of doing it, but it
helped :)
>

Thanks.
Same issue in master even with previous BSP after glib2 has been updated in
poky.
I've just submitted a patch to do something similar but in the recipe level
to avoid patching
the common submodule, as I want to avoid creating a repo to keep track of
changes to it to.

>> Regards.
>>
>> On Thu, Jul 4, 2013 at 6:03 PM, Otavio Salvador <otavio@ossystems.com.br>
> wrote:
>> > On Thu, Jul 4, 2013 at 6:53 PM, Sébastien Taylor <me@staylor.ca> wrote:
>> >> How would one gain access (read-only) to this repository?
>> >
>> > I think it is not public by mistake. Rogerio, can you make this (and
>> > gstreamer/common) public? I think it has nothing private there.
>> >
>> > --
>> > Otavio Salvador                             O.S. Systems
>> > http://www.ossystems.com.br        http://projetos.ossystems.com.br
>> > Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
>> > _______________________________________________
>> > meta-freescale mailing list
>> > meta-freescale@yoctoproject.org
>> > https://lists.yoctoproject.org/listinfo/meta-freescale
>>
>> _______________________________________________
>> meta-freescale mailing list
>> meta-freescale@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-freescale

[-- Attachment #2: Type: text/html, Size: 3873 bytes --]

Re: QtMultimedia on i.MX6

[Philip Craig]

[-- Attachment #1: Type: text/plain, Size: 1404 bytes --]

On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes@gmail.com> wrote:

> On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <thomas.senyk@pelagicore.com>
> wrote:
> > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> >> Repository is public now.
> >> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0 this
> >> afternoon.
> >>
> >> BTW, I still need to update the gst-plugins-gl recipe in the
> >> meta-fsl-arm layer to point to this repo. It's cleaner than the patch
> >> we currently have in the recipe.
> >>
> >
> > Sounds wonderful!
> > If something is ready to test, let me know.
> >
>
> Very small change from previous BSP to 4.0.0 in the gst-plugins-gl package:
>
> https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf217a90e160fd7e4c45f02a23da974130?at=fsl-0.10
> I've just sent a patch to the list (awaiting approval as it was big...) to
> update meta-fsl-arm.
>
> In the bitbucket repo, branch fsl-0.10 has commits that reflect "as is"
> packages from the BSP.
> My idea now is to work on branch 0.10 to generate a set of patches that we
> can upstream.
> I would appreciate any help.
>

I can't speak for upstream, but do note that gstreamer 0.10 is no longer
maintained, and gst-plugins-gl has recently been updated to 1.0, so it
might be worth checking if upstream will accept 0.10 patches before working
on them.

[-- Attachment #2: Type: text/html, Size: 2141 bytes --]

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Wednesday, 10 July, 2013 10:25:09 Philip Craig wrote:
> On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes@gmail.com> wrote:
> > On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <thomas.senyk@pelagicore.com>
> > 
> > wrote:
> > > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> > >> Repository is public now.
> > >> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0 this
> > >> afternoon.
> > >> 
> > >> BTW, I still need to update the gst-plugins-gl recipe in the
> > >> meta-fsl-arm layer to point to this repo. It's cleaner than the patch
> > >> we currently have in the recipe.
> > > 
> > > Sounds wonderful!
> > > If something is ready to test, let me know.
> > 
> > Very small change from previous BSP to 4.0.0 in the gst-plugins-gl
> > package:
> > 
> > https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf2
> > 17a90e160fd7e4c45f02a23da974130?at=fsl-0.10 I've just sent a patch to the
> > list (awaiting approval as it was big...) to update meta-fsl-arm.
> > 
> > In the bitbucket repo, branch fsl-0.10 has commits that reflect "as is"
> > packages from the BSP.
> > My idea now is to work on branch 0.10 to generate a set of patches that we
> > can upstream.
> > I would appreciate any help.
> 
> I can't speak for upstream, but do note that gstreamer 0.10 is no longer
> maintained, and gst-plugins-gl has recently been updated to 1.0, so it
> might be worth checking if upstream will accept 0.10 patches before working
> on them.

 ... again ... a rather long mail ;)


I kind a gave up on 'glupload' (and therefor on gst-plugins-gl) yesterday 
evening.
It sounded very good at first, but after a while I realized it can't work in 
it's current implementation/API ... it least not without X (which is a no go 
for my in any case)

What I wanted to achieve:
Being able to use textures 'filled' by gst-plugins-gl within Qt's OpenGL 
SceneGraph.

Why IMO it will not work:
glupload has to create it's own EGLContext, for that one needs displays and 
window.
The problem is that the current implementation does that be using the vivante-
linuxfb-API which results in a separated, independent "connection" to the 
display.
This means glupload's "external-opengl-context" is not usable as any provided 
EGLContext will be bound to another display connection (I tried and failed, if 
it's supposed to work but I'm doing something wrong, I'm happy to try again 
with help)

This means you'll never be able to use the generated textures outside of 
gstreamer. ... I think ;)
The only use-case left is to use glimagesink to render to framebuffer directly, 
maybe adding some gstreamer-gl-effects or something.

This is not what I indented to achieve (although I've already marked it as my 
back-up plan and doing it in QtMultimedia is trivial, if someone wants to try 
that solution let me know and I tell you what to change)

To achieve what I want I thing the only viable solution would be to replace 
the "external-opengl-context" with a 'makeGLContextCurrent'-callback ...so 
that the EGLContext can be created and managed by the UI-framework rather then 
the multimedia-framework... but I'm not eager to fiddle around with gst-API ;)



If I'm wrong with any part and it's actually possible to use textures 
generated by gst-plugins-gst outside of gstreamer (without X/other 'windowing 
manager'), I happy to try again if someone has a hint/idea how the stack 
should look like.



In the mean while I have an alternative plan:
The important part the thing is replacing the cpu based texture upload (copy) 
with a vivante-based memory-mapping ('no-copy'/'zero-copy')
Today I'm going try to implement this (using the gst-plugins-gl code as 
reference) within QtMultimedia.


Greets
Thomas

Re: QtMultimedia on i.MX6

[Philip Craig]

[-- Attachment #1: Type: text/plain, Size: 4577 bytes --]

On Wed, Jul 10, 2013 at 6:26 PM, Thomas Senyk
<thomas.senyk@pelagicore.com>wrote:

> On Wednesday, 10 July, 2013 10:25:09 Philip Craig wrote:
> > On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes@gmail.com>
> wrote:
> > > On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <
> thomas.senyk@pelagicore.com>
> > >
> > > wrote:
> > > > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> > > >> Repository is public now.
> > > >> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0
> this
> > > >> afternoon.
> > > >>
> > > >> BTW, I still need to update the gst-plugins-gl recipe in the
> > > >> meta-fsl-arm layer to point to this repo. It's cleaner than the
> patch
> > > >> we currently have in the recipe.
> > > >
> > > > Sounds wonderful!
> > > > If something is ready to test, let me know.
> > >
> > > Very small change from previous BSP to 4.0.0 in the gst-plugins-gl
> > > package:
> > >
> > >
> https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf2
> > > 17a90e160fd7e4c45f02a23da974130?at=fsl-0.10 I've just sent a patch to
> the
> > > list (awaiting approval as it was big...) to update meta-fsl-arm.
> > >
> > > In the bitbucket repo, branch fsl-0.10 has commits that reflect "as is"
> > > packages from the BSP.
> > > My idea now is to work on branch 0.10 to generate a set of patches
> that we
> > > can upstream.
> > > I would appreciate any help.
> >
> > I can't speak for upstream, but do note that gstreamer 0.10 is no longer
> > maintained, and gst-plugins-gl has recently been updated to 1.0, so it
> > might be worth checking if upstream will accept 0.10 patches before
> working
> > on them.
>
>  ... again ... a rather long mail ;)
>
>
> I kind a gave up on 'glupload' (and therefor on gst-plugins-gl) yesterday
> evening.
> It sounded very good at first, but after a while I realized it can't work
> in
> it's current implementation/API ... it least not without X (which is a no
> go
> for my in any case)
>
> What I wanted to achieve:
> Being able to use textures 'filled' by gst-plugins-gl within Qt's OpenGL
> SceneGraph.
>
> Why IMO it will not work:
> glupload has to create it's own EGLContext, for that one needs displays and
> window.
> The problem is that the current implementation does that be using the
> vivante-
> linuxfb-API which results in a separated, independent "connection" to the
> display.
> This means glupload's "external-opengl-context" is not usable as any
> provided
> EGLContext will be bound to another display connection (I tried and
> failed, if
> it's supposed to work but I'm doing something wrong, I'm happy to try again
> with help)
>
> This means you'll never be able to use the generated textures outside of
> gstreamer. ... I think ;)
> The only use-case left is to use glimagesink to render to framebuffer
> directly,
> maybe adding some gstreamer-gl-effects or something.
>
> This is not what I indented to achieve (although I've already marked it as
> my
> back-up plan and doing it in QtMultimedia is trivial, if someone wants to
> try
> that solution let me know and I tell you what to change)
>
> To achieve what I want I thing the only viable solution would be to replace
> the "external-opengl-context" with a 'makeGLContextCurrent'-callback ...so
> that the EGLContext can be created and managed by the UI-framework rather
> then
> the multimedia-framework... but I'm not eager to fiddle around with
> gst-API ;)
>
>
>
> If I'm wrong with any part and it's actually possible to use textures
> generated by gst-plugins-gst outside of gstreamer (without X/other
> 'windowing
> manager'), I happy to try again if someone has a hint/idea how the stack
> should look like.
>
>
>
> In the mean while I have an alternative plan:
> The important part the thing is replacing the cpu based texture upload
> (copy)
> with a vivante-based memory-mapping ('no-copy'/'zero-copy')
> Today I'm going try to implement this (using the gst-plugins-gl code as
> reference) within QtMultimedia.
>

I'm very new to opengl, but your analysis sounds correct to me.

You might be interested in looking at the qt-gstreamer package. It has a
qtglvideosink plugin which uses the context you give it, rather than
creating its own display etc. I did try using it, but I was trying to use a
QGLWidget to get the context, and it seems to be incompatible with eglfs.
It displays fine on my laptop with xcb. Maybe you know an alternative way
of getting a context when using eglfs.

[-- Attachment #2: Type: text/html, Size: 5746 bytes --]

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Wednesday, 10 July, 2013 19:13:57 Philip Craig wrote:
> On Wed, Jul 10, 2013 at 6:26 PM, Thomas Senyk
> 
> <thomas.senyk@pelagicore.com>wrote:
> > On Wednesday, 10 July, 2013 10:25:09 Philip Craig wrote:
> > > On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes@gmail.com>
> > 
> > wrote:
> > > > On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <
> > 
> > thomas.senyk@pelagicore.com>
> > 
> > > > wrote:
> > > > > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> > > > >> Repository is public now.
> > > > >> I only have the code from BSP 1.1.0 there yet, but I'll add 4.0.0
> > 
> > this
> > 
> > > > >> afternoon.
> > > > >> 
> > > > >> BTW, I still need to update the gst-plugins-gl recipe in the
> > > > >> meta-fsl-arm layer to point to this repo. It's cleaner than the
> > 
> > patch
> > 
> > > > >> we currently have in the recipe.
> > > > > 
> > > > > Sounds wonderful!
> > > > > If something is ready to test, let me know.
> > > > 
> > > > Very small change from previous BSP to 4.0.0 in the gst-plugins-gl
> > 
> > > > package:
> > https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf2
> > 
> > > > 17a90e160fd7e4c45f02a23da974130?at=fsl-0.10 I've just sent a patch to
> > 
> > the
> > 
> > > > list (awaiting approval as it was big...) to update meta-fsl-arm.
> > > > 
> > > > In the bitbucket repo, branch fsl-0.10 has commits that reflect "as
> > > > is"
> > > > packages from the BSP.
> > > > My idea now is to work on branch 0.10 to generate a set of patches
> > 
> > that we
> > 
> > > > can upstream.
> > > > I would appreciate any help.
> > > 
> > > I can't speak for upstream, but do note that gstreamer 0.10 is no longer
> > > maintained, and gst-plugins-gl has recently been updated to 1.0, so it
> > > might be worth checking if upstream will accept 0.10 patches before
> > 
> > working
> > 
> > > on them.
> >  
> >  ... again ... a rather long mail ;)
> > 
> > I kind a gave up on 'glupload' (and therefor on gst-plugins-gl) yesterday
> > evening.
> > It sounded very good at first, but after a while I realized it can't work
> > in
> > it's current implementation/API ... it least not without X (which is a no
> > go
> > for my in any case)
> > 
> > What I wanted to achieve:
> > Being able to use textures 'filled' by gst-plugins-gl within Qt's OpenGL
> > SceneGraph.
> > 
> > Why IMO it will not work:
> > glupload has to create it's own EGLContext, for that one needs displays
> > and
> > window.
> > The problem is that the current implementation does that be using the
> > vivante-
> > linuxfb-API which results in a separated, independent "connection" to the
> > display.
> > This means glupload's "external-opengl-context" is not usable as any
> > provided
> > EGLContext will be bound to another display connection (I tried and
> > failed, if
> > it's supposed to work but I'm doing something wrong, I'm happy to try
> > again
> > with help)
> > 
> > This means you'll never be able to use the generated textures outside of
> > gstreamer. ... I think ;)
> > The only use-case left is to use glimagesink to render to framebuffer
> > directly,
> > maybe adding some gstreamer-gl-effects or something.
> > 
> > This is not what I indented to achieve (although I've already marked it as
> > my
> > back-up plan and doing it in QtMultimedia is trivial, if someone wants to
> > try
> > that solution let me know and I tell you what to change)
> > 
> > To achieve what I want I thing the only viable solution would be to
> > replace
> > the "external-opengl-context" with a 'makeGLContextCurrent'-callback ...so
> > that the EGLContext can be created and managed by the UI-framework rather
> > then
> > the multimedia-framework... but I'm not eager to fiddle around with
> > gst-API ;)
> > 
> > 
> > 
> > If I'm wrong with any part and it's actually possible to use textures
> > generated by gst-plugins-gst outside of gstreamer (without X/other
> > 'windowing
> > manager'), I happy to try again if someone has a hint/idea how the stack
> > should look like.
> > 
> > 
> > 
> > In the mean while I have an alternative plan:
> > The important part the thing is replacing the cpu based texture upload
> > (copy)
> > with a vivante-based memory-mapping ('no-copy'/'zero-copy')
> > Today I'm going try to implement this (using the gst-plugins-gl code as
> > reference) within QtMultimedia.
> 
> I'm very new to opengl, but your analysis sounds correct to me.
> 
> You might be interested in looking at the qt-gstreamer package. It has a
> qtglvideosink plugin which uses the context you give it, rather than
> creating its own display etc. I did try using it, but I was trying to use a
> QGLWidget to get the context, and it seems to be incompatible with eglfs.
> It displays fine on my laptop with xcb. Maybe you know an alternative way
> of getting a context when using eglfs.

But this is not imx6 specific, is it?
If it's not: we can't get to a optimized stack without using the vivante-
opengl ui 
If it is: I'll take a look (where to find?)

Re: QtMultimedia on i.MX6

[Philip Craig]

[-- Attachment #1: Type: text/plain, Size: 5886 bytes --]

On Wed, Jul 10, 2013 at 7:52 PM, Thomas Senyk
<thomas.senyk@pelagicore.com>wrote:

> On Wednesday, 10 July, 2013 19:13:57 Philip Craig wrote:
> > On Wed, Jul 10, 2013 at 6:26 PM, Thomas Senyk
> >
> > <thomas.senyk@pelagicore.com>wrote:
> > > On Wednesday, 10 July, 2013 10:25:09 Philip Craig wrote:
> > > > On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes@gmail.com>
> > >
> > > wrote:
> > > > > On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <
> > >
> > > thomas.senyk@pelagicore.com>
> > >
> > > > > wrote:
> > > > > > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> > > > > >> Repository is public now.
> > > > > >> I only have the code from BSP 1.1.0 there yet, but I'll add
> 4.0.0
> > >
> > > this
> > >
> > > > > >> afternoon.
> > > > > >>
> > > > > >> BTW, I still need to update the gst-plugins-gl recipe in the
> > > > > >> meta-fsl-arm layer to point to this repo. It's cleaner than the
> > >
> > > patch
> > >
> > > > > >> we currently have in the recipe.
> > > > > >
> > > > > > Sounds wonderful!
> > > > > > If something is ready to test, let me know.
> > > > >
> > > > > Very small change from previous BSP to 4.0.0 in the gst-plugins-gl
> > >
> > > > > package:
> > >
> https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf2
> > >
> > > > > 17a90e160fd7e4c45f02a23da974130?at=fsl-0.10 I've just sent a patch
> to
> > >
> > > the
> > >
> > > > > list (awaiting approval as it was big...) to update meta-fsl-arm.
> > > > >
> > > > > In the bitbucket repo, branch fsl-0.10 has commits that reflect "as
> > > > > is"
> > > > > packages from the BSP.
> > > > > My idea now is to work on branch 0.10 to generate a set of patches
> > >
> > > that we
> > >
> > > > > can upstream.
> > > > > I would appreciate any help.
> > > >
> > > > I can't speak for upstream, but do note that gstreamer 0.10 is no
> longer
> > > > maintained, and gst-plugins-gl has recently been updated to 1.0, so
> it
> > > > might be worth checking if upstream will accept 0.10 patches before
> > >
> > > working
> > >
> > > > on them.
> > >
> > >  ... again ... a rather long mail ;)
> > >
> > > I kind a gave up on 'glupload' (and therefor on gst-plugins-gl)
> yesterday
> > > evening.
> > > It sounded very good at first, but after a while I realized it can't
> work
> > > in
> > > it's current implementation/API ... it least not without X (which is a
> no
> > > go
> > > for my in any case)
> > >
> > > What I wanted to achieve:
> > > Being able to use textures 'filled' by gst-plugins-gl within Qt's
> OpenGL
> > > SceneGraph.
> > >
> > > Why IMO it will not work:
> > > glupload has to create it's own EGLContext, for that one needs displays
> > > and
> > > window.
> > > The problem is that the current implementation does that be using the
> > > vivante-
> > > linuxfb-API which results in a separated, independent "connection" to
> the
> > > display.
> > > This means glupload's "external-opengl-context" is not usable as any
> > > provided
> > > EGLContext will be bound to another display connection (I tried and
> > > failed, if
> > > it's supposed to work but I'm doing something wrong, I'm happy to try
> > > again
> > > with help)
> > >
> > > This means you'll never be able to use the generated textures outside
> of
> > > gstreamer. ... I think ;)
> > > The only use-case left is to use glimagesink to render to framebuffer
> > > directly,
> > > maybe adding some gstreamer-gl-effects or something.
> > >
> > > This is not what I indented to achieve (although I've already marked
> it as
> > > my
> > > back-up plan and doing it in QtMultimedia is trivial, if someone wants
> to
> > > try
> > > that solution let me know and I tell you what to change)
> > >
> > > To achieve what I want I thing the only viable solution would be to
> > > replace
> > > the "external-opengl-context" with a 'makeGLContextCurrent'-callback
> ...so
> > > that the EGLContext can be created and managed by the UI-framework
> rather
> > > then
> > > the multimedia-framework... but I'm not eager to fiddle around with
> > > gst-API ;)
> > >
> > >
> > >
> > > If I'm wrong with any part and it's actually possible to use textures
> > > generated by gst-plugins-gst outside of gstreamer (without X/other
> > > 'windowing
> > > manager'), I happy to try again if someone has a hint/idea how the
> stack
> > > should look like.
> > >
> > >
> > >
> > > In the mean while I have an alternative plan:
> > > The important part the thing is replacing the cpu based texture upload
> > > (copy)
> > > with a vivante-based memory-mapping ('no-copy'/'zero-copy')
> > > Today I'm going try to implement this (using the gst-plugins-gl code as
> > > reference) within QtMultimedia.
> >
> > I'm very new to opengl, but your analysis sounds correct to me.
> >
> > You might be interested in looking at the qt-gstreamer package. It has a
> > qtglvideosink plugin which uses the context you give it, rather than
> > creating its own display etc. I did try using it, but I was trying to
> use a
> > QGLWidget to get the context, and it seems to be incompatible with eglfs.
> > It displays fine on my laptop with xcb. Maybe you know an alternative way
> > of getting a context when using eglfs.
>
> But this is not imx6 specific, is it?
> If it's not: we can't get to a optimized stack without using the vivante-
> opengl ui
> If it is: I'll take a look (where to find?)
>

I'm not sure why it needs to be imx6 specific. Doesn't the opengl API make
it so that it doesn't need to be specific? The Qt eglfs platform is using
the vivante opengl libraries. As I understand it, the parts that need to be
imx6 specific are confined to the eglfs platform plugin.

qt-gstreamer is at http://cgit.freedesktop.org/gstreamer/qt-gstreamer

[-- Attachment #2: Type: text/html, Size: 8097 bytes --]

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
> On Wed, Jul 10, 2013 at 7:52 PM, Thomas Senyk
> 
> <thomas.senyk@pelagicore.com>wrote:
> > On Wednesday, 10 July, 2013 19:13:57 Philip Craig wrote:
> > > On Wed, Jul 10, 2013 at 6:26 PM, Thomas Senyk
> > > 
> > > <thomas.senyk@pelagicore.com>wrote:
> > > > On Wednesday, 10 July, 2013 10:25:09 Philip Craig wrote:
> > > > > On Wed, Jul 10, 2013 at 9:53 AM, Rogerio Nunes <ronunes@gmail.com>
> > > > 
> > > > wrote:
> > > > > > On Tue, Jul 9, 2013 at 4:14 AM, Thomas Senyk <
> > > > 
> > > > thomas.senyk@pelagicore.com>
> > > > 
> > > > > > wrote:
> > > > > > > On Friday, 05 July, 2013 13:16:23 Rogerio Nunes wrote:
> > > > > > >> Repository is public now.
> > > > > > >> I only have the code from BSP 1.1.0 there yet, but I'll add
> > 
> > 4.0.0
> > 
> > > > this
> > > > 
> > > > > > >> afternoon.
> > > > > > >> 
> > > > > > >> BTW, I still need to update the gst-plugins-gl recipe in the
> > > > > > >> meta-fsl-arm layer to point to this repo. It's cleaner than the
> > > > 
> > > > patch
> > > > 
> > > > > > >> we currently have in the recipe.
> > > > > > > 
> > > > > > > Sounds wonderful!
> > > > > > > If something is ready to test, let me know.
> > > > > > 
> > > > > > Very small change from previous BSP to 4.0.0 in the gst-plugins-gl
> > 
> > > > > > package:
> > https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/1d6c0abf2
> > 
> > > > > > 17a90e160fd7e4c45f02a23da974130?at=fsl-0.10 I've just sent a patch
> > 
> > to
> > 
> > > > the
> > > > 
> > > > > > list (awaiting approval as it was big...) to update meta-fsl-arm.
> > > > > > 
> > > > > > In the bitbucket repo, branch fsl-0.10 has commits that reflect
> > > > > > "as
> > > > > > is"
> > > > > > packages from the BSP.
> > > > > > My idea now is to work on branch 0.10 to generate a set of patches
> > > > 
> > > > that we
> > > > 
> > > > > > can upstream.
> > > > > > I would appreciate any help.
> > > > > 
> > > > > I can't speak for upstream, but do note that gstreamer 0.10 is no
> > 
> > longer
> > 
> > > > > maintained, and gst-plugins-gl has recently been updated to 1.0, so
> > 
> > it
> > 
> > > > > might be worth checking if upstream will accept 0.10 patches before
> > > > 
> > > > working
> > > > 
> > > > > on them.
> > > >  
> > > >  ... again ... a rather long mail ;)
> > > > 
> > > > I kind a gave up on 'glupload' (and therefor on gst-plugins-gl)
> > 
> > yesterday
> > 
> > > > evening.
> > > > It sounded very good at first, but after a while I realized it can't
> > 
> > work
> > 
> > > > in
> > > > it's current implementation/API ... it least not without X (which is a
> > 
> > no
> > 
> > > > go
> > > > for my in any case)
> > > > 
> > > > What I wanted to achieve:
> > > > Being able to use textures 'filled' by gst-plugins-gl within Qt's
> > 
> > OpenGL
> > 
> > > > SceneGraph.
> > > > 
> > > > Why IMO it will not work:
> > > > glupload has to create it's own EGLContext, for that one needs
> > > > displays
> > > > and
> > > > window.
> > > > The problem is that the current implementation does that be using the
> > > > vivante-
> > > > linuxfb-API which results in a separated, independent "connection" to
> > 
> > the
> > 
> > > > display.
> > > > This means glupload's "external-opengl-context" is not usable as any
> > > > provided
> > > > EGLContext will be bound to another display connection (I tried and
> > > > failed, if
> > > > it's supposed to work but I'm doing something wrong, I'm happy to try
> > > > again
> > > > with help)
> > > > 
> > > > This means you'll never be able to use the generated textures outside
> > 
> > of
> > 
> > > > gstreamer. ... I think ;)
> > > > The only use-case left is to use glimagesink to render to framebuffer
> > > > directly,
> > > > maybe adding some gstreamer-gl-effects or something.
> > > > 
> > > > This is not what I indented to achieve (although I've already marked
> > 
> > it as
> > 
> > > > my
> > > > back-up plan and doing it in QtMultimedia is trivial, if someone wants
> > 
> > to
> > 
> > > > try
> > > > that solution let me know and I tell you what to change)
> > > > 
> > > > To achieve what I want I thing the only viable solution would be to
> > > > replace
> > > > the "external-opengl-context" with a 'makeGLContextCurrent'-callback
> > 
> > ...so
> > 
> > > > that the EGLContext can be created and managed by the UI-framework
> > 
> > rather
> > 
> > > > then
> > > > the multimedia-framework... but I'm not eager to fiddle around with
> > > > gst-API ;)
> > > > 
> > > > 
> > > > 
> > > > If I'm wrong with any part and it's actually possible to use textures
> > > > generated by gst-plugins-gst outside of gstreamer (without X/other
> > > > 'windowing
> > > > manager'), I happy to try again if someone has a hint/idea how the
> > 
> > stack
> > 
> > > > should look like.
> > > > 
> > > > 
> > > > 
> > > > In the mean while I have an alternative plan:
> > > > The important part the thing is replacing the cpu based texture upload
> > > > (copy)
> > > > with a vivante-based memory-mapping ('no-copy'/'zero-copy')
> > > > Today I'm going try to implement this (using the gst-plugins-gl code
> > > > as
> > > > reference) within QtMultimedia.
> > > 
> > > I'm very new to opengl, but your analysis sounds correct to me.
> > > 
> > > You might be interested in looking at the qt-gstreamer package. It has a
> > > qtglvideosink plugin which uses the context you give it, rather than
> > > creating its own display etc. I did try using it, but I was trying to
> > 
> > use a
> > 
> > > QGLWidget to get the context, and it seems to be incompatible with
> > > eglfs.
> > > It displays fine on my laptop with xcb. Maybe you know an alternative
> > > way
> > > of getting a context when using eglfs.
> > 
> > But this is not imx6 specific, is it?
> > If it's not: we can't get to a optimized stack without using the vivante-
> > opengl ui
> > If it is: I'll take a look (where to find?)
> 
> I'm not sure why it needs to be imx6 specific. Doesn't the opengl API make
> it so that it doesn't need to be specific? The Qt eglfs platform is using
> the vivante opengl libraries. As I understand it, the parts that need to be
> imx6 specific are confined to the eglfs platform plugin.
> 
> qt-gstreamer is at http://cgit.freedesktop.org/gstreamer/qt-gstreamer

No. There is no vendor-independent way to map cpu/vpu based memory into gpu 
(/opengl) memory.
Without 'mapping' one needs to copy/upload via glTexImage2D.
This one single line(!) consumes >100% cpu of one iMX6 core  for 1080p.
If you 'map' the cpu load is very easily <10%

Greets
Thomas

Re: QtMultimedia on i.MX6

[Philip Craig]

On Wed, Jul 10, 2013 at 8:41 PM, Thomas Senyk
<thomas.senyk@pelagicore.com> wrote:
> On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
>> I'm not sure why it needs to be imx6 specific. Doesn't the opengl API make
>> it so that it doesn't need to be specific? The Qt eglfs platform is using
>> the vivante opengl libraries. As I understand it, the parts that need to be
>> imx6 specific are confined to the eglfs platform plugin.
>>
>> qt-gstreamer is at http://cgit.freedesktop.org/gstreamer/qt-gstreamer
>
> No. There is no vendor-independent way to map cpu/vpu based memory into gpu
> (/opengl) memory.
> Without 'mapping' one needs to copy/upload via glTexImage2D.
> This one single line(!) consumes >100% cpu of one iMX6 core  for 1080p.
> If you 'map' the cpu load is very easily <10%

Thanks for the explanation. Just to be sure I understand you: there is
no standard way to allocate dma buffers, so the gpu cannot do the copy
itself? I see that gst-plugins-gl relies on gstbufmeta from the
gst-fsl-plugins package to determine the physical address of dma
buffers allocated by the vpu plugin or by the v4l2src.

And yes, qt-gstreamer is using glTexImage2D. I guess that could be
patched, but then you may as well just patch QtMultimedia like you
were planning.

Re: QtMultimedia on i.MX6

[Robert Winkler]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Wed, Jul 10, 2013 at 6:30 AM, Philip Craig <phil@blackmoth.com.au> wrote:

> On Wed, Jul 10, 2013 at 8:41 PM, Thomas Senyk
> <thomas.senyk@pelagicore.com> wrote:
> > On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
> >> I'm not sure why it needs to be imx6 specific. Doesn't the opengl API
> make
> >> it so that it doesn't need to be specific? The Qt eglfs platform is
> using
> >> the vivante opengl libraries. As I understand it, the parts that need
> to be
> >> imx6 specific are confined to the eglfs platform plugin.
> >>
> >> qt-gstreamer is at http://cgit.freedesktop.org/gstreamer/qt-gstreamer
> >
> > No. There is no vendor-independent way to map cpu/vpu based memory into
> gpu
> > (/opengl) memory.
> > Without 'mapping' one needs to copy/upload via glTexImage2D.
> > This one single line(!) consumes >100% cpu of one iMX6 core  for 1080p.
> > If you 'map' the cpu load is very easily <10%
>

I was under the same impression as Philip that OpenGL does provide the
functions glMapBuffer and glMapBufferRange.  Standard OpenGL does provide
both.
Then I checked and OpenGL ES 2.0 does not have those functions but OpenGL
ES 3.0 has the latter (
https://www.khronos.org/opengles/sdk/docs/man3/xhtml/glMapBufferRange.xml).

So to clarify, if the i.MX 6 did support OpenGL ES 3 would that qualify as
a vendor-independent way to map memory?  Sorry it's slightly unrelated I'm
just curious.


> Thanks for the explanation. Just to be sure I understand you: there is
> no standard way to allocate dma buffers, so the gpu cannot do the copy
> itself? I see that gst-plugins-gl relies on gstbufmeta from the
> gst-fsl-plugins package to determine the physical address of dma
> buffers allocated by the vpu plugin or by the v4l2src.
>
> And yes, qt-gstreamer is using glTexImage2D. I guess that could be
> patched, but then you may as well just patch QtMultimedia like you
> were planning.
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale
>

[-- Attachment #2: Type: text/html, Size: 3250 bytes --]

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Thursday, 11 July, 2013 12:58:34 Robert Winkler wrote:
> On Wed, Jul 10, 2013 at 6:30 AM, Philip Craig <phil@blackmoth.com.au> wrote:
> > On Wed, Jul 10, 2013 at 8:41 PM, Thomas Senyk
> > 
> > <thomas.senyk@pelagicore.com> wrote:
> > > On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
> > >> I'm not sure why it needs to be imx6 specific. Doesn't the opengl API
> > 
> > make
> > 
> > >> it so that it doesn't need to be specific? The Qt eglfs platform is
> > 
> > using
> > 
> > >> the vivante opengl libraries. As I understand it, the parts that need
> > 
> > to be
> > 
> > >> imx6 specific are confined to the eglfs platform plugin.
> > >> 
> > >> qt-gstreamer is at http://cgit.freedesktop.org/gstreamer/qt-gstreamer
> > > 
> > > No. There is no vendor-independent way to map cpu/vpu based memory into
> > 
> > gpu
> > 
> > > (/opengl) memory.
> > > Without 'mapping' one needs to copy/upload via glTexImage2D.
> > > This one single line(!) consumes >100% cpu of one iMX6 core  for 1080p.
> > > If you 'map' the cpu load is very easily <10%
> 
> I was under the same impression as Philip that OpenGL does provide the
> functions glMapBuffer and glMapBufferRange.  Standard OpenGL does provide
> both.
> Then I checked and OpenGL ES 2.0 does not have those functions but OpenGL
> ES 3.0 has the latter (
> https://www.khronos.org/opengles/sdk/docs/man3/xhtml/glMapBufferRange.xml).
> 
> So to clarify, if the i.MX 6 did support OpenGL ES 3 would that qualify as
> a vendor-independent way to map memory?  Sorry it's slightly unrelated I'm
> just curious.

Possibly ... partly! :)
You're still missing YUV support.
So you either have a hardware-converter (unlikely) or you got a opengl-YUV-
extensions (which is part of the vivante api)

So opengl es 3.0 + yuv-extension ... might be possible :)

But then you're back to needing a extension ;) (at least it could be vendor 
independent)


When does the imx6 get opengl es 3.0 anyway?
I've read that the chip (gc2000) is opengl es3.0 capable ..?... so it's just a 
matter of drivers?
I've already seen opengl 3.0 features ... e.g. multisample-texture support!

> 
> > Thanks for the explanation. Just to be sure I understand you: there is
> > no standard way to allocate dma buffers, so the gpu cannot do the copy
> > itself? I see that gst-plugins-gl relies on gstbufmeta from the
> > gst-fsl-plugins package to determine the physical address of dma
> > buffers allocated by the vpu plugin or by the v4l2src.
> > 
> > And yes, qt-gstreamer is using glTexImage2D. I guess that could be
> > patched, but then you may as well just patch QtMultimedia like you
> > were planning.
> > _______________________________________________
> > meta-freescale mailing list
> > meta-freescale@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/meta-freescale

Re: QtMultimedia on i.MX6

[Robert Winkler]

[-- Attachment #1: Type: text/plain, Size: 4075 bytes --]

On Fri, Jul 12, 2013 at 1:56 AM, Thomas Senyk
<thomas.senyk@pelagicore.com>wrote:

> On Thursday, 11 July, 2013 12:58:34 Robert Winkler wrote:
> > On Wed, Jul 10, 2013 at 6:30 AM, Philip Craig <phil@blackmoth.com.au>
> wrote:
> > > On Wed, Jul 10, 2013 at 8:41 PM, Thomas Senyk
> > >
> > > <thomas.senyk@pelagicore.com> wrote:
> > > > On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
> > > >> I'm not sure why it needs to be imx6 specific. Doesn't the opengl
> API
> > >
> > > make
> > >
> > > >> it so that it doesn't need to be specific? The Qt eglfs platform is
> > >
> > > using
> > >
> > > >> the vivante opengl libraries. As I understand it, the parts that
> need
> > >
> > > to be
> > >
> > > >> imx6 specific are confined to the eglfs platform plugin.
> > > >>
> > > >> qt-gstreamer is at
> http://cgit.freedesktop.org/gstreamer/qt-gstreamer
> > > >
> > > > No. There is no vendor-independent way to map cpu/vpu based memory
> into
> > >
> > > gpu
> > >
> > > > (/opengl) memory.
> > > > Without 'mapping' one needs to copy/upload via glTexImage2D.
> > > > This one single line(!) consumes >100% cpu of one iMX6 core  for
> 1080p.
> > > > If you 'map' the cpu load is very easily <10%
> >
> > I was under the same impression as Philip that OpenGL does provide the
> > functions glMapBuffer and glMapBufferRange.  Standard OpenGL does provide
> > both.
> > Then I checked and OpenGL ES 2.0 does not have those functions but OpenGL
> > ES 3.0 has the latter (
> >
> https://www.khronos.org/opengles/sdk/docs/man3/xhtml/glMapBufferRange.xml
> ).
> >
> > So to clarify, if the i.MX 6 did support OpenGL ES 3 would that qualify
> as
> > a vendor-independent way to map memory?  Sorry it's slightly unrelated
> I'm
> > just curious.
>
> Possibly ... partly! :)
> You're still missing YUV support.
> So you either have a hardware-converter (unlikely) or you got a opengl-YUV-
> extensions (which is part of the vivante api)
>
> So opengl es 3.0 + yuv-extension ... might be possible :)
>
> But then you're back to needing a extension ;) (at least it could be vendor
> independent)
>
>
> When does the imx6 get opengl es 3.0 anyway?
> I've read that the chip (gc2000) is opengl es3.0 capable ..?... so it's
> just a
> matter of drivers?
> I've already seen opengl 3.0 features ... e.g. multisample-texture support!
>

I don't think it ever will.  Where did you read that it is 3.0 capable?
That'd be awesome but I've never seen it.  I've only ever seen them say
it supports ES 2.0.

Actually the closest thing I can find right now is this page which lists
several of there GPU's.
http://www.vivantecorp.com/en/technology/3d.html

Unfortunately it doesn't say specifically which ones support 3.0 and I
can't find a footnote to go with the *.
I thought I read once that the GC4000 was the first/lowest to support 3.0.

Also multisample buffers/rasterization/antialiasing and multitexturing are
all supported
in OpenGL ES 2.0.  I think that probably includes what you've seen.  When I
grep for multisample in the gpu packages/examples
I see stuff about rendering to a texture which is probably this which says
it's written to the 2.0 spec:
http://www.khronos.org/registry/gles/extensions/EXT/EXT_multisampled_render_to_texture.txt





> >
> > > Thanks for the explanation. Just to be sure I understand you: there is
> > > no standard way to allocate dma buffers, so the gpu cannot do the copy
> > > itself? I see that gst-plugins-gl relies on gstbufmeta from the
> > > gst-fsl-plugins package to determine the physical address of dma
> > > buffers allocated by the vpu plugin or by the v4l2src.
> > >
> > > And yes, qt-gstreamer is using glTexImage2D. I guess that could be
> > > patched, but then you may as well just patch QtMultimedia like you
> > > were planning.
> > > _______________________________________________
> > > meta-freescale mailing list
> > > meta-freescale@yoctoproject.org
> > > https://lists.yoctoproject.org/listinfo/meta-freescale
>

[-- Attachment #2: Type: text/html, Size: 5960 bytes --]

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Monday, 15 July, 2013 11:18:34 Robert Winkler wrote:
> On Fri, Jul 12, 2013 at 1:56 AM, Thomas Senyk
> 
> <thomas.senyk@pelagicore.com>wrote:
> > On Thursday, 11 July, 2013 12:58:34 Robert Winkler wrote:
> > > On Wed, Jul 10, 2013 at 6:30 AM, Philip Craig <phil@blackmoth.com.au>
> > 
> > wrote:
> > > > On Wed, Jul 10, 2013 at 8:41 PM, Thomas Senyk
> > > > 
> > > > <thomas.senyk@pelagicore.com> wrote:
> > > > > On Wednesday, 10 July, 2013 20:32:05 Philip Craig wrote:
> > > > >> I'm not sure why it needs to be imx6 specific. Doesn't the opengl
> > 
> > API
> > 
> > > > make
> > > > 
> > > > >> it so that it doesn't need to be specific? The Qt eglfs platform is
> > > > 
> > > > using
> > > > 
> > > > >> the vivante opengl libraries. As I understand it, the parts that
> > 
> > need
> > 
> > > > to be
> > > > 
> > > > >> imx6 specific are confined to the eglfs platform plugin.
> > > > >> 
> > > > >> qt-gstreamer is at
> > 
> > http://cgit.freedesktop.org/gstreamer/qt-gstreamer
> > 
> > > > > No. There is no vendor-independent way to map cpu/vpu based memory
> > 
> > into
> > 
> > > > gpu
> > > > 
> > > > > (/opengl) memory.
> > > > > Without 'mapping' one needs to copy/upload via glTexImage2D.
> > > > > This one single line(!) consumes >100% cpu of one iMX6 core  for
> > 
> > 1080p.
> > 
> > > > > If you 'map' the cpu load is very easily <10%
> > > 
> > > I was under the same impression as Philip that OpenGL does provide the
> > > functions glMapBuffer and glMapBufferRange.  Standard OpenGL does
> > > provide
> > > both.
> > > Then I checked and OpenGL ES 2.0 does not have those functions but
> > > OpenGL
> > > ES 3.0 has the latter (
> > 
> > https://www.khronos.org/opengles/sdk/docs/man3/xhtml/glMapBufferRange.xml
> > ).
> > 
> > > So to clarify, if the i.MX 6 did support OpenGL ES 3 would that qualify
> > 
> > as
> > 
> > > a vendor-independent way to map memory?  Sorry it's slightly unrelated
> > 
> > I'm
> > 
> > > just curious.
> > 
> > Possibly ... partly! :)
> > You're still missing YUV support.
> > So you either have a hardware-converter (unlikely) or you got a
> > opengl-YUV-
> > extensions (which is part of the vivante api)
> > 
> > So opengl es 3.0 + yuv-extension ... might be possible :)
> > 
> > But then you're back to needing a extension ;) (at least it could be
> > vendor
> > independent)
> > 
> > 
> > When does the imx6 get opengl es 3.0 anyway?
> > I've read that the chip (gc2000) is opengl es3.0 capable ..?... so it's
> > just a
> > matter of drivers?
> > I've already seen opengl 3.0 features ... e.g. multisample-texture
> > support!
> 
> I don't think it ever will.  Where did you read that it is 3.0 capable?
> That'd be awesome but I've never seen it.  I've only ever seen them say
> it supports ES 2.0.
> 
> Actually the closest thing I can find right now is this page which lists
> several of there GPU's.
> http://www.vivantecorp.com/en/technology/3d.html

There is a '*' behind "ES 3.0" ... which indicates a footnote or something .. 
can't find any ;)

> 
> Unfortunately it doesn't say specifically which ones support 3.0 and I
> can't find a footnote to go with the *.
> I thought I read once that the GC4000 was the first/lowest to support 3.0.

http://www.khronos.org/news/permalink/vivante-shipping-gpu-cores-designed-to-support-the-latest-opengl-es-3.0-spe

quote: "...GC Cores starting with the GC800..."

No clue if this is in any way reliable ... it's khronos so it should, but it's 
more of a 'somebody stated'-news then a 'that's how it is'-news.

> 
> Also multisample buffers/rasterization/antialiasing and multitexturing are
> all supported
> in OpenGL ES 2.0. 

Yes, but they are a extension in es2.0 and core feature of es3.0, that's what 
I meant with es3.0 feature.

> I think that probably includes what you've seen.  When I
> grep for multisample in the gpu packages/examples
> I see stuff about rendering to a texture which is probably this which says
> it's written to the 2.0 spec:
> http://www.khronos.org/registry/gles/extensions/EXT/EXT_multisampled_render_
> to_texture.txt

Yes, this wasn't present right from the beginning,
It come in with 4.0.0 .. or 1.1.0? .. or maybe earlier and I saw it to late :)

> > > > Thanks for the explanation. Just to be sure I understand you: there is
> > > > no standard way to allocate dma buffers, so the gpu cannot do the copy
> > > > itself? I see that gst-plugins-gl relies on gstbufmeta from the
> > > > gst-fsl-plugins package to determine the physical address of dma
> > > > buffers allocated by the vpu plugin or by the v4l2src.
> > > > 
> > > > And yes, qt-gstreamer is using glTexImage2D. I guess that could be
> > > > patched, but then you may as well just patch QtMultimedia like you
> > > > were planning.
> > > > _______________________________________________
> > > > meta-freescale mailing list
> > > > meta-freescale@yoctoproject.org
> > > > https://lists.yoctoproject.org/listinfo/meta-freescale

Re: QtMultimedia on i.MX6

[Eric Nelson]

On 07/04/2013 02:42 PM, Otavio Salvador wrote:
> On Thu, Jul 4, 2013 at 12:14 PM, Thomas Senyk
> <thomas.senyk@pelagicore.com> wrote:
>> On Thursday, 04 July, 2013 17:03:03 Thomas Senyk wrote:
>>> Hi,
>>>
>>> I've started to investigate to see the QtMultimedia-gstreamer integration on
>>> the i.MX6 could be optimized.
>>>
>>> I'm writing this mail to sync with others who might already done some work
>>> or investigations on this matter, to get hints where to start and what's
>>> worth to investigate.
>>>
>>> State right now:
>>>
>>> The HW-decoders are used (according to gstreamer log).
>>> That's a very good start/out of the box experience! Good job..freescale I
>>> guess?
>>>
>>> There is still a lot of cpu load:
>>>     - 720p: smooth playback but~18% complete cpu load => ~70% of one core
>>>     - 1080p: playback not smooth and ~27% complete cpu load -> 100% of one
>>> core + some additional cpu load
>>>
>>> The obvious reason for this cpu load is some sort of memcopy and/or
>>> conversion.
>>>    ... gstreamer doesn't render into gpu-memory directly.
>>>    ... or (if you want it the other way around) the gstreamer (/cpu) memory
>>> isn't mapped into gpu-memory (which is possibly on the imx6)
>>>
>>> Started to do some digging:
>>>
>>> One of the first things I tried was to compile gst-plugin-gl (which is part
>>> of the meta-fsl-arm layer via a .bbappend).
>>> The current recipe is driver-version-1.1.0 based and is not building out of
>>> the box (at least not for me) ... I got gst-plugins-gl-0.10.3.1.tar.gz
>>> (based on 4.0.0) building.**
>>>
>>> So, after I got it deployed I hoped for magic to happen and everything is
>>> perfect now ... no big surprise: it's not :) nothing changed, possibly the
>>> gst-pipline build by qtmultimedia is the ame.
>>>
>>> The reason for this mail / some questions:
>>>
>>> So I though, I write this mail before I waste a lot of time.
>>> What does people think is the best way forward?
>>> Try to convince QtMultimedia to use glsink?
>>> Try to use a fakesink/memory-based-sink and try to map the cpu memory into
>>> gpu memory via vivante APIs?
>>>
>>> Does anyone have c/c++ code I could try?
>>> Ideally for direct framebuffer/EGL use (I'm not using X11)
>>> I found: https://github.com/andreluizeng/i.MX6-Video-Streaming-Texture.git
>>>   .. but it's crashing rather early and I'm not sure the if the time is well
>>> spend fix this code? Anyone used it before?
>>> Does anyone know if gst-plugins-gl is applicable for framebuffer/EGL setups?
>>>
>>>
>>> Greet
>>> Thomas
>>
>> Ok ... what is it with me and sending mails 1minute to early all the time ...
>> After I pressed "send", I had a very simple and obvious idea .. why not just
>> simply try glimagesink instead of mfw_v4lsink:
>>
>> gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux !
>> vpudec !  glimagesink
>>
>>   ... long shot, but:
>> It's actually working!
>> It just opens /dev/fb0 and renders the video into it.
>>
>> 1080p with 0%(!!) cpu load!
>>

Nice! I stopped short of testing this out because I thought that X was
required.

A quick browse of the sources indicates that there's some really good
stuff inside, including what appears to be the ability to upload
planar YUV into a texture:
	http://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/src/629e1efaf87f379a36456f4e42cdec0ca2ed7e2a/gst-libs/gst/gl/gstglvivante.c?at=fsl/0.10.3#cl-103

>> I think that pretty much answers all questions I had ;)
>> The way forward: patch QtMultimedia to use glimagesink
>>   ... I hope it's going to be trivial to just give him a active FBO or texture.
>>
>>
>> Sorry to everybody who read my way to long mail ;)
>
> It was very good read.
+1

 > By the way, Rogerio (add in Cc) started to try
> to get the gst-plugin-gl code more manageble in bitbucket
> (https://bitbucket.org/Freescale/gstreamer-gst-plugins-gl/commits/all)
> but it was not finished. Maybe you could help on this and we could try
> to merge the Vivante backend upstream? This would allow us to drop the
> bbappend completely.
>
+1

>>> ** some more info in gst-plugin-gl:
>>> It's 'detected' on the board:
>>> root@nitrogen-1:~# gst-inspect |grep gl
>>> opengl:  glupload: OpenGL upload
>>> opengl:  gldownload: OpenGL video maker
>>> opengl:  glimagesink: OpenGL video sink
>>> opengl:  glfiltercube: OpenGL cube filter
>>> opengl:  gleffects: Gstreamer OpenGL Effects
>>>
>>> ... if someone is interested on how I got it building, let me know. If I got
>>> something useful out of it (read: "it's tested") I'll start a discussion on
>>> how to upstream it properly into meta-fsl-arm (by the way: I find the
>>> current setup of how to bbappend and it's patch looks like very weird)

I'm interested in the details of how you added Qt5 to your
build. Qt4 seems to be directly accessible in Yocto, but not Qt5.

Regards,


Eric

Re: QtMultimedia on i.MX6

[Trevor Woerner]

On 5 July 2013 19:04, Eric Nelson <eric.nelson@boundarydevices.com> wrote:
> I'm interested in the details of how you added Qt5 to your
> build. Qt4 seems to be directly accessible in Yocto, but not Qt5.


I'm guessing/hoping the meta-qt5 layer was used?
http://layers.openembedded.org/layerindex/layer/meta-qt5/

Re: QtMultimedia on i.MX6

[Otavio Salvador]

On Fri, Jul 5, 2013 at 10:52 PM, Trevor Woerner
<trevor.woerner@linaro.org> wrote:
> On 5 July 2013 19:04, Eric Nelson <eric.nelson@boundarydevices.com> wrote:
>> I'm interested in the details of how you added Qt5 to your
>> build. Qt4 seems to be directly accessible in Yocto, but not Qt5.
>
>
> I'm guessing/hoping the meta-qt5 layer was used?
> http://layers.openembedded.org/layerindex/layer/meta-qt5/

Yes and if you use master-next you win a qt5 binding for mx6 as well.
I hope to finish testing it and looking at X11 this week so we merge
it in master.

--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: QtMultimedia on i.MX6

[Eric Nelson]

On 07/06/2013 06:09 AM, Otavio Salvador wrote:
> On Fri, Jul 5, 2013 at 10:52 PM, Trevor Woerner
> <trevor.woerner@linaro.org> wrote:
>> On 5 July 2013 19:04, Eric Nelson <eric.nelson@boundarydevices.com> wrote:
>>> I'm interested in the details of how you added Qt5 to your
>>> build. Qt4 seems to be directly accessible in Yocto, but not Qt5.
>>
>>
>> I'm guessing/hoping the meta-qt5 layer was used?
>> http://layers.openembedded.org/layerindex/layer/meta-qt5/
>
> Yes and if you use master-next you win a qt5 binding for mx6 as well.
> I hope to finish testing it and looking at X11 this week so we merge
> it in master.
>

Many thanks!

Re: QtMultimedia on i.MX6

[Thomas Senyk]

On Saturday, 06 July, 2013 10:09:23 Eric Nelson wrote:
> On 07/06/2013 06:09 AM, Otavio Salvador wrote:
> > On Fri, Jul 5, 2013 at 10:52 PM, Trevor Woerner
> > 
> > <trevor.woerner@linaro.org> wrote:
> >> On 5 July 2013 19:04, Eric Nelson <eric.nelson@boundarydevices.com> 
wrote:
> >>> I'm interested in the details of how you added Qt5 to your
> >>> build. Qt4 seems to be directly accessible in Yocto, but not Qt5.
> >> 
> >> I'm guessing/hoping the meta-qt5 layer was used?
> >> http://layers.openembedded.org/layerindex/layer/meta-qt5/

Nop sorry :)
I'm building out-side of yocto using configure/make/... directly.
It's simpler for me as I'm doing that for a couple of years now and I have the 
full environment under control (/I know what I'm doing).
 ... at least for development.

> > 
> > Yes and if you use master-next you win a qt5 binding for mx6 as well.
> > I hope to finish testing it and looking at X11 this week so we merge
> > it in master.
> 
> Many thanks!


I got a step further. I got gstreamer also running within QtMultimedia, but 
using glimagesink seems to be wrong (it still renders directly to framebuffer 
which is not what I wanted in the end), but 'glupload' sounds very promising, 
just need to get all the puzzle pieces into the right order :)

So there is still some work to do, but I think I know what to do.
I keep you guys posted.

> 
> 
> _______________________________________________
> meta-freescale mailing list
> meta-freescale@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-freescale