Using OpenCV and OpenGL together

Using OpenCV and OpenGL together

[Fikret Alim]

[-- Attachment #1.1: Type: text/plain, Size: 1402 bytes --]

Hello,



I am trying to use OpenCV and OpenGL together for some image processing
tasks. I have already run OpenCV examples on IMX6, I can also compile and
run the examples under gpu sdk of Freescale with X11, but when I try to use
them together I have some problems.



Actually, what I am trying to make has already been done by Freescale

http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html

http://cache.freescale.com/files/32bit/doc/app_note/AN4629.pdf



I have tried many times by using these examples in order to create a simple
working binary, but unfortunately I could not achieve that. I have attached
one of my trials (the source code, Makefile and the binary). I have
modified the 06_Texturing example given under GLES2_0 folder of gpu sdk. I
have compiled it without any problem, but when I load the binary to IMX6
board and run the example, I see a black cube rotating. I am waiting for
the video instead of black screen. I have attached the output of the
screen, also. If I add an imshow function after I get the image from the
camera, I can see it without any problem. But when it is used by OpenGL, I
can only see black screen as I said.



What can be wrong? I am using Yocto Dora release and I use hard float,
since soft float is not supported. (I have only stubs_hard.h in my sysroots
folder)



Thanks&Best Regards,

Fikret

[-- Attachment #1.2: Type: text/html, Size: 1936 bytes --]

[-- Attachment #2: screenout.jpg --]
[-- Type: image/jpeg, Size: 54944 bytes --]

[-- Attachment #3: opencv_gpu_text.tar.gz --]
[-- Type: application/x-gzip, Size: 87621 bytes --]

Re: Using OpenCV and OpenGL together

[Marek Vasut]

[-- Attachment #1: Type: Text/Plain, Size: 1665 bytes --]

On Wednesday, December 11, 2013 at 03:18:33 PM, Fikret Alim wrote:
> Hello,
> 
> 
> 
> I am trying to use OpenCV and OpenGL together for some image processing
> tasks. I have already run OpenCV examples on IMX6, I can also compile and
> run the examples under gpu sdk of Freescale with X11, but when I try to use
> them together I have some problems.
> 
> 
> 
> Actually, what I am trying to make has already been done by Freescale
> 
> http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
> 
> http://cache.freescale.com/files/32bit/doc/app_note/AN4629.pdf
> 
> 
> 
> I have tried many times by using these examples in order to create a simple
> working binary, but unfortunately I could not achieve that. I have attached
> one of my trials (the source code, Makefile and the binary). I have
> modified the 06_Texturing example given under GLES2_0 folder of gpu sdk. I
> have compiled it without any problem, but when I load the binary to IMX6
> board and run the example, I see a black cube rotating.

Your texturing is broken, right ?

> I am waiting for
> the video instead of black screen. I have attached the output of the
> screen, also. If I add an imshow function after I get the image from the
> camera, I can see it without any problem. But when it is used by OpenGL, I
> can only see black screen as I said.

The examples are broken, they don't check GL errors etc. Find attached patch, it 
should give you an idea how to fix your glplane.cpp in the examples.

glEGLImageTargetTexture2DOES() usually returns GL error because the fragment 
shader is completely bonkers.

Best regards,
Marek Vasut

[-- Attachment #2: glplane.cpp.diff --]
[-- Type: text/x-patch, Size: 4720 bytes --]

--- a/src/glplane.cpp	2013-12-07 13:57:26.410103736 +0100
+++ b/src/glplane.cpp	2013-12-18 18:46:33.193693194 +0100
@@ -8,20 +8,12 @@
 					"{							\n"
 					" gl_FragColor = texture2D(sampler2d,myTexCoord);	\n" 
 					"}							\n";*/
-	"#ifdef GL_FRAGMENT_PRECISION_HIGH				\n"
-	"   precision highp float;					\n"
-	"#else							\n"
-	"   precision mediump float;				\n"
-	"#endif							\n"
-	"								\n"
 	"uniform sampler2D s_texture;				\n"
-	"varying   vec3      g_vVSColor;				\n"
-	"varying   vec2 g_vVSTexCoord;				\n"
+	"varying mediump vec2 g_vVSTexCoord;				\n"
 	"								\n"
 	"void main()						\n"
 	"{								\n"
-	"    gl_FragColor = texture2D(s_texture,g_vVSTexCoord.xy);	\n"
-
+	"    gl_FragColor = texture2D(s_texture,g_vVSTexCoord);	\n"
 	"}								\n";
 
 
@@ -44,17 +36,13 @@
     "uniform   mat4 g_matProj;					\n"
     "								\n"
     "attribute vec4 g_vPosition;				\n"
-    "attribute vec3 g_vColor;					\n"
     "attribute vec2 g_vTexCoord;				\n"
     "								\n"
-    "varying   vec3 g_vVSColor;					\n"
     "varying   vec2 g_vVSTexCoord;				\n"
     "								\n"
     "void main()						\n"
     "{								\n"
-    "    vec4 vPositionES = g_matModelView * g_vPosition;	\n"
-    "    gl_Position  = g_matProj * vPositionES;		\n"
-    "    g_vVSColor = g_vColor;					\n"
+    "    gl_Position  = g_matProj * g_matModelView * g_vPosition;\n"
     "    g_vVSTexCoord = g_vTexCoord;				\n"
     "}								\n";
 
@@ -392,10 +380,10 @@
 //--------------------------------------------------------------------------------------
 void GLPlane::PlaneSetTexBuf (uint8_t *texture_data, int w, int h)
 {
+	glActiveTexture(GL_TEXTURE0);
 	glBindTexture(GL_TEXTURE_2D, _texture);
-	//glTexImage2D (GL_TEXTURE_2D, 0,	GL_RGB,	w, h, 0, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, texture_data);
 	glTexImage2D (GL_TEXTURE_2D, 0,	GL_RGB, w, h, 0, GL_RGB, GL_UNSIGNED_BYTE, texture_data);
-	
+	glUniform1i(_s_texture_location, 0);
 }
 
 //--------------------------------------------------------------------------------------
@@ -470,24 +458,12 @@
 	// Pass the texture coordinates data
 	glVertexAttribPointer (PLANE_TEXCOORD_ARRAY, 2, GL_FLOAT, 0, 0, texcoords);
 	glEnableVertexAttribArray (PLANE_TEXCOORD_ARRAY);
-	
-	// enable depth test
-	glEnable (GL_DEPTH_TEST);
-	glDepthFunc (GL_LEQUAL);
-	glDepthMask (GL_TRUE);
-
-	// cull backside of polygons
-	//glEnable(GL_CULL_FACE);
-	//glCullFace (GL_BACK);
-	glDisable(GL_CULL_FACE);
-
-	// Binds this texture handle so we can load the data into it
-	//glBindTexture(GL_TEXTURE_2D, _texture);
-       /* Select Our Texture */
-	glActiveTexture(GL_TEXTURE0);
-	//Select eglImage
-	glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, g_imgHandle);
-	
+
+
+
+
+
+
 
 	glDrawArrays (GL_TRIANGLES, 0, 6);
 
@@ -632,6 +608,7 @@
 //--------------------------------------------------------------------------------------
 GLuint GLPlane::GenTextures (void)
 {
+#if 0
 	GLuint texture_id = 0;
 
 	// Use tightly packed data
@@ -639,8 +616,38 @@
 
 	// Allocates one texture handle
 	glGenTextures (1, &texture_id);
+#endif
+
+   // Texture object handle
+   GLuint textureId;
+   
+   // 2x2 Image, 3 bytes per pixel (R, G, B)
+   GLubyte pixels[4 * 3] =
+   {  
+      255,   0,   0, // Red
+        0, 255,   0, // Green
+        0,   0, 255, // Blue
+      255, 255,   0  // Yellow
+   };
+
+   // Use tightly packed data
+   glPixelStorei ( GL_UNPACK_ALIGNMENT, 1 );
+
+   // Generate a texture object
+   glGenTextures ( 1, &textureId );
+
+   // Bind the texture object
+   glBindTexture ( GL_TEXTURE_2D, textureId );
+
+   // Load the texture
+   glTexImage2D ( GL_TEXTURE_2D, 0, GL_RGB, 2, 2, 0, GL_RGB, GL_UNSIGNED_BYTE, pixels );
+
+   // Set the filtering mode
+   glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
+   glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
+
 
-	return texture_id;
+	return textureId;
 }
 
 //--------------------------------------------------------------------------------------
@@ -654,10 +661,10 @@
 	_texture_h = texture_h;
 
 	_texture			= GenTextures ();
-	
+#if 0	
 	glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 	glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
-
+#endif
 	return;
 }
 
@@ -765,7 +772,7 @@
     // Get uniform locations
 	_modelview_location = glGetUniformLocation (_shader_program, "g_matModelView");
 	_projview_location  = glGetUniformLocation (_shader_program, "g_matProj");
-
+	_s_texture_location = glGetUniformLocation (_shader_program, "s_texture");
 	return true;
 }
 

Re: Using OpenCV and OpenGL together

[Fabio Estevam]

On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com> wrote:
> Hello,
>
>
>
> I am trying to use OpenCV and OpenGL together for some image processing
> tasks. I have already run OpenCV examples on IMX6, I can also compile and
> run the examples under gpu sdk of Freescale with X11, but when I try to use
> them together I have some problems.
>
>
>
> Actually, what I am trying to make has already been done by Freescale
>
> http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html

Adding Andre.

Regards,

Fabio Estevam

Re: Using OpenCV and OpenGL together

[Fikret Alim]

[-- Attachment #1: Type: text/plain, Size: 2273 bytes --]

Dear Marek,

Thank you for your answer,

I will check the file you sent. I have solved the problem in a different
way, I am not sure if this is the correct way or not, but at least it
worked after this change:

I have only moved the following two lines from PlaneCreateTex function to
PlaneSetTex function.

 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

Then, PlaneSetTex function will be;

void PlaneSetTex (Mat texture_data)
{
 cvtColor (texture_data, _texture_data, CV_BGR2RGB);
 //imshow("_texture_data",_texture_data);
 glBindTexture(GL_TEXTURE_2D, _texture);
 glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB,
GL_UNSIGNED_BYTE, _texture_data.data);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
}

With this change, now I can run Andre's gesture recognition example, and
also I have tried some GLSL shaders for image processing like sobel
detection.

But, let me ask a different question: Is there a limition for the webcam
resolution? I can only use 320x240 resolution for both Andre's example and
gpu sdk examples. If I set a higher resolution, then I get segmentation
fault error. I am sure that my webcam supports higher resolutions, since I
can get video with openCV at higher resolutions. What can be wrong? Is
there a limitation, or do I need to perform some other operations for
higher resolutions?

Thanks&Best Regards,
Fikret


On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com> wrote:

> On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com>
> wrote:
> > Hello,
> >
> >
> >
> > I am trying to use OpenCV and OpenGL together for some image processing
> > tasks. I have already run OpenCV examples on IMX6, I can also compile and
> > run the examples under gpu sdk of Freescale with X11, but when I try to
> use
> > them together I have some problems.
> >
> >
> >
> > Actually, what I am trying to make has already been done by Freescale
> >
> > http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
>
> Adding Andre.
>
> Regards,
>
> Fabio Estevam
>

[-- Attachment #2: Type: text/html, Size: 3045 bytes --]

Re: Using OpenCV and OpenGL together

[Andre Silva]

[-- Attachment #1: Type: text/plain, Size: 2850 bytes --]

Hi Fikret,

I believe that there are another functions in the code using this resolution, if you change the resolution without modifying these other parts the segfault will occur. The only limitation using 320x240 is the performance, since OpenCV runs on cpu, if we use higher resolution the performance will decrease a lot.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com]
Sent: Thursday, December 19, 2013 5:40 AM
To: Fabio Estevam
Cc: meta-freescale@yoctoproject.org; Silva Andre-B22958
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Dear Marek,

Thank you for your answer,

I will check the file you sent. I have solved the problem in a different way, I am not sure if this is the correct way or not, but at least it worked after this change:

I have only moved the following two lines from PlaneCreateTex function to PlaneSetTex function.

 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

Then, PlaneSetTex function will be;

void PlaneSetTex (Mat texture_data)
{
 cvtColor (texture_data, _texture_data, CV_BGR2RGB);
 //imshow("_texture_data",_texture_data);
 glBindTexture(GL_TEXTURE_2D, _texture);
 glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, _texture_data.data);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
}

With this change, now I can run Andre's gesture recognition example, and also I have tried some GLSL shaders for image processing like sobel detection.

But, let me ask a different question: Is there a limition for the webcam resolution? I can only use 320x240 resolution for both Andre's example and gpu sdk examples. If I set a higher resolution, then I get segmentation fault error. I am sure that my webcam supports higher resolutions, since I can get video with openCV at higher resolutions. What can be wrong? Is there a limitation, or do I need to perform some other operations for higher resolutions?

Thanks&Best Regards,
Fikret

On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com<mailto:festevam@gmail.com>> wrote:
On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>> wrote:
> Hello,
>
>
>
> I am trying to use OpenCV and OpenGL together for some image processing
> tasks. I have already run OpenCV examples on IMX6, I can also compile and
> run the examples under gpu sdk of Freescale with X11, but when I try to use
> them together I have some problems.
>
>
>
> Actually, what I am trying to make has already been done by Freescale
>
> http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
Adding Andre.

Regards,

Fabio Estevam


[-- Attachment #2: Type: text/html, Size: 7670 bytes --]

Re: Using OpenCV and OpenGL together

[Fikret Alim]

[-- Attachment #1: Type: text/plain, Size: 3943 bytes --]

Hi Andre,

There are only two places that I change for the resolution inside the code.
This first one is capture resolution that openCV uses, the other one is
glTexImage2D function. I only modify them for the resolution and I don't
see any other functions that can be related to resolution.

cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB,
GL_UNSIGNED_BYTE, bgr_img.data);

The strange thing is; if I set 160x120, 320x240 and even 320x320
resolutions, I don't see any problem. But if I set higher resolutions such
as 640x480, 800x600 or 1024x768, etc., I got segmentation fault. Do you
have a recommendation for me where and what to check inside the code?

Thanks,
Fikret


On Thu, Dec 19, 2013 at 1:55 PM, Andre Silva <Andre.Silva@freescale.com>wrote:

>  Hi Fikret,
>
>
>
> I believe that there are another functions in the code using this
> resolution, if you change the resolution without modifying these other
> parts the segfault will occur. The only limitation using 320x240 is the
> performance, since OpenCV runs on cpu, if we use higher resolution the
> performance will decrease a lot.
>
>
>
> Regards,
>
> andre
>
>
>
> *From:* Fikret Alim [mailto:fikret.alim@gmail.com]
> *Sent:* Thursday, December 19, 2013 5:40 AM
> *To:* Fabio Estevam
> *Cc:* meta-freescale@yoctoproject.org; Silva Andre-B22958
> *Subject:* Re: [meta-freescale] Using OpenCV and OpenGL together
>
>
>
> Dear Marek,
>
>
>
> Thank you for your answer,
>
>
>
> I will check the file you sent. I have solved the problem in a different
> way, I am not sure if this is the correct way or not, but at least it
> worked after this change:
>
>
>
> I have only moved the following two lines from PlaneCreateTex function to
> PlaneSetTex function.
>
>
>
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
>
>
>
> Then, PlaneSetTex function will be;
>
>
>
> void PlaneSetTex (Mat texture_data)
> {
>  cvtColor (texture_data, _texture_data, CV_BGR2RGB);
>  //imshow("_texture_data",_texture_data);
>  glBindTexture(GL_TEXTURE_2D, _texture);
>  glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB,
> GL_UNSIGNED_BYTE, _texture_data.data);
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
> }
>
>
>
> With this change, now I can run Andre's gesture recognition example, and
> also I have tried some GLSL shaders for image processing like sobel
> detection.
>
>
>
> But, let me ask a different question: Is there a limition for the webcam
> resolution? I can only use 320x240 resolution for both Andre's example and
> gpu sdk examples. If I set a higher resolution, then I get segmentation
> fault error. I am sure that my webcam supports higher resolutions, since I
> can get video with openCV at higher resolutions. What can be wrong? Is
> there a limitation, or do I need to perform some other operations for
> higher resolutions?
>
>
>
> Thanks&Best Regards,
>
> Fikret
>
>
>
> On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com> wrote:
>
> On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com>
> wrote:
> > Hello,
> >
> >
> >
> > I am trying to use OpenCV and OpenGL together for some image processing
> > tasks. I have already run OpenCV examples on IMX6, I can also compile and
> > run the examples under gpu sdk of Freescale with X11, but when I try to
> use
> > them together I have some problems.
> >
> >
> >
> > Actually, what I am trying to make has already been done by Freescale
> >
> > http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
>
> Adding Andre.
>
> Regards,
>
> Fabio Estevam
>
>
>

[-- Attachment #2: Type: text/html, Size: 7678 bytes --]

Re: Using OpenCV and OpenGL together

[Andre Silva]

[-- Attachment #1: Type: text/plain, Size: 4294 bytes --]

Hi Friket,

Which OpenCV version are you using ? I remember 1.1 had an issue with setting the camera resolution using these functions, I usually needed to set it directly in CV´s code and then recompiling all the lib.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com]
Sent: Thursday, December 19, 2013 11:56 AM
To: Silva Andre-B22958
Cc: Fabio Estevam; meta-freescale@yoctoproject.org
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Hi Andre,

There are only two places that I change for the resolution inside the code. This first one is capture resolution that openCV uses, the other one is glTexImage2D function. I only modify them for the resolution and I don't see any other functions that can be related to resolution.

cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, bgr_img.data);

The strange thing is; if I set 160x120, 320x240 and even 320x320 resolutions, I don't see any problem. But if I set higher resolutions such as 640x480, 800x600 or 1024x768, etc., I got segmentation fault. Do you have a recommendation for me where and what to check inside the code?

Thanks,
Fikret

On Thu, Dec 19, 2013 at 1:55 PM, Andre Silva <Andre.Silva@freescale.com<mailto:Andre.Silva@freescale.com>> wrote:
Hi Fikret,

I believe that there are another functions in the code using this resolution, if you change the resolution without modifying these other parts the segfault will occur. The only limitation using 320x240 is the performance, since OpenCV runs on cpu, if we use higher resolution the performance will decrease a lot.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>]
Sent: Thursday, December 19, 2013 5:40 AM
To: Fabio Estevam
Cc: meta-freescale@yoctoproject.org<mailto:meta-freescale@yoctoproject.org>; Silva Andre-B22958
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Dear Marek,

Thank you for your answer,

I will check the file you sent. I have solved the problem in a different way, I am not sure if this is the correct way or not, but at least it worked after this change:

I have only moved the following two lines from PlaneCreateTex function to PlaneSetTex function.

 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

Then, PlaneSetTex function will be;

void PlaneSetTex (Mat texture_data)
{
 cvtColor (texture_data, _texture_data, CV_BGR2RGB);
 //imshow("_texture_data",_texture_data);
 glBindTexture(GL_TEXTURE_2D, _texture);
 glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, _texture_data.data);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
}

With this change, now I can run Andre's gesture recognition example, and also I have tried some GLSL shaders for image processing like sobel detection.

But, let me ask a different question: Is there a limition for the webcam resolution? I can only use 320x240 resolution for both Andre's example and gpu sdk examples. If I set a higher resolution, then I get segmentation fault error. I am sure that my webcam supports higher resolutions, since I can get video with openCV at higher resolutions. What can be wrong? Is there a limitation, or do I need to perform some other operations for higher resolutions?

Thanks&Best Regards,
Fikret

On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com<mailto:festevam@gmail.com>> wrote:
On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>> wrote:
> Hello,
>
>
>
> I am trying to use OpenCV and OpenGL together for some image processing
> tasks. I have already run OpenCV examples on IMX6, I can also compile and
> run the examples under gpu sdk of Freescale with X11, but when I try to use
> them together I have some problems.
>
>
>
> Actually, what I am trying to make has already been done by Freescale
>
> http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
Adding Andre.

Regards,

Fabio Estevam



[-- Attachment #2: Type: text/html, Size: 13709 bytes --]

Re: Using OpenCV and OpenGL together

[Fikret Alim]

[-- Attachment #1: Type: text/plain, Size: 5034 bytes --]

Dear Andre,

I am using OpenCV 2.4 and I am sure that OpenCV correctly sets the
resolution, because if I comment out everything related to OpenGL and add
and imshow command after capturing the image with OpenCV, there is no
proble, I can see the video resolution as expected. So, I think that the
problem is related to OpenGL part of the code.

Thanks,
Fikret


On Thu, Dec 19, 2013 at 4:40 PM, Andre Silva <Andre.Silva@freescale.com>wrote:

>  Hi Friket,
>
>
>
> Which OpenCV version are you using ? I remember 1.1 had an issue with
> setting the camera resolution using these functions, I usually needed to
> set it directly in CV´s code and then recompiling all the lib.
>
>
>
> Regards,
>
> andre
>
>
>
> *From:* Fikret Alim [mailto:fikret.alim@gmail.com]
> *Sent:* Thursday, December 19, 2013 11:56 AM
> *To:* Silva Andre-B22958
> *Cc:* Fabio Estevam; meta-freescale@yoctoproject.org
>
> *Subject:* Re: [meta-freescale] Using OpenCV and OpenGL together
>
>
>
> Hi Andre,
>
>
>
> There are only two places that I change for the resolution inside the
> code. This first one is capture resolution that openCV uses, the other one
> is glTexImage2D function. I only modify them for the resolution and I don't
> see any other functions that can be related to resolution.
>
>
>
> cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
>
> cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);
>
>
>
> glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB,
> GL_UNSIGNED_BYTE, bgr_img.data);
>
>
>
> The strange thing is; if I set 160x120, 320x240 and even 320x320
> resolutions, I don't see any problem. But if I set higher resolutions such
> as 640x480, 800x600 or 1024x768, etc., I got segmentation fault. Do you
> have a recommendation for me where and what to check inside the code?
>
>
>
> Thanks,
>
> Fikret
>
>
>
> On Thu, Dec 19, 2013 at 1:55 PM, Andre Silva <Andre.Silva@freescale.com>
> wrote:
>
> Hi Fikret,
>
>
>
> I believe that there are another functions in the code using this
> resolution, if you change the resolution without modifying these other
> parts the segfault will occur. The only limitation using 320x240 is the
> performance, since OpenCV runs on cpu, if we use higher resolution the
> performance will decrease a lot.
>
>
>
> Regards,
>
> andre
>
>
>
> *From:* Fikret Alim [mailto:fikret.alim@gmail.com]
> *Sent:* Thursday, December 19, 2013 5:40 AM
> *To:* Fabio Estevam
> *Cc:* meta-freescale@yoctoproject.org; Silva Andre-B22958
> *Subject:* Re: [meta-freescale] Using OpenCV and OpenGL together
>
>
>
> Dear Marek,
>
>
>
> Thank you for your answer,
>
>
>
> I will check the file you sent. I have solved the problem in a different
> way, I am not sure if this is the correct way or not, but at least it
> worked after this change:
>
>
>
> I have only moved the following two lines from PlaneCreateTex function to
> PlaneSetTex function.
>
>
>
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
>
>
>
> Then, PlaneSetTex function will be;
>
>
>
> void PlaneSetTex (Mat texture_data)
> {
>  cvtColor (texture_data, _texture_data, CV_BGR2RGB);
>  //imshow("_texture_data",_texture_data);
>  glBindTexture(GL_TEXTURE_2D, _texture);
>  glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB,
> GL_UNSIGNED_BYTE, _texture_data.data);
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
>  glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
> }
>
>
>
> With this change, now I can run Andre's gesture recognition example, and
> also I have tried some GLSL shaders for image processing like sobel
> detection.
>
>
>
> But, let me ask a different question: Is there a limition for the webcam
> resolution? I can only use 320x240 resolution for both Andre's example and
> gpu sdk examples. If I set a higher resolution, then I get segmentation
> fault error. I am sure that my webcam supports higher resolutions, since I
> can get video with openCV at higher resolutions. What can be wrong? Is
> there a limitation, or do I need to perform some other operations for
> higher resolutions?
>
>
>
> Thanks&Best Regards,
>
> Fikret
>
>
>
> On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com> wrote:
>
> On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com>
> wrote:
> > Hello,
> >
> >
> >
> > I am trying to use OpenCV and OpenGL together for some image processing
> > tasks. I have already run OpenCV examples on IMX6, I can also compile and
> > run the examples under gpu sdk of Freescale with X11, but when I try to
> use
> > them together I have some problems.
> >
> >
> >
> > Actually, what I am trying to make has already been done by Freescale
> >
> > http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
>
> Adding Andre.
>
> Regards,
>
> Fabio Estevam
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 11241 bytes --]

Re: Using OpenCV and OpenGL together

[Andre Silva]

[-- Attachment #1: Type: text/plain, Size: 5274 bytes --]

Hi Friket,

Please, check the imageSize (image->imageSize) to see if nothing wrong is being passed. I will take a look in the code to see if I find anything else.

Regards,
Andre

From: Fikret Alim [mailto:fikret.alim@gmail.com]
Sent: Thursday, December 19, 2013 12:48 PM
To: Silva Andre-B22958
Cc: Fabio Estevam; meta-freescale@yoctoproject.org
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Dear Andre,

I am using OpenCV 2.4 and I am sure that OpenCV correctly sets the resolution, because if I comment out everything related to OpenGL and add and imshow command after capturing the image with OpenCV, there is no proble, I can see the video resolution as expected. So, I think that the problem is related to OpenGL part of the code.

Thanks,
Fikret

On Thu, Dec 19, 2013 at 4:40 PM, Andre Silva <Andre.Silva@freescale.com<mailto:Andre.Silva@freescale.com>> wrote:
Hi Friket,

Which OpenCV version are you using ? I remember 1.1 had an issue with setting the camera resolution using these functions, I usually needed to set it directly in CV´s code and then recompiling all the lib.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>]
Sent: Thursday, December 19, 2013 11:56 AM
To: Silva Andre-B22958
Cc: Fabio Estevam; meta-freescale@yoctoproject.org<mailto:meta-freescale@yoctoproject.org>

Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Hi Andre,

There are only two places that I change for the resolution inside the code. This first one is capture resolution that openCV uses, the other one is glTexImage2D function. I only modify them for the resolution and I don't see any other functions that can be related to resolution.

cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, bgr_img.data);

The strange thing is; if I set 160x120, 320x240 and even 320x320 resolutions, I don't see any problem. But if I set higher resolutions such as 640x480, 800x600 or 1024x768, etc., I got segmentation fault. Do you have a recommendation for me where and what to check inside the code?

Thanks,
Fikret

On Thu, Dec 19, 2013 at 1:55 PM, Andre Silva <Andre.Silva@freescale.com<mailto:Andre.Silva@freescale.com>> wrote:
Hi Fikret,

I believe that there are another functions in the code using this resolution, if you change the resolution without modifying these other parts the segfault will occur. The only limitation using 320x240 is the performance, since OpenCV runs on cpu, if we use higher resolution the performance will decrease a lot.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>]
Sent: Thursday, December 19, 2013 5:40 AM
To: Fabio Estevam
Cc: meta-freescale@yoctoproject.org<mailto:meta-freescale@yoctoproject.org>; Silva Andre-B22958
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Dear Marek,

Thank you for your answer,

I will check the file you sent. I have solved the problem in a different way, I am not sure if this is the correct way or not, but at least it worked after this change:

I have only moved the following two lines from PlaneCreateTex function to PlaneSetTex function.

 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

Then, PlaneSetTex function will be;

void PlaneSetTex (Mat texture_data)
{
 cvtColor (texture_data, _texture_data, CV_BGR2RGB);
 //imshow("_texture_data",_texture_data);
 glBindTexture(GL_TEXTURE_2D, _texture);
 glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, _texture_data.data);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
}

With this change, now I can run Andre's gesture recognition example, and also I have tried some GLSL shaders for image processing like sobel detection.

But, let me ask a different question: Is there a limition for the webcam resolution? I can only use 320x240 resolution for both Andre's example and gpu sdk examples. If I set a higher resolution, then I get segmentation fault error. I am sure that my webcam supports higher resolutions, since I can get video with openCV at higher resolutions. What can be wrong? Is there a limitation, or do I need to perform some other operations for higher resolutions?

Thanks&Best Regards,
Fikret

On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com<mailto:festevam@gmail.com>> wrote:
On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>> wrote:
> Hello,
>
>
>
> I am trying to use OpenCV and OpenGL together for some image processing
> tasks. I have already run OpenCV examples on IMX6, I can also compile and
> run the examples under gpu sdk of Freescale with X11, but when I try to use
> them together I have some problems.
>
>
>
> Actually, what I am trying to make has already been done by Freescale
>
> http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
Adding Andre.

Regards,

Fabio Estevam




[-- Attachment #2: Type: text/html, Size: 18509 bytes --]

Re: Using OpenCV and OpenGL together

[Andre Silva]

[-- Attachment #1: Type: text/plain, Size: 5306 bytes --]

Hi Friket,

Take a look in the .h in my project, there is this constant TEXTURE_W and TEXTURE_H that is used along the code, make sure that the values match the resolution you are trying to use.

Regards,
Andre

From: Fikret Alim [mailto:fikret.alim@gmail.com]
Sent: Thursday, December 19, 2013 12:48 PM
To: Silva Andre-B22958
Cc: Fabio Estevam; meta-freescale@yoctoproject.org
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Dear Andre,

I am using OpenCV 2.4 and I am sure that OpenCV correctly sets the resolution, because if I comment out everything related to OpenGL and add and imshow command after capturing the image with OpenCV, there is no proble, I can see the video resolution as expected. So, I think that the problem is related to OpenGL part of the code.

Thanks,
Fikret

On Thu, Dec 19, 2013 at 4:40 PM, Andre Silva <Andre.Silva@freescale.com<mailto:Andre.Silva@freescale.com>> wrote:
Hi Friket,

Which OpenCV version are you using ? I remember 1.1 had an issue with setting the camera resolution using these functions, I usually needed to set it directly in CV´s code and then recompiling all the lib.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>]
Sent: Thursday, December 19, 2013 11:56 AM
To: Silva Andre-B22958
Cc: Fabio Estevam; meta-freescale@yoctoproject.org<mailto:meta-freescale@yoctoproject.org>

Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Hi Andre,

There are only two places that I change for the resolution inside the code. This first one is capture resolution that openCV uses, the other one is glTexImage2D function. I only modify them for the resolution and I don't see any other functions that can be related to resolution.

cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, bgr_img.data);

The strange thing is; if I set 160x120, 320x240 and even 320x320 resolutions, I don't see any problem. But if I set higher resolutions such as 640x480, 800x600 or 1024x768, etc., I got segmentation fault. Do you have a recommendation for me where and what to check inside the code?

Thanks,
Fikret

On Thu, Dec 19, 2013 at 1:55 PM, Andre Silva <Andre.Silva@freescale.com<mailto:Andre.Silva@freescale.com>> wrote:
Hi Fikret,

I believe that there are another functions in the code using this resolution, if you change the resolution without modifying these other parts the segfault will occur. The only limitation using 320x240 is the performance, since OpenCV runs on cpu, if we use higher resolution the performance will decrease a lot.

Regards,
andre

From: Fikret Alim [mailto:fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>]
Sent: Thursday, December 19, 2013 5:40 AM
To: Fabio Estevam
Cc: meta-freescale@yoctoproject.org<mailto:meta-freescale@yoctoproject.org>; Silva Andre-B22958
Subject: Re: [meta-freescale] Using OpenCV and OpenGL together

Dear Marek,

Thank you for your answer,

I will check the file you sent. I have solved the problem in a different way, I am not sure if this is the correct way or not, but at least it worked after this change:

I have only moved the following two lines from PlaneCreateTex function to PlaneSetTex function.

 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

Then, PlaneSetTex function will be;

void PlaneSetTex (Mat texture_data)
{
 cvtColor (texture_data, _texture_data, CV_BGR2RGB);
 //imshow("_texture_data",_texture_data);
 glBindTexture(GL_TEXTURE_2D, _texture);
 glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, 320, 240, 0, GL_RGB, GL_UNSIGNED_BYTE, _texture_data.data);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
 glTexParameterf (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
}

With this change, now I can run Andre's gesture recognition example, and also I have tried some GLSL shaders for image processing like sobel detection.

But, let me ask a different question: Is there a limition for the webcam resolution? I can only use 320x240 resolution for both Andre's example and gpu sdk examples. If I set a higher resolution, then I get segmentation fault error. I am sure that my webcam supports higher resolutions, since I can get video with openCV at higher resolutions. What can be wrong? Is there a limitation, or do I need to perform some other operations for higher resolutions?

Thanks&Best Regards,
Fikret

On Wed, Dec 18, 2013 at 8:18 PM, Fabio Estevam <festevam@gmail.com<mailto:festevam@gmail.com>> wrote:
On Wed, Dec 11, 2013 at 12:18 PM, Fikret Alim <fikret.alim@gmail.com<mailto:fikret.alim@gmail.com>> wrote:
> Hello,
>
>
>
> I am trying to use OpenCV and OpenGL together for some image processing
> tasks. I have already run OpenCV examples on IMX6, I can also compile and
> run the examples under gpu sdk of Freescale with X11, but when I try to use
> them together I have some problems.
>
>
>
> Actually, what I am trying to make has already been done by Freescale
>
> http://imxcv.blogspot.com/2012/03/gesture-recognition-on-imx6.html
Adding Andre.

Regards,

Fabio Estevam




[-- Attachment #2: Type: text/html, Size: 18540 bytes --]