Using atmel-isi for direct output on framebuffer ?

Using atmel-isi for direct output on framebuffer ?

[Thomas Petazzoni]

Hello Josh,

I am currently looking at V4L2 and your atmel-isi driver for an AT91
based platform on which I would like the ISI interface to capture the
image from a camera and have this image directly output in RGB format
at a specific location on the screen (so that it can be nicely
integrated into a Qt application for example).

At the moment, I grab frames from the V4L2 device to userspace, do the
YUV -> RGB conversion manually in my application, and then displays the
converted frame on the framebuffer thanks to normal Qt painting
mechanisms. This works, but obviously consumes a lot of CPU.

>From the AT91 datasheet, I understand that the ISI interface is capable
of doing the YUV -> RGB conversion and is also capable of outputting
the frame at some location in the framebuffer, but I don't see how to
use this capability with the Linux V4L2 and framebuffer infrastructures.

Is this possible ? If so, could you provide some pointers or starting
points to get me started ? If not, what is missing in the driver ?

Thanks a lot,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: Using atmel-isi for direct output on framebuffer ?

[Guennadi Liakhovetski]

Hi Thomas

On Thu, 1 Sep 2011, Thomas Petazzoni wrote:

> Hello Josh,
> 
> I am currently looking at V4L2 and your atmel-isi driver for an AT91
> based platform on which I would like the ISI interface to capture the
> image from a camera and have this image directly output in RGB format
> at a specific location on the screen (so that it can be nicely
> integrated into a Qt application for example).

Isn't this what V4L2_CAP_VIDEO_OVERLAY type drivers are doing? I've never 
dealt with those, but it seems to be exactly what you need. ATM there are 
no soc-camera drivers, implementing this capability, so, looks like 
implementing this in atmel-isi won't be a very boring task for you;-)

Thanks
Guennadi

> 
> At the moment, I grab frames from the V4L2 device to userspace, do the
> YUV -> RGB conversion manually in my application, and then displays the
> converted frame on the framebuffer thanks to normal Qt painting
> mechanisms. This works, but obviously consumes a lot of CPU.
> 
> >From the AT91 datasheet, I understand that the ISI interface is capable
> of doing the YUV -> RGB conversion and is also capable of outputting
> the frame at some location in the framebuffer, but I don't see how to
> use this capability with the Linux V4L2 and framebuffer infrastructures.
> 
> Is this possible ? If so, could you provide some pointers or starting
> points to get me started ? If not, what is missing in the driver ?
> 
> Thanks a lot,
> 
> Thomas
> -- 
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

RE: Using atmel-isi for direct output on framebuffer ?

[Wu, Josh]

Hi, Thomas

On Thu, 1 Sep 2011, Thomas Petazzoni wrote:

> Hello Josh,

> I am currently looking at V4L2 and your atmel-isi driver for an AT91
> based platform on which I would like the ISI interface to capture the
> image from a camera and have this image directly output in RGB format
> at a specific location on the screen (so that it can be nicely
> integrated into a Qt application for example).

> At the moment, I grab frames from the V4L2 device to userspace, do the
> YUV -> RGB conversion manually in my application, and then displays the
> converted frame on the framebuffer thanks to normal Qt painting
> mechanisms. This works, but obviously consumes a lot of CPU.

> From the AT91 datasheet, I understand that the ISI interface is capable
> of doing the YUV -> RGB conversion and is also capable of outputting
> the frame at some location in the framebuffer, but I don't see how to
> use this capability with the Linux V4L2 and framebuffer infrastructures.

> Is this possible ? If so, could you provide some pointers or starting
> points to get me started ? If not, what is missing in the driver ?

My understanding is that you want to use Atmel ISI to output RGB data then work with framebuffer. So yes, it is possible.

Since current atmel_isi.c only uses its codec path to output YUV data. So first need add RGB format support in isi_camera_get_formats().
Then you have two choices to enable RGB output of ISI:
  1. Enable isi's preview path(DMA, interrupts) to convert YUV to RGB.
  2. Or still use codec path but don't need add much ISI code, just set camera sensor(if it support RGB565 output) to output RGB565 data for ISI, then what the data ISI output now should be RGB565 format. But in this way you cannot do any scale.

For V4L2_CAP_VIDEO_OVERLAY type driver, I don't know much about that.

Best Regards,
Josh Wu

> Thanks a lot,

> Thomas
> -- 
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Using atmel-isi for direct output on framebuffer ?

[Thomas Petazzoni]

Hello Josh,

Le Fri, 2 Sep 2011 17:08:32 +0800,
"Wu, Josh" <Josh.wu@atmel.com> a écrit :

> My understanding is that you want to use Atmel ISI to output RGB data
> then work with framebuffer. So yes, it is possible.

Good.

> Since current atmel_isi.c only uses its codec path to output YUV
> data. So first need add RGB format support in
> isi_camera_get_formats(). Then you have two choices to enable RGB
> output of ISI: 1. Enable isi's preview path(DMA, interrupts) to
> convert YUV to RGB. 2. Or still use codec path but don't need add
> much ISI code, just set camera sensor(if it support RGB565 output) to
> output RGB565 data for ISI, then what the data ISI output now should
> be RGB565 format. But in this way you cannot do any scale.

Doing the YUV -> RGB within the V4L2 driver is something I understand
quite well. The part I miss is how the V4L2 driver interacts with the
framebuffer driver to output the camera image into the framebuffer.

> For V4L2_CAP_VIDEO_OVERLAY type driver, I don't know much about that.

Hum, ok, found http://v4l2spec.bytesex.org/spec/x6570.htm which seems
to explain a bit the userspace interface for this.

Thanks for your feedback!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: Using atmel-isi for direct output on framebuffer ?

[Sylwester Nawrocki]

Hi,

On 09/02/2011 11:18 AM, Thomas Petazzoni wrote:
> Doing the YUV -> RGB within the V4L2 driver is something I understand
> quite well. The part I miss is how the V4L2 driver interacts with the
> framebuffer driver to output the camera image into the framebuffer.
> 
>> For V4L2_CAP_VIDEO_OVERLAY type driver, I don't know much about that.
> 
> Hum, ok, found http://v4l2spec.bytesex.org/spec/x6570.htm which seems
> to explain a bit the userspace interface for this.

Most up to date documentation is hosted at linuxtv.org:

http://linuxtv.org/downloads/v4l-dvb-apis/overlay.html

Re: Using atmel-isi for direct output on framebuffer ?

[Laurent Pinchart]

Hi Thomas,

On Friday 02 September 2011 11:18:53 Thomas Petazzoni wrote:
> Le Fri, 2 Sep 2011 17:08:32 +0800, "Wu, Josh" a écrit :
> > My understanding is that you want to use Atmel ISI to output RGB data
> > then work with framebuffer. So yes, it is possible.
> 
> Good.
> 
> > Since current atmel_isi.c only uses its codec path to output YUV
> > data. So first need add RGB format support in
> > isi_camera_get_formats(). Then you have two choices to enable RGB
> > output of ISI: 1. Enable isi's preview path(DMA, interrupts) to
> > convert YUV to RGB. 2. Or still use codec path but don't need add
> > much ISI code, just set camera sensor(if it support RGB565 output) to
> > output RGB565 data for ISI, then what the data ISI output now should
> > be RGB565 format. But in this way you cannot do any scale.
> 
> Doing the YUV -> RGB within the V4L2 driver is something I understand
> quite well. The part I miss is how the V4L2 driver interacts with the
> framebuffer driver to output the camera image into the framebuffer.
> 
> > For V4L2_CAP_VIDEO_OVERLAY type driver, I don't know much about that.
> 
> Hum, ok, found http://v4l2spec.bytesex.org/spec/x6570.htm which seems
> to explain a bit the userspace interface for this.

I'm not sure if V4L2_CAP_VIDEO_OVERLAY is a good solution for this. This 
driver type (or rather buffer type) was used on old systems to capture 
directly to the PCI graphics card memory. Nowadays I would advice using 
USERPTR with framebuffer memory.

-- 
Regards,

Laurent Pinchart

Re: Using atmel-isi for direct output on framebuffer ?

[Thomas Petazzoni]

Hi Laurent,

Le Fri, 2 Sep 2011 13:42:03 +0200,
Laurent Pinchart <laurent.pinchart@ideasonboard.com> a écrit :

> I'm not sure if V4L2_CAP_VIDEO_OVERLAY is a good solution for this.
> This driver type (or rather buffer type) was used on old systems to
> capture directly to the PCI graphics card memory. Nowadays I would
> advice using USERPTR with framebuffer memory.

Could you give a short summary of how the USERPTR mechanism works?

Thanks,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: Using atmel-isi for direct output on framebuffer ?

[Laurent Pinchart]

Hi Thomas,

On Friday 02 September 2011 13:51:58 Thomas Petazzoni wrote:
> Le Fri, 2 Sep 2011 13:42:03 +0200, Laurent Pinchart a écrit :
> > I'm not sure if V4L2_CAP_VIDEO_OVERLAY is a good solution for this.
> > This driver type (or rather buffer type) was used on old systems to
> > capture directly to the PCI graphics card memory. Nowadays I would
> > advice using USERPTR with framebuffer memory.
> 
> Could you give a short summary of how the USERPTR mechanism works?

In a nutshell, instead of asking the capture driver to allocate buffers and 
map them to userspace, applications pass userspace buffer pointers to the 
driver.

In your case the application would mmap() the framebuffer memory and give 
pointers to that memory to the capture driver.

-- 
Regards,

Laurent Pinchart