V4L2_PIX_FMT_RAW

V4L2_PIX_FMT_RAW

[Thomas Kaiser]

Why is V4L2_PIX_FMT_RAW not included as a pixel format in V4l2?

I would like to just forward the stream from my webcam "as it is" to user space .

V4L2_PIX_FMT_RAW looks as it is the right thing I need.

Any comments are warmly welcome.

Thomas


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Wed, Feb 20, 2008 at 9:42 PM, Thomas Kaiser
> <linux-dvb@kaiser-linux.li> wrote:
>> H. Willstrand wrote:
>>  > On Wed, Feb 20, 2008 at 9:22 PM, Thomas Kaiser
>>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >> H. Willstrand wrote:
>>  >>  > On Wed, Feb 20, 2008 at 8:25 PM, Thomas Kaiser
>>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >>  >> Why is V4L2_PIX_FMT_RAW not included as a pixel format in V4l2?
>>  >>  >>
>>  >>  >>  I would like to just forward the stream from my webcam "as it is" to user space .
>>  >>  >>
>>  >>  >>  V4L2_PIX_FMT_RAW looks as it is the right thing I need.
>>  >>  >>
>>  >>  >
>>  >>  > V4L2 drivers should not perform any video transformations, the driver
>>  >>  > provides user space with hardware supported formats.
>>  >>
>>  >>  Ok, so, I need a entry for Pixart chips.
>>  >>  PAC207: a line based coding algo.
>>  >>  PAC7311: the interpretation of JPEG from Pixart
>>  >>  PAC7302: the _new_  interpretation of JPEG from Pixart
>>  >>
>>  >>  V4L2_PIX_FMT_RAW would make my live easier.
>>  >>
>>  >>  The option to forward the stream "as it is" would be really nice. When the
>>  >>  manufacture of some video streaming devices (like webcams) do their own thing,
>>  >>  you can forward the raw stream and the user application can take care of the
>>  >>  decoding of the stream.
>>  >>
>>  >>  For me it looks not like such a bad idea!!!???
>>  >>
>>  >
>>  > Yes, it might be a good idea to add something like V4L2_PIX_FMT_PAC207, etc.
>>  > This is anyhow the case with the PWC family.
>>
>>  I think the better is to just forward the stream to userspace.
>>  Then we have to make a lib which can be called from all these video application
>>  around to decode the stream.
>>
>>  Somebody talked already about this on the list.
>>
>>  When the cam is able to send a stream in a good known format it is no problem to
>>  handle this with the right V4L2_PIX_FMT_..., but if not, we need a "official"
>>  way to get this "devil" stream to user space!
>>
> 
> Well, it can go ugly if one piece of hardware supports several "raw"
> formats, they need to be distinct. And in the end of the day the V4L2
> drivers might consist of several identical "raw" formats which then
> aren't consolidated.
> 
> Harri

I don't really understand what you try to say here.
I want to have the stream forwarded "as it is" to user space. Then the v4l2 
driver even have not to know what kind of stream this is!
We just forward the stream (again) "as it is" to user space.
Now, the user space application (viewer app, skype or what ever) has to decide 
if they can handle the stream.

Thomas

PS: Why do you answer OFF-LIST? I think it is a nice topic to discussed with 
everyone on the list.


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Wed, Feb 20, 2008 at 10:11 PM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
>
> H. Willstrand wrote:
>  > On Wed, Feb 20, 2008 at 9:42 PM, Thomas Kaiser
>  > <linux-dvb@kaiser-linux.li> wrote:
>  >> H. Willstrand wrote:
>  >>  > On Wed, Feb 20, 2008 at 9:22 PM, Thomas Kaiser
>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>  >>  >> H. Willstrand wrote:
>  >>  >>  > On Wed, Feb 20, 2008 at 8:25 PM, Thomas Kaiser
>  >>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>  >>  >>  >> Why is V4L2_PIX_FMT_RAW not included as a pixel format in V4l2?
>  >>  >>  >>
>  >>  >>  >>  I would like to just forward the stream from my webcam "as it is" to user space .
>  >>  >>  >>
>  >>  >>  >>  V4L2_PIX_FMT_RAW looks as it is the right thing I need.
>  >>  >>  >>
>  >>  >>  >
>  >>  >>  > V4L2 drivers should not perform any video transformations, the driver
>  >>  >>  > provides user space with hardware supported formats.
>  >>  >>
>  >>  >>  Ok, so, I need a entry for Pixart chips.
>  >>  >>  PAC207: a line based coding algo.
>  >>  >>  PAC7311: the interpretation of JPEG from Pixart
>  >>  >>  PAC7302: the _new_  interpretation of JPEG from Pixart
>  >>  >>
>  >>  >>  V4L2_PIX_FMT_RAW would make my live easier.
>  >>  >>
>  >>  >>  The option to forward the stream "as it is" would be really nice. When the
>  >>  >>  manufacture of some video streaming devices (like webcams) do their own thing,
>  >>  >>  you can forward the raw stream and the user application can take care of the
>  >>  >>  decoding of the stream.
>  >>  >>
>  >>  >>  For me it looks not like such a bad idea!!!???
>  >>  >>
>  >>  >
>  >>  > Yes, it might be a good idea to add something like V4L2_PIX_FMT_PAC207, etc.
>  >>  > This is anyhow the case with the PWC family.
>  >>
>  >>  I think the better is to just forward the stream to userspace.
>  >>  Then we have to make a lib which can be called from all these video application
>  >>  around to decode the stream.
>  >>
>  >>  Somebody talked already about this on the list.
>  >>
>  >>  When the cam is able to send a stream in a good known format it is no problem to
>  >>  handle this with the right V4L2_PIX_FMT_..., but if not, we need a "official"
>  >>  way to get this "devil" stream to user space!
>  >>
>  >
>  > Well, it can go ugly if one piece of hardware supports several "raw"
>  > formats, they need to be distinct. And in the end of the day the V4L2
>  > drivers might consist of several identical "raw" formats which then
>  > aren't consolidated.
>  >
>  > Harri
>
>  I don't really understand what you try to say here.
>  I want to have the stream forwarded "as it is" to user space. Then the v4l2
>  driver even have not to know what kind of stream this is!
>  We just forward the stream (again) "as it is" to user space.
>  Now, the user space application (viewer app, skype or what ever) has to decide
>  if they can handle the stream.
>
Ok, if a driver exposes a RAW format, how can a application know if it
can handle the format? By looking at the driver name? What if the
driver have several RAW formats? (they might have)

Next, standardization is a good thing, and hopefully chip set vendors
tries to keep down the number of formats, so it's a good idea to try
define distinct V4L2_PIX_FMT for each distinct format.

Cheers,
Harri


>  PS: Why do you answer OFF-LIST? I think it is a nice topic to discussed with
>  everyone on the list.
I thought I did, sorry for that.

//Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Daniel Glöckner]

On Wed, Feb 20, 2008 at 10:11:36PM +0100, Thomas Kaiser wrote:
> H. Willstrand wrote:
> >Well, it can go ugly if one piece of hardware supports several "raw"
> >formats, they need to be distinct. And in the end of the day the V4L2
> >drivers might consist of several identical "raw" formats which then
> >aren't consolidated.
> 
> I don't really understand what you try to say here.

Think about an analog TV card.
In the future there might be one where RAW could mean either sampled
CVBS or sampled Y/C. The card may be able to provide the Y/C in planar
and packed format. It may be capable of 16 bit at 13.5Mhz and 8 bit at
27Mhz, ...

If we start defining raw formats, there needs to be a way to choose
between all those variants without defining lots of additional pixel
formats.

Maybe an ioctl VIDIOC_S_RAW where one passes a number to select the
variant. An application would then have to check the driver and version
field returned by VIDIOC_QUERYCAP to determine the number to pass. This
way drivers may freely assign numbers to their raw formats.

Application writers would need to look into all drivers' docs/sources to
find the possible values. They would need to do it anyway to see if they
can decode the raw format.

  Daniel

P.S.: If my mail doesn't reach the list, blame its spam filter

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Wed, Feb 20, 2008 at 10:11 PM, Thomas Kaiser
> <linux-dvb@kaiser-linux.li> wrote:
>> H. Willstrand wrote:
>>  > On Wed, Feb 20, 2008 at 9:42 PM, Thomas Kaiser
>>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >> H. Willstrand wrote:
>>  >>  > On Wed, Feb 20, 2008 at 9:22 PM, Thomas Kaiser
>>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >>  >> H. Willstrand wrote:
>>  >>  >>  > On Wed, Feb 20, 2008 at 8:25 PM, Thomas Kaiser
>>  >>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >>  >>  >> Why is V4L2_PIX_FMT_RAW not included as a pixel format in V4l2?
>>  >>  >>  >>
>>  >>  >>  >>  I would like to just forward the stream from my webcam "as it is" to user space .
>>  >>  >>  >>
>>  >>  >>  >>  V4L2_PIX_FMT_RAW looks as it is the right thing I need.
>>  >>  >>  >>
>>  >>  >>  >
>>  >>  >>  > V4L2 drivers should not perform any video transformations, the driver
>>  >>  >>  > provides user space with hardware supported formats.
>>  >>  >>
>>  >>  >>  Ok, so, I need a entry for Pixart chips.
>>  >>  >>  PAC207: a line based coding algo.
>>  >>  >>  PAC7311: the interpretation of JPEG from Pixart
>>  >>  >>  PAC7302: the _new_  interpretation of JPEG from Pixart
>>  >>  >>
>>  >>  >>  V4L2_PIX_FMT_RAW would make my live easier.
>>  >>  >>
>>  >>  >>  The option to forward the stream "as it is" would be really nice. When the
>>  >>  >>  manufacture of some video streaming devices (like webcams) do their own thing,
>>  >>  >>  you can forward the raw stream and the user application can take care of the
>>  >>  >>  decoding of the stream.
>>  >>  >>
>>  >>  >>  For me it looks not like such a bad idea!!!???
>>  >>  >>
>>  >>  >
>>  >>  > Yes, it might be a good idea to add something like V4L2_PIX_FMT_PAC207, etc.
>>  >>  > This is anyhow the case with the PWC family.
>>  >>
>>  >>  I think the better is to just forward the stream to userspace.
>>  >>  Then we have to make a lib which can be called from all these video application
>>  >>  around to decode the stream.
>>  >>
>>  >>  Somebody talked already about this on the list.
>>  >>
>>  >>  When the cam is able to send a stream in a good known format it is no problem to
>>  >>  handle this with the right V4L2_PIX_FMT_..., but if not, we need a "official"
>>  >>  way to get this "devil" stream to user space!
>>  >>
>>  >
>>  > Well, it can go ugly if one piece of hardware supports several "raw"
>>  > formats, they need to be distinct. And in the end of the day the V4L2
>>  > drivers might consist of several identical "raw" formats which then
>>  > aren't consolidated.
>>  >
>>  > Harri
>>
>>  I don't really understand what you try to say here.
>>  I want to have the stream forwarded "as it is" to user space. Then the v4l2
>>  driver even have not to know what kind of stream this is!
>>  We just forward the stream (again) "as it is" to user space.
>>  Now, the user space application (viewer app, skype or what ever) has to decide
>>  if they can handle the stream.
>>
> Ok, if a driver exposes a RAW format, how can a application know if it
> can handle the format? By looking at the driver name? What if the
> driver have several RAW formats? (they might have)

How does the driver know how to handle the raw stream?

OK, I get USB ID and can guess it should be this format but it is not always 
like this.

Anyway, All information the driver uses to get know the stream can be forwarded 
to user space. We just need the interface for this.
Usually, the SOF gives you the information you need. And this will be included 
when you forward the stream "as it is".

> 
> Next, standardization is a good thing, and hopefully chip set vendors
> tries to keep down the number of formats, so it's a good idea to try
> define distinct V4L2_PIX_FMT for each distinct format.

I don't think that chip set manufacture will look at a "Standard", They just try 
to optimize there chip and lower the cost of manufacturing.

They are not interested in standards. Standards means everybody can copy it!

Thomas

> 
> Cheers,
> Harri
> 
> 
>>  PS: Why do you answer OFF-LIST? I think it is a nice topic to discussed with
>>  everyone on the list.
> I thought I did, sorry for that.
> 
> //Harri


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

Daniel Glöckner wrote:
> On Wed, Feb 20, 2008 at 10:11:36PM +0100, Thomas Kaiser wrote:
>> H. Willstrand wrote:
>>> Well, it can go ugly if one piece of hardware supports several "raw"
>>> formats, they need to be distinct. And in the end of the day the V4L2
>>> drivers might consist of several identical "raw" formats which then
>>> aren't consolidated.
>> I don't really understand what you try to say here.
> 
> Think about an analog TV card.
> In the future there might be one where RAW could mean either sampled
> CVBS or sampled Y/C. The card may be able to provide the Y/C in planar
> and packed format. It may be capable of 16 bit at 13.5Mhz and 8 bit at
> 27Mhz, ...
> 
> If we start defining raw formats, there needs to be a way to choose
> between all those variants without defining lots of additional pixel
> formats.
> 
> Maybe an ioctl VIDIOC_S_RAW where one passes a number to select the
> variant. An application would then have to check the driver and version
> field returned by VIDIOC_QUERYCAP to determine the number to pass. This
> way drivers may freely assign numbers to their raw formats.

Yeh, That's something I mean.

> 
> Application writers would need to look into all drivers' docs/sources to
> find the possible values. They would need to do it anyway to see if they
> can decode the raw format.

That's why we need a user space library to handle all this strange "unknown" 
streams.

When the application can not decode (or does not know) the stream, just get it 
to the decoding lib. When the stream is known, you get a decoded picture back. 
If not you get an error.

Thomas

> 
>   Daniel
> 
> P.S.: If my mail doesn't reach the list, blame its spam filter
> 


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Wed, Feb 20, 2008 at 11:12 PM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
> Daniel Glöckner wrote:
>  > On Wed, Feb 20, 2008 at 10:11:36PM +0100, Thomas Kaiser wrote:
>  >> H. Willstrand wrote:
>  >>> Well, it can go ugly if one piece of hardware supports several "raw"
>  >>> formats, they need to be distinct. And in the end of the day the V4L2
>  >>> drivers might consist of several identical "raw" formats which then
>  >>> aren't consolidated.
>  >> I don't really understand what you try to say here.
>  >
>  > Think about an analog TV card.
>  > In the future there might be one where RAW could mean either sampled
>  > CVBS or sampled Y/C. The card may be able to provide the Y/C in planar
>  > and packed format. It may be capable of 16 bit at 13.5Mhz and 8 bit at
>  > 27Mhz, ...
>  >
>  > If we start defining raw formats, there needs to be a way to choose
>  > between all those variants without defining lots of additional pixel
>  > formats.
>  >
>  > Maybe an ioctl VIDIOC_S_RAW where one passes a number to select the
>  > variant. An application would then have to check the driver and version
>  > field returned by VIDIOC_QUERYCAP to determine the number to pass. This
>  > way drivers may freely assign numbers to their raw formats.
>
>  Yeh, That's something I mean.
>

Okay, suppose we have pixel formats and raw formats. Comparing with
digital cameras raw usually means non processed image in a proprietary
format. What do we mean here?

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Wed, Feb 20, 2008 at 11:12 PM, Thomas Kaiser
> <linux-dvb@kaiser-linux.li> wrote:
>> Daniel Glöckner wrote:
>>  > On Wed, Feb 20, 2008 at 10:11:36PM +0100, Thomas Kaiser wrote:
>>  >> H. Willstrand wrote:
>>  >>> Well, it can go ugly if one piece of hardware supports several "raw"
>>  >>> formats, they need to be distinct. And in the end of the day the V4L2
>>  >>> drivers might consist of several identical "raw" formats which then
>>  >>> aren't consolidated.
>>  >> I don't really understand what you try to say here.
>>  >
>>  > Think about an analog TV card.
>>  > In the future there might be one where RAW could mean either sampled
>>  > CVBS or sampled Y/C. The card may be able to provide the Y/C in planar
>>  > and packed format. It may be capable of 16 bit at 13.5Mhz and 8 bit at
>>  > 27Mhz, ...
>>  >
>>  > If we start defining raw formats, there needs to be a way to choose
>>  > between all those variants without defining lots of additional pixel
>>  > formats.
>>  >
>>  > Maybe an ioctl VIDIOC_S_RAW where one passes a number to select the
>>  > variant. An application would then have to check the driver and version
>>  > field returned by VIDIOC_QUERYCAP to determine the number to pass. This
>>  > way drivers may freely assign numbers to their raw formats.
>>
>>  Yeh, That's something I mean.
>>
> 
> Okay, suppose we have pixel formats and raw formats. Comparing with
> digital cameras raw usually means non processed image in a proprietary
> format. What do we mean here?

I talk about webcams. But It looks like you don't get the point.
A ISOC stream can be received, then we forward this to user space! That's it.

This has nothing to do with pixel format, just deliver  the stream from the cam 
to user space, That's all what I won't.
I think raw means raw, "not manipulated"! Oder in Deutsch Roh equals raw, which 
means "not touched".

You get the point?

I just would like to have a way to get the stream "as it is" to user spcae.

Thomas


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 12:20 AM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
>
> H. Willstrand wrote:
>  > On Wed, Feb 20, 2008 at 11:12 PM, Thomas Kaiser
>  > <linux-dvb@kaiser-linux.li> wrote:
>  >> Daniel Glöckner wrote:
>  >>  > On Wed, Feb 20, 2008 at 10:11:36PM +0100, Thomas Kaiser wrote:
>  >>  >> H. Willstrand wrote:
>  >>  >>> Well, it can go ugly if one piece of hardware supports several "raw"
>  >>  >>> formats, they need to be distinct. And in the end of the day the V4L2
>  >>  >>> drivers might consist of several identical "raw" formats which then
>  >>  >>> aren't consolidated.
>  >>  >> I don't really understand what you try to say here.
>  >>  >
>  >>  > Think about an analog TV card.
>  >>  > In the future there might be one where RAW could mean either sampled
>  >>  > CVBS or sampled Y/C. The card may be able to provide the Y/C in planar
>  >>  > and packed format. It may be capable of 16 bit at 13.5Mhz and 8 bit at
>  >>  > 27Mhz, ...
>  >>  >
>  >>  > If we start defining raw formats, there needs to be a way to choose
>  >>  > between all those variants without defining lots of additional pixel
>  >>  > formats.
>  >>  >
>  >>  > Maybe an ioctl VIDIOC_S_RAW where one passes a number to select the
>  >>  > variant. An application would then have to check the driver and version
>  >>  > field returned by VIDIOC_QUERYCAP to determine the number to pass. This
>  >>  > way drivers may freely assign numbers to their raw formats.
>  >>
>  >>  Yeh, That's something I mean.
>  >>
>  >
>  > Okay, suppose we have pixel formats and raw formats. Comparing with
>  > digital cameras raw usually means non processed image in a proprietary
>  > format. What do we mean here?
>
>  I talk about webcams. But It looks like you don't get the point.
>  A ISOC stream can be received, then we forward this to user space! That's it.
>
>  This has nothing to do with pixel format, just deliver  the stream from the cam
>  to user space, That's all what I won't.
>  I think raw means raw, "not manipulated"! Oder in Deutsch Roh equals raw, which
>  means "not touched".
>
>  You get the point?
>

Yes, I understand what you want to achieve and that's fine.

Still, the hardware produces an image, the image has a format (either
a well know or proprietary), the image might or might not been
processed by the Webcam, etc.

What's the problem with having a name of the formalized data in the
video stream? ie raw do not mean undefined.
I don't see how separate RAW ioctl's will add value to the V4l2 API,
it fits into the current API.

Cheers,
Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Daniel Glöckner]

On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
> What's the problem with having a name of the formalized data in the
> video stream? ie raw do not mean undefined.

I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
exploding number of proprietary formats that are quite similar but still
incompatible. It makes sense for formats that are used by more than one
driver.

> I don't see how separate RAW ioctl's will add value to the V4l2 API,
> it fits into the current API.

Yes, it does. Each driver having multiple raw formats just needs a
private control id to select one.

  Daniel

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>  > What's the problem with having a name of the formalized data in the
>  > video stream? ie raw do not mean undefined.
>
>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>  exploding number of proprietary formats that are quite similar but still
>  incompatible. It makes sense for formats that are used by more than one
>  driver.

Correct, the number of unique pixel formats should be kept down.
Again, comparing with digital cameras there are >200 proprietary
formats and there is a "clean-up" on-going where the "market" is
aiming for a OpenRAW.

However, by declaring a generic RAW format (which is then driver
specific) doesn't help the user mode app developers. Calling a
multitude of libraries to see if you get lucky might not be a good
idea.

Still, I'm suspectious about the definition "raw" used here.
RAW should mean unprocessed image data:
* no white balance adjustment
* no color saturation adjustments
* no contrast adjustments
* no sharpness improvements
* no compression with loss

So, by looking for similarities in the "raw" formats where available
there should be a potential to consolidate them.

>
>
>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>  > it fits into the current API.
>
>  Yes, it does. Each driver having multiple raw formats just needs a
>  private control id to select one.
>
I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
should do the job.
I.e. I think there should be strong reasons to break V4L2 API behavior.

Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>>  > What's the problem with having a name of the formalized data in the
>>  > video stream? ie raw do not mean undefined.
>>
>>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>>  exploding number of proprietary formats that are quite similar but still
>>  incompatible. It makes sense for formats that are used by more than one
>>  driver.
> 
> Correct, the number of unique pixel formats should be kept down.
> Again, comparing with digital cameras there are >200 proprietary
> formats and there is a "clean-up" on-going where the "market" is
> aiming for a OpenRAW.
> 
> However, by declaring a generic RAW format (which is then driver
> specific) doesn't help the user mode app developers. Calling a
> multitude of libraries to see if you get lucky might not be a good
> idea.
> 
> Still, I'm suspectious about the definition "raw" used here.
> RAW should mean unprocessed image data:
> * no white balance adjustment
> * no color saturation adjustments
> * no contrast adjustments
> * no sharpness improvements
> * no compression with loss

Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are 
done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?

> 
> So, by looking for similarities in the "raw" formats where available
> there should be a potential to consolidate them.
> 
>>
>>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>>  > it fits into the current API.
>>
>>  Yes, it does. Each driver having multiple raw formats just needs a
>>  private control id to select one.
>>
> I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
> should do the job.
> I.e. I think there should be strong reasons to break V4L2 API behavior.
> 
> Harri


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Daniel Glöckner]

On Thu, Feb 21, 2008 at 01:00:08PM +0100, Thomas Kaiser wrote:
> H. Willstrand wrote:
> >Still, I'm suspectious about the definition "raw" used here.
> >RAW should mean unprocessed image data:
> >* no white balance adjustment
> >* no color saturation adjustments
> >* no contrast adjustments
> >* no sharpness improvements
> >* no compression with loss
> 
> Yes, raw means "as it is" no stripping, decoding  or removing of SOF 
> headers are done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) 
> is the better name?

IMHO "raw" is the output of the ADC(s).

Uncompressed.

For webcams probably something like V4L2_PIX_FMT_SBGGR16.

Lossless transformations allowed.

  Daniel

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
> H. Willstrand wrote:
>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>  >>  > What's the problem with having a name of the formalized data in the
>  >>  > video stream? ie raw do not mean undefined.
>  >>
>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>  >>  exploding number of proprietary formats that are quite similar but still
>  >>  incompatible. It makes sense for formats that are used by more than one
>  >>  driver.
>  >
>  > Correct, the number of unique pixel formats should be kept down.
>  > Again, comparing with digital cameras there are >200 proprietary
>  > formats and there is a "clean-up" on-going where the "market" is
>  > aiming for a OpenRAW.
>  >
>  > However, by declaring a generic RAW format (which is then driver
>  > specific) doesn't help the user mode app developers. Calling a
>  > multitude of libraries to see if you get lucky might not be a good
>  > idea.
>  >
>  > Still, I'm suspectious about the definition "raw" used here.
>  > RAW should mean unprocessed image data:
>  > * no white balance adjustment
>  > * no color saturation adjustments
>  > * no contrast adjustments
>  > * no sharpness improvements
>  > * no compression with loss
>
>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>

I struggle with the probability to find several CCD's having similar
formats. There aren't so many manifactors of CCD's but they truelly
can generate divergeting formats. Worst case scenario means >200
V4L2_PIX_FMT_RAW_...

I think RAW is a OK name, the question is if the subcomponents of the
RAW formats has similarities, if so they might be standardized.
Looking into different Sony CCD's it's clearly possible, but after the
CCD the data has to be buffered, packaged and transmitted which of
course can be done in several ways...

Cheers,
Harri

>
>  >
>  > So, by looking for similarities in the "raw" formats where available
>  > there should be a potential to consolidate them.
>  >
>  >>
>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>  >>  > it fits into the current API.
>  >>
>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>  >>  private control id to select one.
>  >>
>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>  > should do the job.
>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>  >
>  > Harri
>
>
>
>
> --
>  http://www.kaiser-linux.li
>

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 1:43 PM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
> On Thu, Feb 21, 2008 at 01:00:08PM +0100, Thomas Kaiser wrote:
>  > H. Willstrand wrote:
>
> > >Still, I'm suspectious about the definition "raw" used here.
>  > >RAW should mean unprocessed image data:
>  > >* no white balance adjustment
>  > >* no color saturation adjustments
>  > >* no contrast adjustments
>  > >* no sharpness improvements
>  > >* no compression with loss
>  >
>  > Yes, raw means "as it is" no stripping, decoding  or removing of SOF
>  > headers are done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is)
>  > is the better name?
>
>  IMHO "raw" is the output of the ADC(s).
>
>  Uncompressed.
>
>  For webcams probably something like V4L2_PIX_FMT_SBGGR16.
>
>  Lossless transformations allowed.

Almost agree, meaning lossless transformation can be a compression :)

Cheers,
Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
> <linux-dvb@kaiser-linux.li> wrote:
>> H. Willstrand wrote:
>>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>>  >>  > What's the problem with having a name of the formalized data in the
>>  >>  > video stream? ie raw do not mean undefined.
>>  >>
>>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>>  >>  exploding number of proprietary formats that are quite similar but still
>>  >>  incompatible. It makes sense for formats that are used by more than one
>>  >>  driver.
>>  >
>>  > Correct, the number of unique pixel formats should be kept down.
>>  > Again, comparing with digital cameras there are >200 proprietary
>>  > formats and there is a "clean-up" on-going where the "market" is
>>  > aiming for a OpenRAW.
>>  >
>>  > However, by declaring a generic RAW format (which is then driver
>>  > specific) doesn't help the user mode app developers. Calling a
>>  > multitude of libraries to see if you get lucky might not be a good
>>  > idea.
>>  >
>>  > Still, I'm suspectious about the definition "raw" used here.
>>  > RAW should mean unprocessed image data:
>>  > * no white balance adjustment
>>  > * no color saturation adjustments
>>  > * no contrast adjustments
>>  > * no sharpness improvements
>>  > * no compression with loss
>>
>>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>>
> 
> I struggle with the probability to find several CCD's having similar
> formats. There aren't so many manifactors of CCD's but they truelly
> can generate divergeting formats. Worst case scenario means >200
> V4L2_PIX_FMT_RAW_...
> 
> I think RAW is a OK name, the question is if the subcomponents of the
> RAW formats has similarities, if so they might be standardized.
> Looking into different Sony CCD's it's clearly possible, but after the
> CCD the data has to be buffered, packaged and transmitted which of
> course can be done in several ways...
> 
> Cheers,
> Harri
> 
>>  >
>>  > So, by looking for similarities in the "raw" formats where available
>>  > there should be a potential to consolidate them.
>>  >
>>  >>
>>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>>  >>  > it fits into the current API.
>>  >>
>>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>>  >>  private control id to select one.
>>  >>
>>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>>  > should do the job.
>>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>>  >
>>  > Harri

Actually, in a webcam you have the image sensor and a usb bridge. Usually, the 
sensor capture a picture in Bayer pattern. This gets forwarded to the usb 
bridge. The usb bridge may or may not transfer the picture to an other format 
and/or compress it with a standard compression algo or a proprietary compression 
algo. The resulting data stream will be transmitted over the usb interface.

I just would like to get this resulting stream to user space without 
manipulation/conversion/decoding of the stream in the kernel module.

That means we don't know what the format is in this data which comes trough the 
usb interface. That's way I call it raw.

At the moment with V4L2, I have to forward a stream to user space which is in a 
format v4l2 knows. That means I have sometimes to do heavy data processing in 
the kernel module to decode/convert the data from the usb stream to a known v4l2 
video format.

That's way I want a official way to forward the untouched usb stream to user space!

How the user space application has to react on this stream is an other story, I 
think. But there will be some way to tell the usespace application what to do 
with this "unknown" stream, I am sure.

Thomas


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
>
> H. Willstrand wrote:
>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>  > <linux-dvb@kaiser-linux.li> wrote:
>  >> H. Willstrand wrote:
>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>  >>  >>  > What's the problem with having a name of the formalized data in the
>  >>  >>  > video stream? ie raw do not mean undefined.
>  >>  >>
>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>  >>  >>  exploding number of proprietary formats that are quite similar but still
>  >>  >>  incompatible. It makes sense for formats that are used by more than one
>  >>  >>  driver.
>  >>  >
>  >>  > Correct, the number of unique pixel formats should be kept down.
>  >>  > Again, comparing with digital cameras there are >200 proprietary
>  >>  > formats and there is a "clean-up" on-going where the "market" is
>  >>  > aiming for a OpenRAW.
>  >>  >
>  >>  > However, by declaring a generic RAW format (which is then driver
>  >>  > specific) doesn't help the user mode app developers. Calling a
>  >>  > multitude of libraries to see if you get lucky might not be a good
>  >>  > idea.
>  >>  >
>  >>  > Still, I'm suspectious about the definition "raw" used here.
>  >>  > RAW should mean unprocessed image data:
>  >>  > * no white balance adjustment
>  >>  > * no color saturation adjustments
>  >>  > * no contrast adjustments
>  >>  > * no sharpness improvements
>  >>  > * no compression with loss
>  >>
>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>  >>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>  >>
>  >
>  > I struggle with the probability to find several CCD's having similar
>  > formats. There aren't so many manifactors of CCD's but they truelly
>  > can generate divergeting formats. Worst case scenario means >200
>  > V4L2_PIX_FMT_RAW_...
>  >
>  > I think RAW is a OK name, the question is if the subcomponents of the
>  > RAW formats has similarities, if so they might be standardized.
>  > Looking into different Sony CCD's it's clearly possible, but after the
>  > CCD the data has to be buffered, packaged and transmitted which of
>  > course can be done in several ways...
>  >
>  > Cheers,
>  > Harri
>  >
>  >>  >
>  >>  > So, by looking for similarities in the "raw" formats where available
>  >>  > there should be a potential to consolidate them.
>  >>  >
>  >>  >>
>  >>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>  >>  >>  > it fits into the current API.
>  >>  >>
>  >>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>  >>  >>  private control id to select one.
>  >>  >>
>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>  >>  > should do the job.
>  >>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>  >>  >
>  >>  > Harri
>
>  Actually, in a webcam you have the image sensor and a usb bridge. Usually, the
>  sensor capture a picture in Bayer pattern. This gets forwarded to the usb
>  bridge. The usb bridge may or may not transfer the picture to an other format
>  and/or compress it with a standard compression algo or a proprietary compression
>  algo. The resulting data stream will be transmitted over the usb interface.
>

Yes, the USB bridge buffers, packages and transmits.

>  I just would like to get this resulting stream to user space without
>  manipulation/conversion/decoding of the stream in the kernel module.
>
>  That means we don't know what the format is in this data which comes trough the
>  usb interface. That's way I call it raw.
>
>  At the moment with V4L2, I have to forward a stream to user space which is in a
>  format v4l2 knows. That means I have sometimes to do heavy data processing in
>  the kernel module to decode/convert the data from the usb stream to a known v4l2
>  video format.
>

Drivers should not do any decoding / converting, it's not allowed in
kernel mode.
But you are right, there are a number of V4L1 exceptions:
AR M64278 (arv.c) converts YUV422 to YUV422P
QuickCam (bw-qcam.c) converts RAW to a useful format :)
CPiA (cpia.c) converts 420 to different RGB formats
OmniVision (ov511.c) converts from YUV4:0:0
PWC (V4L2) does decoding
...

However, the Webcams provides only a limited set of formats and the
"raw" are usually available. New drivers with proprietary "raw"
formats should be added to videodev2.h

>  That's way I want a official way to forward the untouched usb stream to user space!
>
>  How the user space application has to react on this stream is an other story, I
>  think. But there will be some way to tell the usespace application what to do
>  with this "unknown" stream, I am sure.
>
>  Thomas
>

Cheers,
Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
> <linux-dvb@kaiser-linux.li> wrote:
>> H. Willstrand wrote:
>>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >> H. Willstrand wrote:
>>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>>  >>  >>  > What's the problem with having a name of the formalized data in the
>>  >>  >>  > video stream? ie raw do not mean undefined.
>>  >>  >>
>>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>>  >>  >>  exploding number of proprietary formats that are quite similar but still
>>  >>  >>  incompatible. It makes sense for formats that are used by more than one
>>  >>  >>  driver.
>>  >>  >
>>  >>  > Correct, the number of unique pixel formats should be kept down.
>>  >>  > Again, comparing with digital cameras there are >200 proprietary
>>  >>  > formats and there is a "clean-up" on-going where the "market" is
>>  >>  > aiming for a OpenRAW.
>>  >>  >
>>  >>  > However, by declaring a generic RAW format (which is then driver
>>  >>  > specific) doesn't help the user mode app developers. Calling a
>>  >>  > multitude of libraries to see if you get lucky might not be a good
>>  >>  > idea.
>>  >>  >
>>  >>  > Still, I'm suspectious about the definition "raw" used here.
>>  >>  > RAW should mean unprocessed image data:
>>  >>  > * no white balance adjustment
>>  >>  > * no color saturation adjustments
>>  >>  > * no contrast adjustments
>>  >>  > * no sharpness improvements
>>  >>  > * no compression with loss
>>  >>
>>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>>  >>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>>  >>
>>  >
>>  > I struggle with the probability to find several CCD's having similar
>>  > formats. There aren't so many manifactors of CCD's but they truelly
>>  > can generate divergeting formats. Worst case scenario means >200
>>  > V4L2_PIX_FMT_RAW_...
>>  >
>>  > I think RAW is a OK name, the question is if the subcomponents of the
>>  > RAW formats has similarities, if so they might be standardized.
>>  > Looking into different Sony CCD's it's clearly possible, but after the
>>  > CCD the data has to be buffered, packaged and transmitted which of
>>  > course can be done in several ways...
>>  >
>>  > Cheers,
>>  > Harri
>>  >
>>  >>  >
>>  >>  > So, by looking for similarities in the "raw" formats where available
>>  >>  > there should be a potential to consolidate them.
>>  >>  >
>>  >>  >>
>>  >>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>>  >>  >>  > it fits into the current API.
>>  >>  >>
>>  >>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>>  >>  >>  private control id to select one.
>>  >>  >>
>>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>>  >>  > should do the job.
>>  >>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>>  >>  >
>>  >>  > Harri
>>
>>  Actually, in a webcam you have the image sensor and a usb bridge. Usually, the
>>  sensor capture a picture in Bayer pattern. This gets forwarded to the usb
>>  bridge. The usb bridge may or may not transfer the picture to an other format
>>  and/or compress it with a standard compression algo or a proprietary compression
>>  algo. The resulting data stream will be transmitted over the usb interface.
>>
> 
> Yes, the USB bridge buffers, packages and transmits.
> 
>>  I just would like to get this resulting stream to user space without
>>  manipulation/conversion/decoding of the stream in the kernel module.
>>
>>  That means we don't know what the format is in this data which comes trough the
>>  usb interface. That's way I call it raw.
>>
>>  At the moment with V4L2, I have to forward a stream to user space which is in a
>>  format v4l2 knows. That means I have sometimes to do heavy data processing in
>>  the kernel module to decode/convert the data from the usb stream to a known v4l2
>>  video format.
>>
> 
> Drivers should not do any decoding / converting, it's not allowed in
> kernel mode.
> But you are right, there are a number of V4L1 exceptions:
> AR M64278 (arv.c) converts YUV422 to YUV422P
> QuickCam (bw-qcam.c) converts RAW to a useful format :)
> CPiA (cpia.c) converts 420 to different RGB formats
> OmniVision (ov511.c) converts from YUV4:0:0
> PWC (V4L2) does decoding

You forgot gspca [1](support of 260 webcams at the moment) and here we even do 
jpeg decoding in kernel space to get the proper format for v4l1!

> ...
> 
> However, the Webcams provides only a limited set of formats and the
> "raw" are usually available. New drivers with proprietary "raw"
> formats should be added to videodev2.h

That means you agree with me?

> 
>>  That's way I want a official way to forward the untouched usb stream to user space!
>>
>>  How the user space application has to react on this stream is an other story, I
>>  think. But there will be some way to tell the usespace application what to do
>>  with this "unknown" stream, I am sure.
>>
>>  Thomas
>>
> 
> Cheers,
> Harri

Thomas


[1] http://mxhaard.free.fr/download.html

-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 9:40 PM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
>
> H. Willstrand wrote:
>  > On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>  > <linux-dvb@kaiser-linux.li> wrote:
>  >> H. Willstrand wrote:
>  >>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>  >>  >> H. Willstrand wrote:
>  >>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>  >>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>  >>  >>  >>  > What's the problem with having a name of the formalized data in the
>  >>  >>  >>  > video stream? ie raw do not mean undefined.
>  >>  >>  >>
>  >>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>  >>  >>  >>  exploding number of proprietary formats that are quite similar but still
>  >>  >>  >>  incompatible. It makes sense for formats that are used by more than one
>  >>  >>  >>  driver.
>  >>  >>  >
>  >>  >>  > Correct, the number of unique pixel formats should be kept down.
>  >>  >>  > Again, comparing with digital cameras there are >200 proprietary
>  >>  >>  > formats and there is a "clean-up" on-going where the "market" is
>  >>  >>  > aiming for a OpenRAW.
>  >>  >>  >
>  >>  >>  > However, by declaring a generic RAW format (which is then driver
>  >>  >>  > specific) doesn't help the user mode app developers. Calling a
>  >>  >>  > multitude of libraries to see if you get lucky might not be a good
>  >>  >>  > idea.
>  >>  >>  >
>  >>  >>  > Still, I'm suspectious about the definition "raw" used here.
>  >>  >>  > RAW should mean unprocessed image data:
>  >>  >>  > * no white balance adjustment
>  >>  >>  > * no color saturation adjustments
>  >>  >>  > * no contrast adjustments
>  >>  >>  > * no sharpness improvements
>  >>  >>  > * no compression with loss
>  >>  >>
>  >>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>  >>  >>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>  >>  >>
>  >>  >
>  >>  > I struggle with the probability to find several CCD's having similar
>  >>  > formats. There aren't so many manifactors of CCD's but they truelly
>  >>  > can generate divergeting formats. Worst case scenario means >200
>  >>  > V4L2_PIX_FMT_RAW_...
>  >>  >
>  >>  > I think RAW is a OK name, the question is if the subcomponents of the
>  >>  > RAW formats has similarities, if so they might be standardized.
>  >>  > Looking into different Sony CCD's it's clearly possible, but after the
>  >>  > CCD the data has to be buffered, packaged and transmitted which of
>  >>  > course can be done in several ways...
>  >>  >
>  >>  > Cheers,
>  >>  > Harri
>  >>  >
>  >>  >>  >
>  >>  >>  > So, by looking for similarities in the "raw" formats where available
>  >>  >>  > there should be a potential to consolidate them.
>  >>  >>  >
>  >>  >>  >>
>  >>  >>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>  >>  >>  >>  > it fits into the current API.
>  >>  >>  >>
>  >>  >>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>  >>  >>  >>  private control id to select one.
>  >>  >>  >>
>  >>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>  >>  >>  > should do the job.
>  >>  >>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>  >>  >>  >
>  >>  >>  > Harri
>  >>
>  >>  Actually, in a webcam you have the image sensor and a usb bridge. Usually, the
>  >>  sensor capture a picture in Bayer pattern. This gets forwarded to the usb
>  >>  bridge. The usb bridge may or may not transfer the picture to an other format
>  >>  and/or compress it with a standard compression algo or a proprietary compression
>  >>  algo. The resulting data stream will be transmitted over the usb interface.
>  >>
>  >
>  > Yes, the USB bridge buffers, packages and transmits.
>  >
>  >>  I just would like to get this resulting stream to user space without
>  >>  manipulation/conversion/decoding of the stream in the kernel module.
>  >>
>  >>  That means we don't know what the format is in this data which comes trough the
>  >>  usb interface. That's way I call it raw.
>  >>
>  >>  At the moment with V4L2, I have to forward a stream to user space which is in a
>  >>  format v4l2 knows. That means I have sometimes to do heavy data processing in
>  >>  the kernel module to decode/convert the data from the usb stream to a known v4l2
>  >>  video format.
>  >>
>  >
>  > Drivers should not do any decoding / converting, it's not allowed in
>  > kernel mode.
>  > But you are right, there are a number of V4L1 exceptions:
>  > AR M64278 (arv.c) converts YUV422 to YUV422P
>  > QuickCam (bw-qcam.c) converts RAW to a useful format :)
>  > CPiA (cpia.c) converts 420 to different RGB formats
>  > OmniVision (ov511.c) converts from YUV4:0:0
>  > PWC (V4L2) does decoding
>
>  You forgot gspca [1](support of 260 webcams at the moment) and here we even do
>  jpeg decoding in kernel space to get the proper format for v4l1!
>
>
>  > ...
>  >
>  > However, the Webcams provides only a limited set of formats and the
>  > "raw" are usually available. New drivers with proprietary "raw"
>  > formats should be added to videodev2.h
>
>  That means you agree with me?
>

Did so from start :)
With the small exception, I still beleive that all RAW formats should
be distinct, not one generic V4L2_PIX_FMT_RAW.

Cheers,
Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

H. Willstrand wrote:
> On Thu, Feb 21, 2008 at 9:40 PM, Thomas Kaiser
> <linux-dvb@kaiser-linux.li> wrote:
>> H. Willstrand wrote:
>>  > On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >> H. Willstrand wrote:
>>  >>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>>  >>  >> H. Willstrand wrote:
>>  >>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>>  >>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>>  >>  >>  >>  > What's the problem with having a name of the formalized data in the
>>  >>  >>  >>  > video stream? ie raw do not mean undefined.
>>  >>  >>  >>
>>  >>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>>  >>  >>  >>  exploding number of proprietary formats that are quite similar but still
>>  >>  >>  >>  incompatible. It makes sense for formats that are used by more than one
>>  >>  >>  >>  driver.
>>  >>  >>  >
>>  >>  >>  > Correct, the number of unique pixel formats should be kept down.
>>  >>  >>  > Again, comparing with digital cameras there are >200 proprietary
>>  >>  >>  > formats and there is a "clean-up" on-going where the "market" is
>>  >>  >>  > aiming for a OpenRAW.
>>  >>  >>  >
>>  >>  >>  > However, by declaring a generic RAW format (which is then driver
>>  >>  >>  > specific) doesn't help the user mode app developers. Calling a
>>  >>  >>  > multitude of libraries to see if you get lucky might not be a good
>>  >>  >>  > idea.
>>  >>  >>  >
>>  >>  >>  > Still, I'm suspectious about the definition "raw" used here.
>>  >>  >>  > RAW should mean unprocessed image data:
>>  >>  >>  > * no white balance adjustment
>>  >>  >>  > * no color saturation adjustments
>>  >>  >>  > * no contrast adjustments
>>  >>  >>  > * no sharpness improvements
>>  >>  >>  > * no compression with loss
>>  >>  >>
>>  >>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>>  >>  >>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>>  >>  >>
>>  >>  >
>>  >>  > I struggle with the probability to find several CCD's having similar
>>  >>  > formats. There aren't so many manifactors of CCD's but they truelly
>>  >>  > can generate divergeting formats. Worst case scenario means >200
>>  >>  > V4L2_PIX_FMT_RAW_...
>>  >>  >
>>  >>  > I think RAW is a OK name, the question is if the subcomponents of the
>>  >>  > RAW formats has similarities, if so they might be standardized.
>>  >>  > Looking into different Sony CCD's it's clearly possible, but after the
>>  >>  > CCD the data has to be buffered, packaged and transmitted which of
>>  >>  > course can be done in several ways...
>>  >>  >
>>  >>  > Cheers,
>>  >>  > Harri
>>  >>  >
>>  >>  >>  >
>>  >>  >>  > So, by looking for similarities in the "raw" formats where available
>>  >>  >>  > there should be a potential to consolidate them.
>>  >>  >>  >
>>  >>  >>  >>
>>  >>  >>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>>  >>  >>  >>  > it fits into the current API.
>>  >>  >>  >>
>>  >>  >>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>>  >>  >>  >>  private control id to select one.
>>  >>  >>  >>
>>  >>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>>  >>  >>  > should do the job.
>>  >>  >>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>>  >>  >>  >
>>  >>  >>  > Harri
>>  >>
>>  >>  Actually, in a webcam you have the image sensor and a usb bridge. Usually, the
>>  >>  sensor capture a picture in Bayer pattern. This gets forwarded to the usb
>>  >>  bridge. The usb bridge may or may not transfer the picture to an other format
>>  >>  and/or compress it with a standard compression algo or a proprietary compression
>>  >>  algo. The resulting data stream will be transmitted over the usb interface.
>>  >>
>>  >
>>  > Yes, the USB bridge buffers, packages and transmits.
>>  >
>>  >>  I just would like to get this resulting stream to user space without
>>  >>  manipulation/conversion/decoding of the stream in the kernel module.
>>  >>
>>  >>  That means we don't know what the format is in this data which comes trough the
>>  >>  usb interface. That's way I call it raw.
>>  >>
>>  >>  At the moment with V4L2, I have to forward a stream to user space which is in a
>>  >>  format v4l2 knows. That means I have sometimes to do heavy data processing in
>>  >>  the kernel module to decode/convert the data from the usb stream to a known v4l2
>>  >>  video format.
>>  >>
>>  >
>>  > Drivers should not do any decoding / converting, it's not allowed in
>>  > kernel mode.
>>  > But you are right, there are a number of V4L1 exceptions:
>>  > AR M64278 (arv.c) converts YUV422 to YUV422P
>>  > QuickCam (bw-qcam.c) converts RAW to a useful format :)
>>  > CPiA (cpia.c) converts 420 to different RGB formats
>>  > OmniVision (ov511.c) converts from YUV4:0:0
>>  > PWC (V4L2) does decoding
>>
>>  You forgot gspca [1](support of 260 webcams at the moment) and here we even do
>>  jpeg decoding in kernel space to get the proper format for v4l1!
>>
>>
>>  > ...
>>  >
>>  > However, the Webcams provides only a limited set of formats and the
>>  > "raw" are usually available. New drivers with proprietary "raw"
>>  > formats should be added to videodev2.h
>>
>>  That means you agree with me?
>>
> 
> Did so from start :)
> With the small exception, I still beleive that all RAW formats should
> be distinct, not one generic V4L2_PIX_FMT_RAW.

Yes and No, It all depends on how the user space app can handle the raw 
"unknown" stream. But as I wrote earlier, this is an other story ;-)

So, now, I have to ask Mauro to include this?
Or what is the proper way to get this official?

Thomas


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

On Thu, Feb 21, 2008 at 10:29 PM, Thomas Kaiser
<linux-dvb@kaiser-linux.li> wrote:
>
> H. Willstrand wrote:
>  > On Thu, Feb 21, 2008 at 9:40 PM, Thomas Kaiser
>  > <linux-dvb@kaiser-linux.li> wrote:
>  >> H. Willstrand wrote:
>  >>  > On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>  >>  >> H. Willstrand wrote:
>  >>  >>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>  >>  >>  > <linux-dvb@kaiser-linux.li> wrote:
>  >>  >>  >> H. Willstrand wrote:
>  >>  >>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> wrote:
>  >>  >>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>  >>  >>  >>  >>  > What's the problem with having a name of the formalized data in the
>  >>  >>  >>  >>  > video stream? ie raw do not mean undefined.
>  >>  >>  >>  >>
>  >>  >>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for an
>  >>  >>  >>  >>  exploding number of proprietary formats that are quite similar but still
>  >>  >>  >>  >>  incompatible. It makes sense for formats that are used by more than one
>  >>  >>  >>  >>  driver.
>  >>  >>  >>  >
>  >>  >>  >>  > Correct, the number of unique pixel formats should be kept down.
>  >>  >>  >>  > Again, comparing with digital cameras there are >200 proprietary
>  >>  >>  >>  > formats and there is a "clean-up" on-going where the "market" is
>  >>  >>  >>  > aiming for a OpenRAW.
>  >>  >>  >>  >
>  >>  >>  >>  > However, by declaring a generic RAW format (which is then driver
>  >>  >>  >>  > specific) doesn't help the user mode app developers. Calling a
>  >>  >>  >>  > multitude of libraries to see if you get lucky might not be a good
>  >>  >>  >>  > idea.
>  >>  >>  >>  >
>  >>  >>  >>  > Still, I'm suspectious about the definition "raw" used here.
>  >>  >>  >>  > RAW should mean unprocessed image data:
>  >>  >>  >>  > * no white balance adjustment
>  >>  >>  >>  > * no color saturation adjustments
>  >>  >>  >>  > * no contrast adjustments
>  >>  >>  >>  > * no sharpness improvements
>  >>  >>  >>  > * no compression with loss
>  >>  >>  >>
>  >>  >>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF headers are
>  >>  >>  >>  done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It Is) is the better name?
>  >>  >>  >>
>  >>  >>  >
>  >>  >>  > I struggle with the probability to find several CCD's having similar
>  >>  >>  > formats. There aren't so many manifactors of CCD's but they truelly
>  >>  >>  > can generate divergeting formats. Worst case scenario means >200
>  >>  >>  > V4L2_PIX_FMT_RAW_...
>  >>  >>  >
>  >>  >>  > I think RAW is a OK name, the question is if the subcomponents of the
>  >>  >>  > RAW formats has similarities, if so they might be standardized.
>  >>  >>  > Looking into different Sony CCD's it's clearly possible, but after the
>  >>  >>  > CCD the data has to be buffered, packaged and transmitted which of
>  >>  >>  > course can be done in several ways...
>  >>  >>  >
>  >>  >>  > Cheers,
>  >>  >>  > Harri
>  >>  >>  >
>  >>  >>  >>  >
>  >>  >>  >>  > So, by looking for similarities in the "raw" formats where available
>  >>  >>  >>  > there should be a potential to consolidate them.
>  >>  >>  >>  >
>  >>  >>  >>  >>
>  >>  >>  >>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2 API,
>  >>  >>  >>  >>  > it fits into the current API.
>  >>  >>  >>  >>
>  >>  >>  >>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>  >>  >>  >>  >>  private control id to select one.
>  >>  >>  >>  >>
>  >>  >>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>  >>  >>  >>  > should do the job.
>  >>  >>  >>  > I.e. I think there should be strong reasons to break V4L2 API behavior.
>  >>  >>  >>  >
>  >>  >>  >>  > Harri
>  >>  >>
>  >>  >>  Actually, in a webcam you have the image sensor and a usb bridge. Usually, the
>  >>  >>  sensor capture a picture in Bayer pattern. This gets forwarded to the usb
>  >>  >>  bridge. The usb bridge may or may not transfer the picture to an other format
>  >>  >>  and/or compress it with a standard compression algo or a proprietary compression
>  >>  >>  algo. The resulting data stream will be transmitted over the usb interface.
>  >>  >>
>  >>  >
>  >>  > Yes, the USB bridge buffers, packages and transmits.
>  >>  >
>  >>  >>  I just would like to get this resulting stream to user space without
>  >>  >>  manipulation/conversion/decoding of the stream in the kernel module.
>  >>  >>
>  >>  >>  That means we don't know what the format is in this data which comes trough the
>  >>  >>  usb interface. That's way I call it raw.
>  >>  >>
>  >>  >>  At the moment with V4L2, I have to forward a stream to user space which is in a
>  >>  >>  format v4l2 knows. That means I have sometimes to do heavy data processing in
>  >>  >>  the kernel module to decode/convert the data from the usb stream to a known v4l2
>  >>  >>  video format.
>  >>  >>
>  >>  >
>  >>  > Drivers should not do any decoding / converting, it's not allowed in
>  >>  > kernel mode.
>  >>  > But you are right, there are a number of V4L1 exceptions:
>  >>  > AR M64278 (arv.c) converts YUV422 to YUV422P
>  >>  > QuickCam (bw-qcam.c) converts RAW to a useful format :)
>  >>  > CPiA (cpia.c) converts 420 to different RGB formats
>  >>  > OmniVision (ov511.c) converts from YUV4:0:0
>  >>  > PWC (V4L2) does decoding
>  >>
>  >>  You forgot gspca [1](support of 260 webcams at the moment) and here we even do
>  >>  jpeg decoding in kernel space to get the proper format for v4l1!
>  >>
>  >>
>  >>  > ...
>  >>  >
>  >>  > However, the Webcams provides only a limited set of formats and the
>  >>  > "raw" are usually available. New drivers with proprietary "raw"
>  >>  > formats should be added to videodev2.h
>  >>
>  >>  That means you agree with me?
>  >>
>  >
>  > Did so from start :)
>  > With the small exception, I still beleive that all RAW formats should
>  > be distinct, not one generic V4L2_PIX_FMT_RAW.
>
>  Yes and No, It all depends on how the user space app can handle the raw
>  "unknown" stream. But as I wrote earlier, this is an other story ;-)
>
>  So, now, I have to ask Mauro to include this?
>  Or what is the proper way to get this official?
>

Correct.

Cheers,
Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Laurent Pinchart]

Hi Thomas and Harri,

On Thursday 21 February 2008, Thomas Kaiser wrote:
> H. Willstrand wrote:
> > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
> >
> > <linux-dvb@kaiser-linux.li> wrote:
> >> H. Willstrand wrote:
> >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> 
wrote:
> >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
> >>  >>  > What's the problem with having a name of the formalized data in
> >>  >>  > the video stream? ie raw do not mean undefined.
> >>  >>
> >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for
> >>  >> an exploding number of proprietary formats that are quite similar
> >>  >> but still incompatible. It makes sense for formats that are used by
> >>  >> more than one driver.
> >>  >
> >>  > Correct, the number of unique pixel formats should be kept down.
> >>  > Again, comparing with digital cameras there are >200 proprietary
> >>  > formats and there is a "clean-up" on-going where the "market" is
> >>  > aiming for a OpenRAW.
> >>  >
> >>  > However, by declaring a generic RAW format (which is then driver
> >>  > specific) doesn't help the user mode app developers. Calling a
> >>  > multitude of libraries to see if you get lucky might not be a good
> >>  > idea.
> >>  >
> >>  > Still, I'm suspectious about the definition "raw" used here.
> >>  > RAW should mean unprocessed image data:
> >>  > * no white balance adjustment
> >>  > * no color saturation adjustments
> >>  > * no contrast adjustments
> >>  > * no sharpness improvements
> >>  > * no compression with loss
> >>
> >>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF
> >> headers are done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It
> >> Is) is the better name?
> >
> > I struggle with the probability to find several CCD's having similar
> > formats. There aren't so many manifactors of CCD's but they truelly
> > can generate divergeting formats. Worst case scenario means >200
> > V4L2_PIX_FMT_RAW_...
> >
> > I think RAW is a OK name, the question is if the subcomponents of the
> > RAW formats has similarities, if so they might be standardized.
> > Looking into different Sony CCD's it's clearly possible, but after the
> > CCD the data has to be buffered, packaged and transmitted which of
> > course can be done in several ways...
> >
> > Cheers,
> > Harri
> >
> >>  > So, by looking for similarities in the "raw" formats where available
> >>  > there should be a potential to consolidate them.
> >>  >
> >>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2
> >>  >>  > API, it fits into the current API.
> >>  >>
> >>  >>  Yes, it does. Each driver having multiple raw formats just needs a
> >>  >>  private control id to select one.
> >>  >
> >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
> >>  > should do the job.
> >>  > I.e. I think there should be strong reasons to break V4L2 API
> >>  > behavior.
> >>  >
> >>  > Harri
>
> Actually, in a webcam you have the image sensor and a usb bridge. Usually,
> the sensor capture a picture in Bayer pattern. This gets forwarded to the
> usb bridge. The usb bridge may or may not transfer the picture to an other
> format and/or compress it with a standard compression algo or a proprietary
> compression algo. The resulting data stream will be transmitted over the
> usb interface.
>
> I just would like to get this resulting stream to user space without
> manipulation/conversion/decoding of the stream in the kernel module.

For most USB devices the resulting stream will be completely unusable. USB 
provides packet boundary information that would be lost. Except when the data 
transmitted over USB is stream-based (MPEG 2/4 for instance), the resulting 
stream will have headers intermixed with data with no way to tell them apart.

> That means we don't know what the format is in this data which comes trough
> the usb interface. That's way I call it raw.

That's not true. If you really don't know what format the data are in, there 
is little point in getting the data.

Video frames can be in a standard format (MJPEG, RGB, ...) or in a proprietary 
format (mostly device-specific compression formats). That format is 
identified at the V4L2 level by a four character code (fourcc). A quick look 
at videodev2.h shows that several vendor-specific formats are defined 
(SN9C10x compression, pwc compression, ...). New formats should be added as 
new devices hit the market.

> At the moment with V4L2, I have to forward a stream to user space which is
> in a format v4l2 knows. That means I have sometimes to do heavy data
> processing in the kernel module to decode/convert the data from the usb
> stream to a known v4l2 video format.

Nothing in V4L2 prevents you from using a private fourcc, although it would be 
easier to submit new fourcc's for inclusion in V4L2. As long as the userspace 
application supports the format and recognises its fourcc your kernel driver 
will only have to handle the USB headers (which are part of the protocol, not 
the format) and won't have to touch the video frame data at all.

> That's way I want a official way to forward the untouched usb stream to
> user space!

Just define a new fourcc and use it to identify your data format.

> How the user space application has to react on this stream is an other
> story, I think. But there will be some way to tell the usespace application
> what to do with this "unknown" stream, I am sure.

With a defined fourcc the stream won't be "unknown".

Best regards,

Laurent Pinchart

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Laurent Pinchart]

On Thursday 21 February 2008, Thomas Kaiser wrote:
> H. Willstrand wrote:
> > On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
> >
> > <linux-dvb@kaiser-linux.li> wrote:
> >> H. Willstrand wrote:
> >>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
> >>  >
> >>  > <linux-dvb@kaiser-linux.li> wrote:
> >>  >> H. Willstrand wrote:
> >>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner 
<daniel-gl@gmx.net> wrote:
> >>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
> >>  >>  >>  > What's the problem with having a name of the formalized data
> >>  >>  >>  > in the video stream? ie raw do not mean undefined.
> >>  >>  >>
> >>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x
> >>  >>  >> for an exploding number of proprietary formats that are quite
> >>  >>  >> similar but still incompatible. It makes sense for formats that
> >>  >>  >> are used by more than one driver.
> >>  >>  >
> >>  >>  > Correct, the number of unique pixel formats should be kept down.
> >>  >>  > Again, comparing with digital cameras there are >200 proprietary
> >>  >>  > formats and there is a "clean-up" on-going where the "market" is
> >>  >>  > aiming for a OpenRAW.
> >>  >>  >
> >>  >>  > However, by declaring a generic RAW format (which is then driver
> >>  >>  > specific) doesn't help the user mode app developers. Calling a
> >>  >>  > multitude of libraries to see if you get lucky might not be a
> >>  >>  > good idea.
> >>  >>  >
> >>  >>  > Still, I'm suspectious about the definition "raw" used here.
> >>  >>  > RAW should mean unprocessed image data:
> >>  >>  > * no white balance adjustment
> >>  >>  > * no color saturation adjustments
> >>  >>  > * no contrast adjustments
> >>  >>  > * no sharpness improvements
> >>  >>  > * no compression with loss
> >>  >>
> >>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of
> >>  >> SOF headers are done in the driver. May be V4L2_PIX_FMT_AII (AII ->
> >>  >> As It Is) is the better name?
> >>  >
> >>  > I struggle with the probability to find several CCD's having similar
> >>  > formats. There aren't so many manifactors of CCD's but they truelly
> >>  > can generate divergeting formats. Worst case scenario means >200
> >>  > V4L2_PIX_FMT_RAW_...
> >>  >
> >>  > I think RAW is a OK name, the question is if the subcomponents of the
> >>  > RAW formats has similarities, if so they might be standardized.
> >>  > Looking into different Sony CCD's it's clearly possible, but after
> >>  > the CCD the data has to be buffered, packaged and transmitted which
> >>  > of course can be done in several ways...
> >>  >
> >>  > Cheers,
> >>  > Harri
> >>  >
> >>  >>  > So, by looking for similarities in the "raw" formats where
> >>  >>  > available there should be a potential to consolidate them.
> >>  >>  >
> >>  >>  >>  > I don't see how separate RAW ioctl's will add value to the
> >>  >>  >>  > V4l2 API, it fits into the current API.
> >>  >>  >>
> >>  >>  >>  Yes, it does. Each driver having multiple raw formats just
> >>  >>  >> needs a private control id to select one.
> >>  >>  >
> >>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
> >>  >>  > should do the job.
> >>  >>  > I.e. I think there should be strong reasons to break V4L2 API
> >>  >>  > behavior.
> >>  >>  >
> >>  >>  > Harri
> >>
> >>  Actually, in a webcam you have the image sensor and a usb bridge.
> >> Usually, the sensor capture a picture in Bayer pattern. This gets
> >> forwarded to the usb bridge. The usb bridge may or may not transfer the
> >> picture to an other format and/or compress it with a standard
> >> compression algo or a proprietary compression algo. The resulting data
> >> stream will be transmitted over the usb interface.
> >
> > Yes, the USB bridge buffers, packages and transmits.
> >
> >>  I just would like to get this resulting stream to user space without
> >>  manipulation/conversion/decoding of the stream in the kernel module.
> >>
> >>  That means we don't know what the format is in this data which comes
> >> trough the usb interface. That's way I call it raw.
> >>
> >>  At the moment with V4L2, I have to forward a stream to user space which
> >> is in a format v4l2 knows. That means I have sometimes to do heavy data
> >> processing in the kernel module to decode/convert the data from the usb
> >> stream to a known v4l2 video format.
> >
> > Drivers should not do any decoding / converting, it's not allowed in
> > kernel mode.
> > But you are right, there are a number of V4L1 exceptions:
> > AR M64278 (arv.c) converts YUV422 to YUV422P
> > QuickCam (bw-qcam.c) converts RAW to a useful format :)
> > CPiA (cpia.c) converts 420 to different RGB formats
> > OmniVision (ov511.c) converts from YUV4:0:0
> > PWC (V4L2) does decoding
>
> You forgot gspca [1](support of 260 webcams at the moment) and here we even
> do jpeg decoding in kernel space to get the proper format for v4l1!

There are historical reasons. Those drivers should be fixed to remove decoding 
from kernelspace. Obviously a new userspace component will be needed to 
handle decoding and conversion, otherwise applications will break. No 
consensus exists today regarding what form that component should take.

> > ...
> >
> > However, the Webcams provides only a limited set of formats and the
> > "raw" are usually available. New drivers with proprietary "raw"
> > formats should be added to videodev2.h
>
> That means you agree with me?
>
> >>  That's way I want a official way to forward the untouched usb stream to
> >> user space!
> >>
> >>  How the user space application has to react on this stream is an other
> >> story, I think. But there will be some way to tell the usespace
> >> application what to do with this "unknown" stream, I am sure.
> >>
> >>  Thomas
> >
> > Cheers,
> > Harri
>
> Thomas
>
> [1] http://mxhaard.free.fr/download.html

Best regards,

Laurent Pinchart

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

Laurent Pinchart wrote:
> On Thursday 21 February 2008, Thomas Kaiser wrote:
>> H. Willstrand wrote:
>>> On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>>>
>>> <linux-dvb@kaiser-linux.li> wrote:
>>>> H. Willstrand wrote:
>>>>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>>>>  >
>>>>  > <linux-dvb@kaiser-linux.li> wrote:
>>>>  >> H. Willstrand wrote:
>>>>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner 
> <daniel-gl@gmx.net> wrote:
>>>>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>>>>  >>  >>  > What's the problem with having a name of the formalized data
>>>>  >>  >>  > in the video stream? ie raw do not mean undefined.
>>>>  >>  >>
>>>>  >>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x
>>>>  >>  >> for an exploding number of proprietary formats that are quite
>>>>  >>  >> similar but still incompatible. It makes sense for formats that
>>>>  >>  >> are used by more than one driver.
>>>>  >>  >
>>>>  >>  > Correct, the number of unique pixel formats should be kept down.
>>>>  >>  > Again, comparing with digital cameras there are >200 proprietary
>>>>  >>  > formats and there is a "clean-up" on-going where the "market" is
>>>>  >>  > aiming for a OpenRAW.
>>>>  >>  >
>>>>  >>  > However, by declaring a generic RAW format (which is then driver
>>>>  >>  > specific) doesn't help the user mode app developers. Calling a
>>>>  >>  > multitude of libraries to see if you get lucky might not be a
>>>>  >>  > good idea.
>>>>  >>  >
>>>>  >>  > Still, I'm suspectious about the definition "raw" used here.
>>>>  >>  > RAW should mean unprocessed image data:
>>>>  >>  > * no white balance adjustment
>>>>  >>  > * no color saturation adjustments
>>>>  >>  > * no contrast adjustments
>>>>  >>  > * no sharpness improvements
>>>>  >>  > * no compression with loss
>>>>  >>
>>>>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of
>>>>  >> SOF headers are done in the driver. May be V4L2_PIX_FMT_AII (AII ->
>>>>  >> As It Is) is the better name?
>>>>  >
>>>>  > I struggle with the probability to find several CCD's having similar
>>>>  > formats. There aren't so many manifactors of CCD's but they truelly
>>>>  > can generate divergeting formats. Worst case scenario means >200
>>>>  > V4L2_PIX_FMT_RAW_...
>>>>  >
>>>>  > I think RAW is a OK name, the question is if the subcomponents of the
>>>>  > RAW formats has similarities, if so they might be standardized.
>>>>  > Looking into different Sony CCD's it's clearly possible, but after
>>>>  > the CCD the data has to be buffered, packaged and transmitted which
>>>>  > of course can be done in several ways...
>>>>  >
>>>>  > Cheers,
>>>>  > Harri
>>>>  >
>>>>  >>  > So, by looking for similarities in the "raw" formats where
>>>>  >>  > available there should be a potential to consolidate them.
>>>>  >>  >
>>>>  >>  >>  > I don't see how separate RAW ioctl's will add value to the
>>>>  >>  >>  > V4l2 API, it fits into the current API.
>>>>  >>  >>
>>>>  >>  >>  Yes, it does. Each driver having multiple raw formats just
>>>>  >>  >> needs a private control id to select one.
>>>>  >>  >
>>>>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>>>>  >>  > should do the job.
>>>>  >>  > I.e. I think there should be strong reasons to break V4L2 API
>>>>  >>  > behavior.
>>>>  >>  >
>>>>  >>  > Harri
>>>>
>>>>  Actually, in a webcam you have the image sensor and a usb bridge.
>>>> Usually, the sensor capture a picture in Bayer pattern. This gets
>>>> forwarded to the usb bridge. The usb bridge may or may not transfer the
>>>> picture to an other format and/or compress it with a standard
>>>> compression algo or a proprietary compression algo. The resulting data
>>>> stream will be transmitted over the usb interface.
>>> Yes, the USB bridge buffers, packages and transmits.
>>>
>>>>  I just would like to get this resulting stream to user space without
>>>>  manipulation/conversion/decoding of the stream in the kernel module.
>>>>
>>>>  That means we don't know what the format is in this data which comes
>>>> trough the usb interface. That's way I call it raw.
>>>>
>>>>  At the moment with V4L2, I have to forward a stream to user space which
>>>> is in a format v4l2 knows. That means I have sometimes to do heavy data
>>>> processing in the kernel module to decode/convert the data from the usb
>>>> stream to a known v4l2 video format.
>>> Drivers should not do any decoding / converting, it's not allowed in
>>> kernel mode.
>>> But you are right, there are a number of V4L1 exceptions:
>>> AR M64278 (arv.c) converts YUV422 to YUV422P
>>> QuickCam (bw-qcam.c) converts RAW to a useful format :)
>>> CPiA (cpia.c) converts 420 to different RGB formats
>>> OmniVision (ov511.c) converts from YUV4:0:0
>>> PWC (V4L2) does decoding
>> You forgot gspca [1](support of 260 webcams at the moment) and here we even
>> do jpeg decoding in kernel space to get the proper format for v4l1!
> 
> There are historical reasons. Those drivers should be fixed to remove decoding 
> from kernelspace. Obviously a new userspace component will be needed to 
> handle decoding and conversion, otherwise applications will break. No 
> consensus exists today regarding what form that component should take.

Yes, but all this transformation which is done in kernel space can be done in 
user space. But it looks like that nobody is interested to move this to user 
space (expect you) ;-)
And I think it should not be that hard to introduce a user space component to 
handle this. When the user space app programmers are willing to do so!

> 
>>> ...
>>>
>>> However, the Webcams provides only a limited set of formats and the
>>> "raw" are usually available. New drivers with proprietary "raw"
>>> formats should be added to videodev2.h
>> That means you agree with me?
>>
>>>>  That's way I want a official way to forward the untouched usb stream to
>>>> user space!
>>>>
>>>>  How the user space application has to react on this stream is an other
>>>> story, I think. But there will be some way to tell the usespace
>>>> application what to do with this "unknown" stream, I am sure.
>>>>
>>>>  Thomas
>>> Cheers,
>>> Harri
>> Thomas
>>
>> [1] http://mxhaard.free.fr/download.html
> 
> Best regards,
> 
> Laurent Pinchart

Best Regards,

Thomas

-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

Laurent Pinchart wrote:
> Hi Thomas and Harri,
> 
> On Thursday 21 February 2008, Thomas Kaiser wrote:
>> H. Willstrand wrote:
>>> On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>>>
>>> <linux-dvb@kaiser-linux.li> wrote:
>>>> H. Willstrand wrote:
>>>>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner <daniel-gl@gmx.net> 
> wrote:
>>>>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand wrote:
>>>>  >>  > What's the problem with having a name of the formalized data in
>>>>  >>  > the video stream? ie raw do not mean undefined.
>>>>  >>
>>>>  >>  I thought you wanted to avoid having to define V4L2_PIX_FMT_x for
>>>>  >> an exploding number of proprietary formats that are quite similar
>>>>  >> but still incompatible. It makes sense for formats that are used by
>>>>  >> more than one driver.
>>>>  >
>>>>  > Correct, the number of unique pixel formats should be kept down.
>>>>  > Again, comparing with digital cameras there are >200 proprietary
>>>>  > formats and there is a "clean-up" on-going where the "market" is
>>>>  > aiming for a OpenRAW.
>>>>  >
>>>>  > However, by declaring a generic RAW format (which is then driver
>>>>  > specific) doesn't help the user mode app developers. Calling a
>>>>  > multitude of libraries to see if you get lucky might not be a good
>>>>  > idea.
>>>>  >
>>>>  > Still, I'm suspectious about the definition "raw" used here.
>>>>  > RAW should mean unprocessed image data:
>>>>  > * no white balance adjustment
>>>>  > * no color saturation adjustments
>>>>  > * no contrast adjustments
>>>>  > * no sharpness improvements
>>>>  > * no compression with loss
>>>>
>>>>  Yes, raw means "as it is" no stripping, decoding  or removing of SOF
>>>> headers are done in the driver. May be V4L2_PIX_FMT_AII (AII -> As It
>>>> Is) is the better name?
>>> I struggle with the probability to find several CCD's having similar
>>> formats. There aren't so many manifactors of CCD's but they truelly
>>> can generate divergeting formats. Worst case scenario means >200
>>> V4L2_PIX_FMT_RAW_...
>>>
>>> I think RAW is a OK name, the question is if the subcomponents of the
>>> RAW formats has similarities, if so they might be standardized.
>>> Looking into different Sony CCD's it's clearly possible, but after the
>>> CCD the data has to be buffered, packaged and transmitted which of
>>> course can be done in several ways...
>>>
>>> Cheers,
>>> Harri
>>>
>>>>  > So, by looking for similarities in the "raw" formats where available
>>>>  > there should be a potential to consolidate them.
>>>>  >
>>>>  >>  > I don't see how separate RAW ioctl's will add value to the V4l2
>>>>  >>  > API, it fits into the current API.
>>>>  >>
>>>>  >>  Yes, it does. Each driver having multiple raw formats just needs a
>>>>  >>  private control id to select one.
>>>>  >
>>>>  > I was more thinking about the VIDIOC_S_RAW stuff, a VIDIOC_S_FMT
>>>>  > should do the job.
>>>>  > I.e. I think there should be strong reasons to break V4L2 API
>>>>  > behavior.
>>>>  >
>>>>  > Harri
>> Actually, in a webcam you have the image sensor and a usb bridge. Usually,
>> the sensor capture a picture in Bayer pattern. This gets forwarded to the
>> usb bridge. The usb bridge may or may not transfer the picture to an other
>> format and/or compress it with a standard compression algo or a proprietary
>> compression algo. The resulting data stream will be transmitted over the
>> usb interface.
>>
>> I just would like to get this resulting stream to user space without
>> manipulation/conversion/decoding of the stream in the kernel module.
> 
> For most USB devices the resulting stream will be completely unusable. USB 
> provides packet boundary information that would be lost. Except when the data 
> transmitted over USB is stream-based (MPEG 2/4 for instance), the resulting 
> stream will have headers intermixed with data with no way to tell them apart.
> 
>> That means we don't know what the format is in this data which comes trough
>> the usb interface. That's way I call it raw.
> 
> That's not true. If you really don't know what format the data are in, there 
> is little point in getting the data.

I mean in known video formats. Some webcames do not send known video formats.

> 
> Video frames can be in a standard format (MJPEG, RGB, ...) or in a proprietary 
> format (mostly device-specific compression formats). That format is 
> identified at the V4L2 level by a four character code (fourcc). A quick look 
> at videodev2.h shows that several vendor-specific formats are defined 
> (SN9C10x compression, pwc compression, ...). New formats should be added as 
> new devices hit the market.
> 
>> At the moment with V4L2, I have to forward a stream to user space which is
>> in a format v4l2 knows. That means I have sometimes to do heavy data
>> processing in the kernel module to decode/convert the data from the usb
>> stream to a known v4l2 video format.
> 
> Nothing in V4L2 prevents you from using a private fourcc, although it would be 
> easier to submit new fourcc's for inclusion in V4L2. As long as the userspace 
> application supports the format and recognises its fourcc your kernel driver 
> will only have to handle the USB headers (which are part of the protocol, not 
> the format) and won't have to touch the video frame data at all.

Looks like I have to do my homework!

> 
>> That's way I want a official way to forward the untouched usb stream to
>> user space!
> 
> Just define a new fourcc and use it to identify your data format.

OK, Homework, again ;-)

> 
>> How the user space application has to react on this stream is an other
>> story, I think. But there will be some way to tell the usespace application
>> what to do with this "unknown" stream, I am sure.
> 
> With a defined fourcc the stream won't be "unknown".

Homework?

> 
> Best regards,
> 
> Laurent Pinchart

Thanks for the information.

Best Regards,

Thomas


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thierry Merle]

Hi Thomas ,
Thomas Kaiser a écrit :
> Laurent Pinchart wrote:
>> On Thursday 21 February 2008, Thomas Kaiser wrote:
>>> H. Willstrand wrote:
>>>> On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>>>>
>>>> <linux-dvb@kaiser-linux.li> wrote:
>>>>> H. Willstrand wrote:
>>>>>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>>>>>  >
>>>>>  > <linux-dvb@kaiser-linux.li> wrote:
>>>>>  >> H. Willstrand wrote:
>>>>>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner 
>> <daniel-gl@gmx.net> wrote:
>>>>>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand
>>>>> wrote:
>>>>>  >>  >>  > What's the problem with having a name of the formalized
>>>>> data
>>>>>  >>  >>  > in the video stream? ie raw do not mean undefined.
>>>>>  >>  >>
>>>>>  >>  >>  I thought you wanted to avoid having to define
>>>>> V4L2_PIX_FMT_x
>>>>>  >>  >> for an exploding number of proprietary formats that are quite
>>>>>  >>  >> similar but still incompatible. It makes sense for formats
>>>>> that
>>>>>  >>  >> are used by more than one driver.
>>>>>  >>  >
>>>>>  >>  > Correct, the number of unique pixel formats should be kept
>>>>> down.
>>>>>  >>  > Again, comparing with digital cameras there are >200
>>>>> proprietary
>>>>>  >>  > formats and there is a "clean-up" on-going where the
>>>>> "market" is
>>>>>  >>  > aiming for a OpenRAW.
>>>>>  >>  >
>>>>>  >>  > However, by declaring a generic RAW format (which is then
>>>>> driver
>>>>>  >>  > specific) doesn't help the user mode app developers. Calling a
>>>>>  >>  > multitude of libraries to see if you get lucky might not be a
>>>>>  >>  > good idea.
>>>>>  >>  >
>>>>>  >>  > Still, I'm suspectious about the definition "raw" used here.
>>>>>  >>  > RAW should mean unprocessed image data:
>>>>>  >>  > * no white balance adjustment
>>>>>  >>  > * no color saturation adjustments
>>>>>  >>  > * no contrast adjustments
>>>>>  >>  > * no sharpness improvements
>>>>>  >>  > * no compression with loss
>>>>>  >>
>>>>>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of
>>>>>  >> SOF headers are done in the driver. May be V4L2_PIX_FMT_AII
>>>>> (AII ->
>>>>>  >> As It Is) is the better name?
>>>>>  >
>>>>>  > I struggle with the probability to find several CCD's having
>>>>> similar
>>>>>  > formats. There aren't so many manifactors of CCD's but they
>>>>> truelly
>>>>>  > can generate divergeting formats. Worst case scenario means >200
>>>>>  > V4L2_PIX_FMT_RAW_...
>>>>>  >
>>>>>  > I think RAW is a OK name, the question is if the subcomponents
>>>>> of the
>>>>>  > RAW formats has similarities, if so they might be standardized.
>>>>>  > Looking into different Sony CCD's it's clearly possible, but after
>>>>>  > the CCD the data has to be buffered, packaged and transmitted
>>>>> which
>>>>>  > of course can be done in several ways...
>>>>>  >
>>>>>  > Cheers,
>>>>>  > Harri
>>>>>  >
>>>>>  >>  > So, by looking for similarities in the "raw" formats where
>>>>>  >>  > available there should be a potential to consolidate them.
>>>>>  >>  >
>>>>>  >>  >>  > I don't see how separate RAW ioctl's will add value to the
>>>>>  >>  >>  > V4l2 API, it fits into the current API.
>>>>>  >>  >>
>>>>>  >>  >>  Yes, it does. Each driver having multiple raw formats just
>>>>>  >>  >> needs a private control id to select one.
>>>>>  >>  >
>>>>>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a
>>>>> VIDIOC_S_FMT
>>>>>  >>  > should do the job.
>>>>>  >>  > I.e. I think there should be strong reasons to break V4L2 API
>>>>>  >>  > behavior.
>>>>>  >>  >
>>>>>  >>  > Harri
>>>>>
>>>>>  Actually, in a webcam you have the image sensor and a usb bridge.
>>>>> Usually, the sensor capture a picture in Bayer pattern. This gets
>>>>> forwarded to the usb bridge. The usb bridge may or may not
>>>>> transfer the
>>>>> picture to an other format and/or compress it with a standard
>>>>> compression algo or a proprietary compression algo. The resulting
>>>>> data
>>>>> stream will be transmitted over the usb interface.
>>>> Yes, the USB bridge buffers, packages and transmits.
>>>>
>>>>>  I just would like to get this resulting stream to user space without
>>>>>  manipulation/conversion/decoding of the stream in the kernel module.
>>>>>
>>>>>  That means we don't know what the format is in this data which comes
>>>>> trough the usb interface. That's way I call it raw.
>>>>>
>>>>>  At the moment with V4L2, I have to forward a stream to user space
>>>>> which
>>>>> is in a format v4l2 knows. That means I have sometimes to do heavy
>>>>> data
>>>>> processing in the kernel module to decode/convert the data from
>>>>> the usb
>>>>> stream to a known v4l2 video format.
>>>> Drivers should not do any decoding / converting, it's not allowed in
>>>> kernel mode.
>>>> But you are right, there are a number of V4L1 exceptions:
>>>> AR M64278 (arv.c) converts YUV422 to YUV422P
>>>> QuickCam (bw-qcam.c) converts RAW to a useful format :)
>>>> CPiA (cpia.c) converts 420 to different RGB formats
>>>> OmniVision (ov511.c) converts from YUV4:0:0
>>>> PWC (V4L2) does decoding
>>> You forgot gspca [1](support of 260 webcams at the moment) and here
>>> we even
>>> do jpeg decoding in kernel space to get the proper format for v4l1!
>>
>> There are historical reasons. Those drivers should be fixed to remove
>> decoding from kernelspace. Obviously a new userspace component will
>> be needed to handle decoding and conversion, otherwise applications
>> will break. No consensus exists today regarding what form that
>> component should take.
>
> Yes, but all this transformation which is done in kernel space can be
> done in user space. But it looks like that nobody is interested to
> move this to user space (expect you) ;-)
> And I think it should not be that hard to introduce a user space
> component to handle this. When the user space app programmers are
> willing to do so!
>
Well, of course you can participate to the v4l2 library mailing-list.
We started a userspace daemon that would do the frame decompression in
userspace and give back uncompressed frames to the application via a
virtual device driver.
You will find more information on here:
http://www.linuxtv.org/v4lwiki/index.php/V4L2UserspaceLibrary
http://www.linuxtv.org/cgi-bin/mailman/listinfo/v4l2-library
I maintain the current code here:
http://linuxtv.org/hg/~tmerle/v4l2_extension

For now, the virtual driver is here, the userspace daemon is here, the
virtual driver <-> daemon command passing is bound to be here thanks to
Jiri Slaby.
Now, we need to implement the frame decompression. I should do that with
the usbvision driver that includes a decompression algorithm in
kernelspace but we can extend it to any v4l2 device driver.
Feel free to join, comment and ask for precisions!

>>
>>>> ...
>>>>
>>>> However, the Webcams provides only a limited set of formats and the
>>>> "raw" are usually available. New drivers with proprietary "raw"
>>>> formats should be added to videodev2.h
>>> That means you agree with me?
>>>
>>>>>  That's way I want a official way to forward the untouched usb
>>>>> stream to
>>>>> user space!
>>>>>
>>>>>  How the user space application has to react on this stream is an
>>>>> other
>>>>> story, I think. But there will be some way to tell the usespace
>>>>> application what to do with this "unknown" stream, I am sure.
>>>>>
>>>>>  Thomas
>>>> Cheers,
>>>> Harri
>>> Thomas
>>>
>>> [1] http://mxhaard.free.fr/download.html
>>
>> Best regards,
>>
>> Laurent Pinchart
>
> Best Regards,
>
> Thomas
>
Cheers,
Thierry

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thomas Kaiser]

Thierry Merle wrote:
> Hi Thomas ,
> Thomas Kaiser a écrit :
>> Laurent Pinchart wrote:
>>> On Thursday 21 February 2008, Thomas Kaiser wrote:
>>>> H. Willstrand wrote:
>>>>> On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>>>>>
>>>>> <linux-dvb@kaiser-linux.li> wrote:
>>>>>> H. Willstrand wrote:
>>>>>>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>>>>>>  >
>>>>>>  > <linux-dvb@kaiser-linux.li> wrote:
>>>>>>  >> H. Willstrand wrote:
>>>>>>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner 
>>> <daniel-gl@gmx.net> wrote:
>>>>>>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand
>>>>>> wrote:
>>>>>>  >>  >>  > What's the problem with having a name of the formalized
>>>>>> data
>>>>>>  >>  >>  > in the video stream? ie raw do not mean undefined.
>>>>>>  >>  >>
>>>>>>  >>  >>  I thought you wanted to avoid having to define
>>>>>> V4L2_PIX_FMT_x
>>>>>>  >>  >> for an exploding number of proprietary formats that are quite
>>>>>>  >>  >> similar but still incompatible. It makes sense for formats
>>>>>> that
>>>>>>  >>  >> are used by more than one driver.
>>>>>>  >>  >
>>>>>>  >>  > Correct, the number of unique pixel formats should be kept
>>>>>> down.
>>>>>>  >>  > Again, comparing with digital cameras there are >200
>>>>>> proprietary
>>>>>>  >>  > formats and there is a "clean-up" on-going where the
>>>>>> "market" is
>>>>>>  >>  > aiming for a OpenRAW.
>>>>>>  >>  >
>>>>>>  >>  > However, by declaring a generic RAW format (which is then
>>>>>> driver
>>>>>>  >>  > specific) doesn't help the user mode app developers. Calling a
>>>>>>  >>  > multitude of libraries to see if you get lucky might not be a
>>>>>>  >>  > good idea.
>>>>>>  >>  >
>>>>>>  >>  > Still, I'm suspectious about the definition "raw" used here.
>>>>>>  >>  > RAW should mean unprocessed image data:
>>>>>>  >>  > * no white balance adjustment
>>>>>>  >>  > * no color saturation adjustments
>>>>>>  >>  > * no contrast adjustments
>>>>>>  >>  > * no sharpness improvements
>>>>>>  >>  > * no compression with loss
>>>>>>  >>
>>>>>>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of
>>>>>>  >> SOF headers are done in the driver. May be V4L2_PIX_FMT_AII
>>>>>> (AII ->
>>>>>>  >> As It Is) is the better name?
>>>>>>  >
>>>>>>  > I struggle with the probability to find several CCD's having
>>>>>> similar
>>>>>>  > formats. There aren't so many manifactors of CCD's but they
>>>>>> truelly
>>>>>>  > can generate divergeting formats. Worst case scenario means >200
>>>>>>  > V4L2_PIX_FMT_RAW_...
>>>>>>  >
>>>>>>  > I think RAW is a OK name, the question is if the subcomponents
>>>>>> of the
>>>>>>  > RAW formats has similarities, if so they might be standardized.
>>>>>>  > Looking into different Sony CCD's it's clearly possible, but after
>>>>>>  > the CCD the data has to be buffered, packaged and transmitted
>>>>>> which
>>>>>>  > of course can be done in several ways...
>>>>>>  >
>>>>>>  > Cheers,
>>>>>>  > Harri
>>>>>>  >
>>>>>>  >>  > So, by looking for similarities in the "raw" formats where
>>>>>>  >>  > available there should be a potential to consolidate them.
>>>>>>  >>  >
>>>>>>  >>  >>  > I don't see how separate RAW ioctl's will add value to the
>>>>>>  >>  >>  > V4l2 API, it fits into the current API.
>>>>>>  >>  >>
>>>>>>  >>  >>  Yes, it does. Each driver having multiple raw formats just
>>>>>>  >>  >> needs a private control id to select one.
>>>>>>  >>  >
>>>>>>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a
>>>>>> VIDIOC_S_FMT
>>>>>>  >>  > should do the job.
>>>>>>  >>  > I.e. I think there should be strong reasons to break V4L2 API
>>>>>>  >>  > behavior.
>>>>>>  >>  >
>>>>>>  >>  > Harri
>>>>>>
>>>>>>  Actually, in a webcam you have the image sensor and a usb bridge.
>>>>>> Usually, the sensor capture a picture in Bayer pattern. This gets
>>>>>> forwarded to the usb bridge. The usb bridge may or may not
>>>>>> transfer the
>>>>>> picture to an other format and/or compress it with a standard
>>>>>> compression algo or a proprietary compression algo. The resulting
>>>>>> data
>>>>>> stream will be transmitted over the usb interface.
>>>>> Yes, the USB bridge buffers, packages and transmits.
>>>>>
>>>>>>  I just would like to get this resulting stream to user space without
>>>>>>  manipulation/conversion/decoding of the stream in the kernel module.
>>>>>>
>>>>>>  That means we don't know what the format is in this data which comes
>>>>>> trough the usb interface. That's way I call it raw.
>>>>>>
>>>>>>  At the moment with V4L2, I have to forward a stream to user space
>>>>>> which
>>>>>> is in a format v4l2 knows. That means I have sometimes to do heavy
>>>>>> data
>>>>>> processing in the kernel module to decode/convert the data from
>>>>>> the usb
>>>>>> stream to a known v4l2 video format.
>>>>> Drivers should not do any decoding / converting, it's not allowed in
>>>>> kernel mode.
>>>>> But you are right, there are a number of V4L1 exceptions:
>>>>> AR M64278 (arv.c) converts YUV422 to YUV422P
>>>>> QuickCam (bw-qcam.c) converts RAW to a useful format :)
>>>>> CPiA (cpia.c) converts 420 to different RGB formats
>>>>> OmniVision (ov511.c) converts from YUV4:0:0
>>>>> PWC (V4L2) does decoding
>>>> You forgot gspca [1](support of 260 webcams at the moment) and here
>>>> we even
>>>> do jpeg decoding in kernel space to get the proper format for v4l1!
>>> There are historical reasons. Those drivers should be fixed to remove
>>> decoding from kernelspace. Obviously a new userspace component will
>>> be needed to handle decoding and conversion, otherwise applications
>>> will break. No consensus exists today regarding what form that
>>> component should take.
>> Yes, but all this transformation which is done in kernel space can be
>> done in user space. But it looks like that nobody is interested to
>> move this to user space (expect you) ;-)
>> And I think it should not be that hard to introduce a user space
>> component to handle this. When the user space app programmers are
>> willing to do so!
>>
> Well, of course you can participate to the v4l2 library mailing-list.
> We started a userspace daemon that would do the frame decompression in
> userspace and give back uncompressed frames to the application via a
> virtual device driver.
> You will find more information on here:
> http://www.linuxtv.org/v4lwiki/index.php/V4L2UserspaceLibrary
> http://www.linuxtv.org/cgi-bin/mailman/listinfo/v4l2-library
> I maintain the current code here:
> http://linuxtv.org/hg/~tmerle/v4l2_extension

Didn't know that :-(. Looks like I have to do homework again ;-)

Thomas

PS: See you soon on the v4l2 library mailing-list :-)

> 
> For now, the virtual driver is here, the userspace daemon is here, the
> virtual driver <-> daemon command passing is bound to be here thanks to
> Jiri Slaby.
> Now, we need to implement the frame decompression. I should do that with
> the usbvision driver that includes a decompression algorithm in
> kernelspace but we can extend it to any v4l2 device driver.
> Feel free to join, comment and ask for precisions!
> 
>>>>> ...
>>>>>
>>>>> However, the Webcams provides only a limited set of formats and the
>>>>> "raw" are usually available. New drivers with proprietary "raw"
>>>>> formats should be added to videodev2.h
>>>> That means you agree with me?
>>>>
>>>>>>  That's way I want a official way to forward the untouched usb
>>>>>> stream to
>>>>>> user space!
>>>>>>
>>>>>>  How the user space application has to react on this stream is an
>>>>>> other
>>>>>> story, I think. But there will be some way to tell the usespace
>>>>>> application what to do with this "unknown" stream, I am sure.
>>>>>>
>>>>>>  Thomas
>>>>> Cheers,
>>>>> Harri
>>>> Thomas
>>>>
>>>> [1] http://mxhaard.free.fr/download.html
>>> Best regards,
>>>
>>> Laurent Pinchart
>> Best Regards,
>>
>> Thomas
>>
> Cheers,
> Thierry


-- 
http://www.kaiser-linux.li

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[H. Willstrand]

Hi Thierry,

>On Fri, Feb 22, 2008 at 10:38 AM, Thierry Merle <thierry.merle@free.fr> wrote:
> Hi Thomas ,
>  Thomas Kaiser a écrit :
>
>
> > Laurent Pinchart wrote:
>  >> On Thursday 21 February 2008, Thomas Kaiser wrote:
>  >>> H. Willstrand wrote:
>  >>>> On Thu, Feb 21, 2008 at 7:55 PM, Thomas Kaiser
>  >>>>
>  >>>> <linux-dvb@kaiser-linux.li> wrote:
>  >>>>> H. Willstrand wrote:
>  >>>>>  > On Thu, Feb 21, 2008 at 1:00 PM, Thomas Kaiser
>  >>>>>  >
>  >>>>>  > <linux-dvb@kaiser-linux.li> wrote:
>  >>>>>  >> H. Willstrand wrote:
>  >>>>>  >>  > On Thu, Feb 21, 2008 at 2:20 AM, Daniel Glöckner
>  >> <daniel-gl@gmx.net> wrote:
>  >>>>>  >>  >> On Thu, Feb 21, 2008 at 01:02:39AM +0100, H. Willstrand
>  >>>>> wrote:
>  >>>>>  >>  >>  > What's the problem with having a name of the formalized
>  >>>>> data
>  >>>>>  >>  >>  > in the video stream? ie raw do not mean undefined.
>  >>>>>  >>  >>
>  >>>>>  >>  >>  I thought you wanted to avoid having to define
>  >>>>> V4L2_PIX_FMT_x
>  >>>>>  >>  >> for an exploding number of proprietary formats that are quite
>  >>>>>  >>  >> similar but still incompatible. It makes sense for formats
>  >>>>> that
>  >>>>>  >>  >> are used by more than one driver.
>  >>>>>  >>  >
>  >>>>>  >>  > Correct, the number of unique pixel formats should be kept
>  >>>>> down.
>  >>>>>  >>  > Again, comparing with digital cameras there are >200
>  >>>>> proprietary
>  >>>>>  >>  > formats and there is a "clean-up" on-going where the
>  >>>>> "market" is
>  >>>>>  >>  > aiming for a OpenRAW.
>  >>>>>  >>  >
>  >>>>>  >>  > However, by declaring a generic RAW format (which is then
>  >>>>> driver
>  >>>>>  >>  > specific) doesn't help the user mode app developers. Calling a
>  >>>>>  >>  > multitude of libraries to see if you get lucky might not be a
>  >>>>>  >>  > good idea.
>  >>>>>  >>  >
>  >>>>>  >>  > Still, I'm suspectious about the definition "raw" used here.
>  >>>>>  >>  > RAW should mean unprocessed image data:
>  >>>>>  >>  > * no white balance adjustment
>  >>>>>  >>  > * no color saturation adjustments
>  >>>>>  >>  > * no contrast adjustments
>  >>>>>  >>  > * no sharpness improvements
>  >>>>>  >>  > * no compression with loss
>  >>>>>  >>
>  >>>>>  >>  Yes, raw means "as it is" no stripping, decoding  or removing of
>  >>>>>  >> SOF headers are done in the driver. May be V4L2_PIX_FMT_AII
>  >>>>> (AII ->
>  >>>>>  >> As It Is) is the better name?
>  >>>>>  >
>  >>>>>  > I struggle with the probability to find several CCD's having
>  >>>>> similar
>  >>>>>  > formats. There aren't so many manifactors of CCD's but they
>  >>>>> truelly
>  >>>>>  > can generate divergeting formats. Worst case scenario means >200
>  >>>>>  > V4L2_PIX_FMT_RAW_...
>  >>>>>  >
>  >>>>>  > I think RAW is a OK name, the question is if the subcomponents
>  >>>>> of the
>  >>>>>  > RAW formats has similarities, if so they might be standardized.
>  >>>>>  > Looking into different Sony CCD's it's clearly possible, but after
>  >>>>>  > the CCD the data has to be buffered, packaged and transmitted
>  >>>>> which
>  >>>>>  > of course can be done in several ways...
>  >>>>>  >
>  >>>>>  > Cheers,
>  >>>>>  > Harri
>  >>>>>  >
>  >>>>>  >>  > So, by looking for similarities in the "raw" formats where
>  >>>>>  >>  > available there should be a potential to consolidate them.
>  >>>>>  >>  >
>  >>>>>  >>  >>  > I don't see how separate RAW ioctl's will add value to the
>  >>>>>  >>  >>  > V4l2 API, it fits into the current API.
>  >>>>>  >>  >>
>  >>>>>  >>  >>  Yes, it does. Each driver having multiple raw formats just
>  >>>>>  >>  >> needs a private control id to select one.
>  >>>>>  >>  >
>  >>>>>  >>  > I was more thinking about the VIDIOC_S_RAW stuff, a
>  >>>>> VIDIOC_S_FMT
>  >>>>>  >>  > should do the job.
>  >>>>>  >>  > I.e. I think there should be strong reasons to break V4L2 API
>  >>>>>  >>  > behavior.
>  >>>>>  >>  >
>  >>>>>  >>  > Harri
>  >>>>>
>  >>>>>  Actually, in a webcam you have the image sensor and a usb bridge.
>  >>>>> Usually, the sensor capture a picture in Bayer pattern. This gets
>  >>>>> forwarded to the usb bridge. The usb bridge may or may not
>  >>>>> transfer the
>  >>>>> picture to an other format and/or compress it with a standard
>  >>>>> compression algo or a proprietary compression algo. The resulting
>  >>>>> data
>  >>>>> stream will be transmitted over the usb interface.
>  >>>> Yes, the USB bridge buffers, packages and transmits.
>  >>>>
>  >>>>>  I just would like to get this resulting stream to user space without
>  >>>>>  manipulation/conversion/decoding of the stream in the kernel module.
>  >>>>>
>  >>>>>  That means we don't know what the format is in this data which comes
>  >>>>> trough the usb interface. That's way I call it raw.
>  >>>>>
>  >>>>>  At the moment with V4L2, I have to forward a stream to user space
>  >>>>> which
>  >>>>> is in a format v4l2 knows. That means I have sometimes to do heavy
>  >>>>> data
>  >>>>> processing in the kernel module to decode/convert the data from
>  >>>>> the usb
>  >>>>> stream to a known v4l2 video format.
>  >>>> Drivers should not do any decoding / converting, it's not allowed in
>  >>>> kernel mode.
>  >>>> But you are right, there are a number of V4L1 exceptions:
>  >>>> AR M64278 (arv.c) converts YUV422 to YUV422P
>  >>>> QuickCam (bw-qcam.c) converts RAW to a useful format :)
>  >>>> CPiA (cpia.c) converts 420 to different RGB formats
>  >>>> OmniVision (ov511.c) converts from YUV4:0:0
>  >>>> PWC (V4L2) does decoding
>  >>> You forgot gspca [1](support of 260 webcams at the moment) and here
>  >>> we even
>  >>> do jpeg decoding in kernel space to get the proper format for v4l1!
>  >>
>  >> There are historical reasons. Those drivers should be fixed to remove
>  >> decoding from kernelspace. Obviously a new userspace component will
>  >> be needed to handle decoding and conversion, otherwise applications
>  >> will break. No consensus exists today regarding what form that
>  >> component should take.
>  >
>  > Yes, but all this transformation which is done in kernel space can be
>  > done in user space. But it looks like that nobody is interested to
>  > move this to user space (expect you) ;-)
>  > And I think it should not be that hard to introduce a user space
>  > component to handle this. When the user space app programmers are
>  > willing to do so!
>  >
>  Well, of course you can participate to the v4l2 library mailing-list.
>  We started a userspace daemon that would do the frame decompression in
>  userspace and give back uncompressed frames to the application via a
>  virtual device driver.
>  You will find more information on here:
>  http://www.linuxtv.org/v4lwiki/index.php/V4L2UserspaceLibrary
>  http://www.linuxtv.org/cgi-bin/mailman/listinfo/v4l2-library
>  I maintain the current code here:
>  http://linuxtv.org/hg/~tmerle/v4l2_extension
>
>  For now, the virtual driver is here, the userspace daemon is here, the
>  virtual driver <-> daemon command passing is bound to be here thanks to
>  Jiri Slaby.
>  Now, we need to implement the frame decompression. I should do that with
>  the usbvision driver that includes a decompression algorithm in
>  kernelspace but we can extend it to any v4l2 device driver.
>  Feel free to join, comment and ask for precisions!
>

Ok, so the idea is to make converting and decompression transparent
for the application.

Are there any plans to run the v4l2_helper as an shared object in the
application process with a direct interface? (to avoid the
kernel-to-user-space, user-to-kernel-space, kernel-to-user-space)

>
>  >>
>  >>>> ...
>  >>>>
>  >>>> However, the Webcams provides only a limited set of formats and the
>  >>>> "raw" are usually available. New drivers with proprietary "raw"
>  >>>> formats should be added to videodev2.h
>  >>> That means you agree with me?
>  >>>
>  >>>>>  That's way I want a official way to forward the untouched usb
>  >>>>> stream to
>  >>>>> user space!
>  >>>>>
>  >>>>>  How the user space application has to react on this stream is an
>  >>>>> other
>  >>>>> story, I think. But there will be some way to tell the usespace
>  >>>>> application what to do with this "unknown" stream, I am sure.
>  >>>>>
>  >>>>>  Thomas
>  >>>> Cheers,
>  >>>> Harri
>  >>> Thomas
>  >>>
>  >>> [1] http://mxhaard.free.fr/download.html
>  >>
>  >> Best regards,
>  >>
>  >> Laurent Pinchart
>  >
>  > Best Regards,
>  >
>  > Thomas
>  >
>  Cheers,
>  Thierry
>
>
>
>  --
>  video4linux-list mailing list
>  Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
>  https://www.redhat.com/mailman/listinfo/video4linux-list
>

Cheers,
Harri

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Thierry Merle]

H. Willstrand a écrit :
> Hi Thierry,
>
>   
[SNIP]
>
> Ok, so the idea is to make converting and decompression transparent
> for the application.
>
> Are there any plans to run the v4l2_helper as an shared object in the
> application process with a direct interface? (to avoid the
> kernel-to-user-space, user-to-kernel-space, kernel-to-user-space)
>
>   
Yes, this is the aim of the project; the final step I hope will show a
new library that will be used by applications, discussing directly with
the base drivers.
Please join to the v4l2-library mailing-list to discuss about the
implementation of the helper daemon in that way.
Following discussions about this specific subject will be easier!
Thanks,
Thierry

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: V4L2_PIX_FMT_RAW

[Laurent Pinchart]

On Saturday 23 February 2008, Thierry Merle wrote:
> H. Willstrand a écrit :
> > Hi Thierry,
>
> [SNIP]
>
> > Ok, so the idea is to make converting and decompression transparent
> > for the application.
> >
> > Are there any plans to run the v4l2_helper as an shared object in the
> > application process with a direct interface? (to avoid the
> > kernel-to-user-space, user-to-kernel-space, kernel-to-user-space)
>
> Yes, this is the aim of the project; the final step I hope will show a
> new library that will be used by applications, discussing directly with
> the base drivers.

I really want to emphasise this. The final solution must not use any userspace 
daemon. The decompression (and rather transcoding) layer will be used 
directly by applications. This will of course not be transparent.

> Please join to the v4l2-library mailing-list to discuss about the
> implementation of the helper daemon in that way.
> Following discussions about this specific subject will be easier!

Best regards,

Laurent Pinchart

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list