output overlay driver and pix format

output overlay driver and pix format

[Matt Porter]

I'm working on a driver for an internal image processing block in an
SoC. This functionality can combine a buffer stream in various YUV/RGB
formats (selectable) with the framebuffer (or any arbitrary buffer one
wishes to overlay).

This fits quite well into the OUTPUT_OVERLAY support for the most part.
However, the driver will not have OUTPUT capability at all. That is, there is
not a direct external output from the image processor so it doesn't not make
sense to define OUTPUT capability. The results of the image processing
are left in a target buffer that may be used for tv/lcd encoding or fed
back in for additional image processing operations.

So the idea is to set the OUTPUT_OVERLAY pix format to one of the supported
formats, set cropping/scaling/blending. Feed it buffers and it blends
with the framebuffer, shoving the result to the internal target buffer.

The problem is that the V4L2 spec seems to imply that an OUTPUT_OVERLAY
device should not touch the fmtdesc pix fields. In my case, the user needs
to configure 1 of N pixelformat types that can be fed to the OUTPUT_OVERLAY
device. Is this allowed or am I using OUTPUT_OVERLAY differently than
intended? It seems that overlay devices may only be intended to be used
with an associated OUTPUT (or INPUT) device that defines the pix format.

The bottom line is: does it make sense to have a driver with only
OUTPUT_OVERLAY capability?

Any clues here are appreciated.

Thanks,
Matt

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: output overlay driver and pix format

[Hans Verkuil]

Hi Matt,

On Monday 27 October 2008 22:18:37 Matt Porter wrote:
> I'm working on a driver for an internal image processing block in an
> SoC. This functionality can combine a buffer stream in various
> YUV/RGB formats (selectable) with the framebuffer (or any arbitrary
> buffer one wishes to overlay).
>
> This fits quite well into the OUTPUT_OVERLAY support for the most
> part. However, the driver will not have OUTPUT capability at all.
> That is, there is not a direct external output from the image
> processor so it doesn't not make sense to define OUTPUT capability.
> The results of the image processing are left in a target buffer that
> may be used for tv/lcd encoding or fed back in for additional image
> processing operations.

Why wouldn't it make sense to define the OUTPUT capability? Based on 
your description I would say that it definitely is an output device. 
Whether the data ends up on a TV-out connector or in an internal target 
buffer is irrelevant.

> So the idea is to set the OUTPUT_OVERLAY pix format to one of the
> supported formats, set cropping/scaling/blending. Feed it buffers and
> it blends with the framebuffer, shoving the result to the internal
> target buffer.

Overlays use the v4l2_window struct, so you need the output capability 
to be able to select the pix format.

> The problem is that the V4L2 spec seems to imply that an
> OUTPUT_OVERLAY device should not touch the fmtdesc pix fields.

Correct, VIDIOC_S_FMT for an overlay uses v4l2_window struct.

> In my 
> case, the user needs to configure 1 of N pixelformat types that can
> be fed to the OUTPUT_OVERLAY device. Is this allowed or am I using
> OUTPUT_OVERLAY differently than intended? It seems that overlay
> devices may only be intended to be used with an associated OUTPUT (or
> INPUT) device that defines the pix format.

Correct.

> The bottom line is: does it make sense to have a driver with only
> OUTPUT_OVERLAY capability?

In this case I don't think it makes sense. But as I said, I think adding 
an OUTPUT capability is not a problem.

Regards,

	Hans

>
> Any clues here are appreciated.
>
> Thanks,
> Matt
>
> --
> video4linux-list mailing list
> Unsubscribe
> mailto:video4linux-list-request@redhat.com?subject=unsubscribe
> https://www.redhat.com/mailman/listinfo/video4linux-list


--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: output overlay driver and pix format

[Matt Porter]

On Mon, Oct 27, 2008 at 10:59:43PM +0100, Hans Verkuil wrote:
> Hi Matt,

Hi Hans,
 
> On Monday 27 October 2008 22:18:37 Matt Porter wrote:
> > I'm working on a driver for an internal image processing block in an
> > SoC. This functionality can combine a buffer stream in various
> > YUV/RGB formats (selectable) with the framebuffer (or any arbitrary
> > buffer one wishes to overlay).
> >
> > This fits quite well into the OUTPUT_OVERLAY support for the most
> > part. However, the driver will not have OUTPUT capability at all.
> > That is, there is not a direct external output from the image
> > processor so it doesn't not make sense to define OUTPUT capability.
> > The results of the image processing are left in a target buffer that
> > may be used for tv/lcd encoding or fed back in for additional image
> > processing operations.
> 
> Why wouldn't it make sense to define the OUTPUT capability? Based on 
> your description I would say that it definitely is an output device. 
> Whether the data ends up on a TV-out connector or in an internal target 
> buffer is irrelevant.

Ok. I guess it does make sense. I've been used to think in terms of
real-world outputs on previous driver work so that's where the confusion
set in. I can define an output that is the internal target buffer as you
suggest. Since it requires the standards ioctls it seems I'll have to 
define a driver specific standard id for a "system buffer". Perhaps that
should be generic...
 
> > So the idea is to set the OUTPUT_OVERLAY pix format to one of the
> > supported formats, set cropping/scaling/blending. Feed it buffers and
> > it blends with the framebuffer, shoving the result to the internal
> > target buffer.
> 
> Overlays use the v4l2_window struct, so you need the output capability 
> to be able to select the pix format.

Ok, that clarifies it. :)

> > The problem is that the V4L2 spec seems to imply that an
> > OUTPUT_OVERLAY device should not touch the fmtdesc pix fields.
> 
> Correct, VIDIOC_S_FMT for an overlay uses v4l2_window struct.

Ok

> > In my 
> > case, the user needs to configure 1 of N pixelformat types that can
> > be fed to the OUTPUT_OVERLAY device. Is this allowed or am I using
> > OUTPUT_OVERLAY differently than intended? It seems that overlay
> > devices may only be intended to be used with an associated OUTPUT (or
> > INPUT) device that defines the pix format.
> 
> Correct.
> 
> > The bottom line is: does it make sense to have a driver with only
> > OUTPUT_OVERLAY capability?
> 
> In this case I don't think it makes sense. But as I said, I think adding 
> an OUTPUT capability is not a problem.

Yes, seems reasonable to me now. There's one other thing this brings up.
Since my hardware can handle 5 different pixelformats as input I'll
obviously S_FMT those on the OUTPUT device. However, it is possible
to configure hardware such that the processed results in the target buffer
are in 4 different pixel formats. Within V4L, it seems that the
way to handle this would be to have 4 different custom (driver specific)
standards that correspond to the 4 possible pixel formats. Does that sound
right?

-Matt

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: output overlay driver and pix format

[Hans Verkuil]

On Tuesday 28 October 2008 03:00:21 Matt Porter wrote:
> On Mon, Oct 27, 2008 at 10:59:43PM +0100, Hans Verkuil wrote:
> > Hi Matt,
>
> Hi Hans,
>
> > On Monday 27 October 2008 22:18:37 Matt Porter wrote:
> > > I'm working on a driver for an internal image processing block in
> > > an SoC. This functionality can combine a buffer stream in various
> > > YUV/RGB formats (selectable) with the framebuffer (or any
> > > arbitrary buffer one wishes to overlay).
> > >
> > > This fits quite well into the OUTPUT_OVERLAY support for the most
> > > part. However, the driver will not have OUTPUT capability at all.
> > > That is, there is not a direct external output from the image
> > > processor so it doesn't not make sense to define OUTPUT
> > > capability. The results of the image processing are left in a
> > > target buffer that may be used for tv/lcd encoding or fed back in
> > > for additional image processing operations.
> >
> > Why wouldn't it make sense to define the OUTPUT capability? Based
> > on your description I would say that it definitely is an output
> > device. Whether the data ends up on a TV-out connector or in an
> > internal target buffer is irrelevant.
>
> Ok. I guess it does make sense. I've been used to think in terms of
> real-world outputs on previous driver work so that's where the
> confusion set in. I can define an output that is the internal target
> buffer as you suggest. Since it requires the standards ioctls it
> seems I'll have to define a driver specific standard id for a "system
> buffer". Perhaps that should be generic...

If there are a fixed number of possible target buffers, then you can 
associate each output with each buffer. But without knowing the precise 
architecture it's hard for me to comment on this.

> > > So the idea is to set the OUTPUT_OVERLAY pix format to one of the
> > > supported formats, set cropping/scaling/blending. Feed it buffers
> > > and it blends with the framebuffer, shoving the result to the
> > > internal target buffer.
> >
> > Overlays use the v4l2_window struct, so you need the output
> > capability to be able to select the pix format.
>
> Ok, that clarifies it. :)
>
> > > The problem is that the V4L2 spec seems to imply that an
> > > OUTPUT_OVERLAY device should not touch the fmtdesc pix fields.
> >
> > Correct, VIDIOC_S_FMT for an overlay uses v4l2_window struct.
>
> Ok
>
> > > In my
> > > case, the user needs to configure 1 of N pixelformat types that
> > > can be fed to the OUTPUT_OVERLAY device. Is this allowed or am I
> > > using OUTPUT_OVERLAY differently than intended? It seems that
> > > overlay devices may only be intended to be used with an
> > > associated OUTPUT (or INPUT) device that defines the pix format.
> >
> > Correct.
> >
> > > The bottom line is: does it make sense to have a driver with only
> > > OUTPUT_OVERLAY capability?
> >
> > In this case I don't think it makes sense. But as I said, I think
> > adding an OUTPUT capability is not a problem.
>
> Yes, seems reasonable to me now. There's one other thing this brings
> up. Since my hardware can handle 5 different pixelformats as input
> I'll obviously S_FMT those on the OUTPUT device. However, it is
> possible to configure hardware such that the processed results in the
> target buffer are in 4 different pixel formats. Within V4L, it seems
> that the way to handle this would be to have 4 different custom
> (driver specific) standards that correspond to the 4 possible pixel
> formats. Does that sound right?

That seems to be the only way to do this at the moment. Is the target 
buffer's pixel format in any way related to the pixel format of the 
framebuffer? Or are they independent?

It feels rather like a hack to abuse S_STD for this. Can you tell a bit 
more about the sort of formats this device can handle? Is it limited to 
PAL/NTSC type images only or can it be different sizes as well? I'm 
planning on adding a new ioctl to support HDTV and it sounds like this 
is something that new ioctl might support as well.

Regards,

	Hans

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list

Re: output overlay driver and pix format

[Matt Porter]

On Tue, Oct 28, 2008 at 08:20:03AM +0100, Hans Verkuil wrote:
> On Tuesday 28 October 2008 03:00:21 Matt Porter wrote:
> > Ok. I guess it does make sense. I've been used to think in terms of
> > real-world outputs on previous driver work so that's where the
> > confusion set in. I can define an output that is the internal target
> > buffer as you suggest. Since it requires the standards ioctls it
> > seems I'll have to define a driver specific standard id for a "system
> > buffer". Perhaps that should be generic...
> 
> If there are a fixed number of possible target buffers, then you can 
> associate each output with each buffer. But without knowing the precise 
> architecture it's hard for me to comment on this.
 
Ok, the hardware can only be programmed with a fixed target buffer. It
seems the best mapping is to have multiple outputs depending on the
functionality of the target buffer. The normal case is that the output
target buffer is the source for the LCD h/w interface.

<snip>

> > > In this case I don't think it makes sense. But as I said, I think
> > > adding an OUTPUT capability is not a problem.
> >
> > Yes, seems reasonable to me now. There's one other thing this brings
> > up. Since my hardware can handle 5 different pixelformats as input
> > I'll obviously S_FMT those on the OUTPUT device. However, it is
> > possible to configure hardware such that the processed results in the
> > target buffer are in 4 different pixel formats. Within V4L, it seems
> > that the way to handle this would be to have 4 different custom
> > (driver specific) standards that correspond to the 4 possible pixel
> > formats. Does that sound right?
> 
> That seems to be the only way to do this at the moment. Is the target 
> buffer's pixel format in any way related to the pixel format of the 
> framebuffer? Or are they independent?

They are completely independent with regards to the hardware capabilities.
However, the normal usage case would be to set the target (output) buffer
configuration the same as the framebuffer.

Specifically, here's what it can do:

On the video input side it handles YUV420, YUV422P, RGB24, RGB555, RGB565
The framebuffer input handles ARGB32, RGB32, ARGB1555, RGB555, RGB565
Target (output) buffer may be ARGB32, RGB32, RGB24, ARGB1555, RGB555, RGB565

So we could feed it YUV420 video, overlay an RGB565 FB, and output RGB555
if desired. I could simply force the output buffer to be the same as
the framebuffer configuration which will probably be the usual usage
model.

> It feels rather like a hack to abuse S_STD for this. Can you tell a bit 
> more about the sort of formats this device can handle? Is it limited to 
> PAL/NTSC type images only or can it be different sizes as well? I'm 
> planning on adding a new ioctl to support HDTV and it sounds like this 
> is something that new ioctl might support as well.

It does feel like a hack for S_STD. It can output more than PAL/NTSC compatible
images. It can also drive an LCD interface with popular progressive formats
like 800x480, 480x272, etc. Since it can crop and scale, the output size
doesn't need to be the same as the framebuffer size. The end result is that
the output isn't necessarily in the form of something that conforms to
the definition of analog standards. I'm not sure handling arbitrary LCD formats
is appropriate for the HDTV ioctl, but maybe another interface could handle
this type of output. Do you have a pointer to some spec for this new ioctl?
I'm not sure if I missed it on the list before.

Thanks,
Matt

--
video4linux-list mailing list
Unsubscribe mailto:video4linux-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/video4linux-list