* Re: RFC: V4L - Support for video timings at the input/output interface
2009-09-14 13:51 RFC: V4L - Support for video timings at the input/output interface Karicheri, Muralidharan
@ 2009-09-15 6:53 ` Hans Verkuil
2009-09-16 21:04 ` Karicheri, Muralidharan
2009-09-15 8:29 ` Vladimir Pantelic
` (2 subsequent siblings)
3 siblings, 1 reply; 12+ messages in thread
From: Hans Verkuil @ 2009-09-15 6:53 UTC (permalink / raw)
To: Karicheri, Muralidharan
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
Hi Murali,
Thanks for your work on this!
See below for a few notes I made.
On Monday 14 September 2009 15:51:28 Karicheri, Muralidharan wrote:
> RFC: V4L - Support for video timings at the input/output interface
>
> Thanks to Hans Verkuil for his initial proposal and feedbacks to help write this RFC.
>
> Version : 1.0
>
> Background
> -----------
>
> Currently v4l2 specification supports capturing video frames from TV signals using tuners (input type V4L2_INPUT_TYPE_TUNER) and baseband TV signals (V4L2_INPUT_TYPE_CAMERA) and sensors. Similarly on the output side, the signals could be TV signal (V4L2_OUTPUT_TYPE_MODULATOR), baseband TV signal (V4L2_OUTPUT_TYPE_ANALOG) or hybrid analog VGA overlay (V4L2_OUTPUT_TYPE_ANALOGVGAOVERLAY) which output from a graphics card and
> then use chromakeying to replace part of the picture with the video. V4L2_OUTPUT_TYPE_ANALOG & V4L2_INPUT_TYPE_CAMERA are for analog interfaces that includes composite, S-Video and VGA (for output only). Note that even though VGA is a supported output, we don't have anyway to set the standard or timing on the output. Standard ids are only defined for TVs using
> v4l2_std_id and a set of bit masks defined for analog TV standards.
>
> Today we have a wide variety of different interfaces available to transmit/receive video or graphics content between source device and destination device. Following are some of the interfaces used in addition to the ones described in the v4l2 specification.
>
> Component analog input/output interface - ED/HD video
> DVI - Digital only, ANALOG only, DVI integrated that support Digital and
> Analog; Dual Link - Where second data link is used for higher
> bandwidth
> SDI - Serial digital interface standardized by SMPTE
> HDMI - HD video and Audio
> DisplayPort - digital audio/video interconnect by VESA
>
> V4L2 specification currently defined NTSC/PAL/SECAM (all variants) standards for describing the timing of the signal transmitted over these interfaces. Even though the specification defined ANALOG output type for VGA, there are no ways to set the timings used for output to VGA or LCD display monitors. Some of the proprietary implementations used existing standards IOCTL, VIDIOC_S_STD, to set these timings over these interfaces.
> For example, TI has Video Processing Back End (VPBE) on various media SOCs
> (Eg, DM6446, DM355 etc) that can output signal for Analog TV and VGA interfaces (using Digital LCD port) and support timing for displaying SD and HD videos (1080i, 1080p and 720p) as well as over VGA interface to a CRT or LCD display monitor. So we need to enhance the v4l2 specification to allow applications to set these timings in the capture or output devices. This RFC proposes to add new IOCTLs for setting/getting timings over the different interfaces described above and freeze the the use of existing standards
> IOCTL and standards IDs for analog TVs only.
>
> Timings
> -------
>
> The timing at the analog or digital interface that are not covered by the v4l2_std_id can be defined using a set of preset timings that are used by the hardware where these timings are predefined or by a set of timing values which can be configured at the hardware to generate the signal expected at the interface. The former will be used for hardware like TVP7002/THS8200 which specifies preset timing required for output HD video such 1080i50/60, 720p50/60 etc. The latter can be used for hardware that require configuration of frame timing such as front porch, hsync length, vsync length, pixel clock etc. For example the earlier mentioned TI SOCs has a Digital LCD port that can be configured to output different timing values expected by LCD Display monitors.
>
> Preset timings (defined by VESA, SMPTE or BT.656/BT.1120 or others) can be defined by the following structure:-
>
> struct v4l2_dv_preset {
> __u32 preset;
> __u32 reserved[4];
> };
>
> Where preset is one of the following values:-
>
> #define V4L2_DV_CUSTOM 0x00000000
> #define V4L2_DV_480I59_94 0x00000001
> #define V4L2_DV_480I60 0x00000002
> #define V4L2_DV_480P23_976 0x00000003
> #define V4L2_DV_480P24 0x00000004
> #define V4L2_DV_480P29_97 0x00000005
> #define V4L2_DV_480P30 0x00000006
> #define V4L2_DV_576I50 0x00000007
> #define V4L2_DV_576P25 0x00000008
> #define V4L2_DV_576P50 0x00000009
> #define V4L2_DV_720P23_976 0x0000000A
> #define V4L2_DV_720P24 0x0000000B
> #define V4L2_DV_720P25 0x0000000C
> #define V4L2_DV_720P29_97 0x0000000C
> #define V4L2_DV_720P30 0x0000000D
> #define V4L2_DV_720P50 0x0000000E
> #define V4L2_DV_720P59_94 0x0000000F
> #define V4L2_DV_720P60 0x00000010
> #define V4L2_DV_1080I50 0x00000011
> #define V4L2_DV_1080I59_94 0x00000012
> #define V4L2_DV_1080I60 0x00000013
> #define V4L2_DV_1080P23_976 0x00000014
> #define V4L2_DV_1080P24 0x00000015
> #define V4L2_DV_1080P25 0x00000016
> #define V4L2_DV_1080P29_97 0x00000017
> #define V4L2_DV_1080P30 0x00000018
> #define V4L2_DV_1080P60 0x00000019
>
>
> /* Following preset value is used by driver to indicate there is no signal
> detected or could not lock to the signal for VIDIOC_QUERY_DV_PRESET.
> */
> #define V4L2_DV_INVALID 0xFFFFFFFF
>
>
> This list is expected grow over time. So a bit mask is not used for this
> (Unlike analog TV standards) so that many presets can be defined as needed in the future.
>
> To enumerate the DV preset timings available, applications call VIDIOC_ENUM_DV_PRESETS
> using the following structure:-
>
> struct v4l2_dv_enum_presets {
> __u32 index;
> __u32 preset;
> __u8 name[32]; /* Name of the preset timing */
> __u32 reserved[4];
> };
>
> Application set/get the preset by calling VIDIOC_S/G_DV_PRESET using v4l2_dv_preset structure.
>
> Also add a capabilities field in the input and output structure to support presets
>
> /*
> * V I D E O I N P U T S
> */
> struct v4l2_input {
> __u32 index; /* Which input */
> __u8 name[32]; /* Label */
> __u32 type; /* Type of input */
> __u32 audioset; /* Associated audios (bitfield) */
> __u32 tuner; /* Associated tuner */
> v4l2_std_id std;
> __u32 status;
> __u32 capabilities;
> __u32 reserved[3];
> };
>
> /*
> * V I D E O O U T P U T S
> */
> struct v4l2_output {
> __u32 index; /* Which output */
> __u8 name[32]; /* Label */
> __u32 type; /* Type of output */
> __u32 audioset; /* Associated audios (bitfield) */
> __u32 modulator; /* Associated modulator */
> v4l2_std_id std;
> __u32 capabilities;
> __u32 reserved[3];
> };
>
> where capabilities are:-
>
> #define V4L2_SUPPORTS_PRESETS 0x00000001
> #define V4L2_SUPPORTS_CUSTOM_TIMINGS 0x00000002
Rather than use 'SUPPORTS' we should rename it as 'IO_CAP'. That's more in
line with the existing naming conventions.
I also suggest adding V4L2_IO_CAP_STD if the regular S_STD is supported.
This will require going through the code and adding it, although we can
probably also set these caps in v4l2-ioctl automatically.
>
> For setting custom timing at the device, following structure is used which defines the complete set of timing values required at the input and output interface:-
>
> /* timing data values specified by various standards such as BT.1120, BT.656 etc. */
>
> /* bt.656/bt.1120 timing data */
> struct v4l2_bt_timings {
> __u32 interlaced;
> __u64 pixelclock;
> __u32 width, height;
> __u32 polarities;
> __u32 hfrontporch, hsync, htotal;
> __u32 vfrontporch, vsync, vtotal;
> /* timings for bottom frame for interlaced formats */
> __u32 il_vtotal;
> __u32 reserved[16];
> };
>
>
> interlaced - Interlaced or progressive. use following values:-
>
> #define V4L2_DV_INTERLACED 0
> #define V4L2_DV_PROGRESSIVE 1
>
> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
> width - number of pixels in horizontal direction
> height - number of lines in vertical direction
> polarities - Bit mask for following polarities to begin with
>
> #define V4L2_DV_VSYNC_POS_POL 0x1
> #define V4L2_DV_VSYNC_NEG_POL 0x2
> #define V4L2_DV_HSYNC_POS_POL 0x3
> #define V4L2_DV_HSYNC_NEG_POL 0x4
Why have separate pos and neg bits? Also, bitmask defines should define the
actual mask, not the bit numbers.
>
> hfrontporch,hsync, and htotal for horizontal direction and vfrontporch,vsync, and vtotal for vertical direction. il_vtotal is the number of vertical lines for interlaced video for bottom field.
>
> To define a particular timing type, following enum is used:-
>
> enum v4l2_dv_timings_type {
> V4L2_DV_BT_656_1120,
> };
>
> This will allow adding new timing types in the future.
>
> If the driver supports a set of custom timing, it can be set/get using VIDIOC_S/G_DV_TIMINGS
> IOCTL and specifying timings using the structure
>
> struct v4l2_dv_timings {
> enum v4l2_dv_timings_type type;
> union {
> struct v4l2_bt_timings bt;
> __u32 reserved[32];
> };
> };
>
>
> If the driver supports custom timing as well as Presets, it will return V4L2_DV_CUSTOM along with other preset timings for the VIDIOC_ENUM_DV_PRESETS IOCTL call. Application can then call VIDIOC_S/G_TIMING to get/set custom timings at the driver.
>
> To detect a preset timing at the input application calls VIDIOC_QUERY_DV_PRESET which returns the preset using the v4l2_dv_preset structure.
>
> Summary of new ioctls added by this RFC.
>
> #define VIDIOC_ENUM_DV_PRESETS _IOWR('V', 79, struct v4l2_dv_enum_presets)
> #define VIDIOC_S_DV_PRESET _IOWR('V', 80, struct v4l2_dv_preset)
> #define VIDIOC_G_DV_PRESET _IOWR('V', 81, struct v4l2_dv_preset)
> #define VIDIOC_QUERY_DV_PRESET _IOR('V', 82, struct v4l2_dv_preset)
> #define VIDIOC_S_DV_TIMINGS _IOWR('V', 83, struct v4l2_dv_timings)
> #define VIDIOC_G_DV_TIMINGS _IOWR('V', 84, struct v4l2_dv_timings)
>
> Open issues
> -----------
>
> 1.How to handle an HDMI transmitter? It can be put in two different modes: DVI compatible
> or HDMI compatible. Some of the choices are
> a) enumerate them as two different outputs when enumerating.
> b) adding a status bit on the input.
> c) change it using a control
The same is also true for a receiver, BTW. They usually can detect whether the
input is in HDMI or DVI mode.
> 2. Detecting whether there is an analog or digital signal on an DVI-I input:
> a) add new status field value for v4l2_input ?
> #define V4L2_IN_ST_DVI_ANALOG_DETECTED 0x10000000
> #define V4L2_IN_ST_DVI_DITIGITAL_DETECTED 0x20000000
Typo: DIGITAL.
I don't think this has to be DVI specific. Just 'ST_DETECTED_ANALOG' and
ST_DETECTED_DIGITAL' should be sufficient and it is more general as well.
> 3. Detecting an EDID.
> a) adding a status field in v4l2_output and two new ioctls that can
> set the EDID for an input or retrieve it for an output. It should
> also be added as an input/output capability.
>
> 4. ATSC bits in v4l2_std_id: how are they used? Are they used at all for
> that matter?
>
>
> 6. HDMI requires additional investigation. HDMI defines a whole bunch of
> infoframe fields. Most of these can probably be exported as controls?? Is
> HDMI audio handled by alsa?
I want to add another two:
7. How to use S_DV_TIMINGS to setup a sensor.
Sensors can be setup either using the full DV_TIMINGS functionality (e.g.
pixelclock, porches, sync width, etc.) or with a subset only: width, height,
frame period. The latter can already be setup with S_PARM (but do we want
that? S_PARM is very, very ugly). I am thinking of creating a new DV_TIMINGS
type for this with just width, height and frame period.
But this leads to another complication: how to tell the user which timings
type to use? Easiest way is to use the capabilities field: CAP_BT_TIMINGS,
CAP_FRAME_TIMINGS. Of course, this limits the number of possible timings
we can define. The alternative is to add yet another field that contains
the actual timings type. But I think this is overkill.
8. Querying non-standard timings.
Currently not in this proposal. This is a complex topic and at this moment
I can't oversee all the possibilities. I know from personal experience
that this is a very difficult topic and I am not sure whether it can be
done reliably through a QUERY_DV_TIMINGS type ioctl. It may need additional
support in the form of receiver-specific ioctls.
I propose to postpone this until someone actually needs it.
Regards,
Hans
--
Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom
^ permalink raw reply [flat|nested] 12+ messages in thread* RE: RFC: V4L - Support for video timings at the input/output interface
2009-09-15 6:53 ` Hans Verkuil
@ 2009-09-16 21:04 ` Karicheri, Muralidharan
2009-09-16 21:49 ` Hans Verkuil
[not found] ` <801963131.20090919082149@ntlworld.com>
0 siblings, 2 replies; 12+ messages in thread
From: Karicheri, Muralidharan @ 2009-09-16 21:04 UTC (permalink / raw)
To: Hans Verkuil
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
Hans,
I was busy with some of the merge work and also some other issues.
So this delayed response...
>
>Thanks for your work on this!
You are most welcome !
>> Where preset is one of the following values:-
>>
>> #define V4L2_DV_CUSTOM 0x00000000
>> #define V4L2_DV_480I59_94 0x00000001
>> #define V4L2_DV_480I60 0x00000002
>> #define V4L2_DV_480P23_976 0x00000003
>> #define V4L2_DV_480P24 0x00000004
>> #define V4L2_DV_480P29_97 0x00000005
>> #define V4L2_DV_480P30 0x00000006
>> #define V4L2_DV_576I50 0x00000007
>> #define V4L2_DV_576P25 0x00000008
>> #define V4L2_DV_576P50 0x00000009
>> #define V4L2_DV_720P23_976 0x0000000A
>> #define V4L2_DV_720P24 0x0000000B
>> #define V4L2_DV_720P25 0x0000000C
>> #define V4L2_DV_720P29_97 0x0000000C
I need to correct this and below for value.
>> #define V4L2_DV_720P30 0x0000000D
>> #define V4L2_DV_720P50 0x0000000E
>> #define V4L2_DV_720P59_94 0x0000000F
>> #define V4L2_DV_720P60 0x00000010
>> #define V4L2_DV_1080I50 0x00000011
>> #define V4L2_DV_1080I59_94 0x00000012
>> #define V4L2_DV_1080I60 0x00000013
>> #define V4L2_DV_1080P23_976 0x00000014
>> #define V4L2_DV_1080P24 0x00000015
>> #define V4L2_DV_1080P25 0x00000016
>> #define V4L2_DV_1080P29_97 0x00000017
>> #define V4L2_DV_1080P30 0x00000018
>> #define V4L2_DV_1080P60 0x00000019
>>
>>
>> where capabilities are:-
>>
>> #define V4L2_SUPPORTS_PRESETS 0x00000001
>> #define V4L2_SUPPORTS_CUSTOM_TIMINGS 0x00000002
>
>Rather than use 'SUPPORTS' we should rename it as 'IO_CAP'. That's more in
>line with the existing naming conventions.
>
>I also suggest adding V4L2_IO_CAP_STD if the regular S_STD is supported.
>This will require going through the code and adding it, although we can
>probably also set these caps in v4l2-ioctl automatically.
Ok.
>
>>
>> For setting custom timing at the device, following structure is used
>which defines the complete set of timing values required at the input and
>output interface:-
>>
>> /* timing data values specified by various standards such as BT.1120,
>BT.656 etc. */
>>
>> /* bt.656/bt.1120 timing data */
>> struct v4l2_bt_timings {
>> __u32 interlaced;
>> __u64 pixelclock;
>> __u32 width, height;
>> __u32 polarities;
>> __u32 hfrontporch, hsync, htotal;
>> __u32 vfrontporch, vsync, vtotal;
>> /* timings for bottom frame for interlaced formats */
>> __u32 il_vtotal;
>> __u32 reserved[16];
>> };
>>
>>
>> interlaced - Interlaced or progressive. use following values:-
>>
>> #define V4L2_DV_INTERLACED 0
>> #define V4L2_DV_PROGRESSIVE 1
>>
>> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
>> width - number of pixels in horizontal direction
>> height - number of lines in vertical direction
>> polarities - Bit mask for following polarities to begin with
>>
>> #define V4L2_DV_VSYNC_POS_POL 0x1
>> #define V4L2_DV_VSYNC_NEG_POL 0x2
>> #define V4L2_DV_HSYNC_POS_POL 0x3
>> #define V4L2_DV_HSYNC_NEG_POL 0x4
>
>Why have separate pos and neg bits? Also, bitmask defines should define the
>actual mask, not the bit numbers.
Oops! That was your original comment. Need to change these to masks.
as
#define V4L2_DV_VSYNC_POS_POL 0x00000001
#define V4L2_DV_VSYNC_NEG_POL 0x00000002
#define V4L2_DV_HSYNC_POS_POL 0x00000004
#define V4L2_DV_HSYNC_NEG_POL 0x00000008
Here is your comment earlier...
==========================================================================
> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
> width - number of pixels in horizontal direction
> height - number of lines in vertical direction
> polarities - Bit mask for following polarities to begin with
>
> #define V4L2_DV_VSYNC_POL 0x1
> #define V4L2_DV_HSYNC_POL 0x2
I suggest a name like V4L2_DV_VSYNC_POS_POL 0x1 to signify that this sets positive polarity.
=========================================================================
>> Open issues
>> -----------
>>
>> 1.How to handle an HDMI transmitter? It can be put in two different
>modes: DVI compatible
>> or HDMI compatible. Some of the choices are
>> a) enumerate them as two different outputs when enumerating.
>> b) adding a status bit on the input.
>> c) change it using a control
>
>The same is also true for a receiver, BTW. They usually can detect whether
>the
>input is in HDMI or DVI mode.
>
>> 2. Detecting whether there is an analog or digital signal on an DVI-I
>input:
>> a) add new status field value for v4l2_input ?
>> #define V4L2_IN_ST_DVI_ANALOG_DETECTED 0x10000000
>> #define V4L2_IN_ST_DVI_DITIGITAL_DETECTED 0x20000000
>
>Typo: DIGITAL.
Ok.
>
>I don't think this has to be DVI specific. Just 'ST_DETECTED_ANALOG' and
>ST_DETECTED_DIGITAL' should be sufficient and it is more general as well.
>
Ok
>> 3. Detecting an EDID.
>> a) adding a status field in v4l2_output and two new ioctls that can
>> set the EDID for an input or retrieve it for an output. It
>should
>> also be added as an input/output capability.
>>
>> 4. ATSC bits in v4l2_std_id: how are they used? Are they used at all for
>> that matter?
>>
>>
>> 6. HDMI requires additional investigation. HDMI defines a whole bunch of
>> infoframe fields. Most of these can probably be exported as controls??
>Is
>> HDMI audio handled by alsa?
>
>I want to add another two:
>
>7. How to use S_DV_TIMINGS to setup a sensor.
> Sensors can be setup either using the full DV_TIMINGS functionality (e.g.
> pixelclock, porches, sync width, etc.) or with a subset only: width,
>height,
> frame period. The latter can already be setup with S_PARM (but do we
>want
> that? S_PARM is very, very ugly). I am thinking of creating a new
>DV_TIMINGS
> type for this with just width, height and frame period.
> But this leads to another complication: how to tell the user which
>timings
> type to use? Easiest way is to use the capabilities field:
>CAP_BT_TIMINGS,
> CAP_FRAME_TIMINGS. Of course, this limits the number of possible timings
> we can define. The alternative is to add yet another field that contains
> the actual timings type. But I think this is overkill.
How about adding two capability flag - CAP_FIXED_FRAME_RATE and CAP_VARIABLE_FRAME_RATE? and change preset type to
struct v4l2_dv_preset {
__u32 preset;
v4l2_fract fps;
__u32 reserved[4];
};
Where preset can be just referring a resolution and scan type.
#define V4L2_DV_720X480P 0x00000001
#define V4L2_DV_640X480P 0x00000002
#define V4L2_DV_720X576P 0x00000003
and so forth. So for a camera that supports pre-defined presets can
set the CAP_FIXED_FRAME_RATE capability. So Auto exposure may not
be available. If Auto exposure is available, the driver can indicate
CAP_VARIABLE_FRAME_RATE. If a driver supports both, then both flags
can be set and based on the value of fps can decide which mode to
operate on (0/0 - for variable mode, 30/1 - to do 30fps rate).
>
>8. Querying non-standard timings.
> Currently not in this proposal. This is a complex topic and at this
>moment
> I can't oversee all the possibilities. I know from personal experience
> that this is a very difficult topic and I am not sure whether it can be
> done reliably through a QUERY_DV_TIMINGS type ioctl. It may need
>additional
> support in the form of receiver-specific ioctls.
> I propose to postpone this until someone actually needs it.
>
Ok. I will add this.
>Regards,
>
> Hans
>
>--
>Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: RFC: V4L - Support for video timings at the input/output interface
2009-09-16 21:04 ` Karicheri, Muralidharan
@ 2009-09-16 21:49 ` Hans Verkuil
2009-09-17 9:18 ` Jean-Philippe François
[not found] ` <801963131.20090919082149@ntlworld.com>
1 sibling, 1 reply; 12+ messages in thread
From: Hans Verkuil @ 2009-09-16 21:49 UTC (permalink / raw)
To: Karicheri, Muralidharan
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
On Wednesday 16 September 2009 23:04:42 Karicheri, Muralidharan wrote:
> Hans,
>
> I was busy with some of the merge work and also some other issues.
> So this delayed response...
>
> >
> >Thanks for your work on this!
>
> You are most welcome !
<snip>
> >> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
> >> width - number of pixels in horizontal direction
> >> height - number of lines in vertical direction
> >> polarities - Bit mask for following polarities to begin with
> >>
> >> #define V4L2_DV_VSYNC_POS_POL 0x1
> >> #define V4L2_DV_VSYNC_NEG_POL 0x2
> >> #define V4L2_DV_HSYNC_POS_POL 0x3
> >> #define V4L2_DV_HSYNC_NEG_POL 0x4
> >
> >Why have separate pos and neg bits? Also, bitmask defines should define the
> >actual mask, not the bit numbers.
>
> Oops! That was your original comment. Need to change these to masks.
> as
>
> #define V4L2_DV_VSYNC_POS_POL 0x00000001
> #define V4L2_DV_VSYNC_NEG_POL 0x00000002
> #define V4L2_DV_HSYNC_POS_POL 0x00000004
> #define V4L2_DV_HSYNC_NEG_POL 0x00000008
Sorry, but why the NEG_POL masks? This is sufficient:
#define V4L2_DV_VSYNC_POS_POL 0x00000001
#define V4L2_DV_HSYNC_POS_POL 0x00000002
If you don't set the bit then the polarity is negative (0 bitvalue, so negative).
> >7. How to use S_DV_TIMINGS to setup a sensor.
> > Sensors can be setup either using the full DV_TIMINGS functionality (e.g.
> > pixelclock, porches, sync width, etc.) or with a subset only: width,
> >height,
> > frame period. The latter can already be setup with S_PARM (but do we
> >want
> > that? S_PARM is very, very ugly). I am thinking of creating a new
> >DV_TIMINGS
> > type for this with just width, height and frame period.
> > But this leads to another complication: how to tell the user which
> >timings
> > type to use? Easiest way is to use the capabilities field:
> >CAP_BT_TIMINGS,
> > CAP_FRAME_TIMINGS. Of course, this limits the number of possible timings
> > we can define. The alternative is to add yet another field that contains
> > the actual timings type. But I think this is overkill.
>
> How about adding two capability flag - CAP_FIXED_FRAME_RATE and CAP_VARIABLE_FRAME_RATE? and change preset type to
>
> struct v4l2_dv_preset {
> __u32 preset;
> v4l2_fract fps;
> __u32 reserved[4];
> };
>
> Where preset can be just referring a resolution and scan type.
>
> #define V4L2_DV_720X480P 0x00000001
> #define V4L2_DV_640X480P 0x00000002
> #define V4L2_DV_720X576P 0x00000003
>
> and so forth. So for a camera that supports pre-defined presets can
> set the CAP_FIXED_FRAME_RATE capability. So Auto exposure may not
> be available. If Auto exposure is available, the driver can indicate
> CAP_VARIABLE_FRAME_RATE. If a driver supports both, then both flags
> can be set and based on the value of fps can decide which mode to
> operate on (0/0 - for variable mode, 30/1 - to do 30fps rate).
Setting up a sensor is rather messy at the moment. You have the
ENUM_FRAMESIZES and ENUM_FRAMEINTERVALS ioctls that basically give you the
'presets' of a sensor. For exposure we have camera controls. Yet we also
have S_PARM to set the framerate. And to set the resolution we abuse S_FMT.
I don't think we should use preset for anything else besides just uniquely
identifying a particular video timing. It is a good idea though to add the
width, height and fps to struct v4l2_dv_enum_presets. That way apps can
actually know what the preset resolution and fps is.
To be honest I don't have any brilliant ideas at the moment on how to solve
setting sensor timings. At the LPC we have both the UVC maintainer (Laurent
Pinchart) and the libv4l and gspca developer Hans de Goede present, so we
should be able to come up with a good solution to this. My knowledge of
sensors is limited, so I will need help with this.
Regards,
Hans
--
Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: RFC: V4L - Support for video timings at the input/output interface
2009-09-16 21:49 ` Hans Verkuil
@ 2009-09-17 9:18 ` Jean-Philippe François
0 siblings, 0 replies; 12+ messages in thread
From: Jean-Philippe François @ 2009-09-17 9:18 UTC (permalink / raw)
To: Hans Verkuil
Cc: Karicheri, Muralidharan,
davinci-linux-open-source@linux.davincidsp.com,
Linux Media Mailing List
Hans Verkuil a écrit :
>> and so forth. So for a camera that supports pre-defined presets can
>> set the CAP_FIXED_FRAME_RATE capability. So Auto exposure may not
>> be available. If Auto exposure is available, the driver can indicate
>> CAP_VARIABLE_FRAME_RATE. If a driver supports both, then both flags
>> can be set and based on the value of fps can decide which mode to
>> operate on (0/0 - for variable mode, 30/1 - to do 30fps rate).
>
> Setting up a sensor is rather messy at the moment. You have the
> ENUM_FRAMESIZES and ENUM_FRAMEINTERVALS ioctls that basically give you the
> 'presets' of a sensor. For exposure we have camera controls. Yet we also
> have S_PARM to set the framerate. And to set the resolution we abuse S_FMT.
>
Some sensor are very versatile, and so a preset is just an arbitrary
resolution. Yet most of the application will be happy with a few preset
to choose from. I agree that it should be easy to just "set the resolution".
We should have preset, it should still be possible to access all
capabilities of the sensor. I have not contributed any code, but here
are some use case that are quite difficult to handle :
* I don't like the presets.
How should I change resolution ?
-VIDIOC_S_FMT ?
-VIDIOC_S_DV_TIMINGS ?
* I use the sensor with another hardware, I can't handle the default
pixelclock :
-VIDIOC_G/S_DV_TIMINGS seems really handy here. Except I don't necessary
care about / can set the other parameters
* I wan't to change the field of view.
-should I use S_CROP ? But I am not really doing any cropping here, just
changing the firts row, first column.
* Both the sensor and the hardware provide the cropping capability.
-For example, let's say I want to capture a 800x600 windows in the
center of a 5 MPixel sensor, and I work with a video port that has
cropping capabilities.
- solution A : The sensor is configured for a 800x600 capture, and the
video port takes it all.
-solution B : the sensor is configured for a full field capture (5Mpx),
and the videoport takes only some part of the data.
Of course solution A is faster, because the sensor does a readout of
much less data. But with some sensor I will only get solution B.
IMO, S/G_FMT should deal with solution A, and S/G_CROP with solution B.
* Repeat the former point with white balance, exposure, gain, gamma,
rgb2yuv etc...
* "Preview" and "capture mode" ex : MT9D131. I can have the following :
- Full field capture (1600x1200) but 800x600 output
- Full field capture, full resolution output
Ok, this can be handheld with ENUM_FRAME_SIZE
- Windowed 800x600 capture, full resolution output (800x600)
Same resolution, similar or identical framerate.
The possibility to move the window, gives you a very quick zoom
capability anywhere in the picture.
But it is a PITA with the current API
That is why I think the VIDIOC_S_DV_PRESET is a good idea, when you just
want to set a standard resolution.
Regarding the custom timings, I think it puts and emphasis on the timing
problem, while CMOS sensor has some other interesting parameters.
I am glad I can change pixelclock, but changing the capture windows
position would be great to !
Should S_FMT be changed ?
This is the format as seen by the application, changing the windows
position does not change the ouptut format.
Should I use S_CROP ?
See my example with solution A and B. Obviously, solution A is not
really cropping.
Should it be exposed in a CTRL ?
That is the current solution i am using. Hacking the driver. Of course
then I move on a new project because my boss does not want me to spend
time having my change merged upstream. Typical embedded reaction :(
Should we enhance the custom timings RFC proposed by TI to include not
only timing, but perhaps windows pos, and perhaps skipping/binning as well ?
> I don't think we should use preset for anything else besides just uniquely
> identifying a particular video timing. It is a good idea though to add the
> width, height and fps to struct v4l2_dv_enum_presets. That way apps can
> actually know what the preset resolution and fps is.
>
> To be honest I don't have any brilliant ideas at the moment on how to solve
> setting sensor timings.
Neither do I, and I prefer a simple driver I can hack, to a "let's
expose all messiness to userspace, and let them fill a dozen struct"
At the LPC we have both the UVC maintainer (Laurent
> Pinchart) and the libv4l and gspca developer Hans de Goede present, so we
> should be able to come up with a good solution to this. My knowledge of
> sensors is limited, so I will need help with this.
>
> Regards,
>
> Hans
>
^ permalink raw reply [flat|nested] 12+ messages in thread
[parent not found: <801963131.20090919082149@ntlworld.com>]
* [linux-dvb] Fwd: Re[2]: RFC: V4L - Support for video timings at the input/output interface
[not found] ` <801963131.20090919082149@ntlworld.com>
@ 2009-09-19 7:27 ` david may
0 siblings, 0 replies; 12+ messages in thread
From: david may @ 2009-09-19 7:27 UTC (permalink / raw)
To: linux-dvb
This is a forwarded message
From: david may <david.may10@ntlworld.com>
To: "Karicheri, Muralidharan" <m-karicheri2@ti.com>
Date: Saturday, September 19, 2009, 8:21:49 AM
Subject: RFC: V4L - Support for video timings at the input/output interface
===8<==============Original message text===============
Hello Muralidharan,
Wednesday, September 16, 2009, 10:04:42 PM, you wrote:
> Hans,
> I was busy with some of the merge work and also some other issues.
> So this delayed response...
>>
>>Thanks for your work on this!
> You are most welcome !
>>> Where preset is one of the following values:-
>>>
>>> #define V4L2_DV_CUSTOM 0x00000000
>>> #define V4L2_DV_480I59_94 0x00000001
>>> #define V4L2_DV_480I60 0x00000002
>>> #define V4L2_DV_480P23_976 0x00000003
>>> #define V4L2_DV_480P24 0x00000004
>>> #define V4L2_DV_480P29_97 0x00000005
>>> #define V4L2_DV_480P30 0x00000006
>>> #define V4L2_DV_576I50 0x00000007
>>> #define V4L2_DV_576P25 0x00000008
>>> #define V4L2_DV_576P50 0x00000009
>>> #define V4L2_DV_720P23_976 0x0000000A
>>> #define V4L2_DV_720P24 0x0000000B
>>> #define V4L2_DV_720P25 0x0000000C
>>> #define V4L2_DV_720P29_97 0x0000000C
> I need to correct this and below for value.
>>> #define V4L2_DV_720P30 0x0000000D
>>> #define V4L2_DV_720P50 0x0000000E
>>> #define V4L2_DV_720P59_94 0x0000000F
>>> #define V4L2_DV_720P60 0x00000010
>>> #define V4L2_DV_1080I50 0x00000011
>>> #define V4L2_DV_1080I59_94 0x00000012
>>> #define V4L2_DV_1080I60 0x00000013
>>> #define V4L2_DV_1080P23_976 0x00000014
>>> #define V4L2_DV_1080P24 0x00000015
>>> #define V4L2_DV_1080P25 0x00000016
>>> #define V4L2_DV_1080P29_97 0x00000017
>>> #define V4L2_DV_1080P30 0x00000018
>>> #define V4L2_DV_1080P60 0x00000019
>>Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
id just like to point out your missing the generic 1080P50 ,
1080P100 hertz ,1080P200 Hz
and the new 2K and 4K that have already been beta tested in the live DVB
broadcasting POC trials..
as you can see adding missed defines will mess up the flow of your
entry's, there most be a better way to allow for mistakes omissions
and future growth!
--
Best regards,
david mailto:david.may10@ntlworld.com
===8<===========End of original message text===========
--
Best regards,
david mailto:david.may10@ntlworld.com
_______________________________________________
linux-dvb users mailing list
For V4L/DVB development, please use instead linux-media@vger.kernel.org
linux-dvb@linuxtv.org
http://www.linuxtv.org/cgi-bin/mailman/listinfo/linux-dvb
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: RFC: V4L - Support for video timings at the input/output interface
2009-09-14 13:51 RFC: V4L - Support for video timings at the input/output interface Karicheri, Muralidharan
2009-09-15 6:53 ` Hans Verkuil
@ 2009-09-15 8:29 ` Vladimir Pantelic
2009-09-16 21:09 ` Karicheri, Muralidharan
2009-09-15 10:13 ` RFC: " Jean-Philippe François
2009-09-17 9:49 ` Jorge Luis Zapata Muga
3 siblings, 1 reply; 12+ messages in thread
From: Vladimir Pantelic @ 2009-09-15 8:29 UTC (permalink / raw)
To: Karicheri, Muralidharan
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
Karicheri, Muralidharan wrote:
<snip>
> Open issues
> -----------
>
> 1.How to handle an HDMI transmitter? It can be put in two different modes: DVI compatible
> or HDMI compatible. Some of the choices are
> a) enumerate them as two different outputs when enumerating.
> b) adding a status bit on the input.
> c) change it using a control
>
> 2. Detecting whether there is an analog or digital signal on an DVI-I input:
> a) add new status field value for v4l2_input ?
> #define V4L2_IN_ST_DVI_ANALOG_DETECTED 0x10000000
> #define V4L2_IN_ST_DVI_DITIGITAL_DETECTED 0x20000000
>
> 3. Detecting an EDID.
> a) adding a status field in v4l2_output and two new ioctls that can
> set the EDID for an input or retrieve it for an output. It should
> also be added as an input/output capability.
>
> 4. ATSC bits in v4l2_std_id: how are they used? Are they used at all for
> that matter?
>
>
> 6. HDMI requires additional investigation. HDMI defines a whole bunch of
> infoframe fields. Most of these can probably be exported as controls?? Is
> HDMI audio handled by alsa?
7. how does this interface/co-exist with something like DSS2 on the omap3?
who will "own" e.g. HDMI setup, DSS2 or V4L2?
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: RFC: V4L - Support for video timings at the input/output interface
2009-09-15 8:29 ` Vladimir Pantelic
@ 2009-09-16 21:09 ` Karicheri, Muralidharan
2009-09-17 6:39 ` Hiremath, Vaibhav
0 siblings, 1 reply; 12+ messages in thread
From: Karicheri, Muralidharan @ 2009-09-16 21:09 UTC (permalink / raw)
To: Vladimir Pantelic
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
Hi,
>>
>> 6. HDMI requires additional investigation. HDMI defines a whole bunch of
>> infoframe fields. Most of these can probably be exported as
>controls?? Is
>> HDMI audio handled by alsa?
>
>7. how does this interface/co-exist with something like DSS2 on the omap3?
>
>who will "own" e.g. HDMI setup, DSS2 or V4L2?
>
I still don't know what DSS2 is capable of doing. Will DSS2 work with
v4l2 sub devices? If so, this RFC applies. I think DSS2 uses its own
interface to integrate a encoder device. Besides it uses (I believe) sysfs
to change the standards at the output. So I don't have an answer to your
question. I will let DSS2 owner to respond.
Murali
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: V4L - Support for video timings at the input/output interface
2009-09-16 21:09 ` Karicheri, Muralidharan
@ 2009-09-17 6:39 ` Hiremath, Vaibhav
0 siblings, 0 replies; 12+ messages in thread
From: Hiremath, Vaibhav @ 2009-09-17 6:39 UTC (permalink / raw)
To: Karicheri, Muralidharan, Vladimir Pantelic
Cc: davinci-linux-open-source@linux.davincidsp.com,
Linux Media Mailing List
> -----Original Message-----
> From: davinci-linux-open-source-bounces@linux.davincidsp.com
> [mailto:davinci-linux-open-source-bounces@linux.davincidsp.com] On
> Behalf Of Karicheri, Muralidharan
> Sent: Thursday, September 17, 2009 2:39 AM
> To: Vladimir Pantelic
> Cc: davinci-linux-open-source@linux.davincidsp.com; Linux Media
> Mailing List
> Subject: RE: V4L - Support for video timings at the input/output
> interface
>
> Hi,
>
> >>
> >> 6. HDMI requires additional investigation. HDMI defines a whole
> bunch of
> >> infoframe fields. Most of these can probably be exported as
> >controls?? Is
> >> HDMI audio handled by alsa?
> >
> >7. how does this interface/co-exist with something like DSS2 on the
> omap3?
> >
> >who will "own" e.g. HDMI setup, DSS2 or V4L2?
> >
>
> I still don't know what DSS2 is capable of doing. Will DSS2 work
> with
> v4l2 sub devices? If so, this RFC applies. I think DSS2 uses its own
> interface to integrate a encoder device. Besides it uses (I believe)
> sysfs
> to change the standards at the output. So I don't have an answer to
> your
> question. I will let DSS2 owner to respond.
>
[Hiremath, Vaibhav] Let me clarify some points here -
DSS2 is nothing but a simple underneath library which allows DSS hardware access for driver layer (V4L2 and Fbdev). So irrespective of whether DSS2 fits into sub-device model I think digital video timing RFC should fit in.
I do agree that, once media controller will come in, we will have to update DSS2 accordingly. As you might be aware of the fact that, Tomi (from Nokia) owns DSS2, I will have initiate discussion with Tomi on this.
Thanks,
Vaibhav
> Murali
> _______________________________________________
> Davinci-linux-open-source mailing list
> Davinci-linux-open-source@linux.davincidsp.com
> http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-
> source
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: RFC: V4L - Support for video timings at the input/output interface
2009-09-14 13:51 RFC: V4L - Support for video timings at the input/output interface Karicheri, Muralidharan
2009-09-15 6:53 ` Hans Verkuil
2009-09-15 8:29 ` Vladimir Pantelic
@ 2009-09-15 10:13 ` Jean-Philippe François
2009-09-17 17:39 ` Karicheri, Muralidharan
2009-09-17 9:49 ` Jorge Luis Zapata Muga
3 siblings, 1 reply; 12+ messages in thread
From: Jean-Philippe François @ 2009-09-15 10:13 UTC (permalink / raw)
To: Karicheri, Muralidharan
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
Karicheri, Muralidharan a écrit :
> RFC: V4L - Support for video timings at the input/output interface
>
> Thanks to Hans Verkuil for his initial proposal and feedbacks to help write this RFC.
>
> Version : 1.0
>
Thank you for this work.
I have a few questions, regarding the other parameters or existing control.
> Background
> -----------
>
> Currently v4l2 specification supports capturing video frames from TV signals using tuners (input type V4L2_INPUT_TYPE_TUNER) and baseband TV signals (V4L2_INPUT_TYPE_CAMERA) and sensors. Similarly on the output side, the signals could be TV signal (V4L2_OUTPUT_TYPE_MODULATOR), baseband TV signal (V4L2_OUTPUT_TYPE_ANALOG) or hybrid analog VGA overlay (V4L2_OUTPUT_TYPE_ANALOGVGAOVERLAY) which output from a graphics card and
> then use chromakeying to replace part of the picture with the video. V4L2_OUTPUT_TYPE_ANALOG & V4L2_INPUT_TYPE_CAMERA are for analog interfaces that includes composite, S-Video and VGA (for output only). Note that even though VGA is a supported output, we don't have anyway to set the standard or timing on the output. Standard ids are only defined for TVs using
> v4l2_std_id and a set of bit masks defined for analog TV standards.
>
> Today we have a wide variety of different interfaces available to transmit/receive video or graphics content between source device and destination device. Following are some of the interfaces used in addition to the ones described in the v4l2 specification.
>
> Component analog input/output interface - ED/HD video
> DVI - Digital only, ANALOG only, DVI integrated that support Digital and
> Analog; Dual Link - Where second data link is used for higher
> bandwidth
> SDI - Serial digital interface standardized by SMPTE
> HDMI - HD video and Audio
> DisplayPort - digital audio/video interconnect by VESA
>
I would like to say that all capture devices are not necessary used in a
video chain or video system.
When it comes to machine vision, resolution and image quality can be
more interesting than timings.
---- snip ---
> /* timing data values specified by various standards such as BT.1120, BT.656 etc. */
>
> /* bt.656/bt.1120 timing data */
> struct v4l2_bt_timings {
> __u32 interlaced;
> __u64 pixelclock;
> __u32 width, height;
> __u32 polarities;
> __u32 hfrontporch, hsync, htotal;
> __u32 vfrontporch, vsync, vtotal;
> /* timings for bottom frame for interlaced formats */
> __u32 il_vtotal;
> __u32 reserved[16];
> };
>
>
> interlaced - Interlaced or progressive. use following values:-
>
> #define V4L2_DV_INTERLACED 0
> #define V4L2_DV_PROGRESSIVE 1
>
> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
> width - number of pixels in horizontal direction
> height - number of lines in vertical direction
> polarities - Bit mask for following polarities to begin with
>
> #define V4L2_DV_VSYNC_POS_POL 0x1
> #define V4L2_DV_VSYNC_NEG_POL 0x2
> #define V4L2_DV_HSYNC_POS_POL 0x3
> #define V4L2_DV_HSYNC_NEG_POL 0x4
>
> hfrontporch,hsync, and htotal for horizontal direction and vfrontporch,vsync, and vtotal for vertical direction. il_vtotal is the number of vertical lines for interlaced video for bottom field.
>
> To define a particular timing type, following enum is used:-
>
> enum v4l2_dv_timings_type {
> V4L2_DV_BT_656_1120,
> };
>
> This will allow adding new timing types in the future.
>
Let's say we have a CMOS sensor, with a driver implementing this various
timings, because the driver writer was "video oriented".
Now here comes another person, who is interested in another set of
features :
width,
height
col_start
row_start
And he wants autoexposure and don't care about the framerate.
How does he extends the driver ?
Is there a way to set the resolution without going through a standard.
What should he do ?
Add another timing_type ?
Use the S_FMT, G_FMT, TRY_FMT ioctl ?
But then what happens when one does a G_DV_TIMINGS ?
Should a new v4l2_dv_timings_type be defined ?
But it is not a timing we are talking about, it is a geometry.
CMOS sensor presents overlapping feature sets :
Resolution + timing is one.
Resolution + windows position + exposure is another.
Resolution + zoom level is yet another
Let's say a driver is written using the VIDIOC_S/G_DV_TIMINGS.
It should be easy to use the same driver with S/G_FMT or S/G_CROP, even
if it leads to a format you can't describe with VIDIOC_S/G_DV_TIMINGS.
I mean, we are heading to a situation where you can set :
hfrontporch, vfrontporch, htotal, vtotal.
But if you want to set :
row_start, col_start, shutter_width, bining or decimation, you have to
write some custom control.
Now if you look at video capture port and cmos sensor :
- video capture hardware cares about polarities, interlacing, and
pixelcount. They don't care about hfrontporch & htotal, as long as it is
over some minimum value.
- CMOS sensor gives you custom size and windows position, bining and
skipping, shutter width.
They give some control over vsync. but not always.
This timings might be interesting when it comes to display and output,
It might be a relevant feature set, but it is not the only one,
especially when it comes to CMOS sensor capture. I hope adopting these
ioctls won't give a higher priority to timing parameters over other
sensible parameters.
Thank you for reading this
Jean-Philippe François
>
> Summary of new ioctls added by this RFC.
>
> #define VIDIOC_ENUM_DV_PRESETS _IOWR('V', 79, struct v4l2_dv_enum_presets)
> #define VIDIOC_S_DV_PRESET _IOWR('V', 80, struct v4l2_dv_preset)
> #define VIDIOC_G_DV_PRESET _IOWR('V', 81, struct v4l2_dv_preset)
> #define VIDIOC_QUERY_DV_PRESET _IOR('V', 82, struct v4l2_dv_preset)
> #define VIDIOC_S_DV_TIMINGS _IOWR('V', 83, struct v4l2_dv_timings)
> #define VIDIOC_G_DV_TIMINGS _IOWR('V', 84, struct v4l2_dv_timings)
>
> Open issues
> -----------
>
> 1.How to handle an HDMI transmitter? It can be put in two different modes: DVI compatible
> or HDMI compatible. Some of the choices are
> a) enumerate them as two different outputs when enumerating.
> b) adding a status bit on the input.
> c) change it using a control
>
> 2. Detecting whether there is an analog or digital signal on an DVI-I input:
> a) add new status field value for v4l2_input ?
> #define V4L2_IN_ST_DVI_ANALOG_DETECTED 0x10000000
> #define V4L2_IN_ST_DVI_DITIGITAL_DETECTED 0x20000000
>
> 3. Detecting an EDID.
> a) adding a status field in v4l2_output and two new ioctls that can
> set the EDID for an input or retrieve it for an output. It should
> also be added as an input/output capability.
>
> 4. ATSC bits in v4l2_std_id: how are they used? Are they used at all for
> that matter?
>
>
> 6. HDMI requires additional investigation. HDMI defines a whole bunch of
> infoframe fields. Most of these can probably be exported as controls?? Is
> HDMI audio handled by alsa?
>
>
> Murali Karicheri
> Software Design Engineer
> Texas Instruments Inc.
> Germantown, MD 20874
> email: m-karicheri2@ti.com
>
> _______________________________________________
> Davinci-linux-open-source mailing list
> Davinci-linux-open-source@linux.davincidsp.com
> http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source
>
^ permalink raw reply [flat|nested] 12+ messages in thread* RE: RFC: V4L - Support for video timings at the input/output interface
2009-09-15 10:13 ` RFC: " Jean-Philippe François
@ 2009-09-17 17:39 ` Karicheri, Muralidharan
0 siblings, 0 replies; 12+ messages in thread
From: Karicheri, Muralidharan @ 2009-09-17 17:39 UTC (permalink / raw)
To: Jean-Philippe François
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
Jean-Philippe François,
>>
>> Component analog input/output interface - ED/HD video
>> DVI - Digital only, ANALOG only, DVI integrated that support Digital and
>> Analog; Dual Link - Where second data link is used for higher
>> bandwidth
>> SDI - Serial digital interface standardized by SMPTE
>> HDMI - HD video and Audio
>> DisplayPort - digital audio/video interconnect by VESA
>>
>I would like to say that all capture devices are not necessary used in a
>video chain or video system.
>When it comes to machine vision, resolution and image quality can be
>more interesting than timings.
>
I will postpone the discussion to your next point...
>---- snip ---
>> /* timing data values specified by various standards such as BT.1120,
>BT.656 etc. */
>>
>> /* bt.656/bt.1120 timing data */
>> struct v4l2_bt_timings {
>> __u32 interlaced;
>> __u64 pixelclock;
>> __u32 width, height;
>> __u32 polarities;
>> __u32 hfrontporch, hsync, htotal;
>> __u32 vfrontporch, vsync, vtotal;
>> /* timings for bottom frame for interlaced formats */
>> __u32 il_vtotal;
>> __u32 reserved[16];
>> };
>>
>>
>> interlaced - Interlaced or progressive. use following values:-
>>
>> #define V4L2_DV_INTERLACED 0
>> #define V4L2_DV_PROGRESSIVE 1
>>
>> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
>> width - number of pixels in horizontal direction
>> height - number of lines in vertical direction
>> polarities - Bit mask for following polarities to begin with
>>
>> #define V4L2_DV_VSYNC_POS_POL 0x1
>> #define V4L2_DV_VSYNC_NEG_POL 0x2
>> #define V4L2_DV_HSYNC_POS_POL 0x3
>> #define V4L2_DV_HSYNC_NEG_POL 0x4
>>
>> hfrontporch,hsync, and htotal for horizontal direction and
>vfrontporch,vsync, and vtotal for vertical direction. il_vtotal is the
>number of vertical lines for interlaced video for bottom field.
>>
>> To define a particular timing type, following enum is used:-
>>
>> enum v4l2_dv_timings_type {
>> V4L2_DV_BT_656_1120,
>> };
>>
>> This will allow adding new timing types in the future.
>>
>
>Let's say we have a CMOS sensor, with a driver implementing this various
> timings, because the driver writer was "video oriented".
>Now here comes another person, who is interested in another set of
>features :
>width,
>height
>col_start
>row_start
>
>And he wants autoexposure and don't care about the framerate.
>How does he extends the driver ?
>Is there a way to set the resolution without going through a standard.
>What should he do ?
>Add another timing_type ?
Yes. I realized that the RFC doesn't address the camera requirement.
I am using MT9T031/MT9T001/MT9P031 in my response below because that is
what I am familiar with. In these sensors there are two mode of operation, continuous mode and Snapshot mode.
The video timing is relevant to continuous mode where the rows and columns
are read out along with pixel clock, hsync and vsync signals to the output bus. So video timing is quite relevant here. Even though there are no
input interface for this similar to video decoders, the output timing to
the bus is still based the settings for row size, column size, blanking
interval etc. In TI's internal release, we have following parameters
for sensor driver as a static table for each of the resolution frames
output to bus. We maintain following values for a given resolution and fps
column_size, row_size, h_blank, v_blank, shutter_width, row_addr_mode,
col_addr_mode, row_start, col_start etc
So when a particular resolution capture is set by user, driver will configure sensor with these values (done using S_STD currently).
In the MT9T031 driver available in v4l2 sub system today, my understanding is that some of these values are taken from S_FMT such as width (column_size), height (row_size). row_start and col_start comes form S_CROP. Some of these values are fixed such as h_blank and v_blank. So I think Hans proposal will work to support both these cases.
Also there are sensors that can support BT timing as well. So these will be
able to use preset or dv_timigs.
Based on the feedback so far, So I propose to make following additions/modifications to the RFC. we define following capabilities:-
case 1) Analog TVs at input/output
add following capability flag for existing v4l2_std_id,
#define V4L2_IO_CAP_STD 0x00000001
This way application knows driver supports standard STD ids for NTSC/PAL/SECAM
case 2) Video timings presets at output/input
To support preset add following capability flag.
#define V4L2_IO_CAP_PRESETS 0x00000002
struct v4l2_dv_preset {
__u32 preset;
__u32 reserved[4];
};
case 3) Camera where frame rate changes with exposure & where frame_rate
is fixed (pre-defined in the driver). Both these needs frame size (width and height). In former case frame rate is based on other factors such as exposure, shutter width etc.
To support camera timing that can be based on width, height, frame rate add following capability flag:-
/* where only width and height can be assured, but not frame rate */
#define V4L2_IO_CAP_FRAME_SIZE 0x00000004
/* where frame rate is fixed (prefined set of resolution and frame rate) or
driver can calculate the frame rate */
#define V4L2_IO_CAP_FRAME_RATE 0x00000008
So if a driver can support both frame size and rate, it can set both flags
struct v4l2_frame_timings {
__u32 width;
__u32 height;
/* nominal frame rate with nominal exposure */
v4l2_fract frame_rate;
__u32 reserved[4];
};
Add new timing type in
enum v4l2_dv_timings_type {
V4L2_DV_BT_656_1120,
V4L2_FRAME_TIMINGS
};
To support BT timing (includes sensors that can set detailed timing)
#define V4L2_IO_CAP_BT_TIMINGS 0x00000010
/* bt.656/bt.1120 timing data */
struct v4l2_bt_timings {
__u32 interlaced;
__u64 pixelclock;
__u32 width, height;
__u32 polarities;
__u32 hfrontporch, hsync, htotal;
__u32 vfrontporch, vsync, vtotal;
/* timings for bottom frame for interlaced formats */
__u32 il_vtotal;
__u32 reserved[16];
};
Some of these parameters applicable for sensors as well.
Some of the sensors can do interlaced or progressive scan type of read out.
width and height refers to the sizes on the sensors output bus. There is
vblank and hblank settings on sensor as well.
case 4) Snapshot mode for camera sensor
In this case, sensor readout happens on a trigger. I think this is covered by case 3 where driver just set the V4L2_IO_CAP_FRAME_SIZE flag and ignore the frame rate in DV_TIMING ioctl.
Let me know if any case is not handled here.
>Use the S_FMT, G_FMT, TRY_FMT ioctl ?
I think we are misusing the API for sensors here. The sensor itself is
only giving an output bus that carries the read out data from rows
and colums with a set of timings that include pixel clock, vertical
blanking & horizontal blanking etc (last two applicable only when frames
are output continuously). There is going to be a host device which sinks this bus and write data to memory using a DMA engine. So there may
be additional capabilities here to pack the data differently (There
is another RFC from Hans that talks about data format on this bus).
So that is where all of S_FMT/G_FMT/TRY_FMT applies. At the sensor,
you just set the output frame size and timings (if doing streaming)
and this RFC address that.
>But then what happens when one does a G_DV_TIMINGS ?
>
>Should a new v4l2_dv_timings_type be defined ?
>But it is not a timing we are talking about, it is a geometry.
>
Yes, the geometry is specified by v4l2_frame_timings structure
where frame rate can be zeros. The driver will indicate that
capability using a flag as suggested above.
>CMOS sensor presents overlapping feature sets :
>Resolution + timing is one.
Can be supported using case & 3) or 4) for sensors. In my view, for sensors, timing refers to the timing at the output bus of sensor.
>Resolution + windows position + exposure is another.
I think what you are referring as window position is the row_start and
column_start of the field of view, right?
Case 3) above can be used to set the frame size at the output bus, but frame rate depends on exposure and final frame size output which in term depends or resize/cropping values used. So this is possible to handle using the new ioctls along with S_CROP and control for exposure.
>Resolution + zoom level is yet another
Zoom is another case of resize IMO. You are specifying the capture
area using S_CROP. In the case of sensor, this will set the row_start
and column_start and row size and column size. So based on final frame size at the output, driver can calculate the zoom factor internally. If host
device can do additional zooming, similar thing happens at the capture
node.
>
>Let's say a driver is written using the VIDIOC_S/G_DV_TIMINGS.
>It should be easy to use the same driver with S/G_FMT or S/G_CROP, even
>if it leads to a format you can't describe with VIDIOC_S/G_DV_TIMINGS.
>
>I mean, we are heading to a situation where you can set :
>hfrontporch, vfrontporch, htotal, vtotal.
>
>But if you want to set :
>row_start, col_start, shutter_width, bining or decimation, you have to
>write some custom control.
>
Why do we need this? S_CROP can be used to set row_start and column_start
etc at the sensor. I know currently all these commands go through the
bridge/host device and it has to decide if scaling/cropping to be at the host device or sub device. I think with the media controller RFC, this can be passed to the sensor directly. shutter width can be changed through a control ioctl, bining/decimation can be used to get the frame timing as specified by the VIDIOC_S_DV_PRESET/VIDIOC_S_DV_TIMINGS ioctl.
>Now if you look at video capture port and cmos sensor :
>- video capture hardware cares about polarities, interlacing, and
>pixelcount. They don't care about hfrontporch & htotal, as long as it is
>over some minimum value.
>
For some cases, these are fixed. So driver can use presets. Other cases detailed timings can be set by hardware for the same. On the output for VESA timing, video hardware do care about hfrontporch & htotal
>- CMOS sensor gives you custom size and windows position, bining and
>skipping, shutter width.
>They give some control over vsync. but not always.
>
>This timings might be interesting when it comes to display and output,
>It might be a relevant feature set, but it is not the only one,
>especially when it comes to CMOS sensor capture. I hope adopting these
>ioctls won't give a higher priority to timing parameters over other
>sensible parameters.
IMO, We should use a common IOCTL, but naming can be adjusted to mean
it is not timing, but rather a frame size for camera devices ( the new
capabilities flag might be enough). If there are settings not controlled by these, MC RFC can be used to change sensor specific settings as a control
or configuration.
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: RFC: V4L - Support for video timings at the input/output interface
2009-09-14 13:51 RFC: V4L - Support for video timings at the input/output interface Karicheri, Muralidharan
` (2 preceding siblings ...)
2009-09-15 10:13 ` RFC: " Jean-Philippe François
@ 2009-09-17 9:49 ` Jorge Luis Zapata Muga
3 siblings, 0 replies; 12+ messages in thread
From: Jorge Luis Zapata Muga @ 2009-09-17 9:49 UTC (permalink / raw)
To: Karicheri, Muralidharan
Cc: Linux Media Mailing List,
davinci-linux-open-source@linux.davincidsp.com
On Mon, Sep 14, 2009 at 3:51 PM, Karicheri, Muralidharan
<m-karicheri2@ti.com> wrote:
> RFC: V4L - Support for video timings at the input/output interface
>
> Thanks to Hans Verkuil for his initial proposal and feedbacks to help write this RFC.
>
> Version : 1.0
>
> Background
> -----------
>
> Currently v4l2 specification supports capturing video frames from TV signals using tuners (input type V4L2_INPUT_TYPE_TUNER) and baseband TV signals (V4L2_INPUT_TYPE_CAMERA) and sensors. Similarly on the output side, the signals could be TV signal (V4L2_OUTPUT_TYPE_MODULATOR), baseband TV signal (V4L2_OUTPUT_TYPE_ANALOG) or hybrid analog VGA overlay (V4L2_OUTPUT_TYPE_ANALOGVGAOVERLAY) which output from a graphics card and
> then use chromakeying to replace part of the picture with the video. V4L2_OUTPUT_TYPE_ANALOG & V4L2_INPUT_TYPE_CAMERA are for analog interfaces that includes composite, S-Video and VGA (for output only). Note that even though VGA is a supported output, we don't have anyway to set the standard or timing on the output. Standard ids are only defined for TVs using
> v4l2_std_id and a set of bit masks defined for analog TV standards.
>
> Today we have a wide variety of different interfaces available to transmit/receive video or graphics content between source device and destination device. Following are some of the interfaces used in addition to the ones described in the v4l2 specification.
>
> Component analog input/output interface - ED/HD video
> DVI - Digital only, ANALOG only, DVI integrated that support Digital and
> Analog; Dual Link - Where second data link is used for higher
> bandwidth
> SDI - Serial digital interface standardized by SMPTE
> HDMI - HD video and Audio
> DisplayPort - digital audio/video interconnect by VESA
>
> V4L2 specification currently defined NTSC/PAL/SECAM (all variants) standards for describing the timing of the signal transmitted over these interfaces. Even though the specification defined ANALOG output type for VGA, there are no ways to set the timings used for output to VGA or LCD display monitors. Some of the proprietary implementations used existing standards IOCTL, VIDIOC_S_STD, to set these timings over these interfaces.
> For example, TI has Video Processing Back End (VPBE) on various media SOCs
> (Eg, DM6446, DM355 etc) that can output signal for Analog TV and VGA interfaces (using Digital LCD port) and support timing for displaying SD and HD videos (1080i, 1080p and 720p) as well as over VGA interface to a CRT or LCD display monitor. So we need to enhance the v4l2 specification to allow applications to set these timings in the capture or output devices. This RFC proposes to add new IOCTLs for setting/getting timings over the different interfaces described above and freeze the the use of existing standards
> IOCTL and standards IDs for analog TVs only.
>
> Timings
> -------
>
> The timing at the analog or digital interface that are not covered by the v4l2_std_id can be defined using a set of preset timings that are used by the hardware where these timings are predefined or by a set of timing values which can be configured at the hardware to generate the signal expected at the interface. The former will be used for hardware like TVP7002/THS8200 which specifies preset timing required for output HD video such 1080i50/60, 720p50/60 etc. The latter can be used for hardware that require configuration of frame timing such as front porch, hsync length, vsync length, pixel clock etc. For example the earlier mentioned TI SOCs has a Digital LCD port that can be configured to output different timing values expected by LCD Display monitors.
>
The proposal seems good, given that v4l2 supports output (input)
devices, it was kind of strange that v4l2 didnt have a way to setup
custom timings on such devices, even so i have some concerns.
For the output device and timings, there's already an API which
supports that on the kernel, the FB API. The framebuffer system can be
seen as an independent system but with some intersection with the v4l2
output device. Some of the differences are modedb, timings, bitblt
acceleration, EDID handling, even a video output switcher via sysfs,
etc and some of the them are now needed for v4l. Also the FB API
suffers on other areas, like YUV planes (as the depth definitions
doesnt support that), chroma key, etc (this are handled by private
ioctls on some fb drivers).
Also the DRM people is adding support for KMS (kernel mode setting)
which also uses a timing definition for that and EDID handling. So,
from a kernel developer/application developer POV, this is a mess. We
have three different API for the same devices/tasks. IMO this RFC is a
good thing, but we need a way to stabilize the kernel graphics area
somehow, maybe share this RFC among the other subsystems (fb, drm)?
Regards
> Preset timings (defined by VESA, SMPTE or BT.656/BT.1120 or others) can be defined by the following structure:-
>
> struct v4l2_dv_preset {
> __u32 preset;
> __u32 reserved[4];
> };
>
> Where preset is one of the following values:-
>
> #define V4L2_DV_CUSTOM 0x00000000
> #define V4L2_DV_480I59_94 0x00000001
> #define V4L2_DV_480I60 0x00000002
> #define V4L2_DV_480P23_976 0x00000003
> #define V4L2_DV_480P24 0x00000004
> #define V4L2_DV_480P29_97 0x00000005
> #define V4L2_DV_480P30 0x00000006
> #define V4L2_DV_576I50 0x00000007
> #define V4L2_DV_576P25 0x00000008
> #define V4L2_DV_576P50 0x00000009
> #define V4L2_DV_720P23_976 0x0000000A
> #define V4L2_DV_720P24 0x0000000B
> #define V4L2_DV_720P25 0x0000000C
> #define V4L2_DV_720P29_97 0x0000000C
> #define V4L2_DV_720P30 0x0000000D
> #define V4L2_DV_720P50 0x0000000E
> #define V4L2_DV_720P59_94 0x0000000F
> #define V4L2_DV_720P60 0x00000010
> #define V4L2_DV_1080I50 0x00000011
> #define V4L2_DV_1080I59_94 0x00000012
> #define V4L2_DV_1080I60 0x00000013
> #define V4L2_DV_1080P23_976 0x00000014
> #define V4L2_DV_1080P24 0x00000015
> #define V4L2_DV_1080P25 0x00000016
> #define V4L2_DV_1080P29_97 0x00000017
> #define V4L2_DV_1080P30 0x00000018
> #define V4L2_DV_1080P60 0x00000019
>
>
> /* Following preset value is used by driver to indicate there is no signal
> detected or could not lock to the signal for VIDIOC_QUERY_DV_PRESET.
> */
> #define V4L2_DV_INVALID 0xFFFFFFFF
>
>
> This list is expected grow over time. So a bit mask is not used for this
> (Unlike analog TV standards) so that many presets can be defined as needed in the future.
>
> To enumerate the DV preset timings available, applications call VIDIOC_ENUM_DV_PRESETS
> using the following structure:-
>
> struct v4l2_dv_enum_presets {
> __u32 index;
> __u32 preset;
> __u8 name[32]; /* Name of the preset timing */
> __u32 reserved[4];
> };
>
> Application set/get the preset by calling VIDIOC_S/G_DV_PRESET using v4l2_dv_preset structure.
>
> Also add a capabilities field in the input and output structure to support presets
>
> /*
> * V I D E O I N P U T S
> */
> struct v4l2_input {
> __u32 index; /* Which input */
> __u8 name[32]; /* Label */
> __u32 type; /* Type of input */
> __u32 audioset; /* Associated audios (bitfield) */
> __u32 tuner; /* Associated tuner */
> v4l2_std_id std;
> __u32 status;
> __u32 capabilities;
> __u32 reserved[3];
> };
>
> /*
> * V I D E O O U T P U T S
> */
> struct v4l2_output {
> __u32 index; /* Which output */
> __u8 name[32]; /* Label */
> __u32 type; /* Type of output */
> __u32 audioset; /* Associated audios (bitfield) */
> __u32 modulator; /* Associated modulator */
> v4l2_std_id std;
> __u32 capabilities;
> __u32 reserved[3];
> };
>
> where capabilities are:-
>
> #define V4L2_SUPPORTS_PRESETS 0x00000001
> #define V4L2_SUPPORTS_CUSTOM_TIMINGS 0x00000002
>
> For setting custom timing at the device, following structure is used which defines the complete set of timing values required at the input and output interface:-
>
> /* timing data values specified by various standards such as BT.1120, BT.656 etc. */
>
> /* bt.656/bt.1120 timing data */
> struct v4l2_bt_timings {
> __u32 interlaced;
> __u64 pixelclock;
> __u32 width, height;
> __u32 polarities;
> __u32 hfrontporch, hsync, htotal;
> __u32 vfrontporch, vsync, vtotal;
> /* timings for bottom frame for interlaced formats */
> __u32 il_vtotal;
> __u32 reserved[16];
> };
>
>
> interlaced - Interlaced or progressive. use following values:-
>
> #define V4L2_DV_INTERLACED 0
> #define V4L2_DV_PROGRESSIVE 1
>
> pixelclock - expressed in HZ. So for 74.25MHz, use 74250000.
> width - number of pixels in horizontal direction
> height - number of lines in vertical direction
> polarities - Bit mask for following polarities to begin with
>
> #define V4L2_DV_VSYNC_POS_POL 0x1
> #define V4L2_DV_VSYNC_NEG_POL 0x2
> #define V4L2_DV_HSYNC_POS_POL 0x3
> #define V4L2_DV_HSYNC_NEG_POL 0x4
>
> hfrontporch,hsync, and htotal for horizontal direction and vfrontporch,vsync, and vtotal for vertical direction. il_vtotal is the number of vertical lines for interlaced video for bottom field.
>
> To define a particular timing type, following enum is used:-
>
> enum v4l2_dv_timings_type {
> V4L2_DV_BT_656_1120,
> };
>
> This will allow adding new timing types in the future.
>
> If the driver supports a set of custom timing, it can be set/get using VIDIOC_S/G_DV_TIMINGS
> IOCTL and specifying timings using the structure
>
> struct v4l2_dv_timings {
> enum v4l2_dv_timings_type type;
> union {
> struct v4l2_bt_timings bt;
> __u32 reserved[32];
> };
> };
>
>
> If the driver supports custom timing as well as Presets, it will return V4L2_DV_CUSTOM along with other preset timings for the VIDIOC_ENUM_DV_PRESETS IOCTL call. Application can then call VIDIOC_S/G_TIMING to get/set custom timings at the driver.
>
> To detect a preset timing at the input application calls VIDIOC_QUERY_DV_PRESET which returns the preset using the v4l2_dv_preset structure.
>
> Summary of new ioctls added by this RFC.
>
> #define VIDIOC_ENUM_DV_PRESETS _IOWR('V', 79, struct v4l2_dv_enum_presets)
> #define VIDIOC_S_DV_PRESET _IOWR('V', 80, struct v4l2_dv_preset)
> #define VIDIOC_G_DV_PRESET _IOWR('V', 81, struct v4l2_dv_preset)
> #define VIDIOC_QUERY_DV_PRESET _IOR('V', 82, struct v4l2_dv_preset)
> #define VIDIOC_S_DV_TIMINGS _IOWR('V', 83, struct v4l2_dv_timings)
> #define VIDIOC_G_DV_TIMINGS _IOWR('V', 84, struct v4l2_dv_timings)
>
> Open issues
> -----------
>
> 1.How to handle an HDMI transmitter? It can be put in two different modes: DVI compatible
> or HDMI compatible. Some of the choices are
> a) enumerate them as two different outputs when enumerating.
> b) adding a status bit on the input.
> c) change it using a control
>
> 2. Detecting whether there is an analog or digital signal on an DVI-I input:
> a) add new status field value for v4l2_input ?
> #define V4L2_IN_ST_DVI_ANALOG_DETECTED 0x10000000
> #define V4L2_IN_ST_DVI_DITIGITAL_DETECTED 0x20000000
>
> 3. Detecting an EDID.
> a) adding a status field in v4l2_output and two new ioctls that can
> set the EDID for an input or retrieve it for an output. It should
> also be added as an input/output capability.
>
> 4. ATSC bits in v4l2_std_id: how are they used? Are they used at all for
> that matter?
>
>
> 6. HDMI requires additional investigation. HDMI defines a whole bunch of
> infoframe fields. Most of these can probably be exported as controls?? Is
> HDMI audio handled by alsa?
>
>
> Murali Karicheri
> Software Design Engineer
> Texas Instruments Inc.
> Germantown, MD 20874
> email: m-karicheri2@ti.com
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 12+ messages in thread