From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp-vbr14.xs4all.nl ([194.109.24.34]:4808 "EHLO smtp-vbr14.xs4all.nl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751087Ab1GZKLd (ORCPT ); Tue, 26 Jul 2011 06:11:33 -0400 From: Hans Verkuil To: Sakari Ailus Subject: Re: [RFC 1/3] v4l: Move event documentation from SUBSCRIBE_EVENT to DQEVENT Date: Tue, 26 Jul 2011 12:11:28 +0200 Cc: linux-media@vger.kernel.org References: <4E2588AD.4070106@maxwell.research.nokia.com> <1311082688-16185-1-git-send-email-sakari.ailus@iki.fi> In-Reply-To: <1311082688-16185-1-git-send-email-sakari.ailus@iki.fi> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201107261211.28542.hverkuil@xs4all.nl> Sender: linux-media-owner@vger.kernel.org List-ID: On Tuesday, July 19, 2011 15:38:06 Sakari Ailus wrote: > Move documentation of structures used in DQEVENT from SUBSCRIBE_EVENT to > DQEVENT. > > Signed-off-by: Sakari Ailus Acked-by: Hans Verkuil This is a much better place! Regards, Hans > --- > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 107 ++++++++++++++++++++ > .../DocBook/media/v4l/vidioc-subscribe-event.xml | 107 -------------------- > 2 files changed, 107 insertions(+), 107 deletions(-) > > diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > index 7769642..5200b68 100644 > --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > @@ -135,6 +135,113 @@ > > > > + > + struct <structname>v4l2_event_vsync</structname> > + > + &cs-str; > + > + > + __u8 > + field > + The upcoming field. See &v4l2-field;. > + > + > + > +
> + > + > + struct <structname>v4l2_event_ctrl</structname> > + > + &cs-str; > + > + > + __u32 > + changes > + > + A bitmask that tells what has changed. See . > + > + > + __u32 > + type > + > + The type of the control. See &v4l2-ctrl-type;. > + > + > + union (anonymous) > + > + > + > + > + > + > + __s32 > + value > + The 32-bit value of the control for 32-bit control types. > + This is 0 for string controls since the value of a string > + cannot be passed using &VIDIOC-DQEVENT;. > + > + > + > + __s64 > + value64 > + The 64-bit value of the control for 64-bit control types. > + > + > + __u32 > + flags > + > + The control flags. See . > + > + > + __s32 > + minimum > + > + The minimum value of the control. See &v4l2-queryctrl;. > + > + > + __s32 > + maximum > + > + The maximum value of the control. See &v4l2-queryctrl;. > + > + > + __s32 > + step > + > + The step value of the control. See &v4l2-queryctrl;. > + > + > + __s32 > + default_value > + > + The default value value of the control. See &v4l2-queryctrl;. > + > + > + > +
> + > + > + Changes > + > + &cs-def; > + > + > + V4L2_EVENT_CTRL_CH_VALUE > + 0x0001 > + This control event was triggered because the value of the control > + changed. Special case: if a button control is pressed, then this > + event is sent as well, even though there is not explicit value > + associated with a button control. > + > + > + V4L2_EVENT_CTRL_CH_FLAGS > + 0x0002 > + This control event was triggered because the control flags > + changed. > + > + > + > +
> > > &return-value; > diff --git a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > index 69c0d8a..275be96 100644 > --- a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > +++ b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > @@ -183,113 +183,6 @@ > > > > - > - struct <structname>v4l2_event_vsync</structname> > - > - &cs-str; > - > - > - __u8 > - field > - The upcoming field. See &v4l2-field;. > - > - > - > -
> - > - > - struct <structname>v4l2_event_ctrl</structname> > - > - &cs-str; > - > - > - __u32 > - changes > - > - A bitmask that tells what has changed. See . > - > - > - __u32 > - type > - > - The type of the control. See &v4l2-ctrl-type;. > - > - > - union (anonymous) > - > - > - > - > - > - > - __s32 > - value > - The 32-bit value of the control for 32-bit control types. > - This is 0 for string controls since the value of a string > - cannot be passed using &VIDIOC-DQEVENT;. > - > - > - > - __s64 > - value64 > - The 64-bit value of the control for 64-bit control types. > - > - > - __u32 > - flags > - > - The control flags. See . > - > - > - __s32 > - minimum > - > - The minimum value of the control. See &v4l2-queryctrl;. > - > - > - __s32 > - maximum > - > - The maximum value of the control. See &v4l2-queryctrl;. > - > - > - __s32 > - step > - > - The step value of the control. See &v4l2-queryctrl;. > - > - > - __s32 > - default_value > - > - The default value value of the control. See &v4l2-queryctrl;. > - > - > - > -
> - > - > - Changes > - > - &cs-def; > - > - > - V4L2_EVENT_CTRL_CH_VALUE > - 0x0001 > - This control event was triggered because the value of the control > - changed. Special case: if a button control is pressed, then this > - event is sent as well, even though there is not explicit value > - associated with a button control. > - > - > - V4L2_EVENT_CTRL_CH_FLAGS > - 0x0002 > - This control event was triggered because the control flags > - changed. > - > - > - > -
> > > &return-value; > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-fx0-f52.google.com ([209.85.161.52]:56954 "EHLO mail-fx0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751569Ab1GUP5r (ORCPT ); Thu, 21 Jul 2011 11:57:47 -0400 Received: by fxd18 with SMTP id 18so3757631fxd.11 for ; Thu, 21 Jul 2011 08:57:46 -0700 (PDT) Message-ID: <4E284C71.7050806@gmail.com> Date: Thu, 21 Jul 2011 17:57:37 +0200 From: Sylwester Nawrocki MIME-Version: 1.0 To: Sakari Ailus CC: "linux-media@vger.kernel.org" Subject: Re: [RFC 0/3] Frame synchronisation events and support for them in the OMAP 3 ISP driver References: <4E2588AD.4070106@maxwell.research.nokia.com> In-Reply-To: <4E2588AD.4070106@maxwell.research.nokia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Sender: linux-media-owner@vger.kernel.org List-ID: Hi Sakari, On 07/19/2011 03:37 PM, Sakari Ailus wrote: > Hi all, > > The OMAP 3 ISP driver implements an HS_VS event which is triggered when > the reception of a frame begins. This functionality is very, very likely > not specific to OMAP 3 ISP so it should be standardised. > > I have a few patches to do that. Additionally the next expected buffer > sequence number is provided with the event, unlike earlier. > > There are a few open questions, however, and this is why I'm sending the > set as RFC. > > > 1) Other frame synchronisation events. The CCDC block in the OMAP 3 ISP > is able to trigger interrupts at two chosen lines of the image. These > naturally can be translated to events. The driver uses both of them > internally at specific points of the frame. Nevertheless, there might be > some use for these in user space. Other hardware might implement a > number of these which wouldn't be used by the driver itself, but I don't > know of that at the moment. On the other hand high resolution timers are > also available in user space, so doing timing based on ISP provided > events is not quite as important as before --- as long as there's one > frame based event produced at a known time, such as V4L2_EVENT_FRAME_START. I'm curious, have you perhaps tried to measure latency of such up calls to a user space process? I mean this is going to be a real time stuff, with HSYNC periods of 50 us order. Could a user space thread be receiving such periodic events reliably ? From my experience I doubt this can work reliably outside of an interrupt handler even with high priority real time threads. V4L2_EVENT_FRAME_START event seems OK, but HSYNC events in user space sound rather tricky to me :-) Also HS_VS looks a bit more descriptive than FRAME_START for me. But unfortunately I can't come up with a better name, e.g. something like V4L2_EVENT_FRAME_AV_START - frame active video start. Just in case in future there are more specific events added. > > Frame end events may be produced as well. This is not exactly the same > as just dequeueing the buffer at video node since the hardware may be > able to produce events even in cases there are no buffers and if the > very hardware block that processes the frame is not outputting it to > memory, handling by further blocks takes more time, and thus delays the > finishing of the buffer from the driver's queue. This is the reason why > the name of the struct related to the event is v4l2_event_frame_sync > rather than v4l2_event_frame_start. > > 2) Buffer sequence number location in the struct v4l2_event. the patches > create a new structure called v4l2_event_frame_sync which contains just > one field, buffer_sequence. Should buffer_sequence be part of this > struct, or should it be part of v4l2_event directly, as the id field? > Both buffer_sequence and id refer to another rather widely used concept > in V4L2. > > > Besides this, the first patch in the series moves the documentation of > structs inside v4l2_event to VIDIOC_DQEVENT documentation. I think it > belongs there rather than to VIDIOC_SUBSCRIBE_EVENT, since that's not > where they are being used. > -- Regards, Sylwester From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp-vbr15.xs4all.nl ([194.109.24.35]:4567 "EHLO smtp-vbr15.xs4all.nl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750946Ab1GZLwr (ORCPT ); Tue, 26 Jul 2011 07:52:47 -0400 From: Hans Verkuil To: Sakari Ailus Subject: Re: [RFC 2/3] v4l: events: Define frame start event Date: Tue, 26 Jul 2011 13:52:44 +0200 Cc: linux-media@vger.kernel.org References: <4E2588AD.4070106@maxwell.research.nokia.com> <1311082688-16185-2-git-send-email-sakari.ailus@iki.fi> In-Reply-To: <1311082688-16185-2-git-send-email-sakari.ailus@iki.fi> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201107261352.44287.hverkuil@xs4all.nl> Sender: linux-media-owner@vger.kernel.org List-ID: On Tuesday, July 19, 2011 15:38:07 Sakari Ailus wrote: > Define a frame start event to tell user space when the reception of a frame > starts. > > Signed-off-by: Sakari Ailus > --- > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 26 ++++++++++++++++++++ > .../DocBook/media/v4l/vidioc-subscribe-event.xml | 18 +++++++++++++ > include/linux/videodev2.h | 12 +++++++-- > 3 files changed, 53 insertions(+), 3 deletions(-) > > diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > index 5200b68..d2cb8db 100644 > --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > @@ -88,6 +88,12 @@ > > > > + &v4l2-event-frame-sync; > + frame > + Event data for event V4L2_EVENT_FRAME_START. The name of the struct and the event are not in sync (pardon the expression :-) ). Both should either be named FRAME_SYNC or FRAME_START. > + > + > + > __u8 > data[64] > Event data. Defined by the event type. The union > @@ -220,6 +226,26 @@ > > > > + > + struct <structname>v4l2_event_frame_sync</structname> > + > + &cs-str; > + > + > + __u32 > + buffer_sequence > + > + The sequence number of the buffer to be handled next or > + currently handled by the driver. So this refers to the sequence field in struct v4l2_buffer? Assuming it is, then you definitely need to refer to that struct. And to answer question 2 in the RFC: buffer_sequence is specific to this event, and so belongs to a v4l2_event_frame_sync struct. BTW, using 'id' to specify a specific line in the future seems reasonable to me. Initially id is just set to 0, meaning the start of the frame. Regards, Hans > + > + > + > + > +
> + > + &v4l2-event-frame-sync; is associated with > + V4L2_EVENT_FRAME_START event. > + > > Changes > > diff --git a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > index 275be96..7ec42bb 100644 > --- a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > +++ b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > @@ -139,6 +139,24 @@ > > > > + V4L2_EVENT_FRAME_START > + 4 > + > + Triggered immediately when the reception of a > + frame has begun. This event has a > + &v4l2-event-frame-sync; associated with it. > + > + A driver will only generate this event when the > + hardware can generate it. This might not be the case > + e.g. when the hardware has no DMA buffer to write the > + image data to. In such cases the > + buffer_sequence field in > + &v4l2-event-frame-sync; will not be incremented either. > + This causes two consecutive buffer sequence numbers to > + have n times frame interval in between them. > + > + > + > V4L2_EVENT_PRIVATE_START > 0x08000000 > Base event number for driver-private events. > diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h > index fca24cc..4265102 100644 > --- a/include/linux/videodev2.h > +++ b/include/linux/videodev2.h > @@ -2006,6 +2006,7 @@ struct v4l2_streamparm { > #define V4L2_EVENT_VSYNC 1 > #define V4L2_EVENT_EOS 2 > #define V4L2_EVENT_CTRL 3 > +#define V4L2_EVENT_FRAME_START 4 > #define V4L2_EVENT_PRIVATE_START 0x08000000 > > /* Payload for V4L2_EVENT_VSYNC */ > @@ -2032,12 +2033,17 @@ struct v4l2_event_ctrl { > __s32 default_value; > }; > > +struct v4l2_event_frame_sync { > + __u32 buffer_sequence; > +}; > + > struct v4l2_event { > __u32 type; > union { > - struct v4l2_event_vsync vsync; > - struct v4l2_event_ctrl ctrl; > - __u8 data[64]; > + struct v4l2_event_vsync vsync; > + struct v4l2_event_ctrl ctrl; > + struct v4l2_event_frame_sync frame_sync; > + __u8 data[64]; > } u; > __u32 pending; > __u32 sequence; > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp.nokia.com ([147.243.1.48]:36386 "EHLO mgw-sa02.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750886Ab1GYJE1 (ORCPT ); Mon, 25 Jul 2011 05:04:27 -0400 Message-ID: <4E2D3183.2040303@iki.fi> Date: Mon, 25 Jul 2011 12:04:03 +0300 From: Sakari Ailus MIME-Version: 1.0 To: Sylwester Nawrocki CC: Sakari Ailus , "linux-media@vger.kernel.org" Subject: Re: [RFC 0/3] Frame synchronisation events and support for them in the OMAP 3 ISP driver References: <4E2588AD.4070106@maxwell.research.nokia.com> <4E284C71.7050806@gmail.com> <4E29536A.3010003@maxwell.research.nokia.com> <4E2987CC.5010303@gmail.com> In-Reply-To: <4E2987CC.5010303@gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-media-owner@vger.kernel.org List-ID: Sylwester Nawrocki wrote: > Hi, Hi Sylwester, Thanks for your comments. > On 07/22/2011 12:39 PM, Sakari Ailus wrote: > ... >>> On 07/19/2011 03:37 PM, Sakari Ailus wrote: >>>> Hi all, >>>> >>>> The OMAP 3 ISP driver implements an HS_VS event which is triggered when >>>> the reception of a frame begins. This functionality is very, very likely >>>> not specific to OMAP 3 ISP so it should be standardised. >>>> >>>> I have a few patches to do that. Additionally the next expected buffer >>>> sequence number is provided with the event, unlike earlier. >>>> >>>> There are a few open questions, however, and this is why I'm sending the >>>> set as RFC. >>>> >>>> >>>> 1) Other frame synchronisation events. The CCDC block in the OMAP 3 ISP >>>> is able to trigger interrupts at two chosen lines of the image. These >>>> naturally can be translated to events. The driver uses both of them >>>> internally at specific points of the frame. Nevertheless, there might be >>>> some use for these in user space. Other hardware might implement a >>>> number of these which wouldn't be used by the driver itself, but I don't >>>> know of that at the moment. On the other hand high resolution timers are >>>> also available in user space, so doing timing based on ISP provided >>>> events is not quite as important as before --- as long as there's one >>>> frame based event produced at a known time, such as V4L2_EVENT_FRAME_START. >>> >>> I'm curious, have you perhaps tried to measure latency of such up calls >>> to a user space process? I mean this is going to be a real time stuff, >>> with HSYNC periods of 50 us order. Could a user space thread be receiving >>> such periodic events reliably ? From my experience I doubt this can work >>> reliably outside of an interrupt handler even with high priority real time >>> threads. >>> >>> V4L2_EVENT_FRAME_START event seems OK, but HSYNC events in user space >>> sound rather tricky to me :-) >> >> I think the user space could be interested in just one or two of these >> per frame, not for every line. But how to subscribe them --- if they are >> needed? > > Yes, that was my understanding. But still we need much better accuracy than > in case of VSYNC/FRAME_START. It seems really hard to guarantee in Linux > that a specific line event will be received in user space when that actual > line is transmitted. There is much better chance that a FRAME_START event There's nothing that a driver may do itself to make the user space receive the event in time, especially if the user space process is not running on real-time priority. This is a completely separate issue. The timestamp is very important here, more important than guaranteed timely event delivery. > is received during the time when a frame that triggered it is being processed. This is what happens in practice and I do not consider it as an issue. Any configuration which is related to the processing of that frame must have been given to the driver well beforehand. > And VSYNC signals last over several horizontal lines (if not tens or thousands) > which makes it easier to receive an event when a VSYNC pulse/period hasn't yet > expired. I'm not sure I can follow you here. It's the responsibility of the driver / hardware to provide the event. If it can't, then it shouldn't allow subscribing it. >> >> Perhaps it'd be better to start with just one and add more once necessary? > > Maybe we could accommodate the struct v4l2_event_subscription::id field for > a horizontal line number, with a special bit indicating any one ? That's an interesting idea. The subscription problem for line events still remains. > But again I'm not convinced to horizontal line events :) There might be > situations when even interrupt handlers are to slow for this. Events may be delivered when there's an interrupt source which can generate an interrupt per event. This is no more complex than generating one in the beginning of the frame. Nevertheless, I don't either think these would be very useful, so I think they could be ignored, at least for the time being. >> >>> Also HS_VS looks a bit more descriptive than FRAME_START for me. >> >> HS_VS doesn't really tell which one it is (horizontal or vertical), and >> we already have a VSYNC event but it's used for a different purpose. >> HS_VS is specific to the CCDC block and doesn't have that meaning in >> context of serial interfaces. >> >> This is why I proposed FRAME_START. > > OK, initially I thought it was that HS_VS means a moment when vertical > and horizontal sync (blanking?) signals go off and thus video data > of the first line in a frame is started to be transmitted. > I agree that HS_VS isn't that relevant in the CSI terminology. The OMAP 3 ISP may produce interrupts for either vertical or horizontal sync events but the driver only uses it for vertical sync. I guess there's little or no imaginable use for HS_VS to generate an interrupt per every line. VD_INT interrupts Laurent mentioned are used to generate an interrupt on a specific line of the frame. >>> But unfortunately I can't come up with a better name, e.g. something like >>> V4L2_EVENT_FRAME_AV_START - frame active video start. Just in case in >>> future there are more specific events added. >> >> What additional information would AV add which isn't evident from >> FRAME_START? > > Given different formats of frame headers across the standards (ITU-R BT.656, > MIPI-CSI2 and the like) it might be not so obvious at which moment the > FRAME_START event is to be triggered. But not being too paranoid I guess > we're perfectly fine with VSYNC and FRAME_START events. It should be trigered when the first bit of data is being received. What about FRAME_DATA_START? Cheers, -- Sakari Ailus sakari.ailus@iki.fi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp-68.nebula.fi ([83.145.220.68]:53559 "EHLO smtp-68.nebula.fi" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751456Ab1GZOPu (ORCPT ); Tue, 26 Jul 2011 10:15:50 -0400 Date: Tue, 26 Jul 2011 17:15:46 +0300 From: Sakari Ailus To: Hans Verkuil Cc: linux-media@vger.kernel.org Subject: Re: [RFC 2/3] v4l: events: Define frame start event Message-ID: <20110726141546.GC32629@valkosipuli.localdomain> References: <4E2588AD.4070106@maxwell.research.nokia.com> <201107261352.44287.hverkuil@xs4all.nl> <20110726135125.GB32629@valkosipuli.localdomain> <201107261559.38397.hverkuil@xs4all.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201107261559.38397.hverkuil@xs4all.nl> Sender: linux-media-owner@vger.kernel.org List-ID: Thanks for your comments, Hans! On Tue, Jul 26, 2011 at 03:59:38PM +0200, Hans Verkuil wrote: > On Tuesday, July 26, 2011 15:51:26 Sakari Ailus wrote: > > On Tue, Jul 26, 2011 at 01:52:44PM +0200, Hans Verkuil wrote: > > > On Tuesday, July 19, 2011 15:38:07 Sakari Ailus wrote: > > > > Define a frame start event to tell user space when the reception of a frame > > > > starts. > > > > > > > > Signed-off-by: Sakari Ailus > > > > --- > > > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 26 ++++++++++++++++++++ > > > > .../DocBook/media/v4l/vidioc-subscribe-event.xml | 18 +++++++++++++ > > > > include/linux/videodev2.h | 12 +++++++-- > > > > 3 files changed, 53 insertions(+), 3 deletions(-) > > > > > > > > diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > > > index 5200b68..d2cb8db 100644 > > > > --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > > > +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > > > @@ -88,6 +88,12 @@ > > > > > > > > > > > > > > > > + &v4l2-event-frame-sync; > > > > + frame > > > > + Event data for event V4L2_EVENT_FRAME_START. > > > > > > The name of the struct and the event are not in sync (pardon the expression :-) ). > > > > > > Both should either be named FRAME_SYNC or FRAME_START. > > > > Should they be in sync? FRAME_START event is for frame start, not for other > > purposes. > > Ah, you expect other events to reuse the same payload struct. I missed > that part. > > > The buffer sequence number, however, could be used by other events, too. > > This is directly related to the question of how to subscribe line-based > > events. Albeit whether they are really ever needed is another question. > > > > Getting _one_ event giving frame synchronisation timestamps is important, > > however; that's also why I sent the RFC. > > > > What I might do now is that we define a FRAME_SYNC (or FRAME_START) event > > and specify the id == 0 always, and worry about the rest later on. It is > > quite possible that line based events will never be needed. > > I would go for that. Ack. > > If they are, then we must also specify how to subscribe them. > > Using 'id' as the line number seems sensible to me, but I would definitely > leave that part out for now. I am not convinced it is possible to use that > reliably in any case due to the difficult timing requirements. Ah, I momentarily forgot how this works on controls. :-) I think FRAME_SYNC is better name since it's reusable in the future as well so I choose that one and resend the set. Cheers, -- Sakari Ailus sakari.ailus@iki.fi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp-68.nebula.fi ([83.145.220.68]:45585 "EHLO smtp-68.nebula.fi" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751211Ab1GZNva (ORCPT ); Tue, 26 Jul 2011 09:51:30 -0400 Date: Tue, 26 Jul 2011 16:51:26 +0300 From: Sakari Ailus To: Hans Verkuil Cc: linux-media@vger.kernel.org Subject: Re: [RFC 2/3] v4l: events: Define frame start event Message-ID: <20110726135125.GB32629@valkosipuli.localdomain> References: <4E2588AD.4070106@maxwell.research.nokia.com> <1311082688-16185-2-git-send-email-sakari.ailus@iki.fi> <201107261352.44287.hverkuil@xs4all.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201107261352.44287.hverkuil@xs4all.nl> Sender: linux-media-owner@vger.kernel.org List-ID: On Tue, Jul 26, 2011 at 01:52:44PM +0200, Hans Verkuil wrote: > On Tuesday, July 19, 2011 15:38:07 Sakari Ailus wrote: > > Define a frame start event to tell user space when the reception of a frame > > starts. > > > > Signed-off-by: Sakari Ailus > > --- > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 26 ++++++++++++++++++++ > > .../DocBook/media/v4l/vidioc-subscribe-event.xml | 18 +++++++++++++ > > include/linux/videodev2.h | 12 +++++++-- > > 3 files changed, 53 insertions(+), 3 deletions(-) > > > > diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > index 5200b68..d2cb8db 100644 > > --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > @@ -88,6 +88,12 @@ > > > > > > > > + &v4l2-event-frame-sync; > > + frame > > + Event data for event V4L2_EVENT_FRAME_START. > > The name of the struct and the event are not in sync (pardon the expression :-) ). > > Both should either be named FRAME_SYNC or FRAME_START. Should they be in sync? FRAME_START event is for frame start, not for other purposes. The buffer sequence number, however, could be used by other events, too. This is directly related to the question of how to subscribe line-based events. Albeit whether they are really ever needed is another question. Getting _one_ event giving frame synchronisation timestamps is important, however; that's also why I sent the RFC. What I might do now is that we define a FRAME_SYNC (or FRAME_START) event and specify the id == 0 always, and worry about the rest later on. It is quite possible that line based events will never be needed. If they are, then we must also specify how to subscribe them. > > + > > + > > + > > __u8 > > data[64] > > Event data. Defined by the event type. The union > > @@ -220,6 +226,26 @@ > > > >
> > > > + > > + struct <structname>v4l2_event_frame_sync</structname> > > + > > + &cs-str; > > + > > + > > + __u32 > > + buffer_sequence > > + > > + The sequence number of the buffer to be handled next or > > + currently handled by the driver. > > So this refers to the sequence field in struct v4l2_buffer? Assuming it is, > then you definitely need to refer to that struct. Yes, it does. > And to answer question 2 in the RFC: buffer_sequence is specific to this > event, and so belongs to a v4l2_event_frame_sync struct. Agreed. > BTW, using 'id' to specify a specific line in the future seems reasonable to me. > Initially id is just set to 0, meaning the start of the frame. > > Regards, > > Hans > > > + > > + > > + > > + > > +
> > + > > + &v4l2-event-frame-sync; is associated with > > + V4L2_EVENT_FRAME_START event. > > + > > > > Changes > > > > diff --git a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > > index 275be96..7ec42bb 100644 > > --- a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > > +++ b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml > > @@ -139,6 +139,24 @@ > > > > > > > > + V4L2_EVENT_FRAME_START > > + 4 > > + > > + Triggered immediately when the reception of a > > + frame has begun. This event has a > > + &v4l2-event-frame-sync; associated with it. > > + > > + A driver will only generate this event when the > > + hardware can generate it. This might not be the case > > + e.g. when the hardware has no DMA buffer to write the > > + image data to. In such cases the > > + buffer_sequence field in > > + &v4l2-event-frame-sync; will not be incremented either. > > + This causes two consecutive buffer sequence numbers to > > + have n times frame interval in between them. > > + > > + > > + > > V4L2_EVENT_PRIVATE_START > > 0x08000000 > > Base event number for driver-private events. > > diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h > > index fca24cc..4265102 100644 > > --- a/include/linux/videodev2.h > > +++ b/include/linux/videodev2.h > > @@ -2006,6 +2006,7 @@ struct v4l2_streamparm { > > #define V4L2_EVENT_VSYNC 1 > > #define V4L2_EVENT_EOS 2 > > #define V4L2_EVENT_CTRL 3 > > +#define V4L2_EVENT_FRAME_START 4 > > #define V4L2_EVENT_PRIVATE_START 0x08000000 > > > > /* Payload for V4L2_EVENT_VSYNC */ > > @@ -2032,12 +2033,17 @@ struct v4l2_event_ctrl { > > __s32 default_value; > > }; > > > > +struct v4l2_event_frame_sync { > > + __u32 buffer_sequence; > > +}; > > + > > struct v4l2_event { > > __u32 type; > > union { > > - struct v4l2_event_vsync vsync; > > - struct v4l2_event_ctrl ctrl; > > - __u8 data[64]; > > + struct v4l2_event_vsync vsync; > > + struct v4l2_event_ctrl ctrl; > > + struct v4l2_event_frame_sync frame_sync; > > + __u8 data[64]; > > } u; > > __u32 pending; > > __u32 sequence; > > -- Sakari Ailus sakari.ailus@iki.fi From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp.nokia.com ([147.243.1.48]:18567 "EHLO mgw-sa02.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751037Ab1GSNiK (ORCPT ); Tue, 19 Jul 2011 09:38:10 -0400 Received: from maxwell.research.nokia.com (maxwell.research.nokia.com [172.21.50.162]) by mgw-sa02.nokia.com (Switch-3.4.4/Switch-3.4.3) with ESMTP id p6JDc8er027793 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 19 Jul 2011 16:38:08 +0300 Received: from kaali.localdomain (kaali.localdomain [192.168.239.7]) by maxwell.research.nokia.com (Postfix) with ESMTPS id AE2C438639A for ; Tue, 19 Jul 2011 16:38:08 +0300 (EEST) Received: from sailus by kaali.localdomain with local (Exim 4.72) (envelope-from ) id 1QjAUy-0004E2-Ka for linux-media@vger.kernel.org; Tue, 19 Jul 2011 16:38:08 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Subject: [RFC 3/3] omap3isp: ccdc: Make frame start event generic Date: Tue, 19 Jul 2011 16:38:08 +0300 Message-Id: <1311082688-16185-3-git-send-email-sakari.ailus@iki.fi> In-Reply-To: <4E2588AD.4070106@maxwell.research.nokia.com> References: <4E2588AD.4070106@maxwell.research.nokia.com> Sender: linux-media-owner@vger.kernel.org List-ID: The ccdc block in the omap3isp produces frame start events. These events were previously specific to the omap3isp. Make them generic. Also add sequence number to the frame. This is stored to the id field. Signed-off-by: Sakari Ailus --- Documentation/video4linux/omap3isp.txt | 9 +++++---- drivers/media/video/omap3isp/ispccdc.c | 7 +++++-- include/linux/omap3isp.h | 2 -- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/Documentation/video4linux/omap3isp.txt b/Documentation/video4linux/omap3isp.txt index 69be2c7..84f24cf 100644 --- a/Documentation/video4linux/omap3isp.txt +++ b/Documentation/video4linux/omap3isp.txt @@ -70,10 +70,11 @@ Events The OMAP 3 ISP driver does support the V4L2 event interface on CCDC and statistics (AEWB, AF and histogram) subdevs. -The CCDC subdev produces V4L2_EVENT_OMAP3ISP_HS_VS type event on HS_VS -interrupt which is used to signal frame start. The event is triggered exactly -when the reception of the first line of the frame starts in the CCDC module. -The event can be subscribed on the CCDC subdev. +The CCDC subdev produces V4L2_EVENT_FRAME_START type event on HS_VS +interrupt which is used to signal frame start. Earlier version of this +driver used V4L2_EVENT_OMAP3ISP_HS_VS for this purpose. The event is +triggered exactly when the reception of the first line of the frame starts +in the CCDC module. The event can be subscribed on the CCDC subdev. (When using parallel interface one must pay account to correct configuration of the VS signal polarity. This is automatically correct when using the serial diff --git a/drivers/media/video/omap3isp/ispccdc.c b/drivers/media/video/omap3isp/ispccdc.c index 6766247..61c9e3d 100644 --- a/drivers/media/video/omap3isp/ispccdc.c +++ b/drivers/media/video/omap3isp/ispccdc.c @@ -1402,11 +1402,14 @@ static int __ccdc_handle_stopping(struct isp_ccdc_device *ccdc, u32 event) static void ccdc_hs_vs_isr(struct isp_ccdc_device *ccdc) { + struct isp_pipeline *pipe = + to_isp_pipeline(&ccdc->video_out.video.entity); struct video_device *vdev = &ccdc->subdev.devnode; struct v4l2_event event; memset(&event, 0, sizeof(event)); - event.type = V4L2_EVENT_OMAP3ISP_HS_VS; + event.type = V4L2_EVENT_FRAME_START; + event.u.frame_sync.buffer_sequence = atomic_read(&pipe->frame_number); v4l2_event_queue(vdev, &event); } @@ -1688,7 +1691,7 @@ static long ccdc_ioctl(struct v4l2_subdev *sd, unsigned int cmd, void *arg) static int ccdc_subscribe_event(struct v4l2_subdev *sd, struct v4l2_fh *fh, struct v4l2_event_subscription *sub) { - if (sub->type != V4L2_EVENT_OMAP3ISP_HS_VS) + if (sub->type != V4L2_EVENT_FRAME_START) return -EINVAL; return v4l2_event_subscribe(fh, sub, OMAP3ISP_CCDC_NEVENTS); diff --git a/include/linux/omap3isp.h b/include/linux/omap3isp.h index b6111f8..c73a34c 100644 --- a/include/linux/omap3isp.h +++ b/include/linux/omap3isp.h @@ -62,14 +62,12 @@ * V4L2_EVENT_OMAP3ISP_AEWB: AEWB statistics data ready * V4L2_EVENT_OMAP3ISP_AF: AF statistics data ready * V4L2_EVENT_OMAP3ISP_HIST: Histogram statistics data ready - * V4L2_EVENT_OMAP3ISP_HS_VS: Horizontal/vertical synchronization detected */ #define V4L2_EVENT_OMAP3ISP_CLASS (V4L2_EVENT_PRIVATE_START | 0x100) #define V4L2_EVENT_OMAP3ISP_AEWB (V4L2_EVENT_OMAP3ISP_CLASS | 0x1) #define V4L2_EVENT_OMAP3ISP_AF (V4L2_EVENT_OMAP3ISP_CLASS | 0x2) #define V4L2_EVENT_OMAP3ISP_HIST (V4L2_EVENT_OMAP3ISP_CLASS | 0x3) -#define V4L2_EVENT_OMAP3ISP_HS_VS (V4L2_EVENT_OMAP3ISP_CLASS | 0x4) struct omap3isp_stat_event_status { __u32 frame_number; -- 1.7.2.5 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp.nokia.com ([147.243.1.47]:42995 "EHLO mgw-sa01.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750933Ab1GSNhx (ORCPT ); Tue, 19 Jul 2011 09:37:53 -0400 Received: from maxwell.research.nokia.com (maxwell.research.nokia.com [172.21.50.162]) by mgw-sa01.nokia.com (Switch-3.4.4/Switch-3.4.3) with ESMTP id p6JDbnCg026800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 19 Jul 2011 16:37:50 +0300 Received: from [127.0.0.1] (localhost [127.0.0.1]) by maxwell.research.nokia.com (Postfix) with ESMTP id A1CF537FCDA for ; Tue, 19 Jul 2011 16:37:49 +0300 (EEST) Message-ID: <4E2588AD.4070106@maxwell.research.nokia.com> Date: Tue, 19 Jul 2011 16:37:49 +0300 From: Sakari Ailus MIME-Version: 1.0 To: "linux-media@vger.kernel.org" Subject: [RFC 0/3] Frame synchronisation events and support for them in the OMAP 3 ISP driver Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-media-owner@vger.kernel.org List-ID: Hi all, The OMAP 3 ISP driver implements an HS_VS event which is triggered when the reception of a frame begins. This functionality is very, very likely not specific to OMAP 3 ISP so it should be standardised. I have a few patches to do that. Additionally the next expected buffer sequence number is provided with the event, unlike earlier. There are a few open questions, however, and this is why I'm sending the set as RFC. 1) Other frame synchronisation events. The CCDC block in the OMAP 3 ISP is able to trigger interrupts at two chosen lines of the image. These naturally can be translated to events. The driver uses both of them internally at specific points of the frame. Nevertheless, there might be some use for these in user space. Other hardware might implement a number of these which wouldn't be used by the driver itself, but I don't know of that at the moment. On the other hand high resolution timers are also available in user space, so doing timing based on ISP provided events is not quite as important as before --- as long as there's one frame based event produced at a known time, such as V4L2_EVENT_FRAME_START. Frame end events may be produced as well. This is not exactly the same as just dequeueing the buffer at video node since the hardware may be able to produce events even in cases there are no buffers and if the very hardware block that processes the frame is not outputting it to memory, handling by further blocks takes more time, and thus delays the finishing of the buffer from the driver's queue. This is the reason why the name of the struct related to the event is v4l2_event_frame_sync rather than v4l2_event_frame_start. 2) Buffer sequence number location in the struct v4l2_event. the patches create a new structure called v4l2_event_frame_sync which contains just one field, buffer_sequence. Should buffer_sequence be part of this struct, or should it be part of v4l2_event directly, as the id field? Both buffer_sequence and id refer to another rather widely used concept in V4L2. Besides this, the first patch in the series moves the documentation of structs inside v4l2_event to VIDIOC_DQEVENT documentation. I think it belongs there rather than to VIDIOC_SUBSCRIBE_EVENT, since that's not where they are being used. -- Sakari Ailus sakari.ailus@maxwell.research.nokia.com From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp.nokia.com ([147.243.1.48]:19111 "EHLO mgw-sa02.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751191Ab1GVKjo (ORCPT ); Fri, 22 Jul 2011 06:39:44 -0400 Message-ID: <4E29536A.3010003@maxwell.research.nokia.com> Date: Fri, 22 Jul 2011 13:39:38 +0300 From: Sakari Ailus MIME-Version: 1.0 To: Sylwester Nawrocki CC: "linux-media@vger.kernel.org" Subject: Re: [RFC 0/3] Frame synchronisation events and support for them in the OMAP 3 ISP driver References: <4E2588AD.4070106@maxwell.research.nokia.com> <4E284C71.7050806@gmail.com> In-Reply-To: <4E284C71.7050806@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Sender: linux-media-owner@vger.kernel.org List-ID: Sylwester Nawrocki wrote: > Hi Sakari, Hi Sylwester, Thanks for the comments. > On 07/19/2011 03:37 PM, Sakari Ailus wrote: >> Hi all, >> >> The OMAP 3 ISP driver implements an HS_VS event which is triggered when >> the reception of a frame begins. This functionality is very, very likely >> not specific to OMAP 3 ISP so it should be standardised. >> >> I have a few patches to do that. Additionally the next expected buffer >> sequence number is provided with the event, unlike earlier. >> >> There are a few open questions, however, and this is why I'm sending the >> set as RFC. >> >> >> 1) Other frame synchronisation events. The CCDC block in the OMAP 3 ISP >> is able to trigger interrupts at two chosen lines of the image. These >> naturally can be translated to events. The driver uses both of them >> internally at specific points of the frame. Nevertheless, there might be >> some use for these in user space. Other hardware might implement a >> number of these which wouldn't be used by the driver itself, but I don't >> know of that at the moment. On the other hand high resolution timers are >> also available in user space, so doing timing based on ISP provided >> events is not quite as important as before --- as long as there's one >> frame based event produced at a known time, such as V4L2_EVENT_FRAME_START. > > I'm curious, have you perhaps tried to measure latency of such up calls > to a user space process? I mean this is going to be a real time stuff, > with HSYNC periods of 50 us order. Could a user space thread be receiving > such periodic events reliably ? From my experience I doubt this can work > reliably outside of an interrupt handler even with high priority real time > threads. > > V4L2_EVENT_FRAME_START event seems OK, but HSYNC events in user space > sound rather tricky to me :-) I think the user space could be interested in just one or two of these per frame, not for every line. But how to subscribe them --- if they are needed? Perhaps it'd be better to start with just one and add more once necessary? > Also HS_VS looks a bit more descriptive than FRAME_START for me. HS_VS doesn't really tell which one it is (horizontal or vertical), and we already have a VSYNC event but it's used for a different purpose. HS_VS is specific to the CCDC block and doesn't have that meaning in context of serial interfaces. This is why I proposed FRAME_START. > But unfortunately I can't come up with a better name, e.g. something like > V4L2_EVENT_FRAME_AV_START - frame active video start. Just in case in > future there are more specific events added. What additional information would AV add which isn't evident from FRAME_START? I admit that there could be differencies in terminology used in this area; terms that are meaningful to some might not be to others, or they could mean different things to them. Regards, -- Sakari Ailus sakari.ailus@maxwell.research.nokia.com From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp.nokia.com ([147.243.1.47]:43210 "EHLO mgw-sa01.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751024Ab1GSNiK (ORCPT ); Tue, 19 Jul 2011 09:38:10 -0400 Received: from maxwell.research.nokia.com (maxwell.research.nokia.com [172.21.50.162]) by mgw-sa01.nokia.com (Switch-3.4.4/Switch-3.4.3) with ESMTP id p6JDc8J9027233 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 19 Jul 2011 16:38:08 +0300 Received: from kaali.localdomain (kaali.localdomain [192.168.239.7]) by maxwell.research.nokia.com (Postfix) with ESMTPS id 9E27D37FCDA for ; Tue, 19 Jul 2011 16:38:08 +0300 (EEST) Received: from sailus by kaali.localdomain with local (Exim 4.72) (envelope-from ) id 1QjAUy-0004Dx-9o for linux-media@vger.kernel.org; Tue, 19 Jul 2011 16:38:08 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Subject: [RFC 1/3] v4l: Move event documentation from SUBSCRIBE_EVENT to DQEVENT Date: Tue, 19 Jul 2011 16:38:06 +0300 Message-Id: <1311082688-16185-1-git-send-email-sakari.ailus@iki.fi> In-Reply-To: <4E2588AD.4070106@maxwell.research.nokia.com> References: <4E2588AD.4070106@maxwell.research.nokia.com> Sender: linux-media-owner@vger.kernel.org List-ID: Move documentation of structures used in DQEVENT from SUBSCRIBE_EVENT to DQEVENT. Signed-off-by: Sakari Ailus --- Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 107 ++++++++++++++++++++ .../DocBook/media/v4l/vidioc-subscribe-event.xml | 107 -------------------- 2 files changed, 107 insertions(+), 107 deletions(-) diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml index 7769642..5200b68 100644 --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml @@ -135,6 +135,113 @@
+ + struct <structname>v4l2_event_vsync</structname> + + &cs-str; + + + __u8 + field + The upcoming field. See &v4l2-field;. + + + +
+ + + struct <structname>v4l2_event_ctrl</structname> + + &cs-str; + + + __u32 + changes + + A bitmask that tells what has changed. See . + + + __u32 + type + + The type of the control. See &v4l2-ctrl-type;. + + + union (anonymous) + + + + + + + __s32 + value + The 32-bit value of the control for 32-bit control types. + This is 0 for string controls since the value of a string + cannot be passed using &VIDIOC-DQEVENT;. + + + + __s64 + value64 + The 64-bit value of the control for 64-bit control types. + + + __u32 + flags + + The control flags. See . + + + __s32 + minimum + + The minimum value of the control. See &v4l2-queryctrl;. + + + __s32 + maximum + + The maximum value of the control. See &v4l2-queryctrl;. + + + __s32 + step + + The step value of the control. See &v4l2-queryctrl;. + + + __s32 + default_value + + The default value value of the control. See &v4l2-queryctrl;. + + + +
+ + + Changes + + &cs-def; + + + V4L2_EVENT_CTRL_CH_VALUE + 0x0001 + This control event was triggered because the value of the control + changed. Special case: if a button control is pressed, then this + event is sent as well, even though there is not explicit value + associated with a button control. + + + V4L2_EVENT_CTRL_CH_FLAGS + 0x0002 + This control event was triggered because the control flags + changed. + + + +
&return-value; diff --git a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml index 69c0d8a..275be96 100644 --- a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml +++ b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml @@ -183,113 +183,6 @@ - - struct <structname>v4l2_event_vsync</structname> - - &cs-str; - - - __u8 - field - The upcoming field. See &v4l2-field;. - - - -
- - - struct <structname>v4l2_event_ctrl</structname> - - &cs-str; - - - __u32 - changes - - A bitmask that tells what has changed. See . - - - __u32 - type - - The type of the control. See &v4l2-ctrl-type;. - - - union (anonymous) - - - - - - - __s32 - value - The 32-bit value of the control for 32-bit control types. - This is 0 for string controls since the value of a string - cannot be passed using &VIDIOC-DQEVENT;. - - - - __s64 - value64 - The 64-bit value of the control for 64-bit control types. - - - __u32 - flags - - The control flags. See . - - - __s32 - minimum - - The minimum value of the control. See &v4l2-queryctrl;. - - - __s32 - maximum - - The maximum value of the control. See &v4l2-queryctrl;. - - - __s32 - step - - The step value of the control. See &v4l2-queryctrl;. - - - __s32 - default_value - - The default value value of the control. See &v4l2-queryctrl;. - - - -
- - - Changes - - &cs-def; - - - V4L2_EVENT_CTRL_CH_VALUE - 0x0001 - This control event was triggered because the value of the control - changed. Special case: if a button control is pressed, then this - event is sent as well, even though there is not explicit value - associated with a button control. - - - V4L2_EVENT_CTRL_CH_FLAGS - 0x0002 - This control event was triggered because the control flags - changed. - - - -
&return-value; -- 1.7.2.5 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp.nokia.com ([147.243.128.24]:34098 "EHLO mgw-da01.nokia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751105Ab1GSNiL (ORCPT ); Tue, 19 Jul 2011 09:38:11 -0400 Received: from maxwell.research.nokia.com (maxwell.research.nokia.com [172.21.50.162]) by mgw-da01.nokia.com (Switch-3.4.4/Switch-3.4.3) with ESMTP id p6JDc8Af032138 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 19 Jul 2011 16:38:10 +0300 Received: from kaali.localdomain (kaali.localdomain [192.168.239.7]) by maxwell.research.nokia.com (Postfix) with ESMTPS id A988738638F for ; Tue, 19 Jul 2011 16:38:08 +0300 (EEST) Received: from sailus by kaali.localdomain with local (Exim 4.72) (envelope-from ) id 1QjAUy-0004Dz-EP for linux-media@vger.kernel.org; Tue, 19 Jul 2011 16:38:08 +0300 From: Sakari Ailus To: linux-media@vger.kernel.org Subject: [RFC 2/3] v4l: events: Define frame start event Date: Tue, 19 Jul 2011 16:38:07 +0300 Message-Id: <1311082688-16185-2-git-send-email-sakari.ailus@iki.fi> In-Reply-To: <4E2588AD.4070106@maxwell.research.nokia.com> References: <4E2588AD.4070106@maxwell.research.nokia.com> Sender: linux-media-owner@vger.kernel.org List-ID: Define a frame start event to tell user space when the reception of a frame starts. Signed-off-by: Sakari Ailus --- Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 26 ++++++++++++++++++++ .../DocBook/media/v4l/vidioc-subscribe-event.xml | 18 +++++++++++++ include/linux/videodev2.h | 12 +++++++-- 3 files changed, 53 insertions(+), 3 deletions(-) diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml index 5200b68..d2cb8db 100644 --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml @@ -88,6 +88,12 @@ + &v4l2-event-frame-sync; + frame + Event data for event V4L2_EVENT_FRAME_START. + + + __u8 data[64] Event data. Defined by the event type. The union @@ -220,6 +226,26 @@ + + struct <structname>v4l2_event_frame_sync</structname> + + &cs-str; + + + __u32 + buffer_sequence + + The sequence number of the buffer to be handled next or + currently handled by the driver. + + + + +
+ + &v4l2-event-frame-sync; is associated with + V4L2_EVENT_FRAME_START event. + Changes diff --git a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml index 275be96..7ec42bb 100644 --- a/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml +++ b/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml @@ -139,6 +139,24 @@ + V4L2_EVENT_FRAME_START + 4 + + Triggered immediately when the reception of a + frame has begun. This event has a + &v4l2-event-frame-sync; associated with it. + + A driver will only generate this event when the + hardware can generate it. This might not be the case + e.g. when the hardware has no DMA buffer to write the + image data to. In such cases the + buffer_sequence field in + &v4l2-event-frame-sync; will not be incremented either. + This causes two consecutive buffer sequence numbers to + have n times frame interval in between them. + + + V4L2_EVENT_PRIVATE_START 0x08000000 Base event number for driver-private events. diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index fca24cc..4265102 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -2006,6 +2006,7 @@ struct v4l2_streamparm { #define V4L2_EVENT_VSYNC 1 #define V4L2_EVENT_EOS 2 #define V4L2_EVENT_CTRL 3 +#define V4L2_EVENT_FRAME_START 4 #define V4L2_EVENT_PRIVATE_START 0x08000000 /* Payload for V4L2_EVENT_VSYNC */ @@ -2032,12 +2033,17 @@ struct v4l2_event_ctrl { __s32 default_value; }; +struct v4l2_event_frame_sync { + __u32 buffer_sequence; +}; + struct v4l2_event { __u32 type; union { - struct v4l2_event_vsync vsync; - struct v4l2_event_ctrl ctrl; - __u8 data[64]; + struct v4l2_event_vsync vsync; + struct v4l2_event_ctrl ctrl; + struct v4l2_event_frame_sync frame_sync; + __u8 data[64]; } u; __u32 pending; __u32 sequence; -- 1.7.2.5 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-fx0-f52.google.com ([209.85.161.52]:54451 "EHLO mail-fx0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753832Ab1GVOXP (ORCPT ); Fri, 22 Jul 2011 10:23:15 -0400 Received: by fxd18 with SMTP id 18so5358720fxd.11 for ; Fri, 22 Jul 2011 07:23:13 -0700 (PDT) Message-ID: <4E2987CC.5010303@gmail.com> Date: Fri, 22 Jul 2011 16:23:08 +0200 From: Sylwester Nawrocki MIME-Version: 1.0 To: Sakari Ailus , "linux-media@vger.kernel.org" Subject: Re: [RFC 0/3] Frame synchronisation events and support for them in the OMAP 3 ISP driver References: <4E2588AD.4070106@maxwell.research.nokia.com> <4E284C71.7050806@gmail.com> <4E29536A.3010003@maxwell.research.nokia.com> In-Reply-To: <4E29536A.3010003@maxwell.research.nokia.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Sender: linux-media-owner@vger.kernel.org List-ID: Hi, On 07/22/2011 12:39 PM, Sakari Ailus wrote: ... >> On 07/19/2011 03:37 PM, Sakari Ailus wrote: >>> Hi all, >>> >>> The OMAP 3 ISP driver implements an HS_VS event which is triggered when >>> the reception of a frame begins. This functionality is very, very likely >>> not specific to OMAP 3 ISP so it should be standardised. >>> >>> I have a few patches to do that. Additionally the next expected buffer >>> sequence number is provided with the event, unlike earlier. >>> >>> There are a few open questions, however, and this is why I'm sending the >>> set as RFC. >>> >>> >>> 1) Other frame synchronisation events. The CCDC block in the OMAP 3 ISP >>> is able to trigger interrupts at two chosen lines of the image. These >>> naturally can be translated to events. The driver uses both of them >>> internally at specific points of the frame. Nevertheless, there might be >>> some use for these in user space. Other hardware might implement a >>> number of these which wouldn't be used by the driver itself, but I don't >>> know of that at the moment. On the other hand high resolution timers are >>> also available in user space, so doing timing based on ISP provided >>> events is not quite as important as before --- as long as there's one >>> frame based event produced at a known time, such as V4L2_EVENT_FRAME_START. >> >> I'm curious, have you perhaps tried to measure latency of such up calls >> to a user space process? I mean this is going to be a real time stuff, >> with HSYNC periods of 50 us order. Could a user space thread be receiving >> such periodic events reliably ? From my experience I doubt this can work >> reliably outside of an interrupt handler even with high priority real time >> threads. >> >> V4L2_EVENT_FRAME_START event seems OK, but HSYNC events in user space >> sound rather tricky to me :-) > > I think the user space could be interested in just one or two of these > per frame, not for every line. But how to subscribe them --- if they are > needed? Yes, that was my understanding. But still we need much better accuracy than in case of VSYNC/FRAME_START. It seems really hard to guarantee in Linux that a specific line event will be received in user space when that actual line is transmitted. There is much better chance that a FRAME_START event is received during the time when a frame that triggered it is being processed. And VSYNC signals last over several horizontal lines (if not tens or thousands) which makes it easier to receive an event when a VSYNC pulse/period hasn't yet expired. > > Perhaps it'd be better to start with just one and add more once necessary? Maybe we could accommodate the struct v4l2_event_subscription::id field for a horizontal line number, with a special bit indicating any one ? But again I'm not convinced to horizontal line events :) There might be situations when even interrupt handlers are to slow for this. > >> Also HS_VS looks a bit more descriptive than FRAME_START for me. > > HS_VS doesn't really tell which one it is (horizontal or vertical), and > we already have a VSYNC event but it's used for a different purpose. > HS_VS is specific to the CCDC block and doesn't have that meaning in > context of serial interfaces. > > This is why I proposed FRAME_START. OK, initially I thought it was that HS_VS means a moment when vertical and horizontal sync (blanking?) signals go off and thus video data of the first line in a frame is started to be transmitted. I agree that HS_VS isn't that relevant in the CSI terminology. > >> But unfortunately I can't come up with a better name, e.g. something like >> V4L2_EVENT_FRAME_AV_START - frame active video start. Just in case in >> future there are more specific events added. > > What additional information would AV add which isn't evident from > FRAME_START? Given different formats of frame headers across the standards (ITU-R BT.656, MIPI-CSI2 and the like) it might be not so obvious at which moment the FRAME_START event is to be triggered. But not being too paranoid I guess we're perfectly fine with VSYNC and FRAME_START events. > > I admit that there could be differencies in terminology used in this > area; terms that are meaningful to some might not be to others, or they > could mean different things to them. Yes, I entirely agree. So perhaps we're better off with V4L2_EVENT_FRAME_START name and having the exact meaning described in details in the documentation. -- Regards, Sylwester From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from smtp-vbr2.xs4all.nl ([194.109.24.22]:1494 "EHLO smtp-vbr2.xs4all.nl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751338Ab1GZN7r (ORCPT ); Tue, 26 Jul 2011 09:59:47 -0400 From: Hans Verkuil To: Sakari Ailus Subject: Re: [RFC 2/3] v4l: events: Define frame start event Date: Tue, 26 Jul 2011 15:59:38 +0200 Cc: linux-media@vger.kernel.org References: <4E2588AD.4070106@maxwell.research.nokia.com> <201107261352.44287.hverkuil@xs4all.nl> <20110726135125.GB32629@valkosipuli.localdomain> In-Reply-To: <20110726135125.GB32629@valkosipuli.localdomain> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201107261559.38397.hverkuil@xs4all.nl> Sender: linux-media-owner@vger.kernel.org List-ID: On Tuesday, July 26, 2011 15:51:26 Sakari Ailus wrote: > On Tue, Jul 26, 2011 at 01:52:44PM +0200, Hans Verkuil wrote: > > On Tuesday, July 19, 2011 15:38:07 Sakari Ailus wrote: > > > Define a frame start event to tell user space when the reception of a frame > > > starts. > > > > > > Signed-off-by: Sakari Ailus > > > --- > > > Documentation/DocBook/media/v4l/vidioc-dqevent.xml | 26 ++++++++++++++++++++ > > > .../DocBook/media/v4l/vidioc-subscribe-event.xml | 18 +++++++++++++ > > > include/linux/videodev2.h | 12 +++++++-- > > > 3 files changed, 53 insertions(+), 3 deletions(-) > > > > > > diff --git a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > > index 5200b68..d2cb8db 100644 > > > --- a/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > > +++ b/Documentation/DocBook/media/v4l/vidioc-dqevent.xml > > > @@ -88,6 +88,12 @@ > > > > > > > > > > > > + &v4l2-event-frame-sync; > > > + frame > > > + Event data for event V4L2_EVENT_FRAME_START. > > > > The name of the struct and the event are not in sync (pardon the expression :-) ). > > > > Both should either be named FRAME_SYNC or FRAME_START. > > Should they be in sync? FRAME_START event is for frame start, not for other > purposes. Ah, you expect other events to reuse the same payload struct. I missed that part. > The buffer sequence number, however, could be used by other events, too. > This is directly related to the question of how to subscribe line-based > events. Albeit whether they are really ever needed is another question. > > Getting _one_ event giving frame synchronisation timestamps is important, > however; that's also why I sent the RFC. > > What I might do now is that we define a FRAME_SYNC (or FRAME_START) event > and specify the id == 0 always, and worry about the rest later on. It is > quite possible that line based events will never be needed. I would go for that. > If they are, then we must also specify how to subscribe them. Using 'id' as the line number seems sensible to me, but I would definitely leave that part out for now. I am not convinced it is possible to use that reliably in any case due to the difficult timing requirements. Regards, Hans