From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from perceval.ideasonboard.com ([95.142.166.194]:53526 "EHLO perceval.ideasonboard.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758152Ab3D2UwZ (ORCPT ); Mon, 29 Apr 2013 16:52:25 -0400 From: Laurent Pinchart To: Hans Verkuil Cc: linux-media , Volokh Konstantin , Pete Eberlein , Ismael Luceno , Sylwester Nawrocki , Kamil Debski Subject: Re: [RFC] Motion Detection API Date: Mon, 29 Apr 2013 22:52:31 +0200 Message-ID: <1925455.QCByddZe4C@avalon> In-Reply-To: <201304121736.16542.hverkuil@xs4all.nl> References: <201304121736.16542.hverkuil@xs4all.nl> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Sender: linux-media-owner@vger.kernel.org List-ID: Hi Hans, Sorry for the late reply. On Friday 12 April 2013 17:36:16 Hans Verkuil wrote: > This RFC looks at adding support for motion detection to V4L2. This is the > main missing piece that prevents the go7007 and solo6x10 drivers from being > moved into mainline from the staging directory. > > Step one is to look at existing drivers/hardware: > > 1) The go7007 driver: > > - divides the frame into blocks of 16x16 pixels each (that's 45x36 blocks > for PAL) > - each block can be assigned to region 0, 1, 2 or 3 > - each region has: > - a pixel change threshold > - a motion vector change threshold > - a trigger level; if this is 0, then motion detection for this > region is disabled > - when streaming the reserved field of v4l2_buffer is used as a bitmask: > one bit for each region where motion is detected. > > 2) The solo6x10 driver: > > - divides the frame into blocks of 16x16 pixels each > - each block has its own threshold > - the driver adds one MOTION_ON buffer flag and one MOTION_DETECTED > buffer flag. > - motion detection can be disabled or enabled. > - the driver has a global motion detection mode with just one threshold: > in that case all blocks are set to the same threshold. > - there is also support for displaying a border around the image if > motion is detected (very hardware specific). > > 3) The tw2804 video encoder (based on the datasheet, not implemented in the > driver): > > - divides the image in 12x12 blocks (block size will differ for NTSC vs > PAL) > - motion detection can be enabled or disabled for each block > - there are four controls: > - luminance level change threshold > - spatial sensitivity threshold > - temporal sensitivity threshold > - velocity control (determines how well slow motions are detected) > - detection is reported by a hardware pin in this case > > Comparing these three examples of motion detection I see quite a lot of > similarities, enough to make a proposal for an API: > > - Add a MOTION_DETECTION menu control: > - Disabled > - Global Motion Detection > - Regional Motion Detection > > If 'Global Motion Detection' is selected, then various threshold controls > become available. What sort of thresholds are available seems to be quite > variable, so I am inclined to leave this as private controls. > > - Add new buffer flags when motion is detected. The go7007 driver would need > 4 bits (one for each region), the others just one. This can be done by > taking 4 bits from the v4l2_buffer flags field. There are still 16 bits > left there, and if it becomes full, then we still have two reserved fields. > I see no reason for adding a 'MOTION_ON' flag as the solo6x10 driver does > today: just check the MOTION_DETECTION control if you want to know if > motion detection is on or not. We're really starting to shove metadata in buffer flags. Isn't it time to add a proper metadata API ? I don't really like the idea of using (valuable) buffer flags for a feature supported by three drivers only. > - Add two new ioctls to get and set the block data: > > #define V4L2_MD_HOR_BLOCKS (64) > #define V4L2_MD_VERT_BLOCKS (48) > > #define V4L2_MD_TYPE_REGION (1) > #define V4L2_MD_TYPE_THRESHOLD (2) > > struct v4l2_md_blocks { > __u32 type; > struct v4l2_rect rect; > __u32 minimum; > __u32 maximum; > __u32 reserved[32]; > __u16 blocks[V4L2_MD_HOR_BLOCKS][V4L2_MD_VERT_BLOCKS]; > }; > > #define VIDIOC_G_MD_BLOCKS _IORW('V', 103, struct v4l2_md_blocks) > #define VIDIOC_S_MD_BLOCKS _IORW('V', 104, struct v4l2_md_blocks) > > Apps must fill in type, then can call G_MD_BLOCKS to get the current block > values for that type. TYPE_REGION returns to which region each block > belongs, TYPE_THRESHOLD returns threshold values for each block. > > rect returns the rectangle of valid blocks, minimum and maximum the min > and max values for each 'blocks' array element. > > To change the blocks apps call S_MD_BLOCKS, fill in type, rect (rect is > useful here to set only a subset of all blocks) and blocks. > > So the go7007 would return 45x36 in rect, type would be REGION, min/max > would be 0-3. > > solo6x10 would return 45x36 in rect, type would be THRESHOLD, min/max would > be 0-65535. > > TW2804 would return 12x12 in rect, type would be THRESHOLD, min/max would be > 0-1. > > Comment? Questions? -- Regards, Laurent Pinchart