* [PATCH v5 0/2] media: imx: staging cleanup and FIM export to v4l2-core
@ 2026-11-22 1:29 Jiaqi Shi
2026-11-22 1:29 ` [PATCH v5 1/2] media: imx: remove legacy video node control and format handling Jiaqi Shi
2026-11-22 1:29 ` [PATCH v5 2/2] media: v4l2-core: add frame interval monitor helper Jiaqi Shi
0 siblings, 2 replies; 3+ messages in thread
From: Jiaqi Shi @ 2026-11-22 1:29 UTC (permalink / raw)
To: linux-media, imx, linux-staging
Cc: slongerbeam, p.zabel, mchehab, gregkh, Frank.Li, kernel, festevam,
Jiaqi Shi
Changes in v5:
- Use the source-independent frame-size enumeration for legacy capture
nodes as well, instead of pairing generic pixel-format enumeration
with source-dependent frame-size enumeration.
- Restrict IC capture format enumeration and fallback to YUV/RGB formats;
CSI capture nodes continue to support the full format set.
- Keep the remaining STD and frame-interval legacy ioctls unchanged.
- Add kernel-doc for the public FIM configuration and helper APIs,
including ownership, callback context, and locking rules.
This series continues the i.MX staging media driver cleanup by removing
legacy control inheritance and source-dependent format handling from
capture video nodes, and exports the generic Frame Interval Monitor to
v4l2-core while retaining the i.MX input capture controls locally.
The new FIM files are already covered by the existing MEDIA INPUT
INFRASTRUCTURE entries for drivers/media/ and include/media/.
Validation:
- allmodconfig build and sparse
- scripts/kernel-doc -none on the FIM header and implementation
- local FIM KUnit tests, not included in this series: 13/13 passed
- local white-box i.MX capture format tests, not included: 21/21 passed
- QEMU vivid: v4l2-compliance 125/125, 30-frame streaming passed
- QEMU vim2m: v4l2-compliance 49/49, 10-frame streaming passed
- i.MX register-level and hardware DMA paths remain untested
Jiaqi Shi (2):
media: imx: remove legacy video node control and format handling
media: v4l2-core: add frame interval monitor helper
Documentation/admin-guide/media/imx.rst | 20 +-
drivers/media/v4l2-core/Kconfig | 4 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-fim.c | 385 +++++++++++++++++
drivers/staging/media/imx/Kconfig | 1 +
drivers/staging/media/imx/TODO | 13 -
drivers/staging/media/imx/imx-ic-prpencvf.c | 10 +-
drivers/staging/media/imx/imx-media-capture.c | 391 +++++++-----------
drivers/staging/media/imx/imx-media-csi.c | 10 +-
.../staging/media/imx/imx-media-dev-common.c | 113 +----
drivers/staging/media/imx/imx-media-fim.c | 361 +++-------------
drivers/staging/media/imx/imx-media-utils.c | 9 +-
drivers/staging/media/imx/imx-media.h | 11 +-
include/media/v4l2-fim.h | 57 +++
14 files changed, 672 insertions(+), 714 deletions(-)
create mode 100644 drivers/media/v4l2-core/v4l2-fim.c
delete mode 100644 drivers/staging/media/imx/TODO
create mode 100644 include/media/v4l2-fim.h
--
2.53.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH v5 1/2] media: imx: remove legacy video node control and format handling
2026-11-22 1:29 [PATCH v5 0/2] media: imx: staging cleanup and FIM export to v4l2-core Jiaqi Shi
@ 2026-11-22 1:29 ` Jiaqi Shi
2026-11-22 1:29 ` [PATCH v5 2/2] media: v4l2-core: add frame interval monitor helper Jiaqi Shi
1 sibling, 0 replies; 3+ messages in thread
From: Jiaqi Shi @ 2026-11-22 1:29 UTC (permalink / raw)
To: linux-media, imx, linux-staging
Cc: slongerbeam, p.zabel, mchehab, gregkh, Frank.Li, kernel, festevam,
Jiaqi Shi
Remove control inheritance from pipeline subdevices to video nodes.
Userspace should access controls through subdev nodes instead.
Use source-independent format ioctls for all capture nodes and
initialize them to the existing UYVY 640x480 default instead of the
source subdev active format. Keep stream-time format validation.
Update the usage documentation and remove the completed TODO entries.
Signed-off-by: Jiaqi Shi <shijiaqi_develop@163.com>
---
Documentation/admin-guide/media/imx.rst | 20 +-
drivers/staging/media/imx/TODO | 10 -
drivers/staging/media/imx/imx-ic-prpencvf.c | 10 +-
drivers/staging/media/imx/imx-media-capture.c | 391 +++++++-----------
drivers/staging/media/imx/imx-media-csi.c | 10 +-
.../staging/media/imx/imx-media-dev-common.c | 113 +----
drivers/staging/media/imx/imx-media-utils.c | 9 +-
drivers/staging/media/imx/imx-media.h | 11 +-
8 files changed, 169 insertions(+), 405 deletions(-)
diff --git a/Documentation/admin-guide/media/imx.rst b/Documentation/admin-guide/media/imx.rst
index bb68100d8acb..812c6ac09ab8 100644
--- a/Documentation/admin-guide/media/imx.rst
+++ b/Documentation/admin-guide/media/imx.rst
@@ -394,13 +394,9 @@ scaling up to 1024x1024, CSC, flip, and rotation:
Usage Notes
-----------
-To aid in configuration and for backward compatibility with V4L2
-applications that access controls only from video device nodes, the
-capture device interfaces inherit controls from the active entities
-in the current pipeline, so controls can be accessed either directly
-from the subdev or from the active capture device interface. For
-example, the FIM controls are available either from the ipuX_csiY
-subdevs or from the active capture device.
+Controls for pipeline entities are accessed through their subdev device
+nodes. For example, the FIM controls are available from the ipuX_csiY
+subdevs.
The following are specific usage notes for the Sabre* reference
boards:
@@ -483,7 +479,7 @@ NTSC, or 'alternate'):
media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]"
media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
# Configure "ipu1_csi0 capture" interface (assumed at /dev/video4)
- v4l2-ctl -d4 --set-fmt-video=field=interlaced_bt
+ v4l2-ctl -d4 --set-fmt-video=width=720,height=480,field=interlaced_bt
Streaming can then begin on /dev/video4. The v4l2-ctl tool can also be
used to select any supported YUV pixelformat on /dev/video4.
@@ -510,7 +506,7 @@ alternating fields (field type 'seq-tb' for PAL, or 'alternate').
media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
# Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
- v4l2-ctl -d2 --set-fmt-video=field=none
+ v4l2-ctl -d2 --set-fmt-video=width=720,height=576,field=none
Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
used to select any supported YUV pixelformat on /dev/video2.
@@ -541,7 +537,7 @@ NTSC, or 'alternate'):
media-ctl -V "'ipu1_csi0_mux':5 [fmt:UYVY2X8/720x480]"
media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]"
# Configure "ipu1_csi0 capture" interface (assumed at /dev/video0)
- v4l2-ctl -d0 --set-fmt-video=field=interlaced_bt
+ v4l2-ctl -d0 --set-fmt-video=width=720,height=480,field=interlaced_bt
Streaming can then begin on /dev/video0. The v4l2-ctl tool can also be
used to select any supported YUV pixelformat on /dev/video0.
@@ -568,7 +564,7 @@ alternating fields (field type 'seq-tb' for PAL, or 'alternate').
media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x576 field:none]"
media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x576 field:none]"
# Configure "ipu1_ic_prpvf capture" interface (assumed at /dev/video2)
- v4l2-ctl -d2 --set-fmt-video=field=none
+ v4l2-ctl -d2 --set-fmt-video=width=720,height=576,field=none
Streaming can then begin on /dev/video2. The v4l2-ctl tool can also be
used to select any supported YUV pixelformat on /dev/video2.
@@ -652,7 +648,7 @@ shows colorspace conversion and scaling at IC output.
media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
# Set a format at the capture interface
- v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
+ v4l2-ctl -d /dev/video1 --set-fmt-video=width=800,height=600,pixelformat=RGB3
Streaming can then begin on "ipu1_ic_prpenc capture" node.
diff --git a/drivers/staging/media/imx/TODO b/drivers/staging/media/imx/TODO
index 11c9e10d34ae..43cd030d4bbd 100644
--- a/drivers/staging/media/imx/TODO
+++ b/drivers/staging/media/imx/TODO
@@ -1,13 +1,3 @@
- The Frame Interval Monitor could be exported to v4l2-core for
general use.
-
-- This media driver supports inheriting V4L2 controls to the
- video capture devices, from the subdevices in the capture device's
- pipeline. The controls for each capture device are updated in the
- link_notify callback when the pipeline is modified. This feature should be
- removed, userspace should use the subdev-based userspace API instead.
-
-- Similarly to the legacy control handling, legacy format handling where
- formats on the video nodes are influenced by the active format of the
- connected subdev should be removed.
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 3430ddeac356..bd4c2e7a0b8d 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -203,7 +203,7 @@ static void prp_vb2_buf_done(struct prp_priv *priv, struct ipuv3_channel *ch)
done = priv->active_vb2_buf[priv->ipu_buf_num];
if (done) {
- done->vbuf.field = vdev->fmt.field;
+ done->vbuf.field = vdev->stream_field;
done->vbuf.sequence = priv->frame_sequence;
vb = &done->vbuf.vb2_buf;
vb->timestamp = ktime_get_ns();
@@ -362,6 +362,7 @@ static int prp_setup_channel(struct prp_priv *priv,
memset(&image, 0, sizeof(image));
image.pix = vdev->fmt;
+ image.pix.field = vdev->stream_field;
image.rect = vdev->compose;
/*
@@ -1280,13 +1281,13 @@ static int prp_registered(struct v4l2_subdev *sd)
priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
&ic_priv->sd,
- PRPENCVF_SRC_PAD, true);
+ PRPENCVF_SRC_PAD, true, true);
if (IS_ERR(priv->vdev))
return PTR_ERR(priv->vdev);
ret = imx_media_capture_device_register(priv->vdev, 0);
if (ret)
- goto remove_vdev;
+ return ret;
ret = prp_init_controls(priv);
if (ret)
@@ -1296,8 +1297,6 @@ static int prp_registered(struct v4l2_subdev *sd)
unreg_vdev:
imx_media_capture_device_unregister(priv->vdev);
-remove_vdev:
- imx_media_capture_device_remove(priv->vdev);
return ret;
}
@@ -1306,7 +1305,6 @@ static void prp_unregistered(struct v4l2_subdev *sd)
struct prp_priv *priv = sd_to_priv(sd);
imx_media_capture_device_unregister(priv->vdev);
- imx_media_capture_device_remove(priv->vdev);
v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
}
diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index 9216a221b72f..0003c20bb610 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -5,6 +5,7 @@
* Copyright (c) 2012-2016 Mentor Graphics Inc.
*/
#include <linux/delay.h>
+#include <linux/math64.h>
#include <linux/module.h>
#include <linux/pinctrl/consumer.h>
#include <linux/platform_device.h>
@@ -12,7 +13,6 @@
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/timer.h>
-#include <media/v4l2-ctrls.h>
#include <media/v4l2-device.h>
#include <media/v4l2-event.h>
#include <media/v4l2-fwnode.h>
@@ -35,16 +35,13 @@ struct capture_priv {
struct v4l2_subdev *src_sd; /* Source subdev */
int src_sd_pad; /* Source subdev pad */
+ bool ic_route; /* Source is an IC entity */
struct mutex mutex; /* Protect vdev operations */
struct vb2_queue q; /* The videobuf2 queue */
struct list_head ready_q; /* List of queued buffers */
spinlock_t q_lock; /* Protect ready_q */
-
- struct v4l2_ctrl_handler ctrl_hdlr; /* Controls inherited from subdevs */
-
- bool legacy_api; /* Use the legacy (pre-MC) API */
};
#define to_capture_priv(v) container_of(v, struct capture_priv, vdev)
@@ -59,22 +56,25 @@ struct capture_priv {
static const struct imx_media_pixfmt *capture_find_format(u32 code, u32 fourcc)
{
const struct imx_media_pixfmt *cc;
+ unsigned int i;
cc = imx_media_find_ipu_format(code, PIXFMT_SEL_YUV_RGB);
if (cc) {
enum imx_pixfmt_sel fmt_sel = cc->cs == IPUV3_COLORSPACE_YUV
- ? PIXFMT_SEL_YUV : PIXFMT_SEL_RGB;
-
- cc = imx_media_find_pixel_format(fourcc, fmt_sel);
- if (!cc) {
- imx_media_enum_pixel_formats(&fourcc, 0, fmt_sel, 0);
- cc = imx_media_find_pixel_format(fourcc, fmt_sel);
- }
+ ? PIXFMT_SEL_YUV : PIXFMT_SEL_RGB;
- return cc;
+ return imx_media_find_pixel_format(fourcc, fmt_sel);
}
- return imx_media_find_mbus_format(code, PIXFMT_SEL_ANY);
+ cc = imx_media_find_pixel_format(fourcc, PIXFMT_SEL_ANY);
+ if (!cc || !cc->codes)
+ return NULL;
+
+ for (i = 0; cc->codes[i]; i++)
+ if (cc->codes[i] == code)
+ return cc;
+
+ return NULL;
}
static int capture_querycap(struct file *file, void *fh,
@@ -93,8 +93,12 @@ static int capture_querycap(struct file *file, void *fh,
static int capture_enum_fmt_vid_cap(struct file *file, void *fh,
struct v4l2_fmtdesc *f)
{
+ struct capture_priv *priv = video_drvdata(file);
+ enum imx_pixfmt_sel fmt_sel = priv->ic_route ? PIXFMT_SEL_YUV_RGB :
+ PIXFMT_SEL_ANY;
+
return imx_media_enum_pixel_formats(&f->pixelformat, f->index,
- PIXFMT_SEL_ANY, f->mbus_code);
+ fmt_sel, f->mbus_code);
}
static int capture_enum_framesizes(struct file *file, void *fh,
@@ -136,39 +140,71 @@ static int capture_g_fmt_vid_cap(struct file *file, void *fh,
}
static const struct imx_media_pixfmt *
-__capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose)
+__capture_try_fmt(struct capture_priv *priv, struct v4l2_pix_format *pixfmt,
+ struct v4l2_rect *compose)
{
struct v4l2_mbus_framefmt fmt_src;
const struct imx_media_pixfmt *cc;
+ enum imx_pixfmt_sel fmt_sel = priv->ic_route ? PIXFMT_SEL_YUV_RGB :
+ PIXFMT_SEL_ANY;
+ int ret;
/*
* Find the pixel format, default to the first supported format if not
* found.
*/
- cc = imx_media_find_pixel_format(pixfmt->pixelformat, PIXFMT_SEL_ANY);
+ cc = imx_media_find_pixel_format(pixfmt->pixelformat, fmt_sel);
if (!cc) {
imx_media_enum_pixel_formats(&pixfmt->pixelformat, 0,
- PIXFMT_SEL_ANY, 0);
+ fmt_sel, 0);
cc = imx_media_find_pixel_format(pixfmt->pixelformat,
- PIXFMT_SEL_ANY);
+ fmt_sel);
}
- /* Allow IDMAC interweave but enforce field order from source. */
- if (V4L2_FIELD_IS_INTERLACED(pixfmt->field)) {
- switch (pixfmt->field) {
- case V4L2_FIELD_SEQ_TB:
- pixfmt->field = V4L2_FIELD_INTERLACED_TB;
- break;
- case V4L2_FIELD_SEQ_BT:
- pixfmt->field = V4L2_FIELD_INTERLACED_BT;
- break;
- default:
- break;
- }
+ switch (pixfmt->field) {
+ case V4L2_FIELD_ANY:
+ pixfmt->field = V4L2_FIELD_NONE;
+ break;
+ case V4L2_FIELD_NONE:
+ case V4L2_FIELD_INTERLACED:
+ case V4L2_FIELD_INTERLACED_TB:
+ case V4L2_FIELD_INTERLACED_BT:
+ case V4L2_FIELD_SEQ_TB:
+ case V4L2_FIELD_SEQ_BT:
+ case V4L2_FIELD_ALTERNATE:
+ break;
+ default:
+ pixfmt->field = V4L2_FIELD_NONE;
+ break;
}
+ /* Clamp dimensions to prevent zero-sized buffers. */
+ pixfmt->width = clamp(pixfmt->width, 1, 65535);
+ pixfmt->height = clamp(pixfmt->height, 1, 65535);
+
v4l2_fill_mbus_format(&fmt_src, pixfmt, 0);
- imx_media_mbus_fmt_to_pix_fmt(pixfmt, &fmt_src, cc);
+ if (cc->cs == IPUV3_COLORSPACE_RGB &&
+ fmt_src.quantization == V4L2_QUANTIZATION_DEFAULT)
+ fmt_src.quantization = V4L2_QUANTIZATION_FULL_RANGE;
+ imx_media_try_colorimetry(&fmt_src, priv->ic_route);
+
+ ret = imx_media_mbus_fmt_to_pix_fmt(pixfmt, &fmt_src, cc);
+ if (ret) {
+ u64 divisor = (u64)pixfmt->bytesperline *
+ (cc->planar ? cc->bpp : 8);
+
+ if (!divisor)
+ return NULL;
+
+ fmt_src.height = div64_u64((u64)U32_MAX * 8, divisor);
+ if (!fmt_src.height)
+ return NULL;
+
+ pixfmt->height = fmt_src.height;
+ ret = imx_media_mbus_fmt_to_pix_fmt(pixfmt, &fmt_src, cc);
+ if (ret)
+ return NULL;
+ }
if (compose) {
compose->width = fmt_src.width;
@@ -181,7 +217,11 @@ __capture_try_fmt(struct v4l2_pix_format *pixfmt, struct v4l2_rect *compose)
static int capture_try_fmt_vid_cap(struct file *file, void *fh,
struct v4l2_format *f)
{
- __capture_try_fmt(&f->fmt.pix, NULL);
+ struct capture_priv *priv = video_drvdata(file);
+
+ if (!__capture_try_fmt(priv, &f->fmt.pix, NULL))
+ return -EINVAL;
+
return 0;
}
@@ -196,10 +236,13 @@ static int capture_s_fmt_vid_cap(struct file *file, void *fh,
return -EBUSY;
}
- cc = __capture_try_fmt(&f->fmt.pix, &priv->vdev.compose);
+ cc = __capture_try_fmt(priv, &f->fmt.pix, &priv->vdev.compose);
+ if (!cc)
+ return -EINVAL;
priv->vdev.cc = cc;
priv->vdev.fmt = f->fmt.pix;
+ priv->vdev.stream_field = f->fmt.pix.field;
return 0;
}
@@ -275,46 +318,6 @@ static const struct v4l2_ioctl_ops capture_ioctl_ops = {
* Legacy Video IOCTLs
*/
-static int capture_legacy_enum_framesizes(struct file *file, void *fh,
- struct v4l2_frmsizeenum *fsize)
-{
- struct capture_priv *priv = video_drvdata(file);
- const struct imx_media_pixfmt *cc;
- struct v4l2_subdev_frame_size_enum fse = {
- .index = fsize->index,
- .pad = priv->src_sd_pad,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
- int ret;
-
- cc = imx_media_find_pixel_format(fsize->pixel_format, PIXFMT_SEL_ANY);
- if (!cc)
- return -EINVAL;
-
- fse.code = cc->codes ? cc->codes[0] : 0;
-
- ret = v4l2_subdev_call(priv->src_sd, pad, enum_frame_size, NULL, &fse);
- if (ret)
- return ret;
-
- if (fse.min_width == fse.max_width &&
- fse.min_height == fse.max_height) {
- fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
- fsize->discrete.width = fse.min_width;
- fsize->discrete.height = fse.min_height;
- } else {
- fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
- fsize->stepwise.min_width = fse.min_width;
- fsize->stepwise.max_width = fse.max_width;
- fsize->stepwise.min_height = fse.min_height;
- fsize->stepwise.max_height = fse.max_height;
- fsize->stepwise.step_width = 1;
- fsize->stepwise.step_height = 1;
- }
-
- return 0;
-}
-
static int capture_legacy_enum_frameintervals(struct file *file, void *fh,
struct v4l2_frmivalenum *fival)
{
@@ -346,133 +349,6 @@ static int capture_legacy_enum_frameintervals(struct file *file, void *fh,
return 0;
}
-static int capture_legacy_enum_fmt_vid_cap(struct file *file, void *fh,
- struct v4l2_fmtdesc *f)
-{
- struct capture_priv *priv = video_drvdata(file);
- const struct imx_media_pixfmt *cc_src;
- struct v4l2_subdev_format fmt_src = {
- .pad = priv->src_sd_pad,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
- u32 fourcc;
- int ret;
-
- ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
- if (ret) {
- dev_err(priv->dev, "failed to get src_sd format\n");
- return ret;
- }
-
- cc_src = imx_media_find_ipu_format(fmt_src.format.code,
- PIXFMT_SEL_YUV_RGB);
- if (cc_src) {
- enum imx_pixfmt_sel fmt_sel =
- (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
- PIXFMT_SEL_YUV : PIXFMT_SEL_RGB;
-
- ret = imx_media_enum_pixel_formats(&fourcc, f->index, fmt_sel,
- 0);
- if (ret)
- return ret;
- } else {
- cc_src = imx_media_find_mbus_format(fmt_src.format.code,
- PIXFMT_SEL_ANY);
- if (WARN_ON(!cc_src))
- return -EINVAL;
-
- if (f->index != 0)
- return -EINVAL;
- fourcc = cc_src->fourcc;
- }
-
- f->pixelformat = fourcc;
-
- return 0;
-}
-
-static const struct imx_media_pixfmt *
-__capture_legacy_try_fmt(struct capture_priv *priv,
- struct v4l2_subdev_format *fmt_src,
- struct v4l2_pix_format *pixfmt)
-{
- const struct imx_media_pixfmt *cc;
-
- cc = capture_find_format(fmt_src->format.code, pixfmt->pixelformat);
- if (WARN_ON(!cc))
- return NULL;
-
- /* allow IDMAC interweave but enforce field order from source */
- if (V4L2_FIELD_IS_INTERLACED(pixfmt->field)) {
- switch (fmt_src->format.field) {
- case V4L2_FIELD_SEQ_TB:
- fmt_src->format.field = V4L2_FIELD_INTERLACED_TB;
- break;
- case V4L2_FIELD_SEQ_BT:
- fmt_src->format.field = V4L2_FIELD_INTERLACED_BT;
- break;
- default:
- break;
- }
- }
-
- imx_media_mbus_fmt_to_pix_fmt(pixfmt, &fmt_src->format, cc);
-
- return cc;
-}
-
-static int capture_legacy_try_fmt_vid_cap(struct file *file, void *fh,
- struct v4l2_format *f)
-{
- struct capture_priv *priv = video_drvdata(file);
- struct v4l2_subdev_format fmt_src = {
- .pad = priv->src_sd_pad,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
- int ret;
-
- ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
- if (ret)
- return ret;
-
- if (!__capture_legacy_try_fmt(priv, &fmt_src, &f->fmt.pix))
- return -EINVAL;
-
- return 0;
-}
-
-static int capture_legacy_s_fmt_vid_cap(struct file *file, void *fh,
- struct v4l2_format *f)
-{
- struct capture_priv *priv = video_drvdata(file);
- struct v4l2_subdev_format fmt_src = {
- .pad = priv->src_sd_pad,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
- const struct imx_media_pixfmt *cc;
- int ret;
-
- if (vb2_is_busy(&priv->q)) {
- dev_err(priv->dev, "%s queue busy\n", __func__);
- return -EBUSY;
- }
-
- ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, &fmt_src);
- if (ret)
- return ret;
-
- cc = __capture_legacy_try_fmt(priv, &fmt_src, &f->fmt.pix);
- if (!cc)
- return -EINVAL;
-
- priv->vdev.cc = cc;
- priv->vdev.fmt = f->fmt.pix;
- priv->vdev.compose.width = fmt_src.format.width;
- priv->vdev.compose.height = fmt_src.format.height;
-
- return 0;
-}
-
static int capture_legacy_querystd(struct file *file, void *fh,
v4l2_std_id *std)
{
@@ -553,8 +429,6 @@ static int capture_legacy_subscribe_event(struct v4l2_fh *fh,
return v4l2_event_subscribe(fh, sub, 0, NULL);
case V4L2_EVENT_SOURCE_CHANGE:
return v4l2_src_change_event_subscribe(fh, sub);
- case V4L2_EVENT_CTRL:
- return v4l2_ctrl_subscribe_event(fh, sub);
default:
return -EINVAL;
}
@@ -563,13 +437,13 @@ static int capture_legacy_subscribe_event(struct v4l2_fh *fh,
static const struct v4l2_ioctl_ops capture_legacy_ioctl_ops = {
.vidioc_querycap = capture_querycap,
- .vidioc_enum_framesizes = capture_legacy_enum_framesizes,
+ .vidioc_enum_framesizes = capture_enum_framesizes,
.vidioc_enum_frameintervals = capture_legacy_enum_frameintervals,
- .vidioc_enum_fmt_vid_cap = capture_legacy_enum_fmt_vid_cap,
+ .vidioc_enum_fmt_vid_cap = capture_enum_fmt_vid_cap,
.vidioc_g_fmt_vid_cap = capture_g_fmt_vid_cap,
- .vidioc_try_fmt_vid_cap = capture_legacy_try_fmt_vid_cap,
- .vidioc_s_fmt_vid_cap = capture_legacy_s_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap = capture_try_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = capture_s_fmt_vid_cap,
.vidioc_querystd = capture_legacy_querystd,
.vidioc_g_std = capture_legacy_g_std,
@@ -672,6 +546,51 @@ static void capture_buf_queue(struct vb2_buffer *vb)
spin_unlock_irqrestore(&priv->q_lock, flags);
}
+static int capture_validate_field(struct capture_priv *priv,
+ const struct v4l2_mbus_framefmt *src)
+{
+ u32 field = priv->vdev.fmt.field;
+
+ switch (field) {
+ case V4L2_FIELD_INTERLACED:
+ switch (src->field) {
+ case V4L2_FIELD_SEQ_TB:
+ field = V4L2_FIELD_INTERLACED_TB;
+ break;
+ case V4L2_FIELD_SEQ_BT:
+ field = V4L2_FIELD_INTERLACED_BT;
+ break;
+ case V4L2_FIELD_INTERLACED:
+ break;
+ case V4L2_FIELD_INTERLACED_TB:
+ case V4L2_FIELD_INTERLACED_BT:
+ field = src->field;
+ break;
+ default:
+ return -EPIPE;
+ }
+ break;
+ case V4L2_FIELD_INTERLACED_TB:
+ if (src->field != V4L2_FIELD_SEQ_TB &&
+ src->field != V4L2_FIELD_INTERLACED_TB)
+ return -EPIPE;
+ break;
+ case V4L2_FIELD_INTERLACED_BT:
+ if (src->field != V4L2_FIELD_SEQ_BT &&
+ src->field != V4L2_FIELD_INTERLACED_BT)
+ return -EPIPE;
+ break;
+ default:
+ if (field != src->field)
+ return -EPIPE;
+ break;
+ }
+
+ priv->vdev.stream_field = field;
+
+ return 0;
+}
+
static int capture_validate_fmt(struct capture_priv *priv)
{
struct v4l2_subdev_format fmt_src = {
@@ -701,11 +620,11 @@ static int capture_validate_fmt(struct capture_priv *priv)
* Verify that the media bus code is compatible with the pixel format
* set on the video node.
*/
- cc = capture_find_format(fmt_src.format.code, 0);
+ cc = capture_find_format(fmt_src.format.code, priv->vdev.fmt.pixelformat);
if (!cc || priv->vdev.cc->cs != cc->cs)
return -EPIPE;
- return 0;
+ return capture_validate_field(priv, &fmt_src.format);
}
static int capture_start_streaming(struct vb2_queue *vq, unsigned int count)
@@ -869,36 +788,22 @@ void imx_media_capture_device_error(struct imx_media_video_dev *vdev)
}
EXPORT_SYMBOL_GPL(imx_media_capture_device_error);
-static int capture_init_format(struct capture_priv *priv)
+static void capture_init_format(struct capture_priv *priv)
{
- struct v4l2_subdev_format fmt_src = {
- .pad = priv->src_sd_pad,
- .which = V4L2_SUBDEV_FORMAT_ACTIVE,
- };
struct imx_media_video_dev *vdev = &priv->vdev;
- int ret;
+ struct v4l2_mbus_framefmt fmt_src;
- if (priv->legacy_api) {
- ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL,
- &fmt_src);
- if (ret) {
- dev_err(priv->dev, "failed to get source format\n");
- return ret;
- }
- } else {
- fmt_src.format.code = MEDIA_BUS_FMT_UYVY8_2X8;
- fmt_src.format.width = IMX_MEDIA_DEF_PIX_WIDTH;
- fmt_src.format.height = IMX_MEDIA_DEF_PIX_HEIGHT;
- }
+ imx_media_init_mbus_fmt(&fmt_src, IMX_MEDIA_DEF_PIX_WIDTH,
+ IMX_MEDIA_DEF_PIX_HEIGHT, MEDIA_BUS_FMT_UYVY8_2X8,
+ V4L2_FIELD_NONE, NULL);
- imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt, &fmt_src.format, NULL);
- vdev->compose.width = fmt_src.format.width;
- vdev->compose.height = fmt_src.format.height;
+ imx_media_mbus_fmt_to_pix_fmt(&vdev->fmt, &fmt_src, NULL);
+ vdev->stream_field = vdev->fmt.field;
+ vdev->compose.width = fmt_src.width;
+ vdev->compose.height = fmt_src.height;
vdev->cc = imx_media_find_pixel_format(vdev->fmt.pixelformat,
PIXFMT_SEL_ANY);
-
- return 0;
}
int imx_media_capture_device_register(struct imx_media_video_dev *vdev,
@@ -916,9 +821,7 @@ int imx_media_capture_device_register(struct imx_media_video_dev *vdev,
vfd->v4l2_dev = v4l2_dev;
/* Initialize the default format and compose rectangle. */
- ret = capture_init_format(priv);
- if (ret < 0)
- return ret;
+ capture_init_format(priv);
/* Register the video device. */
ret = video_register_device(vfd, VFL_TYPE_VIDEO, -1);
@@ -960,7 +863,7 @@ EXPORT_SYMBOL_GPL(imx_media_capture_device_unregister);
struct imx_media_video_dev *
imx_media_capture_device_init(struct device *dev, struct v4l2_subdev *src_sd,
- int pad, bool legacy_api)
+ int pad, bool legacy_api, bool ic_route)
{
struct capture_priv *priv;
struct video_device *vfd;
@@ -974,7 +877,7 @@ imx_media_capture_device_init(struct device *dev, struct v4l2_subdev *src_sd,
priv->src_sd = src_sd;
priv->src_sd_pad = pad;
priv->dev = dev;
- priv->legacy_api = legacy_api;
+ priv->ic_route = ic_route;
mutex_init(&priv->mutex);
INIT_LIST_HEAD(&priv->ready_q);
@@ -1031,24 +934,10 @@ imx_media_capture_device_init(struct device *dev, struct v4l2_subdev *src_sd,
return ERR_PTR(ret);
}
- if (legacy_api) {
- /* Initialize the control handler. */
- v4l2_ctrl_handler_init(&priv->ctrl_hdlr, 0);
- vfd->ctrl_handler = &priv->ctrl_hdlr;
- }
-
return &priv->vdev;
}
EXPORT_SYMBOL_GPL(imx_media_capture_device_init);
-void imx_media_capture_device_remove(struct imx_media_video_dev *vdev)
-{
- struct capture_priv *priv = to_capture_priv(vdev);
-
- v4l2_ctrl_handler_free(&priv->ctrl_hdlr);
-}
-EXPORT_SYMBOL_GPL(imx_media_capture_device_remove);
-
MODULE_DESCRIPTION("i.MX5/6 v4l2 video capture interface driver");
MODULE_AUTHOR("Steve Longerbeam <steve_longerbeam@mentor.com>");
MODULE_LICENSE("GPL");
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 43c637b333f2..51b8a58e02c2 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -266,7 +266,7 @@ static void csi_vb2_buf_done(struct csi_priv *priv)
done = priv->active_vb2_buf[priv->ipu_buf_num];
if (done) {
- done->vbuf.field = vdev->fmt.field;
+ done->vbuf.field = vdev->stream_field;
done->vbuf.sequence = priv->frame_sequence;
vb = &done->vbuf.vb2_buf;
vb->timestamp = ktime_get_ns();
@@ -423,6 +423,7 @@ static int csi_idmac_setup_channel(struct csi_priv *priv,
memset(&image, 0, sizeof(image));
image.pix = vdev->fmt;
+ image.pix.field = vdev->stream_field;
image.rect = vdev->compose;
csi_idmac_setup_vb2_buf(priv, phys);
@@ -1815,7 +1816,7 @@ static int csi_registered(struct v4l2_subdev *sd)
}
priv->vdev = imx_media_capture_device_init(priv->sd.dev, &priv->sd,
- CSI_SRC_PAD_IDMAC, true);
+ CSI_SRC_PAD_IDMAC, true, false);
if (IS_ERR(priv->vdev)) {
ret = PTR_ERR(priv->vdev);
goto free_fim;
@@ -1823,12 +1824,10 @@ static int csi_registered(struct v4l2_subdev *sd)
ret = imx_media_capture_device_register(priv->vdev, 0);
if (ret)
- goto remove_vdev;
+ goto free_fim;
return 0;
-remove_vdev:
- imx_media_capture_device_remove(priv->vdev);
free_fim:
if (priv->fim)
imx_media_fim_free(priv->fim);
@@ -1842,7 +1841,6 @@ static void csi_unregistered(struct v4l2_subdev *sd)
struct csi_priv *priv = v4l2_get_subdevdata(sd);
imx_media_capture_device_unregister(priv->vdev);
- imx_media_capture_device_remove(priv->vdev);
if (priv->fim)
imx_media_fim_free(priv->fim);
diff --git a/drivers/staging/media/imx/imx-media-dev-common.c b/drivers/staging/media/imx/imx-media-dev-common.c
index 0d0ee8627a2d..8a0ce7e394ae 100644
--- a/drivers/staging/media/imx/imx-media-dev-common.c
+++ b/drivers/staging/media/imx/imx-media-dev-common.c
@@ -6,7 +6,6 @@
* Copyright (c) 2016 Mentor Graphics Inc.
*/
-#include <media/v4l2-ctrls.h>
#include <media/v4l2-event.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-mc.h>
@@ -189,116 +188,6 @@ int imx_media_probe_complete(struct v4l2_async_notifier *notifier)
}
EXPORT_SYMBOL_GPL(imx_media_probe_complete);
-/*
- * adds controls to a video device from an entity subdevice.
- * Continues upstream from the entity's sink pads.
- */
-static int imx_media_inherit_controls(struct imx_media_dev *imxmd,
- struct video_device *vfd,
- struct media_entity *entity)
-{
- int i, ret = 0;
-
- if (is_media_entity_v4l2_subdev(entity)) {
- struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
-
- dev_dbg(imxmd->md.dev,
- "adding controls to %s from %s\n",
- vfd->entity.name, sd->entity.name);
-
- ret = v4l2_ctrl_add_handler(vfd->ctrl_handler,
- sd->ctrl_handler,
- NULL, true);
- if (ret)
- return ret;
- }
-
- /* move upstream */
- for (i = 0; i < entity->num_pads; i++) {
- struct media_pad *pad, *spad = &entity->pads[i];
-
- if (!(spad->flags & MEDIA_PAD_FL_SINK))
- continue;
-
- pad = media_pad_remote_pad_first(spad);
- if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
- continue;
-
- ret = imx_media_inherit_controls(imxmd, vfd, pad->entity);
- if (ret)
- break;
- }
-
- return ret;
-}
-
-static int imx_media_link_notify(struct media_link *link, u32 flags,
- unsigned int notification)
-{
- struct imx_media_dev *imxmd = container_of(link->graph_obj.mdev,
- struct imx_media_dev, md);
- struct media_entity *source = link->source->entity;
- struct imx_media_pad_vdev *pad_vdev;
- struct list_head *pad_vdev_list;
- struct video_device *vfd;
- struct v4l2_subdev *sd;
- int pad_idx, ret;
-
- ret = v4l2_pipeline_link_notify(link, flags, notification);
- if (ret)
- return ret;
-
- /* don't bother if source is not a subdev */
- if (!is_media_entity_v4l2_subdev(source))
- return 0;
-
- sd = media_entity_to_v4l2_subdev(source);
- pad_idx = link->source->index;
-
- pad_vdev_list = to_pad_vdev_list(sd, pad_idx);
- if (!pad_vdev_list) {
- /* nothing to do if source sd has no pad vdev list */
- return 0;
- }
-
- /*
- * Before disabling a link, reset controls for all video
- * devices reachable from this link.
- *
- * After enabling a link, refresh controls for all video
- * devices reachable from this link.
- */
- if (notification == MEDIA_DEV_NOTIFY_PRE_LINK_CH &&
- !(flags & MEDIA_LNK_FL_ENABLED)) {
- list_for_each_entry(pad_vdev, pad_vdev_list, list) {
- vfd = pad_vdev->vdev->vfd;
- if (!vfd->ctrl_handler)
- continue;
- dev_dbg(imxmd->md.dev,
- "reset controls for %s\n",
- vfd->entity.name);
- v4l2_ctrl_handler_free(vfd->ctrl_handler);
- v4l2_ctrl_handler_init(vfd->ctrl_handler, 0);
- }
- } else if (notification == MEDIA_DEV_NOTIFY_POST_LINK_CH &&
- (link->flags & MEDIA_LNK_FL_ENABLED)) {
- list_for_each_entry(pad_vdev, pad_vdev_list, list) {
- vfd = pad_vdev->vdev->vfd;
- if (!vfd->ctrl_handler)
- continue;
- dev_dbg(imxmd->md.dev,
- "refresh controls for %s\n",
- vfd->entity.name);
- ret = imx_media_inherit_controls(imxmd, vfd,
- &vfd->entity);
- if (ret)
- break;
- }
- }
-
- return ret;
-}
-
static void imx_media_notify(struct v4l2_subdev *sd, unsigned int notification,
void *arg)
{
@@ -326,7 +215,7 @@ static const struct v4l2_async_notifier_operations imx_media_notifier_ops = {
};
static const struct media_device_ops imx_media_md_ops = {
- .link_notify = imx_media_link_notify,
+ .link_notify = v4l2_pipeline_link_notify,
};
struct imx_media_dev *imx_media_dev_init(struct device *dev,
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 7bdcb88a686b..7cd69a698124 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -522,6 +522,7 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
{
u32 width;
u32 stride;
+ u64 sizeimage;
if (!cc) {
cc = imx_media_find_ipu_format(mbus->code,
@@ -562,8 +563,12 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix,
pix->quantization = mbus->quantization;
pix->field = mbus->field;
pix->bytesperline = stride;
- pix->sizeimage = cc->planar ? ((stride * pix->height * cc->bpp) >> 3) :
- stride * pix->height;
+ sizeimage = (u64)stride * pix->height;
+ if (cc->planar)
+ sizeimage = (sizeimage * cc->bpp) >> 3;
+ if (sizeimage > U32_MAX)
+ return -EINVAL;
+ pix->sizeimage = sizeimage;
return 0;
}
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index 537558a7bece..c09546fcf080 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -109,6 +109,8 @@ struct imx_media_video_dev {
/* the user format */
struct v4l2_pix_format fmt;
+ /* field order resolved for the current stream */
+ u32 stream_field;
/* the compose rectangle */
struct v4l2_rect compose;
const struct imx_media_pixfmt *cc;
@@ -125,10 +127,8 @@ static inline struct imx_media_buffer *to_imx_media_vb(struct vb2_buffer *vb)
}
/*
- * to support control inheritance to video devices, this
- * retrieves a pad's list_head of video devices that can
- * be reached from the pad. Note that only the lists in
- * source pads get populated, sink pads have empty lists.
+ * Retrieve a pad's list of video devices reachable from the pad for
+ * subdev event forwarding. Only source pad lists are populated.
*/
static inline struct list_head *
to_pad_vdev_list(struct v4l2_subdev *sd, int pad_index)
@@ -270,8 +270,7 @@ int imx_media_ic_unregister(struct v4l2_subdev *sd);
/* imx-media-capture.c */
struct imx_media_video_dev *
imx_media_capture_device_init(struct device *dev, struct v4l2_subdev *src_sd,
- int pad, bool legacy_api);
-void imx_media_capture_device_remove(struct imx_media_video_dev *vdev);
+ int pad, bool legacy_api, bool ic_route);
int imx_media_capture_device_register(struct imx_media_video_dev *vdev,
u32 link_flags);
void imx_media_capture_device_unregister(struct imx_media_video_dev *vdev);
--
2.53.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH v5 2/2] media: v4l2-core: add frame interval monitor helper
2026-11-22 1:29 [PATCH v5 0/2] media: imx: staging cleanup and FIM export to v4l2-core Jiaqi Shi
2026-11-22 1:29 ` [PATCH v5 1/2] media: imx: remove legacy video node control and format handling Jiaqi Shi
@ 2026-11-22 1:29 ` Jiaqi Shi
1 sibling, 0 replies; 3+ messages in thread
From: Jiaqi Shi @ 2026-11-22 1:29 UTC (permalink / raw)
To: linux-media, imx, linux-staging
Cc: slongerbeam, p.zabel, mchehab, gregkh, Frank.Li, kernel, festevam,
Jiaqi Shi
Move the generic frame interval monitoring implementation from i.MX
to v4l2-core. The helper supports driver-selected control and event
IDs and accepts EOF or external timestamps.
Keep the i.MX-specific input capture controls in the i.MX adapter,
sharing the original control handler and preserving the existing UAPI
and event semantics.
Select V4L2_FIM from VIDEO_IMX_MEDIA and remove the completed TODO.
Signed-off-by: Jiaqi Shi <shijiaqi_develop@163.com>
---
drivers/media/v4l2-core/Kconfig | 4 +
drivers/media/v4l2-core/Makefile | 1 +
drivers/media/v4l2-core/v4l2-fim.c | 385 ++++++++++++++++++++++
drivers/staging/media/imx/Kconfig | 1 +
drivers/staging/media/imx/TODO | 3 -
drivers/staging/media/imx/imx-media-fim.c | 361 ++++----------------
include/media/v4l2-fim.h | 57 ++++
7 files changed, 503 insertions(+), 309 deletions(-)
create mode 100644 drivers/media/v4l2-core/v4l2-fim.c
delete mode 100644 drivers/staging/media/imx/TODO
create mode 100644 include/media/v4l2-fim.h
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index d50ccac9733c..cff961b24c04 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -40,6 +40,10 @@ config VIDEO_TUNER
config V4L2_JPEG_HELPER
tristate
+# Used by drivers that need v4l2-fim.ko
+config V4L2_FIM
+ tristate
+
# Used by drivers that need v4l2-h264.ko
config V4L2_H264
tristate
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index 329f0eadce99..d5f6699a2669 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -26,6 +26,7 @@ videodev-$(CONFIG_VIDEO_V4L2_I2C) += v4l2-i2c.o
obj-$(CONFIG_V4L2_ASYNC) += v4l2-async.o
obj-$(CONFIG_V4L2_CCI) += v4l2-cci.o
+obj-$(CONFIG_V4L2_FIM) += v4l2-fim.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
obj-$(CONFIG_V4L2_FWNODE) += v4l2-fwnode.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
diff --git a/drivers/media/v4l2-core/v4l2-fim.c b/drivers/media/v4l2-core/v4l2-fim.c
new file mode 100644
index 000000000000..146dcb85caeb
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-fim.c
@@ -0,0 +1,385 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Frame Interval Monitor.
+ *
+ * Copyright (c) 2016 Mentor Graphics Inc.
+ */
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-fim.h>
+#include <media/v4l2-subdev.h>
+
+enum {
+ FIM_CL_ENABLE = 0,
+ FIM_CL_NUM,
+ FIM_CL_TOLERANCE_MIN,
+ FIM_CL_TOLERANCE_MAX,
+ FIM_CL_NUM_SKIP,
+};
+
+#define FIM_CL_ENABLE_DEF 0 /* FIM disabled by default */
+#define FIM_CL_NUM_DEF 8 /* average 8 frames */
+#define FIM_CL_NUM_SKIP_DEF 2 /* skip 2 frames after restart */
+#define FIM_CL_TOLERANCE_MIN_DEF 50 /* usec */
+#define FIM_CL_TOLERANCE_MAX_DEF 0 /* no max tolerance (unbounded) */
+
+struct v4l2_fim {
+ struct v4l2_subdev *sd;
+ struct v4l2_fim_config config;
+ struct v4l2_ctrl *ctrl[V4L2_FIM_NUM_CONTROLS];
+
+ spinlock_t lock; /* protect control values and monitoring state */
+
+ bool enabled;
+ int num_avg;
+ int num_skip;
+ unsigned long tolerance_min;
+ unsigned long tolerance_max;
+ bool external_timestamp;
+
+ int counter;
+ ktime_t last_ts;
+ unsigned long sum;
+ unsigned long nominal;
+ bool stream_on;
+};
+
+static void update_fim_nominal(struct v4l2_fim *fim,
+ const struct v4l2_fract *fi)
+{
+ if (fi->denominator == 0) {
+ dev_dbg(fim->sd->dev, "no frame interval, FIM disabled\n");
+ fim->enabled = false;
+ return;
+ }
+
+ fim->nominal = DIV_ROUND_CLOSEST_ULL(1000000ULL * (u64)fi->numerator,
+ fi->denominator);
+
+ dev_dbg(fim->sd->dev, "FI=%lu usec\n", fim->nominal);
+}
+
+static void reset_fim(struct v4l2_fim *fim, bool curval, struct v4l2_ctrl *ctrl)
+{
+ struct v4l2_ctrl *en = fim->ctrl[FIM_CL_ENABLE];
+ struct v4l2_ctrl *num = fim->ctrl[FIM_CL_NUM];
+ struct v4l2_ctrl *skip = fim->ctrl[FIM_CL_NUM_SKIP];
+ struct v4l2_ctrl *tol_min = fim->ctrl[FIM_CL_TOLERANCE_MIN];
+ struct v4l2_ctrl *tol_max = fim->ctrl[FIM_CL_TOLERANCE_MAX];
+ bool core_active = !curval && ctrl &&
+ ctrl->cluster == en->cluster;
+
+ /*
+ * When curval is false, read ->val only for controls in the cluster
+ * currently being set; other clusters may hold uncommitted values.
+ */
+ if (fim->config.external_timestamp)
+ fim->external_timestamp =
+ fim->config.external_timestamp(fim->config.priv,
+ !curval && !core_active);
+
+ if (!curval && core_active) {
+ fim->enabled = en->val;
+ fim->num_avg = num->val;
+ fim->num_skip = skip->val;
+ fim->tolerance_min = tol_min->val;
+ fim->tolerance_max = tol_max->val;
+ } else {
+ fim->enabled = en->cur.val;
+ fim->num_avg = num->cur.val;
+ fim->num_skip = skip->cur.val;
+ fim->tolerance_min = tol_min->cur.val;
+ fim->tolerance_max = tol_max->cur.val;
+ }
+
+ /* disable tolerance range if max <= min */
+ if (fim->tolerance_max <= fim->tolerance_min)
+ fim->tolerance_max = 0;
+
+ /* num_skip must be >= 1 if input capture not used */
+ if (!fim->external_timestamp)
+ fim->num_skip = max_t(int, fim->num_skip, 1);
+
+ fim->counter = -fim->num_skip;
+ fim->sum = 0;
+}
+
+/**
+ * v4l2_fim_reset - reset the monitor state from control values
+ * @fim: FIM instance
+ * @curval: read committed values instead of values being updated
+ * @ctrl: control in the cluster being updated, or NULL for committed values
+ *
+ * When @curval is false, new values are read only from @ctrl's cluster;
+ * committed values are read from other clusters. The caller must hold the
+ * control-handler lock in that case.
+ */
+void v4l2_fim_reset(struct v4l2_fim *fim, bool curval, struct v4l2_ctrl *ctrl)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&fim->lock, flags);
+ reset_fim(fim, curval, ctrl);
+ spin_unlock_irqrestore(&fim->lock, flags);
+}
+EXPORT_SYMBOL_GPL(v4l2_fim_reset);
+
+static void send_fim_event(struct v4l2_fim *fim)
+{
+ struct v4l2_event ev = {
+ .type = fim->config.event_type,
+ };
+
+ v4l2_subdev_notify_event(fim->sd, &ev);
+}
+
+/*
+ * Monitor an averaged frame interval. If the average deviates too much
+ * from the nominal frame rate, send the frame interval error event. The
+ * frame intervals are averaged in order to quiet noise from
+ * (presumably random) interrupt latency.
+ */
+static void frame_interval_monitor(struct v4l2_fim *fim, ktime_t timestamp)
+{
+ long long interval, error;
+ unsigned long error_avg;
+ bool send_event = false;
+
+ if (!fim->enabled || ++fim->counter <= 0)
+ goto out_update_ts;
+
+ /* max error is less than 100µs, so use 32-bit division or fail */
+ interval = ktime_to_ns(ktime_sub(timestamp, fim->last_ts));
+ error = abs(interval - NSEC_PER_USEC * (s64)fim->nominal);
+ if (error > U32_MAX)
+ error = U32_MAX;
+ else
+ error = abs((u32)error / NSEC_PER_USEC);
+
+ if (fim->tolerance_max && error >= fim->tolerance_max) {
+ dev_dbg(fim->sd->dev,
+ "FIM: %llu ignored, out of tolerance bounds\n",
+ error);
+ fim->counter--;
+ goto out_update_ts;
+ }
+
+ fim->sum += error;
+
+ if (fim->counter == fim->num_avg) {
+ error_avg = DIV_ROUND_CLOSEST(fim->sum, fim->num_avg);
+
+ if (error_avg > fim->tolerance_min)
+ send_event = true;
+
+ dev_dbg(fim->sd->dev, "FIM: error: %lu usec%s\n",
+ error_avg, send_event ? " (!!!)" : "");
+
+ fim->counter = 0;
+ fim->sum = 0;
+ }
+
+out_update_ts:
+ fim->last_ts = timestamp;
+ if (send_event)
+ send_fim_event(fim);
+}
+
+/**
+ * v4l2_fim_eof_monitor - monitor a frame interval from an EOF timestamp
+ * @fim: FIM instance
+ * @timestamp: end-of-frame timestamp
+ *
+ * Does nothing when external timestamps are selected. This function takes
+ * the FIM spinlock and may be called in interrupt context.
+ */
+void v4l2_fim_eof_monitor(struct v4l2_fim *fim, ktime_t timestamp)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&fim->lock, flags);
+
+ if (!fim->external_timestamp)
+ frame_interval_monitor(fim, timestamp);
+
+ spin_unlock_irqrestore(&fim->lock, flags);
+}
+EXPORT_SYMBOL_GPL(v4l2_fim_eof_monitor);
+
+/**
+ * v4l2_fim_monitor_timestamp - monitor a frame interval from a timestamp
+ * @fim: FIM instance
+ * @timestamp: externally supplied frame timestamp
+ *
+ * This function takes the FIM spinlock and may be called in interrupt
+ * context.
+ */
+void v4l2_fim_monitor_timestamp(struct v4l2_fim *fim, ktime_t timestamp)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&fim->lock, flags);
+ frame_interval_monitor(fim, timestamp);
+ spin_unlock_irqrestore(&fim->lock, flags);
+}
+EXPORT_SYMBOL_GPL(v4l2_fim_monitor_timestamp);
+
+/**
+ * v4l2_fim_set_stream - start or stop frame interval monitoring
+ * @fim: FIM instance
+ * @fi: nominal frame interval, or NULL when stopping
+ * @on: true to start monitoring, false to stop
+ *
+ * Starts by resetting the monitor from committed control values. This
+ * function acquires the control-handler lock and must be called in process
+ * context. The optional wait_first_timestamp callback may sleep.
+ */
+void v4l2_fim_set_stream(struct v4l2_fim *fim,
+ const struct v4l2_fract *fi, bool on)
+{
+ unsigned long flags;
+
+ v4l2_ctrl_lock(fim->ctrl[FIM_CL_ENABLE]);
+
+ if (fim->stream_on == on)
+ goto out;
+
+ if (on) {
+ spin_lock_irqsave(&fim->lock, flags);
+ reset_fim(fim, true, NULL);
+ update_fim_nominal(fim, fi);
+ spin_unlock_irqrestore(&fim->lock, flags);
+
+ if (fim->external_timestamp && fim->enabled && !fim->num_skip &&
+ fim->config.wait_first_timestamp)
+ fim->config.wait_first_timestamp(fim->config.priv);
+ }
+
+ fim->stream_on = on;
+out:
+ v4l2_ctrl_unlock(fim->ctrl[FIM_CL_ENABLE]);
+}
+EXPORT_SYMBOL_GPL(v4l2_fim_set_stream);
+
+/**
+ * v4l2_fim_is_streaming - return whether monitoring is active
+ * @fim: FIM instance
+ *
+ * Returns the current stream state without taking locks. Callers that need
+ * serialization with stream changes must provide their own locking.
+ */
+bool v4l2_fim_is_streaming(struct v4l2_fim *fim)
+{
+ return fim->stream_on;
+}
+EXPORT_SYMBOL_GPL(v4l2_fim_is_streaming);
+
+static int fim_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct v4l2_fim *fim = ctrl->priv;
+ unsigned long flags;
+
+ spin_lock_irqsave(&fim->lock, flags);
+ reset_fim(fim, false, ctrl);
+ spin_unlock_irqrestore(&fim->lock, flags);
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops fim_ctrl_ops = {
+ .s_ctrl = fim_s_ctrl,
+};
+
+static const struct v4l2_ctrl_config fim_ctrl[] = {
+ [FIM_CL_ENABLE] = {
+ .ops = &fim_ctrl_ops,
+ .name = "FIM Enable",
+ .type = V4L2_CTRL_TYPE_BOOLEAN,
+ .def = FIM_CL_ENABLE_DEF,
+ .min = 0,
+ .max = 1,
+ .step = 1,
+ },
+ [FIM_CL_NUM] = {
+ .ops = &fim_ctrl_ops,
+ .name = "FIM Num Average",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .def = FIM_CL_NUM_DEF,
+ .min = 1,
+ .max = 64,
+ .step = 1,
+ },
+ [FIM_CL_TOLERANCE_MIN] = {
+ .ops = &fim_ctrl_ops,
+ .name = "FIM Tolerance Min",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .def = FIM_CL_TOLERANCE_MIN_DEF,
+ .min = 2,
+ .max = 200,
+ .step = 1,
+ },
+ [FIM_CL_TOLERANCE_MAX] = {
+ .ops = &fim_ctrl_ops,
+ .name = "FIM Tolerance Max",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .def = FIM_CL_TOLERANCE_MAX_DEF,
+ .min = 0,
+ .max = 500,
+ .step = 1,
+ },
+ [FIM_CL_NUM_SKIP] = {
+ .ops = &fim_ctrl_ops,
+ .name = "FIM Num Skip",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .def = FIM_CL_NUM_SKIP_DEF,
+ .min = 0,
+ .max = 256,
+ .step = 1,
+ },
+};
+
+/**
+ * v4l2_fim_init - allocate and initialize a frame interval monitor
+ * @sd: subdevice that owns the FIM instance and emits its events
+ * @hdlr: control handler to which the five FIM controls are added
+ * @config: driver-specific control IDs, event type, and callbacks
+ *
+ * The FIM instance is managed by @sd->dev. The caller retains ownership of
+ * @hdlr and must free it with v4l2_ctrl_handler_free().
+ *
+ * Return: a FIM instance on success, or an ERR_PTR() on failure.
+ */
+struct v4l2_fim *v4l2_fim_init(struct v4l2_subdev *sd,
+ struct v4l2_ctrl_handler *hdlr,
+ const struct v4l2_fim_config *config)
+{
+ struct v4l2_fim *fim;
+ int i;
+
+ fim = devm_kzalloc(sd->dev, sizeof(*fim), GFP_KERNEL);
+ if (!fim)
+ return ERR_PTR(-ENOMEM);
+
+ fim->sd = sd;
+ fim->config = *config;
+ spin_lock_init(&fim->lock);
+
+ for (i = 0; i < V4L2_FIM_NUM_CONTROLS; i++) {
+ struct v4l2_ctrl_config cfg = fim_ctrl[i];
+
+ cfg.id = config->ctrl_base + i;
+ fim->ctrl[i] = v4l2_ctrl_new_custom(hdlr, &cfg, fim);
+ }
+ if (hdlr->error)
+ return ERR_PTR(hdlr->error);
+
+ v4l2_ctrl_cluster(V4L2_FIM_NUM_CONTROLS, fim->ctrl);
+
+ return fim;
+}
+EXPORT_SYMBOL_GPL(v4l2_fim_init);
+
+MODULE_DESCRIPTION("V4L2 frame interval monitor helpers");
+MODULE_LICENSE("GPL");
diff --git a/drivers/staging/media/imx/Kconfig b/drivers/staging/media/imx/Kconfig
index 1cd48028b641..f559b9979039 100644
--- a/drivers/staging/media/imx/Kconfig
+++ b/drivers/staging/media/imx/Kconfig
@@ -6,6 +6,7 @@ config VIDEO_IMX_MEDIA
depends on VIDEO_DEV
depends on IMX_IPUV3_CORE
select MEDIA_CONTROLLER
+ select V4L2_FIM
select V4L2_FWNODE
select V4L2_MEM2MEM_DEV
select VIDEOBUF2_DMA_CONTIG
diff --git a/drivers/staging/media/imx/TODO b/drivers/staging/media/imx/TODO
deleted file mode 100644
index 43cd030d4bbd..000000000000
--- a/drivers/staging/media/imx/TODO
+++ /dev/null
@@ -1,3 +0,0 @@
-
-- The Frame Interval Monitor could be exported to v4l2-core for
- general use.
diff --git a/drivers/staging/media/imx/imx-media-fim.c b/drivers/staging/media/imx/imx-media-fim.c
index ccbc0371fba2..cb7bd40009d5 100644
--- a/drivers/staging/media/imx/imx-media-fim.c
+++ b/drivers/staging/media/imx/imx-media-fim.c
@@ -4,68 +4,31 @@
*
* Copyright (c) 2016 Mentor Graphics Inc.
*/
-#include <linux/delay.h>
+#include <linux/completion.h>
#include <linux/irq.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
+#include <linux/jiffies.h>
#include <linux/slab.h>
-#include <linux/spinlock.h>
#include <media/v4l2-ctrls.h>
+#include <media/v4l2-fim.h>
#include <media/v4l2-subdev.h>
#include <media/imx.h>
#include "imx-media.h"
-enum {
- FIM_CL_ENABLE = 0,
- FIM_CL_NUM,
- FIM_CL_TOLERANCE_MIN,
- FIM_CL_TOLERANCE_MAX,
- FIM_CL_NUM_SKIP,
- FIM_NUM_CONTROLS,
-};
-
enum {
FIM_CL_ICAP_EDGE = 0,
FIM_CL_ICAP_CHANNEL,
FIM_NUM_ICAP_CONTROLS,
};
-#define FIM_CL_ENABLE_DEF 0 /* FIM disabled by default */
-#define FIM_CL_NUM_DEF 8 /* average 8 frames */
-#define FIM_CL_NUM_SKIP_DEF 2 /* skip 2 frames after restart */
-#define FIM_CL_TOLERANCE_MIN_DEF 50 /* usec */
-#define FIM_CL_TOLERANCE_MAX_DEF 0 /* no max tolerance (unbounded) */
-
struct imx_media_fim {
- /* the owning subdev of this fim instance */
struct v4l2_subdev *sd;
-
- /* FIM's control handler */
+ struct v4l2_fim *core;
struct v4l2_ctrl_handler ctrl_handler;
+ struct v4l2_ctrl *icap_ctrl[FIM_NUM_ICAP_CONTROLS];
- /* control clusters */
- struct v4l2_ctrl *ctrl[FIM_NUM_CONTROLS];
- struct v4l2_ctrl *icap_ctrl[FIM_NUM_ICAP_CONTROLS];
-
- spinlock_t lock; /* protect control values */
-
- /* current control values */
- bool enabled;
- int num_avg;
- int num_skip;
- unsigned long tolerance_min; /* usec */
- unsigned long tolerance_max; /* usec */
- /* input capture method of measuring FI */
- int icap_channel;
- int icap_flags;
-
- int counter;
- ktime_t last_ts;
- unsigned long sum; /* usec */
- unsigned long nominal; /* usec */
-
+ int icap_channel;
+ int icap_flags;
struct completion icap_first_event;
- bool stream_on;
};
static bool icap_enabled(struct imx_media_fim *fim)
@@ -73,171 +36,52 @@ static bool icap_enabled(struct imx_media_fim *fim)
return fim->icap_flags != IRQ_TYPE_NONE;
}
-static void update_fim_nominal(struct imx_media_fim *fim,
- const struct v4l2_fract *fi)
-{
- if (fi->denominator == 0) {
- dev_dbg(fim->sd->dev, "no frame interval, FIM disabled\n");
- fim->enabled = false;
- return;
- }
-
- fim->nominal = DIV_ROUND_CLOSEST_ULL(1000000ULL * (u64)fi->numerator,
- fi->denominator);
-
- dev_dbg(fim->sd->dev, "FI=%lu usec\n", fim->nominal);
-}
-
-static void reset_fim(struct imx_media_fim *fim, bool curval)
+static bool imx_fim_external_timestamp(void *priv, bool use_val)
{
+ struct imx_media_fim *fim = priv;
struct v4l2_ctrl *icap_chan = fim->icap_ctrl[FIM_CL_ICAP_CHANNEL];
struct v4l2_ctrl *icap_edge = fim->icap_ctrl[FIM_CL_ICAP_EDGE];
- struct v4l2_ctrl *en = fim->ctrl[FIM_CL_ENABLE];
- struct v4l2_ctrl *num = fim->ctrl[FIM_CL_NUM];
- struct v4l2_ctrl *skip = fim->ctrl[FIM_CL_NUM_SKIP];
- struct v4l2_ctrl *tol_min = fim->ctrl[FIM_CL_TOLERANCE_MIN];
- struct v4l2_ctrl *tol_max = fim->ctrl[FIM_CL_TOLERANCE_MAX];
- if (curval) {
- fim->enabled = en->cur.val;
- fim->icap_flags = icap_edge->cur.val;
- fim->icap_channel = icap_chan->cur.val;
- fim->num_avg = num->cur.val;
- fim->num_skip = skip->cur.val;
- fim->tolerance_min = tol_min->cur.val;
- fim->tolerance_max = tol_max->cur.val;
- } else {
- fim->enabled = en->val;
+ if (use_val) {
fim->icap_flags = icap_edge->val;
fim->icap_channel = icap_chan->val;
- fim->num_avg = num->val;
- fim->num_skip = skip->val;
- fim->tolerance_min = tol_min->val;
- fim->tolerance_max = tol_max->val;
- }
-
- /* disable tolerance range if max <= min */
- if (fim->tolerance_max <= fim->tolerance_min)
- fim->tolerance_max = 0;
-
- /* num_skip must be >= 1 if input capture not used */
- if (!icap_enabled(fim))
- fim->num_skip = max_t(int, fim->num_skip, 1);
-
- fim->counter = -fim->num_skip;
- fim->sum = 0;
-}
-
-static void send_fim_event(struct imx_media_fim *fim, unsigned long error)
-{
- static const struct v4l2_event ev = {
- .type = V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR,
- };
-
- v4l2_subdev_notify_event(fim->sd, &ev);
-}
-
-/*
- * Monitor an averaged frame interval. If the average deviates too much
- * from the nominal frame rate, send the frame interval error event. The
- * frame intervals are averaged in order to quiet noise from
- * (presumably random) interrupt latency.
- */
-static void frame_interval_monitor(struct imx_media_fim *fim,
- ktime_t timestamp)
-{
- long long interval, error;
- unsigned long error_avg;
- bool send_event = false;
-
- if (!fim->enabled || ++fim->counter <= 0)
- goto out_update_ts;
-
- /* max error is less than l00µs, so use 32-bit division or fail */
- interval = ktime_to_ns(ktime_sub(timestamp, fim->last_ts));
- error = abs(interval - NSEC_PER_USEC * (u64)fim->nominal);
- if (error > U32_MAX)
- error = U32_MAX;
- else
- error = abs((u32)error / NSEC_PER_USEC);
-
- if (fim->tolerance_max && error >= fim->tolerance_max) {
- dev_dbg(fim->sd->dev,
- "FIM: %llu ignored, out of tolerance bounds\n",
- error);
- fim->counter--;
- goto out_update_ts;
- }
-
- fim->sum += error;
-
- if (fim->counter == fim->num_avg) {
- error_avg = DIV_ROUND_CLOSEST(fim->sum, fim->num_avg);
-
- if (error_avg > fim->tolerance_min)
- send_event = true;
-
- dev_dbg(fim->sd->dev, "FIM: error: %lu usec%s\n",
- error_avg, send_event ? " (!!!)" : "");
-
- fim->counter = 0;
- fim->sum = 0;
+ } else {
+ fim->icap_flags = icap_edge->cur.val;
+ fim->icap_channel = icap_chan->cur.val;
}
-out_update_ts:
- fim->last_ts = timestamp;
- if (send_event)
- send_fim_event(fim, error_avg);
+ return icap_enabled(fim);
}
-/*
- * In case we are monitoring the first frame interval after streamon
- * (when fim->num_skip = 0), we need a valid fim->last_ts before we
- * can begin. This only applies to the input capture method. It is not
- * possible to accurately measure the first FI after streamon using the
- * EOF method, so fim->num_skip minimum is set to 1 in that case, so this
- * function is a noop when the EOF method is used.
- */
-static void fim_acquire_first_ts(struct imx_media_fim *fim)
+static void imx_fim_wait_first_timestamp(void *priv)
{
+ struct imx_media_fim *fim = priv;
unsigned long ret;
- if (!fim->enabled || fim->num_skip > 0)
- return;
-
- ret = wait_for_completion_timeout(
- &fim->icap_first_event,
- msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
+ ret = wait_for_completion_timeout(&fim->icap_first_event,
+ msecs_to_jiffies(IMX_MEDIA_EOF_TIMEOUT));
if (ret == 0)
v4l2_warn(fim->sd, "wait first icap event timeout\n");
}
-/* FIM Controls */
static int fim_s_ctrl(struct v4l2_ctrl *ctrl)
{
- struct imx_media_fim *fim = container_of(ctrl->handler,
- struct imx_media_fim,
- ctrl_handler);
- unsigned long flags;
+ struct imx_media_fim *fim = ctrl->priv;
int ret = 0;
- spin_lock_irqsave(&fim->lock, flags);
-
switch (ctrl->id) {
- case V4L2_CID_IMX_FIM_ENABLE:
- break;
case V4L2_CID_IMX_FIM_ICAP_EDGE:
- if (fim->stream_on)
+ if (v4l2_fim_is_streaming(fim->core))
ret = -EBUSY;
break;
default:
ret = -EINVAL;
+ break;
}
if (!ret)
- reset_fim(fim, false);
+ v4l2_fim_reset(fim->core, false, ctrl);
- spin_unlock_irqrestore(&fim->lock, flags);
return ret;
}
@@ -245,66 +89,13 @@ static const struct v4l2_ctrl_ops fim_ctrl_ops = {
.s_ctrl = fim_s_ctrl,
};
-static const struct v4l2_ctrl_config fim_ctrl[] = {
- [FIM_CL_ENABLE] = {
- .ops = &fim_ctrl_ops,
- .id = V4L2_CID_IMX_FIM_ENABLE,
- .name = "FIM Enable",
- .type = V4L2_CTRL_TYPE_BOOLEAN,
- .def = FIM_CL_ENABLE_DEF,
- .min = 0,
- .max = 1,
- .step = 1,
- },
- [FIM_CL_NUM] = {
- .ops = &fim_ctrl_ops,
- .id = V4L2_CID_IMX_FIM_NUM,
- .name = "FIM Num Average",
- .type = V4L2_CTRL_TYPE_INTEGER,
- .def = FIM_CL_NUM_DEF,
- .min = 1, /* no averaging */
- .max = 64, /* average 64 frames */
- .step = 1,
- },
- [FIM_CL_TOLERANCE_MIN] = {
- .ops = &fim_ctrl_ops,
- .id = V4L2_CID_IMX_FIM_TOLERANCE_MIN,
- .name = "FIM Tolerance Min",
- .type = V4L2_CTRL_TYPE_INTEGER,
- .def = FIM_CL_TOLERANCE_MIN_DEF,
- .min = 2,
- .max = 200,
- .step = 1,
- },
- [FIM_CL_TOLERANCE_MAX] = {
- .ops = &fim_ctrl_ops,
- .id = V4L2_CID_IMX_FIM_TOLERANCE_MAX,
- .name = "FIM Tolerance Max",
- .type = V4L2_CTRL_TYPE_INTEGER,
- .def = FIM_CL_TOLERANCE_MAX_DEF,
- .min = 0,
- .max = 500,
- .step = 1,
- },
- [FIM_CL_NUM_SKIP] = {
- .ops = &fim_ctrl_ops,
- .id = V4L2_CID_IMX_FIM_NUM_SKIP,
- .name = "FIM Num Skip",
- .type = V4L2_CTRL_TYPE_INTEGER,
- .def = FIM_CL_NUM_SKIP_DEF,
- .min = 0, /* skip no frames */
- .max = 256, /* skip 256 frames */
- .step = 1,
- },
-};
-
static const struct v4l2_ctrl_config fim_icap_ctrl[] = {
[FIM_CL_ICAP_EDGE] = {
.ops = &fim_ctrl_ops,
.id = V4L2_CID_IMX_FIM_ICAP_EDGE,
.name = "FIM Input Capture Edge",
.type = V4L2_CTRL_TYPE_INTEGER,
- .def = IRQ_TYPE_NONE, /* input capture disabled by default */
+ .def = IRQ_TYPE_NONE,
.min = IRQ_TYPE_NONE,
.max = IRQ_TYPE_EDGE_BOTH,
.step = 1,
@@ -321,108 +112,66 @@ static const struct v4l2_ctrl_config fim_icap_ctrl[] = {
},
};
-static int init_fim_controls(struct imx_media_fim *fim)
-{
- struct v4l2_ctrl_handler *hdlr = &fim->ctrl_handler;
- int i, ret;
-
- v4l2_ctrl_handler_init(hdlr, FIM_NUM_CONTROLS + FIM_NUM_ICAP_CONTROLS);
-
- for (i = 0; i < FIM_NUM_CONTROLS; i++)
- fim->ctrl[i] = v4l2_ctrl_new_custom(hdlr,
- &fim_ctrl[i],
- NULL);
- for (i = 0; i < FIM_NUM_ICAP_CONTROLS; i++)
- fim->icap_ctrl[i] = v4l2_ctrl_new_custom(hdlr,
- &fim_icap_ctrl[i],
- NULL);
- if (hdlr->error) {
- ret = hdlr->error;
- goto err_free;
- }
-
- v4l2_ctrl_cluster(FIM_NUM_CONTROLS, fim->ctrl);
- v4l2_ctrl_cluster(FIM_NUM_ICAP_CONTROLS, fim->icap_ctrl);
-
- return 0;
-err_free:
- v4l2_ctrl_handler_free(hdlr);
- return ret;
-}
-
-/*
- * Monitor frame intervals via EOF interrupt. This method is
- * subject to uncertainty errors introduced by interrupt latency.
- *
- * This is a noop if the Input Capture method is being used, since
- * the frame_interval_monitor() is called by the input capture event
- * callback handler in that case.
- */
void imx_media_fim_eof_monitor(struct imx_media_fim *fim, ktime_t timestamp)
{
- unsigned long flags;
-
- spin_lock_irqsave(&fim->lock, flags);
-
- if (!icap_enabled(fim))
- frame_interval_monitor(fim, timestamp);
-
- spin_unlock_irqrestore(&fim->lock, flags);
+ v4l2_fim_eof_monitor(fim->core, timestamp);
}
-/* Called by the subdev in its s_stream callback */
void imx_media_fim_set_stream(struct imx_media_fim *fim,
- const struct v4l2_fract *fi,
+ const struct v4l2_fract *frame_interval,
bool on)
{
- unsigned long flags;
-
- v4l2_ctrl_lock(fim->ctrl[FIM_CL_ENABLE]);
-
- if (fim->stream_on == on)
- goto out;
-
- if (on) {
- spin_lock_irqsave(&fim->lock, flags);
- reset_fim(fim, true);
- update_fim_nominal(fim, fi);
- spin_unlock_irqrestore(&fim->lock, flags);
-
- if (icap_enabled(fim))
- fim_acquire_first_ts(fim);
- }
-
- fim->stream_on = on;
-out:
- v4l2_ctrl_unlock(fim->ctrl[FIM_CL_ENABLE]);
+ v4l2_fim_set_stream(fim->core, frame_interval, on);
}
int imx_media_fim_add_controls(struct imx_media_fim *fim)
{
- /* add the FIM controls to the calling subdev ctrl handler */
return v4l2_ctrl_add_handler(fim->sd->ctrl_handler,
&fim->ctrl_handler, NULL, true);
}
-/* Called by the subdev in its subdev registered callback */
struct imx_media_fim *imx_media_fim_init(struct v4l2_subdev *sd)
{
+ struct v4l2_fim_config config = {
+ .ctrl_base = V4L2_CID_IMX_FIM_ENABLE,
+ .event_type = V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR,
+ .external_timestamp = imx_fim_external_timestamp,
+ .wait_first_timestamp = imx_fim_wait_first_timestamp,
+ };
struct imx_media_fim *fim;
- int ret;
+ int i, ret;
fim = devm_kzalloc(sd->dev, sizeof(*fim), GFP_KERNEL);
if (!fim)
return ERR_PTR(-ENOMEM);
fim->sd = sd;
+ config.priv = fim;
- spin_lock_init(&fim->lock);
+ v4l2_ctrl_handler_init(&fim->ctrl_handler,
+ V4L2_FIM_NUM_CONTROLS + FIM_NUM_ICAP_CONTROLS);
- ret = init_fim_controls(fim);
- if (ret)
- return ERR_PTR(ret);
+ fim->core = v4l2_fim_init(sd, &fim->ctrl_handler, &config);
+ if (IS_ERR(fim->core)) {
+ ret = PTR_ERR(fim->core);
+ goto err_free;
+ }
+
+ for (i = 0; i < FIM_NUM_ICAP_CONTROLS; i++)
+ fim->icap_ctrl[i] = v4l2_ctrl_new_custom(&fim->ctrl_handler,
+ &fim_icap_ctrl[i], fim);
+ if (fim->ctrl_handler.error) {
+ ret = fim->ctrl_handler.error;
+ goto err_free;
+ }
+
+ v4l2_ctrl_cluster(FIM_NUM_ICAP_CONTROLS, fim->icap_ctrl);
return fim;
+
+err_free:
+ v4l2_ctrl_handler_free(&fim->ctrl_handler);
+ return ERR_PTR(ret);
}
void imx_media_fim_free(struct imx_media_fim *fim)
diff --git a/include/media/v4l2-fim.h b/include/media/v4l2-fim.h
new file mode 100644
index 000000000000..e0faa5673023
--- /dev/null
+++ b/include/media/v4l2-fim.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * V4L2 frame interval monitor helpers
+ *
+ * Copyright (c) 2016 Mentor Graphics Inc.
+ */
+
+#ifndef _V4L2_FIM_H
+#define _V4L2_FIM_H
+
+#include <linux/ktime.h>
+#include <linux/types.h>
+
+struct v4l2_ctrl;
+struct v4l2_ctrl_handler;
+struct v4l2_fract;
+struct v4l2_subdev;
+
+struct v4l2_fim;
+
+#define V4L2_FIM_NUM_CONTROLS 5
+
+/**
+ * struct v4l2_fim_config - configuration for a frame interval monitor
+ * @ctrl_base: first ID of the five FIM controls
+ * @event_type: event emitted when the averaged interval error is too large
+ * @priv: driver-private data passed to the callbacks
+ * @external_timestamp: return whether external timestamps are used
+ * @wait_first_timestamp: optionally wait for the first external timestamp
+ *
+ * @external_timestamp runs in atomic context with the FIM spinlock and
+ * control-handler lock held. If the callback's use_val argument is true, it
+ * reads pending control values; otherwise it reads committed values. It must
+ * not sleep.
+ *
+ * @wait_first_timestamp runs in process context with the control-handler
+ * lock held and may sleep.
+ */
+struct v4l2_fim_config {
+ u32 ctrl_base;
+ u32 event_type;
+ void *priv;
+ bool (*external_timestamp)(void *priv, bool use_val);
+ void (*wait_first_timestamp)(void *priv);
+};
+
+struct v4l2_fim *v4l2_fim_init(struct v4l2_subdev *sd,
+ struct v4l2_ctrl_handler *hdlr,
+ const struct v4l2_fim_config *config);
+void v4l2_fim_reset(struct v4l2_fim *fim, bool curval, struct v4l2_ctrl *ctrl);
+void v4l2_fim_set_stream(struct v4l2_fim *fim,
+ const struct v4l2_fract *frame_interval, bool on);
+void v4l2_fim_eof_monitor(struct v4l2_fim *fim, ktime_t timestamp);
+void v4l2_fim_monitor_timestamp(struct v4l2_fim *fim, ktime_t timestamp);
+bool v4l2_fim_is_streaming(struct v4l2_fim *fim);
+
+#endif
--
2.53.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-11-22 1:29 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-22 1:29 [PATCH v5 0/2] media: imx: staging cleanup and FIM export to v4l2-core Jiaqi Shi
2026-11-22 1:29 ` [PATCH v5 1/2] media: imx: remove legacy video node control and format handling Jiaqi Shi
2026-11-22 1:29 ` [PATCH v5 2/2] media: v4l2-core: add frame interval monitor helper Jiaqi Shi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox