[PATCH 0/3] media: vimc: Add frame rate control support

[PATCH 0/3] media: vimc: Add frame rate control support

[faizel.kb]

From: Faizel K B <faizel.kb@gmail.com>

This patch series adds frame rate configuration support to the vimc
virtual media controller driver, enabling userspace applications to
control frame rates from 1-240 FPS via v4l2-ctl and media-ctl.

Currently, vimc operates at a fixed 60 FPS. This series enables
dynamic frame rate control through standard V4L2 subdevice pad
operations.

Patch 1/3 moves the vimc_sensor_device structure from vimc-sensor.c
to vimc-common.h, making it accessible to the vimc-streamer component
for reading frame rate configuration.

Patch 2/3 implements set_frame_interval and get_frame_interval
callbacks in v4l2_subdev_pad_ops, allowing frame rate configuration
via standard V4L2 ioctls. The configured interval is pre-calculated
into jiffies and stored in the sensor's hardware structure for
efficient runtime access.

Patch 3/3 applies the configured frame rate in the streamer thread
by using the pre-calculated jiffies value to control timing between
frames, replacing the previous fixed 60 FPS implementation.

Testing done using yavta with various frame rates on CIF resolution
in qemu arm64 platform.fps varies depending on the delay on each
pipeline module.

Example usage:
  # Set to 30 FPS
  media-ctl -V '"Sensor A":0/0[fmt:SRGGB8_1X8/176x144@1/30]'

  # Capture and verify frame rate
  yavta /dev/video2 --capture=100

Faizel K B (3):
  media: vimc: sensor: Move vimc_sensor_device to common header
  media: vimc: sensor: Add frame rate configuration support
  media: vimc: streamer: Apply sensor frame rate in streamer thread

 drivers/media/test-drivers/vimc/vimc-common.h | 31 ++++++++
 drivers/media/test-drivers/vimc/vimc-sensor.c | 71 ++++++++++++-------
 .../media/test-drivers/vimc/vimc-streamer.c   | 35 ++++++++-
 3 files changed, 109 insertions(+), 28 deletions(-)

-- 
2.43.0

[PATCH 1/3] media: vimc: sensor: Move vimc_sensor_device to common header

[faizel.kb]

From: Faizel K B <faizel.kb@gmail.com>

Move the vimc_sensor_device structure from vimc-sensor.c to
vimc-common.h to make it accessible to the vimc-streamer component.

The streamer needs access to the sensor structure to read the
configured frame rate interval for timing control.

Signed-off-by: Faizel K B <faizel.kb@gmail.com>
---
 drivers/media/test-drivers/vimc/vimc-common.h | 29 +++++++++++++++++++
 drivers/media/test-drivers/vimc/vimc-sensor.c | 26 -----------------
 2 files changed, 29 insertions(+), 26 deletions(-)

diff --git a/drivers/media/test-drivers/vimc/vimc-common.h b/drivers/media/test-drivers/vimc/vimc-common.h
index 7a45a2117748..7f5f008e407b 100644
--- a/drivers/media/test-drivers/vimc/vimc-common.h
+++ b/drivers/media/test-drivers/vimc/vimc-common.h
@@ -12,6 +12,8 @@
 #include <linux/slab.h>
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
+#include <media/tpg/v4l2-tpg.h>
+#include <media/v4l2-ctrls.h>
 
 #define VIMC_PDEV_NAME "vimc"
 
@@ -159,6 +161,33 @@ struct vimc_ent_config {
 	const struct vimc_ent_type *type;
 };
 
+enum vimc_sensor_osd_mode {
+	VIMC_SENSOR_OSD_SHOW_ALL = 0,
+	VIMC_SENSOR_OSD_SHOW_COUNTERS = 1,
+	VIMC_SENSOR_OSD_SHOW_NONE = 2
+};
+
+struct vimc_sensor_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct tpg_data tpg;
+	struct v4l2_ctrl_handler hdl;
+	struct media_pad pad;
+
+	u8 *frame;
+
+	/*
+	 * Virtual "hardware" configuration, filled when the stream starts or
+	 * when controls are set.
+	 */
+	struct {
+		struct v4l2_area size;
+		enum vimc_sensor_osd_mode osd_value;
+		u64 start_stream_ts;
+	} hw;
+};
+
+
 /**
  * vimc_is_source - returns true if the entity has only source pads
  *
diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
index 027767777763..2b07dc1f1278 100644
--- a/drivers/media/test-drivers/vimc/vimc-sensor.c
+++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
@@ -14,32 +14,6 @@
 
 #include "vimc-common.h"
 
-enum vimc_sensor_osd_mode {
-	VIMC_SENSOR_OSD_SHOW_ALL = 0,
-	VIMC_SENSOR_OSD_SHOW_COUNTERS = 1,
-	VIMC_SENSOR_OSD_SHOW_NONE = 2
-};
-
-struct vimc_sensor_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct tpg_data tpg;
-	struct v4l2_ctrl_handler hdl;
-	struct media_pad pad;
-
-	u8 *frame;
-
-	/*
-	 * Virtual "hardware" configuration, filled when the stream starts or
-	 * when controls are set.
-	 */
-	struct {
-		struct v4l2_area size;
-		enum vimc_sensor_osd_mode osd_value;
-		u64 start_stream_ts;
-	} hw;
-};
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,
-- 
2.43.0

[PATCH 2/3] media: vimc: sensor: Add frame rate configuration support

[faizel.kb]

From: Faizel K B <faizel.kb@gmail.com>

Implement set_frame_interval and get_frame_interval callbacks in
v4l2_subdev_pad_ops to enable frame rate configuration from 1-240 FPS.
The default frame rate is 60 FPS.

The configured frame interval is pre-calculated into jiffies and
stored in the sensor's hw structure for efficient access by the
streamer thread.

Signed-off-by: Faizel K B <faizel.kb@gmail.com>
---
 drivers/media/test-drivers/vimc/vimc-common.h |  2 +
 drivers/media/test-drivers/vimc/vimc-sensor.c | 45 +++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/drivers/media/test-drivers/vimc/vimc-common.h b/drivers/media/test-drivers/vimc/vimc-common.h
index 7f5f008e407b..a71ef761fa77 100644
--- a/drivers/media/test-drivers/vimc/vimc-common.h
+++ b/drivers/media/test-drivers/vimc/vimc-common.h
@@ -173,6 +173,7 @@ struct vimc_sensor_device {
 	struct tpg_data tpg;
 	struct v4l2_ctrl_handler hdl;
 	struct media_pad pad;
+	struct v4l2_fract frame_interval;
 
 	u8 *frame;
 
@@ -184,6 +185,7 @@ struct vimc_sensor_device {
 		struct v4l2_area size;
 		enum vimc_sensor_osd_mode osd_value;
 		u64 start_stream_ts;
+		unsigned long fps_jiffies;
 	} hw;
 };
 
diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
index 2b07dc1f1278..d54425ff28a1 100644
--- a/drivers/media/test-drivers/vimc/vimc-sensor.c
+++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
@@ -140,12 +140,53 @@ static int vimc_sensor_set_fmt(struct v4l2_subdev *sd,
 
 	return 0;
 }
+static int vimc_sensor_get_frame_interval(struct v4l2_subdev *sd,
+					  struct v4l2_subdev_state *state,
+					  struct v4l2_subdev_frame_interval *fi)
+{
+	struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
+
+	fi->interval = vsensor->frame_interval;
+
+	return 0;
+}
+
+static int vimc_sensor_set_frame_interval(struct v4l2_subdev *sd,
+					  struct v4l2_subdev_state *state,
+					  struct v4l2_subdev_frame_interval *fi)
+{
+	struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
+	u32 fps;
+
+	/* Sanitize to default if invalid */
+	if (unlikely(!fi->interval.numerator || !fi->interval.denominator)) {
+		fi->interval.numerator = 1;
+		fi->interval.denominator = 60;
+	} else {
+		/* Clamp FPS to 1-240 range */
+		fps = fi->interval.denominator / fi->interval.numerator;
+		fps = clamp(fps, 1U, 240U);
+
+		fi->interval.numerator = 1;
+		fi->interval.denominator = fps;
+	}
+
+	vsensor->frame_interval = fi->interval;
+
+	/* Update hardware timing configuration */
+	vsensor->hw.fps_jiffies = (HZ * vsensor->frame_interval.numerator) /
+				  vsensor->frame_interval.denominator;
+
+	return 0;
+}
 
 static const struct v4l2_subdev_pad_ops vimc_sensor_pad_ops = {
 	.enum_mbus_code		= vimc_sensor_enum_mbus_code,
 	.enum_frame_size	= vimc_sensor_enum_frame_size,
 	.get_fmt		= v4l2_subdev_get_fmt,
 	.set_fmt		= vimc_sensor_set_fmt,
+	.get_frame_interval     = vimc_sensor_get_frame_interval,
+	.set_frame_interval     = vimc_sensor_set_frame_interval,
 };
 
 static void *vimc_sensor_process_frame(struct vimc_ent_device *ved,
@@ -400,6 +441,10 @@ static struct vimc_ent_device *vimc_sensor_add(struct vimc_device *vimc,
 
 	vsensor->ved.process_frame = vimc_sensor_process_frame;
 	vsensor->ved.dev = vimc->mdev.dev;
+	/* Initialize to 60 FPS */
+	vsensor->frame_interval.numerator = 1;
+	vsensor->frame_interval.denominator = 60;
+	vsensor->hw.fps_jiffies = HZ / 60;
 
 	return &vsensor->ved;
 
-- 
2.43.0

[PATCH 3/3] media: vimc: streamer: Apply sensor frame rate in streamer thread

[faizel.kb]

From: Faizel K B <faizel.kb@gmail.com>

Use the sensor's pre-calculated jiffies value to add appropriate
delay between frames according to the configured frame rate.

Note: The actual frame rate may vary slightly depending on processing
delays in other media pipeline components.

Tested using yavta frame rate display with CIF resolution:
yavta <video-node> --capture=<no_of_frames>

Signed-off-by: Faizel K B <faizel.kb@gmail.com>
---
 .../media/test-drivers/vimc/vimc-streamer.c   | 35 +++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/drivers/media/test-drivers/vimc/vimc-streamer.c b/drivers/media/test-drivers/vimc/vimc-streamer.c
index 15d863f97cbf..e4af6e0916b5 100644
--- a/drivers/media/test-drivers/vimc/vimc-streamer.c
+++ b/drivers/media/test-drivers/vimc/vimc-streamer.c
@@ -139,6 +139,31 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
 	return -EINVAL;
 }
 
+
+/**
+ * vimc_streamer_get_sensor() - Get sensor from pipeline
+ * @stream: the pipeline
+ *
+ * Helper function to find the sensor device in the pipeline.
+ * Returns pointer to sensor device or NULL if not found.
+ */
+static struct vimc_sensor_device *vimc_streamer_get_sensor(struct vimc_stream *stream)
+{
+	int i;
+
+	for (i = 0; i < stream->pipe_size; i++) {
+		struct vimc_ent_device *ved = stream->ved_pipeline[i];
+
+		if (ved && ved->ent &&
+		    ved->ent->function == MEDIA_ENT_F_CAM_SENSOR) {
+			return container_of(ved, struct vimc_sensor_device, ved);
+		}
+	}
+
+	return NULL;
+}
+
+
 /**
  * vimc_streamer_thread - Process frames through the pipeline
  *
@@ -154,25 +179,31 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
 static int vimc_streamer_thread(void *data)
 {
 	struct vimc_stream *stream = data;
+	struct vimc_sensor_device *vsensor;
 	u8 *frame = NULL;
 	int i;
+	unsigned long fps_jiffies;
+	const unsigned long default_jiffies = HZ / 60;
 
 	set_freezable();
+	vsensor = vimc_streamer_get_sensor(stream);
 
 	for (;;) {
 		try_to_freeze();
 		if (kthread_should_stop())
 			break;
 
+		/* Read from hardware configuration */
+		fps_jiffies = vsensor ? vsensor->hw.fps_jiffies : default_jiffies;
+
 		for (i = stream->pipe_size - 1; i >= 0; i--) {
 			frame = stream->ved_pipeline[i]->process_frame(
 					stream->ved_pipeline[i], frame);
 			if (!frame || IS_ERR(frame))
 				break;
 		}
-		//wait for 60hz
 		set_current_state(TASK_UNINTERRUPTIBLE);
-		schedule_timeout(HZ / 60);
+		schedule_timeout(fps_jiffies);
 	}
 
 	return 0;
-- 
2.43.0

Re: [PATCH 2/3] media: vimc: sensor: Add frame rate configuration support

[Kieran Bingham]

Hi Faizel,

Pullling in the libcamera-devel mailing list for relevant awareness too.

Quoting faizel.kb@gmail.com (2026-01-15 19:10:49)
> From: Faizel K B <faizel.kb@gmail.com>
> 
> Implement set_frame_interval and get_frame_interval callbacks in
> v4l2_subdev_pad_ops to enable frame rate configuration from 1-240 FPS.
> The default frame rate is 60 FPS.
> 
> The configured frame interval is pre-calculated into jiffies and
> stored in the sensor's hw structure for efficient access by the
> streamer thread.

This is a really interesting and helpful bit of work - but I'm weary it
might have taken one small mis-direction.

VIMC aims to replicate the usage of camera sensors, and I don't think
we're supposed to use frame interval interface for that.

Instead following the camera sensor model, this should be implemented to
mirror what the hardware in camera sensors actually does/needs which is
to make the frame rate a function of the exposure time and blankings
(both hblank and vblank).

We have a bit of a write up about the calculation at the bottom of this
page:

 - https://libcamera.org/camera-sensor-model.html

Or perhaps the sensor requirements of libcamera might be what a virtual
sensor should aim for:

 - https://libcamera.org/sensor_driver_requirements.html

There's probably more helpful documentation in the linux kernel too in
the upcoming sensor model rework from Sakari.

Aha - in fact I found this:
 - https://www.kernel.org/doc/html/latest/userspace-api/media/drivers/camera-sensor.html#raw-camera-sensors

"2.2. Frame interval configuration

There are two different methods for obtaining possibilities for
different frame intervals as well as configuring the frame interval.
Which one to implement depends on the type of the device."

So perhaps the question we need to clarify is "what type of device is
vimc-sensor".

I'm likely biased to presume it's a virtual raw camera sensor ... But is
it ?

--
Kieran


> Signed-off-by: Faizel K B <faizel.kb@gmail.com>
> ---
>  drivers/media/test-drivers/vimc/vimc-common.h |  2 +
>  drivers/media/test-drivers/vimc/vimc-sensor.c | 45 +++++++++++++++++++
>  2 files changed, 47 insertions(+)
> 
> diff --git a/drivers/media/test-drivers/vimc/vimc-common.h b/drivers/media/test-drivers/vimc/vimc-common.h
> index 7f5f008e407b..a71ef761fa77 100644
> --- a/drivers/media/test-drivers/vimc/vimc-common.h
> +++ b/drivers/media/test-drivers/vimc/vimc-common.h
> @@ -173,6 +173,7 @@ struct vimc_sensor_device {
>         struct tpg_data tpg;
>         struct v4l2_ctrl_handler hdl;
>         struct media_pad pad;
> +       struct v4l2_fract frame_interval;
>  
>         u8 *frame;
>  
> @@ -184,6 +185,7 @@ struct vimc_sensor_device {
>                 struct v4l2_area size;
>                 enum vimc_sensor_osd_mode osd_value;
>                 u64 start_stream_ts;
> +               unsigned long fps_jiffies;
>         } hw;
>  };
>  
> diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
> index 2b07dc1f1278..d54425ff28a1 100644
> --- a/drivers/media/test-drivers/vimc/vimc-sensor.c
> +++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
> @@ -140,12 +140,53 @@ static int vimc_sensor_set_fmt(struct v4l2_subdev *sd,
>  
>         return 0;
>  }
> +static int vimc_sensor_get_frame_interval(struct v4l2_subdev *sd,
> +                                         struct v4l2_subdev_state *state,
> +                                         struct v4l2_subdev_frame_interval *fi)
> +{
> +       struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
> +
> +       fi->interval = vsensor->frame_interval;
> +
> +       return 0;
> +}
> +
> +static int vimc_sensor_set_frame_interval(struct v4l2_subdev *sd,
> +                                         struct v4l2_subdev_state *state,
> +                                         struct v4l2_subdev_frame_interval *fi)
> +{
> +       struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
> +       u32 fps;
> +
> +       /* Sanitize to default if invalid */
> +       if (unlikely(!fi->interval.numerator || !fi->interval.denominator)) {
> +               fi->interval.numerator = 1;
> +               fi->interval.denominator = 60;
> +       } else {
> +               /* Clamp FPS to 1-240 range */
> +               fps = fi->interval.denominator / fi->interval.numerator;
> +               fps = clamp(fps, 1U, 240U);
> +
> +               fi->interval.numerator = 1;
> +               fi->interval.denominator = fps;
> +       }
> +
> +       vsensor->frame_interval = fi->interval;
> +
> +       /* Update hardware timing configuration */
> +       vsensor->hw.fps_jiffies = (HZ * vsensor->frame_interval.numerator) /
> +                                 vsensor->frame_interval.denominator;
> +
> +       return 0;
> +}
>  
>  static const struct v4l2_subdev_pad_ops vimc_sensor_pad_ops = {
>         .enum_mbus_code         = vimc_sensor_enum_mbus_code,
>         .enum_frame_size        = vimc_sensor_enum_frame_size,
>         .get_fmt                = v4l2_subdev_get_fmt,
>         .set_fmt                = vimc_sensor_set_fmt,
> +       .get_frame_interval     = vimc_sensor_get_frame_interval,
> +       .set_frame_interval     = vimc_sensor_set_frame_interval,
>  };
>  
>  static void *vimc_sensor_process_frame(struct vimc_ent_device *ved,
> @@ -400,6 +441,10 @@ static struct vimc_ent_device *vimc_sensor_add(struct vimc_device *vimc,
>  
>         vsensor->ved.process_frame = vimc_sensor_process_frame;
>         vsensor->ved.dev = vimc->mdev.dev;
> +       /* Initialize to 60 FPS */
> +       vsensor->frame_interval.numerator = 1;
> +       vsensor->frame_interval.denominator = 60;
> +       vsensor->hw.fps_jiffies = HZ / 60;
>  
>         return &vsensor->ved;
>  
> -- 
> 2.43.0
>

Re: [PATCH 2/3] media: vimc: sensor: Add frame rate configuration support

[Faizel K B]

On Fri, Jan 16, 2026 at 09:53:00AM +0000, Kieran Bingham wrote:
>Hi Faizel,
>
>Pullling in the libcamera-devel mailing list for relevant awareness too.
>
>Quoting faizel.kb@gmail.com (2026-01-15 19:10:49)
>> From: Faizel K B <faizel.kb@gmail.com>
>>
>> Implement set_frame_interval and get_frame_interval callbacks in
>> v4l2_subdev_pad_ops to enable frame rate configuration from 1-240 FPS.
>> The default frame rate is 60 FPS.
>>
>> The configured frame interval is pre-calculated into jiffies and
>> stored in the sensor's hw structure for efficient access by the
>> streamer thread.
>
>This is a really interesting and helpful bit of work - but I'm weary it
>might have taken one small mis-direction.
>
>VIMC aims to replicate the usage of camera sensors, and I don't think
>we're supposed to use frame interval interface for that.
>
>Instead following the camera sensor model, this should be implemented to
>mirror what the hardware in camera sensors actually does/needs which is
>to make the frame rate a function of the exposure time and blankings
>(both hblank and vblank).
>
>We have a bit of a write up about the calculation at the bottom of this
>page:
>
> - https://libcamera.org/camera-sensor-model.html
>
>Or perhaps the sensor requirements of libcamera might be what a virtual
>sensor should aim for:
>
> - https://libcamera.org/sensor_driver_requirements.html
>

Thanks for the review and links. It makes sense to implement the pixelrate, 
v/h blanking from a sensor point of view.The exposure value also
matters. Sensors even prioritize the exposure to the frame timings or the
other way. This parameters can be added inside using v4l2_ctrl_new_std(),
for pixel rate,blanking and exposure and calculate the sleep time for the
frame rate delay in the streamer thread. By this, a user need to do few
calculations to adjust the frame rate.


>There's probably more helpful documentation in the linux kernel too in
>the upcoming sensor model rework from Sakari.
>
>Aha - in fact I found this:
> - https://www.kernel.org/doc/html/latest/userspace-api/media/drivers/camera-sensor.html#raw-camera-sensors
>
>"2.2. Frame interval configuration
>
>There are two different methods for obtaining possibilities for
>different frame intervals as well as configuring the frame interval.
>Which one to implement depends on the type of the device."
>
>So perhaps the question we need to clarify is "what type of device is
>vimc-sensor".
>
>I'm likely biased to presume it's a virtual raw camera sensor ... But is
>it ?

Would like to hear more about this.

>
>--
>Kieran
>
>
>> Signed-off-by: Faizel K B <faizel.kb@gmail.com>
>> ---
>>  drivers/media/test-drivers/vimc/vimc-common.h |  2 +
>>  drivers/media/test-drivers/vimc/vimc-sensor.c | 45 +++++++++++++++++++
>>  2 files changed, 47 insertions(+)
>>
>> diff --git a/drivers/media/test-drivers/vimc/vimc-common.h b/drivers/media/test-drivers/vimc/vimc-common.h
>> index 7f5f008e407b..a71ef761fa77 100644
>> --- a/drivers/media/test-drivers/vimc/vimc-common.h
>> +++ b/drivers/media/test-drivers/vimc/vimc-common.h
>> @@ -173,6 +173,7 @@ struct vimc_sensor_device {
>>         struct tpg_data tpg;
>>         struct v4l2_ctrl_handler hdl;
>>         struct media_pad pad;
>> +       struct v4l2_fract frame_interval;
>>
>>         u8 *frame;
>>
>> @@ -184,6 +185,7 @@ struct vimc_sensor_device {
>>                 struct v4l2_area size;
>>                 enum vimc_sensor_osd_mode osd_value;
>>                 u64 start_stream_ts;
>> +               unsigned long fps_jiffies;
>>         } hw;
>>  };
>>
>> diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
>> index 2b07dc1f1278..d54425ff28a1 100644
>> --- a/drivers/media/test-drivers/vimc/vimc-sensor.c
>> +++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
>> @@ -140,12 +140,53 @@ static int vimc_sensor_set_fmt(struct v4l2_subdev *sd,
>>
>>         return 0;
>>  }
>> +static int vimc_sensor_get_frame_interval(struct v4l2_subdev *sd,
>> +                                         struct v4l2_subdev_state *state,
>> +                                         struct v4l2_subdev_frame_interval *fi)
>> +{
>> +       struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
>> +
>> +       fi->interval = vsensor->frame_interval;
>> +
>> +       return 0;
>> +}
>> +
>> +static int vimc_sensor_set_frame_interval(struct v4l2_subdev *sd,
>> +                                         struct v4l2_subdev_state *state,
>> +                                         struct v4l2_subdev_frame_interval *fi)
>> +{
>> +       struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
>> +       u32 fps;
>> +
>> +       /* Sanitize to default if invalid */
>> +       if (unlikely(!fi->interval.numerator || !fi->interval.denominator)) {
>> +               fi->interval.numerator = 1;
>> +               fi->interval.denominator = 60;
>> +       } else {
>> +               /* Clamp FPS to 1-240 range */
>> +               fps = fi->interval.denominator / fi->interval.numerator;
>> +               fps = clamp(fps, 1U, 240U);
>> +
>> +               fi->interval.numerator = 1;
>> +               fi->interval.denominator = fps;
>> +       }
>> +
>> +       vsensor->frame_interval = fi->interval;
>> +
>> +       /* Update hardware timing configuration */
>> +       vsensor->hw.fps_jiffies = (HZ * vsensor->frame_interval.numerator) /
>> +                                 vsensor->frame_interval.denominator;
>> +
>> +       return 0;
>> +}
>>
>>  static const struct v4l2_subdev_pad_ops vimc_sensor_pad_ops = {
>>         .enum_mbus_code         = vimc_sensor_enum_mbus_code,
>>         .enum_frame_size        = vimc_sensor_enum_frame_size,
>>         .get_fmt                = v4l2_subdev_get_fmt,
>>         .set_fmt                = vimc_sensor_set_fmt,
>> +       .get_frame_interval     = vimc_sensor_get_frame_interval,
>> +       .set_frame_interval     = vimc_sensor_set_frame_interval,
>>  };
>>
>>  static void *vimc_sensor_process_frame(struct vimc_ent_device *ved,
>> @@ -400,6 +441,10 @@ static struct vimc_ent_device *vimc_sensor_add(struct vimc_device *vimc,
>>
>>         vsensor->ved.process_frame = vimc_sensor_process_frame;
>>         vsensor->ved.dev = vimc->mdev.dev;
>> +       /* Initialize to 60 FPS */
>> +       vsensor->frame_interval.numerator = 1;
>> +       vsensor->frame_interval.denominator = 60;
>> +       vsensor->hw.fps_jiffies = HZ / 60;
>>
>>         return &vsensor->ved;
>>
>> --
>> 2.43.0
>>

Re: [PATCH 2/3] media: vimc: sensor: Add frame rate configuration support

[Laurent Pinchart]

On Fri, Jan 16, 2026 at 12:35:47PM -0800, Faizel K B wrote:
> On Fri, Jan 16, 2026 at 09:53:00AM +0000, Kieran Bingham wrote:
> > Hi Faizel,
> >
> > Pullling in the libcamera-devel mailing list for relevant awareness too.
> >
> > Quoting faizel.kb@gmail.com (2026-01-15 19:10:49)
> >> From: Faizel K B <faizel.kb@gmail.com>
> >>
> >> Implement set_frame_interval and get_frame_interval callbacks in
> >> v4l2_subdev_pad_ops to enable frame rate configuration from 1-240 FPS.
> >> The default frame rate is 60 FPS.
> >>
> >> The configured frame interval is pre-calculated into jiffies and
> >> stored in the sensor's hw structure for efficient access by the
> >> streamer thread.
> >
> > This is a really interesting and helpful bit of work - but I'm weary it
> > might have taken one small mis-direction.
> >
> > VIMC aims to replicate the usage of camera sensors, and I don't think
> > we're supposed to use frame interval interface for that.
> >
> > Instead following the camera sensor model, this should be implemented to
> > mirror what the hardware in camera sensors actually does/needs which is
> > to make the frame rate a function of the exposure time and blankings
> > (both hblank and vblank).
> >
> > We have a bit of a write up about the calculation at the bottom of this
> > page:
> >
> > - https://libcamera.org/camera-sensor-model.html
> >
> > Or perhaps the sensor requirements of libcamera might be what a virtual
> > sensor should aim for:
> >
> > - https://libcamera.org/sensor_driver_requirements.html
> 
> Thanks for the review and links. It makes sense to implement the pixelrate, 
> v/h blanking from a sensor point of view.The exposure value also
> matters. Sensors even prioritize the exposure to the frame timings or the
> other way. This parameters can be added inside using v4l2_ctrl_new_std(),
> for pixel rate,blanking and exposure and calculate the sleep time for the
> frame rate delay in the streamer thread. By this, a user need to do few
> calculations to adjust the frame rate.
> 
> > There's probably more helpful documentation in the linux kernel too in
> > the upcoming sensor model rework from Sakari.
> >
> > Aha - in fact I found this:
> > - https://www.kernel.org/doc/html/latest/userspace-api/media/drivers/camera-sensor.html#raw-camera-sensors
> >
> > "2.2. Frame interval configuration
> >
> > There are two different methods for obtaining possibilities for
> > different frame intervals as well as configuring the frame interval.
> > Which one to implement depends on the type of the device."
> >
> > So perhaps the question we need to clarify is "what type of device is
> > vimc-sensor".
> >
> > I'm likely biased to presume it's a virtual raw camera sensor ... But is
> > it ?
> 
> Would like to hear more about this.

Given that the vimc driver exposes raw bayer formats, and that the
sensor is connected to an entity called debayer, it's safe to say it
should be considered as a raw sensor. This is how libcamera uses vimc,
and I think it's the most important use case for the driver.

This being said, the media graph exposed by the vimc driver doesn't map
exactly to how a raw sensor connected to an inline ISP would look like.
It would be nice to improve this at some point.

> >> Signed-off-by: Faizel K B <faizel.kb@gmail.com>
> >> ---
> >>  drivers/media/test-drivers/vimc/vimc-common.h |  2 +
> >>  drivers/media/test-drivers/vimc/vimc-sensor.c | 45 +++++++++++++++++++
> >>  2 files changed, 47 insertions(+)
> >>
> >> diff --git a/drivers/media/test-drivers/vimc/vimc-common.h b/drivers/media/test-drivers/vimc/vimc-common.h
> >> index 7f5f008e407b..a71ef761fa77 100644
> >> --- a/drivers/media/test-drivers/vimc/vimc-common.h
> >> +++ b/drivers/media/test-drivers/vimc/vimc-common.h
> >> @@ -173,6 +173,7 @@ struct vimc_sensor_device {
> >>         struct tpg_data tpg;
> >>         struct v4l2_ctrl_handler hdl;
> >>         struct media_pad pad;
> >> +       struct v4l2_fract frame_interval;
> >>
> >>         u8 *frame;
> >>
> >> @@ -184,6 +185,7 @@ struct vimc_sensor_device {
> >>                 struct v4l2_area size;
> >>                 enum vimc_sensor_osd_mode osd_value;
> >>                 u64 start_stream_ts;
> >> +               unsigned long fps_jiffies;
> >>         } hw;
> >>  };
> >>
> >> diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
> >> index 2b07dc1f1278..d54425ff28a1 100644
> >> --- a/drivers/media/test-drivers/vimc/vimc-sensor.c
> >> +++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
> >> @@ -140,12 +140,53 @@ static int vimc_sensor_set_fmt(struct v4l2_subdev *sd,
> >>
> >>         return 0;
> >>  }
> >> +static int vimc_sensor_get_frame_interval(struct v4l2_subdev *sd,
> >> +                                         struct v4l2_subdev_state *state,
> >> +                                         struct v4l2_subdev_frame_interval *fi)
> >> +{
> >> +       struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
> >> +
> >> +       fi->interval = vsensor->frame_interval;
> >> +
> >> +       return 0;
> >> +}
> >> +
> >> +static int vimc_sensor_set_frame_interval(struct v4l2_subdev *sd,
> >> +                                         struct v4l2_subdev_state *state,
> >> +                                         struct v4l2_subdev_frame_interval *fi)
> >> +{
> >> +       struct vimc_sensor_device *vsensor = v4l2_get_subdevdata(sd);
> >> +       u32 fps;
> >> +
> >> +       /* Sanitize to default if invalid */
> >> +       if (unlikely(!fi->interval.numerator || !fi->interval.denominator)) {
> >> +               fi->interval.numerator = 1;
> >> +               fi->interval.denominator = 60;
> >> +       } else {
> >> +               /* Clamp FPS to 1-240 range */
> >> +               fps = fi->interval.denominator / fi->interval.numerator;
> >> +               fps = clamp(fps, 1U, 240U);
> >> +
> >> +               fi->interval.numerator = 1;
> >> +               fi->interval.denominator = fps;
> >> +       }
> >> +
> >> +       vsensor->frame_interval = fi->interval;
> >> +
> >> +       /* Update hardware timing configuration */
> >> +       vsensor->hw.fps_jiffies = (HZ * vsensor->frame_interval.numerator) /
> >> +                                 vsensor->frame_interval.denominator;
> >> +
> >> +       return 0;
> >> +}
> >>
> >>  static const struct v4l2_subdev_pad_ops vimc_sensor_pad_ops = {
> >>         .enum_mbus_code         = vimc_sensor_enum_mbus_code,
> >>         .enum_frame_size        = vimc_sensor_enum_frame_size,
> >>         .get_fmt                = v4l2_subdev_get_fmt,
> >>         .set_fmt                = vimc_sensor_set_fmt,
> >> +       .get_frame_interval     = vimc_sensor_get_frame_interval,
> >> +       .set_frame_interval     = vimc_sensor_set_frame_interval,
> >>  };
> >>
> >>  static void *vimc_sensor_process_frame(struct vimc_ent_device *ved,
> >> @@ -400,6 +441,10 @@ static struct vimc_ent_device *vimc_sensor_add(struct vimc_device *vimc,
> >>
> >>         vsensor->ved.process_frame = vimc_sensor_process_frame;
> >>         vsensor->ved.dev = vimc->mdev.dev;
> >> +       /* Initialize to 60 FPS */
> >> +       vsensor->frame_interval.numerator = 1;
> >> +       vsensor->frame_interval.denominator = 60;
> >> +       vsensor->hw.fps_jiffies = HZ / 60;
> >>
> >>         return &vsensor->ved;
> >>

-- 
Regards,

Laurent Pinchart

Re: [PATCH 2/3] media: vimc: sensor: Add frame rate configuration support

[Faizel K B]

On Sat, Jan 17, 2026 at 06:44:13AM +0200, Laurent Pinchart wrote:
>On Fri, Jan 16, 2026 at 12:35:47PM -0800, Faizel K B wrote:
>> On Fri, Jan 16, 2026 at 09:53:00AM +0000, Kieran Bingham wrote:
>> > Hi Faizel,
>> >
>> > Pullling in the libcamera-devel mailing list for relevant awareness too.
>> >
>> > Quoting faizel.kb@gmail.com (2026-01-15 19:10:49)
>> >> From: Faizel K B <faizel.kb@gmail.com>
>> >>
>> >> Implement set_frame_interval and get_frame_interval callbacks in
>> >> v4l2_subdev_pad_ops to enable frame rate configuration from 1-240 FPS.
>> >> The default frame rate is 60 FPS.
>> >>
>> >> The configured frame interval is pre-calculated into jiffies and
>> >> stored in the sensor's hw structure for efficient access by the
>> >> streamer thread.
>> >
>> > This is a really interesting and helpful bit of work - but I'm weary it
>> > might have taken one small mis-direction.
>> >
>> > VIMC aims to replicate the usage of camera sensors, and I don't think
>> > we're supposed to use frame interval interface for that.
>> >
>> > Instead following the camera sensor model, this should be implemented to
>> > mirror what the hardware in camera sensors actually does/needs which is
>> > to make the frame rate a function of the exposure time and blankings
>> > (both hblank and vblank).
>> >
>> > We have a bit of a write up about the calculation at the bottom of this
>> > page:
>> >
>> > - https://libcamera.org/camera-sensor-model.html
>> >
>> > Or perhaps the sensor requirements of libcamera might be what a virtual
>> > sensor should aim for:
>> >
>> > - https://libcamera.org/sensor_driver_requirements.html
>>
>> Thanks for the review and links. It makes sense to implement the pixelrate,
>> v/h blanking from a sensor point of view.The exposure value also
>> matters. Sensors even prioritize the exposure to the frame timings or the
>> other way. This parameters can be added inside using v4l2_ctrl_new_std(),
>> for pixel rate,blanking and exposure and calculate the sleep time for the
>> frame rate delay in the streamer thread. By this, a user need to do few
>> calculations to adjust the frame rate.
>>
>> > There's probably more helpful documentation in the linux kernel too in
>> > the upcoming sensor model rework from Sakari.
>> >
>> > Aha - in fact I found this:
>> > - https://www.kernel.org/doc/html/latest/userspace-api/media/drivers/camera-sensor.html#raw-camera-sensors
>> >
>> > "2.2. Frame interval configuration
>> >
>> > There are two different methods for obtaining possibilities for
>> > different frame intervals as well as configuring the frame interval.
>> > Which one to implement depends on the type of the device."
>> >
>> > So perhaps the question we need to clarify is "what type of device is
>> > vimc-sensor".
>> >
>> > I'm likely biased to presume it's a virtual raw camera sensor ... But is
>> > it ?
>>
>> Would like to hear more about this.
>
>Given that the vimc driver exposes raw bayer formats, and that the
>sensor is connected to an entity called debayer, it's safe to say it
>should be considered as a raw sensor. This is how libcamera uses vimc,
>and I think it's the most important use case for the driver.
>
>This being said, the media graph exposed by the vimc driver doesn't map
>exactly to how a raw sensor connected to an inline ISP would look like.
>It would be nice to improve this at some point.
>

Ok. Thanks. I will send a new patch with frame rate controlled from 

V4L2_CID_PIXEL_RATE
V4L2_CID_HBLANK 

instead of frame interval.Both the above as read only as to closely
match the hardware

V4L2_CID_VBLANK - As read/write, to control the actual frame rate

fps = pixel_rate / ((width+hblank) * (height+vblank))