[PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

The previous frame interval formula used analogue crop dimensions. This
breaks down for some sensors when binning.

For example in imx219 the minimum FLL (frame length in lines) can be
lower than the analogue crop height when binning, which would require a
negative VBLANK to represent the actual timing. Similarly, imx283 allows
a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
the analogue crop width of the full resolution mode.

The CCS specification also describes under section "8.2.6 Line Length
and Frame Length" how the horizontal and vertical readout minimums can
be different when binning.

Replace the formula with the underlying hardware concepts of LLP (line
length in pixels) and FLL (frame length in lines). These terms were
chosen to match the CCS specification on raw sensors, as it is a cleaner
reference compared to a typical sensor vendor datasheet.

Finally, define the blanking controls relative to the active pixel
readout (post-binning) rather than the analogue crop size. This matches
what most sensor drivers already do, and also what applications like
libcamera expect. In "Figure 42" of CCS specification too, we see a
similar definition:

  frame interval = (output width + HBLANK) *
                   (output height + VBLANK) / pixel rate

Also add a note in the "Writing camera sensor drivers" guide, to ensure
this formula is followed by new sensor drivers.

Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
---
 Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
 .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
 2 files changed, 53 insertions(+), 17 deletions(-)

diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
--- a/Documentation/driver-api/media/camera-sensor.rst
+++ b/Documentation/driver-api/media/camera-sensor.rst
@@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
 runtime PM was disabled, in either of which cases the driver may proceed to
 access the device.
 
+Frame interval
+--------------
+
+If a sensor supports cropping or binning, it is the sensor driver's
+responsibility to ensure that the frame interval formula (see
+:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
+pipeline configuration. The driver shall adjust the minimum and maximum allowed
+values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
+changes, so that application developers can always rely on the same formula to
+calculate the frame interval.
+
 Rotation, orientation and flipping
 ----------------------------------
 
diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -49,35 +49,60 @@ depends on the type of the device.
 Raw camera sensors
 ~~~~~~~~~~~~~~~~~~
 
-Instead of a high level parameter such as frame interval, the frame interval is
-a result of the configuration of a number of camera sensor implementation
-specific parameters. Luckily, these parameters tend to be the same for more or
-less all modern raw camera sensors.
+Instead of a high level parameter such as frame interval, the frame interval on
+a raw camera sensor is determined by a number of sensor-specific parameters.
+These parameters tend to be common across most modern raw camera sensors.
 
-The frame interval is calculated using the following equation::
+The pixel array is the full grid of photosensitive elements on the sensor. A
+subregion of it is selected by the analogue crop. The cropped image may then be
+subject to binning (averaging of a NxN block) or subsampling which
+further reduce the image dimensions. The resulting image is then read out by
+the ADC (analogue-to-digital converter) line by line. After ADC readout,
+optional digital crop or scaling may further reduce the image dimensions, see
+:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
 
-	frame interval = (analogue crop width + horizontal blanking) *
-			 (analogue crop height + vertical blanking) / pixel rate
+The frame size is determined by two timing parameters: line length in pixels
+(LLP) and frame length in lines (FLL). These are fundamental sensor timing
+registers that control how fast the ADC reads out the image. They may go
+by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
+VTOTAL, or similar.
 
-The formula is bus independent and is applicable for raw timing parameters on
-large variety of devices beyond camera sensors. Devices that have no analogue
-crop, use the full source image size, i.e. pixel array size.
+LLP is the total number of pixel clock cycles per line, including both the
+active readout width and horizontal blanking. FLL is the total number of lines
+per frame, including both the active readout height and vertical blanking.
+
+The frame interval is::
+
+        frame interval = LLP * FLL / pixel rate
 
 Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
 ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
 is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
-the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
-sub-device. The unit of that control is pixels per second.
+the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
+same sub-device. The unit of that control is pixels per second.
+
+The blanking is defined relative to the size of the image being sent out to the
+host over the bus (like CSI-2)::
+
+        LLP = active width + V4L2_CID_HBLANK
+        FLL = active height + V4L2_CID_VBLANK
+
+The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
+``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
+range permitted by the sensor hardware for the current mode. Sensors that
+support binning often define a lower minimum for LLP or FLL registers, which
+can help achieve higher framerates when binning.
+
+Application developers can calculate the frame interval using the output
+dimensions and the blanking controls::
+
+        frame interval = (output width + horizontal blanking) *
+                         (output height + vertical blanking) / pixel rate
 
 Register list-based drivers need to implement read-only sub-device nodes for the
 purpose. Devices that are not register list based need these to configure the
 device's internal processing pipeline.
 
-The first entity in the linear pipeline is the pixel array. The pixel array may
-be followed by other entities that are there to allow configuring binning,
-skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
-<VIDIOC_SUBDEV_G_SELECTION>`.
-
 USB cameras etc. devices
 ~~~~~~~~~~~~~~~~~~~~~~~~
 

---
base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
change-id: 20260219-media-fps-docs-fd1da722cc38

Best regards,
-- 
Jai Luthra <jai.luthra@ideasonboard.com>

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Sakari Ailus]

Hi Jai,

Thanks for the patch.

On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> The previous frame interval formula used analogue crop dimensions. This
> breaks down for some sensors when binning.
> 
> For example in imx219 the minimum FLL (frame length in lines) can be
> lower than the analogue crop height when binning, which would require a
> negative VBLANK to represent the actual timing. Similarly, imx283 allows
> a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> the analogue crop width of the full resolution mode.

V4L2 integer controls are signed so using negative numbers is a non-issue.
CCS already does this in some cases actually.

> 
> The CCS specification also describes under section "8.2.6 Line Length
> and Frame Length" how the horizontal and vertical readout minimums can
> be different when binning.
> 
> Replace the formula with the underlying hardware concepts of LLP (line
> length in pixels) and FLL (frame length in lines). These terms were
> chosen to match the CCS specification on raw sensors, as it is a cleaner
> reference compared to a typical sensor vendor datasheet.
> 
> Finally, define the blanking controls relative to the active pixel
> readout (post-binning) rather than the analogue crop size. This matches
> what most sensor drivers already do, and also what applications like
> libcamera expect. In "Figure 42" of CCS specification too, we see a
> similar definition:
> 
>   frame interval = (output width + HBLANK) *
>                    (output height + VBLANK) / pixel rate
> 
> Also add a note in the "Writing camera sensor drivers" guide, to ensure
> this formula is followed by new sensor drivers.

We're about to require implementing the common raw sensor model soon by
essentially all new drivers so documenting the soon-to-be-obsolete state
has limited benefits.

But the UAPI documentation remains relevant for quite some time, please see
my comments below.

> 
> Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> ---
>  Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
>  .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
>  2 files changed, 53 insertions(+), 17 deletions(-)
> 
> diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
> index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
> --- a/Documentation/driver-api/media/camera-sensor.rst
> +++ b/Documentation/driver-api/media/camera-sensor.rst
> @@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
>  runtime PM was disabled, in either of which cases the driver may proceed to
>  access the device.
>  
> +Frame interval
> +--------------
> +
> +If a sensor supports cropping or binning, it is the sensor driver's
> +responsibility to ensure that the frame interval formula (see
> +:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
> +pipeline configuration. The driver shall adjust the minimum and maximum allowed
> +values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
> +changes, so that application developers can always rely on the same formula to
> +calculate the frame interval.
> +
>  Rotation, orientation and flipping
>  ----------------------------------
>  
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -49,35 +49,60 @@ depends on the type of the device.
>  Raw camera sensors
>  ~~~~~~~~~~~~~~~~~~
>  
> -Instead of a high level parameter such as frame interval, the frame interval is
> -a result of the configuration of a number of camera sensor implementation
> -specific parameters. Luckily, these parameters tend to be the same for more or
> -less all modern raw camera sensors.
> +Instead of a high level parameter such as frame interval, the frame interval on
> +a raw camera sensor is determined by a number of sensor-specific parameters.
> +These parameters tend to be common across most modern raw camera sensors.
>  
> -The frame interval is calculated using the following equation::
> +The pixel array is the full grid of photosensitive elements on the sensor. A

s/sensor/camera sensor/

> +subregion of it is selected by the analogue crop. The cropped image may then be
> +subject to binning (averaging of a NxN block) or subsampling which
> +further reduce the image dimensions. The resulting image is then read out by
> +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> +optional digital crop or scaling may further reduce the image dimensions, see
> +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
>  
> -	frame interval = (analogue crop width + horizontal blanking) *
> -			 (analogue crop height + vertical blanking) / pixel rate
> +The frame size is determined by two timing parameters: line length in pixels
> +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> +registers that control how fast the ADC reads out the image. They may go
> +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> +VTOTAL, or similar.
>  
> -The formula is bus independent and is applicable for raw timing parameters on
> -large variety of devices beyond camera sensors. Devices that have no analogue
> -crop, use the full source image size, i.e. pixel array size.
> +LLP is the total number of pixel clock cycles per line, including both the
> +active readout width and horizontal blanking. FLL is the total number of lines
> +per frame, including both the active readout height and vertical blanking.
> +
> +The frame interval is::
> +
> +        frame interval = LLP * FLL / pixel rate

How would this look like if you spell out LLP and FLL? The rest aren't
abbreviated either.

>  
>  Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
>  ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
>  is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> -sub-device. The unit of that control is pixels per second.
> +the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
> +same sub-device. The unit of that control is pixels per second.
> +
> +The blanking is defined relative to the size of the image being sent out to the
> +host over the bus (like CSI-2)::
> +
> +        LLP = active width + V4L2_CID_HBLANK
> +        FLL = active height + V4L2_CID_VBLANK
> +
> +The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
> +``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
> +range permitted by the sensor hardware for the current mode. Sensors that
> +support binning often define a lower minimum for LLP or FLL registers, which
> +can help achieve higher framerates when binning.
> +
> +Application developers can calculate the frame interval using the output
> +dimensions and the blanking controls::
> +
> +        frame interval = (output width + horizontal blanking) *
> +                         (output height + vertical blanking) / pixel rate

This is indeed what many presumably non-CCS sensor drivers implement. Are
there any that would use the crop rectangle (the imx219 doesn't seem to)?
For user space variance in this area is of course bad.

The common raw sensor model introduces two new controls for the purpose so
we could re-purpose the old VBLANK/HBLANK controls for this -- apart from
the CCS driver.

>  
>  Register list-based drivers need to implement read-only sub-device nodes for the
>  purpose. Devices that are not register list based need these to configure the
>  device's internal processing pipeline.
>  
> -The first entity in the linear pipeline is the pixel array. The pixel array may
> -be followed by other entities that are there to allow configuring binning,
> -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> -<VIDIOC_SUBDEV_G_SELECTION>`.
> -
>  USB cameras etc. devices
>  ~~~~~~~~~~~~~~~~~~~~~~~~
>  
> 
> ---
> base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> change-id: 20260219-media-fps-docs-fd1da722cc38
> 
> Best regards,

-- 
Kind regards,

Sakari Ailus

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

Hi Sakari,

Thanks for the review.

Quoting Sakari Ailus (2026-02-19 15:21:10)
> Hi Jai,
> 
> Thanks for the patch.
> 
> On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > The previous frame interval formula used analogue crop dimensions. This
> > breaks down for some sensors when binning.
> > 
> > For example in imx219 the minimum FLL (frame length in lines) can be
> > lower than the analogue crop height when binning, which would require a
> > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > the analogue crop width of the full resolution mode.
> 
> V4L2 integer controls are signed so using negative numbers is a non-issue.
> CCS already does this in some cases actually.
> 

Ah, I see. Control being negative was not the only problem, as it would
also diverge from what sensors and applications already do, but I see you
agreed on that below :-)

I'll update the commit message to make this clear(er).

> > 
> > The CCS specification also describes under section "8.2.6 Line Length
> > and Frame Length" how the horizontal and vertical readout minimums can
> > be different when binning.
> > 
> > Replace the formula with the underlying hardware concepts of LLP (line
> > length in pixels) and FLL (frame length in lines). These terms were
> > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > reference compared to a typical sensor vendor datasheet.
> > 
> > Finally, define the blanking controls relative to the active pixel
> > readout (post-binning) rather than the analogue crop size. This matches
> > what most sensor drivers already do, and also what applications like
> > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > similar definition:
> > 
> >   frame interval = (output width + HBLANK) *
> >                    (output height + VBLANK) / pixel rate
> > 
> > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > this formula is followed by new sensor drivers.
> 
> We're about to require implementing the common raw sensor model soon by
> essentially all new drivers so documenting the soon-to-be-obsolete state
> has limited benefits.

Makes sense.

> 
> But the UAPI documentation remains relevant for quite some time, please see
> my comments below.
> 
> > 
> > Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> > ---
> >  Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
> >  .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
> >  2 files changed, 53 insertions(+), 17 deletions(-)
> > 
> > diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
> > index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
> > --- a/Documentation/driver-api/media/camera-sensor.rst
> > +++ b/Documentation/driver-api/media/camera-sensor.rst
> > @@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
> >  runtime PM was disabled, in either of which cases the driver may proceed to
> >  access the device.
> >  
> > +Frame interval
> > +--------------
> > +
> > +If a sensor supports cropping or binning, it is the sensor driver's
> > +responsibility to ensure that the frame interval formula (see
> > +:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
> > +pipeline configuration. The driver shall adjust the minimum and maximum allowed
> > +values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
> > +changes, so that application developers can always rely on the same formula to
> > +calculate the frame interval.
> > +
> >  Rotation, orientation and flipping
> >  ----------------------------------
> >  
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -49,35 +49,60 @@ depends on the type of the device.
> >  Raw camera sensors
> >  ~~~~~~~~~~~~~~~~~~
> >  
> > -Instead of a high level parameter such as frame interval, the frame interval is
> > -a result of the configuration of a number of camera sensor implementation
> > -specific parameters. Luckily, these parameters tend to be the same for more or
> > -less all modern raw camera sensors.
> > +Instead of a high level parameter such as frame interval, the frame interval on
> > +a raw camera sensor is determined by a number of sensor-specific parameters.
> > +These parameters tend to be common across most modern raw camera sensors.
> >  
> > -The frame interval is calculated using the following equation::
> > +The pixel array is the full grid of photosensitive elements on the sensor. A
> 
> s/sensor/camera sensor/
> 

Will fix.

> > +subregion of it is selected by the analogue crop. The cropped image may then be
> > +subject to binning (averaging of a NxN block) or subsampling which
> > +further reduce the image dimensions. The resulting image is then read out by
> > +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> > +optional digital crop or scaling may further reduce the image dimensions, see
> > +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
> >  
> > -     frame interval = (analogue crop width + horizontal blanking) *
> > -                      (analogue crop height + vertical blanking) / pixel rate
> > +The frame size is determined by two timing parameters: line length in pixels
> > +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> > +registers that control how fast the ADC reads out the image. They may go
> > +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> > +VTOTAL, or similar.
> >  
> > -The formula is bus independent and is applicable for raw timing parameters on
> > -large variety of devices beyond camera sensors. Devices that have no analogue
> > -crop, use the full source image size, i.e. pixel array size.
> > +LLP is the total number of pixel clock cycles per line, including both the
> > +active readout width and horizontal blanking. FLL is the total number of lines
> > +per frame, including both the active readout height and vertical blanking.
> > +
> > +The frame interval is::
> > +
> > +        frame interval = LLP * FLL / pixel rate
> 
> How would this look like if you spell out LLP and FLL? The rest aren't
> abbreviated either.

        frame interval = (line length in pixels) *
                         (frame length in lines) / pixel rate

This reads much easier, thanks for the suggestion, will update it in v2.

> 
> >  
> >  Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> >  ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> >  is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> > -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> > -sub-device. The unit of that control is pixels per second.
> > +the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
> > +same sub-device. The unit of that control is pixels per second.
> > +
> > +The blanking is defined relative to the size of the image being sent out to the
> > +host over the bus (like CSI-2)::
> > +
> > +        LLP = active width + V4L2_CID_HBLANK
> > +        FLL = active height + V4L2_CID_VBLANK
> > +
> > +The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
> > +``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
> > +range permitted by the sensor hardware for the current mode. Sensors that
> > +support binning often define a lower minimum for LLP or FLL registers, which
> > +can help achieve higher framerates when binning.
> > +
> > +Application developers can calculate the frame interval using the output
> > +dimensions and the blanking controls::
> > +
> > +        frame interval = (output width + horizontal blanking) *
> > +                         (output height + vertical blanking) / pixel rate
> 
> This is indeed what many presumably non-CCS sensor drivers implement. Are
> there any that would use the crop rectangle (the imx219 doesn't seem to)?
> For user space variance in this area is of course bad.

I am not aware of any, but I haven't checked all drivers.

The ones I checked (IMX219, IMX283, IMX335, OV5647, OV5640), all set:
[h/v]blank = [h/v]total - mode->[width/height]

So IMO updating the documentation to match that is simpler.

> The common raw sensor model introduces two new controls for the purpose so
> we could re-purpose the old VBLANK/HBLANK controls for this -- apart from
> the CCS driver.
> 

I guess that's my final cue to finally take out time and read that series
properly :-)

Thanks,
    Jai

> >  
> >  Register list-based drivers need to implement read-only sub-device nodes for the
> >  purpose. Devices that are not register list based need these to configure the
> >  device's internal processing pipeline.
> >  
> > -The first entity in the linear pipeline is the pixel array. The pixel array may
> > -be followed by other entities that are there to allow configuring binning,
> > -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> > -<VIDIOC_SUBDEV_G_SELECTION>`.
> > -
> >  USB cameras etc. devices
> >  ~~~~~~~~~~~~~~~~~~~~~~~~
> >  
> > 
> > ---
> > base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> > change-id: 20260219-media-fps-docs-fd1da722cc38
> > 
> > Best regards,
> 
> -- 
> Kind regards,
> 
> Sakari Ailus

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Sakari Ailus]

Hi Jai,

On Thu, Feb 19, 2026 at 08:26:43PM +0530, Jai Luthra wrote:
> Hi Sakari,
> 
> Thanks for the review.
> 
> Quoting Sakari Ailus (2026-02-19 15:21:10)
> > Hi Jai,
> > 
> > Thanks for the patch.
> > 
> > On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > > The previous frame interval formula used analogue crop dimensions. This
> > > breaks down for some sensors when binning.
> > > 
> > > For example in imx219 the minimum FLL (frame length in lines) can be
> > > lower than the analogue crop height when binning, which would require a
> > > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > > the analogue crop width of the full resolution mode.
> > 
> > V4L2 integer controls are signed so using negative numbers is a non-issue.
> > CCS already does this in some cases actually.
> > 
> 
> Ah, I see. Control being negative was not the only problem, as it would
> also diverge from what sensors and applications already do, but I see you
> agreed on that below :-)
> 
> I'll update the commit message to make this clear(er).

Ack.

> 
> > > 
> > > The CCS specification also describes under section "8.2.6 Line Length
> > > and Frame Length" how the horizontal and vertical readout minimums can
> > > be different when binning.
> > > 
> > > Replace the formula with the underlying hardware concepts of LLP (line
> > > length in pixels) and FLL (frame length in lines). These terms were
> > > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > > reference compared to a typical sensor vendor datasheet.
> > > 
> > > Finally, define the blanking controls relative to the active pixel
> > > readout (post-binning) rather than the analogue crop size. This matches
> > > what most sensor drivers already do, and also what applications like
> > > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > > similar definition:
> > > 
> > >   frame interval = (output width + HBLANK) *
> > >                    (output height + VBLANK) / pixel rate
> > > 
> > > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > > this formula is followed by new sensor drivers.
> > 
> > We're about to require implementing the common raw sensor model soon by
> > essentially all new drivers so documenting the soon-to-be-obsolete state
> > has limited benefits.
> 
> Makes sense.
> 
> > 
> > But the UAPI documentation remains relevant for quite some time, please see
> > my comments below.
> > 
> > > 
> > > Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> > > ---
> > >  Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
> > >  .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
> > >  2 files changed, 53 insertions(+), 17 deletions(-)
> > > 
> > > diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
> > > index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
> > > --- a/Documentation/driver-api/media/camera-sensor.rst
> > > +++ b/Documentation/driver-api/media/camera-sensor.rst
> > > @@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
> > >  runtime PM was disabled, in either of which cases the driver may proceed to
> > >  access the device.
> > >  
> > > +Frame interval
> > > +--------------
> > > +
> > > +If a sensor supports cropping or binning, it is the sensor driver's
> > > +responsibility to ensure that the frame interval formula (see
> > > +:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
> > > +pipeline configuration. The driver shall adjust the minimum and maximum allowed
> > > +values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
> > > +changes, so that application developers can always rely on the same formula to
> > > +calculate the frame interval.
> > > +
> > >  Rotation, orientation and flipping
> > >  ----------------------------------
> > >  
> > > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
> > > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > @@ -49,35 +49,60 @@ depends on the type of the device.
> > >  Raw camera sensors
> > >  ~~~~~~~~~~~~~~~~~~
> > >  
> > > -Instead of a high level parameter such as frame interval, the frame interval is
> > > -a result of the configuration of a number of camera sensor implementation
> > > -specific parameters. Luckily, these parameters tend to be the same for more or
> > > -less all modern raw camera sensors.
> > > +Instead of a high level parameter such as frame interval, the frame interval on
> > > +a raw camera sensor is determined by a number of sensor-specific parameters.
> > > +These parameters tend to be common across most modern raw camera sensors.
> > >  
> > > -The frame interval is calculated using the following equation::
> > > +The pixel array is the full grid of photosensitive elements on the sensor. A
> > 
> > s/sensor/camera sensor/
> > 
> 
> Will fix.
> 
> > > +subregion of it is selected by the analogue crop. The cropped image may then be
> > > +subject to binning (averaging of a NxN block) or subsampling which
> > > +further reduce the image dimensions. The resulting image is then read out by
> > > +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> > > +optional digital crop or scaling may further reduce the image dimensions, see
> > > +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
> > >  
> > > -     frame interval = (analogue crop width + horizontal blanking) *
> > > -                      (analogue crop height + vertical blanking) / pixel rate
> > > +The frame size is determined by two timing parameters: line length in pixels
> > > +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> > > +registers that control how fast the ADC reads out the image. They may go
> > > +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> > > +VTOTAL, or similar.
> > >  
> > > -The formula is bus independent and is applicable for raw timing parameters on
> > > -large variety of devices beyond camera sensors. Devices that have no analogue
> > > -crop, use the full source image size, i.e. pixel array size.
> > > +LLP is the total number of pixel clock cycles per line, including both the
> > > +active readout width and horizontal blanking. FLL is the total number of lines
> > > +per frame, including both the active readout height and vertical blanking.
> > > +
> > > +The frame interval is::
> > > +
> > > +        frame interval = LLP * FLL / pixel rate
> > 
> > How would this look like if you spell out LLP and FLL? The rest aren't
> > abbreviated either.
> 
>         frame interval = (line length in pixels) *
>                          (frame length in lines) / pixel rate
> 
> This reads much easier, thanks for the suggestion, will update it in v2.
> 
> > 
> > >  
> > >  Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> > >  ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> > >  is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> > > -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> > > -sub-device. The unit of that control is pixels per second.
> > > +the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
> > > +same sub-device. The unit of that control is pixels per second.
> > > +
> > > +The blanking is defined relative to the size of the image being sent out to the
> > > +host over the bus (like CSI-2)::
> > > +
> > > +        LLP = active width + V4L2_CID_HBLANK
> > > +        FLL = active height + V4L2_CID_VBLANK
> > > +
> > > +The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
> > > +``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
> > > +range permitted by the sensor hardware for the current mode. Sensors that
> > > +support binning often define a lower minimum for LLP or FLL registers, which
> > > +can help achieve higher framerates when binning.
> > > +
> > > +Application developers can calculate the frame interval using the output
> > > +dimensions and the blanking controls::
> > > +
> > > +        frame interval = (output width + horizontal blanking) *
> > > +                         (output height + vertical blanking) / pixel rate
> > 
> > This is indeed what many presumably non-CCS sensor drivers implement. Are
> > there any that would use the crop rectangle (the imx219 doesn't seem to)?
> > For user space variance in this area is of course bad.
> 
> I am not aware of any, but I haven't checked all drivers.
> 
> The ones I checked (IMX219, IMX283, IMX335, OV5647, OV5640), all set:
> [h/v]blank = [h/v]total - mode->[width/height]
> 
> So IMO updating the documentation to match that is simpler.

I agree. I was just wondering how consistently this was done over the
years.

> 
> > The common raw sensor model introduces two new controls for the purpose so
> > we could re-purpose the old VBLANK/HBLANK controls for this -- apart from
> > the CCS driver.
> > 
> 
> I guess that's my final cue to finally take out time and read that series
> properly :-)

The two new controls will be part of the new version which I hope to post
soon. I'll cc you when I do.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jacopo Mondi]

Hi Jai

On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> The previous frame interval formula used analogue crop dimensions. This
> breaks down for some sensors when binning.
>
> For example in imx219 the minimum FLL (frame length in lines) can be
> lower than the analogue crop height when binning, which would require a
> negative VBLANK to represent the actual timing. Similarly, imx283 allows
> a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> the analogue crop width of the full resolution mode.
>
> The CCS specification also describes under section "8.2.6 Line Length
> and Frame Length" how the horizontal and vertical readout minimums can
> be different when binning.
>
> Replace the formula with the underlying hardware concepts of LLP (line
> length in pixels) and FLL (frame length in lines). These terms were
> chosen to match the CCS specification on raw sensors, as it is a cleaner
> reference compared to a typical sensor vendor datasheet.
>
> Finally, define the blanking controls relative to the active pixel
> readout (post-binning) rather than the analogue crop size. This matches
> what most sensor drivers already do, and also what applications like
> libcamera expect. In "Figure 42" of CCS specification too, we see a
> similar definition:
>
>   frame interval = (output width + HBLANK) *
>                    (output height + VBLANK) / pixel rate
>
> Also add a note in the "Writing camera sensor drivers" guide, to ensure
> this formula is followed by new sensor drivers.

I agree that using the analogue crop rectangle sizes is not correct,
however, with this new formulation the LLP and FLL values might be
smaller than the crop rectangles reported through the selection API,
which I'm not sure it's great from a consistency point of view ?

Also the below suggested formulation:

        LLP = active width + V4L2_CID_HBLANK
        FLL = active height + V4L2_CID_VBLANK

Assumes the combined effect of [binning + subsampling] always directly
affect the readout time of pixels on the pixel array. The CCS specs
and driver seems to suggest that's not always the case ?

Do we need to distinguish between binning modes that affect the
timings and binning modes that do not do that?

As we're going to introduce a control for binning to report the
binning factor in the image dimension domain, should we introduce
a control to specify the binning factor in the image timing domain ?

        LLP = (analog_crop_width + HBLANK) / binning_timing_h
        FLL = (analog_crop_height + VBLANK) / binning_timing_v
        frame_interval = LLP * FLL / pixel_rate

I'm not 100% sure this is correct however, as the blankings should be
expressed on a different clock domain that the pixel sampling rate,
but I guess this is a reasonable approximation ?

>
> Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> ---
>  Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
>  .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
>  2 files changed, 53 insertions(+), 17 deletions(-)
>
> diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
> index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
> --- a/Documentation/driver-api/media/camera-sensor.rst
> +++ b/Documentation/driver-api/media/camera-sensor.rst
> @@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
>  runtime PM was disabled, in either of which cases the driver may proceed to
>  access the device.
>
> +Frame interval
> +--------------
> +
> +If a sensor supports cropping or binning, it is the sensor driver's
> +responsibility to ensure that the frame interval formula (see
> +:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
> +pipeline configuration. The driver shall adjust the minimum and maximum allowed
> +values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
> +changes, so that application developers can always rely on the same formula to
> +calculate the frame interval.
> +
>  Rotation, orientation and flipping
>  ----------------------------------
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -49,35 +49,60 @@ depends on the type of the device.
>  Raw camera sensors
>  ~~~~~~~~~~~~~~~~~~
>
> -Instead of a high level parameter such as frame interval, the frame interval is
> -a result of the configuration of a number of camera sensor implementation
> -specific parameters. Luckily, these parameters tend to be the same for more or
> -less all modern raw camera sensors.
> +Instead of a high level parameter such as frame interval, the frame interval on
> +a raw camera sensor is determined by a number of sensor-specific parameters.
> +These parameters tend to be common across most modern raw camera sensors.
>
> -The frame interval is calculated using the following equation::
> +The pixel array is the full grid of photosensitive elements on the sensor. A
> +subregion of it is selected by the analogue crop. The cropped image may then be
> +subject to binning (averaging of a NxN block) or subsampling which
> +further reduce the image dimensions. The resulting image is then read out by
> +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> +optional digital crop or scaling may further reduce the image dimensions, see
> +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
>
> -	frame interval = (analogue crop width + horizontal blanking) *
> -			 (analogue crop height + vertical blanking) / pixel rate
> +The frame size is determined by two timing parameters: line length in pixels
> +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> +registers that control how fast the ADC reads out the image. They may go
> +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> +VTOTAL, or similar.
>
> -The formula is bus independent and is applicable for raw timing parameters on
> -large variety of devices beyond camera sensors. Devices that have no analogue
> -crop, use the full source image size, i.e. pixel array size.
> +LLP is the total number of pixel clock cycles per line, including both the
> +active readout width and horizontal blanking. FLL is the total number of lines
> +per frame, including both the active readout height and vertical blanking.
> +
> +The frame interval is::
> +
> +        frame interval = LLP * FLL / pixel rate
>
>  Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
>  ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
>  is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> -sub-device. The unit of that control is pixels per second.
> +the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
> +same sub-device. The unit of that control is pixels per second.
> +
> +The blanking is defined relative to the size of the image being sent out to the
> +host over the bus (like CSI-2)::
> +
> +        LLP = active width + V4L2_CID_HBLANK
> +        FLL = active height + V4L2_CID_VBLANK
> +
> +The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
> +``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
> +range permitted by the sensor hardware for the current mode. Sensors that
> +support binning often define a lower minimum for LLP or FLL registers, which
> +can help achieve higher framerates when binning.
> +
> +Application developers can calculate the frame interval using the output
> +dimensions and the blanking controls::
> +
> +        frame interval = (output width + horizontal blanking) *
> +                         (output height + vertical blanking) / pixel rate
>
>  Register list-based drivers need to implement read-only sub-device nodes for the
>  purpose. Devices that are not register list based need these to configure the
>  device's internal processing pipeline.
>
> -The first entity in the linear pipeline is the pixel array. The pixel array may
> -be followed by other entities that are there to allow configuring binning,
> -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> -<VIDIOC_SUBDEV_G_SELECTION>`.
> -
>  USB cameras etc. devices
>  ~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> ---
> base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> change-id: 20260219-media-fps-docs-fd1da722cc38
>
> Best regards,
> --
> Jai Luthra <jai.luthra@ideasonboard.com>
>

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Sakari Ailus]

Hi Jacopo,

On Mon, Feb 23, 2026 at 10:11:02AM +0100, Jacopo Mondi wrote:
> Hi Jai
> 
> On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > The previous frame interval formula used analogue crop dimensions. This
> > breaks down for some sensors when binning.
> >
> > For example in imx219 the minimum FLL (frame length in lines) can be
> > lower than the analogue crop height when binning, which would require a
> > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > the analogue crop width of the full resolution mode.
> >
> > The CCS specification also describes under section "8.2.6 Line Length
> > and Frame Length" how the horizontal and vertical readout minimums can
> > be different when binning.
> >
> > Replace the formula with the underlying hardware concepts of LLP (line
> > length in pixels) and FLL (frame length in lines). These terms were
> > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > reference compared to a typical sensor vendor datasheet.
> >
> > Finally, define the blanking controls relative to the active pixel
> > readout (post-binning) rather than the analogue crop size. This matches
> > what most sensor drivers already do, and also what applications like
> > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > similar definition:
> >
> >   frame interval = (output width + HBLANK) *
> >                    (output height + VBLANK) / pixel rate
> >
> > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > this formula is followed by new sensor drivers.
> 
> I agree that using the analogue crop rectangle sizes is not correct,
> however, with this new formulation the LLP and FLL values might be
> smaller than the crop rectangles reported through the selection API,
> which I'm not sure it's great from a consistency point of view ?
> 
> Also the below suggested formulation:
> 
>         LLP = active width + V4L2_CID_HBLANK
>         FLL = active height + V4L2_CID_VBLANK
> 
> Assumes the combined effect of [binning + subsampling] always directly
> affect the readout time of pixels on the pixel array. The CCS specs
> and driver seems to suggest that's not always the case ?
> 
> Do we need to distinguish between binning modes that affect the
> timings and binning modes that do not do that?

For existing (pre-common raw sensor model) drivers, I'd say "no". CCS and
common raw sensor model will use line length in pixels and frame length in
lines for this, which reflects what the hardware does.

Existing non-CCS drivers have used the format for this purpose and adjusted
the blanking values accordingly.

> 
> As we're going to introduce a control for binning to report the
> binning factor in the image dimension domain, should we introduce
> a control to specify the binning factor in the image timing domain ?
> 
>         LLP = (analog_crop_width + HBLANK) / binning_timing_h
>         FLL = (analog_crop_height + VBLANK) / binning_timing_v
>         frame_interval = LLP * FLL / pixel_rate
> 
> I'm not 100% sure this is correct however, as the blankings should be
> expressed on a different clock domain that the pixel sampling rate,
> but I guess this is a reasonable approximation ?

If binning is done in analogue domain, the above would hold, but sensor do
digital binning as well. The driver needs to adjust the blanking values
(and in the future frame length and line length values) to accommodate
this.

> 
> >
> > Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> > ---
> >  Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
> >  .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
> >  2 files changed, 53 insertions(+), 17 deletions(-)
> >
> > diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
> > index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
> > --- a/Documentation/driver-api/media/camera-sensor.rst
> > +++ b/Documentation/driver-api/media/camera-sensor.rst
> > @@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
> >  runtime PM was disabled, in either of which cases the driver may proceed to
> >  access the device.
> >
> > +Frame interval
> > +--------------
> > +
> > +If a sensor supports cropping or binning, it is the sensor driver's
> > +responsibility to ensure that the frame interval formula (see
> > +:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
> > +pipeline configuration. The driver shall adjust the minimum and maximum allowed
> > +values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
> > +changes, so that application developers can always rely on the same formula to
> > +calculate the frame interval.
> > +
> >  Rotation, orientation and flipping
> >  ----------------------------------
> >
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -49,35 +49,60 @@ depends on the type of the device.
> >  Raw camera sensors
> >  ~~~~~~~~~~~~~~~~~~
> >
> > -Instead of a high level parameter such as frame interval, the frame interval is
> > -a result of the configuration of a number of camera sensor implementation
> > -specific parameters. Luckily, these parameters tend to be the same for more or
> > -less all modern raw camera sensors.
> > +Instead of a high level parameter such as frame interval, the frame interval on
> > +a raw camera sensor is determined by a number of sensor-specific parameters.
> > +These parameters tend to be common across most modern raw camera sensors.
> >
> > -The frame interval is calculated using the following equation::
> > +The pixel array is the full grid of photosensitive elements on the sensor. A
> > +subregion of it is selected by the analogue crop. The cropped image may then be
> > +subject to binning (averaging of a NxN block) or subsampling which
> > +further reduce the image dimensions. The resulting image is then read out by
> > +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> > +optional digital crop or scaling may further reduce the image dimensions, see
> > +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
> >
> > -	frame interval = (analogue crop width + horizontal blanking) *
> > -			 (analogue crop height + vertical blanking) / pixel rate
> > +The frame size is determined by two timing parameters: line length in pixels
> > +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> > +registers that control how fast the ADC reads out the image. They may go
> > +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> > +VTOTAL, or similar.
> >
> > -The formula is bus independent and is applicable for raw timing parameters on
> > -large variety of devices beyond camera sensors. Devices that have no analogue
> > -crop, use the full source image size, i.e. pixel array size.
> > +LLP is the total number of pixel clock cycles per line, including both the
> > +active readout width and horizontal blanking. FLL is the total number of lines
> > +per frame, including both the active readout height and vertical blanking.
> > +
> > +The frame interval is::
> > +
> > +        frame interval = LLP * FLL / pixel rate
> >
> >  Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> >  ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> >  is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> > -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> > -sub-device. The unit of that control is pixels per second.
> > +the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
> > +same sub-device. The unit of that control is pixels per second.
> > +
> > +The blanking is defined relative to the size of the image being sent out to the
> > +host over the bus (like CSI-2)::
> > +
> > +        LLP = active width + V4L2_CID_HBLANK
> > +        FLL = active height + V4L2_CID_VBLANK
> > +
> > +The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
> > +``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
> > +range permitted by the sensor hardware for the current mode. Sensors that
> > +support binning often define a lower minimum for LLP or FLL registers, which
> > +can help achieve higher framerates when binning.
> > +
> > +Application developers can calculate the frame interval using the output
> > +dimensions and the blanking controls::
> > +
> > +        frame interval = (output width + horizontal blanking) *
> > +                         (output height + vertical blanking) / pixel rate
> >
> >  Register list-based drivers need to implement read-only sub-device nodes for the
> >  purpose. Devices that are not register list based need these to configure the
> >  device's internal processing pipeline.
> >
> > -The first entity in the linear pipeline is the pixel array. The pixel array may
> > -be followed by other entities that are there to allow configuring binning,
> > -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> > -<VIDIOC_SUBDEV_G_SELECTION>`.
> > -
> >  USB cameras etc. devices
> >  ~~~~~~~~~~~~~~~~~~~~~~~~
> >
> >
> > ---
> > base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> > change-id: 20260219-media-fps-docs-fd1da722cc38
> >
> > Best regards,
> > --
> > Jai Luthra <jai.luthra@ideasonboard.com>
> >

-- 
Kind regards,

Sakari Ailus

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jacopo Mondi]

Hi Sakari

On Mon, Feb 23, 2026 at 12:06:01PM +0200, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Mon, Feb 23, 2026 at 10:11:02AM +0100, Jacopo Mondi wrote:
> > Hi Jai
> >
> > On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > > The previous frame interval formula used analogue crop dimensions. This
> > > breaks down for some sensors when binning.
> > >
> > > For example in imx219 the minimum FLL (frame length in lines) can be
> > > lower than the analogue crop height when binning, which would require a
> > > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > > the analogue crop width of the full resolution mode.
> > >
> > > The CCS specification also describes under section "8.2.6 Line Length
> > > and Frame Length" how the horizontal and vertical readout minimums can
> > > be different when binning.
> > >
> > > Replace the formula with the underlying hardware concepts of LLP (line
> > > length in pixels) and FLL (frame length in lines). These terms were
> > > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > > reference compared to a typical sensor vendor datasheet.
> > >
> > > Finally, define the blanking controls relative to the active pixel
> > > readout (post-binning) rather than the analogue crop size. This matches
> > > what most sensor drivers already do, and also what applications like
> > > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > > similar definition:
> > >
> > >   frame interval = (output width + HBLANK) *
> > >                    (output height + VBLANK) / pixel rate
> > >
> > > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > > this formula is followed by new sensor drivers.
> >
> > I agree that using the analogue crop rectangle sizes is not correct,
> > however, with this new formulation the LLP and FLL values might be
> > smaller than the crop rectangles reported through the selection API,
> > which I'm not sure it's great from a consistency point of view ?
> >
> > Also the below suggested formulation:
> >
> >         LLP = active width + V4L2_CID_HBLANK
> >         FLL = active height + V4L2_CID_VBLANK
> >
> > Assumes the combined effect of [binning + subsampling] always directly
> > affect the readout time of pixels on the pixel array. The CCS specs
> > and driver seems to suggest that's not always the case ?
> >
> > Do we need to distinguish between binning modes that affect the
> > timings and binning modes that do not do that?
>
> For existing (pre-common raw sensor model) drivers, I'd say "no". CCS and

I wasn't thinking about existing drivers, they should continue to work
as they do nowadays not to break applications..


> common raw sensor model will use line length in pixels and frame length in
> lines for this, which reflects what the hardware does.
>

This won't support binning modes that do not change the pixel sampling
rate, when binning happens in the digital domain, after all lines and
pixels have been read out, if I understand correctly. I'm stepping
outside of my knowledge domain here as we're getting into the details
of a sensor implementation, but I guess configurations where binning
doesn't impact the frame timinings should be supported ?

> Existing non-CCS drivers have used the format for this purpose and adjusted
> the blanking values accordingly.

I don't think it's a problem for drivers, it's for userspace to have
a way to calculate the frame rate reliably in a way that supports all
sensors

>
> >
> > As we're going to introduce a control for binning to report the
> > binning factor in the image dimension domain, should we introduce
> > a control to specify the binning factor in the image timing domain ?
> >
> >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> >         frame_interval = LLP * FLL / pixel_rate
> >
> > I'm not 100% sure this is correct however, as the blankings should be
> > expressed on a different clock domain that the pixel sampling rate,
> > but I guess this is a reasonable approximation ?
>
> If binning is done in analogue domain, the above would hold, but sensor do
> digital binning as well. The driver needs to adjust the blanking values

In which case, if a digital binning mode is used,
        binning_timing_h = binning_timing_v = 1;

> (and in the future frame length and line length values) to accommodate
> this.

Where should userspace get "frame length" and "line length" from "in
future" ?

If using the format, then the timings will always be affected by the
binning factor, wouldn't they ?

>
> >
> > >
> > > Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> > > ---
> > >  Documentation/driver-api/media/camera-sensor.rst   | 11 ++++
> > >  .../userspace-api/media/drivers/camera-sensor.rst  | 59 +++++++++++++++-------
> > >  2 files changed, 53 insertions(+), 17 deletions(-)
> > >
> > > diff --git a/Documentation/driver-api/media/camera-sensor.rst b/Documentation/driver-api/media/camera-sensor.rst
> > > index 94bd1dae82d5c570b2d11c7faee20dd45d2f4be6..8dcac7551f54ac4ffa71173281ae3bbea331c036 100644
> > > --- a/Documentation/driver-api/media/camera-sensor.rst
> > > +++ b/Documentation/driver-api/media/camera-sensor.rst
> > > @@ -120,6 +120,17 @@ The function returns a non-zero value if it succeeded getting the power count or
> > >  runtime PM was disabled, in either of which cases the driver may proceed to
> > >  access the device.
> > >
> > > +Frame interval
> > > +--------------
> > > +
> > > +If a sensor supports cropping or binning, it is the sensor driver's
> > > +responsibility to ensure that the frame interval formula (see
> > > +:ref:`media_using_camera_sensor_drivers`) remains valid regardless of the
> > > +pipeline configuration. The driver shall adjust the minimum and maximum allowed
> > > +values of ``V4L2_CID_HBLANK`` and ``V4L2_CID_VBLANK`` as needed when the mode
> > > +changes, so that application developers can always rely on the same formula to
> > > +calculate the frame interval.
> > > +
> > >  Rotation, orientation and flipping
> > >  ----------------------------------
> > >
> > > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..30dddea72a12da264fc9c30e37b561c762c09d29 100644
> > > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > @@ -49,35 +49,60 @@ depends on the type of the device.
> > >  Raw camera sensors
> > >  ~~~~~~~~~~~~~~~~~~
> > >
> > > -Instead of a high level parameter such as frame interval, the frame interval is
> > > -a result of the configuration of a number of camera sensor implementation
> > > -specific parameters. Luckily, these parameters tend to be the same for more or
> > > -less all modern raw camera sensors.
> > > +Instead of a high level parameter such as frame interval, the frame interval on
> > > +a raw camera sensor is determined by a number of sensor-specific parameters.
> > > +These parameters tend to be common across most modern raw camera sensors.
> > >
> > > -The frame interval is calculated using the following equation::
> > > +The pixel array is the full grid of photosensitive elements on the sensor. A
> > > +subregion of it is selected by the analogue crop. The cropped image may then be
> > > +subject to binning (averaging of a NxN block) or subsampling which
> > > +further reduce the image dimensions. The resulting image is then read out by
> > > +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> > > +optional digital crop or scaling may further reduce the image dimensions, see
> > > +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
> > >
> > > -	frame interval = (analogue crop width + horizontal blanking) *
> > > -			 (analogue crop height + vertical blanking) / pixel rate
> > > +The frame size is determined by two timing parameters: line length in pixels
> > > +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> > > +registers that control how fast the ADC reads out the image. They may go
> > > +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> > > +VTOTAL, or similar.
> > >
> > > -The formula is bus independent and is applicable for raw timing parameters on
> > > -large variety of devices beyond camera sensors. Devices that have no analogue
> > > -crop, use the full source image size, i.e. pixel array size.
> > > +LLP is the total number of pixel clock cycles per line, including both the
> > > +active readout width and horizontal blanking. FLL is the total number of lines
> > > +per frame, including both the active readout height and vertical blanking.
> > > +
> > > +The frame interval is::
> > > +
> > > +        frame interval = LLP * FLL / pixel rate
> > >
> > >  Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> > >  ``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> > >  is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> > > -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> > > -sub-device. The unit of that control is pixels per second.
> > > +the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the
> > > +same sub-device. The unit of that control is pixels per second.
> > > +
> > > +The blanking is defined relative to the size of the image being sent out to the
> > > +host over the bus (like CSI-2)::
> > > +
> > > +        LLP = active width + V4L2_CID_HBLANK
> > > +        FLL = active height + V4L2_CID_VBLANK
> > > +
> > > +The driver shall set the minimum and maximum values of ``V4L2_CID_HBLANK`` and
> > > +``V4L2_CID_VBLANK`` such that the resulting LLP and FLL values correspond to the
> > > +range permitted by the sensor hardware for the current mode. Sensors that
> > > +support binning often define a lower minimum for LLP or FLL registers, which
> > > +can help achieve higher framerates when binning.
> > > +
> > > +Application developers can calculate the frame interval using the output
> > > +dimensions and the blanking controls::
> > > +
> > > +        frame interval = (output width + horizontal blanking) *
> > > +                         (output height + vertical blanking) / pixel rate
> > >
> > >  Register list-based drivers need to implement read-only sub-device nodes for the
> > >  purpose. Devices that are not register list based need these to configure the
> > >  device's internal processing pipeline.
> > >
> > > -The first entity in the linear pipeline is the pixel array. The pixel array may
> > > -be followed by other entities that are there to allow configuring binning,
> > > -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> > > -<VIDIOC_SUBDEV_G_SELECTION>`.
> > > -
> > >  USB cameras etc. devices
> > >  ~~~~~~~~~~~~~~~~~~~~~~~~
> > >
> > >
> > > ---
> > > base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> > > change-id: 20260219-media-fps-docs-fd1da722cc38
> > >
> > > Best regards,
> > > --
> > > Jai Luthra <jai.luthra@ideasonboard.com>
> > >
>
> --
> Kind regards,
>
> Sakari Ailus

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

Hi Jacopo,

Thanks for the review.

Quoting Jacopo Mondi (2026-02-23 14:41:02)
> Hi Jai
> 
> On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > The previous frame interval formula used analogue crop dimensions. This
> > breaks down for some sensors when binning.
> >
> > For example in imx219 the minimum FLL (frame length in lines) can be
> > lower than the analogue crop height when binning, which would require a
> > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > the analogue crop width of the full resolution mode.
> >
> > The CCS specification also describes under section "8.2.6 Line Length
> > and Frame Length" how the horizontal and vertical readout minimums can
> > be different when binning.
> >
> > Replace the formula with the underlying hardware concepts of LLP (line
> > length in pixels) and FLL (frame length in lines). These terms were
> > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > reference compared to a typical sensor vendor datasheet.
> >
> > Finally, define the blanking controls relative to the active pixel
> > readout (post-binning) rather than the analogue crop size. This matches
> > what most sensor drivers already do, and also what applications like
> > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > similar definition:
> >
> >   frame interval = (output width + HBLANK) *
> >                    (output height + VBLANK) / pixel rate
> >
> > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > this formula is followed by new sensor drivers.
> 
> I agree that using the analogue crop rectangle sizes is not correct,
> however, with this new formulation the LLP and FLL values might be
> smaller than the crop rectangles reported through the selection API,
> which I'm not sure it's great from a consistency point of view ?

While it takes a while to grok, but most sensors I looked at operate in
this way (of reducing the line length or the frame length) when binning,
including CCS (implicitly). A counterexample would really help here.

> 
> Also the below suggested formulation:
> 
>         LLP = active width + V4L2_CID_HBLANK
>         FLL = active height + V4L2_CID_VBLANK
> 
> Assumes the combined effect of [binning + subsampling] always directly
> affect the readout time of pixels on the pixel array. The CCS specs
> and driver seems to suggest that's not always the case ?

Why do you think this model can't work for sensors where the readout time is
unaffected?

Let's say a sensor's pixel array is 1920x1080 with minimum LLP=2000, FLL=1200.

so, HBLANK minimum = 80, VBLANK minimum = 120

It also supports a 2x2 binned mode of 960x540 with the same minimum LLP and FLL
(that is same max framerate, no speed up or change in readout).

so, HBLANK minimum = 2000 - 960 = 1040, VBLANK minimum = 1200 - 540 = 660

If you match this with the CCS Figure 42 it still makes sense.

> 
> Do we need to distinguish between binning modes that affect the
> timings and binning modes that do not do that?
> 
> As we're going to introduce a control for binning to report the
> binning factor in the image dimension domain, should we introduce
> a control to specify the binning factor in the image timing domain ?
> 
>         LLP = (analog_crop_width + HBLANK) / binning_timing_h
>         FLL = (analog_crop_height + VBLANK) / binning_timing_v
>         frame_interval = LLP * FLL / pixel_rate
> 
> I'm not 100% sure this is correct however, as the blankings should be
> expressed on a different clock domain that the pixel sampling rate,
> but I guess this is a reasonable approximation ?

What does these two extra controls really offer us?

All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
values with respect to the digital readout, and not the analogue pixel array.

If we add these two controls, we will have to support two different models for
frame interval calculation in the application layer too. Which I'm fine with if
it has a practical benefit, that is, it makes it easier to deal with some
particular sensor.

And if exposing the LLP/FLL directly offer the same benefit, then that is
cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.

Thanks,
Jai

[...]

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Sakari Ailus]

Hi Jai,

On Tue, Feb 24, 2026 at 10:55:33AM +0530, Jai Luthra wrote:
> >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> >         frame_interval = LLP * FLL / pixel_rate
> > 
> > I'm not 100% sure this is correct however, as the blankings should be
> > expressed on a different clock domain that the pixel sampling rate,
> > but I guess this is a reasonable approximation ?
> 
> What does these two extra controls really offer us?

The conclusion earlier on (well, maybe 10 years ago?) was that we should
have had these to begin with, and not to use the blanking values to control
the frame rate, largely because it requires a reference rectangle -- and
the current drivers use a wrong one, mostly because the API doesn't offer
the right one. The assumption back then was that everything would soon be
SMIA (or now CCS) compliant so this wouldn't matter much. But here we
are...

The VBLANK and HBLANK controls effectively also use the wrong reference
(output size) in most drivers and that can't really be changed meaningfully
without breaking things.

> 
> All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
> values with respect to the digital readout, and not the analogue pixel array.
> 
> If we add these two controls, we will have to support two different models for
> frame interval calculation in the application layer too. Which I'm fine with if
> it has a practical benefit, that is, it makes it easier to deal with some
> particular sensor.

There are three models, effectively:

1. CCS, where vblank and hblank are related to analogue crop rectangle, as in
   hardware;

2. Common raw sensor model, with LLP and FLL; and

3. the rest of the existing drivers (vblank and hblank relative to format
   on source pad).

In the case of frame length vs. blanking, I'd presume the difference in the
user space will be relatively small.

> 
> And if exposing the LLP/FLL directly offer the same benefit, then that is
> cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jacopo Mondi]

Hi Jai

On Tue, Feb 24, 2026 at 10:55:33AM +0530, Jai Luthra wrote:
> Hi Jacopo,
>
> Thanks for the review.
>
> Quoting Jacopo Mondi (2026-02-23 14:41:02)
> > Hi Jai
> >
> > On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > > The previous frame interval formula used analogue crop dimensions. This
> > > breaks down for some sensors when binning.
> > >
> > > For example in imx219 the minimum FLL (frame length in lines) can be
> > > lower than the analogue crop height when binning, which would require a
> > > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > > the analogue crop width of the full resolution mode.
> > >
> > > The CCS specification also describes under section "8.2.6 Line Length
> > > and Frame Length" how the horizontal and vertical readout minimums can
> > > be different when binning.
> > >
> > > Replace the formula with the underlying hardware concepts of LLP (line
> > > length in pixels) and FLL (frame length in lines). These terms were
> > > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > > reference compared to a typical sensor vendor datasheet.
> > >
> > > Finally, define the blanking controls relative to the active pixel
> > > readout (post-binning) rather than the analogue crop size. This matches
> > > what most sensor drivers already do, and also what applications like
> > > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > > similar definition:
> > >
> > >   frame interval = (output width + HBLANK) *
> > >                    (output height + VBLANK) / pixel rate
> > >
> > > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > > this formula is followed by new sensor drivers.
> >
> > I agree that using the analogue crop rectangle sizes is not correct,
> > however, with this new formulation the LLP and FLL values might be
> > smaller than the crop rectangles reported through the selection API,
> > which I'm not sure it's great from a consistency point of view ?
>
> While it takes a while to grok, but most sensors I looked at operate in
> this way (of reducing the line length or the frame length) when binning,
> including CCS (implicitly). A counterexample would really help here.
>
> >
> > Also the below suggested formulation:
> >
> >         LLP = active width + V4L2_CID_HBLANK
> >         FLL = active height + V4L2_CID_VBLANK
> >
> > Assumes the combined effect of [binning + subsampling] always directly
> > affect the readout time of pixels on the pixel array. The CCS specs
> > and driver seems to suggest that's not always the case ?
>
> Why do you think this model can't work for sensors where the readout time is
> unaffected?
>
> Let's say a sensor's pixel array is 1920x1080 with minimum LLP=2000, FLL=1200.
>
> so, HBLANK minimum = 80, VBLANK minimum = 120
>
> It also supports a 2x2 binned mode of 960x540 with the same minimum LLP and FLL
> (that is same max framerate, no speed up or change in readout).
>
> so, HBLANK minimum = 2000 - 960 = 1040, VBLANK minimum = 1200 - 540 = 660
>
> If you match this with the CCS Figure 42 it still makes sense.
>

Ah well, sure if LLP and FFL remain constant, the usage of the
"active sizes" doesn't matter.

This means that the min hblank and min vblank have to be doubled to
compensate for the halved the active sizes.

Speaking in CCS terms, this means the limits reported in Table 86 are
always populated.

And what I'm struggling with at the moment, is the assumptions that's
always be the case for all sensors. Unfortunately I don't have enough
experience across all vendors to tell if that's the case..


> >
> > Do we need to distinguish between binning modes that affect the
> > timings and binning modes that do not do that?
> >
> > As we're going to introduce a control for binning to report the
> > binning factor in the image dimension domain, should we introduce
> > a control to specify the binning factor in the image timing domain ?
> >
> >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> >         frame_interval = LLP * FLL / pixel_rate
> >
> > I'm not 100% sure this is correct however, as the blankings should be
> > expressed on a different clock domain that the pixel sampling rate,
> > but I guess this is a reasonable approximation ?
>
> What does these two extra controls really offer us?
>
> All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
> values with respect to the digital readout, and not the analogue pixel array.
>
> If we add these two controls, we will have to support two different models for
> frame interval calculation in the application layer too. Which I'm fine with if
> it has a practical benefit, that is, it makes it easier to deal with some
> particular sensor.

Not sure why you would need two modes to calculate timings in
userspace.

>
> And if exposing the LLP/FLL directly offer the same benefit, then that is
> cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.
>

As per above, if the consensus is that having the limits updated when
binning to maintaine the LLP/FFL values will work for all sensors,
then I'm fine with that.

> Thanks,
> Jai
>
> [...]

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jacopo Mondi]

Sorry, one correction

On Tue, Feb 24, 2026 at 07:22:24PM +0100, Jacopo Mondi wrote:
> Hi Jai
>
> On Tue, Feb 24, 2026 at 10:55:33AM +0530, Jai Luthra wrote:
> > Hi Jacopo,
> >
> > Thanks for the review.
> >
> > Quoting Jacopo Mondi (2026-02-23 14:41:02)
> > > Hi Jai
> > >
> > > On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > > > The previous frame interval formula used analogue crop dimensions. This
> > > > breaks down for some sensors when binning.
> > > >
> > > > For example in imx219 the minimum FLL (frame length in lines) can be
> > > > lower than the analogue crop height when binning, which would require a
> > > > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > > > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > > > the analogue crop width of the full resolution mode.
> > > >
> > > > The CCS specification also describes under section "8.2.6 Line Length
> > > > and Frame Length" how the horizontal and vertical readout minimums can
> > > > be different when binning.
> > > >
> > > > Replace the formula with the underlying hardware concepts of LLP (line
> > > > length in pixels) and FLL (frame length in lines). These terms were
> > > > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > > > reference compared to a typical sensor vendor datasheet.
> > > >
> > > > Finally, define the blanking controls relative to the active pixel
> > > > readout (post-binning) rather than the analogue crop size. This matches
> > > > what most sensor drivers already do, and also what applications like
> > > > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > > > similar definition:
> > > >
> > > >   frame interval = (output width + HBLANK) *
> > > >                    (output height + VBLANK) / pixel rate
> > > >
> > > > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > > > this formula is followed by new sensor drivers.
> > >
> > > I agree that using the analogue crop rectangle sizes is not correct,
> > > however, with this new formulation the LLP and FLL values might be
> > > smaller than the crop rectangles reported through the selection API,
> > > which I'm not sure it's great from a consistency point of view ?
> >
> > While it takes a while to grok, but most sensors I looked at operate in
> > this way (of reducing the line length or the frame length) when binning,
> > including CCS (implicitly). A counterexample would really help here.
> >
> > >
> > > Also the below suggested formulation:
> > >
> > >         LLP = active width + V4L2_CID_HBLANK
> > >         FLL = active height + V4L2_CID_VBLANK
> > >
> > > Assumes the combined effect of [binning + subsampling] always directly
> > > affect the readout time of pixels on the pixel array. The CCS specs
> > > and driver seems to suggest that's not always the case ?
> >
> > Why do you think this model can't work for sensors where the readout time is
> > unaffected?
> >
> > Let's say a sensor's pixel array is 1920x1080 with minimum LLP=2000, FLL=1200.
> >
> > so, HBLANK minimum = 80, VBLANK minimum = 120
> >
> > It also supports a 2x2 binned mode of 960x540 with the same minimum LLP and FLL
> > (that is same max framerate, no speed up or change in readout).
> >
> > so, HBLANK minimum = 2000 - 960 = 1040, VBLANK minimum = 1200 - 540 = 660
> >
> > If you match this with the CCS Figure 42 it still makes sense.
> >
>
> Ah well, sure if LLP and FFL remain constant, the usage of the
> "active sizes" doesn't matter.
>
> This means that the min hblank and min vblank have to be doubled to
> compensate for the halved the active sizes.

This is of course not correct. blankings doesn't have to be doubled,
but just enalrged enough to maintain LLP/FFL constant.

Sorry for the oversight.

>
> Speaking in CCS terms, this means the limits reported in Table 86 are
> always populated.
>
> And what I'm struggling with at the moment, is the assumptions that's
> always be the case for all sensors. Unfortunately I don't have enough
> experience across all vendors to tell if that's the case..
>
>
> > >
> > > Do we need to distinguish between binning modes that affect the
> > > timings and binning modes that do not do that?
> > >
> > > As we're going to introduce a control for binning to report the
> > > binning factor in the image dimension domain, should we introduce
> > > a control to specify the binning factor in the image timing domain ?
> > >
> > >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> > >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> > >         frame_interval = LLP * FLL / pixel_rate
> > >
> > > I'm not 100% sure this is correct however, as the blankings should be
> > > expressed on a different clock domain that the pixel sampling rate,
> > > but I guess this is a reasonable approximation ?
> >
> > What does these two extra controls really offer us?
> >
> > All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
> > values with respect to the digital readout, and not the analogue pixel array.
> >
> > If we add these two controls, we will have to support two different models for
> > frame interval calculation in the application layer too. Which I'm fine with if
> > it has a practical benefit, that is, it makes it easier to deal with some
> > particular sensor.
>
> Not sure why you would need two modes to calculate timings in
> userspace.
>
> >
> > And if exposing the LLP/FLL directly offer the same benefit, then that is
> > cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.
> >
>
> As per above, if the consensus is that having the limits updated when
> binning to maintaine the LLP/FFL values will work for all sensors,
> then I'm fine with that.
>
> > Thanks,
> > Jai
> >
> > [...]

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

Quoting Sakari Ailus (2026-02-24 14:09:39)
> Hi Jai,
> 
> On Tue, Feb 24, 2026 at 10:55:33AM +0530, Jai Luthra wrote:
> > >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> > >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> > >         frame_interval = LLP * FLL / pixel_rate
> > > 
> > > I'm not 100% sure this is correct however, as the blankings should be
> > > expressed on a different clock domain that the pixel sampling rate,
> > > but I guess this is a reasonable approximation ?
> > 
> > What does these two extra controls really offer us?
> 
> The conclusion earlier on (well, maybe 10 years ago?) was that we should
> have had these to begin with, and not to use the blanking values to control
> the frame rate, largely because it requires a reference rectangle -- and
> the current drivers use a wrong one, mostly because the API doesn't offer
> the right one. The assumption back then was that everything would soon be
> SMIA (or now CCS) compliant so this wouldn't matter much. But here we
> are...
> 
> The VBLANK and HBLANK controls effectively also use the wrong reference
> (output size) in most drivers and that can't really be changed meaningfully
> without breaking things.
> 

Argh, makes sense.

> > 
> > All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
> > values with respect to the digital readout, and not the analogue pixel array.
> > 
> > If we add these two controls, we will have to support two different models for
> > frame interval calculation in the application layer too. Which I'm fine with if
> > it has a practical benefit, that is, it makes it easier to deal with some
> > particular sensor.
> 
> There are three models, effectively:
> 
> 1. CCS, where vblank and hblank are related to analogue crop rectangle, as in
>    hardware;
> 
> 2. Common raw sensor model, with LLP and FLL; and
> 
> 3. the rest of the existing drivers (vblank and hblank relative to format
>    on source pad).
> 

Thanks that's helpful summary. I will update v2 to reflect 1 and 3, while
details for common raw sensor model can be finalized later.

> In the case of frame length vs. blanking, I'd presume the difference in the
> user space will be relatively small.
> 

Indeed, switching to a different LLP/FFL control is easy, what's annoying
is having two different reference rectangles for the H/VBLANK controls.

> > 
> > And if exposing the LLP/FLL directly offer the same benefit, then that is
> > cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.
> 
> -- 
> Kind regards,
> 
> Sakari Ailus

Thanks,
    Jai

Re: [PATCH] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

Quoting Jacopo Mondi (2026-02-25 13:42:35)
> Sorry, one correction
> 
> On Tue, Feb 24, 2026 at 07:22:24PM +0100, Jacopo Mondi wrote:
> > Hi Jai
> >
> > On Tue, Feb 24, 2026 at 10:55:33AM +0530, Jai Luthra wrote:
> > > Hi Jacopo,
> > >
> > > Thanks for the review.
> > >
> > > Quoting Jacopo Mondi (2026-02-23 14:41:02)
> > > > Hi Jai
> > > >
> > > > On Thu, Feb 19, 2026 at 01:20:50PM +0530, Jai Luthra wrote:
> > > > > The previous frame interval formula used analogue crop dimensions. This
> > > > > breaks down for some sensors when binning.
> > > > >
> > > > > For example in imx219 the minimum FLL (frame length in lines) can be
> > > > > lower than the analogue crop height when binning, which would require a
> > > > > negative VBLANK to represent the actual timing. Similarly, imx283 allows
> > > > > a lower minimum HMAX (line length) when doing 2x2 or 3x3 binning than
> > > > > the analogue crop width of the full resolution mode.
> > > > >
> > > > > The CCS specification also describes under section "8.2.6 Line Length
> > > > > and Frame Length" how the horizontal and vertical readout minimums can
> > > > > be different when binning.
> > > > >
> > > > > Replace the formula with the underlying hardware concepts of LLP (line
> > > > > length in pixels) and FLL (frame length in lines). These terms were
> > > > > chosen to match the CCS specification on raw sensors, as it is a cleaner
> > > > > reference compared to a typical sensor vendor datasheet.
> > > > >
> > > > > Finally, define the blanking controls relative to the active pixel
> > > > > readout (post-binning) rather than the analogue crop size. This matches
> > > > > what most sensor drivers already do, and also what applications like
> > > > > libcamera expect. In "Figure 42" of CCS specification too, we see a
> > > > > similar definition:
> > > > >
> > > > >   frame interval = (output width + HBLANK) *
> > > > >                    (output height + VBLANK) / pixel rate
> > > > >
> > > > > Also add a note in the "Writing camera sensor drivers" guide, to ensure
> > > > > this formula is followed by new sensor drivers.
> > > >
> > > > I agree that using the analogue crop rectangle sizes is not correct,
> > > > however, with this new formulation the LLP and FLL values might be
> > > > smaller than the crop rectangles reported through the selection API,
> > > > which I'm not sure it's great from a consistency point of view ?
> > >
> > > While it takes a while to grok, but most sensors I looked at operate in
> > > this way (of reducing the line length or the frame length) when binning,
> > > including CCS (implicitly). A counterexample would really help here.
> > >
> > > >
> > > > Also the below suggested formulation:
> > > >
> > > >         LLP = active width + V4L2_CID_HBLANK
> > > >         FLL = active height + V4L2_CID_VBLANK
> > > >
> > > > Assumes the combined effect of [binning + subsampling] always directly
> > > > affect the readout time of pixels on the pixel array. The CCS specs
> > > > and driver seems to suggest that's not always the case ?
> > >
> > > Why do you think this model can't work for sensors where the readout time is
> > > unaffected?
> > >
> > > Let's say a sensor's pixel array is 1920x1080 with minimum LLP=2000, FLL=1200.
> > >
> > > so, HBLANK minimum = 80, VBLANK minimum = 120
> > >
> > > It also supports a 2x2 binned mode of 960x540 with the same minimum LLP and FLL
> > > (that is same max framerate, no speed up or change in readout).
> > >
> > > so, HBLANK minimum = 2000 - 960 = 1040, VBLANK minimum = 1200 - 540 = 660
> > >
> > > If you match this with the CCS Figure 42 it still makes sense.
> > >
> >
> > Ah well, sure if LLP and FFL remain constant, the usage of the
> > "active sizes" doesn't matter.
> >
> > This means that the min hblank and min vblank have to be doubled to
> > compensate for the halved the active sizes.
> 
> This is of course not correct. blankings doesn't have to be doubled,
> but just enalrged enough to maintain LLP/FFL constant.
> 
> Sorry for the oversight.
> 

No worries, the general idea got across.

> >
> > Speaking in CCS terms, this means the limits reported in Table 86 are
> > always populated.
> >
> > And what I'm struggling with at the moment, is the assumptions that's
> > always be the case for all sensors. Unfortunately I don't have enough
> > experience across all vendors to tell if that's the case..
> >

Indeed, same here. At least the handful of sensors I have dealt with follow
this, so I will go ahead and send a v2 to at least document the current
"Lingua Franca" for blanking controls even if it's as inconsistent as
English :-)

> >
> > > >
> > > > Do we need to distinguish between binning modes that affect the
> > > > timings and binning modes that do not do that?
> > > >
> > > > As we're going to introduce a control for binning to report the
> > > > binning factor in the image dimension domain, should we introduce
> > > > a control to specify the binning factor in the image timing domain ?
> > > >
> > > >         LLP = (analog_crop_width + HBLANK) / binning_timing_h
> > > >         FLL = (analog_crop_height + VBLANK) / binning_timing_v
> > > >         frame_interval = LLP * FLL / pixel_rate
> > > >
> > > > I'm not 100% sure this is correct however, as the blankings should be
> > > > expressed on a different clock domain that the pixel sampling rate,
> > > > but I guess this is a reasonable approximation ?
> > >
> > > What does these two extra controls really offer us?
> > >
> > > All sensors we have seen thus far map their LLP/FLL (or equivalent HTOT/VTOT)
> > > values with respect to the digital readout, and not the analogue pixel array.
> > >
> > > If we add these two controls, we will have to support two different models for
> > > frame interval calculation in the application layer too. Which I'm fine with if
> > > it has a practical benefit, that is, it makes it easier to deal with some
> > > particular sensor.
> >
> > Not sure why you would need two modes to calculate timings in
> > userspace.
> >

As Sakari pointed out, there are in fact 3 models (2 already present in
upstream)

> > >
> > > And if exposing the LLP/FLL directly offer the same benefit, then that is
> > > cleaner, as it leaves the HBLANK/VBLANK as-is in the new model.
> > >
> >
> > As per above, if the consensus is that having the limits updated when
> > binning to maintaine the LLP/FFL values will work for all sensors,
> > then I'm fine with that.
> >

Ack. I think we can discuss that further when common raw sensor model is
being finalized.

> > > Thanks,
> > > Jai
> > >
> > > [...]

Thanks,
    Jai