[PATCH v2] media: Documentation: Fix frame interval calculation for raw camera sensors

[PATCH v2] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jai Luthra]

The previous frame interval formula used analogue crop dimensions. This
diverges from what most non-CCS camera sensor drivers do and what most
applications expect.

The CCS specification describes under section "8.2.6 Line Length and
Frame Length" how the horizontal and vertical readout minimums can be
different when binning.

We see the same thing in non-CCS sensor drivers, for example in imx219
the minimum FLL (frame length in lines) can be lower than the analogue
crop height when doing 2x2 or 4x4 binning. Similarly in imx283 HMAX
(line length) can be lower compared to the analogue crop width when
doing 2x2 or 3x3 binning.

While the CCS driver defines the HBLANK/VBLANK controls with respect to
the analogue crop rectangle, with possibly negative values for blanking,
most other raw sensor drivers instead define them with respect to the
output dimensions (format defined on the source pad) sent on the bus.

So fix the frame interval calculation described in the documentation to
match the reality of existing drivers, separating out the CCS and
non-CCS cases. In future, with the common raw sensor model, this may
need further updates.

While at it, also clarify the order of operations in the sensor pipeline
and give the "hardware" definition for frame interval using LLP (line
length in pixels) and FLL (frame length in lines) before the messy
definitions that use blanking controls.

Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
---
Changes in v2:
- Use reStructureText note box for describing the
  HBLANK/VBLANK/PIXEL_RATE controls
- Document the blanking formula for both CCS and non-CCS raw
  sensor drivers
- Drop driver-api documentation as common raw sensor model is coming
  soon
- Link to v1: https://lore.kernel.org/r/20260219-media-fps-docs-v1-1-0387c5c5368c@ideasonboard.com
---
 .../userspace-api/media/drivers/camera-sensor.rst  | 80 ++++++++++++++++------
 1 file changed, 58 insertions(+), 22 deletions(-)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..e41ed4616593735bf78b73c2ed34b016ab46c783 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -49,34 +49,70 @@ depends on the type of the device.
 Raw camera sensors
 ~~~~~~~~~~~~~~~~~~
 
-Instead of a high level parameter such as frame interval, the frame interval is
-a result of the configuration of a number of camera sensor implementation
-specific parameters. Luckily, these parameters tend to be the same for more or
-less all modern raw camera sensors.
+Instead of a high level parameter such as frame interval, the frame interval on
+a raw camera sensor is determined by a number of sensor-specific parameters.
+These parameters tend to be common across most modern raw camera sensors.
 
-The frame interval is calculated using the following equation::
+The pixel array is the full grid of photosensitive elements on the camera
+sensor. A subregion of it is selected by the analogue crop. The cropped image
+may then be subject to binning (averaging of a NxN block) and subsampling which
+further reduce the image dimensions. The resulting image is then read out by
+the ADC (analogue-to-digital converter) line by line. After ADC readout,
+optional digital crop or scaling may further reduce the image dimensions, see
+:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
 
-	frame interval = (analogue crop width + horizontal blanking) *
-			 (analogue crop height + vertical blanking) / pixel rate
+The frame size is determined by two timing parameters: line length in pixels
+(LLP) and frame length in lines (FLL). These are fundamental sensor timing
+registers that control how fast the ADC reads out the image. They may go
+by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
+VTOTAL, or similar.
 
-The formula is bus independent and is applicable for raw timing parameters on
-large variety of devices beyond camera sensors. Devices that have no analogue
-crop, use the full source image size, i.e. pixel array size.
+LLP is the total number of pixel clock cycles per line, including both the
+active readout width and horizontal blanking. FLL is the total number of lines
+per frame, including both the active readout height and vertical blanking.
 
-Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
-``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
-is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
-the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
-sub-device. The unit of that control is pixels per second.
+The frame interval is::
 
-Register list-based drivers need to implement read-only sub-device nodes for the
-purpose. Devices that are not register list based need these to configure the
-device's internal processing pipeline.
+        frame interval = (line length in pixels) *
+                         (frame length in lines) / pixel rate
 
-The first entity in the linear pipeline is the pixel array. The pixel array may
-be followed by other entities that are there to allow configuring binning,
-skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
-<VIDIOC_SUBDEV_G_SELECTION>`.
+Application developers can calculate the frame interval using the pixel rate
+and blanking controls, relative to the reference rectangle used by the sensor
+driver::
+
+        frame interval = (width + horizontal blanking) *
+                         (height + vertical blanking) / pixel rate
+
+.. note::
+
+        Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK``
+        and ``V4L2_CID_VBLANK``, respectively. The unit of the
+        ``V4L2_CID_HBLANK`` control is pixels and the unit of the
+        ``V4L2_CID_VBLANK`` is lines. The pixel rate in the sensor's **pixel
+        array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device.
+        The unit of that control is pixels per second.
+
+        Drivers need to implement sub-device nodes that expose these controls.
+        They can be read-only or configurable depending on the device.
+
+        For non-CCS sensors, the blanking is defined relative to the size of
+        the image being sent out to the host over the bus (source pad format)::
+
+                LLP = output width + horizontal blanking
+                FLL = output height + vertical blanking
+
+        For CCS-compliant raw sensors (that use the CCS driver), the blanking
+        controls are defined relative to the analogue crop rectangle::
+
+                LLP = analogue crop width + horizontal blanking
+                FLL = analogue crop height + vertical blanking
+
+Sensors may achieve higher framerates by allowing lower minimums for line and
+frame length when binning or subsampling is enabled.
+
+The driver shall set the minimum and maximum values of the blanking controls
+such that the resulting LLP and FLL reigsters are programmed within the range
+permitted by the sensor hardware for the current mode.
 
 USB cameras etc. devices
 ~~~~~~~~~~~~~~~~~~~~~~~~

---
base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
change-id: 20260219-media-fps-docs-fd1da722cc38

Best regards,
-- 
Jai Luthra <jai.luthra@ideasonboard.com>

Re: [PATCH v2] media: Documentation: Fix frame interval calculation for raw camera sensors

[Jacopo Mondi]

Hi Jai,

On Wed, Feb 25, 2026 at 02:19:35PM +0530, Jai Luthra wrote:
> The previous frame interval formula used analogue crop dimensions. This
> diverges from what most non-CCS camera sensor drivers do and what most
> applications expect.
>
> The CCS specification describes under section "8.2.6 Line Length and
> Frame Length" how the horizontal and vertical readout minimums can be
> different when binning.
>
> We see the same thing in non-CCS sensor drivers, for example in imx219
> the minimum FLL (frame length in lines) can be lower than the analogue
> crop height when doing 2x2 or 4x4 binning. Similarly in imx283 HMAX
> (line length) can be lower compared to the analogue crop width when
> doing 2x2 or 3x3 binning.
>
> While the CCS driver defines the HBLANK/VBLANK controls with respect to
> the analogue crop rectangle, with possibly negative values for blanking,
> most other raw sensor drivers instead define them with respect to the
> output dimensions (format defined on the source pad) sent on the bus.

Sakari: are negative blankins the way CCS is intended to be operated
when LLP/FLL are smaller than the analog crop ?

>
> So fix the frame interval calculation described in the documentation to
> match the reality of existing drivers, separating out the CCS and
> non-CCS cases. In future, with the common raw sensor model, this may
> need further updates.
>
> While at it, also clarify the order of operations in the sensor pipeline
> and give the "hardware" definition for frame interval using LLP (line
> length in pixels) and FLL (frame length in lines) before the messy
> definitions that use blanking controls.
>
> Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> ---
> Changes in v2:
> - Use reStructureText note box for describing the
>   HBLANK/VBLANK/PIXEL_RATE controls
> - Document the blanking formula for both CCS and non-CCS raw
>   sensor drivers
> - Drop driver-api documentation as common raw sensor model is coming
>   soon
> - Link to v1: https://lore.kernel.org/r/20260219-media-fps-docs-v1-1-0387c5c5368c@ideasonboard.com
> ---
>  .../userspace-api/media/drivers/camera-sensor.rst  | 80 ++++++++++++++++------
>  1 file changed, 58 insertions(+), 22 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..e41ed4616593735bf78b73c2ed34b016ab46c783 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -49,34 +49,70 @@ depends on the type of the device.
>  Raw camera sensors
>  ~~~~~~~~~~~~~~~~~~
>
> -Instead of a high level parameter such as frame interval, the frame interval is
> -a result of the configuration of a number of camera sensor implementation
> -specific parameters. Luckily, these parameters tend to be the same for more or
> -less all modern raw camera sensors.
> +Instead of a high level parameter such as frame interval, the frame interval on
> +a raw camera sensor is determined by a number of sensor-specific parameters.
> +These parameters tend to be common across most modern raw camera sensors.
>
> -The frame interval is calculated using the following equation::
> +The pixel array is the full grid of photosensitive elements on the camera
> +sensor. A subregion of it is selected by the analogue crop. The cropped image
> +may then be subject to binning (averaging of a NxN block) and subsampling which
> +further reduce the image dimensions. The resulting image is then read out by
> +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> +optional digital crop or scaling may further reduce the image dimensions, see
> +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
>
> -	frame interval = (analogue crop width + horizontal blanking) *
> -			 (analogue crop height + vertical blanking) / pixel rate
> +The frame size is determined by two timing parameters: line length in pixels
> +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> +registers that control how fast the ADC reads out the image. They may go

nitpicking: I would slightly change this to:

The frame duration is determined by two timing parameters: line length in pixels
(LLP) and frame length in lines (FLL). These are fundamental sensor timing
parameters that control how fast the image is read out.


> +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> +VTOTAL, or similar.
>
> -The formula is bus independent and is applicable for raw timing parameters on
> -large variety of devices beyond camera sensors. Devices that have no analogue
> -crop, use the full source image size, i.e. pixel array size.
> +LLP is the total number of pixel clock cycles per line, including both the
> +active readout width and horizontal blanking. FLL is the total number of lines
> +per frame, including both the active readout height and vertical blanking.
>
> -Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> -``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> -is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> -sub-device. The unit of that control is pixels per second.
> +The frame interval is::
>
> -Register list-based drivers need to implement read-only sub-device nodes for the
> -purpose. Devices that are not register list based need these to configure the
> -device's internal processing pipeline.
> +        frame interval = (line length in pixels) *
> +                         (frame length in lines) / pixel rate
>
> -The first entity in the linear pipeline is the pixel array. The pixel array may
> -be followed by other entities that are there to allow configuring binning,
> -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> -<VIDIOC_SUBDEV_G_SELECTION>`.
> +Application developers can calculate the frame interval using the pixel rate
> +and blanking controls, relative to the reference rectangle used by the sensor
> +driver::
> +
> +        frame interval = (width + horizontal blanking) *
> +                         (height + vertical blanking) / pixel rate

As you below provides definitions for LLP and FLL in the CCS and
non-CCS case, should we use:

        frame interval = LLP * FFL / pixel rate

?

> +
> +.. note::
> +
> +        Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK``
> +        and ``V4L2_CID_VBLANK``, respectively. The unit of the
> +        ``V4L2_CID_HBLANK`` control is pixels and the unit of the
> +        ``V4L2_CID_VBLANK`` is lines. The pixel rate in the sensor's **pixel
> +        array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device.
> +        The unit of that control is pixels per second.

Should we link to those control definitions ?

> +
> +        Drivers need to implement sub-device nodes that expose these controls.
> +        They can be read-only or configurable depending on the device.
> +
> +        For non-CCS sensors, the blanking is defined relative to the size of
> +        the image being sent out to the host over the bus (source pad format)::
> +
> +                LLP = output width + horizontal blanking
> +                FLL = output height + vertical blanking
> +
> +        For CCS-compliant raw sensors (that use the CCS driver), the blanking
> +        controls are defined relative to the analogue crop rectangle::
> +
> +                LLP = analogue crop width + horizontal blanking
> +                FLL = analogue crop height + vertical blanking
> +
> +Sensors may achieve higher framerates by allowing lower minimums for line and
> +frame length when binning or subsampling is enabled.

I think what you mean here is that sensors can have different
HBLANK/VBLANK min/max when binning is in use ?

In that case I would say

Sensors may allow lower minimums for line and frame length when
binning or subsampling is enabled.


> +
> +The driver shall set the minimum and maximum values of the blanking controls

and here use "update" instead of "set" to convey that when
binning/subsampling is in use the blankings valid range could be
changed if the sensor supports that.

> +such that the resulting LLP and FLL reigsters are programmed within the range
> +permitted by the sensor hardware for the current mode.

Nits apart the patch looks like a good clarification to me.

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Sakari, I understand you're planning to introduce LLP/FLL controls in
the generic line-based metadata series.

Should we take this patch in that series or can this be fast-tracked ?
I'm asking because if you plan to introduce those controls it might be
worth mentioning them in this patch from the very beginning.

Thanks
  j

>
>  USB cameras etc. devices
>  ~~~~~~~~~~~~~~~~~~~~~~~~
>
> ---
> base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> change-id: 20260219-media-fps-docs-fd1da722cc38
>
> Best regards,
> --
> Jai Luthra <jai.luthra@ideasonboard.com>
>

Re: [PATCH v2] media: Documentation: Fix frame interval calculation for raw camera sensors

[Dave Stevenson]

Hi Jai & Jacopo.

On Fri, 6 Mar 2026 at 13:03, Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote:
>
> Hi Jai,
>
> On Wed, Feb 25, 2026 at 02:19:35PM +0530, Jai Luthra wrote:
> > The previous frame interval formula used analogue crop dimensions. This
> > diverges from what most non-CCS camera sensor drivers do and what most
> > applications expect.
> >
> > The CCS specification describes under section "8.2.6 Line Length and
> > Frame Length" how the horizontal and vertical readout minimums can be
> > different when binning.
> >
> > We see the same thing in non-CCS sensor drivers, for example in imx219
> > the minimum FLL (frame length in lines) can be lower than the analogue
> > crop height when doing 2x2 or 4x4 binning. Similarly in imx283 HMAX
> > (line length) can be lower compared to the analogue crop width when
> > doing 2x2 or 3x3 binning.
> >
> > While the CCS driver defines the HBLANK/VBLANK controls with respect to
> > the analogue crop rectangle, with possibly negative values for blanking,
> > most other raw sensor drivers instead define them with respect to the
> > output dimensions (format defined on the source pad) sent on the bus.
>
> Sakari: are negative blankins the way CCS is intended to be operated
> when LLP/FLL are smaller than the analog crop ?
>
> >
> > So fix the frame interval calculation described in the documentation to
> > match the reality of existing drivers, separating out the CCS and
> > non-CCS cases. In future, with the common raw sensor model, this may
> > need further updates.
> >
> > While at it, also clarify the order of operations in the sensor pipeline
> > and give the "hardware" definition for frame interval using LLP (line
> > length in pixels) and FLL (frame length in lines) before the messy
> > definitions that use blanking controls.
> >
> > Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> > ---
> > Changes in v2:
> > - Use reStructureText note box for describing the
> >   HBLANK/VBLANK/PIXEL_RATE controls
> > - Document the blanking formula for both CCS and non-CCS raw
> >   sensor drivers
> > - Drop driver-api documentation as common raw sensor model is coming
> >   soon
> > - Link to v1: https://lore.kernel.org/r/20260219-media-fps-docs-v1-1-0387c5c5368c@ideasonboard.com
> > ---
> >  .../userspace-api/media/drivers/camera-sensor.rst  | 80 ++++++++++++++++------
> >  1 file changed, 58 insertions(+), 22 deletions(-)
> >
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..e41ed4616593735bf78b73c2ed34b016ab46c783 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -49,34 +49,70 @@ depends on the type of the device.
> >  Raw camera sensors
> >  ~~~~~~~~~~~~~~~~~~
> >
> > -Instead of a high level parameter such as frame interval, the frame interval is
> > -a result of the configuration of a number of camera sensor implementation
> > -specific parameters. Luckily, these parameters tend to be the same for more or
> > -less all modern raw camera sensors.
> > +Instead of a high level parameter such as frame interval, the frame interval on
> > +a raw camera sensor is determined by a number of sensor-specific parameters.
> > +These parameters tend to be common across most modern raw camera sensors.
> >
> > -The frame interval is calculated using the following equation::
> > +The pixel array is the full grid of photosensitive elements on the camera
> > +sensor. A subregion of it is selected by the analogue crop. The cropped image
> > +may then be subject to binning (averaging of a NxN block) and subsampling which
> > +further reduce the image dimensions. The resulting image is then read out by
> > +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> > +optional digital crop or scaling may further reduce the image dimensions, see
> > +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
> >
> > -     frame interval = (analogue crop width + horizontal blanking) *
> > -                      (analogue crop height + vertical blanking) / pixel rate
> > +The frame size is determined by two timing parameters: line length in pixels
> > +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> > +registers that control how fast the ADC reads out the image. They may go
>
> nitpicking: I would slightly change this to:
>
> The frame duration is determined by two timing parameters: line length in pixels
> (LLP) and frame length in lines (FLL). These are fundamental sensor timing
> parameters that control how fast the image is read out.
>
>
> > +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> > +VTOTAL, or similar.

The horizontal register values may also be scaled.
Sensors such as IMX662 and IMX415 (both Sony Starvis2) set HMAX in
terms of "clock number per line designated", which is related to input
clock rather than pixel rate.
For imx662 it is 1/3 x PIXEL_RATE (74.25MHz vs 222.75MPix/s), and for
imx415 is 1/12 x PIXEL_RATE (74.25 or 72MHz vs 891 or 864MPix/s).

In such cases the driver should apply the required scaling so
userspace always sets the controls in units of V4L2_CID_PIXEL_RATE.

> >
> > -The formula is bus independent and is applicable for raw timing parameters on
> > -large variety of devices beyond camera sensors. Devices that have no analogue
> > -crop, use the full source image size, i.e. pixel array size.
> > +LLP is the total number of pixel clock cycles per line, including both the
> > +active readout width and horizontal blanking.

"pixel clock cycles per line" becomes awkward in view of the above.
"pixels per line at PIXEL_RATE" or "normalised pixel clocks per line"?

(Why are image sensors never simple?!?)

  Dave

> > FLL is the total number of lines
> > +per frame, including both the active readout height and vertical blanking.
> >
> > -Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> > -``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> > -is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> > -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> > -sub-device. The unit of that control is pixels per second.
> > +The frame interval is::
> >
> > -Register list-based drivers need to implement read-only sub-device nodes for the
> > -purpose. Devices that are not register list based need these to configure the
> > -device's internal processing pipeline.
> > +        frame interval = (line length in pixels) *
> > +                         (frame length in lines) / pixel rate
> >
> > -The first entity in the linear pipeline is the pixel array. The pixel array may
> > -be followed by other entities that are there to allow configuring binning,
> > -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> > -<VIDIOC_SUBDEV_G_SELECTION>`.
> > +Application developers can calculate the frame interval using the pixel rate
> > +and blanking controls, relative to the reference rectangle used by the sensor
> > +driver::
> > +
> > +        frame interval = (width + horizontal blanking) *
> > +                         (height + vertical blanking) / pixel rate
>
> As you below provides definitions for LLP and FLL in the CCS and
> non-CCS case, should we use:
>
>         frame interval = LLP * FFL / pixel rate
>
> ?
>
> > +
> > +.. note::
> > +
> > +        Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK``
> > +        and ``V4L2_CID_VBLANK``, respectively. The unit of the
> > +        ``V4L2_CID_HBLANK`` control is pixels and the unit of the
> > +        ``V4L2_CID_VBLANK`` is lines. The pixel rate in the sensor's **pixel
> > +        array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device.
> > +        The unit of that control is pixels per second.
>
> Should we link to those control definitions ?
>
> > +
> > +        Drivers need to implement sub-device nodes that expose these controls.
> > +        They can be read-only or configurable depending on the device.
> > +
> > +        For non-CCS sensors, the blanking is defined relative to the size of
> > +        the image being sent out to the host over the bus (source pad format)::
> > +
> > +                LLP = output width + horizontal blanking
> > +                FLL = output height + vertical blanking
> > +
> > +        For CCS-compliant raw sensors (that use the CCS driver), the blanking
> > +        controls are defined relative to the analogue crop rectangle::
> > +
> > +                LLP = analogue crop width + horizontal blanking
> > +                FLL = analogue crop height + vertical blanking
> > +
> > +Sensors may achieve higher framerates by allowing lower minimums for line and
> > +frame length when binning or subsampling is enabled.
>
> I think what you mean here is that sensors can have different
> HBLANK/VBLANK min/max when binning is in use ?
>
> In that case I would say
>
> Sensors may allow lower minimums for line and frame length when
> binning or subsampling is enabled.
>
>
> > +
> > +The driver shall set the minimum and maximum values of the blanking controls
>
> and here use "update" instead of "set" to convey that when
> binning/subsampling is in use the blankings valid range could be
> changed if the sensor supports that.
>
> > +such that the resulting LLP and FLL reigsters are programmed within the range
> > +permitted by the sensor hardware for the current mode.
>
> Nits apart the patch looks like a good clarification to me.
>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>
> Sakari, I understand you're planning to introduce LLP/FLL controls in
> the generic line-based metadata series.
>
> Should we take this patch in that series or can this be fast-tracked ?
> I'm asking because if you plan to introduce those controls it might be
> worth mentioning them in this patch from the very beginning.
>
> Thanks
>   j
>
> >
> >  USB cameras etc. devices
> >  ~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > ---
> > base-commit: 956b9cbd7f156c8672dac94a00de3c6a0939c692
> > change-id: 20260219-media-fps-docs-fd1da722cc38
> >
> > Best regards,
> > --
> > Jai Luthra <jai.luthra@ideasonboard.com>
> >
>

Re: [PATCH v2] media: Documentation: Fix frame interval calculation for raw camera sensors

[Sakari Ailus]

Hi Jacopo,

On Fri, Mar 06, 2026 at 02:02:58PM +0100, Jacopo Mondi wrote:
> Hi Jai,
> 
> On Wed, Feb 25, 2026 at 02:19:35PM +0530, Jai Luthra wrote:
> > The previous frame interval formula used analogue crop dimensions. This
> > diverges from what most non-CCS camera sensor drivers do and what most
> > applications expect.
> >
> > The CCS specification describes under section "8.2.6 Line Length and
> > Frame Length" how the horizontal and vertical readout minimums can be
> > different when binning.
> >
> > We see the same thing in non-CCS sensor drivers, for example in imx219
> > the minimum FLL (frame length in lines) can be lower than the analogue
> > crop height when doing 2x2 or 4x4 binning. Similarly in imx283 HMAX
> > (line length) can be lower compared to the analogue crop width when
> > doing 2x2 or 3x3 binning.
> >
> > While the CCS driver defines the HBLANK/VBLANK controls with respect to
> > the analogue crop rectangle, with possibly negative values for blanking,
> > most other raw sensor drivers instead define them with respect to the
> > output dimensions (format defined on the source pad) sent on the bus.
> 
> Sakari: are negative blankins the way CCS is intended to be operated
> when LLP/FLL are smaller than the analog crop ?

Yes. The formula is rather simple.

> 
> >
> > So fix the frame interval calculation described in the documentation to
> > match the reality of existing drivers, separating out the CCS and
> > non-CCS cases. In future, with the common raw sensor model, this may
> > need further updates.
> >
> > While at it, also clarify the order of operations in the sensor pipeline
> > and give the "hardware" definition for frame interval using LLP (line
> > length in pixels) and FLL (frame length in lines) before the messy
> > definitions that use blanking controls.
> >
> > Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> > ---
> > Changes in v2:
> > - Use reStructureText note box for describing the
> >   HBLANK/VBLANK/PIXEL_RATE controls
> > - Document the blanking formula for both CCS and non-CCS raw
> >   sensor drivers
> > - Drop driver-api documentation as common raw sensor model is coming
> >   soon
> > - Link to v1: https://lore.kernel.org/r/20260219-media-fps-docs-v1-1-0387c5c5368c@ideasonboard.com
> > ---
> >  .../userspace-api/media/drivers/camera-sensor.rst  | 80 ++++++++++++++++------
> >  1 file changed, 58 insertions(+), 22 deletions(-)
> >
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 75fd9166383fdbb2dabdb6384ed0904c4e78a3c6..e41ed4616593735bf78b73c2ed34b016ab46c783 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -49,34 +49,70 @@ depends on the type of the device.
> >  Raw camera sensors
> >  ~~~~~~~~~~~~~~~~~~
> >
> > -Instead of a high level parameter such as frame interval, the frame interval is
> > -a result of the configuration of a number of camera sensor implementation
> > -specific parameters. Luckily, these parameters tend to be the same for more or
> > -less all modern raw camera sensors.
> > +Instead of a high level parameter such as frame interval, the frame interval on
> > +a raw camera sensor is determined by a number of sensor-specific parameters.
> > +These parameters tend to be common across most modern raw camera sensors.
> >
> > -The frame interval is calculated using the following equation::
> > +The pixel array is the full grid of photosensitive elements on the camera
> > +sensor. A subregion of it is selected by the analogue crop. The cropped image
> > +may then be subject to binning (averaging of a NxN block) and subsampling which
> > +further reduce the image dimensions. The resulting image is then read out by
> > +the ADC (analogue-to-digital converter) line by line. After ADC readout,
> > +optional digital crop or scaling may further reduce the image dimensions, see
> > +:ref:`VIDIOC_SUBDEV_G_SELECTION <VIDIOC_SUBDEV_G_SELECTION>`.
> >
> > -	frame interval = (analogue crop width + horizontal blanking) *
> > -			 (analogue crop height + vertical blanking) / pixel rate
> > +The frame size is determined by two timing parameters: line length in pixels
> > +(LLP) and frame length in lines (FLL). These are fundamental sensor timing
> > +registers that control how fast the ADC reads out the image. They may go
> 
> nitpicking: I would slightly change this to:
> 
> The frame duration is determined by two timing parameters: line length in pixels
> (LLP) and frame length in lines (FLL). These are fundamental sensor timing
> parameters that control how fast the image is read out.
> 
> 
> > +by different names for a particular sensor, like HMAX and VMAX, or HTOTAL and
> > +VTOTAL, or similar.
> >
> > -The formula is bus independent and is applicable for raw timing parameters on
> > -large variety of devices beyond camera sensors. Devices that have no analogue
> > -crop, use the full source image size, i.e. pixel array size.
> > +LLP is the total number of pixel clock cycles per line, including both the
> > +active readout width and horizontal blanking. FLL is the total number of lines
> > +per frame, including both the active readout height and vertical blanking.
> >
> > -Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and
> > -``V4L2_CID_VBLANK``, respectively. The unit of the ``V4L2_CID_HBLANK`` control
> > -is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
> > -the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
> > -sub-device. The unit of that control is pixels per second.
> > +The frame interval is::
> >
> > -Register list-based drivers need to implement read-only sub-device nodes for the
> > -purpose. Devices that are not register list based need these to configure the
> > -device's internal processing pipeline.
> > +        frame interval = (line length in pixels) *
> > +                         (frame length in lines) / pixel rate
> >
> > -The first entity in the linear pipeline is the pixel array. The pixel array may
> > -be followed by other entities that are there to allow configuring binning,
> > -skipping, scaling or digital crop, see :ref:`VIDIOC_SUBDEV_G_SELECTION
> > -<VIDIOC_SUBDEV_G_SELECTION>`.
> > +Application developers can calculate the frame interval using the pixel rate
> > +and blanking controls, relative to the reference rectangle used by the sensor
> > +driver::
> > +
> > +        frame interval = (width + horizontal blanking) *
> > +                         (height + vertical blanking) / pixel rate
> 
> As you below provides definitions for LLP and FLL in the CCS and
> non-CCS case, should we use:
> 
>         frame interval = LLP * FFL / pixel rate
> 
> ?
> 
> > +
> > +.. note::
> > +
> > +        Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK``
> > +        and ``V4L2_CID_VBLANK``, respectively. The unit of the
> > +        ``V4L2_CID_HBLANK`` control is pixels and the unit of the
> > +        ``V4L2_CID_VBLANK`` is lines. The pixel rate in the sensor's **pixel
> > +        array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device.
> > +        The unit of that control is pixels per second.
> 
> Should we link to those control definitions ?
> 
> > +
> > +        Drivers need to implement sub-device nodes that expose these controls.
> > +        They can be read-only or configurable depending on the device.
> > +
> > +        For non-CCS sensors, the blanking is defined relative to the size of
> > +        the image being sent out to the host over the bus (source pad format)::
> > +
> > +                LLP = output width + horizontal blanking
> > +                FLL = output height + vertical blanking
> > +
> > +        For CCS-compliant raw sensors (that use the CCS driver), the blanking
> > +        controls are defined relative to the analogue crop rectangle::
> > +
> > +                LLP = analogue crop width + horizontal blanking
> > +                FLL = analogue crop height + vertical blanking
> > +
> > +Sensors may achieve higher framerates by allowing lower minimums for line and
> > +frame length when binning or subsampling is enabled.
> 
> I think what you mean here is that sensors can have different
> HBLANK/VBLANK min/max when binning is in use ?
> 
> In that case I would say
> 
> Sensors may allow lower minimums for line and frame length when
> binning or subsampling is enabled.
> 
> 
> > +
> > +The driver shall set the minimum and maximum values of the blanking controls
> 
> and here use "update" instead of "set" to convey that when
> binning/subsampling is in use the blankings valid range could be
> changed if the sensor supports that.
> 
> > +such that the resulting LLP and FLL reigsters are programmed within the range
> > +permitted by the sensor hardware for the current mode.
> 
> Nits apart the patch looks like a good clarification to me.
> 
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Sakari, I understand you're planning to introduce LLP/FLL controls in
> the generic line-based metadata series.
> 
> Should we take this patch in that series or can this be fast-tracked ?
> I'm asking because if you plan to introduce those controls it might be
> worth mentioning them in this patch from the very beginning.

I think we could merge it before the series; there are few conflicts with
it, too.

-- 
Sakari Ailus