* [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description
[not found] <20200318205034.949531-1-jacopo@jmondi.org>
@ 2020-03-18 20:50 ` Jacopo Mondi
2020-03-20 1:03 ` Rob Herring
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Jacopo Mondi @ 2020-03-18 20:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Hans Verkuil, Sakari Ailus,
Laurent Pinchart, tfiga, pavel
Cc: Jacopo Mondi, open list:MEDIA INPUT INFRASTRUCTURE (V4L/DVB),
libcamera-devel, robh, devicetree, Niklas Soderlund
Replace the 'rotation' property description by providing a definition
relative to the camera sensor pixel array coordinate system and the
captured scene.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Co-developed-by: Niklas Soderlund <niklas.soderlund@ragnatech.se>
---
.../bindings/media/video-interfaces.txt | 359 +++++++++++++++++-
1 file changed, 356 insertions(+), 3 deletions(-)
diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
index 1211bdf80722..36e61d599f34 100644
--- a/Documentation/devicetree/bindings/media/video-interfaces.txt
+++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
@@ -85,9 +85,362 @@ Optional properties
- lens-focus: A phandle to the node of the focus lens controller.
-- rotation: The device, typically an image sensor, is not mounted upright,
- but a number of degrees counter clockwise. Typical values are 0 and 180
- (upside down).
+- rotation: The camera rotation is expressed as the angular difference in
+ degrees between two reference systems, one relative to the camera module, and
+ one defined on the external world scene to be captured when projected on the
+ image sensor pixel array.
+
+ A camera sensor has a 2-dimensional reference system 'Rc' defined by
+ its pixel array read-out order. The origin is set to the first pixel
+ being read out, the X-axis points along the column read-out direction
+ towards the last columns, and the Y-axis along the row read-out
+ direction towards the last row.
+
+ A typical example for a sensor with a 2592x1944 pixel array matrix
+ observed from the front is
+
+ 2591 X-axis 0
+ <------------------------+ 0
+ .......... ... ..........!
+ .......... ... ..........! Y-axis
+ ... !
+ .......... ... ..........!
+ .......... ... ..........! 1943
+ V
+
+ The external world scene reference system 'Rs' is a 2-dimensional
+ reference system on the focal plane of the camera module. The origin is
+ placed on the top-left corner of the visible scene, the X-axis points
+ towards the right, and the Y-axis points towards the bottom of the
+ scene. The top, bottom, left and right directions are intentionally not
+ defined and depend on the environment in which the camera is used.
+
+ A typical example of a (very common) picture of a shark swimming from
+ left to right, as seen from the camera, is
+
+ 0 X-axis
+ 0 +------------------------------------->
+ !
+ !
+ !
+ ! |\____)\___
+ ! ) _____ __`<
+ ! |/ )/
+ !
+ !
+ !
+ V
+ Y-axis
+
+ With the reference system 'Rs' placed on the camera focal plane.
+
+ ¸.·˙!
+ ¸.·˙ !
+ _ ¸.·˙ !
+ +-/ \-+¸.·˙ !
+ | (o) | ! Camera focal plane
+ +-----+˙·.¸ !
+ ˙·.¸ !
+ ˙·.¸ !
+ ˙·.¸!
+
+ When projected on the sensor's pixel array, the image and the associated
+ reference system 'Rs' are typically (but not always) inverted, due to
+ the camera module's lens optical inversion effect.
+
+ Assuming the above represented scene of the swimming shark, the lens
+ inversion projects the scene and its reference system onto the sensor
+ pixel array, seen from the front of the camera sensor, as follow
+
+ Y-axis
+ ^
+ !
+ !
+ !
+ ! |\_____)\__
+ ! ) ____ ___.<
+ ! |/ )/
+ !
+ !
+ !
+ 0 +------------------------------------->
+ 0 X-axis
+
+ Note the shark being upside-down.
+
+ The resulting projected reference system is named 'Rp'.
+
+ The camera rotation property is then defined as the angular difference
+ in the counter-clockwise direction between the camera reference system
+ 'Rc' and the projected scene reference system 'Rp'. It is expressed in
+ degrees as a number in the range [0, 360[.
+
+ Examples
+
+ 0 degrees camera rotation
+
+
+ Y-Rp
+ ^
+ Y-Rc !
+ ^ !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! 0 +------------------------------------->
+ ! 0 X-Rp
+ 0 +------------------------------------->
+ 0 X-Rc
+
+
+ X-Rc 0
+ <------------------------------------+ 0
+ X-Rp 0 !
+ <------------------------------------+ 0 !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! V
+ ! Y-Rc
+ V
+ Y-Rp
+
+ 90 degrees camera rotation
+
+ 0 Y-Rc
+ 0 +-------------------->
+ ! Y-Rp
+ ! ^
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! 0 +------------------------------------->
+ ! 0 X-Rp
+ !
+ !
+ !
+ !
+ V
+ X-Rc
+
+ 180 degrees camera rotation
+
+ 0
+ <------------------------------------+ 0
+ X-Rc !
+ Y-Rp !
+ ^ !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! V
+ ! Y-Rc
+ 0 +------------------------------------->
+ 0 X-Rp
+
+ 270 degrees camera rotation
+
+ 0 Y-Rc
+ 0 +-------------------->
+ ! 0
+ ! <-----------------------------------+ 0
+ ! X-Rp !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! !
+ ! V
+ ! Y-Rp
+ !
+ !
+ !
+ !
+ V
+ X-Rc
+
+
+ Example one - Webcam
+
+ A camera module installed on the user facing part of a laptop screen
+ casing used for video calls. The captured images are meant to be
+ displayed in landscape mode (width > height) on the laptop screen.
+
+ The camera is typically mounted upside-down to compensate the lens
+ optical inversion effect.
+
+ Y-Rp
+ Y-Rc ^
+ ^ !
+ ! !
+ ! ! |\_____)\__
+ ! ! ) ____ ___.<
+ ! ! |/ )/
+ ! !
+ ! !
+ ! !
+ ! 0 +------------------------------------->
+ ! 0 X-Rp
+ 0 +------------------------------------->
+ 0 X-Rc
+
+ The two reference systems are aligned, the resulting camera rotation is
+ 0 degrees, no rotation correction needs to be applied to the resulting
+ image once captured to memory buffers to correctly display it to users.
+
+ +--------------------------------------+
+ ! !
+ ! !
+ ! !
+ ! |\____)\___ !
+ ! ) _____ __`< !
+ ! |/ )/ !
+ ! !
+ ! !
+ ! !
+ +--------------------------------------+
+
+ If the camera sensor is not mounted upside-down to compensate for the
+ lens optical inversion, the two reference systems will not be aligned,
+ with 'Rp' being rotated 180 degrees relatively to 'Rc'.
+
+
+ X-Rc 0
+ <------------------------------------+ 0
+ !
+ Y-Rp !
+ ^ !
+ ! !
+ ! |\_____)\__ !
+ ! ) ____ ___.< !
+ ! |/ )/ !
+ ! !
+ ! !
+ ! V
+ ! Y-Rc
+ 0 +------------------------------------->
+ 0 X-Rp
+
+ The image once captured to memory will then be rotated by 180 degrees
+
+ +--------------------------------------+
+ ! !
+ ! !
+ ! !
+ ! __/(_____/| !
+ ! >.___ ____ ( !
+ ! \( \| !
+ ! !
+ ! !
+ ! !
+ +--------------------------------------+
+
+ A software rotation correction of 180 degrees should be applied to
+ correctly display the image.
+
+ +--------------------------------------+
+ ! !
+ ! !
+ ! !
+ ! |\____)\___ !
+ ! ) _____ __`< !
+ ! |/ )/ !
+ ! !
+ ! !
+ ! !
+ +--------------------------------------+
+
+ Example two - Phone camera
+
+ A camera installed on the back side of a mobile device facing away from
+ the user. The captured images are meant to be displayed in portrait mode
+ (height > width) to match the device screen orientation and the device
+ usage orientation used when taking the picture.
+
+ The camera sensor is typically mounted with its pixel array longer side
+ aligned to the device longer side, upside-down mounted to compensate for
+ the lens optical inversion effect.
+
+ 0 Y-Rc
+ 0 +-------------------->
+ ! Y-Rp
+ ! ^
+ ! !
+ ! !
+ ! !
+ ! ! |\_____)\__
+ ! ! ) ____ ___.<
+ ! ! |/ )/
+ ! !
+ ! !
+ ! !
+ ! 0 +------------------------------------->
+ ! 0 X-Rp
+ !
+ !
+ !
+ !
+ V
+ X-Rc
+
+ The two reference systems are not aligned and the 'Rp' reference
+ system is rotated by 90 degrees in the counter-clockwise direction
+ relatively to the 'Rc' reference system.
+
+ The image once captured to memory will be rotated.
+
+ +-------------------------------------+
+ | _ _ |
+ | \ / |
+ | | | |
+ | | | |
+ | | > |
+ | < | |
+ | | | |
+ | . |
+ | V |
+ +-------------------------------------+
+
+ A correction of 90 degrees in counter-clockwise direction has to be
+ applied to correctly display the image in portrait mode on the device
+ screen.
+
+ +--------------------+
+ | |
+ | |
+ | |
+ | |
+ | |
+ | |
+ | |\____)\___ |
+ | ) _____ __`< |
+ | |/ )/ |
+ | |
+ | |
+ | |
+ | |
+ | |
+ +--------------------+
- location: The mount location of a device (typically an image sensor or a flash
LED) expressed as a position relative to the usage orientation of the system
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description
2020-03-18 20:50 ` [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description Jacopo Mondi
@ 2020-03-20 1:03 ` Rob Herring
2020-03-31 10:21 ` Hans Verkuil
2020-06-08 10:09 ` Sakari Ailus
2 siblings, 0 replies; 5+ messages in thread
From: Rob Herring @ 2020-03-20 1:03 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Mauro Carvalho Chehab, Hans Verkuil, Sakari Ailus,
Laurent Pinchart, tfiga, pavel, Jacopo Mondi,
open list:MEDIA INPUT INFRASTRUCTURE (V4L/DVB), libcamera-devel,
robh, devicetree, Niklas Soderlund
On Wed, 18 Mar 2020 21:50:26 +0100, Jacopo Mondi wrote:
> Replace the 'rotation' property description by providing a definition
> relative to the camera sensor pixel array coordinate system and the
> captured scene.
>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Co-developed-by: Niklas Soderlund <niklas.soderlund@ragnatech.se>
>
> ---
> .../bindings/media/video-interfaces.txt | 359 +++++++++++++++++-
> 1 file changed, 356 insertions(+), 3 deletions(-)
>
Acked-by: Rob Herring <robh@kernel.org>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description
2020-03-18 20:50 ` [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description Jacopo Mondi
2020-03-20 1:03 ` Rob Herring
@ 2020-03-31 10:21 ` Hans Verkuil
2020-06-08 10:09 ` Sakari Ailus
2 siblings, 0 replies; 5+ messages in thread
From: Hans Verkuil @ 2020-03-31 10:21 UTC (permalink / raw)
To: Jacopo Mondi, Mauro Carvalho Chehab, Sakari Ailus,
Laurent Pinchart, tfiga, pavel
Cc: open list:MEDIA INPUT INFRASTRUCTURE (V4L/DVB), libcamera-devel,
robh, devicetree, Niklas Soderlund
On 3/18/20 9:50 PM, Jacopo Mondi wrote:
> Replace the 'rotation' property description by providing a definition
> relative to the camera sensor pixel array coordinate system and the
> captured scene.
>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Co-developed-by: Niklas Soderlund <niklas.soderlund@ragnatech.se>
>
> ---
> .../bindings/media/video-interfaces.txt | 359 +++++++++++++++++-
> 1 file changed, 356 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
> index 1211bdf80722..36e61d599f34 100644
> --- a/Documentation/devicetree/bindings/media/video-interfaces.txt
> +++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
> @@ -85,9 +85,362 @@ Optional properties
>
> - lens-focus: A phandle to the node of the focus lens controller.
>
> -- rotation: The device, typically an image sensor, is not mounted upright,
> - but a number of degrees counter clockwise. Typical values are 0 and 180
> - (upside down).
> +- rotation: The camera rotation is expressed as the angular difference in
> + degrees between two reference systems, one relative to the camera module, and
> + one defined on the external world scene to be captured when projected on the
> + image sensor pixel array.
> +
> + A camera sensor has a 2-dimensional reference system 'Rc' defined by
> + its pixel array read-out order. The origin is set to the first pixel
> + being read out, the X-axis points along the column read-out direction
> + towards the last columns, and the Y-axis along the row read-out
> + direction towards the last row.
> +
> + A typical example for a sensor with a 2592x1944 pixel array matrix
> + observed from the front is
is -> is:
> +
> + 2591 X-axis 0
> + <------------------------+ 0
> + .......... ... ..........!
> + .......... ... ..........! Y-axis
> + ... !
> + .......... ... ..........!
> + .......... ... ..........! 1943
> + V
> +
> + The external world scene reference system 'Rs' is a 2-dimensional
> + reference system on the focal plane of the camera module. The origin is
> + placed on the top-left corner of the visible scene, the X-axis points
> + towards the right, and the Y-axis points towards the bottom of the
> + scene. The top, bottom, left and right directions are intentionally not
> + defined and depend on the environment in which the camera is used.
> +
> + A typical example of a (very common) picture of a shark swimming from
> + left to right, as seen from the camera, is
is -> is:
> +
> + 0 X-axis
> + 0 +------------------------------------->
> + !
> + !
> + !
> + ! |\____)\___
> + ! ) _____ __`<
> + ! |/ )/
> + !
> + !
> + !
> + V
> + Y-axis
> +
> + With the reference system 'Rs' placed on the camera focal plane.
I think here 'With' can be 'with' (since you continue the previous sentence)
and 'plane.' should be 'plane:' since it refers to the picture below that explains
the focal plane.
> +
> + ¸.·˙!
> + ¸.·˙ !
> + _ ¸.·˙ !
> + +-/ \-+¸.·˙ !
> + | (o) | ! Camera focal plane
> + +-----+˙·.¸ !
> + ˙·.¸ !
> + ˙·.¸ !
> + ˙·.¸!
> +
> + When projected on the sensor's pixel array, the image and the associated
> + reference system 'Rs' are typically (but not always) inverted, due to
> + the camera module's lens optical inversion effect.
> +
> + Assuming the above represented scene of the swimming shark, the lens
> + inversion projects the scene and its reference system onto the sensor
> + pixel array, seen from the front of the camera sensor, as follow
follow -> follows:
> +
> + Y-axis
> + ^
> + !
> + !
> + !
> + ! |\_____)\__
> + ! ) ____ ___.<
> + ! |/ )/
> + !
> + !
> + !
> + 0 +------------------------------------->
> + 0 X-axis
> +
> + Note the shark being upside-down.
> +
> + The resulting projected reference system is named 'Rp'.
> +
> + The camera rotation property is then defined as the angular difference
> + in the counter-clockwise direction between the camera reference system
> + 'Rc' and the projected scene reference system 'Rp'. It is expressed in
> + degrees as a number in the range [0, 360[.
> +
> + Examples
> +
> + 0 degrees camera rotation
rotation -> rotation:
> +
> +
> + Y-Rp
> + ^
> + Y-Rc !
> + ^ !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + 0 +------------------------------------->
> + 0 X-Rc
> +
> +
> + X-Rc 0
> + <------------------------------------+ 0
> + X-Rp 0 !
> + <------------------------------------+ 0 !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! V
> + ! Y-Rc
> + V
> + Y-Rp
> +
> + 90 degrees camera rotation
Ditto here and for the 180 and 270 examples below.
> +
> + 0 Y-Rc
> + 0 +-------------------->
> + ! Y-Rp
> + ! ^
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + !
> + !
> + !
> + !
> + V
> + X-Rc
> +
> + 180 degrees camera rotation
> +
> + 0
> + <------------------------------------+ 0
> + X-Rc !
> + Y-Rp !
> + ^ !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! V
> + ! Y-Rc
> + 0 +------------------------------------->
> + 0 X-Rp
> +
> + 270 degrees camera rotation
> +
> + 0 Y-Rc
> + 0 +-------------------->
> + ! 0
> + ! <-----------------------------------+ 0
> + ! X-Rp !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! V
> + ! Y-Rp
> + !
> + !
> + !
> + !
> + V
> + X-Rc
> +
> +
> + Example one - Webcam
> +
> + A camera module installed on the user facing part of a laptop screen
> + casing used for video calls. The captured images are meant to be
> + displayed in landscape mode (width > height) on the laptop screen.
> +
> + The camera is typically mounted upside-down to compensate the lens
> + optical inversion effect.
effect. -> effect:
The : links the sentence with the following picture. I think this should be
done in the remainder of this section, so I won't repeat myself.
> +
> + Y-Rp
> + Y-Rc ^
> + ^ !
> + ! !
> + ! ! |\_____)\__
> + ! ! ) ____ ___.<
> + ! ! |/ )/
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + 0 +------------------------------------->
> + 0 X-Rc
> +
> + The two reference systems are aligned, the resulting camera rotation is
> + 0 degrees, no rotation correction needs to be applied to the resulting
> + image once captured to memory buffers to correctly display it to users.
> +
> + +--------------------------------------+
> + ! !
> + ! !
> + ! !
> + ! |\____)\___ !
> + ! ) _____ __`< !
> + ! |/ )/ !
> + ! !
> + ! !
> + ! !
> + +--------------------------------------+
> +
> + If the camera sensor is not mounted upside-down to compensate for the
> + lens optical inversion, the two reference systems will not be aligned,
> + with 'Rp' being rotated 180 degrees relatively to 'Rc'.
> +
> +
> + X-Rc 0
> + <------------------------------------+ 0
> + !
> + Y-Rp !
> + ^ !
> + ! !
> + ! |\_____)\__ !
> + ! ) ____ ___.< !
> + ! |/ )/ !
> + ! !
> + ! !
> + ! V
> + ! Y-Rc
> + 0 +------------------------------------->
> + 0 X-Rp
> +
> + The image once captured to memory will then be rotated by 180 degrees
> +
> + +--------------------------------------+
> + ! !
> + ! !
> + ! !
> + ! __/(_____/| !
> + ! >.___ ____ ( !
> + ! \( \| !
> + ! !
> + ! !
> + ! !
> + +--------------------------------------+
> +
> + A software rotation correction of 180 degrees should be applied to
> + correctly display the image.
> +
> + +--------------------------------------+
> + ! !
> + ! !
> + ! !
> + ! |\____)\___ !
> + ! ) _____ __`< !
> + ! |/ )/ !
> + ! !
> + ! !
> + ! !
> + +--------------------------------------+
> +
> + Example two - Phone camera
> +
> + A camera installed on the back side of a mobile device facing away from
> + the user. The captured images are meant to be displayed in portrait mode
> + (height > width) to match the device screen orientation and the device
> + usage orientation used when taking the picture.
> +
> + The camera sensor is typically mounted with its pixel array longer side
> + aligned to the device longer side, upside-down mounted to compensate for
> + the lens optical inversion effect.
> +
> + 0 Y-Rc
> + 0 +-------------------->
> + ! Y-Rp
> + ! ^
> + ! !
> + ! !
> + ! !
> + ! ! |\_____)\__
> + ! ! ) ____ ___.<
> + ! ! |/ )/
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + !
> + !
> + !
> + !
> + V
> + X-Rc
> +
> + The two reference systems are not aligned and the 'Rp' reference
> + system is rotated by 90 degrees in the counter-clockwise direction
> + relatively to the 'Rc' reference system.
> +
> + The image once captured to memory will be rotated.
> +
> + +-------------------------------------+
> + | _ _ |
> + | \ / |
> + | | | |
> + | | | |
> + | | > |
> + | < | |
> + | | | |
> + | . |
> + | V |
> + +-------------------------------------+
> +
> + A correction of 90 degrees in counter-clockwise direction has to be
> + applied to correctly display the image in portrait mode on the device
> + screen.
> +
> + +--------------------+
> + | |
> + | |
> + | |
> + | |
> + | |
> + | |
> + | |\____)\___ |
> + | ) _____ __`< |
> + | |/ )/ |
> + | |
> + | |
> + | |
> + | |
> + | |
> + +--------------------+
>
> - location: The mount location of a device (typically an image sensor or a flash
> LED) expressed as a position relative to the usage orientation of the system
> --
> 2.25.1
>
Regards,
Hans
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description
2020-03-18 20:50 ` [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description Jacopo Mondi
2020-03-20 1:03 ` Rob Herring
2020-03-31 10:21 ` Hans Verkuil
@ 2020-06-08 10:09 ` Sakari Ailus
2020-06-08 21:41 ` Laurent Pinchart
2 siblings, 1 reply; 5+ messages in thread
From: Sakari Ailus @ 2020-06-08 10:09 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Mauro Carvalho Chehab, Hans Verkuil, Laurent Pinchart, tfiga,
pavel, open list:MEDIA INPUT INFRASTRUCTURE (V4L/DVB),
libcamera-devel, robh, devicetree, Niklas Soderlund
Hi Jacopo,
Thank you for this great piece of ASCII art. Also my apologies for
reviewing this so late. The latest version is 11 but I believe the content
is the same on the commented matters.
On Wed, Mar 18, 2020 at 09:50:26PM +0100, Jacopo Mondi wrote:
> Replace the 'rotation' property description by providing a definition
> relative to the camera sensor pixel array coordinate system and the
> captured scene.
>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Co-developed-by: Niklas Soderlund <niklas.soderlund@ragnatech.se>
>
> ---
> .../bindings/media/video-interfaces.txt | 359 +++++++++++++++++-
> 1 file changed, 356 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
> index 1211bdf80722..36e61d599f34 100644
> --- a/Documentation/devicetree/bindings/media/video-interfaces.txt
> +++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
> @@ -85,9 +85,362 @@ Optional properties
>
> - lens-focus: A phandle to the node of the focus lens controller.
>
> -- rotation: The device, typically an image sensor, is not mounted upright,
> - but a number of degrees counter clockwise. Typical values are 0 and 180
> - (upside down).
> +- rotation: The camera rotation is expressed as the angular difference in
> + degrees between two reference systems, one relative to the camera module, and
> + one defined on the external world scene to be captured when projected on the
> + image sensor pixel array.
> +
> + A camera sensor has a 2-dimensional reference system 'Rc' defined by
> + its pixel array read-out order. The origin is set to the first pixel
> + being read out, the X-axis points along the column read-out direction
> + towards the last columns, and the Y-axis along the row read-out
> + direction towards the last row.
> +
> + A typical example for a sensor with a 2592x1944 pixel array matrix
> + observed from the front is
> +
> + 2591 X-axis 0
> + <------------------------+ 0
> + .......... ... ..........!
> + .......... ... ..........! Y-axis
> + ... !
> + .......... ... ..........!
> + .......... ... ..........! 1943
> + V
> +
> + The external world scene reference system 'Rs' is a 2-dimensional
> + reference system on the focal plane of the camera module. The origin is
> + placed on the top-left corner of the visible scene, the X-axis points
> + towards the right, and the Y-axis points towards the bottom of the
> + scene. The top, bottom, left and right directions are intentionally not
> + defined and depend on the environment in which the camera is used.
> +
> + A typical example of a (very common) picture of a shark swimming from
> + left to right, as seen from the camera, is
In the previous paragraph, it is stated that "top, bottom, left and right
directions are intentionally not defined". How about something like
A typical example of a (very common) pictore of a shark swimming in
the direction of X axis, as seen from the camera, is
> +
> + 0 X-axis
> + 0 +------------------------------------->
> + !
> + !
> + !
> + ! |\____)\___
> + ! ) _____ __`<
> + ! |/ )/
> + !
> + !
> + !
> + V
> + Y-axis
> +
> + With the reference system 'Rs' placed on the camera focal plane.
> +
> + ¸.·˙!
> + ¸.·˙ !
> + _ ¸.·˙ !
> + +-/ \-+¸.·˙ !
> + | (o) | ! Camera focal plane
> + +-----+˙·.¸ !
> + ˙·.¸ !
> + ˙·.¸ !
> + ˙·.¸!
> +
> + When projected on the sensor's pixel array, the image and the associated
> + reference system 'Rs' are typically (but not always) inverted, due to
> + the camera module's lens optical inversion effect.
> +
> + Assuming the above represented scene of the swimming shark, the lens
> + inversion projects the scene and its reference system onto the sensor
> + pixel array, seen from the front of the camera sensor, as follow
> +
> + Y-axis
> + ^
> + !
> + !
> + !
> + ! |\_____)\__
> + ! ) ____ ___.<
> + ! |/ )/
> + !
> + !
> + !
> + 0 +------------------------------------->
> + 0 X-axis
> +
> + Note the shark being upside-down.
As it's inverted, wouldn't the same happen on the X axis as well?
> +
> + The resulting projected reference system is named 'Rp'.
> +
> + The camera rotation property is then defined as the angular difference
> + in the counter-clockwise direction between the camera reference system
> + 'Rc' and the projected scene reference system 'Rp'. It is expressed in
> + degrees as a number in the range [0, 360[.
> +
> + Examples
> +
> + 0 degrees camera rotation
> +
> +
> + Y-Rp
> + ^
> + Y-Rc !
> + ^ !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + 0 +------------------------------------->
> + 0 X-Rc
> +
> +
> + X-Rc 0
> + <------------------------------------+ 0
> + X-Rp 0 !
> + <------------------------------------+ 0 !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! V
> + ! Y-Rc
> + V
> + Y-Rp
> +
> + 90 degrees camera rotation
> +
> + 0 Y-Rc
> + 0 +-------------------->
> + ! Y-Rp
> + ! ^
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + !
> + !
> + !
> + !
> + V
> + X-Rc
> +
> + 180 degrees camera rotation
> +
> + 0
> + <------------------------------------+ 0
> + X-Rc !
> + Y-Rp !
> + ^ !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! V
> + ! Y-Rc
> + 0 +------------------------------------->
> + 0 X-Rp
> +
> + 270 degrees camera rotation
> +
> + 0 Y-Rc
> + 0 +-------------------->
> + ! 0
> + ! <-----------------------------------+ 0
> + ! X-Rp !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! !
> + ! V
> + ! Y-Rp
> + !
> + !
> + !
> + !
> + V
> + X-Rc
> +
> +
> + Example one - Webcam
> +
> + A camera module installed on the user facing part of a laptop screen
> + casing used for video calls. The captured images are meant to be
> + displayed in landscape mode (width > height) on the laptop screen.
> +
> + The camera is typically mounted upside-down to compensate the lens
> + optical inversion effect.
> +
> + Y-Rp
> + Y-Rc ^
> + ^ !
> + ! !
> + ! ! |\_____)\__
> + ! ! ) ____ ___.<
> + ! ! |/ )/
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + 0 +------------------------------------->
> + 0 X-Rc
> +
> + The two reference systems are aligned, the resulting camera rotation is
> + 0 degrees, no rotation correction needs to be applied to the resulting
> + image once captured to memory buffers to correctly display it to users.
> +
> + +--------------------------------------+
> + ! !
> + ! !
> + ! !
> + ! |\____)\___ !
> + ! ) _____ __`< !
> + ! |/ )/ !
> + ! !
> + ! !
> + ! !
> + +--------------------------------------+
> +
> + If the camera sensor is not mounted upside-down to compensate for the
> + lens optical inversion, the two reference systems will not be aligned,
> + with 'Rp' being rotated 180 degrees relatively to 'Rc'.
I'd say the camera is mounted upright (rotation 0) if the image it produces
is read from the top-left corner towards the bottom. That is what the
current rotation property refers to, and I don't think we can change that
anymore.
Note that the lens is a part of the camera (or camera module). Could this
be defined for the camera module instead, to be compatible with the
existing definition and to take the entire lens system in the module into
account?
> +
> +
> + X-Rc 0
> + <------------------------------------+ 0
> + !
> + Y-Rp !
> + ^ !
> + ! !
> + ! |\_____)\__ !
> + ! ) ____ ___.< !
> + ! |/ )/ !
> + ! !
> + ! !
> + ! V
> + ! Y-Rc
> + 0 +------------------------------------->
> + 0 X-Rp
> +
> + The image once captured to memory will then be rotated by 180 degrees
> +
> + +--------------------------------------+
> + ! !
> + ! !
> + ! !
> + ! __/(_____/| !
> + ! >.___ ____ ( !
> + ! \( \| !
> + ! !
> + ! !
> + ! !
> + +--------------------------------------+
> +
> + A software rotation correction of 180 degrees should be applied to
> + correctly display the image.
> +
> + +--------------------------------------+
> + ! !
> + ! !
> + ! !
> + ! |\____)\___ !
> + ! ) _____ __`< !
> + ! |/ )/ !
> + ! !
> + ! !
> + ! !
> + +--------------------------------------+
> +
> + Example two - Phone camera
> +
> + A camera installed on the back side of a mobile device facing away from
> + the user. The captured images are meant to be displayed in portrait mode
> + (height > width) to match the device screen orientation and the device
> + usage orientation used when taking the picture.
> +
> + The camera sensor is typically mounted with its pixel array longer side
> + aligned to the device longer side, upside-down mounted to compensate for
> + the lens optical inversion effect.
> +
> + 0 Y-Rc
> + 0 +-------------------->
> + ! Y-Rp
> + ! ^
> + ! !
> + ! !
> + ! !
> + ! ! |\_____)\__
> + ! ! ) ____ ___.<
> + ! ! |/ )/
> + ! !
> + ! !
> + ! !
> + ! 0 +------------------------------------->
> + ! 0 X-Rp
> + !
> + !
> + !
> + !
> + V
> + X-Rc
> +
> + The two reference systems are not aligned and the 'Rp' reference
> + system is rotated by 90 degrees in the counter-clockwise direction
> + relatively to the 'Rc' reference system.
> +
> + The image once captured to memory will be rotated.
> +
> + +-------------------------------------+
> + | _ _ |
> + | \ / |
> + | | | |
> + | | | |
> + | | > |
> + | < | |
> + | | | |
> + | . |
> + | V |
> + +-------------------------------------+
> +
> + A correction of 90 degrees in counter-clockwise direction has to be
> + applied to correctly display the image in portrait mode on the device
> + screen.
> +
> + +--------------------+
> + | |
> + | |
> + | |
> + | |
> + | |
> + | |
> + | |\____)\___ |
> + | ) _____ __`< |
> + | |/ )/ |
> + | |
> + | |
> + | |
> + | |
> + | |
> + +--------------------+
>
> - location: The mount location of a device (typically an image sensor or a flash
> LED) expressed as a position relative to the usage orientation of the system
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description
2020-06-08 10:09 ` Sakari Ailus
@ 2020-06-08 21:41 ` Laurent Pinchart
0 siblings, 0 replies; 5+ messages in thread
From: Laurent Pinchart @ 2020-06-08 21:41 UTC (permalink / raw)
To: Sakari Ailus
Cc: Jacopo Mondi, Mauro Carvalho Chehab, Hans Verkuil, tfiga, pavel,
open list:MEDIA INPUT INFRASTRUCTURE (V4L/DVB), libcamera-devel,
robh, devicetree, Niklas Soderlund
Hi Sakari,
On Mon, Jun 08, 2020 at 01:09:03PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
>
> Thank you for this great piece of ASCII art. Also my apologies for
> reviewing this so late. The latest version is 11 but I believe the content
> is the same on the commented matters.
>
> On Wed, Mar 18, 2020 at 09:50:26PM +0100, Jacopo Mondi wrote:
> > Replace the 'rotation' property description by providing a definition
> > relative to the camera sensor pixel array coordinate system and the
> > captured scene.
> >
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > Co-developed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Co-developed-by: Niklas Soderlund <niklas.soderlund@ragnatech.se>
> >
> > ---
> > .../bindings/media/video-interfaces.txt | 359 +++++++++++++++++-
> > 1 file changed, 356 insertions(+), 3 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
> > index 1211bdf80722..36e61d599f34 100644
> > --- a/Documentation/devicetree/bindings/media/video-interfaces.txt
> > +++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
> > @@ -85,9 +85,362 @@ Optional properties
> >
> > - lens-focus: A phandle to the node of the focus lens controller.
> >
> > -- rotation: The device, typically an image sensor, is not mounted upright,
> > - but a number of degrees counter clockwise. Typical values are 0 and 180
> > - (upside down).
> > +- rotation: The camera rotation is expressed as the angular difference in
> > + degrees between two reference systems, one relative to the camera module, and
> > + one defined on the external world scene to be captured when projected on the
> > + image sensor pixel array.
> > +
> > + A camera sensor has a 2-dimensional reference system 'Rc' defined by
> > + its pixel array read-out order. The origin is set to the first pixel
> > + being read out, the X-axis points along the column read-out direction
> > + towards the last columns, and the Y-axis along the row read-out
> > + direction towards the last row.
> > +
> > + A typical example for a sensor with a 2592x1944 pixel array matrix
> > + observed from the front is
> > +
> > + 2591 X-axis 0
> > + <------------------------+ 0
> > + .......... ... ..........!
> > + .......... ... ..........! Y-axis
> > + ... !
> > + .......... ... ..........!
> > + .......... ... ..........! 1943
> > + V
> > +
> > + The external world scene reference system 'Rs' is a 2-dimensional
> > + reference system on the focal plane of the camera module. The origin is
> > + placed on the top-left corner of the visible scene, the X-axis points
> > + towards the right, and the Y-axis points towards the bottom of the
> > + scene. The top, bottom, left and right directions are intentionally not
> > + defined and depend on the environment in which the camera is used.
> > +
> > + A typical example of a (very common) picture of a shark swimming from
> > + left to right, as seen from the camera, is
>
> In the previous paragraph, it is stated that "top, bottom, left and right
> directions are intentionally not defined". How about something like
>
> A typical example of a (very common) pictore of a shark swimming in
> the direction of X axis, as seen from the camera, is
>
> > +
> > + 0 X-axis
> > + 0 +------------------------------------->
> > + !
> > + !
> > + !
> > + ! |\____)\___
> > + ! ) _____ __`<
> > + ! |/ )/
> > + !
> > + !
> > + !
> > + V
> > + Y-axis
> > +
> > + With the reference system 'Rs' placed on the camera focal plane.
> > +
> > + ¸.·˙!
> > + ¸.·˙ !
> > + _ ¸.·˙ !
> > + +-/ \-+¸.·˙ !
> > + | (o) | ! Camera focal plane
> > + +-----+˙·.¸ !
> > + ˙·.¸ !
> > + ˙·.¸ !
> > + ˙·.¸!
> > +
> > + When projected on the sensor's pixel array, the image and the associated
> > + reference system 'Rs' are typically (but not always) inverted, due to
> > + the camera module's lens optical inversion effect.
> > +
> > + Assuming the above represented scene of the swimming shark, the lens
> > + inversion projects the scene and its reference system onto the sensor
> > + pixel array, seen from the front of the camera sensor, as follow
> > +
> > + Y-axis
> > + ^
> > + !
> > + !
> > + !
> > + ! |\_____)\__
> > + ! ) ____ ___.<
> > + ! |/ )/
> > + !
> > + !
> > + !
> > + 0 +------------------------------------->
> > + 0 X-axis
> > +
> > + Note the shark being upside-down.
>
> As it's inverted, wouldn't the same happen on the X axis as well?
There's a double inversion. The lens will invert the image along both
axes, but this picture shows the image as projected on the camera
sensor, while the previous image showed the scene when looking from the
camera sensor. As humans have a tendency to turn their head in the
horizontal plane and not the vertical plane when they want to look
behind them, switching from looking towards the scene to looking towards
the sensor inverts the X axis. The end result is that only the Y axis is
inverted.
> > +
> > + The resulting projected reference system is named 'Rp'.
> > +
> > + The camera rotation property is then defined as the angular difference
> > + in the counter-clockwise direction between the camera reference system
> > + 'Rc' and the projected scene reference system 'Rp'. It is expressed in
> > + degrees as a number in the range [0, 360[.
> > +
> > + Examples
> > +
> > + 0 degrees camera rotation
> > +
> > +
> > + Y-Rp
> > + ^
> > + Y-Rc !
> > + ^ !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! 0 +------------------------------------->
> > + ! 0 X-Rp
> > + 0 +------------------------------------->
> > + 0 X-Rc
> > +
> > +
> > + X-Rc 0
> > + <------------------------------------+ 0
> > + X-Rp 0 !
> > + <------------------------------------+ 0 !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! V
> > + ! Y-Rc
> > + V
> > + Y-Rp
> > +
> > + 90 degrees camera rotation
> > +
> > + 0 Y-Rc
> > + 0 +-------------------->
> > + ! Y-Rp
> > + ! ^
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! 0 +------------------------------------->
> > + ! 0 X-Rp
> > + !
> > + !
> > + !
> > + !
> > + V
> > + X-Rc
> > +
> > + 180 degrees camera rotation
> > +
> > + 0
> > + <------------------------------------+ 0
> > + X-Rc !
> > + Y-Rp !
> > + ^ !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! V
> > + ! Y-Rc
> > + 0 +------------------------------------->
> > + 0 X-Rp
> > +
> > + 270 degrees camera rotation
> > +
> > + 0 Y-Rc
> > + 0 +-------------------->
> > + ! 0
> > + ! <-----------------------------------+ 0
> > + ! X-Rp !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! !
> > + ! V
> > + ! Y-Rp
> > + !
> > + !
> > + !
> > + !
> > + V
> > + X-Rc
> > +
> > +
> > + Example one - Webcam
> > +
> > + A camera module installed on the user facing part of a laptop screen
> > + casing used for video calls. The captured images are meant to be
> > + displayed in landscape mode (width > height) on the laptop screen.
> > +
> > + The camera is typically mounted upside-down to compensate the lens
> > + optical inversion effect.
> > +
> > + Y-Rp
> > + Y-Rc ^
> > + ^ !
> > + ! !
> > + ! ! |\_____)\__
> > + ! ! ) ____ ___.<
> > + ! ! |/ )/
> > + ! !
> > + ! !
> > + ! !
> > + ! 0 +------------------------------------->
> > + ! 0 X-Rp
> > + 0 +------------------------------------->
> > + 0 X-Rc
> > +
> > + The two reference systems are aligned, the resulting camera rotation is
> > + 0 degrees, no rotation correction needs to be applied to the resulting
> > + image once captured to memory buffers to correctly display it to users.
> > +
> > + +--------------------------------------+
> > + ! !
> > + ! !
> > + ! !
> > + ! |\____)\___ !
> > + ! ) _____ __`< !
> > + ! |/ )/ !
> > + ! !
> > + ! !
> > + ! !
> > + +--------------------------------------+
> > +
> > + If the camera sensor is not mounted upside-down to compensate for the
> > + lens optical inversion, the two reference systems will not be aligned,
> > + with 'Rp' being rotated 180 degrees relatively to 'Rc'.
>
> I'd say the camera is mounted upright (rotation 0) if the image it produces
> is read from the top-left corner towards the bottom. That is what the
> current rotation property refers to, and I don't think we can change that
> anymore.
The camera module is mounted upright, while the camera sensor is mounted
upside-down :-)
> Note that the lens is a part of the camera (or camera module). Could this
> be defined for the camera module instead, to be compatible with the
> existing definition and to take the entire lens system in the module into
> account?
This documentation intentionally defines rotation using a scene
reference and a camera sensor reference in order to take the lens (and
anything else between the two) into account.
> > +
> > +
> > + X-Rc 0
> > + <------------------------------------+ 0
> > + !
> > + Y-Rp !
> > + ^ !
> > + ! !
> > + ! |\_____)\__ !
> > + ! ) ____ ___.< !
> > + ! |/ )/ !
> > + ! !
> > + ! !
> > + ! V
> > + ! Y-Rc
> > + 0 +------------------------------------->
> > + 0 X-Rp
> > +
> > + The image once captured to memory will then be rotated by 180 degrees
> > +
> > + +--------------------------------------+
> > + ! !
> > + ! !
> > + ! !
> > + ! __/(_____/| !
> > + ! >.___ ____ ( !
> > + ! \( \| !
> > + ! !
> > + ! !
> > + ! !
> > + +--------------------------------------+
> > +
> > + A software rotation correction of 180 degrees should be applied to
> > + correctly display the image.
> > +
> > + +--------------------------------------+
> > + ! !
> > + ! !
> > + ! !
> > + ! |\____)\___ !
> > + ! ) _____ __`< !
> > + ! |/ )/ !
> > + ! !
> > + ! !
> > + ! !
> > + +--------------------------------------+
> > +
> > + Example two - Phone camera
> > +
> > + A camera installed on the back side of a mobile device facing away from
> > + the user. The captured images are meant to be displayed in portrait mode
> > + (height > width) to match the device screen orientation and the device
> > + usage orientation used when taking the picture.
> > +
> > + The camera sensor is typically mounted with its pixel array longer side
> > + aligned to the device longer side, upside-down mounted to compensate for
> > + the lens optical inversion effect.
> > +
> > + 0 Y-Rc
> > + 0 +-------------------->
> > + ! Y-Rp
> > + ! ^
> > + ! !
> > + ! !
> > + ! !
> > + ! ! |\_____)\__
> > + ! ! ) ____ ___.<
> > + ! ! |/ )/
> > + ! !
> > + ! !
> > + ! !
> > + ! 0 +------------------------------------->
> > + ! 0 X-Rp
> > + !
> > + !
> > + !
> > + !
> > + V
> > + X-Rc
> > +
> > + The two reference systems are not aligned and the 'Rp' reference
> > + system is rotated by 90 degrees in the counter-clockwise direction
> > + relatively to the 'Rc' reference system.
> > +
> > + The image once captured to memory will be rotated.
> > +
> > + +-------------------------------------+
> > + | _ _ |
> > + | \ / |
> > + | | | |
> > + | | | |
> > + | | > |
> > + | < | |
> > + | | | |
> > + | . |
> > + | V |
> > + +-------------------------------------+
> > +
> > + A correction of 90 degrees in counter-clockwise direction has to be
> > + applied to correctly display the image in portrait mode on the device
> > + screen.
> > +
> > + +--------------------+
> > + | |
> > + | |
> > + | |
> > + | |
> > + | |
> > + | |
> > + | |\____)\___ |
> > + | ) _____ __`< |
> > + | |/ )/ |
> > + | |
> > + | |
> > + | |
> > + | |
> > + | |
> > + +--------------------+
> >
> > - location: The mount location of a device (typically an image sensor or a flash
> > LED) expressed as a position relative to the usage orientation of the system
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-06-08 21:41 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20200318205034.949531-1-jacopo@jmondi.org>
2020-03-18 20:50 ` [PATCH v7 03/11] dt-bindings: video-interface: Replace 'rotation' description Jacopo Mondi
2020-03-20 1:03 ` Rob Herring
2020-03-31 10:21 ` Hans Verkuil
2020-06-08 10:09 ` Sakari Ailus
2020-06-08 21:41 ` Laurent Pinchart
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).