[RFC v2 0/3] Document bindings for camera modules and associated flash devices

[RFC v2 0/3] Document bindings for camera modules and associated flash devices

[Sakari Ailus]

This RFC patchset documents properties commonly required by camera modules
and associated camera flash devices.

The camera module is essentially a package consisting of an image sensor, 
a lens, possibly a voice coil to move the lens and a number of other
things that at least the drivers need not to know of. All the devices in a
camera module are declared separately in the system and as such the fact
that they come in a single package isn't generally very useful to driver
software. 

I'm sending the set as RFC as there's no driver implementation, and a 
dependency to the V4L2 async changes:

<URL:http://www.spinics.net/lists/linux-media/msg114915.html>

since RFC v1:

- Remove sentences elaborating applicability of the bindings.

- Say a LED driver is a piece of hardware.

- Otherwise reformulate the descriptions according to the comments.

Sakari Ailus (3):
  dt: bindings: Add a binding for flash devices associated to a sensor
  dt: bindings: Add lens-focus binding for image sensors
  dt: bindings: Add a binding for referencing EEPROM from camera sensors

 .../devicetree/bindings/media/video-interfaces.txt       | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

-- 
2.7.4

[RFC v2 1/3] dt: bindings: Add a binding for flash devices associated to a sensor

[Sakari Ailus]

Camera flash drivers (and LEDs) are separate from the sensor devices in
DT. In order to make an association between the two, provide the
association information to the software.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Pavel Machek <pavel@ucw.cz>
Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>
---
 Documentation/devicetree/bindings/media/video-interfaces.txt | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
index 9cd2a36..dac764b 100644
--- a/Documentation/devicetree/bindings/media/video-interfaces.txt
+++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
@@ -67,6 +67,17 @@ are required in a relevant parent node:
 		    identifier, should be 1.
  - #size-cells    : should be zero.
 
+
+Optional properties
+-------------------
+
+- flash: An array of phandles that refer to the flash light sources
+  related to an image sensor. These could be e.g. LEDs. In case the LED
+  driver (current sink or source chip for the LED(s)) drives more than a
+  single LED, then the phandles here refer to the child nodes of the LED
+  driver describing individual LEDs.
+
+
 Optional endpoint properties
 ----------------------------
 
-- 
2.7.4

Re: [RFC v2 1/3] dt: bindings: Add a binding for flash devices associated to a sensor

[Rob Herring]

On Fri, May 05, 2017 at 11:48:28AM +0300, Sakari Ailus wrote:
> Camera flash drivers (and LEDs) are separate from the sensor devices in
> DT. In order to make an association between the two, provide the
> association information to the software.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Acked-by: Pavel Machek <pavel@ucw.cz>
> Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>
> ---
>  Documentation/devicetree/bindings/media/video-interfaces.txt | 11 +++++++++++
>  1 file changed, 11 insertions(+)

Acked-by: Rob Herring <robh@kernel.org>

> 
> diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
> index 9cd2a36..dac764b 100644
> --- a/Documentation/devicetree/bindings/media/video-interfaces.txt
> +++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
> @@ -67,6 +67,17 @@ are required in a relevant parent node:
>  		    identifier, should be 1.
>   - #size-cells    : should be zero.
>  
> +
> +Optional properties
> +-------------------
> +
> +- flash: An array of phandles that refer to the flash light sources
> +  related to an image sensor. These could be e.g. LEDs. In case the LED
> +  driver (current sink or source chip for the LED(s)) drives more than a
> +  single LED, then the phandles here refer to the child nodes of the LED
> +  driver describing individual LEDs.
> +
> +
>  Optional endpoint properties
>  ----------------------------
>  
> -- 
> 2.7.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC v2 2/3] dt: bindings: Add lens-focus binding for image sensors

[Sakari Ailus]

The lens-focus property contains a phandle to the lens voice coil driver
that is associated to the sensor; typically both are contained in the same
camera module.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Pavel Machek <pavel@ucw.cz>
---
 Documentation/devicetree/bindings/media/video-interfaces.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
index dac764b..0a33240 100644
--- a/Documentation/devicetree/bindings/media/video-interfaces.txt
+++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
@@ -77,6 +77,8 @@ Optional properties
   single LED, then the phandles here refer to the child nodes of the LED
   driver describing individual LEDs.
 
+- lens-focus: A phandle to the node of the focus lens controller.
+
 
 Optional endpoint properties
 ----------------------------
-- 
2.7.4

Re: [RFC v2 2/3] dt: bindings: Add lens-focus binding for image sensors

[Sebastian Reichel]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

Hi,

On Fri, May 05, 2017 at 11:48:29AM +0300, Sakari Ailus wrote:
> The lens-focus property contains a phandle to the lens voice coil driver
> that is associated to the sensor; typically both are contained in the same
> camera module.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Acked-by: Pavel Machek <pavel@ucw.cz>

Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>

> ---
>  Documentation/devicetree/bindings/media/video-interfaces.txt | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
> index dac764b..0a33240 100644
> --- a/Documentation/devicetree/bindings/media/video-interfaces.txt
> +++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
> @@ -77,6 +77,8 @@ Optional properties
>    single LED, then the phandles here refer to the child nodes of the LED
>    driver describing individual LEDs.
>  
> +- lens-focus: A phandle to the node of the focus lens controller.
> +
>  
>  Optional endpoint properties
>  ----------------------------

-- Sebastian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC v2 2/3] dt: bindings: Add lens-focus binding for image sensors

[Rob Herring]

On Fri, May 05, 2017 at 11:48:29AM +0300, Sakari Ailus wrote:
> The lens-focus property contains a phandle to the lens voice coil driver
> that is associated to the sensor; typically both are contained in the same
> camera module.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Acked-by: Pavel Machek <pavel@ucw.cz>
> ---
>  Documentation/devicetree/bindings/media/video-interfaces.txt | 2 ++
>  1 file changed, 2 insertions(+)

Acked-by: Rob Herring <robh@kernel.org>

[RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Sakari Ailus]

Many camera sensor devices contain EEPROM chips that describe the
properties of a given unit --- the data is specific to a given unit can
thus is not stored e.g. in user space or the driver.

Some sensors embed the EEPROM chip and it can be accessed through the
sensor's I2C interface. This property is to be used for devices where the
EEPROM chip is accessed through a different I2C address than the sensor.

The intent is to later provide this information to the user space.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Pavel Machek <pavel@ucw.cz>
---
 Documentation/devicetree/bindings/media/video-interfaces.txt | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
index 0a33240..38e3916 100644
--- a/Documentation/devicetree/bindings/media/video-interfaces.txt
+++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
@@ -79,6 +79,9 @@ Optional properties
 
 - lens-focus: A phandle to the node of the focus lens controller.
 
+- eeprom: A phandle to the node of the EEPROM describing the camera sensor
+  (i.e. device specific calibration data), in case it differs from the
+  sensor node.
 
 Optional endpoint properties
 ----------------------------
-- 
2.7.4

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Sebastian Reichel]

[-- Attachment #1: Type: text/plain, Size: 1447 bytes --]

Hi,

On Fri, May 05, 2017 at 11:48:30AM +0300, Sakari Ailus wrote:
> Many camera sensor devices contain EEPROM chips that describe the
> properties of a given unit --- the data is specific to a given unit can
> thus is not stored e.g. in user space or the driver.
> 
> Some sensors embed the EEPROM chip and it can be accessed through the
> sensor's I2C interface. This property is to be used for devices where the
> EEPROM chip is accessed through a different I2C address than the sensor.
> 
> The intent is to later provide this information to the user space.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Acked-by: Pavel Machek <pavel@ucw.cz>

Reviewed-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk>

> diff --git a/Documentation/devicetree/bindings/media/video-interfaces.txt b/Documentation/devicetree/bindings/media/video-interfaces.txt
> index 0a33240..38e3916 100644
> --- a/Documentation/devicetree/bindings/media/video-interfaces.txt
> +++ b/Documentation/devicetree/bindings/media/video-interfaces.txt
> @@ -79,6 +79,9 @@ Optional properties
>  
>  - lens-focus: A phandle to the node of the focus lens controller.
>  
> +- eeprom: A phandle to the node of the EEPROM describing the camera sensor
> +  (i.e. device specific calibration data), in case it differs from the
> +  sensor node.
>  
>  Optional endpoint properties
>  ----------------------------

-- Sebastian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Rob Herring]

On Fri, May 05, 2017 at 11:48:30AM +0300, Sakari Ailus wrote:
> Many camera sensor devices contain EEPROM chips that describe the
> properties of a given unit --- the data is specific to a given unit can
> thus is not stored e.g. in user space or the driver.
> 
> Some sensors embed the EEPROM chip and it can be accessed through the
> sensor's I2C interface. This property is to be used for devices where the
> EEPROM chip is accessed through a different I2C address than the sensor.

Different I2C address or bus? We already have i2c bindings for sub 
devices downstream of another I2C device. Either the upstream device 
passes thru the I2C transactions or itself is an I2C controller with a 
separate downstream bus. For those cases the EEPROM should be a child 
node. A phandle only makes sense if you have the sensor and eeprom 
connected to 2 entirely separate host buses.

Rob

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Sakari Ailus]

Hi Rob,

On Mon, May 08, 2017 at 12:24:18PM -0500, Rob Herring wrote:
> On Fri, May 05, 2017 at 11:48:30AM +0300, Sakari Ailus wrote:
> > Many camera sensor devices contain EEPROM chips that describe the
> > properties of a given unit --- the data is specific to a given unit can
> > thus is not stored e.g. in user space or the driver.
> > 
> > Some sensors embed the EEPROM chip and it can be accessed through the
> > sensor's I2C interface. This property is to be used for devices where the
> > EEPROM chip is accessed through a different I2C address than the sensor.
> 
> Different I2C address or bus? We already have i2c bindings for sub 
> devices downstream of another I2C device. Either the upstream device 
> passes thru the I2C transactions or itself is an I2C controller with a 
> separate downstream bus. For those cases the EEPROM should be a child 
> node. A phandle only makes sense if you have the sensor and eeprom 
> connected to 2 entirely separate host buses.

Right. It's a different address but located in the same package with the
module, just like the lens. I should have actually said "module", not the
"sensor".

The EEPROM integration to the module is done by the module vendor, but it's
entirely possible to have another module vendor to use the sensor but not
add an EEPROM or use a different EEPROM. It is also possible that the EEPROM
is on the same silicon with the sensor, then it's always guaranteed to be
there.

The bottom line is still that a sensor is simply one of the component in a
camera module; the rest of the module (lens, eeprom etc.) is not defined by
the sensor, the parts are rather picked by the module vendor. Yet the sensor
is a central piece in a module: it's always guaranteed to be there or it's
not a camera module.

Cc Daniel who was asking a related question.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Daniel Mack]

On 05/29/2017 02:20 PM, Sakari Ailus wrote:
> On Mon, May 08, 2017 at 12:24:18PM -0500, Rob Herring wrote:
>> On Fri, May 05, 2017 at 11:48:30AM +0300, Sakari Ailus wrote:
>>> Many camera sensor devices contain EEPROM chips that describe the
>>> properties of a given unit --- the data is specific to a given unit can
>>> thus is not stored e.g. in user space or the driver.
>>>
>>> Some sensors embed the EEPROM chip and it can be accessed through the
>>> sensor's I2C interface. This property is to be used for devices where the
>>> EEPROM chip is accessed through a different I2C address than the sensor.
>>
>> Different I2C address or bus? We already have i2c bindings for sub 
>> devices downstream of another I2C device. Either the upstream device 
>> passes thru the I2C transactions or itself is an I2C controller with a 
>> separate downstream bus. For those cases the EEPROM should be a child 
>> node. A phandle only makes sense if you have the sensor and eeprom 
>> connected to 2 entirely separate host buses.
> 
> Right. It's a different address but located in the same package with the
> module, just like the lens. I should have actually said "module", not the
> "sensor".
> 
> The EEPROM integration to the module is done by the module vendor, but it's
> entirely possible to have another module vendor to use the sensor but not
> add an EEPROM or use a different EEPROM. It is also possible that the EEPROM
> is on the same silicon with the sensor, then it's always guaranteed to be
> there.
> 
> The bottom line is still that a sensor is simply one of the component in a
> camera module; the rest of the module (lens, eeprom etc.) is not defined by
> the sensor, the parts are rather picked by the module vendor. Yet the sensor
> is a central piece in a module: it's always guaranteed to be there or it's
> not a camera module.

I agree, yes. I think the only way to solve this is to have a generic
EEPROM API that allows the camera sensor to read data from it. If
another vendor uses a different type of EEPROM, the sensor driver would
remain the same, as it only reads data from the storage behind the
phandle, not caring about the details.

Same goes for the lens driver, and after thinking about it for awhile,
I'd say it makes most sense to allow referencing a v4l2_subdev device
through a phandle from another v4l2_subdev, and then offload certain
commands such as V4L2_CID_FOCUS_ABSOLUTE to the device that does the
actual work. Opinions?


Thanks,
Daniel

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Sakari Ailus]

Hi Daniel,

On Mon, May 29, 2017 at 02:39:01PM +0200, Daniel Mack wrote:
> On 05/29/2017 02:20 PM, Sakari Ailus wrote:
> > On Mon, May 08, 2017 at 12:24:18PM -0500, Rob Herring wrote:
> >> On Fri, May 05, 2017 at 11:48:30AM +0300, Sakari Ailus wrote:
> >>> Many camera sensor devices contain EEPROM chips that describe the
> >>> properties of a given unit --- the data is specific to a given unit can
> >>> thus is not stored e.g. in user space or the driver.
> >>>
> >>> Some sensors embed the EEPROM chip and it can be accessed through the
> >>> sensor's I2C interface. This property is to be used for devices where the
> >>> EEPROM chip is accessed through a different I2C address than the sensor.
> >>
> >> Different I2C address or bus? We already have i2c bindings for sub 
> >> devices downstream of another I2C device. Either the upstream device 
> >> passes thru the I2C transactions or itself is an I2C controller with a 
> >> separate downstream bus. For those cases the EEPROM should be a child 
> >> node. A phandle only makes sense if you have the sensor and eeprom 
> >> connected to 2 entirely separate host buses.
> > 
> > Right. It's a different address but located in the same package with the
> > module, just like the lens. I should have actually said "module", not the
> > "sensor".
> > 
> > The EEPROM integration to the module is done by the module vendor, but it's
> > entirely possible to have another module vendor to use the sensor but not
> > add an EEPROM or use a different EEPROM. It is also possible that the EEPROM
> > is on the same silicon with the sensor, then it's always guaranteed to be
> > there.
> > 
> > The bottom line is still that a sensor is simply one of the component in a
> > camera module; the rest of the module (lens, eeprom etc.) is not defined by
> > the sensor, the parts are rather picked by the module vendor. Yet the sensor
> > is a central piece in a module: it's always guaranteed to be there or it's
> > not a camera module.
> 
> I agree, yes. I think the only way to solve this is to have a generic
> EEPROM API that allows the camera sensor to read data from it. If

We have one already, and it's defined in Documentation/misc-devices/eeprom .

> another vendor uses a different type of EEPROM, the sensor driver would
> remain the same, as it only reads data from the storage behind the
> phandle, not caring about the details.
> 
> Same goes for the lens driver, and after thinking about it for awhile,
> I'd say it makes most sense to allow referencing a v4l2_subdev device
> through a phandle from another v4l2_subdev, and then offload certain
> commands such as V4L2_CID_FOCUS_ABSOLUTE to the device that does the
> actual work. Opinions?

There are different kinds of lens systems and I don't think the sensor
drivers should be aware of them. The current approach is that the lens is a
separate sub-device --- the intent of the patchset I posted was to document
how the information on the related lens and eeprom components is conveyed to
the software. There's one such driver in the mainline kernel, ad5820.

Unfortunately we don't right now have a good user space interface for
telling which sensor a lens device is related to. The struct
media_entity_desc does have a group_id field for grouping the sub-devices
but that's hardly a good way to describe this.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1669 bytes --]

Hi!

> > I agree, yes. I think the only way to solve this is to have a generic
> > EEPROM API that allows the camera sensor to read data from it. If
> 
> We have one already, and it's defined in Documentation/misc-devices/eeprom .
> 
> > another vendor uses a different type of EEPROM, the sensor driver would
> > remain the same, as it only reads data from the storage behind the
> > phandle, not caring about the details.
> > 
> > Same goes for the lens driver, and after thinking about it for awhile,
> > I'd say it makes most sense to allow referencing a v4l2_subdev device
> > through a phandle from another v4l2_subdev, and then offload certain
> > commands such as V4L2_CID_FOCUS_ABSOLUTE to the device that does the
> > actual work. Opinions?
> 
> There are different kinds of lens systems and I don't think the sensor
> drivers should be aware of them. The current approach is that the lens is a
> separate sub-device --- the intent of the patchset I posted was to document
> how the information on the related lens and eeprom components is conveyed to
> the software. There's one such driver in the mainline kernel, ad5820.
> 
> Unfortunately we don't right now have a good user space interface for
> telling which sensor a lens device is related to. The struct
> media_entity_desc does have a group_id field for grouping the sub-devices
> but that's hardly a good way to describe this.

Yeah, it would be good to get the corresponding patches to be merged
to v4l-utils...

								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC v2 3/3] dt: bindings: Add a binding for referencing EEPROM from camera sensors

[Sakari Ailus]

On Wed, May 31, 2017 at 07:37:18PM +0200, Pavel Machek wrote:
> Yeah, it would be good to get the corresponding patches to be merged
> to v4l-utils...

There are also no drivers to make use of this currently --- I certainly
don't object using the group_id before better solutions are available. But
the kernel needs to have the information before it can use it. :-)

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk