Re: [PATCH 05/14] media: add a V4L2 OF parser

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

On 09/27/2012 04:07 PM, Guennadi Liakhovetski wrote:
> Add a V4L2 OF parser, implementing bindings, documented in
> Documentation/devicetree/bindings/media/v4l2.txt.
> 
> Signed-off-by: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> ---
>   drivers/media/v4l2-core/Makefile  |    3 +
>   drivers/media/v4l2-core/v4l2-of.c |  190 +++++++++++++++++++++++++++++++++++++
>   include/media/v4l2-of.h           |   62 ++++++++++++
>   3 files changed, 255 insertions(+), 0 deletions(-)
>   create mode 100644 drivers/media/v4l2-core/v4l2-of.c
>   create mode 100644 include/media/v4l2-of.h
> 
> diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
> index c2d61d4..00f64d6 100644
> --- a/drivers/media/v4l2-core/Makefile
> +++ b/drivers/media/v4l2-core/Makefile
> @@ -9,6 +9,9 @@ videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
>   ifeq ($(CONFIG_COMPAT),y)
>     videodev-objs += v4l2-compat-ioctl32.o
>   endif
> +ifeq ($(CONFIG_OF),y)
> +  videodev-objs += v4l2-of.o
> +endif
> 
>   obj-$(CONFIG_VIDEO_DEV) += videodev.o v4l2-int-device.o
>   obj-$(CONFIG_VIDEO_V4L2) += v4l2-common.o
> diff --git a/drivers/media/v4l2-core/v4l2-of.c b/drivers/media/v4l2-core/v4l2-of.c
> new file mode 100644
> index 0000000..f45d64b
> --- /dev/null
> +++ b/drivers/media/v4l2-core/v4l2-of.c
> @@ -0,0 +1,190 @@
> +/*
> + * V4L2 OF binding parsing library
> + *
> + * Copyright (C) 2012 Renesas Electronics Corp.
> + * Author: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of version 2 of the GNU General Public License as
> + * published by the Free Software Foundation.
> + */
> +#include<linux/kernel.h>
> +#include<linux/module.h>
> +#include<linux/of.h>
> +#include<linux/types.h>
> +
> +#include<media/v4l2-of.h>
> +
> +/*
> + * All properties are optional. If none are found, we don't set any flags. This
> + * means, the port has a static configuration and no properties have to be
> + * specified explicitly.
> + * If any properties are found, that identify the bus as parallel, and
> + * slave-mode isn't set, we set V4L2_MBUS_MASTER. Similarly, if we recognise the
> + * bus as serial CSI-2 and clock-noncontinuous isn't set, we set the
> + * V4L2_MBUS_CSI2_CONTINUOUS_CLOCK flag.
> + * The caller should hold a reference to "node."
> + */
> +void v4l2_of_parse_link(const struct device_node *node,
> +			struct v4l2_of_link *link)
> +{
> +	const struct device_node *port_node = of_get_parent(node);
> +	int size;
> +	unsigned int v;
> +	u32 data_lanes[ARRAY_SIZE(link->mipi_csi_2.data_lanes)];
> +	bool data_lanes_present;
> +
> +	memset(link, 0, sizeof(*link));
> +
> +	link->local_node = node;
> +
> +	/* Doesn't matter, whether the below two calls succeed */
> +	of_property_read_u32(port_node, "reg",&link->port);
> +	of_property_read_u32(node, "reg",&link->addr);
> +
> +	if (!of_property_read_u32(node, "bus-width",&v))
> +		link->parallel.bus_width = v;
> +
> +	if (!of_property_read_u32(node, "data-shift",&v))
> +		link->parallel.data_shift = v;
> +
> +	if (!of_property_read_u32(node, "hsync-active",&v))
> +		link->mbus_flags |= v ? V4L2_MBUS_HSYNC_ACTIVE_HIGH :
> +			V4L2_MBUS_HSYNC_ACTIVE_LOW;
> +
> +	if (!of_property_read_u32(node, "vsync-active",&v))
> +		link->mbus_flags |= v ? V4L2_MBUS_VSYNC_ACTIVE_HIGH :
> +			V4L2_MBUS_VSYNC_ACTIVE_LOW;
> +
> +	if (!of_property_read_u32(node, "data-active",&v))
> +		link->mbus_flags |= v ? V4L2_MBUS_DATA_ACTIVE_HIGH :
> +			V4L2_MBUS_DATA_ACTIVE_LOW;
> +
> +	if (!of_property_read_u32(node, "pclk-sample",&v))
> +		link->mbus_flags |= v ? V4L2_MBUS_PCLK_SAMPLE_RISING :
> +			V4L2_MBUS_PCLK_SAMPLE_FALLING;
> +
> +	if (!of_property_read_u32(node, "field-even-active",&v))
> +		link->mbus_flags |= v ? V4L2_MBUS_FIELD_EVEN_HIGH :
> +			V4L2_MBUS_FIELD_EVEN_LOW;
> +
> +	if (of_get_property(node, "slave-mode",&size))
> +		link->mbus_flags |= V4L2_MBUS_SLAVE;
> +
> +	/* If any parallel-bus properties have been found, skip serial ones */
> +	if (link->parallel.bus_width || link->parallel.data_shift ||
> +	    link->mbus_flags) {
> +		/* Default parallel bus-master */
> +		if (!(link->mbus_flags&  V4L2_MBUS_SLAVE))
> +			link->mbus_flags |= V4L2_MBUS_MASTER;
> +		return;
> +	}
> +
> +	if (!of_property_read_u32(node, "clock-lanes",&v))
> +		link->mipi_csi_2.clock_lane = v;
> +
> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> +					ARRAY_SIZE(data_lanes))) {
> +		int i;
> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];

It doesn't look like what we aimed for. The data-lanes array is supposed
to be of variable length, thus I don't think it can be parsed like that. 
Or am I missing something ? I think we need more something like below 
(based on of_property_read_u32_array(), not tested):


void v4l2_of_parse_mipi_csi2(const struct device_node *node,
				struct mipi_csi2 *mipi_csi2)
{
	struct property *prop = of_find_property(node, "data-lanes", NULL);
	u8 *out_data_lanes = mipi_csi_2->data_lanes;
	int num_lanes;
	const __be32 *val;

	if (!prop)
		return;

	mipi_csi2->num_lanes = 0;

	if (WARN (!prop->value, "Empty data-lanes property\n"))
		return;

	num_lanes = prop->length / sizeof(u32);
	if (WARN_ON(num_lanes > ARRAY_SIZE(mipi_csi_2->data_lanes))
		num_lanes = ARRAY_SIZE(mipi_csi_2->data_lanes);

	val = prop->value;
	while (num_lanes--)
		*out_data_lanes++ = be32_to_cpup(val++);

	mipi_csi2->num_lanes = num_lanes;
}

	v4l2_of_parse_mipi_csi2(node, &link->mipi_csi2);

> +		data_lanes_present = true;
> +	} else {
> +		data_lanes_present = false;
> +	}
> +
> +	if (of_get_property(node, "clock-noncontinuous",&size))
> +		link->mbus_flags |= V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> +
> +	if ((link->mipi_csi_2.clock_lane || data_lanes_present)&&
> +	    !(link->mbus_flags&  V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK))
> +		/* Default CSI-2: continuous clock */
> +		link->mbus_flags |= V4L2_MBUS_CSI2_CONTINUOUS_CLOCK;
> +}
> +EXPORT_SYMBOL(v4l2_of_parse_link);
> +
...
> diff --git a/include/media/v4l2-of.h b/include/media/v4l2-of.h
> new file mode 100644
> index 0000000..6fafedb
> --- /dev/null
> +++ b/include/media/v4l2-of.h
> @@ -0,0 +1,62 @@
> +/*
> + * V4L2 OF binding parsing library
> + *
> + * Copyright (C) 2012 Renesas Electronics Corp.
> + * Author: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of version 2 of the GNU General Public License as
> + * published by the Free Software Foundation.
> + */
> +#ifndef _V4L2_OF_H
> +#define _V4L2_OF_H
> +
> +#include<linux/list.h>
> +#include<linux/types.h>
> +
> +#include<media/v4l2-mediabus.h>
> +
> +struct device_node;
> +
> +struct v4l2_of_link {
> +	unsigned int port;
> +	unsigned int addr;
> +	struct list_head head;
> +	const struct device_node *local_node;
> +	const __be32 *remote;
> +	unsigned int mbus_flags;
> +	union {
> +		struct {
> +			unsigned char bus_width;
> +			unsigned char data_shift;
> +		} parallel;
> +		struct {
> +			unsigned char data_lanes[4];

Some devices are interested only in absolute number of lanes.
I can't see how we could specify the number of data lanes here.
Shouldn't something like 'unsigned char num_data_lanes;' be
added to this structure ?

> +			unsigned char clock_lane;
> +		} mipi_csi_2;
> +	};
> +};

--

Thanks,
Sylwester

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

Hi Sylwester

On Mon, 1 Oct 2012, Sylwester Nawrocki wrote:

> On 09/27/2012 04:07 PM, Guennadi Liakhovetski wrote:
> > Add a V4L2 OF parser, implementing bindings, documented in
> > Documentation/devicetree/bindings/media/v4l2.txt.
> > 
> > Signed-off-by: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> > ---
> >   drivers/media/v4l2-core/Makefile  |    3 +
> >   drivers/media/v4l2-core/v4l2-of.c |  190 +++++++++++++++++++++++++++++++++++++
> >   include/media/v4l2-of.h           |   62 ++++++++++++
> >   3 files changed, 255 insertions(+), 0 deletions(-)
> >   create mode 100644 drivers/media/v4l2-core/v4l2-of.c
> >   create mode 100644 include/media/v4l2-of.h
> > 
> > diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
> > index c2d61d4..00f64d6 100644
> > --- a/drivers/media/v4l2-core/Makefile
> > +++ b/drivers/media/v4l2-core/Makefile
> > @@ -9,6 +9,9 @@ videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
> >   ifeq ($(CONFIG_COMPAT),y)
> >     videodev-objs += v4l2-compat-ioctl32.o
> >   endif
> > +ifeq ($(CONFIG_OF),y)
> > +  videodev-objs += v4l2-of.o
> > +endif
> > 
> >   obj-$(CONFIG_VIDEO_DEV) += videodev.o v4l2-int-device.o
> >   obj-$(CONFIG_VIDEO_V4L2) += v4l2-common.o
> > diff --git a/drivers/media/v4l2-core/v4l2-of.c b/drivers/media/v4l2-core/v4l2-of.c
> > new file mode 100644
> > index 0000000..f45d64b
> > --- /dev/null
> > +++ b/drivers/media/v4l2-core/v4l2-of.c
> > @@ -0,0 +1,190 @@
> > +/*
> > + * V4L2 OF binding parsing library
> > + *
> > + * Copyright (C) 2012 Renesas Electronics Corp.
> > + * Author: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of version 2 of the GNU General Public License as
> > + * published by the Free Software Foundation.
> > + */
> > +#include<linux/kernel.h>
> > +#include<linux/module.h>
> > +#include<linux/of.h>
> > +#include<linux/types.h>
> > +
> > +#include<media/v4l2-of.h>
> > +
> > +/*
> > + * All properties are optional. If none are found, we don't set any flags. This
> > + * means, the port has a static configuration and no properties have to be
> > + * specified explicitly.
> > + * If any properties are found, that identify the bus as parallel, and
> > + * slave-mode isn't set, we set V4L2_MBUS_MASTER. Similarly, if we recognise the
> > + * bus as serial CSI-2 and clock-noncontinuous isn't set, we set the
> > + * V4L2_MBUS_CSI2_CONTINUOUS_CLOCK flag.
> > + * The caller should hold a reference to "node."
> > + */
> > +void v4l2_of_parse_link(const struct device_node *node,
> > +			struct v4l2_of_link *link)
> > +{
> > +	const struct device_node *port_node = of_get_parent(node);
> > +	int size;
> > +	unsigned int v;
> > +	u32 data_lanes[ARRAY_SIZE(link->mipi_csi_2.data_lanes)];
> > +	bool data_lanes_present;
> > +
> > +	memset(link, 0, sizeof(*link));
> > +
> > +	link->local_node = node;
> > +
> > +	/* Doesn't matter, whether the below two calls succeed */
> > +	of_property_read_u32(port_node, "reg",&link->port);
> > +	of_property_read_u32(node, "reg",&link->addr);
> > +
> > +	if (!of_property_read_u32(node, "bus-width",&v))
> > +		link->parallel.bus_width = v;
> > +
> > +	if (!of_property_read_u32(node, "data-shift",&v))
> > +		link->parallel.data_shift = v;
> > +
> > +	if (!of_property_read_u32(node, "hsync-active",&v))
> > +		link->mbus_flags |= v ? V4L2_MBUS_HSYNC_ACTIVE_HIGH :
> > +			V4L2_MBUS_HSYNC_ACTIVE_LOW;
> > +
> > +	if (!of_property_read_u32(node, "vsync-active",&v))
> > +		link->mbus_flags |= v ? V4L2_MBUS_VSYNC_ACTIVE_HIGH :
> > +			V4L2_MBUS_VSYNC_ACTIVE_LOW;
> > +
> > +	if (!of_property_read_u32(node, "data-active",&v))
> > +		link->mbus_flags |= v ? V4L2_MBUS_DATA_ACTIVE_HIGH :
> > +			V4L2_MBUS_DATA_ACTIVE_LOW;
> > +
> > +	if (!of_property_read_u32(node, "pclk-sample",&v))
> > +		link->mbus_flags |= v ? V4L2_MBUS_PCLK_SAMPLE_RISING :
> > +			V4L2_MBUS_PCLK_SAMPLE_FALLING;
> > +
> > +	if (!of_property_read_u32(node, "field-even-active",&v))
> > +		link->mbus_flags |= v ? V4L2_MBUS_FIELD_EVEN_HIGH :
> > +			V4L2_MBUS_FIELD_EVEN_LOW;
> > +
> > +	if (of_get_property(node, "slave-mode",&size))
> > +		link->mbus_flags |= V4L2_MBUS_SLAVE;
> > +
> > +	/* If any parallel-bus properties have been found, skip serial ones */
> > +	if (link->parallel.bus_width || link->parallel.data_shift ||
> > +	    link->mbus_flags) {
> > +		/* Default parallel bus-master */
> > +		if (!(link->mbus_flags&  V4L2_MBUS_SLAVE))
> > +			link->mbus_flags |= V4L2_MBUS_MASTER;
> > +		return;
> > +	}
> > +
> > +	if (!of_property_read_u32(node, "clock-lanes",&v))
> > +		link->mipi_csi_2.clock_lane = v;
> > +
> > +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> > +					ARRAY_SIZE(data_lanes))) {
> > +		int i;
> > +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> > +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> 
> It doesn't look like what we aimed for. The data-lanes array is supposed
> to be of variable length, thus I don't think it can be parsed like that. 
> Or am I missing something ? I think we need more something like below 
> (based on of_property_read_u32_array(), not tested):

Ok, you're right, that my version only was suitable for fixed-size arrays, 
which wasn't our goal. But I don't think we should go down to manually 
parsing property data. How about (tested;-))

	data = of_find_property(node, "data-lanes", NULL);
	if (data) {
		int i = 0;
		const __be32 *lane = NULL;
		do {
			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
		} while (lane && i++ < ARRAY_SIZE(data_lanes));
		link->mipi_csi_2.num_data_lanes = i;
		while (i--)
			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
	}

> void v4l2_of_parse_mipi_csi2(const struct device_node *node,
> 				struct mipi_csi2 *mipi_csi2)
> {
> 	struct property *prop = of_find_property(node, "data-lanes", NULL);
> 	u8 *out_data_lanes = mipi_csi_2->data_lanes;
> 	int num_lanes;
> 	const __be32 *val;
> 
> 	if (!prop)
> 		return;
> 
> 	mipi_csi2->num_lanes = 0;
> 
> 	if (WARN (!prop->value, "Empty data-lanes property\n"))
> 		return;
> 
> 	num_lanes = prop->length / sizeof(u32);
> 	if (WARN_ON(num_lanes > ARRAY_SIZE(mipi_csi_2->data_lanes))
> 		num_lanes = ARRAY_SIZE(mipi_csi_2->data_lanes);
> 
> 	val = prop->value;
> 	while (num_lanes--)
> 		*out_data_lanes++ = be32_to_cpup(val++);
> 
> 	mipi_csi2->num_lanes = num_lanes;
> }
> 
> 	v4l2_of_parse_mipi_csi2(node, &link->mipi_csi2);

[snip]

> > +struct v4l2_of_link {
> > +	unsigned int port;
> > +	unsigned int addr;
> > +	struct list_head head;
> > +	const struct device_node *local_node;
> > +	const __be32 *remote;
> > +	unsigned int mbus_flags;
> > +	union {
> > +		struct {
> > +			unsigned char bus_width;
> > +			unsigned char data_shift;
> > +		} parallel;
> > +		struct {
> > +			unsigned char data_lanes[4];
> 
> Some devices are interested only in absolute number of lanes.
> I can't see how we could specify the number of data lanes here.
> Shouldn't something like 'unsigned char num_data_lanes;' be
> added to this structure ?

Yes, good point. As you see above, I've added it.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

Hi Guennadi,

On 10/02/2012 11:49 AM, Guennadi Liakhovetski wrote:
>>> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
>>> +					ARRAY_SIZE(data_lanes))) {
>>> +		int i;
>>> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
>>> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
>>
>> It doesn't look like what we aimed for. The data-lanes array is supposed
>> to be of variable length, thus I don't think it can be parsed like that. 
>> Or am I missing something ? I think we need more something like below 
>> (based on of_property_read_u32_array(), not tested):
> 
> Ok, you're right, that my version only was suitable for fixed-size arrays, 
> which wasn't our goal. But I don't think we should go down to manually 
> parsing property data. How about (tested;-))
> 
> 	data = of_find_property(node, "data-lanes", NULL);
> 	if (data) {
> 		int i = 0;
> 		const __be32 *lane = NULL;
> 		do {
> 			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
> 		} while (lane && i++ < ARRAY_SIZE(data_lanes));
> 		link->mipi_csi_2.num_data_lanes = i;
> 		while (i--)
> 			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> 	}

Yes, that looks neat and does what it is supposed to do. :) Thanks!
For now, I'll trust you it works ;)

With regards to the remaining patches, it looks a bit scary to me how
complicated it got, perhaps mostly because of requirement to reference
host devices from subdevs. And it seems to rely on the existing SoC
camera infrastructure, which might imply lot's of work need to be done
for non soc-camera drivers. But I'm going to take a closer look and
comment more on the details at the corresponding patches.

--

Regards,
Sylwester

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Tue, 2 Oct 2012, Sylwester Nawrocki wrote:

> Hi Guennadi,
> 
> On 10/02/2012 11:49 AM, Guennadi Liakhovetski wrote:
> >>> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> >>> +					ARRAY_SIZE(data_lanes))) {
> >>> +		int i;
> >>> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> >>> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> >>
> >> It doesn't look like what we aimed for. The data-lanes array is supposed
> >> to be of variable length, thus I don't think it can be parsed like that. 
> >> Or am I missing something ? I think we need more something like below 
> >> (based on of_property_read_u32_array(), not tested):
> > 
> > Ok, you're right, that my version only was suitable for fixed-size arrays, 
> > which wasn't our goal. But I don't think we should go down to manually 
> > parsing property data. How about (tested;-))
> > 
> > 	data = of_find_property(node, "data-lanes", NULL);
> > 	if (data) {
> > 		int i = 0;
> > 		const __be32 *lane = NULL;
> > 		do {
> > 			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
> > 		} while (lane && i++ < ARRAY_SIZE(data_lanes));
> > 		link->mipi_csi_2.num_data_lanes = i;
> > 		while (i--)
> > 			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > 	}
> 
> Yes, that looks neat and does what it is supposed to do. :) Thanks!
> For now, I'll trust you it works ;)
> 
> With regards to the remaining patches, it looks a bit scary to me how
> complicated it got, perhaps mostly because of requirement to reference
> host devices from subdevs.

If you mean uses of the v4l2_of_get_remote() function, then it's the other 
way round: I'm accessing subdevices from bridges, which is also 
understandable - you need the subdevice.

Or do you mean the need to turn the master clock on and off from the 
subdevice driver? This shall be eliminated, once we switch to using the 
generic clock framework.

> And it seems to rely on the existing SoC
> camera infrastructure, which might imply lot's of work need to be done
> for non soc-camera drivers.

Sorry, what "it" is relying on soc-camera? The patch series consists of 
several generic patches, which have nothing to do with soc-camera, and 
patches, porting soc-camera to use OF with cameras. I think, complexity 
with soc-camera is also higher, than what you'd need with specific single 
bridge drivers, because it is generic.

A big part of the complexity is supporting deferred subdevice (sensor) 
probing. Beginning with what most bridge drivers currently use - static 
subdevice lists in platform data, for which they then register I2C 
devices, it is natural to implement that in 2 steps: (1) support directly 
registered I2C sensors from platform data, that request deferred probing 
until the bridge driver has been probed and is ready to handle them; (2) 
support I2C subdevices in DT. After this your bridge driver will support 3 
(!) modes in which subdevices can be initialised. In principle you could 
drop step (1) - supporting that isn't really required, but then the jump 
to (2) will be larger.

Another part of the complexity is specific to soc-camera, it comes from 
the way, how subdevices are represented in platform data - as platform 
devices with a bus ID, used to link bridges and subdevices. With DT those 
platform devices and bus ID have to be generated dynamically.

And you're right - soc-camera bridge drivers have the advantage, that the 
soc-camera core has taken the most work on supporting DT on itself, so, DT 
support will come to them at only a fraction of the cost;-)

Thanks
Guennadi

> But I'm going to take a closer look and
> comment more on the details at the corresponding patches.
> 
> --
> 
> Regards,
> Sylwester

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Hans Verkuil]

On Tue October 2 2012 12:13:20 Sylwester Nawrocki wrote:
> Hi Guennadi,
> 
> On 10/02/2012 11:49 AM, Guennadi Liakhovetski wrote:
> >>> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> >>> +					ARRAY_SIZE(data_lanes))) {
> >>> +		int i;
> >>> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> >>> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> >>
> >> It doesn't look like what we aimed for. The data-lanes array is supposed
> >> to be of variable length, thus I don't think it can be parsed like that. 
> >> Or am I missing something ? I think we need more something like below 
> >> (based on of_property_read_u32_array(), not tested):
> > 
> > Ok, you're right, that my version only was suitable for fixed-size arrays, 
> > which wasn't our goal. But I don't think we should go down to manually 
> > parsing property data. How about (tested;-))
> > 
> > 	data = of_find_property(node, "data-lanes", NULL);
> > 	if (data) {
> > 		int i = 0;
> > 		const __be32 *lane = NULL;
> > 		do {
> > 			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
> > 		} while (lane && i++ < ARRAY_SIZE(data_lanes));
> > 		link->mipi_csi_2.num_data_lanes = i;
> > 		while (i--)
> > 			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > 	}
> 
> Yes, that looks neat and does what it is supposed to do. :) Thanks!
> For now, I'll trust you it works ;)
> 
> With regards to the remaining patches, it looks a bit scary to me how
> complicated it got, perhaps mostly because of requirement to reference
> host devices from subdevs. And it seems to rely on the existing SoC
> camera infrastructure, which might imply lot's of work need to be done
> for non soc-camera drivers. But I'm going to take a closer look and
> comment more on the details at the corresponding patches.

I have to say that I agree with Sylwester here. It seems awfully complicated,
but I can't really put my finger on the actual cause. It would be really
interesting to see this implemented for a non-SoC device and to compare the
two.

One area that I do not yet completely understand is the i2c bus notifications
(or asynchronous loading or i2c modules).

I would have expected that using OF the i2c devices are still initialized
before the host/bridge driver is initialized. But I gather that's not the
case?

If this deferred probing is a general problem, then I think we need a general
solution as well that's part of the v4l2 core.

Regards,

	Hans

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Fri, 5 Oct 2012, Hans Verkuil wrote:

> On Tue October 2 2012 12:13:20 Sylwester Nawrocki wrote:
> > Hi Guennadi,
> > 
> > On 10/02/2012 11:49 AM, Guennadi Liakhovetski wrote:
> > >>> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> > >>> +					ARRAY_SIZE(data_lanes))) {
> > >>> +		int i;
> > >>> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> > >>> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > >>
> > >> It doesn't look like what we aimed for. The data-lanes array is supposed
> > >> to be of variable length, thus I don't think it can be parsed like that. 
> > >> Or am I missing something ? I think we need more something like below 
> > >> (based on of_property_read_u32_array(), not tested):
> > > 
> > > Ok, you're right, that my version only was suitable for fixed-size arrays, 
> > > which wasn't our goal. But I don't think we should go down to manually 
> > > parsing property data. How about (tested;-))
> > > 
> > > 	data = of_find_property(node, "data-lanes", NULL);
> > > 	if (data) {
> > > 		int i = 0;
> > > 		const __be32 *lane = NULL;
> > > 		do {
> > > 			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
> > > 		} while (lane && i++ < ARRAY_SIZE(data_lanes));
> > > 		link->mipi_csi_2.num_data_lanes = i;
> > > 		while (i--)
> > > 			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > > 	}
> > 
> > Yes, that looks neat and does what it is supposed to do. :) Thanks!
> > For now, I'll trust you it works ;)
> > 
> > With regards to the remaining patches, it looks a bit scary to me how
> > complicated it got, perhaps mostly because of requirement to reference
> > host devices from subdevs. And it seems to rely on the existing SoC
> > camera infrastructure, which might imply lot's of work need to be done
> > for non soc-camera drivers. But I'm going to take a closer look and
> > comment more on the details at the corresponding patches.
> 
> I have to say that I agree with Sylwester here. It seems awfully complicated,
> but I can't really put my finger on the actual cause.

Well, which exactly part? The V4L2 OF part is quite simple.

> It would be really
> interesting to see this implemented for a non-SoC device and to compare the
> two.

Sure, volunteers? ;-) In principle, if I find time, I could try to convert 
sh_vou, which is also interesting, because it's an output driver.

> One area that I do not yet completely understand is the i2c bus notifications
> (or asynchronous loading or i2c modules).
> 
> I would have expected that using OF the i2c devices are still initialized
> before the host/bridge driver is initialized. But I gather that's not the
> case?

No, it's not. I'm not sure, whether it depends on the order of devices in 
the .dts, but, I think, it's better to not have to mandate a certain order 
and I also seem to have seen devices being registered in different order 
with the same DT, but I'm not 100% sure about that.

> If this deferred probing is a general problem, then I think we need a general
> solution as well that's part of the v4l2 core.

That can be done, perhaps. But we can do it as a next step. As soon as 
we're happy with the OF implementation as such, we can commit that, 
possibly leaving soc-camera patches out for now, then we can think where 
to put async I2C handling.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Hans Verkuil]

On Fri October 5 2012 12:58:21 Guennadi Liakhovetski wrote:
> On Fri, 5 Oct 2012, Hans Verkuil wrote:
> 
> > On Tue October 2 2012 12:13:20 Sylwester Nawrocki wrote:
> > > Hi Guennadi,
> > > 
> > > On 10/02/2012 11:49 AM, Guennadi Liakhovetski wrote:
> > > >>> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> > > >>> +					ARRAY_SIZE(data_lanes))) {
> > > >>> +		int i;
> > > >>> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> > > >>> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > > >>
> > > >> It doesn't look like what we aimed for. The data-lanes array is supposed
> > > >> to be of variable length, thus I don't think it can be parsed like that. 
> > > >> Or am I missing something ? I think we need more something like below 
> > > >> (based on of_property_read_u32_array(), not tested):
> > > > 
> > > > Ok, you're right, that my version only was suitable for fixed-size arrays, 
> > > > which wasn't our goal. But I don't think we should go down to manually 
> > > > parsing property data. How about (tested;-))
> > > > 
> > > > 	data = of_find_property(node, "data-lanes", NULL);
> > > > 	if (data) {
> > > > 		int i = 0;
> > > > 		const __be32 *lane = NULL;
> > > > 		do {
> > > > 			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
> > > > 		} while (lane && i++ < ARRAY_SIZE(data_lanes));
> > > > 		link->mipi_csi_2.num_data_lanes = i;
> > > > 		while (i--)
> > > > 			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > > > 	}
> > > 
> > > Yes, that looks neat and does what it is supposed to do. :) Thanks!
> > > For now, I'll trust you it works ;)
> > > 
> > > With regards to the remaining patches, it looks a bit scary to me how
> > > complicated it got, perhaps mostly because of requirement to reference
> > > host devices from subdevs. And it seems to rely on the existing SoC
> > > camera infrastructure, which might imply lot's of work need to be done
> > > for non soc-camera drivers. But I'm going to take a closer look and
> > > comment more on the details at the corresponding patches.
> > 
> > I have to say that I agree with Sylwester here. It seems awfully complicated,
> > but I can't really put my finger on the actual cause.
> 
> Well, which exactly part? The V4L2 OF part is quite simple.

No, the soc_camera part. The V4L2 OF part looks OK. Sorry, I should have
mentioned that!

> > It would be really
> > interesting to see this implemented for a non-SoC device and to compare the
> > two.
> 
> Sure, volunteers? ;-) In principle, if I find time, I could try to convert 
> sh_vou, which is also interesting, because it's an output driver.
> 
> > One area that I do not yet completely understand is the i2c bus notifications
> > (or asynchronous loading or i2c modules).
> > 
> > I would have expected that using OF the i2c devices are still initialized
> > before the host/bridge driver is initialized. But I gather that's not the
> > case?
> 
> No, it's not. I'm not sure, whether it depends on the order of devices in 
> the .dts, but, I think, it's better to not have to mandate a certain order 
> and I also seem to have seen devices being registered in different order 
> with the same DT, but I'm not 100% sure about that.
> 
> > If this deferred probing is a general problem, then I think we need a general
> > solution as well that's part of the v4l2 core.
> 
> That can be done, perhaps. But we can do it as a next step. As soon as 
> we're happy with the OF implementation as such, we can commit that, 
> possibly leaving soc-camera patches out for now, then we can think where 
> to put async I2C handling.

It would be good to have a number of 'Reviewed-by's or 'Acked-by's for the
DT binding documentation at least before it is merged.

I think the soc_camera patches should be left out for now. I suspect that
by adding core support for async i2c handling first, the soc_camera patches
will become a lot easier to understand.

Regards,

	Hans

> 
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Fri, 5 Oct 2012, Hans Verkuil wrote:

> On Fri October 5 2012 12:58:21 Guennadi Liakhovetski wrote:
> > On Fri, 5 Oct 2012, Hans Verkuil wrote:

[snip]

> > > One area that I do not yet completely understand is the i2c bus notifications
> > > (or asynchronous loading or i2c modules).
> > > 
> > > I would have expected that using OF the i2c devices are still initialized
> > > before the host/bridge driver is initialized. But I gather that's not the
> > > case?
> > 
> > No, it's not. I'm not sure, whether it depends on the order of devices in 
> > the .dts, but, I think, it's better to not have to mandate a certain order 
> > and I also seem to have seen devices being registered in different order 
> > with the same DT, but I'm not 100% sure about that.
> > 
> > > If this deferred probing is a general problem, then I think we need a general
> > > solution as well that's part of the v4l2 core.
> > 
> > That can be done, perhaps. But we can do it as a next step. As soon as 
> > we're happy with the OF implementation as such, we can commit that, 
> > possibly leaving soc-camera patches out for now, then we can think where 
> > to put async I2C handling.
> 
> It would be good to have a number of 'Reviewed-by's or 'Acked-by's for the
> DT binding documentation at least before it is merged.

Definitely, I'm sure you'll be honoured to be the first one in the list;-)

> I think the soc_camera patches should be left out for now. I suspect that
> by adding core support for async i2c handling first, the soc_camera patches
> will become a lot easier to understand.

Ok, we can do this.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

Hi Hans

On Fri, 5 Oct 2012, Hans Verkuil wrote:

[snip]

> I think the soc_camera patches should be left out for now. I suspect that
> by adding core support for async i2c handling first,

Ok, let's think, what this meacs - async I2C in media / V4L2 core.

The main reason for our probing order problem is the master clock, 
typically supplied from the camera bridge to I2C subdevices, which we only 
want to start when necessary, i.e. when accessing the subdevice. And the 
subdevice driver needs that clock running during its .probe() to be able 
to access and verify or configure the hardware. Our current solution is to 
not register I2C subdevices from the platform data, as is usual for all 
I2C devices, but from the bridge driver and only after it has switched on 
the master clock. After the subdevice driver has completed its probing we 
switch the clock back off until the subdevice has to be activated, e.g. 
for video capture.

Also important - when we want to unregister the bridge driver we just also 
unregister the I2C device.

Now, to reverse the whole thing and to allow I2C devices be registered as 
usual - via platform data or OF, first of all we have to teach I2C 
subdevice drivers to recognise the "too early" situation and request 
deferred probing in such a case. Then it will be reprobed after each new 
successful probe or unregister on the system. After the bridge driver has 
successfully probed the subdevice driver will be re-probed and at that 
time it should succeed. Now, there is a problem here too: who should 
switch on and off the master clock?

If we do it from the bridge driver, we could install an I2C bus-notifier, 
_before_ the subdevice driver is probed, i.e. upon the 
BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
event to switch the clock back off. BUT - if the subdevice fails probing? 
How do we find out about that and turn the clock back off? There is no 
notification event for that... Possible solutions:

1. timer - ugly and unreliable.
2. add a "probing failed" notifier event to the device core - would this 
   be accepted?
3. let the subdevice turn the master clock on and off for the duration of 
   probing.

My vote goes for (3). Ideally this should be done using the generic clock 
framework. But can we really expect all drivers and platforms to switch to 
it quickly enough? If not, we need a V4L2-specific callback from subdevice 
drivers to bridge drivers to turn the clock on and off. That's what I've 
done "temporarily" in this patch series for soc-camera.

Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
we can think what else we need to add to V4L2 to support asynchronous 
subdevice driver probing.

1. We'll have to create these V4L2 clock start and stop functions, that, 
supplied (in case of I2C) with client address and adapter number will find 
the correct v4l2_device instance and call its callbacks.

2. The I2C notifier. I'm not sure, whether this one should be common. Of 
common tasks we have to refcount the I2C adapter and register the 
subdevice. Then we'd have to call the bridge driver's callback. Is it 
worth it doing this centrally or rather allow individual drivers to do 
that themselves?

Also, ideally OF-compatible (I2C) drivers should run with no platform 
data, but soc-camera is using I2C device platform data intensively. To 
avoid modifying the soc-camera core and all drivers, I also trigger on the 
BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically 
created platform data to the I2C device. Would we also want to do this for 
all V4L2 bridge drivers? We could call this a "prepare" callback or 
something similar...

3. Bridge driver unregistering. Here we have to put the subdevice driver 
back into the deferred-probe state... Ugliness alert: I'm doing this by 
unregistering and re-registering the I2C device... For that I also have to 
create a copy of devices I2C board-info data. Lovely, ain't it? This I'd 
be happy to move to the V4L2 core;-)

Thanks
Guennadi

> the soc_camera patches
> will become a lot easier to understand.
> 
> Regards,
> 
> 	Hans
> 
> > 
> > Thanks
> > Guennadi
> > ---
> > Guennadi Liakhovetski, Ph.D.
> > Freelance Open-Source Software Developer
> > http://www.open-technology.de/
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Hans Verkuil]

On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> Hi Hans
> 
> On Fri, 5 Oct 2012, Hans Verkuil wrote:
> 
> [snip]
> 
> > I think the soc_camera patches should be left out for now. I suspect that
> > by adding core support for async i2c handling first,
> 
> Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> 
> The main reason for our probing order problem is the master clock, 
> typically supplied from the camera bridge to I2C subdevices, which we only 
> want to start when necessary, i.e. when accessing the subdevice. And the 
> subdevice driver needs that clock running during its .probe() to be able 
> to access and verify or configure the hardware. Our current solution is to 
> not register I2C subdevices from the platform data, as is usual for all 
> I2C devices, but from the bridge driver and only after it has switched on 
> the master clock. After the subdevice driver has completed its probing we 
> switch the clock back off until the subdevice has to be activated, e.g. 
> for video capture.
> 
> Also important - when we want to unregister the bridge driver we just also 
> unregister the I2C device.
> 
> Now, to reverse the whole thing and to allow I2C devices be registered as 
> usual - via platform data or OF, first of all we have to teach I2C 
> subdevice drivers to recognise the "too early" situation and request 
> deferred probing in such a case. Then it will be reprobed after each new 
> successful probe or unregister on the system. After the bridge driver has 
> successfully probed the subdevice driver will be re-probed and at that 
> time it should succeed. Now, there is a problem here too: who should 
> switch on and off the master clock?
> 
> If we do it from the bridge driver, we could install an I2C bus-notifier, 
> _before_ the subdevice driver is probed, i.e. upon the 
> BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> event to switch the clock back off. BUT - if the subdevice fails probing? 
> How do we find out about that and turn the clock back off? There is no 
> notification event for that... Possible solutions:
> 
> 1. timer - ugly and unreliable.
> 2. add a "probing failed" notifier event to the device core - would this 
>    be accepted?
> 3. let the subdevice turn the master clock on and off for the duration of 
>    probing.
> 
> My vote goes for (3). Ideally this should be done using the generic clock 
> framework. But can we really expect all drivers and platforms to switch to 
> it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> drivers to bridge drivers to turn the clock on and off. That's what I've 
> done "temporarily" in this patch series for soc-camera.
> 
> Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
> we can think what else we need to add to V4L2 to support asynchronous 
> subdevice driver probing.

I wonder, don't we have the necessary code already? V4L2 subdev drivers can
have internal_ops with register/unregister ops. These are called by
v4l2_device_register_subdev. This happens during the bridge driver's probe.

Suppose the subdev's probe does not actually access the i2c device, but
instead defers that to the register callback? The bridge driver will turn on
the clock before calling v4l2_device_register_subdev to ensure that the
register callback can access the i2c registers. The register callback will
do any initialization and can return an error. In case of an error the i2c
client is automatically unregistered as well.

In addition, during the register op the subdev driver can call into the
bridge driver since it knows the v4l2_device struct.

This has also the advantage that subdev drivers can change to this model
gradually. Only drivers that need master clocks, etc. need to move any probe
code that actually accesses hardware to the register op. Others can remain
as. Nor should this change behavior of existing drivers as this happens
all in the V4L2 core.

The bridge driver may still have to wait until all i2c drivers are loaded,
though. But that can definitely be handled centrally (i.e.: 'I need these
drivers, wait until all are loaded').

> 1. We'll have to create these V4L2 clock start and stop functions, that, 
> supplied (in case of I2C) with client address and adapter number will find 
> the correct v4l2_device instance and call its callbacks.
> 
> 2. The I2C notifier. I'm not sure, whether this one should be common. Of 
> common tasks we have to refcount the I2C adapter and register the 
> subdevice. Then we'd have to call the bridge driver's callback. Is it 
> worth it doing this centrally or rather allow individual drivers to do 
> that themselves?
> 
> Also, ideally OF-compatible (I2C) drivers should run with no platform 
> data, but soc-camera is using I2C device platform data intensively. To 
> avoid modifying the soc-camera core and all drivers, I also trigger on the 
> BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically 
> created platform data to the I2C device. Would we also want to do this for 
> all V4L2 bridge drivers? We could call this a "prepare" callback or 
> something similar...

Well, subdev drivers should either parse the OF data, or use the platform_data.
The way soc_camera uses platform_data is one reason why it is so hard to
reuse subdevs for non-soc_camera drivers. All the callbacks in soc_camera_link
should be replaced by calls to the v4l2_device notify() callback. After that we
can see what is needed to drop struct soc_camera_link altogether as platform_data.

> 3. Bridge driver unregistering. Here we have to put the subdevice driver 
> back into the deferred-probe state... Ugliness alert: I'm doing this by 
> unregistering and re-registering the I2C device... For that I also have to 
> create a copy of devices I2C board-info data. Lovely, ain't it? This I'd 
> be happy to move to the V4L2 core;-)

By just using the unregister ops this should be greatly simplified as well.

Unless I am missing something, which is perfectly possible :-)

Regards,

	Hans

> 
> Thanks
> Guennadi
> 
> > the soc_camera patches
> > will become a lot easier to understand.
> > 
> > Regards,
> > 
> > 	Hans
> > 
> > > 
> > > Thanks
> > > Guennadi
> > > ---
> > > Guennadi Liakhovetski, Ph.D.
> > > Freelance Open-Source Software Developer
> > > http://www.open-technology.de/
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > 
> > 
> 
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> 

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Mon, 8 Oct 2012, Hans Verkuil wrote:

> On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > Hi Hans
> > 
> > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > 
> > [snip]
> > 
> > > I think the soc_camera patches should be left out for now. I suspect that
> > > by adding core support for async i2c handling first,
> > 
> > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > 
> > The main reason for our probing order problem is the master clock, 
> > typically supplied from the camera bridge to I2C subdevices, which we only 
> > want to start when necessary, i.e. when accessing the subdevice. And the 
> > subdevice driver needs that clock running during its .probe() to be able 
> > to access and verify or configure the hardware. Our current solution is to 
> > not register I2C subdevices from the platform data, as is usual for all 
> > I2C devices, but from the bridge driver and only after it has switched on 
> > the master clock. After the subdevice driver has completed its probing we 
> > switch the clock back off until the subdevice has to be activated, e.g. 
> > for video capture.
> > 
> > Also important - when we want to unregister the bridge driver we just also 
> > unregister the I2C device.
> > 
> > Now, to reverse the whole thing and to allow I2C devices be registered as 
> > usual - via platform data or OF, first of all we have to teach I2C 
> > subdevice drivers to recognise the "too early" situation and request 
> > deferred probing in such a case. Then it will be reprobed after each new 
> > successful probe or unregister on the system. After the bridge driver has 
> > successfully probed the subdevice driver will be re-probed and at that 
> > time it should succeed. Now, there is a problem here too: who should 
> > switch on and off the master clock?
> > 
> > If we do it from the bridge driver, we could install an I2C bus-notifier, 
> > _before_ the subdevice driver is probed, i.e. upon the 
> > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> > probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> > event to switch the clock back off. BUT - if the subdevice fails probing? 
> > How do we find out about that and turn the clock back off? There is no 
> > notification event for that... Possible solutions:
> > 
> > 1. timer - ugly and unreliable.
> > 2. add a "probing failed" notifier event to the device core - would this 
> >    be accepted?
> > 3. let the subdevice turn the master clock on and off for the duration of 
> >    probing.
> > 
> > My vote goes for (3). Ideally this should be done using the generic clock 
> > framework. But can we really expect all drivers and platforms to switch to 
> > it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> > drivers to bridge drivers to turn the clock on and off. That's what I've 
> > done "temporarily" in this patch series for soc-camera.
> > 
> > Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
> > we can think what else we need to add to V4L2 to support asynchronous 
> > subdevice driver probing.
> 
> I wonder, don't we have the necessary code already? V4L2 subdev drivers can
> have internal_ops with register/unregister ops. These are called by
> v4l2_device_register_subdev. This happens during the bridge driver's probe.
> 
> Suppose the subdev's probe does not actually access the i2c device, but
> instead defers that to the register callback? The bridge driver will turn on
> the clock before calling v4l2_device_register_subdev to ensure that the
> register callback can access the i2c registers. The register callback will
> do any initialization and can return an error. In case of an error the i2c
> client is automatically unregistered as well.

Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed 
several times before and always what I didn't like in this is, that I2C 
device probe() in this case succeeds without even trying to access the 
hardware. And think about DT. In this case we don't instantiate the I2C 
device, OF code does it for us. What do you do then? If you let probe() 
succeed, then you will have to somehow remember the subdevice to later 
match it against bridges...

> In addition, during the register op the subdev driver can call into the
> bridge driver since it knows the v4l2_device struct.
> 
> This has also the advantage that subdev drivers can change to this model
> gradually. Only drivers that need master clocks, etc. need to move any probe
> code that actually accesses hardware to the register op. Others can remain
> as. Nor should this change behavior of existing drivers as this happens
> all in the V4L2 core.
> 
> The bridge driver may still have to wait until all i2c drivers are loaded,
> though. But that can definitely be handled centrally (i.e.: 'I need these
> drivers, wait until all are loaded').
> 
> > 1. We'll have to create these V4L2 clock start and stop functions, that, 
> > supplied (in case of I2C) with client address and adapter number will find 
> > the correct v4l2_device instance and call its callbacks.
> > 
> > 2. The I2C notifier. I'm not sure, whether this one should be common. Of 
> > common tasks we have to refcount the I2C adapter and register the 
> > subdevice. Then we'd have to call the bridge driver's callback. Is it 
> > worth it doing this centrally or rather allow individual drivers to do 
> > that themselves?
> > 
> > Also, ideally OF-compatible (I2C) drivers should run with no platform 
> > data, but soc-camera is using I2C device platform data intensively. To 
> > avoid modifying the soc-camera core and all drivers, I also trigger on the 
> > BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically 
> > created platform data to the I2C device. Would we also want to do this for 
> > all V4L2 bridge drivers? We could call this a "prepare" callback or 
> > something similar...
> 
> Well, subdev drivers should either parse the OF data, or use the platform_data.
> The way soc_camera uses platform_data is one reason why it is so hard to
> reuse subdevs for non-soc_camera drivers. All the callbacks in soc_camera_link
> should be replaced by calls to the v4l2_device notify() callback. After that we
> can see what is needed to drop struct soc_camera_link altogether as platform_data.

They don't have to be, they are not (or should not be) called by 
subdevices.

> > 3. Bridge driver unregistering. Here we have to put the subdevice driver 
> > back into the deferred-probe state... Ugliness alert: I'm doing this by 
> > unregistering and re-registering the I2C device... For that I also have to 
> > create a copy of devices I2C board-info data. Lovely, ain't it? This I'd 
> > be happy to move to the V4L2 core;-)
> 
> By just using the unregister ops this should be greatly simplified as well.

Sorry, which unregister ops do you mean? internal_ops->unregistered()? 
Yes, but only if we somehow go your way and use dummy probe() methods...

Thanks
Guennadi

> Unless I am missing something, which is perfectly possible :-)
> 
> Regards,
> 
> 	Hans
> 
> > Thanks
> > Guennadi
> > 
> > > the soc_camera patches
> > > will become a lot easier to understand.

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Hans Verkuil]

On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> On Mon, 8 Oct 2012, Hans Verkuil wrote:
> 
> > On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > > Hi Hans
> > > 
> > > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > > 
> > > [snip]
> > > 
> > > > I think the soc_camera patches should be left out for now. I suspect that
> > > > by adding core support for async i2c handling first,
> > > 
> > > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > > 
> > > The main reason for our probing order problem is the master clock, 
> > > typically supplied from the camera bridge to I2C subdevices, which we only 
> > > want to start when necessary, i.e. when accessing the subdevice. And the 
> > > subdevice driver needs that clock running during its .probe() to be able 
> > > to access and verify or configure the hardware. Our current solution is to 
> > > not register I2C subdevices from the platform data, as is usual for all 
> > > I2C devices, but from the bridge driver and only after it has switched on 
> > > the master clock. After the subdevice driver has completed its probing we 
> > > switch the clock back off until the subdevice has to be activated, e.g. 
> > > for video capture.
> > > 
> > > Also important - when we want to unregister the bridge driver we just also 
> > > unregister the I2C device.
> > > 
> > > Now, to reverse the whole thing and to allow I2C devices be registered as 
> > > usual - via platform data or OF, first of all we have to teach I2C 
> > > subdevice drivers to recognise the "too early" situation and request 
> > > deferred probing in such a case. Then it will be reprobed after each new 
> > > successful probe or unregister on the system. After the bridge driver has 
> > > successfully probed the subdevice driver will be re-probed and at that 
> > > time it should succeed. Now, there is a problem here too: who should 
> > > switch on and off the master clock?
> > > 
> > > If we do it from the bridge driver, we could install an I2C bus-notifier, 
> > > _before_ the subdevice driver is probed, i.e. upon the 
> > > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> > > probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> > > event to switch the clock back off. BUT - if the subdevice fails probing? 
> > > How do we find out about that and turn the clock back off? There is no 
> > > notification event for that... Possible solutions:
> > > 
> > > 1. timer - ugly and unreliable.
> > > 2. add a "probing failed" notifier event to the device core - would this 
> > >    be accepted?
> > > 3. let the subdevice turn the master clock on and off for the duration of 
> > >    probing.
> > > 
> > > My vote goes for (3). Ideally this should be done using the generic clock 
> > > framework. But can we really expect all drivers and platforms to switch to 
> > > it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> > > drivers to bridge drivers to turn the clock on and off. That's what I've 
> > > done "temporarily" in this patch series for soc-camera.
> > > 
> > > Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
> > > we can think what else we need to add to V4L2 to support asynchronous 
> > > subdevice driver probing.
> > 
> > I wonder, don't we have the necessary code already? V4L2 subdev drivers can
> > have internal_ops with register/unregister ops. These are called by
> > v4l2_device_register_subdev. This happens during the bridge driver's probe.
> > 
> > Suppose the subdev's probe does not actually access the i2c device, but
> > instead defers that to the register callback? The bridge driver will turn on
> > the clock before calling v4l2_device_register_subdev to ensure that the
> > register callback can access the i2c registers. The register callback will
> > do any initialization and can return an error. In case of an error the i2c
> > client is automatically unregistered as well.
> 
> Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed 
> several times before and always what I didn't like in this is, that I2C 
> device probe() in this case succeeds without even trying to access the 
> hardware. And think about DT. In this case we don't instantiate the I2C 
> device, OF code does it for us. What do you do then? If you let probe() 
> succeed, then you will have to somehow remember the subdevice to later 
> match it against bridges...

Yes, but you need that information anyway. The bridge still needs to call
v4l2_device_register_subdev so it needs to know which subdevs are loaded.
And can't it get that from DT as well?

In my view you cannot do a proper initialization unless you have both the
bridge driver and all subdev drivers loaded and instantiated. They need one
another. So I am perfectly fine with letting the probe function do next to
nothing and postponing that until register() is called. I2C and actual probing
to check if it's the right device is a bad idea in general since you have no
idea what a hardware access to an unknown i2c device will do. There are still
some corner cases where that is needed, but I do not think that that is an
issue here.

It would simplify things a lot IMHO. Also note that the register() op will
work with any device, not just i2c. That may be a useful property as well.

> > In addition, during the register op the subdev driver can call into the
> > bridge driver since it knows the v4l2_device struct.
> > 
> > This has also the advantage that subdev drivers can change to this model
> > gradually. Only drivers that need master clocks, etc. need to move any probe
> > code that actually accesses hardware to the register op. Others can remain
> > as. Nor should this change behavior of existing drivers as this happens
> > all in the V4L2 core.
> > 
> > The bridge driver may still have to wait until all i2c drivers are loaded,
> > though. But that can definitely be handled centrally (i.e.: 'I need these
> > drivers, wait until all are loaded').
> > 
> > > 1. We'll have to create these V4L2 clock start and stop functions, that, 
> > > supplied (in case of I2C) with client address and adapter number will find 
> > > the correct v4l2_device instance and call its callbacks.
> > > 
> > > 2. The I2C notifier. I'm not sure, whether this one should be common. Of 
> > > common tasks we have to refcount the I2C adapter and register the 
> > > subdevice. Then we'd have to call the bridge driver's callback. Is it 
> > > worth it doing this centrally or rather allow individual drivers to do 
> > > that themselves?
> > > 
> > > Also, ideally OF-compatible (I2C) drivers should run with no platform 
> > > data, but soc-camera is using I2C device platform data intensively. To 
> > > avoid modifying the soc-camera core and all drivers, I also trigger on the 
> > > BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically 
> > > created platform data to the I2C device. Would we also want to do this for 
> > > all V4L2 bridge drivers? We could call this a "prepare" callback or 
> > > something similar...
> > 
> > Well, subdev drivers should either parse the OF data, or use the platform_data.
> > The way soc_camera uses platform_data is one reason why it is so hard to
> > reuse subdevs for non-soc_camera drivers. All the callbacks in soc_camera_link
> > should be replaced by calls to the v4l2_device notify() callback. After that we
> > can see what is needed to drop struct soc_camera_link altogether as platform_data.
> 
> They don't have to be, they are not (or should not be) called by 
> subdevices.

Then why are those callbacks in a struct that subdevs can access? I always
have a hard time with soc_camera figuring out who is using what when :-(

> > > 3. Bridge driver unregistering. Here we have to put the subdevice driver 
> > > back into the deferred-probe state... Ugliness alert: I'm doing this by 
> > > unregistering and re-registering the I2C device... For that I also have to 
> > > create a copy of devices I2C board-info data. Lovely, ain't it? This I'd 
> > > be happy to move to the V4L2 core;-)
> > 
> > By just using the unregister ops this should be greatly simplified as well.
> 
> Sorry, which unregister ops do you mean? internal_ops->unregistered()? 

Yes.

> Yes, but only if we somehow go your way and use dummy probe() methods...

Of course.

Regards,

	Hans

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Mon, 8 Oct 2012, Hans Verkuil wrote:

> On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > 
> > > On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > > > Hi Hans
> > > > 
> > > > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > > > 
> > > > [snip]
> > > > 
> > > > > I think the soc_camera patches should be left out for now. I suspect that
> > > > > by adding core support for async i2c handling first,
> > > > 
> > > > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > > > 
> > > > The main reason for our probing order problem is the master clock, 
> > > > typically supplied from the camera bridge to I2C subdevices, which we only 
> > > > want to start when necessary, i.e. when accessing the subdevice. And the 
> > > > subdevice driver needs that clock running during its .probe() to be able 
> > > > to access and verify or configure the hardware. Our current solution is to 
> > > > not register I2C subdevices from the platform data, as is usual for all 
> > > > I2C devices, but from the bridge driver and only after it has switched on 
> > > > the master clock. After the subdevice driver has completed its probing we 
> > > > switch the clock back off until the subdevice has to be activated, e.g. 
> > > > for video capture.
> > > > 
> > > > Also important - when we want to unregister the bridge driver we just also 
> > > > unregister the I2C device.
> > > > 
> > > > Now, to reverse the whole thing and to allow I2C devices be registered as 
> > > > usual - via platform data or OF, first of all we have to teach I2C 
> > > > subdevice drivers to recognise the "too early" situation and request 
> > > > deferred probing in such a case. Then it will be reprobed after each new 
> > > > successful probe or unregister on the system. After the bridge driver has 
> > > > successfully probed the subdevice driver will be re-probed and at that 
> > > > time it should succeed. Now, there is a problem here too: who should 
> > > > switch on and off the master clock?
> > > > 
> > > > If we do it from the bridge driver, we could install an I2C bus-notifier, 
> > > > _before_ the subdevice driver is probed, i.e. upon the 
> > > > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> > > > probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> > > > event to switch the clock back off. BUT - if the subdevice fails probing? 
> > > > How do we find out about that and turn the clock back off? There is no 
> > > > notification event for that... Possible solutions:
> > > > 
> > > > 1. timer - ugly and unreliable.
> > > > 2. add a "probing failed" notifier event to the device core - would this 
> > > >    be accepted?
> > > > 3. let the subdevice turn the master clock on and off for the duration of 
> > > >    probing.
> > > > 
> > > > My vote goes for (3). Ideally this should be done using the generic clock 
> > > > framework. But can we really expect all drivers and platforms to switch to 
> > > > it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> > > > drivers to bridge drivers to turn the clock on and off. That's what I've 
> > > > done "temporarily" in this patch series for soc-camera.
> > > > 
> > > > Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
> > > > we can think what else we need to add to V4L2 to support asynchronous 
> > > > subdevice driver probing.
> > > 
> > > I wonder, don't we have the necessary code already? V4L2 subdev drivers can
> > > have internal_ops with register/unregister ops. These are called by
> > > v4l2_device_register_subdev. This happens during the bridge driver's probe.
> > > 
> > > Suppose the subdev's probe does not actually access the i2c device, but
> > > instead defers that to the register callback? The bridge driver will turn on
> > > the clock before calling v4l2_device_register_subdev to ensure that the
> > > register callback can access the i2c registers. The register callback will
> > > do any initialization and can return an error. In case of an error the i2c
> > > client is automatically unregistered as well.
> > 
> > Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed 
> > several times before and always what I didn't like in this is, that I2C 
> > device probe() in this case succeeds without even trying to access the 
> > hardware. And think about DT. In this case we don't instantiate the I2C 
> > device, OF code does it for us. What do you do then? If you let probe() 
> > succeed, then you will have to somehow remember the subdevice to later 
> > match it against bridges...
> 
> Yes, but you need that information anyway. The bridge still needs to call
> v4l2_device_register_subdev so it needs to know which subdevs are loaded.

But how do you get the subdev pointer? With the notifier I get it from 
i2c_get_clientdata(client) and what do you do without it? How do you get 
to the client?

> And can't it get that from DT as well?

No, I don't think there is a way to get a device pointer from a DT node.

> In my view you cannot do a proper initialization unless you have both the
> bridge driver and all subdev drivers loaded and instantiated. They need one
> another. So I am perfectly fine with letting the probe function do next to
> nothing and postponing that until register() is called. I2C and actual probing
> to check if it's the right device is a bad idea in general since you have no
> idea what a hardware access to an unknown i2c device will do. There are still
> some corner cases where that is needed, but I do not think that that is an
> issue here.
> 
> It would simplify things a lot IMHO. Also note that the register() op will
> work with any device, not just i2c. That may be a useful property as well.

And what if the subdevice device is not yet instantiated by OF by the time 
your bridge driver probes?

Thanks
Guennadi

> > > In addition, during the register op the subdev driver can call into the
> > > bridge driver since it knows the v4l2_device struct.
> > > 
> > > This has also the advantage that subdev drivers can change to this model
> > > gradually. Only drivers that need master clocks, etc. need to move any probe
> > > code that actually accesses hardware to the register op. Others can remain
> > > as. Nor should this change behavior of existing drivers as this happens
> > > all in the V4L2 core.
> > > 
> > > The bridge driver may still have to wait until all i2c drivers are loaded,
> > > though. But that can definitely be handled centrally (i.e.: 'I need these
> > > drivers, wait until all are loaded').
> > > 
> > > > 1. We'll have to create these V4L2 clock start and stop functions, that, 
> > > > supplied (in case of I2C) with client address and adapter number will find 
> > > > the correct v4l2_device instance and call its callbacks.
> > > > 
> > > > 2. The I2C notifier. I'm not sure, whether this one should be common. Of 
> > > > common tasks we have to refcount the I2C adapter and register the 
> > > > subdevice. Then we'd have to call the bridge driver's callback. Is it 
> > > > worth it doing this centrally or rather allow individual drivers to do 
> > > > that themselves?
> > > > 
> > > > Also, ideally OF-compatible (I2C) drivers should run with no platform 
> > > > data, but soc-camera is using I2C device platform data intensively. To 
> > > > avoid modifying the soc-camera core and all drivers, I also trigger on the 
> > > > BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically 
> > > > created platform data to the I2C device. Would we also want to do this for 
> > > > all V4L2 bridge drivers? We could call this a "prepare" callback or 
> > > > something similar...
> > > 
> > > Well, subdev drivers should either parse the OF data, or use the platform_data.
> > > The way soc_camera uses platform_data is one reason why it is so hard to
> > > reuse subdevs for non-soc_camera drivers. All the callbacks in soc_camera_link
> > > should be replaced by calls to the v4l2_device notify() callback. After that we
> > > can see what is needed to drop struct soc_camera_link altogether as platform_data.
> > 
> > They don't have to be, they are not (or should not be) called by 
> > subdevices.
> 
> Then why are those callbacks in a struct that subdevs can access? I always
> have a hard time with soc_camera figuring out who is using what when :-(
> 
> > > > 3. Bridge driver unregistering. Here we have to put the subdevice driver 
> > > > back into the deferred-probe state... Ugliness alert: I'm doing this by 
> > > > unregistering and re-registering the I2C device... For that I also have to 
> > > > create a copy of devices I2C board-info data. Lovely, ain't it? This I'd 
> > > > be happy to move to the V4L2 core;-)
> > > 
> > > By just using the unregister ops this should be greatly simplified as well.
> > 
> > Sorry, which unregister ops do you mean? internal_ops->unregistered()? 
> 
> Yes.
> 
> > Yes, but only if we somehow go your way and use dummy probe() methods...
> 
> Of course.
> 
> Regards,
> 
> 	Hans
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Hans Verkuil]

On Mon October 8 2012 17:15:53 Guennadi Liakhovetski wrote:
> On Mon, 8 Oct 2012, Hans Verkuil wrote:
> 
> > On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> > > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > > 
> > > > On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > > > > Hi Hans
> > > > > 
> > > > > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > > > > 
> > > > > [snip]
> > > > > 
> > > > > > I think the soc_camera patches should be left out for now. I suspect that
> > > > > > by adding core support for async i2c handling first,
> > > > > 
> > > > > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > > > > 
> > > > > The main reason for our probing order problem is the master clock, 
> > > > > typically supplied from the camera bridge to I2C subdevices, which we only 
> > > > > want to start when necessary, i.e. when accessing the subdevice. And the 
> > > > > subdevice driver needs that clock running during its .probe() to be able 
> > > > > to access and verify or configure the hardware. Our current solution is to 
> > > > > not register I2C subdevices from the platform data, as is usual for all 
> > > > > I2C devices, but from the bridge driver and only after it has switched on 
> > > > > the master clock. After the subdevice driver has completed its probing we 
> > > > > switch the clock back off until the subdevice has to be activated, e.g. 
> > > > > for video capture.
> > > > > 
> > > > > Also important - when we want to unregister the bridge driver we just also 
> > > > > unregister the I2C device.
> > > > > 
> > > > > Now, to reverse the whole thing and to allow I2C devices be registered as 
> > > > > usual - via platform data or OF, first of all we have to teach I2C 
> > > > > subdevice drivers to recognise the "too early" situation and request 
> > > > > deferred probing in such a case. Then it will be reprobed after each new 
> > > > > successful probe or unregister on the system. After the bridge driver has 
> > > > > successfully probed the subdevice driver will be re-probed and at that 
> > > > > time it should succeed. Now, there is a problem here too: who should 
> > > > > switch on and off the master clock?
> > > > > 
> > > > > If we do it from the bridge driver, we could install an I2C bus-notifier, 
> > > > > _before_ the subdevice driver is probed, i.e. upon the 
> > > > > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> > > > > probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> > > > > event to switch the clock back off. BUT - if the subdevice fails probing? 
> > > > > How do we find out about that and turn the clock back off? There is no 
> > > > > notification event for that... Possible solutions:
> > > > > 
> > > > > 1. timer - ugly and unreliable.
> > > > > 2. add a "probing failed" notifier event to the device core - would this 
> > > > >    be accepted?
> > > > > 3. let the subdevice turn the master clock on and off for the duration of 
> > > > >    probing.
> > > > > 
> > > > > My vote goes for (3). Ideally this should be done using the generic clock 
> > > > > framework. But can we really expect all drivers and platforms to switch to 
> > > > > it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> > > > > drivers to bridge drivers to turn the clock on and off. That's what I've 
> > > > > done "temporarily" in this patch series for soc-camera.
> > > > > 
> > > > > Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
> > > > > we can think what else we need to add to V4L2 to support asynchronous 
> > > > > subdevice driver probing.
> > > > 
> > > > I wonder, don't we have the necessary code already? V4L2 subdev drivers can
> > > > have internal_ops with register/unregister ops. These are called by
> > > > v4l2_device_register_subdev. This happens during the bridge driver's probe.
> > > > 
> > > > Suppose the subdev's probe does not actually access the i2c device, but
> > > > instead defers that to the register callback? The bridge driver will turn on
> > > > the clock before calling v4l2_device_register_subdev to ensure that the
> > > > register callback can access the i2c registers. The register callback will
> > > > do any initialization and can return an error. In case of an error the i2c
> > > > client is automatically unregistered as well.
> > > 
> > > Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed 
> > > several times before and always what I didn't like in this is, that I2C 
> > > device probe() in this case succeeds without even trying to access the 
> > > hardware. And think about DT. In this case we don't instantiate the I2C 
> > > device, OF code does it for us. What do you do then? If you let probe() 
> > > succeed, then you will have to somehow remember the subdevice to later 
> > > match it against bridges...
> > 
> > Yes, but you need that information anyway. The bridge still needs to call
> > v4l2_device_register_subdev so it needs to know which subdevs are loaded.
> 
> But how do you get the subdev pointer? With the notifier I get it from 
> i2c_get_clientdata(client) and what do you do without it? How do you get 
> to the client?
> 
> > And can't it get that from DT as well?
> 
> No, I don't think there is a way to get a device pointer from a DT node.

Not a device pointer, but the i2c bus and i2c address. With that information
you can get the i2c_client, and with that you can get the subdev pointer.

If there is no way to get that information from the proposed V4L2 DT, then
it needs to be modified since a bridge driver really needs to know which
subdevs it has to register with the v4l2_device struct. That information is
also board specific so it should be part of the DT.

> 
> > In my view you cannot do a proper initialization unless you have both the
> > bridge driver and all subdev drivers loaded and instantiated. They need one
> > another. So I am perfectly fine with letting the probe function do next to
> > nothing and postponing that until register() is called. I2C and actual probing
> > to check if it's the right device is a bad idea in general since you have no
> > idea what a hardware access to an unknown i2c device will do. There are still
> > some corner cases where that is needed, but I do not think that that is an
> > issue here.
> > 
> > It would simplify things a lot IMHO. Also note that the register() op will
> > work with any device, not just i2c. That may be a useful property as well.
> 
> And what if the subdevice device is not yet instantiated by OF by the time 
> your bridge driver probes?

That is where you still need to have a bus notifier mechanism. You have to be
able to wait until all dependent drivers are loaded/instantiated, or
alternatively you have to be able to load them explicitly. But this should
be relatively easy to implement in a generic manner.

I still think this sucks (excuse my language), but I see no way around it as
long as there is no explicit probe order one can rely on.

Regards,

	Hans

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Mon, 8 Oct 2012, Hans Verkuil wrote:

> On Mon October 8 2012 17:15:53 Guennadi Liakhovetski wrote:
> > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > 
> > > On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> > > > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > > > 
> > > > > On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > > > > > Hi Hans
> > > > > > 
> > > > > > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > > > > > 
> > > > > > [snip]
> > > > > > 
> > > > > > > I think the soc_camera patches should be left out for now. I suspect that
> > > > > > > by adding core support for async i2c handling first,
> > > > > > 
> > > > > > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > > > > > 
> > > > > > The main reason for our probing order problem is the master clock, 
> > > > > > typically supplied from the camera bridge to I2C subdevices, which we only 
> > > > > > want to start when necessary, i.e. when accessing the subdevice. And the 
> > > > > > subdevice driver needs that clock running during its .probe() to be able 
> > > > > > to access and verify or configure the hardware. Our current solution is to 
> > > > > > not register I2C subdevices from the platform data, as is usual for all 
> > > > > > I2C devices, but from the bridge driver and only after it has switched on 
> > > > > > the master clock. After the subdevice driver has completed its probing we 
> > > > > > switch the clock back off until the subdevice has to be activated, e.g. 
> > > > > > for video capture.
> > > > > > 
> > > > > > Also important - when we want to unregister the bridge driver we just also 
> > > > > > unregister the I2C device.
> > > > > > 
> > > > > > Now, to reverse the whole thing and to allow I2C devices be registered as 
> > > > > > usual - via platform data or OF, first of all we have to teach I2C 
> > > > > > subdevice drivers to recognise the "too early" situation and request 
> > > > > > deferred probing in such a case. Then it will be reprobed after each new 
> > > > > > successful probe or unregister on the system. After the bridge driver has 
> > > > > > successfully probed the subdevice driver will be re-probed and at that 
> > > > > > time it should succeed. Now, there is a problem here too: who should 
> > > > > > switch on and off the master clock?
> > > > > > 
> > > > > > If we do it from the bridge driver, we could install an I2C bus-notifier, 
> > > > > > _before_ the subdevice driver is probed, i.e. upon the 
> > > > > > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> > > > > > probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> > > > > > event to switch the clock back off. BUT - if the subdevice fails probing? 
> > > > > > How do we find out about that and turn the clock back off? There is no 
> > > > > > notification event for that... Possible solutions:
> > > > > > 
> > > > > > 1. timer - ugly and unreliable.
> > > > > > 2. add a "probing failed" notifier event to the device core - would this 
> > > > > >    be accepted?
> > > > > > 3. let the subdevice turn the master clock on and off for the duration of 
> > > > > >    probing.
> > > > > > 
> > > > > > My vote goes for (3). Ideally this should be done using the generic clock 
> > > > > > framework. But can we really expect all drivers and platforms to switch to 
> > > > > > it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> > > > > > drivers to bridge drivers to turn the clock on and off. That's what I've 
> > > > > > done "temporarily" in this patch series for soc-camera.
> > > > > > 
> > > > > > Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 
> > > > > > we can think what else we need to add to V4L2 to support asynchronous 
> > > > > > subdevice driver probing.
> > > > > 
> > > > > I wonder, don't we have the necessary code already? V4L2 subdev drivers can
> > > > > have internal_ops with register/unregister ops. These are called by
> > > > > v4l2_device_register_subdev. This happens during the bridge driver's probe.
> > > > > 
> > > > > Suppose the subdev's probe does not actually access the i2c device, but
> > > > > instead defers that to the register callback? The bridge driver will turn on
> > > > > the clock before calling v4l2_device_register_subdev to ensure that the
> > > > > register callback can access the i2c registers. The register callback will
> > > > > do any initialization and can return an error. In case of an error the i2c
> > > > > client is automatically unregistered as well.
> > > > 
> > > > Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed 
> > > > several times before and always what I didn't like in this is, that I2C 
> > > > device probe() in this case succeeds without even trying to access the 
> > > > hardware. And think about DT. In this case we don't instantiate the I2C 
> > > > device, OF code does it for us. What do you do then? If you let probe() 
> > > > succeed, then you will have to somehow remember the subdevice to later 
> > > > match it against bridges...
> > > 
> > > Yes, but you need that information anyway. The bridge still needs to call
> > > v4l2_device_register_subdev so it needs to know which subdevs are loaded.
> > 
> > But how do you get the subdev pointer? With the notifier I get it from 
> > i2c_get_clientdata(client) and what do you do without it? How do you get 
> > to the client?
> > 
> > > And can't it get that from DT as well?
> > 
> > No, I don't think there is a way to get a device pointer from a DT node.
> 
> Not a device pointer, but the i2c bus and i2c address. With that information
> you can get the i2c_client, and with that you can get the subdev pointer.

How? How can you get a client pointer from adapter and i2c device address? 
I haven't found anything suitable in i2c.h.

> If there is no way to get that information from the proposed V4L2 DT, then
> it needs to be modified since a bridge driver really needs to know which
> subdevs it has to register with the v4l2_device struct. That information is
> also board specific so it should be part of the DT.
> 
> > 
> > > In my view you cannot do a proper initialization unless you have both the
> > > bridge driver and all subdev drivers loaded and instantiated. They need one
> > > another. So I am perfectly fine with letting the probe function do next to
> > > nothing and postponing that until register() is called. I2C and actual probing
> > > to check if it's the right device is a bad idea in general since you have no
> > > idea what a hardware access to an unknown i2c device will do. There are still
> > > some corner cases where that is needed, but I do not think that that is an
> > > issue here.
> > > 
> > > It would simplify things a lot IMHO. Also note that the register() op will
> > > work with any device, not just i2c. That may be a useful property as well.
> > 
> > And what if the subdevice device is not yet instantiated by OF by the time 
> > your bridge driver probes?
> 
> That is where you still need to have a bus notifier mechanism. You have to be
> able to wait until all dependent drivers are loaded/instantiated,

No, drivers are not the problem, as you say, you can load them. Devices 
are the problem. You don't know when they will be registered.

> or
> alternatively you have to be able to load them explicitly. But this should
> be relatively easy to implement in a generic manner.

So, if you need notifiers anyway, why not use them in all cases instead of 
mixing multiple methods?

Thanks
Guennadi

> I still think this sucks (excuse my language), but I see no way around it as
> long as there is no explicit probe order one can rely on.
> 
> Regards,
> 
> 	Hans
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Mon, 8 Oct 2012, Guennadi Liakhovetski wrote:

> On Mon, 8 Oct 2012, Hans Verkuil wrote:
> 
> > On Mon October 8 2012 17:15:53 Guennadi Liakhovetski wrote:

[snip]

> > > No, I don't think there is a way to get a device pointer from a DT node.
> > 
> > Not a device pointer, but the i2c bus and i2c address. With that information
> > you can get the i2c_client, and with that you can get the subdev pointer.
> 
> How? How can you get a client pointer from adapter and i2c device address? 
> I haven't found anything suitable in i2c.h.

Ok, you could use of_find_i2c_device_by_node(), but the second argument 
remains - if you need notifiers in one case, I don't think it makes sense 
to implement multiple methods.

Thanks
Guennadi

> > If there is no way to get that information from the proposed V4L2 DT, then
> > it needs to be modified since a bridge driver really needs to know which
> > subdevs it has to register with the v4l2_device struct. That information is
> > also board specific so it should be part of the DT.
> > 
> > > 
> > > > In my view you cannot do a proper initialization unless you have both the
> > > > bridge driver and all subdev drivers loaded and instantiated. They need one
> > > > another. So I am perfectly fine with letting the probe function do next to
> > > > nothing and postponing that until register() is called. I2C and actual probing
> > > > to check if it's the right device is a bad idea in general since you have no
> > > > idea what a hardware access to an unknown i2c device will do. There are still
> > > > some corner cases where that is needed, but I do not think that that is an
> > > > issue here.
> > > > 
> > > > It would simplify things a lot IMHO. Also note that the register() op will
> > > > work with any device, not just i2c. That may be a useful property as well.
> > > 
> > > And what if the subdevice device is not yet instantiated by OF by the time 
> > > your bridge driver probes?
> > 
> > That is where you still need to have a bus notifier mechanism. You have to be
> > able to wait until all dependent drivers are loaded/instantiated,
> 
> No, drivers are not the problem, as you say, you can load them. Devices 
> are the problem. You don't know when they will be registered.
> 
> > or
> > alternatively you have to be able to load them explicitly. But this should
> > be relatively easy to implement in a generic manner.
> 
> So, if you need notifiers anyway, why not use them in all cases instead of 
> mixing multiple methods?
> 
> Thanks
> Guennadi
> 
> > I still think this sucks (excuse my language), but I see no way around it as
> > long as there is no explicit probe order one can rely on.
> > 
> > Regards,
> > 
> > 	Hans
> > 
> 
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Hans,

On Monday 08 October 2012 17:41:43 Hans Verkuil wrote:
> On Mon October 8 2012 17:15:53 Guennadi Liakhovetski wrote:
> > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > > On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> > > > On Mon, 8 Oct 2012, Hans Verkuil wrote:

[snip]

> > > > > I wonder, don't we have the necessary code already? V4L2 subdev
> > > > > drivers can have internal_ops with register/unregister ops. These
> > > > > are called by v4l2_device_register_subdev. This happens during the
> > > > > bridge driver's probe.
> > > > > 
> > > > > Suppose the subdev's probe does not actually access the i2c device,
> > > > > but instead defers that to the register callback? The bridge driver
> > > > > will turn on the clock before calling v4l2_device_register_subdev to
> > > > > ensure that the register callback can access the i2c registers. The
> > > > > register callback will do any initialization and can return an
> > > > > error. In case of an error the i2c client is automatically
> > > > > unregistered as well.
> > > > 
> > > > Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed
> > > > several times before and always what I didn't like in this is, that
> > > > I2C device probe() in this case succeeds without even trying to access
> > > > the hardware. And think about DT. In this case we don't instantiate
> > > > the I2C device, OF code does it for us. What do you do then? If you
> > > > let probe() succeed, then you will have to somehow remember the
> > > > subdevice to later match it against bridges...
> > > 
> > > Yes, but you need that information anyway. The bridge still needs to
> > > call v4l2_device_register_subdev so it needs to know which subdevs are
> > > loaded.
> > 
> > But how do you get the subdev pointer? With the notifier I get it from
> > i2c_get_clientdata(client) and what do you do without it? How do you get
> > to the client?
> > 
> > > And can't it get that from DT as well?
> > 
> > No, I don't think there is a way to get a device pointer from a DT node.
> 
> Not a device pointer, but the i2c bus and i2c address. With that information
> you can get the i2c_client, and with that you can get the subdev pointer.

That could work as well, but it might be easier to keep a mapping from the DT 
node to struct device or struct v4l2_subdev instead. I have no strong opinion 
on this at the moment.

> If there is no way to get that information from the proposed V4L2 DT, then
> it needs to be modified since a bridge driver really needs to know which
> subdevs it has to register with the v4l2_device struct. That information is
> also board specific so it should be part of the DT.
> 
> > > In my view you cannot do a proper initialization unless you have both
> > > the bridge driver and all subdev drivers loaded and instantiated. They
> > > need one another. So I am perfectly fine with letting the probe function
> > > do next to nothing and postponing that until register() is called. I2C
> > > and actual probing to check if it's the right device is a bad idea in
> > > general since you have no idea what a hardware access to an unknown i2c
> > > device will do. There are still some corner cases where that is needed,
> > > but I do not think that that is an issue here.
> > > 
> > > It would simplify things a lot IMHO. Also note that the register() op
> > > will work with any device, not just i2c. That may be a useful property
> > > as well.
> > 
> > And what if the subdevice device is not yet instantiated by OF by the time
> > your bridge driver probes?
> 
> That is where you still need to have a bus notifier mechanism. You have to
> be able to wait until all dependent drivers are loaded/instantiated, or
> alternatively you have to be able to load them explicitly. But this should
> be relatively easy to implement in a generic manner.
> 
> I still think this sucks (excuse my language), but I see no way around it as
> long as there is no explicit probe order one can rely on.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Guennadi,

On Monday 08 October 2012 17:15:53 Guennadi Liakhovetski wrote:
> On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> > > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > > > On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > > > > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > > > > 
> > > > > [snip]
> > > > > 
> > > > > > I think the soc_camera patches should be left out for now. I
> > > > > > suspect that by adding core support for async i2c handling first,
> > > > > 
> > > > > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > > > > 
> > > > > The main reason for our probing order problem is the master clock,
> > > > > typically supplied from the camera bridge to I2C subdevices, which
> > > > > we only want to start when necessary, i.e. when accessing the
> > > > > subdevice. And the subdevice driver needs that clock running during
> > > > > its .probe() to be able to access and verify or configure the
> > > > > hardware. Our current solution is to not register I2C subdevices
> > > > > from the platform data, as is usual for all I2C devices, but from
> > > > > the bridge driver and only after it has switched on the master
> > > > > clock. After the subdevice driver has completed its probing we
> > > > > switch the clock back off until the subdevice has to be activated,
> > > > > e.g. for video capture.
> > > > > 
> > > > > Also important - when we want to unregister the bridge driver we
> > > > > just also unregister the I2C device.
> > > > > 
> > > > > Now, to reverse the whole thing and to allow I2C devices be
> > > > > registered as usual - via platform data or OF, first of all we have
> > > > > to teach I2C subdevice drivers to recognise the "too early"
> > > > > situation and request deferred probing in such a case. Then it will
> > > > > be reprobed after each new successful probe or unregister on the
> > > > > system. After the bridge driver has successfully probed the
> > > > > subdevice driver will be re-probed and at that time it should
> > > > > succeed. Now, there is a problem here too: who should switch on and
> > > > > off the master clock?
> > > > > 
> > > > > If we do it from the bridge driver, we could install an I2C
> > > > > bus-notifier, _before_ the subdevice driver is probed, i.e. upon the
> > > > > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If
> > > > > subdevice probing was successful, we can then wait for the
> > > > > BUS_NOTIFY_BOUND_DRIVER event to switch the clock back off. BUT - if
> > > > > the subdevice fails probing?
> > > > > How do we find out about that and turn the clock back off? There is
> > > > > no notification event for that... Possible solutions:
> > > > > 
> > > > > 1. timer - ugly and unreliable.
> > > > > 2. add a "probing failed" notifier event to the device core - would
> > > > > this be accepted?
> > > > > 3. let the subdevice turn the master clock on and off for the
> > > > > duration of probing.
> > > > > 
> > > > > My vote goes for (3). Ideally this should be done using the generic
> > > > > clock framework. But can we really expect all drivers and platforms
> > > > > to switch to it quickly enough? If not, we need a V4L2-specific
> > > > > callback from subdevice drivers to bridge drivers to turn the clock
> > > > > on and off. That's what I've done "temporarily" in this patch series
> > > > > for soc-camera.
> > > > > 
> > > > > Suppose we decide to do the same for V4L2 centrally - add
> > > > > call-backs. Then we can think what else we need to add to V4L2 to
> > > > > support asynchronous subdevice driver probing.
> > > > 
> > > > I wonder, don't we have the necessary code already? V4L2 subdev
> > > > drivers can have internal_ops with register/unregister ops. These are
> > > > called by v4l2_device_register_subdev. This happens during the bridge
> > > > driver's probe.
> > > > 
> > > > Suppose the subdev's probe does not actually access the i2c device,
> > > > but instead defers that to the register callback? The bridge driver
> > > > will turn on the clock before calling v4l2_device_register_subdev to
> > > > ensure that the register callback can access the i2c registers. The
> > > > register callback will do any initialization and can return an error.
> > > > In case of an error the i2c client is automatically unregistered as
> > > > well.
> > > 
> > > Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed
> > > several times before and always what I didn't like in this is, that I2C
> > > device probe() in this case succeeds without even trying to access the
> > > hardware. And think about DT. In this case we don't instantiate the I2C
> > > device, OF code does it for us. What do you do then? If you let probe()
> > > succeed, then you will have to somehow remember the subdevice to later
> > > match it against bridges...
> > 
> > Yes, but you need that information anyway. The bridge still needs to call
> > v4l2_device_register_subdev so it needs to know which subdevs are loaded.
> 
> But how do you get the subdev pointer? With the notifier I get it from
> i2c_get_clientdata(client) and what do you do without it? How do you get
> to the client?
> 
> > And can't it get that from DT as well?
> 
> No, I don't think there is a way to get a device pointer from a DT node.

But we'll need a way. The bridge driver will get sensor DT nodes from the V4L2 
DT bindings, and will need to get the corresponding subdev. This could be 
limited to V4L2 though, we could keep a map of DT nodes to subdevs without 
requiring a generic solution in the device base code (although I'm wondering 
if there's a specific reason not to have a device pointer in the DT node 
structure).

> > In my view you cannot do a proper initialization unless you have both the
> > bridge driver and all subdev drivers loaded and instantiated. They need
> > one another. So I am perfectly fine with letting the probe function do
> > next to nothing and postponing that until register() is called. I2C and
> > actualprobing to check if it's the right device is a bad idea in general
> > since you have no idea what a hardware access to an unknown i2c device
> > will do. There are still some corner cases where that is needed, but I do
> > not think that that is an issue here.
> > 
> > It would simplify things a lot IMHO. Also note that the register() op will
> > work with any device, not just i2c. That may be a useful property as well.
> 
> And what if the subdevice device is not yet instantiated by OF by the time
> your bridge driver probes?

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Stephen Warren]

On 10/10/2012 07:18 AM, Laurent Pinchart wrote:
> On Monday 08 October 2012 17:15:53 Guennadi Liakhovetski wrote:
...
>> But how do you get the subdev pointer? With the notifier I get it from
>> i2c_get_clientdata(client) and what do you do without it? How do you get
>> to the client?
>>
>>> And can't it get that from DT as well?
>>
>> No, I don't think there is a way to get a device pointer from a DT node.

I don't believe there's a generic API for this (although perhaps there
could be), but it can be implemented quite easily.

For example, on Tegra, the SMMU needs to flip a bit in the AHB register
space in order to enable itself. The SMMU DT node contains a phandle
that points at the AHB DT node. The SMMU driver parses the phandle and
passes the DT node pointer to the AHB driver. The AHB driver looks up
the struct device that was instantiated for that node. See
drivers/amba/tegra-ahb.c:tegra_ahb_enable_smmu(). There are a few other
cases of similar code in the kernel, although I can't remember the others!

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

On Wednesday 10 October 2012 10:50:20 Stephen Warren wrote:
> On 10/10/2012 07:18 AM, Laurent Pinchart wrote:
> > On Monday 08 October 2012 17:15:53 Guennadi Liakhovetski wrote:
> ...
> 
> >> But how do you get the subdev pointer? With the notifier I get it from
> >> i2c_get_clientdata(client) and what do you do without it? How do you get
> >> to the client?
> >> 
> >>> And can't it get that from DT as well?
> >> 
> >> No, I don't think there is a way to get a device pointer from a DT node.
> 
> I don't believe there's a generic API for this (although perhaps there
> could be), but it can be implemented quite easily.
>
> For example, on Tegra, the SMMU needs to flip a bit in the AHB register
> space in order to enable itself. The SMMU DT node contains a phandle
> that points at the AHB DT node. The SMMU driver parses the phandle and
> passes the DT node pointer to the AHB driver. The AHB driver looks up
> the struct device that was instantiated for that node. See
> drivers/amba/tegra-ahb.c:tegra_ahb_enable_smmu(). There are a few other
> cases of similar code in the kernel, although I can't remember the others!

That's a very naive approach, but what about storing the struct device in 
struct device_node when the device is instantiated ? It's so simple that 
there's probably a good reason why that hasn't been implemented though.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Stephen Warren]

On 10/10/2012 04:51 PM, Laurent Pinchart wrote:
> On Wednesday 10 October 2012 10:50:20 Stephen Warren wrote:
>> On 10/10/2012 07:18 AM, Laurent Pinchart wrote:
>>> On Monday 08 October 2012 17:15:53 Guennadi Liakhovetski wrote:
>> ...
>>
>>>> But how do you get the subdev pointer? With the notifier I get it from
>>>> i2c_get_clientdata(client) and what do you do without it? How do you get
>>>> to the client?
>>>>
>>>>> And can't it get that from DT as well?
>>>>
>>>> No, I don't think there is a way to get a device pointer from a DT node.
>>
>> I don't believe there's a generic API for this (although perhaps there
>> could be), but it can be implemented quite easily.
>>
>> For example, on Tegra, the SMMU needs to flip a bit in the AHB register
>> space in order to enable itself. The SMMU DT node contains a phandle
>> that points at the AHB DT node. The SMMU driver parses the phandle and
>> passes the DT node pointer to the AHB driver. The AHB driver looks up
>> the struct device that was instantiated for that node. See
>> drivers/amba/tegra-ahb.c:tegra_ahb_enable_smmu(). There are a few other
>> cases of similar code in the kernel, although I can't remember the others!
> 
> That's a very naive approach, but what about storing the struct device in 
> struct device_node when the device is instantiated ? It's so simple that 
> there's probably a good reason why that hasn't been implemented though.

It sounds like that would work.

The advantage of calling a function in the driver for the node, rather
than just grabbing something from the node directly in code unrelated to
the driver for the node, is that it any knowledge of what kind of
device/driver is handling that node is embedded into a function in the
driver for the node, not the driver using the node, which makes
everything a bit more type-safe.

Now, perhaps the implementation of that function could just pull a field
out of struct of_node rather than calling driver_find_device(), but it'd
then have to validate that the struct device that was found was truly
managed by the expected driver, for safety. I assume that's pretty
simple, but haven't checked.

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Hans,

On Monday 08 October 2012 16:53:55 Hans Verkuil wrote:
> On Mon October 8 2012 16:30:53 Guennadi Liakhovetski wrote:
> > On Mon, 8 Oct 2012, Hans Verkuil wrote:
> > > On Mon October 8 2012 14:23:25 Guennadi Liakhovetski wrote:
> > > > On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > > > 
> > > > [snip]
> > > > 
> > > > > I think the soc_camera patches should be left out for now. I suspect
> > > > > that by adding core support for async i2c handling first,
> > > > 
> > > > Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> > > > 
> > > > The main reason for our probing order problem is the master clock,
> > > > typically supplied from the camera bridge to I2C subdevices, which we
> > > > only want to start when necessary, i.e. when accessing the subdevice.
> > > > And the subdevice driver needs that clock running during its .probe()
> > > > to be able to access and verify or configure the hardware. Our current
> > > > solution is to not register I2C subdevices from the platform data, as
> > > > is usual for all I2C devices, but from the bridge driver and only
> > > > after it has switched on the master clock. After the subdevice driver
> > > > has completed its probing we switch the clock back off until the
> > > > subdevice has to be activated, e.g. for video capture.
> > > > 
> > > > Also important - when we want to unregister the bridge driver we just
> > > > also> unregister the I2C device.
> > > > 
> > > > Now, to reverse the whole thing and to allow I2C devices be registered
> > > > as usual - via platform data or OF, first of all we have to teach I2C
> > > > subdevice drivers to recognise the "too early" situation and request
> > > > deferred probing in such a case. Then it will be reprobed after each
> > > > new successful probe or unregister on the system. After the bridge
> > > > driver has successfully probed the subdevice driver will be re-probed
> > > > and at that time it should succeed. Now, there is a problem here too:
> > > > who should switch on and off the master clock?
> > > > 
> > > > If we do it from the bridge driver, we could install an I2C bus-
> > > > notifier, _before_ the subdevice driver is probed, i.e. upon the
> > > > BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice
> > > > probing was successful, we can then wait for the
> > > > BUS_NOTIFY_BOUND_DRIVER event to switch the clock back off. BUT - if
> > > > the subdevice fails probing?
> > > > How do we find out about that and turn the clock back off? There is no
> > > > notification event for that... Possible solutions:
> > > > 
> > > > 1. timer - ugly and unreliable.
> > > > 2. add a "probing failed" notifier event to the device core - would
> > > > this be accepted?
> > > > 3. let the subdevice turn the master clock on and off for the duration
> > > > of probing.
> > > > 
> > > > My vote goes for (3). Ideally this should be done using the generic
> > > > clock framework. But can we really expect all drivers and platforms to
> > > > switch to it quickly enough? If not, we need a V4L2-specific callback
> > > > from subdevice drivers to bridge drivers to turn the clock on and off.
> > > > That's what I've done "temporarily" in this patch series for soc-
> > > > camera.
> > > > 
> > > > Suppose we decide to do the same for V4L2 centrally - add call-backs.
> > > > Then we can think what else we need to add to V4L2 to support
> > > > asynchronous subdevice driver probing.
> > > 
> > > I wonder, don't we have the necessary code already? V4L2 subdev drivers
> > > can have internal_ops with register/unregister ops. These are called by
> > > v4l2_device_register_subdev. This happens during the bridge driver's
> > > probe.
> > > 
> > > Suppose the subdev's probe does not actually access the i2c device, but
> > > instead defers that to the register callback? The bridge driver will
> > > turn on the clock before calling v4l2_device_register_subdev to ensure
> > > that the register callback can access the i2c registers. The register
> > > callback will do any initialization and can return an error. In case of
> > > an error the i2c client is automatically unregistered as well.
> > 
> > Yes, if v4l2_i2c_new_subdev_board() is used. This has been discussed
> > several times before and always what I didn't like in this is, that I2C
> > device probe() in this case succeeds without even trying to access the
> > hardware. And think about DT. In this case we don't instantiate the I2C
> > device, OF code does it for us. What do you do then? If you let probe()
> > succeed, then you will have to somehow remember the subdevice to later
> > match it against bridges...
> 
> Yes, but you need that information anyway. The bridge still needs to call
> v4l2_device_register_subdev so it needs to know which subdevs are loaded.
> And can't it get that from DT as well?

That information will definitely come from the DT, but the bridge won't 
instantiate the I2C devices. They will be instantiated asynchronously by the 
I2C bus master driver when parsing the DT. The bridge will then need to be 
notified or I2C devices registration and finish its initialization when all 
needed I2C (or SPI, ...) devices will be available. That should in my opinion 
be handled by the V4L2 core : the bridge would pass a list of devices 
(possibly DT nodes) to a V4L2 core function along with a callback, and the 
callback would be called when all required devices are available.

I've also thought about adding a synchronous function that would wait until 
all required devices are available, but I'm not sure whether that's a good 
idea.

> In my view you cannot do a proper initialization unless you have both the
> bridge driver and all subdev drivers loaded and instantiated.

You can do a proper initialization of the bridge device. The OMAP3 ISP could 
already be used for mem-to-mem operation for instance.

> They need one another. So I am perfectly fine with letting the probe
> function do next to nothing and postponing that until register() is called.

I don't really like that solution, it's an abuse of the probe() function. It 
would be much better to defer probing of the I2C device until all the needed 
resources (such as clocks) are available. That would also mean that the I2C 
device won't be able to access the bridge directly during probing, which could 
actually be seen as a feature : with proper abstractions in place (such as 
generic clock objects) accessing the bridge at probe() time should not be 
needed.

> I2C and actual probing to check if it's the right device is a bad idea in
> general since you have no idea what a hardware access to an unknown i2c
> device will do. There are still some corner cases where that is needed, but
> I do not think that that is an issue here.
> 
> It would simplify things a lot IMHO. Also note that the register() op will
> work with any device, not just i2c. That may be a useful property as well.
> 
> > > In addition, during the register op the subdev driver can call into the
> > > bridge driver since it knows the v4l2_device struct.
> > > 
> > > This has also the advantage that subdev drivers can change to this model
> > > gradually. Only drivers that need master clocks, etc. need to move any
> > > probe code that actually accesses hardware to the register op. Others
> > > can remain as. Nor should this change behavior of existing drivers as
> > > this happens all in the V4L2 core.
> > > 
> > > The bridge driver may still have to wait until all i2c drivers are
> > > loaded, though. But that can definitely be handled centrally (i.e.: 'I
> > > need these drivers, wait until all are loaded').
> > > 
> > > > 1. We'll have to create these V4L2 clock start and stop functions,
> > > > that, supplied (in case of I2C) with client address and adapter number
> > > > will find the correct v4l2_device instance and call its callbacks.
> > > > 
> > > > 2. The I2C notifier. I'm not sure, whether this one should be common.
> > > > Of common tasks we have to refcount the I2C adapter and register the
> > > > subdevice. Then we'd have to call the bridge driver's callback. Is it
> > > > worth it doing this centrally or rather allow individual drivers to do
> > > > that themselves?
> > > > 
> > > > Also, ideally OF-compatible (I2C) drivers should run with no platform
> > > > data, but soc-camera is using I2C device platform data intensively. To
> > > > avoid modifying the soc-camera core and all drivers, I also trigger on
> > > > the BUS_NOTIFY_BIND_DRIVER event and assign a reference to the
> > > > dynamically created platform data to the I2C device. Would we also
> > > > want to do this for all V4L2 bridge drivers? We could call this a
> > > > "prepare" callback or something similar...
> > > 
> > > Well, subdev drivers should either parse the OF data, or use the
> > > platform_data. The way soc_camera uses platform_data is one reason why
> > > it is so hard to reuse subdevs for non-soc_camera drivers. All the
> > > callbacks in soc_camera_link should be replaced by calls to the
> > > v4l2_device notify() callback. After that we can see what is needed to
> > > drop struct soc_camera_link altogether as platform_data.
> > 
> > They don't have to be, they are not (or should not be) called by
> > subdevices.
> 
> Then why are those callbacks in a struct that subdevs can access? I always
> have a hard time with soc_camera figuring out who is using what when :-(
> 
> > > > 3. Bridge driver unregistering. Here we have to put the subdevice
> > > > driver back into the deferred-probe state... Ugliness alert: I'm doing
> > > > this by unregistering and re-registering the I2C device... For that I
> > > > also have to create a copy of devices I2C board-info data. Lovely,
> > > > isn't it? This I'd be happy to move to the V4L2 core;-)
> > > 
> > > By just using the unregister ops this should be greatly simplified as
> > > well.
> > 
> > Sorry, which unregister ops do you mean? internal_ops->unregistered()?
> 
> Yes.
> 
> > Yes, but only if we somehow go your way and use dummy probe() methods...
> 
> Of course.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Guennadi,

On Monday 08 October 2012 14:23:25 Guennadi Liakhovetski wrote:
> On Fri, 5 Oct 2012, Hans Verkuil wrote:
> 
> [snip]
> 
> > I think the soc_camera patches should be left out for now. I suspect that
> > by adding core support for async i2c handling first,
> 
> Ok, let's think, what this meacs - async I2C in media / V4L2 core.
> 
> The main reason for our probing order problem is the master clock,
> typically supplied from the camera bridge to I2C subdevices, which we only
> want to start when necessary, i.e. when accessing the subdevice. And the
> subdevice driver needs that clock running during its .probe() to be able
> to access and verify or configure the hardware. Our current solution is to
> not register I2C subdevices from the platform data, as is usual for all
> I2C devices, but from the bridge driver and only after it has switched on
> the master clock. After the subdevice driver has completed its probing we
> switch the clock back off until the subdevice has to be activated, e.g.
> for video capture.
> 
> Also important - when we want to unregister the bridge driver we just also
> unregister the I2C device.
> 
> Now, to reverse the whole thing and to allow I2C devices be registered as
> usual - via platform data or OF, first of all we have to teach I2C
> subdevice drivers to recognise the "too early" situation and request
> deferred probing in such a case. Then it will be reprobed after each new
> successful probe or unregister on the system. After the bridge driver has
> successfully probed the subdevice driver will be re-probed and at that
> time it should succeed. Now, there is a problem here too: who should
> switch on and off the master clock?
> 
> If we do it from the bridge driver, we could install an I2C bus-notifier,
> _before_ the subdevice driver is probed, i.e. upon the
> BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice
> probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER
> event to switch the clock back off. BUT - if the subdevice fails probing?
> How do we find out about that and turn the clock back off? There is no
> notification event for that... Possible solutions:
> 
> 1. timer - ugly and unreliable.
> 2. add a "probing failed" notifier event to the device core - would this
>    be accepted?
> 3. let the subdevice turn the master clock on and off for the duration of
>    probing.
> 
> My vote goes for (3). Ideally this should be done using the generic clock
> framework. But can we really expect all drivers and platforms to switch to
> it quickly enough? If not, we need a V4L2-specific callback from subdevice
> drivers to bridge drivers to turn the clock on and off. That's what I've
> done "temporarily" in this patch series for soc-camera.

I vote for (3) as well, using the generic clock framework. You're right that 
we need an interim solution, as not all platforms will switch quickly enough. 
I'm fine with that interim solution being a bit hackish, as long as we push 
new drivers towards the right solution.

> Suppose we decide to do the same for V4L2 centrally - add call-backs. Then
> we can think what else we need to add to V4L2 to support asynchronous
> subdevice driver probing.
> 
> 1. We'll have to create these V4L2 clock start and stop functions, that,
> supplied (in case of I2C) with client address and adapter number will find
> the correct v4l2_device instance and call its callbacks.
> 
> 2. The I2C notifier. I'm not sure, whether this one should be common. Of
> common tasks we have to refcount the I2C adapter and register the
> subdevice. Then we'd have to call the bridge driver's callback. Is it
> worth it doing this centrally or rather allow individual drivers to do
> that themselves?

What about going through board code for platforms that don't support the 
generic clock framework yet ? That's what the OMAP3 ISP driver does. DT 
support would then require the generic clock framework.

> Also, ideally OF-compatible (I2C) drivers should run with no platform
> data, but soc-camera is using I2C device platform data intensively. To
> avoid modifying the soc-camera core and all drivers, I also trigger on the
> BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically
> created platform data to the I2C device. Would we also want to do this for
> all V4L2 bridge drivers? We could call this a "prepare" callback or
> something similar...

If that's going to be an interim solution only I'm fine with keeping it in 
soc-camera.

> 3. Bridge driver unregistering. Here we have to put the subdevice driver
> back into the deferred-probe state... Ugliness alert: I'm doing this by
> unregistering and re-registering the I2C device... For that I also have to
> create a copy of devices I2C board-info data. Lovely, ain't it? This I'd
> be happy to move to the V4L2 core;-)

That's the ugly part. As soon as the I2C device starts using resources 
provided by the bridge, those resources can't disappear behind the scene. 
Reference counting must ensure that the bridge driver doesn't get removed. And 
that's where it gets bad: the bridge uses resources provided by the I2C 
driver, through the subdev operations. This creates circular dependencies that 
we need to break somehow. I currently have no solution for that problem.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Mauro Carvalho Chehab]

Em Wed, 10 Oct 2012 14:54 +0200
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> > Also, ideally OF-compatible (I2C) drivers should run with no platform
> > data, but soc-camera is using I2C device platform data intensively. To
> > avoid modifying the soc-camera core and all drivers, I also trigger on the
> > BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically
> > created platform data to the I2C device. Would we also want to do this for
> > all V4L2 bridge drivers? We could call this a "prepare" callback or
> > something similar...
> 
> If that's going to be an interim solution only I'm fine with keeping it in 
> soc-camera.

I'm far from being an OF expert, but why do we need to export I2C devices to
DT/OF? On my understanding, it is the bridge code that should be responsible
for detecting, binding and initializing the proper I2C devices. On several
cases, it is impossible or it would cause lots of ugly hacks if we ever try
to move away from platform data stuff, as only the bridge driver knows what
initialization is needed for an specific I2C driver.

To make things more complex, it is expected that most I2C drivers to be arch
independent, and they should be allowed to be used by a personal computer
or by an embedded device. 

Let me give 2 such examples:

	- ir-i2c-kbd driver supports lots of IR devices. Platform_data is needed
to specify what hardware will actually be used, and what should be the default
Remote Controller keymap;

	- Sensor drivers like ov2940 is needed by soc_camera and by other
webcam drivers like em28xx. The setup for em28xx should be completely different
than the one for soc_camera: the initial registers init sequence is different
for both. As several registers there aren't properly documented, there's no
easy way to parametrize the configuration.

So, for me, we should not expose the I2C devices directly on OF; it should,
instead, see just the bridge, and let the bridge to map the needed I2C devices
using the needed platform_data.

-- 
Regards,
Mauro

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Mauro,

On Wednesday 10 October 2012 10:45:22 Mauro Carvalho Chehab wrote:
> Em Wed, 10 Oct 2012 14:54 +0200 Laurent Pinchart escreveu:
> > > Also, ideally OF-compatible (I2C) drivers should run with no platform
> > > data, but soc-camera is using I2C device platform data intensively. To
> > > avoid modifying the soc-camera core and all drivers, I also trigger on
> > > the
> > > BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically
> > > created platform data to the I2C device. Would we also want to do this
> > > for
> > > all V4L2 bridge drivers? We could call this a "prepare" callback or
> > > something similar...
> > 
> > If that's going to be an interim solution only I'm fine with keeping it in
> > soc-camera.
> 
> I'm far from being an OF expert, but why do we need to export I2C devices to
> DT/OF? On my understanding, it is the bridge code that should be
> responsible for detecting, binding and initializing the proper I2C devices.
> On several cases, it is impossible or it would cause lots of ugly hacks if
> we ever try to move away from platform data stuff, as only the bridge
> driver knows what initialization is needed for an specific I2C driver.

In a nutshell, a DT-based platform doesn't have any board code (except in rare 
cases, but let's not get into that), and thus doesn't pass any platform data 
structure to drivers. However, drivers receive a DT node, which contains a 
hierarchical description of the hardware, and parse those to extract 
information necessary to configure the device.

One very important point to keep in mind here is that the DT is not Linux-
specific. DT bindings are designed to be portable, and thus must only contain 
hardware descriptions, without any OS-specific information or policy 
information. For instance information such as the maximum video buffers size 
is not allowed in the DT.

The DT is used both to provide platform data to drivers and to instantiate 
devices. I2C device DT nodes are stored as children of the I2C bus master DT 
node, and are automatically instantiated by the I2C bus master. This is a 
significant change compared to our current situation where the V4L2 bridge 
driver receives an array of I2C board information structures and instatiates 
the devices itself. Most of the DT support work will go in supporting that new 
instantiation mechanism. The bridge driver doesn't control instantiation of 
the I2C devices anymore, but need to bind with them at runtime.

> To make things more complex, it is expected that most I2C drivers to be arch
> independent, and they should be allowed to be used by a personal computer
> or by an embedded device.

Agreed. That's why platform data structures won't go away anytime soon, a PCI 
bridge driver for hardware that contain I2C devices will still instantiate the 
I2C devices itself. We don't plan to or even want to get rid of that 
mechanism, as there are perfectly valid use cases. However, for DT-based 
embedded platforms, we need to support a new instantiation mechanism.

> Let me give 2 such examples:
> 
> 	- ir-i2c-kbd driver supports lots of IR devices. Platform_data is needed
> to specify what hardware will actually be used, and what should be the
> default Remote Controller keymap;

That driver isn't used on embedded platforms so it doesn't need to be changed 
now.

> 	- Sensor drivers like ov2940 is needed by soc_camera and by other
> webcam drivers like em28xx. The setup for em28xx should be completely
> different than the one for soc_camera: the initial registers init sequence
> is different for both. As several registers there aren't properly
> documented, there's no easy way to parametrize the configuration.

Such drivers will need to support both DT-based platform data and structure-
based platform data. They will likely parse the DT node and fill a platform 
data structure, to avoid duplicating initialization code.

Please note that the new subdevs instantiation mechanism needed for DT will 
need to handle any instantiation order, as we can't guarantee the I2C device 
and bridge device instantiation order with DT. It should thus then support the 
current instantiation order we use for PCI and USB platforms.

> So, for me, we should not expose the I2C devices directly on OF; it should,
> instead, see just the bridge, and let the bridge to map the needed I2C
> devices using the needed platform_data.

We can't do that, there will be no platform data anymore with DT-based 
platforms.

As a summary, platform data structures won't go away, I2C drivers that need to 
work both on DT-based and non-DT-based platforms will need to support both the 
DT and platform data structures to get platform data. PCI and USB bridges will 
still instantiate their I2C devices, and binding between the I2C devices and 
the bridge can either be handled with two different instantiation mechanisms 
(the new one for DT platforms, with runtime bindings, and the existing one for 
non-DT platforms, with binding at instantiation time) or move to a single 
runtime binding mechanism. It's too early to know which solution will be 
simpler.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Mauro Carvalho Chehab]

Em Wed, 10 Oct 2012 16:48 +0200
Laurent Pinchart <laurent.pinchart@ideasonboard.com> escreveu:

> Hi Mauro,
> 
> On Wednesday 10 October 2012 10:45:22 Mauro Carvalho Chehab wrote:
> > Em Wed, 10 Oct 2012 14:54 +0200 Laurent Pinchart escreveu:
> > > > Also, ideally OF-compatible (I2C) drivers should run with no platform
> > > > data, but soc-camera is using I2C device platform data intensively. To
> > > > avoid modifying the soc-camera core and all drivers, I also trigger on
> > > > the
> > > > BUS_NOTIFY_BIND_DRIVER event and assign a reference to the dynamically
> > > > created platform data to the I2C device. Would we also want to do this
> > > > for
> > > > all V4L2 bridge drivers? We could call this a "prepare" callback or
> > > > something similar...
> > > 
> > > If that's going to be an interim solution only I'm fine with keeping it in
> > > soc-camera.
> > 
> > I'm far from being an OF expert, but why do we need to export I2C devices to
> > DT/OF? On my understanding, it is the bridge code that should be
> > responsible for detecting, binding and initializing the proper I2C devices.
> > On several cases, it is impossible or it would cause lots of ugly hacks if
> > we ever try to move away from platform data stuff, as only the bridge
> > driver knows what initialization is needed for an specific I2C driver.
> 
> In a nutshell, a DT-based platform doesn't have any board code (except in rare 
> cases, but let's not get into that), and thus doesn't pass any platform data 
> structure to drivers. However, drivers receive a DT node, which contains a 
> hierarchical description of the hardware, and parse those to extract 
> information necessary to configure the device.
> 
> One very important point to keep in mind here is that the DT is not Linux-
> specific. DT bindings are designed to be portable, and thus must only contain 
> hardware descriptions, without any OS-specific information or policy 
> information. For instance information such as the maximum video buffers size 
> is not allowed in the DT.
> 
> The DT is used both to provide platform data to drivers and to instantiate 
> devices. I2C device DT nodes are stored as children of the I2C bus master DT 
> node, and are automatically instantiated by the I2C bus master. This is a 
> significant change compared to our current situation where the V4L2 bridge 
> driver receives an array of I2C board information structures and instatiates 
> the devices itself. Most of the DT support work will go in supporting that new 
> instantiation mechanism. The bridge driver doesn't control instantiation of 
> the I2C devices anymore, but need to bind with them at runtime.
> 
> > To make things more complex, it is expected that most I2C drivers to be arch
> > independent, and they should be allowed to be used by a personal computer
> > or by an embedded device.
> 
> Agreed. That's why platform data structures won't go away anytime soon, a PCI 
> bridge driver for hardware that contain I2C devices will still instantiate the 
> I2C devices itself. We don't plan to or even want to get rid of that 
> mechanism, as there are perfectly valid use cases. However, for DT-based 
> embedded platforms, we need to support a new instantiation mechanism.
> 
> > Let me give 2 such examples:
> > 
> > 	- ir-i2c-kbd driver supports lots of IR devices. Platform_data is needed
> > to specify what hardware will actually be used, and what should be the
> > default Remote Controller keymap;
> 
> That driver isn't used on embedded platforms so it doesn't need to be changed 
> now.
> 
> > 	- Sensor drivers like ov2940 is needed by soc_camera and by other
> > webcam drivers like em28xx. The setup for em28xx should be completely
> > different than the one for soc_camera: the initial registers init sequence
> > is different for both. As several registers there aren't properly
> > documented, there's no easy way to parametrize the configuration.
> 
> Such drivers will need to support both DT-based platform data and structure-
> based platform data. They will likely parse the DT node and fill a platform 
> data structure, to avoid duplicating initialization code.
> 
> Please note that the new subdevs instantiation mechanism needed for DT will 
> need to handle any instantiation order, as we can't guarantee the I2C device 
> and bridge device instantiation order with DT. It should thus then support the 
> current instantiation order we use for PCI and USB platforms.
> 
> > So, for me, we should not expose the I2C devices directly on OF; it should,
> > instead, see just the bridge, and let the bridge to map the needed I2C
> > devices using the needed platform_data.
> 
> We can't do that, there will be no platform data anymore with DT-based 
> platforms.
> 
> As a summary, platform data structures won't go away, I2C drivers that need to 
> work both on DT-based and non-DT-based platforms will need to support both the 
> DT and platform data structures to get platform data. PCI and USB bridges will 
> still instantiate their I2C devices, and binding between the I2C devices and 
> the bridge can either be handled with two different instantiation mechanisms 
> (the new one for DT platforms, with runtime bindings, and the existing one for 
> non-DT platforms, with binding at instantiation time) or move to a single 
> runtime binding mechanism. It's too early to know which solution will be 
> simpler.
> 

It seems that you're designing a Frankstein monster with DT/OF and I2C.

Increasing each I2C driver code to support both platform_data and DT way
of doing things seems a huge amount of code that will be added, and I really
fail to understand why this is needed, in the first place.

Ok, I understand that OF specs are OS-independent, but I suspect that
they don't dictate how things should be wired internally at the OS.

So, if DT/OF is so restrictive, and require its own way of doing things, 
instead of changing everything with the risks of adding (more) regressions
to platform drivers, why don't just create a shell driver that will
encapsulate DT/OF specific stuff into the platform_data?

Regards,
Mauro

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Mauro,

On Wednesday 10 October 2012 11:57:03 Mauro Carvalho Chehab wrote:
> Em Wed, 10 Oct 2012 16:48 +0200 Laurent Pinchart escreveu:
> > On Wednesday 10 October 2012 10:45:22 Mauro Carvalho Chehab wrote:
> > > Em Wed, 10 Oct 2012 14:54 +0200 Laurent Pinchart escreveu:
> > > > > Also, ideally OF-compatible (I2C) drivers should run with no
> > > > > platform data, but soc-camera is using I2C device platform data
> > > > > intensively. To avoid modifying the soc-camera core and all drivers,
> > > > > I also trigger on the BUS_NOTIFY_BIND_DRIVER event and assign a
> > > > > reference to the dynamically created platform data to the I2C
> > > > > device. Would we also want to do this for all V4L2 bridge drivers?
> > > > > We could call this a "prepare" callback or something similar...
> > > > 
> > > > If that's going to be an interim solution only I'm fine with keeping
> > > > it in soc-camera.
> > > 
> > > I'm far from being an OF expert, but why do we need to export I2C
> > > devices to DT/OF? On my understanding, it is the bridge code that
> > > should be responsible for detecting, binding and initializing the proper
> > > I2C devices. On several cases, it is impossible or it would cause lots
> > > of ugly hacks if we ever try to move away from platform data stuff, as
> > > only the bridge driver knows what initialization is needed for an
> > > specific I2C driver.
> > 
> > In a nutshell, a DT-based platform doesn't have any board code (except in
> > rare cases, but let's not get into that), and thus doesn't pass any
> > platform data structure to drivers. However, drivers receive a DT node,
> > which contains a hierarchical description of the hardware, and parse
> > those to extract information necessary to configure the device.
> > 
> > One very important point to keep in mind here is that the DT is not Linux-
> > specific. DT bindings are designed to be portable, and thus must only
> > contain hardware descriptions, without any OS-specific information or
> > policy information. For instance information such as the maximum video
> > buffers size is not allowed in the DT.
> > 
> > The DT is used both to provide platform data to drivers and to instantiate
> > devices. I2C device DT nodes are stored as children of the I2C bus master
> > DT node, and are automatically instantiated by the I2C bus master. This
> > is a significant change compared to our current situation where the V4L2
> > bridge driver receives an array of I2C board information structures and
> > instatiates the devices itself. Most of the DT support work will go in
> > supporting that new instantiation mechanism. The bridge driver doesn't
> > control instantiation of the I2C devices anymore, but need to bind with
> > them at runtime.
> > 
> > > To make things more complex, it is expected that most I2C drivers to be
> > > arch independent, and they should be allowed to be used by a personal
> > > computer or by an embedded device.
> > 
> > Agreed. That's why platform data structures won't go away anytime soon, a
> > PCI bridge driver for hardware that contain I2C devices will still
> > instantiate the I2C devices itself. We don't plan to or even want to get
> > rid of that mechanism, as there are perfectly valid use cases. However,
> > for DT-based embedded platforms, we need to support a new instantiation
> > mechanism.
> >
> > > Let me give 2 such examples:
> > > 	- ir-i2c-kbd driver supports lots of IR devices. Platform_data is
> > > 	needed
> > > 
> > > to specify what hardware will actually be used, and what should be the
> > > default Remote Controller keymap;
> > 
> > That driver isn't used on embedded platforms so it doesn't need to be
> > changed now.
> > 
> > > 	- Sensor drivers like ov2940 is needed by soc_camera and by other
> > > 
> > > webcam drivers like em28xx. The setup for em28xx should be completely
> > > different than the one for soc_camera: the initial registers init
> > > sequence
> > > is different for both. As several registers there aren't properly
> > > documented, there's no easy way to parametrize the configuration.
> > 
> > Such drivers will need to support both DT-based platform data and
> > structure- based platform data. They will likely parse the DT node and
> > fill a platform data structure, to avoid duplicating initialization code.
> > 
> > Please note that the new subdevs instantiation mechanism needed for DT
> > will need to handle any instantiation order, as we can't guarantee the I2C
> > device and bridge device instantiation order with DT. It should thus then
> > support the current instantiation order we use for PCI and USB platforms.
> > 
> > > So, for me, we should not expose the I2C devices directly on OF; it
> > > should,
> > > instead, see just the bridge, and let the bridge to map the needed I2C
> > > devices using the needed platform_data.
> > 
> > We can't do that, there will be no platform data anymore with DT-based
> > platforms.
> > 
> > As a summary, platform data structures won't go away, I2C drivers that
> > need to work both on DT-based and non-DT-based platforms will need to
> > support both the DT and platform data structures to get platform data.
> > PCI and USB bridges will still instantiate their I2C devices, and binding
> > between the I2C devices and the bridge can either be handled with two
> > different instantiation mechanisms (the new one for DT platforms, with
> > runtime bindings, and the existing one for non-DT platforms, with binding
> > at instantiation time) or move to a single runtime binding mechanism.
> > It's too early to know which solution will be simpler.
> 
> It seems that you're designing a Frankstein monster with DT/OF and I2C.
> 
> Increasing each I2C driver code to support both platform_data and DT way
> of doing things seems a huge amount of code that will be added, and I really
> fail to understand why this is needed, in the first place.
> 
> Ok, I understand that OF specs are OS-independent, but I suspect that
> they don't dictate how things should be wired internally at the OS.
> 
> So, if DT/OF is so restrictive, and require its own way of doing things,
> instead of changing everything with the risks of adding (more) regressions
> to platform drivers, why don't just create a shell driver that will
> encapsulate DT/OF specific stuff into the platform_data?

DT support requires two independent parts, DT-based probing and device 
instantiation changes.

To support DT probing existing I2C drivers (and only the drivers that need to 
support DT-enabled platforms, we don't have to modify any I2C driver used on 
non-DT platforms only) will need to add a function that parses a DT node to 
fill a platform data structure, and a new OF match table pointer in their 
struct device.

For instance here's what the OMAP I2C bus master does in its probe function:

        match = of_match_device(of_match_ptr(omap_i2c_of_match), &pdev->dev);
        if (match) {
                u32 freq = 100000; /* default to 100000 Hz */

                pdata = match->data;
                dev->dtrev = pdata->rev;
                dev->flags = pdata->flags;

                of_property_read_u32(node, "clock-frequency", &freq);
                /* convert DT freq value in Hz into kHz for speed */
                dev->speed = freq / 1000;
        } else if (pdata != NULL) {
                dev->speed = pdata->clkrate;
                dev->flags = pdata->flags;
                dev->set_mpu_wkup_lat = pdata->set_mpu_wkup_lat;
                dev->dtrev = pdata->rev;
        }

Before DT support only the second branch of the if was there. The above code 
could be rewritten as

	if (DT enabled)
		fill_pdata_from_dt(pdata, DT node);

	Rest of normal pdata-based initialization code here

with the first branch of the if in the fill_pdata_from_dt() function.

It's really not a big deal in my opinion, and creating a new wrapper for that 
would just be overkill.

The device instantiation issue, discusses in this mail thread, is a more 
complex problem for which we need a solution, but isn't related to platform 
data.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sakari Ailus]

Hi Guennadi,

On Mon, Oct 08, 2012 at 02:23:25PM +0200, Guennadi Liakhovetski wrote:
...
> If we do it from the bridge driver, we could install an I2C bus-notifier, 
> _before_ the subdevice driver is probed, i.e. upon the 
> BUS_NOTIFY_BIND_DRIVER event we could turn on the clock. If subdevice 
> probing was successful, we can then wait for the BUS_NOTIFY_BOUND_DRIVER 
> event to switch the clock back off. BUT - if the subdevice fails probing? 
> How do we find out about that and turn the clock back off? There is no 
> notification event for that... Possible solutions:
> 
> 1. timer - ugly and unreliable.
> 2. add a "probing failed" notifier event to the device core - would this 
>    be accepted?
> 3. let the subdevice turn the master clock on and off for the duration of 
>    probing.
> 
> My vote goes for (3). Ideally this should be done using the generic clock 
> framework. But can we really expect all drivers and platforms to switch to 
> it quickly enough? If not, we need a V4L2-specific callback from subdevice 
> drivers to bridge drivers to turn the clock on and off. That's what I've 
> done "temporarily" in this patch series for soc-camera.

I'd say the clock has to be controlled by the sub-device driver. Sensors
also have a power-up (and power-down) sequences that should be followed.
Usually they also involve switching the external clock on (and off) at a
given point of time.

Also the OMAP 3 provides that clock through ISP so it, too, requires the
generic clock framework to function with DT.

> Suppose we decide to do the same for V4L2 centrally - add call-backs. Then 

I'd prefer to use the generic clock framework, albeit I admit it may take
some time till all the relevant platforms support it. Nevertheless, if
there's going to be a temporary solution it should be removed once the
clock framework support is there.

Kind regards,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

Hi

Bearing in mind the local time, I hope my brevity will be excused:-) Just 
wanted to give a sign, that I just finished a first successful run of a 
fully asynchronous and uniform multi-component non-DT video (soc-camera) 
pipeline probing, verified by a sample capture. What this means:

The pipeline consists of 3 components: a sensor, a CSI-2 interface, and a 
bridge. The platform code registers 3 devices: 2 platform devices for the 
bridge and CSI-2 and an I2C device for the sensor. The bridge driver gets 
a list of device descriptors, which it passes to the (soc-camera) generic 
code. That is used to initialise internal data and install bus (platform- 
and i2c-) notifiers. After all components have been collected the normal 
probing continues.

Next week I'll (hopefully) be cleaning all this up and converting from 
soc-camera to V4L2 core... After that I'll start posting, beginning with 
v2 of this DT series, taking into account possibly many comments:-)

I think, it might make sense to first post and have a look at the purely 
asynchronous code, first without DT additions, that we planned to 
implement for bus notifiers and whatever is related to them. If that looks 
reasonable, we can move on with adding DT. (One of) the ugly part(s) is 
going to be, that with this we'll be supporting 3 (!) pipeline 
initialisation modes: current (legacy) mode, generating I2C devices on the 
fly, non-DT asynchronous with all devices probing independently, and DT... 
Let's see if all this actually flies.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

On 10/05/2012 12:58 PM, Guennadi Liakhovetski wrote:
>> One area that I do not yet completely understand is the i2c bus notifications
>> (or asynchronous loading of i2c modules).
>>
>> I would have expected that using OF the i2c devices are still initialized
>> before the host/bridge driver is initialized. But I gather that's not the
>> case?
> 
> No, it's not. I'm not sure, whether it depends on the order of devices in
> the .dts, but, I think, it's better to not have to mandate a certain order
> and I also seem to have seen devices being registered in different order
> with the same DT, but I'm not 100% sure about that.

The device instantiation (and initialization) order is not something that
is supposed to be ensured by a specific device tree source layout, IIUC.
The initialization sequence is probably something that is specific to a
particular operating system. I checked the device tree compiler but couldn't
find if it is free to reorder anything when converting from the human 
readable device tree to its binary representation.

The deferred probing was introduced in Linux to resolve resource 
inter-dependencies in case of FDT based booting AFAIK.

>> If this deferred probing is a general problem, then I think we need a general
>> solution as well that's part of the v4l2 core.
> 
> That can be done, perhaps. But we can do it as a next step. As soon as
> we're happy with the OF implementation as such, we can commit that,
> possibly leaving soc-camera patches out for now, then we can think where
> to put async I2C handling.

I would really like to see more than one user until we add any core code.
No that it couldn't be changed afterwards, but it would be nice to ensure 
the concepts are right and proven in real life.

--

Regards,
Sylwester

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Mark Brown]

On Fri, Oct 05, 2012 at 08:30:59PM +0200, Sylwester Nawrocki wrote:

> The deferred probing was introduced in Linux to resolve resource 
> inter-dependencies in case of FDT based booting AFAIK.

It's completely unrelated to FDT, it's a general issue.  There's no sane
way to use hacks like link ordering to resolve boot time dependencies -
start getting things like regulators connected to I2C or SPI controllers
which may also use GPIOs for some signals and may be parents for other
regulators and mapping out the dependency graph becomes unreasonably
complicated.  Deferred probing is designed to solve this problem by
working things out dynamically at runtime.

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

Hi Sylwester

On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:

> On 10/05/2012 12:58 PM, Guennadi Liakhovetski wrote:
> >> One area that I do not yet completely understand is the i2c bus notifications
> >> (or asynchronous loading of i2c modules).
> >>
> >> I would have expected that using OF the i2c devices are still initialized
> >> before the host/bridge driver is initialized. But I gather that's not the
> >> case?
> > 
> > No, it's not. I'm not sure, whether it depends on the order of devices in
> > the .dts, but, I think, it's better to not have to mandate a certain order
> > and I also seem to have seen devices being registered in different order
> > with the same DT, but I'm not 100% sure about that.
> 
> The device instantiation (and initialization) order is not something that
> is supposed to be ensured by a specific device tree source layout, IIUC.
> The initialization sequence is probably something that is specific to a
> particular operating system. I checked the device tree compiler but couldn't
> find if it is free to reorder anything when converting from the human 
> readable device tree to its binary representation.
> 
> The deferred probing was introduced in Linux to resolve resource 
> inter-dependencies in case of FDT based booting AFAIK.
> 
> >> If this deferred probing is a general problem, then I think we need a general
> >> solution as well that's part of the v4l2 core.
> > 
> > That can be done, perhaps. But we can do it as a next step. As soon as
> > we're happy with the OF implementation as such, we can commit that,
> > possibly leaving soc-camera patches out for now, then we can think where
> > to put async I2C handling.
> 
> I would really like to see more than one user until we add any core code.
> No that it couldn't be changed afterwards, but it would be nice to ensure 
> the concepts are right and proven in real life.

Unfortunately I don't have any more systems on which I could easily enough 
try this. I've got a beagleboard with a camera, but I don't think I'm a 
particularly good candidate for implementing DT support for OMAP3 camera 
drivers;-) Apart from that I've only got soc-camera based systems, of 
which none are _really_ DT-ready... At best I could try an i.MX31 based 
board, but that doesn't have a very well developed .dts and that would be 
soc-camera too anyway.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

Hi Guennadi,

On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
> On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
>> I would really like to see more than one user until we add any core code.
>> No that it couldn't be changed afterwards, but it would be nice to ensure 
>> the concepts are right and proven in real life.
> 
> Unfortunately I don't have any more systems on which I could easily enough 
> try this. I've got a beagleboard with a camera, but I don't think I'm a 
> particularly good candidate for implementing DT support for OMAP3 camera 
> drivers;-) Apart from that I've only got soc-camera based systems, of 
> which none are _really_ DT-ready... At best I could try an i.MX31 based 
> board, but that doesn't have a very well developed .dts and that would be 
> soc-camera too anyway.

I certainly wouldn't expect you would do all the job. I mean it would be good
to possibly have some other developers adding device tree support based on that 
new bindings and new infrastructure related to them. 

There have been recently some progress in device tree support for Exynos SoCs,
including common clock framework support and we hope to add FDT support to the 
Samsung SoC camera devices during this kernel cycle, based on the newly designed 
media bindings. This is going to be a second attempt, after our initial RFC from 
May [1]. It would still be SoC specific implementation, but not soc-camera based.
 
I wasn't a big fan of this asynchronous sub-devices probing, but it now seems
to be a most complete solution to me. I think it just need to be done right
at the v4l2-core so individual drivers don't get complicated too much.

--

Regards,
Sylwester

[1] http://www.spinics.net/lists/linux-media/msg48341.html

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Hans Verkuil]

On Tue 9 October 2012 12:34:48 Sylwester Nawrocki wrote:
> Hi Guennadi,
> 
> On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
> > On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
> >> I would really like to see more than one user until we add any core code.
> >> No that it couldn't be changed afterwards, but it would be nice to ensure 
> >> the concepts are right and proven in real life.
> > 
> > Unfortunately I don't have any more systems on which I could easily enough 
> > try this. I've got a beagleboard with a camera, but I don't think I'm a 
> > particularly good candidate for implementing DT support for OMAP3 camera 
> > drivers;-) Apart from that I've only got soc-camera based systems, of 
> > which none are _really_ DT-ready... At best I could try an i.MX31 based 
> > board, but that doesn't have a very well developed .dts and that would be 
> > soc-camera too anyway.
> 
> I certainly wouldn't expect you would do all the job. I mean it would be good
> to possibly have some other developers adding device tree support based on that 
> new bindings and new infrastructure related to them. 
> 
> There have been recently some progress in device tree support for Exynos SoCs,
> including common clock framework support and we hope to add FDT support to the 
> Samsung SoC camera devices during this kernel cycle, based on the newly designed 
> media bindings. This is going to be a second attempt, after our initial RFC from 
> May [1]. It would still be SoC specific implementation, but not soc-camera based.
>  
> I wasn't a big fan of this asynchronous sub-devices probing, but it now seems
> to be a most complete solution to me. I think it just need to be done right
> at the v4l2-core so individual drivers don't get complicated too much.

After investigating this some more I think I agree with that. There are some
things where we should probably ask for advice from the i2c subsystem devs,
I'm thinking of putting the driver back into the deferred-probe state in
particular.

Creating v4l2-core support for this is crucial as it is quite complex and
without core support this is going to be a nightmare for drivers.

Regards,

	Hans

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Hans,

On Tuesday 09 October 2012 13:00:24 Hans Verkuil wrote:
> On Tue 9 October 2012 12:34:48 Sylwester Nawrocki wrote:
> > On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
> > > On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
> > >> I would really like to see more than one user until we add any core
> > >> code. No that it couldn't be changed afterwards, but it would be nice
> > >> to ensure the concepts are right and proven in real life.
> > > 
> > > Unfortunately I don't have any more systems on which I could easily
> > > enough try this. I've got a beagleboard with a camera, but I don't think
> > > I'm a particularly good candidate for implementing DT support for OMAP3
> > > camera drivers;-) Apart from that I've only got soc-camera based
> > > systems, of which none are _really_ DT-ready... At best I could try an
> > > i.MX31 based board, but that doesn't have a very well developed .dts and
> > > that would be soc-camera too anyway.
> > 
> > I certainly wouldn't expect you would do all the job. I mean it would be
> > good to possibly have some other developers adding device tree support
> > based on that new bindings and new infrastructure related to them.

As I mentioned in another e-mail, I plan to work on DT support for the OMAP3 
ISP, but I first need generic clock framework support for OMAP3.

> > There have been recently some progress in device tree support for Exynos
> > SoCs, including common clock framework support and we hope to add FDT
> > support to the Samsung SoC camera devices during this kernel cycle, based
> > on the newly designed media bindings. This is going to be a second
> > attempt, after our initial RFC from May [1]. It would still be SoC
> > specific implementation, but not soc-camera based.
> > 
> > I wasn't a big fan of this asynchronous sub-devices probing, but it now
> > seems to be a most complete solution to me. I think it just need to be
> > done right at the v4l2-core so individual drivers don't get complicated
> > too much.
>
> After investigating this some more I think I agree with that. There are some
> things where we should probably ask for advice from the i2c subsystem devs,
> I'm thinking of putting the driver back into the deferred-probe state in
> particular.

We might actually not need that, it might be easier to handle the circular 
dependency problem from the other end. We could add a way (ioctl, sysfs, ...) 
to force a V4L2 bridge driver to release its subdevs. Once done, the subdev 
driver could be unloaded and/or the subdev device unregistered, which would 
release the resources used by the subdev, such as clocks. The bridge driver 
could then be unregistered.

> Creating v4l2-core support for this is crucial as it is quite complex and
> without core support this is going to be a nightmare for drivers.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

Hi Laurent,

On 10/10/2012 03:25 PM, Laurent Pinchart wrote:
> On Tuesday 09 October 2012 13:00:24 Hans Verkuil wrote:
>> On Tue 9 October 2012 12:34:48 Sylwester Nawrocki wrote:
>>> On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
>>>> On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
>>>>> I would really like to see more than one user until we add any core
>>>>> code. No that it couldn't be changed afterwards, but it would be nice
>>>>> to ensure the concepts are right and proven in real life.
>>>>
>>>> Unfortunately I don't have any more systems on which I could easily
>>>> enough try this. I've got a beagleboard with a camera, but I don't think
>>>> I'm a particularly good candidate for implementing DT support for OMAP3
>>>> camera drivers;-) Apart from that I've only got soc-camera based
>>>> systems, of which none are _really_ DT-ready... At best I could try an
>>>> i.MX31 based board, but that doesn't have a very well developed .dts and
>>>> that would be soc-camera too anyway.
>>>
>>> I certainly wouldn't expect you would do all the job. I mean it would be
>>> good to possibly have some other developers adding device tree support
>>> based on that new bindings and new infrastructure related to them.
> 
> As I mentioned in another e-mail, I plan to work on DT support for the OMAP3
> ISP, but I first need generic clock framework support for OMAP3.

OK, let's hope it's available soon.

>>> There have been recently some progress in device tree support for Exynos
>>> SoCs, including common clock framework support and we hope to add FDT
>>> support to the Samsung SoC camera devices during this kernel cycle, based
>>> on the newly designed media bindings. This is going to be a second
>>> attempt, after our initial RFC from May [1]. It would still be SoC
>>> specific implementation, but not soc-camera based.
>>>
>>> I wasn't a big fan of this asynchronous sub-devices probing, but it now
>>> seems to be a most complete solution to me. I think it just need to be
>>> done right at the v4l2-core so individual drivers don't get complicated
>>> too much.
>>
>> After investigating this some more I think I agree with that. There are some
>> things where we should probably ask for advice from the i2c subsystem devs,
>> I'm thinking of putting the driver back into the deferred-probe state in
>> particular.
> 
> We might actually not need that, it might be easier to handle the circular
> dependency problem from the other end. We could add a way (ioctl, sysfs, ...)
> to force a V4L2 bridge driver to release its subdevs. Once done, the subdev
> driver could be unloaded and/or the subdev device unregistered, which would
> release the resources used by the subdev, such as clocks. The bridge driver
> could then be unregistered.

That sounds like an option. Perhaps it could be done by v4l2-core, e.g. a sysfs 
entry could be registered for a media or video device if driver requests it.
I'm not sure if we should allow subdevs in "released" state, perhaps it's better
to just unregister subdevs entirely right away ?

>> Creating v4l2-core support for this is crucial as it is quite complex and
>> without core support this is going to be a nightmare for drivers.

--

Regards,
Sylwester

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Guennadi Liakhovetski]

On Wed, 10 Oct 2012, Sylwester Nawrocki wrote:

> Hi Laurent,
> 
> On 10/10/2012 03:25 PM, Laurent Pinchart wrote:
> > On Tuesday 09 October 2012 13:00:24 Hans Verkuil wrote:
> >> On Tue 9 October 2012 12:34:48 Sylwester Nawrocki wrote:
> >>> On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
> >>>> On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
> >>>>> I would really like to see more than one user until we add any core
> >>>>> code. No that it couldn't be changed afterwards, but it would be nice
> >>>>> to ensure the concepts are right and proven in real life.
> >>>>
> >>>> Unfortunately I don't have any more systems on which I could easily
> >>>> enough try this. I've got a beagleboard with a camera, but I don't think
> >>>> I'm a particularly good candidate for implementing DT support for OMAP3
> >>>> camera drivers;-) Apart from that I've only got soc-camera based
> >>>> systems, of which none are _really_ DT-ready... At best I could try an
> >>>> i.MX31 based board, but that doesn't have a very well developed .dts and
> >>>> that would be soc-camera too anyway.
> >>>
> >>> I certainly wouldn't expect you would do all the job. I mean it would be
> >>> good to possibly have some other developers adding device tree support
> >>> based on that new bindings and new infrastructure related to them.
> > 
> > As I mentioned in another e-mail, I plan to work on DT support for the OMAP3
> > ISP, but I first need generic clock framework support for OMAP3.
> 
> OK, let's hope it's available soon.
> 
> >>> There have been recently some progress in device tree support for Exynos
> >>> SoCs, including common clock framework support and we hope to add FDT
> >>> support to the Samsung SoC camera devices during this kernel cycle, based
> >>> on the newly designed media bindings. This is going to be a second
> >>> attempt, after our initial RFC from May [1]. It would still be SoC
> >>> specific implementation, but not soc-camera based.
> >>>
> >>> I wasn't a big fan of this asynchronous sub-devices probing, but it now
> >>> seems to be a most complete solution to me. I think it just need to be
> >>> done right at the v4l2-core so individual drivers don't get complicated
> >>> too much.
> >>
> >> After investigating this some more I think I agree with that. There are some
> >> things where we should probably ask for advice from the i2c subsystem devs,
> >> I'm thinking of putting the driver back into the deferred-probe state in
> >> particular.
> > 
> > We might actually not need that, it might be easier to handle the circular
> > dependency problem from the other end. We could add a way (ioctl, sysfs, ...)
> > to force a V4L2 bridge driver to release its subdevs. Once done, the subdev
> > driver could be unloaded and/or the subdev device unregistered, which would
> > release the resources used by the subdev, such as clocks. The bridge driver
> > could then be unregistered.
> 
> That sounds like an option. Perhaps it could be done by v4l2-core, e.g. a sysfs 
> entry could be registered for a media or video device if driver requests it.
> I'm not sure if we should allow subdevs in "released" state, perhaps it's better
> to just unregister subdevs entirely right away ?

What speaks against holding a clock reference only during streaming, or at 
least between open and close? Wouldn't that solve the problem naturally? 
Yes, after giving up your reference to a clock at close() and re-acquiring 
it at open() you will have to make sure the frequency hasn't change, resp. 
adjust it, but I don't see it as a huge problem, I don't think many users 
on embedded systems will compete for your camera master clock. And if they 
do, you have a different problem, IMHO;-)

Thanks
Guennadi

> >> Creating v4l2-core support for this is crucial as it is quite complex and
> >> without core support this is going to be a nightmare for drivers.
> 
> --
> 
> Regards,
> Sylwester

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

On 10/10/2012 10:32 PM, Guennadi Liakhovetski wrote:
>>> We might actually not need that, it might be easier to handle the circular
>>> dependency problem from the other end. We could add a way (ioctl, sysfs, ...)
>>> to force a V4L2 bridge driver to release its subdevs. Once done, the subdev
>>> driver could be unloaded and/or the subdev device unregistered, which would
>>> release the resources used by the subdev, such as clocks. The bridge driver
>>> could then be unregistered.
>>
>> That sounds like an option. Perhaps it could be done by v4l2-core, e.g. a sysfs
>> entry could be registered for a media or video device if driver requests it.
>> I'm not sure if we should allow subdevs in "released" state, perhaps it's better
>> to just unregister subdevs entirely right away ?
> 
> What speaks against holding a clock reference only during streaming, or at
> least between open and close? Wouldn't that solve the problem naturally?
> Yes, after giving up your reference to a clock at close() and re-acquiring
> it at open() you will have to make sure the frequency hasn't change, resp.
> adjust it, but I don't see it as a huge problem, I don't think many users
> on embedded systems will compete for your camera master clock. And if they
> do, you have a different problem, IMHO;-)

I agree, normally nobody should touch these clocks except the subdev (or as of 
now the host) drivers. It depends on a sensor, video encoder, etc. how much it 
tolerates switching the clock on/off. I suppose it's best to acquire/release it
in .s_power callback, since only then the proper voltage supply, GPIO, clock 
enable/disable sequences could be ensured. I know those things are currently 
mostly ignored, but some sensors might be picky WRT their initialization/shutdown
sequences and it would be good to ensure these sequences are fully controllable 
by the sensor driver itsels, where the host's architecture allows that.

To summarize, I can't see how holding a clock only when a device is active
could cause any problems, in case of camera sensors. I'm not sure about other
devices, like e.g. tuners.

--

Regards,
Sylwester

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Guennadi,

On Wednesday 10 October 2012 22:32:22 Guennadi Liakhovetski wrote:
> On Wed, 10 Oct 2012, Sylwester Nawrocki wrote:
> > On 10/10/2012 03:25 PM, Laurent Pinchart wrote:
> > > On Tuesday 09 October 2012 13:00:24 Hans Verkuil wrote:
> > >> On Tue 9 October 2012 12:34:48 Sylwester Nawrocki wrote:
> > >>> On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
> > >>>> On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
> > >>>>> I would really like to see more than one user until we add any core
> > >>>>> code. No that it couldn't be changed afterwards, but it would be
> > >>>>> nice to ensure the concepts are right and proven in real life.
> > >>>> 
> > >>>> Unfortunately I don't have any more systems on which I could easily
> > >>>> enough try this. I've got a beagleboard with a camera, but I don't
> > >>>> think I'm a particularly good candidate for implementing DT support
> > >>>> for OMAP3 camera drivers;-) Apart from that I've only got soc-camera
> > >>>> based systems, of which none are _really_ DT-ready... At best I could
> > >>>> try an i.MX31 based board, but that doesn't have a very well
> > >>>> developed .dts and that would be soc-camera too anyway.
> > >>> 
> > >>> I certainly wouldn't expect you would do all the job. I mean it would
> > >>> be good to possibly have some other developers adding device tree
> > >>> support based on that new bindings and new infrastructure related to
> > >>> them.
> > > 
> > > As I mentioned in another e-mail, I plan to work on DT support for the
> > > OMAP3 ISP, but I first need generic clock framework support for OMAP3.
> > 
> > OK, let's hope it's available soon.
> > 
> > >>> There have been recently some progress in device tree support for
> > >>> Exynos SoCs, including common clock framework support and we hope to
> > >>> add FDT support to the Samsung SoC camera devices during this kernel
> > >>> cycle, based on the newly designed media bindings. This is going to be
> > >>> a second attempt, after our initial RFC from May [1]. It would still
> > >>> be SoC specific implementation, but not soc-camera based.
> > >>> 
> > >>> I wasn't a big fan of this asynchronous sub-devices probing, but it
> > >>> now seems to be a most complete solution to me. I think it just need
> > >>> to be done right at the v4l2-core so individual drivers don't get
> > >>> complicated too much.
> > >> 
> > >> After investigating this some more I think I agree with that. There are
> > >> some things where we should probably ask for advice from the i2c
> > >> subsystem devs, I'm thinking of putting the driver back into the
> > >> deferred-probe state in particular.
> > > 
> > > We might actually not need that, it might be easier to handle the
> > > circular dependency problem from the other end. We could add a way
> > > (ioctl, sysfs, ...) to force a V4L2 bridge driver to release its
> > > subdevs. Once done, the subdev driver could be unloaded and/or the
> > > subdev device unregistered, which would release the resources used by
> > > the subdev, such as clocks. The bridge driver could then be
> > > unregistered.
> > 
> > That sounds like an option. Perhaps it could be done by v4l2-core, e.g. a
> > sysfs entry could be registered for a media or video device if driver
> > requests it. I'm not sure if we should allow subdevs in "released" state,
> > perhaps it's better to just unregister subdevs entirely right away ?
> 
> What speaks against holding a clock reference only during streaming, or at
> least between open and close? Wouldn't that solve the problem naturally?
> Yes, after giving up your reference to a clock at close() and re-acquiring
> it at open() you will have to make sure the frequency hasn't change, resp.
> adjust it, but I don't see it as a huge problem, I don't think many users
> on embedded systems will compete for your camera master clock. And if they
> do, you have a different problem, IMHO;-)

That's an option as well. I'm a bit worried that it would make subdev drivers 
more complex, as they would need to acquire/release resources in a more fine-
grained fashion at runtime, and handle failures dynamically there instead of 
doing it all at probe time. It could work though, I think we need to 
experiment the possible solutions to find out which one is the best.

Regardless of how we solve the circular dependencies issue at unregistration 
time we will need an easy way for bridge drivers to bind to subdevs. I believe 
that's orthogonal to the unregistration problem, so we can start working on 
registration before knowing exactly how unregistration will be handled.

> > >> Creating v4l2-core support for this is crucial as it is quite complex
> > >> and without core support this is going to be a nightmare for drivers.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

Hi Sylwester,

On Wednesday 10 October 2012 22:23:35 Sylwester Nawrocki wrote:
> On 10/10/2012 03:25 PM, Laurent Pinchart wrote:
> > On Tuesday 09 October 2012 13:00:24 Hans Verkuil wrote:
> >> On Tue 9 October 2012 12:34:48 Sylwester Nawrocki wrote:
> >>> On 10/08/2012 11:40 AM, Guennadi Liakhovetski wrote:
> >>>> On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:
> >>>>> I would really like to see more than one user until we add any core
> >>>>> code. No that it couldn't be changed afterwards, but it would be nice
> >>>>> to ensure the concepts are right and proven in real life.
> >>>> 
> >>>> Unfortunately I don't have any more systems on which I could easily
> >>>> enough try this. I've got a beagleboard with a camera, but I don't
> >>>> think
> >>>> I'm a particularly good candidate for implementing DT support for OMAP3
> >>>> camera drivers;-) Apart from that I've only got soc-camera based
> >>>> systems, of which none are _really_ DT-ready... At best I could try an
> >>>> i.MX31 based board, but that doesn't have a very well developed .dts
> >>>> and
> >>>> that would be soc-camera too anyway.
> >>> 
> >>> I certainly wouldn't expect you would do all the job. I mean it would be
> >>> good to possibly have some other developers adding device tree support
> >>> based on that new bindings and new infrastructure related to them.
> > 
> > As I mentioned in another e-mail, I plan to work on DT support for the
> > OMAP3 ISP, but I first need generic clock framework support for OMAP3.
> 
> OK, let's hope it's available soon.

I've been told a month and a half ago that v3.7 was a plausible target, let's 
see.

> >>> There have been recently some progress in device tree support for Exynos
> >>> SoCs, including common clock framework support and we hope to add FDT
> >>> support to the Samsung SoC camera devices during this kernel cycle,
> >>> based on the newly designed media bindings. This is going to be a second
> >>> attempt, after our initial RFC from May [1]. It would still be SoC
> >>> specific implementation, but not soc-camera based.
> >>> 
> >>> I wasn't a big fan of this asynchronous sub-devices probing, but it now
> >>> seems to be a most complete solution to me. I think it just need to be
> >>> done right at the v4l2-core so individual drivers don't get complicated
> >>> too much.
> >> 
> >> After investigating this some more I think I agree with that. There are
> >> some things where we should probably ask for advice from the i2c
> >> subsystem devs, I'm thinking of putting the driver back into the
> >> deferred-probe state in particular.
> > 
> > We might actually not need that, it might be easier to handle the circular
> > dependency problem from the other end. We could add a way (ioctl, sysfs,
> > ...) to force a V4L2 bridge driver to release its subdevs. Once done, the
> > subdev driver could be unloaded and/or the subdev device unregistered,
> > which would release the resources used by the subdev, such as clocks. The
> > bridge driver could then be unregistered.
> 
> That sounds like an option. Perhaps it could be done by v4l2-core, e.g. a
> sysfs entry could be registered for a media or video device if driver
> requests it.

That's what I was thinking about.

> I'm not sure if we should allow subdevs in "released" state, perhaps it's
> better to just unregister subdevs entirely right away ?

I think we need three states: not created, unbound and bound (names are not 
set in stone). The not created state corresponds to a subdev that hasn't been 
created yet by its I2C/SPI/whatever driver (agreed, it's not really a state 
technically). Upon creation the subdev is not bound to any bridge driver. It 
later gets bound when the bridge driver requests the subdev through the API 
the V4L2 core will provide (most probably using notifiers). The new sysfs 
entry would be used to unbind subdevs (either selectively, or all in one go) 
from the bridge, in which case they will go back to the unbound state, 
allowing driver removal or device release.

> >> Creating v4l2-core support for this is crucial as it is quite complex and
> >> without core support this is going to be a nightmare for drivers.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Laurent Pinchart]

On Friday 05 October 2012 12:58:21 Guennadi Liakhovetski wrote:
> On Fri, 5 Oct 2012, Hans Verkuil wrote:
> > On Tue October 2 2012 12:13:20 Sylwester Nawrocki wrote:
> > > On 10/02/2012 11:49 AM, Guennadi Liakhovetski wrote:
> > > >>> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> > > >>> +					ARRAY_SIZE(data_lanes))) {
> > > >>> +		int i;
> > > >>> +		for (i = 0; i<  ARRAY_SIZE(data_lanes); i++)
> > > >>> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > > >> 
> > > >> It doesn't look like what we aimed for. The data-lanes array is
> > > >> supposed to be of variable length, thus I don't think it can be
> > > >> parsed like that. Or am I missing something ? I think we need more
> > > >> something like below
> > > > 
> > > >> (based on of_property_read_u32_array(), not tested):
> > > > Ok, you're right, that my version only was suitable for fixed-size
> > > > arrays, which wasn't our goal. But I don't think we should go down to
> > > > manually parsing property data. How about (tested;-))
> > > > 
> > > > 	data = of_find_property(node, "data-lanes", NULL);
> > > > 	if (data) {
> > > > 	
> > > > 		int i = 0;
> > > > 		const __be32 *lane = NULL;
> > > > 		do {
> > > > 		
> > > > 			lane = of_prop_next_u32(data, lane, &data_lanes[i]);
> > > > 		
> > > > 		} while (lane && i++ < ARRAY_SIZE(data_lanes));
> > > > 		link->mipi_csi_2.num_data_lanes = i;
> > > > 		while (i--)
> > > > 		
> > > > 			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> > > > 	
> > > > 	}
> > > 
> > > Yes, that looks neat and does what it is supposed to do. :) Thanks!
> > > For now, I'll trust you it works ;)
> > > 
> > > With regards to the remaining patches, it looks a bit scary to me how
> > > complicated it got, perhaps mostly because of requirement to reference
> > > host devices from subdevs. And it seems to rely on the existing SoC
> > > camera infrastructure, which might imply lot's of work need to be done
> > > for non soc-camera drivers. But I'm going to take a closer look and
> > > comment more on the details at the corresponding patches.
> > 
> > I have to say that I agree with Sylwester here. It seems awfully
> > complicated, but I can't really put my finger on the actual cause.
> 
> Well, which exactly part? The V4L2 OF part is quite simple.
> 
> > It would be really interesting to see this implemented for a non-SoC
> > device and to compare the two.
> 
> Sure, volunteers? ;-) In principle, if I find time, I could try to convert
> sh_vou, which is also interesting, because it's an output driver.

The OMAP3 ISP is on my to-do list, but that depends on the generic clock 
availability on the OMAP3, so I have to wait.

> > One area that I do not yet completely understand is the i2c bus
> > notifications (or asynchronous loading or i2c modules).
> > 
> > I would have expected that using OF the i2c devices are still initialized
> > before the host/bridge driver is initialized. But I gather that's not the
> > case?
> 
> No, it's not. I'm not sure, whether it depends on the order of devices in
> the .dts, but, I think, it's better to not have to mandate a certain order
> and I also seem to have seen devices being registered in different order
> with the same DT, but I'm not 100% sure about that.
>
> > If this deferred probing is a general problem, then I think we need a
> > general solution as well that's part of the v4l2 core.
> 
> That can be done, perhaps. But we can do it as a next step. As soon as
> we're happy with the OF implementation as such, we can commit that,
> possibly leaving soc-camera patches out for now, then we can think where
> to put async I2C handling.

I agree that async I2C handling should have V4L2 core helpers, otherwise it's 
going to be pretty complex for ISP drivers.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/14] media: add a V4L2 OF parser

[Sylwester Nawrocki]

Hi Guennadi,

On 09/27/2012 04:07 PM, Guennadi Liakhovetski wrote:
> Add a V4L2 OF parser, implementing bindings, documented in
> Documentation/devicetree/bindings/media/v4l2.txt.
> 
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> ---
>  drivers/media/v4l2-core/Makefile  |    3 +
>  drivers/media/v4l2-core/v4l2-of.c |  190 +++++++++++++++++++++++++++++++++++++
>  include/media/v4l2-of.h           |   62 ++++++++++++
>  3 files changed, 255 insertions(+), 0 deletions(-)
>  create mode 100644 drivers/media/v4l2-core/v4l2-of.c
>  create mode 100644 include/media/v4l2-of.h
> 
> diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
> index c2d61d4..00f64d6 100644
> --- a/drivers/media/v4l2-core/Makefile
> +++ b/drivers/media/v4l2-core/Makefile
> @@ -9,6 +9,9 @@ videodev-objs	:=	v4l2-dev.o v4l2-ioctl.o v4l2-device.o v4l2-fh.o \
>  ifeq ($(CONFIG_COMPAT),y)
>    videodev-objs += v4l2-compat-ioctl32.o
>  endif
> +ifeq ($(CONFIG_OF),y)
> +  videodev-objs += v4l2-of.o
> +endif
>  
>  obj-$(CONFIG_VIDEO_DEV) += videodev.o v4l2-int-device.o
>  obj-$(CONFIG_VIDEO_V4L2) += v4l2-common.o
> diff --git a/drivers/media/v4l2-core/v4l2-of.c b/drivers/media/v4l2-core/v4l2-of.c
> new file mode 100644
> index 0000000..f45d64b
> --- /dev/null
> +++ b/drivers/media/v4l2-core/v4l2-of.c
> @@ -0,0 +1,190 @@
> +/*
> + * V4L2 OF binding parsing library
> + *
> + * Copyright (C) 2012 Renesas Electronics Corp.
> + * Author: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of version 2 of the GNU General Public License as
> + * published by the Free Software Foundation.
> + */
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/types.h>
> +
> +#include <media/v4l2-of.h>
> +
> +/*
> + * All properties are optional. If none are found, we don't set any flags. This
> + * means, the port has a static configuration and no properties have to be
> + * specified explicitly.
> + * If any properties are found, that identify the bus as parallel, and
> + * slave-mode isn't set, we set V4L2_MBUS_MASTER. Similarly, if we recognise the
> + * bus as serial CSI-2 and clock-noncontinuous isn't set, we set the
> + * V4L2_MBUS_CSI2_CONTINUOUS_CLOCK flag.
> + * The caller should hold a reference to "node."
> + */

Since this is a library function, how about converting this description
to kernel-doc ?

> +void v4l2_of_parse_link(const struct device_node *node,
> +			struct v4l2_of_link *link)
> +{
> +	const struct device_node *port_node = of_get_parent(node);
> +	int size;
> +	unsigned int v;
> +	u32 data_lanes[ARRAY_SIZE(link->mipi_csi_2.data_lanes)];
> +	bool data_lanes_present;
> +
> +	memset(link, 0, sizeof(*link));
> +
> +	link->local_node = node;
> +
> +	/* Doesn't matter, whether the below two calls succeed */
> +	of_property_read_u32(port_node, "reg", &link->port);
> +	of_property_read_u32(node, "reg", &link->addr);
> +
> +	if (!of_property_read_u32(node, "bus-width", &v))
> +		link->parallel.bus_width = v;
> +
> +	if (!of_property_read_u32(node, "data-shift", &v))
> +		link->parallel.data_shift = v;
> +
> +	if (!of_property_read_u32(node, "hsync-active", &v))
> +		link->mbus_flags |= v ? V4L2_MBUS_HSYNC_ACTIVE_HIGH :
> +			V4L2_MBUS_HSYNC_ACTIVE_LOW;
> +
> +	if (!of_property_read_u32(node, "vsync-active", &v))
> +		link->mbus_flags |= v ? V4L2_MBUS_VSYNC_ACTIVE_HIGH :
> +			V4L2_MBUS_VSYNC_ACTIVE_LOW;
> +
> +	if (!of_property_read_u32(node, "data-active", &v))
> +		link->mbus_flags |= v ? V4L2_MBUS_DATA_ACTIVE_HIGH :
> +			V4L2_MBUS_DATA_ACTIVE_LOW;
> +
> +	if (!of_property_read_u32(node, "pclk-sample", &v))
> +		link->mbus_flags |= v ? V4L2_MBUS_PCLK_SAMPLE_RISING :
> +			V4L2_MBUS_PCLK_SAMPLE_FALLING;
> +
> +	if (!of_property_read_u32(node, "field-even-active", &v))
> +		link->mbus_flags |= v ? V4L2_MBUS_FIELD_EVEN_HIGH :
> +			V4L2_MBUS_FIELD_EVEN_LOW;
> +
> +	if (of_get_property(node, "slave-mode", &size))
> +		link->mbus_flags |= V4L2_MBUS_SLAVE;
> +
> +	/* If any parallel-bus properties have been found, skip serial ones */
> +	if (link->parallel.bus_width || link->parallel.data_shift ||
> +	    link->mbus_flags) {
> +		/* Default parallel bus-master */
> +		if (!(link->mbus_flags & V4L2_MBUS_SLAVE))
> +			link->mbus_flags |= V4L2_MBUS_MASTER;
> +		return;
> +	}
> +
> +	if (!of_property_read_u32(node, "clock-lanes", &v))
> +		link->mipi_csi_2.clock_lane = v;
> +
> +	if (!of_property_read_u32_array(node, "data-lanes", data_lanes,
> +					ARRAY_SIZE(data_lanes))) {
> +		int i;
> +		for (i = 0; i < ARRAY_SIZE(data_lanes); i++)
> +			link->mipi_csi_2.data_lanes[i] = data_lanes[i];
> +		data_lanes_present = true;
> +	} else {
> +		data_lanes_present = false;
> +	}
> +
> +	if (of_get_property(node, "clock-noncontinuous", &size))
> +		link->mbus_flags |= V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK;
> +
> +	if ((link->mipi_csi_2.clock_lane || data_lanes_present) &&
> +	    !(link->mbus_flags & V4L2_MBUS_CSI2_NONCONTINUOUS_CLOCK))
> +		/* Default CSI-2: continuous clock */
> +		link->mbus_flags |= V4L2_MBUS_CSI2_CONTINUOUS_CLOCK;
> +}
> +EXPORT_SYMBOL(v4l2_of_parse_link);
> +
> +/*
> + * Return a refcounted next "link" DT node. Contrary to the common OF practice,
> + * we do not drop the reference to previous, users have to do it themselves,
> + * when they're done with the node.
> + */
> +struct device_node *v4l2_of_get_next_link(const struct device_node *parent,
> +					struct device_node *previous)

Looks good to me. Only a proper kernel-doc description seems to be missing.
> +{
> +	struct device_node *child, *port;
> +
> +	if (!parent)
> +		return NULL;
> +
> +	if (!previous) {
> +		/*
> +		 * If this is the first call, we have to find a port within this
> +		 * node
> +		 */
> +		for_each_child_of_node(parent, port) {
> +			if (!of_node_cmp(port->name, "port"))
> +				break;
> +		}
> +		if (port) {
> +			/* Found a port, get a link */
> +			child = of_get_next_child(port, NULL);
> +			of_node_put(port);
> +		} else {
> +			child = NULL;
> +		}
> +		if (!child)
> +			pr_err("%s(): Invalid DT: %s has no link children!\n",
> +			       __func__, parent->name);
> +	} else {
> +		port = of_get_parent(previous);
> +		if (!port)
> +			/* Hm, has someone given us the root node?... */
> +			return NULL;
> +
> +		/* Avoid dropping previous refcount to 0 */
> +		of_node_get(previous);
> +		child = of_get_next_child(port, previous);
> +		if (child) {
> +			of_node_put(port);
> +			return child;
> +		}
> +
> +		/* No more links under this port, try the next one */
> +		do {
> +			port = of_get_next_child(parent, port);
> +			if (!port)
> +				return NULL;
> +		} while (of_node_cmp(port->name, "port"));
> +
> +		/* Pick up the first link on this port */
> +		child = of_get_next_child(port, NULL);
> +		of_node_put(port);
> +	}
> +
> +	return child;
> +}
> +EXPORT_SYMBOL(v4l2_of_get_next_link);
> +
> +/* Return a refcounted DT node, owning the link, referenced by "remote" */

Since this returns parent node of of a port on the other end of the link,
how about changing the name to v4l2_of_get_remote_link_parent() ?

Also kernel-doc description would be useful, so one doesn't have to
necessarily dig into the code to see what this function does exactly.

> +struct device_node *v4l2_of_get_remote(const struct device_node *node)
> +{
> +	struct device_node *remote, *tmp;
> +
> +	/* Get remote link DT node */
> +	remote = of_parse_phandle(node, "remote", 0);
> +	if (!remote)
> +		return NULL;
> +
> +	/* remote port */
> +	tmp = of_get_parent(remote);
> +	of_node_put(remote);
> +	if (!tmp)
> +		return NULL;
> +
> +	/* remote DT node */
> +	remote = of_get_parent(tmp);
> +	of_node_put(tmp);
> +
> +	return remote;
> +}
> +EXPORT_SYMBOL(v4l2_of_get_remote);

--

Thanks,
Sylwester

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Sylwester Nawrocki]

Hi Guennadi,

On 09/27/2012 04:07 PM, Guennadi Liakhovetski wrote:
> This patch adds a document, describing common V4L2 device tree bindings.
> 
> Co-authored-by: Sylwester Nawrocki<s.nawrocki@samsung.com>
> Signed-off-by: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> ---
>   Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
>   1 files changed, 162 insertions(+), 0 deletions(-)
>   create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> new file mode 100644
> index 0000000..b8b3f41
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> @@ -0,0 +1,162 @@
> +Video4Linux Version 2 (V4L2)
> +
> +General concept
> +---------------
> +
> +Video pipelines consist of external devices, e.g. camera sensors, controlled
> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> +engines and video data processors.
> +
> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> +blocks. External devices are represented as child nodes of their respective bus
> +controller nodes, e.g. I2C.
> +
> +Data interfaces on all video devices are described by "port" child DT nodes.
> +Configuration of a port depends on other devices participating in the data
> +transfer and is described by "link" DT nodes, specified as children of the
> +"port" nodes:
> +
> +/foo {
> +	port@0 {
> +		link@0 { ... };
> +		link@1 { ... };
> +	};
> +	port@1 { ... };
> +};
> +
> +If a port can be configured to work with more than one other device on the same
> +bus, a "link" child DT node must be provided for each of them. If more than one
> +port is present on a device or more than one link is connected to a port, a
> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> +used.
> +
> +Optional link properties:
> +- remote: phandle to the other endpoint link DT node.
> +- slave-mode: a boolean property, run the link in slave mode. Default is master
> +  mode.
> +- data-shift: on parallel data busses, if data-width is used to specify the
> +  number of data lines, data-shift can be used to specify which data lines are
> +  used, e.g. "data-width=<10>; data-shift=<2>;" means, that lines 9:2 are used.
> +- hsync-active: 1 or 0 for active-high or -low HSYNC signal polarity
> +  respectively.
> +- vsync-active: ditto for VSYNC. Note, that if HSYNC and VSYNC polarities are
> +  not specified, embedded synchronisation may be required, where supported.
> +- data-active: similar to HSYNC and VSYNC specifies data line polarity.
> +- field-even-active: field signal level during the even field data transmission.
> +- pclk-sample: rising (1) or falling (0) edge to sample the pixel clock pin.
> +- data-lanes: array of serial, e.g. MIPI CSI-2, data hardware lane numbers in
> +  the ascending order, beginning with logical lane 0.
> +- clock-lanes: hardware lane number, used for the clock lane.

It is not very clear we are using common contiguous range of logical indexes
for the clock and the data lanes, IMO. Maybe something like this would be
more explicit:

- data-lanes: an array of hardware data lane indexes. Position of an entry 
  determines logical lane number, while the value of an entry indicates hardware
  lane number, e.g. for 2-lane MIPI CSI-2 bus we could have data-lanes = <1>, <2>;,
  assuming the clock lane is on hardware lane 0. This property is applicable to
  serial buses only (e.g. MIPI CSI-2). 

?

--

Thanks,
Sylwester

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Rob Herring]

On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> This patch adds a document, describing common V4L2 device tree bindings.
> 
> Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> ---
>  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
>  1 files changed, 162 insertions(+), 0 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> new file mode 100644
> index 0000000..b8b3f41
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> @@ -0,0 +1,162 @@
> +Video4Linux Version 2 (V4L2)

DT describes the h/w, but V4L2 is Linux specific. I think the binding
looks pretty good in terms of it is describing the h/w and not V4L2
components or settings. So in this case it's really just the name of the
file and title I have issue with.

One other comment below:

> +
> +General concept
> +---------------
> +
> +Video pipelines consist of external devices, e.g. camera sensors, controlled
> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> +engines and video data processors.
> +
> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> +blocks. External devices are represented as child nodes of their respective bus
> +controller nodes, e.g. I2C.
> +
> +Data interfaces on all video devices are described by "port" child DT nodes.
> +Configuration of a port depends on other devices participating in the data
> +transfer and is described by "link" DT nodes, specified as children of the
> +"port" nodes:
> +
> +/foo {
> +	port@0 {
> +		link@0 { ... };
> +		link@1 { ... };
> +	};
> +	port@1 { ... };
> +};
> +
> +If a port can be configured to work with more than one other device on the same
> +bus, a "link" child DT node must be provided for each of them. If more than one
> +port is present on a device or more than one link is connected to a port, a
> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> +used.
> +
> +Optional link properties:
> +- remote: phandle to the other endpoint link DT node.

This name is a little vague. Perhaps "endpoint" would be better.

Rob

> +- slave-mode: a boolean property, run the link in slave mode. Default is master
> +  mode.
> +- data-shift: on parallel data busses, if data-width is used to specify the
> +  number of data lines, data-shift can be used to specify which data lines are
> +  used, e.g. "data-width=<10>; data-shift=<2>;" means, that lines 9:2 are used.
> +- hsync-active: 1 or 0 for active-high or -low HSYNC signal polarity
> +  respectively.
> +- vsync-active: ditto for VSYNC. Note, that if HSYNC and VSYNC polarities are
> +  not specified, embedded synchronisation may be required, where supported.
> +- data-active: similar to HSYNC and VSYNC specifies data line polarity.
> +- field-even-active: field signal level during the even field data transmission.
> +- pclk-sample: rising (1) or falling (0) edge to sample the pixel clock pin.
> +- data-lanes: array of serial, e.g. MIPI CSI-2, data hardware lane numbers in
> +  the ascending order, beginning with logical lane 0.
> +- clock-lanes: hardware lane number, used for the clock lane.
> +- clock-noncontinuous: a boolean property to allow MIPI CSI-2 non-continuous
> +  clock mode.
> +
> +Example:
> +
> +	ceu0: ceu@0xfe910000 {
> +		compatible = "renesas,sh-mobile-ceu";
> +		reg = <0xfe910000 0xa0>;
> +		interrupts = <0x880>;
> +
> +		mclk: master_clock {
> +			compatible = "renesas,ceu-clock";
> +			#clock-cells = <1>;
> +			clock-frequency = <50000000>;	/* max clock frequency */
> +			clock-output-names = "mclk";
> +		};
> +
> +		port {
> +			#address-cells = <1>;
> +			#size-cells = <0>;
> +
> +			ceu0_1: link@1 {
> +				reg = <1>;		/* local link # */
> +				remote = <&ov772x_1_1>;	/* remote phandle */
> +				bus-width = <8>;	/* used data lines */
> +				data-shift = <0>;	/* lines 7:0 are used */
> +
> +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
> +				hsync-active = <1>;	/* active high */
> +				vsync-active = <1>;	/* active high */
> +				data-active = <1>;	/* active high */
> +				pclk-sample = <1>;	/* rising */
> +			};
> +
> +			ceu0_0: link@0 {
> +				reg = <0>;
> +				remote = <&csi2_2>;
> +				immutable;
> +			};
> +		};
> +	};
> +
> +	i2c0: i2c@0xfff20000 {
> +		...
> +		ov772x_1: camera@0x21 {
> +			compatible = "omnivision,ov772x";
> +			reg = <0x21>;
> +			vddio-supply = <&regulator1>;
> +			vddcore-supply = <&regulator2>;
> +
> +			clock-frequency = <20000000>;
> +			clocks = <&mclk 0>;
> +			clock-names = "xclk";
> +
> +			port {
> +				/* With 1 link per port no need in addresses */
> +				ov772x_1_1: link {
> +					bus-width = <8>;
> +					remote = <&ceu0_1>;
> +					hsync-active = <1>;
> +					vsync-active = <0>;	/* who came up with an inverter here?... */
> +					data-active = <1>;
> +					pclk-sample = <1>;
> +				};
> +			};
> +		};
> +
> +		imx074: camera@0x1a {
> +			compatible = "sony,imx074";
> +			reg = <0x1a>;
> +			vddio-supply = <&regulator1>;
> +			vddcore-supply = <&regulator2>;
> +
> +			clock-frequency = <30000000>;	/* shared clock with ov772x_1 */
> +			clocks = <&mclk 0>;
> +			clock-names = "sysclk";		/* assuming this is the name in the datasheet */
> +
> +			port {
> +				imx074_1: link {
> +					clock-lanes = <0>;
> +					data-lanes = <1>, <2>;
> +					remote = <&csi2_1>;
> +				};
> +			};
> +		};
> +	};
> +
> +	csi2: csi2@0xffc90000 {
> +		compatible = "renesas,sh-mobile-csi2";
> +		reg = <0xffc90000 0x1000>;
> +		interrupts = <0x17a0>;
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +
> +		port@1 {
> +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
> +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
> +
> +			csi2_1: link {
> +				clock-lanes = <0>;
> +				data-lanes = <2>, <1>;
> +				remote = <&imx074_1>;
> +			};
> +		};
> +		port@2 {
> +			reg = <2>;			/* port 2: link to the CEU */
> +
> +			csi2_2: link {
> +				immutable;
> +				remote = <&ceu0_0>;
> +			};
> +		};
> +	};
> 

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

Hi Rob

On Tue, 2 Oct 2012, Rob Herring wrote:

> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> > This patch adds a document, describing common V4L2 device tree bindings.
> > 
> > Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > ---
> >  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
> >  1 files changed, 162 insertions(+), 0 deletions(-)
> >  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> > new file mode 100644
> > index 0000000..b8b3f41
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> > @@ -0,0 +1,162 @@
> > +Video4Linux Version 2 (V4L2)
> 
> DT describes the h/w, but V4L2 is Linux specific. I think the binding
> looks pretty good in terms of it is describing the h/w and not V4L2
> components or settings. So in this case it's really just the name of the
> file and title I have issue with.

Hm, I see your point, then, I guess, you'd also like the file name 
changed. What should we use then? Just "video?" But there's already a 
whole directory Documentation/devicetree/bindings/video dedicated to 
graphics output (drm, fbdev). "video-camera" or "video-capture?" But this 
file shall also be describing video output. Use "video.txt" and describe 
inside what exactly this file is for?

> 
> One other comment below:
> 
> > +
> > +General concept
> > +---------------
> > +
> > +Video pipelines consist of external devices, e.g. camera sensors, controlled
> > +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> > +engines and video data processors.
> > +
> > +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> > +blocks. External devices are represented as child nodes of their respective bus
> > +controller nodes, e.g. I2C.
> > +
> > +Data interfaces on all video devices are described by "port" child DT nodes.
> > +Configuration of a port depends on other devices participating in the data
> > +transfer and is described by "link" DT nodes, specified as children of the
> > +"port" nodes:
> > +
> > +/foo {
> > +	port@0 {
> > +		link@0 { ... };
> > +		link@1 { ... };
> > +	};
> > +	port@1 { ... };
> > +};
> > +
> > +If a port can be configured to work with more than one other device on the same
> > +bus, a "link" child DT node must be provided for each of them. If more than one
> > +port is present on a device or more than one link is connected to a port, a
> > +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> > +used.
> > +
> > +Optional link properties:
> > +- remote: phandle to the other endpoint link DT node.
> 
> This name is a little vague. Perhaps "endpoint" would be better.

"endpoint" can also refer to something local like in USB case. Maybe 
rather the description of the "remote" property should be improved?

Thanks
Guennadi

> 
> Rob
> 
> > +- slave-mode: a boolean property, run the link in slave mode. Default is master
> > +  mode.
> > +- data-shift: on parallel data busses, if data-width is used to specify the
> > +  number of data lines, data-shift can be used to specify which data lines are
> > +  used, e.g. "data-width=<10>; data-shift=<2>;" means, that lines 9:2 are used.
> > +- hsync-active: 1 or 0 for active-high or -low HSYNC signal polarity
> > +  respectively.
> > +- vsync-active: ditto for VSYNC. Note, that if HSYNC and VSYNC polarities are
> > +  not specified, embedded synchronisation may be required, where supported.
> > +- data-active: similar to HSYNC and VSYNC specifies data line polarity.
> > +- field-even-active: field signal level during the even field data transmission.
> > +- pclk-sample: rising (1) or falling (0) edge to sample the pixel clock pin.
> > +- data-lanes: array of serial, e.g. MIPI CSI-2, data hardware lane numbers in
> > +  the ascending order, beginning with logical lane 0.
> > +- clock-lanes: hardware lane number, used for the clock lane.
> > +- clock-noncontinuous: a boolean property to allow MIPI CSI-2 non-continuous
> > +  clock mode.
> > +
> > +Example:
> > +
> > +	ceu0: ceu@0xfe910000 {
> > +		compatible = "renesas,sh-mobile-ceu";
> > +		reg = <0xfe910000 0xa0>;
> > +		interrupts = <0x880>;
> > +
> > +		mclk: master_clock {
> > +			compatible = "renesas,ceu-clock";
> > +			#clock-cells = <1>;
> > +			clock-frequency = <50000000>;	/* max clock frequency */
> > +			clock-output-names = "mclk";
> > +		};
> > +
> > +		port {
> > +			#address-cells = <1>;
> > +			#size-cells = <0>;
> > +
> > +			ceu0_1: link@1 {
> > +				reg = <1>;		/* local link # */
> > +				remote = <&ov772x_1_1>;	/* remote phandle */
> > +				bus-width = <8>;	/* used data lines */
> > +				data-shift = <0>;	/* lines 7:0 are used */
> > +
> > +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
> > +				hsync-active = <1>;	/* active high */
> > +				vsync-active = <1>;	/* active high */
> > +				data-active = <1>;	/* active high */
> > +				pclk-sample = <1>;	/* rising */
> > +			};
> > +
> > +			ceu0_0: link@0 {
> > +				reg = <0>;
> > +				remote = <&csi2_2>;
> > +				immutable;
> > +			};
> > +		};
> > +	};
> > +
> > +	i2c0: i2c@0xfff20000 {
> > +		...
> > +		ov772x_1: camera@0x21 {
> > +			compatible = "omnivision,ov772x";
> > +			reg = <0x21>;
> > +			vddio-supply = <&regulator1>;
> > +			vddcore-supply = <&regulator2>;
> > +
> > +			clock-frequency = <20000000>;
> > +			clocks = <&mclk 0>;
> > +			clock-names = "xclk";
> > +
> > +			port {
> > +				/* With 1 link per port no need in addresses */
> > +				ov772x_1_1: link {
> > +					bus-width = <8>;
> > +					remote = <&ceu0_1>;
> > +					hsync-active = <1>;
> > +					vsync-active = <0>;	/* who came up with an inverter here?... */
> > +					data-active = <1>;
> > +					pclk-sample = <1>;
> > +				};
> > +			};
> > +		};
> > +
> > +		imx074: camera@0x1a {
> > +			compatible = "sony,imx074";
> > +			reg = <0x1a>;
> > +			vddio-supply = <&regulator1>;
> > +			vddcore-supply = <&regulator2>;
> > +
> > +			clock-frequency = <30000000>;	/* shared clock with ov772x_1 */
> > +			clocks = <&mclk 0>;
> > +			clock-names = "sysclk";		/* assuming this is the name in the datasheet */
> > +
> > +			port {
> > +				imx074_1: link {
> > +					clock-lanes = <0>;
> > +					data-lanes = <1>, <2>;
> > +					remote = <&csi2_1>;
> > +				};
> > +			};
> > +		};
> > +	};
> > +
> > +	csi2: csi2@0xffc90000 {
> > +		compatible = "renesas,sh-mobile-csi2";
> > +		reg = <0xffc90000 0x1000>;
> > +		interrupts = <0x17a0>;
> > +		#address-cells = <1>;
> > +		#size-cells = <0>;
> > +
> > +		port@1 {
> > +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
> > +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
> > +
> > +			csi2_1: link {
> > +				clock-lanes = <0>;
> > +				data-lanes = <2>, <1>;
> > +				remote = <&imx074_1>;
> > +			};
> > +		};
> > +		port@2 {
> > +			reg = <2>;			/* port 2: link to the CEU */
> > +
> > +			csi2_2: link {
> > +				immutable;
> > +				remote = <&ceu0_0>;
> > +			};
> > +		};
> > +	};
> > 
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Rob Herring]

On 10/02/2012 09:33 AM, Guennadi Liakhovetski wrote:
> Hi Rob
> 
> On Tue, 2 Oct 2012, Rob Herring wrote:
> 
>> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
>>> This patch adds a document, describing common V4L2 device tree bindings.
>>>
>>> Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
>>> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
>>> ---
>>>  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
>>>  1 files changed, 162 insertions(+), 0 deletions(-)
>>>  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
>>>
>>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
>>> new file mode 100644
>>> index 0000000..b8b3f41
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/media/v4l2.txt
>>> @@ -0,0 +1,162 @@
>>> +Video4Linux Version 2 (V4L2)
>>
>> DT describes the h/w, but V4L2 is Linux specific. I think the binding
>> looks pretty good in terms of it is describing the h/w and not V4L2
>> components or settings. So in this case it's really just the name of the
>> file and title I have issue with.
> 
> Hm, I see your point, then, I guess, you'd also like the file name 
> changed. What should we use then? Just "video?" But there's already a 
> whole directory Documentation/devicetree/bindings/video dedicated to 
> graphics output (drm, fbdev). "video-camera" or "video-capture?" But this 
> file shall also be describing video output. Use "video.txt" and describe 
> inside what exactly this file is for?

Video output will probably have a lot of overlap with the graphics side.
How about video-interfaces.txt?

> 
>>
>> One other comment below:
>>
>>> +
>>> +General concept
>>> +---------------
>>> +
>>> +Video pipelines consist of external devices, e.g. camera sensors, controlled
>>> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
>>> +engines and video data processors.
>>> +
>>> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
>>> +blocks. External devices are represented as child nodes of their respective bus
>>> +controller nodes, e.g. I2C.
>>> +
>>> +Data interfaces on all video devices are described by "port" child DT nodes.
>>> +Configuration of a port depends on other devices participating in the data
>>> +transfer and is described by "link" DT nodes, specified as children of the
>>> +"port" nodes:
>>> +
>>> +/foo {
>>> +	port@0 {
>>> +		link@0 { ... };
>>> +		link@1 { ... };
>>> +	};
>>> +	port@1 { ... };
>>> +};
>>> +
>>> +If a port can be configured to work with more than one other device on the same
>>> +bus, a "link" child DT node must be provided for each of them. If more than one
>>> +port is present on a device or more than one link is connected to a port, a
>>> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
>>> +used.
>>> +
>>> +Optional link properties:
>>> +- remote: phandle to the other endpoint link DT node.
>>
>> This name is a little vague. Perhaps "endpoint" would be better.
> 
> "endpoint" can also refer to something local like in USB case. Maybe 
> rather the description of the "remote" property should be improved?
> 

remote-endpoint?

> Thanks
> Guennadi
> 
>>
>> Rob
>>
>>> +- slave-mode: a boolean property, run the link in slave mode. Default is master
>>> +  mode.
>>> +- data-shift: on parallel data busses, if data-width is used to specify the
>>> +  number of data lines, data-shift can be used to specify which data lines are
>>> +  used, e.g. "data-width=<10>; data-shift=<2>;" means, that lines 9:2 are used.
>>> +- hsync-active: 1 or 0 for active-high or -low HSYNC signal polarity
>>> +  respectively.
>>> +- vsync-active: ditto for VSYNC. Note, that if HSYNC and VSYNC polarities are
>>> +  not specified, embedded synchronisation may be required, where supported.
>>> +- data-active: similar to HSYNC and VSYNC specifies data line polarity.
>>> +- field-even-active: field signal level during the even field data transmission.
>>> +- pclk-sample: rising (1) or falling (0) edge to sample the pixel clock pin.
>>> +- data-lanes: array of serial, e.g. MIPI CSI-2, data hardware lane numbers in
>>> +  the ascending order, beginning with logical lane 0.
>>> +- clock-lanes: hardware lane number, used for the clock lane.
>>> +- clock-noncontinuous: a boolean property to allow MIPI CSI-2 non-continuous
>>> +  clock mode.
>>> +
>>> +Example:
>>> +
>>> +	ceu0: ceu@0xfe910000 {
>>> +		compatible = "renesas,sh-mobile-ceu";
>>> +		reg = <0xfe910000 0xa0>;
>>> +		interrupts = <0x880>;
>>> +
>>> +		mclk: master_clock {
>>> +			compatible = "renesas,ceu-clock";
>>> +			#clock-cells = <1>;
>>> +			clock-frequency = <50000000>;	/* max clock frequency */
>>> +			clock-output-names = "mclk";
>>> +		};
>>> +
>>> +		port {
>>> +			#address-cells = <1>;
>>> +			#size-cells = <0>;
>>> +
>>> +			ceu0_1: link@1 {
>>> +				reg = <1>;		/* local link # */
>>> +				remote = <&ov772x_1_1>;	/* remote phandle */
>>> +				bus-width = <8>;	/* used data lines */
>>> +				data-shift = <0>;	/* lines 7:0 are used */
>>> +
>>> +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
>>> +				hsync-active = <1>;	/* active high */
>>> +				vsync-active = <1>;	/* active high */
>>> +				data-active = <1>;	/* active high */
>>> +				pclk-sample = <1>;	/* rising */
>>> +			};
>>> +
>>> +			ceu0_0: link@0 {
>>> +				reg = <0>;
>>> +				remote = <&csi2_2>;
>>> +				immutable;
>>> +			};
>>> +		};
>>> +	};
>>> +
>>> +	i2c0: i2c@0xfff20000 {
>>> +		...
>>> +		ov772x_1: camera@0x21 {
>>> +			compatible = "omnivision,ov772x";
>>> +			reg = <0x21>;
>>> +			vddio-supply = <&regulator1>;
>>> +			vddcore-supply = <&regulator2>;
>>> +
>>> +			clock-frequency = <20000000>;
>>> +			clocks = <&mclk 0>;
>>> +			clock-names = "xclk";
>>> +
>>> +			port {
>>> +				/* With 1 link per port no need in addresses */
>>> +				ov772x_1_1: link {
>>> +					bus-width = <8>;
>>> +					remote = <&ceu0_1>;
>>> +					hsync-active = <1>;
>>> +					vsync-active = <0>;	/* who came up with an inverter here?... */
>>> +					data-active = <1>;
>>> +					pclk-sample = <1>;
>>> +				};
>>> +			};
>>> +		};
>>> +
>>> +		imx074: camera@0x1a {
>>> +			compatible = "sony,imx074";
>>> +			reg = <0x1a>;
>>> +			vddio-supply = <&regulator1>;
>>> +			vddcore-supply = <&regulator2>;
>>> +
>>> +			clock-frequency = <30000000>;	/* shared clock with ov772x_1 */
>>> +			clocks = <&mclk 0>;
>>> +			clock-names = "sysclk";		/* assuming this is the name in the datasheet */
>>> +
>>> +			port {
>>> +				imx074_1: link {
>>> +					clock-lanes = <0>;
>>> +					data-lanes = <1>, <2>;
>>> +					remote = <&csi2_1>;
>>> +				};
>>> +			};
>>> +		};
>>> +	};
>>> +
>>> +	csi2: csi2@0xffc90000 {
>>> +		compatible = "renesas,sh-mobile-csi2";
>>> +		reg = <0xffc90000 0x1000>;
>>> +		interrupts = <0x17a0>;
>>> +		#address-cells = <1>;
>>> +		#size-cells = <0>;
>>> +
>>> +		port@1 {
>>> +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
>>> +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
>>> +
>>> +			csi2_1: link {
>>> +				clock-lanes = <0>;
>>> +				data-lanes = <2>, <1>;
>>> +				remote = <&imx074_1>;
>>> +			};
>>> +		};
>>> +		port@2 {
>>> +			reg = <2>;			/* port 2: link to the CEU */
>>> +
>>> +			csi2_2: link {
>>> +				immutable;
>>> +				remote = <&ceu0_0>;
>>> +			};
>>> +		};
>>> +	};
>>>
>>
> 
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/
> 

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

On Wed, 3 Oct 2012, Rob Herring wrote:

> On 10/02/2012 09:33 AM, Guennadi Liakhovetski wrote:
> > Hi Rob
> > 
> > On Tue, 2 Oct 2012, Rob Herring wrote:
> > 
> >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> >>> This patch adds a document, describing common V4L2 device tree bindings.
> >>>
> >>> Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> >>> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> >>> ---
> >>>  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
> >>>  1 files changed, 162 insertions(+), 0 deletions(-)
> >>>  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> >>>
> >>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> >>> new file mode 100644
> >>> index 0000000..b8b3f41
> >>> --- /dev/null
> >>> +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> >>> @@ -0,0 +1,162 @@
> >>> +Video4Linux Version 2 (V4L2)
> >>
> >> DT describes the h/w, but V4L2 is Linux specific. I think the binding
> >> looks pretty good in terms of it is describing the h/w and not V4L2
> >> components or settings. So in this case it's really just the name of the
> >> file and title I have issue with.
> > 
> > Hm, I see your point, then, I guess, you'd also like the file name 
> > changed. What should we use then? Just "video?" But there's already a 
> > whole directory Documentation/devicetree/bindings/video dedicated to 
> > graphics output (drm, fbdev). "video-camera" or "video-capture?" But this 
> > file shall also be describing video output. Use "video.txt" and describe 
> > inside what exactly this file is for?
> 
> Video output will probably have a lot of overlap with the graphics side.
> How about video-interfaces.txt?

Hm, that's a bit too vague for me. Somewhere on the outskirts of my mind 
I'm still considering making just one standard for both V4L2 and fbdev / 
DRM? Just yesterday we were discussing some common properties with what is 
being proposed in

http://www.mail-archive.com/linux-media@vger.kernel.org/index.html#53322

Still, I think, these two subsystems deserve two separate standards and 
should just try to re-use properties wherever that makes sense. 
video-stream seems a bit better, but this too is just a convention - 
talking about video cameras and TV output as video streaming devices and 
considering displays more static devices. In principle displays can be 
considered taking streaming data just as well as TV encoders. What if we 
just call this camera-tv.txt?

> >> One other comment below:
> >>
> >>> +
> >>> +General concept
> >>> +---------------
> >>> +
> >>> +Video pipelines consist of external devices, e.g. camera sensors, controlled
> >>> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> >>> +engines and video data processors.
> >>> +
> >>> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> >>> +blocks. External devices are represented as child nodes of their respective bus
> >>> +controller nodes, e.g. I2C.
> >>> +
> >>> +Data interfaces on all video devices are described by "port" child DT nodes.
> >>> +Configuration of a port depends on other devices participating in the data
> >>> +transfer and is described by "link" DT nodes, specified as children of the
> >>> +"port" nodes:
> >>> +
> >>> +/foo {
> >>> +	port@0 {
> >>> +		link@0 { ... };
> >>> +		link@1 { ... };
> >>> +	};
> >>> +	port@1 { ... };
> >>> +};
> >>> +
> >>> +If a port can be configured to work with more than one other device on the same
> >>> +bus, a "link" child DT node must be provided for each of them. If more than one
> >>> +port is present on a device or more than one link is connected to a port, a
> >>> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> >>> +used.
> >>> +
> >>> +Optional link properties:
> >>> +- remote: phandle to the other endpoint link DT node.
> >>
> >> This name is a little vague. Perhaps "endpoint" would be better.
> > 
> > "endpoint" can also refer to something local like in USB case. Maybe 
> > rather the description of the "remote" property should be improved?
> 
> remote-endpoint?

Sorry, I really don't want to pull in yet another term here. We've got 
ports and links already, now you're proposing to also use "endpoind." 
Until now everyone was happy with "remote," any more opinions on this?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Hans Verkuil]

On Fri October 5 2012 11:43:27 Guennadi Liakhovetski wrote:
> On Wed, 3 Oct 2012, Rob Herring wrote:
> 
> > On 10/02/2012 09:33 AM, Guennadi Liakhovetski wrote:
> > > Hi Rob
> > > 
> > > On Tue, 2 Oct 2012, Rob Herring wrote:
> > > 
> > >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> > >>> This patch adds a document, describing common V4L2 device tree bindings.
> > >>>
> > >>> Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> > >>> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > >>> ---
> > >>>  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
> > >>>  1 files changed, 162 insertions(+), 0 deletions(-)
> > >>>  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> > >>>
> > >>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> > >>> new file mode 100644
> > >>> index 0000000..b8b3f41
> > >>> --- /dev/null
> > >>> +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> > >>> @@ -0,0 +1,162 @@
> > >>> +Video4Linux Version 2 (V4L2)
> > >>
> > >> DT describes the h/w, but V4L2 is Linux specific. I think the binding
> > >> looks pretty good in terms of it is describing the h/w and not V4L2
> > >> components or settings. So in this case it's really just the name of the
> > >> file and title I have issue with.
> > > 
> > > Hm, I see your point, then, I guess, you'd also like the file name 
> > > changed. What should we use then? Just "video?" But there's already a 
> > > whole directory Documentation/devicetree/bindings/video dedicated to 
> > > graphics output (drm, fbdev). "video-camera" or "video-capture?" But this 
> > > file shall also be describing video output. Use "video.txt" and describe 
> > > inside what exactly this file is for?
> > 
> > Video output will probably have a lot of overlap with the graphics side.
> > How about video-interfaces.txt?
> 
> Hm, that's a bit too vague for me. Somewhere on the outskirts of my mind 
> I'm still considering making just one standard for both V4L2 and fbdev / 
> DRM? Just yesterday we were discussing some common properties with what is 
> being proposed in
> 
> http://www.mail-archive.com/linux-media@vger.kernel.org/index.html#53322
> 
> Still, I think, these two subsystems deserve two separate standards and 
> should just try to re-use properties wherever that makes sense. 
> video-stream seems a bit better, but this too is just a convention - 
> talking about video cameras and TV output as video streaming devices and 
> considering displays more static devices. In principle displays can be 
> considered taking streaming data just as well as TV encoders. What if we 
> just call this camera-tv.txt?
> 
> > >> One other comment below:
> > >>
> > >>> +
> > >>> +General concept
> > >>> +---------------
> > >>> +
> > >>> +Video pipelines consist of external devices, e.g. camera sensors, controlled
> > >>> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> > >>> +engines and video data processors.
> > >>> +
> > >>> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> > >>> +blocks. External devices are represented as child nodes of their respective bus
> > >>> +controller nodes, e.g. I2C.
> > >>> +
> > >>> +Data interfaces on all video devices are described by "port" child DT nodes.
> > >>> +Configuration of a port depends on other devices participating in the data
> > >>> +transfer and is described by "link" DT nodes, specified as children of the
> > >>> +"port" nodes:
> > >>> +
> > >>> +/foo {
> > >>> +	port@0 {
> > >>> +		link@0 { ... };
> > >>> +		link@1 { ... };
> > >>> +	};
> > >>> +	port@1 { ... };
> > >>> +};
> > >>> +
> > >>> +If a port can be configured to work with more than one other device on the same
> > >>> +bus, a "link" child DT node must be provided for each of them. If more than one
> > >>> +port is present on a device or more than one link is connected to a port, a
> > >>> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> > >>> +used.
> > >>> +
> > >>> +Optional link properties:
> > >>> +- remote: phandle to the other endpoint link DT node.
> > >>
> > >> This name is a little vague. Perhaps "endpoint" would be better.
> > > 
> > > "endpoint" can also refer to something local like in USB case. Maybe 
> > > rather the description of the "remote" property should be improved?
> > 
> > remote-endpoint?
> 
> Sorry, I really don't want to pull in yet another term here. We've got 
> ports and links already, now you're proposing to also use "endpoind." 
> Until now everyone was happy with "remote," any more opinions on this?

Actually, when I was reviewing the patch series today I got confused as
well by 'remote'. What about 'remote-link'?

And v4l2_of_get_remote() can be renamed to v4l2_of_get_remote_link() which
I think is a lot clearer.

The text can be improved as well since this:

- remote: phandle to the other endpoint link DT node.

is a bit vague. How about:

- remote-link: phandle to the remote end of this link.

Regards,

	Hans

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

On Fri, 5 Oct 2012, Hans Verkuil wrote:

> On Fri October 5 2012 11:43:27 Guennadi Liakhovetski wrote:
> > On Wed, 3 Oct 2012, Rob Herring wrote:
> > 
> > > On 10/02/2012 09:33 AM, Guennadi Liakhovetski wrote:
> > > > Hi Rob
> > > > 
> > > > On Tue, 2 Oct 2012, Rob Herring wrote:
> > > > 
> > > >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:

[snip]

> > > >>> +Optional link properties:
> > > >>> +- remote: phandle to the other endpoint link DT node.
> > > >>
> > > >> This name is a little vague. Perhaps "endpoint" would be better.
> > > > 
> > > > "endpoint" can also refer to something local like in USB case. Maybe 
> > > > rather the description of the "remote" property should be improved?
> > > 
> > > remote-endpoint?
> > 
> > Sorry, I really don't want to pull in yet another term here. We've got 
> > ports and links already, now you're proposing to also use "endpoind." 
> > Until now everyone was happy with "remote," any more opinions on this?
> 
> Actually, when I was reviewing the patch series today I got confused as
> well by 'remote'. What about 'remote-link'?

Yes, I was thinking about this one too, it looks a bit clumsy, but it does 
make it clearer, what is meant.

> And v4l2_of_get_remote() can be renamed to v4l2_of_get_remote_link() which
> I think is a lot clearer.
> 
> The text can be improved as well since this:
> 
> - remote: phandle to the other endpoint link DT node.
> 
> is a bit vague. How about:
> 
> - remote-link: phandle to the remote end of this link.

Looks good to me.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Stephen Warren]

On 10/02/2012 08:33 AM, Guennadi Liakhovetski wrote:
> On Tue, 2 Oct 2012, Rob Herring wrote:
>> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
>>> This patch adds a document, describing common V4L2 device tree bindings.

>>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt

>> One other comment below:
>>
>>> +
>>> +General concept
>>> +---------------
>>> +
>>> +Video pipelines consist of external devices, e.g. camera sensors, controlled
>>> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
>>> +engines and video data processors.
>>> +
>>> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
>>> +blocks. External devices are represented as child nodes of their respective bus
>>> +controller nodes, e.g. I2C.
>>> +
>>> +Data interfaces on all video devices are described by "port" child DT nodes.
>>> +Configuration of a port depends on other devices participating in the data
>>> +transfer and is described by "link" DT nodes, specified as children of the
>>> +"port" nodes:
>>> +
>>> +/foo {
>>> +	port@0 {
>>> +		link@0 { ... };
>>> +		link@1 { ... };
>>> +	};
>>> +	port@1 { ... };
>>> +};
>>> +
>>> +If a port can be configured to work with more than one other device on the same
>>> +bus, a "link" child DT node must be provided for each of them. If more than one
>>> +port is present on a device or more than one link is connected to a port, a
>>> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
>>> +used.
>>> +
>>> +Optional link properties:
>>> +- remote: phandle to the other endpoint link DT node.
>>
>> This name is a little vague. Perhaps "endpoint" would be better.
> 
> "endpoint" can also refer to something local like in USB case. Maybe 
> rather the description of the "remote" property should be improved?

The documentation doesn't show up in all the .dts files that use it; it
might be useful to try and make the .dts file as obviously readable as
possible.

Perhaps "remote-port" or "connected-port" would be sufficiently descriptive.

(and yes, I know I'm probably bike-shedding now).

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Laurent Pinchart]

On Monday 08 October 2012 14:00:38 Stephen Warren wrote:
> On 10/02/2012 08:33 AM, Guennadi Liakhovetski wrote:
> > On Tue, 2 Oct 2012, Rob Herring wrote:
> >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> >>> This patch adds a document, describing common V4L2 device tree bindings.
> >>> 
> >>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt
> >>> b/Documentation/devicetree/bindings/media/v4l2.txt>> 
> >> One other comment below:
> >>> +
> >>> +General concept
> >>> +---------------
> >>> +
> >>> +Video pipelines consist of external devices, e.g. camera sensors,
> >>> controlled +over an I2C, SPI or UART bus, and SoC internal IP blocks,
> >>> including video DMA +engines and video data processors.
> >>> +
> >>> +SoC internal blocks are described by DT nodes, placed similarly to
> >>> other SoC +blocks. External devices are represented as child nodes of
> >>> their respective bus +controller nodes, e.g. I2C.
> >>> +
> >>> +Data interfaces on all video devices are described by "port" child DT
> >>> nodes. +Configuration of a port depends on other devices participating
> >>> in the data +transfer and is described by "link" DT nodes, specified as
> >>> children of the +"port" nodes:
> >>> +
> >>> +/foo {
> >>> +	port@0 {
> >>> +		link@0 { ... };
> >>> +		link@1 { ... };
> >>> +	};
> >>> +	port@1 { ... };
> >>> +};
> >>> +
> >>> +If a port can be configured to work with more than one other device on
> >>> the same +bus, a "link" child DT node must be provided for each of
> >>> them. If more than one +port is present on a device or more than one
> >>> link is connected to a port, a +common scheme, using "#address-cells,"
> >>> "#size-cells" and "reg" properties is +used.
> >>> +
> >>> +Optional link properties:
> >>> +- remote: phandle to the other endpoint link DT node.
> >> 
> >> This name is a little vague. Perhaps "endpoint" would be better.
> > 
> > "endpoint" can also refer to something local like in USB case. Maybe
> > rather the description of the "remote" property should be improved?
> 
> The documentation doesn't show up in all the .dts files that use it; it
> might be useful to try and make the .dts file as obviously readable as
> possible.
> 
> Perhaps "remote-port" or "connected-port" would be sufficiently descriptive.

I like remote-port better than the already proposed remote-link.

> (and yes, I know I'm probably bike-shedding now).

-- 
Regards,

Laurent Pinchart

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

On Mon, 8 Oct 2012, Laurent Pinchart wrote:

> On Monday 08 October 2012 14:00:38 Stephen Warren wrote:
> > On 10/02/2012 08:33 AM, Guennadi Liakhovetski wrote:
> > > On Tue, 2 Oct 2012, Rob Herring wrote:
> > >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> > >>> This patch adds a document, describing common V4L2 device tree bindings.
> > >>> 
> > >>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt
> > >>> b/Documentation/devicetree/bindings/media/v4l2.txt>> 
> > >> One other comment below:
> > >>> +
> > >>> +General concept
> > >>> +---------------
> > >>> +
> > >>> +Video pipelines consist of external devices, e.g. camera sensors,
> > >>> controlled +over an I2C, SPI or UART bus, and SoC internal IP blocks,
> > >>> including video DMA +engines and video data processors.
> > >>> +
> > >>> +SoC internal blocks are described by DT nodes, placed similarly to
> > >>> other SoC +blocks. External devices are represented as child nodes of
> > >>> their respective bus +controller nodes, e.g. I2C.
> > >>> +
> > >>> +Data interfaces on all video devices are described by "port" child DT
> > >>> nodes. +Configuration of a port depends on other devices participating
> > >>> in the data +transfer and is described by "link" DT nodes, specified as
> > >>> children of the +"port" nodes:
> > >>> +
> > >>> +/foo {
> > >>> +	port@0 {
> > >>> +		link@0 { ... };
> > >>> +		link@1 { ... };
> > >>> +	};
> > >>> +	port@1 { ... };
> > >>> +};
> > >>> +
> > >>> +If a port can be configured to work with more than one other device on
> > >>> the same +bus, a "link" child DT node must be provided for each of
> > >>> them. If more than one +port is present on a device or more than one
> > >>> link is connected to a port, a +common scheme, using "#address-cells,"
> > >>> "#size-cells" and "reg" properties is +used.
> > >>> +
> > >>> +Optional link properties:
> > >>> +- remote: phandle to the other endpoint link DT node.
> > >> 
> > >> This name is a little vague. Perhaps "endpoint" would be better.
> > > 
> > > "endpoint" can also refer to something local like in USB case. Maybe
> > > rather the description of the "remote" property should be improved?
> > 
> > The documentation doesn't show up in all the .dts files that use it; it
> > might be useful to try and make the .dts file as obviously readable as
> > possible.
> > 
> > Perhaps "remote-port" or "connected-port" would be sufficiently descriptive.
> 
> I like remote-port better than the already proposed remote-link.

Yes, remote-port sounds better, than remote-link, but might be more 
difficult to correlate with the fact, that the phandle value of this 
property points to a link DT node, and not to a port.

Thanks
Guennadi

> > (and yes, I know I'm probably bike-shedding now).
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Hans Verkuil]

On Mon 8 October 2012 23:14:01 Guennadi Liakhovetski wrote:
> On Mon, 8 Oct 2012, Laurent Pinchart wrote:
> 
> > On Monday 08 October 2012 14:00:38 Stephen Warren wrote:
> > > On 10/02/2012 08:33 AM, Guennadi Liakhovetski wrote:
> > > > On Tue, 2 Oct 2012, Rob Herring wrote:
> > > >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> > > >>> This patch adds a document, describing common V4L2 device tree bindings.
> > > >>> 
> > > >>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt
> > > >>> b/Documentation/devicetree/bindings/media/v4l2.txt>> 
> > > >> One other comment below:
> > > >>> +
> > > >>> +General concept
> > > >>> +---------------
> > > >>> +
> > > >>> +Video pipelines consist of external devices, e.g. camera sensors,
> > > >>> controlled +over an I2C, SPI or UART bus, and SoC internal IP blocks,
> > > >>> including video DMA +engines and video data processors.
> > > >>> +
> > > >>> +SoC internal blocks are described by DT nodes, placed similarly to
> > > >>> other SoC +blocks. External devices are represented as child nodes of
> > > >>> their respective bus +controller nodes, e.g. I2C.
> > > >>> +
> > > >>> +Data interfaces on all video devices are described by "port" child DT
> > > >>> nodes. +Configuration of a port depends on other devices participating
> > > >>> in the data +transfer and is described by "link" DT nodes, specified as
> > > >>> children of the +"port" nodes:
> > > >>> +
> > > >>> +/foo {
> > > >>> +	port@0 {
> > > >>> +		link@0 { ... };
> > > >>> +		link@1 { ... };
> > > >>> +	};
> > > >>> +	port@1 { ... };
> > > >>> +};
> > > >>> +
> > > >>> +If a port can be configured to work with more than one other device on
> > > >>> the same +bus, a "link" child DT node must be provided for each of
> > > >>> them. If more than one +port is present on a device or more than one
> > > >>> link is connected to a port, a +common scheme, using "#address-cells,"
> > > >>> "#size-cells" and "reg" properties is +used.
> > > >>> +
> > > >>> +Optional link properties:
> > > >>> +- remote: phandle to the other endpoint link DT node.
> > > >> 
> > > >> This name is a little vague. Perhaps "endpoint" would be better.
> > > > 
> > > > "endpoint" can also refer to something local like in USB case. Maybe
> > > > rather the description of the "remote" property should be improved?
> > > 
> > > The documentation doesn't show up in all the .dts files that use it; it
> > > might be useful to try and make the .dts file as obviously readable as
> > > possible.
> > > 
> > > Perhaps "remote-port" or "connected-port" would be sufficiently descriptive.
> > 
> > I like remote-port better than the already proposed remote-link.
> 
> Yes, remote-port sounds better, than remote-link, but might be more 
> difficult to correlate with the fact, that the phandle value of this 
> property points to a link DT node, and not to a port.

I first thought of remote-port as well, but it is just weird that it points to
a link node.

I seem to remember that 'link' was called 'pad' initially, but people didn't
like that due to possible confusion with other meanings of that word.

The problem with the word 'link' is that it doesn't describe a link but just
one endpoint of a link.

Is it an idea to rename 'link' to 'endpoint' and 'remote' to 'remote-endpoint'?

So a port has endpoints, and each endpoint has a remote-endpoint property.

Regards,

	Hans

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

On Tue, 9 Oct 2012, Hans Verkuil wrote:

> On Mon 8 October 2012 23:14:01 Guennadi Liakhovetski wrote:
> > On Mon, 8 Oct 2012, Laurent Pinchart wrote:
> > 
> > > On Monday 08 October 2012 14:00:38 Stephen Warren wrote:
> > > > On 10/02/2012 08:33 AM, Guennadi Liakhovetski wrote:
> > > > > On Tue, 2 Oct 2012, Rob Herring wrote:
> > > > >> On 09/27/2012 09:07 AM, Guennadi Liakhovetski wrote:
> > > > >>> This patch adds a document, describing common V4L2 device tree bindings.
> > > > >>> 
> > > > >>> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt
> > > > >>> b/Documentation/devicetree/bindings/media/v4l2.txt>> 
> > > > >> One other comment below:
> > > > >>> +
> > > > >>> +General concept
> > > > >>> +---------------
> > > > >>> +
> > > > >>> +Video pipelines consist of external devices, e.g. camera sensors,
> > > > >>> controlled +over an I2C, SPI or UART bus, and SoC internal IP blocks,
> > > > >>> including video DMA +engines and video data processors.
> > > > >>> +
> > > > >>> +SoC internal blocks are described by DT nodes, placed similarly to
> > > > >>> other SoC +blocks. External devices are represented as child nodes of
> > > > >>> their respective bus +controller nodes, e.g. I2C.
> > > > >>> +
> > > > >>> +Data interfaces on all video devices are described by "port" child DT
> > > > >>> nodes. +Configuration of a port depends on other devices participating
> > > > >>> in the data +transfer and is described by "link" DT nodes, specified as
> > > > >>> children of the +"port" nodes:
> > > > >>> +
> > > > >>> +/foo {
> > > > >>> +	port@0 {
> > > > >>> +		link@0 { ... };
> > > > >>> +		link@1 { ... };
> > > > >>> +	};
> > > > >>> +	port@1 { ... };
> > > > >>> +};
> > > > >>> +
> > > > >>> +If a port can be configured to work with more than one other device on
> > > > >>> the same +bus, a "link" child DT node must be provided for each of
> > > > >>> them. If more than one +port is present on a device or more than one
> > > > >>> link is connected to a port, a +common scheme, using "#address-cells,"
> > > > >>> "#size-cells" and "reg" properties is +used.
> > > > >>> +
> > > > >>> +Optional link properties:
> > > > >>> +- remote: phandle to the other endpoint link DT node.
> > > > >> 
> > > > >> This name is a little vague. Perhaps "endpoint" would be better.
> > > > > 
> > > > > "endpoint" can also refer to something local like in USB case. Maybe
> > > > > rather the description of the "remote" property should be improved?
> > > > 
> > > > The documentation doesn't show up in all the .dts files that use it; it
> > > > might be useful to try and make the .dts file as obviously readable as
> > > > possible.
> > > > 
> > > > Perhaps "remote-port" or "connected-port" would be sufficiently descriptive.
> > > 
> > > I like remote-port better than the already proposed remote-link.
> > 
> > Yes, remote-port sounds better, than remote-link, but might be more 
> > difficult to correlate with the fact, that the phandle value of this 
> > property points to a link DT node, and not to a port.
> 
> I first thought of remote-port as well, but it is just weird that it points to
> a link node.
> 
> I seem to remember that 'link' was called 'pad' initially, but people didn't
> like that due to possible confusion with other meanings of that word.
> 
> The problem with the word 'link' is that it doesn't describe a link but just
> one endpoint of a link.
> 
> Is it an idea to rename 'link' to 'endpoint' and 'remote' to 'remote-endpoint'?
> 
> So a port has endpoints, and each endpoint has a remote-endpoint property.

I'm fine with that.

Thanks
Guennadi

> Regards,
> 
> 	Hans

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Sascha Hauer]

Hi Guennadi,

Some comments inline.


On Thu, Sep 27, 2012 at 04:07:23PM +0200, Guennadi Liakhovetski wrote:
> This patch adds a document, describing common V4L2 device tree bindings.
> 
> Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> ---
>  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
>  1 files changed, 162 insertions(+), 0 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> new file mode 100644
> index 0000000..b8b3f41
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> @@ -0,0 +1,162 @@
> +Video4Linux Version 2 (V4L2)
> +
> +General concept
> +---------------
> +
> +Video pipelines consist of external devices, e.g. camera sensors, controlled
> +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> +engines and video data processors.
> +
> +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> +blocks. External devices are represented as child nodes of their respective bus
> +controller nodes, e.g. I2C.
> +
> +Data interfaces on all video devices are described by "port" child DT nodes.
> +Configuration of a port depends on other devices participating in the data
> +transfer and is described by "link" DT nodes, specified as children of the
> +"port" nodes:
> +
> +/foo {
> +	port@0 {
> +		link@0 { ... };
> +		link@1 { ... };
> +	};
> +	port@1 { ... };
> +};
> +
> +If a port can be configured to work with more than one other device on the same
> +bus, a "link" child DT node must be provided for each of them. If more than one
> +port is present on a device or more than one link is connected to a port, a
> +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> +used.
> +
> +Optional link properties:
> +- remote: phandle to the other endpoint link DT node.
> +- slave-mode: a boolean property, run the link in slave mode. Default is master
> +  mode.
> +- data-shift: on parallel data busses, if data-width is used to specify the
> +  number of data lines, data-shift can be used to specify which data lines are
> +  used, e.g. "data-width=<10>; data-shift=<2>;" means, that lines 9:2 are used.
> +- hsync-active: 1 or 0 for active-high or -low HSYNC signal polarity
> +  respectively.
> +- vsync-active: ditto for VSYNC. Note, that if HSYNC and VSYNC polarities are
> +  not specified, embedded synchronisation may be required, where supported.
> +- data-active: similar to HSYNC and VSYNC specifies data line polarity.
> +- field-even-active: field signal level during the even field data transmission.
> +- pclk-sample: rising (1) or falling (0) edge to sample the pixel clock pin.
> +- data-lanes: array of serial, e.g. MIPI CSI-2, data hardware lane numbers in
> +  the ascending order, beginning with logical lane 0.
> +- clock-lanes: hardware lane number, used for the clock lane.
> +- clock-noncontinuous: a boolean property to allow MIPI CSI-2 non-continuous
> +  clock mode.
> +
> +Example:
> +
> +	ceu0: ceu@0xfe910000 {
> +		compatible = "renesas,sh-mobile-ceu";
> +		reg = <0xfe910000 0xa0>;
> +		interrupts = <0x880>;
> +
> +		mclk: master_clock {
> +			compatible = "renesas,ceu-clock";
> +			#clock-cells = <1>;
> +			clock-frequency = <50000000>;	/* max clock frequency */
> +			clock-output-names = "mclk";
> +		};
> +
> +		port {
> +			#address-cells = <1>;
> +			#size-cells = <0>;
> +
> +			ceu0_1: link@1 {
> +				reg = <1>;		/* local link # */
> +				remote = <&ov772x_1_1>;	/* remote phandle */
> +				bus-width = <8>;	/* used data lines */
> +				data-shift = <0>;	/* lines 7:0 are used */
> +
> +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
> +				hsync-active = <1>;	/* active high */
> +				vsync-active = <1>;	/* active high */
> +				data-active = <1>;	/* active high */
> +				pclk-sample = <1>;	/* rising */
> +			};
> +
> +			ceu0_0: link@0 {
> +				reg = <0>;
> +				remote = <&csi2_2>;
> +				immutable;
> +			};
> +		};
> +	};
> +
> +	i2c0: i2c@0xfff20000 {
> +		...
> +		ov772x_1: camera@0x21 {
> +			compatible = "omnivision,ov772x";
> +			reg = <0x21>;
> +			vddio-supply = <&regulator1>;
> +			vddcore-supply = <&regulator2>;
> +
> +			clock-frequency = <20000000>;
> +			clocks = <&mclk 0>;
> +			clock-names = "xclk";
> +
> +			port {
> +				/* With 1 link per port no need in addresses */
> +				ov772x_1_1: link {
> +					bus-width = <8>;
> +					remote = <&ceu0_1>;
> +					hsync-active = <1>;
> +					vsync-active = <0>;	/* who came up with an inverter here?... */
> +					data-active = <1>;
> +					pclk-sample = <1>;
> +				};

I currently do not understand why these properties are both in the sensor
and in the link. What happens if they conflict? Are inverters assumed
like suggested above? I think the bus can only have a single bus-width,
why allow multiple bus widths here?


> +		reg = <0xffc90000 0x1000>;
> +		interrupts = <0x17a0>;
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +
> +		port@1 {
> +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
> +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
> +
> +			csi2_1: link {
> +				clock-lanes = <0>;
> +				data-lanes = <2>, <1>;
> +				remote = <&imx074_1>;
> +			};
> +		};
> +		port@2 {
> +			reg = <2>;			/* port 2: link to the CEU */
> +
> +			csi2_2: link {
> +				immutable;
> +				remote = <&ceu0_0>;
> +			};
> +		};

Maybe the example would be clearer if you split it up in two, one simple
case with the csi2_1 <-> imx074_1 and a more advanced with the link in
between. It took me some time until I figured out that these are two
separate camera/sensor pairs. Somehow I was looking for a multiplexer
between them.


I am not sure we really want to have these circular phandles here. I
think phandles in the direction of data flow should be enough. I mean
the devices probe in arbitrary order anyway and the SoC camera core must
keep track of the possible sensor/csi combinations anyway, so it should
be enough to make the connections in a single direction.

Sascha


-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

Hi Sascha

On Fri, 5 Oct 2012, Sascha Hauer wrote:

> Hi Guennadi,
> 
> Some comments inline.
> 
> 
> On Thu, Sep 27, 2012 at 04:07:23PM +0200, Guennadi Liakhovetski wrote:
> > This patch adds a document, describing common V4L2 device tree bindings.
> > 
> > Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > ---
> >  Documentation/devicetree/bindings/media/v4l2.txt |  162 ++++++++++++++++++++++
> >  1 files changed, 162 insertions(+), 0 deletions(-)
> >  create mode 100644 Documentation/devicetree/bindings/media/v4l2.txt
> > 
> > diff --git a/Documentation/devicetree/bindings/media/v4l2.txt b/Documentation/devicetree/bindings/media/v4l2.txt
> > new file mode 100644
> > index 0000000..b8b3f41
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/v4l2.txt
> > @@ -0,0 +1,162 @@
> > +Video4Linux Version 2 (V4L2)
> > +
> > +General concept
> > +---------------
> > +
> > +Video pipelines consist of external devices, e.g. camera sensors, controlled
> > +over an I2C, SPI or UART bus, and SoC internal IP blocks, including video DMA
> > +engines and video data processors.
> > +
> > +SoC internal blocks are described by DT nodes, placed similarly to other SoC
> > +blocks. External devices are represented as child nodes of their respective bus
> > +controller nodes, e.g. I2C.
> > +
> > +Data interfaces on all video devices are described by "port" child DT nodes.
> > +Configuration of a port depends on other devices participating in the data
> > +transfer and is described by "link" DT nodes, specified as children of the
> > +"port" nodes:
> > +
> > +/foo {
> > +	port@0 {
> > +		link@0 { ... };
> > +		link@1 { ... };
> > +	};
> > +	port@1 { ... };
> > +};
> > +
> > +If a port can be configured to work with more than one other device on the same
> > +bus, a "link" child DT node must be provided for each of them. If more than one
> > +port is present on a device or more than one link is connected to a port, a
> > +common scheme, using "#address-cells," "#size-cells" and "reg" properties is
> > +used.
> > +
> > +Optional link properties:
> > +- remote: phandle to the other endpoint link DT node.
> > +- slave-mode: a boolean property, run the link in slave mode. Default is master
> > +  mode.
> > +- data-shift: on parallel data busses, if data-width is used to specify the
> > +  number of data lines, data-shift can be used to specify which data lines are
> > +  used, e.g. "data-width=<10>; data-shift=<2>;" means, that lines 9:2 are used.
> > +- hsync-active: 1 or 0 for active-high or -low HSYNC signal polarity
> > +  respectively.
> > +- vsync-active: ditto for VSYNC. Note, that if HSYNC and VSYNC polarities are
> > +  not specified, embedded synchronisation may be required, where supported.
> > +- data-active: similar to HSYNC and VSYNC specifies data line polarity.
> > +- field-even-active: field signal level during the even field data transmission.
> > +- pclk-sample: rising (1) or falling (0) edge to sample the pixel clock pin.
> > +- data-lanes: array of serial, e.g. MIPI CSI-2, data hardware lane numbers in
> > +  the ascending order, beginning with logical lane 0.
> > +- clock-lanes: hardware lane number, used for the clock lane.
> > +- clock-noncontinuous: a boolean property to allow MIPI CSI-2 non-continuous
> > +  clock mode.
> > +
> > +Example:
> > +
> > +	ceu0: ceu@0xfe910000 {
> > +		compatible = "renesas,sh-mobile-ceu";
> > +		reg = <0xfe910000 0xa0>;
> > +		interrupts = <0x880>;
> > +
> > +		mclk: master_clock {
> > +			compatible = "renesas,ceu-clock";
> > +			#clock-cells = <1>;
> > +			clock-frequency = <50000000>;	/* max clock frequency */
> > +			clock-output-names = "mclk";
> > +		};
> > +
> > +		port {
> > +			#address-cells = <1>;
> > +			#size-cells = <0>;
> > +
> > +			ceu0_1: link@1 {
> > +				reg = <1>;		/* local link # */
> > +				remote = <&ov772x_1_1>;	/* remote phandle */
> > +				bus-width = <8>;	/* used data lines */
> > +				data-shift = <0>;	/* lines 7:0 are used */
> > +
> > +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
> > +				hsync-active = <1>;	/* active high */
> > +				vsync-active = <1>;	/* active high */
> > +				data-active = <1>;	/* active high */
> > +				pclk-sample = <1>;	/* rising */
> > +			};
> > +
> > +			ceu0_0: link@0 {
> > +				reg = <0>;
> > +				remote = <&csi2_2>;
> > +				immutable;
> > +			};
> > +		};
> > +	};
> > +
> > +	i2c0: i2c@0xfff20000 {
> > +		...
> > +		ov772x_1: camera@0x21 {
> > +			compatible = "omnivision,ov772x";
> > +			reg = <0x21>;
> > +			vddio-supply = <&regulator1>;
> > +			vddcore-supply = <&regulator2>;
> > +
> > +			clock-frequency = <20000000>;
> > +			clocks = <&mclk 0>;
> > +			clock-names = "xclk";
> > +
> > +			port {
> > +				/* With 1 link per port no need in addresses */
> > +				ov772x_1_1: link {
> > +					bus-width = <8>;
> > +					remote = <&ceu0_1>;
> > +					hsync-active = <1>;
> > +					vsync-active = <0>;	/* who came up with an inverter here?... */
> > +					data-active = <1>;
> > +					pclk-sample = <1>;
> > +				};
> 
> I currently do not understand why these properties are both in the sensor
> and in the link. What happens if they conflict? Are inverters assumed
> like suggested above? I think the bus can only have a single bus-width,
> why allow multiple bus widths here?

Yes, these nodes represent port configuration of each party on a certain 
link. And they can differ in certain properties, like - as you correctly 
notice - in the case, when there's an inverter on a line. As for other 
properties, some of them must be identical, like bus-width, still, they 
have to be provided on both ends, because generally drivers have to be 
able to perform all the required configuration based only on the 
information from their own nodes, without looking at "remote" partner node 
properties.

> > +		reg = <0xffc90000 0x1000>;
> > +		interrupts = <0x17a0>;
> > +		#address-cells = <1>;
> > +		#size-cells = <0>;
> > +
> > +		port@1 {
> > +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
> > +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
> > +
> > +			csi2_1: link {
> > +				clock-lanes = <0>;
> > +				data-lanes = <2>, <1>;
> > +				remote = <&imx074_1>;
> > +			};
> > +		};
> > +		port@2 {
> > +			reg = <2>;			/* port 2: link to the CEU */
> > +
> > +			csi2_2: link {
> > +				immutable;
> > +				remote = <&ceu0_0>;
> > +			};
> > +		};
> 
> Maybe the example would be clearer if you split it up in two, one simple
> case with the csi2_1 <-> imx074_1 and a more advanced with the link in
> between.

With no link between two ports no connection is possible, so, only 
examples with links make sense.

> It took me some time until I figured out that these are two
> separate camera/sensor pairs. Somehow I was looking for a multiplexer
> between them.

Maybe I can add more comments to the file, perhaps, add an ASCII-art 
chart.

> I am not sure we really want to have these circular phandles here.

It has been suggested and accepted during a discussion at the KS / LPC a 
month ago. The original version only had phandle referencing in one 
direction.

> I
> think phandles in the direction of data flow should be enough. I mean
> the devices probe in arbitrary order anyway and the SoC camera core must
> keep track of the possible sensor/csi combinations anyway, so it should
> be enough to make the connections in a single direction.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Sascha Hauer]

On Fri, Oct 05, 2012 at 05:41:00PM +0200, Guennadi Liakhovetski wrote:
> Hi Sascha
> 
> > > +
> > > +	ceu0: ceu@0xfe910000 {
> > > +		compatible = "renesas,sh-mobile-ceu";
> > > +		reg = <0xfe910000 0xa0>;
> > > +		interrupts = <0x880>;
> > > +
> > > +		mclk: master_clock {
> > > +			compatible = "renesas,ceu-clock";
> > > +			#clock-cells = <1>;
> > > +			clock-frequency = <50000000>;	/* max clock frequency */
> > > +			clock-output-names = "mclk";
> > > +		};
> > > +
> > > +		port {
> > > +			#address-cells = <1>;
> > > +			#size-cells = <0>;
> > > +
> > > +			ceu0_1: link@1 {
> > > +				reg = <1>;		/* local link # */
> > > +				remote = <&ov772x_1_1>;	/* remote phandle */
> > > +				bus-width = <8>;	/* used data lines */
> > > +				data-shift = <0>;	/* lines 7:0 are used */
> > > +
> > > +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
> > > +				hsync-active = <1>;	/* active high */
> > > +				vsync-active = <1>;	/* active high */
> > > +				data-active = <1>;	/* active high */
> > > +				pclk-sample = <1>;	/* rising */
> > > +			};
> > > +
> > > +			ceu0_0: link@0 {
> > > +				reg = <0>;
> > > +				remote = <&csi2_2>;
> > > +				immutable;
> > > +			};
> > > +		};
> > > +	};
> > > +
> > > +	i2c0: i2c@0xfff20000 {
> > > +		...
> > > +		ov772x_1: camera@0x21 {
> > > +			compatible = "omnivision,ov772x";
> > > +			reg = <0x21>;
> > > +			vddio-supply = <&regulator1>;
> > > +			vddcore-supply = <&regulator2>;
> > > +
> > > +			clock-frequency = <20000000>;
> > > +			clocks = <&mclk 0>;
> > > +			clock-names = "xclk";
> > > +
> > > +			port {
> > > +				/* With 1 link per port no need in addresses */
> > > +				ov772x_1_1: link {
> > > +					bus-width = <8>;
> > > +					remote = <&ceu0_1>;
> > > +					hsync-active = <1>;
> > > +					vsync-active = <0>;	/* who came up with an inverter here?... */
> > > +					data-active = <1>;
> > > +					pclk-sample = <1>;
> > > +				};
> > 
> > I currently do not understand why these properties are both in the sensor
> > and in the link. What happens if they conflict? Are inverters assumed
> > like suggested above? I think the bus can only have a single bus-width,
> > why allow multiple bus widths here?
> 
> Yes, these nodes represent port configuration of each party on a certain 
> link. And they can differ in certain properties, like - as you correctly 
> notice - in the case, when there's an inverter on a line. As for other 
> properties, some of them must be identical, like bus-width, still, they 
> have to be provided on both ends, because generally drivers have to be 
> able to perform all the required configuration based only on the 
> information from their own nodes, without looking at "remote" partner node 
> properties.

So the port associated to the ov772x_1 only describes how to configure
the ov772x and it's up to me to make sure that this configuration
matches the partner device. If I don't then it won't work but soc-camera
will happily continue.
Ok, that's good, I thought there would be some kind of matching
mechanism take place here. It may be worth to make this more explicit in
the docs.

> 
> > > +		reg = <0xffc90000 0x1000>;
> > > +		interrupts = <0x17a0>;
> > > +		#address-cells = <1>;
> > > +		#size-cells = <0>;
> > > +
> > > +		port@1 {
> > > +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
> > > +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
> > > +
> > > +			csi2_1: link {
> > > +				clock-lanes = <0>;
> > > +				data-lanes = <2>, <1>;
> > > +				remote = <&imx074_1>;
> > > +			};
> > > +		};
> > > +		port@2 {
> > > +			reg = <2>;			/* port 2: link to the CEU */
> > > +
> > > +			csi2_2: link {
> > > +				immutable;
> > > +				remote = <&ceu0_0>;
> > > +			};
> > > +		};
> > 
> > Maybe the example would be clearer if you split it up in two, one simple
> > case with the csi2_1 <-> imx074_1 and a more advanced with the link in
> > between.
> 
> With no link between two ports no connection is possible, so, only 
> examples with links make sense.

I should have said with the renesas,sh-mobile-ceu in between.

So simple example: csi2_1 <-l-> imx074_1
advanced: csi2_2 <-l-> ceu <-l-> ov772x

> 
> > It took me some time until I figured out that these are two
> > separate camera/sensor pairs. Somehow I was looking for a multiplexer
> > between them.
> 
> Maybe I can add more comments to the file, perhaps, add an ASCII-art 
> chart.

That would be good.

> 
> > I am not sure we really want to have these circular phandles here.
> 
> It has been suggested and accepted during a discussion at the KS / LPC a 
> month ago. The original version only had phandle referencing in one 
> direction.

Ok.

Sascha


-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Guennadi Liakhovetski]

On Fri, 5 Oct 2012, Sascha Hauer wrote:

> On Fri, Oct 05, 2012 at 05:41:00PM +0200, Guennadi Liakhovetski wrote:
> > Hi Sascha
> > 
> > > > +
> > > > +	ceu0: ceu@0xfe910000 {
> > > > +		compatible = "renesas,sh-mobile-ceu";
> > > > +		reg = <0xfe910000 0xa0>;
> > > > +		interrupts = <0x880>;
> > > > +
> > > > +		mclk: master_clock {
> > > > +			compatible = "renesas,ceu-clock";
> > > > +			#clock-cells = <1>;
> > > > +			clock-frequency = <50000000>;	/* max clock frequency */
> > > > +			clock-output-names = "mclk";
> > > > +		};
> > > > +
> > > > +		port {
> > > > +			#address-cells = <1>;
> > > > +			#size-cells = <0>;
> > > > +
> > > > +			ceu0_1: link@1 {
> > > > +				reg = <1>;		/* local link # */
> > > > +				remote = <&ov772x_1_1>;	/* remote phandle */
> > > > +				bus-width = <8>;	/* used data lines */
> > > > +				data-shift = <0>;	/* lines 7:0 are used */
> > > > +
> > > > +				/* If [hv]sync-active are missing, embedded bt.605 sync is used */
> > > > +				hsync-active = <1>;	/* active high */
> > > > +				vsync-active = <1>;	/* active high */
> > > > +				data-active = <1>;	/* active high */
> > > > +				pclk-sample = <1>;	/* rising */
> > > > +			};
> > > > +
> > > > +			ceu0_0: link@0 {
> > > > +				reg = <0>;
> > > > +				remote = <&csi2_2>;
> > > > +				immutable;
> > > > +			};
> > > > +		};
> > > > +	};
> > > > +
> > > > +	i2c0: i2c@0xfff20000 {
> > > > +		...
> > > > +		ov772x_1: camera@0x21 {
> > > > +			compatible = "omnivision,ov772x";
> > > > +			reg = <0x21>;
> > > > +			vddio-supply = <&regulator1>;
> > > > +			vddcore-supply = <&regulator2>;
> > > > +
> > > > +			clock-frequency = <20000000>;
> > > > +			clocks = <&mclk 0>;
> > > > +			clock-names = "xclk";
> > > > +
> > > > +			port {
> > > > +				/* With 1 link per port no need in addresses */
> > > > +				ov772x_1_1: link {
> > > > +					bus-width = <8>;
> > > > +					remote = <&ceu0_1>;
> > > > +					hsync-active = <1>;
> > > > +					vsync-active = <0>;	/* who came up with an inverter here?... */
> > > > +					data-active = <1>;
> > > > +					pclk-sample = <1>;
> > > > +				};
> > > 
> > > I currently do not understand why these properties are both in the sensor
> > > and in the link. What happens if they conflict? Are inverters assumed
> > > like suggested above? I think the bus can only have a single bus-width,
> > > why allow multiple bus widths here?
> > 
> > Yes, these nodes represent port configuration of each party on a certain 
> > link. And they can differ in certain properties, like - as you correctly 
> > notice - in the case, when there's an inverter on a line. As for other 
> > properties, some of them must be identical, like bus-width, still, they 
> > have to be provided on both ends, because generally drivers have to be 
> > able to perform all the required configuration based only on the 
> > information from their own nodes, without looking at "remote" partner node 
> > properties.
> 
> So the port associated to the ov772x_1 only describes how to configure
> the ov772x and it's up to me to make sure that this configuration
> matches the partner device. If I don't then it won't work but soc-camera
> will happily continue.
> Ok, that's good, I thought there would be some kind of matching
> mechanism take place here. It may be worth to make this more explicit in
> the docs.
> 
> > 
> > > > +		reg = <0xffc90000 0x1000>;
> > > > +		interrupts = <0x17a0>;
> > > > +		#address-cells = <1>;
> > > > +		#size-cells = <0>;
> > > > +
> > > > +		port@1 {
> > > > +			compatible = "renesas,csi2c";	/* one of CSI2I and CSI2C */
> > > > +			reg = <1>;			/* CSI-2 PHY #1 of 2: PHY_S, PHY_M has port address 0, is unused */
> > > > +
> > > > +			csi2_1: link {
> > > > +				clock-lanes = <0>;
> > > > +				data-lanes = <2>, <1>;
> > > > +				remote = <&imx074_1>;
> > > > +			};
> > > > +		};
> > > > +		port@2 {
> > > > +			reg = <2>;			/* port 2: link to the CEU */
> > > > +
> > > > +			csi2_2: link {
> > > > +				immutable;
> > > > +				remote = <&ceu0_0>;
> > > > +			};
> > > > +		};
> > > 
> > > Maybe the example would be clearer if you split it up in two, one simple
> > > case with the csi2_1 <-> imx074_1 and a more advanced with the link in
> > > between.
> > 
> > With no link between two ports no connection is possible, so, only 
> > examples with links make sense.
> 
> I should have said with the renesas,sh-mobile-ceu in between.
> 
> So simple example: csi2_1 <-l-> imx074_1
> advanced: csi2_2 <-l-> ceu <-l-> ov772x

No, CEU is the DMA engine with some image processing, so, it's always 
present. The CSI-2 is just a MIPI CSI-2 interface, that in the above case 
is used with the CEU too. So, it's like

       ,-l- ov772x
      /
ceu0 <
      \
       `-l- csi2 -l- imx074

> > > It took me some time until I figured out that these are two
> > > separate camera/sensor pairs. Somehow I was looking for a multiplexer
> > > between them.
> > 
> > Maybe I can add more comments to the file, perhaps, add an ASCII-art 
> > chart.
> 
> That would be good.

Is the above good enough? :)

Thanks
Guennadi

> > > I am not sure we really want to have these circular phandles here.
> > 
> > It has been suggested and accepted during a discussion at the KS / LPC a 
> > month ago. The original version only had phandle referencing in one 
> > direction.
> 
> Ok.
> 
> Sascha

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Sascha Hauer]

Hi Guennadi,

On Mon, Oct 08, 2012 at 09:58:58AM +0200, Guennadi Liakhovetski wrote:
> On Fri, 5 Oct 2012, Sascha Hauer wrote:
> 
> > On Fri, Oct 05, 2012 at 05:41:00PM +0200, Guennadi Liakhovetski wrote:
> > > Hi Sascha
> > > 
> > > > 
> > > > Maybe the example would be clearer if you split it up in two, one simple
> > > > case with the csi2_1 <-> imx074_1 and a more advanced with the link in
> > > > between.
> > > 
> > > With no link between two ports no connection is possible, so, only 
> > > examples with links make sense.
> > 
> > I should have said with the renesas,sh-mobile-ceu in between.
> > 
> > So simple example: csi2_1 <-l-> imx074_1
> > advanced: csi2_2 <-l-> ceu <-l-> ov772x
> 
> No, CEU is the DMA engine with some image processing, so, it's always 
> present. The CSI-2 is just a MIPI CSI-2 interface, that in the above case 
> is used with the CEU too. So, it's like
> 
>        ,-l- ov772x
>       /
> ceu0 <
>       \
>        `-l- csi2 -l- imx074
> 
> > > > It took me some time until I figured out that these are two
> > > > separate camera/sensor pairs. Somehow I was looking for a multiplexer
> > > > between them.
> > > 
> > > Maybe I can add more comments to the file, perhaps, add an ASCII-art 
> > > chart.
> > 
> > That would be good.
> 
> Is the above good enough? :)

Yes, thanks. We thought and disucssed about this devicetree binding
quite much the last days. Finally I understood it and I must say that I
like it. I think more prosa to explain the general concept would be good
in the binding doc.

Mark, when do we get the same for aSoC? ;)

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Mark Brown]

On Wed, Oct 10, 2012 at 10:40:06AM +0200, Sascha Hauer wrote:

> Mark, when do we get the same for aSoC? ;)

Well, I'm unlikely to work on it as I don't have any systems which can
boot with device tree.

The big, big problem you've got doing this is lots of dynamic changes at 
runtime and in general complicated systems.  It's trivial to describe
the physical links but they don't provide anything like enough
information to use things.  Quite frankly I'm not sure device tree is a
useful investment of time for advanced audio systems anyway, it's really
not solving any problems people actually have.

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Sascha Hauer]

On Wed, Oct 10, 2012 at 05:51:27PM +0900, Mark Brown wrote:
> On Wed, Oct 10, 2012 at 10:40:06AM +0200, Sascha Hauer wrote:
> 
> > Mark, when do we get the same for aSoC? ;)
> 
> Well, I'm unlikely to work on it as I don't have any systems which can
> boot with device tree.

If it's only that I'm sure we could spare a i.MX53 LOCO ;)

> 
> The big, big problem you've got doing this is lots of dynamic changes at 
> runtime and in general complicated systems.  It's trivial to describe
> the physical links but they don't provide anything like enough
> information to use things.  Quite frankly I'm not sure device tree is a
> useful investment of time for advanced audio systems anyway, it's really
> not solving any problems people actually have.

Right now the i.MX audmux binding is only enough to say which ports
should be connected, but not which format should be used. Just today
we had the problem where a codec has two DAIs and wanted to add the
information on which port our SSI unit is connected to the devicetree.

So I think it's worthwile to do, but that would be a big big task...

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Mark Brown]

On Wed, Oct 10, 2012 at 11:21:15AM +0200, Sascha Hauer wrote:
> On Wed, Oct 10, 2012 at 05:51:27PM +0900, Mark Brown wrote:

> > Well, I'm unlikely to work on it as I don't have any systems which can
> > boot with device tree.

> If it's only that I'm sure we could spare a i.MX53 LOCO ;)

Well, something with Wolfson hardware would be helpful.

> > The big, big problem you've got doing this is lots of dynamic changes at 
> > runtime and in general complicated systems.  It's trivial to describe
> > the physical links but they don't provide anything like enough
> > information to use things.  Quite frankly I'm not sure device tree is a
> > useful investment of time for advanced audio systems anyway, it's really
> > not solving any problems people actually have.

> Right now the i.MX audmux binding is only enough to say which ports
> should be connected, but not which format should be used. Just today

Why should that be in DT?  For most things it's either fixed by the
hardware or makes no odds.

> we had the problem where a codec has two DAIs and wanted to add the
> information on which port our SSI unit is connected to the devicetree.

There's nothing stopping you doing that right now, the existing DT 

> So I think it's worthwile to do, but that would be a big big task...

For simple devices there's already stuff there and it's not hard to add
new machine bindings, it's trying to cover the general case that's far
too much effort.

Re: [PATCH 04/14] media: add V4L2 DT binding documentation

[Stephen Warren]

On 09/27/2012 08:07 AM, Guennadi Liakhovetski wrote:
> This patch adds a document, describing common V4L2 device tree bindings.
> 
> Co-authored-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>

I think this looks reasonable now, touch wood:-) I guess I won't ack the
binding until the v4l2 -> something-generic rename happens, but I don't
see anything stopping me from doing so once the rename happens.

Re: [PATCH 00/14] V4L2 DT support

[Sylwester Nawrocki]

Hi Guennadi,

Any chance for a GIT tree including this patch series ? I'd like
to see all these pieces put together and I don't seem to find any
base tree that this series would have applied cleanly to.

...(20121005_media_for_v3.7-dt)$ git am -3 \[PATCH\ *
Applying: i2c: add dummy inline functions for when CONFIG_OF_I2C(_MODULE) isn't defined
Applying: of: add a dummy inline function for when CONFIG_OF is not defined
Applying: OF: make a function pointer argument const
Applying: media: add V4L2 DT binding documentation
Applying: media: add a V4L2 OF parser
Applying: media: soc-camera: prepare for asynchronous client probing
Applying: media: soc-camera: support deferred probing of clients
fatal: sha1 information is lacking or useless (drivers/media/platform/soc_camera/soc_camera.c).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.
Patch failed at 0007 media: soc-camera: support deferred probing of clients
When you have resolved this problem run "git am --resolved".
If you would prefer to skip this patch, instead run "git am --skip".
To restore the original branch and stop patching run "git am --abort".

--

Thanks,
Sylwester

Re: [PATCH 00/14] V4L2 DT support

[Guennadi Liakhovetski]

Hi Sylwester

On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:

> Hi Guennadi,
> 
> Any chance for a GIT tree including this patch series ? I'd like
> to see all these pieces put together and I don't seem to find any
> base tree that this series would have applied cleanly to.

Ok, I pushed the patches to

git://linuxtv.org/gliakhovetski/v4l-dvb.git dt-soc_camera

Please, give it a go.

Thanks
Guennadi

> 
> ...(20121005_media_for_v3.7-dt)$ git am -3 \[PATCH\ *
> Applying: i2c: add dummy inline functions for when CONFIG_OF_I2C(_MODULE) isn't defined
> Applying: of: add a dummy inline function for when CONFIG_OF is not defined
> Applying: OF: make a function pointer argument const
> Applying: media: add V4L2 DT binding documentation
> Applying: media: add a V4L2 OF parser
> Applying: media: soc-camera: prepare for asynchronous client probing
> Applying: media: soc-camera: support deferred probing of clients
> fatal: sha1 information is lacking or useless (drivers/media/platform/soc_camera/soc_camera.c).
> Repository lacks necessary blobs to fall back on 3-way merge.
> Cannot fall back to three-way merge.
> Patch failed at 0007 media: soc-camera: support deferred probing of clients
> When you have resolved this problem run "git am --resolved".
> If you would prefer to skip this patch, instead run "git am --skip".
> To restore the original branch and stop patching run "git am --abort".
> 
> --
> 
> Thanks,
> Sylwester
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 10/14] media: soc-camera: support OF cameras

[Sylwester Nawrocki]

On 09/27/2012 04:07 PM, Guennadi Liakhovetski wrote:
> With OF we aren't getting platform data any more. To minimise changes we
> create all the missing data ourselves, including compulsory struct
> soc_camera_link objects. Host-client linking is now done, based on the OF
> data. Media bus numbers also have to be assigned dynamically.
>
> Signed-off-by: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> ---
...
>   static int soc_camera_i2c_notify(struct notifier_block *nb,
>   				 unsigned long action, void *data)
>   {
> @@ -1203,13 +1434,20 @@ static int soc_camera_i2c_notify(struct notifier_block *nb,
>   	struct v4l2_subdev *subdev;
>   	int ret;
>
> -	if (client->addr != icl->board_info->addr ||
> -	    client->adapter->nr != icl->i2c_adapter_id)
> +	dev_dbg(dev, "%s(%lu): %x on %u\n", __func__, action,
> +		client->addr, client->adapter->nr);
> +
> +	if (!soc_camera_i2c_client_match(icl, client))
>   		return NOTIFY_DONE;
>
>   	switch (action) {
>   	case BUS_NOTIFY_BIND_DRIVER:
>   		client->dev.platform_data = icl;
> +		if (icl->of_link) {
> +			struct soc_camera_of_client *sofc = container_of(icl->of_link,
> +						struct soc_camera_of_client, of_link);
> +			soc_camera_of_i2c_ifill(sofc, client);
> +		}
>
>   		return NOTIFY_OK;
>   	case BUS_NOTIFY_BOUND_DRIVER:

There is no need for different handling of this event as well ?
Further, there is code like:

	adap = i2c_get_adapter(icl->i2c_adapter_id);

which is clearly not going to work in OF case. Could you clarify how
it is supposed to work ?

--

Thanks,
Sylwester

Re: [PATCH 10/14] media: soc-camera: support OF cameras

[Guennadi Liakhovetski]

Hi Sylwester

On Fri, 5 Oct 2012, Sylwester Nawrocki wrote:

> On 09/27/2012 04:07 PM, Guennadi Liakhovetski wrote:
> > With OF we aren't getting platform data any more. To minimise changes we
> > create all the missing data ourselves, including compulsory struct
> > soc_camera_link objects. Host-client linking is now done, based on the OF
> > data. Media bus numbers also have to be assigned dynamically.
> > 
> > Signed-off-by: Guennadi Liakhovetski<g.liakhovetski@gmx.de>
> > ---
> ...
> >   static int soc_camera_i2c_notify(struct notifier_block *nb,
> >   				 unsigned long action, void *data)
> >   {
> > @@ -1203,13 +1434,20 @@ static int soc_camera_i2c_notify(struct
> > notifier_block *nb,
> >   	struct v4l2_subdev *subdev;
> >   	int ret;
> > 
> > -	if (client->addr != icl->board_info->addr ||
> > -	    client->adapter->nr != icl->i2c_adapter_id)
> > +	dev_dbg(dev, "%s(%lu): %x on %u\n", __func__, action,
> > +		client->addr, client->adapter->nr);
> > +
> > +	if (!soc_camera_i2c_client_match(icl, client))
> >   		return NOTIFY_DONE;
> > 
> >   	switch (action) {
> >   	case BUS_NOTIFY_BIND_DRIVER:
> >   		client->dev.platform_data = icl;
> > +		if (icl->of_link) {
> > +			struct soc_camera_of_client *sofc =
> > container_of(icl->of_link,
> > +						struct soc_camera_of_client,
> > of_link);
> > +			soc_camera_of_i2c_ifill(sofc, client);
> > +		}
> > 
> >   		return NOTIFY_OK;
> >   	case BUS_NOTIFY_BOUND_DRIVER:
> 
> There is no need for different handling of this event as well ?

There is. The former is entered before the sensor I2C probe method is 
called and prepares the data for probing, the latter is entered after a 
successful sensor I2C probing.

> Further, there is code like:
> 
> 	adap = i2c_get_adapter(icl->i2c_adapter_id);
> 
> which is clearly not going to work in OF case.

It does work. See the call to soc_camera_of_i2c_ifill() under 
BUS_NOTIFY_BIND_DRIVER above. In it

	icl->i2c_adapter_id = client->adapter->nr;

> Could you clarify how it is supposed to work ?

It is not only supposed to work, it actually does work. Does the above 
explain it sufficiently?

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 10/14] media: soc-camera: support OF cameras

[Sylwester Nawrocki]

Hi Guennadi,

On 10/08/2012 10:37 AM, Guennadi Liakhovetski wrote:
>>>   	case BUS_NOTIFY_BOUND_DRIVER:
>>
>> There is no need for different handling of this event as well ?
> 
> There is. The former is entered before the sensor I2C probe method is 
> called and prepares the data for probing, the latter is entered after a 
> successful sensor I2C probing.
> 
>> Further, there is code like:
>>
>> 	adap = i2c_get_adapter(icl->i2c_adapter_id);
>>
>> which is clearly not going to work in OF case.
> 
> It does work. See the call to soc_camera_of_i2c_ifill() under 
> BUS_NOTIFY_BIND_DRIVER above. In it
> 
> 	icl->i2c_adapter_id = client->adapter->nr;
> 
>> Could you clarify how it is supposed to work ?
> 
> It is not only supposed to work, it actually does work. Does the above 
> explain it sufficiently?

Sorry, my fault. Somehow I didn't realize there is being passed
an I2C adapter already assigned by the I2C core. I confused it with
some static adapter nr, which would be -1 for dt case.
Apologies for wasting your time with those non-constructive questions.

--

Thanks,
Sylwester

Re: [PATCH 10/14] media: soc-camera: support OF cameras

[Barry Song]

hi Guennadi,

2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> With OF we aren't getting platform data any more. To minimise changes we
> create all the missing data ourselves, including compulsory struct
> soc_camera_link objects. Host-client linking is now done, based on the OF
> data. Media bus numbers also have to be assigned dynamically.
>
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>

as your V4L2 core DT based supports
[media] Add a V4L2 OF parser
[media] Add common video interfaces OF bindings documentation
have been in media_tree queue for 3.10. i do care about the status of
this patch for soc_camera.

will you have a plan to resend these soc-camera patches based on your
final V4L2 core DT patches? otherwise, we might do some jobs for that.

> ---
>  drivers/media/platform/soc_camera/soc_camera.c |  337 ++++++++++++++++++++++--
>  include/media/soc_camera.h                     |    5 +
>  2 files changed, 326 insertions(+), 16 deletions(-)
>
-barry

Re: [PATCH 10/14] media: soc-camera: support OF cameras

[Guennadi Liakhovetski]

Hi Barry

On Mon, 8 Apr 2013, Barry Song wrote:

> hi Guennadi,
> 
> 2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> > With OF we aren't getting platform data any more. To minimise changes we
> > create all the missing data ourselves, including compulsory struct
> > soc_camera_link objects. Host-client linking is now done, based on the OF
> > data. Media bus numbers also have to be assigned dynamically.
> >
> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> 
> as your V4L2 core DT based supports
> [media] Add a V4L2 OF parser
> [media] Add common video interfaces OF bindings documentation
> have been in media_tree queue for 3.10. i do care about the status of
> this patch for soc_camera.
> 
> will you have a plan to resend these soc-camera patches based on your
> final V4L2 core DT patches? otherwise, we might do some jobs for that.

This patch depends not only on the above two OF patches, that Sylwester 
kindly polished to make them finally suitable for the mainline :-), but 
also on V4L2 (temporary) clock and asynchronous probing patches, which 
I've just re-posted in their v7. Once those patches stabilise, it will be 
possible to re-spin this patch too. However, currently I don't have any 
interested users for this work, so I can only run it as a low priority 
task in my spare time, of which I don't have much.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 10/14] media: soc-camera: support OF cameras

[Barry Song]

Hi Guennadi,

2013/4/8 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> Hi Barry
>
> On Mon, 8 Apr 2013, Barry Song wrote:
>
>> hi Guennadi,
>>
>> 2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> > With OF we aren't getting platform data any more. To minimise changes we
>> > create all the missing data ourselves, including compulsory struct
>> > soc_camera_link objects. Host-client linking is now done, based on the OF
>> > data. Media bus numbers also have to be assigned dynamically.
>> >
>> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
>>
>> as your V4L2 core DT based supports
>> [media] Add a V4L2 OF parser
>> [media] Add common video interfaces OF bindings documentation
>> have been in media_tree queue for 3.10. i do care about the status of
>> this patch for soc_camera.
>>
>> will you have a plan to resend these soc-camera patches based on your
>> final V4L2 core DT patches? otherwise, we might do some jobs for that.
>
> This patch depends not only on the above two OF patches, that Sylwester
> kindly polished to make them finally suitable for the mainline :-), but
> also on V4L2 (temporary) clock and asynchronous probing patches, which
> I've just re-posted in their v7. Once those patches stabilise, it will be
> possible to re-spin this patch too. However, currently I don't have any
> interested users for this work, so I can only run it as a low priority
> task in my spare time, of which I don't have much.

as we are considering putting CSR SiRFprimaII/SiRFatlas6/SiRFmarco
soc_camera based drivers in drivers/media/platform/soc_camera, so we
might become your real user. Old imx, omap users might have lost the
loop.
I will take a careful at all these patches and find the way to make
the things happen faster.

>
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/

-barry

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Barry Song]

Hi Guennadia,

2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> Currently soc-camera doesn't work with independently registered I2C client
> devices, it has to register them itself. This patch adds support for such
> configurations, in which case client drivers have to request deferred
> probing until their host becomes available and enables the data interface.
>
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> ---

it seems deferred probing for i2c camera sensors is a more workaround
than a solution.
currently,  soc-camera-pdrv is the manager of the whole initilization
flow. it all requires the host/client registerred and initilized
synchronously. so that results in strange things like that we fill a
i2c_board_info structure in arch/arm/mach-xxx but we never call
anything like i2c_new_device() to add the i2c client in mach. because
we need to do that in the soc-camera-pdrv driver to make all things
happen orderly.

but now after we move to DT, all i2c device will be registerred
automatically by of_i2c_register_devices() in i2c_host 's probe, that
makes the problem much worse and very urgent to get fixed.

returning DEFERRED_PROBE error until getting the private data filled
by the manager, indirectly, makes the things seem to be asynchronous,
but essentially it is still synchronous because the overall timing
line is still controller by soc-camera-pdrv.

what about another possible way:
we let all host and i2c client driver probed in any order, but let the
final soc-camera-pdrv is the connection of all things. the situation
of soc_camera is very similar with ALSA SoC. it turns out ASoC has
done that very well.

-barry

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Guennadi Liakhovetski]

Hi Barry

On Wed, 10 Apr 2013, Barry Song wrote:

> Hi Guennadia,
> 
> 2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> > Currently soc-camera doesn't work with independently registered I2C client
> > devices, it has to register them itself. This patch adds support for such
> > configurations, in which case client drivers have to request deferred
> > probing until their host becomes available and enables the data interface.
> >
> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> > ---
> 
> it seems deferred probing for i2c camera sensors is a more workaround
> than a solution.
> currently,  soc-camera-pdrv is the manager of the whole initilization
> flow. it all requires the host/client registerred and initilized
> synchronously. so that results in strange things like that we fill a
> i2c_board_info structure in arch/arm/mach-xxx but we never call
> anything like i2c_new_device() to add the i2c client in mach. because
> we need to do that in the soc-camera-pdrv driver to make all things
> happen orderly.
> 
> but now after we move to DT, all i2c device will be registerred
> automatically by of_i2c_register_devices() in i2c_host 's probe, that
> makes the problem much worse and very urgent to get fixed.
> 
> returning DEFERRED_PROBE error until getting the private data filled
> by the manager,

This hasn't been the case since several versions of these patches. We no 
longer use private data to decide whether subdevices can probe 
successfully or have to defer probing.

> indirectly, makes the things seem to be asynchronous,
> but essentially it is still synchronous because the overall timing
> line is still controller by soc-camera-pdrv.

It isn't. If your subdevice driver doesn't have any dependencies, like 
e.g. sh_mobile_csi2.c, it will probe asynchronously whenever it's loaded. 
It is the task of a bridge driver, in our case of the soc-camera core, to 
register notifiers and a list of expected subdevices with the v4l2-async 
subsystem. As subdevices complete their probing they signal that to the 
v4l2-async too, which then calls bridge's notifiers, which then can build 
the pipeline.

> what about another possible way:
> we let all host and i2c client driver probed in any order,

This cannot work, because some I2C devices, e.g. sensors, need a clock 
signal from the camera interface to probe. Before the bridge driver has 
completed its probing and registered a suitable clock source with the 
v4l2-clk framework, sensors cannot be probed. And no, we don't want to 
fake successful probing without actually being able to talk to the 
hardware.

Thanks
Guennadi

> but let the
> final soc-camera-pdrv is the connection of all things. the situation
> of soc_camera is very similar with ALSA SoC. it turns out ASoC has
> done that very well.
> 
> -barry
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Barry Song]

Hi Guennadia,
Thanks!

2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> Hi Barry
>
> On Wed, 10 Apr 2013, Barry Song wrote:
>
>> Hi Guennadia,
>>
>> 2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> > Currently soc-camera doesn't work with independently registered I2C client
>> > devices, it has to register them itself. This patch adds support for such
>> > configurations, in which case client drivers have to request deferred
>> > probing until their host becomes available and enables the data interface.
>> >
>> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
>> > ---
>>
>> it seems deferred probing for i2c camera sensors is a more workaround
>> than a solution.
>> currently,  soc-camera-pdrv is the manager of the whole initilization
>> flow. it all requires the host/client registerred and initilized
>> synchronously. so that results in strange things like that we fill a
>> i2c_board_info structure in arch/arm/mach-xxx but we never call
>> anything like i2c_new_device() to add the i2c client in mach. because
>> we need to do that in the soc-camera-pdrv driver to make all things
>> happen orderly.
>>
>> but now after we move to DT, all i2c device will be registerred
>> automatically by of_i2c_register_devices() in i2c_host 's probe, that
>> makes the problem much worse and very urgent to get fixed.
>>
>> returning DEFERRED_PROBE error until getting the private data filled
>> by the manager,
>
> This hasn't been the case since several versions of these patches. We no
> longer use private data to decide whether subdevices can probe
> successfully or have to defer probing.

sorry for missing.  i will refer to your newer versions.

>
>> indirectly, makes the things seem to be asynchronous,
>> but essentially it is still synchronous because the overall timing
>> line is still controller by soc-camera-pdrv.
>
> It isn't. If your subdevice driver doesn't have any dependencies, like
> e.g. sh_mobile_csi2.c, it will probe asynchronously whenever it's loaded.
> It is the task of a bridge driver, in our case of the soc-camera core, to
> register notifiers and a list of expected subdevices with the v4l2-async
> subsystem. As subdevices complete their probing they signal that to the
> v4l2-async too, which then calls bridge's notifiers, which then can build
> the pipeline.

it seems we didn't describle my idea clearly in the last mail. i
actually mean we don't need that if we put the pipeline building to
the last stage after all things have been placed there.

>
>> what about another possible way:
>> we let all host and i2c client driver probed in any order,
>
> This cannot work, because some I2C devices, e.g. sensors, need a clock
> signal from the camera interface to probe. Before the bridge driver has
> completed its probing and registered a suitable clock source with the
> v4l2-clk framework, sensors cannot be probed. And no, we don't want to
> fake successful probing without actually being able to talk to the
> hardware.

i'd say same dependency also exists on ASoC.  a "fake" successful
probing doesn't mean it should really begin to work if there is no
external trigger source.  ASoC has successfully done that by a machine
driver to connect all DAI.
a way is we put all things ready in their places, finally we connect
them together and launch the whole hardware flow.

anyway, if you have maken the things work by some simple hacking and
that means minimial changes to current soc-camera, i think we can
follow.

>
> Thanks
> Guennadi
>
>> but let the
>> final soc-camera-pdrv is the connection of all things. the situation
>> of soc_camera is very similar with ALSA SoC. it turns out ASoC has
>> done that very well.
>>

-barry

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1248 bytes --]

On Wed, Apr 10, 2013 at 09:53:20PM +0800, Barry Song wrote:
> 2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:

> >> what about another possible way:
> >> we let all host and i2c client driver probed in any order,

> > This cannot work, because some I2C devices, e.g. sensors, need a clock
> > signal from the camera interface to probe. Before the bridge driver has
> > completed its probing and registered a suitable clock source with the
> > v4l2-clk framework, sensors cannot be probed. And no, we don't want to
> > fake successful probing without actually being able to talk to the
> > hardware.

> i'd say same dependency also exists on ASoC.  a "fake" successful
> probing doesn't mean it should really begin to work if there is no
> external trigger source.  ASoC has successfully done that by a machine
> driver to connect all DAI.
> a way is we put all things ready in their places, finally we connect
> them together and launch the whole hardware flow.

In the ASoC case the idea is that drivers should probe as far as they
can with just the chip and then register with the core.  The machine
driver defers probing until all components have probed and then runs
through second stage initialisaton which pulls everything together.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Barry Song]

2013/4/10 Mark Brown <broonie@opensource.wolfsonmicro.com>:
> On Wed, Apr 10, 2013 at 09:53:20PM +0800, Barry Song wrote:
>> 2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>
>> >> what about another possible way:
>> >> we let all host and i2c client driver probed in any order,
>
>> > This cannot work, because some I2C devices, e.g. sensors, need a clock
>> > signal from the camera interface to probe. Before the bridge driver has
>> > completed its probing and registered a suitable clock source with the
>> > v4l2-clk framework, sensors cannot be probed. And no, we don't want to
>> > fake successful probing without actually being able to talk to the
>> > hardware.
>
>> i'd say same dependency also exists on ASoC.  a "fake" successful
>> probing doesn't mean it should really begin to work if there is no
>> external trigger source.  ASoC has successfully done that by a machine
>> driver to connect all DAI.
>> a way is we put all things ready in their places, finally we connect
>> them together and launch the whole hardware flow.
>
> In the ASoC case the idea is that drivers should probe as far as they
> can with just the chip and then register with the core.  The machine
> driver defers probing until all components have probed and then runs
> through second stage initialisaton which pulls everything together.

yes. thanks for clarification, Mark. that is really what i want in
soc-camera too.
put all things in their places, and the final connector wait for
everyone and put them in the initialized status.

-barry

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Guennadi Liakhovetski]

On Wed, 10 Apr 2013, Barry Song wrote:

> Hi Guennadia,

There's a typo above.

> Thanks!
> 
> 2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> > Hi Barry
> >
> > On Wed, 10 Apr 2013, Barry Song wrote:
> >
> >> Hi Guennadia,
> >>
> >> 2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> >> > Currently soc-camera doesn't work with independently registered I2C client
> >> > devices, it has to register them itself. This patch adds support for such
> >> > configurations, in which case client drivers have to request deferred
> >> > probing until their host becomes available and enables the data interface.
> >> >
> >> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> >> > ---
> >>
> >> it seems deferred probing for i2c camera sensors is a more workaround
> >> than a solution.
> >> currently,  soc-camera-pdrv is the manager of the whole initilization
> >> flow. it all requires the host/client registerred and initilized
> >> synchronously. so that results in strange things like that we fill a
> >> i2c_board_info structure in arch/arm/mach-xxx but we never call
> >> anything like i2c_new_device() to add the i2c client in mach. because
> >> we need to do that in the soc-camera-pdrv driver to make all things
> >> happen orderly.
> >>
> >> but now after we move to DT, all i2c device will be registerred
> >> automatically by of_i2c_register_devices() in i2c_host 's probe, that
> >> makes the problem much worse and very urgent to get fixed.
> >>
> >> returning DEFERRED_PROBE error until getting the private data filled
> >> by the manager,
> >
> > This hasn't been the case since several versions of these patches. We no
> > longer use private data to decide whether subdevices can probe
> > successfully or have to defer probing.
> 
> sorry for missing.  i will refer to your newer versions.
> 
> >
> >> indirectly, makes the things seem to be asynchronous,
> >> but essentially it is still synchronous because the overall timing
> >> line is still controller by soc-camera-pdrv.
> >
> > It isn't. If your subdevice driver doesn't have any dependencies, like
> > e.g. sh_mobile_csi2.c, it will probe asynchronously whenever it's loaded.
> > It is the task of a bridge driver, in our case of the soc-camera core, to
> > register notifiers and a list of expected subdevices with the v4l2-async
> > subsystem. As subdevices complete their probing they signal that to the
> > v4l2-async too, which then calls bridge's notifiers, which then can build
> > the pipeline.
> 
> it seems we didn't describle my idea clearly in the last mail. i
> actually mean we don't need that if we put the pipeline building to
> the last stage after all things have been placed there.
> 
> >
> >> what about another possible way:
> >> we let all host and i2c client driver probed in any order,
> >
> > This cannot work, because some I2C devices, e.g. sensors, need a clock
> > signal from the camera interface to probe. Before the bridge driver has
> > completed its probing and registered a suitable clock source with the
> > v4l2-clk framework, sensors cannot be probed. And no, we don't want to
> > fake successful probing without actually being able to talk to the
> > hardware.
> 
> i'd say same dependency also exists on ASoC.  a "fake" successful
> probing doesn't mean it should really begin to work if there is no
> external trigger source.  ASoC has successfully done that by a machine
> driver to connect all DAI.
> a way is we put all things ready in their places, finally we connect
> them together and launch the whole hardware flow.
> 
> anyway, if you have maken the things work by some simple hacking and
> that means minimial changes to current soc-camera, i think we can
> follow.

If you want to volunteer to step up as a new soc-camera maintainer to 
replace my simple hacking with your comprehencive and advanced designs - 
feel free, I'll ack straight away.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Barry Song]

2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> On Wed, 10 Apr 2013, Barry Song wrote:
>
>> Hi Guennadi,
>
> There's a typo above.

sorry for the typo.

>
>> Thanks!
>>
>> 2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> > Hi Barry
>> >
>> > On Wed, 10 Apr 2013, Barry Song wrote:
>> >
>> >> Hi Guennadia,
>> >>
>> >> 2012/9/27 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> >> > Currently soc-camera doesn't work with independently registered I2C client
>> >> > devices, it has to register them itself. This patch adds support for such
>> >> > configurations, in which case client drivers have to request deferred
>> >> > probing until their host becomes available and enables the data interface.
>> >> >
>> >> > Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
>> >> > ---
>> >>
>> >> it seems deferred probing for i2c camera sensors is a more workaround
>> >> than a solution.
>> >> currently,  soc-camera-pdrv is the manager of the whole initilization
>> >> flow. it all requires the host/client registerred and initilized
>> >> synchronously. so that results in strange things like that we fill a
>> >> i2c_board_info structure in arch/arm/mach-xxx but we never call
>> >> anything like i2c_new_device() to add the i2c client in mach. because
>> >> we need to do that in the soc-camera-pdrv driver to make all things
>> >> happen orderly.
>> >>
>> >> but now after we move to DT, all i2c device will be registerred
>> >> automatically by of_i2c_register_devices() in i2c_host 's probe, that
>> >> makes the problem much worse and very urgent to get fixed.
>> >>
>> >> returning DEFERRED_PROBE error until getting the private data filled
>> >> by the manager,
>> >
>> > This hasn't been the case since several versions of these patches. We no
>> > longer use private data to decide whether subdevices can probe
>> > successfully or have to defer probing.
>>
>> sorry for missing.  i will refer to your newer versions.
>>
>> >
>> >> indirectly, makes the things seem to be asynchronous,
>> >> but essentially it is still synchronous because the overall timing
>> >> line is still controller by soc-camera-pdrv.
>> >
>> > It isn't. If your subdevice driver doesn't have any dependencies, like
>> > e.g. sh_mobile_csi2.c, it will probe asynchronously whenever it's loaded.
>> > It is the task of a bridge driver, in our case of the soc-camera core, to
>> > register notifiers and a list of expected subdevices with the v4l2-async
>> > subsystem. As subdevices complete their probing they signal that to the
>> > v4l2-async too, which then calls bridge's notifiers, which then can build
>> > the pipeline.
>>
>> it seems we didn't describle my idea clearly in the last mail. i
>> actually mean we don't need that if we put the pipeline building to
>> the last stage after all things have been placed there.
>>
>> >
>> >> what about another possible way:
>> >> we let all host and i2c client driver probed in any order,
>> >
>> > This cannot work, because some I2C devices, e.g. sensors, need a clock
>> > signal from the camera interface to probe. Before the bridge driver has
>> > completed its probing and registered a suitable clock source with the
>> > v4l2-clk framework, sensors cannot be probed. And no, we don't want to
>> > fake successful probing without actually being able to talk to the
>> > hardware.
>>
>> i'd say same dependency also exists on ASoC.  a "fake" successful
>> probing doesn't mean it should really begin to work if there is no
>> external trigger source.  ASoC has successfully done that by a machine
>> driver to connect all DAI.
>> a way is we put all things ready in their places, finally we connect
>> them together and launch the whole hardware flow.
>>
>> anyway, if you have maken the things work by some simple hacking and
>> that means minimial changes to current soc-camera, i think we can
>> follow.
>
> If you want to volunteer to step up as a new soc-camera maintainer to
> replace my simple hacking with your comprehencive and advanced designs -
> feel free, I'll ack straight away.

i am not sure whether you agree the new way or not. if you also agree
this is a better way, i think we can do something to move ahead. i
need sync and get input from you expert :-)

>
> Thanks
> Guennadi

-barry

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Guennadi Liakhovetski]

On Wed, 10 Apr 2013, Barry Song wrote:

> 2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> > On Wed, 10 Apr 2013, Barry Song wrote:

[snip]

> >> > This cannot work, because some I2C devices, e.g. sensors, need a clock
> >> > signal from the camera interface to probe. Before the bridge driver has
> >> > completed its probing and registered a suitable clock source with the
> >> > v4l2-clk framework, sensors cannot be probed. And no, we don't want to
> >> > fake successful probing without actually being able to talk to the
> >> > hardware.
> >>
> >> i'd say same dependency also exists on ASoC.  a "fake" successful
> >> probing doesn't mean it should really begin to work if there is no
> >> external trigger source.  ASoC has successfully done that by a machine
> >> driver to connect all DAI.
> >> a way is we put all things ready in their places, finally we connect
> >> them together and launch the whole hardware flow.
> >>
> >> anyway, if you have maken the things work by some simple hacking and
> >> that means minimial changes to current soc-camera, i think we can
> >> follow.
> >
> > If you want to volunteer to step up as a new soc-camera maintainer to
> > replace my simple hacking with your comprehencive and advanced designs -
> > feel free, I'll ack straight away.
> 
> i am not sure whether you agree the new way or not. if you also agree
> this is a better way,

In fact I don't.

> i think we can do something to move ahead. i
> need sync and get input from you expert :-)

I suggest you read all the mailing list discussions of these topics over 
last months / years, conference discussion protocols instead of restarting 
a beaten to death topic at the v8 time-frame.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 07/14] media: soc-camera: support deferred probing of clients

[Barry Song]

2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
> On Wed, 10 Apr 2013, Barry Song wrote:
>
>> 2013/4/10 Guennadi Liakhovetski <g.liakhovetski@gmx.de>:
>> > On Wed, 10 Apr 2013, Barry Song wrote:
>
> [snip]
>
>> >> > This cannot work, because some I2C devices, e.g. sensors, need a clock
>> >> > signal from the camera interface to probe. Before the bridge driver has
>> >> > completed its probing and registered a suitable clock source with the
>> >> > v4l2-clk framework, sensors cannot be probed. And no, we don't want to
>> >> > fake successful probing without actually being able to talk to the
>> >> > hardware.
>> >>
>> >> i'd say same dependency also exists on ASoC.  a "fake" successful
>> >> probing doesn't mean it should really begin to work if there is no
>> >> external trigger source.  ASoC has successfully done that by a machine
>> >> driver to connect all DAI.
>> >> a way is we put all things ready in their places, finally we connect
>> >> them together and launch the whole hardware flow.
>> >>
>> >> anyway, if you have maken the things work by some simple hacking and
>> >> that means minimial changes to current soc-camera, i think we can
>> >> follow.
>> >
>> > If you want to volunteer to step up as a new soc-camera maintainer to
>> > replace my simple hacking with your comprehencive and advanced designs -
>> > feel free, I'll ack straight away.
>>
>> i am not sure whether you agree the new way or not. if you also agree
>> this is a better way,
>
> In fact I don't.
>
>> i think we can do something to move ahead. i
>> need sync and get input from you expert :-)
>
> I suggest you read all the mailing list discussions of these topics over
> last months / years, conference discussion protocols instead of restarting
> a beaten to death topic at the v8 time-frame.

you think it has been dead but i think it is still alive since there
are still sombody like me starting a beaten for that.
anyway,  it doesn't really matter too much.
i can definitely follow what you like as you have succefully built a
good soc-camera system.

>
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski, Ph.D.
> Freelance Open-Source Software Developer
> http://www.open-technology.de/

-barry