OMAP 3 ISP and N900 sensor driver update

OMAP 3 ISP and N900 sensor driver update

[Sakari Ailus]

Hi,

I have updated the OMAP 3 ISP driver in Gitorious:

<URL:http://www.gitorious.org/omap3camera>

Major changes since the last update:

- The Nokia N900 (aka rx-51) sensor drivers are available (will be 
posted to the list shortly)
- Say goodbye to v4l2-int-device, welcome the v4l2_subdevice interface 
(thanks to Laurent Pinchart)
- Miscellaneous stability fixes and cleanups
- H3A rework (by David Cohen)
- Resizer rework (by Antti Koskipää)

The next task is then the moving to Media controller, I guess.

Cheers,

-- 
Sakari Ailus
sakari.ailus@maxwell.research.nokia.com

RE: OMAP 3 ISP and N900 sensor driver update

[Hiremath, Vaibhav]

> -----Original Message-----
> From: linux-media-owner@vger.kernel.org [mailto:linux-media-
> owner@vger.kernel.org] On Behalf Of Sakari Ailus
> Sent: Friday, November 06, 2009 6:22 PM
> To: linux-media@vger.kernel.org
> Cc: Laurent Pinchart; Hans Verkuil; Cohen David Abraham; Koskipää
> Antti Jussi Petteri; Toivonen Tuukka Olli Artturi; Zutshi Vimarsh
> (Nokia-D-MSW/Helsinki); talvala@stanford.edu; Aguirre, Sergio; Ivan
> Ivanov; Stan Varbanov; Valeri Ivanov; Atanas Filipov
> Subject: OMAP 3 ISP and N900 sensor driver update
> 
> Hi,
> 
> I have updated the OMAP 3 ISP driver in Gitorious:
> 
> <URL:http://www.gitorious.org/omap3camera>
> 
> Major changes since the last update:
> 
> - The Nokia N900 (aka rx-51) sensor drivers are available (will be
> posted to the list shortly)
> - Say goodbye to v4l2-int-device, welcome the v4l2_subdevice
> interface
> (thanks to Laurent Pinchart)
[Hiremath, Vaibhav] Thanks Sakari and Laurent, this will definitely unblock me.

Thanks,
Vaibhav

> - Miscellaneous stability fixes and cleanups
> - H3A rework (by David Cohen)
> - Resizer rework (by Antti Koskipää)
> 
> The next task is then the moving to Media controller, I guess.
> 
> Cheers,
> 
> --
> Sakari Ailus
> sakari.ailus@maxwell.research.nokia.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-
> media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: OMAP 3 ISP and N900 sensor driver update

[Karicheri, Muralidharan]

Sakari,

Thanks for the update...

Who is working on the CCDC driver for OMAP35xx? After a week or so,
I need to start migrating the CCDC driver to sub device interface so that application can directly configure the parameters with out having to
go through video node. Ultimately it is expected that ccdc will have a
device node that will allow application to open the device and configure
the parameters (in the context of Media controller). But to begin with
I intend to port the existing CCDC driver for DM6446 and DM355 to
sub device interface. Since the VPFE IPs are common across DM6446 &
OMAP 35xx, we could use a common sub device across both platforms.

So I this context, could you please update me on the CCDC development
on OMAP platform that you work?

Murali Karicheri
Software Design Engineer
Texas Instruments Inc.
Germantown, MD 20874
email: m-karicheri2@ti.com

>-----Original Message-----
>From: linux-media-owner@vger.kernel.org [mailto:linux-media-
>owner@vger.kernel.org] On Behalf Of Sakari Ailus
>Sent: Friday, November 06, 2009 7:52 AM
>To: linux-media@vger.kernel.org
>Cc: Laurent Pinchart; Hans Verkuil; Cohen David Abraham; Koskipää Antti
>Jussi Petteri; Toivonen Tuukka Olli Artturi; Zutshi Vimarsh (Nokia-D-
>MSW/Helsinki); talvala@stanford.edu; Aguirre, Sergio; Ivan Ivanov; Stan
>Varbanov; Valeri Ivanov; Atanas Filipov
>Subject: OMAP 3 ISP and N900 sensor driver update
>
>Hi,
>
>I have updated the OMAP 3 ISP driver in Gitorious:
>
><URL:http://www.gitorious.org/omap3camera>
>
>Major changes since the last update:
>
>- The Nokia N900 (aka rx-51) sensor drivers are available (will be
>posted to the list shortly)
>- Say goodbye to v4l2-int-device, welcome the v4l2_subdevice interface
>(thanks to Laurent Pinchart)
>- Miscellaneous stability fixes and cleanups
>- H3A rework (by David Cohen)
>- Resizer rework (by Antti Koskipää)
>
>The next task is then the moving to Media controller, I guess.
>
>Cheers,
>
>--
>Sakari Ailus
>sakari.ailus@maxwell.research.nokia.com
>--
>To unsubscribe from this list: send the line "unsubscribe linux-media" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: OMAP 3 ISP and N900 sensor driver update

[Aguirre, Sergio]

Sakari, 

> -----Original Message-----
> From: Sakari Ailus [mailto:sakari.ailus@maxwell.research.nokia.com] 
> Sent: Friday, November 06, 2009 6:52 AM
> To: linux-media@vger.kernel.org
> Cc: Laurent Pinchart; Hans Verkuil; Cohen David Abraham; 
> Koskipää Antti Jussi Petteri; Toivonen Tuukka Olli Artturi; 
> Zutshi Vimarsh (Nokia-D-MSW/Helsinki); talvala@stanford.edu; 
> Aguirre, Sergio; Ivan Ivanov; Stan Varbanov; Valeri Ivanov; 
> Atanas Filipov
> Subject: OMAP 3 ISP and N900 sensor driver update
> 
> Hi,
> 
> I have updated the OMAP 3 ISP driver in Gitorious:
> 
> <URL:http://www.gitorious.org/omap3camera>
> 
> Major changes since the last update:
> 
> - The Nokia N900 (aka rx-51) sensor drivers are available (will be 
> posted to the list shortly)
> - Say goodbye to v4l2-int-device, welcome the v4l2_subdevice 
> interface 
> (thanks to Laurent Pinchart)
> - Miscellaneous stability fixes and cleanups
> - H3A rework (by David Cohen)
> - Resizer rework (by Antti Koskipää)

Does the v4l2_subdevice conversion had some functional regressions?

Or everything is in place still?

Regards,
Sergio

> 
> The next task is then the moving to Media controller, I guess.
> 
> Cheers,
> 
> -- 
> Sakari Ailus
> sakari.ailus@maxwell.research.nokia.com
> 
> 

Re: OMAP 3 ISP and N900 sensor driver update

[Sakari Ailus]

Aguirre, Sergio wrote:
> Sakari, 

Hi, Sergio!

>> - The Nokia N900 (aka rx-51) sensor drivers are available (will be 
>> posted to the list shortly)
>> - Say goodbye to v4l2-int-device, welcome the v4l2_subdevice 
>> interface 
>> (thanks to Laurent Pinchart)
>> - Miscellaneous stability fixes and cleanups
>> - H3A rework (by David Cohen)
>> - Resizer rework (by Antti Koskipää)
> 
> Does the v4l2_subdevice conversion had some functional regressions?
> 
> Or everything is in place still?

I'd guess more or less so. I'm not aware of any regressions at least. :)

-- 
Sakari Ailus
sakari.ailus@maxwell.research.nokia.com

Re: OMAP 3 ISP and N900 sensor driver update

[Laurent Pinchart]

Hi Murali,

On Friday 06 November 2009 16:25:01 Karicheri, Muralidharan wrote:
> Sakari,
> 
> Thanks for the update...
> 
> Who is working on the CCDC driver for OMAP35xx?

Just for the sake of correctness, we're working on an OMAP34xx, not an 
OMAP35xx. I don't think that makes a big difference though.

As far as I know nobody on our side is currently working on the CCDC driver. 
We're focusing on the previewer and resizer first.

> After a week or so, I need to start migrating the CCDC driver to sub device
> interface so that application can directly configure the parameters with out
> having to go through video node. Ultimately it is expected that ccdc will
> have a device node that will allow application to open the device and
> configure the parameters (in the context of Media controller). But to begin
> with I intend to port the existing CCDC driver for DM6446 and DM355 to sub
> device interface. Since the VPFE IPs are common across DM6446 & OMAP 35xx,
> we could use a common sub device across both platforms.

Coordinating our efforts on that front would indeed be very nice.

> So I this context, could you please update me on the CCDC development
> on OMAP platform that you work?

I haven't checked the Davinci VPFE drivers recently. I suppose they already 
use the v4l2_subdev interface for their I2C sensors and tuners. If not, that 
would be the first step.

On the OMAP34xx platform, the ISP driver is already somehow separated from the 
omap34xxcam driver, although not nicely. In a nutshell, here's the current 
plan. Parts 1 and 2 are already implemented and code is available in Sakari's 
linux-omap tree.

1. Board code registers an omap34xxcam platform device. The platform data 
contains an array of v4l2_subdev_i2c_board_info structures filled with 
information about all I2C sub-devices (sensor, flash controller, lens 
controller, ...).

2. The omap34xxcam driver is loaded. Its probe function is called with a 
pointer to the platform device. The driver registers a v4l2_device and creates 
I2C subdevices using the data supplied through platform data.

3. The omap34xxcam driver calls the ISP core with a pointer to the v4l2_device 
structure to register all ISP subdevices. The ISP core maintains pointers to 
the ISP subdevices.

As ISP submodules (CCDC, previewer, resizer) are not truly independent, we 
were not planning to split them into separate kernel modules. The ISP core 
needs to call explicitly into the submodules for instance to dispatch 
interrupts.

It should be possible to use a single CCDC code base across multiple 
platforms. The ISP core module would depend on the CCDC module directly. I'm 
not sure how the CCDC module should be called though. An omap prefix won't 
work, as it's used on Davinci as well, and an ISP prefix is too generic. Do 
you have any internal code name for the ISP device used on Davinci and OMAP 
platforms ?

The code will be pushed to linux-omap when available, but I can't commit to 
any deadline. If you start working on the Davinci driver before we post 
anything, could you please post patches and/or RFCs on the linux-media mailing 
list (and CC'ing us) ? Confronting our ideas as soon as possible will 
(hopefully) avoid diverging too much in our implementations.

-- 
Regards,

Laurent Pinchart

RE: OMAP 3 ISP and N900 sensor driver update

[Karicheri, Muralidharan]

Hi Laurent,

I have also copied Vaibhav from TI in this email since this is 
of interesting to his area of work as well.

Murali Karicheri
Software Design Engineer
Texas Instruments Inc.
Germantown, MD 20874
email: m-karicheri2@ti.com

>>
>> Who is working on the CCDC driver for OMAP35xx?
>
>Just for the sake of correctness, we're working on an OMAP34xx, not an
>OMAP35xx. I don't think that makes a big difference though.
>

That is right!

>As far as I know nobody on our side is currently working on the CCDC driver.
>We're focusing on the previewer and resizer first.
>
>> After a week or so, I need to start migrating the CCDC driver to sub
>device
>> interface so that application can directly configure the parameters with
>out
>> having to go through video node. Ultimately it is expected that ccdc will
>> have a device node that will allow application to open the device and
>> configure the parameters (in the context of Media controller). But to
>begin
>> with I intend to port the existing CCDC driver for DM6446 and DM355 to
>sub
>> device interface. Since the VPFE IPs are common across DM6446 & OMAP 35xx,
>> we could use a common sub device across both platforms.
>
>Coordinating our efforts on that front would indeed be very nice.
>

Ok.

>> So I this context, could you please update me on the CCDC development
>> on OMAP platform that you work?
>
>I haven't checked the Davinci VPFE drivers recently. I suppose they already
>use the v4l2_subdev interface for their I2C sensors and tuners. If not,
>that
>would be the first step.



Yes.

The vpfe drivers defines all sub device configuration data in the platform
specific code and use that information to load up sub devices. Example
tvp514x is loaded by vpfe capture driver and also mt9t031.


>
>On the OMAP34xx platform, the ISP driver is already somehow separated from
>the
>omap34xxcam driver, although not nicely. In a nutshell, here's the current
>plan. Parts 1 and 2 are already implemented and code is available in
>Sakari's
>linux-omap tree.
>


I will check this out some time next week as I am currently tied up with
internal release deadline.


>1. Board code registers an omap34xxcam platform device. The platform data
>contains an array of v4l2_subdev_i2c_board_info structures filled with
>information about all I2C sub-devices (sensor, flash controller, lens
>controller, ...).
>

vpfe capture does similar things.


>2. The omap34xxcam driver is loaded. Its probe function is called with a
>pointer to the platform device. The driver registers a v4l2_device and
>creates
>I2C subdevices using the data supplied through platform data.
>

vpfe capture does similar things.. The vpfe capture is already part of the
v4l2 sub system. You can find the code under /drivers/media/video/davinci

>3. The omap34xxcam driver calls the ISP core with a pointer to the
>v4l2_device
>structure to register all ISP subdevices. The ISP core maintains pointers
>to
>the ISP subdevices.


I was thinking in similar lines. We could define configuration for the ISP sub devices in the platform code. For example, register IO space for each IP (Preview Engine, Resizer, H3A etc) can be configured at platform level. But how is the sub device operations invoked from the camera driver? Are there wrapper functions in ISP core for each of the operations? 

>
>As ISP submodules (CCDC, previewer, resizer) are not truly independent, we
>were not planning to split them into separate kernel modules. The ISP core
>needs to call explicitly into the submodules for instance to dispatch
>interrupts.
>

IMO, to re-use, we need to define common operations across all of the
ISP sub modules (ccdc ops, resizer ops, preview-ops etc) since at hight
level hardware provide similar functions . I am assuming ISP core is a glue layer that load up all of the ISP sub devices and provide function calls to set format, get format, etc, right? Currently vpfe capture is interfaced
to ccdc modules through an interface (ccdc_hw_if.h). I am planning to
convert it to a sub device. In that case, ccdc along with other sub modules
will be loaded up by a vpfe core similar to ISP core. So by standardizing on
common operations, we could re-use the IP (sub device) across VPFE and OMAP. 
I will investigate the OMAP tree that Sakari maintains before starting my
development. Also the Resizer and Preview Engine will have to be used for
memory to memory operation as well. So re-use from that perspective is needed as well. There was an RFC for this from Samsung recently. Did you
have a chance to review this?

>It should be possible to use a single CCDC code base across multiple
>platforms. The ISP core module would depend on the CCDC module directly.
>I'm
>not sure how the CCDC module should be called though. An omap prefix won't
>work, as it's used on Davinci as well, and an ISP prefix is too generic. Do
>you have any internal code name for the ISP device used on Davinci and OMAP
>platforms ?

The ISP is called VPFE in DaVinci. In our internal release, we call it imp
(short for image pipeline). We use the name dm6446_ccdc and dm355_ccdc for
vpfe capture drivers. Different hardware name for the same IP has put us into this trouble :). omap34xx_dm644x_ccdc.c is a big name to consider. Or simply choose one name omap_34xx.c or dm644x_ccdc.c and document the IP's supported in the KConfig config variable description as well driver file header.

>
>The code will be pushed to linux-omap when available, but I can't commit to
>any deadline. If you start working on the Davinci driver before we post
>anything, could you please post patches and/or RFCs on the linux-media
>mailing
>list (and CC'ing us) ? Confronting our ideas as soon as possible will
>(hopefully) avoid diverging too much in our implementations.

Yes. will do.

>
>--
>Regards,
>
>Laurent Pinchart

RE: OMAP 3 ISP and N900 sensor driver update

[Hiremath, Vaibhav]

> -----Original Message-----
> From: linux-media-owner@vger.kernel.org [mailto:linux-media-
> owner@vger.kernel.org] On Behalf Of Laurent Pinchart
> Sent: Monday, November 09, 2009 7:36 PM
> To: Karicheri, Muralidharan
> Cc: Sakari Ailus; linux-media@vger.kernel.org; Hans Verkuil; Cohen
> David Abraham; Koskipää Antti Jussi Petteri; Toivonen Tuukka Olli
> Artturi; Zutshi Vimarsh (Nokia-D-MSW/Helsinki);
> talvala@stanford.edu; Aguirre, Sergio; Ivan Ivanov; Stan Varbanov;
> Valeri Ivanov; Atanas Filipov
> Subject: Re: OMAP 3 ISP and N900 sensor driver update
> 
> Hi Murali,
> 
> On Friday 06 November 2009 16:25:01 Karicheri, Muralidharan wrote:
> > Sakari,
> >
> > Thanks for the update...
> >
> > Who is working on the CCDC driver for OMAP35xx?
> 
> Just for the sake of correctness, we're working on an OMAP34xx, not
> an
> OMAP35xx. I don't think that makes a big difference though.
> 
[Hiremath, Vaibhav] OMAP3430 and OMAP3530 are exactly same devices.

> As far as I know nobody on our side is currently working on the CCDC
> driver.
> We're focusing on the previewer and resizer first.
> 
[Hiremath, Vaibhav] I believe I should be able to see the current development activity on sakari's repo, right?

Thanks,
Vaibhav
> > After a week or so, I need to start migrating the CCDC driver to
> sub device
> > interface so that application can directly configure the
> parameters with out
> > having to go through video node. Ultimately it is expected that
> ccdc will
> > have a device node that will allow application to open the device
> and
> > configure the parameters (in the context of Media controller). But
> to begin
> > with I intend to port the existing CCDC driver for DM6446 and
> DM355 to sub
> > device interface. Since the VPFE IPs are common across DM6446 &
> OMAP 35xx,
> > we could use a common sub device across both platforms.
> 
> Coordinating our efforts on that front would indeed be very nice.
> 
> > So I this context, could you please update me on the CCDC
> development
> > on OMAP platform that you work?
> 
> I haven't checked the Davinci VPFE drivers recently. I suppose they
> already
> use the v4l2_subdev interface for their I2C sensors and tuners. If
> not, that
> would be the first step.
> 
> On the OMAP34xx platform, the ISP driver is already somehow
> separated from the
> omap34xxcam driver, although not nicely. In a nutshell, here's the
> current
> plan. Parts 1 and 2 are already implemented and code is available in
> Sakari's
> linux-omap tree.
> 
> 1. Board code registers an omap34xxcam platform device. The platform
> data
> contains an array of v4l2_subdev_i2c_board_info structures filled
> with
> information about all I2C sub-devices (sensor, flash controller,
> lens
> controller, ...).
> 
> 2. The omap34xxcam driver is loaded. Its probe function is called
> with a
> pointer to the platform device. The driver registers a v4l2_device
> and creates
> I2C subdevices using the data supplied through platform data.
> 
> 3. The omap34xxcam driver calls the ISP core with a pointer to the
> v4l2_device
> structure to register all ISP subdevices. The ISP core maintains
> pointers to
> the ISP subdevices.
> 
> As ISP submodules (CCDC, previewer, resizer) are not truly
> independent, we
> were not planning to split them into separate kernel modules. The
> ISP core
> needs to call explicitly into the submodules for instance to
> dispatch
> interrupts.
> 
> It should be possible to use a single CCDC code base across multiple
> platforms. The ISP core module would depend on the CCDC module
> directly. I'm
> not sure how the CCDC module should be called though. An omap prefix
> won't
> work, as it's used on Davinci as well, and an ISP prefix is too
> generic. Do
> you have any internal code name for the ISP device used on Davinci
> and OMAP
> platforms ?
> 
> The code will be pushed to linux-omap when available, but I can't
> commit to
> any deadline. If you start working on the Davinci driver before we
> post
> anything, could you please post patches and/or RFCs on the linux-
> media mailing
> list (and CC'ing us) ? Confronting our ideas as soon as possible
> will
> (hopefully) avoid diverging too much in our implementations.
> 
> --
> Regards,
> 
> Laurent Pinchart
> --
> To unsubscribe from this list: send the line "unsubscribe linux-
> media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: OMAP 3 ISP and N900 sensor driver update

[Sakari Ailus]

Hiremath, Vaibhav wrote:
>> As far as I know nobody on our side is currently working on the CCDC
>> driver.
>> We're focusing on the previewer and resizer first.
>>
> [Hiremath, Vaibhav] I believe I should be able to see the current development activity on sakari's repo, right?

That's right. Just use the cont branch if you want to pull. Then the log 
will be garbage. The alternative is to use the devel branch but then you 
cannot pull, instead use fetch always and manually reapply local 
changes. Pick your poison. :-)

-- 
Sakari Ailus
sakari.ailus@maxwell.research.nokia.com