i.mx6 camera interface (CSI) and mainline kernel

i.mx6 camera interface (CSI) and mainline kernel

[Philippe De Muyter]

Hello,

We use a custom imx6 based board with a canera sensor on it.
I have written the driver for the camera sensor, based on
the freescale so-called "3.10" and even "3.14" linux versions.

The camera works perfectly, but we would like to switch to
a mainline kernel for all the usual reasons (including being
able to contribute our fixes).

>From an old mail thread (*), I have found two git repositories
that used to contain not-yet-approved versions of mainline
imx6 ipu-v3 drivers :

git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
https://github.com:slongerbeam/mediatree.git, mx6-camera-staging

I have tried to compile them with the imx_v6_v7_defconfig, but both
fail directly at compile time. because of later changes in the
v4l2_subdev infrastructure, not ported to the those branches.
Can someone point me to compilable versions (either not rebased
versions of those branches, or updated versions of those branches,
or yet another place to look at). ?

Thanks in advance

Philippe

(*) http://linux-media.vger.kernel.narkive.com/cZQ8NrZ2/i-mx6-status-for-ipu-vpu-gpu

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

Re: i.mx6 camera interface (CSI) and mainline kernel

[Philippe De Muyter]

Update.

On Tue, Feb 23, 2016 at 12:49:43PM +0100, Philippe De Muyter wrote:
> Hello,
> 
> We use a custom imx6 based board with a canera sensor on it.
> I have written the driver for the camera sensor, based on
> the freescale so-called "3.10" and even "3.14" linux versions.
> 
> The camera works perfectly, but we would like to switch to
> a mainline kernel for all the usual reasons (including being
> able to contribute our fixes).
> 
> >From an old mail thread (*), I have found two git repositories
> that used to contain not-yet-approved versions of mainline
> imx6 ipu-v3 drivers :
> 
> git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
> https://github.com:slongerbeam/mediatree.git, mx6-camera-staging
> 
> I have tried to compile them with the imx_v6_v7_defconfig, but both
> fail directly at compile time. because of later changes in the
> v4l2_subdev infrastructure, not ported to the those branches.

What I wrote is true for Steve Longerbeam's branch, but for Philipp Zabel's
branch the problem (so far) was only that CONFIG_MEDIA_CONTROLLER
is not defined in imx_v6_v7_defconfig, but is required for a succesfull
compilation of Philipp's tree.

> Can someone point me to compilable versions (either not rebased
> versions of those branches, or updated versions of those branches,
> or yet another place to look at). ?
> 
> Thanks in advance
> 
> Philippe
> 
> (*) http://linux-media.vger.kernel.narkive.com/cZQ8NrZ2/i-mx6-status-for-ipu-vpu-gpu
> 
> -- 
> Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

Re: i.mx6 camera interface (CSI) and mainline kernel

[Laurent Pinchart]

Hello Philippe,

CC'ing Philipp and Steve.

Philipp, Steve, are you still interested in getting a driver for the i.MX6 
camera interface upstreamed ?

On Tuesday 23 February 2016 15:12:58 Philippe De Muyter wrote:
> Update.
> 
> On Tue, Feb 23, 2016 at 12:49:43PM +0100, Philippe De Muyter wrote:
> > Hello,
> > 
> > We use a custom imx6 based board with a canera sensor on it.
> > I have written the driver for the camera sensor, based on
> > the freescale so-called "3.10" and even "3.14" linux versions.
> > 
> > The camera works perfectly, but we would like to switch to
> > a mainline kernel for all the usual reasons (including being
> > able to contribute our fixes).
> > 
> > >From an old mail thread (*), I have found two git repositories
> > 
> > that used to contain not-yet-approved versions of mainline
> > imx6 ipu-v3 drivers :
> > 
> > git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
> > https://github.com:slongerbeam/mediatree.git, mx6-camera-staging
> > 
> > I have tried to compile them with the imx_v6_v7_defconfig, but both
> > fail directly at compile time. because of later changes in the
> > v4l2_subdev infrastructure, not ported to the those branches.
> 
> What I wrote is true for Steve Longerbeam's branch, but for Philipp Zabel's
> branch the problem (so far) was only that CONFIG_MEDIA_CONTROLLER
> is not defined in imx_v6_v7_defconfig, but is required for a succesfull
> compilation of Philipp's tree.
> 
> > Can someone point me to compilable versions (either not rebased
> > versions of those branches, or updated versions of those branches,
> > or yet another place to look at). ?
> > 
> > Thanks in advance
> > 
> > Philippe
> > 
> > (*)
> > http://linux-media.vger.kernel.narkive.com/cZQ8NrZ2/i-mx6-status-for-ipu-> > vpu-gpu

-- 
Regards,

Laurent Pinchart

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

On 02/25/2016 02:05 PM, Laurent Pinchart wrote:
> Hello Philippe,
>
> CC'ing Philipp and Steve.
>
> Philipp, Steve, are you still interested in getting a driver for the i.MX6 
> camera interface upstreamed ?

Hi Laurent, Philippe(s),

I spent a few days updating my mx6-media-staging branch at
git@github.com:slongerbeam/linux-meibp-314.git, moved forward
to latest master at 4.5-rc3.

So far I have retested video capture with the SabreAuto/ADV7180 and
the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
those platforms.

There is also a mem2mem that should work fine, but haven't tested yet.

I removed camera preview support. At Mentor Graphics we have made
quite a few changes to ipu-v3 driver to allow camera preview to initialize
and control an overlay display plane independently of imx-drm, by adding
a subsystem independent ipu-plane sub-unit. Note we also have a video
output overlay driver that also makes use of ipu-plane. But those changes are
extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
and the output overlay driver out (in fact, camera preview is not of much
utility so I probably won't bring it back in upstream version).

The video capture driver is not quite ready for upstream review yet. It still:

- uses the old cropping APIs but should move forward to selection APIs.

- uses custom sensor subdev drivers for ADV7180 and OV564x. Still
  need to switch to upstream subdevs.

- still does not implement the media device framework.


Steve

>
> On Tuesday 23 February 2016 15:12:58 Philippe De Muyter wrote:
>> Update.
>>
>> On Tue, Feb 23, 2016 at 12:49:43PM +0100, Philippe De Muyter wrote:
>>> Hello,
>>>
>>> We use a custom imx6 based board with a canera sensor on it.
>>> I have written the driver for the camera sensor, based on
>>> the freescale so-called "3.10" and even "3.14" linux versions.
>>>
>>> The camera works perfectly, but we would like to switch to
>>> a mainline kernel for all the usual reasons (including being
>>> able to contribute our fixes).
>>>
>>> >From an old mail thread (*), I have found two git repositories
>>>
>>> that used to contain not-yet-approved versions of mainline
>>> imx6 ipu-v3 drivers :
>>>
>>> git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
>>> https://github.com:slongerbeam/mediatree.git, mx6-camera-staging
>>>
>>> I have tried to compile them with the imx_v6_v7_defconfig, but both
>>> fail directly at compile time. because of later changes in the
>>> v4l2_subdev infrastructure, not ported to the those branches.
>> What I wrote is true for Steve Longerbeam's branch, but for Philipp Zabel's
>> branch the problem (so far) was only that CONFIG_MEDIA_CONTROLLER
>> is not defined in imx_v6_v7_defconfig, but is required for a succesfull
>> compilation of Philipp's tree.
>>
>>> Can someone point me to compilable versions (either not rebased
>>> versions of those branches, or updated versions of those branches,
>>> or yet another place to look at). ?
>>>
>>> Thanks in advance
>>>
>>> Philippe
>>>
>>> (*)
>>> http://linux-media.vger.kernel.narkive.com/cZQ8NrZ2/i-mx6-status-for-ipu-> > vpu-gpu

Re: i.mx6 camera interface (CSI) and mainline kernel

[Hans Verkuil]

On 03/03/2016 03:02 AM, Steve Longerbeam wrote:
> On 02/25/2016 02:05 PM, Laurent Pinchart wrote:
>> Hello Philippe,
>>
>> CC'ing Philipp and Steve.
>>
>> Philipp, Steve, are you still interested in getting a driver for the i.MX6 
>> camera interface upstreamed ?
> 
> Hi Laurent, Philippe(s),
> 
> I spent a few days updating my mx6-media-staging branch at
> git@github.com:slongerbeam/linux-meibp-314.git, moved forward
> to latest master at 4.5-rc3.
> 
> So far I have retested video capture with the SabreAuto/ADV7180 and
> the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
> those platforms.
> 
> There is also a mem2mem that should work fine, but haven't tested yet.
> 
> I removed camera preview support. At Mentor Graphics we have made
> quite a few changes to ipu-v3 driver to allow camera preview to initialize
> and control an overlay display plane independently of imx-drm, by adding
> a subsystem independent ipu-plane sub-unit. Note we also have a video
> output overlay driver that also makes use of ipu-plane. But those changes are
> extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
> and the output overlay driver out (in fact, camera preview is not of much
> utility so I probably won't bring it back in upstream version).
> 
> The video capture driver is not quite ready for upstream review yet. It still:
> 
> - uses the old cropping APIs but should move forward to selection APIs.
> 
> - uses custom sensor subdev drivers for ADV7180 and OV564x. Still
>   need to switch to upstream subdevs.
> 
> - still does not implement the media device framework.

After fixing the first two items on that list, would that be a good time to
add this driver to staging?

i.MX6 is quite popular, so having support for this device upstream would be
very nice indeed.

I'd really like to see some upstream support for this SoC soon.

Regards,

	Hans

> 
> 
> Steve
> 
>>
>> On Tuesday 23 February 2016 15:12:58 Philippe De Muyter wrote:
>>> Update.
>>>
>>> On Tue, Feb 23, 2016 at 12:49:43PM +0100, Philippe De Muyter wrote:
>>>> Hello,
>>>>
>>>> We use a custom imx6 based board with a canera sensor on it.
>>>> I have written the driver for the camera sensor, based on
>>>> the freescale so-called "3.10" and even "3.14" linux versions.
>>>>
>>>> The camera works perfectly, but we would like to switch to
>>>> a mainline kernel for all the usual reasons (including being
>>>> able to contribute our fixes).
>>>>
>>>> >From an old mail thread (*), I have found two git repositories
>>>>
>>>> that used to contain not-yet-approved versions of mainline
>>>> imx6 ipu-v3 drivers :
>>>>
>>>> git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
>>>> https://github.com:slongerbeam/mediatree.git, mx6-camera-staging
>>>>
>>>> I have tried to compile them with the imx_v6_v7_defconfig, but both
>>>> fail directly at compile time. because of later changes in the
>>>> v4l2_subdev infrastructure, not ported to the those branches.
>>> What I wrote is true for Steve Longerbeam's branch, but for Philipp Zabel's
>>> branch the problem (so far) was only that CONFIG_MEDIA_CONTROLLER
>>> is not defined in imx_v6_v7_defconfig, but is required for a succesfull
>>> compilation of Philipp's tree.
>>>
>>>> Can someone point me to compilable versions (either not rebased
>>>> versions of those branches, or updated versions of those branches,
>>>> or yet another place to look at). ?
>>>>
>>>> Thanks in advance
>>>>
>>>> Philippe
>>>>
>>>> (*)
>>>> http://linux-media.vger.kernel.narkive.com/cZQ8NrZ2/i-mx6-status-for-ipu-> > vpu-gpu
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: i.mx6 camera interface (CSI) and mainline kernel

[Philippe De Muyter]

Hi Steve,

On Thu, Mar 03, 2016 at 08:19:55AM +0100, Hans Verkuil wrote:
> On 03/03/2016 03:02 AM, Steve Longerbeam wrote:
> > On 02/25/2016 02:05 PM, Laurent Pinchart wrote:
> >> Hello Philippe,
> >>
> >> CC'ing Philipp and Steve.
> >>
> >> Philipp, Steve, are you still interested in getting a driver for the i.MX6 
> >> camera interface upstreamed ?
> > 
> > Hi Laurent, Philippe(s),
> > 
> > I spent a few days updating my mx6-media-staging branch at

First of all, thank you very much, Steve.

> > git@github.com:slongerbeam/linux-meibp-314.git, moved forward
> > to latest master at 4.5-rc3.

Just to be sure : do you mean  https://github.com/slongerbeam/mediatree.git
or something else ?

> > 
> > So far I have retested video capture with the SabreAuto/ADV7180 and
> > the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
> > those platforms.
> > 
> > There is also a mem2mem that should work fine, but haven't tested yet.
> > 
> > I removed camera preview support. At Mentor Graphics we have made
> > quite a few changes to ipu-v3 driver to allow camera preview to initialize
> > and control an overlay display plane independently of imx-drm, by adding
> > a subsystem independent ipu-plane sub-unit. Note we also have a video
> > output overlay driver that also makes use of ipu-plane. But those changes are
> > extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
> > and the output overlay driver out (in fact, camera preview is not of much
> > utility so I probably won't bring it back in upstream version).
> > 
> > The video capture driver is not quite ready for upstream review yet. It still:
> > 
> > - uses the old cropping APIs but should move forward to selection APIs.
> > 
> > - uses custom sensor subdev drivers for ADV7180 and OV564x. Still
> >   need to switch to upstream subdevs.

Is it only a problem of those sensor drivers (which exact files ?) or
is it a problem of the capture driver itself ?

I must update a sensor driver I wrote for the intdev interface found
in the freescale kernel, and I'd like to start from a working subdev
example.  Which driver should I choose as an example ?

> > 
> > - still does not implement the media device framework.
> 
> After fixing the first two items on that list, would that be a good time to
> add this driver to staging?
> 
> i.MX6 is quite popular, so having support for this device upstream would be
> very nice indeed.
> 
> I'd really like to see some upstream support for this SoC soon.

There are a bunch of people expecting that (and trying to help) :)

Best regards

Philippe

> 
> Regards,
> 
> 	Hans
> 
> > 
> > 
> > Steve
> > 
> >>
> >> On Tuesday 23 February 2016 15:12:58 Philippe De Muyter wrote:
> >>> Update.
> >>>
> >>> On Tue, Feb 23, 2016 at 12:49:43PM +0100, Philippe De Muyter wrote:
> >>>> Hello,
> >>>>
> >>>> We use a custom imx6 based board with a canera sensor on it.
> >>>> I have written the driver for the camera sensor, based on
> >>>> the freescale so-called "3.10" and even "3.14" linux versions.
> >>>>
> >>>> The camera works perfectly, but we would like to switch to
> >>>> a mainline kernel for all the usual reasons (including being
> >>>> able to contribute our fixes).
> >>>>
> >>>> >From an old mail thread (*), I have found two git repositories
> >>>>
> >>>> that used to contain not-yet-approved versions of mainline
> >>>> imx6 ipu-v3 drivers :
> >>>>
> >>>> git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
> >>>> https://github.com:slongerbeam/mediatree.git, mx6-camera-staging
> >>>>
> >>>> I have tried to compile them with the imx_v6_v7_defconfig, but both
> >>>> fail directly at compile time. because of later changes in the
> >>>> v4l2_subdev infrastructure, not ported to the those branches.
> >>> What I wrote is true for Steve Longerbeam's branch, but for Philipp Zabel's
> >>> branch the problem (so far) was only that CONFIG_MEDIA_CONTROLLER
> >>> is not defined in imx_v6_v7_defconfig, but is required for a succesfull
> >>> compilation of Philipp's tree.
> >>>
> >>>> Can someone point me to compilable versions (either not rebased
> >>>> versions of those branches, or updated versions of those branches,
> >>>> or yet another place to look at). ?
> >>>>
> >>>> Thanks in advance
> >>>>
> >>>> Philippe
> >>>>
> >>>> (*)
> >>>> http://linux-media.vger.kernel.narkive.com/cZQ8NrZ2/i-mx6-status-for-ipu-> > vpu-gpu
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

Hi Philippe,

On 03/03/2016 12:36 AM, Philippe De Muyter wrote:
>
> Just to be sure : do you mean  https://github.com/slongerbeam/mediatree.git
> or something else ?

Sorry, yes I meant https://github.com/slongerbeam/mediatree.git.

>
>>> So far I have retested video capture with the SabreAuto/ADV7180 and
>>> the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
>>> those platforms.
>>>
>>> There is also a mem2mem that should work fine, but haven't tested yet.
>>>
>>> I removed camera preview support. At Mentor Graphics we have made
>>> quite a few changes to ipu-v3 driver to allow camera preview to initialize
>>> and control an overlay display plane independently of imx-drm, by adding
>>> a subsystem independent ipu-plane sub-unit. Note we also have a video
>>> output overlay driver that also makes use of ipu-plane. But those changes are
>>> extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
>>> and the output overlay driver out (in fact, camera preview is not of much
>>> utility so I probably won't bring it back in upstream version).
>>>
>>> The video capture driver is not quite ready for upstream review yet. It still:
>>>
>>> - uses the old cropping APIs but should move forward to selection APIs.
>>>
>>> - uses custom sensor subdev drivers for ADV7180 and OV564x. Still
>>>   need to switch to upstream subdevs.
> Is it only a problem of those sensor drivers (which exact files ?) or
> is it a problem of the capture driver itself ?

The camera interface driver (drivers/staging/media/imx6/capture/mx6-camif.c)
is binding to these subdevs:

drivers/staging/media/imx6/capture/adv7180.c
drivers/staging/media/imx6/capture/ov5642.c
drivers/staging/media/imx6/capture/ov5640-mipi.c

But instead should use the subdevs under drivers/media/i2c, specifically:

drivers/media/i2c/adv7180.c (and adding whatever standard subdev features
the imx6 interface driver requires).

There is a drivers/media/i2c/soc_camera/ov5642.c, but there is no mipi-csi2
capable subdev for the ov5640 with the mipi-csi2 interface, so that would have
to be created.



> I must update a sensor driver I wrote for the intdev interface found
> in the freescale kernel, and I'd like to start from a working subdev
> example.  Which driver should I choose as an example ?

There's lots of good examples under drivers/media/i2c/.


Steve

Re: i.mx6 camera interface (CSI) and mainline kernel

[Philippe De Muyter]

On Thu, Mar 03, 2016 at 09:45:08AM -0800, Steve Longerbeam wrote:
> Hi Philippe,
> 
> On 03/03/2016 12:36 AM, Philippe De Muyter wrote:
> >
> > Just to be sure : do you mean  https://github.com/slongerbeam/mediatree.git
> > or something else ?
> 
> Sorry, yes I meant https://github.com/slongerbeam/mediatree.git.
> 
> >
> >>> So far I have retested video capture with the SabreAuto/ADV7180 and
> >>> the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
> >>> those platforms.
> >>>
> >>> There is also a mem2mem that should work fine, but haven't tested yet.
> >>>
> >>> I removed camera preview support. At Mentor Graphics we have made
> >>> quite a few changes to ipu-v3 driver to allow camera preview to initialize
> >>> and control an overlay display plane independently of imx-drm, by adding
> >>> a subsystem independent ipu-plane sub-unit. Note we also have a video
> >>> output overlay driver that also makes use of ipu-plane. But those changes are
> >>> extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
> >>> and the output overlay driver out (in fact, camera preview is not of much
> >>> utility so I probably won't bring it back in upstream version).
> >>>
> >>> The video capture driver is not quite ready for upstream review yet. It still:
> >>>
> >>> - uses the old cropping APIs but should move forward to selection APIs.
> >>>
> >>> - uses custom sensor subdev drivers for ADV7180 and OV564x. Still
> >>>   need to switch to upstream subdevs.
> > Is it only a problem of those sensor drivers (which exact files ?) or
> > is it a problem of the capture driver itself ?
> 
> The camera interface driver (drivers/staging/media/imx6/capture/mx6-camif.c)
> is binding to these subdevs:
> 
> drivers/staging/media/imx6/capture/adv7180.c
> drivers/staging/media/imx6/capture/ov5642.c
> drivers/staging/media/imx6/capture/ov5640-mipi.c
> 
> But instead should use the subdevs under drivers/media/i2c, specifically:
> 
> drivers/media/i2c/adv7180.c (and adding whatever standard subdev features
> the imx6 interface driver requires).
> 
> There is a drivers/media/i2c/soc_camera/ov5642.c, but there is no mipi-csi2
> capable subdev for the ov5640 with the mipi-csi2 interface, so that would have
> to be created.
> 
> 
> 
> > I must update a sensor driver I wrote for the intdev interface found
> > in the freescale kernel, and I'd like to start from a working subdev
> > example.  Which driver should I choose as an example ?
> 
> There's lots of good examples under drivers/media/i2c/.

OK.  Thank you

Philippe

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

Re: i.mx6 camera interface (CSI) and mainline kernel

[Tim Harvey]

On Thu, Mar 3, 2016 at 9:45 AM, Steve Longerbeam
<steve_longerbeam@mentor.com> wrote:
> Hi Philippe,
>
> On 03/03/2016 12:36 AM, Philippe De Muyter wrote:
>>
>> Just to be sure : do you mean  https://github.com/slongerbeam/mediatree.git
>> or something else ?
>
> Sorry, yes I meant https://github.com/slongerbeam/mediatree.git.
>
>>
>>>> So far I have retested video capture with the SabreAuto/ADV7180 and
>>>> the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
>>>> those platforms.
>>>>
>>>> There is also a mem2mem that should work fine, but haven't tested yet.
>>>>
>>>> I removed camera preview support. At Mentor Graphics we have made
>>>> quite a few changes to ipu-v3 driver to allow camera preview to initialize
>>>> and control an overlay display plane independently of imx-drm, by adding
>>>> a subsystem independent ipu-plane sub-unit. Note we also have a video
>>>> output overlay driver that also makes use of ipu-plane. But those changes are
>>>> extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
>>>> and the output overlay driver out (in fact, camera preview is not of much
>>>> utility so I probably won't bring it back in upstream version).
>>>>
>>>> The video capture driver is not quite ready for upstream review yet. It still:
>>>>
>>>> - uses the old cropping APIs but should move forward to selection APIs.
>>>>
>>>> - uses custom sensor subdev drivers for ADV7180 and OV564x. Still
>>>>   need to switch to upstream subdevs.
>> Is it only a problem of those sensor drivers (which exact files ?) or
>> is it a problem of the capture driver itself ?
>
> The camera interface driver (drivers/staging/media/imx6/capture/mx6-camif.c)
> is binding to these subdevs:
>
> drivers/staging/media/imx6/capture/adv7180.c
> drivers/staging/media/imx6/capture/ov5642.c
> drivers/staging/media/imx6/capture/ov5640-mipi.c
>
> But instead should use the subdevs under drivers/media/i2c, specifically:
>
> drivers/media/i2c/adv7180.c (and adding whatever standard subdev features
> the imx6 interface driver requires).
>
> There is a drivers/media/i2c/soc_camera/ov5642.c, but there is no mipi-csi2
> capable subdev for the ov5640 with the mipi-csi2 interface, so that would have
> to be created.
>

Steve,

I've built your mx6-media-staging branch and added device-tree config
for the Gateworks Ventana boards which have an on-board ADV7180 and it
works great. I've tested it capturing frames via v4l2-ctl as well as
gstreamer.

Please let me know what kind of testing you need. I would love to see
this get mainlined!

Regards,

Tim

Tim Harvey - Principal Software Engineer
Gateworks Corporation - http://www.gateworks.com/
3026 S. Higuera St. San Luis Obispo CA 93401
805-781-2000

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

On 03/07/2016 08:19 AM, Tim Harvey wrote:
> On Thu, Mar 3, 2016 at 9:45 AM, Steve Longerbeam
> <steve_longerbeam@mentor.com> wrote:
>> Hi Philippe,
>>
>> On 03/03/2016 12:36 AM, Philippe De Muyter wrote:
>>> Just to be sure : do you mean  https://github.com/slongerbeam/mediatree.git
>>> or something else ?
>> Sorry, yes I meant https://github.com/slongerbeam/mediatree.git.
>>
>>>>> So far I have retested video capture with the SabreAuto/ADV7180 and
>>>>> the SabreSD/OV5640-mipi-csi2, and video capture is working fine on
>>>>> those platforms.
>>>>>
>>>>> There is also a mem2mem that should work fine, but haven't tested yet.
>>>>>
>>>>> I removed camera preview support. At Mentor Graphics we have made
>>>>> quite a few changes to ipu-v3 driver to allow camera preview to initialize
>>>>> and control an overlay display plane independently of imx-drm, by adding
>>>>> a subsystem independent ipu-plane sub-unit. Note we also have a video
>>>>> output overlay driver that also makes use of ipu-plane. But those changes are
>>>>> extensive and touch imx-drm as well as ipu-v3, so I am leaving camera preview
>>>>> and the output overlay driver out (in fact, camera preview is not of much
>>>>> utility so I probably won't bring it back in upstream version).
>>>>>
>>>>> The video capture driver is not quite ready for upstream review yet. It still:
>>>>>
>>>>> - uses the old cropping APIs but should move forward to selection APIs.
>>>>>
>>>>> - uses custom sensor subdev drivers for ADV7180 and OV564x. Still
>>>>>   need to switch to upstream subdevs.
>>> Is it only a problem of those sensor drivers (which exact files ?) or
>>> is it a problem of the capture driver itself ?
>> The camera interface driver (drivers/staging/media/imx6/capture/mx6-camif.c)
>> is binding to these subdevs:
>>
>> drivers/staging/media/imx6/capture/adv7180.c
>> drivers/staging/media/imx6/capture/ov5642.c
>> drivers/staging/media/imx6/capture/ov5640-mipi.c
>>
>> But instead should use the subdevs under drivers/media/i2c, specifically:
>>
>> drivers/media/i2c/adv7180.c (and adding whatever standard subdev features
>> the imx6 interface driver requires).
>>
>> There is a drivers/media/i2c/soc_camera/ov5642.c, but there is no mipi-csi2
>> capable subdev for the ov5640 with the mipi-csi2 interface, so that would have
>> to be created.
>>
> Steve,
>
> I've built your mx6-media-staging branch and added device-tree config
> for the Gateworks Ventana boards which have an on-board ADV7180 and it
> works great. I've tested it capturing frames via v4l2-ctl as well as
> gstreamer.
>
> Please let me know what kind of testing you need. I would love to see
> this get mainlined!
>


Hi Tim, good to hear it works for you on the Ventana boards.

I've just pushed some more commits to the mx6-media-staging branch that
get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
of YUYV8_2X8. But I'm holding off on that change because this subdev is used
by Renesas targets and would likely corrupt captured images for those
targets. But I believe UYVY is the correct transmit order according to the
BT.656 standard.

Another thing that should also be changed in drivers/media/i2c/adv7180.c
is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
for PAL.

Steve

Re: i.mx6 camera interface (CSI) and mainline kernel

[Tim Harvey]

On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
<steve_longerbeam@mentor.com> wrote:
> On 03/07/2016 08:19 AM, Tim Harvey wrote:
<snip>
>
>
> Hi Tim, good to hear it works for you on the Ventana boards.
>
> I've just pushed some more commits to the mx6-media-staging branch that
> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
> by Renesas targets and would likely corrupt captured images for those
> targets. But I believe UYVY is the correct transmit order according to the
> BT.656 standard.
>
> Another thing that should also be changed in drivers/media/i2c/adv7180.c
> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
> for PAL.
>
> Steve
>
>

Steve,

with your latest patches, I'm crashing with an null-pointer-deref in
adv7180_set_pad_format. What is your kernel config for
CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?

Your tree contains about 16 or so patches on top of linux-media for
imx6 capture. Are you close to the point where you will be posting a
patch series? If so, please CC me for testing with the adv7180 sensor.

Tim

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

On 03/09/2016 02:44 PM, Tim Harvey wrote:
> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
> <steve_longerbeam@mentor.com> wrote:
>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
> <snip>
>>
>> Hi Tim, good to hear it works for you on the Ventana boards.
>>
>> I've just pushed some more commits to the mx6-media-staging branch that
>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>> by Renesas targets and would likely corrupt captured images for those
>> targets. But I believe UYVY is the correct transmit order according to the
>> BT.656 standard.
>>
>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>> for PAL.
>>
>> Steve
>>
>>
> Steve,
>
> with your latest patches, I'm crashing with an null-pointer-deref in
> adv7180_set_pad_format. What is your kernel config for
> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?

Right, I thought I fixed that, I was passing a NULL pad_cfg for
TRY_FORMAT, but that was fixed. Maybe you fetched a version
of mx6-media-staging while I was in the middle of debugging?
Try fetching again.

I tried with both CONFIG_MEDIA_CONTROLLER and
CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
I don't get the null deref in adv7180_set_pad_format.


>
> Your tree contains about 16 or so patches on top of linux-media for
> imx6 capture. Are you close to the point where you will be posting a
> patch series? If so, please CC me for testing with the adv7180 sensor.

I guess I can try posting a series again, but there will likely be push-back from
Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
their version did not implement scaling, colorspace conversion, or image rotation via
the IC. Instead their driver sends raw camera frames directly to memory, and image
conversion is carried out by separate mem2mem device. Our capture driver does
image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
We also have a mem2mem device that does conversion using IC post-processing,
which I have included in mx6-media-staging.

Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
can multiplex video from different sensors either using the internal mux in the SoC,
or can control an external mux via gpio. Our driver only supports the internal mux,
and does it with a platform data function.

But like I said, I don't what the latest status is of the Pengutronix video capture support.

Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.

Steve

Re: i.mx6 camera interface (CSI) and mainline kernel

[Hans Verkuil]

On 03/10/2016 01:12 AM, Steve Longerbeam wrote:
> On 03/09/2016 02:44 PM, Tim Harvey wrote:
>> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
>> <steve_longerbeam@mentor.com> wrote:
>>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
>> <snip>
>>>
>>> Hi Tim, good to hear it works for you on the Ventana boards.
>>>
>>> I've just pushed some more commits to the mx6-media-staging branch that
>>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>>> by Renesas targets and would likely corrupt captured images for those
>>> targets. But I believe UYVY is the correct transmit order according to the
>>> BT.656 standard.
>>>
>>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>>> for PAL.
>>>
>>> Steve
>>>
>>>
>> Steve,
>>
>> with your latest patches, I'm crashing with an null-pointer-deref in
>> adv7180_set_pad_format. What is your kernel config for
>> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
> 
> Right, I thought I fixed that, I was passing a NULL pad_cfg for
> TRY_FORMAT, but that was fixed. Maybe you fetched a version
> of mx6-media-staging while I was in the middle of debugging?
> Try fetching again.
> 
> I tried with both CONFIG_MEDIA_CONTROLLER and
> CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
> I don't get the null deref in adv7180_set_pad_format.
> 
> 
>>
>> Your tree contains about 16 or so patches on top of linux-media for
>> imx6 capture. Are you close to the point where you will be posting a
>> patch series? If so, please CC me for testing with the adv7180 sensor.
> 
> I guess I can try posting a series again, but there will likely be push-back from
> Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
> their version did not implement scaling, colorspace conversion, or image rotation via
> the IC. Instead their driver sends raw camera frames directly to memory, and image
> conversion is carried out by separate mem2mem device. Our capture driver does
> image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
> We also have a mem2mem device that does conversion using IC post-processing,
> which I have included in mx6-media-staging.
> 
> Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
> can multiplex video from different sensors either using the internal mux in the SoC,
> or can control an external mux via gpio. Our driver only supports the internal mux,
> and does it with a platform data function.
> 
> But like I said, I don't what the latest status is of the Pengutronix video capture support.
> 
> Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.

Steve & Pengutronix,

I would be happy to add drivers to staging, either Steve's, Pengutronix or both.

The i.mx6 is quite popular and the lack of video capture support in the kernel is
a big stumbling block, especially given the pile of crap that freescale provides.

Having decent code in the kernel will help progress a lot I hope.

Regards,

	Hans

Re: i.mx6 camera interface (CSI) and mainline kernel

[Philippe De Muyter]

Hi Steve and all,

I am busy porting to Steve's subdev version a driver that I had written
in intdev style, for the freescale imx6 linux version.  And I have a
problem :

My previous driver had the following ioctl function, which used
the V4L2_FRMIVAL_TYPE_CONTINUOUS type answer.

static int ioctl_enum_frameintervals(struct v4l2_int_device *s,
                                         struct v4l2_frmivalenum *fival)
{
        if (fival->index)
                return -EINVAL;
        fival->type = V4L2_FRMIVAL_TYPE_CONTINUOUS;
        fival->stepwise.min.numerator = 1;
        fival->stepwise.min.denominator = MAX_FPS;
        fival->stepwise.max.numerator = 1;
        fival->stepwise.max.denominator = MIN_FPS;
        fival->stepwise.step.numerator = 1;
        fival->stepwise.step.denominator = 1;
        return 0;
}

Now I see that Steve's related ioctl, using the 'enum_frame_interval'
pad function is :

static int vidioc_enum_frameintervals(struct file *file, void *priv,
                                      struct v4l2_frmivalenum *fival)
{
        struct v4l2_subdev_frame_interval_enum fie ...;
	...

        ret = v4l2_subdev_call(dev->ep->sd, pad, enum_frame_interval,
                               NULL, &fie);

        fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
        fival->discrete = fie.interval;
        return 0;
}

where type is arbitrarily set to V4L2_FRMIVAL_TYPE_DISCRETE, because of the
limitation of the struct v4l2_subdev_frame_interval_enum, which has no 'type',
field nor a 'stepwise' field.

How can I do to continue to provide de V4L2_FRMIVAL_TYPE_CONTINUOUS answer ?

TIA

Philippe

On Thu, Mar 10, 2016 at 08:30:02AM +0100, Hans Verkuil wrote:
> On 03/10/2016 01:12 AM, Steve Longerbeam wrote:
> > On 03/09/2016 02:44 PM, Tim Harvey wrote:
> >> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
> >> <steve_longerbeam@mentor.com> wrote:
> >>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
> >> <snip>
> >>>
> >>> Hi Tim, good to hear it works for you on the Ventana boards.
> >>>
> >>> I've just pushed some more commits to the mx6-media-staging branch that
> >>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
> >>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
> >>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
> >>> by Renesas targets and would likely corrupt captured images for those
> >>> targets. But I believe UYVY is the correct transmit order according to the
> >>> BT.656 standard.
> >>>
> >>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
> >>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
> >>> for PAL.
> >>>
> >>> Steve
> >>>
> >>>
> >> Steve,
> >>
> >> with your latest patches, I'm crashing with an null-pointer-deref in
> >> adv7180_set_pad_format. What is your kernel config for
> >> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
> > 
> > Right, I thought I fixed that, I was passing a NULL pad_cfg for
> > TRY_FORMAT, but that was fixed. Maybe you fetched a version
> > of mx6-media-staging while I was in the middle of debugging?
> > Try fetching again.
> > 
> > I tried with both CONFIG_MEDIA_CONTROLLER and
> > CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
> > I don't get the null deref in adv7180_set_pad_format.
> > 
> > 
> >>
> >> Your tree contains about 16 or so patches on top of linux-media for
> >> imx6 capture. Are you close to the point where you will be posting a
> >> patch series? If so, please CC me for testing with the adv7180 sensor.
> > 
> > I guess I can try posting a series again, but there will likely be push-back from
> > Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
> > their version did not implement scaling, colorspace conversion, or image rotation via
> > the IC. Instead their driver sends raw camera frames directly to memory, and image
> > conversion is carried out by separate mem2mem device. Our capture driver does
> > image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
> > We also have a mem2mem device that does conversion using IC post-processing,
> > which I have included in mx6-media-staging.
> > 
> > Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
> > can multiplex video from different sensors either using the internal mux in the SoC,
> > or can control an external mux via gpio. Our driver only supports the internal mux,
> > and does it with a platform data function.
> > 
> > But like I said, I don't what the latest status is of the Pengutronix video capture support.
> > 
> > Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.
> 
> Steve & Pengutronix,
> 
> I would be happy to add drivers to staging, either Steve's, Pengutronix or both.
> 
> The i.mx6 is quite popular and the lack of video capture support in the kernel is
> a big stumbling block, especially given the pile of crap that freescale provides.
> 
> Having decent code in the kernel will help progress a lot I hope.
> 
> Regards,
> 
> 	Hans

-- 
Philippe De Muyter +32 2 6101532 Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

Re: i.mx6 camera interface (CSI) and mainline kernel

[Tim Harvey]

On Wed, Mar 9, 2016 at 4:12 PM, Steve Longerbeam
<steve_longerbeam@mentor.com> wrote:
> On 03/09/2016 02:44 PM, Tim Harvey wrote:
>> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
>> <steve_longerbeam@mentor.com> wrote:
>>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
>> <snip>
>>>
>>> Hi Tim, good to hear it works for you on the Ventana boards.
>>>
>>> I've just pushed some more commits to the mx6-media-staging branch that
>>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>>> by Renesas targets and would likely corrupt captured images for those
>>> targets. But I believe UYVY is the correct transmit order according to the
>>> BT.656 standard.
>>>
>>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>>> for PAL.
>>>
>>> Steve
>>>
>>>
>> Steve,
>>
>> with your latest patches, I'm crashing with an null-pointer-deref in
>> adv7180_set_pad_format. What is your kernel config for
>> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
>
> Right, I thought I fixed that, I was passing a NULL pad_cfg for
> TRY_FORMAT, but that was fixed. Maybe you fetched a version
> of mx6-media-staging while I was in the middle of debugging?
> Try fetching again.
>
> I tried with both CONFIG_MEDIA_CONTROLLER and
> CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
> I don't get the null deref in adv7180_set_pad_format.
>
>
>>
>> Your tree contains about 16 or so patches on top of linux-media for
>> imx6 capture. Are you close to the point where you will be posting a
>> patch series? If so, please CC me for testing with the adv7180 sensor.
>
> I guess I can try posting a series again, but there will likely be push-back from
> Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
> their version did not implement scaling, colorspace conversion, or image rotation via
> the IC. Instead their driver sends raw camera frames directly to memory, and image
> conversion is carried out by separate mem2mem device. Our capture driver does
> image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
> We also have a mem2mem device that does conversion using IC post-processing,
> which I have included in mx6-media-staging.
>
> Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
> can multiplex video from different sensors either using the internal mux in the SoC,
> or can control an external mux via gpio. Our driver only supports the internal mux,
> and does it with a platform data function.

Philipp (Zabel) / Sascha - are you able to rebase and re-post your
patchset for IMX6 capture or will you defer to Steve's patchset?

We are still missing the media device driver, the video multiplexer,
and the IPUv3 capture driver.

Regards,

Tim

Re: i.mx6 camera interface (CSI) and mainline kernel

[Tim Harvey]

On Wed, Mar 9, 2016 at 4:12 PM, Steve Longerbeam
<steve_longerbeam@mentor.com> wrote:
> On 03/09/2016 02:44 PM, Tim Harvey wrote:
>> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
>> <steve_longerbeam@mentor.com> wrote:
>>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
>> <snip>
>>>
>>> Hi Tim, good to hear it works for you on the Ventana boards.
>>>
>>> I've just pushed some more commits to the mx6-media-staging branch that
>>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>>> by Renesas targets and would likely corrupt captured images for those
>>> targets. But I believe UYVY is the correct transmit order according to the
>>> BT.656 standard.
>>>
>>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>>> for PAL.
>>>
>>> Steve
>>>
>>>
>> Steve,
>>
>> with your latest patches, I'm crashing with an null-pointer-deref in
>> adv7180_set_pad_format. What is your kernel config for
>> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
>
> Right, I thought I fixed that, I was passing a NULL pad_cfg for
> TRY_FORMAT, but that was fixed. Maybe you fetched a version
> of mx6-media-staging while I was in the middle of debugging?
> Try fetching again.
>
> I tried with both CONFIG_MEDIA_CONTROLLER and
> CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
> I don't get the null deref in adv7180_set_pad_format.
>
>
>>
>> Your tree contains about 16 or so patches on top of linux-media for
>> imx6 capture. Are you close to the point where you will be posting a
>> patch series? If so, please CC me for testing with the adv7180 sensor.
>
> I guess I can try posting a series again, but there will likely be push-back from
> Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
> their version did not implement scaling, colorspace conversion, or image rotation via
> the IC. Instead their driver sends raw camera frames directly to memory, and image
> conversion is carried out by separate mem2mem device. Our capture driver does
> image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
> We also have a mem2mem device that does conversion using IC post-processing,
> which I have included in mx6-media-staging.
>
> Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
> can multiplex video from different sensors either using the internal mux in the SoC,
> or can control an external mux via gpio. Our driver only supports the internal mux,
> and does it with a platform data function.
>
> But like I said, I don't what the latest status is of the Pengutronix video capture support.
>
> Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.
>
> Steve
>
>

Steve,

Some time has passed without any IMX6 CSI drivers or response from
Pengutronix and Hans has agreed to add either/both drivers to staging.
Do you have time to rebase and re-post your driver(s)? Maybe that will
get the ball rolling on this final huge missing feature for the IMX6
in mainline.

Regards,

Tim

Re: i.mx6 camera interface (CSI) and mainline kernel

[Hans Verkuil]

On 06/02/2016 03:29 PM, Tim Harvey wrote:
> On Wed, Mar 9, 2016 at 4:12 PM, Steve Longerbeam
> <steve_longerbeam@mentor.com> wrote:
>> On 03/09/2016 02:44 PM, Tim Harvey wrote:
>>> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
>>> <steve_longerbeam@mentor.com> wrote:
>>>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
>>> <snip>
>>>>
>>>> Hi Tim, good to hear it works for you on the Ventana boards.
>>>>
>>>> I've just pushed some more commits to the mx6-media-staging branch that
>>>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>>>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>>>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>>>> by Renesas targets and would likely corrupt captured images for those
>>>> targets. But I believe UYVY is the correct transmit order according to the
>>>> BT.656 standard.
>>>>
>>>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>>>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>>>> for PAL.
>>>>
>>>> Steve
>>>>
>>>>
>>> Steve,
>>>
>>> with your latest patches, I'm crashing with an null-pointer-deref in
>>> adv7180_set_pad_format. What is your kernel config for
>>> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
>>
>> Right, I thought I fixed that, I was passing a NULL pad_cfg for
>> TRY_FORMAT, but that was fixed. Maybe you fetched a version
>> of mx6-media-staging while I was in the middle of debugging?
>> Try fetching again.
>>
>> I tried with both CONFIG_MEDIA_CONTROLLER and
>> CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
>> I don't get the null deref in adv7180_set_pad_format.
>>
>>
>>>
>>> Your tree contains about 16 or so patches on top of linux-media for
>>> imx6 capture. Are you close to the point where you will be posting a
>>> patch series? If so, please CC me for testing with the adv7180 sensor.
>>
>> I guess I can try posting a series again, but there will likely be push-back from
>> Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
>> their version did not implement scaling, colorspace conversion, or image rotation via
>> the IC. Instead their driver sends raw camera frames directly to memory, and image
>> conversion is carried out by separate mem2mem device. Our capture driver does
>> image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
>> We also have a mem2mem device that does conversion using IC post-processing,
>> which I have included in mx6-media-staging.
>>
>> Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
>> can multiplex video from different sensors either using the internal mux in the SoC,
>> or can control an external mux via gpio. Our driver only supports the internal mux,
>> and does it with a platform data function.
>>
>> But like I said, I don't what the latest status is of the Pengutronix video capture support.
>>
>> Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.
>>
>> Steve
>>
>>
> 
> Steve,
> 
> Some time has passed without any IMX6 CSI drivers or response from
> Pengutronix and Hans has agreed to add either/both drivers to staging.
> Do you have time to rebase and re-post your driver(s)? Maybe that will
> get the ball rolling on this final huge missing feature for the IMX6
> in mainline.

Right. All that is needed is for someone to take the latest version, make it compile
in the media_tree in drivers/media/staging and post the patch (just take care to keep
the correct copyrights, Signed-off-by's etc.) and I'll be happy to take it. This is
exactly what staging is for. I think it will greatly increases the chances of this
code being improved for mainline. And I'm happy to take both drivers as well, again,
that's what staging is for.

I've been thinking of doing this myself, but I just don't have the time.

Ideally this is done by the authors, but if they don't have time either then someone
else can do this.

Regards,

	Hans

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

On 06/02/2016 06:55 AM, Hans Verkuil wrote:
>
> On 06/02/2016 03:29 PM, Tim Harvey wrote:
>> On Wed, Mar 9, 2016 at 4:12 PM, Steve Longerbeam
>> <steve_longerbeam@mentor.com> wrote:
>>> On 03/09/2016 02:44 PM, Tim Harvey wrote:
>>>> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
>>>> <steve_longerbeam@mentor.com> wrote:
>>>>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
>>>> <snip>
>>>>> Hi Tim, good to hear it works for you on the Ventana boards.
>>>>>
>>>>> I've just pushed some more commits to the mx6-media-staging branch that
>>>>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>>>>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>>>>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>>>>> by Renesas targets and would likely corrupt captured images for those
>>>>> targets. But I believe UYVY is the correct transmit order according to the
>>>>> BT.656 standard.
>>>>>
>>>>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>>>>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>>>>> for PAL.
>>>>>
>>>>> Steve
>>>>>
>>>>>
>>>> Steve,
>>>>
>>>> with your latest patches, I'm crashing with an null-pointer-deref in
>>>> adv7180_set_pad_format. What is your kernel config for
>>>> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
>>> Right, I thought I fixed that, I was passing a NULL pad_cfg for
>>> TRY_FORMAT, but that was fixed. Maybe you fetched a version
>>> of mx6-media-staging while I was in the middle of debugging?
>>> Try fetching again.
>>>
>>> I tried with both CONFIG_MEDIA_CONTROLLER and
>>> CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
>>> I don't get the null deref in adv7180_set_pad_format.
>>>
>>>
>>>> Your tree contains about 16 or so patches on top of linux-media for
>>>> imx6 capture. Are you close to the point where you will be posting a
>>>> patch series? If so, please CC me for testing with the adv7180 sensor.
>>> I guess I can try posting a series again, but there will likely be push-back from
>>> Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
>>> their version did not implement scaling, colorspace conversion, or image rotation via
>>> the IC. Instead their driver sends raw camera frames directly to memory, and image
>>> conversion is carried out by separate mem2mem device. Our capture driver does
>>> image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
>>> We also have a mem2mem device that does conversion using IC post-processing,
>>> which I have included in mx6-media-staging.
>>>
>>> Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
>>> can multiplex video from different sensors either using the internal mux in the SoC,
>>> or can control an external mux via gpio. Our driver only supports the internal mux,
>>> and does it with a platform data function.
>>>
>>> But like I said, I don't what the latest status is of the Pengutronix video capture support.
>>>
>>> Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.
>>>
>>> Steve
>>>
>>>
>> Steve,
>>
>> Some time has passed without any IMX6 CSI drivers or response from
>> Pengutronix and Hans has agreed to add either/both drivers to staging.
>> Do you have time to rebase and re-post your driver(s)? Maybe that will
>> get the ball rolling on this final huge missing feature for the IMX6
>> in mainline.
> Right. All that is needed is for someone to take the latest version, make it compile
> in the media_tree in drivers/media/staging and post the patch (just take care to keep
> the correct copyrights, Signed-off-by's etc.) and I'll be happy to take it. This is
> exactly what staging is for. I think it will greatly increases the chances of this
> code being improved for mainline. And I'm happy to take both drivers as well, again,
> that's what staging is for.
>
> I've been thinking of doing this myself, but I just don't have the time.
>
> Ideally this is done by the authors, but if they don't have time either then someone
> else can do this.
>

Hi Hans and Tim,

No problem, I will repost the patch-set this week. I've been meaning to,
just busy lately with other tasks.

Steve

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

On 06/02/2016 09:50 AM, Steve Longerbeam wrote:
> On 06/02/2016 06:55 AM, Hans Verkuil wrote:
>> On 06/02/2016 03:29 PM, Tim Harvey wrote:
>>> On Wed, Mar 9, 2016 at 4:12 PM, Steve Longerbeam
>>> <steve_longerbeam@mentor.com> wrote:
>>>> On 03/09/2016 02:44 PM, Tim Harvey wrote:
>>>>> On Tue, Mar 8, 2016 at 6:06 PM, Steve Longerbeam
>>>>> <steve_longerbeam@mentor.com> wrote:
>>>>>> On 03/07/2016 08:19 AM, Tim Harvey wrote:
>>>>> <snip>
>>>>>> Hi Tim, good to hear it works for you on the Ventana boards.
>>>>>>
>>>>>> I've just pushed some more commits to the mx6-media-staging branch that
>>>>>> get the drivers/media/i2c/adv7180.c subdev working with the imx6 capture
>>>>>> backend. Images look perfect when switching to UYVY8_2X8 mbus code instead
>>>>>> of YUYV8_2X8. But I'm holding off on that change because this subdev is used
>>>>>> by Renesas targets and would likely corrupt captured images for those
>>>>>> targets. But I believe UYVY is the correct transmit order according to the
>>>>>> BT.656 standard.
>>>>>>
>>>>>> Another thing that should also be changed in drivers/media/i2c/adv7180.c
>>>>>> is the field type. It should be V4L2_FIELD_SEQ_TB for NTSC and V4L2_FIELD_SEQ_BT
>>>>>> for PAL.
>>>>>>
>>>>>> Steve
>>>>>>
>>>>>>
>>>>> Steve,
>>>>>
>>>>> with your latest patches, I'm crashing with an null-pointer-deref in
>>>>> adv7180_set_pad_format. What is your kernel config for
>>>>> CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API?
>>>> Right, I thought I fixed that, I was passing a NULL pad_cfg for
>>>> TRY_FORMAT, but that was fixed. Maybe you fetched a version
>>>> of mx6-media-staging while I was in the middle of debugging?
>>>> Try fetching again.
>>>>
>>>> I tried with both CONFIG_MEDIA_CONTROLLER and
>>>> CONFIG_VIDEO_V4L2_SUBDEV_API enabled and both disabled, and
>>>> I don't get the null deref in adv7180_set_pad_format.
>>>>
>>>>
>>>>> Your tree contains about 16 or so patches on top of linux-media for
>>>>> imx6 capture. Are you close to the point where you will be posting a
>>>>> patch series? If so, please CC me for testing with the adv7180 sensor.
>>>> I guess I can try posting a series again, but there will likely be push-back from
>>>> Pengutronix. They have their own video capture driver for imx6. Last I heard (a while ago!)
>>>> their version did not implement scaling, colorspace conversion, or image rotation via
>>>> the IC. Instead their driver sends raw camera frames directly to memory, and image
>>>> conversion is carried out by separate mem2mem device. Our capture driver does
>>>> image conversion (scaling, CSC, and rotation) natively using the IC pre-processing channel.
>>>> We also have a mem2mem device that does conversion using IC post-processing,
>>>> which I have included in mx6-media-staging.
>>>>
>>>> Also IIRC they did some pretty slick stuff with a video bus multiplexer subdev, which
>>>> can multiplex video from different sensors either using the internal mux in the SoC,
>>>> or can control an external mux via gpio. Our driver only supports the internal mux,
>>>> and does it with a platform data function.
>>>>
>>>> But like I said, I don't what the latest status is of the Pengutronix video capture support.
>>>>
>>>> Btw, I just pushed an update of mx6-media-staging that implements vidioc_[gs]_selection.
>>>>
>>>> Steve
>>>>
>>>>
>>> Steve,
>>>
>>> Some time has passed without any IMX6 CSI drivers or response from
>>> Pengutronix and Hans has agreed to add either/both drivers to staging.
>>> Do you have time to rebase and re-post your driver(s)? Maybe that will
>>> get the ball rolling on this final huge missing feature for the IMX6
>>> in mainline.
>> Right. All that is needed is for someone to take the latest version, make it compile
>> in the media_tree in drivers/media/staging and post the patch (just take care to keep
>> the correct copyrights, Signed-off-by's etc.) and I'll be happy to take it. This is
>> exactly what staging is for. I think it will greatly increases the chances of this
>> code being improved for mainline. And I'm happy to take both drivers as well, again,
>> that's what staging is for.
>>
>> I've been thinking of doing this myself, but I just don't have the time.
>>
>> Ideally this is done by the authors, but if they don't have time either then someone
>> else can do this.
>>
> Hi Hans and Tim,
>
> No problem, I will repost the patch-set this week. I've been meaning to,
> just busy lately with other tasks.

Hi all, I need a few more days. I would like to bring in the video-switch
subdev from Pengutronix, which will replace the platform data set_video_mux
method. Also re-org the device-tree to better define all the possible 
hardware
connections, and split out mx6-encode.c into mx6-smfc and mx6-ic subdevs.
Once this is done we should have a better base for adding media control
later. I should have this done by end of this week.

Steve

Re: i.mx6 camera interface (CSI) and mainline kernel

[Jack Mitchell]

<snip>

>
> Hi all, I need a few more days. I would like to bring in the video-switch
> subdev from Pengutronix, which will replace the platform data set_video_mux
> method. Also re-org the device-tree to better define all the possible
> hardware
> connections, and split out mx6-encode.c into mx6-smfc and mx6-ic subdevs.
> Once this is done we should have a better base for adding media control
> later. I should have this done by end of this week.
>
> Steve
>

Hi Steve,

Just a heads up that I've also tested and confirmed partially working 
support on a sabrelite + mipi ov5640. The two gotchas I came across were 
dma_allocations fail in the higher resolutions (4x1080p buffers), a 
limit may need to be upped somewhere, the code says it should be able to 
handle up to 64MB? Secondly I can't get the 5MP resolution by default as 
in ov5640_find_nearest_mode, the array ov5640_mode_info_data is hard 
coded to 0.

I gave your v2 branch a quick whirl also but it fails to compile.

Thanks for the awesome work so far.

Cheers,
Jack.

---

Jack Mitchell
Tuxable Ltd
London, UK

Re: i.mx6 camera interface (CSI) and mainline kernel

[Steve Longerbeam]

On 06/10/2016 08:58 AM, Jack Mitchell wrote:
>
> <snip>
>
>>
>> Hi all, I need a few more days. I would like to bring in the video-switch
>> subdev from Pengutronix, which will replace the platform data set_video_mux
>> method. Also re-org the device-tree to better define all the possible
>> hardware
>> connections, and split out mx6-encode.c into mx6-smfc and mx6-ic subdevs.
>> Once this is done we should have a better base for adding media control
>> later. I should have this done by end of this week.
>>
>> Steve
>>
>
> Hi Steve,
>
> Just a heads up that I've also tested and confirmed partially working support on a sabrelite + mipi ov5640. The two gotchas I came across were dma_allocations fail in the higher resolutions (4x1080p
> buffers), a limit may need to be upped somewhere, the code says it should be able to handle up to 64MB? Secondly I can't get the 5MP resolution by default as in ov5640_find_nearest_mode, the array
> ov5640_mode_info_data is hard coded to 0.

Hi Jack, the ov5640 native 5MP (2592x1944) mode is only available at 15 fps, but
the default framerate is 30. So to allow the 5MP mode, set to 15 fps beforehand
with a call to vidioc_s_parm.

>
> I gave your v2 branch a quick whirl also but it fails to compile.

Yes, the v2 branch is the WIP I mentioned above. Making good progress and
hope to have a patchset to post in a few days.


Steve