How to configure resizer in ISP pipeline?

How to configure resizer in ISP pipeline?

[Andreas Nagel]

Hello,

using Media Controller API, I can successfully configure a simple ISP 
pipeline on an OMAP3530 and capture video data. Now I want to include 
the resizer. So the pipeline would look like this (where MEM would be 
the devnode corresponding to "resizer output"):

Sensor --> CCDC --> Resizer --> MEM

My "sensor" (TVP5146) already provides YUV data, so I can skip the 
previewer. I tried setting the input and output pad of the resizer 
subdevice to incoming resolution (input pad) and desired resolution 
(output pad). For example: 720x576 --> 352x288. But it didn't work out 
quite well.

Can someone explain how one properly configures the resizer in an ISP 
pipeline (with Media Controller API) ? I spend some hours researching, 
but this topic seems to be a well guarded secret...

Kind regards,
Andreas

Re: How to configure resizer in ISP pipeline?

[Sakari Ailus]

Hi Andreas,

On Tue, Dec 11, 2012 at 03:48:23PM +0100, Andreas Nagel wrote:
> Hello,
> 
> using Media Controller API, I can successfully configure a simple
> ISP pipeline on an OMAP3530 and capture video data. Now I want to
> include the resizer. So the pipeline would look like this (where MEM
> would be the devnode corresponding to "resizer output"):
> 
> Sensor --> CCDC --> Resizer --> MEM
> 
> My "sensor" (TVP5146) already provides YUV data, so I can skip the
> previewer. I tried setting the input and output pad of the resizer
> subdevice to incoming resolution (input pad) and desired resolution
> (output pad). For example: 720x576 --> 352x288. But it didn't work
> out quite well.

How did it not work quite well? :)

> Can someone explain how one properly configures the resizer in an
> ISP pipeline (with Media Controller API) ? I spend some hours
> researching, but this topic seems to be a well guarded secret...

There's nothing special about it. Really.

The resizing factor is chosen by setting the media bus format on the source
pad to the desired size.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: How to configure resizer in ISP pipeline?

[Andreas Nagel]

Hi Sakari,

thanks for helping.

>> My "sensor" (TVP5146) already provides YUV data, so I can skip the
>> previewer. I tried setting the input and output pad of the resizer
>> subdevice to incoming resolution (input pad) and desired resolution
>> (output pad). For example: 720x576 --> 352x288. But it didn't work
>> out quite well.
> How did it not work quite well? :)

Not sure, if I recall all the details. I haven't done much in this area 
for a few weeks now.
Currently, I actually can configure the resizer, but then 
VIDIOC_STREAMON fails with EINVAL when I configure the devnode. Don't 
know why.
I do connect the resizer source to the resizer output (devnode) and then 
capture from there. I think it is /dev/video6.
If I leave the resizer out and connect the ccdc source to the ccdc 
output (/dev/video2), capturing works just fine.

One reason could be, that the resizer isn't supported right now. (You 
remember, I have to use Technexion's TI kernel 2.6.37 with its exotic 
ISP driver. ;-) )
That's, what one could interpret from this TI wiki page.
http://processors.wiki.ti.com/index.php/UserGuideOmap35xCaptureDriver_PSP_04.02.00.07#Architecture
Under the block diagram, there's a note saying, that only the path with 
the continuous line has been validated. So, the dotted lines are ISP 
paths that might not have been validated ("supported"?) yet.


(I might add, that all this is part of my master thesis and resizing 
would be a nice-to-have goal, but not a must-have. So I can live with it 
if it won't work.)

>> Can someone explain how one properly configures the resizer in an
>> ISP pipeline (with Media Controller API) ? I spend some hours
>> researching, but this topic seems to be a well guarded secret...
> There's nothing special about it. Really.
>
> The resizing factor is chosen by setting the media bus format on the source
> pad to the desired size.

Yes, that is what I figured out eventually.

Re: How to configure resizer in ISP pipeline?

[Sakari Ailus]

Hi Andreas,

Andreas Nagel wrote:
> Hi Sakari,
> 
> thanks for helping.
> 
>>> My "sensor" (TVP5146) already provides YUV data, so I can skip the
>>> previewer. I tried setting the input and output pad of the resizer
>>> subdevice to incoming resolution (input pad) and desired resolution
>>> (output pad). For example: 720x576 --> 352x288. But it didn't work
>>> out quite well.
>> How did it not work quite well? :)
> 
> Not sure, if I recall all the details. I haven't done much in this area
> for a few weeks now.
> Currently, I actually can configure the resizer, but then
> VIDIOC_STREAMON fails with EINVAL when I configure the devnode. Don't
> know why.
> I do connect the resizer source to the resizer output (devnode) and then
> capture from there. I think it is /dev/video6.
> If I leave the resizer out and connect the ccdc source to the ccdc
> output (/dev/video2), capturing works just fine.
> 
> One reason could be, that the resizer isn't supported right now. (You
> remember, I have to use Technexion's TI kernel 2.6.37 with its exotic
> ISP driver. ;-) )

Ouch. There have been many bugfixes and improvements in the omap3isp
driver since it was merged to mainline in this area. I don't know what
TI has backported to their own kernel; it could be something is missing
there.

The fact the ABI and API are incompatible with what's now in mainline
suggests that quite a lot may be missing.

> That's, what one could interpret from this TI wiki page.
> http://processors.wiki.ti.com/index.php/UserGuideOmap35xCaptureDriver_PSP_04.02.00.07#Architecture
> 
> Under the block diagram, there's a note saying, that only the path with
> the continuous line has been validated. So, the dotted lines are ISP
> paths that might not have been validated ("supported"?) yet.

That's very possible.

> (I might add, that all this is part of my master thesis and resizing
> would be a nice-to-have goal, but not a must-have. So I can live with it
> if it won't work.)

Good to hear that... I guess the best starting point to get things
working would definitely be the mainline kernel. The rest is at best
unsupported.

-- 
Kind regards,

Sakari Ailus
sakari.ailus@iki.fi