* Re: Configurable Video Controller Driver
2014-07-21 11:14 ` Sakari Ailus
@ 2014-07-31 8:53 ` Julien BERAUD
0 siblings, 0 replies; 3+ messages in thread
From: Julien BERAUD @ 2014-07-31 8:53 UTC (permalink / raw)
To: Sakari Ailus; +Cc: linux-media
Hi Sakari,
Thank you for your answer.
Le 21/07/2014 13:14, Sakari Ailus a écrit :
> Hi Julien,
>
> On Thu, Jul 10, 2014 at 04:19:06PM +0200, Julien BERAUD wrote:
>> We are developing a driver for our video controller which has the
>> particularity of being very reconfigurable.
>>
>> We have reached a point at which the complexity and variety of the
>> applications we need to implement forces us to
>> design an api/library that allows us to configure the
>> interconnection of the different video processing units(Camera
>> interfaces,
>> LCD interfaces, scalers, rotators, demosaicing, dead pixel
>> correction, etc...) from userland.
>>
>> The media controller api has the limitation of not being able to
>> create links but just browsing and activating/deactivating them.
>> If we just allowed a user to activate/deactivate links, then we
>> would have to declare all the possible connections between
>> the different blocks, which would make it very confusing from a
>> userland point of view. Moreover, the interconnection constraints
>> would have to be dealt with very generically, which would make it
>> very difficult in the kernel too.
> How many different blocks do you have? Can they be connected in arbitrary
> ways? If not, what kind of limitations do you have?
>
> The Media controller is originally intended for modelling complex devices
> with hardware data paths between the sub-blocks. The question is: does your
> device fit into that group, even if could be a little more complex than the
> devices that are currently supported?
We have 44 different hardware blocks that can be connected in arbitrary
ways but with some constraints. Some of the blocks have several
inputs(blenders) and some(one type only) have 2 outputs(the block is a
fork).
There are some limitations. Some blocks only accept Raw Bayer as
input(ISP). There are some small limitations here and there, like if you
use a scaler with a high scaling factor, you need to go to memory
because the internal memory is not sufficient.
Some of the blocks are FIFOs to write/read to memory.
In the end, there are 6 camera inputs and 2 lcd outputs, all of them
parallel, 2 ISPs with 2 blocks each(bayer/yuv), and 2 stat calculators,
12 Fifos, 4 color converters, 2 blenders with 4 inputs each, 2
scalers/rotators, 4 forks, 2 sat, 2 I3D, 2 gamma converters.
Our device might be a little more complex than the devices that are
currently supported, though I am not aware of all the currently
supported devices.
Something else, the Media controller doesn't allow to create links from
userland, which is what I think we would need in order to avoid to
declare all the possible combinations because that would be hard to
understand in userland and complicated to handle in the kernel.
We have come from a totally rigid interconnection declared in the BSP to
become more and more generic but it is probable that becoming too
generic wouldn't make things any better.
>
>> The conclusion we have reached yet is that we have to design an API
>> that allows us to create v4l2 subdevices that have certain
>> capabilities(scaling,rotating, demosaicing, etc...) and then to
>> create links between them from a userland library.
> Can you create arbitrary devices at will, or do these devices exist on
> hardware all the time?
>
The devices exist on hardware at all time. What I would like to do is to
create "virtual" subdevices that have some of the capabilities provided
by the blocks in order to benefit from all the features the hardware
provides in order to have a media controller tree that is clear and that
fits the needs of the current application. We are currently thinking
about designing a kernel driver along with a userland library that
allows us to reconfigure our hardware paths in order to create those
virtual subdevices and then expose them with the Media Controller/V4L2
subdev api.
I have tried to bring the topic up on the v4l irc because it is a
complicated matter. I will try again.
There may be a better way to handle such a complicated video controller,
but I haven't found it yet. If you have any idea, I am totally open to
reconsider the path we are taking.
In the end, it would be possible to handle each hardware block as an
independant subdev or input/output as long as we would be able to create
links with the media api but 2 considerations make it easier to do what
we are currently doing :
-> In our applications, most of the hardware paths between the blocks
won't move at runtime, just a few links would have to be reconfigured at
runtime which could be exposed with the media api.
-> We already have a low level api that allows us to handle the creation
of a hardware block that provide a list of capabilities.
Thank you,
Julien
^ permalink raw reply [flat|nested] 3+ messages in thread