Re: [PATCH 0/5] Introduce ancillary links

[Daniel Scally <djrscally@gmail.com>]

Hi Mauro

On 15/12/2021 09:25, Mauro Carvalho Chehab wrote:
> Em Mon, 13 Dec 2021 23:28:44 +0000
> Daniel Scally <djrscally@gmail.com> escreveu:
>
>> Hello all
>>
>> At present there's no means in the kernel of describing the supporting
>> relationship between subdevices that work together to form an effective single
>> unit - the type example in this case being a camera sensor and its
>> corresponding vcm. To attempt to solve that, this series adds a new type of
>> media link called MEDIA_LNK_FL_ANCILLARY_LINK, which connects two instances of
>> struct media_entity.
>>
>> The mechanism of connection I have modelled as a notifier and async subdev,
>> which seemed the best route since sensor drivers already typically will call
>> v4l2_async_register_subdev_sensor() on probe, and that function already looks
>> for a reference to a firmware node with the reference named "lens-focus". To
>> avoid boilerplate in the sensor drivers, I added some new functions in
>> v4l2-async that are called in v4l2_async_match_notify() to create the ancillary
>> links - checking the entity.function of both notifier and subdev to make sure
>> that's appropriate. I haven't gone further than that yet, but I suspect we could
>> cut down on code elsewhere by, for example, also creating pad-to-pad links in
>> the same place.
>>
>> Thoughts and comments very welcome 
> The idea of ancillary link sounds interesting. I did a quick look at
> the series. 
>
> Laurent already did some good points during his review.
> Besides that, one thing it is missing, though, is an implementation on
> a driver. At least vimc should gain an implementation at this series,
> in order to allow media developers to test and see how the graph will
> be after the patch series.


We have this running through libcamera at the moment; this series piggy
backs onto the notifier that's set up by
v4l2_async_create_sensor_subdev() so that the connection is made and the
links created when a lens controller is linked to the sensor via the
lens-focus firmware property. I've been testing this using the dw9719
driver I posted [1], plus a series that adds support to libcamera [2]. I
believe that some folks from Cros have also tested it with the dw9714
driver too.


[1]
https://lore.kernel.org/linux-media/20211128232115.38833-1-djrscally@gmail.com/

[2]
https://lists.libcamera.org/pipermail/libcamera-devel/2021-December/028082.html

>
> Regards,
> Mauro
>
>> Dan
>>
>> Daniel Scally (5):
>>   media: media.h: Add new media link type
>>   media: entity: Add link_type() helper
>>   media: entity: Skip non-data links in graph iteration
>>   media: entity: Add support for ancillary links
>>   media: v4l2-async: Create links during v4l2_async_match_notify()
>>
>>  drivers/media/mc/mc-entity.c         | 56 ++++++++++++++++++++++++++--
>>  drivers/media/v4l2-core/v4l2-async.c | 51 +++++++++++++++++++++++++
>>  include/media/media-entity.h         | 29 ++++++++++++++
>>  include/uapi/linux/media.h           |  1 +
>>  4 files changed, 134 insertions(+), 3 deletions(-)
>>
>
>
> Thanks,
> Mauro