Re: [Media Summit] Using the selection API to control image sensor subsampling

[Hans Verkuil <hverkuil@xs4all.nl>]

Hi Jacopo,

On 29/08/2022 09:15, Jacopo Mondi wrote:
> Hello
> 
>    I have prepared a slide deck to brief on the discussion for
> the topic specified in the subject. Slides are available at
> https://nc.nibble.pw/s/oib8jzNjjtgB9c6 as an attachment of ~5MB would
> probably be caught by the mailing list filter
> 
> I'll here copy a brief summary as presented in Hans' agenda:
> 
> -------------------------------------------------------------------------------
> 11:45 Jacopo: Representing addition sensor processing stages.
> 
>   How to represent additional processing stages that happens
>   on the sensor side, mostly additional subsampling/cropping that happen
>   between the analogue cropping on the full pixel array and the final
>   image sent on the wire.
> 
>   https://lore.kernel.org/linux-media/CAPY8ntA06L1Xsph79sv9t7MiDSNeSO2vADevuXZdXQdhWpSmow@mail.gmail.com/
> 
>   Dave made a good introduction of the issue his email which got
>   largely unanswered.
> 
>   The issue is particularly relevant for RAW sensors, where applying
>   subsampling has an impact on the sensor's sensitivity and requires
>   to adjust the gains and exposure accordingly.
> -------------------------------------------------------------------------------
> 
> The slide deck:
> 
> - Introduces the topic in slides [1-10]
> 
> - Presents Dave's proposed use case and how it is currently handled in
>   mainline drivers in slides [11-31] using as example the imx274,
>   mt9p031 and CCS drivers.
> 
> - Proposes a best-effort solution for imx219 in slides [32-37]
> 
> - The last two slides presents two discussion topics which I will copy
>   here
> 
>   ------------------------------------------------------------------------
>   Do we need new abstractions ?
> 
>   - An image sensor abstraction built on top of multiple subdevices
>     - Would it really make userspace any simpler ?
> 
>   - Image sensor specific SELECTION_TARGETS
>     - Conflicts with format selection are bound to happen with a single
>       source pad
>       - Can a “special” sink pad that represents the raw pixel array help ?
>     - Does not scale to represent complex sensors with multiple scalers
>   ------------------------------------------------------------------------
> 
> The imx219 proposal can be discussed by email or in person, while the
> questions in the last two slides might be worth being discussed during
> the meeting.

I've always wondered why binning and skipping wasn't implemented as controls.
You set them in userspace based on your needs, and they change the formats
and selection rectangles accordingly.

Trying to model it with selection rectangles is basically impossible.

For the discussion it would be good to know what kind of binning and skipping
schemes there are. Would it be fairly straightforward to translate it to a
control? Or are there many variations between vendors?

Regards,

	Hans

> 
> Thanks
>    j
>