Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Conor Dooley <conor@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 8859 bytes --]

Hey, just some quick thoughts I had, mostly to fish for more information
so I can be more informed..

On Wed, Sep 30, 2026 at 01:18:59PM +0200, Stefan Klug wrote:
> Hi everyone,
> 
> On 15.09.26 01:38, Laurent Pinchart wrote:
> > Hello,
> > 
> > As most of you already know, the Linux Plumbers Conference will host a
> > Camera & ISP BoF in Prague in three weeks ([1]).
> > 
> > The initial proposal for a microconference has unfortunately been
> > downgraded by the program committee to a 45 minutes BoF. We have
> > therefore decided to limit the discussion to three topics at most. Two
> > topics have been approved so far:
> > 
> > - RGB-IR support in V4L2 and libcamera
> > 
> >   by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> > 
> >   The Linux kernel V4L2 API does not support RGB-IR image sensors.
> >   While building blocks necessary to handle those devices are slowly
> >   being merged, RGB-IR support itself hasn't been tackled yet.
> > 
> >   Rishikesh and Devarsh will also present this topic at the OSS Europe
> >   conference ([2]).
> > 
> > - Camera module identification
> > 
> >   by Stefan Klug, Ideas on Board
> > 
> >   Camera tuning and calibration depend not only on the image sensor, but
> >   also on the lens and other characteristics of camera modules. While
> >   Linux supports identifying image sensors, it completely lacks the
> >   concept of camera modules. 
> > 
> >   Identification of camera modules requires coordination between
> >   platform firmware (DT or ACPI), the kernel and userspace. Discussions
> >   will benefit from the presence of DT maintainers.
> > 
> LPC is already next week and we won't have enough time on the BoF for a
> full introduction to the topic. So this email summarizes the Module
> identification topic and contains the questions I'd like to discuss.
> 
> To the device-tree maintainers: I added you to the invitation as this
> document proposes new device tree properties and I'd love to get some
> feedback and common understanding on these. So if you'd be able to join,
> that would be fantastic.

I'll do my best, there's some schedule conflicts with the RISC-V MC for
me, so we'll see how it goes.

> 
> Most of this mail is introduction and context. If you are short on time,
> please jump directly to the last headline "### 4. Camera modules without
> data storage" which contains the most relevant questions.

> 
> Q: Does anyone in this group have more details on which information to
> query and where to get more details from the system?
> A: ...
> 
> ### 2. Camera modules with an embedded EEPROM
> 
> To make use of the EEPROM data from user space we need to carry two
> elements: The EEPROM data itself and some identifier telling which
> format the data in the EEPROM has. This seems relatively straightforward
> and we could model it like this in dts:
> 
> dts-snippet:
> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     eeprom = <&eeprom>;
>     eeprom-format = "vendor-a-eeprom-fmt";

It's nvmem-cells you want here I believe, and the associated layouts
should cover the format?

> };
> 
> Getting the link to the EEPROM from user space is something to
> investigate. Ideas are either sysfs or media-controller.
> The eeprom-format must be specified as well, to tell user space how to
> interpret the EEPROM data, as it is typically not self-contained.

If you have the format of the eeprom, is it not better to provide the
parsed version to userspace directly rather? Or are you concerned that
these things will contain such "random" and non-standardisable info that
it is a lost cause?

> 
> Q: On a first discussion on that topic we were unsure if it would be
> preferred to expose the data directly in sysfs,
> or to symlink to the corresponding sysfs entry for the EEPROM. Any
> preferences?
> A: ...

FWIW, nvmem already has sysfs for this (first option in the nvmem
Kconfig menu).

> 
> Q: Any input/things to consider from ACPI side?
> A:
> 
> Q: What happens if the EEPROM driver is not loaded yet or is just not
> available? Should it block/defer probing of the sensor?
> A:
> 
> Q: We are missing real-life examples for these cases. So input from
> vendors would be welcome.
> A:
> 
> ### 3. Camera modules with a sensor that has an embedded OTP memory
> 
> This case is similar to the previous one. The big difference is that the
> OTP memory can be handled by the sensor driver itself, so no additional
> driver is necessary. In this case a practical solution could be to
> expose the OTP data via sysfs. Handling in user-space is similar to the
> EEPROM case. So the device driver could expose an otp_data and
> otp_format attribute in sysfs for user-space to read.
> 
> The property otp_format contains a name of the format of the OTP data.
> There are cases where the format is specific to the sensor vendor and
> cases where the format is specific to the module vendor. So it should be
> possible to specify that in the device tree.
> 
> ### 4. Camera modules without data storage
> 
> This is the most difficult and most controversial case. In this case we
> can't identify the module automatically by querying the hardware. It is
> also not possible to rely on the machine identifier as with example 1.
> Think of an arbitrary camera module connected to a Raspberry Pi - the
> machine identifier will not change just by connecting a camera.
> Therefore we'd like to supply all necessary information with the device
> tree.
> 
> The difficulty is that the combinations are huge and there is no common
> denominator (for some modules one might be able to get an SKU, for
> others it will even be difficult to get an official name).
> 
> This is quite contrary to typical device tree use-cases where we try to
> tie down everything as far as possible.
> A flat list of module identifiers (like compatible strings) won't work
> as it should be possible to standardize aspects of a module, even if not
> everything is know. Think of a Raspberry Pi Cam HQ with a screw-on lens.
> It would be great to be able to specify that module, but the lens is
> still up to the user.
> 
> In [1] I proposed a possible solution to that problem. A tag-based
> system with a single device tree entry of space separated tags. Every
> tag carries a prefix to denote a category:
> 
> Tag                 Example
> m:<ModuleName>      m:vc-mipi-imx296
> v:<Vendor>          v:RaspberryPi
> l:<Lens>            l:generic-8mm-ircut
> s:<Sku>             s:B030801
> u:<user defined>    u:my-selfmade-module
> 
> This is a bit like the existing "label" property which carries a human
> readable label. So an example for a module could be:
> 
> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     module-info = "v:RaspberryPi m:camera-module-hq
> l:generic-8mm-wide-angle";
> };
> 
> This leads to the immediate question: Should we split the tags into
> properties?

Yes, make use of the standard utilities before rolling your own format
please.

> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     module-name = "camera-module-hq";
>     module-vendor = "RaspberryPi"
>     module-lens = "generic-8mm-wide-angle";
> };
> 
> Pros: We could try to standardize on some values.
> Cons: We add a (possibly increasing) number of properties that have no
> value inside the kernel and will only be populated very sparsely.

What value do they have anywhere? ELI5 why anyone cares that the rpi
foundation made the sensor module, for example. What decisions can be
made on the basis of it? Not trying to be antagonistic, I genuinely
don't know what kind of things it affects. The lens type or some sort of
nd filter info I can understand being useful. That said, and I know very
little about cameras, some of these things could actually vary at runtime
right?

On the module name front, it quickly becomes compatible-v2 I think,
since you're gonna need standardised names for specific modules so that
decisions can be made on the basis of them. Makes me wonder if you need
sensors to have an endpoint connection to whatever sits in front of
them...

> Q: Would the dtb maintainers accept such a property?

I mean, at first glance the idea itself seems reasonable. The info isn't
discoverable and some of it seems to be valuable.

> A:
> 
> Q: Are there other/better ways to convey that information?
> A:
> 
> # Links:
> 
> - [1]
> https://www.linuxtv.org/downloads/presentations/media_summit_2026/Stefan%20-%20Module%20Identification.pdf
> - [2] https://github.com/systemd/systemd/blob/main/rules.d/70-camera.rules
> - [3] https://github.com/systemd/systemd/blob/main/hwdb.d/70-cameras.hwdb
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]