[ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

Hello,

As most of you already know, the Linux Plumbers Conference will host a
Camera & ISP BoF in Prague in three weeks ([1]).

The initial proposal for a microconference has unfortunately been
downgraded by the program committee to a 45 minutes BoF. We have
therefore decided to limit the discussion to three topics at most. Two
topics have been approved so far:

- RGB-IR support in V4L2 and libcamera

  by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments

  The Linux kernel V4L2 API does not support RGB-IR image sensors.
  While building blocks necessary to handle those devices are slowly
  being merged, RGB-IR support itself hasn't been tackled yet.

  Rishikesh and Devarsh will also present this topic at the OSS Europe
  conference ([2]).

- Camera module identification

  by Stefan Klug, Ideas on Board

  Camera tuning and calibration depend not only on the image sensor, but
  also on the lens and other characteristics of camera modules. While
  Linux supports identifying image sensors, it completely lacks the
  concept of camera modules. 

  Identification of camera modules requires coordination between
  platform firmware (DT or ACPI), the kernel and userspace. Discussions
  will benefit from the presence of DT maintainers.

We can consider scheduling a third topic. Proposals are welcome.

Due to the 45 minutes BoF format, we will focus on discussions and won't
be able to afford presentations. All topic leads are expected to
circulate discussion materials on public mailing lists at least a week
before the event to give attendees time to read through proposals. The
materials should summarize the issue at hand and the ongoing work, and
clearly state the questions that will be discussed during the BoF.
Slides that support the discussions, if any, should also be submitted
prior to the event.

If there is enough interest, I am considering booking a meeting room on
Wednesday morning (location to be determined, likely outside of the LPC
venue) to extend the BoF. We could continue discussions on the topics
listed above, as well as schedule additional topics. We would overlap
with the following LPC microconferences:

- Networking Track
- Containers and checkpoint/restore MC
- Rust MC
- Scheduler and Real-Time MC
- Build Systems MC

Please let me know if you would be interested in attending a Wednesday
morning session by replying to this e-mail (publicly or privately). If
you would like to propose additional topics, please do so publicly.


[1] https://lpc.events/event/20/contributions/2364/
[2] https://osselceu2026.sched.com/event/2RaZj/enabling-multi-stream-camera-sensors-in-linux-rgb+ir-streams-and-embedded-metadata-rishikesh-donadkar-devarsh-thakkar-texas-instruments

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Bryan O'Donoghue]

On 15/09/2026 00:38, Laurent Pinchart wrote:
> We can consider scheduling a third topic. Proposals are welcome.

 From my LPC submission:

"Should offline ISPs be implemented in drm/accel"

Where does an offline Camera ISP belong in video4linux or in DRM and why ?

There are two main types of ISP - inline and offline. An inline ISP 
takes sensor input directly and produces some kind of processed image on 
the output. This is a natural fit for v4l2.

An offline ISP takes a memory buffer given to it and processes that 
buffer against supplied parameters. This model can be implemented as a 
video4linux m2m device but it could also be implemented as a DRM device 
and included in the DAG.

Taking Qualcomm SoCs as an example the Camera hardware provides two 
types of ISP. An inline processing engine IFE and an offline processing 
engine based on firmware ICP/HFI.

A use-case emerges to implement the inline ISP in v4l2 and the offline 
in drm/accel.

Some tension with this design would exist as user-space would have to be 
involved to update ISP parameters and process statistics. Another 
tension is including a firmware based system in dma-fencing including 
watchdogging and resetting if required.

There are clear benefits to having a loop of NPU/DSP to Offline ISP to 
Display that drm/accel seems to offer some interesting solutions to.

---
bod

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[johannes.goede]

Hi,

On 17-Sep-26 15:47, Bryan O'Donoghue wrote:
> On 15/09/2026 00:38, Laurent Pinchart wrote:
>> We can consider scheduling a third topic. Proposals are welcome.
> 
> From my LPC submission:
> 
> "Should offline ISPs be implemented in drm/accel"
> 
> Where does an offline Camera ISP belong in video4linux or in DRM and why ?
> 
> There are two main types of ISP - inline and offline. An inline ISP takes sensor input directly and produces some kind of processed image on the output. This is a natural fit for v4l2.
> 
> An offline ISP takes a memory buffer given to it and processes that buffer against supplied parameters. This model can be implemented as a video4linux m2m device but it could also be implemented as a DRM device and included in the DAG.
> 
> Taking Qualcomm SoCs as an example the Camera hardware provides two types of ISP. An inline processing engine IFE and an offline processing engine based on firmware ICP/HFI.

Note that one some Qualcomm SoCs the inline-engine can be switched to
offline mode and then run as a M2M ISP. Also the offline ISPs and
inline ISPs often have similar processing-blocks with similar or
identical parameters.

I do not think that splitting this over 2 subsystems is helpful and will
lead to a lot of duplication of efforts related to ISPs.

> A use-case emerges to implement the inline ISP in v4l2 and the offline in drm/accel.
> > Some tension with this design would exist as user-space would have to be involved to update ISP parameters and process statistics. Another tension is including a firmware based system in dma-fencing including watchdogging and resetting if required.
> 
> There are clear benefits to having a loop of NPU/DSP to Offline ISP to Display that drm/accel seems to offer some interesting solutions to.

This sounds to me like we need a way to better integrate drm and v4l2
subsystems. Other areas like hw-video-decoders might benefit from
having better integration there too.

Regards,

Hans

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

Hy Bryan,

On Thu, Sep 17, 2026 at 02:47:52PM +0100, Bryan O'Donoghue wrote:
> On 15/09/2026 00:38, Laurent Pinchart wrote:
> > We can consider scheduling a third topic. Proposals are welcome.
> 
>  From my LPC submission:
> 
> "Should offline ISPs be implemented in drm/accel"

I personally consider that this question has been discussed enough in
the past and I don't see any new development that would call for
reopening the debate. Proponents of ISP drivers in DRM haven't (in my
opinion) shown any substantial technical advantage, and the few
technical issues that have been pointed out can be addressed (and are
being addressed) in V4L2. I don't see allocating 15 minutes of BoF time
for this changing the status quo.

> Where does an offline Camera ISP belong in video4linux or in DRM and why ?
> 
> There are two main types of ISP - inline and offline. An inline ISP 
> takes sensor input directly and produces some kind of processed image on 
> the output. This is a natural fit for v4l2.
> 
> An offline ISP takes a memory buffer given to it and processes that 
> buffer against supplied parameters. This model can be implemented as a 
> video4linux m2m device but it could also be implemented as a DRM device 
> and included in the DAG.
> 
> Taking Qualcomm SoCs as an example the Camera hardware provides two 
> types of ISP. An inline processing engine IFE and an offline processing 
> engine based on firmware ICP/HFI.
> 
> A use-case emerges to implement the inline ISP in v4l2 and the offline 
> in drm/accel.
> 
> Some tension with this design would exist as user-space would have to be 
> involved to update ISP parameters and process statistics. Another 
> tension is including a firmware based system in dma-fencing including 
> watchdogging and resetting if required.
> 
> There are clear benefits to having a loop of NPU/DSP to Offline ISP to 
> Display that drm/accel seems to offer some interesting solutions to.

What are those interesting solutions ?

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Bryan O'Donoghue]

On 18/09/2026 10:18, Laurent Pinchart wrote:
>> There are clear benefits to having a loop of NPU/DSP to Offline ISP to
>> Display that drm/accel seems to offer some interesting solutions to.
> What are those interesting solutions ?

The chain I'm interested in is

Inline - V4L2 streaming
-> Produces raw or YUV

CVP (&& ||) NPU - Likely implemented as DRM
-> Produces motion vectors and/or further warps image

Offline - Sharpens, GTM, TNR (could be DRM, or v4lm2m)
-> Produces YUV

Display - DRM

V4L2 owning sensor -> inline engine is already supported and won't 
change. For subsequent steps though each stage hands off in-kernel 
without having to involve userspace.

Given though it depends if you need user-space to "do stuff" at the 
offline stage. If the offline stage has a firmware perhaps user-space 
doesn't need to be involved much past setup whereas if that offline 
stage has no firmware, then user-space really needs to drive it. The 
difference between the OPE on-list right now without a firmware and the 
ICP which does have a firmware.

Taking the example of sharing params and stats between Inline and 
Offline - in this case you are constructing a single ISP around two 
blocks basically in libcamera, whereas what I've described above 
traverses several blocks and isn't really an ISP - the ISP is already @ 
stage 1 - you have an additional fully offline block with its own 
firmware, sessions and runtime context. Stage 1 and stage 3 don't share 
kernel-side stats and params; they have their own isolated contexts, in 
contrast to TFE/OPE on Agatti where its integrated and must live in a 
dedicated libcamera pipeline.

But because you have a functional ISP @ stage 1 - the question is does 
the firmware based offline engine provide better overall system 
functionality as a DRM device ...

Its actually the question I'd like to discuss with colleagues in sunny 
Prague :)

So I think the tension comes from wanting to have the DRM scheduler but 
acknowledging that it is probably only for some use-cases possible.

You implement the device as a DRM device, you are still free to have a 
complex user-space context for that if that is your need - whereas if 
you have a use-case which can tolerate non-intervention from user-space, 
then the DRM scheduler _is_ what you want for complex hand-off from one 
device block to the other.

---
bod

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

On Sat, Sep 19, 2026 at 10:45:00PM +0100, Bryan O'Donoghue wrote:
> On 18/09/2026 10:18, Laurent Pinchart wrote:
> >> There are clear benefits to having a loop of NPU/DSP to Offline ISP to
> >> Display that drm/accel seems to offer some interesting solutions to.
> > What are those interesting solutions ?
> 
> The chain I'm interested in is
> 
> Inline - V4L2 streaming
> -> Produces raw or YUV
> 
> CVP (&& ||) NPU - Likely implemented as DRM

Possibly, but that's bit very relevant from a userspace point of view.
An NPU will be controlled through a userspace stack, likely involving
Mesa if the stack is open-source, and exposed through higher-level
frameworks/APIs (TF, ONNX, ...). The fact that the kernel driver lives
in DRM won't have any impact.

> -> Produces motion vectors and/or further warps image
> 
> Offline - Sharpens, GTM, TNR (could be DRM, or v4lm2m)
> -> Produces YUV
> 
> Display - DRM

So likely OpenGL or Vulkan, involving wayland or X11. In rare cases
directly interfacing with KMS for embedded devices.

> V4L2 owning sensor -> inline engine is already supported and won't 
> change. For subsequent steps though each stage hands off in-kernel 
> without having to involve userspace.

That's fences, isn't it ?

> Given though it depends if you need user-space to "do stuff" at the 
> offline stage. If the offline stage has a firmware perhaps user-space 
> doesn't need to be involved much past setup whereas if that offline 
> stage has no firmware, then user-space really needs to drive it. The 
> difference between the OPE on-list right now without a firmware and the 
> ICP which does have a firmware.

Cases where userspace wouldn't need to be involved exist, but then
there's even less userspace involvement than you describe above. The
firmware would control all the components directly (sensor, inline and
offline stages), exposing only the end result (possibly with the ability
to additionally capture raw frames).

> Taking the example of sharing params and stats between Inline and 
> Offline - in this case you are constructing a single ISP around two 
> blocks basically in libcamera, whereas what I've described above 
> traverses several blocks and isn't really an ISP - the ISP is already @ 
> stage 1 - you have an additional fully offline block with its own 
> firmware, sessions and runtime context. Stage 1 and stage 3 don't share 
> kernel-side stats and params; they have their own isolated contexts, in 
> contrast to TFE/OPE on Agatti where its integrated and must live in a 
> dedicated libcamera pipeline.
> 
> But because you have a functional ISP @ stage 1 - the question is does 
> the firmware based offline engine provide better overall system 
> functionality as a DRM device ...

I don't see why that would be the case.

> Its actually the question I'd like to discuss with colleagues in sunny 
> Prague :)
> 
> So I think the tension comes from wanting to have the DRM scheduler but 
> acknowledging that it is probably only for some use-cases possible.
> 
> You implement the device as a DRM device, you are still free to have a 
> complex user-space context for that if that is your need - whereas if 
> you have a use-case which can tolerate non-intervention from user-space, 
> then the DRM scheduler _is_ what you want for complex hand-off from one 
> device block to the other.

Not quite. It's a matter of fences.

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Bryan O'Donoghue]

On 19/09/2026 23:29, Laurent Pinchart wrote:
>> V4L2 owning sensor -> inline engine is already supported and won't
>> change. For subsequent steps though each stage hands off in-kernel
>> without having to involve userspace.
> That's fences, isn't it ?

Perhaps fences in v4l will do exactly the job I want. I'm more 
investigating than specifically pushing a solution.

My understanding of drm benefits are:

- drm_sched
   job scheduling based on dependency fences

- dma_resv
   fencing, sync

- syncobj
   user-space sync primitives which can be
   shared with say a compositor

- TDR
   Builtin fault tolerance.

Probably the best thing to do is to implement both a v4l and drm wrapper 
around the functionality and use it in anger.

There'd be no specific barrier to implementing a libcamera pipeline 
around a DRM device, I mean that's what gpuisp is even if its wrappered 
up in mesa so.. :)

Nicolas mentioned the idea of implementing codec drivers in DRM for 
reasons similar to the above.

Anyway I'm curious to know what the DRM advocates and detractors would 
say about that. It's a question I have not an assertion.

For an inline ISP where you connect a sensor to a CSI block the 
media-graph seems like a nice well defined thing.

The offline ISP I'm describing - truthfully has elements of codec and 
elements of ISP.

Probably the only way to really know is to experiment and see what falls 
out.

---
bod

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 3829 bytes --]

Hi,

Le lundi 21 septembre 2026 à 12:52 +0100, Bryan O'Donoghue a écrit :
> On 19/09/2026 23:29, Laurent Pinchart wrote:
> > > V4L2 owning sensor -> inline engine is already supported and won't
> > > change. For subsequent steps though each stage hands off in-kernel
> > > without having to involve userspace.
> > That's fences, isn't it ?
> 
> Perhaps fences in v4l will do exactly the job I want. I'm more 
> investigating than specifically pushing a solution.

It's something I've thought about a lot on the V4L2 side. So far, all proposals
failed to respect some dma fence contract. There is this strict relationship
that nothing can hold indefinitely on a job once submitted.

In V4L2 m2m (any flavour), jobs are implicit, we queue object, and the presence
of enough objects can trigger a job to start. If you deliver your sync file on
the queue operation, you will deliver a dma-fence without the guaranty it will
fire (or fail).

I've been wanting to solve this extending the request queue for codecs. While
you can queue a request before its matching OUTPUT buffer (which effectively let
user-space delay the completion), we could also query the driver upon if the
request has all the needed resources to be set on a "guarantied to run" queue.
With a new version of the request queue ioctl, we could emit the request fence.
Alternatively, since the request is already kind of a fence, we could export a
sync fs out of it, and do this extra validation/guaranty at the same time.

Could we have the same query from a QBUF(OUTPUT) operation on existing ISP
driver model ... maybe. I'll leave that to ISP drivers expert. What matters is
that userspace cannot pause the process (failing/cancelling it is fine). The
fence does not have to be strictly bound to a dmabuf buffer, its a sync point
where a buffer will be filled. The semantic of which buffer is to be delivered
next will have to be specified, its not the case for v4l2 stateful decoders
notably, which may pick different order to easy the reordering process.

> 
> My understanding of drm benefits are:
> 
> - drm_sched
>    job scheduling based on dependency fences
> 
> - dma_resv
>    fencing, sync
> 
> - syncobj
>    user-space sync primitives which can be
>    shared with say a compositor
> 
> - TDR
>    Builtin fault tolerance.
> 
> Probably the best thing to do is to implement both a v4l and drm wrapper 
> around the functionality and use it in anger.
> 
> There'd be no specific barrier to implementing a libcamera pipeline 
> around a DRM device, I mean that's what gpuisp is even if its wrappered 
> up in mesa so.. :)
> 
> Nicolas mentioned the idea of implementing codec drivers in DRM for 
> reasons similar to the above.

Nice moment to plug my referral track talk at LPC. Appart from early prototype,
it does not exist yet, hence why LPC is a great place to discuss, I already
introduced the subject during Media Summit in may.

https://lpc.events/event/20/contributions/2354/

On codecs, we aim for Vulkan Video as the API, which actually unify userspace
across platform and OS. Since its a low level API, very little design is needed.
Here, libcamera is your API I suppose, but its much higher level, so what the HW
specific driver should expose as API seems a bit more open.

cheers,
Nicolas

> 
> Anyway I'm curious to know what the DRM advocates and detractors would 
> say about that. It's a question I have not an assertion.
> 
> For an inline ISP where you connect a sensor to a CSI block the 
> media-graph seems like a nice well defined thing.
> 
> The offline ISP I'm describing - truthfully has elements of codec and 
> elements of ISP.
> 
> Probably the only way to really know is to experiment and see what falls 
> out.
> 
> ---
> bod

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[johannes.goede]

Hi,

On 15-Sep-26 01:38, Laurent Pinchart wrote:
> Hello,
> 
> As most of you already know, the Linux Plumbers Conference will host a
> Camera & ISP BoF in Prague in three weeks ([1]).
> 
> The initial proposal for a microconference has unfortunately been
> downgraded by the program committee to a 45 minutes BoF. We have
> therefore decided to limit the discussion to three topics at most. Two
> topics have been approved so far:
> 
> - RGB-IR support in V4L2 and libcamera
> 
>   by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> 
>   The Linux kernel V4L2 API does not support RGB-IR image sensors.
>   While building blocks necessary to handle those devices are slowly
>   being merged, RGB-IR support itself hasn't been tackled yet.
> 
>   Rishikesh and Devarsh will also present this topic at the OSS Europe
>   conference ([2]).

Related to this, we are seeing various people working on plain IR
sensors found on Windows Hello capable laptops, both on IPU6/IPU7
as well as on Qualcomm Snapdragon laptops.

I would like us to discuss and ideally agree on a userspace API for
the IR flood LED-s on these devices. I hope we can schedule some
time for this?

I actually just got send a Devicetree snippet off-list which enables
the IR flood LED on a Snapdragon laptop using the already existing
Flash LED support in a PMIC driver.

I think this, combined with an auxiliary media-controller link to
link the Flash LED class device to the IR sensor probably is the best
way to handle IR flood LEDs, but we do need consensus on this and
to document it.

We also may want to offer some sort of auto on in flash-light mode
when streaming support for these (1) to make things easier for userspace
or we clearly need to document that userspace needs to enable this
itself.

Regards,

Hans


1) With a v4l2-control to turn it off when userspace wants more
fine grained control.




> - Camera module identification
> 
>   by Stefan Klug, Ideas on Board
> 
>   Camera tuning and calibration depend not only on the image sensor, but
>   also on the lens and other characteristics of camera modules. While
>   Linux supports identifying image sensors, it completely lacks the
>   concept of camera modules. 
> 
>   Identification of camera modules requires coordination between
>   platform firmware (DT or ACPI), the kernel and userspace. Discussions
>   will benefit from the presence of DT maintainers.
> 
> We can consider scheduling a third topic. Proposals are welcome.
> 
> Due to the 45 minutes BoF format, we will focus on discussions and won't
> be able to afford presentations. All topic leads are expected to
> circulate discussion materials on public mailing lists at least a week
> before the event to give attendees time to read through proposals. The
> materials should summarize the issue at hand and the ongoing work, and
> clearly state the questions that will be discussed during the BoF.
> Slides that support the discussions, if any, should also be submitted
> prior to the event.
> 
> If there is enough interest, I am considering booking a meeting room on
> Wednesday morning (location to be determined, likely outside of the LPC
> venue) to extend the BoF. We could continue discussions on the topics
> listed above, as well as schedule additional topics. We would overlap
> with the following LPC microconferences:
> 
> - Networking Track
> - Containers and checkpoint/restore MC
> - Rust MC
> - Scheduler and Real-Time MC
> - Build Systems MC
> 
> Please let me know if you would be interested in attending a Wednesday
> morning session by replying to this e-mail (publicly or privately). If
> you would like to propose additional topics, please do so publicly.
> 
> 
> [1] https://lpc.events/event/20/contributions/2364/
> [2] https://osselceu2026.sched.com/event/2RaZj/enabling-multi-stream-camera-sensors-in-linux-rgb+ir-streams-and-embedded-metadata-rishikesh-donadkar-devarsh-thakkar-texas-instruments
> 

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[johannes.goede]

Hi,

On 15-Sep-26 01:38, Laurent Pinchart wrote:

<snip>

> If there is enough interest, I am considering booking a meeting room on
> Wednesday morning (location to be determined, likely outside of the LPC
> venue) to extend the BoF. We could continue discussions on the topics
> listed above, as well as schedule additional topics. We would overlap
> with the following LPC microconferences:
> 
> - Networking Track
> - Containers and checkpoint/restore MC
> - Rust MC
> - Scheduler and Real-Time MC
> - Build Systems MC
> 
> Please let me know if you would be interested in attending a Wednesday
> morning session by replying to this e-mail (publicly or privately). If
> you would like to propose additional topics, please do so publicly.

I would be interested in attending a Wednesday session about this.

A possible additional topic for such a Wednesday session would be
restarting the media-controller multi-context work (+Cc Jacopo,
Loic Poulain).

Regards,

Hans

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Stefan Klug]

Hi everyone,

On 15.09.26 01:38, Laurent Pinchart wrote:
> Hello,
> 
> As most of you already know, the Linux Plumbers Conference will host a
> Camera & ISP BoF in Prague in three weeks ([1]).
> 
> The initial proposal for a microconference has unfortunately been
> downgraded by the program committee to a 45 minutes BoF. We have
> therefore decided to limit the discussion to three topics at most. Two
> topics have been approved so far:
> 
> - RGB-IR support in V4L2 and libcamera
> 
>   by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> 
>   The Linux kernel V4L2 API does not support RGB-IR image sensors.
>   While building blocks necessary to handle those devices are slowly
>   being merged, RGB-IR support itself hasn't been tackled yet.
> 
>   Rishikesh and Devarsh will also present this topic at the OSS Europe
>   conference ([2]).
> 
> - Camera module identification
> 
>   by Stefan Klug, Ideas on Board
> 
>   Camera tuning and calibration depend not only on the image sensor, but
>   also on the lens and other characteristics of camera modules. While
>   Linux supports identifying image sensors, it completely lacks the
>   concept of camera modules. 
> 
>   Identification of camera modules requires coordination between
>   platform firmware (DT or ACPI), the kernel and userspace. Discussions
>   will benefit from the presence of DT maintainers.
> 
LPC is already next week and we won't have enough time on the BoF for a
full introduction to the topic. So this email summarizes the Module
identification topic and contains the questions I'd like to discuss.

To the device-tree maintainers: I added you to the invitation as this
document proposes new device tree properties and I'd love to get some
feedback and common understanding on these. So if you'd be able to join,
that would be fantastic.

Most of this mail is introduction and context. If you are short on time,
please jump directly to the last headline "### 4. Camera modules without
data storage" which contains the most relevant questions.

Disclaimer: Even though it uses markdown syntax, this document is
entirely written by a human and therefore the language is not as
polished as one might expect from an AI generated one - sorry for that :-)

Best regards,
Stefan


# Camera Module identification

## Problem statement

In many cases where a camera is involved it is a so called "complex
camera", meaning a sensor is connected to an Image Signal Processing
(ISP) unit that often is part of the SoC. In most cases the sensor is
soldered to a separate PCB together with a bit of electrical
infrastructure like regulators and an eeprom. In this document I call
these "camera modules". Sometimes camera modules are sold separately
like a Raspberry Pi Camera, in other cases the module is part of bigger
device like in a modern laptop.

Sensor and ISP are orchestrated by libcamera to create a proper image
from the RAW data produced by the sensor. To be able to do that
properly, libcamera needs a "tuning file" which contains tuning
parameters for a given camera module. These tuning parameters depend not
only on the sensor model, but also on physical properties of the camera
module. These are e.g. the material and geometry of the lens in the use,
the mounting position of the lens, in extreme cases even the individual
module due to production tolerances.

So to produce a good image, libcamera must be able to select the correct
tuning file for the camera module at hand. Currently libcamera only uses
the sensor name to select the tuning file. This is not sufficient as it
misses the properties mentioned above. There is a prototype of libcamera
that implements a simple scheme that collects as much information as
possible and chooses the best fitting tuning file (in this proposed
implementation the one that matches most tags). However the exact
implementation details are out of scope for this discussion. Side note
here: Tuning files are not necessarily part of libcamera, therefore the
selection process must be a able to select a tuning file that was not
known by libcamera at compile time.

To make this a bit more concrete, here are some examples for such cases
(these are also the reference cases that I'll use in the dicussion):

1. The webcam in a modern IPU6 based laptop (meaning Intel + ACPI) that
could include camera modules with the same sensor but from different
suppliers (meaning slightly different lenses and optical characteristics)

2. Camera modules with an embedded EEPROM that contains additional data.
Think of a camera module connected to a Raspberry Pi. The data stored in
the EEPROM could be anything from a SKU, a module name, or even
calibration data like colour correction matrices and lens shading
descriptions. The format of the data stored in the EEPROM is
vendor-specific.

3. Camera modules with a sensor that has an embedded OTP memory with
additional data. Same case as before, but the sensor driver has direct
access to the data and, depending on the use-case, also knows the
internal format of the data.

4. Camera modules without data storage with the same sensor but from
different manufacturers and with different lenses (screw-on or not, wide
angle, IR cut filter,...). Same case as before but no way to identify
the module. Compared to case 1 the modules are normally sold separately
and can therefore not be inferred from the main device.

Cases 1-3 are easier to solve and added here for completeness sake. Case
4 is the main point I'd like to discuss.

## Related topic (udev/hwdb)

After my talk at the Linux Media Summit 2026 I got the feedback to look
closer at hwdb. I did that and I think it could solve parts of the problem.

For those of us who (as me) are not that familiar with udev & hwdb here
is a quick summary:

Udev and hwdb are part of systemd. They can be used without systemd so
they are also used by non-systemd setups.
Udev consists of the udev-daemon that:

- Is notified when a device is added/removed
- Runs rules that
  - Create device symlinks
  - Query sysfs
  - Query the hwdb
  - Attach properties to devices
  - Attach tags to devices
  - ...

Interestingly there is already /lib/udev/v4l_id that is used by udev to
collect some V4L properties:

```
$ /lib/udev/v4l_id /dev/video6
ID_V4L_VERSION=2
ID_V4L_PRODUCT=HD Webcam C615
ID_V4L_CAPABILITIES=:
```

This is run by udev in /lib/udev/rules.d/60-persistent-v4l.rules
And essentially executes:
SUBSYSTEM=="video4linux", IMPORT{program}="v4l_id $devnode"

hwdb is a complementary utility that adds specific rules for known hardware.

In hwdb there are already some camera specific things:
70-camera.rules and 70-cameras.hwdb  [2],[3]
which set
```
ID_INFRARED_CAMERA=0|1
ID_CAMERA_DIRECTION=front|back
```

which are used in a similar manner:

```
SUBSYSTEM=="video4linux", ENV{ID_BUS}=="usb", \
  IMPORT{builtin}="hwdb
'camera:usb:v$env{ID_VENDOR_ID}p$env{ID_MODEL_ID}:name:$attr{name}:'"
```

So udev/hwdb provides an accepted way to gain more system knowledge and
to provide that to libcamera. It is an open discussion which parts of
the detection logic should live inside libcamera and which are better
located in udev.

## Diving into the examples

Lets look at the example I listed to see how these would be solved.

### 1. Laptop with camera module from various manufacturers

I believe this can be solved by either integrating the detection logic
in libcamera and/or adding rules to udev/hwdb. But we are lacking real
examples. So if you're able to provide identification strategies for
real, existing hardware, I'd be glad to know about it. One strategy
might be to use the machine identifier provided by DMI. This is however
quite coarse and can not handle cases where the manufacturer used
different camera modules from different vendors (using the same sensor
model). ARe there for example ACPI properties that we could query for
additional information?

Q: Does anyone in this group have more details on which information to
query and where to get more details from the system?
A: ...

### 2. Camera modules with an embedded EEPROM

To make use of the EEPROM data from user space we need to carry two
elements: The EEPROM data itself and some identifier telling which
format the data in the EEPROM has. This seems relatively straightforward
and we could model it like this in dts:

dts-snippet:
imx283_0: sensor@1a {
		compatible = "sony,imx283";
    ...
    eeprom = <&eeprom>;
    eeprom-format = "vendor-a-eeprom-fmt";
};

Getting the link to the EEPROM from user space is something to
investigate. Ideas are either sysfs or media-controller.
The eeprom-format must be specified as well, to tell user space how to
interpret the EEPROM data, as it is typically not self-contained.

Q: On a first discussion on that topic we were unsure if it would be
preferred to expose the data directly in sysfs,
or to symlink to the corresponding sysfs entry for the EEPROM. Any
preferences?
A: ...

Q: Any input/things to consider from ACPI side?
A:

Q: What happens if the EEPROM driver is not loaded yet or is just not
available? Should it block/defer probing of the sensor?
A:

Q: We are missing real-life examples for these cases. So input from
vendors would be welcome.
A:

### 3. Camera modules with a sensor that has an embedded OTP memory

This case is similar to the previous one. The big difference is that the
OTP memory can be handled by the sensor driver itself, so no additional
driver is necessary. In this case a practical solution could be to
expose the OTP data via sysfs. Handling in user-space is similar to the
EEPROM case. So the device driver could expose an otp_data and
otp_format attribute in sysfs for user-space to read.

The property otp_format contains a name of the format of the OTP data.
There are cases where the format is specific to the sensor vendor and
cases where the format is specific to the module vendor. So it should be
possible to specify that in the device tree.

### 4. Camera modules without data storage

This is the most difficult and most controversial case. In this case we
can't identify the module automatically by querying the hardware. It is
also not possible to rely on the machine identifier as with example 1.
Think of an arbitrary camera module connected to a Raspberry Pi - the
machine identifier will not change just by connecting a camera.
Therefore we'd like to supply all necessary information with the device
tree.

The difficulty is that the combinations are huge and there is no common
denominator (for some modules one might be able to get an SKU, for
others it will even be difficult to get an official name).

This is quite contrary to typical device tree use-cases where we try to
tie down everything as far as possible.
A flat list of module identifiers (like compatible strings) won't work
as it should be possible to standardize aspects of a module, even if not
everything is know. Think of a Raspberry Pi Cam HQ with a screw-on lens.
It would be great to be able to specify that module, but the lens is
still up to the user.

In [1] I proposed a possible solution to that problem. A tag-based
system with a single device tree entry of space separated tags. Every
tag carries a prefix to denote a category:

Tag                 Example
m:<ModuleName>      m:vc-mipi-imx296
v:<Vendor>          v:RaspberryPi
l:<Lens>            l:generic-8mm-ircut
s:<Sku>             s:B030801
u:<user defined>    u:my-selfmade-module

This is a bit like the existing "label" property which carries a human
readable label. So an example for a module could be:

imx283_0: sensor@1a {
		compatible = "sony,imx283";
    ...
    module-info = "v:RaspberryPi m:camera-module-hq
l:generic-8mm-wide-angle";
};

This leads to the immediate question: Should we split the tags into
properties?

imx283_0: sensor@1a {
		compatible = "sony,imx283";
    ...
    module-name = "camera-module-hq";
    module-vendor = "RaspberryPi"
    module-lens = "generic-8mm-wide-angle";
};

Pros: We could try to standardize on some values.
Cons: We add a (possibly increasing) number of properties that have no
value inside the kernel and will only be populated very sparsely.

Q: Would the dtb maintainers accept such a property?
A:

Q: Are there other/better ways to convey that information?
A:

# Links:

- [1]
https://www.linuxtv.org/downloads/presentations/media_summit_2026/Stefan%20-%20Module%20Identification.pdf
- [2] https://github.com/systemd/systemd/blob/main/rules.d/70-camera.rules
- [3] https://github.com/systemd/systemd/blob/main/hwdb.d/70-cameras.hwdb

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 8859 bytes --]

Hey, just some quick thoughts I had, mostly to fish for more information
so I can be more informed..

On Wed, Sep 30, 2026 at 01:18:59PM +0200, Stefan Klug wrote:
> Hi everyone,
> 
> On 15.09.26 01:38, Laurent Pinchart wrote:
> > Hello,
> > 
> > As most of you already know, the Linux Plumbers Conference will host a
> > Camera & ISP BoF in Prague in three weeks ([1]).
> > 
> > The initial proposal for a microconference has unfortunately been
> > downgraded by the program committee to a 45 minutes BoF. We have
> > therefore decided to limit the discussion to three topics at most. Two
> > topics have been approved so far:
> > 
> > - RGB-IR support in V4L2 and libcamera
> > 
> >   by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> > 
> >   The Linux kernel V4L2 API does not support RGB-IR image sensors.
> >   While building blocks necessary to handle those devices are slowly
> >   being merged, RGB-IR support itself hasn't been tackled yet.
> > 
> >   Rishikesh and Devarsh will also present this topic at the OSS Europe
> >   conference ([2]).
> > 
> > - Camera module identification
> > 
> >   by Stefan Klug, Ideas on Board
> > 
> >   Camera tuning and calibration depend not only on the image sensor, but
> >   also on the lens and other characteristics of camera modules. While
> >   Linux supports identifying image sensors, it completely lacks the
> >   concept of camera modules. 
> > 
> >   Identification of camera modules requires coordination between
> >   platform firmware (DT or ACPI), the kernel and userspace. Discussions
> >   will benefit from the presence of DT maintainers.
> > 
> LPC is already next week and we won't have enough time on the BoF for a
> full introduction to the topic. So this email summarizes the Module
> identification topic and contains the questions I'd like to discuss.
> 
> To the device-tree maintainers: I added you to the invitation as this
> document proposes new device tree properties and I'd love to get some
> feedback and common understanding on these. So if you'd be able to join,
> that would be fantastic.

I'll do my best, there's some schedule conflicts with the RISC-V MC for
me, so we'll see how it goes.

> 
> Most of this mail is introduction and context. If you are short on time,
> please jump directly to the last headline "### 4. Camera modules without
> data storage" which contains the most relevant questions.

> 
> Q: Does anyone in this group have more details on which information to
> query and where to get more details from the system?
> A: ...
> 
> ### 2. Camera modules with an embedded EEPROM
> 
> To make use of the EEPROM data from user space we need to carry two
> elements: The EEPROM data itself and some identifier telling which
> format the data in the EEPROM has. This seems relatively straightforward
> and we could model it like this in dts:
> 
> dts-snippet:
> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     eeprom = <&eeprom>;
>     eeprom-format = "vendor-a-eeprom-fmt";

It's nvmem-cells you want here I believe, and the associated layouts
should cover the format?

> };
> 
> Getting the link to the EEPROM from user space is something to
> investigate. Ideas are either sysfs or media-controller.
> The eeprom-format must be specified as well, to tell user space how to
> interpret the EEPROM data, as it is typically not self-contained.

If you have the format of the eeprom, is it not better to provide the
parsed version to userspace directly rather? Or are you concerned that
these things will contain such "random" and non-standardisable info that
it is a lost cause?

> 
> Q: On a first discussion on that topic we were unsure if it would be
> preferred to expose the data directly in sysfs,
> or to symlink to the corresponding sysfs entry for the EEPROM. Any
> preferences?
> A: ...

FWIW, nvmem already has sysfs for this (first option in the nvmem
Kconfig menu).

> 
> Q: Any input/things to consider from ACPI side?
> A:
> 
> Q: What happens if the EEPROM driver is not loaded yet or is just not
> available? Should it block/defer probing of the sensor?
> A:
> 
> Q: We are missing real-life examples for these cases. So input from
> vendors would be welcome.
> A:
> 
> ### 3. Camera modules with a sensor that has an embedded OTP memory
> 
> This case is similar to the previous one. The big difference is that the
> OTP memory can be handled by the sensor driver itself, so no additional
> driver is necessary. In this case a practical solution could be to
> expose the OTP data via sysfs. Handling in user-space is similar to the
> EEPROM case. So the device driver could expose an otp_data and
> otp_format attribute in sysfs for user-space to read.
> 
> The property otp_format contains a name of the format of the OTP data.
> There are cases where the format is specific to the sensor vendor and
> cases where the format is specific to the module vendor. So it should be
> possible to specify that in the device tree.
> 
> ### 4. Camera modules without data storage
> 
> This is the most difficult and most controversial case. In this case we
> can't identify the module automatically by querying the hardware. It is
> also not possible to rely on the machine identifier as with example 1.
> Think of an arbitrary camera module connected to a Raspberry Pi - the
> machine identifier will not change just by connecting a camera.
> Therefore we'd like to supply all necessary information with the device
> tree.
> 
> The difficulty is that the combinations are huge and there is no common
> denominator (for some modules one might be able to get an SKU, for
> others it will even be difficult to get an official name).
> 
> This is quite contrary to typical device tree use-cases where we try to
> tie down everything as far as possible.
> A flat list of module identifiers (like compatible strings) won't work
> as it should be possible to standardize aspects of a module, even if not
> everything is know. Think of a Raspberry Pi Cam HQ with a screw-on lens.
> It would be great to be able to specify that module, but the lens is
> still up to the user.
> 
> In [1] I proposed a possible solution to that problem. A tag-based
> system with a single device tree entry of space separated tags. Every
> tag carries a prefix to denote a category:
> 
> Tag                 Example
> m:<ModuleName>      m:vc-mipi-imx296
> v:<Vendor>          v:RaspberryPi
> l:<Lens>            l:generic-8mm-ircut
> s:<Sku>             s:B030801
> u:<user defined>    u:my-selfmade-module
> 
> This is a bit like the existing "label" property which carries a human
> readable label. So an example for a module could be:
> 
> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     module-info = "v:RaspberryPi m:camera-module-hq
> l:generic-8mm-wide-angle";
> };
> 
> This leads to the immediate question: Should we split the tags into
> properties?

Yes, make use of the standard utilities before rolling your own format
please.

> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     module-name = "camera-module-hq";
>     module-vendor = "RaspberryPi"
>     module-lens = "generic-8mm-wide-angle";
> };
> 
> Pros: We could try to standardize on some values.
> Cons: We add a (possibly increasing) number of properties that have no
> value inside the kernel and will only be populated very sparsely.

What value do they have anywhere? ELI5 why anyone cares that the rpi
foundation made the sensor module, for example. What decisions can be
made on the basis of it? Not trying to be antagonistic, I genuinely
don't know what kind of things it affects. The lens type or some sort of
nd filter info I can understand being useful. That said, and I know very
little about cameras, some of these things could actually vary at runtime
right?

On the module name front, it quickly becomes compatible-v2 I think,
since you're gonna need standardised names for specific modules so that
decisions can be made on the basis of them. Makes me wonder if you need
sensors to have an endpoint connection to whatever sits in front of
them...

> Q: Would the dtb maintainers accept such a property?

I mean, at first glance the idea itself seems reasonable. The info isn't
discoverable and some of it seems to be valuable.

> A:
> 
> Q: Are there other/better ways to convey that information?
> A:
> 
> # Links:
> 
> - [1]
> https://www.linuxtv.org/downloads/presentations/media_summit_2026/Stefan%20-%20Module%20Identification.pdf
> - [2] https://github.com/systemd/systemd/blob/main/rules.d/70-camera.rules
> - [3] https://github.com/systemd/systemd/blob/main/hwdb.d/70-cameras.hwdb
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

Hello Gjorgji,

On Fri, Sep 18, 2026 at 02:10:07PM +0300, Gjorgji Rosikopulos wrote:
> Hi Laurent, All,
> 
> On 9/15/26 02:38, Laurent Pinchart wrote:
> 
>     We can consider scheduling a third topic. Proposals are welcome.
> 
> 
> Below are my proposed discussion topics. All of them relate to advanced camera
> use cases:
> 
> 1. Electronic Image Stabilization (EIS).
> 2. Multi-frame Bayer or YUV noise reduction filters.
> 3. QUAD CFA reconstruction, including preprocessing algorithms for
> high-resolution Quad Bayer sensors.
> 4. AI-based denoising in Bayer or YUV domains, focusing on image preprocessing
> denoising algorithms.
> 5. DOL and video HDR use cases.
> 6. Background blur for video conferencing applications.
> 7. Face detection algorithms operating directly on image data.
> 8. Multicamera 360 image stitching etc.

Those are all interesting topics, but I won't be able to schedule all of
them in 15 minutes :-)

> All of these use cases and algorithm implementations can be vendor-specific or
> third-party solutions and may run on a GPU, DSP, or other accelerators.
> 
> I also think that, similar to the IPA used for 3A algorithms, we need an
> IPA-like for frame-processing algorithms.
> 
> As discussed, the only practical way to upstream such functionality into
> libcamera is to provide an open-source alternative (even a simplified
> implementation) that defines and covers the use case, including its input/
> output parameters and interfaces.
> 
> The main question is whether these algorithms should live inside libcamera or
> outside of it. Both approaches have their strengths, but I am personally in
> favor of integrating them into libcamera, perhaps as a layer above the pipeline
> handlers, for example through use case plugins or a similar mechanism.

I've long thought that libcamera should make pipeline handlers more
modular. Pipeline handlers would implement components for the pipeline
elements (such as the inline frontend and offline ISP) and declare how
those components are connected. We should be able to move most of the
logic required to run the pipeline into common code.

With a modular approach, inserting elements in the pipeline could become
easier. It's however not clear to me yet if we should try to separate
the processing required for the use cases you list above from the "core"
of pipeline handlers, into a separate layer on top, or if some use cases
would benefit more from being able to inject elements into the pipeline.

Note that we have started experimenting with the concept of layers, see
https://patchwork.libcamera.org/project/libcamera/list/?series=5753/

> We can also learn from Android's approach. Initially, the HAL3 and per-control
> APIs were created to allow applications to implement advanced algorithms such
> as HDR and EIS. Later, CameraX Extensions were introduced so that these
> capabilities could be shared across applications and benefit the broader
> ecosystem:
> 
> https://developer.android.com/media/camera/camerax/extensions-api

It's certainly something we should study more. As usual with Android
there's little documentation on the underlying architecture though :-(

> I believe Linux faces a similar challenge. Once PipeWire and GStreamer plugins
> based on libcamera become widely adopted across Linux distributions, it will
> become increasingly difficult to introduce these advanced use cases at higher
> layers of the software stack. Having a standardized framework within or closely
> integrated with libcamera could make these capabilities more accessible and
> reusable across applications.

No fundamental disagreement there, although we should also be careful
not to reimplement GStreamer or PipeWire inside libcamera.

An important question that will need to be answered is who will be
willing to do all this work.

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Rishikesh Donadkar]

On 15/09/26 05:08, Laurent Pinchart wrote:
> Hello,
> 
> As most of you already know, the Linux Plumbers Conference will host a
> Camera & ISP BoF in Prague in three weeks ([1]).
> 
> The initial proposal for a microconference has unfortunately been
> downgraded by the program committee to a 45 minutes BoF. We have
> therefore decided to limit the discussion to three topics at most. Two
> topics have been approved so far:
> 
> - RGB-IR support in V4L2 and libcamera
> 
>    by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> 
>    The Linux kernel V4L2 API does not support RGB-IR image sensors.
>    While building blocks necessary to handle those devices are slowly
>    being merged, RGB-IR support itself hasn't been tackled yet.
> 
>    Rishikesh and Devarsh will also present this topic at the OSS Europe
>    conference ([2]).

Hello everyone,

I have submitted an RFC for the OV2312 camera sensor driver[1]. This 
patch series proposes the use of array controls to facilitate setting 
controls for both the RGB and IR streams coming out of the camera 
subdevice.

I would greatly appreciate your feedback and thoughts on this approach.

[1] 
https://lore.kernel.org/all/20260925133001.2780868-1-r-donadkar@ti.com/#t

Regards,
Rishikesh

> 
> - Camera module identification
> 
>    by Stefan Klug, Ideas on Board
> 
>    Camera tuning and calibration depend not only on the image sensor, but
>    also on the lens and other characteristics of camera modules. While
>    Linux supports identifying image sensors, it completely lacks the
>    concept of camera modules.
> 
>    Identification of camera modules requires coordination between
>    platform firmware (DT or ACPI), the kernel and userspace. Discussions
>    will benefit from the presence of DT maintainers.
> 
> We can consider scheduling a third topic. Proposals are welcome.
> 
> Due to the 45 minutes BoF format, we will focus on discussions and won't
> be able to afford presentations. All topic leads are expected to
> circulate discussion materials on public mailing lists at least a week
> before the event to give attendees time to read through proposals. The
> materials should summarize the issue at hand and the ongoing work, and
> clearly state the questions that will be discussed during the BoF.
> Slides that support the discussions, if any, should also be submitted
> prior to the event.
> 
> If there is enough interest, I am considering booking a meeting room on
> Wednesday morning (location to be determined, likely outside of the LPC
> venue) to extend the BoF. We could continue discussions on the topics
> listed above, as well as schedule additional topics. We would overlap
> with the following LPC microconferences:
> 
> - Networking Track
> - Containers and checkpoint/restore MC
> - Rust MC
> - Scheduler and Real-Time MC
> - Build Systems MC
> 
> Please let me know if you would be interested in attending a Wednesday
> morning session by replying to this e-mail (publicly or privately). If
> you would like to propose additional topics, please do so publicly.
> 
> 
> [1] https://lpc.events/event/20/contributions/2364/
> [2] https://osselceu2026.sched.com/event/2RaZj/enabling-multi-stream-camera-sensors-in-linux-rgb+ir-streams-and-embedded-metadata-rishikesh-donadkar-devarsh-thakkar-texas-instruments
> 

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[johannes.goede]

Hi Stefan,

Thank you for the write-up.

On 30-Sep-26 13:18, Stefan Klug wrote:

<snip>

> ## Diving into the examples
> 
> Lets look at the example I listed to see how these would be solved.
> 
> ### 1. Laptop with camera module from various manufacturers
> 
> I believe this can be solved by either integrating the detection logic
> in libcamera and/or adding rules to udev/hwdb. But we are lacking real
> examples. So if you're able to provide identification strategies for
> real, existing hardware, I'd be glad to know about it. One strategy
> might be to use the machine identifier provided by DMI. This is however
> quite coarse and can not handle cases where the manufacturer used
> different camera modules from different vendors (using the same sensor
> model). ARe there for example ACPI properties that we could query for
> additional information?
> 
> Q: Does anyone in this group have more details on which information to
> query and where to get more details from the system?
> A: ...

For the Intel IPU6/IPU7 methods there is an ACPI call which the kernel
can do on the sensor fwnode object which will return a string identifying
the modules. These strings don't really have any fixed format, typically
they contain something which look like how some laptop serials look
just a random bunch of numbers + capital letters.

Userspace cannot get to this without the kernel exporting it.

Given all the udev talk, I think a sysfs attribute would make sense
for this. We can make the kernel do the ACPI call and if it is present
add a camera_module sysfs attribute to the v4l2-subdev for the sensor,
which will only be visible when there actually is a module-name.

On laptops using devicetree we could then fill this from a devicetree
property.


> 
> ### 2. Camera modules with an embedded EEPROM
> 
> To make use of the EEPROM data from user space we need to carry two
> elements: The EEPROM data itself and some identifier telling which
> format the data in the EEPROM has. This seems relatively straightforward
> and we could model it like this in dts:
> 
> dts-snippet:
> imx283_0: sensor@1a {
> 		compatible = "sony,imx283";
>     ...
>     eeprom = <&eeprom>;
>     eeprom-format = "vendor-a-eeprom-fmt";
> };
> 
> Getting the link to the EEPROM from user space is something to
> investigate. Ideas are either sysfs or media-controller.
> The eeprom-format must be specified as well, to tell user space how to
> interpret the EEPROM data, as it is typically not self-contained.
> 
> Q: On a first discussion on that topic we were unsure if it would be
> preferred to expose the data directly in sysfs,
> or to symlink to the corresponding sysfs entry for the EEPROM. Any
> preferences?
> A: ...
> 
> Q: Any input/things to consider from ACPI side?
> A:

For IPU6/IPU7 eeprom data is not used for camera-module identification
(AFAIK), it is all done through the (custom, Intel specific) ACPI call
I mentioned above.

> Q: What happens if the EEPROM driver is not loaded yet or is just not
> available? Should it block/defer probing of the sensor?
> A:
> 
> Q: We are missing real-life examples for these cases. So input from
> vendors would be welcome.
> A:
<snip>

Regards,

Hans

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

Hi Hans,

(CC'ing Sakari who may have more insight on ACPI properties for
Intel-based machines, or who may be able to pull the right people in
this conversation)

On Thu, Oct 01, 2026 at 05:15:05PM +0200, johannes.goede@oss.qualcomm.com wrote:
> Hi Stefan,
> 
> Thank you for the write-up.
> 
> On 30-Sep-26 13:18, Stefan Klug wrote:
> 
> <snip>
> 
> > ## Diving into the examples
> > 
> > Lets look at the example I listed to see how these would be solved.
> > 
> > ### 1. Laptop with camera module from various manufacturers
> > 
> > I believe this can be solved by either integrating the detection logic
> > in libcamera and/or adding rules to udev/hwdb. But we are lacking real
> > examples. So if you're able to provide identification strategies for
> > real, existing hardware, I'd be glad to know about it. One strategy
> > might be to use the machine identifier provided by DMI. This is however
> > quite coarse and can not handle cases where the manufacturer used
> > different camera modules from different vendors (using the same sensor
> > model). ARe there for example ACPI properties that we could query for
> > additional information?
> > 
> > Q: Does anyone in this group have more details on which information to
> > query and where to get more details from the system?
> > A: ...
> 
> For the Intel IPU6/IPU7 methods there is an ACPI call which the kernel
> can do on the sensor fwnode object which will return a string identifying
> the modules. These strings don't really have any fixed format, typically
> they contain something which look like how some laptop serials look
> just a random bunch of numbers + capital letters.

We know that hardware manufacturers like to have multiple providers for
camera modules in the same laptop model in order to avoid supply chain
issues. Do you know if this can be confirmed from those strings, do we
have enough data to see if the identification strings are made of a
small number of clusters ?

> Userspace cannot get to this without the kernel exporting it.
> 
> Given all the udev talk, I think a sysfs attribute would make sense
> for this. We can make the kernel do the ACPI call and if it is present
> add a camera_module sysfs attribute to the v4l2-subdev for the sensor,
> which will only be visible when there actually is a module-name.
> 
> On laptops using devicetree we could then fill this from a devicetree
> property.

I would very much like to standardize the API exposed to userspace
across different types of firmwares (ACPI and DT). Having more data
about the ACPI side could help us design DT properties that would fit
nicely with a single API.

> > ### 2. Camera modules with an embedded EEPROM
> > 
> > To make use of the EEPROM data from user space we need to carry two
> > elements: The EEPROM data itself and some identifier telling which
> > format the data in the EEPROM has. This seems relatively straightforward
> > and we could model it like this in dts:
> > 
> > dts-snippet:
> > imx283_0: sensor@1a {
> > 		compatible = "sony,imx283";
> >     ...
> >     eeprom = <&eeprom>;
> >     eeprom-format = "vendor-a-eeprom-fmt";
> > };
> > 
> > Getting the link to the EEPROM from user space is something to
> > investigate. Ideas are either sysfs or media-controller.
> > The eeprom-format must be specified as well, to tell user space how to
> > interpret the EEPROM data, as it is typically not self-contained.
> > 
> > Q: On a first discussion on that topic we were unsure if it would be
> > preferred to expose the data directly in sysfs,
> > or to symlink to the corresponding sysfs entry for the EEPROM. Any
> > preferences?
> > A: ...
> > 
> > Q: Any input/things to consider from ACPI side?
> > A:
> 
> For IPU6/IPU7 eeprom data is not used for camera-module identification
> (AFAIK), it is all done through the (custom, Intel specific) ACPI call
> I mentioned above.

I wouldn't expect an EEPROM, that would be quite costly for little gain.
I wonder if sensor OTP memory is typically used though. Some sensor
manufacturers specify how to store calibration data there, such as lens
shading tables for instance. I wonder if anyone puts the same data in
ACPI tables.

> > Q: What happens if the EEPROM driver is not loaded yet or is just not
> > available? Should it block/defer probing of the sensor?
> > A:
> > 
> > Q: We are missing real-life examples for these cases. So input from
> > vendors would be welcome.
> > A:
> <snip>

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

Hi Rishikesh,

On Thu, Oct 01, 2026 at 11:49:17AM +0530, Rishikesh Donadkar wrote:
> On 15/09/26 05:08, Laurent Pinchart wrote:
> > Hello,
> > 
> > As most of you already know, the Linux Plumbers Conference will host a
> > Camera & ISP BoF in Prague in three weeks ([1]).
> > 
> > The initial proposal for a microconference has unfortunately been
> > downgraded by the program committee to a 45 minutes BoF. We have
> > therefore decided to limit the discussion to three topics at most. Two
> > topics have been approved so far:
> > 
> > - RGB-IR support in V4L2 and libcamera
> > 
> >    by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> > 
> >    The Linux kernel V4L2 API does not support RGB-IR image sensors.
> >    While building blocks necessary to handle those devices are slowly
> >    being merged, RGB-IR support itself hasn't been tackled yet.
> > 
> >    Rishikesh and Devarsh will also present this topic at the OSS Europe
> >    conference ([2]).
> 
> Hello everyone,
> 
> I have submitted an RFC for the OV2312 camera sensor driver[1]. This 
> patch series proposes the use of array controls to facilitate setting 
> controls for both the RGB and IR streams coming out of the camera 
> subdevice.
> 
> I would greatly appreciate your feedback and thoughts on this approach.
> 
> [1] https://lore.kernel.org/all/20260925133001.2780868-1-r-donadkar@ti.com/#t

Thank you for doing so. I strongly encourage every attendee of the BoF
to at least read the proposal ahead of time, and ideally start
discussing it on the mailing list.

> > 
> > - Camera module identification
> > 
> >    by Stefan Klug, Ideas on Board
> > 
> >    Camera tuning and calibration depend not only on the image sensor, but
> >    also on the lens and other characteristics of camera modules. While
> >    Linux supports identifying image sensors, it completely lacks the
> >    concept of camera modules.
> > 
> >    Identification of camera modules requires coordination between
> >    platform firmware (DT or ACPI), the kernel and userspace. Discussions
> >    will benefit from the presence of DT maintainers.
> > 
> > We can consider scheduling a third topic. Proposals are welcome.
> > 
> > Due to the 45 minutes BoF format, we will focus on discussions and won't
> > be able to afford presentations. All topic leads are expected to
> > circulate discussion materials on public mailing lists at least a week
> > before the event to give attendees time to read through proposals. The
> > materials should summarize the issue at hand and the ongoing work, and
> > clearly state the questions that will be discussed during the BoF.
> > Slides that support the discussions, if any, should also be submitted
> > prior to the event.
> > 
> > If there is enough interest, I am considering booking a meeting room on
> > Wednesday morning (location to be determined, likely outside of the LPC
> > venue) to extend the BoF. We could continue discussions on the topics
> > listed above, as well as schedule additional topics. We would overlap
> > with the following LPC microconferences:
> > 
> > - Networking Track
> > - Containers and checkpoint/restore MC
> > - Rust MC
> > - Scheduler and Real-Time MC
> > - Build Systems MC
> > 
> > Please let me know if you would be interested in attending a Wednesday
> > morning session by replying to this e-mail (publicly or privately). If
> > you would like to propose additional topics, please do so publicly.
> > 
> > 
> > [1] https://lpc.events/event/20/contributions/2364/
> > [2] https://osselceu2026.sched.com/event/2RaZj/enabling-multi-stream-camera-sensors-in-linux-rgb+ir-streams-and-embedded-metadata-rishikesh-donadkar-devarsh-thakkar-texas-instruments

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Sakari Ailus]

Hi Laurent, Hans,

On Thu, Oct 01, 2026 at 11:55:05PM +0300, Laurent Pinchart wrote:
> Hi Hans,
> 
> (CC'ing Sakari who may have more insight on ACPI properties for
> Intel-based machines, or who may be able to pull the right people in
> this conversation)
> 
> On Thu, Oct 01, 2026 at 05:15:05PM +0200, johannes.goede@oss.qualcomm.com wrote:
> > Hi Stefan,
> > 
> > Thank you for the write-up.
> > 
> > On 30-Sep-26 13:18, Stefan Klug wrote:
> > 
> > <snip>
> > 
> > > ## Diving into the examples
> > > 
> > > Lets look at the example I listed to see how these would be solved.
> > > 
> > > ### 1. Laptop with camera module from various manufacturers
> > > 
> > > I believe this can be solved by either integrating the detection logic
> > > in libcamera and/or adding rules to udev/hwdb. But we are lacking real
> > > examples. So if you're able to provide identification strategies for
> > > real, existing hardware, I'd be glad to know about it. One strategy
> > > might be to use the machine identifier provided by DMI. This is however
> > > quite coarse and can not handle cases where the manufacturer used
> > > different camera modules from different vendors (using the same sensor
> > > model). ARe there for example ACPI properties that we could query for
> > > additional information?
> > > 
> > > Q: Does anyone in this group have more details on which information to
> > > query and where to get more details from the system?
> > > A: ...
> > 
> > For the Intel IPU6/IPU7 methods there is an ACPI call which the kernel
> > can do on the sensor fwnode object which will return a string identifying
> > the modules. These strings don't really have any fixed format, typically
> > they contain something which look like how some laptop serials look
> > just a random bunch of numbers + capital letters.
> 
> We know that hardware manufacturers like to have multiple providers for
> camera modules in the same laptop model in order to avoid supply chain
> issues. Do you know if this can be confirmed from those strings, do we
> have enough data to see if the identification strings are made of a
> small number of clusters ?

These strings are unique to the module. I'm afraid they could even be
module names as defined by the module vendor.

> 
> > Userspace cannot get to this without the kernel exporting it.
> > 
> > Given all the udev talk, I think a sysfs attribute would make sense
> > for this. We can make the kernel do the ACPI call and if it is present
> > add a camera_module sysfs attribute to the v4l2-subdev for the sensor,
> > which will only be visible when there actually is a module-name.
> > 
> > On laptops using devicetree we could then fill this from a devicetree
> > property.
> 
> I would very much like to standardize the API exposed to userspace
> across different types of firmwares (ACPI and DT). Having more data
> about the ACPI side could help us design DT properties that would fit
> nicely with a single API.

Note that this concerns only the Windows specific ACPI tables. DisCo for
Imaging (or older Chromebook ACPI tables following Linux specific
definitions) doesn't specify anything (largely because DT doesn't). Of
course it'd be nice to change that -- for both.

> 
> > > ### 2. Camera modules with an embedded EEPROM
> > > 
> > > To make use of the EEPROM data from user space we need to carry two
> > > elements: The EEPROM data itself and some identifier telling which
> > > format the data in the EEPROM has. This seems relatively straightforward
> > > and we could model it like this in dts:
> > > 
> > > dts-snippet:
> > > imx283_0: sensor@1a {
> > > 		compatible = "sony,imx283";
> > >     ...
> > >     eeprom = <&eeprom>;
> > >     eeprom-format = "vendor-a-eeprom-fmt";
> > > };
> > > 
> > > Getting the link to the EEPROM from user space is something to
> > > investigate. Ideas are either sysfs or media-controller.
> > > The eeprom-format must be specified as well, to tell user space how to
> > > interpret the EEPROM data, as it is typically not self-contained.
> > > 
> > > Q: On a first discussion on that topic we were unsure if it would be
> > > preferred to expose the data directly in sysfs,
> > > or to symlink to the corresponding sysfs entry for the EEPROM. Any
> > > preferences?
> > > A: ...
> > > 
> > > Q: Any input/things to consider from ACPI side?
> > > A:
> > 
> > For IPU6/IPU7 eeprom data is not used for camera-module identification
> > (AFAIK), it is all done through the (custom, Intel specific) ACPI call
> > I mentioned above.
> 
> I wouldn't expect an EEPROM, that would be quite costly for little gain.
> I wonder if sensor OTP memory is typically used though. Some sensor
> manufacturers specify how to store calibration data there, such as lens
> shading tables for instance. I wonder if anyone puts the same data in
> ACPI tables.

To my knowledge there's no tuning data in system firmware on Intel x86
systems. That could change in the future of course, but it's perhaps
unlikely, due to logistical issues.

> 
> > > Q: What happens if the EEPROM driver is not loaded yet or is just not
> > > available? Should it block/defer probing of the sensor?
> > > A:
> > > 
> > > Q: We are missing real-life examples for these cases. So input from
> > > vendors would be welcome.
> > > A:
> > <snip>
> 

-- 
Regards,

Sakari Ailus

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Stefan Klug]

Hi Conor,

Thank you for your fast reply.

Quoting Conor Dooley (2026-10-01 00:01:44)
> Hey, just some quick thoughts I had, mostly to fish for more information
> so I can be more informed..
> 
> On Wed, Sep 30, 2026 at 01:18:59PM +0200, Stefan Klug wrote:
> > Hi everyone,
> > 
> > On 15.09.26 01:38, Laurent Pinchart wrote:
> > > Hello,
> > > 
> > > As most of you already know, the Linux Plumbers Conference will host a
> > > Camera & ISP BoF in Prague in three weeks ([1]).
> > > 
> > > The initial proposal for a microconference has unfortunately been
> > > downgraded by the program committee to a 45 minutes BoF. We have
> > > therefore decided to limit the discussion to three topics at most. Two
> > > topics have been approved so far:
> > > 
> > > - RGB-IR support in V4L2 and libcamera
> > > 
> > >   by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> > > 
> > >   The Linux kernel V4L2 API does not support RGB-IR image sensors.
> > >   While building blocks necessary to handle those devices are slowly
> > >   being merged, RGB-IR support itself hasn't been tackled yet.
> > > 
> > >   Rishikesh and Devarsh will also present this topic at the OSS Europe
> > >   conference ([2]).
> > > 
> > > - Camera module identification
> > > 
> > >   by Stefan Klug, Ideas on Board
> > > 
> > >   Camera tuning and calibration depend not only on the image sensor, but
> > >   also on the lens and other characteristics of camera modules. While
> > >   Linux supports identifying image sensors, it completely lacks the
> > >   concept of camera modules. 
> > > 
> > >   Identification of camera modules requires coordination between
> > >   platform firmware (DT or ACPI), the kernel and userspace. Discussions
> > >   will benefit from the presence of DT maintainers.
> > > 
> > LPC is already next week and we won't have enough time on the BoF for a
> > full introduction to the topic. So this email summarizes the Module
> > identification topic and contains the questions I'd like to discuss.
> > 
> > To the device-tree maintainers: I added you to the invitation as this
> > document proposes new device tree properties and I'd love to get some
> > feedback and common understanding on these. So if you'd be able to join,
> > that would be fantastic.
> 
> I'll do my best, there's some schedule conflicts with the RISC-V MC for
> me, so we'll see how it goes.
> 
> > 
> > Most of this mail is introduction and context. If you are short on time,
> > please jump directly to the last headline "### 4. Camera modules without
> > data storage" which contains the most relevant questions.
> 
> > 
> > Q: Does anyone in this group have more details on which information to
> > query and where to get more details from the system?
> > A: ...
> > 
> > ### 2. Camera modules with an embedded EEPROM
> > 
> > To make use of the EEPROM data from user space we need to carry two
> > elements: The EEPROM data itself and some identifier telling which
> > format the data in the EEPROM has. This seems relatively straightforward
> > and we could model it like this in dts:
> > 
> > dts-snippet:
> > imx283_0: sensor@1a {
> >               compatible = "sony,imx283";
> >     ...
> >     eeprom = <&eeprom>;
> >     eeprom-format = "vendor-a-eeprom-fmt";
> 
> It's nvmem-cells you want here I believe, and the associated layouts
> should cover the format?

Oh, that looks good. nvmem-cells seems to be the right thing here. I'm
not completely sure how the format would map, but I believe that would
resolve itself on the first practical implementation.

> 
> > };
> > 
> > Getting the link to the EEPROM from user space is something to
> > investigate. Ideas are either sysfs or media-controller.
> > The eeprom-format must be specified as well, to tell user space how to
> > interpret the EEPROM data, as it is typically not self-contained.
> 
> If you have the format of the eeprom, is it not better to provide the
> parsed version to userspace directly rather? Or are you concerned that
> these things will contain such "random" and non-standardisable info that
> it is a lost cause?

As the data is of no use to the kernel, I don't see the benefit in
standardizing it. On the other hand we are missing practical examples. I
feel it is a bit of a chicken and egg situation. There were some
EEPROMSs, but it is difficult to correlate them with the camera module
so no one really used theme even though everyone would like to have
them.

> 
> > 
> > Q: On a first discussion on that topic we were unsure if it would be
> > preferred to expose the data directly in sysfs,
> > or to symlink to the corresponding sysfs entry for the EEPROM. Any
> > preferences?
> > A: ...
> 
> FWIW, nvmem already has sysfs for this (first option in the nvmem
> Kconfig menu).

Thanks for that info. I need to toy around with that.

> 
> > 
> > Q: Any input/things to consider from ACPI side?
> > A:
> > 
> > Q: What happens if the EEPROM driver is not loaded yet or is just not
> > available? Should it block/defer probing of the sensor?
> > A:
> > 
> > Q: We are missing real-life examples for these cases. So input from
> > vendors would be welcome.
> > A:
> > 
> > ### 3. Camera modules with a sensor that has an embedded OTP memory
> > 
> > This case is similar to the previous one. The big difference is that the
> > OTP memory can be handled by the sensor driver itself, so no additional
> > driver is necessary. In this case a practical solution could be to
> > expose the OTP data via sysfs. Handling in user-space is similar to the
> > EEPROM case. So the device driver could expose an otp_data and
> > otp_format attribute in sysfs for user-space to read.
> > 
> > The property otp_format contains a name of the format of the OTP data.
> > There are cases where the format is specific to the sensor vendor and
> > cases where the format is specific to the module vendor. So it should be
> > possible to specify that in the device tree.
> > 
> > ### 4. Camera modules without data storage
> > 
> > This is the most difficult and most controversial case. In this case we
> > can't identify the module automatically by querying the hardware. It is
> > also not possible to rely on the machine identifier as with example 1.
> > Think of an arbitrary camera module connected to a Raspberry Pi - the
> > machine identifier will not change just by connecting a camera.
> > Therefore we'd like to supply all necessary information with the device
> > tree.
> > 
> > The difficulty is that the combinations are huge and there is no common
> > denominator (for some modules one might be able to get an SKU, for
> > others it will even be difficult to get an official name).
> > 
> > This is quite contrary to typical device tree use-cases where we try to
> > tie down everything as far as possible.
> > A flat list of module identifiers (like compatible strings) won't work
> > as it should be possible to standardize aspects of a module, even if not
> > everything is know. Think of a Raspberry Pi Cam HQ with a screw-on lens.
> > It would be great to be able to specify that module, but the lens is
> > still up to the user.
> > 
> > In [1] I proposed a possible solution to that problem. A tag-based
> > system with a single device tree entry of space separated tags. Every
> > tag carries a prefix to denote a category:
> > 
> > Tag                 Example
> > m:<ModuleName>      m:vc-mipi-imx296
> > v:<Vendor>          v:RaspberryPi
> > l:<Lens>            l:generic-8mm-ircut
> > s:<Sku>             s:B030801
> > u:<user defined>    u:my-selfmade-module
> > 
> > This is a bit like the existing "label" property which carries a human
> > readable label. So an example for a module could be:
> > 
> > imx283_0: sensor@1a {
> >               compatible = "sony,imx283";
> >     ...
> >     module-info = "v:RaspberryPi m:camera-module-hq
> > l:generic-8mm-wide-angle";
> > };
> > 
> > This leads to the immediate question: Should we split the tags into
> > properties?
> 
> Yes, make use of the standard utilities before rolling your own format
> please.
> 
> > imx283_0: sensor@1a {
> >               compatible = "sony,imx283";
> >     ...
> >     module-name = "camera-module-hq";
> >     module-vendor = "RaspberryPi"
> >     module-lens = "generic-8mm-wide-angle";
> > };
> > 
> > Pros: We could try to standardize on some values.
> > Cons: We add a (possibly increasing) number of properties that have no
> > value inside the kernel and will only be populated very sparsely.
> 
> What value do they have anywhere? ELI5 why anyone cares that the rpi
> foundation made the sensor module, for example. What decisions can be
> made on the basis of it? Not trying to be antagonistic, I genuinely
> don't know what kind of things it affects. The lens type or some sort of
> nd filter info I can understand being useful. That said, and I know very
> little about cameras, some of these things could actually vary at runtime
> right?

I think the difficulty here is that we want to identify something which
we can't even name precisely and we don't yet see the full spectrum of
possibilities. Regarding the question who cares if the RPi foundation
manufactured a module: It is just a way of identifying a specific piece
of hardware. So there are many modules out there using an imx477 sensor,
but they could differ in the geometry from other ones.

A compatible string works well for devices we know. But then you can
quickly come up with cases, where only an aspect or part is known.

- Raspberry Pi Camera Module HQ
  - Lens is unknown
  - So we could start to unroll that
    - rpi,cmarea-module-hq-8mm-lens
    - rpi,cmarea-module-hq-9mm-lens
    - ... that doesn't scale

So we could introduce a module-lens property:
- vendor-a-8mm
  vendor-a-8mm-ir-cut
  vendor-b-8mm
  ... 1000 vendors more

That doesn't scale as well.

I tried to further reason about the underlying targets and why my
gutfeeling doesn't like the compatible.

I see two main targets: 

1. Finding a solution for well known devices like laptops or phones.
Upstreaming a compatible string per module could work, but we will often
just not know the vendor of the module. So they would be named
(incorrectly) after the manufacturer of the device.

2. Upstreaming overlays for well known camera modules like the ones from
RaspberryPi.  I don't expect that to gain too much traction quickly
because of missing specs for connectors and therefore no way to model a
camera module in dt in a way that works for multiple processing boards.
Still it would be very helpful to be able to write dt overlays for these
modules that work with an upstream kernel without the need for nasty
hacks (like abusing the label property or so).

3. The same applies for full custom solutions where upstreaming
compatible strings will never happen, but we need the overall mechanics.

So in summary I'm looking for a place to put a blob of information
regarding the camera module that is not of any use to the kernel and
only passed on to userspace. Defining a separate node and maintaining
compatible strings seems like a lot of maintenance work for little gain.

Looking for other places to put that information doesn't lead to really
good candidates. One could store it in the software image itself, but
that is a pain as a user would loose it when flashing a new image.

One could put it in a extra partition with the same issues as above and
a huge technical complexity for use cases like the RaspberryPi.

So the devicetree seems to be a good location for it, just that we're
lacking a place to put it there.

I hope that makes a bit of sense.

Best regards,
Stefan

> 
> On the module name front, it quickly becomes compatible-v2 I think,
> since you're gonna need standardised names for specific modules so that
> decisions can be made on the basis of them. Makes me wonder if you need
> sensors to have an endpoint connection to whatever sits in front of
> them...
> 
> > Q: Would the dtb maintainers accept such a property?
> 
> I mean, at first glance the idea itself seems reasonable. The info isn't
> discoverable and some of it seems to be valuable.
> 
> > A:
> > 
> > Q: Are there other/better ways to convey that information?
> > A:
> > 
> > # Links:
> > 
> > - [1]
> > https://www.linuxtv.org/downloads/presentations/media_summit_2026/Stefan%20-%20Module%20Identification.pdf
> > - [2] https://github.com/systemd/systemd/blob/main/rules.d/70-camera.rules
> > - [3] https://github.com/systemd/systemd/blob/main/hwdb.d/70-cameras.hwdb
> >

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Laurent Pinchart]

On Fri, Oct 02, 2026 at 09:56:23AM +0300, Sakari Ailus wrote:
> On Thu, Oct 01, 2026 at 11:55:05PM +0300, Laurent Pinchart wrote:
> > Hi Hans,
> > 
> > (CC'ing Sakari who may have more insight on ACPI properties for
> > Intel-based machines, or who may be able to pull the right people in
> > this conversation)
> > 
> > On Thu, Oct 01, 2026 at 05:15:05PM +0200, johannes.goede@oss.qualcomm.com wrote:
> > > Hi Stefan,
> > > 
> > > Thank you for the write-up.
> > > 
> > > On 30-Sep-26 13:18, Stefan Klug wrote:
> > > 
> > > <snip>
> > > 
> > > > ## Diving into the examples
> > > > 
> > > > Lets look at the example I listed to see how these would be solved.
> > > > 
> > > > ### 1. Laptop with camera module from various manufacturers
> > > > 
> > > > I believe this can be solved by either integrating the detection logic
> > > > in libcamera and/or adding rules to udev/hwdb. But we are lacking real
> > > > examples. So if you're able to provide identification strategies for
> > > > real, existing hardware, I'd be glad to know about it. One strategy
> > > > might be to use the machine identifier provided by DMI. This is however
> > > > quite coarse and can not handle cases where the manufacturer used
> > > > different camera modules from different vendors (using the same sensor
> > > > model). ARe there for example ACPI properties that we could query for
> > > > additional information?
> > > > 
> > > > Q: Does anyone in this group have more details on which information to
> > > > query and where to get more details from the system?
> > > > A: ...
> > > 
> > > For the Intel IPU6/IPU7 methods there is an ACPI call which the kernel
> > > can do on the sensor fwnode object which will return a string identifying
> > > the modules. These strings don't really have any fixed format, typically
> > > they contain something which look like how some laptop serials look
> > > just a random bunch of numbers + capital letters.
> > 
> > We know that hardware manufacturers like to have multiple providers for
> > camera modules in the same laptop model in order to avoid supply chain
> > issues. Do you know if this can be confirmed from those strings, do we
> > have enough data to see if the identification strings are made of a
> > small number of clusters ?
> 
> These strings are unique to the module. I'm afraid they could even be
> module names as defined by the module vendor.

Does that mean that, if a laptop manufacturer sources camera modules
equipped with the same sensor from different module manufacturers to
diversify its supply chain, a single machine model (as report by DMI)
would use a separate module identification string in ACPI for each
module type ?

> > > Userspace cannot get to this without the kernel exporting it.
> > > 
> > > Given all the udev talk, I think a sysfs attribute would make sense
> > > for this. We can make the kernel do the ACPI call and if it is present
> > > add a camera_module sysfs attribute to the v4l2-subdev for the sensor,
> > > which will only be visible when there actually is a module-name.
> > > 
> > > On laptops using devicetree we could then fill this from a devicetree
> > > property.
> > 
> > I would very much like to standardize the API exposed to userspace
> > across different types of firmwares (ACPI and DT). Having more data
> > about the ACPI side could help us design DT properties that would fit
> > nicely with a single API.
> 
> Note that this concerns only the Windows specific ACPI tables. DisCo for
> Imaging (or older Chromebook ACPI tables following Linux specific
> definitions) doesn't specify anything (largely because DT doesn't). Of
> course it'd be nice to change that -- for both.

Understood. The needs are likely the same though, userspace needs to
pick the right tuning data.

Do you know if Intel-based laptops typically ship all tuning files for a
device model in the same file system image and pick the appropriate file
based on data in ACPI tables, or if the system image is tailored to each
SKU and contains only the tuning file for the module used by that
particular machine instance ?

> > > > ### 2. Camera modules with an embedded EEPROM
> > > > 
> > > > To make use of the EEPROM data from user space we need to carry two
> > > > elements: The EEPROM data itself and some identifier telling which
> > > > format the data in the EEPROM has. This seems relatively straightforward
> > > > and we could model it like this in dts:
> > > > 
> > > > dts-snippet:
> > > > imx283_0: sensor@1a {
> > > > 		compatible = "sony,imx283";
> > > >     ...
> > > >     eeprom = <&eeprom>;
> > > >     eeprom-format = "vendor-a-eeprom-fmt";
> > > > };
> > > > 
> > > > Getting the link to the EEPROM from user space is something to
> > > > investigate. Ideas are either sysfs or media-controller.
> > > > The eeprom-format must be specified as well, to tell user space how to
> > > > interpret the EEPROM data, as it is typically not self-contained.
> > > > 
> > > > Q: On a first discussion on that topic we were unsure if it would be
> > > > preferred to expose the data directly in sysfs,
> > > > or to symlink to the corresponding sysfs entry for the EEPROM. Any
> > > > preferences?
> > > > A: ...
> > > > 
> > > > Q: Any input/things to consider from ACPI side?
> > > > A:
> > > 
> > > For IPU6/IPU7 eeprom data is not used for camera-module identification
> > > (AFAIK), it is all done through the (custom, Intel specific) ACPI call
> > > I mentioned above.
> > 
> > I wouldn't expect an EEPROM, that would be quite costly for little gain.
> > I wonder if sensor OTP memory is typically used though. Some sensor
> > manufacturers specify how to store calibration data there, such as lens
> > shading tables for instance. I wonder if anyone puts the same data in
> > ACPI tables.
> 
> To my knowledge there's no tuning data in system firmware on Intel x86
> systems. That could change in the future of course, but it's perhaps
> unlikely, due to logistical issues.

That's my guess too.

I half recall that Linux-based Nokia phones contained camera tuning data
in a special flash partition, but I don't have much details.

> > > > Q: What happens if the EEPROM driver is not loaded yet or is just not
> > > > available? Should it block/defer probing of the sensor?
> > > > A:
> > > > 
> > > > Q: We are missing real-life examples for these cases. So input from
> > > > vendors would be welcome.
> > > > A:
> > > <snip>

-- 
Regards,

Laurent Pinchart

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Loic Poulain]

On Fri, Oct 2, 2026 at 6:14 PM Stefan Klug <stefan.klug@ideasonboard.com> wrote:
>
> Hi Conor,
>
> Thank you for your fast reply.
>
> Quoting Conor Dooley (2026-10-01 00:01:44)
> > Hey, just some quick thoughts I had, mostly to fish for more information
> > so I can be more informed..
> >
> > On Wed, Sep 30, 2026 at 01:18:59PM +0200, Stefan Klug wrote:
> > > Hi everyone,
> > >
> > > On 15.09.26 01:38, Laurent Pinchart wrote:
> > > > Hello,
> > > >
> > > > As most of you already know, the Linux Plumbers Conference will host a
> > > > Camera & ISP BoF in Prague in three weeks ([1]).
> > > >
> > > > The initial proposal for a microconference has unfortunately been
> > > > downgraded by the program committee to a 45 minutes BoF. We have
> > > > therefore decided to limit the discussion to three topics at most. Two
> > > > topics have been approved so far:
> > > >
> > > > - RGB-IR support in V4L2 and libcamera
> > > >
> > > >   by Rishikesh Donadkar and Devarsh Thakkar, Texas Instruments
> > > >
> > > >   The Linux kernel V4L2 API does not support RGB-IR image sensors.
> > > >   While building blocks necessary to handle those devices are slowly
> > > >   being merged, RGB-IR support itself hasn't been tackled yet.
> > > >
> > > >   Rishikesh and Devarsh will also present this topic at the OSS Europe
> > > >   conference ([2]).
> > > >
> > > > - Camera module identification
> > > >
> > > >   by Stefan Klug, Ideas on Board
> > > >
> > > >   Camera tuning and calibration depend not only on the image sensor, but
> > > >   also on the lens and other characteristics of camera modules. While
> > > >   Linux supports identifying image sensors, it completely lacks the
> > > >   concept of camera modules.
> > > >
> > > >   Identification of camera modules requires coordination between
> > > >   platform firmware (DT or ACPI), the kernel and userspace. Discussions
> > > >   will benefit from the presence of DT maintainers.
> > > >
> > > LPC is already next week and we won't have enough time on the BoF for a
> > > full introduction to the topic. So this email summarizes the Module
> > > identification topic and contains the questions I'd like to discuss.
> > >
> > > To the device-tree maintainers: I added you to the invitation as this
> > > document proposes new device tree properties and I'd love to get some
> > > feedback and common understanding on these. So if you'd be able to join,
> > > that would be fantastic.
> >
> > I'll do my best, there's some schedule conflicts with the RISC-V MC for
> > me, so we'll see how it goes.
> >
> > >
> > > Most of this mail is introduction and context. If you are short on time,
> > > please jump directly to the last headline "### 4. Camera modules without
> > > data storage" which contains the most relevant questions.
> >
> > >
> > > Q: Does anyone in this group have more details on which information to
> > > query and where to get more details from the system?
> > > A: ...
> > >
> > > ### 2. Camera modules with an embedded EEPROM
> > >
> > > To make use of the EEPROM data from user space we need to carry two
> > > elements: The EEPROM data itself and some identifier telling which
> > > format the data in the EEPROM has. This seems relatively straightforward
> > > and we could model it like this in dts:
> > >
> > > dts-snippet:
> > > imx283_0: sensor@1a {
> > >               compatible = "sony,imx283";
> > >     ...
> > >     eeprom = <&eeprom>;
> > >     eeprom-format = "vendor-a-eeprom-fmt";
> >
> > It's nvmem-cells you want here I believe, and the associated layouts
> > should cover the format?
>
> Oh, that looks good. nvmem-cells seems to be the right thing here. I'm
> not completely sure how the format would map, but I believe that would
> resolve itself on the first practical implementation.
>
> >
> > > };
> > >
> > > Getting the link to the EEPROM from user space is something to
> > > investigate. Ideas are either sysfs or media-controller.
> > > The eeprom-format must be specified as well, to tell user space how to
> > > interpret the EEPROM data, as it is typically not self-contained.
> >
> > If you have the format of the eeprom, is it not better to provide the
> > parsed version to userspace directly rather? Or are you concerned that
> > these things will contain such "random" and non-standardisable info that
> > it is a lost cause?
>
> As the data is of no use to the kernel, I don't see the benefit in
> standardizing it. On the other hand we are missing practical examples. I
> feel it is a bit of a chicken and egg situation. There were some
> EEPROMSs, but it is difficult to correlate them with the camera module
> so no one really used theme even though everyone would like to have
> them.
>
> >
> > >
> > > Q: On a first discussion on that topic we were unsure if it would be
> > > preferred to expose the data directly in sysfs,
> > > or to symlink to the corresponding sysfs entry for the EEPROM. Any
> > > preferences?
> > > A: ...
> >
> > FWIW, nvmem already has sysfs for this (first option in the nvmem
> > Kconfig menu).
>
> Thanks for that info. I need to toy around with that.
>
> >
> > >
> > > Q: Any input/things to consider from ACPI side?
> > > A:
> > >
> > > Q: What happens if the EEPROM driver is not loaded yet or is just not
> > > available? Should it block/defer probing of the sensor?
> > > A:
> > >
> > > Q: We are missing real-life examples for these cases. So input from
> > > vendors would be welcome.
> > > A:
> > >
> > > ### 3. Camera modules with a sensor that has an embedded OTP memory
> > >
> > > This case is similar to the previous one. The big difference is that the
> > > OTP memory can be handled by the sensor driver itself, so no additional
> > > driver is necessary. In this case a practical solution could be to
> > > expose the OTP data via sysfs. Handling in user-space is similar to the
> > > EEPROM case. So the device driver could expose an otp_data and
> > > otp_format attribute in sysfs for user-space to read.
> > >
> > > The property otp_format contains a name of the format of the OTP data.
> > > There are cases where the format is specific to the sensor vendor and
> > > cases where the format is specific to the module vendor. So it should be
> > > possible to specify that in the device tree.
> > >
> > > ### 4. Camera modules without data storage
> > >
> > > This is the most difficult and most controversial case. In this case we
> > > can't identify the module automatically by querying the hardware. It is
> > > also not possible to rely on the machine identifier as with example 1.
> > > Think of an arbitrary camera module connected to a Raspberry Pi - the
> > > machine identifier will not change just by connecting a camera.
> > > Therefore we'd like to supply all necessary information with the device
> > > tree.
> > >
> > > The difficulty is that the combinations are huge and there is no common
> > > denominator (for some modules one might be able to get an SKU, for
> > > others it will even be difficult to get an official name).
> > >
> > > This is quite contrary to typical device tree use-cases where we try to
> > > tie down everything as far as possible.
> > > A flat list of module identifiers (like compatible strings) won't work
> > > as it should be possible to standardize aspects of a module, even if not
> > > everything is know. Think of a Raspberry Pi Cam HQ with a screw-on lens.
> > > It would be great to be able to specify that module, but the lens is
> > > still up to the user.
> > >
> > > In [1] I proposed a possible solution to that problem. A tag-based
> > > system with a single device tree entry of space separated tags. Every
> > > tag carries a prefix to denote a category:
> > >
> > > Tag                 Example
> > > m:<ModuleName>      m:vc-mipi-imx296
> > > v:<Vendor>          v:RaspberryPi
> > > l:<Lens>            l:generic-8mm-ircut
> > > s:<Sku>             s:B030801
> > > u:<user defined>    u:my-selfmade-module
> > >
> > > This is a bit like the existing "label" property which carries a human
> > > readable label. So an example for a module could be:
> > >
> > > imx283_0: sensor@1a {
> > >               compatible = "sony,imx283";
> > >     ...
> > >     module-info = "v:RaspberryPi m:camera-module-hq
> > > l:generic-8mm-wide-angle";
> > > };
> > >
> > > This leads to the immediate question: Should we split the tags into
> > > properties?
> >
> > Yes, make use of the standard utilities before rolling your own format
> > please.
> >
> > > imx283_0: sensor@1a {
> > >               compatible = "sony,imx283";
> > >     ...
> > >     module-name = "camera-module-hq";
> > >     module-vendor = "RaspberryPi"
> > >     module-lens = "generic-8mm-wide-angle";
> > > };
> > >
> > > Pros: We could try to standardize on some values.
> > > Cons: We add a (possibly increasing) number of properties that have no
> > > value inside the kernel and will only be populated very sparsely.
> >
> > What value do they have anywhere? ELI5 why anyone cares that the rpi
> > foundation made the sensor module, for example. What decisions can be
> > made on the basis of it? Not trying to be antagonistic, I genuinely
> > don't know what kind of things it affects. The lens type or some sort of
> > nd filter info I can understand being useful. That said, and I know very
> > little about cameras, some of these things could actually vary at runtime
> > right?
>
> I think the difficulty here is that we want to identify something which
> we can't even name precisely and we don't yet see the full spectrum of
> possibilities. Regarding the question who cares if the RPi foundation
> manufactured a module: It is just a way of identifying a specific piece
> of hardware. So there are many modules out there using an imx477 sensor,
> but they could differ in the geometry from other ones.
>
> A compatible string works well for devices we know. But then you can
> quickly come up with cases, where only an aspect or part is known.
>
> - Raspberry Pi Camera Module HQ
>   - Lens is unknown
>   - So we could start to unroll that
>     - rpi,cmarea-module-hq-8mm-lens
>     - rpi,cmarea-module-hq-9mm-lens
>     - ... that doesn't scale
>
> So we could introduce a module-lens property:
> - vendor-a-8mm
>   vendor-a-8mm-ir-cut
>   vendor-b-8mm
>   ... 1000 vendors more
>
> That doesn't scale as well.
>
> I tried to further reason about the underlying targets and why my
> gutfeeling doesn't like the compatible.
>
> I see two main targets:
>
> 1. Finding a solution for well known devices like laptops or phones.
> Upstreaming a compatible string per module could work, but we will often
> just not know the vendor of the module. So they would be named
> (incorrectly) after the manufacturer of the device.
>
> 2. Upstreaming overlays for well known camera modules like the ones from
> RaspberryPi.  I don't expect that to gain too much traction quickly
> because of missing specs for connectors and therefore no way to model a
> camera module in dt in a way that works for multiple processing boards.
> Still it would be very helpful to be able to write dt overlays for these
> modules that work with an upstream kernel without the need for nasty
> hacks (like abusing the label property or so).
>
> 3. The same applies for full custom solutions where upstreaming
> compatible strings will never happen, but we need the overall mechanics.
>
> So in summary I'm looking for a place to put a blob of information
> regarding the camera module that is not of any use to the kernel and
> only passed on to userspace. Defining a separate node and maintaining
> compatible strings seems like a lot of maintenance work for little gain.
>
> Looking for other places to put that information doesn't lead to really
> good candidates. One could store it in the software image itself, but
> that is a pain as a user would loose it when flashing a new image.

There are special hardware or logical partitions on eMMC/UFS devices
(e.g. mmcblk0boot0). These partitions are typically preserved across
normal image flashing and even full user-area erases (mmcblk0), making
them a convenient place to store device-specific information when no
dedicated OTP or EEPROM is available. For example, Arduino Uno Q
stores Wi-Fi and Bluetooth addresses this way.

That said, if consumers rely on nvmem-cells, the backend becomes
irrelevant. The data may come from an external EEPROM, SoC OTP,
eMMC/UFS boot partition, sensor OTP, or any other NVMEM provider,
allowing multiple provisioning methods without affecting users.

I also think some level of abstraction should remain in V4L2. While
nvmem-cells can be used to reference the data, the sensor module
attributes should also be definable directly in DT (or ACPI). This
would allow multiple ways of providing the same information behind a
common interface.
Similar to networking, device_get_mac_address() can retrieve the MAC
address either directly from firmware (DT/ACPI) or from an NVMEM cell.
The consumer does not need to care where the data originates.

So in the end, exposing this through a dedicated sysfs attribute sounds
reasonable, perhaps following a model similar to DRM's EDID.

Regards,
Loic


>
> One could put it in a extra partition with the same issues as above and
> a huge technical complexity for use cases like the RaspberryPi.
>
> So the devicetree seems to be a good location for it, just that we're
> lacking a place to put it there.
>
> I hope that makes a bit of sense.

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Gjorgji Rosikopulos]

Hi Laurent,

On 10/1/26 02:23, Laurent Pinchart wrote:
> Hello Gjorgji,
> 
> On Fri, Sep 18, 2026 at 02:10:07PM +0300, Gjorgji Rosikopulos wrote:
>> Hi Laurent, All,
>>
>> On 9/15/26 02:38, Laurent Pinchart wrote:
>>
>>     We can consider scheduling a third topic. Proposals are welcome.
>>
>>
>> Below are my proposed discussion topics. All of them relate to advanced camera
>> use cases:
>>
>> 1. Electronic Image Stabilization (EIS).
>> 2. Multi-frame Bayer or YUV noise reduction filters.
>> 3. QUAD CFA reconstruction, including preprocessing algorithms for
>> high-resolution Quad Bayer sensors.
>> 4. AI-based denoising in Bayer or YUV domains, focusing on image preprocessing
>> denoising algorithms.
>> 5. DOL and video HDR use cases.
>> 6. Background blur for video conferencing applications.
>> 7. Face detection algorithms operating directly on image data.
>> 8. Multicamera 360 image stitching etc.
> 
> Those are all interesting topics, but I won't be able to schedule all of
> them in 15 minutes :-)

I agree those topics will require more time to discuss.

> 
>> All of these use cases and algorithm implementations can be vendor-specific or
>> third-party solutions and may run on a GPU, DSP, or other accelerators.
>>
>> I also think that, similar to the IPA used for 3A algorithms, we need an
>> IPA-like for frame-processing algorithms.
>>
>> As discussed, the only practical way to upstream such functionality into
>> libcamera is to provide an open-source alternative (even a simplified
>> implementation) that defines and covers the use case, including its input/
>> output parameters and interfaces.
>>
>> The main question is whether these algorithms should live inside libcamera or
>> outside of it. Both approaches have their strengths, but I am personally in
>> favor of integrating them into libcamera, perhaps as a layer above the pipeline
>> handlers, for example through use case plugins or a similar mechanism.
> 
> I've long thought that libcamera should make pipeline handlers more
> modular. Pipeline handlers would implement components for the pipeline
> elements (such as the inline frontend and offline ISP) and declare how
> those components are connected. We should be able to move most of the
> logic required to run the pipeline into common code.

That will be really good to have.

> 
> With a modular approach, inserting elements in the pipeline could become
> easier. It's however not clear to me yet if we should try to separate
> the processing required for the use cases you list above from the "core"
> of pipeline handlers, into a separate layer on top, or if some use cases
> would benefit more from being able to inject elements into the pipeline.

I think that we will need some scheduler or code for driving different usecases,
also it should be noted that processing order may depend on the parameters,
An example we have preprocessing noise filter which can be enabled disabled,
or HDR. Depending on that the processing blocks participating in the pipleine
may change. But one step at a time.

> 
> Note that we have started experimenting with the concept of layers, see
> https://patchwork.libcamera.org/project/libcamera/list/?series=5753/

Thanks i will check that work.

> 
>> We can also learn from Android's approach. Initially, the HAL3 and per-control
>> APIs were created to allow applications to implement advanced algorithms such
>> as HDR and EIS. Later, CameraX Extensions were introduced so that these
>> capabilities could be shared across applications and benefit the broader
>> ecosystem:
>>
>> https://developer.android.com/media/camera/camerax/extensions-api
> 
> It's certainly something we should study more. As usual with Android
> there's little documentation on the underlying architecture though :-(
> 
>> I believe Linux faces a similar challenge. Once PipeWire and GStreamer plugins
>> based on libcamera become widely adopted across Linux distributions, it will
>> become increasingly difficult to introduce these advanced use cases at higher
>> layers of the software stack. Having a standardized framework within or closely
>> integrated with libcamera could make these capabilities more accessible and
>> reusable across applications.
> 
> No fundamental disagreement there, although we should also be careful
> not to reimplement GStreamer or PipeWire inside libcamera.

I agree on this, but having some concept like bins in gstreamer will also help for the entire usecase,
the question is whether we need to expose using multiple stages of reprocess the control to the app
(gstreamer element, PipeWire or other libcamera application) or have them build in libcamera. Both have benefits,
having ability the application to add intermediate stages gives freedom to experiment and some vendors to implement their
algos outside of the libcamera, On other hand side standard supported application like teams/slack browser will not 
have those extras example with cameraX extensions in Android which have been mentioned earlier.

> 
> An important question that will need to be answered is who will be
> willing to do all this work.

We have two eng in MMS which can start experimenting with those extensions in libcamera, i will actively involved and drive this activity if
there are no objections :-). Once you are back from Prague i can schedule a meeting for further discussions.

>

Regards,
~Gjorgji

Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Sakari Ailus]

Hi Laurent,

On Fri, Oct 02, 2026 at 09:31:05PM +0300, Laurent Pinchart wrote:
> On Fri, Oct 02, 2026 at 09:56:23AM +0300, Sakari Ailus wrote:
> > On Thu, Oct 01, 2026 at 11:55:05PM +0300, Laurent Pinchart wrote:
> > > Hi Hans,
> > > 
> > > (CC'ing Sakari who may have more insight on ACPI properties for
> > > Intel-based machines, or who may be able to pull the right people in
> > > this conversation)
> > > 
> > > On Thu, Oct 01, 2026 at 05:15:05PM +0200, johannes.goede@oss.qualcomm.com wrote:
> > > > Hi Stefan,
> > > > 
> > > > Thank you for the write-up.
> > > > 
> > > > On 30-Sep-26 13:18, Stefan Klug wrote:
> > > > 
> > > > <snip>
> > > > 
> > > > > ## Diving into the examples
> > > > > 
> > > > > Lets look at the example I listed to see how these would be solved.
> > > > > 
> > > > > ### 1. Laptop with camera module from various manufacturers
> > > > > 
> > > > > I believe this can be solved by either integrating the detection logic
> > > > > in libcamera and/or adding rules to udev/hwdb. But we are lacking real
> > > > > examples. So if you're able to provide identification strategies for
> > > > > real, existing hardware, I'd be glad to know about it. One strategy
> > > > > might be to use the machine identifier provided by DMI. This is however
> > > > > quite coarse and can not handle cases where the manufacturer used
> > > > > different camera modules from different vendors (using the same sensor
> > > > > model). ARe there for example ACPI properties that we could query for
> > > > > additional information?
> > > > > 
> > > > > Q: Does anyone in this group have more details on which information to
> > > > > query and where to get more details from the system?
> > > > > A: ...
> > > > 
> > > > For the Intel IPU6/IPU7 methods there is an ACPI call which the kernel
> > > > can do on the sensor fwnode object which will return a string identifying
> > > > the modules. These strings don't really have any fixed format, typically
> > > > they contain something which look like how some laptop serials look
> > > > just a random bunch of numbers + capital letters.
> > > 
> > > We know that hardware manufacturers like to have multiple providers for
> > > camera modules in the same laptop model in order to avoid supply chain
> > > issues. Do you know if this can be confirmed from those strings, do we
> > > have enough data to see if the identification strings are made of a
> > > small number of clusters ?
> > 
> > These strings are unique to the module. I'm afraid they could even be
> > module names as defined by the module vendor.
> 
> Does that mean that, if a laptop manufacturer sources camera modules
> equipped with the same sensor from different module manufacturers to
> diversify its supply chain, a single machine model (as report by DMI)
> would use a separate module identification string in ACPI for each
> module type ?

That is my understanding. I'm cc'ing Jimmy Su and Serin Yeh, too.

> 
> > > > Userspace cannot get to this without the kernel exporting it.
> > > > 
> > > > Given all the udev talk, I think a sysfs attribute would make sense
> > > > for this. We can make the kernel do the ACPI call and if it is present
> > > > add a camera_module sysfs attribute to the v4l2-subdev for the sensor,
> > > > which will only be visible when there actually is a module-name.
> > > > 
> > > > On laptops using devicetree we could then fill this from a devicetree
> > > > property.
> > > 
> > > I would very much like to standardize the API exposed to userspace
> > > across different types of firmwares (ACPI and DT). Having more data
> > > about the ACPI side could help us design DT properties that would fit
> > > nicely with a single API.
> > 
> > Note that this concerns only the Windows specific ACPI tables. DisCo for
> > Imaging (or older Chromebook ACPI tables following Linux specific
> > definitions) doesn't specify anything (largely because DT doesn't). Of
> > course it'd be nice to change that -- for both.
> 
> Understood. The needs are likely the same though, userspace needs to
> pick the right tuning data.
> 
> Do you know if Intel-based laptops typically ship all tuning files for a
> device model in the same file system image and pick the appropriate file
> based on data in ACPI tables, or if the system image is tailored to each
> SKU and contains only the tuning file for the module used by that
> particular machine instance ?

It's currently probably sensor-based as on DT. I don't think the module
information is exported to userspace currently as we have no appropriate
UAPI for that.

> 
> > > > > ### 2. Camera modules with an embedded EEPROM
> > > > > 
> > > > > To make use of the EEPROM data from user space we need to carry two
> > > > > elements: The EEPROM data itself and some identifier telling which
> > > > > format the data in the EEPROM has. This seems relatively straightforward
> > > > > and we could model it like this in dts:
> > > > > 
> > > > > dts-snippet:
> > > > > imx283_0: sensor@1a {
> > > > > 		compatible = "sony,imx283";
> > > > >     ...
> > > > >     eeprom = <&eeprom>;
> > > > >     eeprom-format = "vendor-a-eeprom-fmt";
> > > > > };
> > > > > 
> > > > > Getting the link to the EEPROM from user space is something to
> > > > > investigate. Ideas are either sysfs or media-controller.
> > > > > The eeprom-format must be specified as well, to tell user space how to
> > > > > interpret the EEPROM data, as it is typically not self-contained.
> > > > > 
> > > > > Q: On a first discussion on that topic we were unsure if it would be
> > > > > preferred to expose the data directly in sysfs,
> > > > > or to symlink to the corresponding sysfs entry for the EEPROM. Any
> > > > > preferences?
> > > > > A: ...
> > > > > 
> > > > > Q: Any input/things to consider from ACPI side?
> > > > > A:
> > > > 
> > > > For IPU6/IPU7 eeprom data is not used for camera-module identification
> > > > (AFAIK), it is all done through the (custom, Intel specific) ACPI call
> > > > I mentioned above.
> > > 
> > > I wouldn't expect an EEPROM, that would be quite costly for little gain.
> > > I wonder if sensor OTP memory is typically used though. Some sensor
> > > manufacturers specify how to store calibration data there, such as lens
> > > shading tables for instance. I wonder if anyone puts the same data in
> > > ACPI tables.
> > 
> > To my knowledge there's no tuning data in system firmware on Intel x86
> > systems. That could change in the future of course, but it's perhaps
> > unlikely, due to logistical issues.
> 
> That's my guess too.
> 
> I half recall that Linux-based Nokia phones contained camera tuning data
> in a special flash partition, but I don't have much details.

There was all sorts of hardware specific tuning in the cal area and AFAIR
this included WIFI but not cameras.

> 
> > > > > Q: What happens if the EEPROM driver is not loaded yet or is just not
> > > > > available? Should it block/defer probing of the sensor?
> > > > > A:
> > > > > 
> > > > > Q: We are missing real-life examples for these cases. So input from
> > > > > vendors would be welcome.
> > > > > A:
> > > > <snip>
> 

-- 
Regards,

Sakari Ailus