Re: [ANNOUNCEMENT] Camera & ISP at Linux Plumbers Conference

[Bryan O'Donoghue <bod@kernel.org>]

On 15/09/2026 00:38, Laurent Pinchart wrote:
> We can consider scheduling a third topic. Proposals are welcome.

 From my LPC submission:

"Should offline ISPs be implemented in drm/accel"

Where does an offline Camera ISP belong in video4linux or in DRM and why ?

There are two main types of ISP - inline and offline. An inline ISP 
takes sensor input directly and produces some kind of processed image on 
the output. This is a natural fit for v4l2.

An offline ISP takes a memory buffer given to it and processes that 
buffer against supplied parameters. This model can be implemented as a 
video4linux m2m device but it could also be implemented as a DRM device 
and included in the DAG.

Taking Qualcomm SoCs as an example the Camera hardware provides two 
types of ISP. An inline processing engine IFE and an offline processing 
engine based on firmware ICP/HFI.

A use-case emerges to implement the inline ISP in v4l2 and the offline 
in drm/accel.

Some tension with this design would exist as user-space would have to be 
involved to update ISP parameters and process statistics. Another 
tension is including a firmware based system in dma-fencing including 
watchdogging and resetting if required.

There are clear benefits to having a loop of NPU/DSP to Offline ISP to 
Display that drm/accel seems to offer some interesting solutions to.

---
bod