* [PATCH v9 00/11] drm: Add new pixel formats for Xilinx Zynqmp
From: Tomi Valkeinen @ 2026-03-25 14:01 UTC (permalink / raw)
To: Vishal Sagar, Anatoliy Klymenko, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Laurent Pinchart,
Michal Simek, Simon Ser
Cc: dri-devel, linux-kernel, linux-arm-kernel, Geert Uytterhoeven,
Dmitry Baryshkov, Pekka Paalanen, Tomi Valkeinen, Pekka Paalanen,
Dmitry Baryshkov
Add new DRM pixel formats and add support for those in the Xilinx zynqmp
display driver.
All other formats except XVUY2101010 are already supported in upstream
gstreamer, but gstreamer's kmssink does not have the support yet, as it
obviously cannot support the formats without kernel having the formats.
Xilinx has support for these formats in their BSP kernel, and Xilinx has
a branch here, adding the support to gstreamer kmssink:
https://github.com/Xilinx/gst-plugins-bad.git xlnx-rebase-v1.18.5
New formats added:
DRM_FORMAT_Y8
- 8-bit Y-only
- fourcc: "GREY"
- gstreamer: GRAY8
DRM_FORMAT_XYYY2101010
- 10-bit Y-only, three pixels packed into 32-bits
- fourcc: "YPA4"
- gstreamer: GRAY10_LE32
DRM_FORMAT_XV20
- Like NV16, but with 10-bit components
- fourcc: "XV20"
- gstreamer: NV16_10LE32
DRM_FORMAT_X403
- 10-bit planar 4:4:4, with three samples packed into 32-bits
- fourcc: "X403"
- gstreamer: Y444_10LE32
XVUY2101010
- 10-bit 4:4:4, one pixel in 32 bits
- fourcc: "XY30"
Some notes:
I know the 8-bit greyscale format has been discussed before, and the
guidance was to use DRM_FORMAT_R8. While I'm not totally against that, I
would argue that adding DRM_FORMAT_Y8 makes sense, as:
1) We can mark it as 'is_yuv' in the drm_format_info, and this can help
the drivers handle e.g. full/limited range. Probably some hardware
handles grayscale as a value used for all RGB components, in which case
R8 makes sense, but when the hardware handles the Y-only pixels as YCbCr,
where Cb and Cr are "neutral", it makes more sense to consider the
format as an YUV format rather than RGB.
2) We can have the same fourcc as in v4l2. While not strictly necessary,
it's a constant source of confusion when the fourccs differ.
3) It (possibly) makes more sense for the user to use Y8/GREY format
instead of R8, as, in my experience, the documentation usually refers
to gray(scale) format or Y-only format.
I have made some adjustments to the formats compared to the Xilinx's
branch. E.g. The DRM_FORMAT_XYYY2101010 format in Xilinx's kmssink uses
fourcc "Y10 ", and DRM_FORMAT_Y10. I didn't like those, as the format is
a packed format, three 10-bit pixels in a 32-bit container, and I think
Y10 means a 10-bit pixel in a 16-bit container. Xilinx also has XV15
format (similar to XV20, but 2x2 subsampled), which already exists in
the kernel as P030.
Generally speaking, if someone has good ideas for the format define
names or fourccs, speak up, as it's not easy to invent good names =).
That said, keeping them the same as in the Xilinx trees will, of course,
be slightly easier for the users of Xilinx platforms.
Note: Earlier versions of the series had DRM_FORMAT_Y10_P32, which is
now DRM_FORMAT_XYYY2101010, and DRM_FORMAT_XV15, which is now removed as
P030 already exists.
[1] https://lore.kernel.org/all/20250109150310.219442-26-tzimmermann%40suse.de/
[2] git@github.com:tomba/kmsxx.git xilinx
[3] git@github.com:tomba/pykms.git xilinx
[4] git@github.com:tomba/pixutils.git xilinx
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Changes in v9:
- Drop DRM_FORMAT_XV15 as DRM_FORMAT_P030 already exists
- Rename DRM_FORMAT_Y10_P32 to DRM_FORMAT_XYYY2101010
- I kept the Reviewed-bys related to the above, although one could say
renaming the format names is a substantial change. Ping me if you want
me to drop the Rb and re-send.
- Link to v8: https://lore.kernel.org/r/20260128-xilinx-formats-v8-0-9ea8adb70269@ideasonboard.com
Changes in v8:
- Expand the "drm/fourcc: Add DRM_FORMAT_Y8" commit description to
explain the rationale
- Add comment to "drm: xlnx: zynqmp: Add support for Y8 and Y10_P32"
explainig the Y-only matrix
- Remove extra blank line
- Link to v7: https://lore.kernel.org/r/20251201-xilinx-formats-v7-0-1e1558adfefc@ideasonboard.com
Changes in v7:
- Added Reviewed-bys
- Rebased on v6.18
- Link to v6: https://lore.kernel.org/r/20251001-xilinx-formats-v6-0-014b076b542a@ideasonboard.com
Changes in v6:
- Added tags for reviews
- Rebased on v6.17
- Link to v5: https://lore.kernel.org/r/20250425-xilinx-formats-v5-0-c74263231630@ideasonboard.com
Changes in v5:
- Add comment about Y-only formats, clarifying how the display pipeline
handles them (they're handled as YCbCr, with Cb and Cr as "neutral")
- Clarify X403 format in the patch description
- Set unused Y-only CSC offsets to 0 (instead of 0x1800).
- Add R-bs
- Link to v4: https://lore.kernel.org/r/20250326-xilinx-formats-v4-0-322a300c6d72@ideasonboard.com
Changes in v4:
- Reformat the drm_format_info entries a bit
- Calculate block size only once in drm_format_info_bpp()
- Declare local variables in separate lines
- Add review tags
- Fix commit message referring to Y10_LE32 (should be Y10_P32)
- Link to v3: https://lore.kernel.org/r/20250212-xilinx-formats-v3-0-90d0fe106995@ideasonboard.com
Changes in v3:
- Drop "drm: xlnx: zynqmp: Fix max dma segment size". It is already
pushed.
- Add XVUY2101010 format.
- Rename DRM_FORMAT_Y10_LE32 to DRM_FORMAT_Y10_P32.
- Link to v2: https://lore.kernel.org/r/20250115-xilinx-formats-v2-0-160327ca652a@ideasonboard.com
Changes in v2:
- I noticed V4L2 already has fourcc Y10P, referring to MIPI-style packed
Y10 format. So I changed Y10_LE32 fourcc to YPA4. If logic has any
relevance here, P means packed, A means 10, 4 means "in 4 bytes".
- Added tags to "Fix max dma segment size" patch
- Updated description for "Add warning for bad bpp"
- Link to v1: https://lore.kernel.org/r/20241204-xilinx-formats-v1-0-0bf2c5147db1@ideasonboard.com
---
Tomi Valkeinen (11):
drm/fourcc: Add warning for bad bpp
drm/fourcc: Add DRM_FORMAT_XV20
drm/fourcc: Add DRM_FORMAT_Y8
drm/fourcc: Add DRM_FORMAT_XYYY2101010
drm/fourcc: Add DRM_FORMAT_X403
drm/fourcc: Add DRM_FORMAT_XVUY2101010
drm: xlnx: zynqmp: Use drm helpers when calculating buffer sizes
drm: xlnx: zynqmp: Add support for P030 & XV20
drm: xlnx: zynqmp: Add support for Y8 and XYYY2101010
drm: xlnx: zynqmp: Add support for X403
drm: xlnx: zynqmp: Add support for XVUY2101010
drivers/gpu/drm/drm_fourcc.c | 25 +++++++++++++++--
drivers/gpu/drm/xlnx/zynqmp_disp.c | 56 +++++++++++++++++++++++++++++++++++---
include/uapi/drm/drm_fourcc.h | 27 ++++++++++++++++++
3 files changed, 101 insertions(+), 7 deletions(-)
---
base-commit: 11439c4635edd669ae435eec308f4ab8a0804808
change-id: 20241120-xilinx-formats-f71901621833
Best regards,
--
Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
^ permalink raw reply
* Re: [PATCH 0/2] arm64: dts: st: Add BSEC support for STM32MP21
From: Alexandre TORGUE @ 2026-03-25 14:00 UTC (permalink / raw)
To: Patrice Chotard, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Maxime Coquelin
Cc: devicetree, linux-stm32, linux-arm-kernel, linux-kernel
In-Reply-To: <20260211-upstream_stm32mp21_bsec_nodes-v1-0-8379bf97b0bc@foss.st.com>
Hi Patrice
On 2/11/26 14:55, Patrice Chotard wrote:
> This series adds BSEC support for STM32MP21.
> It adds bsec node in stm32mp211.dtsi and bootph-all property in bsec
> node in stm32mp215f-dk.dts file.
>
> Signed-off-by: Patrice Chotard <patrice.chotard@foss.st.com>
> ---
> Patrice Chotard (2):
> arm64: dts: st: add bsec support to stm32mp21
> arm64: dts: st: add bootph-all in bsec node to stm32mp215f-dk
>
> arch/arm64/boot/dts/st/stm32mp211.dtsi | 16 ++++++++++++++++
> arch/arm64/boot/dts/st/stm32mp215f-dk.dts | 4 ++++
> 2 files changed, 20 insertions(+)
> ---
> base-commit: fd9678829d6dd0c10fde080b536abf4b1121c346
> change-id: 20260211-upstream_stm32mp21_bsec_nodes-16aa826a73dc
>
> Best regards,
Applied on stm32-next.
Thanks
alex
^ permalink raw reply
* Re: [PATCH v4 18/21] drivers: hv: vmbus: replace deprecated mmap hook with mmap_prepare
From: Vlastimil Babka (SUSE) @ 2026-03-25 13:57 UTC (permalink / raw)
To: Lorenzo Stoakes (Oracle), Andrew Morton
Cc: Jonathan Corbet, Clemens Ladisch, Arnd Bergmann,
Greg Kroah-Hartman, K . Y . Srinivasan, Haiyang Zhang, Wei Liu,
Dexuan Cui, Long Li, Alexander Shishkin, Maxime Coquelin,
Alexandre Torgue, Miquel Raynal, Richard Weinberger,
Vignesh Raghavendra, Bodo Stroesser, Martin K . Petersen,
David Howells, Marc Dionne, Alexander Viro, Christian Brauner,
Jan Kara, David Hildenbrand, Liam R . Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jann Horn, Pedro Falcato,
linux-kernel, linux-doc, linux-hyperv, linux-stm32,
linux-arm-kernel, linux-mtd, linux-staging, linux-scsi,
target-devel, linux-afs, linux-fsdevel, linux-mm, Ryan Roberts
In-Reply-To: <05467cb62267d750e5c770147517d4df0246cda6.1774045440.git.ljs@kernel.org>
On 3/20/26 23:39, Lorenzo Stoakes (Oracle) wrote:
> The f_op->mmap interface is deprecated, so update the vmbus driver to use
> its successor, mmap_prepare.
>
> This updates all callbacks which referenced the function pointer
> hv_mmap_ring_buffer to instead reference hv_mmap_prepare_ring_buffer,
> utilising the newly introduced compat_set_desc_from_vma() and
> __compat_vma_mmap() to be able to implement this change.
>
> The UIO HV generic driver is the only user of hv_create_ring_sysfs(),
> which is the only function which references
> vmbus_channel->mmap_prepare_ring_buffer which, in turn, is the only
> external interface to hv_mmap_prepare_ring_buffer.
>
> This patch therefore updates this caller to use mmap_prepare instead,
> which also previously used vm_iomap_memory(), so this change replaces it
> with its mmap_prepare equivalent, mmap_action_simple_ioremap().
>
> Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [PATCH v10 00/12] barrier: Add smp_cond_load_{relaxed,acquire}_timeout()
From: Catalin Marinas @ 2026-03-25 13:53 UTC (permalink / raw)
To: David Laight
Cc: Ankur Arora, Andrew Morton, linux-kernel, linux-arch,
linux-arm-kernel, linux-pm, bpf, arnd, will, peterz, mark.rutland,
harisokn, cl, ast, rafael, daniel.lezcano, memxor, zhenglifeng1,
xueshuai, rdunlap, joao.m.martins, boris.ostrovsky, konrad.wilk
In-Reply-To: <20260317091705.5a64fc56@pumpkin>
On Tue, Mar 17, 2026 at 09:17:05AM +0000, David Laight wrote:
> On Mon, 16 Mar 2026 23:53:22 -0700
> Ankur Arora <ankur.a.arora@oracle.com> wrote:
> > David Laight <david.laight.linux@gmail.com> writes:
> > > On arm64 I think you could use explicit sev and wfe - but that will wake all
> > > 'sleeping' cpu; and you may not want the 'thundering herd'.
> >
> > Wouldn't we still have the same narrow window where the CPU disregards the IPI?
>
> You need a 'sevl' in the interrupt exit path.
No need to, see the rule below in
https://developer.arm.com/documentation/ddi0487/maa/2983-beijhbbd:
R_XRZRK
The Event Register for a PE is set by any of the following:
[...]
- An exception return.
--
Catalin
^ permalink raw reply
* Re: [PATCH] arm64: dts: renesas: sparrow-hawk: Reserve first 128 MiB of DRAM
From: Marek Vasut @ 2026-03-25 13:52 UTC (permalink / raw)
To: Geert Uytterhoeven, Marek Vasut
Cc: linux-arm-kernel, stable, Conor Dooley, Geert Uytterhoeven,
Krzysztof Kozlowski, Magnus Damm, Rob Herring, devicetree,
linux-kernel, linux-renesas-soc
In-Reply-To: <CAMuHMdVv5KUma8=2T+ibnXyH+45xvqVrQaPho0CSzP1_r+j_hQ@mail.gmail.com>
On 3/25/26 2:08 PM, Geert Uytterhoeven wrote:
Hello Geert,
>> --- a/arch/arm64/boot/dts/renesas/r8a779g3-sparrow-hawk.dts
>> +++ b/arch/arm64/boot/dts/renesas/r8a779g3-sparrow-hawk.dts
>> @@ -118,6 +118,17 @@ memory@600000000 {
>> reg = <0x6 0x00000000 0x1 0x00000000>;
>> };
>>
>> + reserved-memory {
>> + #address-cells = <2>;
>> + #size-cells = <2>;
>> + ranges;
>> +
>> + tfa@40000000 {
>> + reg = <0x0 0x40000000 0x0 0x8000000>;
>> + no-map;
>> + };
>> + };
>
> Obviously I don't like this very much, but I agree there is not much
> else we can do at this point. Shall I add a
>
> /* Temporary workaround for broken TFA BL31 */
>
> comment while applying?
For maximum backward compatibility even with old TFA BL31 versions,
which the users may run for whatever reason, I believe this reserved
memory area is here to stay and won't be "Temporary".
But I will be looking into TFA to fill in this reserved-memory node
properly for this and future SoCs.
> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
> i.e. will queue in renesas-fixes for v7.0.
Thank you
^ permalink raw reply
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Maxime Ripard @ 2026-03-25 13:44 UTC (permalink / raw)
To: Nicolas Frattaroli
Cc: Dave Stevenson, Harry Wentland, Leo Li, Rodrigo Siqueira,
Alex Deucher, Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Thomas Zimmermann, Andrzej Hajda,
Neil Armstrong, Robert Foss, Laurent Pinchart, Jonas Karlman,
Jernej Skrabec, Sandy Huang, Heiko Stübner, Andy Yan,
Jani Nikula, Rodrigo Vivi, Joonas Lahtinen, Tvrtko Ursulin,
Dmitry Baryshkov, Sascha Hauer, Rob Herring, Jonathan Corbet,
Shuah Khan, kernel, amd-gfx, dri-devel, linux-kernel,
linux-arm-kernel, linux-rockchip, intel-gfx, intel-xe, linux-doc,
Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <12425220.nUPlyArG6x@workhorse>
[-- Attachment #1: Type: text/plain, Size: 3249 bytes --]
On Wed, Mar 25, 2026 at 02:21:24PM +0100, Nicolas Frattaroli wrote:
> On Wednesday, 25 March 2026 14:05:25 Central European Standard Time Maxime Ripard wrote:
> > Hi Dave,
> >
> > On Wed, Mar 25, 2026 at 12:49:19PM +0000, Dave Stevenson wrote:
> > > > diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
> > > > index af8b92d2d5b7..bd549f912b76 100644
> > > > --- a/include/drm/drm_connector.h
> > > > +++ b/include/drm/drm_connector.h
> > > > @@ -571,14 +571,102 @@ enum drm_colorspace {
> > > > * YCbCr 4:2:2 output format (ie. with horizontal subsampling)
> > > > * @DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > > > * YCbCr 4:2:0 output format (ie. with horizontal and vertical subsampling)
> > > > + * @DRM_OUTPUT_COLOR_FORMAT_COUNT:
> > > > + * Number of valid output color format values in this enum
> > > > */
> > > > enum drm_output_color_format {
> > > > DRM_OUTPUT_COLOR_FORMAT_RGB444 = 0,
> > > > DRM_OUTPUT_COLOR_FORMAT_YCBCR444,
> > > > DRM_OUTPUT_COLOR_FORMAT_YCBCR422,
> > > > DRM_OUTPUT_COLOR_FORMAT_YCBCR420,
> > > > + DRM_OUTPUT_COLOR_FORMAT_COUNT,
> > > > };
> > > >
> > > > +/**
> > > > + * enum drm_connector_color_format - Connector Color Format Request
> > > > + *
> > > > + * This enum, unlike &enum drm_output_color_format, is used to specify requests
> > > > + * for a specific color format on a connector through the DRM "color format"
> > > > + * property. The difference is that it has an "AUTO" value to specify that
> > > > + * no specific choice has been made.
> > > > + */
> > > > +enum drm_connector_color_format {
> > > > + /**
> > > > + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> > > > + * helpers should pick a suitable color format. All implementations of a
> > > > + * specific display protocol must behave the same way with "AUTO", but
> > > > + * different display protocols do not necessarily have the same "AUTO"
> > > > + * semantics.
> > > > + *
> > > > + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> > > > + * bandwidth required for full-scale RGB is not available, or the mode
> > > > + * is YCbCr 4:2:0-only, as long as the mode and output both support
> > > > + * YCbCr 4:2:0.
> > >
> > > Is there a reason you propose dropping back to YCbCr 4:2:0 without
> > > trying YCbCr 4:2:2 first? Minimising the subsampling is surely
> > > beneficial, and vc4 for one can do 4:2:2 but not 4:2:0.
> >
> > The "auto" behaviour is strictly identical to the one we have right now,
> > and this one stems from i915. Back when all that logic was added, it was
> > decided to align every driver behavior on i915 because that's what most
> > compositors would expect.
>
> would it be okay to extend the behavior while we're at it? 4:2:2 does save
> bandwidth compared to RGB (unlike YCbCr 4:4:4). I do think 4:2:2 instead of
> 4:2:0 will provide benefits in some cases.
I don't really know, we would need to ask some Wayland/Weston devs as
well, but it sure is something worth discussing.
I wouldn't do it in this series but as a follow-up though.
Maxime
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]
^ permalink raw reply
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Ville Syrjälä @ 2026-03-25 13:43 UTC (permalink / raw)
To: Dave Stevenson
Cc: Nicolas Frattaroli, Harry Wentland, Leo Li, Rodrigo Siqueira,
Alex Deucher, Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Sandy Huang, Heiko Stübner,
Andy Yan, Jani Nikula, Rodrigo Vivi, Joonas Lahtinen,
Tvrtko Ursulin, Dmitry Baryshkov, Sascha Hauer, Rob Herring,
Jonathan Corbet, Shuah Khan, kernel, amd-gfx, dri-devel,
linux-kernel, linux-arm-kernel, linux-rockchip, intel-gfx,
intel-xe, linux-doc, Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <CAPY8ntB9f_=f5kru=8w9BpTuqQR+93maGpT61EKU28Uay2vq8Q@mail.gmail.com>
On Wed, Mar 25, 2026 at 12:49:19PM +0000, Dave Stevenson wrote:
> On Tue, 24 Mar 2026 at 16:02, Nicolas Frattaroli
> <nicolas.frattaroli@collabora.com> wrote:
> >
> > Add a new general DRM property named "color format" which can be used by
> > userspace to request the display driver to output a particular color
> > format.
> >
> > Possible options are:
> > - auto (setup by default, driver internally picks the color format)
> > - rgb
> > - ycbcr444
> > - ycbcr422
> > - ycbcr420
> >
> > Drivers should advertise from this list which formats they support.
> > Together with this list and EDID data from the sink we should be able
> > to relay a list of usable color formats to users to pick from.
> >
> > Co-developed-by: Werner Sembach <wse@tuxedocomputers.com>
> > Signed-off-by: Werner Sembach <wse@tuxedocomputers.com>
> > Co-developed-by: Andri Yngvason <andri@yngvason.is>
> > Signed-off-by: Andri Yngvason <andri@yngvason.is>
> > Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
> > Reviewed-by: Maxime Ripard <mripard@kernel.org>
> > Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
> > ---
> > drivers/gpu/drm/drm_atomic_helper.c | 5 ++
> > drivers/gpu/drm/drm_atomic_uapi.c | 11 ++++
> > drivers/gpu/drm/drm_connector.c | 108 ++++++++++++++++++++++++++++++++++++
> > include/drm/drm_connector.h | 104 ++++++++++++++++++++++++++++++++++
> > 4 files changed, 228 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
> > index 26953ed6b53e..b7753454b777 100644
> > --- a/drivers/gpu/drm/drm_atomic_helper.c
> > +++ b/drivers/gpu/drm/drm_atomic_helper.c
> > @@ -737,6 +737,11 @@ drm_atomic_helper_check_modeset(struct drm_device *dev,
> > if (old_connector_state->max_requested_bpc !=
> > new_connector_state->max_requested_bpc)
> > new_crtc_state->connectors_changed = true;
> > +
> > + if (old_connector_state->color_format !=
> > + new_connector_state->color_format)
> > + new_crtc_state->connectors_changed = true;
> > +
> > }
> >
> > if (funcs->atomic_check)
> > diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
> > index 5bd5bf6661df..dee510c85e59 100644
> > --- a/drivers/gpu/drm/drm_atomic_uapi.c
> > +++ b/drivers/gpu/drm/drm_atomic_uapi.c
> > @@ -935,6 +935,15 @@ static int drm_atomic_connector_set_property(struct drm_connector *connector,
> > state->privacy_screen_sw_state = val;
> > } else if (property == connector->broadcast_rgb_property) {
> > state->hdmi.broadcast_rgb = val;
> > + } else if (property == connector->color_format_property) {
> > + if (val > INT_MAX || !drm_connector_color_format_valid(val)) {
> > + drm_dbg_atomic(connector->dev,
> > + "[CONNECTOR:%d:%s] unknown color format %llu\n",
> > + connector->base.id, connector->name, val);
> > + return -EINVAL;
> > + }
> > +
> > + state->color_format = val;
> > } else if (connector->funcs->atomic_set_property) {
> > return connector->funcs->atomic_set_property(connector,
> > state, property, val);
> > @@ -1020,6 +1029,8 @@ drm_atomic_connector_get_property(struct drm_connector *connector,
> > *val = state->privacy_screen_sw_state;
> > } else if (property == connector->broadcast_rgb_property) {
> > *val = state->hdmi.broadcast_rgb;
> > + } else if (property == connector->color_format_property) {
> > + *val = state->color_format;
> > } else if (connector->funcs->atomic_get_property) {
> > return connector->funcs->atomic_get_property(connector,
> > state, property, val);
> > diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
> > index 47dc53c4a738..e848374dee0b 100644
> > --- a/drivers/gpu/drm/drm_connector.c
> > +++ b/drivers/gpu/drm/drm_connector.c
> > @@ -1388,6 +1388,18 @@ static const u32 hdmi_colorspaces =
> > BIT(DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65) |
> > BIT(DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER);
> >
> > +static const u32 hdmi_colorformats =
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_RGB444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR422) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR420);
> > +
> > +static const u32 dp_colorformats =
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_RGB444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR422) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR420);
> > +
> > /*
> > * As per DP 1.4a spec, 2.2.5.7.5 VSC SDP Payload for Pixel Encoding/Colorimetry
> > * Format Table 2-120
> > @@ -2940,6 +2952,102 @@ int drm_connector_attach_colorspace_property(struct drm_connector *connector)
> > }
> > EXPORT_SYMBOL(drm_connector_attach_colorspace_property);
> >
> > +/**
> > + * drm_connector_attach_color_format_property - create and attach color format property
> > + * @connector: connector to create the color format property on
> > + * @supported_color_formats: bitmask of bit-shifted &enum drm_output_color_format
> > + * values the connector supports
> > + *
> > + * Called by a driver to create a color format property. The property is
> > + * attached to the connector automatically on success.
> > + *
> > + * @supported_color_formats should only include color formats the connector
> > + * type can actually support.
> > + *
> > + * Returns:
> > + * 0 on success, negative errno on error
> > + */
> > +int drm_connector_attach_color_format_property(struct drm_connector *connector,
> > + unsigned long supported_color_formats)
> > +{
> > + struct drm_device *dev = connector->dev;
> > + struct drm_prop_enum_list enum_list[DRM_CONNECTOR_COLOR_FORMAT_COUNT];
> > + unsigned int i = 0;
> > + unsigned long fmt;
> > +
> > + if (connector->color_format_property)
> > + return 0;
> > +
> > + if (!supported_color_formats) {
> > + drm_err(dev, "No supported color formats provided on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > +
> > + if (supported_color_formats & ~GENMASK(DRM_OUTPUT_COLOR_FORMAT_COUNT - 1, 0)) {
> > + drm_err(dev, "Unknown color formats provided on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > +
> > + switch (connector->connector_type) {
> > + case DRM_MODE_CONNECTOR_HDMIA:
> > + case DRM_MODE_CONNECTOR_HDMIB:
> > + if (supported_color_formats & ~hdmi_colorformats) {
> > + drm_err(dev, "Color formats not allowed for HDMI on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > + break;
> > + case DRM_MODE_CONNECTOR_DisplayPort:
> > + case DRM_MODE_CONNECTOR_eDP:
> > + if (supported_color_formats & ~dp_colorformats) {
> > + drm_err(dev, "Color formats not allowed for DP on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > + break;
> > + }
> > +
> > + enum_list[0].name = "AUTO";
> > + enum_list[0].type = DRM_CONNECTOR_COLOR_FORMAT_AUTO;
> > +
> > + for_each_set_bit(fmt, &supported_color_formats, DRM_OUTPUT_COLOR_FORMAT_COUNT) {
> > + switch (fmt) {
> > + case DRM_OUTPUT_COLOR_FORMAT_RGB444:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_RGB444;
> > + break;
> > + case DRM_OUTPUT_COLOR_FORMAT_YCBCR444:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR444;
> > + break;
> > + case DRM_OUTPUT_COLOR_FORMAT_YCBCR422:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR422;
> > + break;
> > + case DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR420;
> > + break;
> > + default:
> > + drm_warn(dev, "Unknown supported format %ld on [CONNECTOR:%d:%s]\n",
> > + fmt, connector->base.id, connector->name);
> > + continue;
> > + }
> > + enum_list[i].name = drm_hdmi_connector_get_output_format_name(fmt);
> > + }
> > +
> > + connector->color_format_property =
> > + drm_property_create_enum(dev, DRM_MODE_PROP_ENUM, "color format",
> > + enum_list, i + 1);
> > +
> > + if (!connector->color_format_property)
> > + return -ENOMEM;
> > +
> > + drm_object_attach_property(&connector->base, connector->color_format_property,
> > + DRM_CONNECTOR_COLOR_FORMAT_AUTO);
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL(drm_connector_attach_color_format_property);
> > +
> > /**
> > * drm_connector_atomic_hdr_metadata_equal - checks if the hdr metadata changed
> > * @old_state: old connector state to compare
> > diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
> > index af8b92d2d5b7..bd549f912b76 100644
> > --- a/include/drm/drm_connector.h
> > +++ b/include/drm/drm_connector.h
> > @@ -571,14 +571,102 @@ enum drm_colorspace {
> > * YCbCr 4:2:2 output format (ie. with horizontal subsampling)
> > * @DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > * YCbCr 4:2:0 output format (ie. with horizontal and vertical subsampling)
> > + * @DRM_OUTPUT_COLOR_FORMAT_COUNT:
> > + * Number of valid output color format values in this enum
> > */
> > enum drm_output_color_format {
> > DRM_OUTPUT_COLOR_FORMAT_RGB444 = 0,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR444,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR422,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR420,
> > + DRM_OUTPUT_COLOR_FORMAT_COUNT,
> > };
> >
> > +/**
> > + * enum drm_connector_color_format - Connector Color Format Request
> > + *
> > + * This enum, unlike &enum drm_output_color_format, is used to specify requests
> > + * for a specific color format on a connector through the DRM "color format"
> > + * property. The difference is that it has an "AUTO" value to specify that
> > + * no specific choice has been made.
> > + */
> > +enum drm_connector_color_format {
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> > + * helpers should pick a suitable color format. All implementations of a
> > + * specific display protocol must behave the same way with "AUTO", but
> > + * different display protocols do not necessarily have the same "AUTO"
> > + * semantics.
> > + *
> > + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> > + * bandwidth required for full-scale RGB is not available, or the mode
> > + * is YCbCr 4:2:0-only, as long as the mode and output both support
> > + * YCbCr 4:2:0.
>
> Is there a reason you propose dropping back to YCbCr 4:2:0 without
> trying YCbCr 4:2:2 first? Minimising the subsampling is surely
> beneficial, and vc4 for one can do 4:2:2 but not 4:2:0.
On HDMI 4:2:2 is always 12bpc, so it doesn't save any bandwidth
compared to 8bpc 4:4:4.
--
Ville Syrjälä
Intel
^ permalink raw reply
* Re: [PATCH v4 17/21] mm: allow handling of stacked mmap_prepare hooks in more drivers
From: Vlastimil Babka (SUSE) @ 2026-03-25 13:43 UTC (permalink / raw)
To: Lorenzo Stoakes (Oracle), Andrew Morton
Cc: Jonathan Corbet, Clemens Ladisch, Arnd Bergmann,
Greg Kroah-Hartman, K . Y . Srinivasan, Haiyang Zhang, Wei Liu,
Dexuan Cui, Long Li, Alexander Shishkin, Maxime Coquelin,
Alexandre Torgue, Miquel Raynal, Richard Weinberger,
Vignesh Raghavendra, Bodo Stroesser, Martin K . Petersen,
David Howells, Marc Dionne, Alexander Viro, Christian Brauner,
Jan Kara, David Hildenbrand, Liam R . Howlett, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Jann Horn, Pedro Falcato,
linux-kernel, linux-doc, linux-hyperv, linux-stm32,
linux-arm-kernel, linux-mtd, linux-staging, linux-scsi,
target-devel, linux-afs, linux-fsdevel, linux-mm, Ryan Roberts
In-Reply-To: <24aac3019dd34740e788d169fccbe3c62781e648.1774045440.git.ljs@kernel.org>
On 3/20/26 23:39, Lorenzo Stoakes (Oracle) wrote:
> While the conversion of mmap hooks to mmap_prepare is underway, we will
> encounter situations where mmap hooks need to invoke nested mmap_prepare
> hooks.
>
> The nesting of mmap hooks is termed 'stacking'. In order to flexibly
> facilitate the conversion of custom mmap hooks in drivers which stack, we
> must split up the existing __compat_vma_mmap() function into two separate
> functions:
>
> * compat_set_desc_from_vma() - This allows the setting of a vm_area_desc
> object's fields to the relevant fields of a VMA.
>
> * __compat_vma_mmap() - Once an mmap_prepare hook has been executed upon a
> vm_area_desc object, this function performs any mmap actions specified by
> the mmap_prepare hook and then invokes its vm_ops->mapped() hook if any
> were specified.
>
> In ordinary cases, where a file's f_op->mmap_prepare() hook simply needs
> to be invoked in a stacked mmap() hook, compat_vma_mmap() can be used.
>
> However some drivers define their own nested hooks, which are invoked in
> turn by another hook.
>
> A concrete example is vmbus_channel->mmap_ring_buffer(), which is invoked
> in turn by bin_attribute->mmap():
>
> vmbus_channel->mmap_ring_buffer() has a signature of:
>
> int (*mmap_ring_buffer)(struct vmbus_channel *channel,
> struct vm_area_struct *vma);
>
> And bin_attribute->mmap() has a signature of:
>
> int (*mmap)(struct file *, struct kobject *,
> const struct bin_attribute *attr,
> struct vm_area_struct *vma);
>
> And so compat_vma_mmap() cannot be used here for incremental conversion of
> hooks from mmap() to mmap_prepare().
>
> There are many such instances like this, where conversion to mmap_prepare
> would otherwise cascade to a huge change set due to nesting of this kind.
>
> The changes in this patch mean we could now instead convert
> vmbus_channel->mmap_ring_buffer() to
> vmbus_channel->mmap_prepare_ring_buffer(), and implement something like:
>
> struct vm_area_desc desc;
> int err;
>
> compat_set_desc_from_vma(&desc, file, vma);
> err = channel->mmap_prepare_ring_buffer(channel, &desc);
> if (err)
> return err;
>
> return __compat_vma_mmap(&desc, vma);
>
> Allowing us to incrementally update this logic, and other logic like it.
>
> Unfortunately, as part of this change, we need to be able to flexibly
> assign to the VMA descriptor, so have to remove some of the const
> declarations within the structure.
>
> Also update the VMA tests to reflect the changes.
>
> Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
^ permalink raw reply
* Re: [PATCH] arm64: dts: renesas: sparrow-hawk: Add overlay for Olimex MIPI-HDMI adapter
From: Marek Vasut @ 2026-03-25 13:42 UTC (permalink / raw)
To: Geert Uytterhoeven, Scott Murray
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-renesas-soc
In-Reply-To: <CAMuHMdWO9Qc=mn1SAwZt1HJK2VkU++G_8N+eVuN5AWTrLnVxUQ@mail.gmail.com>
On 3/25/26 2:32 PM, Geert Uytterhoeven wrote:
Hello Geert,
> On Fri, 20 Mar 2026 at 03:53, Marek Vasut
> <marek.vasut+renesas@mailbox.org> wrote:
>>
>> From: Scott Murray <scott.murray@konsulko.com>
>>
>> Add a DT overlay to bind the Olimex MIPI-HDMI dual-lane DSI-to-HDMI adapter
>> on the Retronix R-Car V4H Sparrow Hawk board.
>>
>> Current device link is at https://www.olimex.com/Products/IoT/ESP32-P4/MIPI-HDMI/open-source-hardware
>>
>> Signed-off-by: Scott Murray <scott.murray@konsulko.com>
>> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
>
> Thanks for your patch!
>
>> --- /dev/null
>> +++ b/arch/arm64/boot/dts/renesas/r8a779g3-sparrow-hawk-olimex-dsi-hdmi.dtso
>> @@ -0,0 +1,94 @@
>> +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
>> +/*
>> + * Device Tree Overlay for Olimex MIPI-HDMI adapter connected to J4:DSI
>> + * on R-Car V4H ES3.0 Sparrow Hawk board
>> + *
>> + * Copyright (C) 2026 Scott Murray <scott.murray@konsulko.com>
>> + */
>> +
>> +/dts-v1/;
>> +/plugin/;
>> +
>> +#include <dt-bindings/gpio/gpio.h>
>
> This include is not needed, so I will drop it while applying.
>
> The rest LGTM, so
> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
> i.e. will queue in renesas-devel for v7.1.
Thank you very much.
^ permalink raw reply
* Re: [PATCH v2] KVM: arm64: Prevent the host from using an smc with imm16 != 0
From: Mark Rutland @ 2026-03-25 13:33 UTC (permalink / raw)
To: Will Deacon
Cc: Marc Zyngier, Sebastian Ene, Vincent Donnefort, kvmarm,
linux-arm-kernel, linux-kernel, android-kvm, catalin.marinas,
joey.gouly, oupton, suzuki.poulose, tabba, yuzenghui
In-Reply-To: <acPRIk7_dH5Erul8@willie-the-truck>
On Wed, Mar 25, 2026 at 12:12:18PM +0000, Will Deacon wrote:
> On Wed, Mar 25, 2026 at 11:46:29AM +0000, Marc Zyngier wrote:
> > On Wed, 25 Mar 2026 11:35:18 +0000,
> > Vincent Donnefort <vdonnefort@google.com> wrote:
> > >
> > > On Wed, Mar 25, 2026 at 11:31:38AM +0000, Sebastian Ene wrote:
> > > > The ARM Service Calling Convention (SMCCC) specifies that the function
> > > > identifier and parameters should be passed in registers, leaving the
> > > > 16-bit immediate field of the SMC instruction un-handled.
> > > > Currently, our pKVM handler ignores the immediate value, which could lead
> > > > to non-compliant software relying on implementation-defined behavior.
> > > > Enforce the host kernel running under pKVM to use an immediate value
> > > > of 0 by decoding the ISS from the ESR_EL2 and return a not supported
> > > > error code back to the caller.
> > It also begs the question: if you don't want to handle SMCs with a
> > non-zero immediate, why is it OK to do it for HVCs?
>
> I suppose the difference is that the HVC API is a private interface
> between EL2 and the host. As it stands, EL2 ignores the immediate but we
> don't have a way to know how EL3 responds to the immediate for an SMC.
>
> When proxying an SMC from the host, EL2 therefore has three choices:
>
> 1. Ignore the immediate from the host and always use zero when talking
> to EL3. That's the current behaviour, but it could theoretically
> lead to problems if EL3 is using the immediate for something.
>
> 2. Propagate the immediate from the host. That should work, but it's
> a bit involved.
>
> 3. Reject non-zero immediates (this patch).
I think (3) is the only safe option. As described in:
https://lore.kernel.org/linux-arm-kernel/acPi5V0DgGcgHNGO@J2N7QTR9R3/
... regardless of the the conduit (HVS/SMC) used, a non-zero immediate
is NOT a legitimate SMCCC call. (1) is not safe, becuase it could change
the semantic of the call, and (2) is not safe because the calling
convention could differ (and e.g. things might be clobbered that we
don't expect).
We know that the host *SHOULD* use a zero immediate, and so anything
else represents a bug that we will want to catch.
Mark.
^ permalink raw reply
* Re: [PATCH] arm64: dts: renesas: sparrow-hawk: Add overlay for Olimex MIPI-HDMI adapter
From: Geert Uytterhoeven @ 2026-03-25 13:32 UTC (permalink / raw)
To: Marek Vasut, Scott Murray
Cc: linux-arm-kernel, Conor Dooley, Krzysztof Kozlowski, Magnus Damm,
Rob Herring, devicetree, linux-renesas-soc
In-Reply-To: <20260320025328.509003-1-marek.vasut+renesas@mailbox.org>
Hi Marek, Scott,
On Fri, 20 Mar 2026 at 03:53, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
>
> From: Scott Murray <scott.murray@konsulko.com>
>
> Add a DT overlay to bind the Olimex MIPI-HDMI dual-lane DSI-to-HDMI adapter
> on the Retronix R-Car V4H Sparrow Hawk board.
>
> Current device link is at https://www.olimex.com/Products/IoT/ESP32-P4/MIPI-HDMI/open-source-hardware
>
> Signed-off-by: Scott Murray <scott.murray@konsulko.com>
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Thanks for your patch!
> --- /dev/null
> +++ b/arch/arm64/boot/dts/renesas/r8a779g3-sparrow-hawk-olimex-dsi-hdmi.dtso
> @@ -0,0 +1,94 @@
> +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +/*
> + * Device Tree Overlay for Olimex MIPI-HDMI adapter connected to J4:DSI
> + * on R-Car V4H ES3.0 Sparrow Hawk board
> + *
> + * Copyright (C) 2026 Scott Murray <scott.murray@konsulko.com>
> + */
> +
> +/dts-v1/;
> +/plugin/;
> +
> +#include <dt-bindings/gpio/gpio.h>
This include is not needed, so I will drop it while applying.
The rest LGTM, so
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-devel for v7.1.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH v2] KVM: arm64: Prevent the host from using an smc with imm16 != 0
From: Mark Rutland @ 2026-03-25 13:28 UTC (permalink / raw)
To: Sebastian Ene
Cc: kvmarm, linux-arm-kernel, linux-kernel, android-kvm,
catalin.marinas, joey.gouly, maz, oupton, suzuki.poulose, tabba,
vdonnefort, will, yuzenghui
In-Reply-To: <20260325113138.4171430-1-sebastianene@google.com>
On Wed, Mar 25, 2026 at 11:31:38AM +0000, Sebastian Ene wrote:
> The ARM Service Calling Convention (SMCCC) specifies that the function
> identifier and parameters should be passed in registers, leaving the
> 16-bit immediate field of the SMC instruction un-handled.
That's not quite right; the SMCCC spec says callers must use immediate
0.
See https://developer.arm.com/documentation/den0028/h/ section 2.10
("SME and HVC immediate value"), which says:
| • For all compliant calls, an SMC or HVC immediate value of zero must be
| used.
| • Nonzero immediate values in SMC instructions are reserved.
| • Nonzero immediate values in HVC instructions are designated for use by
| hypervisor vendors.
> Currently, our pKVM handler ignores the immediate value, which could lead
> to non-compliant software relying on implementation-defined behavior.
> Enforce the host kernel running under pKVM to use an immediate value
> of 0 by decoding the ISS from the ESR_EL2 and return a not supported
> error code back to the caller.
From my PoV, it'd be fine to turn a non-zero immediate into an UNDEF:
* For HVC, we can say any non-zero immediate represents a request to KVM to
inject an UNDEF.
* For SMC, the dbehaviour is not defined.
I am also ok with returning a value in x0, BUT that's stronger than SMCCC
actually requires.
Mark.
> Signed-off-by: Sebastian Ene <sebastianene@google.com>
> ---
> v1 -> v2:
>
> - Dropped injecting an UNDEF and return an error instead
> (SMCCC_RET_NOT_SUPPORTED)
> - Used the mask ESR_ELx_xVC_IMM_MASK instead of masking with U16_MAX
> - Updated the title of the commit message from:
> "[PATCH] KVM: arm64: Inject UNDEF when host is executing an
> smc with imm16 != 0"
> ---
> arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> index e7790097db93..4ffe30fd8707 100644
> --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> @@ -762,6 +762,12 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
> handle_host_hcall(host_ctxt);
> break;
> case ESR_ELx_EC_SMC64:
> + if (ESR_ELx_xVC_IMM_MASK & esr) {
> + cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED;
> + kvm_skip_host_instr();
> + break;
> + }
> +
> handle_host_smc(host_ctxt);
> break;
> case ESR_ELx_EC_IABT_LOW:
> --
> 2.53.0.1018.g2bb0e51243-goog
>
^ permalink raw reply
* Re: [PATCH v4 0/6] porting pq compnent for MT8196
From: Jay Liu (刘博) @ 2026-03-25 13:26 UTC (permalink / raw)
To: krzk@kernel.org
Cc: linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org,
chunkuang.hu@kernel.org, devicetree@vger.kernel.org,
tzimmermann@suse.de, simona@ffwll.ch, mripard@kernel.org,
p.zabel@pengutronix.de, maarten.lankhorst@linux.intel.com,
conor+dt@kernel.org, robh@kernel.org,
dri-devel@lists.freedesktop.org, airlied@gmail.com,
linux-arm-kernel@lists.infradead.org, matthias.bgg@gmail.com,
krzk+dt@kernel.org, AngeloGioacchino Del Regno
In-Reply-To: <20260325-hissing-jasmine-crayfish-caef70@quoll>
On Wed, 2026-03-25 at 09:17 +0100, Krzysztof Kozlowski wrote:
> On Tue, Mar 24, 2026 at 08:51:57PM +0800, Jay Liu wrote:
> > Change in v4:
> > - Address coding style comments for disp-tdshp binding.
>
> Coding style? Why so vague? So does that mean you ignored all
> important
> changes but did the "style" things?
>
> > - Rebase ccorr driver patch on top of latest linux-next to fix
> > conficts.
>
> Where is the rest of the changeglog?
>
> Why aren't you using b4 to provide also lore links and solve above
> problem?
>
> Best regards,
> Krzysztof
>
Dear Krzysztof,
I sincerely apologize for the confusion caused by my vague changelog.
My intention was to be concise, but I realize it made it look like I
ignored your technical feedback, which was not the case.
I have actually implemented the changes you requested in v3. Here is
the detailed breakdown of what was fixed in v4:
1. Sibling description: Removed as suggested.
2. Enum: Dropped the items list and used enum directly.
3. Formatting: Removed the extra blank line and fixed the indentation
in the example section.
Regarding the Node Name (disp-tdshp):
This was the only item I hesitated to change. "tdshp" stands for "2D
sharpness", and I kept it to align with MediaTek's internal naming.
However, I understand the upstream requirement for generic names. Since
this is a sharpness engine, would you prefer "sharpness-processor" or
something similar?
And Thank you for the recommendation regarding b4. I will set it up for
future submissions to ensure lore links and changelogs are generated
correctly.
I will send a v5 with a proper changelog and the node name update once
we agree on the name.
Please let me know if I missed any other comments or if you have
further suggestions. I will do my best to address them.
Best regards,
Jay
^ permalink raw reply
* Re: [RFC v1 00/11] Add iMX95 neoisp driver
From: Jacopo Mondi @ 2026-03-25 13:23 UTC (permalink / raw)
To: Antoine Bouyer
Cc: Jacopo Mondi, Michael Riesch, julien.vuillaumier, alexi.birlinger,
daniel.baluta, peng.fan, frank.li, laurent.pinchart, mchehab,
robh, krzk+dt, conor+dt, shawnguo, s.hauer, kernel, festevam,
linux-kernel, linux-media, devicetree, linux-arm-kernel,
niklas soderlund, Anthony McGivern
In-Reply-To: <68cd7c6c-d914-4ea8-9a95-4764f372265a@nxp.com>
Hi Antoine
On Tue, Mar 24, 2026 at 06:44:50PM +0100, Antoine Bouyer wrote:
>
> Hi Jacopo
>
> On 3/23/26 2:18 PM, Jacopo Mondi wrote:
> >
> >
> > Hi Antoine
> >
> > On Fri, Mar 20, 2026 at 05:29:44PM +0100, Antoine Bouyer wrote:
> > > Hi Jacopo
> > >
> > > Quite some updates regarding this RFC after further analysis.
> > >
> > > Le 05/02/2026 à 10:40, Jacopo Mondi a écrit :
> > > >
> > > >
> > > > Hi Antoine
> > > >
> > > > On Wed, Feb 04, 2026 at 07:30:18PM +0100, Antoine Bouyer wrote:
> > > > > Hi Jacopo
> > > > >
> > > > > Le 04/02/2026 à 18:12, Jacopo Mondi a écrit :
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > On Tue, Feb 03, 2026 at 07:37:34PM +0100, Jacopo Mondi wrote:
> > > > > > > Hello
> > > > > > >
> > > > > > > On Thu, Jan 29, 2026 at 12:00:24AM +0100, Michael Riesch wrote:
> > > > > > > > Hi Antoine,
> > > > > > > >
> > > > > > > > Thanks for your response.
> > > > > > > >
> > > > > > > > On 1/28/26 09:17, Antoine Bouyer wrote:
> > > > > > > > > Hi Michael
> > > > > > > > >
> > > > > > > > > On 1/26/26 10:44 AM, Michael Riesch wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi Antoine,
> > > > > > > > > >
> > > > > > > > > > On 1/23/26 09:09, Antoine Bouyer wrote:
> > > > > > > > > > > Hi all,
> > > > > > > > > > >
> > > > > > > > > > > This RFC patch series introduces the NXP Neo Image Signal Processor
> > > > > > > > > > > (ISP)
> > > > > > > > > > > driver, used in the NXP i.MX95 SoC and future devices in the i.MX9
> > > > > > > > > > > family.
> > > > > > > > > > > The series also includes updates to the generic v4l2-isp interface to
> > > > > > > > > > > support extended statistics required by the Neo ISP.
> > > > > > > > > > >
> > > > > > > > > > > The Neo ISP processes one or more camera streams, converting RAW formats
> > > > > > > > > > > into YUV or RGB outputs. Its architecture is largely influenced by the
> > > > > > > > > > > PISP driver. The hardware supports up to eight contexts, with three sink
> > > > > > > > > > > pads (main input, HDR input, and parameter buffers) and three source
> > > > > > > > > > > pads
> > > > > > > > > > > (RGB output, IR output, and statistics metadata).
> > > > > > > > > > >
> > > > > > > > > > > At this stage, both legacy (fixed-size) and extensible (dynamic-size)
> > > > > > > > > > > parameter/statistics buffers are supported through the generic v4l2-isp
> > > > > > > > > > > framework, similar to rkisp1 and Mali-C55. The driver currently supports
> > > > > > > > > > > M2M operation; direct CSI-to-ISP streaming is not yet implemented.
> > > > > > > > > >
> > > > > > > > > > How do you envisage the direct CSI-to-ISP streaming shall be supported?
> > > > > > > > >
> > > > > > > > > At this stage, this streaming mode still needs to be evaluated on
> > > > > > > > > neoisp. We should follow the integration model used by existing ISP
> > > > > > > > > drivers to avoid duplicating solutions.
> > > > > > > >
> > > > > > > > Fair point, but I have had the impression that there are not many
> > > > > > > > examples (if any). The rkisp1 driver, for instance, only supports inline
> > > > > > > > mode although the HW should be able to do both.
> > > > > > > >
> > > > > > > > But any pointers most welcome, I won't claim I have the full overview.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Below are my initial thoughts on the specific points you raised:
> > > > > > > > >
> > > > > > > > > > - How shall the final media graph(s) look like?
> > > > > > > > >
> > > > > > > > > The media entities would remain mostly identical, except for the absence
> > > > > > > > > of ISI. The topology would be a direct linkg from sensor->csi-
> > > > > > > > > > formatter->neoisp.
> > > > > > >
> > > > > > > If support for inline mode has to be added later, the ISP will need to
> > > > > > > be registered in the same media graph of the CSI-2 receiver to be able
> > > > > > > to link the two, right ?
> > > > >
> > > > > yes correct.
> > > > >
> > > > > > >
> > > > > > > How do you envision to control the ISP operating mode, because I'm
> > > > > > > afraid if you register the ISP in its own media graph, you're locking
> > > > > > > yourself there as implementing inline mode would require a different
> > > > > > > media topology with all the implications on the rest of the userspace
> > > > > > > stack.
> > > > > > >
> > > > > > > This might not be a problem if you know that the inline vs m2m mode is
> > > > > > > SoC sythesis time parameter. Some SoCs will integrate neoisp inline, some
> > > > > > > other as m2m. In this case you'll likely need two pipeline handlers
> > > > > > > in libcamera, but if that's per SoC-line maybe is acceptable. The fact
> > > > > > > you suggests in inline mode there won't be an ISI makes me think this
> > > > > > > actually depends on the SoC design ?
> > > > >
> > > > > Actually, this is not really at SoC synthesis time, neoisp HW does support
> > > > > both modes, that is configurable. But ISP HW can run in a single mode only
> > > >
> > > > > once it is configured. Streaming mode is tightly coupled with CSI HW, then
> > > > > ISP cannot be used in M2M mode with another sensor simultaneously.
> > > > >
> > > >
> > > > Yes, my point is trying to understand "how it is configured" and what
> > > > your expectations are.
> > > >
> > > > Will the board .dts (or a camera .dtso) decide how the ISP is operated
> > > > by defining its endpoint connections ? Assuming with the same SoC both
> > > > inline and m2m modes are possible, without differences in the SoC
> > > > design/integration, will users of the same board have to modify the
> > > > .dts or load ad-hoc .dtso to decide what mode is in use ?
> > > >
> > > > Then, the question of how the media topology will look and which
> > > > components registers what has to be clarified.
> > > >
> > > > Let's try to make a taxonomy of the cases we have in mainline (or on
> > > > their way to mainline).
> > > >
> > > > In the mali example I mentioned, the operating mode is selected by the
> > > > .dtsi as Mali can be integrated either inline or in m2m mode in
> > > > different SoCs. RZ/V2H in example, will always be m2m as it doesn't
> > > > interface the CSI-2 receiver with the ISP but rather interfaces the
> > > > ISP with a companion chip the performs memory access on its behalf
> > > > (the IVC). A different design that incorporates Mali inline will
> > > > instead have to interface the CSI-2 receiver with the ISP with
> > > > internal busses/glue logic and will then have to described this in dts.
> > > >
> > > > This is fine as the ISP integration is different and then having the
> > > > description in dts is legit.
> > > >
> > > > The ISP driver unconditionally registers an async notifier and the
> > > > downstream component (csi-2 or IVC) will register its async subdev(s)
> > > > which will all appear in the ISP media graph. This is possible because
> > > > the assumption is that the CSI-2 receiver (or the companion chip)
> > > > won't register their own media graph.
> > > >
> > > > The Renesas V4H example I mentioned is instead different. The ISP can
> > > > be operated in inline and m2m, on the same SoC without any
> > > > modification to hardware and to the dts/dtsi. It's basically a user
> > > > choice we defer to runtime.
> > > >
> > > > The V4H already has a component that registers a media graph: the
> > > > CSI-2/VIN block which is found in many SoCs of the same (and older)
> > > > generations. The ISP is present only in some SoC, but the CSI-2/VIN is
> > > > always there. In this case, to support both inline and m2m modes, the
> > > > VIN registers the media device and, with the trick I pointed you to in
> > > > Niklas' code, the ISP registers a subdev in the VIN media graph. Then
> > > > the inline/m2m mode can be selected by media link enablement at
> > > > run-time. Now, inline mode is not yet supported on V4H and there might
> > > > be dragons there, but at least, both modes should be possible on the same
> > > > SoC.
> > > >
> > > > On the other extremes we have the RaspberryPi PiSP BE and RkISP1.
> > > >
> > > > RPi knows the only SoC where the PiPS will be found is their one. The
> > > > ISP cannot function inline and will always be m2m. In this case, a
> > > > dedicated media graph for the ISP is the simplest and cleanest
> > > > solution.
> > > >
> > > > RkISP1 instead will always be inline only. It registers a media device
> > > > and an async notifier, the connected CSI-2 receiver will register an
> > > > async subdev and will be connected to the device tree endpoint of the
> > > > ISP device node.
> > > >
> > > > What model is the closest one to the neoisp integration that you
> > > > envision on NXP SoCs ?
> > >
> > > Then the closest model is the V4H one I believe: we both support m2m and
> > > streaming (inline) modes on the same SoC. I tested the trick you pointed
> > > out, and let the formatter sharing the media device (owned by ISI) to the
> > > neo ISP, like renesas csisp does. It registers as expected, thanks for the
> > > proposal !
> > >
> > > I think formatter is a good candidate since it is physically connected to
> > > ISP through a pixel link for streaming mode. Moreover, I propose to create a
> > > dedicated pad b/w formatter and ISP and keep the one b/w formatter and ISI
> > > as it is, so that in future we can configure the stream format which is sent
> > > to ISP, and the one sent to ISI.
> >
> > So, if I look at the ISI media graph you shared earlier in the thread,
> > the formatter will gain one source pad to be optionally connected to
> > the ISP, while the existing one that connectes to the crossbar will
> > stay as it is today ?
>
> Yes exactly. However, I don't plan to push the pad changes on the M2M patch
> series yet. I would rather create the pads (formatter and ISP) together with
> the introduction of the inline mode.
>
As long as introducing the two new pads doesn't change the enumeration
of the other ones I think it's fine. The only requirement is that
userspace that used to work with the m2m-only implementation should
continue working once the media graph is modified to support inline.
> >
> > >
> > > I also tested the streaming path can be added in device tree with endpoint
> > > connections between the nodes, so that ISP can create the media link when it
> > > registers itself to the media device.
> >
> > I think this is fine if there actually is a data path between the
> > formatter and the ISP as you have suggested.
>
> Yes there is a pixel link between formatter and ISP (at least on i.MX95
> SoC).
>
> >
> > >
> > > Thus at runtime, if userspace enables this link, then neo runs in streaming
> > > mode, otherwise m2m is used.
> >
> > So if we have an ISI, the ISP can be operated in m2m or inline based
> > on run-time link enablement, right ?
>
> Yes. And as per my understanding, ISI could still be used with inline-ISP,
> to capture raw frame.
>
> >
> > >
> > > If another SoC in future doesn't support streaming path, the endpoints can
> > > be removed from device tree, the ISP would stay in media graph anyway with
> > > m2m mode only.
> >
> > Nice!
> >
> > Do you envision a streaming mode only design, where there is no ISI
> > and the ISP has to register the media device itself ?
>
> AFAIK, there is no such design, ISI is always there.
Ok, that's good
>
> However, I initially though about adding an ISP "standalone" mode, where ISP
> could register its own media device (as it was done before). That could ease
> standalone test I believe, and limit dependency with other drivers. But I
> don't know how this can cohabit with the phandle registration approach,
> except by adding a new optional property on neoisp node to force standalone
> registration, or a module parameter.
>
> Do you think it's worth adding such standalone mode ? and if yes, how can we
> enable it in a proper way ?
>
I don't think it's something that should be mainlined. If it's useful
for your testing please go ahead, but I'm not sure this can be
mainlined...
> >
> > >
> > > Do you think this is good approach ?
> > >
> >
> > Certainly so! Thanks for the effort!
> >
> > > >
> > > > > >
> > > > > > One small correction after some more research:
> > > > > >
> > > > > > we actually already have a pipeline in libcamera that supports inline
> > > > > > and (will soon) support m2m: the mali c55 one. My take on "probably
> > > > > > need two pipeline handlers" was not correct then.
> > > > >
> > > > > Yes, I saw your patchwork on libcamera about this coming upgrade. Spent some
> > > > > time analyzing it ':) Seems we are quite aligned as per my understanding:
> > > > > inline mode (i.e. streaming mode with neoisp) _or_ M2M mode using IVC video
> > > > > device from Mali. Is that right ?
> > > > >
> > > > > >
> > > > > > As said, Mali-C55 can be integrated inline or in m2m mode and this is
> > > > > > decided based on the device tree endpoint connections.
> > > > >
> > > > > Good. Do you have an example available ?
> > > >
> > > > It's in mainline, but there's nothing exciting there as the assumption
> > > > is that there will always be a connection on the first endpoint and
> > > > the driver simply registers a notifier for the connected async subdev. If
> > > > it's a CSI-2 receiver then we're inline. If it's a companion chip
> > > > we're m2m.
> > > >
> > > > The libcamera pipeline (not upstream yet) inspects the media entity
> > > > function of the entity connected to the ISP sink pad#0. If it's a
> > > > CSI-2 reciver we're inline. If it's not, we're m2m. Based on that it
> > > > operated the pipeline differently.
> > > >
> > > > >
> > > > > >
> > > > > > So, if you know neoisp will be integrated either inline or m2m in
> > > > > > different SoC lines, maybe deferring it to device tree is good enough
> > > > > > at the expense of a slightly more complicated pipeline ?
> > > > >
> > > > > As said, SoC/ISP HW does support both modes. But I think that the selection
> > > > > can be done in device tree too. So that after bootup, a camera will be used
> > > > > only in 1 mode.
> > > > >
> > > > > >
> > > > > > I guess this has implications on the bindings definition as well..
> > > > >
> > > > > Most probably yes. Can this be done as second phase once evaluation is
> > > > > completed ?
> > > > >
> > > >
> > > > I think you should asses from the very beginning what is the planned
> > > > integration model of the ISP in order not to corner yourself in a
> > > > place where it will be hard to support inline without re-writing
> > > > the driver's media device registration logic.
> > > >
> > > > Looking at the below media graph of CSI/ISI you should ask the question "how
> > > > will I register the ISP subdev in the CSI-2 media graph when inline"
> > > > and "how will I describe inline vs m2m mode if the underlying hardware
> > > > design doesn't change?" as deferring it to the .dts might not be the
> > > > most correct way to go in that case ?
> > >
> > > So I think we are aligned now: one media graph from the beginning for
> > > supporting both modes, even if first mainline version only supports m2m.
> > > Would that be ok ?
> > >
> >
> > Yes!
> >
> > Let's only just clarify if there will ever be a mode where there is no
> > ISI as in that case I think we need to clarify who will register the
> > media graph and the async notifiers..
>
> Fine. Let me prepare a patchset with all changes already discussed then. I
> keep standalone mode out for the moment.
Fine with me!
Thanks
j
>
> BR
> Antoine
>
> >
> > > >
> > > > > >
> > > > > > >
> > > > > > > However, if you plan to allow deferring inline/m2m mode selection to
> > > > > > > the system integrators or even have it as a run-time parameter, then
> > > > > > > you should really consider having the ISP in the same media graph as
> > > > > > > the CSI-2 receiver and operate the whole CSI-2/ISI/ISP as a single
> > > > > > > media graph, where you could select the operating mode through media link
> > > > > > > enablement or dts endpoint connections
> > > > > > >
> > > > > > > Niklas (in cc) has addressed a similar situation, where inline and m2m
> > > > > > > mode can be selected by link enablement at runtime here
> > > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20251225171054.1370856-3-niklas.soderlund+renesas@ragnatech.se/
> > > > > > > (see risp_cs_internal_ops)
> > > > > > >
> > > > > > > >
> > > > > > > > OK, I thought that ISI was still around...
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > - How many media devices are registered and which driver registers it
> > > > > > > > > > or them?
> > > > > > > > >
> > > > > > > > > That will be part of the evaluation. My initial assumption is that
> > > > > > > > > neoisp would be the appropriate component to register the media device
> > > > > > > > > in this mode, since ISI is not involved, and ISI currently performs the
> > > > > > > > > registration in the M2M configuration.
> > > > > > >
> > > > > > > Isn't the ISP registering its own media graph ?
> > > > >
> > > > > Yes, 8 copies of ISP media graph, that can be used with the 8 output video
> > > > > devices of the ISI media graph.
> > > > >
> > > >
> > > > I suggest you do what RPi does. The mainline driver only registers one
> > > > instance and they carry a little patch downstream that implements the
> > > > for() loop where multiple instances are registered. Duplicating media graphs
> > > > is not desirable (at least in mainline) as we can have ISPs with 256
> > > > contexts, we don't want 256 media graphs.
> > >
> > > Ok. Will do same approach then: 1 neoisp instance on mainline + downstream
> > > patch to create other instances (x8), all in same media graph.
> >
> > Thank you. Let's work on a proper solution and then the downstream
> > patch will be dropped!
> >
> > >
> > > >
> > > > A framework level solution with proper priority handling and job
> > > > scheduling is what is required and that's what the context work should
> > > > end up being.>
> > > >
> > > > > > >
> > > > > > > Can we get a copy of all media graphs on an i.MX95 system including
> > > > > > > the ISI and the CSI-2 receiver ?
> > > > >
> > > > > Here is an example with multiple sensors. Or do you need it in another
> > > > > format ?
> > > >
> > > > No it's fine, thanks!
> > > >
> > > > >
> > > > >
> > > > > digraph board {
> > > > > rankdir=TB
> > > > > n00000001 [label="{{<port0> 0 | <port1> 1 | <port2> 2 | <port3> 3 |
> > > > > <port4> 4} | crossbar\n/dev/v4l-subdev8 | {<port5> 5 | <port6> 6 | <port7> 7
> > > > > | <port8> 8 | <port9> 9 | <port10> 10 | <port11> 11 | <port12> 12}}",
> > > > > shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000001:port5 -> n0000000f:port0 [style=bold]
> > > > > n00000001:port6 -> n0000001a:port0 [style=bold]
> > > > > n00000001:port7 -> n00000025:port0 [style=bold]
> > > > > n00000001:port8 -> n00000030:port0 [style=bold]
> > > > > n00000001:port9 -> n0000003b:port0 [style=bold]
> > > > > n00000001:port10 -> n00000046:port0 [style=bold]
> > > > > n00000001:port11 -> n00000051:port0 [style=bold]
> > > > > n00000001:port12 -> n0000005c:port0 [style=bold]
> > > > > n0000000f [label="{{<port0> 0} | mxc_isi.0\n/dev/v4l-subdev9 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n0000000f:port1 -> n00000012 [style=bold]
> > > > > n00000012 [label="mxc_isi.0.capture\n/dev/video8", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n0000001a [label="{{<port0> 0} | mxc_isi.1\n/dev/v4l-subdev10 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n0000001a:port1 -> n0000001d [style=bold]
> > > > > n0000001d [label="mxc_isi.1.capture\n/dev/video9", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n00000025 [label="{{<port0> 0} | mxc_isi.2\n/dev/v4l-subdev11 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000025:port1 -> n00000028 [style=bold]
> > > > > n00000028 [label="mxc_isi.2.capture\n/dev/video10", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n00000030 [label="{{<port0> 0} | mxc_isi.3\n/dev/v4l-subdev12 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000030:port1 -> n00000033 [style=bold]
> > > > > n00000033 [label="mxc_isi.3.capture\n/dev/video13", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n0000003b [label="{{<port0> 0} | mxc_isi.4\n/dev/v4l-subdev13 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n0000003b:port1 -> n0000003e [style=bold]
> > > > > n0000003e [label="mxc_isi.4.capture\n/dev/video14", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n00000046 [label="{{<port0> 0} | mxc_isi.5\n/dev/v4l-subdev14 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000046:port1 -> n00000049 [style=bold]
> > > > > n00000049 [label="mxc_isi.5.capture\n/dev/video21", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n00000051 [label="{{<port0> 0} | mxc_isi.6\n/dev/v4l-subdev15 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000051:port1 -> n00000054 [style=bold]
> > > > > n00000054 [label="mxc_isi.6.capture\n/dev/video22", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n0000005c [label="{{<port0> 0} | mxc_isi.7\n/dev/v4l-subdev16 |
> > > > > {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n0000005c:port1 -> n0000005f [style=bold]
> > > > > n0000005f [label="mxc_isi.7.capture\n/dev/video23", shape=box,
> > > > > style=filled, fillcolor=yellow]
> > > > > n00000067 [label="mxc_isi.output\n", shape=box, style=filled,
> > > > > fillcolor=yellow]
> > > > > n00000067 -> n00000001:port4 [style=bold]
> > > > > n0000006e [label="{{<port0> 0} |
> > > > > 4ac10000.syscon:formatter@20\n/dev/v4l-subdev17 | {<port1> 1}}",
> > > > > shape=Mrecord, style=filled, fillcolor=green]
> > > > > n0000006e:port1 -> n00000001:port2 [style=bold]
> > > > > n00000073 [label="{{<port0> 0} |
> > > > > csidev-4ad30000.csi\n/dev/v4l-subdev18 | {<port1> 1}}", shape=Mrecord,
> > > > > style=filled, fillcolor=green]
> > > > > n00000073:port1 -> n0000006e:port0 [style=bold]
> > > > > n00000078 [label="{{<port0> 0 | <port1> 1 | <port2> 2 | <port3> 3} |
> > > > > max96724 2-0027\n/dev/v4l-subdev19 | {<port4> 4 | <port5> 5}}",
> > > > > shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000078:port4 -> n00000073:port0 [style=dashed]
> > > > > n00000081 [label="{{} | mx95mbcam 8-0040\n/dev/v4l-subdev20 |
> > > > > {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000081:port0 -> n00000078:port0 [style=bold]
> > > > > n00000085 [label="{{} | mx95mbcam 9-0040\n/dev/v4l-subdev21 |
> > > > > {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000085:port0 -> n00000078:port1 [style=bold]
> > > > > n00000089 [label="{{} | mx95mbcam 10-0040\n/dev/v4l-subdev22 |
> > > > > {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n00000089:port0 -> n00000078:port2 [style=bold]
> > > > > n0000008d [label="{{} | mx95mbcam 11-0040\n/dev/v4l-subdev23 |
> > > > > {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> > > > > n0000008d:port0 -> n00000078:port3 [style=bold]
> > > > > }
> > > > >
> > > > >
> > > > > > >
> > > > > > > If I'm not mistaken you'll have 8 copies of the ISP media graphs, and
> > > > > > > that's exactly what we're working on with the context framework :)
> > > > > > >
> > > > >
> > > > > Ok. Then I should have a look to context framework too ...
> > > > >
> > > >
> > > > Please, I hope to be able to resume working on it sooner or later
> > > > given the right use case.
> > >
> > > Ok. Will continue monitoring the multi context work. Seems to be a nice
> > > feature indeed. But as impact on userspace is more significant, that can be
> > > done as a second step I guess, and will keep the multi instance downstream
> > > patch meanwhile.
> > >
> > > >
> > > > > > >
> > > > > > > >
> > > > > > > > ... since it is not, your assumption seems very reasonable.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > - How can the user decide whether direct (csi2isp) or indirect
> > > > > > > > > > (mem2mem) streaming shall be used?
> > > > > > > > >
> > > > > > > > > That will also be part of the evaluation. From dts would be my first
> > > > > > > > > option, but may prevent using both modes on same platform then.
> > > > > > > >
> > > > > > > > Of course this depends what the hardware is able to do, but in case the
> > > > > > > > HW is reconfigurable easily, I doubt that device tree is a good choice
> > > > > > > > to solve that.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > While it is certainly OK to introduce this support only at a later
> > > > > > > > > > stage, it makes sense to consider this right from the start to avoid
> > > > > > > > > > some nasty changes e.g. in how this hardware is exposed to user space.
> > > > > > > > > >
> > > > > > > > > > Also, we are facing a similiar challenge with recent Rockchip ISP
> > > > > > > > > > hardware (RK3588, RK3576, ...) and it would be great to hear your
> > > > > > > > > > thoughts about that.
> > > > > > > > >
> > > > > > > > > Is there an existing discussion thread available on this topic? I would
> > > > > > > > > be very interested in following it.
> > > > > > > >
> > > > > > > > Not yet, I am afraid. But there should be one or two soon (TM) :-)
> > > > > > >
> > > > > > > It's probably time to have one :)
> > > > >
> > > > > Good. Please loop me in ;)
> > > >
> > > > You are in, this is the conversation ;)
> > > >
> > > > It might be a good discussion point for the media summit in Nice
> > > > co-located with Embedded Recipes if people with interest in the topic
> > > > will going the be there.
> > >
> > > Great ! Will try to join then.
> > >
> >
> > I'm not sure yet how many interested parties will be in Nice and if it
> > would make sense to organize an "ISP design day" there or should we
> > plan a devroom for Plumbers in Prague ?
> >
> >
> > > BR
> > > Antoine
> > >
> > > >
> > > > I'm also adding Anthony from ARM as I know he's going through the same
> > > > inline/m2m duality you're now facing.
> > > >
> > > > Thanks
> > > > j
> > > >
> > > > >
> > > > > BR
> > > > > Antoine
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Thanks and regards,
> > > > > > > > Michael
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > > Antoine
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks in advance and best regards,
> > > > > > > > > > Michael
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > This series is posted as RFC because extending the v4l2-isp interface
> > > > > > > > > > > may
> > > > > > > > > > > overlap with ongoing work. If similar development already exists, I am
> > > > > > > > > > > happy to rebase or adapt the series accordingly. If preferred, the
> > > > > > > > > > > series
> > > > > > > > > > > can also be split into two parts: the v4l2-isp rework and the Neo ISP
> > > > > > > > > > > driver introduction.
> > > > > > > > > > >
> > > > > > > > > > > A few checkpatch warnings in v4l2-ioctl.c remain intentionally to stay
> > > > > > > > > > > consistent with the existing style in that file.
> > > > > > > > > > >
> > > > > > > > > > > Testing was performed on the i.MX95 EVK using the media/next kernel in
> > > > > > > > > > > standalone M2M mode. End-to-end camera-to-ISP capture has been validated
> > > > > > > > > > > using the downstream NXP kernel, as some hardware dependencies are not
> > > > > > > > > > > yet upstreamed.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Antoine
> > > > > > > > > > >
> > > > > > > > > > > ---
> > > > > > > > > > > Here are v4l2-compliance test results:
> > > > > > > > > > >
> > > > > > > > > > > v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t
> > > > > > > > > > > v4l2-compliance SHA: fc15e229d9d3 2024-07-23 19:22:15
> > > > > > > > > > >
> > > > > > > > > > > Compliance test for neoisp device /dev/media0:
> > > > > > > > > > >
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MEDIA_IOC_DEVICE_INFO: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/media0 open: OK
> > > > > > > > > > > test MEDIA_IOC_DEVICE_INFO: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Media Controller ioctls:
> > > > > > > > > > > test MEDIA_IOC_G_TOPOLOGY: OK
> > > > > > > > > > > Entities: 7 Interfaces: 7 Pads: 12 Links: 13
> > > > > > > > > > > test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> > > > > > > > > > > test MEDIA_IOC_SETUP_LINK: OK
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/media0: 8, Succeeded: 8, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/video0:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Card type : neoisp
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x8ca03000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Device Capabilities
> > > > > > > > > > > Device Caps : 0x04202000
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x0300000a
> > > > > > > > > > > Type : V4L Video
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x00000008 (8)
> > > > > > > > > > > Name : neoisp-input0
> > > > > > > > > > > Function : V4L2 I/O
> > > > > > > > > > > Pad 0x01000009 : 0: Source
> > > > > > > > > > > Link 0x0200000c: to remote pad 0x1000002 of entity
> > > > > > > > > > > 'neoisp' (Image Signal Processor): Data, Enabled, Immutable
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/video0 open: OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test VIDIOC_G/S_PRIORITY: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 0 Private Controls: 0
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK
> > > > > > > > > > > test VIDIOC_S_FMT: OK
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/video0: 48, Succeeded: 48, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/video1:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Card type : neoisp
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x8ca03000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Device Capabilities
> > > > > > > > > > > Device Caps : 0x04202000
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x03000010
> > > > > > > > > > > Type : V4L Video
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x0000000e (14)
> > > > > > > > > > > Name : neoisp-input1
> > > > > > > > > > > Function : V4L2 I/O
> > > > > > > > > > > Pad 0x0100000f : 0: Source
> > > > > > > > > > > Link 0x02000012: to remote pad 0x1000003 of entity
> > > > > > > > > > > 'neoisp' (Image Signal Processor): Data
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/video1 open: OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test VIDIOC_G/S_PRIORITY: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 0 Private Controls: 0
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK
> > > > > > > > > > > test VIDIOC_S_FMT: OK
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/video1: 48, Succeeded: 48, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/video2:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Card type : neoisp
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x8ca03000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Device Capabilities
> > > > > > > > > > > Device Caps : 0x0c200000
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x03000016
> > > > > > > > > > > Type : V4L Video
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x00000014 (20)
> > > > > > > > > > > Name : neoisp-params
> > > > > > > > > > > Function : V4L2 I/O
> > > > > > > > > > > Pad 0x01000015 : 0: Source
> > > > > > > > > > > Link 0x02000018: to remote pad 0x1000004 of entity
> > > > > > > > > > > 'neoisp' (Image Signal Processor): Data, Enabled
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/video2 open: OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test VIDIOC_G/S_PRIORITY: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 0 Private Controls: 0
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK
> > > > > > > > > > > test VIDIOC_S_FMT: OK
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK (Not Supported)
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/video2: 48, Succeeded: 48, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/video3:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Card type : neoisp
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x8ca03000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Device Capabilities
> > > > > > > > > > > Device Caps : 0x04201000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x0300001c
> > > > > > > > > > > Type : V4L Video
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x0000001a (26)
> > > > > > > > > > > Name : neoisp-frame
> > > > > > > > > > > Function : V4L2 I/O
> > > > > > > > > > > Pad 0x0100001b : 0: Sink
> > > > > > > > > > > Link 0x0200001e: from remote pad 0x1000005 of entity
> > > > > > > > > > > 'neoisp' (Image Signal Processor): Data, Enabled
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/video3 open: OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test VIDIOC_G/S_PRIORITY: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 0 Private Controls: 0
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK
> > > > > > > > > > > test VIDIOC_S_FMT: OK
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK (Not Supported)
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/video3: 48, Succeeded: 48, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/video4:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Card type : neoisp
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x8ca03000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Device Capabilities
> > > > > > > > > > > Device Caps : 0x04201000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x03000022
> > > > > > > > > > > Type : V4L Video
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x00000020 (32)
> > > > > > > > > > > Name : neoisp-ir
> > > > > > > > > > > Function : V4L2 I/O
> > > > > > > > > > > Pad 0x01000021 : 0: Sink
> > > > > > > > > > > Link 0x02000024: from remote pad 0x1000006 of entity
> > > > > > > > > > > 'neoisp' (Image Signal Processor): Data
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/video4 open: OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test VIDIOC_G/S_PRIORITY: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 0 Private Controls: 0
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK
> > > > > > > > > > > test VIDIOC_S_FMT: OK
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK (Not Supported)
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/video4: 48, Succeeded: 48, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/video5:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Card type : neoisp
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x8ca03000
> > > > > > > > > > > Video Capture Multiplanar
> > > > > > > > > > > Video Output Multiplanar
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Metadata Output
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Device Capabilities
> > > > > > > > > > > Device Caps : 0x04a00000
> > > > > > > > > > > Metadata Capture
> > > > > > > > > > > Streaming
> > > > > > > > > > > Extended Pix Format
> > > > > > > > > > > Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x03000028
> > > > > > > > > > > Type : V4L Video
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x00000026 (38)
> > > > > > > > > > > Name : neoisp-stats
> > > > > > > > > > > Function : V4L2 I/O
> > > > > > > > > > > Pad 0x01000027 : 0: Sink
> > > > > > > > > > > Link 0x0200002a: from remote pad 0x1000007 of entity
> > > > > > > > > > > 'neoisp' (Image Signal Processor): Data, Enabled
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/video5 open: OK
> > > > > > > > > > > test VIDIOC_QUERYCAP: OK
> > > > > > > > > > > test VIDIOC_G/S_PRIORITY: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 0 Private Controls: 0
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK
> > > > > > > > > > > test VIDIOC_S_FMT: OK
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK (Not Supported)
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/video5: 48, Succeeded: 48, Failed: 0,
> > > > > > > > > > > Warnings: 0
> > > > > > > > > > > --------------------------------------------------------------------------------
> > > > > > > > > > > Compliance test for neoisp device /dev/v4l-subdev0:
> > > > > > > > > > >
> > > > > > > > > > > Driver Info:
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Capabilities : 0x00000000
> > > > > > > > > > > Client Capabilities: 0x0000000000000002
> > > > > > > > > > > interval-uses-which Media Driver Info:
> > > > > > > > > > > Driver name : neoisp
> > > > > > > > > > > Model : neoisp
> > > > > > > > > > > Serial :
> > > > > > > > > > > Bus info : platform:4ae00000.isp
> > > > > > > > > > > Media version : 6.19.0
> > > > > > > > > > > Hardware revision: 0x00000002 (2)
> > > > > > > > > > > Driver version : 6.19.0
> > > > > > > > > > > Interface Info:
> > > > > > > > > > > ID : 0x0300002c
> > > > > > > > > > > Type : V4L Sub-Device
> > > > > > > > > > > Entity Info:
> > > > > > > > > > > ID : 0x00000001 (1)
> > > > > > > > > > > Name : neoisp
> > > > > > > > > > > Function : Image Signal Processor
> > > > > > > > > > > Pad 0x01000002 : 0: Sink
> > > > > > > > > > > Link 0x0200000c: from remote pad 0x1000009 of entity
> > > > > > > > > > > 'neoisp-input0' (V4L2 I/O): Data, Enabled, Immutable
> > > > > > > > > > > Pad 0x01000003 : 1: Sink
> > > > > > > > > > > Link 0x02000012: from remote pad 0x100000f of entity
> > > > > > > > > > > 'neoisp-input1' (V4L2 I/O): Data
> > > > > > > > > > > Pad 0x01000004 : 2: Sink
> > > > > > > > > > > Link 0x02000018: from remote pad 0x1000015 of entity
> > > > > > > > > > > 'neoisp-params' (V4L2 I/O): Data, Enabled
> > > > > > > > > > > Pad 0x01000005 : 3: Source
> > > > > > > > > > > Link 0x0200001e: to remote pad 0x100001b of entity 'neoisp-
> > > > > > > > > > > frame' (V4L2 I/O): Data, Enabled
> > > > > > > > > > > Pad 0x01000006 : 4: Source
> > > > > > > > > > > Link 0x02000024: to remote pad 0x1000021 of entity 'neoisp-
> > > > > > > > > > > ir' (V4L2 I/O): Data
> > > > > > > > > > > Pad 0x01000007 : 5: Source
> > > > > > > > > > > Link 0x0200002a: to remote pad 0x1000027 of entity 'neoisp-
> > > > > > > > > > > stats' (V4L2 I/O): Data, Enabled
> > > > > > > > > > >
> > > > > > > > > > > Required ioctls:
> > > > > > > > > > > test MC information (see 'Media Driver Info' above): OK
> > > > > > > > > > > test VIDIOC_SUDBEV_QUERYCAP: OK
> > > > > > > > > > > test invalid ioctls: OK
> > > > > > > > > > >
> > > > > > > > > > > Allow for multiple opens:
> > > > > > > > > > > test second /dev/v4l-subdev0 open: OK
> > > > > > > > > > > test VIDIOC_SUBDEV_QUERYCAP: OK
> > > > > > > > > > > test for unlimited opens: OK
> > > > > > > > > > >
> > > > > > > > > > > Debug ioctls:
> > > > > > > > > > > test VIDIOC_LOG_STATUS: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Input ioctls:
> > > > > > > > > > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > > > > > > > > > > Inputs: 0 Audio Inputs: 0 Tuners: 0
> > > > > > > > > > >
> > > > > > > > > > > Output ioctls:
> > > > > > > > > > > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > > > > > > > > > > Outputs: 0 Audio Outputs: 0 Modulators: 0
> > > > > > > > > > >
> > > > > > > > > > > Input/Output configuration ioctls:
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G/S_EDID: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Sub-Device ioctls (Sink Pad 0):
> > > > > > > > > > > Try Stream 0
> > > > > > > > > > > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > Active Stream 0
> > > > > > > > > > > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Sub-Device ioctls (Sink Pad 1):
> > > > > > > > > > > Try Stream 0
> > > > > > > > > > > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > Active Stream 0
> > > > > > > > > > > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Sub-Device ioctls (Sink Pad 2):
> > > > > > > > > > > Try Stream 0
> > > > > > > > > > > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > Active Stream 0
> > > > > > > > > > > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Sub-Device ioctls (Source Pad 3):
> > > > > > > > > > > Try Stream 0
> > > > > > > > > > > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > Active Stream 0
> > > > > > > > > > > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Sub-Device ioctls (Source Pad 4):
> > > > > > > > > > > Try Stream 0
> > > > > > > > > > > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > Active Stream 0
> > > > > > > > > > > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Sub-Device ioctls (Source Pad 5):
> > > > > > > > > > > Try Stream 0
> > > > > > > > > > > test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > Active Stream 0
> > > > > > > > > > > test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/
> > > > > > > > > > > FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FMT: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
> > > > > > > > > > > test Active VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Control ioctls:
> > > > > > > > > > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> > > > > > > > > > > test VIDIOC_QUERYCTRL: OK
> > > > > > > > > > > test VIDIOC_G/S_CTRL: OK
> > > > > > > > > > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> > > > > > > > > > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> > > > > > > > > > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > > > > > > > > > > Standard Controls: 1 Private Controls: 1
> > > > > > > > > > >
> > > > > > > > > > > Format ioctls:
> > > > > > > > > > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not
> > > > > > > > > > > Supported)
> > > > > > > > > > > test VIDIOC_G/S_PARM: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FBUF: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_FMT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_TRY_FMT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_S_FMT: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > > > > > > > > > > test Cropping: OK (Not Supported)
> > > > > > > > > > > test Composing: OK (Not Supported)
> > > > > > > > > > > test Scaling: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Codec ioctls:
> > > > > > > > > > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > > > > > > > > > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Buffer ioctls:
> > > > > > > > > > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> > > > > > > > > > > test CREATE_BUFS maximum buffers: OK
> > > > > > > > > > > test VIDIOC_REMOVE_BUFS: OK
> > > > > > > > > > > test VIDIOC_EXPBUF: OK (Not Supported)
> > > > > > > > > > > test Requests: OK (Not Supported)
> > > > > > > > > > >
> > > > > > > > > > > Total for neoisp device /dev/v4l-subdev0: 88, Succeeded: 88, Failed:
> > > > > > > > > > > 0, Warnings: 0
> > > > > > > > > > >
> > > > > > > > > > > Grand Total for neoisp device /dev/media0: 384, Succeeded: 384,
> > > > > > > > > > > Failed: 0, Warnings: 0
> > > > > > > > > > >
> > > > > > > > > > > ---
> > > > > > > > > > > Antoine Bouyer (11):
> > > > > > > > > > > media: uapi: v4l2-isp: Add v4l2 ISP extensible statistics definitions
> > > > > > > > > > > media: v4l2-isp: Add helper function to compute extended stats size
> > > > > > > > > > > media: Documentation: uapi: Update V4L2 ISP for extensible stats
> > > > > > > > > > > media: Documentation: Add NXP neoisp driver documentation
> > > > > > > > > > > dt-bindings: media: Add nxp neoisp support
> > > > > > > > > > > media: v4l2-ctrls: Add user control base for NXP neoisp controls
> > > > > > > > > > > media: Add meta formats supported by NXP neoisp driver
> > > > > > > > > > > media: uapi: Add NXP NEOISP user interface header file
> > > > > > > > > > > media: platform: Add NXP Neoisp Image Signal Processor
> > > > > > > > > > > media: platform: neoisp: Add debugfs support
> > > > > > > > > > > arm64: dts: freescale: imx95: Add NXP neoisp device tree node
> > > > > > > > > > >
> > > > > > > > > > > .../admin-guide/media/nxp-neoisp-diagram.dot | 22 +
> > > > > > > > > > > .../admin-guide/media/nxp-neoisp.dot | 16 +
> > > > > > > > > > > .../admin-guide/media/nxp-neoisp.rst | 189 ++
> > > > > > > > > > > .../admin-guide/media/v4l-drivers.rst | 1 +
> > > > > > > > > > > .../devicetree/bindings/media/nxp,neoisp.yaml | 65 +
> > > > > > > > > > > .../userspace-api/media/v4l/meta-formats.rst | 1 +
> > > > > > > > > > > .../media/v4l/metafmt-nxp-neoisp.rst | 114 +
> > > > > > > > > > > .../userspace-api/media/v4l/v4l2-isp.rst | 42 +-
> > > > > > > > > > > MAINTAINERS | 9 +
> > > > > > > > > > > .../boot/dts/freescale/imx95-19x19-evk.dts | 4 +
> > > > > > > > > > > arch/arm64/boot/dts/freescale/imx95.dtsi | 11 +
> > > > > > > > > > > drivers/media/platform/nxp/Kconfig | 1 +
> > > > > > > > > > > drivers/media/platform/nxp/Makefile | 1 +
> > > > > > > > > > > drivers/media/platform/nxp/neoisp/Kconfig | 15 +
> > > > > > > > > > > drivers/media/platform/nxp/neoisp/Makefile | 8 +
> > > > > > > > > > > drivers/media/platform/nxp/neoisp/neoisp.h | 270 ++
> > > > > > > > > > > .../media/platform/nxp/neoisp/neoisp_ctx.c | 2798 +++++++++++++++++
> > > > > > > > > > > .../media/platform/nxp/neoisp/neoisp_ctx.h | 85 +
> > > > > > > > > > > .../platform/nxp/neoisp/neoisp_debugfs.c | 503 +++
> > > > > > > > > > > .../media/platform/nxp/neoisp/neoisp_fmt.h | 509 +++
> > > > > > > > > > > drivers/media/platform/nxp/neoisp/neoisp_hw.h | 577 ++++
> > > > > > > > > > > .../media/platform/nxp/neoisp/neoisp_main.c | 1999 ++++++++++++
> > > > > > > > > > > .../media/platform/nxp/neoisp/neoisp_nodes.h | 60 +
> > > > > > > > > > > .../media/platform/nxp/neoisp/neoisp_regs.h | 2501 +++++++++++++++
> > > > > > > > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 4 +
> > > > > > > > > > > include/media/v4l2-isp.h | 13 +
> > > > > > > > > > > include/uapi/linux/media/nxp/nxp_neoisp.h | 1968 ++++++++++++
> > > > > > > > > > > include/uapi/linux/media/v4l2-isp.h | 85 +
> > > > > > > > > > > include/uapi/linux/v4l2-controls.h | 6 +
> > > > > > > > > > > include/uapi/linux/videodev2.h | 6 +
> > > > > > > > > > > 30 files changed, 11880 insertions(+), 3 deletions(-)
> > > > > > > > > > > create mode 100644 Documentation/admin-guide/media/nxp-neoisp-
> > > > > > > > > > > diagram.dot
> > > > > > > > > > > create mode 100644 Documentation/admin-guide/media/nxp-neoisp.dot
> > > > > > > > > > > create mode 100644 Documentation/admin-guide/media/nxp-neoisp.rst
> > > > > > > > > > > create mode 100644 Documentation/devicetree/bindings/media/
> > > > > > > > > > > nxp,neoisp.yaml
> > > > > > > > > > > create mode 100644 Documentation/userspace-api/media/v4l/metafmt-
> > > > > > > > > > > nxp-neoisp.rst
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/Kconfig
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/Makefile
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp.h
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_ctx.c
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_ctx.h
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_debugfs.c
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_fmt.h
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_hw.h
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_main.c
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_nodes.h
> > > > > > > > > > > create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_regs.h
> > > > > > > > > > > create mode 100644 include/uapi/linux/media/nxp/nxp_neoisp.h
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > >
>
>
^ permalink raw reply
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Nicolas Frattaroli @ 2026-03-25 13:21 UTC (permalink / raw)
To: Dave Stevenson, Maxime Ripard
Cc: Harry Wentland, Leo Li, Rodrigo Siqueira, Alex Deucher,
Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Thomas Zimmermann, Andrzej Hajda,
Neil Armstrong, Robert Foss, Laurent Pinchart, Jonas Karlman,
Jernej Skrabec, Sandy Huang, Heiko Stübner, Andy Yan,
Jani Nikula, Rodrigo Vivi, Joonas Lahtinen, Tvrtko Ursulin,
Dmitry Baryshkov, Sascha Hauer, Rob Herring, Jonathan Corbet,
Shuah Khan, kernel, amd-gfx, dri-devel, linux-kernel,
linux-arm-kernel, linux-rockchip, intel-gfx, intel-xe, linux-doc,
Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <20260325-quaint-bull-of-fortitude-dc68da@houat>
On Wednesday, 25 March 2026 14:05:25 Central European Standard Time Maxime Ripard wrote:
> Hi Dave,
>
> On Wed, Mar 25, 2026 at 12:49:19PM +0000, Dave Stevenson wrote:
> > > diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
> > > index af8b92d2d5b7..bd549f912b76 100644
> > > --- a/include/drm/drm_connector.h
> > > +++ b/include/drm/drm_connector.h
> > > @@ -571,14 +571,102 @@ enum drm_colorspace {
> > > * YCbCr 4:2:2 output format (ie. with horizontal subsampling)
> > > * @DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > > * YCbCr 4:2:0 output format (ie. with horizontal and vertical subsampling)
> > > + * @DRM_OUTPUT_COLOR_FORMAT_COUNT:
> > > + * Number of valid output color format values in this enum
> > > */
> > > enum drm_output_color_format {
> > > DRM_OUTPUT_COLOR_FORMAT_RGB444 = 0,
> > > DRM_OUTPUT_COLOR_FORMAT_YCBCR444,
> > > DRM_OUTPUT_COLOR_FORMAT_YCBCR422,
> > > DRM_OUTPUT_COLOR_FORMAT_YCBCR420,
> > > + DRM_OUTPUT_COLOR_FORMAT_COUNT,
> > > };
> > >
> > > +/**
> > > + * enum drm_connector_color_format - Connector Color Format Request
> > > + *
> > > + * This enum, unlike &enum drm_output_color_format, is used to specify requests
> > > + * for a specific color format on a connector through the DRM "color format"
> > > + * property. The difference is that it has an "AUTO" value to specify that
> > > + * no specific choice has been made.
> > > + */
> > > +enum drm_connector_color_format {
> > > + /**
> > > + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> > > + * helpers should pick a suitable color format. All implementations of a
> > > + * specific display protocol must behave the same way with "AUTO", but
> > > + * different display protocols do not necessarily have the same "AUTO"
> > > + * semantics.
> > > + *
> > > + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> > > + * bandwidth required for full-scale RGB is not available, or the mode
> > > + * is YCbCr 4:2:0-only, as long as the mode and output both support
> > > + * YCbCr 4:2:0.
> >
> > Is there a reason you propose dropping back to YCbCr 4:2:0 without
> > trying YCbCr 4:2:2 first? Minimising the subsampling is surely
> > beneficial, and vc4 for one can do 4:2:2 but not 4:2:0.
>
> The "auto" behaviour is strictly identical to the one we have right now,
> and this one stems from i915. Back when all that logic was added, it was
> decided to align every driver behavior on i915 because that's what most
> compositors would expect.
Hi Maxime,
would it be okay to extend the behavior while we're at it? 4:2:2 does save
bandwidth compared to RGB (unlike YCbCr 4:4:4). I do think 4:2:2 instead of
4:2:0 will provide benefits in some cases. I assume hardware that supports
4:2:2 only instead of 4:2:0 does so in order to save >= 1 horizontal
row of local SRAM buffer in the display controller for any downscaling,
as it'll only need to consider neighbouring pixels on the same row.
Kind regards,
Nicolas Frattaroli
>
> Maxime
>
^ permalink raw reply
* [PATCH] media: verisilicon: AV1: Discard none supported profiles
From: Benjamin Gaignard @ 2026-03-25 13:19 UTC (permalink / raw)
To: nicolas.dufresne, p.zabel, mchehab, heiko
Cc: linux-media, linux-rockchip, linux-kernel, linux-arm-kernel,
kernel, Benjamin Gaignard
AV1 hardware can only decode YUV420 pixel format. That means
that only profile 0 is supported. Discard all other profiles
when receiving AV1 sequence control.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_drv.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c
index 2e81877f640f..12f98e7e7994 100644
--- a/drivers/media/platform/verisilicon/hantro_drv.c
+++ b/drivers/media/platform/verisilicon/hantro_drv.c
@@ -283,6 +283,8 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
if (sequence->bit_depth != 8 && sequence->bit_depth != 10)
return -EINVAL;
+ if (sequence->seq_profile != 0)
+ return -EINVAL;
}
return 0;
--
2.43.0
^ permalink raw reply related
* [PATCH v2] media: verisilicon: Simplify motion vectors and rfc buffers allocation
From: Benjamin Gaignard @ 2026-03-25 13:17 UTC (permalink / raw)
To: nicolas.dufresne, p.zabel, mchehab, heiko
Cc: linux-media, linux-rockchip, linux-kernel, linux-arm-kernel,
kernel, Benjamin Gaignard
Until now we reserve the space needed for motion vectors and reference
frame compression at the end of the frame buffer.
This patch disentanglement mv and rfc from frame buffers by allocating
distinct buffers for each purpose.
That simplify the code by removing lot of offset computation.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- rework commit message
- free mv and rfc buffer before signal the buffer completion.
drivers/media/platform/verisilicon/hantro.h | 17 +-
.../media/platform/verisilicon/hantro_av1.c | 7 -
.../media/platform/verisilicon/hantro_av1.h | 1 -
.../media/platform/verisilicon/hantro_g2.c | 36 --
.../platform/verisilicon/hantro_g2_hevc_dec.c | 24 +-
.../platform/verisilicon/hantro_g2_vp9_dec.c | 12 +-
.../media/platform/verisilicon/hantro_hevc.c | 20 +-
.../media/platform/verisilicon/hantro_hw.h | 99 +-----
.../platform/verisilicon/hantro_postproc.c | 29 +-
.../media/platform/verisilicon/hantro_v4l2.c | 314 ++++++++++++++++--
.../verisilicon/rockchip_vpu981_hw_av1_dec.c | 16 +-
11 files changed, 359 insertions(+), 216 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index 0353de154a1e..c4ceb9c99016 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -31,6 +31,9 @@ struct hantro_ctx;
struct hantro_codec_ops;
struct hantro_postproc_ops;
+#define MAX_MV_BUFFERS MAX_POSTPROC_BUFFERS
+#define MAX_RFC_BUFFERS MAX_POSTPROC_BUFFERS
+
#define HANTRO_JPEG_ENCODER BIT(0)
#define HANTRO_ENCODERS 0x0000ffff
#define HANTRO_MPEG2_DECODER BIT(16)
@@ -237,6 +240,9 @@ struct hantro_dev {
* @need_postproc: Set to true if the bitstream features require to
* use the post-processor.
*
+ * @dec_mv: motion vectors buffers for the context.
+ * @dec_rfc: reference frame compression buffers for the context.
+ *
* @codec_ops: Set of operations related to codec mode.
* @postproc: Post-processing context.
* @h264_dec: H.264-decoding context.
@@ -264,6 +270,9 @@ struct hantro_ctx {
int jpeg_quality;
int bit_depth;
+ struct hantro_aux_buf dec_mv[MAX_MV_BUFFERS];
+ struct hantro_aux_buf dec_rfc[MAX_RFC_BUFFERS];
+
const struct hantro_codec_ops *codec_ops;
struct hantro_postproc_ctx postproc;
bool need_postproc;
@@ -334,14 +343,14 @@ struct hantro_vp9_decoded_buffer_info {
unsigned short width;
unsigned short height;
size_t chroma_offset;
- size_t mv_offset;
+ dma_addr_t mv_addr;
u32 bit_depth : 4;
};
struct hantro_av1_decoded_buffer_info {
/* Info needed when the decoded frame serves as a reference frame. */
size_t chroma_offset;
- size_t mv_offset;
+ dma_addr_t mv_addr;
};
struct hantro_decoded_buffer {
@@ -507,4 +516,8 @@ void hantro_postproc_free(struct hantro_ctx *ctx);
int hanto_postproc_enum_framesizes(struct hantro_ctx *ctx,
struct v4l2_frmsizeenum *fsize);
+dma_addr_t hantro_mv_get_buf_addr(struct hantro_ctx *ctx, int index);
+dma_addr_t hantro_rfc_get_luma_buf_addr(struct hantro_ctx *ctx, int index);
+dma_addr_t hantro_rfc_get_chroma_buf_addr(struct hantro_ctx *ctx, int index);
+
#endif /* HANTRO_H_ */
diff --git a/drivers/media/platform/verisilicon/hantro_av1.c b/drivers/media/platform/verisilicon/hantro_av1.c
index 5a51ac877c9c..3a80a7994f67 100644
--- a/drivers/media/platform/verisilicon/hantro_av1.c
+++ b/drivers/media/platform/verisilicon/hantro_av1.c
@@ -222,13 +222,6 @@ size_t hantro_av1_luma_size(struct hantro_ctx *ctx)
return ctx->ref_fmt.plane_fmt[0].bytesperline * ctx->ref_fmt.height;
}
-size_t hantro_av1_chroma_size(struct hantro_ctx *ctx)
-{
- size_t cr_offset = hantro_av1_luma_size(ctx);
-
- return ALIGN((cr_offset * 3) / 2, 64);
-}
-
static void hantro_av1_tiles_free(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
diff --git a/drivers/media/platform/verisilicon/hantro_av1.h b/drivers/media/platform/verisilicon/hantro_av1.h
index 4e2122b95cdd..330f7938d097 100644
--- a/drivers/media/platform/verisilicon/hantro_av1.h
+++ b/drivers/media/platform/verisilicon/hantro_av1.h
@@ -41,7 +41,6 @@ int hantro_av1_get_order_hint(struct hantro_ctx *ctx, int ref);
int hantro_av1_frame_ref(struct hantro_ctx *ctx, u64 timestamp);
void hantro_av1_clean_refs(struct hantro_ctx *ctx);
size_t hantro_av1_luma_size(struct hantro_ctx *ctx);
-size_t hantro_av1_chroma_size(struct hantro_ctx *ctx);
void hantro_av1_exit(struct hantro_ctx *ctx);
int hantro_av1_init(struct hantro_ctx *ctx);
int hantro_av1_prepare_run(struct hantro_ctx *ctx);
diff --git a/drivers/media/platform/verisilicon/hantro_g2.c b/drivers/media/platform/verisilicon/hantro_g2.c
index 318673b66da8..4ae7df53dcb1 100644
--- a/drivers/media/platform/verisilicon/hantro_g2.c
+++ b/drivers/media/platform/verisilicon/hantro_g2.c
@@ -99,39 +99,3 @@ size_t hantro_g2_chroma_offset(struct hantro_ctx *ctx)
{
return ctx->ref_fmt.plane_fmt[0].bytesperline * ctx->ref_fmt.height;
}
-
-size_t hantro_g2_motion_vectors_offset(struct hantro_ctx *ctx)
-{
- size_t cr_offset = hantro_g2_chroma_offset(ctx);
-
- return ALIGN((cr_offset * 3) / 2, G2_ALIGN);
-}
-
-static size_t hantro_g2_mv_size(struct hantro_ctx *ctx)
-{
- const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
- const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
- unsigned int pic_width_in_ctbs, pic_height_in_ctbs;
- unsigned int max_log2_ctb_size;
-
- max_log2_ctb_size = sps->log2_min_luma_coding_block_size_minus3 + 3 +
- sps->log2_diff_max_min_luma_coding_block_size;
- pic_width_in_ctbs = (sps->pic_width_in_luma_samples +
- (1 << max_log2_ctb_size) - 1) >> max_log2_ctb_size;
- pic_height_in_ctbs = (sps->pic_height_in_luma_samples + (1 << max_log2_ctb_size) - 1)
- >> max_log2_ctb_size;
-
- return pic_width_in_ctbs * pic_height_in_ctbs * (1 << (2 * (max_log2_ctb_size - 4))) * 16;
-}
-
-size_t hantro_g2_luma_compress_offset(struct hantro_ctx *ctx)
-{
- return hantro_g2_motion_vectors_offset(ctx) +
- hantro_g2_mv_size(ctx);
-}
-
-size_t hantro_g2_chroma_compress_offset(struct hantro_ctx *ctx)
-{
- return hantro_g2_luma_compress_offset(ctx) +
- hantro_hevc_luma_compressed_size(ctx->dst_fmt.width, ctx->dst_fmt.height);
-}
diff --git a/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c b/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c
index e8c2e83379de..d0af9fb882ba 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_hevc_dec.c
@@ -383,9 +383,6 @@ static int set_ref(struct hantro_ctx *ctx)
struct vb2_v4l2_buffer *vb2_dst;
struct hantro_decoded_buffer *dst;
size_t cr_offset = hantro_g2_chroma_offset(ctx);
- size_t mv_offset = hantro_g2_motion_vectors_offset(ctx);
- size_t compress_luma_offset = hantro_g2_luma_compress_offset(ctx);
- size_t compress_chroma_offset = hantro_g2_chroma_compress_offset(ctx);
u32 max_ref_frames;
u16 dpb_longterm_e;
static const struct hantro_reg cur_poc[] = {
@@ -453,14 +450,17 @@ static int set_ref(struct hantro_ctx *ctx)
dpb_longterm_e = 0;
for (i = 0; i < decode_params->num_active_dpb_entries &&
i < (V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1); i++) {
+ int index = hantro_hevc_get_ref_buf_index(ctx, dpb[i].pic_order_cnt_val);
luma_addr = hantro_hevc_get_ref_buf(ctx, dpb[i].pic_order_cnt_val);
if (!luma_addr)
return -ENOMEM;
chroma_addr = luma_addr + cr_offset;
- mv_addr = luma_addr + mv_offset;
- compress_luma_addr = luma_addr + compress_luma_offset;
- compress_chroma_addr = luma_addr + compress_chroma_offset;
+ mv_addr = hantro_mv_get_buf_addr(ctx, index);
+ if (ctx->hevc_dec.use_compression) {
+ compress_luma_addr = hantro_rfc_get_luma_buf_addr(ctx, index);
+ compress_chroma_addr = hantro_rfc_get_chroma_buf_addr(ctx, index);
+ }
if (dpb[i].flags & V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE)
dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i);
@@ -478,13 +478,17 @@ static int set_ref(struct hantro_ctx *ctx)
if (!luma_addr)
return -ENOMEM;
- if (hantro_hevc_add_ref_buf(ctx, decode_params->pic_order_cnt_val, luma_addr))
+ if (hantro_hevc_add_ref_buf(ctx, decode_params->pic_order_cnt_val, luma_addr, vb2_dst))
return -EINVAL;
chroma_addr = luma_addr + cr_offset;
- mv_addr = luma_addr + mv_offset;
- compress_luma_addr = luma_addr + compress_luma_offset;
- compress_chroma_addr = luma_addr + compress_chroma_offset;
+ mv_addr = hantro_mv_get_buf_addr(ctx, dst->base.vb.vb2_buf.index);
+ if (ctx->hevc_dec.use_compression) {
+ compress_luma_addr =
+ hantro_rfc_get_luma_buf_addr(ctx, dst->base.vb.vb2_buf.index);
+ compress_chroma_addr =
+ hantro_rfc_get_chroma_buf_addr(ctx, dst->base.vb.vb2_buf.index);
+ }
hantro_write_addr(vpu, G2_REF_LUMA_ADDR(i), luma_addr);
hantro_write_addr(vpu, G2_REF_CHROMA_ADDR(i), chroma_addr);
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 56c79e339030..1e96d0fce72a 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -129,7 +129,7 @@ static void config_output(struct hantro_ctx *ctx,
struct hantro_decoded_buffer *dst,
const struct v4l2_ctrl_vp9_frame *dec_params)
{
- dma_addr_t luma_addr, chroma_addr, mv_addr;
+ dma_addr_t luma_addr, chroma_addr;
hantro_reg_write(ctx->dev, &g2_out_dis, 0);
if (!ctx->dev->variant->legacy_regs)
@@ -142,9 +142,8 @@ static void config_output(struct hantro_ctx *ctx,
hantro_write_addr(ctx->dev, G2_OUT_CHROMA_ADDR, chroma_addr);
dst->vp9.chroma_offset = hantro_g2_chroma_offset(ctx);
- mv_addr = luma_addr + hantro_g2_motion_vectors_offset(ctx);
- hantro_write_addr(ctx->dev, G2_OUT_MV_ADDR, mv_addr);
- dst->vp9.mv_offset = hantro_g2_motion_vectors_offset(ctx);
+ dst->vp9.mv_addr = hantro_mv_get_buf_addr(ctx, dst->base.vb.vb2_buf.index);
+ hantro_write_addr(ctx->dev, G2_OUT_MV_ADDR, dst->vp9.mv_addr);
}
struct hantro_vp9_ref_reg {
@@ -215,15 +214,12 @@ static void config_ref_registers(struct hantro_ctx *ctx,
.c_base = G2_REF_CHROMA_ADDR(5),
},
};
- dma_addr_t mv_addr;
config_ref(ctx, dst, &ref_regs[0], dec_params, dec_params->last_frame_ts);
config_ref(ctx, dst, &ref_regs[1], dec_params, dec_params->golden_frame_ts);
config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params->alt_frame_ts);
- mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
- mv_ref->vp9.mv_offset;
- hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
+ hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_ref->vp9.mv_addr);
hantro_reg_write(ctx->dev, &vp9_last_sign_bias,
dec_params->ref_frame_sign_bias & V4L2_VP9_SIGN_BIAS_LAST ? 1 : 0);
diff --git a/drivers/media/platform/verisilicon/hantro_hevc.c b/drivers/media/platform/verisilicon/hantro_hevc.c
index 83cd12b0ddd6..272ce336b1c6 100644
--- a/drivers/media/platform/verisilicon/hantro_hevc.c
+++ b/drivers/media/platform/verisilicon/hantro_hevc.c
@@ -54,7 +54,24 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx,
return 0;
}
-int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr)
+int hantro_hevc_get_ref_buf_index(struct hantro_ctx *ctx, s32 poc)
+{
+ struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+ int i;
+
+ /* Find the reference buffer in already known ones */
+ for (i = 0; i < NUM_REF_PICTURES; i++) {
+ if (hevc_dec->ref_bufs_poc[i] == poc)
+ return hevc_dec->ref_vb2[i]->vb2_buf.index;
+ }
+
+ return 0;
+}
+
+int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx,
+ int poc,
+ dma_addr_t addr,
+ struct vb2_v4l2_buffer *vb2)
{
struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
int i;
@@ -65,6 +82,7 @@ int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr)
hevc_dec->ref_bufs_used |= 1 << i;
hevc_dec->ref_bufs_poc[i] = poc;
hevc_dec->ref_bufs[i].dma = addr;
+ hevc_dec->ref_vb2[i] = vb2;
return 0;
}
}
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index f0e4bca4b2b2..6a1ee9899b60 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -162,6 +162,7 @@ struct hantro_hevc_dec_hw_ctx {
struct hantro_aux_buf scaling_lists;
s32 ref_bufs_poc[NUM_REF_PICTURES];
u32 ref_bufs_used;
+ struct vb2_v4l2_buffer *ref_vb2[NUM_REF_PICTURES];
struct hantro_hevc_dec_ctrls ctrls;
unsigned int num_tile_cols_allocated;
bool use_compression;
@@ -457,7 +458,10 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
void hantro_hevc_ref_init(struct hantro_ctx *ctx);
dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, s32 poc);
-int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr);
+int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc,
+ dma_addr_t addr,
+ struct vb2_v4l2_buffer *vb2);
+int hantro_hevc_get_ref_buf_index(struct hantro_ctx *ctx, s32 poc);
int rockchip_vpu981_av1_dec_init(struct hantro_ctx *ctx);
void rockchip_vpu981_av1_dec_exit(struct hantro_ctx *ctx);
@@ -469,100 +473,7 @@ static inline unsigned short hantro_vp9_num_sbs(unsigned short dimension)
return (dimension + 63) / 64;
}
-static inline size_t
-hantro_vp9_mv_size(unsigned int width, unsigned int height)
-{
- int num_ctbs;
-
- /*
- * There can be up to (CTBs x 64) number of blocks,
- * and the motion vector for each block needs 16 bytes.
- */
- num_ctbs = hantro_vp9_num_sbs(width) * hantro_vp9_num_sbs(height);
- return (num_ctbs * 64) * 16;
-}
-
-static inline size_t
-hantro_h264_mv_size(unsigned int width, unsigned int height)
-{
- /*
- * A decoded 8-bit 4:2:0 NV12 frame may need memory for up to
- * 448 bytes per macroblock with additional 32 bytes on
- * multi-core variants.
- *
- * The H264 decoder needs extra space on the output buffers
- * to store motion vectors. This is needed for reference
- * frames and only if the format is non-post-processed NV12.
- *
- * Memory layout is as follow:
- *
- * +---------------------------+
- * | Y-plane 256 bytes x MBs |
- * +---------------------------+
- * | UV-plane 128 bytes x MBs |
- * +---------------------------+
- * | MV buffer 64 bytes x MBs |
- * +---------------------------+
- * | MC sync 32 bytes |
- * +---------------------------+
- */
- return 64 * MB_WIDTH(width) * MB_WIDTH(height) + 32;
-}
-
-static inline size_t
-hantro_hevc_mv_size(unsigned int width, unsigned int height)
-{
- /*
- * A CTB can be 64x64, 32x32 or 16x16.
- * Allocated memory for the "worse" case: 16x16
- */
- return width * height / 16;
-}
-
-static inline size_t
-hantro_hevc_luma_compressed_size(unsigned int width, unsigned int height)
-{
- u32 pic_width_in_cbsy =
- round_up((width + CBS_LUMA - 1) / CBS_LUMA, CBS_SIZE);
- u32 pic_height_in_cbsy = (height + CBS_LUMA - 1) / CBS_LUMA;
-
- return round_up(pic_width_in_cbsy * pic_height_in_cbsy, CBS_SIZE);
-}
-
-static inline size_t
-hantro_hevc_chroma_compressed_size(unsigned int width, unsigned int height)
-{
- u32 pic_width_in_cbsc =
- round_up((width + CBS_CHROMA_W - 1) / CBS_CHROMA_W, CBS_SIZE);
- u32 pic_height_in_cbsc = (height / 2 + CBS_CHROMA_H - 1) / CBS_CHROMA_H;
-
- return round_up(pic_width_in_cbsc * pic_height_in_cbsc, CBS_SIZE);
-}
-
-static inline size_t
-hantro_hevc_compressed_size(unsigned int width, unsigned int height)
-{
- return hantro_hevc_luma_compressed_size(width, height) +
- hantro_hevc_chroma_compressed_size(width, height);
-}
-
-static inline unsigned short hantro_av1_num_sbs(unsigned short dimension)
-{
- return DIV_ROUND_UP(dimension, 64);
-}
-
-static inline size_t
-hantro_av1_mv_size(unsigned int width, unsigned int height)
-{
- size_t num_sbs = hantro_av1_num_sbs(width) * hantro_av1_num_sbs(height);
-
- return ALIGN(num_sbs * 384, 16) * 2 + 512;
-}
-
size_t hantro_g2_chroma_offset(struct hantro_ctx *ctx);
-size_t hantro_g2_motion_vectors_offset(struct hantro_ctx *ctx);
-size_t hantro_g2_luma_compress_offset(struct hantro_ctx *ctx);
-size_t hantro_g2_chroma_compress_offset(struct hantro_ctx *ctx);
int hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx);
int rockchip_vpu2_mpeg2_dec_run(struct hantro_ctx *ctx);
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index e94d1ba5ef10..2409353c16e4 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -196,36 +196,11 @@ void hantro_postproc_free(struct hantro_ctx *ctx)
}
}
-static unsigned int hantro_postproc_buffer_size(struct hantro_ctx *ctx)
-{
- unsigned int buf_size;
-
- buf_size = ctx->ref_fmt.plane_fmt[0].sizeimage;
- if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
- buf_size += hantro_h264_mv_size(ctx->ref_fmt.width,
- ctx->ref_fmt.height);
- else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
- buf_size += hantro_vp9_mv_size(ctx->ref_fmt.width,
- ctx->ref_fmt.height);
- else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE) {
- buf_size += hantro_hevc_mv_size(ctx->ref_fmt.width,
- ctx->ref_fmt.height);
- if (ctx->hevc_dec.use_compression)
- buf_size += hantro_hevc_compressed_size(ctx->ref_fmt.width,
- ctx->ref_fmt.height);
- }
- else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_AV1_FRAME)
- buf_size += hantro_av1_mv_size(ctx->ref_fmt.width,
- ctx->ref_fmt.height);
-
- return buf_size;
-}
-
static int hantro_postproc_alloc(struct hantro_ctx *ctx, int index)
{
struct hantro_dev *vpu = ctx->dev;
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[index];
- unsigned int buf_size = hantro_postproc_buffer_size(ctx);
+ unsigned int buf_size = ctx->ref_fmt.plane_fmt[0].sizeimage;
if (!buf_size)
return -EINVAL;
@@ -267,7 +242,7 @@ dma_addr_t
hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index)
{
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[index];
- unsigned int buf_size = hantro_postproc_buffer_size(ctx);
+ unsigned int buf_size = ctx->ref_fmt.plane_fmt[0].sizeimage;
struct hantro_dev *vpu = ctx->dev;
int ret;
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index fcf3bd9bcda2..f8d4dd518368 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -36,6 +36,9 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
static int hantro_set_fmt_cap(struct hantro_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp);
+static void hantro_mv_free(struct hantro_ctx *ctx);
+static void hantro_rfc_free(struct hantro_ctx *ctx);
+
static const struct hantro_fmt *
hantro_get_formats(const struct hantro_ctx *ctx, unsigned int *num_fmts, bool need_postproc)
{
@@ -362,26 +365,6 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
/* Fill remaining fields */
v4l2_fill_pixfmt_mp(pix_mp, fmt->fourcc, pix_mp->width,
pix_mp->height);
- if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE &&
- !hantro_needs_postproc(ctx, fmt))
- pix_mp->plane_fmt[0].sizeimage +=
- hantro_h264_mv_size(pix_mp->width,
- pix_mp->height);
- else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME &&
- !hantro_needs_postproc(ctx, fmt))
- pix_mp->plane_fmt[0].sizeimage +=
- hantro_vp9_mv_size(pix_mp->width,
- pix_mp->height);
- else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE &&
- !hantro_needs_postproc(ctx, fmt))
- pix_mp->plane_fmt[0].sizeimage +=
- hantro_hevc_mv_size(pix_mp->width,
- pix_mp->height);
- else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_AV1_FRAME &&
- !hantro_needs_postproc(ctx, fmt))
- pix_mp->plane_fmt[0].sizeimage +=
- hantro_av1_mv_size(pix_mp->width,
- pix_mp->height);
} else if (!pix_mp->plane_fmt[0].sizeimage) {
/*
* For coded formats the application can specify
@@ -984,6 +967,9 @@ static void hantro_stop_streaming(struct vb2_queue *q)
ctx->codec_ops->exit(ctx);
}
+ hantro_mv_free(ctx);
+ hantro_rfc_free(ctx);
+
/*
* The mem2mem framework calls v4l2_m2m_cancel_job before
* .stop_streaming, so there isn't any job running and
@@ -1025,3 +1011,291 @@ const struct vb2_ops hantro_queue_ops = {
.start_streaming = hantro_start_streaming,
.stop_streaming = hantro_stop_streaming,
};
+
+static inline size_t
+hantro_vp9_mv_size(unsigned int width, unsigned int height)
+{
+ int num_ctbs;
+
+ /*
+ * There can be up to (CTBs x 64) number of blocks,
+ * and the motion vector for each block needs 16 bytes.
+ */
+ num_ctbs = hantro_vp9_num_sbs(width) * hantro_vp9_num_sbs(height);
+ return (num_ctbs * 64) * 16;
+}
+
+static inline size_t
+hantro_h264_mv_size(unsigned int width, unsigned int height)
+{
+ /*
+ * A decoded 8-bit 4:2:0 NV12 frame may need memory for up to
+ * 448 bytes per macroblock with additional 32 bytes on
+ * multi-core variants.
+ *
+ * The H264 decoder needs extra space on the output buffers
+ * to store motion vectors. This is needed for reference
+ * frames and only if the format is non-post-processed NV12.
+ *
+ * Memory layout is as follow:
+ *
+ * +---------------------------+
+ * | Y-plane 256 bytes x MBs |
+ * +---------------------------+
+ * | UV-plane 128 bytes x MBs |
+ * +---------------------------+
+ * | MV buffer 64 bytes x MBs |
+ * +---------------------------+
+ * | MC sync 32 bytes |
+ * +---------------------------+
+ */
+ return 64 * MB_WIDTH(width) * MB_WIDTH(height) + 32;
+}
+
+static inline size_t
+hantro_hevc_mv_size(unsigned int width, unsigned int height, int depth)
+{
+ /*
+ * A CTB can be 64x64, 32x32 or 16x16.
+ * Allocated memory for the "worse" case: 16x16
+ */
+ return DIV_ROUND_UP(width * height * depth / 8, 128);
+}
+
+static inline unsigned short hantro_av1_num_sbs(unsigned short dimension)
+{
+ return DIV_ROUND_UP(dimension, 64);
+}
+
+static inline size_t
+hantro_av1_mv_size(unsigned int width, unsigned int height)
+{
+ size_t num_sbs = hantro_av1_num_sbs(width) * hantro_av1_num_sbs(height);
+
+ return ALIGN(num_sbs * 384, 16) * 2 + 512;
+}
+
+static void hantro_mv_free(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ int i;
+
+ for (i = 0; i < MAX_MV_BUFFERS; i++) {
+ struct hantro_aux_buf *mv = &ctx->dec_mv[i];
+
+ if (!mv->cpu)
+ continue;
+
+ dma_free_attrs(vpu->dev, mv->size, mv->cpu,
+ mv->dma, mv->attrs);
+ mv->cpu = NULL;
+ }
+}
+
+static unsigned int hantro_mv_buffer_size(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ int fourcc = ctx->vpu_src_fmt->fourcc;
+ int width = ctx->ref_fmt.width;
+ int height = ctx->ref_fmt.height;
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_H264_SLICE:
+ return hantro_h264_mv_size(width, height);
+ case V4L2_PIX_FMT_VP9_FRAME:
+ return hantro_vp9_mv_size(width, height);
+ case V4L2_PIX_FMT_HEVC_SLICE:
+ return hantro_hevc_mv_size(width, height, ctx->bit_depth);
+ case V4L2_PIX_FMT_AV1_FRAME:
+ return hantro_av1_mv_size(width, height);
+ }
+
+ /* Should not happen */
+ dev_warn(vpu->dev, "Invalid motion vectors size\n");
+ return 0;
+}
+
+static int hantro_mv_buffer_alloc(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_aux_buf *mv = &ctx->dec_mv[index];
+ unsigned int buf_size = hantro_mv_buffer_size(ctx);
+
+ if (!buf_size)
+ return -EINVAL;
+
+ /*
+ * Motion vectors buffers are only read and write by the
+ * hardware so no mapping is needed.
+ */
+ mv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+ mv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &mv->dma,
+ GFP_KERNEL, mv->attrs);
+ if (!mv->cpu)
+ return -ENOMEM;
+ mv->size = buf_size;
+
+ return 0;
+}
+
+dma_addr_t
+hantro_mv_get_buf_addr(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_aux_buf *mv = &ctx->dec_mv[index];
+ unsigned int buf_size = hantro_mv_buffer_size(ctx);
+ struct hantro_dev *vpu = ctx->dev;
+ int ret;
+
+ if (mv->size < buf_size && mv->cpu) {
+ /* buffer is too small, release it */
+ dma_free_attrs(vpu->dev, mv->size, mv->cpu,
+ mv->dma, mv->attrs);
+ mv->cpu = NULL;
+ }
+
+ if (!mv->cpu) {
+ /* buffer not already allocated, try getting a new one */
+ ret = hantro_mv_buffer_alloc(ctx, index);
+ if (ret)
+ return 0;
+ }
+
+ if (!mv->cpu)
+ return 0;
+
+ return mv->dma;
+}
+
+static inline size_t
+hantro_hevc_luma_compressed_size(unsigned int width, unsigned int height)
+{
+ u32 pic_width_in_cbsy =
+ round_up((width + CBS_LUMA - 1) / CBS_LUMA, CBS_SIZE);
+ u32 pic_height_in_cbsy = (height + CBS_LUMA - 1) / CBS_LUMA;
+
+ return round_up(pic_width_in_cbsy * pic_height_in_cbsy, CBS_SIZE);
+}
+
+static inline size_t
+hantro_hevc_chroma_compressed_size(unsigned int width, unsigned int height)
+{
+ u32 pic_width_in_cbsc =
+ round_up((width + CBS_CHROMA_W - 1) / CBS_CHROMA_W, CBS_SIZE);
+ u32 pic_height_in_cbsc = (height / 2 + CBS_CHROMA_H - 1) / CBS_CHROMA_H;
+
+ return round_up(pic_width_in_cbsc * pic_height_in_cbsc, CBS_SIZE);
+}
+
+static inline size_t
+hantro_hevc_compressed_size(unsigned int width, unsigned int height)
+{
+ return hantro_hevc_luma_compressed_size(width, height) +
+ hantro_hevc_chroma_compressed_size(width, height);
+}
+
+static void hantro_rfc_free(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ int i;
+
+ for (i = 0; i < MAX_MV_BUFFERS; i++) {
+ struct hantro_aux_buf *rfc = &ctx->dec_rfc[i];
+
+ if (!rfc->cpu)
+ continue;
+
+ dma_free_attrs(vpu->dev, rfc->size, rfc->cpu,
+ rfc->dma, rfc->attrs);
+ rfc->cpu = NULL;
+ }
+}
+
+static unsigned int hantro_rfc_buffer_size(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ int fourcc = ctx->vpu_src_fmt->fourcc;
+ int width = ctx->ref_fmt.width;
+ int height = ctx->ref_fmt.height;
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_HEVC_SLICE:
+ return hantro_hevc_compressed_size(width, height);
+ }
+
+ /* Should not happen */
+ dev_warn(vpu->dev, "Invalid rfc size\n");
+ return 0;
+}
+
+static int hantro_rfc_buffer_alloc(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_aux_buf *rfc = &ctx->dec_rfc[index];
+ unsigned int buf_size = hantro_rfc_buffer_size(ctx);
+
+ if (!buf_size)
+ return -EINVAL;
+
+ /*
+ * RFC buffers are only read and write by the
+ * hardware so no mapping is needed.
+ */
+ rfc->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+ rfc->cpu = dma_alloc_attrs(vpu->dev, buf_size, &rfc->dma,
+ GFP_KERNEL, rfc->attrs);
+ if (!rfc->cpu)
+ return -ENOMEM;
+ rfc->size = buf_size;
+
+ return 0;
+}
+
+dma_addr_t
+hantro_rfc_get_luma_buf_addr(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_aux_buf *rfc = &ctx->dec_rfc[index];
+ unsigned int buf_size = hantro_rfc_buffer_size(ctx);
+ struct hantro_dev *vpu = ctx->dev;
+ int ret;
+
+ if (rfc->size < buf_size && rfc->cpu) {
+ /* buffer is too small, release it */
+ dma_free_attrs(vpu->dev, rfc->size, rfc->cpu,
+ rfc->dma, rfc->attrs);
+ rfc->cpu = NULL;
+ }
+
+ if (!rfc->cpu) {
+ /* buffer not already allocated, try getting a new one */
+ ret = hantro_rfc_buffer_alloc(ctx, index);
+ if (ret)
+ return 0;
+ }
+
+ if (!rfc->cpu)
+ return 0;
+
+ return rfc->dma;
+}
+
+dma_addr_t
+hantro_rfc_get_chroma_buf_addr(struct hantro_ctx *ctx, int index)
+{
+ dma_addr_t luma_addr = hantro_rfc_get_luma_buf_addr(ctx, index);
+ struct hantro_dev *vpu = ctx->dev;
+ int fourcc = ctx->vpu_src_fmt->fourcc;
+ int width = ctx->ref_fmt.width;
+ int height = ctx->ref_fmt.height;
+
+ if (!luma_addr)
+ return -EINVAL;
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_HEVC_SLICE:
+ return luma_addr + hantro_hevc_luma_compressed_size(width, height);
+ }
+
+ /* Should not happen */
+ dev_warn(vpu->dev, "Invalid rfc chroma address\n");
+ return 0;
+}
diff --git a/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c b/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
index c1ada14df4c3..21da8ddfc4b3 100644
--- a/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
+++ b/drivers/media/platform/verisilicon/rockchip_vpu981_hw_av1_dec.c
@@ -62,7 +62,7 @@ rockchip_vpu981_av1_dec_set_ref(struct hantro_ctx *ctx, int ref, int idx,
const struct v4l2_ctrl_av1_frame *frame = ctrls->frame;
struct hantro_dev *vpu = ctx->dev;
struct hantro_decoded_buffer *dst;
- dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
+ dma_addr_t luma_addr, chroma_addr = 0;
int cur_width = frame->frame_width_minus_1 + 1;
int cur_height = frame->frame_height_minus_1 + 1;
int scale_width =
@@ -120,11 +120,10 @@ rockchip_vpu981_av1_dec_set_ref(struct hantro_ctx *ctx, int ref, int idx,
dst = vb2_to_hantro_decoded_buf(&av1_dec->frame_refs[idx].vb2_ref->vb2_buf);
luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
chroma_addr = luma_addr + dst->av1.chroma_offset;
- mv_addr = luma_addr + dst->av1.mv_offset;
hantro_write_addr(vpu, AV1_REFERENCE_Y(ref), luma_addr);
hantro_write_addr(vpu, AV1_REFERENCE_CB(ref), chroma_addr);
- hantro_write_addr(vpu, AV1_REFERENCE_MV(ref), mv_addr);
+ hantro_write_addr(vpu, AV1_REFERENCE_MV(ref), dst->av1.mv_addr);
return (scale_width != (1 << AV1_REF_SCALE_SHIFT)) ||
(scale_height != (1 << AV1_REF_SCALE_SHIFT));
@@ -180,11 +179,10 @@ static void rockchip_vpu981_av1_dec_set_segmentation(struct hantro_ctx *ctx)
if (idx >= 0) {
dma_addr_t luma_addr, mv_addr = 0;
struct hantro_decoded_buffer *seg;
- size_t mv_offset = hantro_av1_chroma_size(ctx);
seg = vb2_to_hantro_decoded_buf(&av1_dec->frame_refs[idx].vb2_ref->vb2_buf);
luma_addr = hantro_get_dec_buf_addr(ctx, &seg->base.vb.vb2_buf);
- mv_addr = luma_addr + mv_offset;
+ mv_addr = hantro_mv_get_buf_addr(ctx, seg->base.vb.vb2_buf.index);
hantro_write_addr(vpu, AV1_SEGMENTATION, mv_addr);
hantro_reg_write(vpu, &av1_use_temporal3_mvs, 1);
@@ -1350,22 +1348,20 @@ rockchip_vpu981_av1_dec_set_output_buffer(struct hantro_ctx *ctx)
struct hantro_dev *vpu = ctx->dev;
struct hantro_decoded_buffer *dst;
struct vb2_v4l2_buffer *vb2_dst;
- dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
+ dma_addr_t luma_addr, chroma_addr = 0;
size_t cr_offset = hantro_av1_luma_size(ctx);
- size_t mv_offset = hantro_av1_chroma_size(ctx);
vb2_dst = av1_dec->frame_refs[av1_dec->current_frame_index].vb2_ref;
dst = vb2_to_hantro_decoded_buf(&vb2_dst->vb2_buf);
luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
chroma_addr = luma_addr + cr_offset;
- mv_addr = luma_addr + mv_offset;
dst->av1.chroma_offset = cr_offset;
- dst->av1.mv_offset = mv_offset;
+ dst->av1.mv_addr = hantro_mv_get_buf_addr(ctx, dst->base.vb.vb2_buf.index);
hantro_write_addr(vpu, AV1_TILE_OUT_LU, luma_addr);
hantro_write_addr(vpu, AV1_TILE_OUT_CH, chroma_addr);
- hantro_write_addr(vpu, AV1_TILE_OUT_MV, mv_addr);
+ hantro_write_addr(vpu, AV1_TILE_OUT_MV, dst->av1.mv_addr);
}
int rockchip_vpu981_av1_dec_run(struct hantro_ctx *ctx)
--
2.43.0
^ permalink raw reply related
* Re: [PATCH v2] KVM: arm64: Prevent the host from using an smc with imm16 != 0
From: Sebastian Ene @ 2026-03-25 13:16 UTC (permalink / raw)
To: Fuad Tabba
Cc: Vincent Donnefort, kvmarm, linux-arm-kernel, linux-kernel,
android-kvm, catalin.marinas, joey.gouly, mark.rutland, maz,
oupton, suzuki.poulose, will, yuzenghui
In-Reply-To: <CA+EHjTzeRiUvhDwAExpscWJfPNoRujeRAJ5dEDV4SMpOJK1K6A@mail.gmail.com>
On Wed, Mar 25, 2026 at 12:16:17PM +0000, Fuad Tabba wrote:
> On Wed, 25 Mar 2026 at 11:41, Sebastian Ene <sebastianene@google.com> wrote:
> >
> > On Wed, Mar 25, 2026 at 11:35:18AM +0000, Vincent Donnefort wrote:
> > > On Wed, Mar 25, 2026 at 11:31:38AM +0000, Sebastian Ene wrote:
> > > > The ARM Service Calling Convention (SMCCC) specifies that the function
> > > > identifier and parameters should be passed in registers, leaving the
> > > > 16-bit immediate field of the SMC instruction un-handled.
> > > > Currently, our pKVM handler ignores the immediate value, which could lead
> > > > to non-compliant software relying on implementation-defined behavior.
> > > > Enforce the host kernel running under pKVM to use an immediate value
> > > > of 0 by decoding the ISS from the ESR_EL2 and return a not supported
> > > > error code back to the caller.
> > > >
> > > > Signed-off-by: Sebastian Ene <sebastianene@google.com>
> > > > ---
> > > > v1 -> v2:
> > > >
> > > > - Dropped injecting an UNDEF and return an error instead
> > > > (SMCCC_RET_NOT_SUPPORTED)
> > > > - Used the mask ESR_ELx_xVC_IMM_MASK instead of masking with U16_MAX
> > > > - Updated the title of the commit message from:
> > > > "[PATCH] KVM: arm64: Inject UNDEF when host is executing an
> > > > smc with imm16 != 0
> > >
> > > > ---
> > > > arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 ++++++
> > > > 1 file changed, 6 insertions(+)
> > > >
> > > > diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > > > index e7790097db93..4ffe30fd8707 100644
> > > > --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > > > +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > > > @@ -762,6 +762,12 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
> > > > handle_host_hcall(host_ctxt);
> > > > break;
> > > > case ESR_ELx_EC_SMC64:
> > > > + if (ESR_ELx_xVC_IMM_MASK & esr) {
> > > > + cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED;
> > > > + kvm_skip_host_instr();
> > > > + break;
> > > > + }
> > > > +
> > >
> > > I wonder if it isn't better to move that into handle_host_smc() as this is part
> > > of how we handle the SMC after all? (and it calls that kvm_skip_host_instr()
> > > already)
> > >
> >
> > I was thinking of doing that as well but I prefer this since we don't
> > have to look again at the esr in the callee.
>
> This is a non-trivial amount of code to have in a switch. Can you
> please put it in a helper?
Hey Fuad,
I will move it as part of the handle_host_smc(..) as suggested by other
comment.
>
> Thanks,
> /fuad
>
Thanks,
Sebastian
> >
> > > > handle_host_smc(host_ctxt);
> > > > break;
> > > > case ESR_ELx_EC_IABT_LOW:
> > > > --
> > > > 2.53.0.1018.g2bb0e51243-goog
> > > >
^ permalink raw reply
* Re: [PATCH v2] KVM: arm64: Prevent the host from using an smc with imm16 != 0
From: Sebastian Ene @ 2026-03-25 13:15 UTC (permalink / raw)
To: Marc Zyngier
Cc: Vincent Donnefort, kvmarm, linux-arm-kernel, linux-kernel,
android-kvm, catalin.marinas, joey.gouly, mark.rutland, oupton,
suzuki.poulose, tabba, will, yuzenghui
In-Reply-To: <86ldfg3ze2.wl-maz@kernel.org>
On Wed, Mar 25, 2026 at 11:46:29AM +0000, Marc Zyngier wrote:
> On Wed, 25 Mar 2026 11:35:18 +0000,
> Vincent Donnefort <vdonnefort@google.com> wrote:
> >
> > On Wed, Mar 25, 2026 at 11:31:38AM +0000, Sebastian Ene wrote:
> > > The ARM Service Calling Convention (SMCCC) specifies that the function
> > > identifier and parameters should be passed in registers, leaving the
> > > 16-bit immediate field of the SMC instruction un-handled.
> > > Currently, our pKVM handler ignores the immediate value, which could lead
> > > to non-compliant software relying on implementation-defined behavior.
> > > Enforce the host kernel running under pKVM to use an immediate value
> > > of 0 by decoding the ISS from the ESR_EL2 and return a not supported
> > > error code back to the caller.
> > >
> > > Signed-off-by: Sebastian Ene <sebastianene@google.com>
> > > ---
> > > v1 -> v2:
> > >
> > > - Dropped injecting an UNDEF and return an error instead
> > > (SMCCC_RET_NOT_SUPPORTED)
> > > - Used the mask ESR_ELx_xVC_IMM_MASK instead of masking with U16_MAX
> > > - Updated the title of the commit message from:
> > > "[PATCH] KVM: arm64: Inject UNDEF when host is executing an
> > > smc with imm16 != 0
> >
> > > ---
> > > arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 ++++++
> > > 1 file changed, 6 insertions(+)
> > >
> > > diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > > index e7790097db93..4ffe30fd8707 100644
> > > --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > > +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
> > > @@ -762,6 +762,12 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
> > > handle_host_hcall(host_ctxt);
> > > break;
> > > case ESR_ELx_EC_SMC64:
> > > + if (ESR_ELx_xVC_IMM_MASK & esr) {
> > > + cpu_reg(host_ctxt, 0) = SMCCC_RET_NOT_SUPPORTED;
> > > + kvm_skip_host_instr();
> > > + break;
> > > + }
> > > +
> >
> > I wonder if it isn't better to move that into handle_host_smc() as this is part
> > of how we handle the SMC after all? (and it calls that kvm_skip_host_instr()
> > already)
>
> Yes, that'd be vastly better.
>
good, I will update the patch to do this.
> It also begs the question: if you don't want to handle SMCs with a
> non-zero immediate, why is it OK to do it for HVCs?
I talked a bit with Will about this before writing it. My understanding is that we
don't have to do it for HVCs because the interface with the hypervisor
is controlled by us whereas with non-standard SMCs we need at least to
tell the host that we are not handling non-zero imm16.
>
> Thanks,
>
> M.
>
> --
> Without deviation from the norm, progress is not possible.
Thanks,
Sebastian
^ permalink raw reply
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Nicolas Frattaroli @ 2026-03-25 13:11 UTC (permalink / raw)
To: Dave Stevenson
Cc: Harry Wentland, Leo Li, Rodrigo Siqueira, Alex Deucher,
Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Sandy Huang, Heiko Stübner,
Andy Yan, Jani Nikula, Rodrigo Vivi, Joonas Lahtinen,
Tvrtko Ursulin, Dmitry Baryshkov, Sascha Hauer, Rob Herring,
Jonathan Corbet, Shuah Khan, kernel, amd-gfx, dri-devel,
linux-kernel, linux-arm-kernel, linux-rockchip, intel-gfx,
intel-xe, linux-doc, Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <CAPY8ntB9f_=f5kru=8w9BpTuqQR+93maGpT61EKU28Uay2vq8Q@mail.gmail.com>
On Wednesday, 25 March 2026 13:49:19 Central European Standard Time Dave Stevenson wrote:
> On Tue, 24 Mar 2026 at 16:02, Nicolas Frattaroli
> <nicolas.frattaroli@collabora.com> wrote:
> >
> > Add a new general DRM property named "color format" which can be used by
> > userspace to request the display driver to output a particular color
> > format.
> >
> > Possible options are:
> > - auto (setup by default, driver internally picks the color format)
> > - rgb
> > - ycbcr444
> > - ycbcr422
> > - ycbcr420
> >
> > Drivers should advertise from this list which formats they support.
> > Together with this list and EDID data from the sink we should be able
> > to relay a list of usable color formats to users to pick from.
> >
> > Co-developed-by: Werner Sembach <wse@tuxedocomputers.com>
> > Signed-off-by: Werner Sembach <wse@tuxedocomputers.com>
> > Co-developed-by: Andri Yngvason <andri@yngvason.is>
> > Signed-off-by: Andri Yngvason <andri@yngvason.is>
> > Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
> > Reviewed-by: Maxime Ripard <mripard@kernel.org>
> > Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
> > ---
> > drivers/gpu/drm/drm_atomic_helper.c | 5 ++
> > drivers/gpu/drm/drm_atomic_uapi.c | 11 ++++
> > drivers/gpu/drm/drm_connector.c | 108 ++++++++++++++++++++++++++++++++++++
> > include/drm/drm_connector.h | 104 ++++++++++++++++++++++++++++++++++
> > 4 files changed, 228 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
> > index 26953ed6b53e..b7753454b777 100644
> > --- a/drivers/gpu/drm/drm_atomic_helper.c
> > +++ b/drivers/gpu/drm/drm_atomic_helper.c
> > @@ -737,6 +737,11 @@ drm_atomic_helper_check_modeset(struct drm_device *dev,
> > if (old_connector_state->max_requested_bpc !=
> > new_connector_state->max_requested_bpc)
> > new_crtc_state->connectors_changed = true;
> > +
> > + if (old_connector_state->color_format !=
> > + new_connector_state->color_format)
> > + new_crtc_state->connectors_changed = true;
> > +
> > }
> >
> > if (funcs->atomic_check)
> > diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
> > index 5bd5bf6661df..dee510c85e59 100644
> > --- a/drivers/gpu/drm/drm_atomic_uapi.c
> > +++ b/drivers/gpu/drm/drm_atomic_uapi.c
> > @@ -935,6 +935,15 @@ static int drm_atomic_connector_set_property(struct drm_connector *connector,
> > state->privacy_screen_sw_state = val;
> > } else if (property == connector->broadcast_rgb_property) {
> > state->hdmi.broadcast_rgb = val;
> > + } else if (property == connector->color_format_property) {
> > + if (val > INT_MAX || !drm_connector_color_format_valid(val)) {
> > + drm_dbg_atomic(connector->dev,
> > + "[CONNECTOR:%d:%s] unknown color format %llu\n",
> > + connector->base.id, connector->name, val);
> > + return -EINVAL;
> > + }
> > +
> > + state->color_format = val;
> > } else if (connector->funcs->atomic_set_property) {
> > return connector->funcs->atomic_set_property(connector,
> > state, property, val);
> > @@ -1020,6 +1029,8 @@ drm_atomic_connector_get_property(struct drm_connector *connector,
> > *val = state->privacy_screen_sw_state;
> > } else if (property == connector->broadcast_rgb_property) {
> > *val = state->hdmi.broadcast_rgb;
> > + } else if (property == connector->color_format_property) {
> > + *val = state->color_format;
> > } else if (connector->funcs->atomic_get_property) {
> > return connector->funcs->atomic_get_property(connector,
> > state, property, val);
> > diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
> > index 47dc53c4a738..e848374dee0b 100644
> > --- a/drivers/gpu/drm/drm_connector.c
> > +++ b/drivers/gpu/drm/drm_connector.c
> > @@ -1388,6 +1388,18 @@ static const u32 hdmi_colorspaces =
> > BIT(DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65) |
> > BIT(DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER);
> >
> > +static const u32 hdmi_colorformats =
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_RGB444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR422) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR420);
> > +
> > +static const u32 dp_colorformats =
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_RGB444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR444) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR422) |
> > + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR420);
> > +
> > /*
> > * As per DP 1.4a spec, 2.2.5.7.5 VSC SDP Payload for Pixel Encoding/Colorimetry
> > * Format Table 2-120
> > @@ -2940,6 +2952,102 @@ int drm_connector_attach_colorspace_property(struct drm_connector *connector)
> > }
> > EXPORT_SYMBOL(drm_connector_attach_colorspace_property);
> >
> > +/**
> > + * drm_connector_attach_color_format_property - create and attach color format property
> > + * @connector: connector to create the color format property on
> > + * @supported_color_formats: bitmask of bit-shifted &enum drm_output_color_format
> > + * values the connector supports
> > + *
> > + * Called by a driver to create a color format property. The property is
> > + * attached to the connector automatically on success.
> > + *
> > + * @supported_color_formats should only include color formats the connector
> > + * type can actually support.
> > + *
> > + * Returns:
> > + * 0 on success, negative errno on error
> > + */
> > +int drm_connector_attach_color_format_property(struct drm_connector *connector,
> > + unsigned long supported_color_formats)
> > +{
> > + struct drm_device *dev = connector->dev;
> > + struct drm_prop_enum_list enum_list[DRM_CONNECTOR_COLOR_FORMAT_COUNT];
> > + unsigned int i = 0;
> > + unsigned long fmt;
> > +
> > + if (connector->color_format_property)
> > + return 0;
> > +
> > + if (!supported_color_formats) {
> > + drm_err(dev, "No supported color formats provided on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > +
> > + if (supported_color_formats & ~GENMASK(DRM_OUTPUT_COLOR_FORMAT_COUNT - 1, 0)) {
> > + drm_err(dev, "Unknown color formats provided on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > +
> > + switch (connector->connector_type) {
> > + case DRM_MODE_CONNECTOR_HDMIA:
> > + case DRM_MODE_CONNECTOR_HDMIB:
> > + if (supported_color_formats & ~hdmi_colorformats) {
> > + drm_err(dev, "Color formats not allowed for HDMI on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > + break;
> > + case DRM_MODE_CONNECTOR_DisplayPort:
> > + case DRM_MODE_CONNECTOR_eDP:
> > + if (supported_color_formats & ~dp_colorformats) {
> > + drm_err(dev, "Color formats not allowed for DP on [CONNECTOR:%d:%s]\n",
> > + connector->base.id, connector->name);
> > + return -EINVAL;
> > + }
> > + break;
> > + }
> > +
> > + enum_list[0].name = "AUTO";
> > + enum_list[0].type = DRM_CONNECTOR_COLOR_FORMAT_AUTO;
> > +
> > + for_each_set_bit(fmt, &supported_color_formats, DRM_OUTPUT_COLOR_FORMAT_COUNT) {
> > + switch (fmt) {
> > + case DRM_OUTPUT_COLOR_FORMAT_RGB444:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_RGB444;
> > + break;
> > + case DRM_OUTPUT_COLOR_FORMAT_YCBCR444:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR444;
> > + break;
> > + case DRM_OUTPUT_COLOR_FORMAT_YCBCR422:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR422;
> > + break;
> > + case DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR420;
> > + break;
> > + default:
> > + drm_warn(dev, "Unknown supported format %ld on [CONNECTOR:%d:%s]\n",
> > + fmt, connector->base.id, connector->name);
> > + continue;
> > + }
> > + enum_list[i].name = drm_hdmi_connector_get_output_format_name(fmt);
> > + }
> > +
> > + connector->color_format_property =
> > + drm_property_create_enum(dev, DRM_MODE_PROP_ENUM, "color format",
> > + enum_list, i + 1);
> > +
> > + if (!connector->color_format_property)
> > + return -ENOMEM;
> > +
> > + drm_object_attach_property(&connector->base, connector->color_format_property,
> > + DRM_CONNECTOR_COLOR_FORMAT_AUTO);
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL(drm_connector_attach_color_format_property);
> > +
> > /**
> > * drm_connector_atomic_hdr_metadata_equal - checks if the hdr metadata changed
> > * @old_state: old connector state to compare
> > diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
> > index af8b92d2d5b7..bd549f912b76 100644
> > --- a/include/drm/drm_connector.h
> > +++ b/include/drm/drm_connector.h
> > @@ -571,14 +571,102 @@ enum drm_colorspace {
> > * YCbCr 4:2:2 output format (ie. with horizontal subsampling)
> > * @DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > * YCbCr 4:2:0 output format (ie. with horizontal and vertical subsampling)
> > + * @DRM_OUTPUT_COLOR_FORMAT_COUNT:
> > + * Number of valid output color format values in this enum
> > */
> > enum drm_output_color_format {
> > DRM_OUTPUT_COLOR_FORMAT_RGB444 = 0,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR444,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR422,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR420,
> > + DRM_OUTPUT_COLOR_FORMAT_COUNT,
> > };
> >
> > +/**
> > + * enum drm_connector_color_format - Connector Color Format Request
> > + *
> > + * This enum, unlike &enum drm_output_color_format, is used to specify requests
> > + * for a specific color format on a connector through the DRM "color format"
> > + * property. The difference is that it has an "AUTO" value to specify that
> > + * no specific choice has been made.
> > + */
> > +enum drm_connector_color_format {
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> > + * helpers should pick a suitable color format. All implementations of a
> > + * specific display protocol must behave the same way with "AUTO", but
> > + * different display protocols do not necessarily have the same "AUTO"
> > + * semantics.
> > + *
> > + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> > + * bandwidth required for full-scale RGB is not available, or the mode
> > + * is YCbCr 4:2:0-only, as long as the mode and output both support
> > + * YCbCr 4:2:0.
>
> Is there a reason you propose dropping back to YCbCr 4:2:0 without
> trying YCbCr 4:2:2 first? Minimising the subsampling is surely
> beneficial, and vc4 for one can do 4:2:2 but not 4:2:0.
I worked under the assumption that 4:2:0 was more common than 4:2:2 based
on the existing HDMI state helper code that also doesn't try 4:2:2 first.
But this is a good point, it should try 4:2:2 as well (at least on
implementations where it's supported, e.g. i915 does not support 4:2:2 at
all from what I see.)
Kind regards,
Nicolas Frattaroli
>
> Dave
>
> > + *
> > + * For display protocols other than HDMI, the recursive bridge chain
> > + * format selection picks the first chain of bridge formats that works,
> > + * as has already been the case before the introduction of the "color
> > + * format" property. Non-HDMI bridges should therefore either sort their
> > + * bus output formats by preference, or agree on a unified auto format
> > + * selection logic that's implemented in a common state helper (like
> > + * how HDMI does it).
> > + */
> > + DRM_CONNECTOR_COLOR_FORMAT_AUTO = 0,
> > +
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_RGB444: RGB output format
> > + */
> > + DRM_CONNECTOR_COLOR_FORMAT_RGB444,
> > +
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR444: YCbCr 4:4:4 output format (ie.
> > + * not subsampled)
> > + */
> > + DRM_CONNECTOR_COLOR_FORMAT_YCBCR444,
> > +
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR422: YCbCr 4:2:2 output format (ie.
> > + * with horizontal subsampling)
> > + */
> > + DRM_CONNECTOR_COLOR_FORMAT_YCBCR422,
> > +
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR420: YCbCr 4:2:0 output format (ie.
> > + * with horizontal and vertical subsampling)
> > + */
> > + DRM_CONNECTOR_COLOR_FORMAT_YCBCR420,
> > +
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_COUNT: Number of valid connector color
> > + * format values in this enum
> > + */
> > + DRM_CONNECTOR_COLOR_FORMAT_COUNT,
> > +};
> > +
> > +/**
> > + * drm_connector_color_format_valid - Validate drm_connector_color_format value
> > + * @fmt: value to check against all values of &enum drm_connector_color_format
> > + *
> > + * Checks whether the passed in value of @fmt is one of the allowable values in
> > + * &enum drm_connector_color_format.
> > + *
> > + * Returns: %true if it's a valid value for the enum, %false otherwise.
> > + */
> > +static inline bool __pure
> > +drm_connector_color_format_valid(enum drm_connector_color_format fmt)
> > +{
> > + switch (fmt) {
> > + case DRM_CONNECTOR_COLOR_FORMAT_AUTO:
> > + case DRM_CONNECTOR_COLOR_FORMAT_RGB444:
> > + case DRM_CONNECTOR_COLOR_FORMAT_YCBCR444:
> > + case DRM_CONNECTOR_COLOR_FORMAT_YCBCR422:
> > + case DRM_CONNECTOR_COLOR_FORMAT_YCBCR420:
> > + return true;
> > + default:
> > + return false;
> > + }
> > +}
> > +
> > const char *
> > drm_hdmi_connector_get_output_format_name(enum drm_output_color_format fmt);
> >
> > @@ -1129,6 +1217,13 @@ struct drm_connector_state {
> > */
> > enum drm_colorspace colorspace;
> >
> > + /**
> > + * @color_format: State variable for Connector property to request
> > + * color format change on Sink. This is most commonly used to switch
> > + * between RGB to YUV and vice-versa.
> > + */
> > + enum drm_connector_color_format color_format;
> > +
> > /**
> > * @writeback_job: Writeback job for writeback connectors
> > *
> > @@ -2127,6 +2222,12 @@ struct drm_connector {
> > */
> > struct drm_property *colorspace_property;
> >
> > + /**
> > + * @color_format_property: Connector property to set the suitable
> > + * color format supported by the sink.
> > + */
> > + struct drm_property *color_format_property;
> > +
> > /**
> > * @path_blob_ptr:
> > *
> > @@ -2610,6 +2711,9 @@ bool drm_connector_has_possible_encoder(struct drm_connector *connector,
> > struct drm_encoder *encoder);
> > const char *drm_get_colorspace_name(enum drm_colorspace colorspace);
> >
> > +int drm_connector_attach_color_format_property(struct drm_connector *connector,
> > + unsigned long supported_color_formats);
> > +
> > /**
> > * drm_for_each_connector_iter - connector_list iterator macro
> > * @connector: &struct drm_connector pointer used as cursor
> >
> > --
> > 2.53.0
> >
>
^ permalink raw reply
* Re: [PATCH] arm64: dts: renesas: sparrow-hawk: Reserve first 128 MiB of DRAM
From: Geert Uytterhoeven @ 2026-03-25 13:08 UTC (permalink / raw)
To: Marek Vasut
Cc: linux-arm-kernel, stable, Conor Dooley, Geert Uytterhoeven,
Krzysztof Kozlowski, Magnus Damm, Rob Herring, devicetree,
linux-kernel, linux-renesas-soc
In-Reply-To: <20260324143342.17872-1-marek.vasut+renesas@mailbox.org>
Hi Marek,
On Tue, 24 Mar 2026 at 15:33, Marek Vasut
<marek.vasut+renesas@mailbox.org> wrote:
> Mark the first 128 MiB of DRAM as reserved. The first 128 MiB of DRAM
> may optionally be used by TFA and other firmware for its own purposes,
> and in such case, Linux must not use this memory.
>
> On this platform, U-Boot runs in EL3 and starts TFA BL31 and Linux from
> a single combined fitImage. U-Boot has full access to all memory in the
> 0x40000000..0xbfffffff range, as well memory in the memory banks in the
> 64-bit address ranges, and therefore U-Boot patches this full complete
> view of platform memory layout into the DT that is passed to the next
> stage.
>
> The next stage is TFA BL31 and then the Linux kernel. The TFA BL31 does
> not modify the DT passed from U-Boot to TFA BL31 and then to Linux with
> any new reserved-memory {} node to reserve memory areas used by the TFA
> BL31 to prevent the next stage from using those areas, which lets Linux
> to use all of the available DRAM as described in the DT that was passed
> in by U-Boot, including the areas that are newly utilized by TFA BL31.
>
> In case of high DRAM utilization, for example in case of four instances
> of "memtester 3900M" running in parallel, unless the memory used by TFA
> BL31 is properly reserved, Linux may use and corrupt the memory used by
> TFA BL31, which would often lead to system becoming unresponsive.
>
> Until TFA BL31 can properly fill its own reserved-memory node into the
> DT, and to assure older versions of TFA BL31 do not cause problems, add
> explicitly reserved-memory {} node which prevents Linux from using the
> first 128 MiB of DRAM.
>
> Note that TFA BL31 can be adjusted to use different memory areas, this
> newly added reserved-memory {} node follows longer-term practice on the
> R-Car SoCs where the first 128 MiB of DRAM is reserved for firmware use.
> In case user does modify TFA BL31 to use different memory ranges, they
> must either use a future version of TFA BL31 which properly patches a
> reserved-memory {} node into the DT, or they must adjust the address
> ranges of this reserved-memory {} node accordingly.
>
> Fixes: a719915e76f2 ("arm64: dts: renesas: r8a779g3: Add Retronix R-Car V4H Sparrow Hawk board support")
> Cc: stable@vger.kernel.org
> Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org>
Thanks for your patch!
> --- a/arch/arm64/boot/dts/renesas/r8a779g3-sparrow-hawk.dts
> +++ b/arch/arm64/boot/dts/renesas/r8a779g3-sparrow-hawk.dts
> @@ -118,6 +118,17 @@ memory@600000000 {
> reg = <0x6 0x00000000 0x1 0x00000000>;
> };
>
> + reserved-memory {
> + #address-cells = <2>;
> + #size-cells = <2>;
> + ranges;
> +
> + tfa@40000000 {
> + reg = <0x0 0x40000000 0x0 0x8000000>;
> + no-map;
> + };
> + };
Obviously I don't like this very much, but I agree there is not much
else we can do at this point. Shall I add a
/* Temporary workaround for broken TFA BL31 */
comment while applying?
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-fixes for v7.0.
> +
> /* Page 27 / DSI to Display */
> dp-con {
> compatible = "dp-connector";
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Nicolas Frattaroli @ 2026-03-25 13:05 UTC (permalink / raw)
To: Ville Syrjälä
Cc: Harry Wentland, Leo Li, Rodrigo Siqueira, Alex Deucher,
Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Sandy Huang, Heiko Stübner,
Andy Yan, Jani Nikula, Rodrigo Vivi, Joonas Lahtinen,
Tvrtko Ursulin, Dmitry Baryshkov, Sascha Hauer, Rob Herring,
Jonathan Corbet, Shuah Khan, kernel, amd-gfx, dri-devel,
linux-kernel, linux-arm-kernel, linux-rockchip, intel-gfx,
intel-xe, linux-doc, Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <acLrv5hLyNss-Px5@intel.com>
On Tuesday, 24 March 2026 20:53:35 Central European Standard Time Ville Syrjälä wrote:
> On Tue, Mar 24, 2026 at 08:10:11PM +0100, Nicolas Frattaroli wrote:
> > On Tuesday, 24 March 2026 18:00:45 Central European Standard Time Ville Syrjälä wrote:
> > > On Tue, Mar 24, 2026 at 05:01:07PM +0100, Nicolas Frattaroli wrote:
> > > > +enum drm_connector_color_format {
> > > > + /**
> > > > + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> > > > + * helpers should pick a suitable color format. All implementations of a
> > > > + * specific display protocol must behave the same way with "AUTO", but
> > > > + * different display protocols do not necessarily have the same "AUTO"
> > > > + * semantics.
> > > > + *
> > > > + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> > > > + * bandwidth required for full-scale RGB is not available, or the mode
> > > > + * is YCbCr 4:2:0-only, as long as the mode and output both support
> > > > + * YCbCr 4:2:0.
> > > > + *
> > > > + * For display protocols other than HDMI, the recursive bridge chain
> > > > + * format selection picks the first chain of bridge formats that works,
> > > > + * as has already been the case before the introduction of the "color
> > > > + * format" property. Non-HDMI bridges should therefore either sort their
> > > > + * bus output formats by preference, or agree on a unified auto format
> > > > + * selection logic that's implemented in a common state helper (like
> > > > + * how HDMI does it).
> > > > + */
> > > > + DRM_CONNECTOR_COLOR_FORMAT_AUTO = 0,
> > > > +
> > > > + /**
> > > > + * @DRM_CONNECTOR_COLOR_FORMAT_RGB444: RGB output format
> > > > + */
> > > > + DRM_CONNECTOR_COLOR_FORMAT_RGB444,
> > > > +
> > > > + /**
> > > > + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR444: YCbCr 4:4:4 output format (ie.
> > > > + * not subsampled)
> > > > + */
> > > > + DRM_CONNECTOR_COLOR_FORMAT_YCBCR444,
> > > > +
> > > > + /**
> > > > + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR422: YCbCr 4:2:2 output format (ie.
> > > > + * with horizontal subsampling)
> > > > + */
> > > > + DRM_CONNECTOR_COLOR_FORMAT_YCBCR422,
> > > > +
> > > > + /**
> > > > + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR420: YCbCr 4:2:0 output format (ie.
> > > > + * with horizontal and vertical subsampling)
> > > > + */
> > > > + DRM_CONNECTOR_COLOR_FORMAT_YCBCR420,
> > >
> > > Seems like this should document what the quantization range
> > > should be for each format.
> > >
> >
> > I don't think so? If you want per-component bit depth values,
> > DRM_FORMAT_* defines would be the appropriate values to use. This
> > enum is more abstract than that, and is there to communicate
> > YUV vs. RGB and chroma subsampling, with bit depth being handled
> > by other properties.
> >
> > If you mean the factor used for subsampling, then that'd only be
> > relevant if YCBCR410 was supported where one chroma plane isn't
> > halved but quartered in resolution. I suspect 4:1:0 will never
> > be added; no digital display protocol standard supports it to my
> > knowledge, and hopefully none ever will.
>
> No, I mean the quantization range (16-235 vs. 0-255 etc).
>
> The i915 behaviour is that YCbCr is always limited range,
> RGB can either be full or limited range depending on the
> "Broadcast RGB" property and other related factors.
>
>
I do agree that this would be useful to be precise about, but I'm
not sure mainline is currently capable of even making a definitive
statement about this because every implementation will do its own
thing probably.
If we do add YCbCr quantization range as a separate property (though
I'd really would've loved it if "Broadcast RGB" was named differently
so it wouldn't be confusing if we overloaded that property's meaning
to also apply to YCbCr) then the resulting docs change for these
enums would be to say that quantization range is a separate property.
I do think making it separate is the best path forward, and I'm willing
to adjust the docs to mention this, but I think implementing YCbCr limited
versus full range in this series as well would drastically widen the scope
again.
Kind regards,
Nicolas Frattaroli
^ permalink raw reply
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Maxime Ripard @ 2026-03-25 13:05 UTC (permalink / raw)
To: Dave Stevenson
Cc: Nicolas Frattaroli, Harry Wentland, Leo Li, Rodrigo Siqueira,
Alex Deucher, Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Thomas Zimmermann, Andrzej Hajda,
Neil Armstrong, Robert Foss, Laurent Pinchart, Jonas Karlman,
Jernej Skrabec, Sandy Huang, Heiko Stübner, Andy Yan,
Jani Nikula, Rodrigo Vivi, Joonas Lahtinen, Tvrtko Ursulin,
Dmitry Baryshkov, Sascha Hauer, Rob Herring, Jonathan Corbet,
Shuah Khan, kernel, amd-gfx, dri-devel, linux-kernel,
linux-arm-kernel, linux-rockchip, intel-gfx, intel-xe, linux-doc,
Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <CAPY8ntB9f_=f5kru=8w9BpTuqQR+93maGpT61EKU28Uay2vq8Q@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2493 bytes --]
Hi Dave,
On Wed, Mar 25, 2026 at 12:49:19PM +0000, Dave Stevenson wrote:
> > diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
> > index af8b92d2d5b7..bd549f912b76 100644
> > --- a/include/drm/drm_connector.h
> > +++ b/include/drm/drm_connector.h
> > @@ -571,14 +571,102 @@ enum drm_colorspace {
> > * YCbCr 4:2:2 output format (ie. with horizontal subsampling)
> > * @DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> > * YCbCr 4:2:0 output format (ie. with horizontal and vertical subsampling)
> > + * @DRM_OUTPUT_COLOR_FORMAT_COUNT:
> > + * Number of valid output color format values in this enum
> > */
> > enum drm_output_color_format {
> > DRM_OUTPUT_COLOR_FORMAT_RGB444 = 0,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR444,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR422,
> > DRM_OUTPUT_COLOR_FORMAT_YCBCR420,
> > + DRM_OUTPUT_COLOR_FORMAT_COUNT,
> > };
> >
> > +/**
> > + * enum drm_connector_color_format - Connector Color Format Request
> > + *
> > + * This enum, unlike &enum drm_output_color_format, is used to specify requests
> > + * for a specific color format on a connector through the DRM "color format"
> > + * property. The difference is that it has an "AUTO" value to specify that
> > + * no specific choice has been made.
> > + */
> > +enum drm_connector_color_format {
> > + /**
> > + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> > + * helpers should pick a suitable color format. All implementations of a
> > + * specific display protocol must behave the same way with "AUTO", but
> > + * different display protocols do not necessarily have the same "AUTO"
> > + * semantics.
> > + *
> > + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> > + * bandwidth required for full-scale RGB is not available, or the mode
> > + * is YCbCr 4:2:0-only, as long as the mode and output both support
> > + * YCbCr 4:2:0.
>
> Is there a reason you propose dropping back to YCbCr 4:2:0 without
> trying YCbCr 4:2:2 first? Minimising the subsampling is surely
> beneficial, and vc4 for one can do 4:2:2 but not 4:2:0.
The "auto" behaviour is strictly identical to the one we have right now,
and this one stems from i915. Back when all that logic was added, it was
decided to align every driver behavior on i915 because that's what most
compositors would expect.
Maxime
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 273 bytes --]
^ permalink raw reply
* [PATCH] net: ti: icssg-prueth: fix missing data copy and wrong recycle in ZC RX dispatch
From: David Carlier @ 2026-03-25 12:51 UTC (permalink / raw)
To: danishanwar, rogerq, andrew+netdev, davem, edumazet, kuba, pabeni
Cc: m-malladi, jacob.e.keller, horms, linux-arm-kernel, netdev,
linux-kernel, David Carlier
emac_dispatch_skb_zc() allocates a new skb via napi_alloc_skb() but
never copies the packet data from the XDP buffer into it. The skb is
passed up the stack containing uninitialized heap memory instead of
the actual received packet, leaking kernel heap contents to userspace.
Copy the received packet data from the XDP buffer into the skb using
skb_copy_to_linear_data().
Additionally, remove the skb_mark_for_recycle() call since the skb is
backed by the NAPI page frag allocator, not page_pool. Marking a
non-page_pool skb for recycle causes the free path to return pages to
a page_pool that does not own them, corrupting page_pool state.
The non-ZC path (emac_rx_packet) does not have these issues because it
uses napi_build_skb() to wrap the existing page_pool page directly,
requiring no copy, and correctly marks for recycle since the page comes
from page_pool_dev_alloc_pages().
Fixes: 7a64bb388df3 ("net: ti: icssg-prueth: Add AF_XDP zero copy for RX")
Signed-off-by: David Carlier <devnexen@gmail.com>
---
drivers/net/ethernet/ti/icssg/icssg_common.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/ti/icssg/icssg_common.c b/drivers/net/ethernet/ti/icssg/icssg_common.c
index fd4e7622f123..a28a608f9bf4 100644
--- a/drivers/net/ethernet/ti/icssg/icssg_common.c
+++ b/drivers/net/ethernet/ti/icssg/icssg_common.c
@@ -902,6 +902,7 @@ static void emac_dispatch_skb_zc(struct prueth_emac *emac, struct xdp_buff *xdp,
skb_reserve(skb, headroom);
skb_put(skb, pkt_len);
+ skb_copy_to_linear_data(skb, xdp->data, pkt_len);
skb->dev = ndev;
/* RX HW timestamp */
@@ -912,7 +913,6 @@ static void emac_dispatch_skb_zc(struct prueth_emac *emac, struct xdp_buff *xdp,
skb->offload_fwd_mark = emac->offload_fwd_mark;
skb->protocol = eth_type_trans(skb, ndev);
- skb_mark_for_recycle(skb);
napi_gro_receive(&emac->napi_rx, skb);
ndev->stats.rx_bytes += pkt_len;
ndev->stats.rx_packets++;
--
2.53.0
^ permalink raw reply related
* Re: [PATCH v11 03/22] drm: Add new general DRM property "color format"
From: Dave Stevenson @ 2026-03-25 12:49 UTC (permalink / raw)
To: Nicolas Frattaroli
Cc: Harry Wentland, Leo Li, Rodrigo Siqueira, Alex Deucher,
Christian König, David Airlie, Simona Vetter,
Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann,
Andrzej Hajda, Neil Armstrong, Robert Foss, Laurent Pinchart,
Jonas Karlman, Jernej Skrabec, Sandy Huang, Heiko Stübner,
Andy Yan, Jani Nikula, Rodrigo Vivi, Joonas Lahtinen,
Tvrtko Ursulin, Dmitry Baryshkov, Sascha Hauer, Rob Herring,
Jonathan Corbet, Shuah Khan, kernel, amd-gfx, dri-devel,
linux-kernel, linux-arm-kernel, linux-rockchip, intel-gfx,
intel-xe, linux-doc, Werner Sembach, Andri Yngvason, Marius Vlad
In-Reply-To: <20260324-color-format-v11-3-605559af4fb4@collabora.com>
On Tue, 24 Mar 2026 at 16:02, Nicolas Frattaroli
<nicolas.frattaroli@collabora.com> wrote:
>
> Add a new general DRM property named "color format" which can be used by
> userspace to request the display driver to output a particular color
> format.
>
> Possible options are:
> - auto (setup by default, driver internally picks the color format)
> - rgb
> - ycbcr444
> - ycbcr422
> - ycbcr420
>
> Drivers should advertise from this list which formats they support.
> Together with this list and EDID data from the sink we should be able
> to relay a list of usable color formats to users to pick from.
>
> Co-developed-by: Werner Sembach <wse@tuxedocomputers.com>
> Signed-off-by: Werner Sembach <wse@tuxedocomputers.com>
> Co-developed-by: Andri Yngvason <andri@yngvason.is>
> Signed-off-by: Andri Yngvason <andri@yngvason.is>
> Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
> Reviewed-by: Maxime Ripard <mripard@kernel.org>
> Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli@collabora.com>
> ---
> drivers/gpu/drm/drm_atomic_helper.c | 5 ++
> drivers/gpu/drm/drm_atomic_uapi.c | 11 ++++
> drivers/gpu/drm/drm_connector.c | 108 ++++++++++++++++++++++++++++++++++++
> include/drm/drm_connector.h | 104 ++++++++++++++++++++++++++++++++++
> 4 files changed, 228 insertions(+)
>
> diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c
> index 26953ed6b53e..b7753454b777 100644
> --- a/drivers/gpu/drm/drm_atomic_helper.c
> +++ b/drivers/gpu/drm/drm_atomic_helper.c
> @@ -737,6 +737,11 @@ drm_atomic_helper_check_modeset(struct drm_device *dev,
> if (old_connector_state->max_requested_bpc !=
> new_connector_state->max_requested_bpc)
> new_crtc_state->connectors_changed = true;
> +
> + if (old_connector_state->color_format !=
> + new_connector_state->color_format)
> + new_crtc_state->connectors_changed = true;
> +
> }
>
> if (funcs->atomic_check)
> diff --git a/drivers/gpu/drm/drm_atomic_uapi.c b/drivers/gpu/drm/drm_atomic_uapi.c
> index 5bd5bf6661df..dee510c85e59 100644
> --- a/drivers/gpu/drm/drm_atomic_uapi.c
> +++ b/drivers/gpu/drm/drm_atomic_uapi.c
> @@ -935,6 +935,15 @@ static int drm_atomic_connector_set_property(struct drm_connector *connector,
> state->privacy_screen_sw_state = val;
> } else if (property == connector->broadcast_rgb_property) {
> state->hdmi.broadcast_rgb = val;
> + } else if (property == connector->color_format_property) {
> + if (val > INT_MAX || !drm_connector_color_format_valid(val)) {
> + drm_dbg_atomic(connector->dev,
> + "[CONNECTOR:%d:%s] unknown color format %llu\n",
> + connector->base.id, connector->name, val);
> + return -EINVAL;
> + }
> +
> + state->color_format = val;
> } else if (connector->funcs->atomic_set_property) {
> return connector->funcs->atomic_set_property(connector,
> state, property, val);
> @@ -1020,6 +1029,8 @@ drm_atomic_connector_get_property(struct drm_connector *connector,
> *val = state->privacy_screen_sw_state;
> } else if (property == connector->broadcast_rgb_property) {
> *val = state->hdmi.broadcast_rgb;
> + } else if (property == connector->color_format_property) {
> + *val = state->color_format;
> } else if (connector->funcs->atomic_get_property) {
> return connector->funcs->atomic_get_property(connector,
> state, property, val);
> diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
> index 47dc53c4a738..e848374dee0b 100644
> --- a/drivers/gpu/drm/drm_connector.c
> +++ b/drivers/gpu/drm/drm_connector.c
> @@ -1388,6 +1388,18 @@ static const u32 hdmi_colorspaces =
> BIT(DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65) |
> BIT(DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER);
>
> +static const u32 hdmi_colorformats =
> + BIT(DRM_OUTPUT_COLOR_FORMAT_RGB444) |
> + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR444) |
> + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR422) |
> + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR420);
> +
> +static const u32 dp_colorformats =
> + BIT(DRM_OUTPUT_COLOR_FORMAT_RGB444) |
> + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR444) |
> + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR422) |
> + BIT(DRM_OUTPUT_COLOR_FORMAT_YCBCR420);
> +
> /*
> * As per DP 1.4a spec, 2.2.5.7.5 VSC SDP Payload for Pixel Encoding/Colorimetry
> * Format Table 2-120
> @@ -2940,6 +2952,102 @@ int drm_connector_attach_colorspace_property(struct drm_connector *connector)
> }
> EXPORT_SYMBOL(drm_connector_attach_colorspace_property);
>
> +/**
> + * drm_connector_attach_color_format_property - create and attach color format property
> + * @connector: connector to create the color format property on
> + * @supported_color_formats: bitmask of bit-shifted &enum drm_output_color_format
> + * values the connector supports
> + *
> + * Called by a driver to create a color format property. The property is
> + * attached to the connector automatically on success.
> + *
> + * @supported_color_formats should only include color formats the connector
> + * type can actually support.
> + *
> + * Returns:
> + * 0 on success, negative errno on error
> + */
> +int drm_connector_attach_color_format_property(struct drm_connector *connector,
> + unsigned long supported_color_formats)
> +{
> + struct drm_device *dev = connector->dev;
> + struct drm_prop_enum_list enum_list[DRM_CONNECTOR_COLOR_FORMAT_COUNT];
> + unsigned int i = 0;
> + unsigned long fmt;
> +
> + if (connector->color_format_property)
> + return 0;
> +
> + if (!supported_color_formats) {
> + drm_err(dev, "No supported color formats provided on [CONNECTOR:%d:%s]\n",
> + connector->base.id, connector->name);
> + return -EINVAL;
> + }
> +
> + if (supported_color_formats & ~GENMASK(DRM_OUTPUT_COLOR_FORMAT_COUNT - 1, 0)) {
> + drm_err(dev, "Unknown color formats provided on [CONNECTOR:%d:%s]\n",
> + connector->base.id, connector->name);
> + return -EINVAL;
> + }
> +
> + switch (connector->connector_type) {
> + case DRM_MODE_CONNECTOR_HDMIA:
> + case DRM_MODE_CONNECTOR_HDMIB:
> + if (supported_color_formats & ~hdmi_colorformats) {
> + drm_err(dev, "Color formats not allowed for HDMI on [CONNECTOR:%d:%s]\n",
> + connector->base.id, connector->name);
> + return -EINVAL;
> + }
> + break;
> + case DRM_MODE_CONNECTOR_DisplayPort:
> + case DRM_MODE_CONNECTOR_eDP:
> + if (supported_color_formats & ~dp_colorformats) {
> + drm_err(dev, "Color formats not allowed for DP on [CONNECTOR:%d:%s]\n",
> + connector->base.id, connector->name);
> + return -EINVAL;
> + }
> + break;
> + }
> +
> + enum_list[0].name = "AUTO";
> + enum_list[0].type = DRM_CONNECTOR_COLOR_FORMAT_AUTO;
> +
> + for_each_set_bit(fmt, &supported_color_formats, DRM_OUTPUT_COLOR_FORMAT_COUNT) {
> + switch (fmt) {
> + case DRM_OUTPUT_COLOR_FORMAT_RGB444:
> + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_RGB444;
> + break;
> + case DRM_OUTPUT_COLOR_FORMAT_YCBCR444:
> + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR444;
> + break;
> + case DRM_OUTPUT_COLOR_FORMAT_YCBCR422:
> + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR422;
> + break;
> + case DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> + enum_list[++i].type = DRM_CONNECTOR_COLOR_FORMAT_YCBCR420;
> + break;
> + default:
> + drm_warn(dev, "Unknown supported format %ld on [CONNECTOR:%d:%s]\n",
> + fmt, connector->base.id, connector->name);
> + continue;
> + }
> + enum_list[i].name = drm_hdmi_connector_get_output_format_name(fmt);
> + }
> +
> + connector->color_format_property =
> + drm_property_create_enum(dev, DRM_MODE_PROP_ENUM, "color format",
> + enum_list, i + 1);
> +
> + if (!connector->color_format_property)
> + return -ENOMEM;
> +
> + drm_object_attach_property(&connector->base, connector->color_format_property,
> + DRM_CONNECTOR_COLOR_FORMAT_AUTO);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL(drm_connector_attach_color_format_property);
> +
> /**
> * drm_connector_atomic_hdr_metadata_equal - checks if the hdr metadata changed
> * @old_state: old connector state to compare
> diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
> index af8b92d2d5b7..bd549f912b76 100644
> --- a/include/drm/drm_connector.h
> +++ b/include/drm/drm_connector.h
> @@ -571,14 +571,102 @@ enum drm_colorspace {
> * YCbCr 4:2:2 output format (ie. with horizontal subsampling)
> * @DRM_OUTPUT_COLOR_FORMAT_YCBCR420:
> * YCbCr 4:2:0 output format (ie. with horizontal and vertical subsampling)
> + * @DRM_OUTPUT_COLOR_FORMAT_COUNT:
> + * Number of valid output color format values in this enum
> */
> enum drm_output_color_format {
> DRM_OUTPUT_COLOR_FORMAT_RGB444 = 0,
> DRM_OUTPUT_COLOR_FORMAT_YCBCR444,
> DRM_OUTPUT_COLOR_FORMAT_YCBCR422,
> DRM_OUTPUT_COLOR_FORMAT_YCBCR420,
> + DRM_OUTPUT_COLOR_FORMAT_COUNT,
> };
>
> +/**
> + * enum drm_connector_color_format - Connector Color Format Request
> + *
> + * This enum, unlike &enum drm_output_color_format, is used to specify requests
> + * for a specific color format on a connector through the DRM "color format"
> + * property. The difference is that it has an "AUTO" value to specify that
> + * no specific choice has been made.
> + */
> +enum drm_connector_color_format {
> + /**
> + * @DRM_CONNECTOR_COLOR_FORMAT_AUTO: The driver or display protocol
> + * helpers should pick a suitable color format. All implementations of a
> + * specific display protocol must behave the same way with "AUTO", but
> + * different display protocols do not necessarily have the same "AUTO"
> + * semantics.
> + *
> + * For HDMI, "AUTO" picks RGB, but falls back to YCbCr 4:2:0 if the
> + * bandwidth required for full-scale RGB is not available, or the mode
> + * is YCbCr 4:2:0-only, as long as the mode and output both support
> + * YCbCr 4:2:0.
Is there a reason you propose dropping back to YCbCr 4:2:0 without
trying YCbCr 4:2:2 first? Minimising the subsampling is surely
beneficial, and vc4 for one can do 4:2:2 but not 4:2:0.
Dave
> + *
> + * For display protocols other than HDMI, the recursive bridge chain
> + * format selection picks the first chain of bridge formats that works,
> + * as has already been the case before the introduction of the "color
> + * format" property. Non-HDMI bridges should therefore either sort their
> + * bus output formats by preference, or agree on a unified auto format
> + * selection logic that's implemented in a common state helper (like
> + * how HDMI does it).
> + */
> + DRM_CONNECTOR_COLOR_FORMAT_AUTO = 0,
> +
> + /**
> + * @DRM_CONNECTOR_COLOR_FORMAT_RGB444: RGB output format
> + */
> + DRM_CONNECTOR_COLOR_FORMAT_RGB444,
> +
> + /**
> + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR444: YCbCr 4:4:4 output format (ie.
> + * not subsampled)
> + */
> + DRM_CONNECTOR_COLOR_FORMAT_YCBCR444,
> +
> + /**
> + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR422: YCbCr 4:2:2 output format (ie.
> + * with horizontal subsampling)
> + */
> + DRM_CONNECTOR_COLOR_FORMAT_YCBCR422,
> +
> + /**
> + * @DRM_CONNECTOR_COLOR_FORMAT_YCBCR420: YCbCr 4:2:0 output format (ie.
> + * with horizontal and vertical subsampling)
> + */
> + DRM_CONNECTOR_COLOR_FORMAT_YCBCR420,
> +
> + /**
> + * @DRM_CONNECTOR_COLOR_FORMAT_COUNT: Number of valid connector color
> + * format values in this enum
> + */
> + DRM_CONNECTOR_COLOR_FORMAT_COUNT,
> +};
> +
> +/**
> + * drm_connector_color_format_valid - Validate drm_connector_color_format value
> + * @fmt: value to check against all values of &enum drm_connector_color_format
> + *
> + * Checks whether the passed in value of @fmt is one of the allowable values in
> + * &enum drm_connector_color_format.
> + *
> + * Returns: %true if it's a valid value for the enum, %false otherwise.
> + */
> +static inline bool __pure
> +drm_connector_color_format_valid(enum drm_connector_color_format fmt)
> +{
> + switch (fmt) {
> + case DRM_CONNECTOR_COLOR_FORMAT_AUTO:
> + case DRM_CONNECTOR_COLOR_FORMAT_RGB444:
> + case DRM_CONNECTOR_COLOR_FORMAT_YCBCR444:
> + case DRM_CONNECTOR_COLOR_FORMAT_YCBCR422:
> + case DRM_CONNECTOR_COLOR_FORMAT_YCBCR420:
> + return true;
> + default:
> + return false;
> + }
> +}
> +
> const char *
> drm_hdmi_connector_get_output_format_name(enum drm_output_color_format fmt);
>
> @@ -1129,6 +1217,13 @@ struct drm_connector_state {
> */
> enum drm_colorspace colorspace;
>
> + /**
> + * @color_format: State variable for Connector property to request
> + * color format change on Sink. This is most commonly used to switch
> + * between RGB to YUV and vice-versa.
> + */
> + enum drm_connector_color_format color_format;
> +
> /**
> * @writeback_job: Writeback job for writeback connectors
> *
> @@ -2127,6 +2222,12 @@ struct drm_connector {
> */
> struct drm_property *colorspace_property;
>
> + /**
> + * @color_format_property: Connector property to set the suitable
> + * color format supported by the sink.
> + */
> + struct drm_property *color_format_property;
> +
> /**
> * @path_blob_ptr:
> *
> @@ -2610,6 +2711,9 @@ bool drm_connector_has_possible_encoder(struct drm_connector *connector,
> struct drm_encoder *encoder);
> const char *drm_get_colorspace_name(enum drm_colorspace colorspace);
>
> +int drm_connector_attach_color_format_property(struct drm_connector *connector,
> + unsigned long supported_color_formats);
> +
> /**
> * drm_for_each_connector_iter - connector_list iterator macro
> * @connector: &struct drm_connector pointer used as cursor
>
> --
> 2.53.0
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox