* [PATCH v7 0/2] media: nxp: Add CSI Pixel Formatter support
@ 2026-05-18 2:19 Guoniu Zhou
2026-05-18 2:19 ` [PATCH v7 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings Guoniu Zhou
2026-05-18 2:19 ` [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver Guoniu Zhou
0 siblings, 2 replies; 6+ messages in thread
From: Guoniu Zhou @ 2026-05-18 2:19 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Laurent Pinchart, Frank Li
Cc: imx, linux-media, devicetree, linux-arm-kernel, linux-kernel,
Guoniu Zhou, Frank Li, Krzysztof Kozlowski
CSI Pixel Formatter is a module found on i.MX95. It could unpack the
pixels received by the formatter and reformat them to meet the pixel
link format requirement.
This patch series adds a new V4L2 driver for CSI Pixel Formatter.
v4l2-compliance 1.28.1-5233, 64 bits, 64-bit time_t
v4l2-compliance SHA: fc15e229d9d3 2024-07-23 19:22:15
Compliance test for device /dev/v4l-subdev9:
Driver Info:
Driver version : 7.1.0
Capabilities : 0x00000002
Streams Support
Client Capabilities: 0x0000000000000003
streams interval-uses-which
Required ioctls:
test VIDIOC_SUDBEV_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/v4l-subdev9 open: OK
test VIDIOC_SUBDEV_QUERYCAP: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Sub-Device routing ioctls:
test Try VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: OK
test Active VIDIOC_SUBDEV_G_ROUTING/VIDIOC_SUBDEV_S_ROUTING: OK
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK (Not Supported)
test VIDIOC_TRY_FMT: OK (Not Supported)
test VIDIOC_S_FMT: OK (Not Supported)
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK (Not Supported)
test Requests: OK (Not Supported)
Total for device /dev/v4l-subdev9: 47, Succeeded: 47, Failed: 0, Warnings: 0
Signed-off-by: Guoniu Zhou <guoniu.zhou@nxp.com>
---
Changes in v7:
- Change compatible to imx95-csi-formatter as IP is i.MX95 specific per Marco's suggestion
Link: https://lore.kernel.org/linux-media/20260511-csi_formatter-v6-0-01028e312e2b@oss.nxp.com/T/#mcd135b3de179b3cb69daa1fd6e0e8e27c85b3332
- Update references from imx9 to imx95 for consistency with dt-bindings
- Enable PM runtime before async registration
- Link to v6: https://lore.kernel.org/r/20260511-csi_formatter-v6-0-01028e312e2b@oss.nxp.com
Changes in v6:
- Rebase to latest media/next
- Update v4l2-compliace test
- Remove unused header includes
- Unify macro naming: VCx/VCX -> VC and parameter x -> vc
- Remove unused format field from csi_formatter struct
- Use compact initialization for formats array
- Make find_csi_format() return NULL instead of default format
- Use unsigned int for array index in find_csi_format()
- Add err_ prefix to error handling labels
- Add v4l2_subdev_cleanup() and reorder cleanup sequence
- Update enable_streams debug output format
- Rename VC_MAX to VC_NUM and fix boundary check
- Update CSI formatter Kconfig description
- Use v4l2_subdev_get_frame_desc_passthrough() helper
- Fix error paths in async registration and probe
- Add mutex to protect enabled_streams
- Switch to devm_pm_runtime_enable()
- Remove redundant num_routes check in set_routing
- Optimize get_index_by_dt() and add warning for unsupported type
- csi_formatter_start/stop_stream: Process all streams in mask
- Link to v5: https://lore.kernel.org/r/20260123-csi_formatter-v5-0-d5b803f867bf@nxp.com
Changes in v5:
- Remove CSI_FORMATTER_DRV_NAME macro since only use once.
- Remove sd->owner = THIS_MODULE;
- Simplify code by using DEFINE_RUNTIME_DEV_PM_OPS macro.
- Link to v4: https://lore.kernel.org/r/20260122-csi_formatter-v4-0-6f6fcad1c33a@nxp.com
Changes in v4:
- Rebase to latest media/next.
- Add comments to describe the index field in formatter_dt_to_index_map array.
- Link to v3: https://lore.kernel.org/r/20251219-csi_formatter-v3-0-8680d6d87091@nxp.com
Changes in v3:
- Rename nxp,imx9-csi-formatter.yaml to fsl,imx9-csi-formatter.yaml.
- Drop clock-names property.
- Drop macro IMX95_PD_CAMERA definition and use a constant directly.
[PATCH 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings
- Remove the assignment driver.owner = THIS_MODULE.
- Assign struct fwnode_handle *ep __free(fwnode_handle) when definition.
- Update yaml file name for csi formatter in MAINTAINERS.
[PATCH 2/2] media: nxp: Add i.MX9 CSI pixel formatter v4l2 driver
- Link to v2: https://lore.kernel.org/r/20251217-csi_formatter-v2-0-62168af80210@nxp.com
Changes in v2:
- Delete "|" for description key.
- Add empty line between child node and property.
- Delete labels for endpoint of child nodes.
[PATCH 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings
- Update commit message.
- Use the value defined by bellow macros directly since they are used only once.
#define CSI_FORMATTER_DEF_MBUS_CODE MEDIA_BUS_FMT_UYVY8_1X16
#define CSI_FORMATTER_DEF_PIX_WIDTH 1920U
#define CSI_FORMATTER_DEF_PIX_HEIGHT 1080U
#define CSI_FORMATTER_MAX_PIX_WIDTH 0xffff
#define CSI_FORMATTER_MAX_PIX_HEIGHT 0xffff
- Use macro pm_ptr() to fix build warning when CONFIG_PM is disabled.
- Finish route loop by break statement, instead of goto.
- Return dev_err_probe() when meet errors in probe() function instead of dev_err().
- Remove MODULE_ALIAS().
- Refine .enable(.dsable)_stream callback implementation, include bellow changes:
Add stream checking.
Fix potential pm runtime count unbalance issue.
Add stop stream error handling when enabling remote subdev stream.
- Use __free(fwnode_handle) to drop reference to a device node automatically.
[PATCH 2/2] media: nxp: Add i.MX9 CSI pixel formatter v4l2 driver
- Link to v1: https://lore.kernel.org/r/20251203-csi_formatter-v1-0-eb9e1147b49e@nxp.com
---
Guoniu Zhou (2):
media: dt-bindings: Add CSI Pixel Formatter DT bindings
media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver
.../bindings/media/fsl,imx95-csi-formatter.yaml | 87 +++
MAINTAINERS | 8 +
drivers/media/platform/nxp/Kconfig | 14 +
drivers/media/platform/nxp/Makefile | 1 +
drivers/media/platform/nxp/imx95-csi-formatter.c | 776 +++++++++++++++++++++
5 files changed, 886 insertions(+)
---
base-commit: bc1ba628e37c93cf2abeb2c79716f49087f8a024
change-id: 20251125-csi_formatter-e6d29316dce6
Best regards,
--
Guoniu Zhou <guoniu.zhou@oss.nxp.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v7 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings
2026-05-18 2:19 [PATCH v7 0/2] media: nxp: Add CSI Pixel Formatter support Guoniu Zhou
@ 2026-05-18 2:19 ` Guoniu Zhou
2026-05-20 21:33 ` Laurent Pinchart
2026-05-18 2:19 ` [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver Guoniu Zhou
1 sibling, 1 reply; 6+ messages in thread
From: Guoniu Zhou @ 2026-05-18 2:19 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Laurent Pinchart, Frank Li
Cc: imx, linux-media, devicetree, linux-arm-kernel, linux-kernel,
Guoniu Zhou, Frank Li, Krzysztof Kozlowski
From: Guoniu Zhou <guoniu.zhou@nxp.com>
The i.MX95 CSI pixel formatting module uses packet info, pixel and
non-pixel data from the CSI-2 host controller and reformat them to
match Pixel Link(PL) definition.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Signed-off-by: Guoniu Zhou <guoniu.zhou@nxp.com>
---
Changes in v7:
- Change compatible to imx95-csi-formatter as IP is i.MX95 specific per Marco's suggestion
Link: https://lore.kernel.org/linux-media/20260511-csi_formatter-v6-0-01028e312e2b@oss.nxp.com/T/#mcd135b3de179b3cb69daa1fd6e0e8e27c85b3332
---
.../bindings/media/fsl,imx95-csi-formatter.yaml | 87 ++++++++++++++++++++++
1 file changed, 87 insertions(+)
diff --git a/Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml b/Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml
new file mode 100644
index 000000000000..28adea06c494
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml
@@ -0,0 +1,87 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/fsl,imx95-csi-formatter.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: i.MX95 CSI Pixel Formatter
+
+maintainers:
+ - Guoniu Zhou <guoniu.zhou@nxp.com>
+
+description:
+ The CSI pixel formatting module found on i.MX95 uses packet info, pixel
+ and non-pixel data from the CSI-2 host controller and reformat them to
+ match Pixel Link(PL) definition.
+
+properties:
+ compatible:
+ const: fsl,imx95-csi-formatter
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ power-domains:
+ maxItems: 1
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description: MIPI CSI-2 RX IDI interface
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ port@1:
+ $ref: /schemas/graph.yaml#/properties/port
+ description: Pixel Link Interface
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - power-domains
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/nxp,imx95-clock.h>
+
+ formatter@20 {
+ compatible = "fsl,imx95-csi-formatter";
+ reg = <0x20 0x100>;
+ clocks = <&cameramix_csr IMX95_CLK_CAMBLK_CSI2_FOR0>;
+ power-domains = <&scmi_devpd 3>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@0 {
+ reg = <0>;
+
+ endpoint {
+ remote-endpoint = <&mipi_csi_0_out>;
+ };
+ };
+
+ port@1 {
+ reg = <1>;
+
+ endpoint {
+ remote-endpoint = <&isi_in_2>;
+ };
+ };
+ };
+ };
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver
2026-05-18 2:19 [PATCH v7 0/2] media: nxp: Add CSI Pixel Formatter support Guoniu Zhou
2026-05-18 2:19 ` [PATCH v7 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings Guoniu Zhou
@ 2026-05-18 2:19 ` Guoniu Zhou
2026-05-18 2:46 ` sashiko-bot
2026-05-20 22:29 ` Laurent Pinchart
1 sibling, 2 replies; 6+ messages in thread
From: Guoniu Zhou @ 2026-05-18 2:19 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Laurent Pinchart, Frank Li
Cc: imx, linux-media, devicetree, linux-arm-kernel, linux-kernel,
Guoniu Zhou, Frank Li
From: Guoniu Zhou <guoniu.zhou@nxp.com>
The CSI pixel formatter is a module found on i.MX95 used to reformat
packet info, pixel and non-pixel data from CSI-2 host controller to
match Pixel Link(PL) definition.
Add data formatting support.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Guoniu Zhou <guoniu.zhou@nxp.com>
---
Changes in v7:
- Update references from imx9 to imx95 for consistency with dt-bindings
- Enable PM runtime before async registration
Changes in v6:
- Remove unused header includes
- Unify macro naming: VCx/VCX -> VC and parameter x -> vc
- Remove unused format field from csi_formatter struct
- Use compact initialization for formats array
- Make find_csi_format() return NULL instead of default format
- Use unsigned int for array index in find_csi_format()
- Add err_ prefix to error handling labels
- Add v4l2_subdev_cleanup() and reorder cleanup sequence
- Update enable_streams debug output format
- Rename VC_MAX to VC_NUM and fix boundary check
- Update CSI formatter Kconfig description
- Use v4l2_subdev_get_frame_desc_passthrough() helper
- Fix error paths in async registration and probe
- Add mutex to protect enabled_streams
- Switch to devm_pm_runtime_enable()
- Remove redundant num_routes check in set_routing
- Optimize get_index_by_dt() and add warning for unsupported type
- csi_formatter_start/stop_stream: Process all streams in mask
---
MAINTAINERS | 8 +
drivers/media/platform/nxp/Kconfig | 14 +
drivers/media/platform/nxp/Makefile | 1 +
drivers/media/platform/nxp/imx95-csi-formatter.c | 776 +++++++++++++++++++++++
4 files changed, 799 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 81d53481d3f7..837e14fe7a12 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -19265,6 +19265,14 @@ S: Maintained
F: Documentation/devicetree/bindings/media/nxp,imx8-jpeg.yaml
F: drivers/media/platform/nxp/imx-jpeg
+NXP i.MX 95 CSI PIXEL FORMATTER V4L2 DRIVER
+M: Guoniu Zhou <guoniu.zhou@nxp.com>
+L: imx@lists.linux.dev
+L: linux-media@vger.kernel.org
+S: Maintained
+F: Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml
+F: drivers/media/platform/nxp/imx95-csi-formatter.c
+
NXP i.MX CLOCK DRIVERS
M: Abel Vesa <abelvesa@kernel.org>
R: Peng Fan <peng.fan@nxp.com>
diff --git a/drivers/media/platform/nxp/Kconfig b/drivers/media/platform/nxp/Kconfig
index 40e3436669e2..82caae94a728 100644
--- a/drivers/media/platform/nxp/Kconfig
+++ b/drivers/media/platform/nxp/Kconfig
@@ -28,6 +28,20 @@ config VIDEO_IMX8MQ_MIPI_CSI2
Video4Linux2 driver for the MIPI CSI-2 receiver found on the i.MX8MQ
SoC.
+config VIDEO_IMX95_CSI_FORMATTER
+ tristate "NXP i.MX95 CSI Pixel Formatter driver"
+ depends on ARCH_MXC || COMPILE_TEST
+ depends on VIDEO_DEV
+ select MEDIA_CONTROLLER
+ select V4L2_FWNODE
+ select VIDEO_V4L2_SUBDEV_API
+ help
+ This driver provides support for the CSI Pixel Formatter found on
+ i.MX95 series SoCs. This module unpacks the pixels received from the
+ CSI-2 interface and reformats them to meet pixel link requirements.
+
+ Say Y here to enable CSI Pixel Formater module for i.MX95 SoC.
+
config VIDEO_IMX_MIPI_CSIS
tristate "NXP MIPI CSI-2 CSIS receiver found on i.MX7 and i.MX8 models"
depends on ARCH_MXC || COMPILE_TEST
diff --git a/drivers/media/platform/nxp/Makefile b/drivers/media/platform/nxp/Makefile
index 4d90eb713652..6410115d870e 100644
--- a/drivers/media/platform/nxp/Makefile
+++ b/drivers/media/platform/nxp/Makefile
@@ -6,6 +6,7 @@ obj-y += imx8-isi/
obj-$(CONFIG_VIDEO_IMX7_CSI) += imx7-media-csi.o
obj-$(CONFIG_VIDEO_IMX8MQ_MIPI_CSI2) += imx8mq-mipi-csi2.o
+obj-$(CONFIG_VIDEO_IMX95_CSI_FORMATTER) += imx95-csi-formatter.o
obj-$(CONFIG_VIDEO_IMX_MIPI_CSIS) += imx-mipi-csis.o
obj-$(CONFIG_VIDEO_IMX_PXP) += imx-pxp.o
obj-$(CONFIG_VIDEO_MX2_EMMAPRP) += mx2_emmaprp.o
diff --git a/drivers/media/platform/nxp/imx95-csi-formatter.c b/drivers/media/platform/nxp/imx95-csi-formatter.c
new file mode 100644
index 000000000000..45240b7422b4
--- /dev/null
+++ b/drivers/media/platform/nxp/imx95-csi-formatter.c
@@ -0,0 +1,776 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2025 NXP
+ */
+
+#include <linux/bits.h>
+#include <linux/clk.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/regmap.h>
+
+#include <media/mipi-csi2.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-mc.h>
+#include <media/v4l2-subdev.h>
+
+/* CSI Pixel Formatter registers map */
+
+#define CSI_VC_INTERLACED_LINE_CNT(vc) (0x00 + (vc) * 0x04)
+#define INTERLACED_ODD_LINE_CNT_SET(x) FIELD_PREP(GENMASK(13, 0), (x))
+#define INTERLACED_EVEN_LINE_CNT_SET(x) FIELD_PREP(GENMASK(29, 16), (x))
+
+#define CSI_VC_INTERLACED_CTRL 0x20
+
+#define CSI_VC_INTERLACED_ERR 0x24
+#define CSI_VC_ERR_MASK GENMASK(7, 0)
+#define CSI_VC_ERR(vc) BIT((vc))
+
+#define CSI_VC_YUV420_FIRST_LINE_EVEN 0x28
+#define YUV420_FIRST_LINE_EVEN(vc) BIT((vc))
+
+#define CSI_RAW32_CTRL 0x30
+#define CSI_VC_RAW32_MODE(vc) BIT((vc))
+#define CSI_VC_RAW32_SWAP_MODE(vc) BIT((vc) + 8)
+
+#define STREAM_FENCING_CTRL 0x34
+#define CSI_VC_STREAM_FENCING(vc) BIT((vc))
+#define CSI_VC_STREAM_FENCING_RST(vc) BIT((vc) + 8)
+
+#define STREAM_FENCING_STS 0x38
+#define STREAM_FENCING_STS_MASK GENMASK(7, 0)
+
+#define CSI_VC_NON_PIXEL_DATA_TYPE(vc) (0x40 + (vc) * 0x04)
+
+#define CSI_VC_PIXEL_DATA_CTRL(vc) (0x60 + (vc) * 0x04)
+#define NEW_VC(vc) FIELD_PREP(GENMASK(3, 1), vc)
+#define REROUTE_VC_ENABLE BIT(0)
+
+#define CSI_VC_ROUTE_PIXEL_DATA_TYPE(vc) (0x80 + (vc) * 0x04)
+
+#define CSI_VC_NON_PIXEL_DATA_CTRL(vc) (0xa0 + (vc) * 0x04)
+
+#define CSI_VC_PIXEL_DATA_TYPE(vc) (0xc0 + (vc) * 0x04)
+
+#define CSI_VC_PIXEL_DATA_TYPE_ERR(vc) (0xe0 + (vc) * 0x04)
+
+#define CSI_FORMATTER_PAD_SINK 0
+#define CSI_FORMATTER_PAD_SOURCE 1
+#define CSI_FORMATTER_PAD_NUM 2
+
+#define CSI_FORMATTER_VC_NUM 8 /* Number of virtual channels */
+
+struct formatter_pix_format {
+ u32 code;
+ u32 data_type;
+};
+
+struct csi_formatter {
+ struct device *dev;
+ struct regmap *regs;
+ struct clk *clk;
+
+ struct v4l2_subdev sd;
+ struct v4l2_subdev *csi_sd;
+ struct v4l2_async_notifier notifier;
+ struct media_pad pads[CSI_FORMATTER_PAD_NUM];
+ const struct formatter_pix_format *fmt;
+
+ u32 remote_pad;
+ u32 reg_offset;
+
+ /* Protects enabled_streams */
+ struct mutex lock;
+ u64 enabled_streams;
+};
+
+struct dt_index {
+ u8 dtype;
+ u8 index;
+};
+
+/*
+ * The index should correspond to the bit index define in register
+ * which enable the data type of pixel data transported by Formatter.
+ */
+static const struct dt_index formatter_dt_to_index_map[] = {
+ { .dtype = MIPI_CSI2_DT_YUV420_8B, .index = 0 },
+ { .dtype = MIPI_CSI2_DT_YUV420_8B_LEGACY, .index = 2 },
+ { .dtype = MIPI_CSI2_DT_YUV422_8B, .index = 6 },
+ { .dtype = MIPI_CSI2_DT_RGB444, .index = 8 },
+ { .dtype = MIPI_CSI2_DT_RGB555, .index = 9 },
+ { .dtype = MIPI_CSI2_DT_RGB565, .index = 10 },
+ { .dtype = MIPI_CSI2_DT_RGB666, .index = 11 },
+ { .dtype = MIPI_CSI2_DT_RGB888, .index = 12 },
+ { .dtype = MIPI_CSI2_DT_RAW6, .index = 16 },
+ { .dtype = MIPI_CSI2_DT_RAW7, .index = 17 },
+ { .dtype = MIPI_CSI2_DT_RAW8, .index = 18 },
+ { .dtype = MIPI_CSI2_DT_RAW10, .index = 19 },
+ { .dtype = MIPI_CSI2_DT_RAW12, .index = 20 },
+ { .dtype = MIPI_CSI2_DT_RAW14, .index = 21 },
+ { .dtype = MIPI_CSI2_DT_RAW16, .index = 22 },
+};
+
+static const struct formatter_pix_format formats[] = {
+ /* YUV formats */
+ { MEDIA_BUS_FMT_UYVY8_1X16, MIPI_CSI2_DT_YUV422_8B },
+ /* RGB formats */
+ { MEDIA_BUS_FMT_RGB565_1X16, MIPI_CSI2_DT_RGB565 },
+ { MEDIA_BUS_FMT_RGB888_1X24, MIPI_CSI2_DT_RGB888 },
+ /* RAW (Bayer and greyscale) formats */
+ { MEDIA_BUS_FMT_SBGGR8_1X8, MIPI_CSI2_DT_RAW8 },
+ { MEDIA_BUS_FMT_SGBRG8_1X8, MIPI_CSI2_DT_RAW8 },
+ { MEDIA_BUS_FMT_SGRBG8_1X8, MIPI_CSI2_DT_RAW8 },
+ { MEDIA_BUS_FMT_SRGGB8_1X8, MIPI_CSI2_DT_RAW8 },
+ { MEDIA_BUS_FMT_Y8_1X8, MIPI_CSI2_DT_RAW8 },
+ { MEDIA_BUS_FMT_SBGGR10_1X10, MIPI_CSI2_DT_RAW10 },
+ { MEDIA_BUS_FMT_SGBRG10_1X10, MIPI_CSI2_DT_RAW10 },
+ { MEDIA_BUS_FMT_SGRBG10_1X10, MIPI_CSI2_DT_RAW10 },
+ { MEDIA_BUS_FMT_SRGGB10_1X10, MIPI_CSI2_DT_RAW10 },
+ { MEDIA_BUS_FMT_Y10_1X10, MIPI_CSI2_DT_RAW10 },
+ { MEDIA_BUS_FMT_SBGGR12_1X12, MIPI_CSI2_DT_RAW12 },
+ { MEDIA_BUS_FMT_SGBRG12_1X12, MIPI_CSI2_DT_RAW12 },
+ { MEDIA_BUS_FMT_SGRBG12_1X12, MIPI_CSI2_DT_RAW12 },
+ { MEDIA_BUS_FMT_SRGGB12_1X12, MIPI_CSI2_DT_RAW12 },
+ { MEDIA_BUS_FMT_Y12_1X12, MIPI_CSI2_DT_RAW12 },
+ { MEDIA_BUS_FMT_SBGGR14_1X14, MIPI_CSI2_DT_RAW14 },
+ { MEDIA_BUS_FMT_SGBRG14_1X14, MIPI_CSI2_DT_RAW14 },
+ { MEDIA_BUS_FMT_SGRBG14_1X14, MIPI_CSI2_DT_RAW14 },
+ { MEDIA_BUS_FMT_SRGGB14_1X14, MIPI_CSI2_DT_RAW14 },
+ { MEDIA_BUS_FMT_SBGGR16_1X16, MIPI_CSI2_DT_RAW16 },
+ { MEDIA_BUS_FMT_SGBRG16_1X16, MIPI_CSI2_DT_RAW16 },
+ { MEDIA_BUS_FMT_SGRBG16_1X16, MIPI_CSI2_DT_RAW16 },
+ { MEDIA_BUS_FMT_SRGGB16_1X16, MIPI_CSI2_DT_RAW16 },
+};
+
+static const struct v4l2_mbus_framefmt formatter_default_fmt = {
+ .code = MEDIA_BUS_FMT_UYVY8_1X16,
+ .width = 1920U,
+ .height = 1080U,
+ .field = V4L2_FIELD_NONE,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(V4L2_COLORSPACE_SMPTE170M),
+ .ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(V4L2_COLORSPACE_SMPTE170M),
+ .quantization = V4L2_QUANTIZATION_LIM_RANGE,
+};
+
+static const struct formatter_pix_format *find_csi_format(u32 code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(formats); i++)
+ if (code == formats[i].code)
+ return &formats[i];
+
+ return NULL;
+}
+
+/* -----------------------------------------------------------------------------
+ * V4L2 subdev operations
+ */
+
+static inline struct csi_formatter *sd_to_formatter(struct v4l2_subdev *sdev)
+{
+ return container_of(sdev, struct csi_formatter, sd);
+}
+
+static int __formatter_subdev_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ struct v4l2_subdev_krouting *routing)
+{
+ int ret;
+
+ ret = v4l2_subdev_routing_validate(sd, routing,
+ V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+ if (ret)
+ return ret;
+
+ return v4l2_subdev_set_routing_with_fmt(sd, state, routing,
+ &formatter_default_fmt);
+}
+
+static int formatter_subdev_init_state(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state)
+{
+ struct v4l2_subdev_route routes[] = {
+ {
+ .sink_pad = CSI_FORMATTER_PAD_SINK,
+ .sink_stream = 0,
+ .source_pad = CSI_FORMATTER_PAD_SOURCE,
+ .source_stream = 0,
+ .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+ },
+ };
+
+ struct v4l2_subdev_krouting routing = {
+ .num_routes = ARRAY_SIZE(routes),
+ .routes = routes,
+ };
+
+ return __formatter_subdev_set_routing(sd, sd_state, &routing);
+}
+
+static int formatter_subdev_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad == CSI_FORMATTER_PAD_SOURCE) {
+ struct v4l2_mbus_framefmt *fmt;
+
+ if (code->index > 0)
+ return -EINVAL;
+
+ fmt = v4l2_subdev_state_get_format(sd_state, code->pad,
+ code->stream);
+ code->code = fmt->code;
+ return 0;
+ }
+
+ if (code->index >= ARRAY_SIZE(formats))
+ return -EINVAL;
+
+ code->code = formats[code->index].code;
+
+ return 0;
+}
+
+static int formatter_subdev_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct csi_formatter *formatter = sd_to_formatter(sd);
+ struct formatter_pix_format const *format;
+ struct v4l2_mbus_framefmt *fmt;
+
+ if (sdformat->pad == CSI_FORMATTER_PAD_SOURCE)
+ return v4l2_subdev_get_fmt(sd, sd_state, sdformat);
+
+ /*
+ * Validate the media bus code and clamp and align the size.
+ *
+ * The total number of bits per line must be a multiple of 8. We thus
+ * need to align the width for formats that are not multiples of 8
+ * bits.
+ */
+ format = find_csi_format(sdformat->format.code);
+ if (!format)
+ format = &formats[0];
+
+ v4l_bound_align_image(&sdformat->format.width, 1, 0xffff, 2,
+ &sdformat->format.height, 1, 0xffff, 0, 0);
+
+ fmt = v4l2_subdev_state_get_format(sd_state, sdformat->pad,
+ sdformat->stream);
+ *fmt = sdformat->format;
+
+ /* Set default code if user set an invalid value */
+ fmt->code = format->code;
+
+ /* Propagate the format from sink stream to source stream */
+ fmt = v4l2_subdev_state_get_opposite_stream_format(sd_state, sdformat->pad,
+ sdformat->stream);
+ if (!fmt)
+ return -EINVAL;
+
+ *fmt = sdformat->format;
+
+ /* Store the CSIS format descriptor for active formats. */
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+ formatter->fmt = format;
+
+ return 0;
+}
+
+static int formatter_subdev_set_routing(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ enum v4l2_subdev_format_whence which,
+ struct v4l2_subdev_krouting *routing)
+{
+ if (which == V4L2_SUBDEV_FORMAT_ACTIVE &&
+ media_entity_is_streaming(&sd->entity))
+ return -EBUSY;
+
+ return __formatter_subdev_set_routing(sd, state, routing);
+}
+
+static inline void formatter_write(struct csi_formatter *formatter,
+ unsigned int reg, unsigned int value)
+{
+ u32 offset = formatter->reg_offset;
+
+ regmap_write(formatter->regs, reg + offset, value);
+}
+
+static u8 get_index_by_dt(u8 data_type)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(formatter_dt_to_index_map); ++i) {
+ const struct dt_index *entry = &formatter_dt_to_index_map[i];
+
+ if (data_type == entry->dtype)
+ return entry->index;
+ }
+
+ pr_warn_once("Unsupported data type 0x%x, using default\n", data_type);
+
+ return formatter_dt_to_index_map[0].index;
+}
+
+static int get_vc(struct csi_formatter *formatter, unsigned int stream)
+{
+ struct v4l2_mbus_frame_desc source_fd;
+ struct v4l2_mbus_frame_desc_entry *entry = NULL;
+ unsigned int i;
+ int vc;
+ int ret;
+
+ /*
+ * Return virtual channel 0 as default value when remote subdev
+ * don't implement .get_frame_desc subdev callback
+ */
+ ret = v4l2_subdev_call(formatter->csi_sd, pad, get_frame_desc,
+ formatter->remote_pad, &source_fd);
+ if (ret < 0)
+ return (ret == -ENOIOCTLCMD) ? 0 : ret;
+
+ for (i = 0; i < source_fd.num_entries; ++i) {
+ if (source_fd.entry[i].stream == stream) {
+ entry = &source_fd.entry[i];
+ break;
+ }
+ }
+
+ if (!entry) {
+ dev_err(formatter->dev,
+ "Can't find valid frame desc corresponding to stream %d\n", stream);
+ return -EPIPE;
+ }
+
+ vc = entry->bus.csi2.vc;
+
+ if (vc < 0 || vc >= CSI_FORMATTER_VC_NUM) {
+ dev_err(formatter->dev, "Invalid virtual channel %d\n", vc);
+ return -EINVAL;
+ }
+
+ return vc;
+}
+
+static void csi_formatter_stop_stream(struct csi_formatter *formatter,
+ u64 stream_mask)
+{
+ unsigned int i;
+ int ret;
+ int vc;
+
+ for (i = 0; i < V4L2_FRAME_DESC_ENTRY_MAX; ++i) {
+ if (!(stream_mask & BIT(i)))
+ continue;
+
+ ret = get_vc(formatter, i);
+ if (WARN_ON(ret < 0)) {
+ dev_err(formatter->dev,
+ "Failed to get VC for stream %d: %d\n", i, ret);
+ continue;
+ }
+
+ vc = ret;
+
+ formatter_write(formatter, CSI_VC_PIXEL_DATA_TYPE(vc), 0);
+ }
+}
+
+static int csi_formatter_start_stream(struct csi_formatter *formatter,
+ u64 stream_mask)
+{
+ const struct formatter_pix_format *fmt = formatter->fmt;
+ u64 configured_streams = 0;
+ unsigned int i;
+ u32 val;
+ int ret;
+ int vc;
+
+ for (i = 0; i < V4L2_FRAME_DESC_ENTRY_MAX; ++i) {
+ if (!(stream_mask & BIT(i)))
+ continue;
+
+ val = BIT(get_index_by_dt(fmt->data_type));
+
+ ret = get_vc(formatter, i);
+ if (ret < 0)
+ goto err_cleanup;
+
+ vc = ret;
+
+ formatter_write(formatter, CSI_VC_PIXEL_DATA_TYPE(vc), val);
+ configured_streams |= BIT(i);
+ }
+
+ return 0;
+
+err_cleanup:
+ csi_formatter_stop_stream(formatter, configured_streams);
+ return ret;
+}
+
+static int formatter_subdev_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ u32 pad, u64 streams_mask)
+{
+ struct csi_formatter *formatter = sd_to_formatter(sd);
+ struct device *dev = formatter->dev;
+ u64 sink_streams;
+ int ret;
+
+ sink_streams = v4l2_subdev_state_xlate_streams(state,
+ CSI_FORMATTER_PAD_SOURCE,
+ CSI_FORMATTER_PAD_SINK,
+ &streams_mask);
+ if (!sink_streams || !streams_mask)
+ return -EINVAL;
+
+ dev_dbg(dev, "Enable streams: pad=%u sink=0x%llx source=0x%llx\n",
+ formatter->remote_pad, sink_streams, streams_mask);
+
+ if (!formatter->csi_sd) {
+ dev_err(dev, "CSI controller not linked with formatter\n");
+ return -EPIPE;
+ }
+
+ guard(mutex)(&formatter->lock);
+
+ if (!formatter->enabled_streams) {
+ ret = pm_runtime_resume_and_get(formatter->dev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to resume runtime PM: %d\n", ret);
+ return ret;
+ }
+ }
+
+ ret = csi_formatter_start_stream(formatter, streams_mask);
+ if (ret)
+ goto err_runtime_put;
+
+ ret = v4l2_subdev_enable_streams(formatter->csi_sd,
+ formatter->remote_pad,
+ sink_streams);
+ if (ret)
+ goto err_stop_stream;
+
+ formatter->enabled_streams |= streams_mask;
+
+ return 0;
+
+err_stop_stream:
+ csi_formatter_stop_stream(formatter, streams_mask);
+err_runtime_put:
+ if (!formatter->enabled_streams)
+ pm_runtime_put(formatter->dev);
+ return ret;
+}
+
+static int formatter_subdev_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state,
+ u32 pad, u64 streams_mask)
+{
+ struct csi_formatter *formatter = sd_to_formatter(sd);
+ u64 sink_streams;
+ int ret;
+
+ sink_streams = v4l2_subdev_state_xlate_streams(state,
+ CSI_FORMATTER_PAD_SOURCE,
+ CSI_FORMATTER_PAD_SINK,
+ &streams_mask);
+ if (!sink_streams || !streams_mask)
+ return -EINVAL;
+
+ guard(mutex)(&formatter->lock);
+
+ ret = v4l2_subdev_disable_streams(formatter->csi_sd, formatter->remote_pad,
+ sink_streams);
+ if (ret)
+ dev_err(formatter->dev, "Failed to disable streams: %d\n", ret);
+
+ csi_formatter_stop_stream(formatter, streams_mask);
+
+ formatter->enabled_streams &= ~streams_mask;
+
+ if (!formatter->enabled_streams)
+ pm_runtime_put(formatter->dev);
+
+ return ret;
+}
+
+static const struct v4l2_subdev_pad_ops formatter_subdev_pad_ops = {
+ .enum_mbus_code = formatter_subdev_enum_mbus_code,
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = formatter_subdev_set_fmt,
+ .get_frame_desc = v4l2_subdev_get_frame_desc_passthrough,
+ .set_routing = formatter_subdev_set_routing,
+ .enable_streams = formatter_subdev_enable_streams,
+ .disable_streams = formatter_subdev_disable_streams,
+};
+
+static const struct v4l2_subdev_ops formatter_subdev_ops = {
+ .pad = &formatter_subdev_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops formatter_internal_ops = {
+ .init_state = formatter_subdev_init_state,
+};
+
+/* -----------------------------------------------------------------------------
+ * Media entity operations
+ */
+
+static const struct media_entity_operations formatter_entity_ops = {
+ .link_validate = v4l2_subdev_link_validate,
+ .get_fwnode_pad = v4l2_subdev_get_fwnode_pad_1_to_1,
+};
+
+static int csi_formatter_subdev_init(struct csi_formatter *formatter)
+{
+ struct v4l2_subdev *sd = &formatter->sd;
+ int ret;
+
+ v4l2_subdev_init(sd, &formatter_subdev_ops);
+
+ snprintf(sd->name, sizeof(sd->name), "%s", dev_name(formatter->dev));
+ sd->internal_ops = &formatter_internal_ops;
+
+ sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+ V4L2_SUBDEV_FL_HAS_EVENTS |
+ V4L2_SUBDEV_FL_STREAMS;
+ sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ sd->entity.ops = &formatter_entity_ops;
+ sd->dev = formatter->dev;
+
+ formatter->pads[CSI_FORMATTER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ formatter->pads[CSI_FORMATTER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&sd->entity, CSI_FORMATTER_PAD_NUM,
+ formatter->pads);
+ if (ret) {
+ dev_err(formatter->dev, "Failed to init pads\n");
+ return ret;
+ }
+
+ ret = v4l2_subdev_init_finalize(sd);
+ if (ret)
+ media_entity_cleanup(&sd->entity);
+
+ return ret;
+}
+
+static inline struct csi_formatter *
+notifier_to_formatter(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct csi_formatter, notifier);
+}
+
+static int csi_formatter_notify_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_connection *asc)
+{
+ const unsigned int link_flags = MEDIA_LNK_FL_IMMUTABLE
+ | MEDIA_LNK_FL_ENABLED;
+ struct csi_formatter *formatter = notifier_to_formatter(notifier);
+ struct v4l2_subdev *sdev = &formatter->sd;
+ struct media_pad *sink = &sdev->entity.pads[CSI_FORMATTER_PAD_SINK];
+ struct media_pad *remote_pad;
+ int ret;
+
+ formatter->csi_sd = sd;
+
+ dev_dbg(formatter->dev, "Bound subdev: %s pad\n", sd->name);
+
+ ret = v4l2_create_fwnode_links_to_pad(sd, sink, link_flags);
+ if (ret < 0)
+ return ret;
+
+ remote_pad = media_pad_remote_pad_first(sink);
+ if (!remote_pad) {
+ dev_err(formatter->dev, "Pipe not setup correctly\n");
+ return -EPIPE;
+ }
+ formatter->remote_pad = remote_pad->index;
+
+ return 0;
+}
+
+static const struct v4l2_async_notifier_operations formatter_notify_ops = {
+ .bound = csi_formatter_notify_bound,
+};
+
+static int csi_formatter_async_register(struct csi_formatter *formatter)
+{
+ struct device *dev = formatter->dev;
+ struct v4l2_async_connection *asc;
+ int ret;
+
+ struct fwnode_handle *ep __free(fwnode_handle) =
+ fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0,
+ FWNODE_GRAPH_ENDPOINT_NEXT);
+ if (!ep)
+ return -ENOTCONN;
+
+ v4l2_async_subdev_nf_init(&formatter->notifier, &formatter->sd);
+
+ asc = v4l2_async_nf_add_fwnode_remote(&formatter->notifier, ep,
+ struct v4l2_async_connection);
+ if (IS_ERR(asc)) {
+ ret = PTR_ERR(asc);
+ goto err_cleanup_notifier;
+ }
+
+ formatter->notifier.ops = &formatter_notify_ops;
+
+ ret = v4l2_async_nf_register(&formatter->notifier);
+ if (ret)
+ goto err_cleanup_notifier;
+
+ ret = v4l2_async_register_subdev(&formatter->sd);
+ if (ret)
+ goto err_unregister_notifier;
+
+ return 0;
+
+err_unregister_notifier:
+ v4l2_async_nf_unregister(&formatter->notifier);
+err_cleanup_notifier:
+ v4l2_async_nf_cleanup(&formatter->notifier);
+ return ret;
+}
+
+static void csi_formatter_async_unregister(struct csi_formatter *formatter)
+{
+ v4l2_async_unregister_subdev(&formatter->sd);
+ v4l2_async_nf_unregister(&formatter->notifier);
+ v4l2_async_nf_cleanup(&formatter->notifier);
+}
+
+/* -----------------------------------------------------------------------------
+ * Suspend/resume
+ */
+
+static int csi_formatter_runtime_suspend(struct device *dev)
+{
+ struct v4l2_subdev *sd = dev_get_drvdata(dev);
+ struct csi_formatter *formatter = sd_to_formatter(sd);
+
+ clk_disable_unprepare(formatter->clk);
+
+ return 0;
+}
+
+static int csi_formatter_runtime_resume(struct device *dev)
+{
+ struct v4l2_subdev *sd = dev_get_drvdata(dev);
+ struct csi_formatter *formatter = sd_to_formatter(sd);
+
+ return clk_prepare_enable(formatter->clk);
+}
+
+static DEFINE_RUNTIME_DEV_PM_OPS(csi_formatter_pm_ops,
+ csi_formatter_runtime_suspend,
+ csi_formatter_runtime_resume, NULL);
+
+static int csi_formatter_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct csi_formatter *formatter;
+ u32 val;
+ int ret;
+
+ formatter = devm_kzalloc(dev, sizeof(*formatter), GFP_KERNEL);
+ if (!formatter)
+ return -ENOMEM;
+
+ formatter->dev = dev;
+
+ ret = devm_mutex_init(dev, &formatter->lock);
+ if (ret)
+ return ret;
+
+ formatter->regs = syscon_node_to_regmap(dev->parent->of_node);
+ if (IS_ERR(formatter->regs))
+ return dev_err_probe(dev, PTR_ERR(formatter->regs),
+ "Failed to get csi formatter regmap\n");
+
+ ret = of_property_read_u32(dev->of_node, "reg", &val);
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "Failed to get csi formatter reg property\n");
+
+ formatter->reg_offset = val;
+
+ formatter->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(formatter->clk))
+ return dev_err_probe(dev, PTR_ERR(formatter->clk),
+ "Failed to get pixel clock\n");
+
+ ret = csi_formatter_subdev_init(formatter);
+ if (ret < 0)
+ return dev_err_probe(dev, ret, "formatter subdev init fail\n");
+
+ /* Initialize formatter pixel format */
+ formatter->fmt = &formats[0];
+
+ platform_set_drvdata(pdev, &formatter->sd);
+
+ /* Enable runtime PM. */
+ ret = devm_pm_runtime_enable(dev);
+ if (ret)
+ goto err_cleanup_subdev;
+
+ ret = csi_formatter_async_register(formatter);
+ if (ret < 0) {
+ dev_err_probe(dev, ret, "Failed to register async subdevice\n");
+ goto err_cleanup_subdev;
+ }
+
+ return 0;
+
+err_cleanup_subdev:
+ v4l2_subdev_cleanup(&formatter->sd);
+ media_entity_cleanup(&formatter->sd.entity);
+ return ret;
+}
+
+static void csi_formatter_remove(struct platform_device *pdev)
+{
+ struct v4l2_subdev *sd = platform_get_drvdata(pdev);
+ struct csi_formatter *formatter = sd_to_formatter(sd);
+
+ csi_formatter_async_unregister(formatter);
+
+ v4l2_subdev_cleanup(&formatter->sd);
+ media_entity_cleanup(&formatter->sd.entity);
+}
+
+static const struct of_device_id csi_formatter_of_match[] = {
+ { .compatible = "fsl,imx95-csi-formatter" },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, csi_formatter_of_match);
+
+static struct platform_driver csi_formatter_device_driver = {
+ .driver = {
+ .name = "csi-pixel-formatter",
+ .of_match_table = csi_formatter_of_match,
+ .pm = pm_ptr(&csi_formatter_pm_ops),
+ },
+ .probe = csi_formatter_probe,
+ .remove = csi_formatter_remove,
+};
+
+module_platform_driver(csi_formatter_device_driver);
+
+MODULE_AUTHOR("NXP Semiconductor, Inc.");
+MODULE_DESCRIPTION("NXP i.MX95 CSI Pixel Formatter driver");
+MODULE_LICENSE("GPL");
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver
2026-05-18 2:19 ` [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver Guoniu Zhou
@ 2026-05-18 2:46 ` sashiko-bot
2026-05-20 22:29 ` Laurent Pinchart
1 sibling, 0 replies; 6+ messages in thread
From: sashiko-bot @ 2026-05-18 2:46 UTC (permalink / raw)
To: Guoniu Zhou; +Cc: robh, devicetree, imx, Frank.Li, conor+dt
Thank you for your contribution! Sashiko AI review found 11 potential issue(s) to consider:
- [High] Unchecked return value from v4l2_subdev_state_get_format() causes a kernel panic on invalid stream IDs.
- [High] Missing .unbind callback leaves a dangling pointer to the upstream subdevice, leading to Use-After-Free.
- [High] Global format caching breaks multiplexed multi-stream configurations.
- [High] Register overwrite on shared virtual channels disables concurrent streams.
- [High] Querying upstream frame descriptor after disabling it leads to resource leaks.
- [High] Unprotected format modification and missing streaming check in formatter_subdev_set_fmt().
- [High] Double pm_runtime_put() on stream disable failure leads to unbalanced PM usage counter.
- [Medium] Mixing of goto-based error handling and scope-based cleanup helpers violates subsystem guidelines.
- [Medium] Uninitialized stack variable `source_fd` in get_vc() risks out-of-bounds memory reads.
- [Medium] Stream bitmasks > 7 are silently ignored during start/stop.
- [Medium] formatter_subdev_set_fmt() breaks unrouted streams by improperly returning -EINVAL.
--
commit 4d8a1be6e9f473bbb1ccaf942fe57cfe4870bd41
Author: Guoniu Zhou <guoniu.zhou@nxp.com>
media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver
The CSI pixel formatter is a module found on i.MX95 used to reformat
packet info, pixel and non-pixel data from CSI-2 host controller to
match Pixel Link(PL) definition. Add data formatting support.
> diff --git a/drivers/media/platform/nxp/imx95-csi-formatter.c b/drivers/media/platform/nxp/imx95-csi-formatter.c
> new file mode 100644
> index 0000000000000..45240b7422b47
> --- /dev/null
> +++ b/drivers/media/platform/nxp/imx95-csi-formatter.c
> @@ -0,0 +1,776 @@
[ ... ]
> +static int formatter_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->pad == CSI_FORMATTER_PAD_SOURCE) {
> + struct v4l2_mbus_framefmt *fmt;
> +
> + if (code->index > 0)
> + return -EINVAL;
> +
> + fmt = v4l2_subdev_state_get_format(sd_state, code->pad,
> + code->stream);
> + code->code = fmt->code;
Is it possible for v4l2_subdev_state_get_format() to return NULL here?
If userspace passes an unrouted stream ID, this might result in a NULL
pointer dereference.
> + return 0;
> + }
[ ... ]
> +static int formatter_subdev_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *sdformat)
> +{
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> + struct formatter_pix_format const *format;
> + struct v4l2_mbus_framefmt *fmt;
> +
> + if (sdformat->pad == CSI_FORMATTER_PAD_SOURCE)
> + return v4l2_subdev_get_fmt(sd, sd_state, sdformat);
> +
> + /*
> + * Validate the media bus code and clamp and align the size.
> + *
> + * The total number of bits per line must be a multiple of 8. We thus
> + * need to align the width for formats that are not multiples of 8
> + * bits.
> + */
> + format = find_csi_format(sdformat->format.code);
> + if (!format)
> + format = &formats[0];
> +
> + v4l_bound_align_image(&sdformat->format.width, 1, 0xffff, 2,
> + &sdformat->format.height, 1, 0xffff, 0, 0);
> +
> + fmt = v4l2_subdev_state_get_format(sd_state, sdformat->pad,
> + sdformat->stream);
> + *fmt = sdformat->format;
Can v4l2_subdev_state_get_format() return NULL for an unrouted stream here
as well?
> +
> + /* Set default code if user set an invalid value */
> + fmt->code = format->code;
> +
> + /* Propagate the format from sink stream to source stream */
> + fmt = v4l2_subdev_state_get_opposite_stream_format(sd_state, sdformat->pad,
> + sdformat->stream);
> + if (!fmt)
> + return -EINVAL;
Since the V4L2 Streams API permits setting formats on unrouted streams,
should this skip propagation instead of returning -EINVAL?
> +
> + *fmt = sdformat->format;
> +
> + /* Store the CSIS format descriptor for active formats. */
> + if (sdformat->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> + formatter->fmt = format;
Does this modification of formatter->fmt require formatter->lock protection
to avoid a race with enable_streams()?
Also, could userspace modify the format concurrently with an active stream
since there is no media_entity_is_streaming() check here?
> +
> + return 0;
> +}
[ ... ]
> +static int get_vc(struct csi_formatter *formatter, unsigned int stream)
> +{
> + struct v4l2_mbus_frame_desc source_fd;
> + struct v4l2_mbus_frame_desc_entry *entry = NULL;
> + unsigned int i;
> + int vc;
> + int ret;
> +
> + /*
> + * Return virtual channel 0 as default value when remote subdev
> + * don't implement .get_frame_desc subdev callback
> + */
> + ret = v4l2_subdev_call(formatter->csi_sd, pad, get_frame_desc,
> + formatter->remote_pad, &source_fd);
Could source_fd be left uninitialized if the upstream subdev returns 0
but fails to fully populate it? This might lead to an out-of-bounds read
when iterating over source_fd.num_entries.
> + if (ret < 0)
> + return (ret == -ENOIOCTLCMD) ? 0 : ret;
[ ... ]
> +static void csi_formatter_stop_stream(struct csi_formatter *formatter,
> + u64 stream_mask)
> +{
> + unsigned int i;
> + int ret;
> + int vc;
> +
> + for (i = 0; i < V4L2_FRAME_DESC_ENTRY_MAX; ++i) {
> + if (!(stream_mask & BIT(i)))
> + continue;
Are stream IDs >= V4L2_FRAME_DESC_ENTRY_MAX supported? If the mask
contains bits higher than this limit, they seem to be silently ignored.
> +
> + ret = get_vc(formatter, i);
> + if (WARN_ON(ret < 0)) {
> + dev_err(formatter->dev,
> + "Failed to get VC for stream %d: %d\n", i, ret);
> + continue;
> + }
> +
> + vc = ret;
> +
> + formatter_write(formatter, CSI_VC_PIXEL_DATA_TYPE(vc), 0);
Does overwriting this register completely break other interleaved streams
sharing the same Virtual Channel? A read-modify-write might be necessary
here since the register uses a bitmask.
> + }
> +}
> +
> +static int csi_formatter_start_stream(struct csi_formatter *formatter,
> + u64 stream_mask)
> +{
> + const struct formatter_pix_format *fmt = formatter->fmt;
> + u64 configured_streams = 0;
> + unsigned int i;
> + u32 val;
> + int ret;
> + int vc;
> +
> + for (i = 0; i < V4L2_FRAME_DESC_ENTRY_MAX; ++i) {
> + if (!(stream_mask & BIT(i)))
> + continue;
> +
> + val = BIT(get_index_by_dt(fmt->data_type));
Will this break if userspace configures a multi-stream pipeline with
different formats per stream, since it applies the same global data type
to all enabled streams?
> +
> + ret = get_vc(formatter, i);
> + if (ret < 0)
> + goto err_cleanup;
> +
> + vc = ret;
> +
> + formatter_write(formatter, CSI_VC_PIXEL_DATA_TYPE(vc), val);
> + configured_streams |= BIT(i);
> + }
[ ... ]
> +static int formatter_subdev_enable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + u32 pad, u64 streams_mask)
> +{
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> + struct device *dev = formatter->dev;
> + u64 sink_streams;
> + int ret;
> +
> + sink_streams = v4l2_subdev_state_xlate_streams(state,
> + CSI_FORMATTER_PAD_SOURCE,
> + CSI_FORMATTER_PAD_SINK,
> + &streams_mask);
> + if (!sink_streams || !streams_mask)
> + return -EINVAL;
> +
> + dev_dbg(dev, "Enable streams: pad=%u sink=0x%llx source=0x%llx\n",
> + formatter->remote_pad, sink_streams, streams_mask);
> +
> + if (!formatter->csi_sd) {
> + dev_err(dev, "CSI controller not linked with formatter\n");
> + return -EPIPE;
> + }
> +
> + guard(mutex)(&formatter->lock);
> +
> + if (!formatter->enabled_streams) {
> + ret = pm_runtime_resume_and_get(formatter->dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to resume runtime PM: %d\n", ret);
> + return ret;
> + }
> + }
> +
> + ret = csi_formatter_start_stream(formatter, streams_mask);
> + if (ret)
> + goto err_runtime_put;
Does mixing guard() with goto labels create confusing ownership semantics
here?
> +
> + ret = v4l2_subdev_enable_streams(formatter->csi_sd,
> + formatter->remote_pad,
> + sink_streams);
> + if (ret)
> + goto err_stop_stream;
[ ... ]
> +static int formatter_subdev_disable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + u32 pad, u64 streams_mask)
> +{
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> + u64 sink_streams;
> + int ret;
> +
> + sink_streams = v4l2_subdev_state_xlate_streams(state,
> + CSI_FORMATTER_PAD_SOURCE,
> + CSI_FORMATTER_PAD_SINK,
> + &streams_mask);
> + if (!sink_streams || !streams_mask)
> + return -EINVAL;
> +
> + guard(mutex)(&formatter->lock);
> +
> + ret = v4l2_subdev_disable_streams(formatter->csi_sd, formatter->remote_pad,
> + sink_streams);
> + if (ret)
> + dev_err(formatter->dev, "Failed to disable streams: %d\n", ret);
> +
> + csi_formatter_stop_stream(formatter, streams_mask);
Since the upstream subdevice is disabled first, will its get_frame_desc
callback fail when csi_formatter_stop_stream() calls get_vc()? This could
cause the driver to skip disabling the local hardware stream.
> +
> + formatter->enabled_streams &= ~streams_mask;
> +
> + if (!formatter->enabled_streams)
> + pm_runtime_put(formatter->dev);
> +
> + return ret;
If v4l2_subdev_disable_streams() fails, the V4L2 core retains the stream
as enabled, but the driver clears enabled_streams and calls pm_runtime_put().
Would a subsequent retry to disable the stream cause a second
pm_runtime_put() without a matching get, unbalancing the usage counter?
> +}
[ ... ]
> +static const struct v4l2_async_notifier_operations formatter_notify_ops = {
> + .bound = csi_formatter_notify_bound,
> +};
Is an .unbind callback missing here? If the upstream subdevice is unbound,
formatter->csi_sd would be left dangling, potentially leading to a
use-after-free during subsequent V4L2 operations.
> +
> +static int csi_formatter_async_register(struct csi_formatter *formatter)
> +{
> + struct device *dev = formatter->dev;
> + struct v4l2_async_connection *asc;
> + int ret;
> +
> + struct fwnode_handle *ep __free(fwnode_handle) =
> + fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0,
> + FWNODE_GRAPH_ENDPOINT_NEXT);
> + if (!ep)
> + return -ENOTCONN;
> +
> + v4l2_async_subdev_nf_init(&formatter->notifier, &formatter->sd);
> +
> + asc = v4l2_async_nf_add_fwnode_remote(&formatter->notifier, ep,
> + struct v4l2_async_connection);
> + if (IS_ERR(asc)) {
> + ret = PTR_ERR(asc);
> + goto err_cleanup_notifier;
This mixes __free() and goto-based error handling. Could this lead to
unexpected resource leaks or ownership issues?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260518-csi_formatter-v7-0-562b750557e3@oss.nxp.com?part=2
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v7 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings
2026-05-18 2:19 ` [PATCH v7 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings Guoniu Zhou
@ 2026-05-20 21:33 ` Laurent Pinchart
0 siblings, 0 replies; 6+ messages in thread
From: Laurent Pinchart @ 2026-05-20 21:33 UTC (permalink / raw)
To: Guoniu Zhou
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Frank Li, imx, linux-media, devicetree,
linux-arm-kernel, linux-kernel, Guoniu Zhou, Krzysztof Kozlowski
Hi Guoniu,
Thank you for the patch.
On Mon, May 18, 2026 at 10:19:46AM +0800, Guoniu Zhou wrote:
> From: Guoniu Zhou <guoniu.zhou@nxp.com>
>
> The i.MX95 CSI pixel formatting module uses packet info, pixel and
> non-pixel data from the CSI-2 host controller and reformat them to
> match Pixel Link(PL) definition.
>
> Reviewed-by: Frank Li <Frank.Li@nxp.com>
> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
> Signed-off-by: Guoniu Zhou <guoniu.zhou@nxp.com>
> ---
> Changes in v7:
> - Change compatible to imx95-csi-formatter as IP is i.MX95 specific per Marco's suggestion
> Link: https://lore.kernel.org/linux-media/20260511-csi_formatter-v6-0-01028e312e2b@oss.nxp.com/T/#mcd135b3de179b3cb69daa1fd6e0e8e27c85b3332
> ---
> .../bindings/media/fsl,imx95-csi-formatter.yaml | 87 ++++++++++++++++++++++
> 1 file changed, 87 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml b/Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml
> new file mode 100644
> index 000000000000..28adea06c494
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml
> @@ -0,0 +1,87 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/fsl,imx95-csi-formatter.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: i.MX95 CSI Pixel Formatter
> +
> +maintainers:
> + - Guoniu Zhou <guoniu.zhou@nxp.com>
> +
> +description:
> + The CSI pixel formatting module found on i.MX95 uses packet info, pixel
> + and non-pixel data from the CSI-2 host controller and reformat them to
> + match Pixel Link(PL) definition.
> +
> +properties:
> + compatible:
> + const: fsl,imx95-csi-formatter
> +
> + reg:
> + maxItems: 1
> +
> + clocks:
> + maxItems: 1
> +
> + power-domains:
> + maxItems: 1
> +
> + ports:
> + $ref: /schemas/graph.yaml#/properties/ports
> +
> + properties:
> + port@0:
> + $ref: /schemas/graph.yaml#/$defs/port-base
> + unevaluatedProperties: false
> + description: MIPI CSI-2 RX IDI interface
> +
> + properties:
> + endpoint:
> + $ref: video-interfaces.yaml#
> + unevaluatedProperties: false
What properties defined in video-interfaces.yaml do you expect ? The
example below does not use any, and the driver in 2/2 doesn't appear to
parse any either.
> +
> + port@1:
> + $ref: /schemas/graph.yaml#/properties/port
> + description: Pixel Link Interface
> +
> +required:
> + - compatible
> + - reg
> + - clocks
> + - power-domains
> + - ports
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/clock/nxp,imx95-clock.h>
> +
> + formatter@20 {
> + compatible = "fsl,imx95-csi-formatter";
> + reg = <0x20 0x100>;
> + clocks = <&cameramix_csr IMX95_CLK_CAMBLK_CSI2_FOR0>;
> + power-domains = <&scmi_devpd 3>;
> +
> + ports {
> + #address-cells = <1>;
> + #size-cells = <0>;
> +
> + port@0 {
> + reg = <0>;
> +
> + endpoint {
> + remote-endpoint = <&mipi_csi_0_out>;
> + };
> + };
> +
> + port@1 {
> + reg = <1>;
> +
> + endpoint {
> + remote-endpoint = <&isi_in_2>;
> + };
> + };
> + };
> + };
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver
2026-05-18 2:19 ` [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver Guoniu Zhou
2026-05-18 2:46 ` sashiko-bot
@ 2026-05-20 22:29 ` Laurent Pinchart
1 sibling, 0 replies; 6+ messages in thread
From: Laurent Pinchart @ 2026-05-20 22:29 UTC (permalink / raw)
To: Guoniu Zhou
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Shawn Guo, Sascha Hauer, Pengutronix Kernel Team,
Fabio Estevam, Frank Li, imx, linux-media, devicetree,
linux-arm-kernel, linux-kernel, Guoniu Zhou
Hi Guoniu,
Thank you for the patch.
On Mon, May 18, 2026 at 10:19:47AM +0800, Guoniu Zhou wrote:
> From: Guoniu Zhou <guoniu.zhou@nxp.com>
>
> The CSI pixel formatter is a module found on i.MX95 used to reformat
> packet info, pixel and non-pixel data from CSI-2 host controller to
> match Pixel Link(PL) definition.
>
> Add data formatting support.
>
> Reviewed-by: Frank Li <Frank.Li@nxp.com>
> Signed-off-by: Guoniu Zhou <guoniu.zhou@nxp.com>
> ---
> Changes in v7:
> - Update references from imx9 to imx95 for consistency with dt-bindings
> - Enable PM runtime before async registration
>
> Changes in v6:
> - Remove unused header includes
> - Unify macro naming: VCx/VCX -> VC and parameter x -> vc
> - Remove unused format field from csi_formatter struct
> - Use compact initialization for formats array
> - Make find_csi_format() return NULL instead of default format
> - Use unsigned int for array index in find_csi_format()
> - Add err_ prefix to error handling labels
> - Add v4l2_subdev_cleanup() and reorder cleanup sequence
> - Update enable_streams debug output format
> - Rename VC_MAX to VC_NUM and fix boundary check
> - Update CSI formatter Kconfig description
> - Use v4l2_subdev_get_frame_desc_passthrough() helper
> - Fix error paths in async registration and probe
> - Add mutex to protect enabled_streams
> - Switch to devm_pm_runtime_enable()
> - Remove redundant num_routes check in set_routing
> - Optimize get_index_by_dt() and add warning for unsupported type
> - csi_formatter_start/stop_stream: Process all streams in mask
> ---
> MAINTAINERS | 8 +
> drivers/media/platform/nxp/Kconfig | 14 +
> drivers/media/platform/nxp/Makefile | 1 +
> drivers/media/platform/nxp/imx95-csi-formatter.c | 776 +++++++++++++++++++++++
> 4 files changed, 799 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 81d53481d3f7..837e14fe7a12 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -19265,6 +19265,14 @@ S: Maintained
> F: Documentation/devicetree/bindings/media/nxp,imx8-jpeg.yaml
> F: drivers/media/platform/nxp/imx-jpeg
>
> +NXP i.MX 95 CSI PIXEL FORMATTER V4L2 DRIVER
> +M: Guoniu Zhou <guoniu.zhou@nxp.com>
> +L: imx@lists.linux.dev
> +L: linux-media@vger.kernel.org
> +S: Maintained
> +F: Documentation/devicetree/bindings/media/fsl,imx95-csi-formatter.yaml
> +F: drivers/media/platform/nxp/imx95-csi-formatter.c
> +
> NXP i.MX CLOCK DRIVERS
> M: Abel Vesa <abelvesa@kernel.org>
> R: Peng Fan <peng.fan@nxp.com>
> diff --git a/drivers/media/platform/nxp/Kconfig b/drivers/media/platform/nxp/Kconfig
> index 40e3436669e2..82caae94a728 100644
> --- a/drivers/media/platform/nxp/Kconfig
> +++ b/drivers/media/platform/nxp/Kconfig
> @@ -28,6 +28,20 @@ config VIDEO_IMX8MQ_MIPI_CSI2
> Video4Linux2 driver for the MIPI CSI-2 receiver found on the i.MX8MQ
> SoC.
>
> +config VIDEO_IMX95_CSI_FORMATTER
> + tristate "NXP i.MX95 CSI Pixel Formatter driver"
> + depends on ARCH_MXC || COMPILE_TEST
> + depends on VIDEO_DEV
> + select MEDIA_CONTROLLER
> + select V4L2_FWNODE
> + select VIDEO_V4L2_SUBDEV_API
> + help
> + This driver provides support for the CSI Pixel Formatter found on
> + i.MX95 series SoCs. This module unpacks the pixels received from the
> + CSI-2 interface and reformats them to meet pixel link requirements.
> +
> + Say Y here to enable CSI Pixel Formater module for i.MX95 SoC.
> +
> config VIDEO_IMX_MIPI_CSIS
> tristate "NXP MIPI CSI-2 CSIS receiver found on i.MX7 and i.MX8 models"
> depends on ARCH_MXC || COMPILE_TEST
> diff --git a/drivers/media/platform/nxp/Makefile b/drivers/media/platform/nxp/Makefile
> index 4d90eb713652..6410115d870e 100644
> --- a/drivers/media/platform/nxp/Makefile
> +++ b/drivers/media/platform/nxp/Makefile
> @@ -6,6 +6,7 @@ obj-y += imx8-isi/
>
> obj-$(CONFIG_VIDEO_IMX7_CSI) += imx7-media-csi.o
> obj-$(CONFIG_VIDEO_IMX8MQ_MIPI_CSI2) += imx8mq-mipi-csi2.o
> +obj-$(CONFIG_VIDEO_IMX95_CSI_FORMATTER) += imx95-csi-formatter.o
> obj-$(CONFIG_VIDEO_IMX_MIPI_CSIS) += imx-mipi-csis.o
> obj-$(CONFIG_VIDEO_IMX_PXP) += imx-pxp.o
> obj-$(CONFIG_VIDEO_MX2_EMMAPRP) += mx2_emmaprp.o
> diff --git a/drivers/media/platform/nxp/imx95-csi-formatter.c b/drivers/media/platform/nxp/imx95-csi-formatter.c
> new file mode 100644
> index 000000000000..45240b7422b4
> --- /dev/null
> +++ b/drivers/media/platform/nxp/imx95-csi-formatter.c
> @@ -0,0 +1,776 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright 2025 NXP
> + */
> +
> +#include <linux/bits.h>
> +#include <linux/clk.h>
> +#include <linux/mfd/syscon.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/regmap.h>
> +
> +#include <media/mipi-csi2.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-mc.h>
> +#include <media/v4l2-subdev.h>
> +
> +/* CSI Pixel Formatter registers map */
> +
> +#define CSI_VC_INTERLACED_LINE_CNT(vc) (0x00 + (vc) * 0x04)
> +#define INTERLACED_ODD_LINE_CNT_SET(x) FIELD_PREP(GENMASK(13, 0), (x))
> +#define INTERLACED_EVEN_LINE_CNT_SET(x) FIELD_PREP(GENMASK(29, 16), (x))
> +
> +#define CSI_VC_INTERLACED_CTRL 0x20
> +
> +#define CSI_VC_INTERLACED_ERR 0x24
> +#define CSI_VC_ERR_MASK GENMASK(7, 0)
> +#define CSI_VC_ERR(vc) BIT((vc))
> +
> +#define CSI_VC_YUV420_FIRST_LINE_EVEN 0x28
> +#define YUV420_FIRST_LINE_EVEN(vc) BIT((vc))
> +
> +#define CSI_RAW32_CTRL 0x30
> +#define CSI_VC_RAW32_MODE(vc) BIT((vc))
> +#define CSI_VC_RAW32_SWAP_MODE(vc) BIT((vc) + 8)
> +
> +#define STREAM_FENCING_CTRL 0x34
> +#define CSI_VC_STREAM_FENCING(vc) BIT((vc))
> +#define CSI_VC_STREAM_FENCING_RST(vc) BIT((vc) + 8)
> +
> +#define STREAM_FENCING_STS 0x38
> +#define STREAM_FENCING_STS_MASK GENMASK(7, 0)
> +
> +#define CSI_VC_NON_PIXEL_DATA_TYPE(vc) (0x40 + (vc) * 0x04)
> +
> +#define CSI_VC_PIXEL_DATA_CTRL(vc) (0x60 + (vc) * 0x04)
> +#define NEW_VC(vc) FIELD_PREP(GENMASK(3, 1), vc)
> +#define REROUTE_VC_ENABLE BIT(0)
> +
> +#define CSI_VC_ROUTE_PIXEL_DATA_TYPE(vc) (0x80 + (vc) * 0x04)
> +
> +#define CSI_VC_NON_PIXEL_DATA_CTRL(vc) (0xa0 + (vc) * 0x04)
> +
> +#define CSI_VC_PIXEL_DATA_TYPE(vc) (0xc0 + (vc) * 0x04)
> +
> +#define CSI_VC_PIXEL_DATA_TYPE_ERR(vc) (0xe0 + (vc) * 0x04)
> +
> +#define CSI_FORMATTER_PAD_SINK 0
> +#define CSI_FORMATTER_PAD_SOURCE 1
> +#define CSI_FORMATTER_PAD_NUM 2
> +
> +#define CSI_FORMATTER_VC_NUM 8 /* Number of virtual channels */
> +
> +struct formatter_pix_format {
> + u32 code;
> + u32 data_type;
> +};
> +
> +struct csi_formatter {
> + struct device *dev;
> + struct regmap *regs;
> + struct clk *clk;
> +
> + struct v4l2_subdev sd;
> + struct v4l2_subdev *csi_sd;
> + struct v4l2_async_notifier notifier;
> + struct media_pad pads[CSI_FORMATTER_PAD_NUM];
> + const struct formatter_pix_format *fmt;
State shouldn't be stored in csi_formatter. You can drop this field, and
look up the formatter_pix_format in csi_formatter_start_stream() using
the media bus code stored in the active state.
> +
> + u32 remote_pad;
> + u32 reg_offset;
> +
> + /* Protects enabled_streams */
> + struct mutex lock;
> + u64 enabled_streams;
> +};
> +
> +struct dt_index {
> + u8 dtype;
> + u8 index;
> +};
> +
> +/*
> + * The index should correspond to the bit index define in register
> + * which enable the data type of pixel data transported by Formatter.
> + */
> +static const struct dt_index formatter_dt_to_index_map[] = {
> + { .dtype = MIPI_CSI2_DT_YUV420_8B, .index = 0 },
> + { .dtype = MIPI_CSI2_DT_YUV420_8B_LEGACY, .index = 2 },
> + { .dtype = MIPI_CSI2_DT_YUV422_8B, .index = 6 },
> + { .dtype = MIPI_CSI2_DT_RGB444, .index = 8 },
> + { .dtype = MIPI_CSI2_DT_RGB555, .index = 9 },
> + { .dtype = MIPI_CSI2_DT_RGB565, .index = 10 },
> + { .dtype = MIPI_CSI2_DT_RGB666, .index = 11 },
> + { .dtype = MIPI_CSI2_DT_RGB888, .index = 12 },
> + { .dtype = MIPI_CSI2_DT_RAW6, .index = 16 },
> + { .dtype = MIPI_CSI2_DT_RAW7, .index = 17 },
> + { .dtype = MIPI_CSI2_DT_RAW8, .index = 18 },
> + { .dtype = MIPI_CSI2_DT_RAW10, .index = 19 },
> + { .dtype = MIPI_CSI2_DT_RAW12, .index = 20 },
> + { .dtype = MIPI_CSI2_DT_RAW14, .index = 21 },
> + { .dtype = MIPI_CSI2_DT_RAW16, .index = 22 },
> +};
> +
> +static const struct formatter_pix_format formats[] = {
> + /* YUV formats */
> + { MEDIA_BUS_FMT_UYVY8_1X16, MIPI_CSI2_DT_YUV422_8B },
> + /* RGB formats */
> + { MEDIA_BUS_FMT_RGB565_1X16, MIPI_CSI2_DT_RGB565 },
> + { MEDIA_BUS_FMT_RGB888_1X24, MIPI_CSI2_DT_RGB888 },
> + /* RAW (Bayer and greyscale) formats */
> + { MEDIA_BUS_FMT_SBGGR8_1X8, MIPI_CSI2_DT_RAW8 },
> + { MEDIA_BUS_FMT_SGBRG8_1X8, MIPI_CSI2_DT_RAW8 },
> + { MEDIA_BUS_FMT_SGRBG8_1X8, MIPI_CSI2_DT_RAW8 },
> + { MEDIA_BUS_FMT_SRGGB8_1X8, MIPI_CSI2_DT_RAW8 },
> + { MEDIA_BUS_FMT_Y8_1X8, MIPI_CSI2_DT_RAW8 },
> + { MEDIA_BUS_FMT_SBGGR10_1X10, MIPI_CSI2_DT_RAW10 },
> + { MEDIA_BUS_FMT_SGBRG10_1X10, MIPI_CSI2_DT_RAW10 },
> + { MEDIA_BUS_FMT_SGRBG10_1X10, MIPI_CSI2_DT_RAW10 },
> + { MEDIA_BUS_FMT_SRGGB10_1X10, MIPI_CSI2_DT_RAW10 },
> + { MEDIA_BUS_FMT_Y10_1X10, MIPI_CSI2_DT_RAW10 },
> + { MEDIA_BUS_FMT_SBGGR12_1X12, MIPI_CSI2_DT_RAW12 },
> + { MEDIA_BUS_FMT_SGBRG12_1X12, MIPI_CSI2_DT_RAW12 },
> + { MEDIA_BUS_FMT_SGRBG12_1X12, MIPI_CSI2_DT_RAW12 },
> + { MEDIA_BUS_FMT_SRGGB12_1X12, MIPI_CSI2_DT_RAW12 },
> + { MEDIA_BUS_FMT_Y12_1X12, MIPI_CSI2_DT_RAW12 },
> + { MEDIA_BUS_FMT_SBGGR14_1X14, MIPI_CSI2_DT_RAW14 },
> + { MEDIA_BUS_FMT_SGBRG14_1X14, MIPI_CSI2_DT_RAW14 },
> + { MEDIA_BUS_FMT_SGRBG14_1X14, MIPI_CSI2_DT_RAW14 },
> + { MEDIA_BUS_FMT_SRGGB14_1X14, MIPI_CSI2_DT_RAW14 },
> + { MEDIA_BUS_FMT_SBGGR16_1X16, MIPI_CSI2_DT_RAW16 },
> + { MEDIA_BUS_FMT_SGBRG16_1X16, MIPI_CSI2_DT_RAW16 },
> + { MEDIA_BUS_FMT_SGRBG16_1X16, MIPI_CSI2_DT_RAW16 },
> + { MEDIA_BUS_FMT_SRGGB16_1X16, MIPI_CSI2_DT_RAW16 },
> +};
> +
> +static const struct v4l2_mbus_framefmt formatter_default_fmt = {
> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> + .width = 1920U,
> + .height = 1080U,
> + .field = V4L2_FIELD_NONE,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(V4L2_COLORSPACE_SMPTE170M),
> + .ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(V4L2_COLORSPACE_SMPTE170M),
> + .quantization = V4L2_QUANTIZATION_LIM_RANGE,
> +};
> +
> +static const struct formatter_pix_format *find_csi_format(u32 code)
You have a mix of function naming schemes. Some start with formatter_,
some with csi_formatter_, and some have no driver-specific prefer at
all. Could you please unify them to use the same prefix ?
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++)
> + if (code == formats[i].code)
> + return &formats[i];
> +
> + return NULL;
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * V4L2 subdev operations
> + */
> +
> +static inline struct csi_formatter *sd_to_formatter(struct v4l2_subdev *sdev)
> +{
> + return container_of(sdev, struct csi_formatter, sd);
> +}
> +
> +static int __formatter_subdev_set_routing(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + struct v4l2_subdev_krouting *routing)
> +{
> + int ret;
> +
> + ret = v4l2_subdev_routing_validate(sd, routing,
> + V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
> + if (ret)
> + return ret;
> +
> + return v4l2_subdev_set_routing_with_fmt(sd, state, routing,
> + &formatter_default_fmt);
> +}
> +
> +static int formatter_subdev_init_state(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state)
> +{
> + struct v4l2_subdev_route routes[] = {
> + {
> + .sink_pad = CSI_FORMATTER_PAD_SINK,
> + .sink_stream = 0,
> + .source_pad = CSI_FORMATTER_PAD_SOURCE,
> + .source_stream = 0,
> + .flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> + },
> + };
> +
> + struct v4l2_subdev_krouting routing = {
> + .num_routes = ARRAY_SIZE(routes),
> + .routes = routes,
> + };
> +
> + return __formatter_subdev_set_routing(sd, sd_state, &routing);
> +}
> +
> +static int formatter_subdev_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->pad == CSI_FORMATTER_PAD_SOURCE) {
> + struct v4l2_mbus_framefmt *fmt;
> +
> + if (code->index > 0)
> + return -EINVAL;
> +
> + fmt = v4l2_subdev_state_get_format(sd_state, code->pad,
> + code->stream);
> + code->code = fmt->code;
> + return 0;
> + }
> +
> + if (code->index >= ARRAY_SIZE(formats))
> + return -EINVAL;
> +
> + code->code = formats[code->index].code;
> +
> + return 0;
> +}
> +
> +static int formatter_subdev_set_fmt(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *sdformat)
> +{
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> + struct formatter_pix_format const *format;
> + struct v4l2_mbus_framefmt *fmt;
> +
> + if (sdformat->pad == CSI_FORMATTER_PAD_SOURCE)
> + return v4l2_subdev_get_fmt(sd, sd_state, sdformat);
> +
> + /*
> + * Validate the media bus code and clamp and align the size.
> + *
> + * The total number of bits per line must be a multiple of 8. We thus
> + * need to align the width for formats that are not multiples of 8
> + * bits.
Where is this done ?
> + */
> + format = find_csi_format(sdformat->format.code);
> + if (!format)
> + format = &formats[0];
> +
> + v4l_bound_align_image(&sdformat->format.width, 1, 0xffff, 2,
> + &sdformat->format.height, 1, 0xffff, 0, 0);
> +
> + fmt = v4l2_subdev_state_get_format(sd_state, sdformat->pad,
> + sdformat->stream);
> + *fmt = sdformat->format;
> +
> + /* Set default code if user set an invalid value */
> + fmt->code = format->code;
> +
> + /* Propagate the format from sink stream to source stream */
> + fmt = v4l2_subdev_state_get_opposite_stream_format(sd_state, sdformat->pad,
> + sdformat->stream);
> + if (!fmt)
> + return -EINVAL;
> +
> + *fmt = sdformat->format;
> +
> + /* Store the CSIS format descriptor for active formats. */
> + if (sdformat->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> + formatter->fmt = format;
> +
> + return 0;
> +}
> +
> +static int formatter_subdev_set_routing(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + enum v4l2_subdev_format_whence which,
> + struct v4l2_subdev_krouting *routing)
> +{
> + if (which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> + media_entity_is_streaming(&sd->entity))
> + return -EBUSY;
> +
> + return __formatter_subdev_set_routing(sd, state, routing);
> +}
> +
> +static inline void formatter_write(struct csi_formatter *formatter,
> + unsigned int reg, unsigned int value)
> +{
> + u32 offset = formatter->reg_offset;
> +
> + regmap_write(formatter->regs, reg + offset, value);
> +}
> +
> +static u8 get_index_by_dt(u8 data_type)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formatter_dt_to_index_map); ++i) {
> + const struct dt_index *entry = &formatter_dt_to_index_map[i];
> +
> + if (data_type == entry->dtype)
> + return entry->index;
> + }
> +
> + pr_warn_once("Unsupported data type 0x%x, using default\n", data_type);
> +
> + return formatter_dt_to_index_map[0].index;
> +}
> +
> +static int get_vc(struct csi_formatter *formatter, unsigned int stream)
> +{
> + struct v4l2_mbus_frame_desc source_fd;
> + struct v4l2_mbus_frame_desc_entry *entry = NULL;
> + unsigned int i;
> + int vc;
> + int ret;
> +
> + /*
> + * Return virtual channel 0 as default value when remote subdev
> + * don't implement .get_frame_desc subdev callback
> + */
> + ret = v4l2_subdev_call(formatter->csi_sd, pad, get_frame_desc,
> + formatter->remote_pad, &source_fd);
> + if (ret < 0)
> + return (ret == -ENOIOCTLCMD) ? 0 : ret;
> +
> + for (i = 0; i < source_fd.num_entries; ++i) {
> + if (source_fd.entry[i].stream == stream) {
> + entry = &source_fd.entry[i];
> + break;
> + }
> + }
> +
> + if (!entry) {
> + dev_err(formatter->dev,
> + "Can't find valid frame desc corresponding to stream %d\n", stream);
> + return -EPIPE;
> + }
> +
> + vc = entry->bus.csi2.vc;
> +
> + if (vc < 0 || vc >= CSI_FORMATTER_VC_NUM) {
> + dev_err(formatter->dev, "Invalid virtual channel %d\n", vc);
> + return -EINVAL;
> + }
> +
> + return vc;
> +}
> +
> +static void csi_formatter_stop_stream(struct csi_formatter *formatter,
> + u64 stream_mask)
> +{
> + unsigned int i;
> + int ret;
> + int vc;
> +
> + for (i = 0; i < V4L2_FRAME_DESC_ENTRY_MAX; ++i) {
I don't understand why you iterate over the maximum number of descriptor
entries.
> + if (!(stream_mask & BIT(i)))
> + continue;
> +
> + ret = get_vc(formatter, i);
This function calls the source's get_frame_desc operation, which will
return the exact same descriptors for every iterateion of this loop. You
should call the operation before the loop.
> + if (WARN_ON(ret < 0)) {
> + dev_err(formatter->dev,
> + "Failed to get VC for stream %d: %d\n", i, ret);
> + continue;
> + }
> +
> + vc = ret;
> +
> + formatter_write(formatter, CSI_VC_PIXEL_DATA_TYPE(vc), 0);
> + }
> +}
> +
> +static int csi_formatter_start_stream(struct csi_formatter *formatter,
> + u64 stream_mask)
> +{
> + const struct formatter_pix_format *fmt = formatter->fmt;
> + u64 configured_streams = 0;
> + unsigned int i;
> + u32 val;
> + int ret;
> + int vc;
> +
> + for (i = 0; i < V4L2_FRAME_DESC_ENTRY_MAX; ++i) {
> + if (!(stream_mask & BIT(i)))
> + continue;
> +
> + val = BIT(get_index_by_dt(fmt->data_type));
Is the formatter limited to using the same data type of all VCs ?
> +
> + ret = get_vc(formatter, i);
> + if (ret < 0)
> + goto err_cleanup;
> +
> + vc = ret;
> +
> + formatter_write(formatter, CSI_VC_PIXEL_DATA_TYPE(vc), val);
> + configured_streams |= BIT(i);
> + }
> +
> + return 0;
> +
> +err_cleanup:
> + csi_formatter_stop_stream(formatter, configured_streams);
> + return ret;
> +}
> +
> +static int formatter_subdev_enable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + u32 pad, u64 streams_mask)
> +{
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> + struct device *dev = formatter->dev;
> + u64 sink_streams;
> + int ret;
> +
> + sink_streams = v4l2_subdev_state_xlate_streams(state,
> + CSI_FORMATTER_PAD_SOURCE,
> + CSI_FORMATTER_PAD_SINK,
> + &streams_mask);
> + if (!sink_streams || !streams_mask)
> + return -EINVAL;
> +
> + dev_dbg(dev, "Enable streams: pad=%u sink=0x%llx source=0x%llx\n",
> + formatter->remote_pad, sink_streams, streams_mask);
Is this still useful, especially given that v4l2_subdev_enable_streams()
prints a debug message already ?
> +
> + if (!formatter->csi_sd) {
If you set the MUST_CONNECT flag on the sink pad this will never happen,
as the V4L2 core will refuse to start streaming without an enabled link
on the sink. This will simplify the driver.
> + dev_err(dev, "CSI controller not linked with formatter\n");
> + return -EPIPE;
> + }
> +
> + guard(mutex)(&formatter->lock);
> +
> + if (!formatter->enabled_streams) {
> + ret = pm_runtime_resume_and_get(formatter->dev);
> + if (ret < 0) {
> + dev_err(dev, "Failed to resume runtime PM: %d\n", ret);
> + return ret;
> + }
> + }
> +
> + ret = csi_formatter_start_stream(formatter, streams_mask);
> + if (ret)
> + goto err_runtime_put;
> +
> + ret = v4l2_subdev_enable_streams(formatter->csi_sd,
> + formatter->remote_pad,
> + sink_streams);
> + if (ret)
> + goto err_stop_stream;
> +
> + formatter->enabled_streams |= streams_mask;
> +
> + return 0;
> +
> +err_stop_stream:
> + csi_formatter_stop_stream(formatter, streams_mask);
> +err_runtime_put:
> + if (!formatter->enabled_streams)
> + pm_runtime_put(formatter->dev);
> + return ret;
> +}
> +
> +static int formatter_subdev_disable_streams(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *state,
> + u32 pad, u64 streams_mask)
> +{
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> + u64 sink_streams;
> + int ret;
> +
> + sink_streams = v4l2_subdev_state_xlate_streams(state,
> + CSI_FORMATTER_PAD_SOURCE,
> + CSI_FORMATTER_PAD_SINK,
> + &streams_mask);
> + if (!sink_streams || !streams_mask)
> + return -EINVAL;
> +
> + guard(mutex)(&formatter->lock);
> +
> + ret = v4l2_subdev_disable_streams(formatter->csi_sd, formatter->remote_pad,
> + sink_streams);
> + if (ret)
> + dev_err(formatter->dev, "Failed to disable streams: %d\n", ret);
> +
> + csi_formatter_stop_stream(formatter, streams_mask);
> +
> + formatter->enabled_streams &= ~streams_mask;
> +
> + if (!formatter->enabled_streams)
> + pm_runtime_put(formatter->dev);
> +
> + return ret;
> +}
> +
> +static const struct v4l2_subdev_pad_ops formatter_subdev_pad_ops = {
> + .enum_mbus_code = formatter_subdev_enum_mbus_code,
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = formatter_subdev_set_fmt,
> + .get_frame_desc = v4l2_subdev_get_frame_desc_passthrough,
> + .set_routing = formatter_subdev_set_routing,
> + .enable_streams = formatter_subdev_enable_streams,
> + .disable_streams = formatter_subdev_disable_streams,
> +};
> +
> +static const struct v4l2_subdev_ops formatter_subdev_ops = {
> + .pad = &formatter_subdev_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops formatter_internal_ops = {
> + .init_state = formatter_subdev_init_state,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * Media entity operations
> + */
> +
> +static const struct media_entity_operations formatter_entity_ops = {
> + .link_validate = v4l2_subdev_link_validate,
> + .get_fwnode_pad = v4l2_subdev_get_fwnode_pad_1_to_1,
> +};
> +
> +static int csi_formatter_subdev_init(struct csi_formatter *formatter)
> +{
> + struct v4l2_subdev *sd = &formatter->sd;
> + int ret;
> +
> + v4l2_subdev_init(sd, &formatter_subdev_ops);
> +
> + snprintf(sd->name, sizeof(sd->name), "%s", dev_name(formatter->dev));
> + sd->internal_ops = &formatter_internal_ops;
> +
> + sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
> + V4L2_SUBDEV_FL_HAS_EVENTS |
> + V4L2_SUBDEV_FL_STREAMS;
> + sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
> + sd->entity.ops = &formatter_entity_ops;
> + sd->dev = formatter->dev;
> +
> + formatter->pads[CSI_FORMATTER_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + formatter->pads[CSI_FORMATTER_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&sd->entity, CSI_FORMATTER_PAD_NUM,
> + formatter->pads);
> + if (ret) {
> + dev_err(formatter->dev, "Failed to init pads\n");
> + return ret;
> + }
> +
> + ret = v4l2_subdev_init_finalize(sd);
> + if (ret)
> + media_entity_cleanup(&sd->entity);
> +
> + return ret;
> +}
> +
> +static inline struct csi_formatter *
> +notifier_to_formatter(struct v4l2_async_notifier *n)
> +{
> + return container_of(n, struct csi_formatter, notifier);
> +}
> +
> +static int csi_formatter_notify_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *sd,
> + struct v4l2_async_connection *asc)
> +{
> + const unsigned int link_flags = MEDIA_LNK_FL_IMMUTABLE
> + | MEDIA_LNK_FL_ENABLED;
> + struct csi_formatter *formatter = notifier_to_formatter(notifier);
> + struct v4l2_subdev *sdev = &formatter->sd;
> + struct media_pad *sink = &sdev->entity.pads[CSI_FORMATTER_PAD_SINK];
> + struct media_pad *remote_pad;
> + int ret;
> +
> + formatter->csi_sd = sd;
> +
> + dev_dbg(formatter->dev, "Bound subdev: %s pad\n", sd->name);
> +
> + ret = v4l2_create_fwnode_links_to_pad(sd, sink, link_flags);
> + if (ret < 0)
> + return ret;
> +
> + remote_pad = media_pad_remote_pad_first(sink);
> + if (!remote_pad) {
> + dev_err(formatter->dev, "Pipe not setup correctly\n");
> + return -EPIPE;
> + }
> + formatter->remote_pad = remote_pad->index;
> +
> + return 0;
> +}
> +
> +static const struct v4l2_async_notifier_operations formatter_notify_ops = {
> + .bound = csi_formatter_notify_bound,
> +};
> +
> +static int csi_formatter_async_register(struct csi_formatter *formatter)
> +{
> + struct device *dev = formatter->dev;
> + struct v4l2_async_connection *asc;
> + int ret;
> +
> + struct fwnode_handle *ep __free(fwnode_handle) =
> + fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0,
> + FWNODE_GRAPH_ENDPOINT_NEXT);
> + if (!ep)
> + return -ENOTCONN;
> +
> + v4l2_async_subdev_nf_init(&formatter->notifier, &formatter->sd);
> +
> + asc = v4l2_async_nf_add_fwnode_remote(&formatter->notifier, ep,
> + struct v4l2_async_connection);
> + if (IS_ERR(asc)) {
> + ret = PTR_ERR(asc);
> + goto err_cleanup_notifier;
> + }
> +
> + formatter->notifier.ops = &formatter_notify_ops;
> +
> + ret = v4l2_async_nf_register(&formatter->notifier);
> + if (ret)
> + goto err_cleanup_notifier;
> +
> + ret = v4l2_async_register_subdev(&formatter->sd);
> + if (ret)
> + goto err_unregister_notifier;
> +
> + return 0;
> +
> +err_unregister_notifier:
> + v4l2_async_nf_unregister(&formatter->notifier);
> +err_cleanup_notifier:
> + v4l2_async_nf_cleanup(&formatter->notifier);
> + return ret;
> +}
> +
> +static void csi_formatter_async_unregister(struct csi_formatter *formatter)
> +{
> + v4l2_async_unregister_subdev(&formatter->sd);
> + v4l2_async_nf_unregister(&formatter->notifier);
> + v4l2_async_nf_cleanup(&formatter->notifier);
> +}
> +
> +/* -----------------------------------------------------------------------------
> + * Suspend/resume
> + */
> +
> +static int csi_formatter_runtime_suspend(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> +
> + clk_disable_unprepare(formatter->clk);
> +
> + return 0;
> +}
> +
> +static int csi_formatter_runtime_resume(struct device *dev)
> +{
> + struct v4l2_subdev *sd = dev_get_drvdata(dev);
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> +
> + return clk_prepare_enable(formatter->clk);
> +}
> +
> +static DEFINE_RUNTIME_DEV_PM_OPS(csi_formatter_pm_ops,
> + csi_formatter_runtime_suspend,
> + csi_formatter_runtime_resume, NULL);
> +
> +static int csi_formatter_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct csi_formatter *formatter;
> + u32 val;
> + int ret;
> +
> + formatter = devm_kzalloc(dev, sizeof(*formatter), GFP_KERNEL);
> + if (!formatter)
> + return -ENOMEM;
> +
> + formatter->dev = dev;
> +
> + ret = devm_mutex_init(dev, &formatter->lock);
> + if (ret)
> + return ret;
> +
> + formatter->regs = syscon_node_to_regmap(dev->parent->of_node);
> + if (IS_ERR(formatter->regs))
> + return dev_err_probe(dev, PTR_ERR(formatter->regs),
> + "Failed to get csi formatter regmap\n");
> +
> + ret = of_property_read_u32(dev->of_node, "reg", &val);
> + if (ret < 0)
> + return dev_err_probe(dev, ret, "Failed to get csi formatter reg property\n");
> +
> + formatter->reg_offset = val;
I'd like to see how the device is integrated in DT to validate the usage
of reg here. The example in the DT bindings only shows the formatter's
device node, I don't know what the parent device is.
> +
> + formatter->clk = devm_clk_get(dev, NULL);
> + if (IS_ERR(formatter->clk))
> + return dev_err_probe(dev, PTR_ERR(formatter->clk),
> + "Failed to get pixel clock\n");
> +
> + ret = csi_formatter_subdev_init(formatter);
> + if (ret < 0)
> + return dev_err_probe(dev, ret, "formatter subdev init fail\n");
> +
> + /* Initialize formatter pixel format */
> + formatter->fmt = &formats[0];
> +
> + platform_set_drvdata(pdev, &formatter->sd);
> +
> + /* Enable runtime PM. */
> + ret = devm_pm_runtime_enable(dev);
> + if (ret)
> + goto err_cleanup_subdev;
> +
> + ret = csi_formatter_async_register(formatter);
> + if (ret < 0) {
> + dev_err_probe(dev, ret, "Failed to register async subdevice\n");
> + goto err_cleanup_subdev;
> + }
> +
> + return 0;
> +
> +err_cleanup_subdev:
> + v4l2_subdev_cleanup(&formatter->sd);
> + media_entity_cleanup(&formatter->sd.entity);
> + return ret;
> +}
> +
> +static void csi_formatter_remove(struct platform_device *pdev)
> +{
> + struct v4l2_subdev *sd = platform_get_drvdata(pdev);
> + struct csi_formatter *formatter = sd_to_formatter(sd);
> +
> + csi_formatter_async_unregister(formatter);
> +
> + v4l2_subdev_cleanup(&formatter->sd);
> + media_entity_cleanup(&formatter->sd.entity);
> +}
> +
> +static const struct of_device_id csi_formatter_of_match[] = {
> + { .compatible = "fsl,imx95-csi-formatter" },
> + { /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, csi_formatter_of_match);
> +
> +static struct platform_driver csi_formatter_device_driver = {
> + .driver = {
> + .name = "csi-pixel-formatter",
> + .of_match_table = csi_formatter_of_match,
> + .pm = pm_ptr(&csi_formatter_pm_ops),
> + },
> + .probe = csi_formatter_probe,
> + .remove = csi_formatter_remove,
> +};
> +
> +module_platform_driver(csi_formatter_device_driver);
> +
> +MODULE_AUTHOR("NXP Semiconductor, Inc.");
> +MODULE_DESCRIPTION("NXP i.MX95 CSI Pixel Formatter driver");
> +MODULE_LICENSE("GPL");
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-05-20 22:29 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-18 2:19 [PATCH v7 0/2] media: nxp: Add CSI Pixel Formatter support Guoniu Zhou
2026-05-18 2:19 ` [PATCH v7 1/2] media: dt-bindings: Add CSI Pixel Formatter DT bindings Guoniu Zhou
2026-05-20 21:33 ` Laurent Pinchart
2026-05-18 2:19 ` [PATCH v7 2/2] media: nxp: Add i.MX95 CSI pixel formatter v4l2 driver Guoniu Zhou
2026-05-18 2:46 ` sashiko-bot
2026-05-20 22:29 ` Laurent Pinchart
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox