* [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver
@ 2025-05-24 11:49 Olivia Wen
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
` (9 more replies)
0 siblings, 10 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
Dear Maintainers,
I am pleased to submit this patch series that introduces the MediaTek
ISP7 Image System driver. This series consists of ten patches that
collectively add support for various components and functionalities of
the ISP7 architecture. The patches include device tree bindings, user
API definitions, V4L2 flow support, LARB device integration, CMDQ
support, SCP support, and image processing flows.
MediaTek's Image System (ImgSys), also known as ISP Pass2, is a
memory-to-memory hardware device designed for advanced image processing
tasks. ImgSys includes multiple hardware components such as TRAW, DIP,
PQDIP, ME, and WPE.
- **TRAW (Tile RAW)**:
Provides multiple downscaled resizers for multi-scale noise reduction
and supports RAW/RGB format conversion.
- **DIP (Digital Image Processing) and PQDIP (Picture Quality DIP)**:
Supports temporal and spatial noise reduction (TNR/SNR), edge
enhancement (EE), sharpness (SHP), image scaling, and rotation.
- **ME (Motion Estimation)**:
Supports motion estimation between consecutive frames.
- **WPE (Warping Engine)**:
Handles image warping and de-warping processes.
ImgSys utilizes Multi-Frame Combined Noise Reduction (MCNR) technology to
enhance image quality.
Below is a simplified architecture of the ImgSys driver:
User -> V4L2 Framework -> ImgSys Driver
-> SCP (It packages the hardware settings into commands)
-> ImgSys Driver
-> CMDQ (The packaged commands are sent to the hardware via GCE)
-> Hardware
Thease patches are base on linux-next/master, tag: next-20250523
Thank you for reviewing these patches.
Olivia Wen (10):
dt-bindings: media: Add MediaTek mt8188 ImgSys components
dt-bindings: media: Add MT8188 ImgSys's LARB
uapi: linux: Add MediaTek Imgsys user API
media: mediatek: isp: Add V4L2 flow support for ImgSys driver
media: mediatek: isp: Add the Imgsys LARBs device
media: mediatek: isp: Add module operations structure for ImgSys
media: mediatek: isp: Add CMDQ support for ImgSys driver
media: mediatek: isp: Add SCP support for ImgSys driver
media: mediatek: isp: Add image processing flow
media: mediatek: isp: Add normal data dump flow
.../bindings/media/mediatek,imgsys-larbs.yaml | 75 +
.../bindings/media/mediatek,imgsys.yaml | 180 ++
drivers/media/platform/mediatek/Kconfig | 1 +
drivers/media/platform/mediatek/Makefile | 1 +
drivers/media/platform/mediatek/isp/Kconfig | 23 +
.../platform/mediatek/isp/isp_7x/Makefile | 6 +
.../mediatek/isp/isp_7x/imgsys/Makefile | 25 +
.../imgsys/modules/mtk_dip_v4l2_vnode.h | 594 +++++
.../isp_7x/imgsys/modules/mtk_imgsys-dip.c | 162 ++
.../isp_7x/imgsys/modules/mtk_imgsys-dip.h | 21 +
.../isp/isp_7x/imgsys/modules/mtk_imgsys-me.c | 65 +
.../isp/isp_7x/imgsys/modules/mtk_imgsys-me.h | 30 +
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.c | 165 ++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.h | 20 +
.../isp_7x/imgsys/modules/mtk_imgsys-traw.c | 120 ++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.h | 20 +
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.c | 176 ++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.h | 20 +
.../isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h | 386 ++++
.../imgsys/modules/mtk_pqdip_v4l2_vnode.h | 149 ++
.../imgsys/modules/mtk_traw_v4l2_vnode.h | 371 ++++
.../imgsys/modules/mtk_wpe_v4l2_vnode.h | 317 +++
.../mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h | 151 ++
.../isp/isp_7x/imgsys/mtk_header_desc.h | 129 ++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h | 502 +++++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq.c | 690 ++++++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq.h | 29 +
.../isp/isp_7x/imgsys/mtk_imgsys-dev.c | 299 +++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 570 +++++
.../isp/isp_7x/imgsys/mtk_imgsys-formats.c | 139 ++
.../isp/isp_7x/imgsys/mtk_imgsys-formats.h | 72 +
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.c | 86 +
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.h | 37 +
.../isp_7x/imgsys/mtk_imgsys-module_common.h | 102 +
.../isp_7x/imgsys/mtk_imgsys-module_main.c | 187 ++
.../isp_7x/imgsys/mtk_imgsys-module_main.h | 17 +
.../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 64 +
.../isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c | 342 +++
.../isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h | 64 +
.../isp/isp_7x/imgsys/mtk_imgsys-of.c | 39 +
.../isp/isp_7x/imgsys/mtk_imgsys-of.h | 58 +
.../isp/isp_7x/imgsys/mtk_imgsys-scp.c | 349 +++
.../isp/isp_7x/imgsys/mtk_imgsys-scp.h | 194 ++
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 556 +++++
.../isp/isp_7x/imgsys/mtk_imgsys-sys.h | 50 +
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 1919 +++++++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h | 100 +
.../isp/isp_7x/imgsys/mtk_imgsys-work_buf.c | 113 +
.../isp/isp_7x/imgsys/mtk_imgsys-work_buf.h | 55 +
.../isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h | 135 ++
include/uapi/linux/mtkisp_imgsys.h | 215 ++
51 files changed, 10190 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h
create mode 100644 include/uapi/linux/mtkisp_imgsys.h
--
2.45.2
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
` (8 subsequent siblings)
9 siblings, 1 reply; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
Introduce more Image System (ImgSys) components present in MT8188.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../bindings/media/mediatek,imgsys.yaml | 180 ++++++++++++++++++
1 file changed, 180 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek,imgsys.yaml b/Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
new file mode 100644
index 000000000000..c2899c4b227b
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
@@ -0,0 +1,180 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek,imgsys.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: MediaTek Camera Image System
+
+maintainers:
+ - Olivia Wen <olivia.wen@mediatek.com>
+
+description:
+ MediaTek Camera Image System is the image processing hardware present in MediaTek SoCs
+
+properties:
+ compatible:
+ enum:
+ - mediatek,mt8188-isp-imgsys
+
+ reg:
+ minItems: 1
+ maxItems: 17
+
+ clocks:
+ minItems: 1
+ maxItems: 17
+
+ clock-names:
+ minItems: 1
+ maxItems: 17
+
+ assigned-clocks: true
+
+ assigned-clock-parents: true
+
+ '#address-cells':
+ const: 2
+
+ '#size-cells':
+ const: 2
+
+ iommus:
+ minItems: 1
+ maxItems: 30
+
+ mediatek,larbs:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ minItems: 1
+ maxItems: 32
+ items:
+ maxItems: 1
+ description: |
+ List of phandle to the local arbiters in the current Socs.
+ Refer to bindings/memory-controllers/mediatek,smi-larb.yaml. It must sort
+ according to the local arbiter index, like larb0, larb1, larb2...
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - iommus
+ - mediatek,larbs
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ enum:
+ - mediatek,mt8188-isp-imgsys
+
+ then:
+ required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - iommus
+ - mediatek,larbs
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/mediatek,mt8188-clk.h>
+ #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
+
+ imgsys_fw: imgsys-fw@15000000 {
+ compatible = "mediatek,mt8188-isp-imgsys";
+ reg = <0 0x15000000 0 0x4000>, /* 0 IMGSYS_TOP */
+ <0 0x15020000 0 0x10000>, /* 1 IMGSYS_TRAW */
+ <0 0x15040000 0 0x10000>, /* 2 IMGSYS_LTRAW */
+ <0 0x15640000 0 0x10000>, /* 3 IMGSYS_XTRAW */
+ <0 0x15100000 0 0x10000>, /* 4 IMGSYS_DIP */
+ <0 0x15150000 0 0x10000>, /* 5 IMGSYS_DIP_NR */
+ <0 0x15210000 0 0x10000>, /* 6 IMGSYS_PQDIP_A */
+ <0 0x15510000 0 0x10000>, /* 7 IMGSYS_PQDIP_B */
+ <0 0x15200000 0 0x10000>, /* 8 IMGSYS_WPE_EIS */
+ <0 0x15500000 0 0x10000>, /* 9 IMGSYS_WPE_TNR */
+ <0 0x15600000 0 0x10000>, /* 10 IMGSYS_WPE_LITE */
+ <0 0x15220000 0 0x00100>, /* 11 IMGSYS_WPE1_DIP1 */
+ <0 0x15320000 0 0x10000>, /* 12 IMGSYS_ME */
+ <0 0x15520000 0 0x00100>, /* 13 IMGSYS_WPE2_DIP1 */
+ <0 0x15620000 0 0x00100>, /* 14 IMGSYS_WPE3_DIP1 */
+ <0 0x15110000 0 0x00100>, /* 15 IMGSYS_DIP_TOP */
+ <0 0x15130000 0 0x00100>; /* 16 IMGSYS_DIP_TOP_NR */
+
+ assigned-clocks = <&topckgen CLK_TOP_IMG>;
+ assigned-clock-parents = <&apmixedsys CLK_APMIXED_IMGPLL>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ clocks = <&imgsys CLK_IMGSYS_MAIN_TRAW0>,
+ <&imgsys CLK_IMGSYS_MAIN_TRAW1>,
+ <&imgsys CLK_IMGSYS_MAIN_VCORE_GALS>,
+ <&imgsys CLK_IMGSYS_MAIN_DIP0>,
+ <&imgsys CLK_IMGSYS_MAIN_WPE0>,
+ <&imgsys CLK_IMGSYS_MAIN_WPE1>,
+ <&imgsys CLK_IMGSYS_MAIN_WPE2>,
+ <&imgsys CLK_IMGSYS_MAIN_GALS>,
+ <&imgsys1_dip_top CLK_IMGSYS1_DIP_TOP_DIP_TOP>,
+ <&imgsys1_dip_nr CLK_IMGSYS1_DIP_NR_DIP_NR>,
+ <&imgsys_wpe1 CLK_IMGSYS_WPE1>,
+ <&imgsys_wpe2 CLK_IMGSYS_WPE2>,
+ <&imgsys_wpe3 CLK_IMGSYS_WPE3>,
+ <&imgsys CLK_IMGSYS_MAIN_IPE>,
+ <&ipesys CLK_IPESYS_TOP>,
+ <&ipesys CLK_IPE_ME>,
+ <&ipesys CLK_IPE_SMI_LARB12>;
+ clock-names = "IMGSYS_CG_IMG_TRAW0",
+ "IMGSYS_CG_IMG_TRAW1",
+ "IMGSYS_CG_IMG_VCORE_GALS",
+ "IMGSYS_CG_IMG_DIP0",
+ "IMGSYS_CG_IMG_WPE0",
+ "IMGSYS_CG_IMG_WPE1",
+ "IMGSYS_CG_IMG_WPE2",
+ "IMGSYS_CG_IMG_GALS",
+ "DIP_TOP_DIP_TOP",
+ "DIP_NR_DIP_NR",
+ "WPE1_CG_DIP1_WPE",
+ "WPE2_CG_DIP1_WPE",
+ "WPE3_CG_DIP1_WPE",
+ "ME_CG_IPE",
+ "ME_CG_IPE_TOP",
+ "ME_CG",
+ "ME_CG_LARB12";
+
+ iommus = <&vdo_iommu M4U_PORT_L9_IMGI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_UFDI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_IMGBI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_IMGCI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTI_T4_A>,
+ <&vdo_iommu M4U_PORT_L9_TNCSTI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_TNCSTI_T4_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVBO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVCO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_TIMGO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T2_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T5_A>,
+ <&vdo_iommu M4U_PORT_L9_IMGI_T1_B>,
+ <&vdo_iommu M4U_PORT_L9_IMGBI_T1_B>,
+ <&vdo_iommu M4U_PORT_L9_IMGCI_T1_B>,
+ <&vdo_iommu M4U_PORT_L9_SMTI_T4_B>,
+ <&vdo_iommu M4U_PORT_L9_TNCSO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTO_T4_A>,
+ <&vdo_iommu M4U_PORT_L9_TNCSTO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T2_B>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T5_B>,
+ <&vdo_iommu M4U_PORT_L9_SMTO_T4_B>;
+
+ mediatek,larbs = <&larb9>,
+ <&larb10>,
+ <&larb11a>,
+ <&larb11b>,
+ <&larb11c>,
+ <&larb15>,
+ <&larb12>;
+ };
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-25 4:55 ` Krzysztof Kozlowski
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
` (7 subsequent siblings)
9 siblings, 2 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
This patch adds support for the MT8188 Image System's Local Arbiter
(LARB) in the device tree bindings. The LARB is a crucial component in
MediaTek's ImgSys architecture, responsible for managing memory access
and arbitration between various hardware modules.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../bindings/media/mediatek,imgsys-larbs.yaml | 75 +++++++++++++++++++
1 file changed, 75 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
new file mode 100644
index 000000000000..d2966c64ddb6
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
@@ -0,0 +1,75 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek,imgsys-larbs.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: LARB Information for MediaTek Camera Image System Hardware
+
+maintainers:
+ - Olivia Wen <olivia.wen@mediatek.com>
+
+description:
+ Detailed configuration for MediaTek Camera Image System's Local Arbiter
+ (LARB) hardware integration.
+
+properties:
+ compatible:
+ enum:
+ - mediatek,mt8188-imgsys-larbs
+
+ iommus:
+ minItems: 1
+ maxItems: 30
+
+ '#address-cells':
+ const: 2
+
+ '#size-cells':
+ const: 2
+
+required:
+ - compatible
+ - iommus
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
+
+ imgsys-l11b {
+ compatible = "mediatek,mt8188-imgsys-larb";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ iommus = <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_1>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_4P_0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_4P_1>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_CQ0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_CQ1>,
+ <&vpp_iommu M4U_PORT_L11B_PIMGI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_PIMGBI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_PIMGCI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_IMGI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_IMGBI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_IMGCI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTI_T4_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTI_T6_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVBO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVCO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_WDMA_0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_WDMA_4P_0>,
+ <&vpp_iommu M4U_PORT_L11B_WROT_P1>,
+ <&vpp_iommu M4U_PORT_L11B_TCCSO_P1>,
+ <&vpp_iommu M4U_PORT_L11B_TCCSI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_TIMGO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVO_T2_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVO_T5_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTO_T4_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTO_T6_C>,
+ <&vpp_iommu M4U_PORT_L11B_DBGO_T1_C>;
+ };
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-26 2:13 ` CK Hu (胡俊光)
2025-05-24 11:49 ` [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver Olivia Wen
` (6 subsequent siblings)
9 siblings, 1 reply; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
Add controls and formats for MediaTek Imgsys processing
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
include/uapi/linux/mtkisp_imgsys.h | 215 +++++++++++++++++++++++++++++
1 file changed, 215 insertions(+)
create mode 100644 include/uapi/linux/mtkisp_imgsys.h
diff --git a/include/uapi/linux/mtkisp_imgsys.h b/include/uapi/linux/mtkisp_imgsys.h
new file mode 100644
index 000000000000..c7528c8bf4a2
--- /dev/null
+++ b/include/uapi/linux/mtkisp_imgsys.h
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * MediaTek ISP imgsys User space API
+ *
+ * Copyright (c) 2024 MediaTek Inc.
+ */
+
+#ifndef _MTKISP_IMGSYS_USER_H
+#define _MTKISP_IMGSYS_USER_H
+
+#include <linux/v4l2-controls.h>
+
+/******************************************************/
+/* MediaTek ISP imgsys controls */
+/******************************************************/
+
+/* The base for the MediaTek imgsys driver controls */
+/* We reserve 48 controls for this driver. */
+#define V4L2_CID_USER_MTK_IMG_BASE (V4L2_CID_USER_BASE + 0x1100)
+
+/**
+ * Define a control identifier, V4L2_CID_MTK_IMG_RESIZE_RATIO, for setting the
+ * image processing resize ratio for MediaTek chip. This identifier allows
+ * adjustment of the image resize ratio to accommodate various display or
+ * processing needs. ImgSys hardware supports multiple resizers, including
+ * any ratio, downscaling by factors of 2, 4, and 42, with NULL indicating
+ * the end of menu options.
+ */
+#define V4L2_CID_MTK_IMG_RESIZE_RATIO (V4L2_CID_USER_MTK_IMG_BASE + 34)
+
+/******************************************************/
+/* Vendor specific - MediaTek ISP formats */
+/******************************************************/
+
+/* MediaTek warp map 32-bit, 2 plane */
+#define V4L2_PIX_FMT_WARP2P v4l2_fourcc('M', 'W', '2', 'P')
+/* YUV-10bit packed 4:2:0 2plane, (Y)(UV) */
+#define V4L2_PIX_FMT_YUV_2P010P v4l2_fourcc('U', '0', '2', 'A')
+/* YUV-12bit packed 4:2:0 2plane, (Y)(UV) */
+#define V4L2_PIX_FMT_YUV_2P012P v4l2_fourcc('U', '0', '2', 'C')
+/* Y-32bit */
+#define V4L2_PIX_FMT_MTISP_Y32 v4l2_fourcc('M', 'T', '3', '2')
+/* Y-16bit */
+#define V4L2_PIX_FMT_MTISP_Y16 v4l2_fourcc('M', 'T', '1', '6')
+/* Y-8bit */
+#define V4L2_PIX_FMT_MTISP_Y8 v4l2_fourcc('M', 'T', '0', '8')
+
+/**
+ * Describes the MediaTek APLY8(V4L2_PIX_FMT_MTISP_APLY8) format, which includes
+ * Y-8bit image data and appends APL(Average Pixel Level) data utilized by the
+ * ME(motion estimation) hardware to the end of the image.
+ *
+ * The hardware ME generates motion vectors by processing two consecutive frames,
+ * referred to as the previous and current frames, to enhance graphic processing.
+ * The diagram is shown below.
+ *
+ * +------------------+
+ * the thumbnail of previous frame -> | |
+ * (include the L0 frame and L1 frame)| |
+ * | ME | -> motion vectors
+ * the thumbnail of current frame -> | |
+ * (include the L0 frame and L1 frame)| |
+ * +------------------+
+ *
+ * To accelerate computation, the two images will be resized into two different
+ * smaller images, referred to as L0 frame, L1 frame and APL data.
+ * The L1 frame is a scaled-down version of the L0 frame, reduced to one-fourth
+ * of its original size containing APL data.
+ *
+ * +------------------+
+ * previous L0 frame -> | |
+ * previous L1 frame -> | |
+ * (include APL data) | |
+ * | ME | -> motion vectors
+ * current L0 frame -> | |
+ * current L1 frame -> | |
+ * (include APL data) | |
+ * +------------------+
+ *
+ * Each frame size:
+ * the width of L0 frame (L0_wd) = 576 (fixed value)
+ * the height of L0 frame (L0_ht) =
+ * 16 pixels align((the width of original image * L0_wd) / the height of original image)
+ * the width of L1 frame (L1_wd) = L0_wd / 4
+ * the height of L1 frame (L1_ht) = L0_ht / 4
+ *
+ * Buffer size:
+ * line_size = width * bytes_per_pixel(1)
+ * stride with 16 bytes alignment = ((line_size + 15) / 16) * 16
+ * buffer_size = stride with 16 bytes alignment * height + APL data(16 bytes)
+ *
+ * And the memory layout should be followed as
+ *
+ * @code
+ * APLY8
+ * +------------------+
+ * | Y Plane |
+ * | Bit Stream |
+ * +------------------+
+ * | APL data |
+ * +------------------+
+ * @endcode
+ *
+ * @note The APL data size is 16 bytes.The caller SHOULD NOT modify it.
+ */
+/* Y-8bit image data and append the APL data */
+#define V4L2_PIX_FMT_MTISP_APLY8 v4l2_fourcc('M', 'T', 'A', '8')
+
+/* Packed 10-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR10 v4l2_fourcc('M', 'B', 'B', 'A')
+#define V4L2_PIX_FMT_MTISP_SGBRG10 v4l2_fourcc('M', 'B', 'G', 'A')
+#define V4L2_PIX_FMT_MTISP_SGRBG10 v4l2_fourcc('M', 'B', 'g', 'A')
+#define V4L2_PIX_FMT_MTISP_SRGGB10 v4l2_fourcc('M', 'B', 'R', 'A')
+
+/* Vendor specific - MediaTek ISP parameters for firmware */
+/* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_TUN_PARAMS v4l2_fourcc('M', 'T', 'f', 't')
+/* ISP statistics parameters */
+#define V4L2_META_FMT_MTISP_STAT_PARAMS v4l2_fourcc('M', 'T', 'f', 's')
+/* ISP controlling parameters */
+#define V4L2_META_FMT_MTISP_CTL_PARAMS v4l2_fourcc('M', 'T', 'f', 'c')
+
+/*****************************************************************************/
+/* Define sizes for various formats used in the MediaTek image system driver */
+/*****************************************************************************/
+/**
+ * SIZE_OF_APL_DATA - Size of the APL (Average Pixel Level) data
+ *
+ * This constant defines the size of the APL data, which is 16 bytes.
+ * The APL data is appended to the image format defined by
+ * V4L2_PIX_FMT_MTISP_APLY8. This format includes 8-bit Y image data
+ * followed by the APL data, which is utilized by the motion estimation
+ * hardware for enhanced processing. The APL data provides important
+ * information about the average pixel levels, which can be critical
+ * for various image processing tasks.
+ */
+#define SIZE_OF_APL_DATA 16 /* Size of APL data used by V4L2_PIX_FMT_MTISP_APLY8 */
+
+/**
+ * MediaTek Tuning Metadata Buffer for Image Processing Modules
+ *
+ * This definition is used for format V4L2_META_FMT_MTISP_TUN_PARAMS.
+ * This buffer contains essential tuning information for the MediaTek camera,
+ * including:
+ * - Sensor tuning settings
+ * - Digital gain for adjusting image brightness
+ * - White balance gain
+ *
+ * Note: The individual fields of the tuning metadata structure are not exposed
+ * to the kernel. The kernel only requires the size of the tuning metadata
+ * for memory allocation and management purposes. This encapsulation provides
+ * flexibility in implementing the tuning metadata without impacting the
+ * kernel's interface.
+ */
+ #define SIZE_OF_TUNING_META 219352
+
+/**
+ * MediaTek Control Metadata Buffer for Image Processing Modules
+ *
+ * This definition is used for format V4L2_META_FMT_MTISP_CTL_PARAMS.
+ * The control metadata buffer aggregates control information for multiple
+ * image processing modules, including TRAW, DIP, PQDIP, ME, WPE, and ADL.
+ * It facilitates the configuration and management of these modules by
+ * encapsulating relevant parameters and settings required for processing
+ * image data efficiently. This structure serves as a central point for
+ * coordinating the operation of different hardware components in the image
+ * processing pipeline.
+ *
+ * Note: The individual fields of the control metadata structure are not exposed
+ * to the kernel. The kernel only requires the size of the control metadata
+ * for memory allocation and management purposes. This encapsulation provides
+ * flexibility in implementing the control metadata without impacting the
+ * kernel's interface.
+ */
+#define SIZE_OF_COMMON_CTRL 688
+#define SIZE_OF_WPE_CTRL 240
+#define SIZE_OF_TRAW_CTRL 120
+#define SIZE_OF_DIP_CTRL 112
+#define SIZE_OF_PQDIP_CTRL 112
+#define SIZE_OF_ME_CTRL 8
+#define SIZE_OF_ADL_CTRL 64
+#define SIZE_OF_YUFO_META_INFO_WPE 4228
+#define SIZE_OF_YUFO_META_INFO_DIP 4228
+
+#define SIZE_OF_CTRL_META ( \
+ SIZE_OF_COMMON_CTRL + \
+ SIZE_OF_WPE_CTRL + \
+ SIZE_OF_TRAW_CTRL + \
+ SIZE_OF_DIP_CTRL + \
+ SIZE_OF_PQDIP_CTRL + \
+ SIZE_OF_ME_CTRL + \
+ SIZE_OF_ADL_CTRL + \
+ SIZE_OF_YUFO_META_INFO_WPE + \
+ SIZE_OF_YUFO_META_INFO_DIP \
+)
+
+/**
+ * MediaTek Statistics Metadata Buffer for Image Processing Modules
+ *
+ * This definition is used for format V4L2_META_FMT_MTISP_STAT_PARAMS.
+ * This buffer contains essential statistics information for the MediaTek camera,
+ * including:
+ * - Readiness and corresponding values for tone and contrast statistics
+ * - Motion estimation (ME) statistics
+ * - ME feature match blocks statistics
+ *
+ * Note: The individual fields of the statistics buffer structure are not exposed
+ * to the kernel. The kernel only requires the size of the statistics metadata
+ * for memory allocation and management purposes. This encapsulation provides
+ * flexibility in implementing the statistics metadata without impacting the
+ * kernel's interface.
+ */
+ #define SIZE_OF_STATISTICS_META 88
+
+#endif /* _MTKISP_IMGSYS_USER_H */
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (2 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-26 7:16 ` CK Hu (胡俊光)
2025-05-24 11:49 ` [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device Olivia Wen
` (5 subsequent siblings)
9 siblings, 1 reply; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the V4L2 flow. The MediaTek's Image System (ImgSys), also
known as ISP Pass2. ImgSys is a memory-to-memory hardware device
designed for advanced image processing tasks. It is composed of multiple
hardware components, including TRAW, DIP, PQDIP, ME, and WPE.
TRAW (Tile RAW):
- Provides multiple downscaled resizers to support multi-scale noise
reduction.
- Supports RAW/RGB format conversion.
DIP (Digital Image Processing) and PQDIP (Picture Quality DIP):
- Supports such as temporal and spatial noise reduction (TNR/SNR),
edge enhancement (EE), and sharpness (SHP).
- PQDIP also supports image scaling and rotation.
ME (Motion Estimation):
- Supports motion estimation between two consecutive frames.
WPE (Warping Engine):
- Handles image warping and de-warping processes.
The ImgSys driver, primarily utilized for Google Chromebook products,
uses Multi-Frame Combined Noise Reduction technology to enhance image
quality.
Below is a simplified architecture of the ImgSys driver:
User -> V4L2 Framework -> ImgSys Driver
-> SCP (It packages the hardware settings into commands)
-> ImgSys Driver
-> CMDQ (The packaged commands are sent to the hardware via GCE)
-> Hardware
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
drivers/media/platform/mediatek/Kconfig | 1 +
drivers/media/platform/mediatek/Makefile | 1 +
drivers/media/platform/mediatek/isp/Kconfig | 23 +
.../platform/mediatek/isp/isp_7x/Makefile | 6 +
.../mediatek/isp/isp_7x/imgsys/Makefile | 11 +
.../imgsys/modules/mtk_dip_v4l2_vnode.h | 594 ++++++
.../isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h | 386 ++++
.../imgsys/modules/mtk_pqdip_v4l2_vnode.h | 149 ++
.../imgsys/modules/mtk_traw_v4l2_vnode.h | 371 ++++
.../imgsys/modules/mtk_wpe_v4l2_vnode.h | 317 ++++
| 28 +
.../isp/isp_7x/imgsys/mtk_imgsys-dev.c | 238 +++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 427 +++++
.../isp/isp_7x/imgsys/mtk_imgsys-formats.c | 139 ++
.../isp/isp_7x/imgsys/mtk_imgsys-formats.h | 72 +
.../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 27 +
.../isp/isp_7x/imgsys/mtk_imgsys-of.c | 39 +
.../isp/isp_7x/imgsys/mtk_imgsys-of.h | 55 +
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 27 +
.../isp/isp_7x/imgsys/mtk_imgsys-sys.h | 50 +
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 1684 +++++++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h | 100 +
.../isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h | 135 ++
23 files changed, 4880 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h
diff --git a/drivers/media/platform/mediatek/Kconfig b/drivers/media/platform/mediatek/Kconfig
index 84104e2cd024..4e0a5a43f35e 100644
--- a/drivers/media/platform/mediatek/Kconfig
+++ b/drivers/media/platform/mediatek/Kconfig
@@ -7,3 +7,4 @@ source "drivers/media/platform/mediatek/mdp/Kconfig"
source "drivers/media/platform/mediatek/vcodec/Kconfig"
source "drivers/media/platform/mediatek/vpu/Kconfig"
source "drivers/media/platform/mediatek/mdp3/Kconfig"
+source "drivers/media/platform/mediatek/isp/Kconfig"
diff --git a/drivers/media/platform/mediatek/Makefile b/drivers/media/platform/mediatek/Makefile
index 38e6ba917fe5..611fa95416bc 100644
--- a/drivers/media/platform/mediatek/Makefile
+++ b/drivers/media/platform/mediatek/Makefile
@@ -4,3 +4,4 @@ obj-y += mdp/
obj-y += vcodec/
obj-y += vpu/
obj-y += mdp3/
+obj-y += isp/isp_7x/
diff --git a/drivers/media/platform/mediatek/isp/Kconfig b/drivers/media/platform/mediatek/isp/Kconfig
new file mode 100644
index 000000000000..df92ac67574d
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/Kconfig
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config VIDEO_MTK_ISP_71_IMGSYS
+ tristate "MediaTek ISP71 image processing system function"
+ depends on MTK_CMDQ
+ depends on REMOTEPROC
+ depends on MTK_SCP
+ select VIDEO_V4L2_SUBDEV_API
+ select VIDEOBUF2_DMA_CONTIG
+ select VIDEOBUF2_CORE
+ select VIDEOBUF2_V4L2
+ select VIDEOBUF2_MEMOPS
+ select MEDIA_CONTROLLER
+
+ default n
+ help
+ Support the basic Image processing system driver.
+
+ IMGSYS driver provides image format conversion, resizing,
+ and rotation function through the low power hardware.
+ IMGSYS also supports multiple output feature. It can
+ generate two or more output image with different effect
+ based on a single input image at the same time.
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/Makefile
new file mode 100644
index 000000000000..11d2aee9a3dc
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+subdir-ccflags-y += -Werror
+
+obj-$(CONFIG_VIDEO_MTK_ISP_71_IMGSYS) += imgsys/
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
new file mode 100644
index 000000000000..571d0ae000eb
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+
+mtk_imgsys_util-objs := \
+mtk_imgsys-of.o \
+mtk_imgsys-formats.o \
+mtk_imgsys-dev.o \
+mtk_imgsys-sys.o \
+mtk_imgsys-v4l2.o
+
+obj-$(CONFIG_VIDEO_MTK_ISP_71_IMGSYS) += mtk_imgsys_util.o
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
new file mode 100644
index 000000000000..1b5c8b738a54
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
@@ -0,0 +1,594 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_DIP_V4L2_VNODE_H_
+#define _MTK_DIP_V4L2_VNODE_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-formats.h"
+#include "../mtk_imgsys-vnode_id.h"
+
+static const struct mtk_imgsys_dev_format dip_imgi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_vipi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_rec_dsi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_rec_dpi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y32,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_meta_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_y32,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_tnrw_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 192,
+ .scan_align = 192,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_tnrci_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 144,
+ .scan_align = 108,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_tnrli_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_img2o_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_img3o_fmts[] = {
+ /* YUV422, 3 plane 8 bit */
+ /* YUV420, 2 plane 8 bit */
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ /* Y8 bit */
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format dip_img4o_fmts[] = {
+ /* YUV420, 2 plane 8 bit */
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ /* Y8 bit */
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct v4l2_frmsizeenum dip_in_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .stepwise.min_width = MTK_IMGSYS_CAPTURE_MIN_WIDTH,
+ .stepwise.max_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .stepwise.min_height = MTK_IMGSYS_CAPTURE_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct v4l2_frmsizeenum dip_out_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .stepwise.min_width = MTK_IMGSYS_OUTPUT_MIN_WIDTH,
+ .stepwise.max_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .stepwise.min_height = MTK_IMGSYS_OUTPUT_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct mtk_imgsys_video_device_desc dip_setting[] = {
+ /* Input Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_IMGI_OUT,
+ .name = "Imgi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_imgi_fmts,
+ .num_fmts = ARRAY_SIZE(dip_imgi_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Main image source",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_VIPI_OUT,
+ .name = "Vipi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_vipi_fmts,
+ .num_fmts = ARRAY_SIZE(dip_vipi_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Vipi image source",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_REC_DSI_OUT,
+ .name = "Rec_Dsi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_rec_dsi_fmts,
+ .num_fmts = ARRAY_SIZE(dip_rec_dsi_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Down Source Image",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_REC_DPI_OUT,
+ .name = "Rec_Dpi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_rec_dpi_fmts,
+ .num_fmts = ARRAY_SIZE(dip_rec_dpi_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Down Processed Image",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_CNR_BLURMAPI_OUT,
+ .name = "Bokeh Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Bokehi data",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_LFEI_OUT,
+ .name = "Dmgi_FM Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Dmgi_FM data",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_RFEI_OUT,
+ .name = "Depi_FM Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Depi_FM data",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRSI_OUT,
+ .name = "Tnrsi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Statistics input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRWI_OUT,
+ .name = "Tnrwi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_tnrw_fmts,
+ .num_fmts = ARRAY_SIZE(dip_tnrw_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Weighting input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRMI_OUT,
+ .name = "Tnrmi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Motion Map input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRCI_OUT,
+ .name = "Tnrci Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_tnrci_fmts,
+ .num_fmts = ARRAY_SIZE(dip_tnrci_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Confidence Map input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRLI_OUT,
+ .name = "Tnrli Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_tnrli_fmts,
+ .num_fmts = ARRAY_SIZE(dip_tnrli_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Low Frequency Diff input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRVBI_OUT,
+ .name = "Tnrvbi Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Valid Bit Map input",
+ },
+ /* Output Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_IMG2O_CAPTURE,
+ .name = "Img2o Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_img2o_fmts,
+ .num_fmts = ARRAY_SIZE(dip_img2o_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "Resized output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_IMG3O_CAPTURE,
+ .name = "Img3o Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_img3o_fmts,
+ .num_fmts = ARRAY_SIZE(dip_img3o_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "Dip output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_IMG4O_CAPTURE,
+ .name = "Img4o Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_img4o_fmts,
+ .num_fmts = ARRAY_SIZE(dip_img4o_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "Nr3d output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_FMO_CAPTURE,
+ .name = "FM Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "FM output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRSO_CAPTURE,
+ .name = "Tnrso Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "statistics output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRWO_CAPTURE,
+ .name = "Tnrwo Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_tnrw_fmts,
+ .num_fmts = ARRAY_SIZE(dip_tnrw_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "Weighting output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TNRMO_CAPTURE,
+ .name = "Tnrmo Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = dip_meta_fmts,
+ .num_fmts = ARRAY_SIZE(dip_meta_fmts),
+ .default_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &dip_out_frmsizeenum,
+ .description = "Motion Map output",
+ },
+};
+
+#endif // _MTK_IMGSYS_V4L2_VNODE_H_
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
new file mode 100644
index 000000000000..d33f8847731f
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
@@ -0,0 +1,386 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_ME_V4L2_VNODE_H_
+#define _MTK_ME_V4L2_VNODE_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-formats.h"
+#include "../mtk_imgsys-vnode_id.h"
+
+static const struct mtk_imgsys_dev_format me_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_y32,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y16,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format mewmap_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_warp2p,
+ .align = 8,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format melmi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_y16,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format mel1i_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_aply8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct v4l2_frmsizeenum in_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_IMGSYS_CAPTURE_MAX_WIDTH,
+ .stepwise.min_width = MTK_IMGSYS_CAPTURE_MIN_WIDTH,
+ .stepwise.max_height = MTK_IMGSYS_CAPTURE_MAX_HEIGHT,
+ .stepwise.min_height = MTK_IMGSYS_CAPTURE_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct v4l2_frmsizeenum out_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .stepwise.min_width = MTK_IMGSYS_OUTPUT_MIN_WIDTH,
+ .stepwise.max_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .stepwise.min_height = MTK_IMGSYS_OUTPUT_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct mtk_imgsys_video_device_desc me_setting[] = {
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL0IMG0_OUT,
+ .name = "L0I0I Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME Input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL0IMG1_OUT,
+ .name = "L0I1I Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME Input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL1IMG0_OUT,
+ .name = "L1I0I Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = mel1i_fmts,
+ .num_fmts = ARRAY_SIZE(mel1i_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME L1 Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL1IMG1_OUT,
+ .name = "L1I1I Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = mel1i_fmts,
+ .num_fmts = ARRAY_SIZE(mel1i_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME L1 Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEIMGSTATS_OUT,
+ .name = "STATI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME IMGSTATS Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL0RMV_OUT,
+ .name = "L0RMVI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME L0RMV Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL1RMV_OUT,
+ .name = "L1RMVI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME L1RMV Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL0FMB_OUT,
+ .name = "L0FMBI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME L0FMB Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL1FMB_OUT,
+ .name = "L1FMBI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME L1FMB Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEMIL_OUT,
+ .name = "MEMILI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &in_frmsizeenum,
+ .description = "ME MIL Input ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEMMGMIL_CAPTURE,
+ .name = "MILO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME MMGMIL Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL0WMV_CAPTURE,
+ .name = "L0WMVO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME L0WMV Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL1WMV_CAPTURE,
+ .name = "L1WMVO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME L1WMV Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MECONF_CAPTURE,
+ .name = "CONFO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME CONF Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEWMAP_CAPTURE,
+ .name = "WMAPO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = mewmap_fmts,
+ .num_fmts = ARRAY_SIZE(mewmap_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME WMAP Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEFMV_CAPTURE,
+ .name = "FMVO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME OTHERS Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEFST_CAPTURE,
+ .name = "FSTO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME FST Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MELMI_CAPTURE,
+ .name = "LMIO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = melmi_fmts,
+ .num_fmts = ARRAY_SIZE(melmi_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME LMI Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL0WFMB_CAPTURE,
+ .name = "L0FMBO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME L0FMB Output ",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_MEL1WFMB_CAPTURE,
+ .name = "L1FMBO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = me_fmts,
+ .num_fmts = ARRAY_SIZE(me_fmts),
+ .default_width = MTK_IMGSYS_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_IMGSYS_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &out_frmsizeenum,
+ .description = "ME L1FMB Output ",
+ },
+};
+
+#endif /* _MTK_ME_V4L2_VNODE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
new file mode 100644
index 000000000000..23c8db3b30f3
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_PQDIP_V4L2_VNODE_H_
+#define _MTK_PQDIP_V4L2_VNODE_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-formats.h"
+#include "../mtk_imgsys-vnode_id.h"
+
+#define MTK_PQDIP_OUTPUT_MIN_WIDTH 2U
+#define MTK_PQDIP_OUTPUT_MIN_HEIGHT 2U
+#define MTK_PQDIP_OUTPUT_MAX_WIDTH 5376U
+#define MTK_PQDIP_OUTPUT_MAX_HEIGHT 4032U
+#define MTK_PQDIP_CAPTURE_MIN_WIDTH 2U
+#define MTK_PQDIP_CAPTURE_MIN_HEIGHT 2U
+#define MTK_PQDIP_CAPTURE_MAX_WIDTH 5376U
+#define MTK_PQDIP_CAPTURE_MAX_HEIGHT 4032U
+
+static const struct mtk_imgsys_dev_format pqdip_pimgi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format pqdip_wroto_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format pqdip_tccso_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+};
+
+static const struct v4l2_frmsizeenum pqdip_in_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .stepwise.min_width = MTK_PQDIP_CAPTURE_MIN_WIDTH,
+ .stepwise.max_height = MTK_PQDIP_CAPTURE_MAX_HEIGHT,
+ .stepwise.min_height = MTK_PQDIP_CAPTURE_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct v4l2_frmsizeenum pqdip_out_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_PQDIP_OUTPUT_MAX_WIDTH,
+ .stepwise.min_width = MTK_PQDIP_OUTPUT_MIN_WIDTH,
+ .stepwise.max_height = MTK_PQDIP_OUTPUT_MAX_HEIGHT,
+ .stepwise.min_height = MTK_PQDIP_OUTPUT_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct mtk_imgsys_video_device_desc pqdip_setting[] = {
+ /* Input Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_PIMGI_OUT,
+ .name = "PIMGI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = pqdip_pimgi_fmts,
+ .num_fmts = ARRAY_SIZE(pqdip_pimgi_fmts),
+ .default_width = MTK_PQDIP_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_PQDIP_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &pqdip_in_frmsizeenum,
+ .description = "Imgi image source",
+ },
+ /* Output Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WROT_A_CAPTURE,
+ .name = "WROTO A Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = pqdip_wroto_fmts,
+ .num_fmts = ARRAY_SIZE(pqdip_wroto_fmts),
+ .default_width = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &pqdip_out_frmsizeenum,
+ .description = "Output quality enhanced image",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WROT_B_CAPTURE,
+ .name = "WROTO B Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = pqdip_wroto_fmts,
+ .num_fmts = ARRAY_SIZE(pqdip_wroto_fmts),
+ .default_width = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &pqdip_out_frmsizeenum,
+ .description = "Output quality enhanced image",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TCCSO_A_CAPTURE,
+ .name = "TCCSO A Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = pqdip_tccso_fmts,
+ .num_fmts = ARRAY_SIZE(pqdip_tccso_fmts),
+ .default_width = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &pqdip_out_frmsizeenum,
+ .description = "Output tone curve statistics",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TCCSO_B_CAPTURE,
+ .name = "TCCSO B Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = pqdip_tccso_fmts,
+ .num_fmts = ARRAY_SIZE(pqdip_tccso_fmts),
+ .default_width = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_PQDIP_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &pqdip_out_frmsizeenum,
+ .description = "Output tone curve statistics",
+ },
+};
+
+#endif /* _MTK_PQDIP_V4L2_VNODE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
new file mode 100644
index 000000000000..ec03f7a59be8
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
@@ -0,0 +1,371 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Shih-Fang chuang <shih-fang.chuang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_TRAW_V4L2_VNODE_H_
+#define _MTK_TRAW_V4L2_VNODE_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-formats.h"
+#include "../mtk_imgsys-vnode_id.h"
+
+#define MTK_TRAW_OUTPUT_MIN_WIDTH 2U
+#define MTK_TRAW_OUTPUT_MIN_HEIGHT 2U
+#define MTK_TRAW_OUTPUT_MAX_WIDTH 5376U
+#define MTK_TRAW_OUTPUT_MAX_HEIGHT 4032U
+#define MTK_TRAW_CAPTURE_MIN_WIDTH 2U
+#define MTK_TRAW_CAPTURE_MIN_HEIGHT 2U
+#define MTK_TRAW_CAPTURE_MAX_WIDTH 5376U
+#define MTK_TRAW_CAPTURE_MAX_HEIGHT 4032U
+
+static const struct mtk_imgsys_dev_format traw_imgi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_sgbrg10,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_warp2p,
+ .align = 8,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_srggb10,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_sgrbg10,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format traw_metai_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_metai,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format traw_stato_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_stt,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format traw_yuvo_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_warp2p,
+ .align = 8,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y8,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_aply8,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format traw_pdc_fmts[] = {
+ /* Y8 bit */
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+};
+
+static const struct v4l2_frmsizeenum traw_in_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .stepwise.min_width = MTK_TRAW_CAPTURE_MIN_WIDTH,
+ .stepwise.max_height = MTK_TRAW_CAPTURE_MAX_HEIGHT,
+ .stepwise.min_height = MTK_TRAW_CAPTURE_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct v4l2_frmsizeenum traw_out_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_TRAW_OUTPUT_MAX_WIDTH,
+ .stepwise.min_width = MTK_TRAW_OUTPUT_MIN_WIDTH,
+ .stepwise.max_height = MTK_TRAW_OUTPUT_MAX_HEIGHT,
+ .stepwise.min_height = MTK_TRAW_OUTPUT_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct mtk_imgsys_video_device_desc traw_setting[] = {
+ /* Input Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TIMGI_OUT,
+ .name = "TIMGI Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_imgi_fmts,
+ .num_fmts = ARRAY_SIZE(traw_imgi_fmts),
+ .default_width = MTK_TRAW_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_TRAW_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &traw_in_frmsizeenum,
+ .description = "Imgi image source",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_METAI_OUT,
+ .name = "METAI Input",
+ .cap = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+ .smem_alloc = 1,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_metai_fmts,
+ .num_fmts = ARRAY_SIZE(traw_metai_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &traw_in_frmsizeenum,
+ .description = "METAI image source",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_PDC_OUT,
+ .name = "PDC Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_pdc_fmts,
+ .num_fmts = ARRAY_SIZE(traw_pdc_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &traw_in_frmsizeenum,
+ .description = "PDC image source",
+ },
+ /* Output Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TYUVO_CAPTURE,
+ .name = "TYUVO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_yuvo_fmts,
+ .num_fmts = ARRAY_SIZE(traw_yuvo_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "YUVO output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TYUV2O_CAPTURE,
+ .name = "TYUV2O Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_yuvo_fmts,
+ .num_fmts = ARRAY_SIZE(traw_yuvo_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "YUV2O output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TYUV3O_CAPTURE,
+ .name = "TYUV3O Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_yuvo_fmts,
+ .num_fmts = ARRAY_SIZE(traw_yuvo_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "YUV3O output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TYUV4O_CAPTURE,
+ .name = "TYUV4O Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_yuvo_fmts,
+ .num_fmts = ARRAY_SIZE(traw_yuvo_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "YUV4O output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TYUV5O_CAPTURE,
+ .name = "TYUV5O Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_yuvo_fmts,
+ .num_fmts = ARRAY_SIZE(traw_yuvo_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "YUV5O output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_FEO_CAPTURE,
+ .name = "FEO Output",
+ .cap = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+ .smem_alloc = 1,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_metai_fmts,
+ .num_fmts = ARRAY_SIZE(traw_metai_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "FEO output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TIMGO_CAPTURE,
+ .name = "TIMGO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_imgi_fmts,
+ .num_fmts = ARRAY_SIZE(traw_imgi_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "TIMGO output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_IMGSTATO_CAPTURE,
+ .name = "IMGSTATO Output",
+ .cap = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_META_CAPTURE,
+ .smem_alloc = 1,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_stato_fmts,
+ .num_fmts = ARRAY_SIZE(traw_stato_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "IMGSTATO output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_XTMEO_CAPTURE,
+ .name = "XTMEO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_pdc_fmts,
+ .num_fmts = ARRAY_SIZE(traw_pdc_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "XTMEO output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_XTFDO_CAPTURE,
+ .name = "XTFDO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_yuvo_fmts,
+ .num_fmts = ARRAY_SIZE(traw_yuvo_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "XTFDO output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_XTADLDBGO_CAPTURE,
+ .name = "XTDBGO Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = traw_imgi_fmts,
+ .num_fmts = ARRAY_SIZE(traw_imgi_fmts),
+ .default_width = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_TRAW_CAPTURE_MAX_WIDTH,
+ .frmsizeenum = &traw_out_frmsizeenum,
+ .description = "XTDBGO output",
+ },
+};
+
+#endif /* _MTK_TRAW_V4L2_VNODE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
new file mode 100644
index 000000000000..3e378d54b900
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
@@ -0,0 +1,317 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Floria Huang <floria.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_WPE_V4L2_VNODE_H_
+#define _MTK_WPE_V4L2_VNODE_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-formats.h"
+#include "../mtk_imgsys-vnode_id.h"
+
+#define MTK_WPE_OUTPUT_MIN_WIDTH 2U
+#define MTK_WPE_OUTPUT_MIN_HEIGHT 2U
+#define MTK_WPE_OUTPUT_MAX_WIDTH 18472U
+#define MTK_WPE_OUTPUT_MAX_HEIGHT 13856U
+
+#define MTK_WPE_MAP_OUTPUT_MIN_WIDTH 2U
+#define MTK_WPE_MAP_OUTPUT_MIN_HEIGHT 2U
+#define MTK_WPE_MAP_OUTPUT_MAX_WIDTH 640U
+#define MTK_WPE_MAP_OUTPUT_MAX_HEIGHT 480U
+
+#define MTK_WPE_PSP_OUTPUT_WIDTH 8U
+#define MTK_WPE_PSP_OUTPUT_HEIGHT 33U
+
+#define MTK_WPE_CAPTURE_MIN_WIDTH 2U
+#define MTK_WPE_CAPTURE_MIN_HEIGHT 2U
+#define MTK_WPE_CAPTURE_MAX_WIDTH 18472U
+#define MTK_WPE_CAPTURE_MAX_HEIGHT 13856U
+
+static const struct mtk_imgsys_dev_format wpe_wpei_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ /* Y8 bit */
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 192,
+ .scan_align = 192,
+ },
+ /* Bayer 10 bit */
+ {
+ .fmt = &mtk_imgsys_format_sgbrg10,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format wpe_veci_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_warp2p,
+ .align = 8,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format wpe_pspi_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_y32,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format wpe_wpeo_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_nv12,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_nv21,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 192,
+ .scan_align = 192,
+ },
+ {
+ .fmt = &mtk_imgsys_format_sgbrg10,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p010p,
+ .align = 1,
+ .scan_align = 1,
+ },
+ {
+ .fmt = &mtk_imgsys_format_2p012p,
+ .align = 1,
+ .scan_align = 1,
+ },
+};
+
+static const struct mtk_imgsys_dev_format wpe_msko_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_grey,
+ .align = 16,
+ .scan_align = 1,
+ },
+};
+
+static const struct v4l2_frmsizeenum wpe_in_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .stepwise.min_width = MTK_WPE_CAPTURE_MIN_WIDTH,
+ .stepwise.max_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .stepwise.min_height = MTK_WPE_CAPTURE_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct v4l2_frmsizeenum wpe_in_map_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_WPE_MAP_OUTPUT_MAX_WIDTH,
+ .stepwise.min_width = MTK_WPE_MAP_OUTPUT_MIN_WIDTH,
+ .stepwise.max_height = MTK_WPE_MAP_OUTPUT_MAX_HEIGHT,
+ .stepwise.min_height = MTK_WPE_MAP_OUTPUT_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct v4l2_frmsizeenum wpe_out_frmsizeenum = {
+ .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
+ .stepwise.max_width = MTK_WPE_OUTPUT_MAX_WIDTH,
+ .stepwise.min_width = MTK_WPE_OUTPUT_MIN_WIDTH,
+ .stepwise.max_height = MTK_WPE_OUTPUT_MAX_HEIGHT,
+ .stepwise.min_height = MTK_WPE_OUTPUT_MIN_HEIGHT,
+ .stepwise.step_height = 1,
+ .stepwise.step_width = 1,
+};
+
+static const struct mtk_imgsys_video_device_desc wpe_setting[] = {
+ /* Input Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WWPEI_OUT,
+ .name = "WPEI_E Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_wpei_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_wpei_fmts),
+ .default_width = MTK_WPE_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_WPE_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &wpe_in_frmsizeenum,
+ .description = "WPE main image input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WVECI_OUT,
+ .name = "VECI_E Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_veci_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_veci_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_in_map_frmsizeenum,
+ .description = "WarpMap input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WPSP_COEFI_OUT,
+ .name = "PSPI_E Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_pspi_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_pspi_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_in_frmsizeenum,
+ .description = "PSP coef. table input",
+ },
+ /* WPE_EIS Output Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WWPEO_CAPTURE,
+ .name = "WPEO_E Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_wpeo_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_wpeo_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_out_frmsizeenum,
+ .description = "WPE image output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WMSKO_CAPTURE,
+ .name = "MSKO_E Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_msko_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_msko_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_out_frmsizeenum,
+ .description = "WPE valid map output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WTWPEI_OUT,
+ .name = "WPEI_T Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_wpei_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_wpei_fmts),
+ .default_width = MTK_WPE_OUTPUT_MAX_WIDTH,
+ .default_height = MTK_WPE_OUTPUT_MAX_HEIGHT,
+ .frmsizeenum = &wpe_in_frmsizeenum,
+ .description = "WPE main image input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WTVECI_OUT,
+ .name = "VECI_T Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_veci_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_veci_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_in_map_frmsizeenum,
+ .description = "WarpMap input",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WTPSP_COEFI_OUT,
+ .name = "PSPI_T Input",
+ .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_pspi_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_pspi_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_in_frmsizeenum,
+ .description = "PSP coef. table input",
+ },
+ /* WPE_TNR Output Video Node */
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WTWPEO_CAPTURE,
+ .name = "WPEO_T Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_wpeo_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_wpeo_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_out_frmsizeenum,
+ .description = "WPE image output",
+ },
+ {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_WTMSKO_CAPTURE,
+ .name = "MSKO_T Output",
+ .cap = V4L2_CAP_VIDEO_CAPTURE_MPLANE | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+ .smem_alloc = 0,
+ .supports_ctrls = true,
+ .flags = MEDIA_LNK_FL_DYNAMIC,
+ .fmts = wpe_msko_fmts,
+ .num_fmts = ARRAY_SIZE(wpe_msko_fmts),
+ .default_width = MTK_WPE_CAPTURE_MAX_WIDTH,
+ .default_height = MTK_WPE_CAPTURE_MAX_HEIGHT,
+ .frmsizeenum = &wpe_out_frmsizeenum,
+ .description = "WPE valid map output",
+ },
+};
+
+#endif /* _MTK_WPE_V4L2_VNODE_H_ */
--git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
new file mode 100644
index 000000000000..50c2e3c3395b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ */
+
+#ifndef _HEADER_DESC_
+#define _HEADER_DESC_
+
+#include <linux/videodev2.h>
+
+/**
+ * struct mtk_imgsys_crop - Crop parameters for MediaTek Image System
+ * @c: Rectangle defining the crop area
+ * @left_subpix: Sub-pixel adjustment for the left edge
+ * @top_subpix: Sub-pixel adjustment for the top edge
+ * @width_subpix: Sub-pixel adjustment for the width
+ * @height_subpix: Sub-pixel adjustment for the height
+ */
+struct mtk_imgsys_crop {
+ struct v4l2_rect c; /* Rectangle defining the crop area */
+ struct v4l2_fract left_subpix; /* Sub-pixel adjustment for the left edge */
+ struct v4l2_fract top_subpix; /* Sub-pixel adjustment for the top edge */
+ struct v4l2_fract width_subpix; /* Sub-pixel adjustment for the width */
+ struct v4l2_fract height_subpix; /* Sub-pixel adjustment for the height */
+};
+
+#endif
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
new file mode 100644
index 000000000000..28a35a3226a2
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
@@ -0,0 +1,238 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#include "linux/list.h"
+#include <linux/dma-buf.h>
+#include <linux/dma-direction.h>
+#include <linux/dma-mapping.h>
+#include <linux/platform_device.h>
+#include <linux/hashtable.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-event.h>
+#include "mtk_imgsys-formats.h"
+#include "mtk_imgsys-vnode_id.h"
+#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-sys.h"
+
+int mtk_imgsys_pipe_init(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_pipe *pipe,
+ const struct mtk_imgsys_pipe_desc *setting)
+{
+ u32 i, nodes_num;
+ size_t nodes_size;
+
+ pipe->imgsys_dev = imgsys_dev;
+ pipe->desc = setting;
+ pipe->nodes_enabled = 0ULL;
+ pipe->nodes_streaming = 0ULL;
+
+ atomic_set(&pipe->pipe_job_sequence, 0);
+ INIT_LIST_HEAD(&pipe->pipe_job_running_list);
+ INIT_LIST_HEAD(&pipe->pipe_job_pending_list);
+
+ spin_lock_init(&pipe->pending_job_lock);
+ spin_lock_init(&pipe->running_job_lock);
+ mutex_init(&pipe->lock);
+
+ nodes_num = pipe->desc->total_queues;
+ nodes_size = sizeof(*pipe->nodes) * nodes_num;
+ pipe->nodes = devm_kzalloc(imgsys_dev->dev, nodes_size, GFP_KERNEL);
+ if (!pipe->nodes)
+ return -ENOMEM;
+
+ for (i = 0; i < nodes_num; i++) {
+ pipe->nodes[i].desc = &pipe->desc->queue_descs[i];
+ pipe->nodes[i].flags = pipe->nodes[i].desc->flags;
+ spin_lock_init(&pipe->nodes[i].buf_list_lock);
+ INIT_LIST_HEAD(&pipe->nodes[i].buf_list);
+
+ pipe->nodes[i].crop.left = 0;
+ pipe->nodes[i].crop.top = 0;
+ pipe->nodes[i].crop.width =
+ pipe->nodes[i].desc->frmsizeenum->stepwise.max_width;
+ pipe->nodes[i].crop.height =
+ pipe->nodes[i].desc->frmsizeenum->stepwise.max_height;
+
+ pipe->nodes[i].compose.left = 0;
+ pipe->nodes[i].compose.top = 0;
+ pipe->nodes[i].compose.width =
+ pipe->nodes[i].desc->frmsizeenum->stepwise.max_width;
+ pipe->nodes[i].compose.height =
+ pipe->nodes[i].desc->frmsizeenum->stepwise.max_height;
+ }
+
+ return 0;
+}
+
+int mtk_imgsys_pipe_release(struct mtk_imgsys_pipe *pipe)
+{
+ mutex_destroy(&pipe->lock);
+
+ return 0;
+}
+
+int mtk_imgsys_pipe_next_job_id(struct mtk_imgsys_pipe *pipe)
+{
+ int global_job_id = atomic_inc_return(&pipe->pipe_job_sequence);
+
+ return (global_job_id & 0x0000FFFF) | (pipe->desc->id << 16);
+}
+
+void mtk_imgsys_pipe_debug_job(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_request *req)
+{
+ int i;
+
+ dev_dbg(pipe->imgsys_dev->dev, "%s:%s: pipe-job(%p),id(%d)\n",
+ __func__, pipe->desc->name, req, req->id);
+
+ for (i = 0; i < pipe->desc->total_queues ; i++) {
+ if (req->buf_map[i])
+ dev_dbg(pipe->imgsys_dev->dev, "%s:%s:buf(%p)\n",
+ pipe->desc->name, pipe->nodes[i].desc->name,
+ req->buf_map[i]);
+ }
+}
+
+static unsigned int mtk_imgsys_get_stride(unsigned int width,
+ const struct mtk_imgsys_dev_format *dfmt,
+ unsigned int plane)
+{
+ unsigned int stride =
+ (width + dfmt->fmt->pixels_per_group - 1) /
+ dfmt->fmt->pixels_per_group * dfmt->fmt->bytes_per_group[plane];
+
+ return (stride + dfmt->align - 1) / dfmt->align * dfmt->align;
+}
+
+static unsigned int mtk_imgsys_get_height(unsigned int height,
+ const struct mtk_imgsys_dev_format *dfmt,
+ unsigned int plane)
+{
+ unsigned int plane_height =
+ (height + dfmt->fmt->vertical_sub_sampling[plane] - 1) /
+ dfmt->fmt->vertical_sub_sampling[plane];
+
+ return (plane_height + dfmt->scan_align - 1) / dfmt->scan_align * dfmt->scan_align;
+}
+
+void mtk_imgsys_pipe_try_fmt(struct mtk_imgsys_video_device *node,
+ struct v4l2_format *fmt,
+ const struct v4l2_format *ufmt,
+ const struct mtk_imgsys_dev_format *dfmt)
+{
+ unsigned int i;
+
+ if (!dfmt)
+ return;
+
+ fmt->type = ufmt->type;
+ fmt->fmt.pix_mp.width =
+ clamp_val(ufmt->fmt.pix_mp.width,
+ node->desc->frmsizeenum->stepwise.min_width,
+ node->desc->frmsizeenum->stepwise.max_width);
+ fmt->fmt.pix_mp.height =
+ clamp_val(ufmt->fmt.pix_mp.height,
+ node->desc->frmsizeenum->stepwise.min_height,
+ node->desc->frmsizeenum->stepwise.max_height);
+ fmt->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ fmt->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ fmt->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+ fmt->fmt.pix_mp.field = V4L2_FIELD_NONE;
+
+ fmt->fmt.pix_mp.pixelformat = dfmt->fmt->format;
+ fmt->fmt.pix_mp.num_planes = dfmt->fmt->num_planes;
+
+ for (i = 0; i < fmt->fmt.pix_mp.num_planes; ++i) {
+ struct v4l2_plane_pix_format *plane =
+ &fmt->fmt.pix_mp.plane_fmt[i];
+ const struct v4l2_plane_pix_format *uplane;
+
+ uplane = &ufmt->fmt.pix_mp.plane_fmt[i];
+ plane->bytesperline =
+ mtk_imgsys_get_stride(fmt->fmt.pix_mp.width, dfmt, i);
+ if (plane->bytesperline < uplane->bytesperline)
+ plane->bytesperline = uplane->bytesperline;
+
+ plane->sizeimage = plane->bytesperline *
+ mtk_imgsys_get_height(fmt->fmt.pix_mp.height, dfmt, i);
+
+ plane->sizeimage += dfmt->fmt->ext_data;
+ }
+}
+
+static void set_meta_fmt(struct mtk_imgsys_pipe *pipe,
+ struct v4l2_meta_format *fmt,
+ const struct mtk_imgsys_dev_format *dev_fmt)
+{
+ fmt->dataformat = dev_fmt->fmt->format;
+ fmt->buffersize = dev_fmt->fmt->buffer_size;
+}
+
+void mtk_imgsys_pipe_load_default_fmt(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_video_device *node,
+ struct v4l2_format *fmt)
+{
+ fmt->type = node->desc->buf_type;
+ if (mtk_imgsys_buf_is_meta(node->desc->buf_type)) {
+ set_meta_fmt(pipe, &fmt->fmt.meta,
+ &node->desc->fmts[0]);
+ } else {
+ fmt->fmt.pix_mp.width = node->desc->default_width;
+ fmt->fmt.pix_mp.height = node->desc->default_height;
+ mtk_imgsys_pipe_try_fmt(node, fmt, fmt,
+ &node->desc->fmts[0]);
+ }
+}
+
+const struct mtk_imgsys_dev_format*
+mtk_imgsys_pipe_find_fmt(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_video_device *node,
+ u32 format)
+{
+ int i;
+
+ for (i = 0; i < node->desc->num_fmts; i++) {
+ if (node->desc->fmts[i].fmt->format == format)
+ return &node->desc->fmts[i];
+ }
+
+ return NULL;
+}
+
+void mtk_imgsys_pipe_try_enqueue(struct mtk_imgsys_pipe *pipe)
+{
+ struct mtk_imgsys_request *req = NULL;
+ unsigned long flag;
+
+ if (!pipe->streaming)
+ return;
+
+ spin_lock_irqsave(&pipe->pending_job_lock, flag);
+ if (list_empty(&pipe->pipe_job_pending_list)) {
+ spin_unlock_irqrestore(&pipe->pending_job_lock, flag);
+ return;
+ }
+
+ req = list_first_entry(&pipe->pipe_job_pending_list, struct mtk_imgsys_request, list);
+ list_del(&req->list);
+ pipe->num_pending_jobs--;
+ spin_unlock_irqrestore(&pipe->pending_job_lock, flag);
+
+ spin_lock_irqsave(&pipe->running_job_lock, flag);
+ list_add_tail(&req->list,
+ &pipe->pipe_job_running_list);
+ pipe->num_jobs++;
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+
+ mtk_imgsys_hw_enqueue(pipe->imgsys_dev, req);
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: pending jobs(%d), running jobs(%d)\n",
+ __func__, pipe->desc->name, pipe->num_pending_jobs,
+ pipe->num_jobs);
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
new file mode 100644
index 000000000000..b69cfd0043d3
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
@@ -0,0 +1,427 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_DEV_H_
+#define _MTK_IMGSYS_DEV_H_
+
+#include <linux/completion.h>
+#include <linux/types.h>
+#include <linux/time.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/platform_device.h>
+#include <linux/pm_opp.h>
+#include <linux/regulator/consumer.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-device.h>
+#include <linux/videodev2.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_header_desc.h"
+
+#define MTK_IMGSYS_MEDIA_MODEL_NAME "MTK-ISP-DIP-V4L2"
+
+#define MTK_IMGSYS_OUTPUT_MIN_WIDTH 1U
+#define MTK_IMGSYS_OUTPUT_MIN_HEIGHT 1U
+#define MTK_IMGSYS_OUTPUT_MAX_WIDTH 5376U
+#define MTK_IMGSYS_OUTPUT_MAX_HEIGHT 4032U
+
+#define MTK_IMGSYS_CAPTURE_MIN_WIDTH 1U
+#define MTK_IMGSYS_CAPTURE_MIN_HEIGHT 1U
+#define MTK_IMGSYS_CAPTURE_MAX_WIDTH 5376U
+#define MTK_IMGSYS_CAPTURE_MAX_HEIGHT 4032U
+
+/**
+ * struct mtk_imgsys_dev_format - Device format for MediaTek Image System
+ * @fmt: Pointer to the Image System format structure
+ * @align: Alignment requirement
+ * @scan_align: Scan alignment requirement
+ *
+ * This structure defines the format used by the MediaTek Image System device.
+ * It includes a pointer to the format structure, and alignment and scan
+ * alignment requirements.
+ */
+struct mtk_imgsys_dev_format {
+ const struct mtk_imgsys_format *fmt;
+ u32 align;
+ u32 scan_align;
+};
+
+/**
+ * struct mtk_imgsys_dev_buffer - Device buffer for MediaTek Image System
+ * @vbb: Video buffer structure for V4L2
+ * @fmt: Format of the video buffer
+ * @dev_fmt: Pointer to the device-specific format structure
+ * @isp_daddr: DMA addresses for ISP processing, indexed by plane
+ * @scp_daddr: DMA addresses for SCP processing, indexed by plane
+ * @crop: Crop information for image processing
+ * @compose: Composition rectangle for output
+ * @rotation: Rotation angle for the image
+ * @hflip: Horizontal flip flag
+ * @vflip: Vertical flip flag
+ * @resize_ratio: Ratio for resizing the image
+ * @dataofst: Offset for the data buffer
+ * @list: List head for buffer management
+ *
+ * This structure represents the buffer used by the MediaTek ImgSys device.
+ * It includes various parameters for buffer management and image processing,
+ * such as format, DMA addresses, cropping, composition, rotation, flipping,
+ * resizing, and list management. This buffer is essential for handling
+ * image data in the device's processing pipeline.
+ */
+struct mtk_imgsys_dev_buffer {
+ struct vb2_v4l2_buffer vbb;
+ struct v4l2_format fmt;
+ const struct mtk_imgsys_dev_format *dev_fmt;
+ dma_addr_t isp_daddr[VB2_MAX_PLANES];
+ dma_addr_t scp_daddr[VB2_MAX_PLANES];
+ struct mtk_imgsys_crop crop;
+ struct v4l2_rect compose;
+ __u32 rotation;
+ __u32 hflip;
+ __u32 vflip;
+ __u32 resize_ratio;
+ __u32 dataofst;
+ struct list_head list;
+};
+
+/**
+ * struct mtk_imgsys_pipe_desc - Description of an image processing pipeline
+ * @name: Name of the pipeline
+ * @id: Identifier for the pipeline
+ * @queue_descs: Pointer to an array of video device descriptors
+ * @total_queues: Total number of queues in the pipeline
+ *
+ * This structure describes an image processing pipeline within the MediaTek
+ * image system. It includes the name and identifier of the pipeline, and
+ * an array of video device descriptors along with the total number of queues.
+ */
+struct mtk_imgsys_pipe_desc {
+ char *name;
+ int id;
+ struct mtk_imgsys_video_device_desc *queue_descs;
+ int total_queues;
+};
+
+/**
+ * struct mtk_imgsys_video_device_desc - Description of a video device
+ * @id: Identifier for the video device
+ * @name: Name of the video device
+ * @buf_type: Buffer type used by the video device
+ * @cap: Capabilities of the video device
+ * @smem_alloc: Using the smem_dev as alloc device or not
+ * @supports_ctrls: Flag indicating if the device supports controls
+ * @fmts: Pointer to an array of supported formats
+ * @num_fmts: Number of supported formats
+ * @description: Description of the video device
+ * @default_width: Default width of the video frames
+ * @default_height: Default height of the video frames
+ * @dma_port: DMA port used by the video device
+ * @frmsizeenum: Pointer to an array of frame size enumerations
+ * @flags: Additional flags for the video device
+ *
+ * This structure describes a video device within the MediaTek image system.
+ * It includes various properties such as the identifier, name, buffer type,
+ * capabilities, supported formats, and other relevant details.
+ */
+struct mtk_imgsys_video_device_desc {
+ int id;
+ char *name;
+ u32 buf_type;
+ u32 cap;
+ int smem_alloc;
+ int supports_ctrls;
+ const struct mtk_imgsys_dev_format *fmts;
+ int num_fmts;
+ char *description;
+ int default_width;
+ int default_height;
+ unsigned int dma_port;
+ const struct v4l2_frmsizeenum *frmsizeenum;
+ u32 flags;
+};
+
+/**
+ * struct mtk_imgsys_dev_queue - Device queue structure for image system
+ * @vbq: VB2 queue structure
+ * @lock: Mutex to serialize vb2 queue and video device operations
+ * @dev_fmt: Pointer to the device format structure
+ *
+ * This structure defines the device queue for the image system, including the
+ * VB2 queue, a mutex for serializing operations, and a pointer to the device
+ * format.
+ */
+struct mtk_imgsys_dev_queue {
+ struct vb2_queue vbq;
+ struct mutex lock; /* Protect the vb2_queue */
+ const struct mtk_imgsys_dev_format *dev_fmt;
+};
+
+/**
+ * struct mtk_imgsys_video_device -
+ * Video device structure for MediaTek Image System
+ *
+ * @vdev: Video device structure
+ * @dev_q: Device queue for buffer management
+ * @vdev_fmt: Format of the video device
+ * @vdev_pad: Media pad for the video device
+ * @pad_fmt: Media bus frame format for the pad
+ * @ctrl_handler: Control handler for V4L2 controls
+ * @resize_ratio_ctrl: Pointer to the resize ratio control
+ * @flags: Link attribute flags (e.g., MEDIA_LNK_FL_ENABLED)
+ * @desc: Pointer to the video device description structure
+ * @buf_list: List head for buffer management
+ * @crop: Crop rectangle for the video device
+ * @compose: Composition rectangle for the video device
+ * @rotation: Rotation angle for the video device
+ * @vflip: Vertical flip flag
+ * @hflip: Horizontal flip flag
+ * @resize_ratio: Resize ratio for the video device
+ * @buf_list_lock: Spinlock to protect the in-device buffer list
+ *
+ * This structure represents the video device used by the MediaTek Image System.
+ * It includes various parameters for managing video buffers, formats, media pads,
+ * controls, cropping, composition, rotation, flipping, resizing, and buffer list
+ * protection. This structure is essential for handling video data and device
+ * configuration in the MediaTek Image System.
+ */
+struct mtk_imgsys_video_device {
+ struct video_device vdev;
+ struct mtk_imgsys_dev_queue dev_q;
+ struct v4l2_format vdev_fmt;
+ struct media_pad vdev_pad;
+ struct v4l2_mbus_framefmt pad_fmt;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct v4l2_ctrl *resize_ratio_ctrl;
+ u32 flags; /* Link attribute flags, e.g. MEDIA_LNK_FL_ENABLED */
+ const struct mtk_imgsys_video_device_desc *desc;
+ struct list_head buf_list;
+ struct v4l2_rect crop;
+ struct v4l2_rect compose;
+ int rotation;
+ bool vflip;
+ bool hflip;
+ int resize_ratio;
+ /* protect the in-device buffer list */
+ spinlock_t buf_list_lock;
+};
+
+/**
+ * struct mtk_imgsys_pipe - Pipeline structure for MediaTek Image System
+ * @imgsys_dev: Pointer to the image system device
+ * @nodes: Pointer to the video device nodes associated with the pipeline
+ * @nodes_streaming: Bitmap representing streaming status of nodes
+ * @nodes_enabled: Bitmap representing enabled status of nodes
+ * @streaming: Flag indicating if the pipeline is currently streaming
+ * @subdev_pads: Media pads for subdevice connections
+ * @pipeline: Media pipeline structure
+ * @subdev: V4L2 subdevice structure
+ * @fh: File handle for subdevice operations
+ * @pipe_job_sequence: Atomic counter for job sequencing
+ * @pipe_job_pending_list: List head for pending jobs in the pipeline
+ * @num_pending_jobs: Number of pending jobs in the pipeline
+ * @pending_job_lock: To protect pipe_job_pending_list and num_pending_jobs
+ * @pipe_job_running_list: List head for running jobs in the pipeline
+ * @running_job_lock: Spinlock to protect pipe_job_running_list
+ * @num_jobs: Total number of jobs in the pipeline
+ * @lock: Mutex to serialize stream on/off and buffer enqueue operations
+ * @desc: Pointer to the pipeline description structure
+ *
+ * This structure represents a pipeline in the MediaTek Image System. It includes
+ * various components for managing the image processing pipeline, such as device
+ * pointers, video nodes, streaming flags, media pads, pipeline configuration,
+ * subdevice operations, job management, and synchronization mechanisms.
+ * This structure is essential for coordinating the flow of image data through
+ * the processing pipeline.
+ */
+struct mtk_imgsys_pipe {
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct mtk_imgsys_video_device *nodes;
+ unsigned long long nodes_streaming;
+ unsigned long long nodes_enabled;
+ int streaming;
+ struct media_pad *subdev_pads;
+ struct media_pipeline pipeline;
+ struct v4l2_subdev subdev;
+ struct v4l2_subdev_fh *fh;
+ atomic_t pipe_job_sequence;
+ struct list_head pipe_job_pending_list;
+ int num_pending_jobs;
+ /* protect pipe_job_pending_list and num_pending_jobs */
+ spinlock_t pending_job_lock;
+ struct list_head pipe_job_running_list;
+ /* protect pipe_job_running_list */
+ spinlock_t running_job_lock;
+ int num_jobs;
+ /* Serializes pipe's stream on/off and buffers enqueue operations */
+ struct mutex lock;
+ const struct mtk_imgsys_pipe_desc *desc;
+};
+
+/**
+ * struct mtk_imgsys_dev - MediaTek Image System Device Structure
+ * @dev: Pointer to the device structure.
+ * @imgsys_resource: Resource descriptor for image system hardware.
+ * @mdev: Media device structure for media controller framework integration.
+ * @v4l2_dev: V4L2 device structure for video device operations.
+ * @imgsys_pipe: Image system pipeline configuration structure.
+ * @cust_pipes: Pointer to custom pipeline descriptors.
+ * @clks: Array of clock bulk data for managing multiple clocks.
+ * @num_clks: Number of clocks in the clock array.
+ * @larbs: Array of device pointers for managing LARB (local arbiter).
+ * @num_larbs: Number of LARBs in the system.
+ * @scp: Pointer to the SCP (System Control Processor) structure.
+ * @rproc_handle: Handle for remote processor interface.
+ * @smem_dev: Device structure for shared memory operations.
+ * @num_mods: Number of modules in the image system.
+ */
+struct mtk_imgsys_dev {
+ /* Device and Resource Information */
+ struct device *dev;
+ struct resource *imgsys_resource;
+ /* Media and V4L2 Device Management */
+ struct media_device mdev;
+ struct v4l2_device v4l2_dev;
+ struct mtk_imgsys_pipe imgsys_pipe;
+ /* Imgsys Pipeline Information */
+ const struct mtk_imgsys_pipe_desc *cust_pipes;
+ /* Clock Information */
+ struct clk_bulk_data *clks;
+ int num_clks;
+ /* LARB (Local Arbitration) Control */
+ struct device **larbs;
+ unsigned int num_larbs;
+ /* SCP (System Control Processor) Integration */
+ struct mtk_scp *scp;
+ struct rproc *rproc_handle;
+ struct device *smem_dev;
+ /* Number of modules */
+ int num_mods;
+};
+
+/**
+ * struct req_frameparam - Request frame parameters for MediaTek Image System
+ * @frame_no: Frame number identifier
+ * @state: State of the frame (e.g., init, timeout)
+ *
+ * This structure represents the parameters for a request frame in the MediaTek Image System.
+ * It includes various attributes such as the frame index, frame number, timestamp, state,
+ * and the number of input and output buffers. This structure is used to manage and track
+ * the frames within the image processing pipeline.
+ */
+struct req_frameparam {
+ u32 frame_no;
+ u8 state;
+} __packed;
+
+/**
+ * struct mtk_imgsys_request - Request structure for MediaTek Image System
+ * @req: Media request structure
+ * @imgsys_pipe: Pointer to the image system pipeline
+ * @id: Unique identifier for the request
+ * @buf_map: Pointer to an array of device buffers mapped to the request
+ * @list: List head for request management
+ * @img_fparam: Frame parameters associated with the request
+ * @composer_work: Work structure for composer tasks
+ * @runner_work: Work structure for runner tasks
+ * @working_buf: Pointer to the hardware working buffer
+ * @swfrm_info: Pointer to software frame information
+ * @buf_count: Atomic counter for buffer management
+ * @request_fd: File descriptor for the request
+ *
+ * This structure represents a request in the MediaTek Image System. It includes
+ * various parameters for managing media requests, image processing pipeline,
+ * buffer mapping, frame parameters, work tasks, hardware buffers, software frame
+ * information, buffer count, and request file descriptor. This structure is
+ * essential for handling and processing image requests within the MediaTek Image
+ * System.
+ */
+struct mtk_imgsys_request {
+ struct media_request req;
+ struct mtk_imgsys_pipe *imgsys_pipe;
+ int id;
+ struct mtk_imgsys_dev_buffer **buf_map;
+ struct list_head list;
+ struct req_frameparam img_fparam;
+ atomic_t buf_count;
+ int request_fd;
+};
+
+static inline struct mtk_imgsys_video_device*
+mtk_imgsys_file_to_node(struct file *file)
+{
+ return container_of(video_devdata(file),
+ struct mtk_imgsys_video_device, vdev);
+}
+
+static inline struct mtk_imgsys_pipe*
+mtk_imgsys_subdev_to_pipe(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct mtk_imgsys_pipe, subdev);
+}
+
+static inline struct mtk_imgsys_dev*
+mtk_imgsys_mdev_to_dev(struct media_device *mdev)
+{
+ return container_of(mdev, struct mtk_imgsys_dev, mdev);
+}
+
+static inline struct mtk_imgsys_video_device*
+mtk_imgsys_vbq_to_node(struct vb2_queue *vq)
+{
+ return container_of(vq, struct mtk_imgsys_video_device, dev_q.vbq);
+}
+
+static inline struct mtk_imgsys_dev_buffer*
+mtk_imgsys_vb2_buf_to_dev_buf(struct vb2_buffer *vb)
+{
+ return container_of(vb, struct mtk_imgsys_dev_buffer, vbb.vb2_buf);
+}
+
+static inline struct mtk_imgsys_request*
+mtk_imgsys_media_req_to_imgsys_req(struct media_request *req)
+{
+ return container_of(req, struct mtk_imgsys_request, req);
+}
+
+static inline int mtk_imgsys_buf_is_meta(u32 type)
+{
+ return type == V4L2_BUF_TYPE_META_CAPTURE ||
+ type == V4L2_BUF_TYPE_META_OUTPUT;
+}
+
+int mtk_imgsys_pipe_init(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_pipe *pipe,
+ const struct mtk_imgsys_pipe_desc *setting);
+
+int mtk_imgsys_pipe_release(struct mtk_imgsys_pipe *pipe);
+
+int mtk_imgsys_pipe_next_job_id(struct mtk_imgsys_pipe *pipe);
+
+void mtk_imgsys_pipe_debug_job(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_request *req);
+
+const struct mtk_imgsys_dev_format *
+mtk_imgsys_pipe_find_fmt(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_video_device *node,
+ u32 format);
+
+void mtk_imgsys_pipe_try_fmt(struct mtk_imgsys_video_device *node,
+ struct v4l2_format *fmt,
+ const struct v4l2_format *ufmt,
+ const struct mtk_imgsys_dev_format *dfmt);
+
+void mtk_imgsys_pipe_load_default_fmt(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_video_device *node,
+ struct v4l2_format *fmt_to_fill);
+
+void mtk_imgsys_pipe_try_enqueue(struct mtk_imgsys_pipe *pipe);
+
+#endif /* _MTK_IMGSYS_DEV_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
new file mode 100644
index 000000000000..6a3a98e9cd42
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
@@ -0,0 +1,139 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * MediaTek ISP ImgSys formats
+ *
+ * Copyright (c) 2024 MediaTek Inc.
+ */
+
+#include <linux/mtkisp_imgsys.h>
+#include <linux/videodev2.h>
+#include "mtk_imgsys-formats.h"
+
+const struct mtk_imgsys_format mtk_imgsys_format_nv12 = {
+ .format = V4L2_PIX_FMT_NV12,
+ .num_planes = 2,
+ .pixels_per_group = 2,
+ .bytes_per_group = {2, 2},
+ .vertical_sub_sampling = { 1, 2 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_nv21 = {
+ .format = V4L2_PIX_FMT_NV21,
+ .num_planes = 2,
+ .pixels_per_group = 2,
+ .bytes_per_group = {2, 2},
+ .vertical_sub_sampling = { 1, 2 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_grey = {
+ .format = V4L2_PIX_FMT_GREY,
+ .num_planes = 1,
+ .pixels_per_group = 1,
+ .bytes_per_group = { 1 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_2p012p = {
+ .format = V4L2_PIX_FMT_YUV_2P012P,
+ .num_planes = 2,
+ .pixels_per_group = 64,
+ .bytes_per_group = { 96, 96 },
+ .vertical_sub_sampling = { 1, 2 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_2p010p = {
+ .format = V4L2_PIX_FMT_YUV_2P010P,
+ .num_planes = 2,
+ .pixels_per_group = 64,
+ .bytes_per_group = { 80, 80 },
+ .vertical_sub_sampling = { 1, 2 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_y8 = {
+ .format = V4L2_PIX_FMT_MTISP_Y8,
+ .num_planes = 1,
+ .pixels_per_group = 1,
+ .bytes_per_group = { 1 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_y16 = {
+ .format = V4L2_PIX_FMT_MTISP_Y16,
+ .num_planes = 1,
+ .buffer_size = 0,
+ .pixels_per_group = 1,
+ .bytes_per_group = { 2 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_y32 = {
+ .format = V4L2_PIX_FMT_MTISP_Y32,
+ .num_planes = 1,
+ .buffer_size = 0,
+ .pixels_per_group = 1,
+ .bytes_per_group = { 4 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_warp2p = {
+ .format = V4L2_PIX_FMT_WARP2P,
+ .num_planes = 2,
+ .buffer_size = 0,
+ .pixels_per_group = 4,
+ .bytes_per_group = { 8, 8 },
+ .vertical_sub_sampling = { 1, 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_sgbrg10 = {
+ .format = V4L2_PIX_FMT_MTISP_SGBRG10,
+ .num_planes = 1,
+ .buffer_size = 0,
+ .pixels_per_group = 64,
+ .bytes_per_group = { 80 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_srggb10 = {
+ .format = V4L2_PIX_FMT_MTISP_SRGGB10,
+ .num_planes = 1,
+ .buffer_size = 0,
+ .pixels_per_group = 64,
+ .bytes_per_group = { 80 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_sgrbg10 = {
+ .format = V4L2_PIX_FMT_MTISP_SGRBG10,
+ .num_planes = 1,
+ .buffer_size = 0,
+ .pixels_per_group = 64,
+ .bytes_per_group = { 80 },
+ .vertical_sub_sampling = { 1 },
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_aply8 = {
+ .format = V4L2_PIX_FMT_MTISP_APLY8,
+ .num_planes = 1,
+ .pixels_per_group = 1,
+ .bytes_per_group = { 1 },
+ .vertical_sub_sampling = { 1 },
+ .ext_data = SIZE_OF_APL_DATA,
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_metai = {
+ .format = V4L2_META_FMT_MTISP_TUN_PARAMS,
+ .num_planes = 1,
+ .buffer_size = SIZE_OF_TUNING_META,
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_stt = {
+ .format = V4L2_META_FMT_MTISP_STAT_PARAMS,
+ .num_planes = 1,
+ .buffer_size = SIZE_OF_STATISTICS_META,
+};
+
+const struct mtk_imgsys_format mtk_imgsys_format_ctrlmeta = {
+ .format = V4L2_META_FMT_MTISP_CTL_PARAMS,
+ .num_planes = 1,
+ .buffer_size = SIZE_OF_CTRL_META,
+};
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
new file mode 100644
index 000000000000..cbf8b2b414f8
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * MediaTek ISP ImgSys formats
+ *
+ * Copyright (c) 2024 MediaTek Inc.
+ */
+
+#ifndef _MTK_IMGSYS_FORMATS_H_
+#define _MTK_IMGSYS_FORMATS_H_
+
+/**
+ * struct mtk_imgsys_format - MediaTek Image System format description
+ * @format: Pixel format identifier (e.g., V4L2_PIX_FMT_*).
+ * @num_planes: Number of planes in the image format. For example, 1 for
+ * single-plane formats, 2 or 3 for multi-plane formats like YUV420.
+ * @buffer_size: Total buffer size in bytes required to store the image data.
+ * @pixels_per_group: Number of pixels per group, which s defined as the
+ * minimum number of pixels (including padding) necessary in a row when the
+ * image has only one column of effective pixels.
+ * @bytes_per_group: Array specifying the number of bytes per group for each
+ * plane. The array size is 3 to support up to 3 planes.
+ * @vertical_sub_sampling: Array specifying the vertical subsampling factor for
+ * each plane. The array size is 3 to support up to 3 planes.
+ * @ext_data: Additional data or flags related to the image format.
+ *
+ * This structure is used to describe the format of images processed by the
+ * MediaTek Image System. It includes information about the pixel format,
+ * number of planes, buffer size, and other relevant details.
+ */
+struct mtk_imgsys_format {
+ u32 format;
+ u8 num_planes;
+ u32 buffer_size;
+ u32 pixels_per_group;
+ u32 bytes_per_group[3];
+ u32 vertical_sub_sampling[3];
+ u32 ext_data;
+};
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_nv12;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_nv21;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_grey;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_2p012p;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_2p010p;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_y8;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_y16;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_y32;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_warp2p;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_sgbrg10;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_srggb10;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_sgrbg10;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_aply8;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_metai;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_stt;
+
+extern const struct mtk_imgsys_format mtk_imgsys_format_ctrlmeta;
+
+#endif /* _MTK_IMGSYS_FORMATS_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
new file mode 100644
index 000000000000..6dc4b7b60832
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_MODULES_H_
+#define _MTK_IMGSYS_MODULES_H_
+
+/**
+ * enum mtk_imgsys_module
+ *
+ * Definition about supported hw modules
+ */
+enum mtk_imgsys_module {
+ IMGSYS_MOD_IMGMAIN = 0,
+ IMGSYS_MOD_WPE,
+ IMGSYS_MOD_TRAW,
+ IMGSYS_MOD_DIP,
+ IMGSYS_MOD_PQDIP,
+ IMGSYS_MOD_ME,
+ IMGSYS_MOD_MAX,
+};
+
+#endif /* _MTK_IMGSYS_MODULES_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
new file mode 100644
index 000000000000..94888682a939
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#include "mtk_imgsys-of.h"
+
+static void merge_module_pipelines(const struct mtk_imgsys_pipe_desc *imgsys_pipe,
+ const struct cust_data *data)
+{
+ const struct mtk_imgsys_mod_pipe_desc *module_pipe;
+ unsigned int mod_num;
+ unsigned int i, j, k;
+
+ mod_num = data->mod_num;
+ k = 0;
+ for (i = 0; i < mod_num; i++) {
+ module_pipe = &data->module_pipes[i];
+ for (j = 0; j < module_pipe->node_num; j++) {
+ imgsys_pipe->queue_descs[k] = module_pipe->vnode_desc[j];
+ k++;
+ }
+ }
+}
+
+void init_imgsys_pipeline(const struct cust_data *data)
+{
+ const struct mtk_imgsys_pipe_desc *imgsys_pipe;
+ unsigned int i;
+
+ for (i = 0; i < data->pipe_num; i++) {
+ imgsys_pipe = &data->pipe_settings[i];
+ merge_module_pipelines(imgsys_pipe, data);
+ }
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
new file mode 100644
index 000000000000..5088c9df704c
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_OF_H_
+#define _MTK_IMGSYS_OF_H_
+
+#include <linux/clk.h>
+#include "mtk_imgsys-dev.h"
+
+/**
+ * struct mtk_imgsys_mod_pipe_desc - Description of a module's pipeline
+ * @vnode_desc: Pointer to an array of video device descriptors for the module.
+ * @node_num: Number of video nodes associated with this module.
+ */
+struct mtk_imgsys_mod_pipe_desc {
+ const struct mtk_imgsys_video_device_desc *vnode_desc;
+ unsigned int node_num;
+};
+
+/**
+ * struct cust_data - Custom data structure for image system configuration
+ * @clks: Pointer to an array of clock bulk data for the module.
+ * @clk_num: Number of clocks associated with this module.
+ * @module_pipes: Pointer to an array of module pipeline descriptions.
+ * @mod_num: Number of modules in the pipeline.
+ * @pipe_settings: Pointer to pipeline settings for the module.
+ * @pipe_num: Number of pipeline settings.
+ */
+struct cust_data {
+ struct clk_bulk_data *clks;
+ unsigned int clk_num;
+ const struct mtk_imgsys_mod_pipe_desc *module_pipes;
+ unsigned int mod_num;
+ const struct mtk_imgsys_pipe_desc *pipe_settings;
+ unsigned int pipe_num;
+};
+
+/**
+ * init_imgsys_pipeline - Initialize the video nodes in image system pipeline
+ * @data: Pointer to the custom data structure containing configuration
+ * information.
+ *
+ * This function initializes the image system pipeline based on the provided
+ * custom data. It sets up the necessary modules's video nodes as defined in
+ * the cust_data structure.
+ */
+void init_imgsys_pipeline(const struct cust_data *data);
+
+#endif /* _MTK_IMGSYS_OF_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
new file mode 100644
index 000000000000..9cdcdb0a0200
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -0,0 +1,27 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#include "mtk_imgsys-sys.h"
+
+void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_request *req)
+{
+ /* Function implementation will be provided in subsequent patches */
+}
+
+int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
+{
+ /* Function implementation will be provided in subsequent patches */
+ return 0;
+}
+
+int mtk_imgsys_hw_streamoff(struct mtk_imgsys_pipe *pipe)
+{
+ /* Function implementation will be provided in subsequent patches */
+ return 0;
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
new file mode 100644
index 000000000000..9f9b0b627305
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ */
+
+#ifndef _MTK_IMGSYS_SYS_H_
+#define _MTK_IMGSYS_SYS_H_
+
+#include "mtk_imgsys-dev.h"
+
+/*
+ * mtk_imgsys_hw_streamon - Start streaming on the ImgSys pipeline
+ * @pipe: Pointer to the ImgSys pipeline structure
+ *
+ * This function starts streaming on the specified ImgSys pipeline. It ensures
+ * that the pipeline is properly initialized and ready to process image data.
+ * This is typically used to begin image processing operations after the pipeline
+ * has been configured.
+ *
+ * Return: 0 on success, a negative error code on failure
+ */
+int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe);
+
+/*
+ * mtk_imgsys_hw_streamoff - Stop streaming on the ImgSys pipeline
+ * @pipe: Pointer to the ImgSys pipeline structure
+ *
+ * This function stops streaming on the specified ImgSys pipeline. It ensures
+ * that any ongoing image processing tasks are halted and the pipeline is
+ * properly shut down. This is typically used when the image processing
+ * operations are no longer needed or when the system is being reconfigured.
+ *
+ * Return: 0 on success, a negative error code on failure
+ */
+int mtk_imgsys_hw_streamoff(struct mtk_imgsys_pipe *pipe);
+
+/*
+ * mtk_imgsys_hw_enqueue - Enqueue a request to the ImgSys hardware
+ * @imgsys_dev: Pointer to the ImgSys device structure
+ * @req: Pointer to the ImgSys request structure
+ *
+ * This function enqueues a request to the ImgSys hardware for processing.
+ * The request contains information about the image processing tasks to be
+ * performed by the hardware. The function ensures that the request is properly
+ * queued and ready for execution by the ImgSys hardware.
+ */
+void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_request *req);
+
+#endif /* _MTK_IMGSYS_SYS_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
new file mode 100644
index 000000000000..cfd65858c751
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -0,0 +1,1684 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+#include <linux/platform_device.h>
+#include <linux/of_platform.h>
+#include <linux/of_device.h>
+#include <linux/module.h>
+#include <linux/pm_runtime.h>
+#include <linux/dma-mapping.h>
+#include <linux/vmalloc.h>
+#include <linux/videodev2.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-event.h>
+#include "linux/mtkisp_imgsys.h"
+#include "mtk_imgsys-of.h"
+#include "mtk_imgsys-module_ops.h"
+#include "mtk_imgsys-vnode_id.h"
+#include "mtk_imgsys_v4l2_vnode.h"
+#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-sys.h"
+
+#define IMGSYS_MAX_BUFFERS 256
+#define IMGSYS_SUSPEND_TIME 3000 /* ms */
+
+static int mtk_imgsys_sd_subscribe_event(struct v4l2_subdev *subdev,
+ struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_FRAME_SYNC:
+ return v4l2_event_subscribe(fh, sub, 64, NULL);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct v4l2_subdev_core_ops mtk_imgsys_subdev_core_ops = {
+ .subscribe_event = mtk_imgsys_sd_subscribe_event,
+ .unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+static int mtk_imgsys_subdev_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct mtk_imgsys_pipe *pipe = mtk_imgsys_subdev_to_pipe(sd);
+ int ret;
+
+ if (enable) {
+ ret = mtk_imgsys_hw_streamon(pipe);
+ if (ret)
+ dev_err(pipe->imgsys_dev->dev,
+ "%s: pipe(%d) stream on failed\n",
+ pipe->desc->name, pipe->desc->id);
+ } else {
+ ret = mtk_imgsys_hw_streamoff(pipe);
+ if (ret)
+ dev_err(pipe->imgsys_dev->dev,
+ "%s: pipe(%d) stream off failed\n",
+ pipe->desc->name, pipe->desc->id);
+ }
+
+ return ret;
+}
+
+static const struct v4l2_subdev_video_ops mtk_imgsys_subdev_video_ops = {
+ .s_stream = mtk_imgsys_subdev_s_stream,
+};
+
+static const struct v4l2_subdev_ops mtk_imgsys_subdev_ops = {
+ .core = &mtk_imgsys_subdev_core_ops,
+ .video = &mtk_imgsys_subdev_video_ops,
+};
+
+static int mtk_imgsys_link_setup(struct media_entity *entity,
+ const struct media_pad *local,
+ const struct media_pad *remote,
+ u32 flags)
+{
+ struct mtk_imgsys_pipe *pipe =
+ container_of(entity, struct mtk_imgsys_pipe, subdev.entity);
+ u32 pad = local->index;
+
+ WARN_ON(entity->obj_type != MEDIA_ENTITY_TYPE_V4L2_SUBDEV);
+ WARN_ON(pad >= pipe->desc->total_queues);
+
+ mutex_lock(&pipe->lock);
+
+ if (flags & MEDIA_LNK_FL_ENABLED)
+ pipe->nodes_enabled++;
+ else
+ pipe->nodes_enabled--;
+
+ pipe->nodes[pad].flags &= ~MEDIA_LNK_FL_ENABLED;
+ pipe->nodes[pad].flags |= flags & MEDIA_LNK_FL_ENABLED;
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s: link setup, flags(0x%x), (%s)%d -->(%s)%d, nodes_enabled(0x%llx)\n",
+ pipe->desc->name, flags, local->entity->name, local->index,
+ remote->entity->name, remote->index, pipe->nodes_enabled);
+
+ mutex_unlock(&pipe->lock);
+
+ return 0;
+}
+
+static const struct media_entity_operations mtk_imgsys_media_ops = {
+ .link_setup = mtk_imgsys_link_setup,
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static int mtk_imgsys_vb2_meta_buf_prepare(struct vb2_buffer *vb)
+{
+ struct mtk_imgsys_pipe *pipe = vb2_get_drv_priv(vb->vb2_queue);
+ struct mtk_imgsys_video_device *node =
+ mtk_imgsys_vbq_to_node(vb->vb2_queue);
+ struct device *dev = pipe->imgsys_dev->dev;
+ const struct v4l2_format *fmt = &node->vdev_fmt;
+
+ if (vb->planes[0].length < fmt->fmt.meta.buffersize) {
+ dev_err(dev,
+ "%s:%s: size error(user:%d, required:%d)\n",
+ pipe->desc->name, node->desc->name,
+ vb->planes[0].length, fmt->fmt.meta.buffersize);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int mtk_imgsys_vb2_video_buf_prepare(struct vb2_buffer *vb)
+{
+ struct mtk_imgsys_pipe *pipe = vb2_get_drv_priv(vb->vb2_queue);
+ struct mtk_imgsys_video_device *node =
+ mtk_imgsys_vbq_to_node(vb->vb2_queue);
+ struct device *dev = pipe->imgsys_dev->dev;
+ const struct v4l2_format *fmt = &node->vdev_fmt;
+ unsigned int size;
+ int i;
+
+ for (i = 0; i < vb->num_planes; i++) {
+ size = fmt->fmt.pix_mp.plane_fmt[i].sizeimage;
+ if (vb->planes[i].length < size) {
+ dev_err(dev,
+ "%s:%s: size error(user:%d, required:%d)\n",
+ pipe->desc->name, node->desc->name,
+ vb->planes[i].length, size);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int mtk_imgsys_vb2_buf_out_validate(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+ if (v4l2_buf->field == V4L2_FIELD_ANY)
+ v4l2_buf->field = V4L2_FIELD_NONE;
+
+ if (v4l2_buf->field != V4L2_FIELD_NONE)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int mtk_imgsys_vb2_meta_buf_init(struct vb2_buffer *vb)
+{
+ struct mtk_imgsys_dev_buffer *dev_buf =
+ mtk_imgsys_vb2_buf_to_dev_buf(vb);
+ struct mtk_imgsys_pipe *pipe = vb2_get_drv_priv(vb->vb2_queue);
+ struct mtk_imgsys_video_device *node =
+ mtk_imgsys_vbq_to_node(vb->vb2_queue);
+
+ /*
+ * The meta buffers are allocated from the SCP reserved memory.
+ * Setup the mapping to ImgSys device here.
+ */
+ dev_buf->scp_daddr[0] = vb2_dma_contig_plane_dma_addr(vb, 0);
+ dev_buf->isp_daddr[0] =
+ dma_map_resource(pipe->imgsys_dev->dev,
+ dev_buf->scp_daddr[0],
+ vb->planes[0].length,
+ DMA_BIDIRECTIONAL,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(pipe->imgsys_dev->dev,
+ dev_buf->isp_daddr[0])) {
+ dev_err(pipe->imgsys_dev->dev,
+ "%s:%s: failed to map buffer: s_daddr(%pad)\n",
+ pipe->desc->name, node->desc->name,
+ &dev_buf->scp_daddr[0]);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int mtk_imgsys_vb2_video_buf_init(struct vb2_buffer *vb)
+{
+ struct mtk_imgsys_dev_buffer *dev_buf =
+ mtk_imgsys_vb2_buf_to_dev_buf(vb);
+ unsigned int plane;
+
+ for (plane = 0; plane < vb->num_planes; ++plane)
+ dev_buf->isp_daddr[plane] = vb2_dma_contig_plane_dma_addr(vb, plane);
+
+ return 0;
+}
+
+static void mtk_imgsys_vb2_queue_meta_buf_cleanup(struct vb2_buffer *vb)
+{
+ struct mtk_imgsys_dev_buffer *dev_buf =
+ mtk_imgsys_vb2_buf_to_dev_buf(vb);
+ struct mtk_imgsys_pipe *pipe = vb2_get_drv_priv(vb->vb2_queue);
+
+ dma_unmap_resource(pipe->imgsys_dev->dev,
+ dev_buf->isp_daddr[0],
+ vb->planes[0].length, DMA_BIDIRECTIONAL,
+ DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_imgsys_vb2_buf_queue(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *b = to_vb2_v4l2_buffer(vb);
+ struct mtk_imgsys_dev_buffer *dev_buf =
+ mtk_imgsys_vb2_buf_to_dev_buf(vb);
+ struct mtk_imgsys_request *req = NULL;
+ struct mtk_imgsys_video_device *node =
+ mtk_imgsys_vbq_to_node(vb->vb2_queue);
+ int buf_count;
+
+ if (!vb->request)
+ return;
+
+ req = mtk_imgsys_media_req_to_imgsys_req(vb->request);
+
+ dev_buf->dev_fmt = node->dev_q.dev_fmt;
+ spin_lock(&node->buf_list_lock);
+ list_add_tail(&dev_buf->list, &node->buf_list);
+ spin_unlock(&node->buf_list_lock);
+
+ buf_count = atomic_dec_return(&req->buf_count);
+ if (!buf_count) {
+ dev_dbg(&node->vdev.dev,
+ "framo_no: (%d), reqfd-%d\n",
+ req->img_fparam.frame_no, b->request_fd);
+ req->request_fd = b->request_fd;
+ mutex_lock(&req->imgsys_pipe->lock);
+ mtk_imgsys_pipe_try_enqueue(req->imgsys_pipe);
+ mutex_unlock(&req->imgsys_pipe->lock);
+ }
+}
+
+static int mtk_imgsys_vb2_meta_queue_setup(struct vb2_queue *vq,
+ unsigned int *num_buffers,
+ unsigned int *num_planes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_vbq_to_node(vq);
+ const struct v4l2_format *fmt = &node->vdev_fmt;
+
+ if (*num_planes)
+ return 0;
+
+ *num_planes = 1;
+ sizes[0] = fmt->fmt.meta.buffersize;
+
+ return 0;
+}
+
+static int mtk_imgsys_vb2_video_queue_setup(struct vb2_queue *vq,
+ unsigned int *num_buffers,
+ unsigned int *num_planes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_vbq_to_node(vq);
+ const struct v4l2_format *fmt = &node->vdev_fmt;
+ int i;
+
+ if (*num_planes)
+ return 0;
+
+ *num_planes = fmt->fmt.pix_mp.num_planes;
+
+ for (i = 0; i < *num_planes; i++) {
+ sizes[i] = fmt->fmt.pix_mp.plane_fmt[i].sizeimage;
+ *num_buffers = clamp_val(*num_buffers, 1, IMGSYS_MAX_BUFFERS);
+ }
+ return 0;
+}
+
+static void mtk_imgsys_return_all_buffers(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_video_device *node,
+ enum vb2_buffer_state state)
+{
+ struct mtk_imgsys_dev_buffer *b, *b0;
+
+ spin_lock(&node->buf_list_lock);
+ list_for_each_entry_safe(b, b0, &node->buf_list, list) {
+ list_del(&b->list);
+ vb2_buffer_done(&b->vbb.vb2_buf, state);
+ }
+ spin_unlock(&node->buf_list_lock);
+}
+
+static int mtk_imgsys_vb2_start_streaming(struct vb2_queue *vq,
+ unsigned int count)
+{
+ struct mtk_imgsys_pipe *pipe = vb2_get_drv_priv(vq);
+ struct mtk_imgsys_video_device *node = mtk_imgsys_vbq_to_node(vq);
+ int ret;
+
+ if (!pipe->nodes_streaming) {
+ ret = media_pipeline_start(&node->vdev.entity.pads[0], &pipe->pipeline);
+ if (ret < 0) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s: media_pipeline_start failed(%d)\n",
+ pipe->desc->name, node->desc->name, ret);
+ goto fail_return_bufs;
+ }
+ }
+
+ mutex_lock(&pipe->lock);
+ if (!(node->flags & MEDIA_LNK_FL_ENABLED)) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s: stream on failed, node is not enabled\n",
+ pipe->desc->name, node->desc->name);
+
+ ret = -ENOLINK;
+ goto fail_stop_pipeline;
+ }
+
+ pipe->nodes_streaming++;
+ if (pipe->nodes_streaming == pipe->nodes_enabled) {
+ /* Start streaming of the whole pipeline */
+ ret = v4l2_subdev_call(&pipe->subdev, video, s_stream, 1);
+ if (ret < 0) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s: sub dev s_stream(1) failed(%d)\n",
+ pipe->desc->name, node->desc->name, ret);
+
+ goto fail_stop_pipeline;
+ }
+ }
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s:%s nodes_streaming(0x%llx), nodes_enable(0x%llx)\n",
+ __func__, pipe->desc->name, node->desc->name,
+ pipe->nodes_streaming, pipe->nodes_enabled);
+
+ mutex_unlock(&pipe->lock);
+
+ return 0;
+
+fail_stop_pipeline:
+ mutex_unlock(&pipe->lock);
+ media_pipeline_stop(&node->vdev.entity.pads[0]);
+
+fail_return_bufs:
+ mtk_imgsys_return_all_buffers(pipe, node, VB2_BUF_STATE_QUEUED);
+
+ return ret;
+}
+
+static void mtk_imgsys_vb2_stop_streaming(struct vb2_queue *vq)
+{
+ struct mtk_imgsys_pipe *pipe = vb2_get_drv_priv(vq);
+ struct mtk_imgsys_video_device *node = mtk_imgsys_vbq_to_node(vq);
+ int ret;
+
+ mutex_lock(&pipe->lock);
+
+ if (pipe->streaming) {
+ ret = v4l2_subdev_call(&pipe->subdev, video, s_stream, 0);
+ if (ret)
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s: sub dev s_stream(0) failed(%d)\n",
+ pipe->desc->name, node->desc->name, ret);
+ }
+
+ pipe->nodes_streaming--;
+
+ if (!pipe->nodes_streaming)
+ media_pipeline_stop(&node->vdev.entity.pads[0]);
+
+ mtk_imgsys_return_all_buffers(pipe, node, VB2_BUF_STATE_ERROR);
+
+ mutex_unlock(&pipe->lock);
+}
+
+static void mtk_imgsys_vb2_request_complete(struct vb2_buffer *vb)
+{
+ struct mtk_imgsys_video_device *node =
+ mtk_imgsys_vbq_to_node(vb->vb2_queue);
+
+ v4l2_ctrl_request_complete(vb->req_obj.req,
+ &node->ctrl_handler);
+}
+
+static const struct vb2_ops mtk_imgsys_vb2_meta_ops = {
+ .buf_queue = mtk_imgsys_vb2_buf_queue,
+ .queue_setup = mtk_imgsys_vb2_meta_queue_setup,
+ .buf_init = mtk_imgsys_vb2_meta_buf_init,
+ .buf_prepare = mtk_imgsys_vb2_meta_buf_prepare,
+ .buf_out_validate = mtk_imgsys_vb2_buf_out_validate,
+ .buf_cleanup = mtk_imgsys_vb2_queue_meta_buf_cleanup,
+ .start_streaming = mtk_imgsys_vb2_start_streaming,
+ .stop_streaming = mtk_imgsys_vb2_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .buf_request_complete = mtk_imgsys_vb2_request_complete,
+};
+
+static const struct vb2_ops mtk_imgsys_vb2_video_ops = {
+ .buf_queue = mtk_imgsys_vb2_buf_queue,
+ .queue_setup = mtk_imgsys_vb2_video_queue_setup,
+ .buf_init = mtk_imgsys_vb2_video_buf_init,
+ .buf_prepare = mtk_imgsys_vb2_video_buf_prepare,
+ .buf_out_validate = mtk_imgsys_vb2_buf_out_validate,
+ .start_streaming = mtk_imgsys_vb2_start_streaming,
+ .stop_streaming = mtk_imgsys_vb2_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .buf_request_complete = mtk_imgsys_vb2_request_complete,
+};
+
+static int mtk_imgsys_video_device_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct mtk_imgsys_video_device *node =
+ container_of(ctrl->handler,
+ struct mtk_imgsys_video_device, ctrl_handler);
+
+ switch (ctrl->id) {
+ case V4L2_CID_ROTATE:
+ node->rotation = ctrl->val;
+ break;
+ case V4L2_CID_VFLIP:
+ node->vflip = ctrl->val;
+ break;
+ case V4L2_CID_HFLIP:
+ node->hflip = ctrl->val;
+ break;
+ case V4L2_CID_MTK_IMG_RESIZE_RATIO:
+ node->resize_ratio = ctrl->val;
+ break;
+ default:
+ dev_dbg(&node->vdev.dev, "[%s] doesn't support ctrl(%d)\n",
+ node->desc->name, ctrl->id);
+ return -EINVAL;
+ }
+
+ node->rotation = ctrl->val;
+
+ return 0;
+}
+
+static const struct v4l2_ctrl_ops mtk_imgsys_video_device_ctrl_ops = {
+ .s_ctrl = mtk_imgsys_video_device_s_ctrl,
+};
+
+static int mtk_imgsys_vidioc_qbuf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct mtk_imgsys_pipe *pipe = video_drvdata(file);
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+ struct vb2_buffer *vb = node->dev_q.vbq.bufs[buf->index];
+ struct mtk_imgsys_dev_buffer *dev_buf =
+ mtk_imgsys_vb2_buf_to_dev_buf(vb);
+
+ if (!dev_buf)
+ return -EFAULT;
+
+ dev_buf->fmt = node->vdev_fmt;
+ dev_buf->rotation = node->rotation;
+ dev_buf->crop.c = node->crop;
+ dev_buf->compose = node->compose;
+ dev_buf->hflip = node->hflip;
+ dev_buf->vflip = node->vflip;
+ dev_buf->resize_ratio = node->resize_ratio;
+
+ return vb2_qbuf(node->vdev.queue, &pipe->imgsys_dev->mdev, buf);
+}
+
+static int mtk_imgsys_videoc_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ struct mtk_imgsys_pipe *pipe = video_drvdata(file);
+
+ strscpy(cap->driver, pipe->desc->name, sizeof(cap->driver));
+ strscpy(cap->card, pipe->desc->name, sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info),
+ "platform:%s", dev_name(pipe->imgsys_dev->mdev.dev));
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_try_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_imgsys_pipe *pipe = video_drvdata(file);
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+ const struct mtk_imgsys_dev_format *dev_fmt;
+ struct v4l2_format try_fmt;
+
+ memset(&try_fmt, 0, sizeof(try_fmt));
+
+ dev_fmt = mtk_imgsys_pipe_find_fmt(pipe, node,
+ f->fmt.pix_mp.pixelformat);
+ if (!dev_fmt) {
+ dev_fmt = &node->desc->fmts[0];
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s:%s: dev_fmt(%d) not found, use default(%d)\n",
+ __func__, pipe->desc->name, node->desc->name,
+ f->fmt.pix_mp.pixelformat, dev_fmt->fmt->format);
+ }
+
+ mtk_imgsys_pipe_try_fmt(node, &try_fmt, f, dev_fmt);
+ *f = try_fmt;
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_g_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+
+ *f = node->vdev_fmt;
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_s_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+ struct mtk_imgsys_pipe *pipe = video_drvdata(file);
+ const struct mtk_imgsys_dev_format *dev_fmt;
+
+ dev_fmt = mtk_imgsys_pipe_find_fmt(pipe, node,
+ f->fmt.pix_mp.pixelformat);
+ if (!dev_fmt) {
+ dev_fmt = &node->desc->fmts[0];
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s:%s: dev_fmt(%d) not found, use default(%d)\n",
+ __func__, pipe->desc->name, node->desc->name,
+ f->fmt.pix_mp.pixelformat, dev_fmt->fmt->format);
+ }
+
+ memset(&node->vdev_fmt, 0, sizeof(node->vdev_fmt));
+
+ mtk_imgsys_pipe_try_fmt(node, &node->vdev_fmt, f, dev_fmt);
+ *f = node->vdev_fmt;
+
+ node->dev_q.dev_fmt = dev_fmt;
+ node->crop.left = 0; /* reset crop setting of nodes */
+ node->crop.top = 0;
+ node->crop.width = f->fmt.pix_mp.width;
+ node->crop.height = f->fmt.pix_mp.height;
+ node->compose.left = 0;
+ node->compose.top = 0;
+ node->compose.width = f->fmt.pix_mp.width;
+ node->compose.height = f->fmt.pix_mp.height;
+ if (node->resize_ratio_ctrl)
+ v4l2_ctrl_s_ctrl(node->resize_ratio_ctrl, 0);
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *sizes)
+{
+ struct mtk_imgsys_pipe *pipe = video_drvdata(file);
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+ const struct mtk_imgsys_dev_format *dev_fmt;
+
+ dev_fmt = mtk_imgsys_pipe_find_fmt(pipe, node, sizes->pixel_format);
+
+ if (!dev_fmt || sizes->index)
+ return -EINVAL;
+
+ sizes->type = node->desc->frmsizeenum->type;
+ sizes->stepwise.max_width =
+ node->desc->frmsizeenum->stepwise.max_width;
+ sizes->stepwise.min_width =
+ node->desc->frmsizeenum->stepwise.min_width;
+ sizes->stepwise.max_height =
+ node->desc->frmsizeenum->stepwise.max_height;
+ sizes->stepwise.min_height =
+ node->desc->frmsizeenum->stepwise.min_height;
+ sizes->stepwise.step_height =
+ node->desc->frmsizeenum->stepwise.step_height;
+ sizes->stepwise.step_width =
+ node->desc->frmsizeenum->stepwise.step_width;
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_enum_fmt(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+
+ if (f->index >= node->desc->num_fmts)
+ return -EINVAL;
+
+ strscpy(f->description, node->desc->description, sizeof(f->description));
+ f->pixelformat = node->desc->fmts[f->index].fmt->format;
+ f->flags = 0;
+
+ return 0;
+}
+
+static int mtk_imgsys_meta_enum_format(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+
+ if (f->index > 0)
+ return -EINVAL;
+
+ strscpy(f->description, node->desc->description, sizeof(f->description));
+
+ f->pixelformat = node->vdev_fmt.fmt.meta.dataformat;
+ f->flags = 0;
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_g_meta_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+ *f = node->vdev_fmt;
+
+ return 0;
+}
+
+static int mtk_imgsys_videoc_s_meta_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+ struct mtk_imgsys_pipe *pipe = video_drvdata(file);
+ const struct mtk_imgsys_dev_format *dev_fmt;
+
+ if (pipe->streaming || vb2_is_busy(&node->dev_q.vbq))
+ return -EBUSY;
+
+ dev_fmt = mtk_imgsys_pipe_find_fmt(pipe, node,
+ f->fmt.meta.dataformat);
+ if (!dev_fmt) {
+ dev_fmt = &node->desc->fmts[0];
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s:%s: dev_fmt(%d) not found, use default(%d)\n",
+ __func__, pipe->desc->name, node->desc->name,
+ f->fmt.meta.dataformat, dev_fmt->fmt->format);
+ }
+
+ memset(&node->vdev_fmt, 0, sizeof(node->vdev_fmt));
+
+ f->fmt.meta.dataformat = dev_fmt->fmt->format;
+ f->fmt.meta.buffersize = dev_fmt->fmt->buffer_size;
+
+ node->dev_q.dev_fmt = dev_fmt;
+ node->vdev_fmt = *f;
+
+ return 0;
+}
+
+static int mtk_imgsys_g_selection(struct file *file, void *priv,
+ struct v4l2_selection *s)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+
+ if (!node)
+ return -EINVAL;
+
+ if ((node->desc->buf_type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT) ||
+ (node->desc->buf_type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE))
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ s->r = node->crop;
+ break;
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ s->r.width = node->desc->default_width;
+ s->r.height = node->desc->default_width;
+ s->r.left = 0;
+ s->r.top = 0;
+ break;
+ default:
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static int mtk_imgsys_s_selection(struct file *file, void *priv,
+ struct v4l2_selection *s)
+{
+ struct mtk_imgsys_video_device *node = mtk_imgsys_file_to_node(file);
+
+ if (!node)
+ return -EINVAL;
+
+ if ((node->desc->buf_type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT) ||
+ (node->desc->buf_type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE))
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ node->crop = s->r;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops mtk_imgsys_v4l2_video_out_ioctl_ops = {
+ .vidioc_querycap = mtk_imgsys_videoc_querycap,
+
+ .vidioc_enum_framesizes = mtk_imgsys_videoc_enum_framesizes,
+ .vidioc_enum_fmt_vid_out = mtk_imgsys_videoc_enum_fmt,
+ .vidioc_g_fmt_vid_out_mplane = mtk_imgsys_videoc_g_fmt,
+ .vidioc_s_fmt_vid_out_mplane = mtk_imgsys_videoc_s_fmt,
+ .vidioc_try_fmt_vid_out_mplane = mtk_imgsys_videoc_try_fmt,
+
+ .vidioc_g_selection = mtk_imgsys_g_selection,
+ .vidioc_s_selection = mtk_imgsys_s_selection,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_remove_bufs = vb2_ioctl_remove_bufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = mtk_imgsys_vidioc_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_imgsys_v4l2_video_cap_ioctl_ops = {
+ .vidioc_querycap = mtk_imgsys_videoc_querycap,
+
+ .vidioc_enum_framesizes = mtk_imgsys_videoc_enum_framesizes,
+ .vidioc_enum_fmt_vid_cap = mtk_imgsys_videoc_enum_fmt,
+ .vidioc_g_fmt_vid_cap_mplane = mtk_imgsys_videoc_g_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = mtk_imgsys_videoc_s_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = mtk_imgsys_videoc_try_fmt,
+
+ .vidioc_g_selection = mtk_imgsys_g_selection,
+ .vidioc_s_selection = mtk_imgsys_s_selection,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_remove_bufs = vb2_ioctl_remove_bufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = mtk_imgsys_vidioc_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+
+ .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static const struct v4l2_ioctl_ops mtk_imgsys_v4l2_meta_out_ioctl_ops = {
+ .vidioc_querycap = mtk_imgsys_videoc_querycap,
+
+ .vidioc_enum_fmt_meta_out = mtk_imgsys_meta_enum_format,
+ .vidioc_g_fmt_meta_out = mtk_imgsys_videoc_g_meta_fmt,
+ .vidioc_s_fmt_meta_out = mtk_imgsys_videoc_s_meta_fmt,
+ .vidioc_try_fmt_meta_out = mtk_imgsys_videoc_g_meta_fmt,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_remove_bufs = vb2_ioctl_remove_bufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = mtk_imgsys_vidioc_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static const struct v4l2_ioctl_ops mtk_imgsys_v4l2_meta_cap_ioctl_ops = {
+ .vidioc_querycap = mtk_imgsys_videoc_querycap,
+
+ .vidioc_enum_fmt_meta_cap = mtk_imgsys_meta_enum_format,
+ .vidioc_g_fmt_meta_cap = mtk_imgsys_videoc_g_meta_fmt,
+ .vidioc_s_fmt_meta_cap = mtk_imgsys_videoc_s_meta_fmt,
+ .vidioc_try_fmt_meta_cap = mtk_imgsys_videoc_g_meta_fmt,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_remove_bufs = vb2_ioctl_remove_bufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = mtk_imgsys_vidioc_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static struct media_request *mtk_imgsys_request_alloc(struct media_device *mdev)
+{
+ struct mtk_imgsys_request *req;
+ struct mtk_imgsys_pipe *pipe;
+
+ pipe = &mtk_imgsys_mdev_to_dev(mdev)->imgsys_pipe;
+ req = vzalloc(sizeof(*req));
+ if (!req)
+ return NULL;
+
+ req->buf_map = vzalloc(pipe->desc->total_queues *
+ sizeof(struct mtk_imgsys_dev_buffer *));
+ if (!req->buf_map)
+ goto error;
+
+ return &req->req;
+
+error:
+ vfree(req);
+ return NULL;
+}
+
+static void mtk_imgsys_request_free(struct media_request *req)
+{
+ struct mtk_imgsys_request *imgsys_req =
+ mtk_imgsys_media_req_to_imgsys_req(req);
+
+ vfree(imgsys_req->buf_map);
+ vfree(imgsys_req);
+}
+
+static int mtk_imgsys_vb2_request_validate(struct media_request *req)
+{
+ struct media_request_object *obj;
+ struct mtk_imgsys_dev *imgsys_dev = mtk_imgsys_mdev_to_dev(req->mdev);
+ struct mtk_imgsys_request *imgsys_req =
+ mtk_imgsys_media_req_to_imgsys_req(req);
+ struct mtk_imgsys_pipe *pipe = NULL;
+ struct mtk_imgsys_pipe *pipe_prev = NULL;
+ struct mtk_imgsys_dev_buffer **buf_map = imgsys_req->buf_map;
+ int buf_count = 0;
+ int i;
+
+ for (i = 0; i < imgsys_dev->imgsys_pipe.desc->total_queues; i++)
+ buf_map[i] = NULL;
+
+ list_for_each_entry(obj, &req->objects, list) {
+ struct vb2_buffer *vb;
+ struct mtk_imgsys_dev_buffer *dev_buf;
+ struct mtk_imgsys_video_device *node;
+
+ if (!vb2_request_object_is_buffer(obj))
+ continue;
+
+ vb = container_of(obj, struct vb2_buffer, req_obj);
+ node = mtk_imgsys_vbq_to_node(vb->vb2_queue);
+ pipe = vb2_get_drv_priv(vb->vb2_queue);
+ if (pipe_prev && pipe != pipe_prev) {
+ dev_dbg(imgsys_dev->dev,
+ "%s:%s:%s:found buf of different pipes(%p,%p)\n",
+ __func__, node->desc->name,
+ req->debug_str, pipe, pipe_prev);
+ return -EINVAL;
+ }
+
+ pipe_prev = pipe;
+ dev_buf = mtk_imgsys_vb2_buf_to_dev_buf(vb);
+ imgsys_req->buf_map[node->desc->id] = dev_buf;
+ buf_count++;
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: added buf(%p) to pipe-job(%p), buf_count(%d)\n",
+ pipe->desc->name, node->desc->name, dev_buf,
+ imgsys_req, buf_count);
+ }
+
+ if (!pipe) {
+ dev_dbg(imgsys_dev->dev,
+ "%s: no buffer in the request(%p)\n",
+ req->debug_str, req);
+
+ return -ENOENT;
+ }
+
+ atomic_set(&imgsys_req->buf_count, buf_count);
+ imgsys_req->id = mtk_imgsys_pipe_next_job_id(pipe);
+ imgsys_req->imgsys_pipe = pipe;
+ mtk_imgsys_pipe_debug_job(pipe, imgsys_req);
+
+ return vb2_request_validate(req);
+}
+
+static void mtk_imgsys_vb2_request_queue(struct media_request *req)
+{
+ struct mtk_imgsys_request *imgsys_req =
+ mtk_imgsys_media_req_to_imgsys_req(req);
+ struct mtk_imgsys_pipe *pipe = imgsys_req->imgsys_pipe;
+ unsigned long flag;
+
+ spin_lock_irqsave(&pipe->pending_job_lock, flag);
+ list_add_tail(&imgsys_req->list, &pipe->pipe_job_pending_list);
+ pipe->num_pending_jobs++;
+ dev_dbg(req->mdev->dev,
+ "%s:%s: current num of pending jobs(%d)\n",
+ __func__, pipe->desc->name, pipe->num_pending_jobs);
+ spin_unlock_irqrestore(&pipe->pending_job_lock, flag);
+ vb2_request_queue(req);
+}
+
+static const struct media_device_ops mtk_imgsys_media_req_ops = {
+ .req_validate = mtk_imgsys_vb2_request_validate,
+ .req_queue = mtk_imgsys_vb2_request_queue,
+ .req_alloc = mtk_imgsys_request_alloc,
+ .req_free = mtk_imgsys_request_free,
+};
+
+static const struct v4l2_file_operations mtk_imgsys_v4l2_fops = {
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+static int mtk_imgsys_dev_media_register(struct device *dev,
+ struct media_device *media_dev)
+{
+ int ret;
+
+ media_dev->dev = dev;
+ strscpy(media_dev->model, MTK_IMGSYS_MEDIA_MODEL_NAME, sizeof(media_dev->model));
+ snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
+ "platform:%s", dev_name(dev));
+ media_dev->hw_revision = 0;
+ media_dev->ops = &mtk_imgsys_media_req_ops;
+ media_device_init(media_dev);
+
+ ret = media_device_register(media_dev);
+ if (ret)
+ media_device_cleanup(media_dev);
+
+ return ret;
+}
+
+static int mtk_imgsys_video_device_v4l2_register(struct mtk_imgsys_pipe *pipe,
+ struct mtk_imgsys_video_device *node)
+{
+ struct vb2_queue *vbq = &node->dev_q.vbq;
+ struct video_device *vdev = &node->vdev;
+ struct media_link *link;
+ int ret;
+
+ switch (node->desc->buf_type) {
+ case V4L2_BUF_TYPE_META_OUTPUT:
+ vbq->ops = &mtk_imgsys_vb2_meta_ops;
+ vdev->ioctl_ops = &mtk_imgsys_v4l2_meta_out_ioctl_ops;
+ break;
+ case V4L2_BUF_TYPE_META_CAPTURE:
+ vbq->ops = &mtk_imgsys_vb2_meta_ops;
+ vdev->ioctl_ops = &mtk_imgsys_v4l2_meta_cap_ioctl_ops;
+ break;
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+ vbq->ops = &mtk_imgsys_vb2_video_ops;
+ vdev->ioctl_ops = &mtk_imgsys_v4l2_video_out_ioctl_ops;
+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ vbq->ops = &mtk_imgsys_vb2_video_ops;
+ vdev->ioctl_ops = &mtk_imgsys_v4l2_video_cap_ioctl_ops;
+ break;
+ default:
+ dev_info(pipe->imgsys_dev->dev,
+ "unexpected buf_type %u\n", node->desc->buf_type);
+ return -EFAULT;
+ }
+
+ mutex_init(&node->dev_q.lock);
+ vdev->device_caps = node->desc->cap;
+ node->vdev_fmt.type = node->desc->buf_type;
+ mtk_imgsys_pipe_load_default_fmt(pipe, node, &node->vdev_fmt);
+
+ ret = media_entity_pads_init(&vdev->entity, 1, &node->vdev_pad);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "failed initialize media entity (%d)\n", ret);
+ goto err_mutex_destroy;
+ }
+
+ node->vdev_pad.flags = V4L2_TYPE_IS_OUTPUT(node->desc->buf_type) ?
+ MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+
+ vbq->type = node->vdev_fmt.type;
+ vbq->io_modes = VB2_MMAP | VB2_DMABUF;
+ vbq->mem_ops = &vb2_dma_contig_memops;
+ vbq->supports_requests = true;
+ vbq->requires_requests = true;
+ vbq->buf_struct_size = sizeof(struct mtk_imgsys_dev_buffer);
+ vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ vbq->min_queued_buffers = 0;
+ vbq->drv_priv = pipe;
+ vbq->lock = &node->dev_q.lock;
+ vbq->max_num_buffers = IMGSYS_MAX_BUFFERS;
+
+ ret = vb2_queue_init(vbq);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s:%s: failed to init vb2 queue(%d)\n",
+ __func__, pipe->desc->name, node->desc->name,
+ ret);
+ goto err_media_entity_cleanup;
+ }
+
+ strscpy(vbq->name, node->desc->name, sizeof(vbq->name));
+
+ snprintf(vdev->name, sizeof(vdev->name), "%s %s", pipe->desc->name,
+ node->desc->name);
+ vdev->entity.name = vdev->name;
+ vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+ vdev->entity.ops = NULL;
+ vdev->release = video_device_release_empty;
+ vdev->fops = &mtk_imgsys_v4l2_fops;
+ vdev->lock = &node->dev_q.lock;
+ if (node->desc->supports_ctrls)
+ vdev->ctrl_handler = &node->ctrl_handler;
+ else
+ vdev->ctrl_handler = NULL;
+ vdev->v4l2_dev = &pipe->imgsys_dev->v4l2_dev;
+ vdev->queue = &node->dev_q.vbq;
+ vdev->vfl_dir = V4L2_TYPE_IS_OUTPUT(node->desc->buf_type) ?
+ VFL_DIR_TX : VFL_DIR_RX;
+
+ vdev->queue->dev = pipe->imgsys_dev->dev;
+
+ if (node->desc->smem_alloc) {
+ vdev->queue->dev = pipe->imgsys_dev->smem_dev;
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: select smem_vb2_alloc_ctx(%p)\n",
+ pipe->desc->name, node->desc->name,
+ vdev->queue->dev);
+ } else {
+ vdev->queue->dev = pipe->imgsys_dev->dev;
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: select default_vb2_alloc_ctx(%p)\n",
+ pipe->desc->name, node->desc->name,
+ pipe->imgsys_dev->dev);
+ }
+
+ video_set_drvdata(vdev, pipe);
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "failed to register video device (%d)\n", ret);
+ goto err_vb2_queue_release;
+ }
+ dev_dbg(pipe->imgsys_dev->dev, "registered vdev: %s\n",
+ vdev->name);
+
+ if (V4L2_TYPE_IS_OUTPUT(node->desc->buf_type))
+ ret = media_create_pad_link(&vdev->entity, 0,
+ &pipe->subdev.entity,
+ node->desc->id, node->flags);
+ else
+ ret = media_create_pad_link(&pipe->subdev.entity,
+ node->desc->id, &vdev->entity,
+ 0, node->flags);
+ if (ret)
+ goto err_video_unregister_device;
+
+ vdev->intf_devnode = media_devnode_create(&pipe->imgsys_dev->mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, vdev->minor);
+ if (!vdev->intf_devnode) {
+ ret = -ENOMEM;
+ goto err_rm_links;
+ }
+
+ link = media_create_intf_link(&vdev->entity,
+ &vdev->intf_devnode->intf,
+ node->flags);
+ if (!link) {
+ ret = -ENOMEM;
+ goto err_rm_devnode;
+ }
+
+ return 0;
+
+err_rm_devnode:
+ media_devnode_remove(vdev->intf_devnode);
+
+err_rm_links:
+ media_entity_remove_links(&vdev->entity);
+
+err_video_unregister_device:
+ video_unregister_device(vdev);
+
+err_vb2_queue_release:
+ vb2_queue_release(&node->dev_q.vbq);
+
+err_media_entity_cleanup:
+ media_entity_cleanup(&node->vdev.entity);
+
+err_mutex_destroy:
+ mutex_destroy(&node->dev_q.lock);
+
+ return ret;
+}
+
+/******************************************************************************
+ * @array img_resize_ratio_menu
+ * @brief IMGSYS hardware supports multiple resizers.
+ * - Any Ratio: Allows for any resize ratio, including
+ * down 2, down 4, and down 42.
+ * - Down 4: Specifically for downscaling by a factor of 4.
+ * - Down 2: Specifically for downscaling by a factor of 2.
+ * - Down42: Used for downscaling by a factor of 4 on the
+ * first use, and by a factor of 2 on subsequent uses.
+ * - NULL: Indicates the end of the menu options
+ ******************************************************************************/
+static const char * const img_resize_ratio_menu[] = {
+ "Any Ratio",
+ "Down 4",
+ "Down 2",
+ "Down 42",
+ NULL,
+};
+
+static const struct v4l2_ctrl_config cfg_mtk_resize_ratio = {
+ .ops = &mtk_imgsys_video_device_ctrl_ops,
+ .id = V4L2_CID_MTK_IMG_RESIZE_RATIO,
+ .name = "Image resize ratio",
+ .type = V4L2_CTRL_TYPE_MENU,
+ .max = 3,
+ .def = 0,
+ .qmenu = img_resize_ratio_menu,
+};
+
+static int mtk_imgsys_pipe_v4l2_ctrl_init(struct mtk_imgsys_pipe *imgsys_pipe)
+{
+ int i, ret;
+ struct mtk_imgsys_video_device *ctrl_node;
+
+ for (i = 0; i < MTK_IMGSYS_VIDEO_NODE_ID_TOTAL_NUM; i++) {
+ ctrl_node = &imgsys_pipe->nodes[i];
+ if (!ctrl_node->desc->supports_ctrls)
+ continue;
+
+ v4l2_ctrl_handler_init(&ctrl_node->ctrl_handler, 4);
+ v4l2_ctrl_new_std(&ctrl_node->ctrl_handler,
+ &mtk_imgsys_video_device_ctrl_ops, V4L2_CID_ROTATE,
+ 0, 270, 90, 0);
+ v4l2_ctrl_new_std(&ctrl_node->ctrl_handler,
+ &mtk_imgsys_video_device_ctrl_ops, V4L2_CID_VFLIP,
+ 0, 1, 1, 0);
+ v4l2_ctrl_new_std(&ctrl_node->ctrl_handler,
+ &mtk_imgsys_video_device_ctrl_ops, V4L2_CID_HFLIP,
+ 0, 1, 1, 0);
+ ctrl_node->resize_ratio_ctrl =
+ v4l2_ctrl_new_custom(&ctrl_node->ctrl_handler,
+ &cfg_mtk_resize_ratio, NULL);
+ ret = ctrl_node->ctrl_handler.error;
+ if (ret) {
+ dev_info(imgsys_pipe->imgsys_dev->dev,
+ "%s create rotate ctrl failed:(%d)",
+ ctrl_node->desc->name, ret);
+ goto err_free_ctrl_handlers;
+ }
+ }
+
+ return 0;
+
+err_free_ctrl_handlers:
+ for (; i >= 0; i--) {
+ ctrl_node = &imgsys_pipe->nodes[i];
+ if (!ctrl_node->desc->supports_ctrls)
+ continue;
+ v4l2_ctrl_handler_free(&ctrl_node->ctrl_handler);
+ }
+
+ return ret;
+}
+
+static void mtk_imgsys_pipe_v4l2_ctrl_release(struct mtk_imgsys_pipe *imgsys_pipe)
+{
+ struct mtk_imgsys_video_device *vdev;
+ int i;
+
+ for (i = 0; i < imgsys_pipe->desc->total_queues; ++i) {
+ vdev = &imgsys_pipe->nodes[i];
+ if (vdev->desc->supports_ctrls)
+ v4l2_ctrl_handler_free(&vdev->ctrl_handler);
+ }
+}
+
+static int mtk_imgsys_pipe_v4l2_register(struct mtk_imgsys_pipe *pipe,
+ struct media_device *media_dev,
+ struct v4l2_device *v4l2_dev)
+{
+ int i, ret;
+
+ ret = mtk_imgsys_pipe_v4l2_ctrl_init(pipe);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s: failed(%d) to initialize ctrls\n",
+ pipe->desc->name, ret);
+
+ return ret;
+ }
+
+ pipe->streaming = 0;
+
+ /* Initialize subdev media entity */
+ pipe->subdev_pads = devm_kcalloc(pipe->imgsys_dev->dev,
+ pipe->desc->total_queues,
+ sizeof(*pipe->subdev_pads),
+ GFP_KERNEL);
+ if (!pipe->subdev_pads) {
+ dev_info(pipe->imgsys_dev->dev,
+ "failed to alloc pipe->subdev_pads (%d)\n", ret);
+ ret = -ENOMEM;
+ goto err_release_ctrl;
+ }
+ ret = media_entity_pads_init(&pipe->subdev.entity,
+ pipe->desc->total_queues,
+ pipe->subdev_pads);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "failed initialize subdev media entity (%d)\n", ret);
+ goto err_free_subdev_pads;
+ }
+
+ /* Initialize subdev */
+ v4l2_subdev_init(&pipe->subdev, &mtk_imgsys_subdev_ops);
+
+ pipe->subdev.entity.function =
+ MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ pipe->subdev.entity.ops = &mtk_imgsys_media_ops;
+ pipe->subdev.flags =
+ V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+ pipe->subdev.ctrl_handler = NULL;
+
+ for (i = 0; i < pipe->desc->total_queues; i++)
+ pipe->subdev_pads[pipe->nodes[i].desc->id].flags =
+ V4L2_TYPE_IS_OUTPUT(pipe->nodes[i].desc->buf_type) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+ snprintf(pipe->subdev.name, sizeof(pipe->subdev.name),
+ "%s", pipe->desc->name);
+ v4l2_set_subdevdata(&pipe->subdev, pipe);
+
+ ret = v4l2_device_register_subdev(&pipe->imgsys_dev->v4l2_dev,
+ &pipe->subdev);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "failed initialize subdev (%d)\n", ret);
+ goto err_media_entity_cleanup;
+ }
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "register subdev: %s, ctrl_handler %p\n",
+ pipe->subdev.name, pipe->subdev.ctrl_handler);
+
+ /* Create video nodes and links */
+ for (i = 0; i < pipe->desc->total_queues; i++) {
+ ret =
+ mtk_imgsys_video_device_v4l2_register(pipe,
+ &pipe->nodes[i]);
+ if (ret)
+ goto err_unregister_subdev;
+ }
+
+ return 0;
+
+err_unregister_subdev:
+ v4l2_device_unregister_subdev(&pipe->subdev);
+
+err_media_entity_cleanup:
+ media_entity_cleanup(&pipe->subdev.entity);
+
+err_free_subdev_pads:
+ devm_kfree(pipe->imgsys_dev->dev, pipe->subdev_pads);
+
+err_release_ctrl:
+ mtk_imgsys_pipe_v4l2_ctrl_release(pipe);
+
+ return ret;
+}
+
+static void mtk_imgsys_pipe_v4l2_unregister(struct mtk_imgsys_pipe *pipe)
+{
+ unsigned int i;
+
+ for (i = 0; i < pipe->desc->total_queues; i++) {
+ video_unregister_device(&pipe->nodes[i].vdev);
+ vb2_queue_release(&pipe->nodes[i].dev_q.vbq);
+ media_entity_cleanup(&pipe->nodes[i].vdev.entity);
+ mutex_destroy(&pipe->nodes[i].dev_q.lock);
+ }
+
+ v4l2_device_unregister_subdev(&pipe->subdev);
+ media_entity_cleanup(&pipe->subdev.entity);
+ mtk_imgsys_pipe_v4l2_ctrl_release(pipe);
+}
+
+static void mtk_imgsys_dev_media_unregister(struct mtk_imgsys_dev *imgsys_dev)
+{
+ media_device_unregister(&imgsys_dev->mdev);
+ media_device_cleanup(&imgsys_dev->mdev);
+}
+
+static int mtk_imgsys_dev_v4l2_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct media_device *media_dev = &imgsys_dev->mdev;
+ struct v4l2_device *v4l2_dev = &imgsys_dev->v4l2_dev;
+ int ret;
+
+ ret = mtk_imgsys_dev_media_register(imgsys_dev->dev, media_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: media device register failed(%d)\n",
+ __func__, ret);
+ return ret;
+ }
+
+ v4l2_dev->mdev = media_dev;
+ v4l2_dev->ctrl_handler = NULL;
+
+ ret = v4l2_device_register(imgsys_dev->dev, v4l2_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: v4l2 device register failed(%d)\n",
+ __func__, ret);
+ goto err_release_media_device;
+ }
+
+ ret = mtk_imgsys_pipe_init(imgsys_dev,
+ &imgsys_dev->imgsys_pipe,
+ imgsys_dev->cust_pipes);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: init failed(%d)\n",
+ imgsys_dev->imgsys_pipe.desc->name, ret);
+ goto err_release_pipe;
+ }
+
+ ret = mtk_imgsys_pipe_v4l2_register(&imgsys_dev->imgsys_pipe, &imgsys_dev->mdev,
+ &imgsys_dev->v4l2_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: failed(%d) to create V4L2 devices\n",
+ imgsys_dev->imgsys_pipe.desc->name, ret);
+ goto err_register_pipe;
+ }
+
+ ret = v4l2_device_register_subdev_nodes(&imgsys_dev->v4l2_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "failed to register subdevs (%d)\n", ret);
+ goto err_register_pipe;
+ }
+
+ return 0;
+
+err_register_pipe:
+ mtk_imgsys_pipe_v4l2_unregister(&imgsys_dev->imgsys_pipe);
+err_release_pipe:
+ mtk_imgsys_pipe_release(&imgsys_dev->imgsys_pipe);
+
+ v4l2_device_unregister(v4l2_dev);
+
+err_release_media_device:
+ mtk_imgsys_dev_media_unregister(imgsys_dev);
+
+ return ret;
+}
+
+static void mtk_imgsys_dev_v4l2_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ mtk_imgsys_pipe_v4l2_unregister(&imgsys_dev->imgsys_pipe);
+ mtk_imgsys_pipe_release(&imgsys_dev->imgsys_pipe);
+ v4l2_device_unregister(&imgsys_dev->v4l2_dev);
+ mtk_imgsys_dev_media_unregister(imgsys_dev);
+}
+
+static int mtk_imgsys_of_rproc(struct mtk_imgsys_dev *imgsys,
+ struct platform_device *pdev)
+{
+ struct device *dev = imgsys->dev;
+
+ imgsys->scp = scp_get(pdev);
+ if (!imgsys->scp) {
+ dev_err(dev, "failed to get scp device\n");
+ return -ENODEV;
+ }
+
+ imgsys->rproc_handle = scp_get_rproc(imgsys->scp);
+ dev_dbg(dev, "imgsys rproc_phandle: 0x%pK\n", imgsys->rproc_handle);
+ imgsys->smem_dev = scp_get_device(imgsys->scp);
+
+ return 0;
+}
+
+static int mtk_imgsys_probe(struct platform_device *pdev)
+{
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct device **larb_devs;
+ struct device_link *link;
+ const struct cust_data *data;
+ struct resource *res;
+ int larbs_num, i;
+ int ret;
+
+ imgsys_dev = devm_kzalloc(&pdev->dev, sizeof(*imgsys_dev), GFP_KERNEL);
+ if (!imgsys_dev)
+ return -ENOMEM;
+
+ data = of_device_get_match_data(&pdev->dev);
+
+ init_imgsys_pipeline(data);
+
+ imgsys_dev->cust_pipes = data->pipe_settings;
+ imgsys_dev->dev = &pdev->dev;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res) {
+ dev_err(&pdev->dev, "Failed to get memory resource\n");
+ return -ENODEV;
+ }
+ imgsys_dev->imgsys_resource = res;
+
+ dev_set_drvdata(&pdev->dev, imgsys_dev);
+
+ imgsys_dev->num_mods = data->mod_num;
+
+ /* Get Clocks */
+ imgsys_dev->clks = data->clks;
+ imgsys_dev->num_clks = data->clk_num;
+ ret = devm_clk_bulk_get(&pdev->dev, imgsys_dev->num_clks,
+ imgsys_dev->clks);
+ if (ret)
+ dev_info(&pdev->dev, "Failed to get clks:%d\n", ret);
+
+ /* DMA configuration */
+ if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(34)))
+ dev_info(&pdev->dev, "%s:No DMA available\n", __func__);
+
+ if (!pdev->dev.dma_parms) {
+ pdev->dev.dma_parms =
+ devm_kzalloc(imgsys_dev->dev, sizeof(*pdev->dev.dma_parms), GFP_KERNEL);
+ }
+ if (pdev->dev.dma_parms)
+ dma_set_max_seg_size(imgsys_dev->dev, UINT_MAX);
+
+ /* Get LARB Information */
+ larbs_num = of_count_phandle_with_args(pdev->dev.of_node,
+ "mediatek,larbs", NULL);
+ dev_dbg(imgsys_dev->dev, "%d larbs to be added", larbs_num);
+ larb_devs = devm_kzalloc(&pdev->dev, sizeof(larb_devs) * larbs_num, GFP_KERNEL);
+ if (!larb_devs) {
+ ret = -ENOMEM;
+ goto err_free_dev_alloc;
+ }
+ for (i = 0; i < larbs_num; i++) {
+ struct device_node *larb_node;
+ struct platform_device *larb_pdev;
+
+ larb_node = of_parse_phandle(pdev->dev.of_node, "mediatek,larbs", i);
+ if (!larb_node) {
+ dev_info(imgsys_dev->dev,
+ "%d larb node not found\n", i);
+ continue;
+ }
+
+ larb_pdev = of_find_device_by_node(larb_node);
+ if (!larb_pdev) {
+ of_node_put(larb_node);
+ dev_info(imgsys_dev->dev,
+ "%d larb device not found\n", i);
+ continue;
+ }
+ of_node_put(larb_node);
+
+ link = device_link_add(&pdev->dev, &larb_pdev->dev,
+ DL_FLAG_PM_RUNTIME | DL_FLAG_STATELESS);
+ if (!link)
+ dev_info(imgsys_dev->dev, "unable to link SMI LARB idx %d\n", i);
+
+ larb_devs[i] = &larb_pdev->dev;
+ }
+ imgsys_dev->larbs = larb_devs;
+ imgsys_dev->num_larbs = larbs_num;
+
+ /* Register rproc device */
+ if (mtk_imgsys_of_rproc(imgsys_dev, pdev)) {
+ ret = -EFAULT;
+ goto err_free_larb_alloc;
+ }
+
+ /* Imgsys device initialization */
+ ret = mtk_imgsys_dev_v4l2_init(imgsys_dev);
+ if (ret) {
+ dev_info(&pdev->dev, "v4l2 init failed(%d)\n", ret);
+
+ goto err_free_larb_alloc;
+ }
+
+ /* Enable power*/
+ pm_runtime_set_autosuspend_delay(&pdev->dev, IMGSYS_SUSPEND_TIME);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
+ return 0;
+
+err_free_larb_alloc:
+ devm_kfree(&pdev->dev, imgsys_dev->larbs);
+err_free_dev_alloc:
+ devm_kfree(&pdev->dev, imgsys_dev);
+ return ret;
+}
+
+static void mtk_imgsys_remove(struct platform_device *pdev)
+{
+ struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(&pdev->dev);
+
+ pm_runtime_disable(&pdev->dev);
+ mtk_imgsys_dev_v4l2_release(imgsys_dev);
+ devm_kfree(&pdev->dev, imgsys_dev->larbs);
+ devm_kfree(&pdev->dev, imgsys_dev);
+}
+
+static int __maybe_unused mtk_imgsys_pm_suspend(struct device *dev)
+{
+ int ret;
+
+ if (pm_runtime_suspended(dev)) {
+ dev_info(dev, "%s: pm_runtime_suspended is true, no action\n",
+ __func__);
+ return 0;
+ }
+
+ ret = pm_runtime_force_suspend(dev);
+ if (ret) {
+ dev_info(dev, "%s: pm_runtime_force_suspend failed:(%d)\n",
+ __func__, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused mtk_imgsys_pm_resume(struct device *dev)
+{
+ int ret;
+
+ if (pm_runtime_suspended(dev)) {
+ dev_info(dev, "%s: pm_runtime_suspended is true, no action\n",
+ __func__);
+ return 0;
+ }
+
+ ret = pm_runtime_force_resume(dev);
+ if (ret) {
+ dev_info(dev, "%s: pm_runtime_force_resume failed:(%d)\n",
+ __func__, ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int __maybe_unused mtk_imgsys_runtime_suspend(struct device *dev)
+{
+ struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(dev);
+
+ clk_bulk_disable_unprepare(imgsys_dev->num_clks,
+ imgsys_dev->clks);
+
+ return 0;
+}
+
+static int __maybe_unused mtk_imgsys_runtime_resume(struct device *dev)
+{
+ struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(dev);
+ int ret;
+
+ ret = clk_bulk_prepare_enable(imgsys_dev->num_clks,
+ imgsys_dev->clks);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "failed to enable dip clks(%d)\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static const struct dev_pm_ops mtk_imgsys_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(mtk_imgsys_pm_suspend, mtk_imgsys_pm_resume)
+ SET_RUNTIME_PM_OPS(mtk_imgsys_runtime_suspend,
+ mtk_imgsys_runtime_resume, NULL)
+};
+
+static struct clk_bulk_data imgsys_isp7_clks[] = {
+ { .id = "IMGSYS_CG_IMG_TRAW0" },
+ { .id = "IMGSYS_CG_IMG_TRAW1" },
+ { .id = "IMGSYS_CG_IMG_VCORE_GALS" },
+ { .id = "IMGSYS_CG_IMG_DIP0" },
+ { .id = "IMGSYS_CG_IMG_WPE0" },
+ { .id = "IMGSYS_CG_IMG_WPE1" },
+ { .id = "IMGSYS_CG_IMG_WPE2" },
+ { .id = "IMGSYS_CG_IMG_GALS" },
+ { .id = "DIP_TOP_DIP_TOP" },
+ { .id = "DIP_NR_DIP_NR" },
+ { .id = "WPE1_CG_DIP1_WPE" },
+ { .id = "WPE2_CG_DIP1_WPE" },
+ { .id = "WPE3_CG_DIP1_WPE" },
+ { .id = "ME_CG_IPE" },
+ { .id = "ME_CG_IPE_TOP" },
+ { .id = "ME_CG" },
+ { .id = "ME_CG_LARB12" },
+};
+
+static const struct cust_data imgsys_data[] = {
+ [0] = {
+ .clks = imgsys_isp7_clks,
+ .clk_num = ARRAY_SIZE(imgsys_isp7_clks),
+ .module_pipes = module_pipe_isp7,
+ .mod_num = ARRAY_SIZE(module_pipe_isp7),
+ .pipe_settings = pipe_settings_isp7,
+ .pipe_num = ARRAY_SIZE(pipe_settings_isp7),
+ },
+};
+
+static const struct of_device_id mtk_imgsys_of_match[] = {
+ { .compatible = "mediatek,mt8188-isp-imgsys", .data = (void *)&imgsys_data},
+ {}
+};
+MODULE_DEVICE_TABLE(of, mtk_imgsys_of_match);
+
+static struct platform_driver mtk_imgsys_driver = {
+ .probe = mtk_imgsys_probe,
+ .remove = mtk_imgsys_remove,
+ .driver = {
+ .name = "mtk-imgsys",
+ .owner = THIS_MODULE,
+ .pm = &mtk_imgsys_pm_ops,
+ .of_match_table = of_match_ptr(mtk_imgsys_of_match),
+ }
+};
+
+module_platform_driver(mtk_imgsys_driver);
+
+MODULE_AUTHOR("Frederic Chen <frederic.chen@mediatek.com>");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("MediaTek IMGSYS driver");
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
new file mode 100644
index 000000000000..4a418099d7c0
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
@@ -0,0 +1,100 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+#ifndef _MTK_IMGSYS_VNODE_ID_H_
+#define _MTK_IMGSYS_VNODE_ID_H_
+
+/**
+ * enum imgsys_video_nodes_id - Enumeration of video node IDs for image system
+ *
+ * This enumeration defines the unique identifiers for various video nodes
+ * used in the image processing system. Each ID corresponds to a specific
+ * video node that can be utilized for different functionalities.
+ */
+enum imgsys_video_nodes_id {
+ /* IMGSYS_MODULE_TRAW */
+ MTK_IMGSYS_VIDEO_NODE_ID_TIMGI_OUT = 0,
+ MTK_IMGSYS_VIDEO_NODE_ID_METAI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_PDC_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TYUVO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TYUV2O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TYUV3O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TYUV4O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TYUV5O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_FEO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TIMGO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_IMGSTATO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_XTMEO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_XTFDO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_XTADLDBGO_CAPTURE,
+ /* IMGSYS_MODULE_DIP */
+ MTK_IMGSYS_VIDEO_NODE_ID_IMGI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_VIPI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_REC_DSI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_REC_DPI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_CNR_BLURMAPI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_LFEI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_RFEI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRSI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRWI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRMI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRCI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRLI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRVBI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_IMG2O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_IMG3O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_IMG4O_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_FMO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRSO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRWO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TNRMO_CAPTURE,
+ /* IMGSYS_MODULE_PQDIP */
+ MTK_IMGSYS_VIDEO_NODE_ID_PIMGI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WROT_A_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_WROT_B_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TCCSO_A_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_TCCSO_B_CAPTURE,
+ /* IMGSYS_MODULE_ME */
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL0IMG0_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL0IMG1_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL1IMG0_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL1IMG1_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEIMGSTATS_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL0RMV_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL1RMV_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL0FMB_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL1FMB_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEMIL_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEMMGMIL_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL0WMV_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL1WMV_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MECONF_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEWMAP_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEFMV_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEFST_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MELMI_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL0WFMB_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_MEL1WFMB_CAPTURE,
+ /* IMGSYS_MODULE_WPE */
+ MTK_IMGSYS_VIDEO_NODE_ID_WWPEI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WVECI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WPSP_COEFI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WWPEO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_WMSKO_CAPTURE,
+ /* IMGSYS_MODULE_WPE_TNR */
+ MTK_IMGSYS_VIDEO_NODE_ID_WTWPEI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WTVECI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WTPSP_COEFI_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_WTWPEO_CAPTURE,
+ MTK_IMGSYS_VIDEO_NODE_ID_WTMSKO_CAPTURE,
+ /* other module video nodes to be added */
+ MTK_IMGSYS_VIDEO_NODE_ID_TUNING_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_CTRLMETA_OUT,
+ MTK_IMGSYS_VIDEO_NODE_ID_TOTAL_NUM,
+};
+
+#endif
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h
new file mode 100644
index 000000000000..5a6f3220ab6b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+#ifndef _MTK_IMGSYS_V4L2_VNODE_H_
+#define _MTK_IMGSYS_V4L2_VNODE_H_
+
+#include "mtk_imgsys-of.h"
+#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-formats.h"
+#include "modules/mtk_dip_v4l2_vnode.h"
+#include "modules/mtk_traw_v4l2_vnode.h"
+#include "modules/mtk_pqdip_v4l2_vnode.h"
+#include "modules/mtk_wpe_v4l2_vnode.h"
+#include "modules/mtk_me_v4l2_vnode.h"
+
+/**
+ * enum mtk_imgsys_module_id - Enumeration of image system module IDs
+ * @IMGSYS_MODULE_TRAW: Module for TRAW processing.
+ * @IMGSYS_MODULE_DIP: Module for DIP processing.
+ * @IMGSYS_MODULE_PQDIP: Module for PQDIP processing.
+ * @IMGSYS_MODULE_ME: Module for Motion Estimation (ME).
+ * @IMGSYS_MODULE_WPE: Module for WPE processing.
+ * @IMGSYS_MODULE_MAIN: Main module for image system.
+ * @IMGSYS_MODULE_NUM: Total number of image system modules.
+ */
+enum mtk_imgsys_module_id {
+ IMGSYS_MODULE_TRAW = 0,
+ IMGSYS_MODULE_DIP,
+ IMGSYS_MODULE_PQDIP,
+ IMGSYS_MODULE_ME,
+ IMGSYS_MODULE_WPE,
+ IMGSYS_MODULE_MAIN,
+ IMGSYS_MODULE_NUM,
+};
+
+/**
+ * struct mtk_imgsys_mod_pipe_desc - Pipe description for image system modules
+ * @vnode_desc: Pointer to the node descriptor array
+ * @node_num: Number of nodes in the descriptor array
+ *
+ * This structure defines the pipe description for each module in the MediaTek
+ * image system. It includes a pointer to the node descriptor array and the
+ * number of nodes in that array.
+ */
+static const struct mtk_imgsys_mod_pipe_desc module_pipe_isp7[] = {
+ [IMGSYS_MODULE_TRAW] = {
+ .vnode_desc = traw_setting,
+ .node_num = ARRAY_SIZE(traw_setting),
+ },
+ [IMGSYS_MODULE_DIP] = {
+ .vnode_desc = dip_setting,
+ .node_num = ARRAY_SIZE(dip_setting),
+ },
+ [IMGSYS_MODULE_PQDIP] = {
+ .vnode_desc = pqdip_setting,
+ .node_num = ARRAY_SIZE(pqdip_setting),
+ },
+ [IMGSYS_MODULE_ME] = {
+ .vnode_desc = me_setting,
+ .node_num = ARRAY_SIZE(me_setting),
+ },
+ [IMGSYS_MODULE_WPE] = {
+ .vnode_desc = wpe_setting,
+ .node_num = ARRAY_SIZE(wpe_setting),
+ },
+ [IMGSYS_MODULE_MAIN] = {
+ .vnode_desc = NULL,
+ .node_num = 0,
+ }
+};
+
+/* Tuning formats for MediaTek Image System */
+static const struct mtk_imgsys_dev_format tuning_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_metai,
+ },
+};
+
+/* Control metadata formats for MediaTek Image System */
+static const struct mtk_imgsys_dev_format ctrlmeta_fmts[] = {
+ {
+ .fmt = &mtk_imgsys_format_ctrlmeta,
+ },
+};
+
+/**
+ * queues_setting - Array to store video device descriptions for each module
+ * and meta in ImgSys
+ */
+static struct mtk_imgsys_video_device_desc
+queues_setting[MTK_IMGSYS_VIDEO_NODE_ID_TOTAL_NUM] = {
+ [MTK_IMGSYS_VIDEO_NODE_ID_TUNING_OUT] = {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_TUNING_OUT,
+ .name = "Tuning",
+ .cap = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+ .smem_alloc = 1,
+ .flags = 0,
+ .fmts = tuning_fmts,
+ .num_fmts = ARRAY_SIZE(tuning_fmts),
+ .dma_port = 0,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Tuning data",
+ },
+ [MTK_IMGSYS_VIDEO_NODE_ID_CTRLMETA_OUT] = {
+ .id = MTK_IMGSYS_VIDEO_NODE_ID_CTRLMETA_OUT,
+ .name = "CtrlMeta",
+ .cap = V4L2_CAP_META_OUTPUT | V4L2_CAP_STREAMING,
+ .buf_type = V4L2_BUF_TYPE_META_OUTPUT,
+ .smem_alloc = 1,
+ .flags = 0,
+ .fmts = ctrlmeta_fmts,
+ .num_fmts = ARRAY_SIZE(ctrlmeta_fmts),
+ .dma_port = 0,
+ .frmsizeenum = &dip_in_frmsizeenum,
+ .description = "Control meta data for flow control",
+ },
+};
+
+/**
+ * pipe_settings_isp7 - Array to store pipeline configurations for ISP7 in ImgSys
+ */
+static const struct mtk_imgsys_pipe_desc pipe_settings_isp7[] = {
+ {
+ .name = MTK_IMGSYS_MEDIA_MODEL_NAME,
+ .id = 0,
+ .queue_descs = queues_setting,
+ .total_queues = ARRAY_SIZE(queues_setting),
+ },
+};
+#endif
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (3 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 11:49 ` [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys Olivia Wen
` (4 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the Imgsys LARBs device. This patch adds an independent
device driver for the ImgSys Local Arbiter (LARB) devices. The addition
is necessary because the IOMMU (Input/Output Memory Management Unit)
cannot support multiple LARBs within a single device node.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 61 +++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index cfd65858c751..919c69493bbc 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -28,6 +28,58 @@
#define IMGSYS_MAX_BUFFERS 256
#define IMGSYS_SUSPEND_TIME 3000 /* ms */
+struct mtk_imgsys_larb_device {
+ struct device *dev;
+ struct mtk_imgsys_dev *imgsys_dev;
+};
+
+static int mtk_imgsys_larb_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_imgsys_larb_device *larb_dev;
+
+ larb_dev = devm_kzalloc(dev, sizeof(*dev), GFP_KERNEL);
+ if (!larb_dev)
+ return -ENOMEM;
+
+ if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(34)))
+ dev_err(dev, "No suitable DMA available\n");
+
+ if (!dev->dma_parms) {
+ dev->dma_parms =
+ devm_kzalloc(dev, sizeof(*dev->dma_parms), GFP_KERNEL);
+ if (!dev->dma_parms)
+ return -ENOMEM;
+ }
+ if (pdev->dev.dma_parms)
+ dma_set_max_seg_size(dev, UINT_MAX);
+
+ pm_runtime_enable(dev);
+
+ return 0;
+}
+
+static void mtk_imgsys_larb_remove(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+
+ pm_runtime_disable(dev);
+}
+
+static const struct of_device_id mtk_imgsys_larb_match[] = {
+ {.compatible = "mediatek,mt8188-imgsys-larb",},
+ {},
+};
+
+static struct platform_driver mtk_imgsys_larb_driver = {
+ .probe = mtk_imgsys_larb_probe,
+ .remove = mtk_imgsys_larb_remove,
+ .driver = {
+ .name = "mtk-imgsys-larb",
+ .of_match_table = of_match_ptr(mtk_imgsys_larb_match),
+ },
+};
+
static int mtk_imgsys_sd_subscribe_event(struct v4l2_subdev *subdev,
struct v4l2_fh *fh,
struct v4l2_event_subscription *sub)
@@ -1537,8 +1589,16 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
pm_runtime_use_autosuspend(&pdev->dev);
pm_runtime_enable(&pdev->dev);
+ ret = platform_driver_register(&mtk_imgsys_larb_driver);
+ if (ret) {
+ dev_info(imgsys_dev->dev, "register mtk_imgsys_larb_driver fail\n");
+ goto err_release_deinit_v4l2;
+ }
+
return 0;
+err_release_deinit_v4l2:
+ mtk_imgsys_dev_v4l2_release(imgsys_dev);
err_free_larb_alloc:
devm_kfree(&pdev->dev, imgsys_dev->larbs);
err_free_dev_alloc:
@@ -1551,6 +1611,7 @@ static void mtk_imgsys_remove(struct platform_device *pdev)
struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ platform_driver_unregister(&mtk_imgsys_larb_driver);
mtk_imgsys_dev_v4l2_release(imgsys_dev);
devm_kfree(&pdev->dev, imgsys_dev->larbs);
devm_kfree(&pdev->dev, imgsys_dev);
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (4 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 11:49 ` [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver Olivia Wen
` (3 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on a set of operations for specific modules.
The ImgSys architecture includes various modules such as WPE (Warping
Engine), TRAW (Tile-Raw), DIP (Digital Image Processing), PQDIP (Picture
Quality Digital Image Processing), and ME (Motion Estimation).
Each module requires specific operations for initialization and
deinitialization to ensure proper setup and cleanup.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 8 +
.../isp_7x/imgsys/modules/mtk_imgsys-dip.c | 115 +++++++++++
.../isp_7x/imgsys/modules/mtk_imgsys-dip.h | 18 ++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.c | 113 +++++++++++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.h | 17 ++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.c | 65 +++++++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.h | 17 ++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.c | 116 +++++++++++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.h | 17 ++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 12 +-
.../isp_7x/imgsys/mtk_imgsys-module_common.h | 82 ++++++++
.../isp_7x/imgsys/mtk_imgsys-module_main.c | 183 ++++++++++++++++++
.../isp_7x/imgsys/mtk_imgsys-module_main.h | 17 ++
.../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 26 +++
.../isp/isp_7x/imgsys/mtk_imgsys-of.h | 3 +
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 73 ++++++-
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 45 ++++-
17 files changed, 923 insertions(+), 4 deletions(-)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 571d0ae000eb..26c8c1d39750 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,10 +2,18 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-module_main.o \
mtk_imgsys-of.o \
mtk_imgsys-formats.o \
mtk_imgsys-dev.o \
mtk_imgsys-sys.o \
mtk_imgsys-v4l2.o
+mtk_imgsys_hw_isp-objs := \
+modules/mtk_imgsys-dip.o \
+modules/mtk_imgsys-traw.o \
+modules/mtk_imgsys-pqdip.o \
+modules/mtk_imgsys-wpe.o
+
obj-$(CONFIG_VIDEO_MTK_ISP_71_IMGSYS) += mtk_imgsys_util.o
+obj-$(CONFIG_VIDEO_MTK_ISP_71_IMGSYS) += mtk_imgsys_hw_isp.o
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
new file mode 100644
index 000000000000..1cc1f3e87e8b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-dip.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+#define DIPCTL_D1A_DIPCTL_INT1_EN 0x094
+#define DIPCTL_D1A_DIPCTL_INT2_EN 0x0A0
+#define DIPCTL_D1A_DIPCTL_INT3_EN 0x0AC
+#define DIPCTL_D1A_DIPCTL_CQ_INT1_EN 0x0C4
+#define DIPCTL_D1A_DIPCTL_CQ_INT2_EN 0x0D0
+#define DIPCTL_D1A_DIPCTL_CQ_INT3_EN 0x0DC
+
+#define DIPCQ_D1A_DIPCQ_CQ_THR0_CTL 0x208
+#define DIPCQ_D1A_DIPCQ_CQ_THR1_CTL 0x218
+#define DIPCQ_D1A_DIPCQ_CQ_THR2_CTL 0x228
+#define DIPCQ_D1A_DIPCQ_CQ_THR3_CTL 0x238
+#define DIPCQ_D1A_DIPCQ_CQ_THR4_CTL 0x248
+#define DIPCQ_D1A_DIPCQ_CQ_THR5_CTL 0x258
+#define DIPCQ_D1A_DIPCQ_CQ_THR6_CTL 0x268
+#define DIPCQ_D1A_DIPCQ_CQ_THR7_CTL 0x278
+#define DIPCQ_D1A_DIPCQ_CQ_THR8_CTL 0x288
+#define DIPCQ_D1A_DIPCQ_CQ_THR9_CTL 0x298
+#define DIPCQ_D1A_DIPCQ_CQ_THR10_CTL 0x2A8
+#define DIPCQ_D1A_DIPCQ_CQ_THR11_CTL 0x2B8
+#define DIPCQ_D1A_DIPCQ_CQ_THR12_CTL 0x2C8
+#define DIPCQ_D1A_DIPCQ_CQ_THR13_CTL 0x2D8
+#define DIPCQ_D1A_DIPCQ_CQ_THR14_CTL 0x2E8
+#define DIPCQ_D1A_DIPCQ_CQ_THR15_CTL 0x2F8
+#define DIPCQ_D1A_DIPCQ_CQ_THR16_CTL 0x308
+#define DIPCQ_D1A_DIPCQ_CQ_THR17_CTL 0x318
+#define DIPCQ_D1A_DIPCQ_CQ_THR18_CTL 0x328
+
+#define DIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_dip_init_ary)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_dip_init_ary[] = {
+ { DIPCTL_D1A_DIPCTL_INT1_EN, 0x80000000 },
+ { DIPCTL_D1A_DIPCTL_INT2_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_INT3_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_CQ_INT1_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_CQ_INT2_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_CQ_INT3_EN, 0x0 },
+ { DIPCQ_D1A_DIPCQ_CQ_THR0_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR1_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR2_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR3_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR4_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR5_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR6_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR7_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR8_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR9_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR10_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR11_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR12_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR13_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR14_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR15_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR16_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR17_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR18_CTL, CQ_THRX_CTL },
+};
+
+static void __iomem *dip_reg_base[MTK_IMGSYS_DIP_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_DIP_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_DIP1 + hw_idx;
+ dip_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!dip_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap dip_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < DIP_INIT_ARRAY_COUNT; i++) {
+ offset = dip_reg_base[hw_idx] + mtk_imgsys_dip_init_ary[i].offset;
+ writel(mtk_imgsys_dip_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_dip_init);
+
+void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_DIP_HW_SET; hw_idx++) {
+ iounmap(dip_reg_base[hw_idx]);
+ dip_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_dip_uninit);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
new file mode 100644
index 000000000000..22b4a7c3616d
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_DIP_DIP_H_
+#define _MTK_DIP_DIP_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_DIP_DIP_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
new file mode 100644
index 000000000000..d1d3507555d9
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-pqdip.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+#define PQDIPCTL_P1A_REG_PQDIPCTL_INT1_EN 0x0050
+#define PQDIPCTL_P1A_REG_PQDIPCTL_INT2_EN 0x0060
+#define PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT1_EN 0x0070
+#define PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT2_EN 0x0080
+#define PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT3_EN 0x0090
+
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR0_CTL 0x0208
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR1_CTL 0x0218
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR2_CTL 0x0228
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR3_CTL 0x0238
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR4_CTL 0x0248
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR5_CTL 0x0258
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR6_CTL 0x0268
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR7_CTL 0x0278
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR8_CTL 0x0288
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR9_CTL 0x0298
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR10_CTL 0x02A8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR11_CTL 0x02B8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR12_CTL 0x02C8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR13_CTL 0x02D8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR14_CTL 0x02E8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR15_CTL 0x02F8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR16_CTL 0x0308
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR17_CTL 0x0318
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR18_CTL 0x0328
+
+#define PQDIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_pqdip_init_ary)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_pqdip_init_ary[] = {
+ { PQDIPCTL_P1A_REG_PQDIPCTL_INT1_EN, 0x80000000 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_INT2_EN, 0x0 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT1_EN, 0x0 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT2_EN, 0x0 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT3_EN, 0x0 },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR0_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR1_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR2_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR3_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR4_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR5_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR6_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR7_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR8_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR9_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR10_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR11_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR12_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR13_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR14_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR15_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR16_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR17_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR18_CTL, CQ_THRX_CTL },
+};
+
+static void __iomem *pqdip_reg_base[MTK_IMGSYS_PQDIP_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_PQDIP_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_PQDIP1 + hw_idx;
+ pqdip_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!pqdip_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap pqdip_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < PQDIP_INIT_ARRAY_COUNT; i++) {
+ offset = pqdip_reg_base[hw_idx] + mtk_imgsys_pqdip_init_ary[i].offset;
+ writel(mtk_imgsys_pqdip_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_pqdip_init);
+
+void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_PQDIP_HW_SET; hw_idx++) {
+ iounmap(pqdip_reg_base[hw_idx]);
+ pqdip_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_pqdip_uninit);
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
new file mode 100644
index 000000000000..00f93d536181
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_PQDIP_H_
+#define _MTK_IMGSYS_PQDIP_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_PQDIP_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
new file mode 100644
index 000000000000..292f0ad09c46
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Shih-Fang Chuang <shih-fang.chuang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-traw.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+#define TRAWCTL_INT1_EN 0x00A0
+
+#define TRAW_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_traw_init_ary)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_traw_init_ary[] = {
+ { TRAWCTL_INT1_EN, 0x80000000 },
+};
+
+static void __iomem *traw_reg_base[MTK_IMGSYS_TRAW_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_TRAW_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_TRAW1 + hw_idx;
+ traw_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!traw_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap traw_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < TRAW_INIT_ARRAY_COUNT; i++) {
+ offset = traw_reg_base[hw_idx] + mtk_imgsys_traw_init_ary[i].offset;
+ writel(mtk_imgsys_traw_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_traw_init);
+
+void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_TRAW_HW_SET; hw_idx++) {
+ iounmap(traw_reg_base[hw_idx]);
+ traw_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_traw_uninit);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
new file mode 100644
index 000000000000..f3727216b58c
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Shih-fang Chuang <shih-fang.chuang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_TRAW_H_
+#define _MTK_IMGSYS_TRAW_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_TRAW_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
new file mode 100644
index 000000000000..b19d61529b80
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
@@ -0,0 +1,116 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Floria Huang <floria.huang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-wpe.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+/* Define register offsets */
+#define WPE_TOP_CTL_INT_EN_OFFSET 0x0018
+#define WPE_TOP_CTL_INT_STATUSX_OFFSET 0x0024
+#define WPE_TOP_CQ_IRQ_EN_OFFSET 0x00D4
+#define WPE_TOP_CQ_IRQ_STX_OFFSET 0x00DC
+#define WPE_TOP_CQ_IRQ_EN2_OFFSET 0x00E0
+#define WPE_TOP_CQ_IRQ_STX2_OFFSET 0x00E8
+#define WPE_TOP_CQ_IRQ_EN3_OFFSET 0x00EC
+#define WPE_TOP_CQ_IRQ_STX3_OFFSET 0x00F4
+#define WPE_CACHE_RWCTL_CTL_OFFSET 0x0204
+#define WPE_DMA_DMA_ERR_CTRL_OFFSET 0x03D4
+#define DIPCQ_W1A_DIPCQ_CQ_THR0_CTL_OFFSET 0x0B08
+#define DIPCQ_W1A_DIPCQ_CQ_THR1_CTL_OFFSET 0x0B18
+#define DIPCQ_W1A_DIPCQ_CQ_THR2_CTL_OFFSET 0x0B28
+#define DIPCQ_W1A_DIPCQ_CQ_THR3_CTL_OFFSET 0x0B38
+#define DIPCQ_W1A_DIPCQ_CQ_THR4_CTL_OFFSET 0x0B48
+#define DIPCQ_W1A_DIPCQ_CQ_THR5_CTL_OFFSET 0x0B58
+#define DIPCQ_W1A_DIPCQ_CQ_THR6_CTL_OFFSET 0x0B68
+#define DIPCQ_W1A_DIPCQ_CQ_THR7_CTL_OFFSET 0x0B78
+#define DIPCQ_W1A_DIPCQ_CQ_THR8_CTL_OFFSET 0x0B88
+#define DIPCQ_W1A_DIPCQ_CQ_THR9_CTL_OFFSET 0x0B98
+#define DIPCQ_W1A_DIPCQ_CQ_THR10_CTL_OFFSET 0x0BA8
+#define DIPCQ_W1A_DIPCQ_CQ_THR11_CTL_OFFSET 0x0BB8
+#define DIPCQ_W1A_DIPCQ_CQ_THR12_CTL_OFFSET 0x0BC8
+#define DIPCQ_W1A_DIPCQ_CQ_THR13_CTL_OFFSET 0x0BD8
+#define DIPCQ_W1A_DIPCQ_CQ_THR14_CTL_OFFSET 0x0BE8
+
+#define WPE_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_wpe_init_ary)
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_wpe_init_ary[] = {
+ { WPE_TOP_CTL_INT_EN_OFFSET, 0x80000000 },
+ { WPE_TOP_CTL_INT_STATUSX_OFFSET, 0xFFFFFFFF },
+ { WPE_TOP_CQ_IRQ_EN_OFFSET, 0x80000000 },
+ { WPE_TOP_CQ_IRQ_STX_OFFSET, 0xFFFFFFFF },
+ { WPE_TOP_CQ_IRQ_EN2_OFFSET, 0x80000000 },
+ { WPE_TOP_CQ_IRQ_STX2_OFFSET, 0xFFFFFFFF },
+ { WPE_TOP_CQ_IRQ_EN3_OFFSET, 0x80000000 },
+ { WPE_TOP_CQ_IRQ_STX3_OFFSET, 0xFFFFFFFF },
+ { WPE_CACHE_RWCTL_CTL_OFFSET, 0x00000002 },
+ { WPE_DMA_DMA_ERR_CTRL_OFFSET, 0x80000000 },
+ { DIPCQ_W1A_DIPCQ_CQ_THR0_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR1_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR2_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR3_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR4_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR5_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR6_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR7_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR8_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR9_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR10_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR11_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR12_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR13_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR14_CTL_OFFSET, CQ_THRX_CTL },
+};
+
+static void __iomem *wpe_reg_base[MTK_IMGSYS_WPE_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_WPE1 + hw_idx;
+ wpe_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!wpe_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap wpe_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < WPE_INIT_ARRAY_COUNT; i++) {
+ offset = wpe_reg_base[hw_idx] + mtk_imgsys_wpe_init_ary[i].offset;
+ writel(mtk_imgsys_wpe_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_wpe_init);
+
+void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ iounmap(wpe_reg_base[hw_idx]);
+ wpe_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_wpe_uninit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("MediaTek IMGSYS Modules");
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
new file mode 100644
index 000000000000..6001575265e6
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Floria Huang <floria.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_WPE_H_
+#define _MTK_IMGSYS_WPE_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_WPE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
index b69cfd0043d3..b8c30cb32b97 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
@@ -23,7 +23,7 @@
#include <linux/videodev2.h>
#include <media/videobuf2-core.h>
#include <media/videobuf2-v4l2.h>
-
+#include "mtk_imgsys-module_ops.h"
#include "mtk_header_desc.h"
#define MTK_IMGSYS_MEDIA_MODEL_NAME "MTK-ISP-DIP-V4L2"
@@ -273,6 +273,7 @@ struct mtk_imgsys_pipe {
* @v4l2_dev: V4L2 device structure for video device operations.
* @imgsys_pipe: Image system pipeline configuration structure.
* @cust_pipes: Pointer to custom pipeline descriptors.
+ * @modules: Pointer to image system module operations.
* @clks: Array of clock bulk data for managing multiple clocks.
* @num_clks: Number of clocks in the clock array.
* @larbs: Array of device pointers for managing LARB (local arbiter).
@@ -281,6 +282,9 @@ struct mtk_imgsys_pipe {
* @rproc_handle: Handle for remote processor interface.
* @smem_dev: Device structure for shared memory operations.
* @num_mods: Number of modules in the image system.
+ * @init_kref: Reference counter for initialization tracking.
+ * @hw_op_lock: Mutex for serializing hardware operations.
+ * @stream_cnt: Counter for active streams, increased on stream on.
*/
struct mtk_imgsys_dev {
/* Device and Resource Information */
@@ -292,6 +296,8 @@ struct mtk_imgsys_dev {
struct mtk_imgsys_pipe imgsys_pipe;
/* Imgsys Pipeline Information */
const struct mtk_imgsys_pipe_desc *cust_pipes;
+ /* Imgsys Moudles Information */
+ const struct module_ops *modules;
/* Clock Information */
struct clk_bulk_data *clks;
int num_clks;
@@ -304,6 +310,10 @@ struct mtk_imgsys_dev {
struct device *smem_dev;
/* Number of modules */
int num_mods;
+ /* Synchronization and Flow Control */
+ struct kref init_kref;
+ struct mutex hw_op_lock; /* Protect HW operations */
+ int stream_cnt;
};
/**
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
new file mode 100644
index 000000000000..dd02a8fa28d4
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_CORE_REG_H_
+#define _MTK_IMGSYS_CORE_REG_H_
+
+/**
+ * Define the minimum and maximum hardware IDs
+ * for different MediaTek Image System components
+ */
+#define WPE_HW_MIN_ID IMGSYS_REG_WPE1
+#define WPE_HW_MAX_ID IMGSYS_REG_WPE3
+#define TRAW_HW_MIN_ID IMGSYS_REG_TRAW1
+#define TRAW_HW_MAX_ID IMGSYS_REG_TRAW3
+#define DIP_HW_MIN_ID IMGSYS_REG_DIP1
+#define DIP_HW_MAX_ID IMGSYS_REG_DIP2
+#define PQDIP_HW_MIN_ID IMGSYS_REG_PQDIP1
+#define PQDIP_HW_MAX_ID IMGSYS_REG_PQDIP2
+#define ME_HW_MIN_ID IMGSYS_REG_ME
+#define ME_HW_MAX_ID IMGSYS_REG_ME
+
+/* Macro to calculate the size of a hardware set */
+#define HW_SET_SIZE(min_id, max_id) ((max_id) - (min_id) + 1)
+
+/* Define the hardware set sizes */
+#define MTK_IMGSYS_WPE_HW_SET HW_SET_SIZE(WPE_HW_MIN_ID, WPE_HW_MAX_ID)
+#define MTK_IMGSYS_TRAW_HW_SET HW_SET_SIZE(TRAW_HW_MIN_ID, TRAW_HW_MAX_ID)
+#define MTK_IMGSYS_DIP_HW_SET HW_SET_SIZE(DIP_HW_MIN_ID, DIP_HW_MAX_ID)
+#define MTK_IMGSYS_PQDIP_HW_SET HW_SET_SIZE(PQDIP_HW_MIN_ID, PQDIP_HW_MAX_ID)
+#define MTK_IMGSYS_ME_HW_SET HW_SET_SIZE(ME_HW_MIN_ID, ME_HW_MAX_ID)
+
+/* Define the CQ's control register bits */
+#define CQ_THRX_CTL_EN BIT(0)
+#define CQ_THRX_CTL_MODE BIT(4)
+#define CQ_THRX_CTL (CQ_THRX_CTL_EN | CQ_THRX_CTL_MODE)
+
+/**
+ * enum mtk_imgsys_reg_id
+ *
+ * Definition about hw register map id
+ * The engine order should be the same as register order in dts
+ */
+enum mtk_imgsys_reg_id {
+ IMGSYS_REG_TOP = 0,
+ IMGSYS_REG_TRAW1,
+ IMGSYS_REG_TRAW2,
+ IMGSYS_REG_TRAW3,
+ IMGSYS_REG_DIP1,
+ IMGSYS_REG_DIP2,
+ IMGSYS_REG_PQDIP1,
+ IMGSYS_REG_PQDIP2,
+ IMGSYS_REG_WPE1,
+ IMGSYS_REG_WPE2,
+ IMGSYS_REG_WPE3,
+ IMGSYS_REG_WPE1_DIP1,
+ IMGSYS_REG_ME,
+ IMGSYS_REG_WPE2_DIP1,
+ IMGSYS_REG_WPE3_DIP1,
+ IMGSYS_REG_DIP1_TOP,
+ IMGSYS_REG_DIP2_TOP
+};
+
+/**
+ * struct mtk_imgsys_init_array - Array structure for initializing registers
+ * @offset: Offset of the register
+ * @val: Value to be written to the register
+ *
+ * This structure represents an array element used for initializing registers
+ * in the MediaTek Image System. Each element contains the offset of the register
+ * and the value to be written to that register.
+ */
+struct mtk_imgsys_init_array {
+ unsigned int offset;
+ unsigned int value;
+};
+#endif /* _MTK_IMGSYS_CORE_REG_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
new file mode 100644
index 000000000000..5daa8f863bd2
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "mtk_imgsys-module_common.h"
+#include "mtk_imgsys-module_main.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+/* Define the reset register bits */
+#define SW_RST (0x000C)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static void __iomem *imgsys_main_reg_base;
+static void __iomem *wpe_dip1_reg_base;
+static void __iomem *wpe_dip2_reg_base;
+static void __iomem *wpe_dip3_reg_base;
+static void __iomem *dip1_reg_base;
+static void __iomem *dip2_reg_base;
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+static void imgsys_main_hw_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ void __iomem *wpe_reg_base = NULL;
+ void __iomem *wpe_ctrl = NULL;
+ unsigned int hw_idx = 0;
+
+ /* Enable all LARBs of ImgSys */
+ /**
+ * The flow will be updated based on the patch series 958467 from
+ * the Linux MediaTek project. For more details, refer to the following link:
+ * https://patchwork.kernel.org/project/linux-mediatek/list/?series=958467
+ */
+
+ /* Reset ImgSys hardware */
+ iowrite32(0xFFFFFFFF, dip1_reg_base + SW_RST);
+ iowrite32(0xFFFFFFFF, dip2_reg_base + SW_RST);
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ if (hw_idx == 0)
+ wpe_reg_base = wpe_dip1_reg_base;
+ else if (hw_idx == 1)
+ wpe_reg_base = wpe_dip2_reg_base;
+ else
+ wpe_reg_base = wpe_dip3_reg_base;
+
+ wpe_ctrl = wpe_reg_base + SW_RST;
+ iowrite32(0xFFFFFFFF, wpe_ctrl);
+ iowrite32(0x0, wpe_ctrl);
+ }
+
+ iowrite32(0x00CF00FF, imgsys_main_reg_base + SW_RST);
+ iowrite32(0x0, imgsys_main_reg_base + SW_RST);
+
+ iowrite32(0x0, dip1_reg_base + SW_RST);
+ iowrite32(0x0, dip2_reg_base + SW_RST);
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ if (hw_idx == 0)
+ wpe_reg_base = wpe_dip1_reg_base;
+ else if (hw_idx == 1)
+ wpe_reg_base = wpe_dip2_reg_base;
+ else
+ wpe_reg_base = wpe_dip3_reg_base;
+
+ wpe_ctrl = wpe_reg_base + SW_RST;
+ iowrite32(0xFFFFFFFF, wpe_ctrl);
+ iowrite32(0x0, wpe_ctrl);
+ }
+
+ iowrite32(0x00CF00FF, imgsys_main_reg_base + SW_RST);
+ iowrite32(0x0, imgsys_main_reg_base + SW_RST);
+
+ /* Disable all LARBs of ImgSys*/
+ /**
+ * The flow will be updated based on the patch series 958467 from
+ * the Linux MediaTek project. For more details, refer to the following link:
+ * https://patchwork.kernel.org/project/linux-mediatek/list/?series=958467
+ */
+}
+
+void imgsys_main_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ imgsys_main_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_TOP);
+ if (!imgsys_main_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap imgsys_top registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ wpe_dip1_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_WPE1_DIP1);
+ if (!wpe_dip1_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap wpe_dip1 registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ wpe_dip2_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_WPE2_DIP1);
+ if (!wpe_dip2_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap wpe_dip2 registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ wpe_dip3_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_WPE3_DIP1);
+ if (!wpe_dip3_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap wpe_dip3 registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ dip1_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_DIP1_TOP);
+ if (!dip1_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap dip_top registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ dip2_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_DIP2_TOP);
+ if (!dip2_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap dip_top_nr registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ imgsys_main_hw_init(imgsys_dev);
+}
+
+void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ if (imgsys_main_reg_base) {
+ iounmap(imgsys_main_reg_base);
+ imgsys_main_reg_base = NULL;
+ }
+
+ if (wpe_dip1_reg_base) {
+ iounmap(wpe_dip1_reg_base);
+ wpe_dip1_reg_base = NULL;
+ }
+
+ if (wpe_dip2_reg_base) {
+ iounmap(wpe_dip2_reg_base);
+ wpe_dip2_reg_base = NULL;
+ }
+
+ if (wpe_dip3_reg_base) {
+ iounmap(wpe_dip3_reg_base);
+ wpe_dip3_reg_base = NULL;
+ }
+
+ if (dip1_reg_base) {
+ iounmap(dip1_reg_base);
+ dip1_reg_base = NULL;
+ }
+
+ if (dip2_reg_base) {
+ iounmap(dip2_reg_base);
+ dip2_reg_base = NULL;
+ }
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
new file mode 100644
index 000000000000..2a3fbd08d128
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_DEBUG_H_
+#define _MTK_IMGSYS_DEBUG_H_
+
+#include "mtk_imgsys-dev.h"
+
+void imgsys_main_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_DEBUG_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
index 6dc4b7b60832..a2eea8f4fb3c 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
@@ -24,4 +24,30 @@ enum mtk_imgsys_module {
IMGSYS_MOD_MAX,
};
+struct mtk_imgsys_dev;
+
+/**
+ * struct module_ops - Operations for a specific module in the MediaTek Image System
+ * @module_id: Identifier for the module.
+ * @init: Function pointer to initialize the module.
+ * This function sets the default values for various software-related
+ * parameters and ensures that the module is ready for operation.
+ * Example function: imgsys_dip_init
+ * @uninit: Function pointer to uninitialize the module.
+ * This function resets various software-related parameters to ensure
+ * the module is properly cleaned up and ready for shutdown.
+ * Example function: imgsys_dip_uninit
+ *
+ * This structure defines a set of operations for specific modules within the
+ * MediaTek Image System, such as WPE (Warping Engine), TRAW (Tile-Raw),
+ * DIP (Digital Image Processing), PQDIP (Picture Quality Digital Image
+ * Processing), and ME (Motion Estimation). Each module is identified by
+ * a unique identifier and includes function pointers for operations like
+ * initialization and deinitialization.
+ */
+struct module_ops {
+ int module_id;
+ void (*init)(struct mtk_imgsys_dev *imgsys_dev);
+ void (*uninit)(struct mtk_imgsys_dev *imgsys_dev);
+};
#endif /* _MTK_IMGSYS_MODULES_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
index 5088c9df704c..f34ea9465745 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
@@ -12,6 +12,7 @@
#include <linux/clk.h>
#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-module_ops.h"
/**
* struct mtk_imgsys_mod_pipe_desc - Description of a module's pipeline
@@ -31,6 +32,7 @@ struct mtk_imgsys_mod_pipe_desc {
* @mod_num: Number of modules in the pipeline.
* @pipe_settings: Pointer to pipeline settings for the module.
* @pipe_num: Number of pipeline settings.
+ * @imgsys_modules: Pointer to module operations for image system modules.
*/
struct cust_data {
struct clk_bulk_data *clks;
@@ -39,6 +41,7 @@ struct cust_data {
unsigned int mod_num;
const struct mtk_imgsys_pipe_desc *pipe_settings;
unsigned int pipe_num;
+ const struct module_ops *imgsys_modules;
};
/**
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 9cdcdb0a0200..9f118491b37d 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -6,7 +6,45 @@
*
*/
+#include <linux/pm_runtime.h>
#include "mtk_imgsys-sys.h"
+#include "mtk_imgsys-dev.h"
+
+static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
+{
+ int ret, i;
+
+ ret = pm_runtime_get_sync(imgsys_dev->dev);
+ if (ret < 0) {
+ dev_err(imgsys_dev->dev, "Failed to sync runtime PM: %d\n", ret);
+ return ret;
+ }
+
+ /* Set default value for hardware modules */
+ for (i = 0; i < (imgsys_dev->num_mods); i++)
+ imgsys_dev->modules[i].init(imgsys_dev);
+
+ kref_init(&imgsys_dev->init_kref);
+
+ return 0;
+}
+
+static void module_uninit(struct kref *kref)
+{
+ struct mtk_imgsys_dev *imgsys_dev;
+ int i;
+
+ imgsys_dev = container_of(kref, struct mtk_imgsys_dev, init_kref);
+
+ for (i = 0; i < (imgsys_dev->num_mods); i++)
+ if (imgsys_dev->modules[i].uninit)
+ imgsys_dev->modules[i].uninit(imgsys_dev);
+}
+
+static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
+{
+ kref_put(&imgsys_dev->init_kref, module_uninit);
+}
void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
struct mtk_imgsys_request *req)
@@ -16,12 +54,43 @@ void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
{
- /* Function implementation will be provided in subsequent patches */
+ struct mtk_imgsys_dev *imgsys_dev = pipe->imgsys_dev;
+ int ret;
+
+ mutex_lock(&imgsys_dev->hw_op_lock);
+ if (!imgsys_dev->stream_cnt) {
+ ret = mtk_imgsys_hw_connect(pipe->imgsys_dev);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s: pipe(%d) connect to dip_hw failed\n",
+ __func__, pipe->desc->name, pipe->desc->id);
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+ return ret;
+ }
+ }
+ imgsys_dev->stream_cnt++;
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: stream on, id(%d), stream cnt(%d)\n",
+ __func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
+
return 0;
}
int mtk_imgsys_hw_streamoff(struct mtk_imgsys_pipe *pipe)
{
- /* Function implementation will be provided in subsequent patches */
+ struct mtk_imgsys_dev *imgsys_dev = pipe->imgsys_dev;
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: stream off, id(%d), stream cnt(%d)\n",
+ __func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
+
+ mutex_lock(&imgsys_dev->hw_op_lock);
+ imgsys_dev->stream_cnt--;
+ if (!imgsys_dev->stream_cnt)
+ mtk_imgsys_hw_disconnect(imgsys_dev);
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+
return 0;
}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index 919c69493bbc..e25f17b98649 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -18,8 +18,13 @@
#include <media/v4l2-subdev.h>
#include <media/v4l2-event.h>
#include "linux/mtkisp_imgsys.h"
-#include "mtk_imgsys-of.h"
#include "mtk_imgsys-module_ops.h"
+#include "mtk_imgsys-module_main.h"
+#include "modules/mtk_imgsys-dip.h"
+#include "modules/mtk_imgsys-traw.h"
+#include "modules/mtk_imgsys-pqdip.h"
+#include "modules/mtk_imgsys-wpe.h"
+#include "mtk_imgsys-of.h"
#include "mtk_imgsys-vnode_id.h"
#include "mtk_imgsys_v4l2_vnode.h"
#include "mtk_imgsys-dev.h"
@@ -1499,6 +1504,7 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
init_imgsys_pipeline(data);
imgsys_dev->cust_pipes = data->pipe_settings;
+ imgsys_dev->modules = data->imgsys_modules;
imgsys_dev->dev = &pdev->dev;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -1511,6 +1517,8 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
dev_set_drvdata(&pdev->dev, imgsys_dev);
imgsys_dev->num_mods = data->mod_num;
+ imgsys_dev->stream_cnt = 0;
+ mutex_init(&imgsys_dev->hw_op_lock);
/* Get Clocks */
imgsys_dev->clks = data->clks;
@@ -1613,6 +1621,7 @@ static void mtk_imgsys_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
platform_driver_unregister(&mtk_imgsys_larb_driver);
mtk_imgsys_dev_v4l2_release(imgsys_dev);
+ mutex_destroy(&imgsys_dev->hw_op_lock);
devm_kfree(&pdev->dev, imgsys_dev->larbs);
devm_kfree(&pdev->dev, imgsys_dev);
}
@@ -1709,6 +1718,39 @@ static struct clk_bulk_data imgsys_isp7_clks[] = {
{ .id = "ME_CG_LARB12" },
};
+static const struct module_ops imgsys_isp7_modules[] = {
+ [IMGSYS_MOD_IMGMAIN] = {
+ .module_id = IMGSYS_MOD_IMGMAIN,
+ .init = imgsys_main_init,
+ .uninit = imgsys_main_uninit,
+ },
+ [IMGSYS_MOD_WPE] = {
+ .module_id = IMGSYS_MOD_WPE,
+ .init = imgsys_wpe_init,
+ .uninit = imgsys_wpe_uninit,
+ },
+ [IMGSYS_MOD_TRAW] = {
+ .module_id = IMGSYS_MOD_TRAW,
+ .init = imgsys_traw_init,
+ .uninit = imgsys_traw_uninit,
+ },
+ [IMGSYS_MOD_DIP] = {
+ .module_id = IMGSYS_MOD_DIP,
+ .init = imgsys_dip_init,
+ .uninit = imgsys_dip_uninit,
+ },
+ [IMGSYS_MOD_PQDIP] = {
+ .module_id = IMGSYS_MOD_PQDIP,
+ .init = imgsys_pqdip_init,
+ .uninit = imgsys_pqdip_uninit,
+ },
+ [IMGSYS_MOD_ME] = {
+ .module_id = IMGSYS_MOD_ME,
+ .init = NULL,
+ .uninit = NULL,
+ },
+};
+
static const struct cust_data imgsys_data[] = {
[0] = {
.clks = imgsys_isp7_clks,
@@ -1717,6 +1759,7 @@ static const struct cust_data imgsys_data[] = {
.mod_num = ARRAY_SIZE(module_pipe_isp7),
.pipe_settings = pipe_settings_isp7,
.pipe_num = ARRAY_SIZE(pipe_settings_isp7),
+ .imgsys_modules = imgsys_isp7_modules,
},
};
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (5 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 11:50 ` [PATCH v1 08/10] media: mediatek: isp: Add SCP " Olivia Wen
` (2 subsequent siblings)
9 siblings, 0 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on CMDQ (Command Queue) support for the MediaTek Image System
driver. CMDQ is essential for managing and executing hardware commands
efficiently, ensuring smooth operation of image processing tasks within
the ImgSys architecture.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 1 +
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h | 502 +++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq.c | 690 ++++++++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq.h | 29 +
.../isp_7x/imgsys/mtk_imgsys-module_common.h | 20 +
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 5 +
6 files changed, 1247 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 26c8c1d39750..c0f0f3a0c38a 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,6 +2,7 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-cmdq.o \
mtk_imgsys-module_main.o \
mtk_imgsys-of.o \
mtk_imgsys-formats.o \
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
new file mode 100644
index 000000000000..801a12e70bac
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
@@ -0,0 +1,502 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_CMDQ_PLAT_H_
+#define _MTK_IMGSYS_CMDQ_PLAT_H_
+
+#include "mtk_imgsys-cmdq.h"
+
+/* Maximum number of frame blocks for GCE recording */
+#define GCE_REC_MAX_FRAME_BLOCK 40
+
+/* Maximum number of tile blocks for GCE recording */
+#define GCE_REC_MAX_TILE_BLOCK 40
+
+/* Maximum number of label counts for GCE recording */
+#define GCE_REC_MAX_LABEL_COUNT 2048
+
+/* Maximum number of image system engines */
+#define IMGSYS_ENG_MAX 10
+
+/* Start address of the image system register */
+#define IMGSYS_REG_START 0x15000000
+
+/* End address of the image system register */
+#define IMGSYS_REG_END 0x1570FFFF
+
+/* Number of normal camera's thread */
+#define IMGSYS_NOR_THD 10
+
+/* Define the constants for buffer validation */
+#define HEADER_CODE 0x5A5A5A5A
+#define CHECK_PRE 0x55AA55AA
+#define CHECK_POST 0xAA55AA55
+#define FOOTER_CODE 0xA5A5A5A5
+
+/* Enumeration for Image System (IMGSYS) events */
+enum mtk_imgsys_event {
+ /* HW event */
+ /* TRAW frame done events */
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_0 = 0,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_1,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_2,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_3,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_4,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_5,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_6,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_7,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_8,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_9,
+ /* LTRAW frame done events */
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_0,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_1,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_2,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_3,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_4,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_5,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_6,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_7,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_8,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_9,
+ /* XTRAW frame done events */
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_0,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_1,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_2,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_3,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_4,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_5,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_6,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_7,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_8,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_9,
+ /* DIP frame done events */
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_0,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_1,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_2,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_3,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_4,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_5,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_6,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_7,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_8,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_9,
+ /* PQ_DIP_A frame done events */
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_0,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_1,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_2,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_3,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_4,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_5,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_6,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_7,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_8,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_9,
+ /* PQ_DIP_B frame done events */
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_0,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_1,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_2,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_3,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_4,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_5,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_6,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_7,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_8,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_9,
+ /* WPE_EIS frame done events */
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_0,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_1,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_2,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_3,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_4,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_5,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_6,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_7,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_8,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_9,
+ /* WPE_TNR frame done events */
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_0,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_1,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_2,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_3,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_4,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_5,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_6,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_7,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_8,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_9,
+ /* WPE_LITE frame done events */
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_0,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_1,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_2,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_3,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_4,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_5,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_6,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_7,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_8,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_9,
+ /* ME done events */
+ IMGSYS_EVENT_IMGSYS_IPE_ME_DONE,
+ /* SW events */
+ IMGSYS_EVENT_SYNC_TOKEN_START,
+ IMGSYS_EVENT_SYNC_TOKEN_WPE_EIS = IMGSYS_EVENT_SYNC_TOKEN_START,
+ IMGSYS_EVENT_SYNC_TOKEN_WPE_TNR,
+ IMGSYS_EVENT_SYNC_TOKEN_WPE_LITE,
+ IMGSYS_EVENT_SYNC_TOKEN_TRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_LTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_XTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_DIP,
+ IMGSYS_EVENT_SYNC_TOKEN_PQDIP_A,
+ IMGSYS_EVENT_SYNC_TOKEN_PQDIP_B,
+ IMGSYS_EVENT_SYNC_TOKEN_IPESYS_ME,
+ IMGSYS_EVENT_SYNC_TOKEN_APUSYS_APU,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_TRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_LTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_XTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_DIP,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_START,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_1 = IMGSYS_EVENT_SYNC_TOKEN_POOL_START,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_2,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_3,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_4,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_5,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_6,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_7,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_8,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_9,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_10,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_11,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_12,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_13,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_14,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_15,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_16,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_17,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_18,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_19,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_20,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_21,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_22,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_23,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_24,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_25,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_26,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_27,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_28,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_29,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_30,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_31,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_32,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_33,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_34,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_35,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_36,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_37,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_38,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_39,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_40,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_41,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_42,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_43,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_44,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_45,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_46,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_47,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_48,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_49,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_50,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_51,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_52,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_53,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_54,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_55,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_56,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_57,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_58,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_59,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_60,
+ IMGSYS_EVENT_SYNC_TOKEN_END = IMGSYS_EVENT_SYNC_TOKEN_POOL_60,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_1,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_2,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_3,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_4,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_5,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_6,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_7,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_8,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_9,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_10,
+ IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_WAIT,
+ IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_SET,
+ IMGSYS_EVENT_MAX
+};
+
+/* Structure to represent an IMGSYS event */
+struct imgsys_event_table {
+ u16 event;
+ char dts_name[256];
+};
+
+/* Event table for Image System (IMGSYS) */
+static struct imgsys_event_table imgsys_event[] = {
+ /* TRAW frame done events */
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_0, "traw_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_1, "traw_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_2, "traw_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_3, "traw_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_4, "traw_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_5, "traw_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_6, "traw_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_7, "traw_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_8, "traw_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_9, "traw_cq_thread9_frame_done"},
+ /* LTRAW frame done events */
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_0, "ltraw_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_1, "ltraw_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_2, "ltraw_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_3, "ltraw_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_4, "ltraw_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_5, "ltraw_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_6, "ltraw_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_7, "ltraw_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_8, "ltraw_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_9, "ltraw_cq_thread9_frame_done"},
+ /* XTRAW frame done events */
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_0, "xtraw_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_1, "xtraw_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_2, "xtraw_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_3, "xtraw_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_4, "xtraw_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_5, "xtraw_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_6, "xtraw_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_7, "xtraw_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_8, "xtraw_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_9, "xtraw_cq_thread9_frame_done"},
+ /* DIP frame done events */
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_0, "dip_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_1, "dip_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_2, "dip_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_3, "dip_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_4, "dip_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_5, "dip_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_6, "dip_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_7, "dip_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_8, "dip_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_9, "dip_cq_thread9_frame_done"},
+ /* PQ_DIP_A frame done events */
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_0, "pqa_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_1, "pqa_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_2, "pqa_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_3, "pqa_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_4, "pqa_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_5, "pqa_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_6, "pqa_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_7, "pqa_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_8, "pqa_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_9, "pqa_cq_thread9_frame_done"},
+ /* PQ_DIP_B frame done events */
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_0, "pqb_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_1, "pqb_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_2, "pqb_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_3, "pqb_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_4, "pqb_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_5, "pqb_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_6, "pqb_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_7, "pqb_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_8, "pqb_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_9, "pqb_cq_thread9_frame_done"},
+ /* WPE_EIS frame done events */
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_0, "wpe_eis_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_1, "wpe_eis_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_2, "wpe_eis_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_3, "wpe_eis_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_4, "wpe_eis_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_5, "wpe_eis_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_6, "wpe_eis_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_7, "wpe_eis_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_8, "wpe_eis_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_9, "wpe_eis_cq_thread9_frame_done"},
+ /* WPE_TNR frame done events */
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_0, "wpe_tnr_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_1, "wpe_tnr_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_2, "wpe_tnr_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_3, "wpe_tnr_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_4, "wpe_tnr_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_5, "wpe_tnr_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_6, "wpe_tnr_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_7, "wpe_tnr_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_8, "wpe_tnr_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_9, "wpe_tnr_cq_thread9_frame_done"},
+ /* WPE_LITE frame done events */
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_0, "wpe_lite_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_1, "wpe_lite_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_2, "wpe_lite_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_3, "wpe_lite_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_4, "wpe_lite_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_5, "wpe_lite_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_6, "wpe_lite_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_7, "wpe_lite_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_8, "wpe_lite_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_9, "wpe_lite_cq_thread9_frame_done"},
+ /* ME done events */
+ { IMGSYS_EVENT_IMGSYS_IPE_ME_DONE, "me_done"},
+ /* SW events */
+ { IMGSYS_EVENT_SYNC_TOKEN_WPE_EIS, "wpe_eis_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_WPE_TNR, "wpe_tnr_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_WPE_LITE, "wpe_lite_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_TRAW, "traw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_LTRAW, "ltraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_XTRAW, "xtraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_DIP, "dip_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_PQDIP_A, "pqdip_a_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_PQDIP_B, "pqdip_b_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_IPESYS_ME, "me_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_APUSYS_APU, "apu_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_TRAW, "vss_traw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_LTRAW, "vss_ltraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_XTRAW, "vss_xtraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_DIP, "vss_dip_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_1, "sw_sync_token_pool_1"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_2, "sw_sync_token_pool_2"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_3, "sw_sync_token_pool_3"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_4, "sw_sync_token_pool_4"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_5, "sw_sync_token_pool_5"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_6, "sw_sync_token_pool_6"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_7, "sw_sync_token_pool_7"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_8, "sw_sync_token_pool_8"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_9, "sw_sync_token_pool_9"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_10, "sw_sync_token_pool_10"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_11, "sw_sync_token_pool_11"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_12, "sw_sync_token_pool_12"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_13, "sw_sync_token_pool_13"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_14, "sw_sync_token_pool_14"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_15, "sw_sync_token_pool_15"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_16, "sw_sync_token_pool_16"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_17, "sw_sync_token_pool_17"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_18, "sw_sync_token_pool_18"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_19, "sw_sync_token_pool_19"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_20, "sw_sync_token_pool_20"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_21, "sw_sync_token_pool_21"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_22, "sw_sync_token_pool_22"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_23, "sw_sync_token_pool_23"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_24, "sw_sync_token_pool_24"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_25, "sw_sync_token_pool_25"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_26, "sw_sync_token_pool_26"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_27, "sw_sync_token_pool_27"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_28, "sw_sync_token_pool_28"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_29, "sw_sync_token_pool_29"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_30, "sw_sync_token_pool_30"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_31, "sw_sync_token_pool_31"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_32, "sw_sync_token_pool_32"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_33, "sw_sync_token_pool_33"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_34, "sw_sync_token_pool_34"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_35, "sw_sync_token_pool_35"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_36, "sw_sync_token_pool_36"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_37, "sw_sync_token_pool_37"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_38, "sw_sync_token_pool_38"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_39, "sw_sync_token_pool_39"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_40, "sw_sync_token_pool_40"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_41, "sw_sync_token_pool_41"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_42, "sw_sync_token_pool_42"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_43, "sw_sync_token_pool_43"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_44, "sw_sync_token_pool_44"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_45, "sw_sync_token_pool_45"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_46, "sw_sync_token_pool_46"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_47, "sw_sync_token_pool_47"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_48, "sw_sync_token_pool_48"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_49, "sw_sync_token_pool_49"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_50, "sw_sync_token_pool_50"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_51, "sw_sync_token_pool_51"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_52, "sw_sync_token_pool_52"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_53, "sw_sync_token_pool_53"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_54, "sw_sync_token_pool_54"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_55, "sw_sync_token_pool_55"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_56, "sw_sync_token_pool_56"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_57, "sw_sync_token_pool_57"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_58, "sw_sync_token_pool_58"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_59, "sw_sync_token_pool_59"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_60, "sw_sync_token_pool_60"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_1, "sw_sync_token_camsys_pool_1"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_2, "sw_sync_token_camsys_pool_2"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_3, "sw_sync_token_camsys_pool_3"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_4, "sw_sync_token_camsys_pool_4"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_5, "sw_sync_token_camsys_pool_5"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_6, "sw_sync_token_camsys_pool_6"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_7, "sw_sync_token_camsys_pool_7"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_8, "sw_sync_token_camsys_pool_8"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_9, "sw_sync_token_camsys_pool_9"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_10, "sw_sync_token_camsys_pool_10"},
+ { IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_WAIT, "sw_sync_token_tzmp_isp_wait"},
+ { IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_SET, "sw_sync_token_tzmp_isp_set"},
+ { IMGSYS_EVENT_MAX, "imgsys_event_max"},
+};
+
+/* Structure to record block information */
+struct block_record {
+ u32 label_min; /* Minimum label value */
+ u32 label_max; /* Maximum label value */
+ u32 label_count; /* Count of labels */
+ u32 cmd_offset; /* Offset of the command */
+ u32 cmd_length; /* Length of the command */
+};
+
+/* Enumeration for GCE recording block types */
+enum gce_rec_block_enum {
+ GCE_REC_NONE_BLOCK = -1, /* No block */
+ GCE_REC_FRAME_BLOCK = 0, /* Frame block */
+ GCE_REC_TILE_BLOCK /* Tile block */
+};
+
+/* Enumeration for GCE recording modes */
+enum gce_rec_mode_enum {
+ GCE_REC_APPEND_MODE = 0, /* Append mode */
+ GCE_REC_REPLACE_MODE /* Replace mode */
+};
+
+/**
+ *This structure is utilized by the SCP (Secondary Control Processor),
+ * which is another CPU in the system. It contains necessary information
+ * for managing software frames across different processing units.
+ */
+struct gce_recorder {
+ u32 header_code; /* Header guard word */
+ u32 cmd_offset; /* Command offset */
+ u32 check_pre; /* Pre-check guard word */
+ u32 *p_output; /* Output buffer pointer */
+ u32 check_post; /* Post-check guard word */
+ u32 *p_buffer; /* Buffer pointer */
+ u32 max_length; /* Maximum length of the buffer */
+ u32 curr_length; /* Current length of the buffer */
+
+ struct block_record frame_record[GCE_REC_MAX_FRAME_BLOCK]; /* Frame block records */
+ u32 frame_block; /* Number of frame blocks */
+ u32 curr_frame; /* Current frame block index */
+
+ struct block_record tile_record[GCE_REC_MAX_TILE_BLOCK]; /* Tile block records */
+ u32 tile_block; /* Number of tile blocks */
+ u32 curr_tile; /* Current tile block index */
+
+ enum gce_rec_block_enum curr_block; /* Current block type */
+ enum gce_rec_mode_enum curr_mode; /* Current recording mode */
+
+ u32 orig_index; /* Original index */
+ u32 *p_orig_out; /* Original output buffer pointer */
+ u32 curr_label; /* Current label index */
+
+ enum gce_rec_block_enum label_block; /* Current label block type */
+
+ u32 frame_label[GCE_REC_MAX_LABEL_COUNT]; /* Frame labels */
+ u32 frame_count; /* Number of frame labels */
+
+ u32 tile_label[GCE_REC_MAX_LABEL_COUNT]; /* Tile labels */
+ u32 tile_count; /* Number of tile labels */
+
+ u32 footer_code; /* Footer guard word */
+};
+#endif /* _MTK_IMGSYS_CMDQ_PLAT_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
new file mode 100644
index 000000000000..7628ade90b6b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
@@ -0,0 +1,690 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+#include <linux/dma-mapping.h>
+#include <linux/mailbox_controller.h>
+#include <linux/math64.h>
+#include <linux/platform_device.h>
+#include <linux/soc/mediatek/mtk-cmdq.h>
+#include <linux/pm_opp.h>
+#include <linux/pm_runtime.h>
+#include <linux/regulator/consumer.h>
+#include <linux/vmalloc.h>
+#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-cmdq-plat.h"
+#include "mtk_imgsys-module_common.h"
+
+#define MAX_FRAME_IN_TASK 64
+#define CMDQ_BUF_SIZE 0x4000
+
+static struct workqueue_struct *imgsys_cmdq_wq;
+static u32 is_stream_off;
+
+/**
+ * enum mtk_imgsys_cmd - Commands for image system operations
+ * @IMGSYS_CMD_WRITE: Write operation
+ * @IMGSYS_CMD_POLL: Poll operation
+ * @IMGSYS_CMD_WAIT: Wait operation
+ * @IMGSYS_CMD_UPDATE: Update operation
+ * @IMGSYS_CMD_ACQUIRE: Acquire operation
+ * @IMGSYS_CMD_STOP: Stop operation
+ *
+ * This enumeration defines the various commands that determine how to use the
+ * cmdq API for image system operations.
+ */
+enum mtk_imgsys_cmd {
+ IMGSYS_CMD_WRITE,
+ IMGSYS_CMD_POLL,
+ IMGSYS_CMD_WAIT,
+ IMGSYS_CMD_UPDATE,
+ IMGSYS_CMD_ACQUIRE,
+ IMGSYS_CMD_STOP
+};
+
+/**
+ * struct access - Access parameters for image system commands
+ * @address: Memory address to access
+ * @value: Value to write to the memory address
+ * @mask: Mask for the value
+ *
+ * This structure defines the parameters for memory access operations used by
+ * IMGSYS_CMD_WRITE and IMGSYS_CMD_POLL commands.
+ */
+struct access {
+ u64 address;
+ u32 value;
+ u32 mask;
+} __packed;
+
+/**
+ * struct event - Event parameters for image system commands
+ * @event: Event identifier
+ * @action: Action to perform for the event
+ *
+ * This structure defines the parameters for event-based operations used by
+ * IMGSYS_CMD_WAIT, IMGSYS_CMD_UPDATE and IMGSYS_CMD_ACQUIRE commands.
+ */
+struct event {
+ u32 event;
+ u32 action;
+} __packed;
+
+/**
+ * struct command - Command structure for image system operations
+ * @opcode: Operation code (command)
+ * @param: Union of parameters for different commands
+ * @write: Parameters for write command
+ * @poll: Parameters for poll command
+ * @wait: Parameters for wait command
+ * @update: Parameters for update command
+ * @acquire: Parameters for acquire command
+ *
+ * This structure is used to store the instructions and parameters for how to
+ * use the cmdq driver.
+ */
+struct command {
+ enum mtk_imgsys_cmd opcode;
+ union {
+ struct access write;
+ struct access poll;
+ struct event wait;
+ struct event update;
+ struct event acquire;
+ } param;
+} __packed;
+
+/**
+ * struct mtk_imgsys_cb_param - Callback parameters for image system commands
+ * @cmdq_cb_work: Work structure for command queue callback
+ * @pkt: Command packet
+ * @frm_info: Pointer to frame information
+ * @imgsys_dev: Pointer to image system device
+ * @clt: Pointer to command queue client
+ * @user_cmdq_cb: User-defined command queue callback function
+ * @err: Error code
+ * @frm_idx: Frame index
+ * @frm_num: Number of frames
+ * @blk_idx: Block index
+ * @blk_num: Number of blocks
+ * @is_earlycb: Indicates if it is an early callback
+ * @group_id: Group identifier
+ * @thd_idx: Thread index
+ * @task_id: Task identifier
+ * @task_num: Number of tasks
+ * @task_cnt: Task count
+ * @pkt_ofst: Array of packet offsets for frames in task
+ * @is_blk_last: Indicates if it is the last block
+ * @is_frm_last: Indicates if it is the last frame
+ * @is_task_last: Indicates if it is the last task
+ *
+ * This structure defines the parameters for callbacks related to
+ * image system commands.
+ */
+struct mtk_imgsys_cb_param {
+ struct work_struct cmdq_cb_work;
+ struct cmdq_pkt pkt;
+ struct swfrm_info *frm_info;
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct cmdq_client *clt;
+ void (*user_cmdq_cb)(struct imgsys_cmdq_cb_data data);
+ s32 err;
+ u32 frm_idx;
+ u32 frm_num;
+ u32 blk_idx;
+ u32 blk_num;
+ u32 is_earlycb;
+ s32 group_id;
+ u32 thd_idx;
+ u32 task_id;
+ u32 task_num;
+ u32 task_cnt;
+ size_t pkt_ofst[MAX_FRAME_IN_TASK];
+ bool is_blk_last;
+ bool is_frm_last;
+ bool is_task_last;
+};
+
+static void imgsys_cmdq_task_cb(struct mbox_client *cl, void *mssg);
+static struct cmdq_client *imgsys_clt[IMGSYS_ENG_MAX];
+
+void imgsys_cmdq_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct device *dev = imgsys_dev->dev;
+ u32 idx = 0;
+
+ /* Only first user has to allocate the work queue */
+ if (!imgsys_cmdq_wq) {
+ imgsys_cmdq_wq =
+ alloc_ordered_workqueue("%s", __WQ_LEGACY |
+ WQ_MEM_RECLAIM |
+ WQ_FREEZABLE,
+ "imgsys_cmdq_cb_wq");
+ if (!imgsys_cmdq_wq) {
+ dev_dbg(imgsys_dev->dev,
+ "%s:allcate cmdq callback work queue fail!\n",
+ __func__);
+ return;
+ }
+ }
+
+ /* Request mailbox channels for each IMGSYS engine */
+ for (idx = 0; idx < IMGSYS_ENG_MAX; idx++) {
+ imgsys_clt[idx] = kzalloc(sizeof(*imgsys_clt[idx]), GFP_KERNEL);
+ if (!imgsys_clt[idx])
+ return;
+
+ imgsys_clt[idx]->client.dev = dev;
+ imgsys_clt[idx]->client.tx_block = false;
+ imgsys_clt[idx]->client.knows_txdone = true;
+ imgsys_clt[idx]->client.rx_callback = imgsys_cmdq_task_cb;
+ imgsys_clt[idx]->chan =
+ mbox_request_channel(&imgsys_clt[idx]->client, idx);
+ }
+
+ /* Parse events from dts */
+ for (idx = 0; idx < IMGSYS_EVENT_MAX; idx++)
+ of_property_read_u16(dev->of_node,
+ imgsys_event[idx].dts_name,
+ &imgsys_event[idx].event);
+}
+
+void imgsys_cmdq_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ u32 idx = 0;
+
+ /* Destroy cmdq client */
+ for (idx = 0; idx < IMGSYS_ENG_MAX; idx++) {
+ cmdq_mbox_destroy(imgsys_clt[idx]);
+ imgsys_clt[idx] = NULL;
+ }
+
+ /* Release work_quque */
+ flush_workqueue(imgsys_cmdq_wq);
+ destroy_workqueue(imgsys_cmdq_wq);
+ imgsys_cmdq_wq = NULL;
+}
+
+void imgsys_cmdq_streamon(struct mtk_imgsys_dev *imgsys_dev)
+{
+ dev_info(imgsys_dev->dev, "%s: cmdq stream on (%d)\n", __func__, is_stream_off);
+ is_stream_off = 0;
+}
+
+void imgsys_cmdq_streamoff(struct mtk_imgsys_dev *imgsys_dev)
+{
+ dev_info(imgsys_dev->dev, "%s: cmdq stream off (%d)\n", __func__, is_stream_off);
+ is_stream_off = 1;
+}
+
+static void imgsys_cmdq_cmd_dump(struct swfrm_info *frm_info, u32 frm_idx)
+{
+ struct gce_recorder *cmd_buf = NULL;
+ struct command *cmd = NULL;
+ u32 cmd_num = 0;
+ u32 cmd_idx = 0;
+
+ cmd_buf = (struct gce_recorder *)frm_info->user_info[frm_idx].g_swbuf;
+ cmd_num = cmd_buf->curr_length / sizeof(struct command);
+
+ if (sizeof(struct gce_recorder) != (u64)cmd_buf->cmd_offset) {
+ pr_info("%s: [ERROR] cmd offset is not match (0x%x/0x%zx)!\n",
+ __func__, cmd_buf->cmd_offset,
+ sizeof(struct gce_recorder));
+ return;
+ }
+
+ cmd = (struct command *)((char *)(frm_info->user_info[frm_idx].g_swbuf) +
+ (unsigned long)(cmd_buf->cmd_offset));
+
+ for (cmd_idx = 0; cmd_idx < cmd_num; cmd_idx++) {
+ switch (cmd[cmd_idx].opcode) {
+ case IMGSYS_CMD_WRITE:
+ pr_info("WRITE with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ cmd[cmd_idx].param.write.address,
+ cmd[cmd_idx].param.write.value,
+ cmd[cmd_idx].param.write.mask);
+ break;
+ case IMGSYS_CMD_POLL:
+ pr_info("POLL with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ cmd[cmd_idx].param.poll.address,
+ cmd[cmd_idx].param.poll.value,
+ cmd[cmd_idx].param.poll.mask);
+ break;
+ case IMGSYS_CMD_WAIT:
+ pr_info("WAIT event(%d/%d) action(%d)\n",
+ cmd[cmd_idx].param.wait.event,
+ imgsys_event[cmd[cmd_idx].param.wait.event].event,
+ cmd[cmd_idx].param.wait.action);
+ break;
+ case IMGSYS_CMD_UPDATE:
+ pr_info("UPDATE event(%d/%d) action(%d)\n",
+ cmd[cmd_idx].param.update.event,
+ imgsys_event[cmd[cmd_idx].param.update.event].event,
+ cmd[cmd_idx].param.update.action);
+ break;
+ case IMGSYS_CMD_ACQUIRE:
+ pr_info("ACQUIRE event(%d/%d) action(%d)\n",
+ cmd[cmd_idx].param.acquire.event,
+ imgsys_event[cmd[cmd_idx].param.acquire.event].event,
+ cmd[cmd_idx].param.acquire.action);
+ break;
+ case IMGSYS_CMD_STOP:
+ pr_info("%s: End Of Cmd!\n", __func__);
+ break;
+ default:
+ pr_info("%s: [ERROR]Not Support Cmd(%d)!\n",
+ __func__, cmd[cmd_idx].opcode);
+ break;
+ }
+ }
+}
+
+static void imgsys_cmdq_cb_work(struct work_struct *work)
+{
+ struct mtk_imgsys_cb_param *cb_param = NULL;
+ struct mtk_imgsys_dev *imgsys_dev = NULL;
+ u32 cb_frm_cnt;
+
+ cb_param = container_of(work, struct mtk_imgsys_cb_param, cmdq_cb_work);
+ imgsys_dev = cb_param->imgsys_dev;
+
+ if (cb_param->err != 0) {
+ pr_info("%s: [ERROR] cb(%p) cb_cnt(%d) req fd/no(%d/%d) error(%d).\n",
+ __func__, cb_param,
+ cb_param->frm_info->cb_frmcnt,
+ cb_param->frm_info->request_fd,
+ cb_param->frm_info->request_no,
+ cb_param->err);
+
+ pr_info("%s: Frame Info: frm(%d/%d) blk(%d/%d) earlycb(%d) lst(%d/%d/%d).\n",
+ __func__, cb_param->frm_idx, cb_param->frm_num,
+ cb_param->blk_idx, cb_param->blk_num, cb_param->is_earlycb,
+ cb_param->is_blk_last, cb_param->is_frm_last, cb_param->is_task_last);
+
+ pr_info("%s: CMDQ Info: task(%d/%d/%d) ofst(%zx/%zx/%zx/%zx/%zx).\n",
+ __func__, cb_param->task_id, cb_param->task_num, cb_param->task_cnt,
+ cb_param->pkt_ofst[0], cb_param->pkt_ofst[1], cb_param->pkt_ofst[2],
+ cb_param->pkt_ofst[3], cb_param->pkt_ofst[4]);
+ }
+
+ if (is_stream_off == 1)
+ pr_info("%s: [ERROR] cb(%p) pipe already streamoff(%d)!\n",
+ __func__, cb_param, is_stream_off);
+
+ cb_param->frm_info->cb_frmcnt++;
+ cb_frm_cnt = cb_param->frm_info->cb_frmcnt;
+
+ if (cb_param->is_blk_last && cb_param->user_cmdq_cb &&
+ (cb_param->frm_info->total_taskcnt == cb_frm_cnt ||
+ cb_param->is_earlycb)) {
+ struct imgsys_cmdq_cb_data user_cb_data;
+
+ user_cb_data.cmdq_data.sta = cb_param->err;
+ user_cb_data.cmdq_data.pkt = NULL;
+ user_cb_data.data = (void *)cb_param->frm_info;
+ cb_param->user_cmdq_cb(user_cb_data);
+ }
+
+ cmdq_pkt_destroy(cb_param->clt, &cb_param->pkt);
+ vfree(cb_param);
+}
+
+static void imgsys_cmdq_task_cb(struct mbox_client *cl, void *mssg)
+{
+ struct cmdq_cb_data *data = mssg;
+ struct mtk_imgsys_cb_param *cb_param =
+ container_of(data->pkt, struct mtk_imgsys_cb_param, pkt);
+
+ struct mtk_imgsys_pipe *pipe;
+ u32 frm_idx = 0;
+
+ cb_param->err = data->sta;
+
+ pr_debug("%s: Receive cb(%p) with sta(%d) for frm(%d/%d)\n",
+ __func__, cb_param, data->sta, cb_param->frm_idx,
+ cb_param->frm_num);
+
+ if (cb_param->err != 0) {
+ frm_idx = cb_param->frm_idx;
+ pr_info("%s: [ERROR] cb(%p) error info(%d).\n",
+ __func__, cb_param, cb_param->err);
+
+ pr_info("%s: Frame Info: frm(%d/%d) blk(%d/%d) earlycb(%d) lst(%d/%d/%d).\n",
+ __func__, cb_param->frm_idx, cb_param->frm_num,
+ cb_param->blk_idx, cb_param->blk_num, cb_param->is_earlycb,
+ cb_param->is_blk_last, cb_param->is_frm_last, cb_param->is_task_last);
+
+ pr_info("%s: CMDQ Info: task(%d/%d/%d) ofst(%zx/%zx/%zx/%zx/%zx).\n",
+ __func__, cb_param->task_id, cb_param->task_num, cb_param->task_cnt,
+ cb_param->pkt_ofst[0], cb_param->pkt_ofst[1], cb_param->pkt_ofst[2],
+ cb_param->pkt_ofst[3], cb_param->pkt_ofst[4]);
+
+ if (is_stream_off == 1)
+ pr_info("%s: [ERROR] cb(%p) pipe had been turned off(%d)!\n",
+ __func__, cb_param, is_stream_off);
+
+ pipe = (struct mtk_imgsys_pipe *)cb_param->frm_info->pipe;
+ if (!pipe->streaming) {
+ pr_info("%s: [ERROR] cb(%p): pipe already streamoff\n",
+ __func__, cb_param);
+ }
+
+ imgsys_cmdq_cmd_dump(cb_param->frm_info, frm_idx);
+ }
+
+ INIT_WORK(&cb_param->cmdq_cb_work, imgsys_cmdq_cb_work);
+ queue_work(imgsys_cmdq_wq, &cb_param->cmdq_cb_work);
+}
+
+static int imgsys_cmdq_parser(struct cmdq_pkt *pkt,
+ struct command *cmd, u32 cmd_num)
+{
+ bool stop = false;
+ int count = 0;
+ u32 event_id;
+
+ do {
+ switch (cmd->opcode) {
+ case IMGSYS_CMD_WRITE:
+ if (cmd->param.write.address < IMGSYS_REG_START ||
+ cmd->param.write.address > IMGSYS_REG_END) {
+ pr_info("%s: [ERROR] WRITE: wrong address(0x%08llx)!\n",
+ __func__, cmd->param.poll.address);
+ return -1;
+ }
+ pr_debug("%s: WRITE with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ __func__, cmd->param.write.address,
+ cmd->param.write.value,
+ cmd->param.write.mask);
+
+ cmdq_pkt_assign(pkt, CMDQ_THR_SPR_IDX0,
+ CMDQ_ADDR_HIGH(cmd->param.write.address));
+ if (cmd->param.write.mask != GENMASK(31, 0))
+ cmdq_pkt_write_s_mask_value(pkt, CMDQ_THR_SPR_IDX0,
+ CMDQ_ADDR_LOW(cmd->param.write.address),
+ cmd->param.write.value,
+ cmd->param.write.mask);
+ else
+ cmdq_pkt_write_s_value(pkt, CMDQ_THR_SPR_IDX0,
+ CMDQ_ADDR_LOW(cmd->param.write.address),
+ cmd->param.write.value);
+ break;
+ case IMGSYS_CMD_POLL:
+ if (cmd->param.poll.address < IMGSYS_REG_START ||
+ cmd->param.poll.address > IMGSYS_REG_END) {
+ pr_info("%s: [ERROR] POLL: wrong address(0x%08llx)!\n",
+ __func__, cmd->param.poll.address);
+ return -1;
+ }
+ pr_debug("%s: POLL with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ __func__, cmd->param.poll.address,
+ cmd->param.poll.value, cmd->param.poll.mask);
+
+ cmdq_pkt_poll_addr(pkt, cmd->param.poll.address,
+ cmd->param.poll.value,
+ cmd->param.poll.mask);
+ break;
+ case IMGSYS_CMD_WAIT:
+ event_id = cmd->param.wait.event;
+ if (event_id >= IMGSYS_EVENT_MAX) {
+ pr_info("%s: [ERROR] WAIT: wrong event(%d)!\n",
+ __func__, event_id);
+ return -1;
+ }
+ pr_debug("%s: WAIT event(%d/%d) action(%d)\n",
+ __func__, event_id,
+ imgsys_event[event_id].event,
+ cmd->param.wait.action);
+
+ if (cmd->param.wait.action == 1)
+ cmdq_pkt_wfe(pkt, imgsys_event[event_id].event, true);
+ else if (cmd->param.wait.action == 0)
+ cmdq_pkt_wfe(pkt, imgsys_event[event_id].event, false);
+
+ break;
+ case IMGSYS_CMD_UPDATE:
+ event_id = cmd->param.update.event;
+ if (event_id >= IMGSYS_EVENT_MAX) {
+ pr_info("%s: [ERROR] UPDATE: wrong event(%d)!\n",
+ __func__, event_id);
+ return -1;
+ }
+
+ pr_debug("%s: UPDATE event(%d/%d) action(%d)\n",
+ __func__, event_id,
+ imgsys_event[event_id].event,
+ cmd->param.update.action);
+
+ if (cmd->param.update.action == 1)
+ cmdq_pkt_set_event(pkt, imgsys_event[event_id].event);
+ else if (cmd->param.update.action == 0)
+ cmdq_pkt_clear_event(pkt, imgsys_event[event_id].event);
+
+ break;
+ case IMGSYS_CMD_ACQUIRE:
+ event_id = cmd->param.acquire.event;
+ if (event_id >= IMGSYS_EVENT_MAX) {
+ pr_info("%s: [ERROR] ACQUIRE: wrong event(%d)!\n",
+ __func__, event_id);
+ return -1;
+ }
+
+ pr_debug("%s: ACQUIRE event(%d/%d) action(%d)\n",
+ __func__, event_id,
+ imgsys_event[event_id].event,
+ cmd->param.acquire.action);
+
+ cmdq_pkt_acquire_event(pkt, imgsys_event[event_id].event);
+ break;
+ case IMGSYS_CMD_STOP:
+ pr_debug("%s: End Of Cmd!\n", __func__);
+ stop = true;
+ break;
+ default:
+ pr_info("%s: [ERROR] Not Support Cmd(%d)!\n", __func__, cmd->opcode);
+ return -1;
+ }
+ cmd++;
+ count++;
+ } while ((!stop) && (count < cmd_num));
+
+ return count;
+}
+
+int imgsys_cmdq_sendtask(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *frm_info,
+ void (*cmdq_cb)(struct imgsys_cmdq_cb_data data))
+{
+ struct cmdq_client *clt = NULL;
+ struct gce_recorder *cmd_buf = NULL;
+ struct command *cmd = NULL;
+ struct mtk_imgsys_cb_param *cb_param = NULL;
+ u32 cmd_num = 0;
+ u32 cmd_idx = 0;
+ u32 blk_idx = 0; /* For Vss block cnt */
+ u32 thd_idx = 0;
+ u32 hw_comb = 0;
+ int ret = 0, ret_flush = 0;
+ u32 frm_num = 0, frm_idx = 0;
+ bool is_pack = 0;
+ u32 task_idx = 0;
+ u32 task_id = 0;
+ u32 task_num = 0;
+ u32 task_cnt = 0;
+ size_t pkt_ofst[MAX_FRAME_IN_TASK] = {0};
+
+ frm_num = frm_info->total_frmnum;
+ frm_info->cb_frmcnt = 0;
+ frm_info->total_taskcnt = 0;
+
+ for (frm_idx = 0; frm_idx < frm_num; frm_idx++) {
+ cmd_buf = (struct gce_recorder *)frm_info->user_info[frm_idx].g_swbuf;
+
+ /* Validate the GCE recorder structure */
+ if (cmd_buf->header_code != HEADER_CODE ||
+ cmd_buf->check_pre != CHECK_PRE ||
+ cmd_buf->check_post != CHECK_POST ||
+ cmd_buf->footer_code != FOOTER_CODE) {
+ pr_info("%s: Incorrect guard word: %08x/%08x/%08x/%08x",
+ __func__, cmd_buf->header_code, cmd_buf->check_pre,
+ cmd_buf->check_post, cmd_buf->footer_code);
+ return -1;
+ }
+
+ cmd_num = cmd_buf->curr_length / sizeof(struct command);
+ cmd = (struct command *)((char *)frm_info->user_info[frm_idx].g_swbuf
+ + (unsigned long)cmd_buf->cmd_offset);
+
+ hw_comb = frm_info->user_info[frm_idx].hw_comb;
+
+ if (is_pack == 0) {
+ if (frm_info->group_id == -1) {
+ /* Choose cmdq_client base on hw scenario */
+ for (thd_idx = 0; thd_idx < IMGSYS_ENG_MAX; thd_idx++) {
+ if (hw_comb & 0x1) {
+ clt = imgsys_clt[thd_idx];
+ pr_debug("%s: mbox thread (%d, %p)\n",
+ __func__, thd_idx, clt);
+ pr_debug("%s: frm(%d/%d)\n",
+ __func__, frm_idx, frm_num);
+ break;
+ }
+ hw_comb = hw_comb >> 1;
+ }
+ /* This segment can be removed since user had set dependency */
+ if (frm_info->user_info[frm_idx].hw_comb & IMGSYS_ENG_DIP) {
+ thd_idx = 4;
+ clt = imgsys_clt[thd_idx];
+ }
+ } else {
+ if (frm_info->group_id >= 0 &&
+ frm_info->group_id < IMGSYS_NOR_THD) {
+ thd_idx = frm_info->group_id;
+ clt = imgsys_clt[thd_idx];
+ } else {
+ pr_info("%s: [ERROR] Group id(%d) is not in range(%d).\n",
+ __func__, frm_info->group_id, IMGSYS_NOR_THD);
+ pr_info("%s: [ERROR] Frame Info: frm(%d/%d) hw_comb(%d).\n",
+ __func__, frm_idx, frm_num,
+ frm_info->user_info[frm_idx].hw_comb);
+ return -1;
+ }
+ }
+
+ /* This is work around for low latency flow. */
+ /* If we change to request base, */
+ /* we don't have to take this condition into account. */
+ if (frm_info->sync_id != -1) {
+ thd_idx = 0;
+ clt = imgsys_clt[thd_idx];
+ }
+
+ if (!clt) {
+ pr_info("%s: [ERROR] No HW Found (0x%x) for frm(%d/%d)!\n",
+ __func__, frm_info->user_info[frm_idx].hw_comb,
+ frm_idx, frm_num);
+ return -1;
+ }
+ }
+
+ cmd_idx = 0;
+
+ for (blk_idx = 0; blk_idx < cmd_buf->frame_block; blk_idx++) {
+ if (is_pack == 0) {
+ /* Prepare cb param */
+ cb_param = vzalloc(sizeof(*cb_param));
+ if (!cb_param)
+ return -1;
+
+ /* create pkt and hook clt as pkt's private data */
+ if (cmdq_pkt_create(clt, &cb_param->pkt, CMDQ_BUF_SIZE)) {
+ vfree(cb_param);
+ pr_info("%s: [ERROR] Id %d imgsys_create_pkt fail!\n",
+ __func__, blk_idx);
+ return -1;
+ }
+ }
+
+ ret = imgsys_cmdq_parser(&cb_param->pkt, &cmd[cmd_idx], cmd_num);
+
+ if (ret < 0) {
+ pr_info("%s: [ERROR] Parsing fail. cmd(%d/%d) frm(%d/%d/%d)\n",
+ __func__, cmd_idx, cmd[cmd_idx].opcode,
+ blk_idx, frm_idx, frm_num);
+
+ cmdq_pkt_destroy(clt, &cb_param->pkt);
+ vfree(cb_param);
+ goto sendtask_done;
+ }
+ cmd_idx += ret;
+
+ /* Check for packing gce task */
+ pkt_ofst[task_cnt] = cb_param->pkt.cmd_buf_size - CMDQ_INST_SIZE;
+ task_cnt++;
+ if (frm_info->user_info[frm_idx].is_time_shared ||
+ frm_info->user_info[frm_idx].is_earlycb ||
+ frm_idx + 1 == frm_num) {
+ task_num++;
+ cb_param->frm_info = frm_info;
+ cb_param->frm_idx = frm_idx;
+ cb_param->frm_num = frm_num;
+ cb_param->user_cmdq_cb = cmdq_cb;
+ if ((blk_idx + 1) == cmd_buf->frame_block)
+ cb_param->is_blk_last = 1;
+ else
+ cb_param->is_blk_last = 0;
+ if ((frm_idx + 1) == frm_num)
+ cb_param->is_frm_last = 1;
+ else
+ cb_param->is_frm_last = 0;
+ cb_param->blk_idx = blk_idx;
+ cb_param->blk_num = cmd_buf->frame_block;
+ cb_param->is_earlycb = frm_info->user_info[frm_idx].is_earlycb;
+ cb_param->group_id = frm_info->group_id;
+ cb_param->imgsys_dev = imgsys_dev;
+ cb_param->thd_idx = thd_idx;
+ cb_param->clt = clt;
+ cb_param->task_cnt = task_cnt;
+ for (task_idx = 0; task_idx < task_cnt; task_idx++)
+ cb_param->pkt_ofst[task_idx] = pkt_ofst[task_idx];
+ task_cnt = 0;
+ cb_param->task_id = task_id;
+ task_id++;
+ if (cb_param->is_blk_last && cb_param->is_frm_last) {
+ cb_param->is_task_last = 1;
+ cb_param->task_num = task_num;
+ frm_info->total_taskcnt = task_num;
+ } else {
+ cb_param->is_task_last = 0;
+ cb_param->task_num = 0;
+ }
+
+ /* flush synchronized, block API */
+ cmdq_pkt_eoc(&cb_param->pkt);
+ cmdq_pkt_jump_rel(&cb_param->pkt, CMDQ_INST_SIZE,
+ cmdq_get_shift_pa(clt->chan));
+ dma_sync_single_for_device(clt->chan->mbox->dev,
+ cb_param->pkt.pa_base,
+ cb_param->pkt.cmd_buf_size,
+ DMA_TO_DEVICE);
+
+ mbox_send_message(clt->chan, &cb_param->pkt);
+ mbox_client_txdone(clt->chan, 0);
+ if (ret_flush < 0)
+ pr_info("%s: [ERROR] Frame(%d/%d) flushing fail(%d).\n",
+ __func__, ret_flush, frm_idx, frm_num);
+ is_pack = 0;
+ } else {
+ dev_info(imgsys_dev->dev, "%s:is_pack = 1", __func__);
+ is_pack = 1;
+ }
+ }
+ }
+
+sendtask_done:
+ return 0;
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
new file mode 100644
index 000000000000..47bbd6735d0b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+#ifndef _MTK_IMGSYS_CMDQ_H_
+#define _MTK_IMGSYS_CMDQ_H_
+
+#include <linux/platform_device.h>
+#include <linux/soc/mediatek/mtk-cmdq.h>
+#include "mtk_imgsys-dev.h"
+
+/* Structure to hold command queue callback data */
+struct imgsys_cmdq_cb_data {
+ struct cmdq_cb_data cmdq_data; /* Command queue data */
+ void *data; /* Additional data */
+};
+
+void imgsys_cmdq_streamon(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_cmdq_streamoff(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_cmdq_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_cmdq_release(struct mtk_imgsys_dev *imgsys_dev);
+int imgsys_cmdq_sendtask(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *frm_info,
+ void (*cmdq_cb)(struct imgsys_cmdq_cb_data data));
+
+#endif /* _MTK_IMGSYS_CMDQ_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
index dd02a8fa28d4..3918e360a5a0 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
@@ -39,6 +39,26 @@
#define CQ_THRX_CTL_MODE BIT(4)
#define CQ_THRX_CTL (CQ_THRX_CTL_EN | CQ_THRX_CTL_MODE)
+/**
+ * enum mtk_imgsys_engine_bitmask - Bitmasks for supported hardware engines
+ *
+ * This enumeration defines the bitmasks for the supported hardware engines in
+ * the image system.
+ */
+enum mtk_imgsys_engine_bitmask {
+ IMGSYS_ENG_WPE_EIS = 0x00000001, /* Bitmask for WPE EIS engine */
+ IMGSYS_ENG_WPE_TNR = 0x00000002, /* Bitmask for WPE TNR engine */
+ IMGSYS_ENG_WPE_LITE = 0x00000004, /* Bitmask for WPE Lite engine */
+ IMGSYS_ENG_TRAW = 0x00000020, /* Bitmask for TRAW engine */
+ IMGSYS_ENG_LTR = 0x00000040, /* Bitmask for LTR engine */
+ IMGSYS_ENG_XTR = 0x00000080, /* Bitmask for XTR engine */
+ IMGSYS_ENG_DIP = 0x00000100, /* Bitmask for DIP engine */
+ IMGSYS_ENG_PQDIP_A = 0x00000200, /* Bitmask for PQDIP A engine */
+ IMGSYS_ENG_PQDIP_B = 0x00000400, /* Bitmask for PQDIP B engine */
+ IMGSYS_ENG_ME = 0x00000800, /* Bitmask for ME engine */
+ IMGSYS_ENG_MAX_NUM = 0x00001000 /* Maximum bitmask value */
+};
+
/**
* enum mtk_imgsys_reg_id
*
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 9f118491b37d..184f0e3a4d3b 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -9,6 +9,7 @@
#include <linux/pm_runtime.h>
#include "mtk_imgsys-sys.h"
#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-cmdq.h"
static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
{
@@ -26,6 +27,8 @@ static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
kref_init(&imgsys_dev->init_kref);
+ /* calling cmdq stream on */
+ imgsys_cmdq_streamon(imgsys_dev);
return 0;
}
@@ -43,6 +46,8 @@ static void module_uninit(struct kref *kref)
static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
{
+ /* calling cmdq stream off */
+ imgsys_cmdq_streamoff(imgsys_dev);
kref_put(&imgsys_dev->init_kref, module_uninit);
}
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 08/10] media: mediatek: isp: Add SCP support for ImgSys driver
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (6 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver Olivia Wen
@ 2025-05-24 11:50 ` Olivia Wen
2025-05-24 11:50 ` [PATCH v1 09/10] media: mediatek: isp: Add image processing flow Olivia Wen
2025-05-24 11:50 ` [PATCH v1 10/10] media: mediatek: isp: Add normal data dump flow Olivia Wen
9 siblings, 0 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the SCP support for ImgSys driver. It covers the memory
allocation and transmission interfaces required by SCP. Frame-related
information is transmitted to SCP, where it is packaged into commands
and the necessary hardware information. These commands are then sent to
the kernel for further processing.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 1 +
.../mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h | 143 +++++++
.../isp/isp_7x/imgsys/mtk_imgsys-scp.c | 349 ++++++++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-scp.h | 194 ++++++++++
4 files changed, 687 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index c0f0f3a0c38a..29c18c773fdc 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -3,6 +3,7 @@
mtk_imgsys_util-objs := \
mtk_imgsys-cmdq.o \
+mtk_imgsys-scp.o \
mtk_imgsys-module_main.o \
mtk_imgsys-of.o \
mtk_imgsys-formats.o \
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
new file mode 100644
index 000000000000..850f6ec99a96
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef __MTK_IMG_IPI_H__
+#define __MTK_IMG_IPI_H__
+
+#include <linux/types.h>
+#include "mtk_header_desc.h"
+
+#define HEADER_VER 19481
+#define IMG_MAX_HW_DMAS 72
+#define IMG_MAX_HW 12
+#define WPE_ENG_NUM 3
+#define TMAX 16
+
+/**
+ * enum img_module_id - Enumeration of image processing module IDs
+ * @IMG_MODULE_WPE: ID for the WPE (Warp Engine) module
+ * @IMG_MODULE_TRAW: ID for the TRAW (Tile Raw) module
+ * @IMG_MODULE_DIP: ID for the DIP (Digital Image Processing) module
+ * @IMG_MODULE_PQDIP: ID for the PQDIP (Picture Quality Digital Image Processing) module
+ * @IMG_MODULE_MAX: Maximum ID value, used for boundary checking
+ */
+enum img_module_id {
+ IMG_MODULE_WPE = 0,
+ IMG_MODULE_TRAW = 1,
+ IMG_MODULE_DIP = 2,
+ IMG_MODULE_PQDIP = 3,
+ IMG_MODULE_MAX = 4,
+};
+
+/**
+ * struct module_init_info - Information about module initialization
+ * @c_wbuf: Command queue (CQ) working buffer address
+ * @c_wbuf_dma: DMA address of the CQ working buffer
+ * @c_wbuf_sz: Size of the CQ working buffer
+ * @t_wbuf: Tile dirvier (TDR) working buffer address
+ * @t_wbuf_dma: DMA address of the TDR working buffer
+ * @t_wbuf_sz: Size of the TDR working buffer
+ */
+struct module_init_info {
+ u64 c_wbuf; /* CQ working buffer address */
+ u64 c_wbuf_dma; /* DMA address of the CQ working buffer */
+ u32 c_wbuf_sz; /* Size of the CQ working buffer */
+ u64 t_wbuf; /* TDR working buffer address */
+ u64 t_wbuf_dma; /* DMA address of the TDR working buffer */
+ u32 t_wbuf_sz; /* Size of the TDR working buffer */
+} __packed;
+
+/**
+ * struct img_init_info - Information about image initialization
+ * @header_version: Version of the header
+ * @isp_version: ISP version
+ * @frameparam_size: Size of the frame parameters
+ * @module_info: Array of module working buffer information
+ * @g_wbuf: Global Command Engine (GCE) working buffer address
+ * @g_wbuf_sz: Size of the GCE working buffer
+ *
+ * This structure is used as the parameter for IPI_IMGSYS_INIT_ID and
+ * IPI_IMGSYS_DEINIT_ID. It contains information necessary for initializing and
+ * deinitializing the image system.
+ */
+struct img_init_info {
+ u32 header_version;
+ u32 isp_version;
+ u32 frameparam_size;
+ struct module_init_info module_info[IMG_MODULE_MAX];
+ u64 g_wbuf;
+ u32 g_wbuf_sz;
+} __packed;
+
+/**
+ * struct img_swfrm_info - Software frame information structure for image system
+ * @hw_comb: Hardware combination, which used to store the combination of
+ * enum mtk_imgsys_engine_bitmask
+ * @sw_ridx: Software read index, provided to SCP upon notification to access
+ * GCE buffer location
+ * @is_time_shared: Indicates if the frame is time-shared
+ * @is_earlycb: Indicates if early callback is enabled
+ * @sw_goft: offset of GCE buffer
+ * @g_swbuf: virtual address of GCE buffer
+ *
+ * These fields are primarily set by SCP (System Control Processor) and used by
+ * the kernel for processing and verifing data.
+ */
+struct img_swfrm_info {
+ u32 hw_comb;
+ int sw_ridx;
+ u8 is_time_shared;
+ u8 is_earlycb;
+ u64 sw_goft;
+ void *g_swbuf;
+} __packed;
+
+/**
+ * struct img_ipi_frameparam - Frame parameters for IMGSYS IPI
+ * @dmas_enable: Array indicating enabled DMAs for each hardware unit
+ * @dmas: Array of DMA descriptors for each hardware unit
+ * @tuning_meta: Descriptor for tuning metadata
+ * @ctrl_meta: Descriptor for control metadata
+ *
+ * This structure defines the frame parameters used in the IMGSYS IPI
+ * (Inter-Processor Interface). It includes arrays to indicate enabled DMAs and
+ * their descriptors, as well as descriptors for tuning and control metadata.
+ */
+struct img_ipi_frameparam {
+ u8 dmas_enable[IMG_MAX_HW_DMAS];
+ struct header_desc_norm dmas[IMG_MAX_HW_DMAS];
+ struct header_desc_norm tuning_meta;
+ struct header_desc_norm ctrl_meta;
+};
+
+/**
+ * struct img_sw_buffer - Software buffer address information
+ * @scp_addr: Physical address accessed by SCP
+ * @offset: Address offset
+ *
+ * This structure defines the address information of a software buffer used in
+ * the SCP.
+ */
+struct img_sw_buffer {
+ u32 scp_addr;
+ u32 offset;
+} __packed;
+
+/**
+ * struct img_ipi_param - Parameters for IMGSYS IPI
+ * @frm_param: Software buffer adreess for frame parameters
+ * @req_addr_va: Virtual address for the request
+ *
+ * This structure defines the parameters used in the IMGSYS IPI, including a
+ * software buffer physical address for frame parameters and a virtual address
+ * for the request.
+ */
+struct img_ipi_param {
+ struct img_sw_buffer frm_param;
+ u64 req_addr_va;
+} __packed;
+
+#endif /* __MTK_IMG_IPI_H__ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
new file mode 100644
index 000000000000..4e6126d9f28c
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
@@ -0,0 +1,349 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+#include <linux/slab.h>
+#include <linux/remoteproc.h>
+#include <linux/dma-direction.h>
+#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-scp.h"
+
+#define SCP_TIMEOUT_MS 4000U
+
+/**
+ * struct mtk_scp_reserve_memblock - Represents a memory block reserved by the
+ * kernel for usage by the SCP (System Control Processor).
+ *
+ * @id: Unique identifier for the memory block. This ID is used to
+ * reference and manage the memory block within the system.
+ * @start_phys: Physical address the starting point of the buffer.
+ * @start_virt: Kernel virtual address of the starting point of the buffer.
+ * @start_dma: DMA (Direct Memory Access) address of the starting point of the
+ * buffer.
+ * @size: Size of the allocated buffer, in bytes.
+ * @dma_dir: Direction of DMA data transfer, indicating whether the data is
+ * being transferred to the device, from the device, or
+ * bidirectionally.
+ */
+struct mtk_scp_reserve_memblock {
+ unsigned int id;
+ phys_addr_t start_phys;
+ void *start_virt;
+ phys_addr_t start_dma;
+ unsigned int size;
+ enum dma_data_direction dma_dir;
+};
+
+/* Array of reserved memory blocks for various hardware components */
+static struct mtk_scp_reserve_memblock reserve_memblock[] = {
+ {
+ /*WPE CQ buffer for WPE HW usage*/
+ .id = WPE_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0xE1000, /*900KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*WPE TDR buffer for tile driver usage*/
+ .id = WPE_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x196000, /*1MB + 600KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*TRAW CQ buffer for TRAW driver usage*/
+ .id = TRAW_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x4C8000, /*4MB + 800KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*TRAW TDR buffer for tile driver usage*/
+ .id = TRAW_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x1AC8000, /*26MB + 800KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*DIP CQ buffer for TRAW HW usage*/
+ .id = DIP_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x5C8000, /*5MB + 800KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*DIP TDR buffer for tile driver usage*/
+ .id = DIP_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x1FAF000, /*31MB + 700KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*PQDIP CQ buffer for TRAW HW usage*/
+ .id = PQDIP_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x100000, /*1MB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*PQDIP TDR buffer for tile driver usage*/
+ .id = PQDIP_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x170000, /*1MB + 500KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*GCE command buffer for GCE HW usage*/
+ .id = IMG_MEM_G_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x1800000,
+ .dma_dir = DMA_BIDIRECTIONAL,
+ },
+};
+
+static phys_addr_t imgsys_scp_get_reserve_mem_phys(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT) {
+ pr_err("[SCP] no reserve memory for %d", id);
+ return 0;
+ }
+ return reserve_memblock[id].start_phys;
+}
+
+static phys_addr_t imgsys_scp_get_reserve_mem_dma(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT) {
+ pr_err("[SCP] no reserve memory for %d", id);
+ return 0;
+ }
+ return reserve_memblock[id].start_dma;
+}
+
+void *imgsys_scp_get_reserve_mem_virt(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT)
+ return NULL;
+ else
+ return reserve_memblock[id].start_virt;
+}
+EXPORT_SYMBOL(imgsys_scp_get_reserve_mem_virt);
+
+unsigned int imgsys_scp_get_reserve_mem_size(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT) {
+ pr_err("[SCP] no reserve memory for %d", id);
+ return 0;
+ }
+ return reserve_memblock[id].size;
+}
+
+int imgsys_scp_alloc_reserve_mem(struct mtk_imgsys_dev *imgsys_dev)
+{
+ enum img_mem_id id;
+ unsigned int block_num;
+ void *ptr;
+ dma_addr_t addr;
+
+ block_num = ARRAY_SIZE(reserve_memblock);
+ for (id = 0; id < block_num; id++) {
+ ptr = dma_alloc_coherent(imgsys_dev->smem_dev,
+ reserve_memblock[id].size,
+ &addr, GFP_KERNEL);
+ if (!ptr)
+ return -ENOMEM;
+
+ reserve_memblock[id].start_virt = ptr;
+ reserve_memblock[id].start_phys = addr;
+ reserve_memblock[id].start_dma =
+ dma_map_resource(imgsys_dev->dev, addr,
+ reserve_memblock[id].size,
+ reserve_memblock[id].dma_dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(imgsys_dev->dev,
+ reserve_memblock[id].start_dma)) {
+ dev_err(imgsys_dev->dev, "failed to map scp iova\n");
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+int imgsys_scp_free_reserve_mem(struct mtk_imgsys_dev *imgsys_dev)
+{
+ enum img_mem_id id;
+
+ /* release reserved memory */
+ for (id = 0; id < IMG_MEM_ID_COUNT; id++) {
+ dma_unmap_resource(imgsys_dev->dev,
+ reserve_memblock[id].start_dma,
+ reserve_memblock[id].size,
+ reserve_memblock[id].dma_dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ dma_free_coherent(imgsys_dev->smem_dev,
+ reserve_memblock[id].size,
+ reserve_memblock[id].start_virt,
+ reserve_memblock[id].start_phys);
+
+ reserve_memblock[id].start_dma = 0x0;
+ reserve_memblock[id].start_virt = NULL;
+ reserve_memblock[id].start_phys = 0x0;
+ }
+
+ return 0;
+}
+
+int imgsys_scp_get_reserve_mem_info(struct img_init_info *info)
+{
+ if (!info) {
+ pr_err("%s:NULL info\n", __func__);
+ return -EINVAL;
+ }
+
+ /* WPE */
+ info->module_info[IMG_MODULE_WPE].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(WPE_MEM_C_ID);
+ info->module_info[IMG_MODULE_WPE].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(WPE_MEM_C_ID);
+ info->module_info[IMG_MODULE_WPE].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(WPE_MEM_C_ID);
+ info->module_info[IMG_MODULE_WPE].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(WPE_MEM_T_ID);
+ info->module_info[IMG_MODULE_WPE].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(WPE_MEM_T_ID);
+ info->module_info[IMG_MODULE_WPE].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(WPE_MEM_T_ID);
+
+ /* TRAW */
+ info->module_info[IMG_MODULE_TRAW].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(TRAW_MEM_C_ID);
+ info->module_info[IMG_MODULE_TRAW].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(TRAW_MEM_C_ID);
+ info->module_info[IMG_MODULE_TRAW].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(TRAW_MEM_C_ID);
+ info->module_info[IMG_MODULE_TRAW].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(TRAW_MEM_T_ID);
+ info->module_info[IMG_MODULE_TRAW].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(TRAW_MEM_T_ID);
+ info->module_info[IMG_MODULE_TRAW].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(TRAW_MEM_T_ID);
+
+ /* DIP */
+ info->module_info[IMG_MODULE_DIP].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(DIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_DIP].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(DIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_DIP].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(DIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_DIP].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(DIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_DIP].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(DIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_DIP].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(DIP_MEM_T_ID);
+
+ /* PQDIP */
+ info->module_info[IMG_MODULE_PQDIP].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(PQDIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_PQDIP].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(PQDIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_PQDIP].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(PQDIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_PQDIP].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(PQDIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_PQDIP].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(PQDIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_PQDIP].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(PQDIP_MEM_T_ID);
+
+ /* GCE command */
+ info->g_wbuf = imgsys_scp_get_reserve_mem_phys(IMG_MEM_G_ID);
+ info->g_wbuf_sz = imgsys_scp_get_reserve_mem_size(IMG_MEM_G_ID);
+
+ return 0;
+}
+
+int imgsys_scp_init(struct mtk_imgsys_dev *imgsys_dev,
+ scp_ipi_handler_t imgsys_scp_handler)
+{
+ int ret = 0;
+
+ ret = rproc_boot(imgsys_dev->rproc_handle);
+ if (ret) {
+ dev_err(imgsys_dev->dev,
+ "%s failed to rproc_boot\n", __func__);
+ return ret;
+ }
+
+ ret = scp_ipi_register(imgsys_dev->scp, SCP_IPI_IMGSYS_CMD,
+ imgsys_scp_handler, imgsys_dev);
+ if (ret) {
+ dev_err(imgsys_dev->dev,
+ "%s failed to register IPI cmd\n", __func__);
+ return ret;
+ }
+
+ return ret;
+}
+
+void imgsys_scp_deinit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ rproc_shutdown(imgsys_dev->rproc_handle);
+}
+
+int imgsys_scp_send(struct mtk_imgsys_dev *imgsys_dev,
+ enum imgsys_ipi_id id, void *buf,
+ unsigned int len, int req_fd,
+ unsigned int wait)
+{
+ struct share_buf send_obj = { 0 };
+ int ret = 0;
+
+ dev_dbg(imgsys_dev->dev, "%s id:%d len %d\n",
+ __func__, id, len);
+
+ if (id < IPI_IMGSYS_INIT_ID || id >= IPI_MAX_ID ||
+ len > sizeof(send_obj.share_data) || !buf) {
+ dev_info(imgsys_dev->dev,
+ "%s failed to send scp message (len/sz(%d/%zu)\n",
+ __func__, len, sizeof(send_obj.share_data));
+ return -EINVAL;
+ }
+
+ send_obj.len = len;
+ send_obj.id = id;
+ memcpy((void *)send_obj.share_data, buf, len);
+
+ send_obj.info.send.ipi = id;
+ send_obj.info.send.req = req_fd;
+ send_obj.info.send.ack = (wait ? 1 : 0);
+ send_obj.info.send.seq = 0;
+
+ ret = scp_ipi_send(imgsys_dev->scp, SCP_IPI_IMGSYS_CMD,
+ (void *)&send_obj, sizeof(send_obj),
+ wait ? msecs_to_jiffies(SCP_TIMEOUT_MS) : 0);
+ if (ret)
+ dev_err(imgsys_dev->dev,
+ "%s: send SCP(%d) failed %d\n", __func__, id, ret);
+
+ return 0;
+}
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
new file mode 100644
index 000000000000..2082c4bdf96f
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
@@ -0,0 +1,194 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef MTK_IMGSYS_SCP_H
+#define MTK_IMGSYS_SCP_H
+
+#include <linux/remoteproc/mtk_scp.h>
+#include "mtk-img-ipi.h"
+#include "mtk_imgsys-dev.h"
+
+/* the size of share buffer between the kernel and SCP */
+#define SCP_SHARE_BUF_SIZE 576
+
+/**
+ * struct object_info - Represents information associated with a object,
+ * particularly in the context of inter-processor communication and media
+ * processing.
+ *
+ * @ipi: Specifies the IPI (Inter-Processor Interrupt) ID, using 5 bits.
+ * @ack: Indicates whether an acknowledgment is required, using 1 bit.
+ * @req: Holds the media request ID, using 10 bits.
+ * @seq: A sequence number, using 16 bits, for tracking the order of
+ * objects or messages.
+ * @cmd: Combined command representation.
+ */
+struct object_info {
+ union {
+ struct send {
+ u32 ipi: 5; /* IPI ID */
+ u32 ack: 1; /* Acknowledgment flag */
+ u32 req: 10; /* Media request ID */
+ u32 seq: 16; /* Sequence number */
+ } __packed send;
+ u32 cmd; /* Combined command representation */
+ };
+} __packed;
+
+/**
+ * struct share_buf - DTCM (Data Tightly-Coupled Memory) buffer shared with
+ * AP and hardware components of IMGSYS
+ *
+ * @id: IPI (Inter-Processor Interrupt) ID
+ * @len: Length of the shared buffer
+ * @share_data: Data contained in the shared buffer
+ * @info: Information associated with the object
+ *
+ * This structure represents a DTCM buffer that is shared between the AP and
+ * the hardware components of the MediaTek Image System (IMGSYS). It includes
+ * the IPI ID, the length of the shared buffer, the actual buffer data, and
+ * additional object information.
+ */
+struct share_buf {
+ u32 id; /* IPI (Inter-Processor Interrupt) ID */
+ u32 len; /* Length of the shared buffer */
+ u8 share_data[SCP_SHARE_BUF_SIZE]; /* Data contained in the shared buffer */
+ struct object_info info; /* Information object */
+};
+
+/**
+ * enum img_mem_id - Enumeration of image memory IDs
+ *
+ * @WPE_MEM_C_ID: Warping Engine (WPE) module command queue (CQ) buffer
+ * @WPE_MEM_T_ID: WPE module tile driver (TDR) buffer
+ * @TRAW_MEM_C_ID: Tile-Raw (TRAW) module CQ buffer
+ * @TRAW_MEM_T_ID: TRAW module TDR buffer
+ * @DIP_MEM_C_ID: Digital Image Processing (DIP) module CQ buffer
+ * @DIP_MEM_T_ID: DIP module TDR buffer
+ * @PQDIP_MEM_C_ID: Picture Quality image processing (PQDIP) CQ buffer
+ * @PQDIP_MEM_T_ID: PQDIP TDR buffer
+ * @IMG_MEM_G_ID: GCE command buffer
+ * @IMG_MEM_ID_COUNT: Total number of memory IDs
+ *
+ * This enumeration defines the various memory IDs used in the MediaTek
+ * image system. Each memory ID corresponds to a specific buffer used
+ * by different modules within the image system.
+ */
+enum img_mem_id {
+ WPE_MEM_C_ID, /* WPE module CQ buffer */
+ WPE_MEM_T_ID, /* WPE module TDR buffer */
+ TRAW_MEM_C_ID, /* TRAW module CQ buffer */
+ TRAW_MEM_T_ID, /* TRAW module TDR buffer */
+ DIP_MEM_C_ID, /* DIP module CQ buffer */
+ DIP_MEM_T_ID, /* DIP module TDR buffer */
+ PQDIP_MEM_C_ID, /* PQDIP CQ buffer */
+ PQDIP_MEM_T_ID, /* PQDIP TDR buffer */
+ IMG_MEM_G_ID, /* GCE command buffer */
+ IMG_MEM_ID_COUNT, /* Total number of memory IDs */
+};
+
+/**
+ * enum imgsys_ipi_id - Enumeration of IPI (Inter-Processor Interrupt) IDs
+ * @IPI_IMGSYS_INIT_ID: ID for initializing the image system
+ * @IPI_IMGSYS_FRAME_ID: ID for processing a frame
+ * @IPI_IMGSYS_DEQUE_DONE_ID: ID for indicating frame dequeuing is done
+ * @IPI_IMGSYS_DEINIT_ID: ID for deinitializing the image system
+ * @IPI_MAX_ID: Maximum ID value, used for boundary checking
+ */
+enum imgsys_ipi_id {
+ IPI_IMGSYS_INIT_ID = 0, /* ID for initializing the image system */
+ IPI_IMGSYS_FRAME_ID, /* ID for processing a frame */
+ IPI_IMGSYS_DEQUE_DONE_ID, /* ID for dequeuing frame is done */
+ IPI_IMGSYS_DEINIT_ID, /* ID for deinitializing the image system */
+ IPI_MAX_ID, /* Maximum ID value*/
+};
+
+/**
+ * imgsys_scp_alloc_reserve_mem - allocate reserved memory for use by the
+ * kernel and the SCP (System Control Processor)
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ *
+ **/
+int imgsys_scp_alloc_reserve_mem(struct mtk_imgsys_dev *imgsys_dev);
+
+/**
+ * imgsys_scp_free_reserve_mem - free reserved memory for use by the kernel and
+ * SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ *
+ **/
+int imgsys_scp_free_reserve_mem(struct mtk_imgsys_dev *imgsys_dev);
+
+/**
+ * imgsys_scp_get_reserve_mem_info - get the information about reserved memory
+ *
+ * @info: A pointer to an already allocated structure of type img_init_info,
+ * where this function will populate the reserved memory details.
+ *
+ **/
+int imgsys_scp_get_reserve_mem_info(struct img_init_info *info);
+
+/**
+ * imgsys_scp_get_reserve_mem_virt - get the virtual address of reserved memory
+ * by a particular memory ID
+ *
+ * @id: The identifier of the reserved memory block
+ *
+ **/
+void *imgsys_scp_get_reserve_mem_virt(enum img_mem_id id);
+
+/**
+ * imgsys_scp_get_reserve_mem_size - get the size of reserved memory by a
+ * particular memory ID
+ *
+ * @id: The identifier of the reserved memory block
+ *
+ **/
+unsigned int imgsys_scp_get_reserve_mem_size(enum img_mem_id id);
+
+/**
+ * imgsys_scp_init - start the SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ * @imgsys_scp_handler: Handler for SCP IPI
+ **/
+int imgsys_scp_init(struct mtk_imgsys_dev *imgsys_dev,
+ scp_ipi_handler_t imgsys_scp_handler);
+
+/**
+ * imgsys_scp_deinit - stop the SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ *
+ **/
+void imgsys_scp_deinit(struct mtk_imgsys_dev *imgsys_dev);
+
+/**
+ * imgsys_scp_send - send data from AP to SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ * @id: IPI ID
+ * @buf:the data buffer
+ * @len: the data buffer length
+ * @req_fd: the file descriptor of media request
+ * @wait: wait until ipi done or not
+ *
+ * This function is thread-safe. When this function returns,
+ * SCP has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by scp_ipi_register will be called in interrupt context.
+ *
+ **/
+int imgsys_scp_send(struct mtk_imgsys_dev *imgsys_dev,
+ enum imgsys_ipi_id id,
+ void *buf,
+ unsigned int len,
+ int req_fd,
+ unsigned int wait);
+
+#endif /* MTK_IMGSYS_SCP_H */
+
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 09/10] media: mediatek: isp: Add image processing flow
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (7 preceding siblings ...)
2025-05-24 11:50 ` [PATCH v1 08/10] media: mediatek: isp: Add SCP " Olivia Wen
@ 2025-05-24 11:50 ` Olivia Wen
2025-05-24 11:50 ` [PATCH v1 10/10] media: mediatek: isp: Add normal data dump flow Olivia Wen
9 siblings, 0 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the image processing flow for the MediaTek ImgSys driver.
It is a crucial part of the ImgSys architecture, controlling the flow of
operations from the user applications to the hardware. The ImgSys driver
manages various components (e.g. SCP, CMDQ) and processes to ensure
efficient image processing.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 2 +
| 101 ++++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.c | 61 +++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 139 +++++-
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.c | 82 +++
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.h | 37 ++
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 468 +++++++++++++++++-
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 128 ++++-
.../isp/isp_7x/imgsys/mtk_imgsys-work_buf.c | 113 +++++
.../isp/isp_7x/imgsys/mtk_imgsys-work_buf.h | 55 ++
10 files changed, 1169 insertions(+), 17 deletions(-)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 29c18c773fdc..3aa53e8bfc6d 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,6 +2,8 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-work_buf.o \
+mtk_imgsys-gce_work.o \
mtk_imgsys-cmdq.o \
mtk_imgsys-scp.o \
mtk_imgsys-module_main.o \
--git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
index 50c2e3c3395b..673e84aad362 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
@@ -9,6 +9,39 @@
#include <linux/videodev2.h>
+#define IMGBUF_MAX_PLANES 3 /* Maximum number of planes in an image buffer */
+#define IMG_MAX_HW_DMAS 72 /* Maximum number of hardware DMAs */
+
+/**
+ * struct v4l2_ext_plane - Extended plane information
+ * @offset: Offset within the DMA buffer
+ * @phyaddr: Physical address of the DMA buffer
+ * @isp_addr: ISP (Image Signal Processor) address
+ * @size: Size of the plane
+ */
+struct v4l2_ext_plane {
+ union {
+ struct {
+ __u32 offset; /* Offset within the DMA buffer */
+ __u64 phyaddr; /* Physical address of the DMA buffer */
+ } dma_buf;
+ } m;
+ __u64 isp_addr; /* ISP (Image Signal Processor) address */
+ __u64 size; /* Size of the plane */
+};
+
+/**
+ * struct v4l2_ext_buffer - Extended V4L2 buffer
+ * @planes: Array of extended plane information
+ * @num_planes: Number of planes
+ * @reserved: Reserved for future use
+ */
+struct v4l2_ext_buffer {
+ struct v4l2_ext_plane planes[IMGBUF_MAX_PLANES];
+ __u32 num_planes;
+ __u64 reserved[2];
+};
+
/**
* struct mtk_imgsys_crop - Crop parameters for MediaTek Image System
* @c: Rectangle defining the crop area
@@ -25,4 +58,72 @@ struct mtk_imgsys_crop {
struct v4l2_fract height_subpix; /* Sub-pixel adjustment for the height */
};
+/**
+ * struct plane_pix_format - Pixel format for a single plane
+ * @sizeimage: Size of the image in bytes
+ * @bytesperline: Number of bytes per line
+ */
+struct plane_pix_format {
+ __u32 sizeimage;
+ __u32 bytesperline;
+} __packed;
+
+/**
+ * struct pix_format_mplane - Multi-plane pixel format
+ * @width: Width of the image
+ * @height: Height of the image
+ * @pixelformat: Pixel format identifier
+ * @plane_fmt: Array of pixel formats for each plane
+ */
+struct pix_format_mplane {
+ __u32 width;
+ __u32 height;
+ __u32 pixelformat;
+ struct plane_pix_format plane_fmt[IMGBUF_MAX_PLANES];
+} __packed;
+
+/**
+ * struct frameparams - Frame parameters for MediaTek Image System
+ * @buf: Extended buffer information
+ * @fmt: Multi-plane pixel format
+ * @crop: Crop parameters
+ * @rotation: Rotation angle
+ * @hflip: Horizontal flip flag
+ * @vflip: Vertical flip flag
+ * @resizeratio: Resize ratio
+ */
+struct frameparams {
+ struct v4l2_ext_buffer buf; /* Extended buffer information */
+ struct pix_format_mplane fmt; /* Multi-plane pixel format */
+ struct mtk_imgsys_crop crop; /* Crop parameters */
+ __u32 rotation; /* Rotation angle */
+ __u32 hflip; /* Horizontal flip flag */
+ __u32 vflip; /* Vertical flip flag */
+ __u8 resizeratio; /* Resize ratio */
+};
+
+/**
+ * struct header_desc_norm - Header description for normal camera case
+ * @fparams_num: Number of frame parameters
+ * @fparams: Frame parameters
+ */
+struct header_desc_norm {
+ __u32 fparams_num;
+ struct frameparams fparams;
+};
+
+/**
+ * struct singlenode_desc_norm - Single node description for normal camera case
+ * @dmas_enable: Array indicating enabled DMA channels
+ * @dmas: Array of DMA header descriptions
+ * @tuning_meta: Tuning metadata header description
+ * @ctrl_meta: Control metadata header description
+ */
+struct singlenode_desc_norm {
+ __u8 dmas_enable[IMG_MAX_HW_DMAS];
+ struct header_desc_norm dmas[IMG_MAX_HW_DMAS];
+ struct header_desc_norm tuning_meta;
+ struct header_desc_norm ctrl_meta;
+};
+
#endif
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
index 28a35a3226a2..a52f1cf93d47 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
@@ -83,6 +83,67 @@ int mtk_imgsys_pipe_next_job_id(struct mtk_imgsys_pipe *pipe)
return (global_job_id & 0x0000FFFF) | (pipe->desc->id << 16);
}
+struct mtk_imgsys_request *
+mtk_imgsys_pipe_get_running_job(struct mtk_imgsys_pipe *pipe, int id)
+{
+ struct mtk_imgsys_request *req;
+ unsigned long flag;
+
+ spin_lock_irqsave(&pipe->running_job_lock, flag);
+ list_for_each_entry(req,
+ &pipe->pipe_job_running_list, list) {
+ if (req->id == id) {
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+ return req;
+ }
+ }
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+
+ return NULL;
+}
+
+void mtk_imgsys_pipe_remove_job(struct mtk_imgsys_request *req)
+{
+ unsigned long flag;
+
+ spin_lock_irqsave(&req->imgsys_pipe->running_job_lock, flag);
+ list_del(&req->list);
+ req->imgsys_pipe->num_jobs--;
+ spin_unlock_irqrestore(&req->imgsys_pipe->running_job_lock, flag);
+
+ dev_dbg(req->imgsys_pipe->imgsys_dev->dev,
+ "%s:%s:req->id(%d),num of running jobs(%d) entry(%p)\n", __func__,
+ req->imgsys_pipe->desc->name, req->id,
+ req->imgsys_pipe->num_jobs, &req->list);
+}
+
+void mtk_imgsys_pipe_job_finish(struct mtk_imgsys_request *req,
+ enum vb2_buffer_state vbf_state)
+{
+ struct mtk_imgsys_pipe *pipe = req->imgsys_pipe;
+ int i;
+
+ if (req->req.state != MEDIA_REQUEST_STATE_QUEUED) {
+ dev_info(pipe->imgsys_dev->dev, "%s: req %d 0x%lx flushed in state(%d)", __func__,
+ req->request_fd, (unsigned long)&req->req, req->req.state);
+ return;
+ }
+
+ for (i = 0; i < pipe->desc->total_queues; i++) {
+ struct mtk_imgsys_dev_buffer *dev_buf = req->buf_map[i];
+ struct mtk_imgsys_video_device *node;
+
+ if (!dev_buf)
+ continue;
+
+ node = mtk_imgsys_vbq_to_node(dev_buf->vbb.vb2_buf.vb2_queue);
+ spin_lock(&node->buf_list_lock);
+ list_del(&dev_buf->list);
+ spin_unlock(&node->buf_list_lock);
+ vb2_buffer_done(&dev_buf->vbb.vb2_buf, vbf_state);
+ }
+}
+
void mtk_imgsys_pipe_debug_job(struct mtk_imgsys_pipe *pipe,
struct mtk_imgsys_request *req)
{
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
index b8c30cb32b97..effee4a3b616 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
@@ -25,6 +25,8 @@
#include <media/videobuf2-v4l2.h>
#include "mtk_imgsys-module_ops.h"
#include "mtk_header_desc.h"
+#include "mtk-img-ipi.h"
+#include "mtk_imgsys-work_buf.h"
#define MTK_IMGSYS_MEDIA_MODEL_NAME "MTK-ISP-DIP-V4L2"
@@ -38,6 +40,31 @@
#define MTK_IMGSYS_CAPTURE_MAX_WIDTH 5376U
#define MTK_IMGSYS_CAPTURE_MAX_HEIGHT 4032U
+/**
+ * MTK_IMGSYS_JOB_TIMEOUT_MS - Defines the job timeout in milliseconds.
+ *
+ * This macro calculates the timeout for image system jobs based on the frame rate
+ * (30 frames per second) and the number of working buffers. The timeout is set to
+ * three times the duration of all working buffers.
+ */
+#define MTK_IMGSYS_JOB_TIMEOUT_MS (1000 / 30 * IMGSYS_WORKING_BUF_NUM * 3)
+
+/**
+ * struct mtk_imgsys_hw_working_buf_list - Represents a list of hardware
+ * working buffers
+ *
+ * @list: List head for linking multiple buffer lists
+ * @lock: Spinlock to protect the list
+ *
+ * This structure represents a list of hardware working buffers used in the
+ * MediaTek Image System. It includes a list head for linking multiple buffer
+ * lists and a spinlock to protect the list from concurrent access.
+ */
+struct mtk_imgsys_hw_working_buf_list {
+ struct list_head list;
+ spinlock_t lock; /* protect the list of working buf */
+};
+
/**
* struct mtk_imgsys_dev_format - Device format for MediaTek Image System
* @fmt: Pointer to the Image System format structure
@@ -274,6 +301,7 @@ struct mtk_imgsys_pipe {
* @imgsys_pipe: Image system pipeline configuration structure.
* @cust_pipes: Pointer to custom pipeline descriptors.
* @modules: Pointer to image system module operations.
+ * @num_mods: Number of modules in the image system.
* @clks: Array of clock bulk data for managing multiple clocks.
* @num_clks: Number of clocks in the clock array.
* @larbs: Array of device pointers for managing LARB (local arbiter).
@@ -281,9 +309,18 @@ struct mtk_imgsys_pipe {
* @scp: Pointer to the SCP (System Control Processor) structure.
* @rproc_handle: Handle for remote processor interface.
* @smem_dev: Device structure for shared memory operations.
- * @num_mods: Number of modules in the image system.
+ * @working_bufs: Array of hardware working buffers.
+ * @free_working_bufs: List of free working buffers.
+ * @composer_wq: Workqueue for composing tasks in image processing.
+ * @runner_wq: Workqueue for running tasks in image processing.
+ * @gcecb_wq: Workqueue for handling GCE callback tasks.
+ * @work_pool: Pool of GCE callback work structures.
+ * @flushing_waitq: Wait queue for flushing operations.
+ * @num_composing: Atomic counter for tracking composing operations.
+ * @imgsys_enqueue_cnt: Atomic counter for enqueued image system requests.
* @init_kref: Reference counter for initialization tracking.
* @hw_op_lock: Mutex for serializing hardware operations.
+ * @sem: Semaphore to limit the maximum number of requests sent to SCP.
* @stream_cnt: Counter for active streams, increased on stream on.
*/
struct mtk_imgsys_dev {
@@ -298,6 +335,7 @@ struct mtk_imgsys_dev {
const struct mtk_imgsys_pipe_desc *cust_pipes;
/* Imgsys Moudles Information */
const struct module_ops *modules;
+ int num_mods;
/* Clock Information */
struct clk_bulk_data *clks;
int num_clks;
@@ -308,11 +346,22 @@ struct mtk_imgsys_dev {
struct mtk_scp *scp;
struct rproc *rproc_handle;
struct device *smem_dev;
- /* Number of modules */
- int num_mods;
+ /* Working Buffers */
+ struct mtk_imgsys_hw_working_buf *working_bufs;
+ struct mtk_imgsys_hw_working_buf_list free_working_bufs;
+ /* Workqueues for ImgSys */
+ struct workqueue_struct *composer_wq;
+ struct workqueue_struct *runner_wq;
+ /* GCE (Generic Command Engine) Callback Workqueue */
+ struct workqueue_struct *gcecb_wq;
+ struct gce_cb_work_pool *work_pool;
/* Synchronization and Flow Control */
+ wait_queue_head_t flushing_waitq;
+ atomic_t num_composing;
+ atomic_t imgsys_enqueue_cnt;
struct kref init_kref;
struct mutex hw_op_lock; /* Protect HW operations */
+ struct semaphore sem;
int stream_cnt;
};
@@ -360,10 +409,74 @@ struct mtk_imgsys_request {
struct mtk_imgsys_dev_buffer **buf_map;
struct list_head list;
struct req_frameparam img_fparam;
+ struct work_struct composer_work;
+ struct work_struct runner_work;
+ struct mtk_imgsys_hw_working_buf *working_buf;
+ struct swfrm_info *swfrm_info;
atomic_t buf_count;
int request_fd;
};
+/**
+ * struct swfrm_info - Information about software frame
+ * @req_sbuf_goft: GCE buffer offset
+ * @req_sbuf_kva: GCE buffer virtual address
+ * @swfrminfo_ridx: Ring index of the swfrminfo (referenced by SCP)
+ * @request_fd: Enqueued request file descriptor in V4L2 kernel framework
+ * @request_no: Request number from middleware
+ * @frame_no: Frame number from middleware
+ * @frm_owner: Frame owner identifier from middleware
+ * @fps: Frames per second
+ * @cb_frmcnt: Frame counter
+ * @total_taskcnt: Total task count
+ * @req_vaddr: Kernel request virtual address
+ * @sync_id: Synchronization identifier for camera/image synchronization
+ * @total_frmnum: Total frame number in the current request
+ * @user_info: Array of user information from SCP to kernel
+ * @is_earlycb: Indicates if the request is separated by early callback hint
+ * @earlycb_idx: Frame index for early callback
+ * @is_lastfrm: Indicates if the request is separated by last frame
+ * @group_id: Group identifier for selecting GCE thread
+ * @req: Pointer to mtk_imgsys_request
+ * @pipe: Pointer to pipe mtk_imgsys_pipe
+ * @fail_uinfo_idx: Failed index in user_info
+ * @hw_hang: Indicates if the current failure is caused by hardware hang
+ * @ndd_swfrm_info: Pointer to NDD (Normal data dump) software frame information
+ *
+ * This structure contains detailed information about a software frame,
+ * including various identifiers, addresses, and status flags.
+ */
+struct swfrm_info {
+ u32 req_sbuf_goft;
+ void *req_sbuf_kva;
+ int swfrminfo_ridx;
+ int request_fd;
+ int request_no;
+ int frame_no;
+ u64 frm_owner;
+ int fps;
+ int cb_frmcnt;
+ int total_taskcnt;
+ u64 req_vaddr;
+ int sync_id;
+ int total_frmnum;
+ struct img_swfrm_info user_info[TMAX];
+ u8 is_earlycb;
+ int earlycb_idx;
+ u8 is_lastfrm;
+ s8 group_id;
+ void *req;
+ void *pipe;
+ u32 fail_uinfo_idx;
+ s8 hw_hang;
+ void *ndd_swfrm_info;
+} __attribute((__packed__));
+static inline struct mtk_imgsys_pipe*
+mtk_imgsys_dev_get_pipe(struct mtk_imgsys_dev *imgsys_dev)
+{
+ return &imgsys_dev->imgsys_pipe;
+}
+
static inline struct mtk_imgsys_video_device*
mtk_imgsys_file_to_node(struct file *file)
{
@@ -401,6 +514,18 @@ mtk_imgsys_media_req_to_imgsys_req(struct media_request *req)
return container_of(req, struct mtk_imgsys_request, req);
}
+static inline struct mtk_imgsys_request*
+mtk_imgsys_composer_work_to_req(struct work_struct *composer_work)
+{
+ return container_of(composer_work, struct mtk_imgsys_request, composer_work);
+}
+
+static inline struct mtk_imgsys_request*
+mtk_imgsys_runner_work_to_req(struct work_struct *runner_work)
+{
+ return container_of(runner_work, struct mtk_imgsys_request, runner_work);
+}
+
static inline int mtk_imgsys_buf_is_meta(u32 type)
{
return type == V4L2_BUF_TYPE_META_CAPTURE ||
@@ -413,8 +538,16 @@ int mtk_imgsys_pipe_init(struct mtk_imgsys_dev *imgsys_dev,
int mtk_imgsys_pipe_release(struct mtk_imgsys_pipe *pipe);
+struct mtk_imgsys_request *
+mtk_imgsys_pipe_get_running_job(struct mtk_imgsys_pipe *pipe, int id);
+
+void mtk_imgsys_pipe_remove_job(struct mtk_imgsys_request *req);
+
int mtk_imgsys_pipe_next_job_id(struct mtk_imgsys_pipe *pipe);
+void mtk_imgsys_pipe_job_finish(struct mtk_imgsys_request *req,
+ enum vb2_buffer_state vbf_state);
+
void mtk_imgsys_pipe_debug_job(struct mtk_imgsys_pipe *pipe,
struct mtk_imgsys_request *req);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
new file mode 100644
index 000000000000..ef86410e3d23
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "mtk_imgsys-scp.h"
+#include "mtk_imgsys-gce_work.h"
+
+static void cmdq_cb_done_worker(struct work_struct *work)
+{
+ struct mtk_imgsys_pipe *pipe;
+ struct swfrm_info *gwfrm_info = NULL;
+ struct gce_cb_work *gwork;
+ struct img_sw_buffer swbuf_data = {0};
+
+ gwork = container_of(work, struct gce_cb_work, work);
+ gwfrm_info = gwork->req_sbuf_kva;
+
+ pipe = gwork->pipe;
+ if (!pipe->streaming)
+ return;
+
+ /* send to SCP after frame done & del node from list */
+ swbuf_data.offset = gwfrm_info->req_sbuf_goft;
+ imgsys_scp_send(pipe->imgsys_dev, IPI_IMGSYS_DEQUE_DONE_ID,
+ &swbuf_data, sizeof(struct img_sw_buffer),
+ gwork->reqfd, 1);
+
+ gce_cb_work_free(pipe->imgsys_dev->work_pool, gwork);
+}
+
+int gce_cb_work_pool_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ int i;
+ struct gce_cb_work_pool *work_pool;
+
+ work_pool = kmalloc(sizeof(*work_pool), GFP_KERNEL);
+ if (!work_pool)
+ return -ENOMEM;
+ imgsys_dev->work_pool = work_pool;
+
+ spin_lock_init(&work_pool->lock);
+ INIT_LIST_HEAD(&work_pool->free_list);
+
+ for (i = 0; i < MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS; i++) {
+ INIT_WORK(&work_pool->work_items[i].work, cmdq_cb_done_worker);
+ atomic_set(&work_pool->work_items[i].state, WORK_INIT);
+ work_pool->work_items[i].work_idx = i;
+ list_add_tail(&work_pool->work_items[i].list, &work_pool->free_list);
+ }
+ return 0;
+}
+
+struct gce_cb_work *gce_cb_work_allocate(struct gce_cb_work_pool *pool)
+{
+ struct gce_cb_work *gwork = NULL;
+
+ spin_lock(&pool->lock);
+ if (list_empty(&pool->free_list)) {
+ spin_unlock(&pool->lock);
+ return NULL;
+ }
+
+ gwork = list_first_entry(&pool->free_list, struct gce_cb_work, list);
+ list_del(&gwork->list);
+ spin_unlock(&pool->lock);
+
+ atomic_set(&gwork->state, WORK_ALLOC);
+
+ return gwork;
+}
+
+void gce_cb_work_free(struct gce_cb_work_pool *pool, struct gce_cb_work *gwork)
+{
+ atomic_set(&gwork->state, WORK_FREE);
+
+ spin_lock(&pool->lock);
+ list_add_tail(&gwork->list, &pool->free_list);
+ spin_unlock(&pool->lock);
+}
+
+void gce_cb_work_pool_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ kfree(imgsys_dev->work_pool);
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
new file mode 100644
index 000000000000..ba80b952f38e
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _MTK_IMGSYS_GCE_WORK_H_
+#define _MTK_IMGSYS_GCE_WORK_H_
+
+#define MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS 60
+
+enum mtk_imgsys_gce_cb_work_state {
+ WORK_INIT,
+ WORK_ALLOC,
+ WORK_COMPLETED,
+ WORK_FREE
+};
+
+struct gce_cb_work {
+ struct work_struct work;
+ atomic_t state;
+ u32 work_idx;
+ u32 reqfd;
+ void *req_sbuf_kva;
+ void *pipe;
+ struct list_head list;
+ wait_queue_head_t wait_queue;
+};
+
+struct gce_cb_work_pool {
+ struct gce_cb_work work_items[MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS];
+ struct list_head free_list;
+ spinlock_t lock; /* protect the free_list */
+};
+
+int gce_cb_work_pool_init(struct mtk_imgsys_dev *imgsys_dev);
+struct gce_cb_work *gce_cb_work_allocate(struct gce_cb_work_pool *pool);
+void gce_cb_work_free(struct gce_cb_work_pool *pool, struct gce_cb_work *gwork);
+void gce_cb_work_pool_release(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_GCE_WORK_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 184f0e3a4d3b..36088a5bda8e 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -3,17 +3,261 @@
* Copyright (c) 2021 MediaTek Inc.
*
* Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
*
*/
#include <linux/pm_runtime.h>
#include "mtk_imgsys-sys.h"
#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-scp.h"
#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-work_buf.h"
+#include "mtk_imgsys-gce_work.h"
+#include "mtk_header_desc.h"
+#include "mtk_imgsys_v4l2_vnode.h"
+
+#define FRAME_STATE_INIT 0
+#define FRAME_STATE_HW_TIMEOUT 1
+
+static void mtk_imgsys_notify(struct mtk_imgsys_request *req, uint64_t frm_owner)
+{
+ struct mtk_imgsys_dev *imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ struct mtk_imgsys_pipe *pipe = req->imgsys_pipe;
+ struct req_frameparam *iparam = &req->img_fparam;
+ enum vb2_buffer_state vbf_state;
+
+ if (!mtk_imgsys_pipe_get_running_job(pipe, req->id))
+ goto out;
+
+ if (iparam->state != FRAME_STATE_HW_TIMEOUT)
+ vbf_state = VB2_BUF_STATE_DONE;
+ else
+ vbf_state = VB2_BUF_STATE_ERROR;
+
+ atomic_dec(&imgsys_dev->num_composing);
+
+ mtk_imgsys_hw_working_buf_free(imgsys_dev, req->working_buf);
+ req->working_buf = NULL;
+ mtk_imgsys_pipe_remove_job(req);
+ if (vbf_state == VB2_BUF_STATE_DONE)
+ mtk_imgsys_pipe_job_finish(req, vbf_state);
+
+ wake_up(&imgsys_dev->flushing_waitq);
+ dev_dbg(imgsys_dev->dev,
+ "%s:%s:(reqfd-%d) , frame_no(%d) frm_owner(%s) finished\n",
+ __func__, pipe->desc->name, req->request_fd, iparam->frame_no,
+ ((char *)(&frm_owner)));
+
+out:
+ media_request_put(&req->req);
+}
+
+static void imgsys_cmdq_cb_func(struct imgsys_cmdq_cb_data data)
+{
+ struct mtk_imgsys_pipe *pipe;
+ struct mtk_imgsys_request *req;
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct swfrm_info *swfrminfo_cb;
+ struct gce_cb_work *gwork;
+
+ swfrminfo_cb = data.data;
+ pipe = (struct mtk_imgsys_pipe *)swfrminfo_cb->pipe;
+ if (!pipe->streaming) {
+ pr_info("%s pipe already streamoff\n", __func__);
+ return;
+ }
+
+ req = (struct mtk_imgsys_request *)(swfrminfo_cb->req);
+ if (!req) {
+ pr_info("%s NULL request Address\n", __func__);
+ return;
+ }
+
+ imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ dev_dbg(imgsys_dev->dev, "%s:(reqfd-%d)frame_no(%d) +", __func__,
+ req->request_fd,
+ req->img_fparam.frame_no);
+
+ if (swfrminfo_cb->hw_hang >= 0)
+ req->img_fparam.state = FRAME_STATE_HW_TIMEOUT;
+
+ mtk_imgsys_notify(req, swfrminfo_cb->frm_owner);
+
+ if (swfrminfo_cb->hw_hang < 0) {
+ gwork = gce_cb_work_allocate(imgsys_dev->work_pool);
+ if (!gwork)
+ return;
+
+ gwork->reqfd = swfrminfo_cb->request_fd;
+ gwork->req_sbuf_kva = swfrminfo_cb->req_sbuf_kva;
+ gwork->pipe = swfrminfo_cb->pipe;
+ init_waitqueue_head(&gwork->wait_queue);
+
+ queue_work(req->imgsys_pipe->imgsys_dev->gcecb_wq, &gwork->work);
+ }
+}
+
+static void imgsys_runner_func(struct work_struct *work)
+{
+ struct mtk_imgsys_request *req = mtk_imgsys_runner_work_to_req(work);
+ struct mtk_imgsys_dev *imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ struct swfrm_info *frm_info;
+ int ret;
+
+ frm_info = req->swfrm_info;
+
+ ret = imgsys_cmdq_sendtask(imgsys_dev, frm_info, imgsys_cmdq_cb_func);
+ if (ret)
+ dev_info(imgsys_dev->dev,
+ "%s: imgsys_cmdq_sendtask fail(%d)\n", __func__, ret);
+}
+
+/* processing IPI_IMGSYS_FRAME_ID, share data is struct img_sw_buffer*/
+static void imgsys_scp_handler(void *data, unsigned int len, void *priv)
+{
+ struct share_buf *scp_msg = (struct share_buf *)data;
+ struct mtk_imgsys_pipe *pipe;
+ struct mtk_imgsys_request *req;
+ struct mtk_imgsys_dev *imgsys_dev = (struct mtk_imgsys_dev *)priv;
+ struct img_sw_buffer *swbuf_data;
+ struct swfrm_info *swfrm_info;
+ int i = 0;
+ void *gce_virt;
+ int total_framenum = 0;
+
+ if (!data) {
+ WARN_ONCE(!data, "%s: failed due to NULL data\n", __func__);
+ return;
+ }
+
+ /* this function just handles id IPI_IMGSYS_FRAME_ID */
+ if (scp_msg->id != IPI_IMGSYS_FRAME_ID) {
+ dev_dbg(imgsys_dev->dev,
+ "%s: get invalid IPI ID(%d)\n", __func__, scp_msg->id);
+ return;
+ }
+
+ if (WARN_ONCE(scp_msg->len != sizeof(struct img_sw_buffer),
+ "%s: len(%d) not match img_sw_buffer\n", __func__, len))
+ return;
+
+ swbuf_data = (struct img_sw_buffer *)scp_msg->share_data;
+ gce_virt = imgsys_scp_get_reserve_mem_virt(IMG_MEM_G_ID);
+ if (!gce_virt) {
+ pr_info("%s: invalid gce_virt(%p)\n",
+ __func__, gce_virt);
+ return;
+ }
+
+ swfrm_info = (struct swfrm_info *)(gce_virt + (swbuf_data->offset));
+ if ((int)swbuf_data->offset < 0 ||
+ swbuf_data->offset > imgsys_scp_get_reserve_mem_size(IMG_MEM_G_ID)) {
+ pr_info("%s: invalid swbuf_data->offset(%d), max(%llu)\n",
+ __func__, swbuf_data->offset,
+ (u64)imgsys_scp_get_reserve_mem_size(IMG_MEM_G_ID));
+ return;
+ }
+
+ swfrm_info->req_sbuf_goft = swbuf_data->offset;
+ swfrm_info->req_sbuf_kva = gce_virt + (swbuf_data->offset);
+
+ pipe = mtk_imgsys_dev_get_pipe(imgsys_dev);
+ if (!pipe) {
+ dev_info(imgsys_dev->dev,
+ "%s: get invalid img_ipi_frameparam from firmware\n",
+ __func__);
+ return;
+ }
+
+ req = (struct mtk_imgsys_request *)(uintptr_t)swfrm_info->req_vaddr;
+ if (!req) {
+ WARN_ONCE(!req, "%s: frame_no(%d) is lost\n", __func__, req->id);
+ return;
+ }
+
+ if (!req->working_buf) {
+ dev_dbg(imgsys_dev->dev,
+ "%s: (reqfd-%d) composing\n",
+ __func__, req->request_fd);
+ }
+
+ swfrm_info->req = (void *)req;
+ swfrm_info->pipe = (void *)pipe;
+ swfrm_info->cb_frmcnt = 0;
+ swfrm_info->total_taskcnt = 0;
+ swfrm_info->hw_hang = -1;
+ total_framenum = swfrm_info->total_frmnum;
+
+ if (total_framenum < 0 || total_framenum > TMAX) {
+ dev_info(imgsys_dev->dev,
+ "%s:unexpected total_framenum (%d -> %d), MAX (%d/%d)\n",
+ __func__, swfrm_info->total_frmnum,
+ total_framenum,
+ TMAX, TMAX);
+ return;
+ }
+
+ for (i = 0 ; i < total_framenum ; i++) {
+ swfrm_info->user_info[i].g_swbuf =
+ gce_virt + (swfrm_info->user_info[i].sw_goft);
+ }
+
+ req->swfrm_info = swfrm_info;
+
+ up(&imgsys_dev->sem);
+
+ INIT_WORK(&req->runner_work, imgsys_runner_func);
+ queue_work(req->imgsys_pipe->imgsys_dev->runner_wq,
+ &req->runner_work);
+}
+
+static int mtk_imgsys_hw_flush_pipe_jobs(struct mtk_imgsys_pipe *pipe)
+{
+ struct mtk_imgsys_request *req;
+ struct list_head job_list = LIST_HEAD_INIT(job_list);
+ unsigned long flag;
+ int num;
+ int ret;
+
+ spin_lock_irqsave(&pipe->running_job_lock, flag);
+ list_splice_init(&pipe->pipe_job_running_list, &job_list);
+ pipe->num_jobs = 0;
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+
+ num = atomic_read(&pipe->imgsys_dev->num_composing);
+ ret = wait_event_timeout(pipe->imgsys_dev->flushing_waitq, !(num),
+ msecs_to_jiffies(MTK_IMGSYS_JOB_TIMEOUT_MS));
+ if (!ret && num) {
+ dev_info(pipe->imgsys_dev->dev,
+ "flushing is aborted, num(%d)\n", num);
+ return -EINVAL;
+ }
+
+ list_for_each_entry(req, &job_list, list)
+ mtk_imgsys_pipe_job_finish(req, VB2_BUF_STATE_ERROR);
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s: wakeup num(%d)\n", __func__, num);
+ return 0;
+}
+
+static void module_uninit(struct kref *kref)
+{
+ struct mtk_imgsys_dev *imgsys_dev;
+ int i;
+
+ imgsys_dev = container_of(kref, struct mtk_imgsys_dev, init_kref);
+
+ for (i = 0; i < (imgsys_dev->num_mods); i++)
+ if (imgsys_dev->modules[i].uninit)
+ imgsys_dev->modules[i].uninit(imgsys_dev);
+}
static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
{
int ret, i;
+ struct img_init_info info;
ret = pm_runtime_get_sync(imgsys_dev->dev);
if (ret < 0) {
@@ -23,38 +267,228 @@ static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
/* Set default value for hardware modules */
for (i = 0; i < (imgsys_dev->num_mods); i++)
- imgsys_dev->modules[i].init(imgsys_dev);
+ if (imgsys_dev->modules[i].init)
+ imgsys_dev->modules[i].init(imgsys_dev);
kref_init(&imgsys_dev->init_kref);
+ ret = imgsys_scp_alloc_reserve_mem(imgsys_dev);
+ if (ret) {
+ dev_err(imgsys_dev->dev, "%s: imgsys_scp_alloc_reserve_mem failed %d\n",
+ __func__, ret);
+ return -EBUSY;
+ }
+
+ /* IMGSYS HW INIT */
+ memset(&info, 0, sizeof(info));
+ info.header_version = HEADER_VER;
+ info.frameparam_size = sizeof(struct img_ipi_frameparam);
+
+ imgsys_scp_get_reserve_mem_info(&info);
+
+ imgsys_scp_init(imgsys_dev, imgsys_scp_handler);
+
+ ret = imgsys_scp_send(imgsys_dev, IPI_IMGSYS_INIT_ID,
+ (void *)&info, sizeof(info), 0, 1);
+
+ if (ret) {
+ dev_err(imgsys_dev->dev, "%s: send IPI_IMGSYS_INIT_ID failed %d\n",
+ __func__, ret);
+ return -EBUSY;
+ }
+
/* calling cmdq stream on */
imgsys_cmdq_streamon(imgsys_dev);
+
return 0;
}
-static void module_uninit(struct kref *kref)
+static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
{
- struct mtk_imgsys_dev *imgsys_dev;
+ int ret;
+ struct img_init_info info = {0};
+
+ imgsys_scp_send(imgsys_dev, IPI_IMGSYS_DEINIT_ID,
+ (void *)&info, sizeof(info), 0, 1);
+
+ imgsys_scp_deinit(imgsys_dev);
+ /* calling cmdq stream off */
+ imgsys_cmdq_streamoff(imgsys_dev);
+
+ /* RELEASE IMGSYS WORKING BUFFER FIRST */
+ ret = imgsys_scp_free_reserve_mem(imgsys_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: imgsys_scp_free_reserve_mem failed(%d)\n",
+ __func__, ret);
+ }
+
+ pm_runtime_put_sync(imgsys_dev->dev);
+
+ kref_put(&imgsys_dev->init_kref, module_uninit);
+}
+
+static void imgsys_fill_img_buf(struct mtk_imgsys_dev_buffer *dev_buf,
+ struct header_desc_norm *desc_norm,
+ struct mtk_imgsys_request *req)
+{
+ struct frameparams *buf_info = &desc_norm->fparams;
int i;
- imgsys_dev = container_of(kref, struct mtk_imgsys_dev, init_kref);
+ desc_norm->fparams_num = 1;
- for (i = 0; i < (imgsys_dev->num_mods); i++)
- if (imgsys_dev->modules[i].uninit)
- imgsys_dev->modules[i].uninit(imgsys_dev);
+ buf_info->buf.num_planes = dev_buf->fmt.fmt.pix_mp.num_planes;
+ buf_info->fmt.width = dev_buf->fmt.fmt.pix_mp.width;
+ buf_info->fmt.height = dev_buf->fmt.fmt.pix_mp.height;
+ buf_info->fmt.pixelformat = dev_buf->fmt.fmt.pix_mp.pixelformat;
+ buf_info->crop = dev_buf->crop;
+ buf_info->rotation = dev_buf->rotation;
+ buf_info->hflip = dev_buf->hflip;
+ buf_info->vflip = dev_buf->vflip;
+ buf_info->resizeratio = dev_buf->resize_ratio;
+
+ for (i = 0; i < buf_info->buf.num_planes; i++) {
+ buf_info->buf.planes[i].m.dma_buf.offset =
+ dev_buf->vbb.planes[i].data_offset;
+ buf_info->buf.planes[i].isp_addr = dev_buf->isp_daddr[i];
+ buf_info->buf.planes[i].size = dev_buf->vbb.vb2_buf.planes[i].length;
+ buf_info->fmt.plane_fmt[i].sizeimage =
+ dev_buf->vbb.vb2_buf.planes[i].min_length;
+ buf_info->fmt.plane_fmt[i].bytesperline =
+ dev_buf->fmt.fmt.pix_mp.plane_fmt[i].bytesperline;
+ }
}
-static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
+static void imgsys_fill_meta_buf(struct mtk_imgsys_dev_buffer *dev_buf,
+ struct header_desc_norm *desc_norm,
+ struct mtk_imgsys_request *req)
{
- /* calling cmdq stream off */
- imgsys_cmdq_streamoff(imgsys_dev);
- kref_put(&imgsys_dev->init_kref, module_uninit);
+ struct frameparams *buf_info = &desc_norm->fparams;
+
+ memset(&buf_info->buf, 0, sizeof(buf_info->buf));
+
+ desc_norm->fparams_num = 1;
+
+ buf_info->buf.num_planes = 1;
+
+ buf_info->fmt.width = dev_buf->fmt.fmt.meta.buffersize;
+ buf_info->fmt.height = 1;
+ buf_info->fmt.pixelformat = dev_buf->fmt.fmt.meta.dataformat;
+ buf_info->fmt.plane_fmt[0].sizeimage = dev_buf->fmt.fmt.meta.buffersize;
+ buf_info->fmt.plane_fmt[0].bytesperline = dev_buf->fmt.fmt.meta.buffersize;
+ buf_info->buf.planes[0].isp_addr = dev_buf->isp_daddr[0];
+ buf_info->buf.planes[0].size = dev_buf->vbb.vb2_buf.planes[0].length;
+ buf_info->buf.planes[0].m.dma_buf.phyaddr = dev_buf->scp_daddr[0];
+}
+
+static void imgsys_fill_buf_info(int idx, struct mtk_imgsys_dev_buffer *dev_buf,
+ struct singlenode_desc_norm *sd_norm,
+ struct mtk_imgsys_request *req)
+{
+ if (!dev_buf)
+ return;
+
+ if (idx < 0 || idx >= IMG_MAX_HW_DMAS)
+ return;
+
+ sd_norm->dmas_enable[idx] = 1;
+
+ switch (idx) {
+ case MTK_IMGSYS_VIDEO_NODE_ID_TUNING_OUT:
+ imgsys_fill_meta_buf(dev_buf, &sd_norm->tuning_meta, req);
+ break;
+ case MTK_IMGSYS_VIDEO_NODE_ID_CTRLMETA_OUT:
+ imgsys_fill_meta_buf(dev_buf, &sd_norm->ctrl_meta, req);
+ break;
+ case MTK_IMGSYS_VIDEO_NODE_ID_METAI_OUT:
+ case MTK_IMGSYS_VIDEO_NODE_ID_FEO_CAPTURE:
+ case MTK_IMGSYS_VIDEO_NODE_ID_IMGSTATO_CAPTURE:
+ imgsys_fill_meta_buf(dev_buf, &sd_norm->dmas[idx], req);
+ break;
+ default:
+ imgsys_fill_img_buf(dev_buf, &sd_norm->dmas[idx], req);
+ break;
+ }
+}
+
+static void imgsys_fill_singledev_buf(struct mtk_imgsys_request *req)
+{
+ struct mtk_imgsys_hw_working_buf *buf = req->working_buf;
+ int i;
+
+ if (!buf)
+ return;
+
+ memset(buf->sd_norm, 0, sizeof(*buf->sd_norm));
+ for (i = 0; i < req->imgsys_pipe->desc->total_queues; i++)
+ imgsys_fill_buf_info(i, req->buf_map[i], buf->sd_norm, req);
+}
+
+static void imgsys_composer_workfunc(struct work_struct *work)
+{
+ struct mtk_imgsys_request *req = mtk_imgsys_composer_work_to_req(work);
+ struct mtk_imgsys_dev *imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ struct img_ipi_param ipi_param;
+ int ret;
+
+ imgsys_fill_singledev_buf(req);
+
+ dev_dbg(imgsys_dev->dev,
+ "%s:(reqfd-%d) to send frame_no(%d)\n",
+ __func__, req->request_fd,
+ req->img_fparam.frame_no);
+ media_request_get(&req->req);
+
+ if (down_interruptible(&imgsys_dev->sem))
+ return;
+
+ ipi_param.req_addr_va = (u64)(uintptr_t)req;
+ ipi_param.frm_param.offset = 0;
+ ipi_param.frm_param.scp_addr = req->working_buf->scp_daddr;
+
+ mutex_lock(&imgsys_dev->hw_op_lock);
+ atomic_inc(&imgsys_dev->num_composing);
+
+ ret = imgsys_scp_send(imgsys_dev, IPI_IMGSYS_FRAME_ID,
+ &ipi_param, sizeof(ipi_param),
+ req->request_fd, 0);
+
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "frame_no(%d) send SCP_IPI_DIP_FRAME failed %d\n",
+ req->img_fparam.frame_no, ret);
+ mtk_imgsys_pipe_remove_job(req);
+ mtk_imgsys_hw_working_buf_free(imgsys_dev, req->working_buf);
+ req->working_buf = NULL;
+ mtk_imgsys_pipe_job_finish(req, VB2_BUF_STATE_ERROR);
+ wake_up(&imgsys_dev->flushing_waitq);
+ }
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+
+ dev_dbg(imgsys_dev->dev, "%s:(reqfd-%d) sent\n", __func__,
+ req->request_fd);
}
void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
struct mtk_imgsys_request *req)
{
- /* Function implementation will be provided in subsequent patches */
+ struct req_frameparam *frameparam = &req->img_fparam;
+
+ req->working_buf = mtk_imgsys_hw_working_buf_alloc(imgsys_dev);
+ if (!req->working_buf) {
+ dev_info(imgsys_dev->dev,
+ "%s:req(%p): no free working buffer available\n",
+ req->imgsys_pipe->desc->name, req);
+ return;
+ }
+
+ frameparam->state = FRAME_STATE_INIT;
+ frameparam->frame_no =
+ atomic_inc_return(&imgsys_dev->imgsys_enqueue_cnt);
+
+ INIT_WORK(&req->composer_work, imgsys_composer_workfunc);
+ queue_work(req->imgsys_pipe->imgsys_dev->composer_wq,
+ &req->composer_work);
}
int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
@@ -72,10 +506,13 @@ int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
mutex_unlock(&imgsys_dev->hw_op_lock);
return ret;
}
+ INIT_LIST_HEAD(&pipe->pipe_job_running_list);
}
imgsys_dev->stream_cnt++;
+ atomic_set(&imgsys_dev->num_composing, 0);
mutex_unlock(&imgsys_dev->hw_op_lock);
+ pipe->streaming = 1;
dev_dbg(pipe->imgsys_dev->dev,
"%s:%s: stream on, id(%d), stream cnt(%d)\n",
__func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
@@ -86,11 +523,18 @@ int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
int mtk_imgsys_hw_streamoff(struct mtk_imgsys_pipe *pipe)
{
struct mtk_imgsys_dev *imgsys_dev = pipe->imgsys_dev;
+ int ret;
dev_dbg(pipe->imgsys_dev->dev,
"%s:%s: stream off, id(%d), stream cnt(%d)\n",
__func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
+ pipe->streaming = 0;
+
+ ret = mtk_imgsys_hw_flush_pipe_jobs(pipe);
+ if (ret)
+ return ret;
+
mutex_lock(&imgsys_dev->hw_op_lock);
imgsys_dev->stream_cnt--;
if (!imgsys_dev->stream_cnt)
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index e25f17b98649..ea2ce03c2619 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -5,6 +5,7 @@
* Author: Frederic Chen <frederic.chen@mediatek.com>
*
*/
+
#include <linux/platform_device.h>
#include <linux/of_platform.h>
#include <linux/of_device.h>
@@ -12,6 +13,7 @@
#include <linux/pm_runtime.h>
#include <linux/dma-mapping.h>
#include <linux/vmalloc.h>
+#include <linux/remoteproc/mtk_scp.h>
#include <linux/videodev2.h>
#include <media/videobuf2-dma-contig.h>
#include <media/v4l2-ioctl.h>
@@ -29,6 +31,9 @@
#include "mtk_imgsys_v4l2_vnode.h"
#include "mtk_imgsys-dev.h"
#include "mtk_imgsys-sys.h"
+#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-work_buf.h"
+#include "mtk_imgsys-gce_work.h"
#define IMGSYS_MAX_BUFFERS 256
#define IMGSYS_SUSPEND_TIME 3000 /* ms */
@@ -1467,6 +1472,77 @@ static void mtk_imgsys_dev_v4l2_release(struct mtk_imgsys_dev *imgsys_dev)
mtk_imgsys_dev_media_unregister(imgsys_dev);
}
+static int mtk_imgsys_res_init(struct platform_device *pdev,
+ struct mtk_imgsys_dev *imgsys_dev)
+{
+ int ret;
+
+ imgsys_dev->gcecb_wq =
+ alloc_ordered_workqueue("%s",
+ __WQ_LEGACY | WQ_MEM_RECLAIM |
+ WQ_FREEZABLE,
+ "imgsys_mdp_callback");
+ if (!imgsys_dev->gcecb_wq) {
+ dev_err(imgsys_dev->dev,
+ "%s: unable to alloc mdpcb workqueue\n", __func__);
+ return -ENOMEM;
+ }
+
+ imgsys_dev->composer_wq =
+ alloc_ordered_workqueue("%s",
+ __WQ_LEGACY | WQ_MEM_RECLAIM |
+ WQ_FREEZABLE | WQ_HIGHPRI,
+ "imgsys_composer");
+ if (!imgsys_dev->composer_wq) {
+ dev_err(imgsys_dev->dev,
+ "%s: unable to alloc composer workqueue\n", __func__);
+ ret = -ENOMEM;
+ goto destroy_gcecb_wq;
+ }
+
+ imgsys_dev->runner_wq =
+ alloc_ordered_workqueue("%s",
+ __WQ_LEGACY | WQ_MEM_RECLAIM |
+ WQ_FREEZABLE | WQ_HIGHPRI,
+ "imgsys_runner");
+ if (!imgsys_dev->runner_wq) {
+ dev_err(imgsys_dev->dev,
+ "%s: unable to alloc runner workqueue\n", __func__);
+ ret = -ENOMEM;
+ goto destroy_composer_wq;
+ }
+
+ init_waitqueue_head(&imgsys_dev->flushing_waitq);
+
+ return 0;
+
+destroy_composer_wq:
+ destroy_workqueue(imgsys_dev->composer_wq);
+
+destroy_gcecb_wq:
+ destroy_workqueue(imgsys_dev->gcecb_wq);
+
+ return ret;
+}
+
+static void mtk_imgsys_res_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ flush_workqueue(imgsys_dev->gcecb_wq);
+ destroy_workqueue(imgsys_dev->gcecb_wq);
+ imgsys_dev->gcecb_wq = NULL;
+
+ flush_workqueue(imgsys_dev->composer_wq);
+ destroy_workqueue(imgsys_dev->composer_wq);
+ imgsys_dev->composer_wq = NULL;
+
+ flush_workqueue(imgsys_dev->runner_wq);
+ destroy_workqueue(imgsys_dev->runner_wq);
+ imgsys_dev->runner_wq = NULL;
+
+ atomic_set(&imgsys_dev->num_composing, 0);
+ atomic_set(&imgsys_dev->imgsys_enqueue_cnt, 0);
+}
+
static int mtk_imgsys_of_rproc(struct mtk_imgsys_dev *imgsys,
struct platform_device *pdev)
{
@@ -1519,6 +1595,8 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
imgsys_dev->num_mods = data->mod_num;
imgsys_dev->stream_cnt = 0;
mutex_init(&imgsys_dev->hw_op_lock);
+ atomic_set(&imgsys_dev->imgsys_enqueue_cnt, 0);
+ atomic_set(&imgsys_dev->num_composing, 0);
/* Get Clocks */
imgsys_dev->clks = data->clks;
@@ -1592,6 +1670,33 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
goto err_free_larb_alloc;
}
+ /* Initializing resource for ImgSys */
+ ret = mtk_imgsys_res_init(pdev, imgsys_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: mtk_imgsys_res_init failed(%d)\n", __func__, ret);
+
+ ret = -EBUSY;
+ goto err_release_deinit_v4l2;
+ }
+
+ ret = mtk_imgsys_hw_working_buf_pool_init(imgsys_dev);
+ if (ret) {
+ dev_info(&pdev->dev, "working buffer init failed(%d)\n", ret);
+ goto err_release_deinit_v4l2;
+ }
+
+ /* Initializing resource for CMDQ driver */
+ imgsys_cmdq_init(imgsys_dev);
+
+ sema_init(&imgsys_dev->sem, MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS);
+
+ ret = gce_cb_work_pool_init(imgsys_dev);
+ if (ret) {
+ dev_info(&pdev->dev, "gce callback work pool init failed(%d)\n", ret);
+ goto err_release_working_buf_pool;
+ }
+
/* Enable power*/
pm_runtime_set_autosuspend_delay(&pdev->dev, IMGSYS_SUSPEND_TIME);
pm_runtime_use_autosuspend(&pdev->dev);
@@ -1600,11 +1705,15 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
ret = platform_driver_register(&mtk_imgsys_larb_driver);
if (ret) {
dev_info(imgsys_dev->dev, "register mtk_imgsys_larb_driver fail\n");
- goto err_release_deinit_v4l2;
+ goto err_free_gce_cb_work_pool_alloc;
}
return 0;
+err_free_gce_cb_work_pool_alloc:
+ gce_cb_work_pool_release(imgsys_dev);
+err_release_working_buf_pool:
+ mtk_imgsys_hw_working_buf_pool_release(imgsys_dev);
err_release_deinit_v4l2:
mtk_imgsys_dev_v4l2_release(imgsys_dev);
err_free_larb_alloc:
@@ -1618,17 +1727,32 @@ static void mtk_imgsys_remove(struct platform_device *pdev)
{
struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(&pdev->dev);
+ mtk_imgsys_res_release(imgsys_dev);
pm_runtime_disable(&pdev->dev);
platform_driver_unregister(&mtk_imgsys_larb_driver);
mtk_imgsys_dev_v4l2_release(imgsys_dev);
+ mtk_imgsys_hw_working_buf_pool_release(imgsys_dev);
+ gce_cb_work_pool_release(imgsys_dev);
mutex_destroy(&imgsys_dev->hw_op_lock);
+ imgsys_cmdq_release(imgsys_dev);
devm_kfree(&pdev->dev, imgsys_dev->larbs);
devm_kfree(&pdev->dev, imgsys_dev);
}
static int __maybe_unused mtk_imgsys_pm_suspend(struct device *dev)
{
- int ret;
+ struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(dev);
+ int ret, num;
+
+ num = atomic_read(&imgsys_dev->num_composing);
+ ret = wait_event_timeout(imgsys_dev->flushing_waitq, !(num),
+ msecs_to_jiffies(MTK_IMGSYS_JOB_TIMEOUT_MS));
+ if (!ret && num) {
+ dev_info(dev, "%s: flushing SCP job timeout, num(%d)\n",
+ __func__, num);
+
+ return -EBUSY;
+ }
if (pm_runtime_suspended(dev)) {
dev_info(dev, "%s: pm_runtime_suspended is true, no action\n",
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
new file mode 100644
index 000000000000..5f220bbe8f3f
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#include "mtk_imgsys-work_buf.h"
+
+int mtk_imgsys_hw_working_buf_pool_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct mtk_imgsys_hw_working_buf *buf;
+ void *sd;
+ dma_addr_t scp_addr, isp_daddr;
+ size_t size = sizeof(struct singlenode_desc_norm);
+ int i;
+
+ if (imgsys_dev->working_bufs)
+ return 0;
+
+ INIT_LIST_HEAD(&imgsys_dev->free_working_bufs.list);
+ spin_lock_init(&imgsys_dev->free_working_bufs.lock);
+
+ imgsys_dev->working_bufs = kmalloc_array(IMGSYS_WORKING_BUF_NUM, sizeof(*buf), GFP_KERNEL);
+ if (!imgsys_dev->working_bufs)
+ return -ENOMEM;
+
+ sd = dma_alloc_coherent(imgsys_dev->smem_dev,
+ size * IMGSYS_WORKING_BUF_NUM,
+ &scp_addr, GFP_KERNEL);
+ if (!sd)
+ goto err_alloc;
+
+ isp_daddr = dma_map_resource(imgsys_dev->dev,
+ scp_addr,
+ size * IMGSYS_WORKING_BUF_NUM,
+ DMA_TO_DEVICE, 0);
+ if (!isp_daddr)
+ goto err_map;
+
+ for (i = 0; i < IMGSYS_WORKING_BUF_NUM; i++) {
+ buf = &imgsys_dev->working_bufs[i];
+
+ buf->sd_norm = sd;
+ buf->isp_daddr = isp_daddr;
+ buf->scp_daddr = scp_addr;
+
+ sd += size;
+ isp_daddr += size;
+ scp_addr += size;
+
+ list_add_tail(&buf->list,
+ &imgsys_dev->free_working_bufs.list);
+ }
+
+ return 0;
+
+err_map:
+ dma_free_coherent(imgsys_dev->dev,
+ sizeof(struct singlenode_desc_norm),
+ sd, scp_addr);
+err_alloc:
+ kfree(imgsys_dev->working_bufs);
+
+ return -ENOMEM;
+}
+
+struct mtk_imgsys_hw_working_buf*
+mtk_imgsys_hw_working_buf_alloc(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct mtk_imgsys_hw_working_buf *working_buf;
+
+ spin_lock(&imgsys_dev->free_working_bufs.lock);
+ if (list_empty(&imgsys_dev->free_working_bufs.list)) {
+ spin_unlock(&imgsys_dev->free_working_bufs.lock);
+ return NULL;
+ }
+
+ working_buf = list_first_entry(&imgsys_dev->free_working_bufs.list,
+ struct mtk_imgsys_hw_working_buf,
+ list);
+ list_del(&working_buf->list);
+ spin_unlock(&imgsys_dev->free_working_bufs.lock);
+
+ return working_buf;
+}
+
+void mtk_imgsys_hw_working_buf_free(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_hw_working_buf *working_buf)
+{
+ if (!working_buf)
+ return;
+
+ spin_lock(&imgsys_dev->free_working_bufs.lock);
+ list_add_tail(&working_buf->list,
+ &imgsys_dev->free_working_bufs.list);
+ spin_unlock(&imgsys_dev->free_working_bufs.lock);
+}
+
+void mtk_imgsys_hw_working_buf_pool_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct mtk_imgsys_hw_working_buf *buf = &imgsys_dev->working_bufs[0];
+
+ dma_unmap_resource(imgsys_dev->dev, buf->isp_daddr,
+ sizeof(struct singlenode_desc_norm) * IMGSYS_WORKING_BUF_NUM,
+ DMA_TO_DEVICE, 0);
+ dma_free_coherent(imgsys_dev->dev,
+ sizeof(struct singlenode_desc_norm) * IMGSYS_WORKING_BUF_NUM,
+ buf->sd_norm, buf->scp_daddr);
+ kfree(imgsys_dev->working_bufs);
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
new file mode 100644
index 000000000000..d54b02e6dfa8
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_WORK_BUF_H_
+#define _MTK_IMGSYS_WORK_BUF_H_
+
+#include <linux/clk.h>
+#include <linux/dma-buf.h>
+#include <linux/scatterlist.h>
+#include "mtk_header_desc.h"
+#include "mtk_imgsys-dev.h"
+
+#define IMGSYS_WORKING_BUF_NUM 32
+
+/**
+ * struct mtk_imgsys_hw_working_buf - Represents a hardware working buffer
+ * @sd_norm: Pointer to the single node descriptor normalized structure
+ * @dbuf: Pointer to the DMA buffer
+ * @attach: Pointer to the DMA buffer attachment
+ * @sgt: Pointer to the scatter-gather table
+ * @scp_daddr: Physical address for SCP (System Control Processor)
+ * @isp_daddr: Virtual address for ISP (Image Signal Processor)
+ * @list: List head for linking multiple buffers
+ *
+ * This structure represents a hardware working buffer used in the MediaTek
+ * Image System. It includes pointers to various buffer-related structures
+ * and addresses, as well as a list head for linking multiple buffers.
+ */
+struct mtk_imgsys_hw_working_buf {
+ struct singlenode_desc_norm *sd_norm;
+ struct dma_buf *dbuf;
+ struct dma_buf_attachment *attach;
+ struct sg_table *sgt;
+ dma_addr_t scp_daddr;
+ dma_addr_t isp_daddr;
+ struct list_head list;
+};
+
+int mtk_imgsys_hw_working_buf_pool_init(struct mtk_imgsys_dev *imgsys_dev);
+
+struct mtk_imgsys_hw_working_buf*
+mtk_imgsys_hw_working_buf_alloc(struct mtk_imgsys_dev *imgsys_dev);
+
+void mtk_imgsys_hw_working_buf_free(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_hw_working_buf *working_buf);
+
+void mtk_imgsys_hw_working_buf_pool_release(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_WORK_BUF_H_ */
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v1 10/10] media: mediatek: isp: Add normal data dump flow
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (8 preceding siblings ...)
2025-05-24 11:50 ` [PATCH v1 09/10] media: mediatek: isp: Add image processing flow Olivia Wen
@ 2025-05-24 11:50 ` Olivia Wen
9 siblings, 0 replies; 16+ messages in thread
From: Olivia Wen @ 2025-05-24 11:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the normal data dump flow. It is for debugging purposes
within the MediaTek Image System (ImgSys) driver. The data dump
functionality is essential for diagnosing issues, tracking the flow of
data, and ensuring the proper operation of image processing tasks.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 2 +
.../isp_7x/imgsys/modules/mtk_imgsys-dip.c | 47 +++
.../isp_7x/imgsys/modules/mtk_imgsys-dip.h | 3 +
.../isp/isp_7x/imgsys/modules/mtk_imgsys-me.c | 65 ++++
.../isp/isp_7x/imgsys/modules/mtk_imgsys-me.h | 30 ++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.c | 52 +++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.h | 3 +
.../isp_7x/imgsys/modules/mtk_imgsys-traw.c | 55 +++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.h | 3 +
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.c | 60 +++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.h | 3 +
.../mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h | 8 +
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.c | 4 +
.../isp_7x/imgsys/mtk_imgsys-module_main.c | 4 +
.../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 11 +
.../isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c | 342 ++++++++++++++++++
.../isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h | 64 ++++
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 11 +
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 7 +
19 files changed, 774 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 3aa53e8bfc6d..16044a280d5c 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,6 +2,7 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-ndd_debugfs.o \
mtk_imgsys-work_buf.o \
mtk_imgsys-gce_work.o \
mtk_imgsys-cmdq.o \
@@ -14,6 +15,7 @@ mtk_imgsys-sys.o \
mtk_imgsys-v4l2.o
mtk_imgsys_hw_isp-objs := \
+modules/mtk_imgsys-me.o \
modules/mtk_imgsys-dip.o \
modules/mtk_imgsys-traw.o \
modules/mtk_imgsys-pqdip.o \
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
index 1cc1f3e87e8b..d9563d5a177a 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
@@ -10,6 +10,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-dip.h"
/********************************************************************
@@ -42,6 +43,8 @@
#define DIPCQ_D1A_DIPCQ_CQ_THR17_CTL 0x318
#define DIPCQ_D1A_DIPCQ_CQ_THR18_CTL 0x328
+#define DIP_REG_RANGE 0x17000
+
#define DIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_dip_init_ary)
/********************************************************************
@@ -103,6 +106,50 @@ void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_dip_init);
+void imgsys_dip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char *dip_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ ssize_t ret;
+ void *cq_va;
+
+ if (frm_dump_info->eng_e != IMGSYS_NDD_ENG_DIP)
+ return;
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(DIP_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ dip_name = "REG_DIP_dip.reg";
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, dip_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ dev_err(imgsys_dev->dev, "wrong dip ndd file name %s\n", file_name);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ ret = copy_to_user(frm_dump_info->user_buffer, cq_va, 0x9000);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += 0x48000 + 0x9000;
+
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va + 0x9000, DIP_REG_RANGE - 0x9000);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += DIP_REG_RANGE - 0x9000;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_dip_ndd_dump);
+
void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
index 22b4a7c3616d..1f0b85f1faba 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
@@ -10,8 +10,11 @@
#define _MTK_DIP_DIP_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_dip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_DIP_DIP_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
new file mode 100644
index 000000000000..dacb3d591a71
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+#include <linux/of_address.h>
+#include <linux/uaccess.h>
+#include "mtk_imgsys-me.h"
+
+#define ME_REG_RANGE 0xA20
+
+void imgsys_me_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char *me_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ void __iomem *reg_va;
+ ssize_t ret;
+
+ if (frm_dump_info->eng_e != IMGSYS_NDD_ENG_ME)
+ return;
+
+ reg_va = ioremap(frm_dump_info->cq_ofst[frm_dump_info->eng_e], ME_REG_RANGE);
+ if (!reg_va)
+ return;
+
+ me_name = "REG_ME_me.reg";
+
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, me_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ iounmap(reg_va);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret) {
+ iounmap(reg_va);
+ return;
+ }
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ void *kernel_buffer = kmalloc(ME_REG_RANGE, GFP_KERNEL);
+
+ if (!kernel_buffer) {
+ iounmap(reg_va);
+ return;
+ }
+
+ memcpy_fromio(kernel_buffer, reg_va, ME_REG_RANGE);
+ ret = copy_to_user(frm_dump_info->user_buffer, kernel_buffer, ME_REG_RANGE);
+ kfree(kernel_buffer);
+ if (ret != 0) {
+ iounmap(reg_va);
+ return;
+ }
+ frm_dump_info->user_buffer += ME_REG_RANGE;
+ }
+
+ iounmap(reg_va);
+}
+EXPORT_SYMBOL(imgsys_me_ndd_dump);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
new file mode 100644
index 000000000000..10ed0153b740
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_ME_H_
+#define _MTK_IMGSYS_ME_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
+
+/**
+ * imgsys_me_ndd_dump - Dump the frame information for debugging.
+ * @imgsys_dev: Pointer to the mtk_imgsys_dev structure representing the ISP image
+ * processing device.
+ * @frm_dump_info: Pointer to the imgsys_ndd_frm_dump_info structure containing
+ * frame dump information.
+ *
+ * This function dumps the NDD (Normal Data Dump) frame information for debugging
+ * purposes. It provides insights into the frame data processed by the ISP image
+ * processing device, which can help in identifying issues related to frame
+ * handling.
+ */
+void imgsys_me_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
+
+#endif /* _MTK_IMGSYS_ME_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
index d1d3507555d9..f1f688b13ba5 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-pqdip.h"
/********************************************************************
@@ -40,6 +41,8 @@
#define DIPCQ_P1A_REG_DIPCQ_CQ_THR17_CTL 0x0318
#define DIPCQ_P1A_REG_DIPCQ_CQ_THR18_CTL 0x0328
+#define PQDIP_REG_RANGE 0x6000
+
#define PQDIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_pqdip_init_ary)
/********************************************************************
@@ -100,6 +103,55 @@ void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_pqdip_init);
+void imgsys_pqdip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char *pqdip_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ ssize_t ret;
+ void *cq_va;
+
+ if (frm_dump_info->eng_e != IMGSYS_NDD_ENG_PQDIP_A &&
+ frm_dump_info->eng_e != IMGSYS_NDD_ENG_PQDIP_B)
+ return;
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(PQDIP_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ switch (frm_dump_info->eng_e) {
+ case IMGSYS_NDD_ENG_PQDIP_A:
+ pqdip_name = "REG_PQDIP_A_pqdipa.reg";
+ break;
+ case IMGSYS_NDD_ENG_PQDIP_B:
+ pqdip_name = "REG_PQDIP_B_pqdipb.reg";
+ break;
+ default:
+ return;
+ }
+
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, pqdip_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ dev_err(imgsys_dev->dev, "wrong dip ndd file name %s\n", file_name);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ ret = copy_to_user(frm_dump_info->user_buffer, cq_va, PQDIP_REG_RANGE);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += PQDIP_REG_RANGE;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_pqdip_ndd_dump);
+
void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
index 00f93d536181..09bc51a1c15d 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
@@ -10,8 +10,11 @@
#define _MTK_IMGSYS_PQDIP_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_pqdip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_IMGSYS_PQDIP_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
index 292f0ad09c46..90597f3a776a 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-traw.h"
/********************************************************************
@@ -16,6 +17,8 @@
********************************************************************/
#define TRAWCTL_INT1_EN 0x00A0
+#define TRAW_REG_RANGE 0xC000
+
#define TRAW_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_traw_init_ary)
/********************************************************************
@@ -53,6 +56,58 @@ void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_traw_init);
+void imgsys_traw_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ const char *traw_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ ssize_t ret;
+ void *cq_va;
+
+ if (frm_dump_info->eng_e > IMGSYS_NDD_ENG_XTR ||
+ frm_dump_info->eng_e < IMGSYS_NDD_ENG_TRAW)
+ return;
+
+ switch (frm_dump_info->eng_e) {
+ case IMGSYS_NDD_ENG_TRAW:
+ traw_name = "REG_TRAW_traw.reg";
+ break;
+ case IMGSYS_NDD_ENG_LTR:
+ traw_name = "REG_LTRAW_ltraw.reg";
+ break;
+ case IMGSYS_NDD_ENG_XTR:
+ traw_name = "REG_XTRAW_xtraw.reg";
+ break;
+ default:
+ return;
+ }
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(TRAW_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, traw_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ dev_err(imgsys_dev->dev, "wrong traw ndd file name %s\n", file_name);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ ret = copy_to_user(frm_dump_info->user_buffer, cq_va, TRAW_REG_RANGE);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += TRAW_REG_RANGE;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_traw_ndd_dump);
+
void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
index f3727216b58c..57881bc2e191 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
@@ -10,8 +10,11 @@
#define _MTK_IMGSYS_TRAW_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_traw_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_IMGSYS_TRAW_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
index b19d61529b80..276af83a9ec2 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-wpe.h"
/********************************************************************
@@ -41,6 +42,10 @@
#define DIPCQ_W1A_DIPCQ_CQ_THR13_CTL_OFFSET 0x0BD8
#define DIPCQ_W1A_DIPCQ_CQ_THR14_CTL_OFFSET 0x0BE8
+#define WPE_REG_RANGE 0x1000
+#define WPE_PSP_END_OFST 0x554
+#define WPE_COEF_END_OFST 0x5D8
+
#define WPE_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_wpe_init_ary)
/********************************************************************
* Global Variable
@@ -101,6 +106,61 @@ void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_wpe_init);
+void imgsys_wpe_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char file_name[NDD_FP_SIZE] = "\0";
+ void *cq_va, *psp_va;
+ u64 psp_ofst;
+ ssize_t ret;
+
+ if (frm_dump_info->eng_e > IMGSYS_NDD_ENG_WPE_LITE)
+ return;
+
+ if (!frm_dump_info->user_buffer)
+ return;
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(WPE_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ psp_va = imgsys_scp_get_reserve_mem_virt(WPE_MEM_C_ID);
+ if (!psp_va)
+ return;
+
+ psp_ofst = frm_dump_info->wpe_psp_ofst[frm_dump_info->eng_e];
+ psp_va += psp_ofst;
+
+ ret = snprintf(file_name, sizeof(file_name),
+ "%s%s", frm_dump_info->fp, frm_dump_info->wpe_fp);
+ if (ret < 0 || ret >= sizeof(file_name))
+ return;
+
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ if (psp_ofst != 0) {
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va, WPE_PSP_END_OFST);
+ frm_dump_info->user_buffer += WPE_PSP_END_OFST;
+
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ psp_va, WPE_COEF_END_OFST - WPE_PSP_END_OFST);
+ frm_dump_info->user_buffer += (WPE_COEF_END_OFST - WPE_PSP_END_OFST);
+
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va + WPE_COEF_END_OFST, WPE_REG_RANGE - WPE_COEF_END_OFST);
+ frm_dump_info->user_buffer += (WPE_REG_RANGE - WPE_COEF_END_OFST);
+ } else {
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va, WPE_REG_RANGE);
+ frm_dump_info->user_buffer += WPE_REG_RANGE;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_wpe_ndd_dump);
+
void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
index 6001575265e6..77b713bbe499 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
@@ -10,8 +10,11 @@
#define _MTK_IMGSYS_WPE_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_wpe_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_IMGSYS_WPE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
index 850f6ec99a96..705d8c35ff55 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
@@ -81,6 +81,10 @@ struct img_init_info {
* @is_earlycb: Indicates if early callback is enabled
* @sw_goft: offset of GCE buffer
* @g_swbuf: virtual address of GCE buffer
+ * @ndd_cq_ofst: NDD CQ offset
+ * @ndd_wpe_psp_ofst: NDD WPE PSP offset
+ * @ndd_wpe_fp: NDD WPE file path
+ * @ndd_fp: NDD file path
*
* These fields are primarily set by SCP (System Control Processor) and used by
* the kernel for processing and verifing data.
@@ -92,6 +96,10 @@ struct img_swfrm_info {
u8 is_earlycb;
u64 sw_goft;
void *g_swbuf;
+ u64 ndd_cq_ofst[IMG_MAX_HW];
+ u64 ndd_wpe_psp_ofst[WPE_ENG_NUM];
+ char ndd_wpe_fp[64];
+ char ndd_fp[256];
} __packed;
/**
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
index ef86410e3d23..7e5b7cd383f9 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
@@ -2,6 +2,7 @@
#include "mtk_imgsys-scp.h"
#include "mtk_imgsys-gce_work.h"
+#include "mtk_imgsys-ndd_debugfs.h"
static void cmdq_cb_done_worker(struct work_struct *work)
{
@@ -17,6 +18,9 @@ static void cmdq_cb_done_worker(struct work_struct *work)
if (!pipe->streaming)
return;
+ if (gwfrm_info->user_info[0].ndd_fp[0] != '\0')
+ wait_event(gwork->wait_queue, atomic_read(&gwork->state) == WORK_COMPLETED);
+
/* send to SCP after frame done & del node from list */
swbuf_data.offset = gwfrm_info->req_sbuf_goft;
imgsys_scp_send(pipe->imgsys_dev, IPI_IMGSYS_DEQUE_DONE_ID,
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
index 5daa8f863bd2..c31ada82afcd 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
@@ -10,6 +10,7 @@
#include <linux/of_address.h>
#include "mtk_imgsys-module_common.h"
#include "mtk_imgsys-module_main.h"
+#include "mtk_imgsys-ndd_debugfs.h"
/********************************************************************
* Global Define
@@ -147,6 +148,7 @@ void imgsys_main_init(struct mtk_imgsys_dev *imgsys_dev)
}
imgsys_main_hw_init(imgsys_dev);
+ imgsys_ndd_debugfs_init(imgsys_dev);
}
void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev)
@@ -180,4 +182,6 @@ void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev)
iounmap(dip2_reg_base);
dip2_reg_base = NULL;
}
+
+ imgsys_ndd_debugfs_uninit(imgsys_dev);
}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
index a2eea8f4fb3c..1dc28993bc09 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
@@ -25,14 +25,23 @@ enum mtk_imgsys_module {
};
struct mtk_imgsys_dev;
+struct imgsys_ndd_frm_dump_info;
/**
* struct module_ops - Operations for a specific module in the MediaTek Image System
+ *
* @module_id: Identifier for the module.
* @init: Function pointer to initialize the module.
* This function sets the default values for various software-related
* parameters and ensures that the module is ready for operation.
* Example function: imgsys_dip_init
+ * @ndddump: Function pointer to perform a dump of the module's frame information.
+ * This function is responsible for dumping frame-related information.
+ * It is typically used for debugging and diagnostic purposes to analyze
+ * the state of the system and the frames being processed.
+ * The dumped information can be used to trace issues or verify the
+ * correctness of the frame processing pipeline.
+ * Example function: imgsys_dip_ndd_dump
* @uninit: Function pointer to uninitialize the module.
* This function resets various software-related parameters to ensure
* the module is properly cleaned up and ready for shutdown.
@@ -48,6 +57,8 @@ struct mtk_imgsys_dev;
struct module_ops {
int module_id;
void (*init)(struct mtk_imgsys_dev *imgsys_dev);
+ void (*ndddump)(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void (*uninit)(struct mtk_imgsys_dev *imgsys_dev);
};
#endif /* _MTK_IMGSYS_MODULES_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
new file mode 100644
index 000000000000..d0c1168fbd06
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
@@ -0,0 +1,342 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#include <linux/debugfs.h>
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/of_address.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/uaccess.h>
+#include <soc/mediatek/smi.h>
+#include "mtk_imgsys-ndd_debugfs.h"
+#include "mtk_imgsys-module_common.h"
+#include "mtk_imgsys-gce_work.h"
+
+#define WPE_ENG_NUM 3
+#define TRAW_ENG_NUM 3
+#define DIP_ENG_NUM 1
+#define PQDIP_ENG_NUM 2
+#define ME_ENG_NUM 1
+
+#define WPE_NDD_SIZE 0x1000
+#define TRAW_NDD_SIZE 0xC000
+#define DIP_NDD_SIZE 0x5F000
+#define PQDIP_NDD_SIZE 0x6000
+#define ME_NDD_SIZE 0xA20
+
+struct ndd_request_t {
+ int fd;
+ struct list_head list;
+} __packed;
+
+struct user_ndd_requests_t {
+ struct list_head ndd_req_list; /* ndd_request_t list */
+ struct mutex lock; /* protect the ndd_req_list list*/
+};
+
+struct ndd_frame_info_t {
+ struct list_head ndd_swfrm_list; /* ndd_swfrm_info_t list */
+ struct mutex lock; /* protect the ndd_swfrm_list list*/
+};
+
+struct ndd_wpe_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[WPE_NDD_SIZE];
+};
+
+struct ndd_traw_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[TRAW_NDD_SIZE];
+};
+
+struct ndd_dip_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[DIP_NDD_SIZE];
+};
+
+struct ndd_pqdip_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[PQDIP_NDD_SIZE];
+};
+
+struct ndd_me_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[ME_NDD_SIZE];
+};
+
+struct ndd_frm_info_t {
+ struct ndd_wpe_info_t wpe[WPE_ENG_NUM];
+ struct ndd_traw_info_t traw[TRAW_ENG_NUM];
+ struct ndd_dip_info_t dip[DIP_ENG_NUM];
+ struct ndd_pqdip_info_t pqdip[PQDIP_ENG_NUM];
+ struct ndd_me_info_t me[ME_ENG_NUM];
+} __packed;
+
+static struct ndd_frame_info_t *ndd_frames;
+
+#if defined(CONFIG_DEBUG_FS)
+#define DEBUGFS_DIR "mtk_imgsys_debug"
+#define DEBUGFS_FILE "mtk_imgsys_ndd"
+#define FILE_PERMISSIONS 0644
+
+static struct dentry *debugfs_dir, *debugfs_file;
+static struct user_ndd_requests_t *ndd_reqs;
+
+static ssize_t debugfs_write(struct file *file, const char __user *user_buffer,
+ size_t count, loff_t *ppos)
+{
+ char buffer[64];
+ int ret;
+ struct ndd_request_t *current_request;
+
+ if (count >= sizeof(buffer))
+ return -EINVAL;
+
+ if (copy_from_user(buffer, user_buffer, count))
+ return -EFAULT;
+
+ buffer[count] = '\0';
+
+ current_request = kmalloc(sizeof(*current_request), GFP_KERNEL);
+ if (!current_request)
+ return -ENOMEM;
+
+ ret = sscanf(buffer, "fd=%d", ¤t_request->fd);
+ if (ret != 1)
+ return -EINVAL;
+
+ mutex_lock(&ndd_reqs->lock);
+ list_add_tail(¤t_request->list, &ndd_reqs->ndd_req_list);
+ mutex_unlock(&ndd_reqs->lock);
+
+ *ppos += count;
+
+ return count;
+}
+
+static void imgsys_ndd_dump_routine(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ int i = 0;
+ int mid = 0;
+ unsigned int hw_num = 0x1;
+ unsigned int module_copy_ofst = 0;
+ const struct module_ops *imgsys_modules = imgsys_dev->modules;
+
+ dev_dbg(imgsys_dev->dev,
+ "%s: hw comb set: 0x%x\n", __func__, frm_dump_info->hw_comb);
+
+ for (i = 0; hw_num < IMGSYS_ENG_MAX_NUM; i++, hw_num = (0x1 << i)) {
+ switch (hw_num) {
+ case IMGSYS_ENG_WPE_EIS:
+ case IMGSYS_ENG_WPE_TNR:
+ case IMGSYS_ENG_WPE_LITE:
+ module_copy_ofst = (NDD_FP_SIZE + WPE_NDD_SIZE);
+ mid = IMGSYS_MOD_WPE;
+ break;
+ case IMGSYS_ENG_TRAW:
+ case IMGSYS_ENG_LTR:
+ case IMGSYS_ENG_XTR:
+ module_copy_ofst = (NDD_FP_SIZE + TRAW_NDD_SIZE);
+ mid = IMGSYS_MOD_TRAW;
+ break;
+ case IMGSYS_ENG_DIP:
+ module_copy_ofst = (NDD_FP_SIZE + DIP_NDD_SIZE);
+ mid = IMGSYS_MOD_DIP;
+ break;
+ case IMGSYS_ENG_PQDIP_A:
+ case IMGSYS_ENG_PQDIP_B:
+ module_copy_ofst = (NDD_FP_SIZE + PQDIP_NDD_SIZE);
+ mid = IMGSYS_MOD_PQDIP;
+ break;
+ case IMGSYS_ENG_ME:
+ module_copy_ofst = (NDD_FP_SIZE + ME_NDD_SIZE);
+ mid = IMGSYS_MOD_ME;
+ break;
+ default:
+ dev_err(imgsys_dev->dev, "%s get wrong hw id %d\n", __func__, hw_num);
+ break;
+ }
+
+ if (frm_dump_info->hw_comb & hw_num) {
+ frm_dump_info->eng_e = i;
+ if (imgsys_modules[mid].ndddump &&
+ frm_dump_info->eng_e <= IMGSYS_NDD_ENG_ME)
+ imgsys_modules[mid].ndddump(imgsys_dev, frm_dump_info);
+ } else {
+ if (frm_dump_info->user_buffer)
+ frm_dump_info->user_buffer += module_copy_ofst;
+ else
+ frm_dump_info->user_buffer = NULL;
+ }
+
+ if ((frm_dump_info->hw_comb >> i) == 0)
+ return;
+ }
+}
+
+static ssize_t debugfs_read(struct file *file, char __user *user_buffer,
+ size_t count, loff_t *ppos)
+{
+ struct ndd_swfrm_info_t *cur_ndd_swfrm = NULL, *swfrm_temp = NULL;
+ struct ndd_request_t *cur_user_req = NULL, *req_temp = NULL;
+ struct img_swfrm_info *subfrm_info = NULL;
+ struct imgsys_ndd_frm_dump_info frm_dump_info = { 0 };
+ struct gce_cb_work_pool *work_pool = NULL;
+ ssize_t read_count = 0;
+ size_t buffer_size = count;
+ bool found = false;
+ int frm, ret = 0;
+ u32 gce_cb_work_idx = MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS;
+
+ mutex_lock(&ndd_frames->lock);
+
+ list_for_each_entry_safe(cur_ndd_swfrm, swfrm_temp, &ndd_frames->ndd_swfrm_list, list) {
+ mutex_lock(&ndd_reqs->lock);
+ list_for_each_entry_safe(cur_user_req, req_temp, &ndd_reqs->ndd_req_list, list) {
+ if (cur_ndd_swfrm->gwfrm_info &&
+ cur_ndd_swfrm->gwfrm_info->request_fd == cur_user_req->fd) {
+ found = true;
+ break;
+ }
+ }
+ mutex_unlock(&ndd_reqs->lock);
+
+ if (found) {
+ for (frm = 0; frm < cur_ndd_swfrm->gwfrm_info->total_frmnum; frm++) {
+ subfrm_info = &cur_ndd_swfrm->gwfrm_info->user_info[frm];
+ memset(&frm_dump_info, 0, sizeof(frm_dump_info));
+
+ if (buffer_size < sizeof(struct ndd_frm_info_t)) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ if (subfrm_info->ndd_fp[0] == '\0')
+ continue;
+
+ frm_dump_info.user_buffer = user_buffer +
+ sizeof(struct ndd_frm_info_t) * frm;
+ frm_dump_info.hw_comb = subfrm_info->hw_comb;
+ frm_dump_info.cq_ofst = subfrm_info->ndd_cq_ofst;
+ frm_dump_info.wpe_psp_ofst = subfrm_info->ndd_wpe_psp_ofst;
+ frm_dump_info.wpe_fp = subfrm_info->ndd_wpe_fp;
+ frm_dump_info.fp = subfrm_info->ndd_fp;
+
+ imgsys_ndd_dump_routine(cur_ndd_swfrm->imgsys_dev, &frm_dump_info);
+ read_count += sizeof(struct ndd_frm_info_t);
+ buffer_size -= sizeof(struct ndd_frm_info_t);
+ }
+
+ work_pool = cur_ndd_swfrm->imgsys_dev->work_pool;
+ do {
+ gce_cb_work_idx = atomic_read(&cur_ndd_swfrm->gce_cb_work_idx);
+ } while (gce_cb_work_idx == MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS);
+ atomic_set(&work_pool->work_items[gce_cb_work_idx].state, WORK_COMPLETED);
+ wake_up(&work_pool->work_items[gce_cb_work_idx].wait_queue);
+
+ list_del(&cur_ndd_swfrm->list);
+ kfree(cur_ndd_swfrm);
+ cur_ndd_swfrm = NULL;
+ list_del(&cur_user_req->list);
+ kfree(cur_user_req);
+ cur_user_req = NULL;
+ break;
+ }
+ }
+
+ if (!found)
+ ret = -ENOENT;
+out:
+ mutex_unlock(&ndd_frames->lock);
+
+ return ret == 0 ? read_count : ret;
+}
+
+static const struct file_operations debugfs_fops = {
+ .write = debugfs_write,
+ .read = debugfs_read,
+};
+
+void imgsys_ndd_debugfs_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ debugfs_dir = debugfs_create_dir(DEBUGFS_DIR, NULL);
+ if (!debugfs_dir) {
+ dev_err(imgsys_dev->dev, "%s Failed to create debugfs dir\n", __func__);
+ return;
+ }
+
+ debugfs_file = debugfs_create_file(DEBUGFS_FILE, FILE_PERMISSIONS,
+ debugfs_dir, NULL, &debugfs_fops);
+ if (!debugfs_file) {
+ dev_err(imgsys_dev->dev, "%s Failed to create debugfs file\n", __func__);
+ return;
+ }
+
+ ndd_frames = kmalloc(sizeof(*ndd_frames), GFP_KERNEL);
+ if (!ndd_frames)
+ return;
+ INIT_LIST_HEAD(&ndd_frames->ndd_swfrm_list);
+ mutex_init(&ndd_frames->lock);
+
+ ndd_reqs = kmalloc(sizeof(*ndd_reqs), GFP_KERNEL);
+ if (!ndd_reqs)
+ return;
+ INIT_LIST_HEAD(&ndd_reqs->ndd_req_list);
+ mutex_init(&ndd_reqs->lock);
+}
+EXPORT_SYMBOL_GPL(imgsys_ndd_debugfs_init);
+
+void imgsys_ndd_debugfs_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct ndd_request_t *req, *tmp;
+
+ debugfs_remove(debugfs_file);
+ debugfs_remove(debugfs_dir);
+
+ mutex_lock(&ndd_reqs->lock);
+ list_for_each_entry_safe(req, tmp, &ndd_reqs->ndd_req_list, list) {
+ list_del(&req->list);
+ kfree(req);
+ }
+ mutex_unlock(&ndd_reqs->lock);
+ kfree(ndd_reqs);
+
+ mutex_lock(&ndd_frames->lock);
+ list_for_each_entry_safe(req, tmp, &ndd_frames->ndd_swfrm_list, list) {
+ list_del(&req->list);
+ kfree(req);
+ }
+ mutex_unlock(&ndd_frames->lock);
+ kfree(ndd_frames);
+}
+EXPORT_SYMBOL_GPL(imgsys_ndd_debugfs_uninit);
+#endif
+
+void imgsys_ndd_swfrm_list_add(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *gwfrm_info)
+{
+ struct ndd_swfrm_info_t *new_ndd_swfrm;
+
+ new_ndd_swfrm = kzalloc(sizeof(*new_ndd_swfrm), GFP_KERNEL);
+ if (!new_ndd_swfrm)
+ return;
+
+ new_ndd_swfrm->imgsys_dev = imgsys_dev;
+ new_ndd_swfrm->gwfrm_info = gwfrm_info;
+ atomic_set(&new_ndd_swfrm->gce_cb_work_idx, MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS);
+
+ gwfrm_info->ndd_swfrm_info = (void *)new_ndd_swfrm;
+
+ mutex_lock(&ndd_frames->lock);
+ list_add_tail(&new_ndd_swfrm->list, &ndd_frames->ndd_swfrm_list);
+ mutex_unlock(&ndd_frames->lock);
+}
+EXPORT_SYMBOL_GPL(imgsys_ndd_swfrm_list_add);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
new file mode 100644
index 000000000000..f45cacc7bfff
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_NDD_DEBUGFS_H_
+#define _MTK_IMGSYS_NDD_DEBUGFS_H_
+
+#include "mtk_imgsys-dev.h"
+
+#define NDD_FP_SIZE 256 /* NDD file path size */
+
+enum mtk_imgsys_ndd_eng {
+ IMGSYS_NDD_ENG_WPE_EIS = 0,
+ IMGSYS_NDD_ENG_WPE_TNR = 1,
+ IMGSYS_NDD_ENG_WPE_LITE = 2,
+ IMGSYS_NDD_ENG_ADL_A = 3,
+ IMGSYS_NDD_ENG_ADL_B = 4,
+ IMGSYS_NDD_ENG_TRAW = 5,
+ IMGSYS_NDD_ENG_LTR = 6,
+ IMGSYS_NDD_ENG_XTR = 7,
+ IMGSYS_NDD_ENG_DIP = 8,
+ IMGSYS_NDD_ENG_PQDIP_A = 9,
+ IMGSYS_NDD_ENG_PQDIP_B = 10,
+ IMGSYS_NDD_ENG_ME = 11,
+ IMGSYS_NDD_ENG_MAX_NUM = 12
+};
+
+struct imgsys_ndd_frm_dump_info {
+ char __user *user_buffer;
+ unsigned int hw_comb;
+ enum mtk_imgsys_ndd_eng eng_e;
+ u64 *cq_ofst;
+ u64 *wpe_psp_ofst;
+ char *wpe_fp;
+ char *fp;
+};
+
+struct ndd_swfrm_info_t {
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct swfrm_info *gwfrm_info;
+ struct list_head list;
+ atomic_t gce_cb_work_idx;
+} __packed;
+
+#if defined(CONFIG_DEBUG_FS)
+void imgsys_ndd_debugfs_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_ndd_debugfs_uninit(struct mtk_imgsys_dev *imgsys_dev);
+#else
+static inline void imgsys_ndd_debugfs_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+}
+
+static inline void imgsys_ndd_debugfs_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+}
+#endif
+void imgsys_ndd_swfrm_list_add(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *gwfrm_info);
+#endif /* _MTK_IMGSYS_DEBUG_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 36088a5bda8e..a87b254cd114 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -16,6 +16,7 @@
#include "mtk_imgsys-gce_work.h"
#include "mtk_header_desc.h"
#include "mtk_imgsys_v4l2_vnode.h"
+#include "mtk_imgsys-ndd_debugfs.h"
#define FRAME_STATE_INIT 0
#define FRAME_STATE_HW_TIMEOUT 1
@@ -60,6 +61,7 @@ static void imgsys_cmdq_cb_func(struct imgsys_cmdq_cb_data data)
struct mtk_imgsys_dev *imgsys_dev;
struct swfrm_info *swfrminfo_cb;
struct gce_cb_work *gwork;
+ struct ndd_swfrm_info_t *ndd_swfrm;
swfrminfo_cb = data.data;
pipe = (struct mtk_imgsys_pipe *)swfrminfo_cb->pipe;
@@ -94,6 +96,12 @@ static void imgsys_cmdq_cb_func(struct imgsys_cmdq_cb_data data)
gwork->pipe = swfrminfo_cb->pipe;
init_waitqueue_head(&gwork->wait_queue);
+ if (swfrminfo_cb->user_info[0].ndd_fp[0] != '\0') {
+ ndd_swfrm = (struct ndd_swfrm_info_t *)swfrminfo_cb->ndd_swfrm_info;
+ if (ndd_swfrm)
+ atomic_set(&ndd_swfrm->gce_cb_work_idx, gwork->work_idx);
+ }
+
queue_work(req->imgsys_pipe->imgsys_dev->gcecb_wq, &gwork->work);
}
}
@@ -207,6 +215,9 @@ static void imgsys_scp_handler(void *data, unsigned int len, void *priv)
up(&imgsys_dev->sem);
+ if (swfrm_info->user_info[0].ndd_fp[0] != '\0')
+ imgsys_ndd_swfrm_list_add(imgsys_dev, swfrm_info);
+
INIT_WORK(&req->runner_work, imgsys_runner_func);
queue_work(req->imgsys_pipe->imgsys_dev->runner_wq,
&req->runner_work);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index ea2ce03c2619..b0088d4de69c 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -26,6 +26,7 @@
#include "modules/mtk_imgsys-traw.h"
#include "modules/mtk_imgsys-pqdip.h"
#include "modules/mtk_imgsys-wpe.h"
+#include "modules/mtk_imgsys-me.h"
#include "mtk_imgsys-of.h"
#include "mtk_imgsys-vnode_id.h"
#include "mtk_imgsys_v4l2_vnode.h"
@@ -1846,31 +1847,37 @@ static const struct module_ops imgsys_isp7_modules[] = {
[IMGSYS_MOD_IMGMAIN] = {
.module_id = IMGSYS_MOD_IMGMAIN,
.init = imgsys_main_init,
+ .ndddump = NULL,
.uninit = imgsys_main_uninit,
},
[IMGSYS_MOD_WPE] = {
.module_id = IMGSYS_MOD_WPE,
.init = imgsys_wpe_init,
+ .ndddump = imgsys_wpe_ndd_dump,
.uninit = imgsys_wpe_uninit,
},
[IMGSYS_MOD_TRAW] = {
.module_id = IMGSYS_MOD_TRAW,
.init = imgsys_traw_init,
+ .ndddump = imgsys_traw_ndd_dump,
.uninit = imgsys_traw_uninit,
},
[IMGSYS_MOD_DIP] = {
.module_id = IMGSYS_MOD_DIP,
.init = imgsys_dip_init,
+ .ndddump = imgsys_dip_ndd_dump,
.uninit = imgsys_dip_uninit,
},
[IMGSYS_MOD_PQDIP] = {
.module_id = IMGSYS_MOD_PQDIP,
.init = imgsys_pqdip_init,
+ .ndddump = imgsys_pqdip_ndd_dump,
.uninit = imgsys_pqdip_uninit,
},
[IMGSYS_MOD_ME] = {
.module_id = IMGSYS_MOD_ME,
.init = NULL,
+ .ndddump = imgsys_me_ndd_dump,
.uninit = NULL,
},
};
--
2.45.2
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
@ 2025-05-24 13:37 ` Rob Herring (Arm)
0 siblings, 0 replies; 16+ messages in thread
From: Rob Herring (Arm) @ 2025-05-24 13:37 UTC (permalink / raw)
To: Olivia Wen
Cc: teddy.chen, linux-arm-kernel, yunkec, Mauro Carvalho Chehab,
Conor Dooley, linux-mediatek, devicetree, Matthias Brugger,
Project_Global_Chrome_Upstream_Group, linux-media,
Krzysztof Kozlowski, AngeloGioacchino Del Regno, linux-kernel
On Sat, 24 May 2025 19:49:53 +0800, Olivia Wen wrote:
> Introduce more Image System (ImgSys) components present in MT8188.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> .../bindings/media/mediatek,imgsys.yaml | 180 ++++++++++++++++++
> 1 file changed, 180 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek,imgsys.example.dtb: imgsys-fw@15000000 (mediatek,mt8188-isp-imgsys): reg: [[0, 352321536], [0, 16384], [0, 352452608], [0, 65536], [0, 352583680], [0, 65536], [0, 358875136], [0, 65536], [0, 353370112], [0, 65536], [0, 353697792], [0, 65536], [0, 354484224], [0, 65536], [0, 357629952], [0, 65536], [0, 354418688], [0, 65536], [0, 357564416], [0, 65536], [0, 358612992], [0, 65536], [0, 354549760], [0, 256], [0, 355598336], [0, 65536], [0, 357695488], [0, 256], [0, 358744064], [0, 256], [0, 353435648], [0, 256], [0, 353566720], [0, 256]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek,imgsys.yaml#
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250524115144.3832748-2-olivia.wen@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
@ 2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-25 4:55 ` Krzysztof Kozlowski
1 sibling, 0 replies; 16+ messages in thread
From: Rob Herring (Arm) @ 2025-05-24 13:37 UTC (permalink / raw)
To: Olivia Wen
Cc: teddy.chen, Krzysztof Kozlowski, linux-arm-kernel,
Mauro Carvalho Chehab, AngeloGioacchino Del Regno, linux-kernel,
linux-mediatek, Project_Global_Chrome_Upstream_Group,
Conor Dooley, devicetree, Matthias Brugger, linux-media, yunkec
On Sat, 24 May 2025 19:49:54 +0800, Olivia Wen wrote:
> This patch adds support for the MT8188 Image System's Local Arbiter
> (LARB) in the device tree bindings. The LARB is a crucial component in
> MediaTek's ImgSys architecture, responsible for managing memory access
> and arbitration between various hardware modules.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> .../bindings/media/mediatek,imgsys-larbs.yaml | 75 +++++++++++++++++++
> 1 file changed, 75 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors:
Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.example.dtb: /example-0/imgsys-l11b: failed to match any schema with compatible: ['mediatek,mt8188-imgsys-larb']
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250524115144.3832748-3-olivia.wen@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
@ 2025-05-25 4:55 ` Krzysztof Kozlowski
1 sibling, 0 replies; 16+ messages in thread
From: Krzysztof Kozlowski @ 2025-05-25 4:55 UTC (permalink / raw)
To: Olivia Wen, Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, linux-media, devicetree,
linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
On 24/05/2025 13:49, Olivia Wen wrote:
> This patch adds support for the MT8188 Image System's Local Arbiter
> (LARB) in the device tree bindings. The LARB is a crucial component in
> MediaTek's ImgSys architecture, responsible for managing memory access
> and arbitration between various hardware modules.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> .../bindings/media/mediatek,imgsys-larbs.yaml | 75 +++++++++++++++++++
Your patchset wasn't tested, so I won't do full review. Be sure you test
it prior sending. Few nits below:
Please use subject prefixes matching the subsystem. You can get them for
example with `git log --oneline -- DIRECTORY_OR_FILE` on the directory
your patch is touching. For bindings, the preferred subjects are
explained here:
https://www.kernel.org/doc/html/latest/devicetree/bindings/submitting-patches.html#i-for-patch-submitters
> 1 file changed, 75 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
> new file mode 100644
> index 000000000000..d2966c64ddb6
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
Filename equal to compatible.
> @@ -0,0 +1,75 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/mediatek,imgsys-larbs.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: LARB Information for MediaTek Camera Image System Hardware
> +
> +maintainers:
> + - Olivia Wen <olivia.wen@mediatek.com>
> +
> +description:
> + Detailed configuration for MediaTek Camera Image System's Local Arbiter
> + (LARB) hardware integration.
> +
> +properties:
> + compatible:
> + enum:
> + - mediatek,mt8188-imgsys-larbs
> +
> + iommus:
> + minItems: 1
> + maxItems: 30
This cannot be flexible. List the items, unless it is somehow obvious.
> +
> + '#address-cells':
> + const: 2
> +
> + '#size-cells':
> + const: 2
These are useless.
> +
> +required:
> + - compatible
> + - iommus
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
> +
> + imgsys-l11b {
Node names should be generic. See also an explanation and list of
examples (not exhaustive) in DT specification:
https://devicetree-specification.readthedocs.io/en/latest/chapter2-devicetree-basics.html#generic-names-recommendation
> + compatible = "mediatek,mt8188-imgsys-larb";
> + #address-cells = <2>;
> + #size-cells = <2>;
Why?
> + iommus = <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_0>,
> + <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_1>,
Fix alignment.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
@ 2025-05-26 2:13 ` CK Hu (胡俊光)
0 siblings, 0 replies; 16+ messages in thread
From: CK Hu (胡俊光) @ 2025-05-26 2:13 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, AngeloGioacchino Del Regno,
matthias.bgg@gmail.com, Olivia Wen (温倩苓)
Cc: linux-media@vger.kernel.org, Project_Global_Chrome_Upstream_Group,
devicetree@vger.kernel.org, linux-kernel@vger.kernel.org,
krzk+dt@kernel.org, linux-arm-kernel@lists.infradead.org,
Teddy Chen (陳乾元), conor+dt@kernel.org,
linux-mediatek@lists.infradead.org, yunkec@chromium.org
On Sat, 2025-05-24 at 19:49 +0800, Olivia Wen wrote:
> Add controls and formats for MediaTek Imgsys processing
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> include/uapi/linux/mtkisp_imgsys.h | 215 +++++++++++++++++++++++++++++
> 1 file changed, 215 insertions(+)
> create mode 100644 include/uapi/linux/mtkisp_imgsys.h
>
> diff --git a/include/uapi/linux/mtkisp_imgsys.h b/include/uapi/linux/mtkisp_imgsys.h
> new file mode 100644
> index 000000000000..c7528c8bf4a2
> --- /dev/null
> +++ b/include/uapi/linux/mtkisp_imgsys.h
> @@ -0,0 +1,215 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * MediaTek ISP imgsys User space API
> + *
> + * Copyright (c) 2024 MediaTek Inc.
> + */
> +
> +#ifndef _MTKISP_IMGSYS_USER_H
> +#define _MTKISP_IMGSYS_USER_H
> +
> +#include <linux/v4l2-controls.h>
> +
> +/******************************************************/
> +/* MediaTek ISP imgsys controls */
> +/******************************************************/
> +
> +/* The base for the MediaTek imgsys driver controls */
> +/* We reserve 48 controls for this driver. */
> +#define V4L2_CID_USER_MTK_IMG_BASE (V4L2_CID_USER_BASE + 0x1100)
> +
> +/**
> + * Define a control identifier, V4L2_CID_MTK_IMG_RESIZE_RATIO, for setting the
> + * image processing resize ratio for MediaTek chip. This identifier allows
> + * adjustment of the image resize ratio to accommodate various display or
> + * processing needs. ImgSys hardware supports multiple resizers, including
> + * any ratio, downscaling by factors of 2, 4, and 42, with NULL indicating
> + * the end of menu options.
> + */
> +#define V4L2_CID_MTK_IMG_RESIZE_RATIO (V4L2_CID_USER_MTK_IMG_BASE + 34)
> +
> +/******************************************************/
> +/* Vendor specific - MediaTek ISP formats */
> +/******************************************************/
> +
> +/* MediaTek warp map 32-bit, 2 plane */
> +#define V4L2_PIX_FMT_WARP2P v4l2_fourcc('M', 'W', '2', 'P')
> +/* YUV-10bit packed 4:2:0 2plane, (Y)(UV) */
Pixel format should define in include/uapi/linux/videodev2.h
Add document to describe the detail of V4L2_PIX_FMT_WARP2P. [1] is the example document for V4L2_META_FMT_RPI_FE_CFG.
There are many document in Documentation/userspace-api/media/v4l/ for your reference.
[1] https://patchwork.kernel.org/project/linux-media/patch/20241003-rp1-cfe-v6-1-d6762edd98a8@ideasonboard.com/
> +#define V4L2_PIX_FMT_YUV_2P010P v4l2_fourcc('U', '0', '2', 'A')
> +/* YUV-12bit packed 4:2:0 2plane, (Y)(UV) */
Ditto.
> +#define V4L2_PIX_FMT_YUV_2P012P v4l2_fourcc('U', '0', '2', 'C')
> +/* Y-32bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y32 v4l2_fourcc('M', 'T', '3', '2')
> +/* Y-16bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y16 v4l2_fourcc('M', 'T', '1', '6')
> +/* Y-8bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y8 v4l2_fourcc('M', 'T', '0', '8')
> +
> +/**
> + * Describes the MediaTek APLY8(V4L2_PIX_FMT_MTISP_APLY8) format, which includes
> + * Y-8bit image data and appends APL(Average Pixel Level) data utilized by the
> + * ME(motion estimation) hardware to the end of the image.
> + *
> + * The hardware ME generates motion vectors by processing two consecutive frames,
> + * referred to as the previous and current frames, to enhance graphic processing.
> + * The diagram is shown below.
> + *
> + * +------------------+
> + * the thumbnail of previous frame -> | |
> + * (include the L0 frame and L1 frame)| |
> + * | ME | -> motion vectors
> + * the thumbnail of current frame -> | |
> + * (include the L0 frame and L1 frame)| |
> + * +------------------+
> + *
> + * To accelerate computation, the two images will be resized into two different
> + * smaller images, referred to as L0 frame, L1 frame and APL data.
> + * The L1 frame is a scaled-down version of the L0 frame, reduced to one-fourth
> + * of its original size containing APL data.
> + *
> + * +------------------+
> + * previous L0 frame -> | |
> + * previous L1 frame -> | |
> + * (include APL data) | |
> + * | ME | -> motion vectors
> + * current L0 frame -> | |
> + * current L1 frame -> | |
> + * (include APL data) | |
> + * +------------------+
> + *
> + * Each frame size:
> + * the width of L0 frame (L0_wd) = 576 (fixed value)
> + * the height of L0 frame (L0_ht) =
> + * 16 pixels align((the width of original image * L0_wd) / the height of original image)
> + * the width of L1 frame (L1_wd) = L0_wd / 4
> + * the height of L1 frame (L1_ht) = L0_ht / 4
> + *
> + * Buffer size:
> + * line_size = width * bytes_per_pixel(1)
> + * stride with 16 bytes alignment = ((line_size + 15) / 16) * 16
> + * buffer_size = stride with 16 bytes alignment * height + APL data(16 bytes)
> + *
> + * And the memory layout should be followed as
> + *
> + * @code
> + * APLY8
> + * +------------------+
> + * | Y Plane |
> + * | Bit Stream |
> + * +------------------+
> + * | APL data |
> + * +------------------+
> + * @endcode
> + *
> + * @note The APL data size is 16 bytes.The caller SHOULD NOT modify it.
> + */
> +/* Y-8bit image data and append the APL data */
> +#define V4L2_PIX_FMT_MTISP_APLY8 v4l2_fourcc('M', 'T', 'A', '8')
Ditto.
> +
> +/* Packed 10-bit */
> +#define V4L2_PIX_FMT_MTISP_SBGGR10 v4l2_fourcc('M', 'B', 'B', 'A')
> +#define V4L2_PIX_FMT_MTISP_SGBRG10 v4l2_fourcc('M', 'B', 'G', 'A')
> +#define V4L2_PIX_FMT_MTISP_SGRBG10 v4l2_fourcc('M', 'B', 'g', 'A')
> +#define V4L2_PIX_FMT_MTISP_SRGGB10 v4l2_fourcc('M', 'B', 'R', 'A')
Ditto.
> +
> +/* Vendor specific - MediaTek ISP parameters for firmware */
> +/* ISP tuning parameters */
> +#define V4L2_META_FMT_MTISP_TUN_PARAMS v4l2_fourcc('M', 'T', 'f', 't')
> +/* ISP statistics parameters */
> +#define V4L2_META_FMT_MTISP_STAT_PARAMS v4l2_fourcc('M', 'T', 'f', 's')
> +/* ISP controlling parameters */
> +#define V4L2_META_FMT_MTISP_CTL_PARAMS v4l2_fourcc('M', 'T', 'f', 'c')
Ditto.
> +
> +/*****************************************************************************/
> +/* Define sizes for various formats used in the MediaTek image system driver */
> +/*****************************************************************************/
> +/**
> + * SIZE_OF_APL_DATA - Size of the APL (Average Pixel Level) data
> + *
> + * This constant defines the size of the APL data, which is 16 bytes.
> + * The APL data is appended to the image format defined by
> + * V4L2_PIX_FMT_MTISP_APLY8. This format includes 8-bit Y image data
> + * followed by the APL data, which is utilized by the motion estimation
> + * hardware for enhanced processing. The APL data provides important
> + * information about the average pixel levels, which can be critical
> + * for various image processing tasks.
> + */
> +#define SIZE_OF_APL_DATA 16 /* Size of APL data used by V4L2_PIX_FMT_MTISP_APLY8 */
> +
> +/**
> + * MediaTek Tuning Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_TUN_PARAMS.
> + * This buffer contains essential tuning information for the MediaTek camera,
> + * including:
> + * - Sensor tuning settings
Ditto.
> + * - Digital gain for adjusting image brightness
Ditto.
> + * - White balance gain
Ditto.
> + *
> + * Note: The individual fields of the tuning metadata structure are not exposed
> + * to the kernel. The kernel only requires the size of the tuning metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the tuning metadata without impacting the
> + * kernel's interface.
> + */
> + #define SIZE_OF_TUNING_META 219352
> +
> +/**
> + * MediaTek Control Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_CTL_PARAMS.
> + * The control metadata buffer aggregates control information for multiple
> + * image processing modules, including TRAW, DIP, PQDIP, ME, WPE, and ADL.
> + * It facilitates the configuration and management of these modules by
> + * encapsulating relevant parameters and settings required for processing
> + * image data efficiently. This structure serves as a central point for
> + * coordinating the operation of different hardware components in the image
> + * processing pipeline.
> + *
> + * Note: The individual fields of the control metadata structure are not exposed
> + * to the kernel. The kernel only requires the size of the control metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the control metadata without impacting the
> + * kernel's interface.
How user space to generate these control meta data?
Add document about how generating these control meta data.
> + */
> +#define SIZE_OF_COMMON_CTRL 688
> +#define SIZE_OF_WPE_CTRL 240
> +#define SIZE_OF_TRAW_CTRL 120
> +#define SIZE_OF_DIP_CTRL 112
> +#define SIZE_OF_PQDIP_CTRL 112
> +#define SIZE_OF_ME_CTRL 8
> +#define SIZE_OF_ADL_CTRL 64
> +#define SIZE_OF_YUFO_META_INFO_WPE 4228
> +#define SIZE_OF_YUFO_META_INFO_DIP 4228
> +
> +#define SIZE_OF_CTRL_META ( \
> + SIZE_OF_COMMON_CTRL + \
> + SIZE_OF_WPE_CTRL + \
> + SIZE_OF_TRAW_CTRL + \
> + SIZE_OF_DIP_CTRL + \
> + SIZE_OF_PQDIP_CTRL + \
> + SIZE_OF_ME_CTRL + \
> + SIZE_OF_ADL_CTRL + \
> + SIZE_OF_YUFO_META_INFO_WPE + \
> + SIZE_OF_YUFO_META_INFO_DIP \
> +)
> +
> +/**
> + * MediaTek Statistics Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_STAT_PARAMS.
> + * This buffer contains essential statistics information for the MediaTek camera,
> + * including:
> + * - Readiness and corresponding values for tone and contrast statistics
> + * - Motion estimation (ME) statistics
> + * - ME feature match blocks statistics
> + *
> + * Note: The individual fields of the statistics buffer structure are not exposed
> + * to the kernel. The kernel only requires the size of the statistics metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the statistics metadata without impacting the
> + * kernel's interface.
How user space to generate these statistics meta data?
Add document about how generating these statistics meta data.
Regards,
CK
> + */
> + #define SIZE_OF_STATISTICS_META 88
> +
> +#endif /* _MTKISP_IMGSYS_USER_H */
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver
2025-05-24 11:49 ` [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver Olivia Wen
@ 2025-05-26 7:16 ` CK Hu (胡俊光)
0 siblings, 0 replies; 16+ messages in thread
From: CK Hu (胡俊光) @ 2025-05-26 7:16 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, AngeloGioacchino Del Regno,
matthias.bgg@gmail.com, Olivia Wen (温倩苓)
Cc: linux-media@vger.kernel.org, Project_Global_Chrome_Upstream_Group,
devicetree@vger.kernel.org, linux-kernel@vger.kernel.org,
krzk+dt@kernel.org, linux-arm-kernel@lists.infradead.org,
Teddy Chen (陳乾元), conor+dt@kernel.org,
linux-mediatek@lists.infradead.org, yunkec@chromium.org
On Sat, 2025-05-24 at 19:49 +0800, Olivia Wen wrote:
> The ImgSys driver is implemented as a series of patches, with this patch
> focusing on the V4L2 flow. The MediaTek's Image System (ImgSys), also
> known as ISP Pass2. ImgSys is a memory-to-memory hardware device
> designed for advanced image processing tasks. It is composed of multiple
> hardware components, including TRAW, DIP, PQDIP, ME, and WPE.
>
> TRAW (Tile RAW):
> - Provides multiple downscaled resizers to support multi-scale noise
> reduction.
> - Supports RAW/RGB format conversion.
>
> DIP (Digital Image Processing) and PQDIP (Picture Quality DIP):
> - Supports such as temporal and spatial noise reduction (TNR/SNR),
> edge enhancement (EE), and sharpness (SHP).
> - PQDIP also supports image scaling and rotation.
>
> ME (Motion Estimation):
> - Supports motion estimation between two consecutive frames.
>
> WPE (Warping Engine):
> - Handles image warping and de-warping processes.
>
> The ImgSys driver, primarily utilized for Google Chromebook products,
> uses Multi-Frame Combined Noise Reduction technology to enhance image
> quality.
>
> Below is a simplified architecture of the ImgSys driver:
> User -> V4L2 Framework -> ImgSys Driver
> -> SCP (It packages the hardware settings into commands)
> -> ImgSys Driver
> -> CMDQ (The packaged commands are sent to the hardware via GCE)
> -> Hardware
You break this driver into multiple patches by these function block,
but this would make reviewer (include me) hard to review.
The better way is
1. Add imgsys driver with basic function (include v4l2 framework, scp, cmdq. The basic function may be downscale and format convert only)
2. Add DIP function
3. Add ME function
4. Add WPE function
Let the first patch as simple as possible but it should work.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> drivers/media/platform/mediatek/Kconfig | 1 +
> drivers/media/platform/mediatek/Makefile | 1 +
> drivers/media/platform/mediatek/isp/Kconfig | 23 +
> .../platform/mediatek/isp/isp_7x/Makefile | 6 +
> .../mediatek/isp/isp_7x/imgsys/Makefile | 11 +
> .../imgsys/modules/mtk_dip_v4l2_vnode.h | 594 ++++++
> .../isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h | 386 ++++
> .../imgsys/modules/mtk_pqdip_v4l2_vnode.h | 149 ++
> .../imgsys/modules/mtk_traw_v4l2_vnode.h | 371 ++++
> .../imgsys/modules/mtk_wpe_v4l2_vnode.h | 317 ++++
> .../isp/isp_7x/imgsys/mtk_header_desc.h | 28 +
> .../isp/isp_7x/imgsys/mtk_imgsys-dev.c | 238 +++
> .../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 427 +++++
> .../isp/isp_7x/imgsys/mtk_imgsys-formats.c | 139 ++
> .../isp/isp_7x/imgsys/mtk_imgsys-formats.h | 72 +
> .../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 27 +
> .../isp/isp_7x/imgsys/mtk_imgsys-of.c | 39 +
> .../isp/isp_7x/imgsys/mtk_imgsys-of.h | 55 +
> .../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 27 +
> .../isp/isp_7x/imgsys/mtk_imgsys-sys.h | 50 +
> .../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 1684 +++++++++++++++++
> .../isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h | 100 +
> .../isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h | 135 ++
> 23 files changed, 4880 insertions(+)
> create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h
Place these files in drivers/media/platform/mediatek/imgsys.
This is the first SoC which support imgsys driver so it's not necessary to use so deep folder structure.
Regards,
CK
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2025-05-26 7:16 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-25 4:55 ` Krzysztof Kozlowski
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
2025-05-26 2:13 ` CK Hu (胡俊光)
2025-05-24 11:49 ` [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver Olivia Wen
2025-05-26 7:16 ` CK Hu (胡俊光)
2025-05-24 11:49 ` [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device Olivia Wen
2025-05-24 11:49 ` [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys Olivia Wen
2025-05-24 11:49 ` [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver Olivia Wen
2025-05-24 11:50 ` [PATCH v1 08/10] media: mediatek: isp: Add SCP " Olivia Wen
2025-05-24 11:50 ` [PATCH v1 09/10] media: mediatek: isp: Add image processing flow Olivia Wen
2025-05-24 11:50 ` [PATCH v1 10/10] media: mediatek: isp: Add normal data dump flow Olivia Wen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).