* [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
` (8 subsequent siblings)
9 siblings, 1 reply; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
Introduce more Image System (ImgSys) components present in MT8188.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../bindings/media/mediatek,imgsys.yaml | 180 ++++++++++++++++++
1 file changed, 180 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek,imgsys.yaml b/Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
new file mode 100644
index 000000000000..c2899c4b227b
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
@@ -0,0 +1,180 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek,imgsys.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: MediaTek Camera Image System
+
+maintainers:
+ - Olivia Wen <olivia.wen@mediatek.com>
+
+description:
+ MediaTek Camera Image System is the image processing hardware present in MediaTek SoCs
+
+properties:
+ compatible:
+ enum:
+ - mediatek,mt8188-isp-imgsys
+
+ reg:
+ minItems: 1
+ maxItems: 17
+
+ clocks:
+ minItems: 1
+ maxItems: 17
+
+ clock-names:
+ minItems: 1
+ maxItems: 17
+
+ assigned-clocks: true
+
+ assigned-clock-parents: true
+
+ '#address-cells':
+ const: 2
+
+ '#size-cells':
+ const: 2
+
+ iommus:
+ minItems: 1
+ maxItems: 30
+
+ mediatek,larbs:
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ minItems: 1
+ maxItems: 32
+ items:
+ maxItems: 1
+ description: |
+ List of phandle to the local arbiters in the current Socs.
+ Refer to bindings/memory-controllers/mediatek,smi-larb.yaml. It must sort
+ according to the local arbiter index, like larb0, larb1, larb2...
+
+required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - iommus
+ - mediatek,larbs
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ enum:
+ - mediatek,mt8188-isp-imgsys
+
+ then:
+ required:
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+ - iommus
+ - mediatek,larbs
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/mediatek,mt8188-clk.h>
+ #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
+
+ imgsys_fw: imgsys-fw@15000000 {
+ compatible = "mediatek,mt8188-isp-imgsys";
+ reg = <0 0x15000000 0 0x4000>, /* 0 IMGSYS_TOP */
+ <0 0x15020000 0 0x10000>, /* 1 IMGSYS_TRAW */
+ <0 0x15040000 0 0x10000>, /* 2 IMGSYS_LTRAW */
+ <0 0x15640000 0 0x10000>, /* 3 IMGSYS_XTRAW */
+ <0 0x15100000 0 0x10000>, /* 4 IMGSYS_DIP */
+ <0 0x15150000 0 0x10000>, /* 5 IMGSYS_DIP_NR */
+ <0 0x15210000 0 0x10000>, /* 6 IMGSYS_PQDIP_A */
+ <0 0x15510000 0 0x10000>, /* 7 IMGSYS_PQDIP_B */
+ <0 0x15200000 0 0x10000>, /* 8 IMGSYS_WPE_EIS */
+ <0 0x15500000 0 0x10000>, /* 9 IMGSYS_WPE_TNR */
+ <0 0x15600000 0 0x10000>, /* 10 IMGSYS_WPE_LITE */
+ <0 0x15220000 0 0x00100>, /* 11 IMGSYS_WPE1_DIP1 */
+ <0 0x15320000 0 0x10000>, /* 12 IMGSYS_ME */
+ <0 0x15520000 0 0x00100>, /* 13 IMGSYS_WPE2_DIP1 */
+ <0 0x15620000 0 0x00100>, /* 14 IMGSYS_WPE3_DIP1 */
+ <0 0x15110000 0 0x00100>, /* 15 IMGSYS_DIP_TOP */
+ <0 0x15130000 0 0x00100>; /* 16 IMGSYS_DIP_TOP_NR */
+
+ assigned-clocks = <&topckgen CLK_TOP_IMG>;
+ assigned-clock-parents = <&apmixedsys CLK_APMIXED_IMGPLL>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ clocks = <&imgsys CLK_IMGSYS_MAIN_TRAW0>,
+ <&imgsys CLK_IMGSYS_MAIN_TRAW1>,
+ <&imgsys CLK_IMGSYS_MAIN_VCORE_GALS>,
+ <&imgsys CLK_IMGSYS_MAIN_DIP0>,
+ <&imgsys CLK_IMGSYS_MAIN_WPE0>,
+ <&imgsys CLK_IMGSYS_MAIN_WPE1>,
+ <&imgsys CLK_IMGSYS_MAIN_WPE2>,
+ <&imgsys CLK_IMGSYS_MAIN_GALS>,
+ <&imgsys1_dip_top CLK_IMGSYS1_DIP_TOP_DIP_TOP>,
+ <&imgsys1_dip_nr CLK_IMGSYS1_DIP_NR_DIP_NR>,
+ <&imgsys_wpe1 CLK_IMGSYS_WPE1>,
+ <&imgsys_wpe2 CLK_IMGSYS_WPE2>,
+ <&imgsys_wpe3 CLK_IMGSYS_WPE3>,
+ <&imgsys CLK_IMGSYS_MAIN_IPE>,
+ <&ipesys CLK_IPESYS_TOP>,
+ <&ipesys CLK_IPE_ME>,
+ <&ipesys CLK_IPE_SMI_LARB12>;
+ clock-names = "IMGSYS_CG_IMG_TRAW0",
+ "IMGSYS_CG_IMG_TRAW1",
+ "IMGSYS_CG_IMG_VCORE_GALS",
+ "IMGSYS_CG_IMG_DIP0",
+ "IMGSYS_CG_IMG_WPE0",
+ "IMGSYS_CG_IMG_WPE1",
+ "IMGSYS_CG_IMG_WPE2",
+ "IMGSYS_CG_IMG_GALS",
+ "DIP_TOP_DIP_TOP",
+ "DIP_NR_DIP_NR",
+ "WPE1_CG_DIP1_WPE",
+ "WPE2_CG_DIP1_WPE",
+ "WPE3_CG_DIP1_WPE",
+ "ME_CG_IPE",
+ "ME_CG_IPE_TOP",
+ "ME_CG",
+ "ME_CG_LARB12";
+
+ iommus = <&vdo_iommu M4U_PORT_L9_IMGI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_UFDI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_IMGBI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_IMGCI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTI_T4_A>,
+ <&vdo_iommu M4U_PORT_L9_TNCSTI_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_TNCSTI_T4_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVBO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVCO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_TIMGO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T2_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T5_A>,
+ <&vdo_iommu M4U_PORT_L9_IMGI_T1_B>,
+ <&vdo_iommu M4U_PORT_L9_IMGBI_T1_B>,
+ <&vdo_iommu M4U_PORT_L9_IMGCI_T1_B>,
+ <&vdo_iommu M4U_PORT_L9_SMTI_T4_B>,
+ <&vdo_iommu M4U_PORT_L9_TNCSO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_SMTO_T4_A>,
+ <&vdo_iommu M4U_PORT_L9_TNCSTO_T1_A>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T2_B>,
+ <&vdo_iommu M4U_PORT_L9_YUVO_T5_B>,
+ <&vdo_iommu M4U_PORT_L9_SMTO_T4_B>;
+
+ mediatek,larbs = <&larb9>,
+ <&larb10>,
+ <&larb11a>,
+ <&larb11b>,
+ <&larb11c>,
+ <&larb15>,
+ <&larb12>;
+ };
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
@ 2025-05-24 13:37 ` Rob Herring (Arm)
0 siblings, 0 replies; 15+ messages in thread
From: Rob Herring (Arm) @ 2025-05-24 13:37 UTC (permalink / raw)
To: Olivia Wen
Cc: teddy.chen, linux-arm-kernel, yunkec, Mauro Carvalho Chehab,
Conor Dooley, linux-mediatek, devicetree, Matthias Brugger,
Project_Global_Chrome_Upstream_Group, linux-media,
Krzysztof Kozlowski, AngeloGioacchino Del Regno, linux-kernel
On Sat, 24 May 2025 19:49:53 +0800, Olivia Wen wrote:
> Introduce more Image System (ImgSys) components present in MT8188.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> .../bindings/media/mediatek,imgsys.yaml | 180 ++++++++++++++++++
> 1 file changed, 180 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek,imgsys.example.dtb: imgsys-fw@15000000 (mediatek,mt8188-isp-imgsys): reg: [[0, 352321536], [0, 16384], [0, 352452608], [0, 65536], [0, 352583680], [0, 65536], [0, 358875136], [0, 65536], [0, 353370112], [0, 65536], [0, 353697792], [0, 65536], [0, 354484224], [0, 65536], [0, 357629952], [0, 65536], [0, 354418688], [0, 65536], [0, 357564416], [0, 65536], [0, 358612992], [0, 65536], [0, 354549760], [0, 256], [0, 355598336], [0, 65536], [0, 357695488], [0, 256], [0, 358744064], [0, 256], [0, 353435648], [0, 256], [0, 353566720], [0, 256]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek,imgsys.yaml#
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250524115144.3832748-2-olivia.wen@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-25 4:55 ` Krzysztof Kozlowski
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
` (7 subsequent siblings)
9 siblings, 2 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
This patch adds support for the MT8188 Image System's Local Arbiter
(LARB) in the device tree bindings. The LARB is a crucial component in
MediaTek's ImgSys architecture, responsible for managing memory access
and arbitration between various hardware modules.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../bindings/media/mediatek,imgsys-larbs.yaml | 75 +++++++++++++++++++
1 file changed, 75 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
new file mode 100644
index 000000000000..d2966c64ddb6
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
@@ -0,0 +1,75 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek,imgsys-larbs.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: LARB Information for MediaTek Camera Image System Hardware
+
+maintainers:
+ - Olivia Wen <olivia.wen@mediatek.com>
+
+description:
+ Detailed configuration for MediaTek Camera Image System's Local Arbiter
+ (LARB) hardware integration.
+
+properties:
+ compatible:
+ enum:
+ - mediatek,mt8188-imgsys-larbs
+
+ iommus:
+ minItems: 1
+ maxItems: 30
+
+ '#address-cells':
+ const: 2
+
+ '#size-cells':
+ const: 2
+
+required:
+ - compatible
+ - iommus
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
+
+ imgsys-l11b {
+ compatible = "mediatek,mt8188-imgsys-larb";
+ #address-cells = <2>;
+ #size-cells = <2>;
+ iommus = <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_1>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_4P_0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_4P_1>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_CQ0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_CQ1>,
+ <&vpp_iommu M4U_PORT_L11B_PIMGI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_PIMGBI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_PIMGCI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_IMGI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_IMGBI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_IMGCI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTI_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTI_T4_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTI_T6_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVBO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVCO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_WDMA_0>,
+ <&vpp_iommu M4U_PORT_L11B_WPE_WDMA_4P_0>,
+ <&vpp_iommu M4U_PORT_L11B_WROT_P1>,
+ <&vpp_iommu M4U_PORT_L11B_TCCSO_P1>,
+ <&vpp_iommu M4U_PORT_L11B_TCCSI_P1>,
+ <&vpp_iommu M4U_PORT_L11B_TIMGO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVO_T2_C>,
+ <&vpp_iommu M4U_PORT_L11B_YUVO_T5_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTO_T1_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTO_T4_C>,
+ <&vpp_iommu M4U_PORT_L11B_SMTO_T6_C>,
+ <&vpp_iommu M4U_PORT_L11B_DBGO_T1_C>;
+ };
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
@ 2025-05-24 13:37 ` Rob Herring (Arm)
2025-05-25 4:55 ` Krzysztof Kozlowski
1 sibling, 0 replies; 15+ messages in thread
From: Rob Herring (Arm) @ 2025-05-24 13:37 UTC (permalink / raw)
To: Olivia Wen
Cc: teddy.chen, Krzysztof Kozlowski, linux-arm-kernel,
Mauro Carvalho Chehab, AngeloGioacchino Del Regno, linux-kernel,
linux-mediatek, Project_Global_Chrome_Upstream_Group,
Conor Dooley, devicetree, Matthias Brugger, linux-media, yunkec
On Sat, 24 May 2025 19:49:54 +0800, Olivia Wen wrote:
> This patch adds support for the MT8188 Image System's Local Arbiter
> (LARB) in the device tree bindings. The LARB is a crucial component in
> MediaTek's ImgSys architecture, responsible for managing memory access
> and arbitration between various hardware modules.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> .../bindings/media/mediatek,imgsys-larbs.yaml | 75 +++++++++++++++++++
> 1 file changed, 75 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors:
Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.example.dtb: /example-0/imgsys-l11b: failed to match any schema with compatible: ['mediatek,mt8188-imgsys-larb']
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250524115144.3832748-3-olivia.wen@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
2025-05-24 13:37 ` Rob Herring (Arm)
@ 2025-05-25 4:55 ` Krzysztof Kozlowski
1 sibling, 0 replies; 15+ messages in thread
From: Krzysztof Kozlowski @ 2025-05-25 4:55 UTC (permalink / raw)
To: Olivia Wen, Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, linux-media, devicetree,
linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
On 24/05/2025 13:49, Olivia Wen wrote:
> This patch adds support for the MT8188 Image System's Local Arbiter
> (LARB) in the device tree bindings. The LARB is a crucial component in
> MediaTek's ImgSys architecture, responsible for managing memory access
> and arbitration between various hardware modules.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> .../bindings/media/mediatek,imgsys-larbs.yaml | 75 +++++++++++++++++++
Your patchset wasn't tested, so I won't do full review. Be sure you test
it prior sending. Few nits below:
Please use subject prefixes matching the subsystem. You can get them for
example with `git log --oneline -- DIRECTORY_OR_FILE` on the directory
your patch is touching. For bindings, the preferred subjects are
explained here:
https://www.kernel.org/doc/html/latest/devicetree/bindings/submitting-patches.html#i-for-patch-submitters
> 1 file changed, 75 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
> new file mode 100644
> index 000000000000..d2966c64ddb6
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek,imgsys-larbs.yaml
Filename equal to compatible.
> @@ -0,0 +1,75 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/mediatek,imgsys-larbs.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: LARB Information for MediaTek Camera Image System Hardware
> +
> +maintainers:
> + - Olivia Wen <olivia.wen@mediatek.com>
> +
> +description:
> + Detailed configuration for MediaTek Camera Image System's Local Arbiter
> + (LARB) hardware integration.
> +
> +properties:
> + compatible:
> + enum:
> + - mediatek,mt8188-imgsys-larbs
> +
> + iommus:
> + minItems: 1
> + maxItems: 30
This cannot be flexible. List the items, unless it is somehow obvious.
> +
> + '#address-cells':
> + const: 2
> +
> + '#size-cells':
> + const: 2
These are useless.
> +
> +required:
> + - compatible
> + - iommus
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
> +
> + imgsys-l11b {
Node names should be generic. See also an explanation and list of
examples (not exhaustive) in DT specification:
https://devicetree-specification.readthedocs.io/en/latest/chapter2-devicetree-basics.html#generic-names-recommendation
> + compatible = "mediatek,mt8188-imgsys-larb";
> + #address-cells = <2>;
> + #size-cells = <2>;
Why?
> + iommus = <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_0>,
> + <&vpp_iommu M4U_PORT_L11B_WPE_RDMA_1>,
Fix alignment.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
2025-05-24 11:49 ` [PATCH v1 01/10] dt-bindings: media: Add MediaTek mt8188 ImgSys components Olivia Wen
2025-05-24 11:49 ` [PATCH v1 02/10] dt-bindings: media: Add MT8188 ImgSys's LARB Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-26 2:13 ` CK Hu (胡俊光)
2025-05-24 11:49 ` [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device Olivia Wen
` (6 subsequent siblings)
9 siblings, 1 reply; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
Add controls and formats for MediaTek Imgsys processing
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
include/uapi/linux/mtkisp_imgsys.h | 215 +++++++++++++++++++++++++++++
1 file changed, 215 insertions(+)
create mode 100644 include/uapi/linux/mtkisp_imgsys.h
diff --git a/include/uapi/linux/mtkisp_imgsys.h b/include/uapi/linux/mtkisp_imgsys.h
new file mode 100644
index 000000000000..c7528c8bf4a2
--- /dev/null
+++ b/include/uapi/linux/mtkisp_imgsys.h
@@ -0,0 +1,215 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * MediaTek ISP imgsys User space API
+ *
+ * Copyright (c) 2024 MediaTek Inc.
+ */
+
+#ifndef _MTKISP_IMGSYS_USER_H
+#define _MTKISP_IMGSYS_USER_H
+
+#include <linux/v4l2-controls.h>
+
+/******************************************************/
+/* MediaTek ISP imgsys controls */
+/******************************************************/
+
+/* The base for the MediaTek imgsys driver controls */
+/* We reserve 48 controls for this driver. */
+#define V4L2_CID_USER_MTK_IMG_BASE (V4L2_CID_USER_BASE + 0x1100)
+
+/**
+ * Define a control identifier, V4L2_CID_MTK_IMG_RESIZE_RATIO, for setting the
+ * image processing resize ratio for MediaTek chip. This identifier allows
+ * adjustment of the image resize ratio to accommodate various display or
+ * processing needs. ImgSys hardware supports multiple resizers, including
+ * any ratio, downscaling by factors of 2, 4, and 42, with NULL indicating
+ * the end of menu options.
+ */
+#define V4L2_CID_MTK_IMG_RESIZE_RATIO (V4L2_CID_USER_MTK_IMG_BASE + 34)
+
+/******************************************************/
+/* Vendor specific - MediaTek ISP formats */
+/******************************************************/
+
+/* MediaTek warp map 32-bit, 2 plane */
+#define V4L2_PIX_FMT_WARP2P v4l2_fourcc('M', 'W', '2', 'P')
+/* YUV-10bit packed 4:2:0 2plane, (Y)(UV) */
+#define V4L2_PIX_FMT_YUV_2P010P v4l2_fourcc('U', '0', '2', 'A')
+/* YUV-12bit packed 4:2:0 2plane, (Y)(UV) */
+#define V4L2_PIX_FMT_YUV_2P012P v4l2_fourcc('U', '0', '2', 'C')
+/* Y-32bit */
+#define V4L2_PIX_FMT_MTISP_Y32 v4l2_fourcc('M', 'T', '3', '2')
+/* Y-16bit */
+#define V4L2_PIX_FMT_MTISP_Y16 v4l2_fourcc('M', 'T', '1', '6')
+/* Y-8bit */
+#define V4L2_PIX_FMT_MTISP_Y8 v4l2_fourcc('M', 'T', '0', '8')
+
+/**
+ * Describes the MediaTek APLY8(V4L2_PIX_FMT_MTISP_APLY8) format, which includes
+ * Y-8bit image data and appends APL(Average Pixel Level) data utilized by the
+ * ME(motion estimation) hardware to the end of the image.
+ *
+ * The hardware ME generates motion vectors by processing two consecutive frames,
+ * referred to as the previous and current frames, to enhance graphic processing.
+ * The diagram is shown below.
+ *
+ * +------------------+
+ * the thumbnail of previous frame -> | |
+ * (include the L0 frame and L1 frame)| |
+ * | ME | -> motion vectors
+ * the thumbnail of current frame -> | |
+ * (include the L0 frame and L1 frame)| |
+ * +------------------+
+ *
+ * To accelerate computation, the two images will be resized into two different
+ * smaller images, referred to as L0 frame, L1 frame and APL data.
+ * The L1 frame is a scaled-down version of the L0 frame, reduced to one-fourth
+ * of its original size containing APL data.
+ *
+ * +------------------+
+ * previous L0 frame -> | |
+ * previous L1 frame -> | |
+ * (include APL data) | |
+ * | ME | -> motion vectors
+ * current L0 frame -> | |
+ * current L1 frame -> | |
+ * (include APL data) | |
+ * +------------------+
+ *
+ * Each frame size:
+ * the width of L0 frame (L0_wd) = 576 (fixed value)
+ * the height of L0 frame (L0_ht) =
+ * 16 pixels align((the width of original image * L0_wd) / the height of original image)
+ * the width of L1 frame (L1_wd) = L0_wd / 4
+ * the height of L1 frame (L1_ht) = L0_ht / 4
+ *
+ * Buffer size:
+ * line_size = width * bytes_per_pixel(1)
+ * stride with 16 bytes alignment = ((line_size + 15) / 16) * 16
+ * buffer_size = stride with 16 bytes alignment * height + APL data(16 bytes)
+ *
+ * And the memory layout should be followed as
+ *
+ * @code
+ * APLY8
+ * +------------------+
+ * | Y Plane |
+ * | Bit Stream |
+ * +------------------+
+ * | APL data |
+ * +------------------+
+ * @endcode
+ *
+ * @note The APL data size is 16 bytes.The caller SHOULD NOT modify it.
+ */
+/* Y-8bit image data and append the APL data */
+#define V4L2_PIX_FMT_MTISP_APLY8 v4l2_fourcc('M', 'T', 'A', '8')
+
+/* Packed 10-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR10 v4l2_fourcc('M', 'B', 'B', 'A')
+#define V4L2_PIX_FMT_MTISP_SGBRG10 v4l2_fourcc('M', 'B', 'G', 'A')
+#define V4L2_PIX_FMT_MTISP_SGRBG10 v4l2_fourcc('M', 'B', 'g', 'A')
+#define V4L2_PIX_FMT_MTISP_SRGGB10 v4l2_fourcc('M', 'B', 'R', 'A')
+
+/* Vendor specific - MediaTek ISP parameters for firmware */
+/* ISP tuning parameters */
+#define V4L2_META_FMT_MTISP_TUN_PARAMS v4l2_fourcc('M', 'T', 'f', 't')
+/* ISP statistics parameters */
+#define V4L2_META_FMT_MTISP_STAT_PARAMS v4l2_fourcc('M', 'T', 'f', 's')
+/* ISP controlling parameters */
+#define V4L2_META_FMT_MTISP_CTL_PARAMS v4l2_fourcc('M', 'T', 'f', 'c')
+
+/*****************************************************************************/
+/* Define sizes for various formats used in the MediaTek image system driver */
+/*****************************************************************************/
+/**
+ * SIZE_OF_APL_DATA - Size of the APL (Average Pixel Level) data
+ *
+ * This constant defines the size of the APL data, which is 16 bytes.
+ * The APL data is appended to the image format defined by
+ * V4L2_PIX_FMT_MTISP_APLY8. This format includes 8-bit Y image data
+ * followed by the APL data, which is utilized by the motion estimation
+ * hardware for enhanced processing. The APL data provides important
+ * information about the average pixel levels, which can be critical
+ * for various image processing tasks.
+ */
+#define SIZE_OF_APL_DATA 16 /* Size of APL data used by V4L2_PIX_FMT_MTISP_APLY8 */
+
+/**
+ * MediaTek Tuning Metadata Buffer for Image Processing Modules
+ *
+ * This definition is used for format V4L2_META_FMT_MTISP_TUN_PARAMS.
+ * This buffer contains essential tuning information for the MediaTek camera,
+ * including:
+ * - Sensor tuning settings
+ * - Digital gain for adjusting image brightness
+ * - White balance gain
+ *
+ * Note: The individual fields of the tuning metadata structure are not exposed
+ * to the kernel. The kernel only requires the size of the tuning metadata
+ * for memory allocation and management purposes. This encapsulation provides
+ * flexibility in implementing the tuning metadata without impacting the
+ * kernel's interface.
+ */
+ #define SIZE_OF_TUNING_META 219352
+
+/**
+ * MediaTek Control Metadata Buffer for Image Processing Modules
+ *
+ * This definition is used for format V4L2_META_FMT_MTISP_CTL_PARAMS.
+ * The control metadata buffer aggregates control information for multiple
+ * image processing modules, including TRAW, DIP, PQDIP, ME, WPE, and ADL.
+ * It facilitates the configuration and management of these modules by
+ * encapsulating relevant parameters and settings required for processing
+ * image data efficiently. This structure serves as a central point for
+ * coordinating the operation of different hardware components in the image
+ * processing pipeline.
+ *
+ * Note: The individual fields of the control metadata structure are not exposed
+ * to the kernel. The kernel only requires the size of the control metadata
+ * for memory allocation and management purposes. This encapsulation provides
+ * flexibility in implementing the control metadata without impacting the
+ * kernel's interface.
+ */
+#define SIZE_OF_COMMON_CTRL 688
+#define SIZE_OF_WPE_CTRL 240
+#define SIZE_OF_TRAW_CTRL 120
+#define SIZE_OF_DIP_CTRL 112
+#define SIZE_OF_PQDIP_CTRL 112
+#define SIZE_OF_ME_CTRL 8
+#define SIZE_OF_ADL_CTRL 64
+#define SIZE_OF_YUFO_META_INFO_WPE 4228
+#define SIZE_OF_YUFO_META_INFO_DIP 4228
+
+#define SIZE_OF_CTRL_META ( \
+ SIZE_OF_COMMON_CTRL + \
+ SIZE_OF_WPE_CTRL + \
+ SIZE_OF_TRAW_CTRL + \
+ SIZE_OF_DIP_CTRL + \
+ SIZE_OF_PQDIP_CTRL + \
+ SIZE_OF_ME_CTRL + \
+ SIZE_OF_ADL_CTRL + \
+ SIZE_OF_YUFO_META_INFO_WPE + \
+ SIZE_OF_YUFO_META_INFO_DIP \
+)
+
+/**
+ * MediaTek Statistics Metadata Buffer for Image Processing Modules
+ *
+ * This definition is used for format V4L2_META_FMT_MTISP_STAT_PARAMS.
+ * This buffer contains essential statistics information for the MediaTek camera,
+ * including:
+ * - Readiness and corresponding values for tone and contrast statistics
+ * - Motion estimation (ME) statistics
+ * - ME feature match blocks statistics
+ *
+ * Note: The individual fields of the statistics buffer structure are not exposed
+ * to the kernel. The kernel only requires the size of the statistics metadata
+ * for memory allocation and management purposes. This encapsulation provides
+ * flexibility in implementing the statistics metadata without impacting the
+ * kernel's interface.
+ */
+ #define SIZE_OF_STATISTICS_META 88
+
+#endif /* _MTKISP_IMGSYS_USER_H */
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
@ 2025-05-26 2:13 ` CK Hu (胡俊光)
0 siblings, 0 replies; 15+ messages in thread
From: CK Hu (胡俊光) @ 2025-05-26 2:13 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, AngeloGioacchino Del Regno,
matthias.bgg@gmail.com, Olivia Wen (温倩苓)
Cc: linux-media@vger.kernel.org, Project_Global_Chrome_Upstream_Group,
devicetree@vger.kernel.org, linux-kernel@vger.kernel.org,
krzk+dt@kernel.org, linux-arm-kernel@lists.infradead.org,
Teddy Chen (陳乾元), conor+dt@kernel.org,
linux-mediatek@lists.infradead.org, yunkec@chromium.org
On Sat, 2025-05-24 at 19:49 +0800, Olivia Wen wrote:
> Add controls and formats for MediaTek Imgsys processing
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> include/uapi/linux/mtkisp_imgsys.h | 215 +++++++++++++++++++++++++++++
> 1 file changed, 215 insertions(+)
> create mode 100644 include/uapi/linux/mtkisp_imgsys.h
>
> diff --git a/include/uapi/linux/mtkisp_imgsys.h b/include/uapi/linux/mtkisp_imgsys.h
> new file mode 100644
> index 000000000000..c7528c8bf4a2
> --- /dev/null
> +++ b/include/uapi/linux/mtkisp_imgsys.h
> @@ -0,0 +1,215 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * MediaTek ISP imgsys User space API
> + *
> + * Copyright (c) 2024 MediaTek Inc.
> + */
> +
> +#ifndef _MTKISP_IMGSYS_USER_H
> +#define _MTKISP_IMGSYS_USER_H
> +
> +#include <linux/v4l2-controls.h>
> +
> +/******************************************************/
> +/* MediaTek ISP imgsys controls */
> +/******************************************************/
> +
> +/* The base for the MediaTek imgsys driver controls */
> +/* We reserve 48 controls for this driver. */
> +#define V4L2_CID_USER_MTK_IMG_BASE (V4L2_CID_USER_BASE + 0x1100)
> +
> +/**
> + * Define a control identifier, V4L2_CID_MTK_IMG_RESIZE_RATIO, for setting the
> + * image processing resize ratio for MediaTek chip. This identifier allows
> + * adjustment of the image resize ratio to accommodate various display or
> + * processing needs. ImgSys hardware supports multiple resizers, including
> + * any ratio, downscaling by factors of 2, 4, and 42, with NULL indicating
> + * the end of menu options.
> + */
> +#define V4L2_CID_MTK_IMG_RESIZE_RATIO (V4L2_CID_USER_MTK_IMG_BASE + 34)
> +
> +/******************************************************/
> +/* Vendor specific - MediaTek ISP formats */
> +/******************************************************/
> +
> +/* MediaTek warp map 32-bit, 2 plane */
> +#define V4L2_PIX_FMT_WARP2P v4l2_fourcc('M', 'W', '2', 'P')
> +/* YUV-10bit packed 4:2:0 2plane, (Y)(UV) */
Pixel format should define in include/uapi/linux/videodev2.h
Add document to describe the detail of V4L2_PIX_FMT_WARP2P. [1] is the example document for V4L2_META_FMT_RPI_FE_CFG.
There are many document in Documentation/userspace-api/media/v4l/ for your reference.
[1] https://patchwork.kernel.org/project/linux-media/patch/20241003-rp1-cfe-v6-1-d6762edd98a8@ideasonboard.com/
> +#define V4L2_PIX_FMT_YUV_2P010P v4l2_fourcc('U', '0', '2', 'A')
> +/* YUV-12bit packed 4:2:0 2plane, (Y)(UV) */
Ditto.
> +#define V4L2_PIX_FMT_YUV_2P012P v4l2_fourcc('U', '0', '2', 'C')
> +/* Y-32bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y32 v4l2_fourcc('M', 'T', '3', '2')
> +/* Y-16bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y16 v4l2_fourcc('M', 'T', '1', '6')
> +/* Y-8bit */
Ditto.
> +#define V4L2_PIX_FMT_MTISP_Y8 v4l2_fourcc('M', 'T', '0', '8')
> +
> +/**
> + * Describes the MediaTek APLY8(V4L2_PIX_FMT_MTISP_APLY8) format, which includes
> + * Y-8bit image data and appends APL(Average Pixel Level) data utilized by the
> + * ME(motion estimation) hardware to the end of the image.
> + *
> + * The hardware ME generates motion vectors by processing two consecutive frames,
> + * referred to as the previous and current frames, to enhance graphic processing.
> + * The diagram is shown below.
> + *
> + * +------------------+
> + * the thumbnail of previous frame -> | |
> + * (include the L0 frame and L1 frame)| |
> + * | ME | -> motion vectors
> + * the thumbnail of current frame -> | |
> + * (include the L0 frame and L1 frame)| |
> + * +------------------+
> + *
> + * To accelerate computation, the two images will be resized into two different
> + * smaller images, referred to as L0 frame, L1 frame and APL data.
> + * The L1 frame is a scaled-down version of the L0 frame, reduced to one-fourth
> + * of its original size containing APL data.
> + *
> + * +------------------+
> + * previous L0 frame -> | |
> + * previous L1 frame -> | |
> + * (include APL data) | |
> + * | ME | -> motion vectors
> + * current L0 frame -> | |
> + * current L1 frame -> | |
> + * (include APL data) | |
> + * +------------------+
> + *
> + * Each frame size:
> + * the width of L0 frame (L0_wd) = 576 (fixed value)
> + * the height of L0 frame (L0_ht) =
> + * 16 pixels align((the width of original image * L0_wd) / the height of original image)
> + * the width of L1 frame (L1_wd) = L0_wd / 4
> + * the height of L1 frame (L1_ht) = L0_ht / 4
> + *
> + * Buffer size:
> + * line_size = width * bytes_per_pixel(1)
> + * stride with 16 bytes alignment = ((line_size + 15) / 16) * 16
> + * buffer_size = stride with 16 bytes alignment * height + APL data(16 bytes)
> + *
> + * And the memory layout should be followed as
> + *
> + * @code
> + * APLY8
> + * +------------------+
> + * | Y Plane |
> + * | Bit Stream |
> + * +------------------+
> + * | APL data |
> + * +------------------+
> + * @endcode
> + *
> + * @note The APL data size is 16 bytes.The caller SHOULD NOT modify it.
> + */
> +/* Y-8bit image data and append the APL data */
> +#define V4L2_PIX_FMT_MTISP_APLY8 v4l2_fourcc('M', 'T', 'A', '8')
Ditto.
> +
> +/* Packed 10-bit */
> +#define V4L2_PIX_FMT_MTISP_SBGGR10 v4l2_fourcc('M', 'B', 'B', 'A')
> +#define V4L2_PIX_FMT_MTISP_SGBRG10 v4l2_fourcc('M', 'B', 'G', 'A')
> +#define V4L2_PIX_FMT_MTISP_SGRBG10 v4l2_fourcc('M', 'B', 'g', 'A')
> +#define V4L2_PIX_FMT_MTISP_SRGGB10 v4l2_fourcc('M', 'B', 'R', 'A')
Ditto.
> +
> +/* Vendor specific - MediaTek ISP parameters for firmware */
> +/* ISP tuning parameters */
> +#define V4L2_META_FMT_MTISP_TUN_PARAMS v4l2_fourcc('M', 'T', 'f', 't')
> +/* ISP statistics parameters */
> +#define V4L2_META_FMT_MTISP_STAT_PARAMS v4l2_fourcc('M', 'T', 'f', 's')
> +/* ISP controlling parameters */
> +#define V4L2_META_FMT_MTISP_CTL_PARAMS v4l2_fourcc('M', 'T', 'f', 'c')
Ditto.
> +
> +/*****************************************************************************/
> +/* Define sizes for various formats used in the MediaTek image system driver */
> +/*****************************************************************************/
> +/**
> + * SIZE_OF_APL_DATA - Size of the APL (Average Pixel Level) data
> + *
> + * This constant defines the size of the APL data, which is 16 bytes.
> + * The APL data is appended to the image format defined by
> + * V4L2_PIX_FMT_MTISP_APLY8. This format includes 8-bit Y image data
> + * followed by the APL data, which is utilized by the motion estimation
> + * hardware for enhanced processing. The APL data provides important
> + * information about the average pixel levels, which can be critical
> + * for various image processing tasks.
> + */
> +#define SIZE_OF_APL_DATA 16 /* Size of APL data used by V4L2_PIX_FMT_MTISP_APLY8 */
> +
> +/**
> + * MediaTek Tuning Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_TUN_PARAMS.
> + * This buffer contains essential tuning information for the MediaTek camera,
> + * including:
> + * - Sensor tuning settings
Ditto.
> + * - Digital gain for adjusting image brightness
Ditto.
> + * - White balance gain
Ditto.
> + *
> + * Note: The individual fields of the tuning metadata structure are not exposed
> + * to the kernel. The kernel only requires the size of the tuning metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the tuning metadata without impacting the
> + * kernel's interface.
> + */
> + #define SIZE_OF_TUNING_META 219352
> +
> +/**
> + * MediaTek Control Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_CTL_PARAMS.
> + * The control metadata buffer aggregates control information for multiple
> + * image processing modules, including TRAW, DIP, PQDIP, ME, WPE, and ADL.
> + * It facilitates the configuration and management of these modules by
> + * encapsulating relevant parameters and settings required for processing
> + * image data efficiently. This structure serves as a central point for
> + * coordinating the operation of different hardware components in the image
> + * processing pipeline.
> + *
> + * Note: The individual fields of the control metadata structure are not exposed
> + * to the kernel. The kernel only requires the size of the control metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the control metadata without impacting the
> + * kernel's interface.
How user space to generate these control meta data?
Add document about how generating these control meta data.
> + */
> +#define SIZE_OF_COMMON_CTRL 688
> +#define SIZE_OF_WPE_CTRL 240
> +#define SIZE_OF_TRAW_CTRL 120
> +#define SIZE_OF_DIP_CTRL 112
> +#define SIZE_OF_PQDIP_CTRL 112
> +#define SIZE_OF_ME_CTRL 8
> +#define SIZE_OF_ADL_CTRL 64
> +#define SIZE_OF_YUFO_META_INFO_WPE 4228
> +#define SIZE_OF_YUFO_META_INFO_DIP 4228
> +
> +#define SIZE_OF_CTRL_META ( \
> + SIZE_OF_COMMON_CTRL + \
> + SIZE_OF_WPE_CTRL + \
> + SIZE_OF_TRAW_CTRL + \
> + SIZE_OF_DIP_CTRL + \
> + SIZE_OF_PQDIP_CTRL + \
> + SIZE_OF_ME_CTRL + \
> + SIZE_OF_ADL_CTRL + \
> + SIZE_OF_YUFO_META_INFO_WPE + \
> + SIZE_OF_YUFO_META_INFO_DIP \
> +)
> +
> +/**
> + * MediaTek Statistics Metadata Buffer for Image Processing Modules
> + *
> + * This definition is used for format V4L2_META_FMT_MTISP_STAT_PARAMS.
> + * This buffer contains essential statistics information for the MediaTek camera,
> + * including:
> + * - Readiness and corresponding values for tone and contrast statistics
> + * - Motion estimation (ME) statistics
> + * - ME feature match blocks statistics
> + *
> + * Note: The individual fields of the statistics buffer structure are not exposed
> + * to the kernel. The kernel only requires the size of the statistics metadata
> + * for memory allocation and management purposes. This encapsulation provides
> + * flexibility in implementing the statistics metadata without impacting the
> + * kernel's interface.
How user space to generate these statistics meta data?
Add document about how generating these statistics meta data.
Regards,
CK
> + */
> + #define SIZE_OF_STATISTICS_META 88
> +
> +#endif /* _MTKISP_IMGSYS_USER_H */
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (2 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 03/10] uapi: linux: Add MediaTek Imgsys user API Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 11:49 ` [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys Olivia Wen
` (5 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the Imgsys LARBs device. This patch adds an independent
device driver for the ImgSys Local Arbiter (LARB) devices. The addition
is necessary because the IOMMU (Input/Output Memory Management Unit)
cannot support multiple LARBs within a single device node.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 61 +++++++++++++++++++
1 file changed, 61 insertions(+)
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index cfd65858c751..919c69493bbc 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -28,6 +28,58 @@
#define IMGSYS_MAX_BUFFERS 256
#define IMGSYS_SUSPEND_TIME 3000 /* ms */
+struct mtk_imgsys_larb_device {
+ struct device *dev;
+ struct mtk_imgsys_dev *imgsys_dev;
+};
+
+static int mtk_imgsys_larb_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct mtk_imgsys_larb_device *larb_dev;
+
+ larb_dev = devm_kzalloc(dev, sizeof(*dev), GFP_KERNEL);
+ if (!larb_dev)
+ return -ENOMEM;
+
+ if (dma_set_mask_and_coherent(dev, DMA_BIT_MASK(34)))
+ dev_err(dev, "No suitable DMA available\n");
+
+ if (!dev->dma_parms) {
+ dev->dma_parms =
+ devm_kzalloc(dev, sizeof(*dev->dma_parms), GFP_KERNEL);
+ if (!dev->dma_parms)
+ return -ENOMEM;
+ }
+ if (pdev->dev.dma_parms)
+ dma_set_max_seg_size(dev, UINT_MAX);
+
+ pm_runtime_enable(dev);
+
+ return 0;
+}
+
+static void mtk_imgsys_larb_remove(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+
+ pm_runtime_disable(dev);
+}
+
+static const struct of_device_id mtk_imgsys_larb_match[] = {
+ {.compatible = "mediatek,mt8188-imgsys-larb",},
+ {},
+};
+
+static struct platform_driver mtk_imgsys_larb_driver = {
+ .probe = mtk_imgsys_larb_probe,
+ .remove = mtk_imgsys_larb_remove,
+ .driver = {
+ .name = "mtk-imgsys-larb",
+ .of_match_table = of_match_ptr(mtk_imgsys_larb_match),
+ },
+};
+
static int mtk_imgsys_sd_subscribe_event(struct v4l2_subdev *subdev,
struct v4l2_fh *fh,
struct v4l2_event_subscription *sub)
@@ -1537,8 +1589,16 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
pm_runtime_use_autosuspend(&pdev->dev);
pm_runtime_enable(&pdev->dev);
+ ret = platform_driver_register(&mtk_imgsys_larb_driver);
+ if (ret) {
+ dev_info(imgsys_dev->dev, "register mtk_imgsys_larb_driver fail\n");
+ goto err_release_deinit_v4l2;
+ }
+
return 0;
+err_release_deinit_v4l2:
+ mtk_imgsys_dev_v4l2_release(imgsys_dev);
err_free_larb_alloc:
devm_kfree(&pdev->dev, imgsys_dev->larbs);
err_free_dev_alloc:
@@ -1551,6 +1611,7 @@ static void mtk_imgsys_remove(struct platform_device *pdev)
struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(&pdev->dev);
pm_runtime_disable(&pdev->dev);
+ platform_driver_unregister(&mtk_imgsys_larb_driver);
mtk_imgsys_dev_v4l2_release(imgsys_dev);
devm_kfree(&pdev->dev, imgsys_dev->larbs);
devm_kfree(&pdev->dev, imgsys_dev);
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (3 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 05/10] media: mediatek: isp: Add the Imgsys LARBs device Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 11:49 ` [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver Olivia Wen
` (4 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on a set of operations for specific modules.
The ImgSys architecture includes various modules such as WPE (Warping
Engine), TRAW (Tile-Raw), DIP (Digital Image Processing), PQDIP (Picture
Quality Digital Image Processing), and ME (Motion Estimation).
Each module requires specific operations for initialization and
deinitialization to ensure proper setup and cleanup.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 8 +
.../isp_7x/imgsys/modules/mtk_imgsys-dip.c | 115 +++++++++++
.../isp_7x/imgsys/modules/mtk_imgsys-dip.h | 18 ++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.c | 113 +++++++++++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.h | 17 ++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.c | 65 +++++++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.h | 17 ++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.c | 116 +++++++++++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.h | 17 ++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 12 +-
.../isp_7x/imgsys/mtk_imgsys-module_common.h | 82 ++++++++
.../isp_7x/imgsys/mtk_imgsys-module_main.c | 183 ++++++++++++++++++
.../isp_7x/imgsys/mtk_imgsys-module_main.h | 17 ++
.../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 26 +++
.../isp/isp_7x/imgsys/mtk_imgsys-of.h | 3 +
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 73 ++++++-
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 45 ++++-
17 files changed, 923 insertions(+), 4 deletions(-)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 571d0ae000eb..26c8c1d39750 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,10 +2,18 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-module_main.o \
mtk_imgsys-of.o \
mtk_imgsys-formats.o \
mtk_imgsys-dev.o \
mtk_imgsys-sys.o \
mtk_imgsys-v4l2.o
+mtk_imgsys_hw_isp-objs := \
+modules/mtk_imgsys-dip.o \
+modules/mtk_imgsys-traw.o \
+modules/mtk_imgsys-pqdip.o \
+modules/mtk_imgsys-wpe.o
+
obj-$(CONFIG_VIDEO_MTK_ISP_71_IMGSYS) += mtk_imgsys_util.o
+obj-$(CONFIG_VIDEO_MTK_ISP_71_IMGSYS) += mtk_imgsys_hw_isp.o
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
new file mode 100644
index 000000000000..1cc1f3e87e8b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-dip.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+#define DIPCTL_D1A_DIPCTL_INT1_EN 0x094
+#define DIPCTL_D1A_DIPCTL_INT2_EN 0x0A0
+#define DIPCTL_D1A_DIPCTL_INT3_EN 0x0AC
+#define DIPCTL_D1A_DIPCTL_CQ_INT1_EN 0x0C4
+#define DIPCTL_D1A_DIPCTL_CQ_INT2_EN 0x0D0
+#define DIPCTL_D1A_DIPCTL_CQ_INT3_EN 0x0DC
+
+#define DIPCQ_D1A_DIPCQ_CQ_THR0_CTL 0x208
+#define DIPCQ_D1A_DIPCQ_CQ_THR1_CTL 0x218
+#define DIPCQ_D1A_DIPCQ_CQ_THR2_CTL 0x228
+#define DIPCQ_D1A_DIPCQ_CQ_THR3_CTL 0x238
+#define DIPCQ_D1A_DIPCQ_CQ_THR4_CTL 0x248
+#define DIPCQ_D1A_DIPCQ_CQ_THR5_CTL 0x258
+#define DIPCQ_D1A_DIPCQ_CQ_THR6_CTL 0x268
+#define DIPCQ_D1A_DIPCQ_CQ_THR7_CTL 0x278
+#define DIPCQ_D1A_DIPCQ_CQ_THR8_CTL 0x288
+#define DIPCQ_D1A_DIPCQ_CQ_THR9_CTL 0x298
+#define DIPCQ_D1A_DIPCQ_CQ_THR10_CTL 0x2A8
+#define DIPCQ_D1A_DIPCQ_CQ_THR11_CTL 0x2B8
+#define DIPCQ_D1A_DIPCQ_CQ_THR12_CTL 0x2C8
+#define DIPCQ_D1A_DIPCQ_CQ_THR13_CTL 0x2D8
+#define DIPCQ_D1A_DIPCQ_CQ_THR14_CTL 0x2E8
+#define DIPCQ_D1A_DIPCQ_CQ_THR15_CTL 0x2F8
+#define DIPCQ_D1A_DIPCQ_CQ_THR16_CTL 0x308
+#define DIPCQ_D1A_DIPCQ_CQ_THR17_CTL 0x318
+#define DIPCQ_D1A_DIPCQ_CQ_THR18_CTL 0x328
+
+#define DIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_dip_init_ary)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_dip_init_ary[] = {
+ { DIPCTL_D1A_DIPCTL_INT1_EN, 0x80000000 },
+ { DIPCTL_D1A_DIPCTL_INT2_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_INT3_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_CQ_INT1_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_CQ_INT2_EN, 0x0 },
+ { DIPCTL_D1A_DIPCTL_CQ_INT3_EN, 0x0 },
+ { DIPCQ_D1A_DIPCQ_CQ_THR0_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR1_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR2_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR3_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR4_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR5_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR6_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR7_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR8_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR9_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR10_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR11_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR12_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR13_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR14_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR15_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR16_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR17_CTL, CQ_THRX_CTL },
+ { DIPCQ_D1A_DIPCQ_CQ_THR18_CTL, CQ_THRX_CTL },
+};
+
+static void __iomem *dip_reg_base[MTK_IMGSYS_DIP_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_DIP_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_DIP1 + hw_idx;
+ dip_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!dip_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap dip_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < DIP_INIT_ARRAY_COUNT; i++) {
+ offset = dip_reg_base[hw_idx] + mtk_imgsys_dip_init_ary[i].offset;
+ writel(mtk_imgsys_dip_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_dip_init);
+
+void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_DIP_HW_SET; hw_idx++) {
+ iounmap(dip_reg_base[hw_idx]);
+ dip_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_dip_uninit);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
new file mode 100644
index 000000000000..22b4a7c3616d
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_DIP_DIP_H_
+#define _MTK_DIP_DIP_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_DIP_DIP_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
new file mode 100644
index 000000000000..d1d3507555d9
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-pqdip.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+#define PQDIPCTL_P1A_REG_PQDIPCTL_INT1_EN 0x0050
+#define PQDIPCTL_P1A_REG_PQDIPCTL_INT2_EN 0x0060
+#define PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT1_EN 0x0070
+#define PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT2_EN 0x0080
+#define PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT3_EN 0x0090
+
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR0_CTL 0x0208
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR1_CTL 0x0218
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR2_CTL 0x0228
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR3_CTL 0x0238
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR4_CTL 0x0248
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR5_CTL 0x0258
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR6_CTL 0x0268
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR7_CTL 0x0278
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR8_CTL 0x0288
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR9_CTL 0x0298
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR10_CTL 0x02A8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR11_CTL 0x02B8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR12_CTL 0x02C8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR13_CTL 0x02D8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR14_CTL 0x02E8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR15_CTL 0x02F8
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR16_CTL 0x0308
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR17_CTL 0x0318
+#define DIPCQ_P1A_REG_DIPCQ_CQ_THR18_CTL 0x0328
+
+#define PQDIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_pqdip_init_ary)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_pqdip_init_ary[] = {
+ { PQDIPCTL_P1A_REG_PQDIPCTL_INT1_EN, 0x80000000 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_INT2_EN, 0x0 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT1_EN, 0x0 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT2_EN, 0x0 },
+ { PQDIPCTL_P1A_REG_PQDIPCTL_CQ_INT3_EN, 0x0 },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR0_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR1_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR2_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR3_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR4_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR5_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR6_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR7_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR8_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR9_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR10_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR11_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR12_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR13_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR14_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR15_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR16_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR17_CTL, CQ_THRX_CTL },
+ { DIPCQ_P1A_REG_DIPCQ_CQ_THR18_CTL, CQ_THRX_CTL },
+};
+
+static void __iomem *pqdip_reg_base[MTK_IMGSYS_PQDIP_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_PQDIP_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_PQDIP1 + hw_idx;
+ pqdip_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!pqdip_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap pqdip_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < PQDIP_INIT_ARRAY_COUNT; i++) {
+ offset = pqdip_reg_base[hw_idx] + mtk_imgsys_pqdip_init_ary[i].offset;
+ writel(mtk_imgsys_pqdip_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_pqdip_init);
+
+void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_PQDIP_HW_SET; hw_idx++) {
+ iounmap(pqdip_reg_base[hw_idx]);
+ pqdip_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_pqdip_uninit);
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
new file mode 100644
index 000000000000..00f93d536181
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_PQDIP_H_
+#define _MTK_IMGSYS_PQDIP_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_PQDIP_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
new file mode 100644
index 000000000000..292f0ad09c46
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Shih-Fang Chuang <shih-fang.chuang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-traw.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+#define TRAWCTL_INT1_EN 0x00A0
+
+#define TRAW_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_traw_init_ary)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_traw_init_ary[] = {
+ { TRAWCTL_INT1_EN, 0x80000000 },
+};
+
+static void __iomem *traw_reg_base[MTK_IMGSYS_TRAW_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_TRAW_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_TRAW1 + hw_idx;
+ traw_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!traw_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap traw_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < TRAW_INIT_ARRAY_COUNT; i++) {
+ offset = traw_reg_base[hw_idx] + mtk_imgsys_traw_init_ary[i].offset;
+ writel(mtk_imgsys_traw_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_traw_init);
+
+void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_TRAW_HW_SET; hw_idx++) {
+ iounmap(traw_reg_base[hw_idx]);
+ traw_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_traw_uninit);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
new file mode 100644
index 000000000000..f3727216b58c
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Shih-fang Chuang <shih-fang.chuang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_TRAW_H_
+#define _MTK_IMGSYS_TRAW_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_TRAW_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
new file mode 100644
index 000000000000..b19d61529b80
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
@@ -0,0 +1,116 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Floria Huang <floria.huang@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "../mtk_imgsys-module_common.h"
+#include "mtk_imgsys-wpe.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+/* Define register offsets */
+#define WPE_TOP_CTL_INT_EN_OFFSET 0x0018
+#define WPE_TOP_CTL_INT_STATUSX_OFFSET 0x0024
+#define WPE_TOP_CQ_IRQ_EN_OFFSET 0x00D4
+#define WPE_TOP_CQ_IRQ_STX_OFFSET 0x00DC
+#define WPE_TOP_CQ_IRQ_EN2_OFFSET 0x00E0
+#define WPE_TOP_CQ_IRQ_STX2_OFFSET 0x00E8
+#define WPE_TOP_CQ_IRQ_EN3_OFFSET 0x00EC
+#define WPE_TOP_CQ_IRQ_STX3_OFFSET 0x00F4
+#define WPE_CACHE_RWCTL_CTL_OFFSET 0x0204
+#define WPE_DMA_DMA_ERR_CTRL_OFFSET 0x03D4
+#define DIPCQ_W1A_DIPCQ_CQ_THR0_CTL_OFFSET 0x0B08
+#define DIPCQ_W1A_DIPCQ_CQ_THR1_CTL_OFFSET 0x0B18
+#define DIPCQ_W1A_DIPCQ_CQ_THR2_CTL_OFFSET 0x0B28
+#define DIPCQ_W1A_DIPCQ_CQ_THR3_CTL_OFFSET 0x0B38
+#define DIPCQ_W1A_DIPCQ_CQ_THR4_CTL_OFFSET 0x0B48
+#define DIPCQ_W1A_DIPCQ_CQ_THR5_CTL_OFFSET 0x0B58
+#define DIPCQ_W1A_DIPCQ_CQ_THR6_CTL_OFFSET 0x0B68
+#define DIPCQ_W1A_DIPCQ_CQ_THR7_CTL_OFFSET 0x0B78
+#define DIPCQ_W1A_DIPCQ_CQ_THR8_CTL_OFFSET 0x0B88
+#define DIPCQ_W1A_DIPCQ_CQ_THR9_CTL_OFFSET 0x0B98
+#define DIPCQ_W1A_DIPCQ_CQ_THR10_CTL_OFFSET 0x0BA8
+#define DIPCQ_W1A_DIPCQ_CQ_THR11_CTL_OFFSET 0x0BB8
+#define DIPCQ_W1A_DIPCQ_CQ_THR12_CTL_OFFSET 0x0BC8
+#define DIPCQ_W1A_DIPCQ_CQ_THR13_CTL_OFFSET 0x0BD8
+#define DIPCQ_W1A_DIPCQ_CQ_THR14_CTL_OFFSET 0x0BE8
+
+#define WPE_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_wpe_init_ary)
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static const struct mtk_imgsys_init_array mtk_imgsys_wpe_init_ary[] = {
+ { WPE_TOP_CTL_INT_EN_OFFSET, 0x80000000 },
+ { WPE_TOP_CTL_INT_STATUSX_OFFSET, 0xFFFFFFFF },
+ { WPE_TOP_CQ_IRQ_EN_OFFSET, 0x80000000 },
+ { WPE_TOP_CQ_IRQ_STX_OFFSET, 0xFFFFFFFF },
+ { WPE_TOP_CQ_IRQ_EN2_OFFSET, 0x80000000 },
+ { WPE_TOP_CQ_IRQ_STX2_OFFSET, 0xFFFFFFFF },
+ { WPE_TOP_CQ_IRQ_EN3_OFFSET, 0x80000000 },
+ { WPE_TOP_CQ_IRQ_STX3_OFFSET, 0xFFFFFFFF },
+ { WPE_CACHE_RWCTL_CTL_OFFSET, 0x00000002 },
+ { WPE_DMA_DMA_ERR_CTRL_OFFSET, 0x80000000 },
+ { DIPCQ_W1A_DIPCQ_CQ_THR0_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR1_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR2_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR3_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR4_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR5_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR6_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR7_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR8_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR9_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR10_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR11_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR12_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR13_CTL_OFFSET, CQ_THRX_CTL },
+ { DIPCQ_W1A_DIPCQ_CQ_THR14_CTL_OFFSET, CQ_THRX_CTL },
+};
+
+static void __iomem *wpe_reg_base[MTK_IMGSYS_WPE_HW_SET];
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int i, hw_id, hw_idx;
+ void __iomem *offset = NULL;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ hw_id = IMGSYS_REG_WPE1 + hw_idx;
+ wpe_reg_base[hw_idx] = of_iomap(imgsys_dev->dev->of_node, hw_id);
+ if (!wpe_reg_base[hw_idx]) {
+ dev_info(imgsys_dev->dev,
+ "%s: error: unable to iomap wpe_%d registers, devnode(%s).\n",
+ __func__, hw_id, imgsys_dev->dev->of_node->name);
+ continue;
+ }
+
+ for (i = 0 ; i < WPE_INIT_ARRAY_COUNT; i++) {
+ offset = wpe_reg_base[hw_idx] + mtk_imgsys_wpe_init_ary[i].offset;
+ writel(mtk_imgsys_wpe_init_ary[i].value, offset);
+ }
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_wpe_init);
+
+void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ unsigned int hw_idx;
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ iounmap(wpe_reg_base[hw_idx]);
+ wpe_reg_base[hw_idx] = NULL;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_wpe_uninit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("MediaTek IMGSYS Modules");
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
new file mode 100644
index 000000000000..6001575265e6
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2020 MediaTek Inc.
+ *
+ * Author: Floria Huang <floria.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_WPE_H_
+#define _MTK_IMGSYS_WPE_H_
+
+#include "../mtk_imgsys-dev.h"
+
+void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_WPE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
index b69cfd0043d3..b8c30cb32b97 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
@@ -23,7 +23,7 @@
#include <linux/videodev2.h>
#include <media/videobuf2-core.h>
#include <media/videobuf2-v4l2.h>
-
+#include "mtk_imgsys-module_ops.h"
#include "mtk_header_desc.h"
#define MTK_IMGSYS_MEDIA_MODEL_NAME "MTK-ISP-DIP-V4L2"
@@ -273,6 +273,7 @@ struct mtk_imgsys_pipe {
* @v4l2_dev: V4L2 device structure for video device operations.
* @imgsys_pipe: Image system pipeline configuration structure.
* @cust_pipes: Pointer to custom pipeline descriptors.
+ * @modules: Pointer to image system module operations.
* @clks: Array of clock bulk data for managing multiple clocks.
* @num_clks: Number of clocks in the clock array.
* @larbs: Array of device pointers for managing LARB (local arbiter).
@@ -281,6 +282,9 @@ struct mtk_imgsys_pipe {
* @rproc_handle: Handle for remote processor interface.
* @smem_dev: Device structure for shared memory operations.
* @num_mods: Number of modules in the image system.
+ * @init_kref: Reference counter for initialization tracking.
+ * @hw_op_lock: Mutex for serializing hardware operations.
+ * @stream_cnt: Counter for active streams, increased on stream on.
*/
struct mtk_imgsys_dev {
/* Device and Resource Information */
@@ -292,6 +296,8 @@ struct mtk_imgsys_dev {
struct mtk_imgsys_pipe imgsys_pipe;
/* Imgsys Pipeline Information */
const struct mtk_imgsys_pipe_desc *cust_pipes;
+ /* Imgsys Moudles Information */
+ const struct module_ops *modules;
/* Clock Information */
struct clk_bulk_data *clks;
int num_clks;
@@ -304,6 +310,10 @@ struct mtk_imgsys_dev {
struct device *smem_dev;
/* Number of modules */
int num_mods;
+ /* Synchronization and Flow Control */
+ struct kref init_kref;
+ struct mutex hw_op_lock; /* Protect HW operations */
+ int stream_cnt;
};
/**
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
new file mode 100644
index 000000000000..dd02a8fa28d4
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_CORE_REG_H_
+#define _MTK_IMGSYS_CORE_REG_H_
+
+/**
+ * Define the minimum and maximum hardware IDs
+ * for different MediaTek Image System components
+ */
+#define WPE_HW_MIN_ID IMGSYS_REG_WPE1
+#define WPE_HW_MAX_ID IMGSYS_REG_WPE3
+#define TRAW_HW_MIN_ID IMGSYS_REG_TRAW1
+#define TRAW_HW_MAX_ID IMGSYS_REG_TRAW3
+#define DIP_HW_MIN_ID IMGSYS_REG_DIP1
+#define DIP_HW_MAX_ID IMGSYS_REG_DIP2
+#define PQDIP_HW_MIN_ID IMGSYS_REG_PQDIP1
+#define PQDIP_HW_MAX_ID IMGSYS_REG_PQDIP2
+#define ME_HW_MIN_ID IMGSYS_REG_ME
+#define ME_HW_MAX_ID IMGSYS_REG_ME
+
+/* Macro to calculate the size of a hardware set */
+#define HW_SET_SIZE(min_id, max_id) ((max_id) - (min_id) + 1)
+
+/* Define the hardware set sizes */
+#define MTK_IMGSYS_WPE_HW_SET HW_SET_SIZE(WPE_HW_MIN_ID, WPE_HW_MAX_ID)
+#define MTK_IMGSYS_TRAW_HW_SET HW_SET_SIZE(TRAW_HW_MIN_ID, TRAW_HW_MAX_ID)
+#define MTK_IMGSYS_DIP_HW_SET HW_SET_SIZE(DIP_HW_MIN_ID, DIP_HW_MAX_ID)
+#define MTK_IMGSYS_PQDIP_HW_SET HW_SET_SIZE(PQDIP_HW_MIN_ID, PQDIP_HW_MAX_ID)
+#define MTK_IMGSYS_ME_HW_SET HW_SET_SIZE(ME_HW_MIN_ID, ME_HW_MAX_ID)
+
+/* Define the CQ's control register bits */
+#define CQ_THRX_CTL_EN BIT(0)
+#define CQ_THRX_CTL_MODE BIT(4)
+#define CQ_THRX_CTL (CQ_THRX_CTL_EN | CQ_THRX_CTL_MODE)
+
+/**
+ * enum mtk_imgsys_reg_id
+ *
+ * Definition about hw register map id
+ * The engine order should be the same as register order in dts
+ */
+enum mtk_imgsys_reg_id {
+ IMGSYS_REG_TOP = 0,
+ IMGSYS_REG_TRAW1,
+ IMGSYS_REG_TRAW2,
+ IMGSYS_REG_TRAW3,
+ IMGSYS_REG_DIP1,
+ IMGSYS_REG_DIP2,
+ IMGSYS_REG_PQDIP1,
+ IMGSYS_REG_PQDIP2,
+ IMGSYS_REG_WPE1,
+ IMGSYS_REG_WPE2,
+ IMGSYS_REG_WPE3,
+ IMGSYS_REG_WPE1_DIP1,
+ IMGSYS_REG_ME,
+ IMGSYS_REG_WPE2_DIP1,
+ IMGSYS_REG_WPE3_DIP1,
+ IMGSYS_REG_DIP1_TOP,
+ IMGSYS_REG_DIP2_TOP
+};
+
+/**
+ * struct mtk_imgsys_init_array - Array structure for initializing registers
+ * @offset: Offset of the register
+ * @val: Value to be written to the register
+ *
+ * This structure represents an array element used for initializing registers
+ * in the MediaTek Image System. Each element contains the offset of the register
+ * and the value to be written to that register.
+ */
+struct mtk_imgsys_init_array {
+ unsigned int offset;
+ unsigned int value;
+};
+#endif /* _MTK_IMGSYS_CORE_REG_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
new file mode 100644
index 000000000000..5daa8f863bd2
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/of_address.h>
+#include "mtk_imgsys-module_common.h"
+#include "mtk_imgsys-module_main.h"
+
+/********************************************************************
+ * Global Define
+ ********************************************************************/
+/* Define the reset register bits */
+#define SW_RST (0x000C)
+
+/********************************************************************
+ * Global Variable
+ ********************************************************************/
+static void __iomem *imgsys_main_reg_base;
+static void __iomem *wpe_dip1_reg_base;
+static void __iomem *wpe_dip2_reg_base;
+static void __iomem *wpe_dip3_reg_base;
+static void __iomem *dip1_reg_base;
+static void __iomem *dip2_reg_base;
+
+/********************************************************************
+ * Functions
+ ********************************************************************/
+static void imgsys_main_hw_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ void __iomem *wpe_reg_base = NULL;
+ void __iomem *wpe_ctrl = NULL;
+ unsigned int hw_idx = 0;
+
+ /* Enable all LARBs of ImgSys */
+ /**
+ * The flow will be updated based on the patch series 958467 from
+ * the Linux MediaTek project. For more details, refer to the following link:
+ * https://patchwork.kernel.org/project/linux-mediatek/list/?series=958467
+ */
+
+ /* Reset ImgSys hardware */
+ iowrite32(0xFFFFFFFF, dip1_reg_base + SW_RST);
+ iowrite32(0xFFFFFFFF, dip2_reg_base + SW_RST);
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ if (hw_idx == 0)
+ wpe_reg_base = wpe_dip1_reg_base;
+ else if (hw_idx == 1)
+ wpe_reg_base = wpe_dip2_reg_base;
+ else
+ wpe_reg_base = wpe_dip3_reg_base;
+
+ wpe_ctrl = wpe_reg_base + SW_RST;
+ iowrite32(0xFFFFFFFF, wpe_ctrl);
+ iowrite32(0x0, wpe_ctrl);
+ }
+
+ iowrite32(0x00CF00FF, imgsys_main_reg_base + SW_RST);
+ iowrite32(0x0, imgsys_main_reg_base + SW_RST);
+
+ iowrite32(0x0, dip1_reg_base + SW_RST);
+ iowrite32(0x0, dip2_reg_base + SW_RST);
+
+ for (hw_idx = 0; hw_idx < MTK_IMGSYS_WPE_HW_SET; hw_idx++) {
+ if (hw_idx == 0)
+ wpe_reg_base = wpe_dip1_reg_base;
+ else if (hw_idx == 1)
+ wpe_reg_base = wpe_dip2_reg_base;
+ else
+ wpe_reg_base = wpe_dip3_reg_base;
+
+ wpe_ctrl = wpe_reg_base + SW_RST;
+ iowrite32(0xFFFFFFFF, wpe_ctrl);
+ iowrite32(0x0, wpe_ctrl);
+ }
+
+ iowrite32(0x00CF00FF, imgsys_main_reg_base + SW_RST);
+ iowrite32(0x0, imgsys_main_reg_base + SW_RST);
+
+ /* Disable all LARBs of ImgSys*/
+ /**
+ * The flow will be updated based on the patch series 958467 from
+ * the Linux MediaTek project. For more details, refer to the following link:
+ * https://patchwork.kernel.org/project/linux-mediatek/list/?series=958467
+ */
+}
+
+void imgsys_main_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ imgsys_main_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_TOP);
+ if (!imgsys_main_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap imgsys_top registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ wpe_dip1_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_WPE1_DIP1);
+ if (!wpe_dip1_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap wpe_dip1 registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ wpe_dip2_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_WPE2_DIP1);
+ if (!wpe_dip2_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap wpe_dip2 registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ wpe_dip3_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_WPE3_DIP1);
+ if (!wpe_dip3_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap wpe_dip3 registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ dip1_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_DIP1_TOP);
+ if (!dip1_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap dip_top registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ dip2_reg_base = of_iomap(imgsys_dev->dev->of_node, IMGSYS_REG_DIP2_TOP);
+ if (!dip2_reg_base) {
+ dev_info(imgsys_dev->dev, "%s Unable to ioremap dip_top_nr registers\n",
+ __func__);
+ dev_info(imgsys_dev->dev, "%s of_iomap fail, devnode(%s).\n",
+ __func__, imgsys_dev->dev->of_node->name);
+ return;
+ }
+
+ imgsys_main_hw_init(imgsys_dev);
+}
+
+void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ if (imgsys_main_reg_base) {
+ iounmap(imgsys_main_reg_base);
+ imgsys_main_reg_base = NULL;
+ }
+
+ if (wpe_dip1_reg_base) {
+ iounmap(wpe_dip1_reg_base);
+ wpe_dip1_reg_base = NULL;
+ }
+
+ if (wpe_dip2_reg_base) {
+ iounmap(wpe_dip2_reg_base);
+ wpe_dip2_reg_base = NULL;
+ }
+
+ if (wpe_dip3_reg_base) {
+ iounmap(wpe_dip3_reg_base);
+ wpe_dip3_reg_base = NULL;
+ }
+
+ if (dip1_reg_base) {
+ iounmap(dip1_reg_base);
+ dip1_reg_base = NULL;
+ }
+
+ if (dip2_reg_base) {
+ iounmap(dip2_reg_base);
+ dip2_reg_base = NULL;
+ }
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
new file mode 100644
index 000000000000..2a3fbd08d128
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_DEBUG_H_
+#define _MTK_IMGSYS_DEBUG_H_
+
+#include "mtk_imgsys-dev.h"
+
+void imgsys_main_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_DEBUG_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
index 6dc4b7b60832..a2eea8f4fb3c 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
@@ -24,4 +24,30 @@ enum mtk_imgsys_module {
IMGSYS_MOD_MAX,
};
+struct mtk_imgsys_dev;
+
+/**
+ * struct module_ops - Operations for a specific module in the MediaTek Image System
+ * @module_id: Identifier for the module.
+ * @init: Function pointer to initialize the module.
+ * This function sets the default values for various software-related
+ * parameters and ensures that the module is ready for operation.
+ * Example function: imgsys_dip_init
+ * @uninit: Function pointer to uninitialize the module.
+ * This function resets various software-related parameters to ensure
+ * the module is properly cleaned up and ready for shutdown.
+ * Example function: imgsys_dip_uninit
+ *
+ * This structure defines a set of operations for specific modules within the
+ * MediaTek Image System, such as WPE (Warping Engine), TRAW (Tile-Raw),
+ * DIP (Digital Image Processing), PQDIP (Picture Quality Digital Image
+ * Processing), and ME (Motion Estimation). Each module is identified by
+ * a unique identifier and includes function pointers for operations like
+ * initialization and deinitialization.
+ */
+struct module_ops {
+ int module_id;
+ void (*init)(struct mtk_imgsys_dev *imgsys_dev);
+ void (*uninit)(struct mtk_imgsys_dev *imgsys_dev);
+};
#endif /* _MTK_IMGSYS_MODULES_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
index 5088c9df704c..f34ea9465745 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
@@ -12,6 +12,7 @@
#include <linux/clk.h>
#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-module_ops.h"
/**
* struct mtk_imgsys_mod_pipe_desc - Description of a module's pipeline
@@ -31,6 +32,7 @@ struct mtk_imgsys_mod_pipe_desc {
* @mod_num: Number of modules in the pipeline.
* @pipe_settings: Pointer to pipeline settings for the module.
* @pipe_num: Number of pipeline settings.
+ * @imgsys_modules: Pointer to module operations for image system modules.
*/
struct cust_data {
struct clk_bulk_data *clks;
@@ -39,6 +41,7 @@ struct cust_data {
unsigned int mod_num;
const struct mtk_imgsys_pipe_desc *pipe_settings;
unsigned int pipe_num;
+ const struct module_ops *imgsys_modules;
};
/**
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 9cdcdb0a0200..9f118491b37d 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -6,7 +6,45 @@
*
*/
+#include <linux/pm_runtime.h>
#include "mtk_imgsys-sys.h"
+#include "mtk_imgsys-dev.h"
+
+static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
+{
+ int ret, i;
+
+ ret = pm_runtime_get_sync(imgsys_dev->dev);
+ if (ret < 0) {
+ dev_err(imgsys_dev->dev, "Failed to sync runtime PM: %d\n", ret);
+ return ret;
+ }
+
+ /* Set default value for hardware modules */
+ for (i = 0; i < (imgsys_dev->num_mods); i++)
+ imgsys_dev->modules[i].init(imgsys_dev);
+
+ kref_init(&imgsys_dev->init_kref);
+
+ return 0;
+}
+
+static void module_uninit(struct kref *kref)
+{
+ struct mtk_imgsys_dev *imgsys_dev;
+ int i;
+
+ imgsys_dev = container_of(kref, struct mtk_imgsys_dev, init_kref);
+
+ for (i = 0; i < (imgsys_dev->num_mods); i++)
+ if (imgsys_dev->modules[i].uninit)
+ imgsys_dev->modules[i].uninit(imgsys_dev);
+}
+
+static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
+{
+ kref_put(&imgsys_dev->init_kref, module_uninit);
+}
void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
struct mtk_imgsys_request *req)
@@ -16,12 +54,43 @@ void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
{
- /* Function implementation will be provided in subsequent patches */
+ struct mtk_imgsys_dev *imgsys_dev = pipe->imgsys_dev;
+ int ret;
+
+ mutex_lock(&imgsys_dev->hw_op_lock);
+ if (!imgsys_dev->stream_cnt) {
+ ret = mtk_imgsys_hw_connect(pipe->imgsys_dev);
+ if (ret) {
+ dev_info(pipe->imgsys_dev->dev,
+ "%s:%s: pipe(%d) connect to dip_hw failed\n",
+ __func__, pipe->desc->name, pipe->desc->id);
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+ return ret;
+ }
+ }
+ imgsys_dev->stream_cnt++;
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: stream on, id(%d), stream cnt(%d)\n",
+ __func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
+
return 0;
}
int mtk_imgsys_hw_streamoff(struct mtk_imgsys_pipe *pipe)
{
- /* Function implementation will be provided in subsequent patches */
+ struct mtk_imgsys_dev *imgsys_dev = pipe->imgsys_dev;
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s:%s: stream off, id(%d), stream cnt(%d)\n",
+ __func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
+
+ mutex_lock(&imgsys_dev->hw_op_lock);
+ imgsys_dev->stream_cnt--;
+ if (!imgsys_dev->stream_cnt)
+ mtk_imgsys_hw_disconnect(imgsys_dev);
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+
return 0;
}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index 919c69493bbc..e25f17b98649 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -18,8 +18,13 @@
#include <media/v4l2-subdev.h>
#include <media/v4l2-event.h>
#include "linux/mtkisp_imgsys.h"
-#include "mtk_imgsys-of.h"
#include "mtk_imgsys-module_ops.h"
+#include "mtk_imgsys-module_main.h"
+#include "modules/mtk_imgsys-dip.h"
+#include "modules/mtk_imgsys-traw.h"
+#include "modules/mtk_imgsys-pqdip.h"
+#include "modules/mtk_imgsys-wpe.h"
+#include "mtk_imgsys-of.h"
#include "mtk_imgsys-vnode_id.h"
#include "mtk_imgsys_v4l2_vnode.h"
#include "mtk_imgsys-dev.h"
@@ -1499,6 +1504,7 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
init_imgsys_pipeline(data);
imgsys_dev->cust_pipes = data->pipe_settings;
+ imgsys_dev->modules = data->imgsys_modules;
imgsys_dev->dev = &pdev->dev;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
@@ -1511,6 +1517,8 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
dev_set_drvdata(&pdev->dev, imgsys_dev);
imgsys_dev->num_mods = data->mod_num;
+ imgsys_dev->stream_cnt = 0;
+ mutex_init(&imgsys_dev->hw_op_lock);
/* Get Clocks */
imgsys_dev->clks = data->clks;
@@ -1613,6 +1621,7 @@ static void mtk_imgsys_remove(struct platform_device *pdev)
pm_runtime_disable(&pdev->dev);
platform_driver_unregister(&mtk_imgsys_larb_driver);
mtk_imgsys_dev_v4l2_release(imgsys_dev);
+ mutex_destroy(&imgsys_dev->hw_op_lock);
devm_kfree(&pdev->dev, imgsys_dev->larbs);
devm_kfree(&pdev->dev, imgsys_dev);
}
@@ -1709,6 +1718,39 @@ static struct clk_bulk_data imgsys_isp7_clks[] = {
{ .id = "ME_CG_LARB12" },
};
+static const struct module_ops imgsys_isp7_modules[] = {
+ [IMGSYS_MOD_IMGMAIN] = {
+ .module_id = IMGSYS_MOD_IMGMAIN,
+ .init = imgsys_main_init,
+ .uninit = imgsys_main_uninit,
+ },
+ [IMGSYS_MOD_WPE] = {
+ .module_id = IMGSYS_MOD_WPE,
+ .init = imgsys_wpe_init,
+ .uninit = imgsys_wpe_uninit,
+ },
+ [IMGSYS_MOD_TRAW] = {
+ .module_id = IMGSYS_MOD_TRAW,
+ .init = imgsys_traw_init,
+ .uninit = imgsys_traw_uninit,
+ },
+ [IMGSYS_MOD_DIP] = {
+ .module_id = IMGSYS_MOD_DIP,
+ .init = imgsys_dip_init,
+ .uninit = imgsys_dip_uninit,
+ },
+ [IMGSYS_MOD_PQDIP] = {
+ .module_id = IMGSYS_MOD_PQDIP,
+ .init = imgsys_pqdip_init,
+ .uninit = imgsys_pqdip_uninit,
+ },
+ [IMGSYS_MOD_ME] = {
+ .module_id = IMGSYS_MOD_ME,
+ .init = NULL,
+ .uninit = NULL,
+ },
+};
+
static const struct cust_data imgsys_data[] = {
[0] = {
.clks = imgsys_isp7_clks,
@@ -1717,6 +1759,7 @@ static const struct cust_data imgsys_data[] = {
.mod_num = ARRAY_SIZE(module_pipe_isp7),
.pipe_settings = pipe_settings_isp7,
.pipe_num = ARRAY_SIZE(pipe_settings_isp7),
+ .imgsys_modules = imgsys_isp7_modules,
},
};
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (4 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 06/10] media: mediatek: isp: Add module operations structure for ImgSys Olivia Wen
@ 2025-05-24 11:49 ` Olivia Wen
2025-05-24 11:50 ` [PATCH v1 08/10] media: mediatek: isp: Add SCP " Olivia Wen
` (3 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:49 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on CMDQ (Command Queue) support for the MediaTek Image System
driver. CMDQ is essential for managing and executing hardware commands
efficiently, ensuring smooth operation of image processing tasks within
the ImgSys architecture.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 1 +
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h | 502 +++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq.c | 690 ++++++++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-cmdq.h | 29 +
.../isp_7x/imgsys/mtk_imgsys-module_common.h | 20 +
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 5 +
6 files changed, 1247 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 26c8c1d39750..c0f0f3a0c38a 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,6 +2,7 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-cmdq.o \
mtk_imgsys-module_main.o \
mtk_imgsys-of.o \
mtk_imgsys-formats.o \
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
new file mode 100644
index 000000000000..801a12e70bac
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq-plat.h
@@ -0,0 +1,502 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_CMDQ_PLAT_H_
+#define _MTK_IMGSYS_CMDQ_PLAT_H_
+
+#include "mtk_imgsys-cmdq.h"
+
+/* Maximum number of frame blocks for GCE recording */
+#define GCE_REC_MAX_FRAME_BLOCK 40
+
+/* Maximum number of tile blocks for GCE recording */
+#define GCE_REC_MAX_TILE_BLOCK 40
+
+/* Maximum number of label counts for GCE recording */
+#define GCE_REC_MAX_LABEL_COUNT 2048
+
+/* Maximum number of image system engines */
+#define IMGSYS_ENG_MAX 10
+
+/* Start address of the image system register */
+#define IMGSYS_REG_START 0x15000000
+
+/* End address of the image system register */
+#define IMGSYS_REG_END 0x1570FFFF
+
+/* Number of normal camera's thread */
+#define IMGSYS_NOR_THD 10
+
+/* Define the constants for buffer validation */
+#define HEADER_CODE 0x5A5A5A5A
+#define CHECK_PRE 0x55AA55AA
+#define CHECK_POST 0xAA55AA55
+#define FOOTER_CODE 0xA5A5A5A5
+
+/* Enumeration for Image System (IMGSYS) events */
+enum mtk_imgsys_event {
+ /* HW event */
+ /* TRAW frame done events */
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_0 = 0,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_1,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_2,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_3,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_4,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_5,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_6,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_7,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_8,
+ IMGSYS_EVENT_TRAW0_CQ_THR_DONE_9,
+ /* LTRAW frame done events */
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_0,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_1,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_2,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_3,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_4,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_5,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_6,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_7,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_8,
+ IMGSYS_EVENT_TRAW1_CQ_THR_DONE_9,
+ /* XTRAW frame done events */
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_0,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_1,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_2,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_3,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_4,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_5,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_6,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_7,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_8,
+ IMGSYS_EVENT_XTRAW_CQ_THR_DONE_9,
+ /* DIP frame done events */
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_0,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_1,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_2,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_3,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_4,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_5,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_6,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_7,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_8,
+ IMGSYS_EVENT_DIP_CQ_THR_DONE_9,
+ /* PQ_DIP_A frame done events */
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_0,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_1,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_2,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_3,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_4,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_5,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_6,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_7,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_8,
+ IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_9,
+ /* PQ_DIP_B frame done events */
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_0,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_1,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_2,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_3,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_4,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_5,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_6,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_7,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_8,
+ IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_9,
+ /* WPE_EIS frame done events */
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_0,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_1,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_2,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_3,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_4,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_5,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_6,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_7,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_8,
+ IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_9,
+ /* WPE_TNR frame done events */
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_0,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_1,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_2,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_3,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_4,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_5,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_6,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_7,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_8,
+ IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_9,
+ /* WPE_LITE frame done events */
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_0,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_1,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_2,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_3,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_4,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_5,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_6,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_7,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_8,
+ IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_9,
+ /* ME done events */
+ IMGSYS_EVENT_IMGSYS_IPE_ME_DONE,
+ /* SW events */
+ IMGSYS_EVENT_SYNC_TOKEN_START,
+ IMGSYS_EVENT_SYNC_TOKEN_WPE_EIS = IMGSYS_EVENT_SYNC_TOKEN_START,
+ IMGSYS_EVENT_SYNC_TOKEN_WPE_TNR,
+ IMGSYS_EVENT_SYNC_TOKEN_WPE_LITE,
+ IMGSYS_EVENT_SYNC_TOKEN_TRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_LTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_XTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_DIP,
+ IMGSYS_EVENT_SYNC_TOKEN_PQDIP_A,
+ IMGSYS_EVENT_SYNC_TOKEN_PQDIP_B,
+ IMGSYS_EVENT_SYNC_TOKEN_IPESYS_ME,
+ IMGSYS_EVENT_SYNC_TOKEN_APUSYS_APU,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_TRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_LTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_XTRAW,
+ IMGSYS_EVENT_SYNC_TOKEN_VSS_DIP,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_START,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_1 = IMGSYS_EVENT_SYNC_TOKEN_POOL_START,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_2,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_3,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_4,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_5,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_6,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_7,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_8,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_9,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_10,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_11,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_12,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_13,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_14,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_15,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_16,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_17,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_18,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_19,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_20,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_21,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_22,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_23,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_24,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_25,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_26,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_27,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_28,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_29,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_30,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_31,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_32,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_33,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_34,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_35,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_36,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_37,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_38,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_39,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_40,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_41,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_42,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_43,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_44,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_45,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_46,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_47,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_48,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_49,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_50,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_51,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_52,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_53,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_54,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_55,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_56,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_57,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_58,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_59,
+ IMGSYS_EVENT_SYNC_TOKEN_POOL_60,
+ IMGSYS_EVENT_SYNC_TOKEN_END = IMGSYS_EVENT_SYNC_TOKEN_POOL_60,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_1,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_2,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_3,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_4,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_5,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_6,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_7,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_8,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_9,
+ IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_10,
+ IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_WAIT,
+ IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_SET,
+ IMGSYS_EVENT_MAX
+};
+
+/* Structure to represent an IMGSYS event */
+struct imgsys_event_table {
+ u16 event;
+ char dts_name[256];
+};
+
+/* Event table for Image System (IMGSYS) */
+static struct imgsys_event_table imgsys_event[] = {
+ /* TRAW frame done events */
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_0, "traw_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_1, "traw_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_2, "traw_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_3, "traw_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_4, "traw_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_5, "traw_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_6, "traw_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_7, "traw_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_8, "traw_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_TRAW0_CQ_THR_DONE_9, "traw_cq_thread9_frame_done"},
+ /* LTRAW frame done events */
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_0, "ltraw_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_1, "ltraw_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_2, "ltraw_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_3, "ltraw_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_4, "ltraw_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_5, "ltraw_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_6, "ltraw_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_7, "ltraw_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_8, "ltraw_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_TRAW1_CQ_THR_DONE_9, "ltraw_cq_thread9_frame_done"},
+ /* XTRAW frame done events */
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_0, "xtraw_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_1, "xtraw_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_2, "xtraw_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_3, "xtraw_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_4, "xtraw_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_5, "xtraw_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_6, "xtraw_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_7, "xtraw_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_8, "xtraw_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_XTRAW_CQ_THR_DONE_9, "xtraw_cq_thread9_frame_done"},
+ /* DIP frame done events */
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_0, "dip_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_1, "dip_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_2, "dip_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_3, "dip_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_4, "dip_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_5, "dip_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_6, "dip_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_7, "dip_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_8, "dip_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_DIP_CQ_THR_DONE_9, "dip_cq_thread9_frame_done"},
+ /* PQ_DIP_A frame done events */
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_0, "pqa_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_1, "pqa_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_2, "pqa_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_3, "pqa_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_4, "pqa_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_5, "pqa_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_6, "pqa_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_7, "pqa_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_8, "pqa_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_PQDIP_A_CQ_THR_DONE_9, "pqa_cq_thread9_frame_done"},
+ /* PQ_DIP_B frame done events */
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_0, "pqb_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_1, "pqb_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_2, "pqb_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_3, "pqb_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_4, "pqb_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_5, "pqb_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_6, "pqb_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_7, "pqb_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_8, "pqb_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_PQDIP_B_CQ_THR_DONE_9, "pqb_cq_thread9_frame_done"},
+ /* WPE_EIS frame done events */
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_0, "wpe_eis_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_1, "wpe_eis_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_2, "wpe_eis_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_3, "wpe_eis_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_4, "wpe_eis_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_5, "wpe_eis_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_6, "wpe_eis_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_7, "wpe_eis_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_8, "wpe_eis_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_WPE_EIS_CQ_THR_DONE_9, "wpe_eis_cq_thread9_frame_done"},
+ /* WPE_TNR frame done events */
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_0, "wpe_tnr_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_1, "wpe_tnr_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_2, "wpe_tnr_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_3, "wpe_tnr_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_4, "wpe_tnr_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_5, "wpe_tnr_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_6, "wpe_tnr_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_7, "wpe_tnr_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_8, "wpe_tnr_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_WPE_TNR_CQ_THR_DONE_9, "wpe_tnr_cq_thread9_frame_done"},
+ /* WPE_LITE frame done events */
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_0, "wpe_lite_cq_thread0_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_1, "wpe_lite_cq_thread1_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_2, "wpe_lite_cq_thread2_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_3, "wpe_lite_cq_thread3_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_4, "wpe_lite_cq_thread4_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_5, "wpe_lite_cq_thread5_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_6, "wpe_lite_cq_thread6_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_7, "wpe_lite_cq_thread7_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_8, "wpe_lite_cq_thread8_frame_done"},
+ { IMGSYS_EVENT_WPE_LITE_CQ_THR_DONE_9, "wpe_lite_cq_thread9_frame_done"},
+ /* ME done events */
+ { IMGSYS_EVENT_IMGSYS_IPE_ME_DONE, "me_done"},
+ /* SW events */
+ { IMGSYS_EVENT_SYNC_TOKEN_WPE_EIS, "wpe_eis_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_WPE_TNR, "wpe_tnr_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_WPE_LITE, "wpe_lite_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_TRAW, "traw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_LTRAW, "ltraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_XTRAW, "xtraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_DIP, "dip_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_PQDIP_A, "pqdip_a_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_PQDIP_B, "pqdip_b_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_IPESYS_ME, "me_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_APUSYS_APU, "apu_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_TRAW, "vss_traw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_LTRAW, "vss_ltraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_XTRAW, "vss_xtraw_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_VSS_DIP, "vss_dip_sync_token"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_1, "sw_sync_token_pool_1"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_2, "sw_sync_token_pool_2"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_3, "sw_sync_token_pool_3"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_4, "sw_sync_token_pool_4"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_5, "sw_sync_token_pool_5"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_6, "sw_sync_token_pool_6"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_7, "sw_sync_token_pool_7"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_8, "sw_sync_token_pool_8"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_9, "sw_sync_token_pool_9"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_10, "sw_sync_token_pool_10"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_11, "sw_sync_token_pool_11"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_12, "sw_sync_token_pool_12"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_13, "sw_sync_token_pool_13"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_14, "sw_sync_token_pool_14"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_15, "sw_sync_token_pool_15"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_16, "sw_sync_token_pool_16"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_17, "sw_sync_token_pool_17"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_18, "sw_sync_token_pool_18"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_19, "sw_sync_token_pool_19"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_20, "sw_sync_token_pool_20"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_21, "sw_sync_token_pool_21"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_22, "sw_sync_token_pool_22"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_23, "sw_sync_token_pool_23"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_24, "sw_sync_token_pool_24"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_25, "sw_sync_token_pool_25"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_26, "sw_sync_token_pool_26"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_27, "sw_sync_token_pool_27"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_28, "sw_sync_token_pool_28"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_29, "sw_sync_token_pool_29"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_30, "sw_sync_token_pool_30"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_31, "sw_sync_token_pool_31"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_32, "sw_sync_token_pool_32"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_33, "sw_sync_token_pool_33"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_34, "sw_sync_token_pool_34"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_35, "sw_sync_token_pool_35"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_36, "sw_sync_token_pool_36"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_37, "sw_sync_token_pool_37"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_38, "sw_sync_token_pool_38"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_39, "sw_sync_token_pool_39"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_40, "sw_sync_token_pool_40"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_41, "sw_sync_token_pool_41"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_42, "sw_sync_token_pool_42"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_43, "sw_sync_token_pool_43"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_44, "sw_sync_token_pool_44"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_45, "sw_sync_token_pool_45"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_46, "sw_sync_token_pool_46"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_47, "sw_sync_token_pool_47"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_48, "sw_sync_token_pool_48"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_49, "sw_sync_token_pool_49"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_50, "sw_sync_token_pool_50"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_51, "sw_sync_token_pool_51"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_52, "sw_sync_token_pool_52"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_53, "sw_sync_token_pool_53"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_54, "sw_sync_token_pool_54"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_55, "sw_sync_token_pool_55"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_56, "sw_sync_token_pool_56"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_57, "sw_sync_token_pool_57"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_58, "sw_sync_token_pool_58"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_59, "sw_sync_token_pool_59"},
+ { IMGSYS_EVENT_SYNC_TOKEN_POOL_60, "sw_sync_token_pool_60"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_1, "sw_sync_token_camsys_pool_1"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_2, "sw_sync_token_camsys_pool_2"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_3, "sw_sync_token_camsys_pool_3"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_4, "sw_sync_token_camsys_pool_4"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_5, "sw_sync_token_camsys_pool_5"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_6, "sw_sync_token_camsys_pool_6"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_7, "sw_sync_token_camsys_pool_7"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_8, "sw_sync_token_camsys_pool_8"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_9, "sw_sync_token_camsys_pool_9"},
+ { IMGSYS_EVENT_SYNC_TOKEN_CAMSYS_POOL_10, "sw_sync_token_camsys_pool_10"},
+ { IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_WAIT, "sw_sync_token_tzmp_isp_wait"},
+ { IMGSYS_EVENT_SYNC_TOKEN_TZMP_ISP_SET, "sw_sync_token_tzmp_isp_set"},
+ { IMGSYS_EVENT_MAX, "imgsys_event_max"},
+};
+
+/* Structure to record block information */
+struct block_record {
+ u32 label_min; /* Minimum label value */
+ u32 label_max; /* Maximum label value */
+ u32 label_count; /* Count of labels */
+ u32 cmd_offset; /* Offset of the command */
+ u32 cmd_length; /* Length of the command */
+};
+
+/* Enumeration for GCE recording block types */
+enum gce_rec_block_enum {
+ GCE_REC_NONE_BLOCK = -1, /* No block */
+ GCE_REC_FRAME_BLOCK = 0, /* Frame block */
+ GCE_REC_TILE_BLOCK /* Tile block */
+};
+
+/* Enumeration for GCE recording modes */
+enum gce_rec_mode_enum {
+ GCE_REC_APPEND_MODE = 0, /* Append mode */
+ GCE_REC_REPLACE_MODE /* Replace mode */
+};
+
+/**
+ *This structure is utilized by the SCP (Secondary Control Processor),
+ * which is another CPU in the system. It contains necessary information
+ * for managing software frames across different processing units.
+ */
+struct gce_recorder {
+ u32 header_code; /* Header guard word */
+ u32 cmd_offset; /* Command offset */
+ u32 check_pre; /* Pre-check guard word */
+ u32 *p_output; /* Output buffer pointer */
+ u32 check_post; /* Post-check guard word */
+ u32 *p_buffer; /* Buffer pointer */
+ u32 max_length; /* Maximum length of the buffer */
+ u32 curr_length; /* Current length of the buffer */
+
+ struct block_record frame_record[GCE_REC_MAX_FRAME_BLOCK]; /* Frame block records */
+ u32 frame_block; /* Number of frame blocks */
+ u32 curr_frame; /* Current frame block index */
+
+ struct block_record tile_record[GCE_REC_MAX_TILE_BLOCK]; /* Tile block records */
+ u32 tile_block; /* Number of tile blocks */
+ u32 curr_tile; /* Current tile block index */
+
+ enum gce_rec_block_enum curr_block; /* Current block type */
+ enum gce_rec_mode_enum curr_mode; /* Current recording mode */
+
+ u32 orig_index; /* Original index */
+ u32 *p_orig_out; /* Original output buffer pointer */
+ u32 curr_label; /* Current label index */
+
+ enum gce_rec_block_enum label_block; /* Current label block type */
+
+ u32 frame_label[GCE_REC_MAX_LABEL_COUNT]; /* Frame labels */
+ u32 frame_count; /* Number of frame labels */
+
+ u32 tile_label[GCE_REC_MAX_LABEL_COUNT]; /* Tile labels */
+ u32 tile_count; /* Number of tile labels */
+
+ u32 footer_code; /* Footer guard word */
+};
+#endif /* _MTK_IMGSYS_CMDQ_PLAT_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
new file mode 100644
index 000000000000..7628ade90b6b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.c
@@ -0,0 +1,690 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+#include <linux/dma-mapping.h>
+#include <linux/mailbox_controller.h>
+#include <linux/math64.h>
+#include <linux/platform_device.h>
+#include <linux/soc/mediatek/mtk-cmdq.h>
+#include <linux/pm_opp.h>
+#include <linux/pm_runtime.h>
+#include <linux/regulator/consumer.h>
+#include <linux/vmalloc.h>
+#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-cmdq-plat.h"
+#include "mtk_imgsys-module_common.h"
+
+#define MAX_FRAME_IN_TASK 64
+#define CMDQ_BUF_SIZE 0x4000
+
+static struct workqueue_struct *imgsys_cmdq_wq;
+static u32 is_stream_off;
+
+/**
+ * enum mtk_imgsys_cmd - Commands for image system operations
+ * @IMGSYS_CMD_WRITE: Write operation
+ * @IMGSYS_CMD_POLL: Poll operation
+ * @IMGSYS_CMD_WAIT: Wait operation
+ * @IMGSYS_CMD_UPDATE: Update operation
+ * @IMGSYS_CMD_ACQUIRE: Acquire operation
+ * @IMGSYS_CMD_STOP: Stop operation
+ *
+ * This enumeration defines the various commands that determine how to use the
+ * cmdq API for image system operations.
+ */
+enum mtk_imgsys_cmd {
+ IMGSYS_CMD_WRITE,
+ IMGSYS_CMD_POLL,
+ IMGSYS_CMD_WAIT,
+ IMGSYS_CMD_UPDATE,
+ IMGSYS_CMD_ACQUIRE,
+ IMGSYS_CMD_STOP
+};
+
+/**
+ * struct access - Access parameters for image system commands
+ * @address: Memory address to access
+ * @value: Value to write to the memory address
+ * @mask: Mask for the value
+ *
+ * This structure defines the parameters for memory access operations used by
+ * IMGSYS_CMD_WRITE and IMGSYS_CMD_POLL commands.
+ */
+struct access {
+ u64 address;
+ u32 value;
+ u32 mask;
+} __packed;
+
+/**
+ * struct event - Event parameters for image system commands
+ * @event: Event identifier
+ * @action: Action to perform for the event
+ *
+ * This structure defines the parameters for event-based operations used by
+ * IMGSYS_CMD_WAIT, IMGSYS_CMD_UPDATE and IMGSYS_CMD_ACQUIRE commands.
+ */
+struct event {
+ u32 event;
+ u32 action;
+} __packed;
+
+/**
+ * struct command - Command structure for image system operations
+ * @opcode: Operation code (command)
+ * @param: Union of parameters for different commands
+ * @write: Parameters for write command
+ * @poll: Parameters for poll command
+ * @wait: Parameters for wait command
+ * @update: Parameters for update command
+ * @acquire: Parameters for acquire command
+ *
+ * This structure is used to store the instructions and parameters for how to
+ * use the cmdq driver.
+ */
+struct command {
+ enum mtk_imgsys_cmd opcode;
+ union {
+ struct access write;
+ struct access poll;
+ struct event wait;
+ struct event update;
+ struct event acquire;
+ } param;
+} __packed;
+
+/**
+ * struct mtk_imgsys_cb_param - Callback parameters for image system commands
+ * @cmdq_cb_work: Work structure for command queue callback
+ * @pkt: Command packet
+ * @frm_info: Pointer to frame information
+ * @imgsys_dev: Pointer to image system device
+ * @clt: Pointer to command queue client
+ * @user_cmdq_cb: User-defined command queue callback function
+ * @err: Error code
+ * @frm_idx: Frame index
+ * @frm_num: Number of frames
+ * @blk_idx: Block index
+ * @blk_num: Number of blocks
+ * @is_earlycb: Indicates if it is an early callback
+ * @group_id: Group identifier
+ * @thd_idx: Thread index
+ * @task_id: Task identifier
+ * @task_num: Number of tasks
+ * @task_cnt: Task count
+ * @pkt_ofst: Array of packet offsets for frames in task
+ * @is_blk_last: Indicates if it is the last block
+ * @is_frm_last: Indicates if it is the last frame
+ * @is_task_last: Indicates if it is the last task
+ *
+ * This structure defines the parameters for callbacks related to
+ * image system commands.
+ */
+struct mtk_imgsys_cb_param {
+ struct work_struct cmdq_cb_work;
+ struct cmdq_pkt pkt;
+ struct swfrm_info *frm_info;
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct cmdq_client *clt;
+ void (*user_cmdq_cb)(struct imgsys_cmdq_cb_data data);
+ s32 err;
+ u32 frm_idx;
+ u32 frm_num;
+ u32 blk_idx;
+ u32 blk_num;
+ u32 is_earlycb;
+ s32 group_id;
+ u32 thd_idx;
+ u32 task_id;
+ u32 task_num;
+ u32 task_cnt;
+ size_t pkt_ofst[MAX_FRAME_IN_TASK];
+ bool is_blk_last;
+ bool is_frm_last;
+ bool is_task_last;
+};
+
+static void imgsys_cmdq_task_cb(struct mbox_client *cl, void *mssg);
+static struct cmdq_client *imgsys_clt[IMGSYS_ENG_MAX];
+
+void imgsys_cmdq_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct device *dev = imgsys_dev->dev;
+ u32 idx = 0;
+
+ /* Only first user has to allocate the work queue */
+ if (!imgsys_cmdq_wq) {
+ imgsys_cmdq_wq =
+ alloc_ordered_workqueue("%s", __WQ_LEGACY |
+ WQ_MEM_RECLAIM |
+ WQ_FREEZABLE,
+ "imgsys_cmdq_cb_wq");
+ if (!imgsys_cmdq_wq) {
+ dev_dbg(imgsys_dev->dev,
+ "%s:allcate cmdq callback work queue fail!\n",
+ __func__);
+ return;
+ }
+ }
+
+ /* Request mailbox channels for each IMGSYS engine */
+ for (idx = 0; idx < IMGSYS_ENG_MAX; idx++) {
+ imgsys_clt[idx] = kzalloc(sizeof(*imgsys_clt[idx]), GFP_KERNEL);
+ if (!imgsys_clt[idx])
+ return;
+
+ imgsys_clt[idx]->client.dev = dev;
+ imgsys_clt[idx]->client.tx_block = false;
+ imgsys_clt[idx]->client.knows_txdone = true;
+ imgsys_clt[idx]->client.rx_callback = imgsys_cmdq_task_cb;
+ imgsys_clt[idx]->chan =
+ mbox_request_channel(&imgsys_clt[idx]->client, idx);
+ }
+
+ /* Parse events from dts */
+ for (idx = 0; idx < IMGSYS_EVENT_MAX; idx++)
+ of_property_read_u16(dev->of_node,
+ imgsys_event[idx].dts_name,
+ &imgsys_event[idx].event);
+}
+
+void imgsys_cmdq_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ u32 idx = 0;
+
+ /* Destroy cmdq client */
+ for (idx = 0; idx < IMGSYS_ENG_MAX; idx++) {
+ cmdq_mbox_destroy(imgsys_clt[idx]);
+ imgsys_clt[idx] = NULL;
+ }
+
+ /* Release work_quque */
+ flush_workqueue(imgsys_cmdq_wq);
+ destroy_workqueue(imgsys_cmdq_wq);
+ imgsys_cmdq_wq = NULL;
+}
+
+void imgsys_cmdq_streamon(struct mtk_imgsys_dev *imgsys_dev)
+{
+ dev_info(imgsys_dev->dev, "%s: cmdq stream on (%d)\n", __func__, is_stream_off);
+ is_stream_off = 0;
+}
+
+void imgsys_cmdq_streamoff(struct mtk_imgsys_dev *imgsys_dev)
+{
+ dev_info(imgsys_dev->dev, "%s: cmdq stream off (%d)\n", __func__, is_stream_off);
+ is_stream_off = 1;
+}
+
+static void imgsys_cmdq_cmd_dump(struct swfrm_info *frm_info, u32 frm_idx)
+{
+ struct gce_recorder *cmd_buf = NULL;
+ struct command *cmd = NULL;
+ u32 cmd_num = 0;
+ u32 cmd_idx = 0;
+
+ cmd_buf = (struct gce_recorder *)frm_info->user_info[frm_idx].g_swbuf;
+ cmd_num = cmd_buf->curr_length / sizeof(struct command);
+
+ if (sizeof(struct gce_recorder) != (u64)cmd_buf->cmd_offset) {
+ pr_info("%s: [ERROR] cmd offset is not match (0x%x/0x%zx)!\n",
+ __func__, cmd_buf->cmd_offset,
+ sizeof(struct gce_recorder));
+ return;
+ }
+
+ cmd = (struct command *)((char *)(frm_info->user_info[frm_idx].g_swbuf) +
+ (unsigned long)(cmd_buf->cmd_offset));
+
+ for (cmd_idx = 0; cmd_idx < cmd_num; cmd_idx++) {
+ switch (cmd[cmd_idx].opcode) {
+ case IMGSYS_CMD_WRITE:
+ pr_info("WRITE with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ cmd[cmd_idx].param.write.address,
+ cmd[cmd_idx].param.write.value,
+ cmd[cmd_idx].param.write.mask);
+ break;
+ case IMGSYS_CMD_POLL:
+ pr_info("POLL with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ cmd[cmd_idx].param.poll.address,
+ cmd[cmd_idx].param.poll.value,
+ cmd[cmd_idx].param.poll.mask);
+ break;
+ case IMGSYS_CMD_WAIT:
+ pr_info("WAIT event(%d/%d) action(%d)\n",
+ cmd[cmd_idx].param.wait.event,
+ imgsys_event[cmd[cmd_idx].param.wait.event].event,
+ cmd[cmd_idx].param.wait.action);
+ break;
+ case IMGSYS_CMD_UPDATE:
+ pr_info("UPDATE event(%d/%d) action(%d)\n",
+ cmd[cmd_idx].param.update.event,
+ imgsys_event[cmd[cmd_idx].param.update.event].event,
+ cmd[cmd_idx].param.update.action);
+ break;
+ case IMGSYS_CMD_ACQUIRE:
+ pr_info("ACQUIRE event(%d/%d) action(%d)\n",
+ cmd[cmd_idx].param.acquire.event,
+ imgsys_event[cmd[cmd_idx].param.acquire.event].event,
+ cmd[cmd_idx].param.acquire.action);
+ break;
+ case IMGSYS_CMD_STOP:
+ pr_info("%s: End Of Cmd!\n", __func__);
+ break;
+ default:
+ pr_info("%s: [ERROR]Not Support Cmd(%d)!\n",
+ __func__, cmd[cmd_idx].opcode);
+ break;
+ }
+ }
+}
+
+static void imgsys_cmdq_cb_work(struct work_struct *work)
+{
+ struct mtk_imgsys_cb_param *cb_param = NULL;
+ struct mtk_imgsys_dev *imgsys_dev = NULL;
+ u32 cb_frm_cnt;
+
+ cb_param = container_of(work, struct mtk_imgsys_cb_param, cmdq_cb_work);
+ imgsys_dev = cb_param->imgsys_dev;
+
+ if (cb_param->err != 0) {
+ pr_info("%s: [ERROR] cb(%p) cb_cnt(%d) req fd/no(%d/%d) error(%d).\n",
+ __func__, cb_param,
+ cb_param->frm_info->cb_frmcnt,
+ cb_param->frm_info->request_fd,
+ cb_param->frm_info->request_no,
+ cb_param->err);
+
+ pr_info("%s: Frame Info: frm(%d/%d) blk(%d/%d) earlycb(%d) lst(%d/%d/%d).\n",
+ __func__, cb_param->frm_idx, cb_param->frm_num,
+ cb_param->blk_idx, cb_param->blk_num, cb_param->is_earlycb,
+ cb_param->is_blk_last, cb_param->is_frm_last, cb_param->is_task_last);
+
+ pr_info("%s: CMDQ Info: task(%d/%d/%d) ofst(%zx/%zx/%zx/%zx/%zx).\n",
+ __func__, cb_param->task_id, cb_param->task_num, cb_param->task_cnt,
+ cb_param->pkt_ofst[0], cb_param->pkt_ofst[1], cb_param->pkt_ofst[2],
+ cb_param->pkt_ofst[3], cb_param->pkt_ofst[4]);
+ }
+
+ if (is_stream_off == 1)
+ pr_info("%s: [ERROR] cb(%p) pipe already streamoff(%d)!\n",
+ __func__, cb_param, is_stream_off);
+
+ cb_param->frm_info->cb_frmcnt++;
+ cb_frm_cnt = cb_param->frm_info->cb_frmcnt;
+
+ if (cb_param->is_blk_last && cb_param->user_cmdq_cb &&
+ (cb_param->frm_info->total_taskcnt == cb_frm_cnt ||
+ cb_param->is_earlycb)) {
+ struct imgsys_cmdq_cb_data user_cb_data;
+
+ user_cb_data.cmdq_data.sta = cb_param->err;
+ user_cb_data.cmdq_data.pkt = NULL;
+ user_cb_data.data = (void *)cb_param->frm_info;
+ cb_param->user_cmdq_cb(user_cb_data);
+ }
+
+ cmdq_pkt_destroy(cb_param->clt, &cb_param->pkt);
+ vfree(cb_param);
+}
+
+static void imgsys_cmdq_task_cb(struct mbox_client *cl, void *mssg)
+{
+ struct cmdq_cb_data *data = mssg;
+ struct mtk_imgsys_cb_param *cb_param =
+ container_of(data->pkt, struct mtk_imgsys_cb_param, pkt);
+
+ struct mtk_imgsys_pipe *pipe;
+ u32 frm_idx = 0;
+
+ cb_param->err = data->sta;
+
+ pr_debug("%s: Receive cb(%p) with sta(%d) for frm(%d/%d)\n",
+ __func__, cb_param, data->sta, cb_param->frm_idx,
+ cb_param->frm_num);
+
+ if (cb_param->err != 0) {
+ frm_idx = cb_param->frm_idx;
+ pr_info("%s: [ERROR] cb(%p) error info(%d).\n",
+ __func__, cb_param, cb_param->err);
+
+ pr_info("%s: Frame Info: frm(%d/%d) blk(%d/%d) earlycb(%d) lst(%d/%d/%d).\n",
+ __func__, cb_param->frm_idx, cb_param->frm_num,
+ cb_param->blk_idx, cb_param->blk_num, cb_param->is_earlycb,
+ cb_param->is_blk_last, cb_param->is_frm_last, cb_param->is_task_last);
+
+ pr_info("%s: CMDQ Info: task(%d/%d/%d) ofst(%zx/%zx/%zx/%zx/%zx).\n",
+ __func__, cb_param->task_id, cb_param->task_num, cb_param->task_cnt,
+ cb_param->pkt_ofst[0], cb_param->pkt_ofst[1], cb_param->pkt_ofst[2],
+ cb_param->pkt_ofst[3], cb_param->pkt_ofst[4]);
+
+ if (is_stream_off == 1)
+ pr_info("%s: [ERROR] cb(%p) pipe had been turned off(%d)!\n",
+ __func__, cb_param, is_stream_off);
+
+ pipe = (struct mtk_imgsys_pipe *)cb_param->frm_info->pipe;
+ if (!pipe->streaming) {
+ pr_info("%s: [ERROR] cb(%p): pipe already streamoff\n",
+ __func__, cb_param);
+ }
+
+ imgsys_cmdq_cmd_dump(cb_param->frm_info, frm_idx);
+ }
+
+ INIT_WORK(&cb_param->cmdq_cb_work, imgsys_cmdq_cb_work);
+ queue_work(imgsys_cmdq_wq, &cb_param->cmdq_cb_work);
+}
+
+static int imgsys_cmdq_parser(struct cmdq_pkt *pkt,
+ struct command *cmd, u32 cmd_num)
+{
+ bool stop = false;
+ int count = 0;
+ u32 event_id;
+
+ do {
+ switch (cmd->opcode) {
+ case IMGSYS_CMD_WRITE:
+ if (cmd->param.write.address < IMGSYS_REG_START ||
+ cmd->param.write.address > IMGSYS_REG_END) {
+ pr_info("%s: [ERROR] WRITE: wrong address(0x%08llx)!\n",
+ __func__, cmd->param.poll.address);
+ return -1;
+ }
+ pr_debug("%s: WRITE with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ __func__, cmd->param.write.address,
+ cmd->param.write.value,
+ cmd->param.write.mask);
+
+ cmdq_pkt_assign(pkt, CMDQ_THR_SPR_IDX0,
+ CMDQ_ADDR_HIGH(cmd->param.write.address));
+ if (cmd->param.write.mask != GENMASK(31, 0))
+ cmdq_pkt_write_s_mask_value(pkt, CMDQ_THR_SPR_IDX0,
+ CMDQ_ADDR_LOW(cmd->param.write.address),
+ cmd->param.write.value,
+ cmd->param.write.mask);
+ else
+ cmdq_pkt_write_s_value(pkt, CMDQ_THR_SPR_IDX0,
+ CMDQ_ADDR_LOW(cmd->param.write.address),
+ cmd->param.write.value);
+ break;
+ case IMGSYS_CMD_POLL:
+ if (cmd->param.poll.address < IMGSYS_REG_START ||
+ cmd->param.poll.address > IMGSYS_REG_END) {
+ pr_info("%s: [ERROR] POLL: wrong address(0x%08llx)!\n",
+ __func__, cmd->param.poll.address);
+ return -1;
+ }
+ pr_debug("%s: POLL with addr(0x%08llx) value(0x%08x) mask(0x%08x)\n",
+ __func__, cmd->param.poll.address,
+ cmd->param.poll.value, cmd->param.poll.mask);
+
+ cmdq_pkt_poll_addr(pkt, cmd->param.poll.address,
+ cmd->param.poll.value,
+ cmd->param.poll.mask);
+ break;
+ case IMGSYS_CMD_WAIT:
+ event_id = cmd->param.wait.event;
+ if (event_id >= IMGSYS_EVENT_MAX) {
+ pr_info("%s: [ERROR] WAIT: wrong event(%d)!\n",
+ __func__, event_id);
+ return -1;
+ }
+ pr_debug("%s: WAIT event(%d/%d) action(%d)\n",
+ __func__, event_id,
+ imgsys_event[event_id].event,
+ cmd->param.wait.action);
+
+ if (cmd->param.wait.action == 1)
+ cmdq_pkt_wfe(pkt, imgsys_event[event_id].event, true);
+ else if (cmd->param.wait.action == 0)
+ cmdq_pkt_wfe(pkt, imgsys_event[event_id].event, false);
+
+ break;
+ case IMGSYS_CMD_UPDATE:
+ event_id = cmd->param.update.event;
+ if (event_id >= IMGSYS_EVENT_MAX) {
+ pr_info("%s: [ERROR] UPDATE: wrong event(%d)!\n",
+ __func__, event_id);
+ return -1;
+ }
+
+ pr_debug("%s: UPDATE event(%d/%d) action(%d)\n",
+ __func__, event_id,
+ imgsys_event[event_id].event,
+ cmd->param.update.action);
+
+ if (cmd->param.update.action == 1)
+ cmdq_pkt_set_event(pkt, imgsys_event[event_id].event);
+ else if (cmd->param.update.action == 0)
+ cmdq_pkt_clear_event(pkt, imgsys_event[event_id].event);
+
+ break;
+ case IMGSYS_CMD_ACQUIRE:
+ event_id = cmd->param.acquire.event;
+ if (event_id >= IMGSYS_EVENT_MAX) {
+ pr_info("%s: [ERROR] ACQUIRE: wrong event(%d)!\n",
+ __func__, event_id);
+ return -1;
+ }
+
+ pr_debug("%s: ACQUIRE event(%d/%d) action(%d)\n",
+ __func__, event_id,
+ imgsys_event[event_id].event,
+ cmd->param.acquire.action);
+
+ cmdq_pkt_acquire_event(pkt, imgsys_event[event_id].event);
+ break;
+ case IMGSYS_CMD_STOP:
+ pr_debug("%s: End Of Cmd!\n", __func__);
+ stop = true;
+ break;
+ default:
+ pr_info("%s: [ERROR] Not Support Cmd(%d)!\n", __func__, cmd->opcode);
+ return -1;
+ }
+ cmd++;
+ count++;
+ } while ((!stop) && (count < cmd_num));
+
+ return count;
+}
+
+int imgsys_cmdq_sendtask(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *frm_info,
+ void (*cmdq_cb)(struct imgsys_cmdq_cb_data data))
+{
+ struct cmdq_client *clt = NULL;
+ struct gce_recorder *cmd_buf = NULL;
+ struct command *cmd = NULL;
+ struct mtk_imgsys_cb_param *cb_param = NULL;
+ u32 cmd_num = 0;
+ u32 cmd_idx = 0;
+ u32 blk_idx = 0; /* For Vss block cnt */
+ u32 thd_idx = 0;
+ u32 hw_comb = 0;
+ int ret = 0, ret_flush = 0;
+ u32 frm_num = 0, frm_idx = 0;
+ bool is_pack = 0;
+ u32 task_idx = 0;
+ u32 task_id = 0;
+ u32 task_num = 0;
+ u32 task_cnt = 0;
+ size_t pkt_ofst[MAX_FRAME_IN_TASK] = {0};
+
+ frm_num = frm_info->total_frmnum;
+ frm_info->cb_frmcnt = 0;
+ frm_info->total_taskcnt = 0;
+
+ for (frm_idx = 0; frm_idx < frm_num; frm_idx++) {
+ cmd_buf = (struct gce_recorder *)frm_info->user_info[frm_idx].g_swbuf;
+
+ /* Validate the GCE recorder structure */
+ if (cmd_buf->header_code != HEADER_CODE ||
+ cmd_buf->check_pre != CHECK_PRE ||
+ cmd_buf->check_post != CHECK_POST ||
+ cmd_buf->footer_code != FOOTER_CODE) {
+ pr_info("%s: Incorrect guard word: %08x/%08x/%08x/%08x",
+ __func__, cmd_buf->header_code, cmd_buf->check_pre,
+ cmd_buf->check_post, cmd_buf->footer_code);
+ return -1;
+ }
+
+ cmd_num = cmd_buf->curr_length / sizeof(struct command);
+ cmd = (struct command *)((char *)frm_info->user_info[frm_idx].g_swbuf
+ + (unsigned long)cmd_buf->cmd_offset);
+
+ hw_comb = frm_info->user_info[frm_idx].hw_comb;
+
+ if (is_pack == 0) {
+ if (frm_info->group_id == -1) {
+ /* Choose cmdq_client base on hw scenario */
+ for (thd_idx = 0; thd_idx < IMGSYS_ENG_MAX; thd_idx++) {
+ if (hw_comb & 0x1) {
+ clt = imgsys_clt[thd_idx];
+ pr_debug("%s: mbox thread (%d, %p)\n",
+ __func__, thd_idx, clt);
+ pr_debug("%s: frm(%d/%d)\n",
+ __func__, frm_idx, frm_num);
+ break;
+ }
+ hw_comb = hw_comb >> 1;
+ }
+ /* This segment can be removed since user had set dependency */
+ if (frm_info->user_info[frm_idx].hw_comb & IMGSYS_ENG_DIP) {
+ thd_idx = 4;
+ clt = imgsys_clt[thd_idx];
+ }
+ } else {
+ if (frm_info->group_id >= 0 &&
+ frm_info->group_id < IMGSYS_NOR_THD) {
+ thd_idx = frm_info->group_id;
+ clt = imgsys_clt[thd_idx];
+ } else {
+ pr_info("%s: [ERROR] Group id(%d) is not in range(%d).\n",
+ __func__, frm_info->group_id, IMGSYS_NOR_THD);
+ pr_info("%s: [ERROR] Frame Info: frm(%d/%d) hw_comb(%d).\n",
+ __func__, frm_idx, frm_num,
+ frm_info->user_info[frm_idx].hw_comb);
+ return -1;
+ }
+ }
+
+ /* This is work around for low latency flow. */
+ /* If we change to request base, */
+ /* we don't have to take this condition into account. */
+ if (frm_info->sync_id != -1) {
+ thd_idx = 0;
+ clt = imgsys_clt[thd_idx];
+ }
+
+ if (!clt) {
+ pr_info("%s: [ERROR] No HW Found (0x%x) for frm(%d/%d)!\n",
+ __func__, frm_info->user_info[frm_idx].hw_comb,
+ frm_idx, frm_num);
+ return -1;
+ }
+ }
+
+ cmd_idx = 0;
+
+ for (blk_idx = 0; blk_idx < cmd_buf->frame_block; blk_idx++) {
+ if (is_pack == 0) {
+ /* Prepare cb param */
+ cb_param = vzalloc(sizeof(*cb_param));
+ if (!cb_param)
+ return -1;
+
+ /* create pkt and hook clt as pkt's private data */
+ if (cmdq_pkt_create(clt, &cb_param->pkt, CMDQ_BUF_SIZE)) {
+ vfree(cb_param);
+ pr_info("%s: [ERROR] Id %d imgsys_create_pkt fail!\n",
+ __func__, blk_idx);
+ return -1;
+ }
+ }
+
+ ret = imgsys_cmdq_parser(&cb_param->pkt, &cmd[cmd_idx], cmd_num);
+
+ if (ret < 0) {
+ pr_info("%s: [ERROR] Parsing fail. cmd(%d/%d) frm(%d/%d/%d)\n",
+ __func__, cmd_idx, cmd[cmd_idx].opcode,
+ blk_idx, frm_idx, frm_num);
+
+ cmdq_pkt_destroy(clt, &cb_param->pkt);
+ vfree(cb_param);
+ goto sendtask_done;
+ }
+ cmd_idx += ret;
+
+ /* Check for packing gce task */
+ pkt_ofst[task_cnt] = cb_param->pkt.cmd_buf_size - CMDQ_INST_SIZE;
+ task_cnt++;
+ if (frm_info->user_info[frm_idx].is_time_shared ||
+ frm_info->user_info[frm_idx].is_earlycb ||
+ frm_idx + 1 == frm_num) {
+ task_num++;
+ cb_param->frm_info = frm_info;
+ cb_param->frm_idx = frm_idx;
+ cb_param->frm_num = frm_num;
+ cb_param->user_cmdq_cb = cmdq_cb;
+ if ((blk_idx + 1) == cmd_buf->frame_block)
+ cb_param->is_blk_last = 1;
+ else
+ cb_param->is_blk_last = 0;
+ if ((frm_idx + 1) == frm_num)
+ cb_param->is_frm_last = 1;
+ else
+ cb_param->is_frm_last = 0;
+ cb_param->blk_idx = blk_idx;
+ cb_param->blk_num = cmd_buf->frame_block;
+ cb_param->is_earlycb = frm_info->user_info[frm_idx].is_earlycb;
+ cb_param->group_id = frm_info->group_id;
+ cb_param->imgsys_dev = imgsys_dev;
+ cb_param->thd_idx = thd_idx;
+ cb_param->clt = clt;
+ cb_param->task_cnt = task_cnt;
+ for (task_idx = 0; task_idx < task_cnt; task_idx++)
+ cb_param->pkt_ofst[task_idx] = pkt_ofst[task_idx];
+ task_cnt = 0;
+ cb_param->task_id = task_id;
+ task_id++;
+ if (cb_param->is_blk_last && cb_param->is_frm_last) {
+ cb_param->is_task_last = 1;
+ cb_param->task_num = task_num;
+ frm_info->total_taskcnt = task_num;
+ } else {
+ cb_param->is_task_last = 0;
+ cb_param->task_num = 0;
+ }
+
+ /* flush synchronized, block API */
+ cmdq_pkt_eoc(&cb_param->pkt);
+ cmdq_pkt_jump_rel(&cb_param->pkt, CMDQ_INST_SIZE,
+ cmdq_get_shift_pa(clt->chan));
+ dma_sync_single_for_device(clt->chan->mbox->dev,
+ cb_param->pkt.pa_base,
+ cb_param->pkt.cmd_buf_size,
+ DMA_TO_DEVICE);
+
+ mbox_send_message(clt->chan, &cb_param->pkt);
+ mbox_client_txdone(clt->chan, 0);
+ if (ret_flush < 0)
+ pr_info("%s: [ERROR] Frame(%d/%d) flushing fail(%d).\n",
+ __func__, ret_flush, frm_idx, frm_num);
+ is_pack = 0;
+ } else {
+ dev_info(imgsys_dev->dev, "%s:is_pack = 1", __func__);
+ is_pack = 1;
+ }
+ }
+ }
+
+sendtask_done:
+ return 0;
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
new file mode 100644
index 000000000000..47bbd6735d0b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-cmdq.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Daniel Huang <daniel.huang@mediatek.com>
+ *
+ */
+#ifndef _MTK_IMGSYS_CMDQ_H_
+#define _MTK_IMGSYS_CMDQ_H_
+
+#include <linux/platform_device.h>
+#include <linux/soc/mediatek/mtk-cmdq.h>
+#include "mtk_imgsys-dev.h"
+
+/* Structure to hold command queue callback data */
+struct imgsys_cmdq_cb_data {
+ struct cmdq_cb_data cmdq_data; /* Command queue data */
+ void *data; /* Additional data */
+};
+
+void imgsys_cmdq_streamon(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_cmdq_streamoff(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_cmdq_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_cmdq_release(struct mtk_imgsys_dev *imgsys_dev);
+int imgsys_cmdq_sendtask(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *frm_info,
+ void (*cmdq_cb)(struct imgsys_cmdq_cb_data data));
+
+#endif /* _MTK_IMGSYS_CMDQ_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
index dd02a8fa28d4..3918e360a5a0 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_common.h
@@ -39,6 +39,26 @@
#define CQ_THRX_CTL_MODE BIT(4)
#define CQ_THRX_CTL (CQ_THRX_CTL_EN | CQ_THRX_CTL_MODE)
+/**
+ * enum mtk_imgsys_engine_bitmask - Bitmasks for supported hardware engines
+ *
+ * This enumeration defines the bitmasks for the supported hardware engines in
+ * the image system.
+ */
+enum mtk_imgsys_engine_bitmask {
+ IMGSYS_ENG_WPE_EIS = 0x00000001, /* Bitmask for WPE EIS engine */
+ IMGSYS_ENG_WPE_TNR = 0x00000002, /* Bitmask for WPE TNR engine */
+ IMGSYS_ENG_WPE_LITE = 0x00000004, /* Bitmask for WPE Lite engine */
+ IMGSYS_ENG_TRAW = 0x00000020, /* Bitmask for TRAW engine */
+ IMGSYS_ENG_LTR = 0x00000040, /* Bitmask for LTR engine */
+ IMGSYS_ENG_XTR = 0x00000080, /* Bitmask for XTR engine */
+ IMGSYS_ENG_DIP = 0x00000100, /* Bitmask for DIP engine */
+ IMGSYS_ENG_PQDIP_A = 0x00000200, /* Bitmask for PQDIP A engine */
+ IMGSYS_ENG_PQDIP_B = 0x00000400, /* Bitmask for PQDIP B engine */
+ IMGSYS_ENG_ME = 0x00000800, /* Bitmask for ME engine */
+ IMGSYS_ENG_MAX_NUM = 0x00001000 /* Maximum bitmask value */
+};
+
/**
* enum mtk_imgsys_reg_id
*
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 9f118491b37d..184f0e3a4d3b 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -9,6 +9,7 @@
#include <linux/pm_runtime.h>
#include "mtk_imgsys-sys.h"
#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-cmdq.h"
static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
{
@@ -26,6 +27,8 @@ static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
kref_init(&imgsys_dev->init_kref);
+ /* calling cmdq stream on */
+ imgsys_cmdq_streamon(imgsys_dev);
return 0;
}
@@ -43,6 +46,8 @@ static void module_uninit(struct kref *kref)
static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
{
+ /* calling cmdq stream off */
+ imgsys_cmdq_streamoff(imgsys_dev);
kref_put(&imgsys_dev->init_kref, module_uninit);
}
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 08/10] media: mediatek: isp: Add SCP support for ImgSys driver
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (5 preceding siblings ...)
2025-05-24 11:49 ` [PATCH v1 07/10] media: mediatek: isp: Add CMDQ support for ImgSys driver Olivia Wen
@ 2025-05-24 11:50 ` Olivia Wen
2025-05-24 11:50 ` [PATCH v1 09/10] media: mediatek: isp: Add image processing flow Olivia Wen
` (2 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the SCP support for ImgSys driver. It covers the memory
allocation and transmission interfaces required by SCP. Frame-related
information is transmitted to SCP, where it is packaged into commands
and the necessary hardware information. These commands are then sent to
the kernel for further processing.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 1 +
.../mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h | 143 +++++++
.../isp/isp_7x/imgsys/mtk_imgsys-scp.c | 349 ++++++++++++++++++
.../isp/isp_7x/imgsys/mtk_imgsys-scp.h | 194 ++++++++++
4 files changed, 687 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index c0f0f3a0c38a..29c18c773fdc 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -3,6 +3,7 @@
mtk_imgsys_util-objs := \
mtk_imgsys-cmdq.o \
+mtk_imgsys-scp.o \
mtk_imgsys-module_main.o \
mtk_imgsys-of.o \
mtk_imgsys-formats.o \
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
new file mode 100644
index 000000000000..850f6ec99a96
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef __MTK_IMG_IPI_H__
+#define __MTK_IMG_IPI_H__
+
+#include <linux/types.h>
+#include "mtk_header_desc.h"
+
+#define HEADER_VER 19481
+#define IMG_MAX_HW_DMAS 72
+#define IMG_MAX_HW 12
+#define WPE_ENG_NUM 3
+#define TMAX 16
+
+/**
+ * enum img_module_id - Enumeration of image processing module IDs
+ * @IMG_MODULE_WPE: ID for the WPE (Warp Engine) module
+ * @IMG_MODULE_TRAW: ID for the TRAW (Tile Raw) module
+ * @IMG_MODULE_DIP: ID for the DIP (Digital Image Processing) module
+ * @IMG_MODULE_PQDIP: ID for the PQDIP (Picture Quality Digital Image Processing) module
+ * @IMG_MODULE_MAX: Maximum ID value, used for boundary checking
+ */
+enum img_module_id {
+ IMG_MODULE_WPE = 0,
+ IMG_MODULE_TRAW = 1,
+ IMG_MODULE_DIP = 2,
+ IMG_MODULE_PQDIP = 3,
+ IMG_MODULE_MAX = 4,
+};
+
+/**
+ * struct module_init_info - Information about module initialization
+ * @c_wbuf: Command queue (CQ) working buffer address
+ * @c_wbuf_dma: DMA address of the CQ working buffer
+ * @c_wbuf_sz: Size of the CQ working buffer
+ * @t_wbuf: Tile dirvier (TDR) working buffer address
+ * @t_wbuf_dma: DMA address of the TDR working buffer
+ * @t_wbuf_sz: Size of the TDR working buffer
+ */
+struct module_init_info {
+ u64 c_wbuf; /* CQ working buffer address */
+ u64 c_wbuf_dma; /* DMA address of the CQ working buffer */
+ u32 c_wbuf_sz; /* Size of the CQ working buffer */
+ u64 t_wbuf; /* TDR working buffer address */
+ u64 t_wbuf_dma; /* DMA address of the TDR working buffer */
+ u32 t_wbuf_sz; /* Size of the TDR working buffer */
+} __packed;
+
+/**
+ * struct img_init_info - Information about image initialization
+ * @header_version: Version of the header
+ * @isp_version: ISP version
+ * @frameparam_size: Size of the frame parameters
+ * @module_info: Array of module working buffer information
+ * @g_wbuf: Global Command Engine (GCE) working buffer address
+ * @g_wbuf_sz: Size of the GCE working buffer
+ *
+ * This structure is used as the parameter for IPI_IMGSYS_INIT_ID and
+ * IPI_IMGSYS_DEINIT_ID. It contains information necessary for initializing and
+ * deinitializing the image system.
+ */
+struct img_init_info {
+ u32 header_version;
+ u32 isp_version;
+ u32 frameparam_size;
+ struct module_init_info module_info[IMG_MODULE_MAX];
+ u64 g_wbuf;
+ u32 g_wbuf_sz;
+} __packed;
+
+/**
+ * struct img_swfrm_info - Software frame information structure for image system
+ * @hw_comb: Hardware combination, which used to store the combination of
+ * enum mtk_imgsys_engine_bitmask
+ * @sw_ridx: Software read index, provided to SCP upon notification to access
+ * GCE buffer location
+ * @is_time_shared: Indicates if the frame is time-shared
+ * @is_earlycb: Indicates if early callback is enabled
+ * @sw_goft: offset of GCE buffer
+ * @g_swbuf: virtual address of GCE buffer
+ *
+ * These fields are primarily set by SCP (System Control Processor) and used by
+ * the kernel for processing and verifing data.
+ */
+struct img_swfrm_info {
+ u32 hw_comb;
+ int sw_ridx;
+ u8 is_time_shared;
+ u8 is_earlycb;
+ u64 sw_goft;
+ void *g_swbuf;
+} __packed;
+
+/**
+ * struct img_ipi_frameparam - Frame parameters for IMGSYS IPI
+ * @dmas_enable: Array indicating enabled DMAs for each hardware unit
+ * @dmas: Array of DMA descriptors for each hardware unit
+ * @tuning_meta: Descriptor for tuning metadata
+ * @ctrl_meta: Descriptor for control metadata
+ *
+ * This structure defines the frame parameters used in the IMGSYS IPI
+ * (Inter-Processor Interface). It includes arrays to indicate enabled DMAs and
+ * their descriptors, as well as descriptors for tuning and control metadata.
+ */
+struct img_ipi_frameparam {
+ u8 dmas_enable[IMG_MAX_HW_DMAS];
+ struct header_desc_norm dmas[IMG_MAX_HW_DMAS];
+ struct header_desc_norm tuning_meta;
+ struct header_desc_norm ctrl_meta;
+};
+
+/**
+ * struct img_sw_buffer - Software buffer address information
+ * @scp_addr: Physical address accessed by SCP
+ * @offset: Address offset
+ *
+ * This structure defines the address information of a software buffer used in
+ * the SCP.
+ */
+struct img_sw_buffer {
+ u32 scp_addr;
+ u32 offset;
+} __packed;
+
+/**
+ * struct img_ipi_param - Parameters for IMGSYS IPI
+ * @frm_param: Software buffer adreess for frame parameters
+ * @req_addr_va: Virtual address for the request
+ *
+ * This structure defines the parameters used in the IMGSYS IPI, including a
+ * software buffer physical address for frame parameters and a virtual address
+ * for the request.
+ */
+struct img_ipi_param {
+ struct img_sw_buffer frm_param;
+ u64 req_addr_va;
+} __packed;
+
+#endif /* __MTK_IMG_IPI_H__ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
new file mode 100644
index 000000000000..4e6126d9f28c
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.c
@@ -0,0 +1,349 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+#include <linux/slab.h>
+#include <linux/remoteproc.h>
+#include <linux/dma-direction.h>
+#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-scp.h"
+
+#define SCP_TIMEOUT_MS 4000U
+
+/**
+ * struct mtk_scp_reserve_memblock - Represents a memory block reserved by the
+ * kernel for usage by the SCP (System Control Processor).
+ *
+ * @id: Unique identifier for the memory block. This ID is used to
+ * reference and manage the memory block within the system.
+ * @start_phys: Physical address the starting point of the buffer.
+ * @start_virt: Kernel virtual address of the starting point of the buffer.
+ * @start_dma: DMA (Direct Memory Access) address of the starting point of the
+ * buffer.
+ * @size: Size of the allocated buffer, in bytes.
+ * @dma_dir: Direction of DMA data transfer, indicating whether the data is
+ * being transferred to the device, from the device, or
+ * bidirectionally.
+ */
+struct mtk_scp_reserve_memblock {
+ unsigned int id;
+ phys_addr_t start_phys;
+ void *start_virt;
+ phys_addr_t start_dma;
+ unsigned int size;
+ enum dma_data_direction dma_dir;
+};
+
+/* Array of reserved memory blocks for various hardware components */
+static struct mtk_scp_reserve_memblock reserve_memblock[] = {
+ {
+ /*WPE CQ buffer for WPE HW usage*/
+ .id = WPE_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0xE1000, /*900KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*WPE TDR buffer for tile driver usage*/
+ .id = WPE_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x196000, /*1MB + 600KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*TRAW CQ buffer for TRAW driver usage*/
+ .id = TRAW_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x4C8000, /*4MB + 800KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*TRAW TDR buffer for tile driver usage*/
+ .id = TRAW_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x1AC8000, /*26MB + 800KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*DIP CQ buffer for TRAW HW usage*/
+ .id = DIP_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x5C8000, /*5MB + 800KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*DIP TDR buffer for tile driver usage*/
+ .id = DIP_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x1FAF000, /*31MB + 700KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*PQDIP CQ buffer for TRAW HW usage*/
+ .id = PQDIP_MEM_C_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x100000, /*1MB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*PQDIP TDR buffer for tile driver usage*/
+ .id = PQDIP_MEM_T_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x170000, /*1MB + 500KB*/
+ .dma_dir = DMA_TO_DEVICE,
+ },
+ {
+ /*GCE command buffer for GCE HW usage*/
+ .id = IMG_MEM_G_ID,
+ .start_phys = 0x0,
+ .start_virt = NULL,
+ .start_dma = 0x0,
+ .size = 0x1800000,
+ .dma_dir = DMA_BIDIRECTIONAL,
+ },
+};
+
+static phys_addr_t imgsys_scp_get_reserve_mem_phys(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT) {
+ pr_err("[SCP] no reserve memory for %d", id);
+ return 0;
+ }
+ return reserve_memblock[id].start_phys;
+}
+
+static phys_addr_t imgsys_scp_get_reserve_mem_dma(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT) {
+ pr_err("[SCP] no reserve memory for %d", id);
+ return 0;
+ }
+ return reserve_memblock[id].start_dma;
+}
+
+void *imgsys_scp_get_reserve_mem_virt(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT)
+ return NULL;
+ else
+ return reserve_memblock[id].start_virt;
+}
+EXPORT_SYMBOL(imgsys_scp_get_reserve_mem_virt);
+
+unsigned int imgsys_scp_get_reserve_mem_size(enum img_mem_id id)
+{
+ if (id >= IMG_MEM_ID_COUNT) {
+ pr_err("[SCP] no reserve memory for %d", id);
+ return 0;
+ }
+ return reserve_memblock[id].size;
+}
+
+int imgsys_scp_alloc_reserve_mem(struct mtk_imgsys_dev *imgsys_dev)
+{
+ enum img_mem_id id;
+ unsigned int block_num;
+ void *ptr;
+ dma_addr_t addr;
+
+ block_num = ARRAY_SIZE(reserve_memblock);
+ for (id = 0; id < block_num; id++) {
+ ptr = dma_alloc_coherent(imgsys_dev->smem_dev,
+ reserve_memblock[id].size,
+ &addr, GFP_KERNEL);
+ if (!ptr)
+ return -ENOMEM;
+
+ reserve_memblock[id].start_virt = ptr;
+ reserve_memblock[id].start_phys = addr;
+ reserve_memblock[id].start_dma =
+ dma_map_resource(imgsys_dev->dev, addr,
+ reserve_memblock[id].size,
+ reserve_memblock[id].dma_dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(imgsys_dev->dev,
+ reserve_memblock[id].start_dma)) {
+ dev_err(imgsys_dev->dev, "failed to map scp iova\n");
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+int imgsys_scp_free_reserve_mem(struct mtk_imgsys_dev *imgsys_dev)
+{
+ enum img_mem_id id;
+
+ /* release reserved memory */
+ for (id = 0; id < IMG_MEM_ID_COUNT; id++) {
+ dma_unmap_resource(imgsys_dev->dev,
+ reserve_memblock[id].start_dma,
+ reserve_memblock[id].size,
+ reserve_memblock[id].dma_dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ dma_free_coherent(imgsys_dev->smem_dev,
+ reserve_memblock[id].size,
+ reserve_memblock[id].start_virt,
+ reserve_memblock[id].start_phys);
+
+ reserve_memblock[id].start_dma = 0x0;
+ reserve_memblock[id].start_virt = NULL;
+ reserve_memblock[id].start_phys = 0x0;
+ }
+
+ return 0;
+}
+
+int imgsys_scp_get_reserve_mem_info(struct img_init_info *info)
+{
+ if (!info) {
+ pr_err("%s:NULL info\n", __func__);
+ return -EINVAL;
+ }
+
+ /* WPE */
+ info->module_info[IMG_MODULE_WPE].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(WPE_MEM_C_ID);
+ info->module_info[IMG_MODULE_WPE].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(WPE_MEM_C_ID);
+ info->module_info[IMG_MODULE_WPE].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(WPE_MEM_C_ID);
+ info->module_info[IMG_MODULE_WPE].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(WPE_MEM_T_ID);
+ info->module_info[IMG_MODULE_WPE].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(WPE_MEM_T_ID);
+ info->module_info[IMG_MODULE_WPE].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(WPE_MEM_T_ID);
+
+ /* TRAW */
+ info->module_info[IMG_MODULE_TRAW].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(TRAW_MEM_C_ID);
+ info->module_info[IMG_MODULE_TRAW].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(TRAW_MEM_C_ID);
+ info->module_info[IMG_MODULE_TRAW].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(TRAW_MEM_C_ID);
+ info->module_info[IMG_MODULE_TRAW].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(TRAW_MEM_T_ID);
+ info->module_info[IMG_MODULE_TRAW].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(TRAW_MEM_T_ID);
+ info->module_info[IMG_MODULE_TRAW].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(TRAW_MEM_T_ID);
+
+ /* DIP */
+ info->module_info[IMG_MODULE_DIP].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(DIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_DIP].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(DIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_DIP].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(DIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_DIP].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(DIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_DIP].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(DIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_DIP].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(DIP_MEM_T_ID);
+
+ /* PQDIP */
+ info->module_info[IMG_MODULE_PQDIP].c_wbuf =
+ imgsys_scp_get_reserve_mem_phys(PQDIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_PQDIP].c_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(PQDIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_PQDIP].c_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(PQDIP_MEM_C_ID);
+ info->module_info[IMG_MODULE_PQDIP].t_wbuf =
+ imgsys_scp_get_reserve_mem_phys(PQDIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_PQDIP].t_wbuf_dma =
+ imgsys_scp_get_reserve_mem_dma(PQDIP_MEM_T_ID);
+ info->module_info[IMG_MODULE_PQDIP].t_wbuf_sz =
+ imgsys_scp_get_reserve_mem_size(PQDIP_MEM_T_ID);
+
+ /* GCE command */
+ info->g_wbuf = imgsys_scp_get_reserve_mem_phys(IMG_MEM_G_ID);
+ info->g_wbuf_sz = imgsys_scp_get_reserve_mem_size(IMG_MEM_G_ID);
+
+ return 0;
+}
+
+int imgsys_scp_init(struct mtk_imgsys_dev *imgsys_dev,
+ scp_ipi_handler_t imgsys_scp_handler)
+{
+ int ret = 0;
+
+ ret = rproc_boot(imgsys_dev->rproc_handle);
+ if (ret) {
+ dev_err(imgsys_dev->dev,
+ "%s failed to rproc_boot\n", __func__);
+ return ret;
+ }
+
+ ret = scp_ipi_register(imgsys_dev->scp, SCP_IPI_IMGSYS_CMD,
+ imgsys_scp_handler, imgsys_dev);
+ if (ret) {
+ dev_err(imgsys_dev->dev,
+ "%s failed to register IPI cmd\n", __func__);
+ return ret;
+ }
+
+ return ret;
+}
+
+void imgsys_scp_deinit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ rproc_shutdown(imgsys_dev->rproc_handle);
+}
+
+int imgsys_scp_send(struct mtk_imgsys_dev *imgsys_dev,
+ enum imgsys_ipi_id id, void *buf,
+ unsigned int len, int req_fd,
+ unsigned int wait)
+{
+ struct share_buf send_obj = { 0 };
+ int ret = 0;
+
+ dev_dbg(imgsys_dev->dev, "%s id:%d len %d\n",
+ __func__, id, len);
+
+ if (id < IPI_IMGSYS_INIT_ID || id >= IPI_MAX_ID ||
+ len > sizeof(send_obj.share_data) || !buf) {
+ dev_info(imgsys_dev->dev,
+ "%s failed to send scp message (len/sz(%d/%zu)\n",
+ __func__, len, sizeof(send_obj.share_data));
+ return -EINVAL;
+ }
+
+ send_obj.len = len;
+ send_obj.id = id;
+ memcpy((void *)send_obj.share_data, buf, len);
+
+ send_obj.info.send.ipi = id;
+ send_obj.info.send.req = req_fd;
+ send_obj.info.send.ack = (wait ? 1 : 0);
+ send_obj.info.send.seq = 0;
+
+ ret = scp_ipi_send(imgsys_dev->scp, SCP_IPI_IMGSYS_CMD,
+ (void *)&send_obj, sizeof(send_obj),
+ wait ? msecs_to_jiffies(SCP_TIMEOUT_MS) : 0);
+ if (ret)
+ dev_err(imgsys_dev->dev,
+ "%s: send SCP(%d) failed %d\n", __func__, id, ret);
+
+ return 0;
+}
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
new file mode 100644
index 000000000000..2082c4bdf96f
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-scp.h
@@ -0,0 +1,194 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ */
+
+#ifndef MTK_IMGSYS_SCP_H
+#define MTK_IMGSYS_SCP_H
+
+#include <linux/remoteproc/mtk_scp.h>
+#include "mtk-img-ipi.h"
+#include "mtk_imgsys-dev.h"
+
+/* the size of share buffer between the kernel and SCP */
+#define SCP_SHARE_BUF_SIZE 576
+
+/**
+ * struct object_info - Represents information associated with a object,
+ * particularly in the context of inter-processor communication and media
+ * processing.
+ *
+ * @ipi: Specifies the IPI (Inter-Processor Interrupt) ID, using 5 bits.
+ * @ack: Indicates whether an acknowledgment is required, using 1 bit.
+ * @req: Holds the media request ID, using 10 bits.
+ * @seq: A sequence number, using 16 bits, for tracking the order of
+ * objects or messages.
+ * @cmd: Combined command representation.
+ */
+struct object_info {
+ union {
+ struct send {
+ u32 ipi: 5; /* IPI ID */
+ u32 ack: 1; /* Acknowledgment flag */
+ u32 req: 10; /* Media request ID */
+ u32 seq: 16; /* Sequence number */
+ } __packed send;
+ u32 cmd; /* Combined command representation */
+ };
+} __packed;
+
+/**
+ * struct share_buf - DTCM (Data Tightly-Coupled Memory) buffer shared with
+ * AP and hardware components of IMGSYS
+ *
+ * @id: IPI (Inter-Processor Interrupt) ID
+ * @len: Length of the shared buffer
+ * @share_data: Data contained in the shared buffer
+ * @info: Information associated with the object
+ *
+ * This structure represents a DTCM buffer that is shared between the AP and
+ * the hardware components of the MediaTek Image System (IMGSYS). It includes
+ * the IPI ID, the length of the shared buffer, the actual buffer data, and
+ * additional object information.
+ */
+struct share_buf {
+ u32 id; /* IPI (Inter-Processor Interrupt) ID */
+ u32 len; /* Length of the shared buffer */
+ u8 share_data[SCP_SHARE_BUF_SIZE]; /* Data contained in the shared buffer */
+ struct object_info info; /* Information object */
+};
+
+/**
+ * enum img_mem_id - Enumeration of image memory IDs
+ *
+ * @WPE_MEM_C_ID: Warping Engine (WPE) module command queue (CQ) buffer
+ * @WPE_MEM_T_ID: WPE module tile driver (TDR) buffer
+ * @TRAW_MEM_C_ID: Tile-Raw (TRAW) module CQ buffer
+ * @TRAW_MEM_T_ID: TRAW module TDR buffer
+ * @DIP_MEM_C_ID: Digital Image Processing (DIP) module CQ buffer
+ * @DIP_MEM_T_ID: DIP module TDR buffer
+ * @PQDIP_MEM_C_ID: Picture Quality image processing (PQDIP) CQ buffer
+ * @PQDIP_MEM_T_ID: PQDIP TDR buffer
+ * @IMG_MEM_G_ID: GCE command buffer
+ * @IMG_MEM_ID_COUNT: Total number of memory IDs
+ *
+ * This enumeration defines the various memory IDs used in the MediaTek
+ * image system. Each memory ID corresponds to a specific buffer used
+ * by different modules within the image system.
+ */
+enum img_mem_id {
+ WPE_MEM_C_ID, /* WPE module CQ buffer */
+ WPE_MEM_T_ID, /* WPE module TDR buffer */
+ TRAW_MEM_C_ID, /* TRAW module CQ buffer */
+ TRAW_MEM_T_ID, /* TRAW module TDR buffer */
+ DIP_MEM_C_ID, /* DIP module CQ buffer */
+ DIP_MEM_T_ID, /* DIP module TDR buffer */
+ PQDIP_MEM_C_ID, /* PQDIP CQ buffer */
+ PQDIP_MEM_T_ID, /* PQDIP TDR buffer */
+ IMG_MEM_G_ID, /* GCE command buffer */
+ IMG_MEM_ID_COUNT, /* Total number of memory IDs */
+};
+
+/**
+ * enum imgsys_ipi_id - Enumeration of IPI (Inter-Processor Interrupt) IDs
+ * @IPI_IMGSYS_INIT_ID: ID for initializing the image system
+ * @IPI_IMGSYS_FRAME_ID: ID for processing a frame
+ * @IPI_IMGSYS_DEQUE_DONE_ID: ID for indicating frame dequeuing is done
+ * @IPI_IMGSYS_DEINIT_ID: ID for deinitializing the image system
+ * @IPI_MAX_ID: Maximum ID value, used for boundary checking
+ */
+enum imgsys_ipi_id {
+ IPI_IMGSYS_INIT_ID = 0, /* ID for initializing the image system */
+ IPI_IMGSYS_FRAME_ID, /* ID for processing a frame */
+ IPI_IMGSYS_DEQUE_DONE_ID, /* ID for dequeuing frame is done */
+ IPI_IMGSYS_DEINIT_ID, /* ID for deinitializing the image system */
+ IPI_MAX_ID, /* Maximum ID value*/
+};
+
+/**
+ * imgsys_scp_alloc_reserve_mem - allocate reserved memory for use by the
+ * kernel and the SCP (System Control Processor)
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ *
+ **/
+int imgsys_scp_alloc_reserve_mem(struct mtk_imgsys_dev *imgsys_dev);
+
+/**
+ * imgsys_scp_free_reserve_mem - free reserved memory for use by the kernel and
+ * SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ *
+ **/
+int imgsys_scp_free_reserve_mem(struct mtk_imgsys_dev *imgsys_dev);
+
+/**
+ * imgsys_scp_get_reserve_mem_info - get the information about reserved memory
+ *
+ * @info: A pointer to an already allocated structure of type img_init_info,
+ * where this function will populate the reserved memory details.
+ *
+ **/
+int imgsys_scp_get_reserve_mem_info(struct img_init_info *info);
+
+/**
+ * imgsys_scp_get_reserve_mem_virt - get the virtual address of reserved memory
+ * by a particular memory ID
+ *
+ * @id: The identifier of the reserved memory block
+ *
+ **/
+void *imgsys_scp_get_reserve_mem_virt(enum img_mem_id id);
+
+/**
+ * imgsys_scp_get_reserve_mem_size - get the size of reserved memory by a
+ * particular memory ID
+ *
+ * @id: The identifier of the reserved memory block
+ *
+ **/
+unsigned int imgsys_scp_get_reserve_mem_size(enum img_mem_id id);
+
+/**
+ * imgsys_scp_init - start the SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ * @imgsys_scp_handler: Handler for SCP IPI
+ **/
+int imgsys_scp_init(struct mtk_imgsys_dev *imgsys_dev,
+ scp_ipi_handler_t imgsys_scp_handler);
+
+/**
+ * imgsys_scp_deinit - stop the SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ *
+ **/
+void imgsys_scp_deinit(struct mtk_imgsys_dev *imgsys_dev);
+
+/**
+ * imgsys_scp_send - send data from AP to SCP
+ *
+ * @imgsys_dev: Pointer to the imgsys platform device structure
+ * @id: IPI ID
+ * @buf:the data buffer
+ * @len: the data buffer length
+ * @req_fd: the file descriptor of media request
+ * @wait: wait until ipi done or not
+ *
+ * This function is thread-safe. When this function returns,
+ * SCP has received the data and starts the processing.
+ * When the processing completes, IPI handler registered
+ * by scp_ipi_register will be called in interrupt context.
+ *
+ **/
+int imgsys_scp_send(struct mtk_imgsys_dev *imgsys_dev,
+ enum imgsys_ipi_id id,
+ void *buf,
+ unsigned int len,
+ int req_fd,
+ unsigned int wait);
+
+#endif /* MTK_IMGSYS_SCP_H */
+
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 09/10] media: mediatek: isp: Add image processing flow
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (6 preceding siblings ...)
2025-05-24 11:50 ` [PATCH v1 08/10] media: mediatek: isp: Add SCP " Olivia Wen
@ 2025-05-24 11:50 ` Olivia Wen
2025-05-24 11:50 ` [PATCH v1 10/10] media: mediatek: isp: Add normal data dump flow Olivia Wen
[not found] ` <20250524115144.3832748-5-olivia.wen@mediatek.com>
9 siblings, 0 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the image processing flow for the MediaTek ImgSys driver.
It is a crucial part of the ImgSys architecture, controlling the flow of
operations from the user applications to the hardware. The ImgSys driver
manages various components (e.g. SCP, CMDQ) and processes to ensure
efficient image processing.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 2 +
| 101 ++++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.c | 61 +++
.../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 139 +++++-
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.c | 82 +++
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.h | 37 ++
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 468 +++++++++++++++++-
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 128 ++++-
.../isp/isp_7x/imgsys/mtk_imgsys-work_buf.c | 113 +++++
.../isp/isp_7x/imgsys/mtk_imgsys-work_buf.h | 55 ++
10 files changed, 1169 insertions(+), 17 deletions(-)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 29c18c773fdc..3aa53e8bfc6d 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,6 +2,8 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-work_buf.o \
+mtk_imgsys-gce_work.o \
mtk_imgsys-cmdq.o \
mtk_imgsys-scp.o \
mtk_imgsys-module_main.o \
--git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
index 50c2e3c3395b..673e84aad362 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
@@ -9,6 +9,39 @@
#include <linux/videodev2.h>
+#define IMGBUF_MAX_PLANES 3 /* Maximum number of planes in an image buffer */
+#define IMG_MAX_HW_DMAS 72 /* Maximum number of hardware DMAs */
+
+/**
+ * struct v4l2_ext_plane - Extended plane information
+ * @offset: Offset within the DMA buffer
+ * @phyaddr: Physical address of the DMA buffer
+ * @isp_addr: ISP (Image Signal Processor) address
+ * @size: Size of the plane
+ */
+struct v4l2_ext_plane {
+ union {
+ struct {
+ __u32 offset; /* Offset within the DMA buffer */
+ __u64 phyaddr; /* Physical address of the DMA buffer */
+ } dma_buf;
+ } m;
+ __u64 isp_addr; /* ISP (Image Signal Processor) address */
+ __u64 size; /* Size of the plane */
+};
+
+/**
+ * struct v4l2_ext_buffer - Extended V4L2 buffer
+ * @planes: Array of extended plane information
+ * @num_planes: Number of planes
+ * @reserved: Reserved for future use
+ */
+struct v4l2_ext_buffer {
+ struct v4l2_ext_plane planes[IMGBUF_MAX_PLANES];
+ __u32 num_planes;
+ __u64 reserved[2];
+};
+
/**
* struct mtk_imgsys_crop - Crop parameters for MediaTek Image System
* @c: Rectangle defining the crop area
@@ -25,4 +58,72 @@ struct mtk_imgsys_crop {
struct v4l2_fract height_subpix; /* Sub-pixel adjustment for the height */
};
+/**
+ * struct plane_pix_format - Pixel format for a single plane
+ * @sizeimage: Size of the image in bytes
+ * @bytesperline: Number of bytes per line
+ */
+struct plane_pix_format {
+ __u32 sizeimage;
+ __u32 bytesperline;
+} __packed;
+
+/**
+ * struct pix_format_mplane - Multi-plane pixel format
+ * @width: Width of the image
+ * @height: Height of the image
+ * @pixelformat: Pixel format identifier
+ * @plane_fmt: Array of pixel formats for each plane
+ */
+struct pix_format_mplane {
+ __u32 width;
+ __u32 height;
+ __u32 pixelformat;
+ struct plane_pix_format plane_fmt[IMGBUF_MAX_PLANES];
+} __packed;
+
+/**
+ * struct frameparams - Frame parameters for MediaTek Image System
+ * @buf: Extended buffer information
+ * @fmt: Multi-plane pixel format
+ * @crop: Crop parameters
+ * @rotation: Rotation angle
+ * @hflip: Horizontal flip flag
+ * @vflip: Vertical flip flag
+ * @resizeratio: Resize ratio
+ */
+struct frameparams {
+ struct v4l2_ext_buffer buf; /* Extended buffer information */
+ struct pix_format_mplane fmt; /* Multi-plane pixel format */
+ struct mtk_imgsys_crop crop; /* Crop parameters */
+ __u32 rotation; /* Rotation angle */
+ __u32 hflip; /* Horizontal flip flag */
+ __u32 vflip; /* Vertical flip flag */
+ __u8 resizeratio; /* Resize ratio */
+};
+
+/**
+ * struct header_desc_norm - Header description for normal camera case
+ * @fparams_num: Number of frame parameters
+ * @fparams: Frame parameters
+ */
+struct header_desc_norm {
+ __u32 fparams_num;
+ struct frameparams fparams;
+};
+
+/**
+ * struct singlenode_desc_norm - Single node description for normal camera case
+ * @dmas_enable: Array indicating enabled DMA channels
+ * @dmas: Array of DMA header descriptions
+ * @tuning_meta: Tuning metadata header description
+ * @ctrl_meta: Control metadata header description
+ */
+struct singlenode_desc_norm {
+ __u8 dmas_enable[IMG_MAX_HW_DMAS];
+ struct header_desc_norm dmas[IMG_MAX_HW_DMAS];
+ struct header_desc_norm tuning_meta;
+ struct header_desc_norm ctrl_meta;
+};
+
#endif
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
index 28a35a3226a2..a52f1cf93d47 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
@@ -83,6 +83,67 @@ int mtk_imgsys_pipe_next_job_id(struct mtk_imgsys_pipe *pipe)
return (global_job_id & 0x0000FFFF) | (pipe->desc->id << 16);
}
+struct mtk_imgsys_request *
+mtk_imgsys_pipe_get_running_job(struct mtk_imgsys_pipe *pipe, int id)
+{
+ struct mtk_imgsys_request *req;
+ unsigned long flag;
+
+ spin_lock_irqsave(&pipe->running_job_lock, flag);
+ list_for_each_entry(req,
+ &pipe->pipe_job_running_list, list) {
+ if (req->id == id) {
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+ return req;
+ }
+ }
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+
+ return NULL;
+}
+
+void mtk_imgsys_pipe_remove_job(struct mtk_imgsys_request *req)
+{
+ unsigned long flag;
+
+ spin_lock_irqsave(&req->imgsys_pipe->running_job_lock, flag);
+ list_del(&req->list);
+ req->imgsys_pipe->num_jobs--;
+ spin_unlock_irqrestore(&req->imgsys_pipe->running_job_lock, flag);
+
+ dev_dbg(req->imgsys_pipe->imgsys_dev->dev,
+ "%s:%s:req->id(%d),num of running jobs(%d) entry(%p)\n", __func__,
+ req->imgsys_pipe->desc->name, req->id,
+ req->imgsys_pipe->num_jobs, &req->list);
+}
+
+void mtk_imgsys_pipe_job_finish(struct mtk_imgsys_request *req,
+ enum vb2_buffer_state vbf_state)
+{
+ struct mtk_imgsys_pipe *pipe = req->imgsys_pipe;
+ int i;
+
+ if (req->req.state != MEDIA_REQUEST_STATE_QUEUED) {
+ dev_info(pipe->imgsys_dev->dev, "%s: req %d 0x%lx flushed in state(%d)", __func__,
+ req->request_fd, (unsigned long)&req->req, req->req.state);
+ return;
+ }
+
+ for (i = 0; i < pipe->desc->total_queues; i++) {
+ struct mtk_imgsys_dev_buffer *dev_buf = req->buf_map[i];
+ struct mtk_imgsys_video_device *node;
+
+ if (!dev_buf)
+ continue;
+
+ node = mtk_imgsys_vbq_to_node(dev_buf->vbb.vb2_buf.vb2_queue);
+ spin_lock(&node->buf_list_lock);
+ list_del(&dev_buf->list);
+ spin_unlock(&node->buf_list_lock);
+ vb2_buffer_done(&dev_buf->vbb.vb2_buf, vbf_state);
+ }
+}
+
void mtk_imgsys_pipe_debug_job(struct mtk_imgsys_pipe *pipe,
struct mtk_imgsys_request *req)
{
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
index b8c30cb32b97..effee4a3b616 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
@@ -25,6 +25,8 @@
#include <media/videobuf2-v4l2.h>
#include "mtk_imgsys-module_ops.h"
#include "mtk_header_desc.h"
+#include "mtk-img-ipi.h"
+#include "mtk_imgsys-work_buf.h"
#define MTK_IMGSYS_MEDIA_MODEL_NAME "MTK-ISP-DIP-V4L2"
@@ -38,6 +40,31 @@
#define MTK_IMGSYS_CAPTURE_MAX_WIDTH 5376U
#define MTK_IMGSYS_CAPTURE_MAX_HEIGHT 4032U
+/**
+ * MTK_IMGSYS_JOB_TIMEOUT_MS - Defines the job timeout in milliseconds.
+ *
+ * This macro calculates the timeout for image system jobs based on the frame rate
+ * (30 frames per second) and the number of working buffers. The timeout is set to
+ * three times the duration of all working buffers.
+ */
+#define MTK_IMGSYS_JOB_TIMEOUT_MS (1000 / 30 * IMGSYS_WORKING_BUF_NUM * 3)
+
+/**
+ * struct mtk_imgsys_hw_working_buf_list - Represents a list of hardware
+ * working buffers
+ *
+ * @list: List head for linking multiple buffer lists
+ * @lock: Spinlock to protect the list
+ *
+ * This structure represents a list of hardware working buffers used in the
+ * MediaTek Image System. It includes a list head for linking multiple buffer
+ * lists and a spinlock to protect the list from concurrent access.
+ */
+struct mtk_imgsys_hw_working_buf_list {
+ struct list_head list;
+ spinlock_t lock; /* protect the list of working buf */
+};
+
/**
* struct mtk_imgsys_dev_format - Device format for MediaTek Image System
* @fmt: Pointer to the Image System format structure
@@ -274,6 +301,7 @@ struct mtk_imgsys_pipe {
* @imgsys_pipe: Image system pipeline configuration structure.
* @cust_pipes: Pointer to custom pipeline descriptors.
* @modules: Pointer to image system module operations.
+ * @num_mods: Number of modules in the image system.
* @clks: Array of clock bulk data for managing multiple clocks.
* @num_clks: Number of clocks in the clock array.
* @larbs: Array of device pointers for managing LARB (local arbiter).
@@ -281,9 +309,18 @@ struct mtk_imgsys_pipe {
* @scp: Pointer to the SCP (System Control Processor) structure.
* @rproc_handle: Handle for remote processor interface.
* @smem_dev: Device structure for shared memory operations.
- * @num_mods: Number of modules in the image system.
+ * @working_bufs: Array of hardware working buffers.
+ * @free_working_bufs: List of free working buffers.
+ * @composer_wq: Workqueue for composing tasks in image processing.
+ * @runner_wq: Workqueue for running tasks in image processing.
+ * @gcecb_wq: Workqueue for handling GCE callback tasks.
+ * @work_pool: Pool of GCE callback work structures.
+ * @flushing_waitq: Wait queue for flushing operations.
+ * @num_composing: Atomic counter for tracking composing operations.
+ * @imgsys_enqueue_cnt: Atomic counter for enqueued image system requests.
* @init_kref: Reference counter for initialization tracking.
* @hw_op_lock: Mutex for serializing hardware operations.
+ * @sem: Semaphore to limit the maximum number of requests sent to SCP.
* @stream_cnt: Counter for active streams, increased on stream on.
*/
struct mtk_imgsys_dev {
@@ -298,6 +335,7 @@ struct mtk_imgsys_dev {
const struct mtk_imgsys_pipe_desc *cust_pipes;
/* Imgsys Moudles Information */
const struct module_ops *modules;
+ int num_mods;
/* Clock Information */
struct clk_bulk_data *clks;
int num_clks;
@@ -308,11 +346,22 @@ struct mtk_imgsys_dev {
struct mtk_scp *scp;
struct rproc *rproc_handle;
struct device *smem_dev;
- /* Number of modules */
- int num_mods;
+ /* Working Buffers */
+ struct mtk_imgsys_hw_working_buf *working_bufs;
+ struct mtk_imgsys_hw_working_buf_list free_working_bufs;
+ /* Workqueues for ImgSys */
+ struct workqueue_struct *composer_wq;
+ struct workqueue_struct *runner_wq;
+ /* GCE (Generic Command Engine) Callback Workqueue */
+ struct workqueue_struct *gcecb_wq;
+ struct gce_cb_work_pool *work_pool;
/* Synchronization and Flow Control */
+ wait_queue_head_t flushing_waitq;
+ atomic_t num_composing;
+ atomic_t imgsys_enqueue_cnt;
struct kref init_kref;
struct mutex hw_op_lock; /* Protect HW operations */
+ struct semaphore sem;
int stream_cnt;
};
@@ -360,10 +409,74 @@ struct mtk_imgsys_request {
struct mtk_imgsys_dev_buffer **buf_map;
struct list_head list;
struct req_frameparam img_fparam;
+ struct work_struct composer_work;
+ struct work_struct runner_work;
+ struct mtk_imgsys_hw_working_buf *working_buf;
+ struct swfrm_info *swfrm_info;
atomic_t buf_count;
int request_fd;
};
+/**
+ * struct swfrm_info - Information about software frame
+ * @req_sbuf_goft: GCE buffer offset
+ * @req_sbuf_kva: GCE buffer virtual address
+ * @swfrminfo_ridx: Ring index of the swfrminfo (referenced by SCP)
+ * @request_fd: Enqueued request file descriptor in V4L2 kernel framework
+ * @request_no: Request number from middleware
+ * @frame_no: Frame number from middleware
+ * @frm_owner: Frame owner identifier from middleware
+ * @fps: Frames per second
+ * @cb_frmcnt: Frame counter
+ * @total_taskcnt: Total task count
+ * @req_vaddr: Kernel request virtual address
+ * @sync_id: Synchronization identifier for camera/image synchronization
+ * @total_frmnum: Total frame number in the current request
+ * @user_info: Array of user information from SCP to kernel
+ * @is_earlycb: Indicates if the request is separated by early callback hint
+ * @earlycb_idx: Frame index for early callback
+ * @is_lastfrm: Indicates if the request is separated by last frame
+ * @group_id: Group identifier for selecting GCE thread
+ * @req: Pointer to mtk_imgsys_request
+ * @pipe: Pointer to pipe mtk_imgsys_pipe
+ * @fail_uinfo_idx: Failed index in user_info
+ * @hw_hang: Indicates if the current failure is caused by hardware hang
+ * @ndd_swfrm_info: Pointer to NDD (Normal data dump) software frame information
+ *
+ * This structure contains detailed information about a software frame,
+ * including various identifiers, addresses, and status flags.
+ */
+struct swfrm_info {
+ u32 req_sbuf_goft;
+ void *req_sbuf_kva;
+ int swfrminfo_ridx;
+ int request_fd;
+ int request_no;
+ int frame_no;
+ u64 frm_owner;
+ int fps;
+ int cb_frmcnt;
+ int total_taskcnt;
+ u64 req_vaddr;
+ int sync_id;
+ int total_frmnum;
+ struct img_swfrm_info user_info[TMAX];
+ u8 is_earlycb;
+ int earlycb_idx;
+ u8 is_lastfrm;
+ s8 group_id;
+ void *req;
+ void *pipe;
+ u32 fail_uinfo_idx;
+ s8 hw_hang;
+ void *ndd_swfrm_info;
+} __attribute((__packed__));
+static inline struct mtk_imgsys_pipe*
+mtk_imgsys_dev_get_pipe(struct mtk_imgsys_dev *imgsys_dev)
+{
+ return &imgsys_dev->imgsys_pipe;
+}
+
static inline struct mtk_imgsys_video_device*
mtk_imgsys_file_to_node(struct file *file)
{
@@ -401,6 +514,18 @@ mtk_imgsys_media_req_to_imgsys_req(struct media_request *req)
return container_of(req, struct mtk_imgsys_request, req);
}
+static inline struct mtk_imgsys_request*
+mtk_imgsys_composer_work_to_req(struct work_struct *composer_work)
+{
+ return container_of(composer_work, struct mtk_imgsys_request, composer_work);
+}
+
+static inline struct mtk_imgsys_request*
+mtk_imgsys_runner_work_to_req(struct work_struct *runner_work)
+{
+ return container_of(runner_work, struct mtk_imgsys_request, runner_work);
+}
+
static inline int mtk_imgsys_buf_is_meta(u32 type)
{
return type == V4L2_BUF_TYPE_META_CAPTURE ||
@@ -413,8 +538,16 @@ int mtk_imgsys_pipe_init(struct mtk_imgsys_dev *imgsys_dev,
int mtk_imgsys_pipe_release(struct mtk_imgsys_pipe *pipe);
+struct mtk_imgsys_request *
+mtk_imgsys_pipe_get_running_job(struct mtk_imgsys_pipe *pipe, int id);
+
+void mtk_imgsys_pipe_remove_job(struct mtk_imgsys_request *req);
+
int mtk_imgsys_pipe_next_job_id(struct mtk_imgsys_pipe *pipe);
+void mtk_imgsys_pipe_job_finish(struct mtk_imgsys_request *req,
+ enum vb2_buffer_state vbf_state);
+
void mtk_imgsys_pipe_debug_job(struct mtk_imgsys_pipe *pipe,
struct mtk_imgsys_request *req);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
new file mode 100644
index 000000000000..ef86410e3d23
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "mtk_imgsys-scp.h"
+#include "mtk_imgsys-gce_work.h"
+
+static void cmdq_cb_done_worker(struct work_struct *work)
+{
+ struct mtk_imgsys_pipe *pipe;
+ struct swfrm_info *gwfrm_info = NULL;
+ struct gce_cb_work *gwork;
+ struct img_sw_buffer swbuf_data = {0};
+
+ gwork = container_of(work, struct gce_cb_work, work);
+ gwfrm_info = gwork->req_sbuf_kva;
+
+ pipe = gwork->pipe;
+ if (!pipe->streaming)
+ return;
+
+ /* send to SCP after frame done & del node from list */
+ swbuf_data.offset = gwfrm_info->req_sbuf_goft;
+ imgsys_scp_send(pipe->imgsys_dev, IPI_IMGSYS_DEQUE_DONE_ID,
+ &swbuf_data, sizeof(struct img_sw_buffer),
+ gwork->reqfd, 1);
+
+ gce_cb_work_free(pipe->imgsys_dev->work_pool, gwork);
+}
+
+int gce_cb_work_pool_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ int i;
+ struct gce_cb_work_pool *work_pool;
+
+ work_pool = kmalloc(sizeof(*work_pool), GFP_KERNEL);
+ if (!work_pool)
+ return -ENOMEM;
+ imgsys_dev->work_pool = work_pool;
+
+ spin_lock_init(&work_pool->lock);
+ INIT_LIST_HEAD(&work_pool->free_list);
+
+ for (i = 0; i < MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS; i++) {
+ INIT_WORK(&work_pool->work_items[i].work, cmdq_cb_done_worker);
+ atomic_set(&work_pool->work_items[i].state, WORK_INIT);
+ work_pool->work_items[i].work_idx = i;
+ list_add_tail(&work_pool->work_items[i].list, &work_pool->free_list);
+ }
+ return 0;
+}
+
+struct gce_cb_work *gce_cb_work_allocate(struct gce_cb_work_pool *pool)
+{
+ struct gce_cb_work *gwork = NULL;
+
+ spin_lock(&pool->lock);
+ if (list_empty(&pool->free_list)) {
+ spin_unlock(&pool->lock);
+ return NULL;
+ }
+
+ gwork = list_first_entry(&pool->free_list, struct gce_cb_work, list);
+ list_del(&gwork->list);
+ spin_unlock(&pool->lock);
+
+ atomic_set(&gwork->state, WORK_ALLOC);
+
+ return gwork;
+}
+
+void gce_cb_work_free(struct gce_cb_work_pool *pool, struct gce_cb_work *gwork)
+{
+ atomic_set(&gwork->state, WORK_FREE);
+
+ spin_lock(&pool->lock);
+ list_add_tail(&gwork->list, &pool->free_list);
+ spin_unlock(&pool->lock);
+}
+
+void gce_cb_work_pool_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ kfree(imgsys_dev->work_pool);
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
new file mode 100644
index 000000000000..ba80b952f38e
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _MTK_IMGSYS_GCE_WORK_H_
+#define _MTK_IMGSYS_GCE_WORK_H_
+
+#define MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS 60
+
+enum mtk_imgsys_gce_cb_work_state {
+ WORK_INIT,
+ WORK_ALLOC,
+ WORK_COMPLETED,
+ WORK_FREE
+};
+
+struct gce_cb_work {
+ struct work_struct work;
+ atomic_t state;
+ u32 work_idx;
+ u32 reqfd;
+ void *req_sbuf_kva;
+ void *pipe;
+ struct list_head list;
+ wait_queue_head_t wait_queue;
+};
+
+struct gce_cb_work_pool {
+ struct gce_cb_work work_items[MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS];
+ struct list_head free_list;
+ spinlock_t lock; /* protect the free_list */
+};
+
+int gce_cb_work_pool_init(struct mtk_imgsys_dev *imgsys_dev);
+struct gce_cb_work *gce_cb_work_allocate(struct gce_cb_work_pool *pool);
+void gce_cb_work_free(struct gce_cb_work_pool *pool, struct gce_cb_work *gwork);
+void gce_cb_work_pool_release(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_GCE_WORK_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 184f0e3a4d3b..36088a5bda8e 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -3,17 +3,261 @@
* Copyright (c) 2021 MediaTek Inc.
*
* Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
*
*/
#include <linux/pm_runtime.h>
#include "mtk_imgsys-sys.h"
#include "mtk_imgsys-dev.h"
+#include "mtk_imgsys-scp.h"
#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-work_buf.h"
+#include "mtk_imgsys-gce_work.h"
+#include "mtk_header_desc.h"
+#include "mtk_imgsys_v4l2_vnode.h"
+
+#define FRAME_STATE_INIT 0
+#define FRAME_STATE_HW_TIMEOUT 1
+
+static void mtk_imgsys_notify(struct mtk_imgsys_request *req, uint64_t frm_owner)
+{
+ struct mtk_imgsys_dev *imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ struct mtk_imgsys_pipe *pipe = req->imgsys_pipe;
+ struct req_frameparam *iparam = &req->img_fparam;
+ enum vb2_buffer_state vbf_state;
+
+ if (!mtk_imgsys_pipe_get_running_job(pipe, req->id))
+ goto out;
+
+ if (iparam->state != FRAME_STATE_HW_TIMEOUT)
+ vbf_state = VB2_BUF_STATE_DONE;
+ else
+ vbf_state = VB2_BUF_STATE_ERROR;
+
+ atomic_dec(&imgsys_dev->num_composing);
+
+ mtk_imgsys_hw_working_buf_free(imgsys_dev, req->working_buf);
+ req->working_buf = NULL;
+ mtk_imgsys_pipe_remove_job(req);
+ if (vbf_state == VB2_BUF_STATE_DONE)
+ mtk_imgsys_pipe_job_finish(req, vbf_state);
+
+ wake_up(&imgsys_dev->flushing_waitq);
+ dev_dbg(imgsys_dev->dev,
+ "%s:%s:(reqfd-%d) , frame_no(%d) frm_owner(%s) finished\n",
+ __func__, pipe->desc->name, req->request_fd, iparam->frame_no,
+ ((char *)(&frm_owner)));
+
+out:
+ media_request_put(&req->req);
+}
+
+static void imgsys_cmdq_cb_func(struct imgsys_cmdq_cb_data data)
+{
+ struct mtk_imgsys_pipe *pipe;
+ struct mtk_imgsys_request *req;
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct swfrm_info *swfrminfo_cb;
+ struct gce_cb_work *gwork;
+
+ swfrminfo_cb = data.data;
+ pipe = (struct mtk_imgsys_pipe *)swfrminfo_cb->pipe;
+ if (!pipe->streaming) {
+ pr_info("%s pipe already streamoff\n", __func__);
+ return;
+ }
+
+ req = (struct mtk_imgsys_request *)(swfrminfo_cb->req);
+ if (!req) {
+ pr_info("%s NULL request Address\n", __func__);
+ return;
+ }
+
+ imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ dev_dbg(imgsys_dev->dev, "%s:(reqfd-%d)frame_no(%d) +", __func__,
+ req->request_fd,
+ req->img_fparam.frame_no);
+
+ if (swfrminfo_cb->hw_hang >= 0)
+ req->img_fparam.state = FRAME_STATE_HW_TIMEOUT;
+
+ mtk_imgsys_notify(req, swfrminfo_cb->frm_owner);
+
+ if (swfrminfo_cb->hw_hang < 0) {
+ gwork = gce_cb_work_allocate(imgsys_dev->work_pool);
+ if (!gwork)
+ return;
+
+ gwork->reqfd = swfrminfo_cb->request_fd;
+ gwork->req_sbuf_kva = swfrminfo_cb->req_sbuf_kva;
+ gwork->pipe = swfrminfo_cb->pipe;
+ init_waitqueue_head(&gwork->wait_queue);
+
+ queue_work(req->imgsys_pipe->imgsys_dev->gcecb_wq, &gwork->work);
+ }
+}
+
+static void imgsys_runner_func(struct work_struct *work)
+{
+ struct mtk_imgsys_request *req = mtk_imgsys_runner_work_to_req(work);
+ struct mtk_imgsys_dev *imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ struct swfrm_info *frm_info;
+ int ret;
+
+ frm_info = req->swfrm_info;
+
+ ret = imgsys_cmdq_sendtask(imgsys_dev, frm_info, imgsys_cmdq_cb_func);
+ if (ret)
+ dev_info(imgsys_dev->dev,
+ "%s: imgsys_cmdq_sendtask fail(%d)\n", __func__, ret);
+}
+
+/* processing IPI_IMGSYS_FRAME_ID, share data is struct img_sw_buffer*/
+static void imgsys_scp_handler(void *data, unsigned int len, void *priv)
+{
+ struct share_buf *scp_msg = (struct share_buf *)data;
+ struct mtk_imgsys_pipe *pipe;
+ struct mtk_imgsys_request *req;
+ struct mtk_imgsys_dev *imgsys_dev = (struct mtk_imgsys_dev *)priv;
+ struct img_sw_buffer *swbuf_data;
+ struct swfrm_info *swfrm_info;
+ int i = 0;
+ void *gce_virt;
+ int total_framenum = 0;
+
+ if (!data) {
+ WARN_ONCE(!data, "%s: failed due to NULL data\n", __func__);
+ return;
+ }
+
+ /* this function just handles id IPI_IMGSYS_FRAME_ID */
+ if (scp_msg->id != IPI_IMGSYS_FRAME_ID) {
+ dev_dbg(imgsys_dev->dev,
+ "%s: get invalid IPI ID(%d)\n", __func__, scp_msg->id);
+ return;
+ }
+
+ if (WARN_ONCE(scp_msg->len != sizeof(struct img_sw_buffer),
+ "%s: len(%d) not match img_sw_buffer\n", __func__, len))
+ return;
+
+ swbuf_data = (struct img_sw_buffer *)scp_msg->share_data;
+ gce_virt = imgsys_scp_get_reserve_mem_virt(IMG_MEM_G_ID);
+ if (!gce_virt) {
+ pr_info("%s: invalid gce_virt(%p)\n",
+ __func__, gce_virt);
+ return;
+ }
+
+ swfrm_info = (struct swfrm_info *)(gce_virt + (swbuf_data->offset));
+ if ((int)swbuf_data->offset < 0 ||
+ swbuf_data->offset > imgsys_scp_get_reserve_mem_size(IMG_MEM_G_ID)) {
+ pr_info("%s: invalid swbuf_data->offset(%d), max(%llu)\n",
+ __func__, swbuf_data->offset,
+ (u64)imgsys_scp_get_reserve_mem_size(IMG_MEM_G_ID));
+ return;
+ }
+
+ swfrm_info->req_sbuf_goft = swbuf_data->offset;
+ swfrm_info->req_sbuf_kva = gce_virt + (swbuf_data->offset);
+
+ pipe = mtk_imgsys_dev_get_pipe(imgsys_dev);
+ if (!pipe) {
+ dev_info(imgsys_dev->dev,
+ "%s: get invalid img_ipi_frameparam from firmware\n",
+ __func__);
+ return;
+ }
+
+ req = (struct mtk_imgsys_request *)(uintptr_t)swfrm_info->req_vaddr;
+ if (!req) {
+ WARN_ONCE(!req, "%s: frame_no(%d) is lost\n", __func__, req->id);
+ return;
+ }
+
+ if (!req->working_buf) {
+ dev_dbg(imgsys_dev->dev,
+ "%s: (reqfd-%d) composing\n",
+ __func__, req->request_fd);
+ }
+
+ swfrm_info->req = (void *)req;
+ swfrm_info->pipe = (void *)pipe;
+ swfrm_info->cb_frmcnt = 0;
+ swfrm_info->total_taskcnt = 0;
+ swfrm_info->hw_hang = -1;
+ total_framenum = swfrm_info->total_frmnum;
+
+ if (total_framenum < 0 || total_framenum > TMAX) {
+ dev_info(imgsys_dev->dev,
+ "%s:unexpected total_framenum (%d -> %d), MAX (%d/%d)\n",
+ __func__, swfrm_info->total_frmnum,
+ total_framenum,
+ TMAX, TMAX);
+ return;
+ }
+
+ for (i = 0 ; i < total_framenum ; i++) {
+ swfrm_info->user_info[i].g_swbuf =
+ gce_virt + (swfrm_info->user_info[i].sw_goft);
+ }
+
+ req->swfrm_info = swfrm_info;
+
+ up(&imgsys_dev->sem);
+
+ INIT_WORK(&req->runner_work, imgsys_runner_func);
+ queue_work(req->imgsys_pipe->imgsys_dev->runner_wq,
+ &req->runner_work);
+}
+
+static int mtk_imgsys_hw_flush_pipe_jobs(struct mtk_imgsys_pipe *pipe)
+{
+ struct mtk_imgsys_request *req;
+ struct list_head job_list = LIST_HEAD_INIT(job_list);
+ unsigned long flag;
+ int num;
+ int ret;
+
+ spin_lock_irqsave(&pipe->running_job_lock, flag);
+ list_splice_init(&pipe->pipe_job_running_list, &job_list);
+ pipe->num_jobs = 0;
+ spin_unlock_irqrestore(&pipe->running_job_lock, flag);
+
+ num = atomic_read(&pipe->imgsys_dev->num_composing);
+ ret = wait_event_timeout(pipe->imgsys_dev->flushing_waitq, !(num),
+ msecs_to_jiffies(MTK_IMGSYS_JOB_TIMEOUT_MS));
+ if (!ret && num) {
+ dev_info(pipe->imgsys_dev->dev,
+ "flushing is aborted, num(%d)\n", num);
+ return -EINVAL;
+ }
+
+ list_for_each_entry(req, &job_list, list)
+ mtk_imgsys_pipe_job_finish(req, VB2_BUF_STATE_ERROR);
+
+ dev_dbg(pipe->imgsys_dev->dev,
+ "%s: wakeup num(%d)\n", __func__, num);
+ return 0;
+}
+
+static void module_uninit(struct kref *kref)
+{
+ struct mtk_imgsys_dev *imgsys_dev;
+ int i;
+
+ imgsys_dev = container_of(kref, struct mtk_imgsys_dev, init_kref);
+
+ for (i = 0; i < (imgsys_dev->num_mods); i++)
+ if (imgsys_dev->modules[i].uninit)
+ imgsys_dev->modules[i].uninit(imgsys_dev);
+}
static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
{
int ret, i;
+ struct img_init_info info;
ret = pm_runtime_get_sync(imgsys_dev->dev);
if (ret < 0) {
@@ -23,38 +267,228 @@ static int mtk_imgsys_hw_connect(struct mtk_imgsys_dev *imgsys_dev)
/* Set default value for hardware modules */
for (i = 0; i < (imgsys_dev->num_mods); i++)
- imgsys_dev->modules[i].init(imgsys_dev);
+ if (imgsys_dev->modules[i].init)
+ imgsys_dev->modules[i].init(imgsys_dev);
kref_init(&imgsys_dev->init_kref);
+ ret = imgsys_scp_alloc_reserve_mem(imgsys_dev);
+ if (ret) {
+ dev_err(imgsys_dev->dev, "%s: imgsys_scp_alloc_reserve_mem failed %d\n",
+ __func__, ret);
+ return -EBUSY;
+ }
+
+ /* IMGSYS HW INIT */
+ memset(&info, 0, sizeof(info));
+ info.header_version = HEADER_VER;
+ info.frameparam_size = sizeof(struct img_ipi_frameparam);
+
+ imgsys_scp_get_reserve_mem_info(&info);
+
+ imgsys_scp_init(imgsys_dev, imgsys_scp_handler);
+
+ ret = imgsys_scp_send(imgsys_dev, IPI_IMGSYS_INIT_ID,
+ (void *)&info, sizeof(info), 0, 1);
+
+ if (ret) {
+ dev_err(imgsys_dev->dev, "%s: send IPI_IMGSYS_INIT_ID failed %d\n",
+ __func__, ret);
+ return -EBUSY;
+ }
+
/* calling cmdq stream on */
imgsys_cmdq_streamon(imgsys_dev);
+
return 0;
}
-static void module_uninit(struct kref *kref)
+static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
{
- struct mtk_imgsys_dev *imgsys_dev;
+ int ret;
+ struct img_init_info info = {0};
+
+ imgsys_scp_send(imgsys_dev, IPI_IMGSYS_DEINIT_ID,
+ (void *)&info, sizeof(info), 0, 1);
+
+ imgsys_scp_deinit(imgsys_dev);
+ /* calling cmdq stream off */
+ imgsys_cmdq_streamoff(imgsys_dev);
+
+ /* RELEASE IMGSYS WORKING BUFFER FIRST */
+ ret = imgsys_scp_free_reserve_mem(imgsys_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: imgsys_scp_free_reserve_mem failed(%d)\n",
+ __func__, ret);
+ }
+
+ pm_runtime_put_sync(imgsys_dev->dev);
+
+ kref_put(&imgsys_dev->init_kref, module_uninit);
+}
+
+static void imgsys_fill_img_buf(struct mtk_imgsys_dev_buffer *dev_buf,
+ struct header_desc_norm *desc_norm,
+ struct mtk_imgsys_request *req)
+{
+ struct frameparams *buf_info = &desc_norm->fparams;
int i;
- imgsys_dev = container_of(kref, struct mtk_imgsys_dev, init_kref);
+ desc_norm->fparams_num = 1;
- for (i = 0; i < (imgsys_dev->num_mods); i++)
- if (imgsys_dev->modules[i].uninit)
- imgsys_dev->modules[i].uninit(imgsys_dev);
+ buf_info->buf.num_planes = dev_buf->fmt.fmt.pix_mp.num_planes;
+ buf_info->fmt.width = dev_buf->fmt.fmt.pix_mp.width;
+ buf_info->fmt.height = dev_buf->fmt.fmt.pix_mp.height;
+ buf_info->fmt.pixelformat = dev_buf->fmt.fmt.pix_mp.pixelformat;
+ buf_info->crop = dev_buf->crop;
+ buf_info->rotation = dev_buf->rotation;
+ buf_info->hflip = dev_buf->hflip;
+ buf_info->vflip = dev_buf->vflip;
+ buf_info->resizeratio = dev_buf->resize_ratio;
+
+ for (i = 0; i < buf_info->buf.num_planes; i++) {
+ buf_info->buf.planes[i].m.dma_buf.offset =
+ dev_buf->vbb.planes[i].data_offset;
+ buf_info->buf.planes[i].isp_addr = dev_buf->isp_daddr[i];
+ buf_info->buf.planes[i].size = dev_buf->vbb.vb2_buf.planes[i].length;
+ buf_info->fmt.plane_fmt[i].sizeimage =
+ dev_buf->vbb.vb2_buf.planes[i].min_length;
+ buf_info->fmt.plane_fmt[i].bytesperline =
+ dev_buf->fmt.fmt.pix_mp.plane_fmt[i].bytesperline;
+ }
}
-static void mtk_imgsys_hw_disconnect(struct mtk_imgsys_dev *imgsys_dev)
+static void imgsys_fill_meta_buf(struct mtk_imgsys_dev_buffer *dev_buf,
+ struct header_desc_norm *desc_norm,
+ struct mtk_imgsys_request *req)
{
- /* calling cmdq stream off */
- imgsys_cmdq_streamoff(imgsys_dev);
- kref_put(&imgsys_dev->init_kref, module_uninit);
+ struct frameparams *buf_info = &desc_norm->fparams;
+
+ memset(&buf_info->buf, 0, sizeof(buf_info->buf));
+
+ desc_norm->fparams_num = 1;
+
+ buf_info->buf.num_planes = 1;
+
+ buf_info->fmt.width = dev_buf->fmt.fmt.meta.buffersize;
+ buf_info->fmt.height = 1;
+ buf_info->fmt.pixelformat = dev_buf->fmt.fmt.meta.dataformat;
+ buf_info->fmt.plane_fmt[0].sizeimage = dev_buf->fmt.fmt.meta.buffersize;
+ buf_info->fmt.plane_fmt[0].bytesperline = dev_buf->fmt.fmt.meta.buffersize;
+ buf_info->buf.planes[0].isp_addr = dev_buf->isp_daddr[0];
+ buf_info->buf.planes[0].size = dev_buf->vbb.vb2_buf.planes[0].length;
+ buf_info->buf.planes[0].m.dma_buf.phyaddr = dev_buf->scp_daddr[0];
+}
+
+static void imgsys_fill_buf_info(int idx, struct mtk_imgsys_dev_buffer *dev_buf,
+ struct singlenode_desc_norm *sd_norm,
+ struct mtk_imgsys_request *req)
+{
+ if (!dev_buf)
+ return;
+
+ if (idx < 0 || idx >= IMG_MAX_HW_DMAS)
+ return;
+
+ sd_norm->dmas_enable[idx] = 1;
+
+ switch (idx) {
+ case MTK_IMGSYS_VIDEO_NODE_ID_TUNING_OUT:
+ imgsys_fill_meta_buf(dev_buf, &sd_norm->tuning_meta, req);
+ break;
+ case MTK_IMGSYS_VIDEO_NODE_ID_CTRLMETA_OUT:
+ imgsys_fill_meta_buf(dev_buf, &sd_norm->ctrl_meta, req);
+ break;
+ case MTK_IMGSYS_VIDEO_NODE_ID_METAI_OUT:
+ case MTK_IMGSYS_VIDEO_NODE_ID_FEO_CAPTURE:
+ case MTK_IMGSYS_VIDEO_NODE_ID_IMGSTATO_CAPTURE:
+ imgsys_fill_meta_buf(dev_buf, &sd_norm->dmas[idx], req);
+ break;
+ default:
+ imgsys_fill_img_buf(dev_buf, &sd_norm->dmas[idx], req);
+ break;
+ }
+}
+
+static void imgsys_fill_singledev_buf(struct mtk_imgsys_request *req)
+{
+ struct mtk_imgsys_hw_working_buf *buf = req->working_buf;
+ int i;
+
+ if (!buf)
+ return;
+
+ memset(buf->sd_norm, 0, sizeof(*buf->sd_norm));
+ for (i = 0; i < req->imgsys_pipe->desc->total_queues; i++)
+ imgsys_fill_buf_info(i, req->buf_map[i], buf->sd_norm, req);
+}
+
+static void imgsys_composer_workfunc(struct work_struct *work)
+{
+ struct mtk_imgsys_request *req = mtk_imgsys_composer_work_to_req(work);
+ struct mtk_imgsys_dev *imgsys_dev = req->imgsys_pipe->imgsys_dev;
+ struct img_ipi_param ipi_param;
+ int ret;
+
+ imgsys_fill_singledev_buf(req);
+
+ dev_dbg(imgsys_dev->dev,
+ "%s:(reqfd-%d) to send frame_no(%d)\n",
+ __func__, req->request_fd,
+ req->img_fparam.frame_no);
+ media_request_get(&req->req);
+
+ if (down_interruptible(&imgsys_dev->sem))
+ return;
+
+ ipi_param.req_addr_va = (u64)(uintptr_t)req;
+ ipi_param.frm_param.offset = 0;
+ ipi_param.frm_param.scp_addr = req->working_buf->scp_daddr;
+
+ mutex_lock(&imgsys_dev->hw_op_lock);
+ atomic_inc(&imgsys_dev->num_composing);
+
+ ret = imgsys_scp_send(imgsys_dev, IPI_IMGSYS_FRAME_ID,
+ &ipi_param, sizeof(ipi_param),
+ req->request_fd, 0);
+
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "frame_no(%d) send SCP_IPI_DIP_FRAME failed %d\n",
+ req->img_fparam.frame_no, ret);
+ mtk_imgsys_pipe_remove_job(req);
+ mtk_imgsys_hw_working_buf_free(imgsys_dev, req->working_buf);
+ req->working_buf = NULL;
+ mtk_imgsys_pipe_job_finish(req, VB2_BUF_STATE_ERROR);
+ wake_up(&imgsys_dev->flushing_waitq);
+ }
+ mutex_unlock(&imgsys_dev->hw_op_lock);
+
+ dev_dbg(imgsys_dev->dev, "%s:(reqfd-%d) sent\n", __func__,
+ req->request_fd);
}
void mtk_imgsys_hw_enqueue(struct mtk_imgsys_dev *imgsys_dev,
struct mtk_imgsys_request *req)
{
- /* Function implementation will be provided in subsequent patches */
+ struct req_frameparam *frameparam = &req->img_fparam;
+
+ req->working_buf = mtk_imgsys_hw_working_buf_alloc(imgsys_dev);
+ if (!req->working_buf) {
+ dev_info(imgsys_dev->dev,
+ "%s:req(%p): no free working buffer available\n",
+ req->imgsys_pipe->desc->name, req);
+ return;
+ }
+
+ frameparam->state = FRAME_STATE_INIT;
+ frameparam->frame_no =
+ atomic_inc_return(&imgsys_dev->imgsys_enqueue_cnt);
+
+ INIT_WORK(&req->composer_work, imgsys_composer_workfunc);
+ queue_work(req->imgsys_pipe->imgsys_dev->composer_wq,
+ &req->composer_work);
}
int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
@@ -72,10 +506,13 @@ int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
mutex_unlock(&imgsys_dev->hw_op_lock);
return ret;
}
+ INIT_LIST_HEAD(&pipe->pipe_job_running_list);
}
imgsys_dev->stream_cnt++;
+ atomic_set(&imgsys_dev->num_composing, 0);
mutex_unlock(&imgsys_dev->hw_op_lock);
+ pipe->streaming = 1;
dev_dbg(pipe->imgsys_dev->dev,
"%s:%s: stream on, id(%d), stream cnt(%d)\n",
__func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
@@ -86,11 +523,18 @@ int mtk_imgsys_hw_streamon(struct mtk_imgsys_pipe *pipe)
int mtk_imgsys_hw_streamoff(struct mtk_imgsys_pipe *pipe)
{
struct mtk_imgsys_dev *imgsys_dev = pipe->imgsys_dev;
+ int ret;
dev_dbg(pipe->imgsys_dev->dev,
"%s:%s: stream off, id(%d), stream cnt(%d)\n",
__func__, pipe->desc->name, pipe->desc->id, imgsys_dev->stream_cnt);
+ pipe->streaming = 0;
+
+ ret = mtk_imgsys_hw_flush_pipe_jobs(pipe);
+ if (ret)
+ return ret;
+
mutex_lock(&imgsys_dev->hw_op_lock);
imgsys_dev->stream_cnt--;
if (!imgsys_dev->stream_cnt)
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index e25f17b98649..ea2ce03c2619 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -5,6 +5,7 @@
* Author: Frederic Chen <frederic.chen@mediatek.com>
*
*/
+
#include <linux/platform_device.h>
#include <linux/of_platform.h>
#include <linux/of_device.h>
@@ -12,6 +13,7 @@
#include <linux/pm_runtime.h>
#include <linux/dma-mapping.h>
#include <linux/vmalloc.h>
+#include <linux/remoteproc/mtk_scp.h>
#include <linux/videodev2.h>
#include <media/videobuf2-dma-contig.h>
#include <media/v4l2-ioctl.h>
@@ -29,6 +31,9 @@
#include "mtk_imgsys_v4l2_vnode.h"
#include "mtk_imgsys-dev.h"
#include "mtk_imgsys-sys.h"
+#include "mtk_imgsys-cmdq.h"
+#include "mtk_imgsys-work_buf.h"
+#include "mtk_imgsys-gce_work.h"
#define IMGSYS_MAX_BUFFERS 256
#define IMGSYS_SUSPEND_TIME 3000 /* ms */
@@ -1467,6 +1472,77 @@ static void mtk_imgsys_dev_v4l2_release(struct mtk_imgsys_dev *imgsys_dev)
mtk_imgsys_dev_media_unregister(imgsys_dev);
}
+static int mtk_imgsys_res_init(struct platform_device *pdev,
+ struct mtk_imgsys_dev *imgsys_dev)
+{
+ int ret;
+
+ imgsys_dev->gcecb_wq =
+ alloc_ordered_workqueue("%s",
+ __WQ_LEGACY | WQ_MEM_RECLAIM |
+ WQ_FREEZABLE,
+ "imgsys_mdp_callback");
+ if (!imgsys_dev->gcecb_wq) {
+ dev_err(imgsys_dev->dev,
+ "%s: unable to alloc mdpcb workqueue\n", __func__);
+ return -ENOMEM;
+ }
+
+ imgsys_dev->composer_wq =
+ alloc_ordered_workqueue("%s",
+ __WQ_LEGACY | WQ_MEM_RECLAIM |
+ WQ_FREEZABLE | WQ_HIGHPRI,
+ "imgsys_composer");
+ if (!imgsys_dev->composer_wq) {
+ dev_err(imgsys_dev->dev,
+ "%s: unable to alloc composer workqueue\n", __func__);
+ ret = -ENOMEM;
+ goto destroy_gcecb_wq;
+ }
+
+ imgsys_dev->runner_wq =
+ alloc_ordered_workqueue("%s",
+ __WQ_LEGACY | WQ_MEM_RECLAIM |
+ WQ_FREEZABLE | WQ_HIGHPRI,
+ "imgsys_runner");
+ if (!imgsys_dev->runner_wq) {
+ dev_err(imgsys_dev->dev,
+ "%s: unable to alloc runner workqueue\n", __func__);
+ ret = -ENOMEM;
+ goto destroy_composer_wq;
+ }
+
+ init_waitqueue_head(&imgsys_dev->flushing_waitq);
+
+ return 0;
+
+destroy_composer_wq:
+ destroy_workqueue(imgsys_dev->composer_wq);
+
+destroy_gcecb_wq:
+ destroy_workqueue(imgsys_dev->gcecb_wq);
+
+ return ret;
+}
+
+static void mtk_imgsys_res_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ flush_workqueue(imgsys_dev->gcecb_wq);
+ destroy_workqueue(imgsys_dev->gcecb_wq);
+ imgsys_dev->gcecb_wq = NULL;
+
+ flush_workqueue(imgsys_dev->composer_wq);
+ destroy_workqueue(imgsys_dev->composer_wq);
+ imgsys_dev->composer_wq = NULL;
+
+ flush_workqueue(imgsys_dev->runner_wq);
+ destroy_workqueue(imgsys_dev->runner_wq);
+ imgsys_dev->runner_wq = NULL;
+
+ atomic_set(&imgsys_dev->num_composing, 0);
+ atomic_set(&imgsys_dev->imgsys_enqueue_cnt, 0);
+}
+
static int mtk_imgsys_of_rproc(struct mtk_imgsys_dev *imgsys,
struct platform_device *pdev)
{
@@ -1519,6 +1595,8 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
imgsys_dev->num_mods = data->mod_num;
imgsys_dev->stream_cnt = 0;
mutex_init(&imgsys_dev->hw_op_lock);
+ atomic_set(&imgsys_dev->imgsys_enqueue_cnt, 0);
+ atomic_set(&imgsys_dev->num_composing, 0);
/* Get Clocks */
imgsys_dev->clks = data->clks;
@@ -1592,6 +1670,33 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
goto err_free_larb_alloc;
}
+ /* Initializing resource for ImgSys */
+ ret = mtk_imgsys_res_init(pdev, imgsys_dev);
+ if (ret) {
+ dev_info(imgsys_dev->dev,
+ "%s: mtk_imgsys_res_init failed(%d)\n", __func__, ret);
+
+ ret = -EBUSY;
+ goto err_release_deinit_v4l2;
+ }
+
+ ret = mtk_imgsys_hw_working_buf_pool_init(imgsys_dev);
+ if (ret) {
+ dev_info(&pdev->dev, "working buffer init failed(%d)\n", ret);
+ goto err_release_deinit_v4l2;
+ }
+
+ /* Initializing resource for CMDQ driver */
+ imgsys_cmdq_init(imgsys_dev);
+
+ sema_init(&imgsys_dev->sem, MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS);
+
+ ret = gce_cb_work_pool_init(imgsys_dev);
+ if (ret) {
+ dev_info(&pdev->dev, "gce callback work pool init failed(%d)\n", ret);
+ goto err_release_working_buf_pool;
+ }
+
/* Enable power*/
pm_runtime_set_autosuspend_delay(&pdev->dev, IMGSYS_SUSPEND_TIME);
pm_runtime_use_autosuspend(&pdev->dev);
@@ -1600,11 +1705,15 @@ static int mtk_imgsys_probe(struct platform_device *pdev)
ret = platform_driver_register(&mtk_imgsys_larb_driver);
if (ret) {
dev_info(imgsys_dev->dev, "register mtk_imgsys_larb_driver fail\n");
- goto err_release_deinit_v4l2;
+ goto err_free_gce_cb_work_pool_alloc;
}
return 0;
+err_free_gce_cb_work_pool_alloc:
+ gce_cb_work_pool_release(imgsys_dev);
+err_release_working_buf_pool:
+ mtk_imgsys_hw_working_buf_pool_release(imgsys_dev);
err_release_deinit_v4l2:
mtk_imgsys_dev_v4l2_release(imgsys_dev);
err_free_larb_alloc:
@@ -1618,17 +1727,32 @@ static void mtk_imgsys_remove(struct platform_device *pdev)
{
struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(&pdev->dev);
+ mtk_imgsys_res_release(imgsys_dev);
pm_runtime_disable(&pdev->dev);
platform_driver_unregister(&mtk_imgsys_larb_driver);
mtk_imgsys_dev_v4l2_release(imgsys_dev);
+ mtk_imgsys_hw_working_buf_pool_release(imgsys_dev);
+ gce_cb_work_pool_release(imgsys_dev);
mutex_destroy(&imgsys_dev->hw_op_lock);
+ imgsys_cmdq_release(imgsys_dev);
devm_kfree(&pdev->dev, imgsys_dev->larbs);
devm_kfree(&pdev->dev, imgsys_dev);
}
static int __maybe_unused mtk_imgsys_pm_suspend(struct device *dev)
{
- int ret;
+ struct mtk_imgsys_dev *imgsys_dev = dev_get_drvdata(dev);
+ int ret, num;
+
+ num = atomic_read(&imgsys_dev->num_composing);
+ ret = wait_event_timeout(imgsys_dev->flushing_waitq, !(num),
+ msecs_to_jiffies(MTK_IMGSYS_JOB_TIMEOUT_MS));
+ if (!ret && num) {
+ dev_info(dev, "%s: flushing SCP job timeout, num(%d)\n",
+ __func__, num);
+
+ return -EBUSY;
+ }
if (pm_runtime_suspended(dev)) {
dev_info(dev, "%s: pm_runtime_suspended is true, no action\n",
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
new file mode 100644
index 000000000000..5f220bbe8f3f
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.c
@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#include "mtk_imgsys-work_buf.h"
+
+int mtk_imgsys_hw_working_buf_pool_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct mtk_imgsys_hw_working_buf *buf;
+ void *sd;
+ dma_addr_t scp_addr, isp_daddr;
+ size_t size = sizeof(struct singlenode_desc_norm);
+ int i;
+
+ if (imgsys_dev->working_bufs)
+ return 0;
+
+ INIT_LIST_HEAD(&imgsys_dev->free_working_bufs.list);
+ spin_lock_init(&imgsys_dev->free_working_bufs.lock);
+
+ imgsys_dev->working_bufs = kmalloc_array(IMGSYS_WORKING_BUF_NUM, sizeof(*buf), GFP_KERNEL);
+ if (!imgsys_dev->working_bufs)
+ return -ENOMEM;
+
+ sd = dma_alloc_coherent(imgsys_dev->smem_dev,
+ size * IMGSYS_WORKING_BUF_NUM,
+ &scp_addr, GFP_KERNEL);
+ if (!sd)
+ goto err_alloc;
+
+ isp_daddr = dma_map_resource(imgsys_dev->dev,
+ scp_addr,
+ size * IMGSYS_WORKING_BUF_NUM,
+ DMA_TO_DEVICE, 0);
+ if (!isp_daddr)
+ goto err_map;
+
+ for (i = 0; i < IMGSYS_WORKING_BUF_NUM; i++) {
+ buf = &imgsys_dev->working_bufs[i];
+
+ buf->sd_norm = sd;
+ buf->isp_daddr = isp_daddr;
+ buf->scp_daddr = scp_addr;
+
+ sd += size;
+ isp_daddr += size;
+ scp_addr += size;
+
+ list_add_tail(&buf->list,
+ &imgsys_dev->free_working_bufs.list);
+ }
+
+ return 0;
+
+err_map:
+ dma_free_coherent(imgsys_dev->dev,
+ sizeof(struct singlenode_desc_norm),
+ sd, scp_addr);
+err_alloc:
+ kfree(imgsys_dev->working_bufs);
+
+ return -ENOMEM;
+}
+
+struct mtk_imgsys_hw_working_buf*
+mtk_imgsys_hw_working_buf_alloc(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct mtk_imgsys_hw_working_buf *working_buf;
+
+ spin_lock(&imgsys_dev->free_working_bufs.lock);
+ if (list_empty(&imgsys_dev->free_working_bufs.list)) {
+ spin_unlock(&imgsys_dev->free_working_bufs.lock);
+ return NULL;
+ }
+
+ working_buf = list_first_entry(&imgsys_dev->free_working_bufs.list,
+ struct mtk_imgsys_hw_working_buf,
+ list);
+ list_del(&working_buf->list);
+ spin_unlock(&imgsys_dev->free_working_bufs.lock);
+
+ return working_buf;
+}
+
+void mtk_imgsys_hw_working_buf_free(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_hw_working_buf *working_buf)
+{
+ if (!working_buf)
+ return;
+
+ spin_lock(&imgsys_dev->free_working_bufs.lock);
+ list_add_tail(&working_buf->list,
+ &imgsys_dev->free_working_bufs.list);
+ spin_unlock(&imgsys_dev->free_working_bufs.lock);
+}
+
+void mtk_imgsys_hw_working_buf_pool_release(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct mtk_imgsys_hw_working_buf *buf = &imgsys_dev->working_bufs[0];
+
+ dma_unmap_resource(imgsys_dev->dev, buf->isp_daddr,
+ sizeof(struct singlenode_desc_norm) * IMGSYS_WORKING_BUF_NUM,
+ DMA_TO_DEVICE, 0);
+ dma_free_coherent(imgsys_dev->dev,
+ sizeof(struct singlenode_desc_norm) * IMGSYS_WORKING_BUF_NUM,
+ buf->sd_norm, buf->scp_daddr);
+ kfree(imgsys_dev->working_bufs);
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
new file mode 100644
index 000000000000..d54b02e6dfa8
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-work_buf.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_WORK_BUF_H_
+#define _MTK_IMGSYS_WORK_BUF_H_
+
+#include <linux/clk.h>
+#include <linux/dma-buf.h>
+#include <linux/scatterlist.h>
+#include "mtk_header_desc.h"
+#include "mtk_imgsys-dev.h"
+
+#define IMGSYS_WORKING_BUF_NUM 32
+
+/**
+ * struct mtk_imgsys_hw_working_buf - Represents a hardware working buffer
+ * @sd_norm: Pointer to the single node descriptor normalized structure
+ * @dbuf: Pointer to the DMA buffer
+ * @attach: Pointer to the DMA buffer attachment
+ * @sgt: Pointer to the scatter-gather table
+ * @scp_daddr: Physical address for SCP (System Control Processor)
+ * @isp_daddr: Virtual address for ISP (Image Signal Processor)
+ * @list: List head for linking multiple buffers
+ *
+ * This structure represents a hardware working buffer used in the MediaTek
+ * Image System. It includes pointers to various buffer-related structures
+ * and addresses, as well as a list head for linking multiple buffers.
+ */
+struct mtk_imgsys_hw_working_buf {
+ struct singlenode_desc_norm *sd_norm;
+ struct dma_buf *dbuf;
+ struct dma_buf_attachment *attach;
+ struct sg_table *sgt;
+ dma_addr_t scp_daddr;
+ dma_addr_t isp_daddr;
+ struct list_head list;
+};
+
+int mtk_imgsys_hw_working_buf_pool_init(struct mtk_imgsys_dev *imgsys_dev);
+
+struct mtk_imgsys_hw_working_buf*
+mtk_imgsys_hw_working_buf_alloc(struct mtk_imgsys_dev *imgsys_dev);
+
+void mtk_imgsys_hw_working_buf_free(struct mtk_imgsys_dev *imgsys_dev,
+ struct mtk_imgsys_hw_working_buf *working_buf);
+
+void mtk_imgsys_hw_working_buf_pool_release(struct mtk_imgsys_dev *imgsys_dev);
+
+#endif /* _MTK_IMGSYS_WORK_BUF_H_ */
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v1 10/10] media: mediatek: isp: Add normal data dump flow
2025-05-24 11:49 [PATCH v1 00/10] Add MediaTek ISP7 Image Syatem driver Olivia Wen
` (7 preceding siblings ...)
2025-05-24 11:50 ` [PATCH v1 09/10] media: mediatek: isp: Add image processing flow Olivia Wen
@ 2025-05-24 11:50 ` Olivia Wen
[not found] ` <20250524115144.3832748-5-olivia.wen@mediatek.com>
9 siblings, 0 replies; 15+ messages in thread
From: Olivia Wen @ 2025-05-24 11:50 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: Krzysztof Kozlowski, Conor Dooley, Olivia Wen, linux-media,
devicetree, linux-kernel, linux-arm-kernel, linux-mediatek,
Project_Global_Chrome_Upstream_Group, teddy.chen, yunkec
The ImgSys driver is implemented as a series of patches, with this patch
focusing on the normal data dump flow. It is for debugging purposes
within the MediaTek Image System (ImgSys) driver. The data dump
functionality is essential for diagnosing issues, tracking the flow of
data, and ensuring the proper operation of image processing tasks.
Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
---
.../mediatek/isp/isp_7x/imgsys/Makefile | 2 +
.../isp_7x/imgsys/modules/mtk_imgsys-dip.c | 47 +++
.../isp_7x/imgsys/modules/mtk_imgsys-dip.h | 3 +
.../isp/isp_7x/imgsys/modules/mtk_imgsys-me.c | 65 ++++
.../isp/isp_7x/imgsys/modules/mtk_imgsys-me.h | 30 ++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.c | 52 +++
.../isp_7x/imgsys/modules/mtk_imgsys-pqdip.h | 3 +
.../isp_7x/imgsys/modules/mtk_imgsys-traw.c | 55 +++
.../isp_7x/imgsys/modules/mtk_imgsys-traw.h | 3 +
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.c | 60 +++
.../isp_7x/imgsys/modules/mtk_imgsys-wpe.h | 3 +
.../mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h | 8 +
.../isp/isp_7x/imgsys/mtk_imgsys-gce_work.c | 4 +
.../isp_7x/imgsys/mtk_imgsys-module_main.c | 4 +
.../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 11 +
.../isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c | 342 ++++++++++++++++++
.../isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h | 64 ++++
.../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 11 +
.../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 7 +
19 files changed, 774 insertions(+)
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
index 3aa53e8bfc6d..16044a280d5c 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
@@ -2,6 +2,7 @@
#
mtk_imgsys_util-objs := \
+mtk_imgsys-ndd_debugfs.o \
mtk_imgsys-work_buf.o \
mtk_imgsys-gce_work.o \
mtk_imgsys-cmdq.o \
@@ -14,6 +15,7 @@ mtk_imgsys-sys.o \
mtk_imgsys-v4l2.o
mtk_imgsys_hw_isp-objs := \
+modules/mtk_imgsys-me.o \
modules/mtk_imgsys-dip.o \
modules/mtk_imgsys-traw.o \
modules/mtk_imgsys-pqdip.o \
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
index 1cc1f3e87e8b..d9563d5a177a 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.c
@@ -10,6 +10,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-dip.h"
/********************************************************************
@@ -42,6 +43,8 @@
#define DIPCQ_D1A_DIPCQ_CQ_THR17_CTL 0x318
#define DIPCQ_D1A_DIPCQ_CQ_THR18_CTL 0x328
+#define DIP_REG_RANGE 0x17000
+
#define DIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_dip_init_ary)
/********************************************************************
@@ -103,6 +106,50 @@ void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_dip_init);
+void imgsys_dip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char *dip_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ ssize_t ret;
+ void *cq_va;
+
+ if (frm_dump_info->eng_e != IMGSYS_NDD_ENG_DIP)
+ return;
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(DIP_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ dip_name = "REG_DIP_dip.reg";
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, dip_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ dev_err(imgsys_dev->dev, "wrong dip ndd file name %s\n", file_name);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ ret = copy_to_user(frm_dump_info->user_buffer, cq_va, 0x9000);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += 0x48000 + 0x9000;
+
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va + 0x9000, DIP_REG_RANGE - 0x9000);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += DIP_REG_RANGE - 0x9000;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_dip_ndd_dump);
+
void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
index 22b4a7c3616d..1f0b85f1faba 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-dip.h
@@ -10,8 +10,11 @@
#define _MTK_DIP_DIP_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_dip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_dip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_dip_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_DIP_DIP_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
new file mode 100644
index 000000000000..dacb3d591a71
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.c
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ * Holmes Chiou <holmes.chiou@mediatek.com>
+ *
+ */
+#include <linux/of_address.h>
+#include <linux/uaccess.h>
+#include "mtk_imgsys-me.h"
+
+#define ME_REG_RANGE 0xA20
+
+void imgsys_me_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char *me_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ void __iomem *reg_va;
+ ssize_t ret;
+
+ if (frm_dump_info->eng_e != IMGSYS_NDD_ENG_ME)
+ return;
+
+ reg_va = ioremap(frm_dump_info->cq_ofst[frm_dump_info->eng_e], ME_REG_RANGE);
+ if (!reg_va)
+ return;
+
+ me_name = "REG_ME_me.reg";
+
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, me_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ iounmap(reg_va);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret) {
+ iounmap(reg_va);
+ return;
+ }
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ void *kernel_buffer = kmalloc(ME_REG_RANGE, GFP_KERNEL);
+
+ if (!kernel_buffer) {
+ iounmap(reg_va);
+ return;
+ }
+
+ memcpy_fromio(kernel_buffer, reg_va, ME_REG_RANGE);
+ ret = copy_to_user(frm_dump_info->user_buffer, kernel_buffer, ME_REG_RANGE);
+ kfree(kernel_buffer);
+ if (ret != 0) {
+ iounmap(reg_va);
+ return;
+ }
+ frm_dump_info->user_buffer += ME_REG_RANGE;
+ }
+
+ iounmap(reg_va);
+}
+EXPORT_SYMBOL(imgsys_me_ndd_dump);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
new file mode 100644
index 000000000000..10ed0153b740
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-me.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Frederic Chen <frederic.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_ME_H_
+#define _MTK_IMGSYS_ME_H_
+
+#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
+
+/**
+ * imgsys_me_ndd_dump - Dump the frame information for debugging.
+ * @imgsys_dev: Pointer to the mtk_imgsys_dev structure representing the ISP image
+ * processing device.
+ * @frm_dump_info: Pointer to the imgsys_ndd_frm_dump_info structure containing
+ * frame dump information.
+ *
+ * This function dumps the NDD (Normal Data Dump) frame information for debugging
+ * purposes. It provides insights into the frame data processed by the ISP image
+ * processing device, which can help in identifying issues related to frame
+ * handling.
+ */
+void imgsys_me_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
+
+#endif /* _MTK_IMGSYS_ME_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
index d1d3507555d9..f1f688b13ba5 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-pqdip.h"
/********************************************************************
@@ -40,6 +41,8 @@
#define DIPCQ_P1A_REG_DIPCQ_CQ_THR17_CTL 0x0318
#define DIPCQ_P1A_REG_DIPCQ_CQ_THR18_CTL 0x0328
+#define PQDIP_REG_RANGE 0x6000
+
#define PQDIP_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_pqdip_init_ary)
/********************************************************************
@@ -100,6 +103,55 @@ void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_pqdip_init);
+void imgsys_pqdip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char *pqdip_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ ssize_t ret;
+ void *cq_va;
+
+ if (frm_dump_info->eng_e != IMGSYS_NDD_ENG_PQDIP_A &&
+ frm_dump_info->eng_e != IMGSYS_NDD_ENG_PQDIP_B)
+ return;
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(PQDIP_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ switch (frm_dump_info->eng_e) {
+ case IMGSYS_NDD_ENG_PQDIP_A:
+ pqdip_name = "REG_PQDIP_A_pqdipa.reg";
+ break;
+ case IMGSYS_NDD_ENG_PQDIP_B:
+ pqdip_name = "REG_PQDIP_B_pqdipb.reg";
+ break;
+ default:
+ return;
+ }
+
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, pqdip_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ dev_err(imgsys_dev->dev, "wrong dip ndd file name %s\n", file_name);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ ret = copy_to_user(frm_dump_info->user_buffer, cq_va, PQDIP_REG_RANGE);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += PQDIP_REG_RANGE;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_pqdip_ndd_dump);
+
void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
index 00f93d536181..09bc51a1c15d 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-pqdip.h
@@ -10,8 +10,11 @@
#define _MTK_IMGSYS_PQDIP_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_pqdip_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_pqdip_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_pqdip_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_IMGSYS_PQDIP_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
index 292f0ad09c46..90597f3a776a 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-traw.h"
/********************************************************************
@@ -16,6 +17,8 @@
********************************************************************/
#define TRAWCTL_INT1_EN 0x00A0
+#define TRAW_REG_RANGE 0xC000
+
#define TRAW_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_traw_init_ary)
/********************************************************************
@@ -53,6 +56,58 @@ void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_traw_init);
+void imgsys_traw_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ const char *traw_name;
+ char file_name[NDD_FP_SIZE] = "\0";
+ ssize_t ret;
+ void *cq_va;
+
+ if (frm_dump_info->eng_e > IMGSYS_NDD_ENG_XTR ||
+ frm_dump_info->eng_e < IMGSYS_NDD_ENG_TRAW)
+ return;
+
+ switch (frm_dump_info->eng_e) {
+ case IMGSYS_NDD_ENG_TRAW:
+ traw_name = "REG_TRAW_traw.reg";
+ break;
+ case IMGSYS_NDD_ENG_LTR:
+ traw_name = "REG_LTRAW_ltraw.reg";
+ break;
+ case IMGSYS_NDD_ENG_XTR:
+ traw_name = "REG_XTRAW_xtraw.reg";
+ break;
+ default:
+ return;
+ }
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(TRAW_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ ret = snprintf(file_name, sizeof(file_name), "%s%s", frm_dump_info->fp, traw_name);
+ if (ret < 0 || ret >= sizeof(file_name)) {
+ dev_err(imgsys_dev->dev, "wrong traw ndd file name %s\n", file_name);
+ return;
+ }
+
+ if (frm_dump_info->user_buffer) {
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ ret = copy_to_user(frm_dump_info->user_buffer, cq_va, TRAW_REG_RANGE);
+ if (ret)
+ return;
+ frm_dump_info->user_buffer += TRAW_REG_RANGE;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_traw_ndd_dump);
+
void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
index f3727216b58c..57881bc2e191 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-traw.h
@@ -10,8 +10,11 @@
#define _MTK_IMGSYS_TRAW_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_traw_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_traw_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_traw_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_IMGSYS_TRAW_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
index b19d61529b80..276af83a9ec2 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.c
@@ -9,6 +9,7 @@
#include <linux/device.h>
#include <linux/of_address.h>
#include "../mtk_imgsys-module_common.h"
+#include "../mtk_imgsys-scp.h"
#include "mtk_imgsys-wpe.h"
/********************************************************************
@@ -41,6 +42,10 @@
#define DIPCQ_W1A_DIPCQ_CQ_THR13_CTL_OFFSET 0x0BD8
#define DIPCQ_W1A_DIPCQ_CQ_THR14_CTL_OFFSET 0x0BE8
+#define WPE_REG_RANGE 0x1000
+#define WPE_PSP_END_OFST 0x554
+#define WPE_COEF_END_OFST 0x5D8
+
#define WPE_INIT_ARRAY_COUNT ARRAY_SIZE(mtk_imgsys_wpe_init_ary)
/********************************************************************
* Global Variable
@@ -101,6 +106,61 @@ void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev)
}
EXPORT_SYMBOL_GPL(imgsys_wpe_init);
+void imgsys_wpe_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ char file_name[NDD_FP_SIZE] = "\0";
+ void *cq_va, *psp_va;
+ u64 psp_ofst;
+ ssize_t ret;
+
+ if (frm_dump_info->eng_e > IMGSYS_NDD_ENG_WPE_LITE)
+ return;
+
+ if (!frm_dump_info->user_buffer)
+ return;
+
+ cq_va = imgsys_scp_get_reserve_mem_virt(WPE_MEM_C_ID);
+ if (!cq_va)
+ return;
+
+ cq_va += frm_dump_info->cq_ofst[frm_dump_info->eng_e];
+
+ psp_va = imgsys_scp_get_reserve_mem_virt(WPE_MEM_C_ID);
+ if (!psp_va)
+ return;
+
+ psp_ofst = frm_dump_info->wpe_psp_ofst[frm_dump_info->eng_e];
+ psp_va += psp_ofst;
+
+ ret = snprintf(file_name, sizeof(file_name),
+ "%s%s", frm_dump_info->fp, frm_dump_info->wpe_fp);
+ if (ret < 0 || ret >= sizeof(file_name))
+ return;
+
+ ret = copy_to_user(frm_dump_info->user_buffer, file_name, sizeof(file_name));
+ frm_dump_info->user_buffer += sizeof(file_name);
+
+ if (psp_ofst != 0) {
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va, WPE_PSP_END_OFST);
+ frm_dump_info->user_buffer += WPE_PSP_END_OFST;
+
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ psp_va, WPE_COEF_END_OFST - WPE_PSP_END_OFST);
+ frm_dump_info->user_buffer += (WPE_COEF_END_OFST - WPE_PSP_END_OFST);
+
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va + WPE_COEF_END_OFST, WPE_REG_RANGE - WPE_COEF_END_OFST);
+ frm_dump_info->user_buffer += (WPE_REG_RANGE - WPE_COEF_END_OFST);
+ } else {
+ ret = copy_to_user(frm_dump_info->user_buffer,
+ cq_va, WPE_REG_RANGE);
+ frm_dump_info->user_buffer += WPE_REG_RANGE;
+ }
+}
+EXPORT_SYMBOL_GPL(imgsys_wpe_ndd_dump);
+
void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev)
{
unsigned int hw_idx;
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
index 6001575265e6..77b713bbe499 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_imgsys-wpe.h
@@ -10,8 +10,11 @@
#define _MTK_IMGSYS_WPE_H_
#include "../mtk_imgsys-dev.h"
+#include "../mtk_imgsys-ndd_debugfs.h"
void imgsys_wpe_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_wpe_ndd_dump(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void imgsys_wpe_uninit(struct mtk_imgsys_dev *imgsys_dev);
#endif /* _MTK_IMGSYS_WPE_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
index 850f6ec99a96..705d8c35ff55 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk-img-ipi.h
@@ -81,6 +81,10 @@ struct img_init_info {
* @is_earlycb: Indicates if early callback is enabled
* @sw_goft: offset of GCE buffer
* @g_swbuf: virtual address of GCE buffer
+ * @ndd_cq_ofst: NDD CQ offset
+ * @ndd_wpe_psp_ofst: NDD WPE PSP offset
+ * @ndd_wpe_fp: NDD WPE file path
+ * @ndd_fp: NDD file path
*
* These fields are primarily set by SCP (System Control Processor) and used by
* the kernel for processing and verifing data.
@@ -92,6 +96,10 @@ struct img_swfrm_info {
u8 is_earlycb;
u64 sw_goft;
void *g_swbuf;
+ u64 ndd_cq_ofst[IMG_MAX_HW];
+ u64 ndd_wpe_psp_ofst[WPE_ENG_NUM];
+ char ndd_wpe_fp[64];
+ char ndd_fp[256];
} __packed;
/**
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
index ef86410e3d23..7e5b7cd383f9 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-gce_work.c
@@ -2,6 +2,7 @@
#include "mtk_imgsys-scp.h"
#include "mtk_imgsys-gce_work.h"
+#include "mtk_imgsys-ndd_debugfs.h"
static void cmdq_cb_done_worker(struct work_struct *work)
{
@@ -17,6 +18,9 @@ static void cmdq_cb_done_worker(struct work_struct *work)
if (!pipe->streaming)
return;
+ if (gwfrm_info->user_info[0].ndd_fp[0] != '\0')
+ wait_event(gwork->wait_queue, atomic_read(&gwork->state) == WORK_COMPLETED);
+
/* send to SCP after frame done & del node from list */
swbuf_data.offset = gwfrm_info->req_sbuf_goft;
imgsys_scp_send(pipe->imgsys_dev, IPI_IMGSYS_DEQUE_DONE_ID,
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
index 5daa8f863bd2..c31ada82afcd 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_main.c
@@ -10,6 +10,7 @@
#include <linux/of_address.h>
#include "mtk_imgsys-module_common.h"
#include "mtk_imgsys-module_main.h"
+#include "mtk_imgsys-ndd_debugfs.h"
/********************************************************************
* Global Define
@@ -147,6 +148,7 @@ void imgsys_main_init(struct mtk_imgsys_dev *imgsys_dev)
}
imgsys_main_hw_init(imgsys_dev);
+ imgsys_ndd_debugfs_init(imgsys_dev);
}
void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev)
@@ -180,4 +182,6 @@ void imgsys_main_uninit(struct mtk_imgsys_dev *imgsys_dev)
iounmap(dip2_reg_base);
dip2_reg_base = NULL;
}
+
+ imgsys_ndd_debugfs_uninit(imgsys_dev);
}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
index a2eea8f4fb3c..1dc28993bc09 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
@@ -25,14 +25,23 @@ enum mtk_imgsys_module {
};
struct mtk_imgsys_dev;
+struct imgsys_ndd_frm_dump_info;
/**
* struct module_ops - Operations for a specific module in the MediaTek Image System
+ *
* @module_id: Identifier for the module.
* @init: Function pointer to initialize the module.
* This function sets the default values for various software-related
* parameters and ensures that the module is ready for operation.
* Example function: imgsys_dip_init
+ * @ndddump: Function pointer to perform a dump of the module's frame information.
+ * This function is responsible for dumping frame-related information.
+ * It is typically used for debugging and diagnostic purposes to analyze
+ * the state of the system and the frames being processed.
+ * The dumped information can be used to trace issues or verify the
+ * correctness of the frame processing pipeline.
+ * Example function: imgsys_dip_ndd_dump
* @uninit: Function pointer to uninitialize the module.
* This function resets various software-related parameters to ensure
* the module is properly cleaned up and ready for shutdown.
@@ -48,6 +57,8 @@ struct mtk_imgsys_dev;
struct module_ops {
int module_id;
void (*init)(struct mtk_imgsys_dev *imgsys_dev);
+ void (*ndddump)(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info);
void (*uninit)(struct mtk_imgsys_dev *imgsys_dev);
};
#endif /* _MTK_IMGSYS_MODULES_H_ */
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
new file mode 100644
index 000000000000..d0c1168fbd06
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.c
@@ -0,0 +1,342 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#include <linux/debugfs.h>
+#include <linux/device.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/of_address.h>
+#include <linux/pm_runtime.h>
+#include <linux/remoteproc.h>
+#include <linux/uaccess.h>
+#include <soc/mediatek/smi.h>
+#include "mtk_imgsys-ndd_debugfs.h"
+#include "mtk_imgsys-module_common.h"
+#include "mtk_imgsys-gce_work.h"
+
+#define WPE_ENG_NUM 3
+#define TRAW_ENG_NUM 3
+#define DIP_ENG_NUM 1
+#define PQDIP_ENG_NUM 2
+#define ME_ENG_NUM 1
+
+#define WPE_NDD_SIZE 0x1000
+#define TRAW_NDD_SIZE 0xC000
+#define DIP_NDD_SIZE 0x5F000
+#define PQDIP_NDD_SIZE 0x6000
+#define ME_NDD_SIZE 0xA20
+
+struct ndd_request_t {
+ int fd;
+ struct list_head list;
+} __packed;
+
+struct user_ndd_requests_t {
+ struct list_head ndd_req_list; /* ndd_request_t list */
+ struct mutex lock; /* protect the ndd_req_list list*/
+};
+
+struct ndd_frame_info_t {
+ struct list_head ndd_swfrm_list; /* ndd_swfrm_info_t list */
+ struct mutex lock; /* protect the ndd_swfrm_list list*/
+};
+
+struct ndd_wpe_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[WPE_NDD_SIZE];
+};
+
+struct ndd_traw_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[TRAW_NDD_SIZE];
+};
+
+struct ndd_dip_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[DIP_NDD_SIZE];
+};
+
+struct ndd_pqdip_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[PQDIP_NDD_SIZE];
+};
+
+struct ndd_me_info_t {
+ char fp[NDD_FP_SIZE];
+ u8 data[ME_NDD_SIZE];
+};
+
+struct ndd_frm_info_t {
+ struct ndd_wpe_info_t wpe[WPE_ENG_NUM];
+ struct ndd_traw_info_t traw[TRAW_ENG_NUM];
+ struct ndd_dip_info_t dip[DIP_ENG_NUM];
+ struct ndd_pqdip_info_t pqdip[PQDIP_ENG_NUM];
+ struct ndd_me_info_t me[ME_ENG_NUM];
+} __packed;
+
+static struct ndd_frame_info_t *ndd_frames;
+
+#if defined(CONFIG_DEBUG_FS)
+#define DEBUGFS_DIR "mtk_imgsys_debug"
+#define DEBUGFS_FILE "mtk_imgsys_ndd"
+#define FILE_PERMISSIONS 0644
+
+static struct dentry *debugfs_dir, *debugfs_file;
+static struct user_ndd_requests_t *ndd_reqs;
+
+static ssize_t debugfs_write(struct file *file, const char __user *user_buffer,
+ size_t count, loff_t *ppos)
+{
+ char buffer[64];
+ int ret;
+ struct ndd_request_t *current_request;
+
+ if (count >= sizeof(buffer))
+ return -EINVAL;
+
+ if (copy_from_user(buffer, user_buffer, count))
+ return -EFAULT;
+
+ buffer[count] = '\0';
+
+ current_request = kmalloc(sizeof(*current_request), GFP_KERNEL);
+ if (!current_request)
+ return -ENOMEM;
+
+ ret = sscanf(buffer, "fd=%d", ¤t_request->fd);
+ if (ret != 1)
+ return -EINVAL;
+
+ mutex_lock(&ndd_reqs->lock);
+ list_add_tail(¤t_request->list, &ndd_reqs->ndd_req_list);
+ mutex_unlock(&ndd_reqs->lock);
+
+ *ppos += count;
+
+ return count;
+}
+
+static void imgsys_ndd_dump_routine(struct mtk_imgsys_dev *imgsys_dev,
+ struct imgsys_ndd_frm_dump_info *frm_dump_info)
+{
+ int i = 0;
+ int mid = 0;
+ unsigned int hw_num = 0x1;
+ unsigned int module_copy_ofst = 0;
+ const struct module_ops *imgsys_modules = imgsys_dev->modules;
+
+ dev_dbg(imgsys_dev->dev,
+ "%s: hw comb set: 0x%x\n", __func__, frm_dump_info->hw_comb);
+
+ for (i = 0; hw_num < IMGSYS_ENG_MAX_NUM; i++, hw_num = (0x1 << i)) {
+ switch (hw_num) {
+ case IMGSYS_ENG_WPE_EIS:
+ case IMGSYS_ENG_WPE_TNR:
+ case IMGSYS_ENG_WPE_LITE:
+ module_copy_ofst = (NDD_FP_SIZE + WPE_NDD_SIZE);
+ mid = IMGSYS_MOD_WPE;
+ break;
+ case IMGSYS_ENG_TRAW:
+ case IMGSYS_ENG_LTR:
+ case IMGSYS_ENG_XTR:
+ module_copy_ofst = (NDD_FP_SIZE + TRAW_NDD_SIZE);
+ mid = IMGSYS_MOD_TRAW;
+ break;
+ case IMGSYS_ENG_DIP:
+ module_copy_ofst = (NDD_FP_SIZE + DIP_NDD_SIZE);
+ mid = IMGSYS_MOD_DIP;
+ break;
+ case IMGSYS_ENG_PQDIP_A:
+ case IMGSYS_ENG_PQDIP_B:
+ module_copy_ofst = (NDD_FP_SIZE + PQDIP_NDD_SIZE);
+ mid = IMGSYS_MOD_PQDIP;
+ break;
+ case IMGSYS_ENG_ME:
+ module_copy_ofst = (NDD_FP_SIZE + ME_NDD_SIZE);
+ mid = IMGSYS_MOD_ME;
+ break;
+ default:
+ dev_err(imgsys_dev->dev, "%s get wrong hw id %d\n", __func__, hw_num);
+ break;
+ }
+
+ if (frm_dump_info->hw_comb & hw_num) {
+ frm_dump_info->eng_e = i;
+ if (imgsys_modules[mid].ndddump &&
+ frm_dump_info->eng_e <= IMGSYS_NDD_ENG_ME)
+ imgsys_modules[mid].ndddump(imgsys_dev, frm_dump_info);
+ } else {
+ if (frm_dump_info->user_buffer)
+ frm_dump_info->user_buffer += module_copy_ofst;
+ else
+ frm_dump_info->user_buffer = NULL;
+ }
+
+ if ((frm_dump_info->hw_comb >> i) == 0)
+ return;
+ }
+}
+
+static ssize_t debugfs_read(struct file *file, char __user *user_buffer,
+ size_t count, loff_t *ppos)
+{
+ struct ndd_swfrm_info_t *cur_ndd_swfrm = NULL, *swfrm_temp = NULL;
+ struct ndd_request_t *cur_user_req = NULL, *req_temp = NULL;
+ struct img_swfrm_info *subfrm_info = NULL;
+ struct imgsys_ndd_frm_dump_info frm_dump_info = { 0 };
+ struct gce_cb_work_pool *work_pool = NULL;
+ ssize_t read_count = 0;
+ size_t buffer_size = count;
+ bool found = false;
+ int frm, ret = 0;
+ u32 gce_cb_work_idx = MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS;
+
+ mutex_lock(&ndd_frames->lock);
+
+ list_for_each_entry_safe(cur_ndd_swfrm, swfrm_temp, &ndd_frames->ndd_swfrm_list, list) {
+ mutex_lock(&ndd_reqs->lock);
+ list_for_each_entry_safe(cur_user_req, req_temp, &ndd_reqs->ndd_req_list, list) {
+ if (cur_ndd_swfrm->gwfrm_info &&
+ cur_ndd_swfrm->gwfrm_info->request_fd == cur_user_req->fd) {
+ found = true;
+ break;
+ }
+ }
+ mutex_unlock(&ndd_reqs->lock);
+
+ if (found) {
+ for (frm = 0; frm < cur_ndd_swfrm->gwfrm_info->total_frmnum; frm++) {
+ subfrm_info = &cur_ndd_swfrm->gwfrm_info->user_info[frm];
+ memset(&frm_dump_info, 0, sizeof(frm_dump_info));
+
+ if (buffer_size < sizeof(struct ndd_frm_info_t)) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ if (subfrm_info->ndd_fp[0] == '\0')
+ continue;
+
+ frm_dump_info.user_buffer = user_buffer +
+ sizeof(struct ndd_frm_info_t) * frm;
+ frm_dump_info.hw_comb = subfrm_info->hw_comb;
+ frm_dump_info.cq_ofst = subfrm_info->ndd_cq_ofst;
+ frm_dump_info.wpe_psp_ofst = subfrm_info->ndd_wpe_psp_ofst;
+ frm_dump_info.wpe_fp = subfrm_info->ndd_wpe_fp;
+ frm_dump_info.fp = subfrm_info->ndd_fp;
+
+ imgsys_ndd_dump_routine(cur_ndd_swfrm->imgsys_dev, &frm_dump_info);
+ read_count += sizeof(struct ndd_frm_info_t);
+ buffer_size -= sizeof(struct ndd_frm_info_t);
+ }
+
+ work_pool = cur_ndd_swfrm->imgsys_dev->work_pool;
+ do {
+ gce_cb_work_idx = atomic_read(&cur_ndd_swfrm->gce_cb_work_idx);
+ } while (gce_cb_work_idx == MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS);
+ atomic_set(&work_pool->work_items[gce_cb_work_idx].state, WORK_COMPLETED);
+ wake_up(&work_pool->work_items[gce_cb_work_idx].wait_queue);
+
+ list_del(&cur_ndd_swfrm->list);
+ kfree(cur_ndd_swfrm);
+ cur_ndd_swfrm = NULL;
+ list_del(&cur_user_req->list);
+ kfree(cur_user_req);
+ cur_user_req = NULL;
+ break;
+ }
+ }
+
+ if (!found)
+ ret = -ENOENT;
+out:
+ mutex_unlock(&ndd_frames->lock);
+
+ return ret == 0 ? read_count : ret;
+}
+
+static const struct file_operations debugfs_fops = {
+ .write = debugfs_write,
+ .read = debugfs_read,
+};
+
+void imgsys_ndd_debugfs_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+ debugfs_dir = debugfs_create_dir(DEBUGFS_DIR, NULL);
+ if (!debugfs_dir) {
+ dev_err(imgsys_dev->dev, "%s Failed to create debugfs dir\n", __func__);
+ return;
+ }
+
+ debugfs_file = debugfs_create_file(DEBUGFS_FILE, FILE_PERMISSIONS,
+ debugfs_dir, NULL, &debugfs_fops);
+ if (!debugfs_file) {
+ dev_err(imgsys_dev->dev, "%s Failed to create debugfs file\n", __func__);
+ return;
+ }
+
+ ndd_frames = kmalloc(sizeof(*ndd_frames), GFP_KERNEL);
+ if (!ndd_frames)
+ return;
+ INIT_LIST_HEAD(&ndd_frames->ndd_swfrm_list);
+ mutex_init(&ndd_frames->lock);
+
+ ndd_reqs = kmalloc(sizeof(*ndd_reqs), GFP_KERNEL);
+ if (!ndd_reqs)
+ return;
+ INIT_LIST_HEAD(&ndd_reqs->ndd_req_list);
+ mutex_init(&ndd_reqs->lock);
+}
+EXPORT_SYMBOL_GPL(imgsys_ndd_debugfs_init);
+
+void imgsys_ndd_debugfs_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+ struct ndd_request_t *req, *tmp;
+
+ debugfs_remove(debugfs_file);
+ debugfs_remove(debugfs_dir);
+
+ mutex_lock(&ndd_reqs->lock);
+ list_for_each_entry_safe(req, tmp, &ndd_reqs->ndd_req_list, list) {
+ list_del(&req->list);
+ kfree(req);
+ }
+ mutex_unlock(&ndd_reqs->lock);
+ kfree(ndd_reqs);
+
+ mutex_lock(&ndd_frames->lock);
+ list_for_each_entry_safe(req, tmp, &ndd_frames->ndd_swfrm_list, list) {
+ list_del(&req->list);
+ kfree(req);
+ }
+ mutex_unlock(&ndd_frames->lock);
+ kfree(ndd_frames);
+}
+EXPORT_SYMBOL_GPL(imgsys_ndd_debugfs_uninit);
+#endif
+
+void imgsys_ndd_swfrm_list_add(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *gwfrm_info)
+{
+ struct ndd_swfrm_info_t *new_ndd_swfrm;
+
+ new_ndd_swfrm = kzalloc(sizeof(*new_ndd_swfrm), GFP_KERNEL);
+ if (!new_ndd_swfrm)
+ return;
+
+ new_ndd_swfrm->imgsys_dev = imgsys_dev;
+ new_ndd_swfrm->gwfrm_info = gwfrm_info;
+ atomic_set(&new_ndd_swfrm->gce_cb_work_idx, MTK_IMGSYS_GCE_CB_MAX_WORK_ITEMS);
+
+ gwfrm_info->ndd_swfrm_info = (void *)new_ndd_swfrm;
+
+ mutex_lock(&ndd_frames->lock);
+ list_add_tail(&new_ndd_swfrm->list, &ndd_frames->ndd_swfrm_list);
+ mutex_unlock(&ndd_frames->lock);
+}
+EXPORT_SYMBOL_GPL(imgsys_ndd_swfrm_list_add);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
new file mode 100644
index 000000000000..f45cacc7bfff
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-ndd_debugfs.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018 MediaTek Inc.
+ *
+ * Author: Christopher Chen <christopher.chen@mediatek.com>
+ *
+ */
+
+#ifndef _MTK_IMGSYS_NDD_DEBUGFS_H_
+#define _MTK_IMGSYS_NDD_DEBUGFS_H_
+
+#include "mtk_imgsys-dev.h"
+
+#define NDD_FP_SIZE 256 /* NDD file path size */
+
+enum mtk_imgsys_ndd_eng {
+ IMGSYS_NDD_ENG_WPE_EIS = 0,
+ IMGSYS_NDD_ENG_WPE_TNR = 1,
+ IMGSYS_NDD_ENG_WPE_LITE = 2,
+ IMGSYS_NDD_ENG_ADL_A = 3,
+ IMGSYS_NDD_ENG_ADL_B = 4,
+ IMGSYS_NDD_ENG_TRAW = 5,
+ IMGSYS_NDD_ENG_LTR = 6,
+ IMGSYS_NDD_ENG_XTR = 7,
+ IMGSYS_NDD_ENG_DIP = 8,
+ IMGSYS_NDD_ENG_PQDIP_A = 9,
+ IMGSYS_NDD_ENG_PQDIP_B = 10,
+ IMGSYS_NDD_ENG_ME = 11,
+ IMGSYS_NDD_ENG_MAX_NUM = 12
+};
+
+struct imgsys_ndd_frm_dump_info {
+ char __user *user_buffer;
+ unsigned int hw_comb;
+ enum mtk_imgsys_ndd_eng eng_e;
+ u64 *cq_ofst;
+ u64 *wpe_psp_ofst;
+ char *wpe_fp;
+ char *fp;
+};
+
+struct ndd_swfrm_info_t {
+ struct mtk_imgsys_dev *imgsys_dev;
+ struct swfrm_info *gwfrm_info;
+ struct list_head list;
+ atomic_t gce_cb_work_idx;
+} __packed;
+
+#if defined(CONFIG_DEBUG_FS)
+void imgsys_ndd_debugfs_init(struct mtk_imgsys_dev *imgsys_dev);
+void imgsys_ndd_debugfs_uninit(struct mtk_imgsys_dev *imgsys_dev);
+#else
+static inline void imgsys_ndd_debugfs_init(struct mtk_imgsys_dev *imgsys_dev)
+{
+}
+
+static inline void imgsys_ndd_debugfs_uninit(struct mtk_imgsys_dev *imgsys_dev)
+{
+}
+#endif
+void imgsys_ndd_swfrm_list_add(struct mtk_imgsys_dev *imgsys_dev,
+ struct swfrm_info *gwfrm_info);
+#endif /* _MTK_IMGSYS_DEBUG_H_ */
+
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
index 36088a5bda8e..a87b254cd114 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
@@ -16,6 +16,7 @@
#include "mtk_imgsys-gce_work.h"
#include "mtk_header_desc.h"
#include "mtk_imgsys_v4l2_vnode.h"
+#include "mtk_imgsys-ndd_debugfs.h"
#define FRAME_STATE_INIT 0
#define FRAME_STATE_HW_TIMEOUT 1
@@ -60,6 +61,7 @@ static void imgsys_cmdq_cb_func(struct imgsys_cmdq_cb_data data)
struct mtk_imgsys_dev *imgsys_dev;
struct swfrm_info *swfrminfo_cb;
struct gce_cb_work *gwork;
+ struct ndd_swfrm_info_t *ndd_swfrm;
swfrminfo_cb = data.data;
pipe = (struct mtk_imgsys_pipe *)swfrminfo_cb->pipe;
@@ -94,6 +96,12 @@ static void imgsys_cmdq_cb_func(struct imgsys_cmdq_cb_data data)
gwork->pipe = swfrminfo_cb->pipe;
init_waitqueue_head(&gwork->wait_queue);
+ if (swfrminfo_cb->user_info[0].ndd_fp[0] != '\0') {
+ ndd_swfrm = (struct ndd_swfrm_info_t *)swfrminfo_cb->ndd_swfrm_info;
+ if (ndd_swfrm)
+ atomic_set(&ndd_swfrm->gce_cb_work_idx, gwork->work_idx);
+ }
+
queue_work(req->imgsys_pipe->imgsys_dev->gcecb_wq, &gwork->work);
}
}
@@ -207,6 +215,9 @@ static void imgsys_scp_handler(void *data, unsigned int len, void *priv)
up(&imgsys_dev->sem);
+ if (swfrm_info->user_info[0].ndd_fp[0] != '\0')
+ imgsys_ndd_swfrm_list_add(imgsys_dev, swfrm_info);
+
INIT_WORK(&req->runner_work, imgsys_runner_func);
queue_work(req->imgsys_pipe->imgsys_dev->runner_wq,
&req->runner_work);
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
index ea2ce03c2619..b0088d4de69c 100644
--- a/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
+++ b/drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
@@ -26,6 +26,7 @@
#include "modules/mtk_imgsys-traw.h"
#include "modules/mtk_imgsys-pqdip.h"
#include "modules/mtk_imgsys-wpe.h"
+#include "modules/mtk_imgsys-me.h"
#include "mtk_imgsys-of.h"
#include "mtk_imgsys-vnode_id.h"
#include "mtk_imgsys_v4l2_vnode.h"
@@ -1846,31 +1847,37 @@ static const struct module_ops imgsys_isp7_modules[] = {
[IMGSYS_MOD_IMGMAIN] = {
.module_id = IMGSYS_MOD_IMGMAIN,
.init = imgsys_main_init,
+ .ndddump = NULL,
.uninit = imgsys_main_uninit,
},
[IMGSYS_MOD_WPE] = {
.module_id = IMGSYS_MOD_WPE,
.init = imgsys_wpe_init,
+ .ndddump = imgsys_wpe_ndd_dump,
.uninit = imgsys_wpe_uninit,
},
[IMGSYS_MOD_TRAW] = {
.module_id = IMGSYS_MOD_TRAW,
.init = imgsys_traw_init,
+ .ndddump = imgsys_traw_ndd_dump,
.uninit = imgsys_traw_uninit,
},
[IMGSYS_MOD_DIP] = {
.module_id = IMGSYS_MOD_DIP,
.init = imgsys_dip_init,
+ .ndddump = imgsys_dip_ndd_dump,
.uninit = imgsys_dip_uninit,
},
[IMGSYS_MOD_PQDIP] = {
.module_id = IMGSYS_MOD_PQDIP,
.init = imgsys_pqdip_init,
+ .ndddump = imgsys_pqdip_ndd_dump,
.uninit = imgsys_pqdip_uninit,
},
[IMGSYS_MOD_ME] = {
.module_id = IMGSYS_MOD_ME,
.init = NULL,
+ .ndddump = imgsys_me_ndd_dump,
.uninit = NULL,
},
};
--
2.45.2
^ permalink raw reply related [flat|nested] 15+ messages in thread[parent not found: <20250524115144.3832748-5-olivia.wen@mediatek.com>]
* Re: [PATCH v1 04/10] media: mediatek: isp: Add V4L2 flow support for ImgSys driver
[not found] ` <20250524115144.3832748-5-olivia.wen@mediatek.com>
@ 2025-05-26 7:16 ` CK Hu (胡俊光)
0 siblings, 0 replies; 15+ messages in thread
From: CK Hu (胡俊光) @ 2025-05-26 7:16 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, AngeloGioacchino Del Regno,
matthias.bgg@gmail.com, Olivia Wen (温倩苓)
Cc: linux-media@vger.kernel.org, Project_Global_Chrome_Upstream_Group,
devicetree@vger.kernel.org, linux-kernel@vger.kernel.org,
krzk+dt@kernel.org, linux-arm-kernel@lists.infradead.org,
Teddy Chen (陳乾元), conor+dt@kernel.org,
linux-mediatek@lists.infradead.org, yunkec@chromium.org
On Sat, 2025-05-24 at 19:49 +0800, Olivia Wen wrote:
> The ImgSys driver is implemented as a series of patches, with this patch
> focusing on the V4L2 flow. The MediaTek's Image System (ImgSys), also
> known as ISP Pass2. ImgSys is a memory-to-memory hardware device
> designed for advanced image processing tasks. It is composed of multiple
> hardware components, including TRAW, DIP, PQDIP, ME, and WPE.
>
> TRAW (Tile RAW):
> - Provides multiple downscaled resizers to support multi-scale noise
> reduction.
> - Supports RAW/RGB format conversion.
>
> DIP (Digital Image Processing) and PQDIP (Picture Quality DIP):
> - Supports such as temporal and spatial noise reduction (TNR/SNR),
> edge enhancement (EE), and sharpness (SHP).
> - PQDIP also supports image scaling and rotation.
>
> ME (Motion Estimation):
> - Supports motion estimation between two consecutive frames.
>
> WPE (Warping Engine):
> - Handles image warping and de-warping processes.
>
> The ImgSys driver, primarily utilized for Google Chromebook products,
> uses Multi-Frame Combined Noise Reduction technology to enhance image
> quality.
>
> Below is a simplified architecture of the ImgSys driver:
> User -> V4L2 Framework -> ImgSys Driver
> -> SCP (It packages the hardware settings into commands)
> -> ImgSys Driver
> -> CMDQ (The packaged commands are sent to the hardware via GCE)
> -> Hardware
You break this driver into multiple patches by these function block,
but this would make reviewer (include me) hard to review.
The better way is
1. Add imgsys driver with basic function (include v4l2 framework, scp, cmdq. The basic function may be downscale and format convert only)
2. Add DIP function
3. Add ME function
4. Add WPE function
Let the first patch as simple as possible but it should work.
>
> Signed-off-by: Olivia Wen <olivia.wen@mediatek.com>
> ---
> drivers/media/platform/mediatek/Kconfig | 1 +
> drivers/media/platform/mediatek/Makefile | 1 +
> drivers/media/platform/mediatek/isp/Kconfig | 23 +
> .../platform/mediatek/isp/isp_7x/Makefile | 6 +
> .../mediatek/isp/isp_7x/imgsys/Makefile | 11 +
> .../imgsys/modules/mtk_dip_v4l2_vnode.h | 594 ++++++
> .../isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h | 386 ++++
> .../imgsys/modules/mtk_pqdip_v4l2_vnode.h | 149 ++
> .../imgsys/modules/mtk_traw_v4l2_vnode.h | 371 ++++
> .../imgsys/modules/mtk_wpe_v4l2_vnode.h | 317 ++++
> .../isp/isp_7x/imgsys/mtk_header_desc.h | 28 +
> .../isp/isp_7x/imgsys/mtk_imgsys-dev.c | 238 +++
> .../isp/isp_7x/imgsys/mtk_imgsys-dev.h | 427 +++++
> .../isp/isp_7x/imgsys/mtk_imgsys-formats.c | 139 ++
> .../isp/isp_7x/imgsys/mtk_imgsys-formats.h | 72 +
> .../isp/isp_7x/imgsys/mtk_imgsys-module_ops.h | 27 +
> .../isp/isp_7x/imgsys/mtk_imgsys-of.c | 39 +
> .../isp/isp_7x/imgsys/mtk_imgsys-of.h | 55 +
> .../isp/isp_7x/imgsys/mtk_imgsys-sys.c | 27 +
> .../isp/isp_7x/imgsys/mtk_imgsys-sys.h | 50 +
> .../isp/isp_7x/imgsys/mtk_imgsys-v4l2.c | 1684 +++++++++++++++++
> .../isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h | 100 +
> .../isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h | 135 ++
> 23 files changed, 4880 insertions(+)
> create mode 100644 drivers/media/platform/mediatek/isp/Kconfig
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/Makefile
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_dip_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_me_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_pqdip_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_traw_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/modules/mtk_wpe_v4l2_vnode.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_header_desc.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-dev.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-formats.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-module_ops.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-of.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-sys.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-v4l2.c
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys-vnode_id.h
> create mode 100644 drivers/media/platform/mediatek/isp/isp_7x/imgsys/mtk_imgsys_v4l2_vnode.h
Place these files in drivers/media/platform/mediatek/imgsys.
This is the first SoC which support imgsys driver so it's not necessary to use so deep folder structure.
Regards,
CK
^ permalink raw reply [flat|nested] 15+ messages in thread