* [PATCH v2 00/13] Add MediaTek ISP7.x camera system support
@ 2025-07-07 1:31 shangyao lin
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
` (13 more replies)
0 siblings, 14 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
Based on linux-next tag: next-20250630
This patch set adds the MediaTek ISP7.x camera system hardware driver.
The driver sets up ISP hardware, handles interrupts, and initializes
V4L2 device nodes and functions. It also implements a V4L2 standard video
driver utilizing the media framework APIs, connects sensors and the ISP
via the seninf interface, and communicates with the SCP co-processor to
compose ISP registers in firmware.
These patches include:
CSI data reception from sensors Sensor interface bridge RAW/YUV image
pre-processing ISP utility and control modules
Changelog (v2):
- Fix documentation typos
- Address review comments from v1
Note: At this stage (v2), the related firmware interface and headers are
still under development and review. Once the code is more stable and ready
for upstream submission, we will provide the corresponding linux-firmware
submission and clearly document which MediaTek SCP firmware version is
required for this driver. Thank you for reviewing these patches.
shangyao.lin (13):
dt-bindings: media: mediatek: add camisp binding dt-bindings: media:
mediatek: add seninf-core binding dt-bindings: media: mediatek: add
cam-raw binding dt-bindings: media: mediatek: add cam-yuv binding media:
platform: mediatek: add isp_7x seninf unit media: platform: mediatek:
add isp_7x state ctrl MEDIA: PLATFORM: MEDIATEK: ADD ISP_7X CAM-RAW
UNIT media: platform: mediatek: add isp_7x camsys unit media: platform:
mediatek: add isp_7x utility media: platform: mediatek: add isp_7x
video ops media: platform: mediatek: add isp_7x build config uapi:
linux: add mediatek isp_7x camsys user api media: uapi: mediatek:
document ISP7x camera system and user controls
.../mediatek/mediatek,mt8188-cam-raw.yaml | 156 +
.../mediatek/mediatek,mt8188-cam-yuv.yaml | 134 +
.../mediatek/mediatek,mt8188-camisp.yaml | 68 +
.../mediatek/mediatek,mt8188-seninf-core.yaml | 121 +
.../media/v4l/mtk-isp7x-camsys.rst | 94 +
.../media/v4l/mtk-isp7x-controls.rst | 199 +
drivers/media/platform/mediatek/Kconfig | 1 +
drivers/media/platform/mediatek/Makefile | 2 +
drivers/media/platform/mediatek/isp/Kconfig | 21 +
.../platform/mediatek/isp/isp_7x/Makefile | 7 +
.../mediatek/isp/isp_7x/camsys/Makefile | 14 +
.../isp_7x/camsys/kd_imgsensor_define_v4l2.h | 86 +
.../mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c | 1409 +++++
.../mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h | 131 +
.../mediatek/isp/isp_7x/camsys/mtk_cam-defs.h | 161 +
.../mediatek/isp/isp_7x/camsys/mtk_cam-fmt.h | 87 +
.../mediatek/isp/isp_7x/camsys/mtk_cam-ipi.h | 213 +
.../isp/isp_7x/camsys/mtk_cam-meta-mt8188.h | 2296 ++++++++
.../isp/isp_7x/camsys/mtk_cam-plat-util.c | 141 +
.../isp/isp_7x/camsys/mtk_cam-plat-util.h | 15 +
.../mediatek/isp/isp_7x/camsys/mtk_cam-pool.c | 394 ++
.../mediatek/isp/isp_7x/camsys/mtk_cam-pool.h | 28
+ .../mediatek/isp/isp_7x/camsys/mtk_cam-raw.c | 4883
+++++++++++++++++ .../mediatek/isp/isp_7x/camsys/mtk_cam-raw.h |
323 ++ .../isp/isp_7x/camsys/mtk_cam-regs-mt8188.h | 374
++ .../isp/isp_7x/camsys/mtk_cam-seninf-def.h | 237 +
.../isp/isp_7x/camsys/mtk_cam-seninf-drv.c | 1441 +++++
.../isp/isp_7x/camsys/mtk_cam-seninf-drv.h | 16 +
.../isp/isp_7x/camsys/mtk_cam-seninf-hw.h | 108 +
.../isp/isp_7x/camsys/mtk_cam-seninf-if.h | 24 +
.../isp/isp_7x/camsys/mtk_cam-seninf-regs.h | 44 +
.../isp/isp_7x/camsys/mtk_cam-seninf-route.c | 279 +
.../isp/isp_7x/camsys/mtk_cam-seninf-route.h | 20 +
.../isp/isp_7x/camsys/mtk_cam-seninf.h | 161 +
.../isp/isp_7x/camsys/mtk_cam-timesync.c | 125 +
.../isp/isp_7x/camsys/mtk_cam-timesync.h | 12 +
.../isp/isp_7x/camsys/mtk_cam-video.c | 1594
++++++ .../isp/isp_7x/camsys/mtk_cam-video.h |
223 + .../mediatek/isp/isp_7x/camsys/mtk_cam.c | 3977
++++++++++++++ .../mediatek/isp/isp_7x/camsys/mtk_cam.h |
718 +++ .../isp_7x/camsys/mtk_camera-v4l2-controls.h | 63
+ .../isp_7x/camsys/mtk_csi_phy_2_0/Makefile | 5 +
.../mtk_csi_phy_2_0/mtk_cam-seninf-cammux.h | 679 +++
.../mtk_cam-seninf-csi0-cphy.h | 51 +
.../mtk_cam-seninf-csi0-dphy.h | 98 +
.../mtk_cam-seninf-hw_phy_2_0.c | 1932 +++++++
.../mtk_cam-seninf-mipi-rx-ana-cdphy-csi0a.h | 165 +
.../mtk_cam-seninf-seninf1-csi2.h | 264 +
.../mtk_cam-seninf-seninf1-mux.h | 120 +
.../mtk_csi_phy_2_0/mtk_cam-seninf-seninf1.h | 46 +
.../mtk_csi_phy_2_0/mtk_cam-seninf-tg1.h | 43 +
.../mtk_csi_phy_2_0/mtk_cam-seninf-top-ctrl.h | 76 +
include/uapi/linux/mtkisp_camsys.h | 132 +
include/uapi/linux/videodev2.h | 87 +
54 files changed, 24098 insertions(+) create mode 100755
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml
create mode 100755
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
create mode 100755
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
create mode 100755
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml
create mode 100755
Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst create mode
100755 Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst mode
change 100644 => 100755 drivers/media/platform/mediatek/Kconfig mode
change 100644 => 100755 drivers/media/platform/mediatek/Makefile create
mode 100755 drivers/media/platform/mediatek/isp/Kconfig create mode
100755 drivers/media/platform/mediatek/isp/isp_7x/Makefile create mode
100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/Makefile
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/kd_imgsensor_define_v4l2.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-defs.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-fmt.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ipi.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-meta-mt8188.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-plat-util.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-plat-util.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-pool.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-pool.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-raw.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-raw.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-regs-mt8188.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-def.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-drv.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-drv.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-hw.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-if.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-regs.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-route.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-route.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-timesync.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-timesync.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.h create
mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_camera-v4l2-controls.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/Makefile
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-cammux.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-csi0-cphy.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-csi0-dphy.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-hw_phy_2_0.c
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-mipi-rx-ana-cdphy-csi0a.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-seninf1-csi2.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-seninf1-mux.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-seninf1.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-tg1.h
create mode 100755
drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-top-ctrl.h
create mode 100755 include/uapi/linux/mtkisp_camsys.h mode change 100644
=> 100755 include/uapi/linux/videodev2.h
-- 2.18.0
^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
` (2 more replies)
2025-07-07 1:31 ` [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding shangyao lin
` (12 subsequent siblings)
13 siblings, 3 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Add camera isp7x module device document.
---
Changes in v2:
- Rename binding file to mediatek,mt8188-camisp.yaml
- Split bindings into four separate patches (one per YAML file)
- Various fixes per review comments
- Update maintainers list
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../mediatek/mediatek,mt8188-camisp.yaml | 68 +++++++++++++++++++
1 file changed, 68 insertions(+)
create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
new file mode 100755
index 000000000000..53dbf5152e78
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (c) 2024 MediaTek Inc.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek/mediatek,camisp.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: The camisp unit of MediaTek ISP system
+
+maintainers:
+ - Shangyao Lin <shangyao.lin@mediatek.com>
+ - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
+ - Shun-yi Wang <shun-yi.wang@mediatek.com>
+ - Teddy Chen <teddy.chen@mediatek.com>
+
+description:
+ MediaTek camisp is the ISP auxiliary unit for camera system in MediaTek SoC.
+
+properties:
+ compatible:
+ const: mediatek,mt8188-camisp
+
+ reg:
+ minItems: 1
+ maxItems: 1
+ description:
+ Base address of the camisp hardware block.
+
+ reg-names:
+ items:
+ - const: base
+ minItems: 1
+ maxItems: 1
+ description:
+ Name for the register region. Must be "base".
+
+ power-domains:
+ minItems: 1
+ maxItems: 1
+ description: Power domain for the camisp block.
+
+ mediatek,scp:
+ description: MediaTek co-process unit for ISP system
+ $ref: /schemas/types.yaml#/definitions/phandle
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - power-domains
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/power/mediatek,mt8188-power.h>
+
+ soc {
+ isp@16000000 {
+ compatible = "mediatek,mt8188-camisp";
+ reg = <0 0x16000000 0 0x1000>;
+ reg-names = "base";
+ mediatek,scp = <&scp_dual>;
+ power-domains = <&spm MT8188_POWER_DOMAIN_CAM_MAIN>;
+ };
+ };
+
+...
\ No newline at end of file
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
` (2 more replies)
2025-07-07 1:31 ` [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding shangyao lin
` (11 subsequent siblings)
13 siblings, 3 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
1. Add camera isp7x module device document
---
Changes in v2:
- Rename binding file to mediatek,mt8188-seninf-core.yaml
- Various fixes per review comments
- Update maintainers list
Question for reviewer (CK):
Hi CK,
Thank you for your review and suggestions on this patch, especially for providing the reference patch (https://patchwork.kernel.org/project/linux-mediatek/list/?series=874617) and for mentioning in another patch ([V1,02/10] MEDIA: PLATFORM: MEDIATEK: ADD SENINF CONTROLLER) the suggestion to "Move the phy part to phy/mediatek/ folder. You could refer to phy/mediatek/phy-mtk-mipi-csi-0-5.c".
After reading your comments and the reference patches, my understanding is that only the seninf-core driver should manage all ports internally, and each port corresponds to a PHY. During probe, the driver will parse each port, obtain the corresponding PHY (e.g., devm_phy_get(dev, "csi0"), devm_phy_get(dev, "csi1"), etc.), and operate the PHY for each port individually during stream on/off or power on/off.
Could you please confirm if my understanding is correct?
If you have any additional reference patches or examples, I would greatly appreciate it.
Thank you for your guidance!
Best regards,
Shangyao
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../mediatek/mediatek,mt8188-seninf-core.yaml | 121 ++++++++++++++++++
1 file changed, 121 insertions(+)
create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml
new file mode 100755
index 000000000000..763b96b561cf
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml
@@ -0,0 +1,121 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (c) 2023 MediaTek Inc.
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/mediatek,seninf-core.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: The seninf-core top unit of MediaTek ISP system
+
+maintainers:
+ - Shangyao Lin <shangyao.lin@mediatek.com>
+ - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
+ - Shun-yi Wang <shun-yi.wang@mediatek.com>
+ - Teddy Chen <teddy.chen@mediatek.com>
+
+description: |
+ MediaTek seninf-core is the ISP sensor interface unit in MediaTek SoC.
+ The sensor interface serves as the MIPI-CSI2 top RX controller.
+
+properties:
+ compatible:
+ const: mediatek,mt8188-seninf-core
+
+ reg:
+ minItems: 1
+ maxItems: 1
+ description: |
+ Base address register region.
+
+ reg-names:
+ items:
+ - const: base
+ minItems: 1
+ maxItems: 1
+
+ mtk_csi_phy_ver:
+ description: Describes MediaTek camera Rx controller version
+ $ref: /schemas/types.yaml#/definitions/string
+
+ interrupts:
+ minItems: 1
+ maxItems: 2
+
+ power-domains:
+ minItems: 1
+ maxItems: 4
+
+ clocks:
+ minItems: 4
+ maxItems: 4
+ description: List of clock phandles required by the controller.
+
+ clock-names:
+ items:
+ - const: clk_cam_seninf
+ - const: clk_top_seninf
+ - const: clk_top_seninf1
+ - const: clk_top_camtm
+ minItems: 4
+ maxItems: 4
+
+ mediatek,larbs:
+ description: |
+ List of phandles to the local arbiters in the current SoCs.
+ Refer to bindings/memory-controllers/mediatek,smi-larb.yaml.
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ minItems: 1
+ maxItems: 32
+
+ dma-ranges:
+ description: |
+ Describes the address information of IOMMU mapping to memory.
+ Defines six fields for the MediaTek IOMMU extended iova, pa, and size.
+ minItems: 1
+
+ phys:
+ description: List of phandle and args to the PHY provider.
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+
+ phy-names:
+ description: Names of the PHYs.
+ $ref: /schemas/types.yaml#/definitions/string-array
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - power-domains
+ - clocks
+ - clock-names
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+ #include <dt-bindings/power/mediatek,mt8188-power.h>
+ #include <dt-bindings/clock/mediatek,mt8188-clk.h>
+
+ seninf@16010000 {
+ compatible = "mediatek,mt8188-seninf-core";
+ reg = <0 0x16010000 0 0x8000>;
+ reg-names = "base";
+ mtk_csi_phy_ver = "mtk_csi_phy_2_0";
+ interrupts = <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH 0>;
+ power-domains = <&spm MT8188_POWER_DOMAIN_CSIRX_TOP>,
+ <&spm MT8188_POWER_DOMAIN_CAM_VCORE>,
+ <&spm MT8188_POWER_DOMAIN_CAM_MAIN>;
+ clocks = <&camsys CLK_CAM_MAIN_SENINF>,
+ <&topckgen CLK_TOP_SENINF>,
+ <&topckgen CLK_TOP_SENINF1>,
+ <&topckgen CLK_TOP_CAMTM>;
+ clock-names = "clk_cam_seninf",
+ "clk_top_seninf",
+ "clk_top_seninf1",
+ "clk_top_camtm";
+ };
+
+...
\ No newline at end of file
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
2025-07-07 1:31 ` [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
` (2 more replies)
2025-07-07 1:31 ` [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding shangyao lin
` (10 subsequent siblings)
13 siblings, 3 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Add camera isp7x module device document.
---
Changes in v2:
- Rename binding file to mediatek,mt8188-cam-raw.yaml
- Various fixes per review comments
- Update maintainers list
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../mediatek/mediatek,mt8188-cam-raw.yaml | 156 ++++++++++++++++++
1 file changed, 156 insertions(+)
create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml
new file mode 100755
index 000000000000..dfedb229e79c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml
@@ -0,0 +1,156 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (c) 2024 MediaTek Inc.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-raw.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: The cam-raw unit of MediaTek ISP system
+
+maintainers:
+ - Shangyao Lin <shangyao.lin@mediatek.com>
+ - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
+ - Shun-yi Wang <shun-yi.wang@mediatek.com>
+ - Teddy Chen <teddy.chen@mediatek.com>
+
+description:
+ MediaTek cam-raw is the camera RAW processing unit in MediaTek SoC.
+
+properties:
+ compatible:
+ const: mediatek,mt8188-cam-raw
+
+ reg:
+ minItems: 1
+ maxItems: 2
+ description:
+ Base address and optional inner base address of the cam-raw hardware block.
+
+ reg-names:
+ items:
+ - const: base
+ - const: inner_base
+ minItems: 1
+ maxItems: 2
+ description:
+ Names for each register region. Must be "base" and optionally "inner_base".
+
+ mediatek,larbs:
+ description:
+ List of phandles to the local arbiters in the current SoCs.
+ Refer to bindings/memory-controllers/mediatek,smi-larb.yaml.
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ minItems: 1
+ maxItems: 32
+
+ interrupts:
+ minItems: 1
+ description: Interrupts for the cam-raw block.
+
+ dma-ranges:
+ minItems: 1
+ description: Address information of IOMMU mapping to memory.
+
+ power-domains:
+ minItems: 1
+ description: Power domains for the cam-raw block.
+
+ clocks:
+ minItems: 4
+ maxItems: 16
+ description: List of phandles to the clocks required by the cam-raw block.
+
+ clock-names:
+ items:
+ - const: camsys_cam2mm0_cgpdn
+ - const: camsys_cam2mm1_cgpdn
+ - const: camsys_cam2sys_cgpdn
+ - const: camsys_cam_cgpdn
+ - const: camsys_camtg_cgpdn
+ - const: camsys_rawa_larbx_cgpdn
+ - const: camsys_rawa_cam_cgpdn
+ - const: camsys_rawa_camtg_cgpdn
+ - const: topckgen_top_cam
+ - const: topckgen_top_camtg
+ - const: topckgen_top_camtm
+ minItems: 4
+ maxItems: 16
+ description: Names of the clocks, must match the order of the clocks property.
+
+ iommus:
+ minItems: 1
+ maxItems: 32
+ description: Points to the respective IOMMU block with master port as argument.
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - power-domains
+ - clocks
+ - clock-names
+ - iommus
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+ #include <dt-bindings/power/mediatek,mt8188-power.h>
+ #include <dt-bindings/clock/mediatek,mt8188-clk.h>
+ #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
+
+ soc {
+ raw@16030000 {
+ compatible = "mediatek,mt8188-cam-raw";
+ reg = <0 0x16030000 0 0x8000>,
+ <0 0x16038000 0 0x8000>;
+ reg-names = "base", "inner_base";
+ interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH 0>;
+ dma-ranges = <0x2 0x0 0x0 0x40000000 0x1 0x0>;
+ power-domains = <&spm MT8188_POWER_DOMAIN_CAM_SUBA>;
+ clocks = <&camsys CLK_CAM_MAIN_CAM2MM0_GALS>,
+ <&camsys CLK_CAM_MAIN_CAM2MM1_GALS>,
+ <&camsys CLK_CAM_MAIN_CAM2SYS_GALS>,
+ <&camsys CLK_CAM_MAIN_CAM>,
+ <&camsys CLK_CAM_MAIN_CAMTG>,
+ <&camsys_rawa CLK_CAM_RAWA_LARBX>,
+ <&camsys_rawa CLK_CAM_RAWA_CAM>,
+ <&camsys_rawa CLK_CAM_RAWA_CAMTG>,
+ <&topckgen CLK_TOP_CAM>,
+ <&topckgen CLK_TOP_CAMTG>,
+ <&topckgen CLK_TOP_CAMTM>;
+ clock-names = "camsys_cam2mm0_cgpdn",
+ "camsys_cam2mm1_cgpdn",
+ "camsys_cam2sys_cgpdn",
+ "camsys_cam_cgpdn",
+ "camsys_camtg_cgpdn",
+ "camsys_rawa_larbx_cgpdn",
+ "camsys_rawa_cam_cgpdn",
+ "camsys_rawa_camtg_cgpdn",
+ "topckgen_top_cam",
+ "topckgen_top_camtg",
+ "topckgen_top_camtm";
+ iommus = <&vpp_iommu M4U_PORT_L16A_IMGO_R1>,
+ <&vpp_iommu M4U_PORT_L16A_CQI_R1>,
+ <&vpp_iommu M4U_PORT_L16A_CQI_R2>,
+ <&vpp_iommu M4U_PORT_L16A_BPCI_R1>,
+ <&vpp_iommu M4U_PORT_L16A_LSCI_R1>,
+ <&vpp_iommu M4U_PORT_L16A_RAWI_R2>,
+ <&vpp_iommu M4U_PORT_L16A_RAWI_R3>,
+ <&vpp_iommu M4U_PORT_L16A_UFDI_R2>,
+ <&vpp_iommu M4U_PORT_L16A_UFDI_R3>,
+ <&vpp_iommu M4U_PORT_L16A_RAWI_R4>,
+ <&vpp_iommu M4U_PORT_L16A_RAWI_R5>,
+ <&vpp_iommu M4U_PORT_L16A_AAI_R1>,
+ <&vpp_iommu M4U_PORT_L16A_UFDI_R5>,
+ <&vpp_iommu M4U_PORT_L16A_FHO_R1>,
+ <&vpp_iommu M4U_PORT_L16A_AAO_R1>,
+ <&vpp_iommu M4U_PORT_L16A_TSFSO_R1>,
+ <&vpp_iommu M4U_PORT_L16A_FLKO_R1>;
+ };
+ };
+
+...
\ No newline at end of file
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (2 preceding siblings ...)
2025-07-07 1:31 ` [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
` (2 more replies)
2025-07-07 1:31 ` [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl shangyao lin
` (9 subsequent siblings)
13 siblings, 3 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Add camera isp7x module device document.
---
Changes in v2:
- Rename binding file to mediatek,mt8188-cam-yuv.yaml
- Various fixes per review comments
- Update maintainers list
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../mediatek/mediatek,mt8188-cam-yuv.yaml | 134 ++++++++++++++++++
1 file changed, 134 insertions(+)
create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
diff --git a/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
new file mode 100755
index 000000000000..0de120d3c6e8
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
@@ -0,0 +1,134 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (c) 2024 MediaTek Inc.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-yuv.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: The cam-yuv unit of MediaTek ISP system
+
+maintainers:
+ - Shangyao Lin <shangyao.lin@mediatek.com>
+ - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
+ - Shun-yi Wang <shun-yi.wang@mediatek.com>
+ - Teddy Chen <teddy.chen@mediatek.com>
+
+description:
+ MediaTek cam-yuv is the camera YUV processing unit in MediaTek SoC.
+
+properties:
+ compatible:
+ const: mediatek,mt8188-cam-yuv
+
+ reg:
+ minItems: 1
+ maxItems: 2
+ description:
+ Base address and optional inner base address of the cam-yuv hardware block.
+
+ reg-names:
+ items:
+ - const: base
+ - const: inner_base
+ minItems: 1
+ maxItems: 2
+ description:
+ Names for each register region. Must be "base" and optionally "inner_base".
+
+ mediatek,larbs:
+ description:
+ List of phandles to the local arbiters in the current SoCs.
+ Refer to bindings/memory-controllers/mediatek,smi-larb.yaml.
+ $ref: /schemas/types.yaml#/definitions/phandle-array
+ minItems: 1
+ maxItems: 32
+
+ interrupts:
+ minItems: 1
+ description: Interrupts for the cam-yuv block.
+
+ dma-ranges:
+ minItems: 1
+ description: Address information of IOMMU mapping to memory.
+
+ power-domains:
+ minItems: 1
+ description: Power domains for the cam-yuv block.
+
+ clocks:
+ minItems: 4
+ maxItems: 16
+ description: List of phandles to the clocks required by the cam-yuv block.
+
+ clock-names:
+ items:
+ - const: camsys_cam2mm0_cgpdn
+ - const: camsys_cam2mm1_cgpdn
+ - const: camsys_cam2sys_cgpdn
+ - const: camsys_yuva_larbx
+ - const: camsys_yuva_cam_cgpdn
+ - const: camsys_yuva_camtg_cgpdn
+ - const: camsys_yuvb_larbx_cgpdn
+ - const: camsys_yuvb_cam_cgpdn
+ - const: camsys_yuvb_camtg_cgpdn
+ minItems: 4
+ maxItems: 16
+ description: Names of the clocks, must match the order of the clocks property.
+
+ iommus:
+ minItems: 1
+ maxItems: 32
+ description: Points to the respective IOMMU block with master port as argument.
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - power-domains
+ - clocks
+ - clock-names
+ - iommus
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+ #include <dt-bindings/power/mediatek,mt8188-power.h>
+ #include <dt-bindings/clock/mediatek,mt8188-clk.h>
+ #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
+
+ soc {
+ yuv@16050000 {
+ compatible = "mediatek,mt8188-cam-yuv";
+ reg = <0 0x16050000 0 0x8000>;
+ reg-names = "base";
+ mediatek,larbs = <&larb17a>;
+ interrupts = <GIC_SPI 301 IRQ_TYPE_LEVEL_HIGH 0>;
+ dma-ranges = <0x2 0x0 0x0 0x40000000 0x1 0x0>;
+ power-domains = <&spm MT8188_POWER_DOMAIN_CAM_SUBA>;
+ clocks = <&camsys CLK_CAM_MAIN_CAM2MM0_GALS>,
+ <&camsys CLK_CAM_MAIN_CAM2MM1_GALS>,
+ <&camsys CLK_CAM_MAIN_CAM2SYS_GALS>,
+ <&camsys_yuva CLK_CAM_YUVA_LARBX>,
+ <&camsys_yuva CLK_CAM_YUVA_CAM>,
+ <&camsys_yuva CLK_CAM_YUVA_CAMTG>;
+ clock-names = "camsys_cam2mm0_cgpdn",
+ "camsys_cam2mm1_cgpdn",
+ "camsys_cam2sys_cgpdn",
+ "camsys_yuva_larbx_cgpdn",
+ "camsys_yuva_cam_cgpdn",
+ "camsys_yuva_camtg_cgpdn";
+ iommus = <&vpp_iommu M4U_PORT_L17A_YUVO_R1>,
+ <&vpp_iommu M4U_PORT_L17A_YUVO_R3>,
+ <&vpp_iommu M4U_PORT_L17A_YUVCO_R1>,
+ <&vpp_iommu M4U_PORT_L17A_YUVO_R2>,
+ <&vpp_iommu M4U_PORT_L17A_RZH1N2TO_R1>,
+ <&vpp_iommu M4U_PORT_L17A_DRZS4NO_R1>,
+ <&vpp_iommu M4U_PORT_L17A_TNCSO_R1>;
+ };
+ };
+
+...
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (3 preceding siblings ...)
2025-07-07 1:31 ` [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-11 8:56 ` CK Hu (胡俊光)
2025-07-21 1:40 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 10/13] media: platform: mediatek: add isp_7x video ops shangyao lin
` (8 subsequent siblings)
13 siblings, 2 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Introduce state management and debugging mechanisms for the MediaTek ISP7.x
camsys platform. State management establishes control over ISP operations and
events, defining distinct states for request handling, sensor control, and
frame synchronization, ensuring proper event processing. The debugging
mechanism ensures stable operation and timely data collection during anomalies.
---
Changes in v2:
- Removed mtk_cam-debug.c and mtk_cam-debug.h, along with related code
- Various fixes per review comments
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c | 1409 +++++++++++++++++
.../mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h | 131 ++
2 files changed, 1540 insertions(+)
create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
new file mode 100755
index 000000000000..15af1eac5444
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
@@ -0,0 +1,1409 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ */
+
+#include <linux/list.h>
+#include <linux/math64.h>
+#include <linux/of.h>
+#include <linux/videodev2.h>
+#include <linux/hrtimer.h>
+
+#include <media/v4l2-event.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-ioctl.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-ctrl.h"
+#include "mtk_cam-pool.h"
+#include "mtk_cam-raw.h"
+
+#include "mtk_cam-regs-mt8188.h"
+
+#include "mtk_camera-v4l2-controls.h"
+
+#define SENSOR_SET_DEADLINE_MS 18
+#define SENSOR_SET_RESERVED_MS 7
+#define SENSOR_SET_DEADLINE_MS_60FPS 6
+#define SENSOR_SET_RESERVED_MS_60FPS 6
+#define STATE_NUM_AT_SOF 3
+#define INITIAL_DROP_FRAME_CNT 1
+
+enum MTK_CAMSYS_STATE_RESULT {
+ STATE_RESULT_TRIGGER_CQ = 0,
+ STATE_RESULT_PASS_CQ_INIT,
+ STATE_RESULT_PASS_CQ_SW_DELAY,
+ STATE_RESULT_PASS_CQ_SCQ_DELAY,
+ STATE_RESULT_PASS_CQ_HW_DELAY,
+};
+
+void state_transition(struct mtk_camsys_ctrl_state *state_entry,
+ enum MTK_CAMSYS_STATE_IDX from,
+ enum MTK_CAMSYS_STATE_IDX to)
+{
+ if (state_entry->estate == from)
+ state_entry->estate = to;
+}
+
+static void mtk_cam_event_eos(struct mtk_raw_pipeline *pipeline)
+{
+ struct v4l2_event event = {
+ .type = V4L2_EVENT_EOS,
+ };
+ v4l2_event_queue(pipeline->subdev.devnode, &event);
+}
+
+static void mtk_cam_event_frame_sync(struct mtk_raw_pipeline *pipeline,
+ unsigned int frame_seq_no)
+{
+ struct v4l2_event event = {
+ .type = V4L2_EVENT_FRAME_SYNC,
+ .u.frame_sync.frame_sequence = frame_seq_no,
+ };
+ v4l2_event_queue(pipeline->subdev.devnode, &event);
+}
+
+static void mtk_cam_event_request_drained(struct mtk_raw_pipeline *pipeline)
+{
+ struct v4l2_event event = {
+ .type = V4L2_EVENT_REQUEST_DRAINED,
+ };
+ v4l2_event_queue(pipeline->subdev.devnode, &event);
+}
+
+static bool mtk_cam_request_drained(struct mtk_camsys_sensor_ctrl *sensor_ctrl)
+{
+ struct mtk_cam_ctx *ctx = sensor_ctrl->ctx;
+ int sensor_seq_no_next =
+ atomic_read(&sensor_ctrl->sensor_request_seq_no) + 1;
+ int res = 0;
+
+ if (sensor_seq_no_next <= atomic_read(&ctx->enqueued_frame_seq_no))
+ res = 1;
+ /* Send V4L2_EVENT_REQUEST_DRAINED event */
+ if (res == 0) {
+ mtk_cam_event_request_drained(ctx->pipe);
+ dev_dbg(ctx->cam->dev, "request_drained:(%d)\n",
+ sensor_seq_no_next);
+ }
+ return (res == 0);
+}
+
+static bool mtk_cam_req_frame_sync_start(struct mtk_cam_request *req)
+{
+ /* All ctx with sensor is in ready state */
+ struct mtk_cam_device *cam =
+ container_of(req->req.mdev, struct mtk_cam_device, media_dev);
+ struct mtk_cam_ctx *sync_ctx[MTKCAM_SUBDEV_MAX];
+ int i;
+ u32 ctx_cnt = 0;
+ bool ret = false;
+
+ /* pick out the used ctxs */
+ for (i = 0; i < cam->max_stream_num; i++) {
+ if (!(1 << i & req->ctx_used))
+ continue;
+
+ sync_ctx[ctx_cnt] = &cam->ctxs[i];
+ ctx_cnt++;
+ }
+
+ dev_dbg(cam->dev, "%s:%s:target/on/off(%d/%d/%d)\n", __func__,
+ req->req.debug_str, req->fs.target, req->fs.on_cnt,
+ req->fs.off_cnt);
+ return ret;
+}
+
+static bool mtk_cam_req_frame_sync_end(struct mtk_cam_request *req)
+{
+ /* All ctx with sensor is not in ready state */
+ struct mtk_cam_device *cam =
+ container_of(req->req.mdev, struct mtk_cam_device, media_dev);
+ bool ret = false;
+
+ mutex_lock(&req->fs.op_lock);
+ if (req->fs.target && req->fs.on_cnt) {
+ /* check fs on */
+ req->fs.off_cnt++;
+ if (req->fs.on_cnt != req->fs.target ||
+ req->fs.off_cnt != req->fs.target) {
+ /* not the last */
+ goto EXIT;
+ }
+ dev_dbg(cam->dev,
+ "%s:%s:fs_sync_frame(0): ctxs: 0x%x\n",
+ __func__, req->req.debug_str, req->ctx_used);
+ ret = true;
+ goto EXIT;
+ }
+EXIT:
+ dev_dbg(cam->dev, "%s:%s:target/on/off(%d/%d/%d)\n", __func__,
+ req->req.debug_str, req->fs.target, req->fs.on_cnt,
+ req->fs.off_cnt);
+ mutex_unlock(&req->fs.op_lock);
+ return ret;
+}
+
+static void mtk_cam_stream_on(struct mtk_raw_device *raw_dev,
+ struct mtk_cam_ctx *ctx)
+{
+ spin_lock(&ctx->streaming_lock);
+ if (ctx->streaming)
+ mtk_cam_raw_stream_on(raw_dev, 1);
+
+ spin_unlock(&ctx->streaming_lock);
+}
+
+void mtk_cam_set_sensor_full(struct mtk_cam_request_stream_data *s_data,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl)
+{
+ struct mtk_cam_device *cam;
+ struct mtk_cam_ctx *ctx;
+ struct mtk_cam_request *req;
+ struct mtk_raw_device *raw_dev;
+ unsigned int time_after_sof = 0;
+
+ /* EnQ this request's state element to state_list (STATE:READY) */
+ spin_lock(&sensor_ctrl->camsys_state_lock);
+ list_add_tail(&s_data->state.state_element,
+ &sensor_ctrl->camsys_state_list);
+ atomic_set(&sensor_ctrl->sensor_request_seq_no, s_data->frame_seq_no);
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+
+ /* sensor_worker task */
+ ctx = mtk_cam_s_data_get_ctx(s_data);
+ cam = ctx->cam;
+ req = mtk_cam_s_data_get_req(s_data);
+
+ dev_dbg(cam->dev, "%s:%s:ctx(%d) req(%d):sensor try set start\n",
+ __func__, req->req.debug_str, ctx->stream_id, s_data->frame_seq_no);
+
+ if (ctx->used_raw_num && mtk_cam_req_frame_sync_start(req))
+ dev_dbg(cam->dev,
+ "%s:%s:ctx(%d): sensor ctrl with frame sync - start\n",
+ __func__, req->req.debug_str, ctx->stream_id);
+
+ /*
+ * request setup
+ * 1st frame sensor setting is treated like normal frame
+ * and is set with other settings like max fps.
+ * 2nd is special, only exposure is set.
+ */
+ if (s_data->flags & MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_EN) {
+ v4l2_ctrl_request_setup(&req->req,
+ s_data->sensor->ctrl_handler);
+ time_after_sof =
+ div_u64(ktime_get_boottime_ns(), 1000000) -
+ ctx->sensor_ctrl.sof_time;
+ dev_dbg(cam->dev,
+ "[SOF+%dms] Sensor request:%d[ctx:%d] setup\n",
+ time_after_sof, s_data->frame_seq_no,
+ ctx->stream_id);
+ }
+
+ state_transition(&s_data->state, E_STATE_READY, E_STATE_SENSOR);
+
+ if (ctx->used_raw_num && mtk_cam_req_frame_sync_end(req))
+ dev_dbg(cam->dev,
+ "%s:ctx(%d): sensor ctrl with frame sync - stop\n",
+ __func__, ctx->stream_id);
+
+ if (ctx->used_raw_num) {
+ raw_dev = get_main_raw_dev(ctx->cam, ctx->pipe);
+ if (raw_dev && atomic_read(&raw_dev->vf_en) == 0 &&
+ ctx->sensor_ctrl.initial_cq_done == 1 &&
+ s_data->frame_seq_no == 1)
+ mtk_cam_stream_on(raw_dev, ctx);
+ }
+
+ dev_dbg(cam->dev, "%s:%s:ctx(%d)req(%d):sensor done at SOF+%dms\n",
+ __func__, req->req.debug_str, ctx->stream_id,
+ s_data->frame_seq_no, time_after_sof);
+
+ mtk_cam_complete_sensor_hdl(s_data);
+}
+
+static void mtk_cam_try_set_sensor(struct mtk_cam_ctx *ctx)
+{
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
+ struct mtk_cam_device *cam = ctx->cam;
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_camsys_ctrl_state *state_entry;
+ struct mtk_cam_request *req;
+ int sensor_seq_no_next =
+ atomic_read(&sensor_ctrl->sensor_request_seq_no) + 1;
+ int time_after_sof = div_u64(ktime_get_boottime_ns(), 1000000) -
+ ctx->sensor_ctrl.sof_time;
+ /*for 1st unsync, sensor setting will be set at enque thread*/
+ if (ctx->used_raw_num) {
+ if (ctx->pipe->feature_active == 0 && sensor_seq_no_next <= 2)
+ return;
+ } else {
+ if (sensor_seq_no_next <= 2)
+ return;
+ }
+ spin_lock(&sensor_ctrl->camsys_state_lock);
+ /* Check if previous state was without cq done */
+ list_for_each_entry(state_entry, &sensor_ctrl->camsys_state_list,
+ state_element) {
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_entry);
+ if (req_stream_data->frame_seq_no ==
+ atomic_read(&sensor_ctrl->sensor_request_seq_no)) {
+ if (state_entry->estate == E_STATE_CQ &&
+ req_stream_data->frame_seq_no > INITIAL_DROP_FRAME_CNT) {
+ state_entry->estate = E_STATE_CQ_SCQ_DELAY;
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+ dev_dbg(ctx->cam->dev,
+ "[%s] SCQ DELAY STATE at SOF+%dms\n",
+ __func__, time_after_sof);
+ return;
+ } else if (state_entry->estate == E_STATE_CAMMUX_OUTER_CFG) {
+ state_entry->estate =
+ E_STATE_CAMMUX_OUTER_CFG_DELAY;
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+
+ dev_dbg(ctx->cam->dev,
+ "[%s] CAMMUX OUTTER CFG DELAY STATE\n",
+ __func__);
+ return;
+
+ } else if (state_entry->estate <= E_STATE_SENSOR) {
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+ dev_dbg(ctx->cam->dev,
+ "[%s] wrong state:%d (sensor workqueue delay)\n",
+ __func__, state_entry->estate);
+ return;
+ }
+ } else if (req_stream_data->frame_seq_no ==
+ atomic_read(&sensor_ctrl->sensor_request_seq_no) - 1) {
+ if (state_entry->estate < E_STATE_INNER) {
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+ dev_dbg(ctx->cam->dev,
+ "[%s] req:%d isn't arrive inner at SOF+%dms\n",
+ __func__, req_stream_data->frame_seq_no,
+ time_after_sof);
+ return;
+ }
+ }
+ }
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+
+ /** Frame sync:
+ * make sure the all ctxs of previous request are triggered
+ */
+ req_stream_data = mtk_cam_get_req_s_data(ctx, ctx->stream_id,
+ sensor_seq_no_next - 1);
+ if (req_stream_data) {
+ req = mtk_cam_s_data_get_req(req_stream_data);
+ /* fs complete: fs.target <= off and on == off */
+ if (!(req->fs.target <= req->fs.off_cnt &&
+ req->fs.off_cnt == req->fs.on_cnt)) {
+ dev_info(ctx->cam->dev,
+ "[TimerIRQ] ctx:%d the fs of req(%s/%d) is not completed, target/on/off(%d/%d/%d)\n",
+ ctx->stream_id, req->req.debug_str,
+ sensor_seq_no_next - 1, req->fs.target,
+ req->fs.on_cnt, req->fs.off_cnt);
+ return;
+ }
+ }
+
+ req_stream_data =
+ mtk_cam_get_req_s_data(ctx, ctx->stream_id, sensor_seq_no_next);
+ if (req_stream_data) {
+ req = mtk_cam_s_data_get_req(req_stream_data);
+ mtk_cam_set_sensor_full(req_stream_data, &ctx->sensor_ctrl);
+
+ dev_dbg(cam->dev,
+ "%s:[TimerIRQ [SOF+%dms]:] ctx:%d, sensor_req_seq_no:%d\n",
+ __func__, time_after_sof, ctx->stream_id,
+ sensor_seq_no_next);
+ } else {
+ dev_dbg(cam->dev,
+ "%s:[TimerIRQ [SOF+%dms]] ctx:%d, empty req_queue, sensor_req_seq_no:%d\n",
+ __func__, time_after_sof, ctx->stream_id,
+ atomic_read(&sensor_ctrl->sensor_request_seq_no));
+ }
+}
+
+static void mtk_cam_sensor_worker_in_sensorctrl(struct kthread_work *work)
+{
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl =
+ container_of(work, struct mtk_camsys_sensor_ctrl, work);
+ struct mtk_cam_ctx *ctx;
+
+ ctx = sensor_ctrl->ctx;
+ mtk_cam_try_set_sensor(ctx);
+}
+
+bool mtk_cam_submit_kwork_in_sensorctrl(struct kthread_worker *worker,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl)
+{
+ if (!worker) {
+ pr_info("%s: not queue work since kthread_worker is null\n",
+ __func__);
+
+ return false;
+ }
+
+ return kthread_queue_work(worker, &sensor_ctrl->work);
+}
+
+static enum hrtimer_restart sensor_set_handler(struct hrtimer *t)
+{
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl =
+ container_of(t, struct mtk_camsys_sensor_ctrl,
+ sensor_deadline_timer);
+
+ mtk_cam_submit_kwork_in_sensorctrl(sensor_ctrl->sensorsetting_wq,
+ sensor_ctrl);
+
+ return HRTIMER_NORESTART;
+}
+
+static enum hrtimer_restart sensor_deadline_timer_handler(struct hrtimer *t)
+{
+ unsigned int enq_no, sen_no;
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl =
+ container_of(t, struct mtk_camsys_sensor_ctrl,
+ sensor_deadline_timer);
+ struct mtk_cam_ctx *ctx = sensor_ctrl->ctx;
+ struct mtk_cam_device *cam = ctx->cam;
+ ktime_t m_kt;
+ int time_after_sof = div_u64(ktime_get_boottime_ns(), 1000000) -
+ sensor_ctrl->sof_time;
+ bool drained_res = false;
+
+ sensor_ctrl->sensor_deadline_timer.function = sensor_set_handler;
+
+ m_kt = ktime_set(0, sensor_ctrl->timer_req_sensor * 1000000);
+
+ if (ctx->used_raw_num) {
+ /* handle V4L2_EVENT_REQUEST_DRAINED event */
+ drained_res = mtk_cam_request_drained(sensor_ctrl);
+ }
+ dev_dbg(cam->dev,
+ "[TimerIRQ [SOF+%dms]] ctx:%d, sensor_req_seq_no:%d\n",
+ time_after_sof, ctx->stream_id,
+ atomic_read(&sensor_ctrl->sensor_request_seq_no));
+ if (drained_res == 0) {
+ sen_no = atomic_read(&sensor_ctrl->sensor_enq_seq_no);
+ enq_no = atomic_read(&ctx->enqueued_frame_seq_no);
+ if (enq_no == sen_no) {
+ mtk_cam_submit_kwork_in_sensorctrl(sensor_ctrl->sensorsetting_wq,
+ sensor_ctrl);
+ return HRTIMER_NORESTART;
+ }
+ dev_dbg(cam->dev,
+ "[TimerIRQ [SOF+%dms]] ctx:%d, enq:%d/sensor_enq:%d\n",
+ time_after_sof, ctx->stream_id, enq_no, sen_no);
+ }
+ /*using enque timing for sensor setting*/
+ if (ctx->used_raw_num) {
+ if (ctx->pipe->feature_active == 0) {
+ int drained_seq_no =
+ atomic_read(&sensor_ctrl->sensor_request_seq_no) + 1;
+ atomic_set(&sensor_ctrl->last_drained_seq_no, drained_seq_no);
+ return HRTIMER_NORESTART;
+ }
+ }
+ hrtimer_forward_now(&sensor_ctrl->sensor_deadline_timer, m_kt);
+
+ return HRTIMER_RESTART;
+}
+
+static void mtk_cam_sof_timer_setup(struct mtk_cam_ctx *ctx)
+{
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
+ ktime_t m_kt;
+ int after_sof_ms = div_u64(ktime_get_boottime_ns(), 1000000) -
+ sensor_ctrl->sof_time;
+
+ sensor_ctrl->sensor_deadline_timer.function =
+ sensor_deadline_timer_handler;
+ sensor_ctrl->ctx = ctx;
+ if (after_sof_ms < 0)
+ after_sof_ms = 0;
+ else if (after_sof_ms > sensor_ctrl->timer_req_event)
+ after_sof_ms = sensor_ctrl->timer_req_event - 1;
+ m_kt = ktime_set(0, sensor_ctrl->timer_req_event * 1000000 -
+ after_sof_ms * 1000000);
+ hrtimer_start(&sensor_ctrl->sensor_deadline_timer, m_kt, HRTIMER_MODE_REL);
+}
+
+static void
+mtk_cam_set_timestamp(struct mtk_cam_request_stream_data *stream_data,
+ u64 time_boot,
+ u64 time_mono)
+{
+ stream_data->timestamp = time_boot;
+ stream_data->timestamp_mono = time_mono;
+}
+
+static int
+mtk_camsys_raw_state_handle(struct mtk_raw_device *raw_dev,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl,
+ struct mtk_camsys_ctrl_state **current_state,
+ struct mtk_camsys_irq_info *irq_info)
+{
+ struct mtk_cam_ctx *ctx = sensor_ctrl->ctx;
+ struct mtk_camsys_ctrl_state *state_temp, *state_outer = NULL;
+ struct mtk_camsys_ctrl_state *state_inner = NULL;
+ struct mtk_camsys_ctrl_state *state_rec[STATE_NUM_AT_SOF];
+ struct mtk_cam_request_stream_data *req_stream_data;
+ int frame_idx_inner = irq_info->frame_idx_inner;
+ int stateidx;
+ int que_cnt = 0;
+ int write_cnt;
+ u64 time_boot = ktime_get_boottime_ns();
+ u64 time_mono = ktime_get_ns();
+ int working_req_found = 0;
+
+ /* List state-queue status*/
+ spin_lock(&sensor_ctrl->camsys_state_lock);
+ list_for_each_entry(state_temp, &sensor_ctrl->camsys_state_list,
+ state_element) {
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_temp);
+ stateidx = atomic_read(&sensor_ctrl->sensor_request_seq_no) -
+ req_stream_data->frame_seq_no;
+ if (stateidx < STATE_NUM_AT_SOF && stateidx > -1) {
+ state_rec[stateidx] = state_temp;
+ if (stateidx == 0)
+ working_req_found = 1;
+
+ /* Find outer state element */
+ if (state_temp->estate == E_STATE_OUTER ||
+ state_temp->estate == E_STATE_CAMMUX_OUTER ||
+ state_temp->estate == E_STATE_OUTER_HW_DELAY) {
+ state_outer = state_temp;
+ mtk_cam_set_timestamp(req_stream_data,
+ time_boot, time_mono);
+ }
+
+ /* Find inner state element request */
+ if (state_temp->estate == E_STATE_INNER ||
+ state_temp->estate == E_STATE_INNER_HW_DELAY)
+ state_inner = state_temp;
+
+ dev_dbg(raw_dev->dev,
+ "[SOF] STATE_CHECK [N-%d] Req:%d / State:%d\n",
+ stateidx, req_stream_data->frame_seq_no,
+ state_rec[stateidx]->estate);
+ }
+ /* counter for state queue*/
+ que_cnt++;
+ }
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+
+ /* HW imcomplete case */
+ if (state_inner) {
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_inner);
+ write_cnt = (atomic_read(&sensor_ctrl->isp_request_seq_no) / 256)
+ * 256 + irq_info->write_cnt;
+
+ if (frame_idx_inner > atomic_read(&sensor_ctrl->isp_request_seq_no) ||
+ atomic_read(&req_stream_data->frame_done_work.is_queued) == 1) {
+ dev_dbg_ratelimited(raw_dev->dev,
+ "[SOF] frame done work too late frames. req(%d),ts(%llu)\n",
+ req_stream_data->frame_seq_no,
+ irq_info->ts_ns);
+ } else if (write_cnt >= req_stream_data->frame_seq_no) {
+ dev_info_ratelimited(raw_dev->dev,
+ "[SOF] frame done reading lost %d frames. req(%d),ts(%llu)\n",
+ write_cnt - req_stream_data->frame_seq_no + 1,
+ req_stream_data->frame_seq_no,
+ irq_info->ts_ns);
+ mtk_cam_set_timestamp(req_stream_data,
+ time_boot - 1000, time_mono - 1000);
+ mtk_camsys_frame_done(ctx, write_cnt, ctx->stream_id);
+ } else if ((write_cnt >= req_stream_data->frame_seq_no - 1) &&
+ irq_info->fbc_cnt == 0) {
+ dev_info_ratelimited(raw_dev->dev,
+ "[SOF] frame done reading lost frames. req(%d),ts(%llu)\n",
+ req_stream_data->frame_seq_no, irq_info->ts_ns);
+ mtk_cam_set_timestamp(req_stream_data,
+ time_boot - 1000, time_mono - 1000);
+ mtk_camsys_frame_done(ctx, write_cnt + 1, ctx->stream_id);
+ } else {
+ state_transition(state_inner,
+ E_STATE_INNER, E_STATE_INNER_HW_DELAY);
+ if (state_outer) {
+ state_transition(state_outer,
+ E_STATE_OUTER,
+ E_STATE_OUTER_HW_DELAY);
+ state_transition(state_outer,
+ E_STATE_CAMMUX_OUTER,
+ E_STATE_OUTER_HW_DELAY);
+ }
+ dev_info_ratelimited(raw_dev->dev,
+ "[SOF] HW_IMCOMPLETE state cnt(%d,%d),req(%d),ts(%llu)\n",
+ write_cnt, irq_info->write_cnt,
+ req_stream_data->frame_seq_no,
+ irq_info->ts_ns);
+ return STATE_RESULT_PASS_CQ_HW_DELAY;
+ }
+ }
+
+ /* Transit outer state to inner state */
+ if (state_outer && sensor_ctrl->sensorsetting_wq) {
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_outer);
+ if (atomic_read(&sensor_ctrl->initial_drop_frame_cnt) == 0 &&
+ req_stream_data->frame_seq_no > frame_idx_inner) {
+ dev_info(raw_dev->dev,
+ "[SOF-noDBLOAD] HW delay outer_no:%d, inner_idx:%d <= processing_idx:%d,ts:%llu\n",
+ req_stream_data->frame_seq_no, frame_idx_inner,
+ atomic_read(&sensor_ctrl->isp_request_seq_no),
+ irq_info->ts_ns);
+ return STATE_RESULT_PASS_CQ_HW_DELAY;
+ }
+
+ if (atomic_read(&sensor_ctrl->initial_drop_frame_cnt) == 0 &&
+ req_stream_data->frame_seq_no == frame_idx_inner) {
+ if (frame_idx_inner >
+ atomic_read(&sensor_ctrl->isp_request_seq_no)) {
+ state_transition(state_outer,
+ E_STATE_OUTER_HW_DELAY,
+ E_STATE_INNER_HW_DELAY);
+ state_transition(state_outer,
+ E_STATE_OUTER,
+ E_STATE_INNER);
+ state_transition(state_outer,
+ E_STATE_CAMMUX_OUTER,
+ E_STATE_INNER);
+ atomic_set(&sensor_ctrl->isp_request_seq_no, frame_idx_inner);
+ dev_dbg(raw_dev->dev,
+ "[SOF-DBLOAD] frame_seq_no:%d, OUTER->INNER state:%d,ts:%llu\n",
+ req_stream_data->frame_seq_no,
+ state_outer->estate, irq_info->ts_ns);
+ }
+ }
+ }
+
+ /* Trigger high resolution timer to try sensor setting */
+ sensor_ctrl->sof_time = div_u64(irq_info->ts_ns, 1000000);
+ mtk_cam_sof_timer_setup(ctx);
+
+ if (que_cnt > 1 && state_rec[1]) {
+ state_temp = state_rec[1];
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_temp);
+ if (req_stream_data->frame_seq_no == 1)
+ state_transition(state_temp,
+ E_STATE_SENSOR, E_STATE_INNER);
+ }
+
+ if (que_cnt > 0) {
+ if (working_req_found && state_rec[0]) {
+ if (state_rec[0]->estate == E_STATE_READY) {
+ dev_info(raw_dev->dev, "[SOF] sensor delay ts:%llu\n",
+ irq_info->ts_ns);
+ req_stream_data =
+ mtk_cam_ctrl_state_to_req_s_data(state_rec[0]);
+ req_stream_data->flags |=
+ MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_DELAYED;
+ }
+
+ if (state_rec[0]->estate == E_STATE_SENINF)
+ dev_info(raw_dev->dev, "[SOF] sensor switch delay\n");
+
+ /* CQ triggering judgment*/
+ if (state_rec[0]->estate == E_STATE_SENSOR) {
+ *current_state = state_rec[0];
+ return STATE_RESULT_TRIGGER_CQ;
+ }
+ /* last SCQ triggering delay judgment*/
+ if (state_rec[0]->estate == E_STATE_CQ_SCQ_DELAY) {
+ state_transition(state_rec[0],
+ E_STATE_CQ_SCQ_DELAY,
+ E_STATE_OUTER);
+ dev_info(raw_dev->dev, "[SOF] SCQ_DELAY state:%d ts:%llu\n",
+ state_rec[0]->estate, irq_info->ts_ns);
+ return STATE_RESULT_PASS_CQ_SCQ_DELAY;
+ }
+ } else {
+ dev_dbg(raw_dev->dev, "[SOF] working request not found\n");
+ }
+ }
+ return STATE_RESULT_PASS_CQ_SW_DELAY;
+}
+
+static void mtk_camsys_raw_frame_start(struct mtk_raw_device *raw_dev,
+ struct mtk_cam_ctx *ctx,
+ struct mtk_camsys_irq_info *irq_info)
+{
+ unsigned int dequeued_frame_seq_no = irq_info->frame_idx_inner;
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
+ struct mtk_cam_working_buf_entry *buf_entry;
+ struct mtk_camsys_ctrl_state *current_state;
+ struct mtk_cam_buffer *buf;
+
+ dma_addr_t base_addr;
+ enum MTK_CAMSYS_STATE_RESULT state_handle_ret;
+
+ /* touch watchdog*/
+ if (watchdog_scenario(ctx))
+ mtk_ctx_watchdog_kick(ctx);
+ /* inner register dequeue number */
+ ctx->dequeued_frame_seq_no = dequeued_frame_seq_no;
+ /* Send V4L2_EVENT_FRAME_SYNC event */
+ mtk_cam_event_frame_sync(ctx->pipe, dequeued_frame_seq_no);
+
+ /* Handle directly enque buffers */
+ spin_lock(&ctx->cam->dma_processing_lock);
+ list_for_each_entry(buf, &ctx->cam->dma_processing, list) {
+ if (buf->state.estate == E_BUF_STATE_COMPOSED) {
+ spin_unlock(&ctx->cam->dma_processing_lock);
+ goto apply_cq;
+ }
+ }
+ spin_unlock(&ctx->cam->dma_processing_lock);
+ buf = NULL;
+ current_state = NULL;
+
+ /* Find request of this dequeued frame */
+ req_stream_data =
+ mtk_cam_get_req_s_data(ctx, ctx->stream_id, dequeued_frame_seq_no);
+
+ if (ctx->sensor) {
+ state_handle_ret =
+ mtk_camsys_raw_state_handle(raw_dev, sensor_ctrl,
+ ¤t_state, irq_info);
+
+ if (state_handle_ret != STATE_RESULT_TRIGGER_CQ) {
+ dev_dbg(raw_dev->dev, "[SOF] CQ drop s:%d deq:%d\n",
+ state_handle_ret, dequeued_frame_seq_no);
+ return;
+ }
+ }
+
+apply_cq:
+ /* Update CQ base address if needed */
+ if (ctx->composed_frame_seq_no <= dequeued_frame_seq_no) {
+ dev_info_ratelimited(raw_dev->dev,
+ "SOF[ctx:%d-#%d], CQ isn't updated [composed_frame_deq (%d) ts:%llu]\n",
+ ctx->stream_id, dequeued_frame_seq_no,
+ ctx->composed_frame_seq_no, irq_info->ts_ns);
+ return;
+ }
+ /* apply next composed buffer */
+ spin_lock(&ctx->composed_buffer_list.lock);
+ if (list_empty(&ctx->composed_buffer_list.list)) {
+ dev_info_ratelimited(raw_dev->dev,
+ "SOF_INT_ST, no buffer update, cq_num:%d, frame_seq:%d, ts:%llu\n",
+ ctx->composed_frame_seq_no, dequeued_frame_seq_no,
+ irq_info->ts_ns);
+ spin_unlock(&ctx->composed_buffer_list.lock);
+ } else {
+ buf_entry = list_first_entry(&ctx->composed_buffer_list.list,
+ struct mtk_cam_working_buf_entry,
+ list_entry);
+ list_del(&buf_entry->list_entry);
+ ctx->composed_buffer_list.cnt--;
+ spin_unlock(&ctx->composed_buffer_list.lock);
+ spin_lock(&ctx->processing_buffer_list.lock);
+ list_add_tail(&buf_entry->list_entry,
+ &ctx->processing_buffer_list.list);
+ ctx->processing_buffer_list.cnt++;
+ spin_unlock(&ctx->processing_buffer_list.lock);
+ base_addr = buf_entry->buffer.iova;
+ mtk_cam_raw_apply_cq(raw_dev, base_addr,
+ buf_entry->cq_desc_size,
+ buf_entry->cq_desc_offset,
+ buf_entry->sub_cq_desc_size,
+ buf_entry->sub_cq_desc_offset);
+
+ if (buf) {
+ buf->state.estate = E_BUF_STATE_CQ;
+ return;
+ }
+
+ /* Transit state from Sensor -> CQ */
+ if (ctx->sensor) {
+ /* req_stream_data of req_cq*/
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(current_state);
+ state_transition(current_state,
+ E_STATE_SENSOR, E_STATE_CQ);
+
+ dev_dbg(raw_dev->dev,
+ "SOF[ctx:%d-#%d], CQ-%d is update, composed:%d, cq_addr:0x%pad, time:%lld, monotime:%lld\n",
+ ctx->stream_id, dequeued_frame_seq_no,
+ req_stream_data->frame_seq_no,
+ ctx->composed_frame_seq_no, &base_addr,
+ req_stream_data->timestamp,
+ req_stream_data->timestamp_mono);
+ }
+ }
+}
+
+static void mtk_camsys_raw_cq_done(struct mtk_raw_device *raw_dev,
+ struct mtk_cam_ctx *ctx,
+ unsigned int frame_seq_no_outer)
+{
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
+ struct mtk_camsys_ctrl_state *state_entry;
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_cam_buffer *buf;
+
+ /* initial CQ done */
+ if (raw_dev->sof_count == 0) {
+ sensor_ctrl->initial_cq_done = 1;
+ spin_lock(&ctx->cam->dma_processing_lock);
+ if (!list_empty(&ctx->cam->dma_processing)) {
+ buf = list_first_entry(&ctx->cam->dma_processing,
+ struct mtk_cam_buffer,
+ list);
+ if (buf->state.estate == E_BUF_STATE_CQ)
+ buf->state.estate = E_BUF_STATE_OUTER;
+ spin_unlock(&ctx->cam->dma_processing_lock);
+ mtk_cam_stream_on(raw_dev, ctx);
+ return;
+ }
+ spin_unlock(&ctx->cam->dma_processing_lock);
+ req_stream_data = mtk_cam_get_req_s_data(ctx, ctx->stream_id, 1);
+ if (!req_stream_data) {
+ dev_err(raw_dev->dev, "Cannot find req stream data with stream_id:%d\n",
+ ctx->stream_id);
+ return;
+ }
+ if (req_stream_data->state.estate >= E_STATE_SENSOR || !ctx->sensor) {
+ mtk_cam_stream_on(raw_dev, ctx);
+ } else {
+ dev_dbg(raw_dev->dev,
+ "[CQD] 1st sensor not set yet, req:%d, state:%d\n",
+ req_stream_data->frame_seq_no, req_stream_data->state.estate);
+ }
+ }
+ /* Legacy CQ done will be always happened at frame done */
+ spin_lock(&sensor_ctrl->camsys_state_lock);
+ list_for_each_entry(state_entry, &sensor_ctrl->camsys_state_list,
+ state_element) {
+ req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_entry);
+ if (req_stream_data->frame_seq_no == frame_seq_no_outer) {
+ if (frame_seq_no_outer > atomic_read(&sensor_ctrl->isp_request_seq_no)) {
+ /*
+ * outer number is 1 more from last SOF's
+ * inner number
+ */
+ if (frame_seq_no_outer == 1)
+ state_entry->estate = E_STATE_OUTER;
+ state_transition(state_entry, E_STATE_CQ,
+ E_STATE_OUTER);
+ state_transition(state_entry, E_STATE_CQ_SCQ_DELAY,
+ E_STATE_OUTER);
+ state_transition(state_entry, E_STATE_SENINF,
+ E_STATE_OUTER);
+
+ dev_dbg(raw_dev->dev,
+ "[CQD] req:%d, CQ->OUTER state:%d\n",
+ req_stream_data->frame_seq_no,
+ state_entry->estate);
+ }
+ }
+ }
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+}
+
+static bool
+mtk_camsys_raw_prepare_frame_done(struct mtk_raw_device *raw_dev,
+ struct mtk_cam_ctx *ctx,
+ unsigned int dequeued_frame_seq_no)
+{
+ struct mtk_cam_device *cam = ctx->cam;
+ struct mtk_camsys_sensor_ctrl *camsys_sensor_ctrl = &ctx->sensor_ctrl;
+ struct mtk_camsys_ctrl_state *state_entry;
+ struct mtk_cam_request *state_req;
+ struct mtk_cam_request_stream_data *s_data;
+
+ if (!ctx->sensor) {
+ dev_info(cam->dev, "%s: no sensor found in ctx:%d, req:%d",
+ __func__, ctx->stream_id, dequeued_frame_seq_no);
+
+ return true;
+ }
+
+ spin_lock(&camsys_sensor_ctrl->camsys_state_lock);
+ /**
+ * Find inner register number's request and transit to
+ * STATE_DONE_xxx
+ */
+ list_for_each_entry(state_entry, &camsys_sensor_ctrl->camsys_state_list,
+ state_element) {
+ state_req = mtk_cam_ctrl_state_get_req(state_entry);
+ s_data = mtk_cam_ctrl_state_to_req_s_data(state_entry);
+ if (s_data->frame_seq_no == dequeued_frame_seq_no) {
+ state_transition(state_entry,
+ E_STATE_INNER_HW_DELAY, E_STATE_DONE_MISMATCH);
+ state_transition(state_entry,
+ E_STATE_INNER, E_STATE_DONE_NORMAL);
+ if (atomic_read(&camsys_sensor_ctrl->isp_request_seq_no) == 0)
+ state_transition(state_entry,
+ E_STATE_CQ, E_STATE_OUTER);
+
+ dev_dbg(cam->dev,
+ "[SWD] req:%d/state:%d/time:%lld/sync_id:%lld\n",
+ s_data->frame_seq_no, state_entry->estate,
+ s_data->timestamp, state_req->sync_id);
+ }
+ }
+ spin_unlock(&camsys_sensor_ctrl->camsys_state_lock);
+
+ return true;
+}
+
+static void mtk_cam_handle_frame_done(struct mtk_cam_ctx *ctx,
+ unsigned int frame_seq_no,
+ unsigned int pipe_id)
+{
+ struct mtk_raw_device *raw_dev;
+ bool need_dequeue;
+
+ /**
+ * If ctx is already off, just return; mtk_cam_dev_req_cleanup()
+ * triggered by mtk_cam_vb2_stop_streaming() puts the all media
+ * requests back.
+ */
+ spin_lock(&ctx->streaming_lock);
+ if (!ctx->streaming) {
+ dev_info(ctx->cam->dev,
+ "%s: skip frame done for stream off ctx:%d\n",
+ __func__, ctx->stream_id);
+ spin_unlock(&ctx->streaming_lock);
+ return;
+ }
+ spin_unlock(&ctx->streaming_lock);
+ raw_dev = get_main_raw_dev(ctx->cam, ctx->pipe);
+ if (!raw_dev) {
+ dev_err(ctx->cam->dev, "%s: not found raw device\n", __func__);
+ return;
+ }
+
+ need_dequeue =
+ mtk_camsys_raw_prepare_frame_done(raw_dev, ctx, frame_seq_no);
+
+ if (!need_dequeue)
+ return;
+
+ dev_dbg(ctx->cam->dev, "[%s] job done ctx-%d:pipe-%d:req(%d)\n",
+ __func__, ctx->stream_id, pipe_id, frame_seq_no);
+ if (mtk_cam_dequeue_req_frame(ctx, frame_seq_no, pipe_id)) {
+ mutex_lock(&ctx->cam->queue_lock);
+ mtk_cam_dev_req_try_queue(ctx->cam);
+ mutex_unlock(&ctx->cam->queue_lock);
+ }
+}
+
+void mtk_cam_meta1_done_work(struct work_struct *work)
+{
+ struct mtk_cam_req_work *meta1_done_work =
+ (struct mtk_cam_req_work *)work;
+ struct mtk_cam_request_stream_data *s_data, *s_data_ctx;
+ struct mtk_cam_ctx *ctx;
+ struct mtk_cam_request *req;
+ struct mtk_cam_buffer *buf;
+ struct vb2_buffer *vb;
+ void *vaddr;
+ bool unreliable = false;
+
+ s_data = mtk_cam_req_work_get_s_data(meta1_done_work);
+ ctx = mtk_cam_s_data_get_ctx(s_data);
+ req = mtk_cam_s_data_get_req(s_data);
+ s_data_ctx = mtk_cam_req_get_s_data(req, ctx->stream_id, 0);
+
+ dev_dbg(ctx->cam->dev, "%s: ctx:%d\n", __func__, ctx->stream_id);
+
+ spin_lock(&ctx->streaming_lock);
+ if (!ctx->streaming) {
+ spin_unlock(&ctx->streaming_lock);
+ dev_info(ctx->cam->dev, "%s: skip for stream off ctx:%d\n",
+ __func__, ctx->stream_id);
+ return;
+ }
+ spin_unlock(&ctx->streaming_lock);
+
+ if (!s_data) {
+ dev_info(ctx->cam->dev,
+ "%s:ctx(%d): can't get s_data\n",
+ __func__, ctx->stream_id);
+ return;
+ }
+
+ /* Copy the meta1 output content to user buffer */
+ buf = mtk_cam_s_data_get_vbuf(s_data, MTK_RAW_META_OUT_1);
+ if (!buf) {
+ dev_info(ctx->cam->dev,
+ "ctx(%d): can't get MTK_RAW_META_OUT_1 buf from req(%d)\n",
+ ctx->stream_id, s_data->frame_seq_no);
+ return;
+ }
+
+ vb = &buf->vbb.vb2_buf;
+ if (!vb) {
+ dev_info(ctx->cam->dev,
+ "%s:ctx(%d): can't get vb2 buf\n",
+ __func__, ctx->stream_id);
+ return;
+ }
+
+ vaddr = vb2_plane_vaddr(&buf->vbb.vb2_buf, 0);
+ if (!vaddr) {
+ dev_info(ctx->cam->dev,
+ "%s:ctx(%d): can't get plane_vadd\n",
+ __func__, ctx->stream_id);
+ return;
+ }
+
+ /* Update the timestamp for the buffer*/
+ mtk_cam_s_data_update_timestamp(buf, s_data_ctx);
+
+ /* clean the stream data for req reinit case */
+ mtk_cam_s_data_reset_vbuf(s_data, MTK_RAW_META_OUT_1);
+
+ /* Let user get the buffer */
+ if (unreliable)
+ vb2_buffer_done(&buf->vbb.vb2_buf, VB2_BUF_STATE_ERROR);
+ else
+ vb2_buffer_done(&buf->vbb.vb2_buf, VB2_BUF_STATE_DONE);
+
+ dev_dbg(ctx->cam->dev, "%s:%s: req(%d) done\n",
+ __func__, req->req.debug_str, s_data->frame_seq_no);
+}
+
+static void mtk_cam_meta1_done(struct mtk_cam_ctx *ctx,
+ unsigned int frame_seq_no,
+ unsigned int pipe_id)
+{
+ struct mtk_cam_request *req;
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_cam_req_work *meta1_done_work;
+
+ req = mtk_cam_get_req(ctx, frame_seq_no);
+ if (!req) {
+ dev_info(ctx->cam->dev, "%s:ctx-%d:pipe-%d:req(%d) not found!\n",
+ __func__, ctx->stream_id, pipe_id, frame_seq_no);
+ return;
+ }
+
+ req_stream_data = mtk_cam_req_get_s_data(req, pipe_id, 0);
+ if (!req_stream_data) {
+ dev_info(ctx->cam->dev, "%s:ctx-%d:pipe-%d:s_data not found!\n",
+ __func__, ctx->stream_id, pipe_id);
+ return;
+ }
+
+ if (!(req_stream_data->flags & MTK_CAM_REQ_S_DATA_FLAG_META1_INDEPENDENT))
+ return;
+
+ meta1_done_work = &req_stream_data->meta1_done_work;
+ atomic_set(&meta1_done_work->is_queued, 1);
+ queue_work(ctx->frame_done_wq, &meta1_done_work->work);
+}
+
+void mtk_cam_frame_done_work(struct work_struct *work)
+{
+ struct mtk_cam_req_work *frame_done_work =
+ (struct mtk_cam_req_work *)work;
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_cam_ctx *ctx;
+
+ req_stream_data = mtk_cam_req_work_get_s_data(frame_done_work);
+ ctx = mtk_cam_s_data_get_ctx(req_stream_data);
+
+ mtk_cam_handle_frame_done(ctx,
+ req_stream_data->frame_seq_no,
+ req_stream_data->pipe_id);
+}
+
+void mtk_camsys_frame_done(struct mtk_cam_ctx *ctx,
+ unsigned int frame_seq_no,
+ unsigned int pipe_id)
+{
+ struct mtk_cam_req_work *frame_done_work;
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_cam_buffer *buf;
+ struct mtk_cam_working_buf_entry *buf_entry = NULL;
+ bool is_pending_buffer = false;
+
+ spin_lock(&ctx->cam->dma_processing_lock);
+ if (!list_empty(&ctx->cam->dma_processing)) {
+ buf = list_first_entry(&ctx->cam->dma_processing,
+ struct mtk_cam_buffer, list);
+ list_del(&buf->list);
+ ctx->cam->dma_processing_count--;
+ vb2_buffer_done(&buf->vbb.vb2_buf, VB2_BUF_STATE_DONE);
+ is_pending_buffer = true;
+ }
+ spin_unlock(&ctx->cam->dma_processing_lock);
+ if (is_pending_buffer) {
+ spin_lock(&ctx->processing_buffer_list.lock);
+ if (!list_empty(&ctx->processing_buffer_list.list)) {
+ buf_entry =
+ list_first_entry(&ctx->processing_buffer_list.list,
+ struct mtk_cam_working_buf_entry,
+ list_entry);
+ list_del(&buf_entry->list_entry);
+ ctx->processing_buffer_list.cnt--;
+ }
+ spin_unlock(&ctx->processing_buffer_list.lock);
+ if (buf_entry)
+ mtk_cam_working_buf_put(buf_entry);
+
+ mtk_cam_buf_try_queue(ctx);
+ return;
+ }
+
+ req_stream_data = mtk_cam_get_req_s_data(ctx, pipe_id, frame_seq_no);
+ if (!req_stream_data) {
+ dev_err(ctx->cam->dev,
+ "%s:ctx-%d:pipe-%d:req(%d) not found!\n",
+ __func__, ctx->stream_id, pipe_id, frame_seq_no);
+ return;
+ }
+
+ if (atomic_read(&req_stream_data->frame_done_work.is_queued)) {
+ dev_info(ctx->cam->dev,
+ "already queue done work req:%d seq:%d pipe_id:%d\n",
+ req_stream_data->frame_seq_no, frame_seq_no, pipe_id);
+ return;
+ }
+
+ atomic_set(&req_stream_data->frame_done_work.is_queued, 1);
+ frame_done_work = &req_stream_data->frame_done_work;
+ queue_work(ctx->frame_done_wq, &frame_done_work->work);
+}
+
+void mtk_camsys_state_delete(struct mtk_cam_ctx *ctx,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl,
+ struct mtk_cam_request *req)
+{
+ struct mtk_camsys_ctrl_state *state_entry, *state_entry_prev;
+ struct mtk_cam_request_stream_data *s_data;
+ struct mtk_camsys_ctrl_state *req_state;
+ int state_found = 0;
+
+ if (ctx->sensor) {
+ spin_lock(&sensor_ctrl->camsys_state_lock);
+ list_for_each_entry_safe(state_entry, state_entry_prev,
+ &sensor_ctrl->camsys_state_list,
+ state_element) {
+ s_data = mtk_cam_req_get_s_data(req, ctx->stream_id, 0);
+ req_state = &s_data->state;
+
+ if (state_entry == req_state) {
+ list_del(&state_entry->state_element);
+ state_found = 1;
+ }
+ }
+ spin_unlock(&sensor_ctrl->camsys_state_lock);
+ if (state_found == 0)
+ dev_dbg(ctx->cam->dev, "state not found\n");
+ }
+}
+
+static bool mtk_camsys_is_all_cq_done(struct mtk_cam_ctx *ctx,
+ unsigned int pipe_id)
+{
+ unsigned int all_subdevs = 0;
+ bool ret = false;
+
+ spin_lock(&ctx->first_cq_lock);
+ if (ctx->is_first_cq_done) {
+ ret = true;
+ spin_unlock(&ctx->first_cq_lock);
+ goto EXIT;
+ }
+
+ // update cq done status
+ ctx->cq_done_status |= (1 << pipe_id);
+
+ // check cq done status
+ if (ctx->used_raw_num)
+ all_subdevs |= (1 << ctx->pipe->id);
+ if ((ctx->cq_done_status & all_subdevs) == all_subdevs) {
+ ctx->is_first_cq_done = 1;
+ ret = true;
+ }
+ spin_unlock(&ctx->first_cq_lock);
+ dev_info(ctx->cam->dev,
+ "[1st-CQD] all done:%d, pipe_id:%d (using raw:%d)\n",
+ ctx->is_first_cq_done, pipe_id, ctx->used_raw_num);
+EXIT:
+ return ret;
+}
+
+static int mtk_camsys_event_handle_raw(struct mtk_cam_device *cam,
+ unsigned int engine_id,
+ struct mtk_camsys_irq_info *irq_info)
+{
+ struct mtk_raw_device *raw_dev;
+ struct mtk_cam_ctx *ctx;
+
+ raw_dev = dev_get_drvdata(cam->raw.devs[engine_id]);
+ ctx = mtk_cam_find_ctx(cam, &raw_dev->pipeline->subdev.entity);
+ if (!ctx) {
+ dev_err(raw_dev->dev, "cannot find ctx\n");
+ return -EINVAL;
+ }
+
+ /* raw's CQ done */
+ if (irq_info->irq_type & 1 << CAMSYS_IRQ_SETTING_DONE) {
+ if (mtk_camsys_is_all_cq_done(ctx, ctx->pipe->id))
+ mtk_camsys_raw_cq_done(raw_dev, ctx,
+ irq_info->frame_idx);
+ }
+
+ /* raw's DMA done, we only allow AFO done here */
+ if (irq_info->irq_type & 1 << CAMSYS_IRQ_AFO_DONE)
+ mtk_cam_meta1_done(ctx, ctx->dequeued_frame_seq_no, ctx->stream_id);
+
+ /* raw's SW done */
+ if (irq_info->irq_type & 1 << CAMSYS_IRQ_FRAME_DONE) {
+ mtk_camsys_frame_done(ctx, ctx->dequeued_frame_seq_no,
+ ctx->stream_id);
+ }
+ /* raw's SOF */
+ if (irq_info->irq_type & 1 << CAMSYS_IRQ_FRAME_START) {
+ if (atomic_read(&raw_dev->vf_en) == 0) {
+ dev_info(raw_dev->dev, "skip sof event when vf off\n");
+ return 0;
+ }
+ mtk_camsys_raw_frame_start(raw_dev, ctx, irq_info);
+ }
+
+ return 0;
+}
+
+int mtk_camsys_isr_event(struct mtk_cam_device *cam,
+ unsigned int engine_id,
+ struct mtk_camsys_irq_info *irq_info)
+{
+ int ret = 0;
+
+ ret = mtk_camsys_event_handle_raw(cam, engine_id, irq_info);
+
+ return ret;
+}
+
+void mtk_cam_initial_sensor_setup(struct mtk_cam_request *initial_req,
+ struct mtk_cam_ctx *ctx)
+{
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
+ struct mtk_cam_request_stream_data *req_stream_data;
+
+ sensor_ctrl->ctx = ctx;
+ req_stream_data = mtk_cam_req_get_s_data(initial_req, ctx->stream_id, 0);
+ req_stream_data->ctx = ctx;
+ mtk_cam_set_sensor_full(req_stream_data, &ctx->sensor_ctrl);
+ dev_info(ctx->cam->dev, "Directly setup sensor req:%d\n",
+ req_stream_data->frame_seq_no);
+}
+
+static void mtk_cam_complete_hdl(struct mtk_cam_request_stream_data *s_data,
+ struct media_request_object *hdl_obj,
+ char *name)
+{
+ char *debug_str;
+ u64 start, cost;
+
+ debug_str = mtk_cam_s_data_get_dbg_str(s_data);
+
+ start = ktime_get_boottime_ns();
+ if (hdl_obj->ops)
+ hdl_obj->ops->unbind(hdl_obj); /* mutex used */
+ else
+ pr_info("%s:%s:pipe(%d):seq(%d): cannot unbind %s hd\n",
+ __func__, debug_str, s_data->pipe_id,
+ s_data->frame_seq_no, name);
+
+ cost = ktime_get_boottime_ns() - start;
+ if (cost > 1000000)
+ pr_info("%s:%s:pipe(%d):seq(%d): complete hdl:%s, cost:%llu ns\n",
+ __func__, debug_str, s_data->pipe_id,
+ s_data->frame_seq_no, name, cost);
+ else
+ pr_debug("%s:%s:pipe(%d):seq(%d): complete hdl:%s, cost:%llu ns\n",
+ __func__, debug_str, s_data->pipe_id,
+ s_data->frame_seq_no, name, cost);
+
+ media_request_object_complete(hdl_obj);
+}
+
+void mtk_cam_complete_sensor_hdl(struct mtk_cam_request_stream_data *s_data)
+{
+ if (s_data->flags & MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_EN &&
+ !(s_data->flags & MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_COMPLETE) &&
+ s_data->sensor_hdl_obj) {
+ mtk_cam_complete_hdl(s_data, s_data->sensor_hdl_obj, "sensor");
+ s_data->flags |= MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_COMPLETE;
+ }
+}
+
+void mtk_cam_complete_raw_hdl(struct mtk_cam_request_stream_data *s_data)
+{
+ if ((s_data->flags & MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_EN) &&
+ !(s_data->flags & MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_COMPLETE) &&
+ s_data->raw_hdl_obj) {
+ mtk_cam_complete_hdl(s_data, s_data->raw_hdl_obj, "raw");
+ s_data->flags |= MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_COMPLETE;
+ }
+}
+
+void mtk_cam_req_ctrl_setup(struct mtk_raw_pipeline *raw_pipe,
+ struct mtk_cam_request *req)
+{
+ struct mtk_cam_request_stream_data *req_stream_data;
+
+ req_stream_data = mtk_cam_req_get_s_data(req, raw_pipe->id, 0);
+
+ /* request setup*/
+ if (req_stream_data->flags & MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_EN &&
+ !(req_stream_data->flags & MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_COMPLETE) &&
+ req_stream_data->raw_hdl_obj) {
+ dev_dbg(raw_pipe->subdev.v4l2_dev->dev,
+ "%s:%s:%s:raw ctrl set start (seq:%d)\n",
+ __func__, raw_pipe->subdev.name, req->req.debug_str,
+ req_stream_data->frame_seq_no);
+ v4l2_ctrl_request_setup(&req->req, &raw_pipe->ctrl_handler);
+
+ mtk_cam_complete_raw_hdl(req_stream_data);
+ }
+}
+
+static int timer_reqdrained_chk(int fps_ratio)
+{
+ int timer_ms = 0;
+
+ if (fps_ratio > 1)
+ timer_ms = SENSOR_SET_DEADLINE_MS / fps_ratio;
+ else
+ timer_ms = SENSOR_SET_DEADLINE_MS;
+ /* earlier request drained event*/
+ if (fps_ratio > 1)
+ timer_ms = SENSOR_SET_DEADLINE_MS_60FPS;
+
+ return timer_ms;
+}
+
+static int timer_setsensor(int fps_ratio)
+{
+ int timer_ms = 0;
+
+ if (fps_ratio > 1)
+ timer_ms = SENSOR_SET_RESERVED_MS / fps_ratio;
+ else
+ timer_ms = SENSOR_SET_RESERVED_MS;
+
+ /* faster sensor setting*/
+ if (fps_ratio > 1)
+ timer_ms = SENSOR_SET_RESERVED_MS_60FPS;
+
+ return timer_ms;
+}
+
+int mtk_camsys_ctrl_start(struct mtk_cam_ctx *ctx)
+{
+ struct mtk_camsys_sensor_ctrl *camsys_sensor_ctrl = &ctx->sensor_ctrl;
+ struct v4l2_subdev_frame_interval fi = {
+ .pad = 0,
+ .interval = {
+ .numerator = 1,
+ .denominator = 30
+ },
+ };
+ int fps_factor = 1;
+
+ if (ctx->used_raw_num) {
+ v4l2_subdev_call_state_active(ctx->sensor, pad, get_frame_interval, &fi);
+ fps_factor = (fi.interval.numerator > 0) ?
+ (fi.interval.denominator / fi.interval.numerator / 30) : 1;
+ }
+
+ camsys_sensor_ctrl->ctx = ctx;
+ atomic_set(&camsys_sensor_ctrl->sensor_enq_seq_no, 0);
+ atomic_set(&camsys_sensor_ctrl->sensor_request_seq_no, 0);
+ atomic_set(&camsys_sensor_ctrl->isp_request_seq_no, 0);
+ atomic_set(&camsys_sensor_ctrl->isp_enq_seq_no, 0);
+ atomic_set(&camsys_sensor_ctrl->last_drained_seq_no, 0);
+ camsys_sensor_ctrl->initial_cq_done = 0;
+ camsys_sensor_ctrl->sof_time = 0;
+ if (ctx->used_raw_num)
+ atomic_set(&camsys_sensor_ctrl->initial_drop_frame_cnt, 0);
+
+ camsys_sensor_ctrl->timer_req_event =
+ timer_reqdrained_chk(fps_factor);
+ camsys_sensor_ctrl->timer_req_sensor =
+ timer_setsensor(fps_factor);
+ INIT_LIST_HEAD(&camsys_sensor_ctrl->camsys_state_list);
+ spin_lock_init(&camsys_sensor_ctrl->camsys_state_lock);
+ if (ctx->sensor) {
+ hrtimer_init(&camsys_sensor_ctrl->sensor_deadline_timer,
+ CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+ camsys_sensor_ctrl->sensor_deadline_timer.function =
+ sensor_deadline_timer_handler;
+ camsys_sensor_ctrl->sensorsetting_wq = &ctx->sensor_worker;
+ }
+ kthread_init_work(&camsys_sensor_ctrl->work,
+ mtk_cam_sensor_worker_in_sensorctrl);
+
+ dev_info(ctx->cam->dev,
+ "[%s] ctx:%d/raw_dev:0x%x drained/sensor (%d)%d/%d\n",
+ __func__, ctx->stream_id, ctx->used_raw_dev, fps_factor,
+ camsys_sensor_ctrl->timer_req_event,
+ camsys_sensor_ctrl->timer_req_sensor);
+
+ return 0;
+}
+
+void mtk_camsys_ctrl_update(struct mtk_cam_ctx *ctx)
+{
+ struct mtk_camsys_sensor_ctrl *camsys_sensor_ctrl = &ctx->sensor_ctrl;
+ struct v4l2_subdev_frame_interval fi = {
+ .pad = 0,
+ .interval = {
+ .numerator = 1,
+ .denominator = 30
+ },
+ };
+ int fps_factor = 1;
+
+ if (ctx->used_raw_num) {
+ fi.pad = 0;
+ v4l2_subdev_call_state_active(ctx->sensor, pad, get_frame_interval, &fi);
+ fps_factor = (fi.interval.numerator > 0) ?
+ (fi.interval.denominator / fi.interval.numerator / 30) : 1;
+ }
+
+ camsys_sensor_ctrl->timer_req_event =
+ timer_reqdrained_chk(fps_factor);
+ camsys_sensor_ctrl->timer_req_sensor =
+ timer_setsensor(fps_factor);
+
+ dev_info(ctx->cam->dev,
+ "[%s] ctx:%d/raw_dev:0x%x drained/sensor (%d)%d/%d\n",
+ __func__, ctx->stream_id, ctx->used_raw_dev, fps_factor,
+ camsys_sensor_ctrl->timer_req_event,
+ camsys_sensor_ctrl->timer_req_sensor);
+}
+
+void mtk_camsys_ctrl_stop(struct mtk_cam_ctx *ctx)
+{
+ struct mtk_camsys_sensor_ctrl *camsys_sensor_ctrl = &ctx->sensor_ctrl;
+ struct mtk_camsys_ctrl_state *state_entry, *state_entry_prev;
+
+ spin_lock(&camsys_sensor_ctrl->camsys_state_lock);
+ list_for_each_entry_safe(state_entry, state_entry_prev,
+ &camsys_sensor_ctrl->camsys_state_list,
+ state_element) {
+ list_del(&state_entry->state_element);
+ }
+ spin_unlock(&camsys_sensor_ctrl->camsys_state_lock);
+
+ if (ctx->sensor) {
+ hrtimer_cancel(&camsys_sensor_ctrl->sensor_deadline_timer);
+ camsys_sensor_ctrl->sensorsetting_wq = NULL;
+ }
+ kthread_flush_work(&camsys_sensor_ctrl->work);
+ if (ctx->used_raw_num)
+ mtk_cam_event_eos(ctx->pipe);
+
+ dev_info(ctx->cam->dev, "[%s] ctx:%d/raw_dev:0x%x\n",
+ __func__, ctx->stream_id, ctx->used_raw_dev);
+}
\ No newline at end of file
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h
new file mode 100755
index 000000000000..c59e0241a706
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_CTRL_H
+#define __MTK_CAM_CTRL_H
+
+#include <linux/hrtimer.h>
+#include <linux/timer.h>
+
+struct mtk_cam_device;
+
+enum MTK_CAMSYS_IRQ_EVENT {
+ /* with normal_data */
+ CAMSYS_IRQ_SETTING_DONE = 0,
+ CAMSYS_IRQ_FRAME_START,
+ CAMSYS_IRQ_AFO_DONE,
+ CAMSYS_IRQ_FRAME_DONE,
+ CAMSYS_IRQ_SUBSAMPLE_SENSOR_SET,
+ CAMSYS_IRQ_FRAME_DROP,
+
+ /* with error_data */
+ CAMSYS_IRQ_ERROR,
+};
+
+struct mtk_camsys_irq_normal_data {
+ /* with normal_status */
+};
+
+struct mtk_camsys_irq_error_data {
+ /* with error_status */
+ int err_status;
+};
+
+struct mtk_camsys_irq_info {
+ enum MTK_CAMSYS_IRQ_EVENT irq_type;
+ u64 ts_ns;
+ int frame_idx;
+ int frame_idx_inner;
+ bool sub_engine;
+ int write_cnt;
+ int fbc_cnt;
+ union {
+ struct mtk_camsys_irq_normal_data n;
+ struct mtk_camsys_irq_error_data e;
+ };
+};
+
+enum MTK_CAMSYS_STATE_IDX {
+ /* For state analysis and controlling for request */
+ E_STATE_READY = 0x0,
+ E_STATE_SENINF,
+ E_STATE_SENSOR,
+ E_STATE_CQ,
+ E_STATE_OUTER,
+ E_STATE_CAMMUX_OUTER_CFG,
+ E_STATE_CAMMUX_OUTER,
+ E_STATE_INNER,
+ E_STATE_DONE_NORMAL,
+ E_STATE_CQ_SCQ_DELAY,
+ E_STATE_CAMMUX_OUTER_CFG_DELAY,
+ E_STATE_OUTER_HW_DELAY,
+ E_STATE_INNER_HW_DELAY,
+ E_STATE_DONE_MISMATCH,
+};
+
+struct mtk_camsys_ctrl_state {
+ enum MTK_CAMSYS_STATE_IDX estate;
+ struct list_head state_element;
+};
+
+struct mtk_camsys_link_ctrl {
+ struct mtk_raw_pipeline *pipe;
+ struct media_pad remote;
+ struct mtk_cam_ctx *swapping_ctx;
+ u8 active;
+ u8 wait_exchange;
+};
+
+/* per stream from sensor */
+struct mtk_camsys_sensor_ctrl {
+ struct mtk_cam_ctx *ctx;
+ struct kthread_worker *sensorsetting_wq;
+ struct kthread_work work;
+ struct hrtimer sensor_deadline_timer;
+ u64 sof_time;
+ int timer_req_sensor;
+ int timer_req_event;
+ atomic_t sensor_enq_seq_no;
+ atomic_t sensor_request_seq_no;
+ atomic_t isp_request_seq_no;
+ atomic_t isp_enq_seq_no;
+ atomic_t last_drained_seq_no;
+ int initial_cq_done;
+ atomic_t initial_drop_frame_cnt;
+ struct list_head camsys_state_list;
+ spinlock_t camsys_state_lock; /* protect camsys_state_list */
+ /* link change ctrl */
+ struct mtk_camsys_link_ctrl link_ctrl;
+ struct mtk_cam_request *link_change_req;
+};
+
+void mtk_camsys_state_delete(struct mtk_cam_ctx *ctx,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl,
+ struct mtk_cam_request *req);
+void mtk_camsys_frame_done(struct mtk_cam_ctx *ctx, unsigned int frame_seq_no,
+ unsigned int pipe_id);
+int mtk_camsys_isr_event(struct mtk_cam_device *cam,
+ unsigned int engine_id,
+ struct mtk_camsys_irq_info *irq_info);
+bool mtk_cam_submit_kwork_in_sensorctrl(struct kthread_worker *worker,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl);
+void mtk_cam_initial_sensor_setup(struct mtk_cam_request *req,
+ struct mtk_cam_ctx *ctx);
+void mtk_cam_req_ctrl_setup(struct mtk_raw_pipeline *raw_pipe,
+ struct mtk_cam_request *req);
+int mtk_camsys_ctrl_start(struct mtk_cam_ctx *ctx); /* ctx_stream_on */
+void mtk_camsys_ctrl_update(struct mtk_cam_ctx *ctx);
+void mtk_camsys_ctrl_stop(struct mtk_cam_ctx *ctx); /* ctx_stream_off */
+void mtk_cam_frame_done_work(struct work_struct *work);
+void mtk_cam_meta1_done_work(struct work_struct *work);
+void mtk_cam_set_sensor_full(struct mtk_cam_request_stream_data *s_data,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl);
+void state_transition(struct mtk_camsys_ctrl_state *state_entry,
+ enum MTK_CAMSYS_STATE_IDX from,
+ enum MTK_CAMSYS_STATE_IDX to);
+void mtk_cam_set_sensor_switch(struct mtk_cam_request_stream_data *s_data,
+ struct mtk_camsys_sensor_ctrl *sensor_ctrl);
+
+#endif /* __MTK_CAM_CTRL_H */
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 10/13] media: platform: mediatek: add isp_7x video ops
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (4 preceding siblings ...)
2025-07-07 1:31 ` [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-08-01 5:45 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 11/13] media: platform: mediatek: add isp_7x build config shangyao lin
` (7 subsequent siblings)
13 siblings, 1 reply; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Introduce the V4L2 video interface and feature management for the MediaTek
ISP7.x CAMSYS. These interfaces provide functionalities such as video operation
initialization and registration. They also manage MediaTek-specific formats and
handle buffers for MediaTek camera video devices. This enables CAMSYS
functionalities to be compatible with the V4L2 framework.
Changes in v2:
- Removed mtk_cam-feature.c and mtk_cam-feature.h, along with related code
- Various fixes per review comments
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../isp/isp_7x/camsys/mtk_cam-video.c | 1594 +++++++++++++++++
.../isp/isp_7x/camsys/mtk_cam-video.h | 223 +++
2 files changed, 1817 insertions(+)
create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.c
create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.h
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.c b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.c
new file mode 100755
index 000000000000..8f071c259851
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.c
@@ -0,0 +1,1594 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (c) 2022 MediaTek Inc.
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <linux/videodev2.h>
+#include <linux/iommu.h>
+
+#include "mtk_cam.h"
+#include "mtk_cam-video.h"
+#include "mtk_camera-v4l2-controls.h"
+
+/*
+ * differt dma (fmt) would have different bus_size
+ * align xsize(bytes per line) with [bus_size * pixel_mode]
+ */
+static inline int mtk_cam_is_fullg(unsigned int ipi_fmt)
+{
+ return (ipi_fmt == MTKCAM_IPI_IMG_FMT_FG_BAYER8) ||
+ (ipi_fmt == MTKCAM_IPI_IMG_FMT_FG_BAYER10) ||
+ (ipi_fmt == MTKCAM_IPI_IMG_FMT_FG_BAYER12);
+}
+
+static inline
+unsigned int mtk_cam_dma_bus_size(int bpp, int pixel_mode_shift, int is_fg)
+{
+ unsigned int bus_size = ALIGN(bpp, 16) << pixel_mode_shift;
+
+ if (is_fg)
+ bus_size <<= 1;
+ return bus_size / 8; /* in bytes */
+}
+
+static inline
+unsigned int mtk_cam_yuv_dma_bus_size(int bpp, int pixel_mode_shift)
+{
+ unsigned int bus_size = ALIGN(bpp, 32) << pixel_mode_shift;
+
+ return bus_size / 8; /* in bytes */
+}
+
+static inline
+unsigned int mtk_cam_dmao_xsize(int w, unsigned int ipi_fmt, int pixel_mode_shift)
+{
+ const unsigned int is_fg = mtk_cam_is_fullg(ipi_fmt);
+ const unsigned int bpp = mtk_cam_get_pixel_bits(ipi_fmt);
+ const unsigned int bytes = is_fg ?
+ DIV_ROUND_UP(w * bpp * 3 / 2, 8) : DIV_ROUND_UP(w * bpp, 8);
+ const unsigned int bus_size =
+ mtk_cam_dma_bus_size(bpp, pixel_mode_shift, is_fg);
+
+ return ALIGN(bytes, bus_size);
+}
+
+static void mtk_cam_release_all_buffer(struct mtk_cam_device *cam)
+{
+ struct mtk_cam_buffer *buf, *tmp;
+
+ spin_lock(&cam->dma_pending_lock);
+ list_for_each_entry_safe(buf, tmp, &cam->dma_pending, list) {
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vbb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ spin_unlock(&cam->dma_pending_lock);
+
+ spin_lock(&cam->dma_processing_lock);
+ list_for_each_entry_safe(buf, tmp, &cam->dma_processing, list) {
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vbb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ spin_unlock(&cam->dma_processing_lock);
+}
+
+static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
+ unsigned int *num_buffers,
+ unsigned int *num_planes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vq);
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+ struct mtk_raw_pipeline *raw_pipeline;
+ unsigned int max_buffer_count = node->desc.max_buf_count;
+ const struct v4l2_format *fmt = &node->active_fmt;
+ unsigned int size;
+ int i;
+
+ /* Check the limitation of buffer size */
+ if (max_buffer_count)
+ *num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
+
+ if (node->desc.smem_alloc)
+ vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
+
+ if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
+ vq->type == V4L2_BUF_TYPE_META_CAPTURE)
+ size = fmt->fmt.meta.buffersize;
+ else
+ size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ /* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
+ if (*num_planes) {
+ if (sizes[0] < size || *num_planes != 1)
+ return -EINVAL;
+ } else {
+ /* Set default as one plane */
+ *num_planes = 1;
+ sizes[0] = size;
+
+ if (is_raw_subdev(node->uid.pipe_id)) {
+ raw_pipeline =
+ mtk_cam_dev_get_raw_pipeline(cam, node->uid.pipe_id);
+
+ if (raw_pipeline &&
+ raw_pipeline->user_res.raw_res.feature &&
+ fmt->fmt.pix_mp.num_planes > 1) {
+ *num_planes = fmt->fmt.pix_mp.num_planes;
+ for (i = 0; i < *num_planes; i++)
+ sizes[i] = size;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int mtk_cam_vb2_buf_init(struct vb2_buffer *vb)
+{
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+ struct mtk_cam_buffer *buf;
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vb->vb2_queue);
+ struct device *dev = cam->raw.devs[0];
+ dma_addr_t addr;
+
+ buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+ buf->daddr = vb2_dma_contig_plane_dma_addr(vb, 0);
+ buf->scp_addr = 0;
+
+ /* SCP address is only valid for meta input buffer */
+ if (!node->desc.smem_alloc)
+ return 0;
+
+ /* Use coherent address to get iova address */
+ addr = dma_map_resource(dev, buf->daddr, vb->planes[0].length,
+ DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+ if (dma_mapping_error(dev, addr)) {
+ dev_err(dev, "failed to map addr:%pad\n", &buf->daddr);
+ return -EFAULT;
+ }
+ buf->scp_addr = buf->daddr;
+ buf->daddr = addr;
+
+ return 0;
+}
+
+static int mtk_cam_vb2_buf_prepare(struct vb2_buffer *vb)
+{
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+ struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+ const struct v4l2_format *fmt = &node->active_fmt;
+ unsigned int size;
+
+ dev_dbg(vb->vb2_queue->dev, "%s: %s\n", __func__, node->desc.name);
+
+ if (vb->vb2_queue->type == V4L2_BUF_TYPE_META_OUTPUT ||
+ vb->vb2_queue->type == V4L2_BUF_TYPE_META_CAPTURE)
+ size = fmt->fmt.meta.buffersize;
+ else
+ size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
+
+ if (vb2_plane_size(vb, 0) < size) {
+ dev_info(vb->vb2_queue->dev,
+ "plane size is too small:%lu<%u\n",
+ vb2_plane_size(vb, 0), size);
+ }
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+ if ((vb2_get_plane_payload(vb, 0) != size) &&
+ vb->vb2_queue->streaming) {
+ dev_dbg(vb->vb2_queue->dev,
+ "plane payload is mismatch:%lu:%u\n",
+ vb2_get_plane_payload(vb, 0), size);
+ }
+ return 0;
+ }
+
+ v4l2_buf->field = V4L2_FIELD_NONE;
+ vb2_set_plane_payload(vb, 0, size);
+
+ return 0;
+}
+
+static int mtk_cam_vb2_start_streaming(struct vb2_queue *vq,
+ unsigned int count)
+{
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vq);
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+ struct media_entity *entity = &node->vdev.entity;
+ struct mtk_cam_ctx *ctx = NULL;
+ struct device *dev = cam->dev;
+ int ret;
+
+ /* check entity is linked */
+ if (!node->enabled) {
+ dev_info(cam->dev,
+ "%s: stream on failed, node is not enabled\n",
+ node->desc.name);
+ ret = -ENOLINK;
+ goto fail_return_buffer;
+ }
+
+ if (!media_entity_is_streaming(entity)) {
+ ctx = mtk_cam_start_ctx(cam, node);
+ if (!ctx) {
+ ret = -ENOLINK;
+ goto fail_return_buffer;
+ }
+ } else {
+ ctx = mtk_cam_find_ctx(cam, entity);
+ if (WARN_ON(!ctx)) {
+ ret = -ENOLINK;
+ goto fail_return_buffer;
+ }
+ }
+
+ cam->streaming_pipe |= (1 << node->uid.pipe_id);
+ ctx->streaming_pipe |= (1 << node->uid.pipe_id);
+ ctx->streaming_node_cnt++;
+
+ if (ctx->streaming_node_cnt == 1)
+ if (is_raw_subdev(node->uid.pipe_id)) {
+ if (!isp_composer_create_session(ctx)) {
+ ctx->session_created = 1;
+ } else {
+ complete(&ctx->session_complete);
+ ret = -EBUSY;
+ goto fail_stop_ctx;
+ }
+ }
+
+ dev_dbg(dev, "%s:%s:ctx(%d): node:%d count info:%d\n", __func__,
+ node->desc.name, ctx->stream_id, node->desc.id,
+ ctx->streaming_node_cnt);
+
+ ret = mtk_cam_ctx_stream_on(ctx, node);
+ if (ret)
+ goto fail_destroy_session;
+
+ mtk_cam_buf_try_queue(ctx);
+
+ return 0;
+
+fail_destroy_session:
+ if (ctx->session_created)
+ isp_composer_destroy_session(ctx);
+fail_stop_ctx:
+ ctx->streaming_node_cnt--;
+ ctx->streaming_pipe &= ~(1 << node->uid.pipe_id);
+ cam->streaming_pipe &= ~(1 << node->uid.pipe_id);
+ mtk_cam_dev_req_cleanup(ctx, node->uid.pipe_id, VB2_BUF_STATE_QUEUED);
+ mtk_cam_stop_ctx(ctx, node);
+fail_return_buffer:
+ mtk_cam_release_all_buffer(cam);
+ /* relese bufs by request */
+ return ret;
+}
+
+static void mtk_cam_vb2_stop_streaming(struct vb2_queue *vq)
+{
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vq);
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
+ struct device *dev = cam->dev;
+ struct mtk_cam_ctx *ctx;
+
+ ctx = mtk_cam_find_ctx(cam, &node->vdev.entity);
+ if (WARN_ON(!ctx)) {
+ /* the ctx is stop, media_pipeline_stop is called */
+ mtk_cam_dev_req_clean_pending(cam, node->uid.pipe_id,
+ VB2_BUF_STATE_ERROR);
+ return;
+ }
+
+ dev_dbg(dev, "%s:%s:ctx(%d): node:%d count info:%d\n", __func__,
+ node->desc.name, ctx->stream_id, node->desc.id,
+ ctx->streaming_node_cnt);
+
+ mtk_cam_ctx_stream_off(ctx, node);
+
+ if (cam->streaming_pipe & (1 << node->uid.pipe_id)) {
+ /* NOTE: take multi-pipelines case into consideration */
+ /* Moreover, must clean bit mask before req cleanup */
+ /* Otherwise, would cause req not removed in pending list */
+ cam->streaming_pipe &= ~(1 << node->uid.pipe_id);
+ mtk_cam_dev_req_cleanup(ctx, node->uid.pipe_id, VB2_BUF_STATE_ERROR);
+ }
+
+ /* all bufs of node should be return by per requests */
+ mtk_cam_release_all_buffer(ctx->cam);
+
+ /* NOTE: take multi-pipelines case into consideration */
+ cam->streaming_pipe &= ~(1 << node->uid.pipe_id);
+ ctx->streaming_node_cnt--;
+ if (ctx->streaming_node_cnt)
+ return;
+
+ mtk_cam_stop_ctx(ctx, node);
+}
+
+int is_mtk_format(u32 pixelformat)
+{
+ switch (pixelformat) {
+ case V4L2_PIX_FMT_YUYV10:
+ case V4L2_PIX_FMT_YVYU10:
+ case V4L2_PIX_FMT_UYVY10:
+ case V4L2_PIX_FMT_VYUY10:
+ case V4L2_PIX_FMT_YUYV12:
+ case V4L2_PIX_FMT_YVYU12:
+ case V4L2_PIX_FMT_UYVY12:
+ case V4L2_PIX_FMT_VYUY12:
+ case V4L2_PIX_FMT_MTISP_YUYV10P:
+ case V4L2_PIX_FMT_MTISP_YVYU10P:
+ case V4L2_PIX_FMT_MTISP_UYVY10P:
+ case V4L2_PIX_FMT_MTISP_VYUY10P:
+ case V4L2_PIX_FMT_MTISP_YUYV12P:
+ case V4L2_PIX_FMT_MTISP_YVYU12P:
+ case V4L2_PIX_FMT_MTISP_UYVY12P:
+ case V4L2_PIX_FMT_MTISP_VYUY12P:
+ case V4L2_PIX_FMT_NV12_10:
+ case V4L2_PIX_FMT_NV21_10:
+ case V4L2_PIX_FMT_NV16_10:
+ case V4L2_PIX_FMT_NV61_10:
+ case V4L2_PIX_FMT_NV12_12:
+ case V4L2_PIX_FMT_NV21_12:
+ case V4L2_PIX_FMT_NV16_12:
+ case V4L2_PIX_FMT_NV61_12:
+ case V4L2_PIX_FMT_MTISP_NV12_10P:
+ case V4L2_PIX_FMT_MTISP_NV21_10P:
+ case V4L2_PIX_FMT_MTISP_NV16_10P:
+ case V4L2_PIX_FMT_MTISP_NV61_10P:
+ case V4L2_PIX_FMT_MTISP_NV12_12P:
+ case V4L2_PIX_FMT_MTISP_NV21_12P:
+ case V4L2_PIX_FMT_MTISP_NV16_12P:
+ case V4L2_PIX_FMT_MTISP_NV61_12P:
+ case V4L2_PIX_FMT_MTISP_NV12_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV21_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV12_10_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV21_10_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV12_12_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV21_12_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER8_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER10_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER12_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER14_UFBC:
+ case V4L2_PIX_FMT_MTISP_SGRB8F:
+ case V4L2_PIX_FMT_MTISP_SGRB10F:
+ case V4L2_PIX_FMT_MTISP_SGRB12F:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+int is_yuv_ufo(u32 pixelformat)
+{
+ switch (pixelformat) {
+ case V4L2_PIX_FMT_MTISP_NV12_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV21_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV12_10_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV21_10_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV12_12_UFBC:
+ case V4L2_PIX_FMT_MTISP_NV21_12_UFBC:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+int is_raw_ufo(u32 pixelformat)
+{
+ switch (pixelformat) {
+ case V4L2_PIX_FMT_MTISP_BAYER8_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER10_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER12_UFBC:
+ case V4L2_PIX_FMT_MTISP_BAYER14_UFBC:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+int is_fullg_rb(u32 pixelformat)
+{
+ switch (pixelformat) {
+ case V4L2_PIX_FMT_MTISP_SGRB8F:
+ case V4L2_PIX_FMT_MTISP_SGRB10F:
+ case V4L2_PIX_FMT_MTISP_SGRB12F:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+const struct mtk_format_info *mtk_format_info(u32 format)
+{
+ static const struct mtk_format_info formats[] = {
+ /* YUV planar formats */
+ { .format = V4L2_PIX_FMT_NV12_10,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV21_10,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV16_10,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV61_10,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_YUYV10,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_YVYU10,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_UYVY10,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_VYUY10,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV12_12,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV21_12,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV16_12,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_NV61_12,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_YUYV12,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_YVYU12,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_UYVY12,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_VYUY12,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
+ /* YUV packed formats */
+ { .format = V4L2_PIX_FMT_MTISP_YUYV10P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_YVYU10P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_UYVY10P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_VYUY10P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_NV12_10P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_NV21_10P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_NV16_10P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_NV61_10P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_YUYV12P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_YVYU12P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_UYVY12P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_VYUY12P,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_NV12_12P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_NV21_12P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_NV16_12P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_NV61_12P,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ /* YUV UFBC formats */
+ { .format = V4L2_PIX_FMT_MTISP_NV12_UFBC,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_MTISP_NV21_UFBC,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_MTISP_NV12_10_UFBC,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_NV21_10_UFBC,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_NV12_12_UFBC,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_NV21_12_UFBC,
+ .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_BAYER8_UFBC,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
+ .hdiv = 1, .vdiv = 1, .bit_r_num = 1, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_MTISP_BAYER10_UFBC,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
+ .hdiv = 1, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_BAYER12_UFBC,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
+ .hdiv = 1, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
+ { .format = V4L2_PIX_FMT_MTISP_BAYER14_UFBC,
+ .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
+ .hdiv = 1, .vdiv = 1, .bit_r_num = 7, .bit_r_den = 4 },
+ /* Full-G RGB formats */
+ { .format = V4L2_PIX_FMT_MTISP_SGRB8F,
+ .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 1, .bit_r_den = 1 },
+ { .format = V4L2_PIX_FMT_MTISP_SGRB10F,
+ .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
+ { .format = V4L2_PIX_FMT_MTISP_SGRB12F,
+ .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
+ .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
+ };
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(formats); ++i)
+ if (formats[i].format == format)
+ return &formats[i];
+ return NULL;
+}
+
+static void mtk_cam_vb2_buf_queue(struct vb2_buffer *vb)
+{
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vb->vb2_queue);
+ struct mtk_cam_buffer *buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+ struct mtk_cam_request *req = to_mtk_cam_req(vb->request);
+ struct mtk_cam_request_stream_data *req_stream_data;
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+ struct mtk_raw_pde_config *pde_cfg;
+ struct device *dev = cam->dev;
+ struct mtkcam_ipi_frame_param *frame_param;
+ struct mtkcam_ipi_meta_input *meta_in;
+ struct mtkcam_ipi_meta_output *meta_out;
+ struct mtk_cam_ctx *ctx;
+ unsigned int pipe_id;
+ unsigned int desc_id;
+ unsigned int dma_port = node->desc.dma_port;
+
+ if (!vb->vb2_queue->uses_requests) {
+ spin_lock(&cam->dma_pending_lock);
+ list_add_tail(&buf->list, &cam->dma_pending);
+ spin_unlock(&cam->dma_pending_lock);
+ buf->state.estate = E_BUF_STATE_QUEUED;
+ if (media_entity_is_streaming(&node->vdev.entity)) {
+ ctx = mtk_cam_find_ctx(cam, &node->vdev.entity);
+ mtk_cam_buf_try_queue(ctx);
+ }
+ return;
+ }
+
+ dma_port = node->desc.dma_port;
+ pipe_id = node->uid.pipe_id;
+ req_stream_data = mtk_cam_req_get_s_data(req, pipe_id, 0);
+ frame_param = &req_stream_data->frame_params;
+ mtk_cam_s_data_set_vbuf(req_stream_data, buf, node->desc.id);
+
+ /* update buffer internal address */
+ switch (dma_port) {
+ case MTKCAM_ISP_META_STATS_CFG:
+ desc_id = node->desc.id - MTK_RAW_SINK_NUM;
+ meta_in = &frame_param->meta_inputs[desc_id];
+ meta_in->buf.size = node->active_fmt.fmt.meta.buffersize;
+ meta_in->buf.iova = buf->daddr;
+ meta_in->buf.scp_addr = buf->scp_addr;
+ meta_in->uid.id = dma_port;
+ break;
+ case MTKCAM_ISP_META_STATS_0:
+ case MTKCAM_ISP_META_STATS_1:
+ case MTKCAM_ISP_META_STATS_2:
+ pde_cfg = &cam->raw.pipelines[node->uid.pipe_id].pde_config;
+ desc_id = node->desc.id - MTK_RAW_META_OUT_BEGIN;
+ meta_out = &frame_param->meta_outputs[desc_id];
+ meta_out->buf.size = node->active_fmt.fmt.meta.buffersize;
+ meta_out->buf.iova = buf->daddr;
+ meta_out->buf.scp_addr = buf->scp_addr;
+ meta_out->uid.id = dma_port;
+ camsys_set_meta_stats_info(dma_port, vb, pde_cfg);
+ break;
+ default:
+ dev_dbg(dev, "%s:pipe(%d):buffer with invalid port(%d)\n",
+ __func__, pipe_id, dma_port);
+ break;
+ }
+}
+
+static void mtk_cam_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+ struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vb->vb2_queue);
+ struct mtk_cam_buffer *buf;
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vb->vb2_queue);
+ struct device *dev = cam->raw.devs[0];
+
+ /* SCP address is only valid for meta input buffer */
+ if (!node->desc.smem_alloc)
+ return;
+
+ buf = mtk_cam_vb2_buf_to_dev_buf(vb);
+ dma_unmap_page_attrs(dev, buf->daddr, vb->planes[0].length,
+ DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
+}
+
+static void mtk_cam_vb2_request_complete(struct vb2_buffer *vb)
+{
+ struct mtk_cam_device *cam = vb2_get_drv_priv(vb->vb2_queue);
+
+ dev_dbg(vb->vb2_queue->dev, "camsys | request %s\n", __func__);
+
+ v4l2_ctrl_request_complete(vb->req_obj.req, cam->v4l2_dev.ctrl_handler);
+}
+
+static int mtk_cam_vb2_buf_out_validate(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *v4l2_buf = to_vb2_v4l2_buffer(vb);
+
+ if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ if (v4l2_buf->field == V4L2_FIELD_ANY)
+ v4l2_buf->field = V4L2_FIELD_NONE;
+
+ if (v4l2_buf->field != V4L2_FIELD_NONE)
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static const struct vb2_ops mtk_cam_vb2_ops = {
+ .queue_setup = mtk_cam_vb2_queue_setup,
+
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+
+ .buf_out_validate = mtk_cam_vb2_buf_out_validate,
+ .buf_init = mtk_cam_vb2_buf_init,
+ .buf_prepare = mtk_cam_vb2_buf_prepare,
+
+ .start_streaming = mtk_cam_vb2_start_streaming,
+ .stop_streaming = mtk_cam_vb2_stop_streaming,
+
+ .buf_queue = mtk_cam_vb2_buf_queue,
+ .buf_cleanup = mtk_cam_vb2_buf_cleanup,
+ .buf_request_complete = mtk_cam_vb2_request_complete,
+};
+
+static const struct v4l2_file_operations mtk_cam_v4l2_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl32 = v4l2_compat_ioctl32,
+#endif
+};
+
+unsigned int mtk_cam_get_sensor_pixel_id(unsigned int fmt)
+{
+ switch (fmt & SENSOR_FMT_MASK) {
+ case MEDIA_BUS_FMT_SBGGR8_1X8:
+ case MEDIA_BUS_FMT_SBGGR10_1X10:
+ case MEDIA_BUS_FMT_SBGGR12_1X12:
+ case MEDIA_BUS_FMT_SBGGR14_1X14:
+ return MTKCAM_IPI_BAYER_PXL_ID_B;
+ case MEDIA_BUS_FMT_SGBRG8_1X8:
+ case MEDIA_BUS_FMT_SGBRG10_1X10:
+ case MEDIA_BUS_FMT_SGBRG12_1X12:
+ case MEDIA_BUS_FMT_SGBRG14_1X14:
+ return MTKCAM_IPI_BAYER_PXL_ID_GB;
+ case MEDIA_BUS_FMT_SGRBG8_1X8:
+ case MEDIA_BUS_FMT_SGRBG10_1X10:
+ case MEDIA_BUS_FMT_SGRBG12_1X12:
+ case MEDIA_BUS_FMT_SGRBG14_1X14:
+ return MTKCAM_IPI_BAYER_PXL_ID_GR;
+ case MEDIA_BUS_FMT_SRGGB8_1X8:
+ case MEDIA_BUS_FMT_SRGGB10_1X10:
+ case MEDIA_BUS_FMT_SRGGB12_1X12:
+ case MEDIA_BUS_FMT_SRGGB14_1X14:
+ return MTKCAM_IPI_BAYER_PXL_ID_R;
+ default:
+ return MTKCAM_IPI_BAYER_PXL_ID_UNKNOWN;
+ }
+}
+
+unsigned int mtk_cam_get_sensor_fmt(unsigned int fmt)
+{
+ switch (fmt & SENSOR_FMT_MASK) {
+ case MEDIA_BUS_FMT_SBGGR8_1X8:
+ case MEDIA_BUS_FMT_SGBRG8_1X8:
+ case MEDIA_BUS_FMT_SGRBG8_1X8:
+ case MEDIA_BUS_FMT_SRGGB8_1X8:
+ return MTKCAM_IPI_IMG_FMT_BAYER8;
+ case MEDIA_BUS_FMT_SBGGR10_1X10:
+ case MEDIA_BUS_FMT_SGBRG10_1X10:
+ case MEDIA_BUS_FMT_SGRBG10_1X10:
+ case MEDIA_BUS_FMT_SRGGB10_1X10:
+ return MTKCAM_IPI_IMG_FMT_BAYER10;
+ case MEDIA_BUS_FMT_SBGGR12_1X12:
+ case MEDIA_BUS_FMT_SGBRG12_1X12:
+ case MEDIA_BUS_FMT_SGRBG12_1X12:
+ case MEDIA_BUS_FMT_SRGGB12_1X12:
+ return MTKCAM_IPI_IMG_FMT_BAYER12;
+ case MEDIA_BUS_FMT_SBGGR14_1X14:
+ case MEDIA_BUS_FMT_SGBRG14_1X14:
+ case MEDIA_BUS_FMT_SGRBG14_1X14:
+ case MEDIA_BUS_FMT_SRGGB14_1X14:
+ return MTKCAM_IPI_IMG_FMT_BAYER14;
+ default:
+ return MTKCAM_IPI_IMG_FMT_UNKNOWN;
+ }
+}
+
+unsigned int mtk_cam_get_pixel_bits(unsigned int ipi_fmt)
+{
+ switch (ipi_fmt) {
+ case MTKCAM_IPI_IMG_FMT_BAYER8:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER8:
+ return 8;
+ case MTKCAM_IPI_IMG_FMT_BAYER10:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER10:
+ case MTKCAM_IPI_IMG_FMT_BAYER10_MIPI:
+ return 10;
+ case MTKCAM_IPI_IMG_FMT_BAYER12:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER12:
+ return 12;
+ case MTKCAM_IPI_IMG_FMT_BAYER14:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER14:
+ return 14;
+ case MTKCAM_IPI_IMG_FMT_BAYER10_UNPACKED:
+ case MTKCAM_IPI_IMG_FMT_BAYER12_UNPACKED:
+ case MTKCAM_IPI_IMG_FMT_BAYER14_UNPACKED:
+ case MTKCAM_IPI_IMG_FMT_BAYER16:
+ case MTKCAM_IPI_IMG_FMT_YUYV:
+ case MTKCAM_IPI_IMG_FMT_YVYU:
+ case MTKCAM_IPI_IMG_FMT_UYVY:
+ case MTKCAM_IPI_IMG_FMT_VYUY:
+ return 16;
+ case MTKCAM_IPI_IMG_FMT_Y8:
+ case MTKCAM_IPI_IMG_FMT_YUV_422_2P:
+ case MTKCAM_IPI_IMG_FMT_YVU_422_2P:
+ case MTKCAM_IPI_IMG_FMT_YUV_422_3P:
+ case MTKCAM_IPI_IMG_FMT_YVU_422_3P:
+ case MTKCAM_IPI_IMG_FMT_YUV_420_2P:
+ case MTKCAM_IPI_IMG_FMT_YVU_420_2P:
+ case MTKCAM_IPI_IMG_FMT_YUV_420_3P:
+ case MTKCAM_IPI_IMG_FMT_YVU_420_3P:
+ return 8;
+ case MTKCAM_IPI_IMG_FMT_YUYV_Y210:
+ case MTKCAM_IPI_IMG_FMT_YVYU_Y210:
+ case MTKCAM_IPI_IMG_FMT_UYVY_Y210:
+ case MTKCAM_IPI_IMG_FMT_VYUY_Y210:
+ return 32;
+ case MTKCAM_IPI_IMG_FMT_YUV_P210:
+ case MTKCAM_IPI_IMG_FMT_YVU_P210:
+ case MTKCAM_IPI_IMG_FMT_YUV_P010:
+ case MTKCAM_IPI_IMG_FMT_YVU_P010:
+ case MTKCAM_IPI_IMG_FMT_YUV_P212:
+ case MTKCAM_IPI_IMG_FMT_YVU_P212:
+ case MTKCAM_IPI_IMG_FMT_YUV_P012:
+ case MTKCAM_IPI_IMG_FMT_YVU_P012:
+ return 16;
+ case MTKCAM_IPI_IMG_FMT_YUYV_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVYU_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_UYVY_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_VYUY_Y210_PACKED:
+ return 20;
+ case MTKCAM_IPI_IMG_FMT_YUV_P210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YUV_P010_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P010_PACKED:
+ return 10;
+ case MTKCAM_IPI_IMG_FMT_YUV_P212_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P212_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YUV_P012_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P012_PACKED:
+ return 12;
+ case MTKCAM_IPI_IMG_FMT_RGB_8B_3P:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER8_3P:
+ case MTKCAM_IPI_IMG_FMT_UFBC_NV12:
+ case MTKCAM_IPI_IMG_FMT_UFBC_NV21:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER8:
+ return 8;
+ case MTKCAM_IPI_IMG_FMT_RGB_10B_3P_PACKED:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER10_3P_PACKED:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YUV_P010:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YVU_P010:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER10:
+ return 10;
+ case MTKCAM_IPI_IMG_FMT_RGB_12B_3P_PACKED:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER12_3P_PACKED:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YUV_P012:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YVU_P012:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER12:
+ return 12;
+ case MTKCAM_IPI_IMG_FMT_RGB_10B_3P:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER10_3P:
+ case MTKCAM_IPI_IMG_FMT_RGB_12B_3P:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER12_3P:
+ return 16;
+
+ default:
+ break;
+ }
+ pr_debug("not supported ipi-fmt 0x%08x", ipi_fmt);
+
+ return -1;
+}
+
+unsigned int mtk_cam_get_img_fmt(unsigned int fourcc)
+{
+ switch (fourcc) {
+ case V4L2_PIX_FMT_GREY:
+ return MTKCAM_IPI_IMG_FMT_Y8;
+ case V4L2_PIX_FMT_YUYV:
+ return MTKCAM_IPI_IMG_FMT_YUYV;
+ case V4L2_PIX_FMT_YVYU:
+ return MTKCAM_IPI_IMG_FMT_YVYU;
+ case V4L2_PIX_FMT_NV16:
+ return MTKCAM_IPI_IMG_FMT_YUV_422_2P;
+ case V4L2_PIX_FMT_NV61:
+ return MTKCAM_IPI_IMG_FMT_YVU_422_2P;
+ case V4L2_PIX_FMT_NV12:
+ return MTKCAM_IPI_IMG_FMT_YUV_420_2P;
+ case V4L2_PIX_FMT_NV21:
+ return MTKCAM_IPI_IMG_FMT_YVU_420_2P;
+ case V4L2_PIX_FMT_YUV422P:
+ return MTKCAM_IPI_IMG_FMT_YUV_422_3P;
+ case V4L2_PIX_FMT_YUV420:
+ return MTKCAM_IPI_IMG_FMT_YUV_420_3P;
+ case V4L2_PIX_FMT_YVU420:
+ return MTKCAM_IPI_IMG_FMT_YVU_420_3P;
+ case V4L2_PIX_FMT_NV12_10:
+ return MTKCAM_IPI_IMG_FMT_YUV_P010;
+ case V4L2_PIX_FMT_NV21_10:
+ return MTKCAM_IPI_IMG_FMT_YVU_P010;
+ case V4L2_PIX_FMT_NV16_10:
+ return MTKCAM_IPI_IMG_FMT_YUV_P210;
+ case V4L2_PIX_FMT_NV61_10:
+ return MTKCAM_IPI_IMG_FMT_YVU_P210;
+ case V4L2_PIX_FMT_MTISP_NV12_10P:
+ return MTKCAM_IPI_IMG_FMT_YUV_P010_PACKED;
+ case V4L2_PIX_FMT_MTISP_NV21_10P:
+ return MTKCAM_IPI_IMG_FMT_YVU_P010_PACKED;
+ case V4L2_PIX_FMT_MTISP_NV16_10P:
+ return MTKCAM_IPI_IMG_FMT_YUV_P210_PACKED;
+ case V4L2_PIX_FMT_MTISP_NV61_10P:
+ return MTKCAM_IPI_IMG_FMT_YVU_P210_PACKED;
+ case V4L2_PIX_FMT_YUYV10:
+ return MTKCAM_IPI_IMG_FMT_YUYV_Y210;
+ case V4L2_PIX_FMT_YVYU10:
+ return MTKCAM_IPI_IMG_FMT_YVYU_Y210;
+ case V4L2_PIX_FMT_UYVY10:
+ return MTKCAM_IPI_IMG_FMT_UYVY_Y210;
+ case V4L2_PIX_FMT_VYUY10:
+ return MTKCAM_IPI_IMG_FMT_VYUY_Y210;
+ case V4L2_PIX_FMT_MTISP_YUYV10P:
+ return MTKCAM_IPI_IMG_FMT_YUYV_Y210_PACKED;
+ case V4L2_PIX_FMT_MTISP_YVYU10P:
+ return MTKCAM_IPI_IMG_FMT_YVYU_Y210_PACKED;
+ case V4L2_PIX_FMT_MTISP_UYVY10P:
+ return MTKCAM_IPI_IMG_FMT_UYVY_Y210_PACKED;
+ case V4L2_PIX_FMT_MTISP_VYUY10P:
+ return MTKCAM_IPI_IMG_FMT_VYUY_Y210_PACKED;
+ case V4L2_PIX_FMT_NV12_12:
+ return MTKCAM_IPI_IMG_FMT_YUV_P012;
+ case V4L2_PIX_FMT_NV21_12:
+ return MTKCAM_IPI_IMG_FMT_YVU_P012;
+ case V4L2_PIX_FMT_NV16_12:
+ return MTKCAM_IPI_IMG_FMT_YUV_P212;
+ case V4L2_PIX_FMT_NV61_12:
+ return MTKCAM_IPI_IMG_FMT_YVU_P212;
+ case V4L2_PIX_FMT_MTISP_NV12_12P:
+ return MTKCAM_IPI_IMG_FMT_YUV_P012_PACKED;
+ case V4L2_PIX_FMT_MTISP_NV21_12P:
+ return MTKCAM_IPI_IMG_FMT_YVU_P012_PACKED;
+ case V4L2_PIX_FMT_MTISP_NV16_12P:
+ return MTKCAM_IPI_IMG_FMT_YUV_P212_PACKED;
+ case V4L2_PIX_FMT_MTISP_NV61_12P:
+ return MTKCAM_IPI_IMG_FMT_YVU_P212_PACKED;
+ case V4L2_PIX_FMT_SBGGR8:
+ case V4L2_PIX_FMT_SGBRG8:
+ case V4L2_PIX_FMT_SGRBG8:
+ case V4L2_PIX_FMT_SRGGB8:
+ return MTKCAM_IPI_IMG_FMT_BAYER8;
+ case V4L2_PIX_FMT_MTISP_SBGGR8F:
+ case V4L2_PIX_FMT_MTISP_SGBRG8F:
+ case V4L2_PIX_FMT_MTISP_SGRBG8F:
+ case V4L2_PIX_FMT_MTISP_SRGGB8F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER8;
+ case V4L2_PIX_FMT_SBGGR10:
+ case V4L2_PIX_FMT_SGBRG10:
+ case V4L2_PIX_FMT_SGRBG10:
+ case V4L2_PIX_FMT_SRGGB10:
+ return MTKCAM_IPI_IMG_FMT_BAYER10_UNPACKED;
+ case V4L2_PIX_FMT_SBGGR10P:
+ case V4L2_PIX_FMT_SGBRG10P:
+ case V4L2_PIX_FMT_SGRBG10P:
+ case V4L2_PIX_FMT_SRGGB10P:
+ return MTKCAM_IPI_IMG_FMT_BAYER10_MIPI;
+ case V4L2_PIX_FMT_MTISP_SBGGR10:
+ case V4L2_PIX_FMT_MTISP_SGBRG10:
+ case V4L2_PIX_FMT_MTISP_SGRBG10:
+ case V4L2_PIX_FMT_MTISP_SRGGB10:
+ return MTKCAM_IPI_IMG_FMT_BAYER10;
+ case V4L2_PIX_FMT_MTISP_SBGGR10F:
+ case V4L2_PIX_FMT_MTISP_SGBRG10F:
+ case V4L2_PIX_FMT_MTISP_SGRBG10F:
+ case V4L2_PIX_FMT_MTISP_SRGGB10F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER10;
+ case V4L2_PIX_FMT_SBGGR12:
+ case V4L2_PIX_FMT_SGBRG12:
+ case V4L2_PIX_FMT_SGRBG12:
+ case V4L2_PIX_FMT_SRGGB12:
+ return MTKCAM_IPI_IMG_FMT_BAYER12_UNPACKED;
+ case V4L2_PIX_FMT_MTISP_SBGGR12:
+ case V4L2_PIX_FMT_MTISP_SGBRG12:
+ case V4L2_PIX_FMT_MTISP_SGRBG12:
+ case V4L2_PIX_FMT_MTISP_SRGGB12:
+ return MTKCAM_IPI_IMG_FMT_BAYER12;
+ case V4L2_PIX_FMT_MTISP_SBGGR12F:
+ case V4L2_PIX_FMT_MTISP_SGBRG12F:
+ case V4L2_PIX_FMT_MTISP_SGRBG12F:
+ case V4L2_PIX_FMT_MTISP_SRGGB12F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER12;
+ case V4L2_PIX_FMT_SBGGR14:
+ case V4L2_PIX_FMT_SGBRG14:
+ case V4L2_PIX_FMT_SGRBG14:
+ case V4L2_PIX_FMT_SRGGB14:
+ return MTKCAM_IPI_IMG_FMT_BAYER14_UNPACKED;
+ case V4L2_PIX_FMT_MTISP_SBGGR14:
+ case V4L2_PIX_FMT_MTISP_SGBRG14:
+ case V4L2_PIX_FMT_MTISP_SGRBG14:
+ case V4L2_PIX_FMT_MTISP_SRGGB14:
+ return MTKCAM_IPI_IMG_FMT_BAYER14;
+ case V4L2_PIX_FMT_MTISP_SBGGR14F:
+ case V4L2_PIX_FMT_MTISP_SGBRG14F:
+ case V4L2_PIX_FMT_MTISP_SGRBG14F:
+ case V4L2_PIX_FMT_MTISP_SRGGB14F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER14;
+ case V4L2_PIX_FMT_SBGGR16:
+ case V4L2_PIX_FMT_SGBRG16:
+ case V4L2_PIX_FMT_SGRBG16:
+ case V4L2_PIX_FMT_SRGGB16:
+ return MTKCAM_IPI_IMG_FMT_BAYER16;
+ case V4L2_PIX_FMT_MTISP_NV12_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_NV12;
+ case V4L2_PIX_FMT_MTISP_NV21_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_NV21;
+ case V4L2_PIX_FMT_MTISP_NV12_10_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_YUV_P010;
+ case V4L2_PIX_FMT_MTISP_NV21_10_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_YVU_P010;
+ case V4L2_PIX_FMT_MTISP_NV12_12_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_YUV_P012;
+ case V4L2_PIX_FMT_MTISP_NV21_12_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_YVU_P012;
+ case V4L2_PIX_FMT_MTISP_BAYER8_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_BAYER8;
+ case V4L2_PIX_FMT_MTISP_BAYER10_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_BAYER10;
+ case V4L2_PIX_FMT_MTISP_BAYER12_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_BAYER12;
+ case V4L2_PIX_FMT_MTISP_BAYER14_UFBC:
+ return MTKCAM_IPI_IMG_FMT_UFBC_BAYER14;
+ case V4L2_PIX_FMT_MTISP_SGRB8F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER8_3P;
+ case V4L2_PIX_FMT_MTISP_SGRB10F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER10_3P_PACKED;
+ case V4L2_PIX_FMT_MTISP_SGRB12F:
+ return MTKCAM_IPI_IMG_FMT_FG_BAYER12_3P_PACKED;
+ default:
+ return MTKCAM_IPI_IMG_FMT_UNKNOWN;
+ }
+}
+
+static void mtk_cam_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
+ u32 pixelformat, u32 width, u32 height)
+{
+ struct v4l2_plane_pix_format *plane;
+ unsigned int ipi_fmt = mtk_cam_get_img_fmt(pixelformat);
+ u8 pixel_bits = mtk_cam_get_pixel_bits(ipi_fmt);
+ u32 stride;
+ u32 aligned_width;
+ u8 bus_size;
+ u8 i;
+
+ pixfmt->width = width;
+ pixfmt->height = height;
+ pixfmt->pixelformat = pixelformat;
+ plane = &pixfmt->plane_fmt[0];
+ bus_size = mtk_cam_yuv_dma_bus_size(pixel_bits, 0);
+ plane->sizeimage = 0;
+
+ if (is_mtk_format(pixelformat)) {
+ const struct mtk_format_info *info;
+
+ info = mtk_format_info(pixelformat);
+ if (!info)
+ return;
+
+ pixfmt->num_planes = info->mem_planes;
+ if (info->mem_planes == 1) {
+ if (is_yuv_ufo(pixelformat)) {
+ /* UFO format width should align 64 pixel */
+ aligned_width = ALIGN(width, 64);
+ stride = aligned_width * info->bit_r_num / info->bit_r_den;
+
+ if (stride > plane->bytesperline)
+ plane->bytesperline = stride;
+ plane->sizeimage = stride * height;
+ plane->sizeimage += stride * height / 2;
+ plane->sizeimage += ALIGN((aligned_width / 64), 8) * height;
+ plane->sizeimage += ALIGN((aligned_width / 64), 8) * height / 2;
+ } else if (is_raw_ufo(pixelformat)) {
+ /* UFO format width should align 64 pixel */
+ aligned_width = ALIGN(width, 64);
+ stride = aligned_width * info->bit_r_num / info->bit_r_den;
+
+ if (stride > plane->bytesperline)
+ plane->bytesperline = stride;
+ plane->sizeimage = stride * height;
+ plane->sizeimage += ALIGN((aligned_width / 64), 8) * height;
+ } else {
+ /* width should be bus_size align */
+ aligned_width = ALIGN(DIV_ROUND_UP(width
+ * info->bit_r_num, info->bit_r_den), bus_size);
+ stride = aligned_width * info->bpp[0];
+
+ if (stride > plane->bytesperline)
+ plane->bytesperline = stride;
+
+ for (i = 0; i < info->comp_planes; i++) {
+ unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
+ unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
+
+ if (plane->bytesperline > stride &&
+ is_fullg_rb(pixelformat)) {
+ plane->sizeimage +=
+ DIV_ROUND_UP(plane->bytesperline, hdiv)
+ * DIV_ROUND_UP(height, vdiv);
+ } else if (plane->bytesperline > stride &&
+ !is_fullg_rb(pixelformat)) {
+ plane->sizeimage +=
+ plane->bytesperline
+ * DIV_ROUND_UP(height, vdiv);
+ } else {
+ plane->sizeimage += info->bpp[i]
+ * DIV_ROUND_UP(aligned_width, hdiv)
+ * DIV_ROUND_UP(height, vdiv);
+ }
+ }
+ }
+ pr_debug("%s stride %d sizeimage %d\n", __func__,
+ plane->bytesperline, plane->sizeimage);
+ } else {
+ pr_debug("do not support non contiguous mplane\n");
+ }
+ } else {
+ const struct v4l2_format_info *info;
+
+ pr_debug("pixelformat:0x%x sizeimage:%d\n",
+ pixelformat, plane->sizeimage);
+ info = v4l2_format_info(pixelformat);
+ if (!info)
+ return;
+
+ pixfmt->num_planes = info->mem_planes;
+ if (info->mem_planes == 1) {
+ aligned_width = ALIGN(width, bus_size);
+ stride = aligned_width * info->bpp[0];
+ if (stride > plane->bytesperline)
+ plane->bytesperline = stride;
+
+ for (i = 0; i < info->comp_planes; i++) {
+ unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
+ unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
+
+ plane->sizeimage += info->bpp[i]
+ * DIV_ROUND_UP(aligned_width, hdiv)
+ * DIV_ROUND_UP(height, vdiv);
+ }
+ pr_debug("%s stride %d sizeimage %d\n", __func__,
+ plane->bytesperline, plane->sizeimage);
+ } else {
+ pr_warn("do not support non contiguous mplane\n");
+ }
+ }
+}
+
+static void cal_image_pix_mp(unsigned int node_id,
+ struct v4l2_pix_format_mplane *mp,
+ unsigned int pixel_mode)
+{
+ unsigned int ipi_fmt = mtk_cam_get_img_fmt(mp->pixelformat);
+ unsigned int width = mp->width;
+ unsigned int height = mp->height;
+ unsigned int stride, i;
+
+ pr_debug("fmt:0x%x ipi_fmt:%d\n", mp->pixelformat, ipi_fmt);
+ switch (ipi_fmt) {
+ case MTKCAM_IPI_IMG_FMT_BAYER8:
+ case MTKCAM_IPI_IMG_FMT_BAYER10:
+ case MTKCAM_IPI_IMG_FMT_BAYER12:
+ case MTKCAM_IPI_IMG_FMT_BAYER14:
+ case MTKCAM_IPI_IMG_FMT_BAYER16:
+ case MTKCAM_IPI_IMG_FMT_BAYER10_MIPI:
+ case MTKCAM_IPI_IMG_FMT_BAYER10_UNPACKED:
+ case MTKCAM_IPI_IMG_FMT_BAYER12_UNPACKED:
+ case MTKCAM_IPI_IMG_FMT_BAYER14_UNPACKED:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER8:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER10:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER12:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER14:
+ stride = mtk_cam_dmao_xsize(width, ipi_fmt, pixel_mode);
+ for (i = 0; i < mp->num_planes; i++) {
+ if (stride > mp->plane_fmt[i].bytesperline)
+ mp->plane_fmt[i].bytesperline = stride;
+ mp->plane_fmt[i].sizeimage =
+ mp->plane_fmt[i].bytesperline * height;
+ }
+ break;
+ case MTKCAM_IPI_IMG_FMT_YUYV:
+ case MTKCAM_IPI_IMG_FMT_YVYU:
+ case MTKCAM_IPI_IMG_FMT_UYVY:
+ case MTKCAM_IPI_IMG_FMT_VYUY:
+ case MTKCAM_IPI_IMG_FMT_YUV_422_2P:
+ case MTKCAM_IPI_IMG_FMT_YVU_422_2P:
+ case MTKCAM_IPI_IMG_FMT_YUV_422_3P:
+ case MTKCAM_IPI_IMG_FMT_YVU_422_3P:
+ case MTKCAM_IPI_IMG_FMT_YUV_420_2P:
+ case MTKCAM_IPI_IMG_FMT_YVU_420_2P:
+ case MTKCAM_IPI_IMG_FMT_YUV_420_3P:
+ case MTKCAM_IPI_IMG_FMT_YVU_420_3P:
+ case MTKCAM_IPI_IMG_FMT_Y8:
+ case MTKCAM_IPI_IMG_FMT_YUYV_Y210:
+ case MTKCAM_IPI_IMG_FMT_YVYU_Y210:
+ case MTKCAM_IPI_IMG_FMT_UYVY_Y210:
+ case MTKCAM_IPI_IMG_FMT_VYUY_Y210:
+ case MTKCAM_IPI_IMG_FMT_YUYV_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVYU_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_UYVY_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_VYUY_Y210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YUV_P210:
+ case MTKCAM_IPI_IMG_FMT_YVU_P210:
+ case MTKCAM_IPI_IMG_FMT_YUV_P010:
+ case MTKCAM_IPI_IMG_FMT_YVU_P010:
+ case MTKCAM_IPI_IMG_FMT_YUV_P210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P210_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YUV_P010_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P010_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YUV_P212:
+ case MTKCAM_IPI_IMG_FMT_YVU_P212:
+ case MTKCAM_IPI_IMG_FMT_YUV_P012:
+ case MTKCAM_IPI_IMG_FMT_YVU_P012:
+ case MTKCAM_IPI_IMG_FMT_YUV_P212_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P212_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YUV_P012_PACKED:
+ case MTKCAM_IPI_IMG_FMT_YVU_P012_PACKED:
+ case MTKCAM_IPI_IMG_FMT_UFBC_NV12:
+ case MTKCAM_IPI_IMG_FMT_UFBC_NV21:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YUV_P010:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YVU_P010:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YUV_P012:
+ case MTKCAM_IPI_IMG_FMT_UFBC_YVU_P012:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER8:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER10:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER12:
+ case MTKCAM_IPI_IMG_FMT_UFBC_BAYER14:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER8_3P:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER10_3P_PACKED:
+ case MTKCAM_IPI_IMG_FMT_FG_BAYER12_3P_PACKED:
+ mtk_cam_fill_pixfmt_mp(mp, mp->pixelformat, width, height);
+ break;
+ default:
+ break;
+ }
+}
+
+static int mtk_video_init_format(struct mtk_cam_video_device *video)
+{
+ struct mtk_cam_dev_node_desc *desc = &video->desc;
+ struct v4l2_format *active = &video->active_fmt;
+ const struct v4l2_format *default_fmt =
+ &desc->fmts[desc->default_fmt_idx].vfmt;
+
+ active->type = desc->buf_type;
+
+ if (!desc->image) {
+ active->fmt.meta.dataformat = default_fmt->fmt.meta.dataformat;
+ active->fmt.meta.buffersize = default_fmt->fmt.meta.buffersize;
+ return 0;
+ }
+
+ active->fmt.pix_mp.pixelformat = default_fmt->fmt.pix_mp.pixelformat;
+ active->fmt.pix_mp.width = default_fmt->fmt.pix_mp.width;
+ active->fmt.pix_mp.height = default_fmt->fmt.pix_mp.height;
+ active->fmt.pix_mp.num_planes = default_fmt->fmt.pix_mp.num_planes;
+
+ cal_image_pix_mp(desc->id, &active->fmt.pix_mp, 0);
+
+ /* set init one-plane */
+ active->fmt.pix_mp.num_planes = 1;
+ active->fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+ active->fmt.pix_mp.field = V4L2_FIELD_NONE;
+ active->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ active->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ active->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+ return 0;
+}
+
+int mtk_cam_video_register(struct mtk_cam_video_device *video,
+ struct v4l2_device *v4l2_dev)
+{
+ struct mtk_cam_device *cam =
+ container_of(v4l2_dev, struct mtk_cam_device, v4l2_dev);
+ struct media_pad *pad = &video->pad;
+ struct video_device *vdev = &video->vdev;
+ struct vb2_queue *q = &video->vb2_q;
+ unsigned int output = V4L2_TYPE_IS_OUTPUT(video->desc.buf_type);
+ int ret;
+
+ if (video->desc.link_flags & MEDIA_LNK_FL_ENABLED)
+ video->enabled = true;
+ else
+ video->enabled = false;
+
+ mutex_init(&video->q_lock);
+
+ /* initialize vb2_queue */
+ q->type = video->desc.buf_type;
+ q->io_modes = VB2_MMAP | VB2_DMABUF;
+
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+
+ if (video->desc.smem_alloc) {
+ q->bidirectional = 1;
+ /* reserved memory */
+ q->dev = cam->smem_dev;
+ } else if (is_yuv_node(video->desc.id)) {
+ q->dev = cam->raw.yuvs[0];
+ } else {
+ q->dev = cam->raw.devs[0];
+ }
+
+ q->supports_requests = true;
+ q->lock = &video->q_lock;
+ q->ops = &mtk_cam_vb2_ops;
+ q->mem_ops = &vb2_dma_contig_memops;
+ q->drv_priv = cam;
+ q->buf_struct_size = sizeof(struct mtk_cam_buffer);
+
+ if (output)
+ q->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_EOF;
+ else
+ q->timestamp_flags |= V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
+
+ /* No minimum buffers limitation */
+ q->min_queued_buffers = 0;
+
+ ret = vb2_queue_init(q);
+ if (ret < 0) {
+ dev_info(v4l2_dev->dev, "Failed to init vb2 queue: %d\n", ret);
+ goto error_vb2_init;
+ }
+
+ pad->flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&vdev->entity, 1, pad);
+ if (ret < 0) {
+ dev_info(v4l2_dev->dev, "Failed to init video entity: %d\n", ret);
+ goto error_media_init;
+ }
+
+ ret = mtk_video_init_format(video);
+ if (ret < 0) {
+ dev_info(v4l2_dev->dev, "Failed to init format: %d\n", ret);
+ goto error_video_register;
+ }
+
+ vdev->entity.function = MEDIA_ENT_F_IO_V4L;
+ vdev->entity.ops = NULL;
+ vdev->fops = &mtk_cam_v4l2_fops;
+ vdev->device_caps = video->desc.cap | V4L2_CAP_STREAMING;
+ vdev->v4l2_dev = v4l2_dev;
+
+ vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX;
+ vdev->queue = &video->vb2_q;
+ vdev->ioctl_ops = video->desc.ioctl_ops;
+ vdev->release = video_device_release_empty;
+ /* share q_lock */
+ vdev->lock = &video->q_lock;
+ strscpy(vdev->name, video->desc.name, sizeof(vdev->name));
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret < 0) {
+ dev_info(v4l2_dev->dev, "Failed to register video device: %d\n",
+ ret);
+ goto error_video_register;
+ }
+ video_set_drvdata(vdev, cam);
+
+ dev_dbg(v4l2_dev->dev, "registered vdev:%d:%s\n",
+ video->desc.id, vdev->name);
+
+ return 0;
+
+error_video_register:
+ media_entity_cleanup(&vdev->entity);
+error_media_init:
+ vb2_queue_release(&video->vb2_q);
+error_vb2_init:
+ mutex_destroy(&video->q_lock);
+
+ return ret;
+}
+
+void mtk_cam_video_unregister(struct mtk_cam_video_device *video)
+{
+ video_unregister_device(&video->vdev);
+ vb2_queue_release(&video->vb2_q);
+ media_entity_cleanup(&video->vdev.entity);
+ mutex_destroy(&video->q_lock);
+}
+
+const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format)
+{
+ unsigned int i;
+ const struct v4l2_format *fmt;
+
+ for (i = 0; i < desc->num_fmts; i++) {
+ fmt = &desc->fmts[i].vfmt;
+ if (fmt->fmt.pix_mp.pixelformat == format)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ struct mtk_cam_device *cam = video_drvdata(file);
+
+ strscpy(cap->driver, dev_driver_string(cam->dev), sizeof(cap->driver));
+ strscpy(cap->card, dev_driver_string(cam->dev), sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+ dev_name(cam->dev));
+
+ return 0;
+}
+
+int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+ struct v4l2_frmsizeenum *sizes)
+{
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(filp);
+ const struct v4l2_format *dev_fmt;
+
+ dev_fmt = mtk_cam_dev_find_fmt(&node->desc, sizes->pixel_format);
+ if (!dev_fmt || sizes->index)
+ return -EINVAL;
+
+ sizes->type = node->desc.frmsizes->type;
+ memcpy(&sizes->stepwise, &node->desc.frmsizes->stepwise,
+ sizeof(sizes->stepwise));
+ return 0;
+}
+
+int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+ if (f->index >= node->desc.num_fmts)
+ return -EINVAL;
+
+ f->pixelformat = node->desc.fmts[f->index].vfmt.fmt.pix_mp.pixelformat;
+ f->flags = 0;
+ /* fmt description is filled in v4l_fill_fmtdesc */
+
+ return 0;
+}
+
+int mtk_cam_vidioc_g_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+ f->fmt = node->active_fmt.fmt;
+
+ return 0;
+}
+
+int mtk_cam_vidioc_s_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_cam_device *cam = video_drvdata(file);
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+ struct mtk_raw_pipeline *raw_pipeline;
+ int raw_feature = 0;
+
+ raw_pipeline = mtk_cam_dev_get_raw_pipeline(cam, node->uid.pipe_id);
+
+ if (!vb2_is_busy(node->vdev.queue)) {
+ /* Get the valid format */
+ if (raw_pipeline)
+ raw_feature = raw_pipeline->user_res.raw_res.feature;
+
+ mtk_cam_video_set_fmt(node, f, raw_feature);
+
+ /* Configure to video device */
+ node->active_fmt = *f;
+ return 0;
+ }
+
+ dev_info(cam->dev,
+ "%s:pipe(%d):%s:Cannot change format while streaming\n",
+ __func__, node->uid.pipe_id, node->desc.name);
+
+ return -EBUSY;
+}
+
+int mtk_cam_video_set_fmt(struct mtk_cam_video_device *node,
+ struct v4l2_format *f, int raw_feature)
+{
+ struct mtk_cam_device *cam = video_get_drvdata(&node->vdev);
+ const struct v4l2_format *dev_fmt;
+ struct v4l2_format try_fmt;
+ s32 i;
+
+ dev_dbg(cam->dev,
+ "%s:pipe(%d):%s:feature(0x%x)\n",
+ __func__, node->uid.pipe_id, node->desc.name, raw_feature);
+
+ memset(&try_fmt, 0, sizeof(try_fmt));
+ try_fmt.type = f->type;
+
+ /* Validate pixelformat */
+ dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
+ if (!dev_fmt) {
+ dev_dbg(cam->dev, "unknown fmt:%d\n",
+ f->fmt.pix_mp.pixelformat);
+ dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx].vfmt;
+ }
+ try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
+
+ /* Validate image width & height range */
+ try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
+ IMG_MIN_WIDTH, IMG_MAX_WIDTH);
+ try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
+ IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
+ /* 4 bytes alignment for width */
+ /* width and stride should align bus_size */
+ try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, IMG_PIX_ALIGN);
+ try_fmt.fmt.pix_mp.num_planes = 1;
+
+ for (i = 0 ; i < try_fmt.fmt.pix_mp.num_planes ; i++)
+ try_fmt.fmt.pix_mp.plane_fmt[i].bytesperline =
+ f->fmt.pix_mp.plane_fmt[i].bytesperline;
+
+ /* bytesperline & sizeimage calculation */
+ cal_image_pix_mp(node->desc.id, &try_fmt.fmt.pix_mp, 0);
+
+ /* Constant format fields */
+ try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
+ try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
+ try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+ try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
+
+ *f = try_fmt;
+
+ return 0;
+}
+
+int mtk_cam_vidioc_try_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+ mtk_cam_video_set_fmt(node, f, 0);
+
+ return 0;
+}
+
+int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+
+ if (f->index)
+ return -EINVAL;
+
+ f->pixelformat = node->active_fmt.fmt.meta.dataformat;
+ f->flags = 0;
+ /*fmt description is filled in v4l_fill_fmtdesc */
+
+ return 0;
+}
+
+int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct mtk_cam_device *cam = video_drvdata(file);
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+ struct mtk_cam_dev_node_desc *desc = &node->desc;
+ const struct v4l2_format *default_fmt =
+ &desc->fmts[desc->default_fmt_idx].vfmt;
+ struct mtk_raw_pde_config *pde_cfg;
+ struct mtk_cam_pde_info *pde_info;
+
+ if (node->desc.dma_port == MTKCAM_ISP_META_STATS_CFG) {
+ pde_cfg = &cam->raw.pipelines[node->uid.pipe_id].pde_config;
+ pde_info = &pde_cfg->pde_info;
+ if (pde_info->pd_table_offset) {
+ node->active_fmt.fmt.meta.buffersize =
+ default_fmt->fmt.meta.buffersize +
+ pde_info->pdi_max_size;
+
+ dev_dbg(cam->dev, "PDE: node(%d), enlarge meta size(%u)",
+ node->desc.dma_port,
+ node->active_fmt.fmt.meta.buffersize);
+ }
+ }
+ if (node->desc.dma_port == MTKCAM_ISP_META_STATS_0) {
+ pde_cfg = &cam->raw.pipelines[node->uid.pipe_id].pde_config;
+ pde_info = &pde_cfg->pde_info;
+ if (pde_info->pd_table_offset) {
+ node->active_fmt.fmt.meta.buffersize =
+ default_fmt->fmt.meta.buffersize +
+ pde_info->pdo_max_size;
+
+ dev_dbg(cam->dev, "PDE: node(%d), enlarge meta size(%u)",
+ node->desc.dma_port,
+ node->active_fmt.fmt.meta.buffersize);
+ }
+ }
+ f->fmt.meta.dataformat = node->active_fmt.fmt.meta.dataformat;
+ f->fmt.meta.buffersize = node->active_fmt.fmt.meta.buffersize;
+
+ return 0;
+}
+
+int mtk_cam_vidioc_s_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ struct mtk_cam_device *cam = video_drvdata(file);
+ struct mtk_cam_video_device *node = file_to_mtk_cam_node(file);
+ struct mtk_raw_pipeline *raw_pipeline;
+
+ raw_pipeline = mtk_cam_dev_get_raw_pipeline(cam, node->uid.pipe_id);
+ if (raw_pipeline) {
+ node->active_crop = *s;
+
+ dev_dbg(raw_pipeline->subdev.v4l2_dev->dev,
+ "%s:%s:%s:Set selection (%d,%d,%d,%d)\n",
+ __func__, raw_pipeline->subdev.name, node->desc.name,
+ s->r.left, s->r.top, s->r.width, s->r.height);
+ }
+
+ return 0;
+}
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.h b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.h
new file mode 100755
index 000000000000..38b89cc2b81e
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-video.h
@@ -0,0 +1,223 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ */
+
+#ifndef __MTK_CAM_VIDEO_H
+#define __MTK_CAM_VIDEO_H
+
+#include <linux/types.h>
+#include <linux/videodev2.h>
+#include <media/media-entity.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-dev.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "mtk_cam-ipi.h"
+
+#define MAX_PLANE_NUM 3
+
+struct mtk_cam_resource;
+struct mtk_raw_pde_config;
+
+typedef int (*set_pad_fmt_func_t)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_mbus_framefmt *sink_fmt,
+ struct mtk_cam_resource *res,
+ int pad, int which);
+
+typedef int (*set_pad_selection_func_t)(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_mbus_framefmt *sink_fmt,
+ struct mtk_cam_resource *res,
+ int pad, int which);
+
+/*For state analysis and controlling for request*/
+enum MTK_BUF_STATE {
+ E_BUF_STATE_QUEUED = 0x0,
+ E_BUF_STATE_COMPOSED,
+ E_BUF_STATE_CQ,
+ E_BUF_STATE_OUTER,
+};
+
+struct mtk_buf_state {
+ enum MTK_BUF_STATE estate;
+ struct list_head list;
+};
+
+/**
+ * struct mtk_cam_buffer - MTK camera device buffer.
+ *
+ * @vbb: Embedded struct vb2_v4l2_buffer.
+ * @list: List entry for the buffer queue
+ * @daddr: The DMA address of this buffer.
+ * @scp_addr: The SCP address of this buffer which
+ * is only supported for meta input node.
+ * @state: The camera buffer status.
+ */
+struct mtk_cam_buffer {
+ struct vb2_v4l2_buffer vbb;
+ struct list_head list;
+
+ dma_addr_t daddr;
+ dma_addr_t scp_addr;
+ struct mtk_buf_state state;
+};
+
+struct mtk_cam_format_desc {
+ struct v4l2_format vfmt;
+ struct v4l2_mbus_framefmt pfmt;
+};
+
+struct mtk_cam_pad_ops {
+ set_pad_fmt_func_t set_pad_fmt;
+ set_pad_selection_func_t set_pad_selection;
+};
+
+/**
+ * struct mtk_cam_dev_node_desc - MTK camera device node descriptor
+ *
+ * @id: id of the node
+ * @name: name of the node
+ * @cap: supported V4L2 capabilities
+ * @buf_type: supported V4L2 buffer type
+ * @dma_port: the dma ports associated to the node
+ * @link_flags: default media link flags
+ * @smem_alloc: using the smem_dev as alloc device or not
+ * @need_cache_sync_on_prepare: do cache sync at buf_prepare (userspace sync)
+ * @need_cache_sync_on_finish: do cache sync at buf_finish (userspace sync)
+ * @image: true for image node, false for meta node
+ * @num_fmts: the number of supported node formats
+ * @default_fmt_idx: default format of this node
+ * @max_buf_count: maximum VB2 buffer count
+ * @ioctl_ops: mapped to v4l2_ioctl_ops
+ * @fmts: supported format
+ * @frmsizes: supported V4L2 frame size number
+ * @pad_ops: set and select pad configurations and formats
+ */
+struct mtk_cam_dev_node_desc {
+ u8 id;
+ const char *name;
+ u32 cap;
+ u32 buf_type;
+ u32 dma_port;
+ u32 link_flags;
+ u8 smem_alloc:1;
+ u8 image:1;
+ u8 num_fmts;
+ u8 default_fmt_idx;
+ u8 max_buf_count;
+ const struct v4l2_ioctl_ops *ioctl_ops;
+ const struct mtk_cam_format_desc *fmts;
+ const struct v4l2_frmsizeenum *frmsizes;
+ struct mtk_cam_pad_ops *pad_ops;
+};
+
+/**
+ * struct mtk_cam_video_device - MediaTek video device structure.
+ */
+struct mtk_cam_video_device {
+ struct mtkcam_ipi_uid uid;
+ struct mtk_cam_dev_node_desc desc;
+ unsigned int enabled;
+
+ struct vb2_queue vb2_q;
+ struct video_device vdev;
+ struct media_pad pad;
+ struct v4l2_format active_fmt;
+ /* use first 4 elements of reserved field of v4l2_pix_format_mplane as request fd */
+ struct v4l2_format pending_fmt;
+ /* use first elements of reserved field of v4l2_selection as request fd*/
+ struct v4l2_selection active_crop;
+ /* Serializes vb2 queue and video device operations */
+ struct mutex q_lock;
+ int streaming_id;
+
+ /* cached ctx info */
+ struct mtk_cam_ctx *ctx;
+};
+
+struct mtk_format_info {
+ u32 format;
+ u8 mem_planes;
+ u8 comp_planes;
+ u8 bpp[4];
+ u8 hdiv;
+ u8 vdiv;
+ u8 bit_r_num; /* numerator of bit ratio */
+ u8 bit_r_den; /* denominator of bit ratio */
+};
+
+int mtk_cam_video_register(struct mtk_cam_video_device *video,
+ struct v4l2_device *v4l2_dev);
+void mtk_cam_video_unregister(struct mtk_cam_video_device *video);
+
+static inline struct mtk_cam_video_device *
+file_to_mtk_cam_node(struct file *__file)
+{
+ return container_of(video_devdata(__file), struct mtk_cam_video_device, vdev);
+}
+
+static inline struct mtk_cam_buffer *
+mtk_cam_vb2_buf_to_dev_buf(struct vb2_buffer *__vb)
+{
+ return container_of(__vb, struct mtk_cam_buffer, vbb.vb2_buf);
+}
+
+static inline struct mtk_cam_video_device *
+mtk_cam_vbq_to_vdev(struct vb2_queue *__vq)
+{
+ return container_of(__vq, struct mtk_cam_video_device, vb2_q);
+}
+
+const struct v4l2_format *
+mtk_cam_dev_find_fmt(struct mtk_cam_dev_node_desc *desc, u32 format);
+
+int mtk_cam_vidioc_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap);
+
+int mtk_cam_vidioc_enum_framesizes(struct file *filp, void *priv,
+ struct v4l2_frmsizeenum *sizes);
+
+int mtk_cam_vidioc_enum_fmt(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f);
+
+int mtk_cam_vidioc_g_fmt(struct file *file, void *fh, struct v4l2_format *f);
+
+int mtk_cam_vidioc_s_fmt(struct file *file, void *fh, struct v4l2_format *f);
+
+int mtk_cam_vidioc_try_fmt(struct file *file, void *fh, struct v4l2_format *f);
+
+int mtk_cam_vidioc_meta_enum_fmt(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f);
+
+int mtk_cam_vidioc_s_selection(struct file *file, void *fh,
+ struct v4l2_selection *s);
+
+int mtk_cam_vidioc_g_meta_fmt(struct file *file, void *fh,
+ struct v4l2_format *f);
+
+/* Utility functions to convert format enum */
+unsigned int mtk_cam_get_sensor_pixel_id(unsigned int fmt);
+
+unsigned int mtk_cam_get_sensor_fmt(unsigned int fmt);
+
+unsigned int mtk_cam_get_pixel_bits(unsigned int pix_fmt);
+
+unsigned int mtk_cam_get_img_fmt(unsigned int fourcc);
+
+int mtk_cam_video_set_fmt(struct mtk_cam_video_device *node,
+ struct v4l2_format *f, int feature);
+
+int is_mtk_format(u32 pixelformat);
+
+int is_yuv_ufo(u32 pixelformat);
+
+int is_raw_ufo(u32 pixelformat);
+
+int is_fullg_rb(u32 pixelformat);
+
+const struct mtk_format_info *mtk_format_info(u32 format);
+
+#endif /*__MTK_CAM_VIDEO_H */
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 11/13] media: platform: mediatek: add isp_7x build config
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (5 preceding siblings ...)
2025-07-07 1:31 ` [PATCH v2 10/13] media: platform: mediatek: add isp_7x video ops shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-07 1:31 ` [PATCH v2 12/13] uapi: linux: add mediatek isp_7x camsys user api shangyao lin
` (6 subsequent siblings)
13 siblings, 0 replies; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Add Kconfig and Makefile to include MTK ISP CAMSYS, integrating the driver into
the kernel build and configuration process.
Question for CK:
Hi CK,
Thank you for your review and comments.
Just to confirm, do you mean that when I add mtk_cam-seninf-route.c and
mtk_cam-seninf-drv.c in patch [3/10], I should also create or update the
corresponding Makefile to ensure these two .c files are compiled as part of the
driver? Is my understanding correct?
Thank you for your clarification.
Best regards,
Shangyao
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
drivers/media/platform/mediatek/Kconfig | 1 +
drivers/media/platform/mediatek/Makefile | 2 ++
drivers/media/platform/mediatek/isp/Kconfig | 21 +++++++++++++++++++
.../platform/mediatek/isp/isp_7x/Makefile | 7 +++++++
.../mediatek/isp/isp_7x/camsys/Makefile | 14 +++++++++++++
5 files changed, 45 insertions(+)
mode change 100644 => 100755 drivers/media/platform/mediatek/Kconfig
mode change 100644 => 100755 drivers/media/platform/mediatek/Makefile
create mode 100755 drivers/media/platform/mediatek/isp/Kconfig
create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/Makefile
create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/Makefile
diff --git a/drivers/media/platform/mediatek/Kconfig b/drivers/media/platform/mediatek/Kconfig
old mode 100644
new mode 100755
index 84104e2cd024..a405d5701329
--- a/drivers/media/platform/mediatek/Kconfig
+++ b/drivers/media/platform/mediatek/Kconfig
@@ -2,6 +2,7 @@
comment "Mediatek media platform drivers"
+source "drivers/media/platform/mediatek/isp/Kconfig"
source "drivers/media/platform/mediatek/jpeg/Kconfig"
source "drivers/media/platform/mediatek/mdp/Kconfig"
source "drivers/media/platform/mediatek/vcodec/Kconfig"
diff --git a/drivers/media/platform/mediatek/Makefile b/drivers/media/platform/mediatek/Makefile
old mode 100644
new mode 100755
index 38e6ba917fe5..74164df8c68d
--- a/drivers/media/platform/mediatek/Makefile
+++ b/drivers/media/platform/mediatek/Makefile
@@ -1,6 +1,8 @@
# SPDX-License-Identifier: GPL-2.0-only
+
obj-y += jpeg/
obj-y += mdp/
obj-y += vcodec/
obj-y += vpu/
obj-y += mdp3/
+obj-y += isp/isp_7x/
diff --git a/drivers/media/platform/mediatek/isp/Kconfig b/drivers/media/platform/mediatek/isp/Kconfig
new file mode 100755
index 000000000000..ce4dd21e3e79
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/Kconfig
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config VIDEO_MTK_ISP_71_CAMSYS
+ tristate "MediaTek ISP 7.1 camsys driver"
+ depends on ARCH_MEDIATEK
+ select V4L2_FWNODE
+ select VIDEO_V4L2_SUBDEV_API
+ select VIDEOBUF2_DMA_CONTIG
+ select RPMSG_MTK_CCD
+ select MTKCCD_REMOTEPROC
+ select MTK_SCP
+
+ default n
+ help
+ Camsys driver controls 3A (auto-focus, exposure,
+ and white balance) with tuning feature and outputs
+ the captured image buffers in MediaTek's ISP7 system.
+
+ Choose y if you want to use MediaTek SoCs to create image
+ captured application such as video recording and still image
+ capturing.
\ No newline at end of file
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/Makefile
new file mode 100755
index 000000000000..be70346d224b
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+subdir-ccflags-y += -Werror
+
+subdir-ccflags-$(CONFIG_VIDEO_MTK_ISP_71_CAMSYS) += -DISP7_1
+
+obj-$(CONFIG_VIDEO_MTK_ISP_71_CAMSYS) += camsys/
\ No newline at end of file
diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/Makefile b/drivers/media/platform/mediatek/isp/isp_7x/camsys/Makefile
new file mode 100755
index 000000000000..700ac2a85ac4
--- /dev/null
+++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/Makefile
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2022 MediaTek Inc.
+
+mtk-cam-isp-objs := mtk_cam.o mtk_cam-raw.o mtk_cam-pool.o \
+ mtk_cam-video.o mtk_cam-ctrl.o \
+ mtk_cam-seninf-route.o mtk_cam-seninf-drv.o \
+ mtk_cam-timesync.o
+
+mtk-cam-plat-util-objs := mtk_cam-plat-util.o
+
+include $(src)/mtk_csi_phy_2_0/Makefile
+
+obj-$(CONFIG_VIDEO_MTK_ISP_71_CAMSYS) += mtk-cam-plat-util.o
+obj-$(CONFIG_VIDEO_MTK_ISP_71_CAMSYS) += mtk-cam-isp.o
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 12/13] uapi: linux: add mediatek isp_7x camsys user api
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (6 preceding siblings ...)
2025-07-07 1:31 ` [PATCH v2 11/13] media: platform: mediatek: add isp_7x build config shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-08 1:34 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 13/13] media: uapi: mediatek: document ISP7x camera system and user controls shangyao lin
` (5 subsequent siblings)
13 siblings, 1 reply; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
Add UAPI for the MediaTek ISP platform, providing user-space interfaces for the
new CAMSYS driver.
Changes in v2:
- Moved V4L2 pixel format definitions to include/uapi/linux/videodev2.h
Notes:
Hi Laurent Pinchart and CK,
Thank you very much for your detailed feedback and suggestions.
I would like to let you know that in the next patch, I have added an initial
version of an .rst document describing the MTK ISP CAMSYS controls. Most of
these ISP CAMSYS controls are intended for other advanced features. For normal
main-stream use cases, only controls such as V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC
are actually used.
Would you prefer that we remove the user space interfaces for these advanced
features for now, and only add them back in a future patch series dedicated to
advanced functionality? I am happy to follow your recommendation on this.
Additionally, I have also written an initial version of a brief .rst document
providing a high-level description of the hardware. If you think there is
anything else that should be added or clarified, please kindly let me know.
Thank you again for your valuable feedback!
Best regards,
Shangyao
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
include/uapi/linux/mtkisp_camsys.h | 132 +++++++++++++++++++++++++++++
include/uapi/linux/videodev2.h | 87 +++++++++++++++++++
2 files changed, 219 insertions(+)
create mode 100755 include/uapi/linux/mtkisp_camsys.h
mode change 100644 => 100755 include/uapi/linux/videodev2.h
diff --git a/include/uapi/linux/mtkisp_camsys.h b/include/uapi/linux/mtkisp_camsys.h
new file mode 100755
index 000000000000..4d563c12077b
--- /dev/null
+++ b/include/uapi/linux/mtkisp_camsys.h
@@ -0,0 +1,132 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * MediaTek ISP camsys User space API
+ *
+ * Copyright (c) 2024 MediaTek Inc.
+ */
+
+#ifndef _MTKISP_CAMSYS_USER_H
+#define _MTKISP_CAMSYS_USER_H
+
+#include <linux/videodev2.h>
+#include <linux/v4l2-controls.h>
+
+#define V4L2_BUF_FLAG_TIMESTAMP_COPY 0x00004000
+#define V4L2_BUF_FLAG_TIMESTAMP_BOOTIME 0x00008000
+
+/* MTK ISP camsys events */
+#define V4L2_EVENT_REQUEST_DRAINED (V4L2_EVENT_PRIVATE_START + 1)
+#define V4L2_EVENT_REQUEST_DUMPED (V4L2_EVENT_PRIVATE_START + 2)
+
+/* The base for the mediatek camsys driver controls */
+/* We reserve 48 controls for this driver. */
+#define V4L2_CID_USER_MTK_CAM_BASE (V4L2_CID_USER_BASE + 0x10d0)
+
+/* MTK ISP camsys controls */
+#define V4L2_CID_MTK_CAM_USED_ENGINE_LIMIT (V4L2_CID_USER_MTK_CAM_BASE + 1)
+#define V4L2_CID_MTK_CAM_BIN_LIMIT (V4L2_CID_USER_MTK_CAM_BASE + 2)
+#define V4L2_CID_MTK_CAM_FRZ_LIMIT (V4L2_CID_USER_MTK_CAM_BASE + 3)
+#define V4L2_CID_MTK_CAM_RESOURCE_PLAN_POLICY (V4L2_CID_USER_MTK_CAM_BASE + 4)
+#define V4L2_CID_MTK_CAM_USED_ENGINE (V4L2_CID_USER_MTK_CAM_BASE + 5)
+#define V4L2_CID_MTK_CAM_BIN (V4L2_CID_USER_MTK_CAM_BASE + 6)
+#define V4L2_CID_MTK_CAM_FRZ (V4L2_CID_USER_MTK_CAM_BASE + 7)
+#define V4L2_CID_MTK_CAM_USED_ENGINE_TRY (V4L2_CID_USER_MTK_CAM_BASE + 8)
+#define V4L2_CID_MTK_CAM_BIN_TRY (V4L2_CID_USER_MTK_CAM_BASE + 9)
+#define V4L2_CID_MTK_CAM_FRZ_TRY (V4L2_CID_USER_MTK_CAM_BASE + 10)
+#define V4L2_CID_MTK_CAM_PIXEL_RATE (V4L2_CID_USER_MTK_CAM_BASE + 11)
+#define V4L2_CID_MTK_CAM_FEATURE (V4L2_CID_USER_MTK_CAM_BASE + 12)
+#define V4L2_CID_MTK_CAM_SYNC_ID (V4L2_CID_USER_MTK_CAM_BASE + 13)
+#define V4L2_CID_MTK_CAM_RAW_PATH_SELECT (V4L2_CID_USER_MTK_CAM_BASE + 14)
+#define V4L2_CID_MTK_CAM_HSF_EN (V4L2_CID_USER_MTK_CAM_BASE + 15)
+#define V4L2_CID_MTK_CAM_PDE_INFO (V4L2_CID_USER_MTK_CAM_BASE + 16)
+#define V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC (V4L2_CID_USER_MTK_CAM_BASE + 18)
+#define V4L2_CID_MTK_CAM_TG_FLASH_CFG (V4L2_CID_USER_MTK_CAM_BASE + 19)
+#define V4L2_CID_MTK_CAM_RAW_RESOURCE_UPDATE (V4L2_CID_USER_MTK_CAM_BASE + 20)
+#define V4L2_CID_MTK_CAM_CAMSYS_HW_MODE (V4L2_CID_USER_MTK_CAM_BASE + 21)
+
+/*
+ * struct mtk_cam_resource_sensor - sensor resoruces for format negotiation
+ *
+ */
+struct mtk_cam_resource_sensor {
+ struct v4l2_fract interval;
+ __u32 hblank;
+ __u32 vblank;
+ __u64 pixel_rate;
+ __u64 cust_pixel_rate;
+};
+
+/*
+ * struct mtk_cam_resource_raw - MTK camsys raw resoruces for format negotiation
+ *
+ * @feature: value of V4L2_CID_MTK_CAM_FEATURE the user want to check the
+ * resource with. If it is used in set CTRL, we will apply the value
+ * to V4L2_CID_MTK_CAM_FEATURE ctrl directly.
+ * @strategy: indicate the order of multiple raws, binning or DVFS to be selected
+ * when doing format negotiation of raw's source pads (output pads).
+ * Please pass MTK_CAM_RESOURCE_DEFAULT if you want camsys driver to
+ * determine it.
+ * @raw_max: indicate the max number of raw to be used for the raw pipeline.
+ * Please pass MTK_CAM_RESOURCE_DEFAULT if you want camsys driver to
+ * determine it.
+ * @raw_min: indicate the max number of raw to be used for the raw pipeline.
+ * Please pass MTK_CAM_RESOURCE_DEFAULT if you want camsys driver to
+ * determine it.
+ * @raw_used: The number of raw used. The used don't need to writ this failed,
+ * the driver always updates the field.
+ * @bin: indicate if the driver should enable the bining or not. The driver
+ * update the field depanding the hardware supporting status. Please pass
+ * MTK_CAM_RESOURCE_DEFAULT if you want camsys driver to determine it.
+ * @path_sel: indicate the user selected raw path. The driver
+ * update the field depanding the hardware supporting status. Please
+ * pass MTK_CAM_RESOURCE_DEFAULT if you want camsys driver to
+ * determine it.
+ * @pixel_mode: the pixel mode driver used in the raw pipeline. It is written by
+ * driver only.
+ * @throughput: the throughput be used in the raw pipeline. It is written by
+ * driver only.
+ *
+ */
+struct mtk_cam_resource_raw {
+ __s64 feature;
+ __u16 strategy;
+ __u8 raw_max;
+ __u8 raw_min;
+ __u8 raw_used;
+ __u8 bin;
+ __u8 path_sel;
+ __u8 pixel_mode;
+ __u64 throughput;
+};
+
+/*
+ * struct mtk_cam_resource - MTK camsys resoruces for format negotiation
+ *
+ * @sink_fmt: sink_fmt pad's format, it must be return by g_fmt or s_fmt
+ * from driver.
+ * @sensor_res: senor information to calculate the required resource, it is
+ * read-only and camsys driver will not change it.
+ * @raw_res: user hint and resource negotiation result.
+ * @status: resource negotiation status.
+ *
+ */
+struct mtk_cam_resource {
+ __u64 sink_fmt;
+ struct mtk_cam_resource_sensor sensor_res;
+ struct mtk_cam_resource_raw raw_res;
+ __u8 status;
+};
+
+/**
+ * struct mtk_cam_pde_info - PDE module information for raw
+ *
+ * @pdo_max_size: the max pdo size of pde sensor.
+ * @pdi_max_size: the max pdi size of pde sensor or max pd table size.
+ * @pd_table_offset: the offest of meta config for pd table content.
+ */
+struct mtk_cam_pde_info {
+ __u32 pdo_max_size;
+ __u32 pdi_max_size;
+ __u32 pd_table_offset;
+};
+#endif /* _MTKISP_CAMSYS_USER_H */
\ No newline at end of file
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
old mode 100644
new mode 100755
index 9e3b366d5fc7..abaee64bebd1
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -840,6 +840,93 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_PISP_COMP2_BGGR v4l2_fourcc('P', 'C', '2', 'B') /* PiSP 8-bit mode 2 compressed BGGR bayer */
#define V4L2_PIX_FMT_PISP_COMP2_MONO v4l2_fourcc('P', 'C', '2', 'M') /* PiSP 8-bit mode 2 compressed monochrome */
+/* Luminance+Chrominance formats */
+#define V4L2_PIX_FMT_YUYV10 v4l2_fourcc('Y', 'U', 'Y', 'A') /* 16 YUV 4:2:2 10-bit */
+#define V4L2_PIX_FMT_YVYU10 v4l2_fourcc('Y', 'V', 'Y', 'A') /* 16 YUV 4:2:2 10-bit */
+#define V4L2_PIX_FMT_UYVY10 v4l2_fourcc('U', 'Y', 'V', 'A') /* 16 YUV 4:2:2 10-bit */
+#define V4L2_PIX_FMT_VYUY10 v4l2_fourcc('V', 'Y', 'U', 'A') /* 16 YUV 4:2:2 10-bit */
+#define V4L2_PIX_FMT_YUYV12 v4l2_fourcc('Y', 'U', 'Y', 'C') /* 16 YUV 4:2:2 12-bit */
+#define V4L2_PIX_FMT_YVYU12 v4l2_fourcc('Y', 'V', 'Y', 'C') /* 16 YUV 4:2:2 12-bit */
+#define V4L2_PIX_FMT_UYVY12 v4l2_fourcc('U', 'Y', 'V', 'C') /* 16 YUV 4:2:2 12-bit */
+#define V4L2_PIX_FMT_VYUY12 v4l2_fourcc('V', 'Y', 'U', 'C') /* 16 YUV 4:2:2 12-bit */
+
+/* two planes -- one Y, one Cr + Cb interleaved */
+#define V4L2_PIX_FMT_NV12_10 v4l2_fourcc('1', '2', 'A', 'U') /* 12 Y/CbCr 4:2:0 10 bits un-packed */
+#define V4L2_PIX_FMT_NV21_10 v4l2_fourcc('2', '1', 'A', 'U') /* 12 Y/CrCb 4:2:0 10 bits un-packed */
+#define V4L2_PIX_FMT_NV16_10 v4l2_fourcc('1', '6', 'A', 'U') /* 16 Y/CbCr 4:2:2 10 bits un-packed */
+#define V4L2_PIX_FMT_NV61_10 v4l2_fourcc('6', '1', 'A', 'U') /* 16 Y/CrCb 4:2:2 10 bits un-packed */
+#define V4L2_PIX_FMT_NV12_12 v4l2_fourcc('1', '2', 'C', 'U') /* 12 Y/CbCr 4:2:0 12 bits un-packed */
+#define V4L2_PIX_FMT_NV21_12 v4l2_fourcc('2', '1', 'C', 'U') /* 12 Y/CrCb 4:2:0 12 bits un-packed */
+#define V4L2_PIX_FMT_NV16_12 v4l2_fourcc('1', '6', 'C', 'U') /* 16 Y/CbCr 4:2:2 12 bits un-packed */
+#define V4L2_PIX_FMT_NV61_12 v4l2_fourcc('6', '1', 'C', 'U') /* 16 Y/CrCb 4:2:2 12 bits un-packed */
+
+/* Vendor specific - MediaTek ISP bayer formats */
+#define V4L2_PIX_FMT_MTISP_SBGGR8 v4l2_fourcc('M', 'B', 'B', '8') /* Packed 8-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG8 v4l2_fourcc('M', 'B', 'G', '8') /* Packed 8-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG8 v4l2_fourcc('M', 'B', 'g', '8') /* Packed 8-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB8 v4l2_fourcc('M', 'B', 'R', '8') /* Packed 8-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR10 v4l2_fourcc('M', 'B', 'B', 'A') /* Packed 10-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG10 v4l2_fourcc('M', 'B', 'G', 'A') /* Packed 10-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG10 v4l2_fourcc('M', 'B', 'g', 'A') /* Packed 10-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB10 v4l2_fourcc('M', 'B', 'R', 'A') /* Packed 10-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR12 v4l2_fourcc('M', 'B', 'B', 'C') /* Packed 12-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG12 v4l2_fourcc('M', 'B', 'G', 'C') /* Packed 12-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG12 v4l2_fourcc('M', 'B', 'g', 'C') /* Packed 12-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB12 v4l2_fourcc('M', 'B', 'R', 'C') /* Packed 12-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR14 v4l2_fourcc('M', 'B', 'B', 'E') /* Packed 14-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG14 v4l2_fourcc('M', 'B', 'G', 'E') /* Packed 14-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG14 v4l2_fourcc('M', 'B', 'g', 'E') /* Packed 14-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB14 v4l2_fourcc('M', 'B', 'R', 'E') /* Packed 14-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR8F v4l2_fourcc('M', 'F', 'B', '8') /* Full-G 8-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG8F v4l2_fourcc('M', 'F', 'G', '8') /* Full-G 8-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG8F v4l2_fourcc('M', 'F', 'g', '8') /* Full-G 8-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB8F v4l2_fourcc('M', 'F', 'R', '8') /* Full-G 8-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR10F v4l2_fourcc('M', 'F', 'B', 'A') /* Full-G 10-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG10F v4l2_fourcc('M', 'F', 'G', 'A') /* Full-G 10-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG10F v4l2_fourcc('M', 'F', 'g', 'A') /* Full-G 10-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB10F v4l2_fourcc('M', 'F', 'R', 'A') /* Full-G 10-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR12F v4l2_fourcc('M', 'F', 'B', 'C') /* Full-G 12-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG12F v4l2_fourcc('M', 'F', 'G', 'C') /* Full-G 12-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG12F v4l2_fourcc('M', 'F', 'g', 'C') /* Full-G 12-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB12F v4l2_fourcc('M', 'F', 'R', 'C') /* Full-G 12-bit */
+#define V4L2_PIX_FMT_MTISP_SBGGR14F v4l2_fourcc('M', 'F', 'B', 'E') /* Full-G 14-bit */
+#define V4L2_PIX_FMT_MTISP_SGBRG14F v4l2_fourcc('M', 'F', 'G', 'E') /* Full-G 14-bit */
+#define V4L2_PIX_FMT_MTISP_SGRBG14F v4l2_fourcc('M', 'F', 'g', 'E') /* Full-G 14-bit */
+#define V4L2_PIX_FMT_MTISP_SRGGB14F v4l2_fourcc('M', 'F', 'R', 'E') /* Full-G 14-bit */
+#define V4L2_PIX_FMT_MTISP_SGRB8F v4l2_fourcc('M', 'F', '8', 'P') /* three planes Full-G 8-bit */
+#define V4L2_PIX_FMT_MTISP_SGRB10F v4l2_fourcc('M', 'F', 'A', 'P') /* three planes Full-G 10-bit */
+#define V4L2_PIX_FMT_MTISP_SGRB12F v4l2_fourcc('M', 'F', 'C', 'P') /* three planes Full-G 12-bit */
+
+/* Vendor specific - MediaTek Luminance+Chrominance formats */
+#define V4L2_PIX_FMT_MTISP_YUYV10P v4l2_fourcc('Y', 'U', 'A', 'P') /* YUV 4:2:2 10-bit packed */
+#define V4L2_PIX_FMT_MTISP_YVYU10P v4l2_fourcc('Y', 'V', 'A', 'P') /* YUV 4:2:2 10-bit packed */
+#define V4L2_PIX_FMT_MTISP_UYVY10P v4l2_fourcc('U', 'Y', 'A', 'P') /* YUV 4:2:2 10-bit packed */
+#define V4L2_PIX_FMT_MTISP_VYUY10P v4l2_fourcc('V', 'Y', 'A', 'P') /* YUV 4:2:2 10-bit packed */
+#define V4L2_PIX_FMT_MTISP_NV12_10P v4l2_fourcc('1', '2', 'A', 'P') /* Y/CbCr 4:2:0 10 bits packed */
+#define V4L2_PIX_FMT_MTISP_NV21_10P v4l2_fourcc('2', '1', 'A', 'P') /* Y/CrCb 4:2:0 10 bits packed */
+#define V4L2_PIX_FMT_MTISP_NV16_10P v4l2_fourcc('1', '6', 'A', 'P') /* Y/CbCr 4:2:2 10 bits packed */
+#define V4L2_PIX_FMT_MTISP_NV61_10P v4l2_fourcc('6', '1', 'A', 'P') /* Y/CrCb 4:2:2 10 bits packed */
+#define V4L2_PIX_FMT_MTISP_YUYV12P v4l2_fourcc('Y', 'U', 'C', 'P') /* YUV 4:2:2 12-bit packed */
+#define V4L2_PIX_FMT_MTISP_YVYU12P v4l2_fourcc('Y', 'V', 'C', 'P') /* YUV 4:2:2 12-bit packed */
+#define V4L2_PIX_FMT_MTISP_UYVY12P v4l2_fourcc('U', 'Y', 'C', 'P') /* YUV 4:2:2 12-bit packed */
+#define V4L2_PIX_FMT_MTISP_VYUY12P v4l2_fourcc('V', 'Y', 'C', 'P') /* YUV 4:2:2 12-bit packed */
+#define V4L2_PIX_FMT_MTISP_NV12_12P v4l2_fourcc('1', '2', 'C', 'P') /* Y/CbCr 4:2:0 12 bits packed */
+#define V4L2_PIX_FMT_MTISP_NV21_12P v4l2_fourcc('2', '1', 'C', 'P') /* Y/CrCb 4:2:0 12 bits packed */
+#define V4L2_PIX_FMT_MTISP_NV16_12P v4l2_fourcc('1', '6', 'C', 'P') /* Y/CbCr 4:2:2 12 bits packed */
+#define V4L2_PIX_FMT_MTISP_NV61_12P v4l2_fourcc('6', '1', 'C', 'P') /* Y/CrCb 4:2:2 12 bits packed */
+
+/* Vendor specific - MediaTek specified compressed format */
+#define V4L2_PIX_FMT_MTISP_NV12_UFBC v4l2_fourcc('1', '2', '8', 'F') /* Y/CbCr 4:2:0 8 bits compressed */
+#define V4L2_PIX_FMT_MTISP_NV21_UFBC v4l2_fourcc('2', '1', '8', 'F') /* Y/CrCb 4:2:0 8 bits compressed */
+#define V4L2_PIX_FMT_MTISP_NV12_10_UFBC v4l2_fourcc('1', '2', 'A', 'F') /* Y/CbCr 4:2:0 10 bits compressed */
+#define V4L2_PIX_FMT_MTISP_NV21_10_UFBC v4l2_fourcc('2', '1', 'A', 'F') /* Y/CrCb 4:2:0 10 bits compressed */
+#define V4L2_PIX_FMT_MTISP_NV12_12_UFBC v4l2_fourcc('1', '2', 'C', 'F') /* Y/CbCr 4:2:0 12 bits compressed */
+#define V4L2_PIX_FMT_MTISP_NV21_12_UFBC v4l2_fourcc('2', '1', 'C', 'F') /* Y/CrCb 4:2:0 12 bits compressed */
+#define V4L2_PIX_FMT_MTISP_BAYER8_UFBC v4l2_fourcc('M', 'B', '8', 'U') /* Raw 8 bits compressed */
+#define V4L2_PIX_FMT_MTISP_BAYER10_UFBC v4l2_fourcc('M', 'B', 'A', 'U') /* Raw 10 bits compressed */
+#define V4L2_PIX_FMT_MTISP_BAYER12_UFBC v4l2_fourcc('M', 'B', 'C', 'U') /* Raw 12 bits compressed */
+#define V4L2_PIX_FMT_MTISP_BAYER14_UFBC v4l2_fourcc('M', 'B', 'E', 'U') /* Raw 14 bits compressed */
+
/* SDR formats - used only for Software Defined Radio devices */
#define V4L2_SDR_FMT_CU8 v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */
#define V4L2_SDR_FMT_CU16LE v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [PATCH v2 13/13] media: uapi: mediatek: document ISP7x camera system and user controls
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (7 preceding siblings ...)
2025-07-07 1:31 ` [PATCH v2 12/13] uapi: linux: add mediatek isp_7x camsys user api shangyao lin
@ 2025-07-07 1:31 ` shangyao lin
2025-07-11 3:12 ` CK Hu (胡俊光)
[not found] ` <20250707013154.4055874-6-shangyao.lin@mediatek.com>
` (4 subsequent siblings)
13 siblings, 1 reply; 37+ messages in thread
From: shangyao lin @ 2025-07-07 1:31 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno
Cc: Shangyao Lin, linux-media, devicetree, linux-kernel,
linux-arm-kernel, linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
From: "shangyao.lin" <shangyao.lin@mediatek.com>
This document covers:
- The hardware pipeline and firmware interface
- Integration with the V4L2 CAMSYS/media controller framework
- The list of custom V4L2 controls, their usage, valid ranges, and typical usage flow
Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
---
.../media/v4l/mtk-isp7x-camsys.rst | 94 +++++++++
.../media/v4l/mtk-isp7x-controls.rst | 199 ++++++++++++++++++
2 files changed, 293 insertions(+)
create mode 100755 Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst
create mode 100755 Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst
diff --git a/Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst b/Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst
new file mode 100755
index 000000000000..04a1fe1c448d
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst
@@ -0,0 +1,94 @@
+==========================
+MediaTek ISP7x Camera System
+==========================
+
+Overview
+========
+
+The MediaTek ISP7x camera system is designed to provide a flexible, high-performance image capture pipeline for SoCs, leveraging the Linux V4L2 framework. The system is composed of multiple hardware modules (Sensor, SENINF, RAW ISP, etc.) and a firmware backend, with a clear separation of control and data flow. This document describes the hardware architecture from the perspective of the interface exposed to the firmware (backend), and how it integrates with the V4L2 CAMSYS framework.
+
+V4L2 CAMSYS Framework Overview
+==============================
+
+The V4L2 CAMSYS framework models the camera pipeline as a set of interconnected devices and subdevices:
+
+- **Video Device**: Exposes buffer queue/dequeue (enque/deque) operations to user space, allowing applications to stream image data.
+- **Subdevice**: Represents a hardware block (e.g., sensor, SENINF, RAW ISP) and handles control/configuration operations.
+- **Pads and Links**: Each subdevice has pads (input/output endpoints). Media links connect pads, representing the physical and logical data flow between hardware blocks.
+
+The topology and link state are established at probe time and can be dynamically enabled/disabled by user space, depending on the desired pipeline configuration.
+
+Hardware Pipeline
+=================
+
+A typical pipeline consists of:
+
+.. code-block::
+
+ Sensor --> SENINF --> RAW ISP --> (optional: YUV ISP) --> Memory
+
+- **Sensor**: Provides MIPI CSI-2 data.
+- **SENINF**: Receives and demultiplexes sensor data, supports virtual channel (VC) and data type (DT) routing.
+- **RAW ISP**: Main image signal processor, handles demosaicing, noise reduction, and other image enhancements.
+- **YUV ISP** (optional): Further processing for YUV output.
+- **CAMSYS**: System controller, manages pipeline state, buffer flow, and synchronization.
+
+Firmware Interface
+==================
+
+The firmware (backend, e.g., CCD/SCP) is responsible for:
+
+- Calculating hardware settings (3A, tuning, DMA, FBC, Twin, etc.) for each frame.
+- Generating command queues (CQ) and descriptors to program the hardware.
+- Abstracting hardware-specific details from the kernel driver.
+
+The kernel driver communicates with the firmware via an inter-processor interface (IPI), typically using RPMSG. The main interactions are:
+
+1. **Session Management**: The kernel establishes a session with the firmware at stream-on, and destroys it at stream-off.
+2. **Per-frame Request Scheduling**: For each user request (frame), the kernel sends a request to the firmware, including sensor settings, buffer addresses, and control parameters.
+3. **CQ Generation**: The firmware generates the necessary command queues and returns them to the kernel.
+4. **Event Notification**: The firmware notifies the kernel of CQ completion, errors, or other events.
+
+Request Scheduling and Buffer Flow
+==================================
+
+The request scheduling flow is as follows:
+
+1. **User space** submits a request (with buffer and control settings) via V4L2.
+2. **PipeMgr** (user space) collects requests, groups them by file descriptor, and sends them to the kernel CAMSYS driver.
+3. **CAMSYS** schedules requests, manages buffer flow, and coordinates with the firmware for hardware programming.
+4. **Firmware** processes each request, generates CQ, and signals completion.
+5. **Kernel** dequeues the buffer and notifies user space.
+
+The system supports per-frame control, allowing sensor settings and ISP parameters to be updated on a frame-by-frame basis.
+
+Media Controller Topology
+=========================
+
+The media controller graph models the hardware pipeline as a set of entities (video devices, subdevices) connected by links. Each entity exposes pads, and links are established at probe time based on the hardware configuration and device tree.
+
+.. code-block::
+
+ +---------+ +--------+ +--------+ +------+
+ | Sensor | --> | SENINF | --> | RAW ISP| --> | Video|
+ +---------+ +--------+ +--------+ +------+
+
+- Solid lines: active links (enabled)
+- Dashed lines: inactive links (disabled)
+
+User space can query and configure the topology using standard V4L2 and media controller APIs.
+
+Backend Abstraction
+===================
+
+The backend firmware abstracts hardware-specific programming, allowing the kernel driver to remain generic and maintainable. All hardware register programming, tuning, and advanced features are handled in the firmware, with the kernel acting as a control and data flow manager.
+
+This separation ensures that proprietary hardware details remain in the firmware, while the kernel driver exposes a standard V4L2 interface.
+
+Summary
+=======
+
+- The MediaTek ISP7x camera system is built on the V4L2 media controller framework, with a modular pipeline of video devices and subdevices.
+- The kernel driver manages pipeline state, buffer flow, and per-frame control, while delegating hardware programming to the backend firmware via IPI.
+- The firmware interface is session-based, supports per-frame CQ generation, and abstracts all hardware-specific details from the kernel.
+- The media controller graph models the hardware pipeline, allowing flexible configuration and dynamic topology management.
\ No newline at end of file
diff --git a/Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst b/Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst
new file mode 100755
index 000000000000..0e8d41d339e7
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst
@@ -0,0 +1,199 @@
+==========================================
+MediaTek ISP Camsys User Space Interface
+==========================================
+
+Overview
+--------
+
+The MediaTek ISP camsys driver provides a set of custom V4L2 controls for user space
+applications to configure and query the ISP hardware. These controls allow
+fine-grained management of resource allocation, feature enablement, and
+hardware-specific settings.
+
+Typical Usage Flow
+------------------
+
+1. Open the video device node (e.g., ``/dev/video0``).
+2. Set required V4L2 controls using ``VIDIOC_S_CTRL`` or ``VIDIOC_S_EXT_CTRLS``.
+3. Set up video format and buffers (``VIDIOC_S_FMT``, ``VIDIOC_REQBUFS``, etc.).
+4. Start streaming with ``VIDIOC_STREAMON``.
+5. Queue and dequeue buffers for image processing (``VIDIOC_QBUF``, ``VIDIOC_DQBUF``).
+6. Stop streaming with ``VIDIOC_STREAMOFF``.
+
+Control List and Documentation
+-----------------------------
+
+Below is a list of custom V4L2 controls provided by the camsys driver, with detailed descriptions.
+
+.. list-table::
+ :header-rows: 1
+
+ * - Control Name
+ - Type
+ - Description
+ - Valid Values / Range
+ - When to Set
+
+ * - ``V4L2_CID_MTK_CAM_USED_ENGINE_LIMIT``
+ - Integer
+ - Limits the maximum number of RAW engines (hardware pipelines) that can be used for this stream.
+ - 1 ~ 2 (default: 2)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_BIN_LIMIT``
+ - Integer
+ - Sets the maximum binning ratio allowed for the stream.
+ - 0 ~ 0xFFF (default: 0)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_FRZ_LIMIT``
+ - Integer
+ - Sets the maximum resizer (FRZ) ratio allowed (as a percentage).
+ - 70 ~ 100 (default: 100)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_RESOURCE_PLAN_POLICY``
+ - Integer (Enum)
+ - Selects the resource planning policy for the stream.
+ - 0: Default, 1: Power-saving, 2: Performance, ... (0~10)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_USED_ENGINE``
+ - Integer
+ - Reports or sets the number of RAW engines actually used for this stream after negotiation.
+ - 1 ~ 2 (default: 2)
+ - After resource negotiation
+
+ * - ``V4L2_CID_MTK_CAM_BIN``
+ - Boolean
+ - Enables or disables binning for the stream.
+ - 0 (disable), 1 (enable, default)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_FRZ``
+ - Integer
+ - Sets the resizer (FRZ) ratio (as a percentage) for the stream.
+ - 70 ~ 100 (default: 100)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_USED_ENGINE_TRY``
+ - Integer
+ - Reports the number of RAW engines that would be used for the current "try" format negotiation.
+ - 1 ~ 2 (default: 2)
+ - After VIDIOC_TRY_FMT
+
+ * - ``V4L2_CID_MTK_CAM_BIN_TRY``
+ - Boolean
+ - Reports whether binning would be enabled for the current "try" format negotiation.
+ - 0, 1 (default: 1)
+ - After VIDIOC_TRY_FMT
+
+ * - ``V4L2_CID_MTK_CAM_FRZ_TRY``
+ - Integer
+ - Reports the resizer ratio that would be used for the current "try" format negotiation.
+ - 70 ~ 100 (default: 100)
+ - After VIDIOC_TRY_FMT
+
+ * - ``V4L2_CID_MTK_CAM_PIXEL_RATE``
+ - Integer64
+ - Sets the pixel rate (in pixels per second) of the sensor for resource calculation.
+ - 0 ~ 0xFFFFFFFF
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_FEATURE``
+ - Bitmask (Integer64)
+ - Bitmask of enabled features for the stream (e.g., HDR, 3A, etc.). See driver header for details.
+ - Bitmask (0 ~ RAW_FUNCTION_END)
+ - Before streaming or per-frame
+
+ * - ``V4L2_CID_MTK_CAM_SYNC_ID``
+ - Integer64
+ - Frame synchronization ID for multi-sensor or multi-pipeline synchronization.
+ - -1 ~ 0x7FFFFFFF (default: -1)
+ - Before streaming or per-frame
+
+ * - ``V4L2_CID_MTK_CAM_RAW_PATH_SELECT``
+ - Integer
+ - Selects the RAW path (e.g., main or secondary) for the stream.
+ - 0, 1 (default: 1)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_HSF_EN``
+ - Integer
+ - Enables or disables hardware secure flow (HSF) for the stream.
+ - 0 (disable), 1 (enable)
+ - Before streaming
+
+ * - ``V4L2_CID_MTK_CAM_PDE_INFO``
+ - Integer (struct mtk_cam_pde_info)
+ - Provides or sets phase detection (PDE) information for PDAF/AF features.
+ - See struct definition
+ - Before streaming or per-frame
+
+ * - ``V4L2_CID_MTK_CAM_MSTREAM_EXPOSURE``
+ - Integer/Integer64
+ - Controls multi-stream exposure settings for advanced HDR or multi-exposure modes.
+ - See driver header
+ - Before streaming or per-frame
+
+ * - ``V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC``
+ - Compound (struct mtk_cam_resource)
+ - Triggers resource calculation for the current format and settings.
+ - See struct definition
+ - During format negotiation
+
+ * - ``V4L2_CID_MTK_CAM_TG_FLASH_CFG``
+ - Integer/struct
+ - Configures timing generator (TG) and flash synchronization.
+ - See driver header
+ - Before streaming or per-frame
+
+ * - ``V4L2_CID_MTK_CAM_RAW_RESOURCE_UPDATE``
+ - Integer
+ - Notifies the driver of a sensor mode update (e.g., seamless switch).
+ - 0 ~ 0xF (default: 0)
+ - When sensor mode changes
+
+ * - ``V4L2_CID_MTK_CAM_CAMSYS_HW_MODE``
+ - Integer64
+ - Selects the hardware mode for the CAMSYS pipeline (e.g., normal, secure, etc.).
+ - 0 ~ 0x7FFFFFFF
+ - Before streaming
+
+Control Usage Patterns
+---------------------
+
+- Controls with ``_TRY`` suffix are used to query the result of a "try" format negotiation (``VIDIOC_TRY_FMT``), without committing changes.
+- Controls without ``_TRY`` are used to set or get the actual configuration (``VIDIOC_S_FMT``, ``VIDIOC_G_CTRL``).
+- Use ``V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC`` to check if the requested format and settings are feasible, and to get recommended values.
+- Use ``V4L2_CID_MTK_CAM_RAW_RESOURCE_UPDATE`` to notify the driver of a sensor mode change (e.g., seamless switch).
+- Use ``V4L2_CID_MTK_CAM_SYNC_ID`` to synchronize frames across multiple pipelines.
+- Use ``V4L2_CID_MTK_CAM_FEATURE`` to enable/disable advanced features (bitmask).
+- Use ``V4L2_CID_MTK_CAM_PDE_INFO`` to set/get phase detection information for PDAF/AF.
+- Use ``V4L2_CID_MTK_CAM_PIXEL_RATE`` to inform the driver of the sensor's pixel rate for bandwidth/resource calculation.
+
+Example: Setting a Control
+--------------------------
+
+.. code-block:: c
+
+ int fd = open("/dev/video0", O_RDWR);
+ struct v4l2_control ctrl = {
+ .id = V4L2_CID_MTK_CAM_BIN_LIMIT,
+ .value = 2,
+ };
+ if (ioctl(fd, VIDIOC_S_CTRL, &ctrl) < 0) {
+ perror("Failed to set BIN_LIMIT");
+ }
+
+Notes
+-----
+
+- All controls should be set before starting the video stream unless otherwise specified.
+- Refer to the driver header file for detailed struct definitions and bitmask values.
+- The driver will reject invalid values or unsupported configurations with -EINVAL.
+
+References
+----------
+
+- V4L2 documentation: https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2.html
\ No newline at end of file
--
2.18.0
^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
@ 2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:42 ` Krzysztof Kozlowski
2025-07-15 8:25 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Rob Herring (Arm) @ 2025-07-07 2:46 UTC (permalink / raw)
To: shangyao lin
Cc: Matthias Brugger, Project_Global_Chrome_Upstream_Group,
linaro-mm-sig, AngeloGioacchino Del Regno, linux-mediatek,
Krzysztof Kozlowski, linux-kernel, linux-arm-kernel, devicetree,
linux-media, dri-devel, Conor Dooley, Mauro Carvalho Chehab
On Mon, 07 Jul 2025 09:31:42 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-camisp.yaml
> - Split bindings into four separate patches (one per YAML file)
> - Various fixes per review comments
> - Update maintainers list
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/mediatek,mt8188-camisp.yaml | 68 +++++++++++++++++++
> 1 file changed, 68 insertions(+)
> create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
./Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml:68:4: [error] no new line character at the end of file (new-line-at-end-of-file)
dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml: properties:reg-names: {'items': [{'const': 'base'}], 'minItems': 1, 'maxItems': 1, 'description': 'Name for the register region. Must be "base".'} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml: properties:reg-names: 'oneOf' conditional failed, one must be fixed:
[{'const': 'base'}] is too short
False schema does not allow 1
hint: "minItems" is only needed if less than the "items" list length
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml: properties:reg: 'anyOf' conditional failed, one must be fixed:
'minItems' is not one of ['maxItems', 'description', 'deprecated']
hint: Only "maxItems" is required for a single entry if there are no constraints defined for the values.
'minItems' is not one of ['description', 'deprecated', 'const', 'enum', 'minimum', 'maximum', 'multipleOf', 'default', '$ref', 'oneOf']
'maxItems' is not one of ['description', 'deprecated', 'const', 'enum', 'minimum', 'maximum', 'multipleOf', 'default', '$ref', 'oneOf']
1 is less than the minimum of 2
hint: Arrays must be described with a combination of minItems/maxItems/items
hint: cell array properties must define how many entries and what the entries are when there is more than one entry.
from schema $id: http://devicetree.org/meta-schemas/core.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml: properties:power-domains: 'anyOf' conditional failed, one must be fixed:
'minItems' is not one of ['maxItems', 'description', 'deprecated']
hint: Only "maxItems" is required for a single entry if there are no constraints defined for the values.
'minItems' is not one of ['description', 'deprecated', 'const', 'enum', 'minimum', 'maximum', 'multipleOf', 'default', '$ref', 'oneOf']
'maxItems' is not one of ['description', 'deprecated', 'const', 'enum', 'minimum', 'maximum', 'multipleOf', 'default', '$ref', 'oneOf']
1 is less than the minimum of 2
hint: Arrays must be described with a combination of minItems/maxItems/items
hint: cell array properties must define how many entries and what the entries are when there is more than one entry.
from schema $id: http://devicetree.org/meta-schemas/power-domain.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml: $id: Cannot determine base path from $id, relative path/filename doesn't match actual path or filename
$id: http://devicetree.org/schemas/media/mediatek/mediatek,camisp.yaml
file: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dts:23.13-43: Warning (reg_format): /example-0/soc/isp@16000000:reg: property has invalid length (16 bytes) (#address-cells == 2, #size-cells == 1)
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: Warning (pci_device_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: Warning (pci_device_bus_num): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: Warning (simple_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: Warning (i2c_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: Warning (spi_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dts:21.24-27.13: Warning (avoid_default_addr_size): /example-0/soc/isp@16000000: Relying on default #address-cells value
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dts:21.24-27.13: Warning (avoid_default_addr_size): /example-0/soc/isp@16000000: Relying on default #size-cells value
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: Warning (unique_unit_address_if_enabled): Failed prerequisite 'avoid_default_addr_size'
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.example.dtb: isp@16000000 (mediatek,mt8188-camisp): reg: [[0, 369098752], [0, 4096]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek/mediatek,camisp.yaml#
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250707013154.4055874-2-shangyao.lin@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding
2025-07-07 1:31 ` [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding shangyao lin
@ 2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:44 ` Krzysztof Kozlowski
2025-07-15 7:28 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Rob Herring (Arm) @ 2025-07-07 2:46 UTC (permalink / raw)
To: shangyao lin
Cc: Conor Dooley, Krzysztof Kozlowski, Mauro Carvalho Chehab,
Matthias Brugger, AngeloGioacchino Del Regno, dri-devel,
Project_Global_Chrome_Upstream_Group, devicetree,
linux-arm-kernel, linaro-mm-sig, linux-mediatek, linux-media,
linux-kernel
On Mon, 07 Jul 2025 09:31:43 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> 1. Add camera isp7x module device document
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-seninf-core.yaml
> - Various fixes per review comments
> - Update maintainers list
>
> Question for reviewer (CK):
>
> Hi CK,
>
> Thank you for your review and suggestions on this patch, especially for providing the reference patch (https://patchwork.kernel.org/project/linux-mediatek/list/?series=874617) and for mentioning in another patch ([V1,02/10] MEDIA: PLATFORM: MEDIATEK: ADD SENINF CONTROLLER) the suggestion to "Move the phy part to phy/mediatek/ folder. You could refer to phy/mediatek/phy-mtk-mipi-csi-0-5.c".
>
> After reading your comments and the reference patches, my understanding is that only the seninf-core driver should manage all ports internally, and each port corresponds to a PHY. During probe, the driver will parse each port, obtain the corresponding PHY (e.g., devm_phy_get(dev, "csi0"), devm_phy_get(dev, "csi1"), etc.), and operate the PHY for each port individually during stream on/off or power on/off.
>
> Could you please confirm if my understanding is correct?
> If you have any additional reference patches or examples, I would greatly appreciate it.
>
> Thank you for your guidance!
>
> Best regards,
> Shangyao
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/mediatek,mt8188-seninf-core.yaml | 121 ++++++++++++++++++
> 1 file changed, 121 insertions(+)
> create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
./Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml:6:10: [error] string value is redundantly quoted with any quotes (quoted-strings)
./Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml:121:4: [error] no new line character at the end of file (new-line-at-end-of-file)
dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml: properties:clock-names: {'items': [{'const': 'clk_cam_seninf'}, {'const': 'clk_top_seninf'}, {'const': 'clk_top_seninf1'}, {'const': 'clk_top_camtm'}], 'minItems': 4, 'maxItems': 4} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml: properties:clock-names: 'oneOf' conditional failed, one must be fixed:
[{'const': 'clk_cam_seninf'}, {'const': 'clk_top_seninf'}, {'const': 'clk_top_seninf1'}, {'const': 'clk_top_camtm'}] is too long
[{'const': 'clk_cam_seninf'}, {'const': 'clk_top_seninf'}, {'const': 'clk_top_seninf1'}, {'const': 'clk_top_camtm'}] is too short
False schema does not allow 4
1 was expected
4 is greater than the maximum of 2
4 is greater than the maximum of 3
hint: "minItems" is only needed if less than the "items" list length
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml: properties:reg-names: {'items': [{'const': 'base'}], 'minItems': 1, 'maxItems': 1} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml: properties:reg-names: 'oneOf' conditional failed, one must be fixed:
[{'const': 'base'}] is too short
False schema does not allow 1
hint: "minItems" is only needed if less than the "items" list length
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml: properties:reg: 'anyOf' conditional failed, one must be fixed:
'minItems' is not one of ['maxItems', 'description', 'deprecated']
hint: Only "maxItems" is required for a single entry if there are no constraints defined for the values.
'minItems' is not one of ['description', 'deprecated', 'const', 'enum', 'minimum', 'maximum', 'multipleOf', 'default', '$ref', 'oneOf']
'maxItems' is not one of ['description', 'deprecated', 'const', 'enum', 'minimum', 'maximum', 'multipleOf', 'default', '$ref', 'oneOf']
1 is less than the minimum of 2
hint: Arrays must be described with a combination of minItems/maxItems/items
hint: cell array properties must define how many entries and what the entries are when there is more than one entry.
from schema $id: http://devicetree.org/meta-schemas/core.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml: $id: Cannot determine base path from $id, relative path/filename doesn't match actual path or filename
$id: http://devicetree.org/schemas/media/mediatek,seninf-core.yaml
file: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.yaml
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-seninf-core.example.dtb: seninf@16010000 (mediatek,mt8188-seninf-core): reg: [[0, 369164288], [0, 32768]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek,seninf-core.yaml#
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250707013154.4055874-3-shangyao.lin@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding
2025-07-07 1:31 ` [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding shangyao lin
@ 2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:48 ` Krzysztof Kozlowski
2025-07-15 8:13 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Rob Herring (Arm) @ 2025-07-07 2:46 UTC (permalink / raw)
To: shangyao lin
Cc: dri-devel, linux-media, linux-mediatek, Matthias Brugger,
devicetree, linaro-mm-sig, linux-kernel, Krzysztof Kozlowski,
Conor Dooley, AngeloGioacchino Del Regno, Mauro Carvalho Chehab,
Project_Global_Chrome_Upstream_Group, linux-arm-kernel
On Mon, 07 Jul 2025 09:31:44 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-cam-raw.yaml
> - Various fixes per review comments
> - Update maintainers list
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/mediatek,mt8188-cam-raw.yaml | 156 ++++++++++++++++++
> 1 file changed, 156 insertions(+)
> create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
./Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml:156:4: [error] no new line character at the end of file (new-line-at-end-of-file)
dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml: properties:clock-names: {'items': [{'const': 'camsys_cam2mm0_cgpdn'}, {'const': 'camsys_cam2mm1_cgpdn'}, {'const': 'camsys_cam2sys_cgpdn'}, {'const': 'camsys_cam_cgpdn'}, {'const': 'camsys_camtg_cgpdn'}, {'const': 'camsys_rawa_larbx_cgpdn'}, {'const': 'camsys_rawa_cam_cgpdn'}, {'const': 'camsys_rawa_camtg_cgpdn'}, {'const': 'topckgen_top_cam'}, {'const': 'topckgen_top_camtg'}, {'const': 'topckgen_top_camtm'}], 'minItems': 4, 'maxItems': 16, 'description': 'Names of the clocks, must match the order of the clocks property.'} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml: properties:reg-names: {'items': [{'const': 'base'}, {'const': 'inner_base'}], 'minItems': 1, 'maxItems': 2, 'description': 'Names for each register region. Must be "base" and optionally "inner_base".'} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml: $id: Cannot determine base path from $id, relative path/filename doesn't match actual path or filename
$id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-raw.yaml
file: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.yaml
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dts:33.13-34.43: Warning (reg_format): /example-0/soc/raw@16030000:reg: property has invalid length (32 bytes) (#address-cells == 2, #size-cells == 1)
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dts:37.13-59: Warning (dma_ranges_format): /example-0/soc/raw@16030000:dma-ranges: "dma-ranges" property has invalid length (24 bytes) (parent #address-cells == 2, child #address-cells == 2, #size-cells == 1)
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: Warning (pci_device_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: Warning (pci_device_bus_num): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: Warning (simple_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: Warning (i2c_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: Warning (spi_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dts:31.24-78.13: Warning (avoid_default_addr_size): /example-0/soc/raw@16030000: Relying on default #address-cells value
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dts:31.24-78.13: Warning (avoid_default_addr_size): /example-0/soc/raw@16030000: Relying on default #size-cells value
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: Warning (unique_unit_address_if_enabled): Failed prerequisite 'avoid_default_addr_size'
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: raw@16030000 (mediatek,mt8188-cam-raw): dma-ranges: [[2], [0], [0], [1073741824], [1], [0]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-raw.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-raw.example.dtb: raw@16030000 (mediatek,mt8188-cam-raw): reg: [[0, 369295360], [0, 32768], [0, 369328128], [0, 32768]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-raw.yaml#
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250707013154.4055874-4-shangyao.lin@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding
2025-07-07 1:31 ` [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding shangyao lin
@ 2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:47 ` Krzysztof Kozlowski
2025-07-15 7:41 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Rob Herring (Arm) @ 2025-07-07 2:46 UTC (permalink / raw)
To: shangyao lin
Cc: Conor Dooley, dri-devel, Matthias Brugger, linaro-mm-sig,
Mauro Carvalho Chehab, linux-kernel, linux-media,
Project_Global_Chrome_Upstream_Group, Krzysztof Kozlowski,
devicetree, linux-mediatek, linux-arm-kernel,
AngeloGioacchino Del Regno
On Mon, 07 Jul 2025 09:31:45 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-cam-yuv.yaml
> - Various fixes per review comments
> - Update maintainers list
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/mediatek,mt8188-cam-yuv.yaml | 134 ++++++++++++++++++
> 1 file changed, 134 insertions(+)
> create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml: properties:reg-names: {'items': [{'const': 'base'}, {'const': 'inner_base'}], 'minItems': 1, 'maxItems': 2, 'description': 'Names for each register region. Must be "base" and optionally "inner_base".'} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml: properties:clock-names: {'items': [{'const': 'camsys_cam2mm0_cgpdn'}, {'const': 'camsys_cam2mm1_cgpdn'}, {'const': 'camsys_cam2sys_cgpdn'}, {'const': 'camsys_yuva_larbx'}, {'const': 'camsys_yuva_cam_cgpdn'}, {'const': 'camsys_yuva_camtg_cgpdn'}, {'const': 'camsys_yuvb_larbx_cgpdn'}, {'const': 'camsys_yuvb_cam_cgpdn'}, {'const': 'camsys_yuvb_camtg_cgpdn'}], 'minItems': 4, 'maxItems': 16, 'description': 'Names of the clocks, must match the order of the clocks property.'} should not be valid under {'required': ['maxItems']}
hint: "maxItems" is not needed with an "items" list
from schema $id: http://devicetree.org/meta-schemas/items.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml: $id: Cannot determine base path from $id, relative path/filename doesn't match actual path or filename
$id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-yuv.yaml
file: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dts:33.13-43: Warning (reg_format): /example-0/soc/yuv@16050000:reg: property has invalid length (16 bytes) (#address-cells == 2, #size-cells == 1)
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dts:37.13-59: Warning (dma_ranges_format): /example-0/soc/yuv@16050000:dma-ranges: "dma-ranges" property has invalid length (24 bytes) (parent #address-cells == 2, child #address-cells == 2, #size-cells == 1)
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: Warning (pci_device_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: Warning (pci_device_bus_num): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: Warning (simple_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: Warning (i2c_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: Warning (spi_bus_reg): Failed prerequisite 'reg_format'
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dts:31.24-58.13: Warning (avoid_default_addr_size): /example-0/soc/yuv@16050000: Relying on default #address-cells value
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dts:31.24-58.13: Warning (avoid_default_addr_size): /example-0/soc/yuv@16050000: Relying on default #size-cells value
Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: Warning (unique_unit_address_if_enabled): Failed prerequisite 'avoid_default_addr_size'
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: yuv@16050000 (mediatek,mt8188-cam-yuv): clock-names:3: 'camsys_yuva_larbx' was expected
from schema $id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-yuv.yaml#
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.example.dtb: yuv@16050000 (mediatek,mt8188-cam-yuv): dma-ranges: [[2], [0], [0], [1073741824], [1], [0]] is too long
from schema $id: http://devicetree.org/schemas/media/mediatek/mediatek,cam-yuv.yaml#
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/project/devicetree-bindings/patch/20250707013154.4055874-5-shangyao.lin@mediatek.com
The base for the series is generally the latest rc1. A different dependency
should be noted in *this* patch.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit after running the above command yourself. Note
that DT_SCHEMA_FILES can be set to your schema file to speed up checking
your schema. However, it must be unset to test all examples with your schema.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
@ 2025-07-07 5:42 ` Krzysztof Kozlowski
2025-07-15 8:25 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:42 UTC (permalink / raw)
To: shangyao lin
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno,
linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On Mon, Jul 07, 2025 at 09:31:42AM +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
Missing SoB, missing checkpatch.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-camisp.yaml
> - Split bindings into four separate patches (one per YAML file)
> - Various fixes per review comments
> - Update maintainers list
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
Apply this patch and see by yourself.
This wasn't tested either, so I will skip review.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding
2025-07-07 1:31 ` [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
@ 2025-07-07 5:44 ` Krzysztof Kozlowski
2025-07-15 7:28 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:44 UTC (permalink / raw)
To: shangyao lin
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno,
linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On Mon, Jul 07, 2025 at 09:31:43AM +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> 1. Add camera isp7x module device document
>
Still no SoB, still not tested, still no checkpatch...
and also:
...
> +
> +...
> \ No newline at end of file
Look, you have patch warnings. Review your patches before you post them.
You should reach internally to mediatek colleagues to explain you how
this process works. I really do not understand why mediatek has so poor
quality of work and cannot improve over last 1-2 years!
> --
> 2.18.0
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding
2025-07-07 1:31 ` [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
@ 2025-07-07 5:47 ` Krzysztof Kozlowski
2025-07-15 7:41 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:47 UTC (permalink / raw)
To: shangyao lin
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno,
linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On Mon, Jul 07, 2025 at 09:31:45AM +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-cam-yuv.yaml
> - Various fixes per review comments
> - Update maintainers list
Where did you post v1?
Please use standard email subjects, so with the PATCH keyword in the
title. 'git format-patch -vX' helps here to create proper versioned patches.
Another useful tool is b4. Skipping the PATCH keyword makes filtering of
emails more difficult thus making the review process less convenient.
A nit, subject: drop second/last, redundant "bindings". The
"dt-bindings" prefix is already stating that these are bindings.
See also:
https://elixir.bootlin.com/linux/v6.7-rc8/source/Documentation/devicetree/bindings/submitting-patches.rst#L18
> +properties:
> + compatible:
> + const: mediatek,mt8188-cam-yuv
> +
> + reg:
> + minItems: 1
What, why?
Look at other bindings.
> + maxItems: 2
> + description:
> + Base address and optional inner base address of the cam-yuv hardware block.
Why are you stating obvious? From where did you take it?
> +
> + reg-names:
> + items:
> + - const: base
> + - const: inner_base
> + minItems: 1
> + maxItems: 2
No, really no. You did not follow any existing patterns and this binding
does not look at all as anything else. Why making this things up? Just
use recently reviewed binding as starting point.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding
2025-07-07 1:31 ` [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
@ 2025-07-07 5:48 ` Krzysztof Kozlowski
2025-07-15 8:13 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:48 UTC (permalink / raw)
To: shangyao lin
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno,
linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On Mon, Jul 07, 2025 at 09:31:44AM +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-cam-raw.yaml
> - Various fixes per review comments
Which fixes? This is way too vague, considering how buggy and poor this
code is. Where was v1 of this?
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 05/13] media: platform: mediatek: add isp_7x seninf unit
[not found] ` <20250707013154.4055874-6-shangyao.lin@mediatek.com>
@ 2025-07-07 5:50 ` Krzysztof Kozlowski
2025-07-10 6:39 ` CK Hu (胡俊光)
2025-07-16 4:06 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:50 UTC (permalink / raw)
To: shangyao lin
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Matthias Brugger, AngeloGioacchino Del Regno,
linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On Mon, Jul 07, 2025 at 09:31:46AM +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
My most frequent comment last year to Mediatek. I even asked to come
with some internal procedure so you will not keep repating the same
mistake in author's name.
Any success?
>
> Introduce support for the MediaTek sensor interface (seninf) in the SoC camera
...
>
> ---
>
> Note:
> The PHY operations have been refactored and separated from the seninf driver,
> but there are still some issues to confirm with reviewers in this v2 patch
> (dt-bindings: media: mediatek: add seninf-core binding). The PHY part will be
> moved to drivers/phy/mediatek/ in v3.
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
and here the same.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 00/13] Add MediaTek ISP7.x camera system support
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
` (9 preceding siblings ...)
[not found] ` <20250707013154.4055874-6-shangyao.lin@mediatek.com>
@ 2025-07-07 5:55 ` Krzysztof Kozlowski
[not found] ` <20250707013154.4055874-8-shangyao.lin@mediatek.com>
` (2 subsequent siblings)
13 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:55 UTC (permalink / raw)
To: shangyao lin, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On 07/07/2025 03:31, shangyao lin wrote:
> Based on linux-next tag: next-20250630
>
> This patch set adds the MediaTek ISP7.x camera system hardware driver.
>
> The driver sets up ISP hardware, handles interrupts, and initializes
> V4L2 device nodes and functions. It also implements a V4L2 standard video
> driver utilizing the media framework APIs, connects sensors and the ISP
> via the seninf interface, and communicates with the SCP co-processor to
> compose ISP registers in firmware.
>
Now I found v1. How did you address comment about compliance report?
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit
[not found] ` <20250707013154.4055874-9-shangyao.lin@mediatek.com>
@ 2025-07-07 5:58 ` Krzysztof Kozlowski
2025-07-10 5:20 ` CK Hu (胡俊光)
` (2 subsequent siblings)
3 siblings, 0 replies; 37+ messages in thread
From: Krzysztof Kozlowski @ 2025-07-07 5:58 UTC (permalink / raw)
To: shangyao lin, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Matthias Brugger,
AngeloGioacchino Del Regno
Cc: linux-media, devicetree, linux-kernel, linux-arm-kernel,
linux-mediatek, dri-devel, linaro-mm-sig,
Project_Global_Chrome_Upstream_Group
On 07/07/2025 03:31, shangyao lin wrote:
> +static int mtk_cam_create_links(struct mtk_cam_device *cam)
> +{
> + struct v4l2_subdev *sd;
> + int ret = 0;
> +
> + v4l2_device_for_each_subdev(sd, &cam->v4l2_dev) {
> + if (sd->entity.function == MEDIA_ENT_F_VID_IF_BRIDGE)
> + ret = config_bridge_pad_links(cam, sd);
> + }
> +
> + return ret;
> +}
> +
> +static int mtk_cam_master_register(struct device *dev)
> +{
> + struct mtk_cam_device *cam_dev = dev_get_drvdata(dev);
> + struct media_device *media_dev = &cam_dev->media_dev;
> + int ret;
> +
> + dev_info(dev, "camsys | start %s\n", __func__);
NAK, nothing improved, you already got comments on that. Provide
detailed changelog next time. And respond to EACH comment acknowledging
that you understood it.
This code still has so many trivial issues and terrible coding style,
including wrong indentation, wrong alignment/wrapping.
> +
> + media_dev->dev = cam_dev->dev;
> + strscpy(media_dev->model, dev_driver_string(dev),
> + sizeof(media_dev->model));
> + snprintf(media_dev->bus_info, sizeof(media_dev->bus_info),
> + "platform:%s", dev_name(dev));
> + media_dev->hw_revision = 0;
> + media_dev->ops = &mtk_cam_dev_ops;
> + media_device_init(media_dev);
> +
> + cam_dev->v4l2_dev.mdev = media_dev;
> + ret = v4l2_device_register(cam_dev->dev, &cam_dev->v4l2_dev);
> + if (ret) {
> + dev_dbg(dev, "Failed to register V4L2 device: %d\n", ret);
> + goto fail_media_device_cleanup;
> + }
> +
> + ret = media_device_register(media_dev);
> + if (ret) {
> + dev_dbg(dev, "Failed to register media device: %d\n",
> + ret);
> + goto fail_v4l2_device_unreg;
> + }
> +
> + dev_info(dev, "%s success\n", __func__);
> + return 0;
> +
> +fail_v4l2_device_unreg:
> + v4l2_device_unregister(&cam_dev->v4l2_dev);
> +
> +fail_media_device_cleanup:
> + media_device_cleanup(&cam_dev->media_dev);
> +
> + return ret;
> +}
> +
> +static void mtk_cam_master_unregister(struct device *dev)
> +{
> + struct mtk_cam_device *cam_dev = dev_get_drvdata(dev);
> +
> + dev_info(dev, "camsys | start %s\n", __func__);
NAK
> +
> + media_device_unregister(&cam_dev->media_dev);
> + v4l2_device_unregister(&cam_dev->v4l2_dev);
> + media_device_cleanup(&cam_dev->media_dev);
> +}
> +
> +static int mtk_cam_async_add_by_driver(struct device *dev,
> + struct platform_driver *drv,
> + struct v4l2_async_notifier *notifier)
> +{
> + struct fwnode_handle *fwnode;
> + struct device *p;
> + struct v4l2_async_connection *asc;
> + int dev_num = 0;
> +
> + p = platform_find_device_by_driver(NULL, &drv->driver);
> + while (p) {
> + dev_info(dev, "camsys | %s add %s\n", __func__, p->kobj.name);
> +
> + fwnode = dev_fwnode(p);
> + asc = v4l2_async_nf_add_fwnode(notifier, fwnode,
> + struct v4l2_async_connection);
> + put_device(p);
> +
...
> +
> + cam_dev->base = devm_ioremap_resource(dev, res);
> + if (IS_ERR(cam_dev->base)) {
> + dev_err(dev, "failed to map register base\n");
> + return PTR_ERR(cam_dev->base);
> + }
> +
> + cam_dev->dev = dev;
> + dev_set_drvdata(dev, cam_dev);
> +
> + cam_dev->composer_cnt = 0;
> + cam_dev->num_seninf_devices = 0;
> +
> + cam_dev->max_stream_num = MTKCAM_SUBDEV_MAX;
> + cam_dev->ctxs = devm_kcalloc(dev, cam_dev->max_stream_num,
> + sizeof(*cam_dev->ctxs), GFP_KERNEL);
> + if (!cam_dev->ctxs)
> + return -ENOMEM;
> +
> + cam_dev->streaming_ctx = 0;
> + for (i = 0; i < cam_dev->max_stream_num; i++)
> + mtk_cam_ctx_init(cam_dev->ctxs + i, cam_dev, i);
> +
> + cam_dev->running_job_count = 0;
> + spin_lock_init(&cam_dev->pending_job_lock);
> + spin_lock_init(&cam_dev->running_job_lock);
> + INIT_LIST_HEAD(&cam_dev->pending_job_list);
> + INIT_LIST_HEAD(&cam_dev->running_job_list);
> +
> + cam_dev->dma_processing_count = 0;
> + spin_lock_init(&cam_dev->dma_pending_lock);
> + spin_lock_init(&cam_dev->dma_processing_lock);
> + INIT_LIST_HEAD(&cam_dev->dma_pending);
> + INIT_LIST_HEAD(&cam_dev->dma_processing);
> +
> + mutex_init(&cam_dev->queue_lock);
> +
> + pm_runtime_enable(dev);
> +
> + ret = mtk_cam_of_rproc(cam_dev, pdev);
> + if (ret)
> + goto fail_destroy_mutex;
> +
> + ret = register_sub_drivers(dev);
> + if (ret) {
> + dev_err(dev, "fail to register_sub_drivers\n");
> + goto fail_destroy_mutex;
> + }
> +
> + cam_dev->clks_pdev = platform_device_register_data(
> + dev, "clk-mt8188-cam", PLATFORM_DEVID_AUTO, NULL, 0);
> + if (IS_ERR(cam_dev->clks_pdev)) {
> + ret = PTR_ERR(cam_dev->clks_pdev);
> + dev_err(dev, "Failed to register cam clk device: %d\n", ret);
> + goto fail_unregister_sub_drivers;
> + }
> +
> + /* register mtk_cam as all isp subdev async parent */
> + cam_dev->notifier.ops = &mtk_cam_async_nf_ops;
> + v4l2_async_nf_init(&cam_dev->notifier, &cam_dev->v4l2_dev);
> + ret = mtk_cam_async_subdev_add(dev); /* wait all isp sub drivers */
> + if (ret) {
> + dev_err(dev, "%s failed mtk_cam_async_subdev_add\n", __func__);
> + goto fail_unregister_clks;
> + }
> +
> + ret = v4l2_async_nf_register(&cam_dev->notifier);
> + if (ret) {
> + dev_err(dev, "%s async_nf_register ret:%d\n", __func__, ret);
> + v4l2_async_nf_cleanup(&cam_dev->notifier);
> + goto fail_unregister_clks;
> + }
> +
> + dev_info(dev, "camsys | [%s] success\n", __func__);
You already got comments. Nothing improved here.
> +
> + return 0;
> +
> +fail_unregister_clks:
> + if (cam_dev->clks_pdev)
> + platform_device_unregister(cam_dev->clks_pdev);
> +
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 12/13] uapi: linux: add mediatek isp_7x camsys user api
2025-07-07 1:31 ` [PATCH v2 12/13] uapi: linux: add mediatek isp_7x camsys user api shangyao lin
@ 2025-07-08 1:34 ` CK Hu (胡俊光)
0 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-08 1:34 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add UAPI for the MediaTek ISP platform, providing user-space interfaces for the
> new CAMSYS driver.
>
> Changes in v2:
> - Moved V4L2 pixel format definitions to include/uapi/linux/videodev2.h
>
> Notes:
>
> Hi Laurent Pinchart and CK,
>
> Thank you very much for your detailed feedback and suggestions.
>
> I would like to let you know that in the next patch, I have added an initial
> version of an .rst document describing the MTK ISP CAMSYS controls. Most of
> these ISP CAMSYS controls are intended for other advanced features. For normal
> main-stream use cases, only controls such as V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC
> are actually used.
>
> Would you prefer that we remove the user space interfaces for these advanced
> features for now, and only add them back in a future patch series dedicated to
> advanced functionality? I am happy to follow your recommendation on this.
I suggest that you send the basic function first.
For you and the reviewer, the process would be much easier.
Reviewer review fewer codes, and you get fewer comment.
Advanced function could be sent later at any time you want.
Regards,
CK
>
> Additionally, I have also written an initial version of a brief .rst document
> providing a high-level description of the hardware. If you think there is
> anything else that should be added or clarified, please kindly let me know.
>
> Thank you again for your valuable feedback!
>
> Best regards,
> Shangyao
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit
[not found] ` <20250707013154.4055874-9-shangyao.lin@mediatek.com>
2025-07-07 5:58 ` [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit Krzysztof Kozlowski
@ 2025-07-10 5:20 ` CK Hu (胡俊光)
2025-07-10 7:46 ` CK Hu (胡俊光)
2025-07-24 8:36 ` CK Hu (胡俊光)
3 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-10 5:20 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduces the top media device driver for the MediaTek ISP7X CAMSYS. The driver maintains the camera system, including sub-device management, DMA operations, and integration with the V4L2 framework. It handles request stream data, buffer management, MediaTek-specific features, pipeline management, streaming control, and error handling mechanisms. Additionally, it aggregates sub-drivers for the camera interface, raw, and yuv pipelines.
>
> ---
>
> Changes in v2:
> - Removed mtk_cam-ufbc-def.h along with related code
> - Various fixes per review comments
>
> ---
>
> [Question for CK]:
>
> Hi CK,
>
> Is your suggestion to simplify buffer management to a single buf_list with a buffer status (estate) state machine, so all buffers stay in one list and only the estate field changes (no more multiple lists)? The buffer life cycle would be: WAITING → SCP_GENERATE_CQ → CQ_READY → CQ_APPLY → DONE.
>
> I have some concerns about concurrency and race conditions:
>
> Multiple lists can use different locks to reduce contention.
> With a single list, all state changes need to lock the same list, which may require more careful lock design to avoid race conditions or deadlocks.
> When multiple threads (interrupts, workqueues, user requests) operate at the same time, synchronization becomes more difficult.
> Do you have any reference code or best practices for this kind of design?
Deadlock happen only when code hold two lock at the same time.
I think when you hold buf_list lock, it's not necessary to hold other lock, so it's not necessary to worry about deadlock.
The buffer management in this driver is simple.
All the buffer is processed step by step in order.
I have no sample code to provide to you.
You may point out the worry code in this patch and we could discuss on that point.
>
> Thanks!
>
>
>
> Reply regarding stream_id and pipe_id naming:
>
> Hi CK,
>
> Thank you for your suggestion. In our current design, stream_id and pipe_id are not always in a 1-to-1 relationship. In some cases, one stream may correspond to multiple pipes, or vice versa. That’s why we use both names in different contexts. If you have any suggestions on how to unify the naming while keeping this distinction clear, please let us know.
>
So stream_id and pipe_id are different concept.
Below is the code in this patch.
+struct mtk_cam_request_stream_data*
+mtk_cam_get_req_s_data(struct mtk_cam_ctx *ctx, unsigned int pipe_id,
+ unsigned int frame_seq_no)
+struct mtk_cam_request *mtk_cam_get_req(struct mtk_cam_ctx *ctx,
+ unsigned int frame_seq_no)
+{
+ struct mtk_cam_request_stream_data *req_stream_data;
+
+ req_stream_data = mtk_cam_get_req_s_data(ctx, ctx->stream_id, frame_seq_no);
The second parameter of mtk_cam_get_req_s_data() is pipe_id, but you pass stream_id into it.
Because stream_id and pipe_id are different concept, so you should not do this.
You should pass pipe_id into mtk_cam_get_req_s_data().
Regards,
CK
+ if (!req_stream_data)
+ return NULL;
+
+ return req_stream_data->req;
+}
> Thanks!
>
> Best regards,
> Shangyao
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 05/13] media: platform: mediatek: add isp_7x seninf unit
[not found] ` <20250707013154.4055874-6-shangyao.lin@mediatek.com>
2025-07-07 5:50 ` [PATCH v2 05/13] media: platform: mediatek: add isp_7x seninf unit Krzysztof Kozlowski
@ 2025-07-10 6:39 ` CK Hu (胡俊光)
2025-07-16 4:06 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-10 6:39 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduce support for the MediaTek sensor interface (seninf) in the SoC camera
> subsystem, focusing on CSI and stream control. The driver manages parameter
> control, metering, status tracking, and interrupt handling for the camera sensor
> interface hardware. It integrates with the MediaTek ISP CAMSYS, bridging camera
> sensors and the ISP system, and provides V4L2 framework support for dynamic
> stream configuration and virtual channel management.
>
> ---
[snip]
> diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-hw_phy_2_0.c b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-hw_phy_2_0.c
> new file mode 100755
> index 000000000000..a3c926cc3cee
> --- /dev/null
> +++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-hw_phy_2_0.c
> @@ -0,0 +1,1932 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// Copyright (c) 2022 MediaTek Inc.
> +
> +#include <linux/module.h>
> +#include <linux/delay.h>
> +#include <linux/bitfield.h>
> +#include <linux/bitops.h>
> +
> +#include "../mtk_cam-seninf.h"
> +#include "../mtk_cam-seninf-hw.h"
> +#include "../mtk_cam-seninf-regs.h"
> +#include "mtk_cam-seninf-top-ctrl.h"
> +#include "mtk_cam-seninf-seninf1-mux.h"
> +#include "mtk_cam-seninf-seninf1.h"
> +#include "mtk_cam-seninf-seninf1-csi2.h"
> +#include "mtk_cam-seninf-tg1.h"
> +#include "mtk_cam-seninf-cammux.h"
> +#include "mtk_cam-seninf-mipi-rx-ana-cdphy-csi0a.h"
> +#include "mtk_cam-seninf-csi0-cphy.h"
> +#include "mtk_cam-seninf-csi0-dphy.h"
> +#include "../kd_imgsensor_define_v4l2.h"
> +
> +static struct mtk_cam_seninf_cfg _seninf_cfg = {
> + .mux_num = 8,
> + .seninf_num = 4,
> + .cam_mux_num = 11,
> + .pref_mux_num = 11,
> +};
> +
> +struct mtk_cam_seninf_cfg *g_seninf_cfg = &_seninf_cfg;
> +
> +static inline u32 get_port_num(int port)
> +{
> + if (port >= CSI_PORT_0A)
> + return FIELD_GET(CSI_PORT_A_MASK, port - CSI_PORT_0A);
> + else
> + return port;
> +}
> +
> +static inline void mtk_cam_seninf_set_di_ch_ctrl(void __iomem *pseninf,
> + unsigned int stream_id,
> + struct seninf_vc *vc)
> +{
> + if (stream_id > SENINF_MAX_STREAM)
> + return;
> +
> + SENINF_BITS(pseninf, SENINF_CSI2_S0_DI_CTRL + (stream_id << 0x2),
> + RG_CSI2_DT_SEL, vc->dt);
> + SENINF_BITS(pseninf, SENINF_CSI2_S0_DI_CTRL + (stream_id << 0x2),
> + RG_CSI2_VC_SEL, 0);
> + SENINF_BITS(pseninf, SENINF_CSI2_S0_DI_CTRL + (stream_id << 0x2),
> + RG_CSI2_DT_INTERLEAVE_MODE, 1);
> + SENINF_BITS(pseninf, SENINF_CSI2_S0_DI_CTRL + (stream_id << 0x2),
> + RG_CSI2_VC_INTERLEAVE_EN, 1);
> +
> + switch (stream_id) {
> + case 0:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S0_GRP_EN, 1);
> + break;
> + case 1:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S1_GRP_EN, 1);
> + break;
> + case 2:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S2_GRP_EN, 1);
> + break;
> + case 3:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S3_GRP_EN, 1);
> + break;
> + case 4:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S4_GRP_EN, 1);
> + break;
> + case 5:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S5_GRP_EN, 1);
> + break;
> + case 6:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S6_GRP_EN, 1);
> + break;
> + case 7:
> + SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
> + RG_CSI2_S7_GRP_EN, 1);
> + break;
> + default:
> + return;
> + }
#define RG_CSI2_SX_GRP_EN(x) BIT(x + 8)
and this switch case could be reduced as
SENINF_BITS(pseninf, SENINF_CSI2_CH0_CTRL,
RG_CSI2_SX_GRP_EN(stream_id), 1);
> +}
> +
[snip]
> +
> +int mtk_cam_seninf_set_top_mux_ctrl(struct seninf_ctx *ctx, int mux_idx,
> + int seninf_src)
> +{
> + void __iomem *seninf = ctx->reg_if_top;
> +
> + switch (mux_idx) {
> + case SENINF_MUX1:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_0,
> + RG_SENINF_MUX1_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX2:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_0,
> + RG_SENINF_MUX2_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX3:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_0,
> + RG_SENINF_MUX3_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX4:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_0,
> + RG_SENINF_MUX4_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX5:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_1,
> + RG_SENINF_MUX5_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX6:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_1,
> + RG_SENINF_MUX6_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX7:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_1,
> + RG_SENINF_MUX7_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX8:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_1,
> + RG_SENINF_MUX8_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX9:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_2,
> + RG_SENINF_MUX9_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX10:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_2,
> + RG_SENINF_MUX10_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX11:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_2,
> + RG_SENINF_MUX11_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX12:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_2,
> + RG_SENINF_MUX12_SRC_SEL, seninf_src);
> + break;
> + case SENINF_MUX13:
> + SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL_3,
> + RG_SENINF_MUX13_SRC_SEL, seninf_src);
> + break;
> + default:
> + dev_dbg(ctx->dev, "invalid mux_idx %d\n", mux_idx);
> + return -EINVAL;
> + }
> +
#define SENINF_TOP_MUX_CTRL(x) (0x10 + 4 * x)
#define RG_SENINF_MUX_SRC_SEL(x) GENMASK((x % 4) * 4 + 3, (x % 4) * 4)
And this switch case could be reduced as
SENINF_BITS(seninf, SENINF_TOP_MUX_CTRL(mux_idx / 4),
RG_SENINF_MUX_SRC_SEL(mux_idx), seninf_src);
Regards,
CK
> + return 0;
> +}
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit
[not found] ` <20250707013154.4055874-9-shangyao.lin@mediatek.com>
2025-07-07 5:58 ` [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit Krzysztof Kozlowski
2025-07-10 5:20 ` CK Hu (胡俊光)
@ 2025-07-10 7:46 ` CK Hu (胡俊光)
2025-07-24 8:36 ` CK Hu (胡俊光)
3 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-10 7:46 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduces the top media device driver for the MediaTek ISP7X CAMSYS. The driver maintains the camera system, including sub-device management, DMA operations, and integration with the V4L2 framework. It handles request stream data, buffer management, MediaTek-specific features, pipeline management, streaming control, and error handling mechanisms. Additionally, it aggregates sub-drivers for the camera interface, raw, and yuv pipelines.
>
> ---
[snip]
> +struct mtk_cam_ctx *mtk_cam_start_ctx(struct mtk_cam_device *cam,
> + struct mtk_cam_video_device *node)
> +{
> + struct mtk_cam_ctx *ctx = node->ctx;
> + struct device *dev;
> + struct v4l2_subdev **target_sd;
> + int ret, i, is_first_ctx;
> + struct media_entity *entity = &node->vdev.entity;
> + struct media_graph graph;
> +
> + dev_info(cam->dev, "%s:ctx(%d): triggered by %s\n",
> + __func__, ctx->stream_id, entity->name);
> +
> + atomic_set(&ctx->enqueued_frame_seq_no, 0);
> + ctx->composed_frame_seq_no = 0;
> + ctx->dequeued_frame_seq_no = 0;
> + atomic_set(&ctx->running_s_data_cnt, 0);
> + init_completion(&ctx->session_complete);
> +
> + is_first_ctx = !cam->composer_cnt;
> + if (is_first_ctx) {
> + spin_lock(&cam->dma_processing_lock);
> + cam->dma_processing_count = 0;
> + spin_unlock(&cam->dma_processing_lock);
> + spin_lock(&cam->running_job_lock);
> + cam->running_job_count = 0;
> + spin_unlock(&cam->running_job_lock);
> +
> + dev_info(cam->dev, "%s: power on camsys\n", __func__);
> + ret = pm_runtime_resume_and_get(cam->dev);
> + if (ret < 0) {
> + dev_info(cam->dev, "%s: power on camsys failed\n",
> + __func__);
> + return NULL;
> + }
> +
> + ret = isp_composer_init(cam);
> + if (ret)
> + goto fail_shutdown;
> + }
> + cam->composer_cnt++;
> + if (is_yuv_node(node->desc.id))
> + dev = cam->raw.yuvs[0];
> + else
> + dev = cam->raw.devs[0];
> +
> + ret = mtk_cam_working_buf_pool_init(ctx, dev);
> + if (ret) {
> + dev_info(cam->dev, "failed to reserve DMA memory:%d\n", ret);
> + goto fail_uninit_composer;
> + }
> +
> + kthread_init_worker(&ctx->sensor_worker);
You does not do anything to sensor_worker, so it's usedless.
Drop it.
> + ctx->sensor_worker_task = kthread_run(kthread_worker_fn,
> + &ctx->sensor_worker,
> + "sensor_worker-%d",
> + ctx->stream_id);
> + if (IS_ERR(ctx->sensor_worker_task)) {
> + dev_info(cam->dev,
> + "%s:ctx(%d): could not create sensor_worker_task\n",
> + __func__, ctx->stream_id);
> + goto fail_release_buffer_pool;
> + }
> +
> + sched_set_fifo(ctx->sensor_worker_task);
> +
> + ctx->composer_wq = alloc_ordered_workqueue(dev_name(cam->dev),
> + WQ_HIGHPRI | WQ_FREEZABLE);
> + if (!ctx->composer_wq) {
> + dev_info(cam->dev, "failed to alloc composer workqueue\n");
> + goto fail_uninit_sensor_worker_task;
> + }
> +
> + ctx->frame_done_wq = alloc_ordered_workqueue(dev_name(cam->dev),
> + WQ_HIGHPRI | WQ_FREEZABLE);
> + if (!ctx->frame_done_wq) {
> + dev_info(cam->dev, "failed to alloc frame_done workqueue\n");
> + goto fail_uninit_composer_wq;
> + }
> +
> + ret = media_pipeline_start(&entity->pads[0], &ctx->pipeline);
> + if (ret) {
> + dev_warn(cam->dev,
> + "%s:pipe(%d):failed in media_pipeline_start:%d\n",
> + __func__, node->uid.pipe_id, ret);
> + goto fail_uninit_frame_done_wq;
> + }
> +
> + /* traverse to update used subdevs & number of nodes */
> + i = 0;
> + ret = media_graph_walk_init(&graph, entity->graph_obj.mdev);
> + if (ret)
> + goto fail_stop_pipeline;
> +
> + media_graph_walk_start(&graph, entity);
> + while ((entity = media_graph_walk_next(&graph))) {
> + dev_dbg(cam->dev, "linked entity %s\n", entity->name);
> +
> + target_sd = NULL;
> +
> + switch (entity->function) {
> + case MEDIA_ENT_F_IO_V4L:
> + ctx->enabled_node_cnt++;
> + break;
> + case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: /* pipeline */
> + if (i >= MAX_PIPES_PER_STREAM)
> + goto fail_stop_pipeline;
> + target_sd = ctx->pipe_subdevs + i;
> + i++;
> + break;
> + case MEDIA_ENT_F_VID_IF_BRIDGE: /* seninf */
> + target_sd = &ctx->seninf;
> + break;
> + case MEDIA_ENT_F_CAM_SENSOR:
> + target_sd = &ctx->sensor;
In this series, no entity's function is MEDIA_ENT_F_CAM_SENSOR,
so ctx->sensor is always NULL.
Drop it.
Regards,
CK
> + break;
> + default:
> + break;
> + }
> +
> + if (!target_sd)
> + continue;
> +
> + if (*target_sd) {
> + dev_info(cam->dev, "duplicated subdevs!!!\n");
> + goto fail_traverse_subdev;
> + }
> +
> + if (is_media_entity_v4l2_subdev(entity))
> + *target_sd = media_entity_to_v4l2_subdev(entity);
> + }
> + media_graph_walk_cleanup(&graph);
> +
> + return ctx;
> +
> +fail_traverse_subdev:
> + media_graph_walk_cleanup(&graph);
> +fail_stop_pipeline:
> + media_pipeline_stop(&entity->pads[0]);
> +fail_uninit_frame_done_wq:
> + destroy_workqueue(ctx->frame_done_wq);
> +fail_uninit_composer_wq:
> + destroy_workqueue(ctx->composer_wq);
> +fail_uninit_sensor_worker_task:
> + kthread_stop(ctx->sensor_worker_task);
> + ctx->sensor_worker_task = NULL;
> +fail_release_buffer_pool:
> + mtk_cam_working_buf_pool_release(ctx, dev);
> +fail_uninit_composer:
> + isp_composer_uninit(cam);
> + cam->composer_cnt--;
> +fail_shutdown:
> + if (is_first_ctx)
> + rproc_shutdown(cam->rproc_handle);
> +
> + return NULL;
> +}
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 13/13] media: uapi: mediatek: document ISP7x camera system and user controls
2025-07-07 1:31 ` [PATCH v2 13/13] media: uapi: mediatek: document ISP7x camera system and user controls shangyao lin
@ 2025-07-11 3:12 ` CK Hu (胡俊光)
0 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-11 3:12 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> This document covers:
> - The hardware pipeline and firmware interface
> - Integration with the V4L2 CAMSYS/media controller framework
> - The list of custom V4L2 controls, their usage, valid ranges, and typical usage flow
You may drop some advanced function, but I don't know which are advanced function, so I would review all.
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../media/v4l/mtk-isp7x-camsys.rst | 94 +++++++++
> .../media/v4l/mtk-isp7x-controls.rst | 199 ++++++++++++++++++
> 2 files changed, 293 insertions(+)
> create mode 100755 Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst
> create mode 100755 Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst b/Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst
> new file mode 100755
> index 000000000000..04a1fe1c448d
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/mtk-isp7x-camsys.rst
> @@ -0,0 +1,94 @@
> +==========================
> +MediaTek ISP7x Camera System
> +==========================
> +
> +Overview
> +========
> +
> +The MediaTek ISP7x camera system is designed to provide a flexible, high-performance image capture pipeline for SoCs, leveraging the Linux V4L2 framework. The system is composed of multiple hardware modules (Sensor, SENINF, RAW ISP, etc.) and a firmware backend, with a clear separation of control and data flow. This document describes the hardware architecture from the perspective of the interface exposed to the firmware (backend), and how it integrates with the V4L2 CAMSYS framework.
> +
> +V4L2 CAMSYS Framework Overview
> +==============================
> +
> +The V4L2 CAMSYS framework models the camera pipeline as a set of interconnected devices and subdevices:
> +
> +- **Video Device**: Exposes buffer queue/dequeue (enque/deque) operations to user space, allowing applications to stream image data.
> +- **Subdevice**: Represents a hardware block (e.g., sensor, SENINF, RAW ISP) and handles control/configuration operations.
> +- **Pads and Links**: Each subdevice has pads (input/output endpoints). Media links connect pads, representing the physical and logical data flow between hardware blocks.
> +
> +The topology and link state are established at probe time and can be dynamically enabled/disabled by user space, depending on the desired pipeline configuration.
> +
> +Hardware Pipeline
> +=================
> +
> +A typical pipeline consists of:
> +
> +.. code-block::
> +
> + Sensor --> SENINF --> RAW ISP --> (optional: YUV ISP) --> Memory
> +
> +- **Sensor**: Provides MIPI CSI-2 data.
> +- **SENINF**: Receives and demultiplexes sensor data, supports virtual channel (VC) and data type (DT) routing.
> +- **RAW ISP**: Main image signal processor, handles demosaicing, noise reduction, and other image enhancements.
> +- **YUV ISP** (optional): Further processing for YUV output.
> +- **CAMSYS**: System controller, manages pipeline state, buffer flow, and synchronization.
> +
> +Firmware Interface
> +==================
> +
> +The firmware (backend, e.g., CCD/SCP) is responsible for:
> +
> +- Calculating hardware settings (3A, tuning, DMA, FBC, Twin, etc.) for each frame.
> +- Generating command queues (CQ) and descriptors to program the hardware.
> +- Abstracting hardware-specific details from the kernel driver.
> +
> +The kernel driver communicates with the firmware via an inter-processor interface (IPI), typically using RPMSG. The main interactions are:
> +
> +1. **Session Management**: The kernel establishes a session with the firmware at stream-on, and destroys it at stream-off.
> +2. **Per-frame Request Scheduling**: For each user request (frame), the kernel sends a request to the firmware, including sensor settings, buffer addresses, and control parameters.
> +3. **CQ Generation**: The firmware generates the necessary command queues and returns them to the kernel.
> +4. **Event Notification**: The firmware notifies the kernel of CQ completion, errors, or other events.
> +
> +Request Scheduling and Buffer Flow
> +==================================
> +
> +The request scheduling flow is as follows:
> +
> +1. **User space** submits a request (with buffer and control settings) via V4L2.
> +2. **PipeMgr** (user space) collects requests, groups them by file descriptor, and sends them to the kernel CAMSYS driver.
> +3. **CAMSYS** schedules requests, manages buffer flow, and coordinates with the firmware for hardware programming.
> +4. **Firmware** processes each request, generates CQ, and signals completion.
> +5. **Kernel** dequeues the buffer and notifies user space.
> +
> +The system supports per-frame control, allowing sensor settings and ISP parameters to be updated on a frame-by-frame basis.
> +
> +Media Controller Topology
> +=========================
> +
> +The media controller graph models the hardware pipeline as a set of entities (video devices, subdevices) connected by links. Each entity exposes pads, and links are established at probe time based on the hardware configuration and device tree.
> +
> +.. code-block::
> +
> + +---------+ +--------+ +--------+ +------+
> + | Sensor | --> | SENINF | --> | RAW ISP| --> | Video|
> + +---------+ +--------+ +--------+ +------+
> +
> +- Solid lines: active links (enabled)
> +- Dashed lines: inactive links (disabled)
> +
> +User space can query and configure the topology using standard V4L2 and media controller APIs.
> +
> +Backend Abstraction
> +===================
> +
> +The backend firmware abstracts hardware-specific programming, allowing the kernel driver to remain generic and maintainable. All hardware register programming, tuning, and advanced features are handled in the firmware, with the kernel acting as a control and data flow manager.
> +
> +This separation ensures that proprietary hardware details remain in the firmware, while the kernel driver exposes a standard V4L2 interface.
> +
> +Summary
> +=======
> +
> +- The MediaTek ISP7x camera system is built on the V4L2 media controller framework, with a modular pipeline of video devices and subdevices.
> +- The kernel driver manages pipeline state, buffer flow, and per-frame control, while delegating hardware programming to the backend firmware via IPI.
> +- The firmware interface is session-based, supports per-frame CQ generation, and abstracts all hardware-specific details from the kernel.
> +- The media controller graph models the hardware pipeline, allowing flexible configuration and dynamic topology management.
> \ No newline at end of file
> diff --git a/Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst b/Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst
> new file mode 100755
> index 000000000000..0e8d41d339e7
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/mtk-isp7x-controls.rst
> @@ -0,0 +1,199 @@
> +==========================================
> +MediaTek ISP Camsys User Space Interface
> +==========================================
> +
> +Overview
> +--------
> +
> +The MediaTek ISP camsys driver provides a set of custom V4L2 controls for user space
> +applications to configure and query the ISP hardware. These controls allow
> +fine-grained management of resource allocation, feature enablement, and
> +hardware-specific settings.
> +
> +Typical Usage Flow
> +------------------
> +
> +1. Open the video device node (e.g., ``/dev/video0``).
> +2. Set required V4L2 controls using ``VIDIOC_S_CTRL`` or ``VIDIOC_S_EXT_CTRLS``.
> +3. Set up video format and buffers (``VIDIOC_S_FMT``, ``VIDIOC_REQBUFS``, etc.).
> +4. Start streaming with ``VIDIOC_STREAMON``.
> +5. Queue and dequeue buffers for image processing (``VIDIOC_QBUF``, ``VIDIOC_DQBUF``).
> +6. Stop streaming with ``VIDIOC_STREAMOFF``.
> +
> +Control List and Documentation
> +-----------------------------
> +
> +Below is a list of custom V4L2 controls provided by the camsys driver, with detailed descriptions.
> +
> +.. list-table::
> + :header-rows: 1
> +
> + * - Control Name
> + - Type
> + - Description
> + - Valid Values / Range
> + - When to Set
> +
> + * - ``V4L2_CID_MTK_CAM_USED_ENGINE_LIMIT``
> + - Integer
> + - Limits the maximum number of RAW engines (hardware pipelines) that can be used for this stream.
> + - 1 ~ 2 (default: 2)
I don't know why you have this interface.
You have V4L2_CID_MTK_CAM_USED_ENGINE which would decide the engine number, why have this interface to limit it?
If I want 1 engine, I just use V4L2_CID_MTK_CAM_USED_ENGINE to set. V4L2_CID_MTK_CAM_USED_ENGINE_LIMIT is redundant.
So drop this.
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_BIN_LIMIT``
> + - Integer
> + - Sets the maximum binning ratio allowed for the stream.
> + - 0 ~ 0xFFF (default: 0)
Is the binning equal to pixel binning [1]?
If it is, does 0 mean no binning effect?
What does the level mean?
I guess is:
The maximum is 16x16 binning, so 0xFFF means 16x16 binning, and 0x800 means 4x4 binning.
This is just my guess, please provide more information what the level mean.
[1] https://en.wikipedia.org/wiki/Pixel_binning
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_FRZ_LIMIT``
> + - Integer
> + - Sets the maximum resizer (FRZ) ratio allowed (as a percentage).
> + - 70 ~ 100 (default: 100)
I don't know why you have this interface.
You have V4L2_CID_MTK_CAM_FRZ which would decide the resize ratio, why have this interface to limit it?
If I want 0.8 ratio, I just use V4L2_CID_MTK_CAM_FRZ to set. V4L2_CID_MTK_CAM_FRZ_LIMIT is redundant.
So drop this.
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_RESOURCE_PLAN_POLICY``
> + - Integer (Enum)
> + - Selects the resource planning policy for the stream.
> + - 0: Default, 1: Power-saving, 2: Performance, ... (0~10)
You just list 0, 1, 2.
What about 3 ~ 10?
Add the rest information.
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_USED_ENGINE``
> + - Integer
> + - Reports or sets the number of RAW engines actually used for this stream after negotiation.
> + - 1 ~ 2 (default: 2)
> + - After resource negotiation
What's the negotiation?
Is it V4L2_CID_MTK_CAM_USED_ENGINE_TRY?
or all XXX_TRY interface?
Please explicitly list all these interface.
> +
> + * - ``V4L2_CID_MTK_CAM_BIN``
> + - Boolean
> + - Enables or disables binning for the stream.
> + - 0 (disable), 1 (enable, default)
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_FRZ``
> + - Integer
> + - Sets the resizer (FRZ) ratio (as a percentage) for the stream.
> + - 70 ~ 100 (default: 100)
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_USED_ENGINE_TRY``
> + - Integer
> + - Reports the number of RAW engines that would be used for the current "try" format negotiation.
> + - 1 ~ 2 (default: 2)
> + - After VIDIOC_TRY_FMT
> +
> + * - ``V4L2_CID_MTK_CAM_BIN_TRY``
> + - Boolean
> + - Reports whether binning would be enabled for the current "try" format negotiation.
> + - 0, 1 (default: 1)
> + - After VIDIOC_TRY_FMT
> +
> + * - ``V4L2_CID_MTK_CAM_FRZ_TRY``
> + - Integer
> + - Reports the resizer ratio that would be used for the current "try" format negotiation.
> + - 70 ~ 100 (default: 100)
> + - After VIDIOC_TRY_FMT
> +
> + * - ``V4L2_CID_MTK_CAM_PIXEL_RATE``
> + - Integer64
> + - Sets the pixel rate (in pixels per second) of the sensor for resource calculation.
> + - 0 ~ 0xFFFFFFFF
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_FEATURE``
> + - Bitmask (Integer64)
> + - Bitmask of enabled features for the stream (e.g., HDR, 3A, etc.). See driver header for details.
In patch [12/13], I does not see any information about these feature.
Add these feature definition in patch [12/13].
> + - Bitmask (0 ~ RAW_FUNCTION_END)
> + - Before streaming or per-frame
> +
> + * - ``V4L2_CID_MTK_CAM_SYNC_ID``
> + - Integer64
> + - Frame synchronization ID for multi-sensor or multi-pipeline synchronization.
Describe how to use this ID for multi-sensor or multi-pipeline.
I think every frame has a timestamp which could be used as ID.
Why not use timestamp instead of this ID?
> + - -1 ~ 0x7FFFFFFF (default: -1)
> + - Before streaming or per-frame
> +
> + * - ``V4L2_CID_MTK_CAM_RAW_PATH_SELECT``
> + - Integer
> + - Selects the RAW path (e.g., main or secondary) for the stream.
> + - 0, 1 (default: 1)
You have draw a graph like this, but it does not show two raw path.
+---------+ +--------+ +--------+ +------+
| Sensor | --> | SENINF | --> | RAW ISP| --> | Video|
+---------+ +--------+ +--------+ +------+
Does it mean this?
+---------+ +--------+ +---------------+ +------+
| Sensor | --> | SENINF | +-> | MAIN RAW ISP | --> | Video|
+---------+ +--------+ | +---------------+ +------+
|
| +---------------+ +------+
+-> | SECOND RAW ISP| --> | Video|
+---------------+ +------+
If so, change your graph.
How do we decide which should we select?
What's the difference of these two RAW?
When we configure camsys driver, both RAW has the same setting or each RAW has different setting?
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_HSF_EN``
> + - Integer
> + - Enables or disables hardware secure flow (HSF) for the stream.
What is HSF?
Does it mean output video buffer is accessible only by trusted CPU or hardware?
Does it mean camsys hardware could be configured only by trusted CPU or firmware?
If so, add this information.
> + - 0 (disable), 1 (enable)
> + - Before streaming
> +
> + * - ``V4L2_CID_MTK_CAM_PDE_INFO``
> + - Integer (struct mtk_cam_pde_info)
> + - Provides or sets phase detection (PDE) information for PDAF/AF features.
> + - See struct definition
After seeing the structure, I still don't understand what it is.
What is pdo? What is pdi? What is pd table offset?
Try to add more information to let us know how to configure it.
> + - Before streaming or per-frame
> +
> + * - ``V4L2_CID_MTK_CAM_MSTREAM_EXPOSURE``
> + - Integer/Integer64
> + - Controls multi-stream exposure settings for advanced HDR or multi-exposure modes.
> + - See driver header
I does not see this in driver header, so drop this.
> + - Before streaming or per-frame
> +
> + * - ``V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC``
> + - Compound (struct mtk_cam_resource)
> + - Triggers resource calculation for the current format and settings.
> + - See struct definition
> + - During format negotiation
Call this between some XXX_TRY interface?
The negotiation interface could be called in a random sequence?
If there are dependencies, explicitly describe it.
> +
> + * - ``V4L2_CID_MTK_CAM_TG_FLASH_CFG``
> + - Integer/struct
> + - Configures timing generator (TG) and flash synchronization.
> + - See driver header
I could not find this in driver, so drop it.
> + - Before streaming or per-frame
> +
> + * - ``V4L2_CID_MTK_CAM_RAW_RESOURCE_UPDATE``
> + - Integer
> + - Notifies the driver of a sensor mode update (e.g., seamless switch).
> + - 0 ~ 0xF (default: 0)
What is sensor mode?
I still don't understand to set value 3 or value 7?
Describe more about how to set these value.
> + - When sensor mode changes
> +
> + * - ``V4L2_CID_MTK_CAM_CAMSYS_HW_MODE``
> + - Integer64
> + - Selects the hardware mode for the CAMSYS pipeline (e.g., normal, secure, etc.).
> + - 0 ~ 0x7FFFFFFF
What is hardware mode?
I still don't understand to set value 3 or value 7?
Describe more about how to set these value.
Do you understand the meaning of upstream?
Upstream is not a cloud server to store your code.
Upstream is to share your code so others could use your code to control this hardware.
Try to let us understand how to use this code to control hardware.
Regards,
CK
> + - Before streaming
> +
> +Control Usage Patterns
> +---------------------
> +
> +- Controls with ``_TRY`` suffix are used to query the result of a "try" format negotiation (``VIDIOC_TRY_FMT``), without committing changes.
> +- Controls without ``_TRY`` are used to set or get the actual configuration (``VIDIOC_S_FMT``, ``VIDIOC_G_CTRL``).
> +- Use ``V4L2_CID_MTK_CAM_RAW_RESOURCE_CALC`` to check if the requested format and settings are feasible, and to get recommended values.
> +- Use ``V4L2_CID_MTK_CAM_RAW_RESOURCE_UPDATE`` to notify the driver of a sensor mode change (e.g., seamless switch).
> +- Use ``V4L2_CID_MTK_CAM_SYNC_ID`` to synchronize frames across multiple pipelines.
> +- Use ``V4L2_CID_MTK_CAM_FEATURE`` to enable/disable advanced features (bitmask).
> +- Use ``V4L2_CID_MTK_CAM_PDE_INFO`` to set/get phase detection information for PDAF/AF.
> +- Use ``V4L2_CID_MTK_CAM_PIXEL_RATE`` to inform the driver of the sensor's pixel rate for bandwidth/resource calculation.
> +
> +Example: Setting a Control
> +--------------------------
> +
> +.. code-block:: c
> +
> + int fd = open("/dev/video0", O_RDWR);
> + struct v4l2_control ctrl = {
> + .id = V4L2_CID_MTK_CAM_BIN_LIMIT,
> + .value = 2,
> + };
> + if (ioctl(fd, VIDIOC_S_CTRL, &ctrl) < 0) {
> + perror("Failed to set BIN_LIMIT");
> + }
> +
> +Notes
> +-----
> +
> +- All controls should be set before starting the video stream unless otherwise specified.
> +- Refer to the driver header file for detailed struct definitions and bitmask values.
> +- The driver will reject invalid values or unsupported configurations with -EINVAL.
> +
> +References
> +----------
> +
> +- V4L2 documentation: https://urldefense.com/v3/__https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/v4l2.html__;!!CTRNKA9wMg0ARbw!koPmGRLtqjG7_WecTl2EBNDMmuZO-_VzYEAbKzsTJhueUGekgy_ky5-FeOERXvI6377LscGrivr2cMlRuXbU3pM$
> \ No newline at end of file
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl
2025-07-07 1:31 ` [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl shangyao lin
@ 2025-07-11 8:56 ` CK Hu (胡俊光)
2025-07-21 1:40 ` CK Hu (胡俊光)
1 sibling, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-11 8:56 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduce state management and debugging mechanisms for the MediaTek ISP7.x
> camsys platform. State management establishes control over ISP operations and
> events, defining distinct states for request handling, sensor control, and
> frame synchronization, ensuring proper event processing. The debugging
> mechanism ensures stable operation and timely data collection during anomalies.
>
> ---
>
> Changes in v2:
> - Removed mtk_cam-debug.c and mtk_cam-debug.h, along with related code
> - Various fixes per review comments
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
[snip]
> +static bool mtk_cam_request_drained(struct mtk_camsys_sensor_ctrl *sensor_ctrl)
> +{
> + struct mtk_cam_ctx *ctx = sensor_ctrl->ctx;
You use struct mtk_cam_ctx in this patch, but it's defined in patch [8/13].
You should define it first and then use it.
But here you use struct mtk_cam_ctx which is defined in later patch.
To prevent this, you could apply one patch and build code once.
For example,
After apply patch [5/13], build code.
After apply patch [6/13], build code.
After apply patch [7/13], build code.
...
The step would help you to prevent using symbol which has not defined.
In addition, this patch just add .c file but not add it in Makefile.
When you add .c, also add it in Makefile so you could check it could be built successfully.
Regards,
CK
> + int sensor_seq_no_next =
> + atomic_read(&sensor_ctrl->sensor_request_seq_no) + 1;
> + int res = 0;
> +
> + if (sensor_seq_no_next <= atomic_read(&ctx->enqueued_frame_seq_no))
> + res = 1;
> + /* Send V4L2_EVENT_REQUEST_DRAINED event */
> + if (res == 0) {
> + mtk_cam_event_request_drained(ctx->pipe);
> + dev_dbg(ctx->cam->dev, "request_drained:(%d)\n",
> + sensor_seq_no_next);
> + }
> + return (res == 0);
> +}
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 07/13] MEDIA: PLATFORM: MEDIATEK: ADD ISP_7X CAM-RAW UNIT
[not found] ` <20250707013154.4055874-8-shangyao.lin@mediatek.com>
@ 2025-07-15 4:07 ` CK Hu (胡俊光)
0 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-15 4:07 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduces the ISP pipeline driver for the MediaTek ISP raw and yuv
> modules. Key functionalities include data processing, V4L2 integration,
> resource management, debug support, and various control operations.
> Additionally, IRQ handling, platform device management, and MediaTek
> ISP DMA format support are also included.
>
> Changes in v2:
>
> - Removed mtk_cam-raw.c and mtk_cam-raw.h, along with related code
> - Removed M2M related code
> - Various fixes per review comments
>
> Question for CK
>
> Hi CK,
>
> May I ask if it is acceptable to keep the id field for the following reason?
> When there is more than one RAW engine on the platform, having an id makes it
> much easier to manage and coordinate different RAW pipelines. It allows us to
> correctly associate events and interrupts with the corresponding RAW device
> context, which improves code maintainability and scalability.
>
> Would you agree with keeping the id for this purpose, or do you have a preferred
> alternative?
You could keep it now. I would try to review where use this id.
If every where is not necessary, this id would finally be removed.
>
> Explanation:
>
> Reply to CK: "Remove resource calculation related code"
>
> Thank you for your suggestion. The resource calculation code is retained even
> for the unprocessed IMGO path. All image streams, including IMGO unprocessed,
> pass through the raw pipeline and require proper ISP hardware configuration.
It's better to have a comment to explain the pipeline and function block in the pipeline.
When more information, I would give better advice.
>
> Reply to CK: "It seems yuv is an independent function. If so, separate yuv
> function to an independent patch."
>
> Thank you for your comment. The raw and yuv functions are not separated into
> independent patches because, in our hardware design, both are handled by the
> same hardware block and share the same register set. The yuv function is just a
> different mode of operation within the same unit, not an independent hardware
> module. Splitting them would not reflect the actual hardware architecture and
> could make the code harder to maintain.
I just want you to separate yuv function to another patch.
You could add it back in later patch.
Finally the code would be the same as now.
Finally it would show hardware architecture and you could maintain the same code.
>
> Please let me know if you have any further suggestions. Thank you!
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
[snip]
> + {
> + .id = MTK_RAW_RAWI_2_IN,
> + .name = "rawi 2",
> + .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE,
> + .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
For camera, capture device is basic function and output device is advanced function.
If this series just keep basic function, drop the output device code.
Regards,
CK
> + .link_flags = MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE,
> + .image = true,
> + .smem_alloc = false,
> + .dma_port = MTKCAM_ISP_RAWI_2,
> + .fmts = stream_out_fmts,
> + .num_fmts = ARRAY_SIZE(stream_out_fmts),
> + .default_fmt_idx = 0,
> + .ioctl_ops = &mtk_cam_v4l2_vout_ioctl_ops,
> + .frmsizes = &(struct v4l2_frmsizeenum) {
> + .index = 0,
> + .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> + .stepwise = {
> + .max_width = IMG_MAX_WIDTH,
> + .min_width = IMG_MIN_WIDTH,
> + .max_height = IMG_MAX_HEIGHT,
> + .min_height = IMG_MIN_HEIGHT,
> + .step_height = 1,
> + .step_width = 1,
> + },
> + },
> + },
> + {
> + .id = MTK_RAW_RAWI_3_IN,
> + .name = "rawi 3",
> + .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE,
> + .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + .link_flags = 0,
> + .image = true,
> + .smem_alloc = false,
> + .dma_port = MTKCAM_ISP_RAWI_3,
> + .fmts = stream_out_fmts,
> + .num_fmts = ARRAY_SIZE(stream_out_fmts),
> + .default_fmt_idx = 0,
> + .ioctl_ops = &mtk_cam_v4l2_vout_ioctl_ops,
> + .frmsizes = &(struct v4l2_frmsizeenum) {
> + .index = 0,
> + .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> + .stepwise = {
> + .max_width = IMG_MAX_WIDTH,
> + .min_width = IMG_MIN_WIDTH,
> + .max_height = IMG_MAX_HEIGHT,
> + .min_height = IMG_MIN_HEIGHT,
> + .step_height = 1,
> + .step_width = 1,
> + },
> + },
> + },
> + {
> + .id = MTK_RAW_RAWI_4_IN,
> + .name = "rawi 4",
> + .cap = V4L2_CAP_VIDEO_OUTPUT_MPLANE,
> + .buf_type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> + .link_flags = 0,
> + .image = true,
> + .smem_alloc = false,
> + .dma_port = MTKCAM_ISP_RAWI_3, /* align backend RAWI_3 */
> + .fmts = stream_out_fmts,
> + .num_fmts = ARRAY_SIZE(stream_out_fmts),
> + .default_fmt_idx = 0,
> + .ioctl_ops = &mtk_cam_v4l2_vout_ioctl_ops,
> + .frmsizes = &(struct v4l2_frmsizeenum) {
> + .index = 0,
> + .type = V4L2_FRMSIZE_TYPE_CONTINUOUS,
> + .stepwise = {
> + .max_width = IMG_MAX_WIDTH,
> + .min_width = IMG_MIN_WIDTH,
> + .max_height = IMG_MAX_HEIGHT,
> + .min_height = IMG_MIN_HEIGHT,
> + .step_height = 1,
> + .step_width = 1,
> + },
> + },
> + }
> +};
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding
2025-07-07 1:31 ` [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:44 ` Krzysztof Kozlowski
@ 2025-07-15 7:28 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-15 7:28 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> 1. Add camera isp7x module device document
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-seninf-core.yaml
> - Various fixes per review comments
> - Update maintainers list
>
> Question for reviewer (CK):
>
> Hi CK,
>
> Thank you for your review and suggestions on this patch, especially for providing the reference patch (https://urldefense.com/v3/__https://patchwork.kernel.org/project/linux-mediatek/list/?series=874617__;!!CTRNKA9wMg0ARbw!lK62bVQ9-iOIdvsqy3Y0MZIGQl5Szp23K-rB_IB5uffV-tfbML9p144Cgt-4xTpgeYQWungRDBzhfuY1fYflBnk$ ) and for mentioning in another patch ([V1,02/10] MEDIA: PLATFORM: MEDIATEK: ADD SENINF CONTROLLER) the suggestion to "Move the phy part to phy/mediatek/ folder. You could refer to phy/mediatek/phy-mtk-mipi-csi-0-5.c".
>
> After reading your comments and the reference patches, my understanding is that only the seninf-core driver should manage all ports internally, and each port corresponds to a PHY. During probe, the driver will parse each port, obtain the corresponding PHY (e.g., devm_phy_get(dev, "csi0"), devm_phy_get(dev, "csi1"), etc.), and operate the PHY for each port individually during stream on/off or power on/off.
>
> Could you please confirm if my understanding is correct?
> If you have any additional reference patches or examples, I would greatly appreciate it.
I'm not expert on seninf and I find previous link [1] is invalid now.
I provide [2] here for your reference.
I don't know the relation of port and phy. You may find some hint in that link.
[1] https://patchwork.kernel.org/project/linux-mediatek/list/?series=874617
[2] https://patchwork.kernel.org/project/linux-mediatek/patch/20241121-add-mtk-isp-3-0-support-v7-3-b04dc9610619@baylibre.com/
>
> Thank you for your guidance!
>
> Best regards,
> Shangyao
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
[snip]
> +
> +examples:
> + - |
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> + #include <dt-bindings/interrupt-controller/irq.h>
> + #include <dt-bindings/power/mediatek,mt8188-power.h>
> + #include <dt-bindings/clock/mediatek,mt8188-clk.h>
> +
> + seninf@16010000 {
> + compatible = "mediatek,mt8188-seninf-core";
> + reg = <0 0x16010000 0 0x8000>;
> + reg-names = "base";
> + mtk_csi_phy_ver = "mtk_csi_phy_2_0";
mtk_csi_phy_ver is not necessary. compatible imply this.
So drop this.
Regards,
CK
> + interrupts = <GIC_SPI 316 IRQ_TYPE_LEVEL_HIGH 0>;
> + power-domains = <&spm MT8188_POWER_DOMAIN_CSIRX_TOP>,
> + <&spm MT8188_POWER_DOMAIN_CAM_VCORE>,
> + <&spm MT8188_POWER_DOMAIN_CAM_MAIN>;
> + clocks = <&camsys CLK_CAM_MAIN_SENINF>,
> + <&topckgen CLK_TOP_SENINF>,
> + <&topckgen CLK_TOP_SENINF1>,
> + <&topckgen CLK_TOP_CAMTM>;
> + clock-names = "clk_cam_seninf",
> + "clk_top_seninf",
> + "clk_top_seninf1",
> + "clk_top_camtm";
> + };
> +
> +...
> \ No newline at end of file
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding
2025-07-07 1:31 ` [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:47 ` Krzysztof Kozlowski
@ 2025-07-15 7:41 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-15 7:41 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-cam-yuv.yaml
> - Various fixes per review comments
> - Update maintainers list
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/mediatek,mt8188-cam-yuv.yaml | 134 ++++++++++++++++++
> 1 file changed, 134 insertions(+)
> create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
> new file mode 100755
> index 000000000000..0de120d3c6e8
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-cam-yuv.yaml
> @@ -0,0 +1,134 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +# Copyright (c) 2024 MediaTek Inc.
> +%YAML 1.2
> +---
> +$id: https://urldefense.com/v3/__http://devicetree.org/schemas/media/mediatek/mediatek,cam-yuv.yaml*__;Iw!!CTRNKA9wMg0ARbw!m5llIhxb52atfbJu7JZcPOjWHR2ypTsl8hAhZbX5kM91Vn1dPv9RPDCXHxNwMJDLf05csXuKqchuR_f0YjO6FS8$
> +$schema: https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!m5llIhxb52atfbJu7JZcPOjWHR2ypTsl8hAhZbX5kM91Vn1dPv9RPDCXHxNwMJDLf05csXuKqchuR_f0cW6r7vU$
> +
> +title: The cam-yuv unit of MediaTek ISP system
> +
> +maintainers:
> + - Shangyao Lin <shangyao.lin@mediatek.com>
> + - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
> + - Shun-yi Wang <shun-yi.wang@mediatek.com>
> + - Teddy Chen <teddy.chen@mediatek.com>
> +
> +description:
> + MediaTek cam-yuv is the camera YUV processing unit in MediaTek SoC.
> +
> +properties:
> + compatible:
> + const: mediatek,mt8188-cam-yuv
In patch [7/13], you say yuv and raw use the same hardware with difference software configuration.
So you should not have different binding for yuv and raw.
Use one (maybe mediatek,mt8188-cam-raw) for both yuv and raw.
In your driver, try another way to decide it works as yuv or raw.
Regards,
CK
> +
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding
2025-07-07 1:31 ` [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:48 ` Krzysztof Kozlowski
@ 2025-07-15 8:13 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-15 8:13 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
[snip]
> +$id: https://urldefense.com/v3/__http://devicetree.org/schemas/media/mediatek/mediatek,cam-raw.yaml*__;Iw!!CTRNKA9wMg0ARbw!huoB5nm0-eewqNovfUlZpTbH1XdMjr0yWt3-_bNow3GppcMua0_ihDD4W1-due6CzcSdZJqTKs9hUNrWwljl0mM$
> +$schema: https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!huoB5nm0-eewqNovfUlZpTbH1XdMjr0yWt3-_bNow3GppcMua0_ihDD4W1-due6CzcSdZJqTKs9hUNrWTOQ4FiY$
> +
> +title: The cam-raw unit of MediaTek ISP system
> +
> +maintainers:
> + - Shangyao Lin <shangyao.lin@mediatek.com>
> + - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
> + - Shun-yi Wang <shun-yi.wang@mediatek.com>
> + - Teddy Chen <teddy.chen@mediatek.com>
> +
> +description:
> + MediaTek cam-raw is the camera RAW processing unit in MediaTek SoC.
After reading this description, I still don't understand what is cam-raw.
Describe more here.
What's the main function of it?
It has many iommu port, introduce the function of each port.
Regards,
CK
> +
> +properties:
> + compatible:
> + const: mediatek,mt8188-cam-raw
> +
> + reg:
> + minItems: 1
> + maxItems: 2
> + description:
> + Base address and optional inner base address of the cam-raw hardware block.
> +
> + reg-names:
> + items:
> + - const: base
> + - const: inner_base
> + minItems: 1
> + maxItems: 2
> + description:
> + Names for each register region. Must be "base" and optionally "inner_base".
> +
> + mediatek,larbs:
> + description:
> + List of phandles to the local arbiters in the current SoCs.
> + Refer to bindings/memory-controllers/mediatek,smi-larb.yaml.
> + $ref: /schemas/types.yaml#/definitions/phandle-array
> + minItems: 1
> + maxItems: 32
> +
> + interrupts:
> + minItems: 1
> + description: Interrupts for the cam-raw block.
> +
> + dma-ranges:
> + minItems: 1
> + description: Address information of IOMMU mapping to memory.
> +
> + power-domains:
> + minItems: 1
> + description: Power domains for the cam-raw block.
> +
> + clocks:
> + minItems: 4
> + maxItems: 16
> + description: List of phandles to the clocks required by the cam-raw block.
> +
> + clock-names:
> + items:
> + - const: camsys_cam2mm0_cgpdn
> + - const: camsys_cam2mm1_cgpdn
> + - const: camsys_cam2sys_cgpdn
> + - const: camsys_cam_cgpdn
> + - const: camsys_camtg_cgpdn
> + - const: camsys_rawa_larbx_cgpdn
> + - const: camsys_rawa_cam_cgpdn
> + - const: camsys_rawa_camtg_cgpdn
> + - const: topckgen_top_cam
> + - const: topckgen_top_camtg
> + - const: topckgen_top_camtm
> + minItems: 4
> + maxItems: 16
> + description: Names of the clocks, must match the order of the clocks property.
> +
> + iommus:
> + minItems: 1
> + maxItems: 32
> + description: Points to the respective IOMMU block with master port as argument.
> +
> +required:
> + - compatible
> + - reg
> + - reg-names
> + - interrupts
> + - power-domains
> + - clocks
> + - clock-names
> + - iommus
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> + #include <dt-bindings/interrupt-controller/irq.h>
> + #include <dt-bindings/power/mediatek,mt8188-power.h>
> + #include <dt-bindings/clock/mediatek,mt8188-clk.h>
> + #include <dt-bindings/memory/mediatek,mt8188-memory-port.h>
> +
> + soc {
> + raw@16030000 {
> + compatible = "mediatek,mt8188-cam-raw";
> + reg = <0 0x16030000 0 0x8000>,
> + <0 0x16038000 0 0x8000>;
> + reg-names = "base", "inner_base";
> + interrupts = <GIC_SPI 300 IRQ_TYPE_LEVEL_HIGH 0>;
> + dma-ranges = <0x2 0x0 0x0 0x40000000 0x1 0x0>;
> + power-domains = <&spm MT8188_POWER_DOMAIN_CAM_SUBA>;
> + clocks = <&camsys CLK_CAM_MAIN_CAM2MM0_GALS>,
> + <&camsys CLK_CAM_MAIN_CAM2MM1_GALS>,
> + <&camsys CLK_CAM_MAIN_CAM2SYS_GALS>,
> + <&camsys CLK_CAM_MAIN_CAM>,
> + <&camsys CLK_CAM_MAIN_CAMTG>,
> + <&camsys_rawa CLK_CAM_RAWA_LARBX>,
> + <&camsys_rawa CLK_CAM_RAWA_CAM>,
> + <&camsys_rawa CLK_CAM_RAWA_CAMTG>,
> + <&topckgen CLK_TOP_CAM>,
> + <&topckgen CLK_TOP_CAMTG>,
> + <&topckgen CLK_TOP_CAMTM>;
> + clock-names = "camsys_cam2mm0_cgpdn",
> + "camsys_cam2mm1_cgpdn",
> + "camsys_cam2sys_cgpdn",
> + "camsys_cam_cgpdn",
> + "camsys_camtg_cgpdn",
> + "camsys_rawa_larbx_cgpdn",
> + "camsys_rawa_cam_cgpdn",
> + "camsys_rawa_camtg_cgpdn",
> + "topckgen_top_cam",
> + "topckgen_top_camtg",
> + "topckgen_top_camtm";
> + iommus = <&vpp_iommu M4U_PORT_L16A_IMGO_R1>,
> + <&vpp_iommu M4U_PORT_L16A_CQI_R1>,
> + <&vpp_iommu M4U_PORT_L16A_CQI_R2>,
> + <&vpp_iommu M4U_PORT_L16A_BPCI_R1>,
> + <&vpp_iommu M4U_PORT_L16A_LSCI_R1>,
> + <&vpp_iommu M4U_PORT_L16A_RAWI_R2>,
> + <&vpp_iommu M4U_PORT_L16A_RAWI_R3>,
> + <&vpp_iommu M4U_PORT_L16A_UFDI_R2>,
> + <&vpp_iommu M4U_PORT_L16A_UFDI_R3>,
> + <&vpp_iommu M4U_PORT_L16A_RAWI_R4>,
> + <&vpp_iommu M4U_PORT_L16A_RAWI_R5>,
> + <&vpp_iommu M4U_PORT_L16A_AAI_R1>,
> + <&vpp_iommu M4U_PORT_L16A_UFDI_R5>,
> + <&vpp_iommu M4U_PORT_L16A_FHO_R1>,
> + <&vpp_iommu M4U_PORT_L16A_AAO_R1>,
> + <&vpp_iommu M4U_PORT_L16A_TSFSO_R1>,
> + <&vpp_iommu M4U_PORT_L16A_FLKO_R1>;
> + };
> + };
> +
> +...
> \ No newline at end of file
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:42 ` Krzysztof Kozlowski
@ 2025-07-15 8:25 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-15 8:25 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Add camera isp7x module device document.
>
> ---
>
> Changes in v2:
> - Rename binding file to mediatek,mt8188-camisp.yaml
> - Split bindings into four separate patches (one per YAML file)
> - Various fixes per review comments
> - Update maintainers list
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/mediatek,mt8188-camisp.yaml | 68 +++++++++++++++++++
> 1 file changed, 68 insertions(+)
> create mode 100755 Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
> new file mode 100755
> index 000000000000..53dbf5152e78
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/mediatek/mediatek,mt8188-camisp.yaml
> @@ -0,0 +1,68 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +# Copyright (c) 2024 MediaTek Inc.
> +%YAML 1.2
> +---
> +$id: https://urldefense.com/v3/__http://devicetree.org/schemas/media/mediatek/mediatek,camisp.yaml*__;Iw!!CTRNKA9wMg0ARbw!lF1heQiKVAbFIXXXQfR2iaCncLFFZQ6cRvhFJYGAj3MUbQkTc7AvH3yPBQjlsJy3RzRtOShBXxc4egYMq4Eka6A$
> +$schema: https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!lF1heQiKVAbFIXXXQfR2iaCncLFFZQ6cRvhFJYGAj3MUbQkTc7AvH3yPBQjlsJy3RzRtOShBXxc4egYMBse--Nk$
> +
> +title: The camisp unit of MediaTek ISP system
> +
> +maintainers:
> + - Shangyao Lin <shangyao.lin@mediatek.com>
> + - Shu-hsiang Yang <shu-hsiang.yang@mediatek.com>
> + - Shun-yi Wang <shun-yi.wang@mediatek.com>
> + - Teddy Chen <teddy.chen@mediatek.com>
> +
> +description:
> + MediaTek camisp is the ISP auxiliary unit for camera system in MediaTek SoC.
> +
> +properties:
> + compatible:
> + const: mediatek,mt8188-camisp
> +
> + reg:
> + minItems: 1
> + maxItems: 1
> + description:
> + Base address of the camisp hardware block.
> +
> + reg-names:
> + items:
> + - const: base
> + minItems: 1
> + maxItems: 1
> + description:
> + Name for the register region. Must be "base".
> +
> + power-domains:
> + minItems: 1
> + maxItems: 1
> + description: Power domain for the camisp block.
> +
> + mediatek,scp:
> + description: MediaTek co-process unit for ISP system
> + $ref: /schemas/types.yaml#/definitions/phandle
> +
> +required:
> + - compatible
> + - reg
> + - reg-names
> + - power-domains
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/power/mediatek,mt8188-power.h>
> +
> + soc {
> + isp@16000000 {
> + compatible = "mediatek,mt8188-camisp";
In [1], I has already says in the same address, it should not have more than one device with different compatible.
Follow [1] to modify.
[1] https://patchwork.kernel.org/project/linux-mediatek/patch/20241009111551.27052-2-Shu-hsiang.Yang@mediatek.com/#26114958
Regards,
CK
> + reg = <0 0x16000000 0 0x1000>;
> + reg-names = "base";
> + mediatek,scp = <&scp_dual>;
> + power-domains = <&spm MT8188_POWER_DOMAIN_CAM_MAIN>;
> + };
> + };
> +
> +...
> \ No newline at end of file
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 05/13] media: platform: mediatek: add isp_7x seninf unit
[not found] ` <20250707013154.4055874-6-shangyao.lin@mediatek.com>
2025-07-07 5:50 ` [PATCH v2 05/13] media: platform: mediatek: add isp_7x seninf unit Krzysztof Kozlowski
2025-07-10 6:39 ` CK Hu (胡俊光)
@ 2025-07-16 4:06 ` CK Hu (胡俊光)
2 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-16 4:06 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduce support for the MediaTek sensor interface (seninf) in the SoC camera
> subsystem, focusing on CSI and stream control. The driver manages parameter
> control, metering, status tracking, and interrupt handling for the camera sensor
> interface hardware. It integrates with the MediaTek ISP CAMSYS, bridging camera
> sensors and the ISP system, and provides V4L2 framework support for dynamic
> stream configuration and virtual channel management.
>
> ---
[snip]
> .../isp_7x/camsys/kd_imgsensor_define_v4l2.h | 86 +
> .../isp/isp_7x/camsys/mtk_cam-seninf-def.h | 237 ++
> .../isp/isp_7x/camsys/mtk_cam-seninf-drv.c | 1441 ++++++++++++
> .../isp/isp_7x/camsys/mtk_cam-seninf-drv.h | 16 +
> .../isp/isp_7x/camsys/mtk_cam-seninf-hw.h | 108 +
> .../isp/isp_7x/camsys/mtk_cam-seninf-if.h | 24 +
> .../isp/isp_7x/camsys/mtk_cam-seninf-regs.h | 44 +
> .../isp/isp_7x/camsys/mtk_cam-seninf-route.c | 279 +++
> .../isp/isp_7x/camsys/mtk_cam-seninf-route.h | 20 +
> .../isp/isp_7x/camsys/mtk_cam-seninf.h | 161 ++
Add Makefile to build mtk-cam-seninfo-xxx.c in this patch.
Make this patch could be built.
> .../isp_7x/camsys/mtk_csi_phy_2_0/Makefile | 5 +
> .../mtk_csi_phy_2_0/mtk_cam-seninf-cammux.h | 679 ++++++
> .../mtk_cam-seninf-csi0-cphy.h | 51 +
> .../mtk_cam-seninf-csi0-dphy.h | 98 +
> .../mtk_cam-seninf-hw_phy_2_0.c | 1932 +++++++++++++++++
> .../mtk_cam-seninf-mipi-rx-ana-cdphy-csi0a.h | 165 ++
> .../mtk_cam-seninf-seninf1-csi2.h | 264 +++
> .../mtk_cam-seninf-seninf1-mux.h | 120 +
> .../mtk_csi_phy_2_0/mtk_cam-seninf-seninf1.h | 46 +
> .../mtk_csi_phy_2_0/mtk_cam-seninf-tg1.h | 43 +
> .../mtk_csi_phy_2_0/mtk_cam-seninf-top-ctrl.h | 76 +
> 21 files changed, 5895 insertions(+)
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/kd_imgsensor_define_v4l2.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-def.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-drv.c
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-drv.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-hw.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-if.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-regs.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-route.c
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-route.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/Makefile
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-cammux.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-csi0-cphy.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-csi0-dphy.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-hw_phy_2_0.c
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-mipi-rx-ana-cdphy-csi0a.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-seninf1-csi2.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-seninf1-mux.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-seninf1.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-tg1.h
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_csi_phy_2_0/mtk_cam-seninf-top-ctrl.h
>
> diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/kd_imgsensor_define_v4l2.h b/drivers/media/platform/mediatek/isp/isp_7x/camsys/kd_imgsensor_define_v4l2.h
> new file mode 100755
> index 000000000000..54c23afe64bd
> --- /dev/null
> +++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/kd_imgsensor_define_v4l2.h
> @@ -0,0 +1,86 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2019 MediaTek Inc.
> + */
> +
> +#ifndef _KD_IMGSENSOR_DATA_V4L2_H
> +#define _KD_IMGSENSOR_DATA_V4L2_H
> +
> +enum VC_FEATURE {
Because mtk_cam_seninf_get_vc_feature() is useless, so VC_FEATURE is useless.
Drop these.
> + VC_NONE = 0,
> + VC_MIN_NUM,
> + VC_RAW_DATA = VC_MIN_NUM,
> + VC_RAW_DATA_MAX,
> +
> + VC_PDAF_MIN_NUM = VC_RAW_DATA_MAX,
> + VC_PDAF_STATS = VC_PDAF_MIN_NUM,
> + VC_PDAF_STATS_NE,
> + VC_PDAF_STATS_ME,
> + VC_PDAF_STATS_SE,
> + VC_PDAF_STATS_PIX_1,
> + VC_PDAF_STATS_PIX_2,
> + VC_PDAF_STATS_NE_PIX_1 = VC_PDAF_STATS_PIX_1,
> + VC_PDAF_STATS_NE_PIX_2 = VC_PDAF_STATS_PIX_2,
> + VC_PDAF_STATS_ME_PIX_1,
> + VC_PDAF_STATS_ME_PIX_2,
> + VC_PDAF_STATS_SE_PIX_2,
> + VC_PDAF_MAX_NUM,
> +
> + VC_HDR_MIN_NUM = VC_PDAF_MAX_NUM,
> + VC_HDR_MVHDR = VC_HDR_MIN_NUM,
> + VC_HDR_MAX_NUM,
> +
> + VC_3HDR_MIN_NUM = VC_HDR_MAX_NUM,
> + VC_3HDR_EMBEDDED = VC_3HDR_MIN_NUM,
> + VC_3HDR_FLICKER,
> + VC_3HDR_Y,
> + VC_3HDR_AE,
> + VC_3HDR_MAX_NUM,
> +
> + VC_STAGGER_MIN_NUM = VC_3HDR_MAX_NUM,
> + VC_STAGGER_EMBEDDED = VC_STAGGER_MIN_NUM,
> + VC_STAGGER_NE,
> + VC_STAGGER_ME,
> + VC_STAGGER_SE,
> + VC_STAGGER_MAX_NUM,
> +
> + VC_YUV_MIN_NUM = VC_STAGGER_MAX_NUM,
> + VC_YUV_Y = VC_YUV_MIN_NUM,
> + VC_YUV_UV,
> + VC_YUV_MAX_NUM,
> +
> + VC_RAW_EXT_MIN_NUM = VC_YUV_MAX_NUM,
> + VC_RAW_W_DATA = VC_RAW_EXT_MIN_NUM,
> + VC_RAW_PROCESSED_DATA,
> + VC_RAW_EXT_MAX_NUM,
> +
> + VC_GENERAL_DATA_MIN_NUM = VC_RAW_EXT_MAX_NUM,
> + VC_GENERAL_EMBEDDED = VC_GENERAL_DATA_MIN_NUM,
> + VC_GENERAL_DATA_MAX_NUM,
> +
> + VC_MAX_NUM = VC_GENERAL_DATA_MAX_NUM,
> +};
> +
[snip]
> +
> +#endif /* _KD_IMGSENSOR_DATA_H */
> diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-def.h b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-def.h
> new file mode 100755
> index 000000000000..39f6005c8227
> --- /dev/null
> +++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-seninf-def.h
> @@ -0,0 +1,237 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +// Copyright (c) 2022 MediaTek Inc.
> +
> +#ifndef __MTK_CAM_SENINF_DEF_H__
> +#define __MTK_CAM_SENINF_DEF_H__
> +
> +#include <linux/bitfield.h>
> +
> +#define SENINF_VC_MAXCNT 8
> +#define SENINF_DEF_PIXEL_MODE 2
> +
> +#define SENINF_CLK_MARGIN_IN_PERCENT 0
> +#define HW_BUF_EFFECT 10
> +
> +#define SENINF_HS_TRAIL_EN_CONDITION 1450000000
SENINF_HS_TRAIL_EN_CONDITION is useless. Drop it.
> +#define SENINF_TIMESTAMP_CLK 1000
SENINF_TIMESTAMP_CLK is useless. Drop it.
> +#define ISP_CLK_LOW 273000000
> +
> +#define SENINF_CPHY_SETTLE_DELAY_DT 0x10
> +#define SENINF_DPHY_SETTLE_DELAY_DT 0x10
> +#define SENINF_SETTLE_DELAY_CK 0x11
> +#define SENINF_HS_TRAIL_PARAMETER 0x34
> +
> +#define SENSOR_CLOCK_POLARITY_HIGH 0
SENSOR_CLOCK_POLARITY_HIGH is useless. Drop it.
> +#define SENSOR_CLOCK_POLARITY_LOW 1
SENSOR_CLOCK_POLARITY_LOW is useless. Drop it.
> +#define NUM_PORTS 2
NUM_PORTS is useless. Drop it.
> +#define DEFAULT_WIDTH 1600
> +#define DEFAULT_HEIGHT 1200
> +#define CSI_PORT_A_MASK GENMASK(3, 1)
> +#define SENINF_MAX_STREAM 7
> +
[snip]
> +
> +enum SENINF_PHY_VER_ENUM {
> + SENINF_PHY_2_0,
> + SENINF_PHY_VER_NUM,
> +};
SENINF_PHY_VER_ENUM, SENINF_PHY_2_0 and SENINF_PHY_VER_NUM are useless.
Drop them.
> +
> +#define MTK_CSI_PHY_VERSIONS "mtk_csi_phy_2_0"
> +
[snip]
> +
> +static const char * const csi_port_names[] = {
> + SENINF_CSI_PORT_NAMES
Drop SENINF_CSI_PORT_NAMES and place string here.
> +};
> +
> +static const char * const clk_names[] = {
> + SENINF_CLK_NAMES
Drop SENINF_CLK_NAMES and place string here.
> +};
> +
> +static const char * const set_reg_names[] = {
> + SET_REG_KEYS_NAMES
Drop SET_REG_KEYS_NAMES and place string here.
> +};
> +
> +enum REG_OPS_CMD {
> + REG_OPS_CMD_ID,
> + REG_OPS_CMD_CSI,
> + REG_OPS_CMD_RG,
> + REG_OPS_CMD_VAL,
> + REG_OPS_CMD_MAX_NUM,
> +};
REG_OPS_CMD_XXX is not defined. Drop all of them.
> +
[snip]
> +
> +static int seninf_core_probe(struct platform_device *pdev)
> +{
> + int i, ret, irq;
> + struct resource *res;
> + struct seninf_core *core;
> + struct device *dev = &pdev->dev;
> +
> + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL);
> + if (!core)
> + return -ENOMEM;
> +
> + dev_set_drvdata(dev, core);
> + core->dev = dev;
> + mutex_init(&core->mutex);
> + INIT_LIST_HEAD(&core->list);
> +
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "base");
> + core->reg_if = devm_ioremap_resource(dev, res);
> + if (IS_ERR(core->reg_if))
> + return PTR_ERR(core->reg_if);
> +
> + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "ana-rx");
"ana-rx" is not defined in binding document. Drop this.
> + core->reg_ana = devm_ioremap_resource(dev, res);
> + if (IS_ERR(core->reg_ana))
> + return PTR_ERR(core->reg_ana);
> +
> + ret = get_seninf_cfg(dev, core);
> + if (ret) {
> + dev_err(dev, "failed to get seninf ops\n");
> + return ret;
> + }
> + mtk_cam_seninf_init_res(core);
> +
> + spin_lock_init(&core->spinlock_irq);
> + irq = platform_get_irq(pdev, 0);
> + if (!irq) {
> + dev_err(dev, "failed to get irq\n");
> + return -ENODEV;
> + }
> +
> + ret = devm_request_irq(dev, irq, mtk_irq_seninf, 0,
> + dev_name(dev), core);
> + if (ret) {
> + dev_err(dev, "failed to request irq=%d\n", irq);
> + return ret;
> + }
> +
> + /* default platform properties */
> + core->cphy_settle_delay_dt = SENINF_CPHY_SETTLE_DELAY_DT;
> + core->dphy_settle_delay_dt = SENINF_DPHY_SETTLE_DELAY_DT;
> + core->settle_delay_ck = SENINF_SETTLE_DELAY_CK;
> + core->hs_trail_parameter = SENINF_HS_TRAIL_PARAMETER;
> +
> + /* read platform properties from device tree */
> + of_property_read_u32(dev->of_node, "cphy_settle_delay_dt",
> + &core->cphy_settle_delay_dt);
"cphy_settle_delay_dt" is not defined in binding document. Drop this.
> + of_property_read_u32(dev->of_node, "dphy_settle_delay_dt",
> + &core->dphy_settle_delay_dt);
"dphy_settle_delay_dt" is not defined in binding document. Drop this.
> + of_property_read_u32(dev->of_node, "settle_delay_ck",
> + &core->settle_delay_ck);
"settle_delay_ck" is not defined in binding document. Drop this.
> + of_property_read_u32(dev->of_node, "hs_trail_parameter",
> + &core->hs_trail_parameter);
"hs_trail_parameter" is not defined in binding document. Drop this.
> +
> + core->dfs.cnt = 0;
> + core->dfs.reg = NULL;
> +
> + ret = seninf_core_pm_runtime_enable(core);
> + if (ret) {
> + dev_info(dev, "failed to enable seninf_core_pm_runtime\n");
> + return ret;
> + }
> +
> + for (i = 0; i < CLK_MAXCNT; i++) {
> + core->clk[i] = devm_clk_get(dev, clk_names[i]);
> + if (IS_ERR(core->clk[i])) {
> + dev_info(dev, "failed to get %s\n", clk_names[i]);
> + core->clk[i] = NULL;
> + /* ignore not define seninf */
> + }
> + }
> +
> + ret = of_platform_populate(dev->of_node, NULL, NULL, dev);
It seems "mediatek,mt8188-seninf-core" would probe "mediatek,mt8188-seninf",
but I does not find any relation between them.
How "mediatek,mt8188-seninf-core" proble "mediatek,mt8188-seninf"?
> + if (ret) {
> + dev_info(dev, "%s: failed to create sub devices\n", __func__);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
[snip]
> +
> +int update_isp_clk(struct seninf_ctx *ctx)
> +{
> + int i, pixelmode;
> + struct seninf_dfs *dfs = &ctx->core->dfs;
> + s64 pixel_rate = -1;
> + u64 dfs_freq;
> + struct seninf_vc *vc;
> + int ret = 0;
> +
> + if (!dfs->cnt) {
dfs->cnt is always zero, so drop dfs and drop this function.
> + dev_info(ctx->dev, "dfs not ready.\n");
> + return ret;
> + }
> +
> + vc = mtk_cam_seninf_get_vc_by_pad(ctx, PAD_SRC_RAW0);
> + if (!vc) {
> + dev_info(ctx->dev, "failed to get vc\n");
> + return -1;
> + }
> + dev_info(ctx->dev,
> + "%s dfs->cnt %d pixel mode %d customized_pixel_rate %lld, buffered_pixel_rate %lld mipi_pixel_rate %lld\n",
> + __func__, dfs->cnt, vc->pixel_mode, ctx->customized_pixel_rate,
> + ctx->buffered_pixel_rate, ctx->mipi_pixel_rate);
> +
> + /* Use SensorPixelrate */
> + if (ctx->customized_pixel_rate) {
> + pixel_rate = ctx->customized_pixel_rate;
> + } else if (ctx->buffered_pixel_rate) {
> + pixel_rate = ctx->buffered_pixel_rate;
> + } else if (ctx->mipi_pixel_rate) {
> + pixel_rate = ctx->mipi_pixel_rate;
> + } else {
> + dev_info(ctx->dev, "failed to get pixel_rate\n");
> + return -EINVAL;
> + }
> +
> + pixelmode = vc->pixel_mode;
> + for (i = 0; i < dfs->cnt; i++) {
> + dfs_freq = dfs->freqs[i];
> + dfs_freq = dfs_freq * (100 - SENINF_CLK_MARGIN_IN_PERCENT);
> + do_div(dfs_freq, 100);
> + if ((dfs_freq << pixelmode) >= pixel_rate)
> + break;
> + }
> +
> + if (i == dfs->cnt) {
> + dev_info(ctx->dev, "mux is overrun. please adjust pixelmode\n");
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
[snip]
> +
> +int mtk_cam_seninf_calc_pixelrate(struct device *dev, s64 width, s64 height,
> + s64 hblank, s64 vblank,
> + int fps_n, int fps_d,
> + s64 sensor_pixel_rate)
> +{
> + int ret;
> + s64 p_pixel_rate = sensor_pixel_rate;
> +
> + ret = calc_buffered_pixel_rate(dev, width, height, hblank, vblank,
> + fps_n, fps_d, &p_pixel_rate);
> + if (ret)
> + return sensor_pixel_rate;
> +
> + return p_pixel_rate;
> +}
> +
> +int mtk_cam_seninf_dump(struct v4l2_subdev *sd)
This function does nothing. Drop it.
> +{
> + int ret = 0;
> + //TODO debug patch
> + return ret;
> +}
> +
> +void mtk_cam_seninf_set_secure(struct v4l2_subdev *sd,
> + int enable, unsigned int sec_info_addr)
This function is useless. Drop it.
> +{
> + struct seninf_ctx *ctx = sd_to_ctx(sd);
> +
> + ctx->sec_info_addr = sec_info_addr;
> + dev_info(ctx->dev, "[%s]: %x, enable: %d\n",
> + __func__, sec_info_addr, enable);
> + ctx->is_secure = enable ? 1 : 0;
> +}
>
[snip]
> +
> +enum SET_REG_KEYS {
> + REG_KEY_MIN = 0,
> + REG_KEY_SETTLE_CK = REG_KEY_MIN,
> + REG_KEY_SETTLE_DT,
> + REG_KEY_HS_TRAIL_EN,
> + REG_KEY_HS_TRAIL_PARAM,
> + REG_KEY_CSI_IRQ_STAT,
> + REG_KEY_CSI_RESYNC_CYCLE,
> + REG_KEY_MUX_IRQ_STAT,
> + REG_KEY_CAMMUX_IRQ_STAT,
> + REG_KEY_CAMMUX_VSYNC_IRQ_EN,
> + REG_KEY_CSI_IRQ_EN,
> + REG_KEY_MAX_NUM
Because mtk_cam_seninf_set_reg() is useless and drop,
these enum is also useless. Drop these.
> +};
> +
> +#define SET_REG_KEYS_NAMES \
> + "RG_SETTLE_CK", \
> + "RG_SETTLE_DT", \
> + "RG_HS_TRAIL_EN", \
> + "RG_HS_TRAIL_PARAM", \
> + "RG_CSI_IRQ_STAT", \
> + "RG_CSI_RESYNC_CYCLE", \
> + "RG_MUX_IRQ_STAT", \
> + "RG_CAMMUX_IRQ_STAT", \
> + "REG_VSYNC_IRQ_EN", \
> + "RG_CSI_IRQ_EN", \
> +
> +struct mtk_cam_seninf_mux_meter {
mtk_cam_seninf_mux_meter is useless. Drop it.
> + u32 width;
> + u32 height;
> + u32 h_valid;
> + u32 h_blank;
> + u32 v_valid;
> + u32 v_blank;
> + s64 mipi_pixel_rate;
> + s64 vb_in_us;
> + s64 hb_in_us;
> + s64 line_time_in_us;
> +};
> +
[snip]
> +
> +#define to_std_fmt_code(code) \
> + ((code) & 0xFFFF)
to_std_fmt_code() is not necessary. Drop it.
> +
> +
[snip]
> +
> +void mtk_cam_seninf_mux_put(struct seninf_ctx *ctx, struct seninf_mux *mux)
> +{
> + struct seninf_core *core = ctx->core;
> +
> + mutex_lock(&core->mutex);
> + list_move_tail(&mux->list, &core->list_mux);
> + mutex_unlock(&core->mutex);
> +}
I think using list to manage mux resource is a little over coding.
An array could be more simple.
struct sensor_core {
...
bool *mux_used;
...
};
void mtk_cam_seninf_init_res(struct seninf_core *core)
{
int i;
core->mux_used = kzalloc(g_seninf_cfg->mux_num * sizeof(*core->mux_used));
}
int mtk_cam_seninf_mux_get_pref(struct seninf_ctx *ctx,
int pref_cnt)
{
int mux;
mutex_lock(&core->mutex);
if (!core->mux_used[pref_cnt]) {
core->mux_used[pref_cnt] = true;
mutex_unlock(&core->mutex);
return pref_cnt;
}
for (mux = 0; mux < g_seninf_cfg->mux_num; mux++)
if (!core->mux_used[mux]) {
core->mux_used[mux] = true;
mutex_unlock(&core->mutex);
return mux
}
mutex_unlock(&core->mutex);
return -1;
}
void mtk_cam_seninf_release_mux(struct seninf_ctx *ctx)
{
int mux;
mutex_lock(&core->mutex);
for (mux = 0; mux < g_seninf_cfg->mux_num; mux++)
core->mux_used[mux] = false;
mutex_unlock(&core->mutex);
}
> +
> +struct seninf_vc *mtk_cam_seninf_get_vc_by_pad(struct seninf_ctx *ctx, int idx)
> +{
> + int i;
> + struct seninf_vcinfo *vcinfo = &ctx->vcinfo;
> +
> + for (i = 0; i < vcinfo->cnt; i++) {
> + if (vcinfo->vc[i].out_pad == idx)
> + return &vcinfo->vc[i];
> + }
> +
> + return NULL;
> +}
> +
> +unsigned int mtk_cam_seninf_get_vc_feature(struct v4l2_subdev *sd,
> + unsigned int pad)
This function is useless. Drop it.
> +{
> + struct seninf_vc *pvc = NULL;
> + struct seninf_ctx *ctx = container_of(sd, struct seninf_ctx, subdev);
> +
> + pvc = mtk_cam_seninf_get_vc_by_pad(ctx, pad);
> + if (pvc)
> + return pvc->feature;
> +
> + return VC_NONE;
> +}
> +
[snip]
> +
> +void mtk_cam_seninf_release_mux(struct seninf_ctx *ctx)
> +{
> + struct seninf_mux *ent, *tmp;
> +
> + list_for_each_entry_safe(ent, tmp, &ctx->list_mux, list) {
> + mtk_cam_seninf_mux_put(ctx, ent);
> + }
> +}
> +
> +int mtk_cam_seninf_get_pixelmode(struct v4l2_subdev *sd,
> + int pad_id, int *pixel_mode)
This function is useless. Drop it.
> +{
> + struct seninf_ctx *ctx = container_of(sd, struct seninf_ctx, subdev);
> + struct seninf_vc *vc;
> +
> + vc = mtk_cam_seninf_get_vc_by_pad(ctx, pad_id);
> + if (!vc) {
> + pr_info("%s: invalid pad=%d\n", __func__, pad_id);
> + return -1;
> + }
> +
> + *pixel_mode = vc->pixel_mode;
> +
> + return 0;
> +}
> +
[snip]
> +int mtk_cam_seninf_set_camtg(struct v4l2_subdev *sd, int pad_id, int camtg)
> +{
> + return _mtk_cam_seninf_set_camtg(sd, pad_id, camtg, true);
Squash _mtk_cam_seninf_set_camtg() into mtk_cam_seninf_set_camtg().
> +}
>
[snip]
> +
> +struct seninf_vc {
> + u8 dt;
> + u8 feature;
Because mtk_cam_seninf_get_vc_feature() is useless, so feature is useless.
Drop it.
> + u8 out_pad;
> + u8 pixel_mode;
> + u8 mux; /* allocated per group */
> + u8 cam; /* assigned by cam driver */
> + u8 enable;
> +};
> +
[snip]
> +
> +struct seninf_ctx {
> + struct v4l2_subdev subdev;
> + struct v4l2_async_notifier notifier;
> + struct device *dev;
> + struct v4l2_ctrl_handler ctrl_handler;
> + struct media_pad pads[PAD_MAXCNT];
> + struct v4l2_subdev_format fmt[PAD_MAXCNT];
> + struct seninf_core *core;
> + struct list_head list;
> +
> + u32 port;
> + u32 port_a;
> + u32 port_b;
> + u32 port_num;
> + u32 num_data_lanes;
> + s64 mipi_pixel_rate;
mipi_pixel_rate is never initialized. Drop it.
> + s64 buffered_pixel_rate;
buffered_pixel_rate is used only in update_isp_clk().
update_isp_clk() is useless and drop, so buffered_pixel_rate is useless.
Drop it.
> + s64 customized_pixel_rate;
customized_pixel_rate is useless. Drop it.
> + unsigned int m_csi_efuse;
> +
> + unsigned int is_4d1c:1;
> + unsigned int is_cphy:1;
> + unsigned int is_test_model:4;
is_test_model is useless. Drop it.
> + unsigned int is_secure:1;
> + unsigned int sec_info_addr;
Because mtk_cam_seninf_set_secure() is useless, so is_secure and sec_info_addr are useless.
Drop them.
> + u32 seninf_idx;
> + int pad2cam[PAD_MAXCNT];
> +
> + /* remote sensor */
> + struct v4l2_subdev *sensor_sd;
> + u32 sensor_pad_idx;
> +
> + /* provided by sensor */
> + struct seninf_vcinfo vcinfo;
> + int fps_n;
> + int fps_d;
> +
> + /* dfs */
> + int isp_freq;
isp_freq is useless. Drop it.
> +
> + void __iomem *reg_ana_csi_rx[CSI_PORT_MAX_NUM];
> + void __iomem *reg_ana_dphy_top[CSI_PORT_MAX_NUM];
> + void __iomem *reg_ana_cphy_top[CSI_PORT_MAX_NUM];
> + void __iomem *reg_if_top;
> + void __iomem *reg_if_ctrl[SENINF_NUM];
> + void __iomem *reg_if_cam_mux;
> + void __iomem *reg_if_cam_mux_gcsr;
> + void __iomem *reg_if_cam_mux_pcsr[SENINF_CAM_MUX_NUM];
> + void __iomem *reg_if_tg[SENINF_NUM];
> + void __iomem *reg_if_csi2[SENINF_NUM];
> + void __iomem *reg_if_mux[SENINF_MUX_NUM];
> +
> + /* resources */
> + struct list_head list_mux; /* work mux */
> + struct list_head list_cam_mux;
> +
> + /* flags */
> + unsigned int streaming:1;
> +
> + int cphy_settle_delay_dt;
> + int dphy_settle_delay_dt;
> + int settle_delay_ck;
> + int hs_trail_parameter;
> +
> + int open_refcnt;
> + struct mutex mutex; /* protect seninf context */
> +
> + /* csi irq */
> + unsigned int data_not_enough_cnt;
> + unsigned int err_lane_resync_cnt;
> + unsigned int crc_err_cnt;
> + unsigned int ecc_err_double_cnt;
> + unsigned int ecc_err_corrected_cnt;
> + /* seninf_mux fifo overrun irq */
> + unsigned int fifo_overrun_cnt;
> + /* cam_mux h/v size irq */
> + unsigned int size_err_cnt;
> + /* error flag */
> + unsigned int data_not_enough_flag;
> + unsigned int err_lane_resync_flag;
> + unsigned int crc_err_flag;
> + unsigned int ecc_err_double_flag;
> + unsigned int ecc_err_corrected_flag;
> + unsigned int fifo_overrun_flag;
> + unsigned int size_err_flag;
> +};
> +
[snip]
> +
> +int mtk_cam_seninf_set_cammux_vc(struct seninf_ctx *ctx, int cam_mux,
> + int vc_sel, int dt_sel, int vc_en,
> + int dt_en)
vc_sel is always zero, so just assign 0 in this function and drop vc_sel parameter.
vc_en and dt_en are always !!dt_sel, so it's not necessary to pass vc_en and dt_en as parameter.
Drop these two parameter and use !!dt_sel in this function.
> +{
> + void __iomem *seninf_cam_mux_vc_addr = ctx->reg_if_cam_mux + (cam_mux << 0x2);
> +
> + SENINF_BITS(seninf_cam_mux_vc_addr, SENINF_CAM_MUX0_OPT,
> + RG_SENINF_CAM_MUX0_VC_SEL, vc_sel);
> + SENINF_BITS(seninf_cam_mux_vc_addr, SENINF_CAM_MUX0_OPT,
> + RG_SENINF_CAM_MUX0_DT_SEL, dt_sel);
> + SENINF_BITS(seninf_cam_mux_vc_addr, SENINF_CAM_MUX0_OPT,
> + RG_SENINF_CAM_MUX0_VC_EN, vc_en);
> + SENINF_BITS(seninf_cam_mux_vc_addr, SENINF_CAM_MUX0_OPT,
> + RG_SENINF_CAM_MUX0_DT_EN, dt_en);
> +
> + return 0;
> +}
> +
[snip]
> +int mtk_cam_seninf_set_vc(struct seninf_ctx *ctx, u32 seninf_idx,
> + struct seninf_vcinfo *vcinfo)
> +{
> + void __iomem *seninf_csi2 = ctx->reg_if_csi2[seninf_idx];
> + int i;
> + struct seninf_vc *vc;
> +
> + if (WARN_ON(!vcinfo || !vcinfo->cnt))
> + return -EINVAL;
> +
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S0_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S1_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S2_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S3_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S4_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S5_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S6_DI_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_S7_DI_CTRL, 0);
> +
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_CH0_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_CH1_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_CH2_CTRL, 0);
> + SENINF_WRITE_REG(seninf_csi2, SENINF_CSI2_CH3_CTRL, 0);
> +
> + for (i = 0; i < vcinfo->cnt; i++) {
> + vc = &vcinfo->vc[i];
> +
> + /* General Long Packet Data Types: 0x10-0x17 */
> + if (vc->dt >= 0x10 && vc->dt <= 0x17) {
vc->dt would only be 0, 0x2a, 0x2b, 0x2c, so this checking would always be false.
Drop this.
> + SENINF_BITS(seninf_csi2, SENINF_CSI2_OPT,
> + RG_CSI2_GENERIC_LONG_PACKET_EN, 1);
> + }
> +
> + mtk_cam_seninf_set_di_ch_ctrl(seninf_csi2, i, vc);
> + }
> +
> + dev_dbg(ctx->dev, "DI_CTRL 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x\n",
> + readl(seninf_csi2 + SENINF_CSI2_S0_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S1_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S2_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S3_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S4_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S5_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S6_DI_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_S7_DI_CTRL));
> +
> + dev_dbg(ctx->dev, "CH_CTRL 0x%x 0x%x 0x%x 0x%x\n",
> + readl(seninf_csi2 + SENINF_CSI2_CH0_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_CH1_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_CH2_CTRL),
> + readl(seninf_csi2 + SENINF_CSI2_CH3_CTRL));
> +
> + return 0;
> +}
> +
[snip]
> +
> +int mtk_cam_seninf_irq_handler(int irq, void *data)
> +{
> + struct seninf_core *core = (struct seninf_core *)data;
> + unsigned long flags; /* for mipi err detection */
> + struct seninf_ctx *ctx;
> + struct seninf_vc *vc;
> + void __iomem *csi2, *pmux, *seninf_cam_mux;
> + int i;
> + unsigned int csi_irq_ro;
> + unsigned int mux_irq_ro;
> + unsigned int cam_irq_exp_ro;
> + unsigned int cam_irq_res_ro;
> +
> + spin_lock_irqsave(&core->spinlock_irq, flags);
> +
> + /* debug for set_reg case: REG_KEY_CSI_IRQ_EN */
> + if (core->csi_irq_en_flag) {
Because mtk_cam_seninf_set_reg() is useless, so core->csi_irq_en_flag is always zero.
Drop these code.
> + list_for_each_entry(ctx, &core->list, list) {
> + csi2 = ctx->reg_if_csi2[ctx->seninf_idx];
> + csi_irq_ro =
> + readl(csi2 + SENINF_CSI2_IRQ_STATUS);
> +
> + if (csi_irq_ro) {
> + SENINF_WRITE_REG(csi2, SENINF_CSI2_IRQ_STATUS,
> + 0xFFFFFFFF);
> + }
> +
> + if (csi_irq_ro & RO_CSI2_ECC_ERR_CORRECTED_IRQ)
> + ctx->ecc_err_corrected_cnt++;
> + if (csi_irq_ro & RO_CSI2_ECC_ERR_DOUBLE_IRQ)
> + ctx->ecc_err_double_cnt++;
> + if (csi_irq_ro & RO_CSI2_CRC_ERR_IRQ)
> + ctx->crc_err_cnt++;
> + if (csi_irq_ro & RO_CSI2_ERR_LANE_RESYNC_IRQ)
> + ctx->err_lane_resync_cnt++;
> + if (csi_irq_ro & RO_CSI2_RECEIVE_DATA_NOT_ENOUGH_IRQ)
> + ctx->data_not_enough_cnt++;
> +
> + for (i = 0; i < ctx->vcinfo.cnt; i++) {
> + vc = &ctx->vcinfo.vc[i];
> + pmux = ctx->reg_if_mux[vc->mux];
> + seninf_cam_mux = ctx->reg_if_cam_mux;
> +
> + mux_irq_ro = readl(pmux +
> + SENINF_MUX_IRQ_STATUS);
> +
> + cam_irq_exp_ro = readl(seninf_cam_mux +
> + SENINF_CAM_MUX0_CHK_CTL_1 +
> + (0x10 * (vc->cam)));
> +
> + cam_irq_res_ro = readl(seninf_cam_mux +
> + SENINF_CAM_MUX0_CHK_RES +
> + (0x10 * (vc->cam)));
> +
> + if (mux_irq_ro)
> + SENINF_WRITE_REG(pmux,
> + SENINF_MUX_IRQ_STATUS,
> + 0xFFFFFFFF);
> +
> + if (cam_irq_res_ro != cam_irq_exp_ro)
> + SENINF_WRITE_REG(seninf_cam_mux,
> + SENINF_CAM_MUX0_CHK_RES +
> + (0x10 * (vc->cam)),
> + 0xFFFFFFFF);
> +
> + if (mux_irq_ro & (0x1 << 0))
> + ctx->fifo_overrun_cnt++;
> +
> + if (cam_irq_res_ro != cam_irq_exp_ro)
> + ctx->size_err_cnt++;
> + }
> +
> + /* dump status counter: debug for electrical signal */
> + if (ctx->data_not_enough_cnt >= core->detection_cnt ||
> + ctx->err_lane_resync_cnt >= core->detection_cnt ||
> + ctx->crc_err_cnt >= core->detection_cnt ||
> + ctx->ecc_err_double_cnt >= core->detection_cnt ||
> + ctx->ecc_err_corrected_cnt >= core->detection_cnt ||
> + ctx->fifo_overrun_cnt >= core->detection_cnt ||
> + ctx->size_err_cnt >= core->detection_cnt) {
> + /* disable all interrupts */
> + SENINF_WRITE_REG(csi2, SENINF_CSI2_IRQ_EN, 0x80000000);
> +
> + if (ctx->data_not_enough_cnt >= core->detection_cnt)
> + ctx->data_not_enough_flag = 1;
> + if (ctx->err_lane_resync_cnt >= core->detection_cnt)
> + ctx->err_lane_resync_flag = 1;
> + if (ctx->crc_err_cnt >= core->detection_cnt)
> + ctx->crc_err_flag = 1;
> + if (ctx->ecc_err_double_cnt >= core->detection_cnt)
> + ctx->ecc_err_double_flag = 1;
> + if (ctx->ecc_err_corrected_cnt >= core->detection_cnt)
> + ctx->ecc_err_corrected_flag = 1;
> + if (ctx->fifo_overrun_cnt >= core->detection_cnt)
> + ctx->fifo_overrun_flag = 1;
> + if (ctx->size_err_cnt >= core->detection_cnt)
> + ctx->size_err_flag = 1;
> + }
> + }
> + }
> +
> + spin_unlock_irqrestore(&core->spinlock_irq, flags);
> +
> + return 0;
> +}
> +
> +int mtk_cam_seninf_set_sw_cfg_busy(struct seninf_ctx *ctx, bool enable, int index)
This function is useless, drop it.
> +{
> + void __iomem *seninf_cam_mux = ctx->reg_if_cam_mux;
> + unsigned int mask;
> +
> + mask = index ? RG_SENINF_CAM_MUX_DYN_SWITCH_BSY1 : RG_SENINF_CAM_MUX_DYN_SWITCH_BSY0;
> +
> + SENINF_BITS_VAR(seninf_cam_mux, SENINF_CAM_MUX_DYN_CTRL, mask, enable);
> +
> + return 0;
> +}
> +
> +int mtk_cam_seninf_reset_cam_mux_dyn_en(struct seninf_ctx *ctx, int index)
This function is useless, drop it.
> +{
> + void __iomem *seninf_cam_mux = ctx->reg_if_cam_mux;
> +
> + if (index == 0)
> + SENINF_BITS(seninf_cam_mux, SENINF_CAM_MUX_DYN_EN,
> + RG_SENINF_CAM_MUX_DYN_SWITCH_EN0, 0);
> + else
> + SENINF_BITS(seninf_cam_mux, SENINF_CAM_MUX_DYN_EN,
> + RG_SENINF_CAM_MUX_DYN_SWITCH_EN1, 0);
> + return 0;
> +}
> +
> +int mtk_cam_seninf_enable_global_drop_irq(struct seninf_ctx *ctx, bool enable,
> + int index)
This function is useless, drop it.
> +{
> + void __iomem *seninf_cam_mux = ctx->reg_if_cam_mux;
> +
> + if (index == 0)
> + SENINF_BITS(seninf_cam_mux, SENINF_CAM_MUX_IRQ_EN,
> + RG_SENINF_CAM_MUX15_HSIZE_ERR_IRQ_EN, enable);
> + else
> + SENINF_BITS(seninf_cam_mux, SENINF_CAM_MUX_IRQ_EN,
> + RG_SENINF_CAM_MUX15_VSIZE_ERR_IRQ_EN, enable);
> + return 0;
> +}
> +
> +int mtk_cam_seninf_enable_cam_mux_vsync_irq(struct seninf_ctx *ctx, bool enable,
> + int cam_mux)
This function is useless, drop it.
> +{
> + void __iomem *seninf_cam_mux = ctx->reg_if_cam_mux;
> + int tmp = 0;
> +
> + tmp = SENINF_READ_BITS(seninf_cam_mux, SENINF_CAM_MUX_VSYNC_IRQ_EN,
> + RG_SENINF_CAM_MUX_VSYNC_IRQ_EN);
> + if (enable)
> + tmp |= (enable << cam_mux);
> + else
> + tmp &= ~(enable << cam_mux);
> + SENINF_BITS(seninf_cam_mux, SENINF_CAM_MUX_VSYNC_IRQ_EN,
> + RG_SENINF_CAM_MUX_VSYNC_IRQ_EN, tmp);
> + return 0;
> +}
> +
> +int mtk_cam_seninf_disable_all_cam_mux_vsync_irq(struct seninf_ctx *ctx)
This function is useless, drop it.
> +{
> + void __iomem *seninf_cam_mux = ctx->reg_if_cam_mux;
> +
> + SENINF_BITS(seninf_cam_mux, SENINF_CAM_MUX_VSYNC_IRQ_EN,
> + RG_SENINF_CAM_MUX_VSYNC_IRQ_EN, 0);
> + return 0;
> +}
> +
> +int mtk_cam_seninf_set_reg(struct seninf_ctx *ctx, u32 key, u32 val)
This function is useless, drop it.
Regards,
CK
> +{
> + int i;
> + void __iomem *base = ctx->reg_ana_dphy_top[ctx->port];
> + void __iomem *csi2 = ctx->reg_if_csi2[ctx->seninf_idx];
> + void __iomem *pmux, *pcammux;
> + struct seninf_vc *vc;
> + struct seninf_core *core;
> + struct seninf_ctx *ctx_;
> + void __iomem *csi2_;
> +
> + core = dev_get_drvdata(ctx->dev->parent);
> +
> + if (!ctx->streaming)
> + return 0;
> +
> + dev_dbg(ctx->dev, "%s key = 0x%x, val = 0x%x\n", __func__, key, val);
> +
> + switch (key) {
> + case REG_KEY_SETTLE_CK:
> + SENINF_BITS(base, DPHY_RX_CLOCK_LANE0_HS_PARAMETER,
> + RG_DPHY_RX_LC0_HS_SETTLE_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_CLOCK_LANE1_HS_PARAMETER,
> + RG_DPHY_RX_LC1_HS_SETTLE_PARAMETER, val);
> + break;
> + case REG_KEY_SETTLE_DT:
> + SENINF_BITS(base, DPHY_RX_DATA_LANE0_HS_PARAMETER,
> + RG_CDPHY_RX_LD0_TRIO0_HS_SETTLE_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE1_HS_PARAMETER,
> + RG_CDPHY_RX_LD1_TRIO1_HS_SETTLE_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE2_HS_PARAMETER,
> + RG_CDPHY_RX_LD2_TRIO2_HS_SETTLE_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE3_HS_PARAMETER,
> + RG_CDPHY_RX_LD3_TRIO3_HS_SETTLE_PARAMETER, val);
> + break;
> + case REG_KEY_HS_TRAIL_EN:
> + SENINF_BITS(base, DPHY_RX_DATA_LANE0_HS_PARAMETER,
> + RG_DPHY_RX_LD0_HS_TRAIL_EN, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE1_HS_PARAMETER,
> + RG_DPHY_RX_LD1_HS_TRAIL_EN, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE2_HS_PARAMETER,
> + RG_DPHY_RX_LD2_HS_TRAIL_EN, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE3_HS_PARAMETER,
> + RG_DPHY_RX_LD3_HS_TRAIL_EN, val);
> + break;
> + case REG_KEY_HS_TRAIL_PARAM:
> + SENINF_BITS(base, DPHY_RX_DATA_LANE0_HS_PARAMETER,
> + RG_DPHY_RX_LD0_HS_TRAIL_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE1_HS_PARAMETER,
> + RG_DPHY_RX_LD1_HS_TRAIL_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE2_HS_PARAMETER,
> + RG_DPHY_RX_LD2_HS_TRAIL_PARAMETER, val);
> + SENINF_BITS(base, DPHY_RX_DATA_LANE3_HS_PARAMETER,
> + RG_DPHY_RX_LD3_HS_TRAIL_PARAMETER, val);
> + break;
> + case REG_KEY_CSI_IRQ_STAT:
> + SENINF_WRITE_REG(csi2, SENINF_CSI2_IRQ_STATUS,
> + val & 0xFFFFFFFF);
> + break;
> + case REG_KEY_MUX_IRQ_STAT:
> + for (i = 0; i < ctx->vcinfo.cnt; i++) {
> + vc = &ctx->vcinfo.vc[i];
> + pmux = ctx->reg_if_mux[vc->mux];
> + SENINF_WRITE_REG(pmux, SENINF_MUX_IRQ_STATUS,
> + val & 0xFFFFFFFF);
> + }
> + break;
> + case REG_KEY_CAMMUX_IRQ_STAT:
> + for (i = 0; i < ctx->vcinfo.cnt; i++) {
> + vc = &ctx->vcinfo.vc[i];
> + pcammux = ctx->reg_if_cam_mux;
> + SENINF_WRITE_REG(pcammux, SENINF_CAM_MUX_IRQ_STATUS,
> + val & 0xFFFFFFFF);
> + }
> + break;
> + case REG_KEY_CSI_IRQ_EN:
> + {
> + if (!val) {
> + core->csi_irq_en_flag = 0;
> + return 0;
> + }
> +
> + core->csi_irq_en_flag = 1;
> + core->detection_cnt = val;
> + list_for_each_entry(ctx_, &core->list, list) {
> + csi2_ = ctx_->reg_if_csi2[ctx_->seninf_idx];
> + SENINF_WRITE_REG(csi2_, SENINF_CSI2_IRQ_EN,
> + 0xA0002058);
> + }
> + }
> + break;
> + }
> +
> + return 0;
> +}
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl
2025-07-07 1:31 ` [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl shangyao lin
2025-07-11 8:56 ` CK Hu (胡俊光)
@ 2025-07-21 1:40 ` CK Hu (胡俊光)
1 sibling, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-21 1:40 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduce state management and debugging mechanisms for the MediaTek ISP7.x
> camsys platform. State management establishes control over ISP operations and
> events, defining distinct states for request handling, sensor control, and
> frame synchronization, ensuring proper event processing. The debugging
> mechanism ensures stable operation and timely data collection during anomalies.
>
> ---
>
> Changes in v2:
> - Removed mtk_cam-debug.c and mtk_cam-debug.h, along with related code
> - Various fixes per review comments
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
> .../mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c | 1409 +++++++++++++++++
> .../mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h | 131 ++
> 2 files changed, 1540 insertions(+)
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
Why this file is executable?
> create mode 100755 drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.h
Ditto.
>
> diff --git a/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
> new file mode 100755
> index 000000000000..15af1eac5444
> --- /dev/null
> +++ b/drivers/media/platform/mediatek/isp/isp_7x/camsys/mtk_cam-ctrl.c
> @@ -0,0 +1,1409 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2022 MediaTek Inc.
2025
> + */
> +
> +#include <linux/list.h>
> +#include <linux/math64.h>
> +#include <linux/of.h>
> +#include <linux/videodev2.h>
> +#include <linux/hrtimer.h>
Alphabetic order.
> +
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-subdev.h>
> +#include <media/v4l2-ioctl.h>
Alphabetic order.
> +
> +#include "mtk_cam.h"
> +#include "mtk_cam-ctrl.h"
> +#include "mtk_cam-pool.h"
> +#include "mtk_cam-raw.h"
> +
Drop this blank line.
> +#include "mtk_cam-regs-mt8188.h"
> +
Drop this blank line.
> +#include "mtk_camera-v4l2-controls.h"
> +
> +#define SENSOR_SET_DEADLINE_MS 18
> +#define SENSOR_SET_RESERVED_MS 7
> +#define SENSOR_SET_DEADLINE_MS_60FPS 6
> +#define SENSOR_SET_RESERVED_MS_60FPS 6
> +#define STATE_NUM_AT_SOF 3
> +#define INITIAL_DROP_FRAME_CNT 1
> +
> +enum MTK_CAMSYS_STATE_RESULT {
> + STATE_RESULT_TRIGGER_CQ = 0,
> + STATE_RESULT_PASS_CQ_INIT,
> + STATE_RESULT_PASS_CQ_SW_DELAY,
> + STATE_RESULT_PASS_CQ_SCQ_DELAY,
> + STATE_RESULT_PASS_CQ_HW_DELAY,
After call mtk_camsys_raw_state_handle(), just check STATE_RESULT_TRIGGER_CQ or not.
There result is just for print error log.
Direct print error in mtk_camsys_raw_state_handle() and drop these definition.
> +};
> +
> +void state_transition(struct mtk_camsys_ctrl_state *state_entry,
static void state_transition(struct mtk_camsys_ctrl_state *state_entry,
> + enum MTK_CAMSYS_STATE_IDX from,
> + enum MTK_CAMSYS_STATE_IDX to)
> +{
> + if (state_entry->estate == from)
> + state_entry->estate = to;
> +}
> +
[snip]
> +void mtk_cam_set_sensor_full(struct mtk_cam_request_stream_data *s_data,
> + struct mtk_camsys_sensor_ctrl *sensor_ctrl)
> +{
> + struct mtk_cam_device *cam;
> + struct mtk_cam_ctx *ctx;
> + struct mtk_cam_request *req;
> + struct mtk_raw_device *raw_dev;
> + unsigned int time_after_sof = 0;
> +
> + /* EnQ this request's state element to state_list (STATE:READY) */
> + spin_lock(&sensor_ctrl->camsys_state_lock);
> + list_add_tail(&s_data->state.state_element,
> + &sensor_ctrl->camsys_state_list);
> + atomic_set(&sensor_ctrl->sensor_request_seq_no, s_data->frame_seq_no);
> + spin_unlock(&sensor_ctrl->camsys_state_lock);
> +
> + /* sensor_worker task */
> + ctx = mtk_cam_s_data_get_ctx(s_data);
> + cam = ctx->cam;
> + req = mtk_cam_s_data_get_req(s_data);
> +
> + dev_dbg(cam->dev, "%s:%s:ctx(%d) req(%d):sensor try set start\n",
> + __func__, req->req.debug_str, ctx->stream_id, s_data->frame_seq_no);
> +
> + if (ctx->used_raw_num && mtk_cam_req_frame_sync_start(req))
> + dev_dbg(cam->dev,
> + "%s:%s:ctx(%d): sensor ctrl with frame sync - start\n",
> + __func__, req->req.debug_str, ctx->stream_id);
> +
> + /*
> + * request setup
> + * 1st frame sensor setting is treated like normal frame
> + * and is set with other settings like max fps.
> + * 2nd is special, only exposure is set.
> + */
> + if (s_data->flags & MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_EN) {
In patch [8/13] I says ctx->sensor is always NULL, and MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_EN is set only when ctx->sensor is not NULL.
So drop MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_EN.
> + v4l2_ctrl_request_setup(&req->req,
> + s_data->sensor->ctrl_handler);
> + time_after_sof =
> + div_u64(ktime_get_boottime_ns(), 1000000) -
> + ctx->sensor_ctrl.sof_time;
> + dev_dbg(cam->dev,
> + "[SOF+%dms] Sensor request:%d[ctx:%d] setup\n",
> + time_after_sof, s_data->frame_seq_no,
> + ctx->stream_id);
> + }
> +
> + state_transition(&s_data->state, E_STATE_READY, E_STATE_SENSOR);
> +
> + if (ctx->used_raw_num && mtk_cam_req_frame_sync_end(req))
> + dev_dbg(cam->dev,
> + "%s:ctx(%d): sensor ctrl with frame sync - stop\n",
> + __func__, ctx->stream_id);
> +
> + if (ctx->used_raw_num) {
> + raw_dev = get_main_raw_dev(ctx->cam, ctx->pipe);
> + if (raw_dev && atomic_read(&raw_dev->vf_en) == 0 &&
> + ctx->sensor_ctrl.initial_cq_done == 1 &&
> + s_data->frame_seq_no == 1)
> + mtk_cam_stream_on(raw_dev, ctx);
> + }
> +
> + dev_dbg(cam->dev, "%s:%s:ctx(%d)req(%d):sensor done at SOF+%dms\n",
> + __func__, req->req.debug_str, ctx->stream_id,
> + s_data->frame_seq_no, time_after_sof);
> +
> + mtk_cam_complete_sensor_hdl(s_data);
> +}
> +
[snip]
> +static int
> +mtk_camsys_raw_state_handle(struct mtk_raw_device *raw_dev,
> + struct mtk_camsys_sensor_ctrl *sensor_ctrl,
> + struct mtk_camsys_ctrl_state **current_state,
> + struct mtk_camsys_irq_info *irq_info)
> +{
> + struct mtk_cam_ctx *ctx = sensor_ctrl->ctx;
> + struct mtk_camsys_ctrl_state *state_temp, *state_outer = NULL;
> + struct mtk_camsys_ctrl_state *state_inner = NULL;
> + struct mtk_camsys_ctrl_state *state_rec[STATE_NUM_AT_SOF];
> + struct mtk_cam_request_stream_data *req_stream_data;
> + int frame_idx_inner = irq_info->frame_idx_inner;
> + int stateidx;
> + int que_cnt = 0;
> + int write_cnt;
> + u64 time_boot = ktime_get_boottime_ns();
> + u64 time_mono = ktime_get_ns();
> + int working_req_found = 0;
> +
> + /* List state-queue status*/
> + spin_lock(&sensor_ctrl->camsys_state_lock);
> + list_for_each_entry(state_temp, &sensor_ctrl->camsys_state_list,
> + state_element) {
> + req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_temp);
> + stateidx = atomic_read(&sensor_ctrl->sensor_request_seq_no) -
> + req_stream_data->frame_seq_no;
> + if (stateidx < STATE_NUM_AT_SOF && stateidx > -1) {
> + state_rec[stateidx] = state_temp;
> + if (stateidx == 0)
> + working_req_found = 1;
> +
> + /* Find outer state element */
> + if (state_temp->estate == E_STATE_OUTER ||
> + state_temp->estate == E_STATE_CAMMUX_OUTER ||
> + state_temp->estate == E_STATE_OUTER_HW_DELAY) {
> + state_outer = state_temp;
> + mtk_cam_set_timestamp(req_stream_data,
> + time_boot, time_mono);
> + }
> +
> + /* Find inner state element request */
> + if (state_temp->estate == E_STATE_INNER ||
> + state_temp->estate == E_STATE_INNER_HW_DELAY)
> + state_inner = state_temp;
> +
> + dev_dbg(raw_dev->dev,
> + "[SOF] STATE_CHECK [N-%d] Req:%d / State:%d\n",
> + stateidx, req_stream_data->frame_seq_no,
> + state_rec[stateidx]->estate);
> + }
> + /* counter for state queue*/
> + que_cnt++;
> + }
> + spin_unlock(&sensor_ctrl->camsys_state_lock);
> +
> + /* HW imcomplete case */
> + if (state_inner) {
> + req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_inner);
> + write_cnt = (atomic_read(&sensor_ctrl->isp_request_seq_no) / 256)
> + * 256 + irq_info->write_cnt;
> +
> + if (frame_idx_inner > atomic_read(&sensor_ctrl->isp_request_seq_no) ||
> + atomic_read(&req_stream_data->frame_done_work.is_queued) == 1) {
> + dev_dbg_ratelimited(raw_dev->dev,
> + "[SOF] frame done work too late frames. req(%d),ts(%llu)\n",
> + req_stream_data->frame_seq_no,
> + irq_info->ts_ns);
> + } else if (write_cnt >= req_stream_data->frame_seq_no) {
> + dev_info_ratelimited(raw_dev->dev,
> + "[SOF] frame done reading lost %d frames. req(%d),ts(%llu)\n",
> + write_cnt - req_stream_data->frame_seq_no + 1,
> + req_stream_data->frame_seq_no,
> + irq_info->ts_ns);
> + mtk_cam_set_timestamp(req_stream_data,
> + time_boot - 1000, time_mono - 1000);
> + mtk_camsys_frame_done(ctx, write_cnt, ctx->stream_id);
> + } else if ((write_cnt >= req_stream_data->frame_seq_no - 1) &&
> + irq_info->fbc_cnt == 0) {
> + dev_info_ratelimited(raw_dev->dev,
> + "[SOF] frame done reading lost frames. req(%d),ts(%llu)\n",
> + req_stream_data->frame_seq_no, irq_info->ts_ns);
> + mtk_cam_set_timestamp(req_stream_data,
> + time_boot - 1000, time_mono - 1000);
> + mtk_camsys_frame_done(ctx, write_cnt + 1, ctx->stream_id);
> + } else {
> + state_transition(state_inner,
> + E_STATE_INNER, E_STATE_INNER_HW_DELAY);
> + if (state_outer) {
> + state_transition(state_outer,
> + E_STATE_OUTER,
> + E_STATE_OUTER_HW_DELAY);
> + state_transition(state_outer,
> + E_STATE_CAMMUX_OUTER,
> + E_STATE_OUTER_HW_DELAY);
> + }
> + dev_info_ratelimited(raw_dev->dev,
> + "[SOF] HW_IMCOMPLETE state cnt(%d,%d),req(%d),ts(%llu)\n",
> + write_cnt, irq_info->write_cnt,
> + req_stream_data->frame_seq_no,
> + irq_info->ts_ns);
> + return STATE_RESULT_PASS_CQ_HW_DELAY;
> + }
> + }
> +
> + /* Transit outer state to inner state */
> + if (state_outer && sensor_ctrl->sensorsetting_wq) {
> + req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_outer);
> + if (atomic_read(&sensor_ctrl->initial_drop_frame_cnt) == 0 &&
> + req_stream_data->frame_seq_no > frame_idx_inner) {
> + dev_info(raw_dev->dev,
> + "[SOF-noDBLOAD] HW delay outer_no:%d, inner_idx:%d <= processing_idx:%d,ts:%llu\n",
> + req_stream_data->frame_seq_no, frame_idx_inner,
> + atomic_read(&sensor_ctrl->isp_request_seq_no),
> + irq_info->ts_ns);
> + return STATE_RESULT_PASS_CQ_HW_DELAY;
> + }
> +
> + if (atomic_read(&sensor_ctrl->initial_drop_frame_cnt) == 0 &&
> + req_stream_data->frame_seq_no == frame_idx_inner) {
> + if (frame_idx_inner >
> + atomic_read(&sensor_ctrl->isp_request_seq_no)) {
> + state_transition(state_outer,
> + E_STATE_OUTER_HW_DELAY,
> + E_STATE_INNER_HW_DELAY);
> + state_transition(state_outer,
> + E_STATE_OUTER,
> + E_STATE_INNER);
> + state_transition(state_outer,
> + E_STATE_CAMMUX_OUTER,
> + E_STATE_INNER);
State would change to E_STATE_INNER only in mtk_camsys_raw_state_handle().
mtk_camsys_raw_state_handle() is called only when ctx->sensor is not NULL.
In patch [8/13] I say ctx-sensor is always NULL,
so state would never change to E_STATE_INNER.
Drop it.
> + atomic_set(&sensor_ctrl->isp_request_seq_no, frame_idx_inner);
> + dev_dbg(raw_dev->dev,
> + "[SOF-DBLOAD] frame_seq_no:%d, OUTER->INNER state:%d,ts:%llu\n",
> + req_stream_data->frame_seq_no,
> + state_outer->estate, irq_info->ts_ns);
> + }
> + }
> + }
> +
> + /* Trigger high resolution timer to try sensor setting */
> + sensor_ctrl->sof_time = div_u64(irq_info->ts_ns, 1000000);
> + mtk_cam_sof_timer_setup(ctx);
> +
> + if (que_cnt > 1 && state_rec[1]) {
> + state_temp = state_rec[1];
> + req_stream_data = mtk_cam_ctrl_state_to_req_s_data(state_temp);
> + if (req_stream_data->frame_seq_no == 1)
> + state_transition(state_temp,
> + E_STATE_SENSOR, E_STATE_INNER);
> + }
> +
> + if (que_cnt > 0) {
> + if (working_req_found && state_rec[0]) {
> + if (state_rec[0]->estate == E_STATE_READY) {
> + dev_info(raw_dev->dev, "[SOF] sensor delay ts:%llu\n",
> + irq_info->ts_ns);
> + req_stream_data =
> + mtk_cam_ctrl_state_to_req_s_data(state_rec[0]);
> + req_stream_data->flags |=
> + MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_DELAYED;
> + }
> +
> + if (state_rec[0]->estate == E_STATE_SENINF)
> + dev_info(raw_dev->dev, "[SOF] sensor switch delay\n");
> +
> + /* CQ triggering judgment*/
> + if (state_rec[0]->estate == E_STATE_SENSOR) {
> + *current_state = state_rec[0];
> + return STATE_RESULT_TRIGGER_CQ;
> + }
> + /* last SCQ triggering delay judgment*/
> + if (state_rec[0]->estate == E_STATE_CQ_SCQ_DELAY) {
> + state_transition(state_rec[0],
> + E_STATE_CQ_SCQ_DELAY,
> + E_STATE_OUTER);
> + dev_info(raw_dev->dev, "[SOF] SCQ_DELAY state:%d ts:%llu\n",
> + state_rec[0]->estate, irq_info->ts_ns);
> + return STATE_RESULT_PASS_CQ_SCQ_DELAY;
> + }
> + } else {
> + dev_dbg(raw_dev->dev, "[SOF] working request not found\n");
> + }
> + }
> + return STATE_RESULT_PASS_CQ_SW_DELAY;
> +}
> +
> +static void mtk_camsys_raw_frame_start(struct mtk_raw_device *raw_dev,
> + struct mtk_cam_ctx *ctx,
> + struct mtk_camsys_irq_info *irq_info)
> +{
> + unsigned int dequeued_frame_seq_no = irq_info->frame_idx_inner;
> + struct mtk_cam_request_stream_data *req_stream_data;
> + struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
> + struct mtk_cam_working_buf_entry *buf_entry;
> + struct mtk_camsys_ctrl_state *current_state;
> + struct mtk_cam_buffer *buf;
> +
> + dma_addr_t base_addr;
> + enum MTK_CAMSYS_STATE_RESULT state_handle_ret;
> +
> + /* touch watchdog*/
> + if (watchdog_scenario(ctx))
In patch [8/13] I says ctx->sensor is always NULL.
watchdog_scenario() return true only when ctx->sensor is not NULL,
so watchdog_scenario() would always return false.
Drop all watchdog related code.
> + mtk_ctx_watchdog_kick(ctx);
> + /* inner register dequeue number */
> + ctx->dequeued_frame_seq_no = dequeued_frame_seq_no;
> + /* Send V4L2_EVENT_FRAME_SYNC event */
> + mtk_cam_event_frame_sync(ctx->pipe, dequeued_frame_seq_no);
> +
> + /* Handle directly enque buffers */
> + spin_lock(&ctx->cam->dma_processing_lock);
> + list_for_each_entry(buf, &ctx->cam->dma_processing, list) {
> + if (buf->state.estate == E_BUF_STATE_COMPOSED) {
> + spin_unlock(&ctx->cam->dma_processing_lock);
> + goto apply_cq;
> + }
> + }
> + spin_unlock(&ctx->cam->dma_processing_lock);
> + buf = NULL;
> + current_state = NULL;
> +
> + /* Find request of this dequeued frame */
> + req_stream_data =
> + mtk_cam_get_req_s_data(ctx, ctx->stream_id, dequeued_frame_seq_no);
> +
> + if (ctx->sensor) {
In patch [8/13] I says ctx->sensor is always NULL, so drop these code.
> + state_handle_ret =
> + mtk_camsys_raw_state_handle(raw_dev, sensor_ctrl,
> + ¤t_state, irq_info);
mtk_camsys_raw_state_handle() is called only here, but ctx->sensor is always NULL, so mtk_camsys_raw_state_handle() would never be called.
Drop mtk_camsys_raw_state_handle().
> +
> + if (state_handle_ret != STATE_RESULT_TRIGGER_CQ) {
> + dev_dbg(raw_dev->dev, "[SOF] CQ drop s:%d deq:%d\n",
> + state_handle_ret, dequeued_frame_seq_no);
> + return;
> + }
> + }
> +
> +apply_cq:
> + /* Update CQ base address if needed */
> + if (ctx->composed_frame_seq_no <= dequeued_frame_seq_no) {
> + dev_info_ratelimited(raw_dev->dev,
> + "SOF[ctx:%d-#%d], CQ isn't updated [composed_frame_deq (%d) ts:%llu]\n",
> + ctx->stream_id, dequeued_frame_seq_no,
> + ctx->composed_frame_seq_no, irq_info->ts_ns);
> + return;
> + }
> + /* apply next composed buffer */
> + spin_lock(&ctx->composed_buffer_list.lock);
> + if (list_empty(&ctx->composed_buffer_list.list)) {
> + dev_info_ratelimited(raw_dev->dev,
> + "SOF_INT_ST, no buffer update, cq_num:%d, frame_seq:%d, ts:%llu\n",
> + ctx->composed_frame_seq_no, dequeued_frame_seq_no,
> + irq_info->ts_ns);
> + spin_unlock(&ctx->composed_buffer_list.lock);
> + } else {
> + buf_entry = list_first_entry(&ctx->composed_buffer_list.list,
> + struct mtk_cam_working_buf_entry,
> + list_entry);
> + list_del(&buf_entry->list_entry);
> + ctx->composed_buffer_list.cnt--;
> + spin_unlock(&ctx->composed_buffer_list.lock);
> + spin_lock(&ctx->processing_buffer_list.lock);
> + list_add_tail(&buf_entry->list_entry,
> + &ctx->processing_buffer_list.list);
> + ctx->processing_buffer_list.cnt++;
> + spin_unlock(&ctx->processing_buffer_list.lock);
> + base_addr = buf_entry->buffer.iova;
> + mtk_cam_raw_apply_cq(raw_dev, base_addr,
> + buf_entry->cq_desc_size,
> + buf_entry->cq_desc_offset,
> + buf_entry->sub_cq_desc_size,
> + buf_entry->sub_cq_desc_offset);
> +
> + if (buf) {
> + buf->state.estate = E_BUF_STATE_CQ;
> + return;
> + }
> +
> + /* Transit state from Sensor -> CQ */
> + if (ctx->sensor) {
> + /* req_stream_data of req_cq*/
> + req_stream_data = mtk_cam_ctrl_state_to_req_s_data(current_state);
> + state_transition(current_state,
> + E_STATE_SENSOR, E_STATE_CQ);
In patch [8/13] I says ctx->sensor is always NULL, so these code would not execute.
This is the only code that change state to E_STATE_CQ but it never execute,
so E_STATE_CQ is useless. Drop it.
> +
> + dev_dbg(raw_dev->dev,
> + "SOF[ctx:%d-#%d], CQ-%d is update, composed:%d, cq_addr:0x%pad, time:%lld, monotime:%lld\n",
> + ctx->stream_id, dequeued_frame_seq_no,
> + req_stream_data->frame_seq_no,
> + ctx->composed_frame_seq_no, &base_addr,
> + req_stream_data->timestamp,
> + req_stream_data->timestamp_mono);
> + }
> + }
> +}
> +
[snip]
> +void mtk_camsys_frame_done(struct mtk_cam_ctx *ctx,
> + unsigned int frame_seq_no,
> + unsigned int pipe_id)
static void mtk_camsys_frame_done(
> +{
> + struct mtk_cam_req_work *frame_done_work;
> + struct mtk_cam_request_stream_data *req_stream_data;
> + struct mtk_cam_buffer *buf;
> + struct mtk_cam_working_buf_entry *buf_entry = NULL;
> + bool is_pending_buffer = false;
> +
> + spin_lock(&ctx->cam->dma_processing_lock);
> + if (!list_empty(&ctx->cam->dma_processing)) {
> + buf = list_first_entry(&ctx->cam->dma_processing,
> + struct mtk_cam_buffer, list);
> + list_del(&buf->list);
> + ctx->cam->dma_processing_count--;
> + vb2_buffer_done(&buf->vbb.vb2_buf, VB2_BUF_STATE_DONE);
> + is_pending_buffer = true;
> + }
> + spin_unlock(&ctx->cam->dma_processing_lock);
> + if (is_pending_buffer) {
> + spin_lock(&ctx->processing_buffer_list.lock);
> + if (!list_empty(&ctx->processing_buffer_list.list)) {
> + buf_entry =
> + list_first_entry(&ctx->processing_buffer_list.list,
> + struct mtk_cam_working_buf_entry,
> + list_entry);
> + list_del(&buf_entry->list_entry);
> + ctx->processing_buffer_list.cnt--;
> + }
> + spin_unlock(&ctx->processing_buffer_list.lock);
> + if (buf_entry)
> + mtk_cam_working_buf_put(buf_entry);
> +
> + mtk_cam_buf_try_queue(ctx);
> + return;
> + }
> +
> + req_stream_data = mtk_cam_get_req_s_data(ctx, pipe_id, frame_seq_no);
> + if (!req_stream_data) {
> + dev_err(ctx->cam->dev,
> + "%s:ctx-%d:pipe-%d:req(%d) not found!\n",
> + __func__, ctx->stream_id, pipe_id, frame_seq_no);
> + return;
> + }
> +
> + if (atomic_read(&req_stream_data->frame_done_work.is_queued)) {
> + dev_info(ctx->cam->dev,
> + "already queue done work req:%d seq:%d pipe_id:%d\n",
> + req_stream_data->frame_seq_no, frame_seq_no, pipe_id);
> + return;
> + }
> +
> + atomic_set(&req_stream_data->frame_done_work.is_queued, 1);
> + frame_done_work = &req_stream_data->frame_done_work;
> + queue_work(ctx->frame_done_wq, &frame_done_work->work);
> +}
> +
[snip]
> +static bool mtk_camsys_is_all_cq_done(struct mtk_cam_ctx *ctx,
> + unsigned int pipe_id)
> +{
> + unsigned int all_subdevs = 0;
> + bool ret = false;
> +
> + spin_lock(&ctx->first_cq_lock);
> + if (ctx->is_first_cq_done) {
> + ret = true;
> + spin_unlock(&ctx->first_cq_lock);
> + goto EXIT;
> + }
> +
> + // update cq done status
Run checkpatch.pl before you send out patch.
checkpatch would tell you not to use c++ style comment.
> + ctx->cq_done_status |= (1 << pipe_id);
> +
> + // check cq done status
> + if (ctx->used_raw_num)
> + all_subdevs |= (1 << ctx->pipe->id);
> + if ((ctx->cq_done_status & all_subdevs) == all_subdevs) {
> + ctx->is_first_cq_done = 1;
> + ret = true;
> + }
> + spin_unlock(&ctx->first_cq_lock);
> + dev_info(ctx->cam->dev,
> + "[1st-CQD] all done:%d, pipe_id:%d (using raw:%d)\n",
> + ctx->is_first_cq_done, pipe_id, ctx->used_raw_num);
Why print is_first_cq_done?
I think this is not necessary.
> +EXIT:
> + return ret;
> +}
> +
> +static int mtk_camsys_event_handle_raw(struct mtk_cam_device *cam,
> + unsigned int engine_id,
> + struct mtk_camsys_irq_info *irq_info)
> +{
> + struct mtk_raw_device *raw_dev;
> + struct mtk_cam_ctx *ctx;
> +
> + raw_dev = dev_get_drvdata(cam->raw.devs[engine_id]);
> + ctx = mtk_cam_find_ctx(cam, &raw_dev->pipeline->subdev.entity);
> + if (!ctx) {
> + dev_err(raw_dev->dev, "cannot find ctx\n");
> + return -EINVAL;
> + }
> +
> + /* raw's CQ done */
> + if (irq_info->irq_type & 1 << CAMSYS_IRQ_SETTING_DONE) {
> + if (mtk_camsys_is_all_cq_done(ctx, ctx->pipe->id))
> + mtk_camsys_raw_cq_done(raw_dev, ctx,
> + irq_info->frame_idx);
> + }
> +
> + /* raw's DMA done, we only allow AFO done here */
> + if (irq_info->irq_type & 1 << CAMSYS_IRQ_AFO_DONE)
> + mtk_cam_meta1_done(ctx, ctx->dequeued_frame_seq_no, ctx->stream_id);
> +
> + /* raw's SW done */
> + if (irq_info->irq_type & 1 << CAMSYS_IRQ_FRAME_DONE) {
> + mtk_camsys_frame_done(ctx, ctx->dequeued_frame_seq_no,
> + ctx->stream_id);
> + }
> + /* raw's SOF */
> + if (irq_info->irq_type & 1 << CAMSYS_IRQ_FRAME_START) {
> + if (atomic_read(&raw_dev->vf_en) == 0) {
> + dev_info(raw_dev->dev, "skip sof event when vf off\n");
> + return 0;
> + }
> + mtk_camsys_raw_frame_start(raw_dev, ctx, irq_info);
> + }
> +
> + return 0;
> +}
> +
[snip]
> +
> +struct mtk_camsys_irq_normal_data {
> + /* with normal_status */
mtk_camsys_irq_normal_data is nothing. Drop it.
> +};
> +
> +struct mtk_camsys_irq_error_data {
> + /* with error_status */
> + int err_status;
> +};
> +
> +struct mtk_camsys_irq_info {
> + enum MTK_CAMSYS_IRQ_EVENT irq_type;
> + u64 ts_ns;
> + int frame_idx;
> + int frame_idx_inner;
> + bool sub_engine;
sub_engine is useless. Drop it.
> + int write_cnt;
> + int fbc_cnt;
> + union {
> + struct mtk_camsys_irq_normal_data n;
> + struct mtk_camsys_irq_error_data e;
e is never used. Drop it.
> + };
> +};
> +
> +enum MTK_CAMSYS_STATE_IDX {
> + /* For state analysis and controlling for request */
> + E_STATE_READY = 0x0,
> + E_STATE_SENINF,
state never change to E_STATE_SENINF. Drop it.
> + E_STATE_SENSOR,
> + E_STATE_CQ,
> + E_STATE_OUTER,
> + E_STATE_CAMMUX_OUTER_CFG,
state never change to E_STATE_CAMMUX_OUTER_CFG. Drop it.
> + E_STATE_CAMMUX_OUTER,
state never change to E_STATE_CAMMUX_OUTER. Drop it.
> + E_STATE_INNER,
> + E_STATE_DONE_NORMAL,
> + E_STATE_CQ_SCQ_DELAY,
> + E_STATE_CAMMUX_OUTER_CFG_DELAY,
state never change to E_STATE_CAMMUX_OUTER_CFG_DELAY. Drop it.
> + E_STATE_OUTER_HW_DELAY,
> + E_STATE_INNER_HW_DELAY,
> + E_STATE_DONE_MISMATCH,
> +};
> +
> +struct mtk_camsys_ctrl_state {
> + enum MTK_CAMSYS_STATE_IDX estate;
> + struct list_head state_element;
> +};
> +
> +struct mtk_camsys_link_ctrl {
mtk_camsys_link_ctrl is useless. Drop it.
> + struct mtk_raw_pipeline *pipe;
> + struct media_pad remote;
> + struct mtk_cam_ctx *swapping_ctx;
> + u8 active;
> + u8 wait_exchange;
> +};
> +
> +/* per stream from sensor */
> +struct mtk_camsys_sensor_ctrl {
> + struct mtk_cam_ctx *ctx;
> + struct kthread_worker *sensorsetting_wq;
> + struct kthread_work work;
> + struct hrtimer sensor_deadline_timer;
> + u64 sof_time;
> + int timer_req_sensor;
> + int timer_req_event;
> + atomic_t sensor_enq_seq_no;
> + atomic_t sensor_request_seq_no;
> + atomic_t isp_request_seq_no;
> + atomic_t isp_enq_seq_no;
> + atomic_t last_drained_seq_no;
> + int initial_cq_done;
> + atomic_t initial_drop_frame_cnt;
> + struct list_head camsys_state_list;
> + spinlock_t camsys_state_lock; /* protect camsys_state_list */
> + /* link change ctrl */
> + struct mtk_camsys_link_ctrl link_ctrl;
link_ctrl is useless. Drop it.
> + struct mtk_cam_request *link_change_req;
link_change_req is useless. Drop it.
Regards,
CK
> +};
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 09/13] media: platform: mediatek: add isp_7x utility
[not found] ` <20250707013154.4055874-10-shangyao.lin@mediatek.com>
@ 2025-07-22 2:19 ` CK Hu (胡俊光)
0 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-22 2:19 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduce utility files for the MediaTek ISP7.x camsys driver. These utilities
> provide essential platform definitions, debugging tools, and management
> functions to support ISP operations and SCP communication.
>
> Key functionalities include:
> - Hardware pipeline and register definitions for managing image processing
> modules
> - Buffer management functions
> - Definitions of supported image formats for proper data handling
> - IPI and SCP communication structures for module state management and ISP
> configuration
> - Metadata parameters for configuring image processing
>
> Changes in v2:
> - Removed mtk_cam-dmadbg.h along with related code
> - Various fixes per review comments
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
[snip]
> +/*
> + * A U T O F O C U S
> + */
> +
> +/*
> + * struct mtk_cam_uapi_af_param - af statistic parameters
> + * @roi: AF roi rectangle (in pixel) for AF statistic covered, including
> + * x, y, width, height
> + * @th_sat_g: green channel pixel value saturation threshold (0~255)
> + * @th_h[3]: horizontal AF filters response threshold (0~50) for H0, H1,
> + * and H2
> + * @th_v: vertical AF filter response threshold (0~50)
> + * @blk_pixel_xnum: horizontal number of pixel per block
> + * @blk_pixel_ynum: vertical number of pixel per block
> + * @fir_type: to select FIR filter by AF target type (0,1,2,3)
> + * @iir_type: to select IIR filter by AF target type (0,1,2,3)
> + * @data_gain[7]: gamma curve gain for AF source data
> + */
> +struct mtk_cam_uapi_af_param {
Move all uapi definition to uapi folder.
But I does not see driver use mtk_cam_uapi_af_param.
If this is useless. Drop it.
> + struct mtk_cam_uapi_meta_rect roi;
> + u32 th_sat_g;
> + u32 th_h[3];
> + u32 th_v;
> + u32 blk_pixel_xnum;
> + u32 blk_pixel_ynum;
> + u32 fir_type;
> + u32 iir_type;
> + u32 data_gain[7];
> +} __packed;
> +
[snip]
> +/*
> + * struct mtk_cam_uapi_pdp_stats - statistics of pd
> + *
> + * @stats_src: source width and heitgh of the statistics.
> + * @stride: stride value used by
> + * @pdo_buf: The buffer for PD statistic hardware output.
> + *
> + * This is the PD statistic returned to user.
> + */
> +struct mtk_cam_uapi_pdp_stats {
I does not see driver use mtk_cam_uapi_pdp_stats.
If this is useless. Drop it.
> + struct mtk_cam_uapi_meta_size stats_src;
> + u32 stride;
> + struct mtk_cam_uapi_meta_hw_buf pdo_buf;
> +} __packed;
> +
[snip]
> +/*
> + * struct mtk_cam_uapi_tnc_stats - Tone2 statistic data for
> + * Mediatek proprietary algorithm
> + *
> + * @tncso_buf: The buffer for tnc statistic hardware output. The buffer size
> + * is defined in MTK_CAM_UAPI_TNCSO_SIZE (680*510*2)
> + */
> +struct mtk_cam_uapi_tnc_stats {
> + struct mtk_cam_uapi_meta_hw_buf tncso_buf;
It's weird that define one structure including only one structure.
Drop mtk_cam_uapi_tnc_stats.
> +} __packed;
> +
> +/*
> + * struct mtk_cam_uapi_tnch_stats - Tone3 statistic data for Mediatek
> + * proprietary algorithm
> + *
> + * @tncsho_buf: The buffer for tnch statistic hardware output. The buffer size
> + * is defined in MTK_CAM_UAPI_TNCSHO_SIZE (1544)
> + */
> +struct mtk_cam_uapi_tnch_stats {
> + struct mtk_cam_uapi_meta_hw_buf tncsho_buf;
It's weird that define one structure including only one structure.
Drop mtk_cam_uapi_tnch_stats.
> +} __packed;
> +
> +/*
> + * struct mtk_cam_uapi_tncb_stats - Tone4 statistic data for Mediatek
> + * proprietary algorithm
> + *
> + * @tncsbo_buf: The buffer for tncb statistic hardware output. The buffer size
> + * is defined in MTK_CAM_UAPI_TNCSBO_SIZE (3888)
> + */
> +struct mtk_cam_uapi_tncb_stats {
> + struct mtk_cam_uapi_meta_hw_buf tncsbo_buf;
It's weird that define one structure including only one structure.
Drop mtk_cam_uapi_tncb_stats.
> +} __packed;
> +
> +/*
> + * struct mtk_cam_uapi_tncy_stats - Tone3 statistic data for Mediatek
> + * proprietary algorithm
> + *
> + * @tncsyo_buf: The buffer for tncy statistic hardware output. The buffer size
> + * is defined in MTK_CAM_UAPI_TNCSYO_SIZE (68)
> + */
> +struct mtk_cam_uapi_tncy_stats {
> + struct mtk_cam_uapi_meta_hw_buf tncsyo_buf;
It's weird that define one structure including only one structure.
Drop mtk_cam_uapi_tncy_stats.
> +} __packed;
> +
> +/*
> + * struct mtk_cam_uapi_act_stats - act statistic data for Mediatek
> + * proprietary algorithm
> + *
> + * @actso_buf: The buffer for tncy statistic hardware output. The buffer size
> + * is defined in MTK_CAM_UAPI_ACTSO_SIZE (768)
> + */
> +#define MTK_CAM_UAPI_ACTSO_SIZE (768)
> +struct mtk_cam_uapi_act_stats {
> + struct mtk_cam_uapi_meta_hw_buf actso_buf;
It's weird that define one structure including only one structure.
Drop mtk_cam_uapi_act_stats.
> +} __packed;
> +
[snip]
> +/*
> + * Usage example: To print value of "MTK_CAM_BPC_BPC_FUNC_CON_BPC_BPC_LUT_BIT_EXTEND_EN" in "val"
> + * > printf("%x", GET_MTK_CAM(val, MTK_CAM_BPC_BPC_FUNC_CON_BPC_BPC_LUT_BIT_EXTEND_EN));
> + */
> +#define GET_MTK_CAM(val, field) (((val) & field##_MASK) >> \
> + field##_SHIFT)
> +/*
> + * Usage example: To set "val_of_bpc_lut_bit_extend_en" to bits
> + * of "MTK_CAM_BPC_BPC_FUNC_CON_BPC_BPC_LUT_BIT_EXTEND_EN" in "val"
> + * > val = SET_MTK_CAM(val, MTK_CAM_BPC_BPC_FUNC_CON_BPC_BPC_LUT_BIT_EXTEND_EN,
> + * val_of_bpc_lut_bit_extend_en);
> + */
> +#define SET_MTK_CAM(val, field, set_val) (((val) & ~field##_MASK) | \
> + (((set_val) << field##_SHIFT) & field##_MASK))
> +
> +/*
> + * Bit Feild of BPC_FUNC_CON: BPC_EN
> + * MTK_CAM_BPC_FUNC_CON_BPC_EN: [31, 31]
> + * Enable/disable for BPC Correction
> + * 1'd1: enable the function
> + * 1'd0: disable the function
> + */
> +#define MTK_CAM_BPC_FUNC_CON_BPC_EN_MASK 0x80000000
I can not find anywhere to use this.
If it is useless, drop it.
If it is usable, it seams a structure could simplify these definition.
struct bpc_ctrl {
u32 bpc_lut_bit_extend_en:1;
u32 reserved:27;
u32 bpc_lut_en:1;
u32 bpc_pdc_en:1;
u32 bpc_ct_en:1;
u32 bpc_en:1;
};
> +#define MTK_CAM_BPC_FUNC_CON_BPC_EN_SHIFT 31
> +
> +/*
> + * Bit Feild of BPC_FUNC_CON: BPC_CT_EN
> + * MTK_CAM_BPC_FUNC_CON_BPC_CT_EN: [30, 30]
> + * Enable/disable for Cross-Talk compensation
> + * 1'd1: enable
> + * 1'd0: disable
> + */
> +#define MTK_CAM_BPC_FUNC_CON_BPC_CT_EN_MASK 0x40000000
> +#define MTK_CAM_BPC_FUNC_CON_BPC_CT_EN_SHIFT 30
> +
> +/*
> + * Bit Feild of BPC_FUNC_CON: BPC_PDC_EN
> + * MTK_CAM_BPC_FUNC_CON_BPC_PDC_EN: [29, 29]
> + * Enable/disable for PDC correction
> + * 1'd1: enable
> + * 1'd0: disable
> + */
> +#define MTK_CAM_BPC_FUNC_CON_BPC_PDC_EN_MASK 0x20000000
> +#define MTK_CAM_BPC_FUNC_CON_BPC_PDC_EN_SHIFT 29
> +
> +/*
> + * Bit Feild of BPC_FUNC_CON: BPC_LUT_EN
> + * MTK_CAM_BPC_FUNC_CON_BPC_LUT_EN: [28, 28]
> + * Enable table lookup
> + * 1'd1: enable BPC with default table mode
> + * 1'd0: disable BPC with default table mode
> + */
> +#define MTK_CAM_BPC_FUNC_CON_BPC_LUT_EN_MASK 0x10000000
> +#define MTK_CAM_BPC_FUNC_CON_BPC_LUT_EN_SHIFT 28
> +
> +/*
> + * Bit Feild of BPC_FUNC_CON: BPC_LUT_BIT_EXTEND_EN
> + * MTK_CAM_BPC_FUNC_CON_BPC_LUT_BIT_EXTEND_EN: [0, 0]
> + * Enable table 24 bits mode
> + * 1'd1: Table format to be 24 bits
> + * 1'd0: @ the original format, tbale to be 16 bits mode
> + */
> +#define MTK_CAM_BPC_FUNC_CON_BPC_LUT_BIT_EXTEND_EN_MASK 0x00000001
> +#define MTK_CAM_BPC_FUNC_CON_BPC_LUT_BIT_EXTEND_EN_SHIFT 0
> +
[snip]
> +int mtk_cam_img_working_buf_pool_init(struct mtk_cam_ctx *ctx, int buf_num,
> + struct device *dev)
mtk_cam_img_working_buf_pool_init() is useless. Drop it.
> +{
> + int i, ret;
> + u32 working_buf_size;
> + void *ptr;
> + dma_addr_t addr;
> + struct mtk_cam_device *cam = ctx->cam;
> + struct mtk_cam_video_device *vdev;
> +
> + if (buf_num > CAM_IMG_BUF_NUM) {
> + dev_err(ctx->cam->dev,
> + "%s: ctx(%d): image buffers number too large(%d)\n",
> + __func__, ctx->stream_id, buf_num);
> + WARN_ON(1);
> + return 0;
> + }
> +
> + vdev = &ctx->pipe->vdev_nodes[MTK_RAW_MAIN_STREAM_OUT - MTK_RAW_SINK_NUM];
> + working_buf_size = vdev->active_fmt.fmt.pix_mp.plane_fmt[0].sizeimage;
> + INIT_LIST_HEAD(&ctx->img_buf_pool.cam_freeimglist.list);
> + spin_lock_init(&ctx->img_buf_pool.cam_freeimglist.lock);
> + ctx->img_buf_pool.cam_freeimglist.cnt = 0;
> + ctx->img_buf_pool.working_img_buf_size = buf_num * working_buf_size;
> + ptr = dma_alloc_coherent(cam->smem_dev,
> + ctx->img_buf_pool.working_img_buf_size,
> + &addr, GFP_KERNEL);
> + if (!ptr)
> + return -ENOMEM;
> + ctx->img_buf_pool.working_img_buf_scp_addr = addr;
> + ctx->img_buf_pool.working_img_buf_va = ptr;
> + addr = dma_map_resource(dev, addr, ctx->img_buf_pool.working_img_buf_size,
> + DMA_BIDIRECTIONAL, DMA_ATTR_SKIP_CPU_SYNC);
> + if (dma_mapping_error(dev, addr)) {
> + dev_err(dev, "failed to map scp iova\n");
> + ret = -ENOMEM;
> + goto fail_free_mem;
> + }
> + ctx->img_buf_pool.working_img_buf_iova = addr;
> + dev_dbg(dev,
> + "[%s] img working buf scp addr:%pad va:%pK iova: %pad size %d\n",
> + __func__,
> + &ctx->img_buf_pool.working_img_buf_scp_addr,
> + ctx->img_buf_pool.working_img_buf_va,
> + &ctx->img_buf_pool.working_img_buf_iova,
> + ctx->img_buf_pool.working_img_buf_size);
> +
> + for (i = 0; i < buf_num; i++) {
> + struct mtk_cam_img_working_buf_entry *buf =
> + &ctx->img_buf_pool.img_working_buf[i];
> + int offset = i * working_buf_size;
> +
> + buf->ctx = ctx;
> + buf->img_buffer.va =
> + ctx->img_buf_pool.working_img_buf_va + offset;
> + buf->img_buffer.scp_addr =
> + ctx->img_buf_pool.working_img_buf_scp_addr + offset;
> + buf->img_buffer.iova =
> + ctx->img_buf_pool.working_img_buf_iova + offset;
> + buf->img_buffer.size = working_buf_size;
> +
> + list_add_tail(&buf->list_entry,
> + &ctx->img_buf_pool.cam_freeimglist.list);
> + ctx->img_buf_pool.cam_freeimglist.cnt++;
> + }
> +
> + dev_dbg(dev, "%s: ctx(%d): image buffers init, freebuf cnt(%d)\n",
> + __func__, ctx->stream_id, ctx->img_buf_pool.cam_freeimglist.cnt);
> + return 0;
> +
> +fail_free_mem:
> + dma_free_coherent(cam->smem_dev, ctx->img_buf_pool.working_img_buf_size,
> + ctx->img_buf_pool.working_img_buf_va,
> + ctx->img_buf_pool.working_img_buf_scp_addr);
> + return ret;
> +}
> +
> +void mtk_cam_img_working_buf_pool_release(struct mtk_cam_ctx *ctx,
> + struct device *dev)
Becasue mtk_cam_img_working_buf_pool_init is useless,
so mtk_cam_img_working_buf_pool_release() is useless. Drop it.
> +{
> + struct mtk_cam_device *cam = ctx->cam;
> +
> + dma_unmap_page_attrs(dev, ctx->img_buf_pool.working_img_buf_iova,
> + ctx->img_buf_pool.working_img_buf_size,
> + DMA_BIDIRECTIONAL,
> + DMA_ATTR_SKIP_CPU_SYNC);
> + dma_free_coherent(cam->smem_dev, ctx->img_buf_pool.working_img_buf_size,
> + ctx->img_buf_pool.working_img_buf_va,
> + ctx->img_buf_pool.working_img_buf_scp_addr);
> + dev_dbg(dev,
> + "%s:ctx(%d):img working buf release, scp addr %pad va %pK iova %pad, sz %u\n",
> + __func__, ctx->stream_id,
> + &ctx->img_buf_pool.working_img_buf_scp_addr,
> + ctx->img_buf_pool.working_img_buf_va,
> + &ctx->img_buf_pool.working_img_buf_iova,
> + ctx->img_buf_pool.working_img_buf_size);
> +}
> +
> +void mtk_cam_img_working_buf_put(struct mtk_cam_img_working_buf_entry *buf_entry)
mtk_cam_img_working_buf_put() is useless. Drop it.
> +{
> + struct mtk_cam_ctx *ctx = buf_entry->ctx;
> + int cnt;
> +
> + spin_lock(&ctx->img_buf_pool.cam_freeimglist.lock);
> +
> + list_add_tail(&buf_entry->list_entry,
> + &ctx->img_buf_pool.cam_freeimglist.list);
> + cnt = ++ctx->img_buf_pool.cam_freeimglist.cnt;
> +
> + spin_unlock(&ctx->img_buf_pool.cam_freeimglist.lock);
> +
> + dev_dbg(ctx->cam->dev, "%s:ctx(%d):iova(0x%pad), free cnt(%d)\n",
> + __func__, ctx->stream_id, &buf_entry->img_buffer.iova, cnt);
> +}
> +
> +struct mtk_cam_img_working_buf_entry *
> +mtk_cam_img_working_buf_get(struct mtk_cam_ctx *ctx)
mtk_cam_img_working_buf_get() is useless. Drop it.
> +{
> + struct mtk_cam_img_working_buf_entry *buf_entry;
> + int cnt;
> +
> + /* get from free list */
> + spin_lock(&ctx->img_buf_pool.cam_freeimglist.lock);
> + if (list_empty(&ctx->img_buf_pool.cam_freeimglist.list)) {
> + spin_unlock(&ctx->img_buf_pool.cam_freeimglist.lock);
> +
> + dev_info(ctx->cam->dev, "%s:ctx(%d):no free buf\n",
> + __func__, ctx->stream_id);
> + return NULL;
> + }
> +
> + buf_entry = list_first_entry(&ctx->img_buf_pool.cam_freeimglist.list,
> + struct mtk_cam_img_working_buf_entry,
> + list_entry);
> + list_del(&buf_entry->list_entry);
> + cnt = --ctx->img_buf_pool.cam_freeimglist.cnt;
> +
> + spin_unlock(&ctx->img_buf_pool.cam_freeimglist.lock);
> +
> + dev_dbg(ctx->cam->dev, "%s:ctx(%d):iova(0x%pad), free cnt(%d)\n",
> + __func__, ctx->stream_id, &buf_entry->img_buffer.iova, cnt);
> +
> + return buf_entry;
> +}
> +
[snip]
> +/* IRQ signal mask */
> +#define INT_ST_MASK_CAM (VS_INT_ST |\
> + TG_INT1_ST |\
> + TG_INT2_ST |\
> + EXPDON_ST |\
> + HW_PASS1_DON_ST |\
> + SOF_INT_ST |\
> + SW_PASS1_DON_ST)
> +
> +/* IRQ Error Mask */
> +#define INT_ST_MASK_CAM_ERR (TG_OVRUN_ST |\
> + TG_GBERR_ST |\
> + CQ_DB_LOAD_ERR_ST |\
> + CQ_MAIN_CODE_ERR_ST |\
> + CQ_MAIN_VS_ERR_ST |\
> + DMA_ERR_ST)
> +
> +/* camsys */
> +#define REG_CAMSYS_CG_SET 0x0004
REG_CAMSYS_CG_SET is useless. Drop it.
> +#define REG_CAMSYS_CG_CLR 0x0008
REG_CAMSYS_CG_CLR is useless. Drop it.
> +
> +#define REG_HALT1_EN 0x0350
> +#define REG_HALT2_EN 0x0354
> +#define REG_HALT3_EN 0x0358
> +#define REG_HALT4_EN 0x035c
> +#define REG_HALT5_EN 0x0360
> +#define REG_HALT6_EN 0x0364
> +#define REG_FLASH 0x03A0
REG_FLASH is useless. Drop it.
> +#define REG_ULTRA_HALT1_EN 0x03c0
> +#define REG_ULTRA_HALT2_EN 0x03c4
> +#define REG_ULTRA_HALT3_EN 0x03c8
> +#define REG_ULTRA_HALT4_EN 0x03cc
> +#define REG_ULTRA_HALT5_EN 0x03d0
> +#define REG_ULTRA_HALT6_EN 0x03d4
> +#define REG_PREULTRA_HALT1_EN 0x03f0
> +#define REG_PREULTRA_HALT2_EN 0x03f4
> +#define REG_PREULTRA_HALT3_EN 0x03f8
> +#define REG_PREULTRA_HALT4_EN 0x03fc
> +#define REG_PREULTRA_HALT5_EN 0x0400
> +#define REG_PREULTRA_HALT6_EN 0x0404
> +
> +/* Status check */
> +#define REG_CTL_EN 0x0000
> +#define REG_CTL_EN2 0x0004
REG_CTL_EN2 is useless. Drop it.
> +
> +/* DMA Enable Register, DMA_EN */
> +#define REG_CTL_MOD5_EN 0x0010
REG_CTL_MOD5_EN is useless. Drop it.
> +#define REG_CTL_MOD6_EN 0x0014
> +/* RAW input trigger*/
> +#define REG_CTL_RAWI_TRIG 0x00C0
> +
> +#define REG_CTL_MISC 0x0060
> +#define CTL_DB_EN BIT(4)
> +
> +#define REG_CTL_SW_CTL 0x00C4
> +#define REG_CTL_START 0x00B0
> +
> +#define REG_CTL_RAW_INT_EN 0x0100
REG_CTL_RAW_INT_EN is useless. Drop it.
> +#define REG_CTL_RAW_INT_STAT 0x0104
> +#define REG_CTL_RAW_INT2_EN 0x0110
REG_CTL_RAW_INT2_EN is useless. Drop it.
> +#define REG_CTL_RAW_INT2_STAT 0x0114
> +#define REG_CTL_RAW_INT3_STAT 0x0124
> +#define REG_CTL_RAW_INT4_STAT 0x0134
> +#define REG_CTL_RAW_INT5_STAT 0x0144
> +#define REG_CTL_RAW_INT6_EN 0x0150
> +#define REG_CTL_RAW_INT6_STAT 0x0154
> +#define REG_CTL_RAW_INT7_EN 0x0160
> +#define REG_CTL_RAW_INT7_STAT 0x0164
> +
> +#define REG_CTL_RAW_MOD_DCM_DIS 0x0300
REG_CTL_RAW_MOD_DCM_DIS is useless. Drop it.
> +#define REG_CTL_RAW_MOD2_DCM_DIS 0x0304
REG_CTL_RAW_MOD2_DCM_DIS is useless. Drop it.
> +#define REG_CTL_RAW_MOD3_DCM_DIS 0x0308
REG_CTL_RAW_MOD3_DCM_DIS is useless. Drop it.
> +#define REG_CTL_RAW_MOD5_DCM_DIS 0x0310
> +#define REG_CTL_RAW_MOD6_DCM_DIS 0x0314
> +
> +#define REG_CTL_DBG_SET 0x00F0
REG_CTL_DBG_SET is useless. Drop it.
> +#define REG_CTL_DBG_PORT 0x00F4
REG_CTL_DBG_PORT is useless. Drop it.
> +#define REG_DMA_DBG_SEL 0x4070
REG_DMA_DBG_SEL is useless. Drop it.
> +#define REG_DMA_DBG_PORT 0x4074
> +#define REG_CTL_DBG_SET2 0x00F8
> +
> +#define REG_CTL_RAW_MOD_REQ_STAT 0x0340
REG_CTL_RAW_MOD_REQ_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD2_REQ_STAT 0x0344
REG_CTL_RAW_MOD2_REQ_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD3_REQ_STAT 0x0348
REG_CTL_RAW_MOD3_REQ_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD4_REQ_STAT 0x034c
REG_CTL_RAW_MOD4_REQ_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD5_REQ_STAT 0x0350
REG_CTL_RAW_MOD5_REQ_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD6_REQ_STAT 0x0354
REG_CTL_RAW_MOD6_REQ_STAT is useless. Drop it.
> +
> +#define REG_CTL_RAW_MOD_RDY_STAT 0x0360
REG_CTL_RAW_MOD_RDY_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD2_RDY_STAT 0x0364
REG_CTL_RAW_MOD2_RDY_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD3_RDY_STAT 0x0368
REG_CTL_RAW_MOD3_RDY_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD4_RDY_STAT 0x036c
REG_CTL_RAW_MOD4_RDY_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD5_RDY_STAT 0x0370
REG_CTL_RAW_MOD5_RDY_STAT is useless. Drop it.
> +#define REG_CTL_RAW_MOD6_RDY_STAT 0x0374
REG_CTL_RAW_MOD6_RDY_STAT is useless. Drop it.
> +
> +#define REG_CQ_EN 0x0400
> +#define REG_SCQ_START_PERIOD 0x0408
> +#define REG_CQ_THR0_CTL 0x0410
> +#define REG_CQ_SUB_CQ_EN 0x06B0
> +#define REG_CQ_SUB_THR0_CTL 0x06C0
> +
[snip]
> +#define REG_TG_INTER_ST 0x073C
> +/* use this MASK to extract TG_CAM_CS from TG_INTER_ST */
> +#define TG_CAM_CS_MASK 0x3f00
> +#define TG_IDLE_ST BIT(8)
> +
> +#define REG_TG_FRMSIZE_ST 0x0738
REG_TG_FRMSIZE_ST is useless. Drop it.
> +#define REG_TG_DCIF_CTL 0x075C
> +#define TG_DCIF_EN BIT(16)
> +
> +#define REG_TG_FRMSIZE_ST_R 0x076C
REG_TG_FRMSIZE_ST_R is useless. Drop it.
> +#define REG_TG_TIME_STAMP 0x0778
> +#define REG_TG_TIME_STAMP_CNT 0x077C
> +
> +/* tg flash */
> +#define REG_TG_XENON_FLASH_CTL 0x0780
> +#define REG_TG_XENON_FLASH_OFFSET 0x0784
> +#define REG_TG_XENON_FLASH_HIGH_WIDTH 0x0788
> +#define REG_TG_XENON_FLASH_LOW_WIDTH 0x078C
> +#define REG_TG_IR_FLASH_CTL 0x0798
> +#define REG_TG_IR_FLASH_OFFSET 0x079C
> +#define REG_TG_IR_FLASH_HIGH_WIDTH 0x07A0
> +#define REG_TG_IR_FLASH_LOW_WIDTH 0x07A4
> +
> +/* for raw & yuv's dma top base */
> +#define CAMDMATOP_BASE 0x4000
CAMDMATOP_BASE is useless. Drop it.
Regards,
CK
> +
> +#define REG_DMA_SOFT_RST_STAT 0x4068
> +#define REG_DMA_SOFT_RST_STAT2 0x406C
> +#define REG_DMA_DBG_CHASING_STATUS 0x4098
> +#define REG_DMA_DBG_CHASING_STATUS2 0x409c
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit
[not found] ` <20250707013154.4055874-9-shangyao.lin@mediatek.com>
` (2 preceding siblings ...)
2025-07-10 7:46 ` CK Hu (胡俊光)
@ 2025-07-24 8:36 ` CK Hu (胡俊光)
3 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-07-24 8:36 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduces the top media device driver for the MediaTek ISP7X CAMSYS. The driver maintains the camera system, including sub-device management, DMA operations, and integration with the V4L2 framework. It handles request stream data, buffer management, MediaTek-specific features, pipeline management, streaming control, and error handling mechanisms. Additionally, it aggregates sub-drivers for the camera interface, raw, and yuv pipelines.
>
> ---
[snip]
> +int mtk_cam_dequeue_req_frame(struct mtk_cam_ctx *ctx,
> + unsigned int dequeued_frame_seq_no,
> + int pipe_id)
> +{
> + struct mtk_cam_request *req, *req_prev;
> + struct mtk_cam_request_stream_data *s_data, *s_data_pipe;
> + struct mtk_cam_request_stream_data *deq_s_data[RUNNING_JOB_DEPTH];
> + struct mtk_raw_pipeline *pipe = ctx->pipe;
> + struct mtk_camsys_sensor_ctrl *sensor_ctrl = &ctx->sensor_ctrl;
> + struct mtk_ae_debug_data ae_data;
> + int buf_state;
> + u32 dequeue_cnt, s_data_cnt, handled_cnt;
> + bool del_job, del_req;
> + bool unreliable = false;
> + unsigned int done_status_latch;
> +
> + memset(&ae_data, 0, sizeof(struct mtk_ae_debug_data));
> + dequeue_cnt = 0;
> + s_data_cnt = 0;
> + spin_lock(&ctx->cam->running_job_lock);
> + list_for_each_entry_safe(req, req_prev, &ctx->cam->running_job_list, list) {
> + if (!(req->pipe_used & (1 << pipe_id)))
> + continue;
> +
> + s_data = mtk_cam_req_get_s_data(req, ctx->stream_id, 0);
> + if (!s_data) {
> + dev_info(ctx->cam->dev,
> + "frame_seq:%d[ctx=%d,pipe=%d], de-queue request not found\n",
> + dequeued_frame_seq_no, ctx->stream_id, pipe_id);
> + continue;
> + }
> +
> + if (s_data->frame_seq_no > dequeued_frame_seq_no)
> + goto STOP_SCAN;
> +
> + deq_s_data[s_data_cnt++] = s_data;
> + if (s_data_cnt >= RUNNING_JOB_DEPTH) {
> + dev_info(ctx->cam->dev,
> + "%s:%s:ctx(%d):pipe(%d):seq(%d/%d) dequeue s_data over local buffer cnt(%d)\n",
> + __func__, req->req.debug_str, ctx->stream_id, pipe_id,
> + s_data->frame_seq_no, dequeued_frame_seq_no,
> + s_data_cnt);
> + goto STOP_SCAN;
> + }
> + }
> +
> +STOP_SCAN:
> + spin_unlock(&ctx->cam->running_job_lock);
> +
> + for (handled_cnt = 0; handled_cnt < s_data_cnt; handled_cnt++) {
> + s_data = deq_s_data[handled_cnt];
> + del_req = false;
> + del_job = false;
> + req = mtk_cam_s_data_get_req(s_data);
> + if (!req) {
> + dev_info(ctx->cam->dev,
> + "%s:ctx(%d):pipe(%d):seq(%d) req not found\n",
> + __func__, ctx->stream_id, pipe_id,
> + s_data->frame_seq_no);
> + continue;
> + }
> +
> + spin_lock(&req->done_status_lock);
> +
> + if (req->done_status & 1 << pipe_id) {
> + /* already handled by another job done work */
> + spin_unlock(&req->done_status_lock);
> + continue;
> + }
> +
> + /* Check whether all pipelines of single ctx are done */
> + req->done_status |= 1 << pipe_id;
> + if ((req->done_status & ctx->streaming_pipe) ==
> + (req->pipe_used & ctx->streaming_pipe))
> + del_job = true;
> +
> + if ((req->done_status & ctx->cam->streaming_pipe) ==
> + (req->pipe_used & ctx->cam->streaming_pipe)) {
> + if (MTK_CAM_REQ_STATE_RUNNING ==
> + atomic_cmpxchg(&req->state,
> + MTK_CAM_REQ_STATE_RUNNING,
> + MTK_CAM_REQ_STATE_DELETING))
> + del_req = true;
> + }
> + done_status_latch = req->done_status;
> + spin_unlock(&req->done_status_lock);
> +
> + if (is_raw_subdev(pipe_id) && debug_ae) {
> + mtk_cam_raw_dump_aa_info(ctx, &ae_data);
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:ctx(%d):pipe(%d):de-queue seq(%d):handle seq(%d),done(0x%x),pipes(req:0x%x,ctx:0x%x,all:0x%x),del_job(%d),del_req(%d),metaout,size(%u,%u),AA(0x%llx,0x%llx,0x%llx,0x%llx)(0x%llx,0x%llx,0x%llx,0x%llx)(0x%llx,0x%llx,0x%llx,0x%llx)(0x%llx,0x%llx,0x%llx,0x%llx)(0x%llx,0x%llx,0x%llx,0x%llx)\n",
> + __func__, req->req.debug_str, ctx->stream_id, pipe_id,
> + dequeued_frame_seq_no, s_data->frame_seq_no,
> + done_status_latch, req->pipe_used,
> + ctx->streaming_pipe, ctx->cam->streaming_pipe,
> + del_job, del_req,
> + pipe->res_config.sink_fmt.width,
> + pipe->res_config.sink_fmt.height,
> + ae_data.obc_r1_sum[0], ae_data.obc_r1_sum[1],
> + ae_data.obc_r1_sum[2], ae_data.obc_r1_sum[3],
> + ae_data.obc_r2_sum[0], ae_data.obc_r2_sum[1],
> + ae_data.obc_r2_sum[2], ae_data.obc_r2_sum[3],
> + ae_data.obc_r3_sum[0], ae_data.obc_r3_sum[1],
> + ae_data.obc_r3_sum[2], ae_data.obc_r3_sum[3],
> + ae_data.aa_sum[0], ae_data.aa_sum[1],
> + ae_data.aa_sum[2], ae_data.aa_sum[3],
> + ae_data.ltm_sum[0], ae_data.ltm_sum[1],
> + ae_data.ltm_sum[2], ae_data.ltm_sum[3]);
Add comment to describe what these information is and how to use these information to debug.
> + } else {
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:ctx(%d):pipe(%d):de-queue seq(%d):handle seq(%d),done(0x%x),pipes(req:0x%x,ctx:0x%x,all:0x%x),del_job(%d),del_req(%d)\n",
> + dequeued_frame_seq_no, s_data->frame_seq_no,
> + done_status_latch, req->pipe_used,
> + ctx->streaming_pipe, ctx->cam->streaming_pipe,
> + del_job, del_req);
> + }
> +
> + if (is_raw_subdev(pipe_id)) {
> + mtk_cam_get_timestamp(ctx, s_data);
> + mtk_cam_req_works_clean(s_data);
> + }
> +
> + if (del_job) {
> + atomic_dec(&ctx->running_s_data_cnt);
> + mtk_camsys_state_delete(ctx, sensor_ctrl, req);
> +
> + /* release internal buffers */
> + finish_cq_buf(s_data);
> + }
> +
> + if (del_req) {
> + mtk_cam_del_req_from_running(ctx, req, pipe_id);
> + dequeue_cnt++;
> + }
> +
> + /* release vb2 buffers of the pipe */
> + s_data_pipe = mtk_cam_req_get_s_data(req, pipe_id, 0);
> + if (!s_data_pipe) {
> + dev_info(ctx->cam->dev,
> + "%s:%s:ctx(%d):pipe(%d):seq(%d) s_data_pipe not found\n",
> + __func__, req->req.debug_str, ctx->stream_id, pipe_id,
> + s_data->frame_seq_no);
> + continue;
> + }
> +
> + if (s_data->frame_seq_no < dequeued_frame_seq_no) {
> + buf_state = VB2_BUF_STATE_ERROR;
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:pipe(%d) seq:%d, time:%lld drop, ctx:%d\n",
> + __func__, req->req.debug_str, pipe_id,
> + s_data->frame_seq_no, s_data->timestamp,
> + ctx->stream_id);
> + } else if (s_data->state.estate == E_STATE_DONE_MISMATCH) {
> + buf_state = VB2_BUF_STATE_ERROR;
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:pipe(%d) seq:%d, state done mismatch",
> + __func__, req->req.debug_str, pipe_id,
> + s_data->frame_seq_no);
> + } else if (unreliable) {
> + buf_state = VB2_BUF_STATE_ERROR;
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:pipe(%d) seq:%d, done (unreliable)",
> + __func__, req->req.debug_str, pipe_id,
> + s_data->frame_seq_no);
> + } else {
> + buf_state = VB2_BUF_STATE_DONE;
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:pipe(%d) seq:%d, done success",
> + __func__, req->req.debug_str, pipe_id,
> + s_data->frame_seq_no);
> + }
> +
> + if (mtk_cam_s_data_set_buf_state(s_data_pipe, buf_state)) {
> + /* handle vb2_buffer_done */
> + if (mtk_cam_req_put(req, pipe_id))
> + dev_dbg(ctx->cam->dev,
> + "%s:%s:pipe(%d) return request",
> + __func__, req->req.debug_str, pipe_id);
> + }
> + }
> +
> + return dequeue_cnt;
> +}
> +
[snip]
> +
> +void isp_composer_destroy_session(struct mtk_cam_ctx *ctx)
Only error case would call isp_composer_destroy_session().
In normal stop flow, it does not call isp_composer_destroy_session().
This would result in hang up in mtk_cam_stop_ctx().
It would wait compeletion but it would never complete because isp_composer_destroy_session() is not called.
Test your driver with stop flow and fix it.
> +{
> + struct mtk_cam_device *cam = ctx->cam;
> + struct mtkcam_ipi_event event;
> + struct mtkcam_ipi_session_cookie *session = &event.cookie;
> +
> + memset(&event, 0, sizeof(event));
> + event.cmd_id = CAM_CMD_DESTROY_SESSION;
> + session->session_id = ctx->stream_id;
> + scp_ipi_send(cam->scp, SCP_IPI_ISP_CMD, &event,
> + sizeof(event), MTK_CAM_IPI_SEND_TIMEOUT);
> + dev_info(cam->dev, "IPI send id: %d\n", event.cmd_id);
> +}
> +
[snip]
> +void mtk_cam_apply_pending_dev_config(struct mtk_cam_request_stream_data *s_data)
mtk_cam_apply_pending_dev_config() is useless. Drop it.
> +{
> + struct mtk_cam_req_raw_pipe_data *s_raw_pipe_data;
> + struct mtk_cam_ctx *ctx;
> + char *debug_str = mtk_cam_s_data_get_dbg_str(s_data);
> +
> + s_raw_pipe_data = mtk_cam_s_data_get_raw_pipe_data(s_data);
> + if (!s_raw_pipe_data)
> + return;
> +
> + ctx = mtk_cam_s_data_get_ctx(s_data);
> + if (!ctx)
> + return;
> + ctx->pipe->feature_active = ctx->pipe->user_res.raw_res.feature;
> + ctx->pipe->enabled_raw = s_raw_pipe_data->enabled_raw;
> + ctx->used_raw_dev = s_raw_pipe_data->enabled_raw;
> +
> + dev_info(ctx->cam->dev,
> + "%s:%s:pipe(%d):seq(%d):feature_active(0x%llx), ctx->pipe->user_res.raw_res.feature(%lld), enabled_raw(0x%x)\n",
> + __func__, debug_str, ctx->stream_id, s_data->frame_seq_no,
> + ctx->pipe->feature_active,
> + ctx->pipe->user_res.raw_res.feature,
> + ctx->pipe->enabled_raw);
> +}
> +
[snip]
> +int mtk_cam_call_seninf_set_pixelmode(struct mtk_cam_ctx *ctx,
> + struct v4l2_subdev *sd,
> + int pad_id, int pixel_mode)
static int mtk_cam_call_seninf_set_pixelmode(...
pad_idx is always PAD_SRC_RAW0, so use PAD_SRC_RAW0 to replace pad_id. Drop pad_id.
> +{
> + int ret;
> +
> + ret = mtk_cam_seninf_set_pixelmode(sd, pad_id, pixel_mode);
mtk_cam_seninf_set_pixelmode() would always get PAD_SRC_RAW0 as pad_id,
so drop pad_id parameter of mtk_cam_seninf_set_pixelmode().
> + dev_dbg(ctx->cam->dev,
> + "%s:ctx(%d): seninf(%s): pad(%d), pixel_mode(%d)\n, ret(%d)",
> + __func__, ctx->stream_id, sd->name, pad_id, pixel_mode,
> + ret);
> +
> + return ret;
> +}
> +
> +int mtk_cam_ctx_stream_on(struct mtk_cam_ctx *ctx, struct mtk_cam_video_device *node)
> +{
> + struct mtk_cam_device *cam = ctx->cam;
> + struct device *dev;
> + struct mtk_raw_device *raw_dev;
> + int i, ret;
> + int tgo_pxl_mode;
> + unsigned int streaming_ctx_latch;
> +
> + dev_dbg(cam->dev, "ctx %d stream on, streaming_pipe:0x%x\n",
> + ctx->stream_id, ctx->streaming_pipe);
> +
> + if (ctx->streaming) {
> + dev_dbg(cam->dev, "ctx-%d is already streaming on\n", ctx->stream_id);
> + return 0;
> + }
> +
> + for (i = 0; i < MAX_PIPES_PER_STREAM && ctx->pipe_subdevs[i]; i++) {
> + ret = v4l2_subdev_call(ctx->pipe_subdevs[i], video, s_stream, 1);
> + if (ret) {
> + dev_info(cam->dev, "failed to stream on %s: %d\n",
> + ctx->pipe_subdevs[i]->name, ret);
> + goto fail_pipe_off;
> + }
> + }
> +
> + if (ctx->used_raw_num) {
> + tgo_pxl_mode = ctx->pipe->res_config.tgo_pxl_mode;
> +
> + ret = mtk_cam_dev_config(ctx);
> + if (ret)
> + goto fail_pipe_off;
> + dev = mtk_cam_find_raw_dev(cam, ctx->used_raw_dev);
> + if (!dev) {
> + dev_info(cam->dev, "streamon raw device not found\n");
> + goto fail_pipe_off;
> + }
> + raw_dev = dev_get_drvdata(dev);
> +
> + mtk_cam_call_seninf_set_pixelmode(ctx, ctx->seninf,
> + PAD_SRC_RAW0,
> + tgo_pxl_mode);
> + mtk_cam_seninf_set_camtg(ctx->seninf, PAD_SRC_RAW0,
> + pipeid_to_tgidx(raw_dev->id));
mtk_cam_seninf_set_camtg() parameter pad_id is always PAD_SRC_RAW0,
so use PAD_SRC_RAW0 to replace pad_id and drop pad_id parameter.
> + }
> +
> +
> + ret = v4l2_subdev_call(ctx->seninf, video, s_stream, 1);
> + if (ret) {
> + dev_info(cam->dev, "failed to stream on seninf %s:%d\n",
> + ctx->seninf->name, ret);
> + goto fail_pipe_off;
> + }
> +
> + if (ctx->used_raw_num) {
> + mtk_cam_raw_initialize(raw_dev, 0);
> + /* Twin */
> + if (ctx->pipe->res_config.raw_num_used != 1) {
> + struct mtk_raw_device *raw_dev_sub =
> + get_sub_raw_dev(cam, ctx->pipe);
> + mtk_cam_raw_initialize(raw_dev_sub, 1);
> + if (ctx->pipe->res_config.raw_num_used == 3) {
> + struct mtk_raw_device *raw_dev_sub2 =
> + get_sub2_raw_dev(cam, ctx->pipe);
> + mtk_cam_raw_initialize(raw_dev_sub2, 1);
> + }
> + }
> + }
> +
> + spin_lock(&ctx->streaming_lock);
> +
> + streaming_ctx_latch = cam->streaming_ctx;
> + ctx->streaming = true;
> + cam->streaming_ctx |= 1 << ctx->stream_id;
> + spin_unlock(&ctx->streaming_lock);
> +
> + ret = mtk_camsys_ctrl_start(ctx);
> + if (ret)
> + goto fail_streaming_off;
> +
> + mutex_lock(&cam->queue_lock);
> + mtk_cam_dev_req_try_queue(cam); /* request moved into working list */
> + mutex_unlock(&cam->queue_lock);
> + if (watchdog_scenario(ctx))
> + mtk_ctx_watchdog_start(ctx, 4);
> +
> + dev_dbg(cam->dev, "streamed on camsys ctx:%d\n", ctx->stream_id);
> +
> + return 0;
> +
> +fail_streaming_off:
> + spin_lock(&ctx->streaming_lock);
> + ctx->streaming = false;
> + cam->streaming_ctx &= ~(1 << ctx->stream_id);
> + spin_unlock(&ctx->streaming_lock);
> + v4l2_subdev_call(ctx->seninf, video, s_stream, 0);
> +fail_pipe_off:
> + for (i = 0; i < MAX_PIPES_PER_STREAM && ctx->pipe_subdevs[i]; i++)
> + v4l2_subdev_call(ctx->pipe_subdevs[i], video, s_stream, 0);
> +
> + return ret;
> +}
> +
[snip]
> +static int mtk_cam_master_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_connection *asc)
> +{
> + struct mtk_cam_device *cam_dev =
> + container_of(notifier, struct mtk_cam_device, notifier);
> + struct mtk_raw *raw = &cam_dev->raw;
> + struct device *cam = cam_dev->dev;
> + struct device *dev = subdev->dev;
> +
> + dev_info(cam, "cmasys | %s trigger %s\n", subdev->name, __func__);
> +
> + if (strcmp(dev_driver_string(dev), "seninf") == 0) {
> + dev_dbg(cam, "%s@(seninf) done\n", __func__);
> + } else if (strcmp(dev_driver_string(dev), "mtk-cam raw") == 0) {
> + struct mtk_raw_device *raw_dev = dev_get_drvdata(dev);
> +
> + raw_dev->cam = cam_dev;
> + raw->devs[raw_dev->id] = dev;
raw_dev->id is never set, so raw_dev->id is always 0.
Does this driver support only one raw?
If this driver support only one raw, remove all code related to multiple raw.
One example is changing raw->devs[] to raw->dev, and
raw->dev = dev;
And pipeline number is equal to the raw number.
If raw number is one, imply there is only one pipeline.
If only one pipeline, it's not necessary to have pipe id to identify pipeline.
> + raw->cam_dev = cam_dev->dev;
> + dev_dbg(cam, "%s@(mtk-cam raw) done\n", __func__);
> + } else if (strcmp(dev_driver_string(dev), "mtk-cam yuv") == 0) {
> + struct mtk_yuv_device *yuv_dev = dev_get_drvdata(dev);
> +
> + raw->yuvs[yuv_dev->id] = dev;
yuv_dev->id is never set, so yuv_dev->id is always 0.
Does this driver support only one yuv?
If this driver support only one yuv, remove all code related to multiple yuv.
One example is changing raw->yuvs[] to raw->yuv, and
raw->yuv = dev;
> + dev_dbg(cam, "%s@(mtk-cam yuv) done\n", __func__);
> + } else {
> + dev_warn(cam, "%s got unrecongized device\n", __func__);
> + }
> +
> + return 0;
> +}
> +
[snip]
> +
> +#define SENSOR_FMT_MASK 0xFFFF
> +
> +/* flags of mtk_cam_request */
> +#define MTK_CAM_REQ_FLAG_SENINF_IMMEDIATE_UPDATE BIT(1)
> +
> +/* flags of mtk_cam_request_stream_data */
> +#define MTK_CAM_REQ_S_DATA_FLAG_TG_FLASH BIT(0)
MTK_CAM_REQ_S_DATA_FLAG_TG_FLASH is useless. Drop it.
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_META1_INDEPENDENT BIT(1)
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_SINK_FMT_UPDATE BIT(2)
> +
> +/* Apply sensor mode and the timing is 1 vsync before */
> +#define MTK_CAM_REQ_S_DATA_FLAG_SENSOR_MODE_UPDATE_T1 BIT(3)
MTK_CAM_REQ_S_DATA_FLAG_SENSOR_MODE_UPDATE_T1 is useless. Drop it.
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_EN BIT(4)
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_EN BIT(5)
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_COMPLETE BIT(6)
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_RAW_HDL_COMPLETE BIT(7)
> +
> +#define MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_DELAYED BIT(8)
MTK_CAM_REQ_S_DATA_FLAG_SENSOR_HDL_DELAYED is useless. Drop it.
> +
> +struct mtk_cam_working_buf {
> + void *va;
> + dma_addr_t iova;
> + dma_addr_t scp_addr;
> + unsigned int size;
> +};
> +
[snip]
>
> +
> +struct mtk_cam_req_feature {
> + int raw_feature;
raw_feature is useless. Drop it.
> + bool switch_prev_frame_done;
switch_prev_frame_done is useless. Drop it.
> + bool switch_curr_setting_done;
switch_curr_setting_done is useless. Drop it.
> + bool switch_done;
switch_done is useless. Drop it.
> +};
> +
> +struct mtk_cam_sensor_work {
> + struct kthread_work work;
> + atomic_t is_queued;
> +};
> +
> +/*
> + * struct mtk_cam_request_stream_data - per stream members of a request
> + *
> + * @pad_fmt: pad format configurtion for sensor switch.
> + * @frame_params: The frame info. & address info. of enabled DMA nodes.
> + * @frame_work: work queue entry for frame transmission to SCP.
> + * @working_buf: command queue buffer associated to this request
> + * @deque_list_node: the entry node of s_data for deque
> + * @cleanup_list_node: the entry node of s_data for cleanup
> + *
> + */
> +struct mtk_cam_request_stream_data {
> + u32 index;
> + struct mtk_cam_request *req;
> + struct mtk_cam_ctx *ctx;
> + u32 pipe_id;
> + u32 frame_seq_no;
> + u32 flags;
> + unsigned long raw_dmas;
raw_dmas is useless. Drop it.
> + u64 timestamp;
> + u64 timestamp_mono;
> + atomic_t buf_state; /* default: -1 */
> + struct mtk_cam_buffer *bufs[MTK_RAW_TOTAL_NODES];
> + struct v4l2_subdev *sensor;
> + struct media_request_object *sensor_hdl_obj; /* for complete only */
> + struct media_request_object *raw_hdl_obj; /* for complete only */
> + struct v4l2_subdev_format seninf_fmt;
> + struct v4l2_subdev_format pad_fmt[MTK_RAW_PIPELINE_PADS_NUM];
> + struct v4l2_rect pad_selection[MTK_RAW_PIPELINE_PADS_NUM];
> + struct v4l2_format vdev_fmt[MTK_RAW_TOTAL_NODES];
> + struct v4l2_selection vdev_selection[MTK_RAW_TOTAL_NODES];
> + struct mtkcam_ipi_frame_param frame_params;
> + struct mtk_cam_sensor_work sensor_work;
> + struct mtk_cam_req_work seninf_s_fmt_work;
seninf_s_fmt_work is useless. Drop it.
> + struct mtk_cam_req_work frame_work;
You queue this work from mtk_cam_dev_req_enqueue().
I think the frame work job could be done in mtk_cam_dev_req_enqueue().
So it's not necessary to queue the job to work queue.
Drop frame_work.
> + struct mtk_cam_req_work meta1_done_work;
You queue this work from irq thread.
I think the meta1 done job could be done in irq thread.
So it's not necessary to queue the job to work queue.
Drop meta1_done_work.
> + struct mtk_cam_req_work frame_done_work;
You queue this work from irq thread.
I think the frame done job could be done in irq thread.
So it's not necessary to queue the job to work queue.
Drop frame_done_work.
> + struct mtk_camsys_ctrl_state state;
> + struct mtk_cam_working_buf_entry *working_buf;
> + unsigned int no_frame_done_cnt;
no_frame_done_cnt is useless. Drop it.
> + atomic_t seninf_dump_state;
seninf_dump_state is useless. Drop it.
> + struct mtk_cam_req_feature feature;
mtk_cam_req_feature is empty. Drop feature.
> + struct list_head deque_list_node;
> + struct list_head cleanup_list_node;
> + atomic_t first_setting_check;
> +};
> +
> +struct mtk_cam_req_pipe {
> + int s_data_num;
s_data_num is only set in mtk_cam_req_p_data_init().
mtk_cam_req_p_data_init() is only called in mtk_cam_req_get_pipe_used() with s_data_num = 1.
So s_data_num is always 1.
Use 1 to replace s_data_num and drop s_data_num.
> + int req_seq;
req_seq is useless. Drop it.
> + struct mtk_cam_request_stream_data s_data[MTK_CAM_REQ_MAX_S_DATA];
After previous drop. struct mtk_cam_req_pipe include only one structure. This is weird.
So drop struct mtk_cam_req_pipe and use struct mtk_cam_request_stream_data directly.
> +};
> +
> +enum mtk_cam_request_state {
> + MTK_CAM_REQ_STATE_PENDING,
> + MTK_CAM_REQ_STATE_RUNNING,
> + MTK_CAM_REQ_STATE_DELETING,
> + MTK_CAM_REQ_STATE_COMPLETE,
> + MTK_CAM_REQ_STATE_CLEANUP,
> + NR_OF_MTK_CAM_REQ_STATE,
> +};
> +
[snip]
> +
> +struct mtk_cam_img_working_buf_pool {
In patch [9/13], I says img working buf related things are useless.
Drop them.
> + struct mtk_cam_ctx *ctx;
> + struct dma_buf *working_img_buf_dmabuf;
> + void *working_img_buf_va;
> + dma_addr_t working_img_buf_iova;
> + dma_addr_t working_img_buf_scp_addr;
> + unsigned int working_img_buf_size;
> + struct mtk_cam_img_working_buf_entry img_working_buf[CAM_IMG_BUF_NUM];
> + struct mtk_cam_working_buf_list cam_freeimglist;
> +};
> +
[snip]
> +
> +struct mtk_cam_device {
> + struct device *dev;
> +
> + struct platform_device *clks_pdev;
> + struct v4l2_device v4l2_dev;
> + struct v4l2_async_notifier notifier;
> + struct media_device media_dev;
> + void __iomem *base;
> +
> + struct mtk_scp *scp;
> + struct device *smem_dev;
> + struct rproc *rproc_handle;
> +
> + unsigned int composer_cnt;
> +
> + unsigned int num_seninf_devices;
num_seninf_devices is useless. Drop it.
> + unsigned int num_raw_devices;
> +
> + /* raw_pipe controller subdev */
> + struct mtk_raw raw;
> + struct mutex queue_lock; /* protect queue request */
> +
> + unsigned int max_stream_num;
> + unsigned int streaming_ctx;
> + unsigned int streaming_pipe;
> + struct mtk_cam_ctx *ctxs;
> +
> + /* request related */
> + struct list_head pending_job_list;
> + spinlock_t pending_job_lock; /* protect pending_job_list */
> + struct list_head running_job_list;
> + unsigned int running_job_count;
> + spinlock_t running_job_lock; /* protect running_job_list */
> +
> + /* standard v4l2 buffer control */
> + struct list_head dma_pending;
> + spinlock_t dma_pending_lock; /* protect dma_pending_list */
> + struct list_head dma_processing;
If you don't know how to merge all buffer list together,
you may try merge dma_pending and dma_processing first.
The merged name may be dma_buf_list.
And the buffer has a flag 'processing' to indicate it's processing or not.
> + spinlock_t dma_processing_lock; /* protect dma_processing_list and dma_processing_count */
> + unsigned int dma_processing_count;
> +
> + struct workqueue_struct *debug_wq;
debug_wq is useless. Drop it.
> + struct workqueue_struct *debug_exception_wq;
debug_exception_wq is useless. Drop it.
Regards,
CK
> +};
> +
> +static inline struct mtk_cam_request_stream_data*
> +mtk_cam_ctrl_state_to_req_s_data(struct mtk_camsys_ctrl_state *state)
> +{
> + return container_of(state, struct mtk_cam_request_stream_data, state);
> +}
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 10/13] media: platform: mediatek: add isp_7x video ops
2025-07-07 1:31 ` [PATCH v2 10/13] media: platform: mediatek: add isp_7x video ops shangyao lin
@ 2025-08-01 5:45 ` CK Hu (胡俊光)
0 siblings, 0 replies; 37+ messages in thread
From: CK Hu (胡俊光) @ 2025-08-01 5:45 UTC (permalink / raw)
To: robh@kernel.org, mchehab@kernel.org, krzk+dt@kernel.org,
conor+dt@kernel.org, matthias.bgg@gmail.com,
Shangyao Lin (林上堯),
AngeloGioacchino Del Regno
Cc: linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
Project_Global_Chrome_Upstream_Group, devicetree@vger.kernel.org,
dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org
On Mon, 2025-07-07 at 09:31 +0800, shangyao lin wrote:
> From: "shangyao.lin" <shangyao.lin@mediatek.com>
>
> Introduce the V4L2 video interface and feature management for the MediaTek
> ISP7.x CAMSYS. These interfaces provide functionalities such as video operation
> initialization and registration. They also manage MediaTek-specific formats and
> handle buffers for MediaTek camera video devices. This enables CAMSYS
> functionalities to be compatible with the V4L2 framework.
>
> Changes in v2:
> - Removed mtk_cam-feature.c and mtk_cam-feature.h, along with related code
> - Various fixes per review comments
>
> Signed-off-by: shangyao.lin <shangyao.lin@mediatek.com>
> ---
[snip]
> +static int mtk_cam_vb2_queue_setup(struct vb2_queue *vq,
> + unsigned int *num_buffers,
> + unsigned int *num_planes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct mtk_cam_device *cam = vb2_get_drv_priv(vq);
> + struct mtk_cam_video_device *node = mtk_cam_vbq_to_vdev(vq);
> + struct mtk_raw_pipeline *raw_pipeline;
> + unsigned int max_buffer_count = node->desc.max_buf_count;
> + const struct v4l2_format *fmt = &node->active_fmt;
> + unsigned int size;
> + int i;
> +
> + /* Check the limitation of buffer size */
> + if (max_buffer_count)
> + *num_buffers = clamp_val(*num_buffers, 1, max_buffer_count);
> +
> + if (node->desc.smem_alloc)
> + vq->dma_attrs |= DMA_ATTR_NO_KERNEL_MAPPING;
I think all buffer should be no kernel mapping.
Which buffer you would access in kernel?
> +
> + if (vq->type == V4L2_BUF_TYPE_META_OUTPUT ||
> + vq->type == V4L2_BUF_TYPE_META_CAPTURE)
> + size = fmt->fmt.meta.buffersize;
> + else
> + size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage;
> +
> + /* Add for q.create_bufs with fmt.g_sizeimage(p) / 2 test */
> + if (*num_planes) {
> + if (sizes[0] < size || *num_planes != 1)
> + return -EINVAL;
> + } else {
> + /* Set default as one plane */
> + *num_planes = 1;
> + sizes[0] = size;
> +
> + if (is_raw_subdev(node->uid.pipe_id)) {
> + raw_pipeline =
> + mtk_cam_dev_get_raw_pipeline(cam, node->uid.pipe_id);
> +
> + if (raw_pipeline &&
> + raw_pipeline->user_res.raw_res.feature &&
> + fmt->fmt.pix_mp.num_planes > 1) {
> + *num_planes = fmt->fmt.pix_mp.num_planes;
> + for (i = 0; i < *num_planes; i++)
> + sizes[i] = size;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
[snip]
> +
> +int is_mtk_format(u32 pixelformat)
Use mtk_cam_get_img_fmt() to replace is_mtk_format().
When mtk_cam_get_img_fmt() return MTK_CAM_IPI_IMG_FMT_UNKNOWN, it's not mtk image format.
> +{
> + switch (pixelformat) {
> + case V4L2_PIX_FMT_YUYV10:
> + case V4L2_PIX_FMT_YVYU10:
> + case V4L2_PIX_FMT_UYVY10:
> + case V4L2_PIX_FMT_VYUY10:
> + case V4L2_PIX_FMT_YUYV12:
> + case V4L2_PIX_FMT_YVYU12:
> + case V4L2_PIX_FMT_UYVY12:
> + case V4L2_PIX_FMT_VYUY12:
> + case V4L2_PIX_FMT_MTISP_YUYV10P:
> + case V4L2_PIX_FMT_MTISP_YVYU10P:
> + case V4L2_PIX_FMT_MTISP_UYVY10P:
> + case V4L2_PIX_FMT_MTISP_VYUY10P:
> + case V4L2_PIX_FMT_MTISP_YUYV12P:
> + case V4L2_PIX_FMT_MTISP_YVYU12P:
> + case V4L2_PIX_FMT_MTISP_UYVY12P:
> + case V4L2_PIX_FMT_MTISP_VYUY12P:
> + case V4L2_PIX_FMT_NV12_10:
> + case V4L2_PIX_FMT_NV21_10:
> + case V4L2_PIX_FMT_NV16_10:
> + case V4L2_PIX_FMT_NV61_10:
> + case V4L2_PIX_FMT_NV12_12:
> + case V4L2_PIX_FMT_NV21_12:
> + case V4L2_PIX_FMT_NV16_12:
> + case V4L2_PIX_FMT_NV61_12:
> + case V4L2_PIX_FMT_MTISP_NV12_10P:
> + case V4L2_PIX_FMT_MTISP_NV21_10P:
> + case V4L2_PIX_FMT_MTISP_NV16_10P:
> + case V4L2_PIX_FMT_MTISP_NV61_10P:
> + case V4L2_PIX_FMT_MTISP_NV12_12P:
> + case V4L2_PIX_FMT_MTISP_NV21_12P:
> + case V4L2_PIX_FMT_MTISP_NV16_12P:
> + case V4L2_PIX_FMT_MTISP_NV61_12P:
> + case V4L2_PIX_FMT_MTISP_NV12_UFBC:
> + case V4L2_PIX_FMT_MTISP_NV21_UFBC:
> + case V4L2_PIX_FMT_MTISP_NV12_10_UFBC:
> + case V4L2_PIX_FMT_MTISP_NV21_10_UFBC:
> + case V4L2_PIX_FMT_MTISP_NV12_12_UFBC:
> + case V4L2_PIX_FMT_MTISP_NV21_12_UFBC:
> + case V4L2_PIX_FMT_MTISP_BAYER8_UFBC:
> + case V4L2_PIX_FMT_MTISP_BAYER10_UFBC:
> + case V4L2_PIX_FMT_MTISP_BAYER12_UFBC:
> + case V4L2_PIX_FMT_MTISP_BAYER14_UFBC:
> + case V4L2_PIX_FMT_MTISP_SGRB8F:
> + case V4L2_PIX_FMT_MTISP_SGRB10F:
> + case V4L2_PIX_FMT_MTISP_SGRB12F:
> + return 1;
> + default:
> + return 0;
> + }
> +}
> +
[snip]
> +const struct mtk_format_info *mtk_format_info(u32 format)
> +{
> + static const struct mtk_format_info formats[] = {
Move this array out of this function.
> + /* YUV planar formats */
> + { .format = V4L2_PIX_FMT_NV12_10,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
mem_planes is always 1. Use 1 to replace mem_planes. Drop mem_planes.
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV21_10,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
bpp[3] is always 0. So shrink array size of bpp[] to 3.
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV16_10,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV61_10,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_YUYV10,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_YVYU10,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_UYVY10,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_VYUY10,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV12_12,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV21_12,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV16_12,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_NV61_12,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_YUYV12,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_YVYU12,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_UYVY12,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_VYUY12,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 2, .bit_r_den = 1 },
> + /* YUV packed formats */
> + { .format = V4L2_PIX_FMT_MTISP_YUYV10P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_YVYU10P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_UYVY10P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_VYUY10P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_NV12_10P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_NV21_10P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_NV16_10P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_NV61_10P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_YUYV12P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_YVYU12P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_UYVY12P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_VYUY12P,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_NV12_12P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_NV21_12P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_NV16_12P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_NV61_12P,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + /* YUV UFBC formats */
> + { .format = V4L2_PIX_FMT_MTISP_NV12_UFBC,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_MTISP_NV21_UFBC,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 2, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_MTISP_NV12_10_UFBC,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_NV21_10_UFBC,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_NV12_12_UFBC,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_NV21_12_UFBC,
> + .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_BAYER8_UFBC,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
> + .hdiv = 1, .vdiv = 1, .bit_r_num = 1, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_MTISP_BAYER10_UFBC,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
> + .hdiv = 1, .vdiv = 1, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_BAYER12_UFBC,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
> + .hdiv = 1, .vdiv = 1, .bit_r_num = 3, .bit_r_den = 2 },
> + { .format = V4L2_PIX_FMT_MTISP_BAYER14_UFBC,
> + .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 },
> + .hdiv = 1, .vdiv = 1, .bit_r_num = 7, .bit_r_den = 4 },
> + /* Full-G RGB formats */
> + { .format = V4L2_PIX_FMT_MTISP_SGRB8F,
> + .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 1, .bit_r_den = 1 },
> + { .format = V4L2_PIX_FMT_MTISP_SGRB10F,
> + .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 5, .bit_r_den = 4 },
> + { .format = V4L2_PIX_FMT_MTISP_SGRB12F,
> + .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 },
> + .hdiv = 2, .vdiv = 2, .bit_r_num = 3, .bit_r_den = 2 },
> + };
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); ++i)
> + if (formats[i].format == format)
> + return &formats[i];
> + return NULL;
> +}
> +
[snip]
> +static void mtk_cam_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
> + u32 pixelformat, u32 width, u32 height)
> +{
> + struct v4l2_plane_pix_format *plane;
> + unsigned int ipi_fmt = mtk_cam_get_img_fmt(pixelformat);
mtk_cam_fill_pixfmt_mp() is called only by cal_image_pix_mp() and cal_image_pix_mp() already has ipi_fmt.
So pass ipi_fmt as parameter and do not call mtk_cam_get_img_fmt() again.
> + u8 pixel_bits = mtk_cam_get_pixel_bits(ipi_fmt);
> + u32 stride;
> + u32 aligned_width;
> + u8 bus_size;
> + u8 i;
> +
> + pixfmt->width = width;
> + pixfmt->height = height;
> + pixfmt->pixelformat = pixelformat;
> + plane = &pixfmt->plane_fmt[0];
> + bus_size = mtk_cam_yuv_dma_bus_size(pixel_bits, 0);
> + plane->sizeimage = 0;
> +
> + if (is_mtk_format(pixelformat)) {
mtk_cam_fill_pixfmt_mp() is called only by cal_image_pix_mp(), so this checking is always true.
Drop this checking.
> + const struct mtk_format_info *info;
> +
> + info = mtk_format_info(pixelformat);
> + if (!info)
> + return;
> +
> + pixfmt->num_planes = info->mem_planes;
> + if (info->mem_planes == 1) {
mem_planes is always 1. Drop this checking.
> + if (is_yuv_ufo(pixelformat)) {
> + /* UFO format width should align 64 pixel */
> + aligned_width = ALIGN(width, 64);
> + stride = aligned_width * info->bit_r_num / info->bit_r_den;
> +
> + if (stride > plane->bytesperline)
> + plane->bytesperline = stride;
> + plane->sizeimage = stride * height;
> + plane->sizeimage += stride * height / 2;
> + plane->sizeimage += ALIGN((aligned_width / 64), 8) * height;
> + plane->sizeimage += ALIGN((aligned_width / 64), 8) * height / 2;
> + } else if (is_raw_ufo(pixelformat)) {
> + /* UFO format width should align 64 pixel */
> + aligned_width = ALIGN(width, 64);
> + stride = aligned_width * info->bit_r_num / info->bit_r_den;
> +
> + if (stride > plane->bytesperline)
> + plane->bytesperline = stride;
> + plane->sizeimage = stride * height;
> + plane->sizeimage += ALIGN((aligned_width / 64), 8) * height;
> + } else {
> + /* width should be bus_size align */
> + aligned_width = ALIGN(DIV_ROUND_UP(width
> + * info->bit_r_num, info->bit_r_den), bus_size);
> + stride = aligned_width * info->bpp[0];
> +
> + if (stride > plane->bytesperline)
> + plane->bytesperline = stride;
> +
> + for (i = 0; i < info->comp_planes; i++) {
> + unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> + unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
> +
> + if (plane->bytesperline > stride &&
> + is_fullg_rb(pixelformat)) {
> + plane->sizeimage +=
> + DIV_ROUND_UP(plane->bytesperline, hdiv)
> + * DIV_ROUND_UP(height, vdiv);
> + } else if (plane->bytesperline > stride &&
> + !is_fullg_rb(pixelformat)) {
> + plane->sizeimage +=
> + plane->bytesperline
> + * DIV_ROUND_UP(height, vdiv);
> + } else {
> + plane->sizeimage += info->bpp[i]
> + * DIV_ROUND_UP(aligned_width, hdiv)
> + * DIV_ROUND_UP(height, vdiv);
> + }
> + }
> + }
> + pr_debug("%s stride %d sizeimage %d\n", __func__,
> + plane->bytesperline, plane->sizeimage);
> + } else {
> + pr_debug("do not support non contiguous mplane\n");
> + }
> + } else {
> + const struct v4l2_format_info *info;
> +
> + pr_debug("pixelformat:0x%x sizeimage:%d\n",
> + pixelformat, plane->sizeimage);
> + info = v4l2_format_info(pixelformat);
> + if (!info)
> + return;
> +
> + pixfmt->num_planes = info->mem_planes;
> + if (info->mem_planes == 1) {
> + aligned_width = ALIGN(width, bus_size);
> + stride = aligned_width * info->bpp[0];
> + if (stride > plane->bytesperline)
> + plane->bytesperline = stride;
> +
> + for (i = 0; i < info->comp_planes; i++) {
> + unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> + unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
> +
> + plane->sizeimage += info->bpp[i]
> + * DIV_ROUND_UP(aligned_width, hdiv)
> + * DIV_ROUND_UP(height, vdiv);
> + }
> + pr_debug("%s stride %d sizeimage %d\n", __func__,
> + plane->bytesperline, plane->sizeimage);
> + } else {
> + pr_warn("do not support non contiguous mplane\n");
> + }
> + }
> +}
> +
[snip]
> +int mtk_cam_video_set_fmt(struct mtk_cam_video_device *node,
> + struct v4l2_format *f, int raw_feature)
> +{
> + struct mtk_cam_device *cam = video_get_drvdata(&node->vdev);
> + const struct v4l2_format *dev_fmt;
> + struct v4l2_format try_fmt;
> + s32 i;
> +
> + dev_dbg(cam->dev,
> + "%s:pipe(%d):%s:feature(0x%x)\n",
> + __func__, node->uid.pipe_id, node->desc.name, raw_feature);
> +
> + memset(&try_fmt, 0, sizeof(try_fmt));
> + try_fmt.type = f->type;
> +
> + /* Validate pixelformat */
> + dev_fmt = mtk_cam_dev_find_fmt(&node->desc, f->fmt.pix_mp.pixelformat);
> + if (!dev_fmt) {
> + dev_dbg(cam->dev, "unknown fmt:%d\n",
> + f->fmt.pix_mp.pixelformat);
> + dev_fmt = &node->desc.fmts[node->desc.default_fmt_idx].vfmt;
> + }
> + try_fmt.fmt.pix_mp.pixelformat = dev_fmt->fmt.pix_mp.pixelformat;
> +
> + /* Validate image width & height range */
> + try_fmt.fmt.pix_mp.width = clamp_val(f->fmt.pix_mp.width,
> + IMG_MIN_WIDTH, IMG_MAX_WIDTH);
> + try_fmt.fmt.pix_mp.height = clamp_val(f->fmt.pix_mp.height,
> + IMG_MIN_HEIGHT, IMG_MAX_HEIGHT);
> + /* 4 bytes alignment for width */
> + /* width and stride should align bus_size */
> + try_fmt.fmt.pix_mp.width = ALIGN(try_fmt.fmt.pix_mp.width, IMG_PIX_ALIGN);
> + try_fmt.fmt.pix_mp.num_planes = 1;
> +
> + for (i = 0 ; i < try_fmt.fmt.pix_mp.num_planes ; i++)
> + try_fmt.fmt.pix_mp.plane_fmt[i].bytesperline =
> + f->fmt.pix_mp.plane_fmt[i].bytesperline;
Why do you assign this?
You would calculate bytesperline in cal_image_pix_mp().
And you compare new cal_image_pix_mp and old cal_image_pix_mp to do something,
but I think it's not necessary.
Just calculate new bytesperline in cal_image_pix_mp() and don't care about old bytesperline.
> +
> + /* bytesperline & sizeimage calculation */
> + cal_image_pix_mp(node->desc.id, &try_fmt.fmt.pix_mp, 0);
> +
> + /* Constant format fields */
> + try_fmt.fmt.pix_mp.colorspace = V4L2_COLORSPACE_SRGB;
> + try_fmt.fmt.pix_mp.field = V4L2_FIELD_NONE;
> + try_fmt.fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> + try_fmt.fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
> + try_fmt.fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_SRGB;
> +
> + *f = try_fmt;
> +
> + return 0;
> +}
> +
[snip]
> +/**
> + * struct mtk_cam_video_device - MediaTek video device structure.
> + */
> +struct mtk_cam_video_device {
> + struct mtkcam_ipi_uid uid;
> + struct mtk_cam_dev_node_desc desc;
> + unsigned int enabled;
> +
> + struct vb2_queue vb2_q;
> + struct video_device vdev;
> + struct media_pad pad;
> + struct v4l2_format active_fmt;
> + /* use first 4 elements of reserved field of v4l2_pix_format_mplane as request fd */
> + struct v4l2_format pending_fmt;
pending_fmt is useless. Drop it.
> + /* use first elements of reserved field of v4l2_selection as request fd*/
> + struct v4l2_selection active_crop;
> + /* Serializes vb2 queue and video device operations */
> + struct mutex q_lock;
> + int streaming_id;
> +
> + /* cached ctx info */
> + struct mtk_cam_ctx *ctx;
> +};
> +
> +struct mtk_format_info {
> + u32 format;
> + u8 mem_planes;
mem_planes is always 1. Drop it.
Regards,
CK
> + u8 comp_planes;
> + u8 bpp[4];
> + u8 hdiv;
> + u8 vdiv;
> + u8 bit_r_num; /* numerator of bit ratio */
> + u8 bit_r_den; /* denominator of bit ratio */
> +};
> +
^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2025-08-01 5:48 UTC | newest]
Thread overview: 37+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-07 1:31 [PATCH v2 00/13] Add MediaTek ISP7.x camera system support shangyao lin
2025-07-07 1:31 ` [PATCH v2 01/13] dt-bindings: media: mediatek: add camisp binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:42 ` Krzysztof Kozlowski
2025-07-15 8:25 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 02/13] dt-bindings: media: mediatek: add seninf-core binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:44 ` Krzysztof Kozlowski
2025-07-15 7:28 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 03/13] dt-bindings: media: mediatek: add cam-raw binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:48 ` Krzysztof Kozlowski
2025-07-15 8:13 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 04/13] dt-bindings: media: mediatek: add cam-yuv binding shangyao lin
2025-07-07 2:46 ` Rob Herring (Arm)
2025-07-07 5:47 ` Krzysztof Kozlowski
2025-07-15 7:41 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 06/13] media: platform: mediatek: add isp_7x state ctrl shangyao lin
2025-07-11 8:56 ` CK Hu (胡俊光)
2025-07-21 1:40 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 10/13] media: platform: mediatek: add isp_7x video ops shangyao lin
2025-08-01 5:45 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 11/13] media: platform: mediatek: add isp_7x build config shangyao lin
2025-07-07 1:31 ` [PATCH v2 12/13] uapi: linux: add mediatek isp_7x camsys user api shangyao lin
2025-07-08 1:34 ` CK Hu (胡俊光)
2025-07-07 1:31 ` [PATCH v2 13/13] media: uapi: mediatek: document ISP7x camera system and user controls shangyao lin
2025-07-11 3:12 ` CK Hu (胡俊光)
[not found] ` <20250707013154.4055874-6-shangyao.lin@mediatek.com>
2025-07-07 5:50 ` [PATCH v2 05/13] media: platform: mediatek: add isp_7x seninf unit Krzysztof Kozlowski
2025-07-10 6:39 ` CK Hu (胡俊光)
2025-07-16 4:06 ` CK Hu (胡俊光)
2025-07-07 5:55 ` [PATCH v2 00/13] Add MediaTek ISP7.x camera system support Krzysztof Kozlowski
[not found] ` <20250707013154.4055874-8-shangyao.lin@mediatek.com>
2025-07-15 4:07 ` [PATCH v2 07/13] MEDIA: PLATFORM: MEDIATEK: ADD ISP_7X CAM-RAW UNIT CK Hu (胡俊光)
[not found] ` <20250707013154.4055874-10-shangyao.lin@mediatek.com>
2025-07-22 2:19 ` [PATCH v2 09/13] media: platform: mediatek: add isp_7x utility CK Hu (胡俊光)
[not found] ` <20250707013154.4055874-9-shangyao.lin@mediatek.com>
2025-07-07 5:58 ` [PATCH v2 08/13] media: platform: mediatek: add isp_7x camsys unit Krzysztof Kozlowski
2025-07-10 5:20 ` CK Hu (胡俊光)
2025-07-10 7:46 ` CK Hu (胡俊光)
2025-07-24 8:36 ` CK Hu (胡俊光)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).