[PATCH v5 00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[PATCH v5 00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[Kyrie Wu]

This series adds support for mt8196 multi-hardwares jpeg enc & dec,
by first adding mt8196 jpegdec and jpegenc compatible to install
kernel driver. Add smmu setting to support smmu and iommu at the
same time.
Secondly refactor buffer and clock setting to support multi-hw jpeg
working.
Lastly, fix some bugs, including resolution change handleing, stop
streaming sw flow and others.

This series has been tested with MT8196 tast test.
Encoding and decoding worked for this chip.

Patches 1-3 Adds jpeg encoder and decoder compatible.
Patches 4 add jpeg smmu sid setting.
Patches 5 fix jpeg hw count setting to support different chips.
Patches 6 refactor jpeg buffer payload setting to handle buffer
size bug while resolution changed.
Patches 7 reconstruct jpeg dst buffer layout.
Patches 8 fix multi-core stop streaming flow
Patches 9 refactor multi-core clk suspend/resume setting
Patches 10 fix decoding buffer number setting timing issue
Patches 11 refactor decoding resolution change operation
Patches 12 fix remove buffer operation

---
This series patches dependent on:
[1]
https://patchwork.linuxtv.org/project/linux-media/patch/20250424090824.5309-1-jianhua.lin@mediatek.com/

Changes compared with v4:
--fix kernel robot build errors for patch 4.
--add reviewer for patch 1 and patch 2.

Changes compared with v3:
--change patch subject of jpeg encoder and decoder compatible.

Changes compared with v2:
--refactor smmu sid setting function interface
--Some modifications for patch v2's review comments.

Changes compared with v1:
--refine jpeg dt-bindings for MT8196
--optimize software code to manage jpeg HW count
--refactor smmu sid setting function interface
--Some modifications for patch v1's review comments.

Kyrie Wu (12):
  media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgdec
    compatible
  media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgenc
    compatible
  media: mediatek: jpeg: add jpeg compatible
  media: mediatek: jpeg: add jpeg smmu sid setting
  media: mediatek: jpeg: fix jpeg hw count setting
  media: mediatek: jpeg: refactor jpeg buffer payload setting
  media: mediatek: jpeg: refactor jpeg dst buffer layout
  media: mediatek: jpeg: fix stop streaming flow for multi-core
  media: mediatek: jpeg: refactor multi-core clk suspend and resume
    setting
  media: mediatek: jpeg: fix decoding buffer number setting timing issue
  media: mediatek: jpeg: refactor decoding resolution change operation
  media: mediatek: jpeg: fix remove buffer operation for multi-core

 .../media/mediatek,mt8195-jpegdec.yaml        |   8 +-
 .../media/mediatek,mt8195-jpegenc.yaml        |   8 +-
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 169 +++++++++++++-----
 .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  21 ++-
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 112 +++++++++++-
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 112 +++++++++++-
 6 files changed, 377 insertions(+), 53 deletions(-)

-- 
2.46.0

[PATCH v5 01/12] media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgdec compatible

[Kyrie Wu]

Compared to the previous generation IC, the MT8196 uses SMMU
instead of IOMMU and supports features such as dynamic voltage
and frequency scaling. Therefore, add "mediatek,mt8196-jpgdec"
compatible to the binding document.

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
---
 .../bindings/media/mediatek,mt8195-jpegdec.yaml           | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml b/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
index e5448c60e3eb..28a9a9bfdbf8 100644
--- a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
@@ -14,7 +14,9 @@ description:
 
 properties:
   compatible:
-    const: mediatek,mt8195-jpgdec
+    enum:
+      - mediatek,mt8195-jpgdec
+      - mediatek,mt8196-jpgdec
 
   power-domains:
     maxItems: 1
@@ -44,7 +46,9 @@ patternProperties:
 
     properties:
       compatible:
-        const: mediatek,mt8195-jpgdec-hw
+        enum:
+          - mediatek,mt8195-jpgdec-hw
+          - mediatek,mt8196-jpgdec-hw
 
       reg:
         maxItems: 1
-- 
2.46.0

[PATCH v5 02/12] media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgenc compatible

[Kyrie Wu]

Compared to the previous generation IC, the MT8196 uses SMMU
instead of IOMMU and supports features such as dynamic voltage
and frequency scaling. Therefore, add "mediatek,mt8196-jpgenc"
compatible to the binding document.

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
---
 .../bindings/media/mediatek,mt8195-jpegenc.yaml           | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml b/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml
index 596186497b68..e2d772ea0fb0 100644
--- a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml
@@ -14,7 +14,9 @@ description:
 
 properties:
   compatible:
-    const: mediatek,mt8195-jpgenc
+    enum:
+      - mediatek,mt8195-jpgenc
+      - mediatek,mt8196-jpgenc
 
   power-domains:
     maxItems: 1
@@ -44,7 +46,9 @@ patternProperties:
 
     properties:
       compatible:
-        const: mediatek,mt8195-jpgenc-hw
+        enum:
+          - mediatek,mt8195-jpgenc-hw
+          - mediatek,mt8196-jpgenc-hw
 
       reg:
         maxItems: 1
-- 
2.46.0

[PATCH v5 03/12] media: mediatek: jpeg: add jpeg compatible

[Kyrie Wu]

Add jpeg dec and enc compatible for mt8196

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 34 +++++++++++++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  |  3 ++
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  |  3 ++
 3 files changed, 40 insertions(+)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 7eb12449b63a..e89ccb8c904c 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1918,6 +1918,19 @@ static struct mtk_jpeg_variant mtk8195_jpegenc_drvdata = {
 	.jpeg_worker = mtk_jpegenc_worker,
 };
 
+static struct mtk_jpeg_variant mtk8196_jpegenc_drvdata = {
+	.formats = mtk_jpeg_enc_formats,
+	.num_formats = MTK_JPEG_ENC_NUM_FORMATS,
+	.qops = &mtk_jpeg_enc_qops,
+	.m2m_ops = &mtk_jpeg_multicore_enc_m2m_ops,
+	.dev_name = "mtk-jpeg-enc",
+	.ioctl_ops = &mtk_jpeg_enc_ioctl_ops,
+	.out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
+	.cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
+	.multi_core = true,
+	.jpeg_worker = mtk_jpegenc_worker,
+};
+
 static const struct mtk_jpeg_variant mtk8195_jpegdec_drvdata = {
 	.formats = mtk_jpeg_dec_formats,
 	.num_formats = MTK_JPEG_DEC_NUM_FORMATS,
@@ -1931,6 +1944,19 @@ static const struct mtk_jpeg_variant mtk8195_jpegdec_drvdata = {
 	.jpeg_worker = mtk_jpegdec_worker,
 };
 
+static const struct mtk_jpeg_variant mtk8196_jpegdec_drvdata = {
+	.formats = mtk_jpeg_dec_formats,
+	.num_formats = MTK_JPEG_DEC_NUM_FORMATS,
+	.qops = &mtk_jpeg_dec_qops,
+	.m2m_ops = &mtk_jpeg_multicore_dec_m2m_ops,
+	.dev_name = "mtk-jpeg-dec",
+	.ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
+	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
+	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
+	.multi_core = true,
+	.jpeg_worker = mtk_jpegdec_worker,
+};
+
 static const struct of_device_id mtk_jpeg_match[] = {
 	{
 		.compatible = "mediatek,mt8173-jpgdec",
@@ -1952,6 +1978,14 @@ static const struct of_device_id mtk_jpeg_match[] = {
 		.compatible = "mediatek,mt8195-jpgdec",
 		.data = &mtk8195_jpegdec_drvdata,
 	},
+	{
+		.compatible = "mediatek,mt8196-jpgenc",
+		.data = &mtk8196_jpegenc_drvdata,
+	},
+	{
+		.compatible = "mediatek,mt8196-jpgdec",
+		.data = &mtk8196_jpegdec_drvdata,
+	},
 	{},
 };
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index e78e1d11093c..20553bf541df 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -45,6 +45,9 @@ static const struct of_device_id mtk_jpegdec_hw_ids[] = {
 	{
 		.compatible = "mediatek,mt8195-jpgdec-hw",
 	},
+	{
+		.compatible = "mediatek,mt8196-jpgdec-hw",
+	},
 	{},
 };
 MODULE_DEVICE_TABLE(of, mtk_jpegdec_hw_ids);
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index 9ab27aee302a..69c95113f205 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -52,6 +52,9 @@ static const struct of_device_id mtk_jpegenc_drv_ids[] = {
 	{
 		.compatible = "mediatek,mt8195-jpgenc-hw",
 	},
+	{
+		.compatible = "mediatek,mt8196-jpgenc-hw",
+	},
 	{},
 };
 MODULE_DEVICE_TABLE(of, mtk_jpegenc_drv_ids);
-- 
2.46.0

[PATCH v5 04/12] media: mediatek: jpeg: add jpeg smmu sid setting

[Kyrie Wu]

Add a configuration to set jpeg dec & enc smmu sid

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 37 +++++++++++++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_core.h    | 15 ++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 23 ++++++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 23 ++++++++++++
 4 files changed, 98 insertions(+)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index e89ccb8c904c..ea890f01fa9e 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -15,6 +15,7 @@
 #include <linux/of_platform.h>
 #include <linux/platform_device.h>
 #include <linux/pm_runtime.h>
+#include <linux/regmap.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
 #include <media/v4l2-event.h>
@@ -1584,6 +1585,20 @@ static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
 	return IRQ_HANDLED;
 }
 
+static void mtk_jpeg_enc_set_smmu_sid(struct mtk_jpegenc_comp_dev *jpeg)
+{
+	struct mtk_jpeg_dev *mjpeg = jpeg->master_dev;
+
+	if (!mjpeg->variant->support_smmu || !jpeg->smmu_regmap)
+		return;
+
+	regmap_update_bits(jpeg->smmu_regmap, JPEG_ENC_SMMU_SID,
+			   JPG_REG_GUSER_ID_MASK <<
+			   JPG_REG_ENC_GUSER_ID_SHIFT,
+			   JPG_REG_GUSER_ID_ENC_SID <<
+			   JPG_REG_ENC_GUSER_ID_SHIFT);
+}
+
 static void mtk_jpegenc_worker(struct work_struct *work)
 {
 	struct mtk_jpegenc_comp_dev *comp_jpeg[MTK_JPEGENC_HW_MAX];
@@ -1655,6 +1670,9 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 	jpeg_dst_buf->frame_num = ctx->total_frame_num;
 	ctx->total_frame_num++;
 	mtk_jpeg_enc_reset(comp_jpeg[hw_id]->reg_base);
+
+	mtk_jpeg_enc_set_smmu_sid(comp_jpeg[hw_id]);
+
 	mtk_jpeg_set_enc_dst(ctx,
 			     comp_jpeg[hw_id]->reg_base,
 			     &dst_buf->vb2_buf);
@@ -1679,6 +1697,20 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 }
 
+static void mtk_jpeg_dec_set_smmu_sid(struct mtk_jpegdec_comp_dev *jpeg)
+{
+	struct mtk_jpeg_dev *mjpeg = jpeg->master_dev;
+
+	if (!mjpeg->variant->support_smmu || !jpeg->smmu_regmap)
+		return;
+
+	regmap_update_bits(jpeg->smmu_regmap, JPEG_DEC_SMMU_SID,
+			   JPG_REG_GUSER_ID_MASK <<
+			   JPG_REG_DEC_GUSER_ID_SHIFT,
+			   JPG_REG_GUSER_ID_DEC_SID <<
+			   JPG_REG_DEC_GUSER_ID_SHIFT);
+}
+
 static void mtk_jpegdec_worker(struct work_struct *work)
 {
 	struct mtk_jpeg_ctx *ctx = container_of(work, struct mtk_jpeg_ctx,
@@ -1771,6 +1803,9 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 	spin_lock_irqsave(&comp_jpeg[hw_id]->hw_lock, flags);
 	ctx->total_frame_num++;
 	mtk_jpeg_dec_reset(comp_jpeg[hw_id]->reg_base);
+
+	mtk_jpeg_dec_set_smmu_sid(comp_jpeg[hw_id]);
+
 	mtk_jpeg_dec_set_config(comp_jpeg[hw_id]->reg_base,
 				jpeg->variant->support_34bit,
 				&jpeg_src_buf->dec_param,
@@ -1929,6 +1964,7 @@ static struct mtk_jpeg_variant mtk8196_jpegenc_drvdata = {
 	.cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
 	.multi_core = true,
 	.jpeg_worker = mtk_jpegenc_worker,
+	.support_smmu = true,
 };
 
 static const struct mtk_jpeg_variant mtk8195_jpegdec_drvdata = {
@@ -1955,6 +1991,7 @@ static const struct mtk_jpeg_variant mtk8196_jpegdec_drvdata = {
 	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
 	.multi_core = true,
 	.jpeg_worker = mtk_jpegdec_worker,
+	.support_smmu = true,
 };
 
 static const struct of_device_id mtk_jpeg_match[] = {
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index 02ed0ed5b736..46899512a5e6 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -11,6 +11,7 @@
 
 #include <linux/clk.h>
 #include <linux/interrupt.h>
+#include <linux/mfd/syscon.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-fh.h>
@@ -34,6 +35,14 @@
 
 #define MTK_JPEG_MAX_EXIF_SIZE	(64 * 1024)
 
+#define JPEG_DEC_SMMU_SID				0
+#define JPEG_ENC_SMMU_SID				0
+#define JPG_REG_GUSER_ID_MASK			0x7
+#define JPG_REG_GUSER_ID_DEC_SID		0x4
+#define JPG_REG_GUSER_ID_ENC_SID		0x5
+#define JPG_REG_DEC_GUSER_ID_SHIFT		8
+#define JPG_REG_ENC_GUSER_ID_SHIFT		4
+
 #define MTK_JPEG_ADDR_MASK GENMASK(1, 0)
 
 /**
@@ -65,6 +74,7 @@ enum mtk_jpeg_ctx_state {
  * @multi_core:		mark jpeg hw is multi_core or not
  * @jpeg_worker:		jpeg dec or enc worker
  * @support_34bit:	flag to check support for 34-bit DMA address
+ * @support_smmu:	flag to check if support smmu
  */
 struct mtk_jpeg_variant {
 	struct clk_bulk_data *clks;
@@ -82,6 +92,7 @@ struct mtk_jpeg_variant {
 	bool multi_core;
 	void (*jpeg_worker)(struct work_struct *work);
 	bool support_34bit;
+	bool support_smmu;
 };
 
 struct mtk_jpeg_src_buf {
@@ -150,6 +161,7 @@ struct mtk_jpegdec_clk {
  * @hw_param:		jpeg encode hw parameters
  * @hw_state:		record hw state
  * @hw_lock:		spinlock protecting the hw device resource
+ * @smmu_regmap:	SMMU registers mapping
  */
 struct mtk_jpegenc_comp_dev {
 	struct device *dev;
@@ -163,6 +175,7 @@ struct mtk_jpegenc_comp_dev {
 	enum mtk_jpeg_hw_state hw_state;
 	/* spinlock protecting the hw device resource */
 	spinlock_t hw_lock;
+	struct regmap *smmu_regmap;
 };
 
 /**
@@ -177,6 +190,7 @@ struct mtk_jpegenc_comp_dev {
  * @hw_param:			jpeg decode hw parameters
  * @hw_state:			record hw state
  * @hw_lock:			spinlock protecting hw
+ * @smmu_regmap:		SMMU registers mapping
  */
 struct mtk_jpegdec_comp_dev {
 	struct device *dev;
@@ -190,6 +204,7 @@ struct mtk_jpegdec_comp_dev {
 	enum mtk_jpeg_hw_state hw_state;
 	/* spinlock protecting the hw device resource */
 	spinlock_t hw_lock;
+	struct regmap *smmu_regmap;
 };
 
 /**
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index 20553bf541df..2538c49f8eda 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -616,6 +616,25 @@ static int mtk_jpegdec_hw_init_irq(struct mtk_jpegdec_comp_dev *dev)
 	return 0;
 }
 
+static int mtk_jpegdec_smmu_init(struct mtk_jpegdec_comp_dev *dev)
+{
+	struct mtk_jpeg_dev *master_dev = dev->master_dev;
+
+	if (!master_dev->variant->support_smmu)
+		return 0;
+
+	dev->smmu_regmap =
+		syscon_regmap_lookup_by_phandle(dev->plat_dev->dev.of_node,
+						"mediatek,smmu-config");
+	if (IS_ERR(dev->smmu_regmap)) {
+		return dev_err_probe(dev->dev, PTR_ERR(dev->smmu_regmap),
+				     "mmap smmu_base failed(%ld)\n",
+				     PTR_ERR(dev->smmu_regmap));
+	}
+
+	return 0;
+}
+
 static int mtk_jpegdec_hw_probe(struct platform_device *pdev)
 {
 	struct mtk_jpegdec_clk *jpegdec_clk;
@@ -668,6 +687,10 @@ static int mtk_jpegdec_hw_probe(struct platform_device *pdev)
 	master_dev->reg_decbase[i] = dev->reg_base;
 	dev->master_dev = master_dev;
 
+	ret = mtk_jpegdec_smmu_init(dev);
+	if (ret)
+		return ret;
+
 	platform_set_drvdata(pdev, dev);
 	pm_runtime_enable(&pdev->dev);
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index 69c95113f205..fc7b7d404253 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -340,6 +340,25 @@ static int mtk_jpegenc_hw_init_irq(struct mtk_jpegenc_comp_dev *dev)
 	return 0;
 }
 
+static int mtk_jpegenc_smmu_init(struct mtk_jpegenc_comp_dev *dev)
+{
+	struct mtk_jpeg_dev *master_dev = dev->master_dev;
+
+	if (!master_dev->variant->support_smmu)
+		return 0;
+
+	dev->smmu_regmap =
+		syscon_regmap_lookup_by_phandle(dev->plat_dev->dev.of_node,
+						"mediatek,smmu-config");
+	if (IS_ERR(dev->smmu_regmap)) {
+		return dev_err_probe(dev->dev, PTR_ERR(dev->smmu_regmap),
+				     "mmap smmu_base failed(%ld)\n",
+				     PTR_ERR(dev->smmu_regmap));
+	}
+
+	return 0;
+}
+
 static int mtk_jpegenc_hw_probe(struct platform_device *pdev)
 {
 	struct mtk_jpegenc_clk *jpegenc_clk;
@@ -390,6 +409,10 @@ static int mtk_jpegenc_hw_probe(struct platform_device *pdev)
 	master_dev->reg_encbase[i] = dev->reg_base;
 	dev->master_dev = master_dev;
 
+	ret = mtk_jpegenc_smmu_init(dev);
+	if (ret)
+		return ret;
+
 	platform_set_drvdata(pdev, dev);
 	pm_runtime_enable(&pdev->dev);
 
-- 
2.46.0

[PATCH v5 05/12] media: mediatek: jpeg: fix jpeg hw count setting

[Kyrie Wu]

1. different IC has different hw core;
2. use a parameter to set jpeg hw count.

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 8 ++++----
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h   | 2 ++
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 1 +
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 1 +
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index ea890f01fa9e..0074d1edb534 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1469,7 +1469,7 @@ static int mtk_jpegenc_get_hw(struct mtk_jpeg_ctx *ctx)
 	int i;
 
 	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	for (i = 0; i < MTK_JPEGENC_HW_MAX; i++) {
+	for (i = 0; i < jpeg->max_hw_count; i++) {
 		comp_jpeg = jpeg->enc_hw_dev[i];
 		if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
 			hw_id = i;
@@ -1516,7 +1516,7 @@ static int mtk_jpegdec_get_hw(struct mtk_jpeg_ctx *ctx)
 	int i;
 
 	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++) {
+	for (i = 0; i < jpeg->max_hw_count; i++) {
 		comp_jpeg = jpeg->dec_hw_dev[i];
 		if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
 			hw_id = i;
@@ -1613,7 +1613,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 		jpeg_work);
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	for (i = 0; i < MTK_JPEGENC_HW_MAX; i++)
+	for (i = 0; i < jpeg->max_hw_count; i++)
 		comp_jpeg[i] = jpeg->enc_hw_dev[i];
 	i = 0;
 
@@ -1725,7 +1725,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 	struct mtk_jpeg_fb fb;
 	unsigned long flags;
 
-	for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++)
+	for (i = 0; i < jpeg->max_hw_count; i++)
 		comp_jpeg[i] = jpeg->dec_hw_dev[i];
 	i = 0;
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index 46899512a5e6..655dc9c3280c 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -227,6 +227,7 @@ struct mtk_jpegdec_comp_dev {
  * @reg_decbase:	jpg decode register base addr
  * @dec_hw_dev:	jpg decode hardware device
  * @hw_index:		jpg hw index
+ * @max_hw_count:	jpeg hw-core count
  */
 struct mtk_jpeg_dev {
 	struct mutex		lock;
@@ -249,6 +250,7 @@ struct mtk_jpeg_dev {
 	void __iomem *reg_decbase[MTK_JPEGDEC_HW_MAX];
 	struct mtk_jpegdec_comp_dev *dec_hw_dev[MTK_JPEGDEC_HW_MAX];
 	atomic_t hw_index;
+	u32 max_hw_count;
 };
 
 /**
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index 2538c49f8eda..2200f3b628dc 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -686,6 +686,7 @@ static int mtk_jpegdec_hw_probe(struct platform_device *pdev)
 	master_dev->dec_hw_dev[i] = dev;
 	master_dev->reg_decbase[i] = dev->reg_base;
 	dev->master_dev = master_dev;
+	master_dev->max_hw_count++;
 
 	ret = mtk_jpegdec_smmu_init(dev);
 	if (ret)
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index fc7b7d404253..4c264c14ad83 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -408,6 +408,7 @@ static int mtk_jpegenc_hw_probe(struct platform_device *pdev)
 	master_dev->enc_hw_dev[i] = dev;
 	master_dev->reg_encbase[i] = dev->reg_base;
 	dev->master_dev = master_dev;
+	master_dev->max_hw_count++;
 
 	ret = mtk_jpegenc_smmu_init(dev);
 	if (ret)
-- 
2.46.0

[PATCH v5 06/12] media: mediatek: jpeg: refactor jpeg buffer payload setting

[Kyrie Wu]

1. for multi-core jpegdec:
   core0: |<-------- decoding buffer0 and resolution changed to smaller
   core1:  |<-------- decoding buffer1
   core0:   |<- handling resolution changing
   core0:    |<- vb2_set_plane_payload
2. the payload size is changed on the step of set format. Because core1
is running and streaming has not been stopped, the format cannot be
set again, resulting in no change in the payload size.
3. at this time, the payload size is bigger than buffer length,
it will print a warnning call trace
4. set payload size must less than buffer length

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c     | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 0074d1edb534..52d59bb5c9ad 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -720,10 +720,22 @@ static int mtk_jpeg_buf_prepare(struct vb2_buffer *vb)
 		plane_fmt = q_data->pix_mp.plane_fmt[i];
 		if (ctx->enable_exif &&
 		    q_data->fmt->fourcc == V4L2_PIX_FMT_JPEG)
-			vb2_set_plane_payload(vb, i, plane_fmt.sizeimage +
-					      MTK_JPEG_MAX_EXIF_SIZE);
+			if (vb->planes[i].length > (plane_fmt.sizeimage +
+			    MTK_JPEG_MAX_EXIF_SIZE))
+				vb2_set_plane_payload(vb, i,
+						      plane_fmt.sizeimage +
+						      MTK_JPEG_MAX_EXIF_SIZE);
+			else
+				vb2_set_plane_payload(vb, i,
+						      vb->planes[i].length);
+
 		else
-			vb2_set_plane_payload(vb, i,  plane_fmt.sizeimage);
+			if (vb->planes[i].length > plane_fmt.sizeimage)
+				vb2_set_plane_payload(vb, i,
+						      plane_fmt.sizeimage);
+			else
+				vb2_set_plane_payload(vb, i,
+						      vb->planes[i].length);
 	}
 
 	return 0;
-- 
2.46.0

[PATCH v5 07/12] media: mediatek: jpeg: refactor jpeg dst buffer layout

[Kyrie Wu]

1. change dst buffer size to same as struct mtk_jpeg_src_buf
to make sure all params of mtk_jpeg_src_buf could get a memory.
2. For memory alloc operation:
the v4l2 framework malloc a memory, the base addr is vb2_buffer and
the size is sizeof(struct mtk_jpeg_src_buf), mtk_jpeg_src_buf could get
itself addr by container_of like that:
vb2_buffer -> vb2_v4l2_buffer -> mtk_jpeg_src_buf.
vb2_v4l2_buffer must keep on the top of mtk_jpeg_src_buf.

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 2 +-
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 52d59bb5c9ad..7e3509be6f69 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1103,7 +1103,7 @@ static int mtk_jpeg_queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
 	dst_vq->drv_priv = ctx;
-	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
 	dst_vq->ops = jpeg->variant->qops;
 	dst_vq->mem_ops = &vb2_dma_contig_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index 655dc9c3280c..186cd1862028 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -96,10 +96,10 @@ struct mtk_jpeg_variant {
 };
 
 struct mtk_jpeg_src_buf {
-	u32 frame_num;
 	struct vb2_v4l2_buffer b;
 	struct list_head list;
 	u32 bs_size;
+	u32 frame_num;
 	struct mtk_jpeg_dec_param dec_param;
 
 	struct mtk_jpeg_ctx *curr_ctx;
-- 
2.46.0

[PATCH v5 08/12] media: mediatek: jpeg: fix stop streaming flow for multi-core

[Kyrie Wu]

1. For multi-core jpegdec, the all hws may run at the same time,
if one hw decoded firstly, the function of mtk_jpeg_dec_stop_streaming
would be called, but others input buffers are decoding, this will
cause some running buffers to be buffer done, causing errors;
2. add a parameter to calculate the decoding buffer counts, it
wil decrease to 0 until the all buffers decoded and the
mtk_jpeg_dec_stop_streaming could continue to be executed.

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 .../media/platform/mediatek/jpeg/mtk_jpeg_core.c | 16 ++++++++++++++++
 .../media/platform/mediatek/jpeg/mtk_jpeg_core.h |  2 ++
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c     |  9 +++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c     |  9 +++++++++
 4 files changed, 36 insertions(+)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 7e3509be6f69..1d3df1230191 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -861,8 +861,12 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct vb2_v4l2_buffer *vb;
 
+	if (jpeg->variant->multi_core)
+		wait_event(jpeg->hw_wq, (atomic_read(&ctx->buf_list_cnt) == 0));
+
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
 }
@@ -870,6 +874,7 @@ static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct vb2_v4l2_buffer *vb;
 
 	/*
@@ -877,6 +882,9 @@ static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 	 * Before STREAMOFF, we still have to return the old resolution and
 	 * subsampling. Update capture queue when the stream is off.
 	 */
+	if (jpeg->variant->multi_core)
+		wait_event(jpeg->hw_wq, (atomic_read(&ctx->buf_list_cnt) == 0));
+
 	if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
 	    V4L2_TYPE_IS_CAPTURE(q->type)) {
 		struct mtk_jpeg_src_buf *src_buf;
@@ -1186,6 +1194,7 @@ static int mtk_jpeg_open(struct file *file)
 	v4l2_fh_init(&ctx->fh, vfd);
 	file->private_data = &ctx->fh;
 	v4l2_fh_add(&ctx->fh);
+	atomic_set(&ctx->buf_list_cnt, 0);
 
 	ctx->jpeg = jpeg;
 	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
@@ -1568,6 +1577,11 @@ static int mtk_jpegdec_set_hw_param(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static void jpeg_buf_queue_inc(struct mtk_jpeg_ctx *ctx)
+{
+	atomic_inc(&ctx->buf_list_cnt);
+}
+
 static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
 {
 	struct mtk_jpeg_ctx *ctx;
@@ -1693,6 +1707,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 			     &src_buf->vb2_buf);
 	mtk_jpeg_set_enc_params(ctx, comp_jpeg[hw_id]->reg_base);
 	mtk_jpeg_enc_start(comp_jpeg[hw_id]->reg_base);
+	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
 
@@ -1825,6 +1840,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 				&bs,
 				&fb);
 	mtk_jpeg_dec_start(comp_jpeg[hw_id]->reg_base);
+	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index 186cd1862028..6e8304680393 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -303,6 +303,7 @@ struct mtk_jpeg_q_data {
  * @dst_done_queue:		encoded frame buffer queue
  * @done_queue_lock:		encoded frame operation spinlock
  * @last_done_frame_num:	the last encoded frame number
+ * @buf_list_cnt:		the frame buffer count own by jpeg driver
  */
 struct mtk_jpeg_ctx {
 	struct mtk_jpeg_dev		*jpeg;
@@ -321,6 +322,7 @@ struct mtk_jpeg_ctx {
 	/* spinlock protecting the encode done buffer */
 	spinlock_t done_queue_lock;
 	u32 last_done_frame_num;
+	atomic_t buf_list_cnt;
 };
 
 #endif /* _MTK_JPEG_CORE_H */
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index 2200f3b628dc..2e6da8617484 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -522,6 +522,11 @@ static void mtk_jpegdec_put_buf(struct mtk_jpegdec_comp_dev *jpeg)
 	spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
 }
 
+static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
+{
+	atomic_dec(&ctx->buf_list_cnt);
+}
+
 static void mtk_jpegdec_timeout_work(struct work_struct *work)
 {
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
@@ -530,9 +535,11 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
 			     job_timeout_work.work);
 	struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_ctx *ctx;
 
 	src_buf = cjpeg->hw_param.src_buffer;
 	dst_buf = cjpeg->hw_param.dst_buffer;
+	ctx = cjpeg->hw_param.curr_ctx;
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_dec_reset(cjpeg->reg_base);
@@ -543,6 +550,7 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(cjpeg);
+	jpeg_buf_queue_dec(ctx);
 }
 
 static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
@@ -583,6 +591,7 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	buf_state = VB2_BUF_STATE_DONE;
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(jpeg);
+	jpeg_buf_queue_dec(ctx);
 	pm_runtime_put(ctx->jpeg->dev);
 	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index 4c264c14ad83..ff73393a2417 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -251,6 +251,11 @@ static void mtk_jpegenc_put_buf(struct mtk_jpegenc_comp_dev *jpeg)
 	spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
 }
 
+static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
+{
+	atomic_dec(&ctx->buf_list_cnt);
+}
+
 static void mtk_jpegenc_timeout_work(struct work_struct *work)
 {
 	struct delayed_work *dly_work = to_delayed_work(work);
@@ -261,9 +266,11 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
 	struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_ctx *ctx;
 
 	src_buf = cjpeg->hw_param.src_buffer;
 	dst_buf = cjpeg->hw_param.dst_buffer;
+	ctx = cjpeg->hw_param.curr_ctx;
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_enc_reset(cjpeg->reg_base);
@@ -274,6 +281,7 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(cjpeg);
+	jpeg_buf_queue_dec(ctx);
 }
 
 static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
@@ -307,6 +315,7 @@ static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
 	buf_state = VB2_BUF_STATE_DONE;
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(jpeg);
+	jpeg_buf_queue_dec(ctx);
 	pm_runtime_put(ctx->jpeg->dev);
 	clk_disable_unprepare(jpeg->venc_clk.clks->clk);
 
-- 
2.46.0

[PATCH v5 09/12] media: mediatek: jpeg: refactor multi-core clk suspend and resume setting

[Kyrie Wu]

refactor jpeg clk suspend and resume setting for multi-core

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++----
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 75 ++++++++++++++++++-
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 75 ++++++++++++++++++-
 3 files changed, 151 insertions(+), 27 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 1d3df1230191..c1d2de92f125 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1126,6 +1126,9 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 {
 	int ret;
 
+	if (jpeg->variant->multi_core)
+		return;
+
 	ret = clk_bulk_prepare_enable(jpeg->variant->num_clks,
 				      jpeg->variant->clks);
 	if (ret)
@@ -1134,6 +1137,9 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
+	if (jpeg->variant->multi_core)
+		return;
+
 	clk_bulk_disable_unprepare(jpeg->variant->num_clks,
 				   jpeg->variant->clks);
 }
@@ -1677,13 +1683,6 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 		goto enc_end;
 	}
 
-	ret = clk_prepare_enable(comp_jpeg[hw_id]->venc_clk.clks->clk);
-	if (ret) {
-		dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
-			__func__, __LINE__);
-		goto enc_end;
-	}
-
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 
@@ -1798,20 +1797,13 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 	jpeg_dst_buf->frame_num = ctx->total_frame_num;
 
 	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
-	ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
+	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
 	if (ret < 0) {
 		dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail !!!\n",
 			__func__, __LINE__);
 		goto dec_end;
 	}
 
-	ret = clk_prepare_enable(comp_jpeg[hw_id]->jdec_clk.clks->clk);
-	if (ret) {
-		dev_err(jpeg->dev, "%s : %d, jpegdec clk_prepare_enable fail\n",
-			__func__, __LINE__);
-		goto clk_end;
-	}
-
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 
@@ -1821,7 +1813,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 				 &dst_buf->vb2_buf, &fb)) {
 		dev_err(jpeg->dev, "%s : %d, mtk_jpeg_set_dec_dst fail\n",
 			__func__, __LINE__);
-		goto setdst_end;
+		goto set_dst_fail;
 	}
 
 	schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
@@ -1846,9 +1838,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 
 	return;
 
-setdst_end:
-	clk_disable_unprepare(comp_jpeg[hw_id]->jdec_clk.clks->clk);
-clk_end:
+set_dst_fail:
 	pm_runtime_put(comp_jpeg[hw_id]->dev);
 dec_end:
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index 2e6da8617484..db2afc5151ad 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -543,14 +543,13 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_dec_reset(cjpeg->reg_base);
-	clk_disable_unprepare(cjpeg->jdec_clk.clks->clk);
-	pm_runtime_put(cjpeg->dev);
 	cjpeg->hw_state = MTK_JPEG_HW_IDLE;
 	atomic_inc(&master_jpeg->hw_rdy);
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(cjpeg);
 	jpeg_buf_queue_dec(ctx);
+	pm_runtime_put(cjpeg->dev);
 }
 
 static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
@@ -592,12 +591,11 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(jpeg);
 	jpeg_buf_queue_dec(ctx);
-	pm_runtime_put(ctx->jpeg->dev);
-	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
 
 	jpeg->hw_state = MTK_JPEG_HW_IDLE;
 	wake_up(&master_jpeg->hw_wq);
 	atomic_inc(&master_jpeg->hw_rdy);
+	pm_runtime_put(jpeg->dev);
 
 	return IRQ_HANDLED;
 }
@@ -703,15 +701,84 @@ static int mtk_jpegdec_hw_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, dev);
 	pm_runtime_enable(&pdev->dev);
+	ret = devm_clk_bulk_get(dev->dev,
+				jpegdec_clk->clk_num,
+				jpegdec_clk->clks);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to init clk\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static void mtk_jpeg_clk_on(struct mtk_jpegdec_comp_dev *jpeg)
+{
+	int ret;
+
+	ret = clk_bulk_prepare_enable(jpeg->jdec_clk.clk_num,
+				      jpeg->jdec_clk.clks);
+	if (ret)
+		dev_err(jpeg->dev, "%s : %d, jpegdec clk_prepare_enable fail\n",
+			__func__, __LINE__);
+}
+
+static void mtk_jpeg_clk_off(struct mtk_jpegdec_comp_dev *jpeg)
+{
+	clk_bulk_disable_unprepare(jpeg->jdec_clk.clk_num,
+				   jpeg->jdec_clk.clks);
+}
+
+static __maybe_unused int mtk_jpegdec_pm_suspend(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
 }
 
+static __maybe_unused int mtk_jpegdec_pm_resume(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_on(jpeg);
+
+	return 0;
+}
+
+static __maybe_unused int mtk_jpegdec_suspend(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
+	return pm_runtime_force_suspend(dev);
+}
+
+static __maybe_unused int mtk_jpegdec_resume(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+	int ret;
+
+	ret = pm_runtime_force_resume(dev);
+	if (ret < 0)
+		return ret;
+
+	v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
+	return ret;
+}
+
+static const struct dev_pm_ops mtk_jpegdec_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegdec_suspend, mtk_jpegdec_resume)
+	SET_RUNTIME_PM_OPS(mtk_jpegdec_pm_suspend, mtk_jpegdec_pm_resume, NULL)
+};
+
 static struct platform_driver mtk_jpegdec_hw_driver = {
 	.probe = mtk_jpegdec_hw_probe,
 	.driver = {
 		.name = "mtk-jpegdec-hw",
 		.of_match_table = mtk_jpegdec_hw_ids,
+		.pm             = &mtk_jpegdec_pm_ops,
 	},
 };
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index ff73393a2417..27da2a9922a6 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -274,14 +274,13 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_enc_reset(cjpeg->reg_base);
-	clk_disable_unprepare(cjpeg->venc_clk.clks->clk);
-	pm_runtime_put(cjpeg->dev);
 	cjpeg->hw_state = MTK_JPEG_HW_IDLE;
 	atomic_inc(&master_jpeg->hw_rdy);
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(cjpeg);
 	jpeg_buf_queue_dec(ctx);
+	pm_runtime_put(cjpeg->dev);
 }
 
 static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
@@ -316,12 +315,11 @@ static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(jpeg);
 	jpeg_buf_queue_dec(ctx);
-	pm_runtime_put(ctx->jpeg->dev);
-	clk_disable_unprepare(jpeg->venc_clk.clks->clk);
 
 	jpeg->hw_state = MTK_JPEG_HW_IDLE;
 	wake_up(&master_jpeg->hw_wq);
 	atomic_inc(&master_jpeg->hw_rdy);
+	pm_runtime_put(jpeg->dev);
 
 	return IRQ_HANDLED;
 }
@@ -425,15 +423,84 @@ static int mtk_jpegenc_hw_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, dev);
 	pm_runtime_enable(&pdev->dev);
+	ret = devm_clk_bulk_get(dev->dev,
+				jpegenc_clk->clk_num,
+				jpegenc_clk->clks);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to init clk\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static void mtk_jpeg_clk_on(struct mtk_jpegenc_comp_dev *jpeg)
+{
+	int ret;
+
+	ret = clk_bulk_prepare_enable(jpeg->venc_clk.clk_num,
+				      jpeg->venc_clk.clks);
+	if (ret)
+		dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
+			__func__, __LINE__);
+}
+
+static void mtk_jpeg_clk_off(struct mtk_jpegenc_comp_dev *jpeg)
+{
+	clk_bulk_disable_unprepare(jpeg->venc_clk.clk_num,
+				   jpeg->venc_clk.clks);
+}
+
+static __maybe_unused int mtk_jpegenc_pm_suspend(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
 }
 
+static __maybe_unused int mtk_jpegenc_pm_resume(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_on(jpeg);
+
+	return 0;
+}
+
+static __maybe_unused int mtk_jpegenc_suspend(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
+	return pm_runtime_force_suspend(dev);
+}
+
+static __maybe_unused int mtk_jpegenc_resume(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+	int ret;
+
+	ret = pm_runtime_force_resume(dev);
+	if (ret < 0)
+		return ret;
+
+	v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
+	return ret;
+}
+
+static const struct dev_pm_ops mtk_jpegenc_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegenc_suspend, mtk_jpegenc_resume)
+	SET_RUNTIME_PM_OPS(mtk_jpegenc_pm_suspend, mtk_jpegenc_pm_resume, NULL)
+};
+
 static struct platform_driver mtk_jpegenc_hw_driver = {
 	.probe = mtk_jpegenc_hw_probe,
 	.driver = {
 		.name = "mtk-jpegenc-hw",
 		.of_match_table = mtk_jpegenc_drv_ids,
+		.pm = &mtk_jpegenc_pm_ops,
 	},
 };
 
-- 
2.46.0

[PATCH v5 10/12] media: mediatek: jpeg: fix decoding buffer number setting timing issue

[Kyrie Wu]

move decoding buffer increase code into spinlock
protecting aera for multi-core

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 9 +++------
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 1 +
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 1 +
 3 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index c1d2de92f125..795916206c7c 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1782,7 +1782,6 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
-	jpeg_dst_buf = mtk_jpeg_vb2_to_srcbuf(&dst_buf->vb2_buf);
 
 	if (mtk_jpeg_check_resolution_change(ctx,
 					     &jpeg_src_buf->dec_param)) {
@@ -1791,11 +1790,6 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 		goto getbuf_fail;
 	}
 
-	jpeg_src_buf->curr_ctx = ctx;
-	jpeg_src_buf->frame_num = ctx->total_frame_num;
-	jpeg_dst_buf->curr_ctx = ctx;
-	jpeg_dst_buf->frame_num = ctx->total_frame_num;
-
 	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
 	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
 	if (ret < 0) {
@@ -1820,6 +1814,9 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 			      msecs_to_jiffies(MTK_JPEG_HW_TIMEOUT_MSEC));
 
 	spin_lock_irqsave(&comp_jpeg[hw_id]->hw_lock, flags);
+	jpeg_dst_buf = mtk_jpeg_vb2_to_srcbuf(&dst_buf->vb2_buf);
+	jpeg_dst_buf->curr_ctx = ctx;
+	jpeg_dst_buf->frame_num = ctx->total_frame_num;
 	ctx->total_frame_num++;
 	mtk_jpeg_dec_reset(comp_jpeg[hw_id]->reg_base);
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index db2afc5151ad..da753a636eaa 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -516,6 +516,7 @@ static void mtk_jpegdec_put_buf(struct mtk_jpegdec_comp_dev *jpeg)
 				v4l2_m2m_buf_done(&tmp_dst_done_buf->b,
 						  VB2_BUF_STATE_DONE);
 				ctx->last_done_frame_num++;
+				break;
 			}
 		}
 	}
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index 27da2a9922a6..5e8a5cb4850e 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -245,6 +245,7 @@ static void mtk_jpegenc_put_buf(struct mtk_jpegenc_comp_dev *jpeg)
 				v4l2_m2m_buf_done(&tmp_dst_done_buf->b,
 						  VB2_BUF_STATE_DONE);
 				ctx->last_done_frame_num++;
+				break;
 			}
 		}
 	}
-- 
2.46.0

[PATCH v5 11/12] media: mediatek: jpeg: refactor decoding resolution change operation

[Kyrie Wu]

For multi-core:
1.if there are only two src buffers, hw0 and hw1 are decoding
at the same time. When resolution change occurred on one hw,
the src buffer cannot be got in the stop streaming function,
which will cause a crash.
2.When a resolution change occurs, immediately set the new
resolution parameter to save the new resolution;
3.After a resolution change occurred, decoding should not continue,
needs to wait until new buffers are ready and the state machine
changed.

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 795916206c7c..eba9f58b9198 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -891,7 +891,8 @@ static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 
 		vb = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 		src_buf = mtk_jpeg_vb2_to_srcbuf(&vb->vb2_buf);
-		mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
+		if (!IS_ERR_OR_NULL(src_buf))
+			mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
 		ctx->state = MTK_JPEG_RUNNING;
 	} else if (V4L2_TYPE_IS_OUTPUT(q->type)) {
 		ctx->state = MTK_JPEG_INIT;
@@ -1785,11 +1786,15 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 
 	if (mtk_jpeg_check_resolution_change(ctx,
 					     &jpeg_src_buf->dec_param)) {
-		mtk_jpeg_queue_src_chg_event(ctx);
+		mtk_jpeg_set_queue_data(ctx, &jpeg_src_buf->dec_param);
 		ctx->state = MTK_JPEG_SOURCE_CHANGE;
+		mtk_jpeg_queue_src_chg_event(ctx);
 		goto getbuf_fail;
 	}
 
+	if (ctx->state == MTK_JPEG_SOURCE_CHANGE)
+		goto getbuf_fail;
+
 	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
 	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
 	if (ret < 0) {
-- 
2.46.0

[PATCH v5 12/12] media: mediatek: jpeg: fix remove buffer operation for multi-core

[Kyrie Wu]

move remove buffer code to spinlock protect area for multi-core

Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index eba9f58b9198..34fa03bcfdd3 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1684,9 +1684,6 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 		goto enc_end;
 	}
 
-	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
-	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
-
 	schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
 			      msecs_to_jiffies(MTK_JPEG_HW_TIMEOUT_MSEC));
 
@@ -1707,6 +1704,8 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 			     &src_buf->vb2_buf);
 	mtk_jpeg_set_enc_params(ctx, comp_jpeg[hw_id]->reg_base);
 	mtk_jpeg_enc_start(comp_jpeg[hw_id]->reg_base);
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
@@ -1803,9 +1802,6 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 		goto dec_end;
 	}
 
-	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
-	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
-
 	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
 	if (mtk_jpeg_set_dec_dst(ctx,
 				 &jpeg_src_buf->dec_param,
@@ -1833,6 +1829,8 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 				jpeg_src_buf->bs_size,
 				&bs,
 				&fb);
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	mtk_jpeg_dec_start(comp_jpeg[hw_id]->reg_base);
 	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
-- 
2.46.0

Re: [PATCH v5 06/12] media: mediatek: jpeg: refactor jpeg buffer payload setting

[Nicolas Dufresne]

Hi Kyrie,

Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> 1. for multi-core jpegdec:
>    core0: |<-------- decoding buffer0 and resolution changed to smaller
>    core1:  |<-------- decoding buffer1
>    core0:   |<- handling resolution changing
>    core0:    |<- vb2_set_plane_payload
> 2. the payload size is changed on the step of set format. Because core1
> is running and streaming has not been stopped, the format cannot be
> set again, resulting in no change in the payload size.
> 3. at this time, the payload size is bigger than buffer length,
> it will print a warnning call trace
> 4. set payload size must less than buffer length

You'll have to rework the text in this commit message, I must admit I don't
understand from this text what exactly the problem is, make it really hard to
review your solution.

> 
> Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> ---
>  .../platform/mediatek/jpeg/mtk_jpeg_core.c     | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index 0074d1edb534..52d59bb5c9ad 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -720,10 +720,22 @@ static int mtk_jpeg_buf_prepare(struct vb2_buffer *vb)
>  		plane_fmt = q_data->pix_mp.plane_fmt[i];
>  		if (ctx->enable_exif &&
>  		    q_data->fmt->fourcc == V4L2_PIX_FMT_JPEG)
> -			vb2_set_plane_payload(vb, i, plane_fmt.sizeimage +
> -					      MTK_JPEG_MAX_EXIF_SIZE);
> +			if (vb->planes[i].length > (plane_fmt.sizeimage +
> +			    MTK_JPEG_MAX_EXIF_SIZE))
> +				vb2_set_plane_payload(vb, i,
> +						      plane_fmt.sizeimage +
> +						      MTK_JPEG_MAX_EXIF_SIZE);
> +			else
> +				vb2_set_plane_payload(vb, i,
> +						      vb->planes[i].length);
> +
>  		else
> -			vb2_set_plane_payload(vb, i,  plane_fmt.sizeimage);
> +			if (vb->planes[i].length > plane_fmt.sizeimage)
> +				vb2_set_plane_payload(vb, i,
> +						      plane_fmt.sizeimage);
> +			else
> +				vb2_set_plane_payload(vb, i,
> +						      vb->planes[i].length);

Is this the same as ?

		unsigned long max_size = plane_fmt.sizeimage;

		if (ctx->enable_exif && q_data->fmt->fourcc == V4L2_PIX_FMT_JPEG)
			max_size += MTK_JPEG_MAX_EXIF_SIZE;

		vb2_set_plane_payload(vb, i, MIN(vb->planes[i].length, max_size));

It is unclear to me how this isn't just a workaround though, looking forward
your reworked commit message.

Nicolas

>  	}
>  
>  	return 0;

Re: [PATCH v5 07/12] media: mediatek: jpeg: refactor jpeg dst buffer layout

[Nicolas Dufresne]

Hi,

Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> 1. change dst buffer size to same as struct mtk_jpeg_src_buf
> to make sure all params of mtk_jpeg_src_buf could get a memory.
> 2. For memory alloc operation:
> the v4l2 framework malloc a memory, the base addr is vb2_buffer and
> the size is sizeof(struct mtk_jpeg_src_buf), mtk_jpeg_src_buf could get
> itself addr by container_of like that:
> vb2_buffer -> vb2_v4l2_buffer -> mtk_jpeg_src_buf.
> vb2_v4l2_buffer must keep on the top of mtk_jpeg_src_buf.

The subject imply a refactoring, but the most important change in your
patch is to fix the wrong buf strut size. Can you rework the subject
and message to state what you are fixing please.

Add a Fixes tag, and moved it at the start of the series to show this
isn't a problem you have introduced in previous patch.

> 
> Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> ---
>  drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 2 +-
>  drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index 52d59bb5c9ad..7e3509be6f69 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -1103,7 +1103,7 @@ static int mtk_jpeg_queue_init(void *priv, struct vb2_queue *src_vq,
>  	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>  	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
>  	dst_vq->drv_priv = ctx;
> -	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> +	dst_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
>  	dst_vq->ops = jpeg->variant->qops;
>  	dst_vq->mem_ops = &vb2_dma_contig_memops;
>  	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> index 655dc9c3280c..186cd1862028 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> @@ -96,10 +96,10 @@ struct mtk_jpeg_variant {
>  };
>  
>  struct mtk_jpeg_src_buf {
> -	u32 frame_num;
>  	struct vb2_v4l2_buffer b;
>  	struct list_head list;
>  	u32 bs_size;
> +	u32 frame_num;

This "refactoring" should be split, it does not fix anything.

Nicolas

>  	struct mtk_jpeg_dec_param dec_param;
>  
>  	struct mtk_jpeg_ctx *curr_ctx;

Re: [PATCH v5 08/12] media: mediatek: jpeg: fix stop streaming flow for multi-core

[Nicolas Dufresne]

Hi,

Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> 1. For multi-core jpegdec, the all hws may run at the same time,
> if one hw decoded firstly, the function of mtk_jpeg_dec_stop_streaming
> would be called, but others input buffers are decoding, this will
> cause some running buffers to be buffer done, causing errors;
> 2. add a parameter to calculate the decoding buffer counts, it
> wil decrease to 0 until the all buffers decoded and the
> mtk_jpeg_dec_stop_streaming could continue to be executed.

This one is equally unclear to me. If you run different queues per core,
why does this matter ?

Nicolas

> 
> Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> ---
>  .../media/platform/mediatek/jpeg/mtk_jpeg_core.c | 16 ++++++++++++++++
>  .../media/platform/mediatek/jpeg/mtk_jpeg_core.h |  2 ++
>  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c     |  9 +++++++++
>  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c     |  9 +++++++++
>  4 files changed, 36 insertions(+)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index 7e3509be6f69..1d3df1230191 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -861,8 +861,12 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
>  static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
>  {
>  	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
> +	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
>  	struct vb2_v4l2_buffer *vb;
>  
> +	if (jpeg->variant->multi_core)
> +		wait_event(jpeg->hw_wq, (atomic_read(&ctx->buf_list_cnt) == 0));
> +
>  	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
>  		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
>  }
> @@ -870,6 +874,7 @@ static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
>  static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
>  {
>  	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
> +	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
>  	struct vb2_v4l2_buffer *vb;
>  
>  	/*
> @@ -877,6 +882,9 @@ static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
>  	 * Before STREAMOFF, we still have to return the old resolution and
>  	 * subsampling. Update capture queue when the stream is off.
>  	 */
> +	if (jpeg->variant->multi_core)
> +		wait_event(jpeg->hw_wq, (atomic_read(&ctx->buf_list_cnt) == 0));
> +
>  	if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
>  	    V4L2_TYPE_IS_CAPTURE(q->type)) {
>  		struct mtk_jpeg_src_buf *src_buf;
> @@ -1186,6 +1194,7 @@ static int mtk_jpeg_open(struct file *file)
>  	v4l2_fh_init(&ctx->fh, vfd);
>  	file->private_data = &ctx->fh;
>  	v4l2_fh_add(&ctx->fh);
> +	atomic_set(&ctx->buf_list_cnt, 0);
>  
>  	ctx->jpeg = jpeg;
>  	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
> @@ -1568,6 +1577,11 @@ static int mtk_jpegdec_set_hw_param(struct mtk_jpeg_ctx *ctx,
>  	return 0;
>  }
>  
> +static void jpeg_buf_queue_inc(struct mtk_jpeg_ctx *ctx)
> +{
> +	atomic_inc(&ctx->buf_list_cnt);
> +}
> +
>  static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
>  {
>  	struct mtk_jpeg_ctx *ctx;
> @@ -1693,6 +1707,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
>  			     &src_buf->vb2_buf);
>  	mtk_jpeg_set_enc_params(ctx, comp_jpeg[hw_id]->reg_base);
>  	mtk_jpeg_enc_start(comp_jpeg[hw_id]->reg_base);
> +	jpeg_buf_queue_inc(ctx);
>  	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
>  	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
>  
> @@ -1825,6 +1840,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
>  				&bs,
>  				&fb);
>  	mtk_jpeg_dec_start(comp_jpeg[hw_id]->reg_base);
> +	jpeg_buf_queue_inc(ctx);
>  	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
>  	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
>  
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> index 186cd1862028..6e8304680393 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> @@ -303,6 +303,7 @@ struct mtk_jpeg_q_data {
>   * @dst_done_queue:		encoded frame buffer queue
>   * @done_queue_lock:		encoded frame operation spinlock
>   * @last_done_frame_num:	the last encoded frame number
> + * @buf_list_cnt:		the frame buffer count own by jpeg driver
>   */
>  struct mtk_jpeg_ctx {
>  	struct mtk_jpeg_dev		*jpeg;
> @@ -321,6 +322,7 @@ struct mtk_jpeg_ctx {
>  	/* spinlock protecting the encode done buffer */
>  	spinlock_t done_queue_lock;
>  	u32 last_done_frame_num;
> +	atomic_t buf_list_cnt;
>  };
>  
>  #endif /* _MTK_JPEG_CORE_H */
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> index 2200f3b628dc..2e6da8617484 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> @@ -522,6 +522,11 @@ static void mtk_jpegdec_put_buf(struct mtk_jpegdec_comp_dev *jpeg)
>  	spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
>  }
>  
> +static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
> +{
> +	atomic_dec(&ctx->buf_list_cnt);
> +}
> +
>  static void mtk_jpegdec_timeout_work(struct work_struct *work)
>  {
>  	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
> @@ -530,9 +535,11 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
>  			     job_timeout_work.work);
>  	struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
>  	struct vb2_v4l2_buffer *src_buf, *dst_buf;
> +	struct mtk_jpeg_ctx *ctx;
>  
>  	src_buf = cjpeg->hw_param.src_buffer;
>  	dst_buf = cjpeg->hw_param.dst_buffer;
> +	ctx = cjpeg->hw_param.curr_ctx;
>  	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
>  
>  	mtk_jpeg_dec_reset(cjpeg->reg_base);
> @@ -543,6 +550,7 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
>  	wake_up(&master_jpeg->hw_wq);
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegdec_put_buf(cjpeg);
> +	jpeg_buf_queue_dec(ctx);
>  }
>  
>  static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> @@ -583,6 +591,7 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
>  	buf_state = VB2_BUF_STATE_DONE;
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegdec_put_buf(jpeg);
> +	jpeg_buf_queue_dec(ctx);
>  	pm_runtime_put(ctx->jpeg->dev);
>  	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
>  
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> index 4c264c14ad83..ff73393a2417 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> @@ -251,6 +251,11 @@ static void mtk_jpegenc_put_buf(struct mtk_jpegenc_comp_dev *jpeg)
>  	spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
>  }
>  
> +static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
> +{
> +	atomic_dec(&ctx->buf_list_cnt);
> +}
> +
>  static void mtk_jpegenc_timeout_work(struct work_struct *work)
>  {
>  	struct delayed_work *dly_work = to_delayed_work(work);
> @@ -261,9 +266,11 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
>  	struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
>  	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
>  	struct vb2_v4l2_buffer *src_buf, *dst_buf;
> +	struct mtk_jpeg_ctx *ctx;
>  
>  	src_buf = cjpeg->hw_param.src_buffer;
>  	dst_buf = cjpeg->hw_param.dst_buffer;
> +	ctx = cjpeg->hw_param.curr_ctx;
>  	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
>  
>  	mtk_jpeg_enc_reset(cjpeg->reg_base);
> @@ -274,6 +281,7 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
>  	wake_up(&master_jpeg->hw_wq);
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegenc_put_buf(cjpeg);
> +	jpeg_buf_queue_dec(ctx);
>  }
>  
>  static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> @@ -307,6 +315,7 @@ static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
>  	buf_state = VB2_BUF_STATE_DONE;
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegenc_put_buf(jpeg);
> +	jpeg_buf_queue_dec(ctx);
>  	pm_runtime_put(ctx->jpeg->dev);
>  	clk_disable_unprepare(jpeg->venc_clk.clks->clk);
>  

Re: [PATCH v5 09/12] media: mediatek: jpeg: refactor multi-core clk suspend and resume setting

[Nicolas Dufresne]

Hi,

Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> refactor jpeg clk suspend and resume setting for multi-core

You'll have to write a lot more to support such a large and I
must say slightly convoluted change. Why do you need a special
case for 1 core in the first place ? What about multi-core
design that support from 1 to N cores without using different
code path ?

Nicolas

> 
> Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> ---
>  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++----
>  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 75 ++++++++++++++++++-
>  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 75 ++++++++++++++++++-
>  3 files changed, 151 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index 1d3df1230191..c1d2de92f125 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -1126,6 +1126,9 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
>  {
>  	int ret;
>  
> +	if (jpeg->variant->multi_core)
> +		return;
> +
>  	ret = clk_bulk_prepare_enable(jpeg->variant->num_clks,
>  				      jpeg->variant->clks);
>  	if (ret)
> @@ -1134,6 +1137,9 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
>  
>  static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
>  {
> +	if (jpeg->variant->multi_core)
> +		return;
> +
>  	clk_bulk_disable_unprepare(jpeg->variant->num_clks,
>  				   jpeg->variant->clks);
>  }
> @@ -1677,13 +1683,6 @@ static void mtk_jpegenc_worker(struct work_struct *work)
>  		goto enc_end;
>  	}
>  
> -	ret = clk_prepare_enable(comp_jpeg[hw_id]->venc_clk.clks->clk);
> -	if (ret) {
> -		dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
> -			__func__, __LINE__);
> -		goto enc_end;
> -	}
> -
>  	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
>  	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>  
> @@ -1798,20 +1797,13 @@ static void mtk_jpegdec_worker(struct work_struct *work)
>  	jpeg_dst_buf->frame_num = ctx->total_frame_num;
>  
>  	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
> -	ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
> +	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
>  	if (ret < 0) {
>  		dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail !!!\n",
>  			__func__, __LINE__);
>  		goto dec_end;
>  	}
>  
> -	ret = clk_prepare_enable(comp_jpeg[hw_id]->jdec_clk.clks->clk);
> -	if (ret) {
> -		dev_err(jpeg->dev, "%s : %d, jpegdec clk_prepare_enable fail\n",
> -			__func__, __LINE__);
> -		goto clk_end;
> -	}
> -
>  	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
>  	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>  
> @@ -1821,7 +1813,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
>  				 &dst_buf->vb2_buf, &fb)) {
>  		dev_err(jpeg->dev, "%s : %d, mtk_jpeg_set_dec_dst fail\n",
>  			__func__, __LINE__);
> -		goto setdst_end;
> +		goto set_dst_fail;
>  	}
>  
>  	schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
> @@ -1846,9 +1838,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
>  
>  	return;
>  
> -setdst_end:
> -	clk_disable_unprepare(comp_jpeg[hw_id]->jdec_clk.clks->clk);
> -clk_end:
> +set_dst_fail:
>  	pm_runtime_put(comp_jpeg[hw_id]->dev);
>  dec_end:
>  	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> index 2e6da8617484..db2afc5151ad 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> @@ -543,14 +543,13 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
>  	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
>  
>  	mtk_jpeg_dec_reset(cjpeg->reg_base);
> -	clk_disable_unprepare(cjpeg->jdec_clk.clks->clk);
> -	pm_runtime_put(cjpeg->dev);
>  	cjpeg->hw_state = MTK_JPEG_HW_IDLE;
>  	atomic_inc(&master_jpeg->hw_rdy);
>  	wake_up(&master_jpeg->hw_wq);
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegdec_put_buf(cjpeg);
>  	jpeg_buf_queue_dec(ctx);
> +	pm_runtime_put(cjpeg->dev);
>  }
>  
>  static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> @@ -592,12 +591,11 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegdec_put_buf(jpeg);
>  	jpeg_buf_queue_dec(ctx);
> -	pm_runtime_put(ctx->jpeg->dev);
> -	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
>  
>  	jpeg->hw_state = MTK_JPEG_HW_IDLE;
>  	wake_up(&master_jpeg->hw_wq);
>  	atomic_inc(&master_jpeg->hw_rdy);
> +	pm_runtime_put(jpeg->dev);
>  
>  	return IRQ_HANDLED;
>  }
> @@ -703,15 +701,84 @@ static int mtk_jpegdec_hw_probe(struct platform_device *pdev)
>  
>  	platform_set_drvdata(pdev, dev);
>  	pm_runtime_enable(&pdev->dev);
> +	ret = devm_clk_bulk_get(dev->dev,
> +				jpegdec_clk->clk_num,
> +				jpegdec_clk->clks);
> +	if (ret) {
> +		dev_err(&pdev->dev, "Failed to init clk\n");
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_jpeg_clk_on(struct mtk_jpegdec_comp_dev *jpeg)
> +{
> +	int ret;
> +
> +	ret = clk_bulk_prepare_enable(jpeg->jdec_clk.clk_num,
> +				      jpeg->jdec_clk.clks);
> +	if (ret)
> +		dev_err(jpeg->dev, "%s : %d, jpegdec clk_prepare_enable fail\n",
> +			__func__, __LINE__);
> +}
> +
> +static void mtk_jpeg_clk_off(struct mtk_jpegdec_comp_dev *jpeg)
> +{
> +	clk_bulk_disable_unprepare(jpeg->jdec_clk.clk_num,
> +				   jpeg->jdec_clk.clks);
> +}
> +
> +static __maybe_unused int mtk_jpegdec_pm_suspend(struct device *dev)
> +{
> +	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> +
> +	mtk_jpeg_clk_off(jpeg);
>  
>  	return 0;
>  }
>  
> +static __maybe_unused int mtk_jpegdec_pm_resume(struct device *dev)
> +{
> +	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> +
> +	mtk_jpeg_clk_on(jpeg);
> +
> +	return 0;
> +}
> +
> +static __maybe_unused int mtk_jpegdec_suspend(struct device *dev)
> +{
> +	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> +
> +	v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
> +	return pm_runtime_force_suspend(dev);
> +}
> +
> +static __maybe_unused int mtk_jpegdec_resume(struct device *dev)
> +{
> +	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = pm_runtime_force_resume(dev);
> +	if (ret < 0)
> +		return ret;
> +
> +	v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
> +	return ret;
> +}
> +
> +static const struct dev_pm_ops mtk_jpegdec_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegdec_suspend, mtk_jpegdec_resume)
> +	SET_RUNTIME_PM_OPS(mtk_jpegdec_pm_suspend, mtk_jpegdec_pm_resume, NULL)
> +};
> +
>  static struct platform_driver mtk_jpegdec_hw_driver = {
>  	.probe = mtk_jpegdec_hw_probe,
>  	.driver = {
>  		.name = "mtk-jpegdec-hw",
>  		.of_match_table = mtk_jpegdec_hw_ids,
> +		.pm             = &mtk_jpegdec_pm_ops,
>  	},
>  };
>  
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> index ff73393a2417..27da2a9922a6 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> @@ -274,14 +274,13 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
>  	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
>  
>  	mtk_jpeg_enc_reset(cjpeg->reg_base);
> -	clk_disable_unprepare(cjpeg->venc_clk.clks->clk);
> -	pm_runtime_put(cjpeg->dev);
>  	cjpeg->hw_state = MTK_JPEG_HW_IDLE;
>  	atomic_inc(&master_jpeg->hw_rdy);
>  	wake_up(&master_jpeg->hw_wq);
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegenc_put_buf(cjpeg);
>  	jpeg_buf_queue_dec(ctx);
> +	pm_runtime_put(cjpeg->dev);
>  }
>  
>  static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> @@ -316,12 +315,11 @@ static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
>  	v4l2_m2m_buf_done(src_buf, buf_state);
>  	mtk_jpegenc_put_buf(jpeg);
>  	jpeg_buf_queue_dec(ctx);
> -	pm_runtime_put(ctx->jpeg->dev);
> -	clk_disable_unprepare(jpeg->venc_clk.clks->clk);
>  
>  	jpeg->hw_state = MTK_JPEG_HW_IDLE;
>  	wake_up(&master_jpeg->hw_wq);
>  	atomic_inc(&master_jpeg->hw_rdy);
> +	pm_runtime_put(jpeg->dev);
>  
>  	return IRQ_HANDLED;
>  }
> @@ -425,15 +423,84 @@ static int mtk_jpegenc_hw_probe(struct platform_device *pdev)
>  
>  	platform_set_drvdata(pdev, dev);
>  	pm_runtime_enable(&pdev->dev);
> +	ret = devm_clk_bulk_get(dev->dev,
> +				jpegenc_clk->clk_num,
> +				jpegenc_clk->clks);
> +	if (ret) {
> +		dev_err(&pdev->dev, "Failed to init clk\n");
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +static void mtk_jpeg_clk_on(struct mtk_jpegenc_comp_dev *jpeg)
> +{
> +	int ret;
> +
> +	ret = clk_bulk_prepare_enable(jpeg->venc_clk.clk_num,
> +				      jpeg->venc_clk.clks);
> +	if (ret)
> +		dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
> +			__func__, __LINE__);
> +}
> +
> +static void mtk_jpeg_clk_off(struct mtk_jpegenc_comp_dev *jpeg)
> +{
> +	clk_bulk_disable_unprepare(jpeg->venc_clk.clk_num,
> +				   jpeg->venc_clk.clks);
> +}
> +
> +static __maybe_unused int mtk_jpegenc_pm_suspend(struct device *dev)
> +{
> +	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> +
> +	mtk_jpeg_clk_off(jpeg);
>  
>  	return 0;
>  }
>  
> +static __maybe_unused int mtk_jpegenc_pm_resume(struct device *dev)
> +{
> +	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> +
> +	mtk_jpeg_clk_on(jpeg);
> +
> +	return 0;
> +}
> +
> +static __maybe_unused int mtk_jpegenc_suspend(struct device *dev)
> +{
> +	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> +
> +	v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
> +	return pm_runtime_force_suspend(dev);
> +}
> +
> +static __maybe_unused int mtk_jpegenc_resume(struct device *dev)
> +{
> +	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> +	int ret;
> +
> +	ret = pm_runtime_force_resume(dev);
> +	if (ret < 0)
> +		return ret;
> +
> +	v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
> +	return ret;
> +}
> +
> +static const struct dev_pm_ops mtk_jpegenc_pm_ops = {
> +	SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegenc_suspend, mtk_jpegenc_resume)
> +	SET_RUNTIME_PM_OPS(mtk_jpegenc_pm_suspend, mtk_jpegenc_pm_resume, NULL)
> +};
> +
>  static struct platform_driver mtk_jpegenc_hw_driver = {
>  	.probe = mtk_jpegenc_hw_probe,
>  	.driver = {
>  		.name = "mtk-jpegenc-hw",
>  		.of_match_table = mtk_jpegenc_drv_ids,
> +		.pm = &mtk_jpegenc_pm_ops,
>  	},
>  };
>  

Re: [PATCH v5 00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[Nicolas Dufresne]

Hi,

Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> This series adds support for mt8196 multi-hardwares jpeg enc & dec,
> by first adding mt8196 jpegdec and jpegenc compatible to install
> kernel driver. Add smmu setting to support smmu and iommu at the
> same time.
> Secondly refactor buffer and clock setting to support multi-hw jpeg
> working.
> Lastly, fix some bugs, including resolution change handleing, stop
> streaming sw flow and others.
> 
> This series has been tested with MT8196 tast test.
> Encoding and decoding worked for this chip.
> 
> Patches 1-3 Adds jpeg encoder and decoder compatible.
> Patches 4 add jpeg smmu sid setting.
> Patches 5 fix jpeg hw count setting to support different chips.
> Patches 6 refactor jpeg buffer payload setting to handle buffer
> size bug while resolution changed.
> Patches 7 reconstruct jpeg dst buffer layout.
> Patches 8 fix multi-core stop streaming flow
> Patches 9 refactor multi-core clk suspend/resume setting
> Patches 10 fix decoding buffer number setting timing issue
> Patches 11 refactor decoding resolution change operation
> Patches 12 fix remove buffer operation

Just general comment, you built your patchset up-side-down. Start
with the fixes of things that was already broken, then do your multi-core
support refactoring, and only then add MT8196. Looking for a v6 with
a re-organization of the set.

Nicolas

> 
> ---
> This series patches dependent on:
> [1]
> https://patchwork.linuxtv.org/project/linux-media/patch/20250424090824.5309-1-jianhua.lin@mediatek.com/
> 
> Changes compared with v4:
> --fix kernel robot build errors for patch 4.
> --add reviewer for patch 1 and patch 2.
> 
> Changes compared with v3:
> --change patch subject of jpeg encoder and decoder compatible.
> 
> Changes compared with v2:
> --refactor smmu sid setting function interface
> --Some modifications for patch v2's review comments.
> 
> Changes compared with v1:
> --refine jpeg dt-bindings for MT8196
> --optimize software code to manage jpeg HW count
> --refactor smmu sid setting function interface
> --Some modifications for patch v1's review comments.
> 
> Kyrie Wu (12):
>   media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgdec
>     compatible
>   media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgenc
>     compatible
>   media: mediatek: jpeg: add jpeg compatible
>   media: mediatek: jpeg: add jpeg smmu sid setting
>   media: mediatek: jpeg: fix jpeg hw count setting
>   media: mediatek: jpeg: refactor jpeg buffer payload setting
>   media: mediatek: jpeg: refactor jpeg dst buffer layout
>   media: mediatek: jpeg: fix stop streaming flow for multi-core
>   media: mediatek: jpeg: refactor multi-core clk suspend and resume
>     setting
>   media: mediatek: jpeg: fix decoding buffer number setting timing issue
>   media: mediatek: jpeg: refactor decoding resolution change operation
>   media: mediatek: jpeg: fix remove buffer operation for multi-core
> 
>  .../media/mediatek,mt8195-jpegdec.yaml        |   8 +-
>  .../media/mediatek,mt8195-jpegenc.yaml        |   8 +-
>  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 169 +++++++++++++-----
>  .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  21 ++-
>  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 112 +++++++++++-
>  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 112 +++++++++++-
>  6 files changed, 377 insertions(+), 53 deletions(-)

Re: [PATCH v5 00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[Kyrie Wu (吴晗)]

On Fri, 2025-05-30 at 13:45 -0400, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi,
> 
> Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> > This series adds support for mt8196 multi-hardwares jpeg enc & dec,
> > by first adding mt8196 jpegdec and jpegenc compatible to install
> > kernel driver. Add smmu setting to support smmu and iommu at the
> > same time.
> > Secondly refactor buffer and clock setting to support multi-hw jpeg
> > working.
> > Lastly, fix some bugs, including resolution change handleing, stop
> > streaming sw flow and others.
> > 
> > This series has been tested with MT8196 tast test.
> > Encoding and decoding worked for this chip.
> > 
> > Patches 1-3 Adds jpeg encoder and decoder compatible.
> > Patches 4 add jpeg smmu sid setting.
> > Patches 5 fix jpeg hw count setting to support different chips.
> > Patches 6 refactor jpeg buffer payload setting to handle buffer
> > size bug while resolution changed.
> > Patches 7 reconstruct jpeg dst buffer layout.
> > Patches 8 fix multi-core stop streaming flow
> > Patches 9 refactor multi-core clk suspend/resume setting
> > Patches 10 fix decoding buffer number setting timing issue
> > Patches 11 refactor decoding resolution change operation
> > Patches 12 fix remove buffer operation
> 
> Just general comment, you built your patchset up-side-down. Start
> with the fixes of things that was already broken, then do your multi-
> core
> support refactoring, and only then add MT8196. Looking for a v6 with
> a re-organization of the set.
> 
> Nicolas

Dear Nicolas,

Thanks for your kind comments, I will re-organize my patch set in v6.

Regards,
Kyrie.
> 
> > 
> > ---
> > This series patches dependent on:
> > [1]
> > 
https://urldefense.com/v3/__https://patchwork.linuxtv.org/project/linux-media/patch/20250424090824.5309-1-jianhua.lin@mediatek.com/__;!!CTRNKA9wMg0ARbw!mKadXMSwCGnM_vu578CgBAihMGSqgm-6n24akvuHZKkeHA9xKZzvRJuh6K9BHIQ4HMvfHY4xK2x7OkwVaYfYUg$
> > 
> > Changes compared with v4:
> > --fix kernel robot build errors for patch 4.
> > --add reviewer for patch 1 and patch 2.
> > 
> > Changes compared with v3:
> > --change patch subject of jpeg encoder and decoder compatible.
> > 
> > Changes compared with v2:
> > --refactor smmu sid setting function interface
> > --Some modifications for patch v2's review comments.
> > 
> > Changes compared with v1:
> > --refine jpeg dt-bindings for MT8196
> > --optimize software code to manage jpeg HW count
> > --refactor smmu sid setting function interface
> > --Some modifications for patch v1's review comments.
> > 
> > Kyrie Wu (12):
> >   media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgdec
> >     compatible
> >   media: dt-bindings: mediatek,jpeg: Add mediatek, mt8196-jpgenc
> >     compatible
> >   media: mediatek: jpeg: add jpeg compatible
> >   media: mediatek: jpeg: add jpeg smmu sid setting
> >   media: mediatek: jpeg: fix jpeg hw count setting
> >   media: mediatek: jpeg: refactor jpeg buffer payload setting
> >   media: mediatek: jpeg: refactor jpeg dst buffer layout
> >   media: mediatek: jpeg: fix stop streaming flow for multi-core
> >   media: mediatek: jpeg: refactor multi-core clk suspend and resume
> >     setting
> >   media: mediatek: jpeg: fix decoding buffer number setting timing
> > issue
> >   media: mediatek: jpeg: refactor decoding resolution change
> > operation
> >   media: mediatek: jpeg: fix remove buffer operation for multi-core
> > 
> >  .../media/mediatek,mt8195-jpegdec.yaml        |   8 +-
> >  .../media/mediatek,mt8195-jpegenc.yaml        |   8 +-
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 169 +++++++++++++-
> > ----
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  21 ++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 112 +++++++++++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 112 +++++++++++-
> >  6 files changed, 377 insertions(+), 53 deletions(-)

Re: [PATCH v5 06/12] media: mediatek: jpeg: refactor jpeg buffer payload setting

[Kyrie Wu (吴晗)]

On Fri, 2025-05-30 at 13:33 -0400, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi Kyrie,
> 
> Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> > 1. for multi-core jpegdec:
> >    core0: |<-------- decoding buffer0 and resolution changed to
> > smaller
> >    core1:  |<-------- decoding buffer1
> >    core0:   |<- handling resolution changing
> >    core0:    |<- vb2_set_plane_payload
> > 2. the payload size is changed on the step of set format. Because
> > core1
> > is running and streaming has not been stopped, the format cannot be
> > set again, resulting in no change in the payload size.
> > 3. at this time, the payload size is bigger than buffer length,
> > it will print a warnning call trace
> > 4. set payload size must less than buffer length
> 
> You'll have to rework the text in this commit message, I must admit I
> don't
> understand from this text what exactly the problem is, make it really
> hard to
> review your solution.

Dear Nicolas,

I'm terribly sorry for the inconvenience caused to you. I will change
the commit message and let it read easier.

Thanks.
> 
> > 
> > Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> > ---
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.c     | 18
> > +++++++++++++++---
> >  1 file changed, 15 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index 0074d1edb534..52d59bb5c9ad 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -720,10 +720,22 @@ static int mtk_jpeg_buf_prepare(struct
> > vb2_buffer *vb)
> >               plane_fmt = q_data->pix_mp.plane_fmt[i];
> >               if (ctx->enable_exif &&
> >                   q_data->fmt->fourcc == V4L2_PIX_FMT_JPEG)
> > -                     vb2_set_plane_payload(vb, i,
> > plane_fmt.sizeimage +
> > -                                           MTK_JPEG_MAX_EXIF_SIZE)
> > ;
> > +                     if (vb->planes[i].length >
> > (plane_fmt.sizeimage +
> > +                         MTK_JPEG_MAX_EXIF_SIZE))
> > +                             vb2_set_plane_payload(vb, i,
> > +                                                   plane_fmt.sizei
> > mage +
> > +                                                   MTK_JPEG_MAX_EX
> > IF_SIZE);
> > +                     else
> > +                             vb2_set_plane_payload(vb, i,
> > +                                                   vb-
> > >planes[i].length);
> > +
> >               else
> > -                     vb2_set_plane_payload(vb,
> > i,  plane_fmt.sizeimage);
> > +                     if (vb->planes[i].length >
> > plane_fmt.sizeimage)
> > +                             vb2_set_plane_payload(vb, i,
> > +                                                   plane_fmt.sizei
> > mage);
> > +                     else
> > +                             vb2_set_plane_payload(vb, i,
> > +                                                   vb-
> > >planes[i].length);
> 
> Is this the same as ?
> 
>                 unsigned long max_size = plane_fmt.sizeimage;
> 
>                 if (ctx->enable_exif && q_data->fmt->fourcc ==
> V4L2_PIX_FMT_JPEG)
>                         max_size += MTK_JPEG_MAX_EXIF_SIZE;
> 
>                 vb2_set_plane_payload(vb, i, MIN(vb-
> >planes[i].length, max_size));
> 
> It is unclear to me how this isn't just a workaround though, looking
> forward
> your reworked commit message.
> 
> Nicolas

Yes, Thanks for you suggestion. I will fix it in the next version.

Regards,
Kyrie.
> 
> >       }
> > 
> >       return 0;

Re: [PATCH v5 07/12] media: mediatek: jpeg: refactor jpeg dst buffer layout

[Kyrie Wu (吴晗)]

On Fri, 2025-05-30 at 13:38 -0400, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi,
> 
> Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> > 1. change dst buffer size to same as struct mtk_jpeg_src_buf
> > to make sure all params of mtk_jpeg_src_buf could get a memory.
> > 2. For memory alloc operation:
> > the v4l2 framework malloc a memory, the base addr is vb2_buffer and
> > the size is sizeof(struct mtk_jpeg_src_buf), mtk_jpeg_src_buf could
> > get
> > itself addr by container_of like that:
> > vb2_buffer -> vb2_v4l2_buffer -> mtk_jpeg_src_buf.
> > vb2_v4l2_buffer must keep on the top of mtk_jpeg_src_buf.
> 
> The subject imply a refactoring, but the most important change in
> your
> patch is to fix the wrong buf strut size. Can you rework the subject
> and message to state what you are fixing please.
> 
> Add a Fixes tag, and moved it at the start of the series to show this
> isn't a problem you have introduced in previous patch.

Dear Nicolas,

Thanks for your suggestion and I will change the tag from "refactoring"
to "fixes" and add the tag.

Thanks.
> 
> > 
> > Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> > ---
> >  drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 2 +-
> >  drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h | 2 +-
> >  2 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index 52d59bb5c9ad..7e3509be6f69 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -1103,7 +1103,7 @@ static int mtk_jpeg_queue_init(void *priv,
> > struct vb2_queue *src_vq,
> >       dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> >       dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
> >       dst_vq->drv_priv = ctx;
> > -     dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
> > +     dst_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
> >       dst_vq->ops = jpeg->variant->qops;
> >       dst_vq->mem_ops = &vb2_dma_contig_memops;
> >       dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > index 655dc9c3280c..186cd1862028 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > @@ -96,10 +96,10 @@ struct mtk_jpeg_variant {
> >  };
> > 
> >  struct mtk_jpeg_src_buf {
> > -     u32 frame_num;
> >       struct vb2_v4l2_buffer b;
> >       struct list_head list;
> >       u32 bs_size;
> > +     u32 frame_num;
> 
> This "refactoring" should be split, it does not fix anything.

Thanks. I will change it in the coming version 6.
> 
> Nicolas

Regards,
Kyrie.
> 
> >       struct mtk_jpeg_dec_param dec_param;
> > 
> >       struct mtk_jpeg_ctx *curr_ctx;

Re: [PATCH v5 08/12] media: mediatek: jpeg: fix stop streaming flow for multi-core

[Kyrie Wu (吴晗)]

On Fri, 2025-05-30 at 13:40 -0400, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi,
> 
> Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> > 1. For multi-core jpegdec, the all hws may run at the same time,
> > if one hw decoded firstly, the function of
> > mtk_jpeg_dec_stop_streaming
> > would be called, but others input buffers are decoding, this will
> > cause some running buffers to be buffer done, causing errors;
> > 2. add a parameter to calculate the decoding buffer counts, it
> > wil decrease to 0 until the all buffers decoded and the
> > mtk_jpeg_dec_stop_streaming could continue to be executed.
> 
> This one is equally unclear to me. If you run different queues per
> core,
> why does this matter ?
> 
> Nicolas

Dear Nicoals,

Thank you for your question. I will reorganize my commit message in the
next version. Now, let me give a brief explanation of this patch:
This patch fixes the bug in multi-core jpeg decoding when executing
stop streaming.
Take 3 hardware as an example. If core0, core1 and core2 decode each
image at the same time, core0 completes decoding firstly. the app calls
stop streaming when it receives the result of core0. At this time,
core1 and core2 are still decoding. In stop streaming, it is necessary
to ensure that all hardware (core0-3) have completed decoding before
releasing the related buffers. Otherwise, some buffers are being used
but released, resulting in errors. This patch solves this problem.

Thanks.

Regards,
Kyrie.
> 
> > 
> > Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> > ---
> >  .../media/platform/mediatek/jpeg/mtk_jpeg_core.c | 16
> > ++++++++++++++++
> >  .../media/platform/mediatek/jpeg/mtk_jpeg_core.h |  2 ++
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c     |  9 +++++++++
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c     |  9 +++++++++
> >  4 files changed, 36 insertions(+)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index 7e3509be6f69..1d3df1230191 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -861,8 +861,12 @@ static struct vb2_v4l2_buffer
> > *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
> >  static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
> >  {
> >       struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
> > +     struct mtk_jpeg_dev *jpeg = ctx->jpeg;
> >       struct vb2_v4l2_buffer *vb;
> > 
> > +     if (jpeg->variant->multi_core)
> > +             wait_event(jpeg->hw_wq, (atomic_read(&ctx-
> > >buf_list_cnt) == 0));
> > +
> >       while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
> >               v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
> >  }
> > @@ -870,6 +874,7 @@ static void mtk_jpeg_enc_stop_streaming(struct
> > vb2_queue *q)
> >  static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
> >  {
> >       struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
> > +     struct mtk_jpeg_dev *jpeg = ctx->jpeg;
> >       struct vb2_v4l2_buffer *vb;
> > 
> >       /*
> > @@ -877,6 +882,9 @@ static void mtk_jpeg_dec_stop_streaming(struct
> > vb2_queue *q)
> >        * Before STREAMOFF, we still have to return the old
> > resolution and
> >        * subsampling. Update capture queue when the stream is off.
> >        */
> > +     if (jpeg->variant->multi_core)
> > +             wait_event(jpeg->hw_wq, (atomic_read(&ctx-
> > >buf_list_cnt) == 0));
> > +
> >       if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
> >           V4L2_TYPE_IS_CAPTURE(q->type)) {
> >               struct mtk_jpeg_src_buf *src_buf;
> > @@ -1186,6 +1194,7 @@ static int mtk_jpeg_open(struct file *file)
> >       v4l2_fh_init(&ctx->fh, vfd);
> >       file->private_data = &ctx->fh;
> >       v4l2_fh_add(&ctx->fh);
> > +     atomic_set(&ctx->buf_list_cnt, 0);
> > 
> >       ctx->jpeg = jpeg;
> >       ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
> > @@ -1568,6 +1577,11 @@ static int mtk_jpegdec_set_hw_param(struct
> > mtk_jpeg_ctx *ctx,
> >       return 0;
> >  }
> > 
> > +static void jpeg_buf_queue_inc(struct mtk_jpeg_ctx *ctx)
> > +{
> > +     atomic_inc(&ctx->buf_list_cnt);
> > +}
> > +
> >  static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
> >  {
> >       struct mtk_jpeg_ctx *ctx;
> > @@ -1693,6 +1707,7 @@ static void mtk_jpegenc_worker(struct
> > work_struct *work)
> >                            &src_buf->vb2_buf);
> >       mtk_jpeg_set_enc_params(ctx, comp_jpeg[hw_id]->reg_base);
> >       mtk_jpeg_enc_start(comp_jpeg[hw_id]->reg_base);
> > +     jpeg_buf_queue_inc(ctx);
> >       v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
> >       spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
> > 
> > @@ -1825,6 +1840,7 @@ static void mtk_jpegdec_worker(struct
> > work_struct *work)
> >                               &bs,
> >                               &fb);
> >       mtk_jpeg_dec_start(comp_jpeg[hw_id]->reg_base);
> > +     jpeg_buf_queue_inc(ctx);
> >       v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
> >       spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > index 186cd1862028..6e8304680393 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > @@ -303,6 +303,7 @@ struct mtk_jpeg_q_data {
> >   * @dst_done_queue:          encoded frame buffer queue
> >   * @done_queue_lock:         encoded frame operation spinlock
> >   * @last_done_frame_num:     the last encoded frame number
> > + * @buf_list_cnt:            the frame buffer count own by jpeg
> > driver
> >   */
> >  struct mtk_jpeg_ctx {
> >       struct mtk_jpeg_dev             *jpeg;
> > @@ -321,6 +322,7 @@ struct mtk_jpeg_ctx {
> >       /* spinlock protecting the encode done buffer */
> >       spinlock_t done_queue_lock;
> >       u32 last_done_frame_num;
> > +     atomic_t buf_list_cnt;
> >  };
> > 
> >  #endif /* _MTK_JPEG_CORE_H */
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > index 2200f3b628dc..2e6da8617484 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > @@ -522,6 +522,11 @@ static void mtk_jpegdec_put_buf(struct
> > mtk_jpegdec_comp_dev *jpeg)
> >       spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
> >  }
> > 
> > +static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
> > +{
> > +     atomic_dec(&ctx->buf_list_cnt);
> > +}
> > +
> >  static void mtk_jpegdec_timeout_work(struct work_struct *work)
> >  {
> >       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
> > @@ -530,9 +535,11 @@ static void mtk_jpegdec_timeout_work(struct
> > work_struct *work)
> >                            job_timeout_work.work);
> >       struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
> >       struct vb2_v4l2_buffer *src_buf, *dst_buf;
> > +     struct mtk_jpeg_ctx *ctx;
> > 
> >       src_buf = cjpeg->hw_param.src_buffer;
> >       dst_buf = cjpeg->hw_param.dst_buffer;
> > +     ctx = cjpeg->hw_param.curr_ctx;
> >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > 
> >       mtk_jpeg_dec_reset(cjpeg->reg_base);
> > @@ -543,6 +550,7 @@ static void mtk_jpegdec_timeout_work(struct
> > work_struct *work)
> >       wake_up(&master_jpeg->hw_wq);
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegdec_put_buf(cjpeg);
> > +     jpeg_buf_queue_dec(ctx);
> >  }
> > 
> >  static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> > @@ -583,6 +591,7 @@ static irqreturn_t
> > mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> >       buf_state = VB2_BUF_STATE_DONE;
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegdec_put_buf(jpeg);
> > +     jpeg_buf_queue_dec(ctx);
> >       pm_runtime_put(ctx->jpeg->dev);
> >       clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > index 4c264c14ad83..ff73393a2417 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > @@ -251,6 +251,11 @@ static void mtk_jpegenc_put_buf(struct
> > mtk_jpegenc_comp_dev *jpeg)
> >       spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
> >  }
> > 
> > +static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
> > +{
> > +     atomic_dec(&ctx->buf_list_cnt);
> > +}
> > +
> >  static void mtk_jpegenc_timeout_work(struct work_struct *work)
> >  {
> >       struct delayed_work *dly_work = to_delayed_work(work);
> > @@ -261,9 +266,11 @@ static void mtk_jpegenc_timeout_work(struct
> > work_struct *work)
> >       struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
> >       enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
> >       struct vb2_v4l2_buffer *src_buf, *dst_buf;
> > +     struct mtk_jpeg_ctx *ctx;
> > 
> >       src_buf = cjpeg->hw_param.src_buffer;
> >       dst_buf = cjpeg->hw_param.dst_buffer;
> > +     ctx = cjpeg->hw_param.curr_ctx;
> >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > 
> >       mtk_jpeg_enc_reset(cjpeg->reg_base);
> > @@ -274,6 +281,7 @@ static void mtk_jpegenc_timeout_work(struct
> > work_struct *work)
> >       wake_up(&master_jpeg->hw_wq);
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegenc_put_buf(cjpeg);
> > +     jpeg_buf_queue_dec(ctx);
> >  }
> > 
> >  static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> > @@ -307,6 +315,7 @@ static irqreturn_t
> > mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> >       buf_state = VB2_BUF_STATE_DONE;
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegenc_put_buf(jpeg);
> > +     jpeg_buf_queue_dec(ctx);
> >       pm_runtime_put(ctx->jpeg->dev);
> >       clk_disable_unprepare(jpeg->venc_clk.clks->clk)

Re: [PATCH v5 09/12] media: mediatek: jpeg: refactor multi-core clk suspend and resume setting

[Kyrie Wu (吴晗)]

On Fri, 2025-05-30 at 13:43 -0400, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi,
> 
> Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> > refactor jpeg clk suspend and resume setting for multi-core
> 
> You'll have to write a lot more to support such a large and I
> must say slightly convoluted change. Why do you need a special
> case for 1 core in the first place ? What about multi-core
> design that support from 1 to N cores without using different
> code path ?
> 
> Nicolas

Dear Nicolas,

For single core, the clock information is parsed and stored in the
struct of mtk_jpeg_dev, but multi-core's are in the struct of
mtk_jpegdec_comp_dev, it can not use a same software interface.

For a further thinking, it is also a fixes patch rather than
refactoring one. I will optimize this patch in v6.

Thanks.

Regards,
kyrie.
> 
> > 
> > Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> > ---
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++----
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 75
> > ++++++++++++++++++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 75
> > ++++++++++++++++++-
> >  3 files changed, 151 insertions(+), 27 deletions(-)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index 1d3df1230191..c1d2de92f125 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -1126,6 +1126,9 @@ static void mtk_jpeg_clk_on(struct
> > mtk_jpeg_dev *jpeg)
> >  {
> >       int ret;
> > 
> > +     if (jpeg->variant->multi_core)
> > +             return;
> > +
> >       ret = clk_bulk_prepare_enable(jpeg->variant->num_clks,
> >                                     jpeg->variant->clks);
> >       if (ret)
> > @@ -1134,6 +1137,9 @@ static void mtk_jpeg_clk_on(struct
> > mtk_jpeg_dev *jpeg)
> > 
> >  static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
> >  {
> > +     if (jpeg->variant->multi_core)
> > +             return;
> > +
> >       clk_bulk_disable_unprepare(jpeg->variant->num_clks,
> >                                  jpeg->variant->clks);
> >  }
> > @@ -1677,13 +1683,6 @@ static void mtk_jpegenc_worker(struct
> > work_struct *work)
> >               goto enc_end;
> >       }
> > 
> > -     ret = clk_prepare_enable(comp_jpeg[hw_id]->venc_clk.clks-
> > >clk);
> > -     if (ret) {
> > -             dev_err(jpeg->dev, "%s : %d, jpegenc
> > clk_prepare_enable fail\n",
> > -                     __func__, __LINE__);
> > -             goto enc_end;
> > -     }
> > -
> >       v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> >       v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
> > 
> > @@ -1798,20 +1797,13 @@ static void mtk_jpegdec_worker(struct
> > work_struct *work)
> >       jpeg_dst_buf->frame_num = ctx->total_frame_num;
> > 
> >       mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
> > -     ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
> > +     ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
> >       if (ret < 0) {
> >               dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail
> > !!!\n",
> >                       __func__, __LINE__);
> >               goto dec_end;
> >       }
> > 
> > -     ret = clk_prepare_enable(comp_jpeg[hw_id]->jdec_clk.clks-
> > >clk);
> > -     if (ret) {
> > -             dev_err(jpeg->dev, "%s : %d, jpegdec
> > clk_prepare_enable fail\n",
> > -                     __func__, __LINE__);
> > -             goto clk_end;
> > -     }
> > -
> >       v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> >       v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
> > 
> > @@ -1821,7 +1813,7 @@ static void mtk_jpegdec_worker(struct
> > work_struct *work)
> >                                &dst_buf->vb2_buf, &fb)) {
> >               dev_err(jpeg->dev, "%s : %d, mtk_jpeg_set_dec_dst
> > fail\n",
> >                       __func__, __LINE__);
> > -             goto setdst_end;
> > +             goto set_dst_fail;
> >       }
> > 
> >       schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
> > @@ -1846,9 +1838,7 @@ static void mtk_jpegdec_worker(struct
> > work_struct *work)
> > 
> >       return;
> > 
> > -setdst_end:
> > -     clk_disable_unprepare(comp_jpeg[hw_id]->jdec_clk.clks->clk);
> > -clk_end:
> > +set_dst_fail:
> >       pm_runtime_put(comp_jpeg[hw_id]->dev);
> >  dec_end:
> >       v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > index 2e6da8617484..db2afc5151ad 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > @@ -543,14 +543,13 @@ static void mtk_jpegdec_timeout_work(struct
> > work_struct *work)
> >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > 
> >       mtk_jpeg_dec_reset(cjpeg->reg_base);
> > -     clk_disable_unprepare(cjpeg->jdec_clk.clks->clk);
> > -     pm_runtime_put(cjpeg->dev);
> >       cjpeg->hw_state = MTK_JPEG_HW_IDLE;
> >       atomic_inc(&master_jpeg->hw_rdy);
> >       wake_up(&master_jpeg->hw_wq);
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegdec_put_buf(cjpeg);
> >       jpeg_buf_queue_dec(ctx);
> > +     pm_runtime_put(cjpeg->dev);
> >  }
> > 
> >  static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> > @@ -592,12 +591,11 @@ static irqreturn_t
> > mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegdec_put_buf(jpeg);
> >       jpeg_buf_queue_dec(ctx);
> > -     pm_runtime_put(ctx->jpeg->dev);
> > -     clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
> > 
> >       jpeg->hw_state = MTK_JPEG_HW_IDLE;
> >       wake_up(&master_jpeg->hw_wq);
> >       atomic_inc(&master_jpeg->hw_rdy);
> > +     pm_runtime_put(jpeg->dev);
> > 
> >       return IRQ_HANDLED;
> >  }
> > @@ -703,15 +701,84 @@ static int mtk_jpegdec_hw_probe(struct
> > platform_device *pdev)
> > 
> >       platform_set_drvdata(pdev, dev);
> >       pm_runtime_enable(&pdev->dev);
> > +     ret = devm_clk_bulk_get(dev->dev,
> > +                             jpegdec_clk->clk_num,
> > +                             jpegdec_clk->clks);
> > +     if (ret) {
> > +             dev_err(&pdev->dev, "Failed to init clk\n");
> > +             return ret;
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static void mtk_jpeg_clk_on(struct mtk_jpegdec_comp_dev *jpeg)
> > +{
> > +     int ret;
> > +
> > +     ret = clk_bulk_prepare_enable(jpeg->jdec_clk.clk_num,
> > +                                   jpeg->jdec_clk.clks);
> > +     if (ret)
> > +             dev_err(jpeg->dev, "%s : %d, jpegdec
> > clk_prepare_enable fail\n",
> > +                     __func__, __LINE__);
> > +}
> > +
> > +static void mtk_jpeg_clk_off(struct mtk_jpegdec_comp_dev *jpeg)
> > +{
> > +     clk_bulk_disable_unprepare(jpeg->jdec_clk.clk_num,
> > +                                jpeg->jdec_clk.clks);
> > +}
> > +
> > +static __maybe_unused int mtk_jpegdec_pm_suspend(struct device
> > *dev)
> > +{
> > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > +
> > +     mtk_jpeg_clk_off(jpeg);
> > 
> >       return 0;
> >  }
> > 
> > +static __maybe_unused int mtk_jpegdec_pm_resume(struct device
> > *dev)
> > +{
> > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > +
> > +     mtk_jpeg_clk_on(jpeg);
> > +
> > +     return 0;
> > +}
> > +
> > +static __maybe_unused int mtk_jpegdec_suspend(struct device *dev)
> > +{
> > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > +
> > +     v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
> > +     return pm_runtime_force_suspend(dev);
> > +}
> > +
> > +static __maybe_unused int mtk_jpegdec_resume(struct device *dev)
> > +{
> > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > +     int ret;
> > +
> > +     ret = pm_runtime_force_resume(dev);
> > +     if (ret < 0)
> > +             return ret;
> > +
> > +     v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
> > +     return ret;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_jpegdec_pm_ops = {
> > +     SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegdec_suspend,
> > mtk_jpegdec_resume)
> > +     SET_RUNTIME_PM_OPS(mtk_jpegdec_pm_suspend,
> > mtk_jpegdec_pm_resume, NULL)
> > +};
> > +
> >  static struct platform_driver mtk_jpegdec_hw_driver = {
> >       .probe = mtk_jpegdec_hw_probe,
> >       .driver = {
> >               .name = "mtk-jpegdec-hw",
> >               .of_match_table = mtk_jpegdec_hw_ids,
> > +             .pm             = &mtk_jpegdec_pm_ops,
> >       },
> >  };
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > index ff73393a2417..27da2a9922a6 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > @@ -274,14 +274,13 @@ static void mtk_jpegenc_timeout_work(struct
> > work_struct *work)
> >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > 
> >       mtk_jpeg_enc_reset(cjpeg->reg_base);
> > -     clk_disable_unprepare(cjpeg->venc_clk.clks->clk);
> > -     pm_runtime_put(cjpeg->dev);
> >       cjpeg->hw_state = MTK_JPEG_HW_IDLE;
> >       atomic_inc(&master_jpeg->hw_rdy);
> >       wake_up(&master_jpeg->hw_wq);
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegenc_put_buf(cjpeg);
> >       jpeg_buf_queue_dec(ctx);
> > +     pm_runtime_put(cjpeg->dev);
> >  }
> > 
> >  static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> > @@ -316,12 +315,11 @@ static irqreturn_t
> > mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegenc_put_buf(jpeg);
> >       jpeg_buf_queue_dec(ctx);
> > -     pm_runtime_put(ctx->jpeg->dev);
> > -     clk_disable_unprepare(jpeg->venc_clk.clks->clk);
> > 
> >       jpeg->hw_state = MTK_JPEG_HW_IDLE;
> >       wake_up(&master_jpeg->hw_wq);
> >       atomic_inc(&master_jpeg->hw_rdy);
> > +     pm_runtime_put(jpeg->dev);
> > 
> >       return IRQ_HANDLED;
> >  }
> > @@ -425,15 +423,84 @@ static int mtk_jpegenc_hw_probe(struct
> > platform_device *pdev)
> > 
> >       platform_set_drvdata(pdev, dev);
> >       pm_runtime_enable(&pdev->dev);
> > +     ret = devm_clk_bulk_get(dev->dev,
> > +                             jpegenc_clk->clk_num,
> > +                             jpegenc_clk->clks);
> > +     if (ret) {
> > +             dev_err(&pdev->dev, "Failed to init clk\n");
> > +             return ret;
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static void mtk_jpeg_clk_on(struct mtk_jpegenc_comp_dev *jpeg)
> > +{
> > +     int ret;
> > +
> > +     ret = clk_bulk_prepare_enable(jpeg->venc_clk.clk_num,
> > +                                   jpeg->venc_clk.clks);
> > +     if (ret)
> > +             dev_err(jpeg->dev, "%s : %d, jpegenc
> > clk_prepare_enable fail\n",
> > +                     __func__, __LINE__);
> > +}
> > +
> > +static void mtk_jpeg_clk_off(struct mtk_jpegenc_comp_dev *jpeg)
> > +{
> > +     clk_bulk_disable_unprepare(jpeg->venc_clk.clk_num,
> > +                                jpeg->venc_clk.clks);
> > +}
> > +
> > +static __maybe_unused int mtk_jpegenc_pm_suspend(struct device
> > *dev)
> > +{
> > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > +
> > +     mtk_jpeg_clk_off(jpeg);
> > 
> >       return 0;
> >  }
> > 
> > +static __maybe_unused int mtk_jpegenc_pm_resume(struct device
> > *dev)
> > +{
> > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > +
> > +     mtk_jpeg_clk_on(jpeg);
> > +
> > +     return 0;
> > +}
> > +
> > +static __maybe_unused int mtk_jpegenc_suspend(struct device *dev)
> > +{
> > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > +
> > +     v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
> > +     return pm_runtime_force_suspend(dev);
> > +}
> > +
> > +static __maybe_unused int mtk_jpegenc_resume(struct device *dev)
> > +{
> > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > +     int ret;
> > +
> > +     ret = pm_runtime_force_resume(dev);
> > +     if (ret < 0)
> > +             return ret;
> > +
> > +     v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
> > +     return ret;
> > +}
> > +
> > +static const struct dev_pm_ops mtk_jpegenc_pm_ops = {
> > +     SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegenc_suspend,
> > mtk_jpegenc_resume)
> > +     SET_RUNTIME_PM_OPS(mtk_jpegenc_pm_suspend,
> > mtk_jpegenc_pm_resume, NULL)
> > +};
> > +
> >  static struct platform_driver mtk_jpegenc_hw_driver = {
> >       .probe = mtk_jpegenc_hw_probe,
> >       .driver = {
> >               .name = "mtk-jpegenc-hw",
> >               .of_match_table = mtk_jpegenc_drv_ids,
> > +             .pm = &mtk_jpegenc_pm_ops,
> >       },
> >  };
> > 

Re: [PATCH v5 09/12] media: mediatek: jpeg: refactor multi-core clk suspend and resume setting

[Nicolas Dufresne]

Hi Kyrie,

Le vendredi 06 juin 2025 à 03:23 +0000, Kyrie Wu (吴晗) a écrit :
> On Fri, 2025-05-30 at 13:43 -0400, Nicolas Dufresne wrote:
> > External email : Please do not click links or open attachments until
> > you have verified the sender or the content.
> > 
> > 
> > Hi,
> > 
> > Le vendredi 30 mai 2025 à 15:45 +0800, Kyrie Wu a écrit :
> > > refactor jpeg clk suspend and resume setting for multi-core
> > 
> > You'll have to write a lot more to support such a large and I
> > must say slightly convoluted change. Why do you need a special
> > case for 1 core in the first place ? What about multi-core
> > design that support from 1 to N cores without using different
> > code path ?
> > 
> > Nicolas
> 
> Dear Nicolas,
> 
> For single core, the clock information is parsed and stored in the
> struct of mtk_jpeg_dev, but multi-core's are in the struct of
> mtk_jpegdec_comp_dev, it can not use a same software interface.
> 
> For a further thinking, it is also a fixes patch rather than
> refactoring one. I will optimize this patch in v6.

thanks for the feedback on this patch and other patches. I will have
a fresh look once a new version is sent.

thanks for your work,
Nicolas

> 
> Thanks.
> 
> Regards,
> kyrie.
> > 
> > > 
> > > Signed-off-by: Kyrie Wu <kyrie.wu@mediatek.com>
> > > ---
> > >  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++----
> > >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 75
> > > ++++++++++++++++++-
> > >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 75
> > > ++++++++++++++++++-
> > >  3 files changed, 151 insertions(+), 27 deletions(-)
> > > 
> > > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > > index 1d3df1230191..c1d2de92f125 100644
> > > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > > @@ -1126,6 +1126,9 @@ static void mtk_jpeg_clk_on(struct
> > > mtk_jpeg_dev *jpeg)
> > >  {
> > >       int ret;
> > > 
> > > +     if (jpeg->variant->multi_core)
> > > +             return;
> > > +
> > >       ret = clk_bulk_prepare_enable(jpeg->variant->num_clks,
> > >                                     jpeg->variant->clks);
> > >       if (ret)
> > > @@ -1134,6 +1137,9 @@ static void mtk_jpeg_clk_on(struct
> > > mtk_jpeg_dev *jpeg)
> > > 
> > >  static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
> > >  {
> > > +     if (jpeg->variant->multi_core)
> > > +             return;
> > > +
> > >       clk_bulk_disable_unprepare(jpeg->variant->num_clks,
> > >                                  jpeg->variant->clks);
> > >  }
> > > @@ -1677,13 +1683,6 @@ static void mtk_jpegenc_worker(struct
> > > work_struct *work)
> > >               goto enc_end;
> > >       }
> > > 
> > > -     ret = clk_prepare_enable(comp_jpeg[hw_id]->venc_clk.clks-
> > > > clk);
> > > -     if (ret) {
> > > -             dev_err(jpeg->dev, "%s : %d, jpegenc
> > > clk_prepare_enable fail\n",
> > > -                     __func__, __LINE__);
> > > -             goto enc_end;
> > > -     }
> > > -
> > >       v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > >       v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
> > > 
> > > @@ -1798,20 +1797,13 @@ static void mtk_jpegdec_worker(struct
> > > work_struct *work)
> > >       jpeg_dst_buf->frame_num = ctx->total_frame_num;
> > > 
> > >       mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
> > > -     ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
> > > +     ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
> > >       if (ret < 0) {
> > >               dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail
> > > !!!\n",
> > >                       __func__, __LINE__);
> > >               goto dec_end;
> > >       }
> > > 
> > > -     ret = clk_prepare_enable(comp_jpeg[hw_id]->jdec_clk.clks-
> > > > clk);
> > > -     if (ret) {
> > > -             dev_err(jpeg->dev, "%s : %d, jpegdec
> > > clk_prepare_enable fail\n",
> > > -                     __func__, __LINE__);
> > > -             goto clk_end;
> > > -     }
> > > -
> > >       v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > >       v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
> > > 
> > > @@ -1821,7 +1813,7 @@ static void mtk_jpegdec_worker(struct
> > > work_struct *work)
> > >                                &dst_buf->vb2_buf, &fb)) {
> > >               dev_err(jpeg->dev, "%s : %d, mtk_jpeg_set_dec_dst
> > > fail\n",
> > >                       __func__, __LINE__);
> > > -             goto setdst_end;
> > > +             goto set_dst_fail;
> > >       }
> > > 
> > >       schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
> > > @@ -1846,9 +1838,7 @@ static void mtk_jpegdec_worker(struct
> > > work_struct *work)
> > > 
> > >       return;
> > > 
> > > -setdst_end:
> > > -     clk_disable_unprepare(comp_jpeg[hw_id]->jdec_clk.clks->clk);
> > > -clk_end:
> > > +set_dst_fail:
> > >       pm_runtime_put(comp_jpeg[hw_id]->dev);
> > >  dec_end:
> > >       v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > > index 2e6da8617484..db2afc5151ad 100644
> > > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > > @@ -543,14 +543,13 @@ static void mtk_jpegdec_timeout_work(struct
> > > work_struct *work)
> > >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > > 
> > >       mtk_jpeg_dec_reset(cjpeg->reg_base);
> > > -     clk_disable_unprepare(cjpeg->jdec_clk.clks->clk);
> > > -     pm_runtime_put(cjpeg->dev);
> > >       cjpeg->hw_state = MTK_JPEG_HW_IDLE;
> > >       atomic_inc(&master_jpeg->hw_rdy);
> > >       wake_up(&master_jpeg->hw_wq);
> > >       v4l2_m2m_buf_done(src_buf, buf_state);
> > >       mtk_jpegdec_put_buf(cjpeg);
> > >       jpeg_buf_queue_dec(ctx);
> > > +     pm_runtime_put(cjpeg->dev);
> > >  }
> > > 
> > >  static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> > > @@ -592,12 +591,11 @@ static irqreturn_t
> > > mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> > >       v4l2_m2m_buf_done(src_buf, buf_state);
> > >       mtk_jpegdec_put_buf(jpeg);
> > >       jpeg_buf_queue_dec(ctx);
> > > -     pm_runtime_put(ctx->jpeg->dev);
> > > -     clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
> > > 
> > >       jpeg->hw_state = MTK_JPEG_HW_IDLE;
> > >       wake_up(&master_jpeg->hw_wq);
> > >       atomic_inc(&master_jpeg->hw_rdy);
> > > +     pm_runtime_put(jpeg->dev);
> > > 
> > >       return IRQ_HANDLED;
> > >  }
> > > @@ -703,15 +701,84 @@ static int mtk_jpegdec_hw_probe(struct
> > > platform_device *pdev)
> > > 
> > >       platform_set_drvdata(pdev, dev);
> > >       pm_runtime_enable(&pdev->dev);
> > > +     ret = devm_clk_bulk_get(dev->dev,
> > > +                             jpegdec_clk->clk_num,
> > > +                             jpegdec_clk->clks);
> > > +     if (ret) {
> > > +             dev_err(&pdev->dev, "Failed to init clk\n");
> > > +             return ret;
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static void mtk_jpeg_clk_on(struct mtk_jpegdec_comp_dev *jpeg)
> > > +{
> > > +     int ret;
> > > +
> > > +     ret = clk_bulk_prepare_enable(jpeg->jdec_clk.clk_num,
> > > +                                   jpeg->jdec_clk.clks);
> > > +     if (ret)
> > > +             dev_err(jpeg->dev, "%s : %d, jpegdec
> > > clk_prepare_enable fail\n",
> > > +                     __func__, __LINE__);
> > > +}
> > > +
> > > +static void mtk_jpeg_clk_off(struct mtk_jpegdec_comp_dev *jpeg)
> > > +{
> > > +     clk_bulk_disable_unprepare(jpeg->jdec_clk.clk_num,
> > > +                                jpeg->jdec_clk.clks);
> > > +}
> > > +
> > > +static __maybe_unused int mtk_jpegdec_pm_suspend(struct device
> > > *dev)
> > > +{
> > > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +
> > > +     mtk_jpeg_clk_off(jpeg);
> > > 
> > >       return 0;
> > >  }
> > > 
> > > +static __maybe_unused int mtk_jpegdec_pm_resume(struct device
> > > *dev)
> > > +{
> > > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +
> > > +     mtk_jpeg_clk_on(jpeg);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static __maybe_unused int mtk_jpegdec_suspend(struct device *dev)
> > > +{
> > > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +
> > > +     v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
> > > +     return pm_runtime_force_suspend(dev);
> > > +}
> > > +
> > > +static __maybe_unused int mtk_jpegdec_resume(struct device *dev)
> > > +{
> > > +     struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +     int ret;
> > > +
> > > +     ret = pm_runtime_force_resume(dev);
> > > +     if (ret < 0)
> > > +             return ret;
> > > +
> > > +     v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
> > > +     return ret;
> > > +}
> > > +
> > > +static const struct dev_pm_ops mtk_jpegdec_pm_ops = {
> > > +     SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegdec_suspend,
> > > mtk_jpegdec_resume)
> > > +     SET_RUNTIME_PM_OPS(mtk_jpegdec_pm_suspend,
> > > mtk_jpegdec_pm_resume, NULL)
> > > +};
> > > +
> > >  static struct platform_driver mtk_jpegdec_hw_driver = {
> > >       .probe = mtk_jpegdec_hw_probe,
> > >       .driver = {
> > >               .name = "mtk-jpegdec-hw",
> > >               .of_match_table = mtk_jpegdec_hw_ids,
> > > +             .pm             = &mtk_jpegdec_pm_ops,
> > >       },
> > >  };
> > > 
> > > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > > index ff73393a2417..27da2a9922a6 100644
> > > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > > @@ -274,14 +274,13 @@ static void mtk_jpegenc_timeout_work(struct
> > > work_struct *work)
> > >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > > 
> > >       mtk_jpeg_enc_reset(cjpeg->reg_base);
> > > -     clk_disable_unprepare(cjpeg->venc_clk.clks->clk);
> > > -     pm_runtime_put(cjpeg->dev);
> > >       cjpeg->hw_state = MTK_JPEG_HW_IDLE;
> > >       atomic_inc(&master_jpeg->hw_rdy);
> > >       wake_up(&master_jpeg->hw_wq);
> > >       v4l2_m2m_buf_done(src_buf, buf_state);
> > >       mtk_jpegenc_put_buf(cjpeg);
> > >       jpeg_buf_queue_dec(ctx);
> > > +     pm_runtime_put(cjpeg->dev);
> > >  }
> > > 
> > >  static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> > > @@ -316,12 +315,11 @@ static irqreturn_t
> > > mtk_jpegenc_hw_irq_handler(int irq, void *priv)
> > >       v4l2_m2m_buf_done(src_buf, buf_state);
> > >       mtk_jpegenc_put_buf(jpeg);
> > >       jpeg_buf_queue_dec(ctx);
> > > -     pm_runtime_put(ctx->jpeg->dev);
> > > -     clk_disable_unprepare(jpeg->venc_clk.clks->clk);
> > > 
> > >       jpeg->hw_state = MTK_JPEG_HW_IDLE;
> > >       wake_up(&master_jpeg->hw_wq);
> > >       atomic_inc(&master_jpeg->hw_rdy);
> > > +     pm_runtime_put(jpeg->dev);
> > > 
> > >       return IRQ_HANDLED;
> > >  }
> > > @@ -425,15 +423,84 @@ static int mtk_jpegenc_hw_probe(struct
> > > platform_device *pdev)
> > > 
> > >       platform_set_drvdata(pdev, dev);
> > >       pm_runtime_enable(&pdev->dev);
> > > +     ret = devm_clk_bulk_get(dev->dev,
> > > +                             jpegenc_clk->clk_num,
> > > +                             jpegenc_clk->clks);
> > > +     if (ret) {
> > > +             dev_err(&pdev->dev, "Failed to init clk\n");
> > > +             return ret;
> > > +     }
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static void mtk_jpeg_clk_on(struct mtk_jpegenc_comp_dev *jpeg)
> > > +{
> > > +     int ret;
> > > +
> > > +     ret = clk_bulk_prepare_enable(jpeg->venc_clk.clk_num,
> > > +                                   jpeg->venc_clk.clks);
> > > +     if (ret)
> > > +             dev_err(jpeg->dev, "%s : %d, jpegenc
> > > clk_prepare_enable fail\n",
> > > +                     __func__, __LINE__);
> > > +}
> > > +
> > > +static void mtk_jpeg_clk_off(struct mtk_jpegenc_comp_dev *jpeg)
> > > +{
> > > +     clk_bulk_disable_unprepare(jpeg->venc_clk.clk_num,
> > > +                                jpeg->venc_clk.clks);
> > > +}
> > > +
> > > +static __maybe_unused int mtk_jpegenc_pm_suspend(struct device
> > > *dev)
> > > +{
> > > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +
> > > +     mtk_jpeg_clk_off(jpeg);
> > > 
> > >       return 0;
> > >  }
> > > 
> > > +static __maybe_unused int mtk_jpegenc_pm_resume(struct device
> > > *dev)
> > > +{
> > > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +
> > > +     mtk_jpeg_clk_on(jpeg);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static __maybe_unused int mtk_jpegenc_suspend(struct device *dev)
> > > +{
> > > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +
> > > +     v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
> > > +     return pm_runtime_force_suspend(dev);
> > > +}
> > > +
> > > +static __maybe_unused int mtk_jpegenc_resume(struct device *dev)
> > > +{
> > > +     struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
> > > +     int ret;
> > > +
> > > +     ret = pm_runtime_force_resume(dev);
> > > +     if (ret < 0)
> > > +             return ret;
> > > +
> > > +     v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
> > > +     return ret;
> > > +}
> > > +
> > > +static const struct dev_pm_ops mtk_jpegenc_pm_ops = {
> > > +     SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegenc_suspend,
> > > mtk_jpegenc_resume)
> > > +     SET_RUNTIME_PM_OPS(mtk_jpegenc_pm_suspend,
> > > mtk_jpegenc_pm_resume, NULL)
> > > +};
> > > +
> > >  static struct platform_driver mtk_jpegenc_hw_driver = {
> > >       .probe = mtk_jpegenc_hw_probe,
> > >       .driver = {
> > >               .name = "mtk-jpegenc-hw",
> > >               .of_match_table = mtk_jpegenc_drv_ids,
> > > +             .pm = &mtk_jpegenc_pm_ops,
> > >       },
> > >  };
> > >