[V1,00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[V1,00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[kyrie.wu]

This series adds support for mt8196 multi-hardwares jpeg enc & dec,
by first adding mt8196 jpegdec and jpegenc compatible to install
kernel driver. Add smmu setting to support smmu and iommu at the
same time.
Secondly refactor buffer and clock setting to support multi-hw jpeg
working.
Lastly, fix some bugs, including resolution change handleing, stop
streaming sw flow and others.

This series has been tested with MT8196 tast test.
Encoding and decoding worked for this chip.

Patches 1-3 Adds jpeg encoder and decoder compatible.
Patches 4 add jpeg smmu sid setting.
Patches 5 fix jpeg hw count setting to support different chips.
Patches 6 refactor jpeg buffer payload setting to handle buffer
size bug while resolution changed.
Patches 7 reconstruct jpeg dst buffer layout.
Patches 8 fix multi-core stop streaming flow
Patches 9 refactor multi-core clk suspend/resume setting
Patches 10 fix decoding buffer number setting timing issue
Patches 11 refactor decoding resolution change operation
Patches 12 fix remove buffer operation

---
This series patches dependent on:
[1]
https://patchwork.kernel.org/project/linux-mediatek/patch/20240808092555.12999-1-jianhua.lin@mediatek.com/

kyrie.wu (11):
  dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible
  dt-bindings: mediatek: Add mediatek, mt8196-jpgenc compatible
  media: mediatek: jpeg: add jpeg compatible
  media: mediatek: jpeg: add jpeg smmu sid setting
  media: mediatek: jpeg: fix jpeg hw count setting
  media: mediatek: jpeg: refactor jpeg buffer payload setting
  media: mediatek: jpeg: refactor jpeg dst buffer layout
  media: mediatek: jpeg: fix stop streaming flow for multi-core
  media: mediatek: jpeg: refactor multi-core clk suspend and resume
    setting
  media: mediatek: jpeg: fix decoding buffer number setting timing issue
  media: mediatek: jpeg: refactor decoding resolution change operation
  media: mediatek: jpeg: fix remove buffer operation for multi-core

 ....yaml => mediatek,multi-core-jpegdec.yaml} |  10 +-
 ....yaml => mediatek,multi-core-jpegenc.yaml} |  10 +-
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 126 ++++++++++++------
 .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  17 ++-
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 116 +++++++++++++++-
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |   4 +
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 113 +++++++++++++++-
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |   4 +
 8 files changed, 343 insertions(+), 57 deletions(-)
 rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)
 rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegenc.yaml => mediatek,multi-core-jpegenc.yaml} (94%)

-- 
2.46.0

[V1,01/12] dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible

[kyrie.wu]

Add mediatek,mt8196-jpgdec compatible to binding document.

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 ...5-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
 rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)

diff --git a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml b/Documentation/devicetree/bindings/media/mediatek,multi-core-jpegdec.yaml
similarity index 95%
rename from Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
rename to Documentation/devicetree/bindings/media/mediatek,multi-core-jpegdec.yaml
index e5448c60e3eb..5ec98ddad506 100644
--- a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,multi-core-jpegdec.yaml
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
 %YAML 1.2
 ---
-$id: http://devicetree.org/schemas/media/mediatek,mt8195-jpegdec.yaml#
+$id: http://devicetree.org/schemas/media/mediatek,multi-core-jpegdec.yaml#
 $schema: http://devicetree.org/meta-schemas/core.yaml#
 
 title: MediaTek JPEG Decoder
@@ -14,7 +14,9 @@ description:
 
 properties:
   compatible:
-    const: mediatek,mt8195-jpgdec
+    enum:
+      - mediatek,mt8195-jpgdec
+      - mediatek,mt8196-jpgdec
 
   power-domains:
     maxItems: 1
@@ -44,7 +46,9 @@ patternProperties:
 
     properties:
       compatible:
-        const: mediatek,mt8195-jpgdec-hw
+        enum:
+          - mediatek,mt8195-jpgdec-hw
+          - mediatek,mt8196-jpgdec-hw
 
       reg:
         maxItems: 1
-- 
2.46.0

Re: [V1,01/12] dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible

[Krzysztof Kozlowski]

On 09/01/2025 14:35, kyrie.wu wrote:
> Add mediatek,mt8196-jpgdec compatible to binding document.

Please use standard email subjects, so with the PATCH keyword in the
title. `git format-patch -vX` helps here to create proper versioned
patches. Another useful tool is b4. Skipping the PATCH keyword makes
filtering of emails more difficult thus making the review process less
convenient.

> 
> Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> ---
>  ...5-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
>  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)
> 
> diff --git a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml b/Documentation/devicetree/bindings/media/mediatek,multi-core-jpegdec.yaml
> similarity index 95%
> rename from Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
> rename to Documentation/devicetree/bindings/media/mediatek,multi-core-jpegdec.yaml


Don't rename the file.

> index e5448c60e3eb..5ec98ddad506 100644
> --- a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegdec.yaml
> +++ b/Documentation/devicetree/bindings/media/mediatek,multi-core-jpegdec.yaml
> @@ -1,7 +1,7 @@
>  # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
>  %YAML 1.2
>  ---
> -$id: http://devicetree.org/schemas/media/mediatek,mt8195-jpegdec.yaml#
> +$id: http://devicetree.org/schemas/media/mediatek,multi-core-jpegdec.yaml#
>  $schema: http://devicetree.org/meta-schemas/core.yaml#
>  
>  title: MediaTek JPEG Decoder
> @@ -14,7 +14,9 @@ description:
>  
>  properties:
>    compatible:
> -    const: mediatek,mt8195-jpgdec
> +    enum:
> +      - mediatek,mt8195-jpgdec
> +      - mediatek,mt8196-jpgdec

Devices look compatible, so express it with fallback.

Best regards,
Krzysztof

Re: [V1,01/12] dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible

[Kyrie Wu (吴晗)]

On Thu, 2025-01-09 at 14:54 +0100, Krzysztof Kozlowski wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 09/01/2025 14:35, kyrie.wu wrote:
> > Add mediatek,mt8196-jpgdec compatible to binding document.
> 
> Please use standard email subjects, so with the PATCH keyword in the
> title. `git format-patch -vX` helps here to create proper versioned
> patches. Another useful tool is b4. Skipping the PATCH keyword makes
> filtering of emails more difficult thus making the review process
> less
> convenient.
> 
> > 
> > Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> > ---
> >  ...5-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} | 10
> > +++++++---
> >  1 file changed, 7 insertions(+), 3 deletions(-)
> >  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-
> > jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)
> > 
> > diff --git
> > a/Documentation/devicetree/bindings/media/mediatek,mt8195-
> > jpegdec.yaml
> > b/Documentation/devicetree/bindings/media/mediatek,multi-core-
> > jpegdec.yaml
> > similarity index 95%
> > rename from
> > Documentation/devicetree/bindings/media/mediatek,mt8195-
> > jpegdec.yaml
> > rename to Documentation/devicetree/bindings/media/mediatek,multi-
> > core-jpegdec.yaml
> 
> 
> Don't rename the file.
> 
> > index e5448c60e3eb..5ec98ddad506 100644
> > --- a/Documentation/devicetree/bindings/media/mediatek,mt8195-
> > jpegdec.yaml
> > +++ b/Documentation/devicetree/bindings/media/mediatek,multi-core-
> > jpegdec.yaml
> > @@ -1,7 +1,7 @@
> >  # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> >  %YAML 1.2
> >  ---
> > -$id: 
> > https://urldefense.com/v3/__http://devicetree.org/schemas/media/mediatek,mt8195-jpegdec.yaml*__;Iw!!CTRNKA9wMg0ARbw!lg1d9e5vm7JQRbUIRKg64b44cKCTOgxo8qiwE2QuYSBLcMHUaaFKbikp6yOXI4lvceYOAwLELlqtbQ$
> > +$id: 
> > https://urldefense.com/v3/__http://devicetree.org/schemas/media/mediatek,multi-core-jpegdec.yaml*__;Iw!!CTRNKA9wMg0ARbw!lg1d9e5vm7JQRbUIRKg64b44cKCTOgxo8qiwE2QuYSBLcMHUaaFKbikp6yOXI4lvceYOAwJqmnSMWA$
> >  $schema: 
> > https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!lg1d9e5vm7JQRbUIRKg64b44cKCTOgxo8qiwE2QuYSBLcMHUaaFKbikp6yOXI4lvceYOAwIJtjn37Q$
> > 
> >  title: MediaTek JPEG Decoder
> > @@ -14,7 +14,9 @@ description:
> > 
> >  properties:
> >    compatible:
> > -    const: mediatek,mt8195-jpgdec
> > +    enum:
> > +      - mediatek,mt8195-jpgdec
> > +      - mediatek,mt8196-jpgdec
> 
> Devices look compatible, so express it with fallback.
> 
> Best regards,
> Krzysztof

Dear Krzysztof,

The MT8195 and MT8196 are both multi-hardware chips, I renamed the file
to treat it as a common file, if I don't do that it will need to
recreate a file with similar contents. Is this acceptable?
If the above description is acceptable, I will apply your request in
the next version.

Thanks.

Best regards,
Kyrie

Re: [V1,01/12] dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible

[Krzysztof Kozlowski]

On 07/04/2025 09:13, Kyrie Wu (吴晗) wrote:
> On Thu, 2025-01-09 at 14:54 +0100, Krzysztof Kozlowski wrote:

...


>>>
>>>  properties:
>>>    compatible:
>>> -    const: mediatek,mt8195-jpgdec
>>> +    enum:
>>> +      - mediatek,mt8195-jpgdec
>>> +      - mediatek,mt8196-jpgdec
>>
>> Devices look compatible, so express it with fallback.
>>
>> Best regards,
>> Krzysztof
> 
> Dear Krzysztof,
> 
> The MT8195 and MT8196 are both multi-hardware chips, I renamed the file
> to treat it as a common file, if I don't do that it will need to
> recreate a file with similar contents. Is this acceptable?

No, do not create own rules. I do not see how this is related to my
comment at all.

You responded three months after my review, so entire context including
actual mails in my inbox are gone.

If you disagree with feedback, respond in timely manner. I feel
discussing something after three months like wasting my time.


> If the above description is acceptable, I will apply your request in
> the next version.

Best regards,
Krzysztof

Re: [V1,01/12] dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible

[Kyrie Wu (吴晗)]

On Mon, 2025-04-07 at 09:19 +0200, Krzysztof Kozlowski wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 07/04/2025 09:13, Kyrie Wu (吴晗) wrote:
> > On Thu, 2025-01-09 at 14:54 +0100, Krzysztof Kozlowski wrote:
> 
> ...
> 
> 
> > > > 
> > > >  properties:
> > > >    compatible:
> > > > -    const: mediatek,mt8195-jpgdec
> > > > +    enum:
> > > > +      - mediatek,mt8195-jpgdec
> > > > +      - mediatek,mt8196-jpgdec
> > > 
> > > Devices look compatible, so express it with fallback.
> > > 
> > > Best regards,
> > > Krzysztof
> > 
> > Dear Krzysztof,
> > 
> > The MT8195 and MT8196 are both multi-hardware chips, I renamed the
> > file
> > to treat it as a common file, if I don't do that it will need to
> > recreate a file with similar contents. Is this acceptable?
> 
> No, do not create own rules. I do not see how this is related to my
> comment at all.
> 
> You responded three months after my review, so entire context
> including
> actual mails in my inbox are gone.
> 
> If you disagree with feedback, respond in timely manner. I feel
> discussing something after three months like wasting my time.
> 
> 
> > If the above description is acceptable, I will apply your request
> > in
> > the next version.
> 
> Best regards,
> Krzysztof

Dear Krzysztof,
 
I'm truly sorry for what I have done. I feel very ashamed and self-
blame after I tried to stand in your shoes. 
 
You contributed a lot of time to help me review my patch, you could
have spent it with your family or your friends, and I failed you in
your selfless kindness. I have not only wasted your precious time but
also disrespected your contribution by ignoring your reply,
undoubtedly.

This was never my intention, and I‘m committed to improving my
communication and collaboration within the community. I value your
expertise and feedback, and I will ensure that I respond promptly and
thoughtfully to any future reviews. (I have improved my email
notifications, and I will share this experience within my team to
remind all developers.)
 
Thank you for your understanding and patience. I hope to rebuild your
trust and continue contributing positively to the community.
 
Sincerely,
Kyrie

[V1,02/12] dt-bindings: mediatek: Add mediatek, mt8196-jpgenc compatible

[kyrie.wu]

Add mediatek,mt8196-jpgenc compatible to binding document.

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 ...5-jpegenc.yaml => mediatek,multi-core-jpegenc.yaml} | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
 rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegenc.yaml => mediatek,multi-core-jpegenc.yaml} (94%)

diff --git a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml b/Documentation/devicetree/bindings/media/mediatek,multi-core-jpegenc.yaml
similarity index 94%
rename from Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml
rename to Documentation/devicetree/bindings/media/mediatek,multi-core-jpegenc.yaml
index 596186497b68..5d3f2adf3a3a 100644
--- a/Documentation/devicetree/bindings/media/mediatek,mt8195-jpegenc.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,multi-core-jpegenc.yaml
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
 %YAML 1.2
 ---
-$id: http://devicetree.org/schemas/media/mediatek,mt8195-jpegenc.yaml#
+$id: http://devicetree.org/schemas/media/mediatek,multi-core-jpegenc.yaml#
 $schema: http://devicetree.org/meta-schemas/core.yaml#
 
 title: MediaTek JPEG Encoder
@@ -14,7 +14,9 @@ description:
 
 properties:
   compatible:
-    const: mediatek,mt8195-jpgenc
+    enum:
+      - mediatek,mt8195-jpgenc
+      - mediatek,mt8196-jpgenc
 
   power-domains:
     maxItems: 1
@@ -44,7 +46,9 @@ patternProperties:
 
     properties:
       compatible:
-        const: mediatek,mt8195-jpgenc-hw
+        enum:
+          - mediatek,mt8195-jpgenc-hw
+          - mediatek,mt8196-jpgenc-hw
 
       reg:
         maxItems: 1
-- 
2.46.0

[V1,03/12] media: mediatek: jpeg: add jpeg compatible

[kyrie.wu]

Add jpeg dec and enc compatible for mt8196

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 8 ++++++++
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 3 +++
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 3 +++
 3 files changed, 14 insertions(+)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 9517ebed0701..c3ccc525d9fd 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1996,6 +1996,14 @@ static const struct of_device_id mtk_jpeg_match[] = {
 		.compatible = "mediatek,mt8188-jpgdec",
 		.data = &mtk8188_jpegdec_drvdata,
 	},
+	{
+		.compatible = "mediatek,mt8196-jpgenc",
+		.data = &mtk8195_jpegenc_drvdata,
+	},
+	{
+		.compatible = "mediatek,mt8196-jpgdec",
+		.data = &mtk8195_jpegdec_drvdata,
+	},
 	{},
 };
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index ebded06ba92d..d868e46aaf37 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -45,6 +45,9 @@ static const struct of_device_id mtk_jpegdec_hw_ids[] = {
 	{
 		.compatible = "mediatek,mt8195-jpgdec-hw",
 	},
+	{
+		.compatible = "mediatek,mt8196-jpgdec-hw",
+	},
 	{},
 };
 MODULE_DEVICE_TABLE(of, mtk_jpegdec_hw_ids);
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index 87fe1f324f23..ca06d4f435cd 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -52,6 +52,9 @@ static const struct of_device_id mtk_jpegenc_drv_ids[] = {
 	{
 		.compatible = "mediatek,mt8195-jpgenc-hw",
 	},
+	{
+		.compatible = "mediatek,mt8196-jpgenc-hw",
+	},
 	{},
 };
 MODULE_DEVICE_TABLE(of, mtk_jpegenc_drv_ids);
-- 
2.46.0

Re: [V1,03/12] media: mediatek: jpeg: add jpeg compatible

[AngeloGioacchino Del Regno]

Il 09/01/25 14:35, kyrie.wu ha scritto:
> Add jpeg dec and enc compatible for mt8196
> 
> Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> ---
>   drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 8 ++++++++
>   drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 3 +++
>   drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 3 +++
>   3 files changed, 14 insertions(+)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index 9517ebed0701..c3ccc525d9fd 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -1996,6 +1996,14 @@ static const struct of_device_id mtk_jpeg_match[] = {
>   		.compatible = "mediatek,mt8188-jpgdec",
>   		.data = &mtk8188_jpegdec_drvdata,
>   	},
> +	{
> +		.compatible = "mediatek,mt8196-jpgenc",
> +		.data = &mtk8195_jpegenc_drvdata,
> +	},
> +	{
> +		.compatible = "mediatek,mt8196-jpgdec",
> +		.data = &mtk8195_jpegdec_drvdata,

You're adding this with MT8195 platform data, and then you're changing it to
MT8196 specific platform data.

Just put this commit at the end of the series, at this point.

Regards,
Angelo

> +	},
>   	{},
>   };
>   
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> index ebded06ba92d..d868e46aaf37 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> @@ -45,6 +45,9 @@ static const struct of_device_id mtk_jpegdec_hw_ids[] = {
>   	{
>   		.compatible = "mediatek,mt8195-jpgdec-hw",
>   	},
> +	{
> +		.compatible = "mediatek,mt8196-jpgdec-hw",
> +	},
>   	{},
>   };
>   MODULE_DEVICE_TABLE(of, mtk_jpegdec_hw_ids);
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> index 87fe1f324f23..ca06d4f435cd 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> @@ -52,6 +52,9 @@ static const struct of_device_id mtk_jpegenc_drv_ids[] = {
>   	{
>   		.compatible = "mediatek,mt8195-jpgenc-hw",
>   	},
> +	{
> +		.compatible = "mediatek,mt8196-jpgenc-hw",
> +	},
>   	{},
>   };
>   MODULE_DEVICE_TABLE(of, mtk_jpegenc_drv_ids);

Re: [V1,03/12] media: mediatek: jpeg: add jpeg compatible

[Kyrie Wu (吴晗)]

On Fri, 2025-01-10 at 10:46 +0100, AngeloGioacchino Del Regno wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Il 09/01/25 14:35, kyrie.wu ha scritto:
> > Add jpeg dec and enc compatible for mt8196
> > 
> > Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> > ---
> >   drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 8
> > ++++++++
> >   drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 3 +++
> >   drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 3 +++
> >   3 files changed, 14 insertions(+)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index 9517ebed0701..c3ccc525d9fd 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -1996,6 +1996,14 @@ static const struct of_device_id
> > mtk_jpeg_match[] = {
> >               .compatible = "mediatek,mt8188-jpgdec",
> >               .data = &mtk8188_jpegdec_drvdata,
> >       },
> > +     {
> > +             .compatible = "mediatek,mt8196-jpgenc",
> > +             .data = &mtk8195_jpegenc_drvdata,
> > +     },
> > +     {
> > +             .compatible = "mediatek,mt8196-jpgdec",
> > +             .data = &mtk8195_jpegdec_drvdata,
> 
> You're adding this with MT8195 platform data, and then you're
> changing it to
> MT8196 specific platform data.
> 
> Just put this commit at the end of the series, at this point.
> 
> Regards,
> Angelo

Dear Angelo,

Thanks for your advices, I will add smmu info. into the platform data
and push it to the end.

Thanks.

Regards,
Kyrie.

> 
> > +     },
> >       {},
> >   };
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > index ebded06ba92d..d868e46aaf37 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > @@ -45,6 +45,9 @@ static const struct of_device_id
> > mtk_jpegdec_hw_ids[] = {
> >       {
> >               .compatible = "mediatek,mt8195-jpgdec-hw",
> >       },
> > +     {
> > +             .compatible = "mediatek,mt8196-jpgdec-hw",
> > +     },
> >       {},
> >   };
> >   MODULE_DEVICE_TABLE(of, mtk_jpegdec_hw_ids);
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > index 87fe1f324f23..ca06d4f435cd 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
> > @@ -52,6 +52,9 @@ static const struct of_device_id
> > mtk_jpegenc_drv_ids[] = {
> >       {
> >               .compatible = "mediatek,mt8195-jpgenc-hw",
> >       },
> > +     {
> > +             .compatible = "mediatek,mt8196-jpgenc-hw",
> > +     },
> >       {},
> >   };
> >   MODULE_DEVICE_TABLE(of, mtk_jpegenc_drv_ids);
> 
> 
> 

[V1,04/12] media: mediatek: jpeg: add jpeg smmu sid setting

[kyrie.wu]

Add a configuration to set jpeg dec & enc smmu sid

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    |  6 ++++
 .../platform/mediatek/jpeg/mtk_jpeg_core.h    | 11 +++++++
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 30 +++++++++++++++++--
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |  4 +++
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 26 ++++++++++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |  4 +++
 6 files changed, 78 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index c3ccc525d9fd..77b3bd6c4d3f 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1655,6 +1655,9 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 	jpeg_dst_buf->frame_num = ctx->total_frame_num;
 	ctx->total_frame_num++;
 	mtk_jpeg_enc_reset(comp_jpeg[hw_id]->reg_base);
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+	mtk_jpeg_enc_set_smmu_sid(hw_id);
+#endif
 	mtk_jpeg_set_enc_dst(ctx,
 			     comp_jpeg[hw_id]->reg_base,
 			     &dst_buf->vb2_buf);
@@ -1771,6 +1774,9 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 	spin_lock_irqsave(&comp_jpeg[hw_id]->hw_lock, flags);
 	ctx->total_frame_num++;
 	mtk_jpeg_dec_reset(comp_jpeg[hw_id]->reg_base);
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+	mtk_jpeg_dec_set_smmu_sid(hw_id);
+#endif
 	mtk_jpeg_dec_set_config(comp_jpeg[hw_id]->reg_base,
 				jpeg->variant->support_34bit,
 				&jpeg_src_buf->dec_param,
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index 8fddc133c46c..d3aba1e6cae8 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -36,6 +36,17 @@
 
 #define MTK_JPEG_ADDR_MASK GENMASK(1, 0)
 
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+#define JPG_REG_CORE0_GUSER_ID                          0x380d0000
+#define JPG_REG_CORE1_GUSER_ID                          0x388d0000
+#define JPG_REG_GUSER_ID_MASK                           0x7
+#define JPG_REG_GUSER_ID_DEC_SID                        0x4
+#define JPG_REG_GUSER_ID_ENC_SID                        0x5
+#define JPG_REG_DEC_GUSER_ID_SHIFT                      8
+#define JPG_REG_ENC_GUSER_ID_SHIFT                      4
+#define GUSER_ID_MAPRANGE                               4
+#endif
+
 /**
  * enum mtk_jpeg_ctx_state - states of the context state machine
  * @MTK_JPEG_INIT:		current state is initialized
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index d868e46aaf37..fadfc4b5e366 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -274,6 +274,32 @@ void mtk_jpeg_dec_reset(void __iomem *base)
 }
 EXPORT_SYMBOL_GPL(mtk_jpeg_dec_reset);
 
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+void mtk_jpeg_dec_set_smmu_sid(int hwid)
+{
+	void __iomem *dec_reg_base;
+	u32 val, mask;
+
+	if (hwid)
+		dec_reg_base = ioremap(JPG_REG_CORE1_GUSER_ID, GUSER_ID_MAPRANGE);
+	else
+		dec_reg_base = ioremap(JPG_REG_CORE0_GUSER_ID, GUSER_ID_MAPRANGE);
+	if (!dec_reg_base) {
+		dev_err(jpeg->dev, "Failed to map jpgdec JPG_REG_GUSER_ID\n");
+		return;
+	}
+
+	val = ioread32(dec_reg_base);
+	mask = ~(JPG_REG_GUSER_ID_MASK << JPG_REG_DEC_GUSER_ID_SHIFT);
+	val &= mask;
+	val |= (JPG_REG_GUSER_ID_DEC_SID << JPG_REG_DEC_GUSER_ID_SHIFT);
+
+	iowrite32(val, dec_reg_base);
+	iounmap(dec_reg_base);
+}
+EXPORT_SYMBOL_GPL(mtk_jpeg_dec_set_smmu_sid);
+#endif
+
 static void mtk_jpeg_dec_set_brz_factor(void __iomem *base, u8 yscale_w,
 				u8 yscale_h, u8 uvscale_w, u8 uvscale_h)
 {
@@ -552,7 +578,6 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	enum vb2_buffer_state buf_state;
-	struct mtk_jpeg_ctx *ctx;
 	u32 dec_irq_ret;
 	u32 irq_status;
 	int i;
@@ -562,7 +587,6 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 
 	cancel_delayed_work(&jpeg->job_timeout_work);
 
-	ctx = jpeg->hw_param.curr_ctx;
 	src_buf = jpeg->hw_param.src_buffer;
 	dst_buf = jpeg->hw_param.dst_buffer;
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
@@ -585,7 +609,7 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	buf_state = VB2_BUF_STATE_DONE;
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(jpeg);
-	pm_runtime_put(ctx->jpeg->dev);
+	pm_runtime_put(jpeg->dev);
 	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
 
 	jpeg->hw_state = MTK_JPEG_HW_IDLE;
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.h
index 2948c9c300a4..2a5680f52ccb 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.h
@@ -79,4 +79,8 @@ void mtk_jpeg_dec_set_config(void __iomem *base,
 void mtk_jpeg_dec_reset(void __iomem *dec_reg_base);
 void mtk_jpeg_dec_start(void __iomem *dec_reg_base);
 
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+void mtk_jpeg_dec_set_smmu_sid(int hwid);
+#endif
+
 #endif /* _MTK_JPEG_HW_H */
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index ca06d4f435cd..fc7676d13adb 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -67,6 +67,32 @@ void mtk_jpeg_enc_reset(void __iomem *base)
 }
 EXPORT_SYMBOL_GPL(mtk_jpeg_enc_reset);
 
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+void mtk_jpeg_enc_set_smmu_sid(int hwid)
+{
+	void __iomem *enc_reg_base;
+	u32 val, mask;
+
+	if (hwid)
+		enc_reg_base = ioremap(JPG_REG_CORE1_GUSER_ID, GUSER_ID_MAPRANGE);
+	else
+		enc_reg_base = ioremap(JPG_REG_CORE0_GUSER_ID, GUSER_ID_MAPRANGE);
+	if (!enc_reg_base) {
+		dev_err(jpeg->dev, "Failed to map jpgenc JPG_REG_GUSER_ID\n");
+		return;
+	}
+
+	val = ioread32(enc_reg_base);
+	mask = ~(JPG_REG_GUSER_ID_MASK << JPG_REG_ENC_GUSER_ID_SHIFT);
+	val &= mask;
+	val |= (JPG_REG_GUSER_ID_ENC_SID << JPG_REG_ENC_GUSER_ID_SHIFT);
+
+	iowrite32(val, enc_reg_base);
+	iounmap(enc_reg_base);
+}
+EXPORT_SYMBOL_GPL(mtk_jpeg_enc_set_smmu_sid);
+#endif
+
 u32 mtk_jpeg_enc_get_file_size(void __iomem *base, bool support_34bit)
 {
 	/*
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.h
index 31ec9030ae88..723391aba639 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.h
@@ -93,4 +93,8 @@ void mtk_jpeg_set_enc_dst(struct mtk_jpeg_ctx *ctx, void __iomem *base,
 			  struct vb2_buffer *dst_buf);
 void mtk_jpeg_set_enc_params(struct mtk_jpeg_ctx *ctx,  void __iomem *base);
 
+#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
+void mtk_jpeg_enc_set_smmu_sid(int hwid);
+#endif
+
 #endif /* _MTK_JPEG_ENC_HW_H */
-- 
2.46.0

Re: [V1,04/12] media: mediatek: jpeg: add jpeg smmu sid setting

[AngeloGioacchino Del Regno]

Il 09/01/25 14:35, kyrie.wu ha scritto:
> Add a configuration to set jpeg dec & enc smmu sid
> 
> Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> ---
>   .../platform/mediatek/jpeg/mtk_jpeg_core.c    |  6 ++++
>   .../platform/mediatek/jpeg/mtk_jpeg_core.h    | 11 +++++++
>   .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 30 +++++++++++++++++--
>   .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |  4 +++
>   .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 26 ++++++++++++++++
>   .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |  4 +++
>   6 files changed, 78 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index c3ccc525d9fd..77b3bd6c4d3f 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -1655,6 +1655,9 @@ static void mtk_jpegenc_worker(struct work_struct *work)
>   	jpeg_dst_buf->frame_num = ctx->total_frame_num;
>   	ctx->total_frame_num++;
>   	mtk_jpeg_enc_reset(comp_jpeg[hw_id]->reg_base);
> +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> +	mtk_jpeg_enc_set_smmu_sid(hw_id);
> +#endif
>   	mtk_jpeg_set_enc_dst(ctx,
>   			     comp_jpeg[hw_id]->reg_base,
>   			     &dst_buf->vb2_buf);
> @@ -1771,6 +1774,9 @@ static void mtk_jpegdec_worker(struct work_struct *work)
>   	spin_lock_irqsave(&comp_jpeg[hw_id]->hw_lock, flags);
>   	ctx->total_frame_num++;
>   	mtk_jpeg_dec_reset(comp_jpeg[hw_id]->reg_base);
> +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)

You can avoid using preprocessor conditionals - and then, this doesn't look like an
ARM SMMUv3 configuration, but rather a JPEG dec/enc HW config, so in that case the
enclosing of this in the proposed config option would even be wrong.

Use platform data for that.

> +	mtk_jpeg_dec_set_smmu_sid(hw_id);
> +#endif
>   	mtk_jpeg_dec_set_config(comp_jpeg[hw_id]->reg_base,
>   				jpeg->variant->support_34bit,
>   				&jpeg_src_buf->dec_param,
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> index 8fddc133c46c..d3aba1e6cae8 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> @@ -36,6 +36,17 @@
>   
>   #define MTK_JPEG_ADDR_MASK GENMASK(1, 0)
>   
> +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> +#define JPG_REG_CORE0_GUSER_ID                          0x380d0000
> +#define JPG_REG_CORE1_GUSER_ID                          0x388d0000

At least these two definitions shall go in the devicetree reg node.

> +#define JPG_REG_GUSER_ID_MASK                           0x7
> +#define JPG_REG_GUSER_ID_DEC_SID                        0x4
> +#define JPG_REG_GUSER_ID_ENC_SID                        0x5
> +#define JPG_REG_DEC_GUSER_ID_SHIFT                      8
> +#define JPG_REG_ENC_GUSER_ID_SHIFT                      4

If this is setting an IOMMU SID, then you can just use the "iommus" property
to pass a handle to the IOMMU that this device is using plus the streamid,
which you can retrieve and write to the guser_id_{dec,enc}_sid register.

There's no reason to hardcode that in this header, and actually hardcoding
will give you a number of issues (example: unpowered/unclocked access to the
IOMMU, and many others).

> +#define GUSER_ID_MAPRANGE                               4
> +#endif
> +
>   /**
>    * enum mtk_jpeg_ctx_state - states of the context state machine
>    * @MTK_JPEG_INIT:		current state is initialized
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> index d868e46aaf37..fadfc4b5e366 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> @@ -274,6 +274,32 @@ void mtk_jpeg_dec_reset(void __iomem *base)
>   }
>   EXPORT_SYMBOL_GPL(mtk_jpeg_dec_reset);
>   
> +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> +void mtk_jpeg_dec_set_smmu_sid(int hwid)
> +{
> +	void __iomem *dec_reg_base;
> +	u32 val, mask;
> +
> +	if (hwid)
> +		dec_reg_base = ioremap(JPG_REG_CORE1_GUSER_ID, GUSER_ID_MAPRANGE);
> +	else
> +		dec_reg_base = ioremap(JPG_REG_CORE0_GUSER_ID, GUSER_ID_MAPRANGE);
> +	if (!dec_reg_base) {
> +		dev_err(jpeg->dev, "Failed to map jpgdec JPG_REG_GUSER_ID\n");
> +		return;
> +	}
> +
> +	val = ioread32(dec_reg_base);
> +	mask = ~(JPG_REG_GUSER_ID_MASK << JPG_REG_DEC_GUSER_ID_SHIFT);
> +	val &= mask;
> +	val |= (JPG_REG_GUSER_ID_DEC_SID << JPG_REG_DEC_GUSER_ID_SHIFT);
> +
> +	iowrite32(val, dec_reg_base);
> +	iounmap(dec_reg_base);
> +}
> +EXPORT_SYMBOL_GPL(mtk_jpeg_dec_set_smmu_sid);
> +#endif
> +
>   static void mtk_jpeg_dec_set_brz_factor(void __iomem *base, u8 yscale_w,
>   				u8 yscale_h, u8 uvscale_w, u8 uvscale_h)
>   {
> @@ -552,7 +578,6 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
>   	struct vb2_v4l2_buffer *src_buf, *dst_buf;
>   	struct mtk_jpeg_src_buf *jpeg_src_buf;
>   	enum vb2_buffer_state buf_state;
> -	struct mtk_jpeg_ctx *ctx;
>   	u32 dec_irq_ret;
>   	u32 irq_status;
>   	int i;
> @@ -562,7 +587,6 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
>   
>   	cancel_delayed_work(&jpeg->job_timeout_work);
>   
> -	ctx = jpeg->hw_param.curr_ctx;
>   	src_buf = jpeg->hw_param.src_buffer;
>   	dst_buf = jpeg->hw_param.dst_buffer;
>   	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> @@ -585,7 +609,7 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
>   	buf_state = VB2_BUF_STATE_DONE;
>   	v4l2_m2m_buf_done(src_buf, buf_state);
>   	mtk_jpegdec_put_buf(jpeg);
> -	pm_runtime_put(ctx->jpeg->dev);
> +	pm_runtime_put(jpeg->dev);

You're doing more than what you're describing in the commit title and description.

If this change was intentional, please move it to a diffrent commit.

Regards,
Angelo

Re: [V1,04/12] media: mediatek: jpeg: add jpeg smmu sid setting

[Kyrie Wu (吴晗)]

On Fri, 2025-01-10 at 10:39 +0100, AngeloGioacchino Del Regno wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Il 09/01/25 14:35, kyrie.wu ha scritto:
> > Add a configuration to set jpeg dec & enc smmu sid
> > 
> > Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> > ---
> >   .../platform/mediatek/jpeg/mtk_jpeg_core.c    |  6 ++++
> >   .../platform/mediatek/jpeg/mtk_jpeg_core.h    | 11 +++++++
> >   .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 30
> > +++++++++++++++++--
> >   .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |  4 +++
> >   .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 26
> > ++++++++++++++++
> >   .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |  4 +++
> >   6 files changed, 78 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index c3ccc525d9fd..77b3bd6c4d3f 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -1655,6 +1655,9 @@ static void mtk_jpegenc_worker(struct
> > work_struct *work)
> >       jpeg_dst_buf->frame_num = ctx->total_frame_num;
> >       ctx->total_frame_num++;
> >       mtk_jpeg_enc_reset(comp_jpeg[hw_id]->reg_base);
> > +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> > +     mtk_jpeg_enc_set_smmu_sid(hw_id);
> > +#endif
> >       mtk_jpeg_set_enc_dst(ctx,
> >                            comp_jpeg[hw_id]->reg_base,
> >                            &dst_buf->vb2_buf);
> > @@ -1771,6 +1774,9 @@ static void mtk_jpegdec_worker(struct
> > work_struct *work)
> >       spin_lock_irqsave(&comp_jpeg[hw_id]->hw_lock, flags);
> >       ctx->total_frame_num++;
> >       mtk_jpeg_dec_reset(comp_jpeg[hw_id]->reg_base);
> > +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> 
> You can avoid using preprocessor conditionals - and then, this
> doesn't look like an
> ARM SMMUv3 configuration, but rather a JPEG dec/enc HW config, so in
> that case the
> enclosing of this in the proposed config option would even be wrong.
> 
> Use platform data for that.
> 
> > +     mtk_jpeg_dec_set_smmu_sid(hw_id);
> > +#endif
> >       mtk_jpeg_dec_set_config(comp_jpeg[hw_id]->reg_base,
> >                               jpeg->variant->support_34bit,
> >                               &jpeg_src_buf->dec_param,
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > index 8fddc133c46c..d3aba1e6cae8 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > @@ -36,6 +36,17 @@
> > 
> >   #define MTK_JPEG_ADDR_MASK GENMASK(1, 0)
> > 
> > +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> > +#define JPG_REG_CORE0_GUSER_ID                          0x380d0000
> > +#define JPG_REG_CORE1_GUSER_ID                          0x388d0000
> 
> At least these two definitions shall go in the devicetree reg node.
> 
> > +#define JPG_REG_GUSER_ID_MASK                           0x7
> > +#define JPG_REG_GUSER_ID_DEC_SID                        0x4
> > +#define JPG_REG_GUSER_ID_ENC_SID                        0x5
> > +#define JPG_REG_DEC_GUSER_ID_SHIFT                      8
> > +#define JPG_REG_ENC_GUSER_ID_SHIFT                      4
> 
> If this is setting an IOMMU SID, then you can just use the "iommus"
> property
> to pass a handle to the IOMMU that this device is using plus the
> streamid,
> which you can retrieve and write to the guser_id_{dec,enc}_sid
> register.
> 
> There's no reason to hardcode that in this header, and actually
> hardcoding
> will give you a number of issues (example: unpowered/unclocked access
> to the
> IOMMU, and many others).
> 
> > +#define GUSER_ID_MAPRANGE                               4
> > +#endif
> > +
> >   /**
> >    * enum mtk_jpeg_ctx_state - states of the context state machine
> >    * @MTK_JPEG_INIT:          current state is initialized
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > index d868e46aaf37..fadfc4b5e366 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
> > @@ -274,6 +274,32 @@ void mtk_jpeg_dec_reset(void __iomem *base)
> >   }
> >   EXPORT_SYMBOL_GPL(mtk_jpeg_dec_reset);
> > 
> > +#if IS_ENABLED(CONFIG_ARM_SMMU_V3)
> > +void mtk_jpeg_dec_set_smmu_sid(int hwid)
> > +{
> > +     void __iomem *dec_reg_base;
> > +     u32 val, mask;
> > +
> > +     if (hwid)
> > +             dec_reg_base = ioremap(JPG_REG_CORE1_GUSER_ID,
> > GUSER_ID_MAPRANGE);
> > +     else
> > +             dec_reg_base = ioremap(JPG_REG_CORE0_GUSER_ID,
> > GUSER_ID_MAPRANGE);
> > +     if (!dec_reg_base) {
> > +             dev_err(jpeg->dev, "Failed to map jpgdec
> > JPG_REG_GUSER_ID\n");
> > +             return;
> > +     }
> > +
> > +     val = ioread32(dec_reg_base);
> > +     mask = ~(JPG_REG_GUSER_ID_MASK <<
> > JPG_REG_DEC_GUSER_ID_SHIFT);
> > +     val &= mask;
> > +     val |= (JPG_REG_GUSER_ID_DEC_SID <<
> > JPG_REG_DEC_GUSER_ID_SHIFT);
> > +
> > +     iowrite32(val, dec_reg_base);
> > +     iounmap(dec_reg_base);
> > +}
> > +EXPORT_SYMBOL_GPL(mtk_jpeg_dec_set_smmu_sid);
> > +#endif
> > +
> >   static void mtk_jpeg_dec_set_brz_factor(void __iomem *base, u8
> > yscale_w,
> >                               u8 yscale_h, u8 uvscale_w, u8
> > uvscale_h)
> >   {
> > @@ -552,7 +578,6 @@ static irqreturn_t
> > mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> >       struct vb2_v4l2_buffer *src_buf, *dst_buf;
> >       struct mtk_jpeg_src_buf *jpeg_src_buf;
> >       enum vb2_buffer_state buf_state;
> > -     struct mtk_jpeg_ctx *ctx;
> >       u32 dec_irq_ret;
> >       u32 irq_status;
> >       int i;
> > @@ -562,7 +587,6 @@ static irqreturn_t
> > mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> > 
> >       cancel_delayed_work(&jpeg->job_timeout_work);
> > 
> > -     ctx = jpeg->hw_param.curr_ctx;
> >       src_buf = jpeg->hw_param.src_buffer;
> >       dst_buf = jpeg->hw_param.dst_buffer;
> >       v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
> > @@ -585,7 +609,7 @@ static irqreturn_t
> > mtk_jpegdec_hw_irq_handler(int irq, void *priv)
> >       buf_state = VB2_BUF_STATE_DONE;
> >       v4l2_m2m_buf_done(src_buf, buf_state);
> >       mtk_jpegdec_put_buf(jpeg);
> > -     pm_runtime_put(ctx->jpeg->dev);
> > +     pm_runtime_put(jpeg->dev);
> 
> You're doing more than what you're describing in the commit title and
> description.
> 
> If this change was intentional, please move it to a diffrent commit.
> 
> Regards,
> Angelo

Dear Angelo,

Thanks for your comments, let's have a summary for them:
1. add smmu config enalbing flag in platform data.
2. move the hw smmu setting to devicetree and get it in probe func.
3. refine the patch.

I will fix ithem.

Thanks.

Regards,
Kyrie.

[V1,05/12] media: mediatek: jpeg: fix jpeg hw count setting

[kyrie.wu]

1. different IC has different hw core;
2. use a parameter to set jpeg hw count.

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++++++++++++++----
 .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  2 ++
 2 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 77b3bd6c4d3f..4dc6f82d74fa 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1468,7 +1468,7 @@ static int mtk_jpegenc_get_hw(struct mtk_jpeg_ctx *ctx)
 	int i;
 
 	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	for (i = 0; i < MTK_JPEGENC_HW_MAX; i++) {
+	for (i = 0; i < jpeg->variant->max_hw_count; i++) {
 		comp_jpeg = jpeg->enc_hw_dev[i];
 		if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
 			hw_id = i;
@@ -1515,7 +1515,7 @@ static int mtk_jpegdec_get_hw(struct mtk_jpeg_ctx *ctx)
 	int i;
 
 	spin_lock_irqsave(&jpeg->hw_lock, flags);
-	for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++) {
+	for (i = 0; i < jpeg->variant->max_hw_count; i++) {
 		comp_jpeg = jpeg->dec_hw_dev[i];
 		if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
 			hw_id = i;
@@ -1598,7 +1598,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 		jpeg_work);
 	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 
-	for (i = 0; i < MTK_JPEGENC_HW_MAX; i++)
+	for (i = 0; i < jpeg->variant->max_hw_count; i++)
 		comp_jpeg[i] = jpeg->enc_hw_dev[i];
 	i = 0;
 
@@ -1696,7 +1696,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 	struct mtk_jpeg_fb fb;
 	unsigned long flags;
 
-	for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++)
+	for (i = 0; i < jpeg->variant->max_hw_count; i++)
 		comp_jpeg[i] = jpeg->dec_hw_dev[i];
 	i = 0;
 
@@ -1925,6 +1925,7 @@ static struct mtk_jpeg_variant mtk8195_jpegenc_drvdata = {
 	.out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
 	.cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
 	.multi_core = true,
+	.max_hw_count = 2,
 	.jpeg_worker = mtk_jpegenc_worker,
 };
 
@@ -1938,6 +1939,21 @@ static const struct mtk_jpeg_variant mtk8195_jpegdec_drvdata = {
 	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
 	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
 	.multi_core = true,
+	.max_hw_count = 3,
+	.jpeg_worker = mtk_jpegdec_worker,
+};
+
+static const struct mtk_jpeg_variant mtk8196_jpegdec_drvdata = {
+	.formats = mtk_jpeg_dec_formats,
+	.num_formats = MTK_JPEG_DEC_NUM_FORMATS,
+	.qops = &mtk_jpeg_dec_qops,
+	.m2m_ops = &mtk_jpeg_multicore_dec_m2m_ops,
+	.dev_name = "mtk-jpeg-dec",
+	.ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
+	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
+	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
+	.multi_core = true,
+	.max_hw_count = 2,
 	.jpeg_worker = mtk_jpegdec_worker,
 };
 
@@ -1954,6 +1970,7 @@ static const struct mtk_jpeg_variant mtk8188_jpegenc_drvdata = {
 	.ioctl_ops = &mtk_jpeg_enc_ioctl_ops,
 	.out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
 	.cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
+	.max_hw_count = 1,
 	.support_34bit = true,
 };
 
@@ -1970,6 +1987,7 @@ static const struct mtk_jpeg_variant mtk8188_jpegdec_drvdata = {
 	.ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
 	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
 	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
+	.max_hw_count = 1,
 	.support_34bit = true,
 };
 
@@ -2008,7 +2026,7 @@ static const struct of_device_id mtk_jpeg_match[] = {
 	},
 	{
 		.compatible = "mediatek,mt8196-jpgdec",
-		.data = &mtk8195_jpegdec_drvdata,
+		.data = &mtk8196_jpegdec_drvdata,
 	},
 	{},
 };
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index d3aba1e6cae8..38672499ca18 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -74,6 +74,7 @@ enum mtk_jpeg_ctx_state {
  * @out_q_default_fourcc:	output queue default fourcc
  * @cap_q_default_fourcc:	capture queue default fourcc
  * @multi_core:		mark jpeg hw is multi_core or not
+ * @max_hw_count:		jpeg hw-core count
  * @jpeg_worker:		jpeg dec or enc worker
  * @support_34bit:	flag to check if support dma_address 34bit
  */
@@ -91,6 +92,7 @@ struct mtk_jpeg_variant {
 	u32 out_q_default_fourcc;
 	u32 cap_q_default_fourcc;
 	bool multi_core;
+	u32 max_hw_count;
 	void (*jpeg_worker)(struct work_struct *work);
 	bool support_34bit;
 };
-- 
2.46.0

Re: [V1,05/12] media: mediatek: jpeg: fix jpeg hw count setting

[AngeloGioacchino Del Regno]

Il 09/01/25 14:35, kyrie.wu ha scritto:
> 1. different IC has different hw core;
> 2. use a parameter to set jpeg hw count.

In MT8195, each decoder core has its own devicetree node, so you can count
how many cores are actually present and registered.

Please do the same with MT8196, and replace MTK_JPEG{ENC,DEC}_HW_MAX with a
variable that stores the number of cores, counted at probe time.

Regards,
Angelo

> 
> Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> ---
>   .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++++++++++++++----
>   .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  2 ++
>   2 files changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> index 77b3bd6c4d3f..4dc6f82d74fa 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> @@ -1468,7 +1468,7 @@ static int mtk_jpegenc_get_hw(struct mtk_jpeg_ctx *ctx)
>   	int i;
>   
>   	spin_lock_irqsave(&jpeg->hw_lock, flags);
> -	for (i = 0; i < MTK_JPEGENC_HW_MAX; i++) {
> +	for (i = 0; i < jpeg->variant->max_hw_count; i++) {
>   		comp_jpeg = jpeg->enc_hw_dev[i];
>   		if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
>   			hw_id = i;
> @@ -1515,7 +1515,7 @@ static int mtk_jpegdec_get_hw(struct mtk_jpeg_ctx *ctx)
>   	int i;
>   
>   	spin_lock_irqsave(&jpeg->hw_lock, flags);
> -	for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++) {
> +	for (i = 0; i < jpeg->variant->max_hw_count; i++) {
>   		comp_jpeg = jpeg->dec_hw_dev[i];
>   		if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
>   			hw_id = i;
> @@ -1598,7 +1598,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
>   		jpeg_work);
>   	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
>   
> -	for (i = 0; i < MTK_JPEGENC_HW_MAX; i++)
> +	for (i = 0; i < jpeg->variant->max_hw_count; i++)
>   		comp_jpeg[i] = jpeg->enc_hw_dev[i];
>   	i = 0;
>   
> @@ -1696,7 +1696,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
>   	struct mtk_jpeg_fb fb;
>   	unsigned long flags;
>   
> -	for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++)
> +	for (i = 0; i < jpeg->variant->max_hw_count; i++)
>   		comp_jpeg[i] = jpeg->dec_hw_dev[i];
>   	i = 0;
>   
> @@ -1925,6 +1925,7 @@ static struct mtk_jpeg_variant mtk8195_jpegenc_drvdata = {
>   	.out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
>   	.cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
>   	.multi_core = true,
> +	.max_hw_count = 2,
>   	.jpeg_worker = mtk_jpegenc_worker,
>   };
>   
> @@ -1938,6 +1939,21 @@ static const struct mtk_jpeg_variant mtk8195_jpegdec_drvdata = {
>   	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
>   	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
>   	.multi_core = true,
> +	.max_hw_count = 3,
> +	.jpeg_worker = mtk_jpegdec_worker,
> +};
> +
> +static const struct mtk_jpeg_variant mtk8196_jpegdec_drvdata = {
> +	.formats = mtk_jpeg_dec_formats,
> +	.num_formats = MTK_JPEG_DEC_NUM_FORMATS,
> +	.qops = &mtk_jpeg_dec_qops,
> +	.m2m_ops = &mtk_jpeg_multicore_dec_m2m_ops,
> +	.dev_name = "mtk-jpeg-dec",
> +	.ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
> +	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> +	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
> +	.multi_core = true,
> +	.max_hw_count = 2,
>   	.jpeg_worker = mtk_jpegdec_worker,
>   };
>   
> @@ -1954,6 +1970,7 @@ static const struct mtk_jpeg_variant mtk8188_jpegenc_drvdata = {
>   	.ioctl_ops = &mtk_jpeg_enc_ioctl_ops,
>   	.out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
>   	.cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> +	.max_hw_count = 1,
>   	.support_34bit = true,
>   };
>   
> @@ -1970,6 +1987,7 @@ static const struct mtk_jpeg_variant mtk8188_jpegdec_drvdata = {
>   	.ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
>   	.out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
>   	.cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
> +	.max_hw_count = 1,
>   	.support_34bit = true,
>   };
>   
> @@ -2008,7 +2026,7 @@ static const struct of_device_id mtk_jpeg_match[] = {
>   	},
>   	{
>   		.compatible = "mediatek,mt8196-jpgdec",
> -		.data = &mtk8195_jpegdec_drvdata,
> +		.data = &mtk8196_jpegdec_drvdata,
>   	},
>   	{},
>   };
> diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> index d3aba1e6cae8..38672499ca18 100644
> --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> @@ -74,6 +74,7 @@ enum mtk_jpeg_ctx_state {
>    * @out_q_default_fourcc:	output queue default fourcc
>    * @cap_q_default_fourcc:	capture queue default fourcc
>    * @multi_core:		mark jpeg hw is multi_core or not
> + * @max_hw_count:		jpeg hw-core count
>    * @jpeg_worker:		jpeg dec or enc worker
>    * @support_34bit:	flag to check if support dma_address 34bit
>    */
> @@ -91,6 +92,7 @@ struct mtk_jpeg_variant {
>   	u32 out_q_default_fourcc;
>   	u32 cap_q_default_fourcc;
>   	bool multi_core;
> +	u32 max_hw_count;
>   	void (*jpeg_worker)(struct work_struct *work);
>   	bool support_34bit;
>   };

Re: [V1,05/12] media: mediatek: jpeg: fix jpeg hw count setting

[Kyrie Wu (吴晗)]

On Fri, 2025-01-10 at 10:45 +0100, AngeloGioacchino Del Regno wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Il 09/01/25 14:35, kyrie.wu ha scritto:
> > 1. different IC has different hw core;
> > 2. use a parameter to set jpeg hw count.
> 
> In MT8195, each decoder core has its own devicetree node, so you can
> count
> how many cores are actually present and registered.
> 
> Please do the same with MT8196, and replace MTK_JPEG{ENC,DEC}_HW_MAX
> with a
> variable that stores the number of cores, counted at probe time.
> 
> Regards,
> Angelo

Dear Angelo,

I will fix it in the coming version.

Thanks.

Regards,
Kyrie.
> 
> > 
> > Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
> > ---
> >   .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28
> > +++++++++++++++----
> >   .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  2 ++
> >   2 files changed, 25 insertions(+), 5 deletions(-)
> > 
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > index 77b3bd6c4d3f..4dc6f82d74fa 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
> > @@ -1468,7 +1468,7 @@ static int mtk_jpegenc_get_hw(struct
> > mtk_jpeg_ctx *ctx)
> >       int i;
> > 
> >       spin_lock_irqsave(&jpeg->hw_lock, flags);
> > -     for (i = 0; i < MTK_JPEGENC_HW_MAX; i++) {
> > +     for (i = 0; i < jpeg->variant->max_hw_count; i++) {
> >               comp_jpeg = jpeg->enc_hw_dev[i];
> >               if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
> >                       hw_id = i;
> > @@ -1515,7 +1515,7 @@ static int mtk_jpegdec_get_hw(struct
> > mtk_jpeg_ctx *ctx)
> >       int i;
> > 
> >       spin_lock_irqsave(&jpeg->hw_lock, flags);
> > -     for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++) {
> > +     for (i = 0; i < jpeg->variant->max_hw_count; i++) {
> >               comp_jpeg = jpeg->dec_hw_dev[i];
> >               if (comp_jpeg->hw_state == MTK_JPEG_HW_IDLE) {
> >                       hw_id = i;
> > @@ -1598,7 +1598,7 @@ static void mtk_jpegenc_worker(struct
> > work_struct *work)
> >               jpeg_work);
> >       struct mtk_jpeg_dev *jpeg = ctx->jpeg;
> > 
> > -     for (i = 0; i < MTK_JPEGENC_HW_MAX; i++)
> > +     for (i = 0; i < jpeg->variant->max_hw_count; i++)
> >               comp_jpeg[i] = jpeg->enc_hw_dev[i];
> >       i = 0;
> > 
> > @@ -1696,7 +1696,7 @@ static void mtk_jpegdec_worker(struct
> > work_struct *work)
> >       struct mtk_jpeg_fb fb;
> >       unsigned long flags;
> > 
> > -     for (i = 0; i < MTK_JPEGDEC_HW_MAX; i++)
> > +     for (i = 0; i < jpeg->variant->max_hw_count; i++)
> >               comp_jpeg[i] = jpeg->dec_hw_dev[i];
> >       i = 0;
> > 
> > @@ -1925,6 +1925,7 @@ static struct mtk_jpeg_variant
> > mtk8195_jpegenc_drvdata = {
> >       .out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
> >       .cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> >       .multi_core = true,
> > +     .max_hw_count = 2,
> >       .jpeg_worker = mtk_jpegenc_worker,
> >   };
> > 
> > @@ -1938,6 +1939,21 @@ static const struct mtk_jpeg_variant
> > mtk8195_jpegdec_drvdata = {
> >       .out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> >       .cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
> >       .multi_core = true,
> > +     .max_hw_count = 3,
> > +     .jpeg_worker = mtk_jpegdec_worker,
> > +};
> > +
> > +static const struct mtk_jpeg_variant mtk8196_jpegdec_drvdata = {
> > +     .formats = mtk_jpeg_dec_formats,
> > +     .num_formats = MTK_JPEG_DEC_NUM_FORMATS,
> > +     .qops = &mtk_jpeg_dec_qops,
> > +     .m2m_ops = &mtk_jpeg_multicore_dec_m2m_ops,
> > +     .dev_name = "mtk-jpeg-dec",
> > +     .ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
> > +     .out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> > +     .cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
> > +     .multi_core = true,
> > +     .max_hw_count = 2,
> >       .jpeg_worker = mtk_jpegdec_worker,
> >   };
> > 
> > @@ -1954,6 +1970,7 @@ static const struct mtk_jpeg_variant
> > mtk8188_jpegenc_drvdata = {
> >       .ioctl_ops = &mtk_jpeg_enc_ioctl_ops,
> >       .out_q_default_fourcc = V4L2_PIX_FMT_YUYV,
> >       .cap_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> > +     .max_hw_count = 1,
> >       .support_34bit = true,
> >   };
> > 
> > @@ -1970,6 +1987,7 @@ static const struct mtk_jpeg_variant
> > mtk8188_jpegdec_drvdata = {
> >       .ioctl_ops = &mtk_jpeg_dec_ioctl_ops,
> >       .out_q_default_fourcc = V4L2_PIX_FMT_JPEG,
> >       .cap_q_default_fourcc = V4L2_PIX_FMT_YUV420M,
> > +     .max_hw_count = 1,
> >       .support_34bit = true,
> >   };
> > 
> > @@ -2008,7 +2026,7 @@ static const struct of_device_id
> > mtk_jpeg_match[] = {
> >       },
> >       {
> >               .compatible = "mediatek,mt8196-jpgdec",
> > -             .data = &mtk8195_jpegdec_drvdata,
> > +             .data = &mtk8196_jpegdec_drvdata,
> >       },
> >       {},
> >   };
> > diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > index d3aba1e6cae8..38672499ca18 100644
> > --- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > +++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
> > @@ -74,6 +74,7 @@ enum mtk_jpeg_ctx_state {
> >    * @out_q_default_fourcc:   output queue default fourcc
> >    * @cap_q_default_fourcc:   capture queue default fourcc
> >    * @multi_core:             mark jpeg hw is multi_core or not
> > + * @max_hw_count:            jpeg hw-core count
> >    * @jpeg_worker:            jpeg dec or enc worker
> >    * @support_34bit:  flag to check if support dma_address 34bit
> >    */
> > @@ -91,6 +92,7 @@ struct mtk_jpeg_variant {
> >       u32 out_q_default_fourcc;
> >       u32 cap_q_default_fourcc;
> >       bool multi_core;
> > +     u32 max_hw_count;
> >       void (*jpeg_worker)(struct work_struct *work);
> >       bool support_34bit;
> >   };
> 
> 

[V1,06/12] media: mediatek: jpeg: refactor jpeg buffer payload setting

[kyrie.wu]

1. for multi-core jpegdec:
   core0: |<-------- decoding buffer0 and resolution changed to smaller
   core1:  |<-------- decoding buffer1
   core0:   |<- handling resolution changing
   core0:    |<- vb2_set_plane_payload
2. the payload size is changed on the step of set format. Because core1
is running and streaming has not been stopped, the format cannot be
set again, resulting in no change in the payload size.
3. at this time, the payload size is bigger than buffer length,
it will print a warnning call trace
4. set payload size must less than buffer length

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 .../media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 4dc6f82d74fa..83f9451fd3ba 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -719,10 +719,18 @@ static int mtk_jpeg_buf_prepare(struct vb2_buffer *vb)
 		plane_fmt = q_data->pix_mp.plane_fmt[i];
 		if (ctx->enable_exif &&
 		    q_data->fmt->fourcc == V4L2_PIX_FMT_JPEG)
-			vb2_set_plane_payload(vb, i, plane_fmt.sizeimage +
-					      MTK_JPEG_MAX_EXIF_SIZE);
+			if (vb->planes[i].length > (plane_fmt.sizeimage +
+			    MTK_JPEG_MAX_EXIF_SIZE))
+				vb2_set_plane_payload(vb, i, plane_fmt.sizeimage +
+						      MTK_JPEG_MAX_EXIF_SIZE);
+			else
+				vb2_set_plane_payload(vb, i, vb->planes[i].length);
+
 		else
-			vb2_set_plane_payload(vb, i,  plane_fmt.sizeimage);
+			if (vb->planes[i].length > plane_fmt.sizeimage)
+				vb2_set_plane_payload(vb, i, plane_fmt.sizeimage);
+			else
+				vb2_set_plane_payload(vb, i, vb->planes[i].length);
 	}
 
 	return 0;
-- 
2.46.0

[V1,07/12] media: mediatek: jpeg: refactor jpeg dst buffer layout

[kyrie.wu]

1. change dst buffer size to same as struct mtk_jpeg_src_buf
to make sure all params of mtk_jpeg_src_buf could get a memory.
2. For memory alloc operation:
the v4l2 framework malloc a memory, the base addr is vb2_buffer and
the size is sizeof(struct mtk_jpeg_src_buf), mtk_jpeg_src_buf could get
itself addr by container_of like that:
vb2_buffer -> vb2_v4l2_buffer -> mtk_jpeg_src_buf.
vb2_v4l2_buffer must keep on the top of mtk_jpeg_src_buf.

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 2 +-
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 83f9451fd3ba..3082d0fcf974 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1098,7 +1098,7 @@ static int mtk_jpeg_queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	dst_vq->io_modes = VB2_DMABUF | VB2_MMAP;
 	dst_vq->drv_priv = ctx;
-	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->buf_struct_size = sizeof(struct mtk_jpeg_src_buf);
 	dst_vq->ops = jpeg->variant->qops;
 	dst_vq->mem_ops = &vb2_dma_contig_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index 38672499ca18..ce9f14e711f5 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -98,10 +98,10 @@ struct mtk_jpeg_variant {
 };
 
 struct mtk_jpeg_src_buf {
-	u32 frame_num;
 	struct vb2_v4l2_buffer b;
 	struct list_head list;
 	u32 bs_size;
+	u32 frame_num;
 	struct mtk_jpeg_dec_param dec_param;
 
 	struct mtk_jpeg_ctx *curr_ctx;
-- 
2.46.0

[V1,08/12] media: mediatek: jpeg: fix stop streaming flow for multi-core

[kyrie.wu]

1. For multi-core jpegdec, the all hws may run at the same time,
if one hw decoded firstly, the function of mtk_jpeg_dec_stop_streaming
would be called, but others input buffers are decoding, this will
cause some running buffers to be buffer done, causing errors;
2. add a parameter to calculate the decoding buffer counts, it
wil decrease to 0 until the all buffers decoded and the
mtk_jpeg_dec_stop_streaming could continue to be executed.

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 .../media/platform/mediatek/jpeg/mtk_jpeg_core.c | 16 ++++++++++++++++
 .../media/platform/mediatek/jpeg/mtk_jpeg_core.h |  2 ++
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c     | 12 ++++++++++++
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c     | 11 ++++++++++-
 4 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 3082d0fcf974..9556accacfa0 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -856,8 +856,12 @@ static struct vb2_v4l2_buffer *mtk_jpeg_buf_remove(struct mtk_jpeg_ctx *ctx,
 static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct vb2_v4l2_buffer *vb;
 
+	if (jpeg->variant->multi_core)
+		wait_event(jpeg->hw_wq, (atomic_read(&ctx->buf_list_cnt) == 0));
+
 	while ((vb = mtk_jpeg_buf_remove(ctx, q->type)))
 		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
 }
@@ -865,6 +869,7 @@ static void mtk_jpeg_enc_stop_streaming(struct vb2_queue *q)
 static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 {
 	struct mtk_jpeg_ctx *ctx = vb2_get_drv_priv(q);
+	struct mtk_jpeg_dev *jpeg = ctx->jpeg;
 	struct vb2_v4l2_buffer *vb;
 
 	/*
@@ -872,6 +877,9 @@ static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 	 * Before STREAMOFF, we still have to return the old resolution and
 	 * subsampling. Update capture queue when the stream is off.
 	 */
+	if (jpeg->variant->multi_core)
+		wait_event(jpeg->hw_wq, (atomic_read(&ctx->buf_list_cnt) == 0));
+
 	if (ctx->state == MTK_JPEG_SOURCE_CHANGE &&
 	    V4L2_TYPE_IS_CAPTURE(q->type)) {
 		struct mtk_jpeg_src_buf *src_buf;
@@ -1181,6 +1189,7 @@ static int mtk_jpeg_open(struct file *file)
 	v4l2_fh_init(&ctx->fh, vfd);
 	file->private_data = &ctx->fh;
 	v4l2_fh_add(&ctx->fh);
+	atomic_set(&ctx->buf_list_cnt, 0);
 
 	ctx->jpeg = jpeg;
 	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpeg->m2m_dev, ctx,
@@ -1563,6 +1572,11 @@ static int mtk_jpegdec_set_hw_param(struct mtk_jpeg_ctx *ctx,
 	return 0;
 }
 
+static void jpeg_buf_queue_inc(struct mtk_jpeg_ctx *ctx)
+{
+	atomic_inc(&ctx->buf_list_cnt);
+}
+
 static irqreturn_t mtk_jpeg_enc_done(struct mtk_jpeg_dev *jpeg)
 {
 	struct mtk_jpeg_ctx *ctx;
@@ -1674,6 +1688,7 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 			     &src_buf->vb2_buf);
 	mtk_jpeg_set_enc_params(ctx, comp_jpeg[hw_id]->reg_base);
 	mtk_jpeg_enc_start(comp_jpeg[hw_id]->reg_base);
+	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
 
@@ -1792,6 +1807,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 				&bs,
 				&fb);
 	mtk_jpeg_dec_start(comp_jpeg[hw_id]->reg_base);
+	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
index ce9f14e711f5..ce884c190ce5 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.h
@@ -299,6 +299,7 @@ struct mtk_jpeg_q_data {
  * @dst_done_queue:		encoded frame buffer queue
  * @done_queue_lock:		encoded frame operation spinlock
  * @last_done_frame_num:	the last encoded frame number
+ * @buf_list_cnt:		the frame buffer count own by jpeg driver
  */
 struct mtk_jpeg_ctx {
 	struct mtk_jpeg_dev		*jpeg;
@@ -317,6 +318,7 @@ struct mtk_jpeg_ctx {
 	/* spinlock protecting the encode done buffer */
 	spinlock_t done_queue_lock;
 	u32 last_done_frame_num;
+	atomic_t buf_list_cnt;
 };
 
 #endif /* _MTK_JPEG_CORE_H */
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index fadfc4b5e366..37c015236524 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -550,6 +550,11 @@ static void mtk_jpegdec_put_buf(struct mtk_jpegdec_comp_dev *jpeg)
 	spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
 }
 
+static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
+{
+	atomic_dec(&ctx->buf_list_cnt);
+}
+
 static void mtk_jpegdec_timeout_work(struct work_struct *work)
 {
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
@@ -558,9 +563,11 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
 			     job_timeout_work.work);
 	struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_ctx *ctx;
 
 	src_buf = cjpeg->hw_param.src_buffer;
 	dst_buf = cjpeg->hw_param.dst_buffer;
+	ctx = cjpeg->hw_param.curr_ctx;
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_dec_reset(cjpeg->reg_base);
@@ -571,6 +578,7 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(cjpeg);
+	jpeg_buf_queue_dec(ctx);
 }
 
 static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
@@ -578,12 +586,14 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
 	struct mtk_jpeg_src_buf *jpeg_src_buf;
 	enum vb2_buffer_state buf_state;
+	struct mtk_jpeg_ctx *ctx;
 	u32 dec_irq_ret;
 	u32 irq_status;
 	int i;
 
 	struct mtk_jpegdec_comp_dev *jpeg = priv;
 	struct mtk_jpeg_dev *master_jpeg = jpeg->master_dev;
+	ctx = jpeg->hw_param.curr_ctx;
 
 	cancel_delayed_work(&jpeg->job_timeout_work);
 
@@ -609,6 +619,8 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	buf_state = VB2_BUF_STATE_DONE;
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(jpeg);
+	jpeg_buf_queue_dec(ctx);
+
 	pm_runtime_put(jpeg->dev);
 	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index fc7676d13adb..c7d98f8aabfa 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -290,6 +290,11 @@ static void mtk_jpegenc_put_buf(struct mtk_jpegenc_comp_dev *jpeg)
 	spin_unlock_irqrestore(&ctx->done_queue_lock, flags);
 }
 
+static void jpeg_buf_queue_dec(struct mtk_jpeg_ctx *ctx)
+{
+	atomic_dec(&ctx->buf_list_cnt);
+}
+
 static void mtk_jpegenc_timeout_work(struct work_struct *work)
 {
 	struct delayed_work *dly_work = to_delayed_work(work);
@@ -300,9 +305,11 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
 	struct mtk_jpeg_dev *master_jpeg = cjpeg->master_dev;
 	enum vb2_buffer_state buf_state = VB2_BUF_STATE_ERROR;
 	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct mtk_jpeg_ctx *ctx;
 
 	src_buf = cjpeg->hw_param.src_buffer;
 	dst_buf = cjpeg->hw_param.dst_buffer;
+	ctx = cjpeg->hw_param.curr_ctx;
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_enc_reset(cjpeg->reg_base);
@@ -313,6 +320,7 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(cjpeg);
+	jpeg_buf_queue_dec(ctx);
 }
 
 static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
@@ -346,7 +354,8 @@ static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
 	buf_state = VB2_BUF_STATE_DONE;
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(jpeg);
-	pm_runtime_put(ctx->jpeg->dev);
+	jpeg_buf_queue_dec(ctx);
+	pm_runtime_put(jpeg->dev);
 	clk_disable_unprepare(jpeg->venc_clk.clks->clk);
 
 	jpeg->hw_state = MTK_JPEG_HW_IDLE;
-- 
2.46.0

[V1,09/12] media: mediatek: jpeg: refactor multi-core clk suspend and resume setting

[kyrie.wu]

refactor jpeg clk suspend and resume setting for multi-core

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 28 +++----
 .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 75 +++++++++++++++++--
 .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 73 +++++++++++++++++-
 3 files changed, 147 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 9556accacfa0..48beb97015bf 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1121,6 +1121,9 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 {
 	int ret;
 
+	if (jpeg->variant->multi_core)
+		return;
+
 	ret = clk_bulk_prepare_enable(jpeg->variant->num_clks,
 				      jpeg->variant->clks);
 	if (ret)
@@ -1129,6 +1132,9 @@ static void mtk_jpeg_clk_on(struct mtk_jpeg_dev *jpeg)
 
 static void mtk_jpeg_clk_off(struct mtk_jpeg_dev *jpeg)
 {
+	if (jpeg->variant->multi_core)
+		return;
+
 	clk_bulk_disable_unprepare(jpeg->variant->num_clks,
 				   jpeg->variant->clks);
 }
@@ -1658,13 +1664,6 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 		goto enc_end;
 	}
 
-	ret = clk_prepare_enable(comp_jpeg[hw_id]->venc_clk.clks->clk);
-	if (ret) {
-		dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
-			__func__, __LINE__);
-		goto enc_end;
-	}
-
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 
@@ -1765,20 +1764,13 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 	jpeg_dst_buf->frame_num = ctx->total_frame_num;
 
 	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
-	ret = pm_runtime_get_sync(comp_jpeg[hw_id]->dev);
+	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
 	if (ret < 0) {
 		dev_err(jpeg->dev, "%s : %d, pm_runtime_get_sync fail !!!\n",
 			__func__, __LINE__);
 		goto dec_end;
 	}
 
-	ret = clk_prepare_enable(comp_jpeg[hw_id]->jdec_clk.clks->clk);
-	if (ret) {
-		dev_err(jpeg->dev, "%s : %d, jpegdec clk_prepare_enable fail\n",
-			__func__, __LINE__);
-		goto clk_end;
-	}
-
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 
@@ -1788,7 +1780,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 				 &dst_buf->vb2_buf, &fb)) {
 		dev_err(jpeg->dev, "%s : %d, mtk_jpeg_set_dec_dst fail\n",
 			__func__, __LINE__);
-		goto setdst_end;
+		goto set_dst_fail;
 	}
 
 	schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
@@ -1813,9 +1805,7 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 
 	return;
 
-setdst_end:
-	clk_disable_unprepare(comp_jpeg[hw_id]->jdec_clk.clks->clk);
-clk_end:
+set_dst_fail:
 	pm_runtime_put(comp_jpeg[hw_id]->dev);
 dec_end:
 	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index 37c015236524..2c540d2c9c49 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -571,14 +571,13 @@ static void mtk_jpegdec_timeout_work(struct work_struct *work)
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_dec_reset(cjpeg->reg_base);
-	clk_disable_unprepare(cjpeg->jdec_clk.clks->clk);
-	pm_runtime_put(cjpeg->dev);
 	cjpeg->hw_state = MTK_JPEG_HW_IDLE;
 	atomic_inc(&master_jpeg->hw_rdy);
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(cjpeg);
 	jpeg_buf_queue_dec(ctx);
+	pm_runtime_put(cjpeg->dev);
 }
 
 static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
@@ -620,13 +619,10 @@ static irqreturn_t mtk_jpegdec_hw_irq_handler(int irq, void *priv)
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegdec_put_buf(jpeg);
 	jpeg_buf_queue_dec(ctx);
-
-	pm_runtime_put(jpeg->dev);
-	clk_disable_unprepare(jpeg->jdec_clk.clks->clk);
-
 	jpeg->hw_state = MTK_JPEG_HW_IDLE;
 	wake_up(&master_jpeg->hw_wq);
 	atomic_inc(&master_jpeg->hw_rdy);
+	pm_runtime_put(jpeg->dev);
 
 	return IRQ_HANDLED;
 }
@@ -708,15 +704,82 @@ static int mtk_jpegdec_hw_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, dev);
 	pm_runtime_enable(&pdev->dev);
+	ret = devm_clk_bulk_get(dev->dev,
+				jpegdec_clk->clk_num,
+				jpegdec_clk->clks);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to init clk\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static void mtk_jpeg_clk_on(struct mtk_jpegdec_comp_dev *jpeg)
+{
+	int ret;
+
+	ret = clk_bulk_prepare_enable(jpeg->jdec_clk.clk_num, jpeg->jdec_clk.clks);
+	if (ret)
+		dev_err(jpeg->dev, "%s : %d, jpegdec clk_prepare_enable fail\n",
+			__func__, __LINE__);
+}
+
+static void mtk_jpeg_clk_off(struct mtk_jpegdec_comp_dev *jpeg)
+{
+	clk_bulk_disable_unprepare(jpeg->jdec_clk.clk_num, jpeg->jdec_clk.clks);
+}
+
+static __maybe_unused int mtk_jpegdec_pm_suspend(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_off(jpeg);
 
 	return 0;
 }
 
+static __maybe_unused int mtk_jpegdec_pm_resume(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_on(jpeg);
+
+	return 0;
+}
+
+static __maybe_unused int mtk_jpegdec_suspend(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
+	return pm_runtime_force_suspend(dev);
+}
+
+static __maybe_unused int mtk_jpegdec_resume(struct device *dev)
+{
+	struct mtk_jpegdec_comp_dev *jpeg = dev_get_drvdata(dev);
+	int ret;
+
+	ret = pm_runtime_force_resume(dev);
+	if (ret < 0)
+		return ret;
+
+	v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
+	return ret;
+}
+
+static const struct dev_pm_ops mtk_jpegdec_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegdec_suspend, mtk_jpegdec_resume)
+	SET_RUNTIME_PM_OPS(mtk_jpegdec_pm_suspend, mtk_jpegdec_pm_resume, NULL)
+};
+
 static struct platform_driver mtk_jpegdec_hw_driver = {
 	.probe = mtk_jpegdec_hw_probe,
 	.driver = {
 		.name = "mtk-jpegdec-hw",
 		.of_match_table = mtk_jpegdec_hw_ids,
+		.pm             = &mtk_jpegdec_pm_ops,
 	},
 };
 
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index c7d98f8aabfa..2fe50e52caeb 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -313,14 +313,13 @@ static void mtk_jpegenc_timeout_work(struct work_struct *work)
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 
 	mtk_jpeg_enc_reset(cjpeg->reg_base);
-	clk_disable_unprepare(cjpeg->venc_clk.clks->clk);
-	pm_runtime_put(cjpeg->dev);
 	cjpeg->hw_state = MTK_JPEG_HW_IDLE;
 	atomic_inc(&master_jpeg->hw_rdy);
 	wake_up(&master_jpeg->hw_wq);
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(cjpeg);
 	jpeg_buf_queue_dec(ctx);
+	pm_runtime_put(cjpeg->dev);
 }
 
 static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
@@ -355,12 +354,11 @@ static irqreturn_t mtk_jpegenc_hw_irq_handler(int irq, void *priv)
 	v4l2_m2m_buf_done(src_buf, buf_state);
 	mtk_jpegenc_put_buf(jpeg);
 	jpeg_buf_queue_dec(ctx);
-	pm_runtime_put(jpeg->dev);
-	clk_disable_unprepare(jpeg->venc_clk.clks->clk);
 
 	jpeg->hw_state = MTK_JPEG_HW_IDLE;
 	wake_up(&master_jpeg->hw_wq);
 	atomic_inc(&master_jpeg->hw_rdy);
+	pm_runtime_put(jpeg->dev);
 
 	return IRQ_HANDLED;
 }
@@ -440,15 +438,82 @@ static int mtk_jpegenc_hw_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, dev);
 	pm_runtime_enable(&pdev->dev);
+	ret = devm_clk_bulk_get(dev->dev,
+				jpegenc_clk->clk_num,
+				jpegenc_clk->clks);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to init clk\n");
+		return ret;
+	}
 
 	return 0;
 }
 
+static void mtk_jpeg_clk_on(struct mtk_jpegenc_comp_dev *jpeg)
+{
+	int ret;
+
+	ret = clk_bulk_prepare_enable(jpeg->venc_clk.clk_num, jpeg->venc_clk.clks);
+	if (ret)
+		dev_err(jpeg->dev, "%s : %d, jpegenc clk_prepare_enable fail\n",
+			__func__, __LINE__);
+}
+
+static void mtk_jpeg_clk_off(struct mtk_jpegenc_comp_dev *jpeg)
+{
+	clk_bulk_disable_unprepare(jpeg->venc_clk.clk_num, jpeg->venc_clk.clks);
+}
+
+static __maybe_unused int mtk_jpegenc_pm_suspend(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_off(jpeg);
+
+	return 0;
+}
+
+static __maybe_unused int mtk_jpegenc_pm_resume(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	mtk_jpeg_clk_on(jpeg);
+
+	return 0;
+}
+
+static __maybe_unused int mtk_jpegenc_suspend(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+
+	v4l2_m2m_suspend(jpeg->master_dev->m2m_dev);
+	return pm_runtime_force_suspend(dev);
+}
+
+static __maybe_unused int mtk_jpegenc_resume(struct device *dev)
+{
+	struct mtk_jpegenc_comp_dev *jpeg = dev_get_drvdata(dev);
+	int ret;
+
+	ret = pm_runtime_force_resume(dev);
+	if (ret < 0)
+		return ret;
+
+	v4l2_m2m_resume(jpeg->master_dev->m2m_dev);
+	return ret;
+}
+
+static const struct dev_pm_ops mtk_jpegenc_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(mtk_jpegenc_suspend, mtk_jpegenc_resume)
+	SET_RUNTIME_PM_OPS(mtk_jpegenc_pm_suspend, mtk_jpegenc_pm_resume, NULL)
+};
+
 static struct platform_driver mtk_jpegenc_hw_driver = {
 	.probe = mtk_jpegenc_hw_probe,
 	.driver = {
 		.name = "mtk-jpegenc-hw",
 		.of_match_table = mtk_jpegenc_drv_ids,
+		.pm = &mtk_jpegenc_pm_ops,
 	},
 };
 
-- 
2.46.0

[V1,10/12] media: mediatek: jpeg: fix decoding buffer number setting timing issue

[kyrie.wu]

move decoding buffer increase code into spinlock
protecting aera for multi-core

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c   | 9 +++------
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c | 1 +
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c | 1 +
 3 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 48beb97015bf..791fc064bc0a 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1749,7 +1749,6 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, true);
 	jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(&src_buf->vb2_buf);
-	jpeg_dst_buf = mtk_jpeg_vb2_to_srcbuf(&dst_buf->vb2_buf);
 
 	if (mtk_jpeg_check_resolution_change(ctx,
 					     &jpeg_src_buf->dec_param)) {
@@ -1758,11 +1757,6 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 		goto getbuf_fail;
 	}
 
-	jpeg_src_buf->curr_ctx = ctx;
-	jpeg_src_buf->frame_num = ctx->total_frame_num;
-	jpeg_dst_buf->curr_ctx = ctx;
-	jpeg_dst_buf->frame_num = ctx->total_frame_num;
-
 	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
 	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
 	if (ret < 0) {
@@ -1787,6 +1781,9 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 			      msecs_to_jiffies(MTK_JPEG_HW_TIMEOUT_MSEC));
 
 	spin_lock_irqsave(&comp_jpeg[hw_id]->hw_lock, flags);
+	jpeg_dst_buf = mtk_jpeg_vb2_to_srcbuf(&dst_buf->vb2_buf);
+	jpeg_dst_buf->curr_ctx = ctx;
+	jpeg_dst_buf->frame_num = ctx->total_frame_num;
 	ctx->total_frame_num++;
 	mtk_jpeg_dec_reset(comp_jpeg[hw_id]->reg_base);
 #if IS_ENABLED(CONFIG_ARM_SMMU_V3)
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
index 2c540d2c9c49..6467b9bec595 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_dec_hw.c
@@ -544,6 +544,7 @@ static void mtk_jpegdec_put_buf(struct mtk_jpegdec_comp_dev *jpeg)
 				v4l2_m2m_buf_done(&tmp_dst_done_buf->b,
 						  VB2_BUF_STATE_DONE);
 				ctx->last_done_frame_num++;
+				break;
 			}
 		}
 	}
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
index 2fe50e52caeb..ea3c909ab250 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_enc_hw.c
@@ -284,6 +284,7 @@ static void mtk_jpegenc_put_buf(struct mtk_jpegenc_comp_dev *jpeg)
 				v4l2_m2m_buf_done(&tmp_dst_done_buf->b,
 						  VB2_BUF_STATE_DONE);
 				ctx->last_done_frame_num++;
+				break;
 			}
 		}
 	}
-- 
2.46.0

[V1,11/12] media: mediatek: jpeg: refactor decoding resolution change operation

[kyrie.wu]

For multi-core:
1.if there are only two src buffers, hw0 and hw1 are decoding
at the same time. When resolution change occurred on one hw,
the src buffer cannot be got in the stop streaming function,
which will cause a crash.
2.When a resolution change occurs, immediately set the new
resolution parameter to save the new resolution;
3.After a resolution change occurred, decoding should not continue,
needs to wait until new buffers are ready and the state machine
changed.

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 791fc064bc0a..2919fdd92d45 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -886,7 +886,8 @@ static void mtk_jpeg_dec_stop_streaming(struct vb2_queue *q)
 
 		vb = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 		src_buf = mtk_jpeg_vb2_to_srcbuf(&vb->vb2_buf);
-		mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
+		if (!IS_ERR_OR_NULL(src_buf))
+			mtk_jpeg_set_queue_data(ctx, &src_buf->dec_param);
 		ctx->state = MTK_JPEG_RUNNING;
 	} else if (V4L2_TYPE_IS_OUTPUT(q->type)) {
 		ctx->state = MTK_JPEG_INIT;
@@ -1752,11 +1753,15 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 
 	if (mtk_jpeg_check_resolution_change(ctx,
 					     &jpeg_src_buf->dec_param)) {
-		mtk_jpeg_queue_src_chg_event(ctx);
+		mtk_jpeg_set_queue_data(ctx, &jpeg_src_buf->dec_param);
 		ctx->state = MTK_JPEG_SOURCE_CHANGE;
+		mtk_jpeg_queue_src_chg_event(ctx);
 		goto getbuf_fail;
 	}
 
+	if (ctx->state == MTK_JPEG_SOURCE_CHANGE)
+		goto getbuf_fail;
+
 	mtk_jpegdec_set_hw_param(ctx, hw_id, src_buf, dst_buf);
 	ret = pm_runtime_resume_and_get(comp_jpeg[hw_id]->dev);
 	if (ret < 0) {
-- 
2.46.0

[V1,12/12] media: mediatek: jpeg: fix remove buffer operation for multi-core

[kyrie.wu]

move remove buffer code to spinlock protect area for multi-core

Signed-off-by: kyrie.wu <kyrie.wu@mediatek.com>
---
 drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 2919fdd92d45..065b35122acf 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -1665,9 +1665,6 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 		goto enc_end;
 	}
 
-	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
-	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
-
 	schedule_delayed_work(&comp_jpeg[hw_id]->job_timeout_work,
 			      msecs_to_jiffies(MTK_JPEG_HW_TIMEOUT_MSEC));
 
@@ -1688,6 +1685,8 @@ static void mtk_jpegenc_worker(struct work_struct *work)
 			     &src_buf->vb2_buf);
 	mtk_jpeg_set_enc_params(ctx, comp_jpeg[hw_id]->reg_base);
 	mtk_jpeg_enc_start(comp_jpeg[hw_id]->reg_base);
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
 	spin_unlock_irqrestore(&comp_jpeg[hw_id]->hw_lock, flags);
@@ -1770,9 +1769,6 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 		goto dec_end;
 	}
 
-	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
-	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
-
 	mtk_jpeg_set_dec_src(ctx, &src_buf->vb2_buf, &bs);
 	if (mtk_jpeg_set_dec_dst(ctx,
 				 &jpeg_src_buf->dec_param,
@@ -1800,6 +1796,8 @@ static void mtk_jpegdec_worker(struct work_struct *work)
 				jpeg_src_buf->bs_size,
 				&bs,
 				&fb);
+	v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	mtk_jpeg_dec_start(comp_jpeg[hw_id]->reg_base);
 	jpeg_buf_queue_inc(ctx);
 	v4l2_m2m_job_finish(jpeg->m2m_dev, ctx->fh.m2m_ctx);
-- 
2.46.0

Re: [V1,00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[Nicolas Dufresne]

Hi,

Le jeudi 09 janvier 2025 à 21:35 +0800, kyrie.wu a écrit :
> This series adds support for mt8196 multi-hardwares jpeg enc & dec,
> by first adding mt8196 jpegdec and jpegenc compatible to install
> kernel driver. Add smmu setting to support smmu and iommu at the
> same time.
> Secondly refactor buffer and clock setting to support multi-hw jpeg
> working.
> Lastly, fix some bugs, including resolution change handleing, stop
> streaming sw flow and others.
> 
> This series has been tested with MT8196 tast test.
> Encoding and decoding worked for this chip.
> 
> Patches 1-3 Adds jpeg encoder and decoder compatible.
> Patches 4 add jpeg smmu sid setting.
> Patches 5 fix jpeg hw count setting to support different chips.
> Patches 6 refactor jpeg buffer payload setting to handle buffer
> size bug while resolution changed.
> Patches 7 reconstruct jpeg dst buffer layout.
> Patches 8 fix multi-core stop streaming flow
> Patches 9 refactor multi-core clk suspend/resume setting
> Patches 10 fix decoding buffer number setting timing issue
> Patches 11 refactor decoding resolution change operation
> Patches 12 fix remove buffer operation
> 
> ---
> This series patches dependent on:
> [1]
> https://patchwork.kernel.org/project/linux-mediatek/patch/20240808092555.12999-1-jianhua.lin@mediatek.com/
> 
> kyrie.wu (11):

would you mind fixing the --author in your configuration, write your name
instead of your user name. UT8 is allowed, many people will use a ASCI spelling
(some approximation) and then add their real name in parenthesis.

regards,
Nicolas

>   dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible
>   dt-bindings: mediatek: Add mediatek, mt8196-jpgenc compatible
>   media: mediatek: jpeg: add jpeg compatible
>   media: mediatek: jpeg: add jpeg smmu sid setting
>   media: mediatek: jpeg: fix jpeg hw count setting
>   media: mediatek: jpeg: refactor jpeg buffer payload setting
>   media: mediatek: jpeg: refactor jpeg dst buffer layout
>   media: mediatek: jpeg: fix stop streaming flow for multi-core
>   media: mediatek: jpeg: refactor multi-core clk suspend and resume
>     setting
>   media: mediatek: jpeg: fix decoding buffer number setting timing issue
>   media: mediatek: jpeg: refactor decoding resolution change operation
>   media: mediatek: jpeg: fix remove buffer operation for multi-core
> 
>  ....yaml => mediatek,multi-core-jpegdec.yaml} |  10 +-
>  ....yaml => mediatek,multi-core-jpegenc.yaml} |  10 +-
>  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 126 ++++++++++++------
>  .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  17 ++-
>  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 116 +++++++++++++++-
>  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |   4 +
>  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 113 +++++++++++++++-
>  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |   4 +
>  8 files changed, 343 insertions(+), 57 deletions(-)
>  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)
>  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-jpegenc.yaml => mediatek,multi-core-jpegenc.yaml} (94%)
> 

Re: [V1,00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[Kyrie Wu (吴晗)]

On Thu, 2025-01-09 at 09:06 -0500, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi,
> 
> Le jeudi 09 janvier 2025 à 21:35 +0800, kyrie.wu a écrit :
> > This series adds support for mt8196 multi-hardwares jpeg enc & dec,
> > by first adding mt8196 jpegdec and jpegenc compatible to install
> > kernel driver. Add smmu setting to support smmu and iommu at the
> > same time.
> > Secondly refactor buffer and clock setting to support multi-hw jpeg
> > working.
> > Lastly, fix some bugs, including resolution change handleing, stop
> > streaming sw flow and others.
> > 
> > This series has been tested with MT8196 tast test.
> > Encoding and decoding worked for this chip.
> > 
> > Patches 1-3 Adds jpeg encoder and decoder compatible.
> > Patches 4 add jpeg smmu sid setting.
> > Patches 5 fix jpeg hw count setting to support different chips.
> > Patches 6 refactor jpeg buffer payload setting to handle buffer
> > size bug while resolution changed.
> > Patches 7 reconstruct jpeg dst buffer layout.
> > Patches 8 fix multi-core stop streaming flow
> > Patches 9 refactor multi-core clk suspend/resume setting
> > Patches 10 fix decoding buffer number setting timing issue
> > Patches 11 refactor decoding resolution change operation
> > Patches 12 fix remove buffer operation
> > 
> > ---
> > This series patches dependent on:
> > [1]
> > 
https://urldefense.com/v3/__https://patchwork.kernel.org/project/linux-mediatek/patch/20240808092555.12999-1-jianhua.lin@mediatek.com/__;!!CTRNKA9wMg0ARbw!mJmYHl_SoomWL8beP_5XuUZ0bpvhI_TqJVcDjN5mkxOnrNPnEkfu8-JdTfxx3Q6SZIEBU-3CvddOw-3uio1MxA$
> > 
> > kyrie.wu (11):
> 
> would you mind fixing the --author in your configuration, write your
> name
> instead of your user name. UT8 is allowed, many people will use a
> ASCI spelling
> (some approximation) and then add their real name in parenthesis.
> 
> regards,
> Nicolas

Dear Nicolas,

I have resent a new version, but I didn't view this mail in time.
Thanks for your advice, and I will fix it in the next version.

Thanks.

Regards,
Kyrie.
> 
> >   dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible
> >   dt-bindings: mediatek: Add mediatek, mt8196-jpgenc compatible
> >   media: mediatek: jpeg: add jpeg compatible
> >   media: mediatek: jpeg: add jpeg smmu sid setting
> >   media: mediatek: jpeg: fix jpeg hw count setting
> >   media: mediatek: jpeg: refactor jpeg buffer payload setting
> >   media: mediatek: jpeg: refactor jpeg dst buffer layout
> >   media: mediatek: jpeg: fix stop streaming flow for multi-core
> >   media: mediatek: jpeg: refactor multi-core clk suspend and resume
> >     setting
> >   media: mediatek: jpeg: fix decoding buffer number setting timing
> > issue
> >   media: mediatek: jpeg: refactor decoding resolution change
> > operation
> >   media: mediatek: jpeg: fix remove buffer operation for multi-core
> > 
> >  ....yaml => mediatek,multi-core-jpegdec.yaml} |  10 +-
> >  ....yaml => mediatek,multi-core-jpegenc.yaml} |  10 +-
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 126 ++++++++++++
> > ------
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  17 ++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 116
> > +++++++++++++++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |   4 +
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 113
> > +++++++++++++++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |   4 +
> >  8 files changed, 343 insertions(+), 57 deletions(-)
> >  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-
> > jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)
> >  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-
> > jpegenc.yaml => mediatek,multi-core-jpegenc.yaml} (94%)
> > 
> 
> 

Re: [V1,00/12] Enable jpeg enc & dec multi-hardwares for MT8196

[Kyrie Wu (吴晗)]

On Thu, 2025-01-09 at 09:06 -0500, Nicolas Dufresne wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Hi,
> 
> Le jeudi 09 janvier 2025 à 21:35 +0800, kyrie.wu a écrit :
> > This series adds support for mt8196 multi-hardwares jpeg enc & dec,
> > by first adding mt8196 jpegdec and jpegenc compatible to install
> > kernel driver. Add smmu setting to support smmu and iommu at the
> > same time.
> > Secondly refactor buffer and clock setting to support multi-hw jpeg
> > working.
> > Lastly, fix some bugs, including resolution change handleing, stop
> > streaming sw flow and others.
> > 
> > This series has been tested with MT8196 tast test.
> > Encoding and decoding worked for this chip.
> > 
> > Patches 1-3 Adds jpeg encoder and decoder compatible.
> > Patches 4 add jpeg smmu sid setting.
> > Patches 5 fix jpeg hw count setting to support different chips.
> > Patches 6 refactor jpeg buffer payload setting to handle buffer
> > size bug while resolution changed.
> > Patches 7 reconstruct jpeg dst buffer layout.
> > Patches 8 fix multi-core stop streaming flow
> > Patches 9 refactor multi-core clk suspend/resume setting
> > Patches 10 fix decoding buffer number setting timing issue
> > Patches 11 refactor decoding resolution change operation
> > Patches 12 fix remove buffer operation
> > 
> > ---
> > This series patches dependent on:
> > [1]
> > 
https://urldefense.com/v3/__https://patchwork.kernel.org/project/linux-mediatek/patch/20240808092555.12999-1-jianhua.lin@mediatek.com/__;!!CTRNKA9wMg0ARbw!mJmYHl_SoomWL8beP_5XuUZ0bpvhI_TqJVcDjN5mkxOnrNPnEkfu8-JdTfxx3Q6SZIEBU-3CvddOw-3uio1MxA$
> > 
> > kyrie.wu (11):
> 
> would you mind fixing the --author in your configuration, write your
> name
> instead of your user name. UT8 is allowed, many people will use a
> ASCI spelling
> (some approximation) and then add their real name in parenthesis.
> 
> regards,
> Nicolas
> 
Hi Nicolas,

I will fix it in the coming version.

Thanks.

Regards,
Kyrie

> >   dt-bindings: mediatek: Add mediatek, mt8196-jpgdec compatible
> >   dt-bindings: mediatek: Add mediatek, mt8196-jpgenc compatible
> >   media: mediatek: jpeg: add jpeg compatible
> >   media: mediatek: jpeg: add jpeg smmu sid setting
> >   media: mediatek: jpeg: fix jpeg hw count setting
> >   media: mediatek: jpeg: refactor jpeg buffer payload setting
> >   media: mediatek: jpeg: refactor jpeg dst buffer layout
> >   media: mediatek: jpeg: fix stop streaming flow for multi-core
> >   media: mediatek: jpeg: refactor multi-core clk suspend and resume
> >     setting
> >   media: mediatek: jpeg: fix decoding buffer number setting timing
> > issue
> >   media: mediatek: jpeg: refactor decoding resolution change
> > operation
> >   media: mediatek: jpeg: fix remove buffer operation for multi-core
> > 
> >  ....yaml => mediatek,multi-core-jpegdec.yaml} |  10 +-
> >  ....yaml => mediatek,multi-core-jpegenc.yaml} |  10 +-
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.c    | 126 ++++++++++++
> > ------
> >  .../platform/mediatek/jpeg/mtk_jpeg_core.h    |  17 ++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.c  | 116
> > +++++++++++++++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_dec_hw.h  |   4 +
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.c  | 113
> > +++++++++++++++-
> >  .../platform/mediatek/jpeg/mtk_jpeg_enc_hw.h  |   4 +
> >  8 files changed, 343 insertions(+), 57 deletions(-)
> >  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-
> > jpegdec.yaml => mediatek,multi-core-jpegdec.yaml} (95%)
> >  rename Documentation/devicetree/bindings/media/{mediatek,mt8195-
> > jpegenc.yaml => mediatek,multi-core-jpegenc.yaml} (94%)
> > 
> 
>