[PATCH v3 1/2] ASoC: dt-bindings: sound-card-common: List DAPM endpoints ignoring system suspend

[PATCH v3 1/2] ASoC: dt-bindings: sound-card-common: List DAPM endpoints ignoring system suspend

[Chancel Liu]

Add a property to list DAPM endpoints which mark paths between these
endpoints should not be disabled when system enters in suspend state.

LPA means low power audio case. For example on asymmetric
multiprocessor, there are Cortex-A core and Cortex-M core, Linux is
running on Cortex-A core, RTOS or other OS is running on Cortex-M core.
The audio hardware devices can be controlled by Cortex-M. LPA can be
explained as a mechanism that Cortex-A allocates a large buffer and
fill audio data, then Cortex-A can enter into suspend for the purpose
of power saving. Cortex-M continues to play the sound during suspend
phase of Cortex-A. LPA requires some audio paths still enabled when
Cortex-A enters into suspend.

Signed-off-by: Chancel Liu <chancel.liu@nxp.com>
---
 .../bindings/sound/sound-card-common.yaml          | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/Documentation/devicetree/bindings/sound/sound-card-common.yaml b/Documentation/devicetree/bindings/sound/sound-card-common.yaml
index 3a941177f684..f43147c78651 100644
--- a/Documentation/devicetree/bindings/sound/sound-card-common.yaml
+++ b/Documentation/devicetree/bindings/sound/sound-card-common.yaml
@@ -17,6 +17,20 @@ properties:
       pair of strings, the first being the connection's sink, the second
       being the connection's source.
 
+  lpa-widgets:
+    $ref: /schemas/types.yaml#/definitions/non-unique-string-array
+    description: |
+      A list of DAPM endpoints which mark paths between these endpoints should
+      not be disabled when system enters in suspend state. LPA means low power
+      audio case. For example on asymmetric multiprocessor, there are Cortex-A
+      core and Cortex-M core, Linux is running on Cortex-A core, RTOS or other
+      OS is running on Cortex-M core. The audio hardware devices can be
+      controlled by Cortex-M. LPA can be explained as a mechanism that Cortex-A
+      allocates a large buffer and fill audio data, then Cortex-A can enter
+      into suspend for the purpose of power saving. Cortex-M continues to play
+      the sound during suspend phase of Cortex-A. LPA requires some audio paths
+      still enabled when Cortex-A enters into suspend.
+
   model:
     $ref: /schemas/types.yaml#/definitions/string
     description: User specified audio sound card name
-- 
2.25.1

[PATCH v3 2/2] ASoC: imx-rpmsg: Force codec power on in low power audio mode

[Chancel Liu]

Low power audio mode requires binding codec still power on while Acore
enters into suspend so Mcore can continue playback music.

ASoC machine driver acquires DAPM endpoints through reading
"lpa-widgets" property from DT and then forces the path between these
endpoints ignoring suspend.

If the rpmsg sound card is in low power audio mode, the suspend/resume
callback of binding codec is overridden to disable the suspend/resume.

Signed-off-by: Chancel Liu <chancel.liu@nxp.com>
---
 sound/soc/fsl/imx-rpmsg.c | 58 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/sound/soc/fsl/imx-rpmsg.c b/sound/soc/fsl/imx-rpmsg.c
index a9324712e3fa..2c54c92fb911 100644
--- a/sound/soc/fsl/imx-rpmsg.c
+++ b/sound/soc/fsl/imx-rpmsg.c
@@ -20,8 +20,11 @@ struct imx_rpmsg {
 	struct snd_soc_dai_link dai;
 	struct snd_soc_card card;
 	unsigned long sysclk;
+	bool lpa;
 };
 
+static struct dev_pm_ops lpa_pm;
+
 static const struct snd_soc_dapm_widget imx_rpmsg_dapm_widgets[] = {
 	SND_SOC_DAPM_HP("Headphone Jack", NULL),
 	SND_SOC_DAPM_SPK("Ext Spk", NULL),
@@ -38,6 +41,58 @@ static int imx_rpmsg_late_probe(struct snd_soc_card *card)
 	struct device *dev = card->dev;
 	int ret;
 
+	if (data->lpa) {
+		struct snd_soc_component *codec_comp;
+		struct device_node *codec_np;
+		struct device_driver *codec_drv;
+		struct device *codec_dev = NULL;
+
+		codec_np = data->dai.codecs->of_node;
+		if (codec_np) {
+			struct platform_device *codec_pdev;
+			struct i2c_client *codec_i2c;
+
+			codec_i2c = of_find_i2c_device_by_node(codec_np);
+			if (codec_i2c)
+				codec_dev = &codec_i2c->dev;
+			if (!codec_dev) {
+				codec_pdev = of_find_device_by_node(codec_np);
+				if (codec_pdev)
+					codec_dev = &codec_pdev->dev;
+			}
+		}
+		if (codec_dev) {
+			codec_comp = snd_soc_lookup_component_nolocked(codec_dev, NULL);
+			if (codec_comp) {
+				int i, num_widgets;
+				const char *widgets;
+				struct snd_soc_dapm_context *dapm;
+
+				num_widgets = of_property_count_strings(data->card.dev->of_node,
+									"lpa-widgets");
+				for (i = 0; i < num_widgets; i++) {
+					of_property_read_string_index(data->card.dev->of_node,
+								      "lpa-widgets",
+								      i, &widgets);
+					dapm = snd_soc_component_get_dapm(codec_comp);
+					snd_soc_dapm_ignore_suspend(dapm, widgets);
+				}
+			}
+			codec_drv = codec_dev->driver;
+			if (codec_drv->pm) {
+				memcpy(&lpa_pm, codec_drv->pm, sizeof(lpa_pm));
+				lpa_pm.suspend = NULL;
+				lpa_pm.resume = NULL;
+				lpa_pm.freeze = NULL;
+				lpa_pm.thaw = NULL;
+				lpa_pm.poweroff = NULL;
+				lpa_pm.restore = NULL;
+				codec_drv->pm = &lpa_pm;
+			}
+			put_device(codec_dev);
+		}
+	}
+
 	if (!data->sysclk)
 		return 0;
 
@@ -137,6 +192,9 @@ static int imx_rpmsg_probe(struct platform_device *pdev)
 		goto fail;
 	}
 
+	if (of_property_read_bool(np, "fsl,enable-lpa"))
+		data->lpa = true;
+
 	data->card.dev = &pdev->dev;
 	data->card.owner = THIS_MODULE;
 	data->card.dapm_widgets = imx_rpmsg_dapm_widgets;
-- 
2.25.1

Re: [PATCH v3 1/2] ASoC: dt-bindings: sound-card-common: List DAPM endpoints ignoring system suspend

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]

On Wed, Oct 11, 2023 at 07:47:58PM +0800, Chancel Liu wrote:

> +  lpa-widgets:
> +    $ref: /schemas/types.yaml#/definitions/non-unique-string-array
> +    description: |
> +      A list of DAPM endpoints which mark paths between these endpoints should
> +      not be disabled when system enters in suspend state. LPA means low power
> +      audio case. For example on asymmetric multiprocessor, there are Cortex-A

I suspect that the DT maintainers would prefer that this description be
workshopped a bit to remove the Linux specifics.  I think the key thing
here is that these are endpoints that can be active over suspend of the
main application processor that the current operating system is running
(system DT stuff is an interesting corner case here...), and the example
is probably a bit specific.  Other bindings use "audio sound widgets"
rather than "DAPM widgets".

We also shouldn't see that these endpoints "should not be disabled"
since that implies that they should be left on even if they aren't
active which isn't quite the case, instead it's that we can continue
playing an audio stream through them in suspend.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v3 1/2] ASoC: dt-bindings: sound-card-common: List DAPM endpoints ignoring system suspend

[Rob Herring]

On Wed, Oct 11, 2023 at 10:21:33PM +0100, Mark Brown wrote:
> On Wed, Oct 11, 2023 at 07:47:58PM +0800, Chancel Liu wrote:
> 
> > +  lpa-widgets:
> > +    $ref: /schemas/types.yaml#/definitions/non-unique-string-array
> > +    description: |
> > +      A list of DAPM endpoints which mark paths between these endpoints should
> > +      not be disabled when system enters in suspend state. LPA means low power
> > +      audio case. For example on asymmetric multiprocessor, there are Cortex-A
> 
> I suspect that the DT maintainers would prefer that this description be
> workshopped a bit to remove the Linux specifics.

And Cortex A/M specifics if this is a common binding.


>  I think the key thing
> here is that these are endpoints that can be active over suspend of the
> main application processor that the current operating system is running
> (system DT stuff is an interesting corner case here...), and the example
> is probably a bit specific.  Other bindings use "audio sound widgets"
> rather than "DAPM widgets".
> 
> We also shouldn't see that these endpoints "should not be disabled"
> since that implies that they should be left on even if they aren't
> active which isn't quite the case, instead it's that we can continue
> playing an audio stream through them in suspend.

This seems like one of those things that everyone has/does, and everyone 
handles it a bit differently. I applaud trying to do something common, 
but it isn't really common until we have multiple users.

Rob

RE: Re: [PATCH v3 1/2] ASoC: dt-bindings: sound-card-common: List DAPM endpoints ignoring system suspend

[Chancel Liu]

> >  I think the key thing
> > here is that these are endpoints that can be active over suspend of
> > the main application processor that the current operating system is
> > running (system DT stuff is an interesting corner case here...), and
> > the example is probably a bit specific.  Other bindings use "audio sound
> widgets"
> > rather than "DAPM widgets".
> >
> > We also shouldn't see that these endpoints "should not be disabled"
> > since that implies that they should be left on even if they aren't
> > active which isn't quite the case, instead it's that we can continue
> > playing an audio stream through them in suspend.
> 
> This seems like one of those things that everyone has/does, and everyone
> handles it a bit differently. I applaud trying to do something common, but it
> isn't really common until we have multiple users.
> 
> Rob

Thanks Mark and Rob for your advice. In fact, it's common use case. We can see
many drivers set widgets ignoring suspend. I will remove the linux specifics
and focus on the key concept. How about the modification on the property name
and description as following:
  ignore-suspend-widgets:
    description: |
      A list of audio sound widgets which are marked ignoring system suspend.
	  Paths between these endpoints are still active over suspend of the main
	  application processor that the current operating system is running.

Regards, 
Chancel Liu

Re: Re: [PATCH v3 1/2] ASoC: dt-bindings: sound-card-common: List DAPM endpoints ignoring system suspend

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Mon, Oct 16, 2023 at 12:08:56PM +0000, Chancel Liu wrote:

> Thanks Mark and Rob for your advice. In fact, it's common use case. We can see
> many drivers set widgets ignoring suspend. I will remove the linux specifics
> and focus on the key concept. How about the modification on the property name
> and description as following:
>   ignore-suspend-widgets:
>     description: |
>       A list of audio sound widgets which are marked ignoring system suspend.
> 	  Paths between these endpoints are still active over suspend of the main
> 	  application processor that the current operating system is running.

That's probably fine from my point of view.  There's likely a better way
of saying system suspend but I'm not immediately seeing it and it could
always be improved later.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]