Re: [PATCH] arm64: dts: imx8mp: add cpuidle state "cpu-pd-wait"

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

Hi Marco,

On Tue, Apr 15, 2025 at 08:14:23PM +0200, Marco Felsch wrote:
> On 25-04-15, Laurent Pinchart wrote:
> > On Tue, Apr 15, 2025 at 06:47:26PM +0300, Laurent Pinchart wrote:
> > > On Tue, Apr 15, 2025 at 03:42:22PM +0000, POPESCU Catalin wrote:
> > > > Hi Jai,
> > > > 
> > > > This issue was already reported by Stefan. The problem is that I don't 
> > > > have a Debix board to investigate.
> > > > The main difference b/w WFI and cpu-pd-wait is that the first doesn't 
> > > > call PSCI/TF-A. So, the issue looks to be related to some settings in 
> > > > the TF-A.
> > > 
> > > Jai, are you using mainline U-Boot and TF-A, or a downstream version of
> > > either (or both) ?
> > 
> > Actually, same question for Calatin :-)
> > 
> > I'm running mainline U-Boot 2025.01 and TF-A rel_imx_5.4.70_2.3.6 (from
> > https://github.com/nxp-imx/imx-atf) and don't seem to experience the
> > issue:
> 
> Interessting, can you share your imx-atf build-config please?

I've followed the instructions in the doc/board/nxp/imx8mp_evk.rst file
(in the U-Boot source):

```
Get and Build the ARM Trusted firmware
--------------------------------------

Get ATF from: https://github.com/nxp-imx/imx-atf
branch: imx_5.4.70_2.3.0

.. code-block:: bash

   $ make PLAT=imx8mp bl31
```

I had to patch the TF-A makefile as follows to fix errors with recent
versions of gcc:

```
diff --git a/Makefile b/Makefile
index 2ae12fda741a..46512993c6e0 100644
--- a/Makefile
+++ b/Makefile
@@ -285,7 +285,8 @@ ASFLAGS			+=	$(CPPFLAGS) $(ASFLAGS_$(ARCH))			\
 				-ffreestanding -Wa,--fatal-warnings
 TF_CFLAGS		+=	$(CPPFLAGS) $(TF_CFLAGS_$(ARCH))		\
 				-ffreestanding -fno-builtin -Wall -std=gnu99	\
-				-Os -ffunction-sections -fdata-sections
+				-Os -ffunction-sections -fdata-sections		\
+				-Wno-error=array-bounds

 ifeq (${SANITIZE_UB},on)
 TF_CFLAGS		+=	-fsanitize=undefined -fno-sanitize-recover
@@ -300,9 +301,11 @@ GCC_V_OUTPUT		:=	$(shell $(CC) -v 2>&1)
 ifneq ($(findstring armlink,$(notdir $(LD))),)
 TF_LDFLAGS		+=	--diag_error=warning --lto_level=O1
 TF_LDFLAGS		+=	--remove --info=unused,unusedsymbols
+TF_LDFLAGS		+=	-Wl,--no-warn-rwx-segment
 else
 TF_LDFLAGS		+=	--fatal-warnings -O1
 TF_LDFLAGS		+=	--gc-sections
+TF_LDFLAGS		+=	--no-warn-rwx-segment
 endif
 TF_LDFLAGS		+=	$(TF_LDFLAGS_$(ARCH))

```

> I checked the code base and found a missing SLPCR_A53_FASTWUP_STOP_MODE
> during the imx_set_sys_lpm() which is called during
> .pwr_domain_suspend(). Can you check/trace if pwr_domain_suspend() was
> entered?

I'm no TF-A expert, can you tell me how to check/trace that ?

> > # cat /sys/devices/system/cpu/cpu*/cpuidle/state1/disable
> > 0
> > 0
> > 0
> > 0
> > 
> > $ ping debix
> > PING debix.farm.ideasonboard.com (192.168.2.230) 56(84) bytes of data.
> > 64 bytes from debix.farm.ideasonboard.com (192.168.2.230): icmp_seq=1 ttl=64 time=1.03 ms
> > 64 bytes from debix.farm.ideasonboard.com (192.168.2.230): icmp_seq=2 ttl=64 time=0.800 ms
> > 64 bytes from debix.farm.ideasonboard.com (192.168.2.230): icmp_seq=3 ttl=64 time=0.935 ms
> > 64 bytes from debix.farm.ideasonboard.com (192.168.2.230): icmp_seq=4 ttl=64 time=0.902 ms
> > 64 bytes from debix.farm.ideasonboard.com (192.168.2.230): icmp_seq=5 ttl=64 time=0.738 ms
> > 64 bytes from debix.farm.ideasonboard.com (192.168.2.230): icmp_seq=6 ttl=64 time=0.939 ms
> > 
> > > > What I don't get is why I don't see this issue neither on our IMX8MP 
> > > > specific design nor on the EVK, which uses the same PHY as the Debix board.
> > > >
> > > > On 14/04/2025 14:07, Jai Luthra wrote:
> > > > > On Oct 21, 2024 at 17:42:34 +0800, Shawn Guo wrote:
> > > > >> On Mon, Oct 07, 2024 at 03:44:24PM +0200, Catalin Popescu wrote:
> > > > >>> So far, only WFI is supported on i.MX8mp platform. Add support for
> > > > >>> deeper cpuidle state "cpu-pd-wait" that would allow for better power
> > > > >>> usage during runtime. This is a port from NXP downstream kernel.
> > > > >>>
> > > > > Since the introduction of this patch in mainline, I am facing sluggish
> > > > > network performance with my Debix Model-A board with i.MX8mp SoC.
> > > > >
> > > > > The network latency jumps to >1s after almost every other packet:
> > > > >
> > > > > PING debix (10.0.42.5) 56(84) bytes of data.
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=1 ttl=64 time=1008 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=2 ttl=64 time=0.488 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=3 ttl=64 time=1025 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=4 ttl=64 time=0.810 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=5 ttl=64 time=590 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=6 ttl=64 time=0.351 ms
> > > > > ^C
> > > > > --- debix ping statistics ---
> > > > > 7 packets transmitted, 6 received, 14.2857% packet loss, time 6126ms
> > > > > rtt min/avg/max/mdev = 0.351/437.416/1024.755/459.370 ms, pipe 2
> > > > > darkapex at freya in ~
> > > > >
> > > > > If I revert the patch, or disable the deeper cpuidle state through
> > > > > sysfs, the issue goes away.
> > > > >
> > > > > # echo 1 > /sys/devices/system/cpu/cpu$i/cpuidle/state1/disable
> > > > >
> > > > > PING debix (10.0.42.5) 56(84) bytes of data.
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=1 ttl=64 time=0.482 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=2 ttl=64 time=2.28 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=3 ttl=64 time=2.26 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=4 ttl=64 time=0.848 ms
> > > > > 64 bytes from debix (10.0.42.5): icmp_seq=5 ttl=64 time=0.406 ms
> > > > > ^C
> > > > > --- debix ping statistics ---
> > > > > 5 packets transmitted, 5 received, 0% packet loss, time 4051ms
> > > > > rtt min/avg/max/mdev = 0.406/1.255/2.280/0.842 ms
> > > > >
> > > > >>> Signed-off-by: Catalin Popescu <catalin.popescu@leica-geosystems.com>
> > > > >>
> > > > >> Applied, thanks!

-- 
Regards,

Laurent Pinchart