power management problems in ehci-omap

power management problems in ehci-omap

[Tony Lindgren]

* Andreas Kemnade <andreas@kemnade.info> [180206 16:56]:
> On Tue, 6 Feb 2018 08:04:52 -0800
> Tony Lindgren <tony@atomide.com> wrote:
> 
> > * Andreas Kemnade <andreas@kemnade.info> [180206 06:42]:
> > > rechecked with a board with really nothing connected there
> > > Same behaviour  
> > 
> > I've just verified that my test board power consumption goes
> > back to normal after rmmod ehci-omap with v4.15.
> > 
> yes, for me too, I initially used a test script which does an
> echo rmmod ehci-omap
> 
> without a real
> rmmod ehci-omap

Ah OK :)

> It just seems to be consistent with my observations in a fully booted
> system (where many things can play a role). Sorry for that confusion.

Try with a minimal set of modules first, then modprobe and rmmod one
at a time until you find the module breaking PM?

You probably know this already, but just in case it helps..

First idle the UARTs and enable off mode with something like:

uarts=$(find /sys/class/tty/tty[SO]*/device/power/ -type d)
for uart in $uarts; do
	echo 3000 > $uart/autosuspend_delay_ms 2>&1
done

uarts=$(find /sys/class/tty/tty[SO]*/power/ -type d 2>/dev/null)
for uart in $uarts; do
	echo enabled > $uart/wakeup 2>&1
	echo auto > $uart/control 2>&1
done

echo -n 1 > /sys/kernel/debug/pm_debug/enable_off_mode

Then if using omap3, the attached debug hack patch can be used to
see which devices are not idling:

> But suspend current is a problem. I have repeated the measurement with
> another board, so it is not a board problem. 

I also verified v4.15 behaves for suspend current even with
echi-omap loaded just in case.

Regards,

Tony

8< -------------------
From tony Mon Sep 17 00:00:00 2001
From: Tony Lindgren <tony@atomide.com>
Date: Wed, 13 Dec 2017 16:36:45 -0800
Subject: [PATCH] NOT FOR MERGING: Test patch for dumping omap3 off idle
 blocking bits

Allows seeing the deeper idle state blockers in
/sys/kernel/debug/pm_debug/count. For example, when
off idle is working on beaglboard xm, this is what
i see:

# sleep 5; cat /sys/kernel/debug/pm_debug/count
...
0006ffff 48005020 (fa005020) cm_idlest_per blocking bits: 00010000
e7ffffbd 48004a20 (fa004a20) cm_idlest1_core blocking bits: 00000042
0000000d 48004a28 (fa004a28) cm_idlest3_core
---
 arch/arm/mach-omap2/pm-debug.c | 68 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/arch/arm/mach-omap2/pm-debug.c b/arch/arm/mach-omap2/pm-debug.c
--- a/arch/arm/mach-omap2/pm-debug.c
+++ b/arch/arm/mach-omap2/pm-debug.c
@@ -142,10 +142,78 @@ static int pwrdm_dbg_show_timer(struct powerdomain *pwrdm, void *user)
 	return 0;
 }
 
+#include "iomap.h"
+
+struct dregs {
+	const char	*desc;
+	u32		phys;
+	void __iomem	*virt;
+	u32		mask;
+};
+
+#define PER_CM_BASE	0x48005000
+#define PER_CM_REG(name, offset, mask)				\
+	{ name, PER_CM_BASE + offset,				\
+	OMAP2_L4_IO_ADDRESS(PER_CM_BASE + offset), mask, }
+
+static struct dregs cm_per[] = {
+	PER_CM_REG("cm_idlest_per", 0x20, 0xfff80000), /* p 513 */
+	{ NULL, },
+};
+
+#define CORE_CM_BASE	0x48004a00
+#define CORE_CM_REG(name, offset, mask)				\
+	{ name, CORE_CM_BASE + offset,				\
+	OMAP2_L4_IO_ADDRESS(CORE_CM_BASE + offset), mask, }
+
+static struct dregs cm_core[] = {
+	CORE_CM_REG("cm_idlest1_core", 0x20, 0x9c800109), /* p 467 */
+	CORE_CM_REG("cm_idlest3_core", 0x28, 0xfffffffb),
+	{ NULL, },
+};
+
+void __dregs_dump(struct dregs *dregs, struct seq_file *s)
+{
+	for (; dregs->desc; dregs++) {
+		u32 val, blockers;
+
+		val = __raw_readl(dregs->virt);
+
+		seq_printf(s, "%08x %08x (%p) %s",
+			   val, dregs->phys, dregs->virt,
+			   dregs->desc);
+
+		if (dregs->mask) {
+			blockers = ~val;
+			blockers &= ~dregs->mask;
+
+			if (blockers)
+				seq_printf(s, " blocking bits: %08x",
+					   blockers);
+		}
+
+		seq_printf(s, "\n");
+	}
+}
+
+void cm_per_dump(struct seq_file *s)
+{
+	__dregs_dump(cm_per, s);
+}
+
+void cm_core_dump(struct seq_file *s)
+{
+	__dregs_dump(cm_core, s);
+}
+
 static int pm_dbg_show_counters(struct seq_file *s, void *unused)
 {
 	pwrdm_for_each(pwrdm_dbg_show_counter, s);
 	clkdm_for_each(clkdm_dbg_show_counter, s);
+	if (cpu_is_omap34xx()) {
+		cm_per_dump(s);
+		cm_core_dump(s);
+	}
 
 	return 0;
 }

power management problems in ehci-omap

[Andreas Kemnade]

On Tue, 6 Feb 2018 09:17:37 -0800
Tony Lindgren <tony@atomide.com> wrote:

> * Andreas Kemnade <andreas@kemnade.info> [180206 16:56]:
> > On Tue, 6 Feb 2018 08:04:52 -0800
> > Tony Lindgren <tony@atomide.com> wrote:
> >   
> > > * Andreas Kemnade <andreas@kemnade.info> [180206 06:42]:  
> > > > rechecked with a board with really nothing connected there
> > > > Same behaviour    
> > > 
> > > I've just verified that my test board power consumption goes
> > > back to normal after rmmod ehci-omap with v4.15.
> > >   
> > yes, for me too, I initially used a test script which does an
> > echo rmmod ehci-omap
> > 
> > without a real
> > rmmod ehci-omap  
> 
> Ah OK :)
> 
> > It just seems to be consistent with my observations in a fully booted
> > system (where many things can play a role). Sorry for that confusion.  
> 
> Try with a minimal set of modules first, then modprobe and rmmod one
> at a time until you find the module breaking PM?
> 
Yes, that is basically what I do with this script:

https://misc.andi.de1.cc/measure4.sh

and

https://misc.andi.de1.cc/measure5.sh

I start from a bare kernel, check currents, load ehci-omap (I have also
debugged many other pm things that way)
check currents again. 
But since my rough observations with a fully loaded system
seem to match with the 
echo rmmod behaviour, I did not notice my mistake.

> You probably know this already, but just in case it helps..
> 
> First idle the UARTs and enable off mode with something like:
> 
> uarts=$(find /sys/class/tty/tty[SO]*/device/power/ -type d)
> for uart in $uarts; do
> 	echo 3000 > $uart/autosuspend_delay_ms 2>&1
> done
> 
> uarts=$(find /sys/class/tty/tty[SO]*/power/ -type d 2>/dev/null)
> for uart in $uarts; do
> 	echo enabled > $uart/wakeup 2>&1
> 	echo auto > $uart/control 2>&1
> done
> 

hmm, this looks a bit like runtime suspend.

I mean suspend aka echo mem >/sys/power/state

> echo -n 1 > /sys/kernel/debug/pm_debug/enable_off_mode
> 
In earlier times this just caused trouble and a little lower current,
maybe it is time to check again.

> Then if using omap3, the attached debug hack patch can be used to
> see which devices are not idling:
> 
Yes, I will try out. It it about detecting whether the soc itself went
into low power mode. But it does not check e.g. whether the phy is put
into low power mode correctly. So ehci-usb in soc might be powered off
and phy is still on.

When I go to suspend, I get this message:
"Successfully put all powerdomains to target state"

@Nikolaus: Can you check IR at the end of the suspend time in
https://misc.andi.de1.cc/measure5.sh
the second suspend compared with the first whether the phy (the USB
2233) stays on. 

Regards,
Andreas

power management problems in ehci-omap

[Tony Lindgren]

* Andreas Kemnade <andreas@kemnade.info> [180206 18:04]:
> On Tue, 6 Feb 2018 09:17:37 -0800
> Tony Lindgren <tony@atomide.com> wrote:
> > uarts=$(find /sys/class/tty/tty[SO]*/power/ -type d 2>/dev/null)
> > for uart in $uarts; do
> > 	echo enabled > $uart/wakeup 2>&1
> > 	echo auto > $uart/control 2>&1
> > done
> > 
> 
> hmm, this looks a bit like runtime suspend.

Not only that, it enables wakeup for UART also for suspend :)

That is if your dts has it configured with interrupts-extended
for the console UART like omap3-beagle-xm.dts has for example.
Seems like the gta04 dts don't have these.. And you also want
to have chosen with stdout-path = &uart3 or whatever the debug
UART is for earlycon to work.

> I mean suspend aka echo mem >/sys/power/state
> 
> > echo -n 1 > /sys/kernel/debug/pm_debug/enable_off_mode

And the above will enable SoC and PMIC off modes, which will also
take the suspend power to some much much lower value :) You need
to configure the PMIC too depending if the oscillator can be turned
off, in that case set "ti,twl4030-power-idle-osc-off". That too
seems to be missing in gta04 dts files..

Regards,

Tony
---
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

power management problems in ehci-omap

[Andreas Kemnade]

On Tue, 6 Feb 2018 10:16:23 -0800
Tony Lindgren <tony@atomide.com> wrote:

> * Andreas Kemnade <andreas@kemnade.info> [180206 18:04]:
> > On Tue, 6 Feb 2018 09:17:37 -0800
> > Tony Lindgren <tony@atomide.com> wrote:  
> > > uarts=$(find /sys/class/tty/tty[SO]*/power/ -type d 2>/dev/null)
> > > for uart in $uarts; do
> > > 	echo enabled > $uart/wakeup 2>&1
> > > 	echo auto > $uart/control 2>&1
> > > done
> > >   
> > 
> > hmm, this looks a bit like runtime suspend.  
> 
> Not only that, it enables wakeup for UART also for suspend :)
> 
We are using the rtc for wakeup and measure discharge of battery
for a time frame of about 300 seconds.

> That is if your dts has it configured with interrupts-extended
> for the console UART like omap3-beagle-xm.dts has for example.
> Seems like the gta04 dts don't have these.. And you also want
> to have chosen with stdout-path = &uart3 or whatever the debug
> UART is for earlycon to work.
> 
> > I mean suspend aka echo mem >/sys/power/state
> >   
> > > echo -n 1 > /sys/kernel/debug/pm_debug/enable_off_mode  
> 
> And the above will enable SoC and PMIC off modes, which will also
> take the suspend power to some much much lower value :) You need
> to configure the PMIC too depending if the oscillator can be turned
> off, in that case set "ti,twl4030-power-idle-osc-off". That too
> seems to be missing in gta04 dts files..
> 
It was in our tree. It can be enabled for the gta04a5. We have even done
that. But then suspend while charging breaks. I have no idea how to do a
proper if-not-charging-power-idle-osc-off patch... 

Yes there are other places where we can optimize suspend current. But
lets first find out why ehci-omap seems to cause trouble here.
So we are looking for around 15mA of additional suspend current when the
module is loaded. 
Shouldn't the reset line of the phy (usb-nop-xceiv) be set to low when
going to suspend? I do not see code how to do it. I guess that is the
reason.

BTW:
root@letux:~# cat /sys/bus/platform/devices/48064800.ehci/power/runtime_status 
active

Regards,
Andreas

power management problems in ehci-omap

[Roger Quadros]

Hi,

On 06/02/18 20:40, Andreas Kemnade wrote:
> On Tue, 6 Feb 2018 10:16:23 -0800
> Tony Lindgren <tony@atomide.com> wrote:
> 
>> * Andreas Kemnade <andreas@kemnade.info> [180206 18:04]:
>>> On Tue, 6 Feb 2018 09:17:37 -0800
>>> Tony Lindgren <tony@atomide.com> wrote:  
>>>> uarts=$(find /sys/class/tty/tty[SO]*/power/ -type d 2>/dev/null)
>>>> for uart in $uarts; do
>>>> 	echo enabled > $uart/wakeup 2>&1
>>>> 	echo auto > $uart/control 2>&1
>>>> done
>>>>   
>>>
>>> hmm, this looks a bit like runtime suspend.  
>>
>> Not only that, it enables wakeup for UART also for suspend :)
>>
> We are using the rtc for wakeup and measure discharge of battery
> for a time frame of about 300 seconds.
> 
>> That is if your dts has it configured with interrupts-extended
>> for the console UART like omap3-beagle-xm.dts has for example.
>> Seems like the gta04 dts don't have these.. And you also want
>> to have chosen with stdout-path = &uart3 or whatever the debug
>> UART is for earlycon to work.
>>
>>> I mean suspend aka echo mem >/sys/power/state
>>>   
>>>> echo -n 1 > /sys/kernel/debug/pm_debug/enable_off_mode  
>>
>> And the above will enable SoC and PMIC off modes, which will also
>> take the suspend power to some much much lower value :) You need
>> to configure the PMIC too depending if the oscillator can be turned
>> off, in that case set "ti,twl4030-power-idle-osc-off". That too
>> seems to be missing in gta04 dts files..
>>
> It was in our tree. It can be enabled for the gta04a5. We have even done
> that. But then suspend while charging breaks. I have no idea how to do a
> proper if-not-charging-power-idle-osc-off patch... 
> 
> Yes there are other places where we can optimize suspend current. But
> lets first find out why ehci-omap seems to cause trouble here.
> So we are looking for around 15mA of additional suspend current when the
> module is loaded. 
> Shouldn't the reset line of the phy (usb-nop-xceiv) be set to low when
> going to suspend? I do not see code how to do it. I guess that is the
> reason.
> 
> BTW:
> root@letux:~# cat /sys/bus/platform/devices/48064800.ehci/power/runtime_status 
> active


PM handling is not yet there in the ehci-omap driver.

> static struct platform_driver ehci_hcd_omap_driver = {
>         .probe                  = ehci_hcd_omap_probe,
>         .remove                 = ehci_hcd_omap_remove,
>         .shutdown               = usb_hcd_platform_shutdown,
>         /*.suspend              = ehci_hcd_omap_suspend, */
>         /*.resume               = ehci_hcd_omap_resume, */
>         .driver = {
>                 .name           = hcd_name,
>                 .of_match_table = omap_ehci_dt_ids,
>         }
> };

There is also some co-relation with the parent driver
drivers/mfd/omap-usb-host.c

Getting low power on system suspend should be easy as we most probably don't
need wakeup to work in this case. This should fix the increased power during
system suspend.

To fix active power consumption we need to implement runtime PM.
However, for runtime suspend case we do need remote-wakeup to work else we
won't be able to detect any new USB devices after the host runtime suspends.
For this we need to remux one of the PHY lines into a GPIO to capture the wake event.
This is where the generic wakeirq support comes in.