Re: arm64 s2idle vs. workqueues

[Sudeep Holla <sudeep.holla@arm.com>]

On Thu, Oct 10, 2024 at 11:33:09AM +0100, Christian Loehle wrote:
> On 10/10/24 11:23, Rafael J. Wysocki wrote:
> > Hi Kevin,
> > 
> > On Thu, Oct 10, 2024 at 2:19 AM Kevin Hilman <khilman@baylibre.com> wrote:
> >>
> >> Hello,
> >>
> >> Looking for some pointers/tips on debugging s2idle, and in particular
> >> why it is not staying in an idle state as long as expected.
> >>
> >> I'm attempting to use s2idle on a 4-core, single cluster ARM64 SoC (TI
> >> AM62x), which doesn't (yet) have any DT defined idle-states, so is just
> >> doing a WFI when idle.
> >>
> >> I'm doing an 8-second s2idle with RTC wakeup by using:
> >>
> >>   rtcwake -m freeze -s8
> >>
> >> and what I see is that 3 of the CPUs stay in their idle state for the
> >> full 8 seconds, but one of them keeps waking due to the A53
> >> arm_arch_timer firing, and processing misc. workqueue related activity
> >> (example work listed below[1].)
> >>
> >> I realize that these workqueues are not WQ_FREEZABLE, so I don't expect
> >> the freezer part of suspend to stop/freeze them.  However, I am a bit
> >> surprised to see this non-frozen workqueue activity happening often
> >> enough (few times per second) to prevent all 4 CPUs from being idle for
> >> long periods at the same time, thus preventing a deeper cluster-idle
> >> state.
> >>
> >> Is there something else I'm missing that is needed to keep these
> >> workqueues quiet for longer?  I had assumed that most of this workqueue
> >> work would be deferred, and shouldn't need to wakeup a CPU just to run.
> >>
> >> In case it's helpful I have published a trace.dat from trace-cmd which
> >> captures power, sched, irq, timer and workqueue events.  With
> >> kernelshark, it's pretty obvious to visualize what's happening: CPU0,1,3
> >> are all nicely idle for 8 sec while CPU2 is waking due to the timer and
> >> workqueue activity.
> >>
> >> Any pointers to how to improve this situation, or what else needs to be
> >> tweaked here would be greatly appreciated,
> > 
> > It looks like tick_freeze() is not called, which only happens in
> > enter_s2idle_proper() that is called from cpuidle_enter_s2idle() if
> > there are any idle states with an .enter_s2idle() callback.
> > 
> > So does the cpuidle driver for this platform provide .enter_s2idle()
> > callbacks for any of its idle states?
> 
> AFAICT there shouldn't be a cpuidle driver that initialized and WFI is
> entered through arch code. The trace.dat indicates that, too.
> 
> @Kevin
> I assume you can add a state in the DT, disable it and everything works
> as expected?
>

Good point, I didn't realise that will be quickest way, I was thinking of
writing a small wfi only idle driver.

-- 
Regards,
Sudeep