Re: arm64 s2idle vs. workqueues

[Sudeep Holla <sudeep.holla@arm.com>]

On Thu, Oct 10, 2024 at 12:23:25PM +0200, Rafael J. Wysocki wrote:
> Hi Kevin,
> 
> On Thu, Oct 10, 2024 at 2:19 AM Kevin Hilman <khilman@baylibre.com> wrote:
> >
> > Hello,
> >
> > Looking for some pointers/tips on debugging s2idle, and in particular
> > why it is not staying in an idle state as long as expected.
> >
> > I'm attempting to use s2idle on a 4-core, single cluster ARM64 SoC (TI
> > AM62x), which doesn't (yet) have any DT defined idle-states, so is just
> > doing a WFI when idle.
> >
> > I'm doing an 8-second s2idle with RTC wakeup by using:
> >
> >   rtcwake -m freeze -s8
> >
> > and what I see is that 3 of the CPUs stay in their idle state for the
> > full 8 seconds, but one of them keeps waking due to the A53
> > arm_arch_timer firing, and processing misc. workqueue related activity
> > (example work listed below[1].)
> >
> > I realize that these workqueues are not WQ_FREEZABLE, so I don't expect
> > the freezer part of suspend to stop/freeze them.  However, I am a bit
> > surprised to see this non-frozen workqueue activity happening often
> > enough (few times per second) to prevent all 4 CPUs from being idle for
> > long periods at the same time, thus preventing a deeper cluster-idle
> > state.
> >
> > Is there something else I'm missing that is needed to keep these
> > workqueues quiet for longer?  I had assumed that most of this workqueue
> > work would be deferred, and shouldn't need to wakeup a CPU just to run.
> >
> > In case it's helpful I have published a trace.dat from trace-cmd which
> > captures power, sched, irq, timer and workqueue events.  With
> > kernelshark, it's pretty obvious to visualize what's happening: CPU0,1,3
> > are all nicely idle for 8 sec while CPU2 is waking due to the timer and
> > workqueue activity.
> >
> > Any pointers to how to improve this situation, or what else needs to be
> > tweaked here would be greatly appreciated,
> 
> It looks like tick_freeze() is not called, which only happens in
> enter_s2idle_proper() that is called from cpuidle_enter_s2idle() if
> there are any idle states with an .enter_s2idle() callback.
> 
> So does the cpuidle driver for this platform provide .enter_s2idle()
> callbacks for any of its idle states?

IIUC, his setup/platform doesn't support any deeper idle states than the
shallowest WFI on all arm systems. I was about to ask if he has already
tried to build a cpuidle driver with just that WFI state or is he relying
on default arch_cpu_idle() which is one possible reason to take different
path and hence not executing the expected s2idle path.

-- 
Regards,
Sudeep