Re: My AlderLake Dell (XPS-9320) needs these patches to get full standby/low-power modes

[Bjorn Helgaas <helgaas@kernel.org>]

[+cc Mika, Sathy, Rafael, David, Ilpo, Ricky, Mario, linux-pci]

On Sat, Nov 04, 2023 at 10:13:24AM -0700, Kenneth R. Crudup wrote:
> 
> I have a Dell XPS-9320 with an Alderlake chipset, and the NVMe behind a
> VMD device:
> 
> ----
> [    0.127342] smpboot: CPU0: 12th Gen Intel(R) Core(TM) i7-1280P (family: 0x6, model: 0x9a, stepping: 0x3)
> ----
> 0000:00:0e.0 0104: 8086:467f
>         Subsystem: 1028:0af3
>         Flags: bus master, fast devsel, latency 0, IOMMU group 9
>         Memory at 603c000000 (64-bit, non-prefetchable) [size=32M]
>         Memory at 72000000 (32-bit, non-prefetchable) [size=32M]
> a7152be79b6        Memory at 6040100000 (64-bit, non-prefetchable) [size=1M]
>         Capabilities: <access denied>
>         Kernel driver in use: vmd
> ----
> 
> The only release kernel that was able to get this laptop to fully get into
> low-power (unfortunately only s0ix) was the Ubuntu-6.2.0- ... series from
> Ubuntu
> (remote git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/lunar).
> 
> I'd bisected it to the following commits (in this order):
> 
> 4ff116d0d5fd PCI/ASPM: Save L1 PM Substates Capability for suspend/resume
> 5e85eba6f50d PCI/ASPM: Refactor L1 PM Substates Control Register programming
> 1a0102a08f20 UBUNTU: SAUCE: PCI/ASPM: Enable ASPM for links under VMD domain
> 47c7bfd31514 UBUNTU: SAUCE: PCI/ASPM: Enable LTR for endpoints behind VMD
> 154d48da2c57 UBUNTU: SAUCE: vmd: fixup bridge ASPM by driver name instead

Thanks for these.  You don't happen to have URLs for those Ubuntu
commits, do you?  E.g., https://git.kernel.org/linus/4ff116d0d5fd
(which was reverted by a7152be79b62 ("Revert "PCI/ASPM: Save L1 PM
Substates Capability for suspend/resume"")).

> Without the patches I never see Pkg%PC8 or higher(? lower?), nor i915 states
> DC5/6, all necssary for SYS%LPI/CPU%LPI. I've attached a little script I use
> alongside turbostat for verifying low-power operation (and also for seeing
> what chipset subsystem may be preventing it).
> 
> The first two are in Linus' trees, but were reverted (4ff116d0d5fd in
> a7152be79b6, 5e85eba6f50d in ff209ecc376a). The last three come from Ubuntu's
> Linux trees (see remote spec above). The first two remain reverted in the
> Ubuntu trees, but if I put them back, I get increased power savings during
> suspend/resume cycles.
> 
> Considering the power draw is really significant without these patches (10s
> of %s per hour) and I'd think Dell would have sold some decent number of
> these laptops, I'd been patiently waiting for these patches, or some variant
> to show up in the stable trees, but so far I'm up to the 6.6 stable kernel
> and still having to manually cherry-pick these, so I thought maybe I could
> bring this to the PM maintainers' attention so at least start a discussion
> about this issue.

Thank you very much for raising this again.  We really need to make
some progress, and Mika recently posted a patch to add the
4ff116d0d5fd functionality again:
https://lore.kernel.org/r/20231002070044.2299644-1-mika.westerberg@linux.intel.com

The big problem is that it works on *most* systems, but it still seems
to break a few.  So Mika's current patch relies on a denylist of
systems where we *don't* restore the substates.

It's possible we'll have to give in and use a denylist, but it's
obviously not ideal because (a) we don't know *why* it doesn't work on
those systems, and (b) it means substates work before suspend but not
after resume, which is a poor user experience.

Bjorn

> #!/bin/bash -e
> date
> egrep -Hr . /sys/class/drm/card0/power/rc6_residency_ms \
>  /sys/devices/system/cpu/cpuidle/low_power_idle_system_residency_us \
>  /sys/kernel/debug/pmc_core/package_cstate_show \
>  /sys/kernel/debug/pmc_core/slp_s0_residency_usec \
>  /sys/kernel/debug/dri/0/i915_edp_psr_status \
>  /sys/kernel/debug/dri/0/i915_dmc_info | tee -a ~/Dropbox/XPS-7390/sleep-params
> egrep '\(ns\): [^0]' /sys/kernel/debug/pmc_core/ltr_show | cut -d'	' -f1,3,4 | sed -e 's;[	][	]*; ;' | tee -a ~/Dropbox/XPS-7390/sleep-params
> egrep -Hr ": On" /sys/kernel/debug/pmc_core/pch_ip_power_gating_status | tee -a /dev/tty | tee -a ~/Dropbox/XPS-7390/sleep-params | wc -l
> egrep No /sys/kernel/debug/pmc_core/slp_s0_debug_status 2>/dev/null | tee -a ~/Dropbox/XPS-7390/sleep-params