Re: [PATCH v2 00/11] KVM: arm64: Implement PSCI SYSTEM_SUSPEND support

[Oliver Upton <oupton@google.com>]

On Thu, Sep 23, 2021 at 07:15:59PM +0000, Oliver Upton wrote:
> Certain VMMs/operators may wish to give their guests the ability to
> initiate a system suspend that could result in the VM being saved to
> persistent storage to be resumed at a later time. The PSCI v1.0
> specification describes an SMC, SYSTEM_SUSPEND, that allows a kernel to
> request a system suspend. This call is optional for v1.0, and KVM
> elected to not support the call in its v1.0 implementation.
> 
> This series adds support for the SYSTEM_SUSPEND PSCI call to KVM/arm64.
> Since this is a system-scoped event, KVM cannot quiesce the VM on its
> own. We add a new system exit type in this series to clue in userspace
> that a suspend was requested. Per the KVM_EXIT_SYSTEM_EVENT ABI, a VMM
> that doesn't care about this event can simply resume the guest without
> issue (we set up the calling vCPU to come out of reset correctly on next
> KVM_RUN). If a VMM would like to have KVM emulate the suspend, it can do
> so by setting the vCPU's MP state to KVM_MP_STATE_HALTED. Support for
> this state has been added in this series.
> 
> Patch 1 is an unrelated cleanup, dropping an unused parameter
> 
> Patch 2 simplifies how KVM filters SMC64 functions for AArch32 guests.
> 
> Patch 3 wraps up the vCPU reset logic used by the PSCI CPU_ON
> implementation in KVM for subsequent use, as we must queue up a reset
> for the vCPU that requested a system suspend.
> 
> Patch 4 is another unrelated cleanup, fixing the naming for the
> KVM_REQ_SLEEP handler to avoid confusion and remain consistent with the
> handler introduced in this series.
> 
> Patch 5 changes how WFI-like events are handled in KVM (WFI instruction,
> PSCI CPU_SUSPEND). Instead of directly blocking the vCPU in the
> respective handlers, set a request bit and block before resuming the
> guest. WFI and PSCI CPU_SUSPEND do not require deferral of
> kvm_vcpu_block(), but SYSTEM_SUSPEND does. Rather than adding a deferral
> mechanism just for SYSTEM_SUSPEND, it is a bit cleaner to have all
> blocking events just request the event.
> 
> Patch 6 actually adds PSCI SYSTEM_SUSPEND support to KVM, and adds the
> necessary UAPI to pair with the call.
> 
> Patch 7 renames the PSCI selftest to something more generic, as we will
> test more than just CPU_ON.
> 
> Patch 8 creates a common helper for making SMC64 calls in KVM selftests,
> rather than having tests open-code their own approach.
> 
> Patch 9 makes the PSCI test use KVM_SET_MP_STATE for powering off a vCPU
> rather than the vCPU init flag. This change is necessary to separate
> generic VM setup from the setup for a particular PSCI test.
> 
> Patch 10 reworks psci_test into a bunch of helpers, making it easier to
> build additional test cases with the common parts.
> 
> Finally, patch 11 adds 2 test cases for the SYSTEM_SUSPEND PSCI call.
> Verify that the call succeeds if all other vCPUs have been powered off
> and that it fails if more than the calling vCPU is powered on.
> 
> This series applies cleanly to v5.15-rc2. Testing was performed on an
> Ampere Mt. Jade system.

Gah, forgot to summarize updates:

v1 -> v2:
 - Rebase to 5.15-rc2
 - Allow userspace to request in-kernel suspend emulation (Marc)
 - Add another test case for SYSTEM_SUSPEND, cleaning up the PSCI
   selftest
 - Create a common SMCCC function for KVM selftests

v1: http://lore.kernel.org/r/20210819223640.3564975-1-oupton@google.com

> Oliver Upton (11):
>   KVM: arm64: Drop unused vcpu param to kvm_psci_valid_affinity()
>   KVM: arm64: Clean up SMC64 PSCI filtering for AArch32 guests
>   KVM: arm64: Encapsulate reset request logic in a helper function
>   KVM: arm64: Rename the KVM_REQ_SLEEP handler
>   KVM: arm64: Defer WFI emulation as a requested event
>   KVM: arm64: Add support for SYSTEM_SUSPEND PSCI call
>   selftests: KVM: Rename psci_cpu_on_test to psci_test
>   selftests: KVM: Create helper for making SMCCC calls
>   selftests: KVM: Use KVM_SET_MP_STATE to power off vCPU in psci_test
>   selftests: KVM: Refactor psci_test to make it amenable to new tests
>   selftests: KVM: Test SYSTEM_SUSPEND PSCI call
> 
>  Documentation/virt/kvm/api.rst                |   6 +
>  arch/arm64/include/asm/kvm_host.h             |   4 +
>  arch/arm64/kvm/arm.c                          |  21 +-
>  arch/arm64/kvm/handle_exit.c                  |   3 +-
>  arch/arm64/kvm/psci.c                         | 138 ++++++++---
>  include/uapi/linux/kvm.h                      |   2 +
>  tools/testing/selftests/kvm/.gitignore        |   2 +-
>  tools/testing/selftests/kvm/Makefile          |   2 +-
>  .../selftests/kvm/aarch64/psci_cpu_on_test.c  | 121 ----------
>  .../testing/selftests/kvm/aarch64/psci_test.c | 218 ++++++++++++++++++
>  .../selftests/kvm/include/aarch64/processor.h |  22 ++
>  .../selftests/kvm/lib/aarch64/processor.c     |  25 ++
>  tools/testing/selftests/kvm/steal_time.c      |  13 +-
>  13 files changed, 403 insertions(+), 174 deletions(-)
>  delete mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
>  create mode 100644 tools/testing/selftests/kvm/aarch64/psci_test.c
> 
> -- 
> 2.33.0.685.g46640cef36-goog
>