Kernel KVM virtualization development
 help / color / mirror / Atom feed
* [PATCH v4 0/4] KVM: PPC: Expose CPU compatibility modes for nested guests
@ 2026-06-16 12:33 Amit Machhiwal
  2026-06-16 12:33 ` [PATCH v4 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl Amit Machhiwal
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Amit Machhiwal @ 2026-06-16 12:33 UTC (permalink / raw)
  To: linuxppc-dev, Madhavan Srinivasan
  Cc: Vaibhav Jain, Amit Machhiwal, Anushree Mathur, Paolo Bonzini,
	Nicholas Piggin, Michael Ellerman, Christophe Leroy (CS GROUP),
	Jonathan Corbet, Shuah Khan, kvm, linux-kernel, linux-doc, lkp

On POWER systems, newer processor generations can operate in compatibility
modes corresponding to earlier generations (e.g., a Power11 system running
in Power10 compatibility mode). In such cases, the effective CPU level
exposed to guests differs from the physical processor generation.

This creates a problem for nested virtualization. When booting a nested KVM
guest (L2) inside a host KVM guest (L1) running in a compatibility mode,
userspace (e.g., QEMU) may derive the CPU model from the raw hardware PVR
and attempt to configure the nested guest accordingly. However, the L1
partition is constrained by the compatibility level negotiated with the
hypervisor (L0), and requests exceeding that level are rejected, leading to
guest boot failures such as:

  KVM-NESTEDv2: couldn't set guest wide elements

This series provides a mechanism for userspace to query the effective CPU
compatibility modes supported by the host, so it can select an appropriate
CPU model for nested guests.

To achieve this, the series introduces a new KVM capability and ioctl
(KVM_CAP_PPC_COMPAT_CAPS / KVM_PPC_GET_COMPAT_CAPS) that expose the
compatibility modes supported by the host.

Why a new UAPI?
---------------
While cpu-version is available in /proc/device-tree/cpus/<cpu#>/cpu-version
on both L1 booted on PowerNV and PowerVM LPARs, the UAPI approach is
preferable for several reasons:

1. pHYP (L0) capabilities: On PowerVM, we need to rely on capabilities
   negotiated with pHYP in KVM, not just device tree properties. The
   cpu-version property depicts the current compat mode but doesn't point
   to what all compat modes are supported for the nested guest.

2. procfs dependency: Not all systems run with procfs enabled (CONFIG_PROC_FS
   is optional). Minimal configurations like buildroot might disable it, but
   KVM ioctl works regardless since it accesses kernel data structures
   directly.

3. Kernel validation: The kernel validates and normalizes the compatibility
   information, ensuring userspace gets validated, consistent data.

4. Abstraction & stability: /proc/device-tree is an implementation detail.
   The UAPI provides a stable interface that won't break if the underlying
   mechanism changes.

5. Semantic clarity: KVM_PPC_GET_COMPAT_CAPS clearly expresses what
   compatibility modes can be used for KVM guests, vs. parsing device tree
   which requires understanding the semantic meaning of cpu-version.

The implementation supports both:

  - PowerVM (nested API v2), where compatibility information is obtained
    via the H_GUEST_GET_CAPABILITIES hypercall.
  - PowerNV (nested API v1), where compatibility is derived from the device
    tree ("cpu-version") representing the effective processor compatibility
    level.

This allows userspace (e.g., QEMU) to select a CPU model consistent with
the host compatibility mode, avoiding mismatches and enabling successful
nested guest boot.

Changes in v4:
  - Added 'size' field to struct kvm_ppc_compat_caps for forward
    compatibility and ABI extensibility
  - Implemented size validation in ioctl handler to ensure correct structure
    size from userspace
  - Introduced KVM-specific capability constants (KVM_PPC_COMPAT_CAP_POWER9/
    10/11) instead of exposing hypervisor-internal H_GUEST_CAP_* constants
  - Added capability masking using KVM_PPC_COMPAT_BITMASK to ensure only
    supported processor modes are exposed
  - Enhanced error handling with comprehensive error codes (EINVAL, EFAULT,
    ENOTTY) and detailed documentation
  - Removed Tested-by tags pending re-testing with v4 changes
  - Separated validation patch (patch 1 from v3) and sent independently [1]

Note: This series is built on top of patches [1] and [2] which must be
applied first. Patch [1] ensures arch_compat is validated against the host
compatibility mode before this series adds the capability query mechanism.
Patch [2] sets CPU_FTR_P11_PVR for Power11 and later processors, which is
needed for proper CPU feature detection in dt-cpu-ftrs environments.

Changes in v3:
  - Added "Why a new UAPI?" section to cover letter addressing questions
    about the need for a new UAPI vs. using existing mechanisms like
    /proc/device-tree
  - Fixed initialization of 'r' in KVM_PPC_GET_COMPAT_CAPS ioctl handler
    from 0 to -ENOTTY for proper error handling when the operation is not
    supported
  - Added Vaibhav's "Suggested-by" tags
  - Have retained Anushree's "Tested-by" tags as no major code changes
  - Fixed documentation build warning reported by kernel test robot and
    added "Reported-by" and "Closes" tags to patch 5

Changes in v2:
  - Squashed patches 2 and 3 from v1 (capability introduction and ioctl
    wiring) into a single patch for better logical grouping
  - Changed kvm_ppc_compat_caps.flags from __u32 to __u64 for consistency
    and future extensibility
  - Addressed other review comments
  - Improved commit messages with clearer explanations of the changes

Patch summary:
  [1/4] Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
  [2/4] Implement capability retrieval for PowerVM (API v2)
  [3/4] Add PowerNV support (API v1)
  [4/4] Document the new ioctl

Testing (with QEMU v3 patches and on top of patches [1] and [2]):

KVM APIv1 Testing
=================
  On P10 PowerNV machine (L0)
  ---------------------------
    - P10 L1 KVM guest -> works
      - P10 nested L2 KVM guest -> works
      - P9 compat nested L2 KVM guest -> works
    - P9 compat L1 KVM guest -> works
      - P9 nested L2 KVM guest -> works
  
  On Powernv11 TCG Guest (L0)
  ---------------------------
    - P11 L1 KVM guest -> works
      - P11 L2 KVM guest -> works
    - P10 compat L1 KVM guest -> works
      - P10 L2 KVM guest -> works
    - P9  compat L1 KVM guest -> works
      - P9 L2 KVM guest -> works

KVM APIv2 Testing
=================
  On P11 PowerVM LPAR (L1)
  ------------------------
    - P11 L2 KVM guest -> works
    - P10 compat L2 KVM guest -> works
  
  On P11 LPAR in P10 compat (L1)
  ------------------------------
    - P10 (host compat) L2 KVM guest -> works

  On P10 PowerVM LPAR (L1)
  ------------------------
    - P10 L2 KVM guest -> works

With this series, nested guests boot successfully in configurations where
they previously failed due to compatibility mismatches.

Related QEMU series:
--------------------
A corresponding QEMU v3 series adds support for querying and using these
compatibility capabilities when configuring nested KVM guests:

v3: https://lore.kernel.org/all/20260616113915.25589-1-amachhiw@linux.ibm.com/
v2: https://lore.kernel.org/all/20260502140021.69712-1-amachhiw@linux.ibm.com/
v1: https://lore.kernel.org/all/20260430061333.37905-1-amachhiw@linux.ibm.com/

Previous versions:
------------------
v3: https://lore.kernel.org/linuxppc-dev/20260522152744.55251-1-amachhiw@linux.ibm.com/
v2: https://lore.kernel.org/linuxppc-dev/20260513100755.83215-1-amachhiw@linux.ibm.com/
v1: https://lore.kernel.org/linuxppc-dev/20260430054906.94431-1-amachhiw@linux.ibm.com/

References:
-----------
[1] https://lore.kernel.org/all/20260609053327.61563-1-amachhiw@linux.ibm.com/
[2] https://lore.kernel.org/all/20260614173437.26352-1-amachhiw@linux.ibm.com/

Amit Machhiwal (4):
  KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
  KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM
    on PowerVM
  KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM
    on PowerNV
  KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl

 Documentation/virt/kvm/api.rst      | 47 ++++++++++++++++++++++++++
 arch/powerpc/include/asm/kvm_ppc.h  |  1 +
 arch/powerpc/include/uapi/asm/kvm.h | 16 +++++++++
 arch/powerpc/kvm/book3s_hv.c        | 52 +++++++++++++++++++++++++++++
 arch/powerpc/kvm/powerpc.c          | 35 +++++++++++++++++++
 include/uapi/linux/kvm.h            |  4 +++
 6 files changed, 155 insertions(+)


base-commit: 8b308f96484e37d92d2fc6b72b091f60496c000e
prerequisite-patch-id: ce9521668d549f2bd731d321a38f720b789e0b4e
prerequisite-patch-id: 4662f01d2101cfae8502f04290658deed60eec26
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-06-16 13:00 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-16 12:33 [PATCH v4 0/4] KVM: PPC: Expose CPU compatibility modes for nested guests Amit Machhiwal
2026-06-16 12:33 ` [PATCH v4 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl Amit Machhiwal
2026-06-16 12:45   ` sashiko-bot
2026-06-16 12:33 ` [PATCH v4 2/4] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM Amit Machhiwal
2026-06-16 13:00   ` sashiko-bot
2026-06-16 12:33 ` [PATCH v4 3/4] KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV Amit Machhiwal
2026-06-16 12:47   ` sashiko-bot
2026-06-16 12:33 ` [PATCH v4 4/4] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl Amit Machhiwal
2026-06-16 12:50   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox