[PATCH 0/2] ppc/kvm: Handle CPU compatibility mode correctly for nested guests

[PATCH 0/2] ppc/kvm: Handle CPU compatibility mode correctly for nested guests

[Amit Machhiwal]

On POWER systems, newer processor generations can operate in compatibility
modes corresponding to earlier generations (e.g., a Power11 system running
in Power10 compatibility mode). In such cases, the effective CPU level
exposed to guests differs from the physical processor generation.

This creates issues for nested virtualization. When booting a nested KVM
guest, QEMU may derive the CPU model from the raw hardware PVR and attempt
to configure the guest accordingly. However, the host is constrained by the
compatibility level negotiated with the hypervisor, and requests exceeding
that level are rejected by KVM, leading to guest boot failures such as:

  KVM-NESTEDv2: couldn't set guest wide elements

This series addresses the issue in two ways:

1. Do not silently fall back to raw mode when KVM rejects a requested
   compatibility level during CAS. Instead, propagate the error so invalid
   configurations are visible and fail early.

2. Query the effective CPU compatibility modes supported by the host via
   KVM and use this information to select an appropriate CPU model for
   nested guests.

With these changes, QEMU avoids masking KVM errors and ensures that nested
guests are configured with CPU models consistent with the host
compatibility mode, allowing them to boot correctly.

Patch summary:
  [1/2] hw/ppc/spapr: Do not fallback to raw mode when KVM rejects compat
  [2/2] target/ppc/kvm: Use host compatibility mode for nested guests

Tested on:
  - Power11 pSeries LPAR in Power10 compatibility mode
  - Power10 PowerNV and QEMU PowerNV 11 TCG L0 host

Note: The corresponding Linux patches have been posted [1]

[1] https://lore.kernel.org/all/20260430054906.94431-1-amachhiw@linux.ibm.com/

Amit Machhiwal (2):
  hw/ppc/spapr: Do not fallback to raw mode when KVM rejects compat
  target/ppc/kvm: Use host compatibility mode for nested guests

 hw/ppc/spapr_hcall.c |  9 ++++++++
 target/ppc/kvm.c     | 52 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)


base-commit: 759c456b1d22fe4083c8b384da27d3f56fd53f82
--
2.50.1

[PATCH 2/2] target/ppc/kvm: Use host compatibility mode for nested guests

[Amit Machhiwal]

On POWER systems, the host CPU may run in a compatibility mode (e.g.,
a Power11 processor operating in Power10 compatibility mode). In such
cases, the effective CPU level exposed to guests differs from the
physical processor generation.

When running nested KVM guests, QEMU currently derives the host CPU type
using mfpvr(), which reflects the physical processor version. This can
result in a mismatch between the CPU model used by QEMU and the
compatibility mode enforced by the host, leading to guest boot failures.

In particular, booting a nested guest on a Power11 LPAR configured in
Power10 compatibility mode fails with errors such as:

  KVM-NESTEDv2: couldn't set guest wide elements

This occurs because QEMU selects a CPU model based on the physical
processor version, while the host operates in a lower compatibility
mode. As a result, KVM rejects the requested compatibility level during
guest initialization.

Add support for querying host compatibility capabilities via the
KVM_PPC_GET_COMPAT_CAPS ioctl and derive the effective PVR based on the
compatibility mode reported by KVM. When available, use this
compatibility PVR instead of the raw hardware PVR when selecting
the CPU model.

If the capability is not supported or the query fails, fall back to the
existing behavior.

This ensures that QEMU selects a CPU model consistent with the host
compatibility mode, allowing nested guests to boot correctly.

Signed-off-by: Amit Machhiwal <amachhiw@linux.ibm.com>
---
 target/ppc/kvm.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 25c28ad089c6..d384161f780b 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -2602,11 +2602,63 @@ bool kvmppc_supports_ail_3(void)
     return cap_ail_mode_3;
 }
 
+static target_ulong kvmppc_get_compat_caps(void)
+{
+    struct kvm_ppc_compat_caps host_compat;
+    target_ulong host_caps;
+    int ret;
+
+    if (!kvm_check_extension(kvm_state, KVM_CAP_PPC_COMPAT_CAPS)) {
+        return 0;
+    }
+
+    ret = kvm_vm_ioctl(kvm_state, KVM_PPC_GET_COMPAT_CAPS, &host_compat);
+    if (ret < 0) {
+        fprintf(stderr, "KVM: failed to get host capabilities\n");
+        return 0;
+    }
+
+    host_caps = host_compat.compat_capabilities;
+    return host_caps;
+}
+
+static uint32_t kvm_ppc_host_compat_pvr(void)
+{
+    uint32_t compat_host_pvr = 0;
+    int cap_idx = 0;
+    target_ulong host_caps = kvmppc_get_compat_caps();
+
+    if (host_caps) {
+        cap_idx = 63 - __builtin_ctzll(host_caps);
+        switch (cap_idx) {
+        case H_GUEST_CAP_P9_MODE_BMAP:
+            compat_host_pvr = CPU_POWERPC_POWER9_DD22;
+            break;
+        case H_GUEST_CAP_P10_MODE_BMAP:
+            compat_host_pvr = CPU_POWERPC_POWER10_DD20;
+            break;
+        case H_GUEST_CAP_P11_MODE_BMAP:
+            compat_host_pvr = CPU_POWERPC_POWER11_DD20;
+            break;
+        default:
+            break;
+        }
+    }
+
+    return compat_host_pvr;
+}
+
 PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
 {
     uint32_t host_pvr = mfpvr();
+    uint32_t compat_host_pvr;
     PowerPCCPUClass *pvr_pcc;
 
+    compat_host_pvr = kvm_ppc_host_compat_pvr();
+    if (compat_host_pvr) {
+        host_pvr = compat_host_pvr;
+    }
+
     pvr_pcc = ppc_cpu_class_by_pvr(host_pvr);
     if (pvr_pcc == NULL) {
         pvr_pcc = ppc_cpu_class_by_pvr_mask(host_pvr);
-- 
2.50.1 (Apple Git-155)

Re: [PATCH 2/2] target/ppc/kvm: Use host compatibility mode for nested guests

[Amit Machhiwal]

I found a compilation issue with this patch when building for ppc32. The
host PVR constants (CPU_POWERPC_POWER9_DD22, etc.) are only defined under
TARGET_PPC64, and since POWER9/10/11 compatibility modes are 64-bit only,
the code should be guarded with #if defined(TARGET_PPC64). I'll send v2
shortly with this fix.

~Amit

On 2026/04/30 11:43 AM, Amit Machhiwal wrote:
> On POWER systems, the host CPU may run in a compatibility mode (e.g.,
> a Power11 processor operating in Power10 compatibility mode). In such
> cases, the effective CPU level exposed to guests differs from the
> physical processor generation.
> 
> When running nested KVM guests, QEMU currently derives the host CPU type
> using mfpvr(), which reflects the physical processor version. This can
> result in a mismatch between the CPU model used by QEMU and the
> compatibility mode enforced by the host, leading to guest boot failures.
> 
> In particular, booting a nested guest on a Power11 LPAR configured in
> Power10 compatibility mode fails with errors such as:
> 
>   KVM-NESTEDv2: couldn't set guest wide elements
> 
> This occurs because QEMU selects a CPU model based on the physical
> processor version, while the host operates in a lower compatibility
> mode. As a result, KVM rejects the requested compatibility level during
> guest initialization.
> 
> Add support for querying host compatibility capabilities via the
> KVM_PPC_GET_COMPAT_CAPS ioctl and derive the effective PVR based on the
> compatibility mode reported by KVM. When available, use this
> compatibility PVR instead of the raw hardware PVR when selecting
> the CPU model.
> 
> If the capability is not supported or the query fails, fall back to the
> existing behavior.
> 
> This ensures that QEMU selects a CPU model consistent with the host
> compatibility mode, allowing nested guests to boot correctly.
> 
> Signed-off-by: Amit Machhiwal <amachhiw@linux.ibm.com>
> ---
>  target/ppc/kvm.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 52 insertions(+)
> 
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 25c28ad089c6..d384161f780b 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -2602,11 +2602,63 @@ bool kvmppc_supports_ail_3(void)
>      return cap_ail_mode_3;
>  }
>  
> +static target_ulong kvmppc_get_compat_caps(void)
> +{
> +    struct kvm_ppc_compat_caps host_compat;
> +    target_ulong host_caps;
> +    int ret;
> +
> +    if (!kvm_check_extension(kvm_state, KVM_CAP_PPC_COMPAT_CAPS)) {
> +        return 0;
> +    }
> +
> +    ret = kvm_vm_ioctl(kvm_state, KVM_PPC_GET_COMPAT_CAPS, &host_compat);
> +    if (ret < 0) {
> +        fprintf(stderr, "KVM: failed to get host capabilities\n");
> +        return 0;
> +    }
> +
> +    host_caps = host_compat.compat_capabilities;
> +    return host_caps;
> +}
> +
> +static uint32_t kvm_ppc_host_compat_pvr(void)
> +{
> +    uint32_t compat_host_pvr = 0;
> +    int cap_idx = 0;
> +    target_ulong host_caps = kvmppc_get_compat_caps();
> +
> +    if (host_caps) {
> +        cap_idx = 63 - __builtin_ctzll(host_caps);
> +        switch (cap_idx) {
> +        case H_GUEST_CAP_P9_MODE_BMAP:
> +            compat_host_pvr = CPU_POWERPC_POWER9_DD22;
> +            break;
> +        case H_GUEST_CAP_P10_MODE_BMAP:
> +            compat_host_pvr = CPU_POWERPC_POWER10_DD20;
> +            break;
> +        case H_GUEST_CAP_P11_MODE_BMAP:
> +            compat_host_pvr = CPU_POWERPC_POWER11_DD20;
> +            break;
> +        default:
> +            break;
> +        }
> +    }
> +
> +    return compat_host_pvr;
> +}
> +
>  PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
>  {
>      uint32_t host_pvr = mfpvr();
> +    uint32_t compat_host_pvr;
>      PowerPCCPUClass *pvr_pcc;
>  
> +    compat_host_pvr = kvm_ppc_host_compat_pvr();
> +    if (compat_host_pvr) {
> +        host_pvr = compat_host_pvr;
> +    }
> +
>      pvr_pcc = ppc_cpu_class_by_pvr(host_pvr);
>      if (pvr_pcc == NULL) {
>          pvr_pcc = ppc_cpu_class_by_pvr_mask(host_pvr);
> -- 
> 2.50.1 (Apple Git-155)
>