[PATCH 0/7] target/i386: VM type infrastructure and KVM_SEV_INIT2 support

[PATCH 0/7] target/i386: VM type infrastructure and KVM_SEV_INIT2 support

[Paolo Bonzini]

This series adds another vendor-neutral part of the SEV-SNP/TDX support
patches, namely support for KVM_CAP_VM_TYPES.  In Linux 6.10 this will
also be available for SEV and SEV-ES, so introduce it now already.

Also, Linux 6.10 will _not_ allow KVM_GET/SET_* ioctls for VMs with
encrypted state and a VM type other than KVM_X86_DEFAULT_VM, so prepare
for that.

The patches are not yet available in kvm.git, hence the hackish
linux-headers update in patch 1.  Apart from that, however, the API
should be final.

Tested by booting a SEV-ES guest.

Paolo

Based-on: <20240229060038.606591-1-xiaoyao.li@intel.com>

Paolo Bonzini (6):
  linux-headers hack
  runstate: skip initial CPU reset if reset is not actually possible
  KVM: track whether guest state is encrypted
  KVM: remove kvm_arch_cpu_check_are_resettable
  target/i386: introduce x86-confidential-guest
  target/i386: SEV: use KVM_SEV_INIT2 if possible

Xiaoyao Li (1):
  target/i386: Implement mc->kvm_type() to get VM type

 include/sysemu/kvm.h             | 12 ++-----
 include/sysemu/kvm_int.h         |  1 +
 linux-headers/asm-x86/kvm.h      |  8 +++++
 linux-headers/linux/kvm.h        |  2 ++
 target/i386/confidential-guest.h | 59 ++++++++++++++++++++++++++++++++
 target/i386/kvm/kvm_i386.h       |  2 ++
 accel/kvm/kvm-accel-ops.c        |  2 +-
 accel/kvm/kvm-all.c              | 19 ++++++----
 hw/i386/x86.c                    |  6 ++++
 system/runstate.c                | 15 +++++++-
 target/arm/kvm.c                 |  5 ---
 target/i386/confidential-guest.c | 33 ++++++++++++++++++
 target/i386/kvm/kvm.c            | 49 +++++++++++++++++++++++---
 target/i386/sev.c                | 48 ++++++++++++++++++++++----
 target/loongarch/kvm/kvm.c       |  5 ---
 target/mips/kvm.c                |  5 ---
 target/ppc/kvm.c                 |  5 ---
 target/riscv/kvm/kvm-cpu.c       |  5 ---
 target/s390x/kvm/kvm.c           |  5 ---
 target/i386/meson.build          |  2 +-
 20 files changed, 226 insertions(+), 62 deletions(-)
 create mode 100644 target/i386/confidential-guest.h
 create mode 100644 target/i386/confidential-guest.c

-- 
2.44.0

[PATCH 1/7] linux-headers hack

[Paolo Bonzini]

To be replaced by update to kvm/next branch from Linux 6.9, once the
new API is committed.
---
 linux-headers/asm-x86/kvm.h | 8 ++++++++
 linux-headers/linux/kvm.h   | 2 ++
 2 files changed, 10 insertions(+)

diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index 003fb745347..8f58c32d37d 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -562,5 +562,13 @@ struct kvm_pmu_event_filter {
 
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_SW_PROTECTED_VM	1
+#define KVM_X86_SEV_VM          2
+#define KVM_X86_SEV_ES_VM       3
+
+struct kvm_sev_init {
+        __u64 vmsa_features;
+        __u32 flags;
+        __u32 pad[9];
+};
 
 #endif /* _ASM_X86_KVM_H */
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 17839229b2a..5fd84fd7d0c 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -1865,6 +1865,8 @@ enum sev_cmd_id {
 	/* Guest Migration Extension */
 	KVM_SEV_SEND_CANCEL,
 
+	KVM_SEV_INIT2,
+
 	KVM_SEV_NR_MAX,
 };
 
-- 
2.44.0

[PATCH 2/7] runstate: skip initial CPU reset if reset is not actually possible

[Paolo Bonzini]

Right now, the system reset is concluded by a call to
cpu_synchronize_all_post_reset() in order to sync any changes
that the machine reset callback applied to the CPU state.

However, for VMs with encrypted state such as SEV-ES guests (currently
the only case of guests with non-resettable CPUs) this cannot be done,
because guest state has already been finalized by machine-init-done notifiers.
cpu_synchronize_all_post_reset() does nothing on these guests, and actually
we would like to make it fail if called once guest has been encrypted.
So, assume that boards that support non-resettable CPUs do not touch
CPU state and that all such setup is done before, at the time of
cpu_synchronize_all_post_init().

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 system/runstate.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/system/runstate.c b/system/runstate.c
index d6ab860ecaa..843e2b9853d 100644
--- a/system/runstate.c
+++ b/system/runstate.c
@@ -501,7 +501,20 @@ void qemu_system_reset(ShutdownCause reason)
     default:
         qapi_event_send_reset(shutdown_caused_by_guest(reason), reason);
     }
-    cpu_synchronize_all_post_reset();
+
+    /*
+     * Some boards us the machine reset callback to point CPUs to the firmware
+     * entry point.  Assume that this is not the case for boards that support
+     * non-resettable CPUs (currently used only for confidential guests), in
+     * which case cpu_synchronize_all_post_init() is enough because
+     * it does _more_  than cpu_synchronize_all_post_reset().
+     */
+    if (cpus_are_resettable()) {
+        cpu_synchronize_all_post_reset();
+    } else {
+        assert(runstate_check(RUN_STATE_PRELAUNCH));
+    }
+
     vm_set_suspended(false);
 }
 
-- 
2.44.0

[PATCH 3/7] KVM: track whether guest state is encrypted

[Paolo Bonzini]

So far, KVM has allowed KVM_GET/SET_* ioctls to execute even if the
guest state is encrypted, in which case they do nothing.  For the new
API using VM types, instead, the ioctls will fail which is a safer and
more robust approach.

The new API will be the only one available for SEV-SNP and TDX, but it
is also usable for SEV and SEV-ES.  In preparation for that, require
architecture-specific KVM code to communicate the point at which guest
state is protected (which must be after kvm_cpu_synchronize_post_init(),
though that might change in the future in order to suppor migration).
From that point, skip reading registers so that cpu->vcpu_dirty is
never true: if it ever becomes true, kvm_arch_put_registers() will
fail miserably.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/sysemu/kvm.h     |  2 ++
 include/sysemu/kvm_int.h |  1 +
 accel/kvm/kvm-all.c      | 14 ++++++++++++--
 target/i386/sev.c        |  1 +
 4 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index fad9a7e8ff3..302e8f6f1e5 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -539,6 +539,8 @@ bool kvm_dirty_ring_enabled(void);
 
 uint32_t kvm_dirty_ring_size(void);
 
+void kvm_mark_guest_state_protected(void);
+
 /**
  * kvm_hwpoisoned_mem - indicate if there is any hwpoisoned page
  * reported for the VM.
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index 882e37e12c5..3496be7997a 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -87,6 +87,7 @@ struct KVMState
     bool kernel_irqchip_required;
     OnOffAuto kernel_irqchip_split;
     bool sync_mmu;
+    bool guest_state_protected;
     uint64_t manual_dirty_log_protect;
     /* The man page (and posix) say ioctl numbers are signed int, but
      * they're not.  Linux, glibc and *BSD all treat ioctl numbers as
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index a8cecd040eb..05fa3533c66 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2698,7 +2698,7 @@ bool kvm_cpu_check_are_resettable(void)
 
 static void do_kvm_cpu_synchronize_state(CPUState *cpu, run_on_cpu_data arg)
 {
-    if (!cpu->vcpu_dirty) {
+    if (!cpu->vcpu_dirty && !kvm_state->guest_state_protected) {
         int ret = kvm_arch_get_registers(cpu);
         if (ret) {
             error_report("Failed to get registers: %s", strerror(-ret));
@@ -2712,7 +2712,7 @@ static void do_kvm_cpu_synchronize_state(CPUState *cpu, run_on_cpu_data arg)
 
 void kvm_cpu_synchronize_state(CPUState *cpu)
 {
-    if (!cpu->vcpu_dirty) {
+    if (!cpu->vcpu_dirty && !kvm_state->guest_state_protected) {
         run_on_cpu(cpu, do_kvm_cpu_synchronize_state, RUN_ON_CPU_NULL);
     }
 }
@@ -2747,6 +2747,11 @@ static void do_kvm_cpu_synchronize_post_init(CPUState *cpu, run_on_cpu_data arg)
 
 void kvm_cpu_synchronize_post_init(CPUState *cpu)
 {
+    /*
+     * This runs before the machine_init_done notifiers, and is the last
+     * opportunity to synchronize the state of confidential guests.
+     */
+    assert(!kvm_state->guest_state_protected);
     run_on_cpu(cpu, do_kvm_cpu_synchronize_post_init, RUN_ON_CPU_NULL);
 }
 
@@ -4094,3 +4099,8 @@ void query_stats_schemas_cb(StatsSchemaList **result, Error **errp)
         query_stats_schema_vcpu(first_cpu, &stats_args);
     }
 }
+
+void kvm_mark_guest_state_protected(void)
+{
+    kvm_state->guest_state_protected = true;
+}
diff --git a/target/i386/sev.c b/target/i386/sev.c
index b8f79d34d19..c49a8fd55eb 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -755,6 +755,7 @@ sev_launch_get_measure(Notifier *notifier, void *unused)
         if (ret) {
             exit(1);
         }
+        kvm_mark_guest_state_protected();
     }
 
     /* query the measurement blob length */
-- 
2.44.0

[PATCH 4/7] KVM: remove kvm_arch_cpu_check_are_resettable

[Paolo Bonzini]

Board reset requires writing a fresh CPU state.  As far as KVM is
concerned, the only thing that blocks reset is that CPU state is
encrypted; therefore, kvm_cpus_are_resettable() can simply check
if that is the case.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 include/sysemu/kvm.h       | 10 ----------
 accel/kvm/kvm-accel-ops.c  |  2 +-
 accel/kvm/kvm-all.c        |  5 -----
 target/arm/kvm.c           |  5 -----
 target/i386/kvm/kvm.c      |  5 -----
 target/loongarch/kvm/kvm.c |  5 -----
 target/mips/kvm.c          |  5 -----
 target/ppc/kvm.c           |  5 -----
 target/riscv/kvm/kvm-cpu.c |  5 -----
 target/s390x/kvm/kvm.c     |  5 -----
 10 files changed, 1 insertion(+), 51 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 302e8f6f1e5..54f4d83a370 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -525,16 +525,6 @@ int kvm_get_one_reg(CPUState *cs, uint64_t id, void *target);
 /* Notify resamplefd for EOI of specific interrupts. */
 void kvm_resample_fd_notify(int gsi);
 
-/**
- * kvm_cpu_check_are_resettable - return whether CPUs can be reset
- *
- * Returns: true: CPUs are resettable
- *          false: CPUs are not resettable
- */
-bool kvm_cpu_check_are_resettable(void);
-
-bool kvm_arch_cpu_check_are_resettable(void);
-
 bool kvm_dirty_ring_enabled(void);
 
 uint32_t kvm_dirty_ring_size(void);
diff --git a/accel/kvm/kvm-accel-ops.c b/accel/kvm/kvm-accel-ops.c
index b3c946dc4b4..74e3c5785b5 100644
--- a/accel/kvm/kvm-accel-ops.c
+++ b/accel/kvm/kvm-accel-ops.c
@@ -82,7 +82,7 @@ static bool kvm_vcpu_thread_is_idle(CPUState *cpu)
 
 static bool kvm_cpus_are_resettable(void)
 {
-    return !kvm_enabled() || kvm_cpu_check_are_resettable();
+    return !kvm_enabled() || !kvm_state->guest_state_protected;
 }
 
 #ifdef KVM_CAP_SET_GUEST_DEBUG
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 05fa3533c66..a05dea23133 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2691,11 +2691,6 @@ void kvm_flush_coalesced_mmio_buffer(void)
     s->coalesced_flush_in_progress = false;
 }
 
-bool kvm_cpu_check_are_resettable(void)
-{
-    return kvm_arch_cpu_check_are_resettable();
-}
-
 static void do_kvm_cpu_synchronize_state(CPUState *cpu, run_on_cpu_data arg)
 {
     if (!cpu->vcpu_dirty && !kvm_state->guest_state_protected) {
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index ab85d628a8b..21ebbf3b8f8 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -1598,11 +1598,6 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
     return (data - 32) & 0xffff;
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return true;
-}
-
 static void kvm_arch_get_eager_split_size(Object *obj, Visitor *v,
                                           const char *name, void *opaque,
                                           Error **errp)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index de10155b37a..0ec69109a2b 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -5614,11 +5614,6 @@ bool kvm_has_waitpkg(void)
     return has_msr_umwait;
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return !sev_es_enabled();
-}
-
 #define ARCH_REQ_XCOMP_GUEST_PERM       0x1025
 
 void kvm_request_xsave_components(X86CPU *cpu, uint64_t mask)
diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
index d630cc39cb2..8224d943331 100644
--- a/target/loongarch/kvm/kvm.c
+++ b/target/loongarch/kvm/kvm.c
@@ -733,11 +733,6 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cs)
     return true;
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return true;
-}
-
 int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
 {
     int ret = 0;
diff --git a/target/mips/kvm.c b/target/mips/kvm.c
index 6c52e59f55d..a631ab544f5 100644
--- a/target/mips/kvm.c
+++ b/target/mips/kvm.c
@@ -1273,11 +1273,6 @@ int kvm_arch_get_default_type(MachineState *machine)
     return -1;
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return true;
-}
-
 void kvm_arch_accel_class_init(ObjectClass *oc)
 {
 }
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 8231feb2d45..63930d4a77d 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -2956,11 +2956,6 @@ void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset)
     }
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return true;
-}
-
 void kvm_arch_accel_class_init(ObjectClass *oc)
 {
 }
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index cda7d78a778..135d87dc3f5 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -1466,11 +1466,6 @@ void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level)
     }
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return true;
-}
-
 static int aia_mode;
 
 static const char *kvm_aia_mode_str(uint64_t mode)
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 4ce809c5d46..4dcd757cdcc 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2622,11 +2622,6 @@ void kvm_s390_stop_interrupt(S390CPU *cpu)
     kvm_s390_vcpu_interrupt(cpu, &irq);
 }
 
-bool kvm_arch_cpu_check_are_resettable(void)
-{
-    return true;
-}
-
 int kvm_s390_get_zpci_op(void)
 {
     return cap_zpci_op;
-- 
2.44.0

[PATCH 5/7] target/i386: introduce x86-confidential-guest

[Paolo Bonzini]

Introduce a common superclass for x86 confidential guest implementations.
It will extend ConfidentialGuestSupportClass with a method that provides
the VM type to be passed to KVM_CREATE_VM.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/confidential-guest.h | 40 ++++++++++++++++++++++++++++++++
 target/i386/confidential-guest.c | 33 ++++++++++++++++++++++++++
 target/i386/sev.c                |  6 ++---
 target/i386/meson.build          |  2 +-
 4 files changed, 77 insertions(+), 4 deletions(-)
 create mode 100644 target/i386/confidential-guest.h
 create mode 100644 target/i386/confidential-guest.c

diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
new file mode 100644
index 00000000000..ca12d5a8fba
--- /dev/null
+++ b/target/i386/confidential-guest.h
@@ -0,0 +1,40 @@
+/*
+ * x86-specific confidential guest methods.
+ *
+ * Copyright (c) 2024 Red Hat Inc.
+ *
+ * Authors:
+ *  Paolo Bonzini <pbonzini@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#ifndef TARGET_I386_CG_H
+#define TARGET_I386_CG_H
+
+#include "qom/object.h"
+
+#include "exec/confidential-guest-support.h"
+
+#define TYPE_X86_CONFIDENTIAL_GUEST "x86-confidential-guest"
+
+OBJECT_DECLARE_TYPE(X86ConfidentialGuest,
+                    X86ConfidentialGuestClass,
+                    X86_CONFIDENTIAL_GUEST)
+
+struct X86ConfidentialGuest {
+    /* <private> */
+    ConfidentialGuestSupport parent_obj;
+};
+
+/**
+ * X86ConfidentialGuestClass:
+ *
+ * Class to be implemented by confidential-guest-support concrete objects
+ * for the x86 target.
+ */
+struct X86ConfidentialGuestClass {
+    /* <private> */
+    ConfidentialGuestSupportClass parent;
+};
+#endif
diff --git a/target/i386/confidential-guest.c b/target/i386/confidential-guest.c
new file mode 100644
index 00000000000..b3727845adc
--- /dev/null
+++ b/target/i386/confidential-guest.c
@@ -0,0 +1,33 @@
+/*
+ * QEMU Confidential Guest support
+ *
+ * Copyright (C) 2024 Red Hat, Inc.
+ *
+ * Authors:
+ *  Paolo Bonzini <pbonzini@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later.  See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "confidential-guest.h"
+
+OBJECT_DEFINE_ABSTRACT_TYPE(X86ConfidentialGuest,
+                            x86_confidential_guest,
+                            X86_CONFIDENTIAL_GUEST,
+                            CONFIDENTIAL_GUEST_SUPPORT)
+
+static void x86_confidential_guest_class_init(ObjectClass *oc, void *data)
+{
+}
+
+static void x86_confidential_guest_init(Object *obj)
+{
+}
+
+static void x86_confidential_guest_finalize(Object *obj)
+{
+}
diff --git a/target/i386/sev.c b/target/i386/sev.c
index c49a8fd55eb..ebe36d4c10c 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -35,7 +35,7 @@
 #include "monitor/monitor.h"
 #include "monitor/hmp-target.h"
 #include "qapi/qapi-commands-misc-target.h"
-#include "exec/confidential-guest-support.h"
+#include "confidential-guest.h"
 #include "hw/i386/pc.h"
 #include "exec/address-spaces.h"
 
@@ -54,7 +54,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
  *         -machine ...,memory-encryption=sev0
  */
 struct SevGuestState {
-    ConfidentialGuestSupport parent_obj;
+    X86ConfidentialGuest parent_obj;
 
     /* configuration parameters */
     char *sev_device;
@@ -1372,7 +1372,7 @@ sev_guest_instance_init(Object *obj)
 
 /* sev guest info */
 static const TypeInfo sev_guest_info = {
-    .parent = TYPE_CONFIDENTIAL_GUEST_SUPPORT,
+    .parent = TYPE_X86_CONFIDENTIAL_GUEST,
     .name = TYPE_SEV_GUEST,
     .instance_size = sizeof(SevGuestState),
     .instance_finalize = sev_guest_finalize,
diff --git a/target/i386/meson.build b/target/i386/meson.build
index 7c74bfa8591..8abce725f86 100644
--- a/target/i386/meson.build
+++ b/target/i386/meson.build
@@ -6,7 +6,7 @@ i386_ss.add(files(
   'xsave_helper.c',
   'cpu-dump.c',
 ))
-i386_ss.add(when: 'CONFIG_SEV', if_true: files('host-cpu.c'))
+i386_ss.add(when: 'CONFIG_SEV', if_true: files('host-cpu.c', 'confidential-guest.c'))
 
 # x86 cpu type
 i386_ss.add(when: 'CONFIG_KVM', if_true: files('host-cpu.c'))
-- 
2.44.0

[PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Paolo Bonzini]

From: Xiaoyao Li <xiaoyao.li@intel.com>

KVM is introducing a new API to create confidential guests, which
will be used by TDX and SEV-SNP but is also available for SEV and
SEV-ES.  The API uses the VM type argument to KVM_CREATE_VM to
identify which confidential computing technology to use.

Since there are no other expected uses of VM types, delegate
mc->kvm_type() for x86 boards to the confidential-guest-support
object pointed to by ms->cgs.

For example, if a sev-guest object is specified to confidential-guest-support,
like,

  qemu -machine ...,confidential-guest-support=sev0 \
       -object sev-guest,id=sev0,...

it will check if a VM type KVM_X86_SEV_VM or KVM_X86_SEV_ES_VM
is supported, and if so use them together with the KVM_SEV_INIT2
function of the KVM_MEMORY_ENCRYPT_OP ioctl. If not, it will fall back to
KVM_SEV_INIT and KVM_SEV_ES_INIT.

This is a preparatory work towards TDX and SEV-SNP support, but it
will also enable support for VMSA features such as DebugSwap, which
are only available via KVM_SEV_INIT2.

Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/confidential-guest.h | 19 ++++++++++++++
 target/i386/kvm/kvm_i386.h       |  2 ++
 hw/i386/x86.c                    |  6 +++++
 target/i386/kvm/kvm.c            | 44 ++++++++++++++++++++++++++++++++
 4 files changed, 71 insertions(+)

diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
index ca12d5a8fba..532e172a60b 100644
--- a/target/i386/confidential-guest.h
+++ b/target/i386/confidential-guest.h
@@ -36,5 +36,24 @@ struct X86ConfidentialGuest {
 struct X86ConfidentialGuestClass {
     /* <private> */
     ConfidentialGuestSupportClass parent;
+
+    /* <public> */
+    int (*kvm_type)(X86ConfidentialGuest *cg);
 };
+
+/**
+ * x86_confidential_guest_kvm_type:
+ *
+ * Calls #X86ConfidentialGuestClass.unplug callback of @plug_handler.
+ */
+static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
+{
+    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
+
+    if (klass->kvm_type) {
+        return klass->kvm_type(cg);
+    } else {
+        return 0;
+    }
+}
 #endif
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index 30fedcffea3..02168122787 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -37,6 +37,7 @@ bool kvm_hv_vpindex_settable(void);
 bool kvm_enable_sgx_provisioning(KVMState *s);
 bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp);
 
+int kvm_get_vm_type(MachineState *ms, const char *vm_type);
 void kvm_arch_reset_vcpu(X86CPU *cs);
 void kvm_arch_after_reset_vcpu(X86CPU *cpu);
 void kvm_arch_do_init_vcpu(X86CPU *cs);
@@ -49,6 +50,7 @@ void kvm_request_xsave_components(X86CPU *cpu, uint64_t mask);
 
 #ifdef CONFIG_KVM
 
+bool kvm_is_vm_type_supported(int type);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index ffbda48917f..2d4b148cd25 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1389,6 +1389,11 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
     qapi_free_SgxEPCList(list);
 }
 
+static int x86_kvm_type(MachineState *ms, const char *vm_type)
+{
+    return kvm_enabled() ? kvm_get_vm_type(ms, vm_type) : 0;
+}
+
 static void x86_machine_initfn(Object *obj)
 {
     X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1413,6 +1418,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     mc->cpu_index_to_instance_props = x86_cpu_index_to_props;
     mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
     mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
+    mc->kvm_type = x86_kvm_type;
     x86mc->save_tsc_khz = true;
     x86mc->fwcfg_dma_enabled = true;
     nc->nmi_monitor_handler = x86_nmi;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 0ec69109a2b..e109648f260 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -31,6 +31,7 @@
 #include "sysemu/kvm_int.h"
 #include "sysemu/runstate.h"
 #include "kvm_i386.h"
+#include "../confidential-guest.h"
 #include "sev.h"
 #include "xen-emu.h"
 #include "hyperv.h"
@@ -161,6 +162,49 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];
 static RateLimit bus_lock_ratelimit_ctrl;
 static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
 
+static const char *vm_type_name[] = {
+    [KVM_X86_DEFAULT_VM] = "default",
+};
+
+bool kvm_is_vm_type_supported(int type)
+{
+    uint32_t machine_types;
+
+    /*
+     * old KVM doesn't support KVM_CAP_VM_TYPES but KVM_X86_DEFAULT_VM
+     * is always supported
+     */
+    if (type == KVM_X86_DEFAULT_VM) {
+        return true;
+    }
+
+    machine_types = kvm_check_extension(KVM_STATE(current_machine->accelerator),
+                                        KVM_CAP_VM_TYPES);
+    return !!(machine_types & BIT(type));
+}
+
+int kvm_get_vm_type(MachineState *ms, const char *vm_type)
+{
+    int kvm_type = KVM_X86_DEFAULT_VM;
+
+    if (ms->cgs) {
+        if (!object_dynamic_cast(OBJECT(ms->cgs), TYPE_X86_CONFIDENTIAL_GUEST)) {
+            error_report("configuration type %s not supported for x86 guests",
+                         object_get_typename(OBJECT(ms->cgs)));
+            exit(1);
+        }
+        kvm_type = x86_confidential_guest_kvm_type(
+            X86_CONFIDENTIAL_GUEST(ms->cgs));
+    }
+
+    if (!kvm_is_vm_type_supported(kvm_type)) {
+        error_report("vm-type %s not supported by KVM", vm_type_name[kvm_type]);
+        exit(1);
+    }
+
+    return kvm_type;
+}
+
 bool kvm_has_smm(void)
 {
     return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
-- 
2.44.0

[PATCH 7/7] target/i386: SEV: use KVM_SEV_INIT2 if possible

[Paolo Bonzini]

Implement support for the KVM_X86_SEV_VM and KVM_X86_SEV_ES_VM virtual
machine types, and the KVM_SEV_INIT2 function of KVM_MEMORY_ENCRYPT_OP.

These replace the KVM_SEV_INIT and KVM_SEV_ES_INIT functions, and have
several advantages:

- sharing the initialization sequence with SEV-SNP and TDX

- allowing arguments including the set of desired VMSA features

- protection against invalid use of KVM_GET/SET_* ioctls for guests
  with encrypted state

If the KVM_X86_SEV_VM and KVM_X86_SEV_ES_VM types are not supported,
fall back to KVM_SEV_INIT and KVM_SEV_ES_INIT (which use the
default x86 VM type).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 target/i386/sev.c | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)

diff --git a/target/i386/sev.c b/target/i386/sev.c
index ebe36d4c10c..9dab4060b84 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -26,6 +26,7 @@
 #include "qemu/error-report.h"
 #include "crypto/hash.h"
 #include "sysemu/kvm.h"
+#include "kvm/kvm_i386.h"
 #include "sev.h"
 #include "sysemu/sysemu.h"
 #include "sysemu/runstate.h"
@@ -56,6 +57,8 @@ OBJECT_DECLARE_SIMPLE_TYPE(SevGuestState, SEV_GUEST)
 struct SevGuestState {
     X86ConfidentialGuest parent_obj;
 
+    int kvm_type;
+
     /* configuration parameters */
     char *sev_device;
     uint32_t policy;
@@ -850,6 +853,26 @@ sev_vm_state_change(void *opaque, bool running, RunState state)
     }
 }
 
+static int sev_kvm_type(X86ConfidentialGuest *cg)
+{
+    SevGuestState *sev = SEV_GUEST(cg);
+    int kvm_type;
+
+    if (sev->kvm_type != -1) {
+        goto out;
+    }
+
+    kvm_type = (sev->policy & SEV_POLICY_ES) ? KVM_X86_SEV_ES_VM : KVM_X86_SEV_VM;
+    if (kvm_is_vm_type_supported(kvm_type)) {
+        sev->kvm_type = kvm_type;
+    } else {
+        sev->kvm_type = KVM_X86_DEFAULT_VM;
+    }
+
+out:
+    return sev->kvm_type;
+}
+
 static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     SevGuestState *sev = SEV_GUEST(cgs);
@@ -929,13 +952,19 @@ static int sev_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
                          __func__);
             goto err;
         }
-        cmd = KVM_SEV_ES_INIT;
-    } else {
-        cmd = KVM_SEV_INIT;
     }
 
     trace_kvm_sev_init();
-    ret = sev_ioctl(sev->sev_fd, cmd, NULL, &fw_error);
+    if (sev_kvm_type(X86_CONFIDENTIAL_GUEST(sev)) == KVM_X86_DEFAULT_VM) {
+        cmd = sev_es_enabled() ? KVM_SEV_ES_INIT : KVM_SEV_INIT;
+
+        ret = sev_ioctl(sev->sev_fd, cmd, NULL, &fw_error);
+    } else {
+        struct kvm_sev_init args = { 0 };
+
+        ret = sev_ioctl(sev->sev_fd, KVM_SEV_INIT2, &args, &fw_error);
+    }
+
     if (ret) {
         error_setg(errp, "%s: failed to initialize ret=%d fw_error=%d '%s'",
                    __func__, ret, fw_error, fw_error_to_str(fw_error));
@@ -1327,8 +1356,10 @@ static void
 sev_guest_class_init(ObjectClass *oc, void *data)
 {
     ConfidentialGuestSupportClass *klass = CONFIDENTIAL_GUEST_SUPPORT_CLASS(oc);
+    X86ConfidentialGuestClass *x86_klass = X86_CONFIDENTIAL_GUEST_CLASS(oc);
 
     klass->kvm_init = sev_kvm_init;
+    x86_klass->kvm_type = sev_kvm_type;
 
     object_class_property_add_str(oc, "sev-device",
                                   sev_guest_get_sev_device,
@@ -1357,6 +1388,8 @@ sev_guest_instance_init(Object *obj)
 {
     SevGuestState *sev = SEV_GUEST(obj);
 
+    sev->kvm_type = -1;
+
     sev->sev_device = g_strdup(DEFAULT_SEV_DEVICE);
     sev->policy = DEFAULT_GUEST_POLICY;
     object_property_add_uint32_ptr(obj, "policy", &sev->policy,
-- 
2.44.0

Re: [PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Daniel P. Berrangé]

On Tue, Mar 19, 2024 at 02:59:59PM +0100, Paolo Bonzini wrote:
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> 
> KVM is introducing a new API to create confidential guests, which
> will be used by TDX and SEV-SNP but is also available for SEV and
> SEV-ES.  The API uses the VM type argument to KVM_CREATE_VM to
> identify which confidential computing technology to use.
> 
> Since there are no other expected uses of VM types, delegate
> mc->kvm_type() for x86 boards to the confidential-guest-support
> object pointed to by ms->cgs.
> 
> For example, if a sev-guest object is specified to confidential-guest-support,
> like,
> 
>   qemu -machine ...,confidential-guest-support=sev0 \
>        -object sev-guest,id=sev0,...
> 
> it will check if a VM type KVM_X86_SEV_VM or KVM_X86_SEV_ES_VM
> is supported, and if so use them together with the KVM_SEV_INIT2
> function of the KVM_MEMORY_ENCRYPT_OP ioctl. If not, it will fall back to
> KVM_SEV_INIT and KVM_SEV_ES_INIT.
> 
> This is a preparatory work towards TDX and SEV-SNP support, but it
> will also enable support for VMSA features such as DebugSwap, which
> are only available via KVM_SEV_INIT2.
> 
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  target/i386/confidential-guest.h | 19 ++++++++++++++
>  target/i386/kvm/kvm_i386.h       |  2 ++
>  hw/i386/x86.c                    |  6 +++++
>  target/i386/kvm/kvm.c            | 44 ++++++++++++++++++++++++++++++++
>  4 files changed, 71 insertions(+)
> 
> diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
> index ca12d5a8fba..532e172a60b 100644
> --- a/target/i386/confidential-guest.h
> +++ b/target/i386/confidential-guest.h
> @@ -36,5 +36,24 @@ struct X86ConfidentialGuest {
>  struct X86ConfidentialGuestClass {
>      /* <private> */
>      ConfidentialGuestSupportClass parent;
> +
> +    /* <public> */
> +    int (*kvm_type)(X86ConfidentialGuest *cg);
>  };
> +
> +/**
> + * x86_confidential_guest_kvm_type:
> + *
> + * Calls #X86ConfidentialGuestClass.unplug callback of @plug_handler.
> + */
> +static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
> +{
> +    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
> +
> +    if (klass->kvm_type) {
> +        return klass->kvm_type(cg);
> +    } else {
> +        return 0;
> +    }
> +}
>  #endif
> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> index 30fedcffea3..02168122787 100644
> --- a/target/i386/kvm/kvm_i386.h
> +++ b/target/i386/kvm/kvm_i386.h
> @@ -37,6 +37,7 @@ bool kvm_hv_vpindex_settable(void);
>  bool kvm_enable_sgx_provisioning(KVMState *s);
>  bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp);
>  
> +int kvm_get_vm_type(MachineState *ms, const char *vm_type);
>  void kvm_arch_reset_vcpu(X86CPU *cs);
>  void kvm_arch_after_reset_vcpu(X86CPU *cpu);
>  void kvm_arch_do_init_vcpu(X86CPU *cs);
> @@ -49,6 +50,7 @@ void kvm_request_xsave_components(X86CPU *cpu, uint64_t mask);
>  
>  #ifdef CONFIG_KVM
>  
> +bool kvm_is_vm_type_supported(int type);
>  bool kvm_has_adjust_clock_stable(void);
>  bool kvm_has_exception_payload(void);
>  void kvm_synchronize_all_tsc(void);
> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> index ffbda48917f..2d4b148cd25 100644
> --- a/hw/i386/x86.c
> +++ b/hw/i386/x86.c
> @@ -1389,6 +1389,11 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
>      qapi_free_SgxEPCList(list);
>  }
>  
> +static int x86_kvm_type(MachineState *ms, const char *vm_type)
> +{
> +    return kvm_enabled() ? kvm_get_vm_type(ms, vm_type) : 0;
> +}
> +
>  static void x86_machine_initfn(Object *obj)
>  {
>      X86MachineState *x86ms = X86_MACHINE(obj);
> @@ -1413,6 +1418,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
>      mc->cpu_index_to_instance_props = x86_cpu_index_to_props;
>      mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
>      mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
> +    mc->kvm_type = x86_kvm_type;
>      x86mc->save_tsc_khz = true;
>      x86mc->fwcfg_dma_enabled = true;
>      nc->nmi_monitor_handler = x86_nmi;
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 0ec69109a2b..e109648f260 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -31,6 +31,7 @@
>  #include "sysemu/kvm_int.h"
>  #include "sysemu/runstate.h"
>  #include "kvm_i386.h"
> +#include "../confidential-guest.h"
>  #include "sev.h"
>  #include "xen-emu.h"
>  #include "hyperv.h"
> @@ -161,6 +162,49 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];
>  static RateLimit bus_lock_ratelimit_ctrl;
>  static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
>  
> +static const char *vm_type_name[] = {
> +    [KVM_X86_DEFAULT_VM] = "default",
> +};
> +
> +bool kvm_is_vm_type_supported(int type)
> +{
> +    uint32_t machine_types;
> +
> +    /*
> +     * old KVM doesn't support KVM_CAP_VM_TYPES but KVM_X86_DEFAULT_VM
> +     * is always supported
> +     */
> +    if (type == KVM_X86_DEFAULT_VM) {
> +        return true;
> +    }
> +
> +    machine_types = kvm_check_extension(KVM_STATE(current_machine->accelerator),
> +                                        KVM_CAP_VM_TYPES);
> +    return !!(machine_types & BIT(type));
> +}
> +
> +int kvm_get_vm_type(MachineState *ms, const char *vm_type)

The 'vm_type' parameter is never used here. What value is it expected
to have, and should be diagnosing an error if some unexpected value
is provided.

> +{
> +    int kvm_type = KVM_X86_DEFAULT_VM;
> +
> +    if (ms->cgs) {
> +        if (!object_dynamic_cast(OBJECT(ms->cgs), TYPE_X86_CONFIDENTIAL_GUEST)) {
> +            error_report("configuration type %s not supported for x86 guests",
> +                         object_get_typename(OBJECT(ms->cgs)));
> +            exit(1);
> +        }
> +        kvm_type = x86_confidential_guest_kvm_type(
> +            X86_CONFIDENTIAL_GUEST(ms->cgs));
> +    }
> +
> +    if (!kvm_is_vm_type_supported(kvm_type)) {
> +        error_report("vm-type %s not supported by KVM", vm_type_name[kvm_type]);
> +        exit(1);
> +    }
> +
> +    return kvm_type;
> +}
> +
>  bool kvm_has_smm(void)
>  {
>      return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
> -- 
> 2.44.0
> 
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 2/7] runstate: skip initial CPU reset if reset is not actually possible

[Daniel P. Berrangé]

On Tue, Mar 19, 2024 at 02:59:55PM +0100, Paolo Bonzini wrote:
> Right now, the system reset is concluded by a call to
> cpu_synchronize_all_post_reset() in order to sync any changes
> that the machine reset callback applied to the CPU state.
> 
> However, for VMs with encrypted state such as SEV-ES guests (currently
> the only case of guests with non-resettable CPUs) this cannot be done,
> because guest state has already been finalized by machine-init-done notifiers.
> cpu_synchronize_all_post_reset() does nothing on these guests, and actually
> we would like to make it fail if called once guest has been encrypted.
> So, assume that boards that support non-resettable CPUs do not touch
> CPU state and that all such setup is done before, at the time of
> cpu_synchronize_all_post_init().
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  system/runstate.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/system/runstate.c b/system/runstate.c
> index d6ab860ecaa..843e2b9853d 100644
> --- a/system/runstate.c
> +++ b/system/runstate.c
> @@ -501,7 +501,20 @@ void qemu_system_reset(ShutdownCause reason)
>      default:
>          qapi_event_send_reset(shutdown_caused_by_guest(reason), reason);
>      }
> -    cpu_synchronize_all_post_reset();
> +
> +    /*
> +     * Some boards us the machine reset callback to point CPUs to the firmware

ITYM s/us/use/

> +     * entry point.  Assume that this is not the case for boards that support
> +     * non-resettable CPUs (currently used only for confidential guests), in
> +     * which case cpu_synchronize_all_post_init() is enough because
> +     * it does _more_  than cpu_synchronize_all_post_reset().
> +     */
> +    if (cpus_are_resettable()) {
> +        cpu_synchronize_all_post_reset();
> +    } else {
> +        assert(runstate_check(RUN_STATE_PRELAUNCH));
> +    }
> +
>      vm_set_suspended(false);
>  }
>  
> -- 
> 2.44.0
> 
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Paolo Bonzini]

On Tue, Mar 19, 2024 at 3:15 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > +int kvm_get_vm_type(MachineState *ms, const char *vm_type)
>
> The 'vm_type' parameter is never used here. What value is it expected
> to have, and should be diagnosing an error if some unexpected value
> is provided.

It's the value of the kvm-type machine property, if any; but no x86
machine defines one, so right now it's always NULL. I left it in
because then it's clearer than this is an implementation of
mc->kvm_type, but I can remove it or pass it down to
x86_confidential_guest_kvm_type().

Paolo

> > +{
> > +    int kvm_type = KVM_X86_DEFAULT_VM;
> > +
> > +    if (ms->cgs) {
> > +        if (!object_dynamic_cast(OBJECT(ms->cgs), TYPE_X86_CONFIDENTIAL_GUEST)) {
> > +            error_report("configuration type %s not supported for x86 guests",
> > +                         object_get_typename(OBJECT(ms->cgs)));
> > +            exit(1);
> > +        }
> > +        kvm_type = x86_confidential_guest_kvm_type(
> > +            X86_CONFIDENTIAL_GUEST(ms->cgs));
> > +    }
> > +
> > +    if (!kvm_is_vm_type_supported(kvm_type)) {
> > +        error_report("vm-type %s not supported by KVM", vm_type_name[kvm_type]);
> > +        exit(1);
> > +    }
> > +
> > +    return kvm_type;
> > +}
> > +
> >  bool kvm_has_smm(void)
> >  {
> >      return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
> > --
> > 2.44.0
> >
> >
>
> With regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>

Re: [PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Daniel P. Berrangé]

On Tue, Mar 19, 2024 at 03:25:53PM +0100, Paolo Bonzini wrote:
> On Tue, Mar 19, 2024 at 3:15 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > +int kvm_get_vm_type(MachineState *ms, const char *vm_type)
> >
> > The 'vm_type' parameter is never used here. What value is it expected
> > to have, and should be diagnosing an error if some unexpected value
> > is provided.
> 
> It's the value of the kvm-type machine property, if any; but no x86
> machine defines one, so right now it's always NULL. I left it in
> because then it's clearer than this is an implementation of
> mc->kvm_type, but I can remove it or pass it down to
> x86_confidential_guest_kvm_type().

If we expect it to always be NULL, lets validate that is the
case and error_report + exit, if not.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Paolo Bonzini]

On Tue, Mar 19, 2024 at 3:27 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Mar 19, 2024 at 03:25:53PM +0100, Paolo Bonzini wrote:
> > On Tue, Mar 19, 2024 at 3:15 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > > +int kvm_get_vm_type(MachineState *ms, const char *vm_type)
> > >
> > > The 'vm_type' parameter is never used here. What value is it expected
> > > to have, and should be diagnosing an error if some unexpected value
> > > is provided.
> >
> > It's the value of the kvm-type machine property, if any; but no x86
> > machine defines one, so right now it's always NULL. I left it in
> > because then it's clearer than this is an implementation of
> > mc->kvm_type, but I can remove it or pass it down to
> > x86_confidential_guest_kvm_type().
>
> If we expect it to always be NULL, lets validate that is the
> case and error_report + exit, if not.

I think it's enough to have an assertion in x86_kvm_type():

    /*
     * No x86 machine has a kvm-type property.  If one is added that has
     * it, it should call kvm_get_vm_type() directly or not use it at all.
     */
    assert(vm_type == NULL);

Paolo

Re: [PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Daniel P. Berrangé]

On Tue, Mar 19, 2024 at 03:29:14PM +0100, Paolo Bonzini wrote:
> On Tue, Mar 19, 2024 at 3:27 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Tue, Mar 19, 2024 at 03:25:53PM +0100, Paolo Bonzini wrote:
> > > On Tue, Mar 19, 2024 at 3:15 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > > > +int kvm_get_vm_type(MachineState *ms, const char *vm_type)
> > > >
> > > > The 'vm_type' parameter is never used here. What value is it expected
> > > > to have, and should be diagnosing an error if some unexpected value
> > > > is provided.
> > >
> > > It's the value of the kvm-type machine property, if any; but no x86
> > > machine defines one, so right now it's always NULL. I left it in
> > > because then it's clearer than this is an implementation of
> > > mc->kvm_type, but I can remove it or pass it down to
> > > x86_confidential_guest_kvm_type().
> >
> > If we expect it to always be NULL, lets validate that is the
> > case and error_report + exit, if not.
> 
> I think it's enough to have an assertion in x86_kvm_type():
> 
>     /*
>      * No x86 machine has a kvm-type property.  If one is added that has
>      * it, it should call kvm_get_vm_type() directly or not use it at all.
>      */
>     assert(vm_type == NULL);

Sure, that's fine too.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 6/7] target/i386: Implement mc->kvm_type() to get VM type

[Xiaoyao Li]

On 3/19/2024 9:59 PM, Paolo Bonzini wrote:
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> 
> KVM is introducing a new API to create confidential guests, which
> will be used by TDX and SEV-SNP but is also available for SEV and
> SEV-ES.  The API uses the VM type argument to KVM_CREATE_VM to
> identify which confidential computing technology to use.
> 
> Since there are no other expected uses of VM types, delegate
> mc->kvm_type() for x86 boards to the confidential-guest-support
> object pointed to by ms->cgs.
> 
> For example, if a sev-guest object is specified to confidential-guest-support,
> like,
> 
>    qemu -machine ...,confidential-guest-support=sev0 \
>         -object sev-guest,id=sev0,...
> 
> it will check if a VM type KVM_X86_SEV_VM or KVM_X86_SEV_ES_VM
> is supported, and if so use them together with the KVM_SEV_INIT2
> function of the KVM_MEMORY_ENCRYPT_OP ioctl. If not, it will fall back to
> KVM_SEV_INIT and KVM_SEV_ES_INIT.
> 
> This is a preparatory work towards TDX and SEV-SNP support, but it
> will also enable support for VMSA features such as DebugSwap, which
> are only available via KVM_SEV_INIT2.
> 
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>

some nits below.

> ---
>   target/i386/confidential-guest.h | 19 ++++++++++++++
>   target/i386/kvm/kvm_i386.h       |  2 ++
>   hw/i386/x86.c                    |  6 +++++
>   target/i386/kvm/kvm.c            | 44 ++++++++++++++++++++++++++++++++
>   4 files changed, 71 insertions(+)
> 
> diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
> index ca12d5a8fba..532e172a60b 100644
> --- a/target/i386/confidential-guest.h
> +++ b/target/i386/confidential-guest.h
> @@ -36,5 +36,24 @@ struct X86ConfidentialGuest {
>   struct X86ConfidentialGuestClass {
>       /* <private> */
>       ConfidentialGuestSupportClass parent;
> +
> +    /* <public> */
> +    int (*kvm_type)(X86ConfidentialGuest *cg);
>   };
> +
> +/**
> + * x86_confidential_guest_kvm_type:
> + *
> + * Calls #X86ConfidentialGuestClass.unplug callback of @plug_handler.

ah, forgot to change the callback name after copy+paste.

> + */
> +static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
> +{
> +    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
> +
> +    if (klass->kvm_type) {
> +        return klass->kvm_type(cg);
> +    } else {
> +        return 0;
> +    }
> +}
>   #endif
> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> index 30fedcffea3..02168122787 100644
> --- a/target/i386/kvm/kvm_i386.h
> +++ b/target/i386/kvm/kvm_i386.h
> @@ -37,6 +37,7 @@ bool kvm_hv_vpindex_settable(void);
>   bool kvm_enable_sgx_provisioning(KVMState *s);
>   bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp);
>   
> +int kvm_get_vm_type(MachineState *ms, const char *vm_type);
>   void kvm_arch_reset_vcpu(X86CPU *cs);
>   void kvm_arch_after_reset_vcpu(X86CPU *cpu);
>   void kvm_arch_do_init_vcpu(X86CPU *cs);
> @@ -49,6 +50,7 @@ void kvm_request_xsave_components(X86CPU *cpu, uint64_t mask);
>   
>   #ifdef CONFIG_KVM
>   
> +bool kvm_is_vm_type_supported(int type);
>   bool kvm_has_adjust_clock_stable(void);
>   bool kvm_has_exception_payload(void);
>   void kvm_synchronize_all_tsc(void);
> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> index ffbda48917f..2d4b148cd25 100644
> --- a/hw/i386/x86.c
> +++ b/hw/i386/x86.c
> @@ -1389,6 +1389,11 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
>       qapi_free_SgxEPCList(list);
>   }
>   
> +static int x86_kvm_type(MachineState *ms, const char *vm_type)
> +{
> +    return kvm_enabled() ? kvm_get_vm_type(ms, vm_type) : 0;
> +}
> +
>   static void x86_machine_initfn(Object *obj)
>   {
>       X86MachineState *x86ms = X86_MACHINE(obj);
> @@ -1413,6 +1418,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
>       mc->cpu_index_to_instance_props = x86_cpu_index_to_props;
>       mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
>       mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
> +    mc->kvm_type = x86_kvm_type;
>       x86mc->save_tsc_khz = true;
>       x86mc->fwcfg_dma_enabled = true;
>       nc->nmi_monitor_handler = x86_nmi;
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 0ec69109a2b..e109648f260 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -31,6 +31,7 @@
>   #include "sysemu/kvm_int.h"
>   #include "sysemu/runstate.h"
>   #include "kvm_i386.h"
> +#include "../confidential-guest.h"
>   #include "sev.h"
>   #include "xen-emu.h"
>   #include "hyperv.h"
> @@ -161,6 +162,49 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];
>   static RateLimit bus_lock_ratelimit_ctrl;
>   static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
>   
> +static const char *vm_type_name[] = {
> +    [KVM_X86_DEFAULT_VM] = "default",
> +};
> +
> +bool kvm_is_vm_type_supported(int type)
> +{
> +    uint32_t machine_types;

The name of machine_types confuses me a lot. why not supported_vm_types?

> +
> +    /*
> +     * old KVM doesn't support KVM_CAP_VM_TYPES but KVM_X86_DEFAULT_VM
> +     * is always supported
> +     */
> +    if (type == KVM_X86_DEFAULT_VM) {
> +        return true;
> +    }
> +
> +    machine_types = kvm_check_extension(KVM_STATE(current_machine->accelerator),
> +                                        KVM_CAP_VM_TYPES);
> +    return !!(machine_types & BIT(type));
> +}
> +
> +int kvm_get_vm_type(MachineState *ms, const char *vm_type)
> +{
> +    int kvm_type = KVM_X86_DEFAULT_VM;
> +
> +    if (ms->cgs) {
> +        if (!object_dynamic_cast(OBJECT(ms->cgs), TYPE_X86_CONFIDENTIAL_GUEST)) {
> +            error_report("configuration type %s not supported for x86 guests",
> +                         object_get_typename(OBJECT(ms->cgs)));
> +            exit(1);
> +        }
> +        kvm_type = x86_confidential_guest_kvm_type(
> +            X86_CONFIDENTIAL_GUEST(ms->cgs));
> +    }
> +
> +    if (!kvm_is_vm_type_supported(kvm_type)) {
> +        error_report("vm-type %s not supported by KVM", vm_type_name[kvm_type]);
> +        exit(1);
> +    }
> +
> +    return kvm_type;
> +}
> +
>   bool kvm_has_smm(void)
>   {
>       return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);

Re: [PATCH 5/7] target/i386: introduce x86-confidential-guest

[Xiaoyao Li]

On 3/19/2024 9:59 PM, Paolo Bonzini wrote:
> Introduce a common superclass for x86 confidential guest implementations.
> It will extend ConfidentialGuestSupportClass with a method that provides
> the VM type to be passed to KVM_CREATE_VM.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>

Re: [PATCH 3/7] KVM: track whether guest state is encrypted

[Xiaoyao Li]

On 3/19/2024 9:59 PM, Paolo Bonzini wrote:
> So far, KVM has allowed KVM_GET/SET_* ioctls to execute even if the
> guest state is encrypted, in which case they do nothing.  For the new
> API using VM types, instead, the ioctls will fail which is a safer and
> more robust approach.
> 
> The new API will be the only one available for SEV-SNP and TDX, but it
> is also usable for SEV and SEV-ES.  In preparation for that, require
> architecture-specific KVM code to communicate the point at which guest
> state is protected (which must be after kvm_cpu_synchronize_post_init(),
> though that might change in the future in order to suppor migration).
>  From that point, skip reading registers so that cpu->vcpu_dirty is
> never true: if it ever becomes true, kvm_arch_put_registers() will
> fail miserably.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>

Re: [PATCH 4/7] KVM: remove kvm_arch_cpu_check_are_resettable

[Xiaoyao Li]

On 3/19/2024 9:59 PM, Paolo Bonzini wrote:
> Board reset requires writing a fresh CPU state.  As far as KVM is
> concerned, the only thing that blocks reset is that CPU state is
> encrypted; therefore, kvm_cpus_are_resettable() can simply check
> if that is the case.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>