[RFC PATCH v1 00/17] target/riscv: Add KVM CoVE confidential VM support

[RFC PATCH v1 00/17] target/riscv: Add KVM CoVE confidential VM support

[Baolong Duan]

This series adds RISC-V CoVE (Confidential VM Extension) support to the
virt machine when running with KVM acceleration.

CoVE is the RISC-V equivalent of AMD SEV / Intel TDX, defined by the
RISC-V AP-TEE specification [1].  A CoVE guest is a TEE VM (TVM) whose
memory and vCPU state are owned by the TEE Security Manager (TSM),
making them inaccessible to the host.

The series adds a "cove-vm" machine property to the virt board and adapts
the boot flow, interrupt routing, memory registration and device model to
the constraints of a confidential guest:

  - kernel, initrd and device tree are measured before the guest starts
  - interrupts are MSI-only (IMSIC mandatory, APLIC emulated by QEMU)
  - virtio uses modern PCI transport with VIRTIO_F_ACCESS_PLATFORM
  - vhost is disabled (host kernel has no access to guest memory)
  - vCPU threads are pinned (TSM binds vCPUs to harts)

The KVM interface used here (KVM_VM_TYPE_RISCV_COVE and
KVM_RISCV_COVE_MEASURE_REGION) is not yet part of an upstream Linux
release.  The definitions are kept locally and will be replaced by a
linux-headers update once the kernel side has been merged.

Patches 15 and 17 contain known workarounds and explicitly ask for design
guidance:

  - Patch 15 pins vCPU threads and adds a yield after KVM_RUN to avoid
    host RCU stalls.  Thread placement should be left to the user or
    management layer.

  - Patch 17 converts system_reset into a shutdown because a TVM cannot be
    recreated.  The proper approach may be to implement
    ConfidentialGuestSupport and use the existing can_rebuild_state()
    machinery.

Tested with QEMU TCG emulating a CoVE-capable Xuantie C930 platform.

[1] https://github.com/riscv-non-isa/riscv-ap-tee

Baolong Duan (17):
  docs/system/riscv/virt: document the cove-vm machine option
  hw/riscv/virt: add the cove-vm machine property
  target/riscv/kvm: create and measure a CoVE TEE VM
  accel/kvm: add kvm_gpa_to_userspace_addr()
  hw/riscv/boot: load and measure the images of a CoVE guest
  hw/riscv/virt: measure the device tree of a CoVE guest
  hw/riscv: adapt the device tree of a CoVE guest
  hw/riscv/virt: use MSIs only for a CoVE guest
  hw/intc/riscv_aplic: emulate the APLIC for a CoVE guest
  target/riscv/kvm: skip unsupported KVM requests for a CoVE guest
  hw/virtio: force modern virtio for a CoVE guest
  hw/net/virtio-net: do not use vhost for a CoVE guest
  accel/kvm: only register the DRAM slot of a CoVE guest
  accel/kvm: skip MSI route updates for a CoVE guest
  accel/kvm: pin the vCPU threads of a CoVE guest
  accel/kvm: terminate on a system event of a CoVE guest
  hw/core/machine-qmp-cmds: shut down a CoVE guest on reset

 MAINTAINERS                  |   6 ++
 accel/kvm/kvm-accel-ops.c    |  16 +++++
 accel/kvm/kvm-all.c          |  60 +++++++++++++++++
 docs/system/riscv/virt.rst   |  39 +++++++++++
 hw/core/machine-qmp-cmds.c   |  10 +++
 hw/core/machine.c            |  18 +++++
 hw/intc/riscv_aplic.c        |   9 +++
 hw/net/virtio-net.c          |   9 +++
 hw/riscv/boot.c              |  91 +++++++++++++++++++++++++
 hw/riscv/fdt-common.c        |   4 +-
 hw/riscv/trace-events        |   4 ++
 hw/riscv/virt.c              | 125 +++++++++++++++++++++++++++++------
 hw/virtio/virtio-bus.c       |  12 ++++
 hw/virtio/virtio-pci.c       |   9 ++-
 include/hw/riscv/cove.h      |  24 +++++++
 include/hw/riscv/virt.h      |   1 +
 include/system/kvm.h         |  18 +++++
 target/riscv/kvm/kvm-cpu.c   |  67 ++++++++++++++++++-
 target/riscv/kvm/kvm_riscv.h |  10 +++
 19 files changed, 508 insertions(+), 24 deletions(-)
 create mode 100644 include/hw/riscv/cove.h

-- 
2.34.1

[RFC PATCH v1 01/17] docs/system/riscv/virt: document the cove-vm machine option

[Baolong Duan]

Describe the cove-vm option of the virt machine and how a CoVE TEE VM
(TVM) differs from a regular KVM guest: its memory and vCPU state are
owned by the TEE Security Manager (TSM), the guest images are measured
before it starts, interrupts are delivered as MSIs only, there is no
virtio-mmio transport and vhost cannot be used.

The option itself is added by the following patches.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 docs/system/riscv/virt.rst | 39 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst
index 60850970ce..d053dbc300 100644
--- a/docs/system/riscv/virt.rst
+++ b/docs/system/riscv/virt.rst
@@ -146,6 +146,45 @@ The following machine-specific options are supported:
 
   Enables the riscv-iommu-sys platform device. Defaults to 'off'.
 
+- cove-vm=[on|off]
+
+  When this option is "on" the guest is created as a RISC-V CoVE
+  (Confidential VM Extension) TEE VM, or TVM. Defaults to "off". See
+  `Running a confidential VM`_ below.
+
+Running a confidential VM
+-------------------------
+
+With "cove-vm=on" the ``virt`` machine creates a TEE VM (TVM) instead of a
+regular KVM guest. The memory and the vCPU state of a TVM are owned by the
+TEE Security Manager (TSM) running in M-mode and are not accessible to the
+host, which changes the machine in a few ways:
+
+- the kernel, the initrd and the device tree are added to the initial
+  measurement of the guest before it starts, so that the guest can be
+  attested later on;
+
+- interrupts are delivered as MSIs only. An IMSIC is therefore mandatory and
+  "aia=aplic-imsic" is selected automatically when no AIA mode is requested.
+  The APLIC is emulated by QEMU because KVM does not implement one for a TVM;
+
+- no virtio-mmio transport is created. Devices have to be attached to the
+  PCIe host bridge, and they always negotiate VIRTIO_F_ACCESS_PLATFORM so
+  that DMA is bounced through memory the guest shares explicitly;
+
+- vhost cannot be used, as the kernel datapath has no access to guest memory.
+
+This requires a host with CoVE support, both in the firmware and in KVM, and
+it only works with ``-accel kvm``. An example command line is:
+
+.. code-block:: bash
+
+  $ qemu-system-riscv64 -M virt,cove-vm=on -accel kvm \
+      -m 256M -smp 1 -nographic \
+      -kernel Image -append "console=ttyS0 root=/dev/vda" \
+      -drive file=rootfs.ext4,format=raw,id=hd0,if=none \
+      -device virtio-blk-pci,drive=hd0,disable-legacy=on
+
 Running Linux kernel
 --------------------
 
-- 
2.34.1

[RFC PATCH v1 02/17] hw/riscv/virt: add the cove-vm machine property

[Baolong Duan]

A RISC-V CoVE guest is a TEE VM (TVM) whose memory and vCPU state are
owned by the TEE Security Manager (TSM) rather than by the host.  Add a
'cove-vm' property to the virt machine to ask for one.

Subsystems outside of hw/riscv/, such as the KVM accelerator and the
virtio code, have to know whether the guest is a TVM.  The state is
therefore mirrored into target independent code and queried through
riscv_cove_vm_active(); keeping the implementation in hw/core/machine.c
avoids a link time dependency on target/riscv.

Nothing looks at the property yet.  Add a MAINTAINERS entry for the new
header.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 MAINTAINERS             |  6 ++++++
 hw/core/machine.c       | 18 ++++++++++++++++++
 hw/riscv/virt.c         | 26 ++++++++++++++++++++++++++
 include/hw/riscv/cove.h | 24 ++++++++++++++++++++++++
 include/hw/riscv/virt.h |  1 +
 5 files changed, 75 insertions(+)
 create mode 100644 include/hw/riscv/cove.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 902db77218..dab130a45a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -391,6 +391,12 @@ F: target/riscv/XVentanaCondOps.decode
 F: target/riscv/insn_trans/trans_xventanacondops.c.inc
 F: disas/riscv-xventana*
 
+RISC-V CoVE
+M: Baolong Duan <blduan@linux.alibaba.com>
+L: qemu-riscv@nongnu.org
+S: Maintained
+F: include/hw/riscv/cove.h
+
 RENESAS RX CPUs
 R: Yoshinori Sato <yoshinori.sato@nifty.com>
 S: Orphan
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 73b4d82b4a..578a5cdde5 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -16,6 +16,7 @@
 #include "system/replay.h"
 #include "hw/core/boards.h"
 #include "hw/core/loader.h"
+#include "hw/riscv/cove.h"
 #include "qemu/error-report.h"
 #include "qapi/error.h"
 #include "qapi/qapi-visit-machine.h"
@@ -1335,6 +1336,23 @@ bool machine_require_guest_memfd(MachineState *machine)
     return machine->cgs && machine->cgs->require_guest_memfd;
 }
 
+/*
+ * Whether the machine runs as a RISC-V CoVE guest. This lives here, and not
+ * in hw/riscv/, because target independent code has to query it and must not
+ * depend on the RISC-V machine being linked in.
+ */
+static bool riscv_cove_vm;
+
+bool riscv_cove_vm_active(void)
+{
+    return riscv_cove_vm;
+}
+
+void riscv_cove_vm_set_active(bool active)
+{
+    riscv_cove_vm = active;
+}
+
 static char *cpu_slot_to_string(const CPUArchId *cpu)
 {
     GString *s = g_string_new(NULL);
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 51bac47a91..a76587d362 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -35,6 +35,7 @@
 #include "hw/riscv/riscv-iommu-bits.h"
 #include "hw/riscv/virt.h"
 #include "hw/riscv/boot.h"
+#include "hw/riscv/cove.h"
 #include "hw/riscv/fdt-common.h"
 #include "hw/riscv/machines-qom.h"
 #include "hw/riscv/numa.h"
@@ -1321,6 +1322,11 @@ static void virt_machine_init(MachineState *machine)
         exit(1);
     }
 
+    if (s->cove_vm && !kvm_enabled()) {
+        error_report("'cove-vm' is only available with KVM acceleration");
+        exit(1);
+    }
+
     /* Initialize sockets */
     mmio_irqchip = virtio_irqchip = pcie_irqchip = NULL;
     for (i = 0; i < socket_count; i++) {
@@ -1652,6 +1658,21 @@ static void virt_set_iommu_sys(Object *obj, Visitor *v, const char *name,
     visit_type_OnOffAuto(v, name, &s->iommu_sys, errp);
 }
 
+static bool virt_get_cove_vm(Object *obj, Error **errp)
+{
+    RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+    return s->cove_vm;
+}
+
+static void virt_set_cove_vm(Object *obj, bool value, Error **errp)
+{
+    RISCVVirtState *s = RISCV_VIRT_MACHINE(obj);
+
+    s->cove_vm = value;
+    riscv_cove_vm_set_active(value);
+}
+
 bool virt_is_acpi_enabled(RISCVVirtState *s)
 {
     return s->acpi != ON_OFF_AUTO_OFF;
@@ -1780,6 +1801,11 @@ static void virt_machine_class_init(ObjectClass *oc, const void *data)
                               NULL, NULL);
     object_class_property_set_description(oc, "iommu-sys",
                                           "Enable IOMMU platform device");
+
+    object_class_property_add_bool(oc, "cove-vm", virt_get_cove_vm,
+                                   virt_set_cove_vm);
+    object_class_property_set_description(oc, "cove-vm",
+                                          "Enable CoVE confidential VM");
 }
 
 static const TypeInfo virt_machine_typeinfo = {
diff --git a/include/hw/riscv/cove.h b/include/hw/riscv/cove.h
new file mode 100644
index 0000000000..0b29a00786
--- /dev/null
+++ b/include/hw/riscv/cove.h
@@ -0,0 +1,24 @@
+/*
+ * RISC-V Confidential VM Extension (CoVE)
+ *
+ * Copyright (c) 2026 Alibaba Group
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_RISCV_COVE_H
+#define HW_RISCV_COVE_H
+
+/*
+ * A CoVE guest is a TEE VM (TVM) whose memory and vCPU state are owned by
+ * the TEE Security Manager (TSM) instead of the host. Subsystems outside of
+ * hw/riscv/ have to behave differently for such a guest, so the state is
+ * kept in target independent code.
+ *
+ * riscv_cove_vm_set_active() is called by the machine that implements CoVE
+ * while its properties are parsed, before any device is created.
+ */
+bool riscv_cove_vm_active(void);
+void riscv_cove_vm_set_active(bool active);
+
+#endif /* HW_RISCV_COVE_H */
diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h
index 36a2def410..99e073ef9a 100644
--- a/include/hw/riscv/virt.h
+++ b/include/hw/riscv/virt.h
@@ -60,6 +60,7 @@ struct RISCVVirtState {
     char *oem_id;
     char *oem_table_id;
     OnOffAuto acpi;
+    bool cove_vm;
     const MemMapEntry *memmap;
     struct GPEXHost *gpex_host;
     OnOffAuto iommu_sys;
-- 
2.34.1

[RFC PATCH v1 03/17] target/riscv/kvm: create and measure a CoVE TEE VM

[Baolong Duan]

Ask KVM for a TEE VM (TVM) instead of a regular guest when the machine
runs in CoVE mode, and add kvm_riscv_cove_measure_region() to add a
memory region to the initial measurement of that TVM.

The measurement is what allows a CoVE guest to be attested later on, so
failing to add a region to it is fatal.

The KVM ABI this relies on is not part of an upstream Linux release yet, so
KVM_VM_TYPE_RISCV_COVE and KVM_RISCV_COVE_MEASURE_REGION are defined here
rather than imported into linux-headers/.  They have to be replaced by a
regular scripts/update-linux-headers.sh run once the kernel side has been
merged.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 target/riscv/kvm/kvm-cpu.c   | 46 ++++++++++++++++++++++++++++++++++++
 target/riscv/kvm/kvm_riscv.h | 10 ++++++++
 2 files changed, 56 insertions(+)

diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index 495cb42dc8..67c99d68ce 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -48,10 +48,27 @@
 #include "migration/misc.h"
 #include "system/runstate.h"
 #include "hw/riscv/numa.h"
+#include "hw/riscv/cove.h"
 
 #define PR_RISCV_V_SET_CONTROL            69
 #define PR_RISCV_V_VSTATE_CTRL_ON          2
 
+/*
+ * CoVE KVM ABI.  These definitions are not part of an upstream Linux release
+ * yet, so they cannot be imported into linux-headers/ and are kept here until
+ * the kernel side has been merged.
+ */
+#define KVM_VM_TYPE_RISCV_COVE          (1UL << 9)
+
+struct kvm_riscv_cove_measure_region {
+    uint64_t user_addr;
+    uint64_t gpa;
+    uint64_t size;
+};
+
+#define KVM_RISCV_COVE_MEASURE_REGION \
+    _IOR(KVMIO, 0xb5, struct kvm_riscv_cove_measure_region)
+
 void riscv_kvm_aplic_request(void *opaque, int irq, int level)
 {
     kvm_set_irq(kvm_state, irq, !!level);
@@ -1549,6 +1566,9 @@ int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
 
 int kvm_arch_get_default_type(MachineState *ms)
 {
+    if (riscv_cove_vm_active()) {
+        return KVM_VM_TYPE_RISCV_COVE;
+    }
     return 0;
 }
 
@@ -1829,6 +1849,32 @@ void kvm_arch_accel_class_init(ObjectClass *oc)
                                     "auto");
 }
 
+/*
+ * Add the contents of a memory region to the initial measurement of the TVM.
+ * Nothing is measured for a guest that is not confidential.
+ */
+void kvm_riscv_cove_measure_region(uint64_t user_addr, uint64_t gpa,
+                                   uint64_t size)
+{
+    struct kvm_riscv_cove_measure_region mr;
+    int ret;
+
+    if (!riscv_cove_vm_active()) {
+        return;
+    }
+
+    mr.user_addr = user_addr;
+    mr.gpa = gpa;
+    mr.size = size;
+
+    ret = kvm_vm_ioctl(kvm_state, KVM_RISCV_COVE_MEASURE_REGION, &mr);
+    if (ret < 0) {
+        error_report("Unable to measure CoVE region at 0x%" PRIx64 ": %s",
+                     gpa, strerror(-ret));
+        exit(EXIT_FAILURE);
+    }
+}
+
 void kvm_riscv_aia_create(MachineState *machine, uint64_t group_shift,
                           uint64_t aia_irq_num, uint64_t aia_msi_num,
                           uint64_t aplic_base, uint64_t imsic_base,
diff --git a/target/riscv/kvm/kvm_riscv.h b/target/riscv/kvm/kvm_riscv.h
index b2bcd1041f..cd45f1266b 100644
--- a/target/riscv/kvm/kvm_riscv.h
+++ b/target/riscv/kvm/kvm_riscv.h
@@ -23,6 +23,16 @@
 
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
 void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
+#ifdef CONFIG_KVM
+void kvm_riscv_cove_measure_region(uint64_t user_addr, uint64_t gpa,
+                                   uint64_t size);
+#else
+static inline void kvm_riscv_cove_measure_region(uint64_t user_addr,
+                                                 uint64_t gpa, uint64_t size)
+{
+    /* A CoVE guest cannot be created without KVM, nothing to measure. */
+}
+#endif
 void kvm_riscv_aia_create(MachineState *machine, uint64_t group_shift,
                           uint64_t aia_irq_num, uint64_t aia_msi_num,
                           uint64_t aplic_base, uint64_t imsic_base,
-- 
2.34.1

[RFC PATCH v1 04/17] accel/kvm: add kvm_gpa_to_userspace_addr()

[Baolong Duan]

Translate a guest physical address into the host userspace address
backing it by walking the memory slots of the accelerator.

The RISC-V CoVE code needs this to tell the TSM which host memory holds
the guest images that have to be measured.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 accel/kvm/kvm-all.c  | 21 +++++++++++++++++++++
 include/system/kvm.h | 18 ++++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 83cbd120a8..005abb1c54 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -366,6 +366,27 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
     return ret;
 }
 
+void *kvm_gpa_to_userspace_addr(KVMState *s, hwaddr gpa)
+{
+    KVMMemoryListener *kml = &s->memory_listener;
+    void *result = NULL;
+    int i;
+    KVMSlot *mem;
+
+    kvm_slots_lock();
+    for (i = 0; i < kml->nr_slots_allocated; i++) {
+        mem = &kml->slots[i];
+        if (gpa >= mem->start_addr &&
+            gpa < mem->start_addr + mem->memory_size) {
+            result = mem->ram + (gpa - mem->start_addr);
+            break;
+        }
+    }
+    kvm_slots_unlock();
+
+    return result;
+}
+
 static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, bool new)
 {
     KVMState *s = kvm_state;
diff --git a/include/system/kvm.h b/include/system/kvm.h
index 714b8c7b01..67a0b9f7ac 100644
--- a/include/system/kvm.h
+++ b/include/system/kvm.h
@@ -454,6 +454,24 @@ void kvm_set_sigmask_len(KVMState *s, unsigned int sigmask_len);
 int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
                                        hwaddr *phys_addr);
 
+/**
+ * kvm_gpa_to_userspace_addr:
+ * @s: #KVMState
+ * @gpa: guest physical address
+ *
+ * Returns the host userspace address @gpa is backed by, or NULL if @gpa is
+ * not covered by any memory slot.
+ */
+#ifdef CONFIG_KVM
+void *kvm_gpa_to_userspace_addr(KVMState *s, hwaddr gpa);
+#else
+static inline void *kvm_gpa_to_userspace_addr(KVMState *s, hwaddr gpa)
+{
+    /* Without KVM there are no memory slots to look up. */
+    return NULL;
+}
+#endif
+
 #endif /* COMPILING_PER_TARGET */
 
 bool kvm_arch_supports_vmfd_change(void);
-- 
2.34.1

[RFC PATCH v1 05/17] hw/riscv/boot: load and measure the images of a CoVE guest

[Baolong Duan]

The kernel and the initrd of a CoVE guest are part of the initial
measurement of the TEE VM, which means they have to be written to guest
memory and measured before the guest is started.

Add riscv_cove_load_kernel(), which handles ELF, uImage and raw images,
and measure the initrd as well when one is given.  The host address of a
loaded image is derived from the RAM memory region because the memory
slots of a TVM cannot be used for that this early during machine
initialisation.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/riscv/boot.c       | 91 +++++++++++++++++++++++++++++++++++++++++++
 hw/riscv/trace-events |  4 ++
 2 files changed, 95 insertions(+)

diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c
index 5e2dfa091a..70c4009a66 100644
--- a/hw/riscv/boot.c
+++ b/hw/riscv/boot.c
@@ -27,11 +27,14 @@
 #include "hw/core/loader.h"
 #include "hw/riscv/boot.h"
 #include "hw/riscv/boot_opensbi.h"
+#include "hw/riscv/cove.h"
+#include "kvm/kvm_riscv.h"
 #include "elf.h"
 #include "system/device_tree.h"
 #include "system/qtest.h"
 #include "system/kvm.h"
 #include "system/reset.h"
+#include "trace.h"
 
 #include <libfdt.h>
 
@@ -261,6 +264,86 @@ static void riscv_load_initrd(MachineState *machine, RISCVBootInfo *info)
         qemu_fdt_setprop_u64(fdt, "/chosen", "linux,initrd-start", start);
         qemu_fdt_setprop_u64(fdt, "/chosen", "linux,initrd-end", end);
     }
+
+    /* The initrd is part of the initial measurement of a CoVE guest. */
+    if (riscv_cove_vm_active()) {
+        void *initrd_host = kvm_gpa_to_userspace_addr(kvm_state, start);
+        uint64_t aligned_size = ROUND_UP(size, 4 * KiB);
+
+        trace_riscv_cove_measure_initrd(start, aligned_size);
+        kvm_riscv_cove_measure_region((uint64_t)(uintptr_t)initrd_host,
+                                      start, aligned_size);
+    }
+}
+
+/* Load address of a raw kernel image, relative to the base of the RAM. */
+#define RISCV_COVE_KERNEL_OFFSET 0x200000
+
+/*
+ * Load the kernel of a CoVE guest and add it to the initial measurement of
+ * the TVM. ELF, uImage and raw images are supported.
+ *
+ * Unlike riscv_load_kernel(), the host address of the loaded image has to be
+ * computed from the RAM memory region: the KVM memory slots of a TVM are not
+ * usable for that yet at this point of the machine initialisation.
+ *
+ * Returns the guest physical address to start the guest from.
+ */
+static uint64_t riscv_cove_load_kernel(MachineState *machine,
+                                       const char *kernel_filename,
+                                       uint64_t mem_size,
+                                       symbol_fn_t sym_cb)
+{
+    MemoryRegion *ram = machine->ram;
+    void *ram_base = memory_region_get_ram_ptr(ram);
+    uint64_t elf_entry, elf_low, elf_high;
+    hwaddr uimage_ep, uimage_loadaddr;
+    hwaddr kernel_gpa;
+    ssize_t size;
+
+    size = load_elf_ram_sym(kernel_filename, NULL, NULL, NULL,
+                            &elf_entry, &elf_low, &elf_high,
+                            NULL, 0, EM_RISCV, 1, 0,
+                            NULL, false, sym_cb);
+    if (size > 0) {
+        trace_riscv_cove_measure_kernel("ELF", elf_low, elf_high - elf_low);
+        kvm_riscv_cove_measure_region((uint64_t)(uintptr_t)ram_base +
+                                      (elf_low - ram->addr),
+                                      elf_low, elf_high - elf_low);
+        return elf_entry;
+    }
+
+    uimage_loadaddr = LOAD_UIMAGE_LOADADDR_INVALID;
+    size = load_uimage_as(kernel_filename, &uimage_ep, &uimage_loadaddr,
+                          NULL, NULL, NULL, NULL);
+    if (size > 0) {
+        trace_riscv_cove_measure_kernel("uImage", uimage_loadaddr, size);
+        kvm_riscv_cove_measure_region((uint64_t)(uintptr_t)ram_base +
+                                      (uimage_loadaddr - ram->addr),
+                                      uimage_loadaddr, size);
+        return uimage_ep;
+    }
+
+    /*
+     * Read the image straight into RAM: load_image_targphys_as() would
+     * register a ROM blob, whose contents are only written to guest memory
+     * when the machine is reset, long after the measurement has been taken.
+     */
+    kernel_gpa = ram->addr + RISCV_COVE_KERNEL_OFFSET;
+    size = load_image_size(kernel_filename,
+                           (char *)ram_base + RISCV_COVE_KERNEL_OFFSET,
+                           mem_size - RISCV_COVE_KERNEL_OFFSET);
+    if (size < 0) {
+        error_report("could not load kernel '%s': %s", kernel_filename,
+                     strerror(errno));
+        exit(1);
+    }
+
+    trace_riscv_cove_measure_kernel("raw", kernel_gpa, size);
+    kvm_riscv_cove_measure_region((uint64_t)(uintptr_t)ram_base +
+                                  RISCV_COVE_KERNEL_OFFSET,
+                                  kernel_gpa, size);
+    return kernel_gpa;
 }
 
 void riscv_load_kernel(MachineState *machine,
@@ -276,6 +359,14 @@ void riscv_load_kernel(MachineState *machine,
 
     g_assert(kernel_filename != NULL);
 
+    /* A CoVE guest is loaded and measured before the TVM is finalised. */
+    if (riscv_cove_vm_active()) {
+        info->image_low_addr = riscv_cove_load_kernel(machine, kernel_filename,
+                                                      mem_size, sym_cb);
+        info->image_high_addr = info->image_low_addr;
+        goto out;
+    }
+
     /*
      * NB: Use low address not ELF entry point to ensure that the fw_dynamic
      * behaviour when loading an ELF matches the fw_payload, fw_jump and BBL
diff --git a/hw/riscv/trace-events b/hw/riscv/trace-events
index b50b14a654..04f4b90efa 100644
--- a/hw/riscv/trace-events
+++ b/hw/riscv/trace-events
@@ -24,3 +24,7 @@ riscv_iommu_hpm_incr_ctr(uint64_t cntr_val) "cntr_val 0x%"PRIx64
 riscv_iommu_hpm_iocntinh_cy(bool prev_cy_inh) "prev_cy_inh %d"
 riscv_iommu_hpm_cycle_write(uint32_t ovf, uint64_t val) "ovf 0x%x val 0x%"PRIx64
 riscv_iommu_hpm_evt_write(uint32_t ctr_idx, uint32_t ovf, uint64_t val) "ctr_idx 0x%x ovf 0x%x val 0x%"PRIx64
+
+# boot.c
+riscv_cove_measure_kernel(const char *format, uint64_t gpa, uint64_t size) "%s kernel gpa 0x%"PRIx64" size 0x%"PRIx64
+riscv_cove_measure_initrd(uint64_t gpa, uint64_t size) "initrd gpa 0x%"PRIx64" size 0x%"PRIx64
-- 
2.34.1

[RFC PATCH v1 06/17] hw/riscv/virt: measure the device tree of a CoVE guest

[Baolong Duan]

The device tree is the last piece of guest state that has to be part of
the initial measurement of a TEE VM.  The ROM blob holding it is only
written to guest memory when the machine is reset, which is too late, so
copy it into RAM and measure it right after its address is known.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/riscv/virt.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index a76587d362..c15d8859ce 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -1277,6 +1277,24 @@ static void virt_machine_done(Notifier *notifier, void *data)
                                            machine, &boot_info);
     riscv_load_fdt(fdt_load_addr, machine->fdt);
 
+    /*
+     * The device tree is part of the initial measurement of a TVM. The ROM
+     * blobs holding it are not written to guest memory before the machine is
+     * reset, so copy it into RAM now to be able to measure it.
+     */
+    if (s->cove_vm) {
+        void *ram_base = memory_region_get_ram_ptr(machine->ram);
+        hwaddr fdt_offset = fdt_load_addr - s->memmap[VIRT_DRAM].base;
+        void *fdt_host;
+
+        memcpy((char *)ram_base + fdt_offset, machine->fdt, s->fdt_size);
+
+        fdt_host = kvm_gpa_to_userspace_addr(kvm_state, fdt_load_addr);
+        kvm_riscv_cove_measure_region((uint64_t)(uintptr_t)fdt_host,
+                                      fdt_load_addr,
+                                      ROUND_UP(s->fdt_size, 4 * KiB));
+    }
+
     /* load the reset vector */
     riscv_setup_rom_reset_vec(machine, &s->soc[0], start_addr,
                               s->memmap[VIRT_MROM].base,
-- 
2.34.1

[RFC PATCH v1 07/17] hw/riscv: adapt the device tree of a CoVE guest

[Baolong Duan]

A CoVE guest has neither wired interrupts nor a virtio-mmio transport,
and the TSM does not expose Zicboz to it, so drop the corresponding
device tree properties and nodes.

The kernel command line has to be added here as well: a CoVE guest does
not use the generic riscv_load_kernel() path that normally does it, and
the command line has to be in the device tree before it is measured.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/riscv/fdt-common.c |  4 ++-
 hw/riscv/virt.c       | 62 +++++++++++++++++++++++++++++++------------
 2 files changed, 48 insertions(+), 18 deletions(-)

diff --git a/hw/riscv/fdt-common.c b/hw/riscv/fdt-common.c
index aa143a618b..c065724be4 100644
--- a/hw/riscv/fdt-common.c
+++ b/hw/riscv/fdt-common.c
@@ -11,6 +11,7 @@
 #include "qemu/error-report.h"
 #include "system/device_tree.h"
 #include "hw/core/boards.h"
+#include "hw/riscv/cove.h"
 #include "hw/riscv/fdt-common.h"
 #include "target/riscv/cpu_bits.h"
 
@@ -132,7 +133,8 @@ create_fdt_socket_cpu_internal(void *fdt, char *clust_name, RISCVCPU *cpu_ptr,
                                   cpu_ptr->cfg.cbom_blocksize);
         }
 
-        if (cpu_ptr->cfg.ext_zicboz) {
+        /* The TSM does not expose Zicboz to a CoVE guest. */
+        if (cpu_ptr->cfg.ext_zicboz && !riscv_cove_vm_active()) {
             qemu_fdt_setprop_cell(fdt, cpu_name, "riscv,cboz-block-size",
                                   cpu_ptr->cfg.cboz_blocksize);
         }
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index c15d8859ce..777859baa2 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -225,12 +225,15 @@ static void create_pcie_irq_map(RISCVVirtState *s, void *fdt, char *nodename,
         }
     }
 
-    qemu_fdt_setprop(fdt, nodename, "interrupt-map", full_irq_map,
-                     PCI_NUM_PINS * PCI_NUM_PINS *
-                     irq_map_stride * sizeof(uint32_t));
-
-    qemu_fdt_setprop_cells(fdt, nodename, "interrupt-map-mask",
-                           0x1800, 0, 0, 0x7);
+    /* A CoVE guest only supports MSIs, so it has no interrupt map. */
+    if (!s->cove_vm) {
+        qemu_fdt_setprop(fdt, nodename, "interrupt-map", full_irq_map,
+                         PCI_NUM_PINS * PCI_NUM_PINS *
+                         irq_map_stride * sizeof(uint32_t));
+
+        qemu_fdt_setprop_cells(fdt, nodename, "interrupt-map-mask",
+                               0x1800, 0, 0, 0x7);
+    }
 }
 
 static void create_fdt_socket_aclint(RISCVVirtState *s,
@@ -697,6 +700,14 @@ static void create_fdt_virtio(RISCVVirtState *s, uint32_t irq_virtio_phandle)
     MachineState *ms = MACHINE(s);
     hwaddr virtio_base = s->memmap[VIRT_VIRTIO].base;
 
+    /*
+     * A CoVE guest has no virtio-mmio transport, its virtio devices are
+     * attached to the PCIe host bridge instead.
+     */
+    if (s->cove_vm) {
+        return;
+    }
+
     for (i = 0; i < VIRTIO_COUNT; i++) {
         g_autofree char *name = NULL;
         uint64_t size = s->memmap[VIRT_VIRTIO].size;
@@ -819,11 +830,15 @@ static void create_fdt_uart(RISCVVirtState *s,
                                  2, s->memmap[VIRT_UART0].base,
                                  2, s->memmap[VIRT_UART0].size);
     qemu_fdt_setprop_cell(ms->fdt, name, "clock-frequency", 3686400);
-    qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent", irq_mmio_phandle);
-    if (s->aia_type == VIRT_AIA_TYPE_NONE) {
-        qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", UART0_IRQ);
-    } else {
-        qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", UART0_IRQ, 0x4);
+    /* A CoVE guest has no wired interrupts, the UART is polled. */
+    if (!s->cove_vm) {
+        qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent",
+                              irq_mmio_phandle);
+        if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+            qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", UART0_IRQ);
+        } else {
+            qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", UART0_IRQ, 0x4);
+        }
     }
 
     qemu_fdt_setprop_string(ms->fdt, "/chosen", "stdout-path", name);
@@ -844,12 +859,15 @@ static void create_fdt_rtc(RISCVVirtState *s,
     qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg",
                                  2, s->memmap[VIRT_RTC].base,
                                  2, s->memmap[VIRT_RTC].size);
-    qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent",
-        irq_mmio_phandle);
-    if (s->aia_type == VIRT_AIA_TYPE_NONE) {
-        qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", RTC_IRQ);
-    } else {
-        qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", RTC_IRQ, 0x4);
+    /* A CoVE guest has no wired interrupts, the RTC is polled. */
+    if (!s->cove_vm) {
+        qemu_fdt_setprop_cell(ms->fdt, name, "interrupt-parent",
+            irq_mmio_phandle);
+        if (s->aia_type == VIRT_AIA_TYPE_NONE) {
+            qemu_fdt_setprop_cell(ms->fdt, name, "interrupts", RTC_IRQ);
+        } else {
+            qemu_fdt_setprop_cells(ms->fdt, name, "interrupts", RTC_IRQ, 0x4);
+        }
     }
 }
 
@@ -1021,6 +1039,16 @@ static void create_fdt(RISCVVirtState *s)
 
     qemu_fdt_add_subnode(ms->fdt, "/chosen");
 
+    /*
+     * The kernel command line of a CoVE guest has to be part of the device
+     * tree before it is measured, so it cannot be added by the generic
+     * riscv_load_kernel() path.
+     */
+    if (s->cove_vm && ms->kernel_cmdline && *ms->kernel_cmdline) {
+        qemu_fdt_setprop_string(ms->fdt, "/chosen", "bootargs",
+                                ms->kernel_cmdline);
+    }
+
     /* Pass seed to RNG */
     qemu_guest_getrandom_nofail(rng_seed, sizeof(rng_seed));
     qemu_fdt_setprop(ms->fdt, "/chosen", "rng-seed",
-- 
2.34.1

[RFC PATCH v1 08/17] hw/riscv/virt: use MSIs only for a CoVE guest

[Baolong Duan]

Interrupts of a CoVE guest can only be delivered as MSIs, so an IMSIC is
mandatory: without one PCIe devices have no interrupt source at all and
their drivers fail to probe.  Select aia=aplic-imsic when no AIA mode has
been requested.

Also stop creating the virtio-mmio transports, which a CoVE guest cannot
use; its devices are attached to the PCIe host bridge instead.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/riscv/virt.c | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 777859baa2..dbb82bbd21 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -1373,6 +1373,16 @@ static void virt_machine_init(MachineState *machine)
         exit(1);
     }
 
+    /*
+     * A CoVE guest can only receive MSIs, so an IMSIC is mandatory: without
+     * it PCIe devices have no interrupt source at all and their drivers fail
+     * to probe.
+     */
+    if (s->cove_vm && s->aia_type == VIRT_AIA_TYPE_NONE) {
+        s->aia_type = VIRT_AIA_TYPE_APLIC_IMSIC;
+        warn_report("CoVE VM: forcing aia=aplic-imsic");
+    }
+
     /* Initialize sockets */
     mmio_irqchip = virtio_irqchip = pcie_irqchip = NULL;
     for (i = 0; i < socket_count; i++) {
@@ -1524,10 +1534,13 @@ static void virt_machine_init(MachineState *machine)
     sifive_test_create(s->memmap[VIRT_TEST].base);
 
     /* VirtIO MMIO devices */
-    for (i = 0; i < VIRTIO_COUNT; i++) {
-        sysbus_create_simple("virtio-mmio",
-            s->memmap[VIRT_VIRTIO].base + i * s->memmap[VIRT_VIRTIO].size,
-            qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
+    /* A CoVE guest has no virtio-mmio transport, see create_fdt_virtio(). */
+    if (!s->cove_vm) {
+        for (i = 0; i < VIRTIO_COUNT; i++) {
+            sysbus_create_simple("virtio-mmio",
+                s->memmap[VIRT_VIRTIO].base + i * s->memmap[VIRT_VIRTIO].size,
+                qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
+        }
     }
 
     gpex_pcie_init(system_memory, pcie_irqchip, s);
-- 
2.34.1

[RFC PATCH v1 09/17] hw/intc/riscv_aplic: emulate the APLIC for a CoVE guest

[Baolong Duan]

KVM does not implement an APLIC for a TEE VM, so emulate it in QEMU
while still using the in-kernel IMSIC, exactly like in split irqchip
mode.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/intc/riscv_aplic.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
index 84606e9f3d..897147c2d5 100644
--- a/hw/intc/riscv_aplic.c
+++ b/hw/intc/riscv_aplic.c
@@ -36,6 +36,7 @@
 #include "kvm/kvm_riscv.h"
 #include "migration/vmstate.h"
 #include "trace.h"
+#include "hw/riscv/cove.h"
 
 #define APLIC_MAX_IDC                  (1UL << 14)
 #define APLIC_MAX_SOURCE               1024
@@ -172,6 +173,14 @@ bool riscv_use_emulated_aplic(bool msimode)
         return true;
     }
 
+    /*
+     * KVM does not implement an APLIC for a CoVE guest, so emulate it in
+     * QEMU while still using the in-kernel IMSIC, like in split mode.
+     */
+    if (riscv_cove_vm_active()) {
+        return true;
+    }
+
     if (!riscv_is_kvm_aia_aplic_imsic(msimode)) {
         return true;
     }
-- 
2.34.1

[RFC PATCH v1 10/17] target/riscv/kvm: skip unsupported KVM requests for a CoVE guest

[Baolong Duan]

A TEE VM has no APLIC and no PLIC, and its timer state lives inside the
TVM where the host cannot read it.  Skip the KVM requests that would fail
or return meaningless data: reading the timer back, injecting a wired
interrupt, and the APLIC part of the in-kernel AIA setup.

Note that kvm_mark_guest_state_protected(), which SEV and TDX use, would
cover the vCPU state part of this generically.  It is not used here
because it also changes the reset and migration paths, which have not
been evaluated for CoVE yet.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 target/riscv/kvm/kvm-cpu.c | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index 67c99d68ce..23c2ae2214 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -1477,7 +1477,13 @@ static void kvm_riscv_vm_state_change(void *opaque, bool running,
     if (running) {
         kvm_riscv_put_regs_timer(cs);
     } else {
-        kvm_riscv_get_regs_timer(cs);
+        /*
+         * The timer state of a CoVE guest is kept inside the TVM and cannot
+         * be read back by the host.
+         */
+        if (!riscv_cove_vm_active()) {
+            kvm_riscv_get_regs_timer(cs);
+        }
     }
 }
 
@@ -1791,6 +1797,14 @@ void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level)
     int ret;
     unsigned virq = level ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET;
 
+    /*
+     * A CoVE guest has neither APLIC nor PLIC, so there is no wired
+     * interrupt to inject.
+     */
+    if (riscv_cove_vm_active()) {
+        return;
+    }
+
     if (irq != IRQ_S_EXT) {
         perror("kvm riscv set irq != IRQ_S_EXT\n");
         abort();
@@ -1924,8 +1938,11 @@ void kvm_riscv_aia_create(MachineState *machine, uint64_t group_shift,
      * This is done by leaving KVM_DEV_RISCV_AIA_CONFIG_SRCS
      * unset. We can also skip KVM_DEV_RISCV_AIA_ADDR_APLIC
      * since KVM won't be using it.
+     *
+     * The same applies to a CoVE guest: KVM does not implement an
+     * APLIC for a TVM, only the IMSIC is used.
      */
-    if (!kvm_kernel_irqchip_split()) {
+    if (!kvm_kernel_irqchip_split() && !riscv_cove_vm_active()) {
         ret = kvm_device_access(aia_fd, KVM_DEV_RISCV_AIA_GRP_CONFIG,
                                 KVM_DEV_RISCV_AIA_CONFIG_SRCS,
                                 &aia_irq_num, true, NULL);
-- 
2.34.1

[RFC PATCH v1 11/17] hw/virtio: force modern virtio for a CoVE guest

[Baolong Duan]

Devices have no direct access to the memory of a CoVE guest, so DMA has
to be bounced through buffers that the guest shares explicitly.  Offer
VIRTIO_F_ACCESS_PLATFORM unconditionally, together with
VIRTIO_F_VERSION_1 which Linux requires whenever ACCESS_PLATFORM is
offered.

Such a device therefore always negotiates in modern mode, so the legacy
IOMMU_PLATFORM check in virtio-pci does not apply to it either.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/virtio/virtio-bus.c | 12 ++++++++++++
 hw/virtio/virtio-pci.c |  9 ++++++++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index 9b545acda3..2e2917236f 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -29,6 +29,7 @@
 #include "hw/virtio/virtio-bus.h"
 #include "hw/virtio/virtio.h"
 #include "system/address-spaces.h"
+#include "hw/riscv/cove.h"
 
 /* #define DEBUG_VIRTIO_BUS */
 
@@ -75,6 +76,17 @@ void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp)
         return;
     }
 
+    /*
+     * Devices cannot access the memory of a CoVE guest directly, so all DMA
+     * has to be bounced through buffers the guest shares explicitly: offer
+     * VIRTIO_F_ACCESS_PLATFORM unconditionally. VIRTIO_F_VERSION_1 has to be
+     * offered as well because Linux refuses ACCESS_PLATFORM without it.
+     */
+    if (riscv_cove_vm_active()) {
+        vdev->host_features |= 1ULL << VIRTIO_F_ACCESS_PLATFORM;
+        vdev->host_features |= 1ULL << VIRTIO_F_VERSION_1;
+    }
+
     if (klass->device_plugged != NULL) {
         klass->device_plugged(qbus->parent, &local_err);
     }
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 6f5db5fc42..b257d4a09f 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -34,6 +34,7 @@
 #include "hw/pci/msi.h"
 #include "hw/pci/msix.h"
 #include "hw/core/loader.h"
+#include "hw/riscv/cove.h"
 #include "system/accel-irq.h"
 #include "system/kvm.h"
 #include "hw/virtio/virtio-pci.h"
@@ -2048,7 +2049,13 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)
                 return;
             }
         }
-        if (virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
+        /*
+         * A CoVE guest always negotiates in modern mode: ACCESS_PLATFORM,
+         * which shares its feature bit with IOMMU_PLATFORM, is forced
+         * together with VERSION_1, so this check does not apply.
+         */
+        if (virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM) &&
+            !riscv_cove_vm_active()) {
             error_setg(errp, "VIRTIO_F_IOMMU_PLATFORM was supported by"
                        " neither legacy nor transitional device");
             return;
-- 
2.34.1

[RFC PATCH v1 12/17] hw/net/virtio-net: do not use vhost for a CoVE guest

[Baolong Duan]

The in-kernel vhost datapath has no access to the memory of a TEE VM, so
it cannot be used.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/net/virtio-net.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 814b99a43d..4f8db1a1aa 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -28,6 +28,7 @@
 #include "qemu/config-file.h"
 #include "qobject/qdict.h"
 #include "hw/virtio/virtio-net.h"
+#include "hw/riscv/cove.h"
 #include "net/vhost_net.h"
 #include "net/announce.h"
 #include "hw/virtio/virtio-bus.h"
@@ -326,6 +327,14 @@ static void virtio_net_vhost_status(VirtIONet *n, uint8_t status)
             }
         }
 
+        /*
+         * vhost cannot be used with a CoVE guest: the kernel datapath has no
+         * access to the memory of the TVM.
+         */
+        if (riscv_cove_vm_active()) {
+            return;
+        }
+
         n->vhost_started = 1;
         r = vhost_net_start(vdev, n->nic->ncs, queue_pairs, cvq);
         if (r < 0) {
-- 
2.34.1

[RFC PATCH v1 13/17] accel/kvm: only register the DRAM slot of a CoVE guest

[Baolong Duan]

The TSM tracks the confidential region of a TEE VM itself.  Registering
the remaining memory slots of the machine, such as the mask ROM or the
flash, overwrites that region and the guest fails to start.

Filtering them by slot number is a workaround rather than a solution:
the machine should describe which regions belong to the confidential
guest instead.  Suggestions on how to express that are very welcome.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 accel/kvm/kvm-all.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 005abb1c54..2753bf8b8b 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -26,6 +26,7 @@
 #include "qapi/error.h"
 #include "hw/pci/msi.h"
 #include "hw/pci/msix.h"
+#include "hw/riscv/cove.h"
 #include "hw/s390x/adapter.h"
 #include "gdbstub/enums.h"
 #include "system/kvm_int.h"
@@ -393,6 +394,16 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
     struct kvm_userspace_memory_region2 mem = {};
     int ret;
 
+    /*
+     * Only the DRAM slot is registered with KVM for a CoVE guest: the TSM
+     * tracks the confidential region of the TVM itself and registering the
+     * other slots (MROM, flash, ...) would overwrite it.
+     */
+    if (riscv_cove_vm_active() && (slot->slot & 0xffff) != 0 &&
+        slot->memory_size > 0) {
+        return 0;
+    }
+
     mem.slot = slot->slot | (kml->as_id << 16);
     mem.guest_phys_addr = slot->start_addr;
     mem.userspace_addr = (unsigned long)slot->ram;
-- 
2.34.1

[RFC PATCH v1 14/17] accel/kvm: skip MSI route updates for a CoVE guest

[Baolong Duan]

A CoVE guest has no in-kernel APLIC, so there is no MSI route for KVM to
update and KVM_SET_GSI_ROUTING fails.

This is a workaround: the accelerator should rather be told that GSI
routing is unavailable than have CoVE special cased here.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 accel/kvm/kvm-all.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 2753bf8b8b..aaaa88bd2b 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2448,6 +2448,14 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
 {
     struct kvm_irq_routing_entry kroute = {};
 
+    /*
+     * A CoVE guest has no in-kernel APLIC, so there is no MSI route for KVM
+     * to update.
+     */
+    if (riscv_cove_vm_active()) {
+        return 0;
+    }
+
     if (kvm_gsi_direct_mapping()) {
         return 0;
     }
-- 
2.34.1

[RFC PATCH v1 15/17] accel/kvm: pin the vCPU threads of a CoVE guest

[Baolong Duan]

The TSM binds a TVM vCPU to the hart it first runs on, so the thread
running it has to stay on that host CPU.  With every host CPU running a
pinned vCPU thread the host is then starved, which shows up as RCU
stalls, hence the short sleep after each KVM_RUN.

Both are workarounds that do not belong in target independent code:
thread placement is normally left to the user or to the management layer,
and the sleep papers over a host scheduling problem.  Guidance on how to
express the TSM requirement properly would be appreciated.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 accel/kvm/kvm-accel-ops.c | 16 ++++++++++++++++
 accel/kvm/kvm-all.c       |  9 +++++++++
 2 files changed, 25 insertions(+)

diff --git a/accel/kvm/kvm-accel-ops.c b/accel/kvm/kvm-accel-ops.c
index c8e7aa3870..30e1cb181c 100644
--- a/accel/kvm/kvm-accel-ops.c
+++ b/accel/kvm/kvm-accel-ops.c
@@ -24,6 +24,7 @@
 #include "system/cpus.h"
 #include "qemu/guest-random.h"
 #include "qapi/error.h"
+#include "hw/riscv/cove.h"
 
 #include <linux/kvm.h>
 #include "kvm-cpus.h"
@@ -43,6 +44,21 @@ static void *kvm_vcpu_thread_fn(void *arg)
     r = kvm_init_vcpu(cpu, &error_fatal);
     kvm_init_cpu_signals(cpu);
 
+    /*
+     * The TSM binds a TVM vCPU to the hart it first runs on, so pin vCPU N
+     * to host CPU N before the first KVM_RUN.
+     */
+    if (riscv_cove_vm_active()) {
+        cpu_set_t cpuset;
+
+        CPU_ZERO(&cpuset);
+        CPU_SET(cpu->cpu_index, &cpuset);
+        if (sched_setaffinity(0, sizeof(cpuset), &cpuset) < 0) {
+            error_report("Unable to pin vCPU %d: %s", cpu->cpu_index,
+                         strerror(errno));
+        }
+    }
+
     /* signal CPU creation */
     cpu_thread_signal_created(cpu);
     qemu_guest_random_seed_thread_part2(cpu->random_seed);
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index aaaa88bd2b..7b33a9aa3a 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -3510,6 +3510,15 @@ int kvm_cpu_exec(CPUState *cpu)
          * as true, cpu->exit_request will always read as true.
          */
 
+        /*
+         * Yield briefly after each KVM_RUN of a CoVE guest: with one pinned
+         * vCPU thread per host CPU the host is otherwise starved and reports
+         * RCU stalls.
+         */
+        if (riscv_cove_vm_active()) {
+            usleep(100);
+        }
+
         attrs = kvm_arch_post_run(cpu, run);
 
 #ifdef KVM_HAVE_MCE_INJECTION
-- 
2.34.1

[RFC PATCH v1 16/17] accel/kvm: terminate on a system event of a CoVE guest

[Baolong Duan]

The TSM does not honour a power off request for a TEE VM: the vCPU
re-enters KVM_RUN and reports the same system event again, so QEMU never
shuts the guest down.  Terminate the process instead.

This works around a missing piece of the TSM ABI rather than something
QEMU should be doing.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 accel/kvm/kvm-all.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 7b33a9aa3a..938bbebcd1 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -3613,6 +3613,17 @@ int kvm_cpu_exec(CPUState *cpu)
             break;
         case KVM_EXIT_SYSTEM_EVENT:
             trace_kvm_run_exit_system_event(cpu->cpu_index, run->system_event.type);
+            /*
+             * The TSM does not honour power_off for a TVM: the vCPU would
+             * re-enter KVM_RUN and report the same event forever, so
+             * terminate QEMU right away.
+             */
+            if (riscv_cove_vm_active()) {
+                warn_report("CoVE VM: system event %d, terminating",
+                            run->system_event.type);
+                exit(run->system_event.type == KVM_SYSTEM_EVENT_SHUTDOWN ?
+                     0 : 1);
+            }
             switch (run->system_event.type) {
             case KVM_SYSTEM_EVENT_SHUTDOWN:
                 qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
-- 
2.34.1

[RFC PATCH v1 17/17] hw/core/machine-qmp-cmds: shut down a CoVE guest on reset

[Baolong Duan]

A TEE VM cannot be reset: the TSM destroys it and the host cannot
recreate it with the same measurement, so treat system_reset as a
shutdown request.

Changing the meaning of a QMP command like this is not acceptable as is.
QEMU already has a generic mechanism for guests that cannot be rebuilt,
qemu_system_reset_request() checks cpus_are_resettable() together with
confidential_guest_can_rebuild_state(), and using it would require CoVE
to provide a ConfidentialGuestSupport object.  This patch is included to
document the limitation and to ask whether that is the direction to
take.

Signed-off-by: Baolong Duan <blduan@linux.alibaba.com>
---
 hw/core/machine-qmp-cmds.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index e62cb4ec88..74a9274ab3 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -12,6 +12,7 @@
 #include "hw/core/boards.h"
 #include "hw/intc/intc.h"
 #include "hw/mem/memory-device.h"
+#include "hw/riscv/cove.h"
 #include "qapi/error.h"
 #include "qapi/qapi-builtin-visit.h"
 #include "qapi/qapi-commands-accelerator.h"
@@ -302,6 +303,15 @@ UuidInfo *qmp_query_uuid(Error **errp)
 
 void qmp_system_reset(Error **errp)
 {
+    /*
+     * Resetting a CoVE guest destroys the TVM, which cannot be recreated
+     * from the host, so shut the guest down instead.
+     */
+    if (riscv_cove_vm_active()) {
+        qemu_system_shutdown_request(SHUTDOWN_CAUSE_HOST_QMP_SYSTEM_RESET);
+        return;
+    }
+
     qemu_system_reset_request(SHUTDOWN_CAUSE_HOST_QMP_SYSTEM_RESET);
 }
 
-- 
2.34.1