[PATCH v3 00/26] arm: Run Arm CCA VMs with KVM

[PATCH v3 00/26] arm: Run Arm CCA VMs with KVM

[Jean-Philippe Brucker]

This series enables running confidential VMs on Arm CCA. The host KVM
support is progressing but still under discussion [1], so there is no
urgency to upstream this series. I'm sending this new version to give a
status update, and also to discuss remote attestation below.

Since v2 [2] I addressed comments on the QAPI patches. The support for
running Linux in a Realm will be in Linux v6.13 [3], so the guest-facing
interface is now stable. One important change since v2 is the requirement
to initialize the whole GPA space in RMM before boot, which we do in patch
9. The 'earlycon' kernel parameter now requires an unprotected address
parameter (see Documentation/arch/arm64/arm-cca.rst in Linux v6.13).

Documentation to try this series out:
https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/Building+an+RME+stack+for+QEMU

The new RFC patches 21-26 are ideas to make remote attestation easier.
They're all optional, and in order to understand which patches should be
kept and what should be standardized, we'd like feedback from people who
will provide hosting and verification services, and also from people
working on other VMMs, who will be faced with the same problems.


QEMU and Realm attestation
==========================

Confidential computing wouldn't be complete without the ability to attest
that a remote VM you own is running your software on the correct hardware.
There are many components in enabling remote attestation, but I'd like to
discuss specifically how to independently reconstruct the initial state of
the VM created by QEMU.

We can illustrate this problem using the keybroker attestation demo [4]:

          ┌──────┬───────────────┐                           
          │  NS  │    REALM      │            RELYING PARTY 
      ┌───┼──────┼───────────────┤     (b)    (you are here) 
      │   │      │               │  evidence  ┌────────────┐ 
      │EL0│ QEMU │ keybroker-app ┼────────────► keybroker- │ 
      │   │   │  │      ▲        ◄────────────┼   server   │ 
      ├───┼───┼──┼──────┼────────┤   secret   └─────▲──────┘ 
      │   │   │  │      │        │     (d)          │ (c)    
      │EL1│   │  │ guest kernel  │              ┌───▼────┐   
      │   │   │  │      ▲        │              │veraison│   
      │   │   │  ├──────┼────────┤              └────────┘   
      │   │   ▼  │ (a)  │        │               VERIFIER    
      │EL2│  KVM ┼───► RMM       │                           
      │   │      │               │                           
      └───┴──────┴───────────────┘                           
                ATTESTER                                       

QEMU and KVM load data into the VM memory (kernel, initrd, firmware, etc),
before activating it and sealing its initial state (a). The initial state
is recorded by the RMM as the Realm Initial Measurement (RIM). Additional
data may be loaded at runtime, for example a bootloader loading an initrd
from untrusted storage. Those are recorded in Realm Extensible
Measurements (REM).

Now say you want the VM to run an application, keybroker-app, that
downloads a secret from your keybroker-server, for example a private key
to open an encrypted archive.

  keybroker-server -e http://10.0.2.2 -vv
  keybroker-app -e http://10.0.2.2:8088 -v <key> 

keybroker-app obtains signed evidence using Linux configfs-tsm, sends it
to keybroker-server (b). The evidence contains a platform token describing
the platform state (host hardware, firmware and RMM), and a realm token.
The signature and platform verification, proving that the VM is running in
a confidential environment, is offloaded to a trusted third party in this
example (c), though you could of course run veraison yourself.
keybroker-server then checks the RIM and REMs contained in the realm
token, to ensure the Realm is running the correct payload, and sends the
secret key (d).


Computing the Realm Initial Measurement
---------------------------------------

The simplest way to obtain the RIM is to run the payload once on a trusted
machine, extract the reference values and provision keybroker-server with
them. On the first run the demo tells you how to do this:

  INFO Known-good RIM values are missing. If you trust the client that
    submitted evidence for challenge 352278775, you should restart the
    keybroker-server with the following command-line option to populate it
    with known-good RIM values:
    --reference-values <(echo '{ "reference-values": [ "Up9LS4jrkX002..." ] }')

This is fine for testing, but does not scale well. For one thing you may
not have access to a trusted machine to run the payload. Also, the RIM is
relatively unstable: it depends on the host features (eg. SVE vector
length), the VM configuration (eg. amount of RAM) and the images loaded
into the Realm. Since the DTB/ACPI tables are included in the
measurements, changing the number of CPUs or MMIO devices will also
produce a different RIM. Some VMMs even have unstable IRQ allocation.

To pre-calculate the RIM for a set of VM parameters, rather than running
the workload once you can use the realm-measurements tool [5]:

  realm-measurements --print-b64
    -c configs/qemu-max-8.2.conf -c configs/kvm.conf  # host caps
    -k <kernel> -i <initrd>                           # payload
    qemu <qemu command-line parameters>

  RIM: Up9LS4jrkX002...

Internally realm-measurements generates a DTB for the given machine
configuration and uses it to calculate the RIM. This means that QEMU must
be run with the same generated DTB, which can be obtained by running
realm-measurements with `--output-dtb qemu.dtb --no-measurements` on the
host. See patch 21, which enables loading this DTB.

I've also experimented with reproducing the DTB generated by QEMU,
from a template obtained with `-M dumpdtb=<tpl.dtb>` [6]. It's rather
fragile due to phandle allocation, property names and libfdt internals.
You can try this mode by passing `--input-dtb <tpl.dtb>`, in which case
there is no need to pass a DTB to QEMU. But this solution will probably
remain experimental.


An event log from the VMM
-------------------------

Reconstructing the RIM this way requires a strict specification of the VMM
behavior [7]. To relax this, the VMM could provide an event log,
describing the different operations that contributed to the RIM: Realm
parameters, images loaded into RAM, initial CPU registers. The Trusted
Computing Group (TCG) defines a log format that describes things measured
by a TPM. Since this format will probably be reused to describe images
measured into the REM, it would make sense to adopt it for the RIM as
well. UEFI normally creates this log in RAM and passes it to the OS via
firmware table (CCEL for ACPI, TBD [8] for device tree)

Patches 23-26 show how we could load such event log into Realm memory.
Something to improve is the image identifiers passed in the log, that
allow the verifier to locate the image in its store of known-good images.
At the moment we use a hash function for this, but it would be useful to
either let the host pass the pre-calculated hash, or we could extend RMM
to return the hashes that it already computes when measuring data
granules.

Using a modified keybroker-demo that supports the event log [10], you can
pass a list of known-good images loaded into the realm:

  sha512sum path/to/images/* > checksums.txt
  keybroker-server -e http://10.0.2.2 -vv --images checksums.txt

Now when receiving evidence, the server can reconstruct the RIM and REMs
by parsing the event log. To reduce bandwidth use, the server caches the
computed reference values, and only requires an event log when reference
values are absent.

Note that the PoC is incomplete, because it trusts the log too much when
constructing the RIM: although data loaded into the Realm is correctly
verified, some events aren't properly checked by the policy at the moment.
In particular the REC event (vCPU initial state) must have sane initial
registers (which can be predicted from the DTB, KERNEL and FIRMWARE log
events, so should be easy to fix). The kernel parameters, contained in the
DTB, will also need to be validated by the policy.

The server reconstructs the DTB using a few VM properties passed in the
log (number of CPUs, RAM size). Another possible extension is to send the
DTB/ACPI tables in the log, so the tool can verify their content rather
than generate them, but I'm not sure it's even worth attempting.


Loading IGVM payload 
--------------------

IGVM [9] packs the payload along with instructions for creating the VM,
into a file that is fed to QEMU. We'll likely want to support it on Arm in
order to load complex payloads like COCONUT-SVSM, but probably not for
lightweight container payloads where firmware would be unnecessary
overhead.

If the IGVM file describes all operations leading to a RIM, then a strict
VM specification isn't needed in this case either. However to get more
flexibility it may be useful to combine IGVM with an event log: the host
packs the IGVM file including an event log, and the relying party only
needs the event log to reconstruct the RIM, it doesn't need to upload a
pack for each different configuration.



[1] [PATCH v5] arm64: Support for Arm CCA in KVM
    https://lore.kernel.org/all/20241004152804.72508-1-steven.price@arm.com/

[2] [PATCH v2] arm: Run CCA VMs with KVM
    https://lore.kernel.org/all/20240419155709.318866-2-jean-philippe@linaro.org/

[3] [PATCH v7] arm64: Support for running as a guest in Arm CCA
    https://lore.kernel.org/all/172967739783.1412028.8494484908145931121.b4-ty@arm.com/

[4] Veraison keybroker demo
    https://github.com/veraison/keybroker-demo/

[5] Realm measurements tool
    https://github.com/veraison/cca-realm-measurements
    https://docs.rs/cca-realm-measurements/0.1.0/cca_realm_measurements/

[6] DTB modification and libfdt changes
    https://github.com/veraison/cca-realm-measurements/commit/4b7aa3cbd7ae7ce114726979367329f57b8ef2c6
    https://git.codelinaro.org/linaro/dcap/buildroot-external-cca/-/commit/28f4afa6120b467cadab9bc0af67e7bab239797b

[7] VM specification
    https://github.com/veraison/cca-realm-measurements/blob/main/docs/realm-vm.md

[8] Confidential computing event log for device-tree
    https://jpbrucker.net/git/linux/log/?h=cca/ccel-dt

[9] IGVM in QEMU, COCONUT-SVSM
    https://lore.kernel.org/qemu-devel/cover.1727341768.git.roy.hopkins@suse.com/
    https://github.com/coconut-svsm/svsm/blob/main/Documentation/docs/installation/INSTALL.md

[10] keybroker-demo with event log support
     https://github.com/jpbrucker/keybroker-demo/commits/event-log/

Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Matias Ezequiel Vara Larsen <mvaralar@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Gonda <pgonda@google.com>
Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>

Jean-Philippe Brucker (26):
  kvm: Merge kvm_check_extension() and kvm_vm_check_extension()
  target/arm: Add confidential guest support
  target/arm/kvm: Return immediately on error in kvm_arch_init()
  target/arm/kvm-rme: Initialize realm
  target/arm/kvm: Split kvm_arch_get/put_registers
  target/arm/kvm-rme: Initialize vCPU
  target/arm/kvm: Create scratch VM as Realm if necessary
  hw/core/loader: Add ROM loader notifier
  target/arm/kvm-rme: Initialize Realm memory
  target/arm/kvm-rme: Add Realm Personalization Value parameter
  target/arm/kvm-rme: Add measurement algorithm property
  target/arm/cpu: Set number of breakpoints and watchpoints in KVM
  target/arm/cpu: Set number of PMU counters in KVM
  target/arm/cpu: Inform about reading confidential CPU registers
  hw/arm/virt: Add support for Arm RME
  hw/arm/virt: Disable DTB randomness for confidential VMs
  hw/arm/virt: Reserve one bit of guest-physical address for RME
  hw/arm/boot: Mark all guest memory as RIPAS_RAM.
  hw/arm/virt: Move virt_flash_create() to machvirt_init()
  hw/arm/virt: Use RAM instead of flash for confidential guest firmware
  hw/arm/boot: Load DTB as is for confidential VMs
  hw/arm/boot: Skip bootloader for confidential guests
  hw/tpm: Add TPM event log
  hw/core/loader: Add fields to RomLoaderNotify
  target/arm/kvm-rme: Add measurement log
  hw/arm/virt: Add measurement log for confidential boot

 docs/system/arm/virt.rst                   |   9 +-
 docs/system/confidential-guest-support.rst |   1 +
 qapi/qom.json                              |  40 +
 qapi/tpm.json                              |  14 +
 include/hw/arm/boot.h                      |  18 +
 include/hw/arm/virt.h                      |   3 +-
 include/hw/loader.h                        |  17 +
 include/hw/tpm/tpm_log.h                   |  89 +++
 include/sysemu/kvm.h                       |   2 -
 include/sysemu/kvm_int.h                   |   1 +
 target/arm/cpu.h                           |  10 +
 target/arm/kvm_arm.h                       |  98 +++
 accel/kvm/kvm-all.c                        |  41 +-
 hw/arm/boot.c                              | 115 ++-
 hw/arm/virt.c                              | 143 +++-
 hw/core/loader.c                           |  17 +
 hw/tpm/tpm_log.c                           | 325 ++++++++
 target/arm/arm-qmp-cmds.c                  |   1 +
 target/arm/cpu.c                           |   5 +
 target/arm/cpu64.c                         | 118 +++
 target/arm/kvm-rme.c                       | 843 +++++++++++++++++++++
 target/arm/kvm.c                           | 201 ++++-
 target/i386/kvm/kvm.c                      |   6 +-
 target/ppc/kvm.c                           |  36 +-
 hw/tpm/Kconfig                             |   4 +
 hw/tpm/meson.build                         |   1 +
 target/arm/Kconfig                         |   1 +
 target/arm/meson.build                     |   7 +-
 28 files changed, 2075 insertions(+), 91 deletions(-)
 create mode 100644 include/hw/tpm/tpm_log.h
 create mode 100644 hw/tpm/tpm_log.c
 create mode 100644 target/arm/kvm-rme.c

-- 
2.47.0

[PATCH v3 01/26] kvm: Merge kvm_check_extension() and kvm_vm_check_extension()

[Jean-Philippe Brucker]

The KVM_CHECK_EXTENSION ioctl can be issued either on the global fd
(/dev/kvm), or on the VM fd obtained with KVM_CREATE_VM. For most
extensions, KVM returns the same value with either method, but for some
of them it can refine the returned value depending on the VM type. The
KVM documentation [1] advises to use the VM fd:

  Based on their initialization different VMs may have different
  capabilities. It is thus encouraged to use the vm ioctl to query for
  capabilities (available with KVM_CAP_CHECK_EXTENSION_VM on the vm fd)

Ongoing work on Arm confidential VMs confirms this, as some capabilities
become unavailable to confidential VMs, requiring changes in QEMU to use
kvm_vm_check_extension() instead of kvm_check_extension() [2]. Rather
than changing each check one by one, change kvm_check_extension() to
always issue the ioctl on the VM fd when available, and remove
kvm_vm_check_extension().

Fall back to the global fd when the VM check is unavailable:

* Ancient kernels do not support KVM_CHECK_EXTENSION on the VM fd, since
  it was added by commit 92b591a4c46b ("KVM: Allow KVM_CHECK_EXTENSION
  on the vm fd") in Linux 3.17 [3]. Support for Linux 3.16 ended in June
  2020, but there may still be old images around.

* A couple of calls must be issued before the VM fd is available, since
  they determine the VM type: KVM_CAP_MIPS_VZ and KVM_CAP_ARM_VM_IPA_SIZE

Does any user actually depend on the check being done on the global fd
instead of the VM fd?  I surveyed all cases where KVM presently returns
different values depending on the query method. Luckily QEMU already
calls kvm_vm_check_extension() for most of those. Only three of them are
ambiguous, because currently done on the global fd:

* KVM_CAP_MAX_VCPUS and KVM_CAP_MAX_VCPU_ID on Arm, changes value if the
  user requests a vGIC different from the default. But QEMU queries this
  before vGIC configuration, so the reported value will be the same.

* KVM_CAP_SW_TLB on PPC. When issued on the global fd, returns false if
  the kvm-hv module is loaded; when issued on the VM fd, returns false
  only if the VM type is HV instead of PR. If this returns false, then
  QEMU will fail to initialize a BOOKE206 MMU model.

  So this patch supposedly improves things, as it allows to run this
  type of vCPU even when both KVM modules are loaded.

* KVM_CAP_PPC_SECURE_GUEST. Similarly, doing this check on a VM fd
  refines the returned value, and ensures that SVM is actually
  supported. Since QEMU follows the check with kvm_vm_enable_cap(), this
  patch should only provide better error reporting.

[1] https://www.kernel.org/doc/html/latest/virt/kvm/api.html#kvm-check-extension
[2] https://lore.kernel.org/kvm/875ybi0ytc.fsf@redhat.com/
[3] https://github.com/torvalds/linux/commit/92b591a4c46b

Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Daniel Henrique Barboza <danielhb413@gmail.com>
Cc: qemu-ppc@nongnu.org
Suggested-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 include/sysemu/kvm.h     |  2 --
 include/sysemu/kvm_int.h |  1 +
 accel/kvm/kvm-all.c      | 41 +++++++++++++++++++---------------------
 target/arm/kvm.c         |  2 +-
 target/i386/kvm/kvm.c    |  6 +++---
 target/ppc/kvm.c         | 36 +++++++++++++++++------------------
 6 files changed, 42 insertions(+), 46 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index c3a60b2890..63c96d0096 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -437,8 +437,6 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cpu);
 
 int kvm_check_extension(KVMState *s, unsigned int extension);
 
-int kvm_vm_check_extension(KVMState *s, unsigned int extension);
-
 #define kvm_vm_enable_cap(s, capability, cap_flags, ...)             \
     ({                                                               \
         struct kvm_enable_cap cap = {                                \
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index a1e72763da..cb38085d54 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -166,6 +166,7 @@ struct KVMState
     uint16_t xen_gnttab_max_frames;
     uint16_t xen_evtchn_max_pirq;
     char *device;
+    bool check_extension_vm;
 };
 
 void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 801cff16a5..7ea016d598 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1238,7 +1238,11 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
 {
     int ret;
 
-    ret = kvm_ioctl(s, KVM_CHECK_EXTENSION, extension);
+    if (!s->check_extension_vm) {
+        ret = kvm_ioctl(s, KVM_CHECK_EXTENSION, extension);
+    } else {
+        ret = kvm_vm_ioctl(s, KVM_CHECK_EXTENSION, extension);
+    }
     if (ret < 0) {
         ret = 0;
     }
@@ -1246,19 +1250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
     return ret;
 }
 
-int kvm_vm_check_extension(KVMState *s, unsigned int extension)
-{
-    int ret;
-
-    ret = kvm_vm_ioctl(s, KVM_CHECK_EXTENSION, extension);
-    if (ret < 0) {
-        /* VM wide version not implemented, use global one instead */
-        ret = kvm_check_extension(s, extension);
-    }
-
-    return ret;
-}
-
 /*
  * We track the poisoned pages to be able to:
  * - replace them on VM reset
@@ -1622,10 +1613,10 @@ static int kvm_dirty_ring_init(KVMState *s)
      * Read the max supported pages. Fall back to dirty logging mode
      * if the dirty ring isn't supported.
      */
-    ret = kvm_vm_check_extension(s, capability);
+    ret = kvm_check_extension(s, capability);
     if (ret <= 0) {
         capability = KVM_CAP_DIRTY_LOG_RING_ACQ_REL;
-        ret = kvm_vm_check_extension(s, capability);
+        ret = kvm_check_extension(s, capability);
     }
 
     if (ret <= 0) {
@@ -1648,7 +1639,7 @@ static int kvm_dirty_ring_init(KVMState *s)
     }
 
     /* Enable the backup bitmap if it is supported */
-    ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP);
+    ret = kvm_check_extension(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP);
     if (ret > 0) {
         ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP, 0);
         if (ret) {
@@ -2404,7 +2395,7 @@ static void kvm_irqchip_create(KVMState *s)
  */
 static int kvm_recommended_vcpus(KVMState *s)
 {
-    int ret = kvm_vm_check_extension(s, KVM_CAP_NR_VCPUS);
+    int ret = kvm_check_extension(s, KVM_CAP_NR_VCPUS);
     return (ret) ? ret : 4;
 }
 
@@ -2625,7 +2616,12 @@ static int kvm_init(MachineState *ms)
 
     s->vmfd = ret;
 
-    s->nr_as = kvm_vm_check_extension(s, KVM_CAP_MULTI_ADDRESS_SPACE);
+    ret = kvm_vm_ioctl(s, KVM_CHECK_EXTENSION, KVM_CAP_CHECK_EXTENSION_VM);
+    if (ret > 0) {
+        s->check_extension_vm = true;
+    }
+
+    s->nr_as = kvm_check_extension(s, KVM_CAP_MULTI_ADDRESS_SPACE);
     if (s->nr_as <= 1) {
         s->nr_as = 1;
     }
@@ -2683,7 +2679,7 @@ static int kvm_init(MachineState *ms)
     }
 
     kvm_readonly_mem_allowed =
-        (kvm_vm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
+        (kvm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
 
     kvm_resamplefds_allowed =
         (kvm_check_extension(s, KVM_CAP_IRQFD_RESAMPLE) > 0);
@@ -2717,7 +2713,8 @@ static int kvm_init(MachineState *ms)
         goto err;
     }
 
-    kvm_supported_memory_attributes = kvm_vm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
+    kvm_supported_memory_attributes =
+        kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
     kvm_guest_memfd_supported =
         kvm_check_extension(s, KVM_CAP_GUEST_MEMFD) &&
         kvm_check_extension(s, KVM_CAP_USER_MEMORY2) &&
@@ -2743,7 +2740,7 @@ static int kvm_init(MachineState *ms)
     memory_listener_register(&kvm_io_listener,
                              &address_space_io);
 
-    s->sync_mmu = !!kvm_vm_check_extension(kvm_state, KVM_CAP_SYNC_MMU);
+    s->sync_mmu = !!kvm_check_extension(kvm_state, KVM_CAP_SYNC_MMU);
     if (!s->sync_mmu) {
         ret = ram_block_discard_disable(true);
         assert(!ret);
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 7b6812c0de..8bdf4abeb6 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -601,7 +601,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     if (s->kvm_eager_split_size) {
         uint32_t sizes;
 
-        sizes = kvm_vm_check_extension(s, KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES);
+        sizes = kvm_check_extension(s, KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES);
         if (!sizes) {
             s->kvm_eager_split_size = 0;
             warn_report("Eager Page Split support not available");
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 8e17942c3b..2f35e7468c 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -244,7 +244,7 @@ bool kvm_enable_hypercall(uint64_t enable_mask)
 
 bool kvm_has_smm(void)
 {
-    return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
+    return kvm_check_extension(kvm_state, KVM_CAP_X86_SMM);
 }
 
 bool kvm_has_adjust_clock_stable(void)
@@ -3320,7 +3320,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
         }
     }
 
-    if (kvm_vm_check_extension(s, KVM_CAP_X86_USER_SPACE_MSR)) {
+    if (kvm_check_extension(s, KVM_CAP_X86_USER_SPACE_MSR)) {
         ret = kvm_vm_enable_userspace_msr(s);
         if (ret < 0) {
             return ret;
@@ -5936,7 +5936,7 @@ static bool __kvm_enable_sgx_provisioning(KVMState *s)
 {
     int fd, ret;
 
-    if (!kvm_vm_check_extension(s, KVM_CAP_SGX_ATTRIBUTE)) {
+    if (!kvm_check_extension(s, KVM_CAP_SGX_ATTRIBUTE)) {
         return false;
     }
 
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 3efc28f18b..8bcb0368ce 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -110,7 +110,7 @@ static uint32_t debug_inst_opcode;
 static bool kvmppc_is_pr(KVMState *ks)
 {
     /* Assume KVM-PR if the GET_PVINFO capability is available */
-    return kvm_vm_check_extension(ks, KVM_CAP_PPC_GET_PVINFO) != 0;
+    return kvm_check_extension(ks, KVM_CAP_PPC_GET_PVINFO) != 0;
 }
 
 static int kvm_ppc_register_host_cpu_type(void);
@@ -127,11 +127,11 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_interrupt_unset = kvm_check_extension(s, KVM_CAP_PPC_UNSET_IRQ);
     cap_segstate = kvm_check_extension(s, KVM_CAP_PPC_SEGSTATE);
     cap_booke_sregs = kvm_check_extension(s, KVM_CAP_PPC_BOOKE_SREGS);
-    cap_ppc_smt_possible = kvm_vm_check_extension(s, KVM_CAP_PPC_SMT_POSSIBLE);
+    cap_ppc_smt_possible = kvm_check_extension(s, KVM_CAP_PPC_SMT_POSSIBLE);
     cap_spapr_tce = kvm_check_extension(s, KVM_CAP_SPAPR_TCE);
     cap_spapr_tce_64 = kvm_check_extension(s, KVM_CAP_SPAPR_TCE_64);
     cap_spapr_multitce = kvm_check_extension(s, KVM_CAP_SPAPR_MULTITCE);
-    cap_spapr_vfio = kvm_vm_check_extension(s, KVM_CAP_SPAPR_TCE_VFIO);
+    cap_spapr_vfio = kvm_check_extension(s, KVM_CAP_SPAPR_TCE_VFIO);
     cap_one_reg = kvm_check_extension(s, KVM_CAP_ONE_REG);
     cap_hior = kvm_check_extension(s, KVM_CAP_PPC_HIOR);
     cap_epr = kvm_check_extension(s, KVM_CAP_PPC_EPR);
@@ -140,23 +140,23 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
      * Note: we don't set cap_papr here, because this capability is
      * only activated after this by kvmppc_set_papr()
      */
-    cap_htab_fd = kvm_vm_check_extension(s, KVM_CAP_PPC_HTAB_FD);
+    cap_htab_fd = kvm_check_extension(s, KVM_CAP_PPC_HTAB_FD);
     cap_fixup_hcalls = kvm_check_extension(s, KVM_CAP_PPC_FIXUP_HCALL);
-    cap_ppc_smt = kvm_vm_check_extension(s, KVM_CAP_PPC_SMT);
-    cap_htm = kvm_vm_check_extension(s, KVM_CAP_PPC_HTM);
-    cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
-    cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
-    cap_xive = kvm_vm_check_extension(s, KVM_CAP_PPC_IRQ_XIVE);
-    cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
+    cap_ppc_smt = kvm_check_extension(s, KVM_CAP_PPC_SMT);
+    cap_htm = kvm_check_extension(s, KVM_CAP_PPC_HTM);
+    cap_mmu_radix = kvm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
+    cap_mmu_hash_v3 = kvm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
+    cap_xive = kvm_check_extension(s, KVM_CAP_PPC_IRQ_XIVE);
+    cap_resize_hpt = kvm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
     kvmppc_get_cpu_characteristics(s);
-    cap_ppc_nested_kvm_hv = kvm_vm_check_extension(s, KVM_CAP_PPC_NESTED_HV);
+    cap_ppc_nested_kvm_hv = kvm_check_extension(s, KVM_CAP_PPC_NESTED_HV);
     cap_large_decr = kvmppc_get_dec_bits();
-    cap_fwnmi = kvm_vm_check_extension(s, KVM_CAP_PPC_FWNMI);
+    cap_fwnmi = kvm_check_extension(s, KVM_CAP_PPC_FWNMI);
     /*
      * Note: setting it to false because there is not such capability
      * in KVM at this moment.
      *
-     * TODO: call kvm_vm_check_extension() with the right capability
+     * TODO: call kvm_check_extension() with the right capability
      * after the kernel starts implementing it.
      */
     cap_ppc_pvr_compat = false;
@@ -166,8 +166,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
         exit(1);
     }
 
-    cap_rpt_invalidate = kvm_vm_check_extension(s, KVM_CAP_PPC_RPT_INVALIDATE);
-    cap_ail_mode_3 = kvm_vm_check_extension(s, KVM_CAP_PPC_AIL_MODE_3);
+    cap_rpt_invalidate = kvm_check_extension(s, KVM_CAP_PPC_RPT_INVALIDATE);
+    cap_ail_mode_3 = kvm_check_extension(s, KVM_CAP_PPC_AIL_MODE_3);
     kvm_ppc_register_host_cpu_type();
 
     return 0;
@@ -1976,7 +1976,7 @@ static int kvmppc_get_pvinfo(CPUPPCState *env, struct kvm_ppc_pvinfo *pvinfo)
 {
     CPUState *cs = env_cpu(env);
 
-    if (kvm_vm_check_extension(cs->kvm_state, KVM_CAP_PPC_GET_PVINFO) &&
+    if (kvm_check_extension(cs->kvm_state, KVM_CAP_PPC_GET_PVINFO) &&
         !kvm_vm_ioctl(cs->kvm_state, KVM_PPC_GET_PVINFO, pvinfo)) {
         return 0;
     }
@@ -2298,7 +2298,7 @@ int kvmppc_reset_htab(int shift_hint)
         /* Full emulation, tell caller to allocate htab itself */
         return 0;
     }
-    if (kvm_vm_check_extension(kvm_state, KVM_CAP_PPC_ALLOC_HTAB)) {
+    if (kvm_check_extension(kvm_state, KVM_CAP_PPC_ALLOC_HTAB)) {
         int ret;
         ret = kvm_vm_ioctl(kvm_state, KVM_PPC_ALLOCATE_HTAB, &shift);
         if (ret == -ENOTTY) {
@@ -2507,7 +2507,7 @@ static void kvmppc_get_cpu_characteristics(KVMState *s)
     cap_ppc_safe_bounds_check = 0;
     cap_ppc_safe_indirect_branch = 0;
 
-    ret = kvm_vm_check_extension(s, KVM_CAP_PPC_GET_CPU_CHAR);
+    ret = kvm_check_extension(s, KVM_CAP_PPC_GET_CPU_CHAR);
     if (!ret) {
         return;
     }
-- 
2.47.0

[PATCH v3 02/26] target/arm: Add confidential guest support

[Jean-Philippe Brucker]

Add a new RmeGuest object, inheriting from ConfidentialGuestSupport, to
support the Arm Realm Management Extension (RME). It is instantiated by
passing on the command-line:

  -M virt,confidential-guest-support=<id>
  -object rme-guest,id=<id>[,options...]

This is only the skeleton. Support will be added in following patches.

Cc: Eric Blake <eblake@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Eduardo Habkost <eduardo@habkost.net>
Acked-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: remove some boilerplate with OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES
---
 docs/system/confidential-guest-support.rst |  1 +
 target/arm/kvm-rme.c                       | 40 ++++++++++++++++++++++
 target/arm/meson.build                     |  7 +++-
 3 files changed, 47 insertions(+), 1 deletion(-)
 create mode 100644 target/arm/kvm-rme.c

diff --git a/docs/system/confidential-guest-support.rst b/docs/system/confidential-guest-support.rst
index 0c490dbda2..acf46d8856 100644
--- a/docs/system/confidential-guest-support.rst
+++ b/docs/system/confidential-guest-support.rst
@@ -40,5 +40,6 @@ Currently supported confidential guest mechanisms are:
 * AMD Secure Encrypted Virtualization (SEV) (see :doc:`i386/amd-memory-encryption`)
 * POWER Protected Execution Facility (PEF) (see :ref:`power-papr-protected-execution-facility-pef`)
 * s390x Protected Virtualization (PV) (see :doc:`s390x/protvirt`)
+* Arm Realm Management Extension (RME)
 
 Other mechanisms may be supported in future.
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
new file mode 100644
index 0000000000..67909349c1
--- /dev/null
+++ b/target/arm/kvm-rme.c
@@ -0,0 +1,40 @@
+/*
+ * QEMU Arm RME support
+ *
+ * Copyright Linaro 2024
+ */
+
+#include "qemu/osdep.h"
+
+#include "exec/confidential-guest-support.h"
+#include "hw/boards.h"
+#include "hw/core/cpu.h"
+#include "kvm_arm.h"
+#include "migration/blocker.h"
+#include "qapi/error.h"
+#include "qom/object_interfaces.h"
+#include "sysemu/kvm.h"
+#include "sysemu/runstate.h"
+
+#define TYPE_RME_GUEST "rme-guest"
+OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
+
+struct RmeGuest {
+    ConfidentialGuestSupport parent_obj;
+};
+
+OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
+                                          CONFIDENTIAL_GUEST_SUPPORT,
+                                          { TYPE_USER_CREATABLE }, { })
+
+static void rme_guest_class_init(ObjectClass *oc, void *data)
+{
+}
+
+static void rme_guest_init(Object *obj)
+{
+}
+
+static void rme_guest_finalize(Object *obj)
+{
+}
diff --git a/target/arm/meson.build b/target/arm/meson.build
index 2e10464dbb..c610c078f7 100644
--- a/target/arm/meson.build
+++ b/target/arm/meson.build
@@ -8,7 +8,12 @@ arm_ss.add(files(
 ))
 arm_ss.add(zlib)
 
-arm_ss.add(when: 'CONFIG_KVM', if_true: files('hyp_gdbstub.c', 'kvm.c'), if_false: files('kvm-stub.c'))
+arm_ss.add(when: 'CONFIG_KVM',
+  if_true: files(
+    'hyp_gdbstub.c',
+    'kvm.c',
+    'kvm-rme.c'),
+  if_false: files('kvm-stub.c'))
 arm_ss.add(when: 'CONFIG_HVF', if_true: files('hyp_gdbstub.c'))
 
 arm_ss.add(when: 'TARGET_AARCH64', if_true: files(
-- 
2.47.0

[PATCH v3 03/26] target/arm/kvm: Return immediately on error in kvm_arch_init()

[Jean-Philippe Brucker]

Returning an error to kvm_init() is fatal anyway, no need to continue
the initialization.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/kvm.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 8bdf4abeb6..95bcecf804 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -563,7 +563,7 @@ int kvm_arch_get_default_type(MachineState *ms)
 
 int kvm_arch_init(MachineState *ms, KVMState *s)
 {
-    int ret = 0;
+    int ret;
     /* For ARM interrupt delivery is always asynchronous,
      * whether we are using an in-kernel VGIC or not.
      */
@@ -585,7 +585,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
         !kvm_check_extension(s, KVM_CAP_ARM_IRQ_LINE_LAYOUT_2)) {
         error_report("Using more than 256 vcpus requires a host kernel "
                      "with KVM_CAP_ARM_IRQ_LINE_LAYOUT_2");
-        ret = -EINVAL;
+        return -EINVAL;
     }
 
     if (kvm_check_extension(s, KVM_CAP_ARM_NISV_TO_USER)) {
@@ -607,13 +607,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
             warn_report("Eager Page Split support not available");
         } else if (!(s->kvm_eager_split_size & sizes)) {
             error_report("Eager Page Split requested chunk size not valid");
-            ret = -EINVAL;
+            return -EINVAL;
         } else {
             ret = kvm_vm_enable_cap(s, KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE, 0,
                                     s->kvm_eager_split_size);
             if (ret < 0) {
                 error_report("Enabling of Eager Page Split failed: %s",
                              strerror(-ret));
+                return ret;
             }
         }
     }
@@ -626,7 +627,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     hw_breakpoints = g_array_sized_new(true, true,
                                        sizeof(HWBreakpoint), max_hw_bps);
 
-    return ret;
+    return 0;
 }
 
 unsigned long kvm_arch_vcpu_id(CPUState *cpu)
-- 
2.47.0

[PATCH v3 04/26] target/arm/kvm-rme: Initialize realm

[Jean-Philippe Brucker]

The machine code calls kvm_arm_rme_vm_type() to get the VM flag and KVM
calls kvm_arm_rme_init() to prepare for launching a Realm. Once VM
creation is complete, create the Realm:

* Create the realm descriptor,
* load images into Realm RAM (in another patch),
* finalize the REC (vCPU) after the registers are reset,
* activate the realm, at which point the realm is sealed.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3:
* Cleaner error handling
---
 target/arm/kvm_arm.h |  39 ++++++++++++++++
 target/arm/kvm-rme.c | 106 +++++++++++++++++++++++++++++++++++++++++++
 target/arm/kvm.c     |   9 +++-
 3 files changed, 152 insertions(+), 2 deletions(-)

diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 2e6b49bf13..9d6a89f9b1 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -153,6 +153,14 @@ void kvm_arm_set_cpu_features_from_host(ARMCPU *cpu);
  */
 void kvm_arm_add_vcpu_properties(ARMCPU *cpu);
 
+/**
+ * @cpu: The CPU object to finalize
+ * @feature: a KVM_ARM_VCPU_* feature
+ *
+ * Finalize the configuration of the given vcpu feature.
+ */
+int kvm_arm_vcpu_finalize(ARMCPU *cpu, int feature);
+
 /**
  * kvm_arm_steal_time_finalize:
  * @cpu: ARMCPU for which to finalize kvm-steal-time
@@ -221,6 +229,22 @@ int kvm_arm_set_irq(int cpu, int irqtype, int irq, int level);
 
 void kvm_arm_enable_mte(Object *cpuobj, Error **errp);
 
+/**
+ * kvm_arm_rme_init
+ * @ms: the machine state
+ *
+ * Prepare the machine to be a Realm, if the user enabled it.
+ */
+int kvm_arm_rme_init(MachineState *ms);
+
+/**
+ * kvm_arm_rme_vm_type
+ * @ms: the machine state
+ *
+ * Returns the Realm KVM VM type if the user requested a Realm, 0 otherwise.
+ */
+int kvm_arm_rme_vm_type(MachineState *ms);
+
 #else
 
 /*
@@ -260,6 +284,11 @@ static inline void kvm_arm_add_vcpu_properties(ARMCPU *cpu)
     g_assert_not_reached();
 }
 
+static inline int kvm_arm_vcpu_finalize(ARMCPU *cpu, int feature)
+{
+    g_assert_not_reached();
+}
+
 static inline int kvm_arm_get_max_vm_ipa_size(MachineState *ms, bool *fixed_ipa)
 {
     g_assert_not_reached();
@@ -300,6 +329,16 @@ static inline void kvm_arm_enable_mte(Object *cpuobj, Error **errp)
     g_assert_not_reached();
 }
 
+static inline int kvm_arm_rme_init(MachineState *ms)
+{
+    g_assert_not_reached();
+}
+
+static inline int kvm_arm_rme_vm_type(MachineState *ms)
+{
+    g_assert_not_reached();
+}
+
 #endif
 
 #endif
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
index 67909349c1..60d967a842 100644
--- a/target/arm/kvm-rme.c
+++ b/target/arm/kvm-rme.c
@@ -12,6 +12,7 @@
 #include "kvm_arm.h"
 #include "migration/blocker.h"
 #include "qapi/error.h"
+#include "qemu/error-report.h"
 #include "qom/object_interfaces.h"
 #include "sysemu/kvm.h"
 #include "sysemu/runstate.h"
@@ -27,14 +28,119 @@ OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
                                           CONFIDENTIAL_GUEST_SUPPORT,
                                           { TYPE_USER_CREATABLE }, { })
 
+static RmeGuest *rme_guest;
+
+static int rme_init_cpus(Error **errp)
+{
+    int ret;
+    CPUState *cs;
+
+    /*
+     * Now that do_cpu_reset() initialized the boot PC and
+     * kvm_cpu_synchronize_post_reset() registered it, we can finalize the REC.
+     */
+    CPU_FOREACH(cs) {
+        ret = kvm_arm_vcpu_finalize(ARM_CPU(cs), KVM_ARM_VCPU_REC);
+        if (ret) {
+            error_setg_errno(errp, -ret, "failed to finalize vCPU");
+            return ret;
+        }
+    }
+    return 0;
+}
+
+static int rme_create_realm(Error **errp)
+{
+    int ret;
+
+    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
+                            KVM_CAP_ARM_RME_CREATE_RD);
+    if (ret) {
+        error_setg_errno(errp, -ret, "failed to create Realm Descriptor");
+        return -1;
+    }
+
+    if (rme_init_cpus(errp)) {
+        return -1;
+    }
+
+    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
+                            KVM_CAP_ARM_RME_ACTIVATE_REALM);
+    if (ret) {
+        error_setg_errno(errp, -ret, "failed to activate realm");
+        return -1;
+    }
+
+    kvm_mark_guest_state_protected();
+    return 0;
+}
+
+static void rme_vm_state_change(void *opaque, bool running, RunState state)
+{
+    Error *err = NULL;
+
+    if (!running) {
+        return;
+    }
+
+    if (rme_create_realm(&err)) {
+        error_propagate_prepend(&error_fatal, err, "RME: ");
+    }
+}
+
 static void rme_guest_class_init(ObjectClass *oc, void *data)
 {
 }
 
 static void rme_guest_init(Object *obj)
 {
+    if (rme_guest) {
+        error_report("a single instance of RmeGuest is supported");
+        exit(1);
+    }
+    rme_guest = RME_GUEST(obj);
 }
 
 static void rme_guest_finalize(Object *obj)
 {
 }
+
+int kvm_arm_rme_init(MachineState *ms)
+{
+    static Error *rme_mig_blocker;
+    ConfidentialGuestSupport *cgs = ms->cgs;
+
+    if (!rme_guest) {
+        return 0;
+    }
+
+    if (!cgs) {
+        error_report("missing -machine confidential-guest-support parameter");
+        return -EINVAL;
+    }
+
+    if (!kvm_check_extension(kvm_state, KVM_CAP_ARM_RME)) {
+        return -ENODEV;
+    }
+
+    error_setg(&rme_mig_blocker, "RME: migration is not implemented");
+    migrate_add_blocker(&rme_mig_blocker, &error_fatal);
+
+    /*
+     * The realm activation is done last, when the VM starts, after all images
+     * have been loaded and all vcpus finalized.
+     */
+    qemu_add_vm_change_state_handler(rme_vm_state_change, NULL);
+
+    cgs->require_guest_memfd = true;
+    cgs->ready = true;
+    return 0;
+}
+
+int kvm_arm_rme_vm_type(MachineState *ms)
+{
+    if (rme_guest) {
+        return KVM_VM_TYPE_ARM_REALM;
+    }
+    return 0;
+}
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 95bcecf804..440c3ac8c6 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -95,7 +95,7 @@ static int kvm_arm_vcpu_init(ARMCPU *cpu)
  *
  * Returns: 0 if success else < 0 error code
  */
-static int kvm_arm_vcpu_finalize(ARMCPU *cpu, int feature)
+int kvm_arm_vcpu_finalize(ARMCPU *cpu, int feature)
 {
     return kvm_vcpu_ioctl(CPU(cpu), KVM_ARM_VCPU_FINALIZE, &feature);
 }
@@ -627,7 +627,12 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     hw_breakpoints = g_array_sized_new(true, true,
                                        sizeof(HWBreakpoint), max_hw_bps);
 
-    return 0;
+    ret = kvm_arm_rme_init(ms);
+    if (ret) {
+        error_report("Failed to enable RME: %s", strerror(-ret));
+    }
+
+    return ret;
 }
 
 unsigned long kvm_arch_vcpu_id(CPUState *cpu)
-- 
2.47.0

[PATCH v3 05/26] target/arm/kvm: Split kvm_arch_get/put_registers

[Jean-Philippe Brucker]

The confidential guest support in KVM limits the number of registers
that we can read and write. Split the get/put_registers function to
prepare for it.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/kvm.c | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 440c3ac8c6..0c80992f7c 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -2062,7 +2062,7 @@ static int kvm_arch_put_sve(CPUState *cs)
     return 0;
 }
 
-int kvm_arch_put_registers(CPUState *cs, int level, Error **errp)
+static int kvm_arm_put_core_regs(CPUState *cs, int level, Error **errp)
 {
     uint64_t val;
     uint32_t fpr;
@@ -2165,6 +2165,19 @@ int kvm_arch_put_registers(CPUState *cs, int level, Error **errp)
         return ret;
     }
 
+    return 0;
+}
+
+int kvm_arch_put_registers(CPUState *cs, int level, Error **errp)
+{
+    int ret;
+    ARMCPU *cpu = ARM_CPU(cs);
+
+    ret = kvm_arm_put_core_regs(cs, level, errp);
+    if (ret) {
+        return ret;
+    }
+
     write_cpustate_to_list(cpu, true);
 
     if (!write_list_to_kvmstate(cpu, level)) {
@@ -2246,7 +2259,7 @@ static int kvm_arch_get_sve(CPUState *cs)
     return 0;
 }
 
-int kvm_arch_get_registers(CPUState *cs, Error **errp)
+static int kvm_arm_get_core_regs(CPUState *cs, Error **errp)
 {
     uint64_t val;
     unsigned int el;
@@ -2349,6 +2362,19 @@ int kvm_arch_get_registers(CPUState *cs, Error **errp)
     }
     vfp_set_fpcr(env, fpr);
 
+    return 0;
+}
+
+int kvm_arch_get_registers(CPUState *cs, Error **errp)
+{
+    int ret;
+    ARMCPU *cpu = ARM_CPU(cs);
+
+    ret = kvm_arm_get_core_regs(cs, errp);
+    if (ret) {
+        return ret;
+    }
+
     ret = kvm_get_vcpu_events(cpu);
     if (ret) {
         return ret;
-- 
2.47.0

[PATCH v3 06/26] target/arm/kvm-rme: Initialize vCPU

[Jean-Philippe Brucker]

The target code calls kvm_arm_vcpu_init() to mark the vCPU as part of a
Realm. For a Realm vCPU, only x0-x7 can be set at runtime. Before boot,
the PC can also be set, and is ignored at runtime. KVM also accepts a
few system register changes during initial configuration, as returned by
KVM_GET_REG_LIST.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/cpu.h     |  3 +++
 target/arm/kvm_arm.h | 15 +++++++++++
 target/arm/kvm-rme.c | 10 ++++++++
 target/arm/kvm.c     | 61 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index d86e641280..f617591921 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -961,6 +961,9 @@ struct ArchCPU {
     OnOffAuto kvm_steal_time;
 #endif /* CONFIG_KVM */
 
+    /* Realm Management Extension */
+    bool kvm_rme;
+
     /* Uniprocessor system with MP extensions */
     bool mp_is_up;
 
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 9d6a89f9b1..8b52a881b0 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -245,6 +245,16 @@ int kvm_arm_rme_init(MachineState *ms);
  */
 int kvm_arm_rme_vm_type(MachineState *ms);
 
+/**
+ * kvm_arm_rme_vcpu_init
+ * @cs: the CPU
+ *
+ * If the user requested a Realm, setup the given vCPU accordingly. Realm vCPUs
+ * behave a little differently, for example most of their register state is
+ * hidden from the host.
+ */
+int kvm_arm_rme_vcpu_init(CPUState *cs);
+
 #else
 
 /*
@@ -339,6 +349,11 @@ static inline int kvm_arm_rme_vm_type(MachineState *ms)
     g_assert_not_reached();
 }
 
+static inline int kvm_arm_rme_vcpu_init(CPUState *cs)
+{
+    g_assert_not_reached();
+}
+
 #endif
 
 #endif
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
index 60d967a842..e3cc37538a 100644
--- a/target/arm/kvm-rme.c
+++ b/target/arm/kvm-rme.c
@@ -137,6 +137,16 @@ int kvm_arm_rme_init(MachineState *ms)
     return 0;
 }
 
+int kvm_arm_rme_vcpu_init(CPUState *cs)
+{
+    ARMCPU *cpu = ARM_CPU(cs);
+
+    if (rme_guest) {
+        cpu->kvm_rme = true;
+    }
+    return 0;
+}
+
 int kvm_arm_rme_vm_type(MachineState *ms)
 {
     if (rme_guest) {
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 0c80992f7c..a0de2efc41 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -1926,6 +1926,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
         return ret;
     }
 
+    ret = kvm_arm_rme_vcpu_init(cs);
+    if (ret) {
+        return ret;
+    }
+
     if (cpu_isar_feature(aa64_sve, cpu)) {
         ret = kvm_arm_sve_set_vls(cpu);
         if (ret) {
@@ -2062,6 +2067,35 @@ static int kvm_arch_put_sve(CPUState *cs)
     return 0;
 }
 
+static int kvm_arm_rme_put_core_regs(CPUState *cs, Error **errp)
+{
+    int i, ret;
+    struct kvm_one_reg reg;
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+
+    /*
+     * The RME ABI only allows us to set 8 GPRs and the PC
+     */
+    for (i = 0; i < 8; i++) {
+        reg.id = AARCH64_CORE_REG(regs.regs[i]);
+        reg.addr = (uintptr_t) &env->xregs[i];
+        ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    reg.id = AARCH64_CORE_REG(regs.pc);
+    reg.addr = (uintptr_t) &env->pc;
+    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
+    if (ret) {
+        return ret;
+    }
+
+    return 0;
+}
+
 static int kvm_arm_put_core_regs(CPUState *cs, int level, Error **errp)
 {
     uint64_t val;
@@ -2072,6 +2106,10 @@ static int kvm_arm_put_core_regs(CPUState *cs, int level, Error **errp)
     ARMCPU *cpu = ARM_CPU(cs);
     CPUARMState *env = &cpu->env;
 
+    if (cpu->kvm_rme) {
+        return kvm_arm_rme_put_core_regs(cs, errp);
+    }
+
     /* If we are in AArch32 mode then we need to copy the AArch32 regs to the
      * AArch64 registers before pushing them out to 64-bit KVM.
      */
@@ -2259,6 +2297,25 @@ static int kvm_arch_get_sve(CPUState *cs)
     return 0;
 }
 
+static int kvm_arm_rme_get_core_regs(CPUState *cs, Error **errp)
+{
+    int i, ret;
+    struct kvm_one_reg reg;
+    ARMCPU *cpu = ARM_CPU(cs);
+    CPUARMState *env = &cpu->env;
+
+    for (i = 0; i < 8; i++) {
+        reg.id = AARCH64_CORE_REG(regs.regs[i]);
+        reg.addr = (uintptr_t) &env->xregs[i];
+        ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &reg);
+        if (ret) {
+            return ret;
+        }
+    }
+
+    return 0;
+}
+
 static int kvm_arm_get_core_regs(CPUState *cs, Error **errp)
 {
     uint64_t val;
@@ -2269,6 +2326,10 @@ static int kvm_arm_get_core_regs(CPUState *cs, Error **errp)
     ARMCPU *cpu = ARM_CPU(cs);
     CPUARMState *env = &cpu->env;
 
+    if (cpu->kvm_rme) {
+        return kvm_arm_rme_get_core_regs(cs, errp);
+    }
+
     for (i = 0; i < 31; i++) {
         ret = kvm_get_one_reg(cs, AARCH64_CORE_REG(regs.regs[i]),
                               &env->xregs[i]);
-- 
2.47.0

[PATCH v3 07/26] target/arm/kvm: Create scratch VM as Realm if necessary

[Jean-Philippe Brucker]

Some ID registers have a different value for a Realm VM, for example
ID_AA64DFR0_EL1 contains the number of breakpoints/watchpoints
implemented by RMM instead of the hardware.

Even though RMM is in charge of setting up most Realm registers, KVM
still provides GET_ONE_REG interface on a Realm VM to probe the VM's
capabilities.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/kvm.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index a0de2efc41..870f51bf02 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -106,6 +106,7 @@ bool kvm_arm_create_scratch_host_vcpu(const uint32_t *cpus_to_try,
 {
     int ret = 0, kvmfd = -1, vmfd = -1, cpufd = -1;
     int max_vm_pa_size;
+    int vm_type;
 
     kvmfd = qemu_open_old("/dev/kvm", O_RDWR);
     if (kvmfd < 0) {
@@ -115,8 +116,9 @@ bool kvm_arm_create_scratch_host_vcpu(const uint32_t *cpus_to_try,
     if (max_vm_pa_size < 0) {
         max_vm_pa_size = 0;
     }
+    vm_type = kvm_arm_rme_vm_type(MACHINE(qdev_get_machine()));
     do {
-        vmfd = ioctl(kvmfd, KVM_CREATE_VM, max_vm_pa_size);
+        vmfd = ioctl(kvmfd, KVM_CREATE_VM, max_vm_pa_size | vm_type);
     } while (vmfd == -1 && errno == EINTR);
     if (vmfd < 0) {
         goto err;
-- 
2.47.0

[PATCH v3 08/26] hw/core/loader: Add ROM loader notifier

[Jean-Philippe Brucker]

Add a function to register a notifier, that is invoked after a ROM gets
loaded into guest memory.

It will be used by Arm confidential guest support, in order to register
all blobs loaded into memory with KVM, so that their content is moved
into Realm state and measured into the initial VM state.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 include/hw/loader.h | 15 +++++++++++++++
 hw/core/loader.c    | 15 +++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/include/hw/loader.h b/include/hw/loader.h
index 7f6d06b956..0cd9905f97 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -353,6 +353,21 @@ void *rom_ptr_for_as(AddressSpace *as, hwaddr addr, size_t size);
 ssize_t rom_add_vga(const char *file);
 ssize_t rom_add_option(const char *file, int32_t bootindex);
 
+typedef struct RomLoaderNotify {
+    /* Parameters passed to rom_add_blob() */
+    hwaddr addr;
+    size_t len;
+    size_t max_len;
+} RomLoaderNotify;
+
+/**
+ * rom_add_load_notifier - Add a notifier for loaded images
+ *
+ * Add a notifier that will be invoked with a RomLoaderNotify structure for each
+ * blob loaded into guest memory, after the blob is loaded.
+ */
+void rom_add_load_notifier(Notifier *notifier);
+
 /* This is the usual maximum in uboot, so if a uImage overflows this, it would
  * overflow on real hardware too. */
 #define UBOOT_MAX_GUNZIP_BYTES (64 << 20)
diff --git a/hw/core/loader.c b/hw/core/loader.c
index 31593a1171..759a62cf58 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -67,6 +67,8 @@
 #include <zlib.h>
 
 static int roms_loaded;
+static NotifierList rom_loader_notifier =
+    NOTIFIER_LIST_INITIALIZER(rom_loader_notifier);
 
 /* return the size or -1 if error */
 int64_t get_image_size(const char *filename)
@@ -1179,6 +1181,11 @@ MemoryRegion *rom_add_blob(const char *name, const void *blob, size_t len,
     return mr;
 }
 
+void rom_add_load_notifier(Notifier *notifier)
+{
+    notifier_list_add(&rom_loader_notifier, notifier);
+}
+
 /* This function is specific for elf program because we don't need to allocate
  * all the rom. We just allocate the first part and the rest is just zeros. This
  * is why romsize and datasize are different. Also, this function takes its own
@@ -1220,6 +1227,7 @@ ssize_t rom_add_option(const char *file, int32_t bootindex)
 static void rom_reset(void *unused)
 {
     Rom *rom;
+    RomLoaderNotify notify;
 
     QTAILQ_FOREACH(rom, &roms, next) {
         if (rom->fw_file) {
@@ -1268,6 +1276,13 @@ static void rom_reset(void *unused)
         cpu_flush_icache_range(rom->addr, rom->datasize);
 
         trace_loader_write_rom(rom->name, rom->addr, rom->datasize, rom->isrom);
+
+        notify = (RomLoaderNotify) {
+            .addr = rom->addr,
+            .len = rom->datasize,
+            .max_len = rom->romsize,
+        };
+        notifier_list_notify(&rom_loader_notifier, &notify);
     }
 }
 
-- 
2.47.0

[PATCH v3 09/26] target/arm/kvm-rme: Initialize Realm memory

[Jean-Philippe Brucker]

Initialize the IPA state of RAM. Collect the images copied into guest
RAM into a sorted list, and issue POPULATE_REALM KVM ioctls once we've
created the Realm Descriptor. The images are part of the Realm Initial
Measurement.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: RIPAS is now initialized separately
---
 target/arm/kvm_arm.h |  14 +++++
 target/arm/kvm-rme.c | 128 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 142 insertions(+)

diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 8b52a881b0..67db09a424 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -255,6 +255,16 @@ int kvm_arm_rme_vm_type(MachineState *ms);
  */
 int kvm_arm_rme_vcpu_init(CPUState *cs);
 
+/*
+ * kvm_arm_rme_init_guest_ram
+ * @base: base address of RAM
+ * @size: size of RAM
+ *
+ * If the user requested a Realm, set the base and size of guest RAM, in order
+ * to initialize the Realm IPA space.
+ */
+void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size);
+
 #else
 
 /*
@@ -281,6 +291,10 @@ static inline bool kvm_arm_mte_supported(void)
     return false;
 }
 
+static inline void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size)
+{
+}
+
 /*
  * These functions should never actually be called without KVM support.
  */
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
index e3cc37538a..83a29421df 100644
--- a/target/arm/kvm-rme.c
+++ b/target/arm/kvm-rme.c
@@ -9,6 +9,7 @@
 #include "exec/confidential-guest-support.h"
 #include "hw/boards.h"
 #include "hw/core/cpu.h"
+#include "hw/loader.h"
 #include "kvm_arm.h"
 #include "migration/blocker.h"
 #include "qapi/error.h"
@@ -20,16 +21,85 @@
 #define TYPE_RME_GUEST "rme-guest"
 OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
 
+#define RME_PAGE_SIZE qemu_real_host_page_size()
+
 struct RmeGuest {
     ConfidentialGuestSupport parent_obj;
+    Notifier rom_load_notifier;
+    GSList *ram_regions;
+
+    hwaddr ram_base;
+    size_t ram_size;
 };
 
 OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
                                           CONFIDENTIAL_GUEST_SUPPORT,
                                           { TYPE_USER_CREATABLE }, { })
 
+typedef struct {
+    hwaddr base;
+    hwaddr size;
+} RmeRamRegion;
+
 static RmeGuest *rme_guest;
 
+static int rme_init_ram(hwaddr base, size_t size, Error **errp)
+{
+    int ret;
+    uint64_t start = QEMU_ALIGN_DOWN(base, RME_PAGE_SIZE);
+    uint64_t end = QEMU_ALIGN_UP(base + size, RME_PAGE_SIZE);
+    struct kvm_cap_arm_rme_init_ipa_args init_args = {
+        .init_ipa_base = start,
+        .init_ipa_size = end - start,
+    };
+
+    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
+                            KVM_CAP_ARM_RME_INIT_IPA_REALM,
+                            (intptr_t)&init_args);
+    if (ret) {
+        error_setg_errno(errp, -ret,
+                         "failed to init RAM [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
+                         start, end);
+    }
+
+    return ret;
+}
+
+static int rme_populate_range(hwaddr base, size_t size, bool measure,
+                              Error **errp)
+{
+    int ret;
+    uint64_t start = QEMU_ALIGN_DOWN(base, RME_PAGE_SIZE);
+    uint64_t end = QEMU_ALIGN_UP(base + size, RME_PAGE_SIZE);
+    struct kvm_cap_arm_rme_populate_realm_args populate_args = {
+        .populate_ipa_base = start,
+        .populate_ipa_size = end - start,
+        .flags = measure ? KVM_ARM_RME_POPULATE_FLAGS_MEASURE : 0,
+    };
+
+    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
+                            KVM_CAP_ARM_RME_POPULATE_REALM,
+                            (intptr_t)&populate_args);
+    if (ret) {
+        error_setg_errno(errp, -ret,
+                   "failed to populate realm [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
+                   start, end);
+    }
+    return ret;
+}
+
+static void rme_populate_ram_region(gpointer data, gpointer err)
+{
+    Error **errp = err;
+    const RmeRamRegion *region = data;
+
+    if (*errp) {
+        return;
+    }
+
+    rme_populate_range(region->base, region->size, /* measure */ true, errp);
+}
+
 static int rme_init_cpus(Error **errp)
 {
     int ret;
@@ -60,6 +130,16 @@ static int rme_create_realm(Error **errp)
         return -1;
     }
 
+    if (rme_init_ram(rme_guest->ram_base, rme_guest->ram_size, errp)) {
+        return -1;
+    }
+
+    g_slist_foreach(rme_guest->ram_regions, rme_populate_ram_region, errp);
+    g_slist_free_full(g_steal_pointer(&rme_guest->ram_regions), g_free);
+    if (*errp) {
+        return -1;
+    }
+
     if (rme_init_cpus(errp)) {
         return -1;
     }
@@ -105,6 +185,43 @@ static void rme_guest_finalize(Object *obj)
 {
 }
 
+static gint rme_compare_ram_regions(gconstpointer a, gconstpointer b)
+{
+        const RmeRamRegion *ra = a;
+        const RmeRamRegion *rb = b;
+
+        g_assert(ra->base != rb->base);
+        return ra->base < rb->base ? -1 : 1;
+}
+
+static void rme_rom_load_notify(Notifier *notifier, void *data)
+{
+    RmeRamRegion *region;
+    RomLoaderNotify *rom = data;
+
+    if (rom->addr == -1) {
+        /*
+         * These blobs (ACPI tables) are not loaded into guest RAM at reset.
+         * Instead the firmware will load them via fw_cfg and measure them
+         * itself.
+         */
+        return;
+    }
+
+    region = g_new0(RmeRamRegion, 1);
+    region->base = rom->addr;
+    region->size = rom->len;
+
+    /*
+     * The Realm Initial Measurement (RIM) depends on the order in which we
+     * initialize and populate the RAM regions. To help a verifier
+     * independently calculate the RIM, sort regions by GPA.
+     */
+    rme_guest->ram_regions = g_slist_insert_sorted(rme_guest->ram_regions,
+                                                   region,
+                                                   rme_compare_ram_regions);
+}
+
 int kvm_arm_rme_init(MachineState *ms)
 {
     static Error *rme_mig_blocker;
@@ -132,11 +249,22 @@ int kvm_arm_rme_init(MachineState *ms)
      */
     qemu_add_vm_change_state_handler(rme_vm_state_change, NULL);
 
+    rme_guest->rom_load_notifier.notify = rme_rom_load_notify;
+    rom_add_load_notifier(&rme_guest->rom_load_notifier);
+
     cgs->require_guest_memfd = true;
     cgs->ready = true;
     return 0;
 }
 
+void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size)
+{
+    if (rme_guest) {
+        rme_guest->ram_base = base;
+        rme_guest->ram_size = size;
+    }
+}
+
 int kvm_arm_rme_vcpu_init(CPUState *cs)
 {
     ARMCPU *cpu = ARM_CPU(cs);
-- 
2.47.0

[PATCH v3 10/26] target/arm/kvm-rme: Add Realm Personalization Value parameter

[Jean-Philippe Brucker]

The Realm Personalization Value (RPV) is provided by the user to
distinguish Realms that have the same initial measurement.

The user provides up to 64 hexadecimal bytes. They are stored into the
RPV in the same order, zero-padded on the right.

Cc: Eric Blake <eblake@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Eduardo Habkost <eduardo@habkost.net>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: Fix documentation
---
 qapi/qom.json        |  15 ++++++
 target/arm/kvm-rme.c | 111 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 126 insertions(+)

diff --git a/qapi/qom.json b/qapi/qom.json
index a8beeabf1f..f982850bca 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1068,6 +1068,19 @@
   'data': { '*cpu-affinity': ['uint16'],
             '*node-affinity': ['uint16'] } }
 
+##
+# @RmeGuestProperties:
+#
+# Properties for rme-guest objects.
+#
+# @personalization-value: Realm personalization value, as a 64-byte
+#     hex string. This optional parameter allows to uniquely identify
+#     the VM instance during attestation. (default: 0)
+#
+# Since: 9.3
+##
+{ 'struct': 'RmeGuestProperties',
+  'data': { '*personalization-value': 'str' } }
 
 ##
 # @ObjectType:
@@ -1121,6 +1134,7 @@
     { 'name': 'pr-manager-helper',
       'if': 'CONFIG_LINUX' },
     'qtest',
+    'rme-guest',
     'rng-builtin',
     'rng-egd',
     { 'name': 'rng-random',
@@ -1195,6 +1209,7 @@
       'pr-manager-helper':          { 'type': 'PrManagerHelperProperties',
                                       'if': 'CONFIG_LINUX' },
       'qtest':                      'QtestProperties',
+      'rme-guest':                  'RmeGuestProperties',
       'rng-builtin':                'RngProperties',
       'rng-egd':                    'RngEgdProperties',
       'rng-random':                 { 'type': 'RngRandomProperties',
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
index 83a29421df..0be55867ee 100644
--- a/target/arm/kvm-rme.c
+++ b/target/arm/kvm-rme.c
@@ -23,11 +23,15 @@ OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
 
 #define RME_PAGE_SIZE qemu_real_host_page_size()
 
+#define RME_MAX_CFG         1
+
 struct RmeGuest {
     ConfidentialGuestSupport parent_obj;
     Notifier rom_load_notifier;
     GSList *ram_regions;
 
+    uint8_t *personalization_value;
+
     hwaddr ram_base;
     size_t ram_size;
 };
@@ -43,6 +47,48 @@ typedef struct {
 
 static RmeGuest *rme_guest;
 
+static int rme_configure_one(RmeGuest *guest, uint32_t cfg, Error **errp)
+{
+    int ret;
+    const char *cfg_str;
+    struct kvm_cap_arm_rme_config_item args = {
+        .cfg = cfg,
+    };
+
+    switch (cfg) {
+    case KVM_CAP_ARM_RME_CFG_RPV:
+        if (!guest->personalization_value) {
+            return 0;
+        }
+        memcpy(args.rpv, guest->personalization_value, KVM_CAP_ARM_RME_RPV_SIZE);
+        cfg_str = "personalization value";
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
+                            KVM_CAP_ARM_RME_CONFIG_REALM, (intptr_t)&args);
+    if (ret) {
+        error_setg_errno(errp, -ret, "failed to configure %s", cfg_str);
+    }
+    return ret;
+}
+
+static int rme_configure(Error **errp)
+{
+    int ret;
+    int cfg;
+
+    for (cfg = 0; cfg < RME_MAX_CFG; cfg++) {
+        ret = rme_configure_one(rme_guest, cfg, errp);
+        if (ret) {
+            return ret;
+        }
+    }
+    return 0;
+}
+
 static int rme_init_ram(hwaddr base, size_t size, Error **errp)
 {
     int ret;
@@ -123,6 +169,10 @@ static int rme_create_realm(Error **errp)
 {
     int ret;
 
+    if (rme_configure(errp)) {
+        return -1;
+    }
+
     ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
                             KVM_CAP_ARM_RME_CREATE_RD);
     if (ret) {
@@ -168,8 +218,69 @@ static void rme_vm_state_change(void *opaque, bool running, RunState state)
     }
 }
 
+static char *rme_get_rpv(Object *obj, Error **errp)
+{
+    RmeGuest *guest = RME_GUEST(obj);
+    GString *s;
+    int i;
+
+    if (!guest->personalization_value) {
+        return NULL;
+    }
+
+    s = g_string_sized_new(KVM_CAP_ARM_RME_RPV_SIZE * 2 + 1);
+
+    for (i = 0; i < KVM_CAP_ARM_RME_RPV_SIZE; i++) {
+        g_string_append_printf(s, "%02x", guest->personalization_value[i]);
+    }
+
+    return g_string_free(s, /* free_segment */ false);
+}
+
+static void rme_set_rpv(Object *obj, const char *value, Error **errp)
+{
+    RmeGuest *guest = RME_GUEST(obj);
+    size_t len = strlen(value);
+    uint8_t *out;
+    int i = 1;
+    int ret;
+
+    g_free(guest->personalization_value);
+    guest->personalization_value = out = g_malloc0(KVM_CAP_ARM_RME_RPV_SIZE);
+
+    /* Two chars per byte */
+    if (len > KVM_CAP_ARM_RME_RPV_SIZE * 2) {
+        error_setg(errp, "Realm Personalization Value is too large");
+        return;
+    }
+
+    /* First byte may have a single char */
+    if (len % 2) {
+        ret = sscanf(value, "%1hhx", out++);
+    } else {
+        ret = sscanf(value, "%2hhx", out++);
+        i++;
+    }
+    if (ret != 1) {
+        error_setg(errp, "Invalid Realm Personalization Value");
+        return;
+    }
+
+    for (; i < len; i += 2) {
+        ret = sscanf(value + i, "%2hhx", out++);
+        if (ret != 1) {
+            error_setg(errp, "Invalid Realm Personalization Value");
+            return;
+        }
+    }
+}
+
 static void rme_guest_class_init(ObjectClass *oc, void *data)
 {
+    object_class_property_add_str(oc, "personalization-value", rme_get_rpv,
+                                  rme_set_rpv);
+    object_class_property_set_description(oc, "personalization-value",
+            "Realm personalization value (512-bit hexadecimal number)");
 }
 
 static void rme_guest_init(Object *obj)
-- 
2.47.0

[PATCH v3 11/26] target/arm/kvm-rme: Add measurement algorithm property

[Jean-Philippe Brucker]

This option selects which measurement algorithm to use for attestation.
Supported values are SHA256 and SHA512. Default to SHA512 arbitrarily.

SHA512 is generally faster on 64-bit architectures. On a few arm64 CPUs
I tested SHA256 is much faster, but that's most likely because they only
support acceleration via FEAT_SHA256 (Armv8.0) and not FEAT_SHA512
(Armv8.2). Future CPUs supporting RME are likely to also support
FEAT_SHA512.

Cc: Eric Blake <eblake@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Eduardo Habkost <eduardo@habkost.net>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: Rename measurement-algorithm
---
 qapi/qom.json        | 20 +++++++++++++++++++-
 target/arm/kvm-rme.c | 39 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index f982850bca..901ba67634 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1068,6 +1068,20 @@
   'data': { '*cpu-affinity': ['uint16'],
             '*node-affinity': ['uint16'] } }
 
+##
+# @RmeGuestMeasurementAlgorithm:
+#
+# @sha256: Use the SHA256 algorithm
+#
+# @sha512: Use the SHA512 algorithm
+#
+# Algorithm to use for realm measurements
+#
+# Since: 9.3
+##
+{ 'enum': 'RmeGuestMeasurementAlgorithm',
+  'data': ['sha256', 'sha512'] }
+
 ##
 # @RmeGuestProperties:
 #
@@ -1077,10 +1091,14 @@
 #     hex string. This optional parameter allows to uniquely identify
 #     the VM instance during attestation. (default: 0)
 #
+# @measurement-algorithm: Realm measurement algorithm
+#     (default: sha512)
+#
 # Since: 9.3
 ##
 { 'struct': 'RmeGuestProperties',
-  'data': { '*personalization-value': 'str' } }
+  'data': { '*personalization-value': 'str',
+            '*measurement-algorithm': 'RmeGuestMeasurementAlgorithm' } }
 
 ##
 # @ObjectType:
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
index 0be55867ee..bf0bcf9a38 100644
--- a/target/arm/kvm-rme.c
+++ b/target/arm/kvm-rme.c
@@ -23,7 +23,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
 
 #define RME_PAGE_SIZE qemu_real_host_page_size()
 
-#define RME_MAX_CFG         1
+#define RME_MAX_CFG         2
 
 struct RmeGuest {
     ConfidentialGuestSupport parent_obj;
@@ -31,6 +31,7 @@ struct RmeGuest {
     GSList *ram_regions;
 
     uint8_t *personalization_value;
+    RmeGuestMeasurementAlgorithm measurement_algo;
 
     hwaddr ram_base;
     size_t ram_size;
@@ -63,6 +64,19 @@ static int rme_configure_one(RmeGuest *guest, uint32_t cfg, Error **errp)
         memcpy(args.rpv, guest->personalization_value, KVM_CAP_ARM_RME_RPV_SIZE);
         cfg_str = "personalization value";
         break;
+    case KVM_CAP_ARM_RME_CFG_HASH_ALGO:
+        switch (guest->measurement_algo) {
+        case RME_GUEST_MEASUREMENT_ALGORITHM_SHA256:
+            args.hash_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
+            break;
+        case RME_GUEST_MEASUREMENT_ALGORITHM_SHA512:
+            args.hash_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA512;
+            break;
+        default:
+            g_assert_not_reached();
+        }
+        cfg_str = "hash algorithm";
+        break;
     default:
         g_assert_not_reached();
     }
@@ -275,12 +289,34 @@ static void rme_set_rpv(Object *obj, const char *value, Error **errp)
     }
 }
 
+static int rme_get_measurement_algo(Object *obj, Error **errp)
+{
+    RmeGuest *guest = RME_GUEST(obj);
+
+    return guest->measurement_algo;
+}
+
+static void rme_set_measurement_algo(Object *obj, int algo, Error **errp)
+{
+    RmeGuest *guest = RME_GUEST(obj);
+
+    guest->measurement_algo = algo;
+}
+
 static void rme_guest_class_init(ObjectClass *oc, void *data)
 {
     object_class_property_add_str(oc, "personalization-value", rme_get_rpv,
                                   rme_set_rpv);
     object_class_property_set_description(oc, "personalization-value",
             "Realm personalization value (512-bit hexadecimal number)");
+
+    object_class_property_add_enum(oc, "measurement-algorithm",
+                                   "RmeGuestMeasurementAlgorithm",
+                                   &RmeGuestMeasurementAlgorithm_lookup,
+                                   rme_get_measurement_algo,
+                                   rme_set_measurement_algo);
+    object_class_property_set_description(oc, "measurement-algorithm",
+            "Realm measurement algorithm ('sha256', 'sha512')");
 }
 
 static void rme_guest_init(Object *obj)
@@ -290,6 +326,7 @@ static void rme_guest_init(Object *obj)
         exit(1);
     }
     rme_guest = RME_GUEST(obj);
+    rme_guest->measurement_algo = RME_GUEST_MEASUREMENT_ALGORITHM_SHA512;
 }
 
 static void rme_guest_finalize(Object *obj)
-- 
2.47.0

[PATCH v3 12/26] target/arm/cpu: Set number of breakpoints and watchpoints in KVM

[Jean-Philippe Brucker]

Add "num-breakpoints" and "num-watchpoints" CPU parameters to configure
the debug features that KVM presents to the guest. The KVM vCPU
configuration is modified by calling SET_ONE_REG on the ID register.

This is needed for Realm VMs, whose parameters include breakpoints and
watchpoints, and influence the Realm Initial Measurement.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/cpu.h          |  4 ++
 target/arm/kvm_arm.h      |  2 +
 target/arm/arm-qmp-cmds.c |  1 +
 target/arm/cpu64.c        | 77 +++++++++++++++++++++++++++++++++++++++
 target/arm/kvm.c          | 56 +++++++++++++++++++++++++++-
 5 files changed, 139 insertions(+), 1 deletion(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index f617591921..5cef43a8d2 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -1109,6 +1109,10 @@ struct ArchCPU {
 
     /* Generic timer counter frequency, in Hz */
     uint64_t gt_cntfrq_hz;
+
+    /* Allows to override the default configuration */
+    uint8_t num_bps;
+    uint8_t num_wps;
 };
 
 typedef struct ARMCPUInfo {
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 67db09a424..28ebec8580 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -16,6 +16,8 @@
 #define KVM_ARM_VGIC_V2   (1 << 0)
 #define KVM_ARM_VGIC_V3   (1 << 1)
 
+#define KVM_REG_ARM_ID_AA64DFR0_EL1     ARM64_SYS_REG(3, 0, 0, 5, 0)
+
 /**
  * kvm_arm_register_device:
  * @mr: memory region for this device
diff --git a/target/arm/arm-qmp-cmds.c b/target/arm/arm-qmp-cmds.c
index 3cc8cc738b..0f574bb1dd 100644
--- a/target/arm/arm-qmp-cmds.c
+++ b/target/arm/arm-qmp-cmds.c
@@ -95,6 +95,7 @@ static const char *cpu_model_advertised_features[] = {
     "sve1408", "sve1536", "sve1664", "sve1792", "sve1920", "sve2048",
     "kvm-no-adjvtime", "kvm-steal-time",
     "pauth", "pauth-impdef", "pauth-qarma3",
+    "num-breakpoints", "num-watchpoints",
     NULL
 };
 
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 458d1cee01..1d4f4c134d 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -576,6 +576,82 @@ void aarch64_add_pauth_properties(Object *obj)
     }
 }
 
+#if defined(CONFIG_KVM)
+static void arm_cpu_get_num_wps(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    uint8_t val;
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    val = cpu->num_wps;
+    if (val == 0) {
+        val = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, WRPS) + 1;
+    }
+
+    visit_type_uint8(v, name, &val, errp);
+}
+
+static void arm_cpu_set_num_wps(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    uint8_t val;
+    ARMCPU *cpu = ARM_CPU(obj);
+    uint8_t max_wps = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, WRPS) + 1;
+
+    if (!visit_type_uint8(v, name, &val, errp)) {
+        return;
+    }
+
+    if (val < 2 || val > max_wps) {
+        error_setg(errp, "invalid number of watchpoints");
+        return;
+    }
+
+    cpu->num_wps = val;
+}
+
+static void arm_cpu_get_num_bps(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    uint8_t val;
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    val = cpu->num_bps;
+    if (val == 0) {
+        val = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, BRPS) + 1;
+    }
+
+    visit_type_uint8(v, name, &val, errp);
+}
+
+static void arm_cpu_set_num_bps(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    uint8_t val;
+    ARMCPU *cpu = ARM_CPU(obj);
+    uint8_t max_bps = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, BRPS) + 1;
+
+    if (!visit_type_uint8(v, name, &val, errp)) {
+        return;
+    }
+
+    if (val < 2 || val > max_bps) {
+        error_setg(errp, "invalid number of breakpoints");
+        return;
+    }
+
+    cpu->num_bps = val;
+}
+
+static void aarch64_add_kvm_writable_properties(Object *obj)
+{
+    object_property_add(obj, "num-breakpoints", "uint8", arm_cpu_get_num_bps,
+                        arm_cpu_set_num_bps, NULL, NULL);
+    object_property_add(obj, "num-watchpoints", "uint8", arm_cpu_get_num_wps,
+                        arm_cpu_set_num_wps, NULL, NULL);
+}
+#endif /* CONFIG_KVM */
+
 void arm_cpu_lpa2_finalize(ARMCPU *cpu, Error **errp)
 {
     uint64_t t;
@@ -726,6 +802,7 @@ static void aarch64_host_initfn(Object *obj)
     if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
         aarch64_add_sve_properties(obj);
         aarch64_add_pauth_properties(obj);
+        aarch64_add_kvm_writable_properties(obj);
     }
 #elif defined(CONFIG_HVF)
     ARMCPU *cpu = ARM_CPU(obj);
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 870f51bf02..f6d45476b4 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -336,7 +336,7 @@ static bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64smfr0,
                               ARM64_SYS_REG(3, 0, 0, 4, 5));
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64dfr0,
-                              ARM64_SYS_REG(3, 0, 0, 5, 0));
+                              KVM_REG_ARM_ID_AA64DFR0_EL1);
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64dfr1,
                               ARM64_SYS_REG(3, 0, 0, 5, 1));
         err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64isar0,
@@ -887,6 +887,54 @@ out:
     return ret;
 }
 
+static void kvm_arm_configure_aa64dfr0(ARMCPU *cpu)
+{
+    int ret;
+    uint64_t val, newval;
+    CPUState *cs = CPU(cpu);
+
+    if (!cpu->num_bps && !cpu->num_wps) {
+        return;
+    }
+
+    newval = cpu->isar.id_aa64dfr0;
+    if (cpu->num_bps) {
+        uint64_t ctx_cmps = FIELD_EX64(newval, ID_AA64DFR0, CTX_CMPS);
+
+        /* CTX_CMPs is never greater than BRPs */
+        ctx_cmps = MIN(ctx_cmps, cpu->num_bps - 1);
+        newval = FIELD_DP64(newval, ID_AA64DFR0, BRPS, cpu->num_bps - 1);
+        newval = FIELD_DP64(newval, ID_AA64DFR0, CTX_CMPS, ctx_cmps);
+    }
+    if (cpu->num_wps) {
+        newval = FIELD_DP64(newval, ID_AA64DFR0, WRPS, cpu->num_wps - 1);
+    }
+    ret = kvm_set_one_reg(cs, KVM_REG_ARM_ID_AA64DFR0_EL1, &newval);
+    if (ret) {
+        error_report("Failed to set KVM_REG_ARM_ID_AA64DFR0_EL1");
+        return;
+    }
+
+    /*
+     * Check if the write succeeded. KVM does offer the writable mask for this
+     * register, but this way we also check if the value we wrote was sane.
+     */
+    ret = kvm_get_one_reg(cs, KVM_REG_ARM_ID_AA64DFR0_EL1, &val);
+    if (ret) {
+        error_report("Failed to get KVM_REG_ARM_ID_AA64DFR0_EL1");
+        return;
+    }
+
+    if (val != newval) {
+        error_report("Failed to update KVM_REG_ARM_ID_AA64DFR0_EL1");
+    }
+}
+
+static void kvm_arm_configure_vcpu_regs(ARMCPU *cpu)
+{
+    kvm_arm_configure_aa64dfr0(cpu);
+}
+
 /**
  * kvm_arm_cpreg_level:
  * @regidx: KVM register index
@@ -1006,6 +1054,12 @@ void kvm_arm_reset_vcpu(ARMCPU *cpu)
         fprintf(stderr, "kvm_arm_vcpu_init failed: %s\n", strerror(-ret));
         abort();
     }
+
+    /*
+     * Before loading the KVM values into CPUState, update the KVM configuration
+     */
+    kvm_arm_configure_vcpu_regs(cpu);
+
     if (!write_kvmstate_to_list(cpu)) {
         fprintf(stderr, "write_kvmstate_to_list failed\n");
         abort();
-- 
2.47.0

[PATCH v3 13/26] target/arm/cpu: Set number of PMU counters in KVM

[Jean-Philippe Brucker]

Add a "num-pmu-counters" CPU parameter to configure the number of
counters that KVM presents to the guest. This is needed for Realm VMs,
whose parameters include the number of PMU counters and influence the
Realm Initial Measurement.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/cpu.h          |  3 +++
 target/arm/kvm_arm.h      |  1 +
 target/arm/arm-qmp-cmds.c |  2 +-
 target/arm/cpu64.c        | 41 +++++++++++++++++++++++++++++++++++++++
 target/arm/kvm.c          | 34 +++++++++++++++++++++++++++++++-
 5 files changed, 79 insertions(+), 2 deletions(-)

diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 5cef43a8d2..382dd4ded9 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -1113,6 +1113,7 @@ struct ArchCPU {
     /* Allows to override the default configuration */
     uint8_t num_bps;
     uint8_t num_wps;
+    int8_t num_pmu_ctrs;
 };
 
 typedef struct ARMCPUInfo {
@@ -2393,6 +2394,8 @@ FIELD(MFAR, FPA, 12, 40)
 FIELD(MFAR, NSE, 62, 1)
 FIELD(MFAR, NS, 63, 1)
 
+FIELD(PMCR, N, 11, 5)
+
 QEMU_BUILD_BUG_ON(ARRAY_SIZE(((ARMCPU *)0)->ccsidr) <= R_V7M_CSSELR_INDEX_MASK);
 
 /* If adding a feature bit which corresponds to a Linux ELF
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 28ebec8580..77680f238a 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -17,6 +17,7 @@
 #define KVM_ARM_VGIC_V3   (1 << 1)
 
 #define KVM_REG_ARM_ID_AA64DFR0_EL1     ARM64_SYS_REG(3, 0, 0, 5, 0)
+#define KVM_REG_ARM_PMCR_EL0            ARM64_SYS_REG(3, 3, 9, 12, 0)
 
 /**
  * kvm_arm_register_device:
diff --git a/target/arm/arm-qmp-cmds.c b/target/arm/arm-qmp-cmds.c
index 0f574bb1dd..985d4270b8 100644
--- a/target/arm/arm-qmp-cmds.c
+++ b/target/arm/arm-qmp-cmds.c
@@ -95,7 +95,7 @@ static const char *cpu_model_advertised_features[] = {
     "sve1408", "sve1536", "sve1664", "sve1792", "sve1920", "sve2048",
     "kvm-no-adjvtime", "kvm-steal-time",
     "pauth", "pauth-impdef", "pauth-qarma3",
-    "num-breakpoints", "num-watchpoints",
+    "num-breakpoints", "num-watchpoints", "num-pmu-counters",
     NULL
 };
 
diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c
index 1d4f4c134d..cf1d17fb90 100644
--- a/target/arm/cpu64.c
+++ b/target/arm/cpu64.c
@@ -643,12 +643,53 @@ static void arm_cpu_set_num_bps(Object *obj, Visitor *v, const char *name,
     cpu->num_bps = val;
 }
 
+static void arm_cpu_get_num_pmu_ctrs(Object *obj, Visitor *v, const char *name,
+                                     void *opaque, Error **errp)
+{
+    uint8_t val;
+    ARMCPU *cpu = ARM_CPU(obj);
+
+    if (cpu->num_pmu_ctrs == -1) {
+        val = FIELD_EX64(cpu->isar.reset_pmcr_el0, PMCR, N);
+    } else {
+        val = cpu->num_pmu_ctrs;
+    }
+
+    visit_type_uint8(v, name, &val, errp);
+}
+
+static void arm_cpu_set_num_pmu_ctrs(Object *obj, Visitor *v, const char *name,
+                                     void *opaque, Error **errp)
+{
+    uint8_t val;
+    ARMCPU *cpu = ARM_CPU(obj);
+    uint8_t max_ctrs = FIELD_EX64(cpu->isar.reset_pmcr_el0, PMCR, N);
+
+    if (!visit_type_uint8(v, name, &val, errp)) {
+        return;
+    }
+
+    if (val > max_ctrs) {
+        error_setg(errp, "invalid number of PMU counters");
+        return;
+    }
+
+    cpu->num_pmu_ctrs = val;
+}
+
 static void aarch64_add_kvm_writable_properties(Object *obj)
 {
+    ARMCPU *cpu = ARM_CPU(obj);
+
     object_property_add(obj, "num-breakpoints", "uint8", arm_cpu_get_num_bps,
                         arm_cpu_set_num_bps, NULL, NULL);
     object_property_add(obj, "num-watchpoints", "uint8", arm_cpu_get_num_wps,
                         arm_cpu_set_num_wps, NULL, NULL);
+
+    cpu->num_pmu_ctrs = -1;
+    object_property_add(obj, "num-pmu-counters", "uint8",
+                        arm_cpu_get_num_pmu_ctrs, arm_cpu_set_num_pmu_ctrs,
+                        NULL, NULL);
 }
 #endif /* CONFIG_KVM */
 
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index f6d45476b4..9784c47ff5 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -438,7 +438,7 @@ static bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
         if (pmu_supported) {
             /* PMCR_EL0 is only accessible if the vCPU has feature PMU_V3 */
             err |= read_sys_reg64(fdarray[2], &ahcf->isar.reset_pmcr_el0,
-                                  ARM64_SYS_REG(3, 3, 9, 12, 0));
+                                  KVM_REG_ARM_PMCR_EL0);
         }
 
         if (sve_supported) {
@@ -930,9 +930,41 @@ static void kvm_arm_configure_aa64dfr0(ARMCPU *cpu)
     }
 }
 
+static void kvm_arm_configure_pmcr(ARMCPU *cpu)
+{
+    int ret;
+    uint64_t val, newval;
+    CPUState *cs = CPU(cpu);
+
+    if (cpu->num_pmu_ctrs == -1) {
+        return;
+    }
+
+    newval = FIELD_DP64(cpu->isar.reset_pmcr_el0, PMCR, N, cpu->num_pmu_ctrs);
+    ret = kvm_set_one_reg(cs, KVM_REG_ARM_PMCR_EL0, &newval);
+    if (ret) {
+        error_report("Failed to set KVM_REG_ARM_PMCR_EL0");
+        return;
+    }
+
+    /*
+     * Check if the write succeeded, since older versions of KVM ignore it.
+     */
+    ret = kvm_get_one_reg(cs, KVM_REG_ARM_PMCR_EL0, &val);
+    if (ret) {
+        error_report("Failed to get KVM_REG_ARM_PMCR_EL0");
+        return;
+    }
+
+    if (val != newval) {
+        error_report("Failed to update KVM_REG_ARM_PMCR_EL0");
+    }
+}
+
 static void kvm_arm_configure_vcpu_regs(ARMCPU *cpu)
 {
     kvm_arm_configure_aa64dfr0(cpu);
+    kvm_arm_configure_pmcr(cpu);
 }
 
 /**
-- 
2.47.0

[PATCH v3 14/26] target/arm/cpu: Inform about reading confidential CPU registers

[Jean-Philippe Brucker]

The host cannot access registers of a Realm. Instead of showing all
registers as zero in "info registers", display a message about this
restriction.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 target/arm/cpu.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 6938161b95..7f6569e87e 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1232,6 +1232,11 @@ static void aarch64_cpu_dump_state(CPUState *cs, FILE *f, int flags)
     const char *ns_status;
     bool sve;
 
+    if (cpu->kvm_rme) {
+        qemu_fprintf(f, "the CPU registers are confidential to the realm\n");
+        return;
+    }
+
     qemu_fprintf(f, " PC=%016" PRIx64 " ", env->pc);
     for (i = 0; i < 32; i++) {
         if (i == 31) {
-- 
2.47.0

[PATCH v3 15/26] hw/arm/virt: Add support for Arm RME

[Jean-Philippe Brucker]

When confidential-guest-support is enabled for the virt machine, add the
RME flag to the VM type.

The HVC conduit for PSCI is not supported for Realms.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 hw/arm/virt.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 1a381e9a2b..2d36640733 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -229,6 +229,11 @@ static const int a15irqmap[] = {
     [VIRT_PLATFORM_BUS] = 112, /* ...to 112 + PLATFORM_BUS_NUM_IRQS -1 */
 };
 
+static bool virt_machine_is_confidential(VirtMachineState *vms)
+{
+    return MACHINE(vms)->cgs;
+}
+
 static void create_randomness(MachineState *ms, const char *node)
 {
     struct {
@@ -2165,10 +2170,11 @@ static void machvirt_init(MachineState *machine)
      * if the guest has EL2 then we will use SMC as the conduit,
      * and otherwise we will use HVC (for backwards compatibility and
      * because if we're using KVM then we must use HVC).
+     * Realm guests must also use SMC.
      */
     if (vms->secure && firmware_loaded) {
         vms->psci_conduit = QEMU_PSCI_CONDUIT_DISABLED;
-    } else if (vms->virt) {
+    } else if (vms->virt || virt_machine_is_confidential(vms)) {
         vms->psci_conduit = QEMU_PSCI_CONDUIT_SMC;
     } else {
         vms->psci_conduit = QEMU_PSCI_CONDUIT_HVC;
@@ -3013,6 +3019,7 @@ static HotplugHandler *virt_machine_get_hotplug_handler(MachineState *machine,
 static int virt_kvm_type(MachineState *ms, const char *type_str)
 {
     VirtMachineState *vms = VIRT_MACHINE(ms);
+    int rme_vm_type = kvm_arm_rme_vm_type(ms);
     int max_vm_pa_size, requested_pa_size;
     bool fixed_ipa;
 
@@ -3042,7 +3049,11 @@ static int virt_kvm_type(MachineState *ms, const char *type_str)
      * the implicit legacy 40b IPA setting, in which case the kvm_type
      * must be 0.
      */
-    return fixed_ipa ? 0 : requested_pa_size;
+    if (fixed_ipa) {
+        return 0;
+    }
+
+    return requested_pa_size | rme_vm_type;
 }
 
 static int virt_hvf_get_physical_address_range(MachineState *ms)
-- 
2.47.0

[PATCH v3 16/26] hw/arm/virt: Disable DTB randomness for confidential VMs

[Jean-Philippe Brucker]

The dtb-randomness feature, which adds random seeds to the DTB, isn't
really compatible with confidential VMs since it randomizes the Realm
Initial Measurement. Enabling it is not an error, but it prevents
attestation. It also isn't useful to a Realm, which doesn't trust host
input.

Currently the feature is automatically enabled, unless the user disables
it on the command-line. Change it to OnOffAuto, and automatically
disable it for confidential VMs, unless the user explicitly enables it.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 docs/system/arm/virt.rst |  9 +++++----
 include/hw/arm/virt.h    |  2 +-
 hw/arm/virt.c            | 41 +++++++++++++++++++++++++---------------
 3 files changed, 32 insertions(+), 20 deletions(-)

diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index e67e7f0f7c..c8319555a5 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -176,10 +176,11 @@ dtb-randomness
   rng-seed and kaslr-seed nodes (in both "/chosen" and
   "/secure-chosen") to use for features like the random number
   generator and address space randomisation. The default is
-  ``on``. You will want to disable it if your trusted boot chain
-  will verify the DTB it is passed, since this option causes the
-  DTB to be non-deterministic. It would be the responsibility of
-  the firmware to come up with a seed and pass it on if it wants to.
+  ``off`` for confidential VMs, and ``on`` otherwise. You will want
+  to disable it if your trusted boot chain will verify the DTB it is
+  passed, since this option causes the DTB to be non-deterministic.
+  It would be the responsibility of the firmware to come up with a
+  seed and pass it on if it wants to.
 
 dtb-kaslr-seed
   A deprecated synonym for dtb-randomness.
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index aca4f8061b..e5e9f67f52 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -154,7 +154,7 @@ struct VirtMachineState {
     bool virt;
     bool ras;
     bool mte;
-    bool dtb_randomness;
+    OnOffAuto dtb_randomness;
     bool second_ns_uart_present;
     OnOffAuto acpi;
     VirtGICType gic_version;
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 2d36640733..9836dfbdfb 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -264,6 +264,7 @@ static bool ns_el2_virt_timer_present(void)
 
 static void create_fdt(VirtMachineState *vms)
 {
+    bool dtb_randomness = true;
     MachineState *ms = MACHINE(vms);
     int nb_numa_nodes = ms->numa_state->num_nodes;
     void *fdt = create_device_tree(&vms->fdt_size);
@@ -273,6 +274,16 @@ static void create_fdt(VirtMachineState *vms)
         exit(1);
     }
 
+    /*
+     * Including random data in the DTB causes random intial measurement on CCA,
+     * so disable it for confidential VMs.
+     */
+    if (vms->dtb_randomness == ON_OFF_AUTO_OFF ||
+        (vms->dtb_randomness == ON_OFF_AUTO_AUTO &&
+         virt_machine_is_confidential(vms))) {
+        dtb_randomness = false;
+    }
+
     ms->fdt = fdt;
 
     /* Header */
@@ -294,13 +305,13 @@ static void create_fdt(VirtMachineState *vms)
 
     /* /chosen must exist for load_dtb to fill in necessary properties later */
     qemu_fdt_add_subnode(fdt, "/chosen");
-    if (vms->dtb_randomness) {
+    if (dtb_randomness) {
         create_randomness(ms, "/chosen");
     }
 
     if (vms->secure) {
         qemu_fdt_add_subnode(fdt, "/secure-chosen");
-        if (vms->dtb_randomness) {
+        if (dtb_randomness) {
             create_randomness(ms, "/secure-chosen");
         }
     }
@@ -2570,18 +2581,21 @@ static void virt_set_its(Object *obj, bool value, Error **errp)
     vms->its = value;
 }
 
-static bool virt_get_dtb_randomness(Object *obj, Error **errp)
+static void virt_get_dtb_randomness(Object *obj, Visitor *v, const char *name,
+                                    void *opaque, Error **errp)
 {
     VirtMachineState *vms = VIRT_MACHINE(obj);
+    OnOffAuto dtb_randomness = vms->dtb_randomness;
 
-    return vms->dtb_randomness;
+    visit_type_OnOffAuto(v, name, &dtb_randomness, errp);
 }
 
-static void virt_set_dtb_randomness(Object *obj, bool value, Error **errp)
+static void virt_set_dtb_randomness(Object *obj, Visitor *v, const char *name,
+                                    void *opaque, Error **errp)
 {
     VirtMachineState *vms = VIRT_MACHINE(obj);
 
-    vms->dtb_randomness = value;
+    visit_type_OnOffAuto(v, name, &vms->dtb_randomness, errp);
 }
 
 static char *virt_get_oem_id(Object *obj, Error **errp)
@@ -3253,16 +3267,16 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
                                           "Set on/off to enable/disable "
                                           "ITS instantiation");
 
-    object_class_property_add_bool(oc, "dtb-randomness",
-                                   virt_get_dtb_randomness,
-                                   virt_set_dtb_randomness);
+    object_class_property_add(oc, "dtb-randomness", "OnOffAuto",
+                              virt_get_dtb_randomness, virt_set_dtb_randomness,
+                              NULL, NULL);
     object_class_property_set_description(oc, "dtb-randomness",
                                           "Set off to disable passing random or "
                                           "non-deterministic dtb nodes to guest");
 
-    object_class_property_add_bool(oc, "dtb-kaslr-seed",
-                                   virt_get_dtb_randomness,
-                                   virt_set_dtb_randomness);
+    object_class_property_add(oc, "dtb-kaslr-seed", "OnOffAuto",
+                              virt_get_dtb_randomness, virt_set_dtb_randomness,
+                              NULL, NULL);
     object_class_property_set_description(oc, "dtb-kaslr-seed",
                                           "Deprecated synonym of dtb-randomness");
 
@@ -3333,9 +3347,6 @@ static void virt_instance_init(Object *obj)
     /* MTE is disabled by default.  */
     vms->mte = false;
 
-    /* Supply kaslr-seed and rng-seed by default */
-    vms->dtb_randomness = true;
-
     vms->irqmap = a15irqmap;
 
     virt_flash_create(vms);
-- 
2.47.0

[PATCH v3 17/26] hw/arm/virt: Reserve one bit of guest-physical address for RME

[Jean-Philippe Brucker]

When RME is enabled, the upper GPA bit is used to distinguish protected
from unprotected addresses. Reserve it when setting up the guest memory
map.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 hw/arm/virt.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 9836dfbdfb..eb94997914 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -3035,14 +3035,24 @@ static int virt_kvm_type(MachineState *ms, const char *type_str)
     VirtMachineState *vms = VIRT_MACHINE(ms);
     int rme_vm_type = kvm_arm_rme_vm_type(ms);
     int max_vm_pa_size, requested_pa_size;
+    int rme_reserve_bit = 0;
     bool fixed_ipa;
 
-    max_vm_pa_size = kvm_arm_get_max_vm_ipa_size(ms, &fixed_ipa);
+    if (rme_vm_type) {
+        /*
+         * With RME, the upper GPA bit differentiates Realm from NS memory.
+         * Reserve the upper bit to ensure that highmem devices will fit.
+         */
+        rme_reserve_bit = 1;
+    }
+
+    max_vm_pa_size = kvm_arm_get_max_vm_ipa_size(ms, &fixed_ipa) -
+                     rme_reserve_bit;
 
     /* we freeze the memory map to compute the highest gpa */
     virt_set_memmap(vms, max_vm_pa_size);
 
-    requested_pa_size = 64 - clz64(vms->highest_gpa);
+    requested_pa_size = 64 - clz64(vms->highest_gpa) + rme_reserve_bit;
 
     /*
      * KVM requires the IPA size to be at least 32 bits.
-- 
2.47.0

[PATCH v3 18/26] hw/arm/boot: Mark all guest memory as RIPAS_RAM.

[Jean-Philippe Brucker]

All Realm IPA states are by default RIPAS_EMPTY, and accessing them in
that state causes injection of synchronous exception. Either the loader
or the guest needs to set IPA state to RIPAS_RAM before accessing it.
Since a Linux guest needs all memory ready at boot [1], initialize it
here.

[1] https://docs.kernel.org/arch/arm64/booting.html
    https://lore.kernel.org/all/20241004144307.66199-12-steven.price@arm.com/

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: New: the Linux guest does not initialize RIPAS itself anymore,
and expects the loader to do it.
---
 hw/arm/boot.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 5301d8d318..a2414b1f98 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -26,6 +26,7 @@
 #include "qemu/config-file.h"
 #include "qemu/option.h"
 #include "qemu/units.h"
+#include "kvm_arm.h"
 
 /* Kernel boot protocol is specified in the kernel docs
  * Documentation/arm/Booting and Documentation/arm64/booting.txt
@@ -1238,6 +1239,9 @@ void arm_load_kernel(ARMCPU *cpu, MachineState *ms, struct arm_boot_info *info)
     info->dtb_filename = ms->dtb;
     info->dtb_limit = 0;
 
+    /* Mark all Realm memory as RAM */
+    kvm_arm_rme_init_guest_ram(info->loader_start, info->ram_size);
+
     /* Load the kernel.  */
     if (!info->kernel_filename || info->firmware_loaded) {
         arm_setup_firmware_boot(cpu, info);
-- 
2.47.0

[PATCH v3 19/26] hw/arm/virt: Move virt_flash_create() to machvirt_init()

[Jean-Philippe Brucker]

For confidential VMs we'll want to skip flash device creation.
Unfortunately, in virt_instance_init() the machine->cgs member has not
yet been initialized, so we cannot check whether confidential guest is
enabled. Move virt_flash_create() to machvirt_init(), where we can
access the machine->cgs member.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 hw/arm/virt.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index eb94997914..c4cf69ea33 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2124,6 +2124,8 @@ static void machvirt_init(MachineState *machine)
     unsigned int smp_cpus = machine->smp.cpus;
     unsigned int max_cpus = machine->smp.max_cpus;
 
+    virt_flash_create(vms);
+
     possible_cpus = mc->possible_cpu_arch_ids(machine);
 
     /*
@@ -3359,8 +3361,6 @@ static void virt_instance_init(Object *obj)
 
     vms->irqmap = a15irqmap;
 
-    virt_flash_create(vms);
-
     vms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6);
     vms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
 }
-- 
2.47.0

[PATCH v3 20/26] hw/arm/virt: Use RAM instead of flash for confidential guest firmware

[Jean-Philippe Brucker]

The flash device that holds firmware code relies on read-only stage-2
mappings. Read accesses behave as RAM and write accesses as MMIO. Since
the RMM does not support read-only mappings we cannot use the flash
device as-is.

That isn't a problem because the firmware does not want to disclose any
information to the host, hence will not store its variables in clear
persistent memory. We can therefore replace the flash device with RAM,
and load the firmware there.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 include/hw/arm/boot.h |  9 +++++++++
 hw/arm/boot.c         | 32 ++++++++++++++++++++++++++++--
 hw/arm/virt.c         | 45 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
index 80c492d742..d91cfc6942 100644
--- a/include/hw/arm/boot.h
+++ b/include/hw/arm/boot.h
@@ -112,6 +112,10 @@ struct arm_boot_info {
      */
     bool firmware_loaded;
 
+    /* Used when loading firmware into RAM */
+    hwaddr firmware_base;
+    hwaddr firmware_max_size;
+
     /* Address at which board specific loader/setup code exists. If enabled,
      * this code-blob will run before anything else. It must return to the
      * caller via the link register. There is no stack set up. Enabled by
@@ -132,6 +136,11 @@ struct arm_boot_info {
     bool secure_board_setup;
 
     arm_endianness endianness;
+
+    /*
+     * Confidential guest boot loads everything into RAM so it can be measured.
+     */
+    bool confidential;
 };
 
 /**
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index a2414b1f98..4cf7dd5b4d 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -1150,7 +1150,31 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
     }
 }
 
-static void arm_setup_firmware_boot(ARMCPU *cpu, struct arm_boot_info *info)
+static void arm_setup_confidential_firmware_boot(ARMCPU *cpu,
+                                                 struct arm_boot_info *info,
+                                                 const char *firmware_filename)
+{
+    ssize_t fw_size;
+    const char *fname;
+    AddressSpace *as = arm_boot_address_space(cpu, info);
+
+    fname = qemu_find_file(QEMU_FILE_TYPE_BIOS, firmware_filename);
+    if (!fname) {
+        error_report("Could not find firmware image '%s'", firmware_filename);
+        exit(1);
+    }
+
+    fw_size = load_image_targphys_as(firmware_filename,
+                                     info->firmware_base,
+                                     info->firmware_max_size, as);
+    if (fw_size <= 0) {
+        error_report("could not load firmware '%s'", firmware_filename);
+        exit(1);
+    }
+}
+
+static void arm_setup_firmware_boot(ARMCPU *cpu, struct arm_boot_info *info,
+                                    const char *firmware_filename)
 {
     /* Set up for booting firmware (which might load a kernel via fw_cfg) */
 
@@ -1201,6 +1225,10 @@ static void arm_setup_firmware_boot(ARMCPU *cpu, struct arm_boot_info *info)
         }
     }
 
+    if (info->confidential) {
+        arm_setup_confidential_firmware_boot(cpu, info, firmware_filename);
+    }
+
     /*
      * We will start from address 0 (typically a boot ROM image) in the
      * same way as hardware. Leave env->boot_info NULL, so that
@@ -1244,7 +1272,7 @@ void arm_load_kernel(ARMCPU *cpu, MachineState *ms, struct arm_boot_info *info)
 
     /* Load the kernel.  */
     if (!info->kernel_filename || info->firmware_loaded) {
-        arm_setup_firmware_boot(cpu, info);
+        arm_setup_firmware_boot(cpu, info, ms->firmware);
     } else {
         arm_setup_direct_kernel_boot(cpu, info);
     }
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index c4cf69ea33..0750c83fae 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1228,6 +1228,10 @@ static PFlashCFI01 *virt_flash_create1(VirtMachineState *vms,
 
 static void virt_flash_create(VirtMachineState *vms)
 {
+    if (virt_machine_is_confidential(vms)) {
+        return;
+    }
+
     vms->flash[0] = virt_flash_create1(vms, "virt.flash0", "pflash0");
     vms->flash[1] = virt_flash_create1(vms, "virt.flash1", "pflash1");
 }
@@ -1263,6 +1267,10 @@ static void virt_flash_map(VirtMachineState *vms,
     hwaddr flashsize = vms->memmap[VIRT_FLASH].size / 2;
     hwaddr flashbase = vms->memmap[VIRT_FLASH].base;
 
+    if (virt_machine_is_confidential(vms)) {
+        return;
+    }
+
     virt_flash_map1(vms->flash[0], flashbase, flashsize,
                     secure_sysmem);
     virt_flash_map1(vms->flash[1], flashbase + flashsize, flashsize,
@@ -1278,6 +1286,10 @@ static void virt_flash_fdt(VirtMachineState *vms,
     MachineState *ms = MACHINE(vms);
     char *nodename;
 
+    if (virt_machine_is_confidential(vms)) {
+        return;
+    }
+
     if (sysmem == secure_sysmem) {
         /* Report both flash devices as a single node in the DT */
         nodename = g_strdup_printf("/flash@%" PRIx64, flashbase);
@@ -1313,6 +1325,27 @@ static void virt_flash_fdt(VirtMachineState *vms,
     }
 }
 
+static bool virt_confidential_firmware_init(VirtMachineState *vms,
+                                            MemoryRegion *sysmem)
+{
+    MemoryRegion *fw_ram;
+    hwaddr fw_base = vms->memmap[VIRT_FLASH].base;
+    hwaddr fw_size = vms->memmap[VIRT_FLASH].size;
+
+    if (!MACHINE(vms)->firmware) {
+        return false;
+    }
+
+    assert(machine_require_guest_memfd(MACHINE(vms)));
+
+    fw_ram = g_new(MemoryRegion, 1);
+    memory_region_init_ram_guest_memfd(fw_ram, NULL, "fw_ram", fw_size,
+                                       &error_fatal);
+    memory_region_add_subregion(sysmem, fw_base, fw_ram);
+
+    return true;
+}
+
 static bool virt_firmware_init(VirtMachineState *vms,
                                MemoryRegion *sysmem,
                                MemoryRegion *secure_sysmem)
@@ -1321,6 +1354,15 @@ static bool virt_firmware_init(VirtMachineState *vms,
     const char *bios_name;
     BlockBackend *pflash_blk0;
 
+    /*
+     * For a confidential VM, the firmware image and any boot information,
+     * including EFI variables, are stored in RAM in order to be measurable and
+     * private. Create a RAM region and load the firmware image there.
+     */
+    if (virt_machine_is_confidential(vms)) {
+        return virt_confidential_firmware_init(vms, sysmem);
+    }
+
     /* Map legacy -drive if=pflash to machine properties */
     for (i = 0; i < ARRAY_SIZE(vms->flash); i++) {
         pflash_cfi01_legacy_drive(vms->flash[i],
@@ -2463,7 +2505,10 @@ static void machvirt_init(MachineState *machine)
     vms->bootinfo.get_dtb = machvirt_dtb;
     vms->bootinfo.skip_dtb_autoload = true;
     vms->bootinfo.firmware_loaded = firmware_loaded;
+    vms->bootinfo.firmware_base = vms->memmap[VIRT_FLASH].base;
+    vms->bootinfo.firmware_max_size = vms->memmap[VIRT_FLASH].size;
     vms->bootinfo.psci_conduit = vms->psci_conduit;
+    vms->bootinfo.confidential = virt_machine_is_confidential(vms);
     arm_load_kernel(ARM_CPU(first_cpu), machine, &vms->bootinfo);
 
     vms->machine_done.notify = virt_machine_done;
-- 
2.47.0

[RFC PATCH v3 21/26] hw/arm/boot: Load DTB as is for confidential VMs

[Jean-Philippe Brucker]

For confidential VMs it may be necessary to measure the DTB, to ensure a
malicious host does not insert harmful information in there. In case an
external tool can generated and measured the DTB, load it as is without
patching it.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: new
---
 hw/arm/boot.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 4cf7dd5b4d..20b3071339 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -523,7 +523,14 @@ int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo,
     char **node_path;
     Error *err = NULL;
 
-    if (binfo->dtb_filename) {
+    if (binfo->dtb_filename && binfo->confidential) {
+        /*
+         * If the user is providing a DTB for a confidential VM, it is already
+         * tailored to this configuration and measured. Load it as is, without
+         * any modification.
+         */
+        return rom_add_file_fixed_as(binfo->dtb_filename, addr, -1, as);
+    } else if (binfo->dtb_filename) {
         char *filename;
         filename = qemu_find_file(QEMU_FILE_TYPE_BIOS, binfo->dtb_filename);
         if (!filename) {
-- 
2.47.0

[RFC PATCH v3 22/26] hw/arm/boot: Skip bootloader for confidential guests

[Jean-Philippe Brucker]

An independent verifier needs to reconstruct the content of guest memory
in order to attest that it is running trusted code. To avoid having to
reconstruct the bootloader generated by QEMU, skip this step and jump
directly to the kernel, with the DTB address in x0 as specified by the
Linux boot protocol [1].

[1] https://docs.kernel.org/arch/arm64/booting.html

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: new
---
 include/hw/arm/boot.h |  6 ++++++
 hw/arm/boot.c         | 23 +++++++++++++++++------
 hw/arm/virt.c         |  1 +
 3 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
index d91cfc6942..5fcbaa2625 100644
--- a/include/hw/arm/boot.h
+++ b/include/hw/arm/boot.h
@@ -137,6 +137,12 @@ struct arm_boot_info {
 
     arm_endianness endianness;
 
+    /*
+     * Instead of starting in a small bootloader that jumps to the kernel,
+     * immediately start in the kernel.
+     */
+    bool skip_bootloader;
+
     /*
      * Confidential guest boot loads everything into RAM so it can be measured.
      */
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index 20b3071339..e461137595 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -762,7 +762,13 @@ static void do_cpu_reset(void *opaque)
             if (cs == first_cpu) {
                 AddressSpace *as = arm_boot_address_space(cpu, info);
 
-                cpu_set_pc(cs, info->loader_start);
+                if (info->skip_bootloader)  {
+                    assert(is_a64(env));
+                    env->xregs[0] = info->dtb_start;
+                    cpu_set_pc(cs, info->entry);
+                } else {
+                    cpu_set_pc(cs, info->loader_start);
+                }
 
                 if (!have_dtb(info)) {
                     if (old_param) {
@@ -860,7 +866,8 @@ static ssize_t arm_load_elf(struct arm_boot_info *info, uint64_t *pentry,
 }
 
 static uint64_t load_aarch64_image(const char *filename, hwaddr mem_base,
-                                   hwaddr *entry, AddressSpace *as)
+                                   hwaddr *entry, AddressSpace *as,
+                                   bool skip_bootloader)
 {
     hwaddr kernel_load_offset = KERNEL64_LOAD_ADDR;
     uint64_t kernel_size = 0;
@@ -912,7 +919,8 @@ static uint64_t load_aarch64_image(const char *filename, hwaddr mem_base,
              * bootloader, we can just load it starting at 2MB+offset rather
              * than 0MB + offset.
              */
-            if (kernel_load_offset < BOOTLOADER_MAX_SIZE) {
+            if (kernel_load_offset < BOOTLOADER_MAX_SIZE &&
+                !skip_bootloader) {
                 kernel_load_offset += 2 * MiB;
             }
         }
@@ -996,7 +1004,8 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
     }
     if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64) && kernel_size < 0) {
         kernel_size = load_aarch64_image(info->kernel_filename,
-                                         info->loader_start, &entry, as);
+                                         info->loader_start, &entry, as,
+                                         info->skip_bootloader);
         is_linux = 1;
         if (kernel_size >= 0) {
             image_low_addr = entry;
@@ -1136,8 +1145,10 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
         fixupcontext[FIXUP_ENTRYPOINT_LO] = entry;
         fixupcontext[FIXUP_ENTRYPOINT_HI] = entry >> 32;
 
-        arm_write_bootloader("bootloader", as, info->loader_start,
-                             primary_loader, fixupcontext);
+        if (!info->skip_bootloader) {
+            arm_write_bootloader("bootloader", as, info->loader_start,
+                                 primary_loader, fixupcontext);
+        }
 
         if (info->write_board_setup) {
             info->write_board_setup(cpu, info);
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 0750c83fae..5247f53882 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2509,6 +2509,7 @@ static void machvirt_init(MachineState *machine)
     vms->bootinfo.firmware_max_size = vms->memmap[VIRT_FLASH].size;
     vms->bootinfo.psci_conduit = vms->psci_conduit;
     vms->bootinfo.confidential = virt_machine_is_confidential(vms);
+    vms->bootinfo.skip_bootloader = vms->bootinfo.confidential;
     arm_load_kernel(ARM_CPU(first_cpu), machine, &vms->bootinfo);
 
     vms->machine_done.notify = virt_machine_done;
-- 
2.47.0

[RFC PATCH v3 23/26] hw/tpm: Add TPM event log

[Jean-Philippe Brucker]

Provide a library allowing the VMM to create an event log that describes
what is loaded into memory. During remote attestation in confidential
computing this helps an independent verifier reconstruct the initial
measurements of a VM, which contain the initial state of memory and
CPUs.

We provide some definitions and structures described by the Trusted
Computing Group (TCG) in "TCG PC Client Platform Firmware Profile
Specification" Level 00 Version 1.06 Revision 52 [1]. This is the same
format used by UEFI, and UEFI could reuse this log after finding it in
DT or ACPI tables, but can also copy its content into a new one.

[1] https://trustedcomputinggroup.org/resource/pc-client-specific-platform-firmware-profile-specification/

Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: New
---
 qapi/tpm.json            |  14 ++
 include/hw/tpm/tpm_log.h |  89 +++++++++++
 hw/tpm/tpm_log.c         | 325 +++++++++++++++++++++++++++++++++++++++
 hw/tpm/Kconfig           |   4 +
 hw/tpm/meson.build       |   1 +
 5 files changed, 433 insertions(+)
 create mode 100644 include/hw/tpm/tpm_log.h
 create mode 100644 hw/tpm/tpm_log.c

diff --git a/qapi/tpm.json b/qapi/tpm.json
index a16a72edb9..697e7150ee 100644
--- a/qapi/tpm.json
+++ b/qapi/tpm.json
@@ -188,3 +188,17 @@
 ##
 { 'command': 'query-tpm', 'returns': ['TPMInfo'],
   'if': 'CONFIG_TPM' }
+
+##
+# @TpmLogDigestAlgo:
+#
+# @sha256: Use the SHA256 algorithm
+#
+# @sha512: Use the SHA512 algorithm
+#
+# Algorithm to use for event log digests
+#
+# Since: 9.3
+##
+{ 'enum': 'TpmLogDigestAlgo',
+  'data': ['sha256', 'sha512'] }
diff --git a/include/hw/tpm/tpm_log.h b/include/hw/tpm/tpm_log.h
new file mode 100644
index 0000000000..b3cd2e7563
--- /dev/null
+++ b/include/hw/tpm/tpm_log.h
@@ -0,0 +1,89 @@
+#ifndef QEMU_TPM_LOG_H
+#define QEMU_TPM_LOG_H
+
+#include "qom/object.h"
+#include "sysemu/tpm.h"
+
+/*
+ * Defined in: TCG Algorithm Registry
+ * Family 2.0 Level 00 Revision 01.34
+ *
+ * (Here TCG stands for Trusted Computing Group)
+ */
+#define TCG_ALG_SHA256  0xB
+#define TCG_ALG_SHA512  0xD
+
+/* Size of a digest in bytes */
+#define TCG_ALG_SHA256_DIGEST_SIZE      32
+#define TCG_ALG_SHA512_DIGEST_SIZE      64
+
+/*
+ * Defined in: TCG PC Client Platform Firmware Profile Specification
+ * Version 1.06 revision 52
+ */
+#define TCG_EV_NO_ACTION                        0x00000003
+#define TCG_EV_EVENT_TAG                        0x00000006
+#define TCG_EV_POST_CODE2                       0x00000013
+#define TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2      0x8000000A
+
+struct UefiPlatformFirmwareBlob2Head {
+        uint8_t blob_description_size;
+        uint8_t blob_description[];
+} __attribute__((packed));
+
+struct UefiPlatformFirmwareBlob2Tail {
+        uint64_t blob_base;
+        uint64_t blob_size;
+} __attribute__((packed));
+
+#define TYPE_TPM_LOG "tpm-log"
+
+OBJECT_DECLARE_SIMPLE_TYPE(TpmLog, TPM_LOG)
+
+/**
+ * tpm_log_create - Create the event log
+ * @log: the log object
+ * @max_size: maximum size of the log. Adding an event past that size will
+ *            return an error
+ * @errp: pointer to a NULL-initialized error object
+ *
+ * Allocate the event log and create the initial entry (Spec ID Event03)
+ * describing the log format.
+ *
+ * Returns: 0 on success, -1 on error
+ */
+int tpm_log_create(TpmLog *log, size_t max_size, Error **errp);
+
+/**
+ * tpm_log_add_event - Append an event to the log
+ * @log: the log object
+ * @event_type: the `eventType` field in TCG_PCR_EVENT2
+ * @event: the `event` field in TCG_PCR_EVENT2
+ * @event_size: the `eventSize` field in TCG_PCR_EVENT2
+ * @data: content to be hashed into the event digest. May be NULL.
+ * @data_size: size of @data. Should be zero when @data is NULL.
+ * @errp: pointer to a NULL-initialized error object
+ *
+ * Add a TCG_PCR_EVENT2 event to the event log. Depending on the event type, a
+ * data buffer may be hashed into the event digest (for example
+ * TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2 contains a digest of the blob.)
+ *
+ * Returns: 0 on success, -1 on error
+ */
+int tpm_log_add_event(TpmLog *log, uint32_t event_type, const uint8_t *event,
+                      size_t event_size, const uint8_t *data, size_t data_size,
+                      Error **errp);
+
+/**
+ * tpm_log_write_and_close - Move the log to guest memory
+ * @log: the log object
+ * @errp: pointer to a NULL-initialized error object
+ *
+ * Write the log into memory, at the address set in the load-addr property.
+ * After this operation, the log is not writable anymore.
+ *
+ * Return: 0 on success, -1 on error
+ */
+int tpm_log_write_and_close(TpmLog *log, Error **errp);
+
+#endif
diff --git a/hw/tpm/tpm_log.c b/hw/tpm/tpm_log.c
new file mode 100644
index 0000000000..e6183a6e70
--- /dev/null
+++ b/hw/tpm/tpm_log.c
@@ -0,0 +1,325 @@
+/*
+ * tpm_log.c - Event log as described by the Trusted Computing Group (TCG)
+ *
+ * Copyright (c) 2024 Linaro Ltd.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ * Create an event log in the format specified by:
+ *
+ *  TCG PC Client Platform Firmware Profile Specification
+ *  Level 00 Version 1.06 Revision 52
+ *  Family “2.0”
+ */
+
+#include "qemu/osdep.h"
+
+#include "crypto/hash.h"
+#include "exec/address-spaces.h"
+#include "exec/memory.h"
+#include "hw/tpm/tpm_log.h"
+#include "qapi/error.h"
+#include "qemu/bswap.h"
+#include "qom/object_interfaces.h"
+
+/*
+ * Legacy structure used only in the first event in the log, for compatibility
+ */
+struct TcgPcClientPcrEvent {
+        uint32_t pcr_index;
+        uint32_t event_type;
+        uint8_t  digest[20];
+        uint32_t event_data_size;
+        uint8_t  event[];
+} __attribute__((packed));
+
+struct TcgEfiSpecIdEvent {
+        uint8_t  signature[16];
+        uint32_t platform_class;
+        uint8_t  family_version_minor;
+        uint8_t  family_version_major;
+        uint8_t  spec_revision;
+        uint8_t  uintn_size;
+        uint32_t number_of_algorithms; /* 1 */
+        /*
+         * For now we declare a single algo, but if we want UEFI to reuse this
+         * header then we'd need to add entries here for all algos supported by
+         * UEFI (and expand the digest field for EV_NO_ACTION).
+         */
+        uint16_t algorithm_id;
+        uint16_t digest_size;
+        uint8_t  vendor_info_size;
+        uint8_t  vendor_info[];
+} __attribute__((packed));
+
+struct TcgPcrEvent2Head {
+        uint32_t pcr_index;
+        uint32_t event_type;
+        /* variable-sized digests */
+        uint8_t  digests[];
+} __attribute__((packed));
+
+struct TcgPcrEvent2Tail {
+        uint32_t event_size;
+        uint8_t  event[];
+} __attribute__((packed));
+
+struct TpmlDigestValues {
+        uint32_t count;     /* 1 */
+        uint16_t hash_alg;
+        uint8_t  digest[];
+} __attribute__((packed));
+
+struct TpmLog {
+    Object parent_obj;
+
+    TpmLogDigestAlgo digest_algo;
+    size_t max_size;
+    uint64_t load_addr;
+
+    uint16_t tcg_algo;
+    GByteArray *content;
+    uint8_t *digest;
+    size_t digest_size;
+};
+
+OBJECT_DEFINE_SIMPLE_TYPE(TpmLog, tpm_log, TPM_LOG, OBJECT)
+
+static void tpm_log_init(Object *obj)
+{
+    TpmLog *log = TPM_LOG(obj);
+
+    log->digest_algo = TPM_LOG_DIGEST_ALGO_SHA256;
+}
+
+static void tpm_log_destroy(TpmLog *log)
+{
+    if (!log->content) {
+        return;
+    }
+    g_free(log->digest);
+    log->digest = NULL;
+    g_byte_array_free(log->content, /* free_segment */ true);
+    log->content = NULL;
+}
+
+static void tpm_log_finalize(Object *obj)
+{
+    tpm_log_destroy(TPM_LOG(obj));
+}
+
+static int tpm_log_get_digest_algo(Object *obj, Error **errp)
+{
+    TpmLog *log = TPM_LOG(obj);
+
+    return log->digest_algo;
+}
+
+static void tpm_log_set_digest_algo(Object *obj, int algo, Error **errp)
+{
+    TpmLog *log = TPM_LOG(obj);
+
+    if (log->content != NULL) {
+        error_setg(errp, "cannot set digest algo after log creation");
+        return;
+    }
+
+    log->digest_algo = algo;
+}
+
+static void tpm_log_get_max_size(Object *obj, Visitor *v, const char *name,
+                                void *opaque, Error **errp)
+{
+    TpmLog *log = TPM_LOG(obj);
+    uint64_t value = log->max_size;
+
+    visit_type_uint64(v, name, &value, errp);
+}
+
+static void tpm_log_get_load_addr(Object *obj, Visitor *v, const char *name,
+                                  void *opaque, Error **errp)
+{
+    TpmLog *log = TPM_LOG(obj);
+    uint64_t value = log->load_addr;
+
+    visit_type_uint64(v, name, &value, errp);
+}
+
+static void tpm_log_set_load_addr(Object *obj, Visitor *v, const char *name,
+                                  void *opaque, Error **errp)
+{
+    TpmLog *log = TPM_LOG(obj);
+    uint64_t value;
+
+    if (!visit_type_uint64(v, name, &value, errp)) {
+        return;
+    }
+
+    log->load_addr = value;
+}
+
+
+static void tpm_log_class_init(ObjectClass *oc, void *data)
+{
+    object_class_property_add_enum(oc, "digest-algo",
+                                   "TpmLogDigestAlgo",
+                                   &TpmLogDigestAlgo_lookup,
+                                   tpm_log_get_digest_algo,
+                                   tpm_log_set_digest_algo);
+    object_class_property_set_description(oc, "digest-algo",
+            "Algorithm used to hash blobs added as events ('sha256', 'sha512')");
+
+    /* max_size is set while allocating the log in tpm_log_create */
+    object_class_property_add(oc, "max-size", "uint64", tpm_log_get_max_size,
+                              NULL, NULL, NULL);
+    object_class_property_set_description(oc, "max-size",
+            "Maximum size of the log, reserved in guest memory");
+
+    object_class_property_add(oc, "load-addr", "uint64", tpm_log_get_load_addr,
+                              tpm_log_set_load_addr, NULL, NULL);
+    object_class_property_set_description(oc, "load-addr",
+            "Base address of the log in guest memory");
+}
+
+int tpm_log_create(TpmLog *log, size_t max_size, Error **errp)
+{
+    struct TcgEfiSpecIdEvent event;
+    struct TcgPcClientPcrEvent header = {
+        .pcr_index = 0,
+        .event_type = cpu_to_le32(TCG_EV_NO_ACTION),
+        .digest = {0},
+        .event_data_size = cpu_to_le32(sizeof(event)),
+    };
+
+    log->content = g_byte_array_sized_new(max_size);
+    log->max_size = max_size;
+
+    switch (log->digest_algo) {
+    case TPM_LOG_DIGEST_ALGO_SHA256:
+        log->tcg_algo = TCG_ALG_SHA256;
+        log->digest_size = TCG_ALG_SHA256_DIGEST_SIZE;
+        break;
+    case TPM_LOG_DIGEST_ALGO_SHA512:
+        log->tcg_algo = TCG_ALG_SHA512;
+        log->digest_size = TCG_ALG_SHA512_DIGEST_SIZE;
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    log->digest = g_malloc0(log->digest_size);
+
+    event = (struct TcgEfiSpecIdEvent) {
+        .signature = "Spec ID Event03",
+        .platform_class = 0,
+        .family_version_minor = 0,
+        .family_version_major = 2,
+        .spec_revision = 106,
+        .uintn_size = 2, /* UINT64 */
+        .number_of_algorithms = cpu_to_le32(1),
+        .algorithm_id = cpu_to_le16(log->tcg_algo),
+        .digest_size = cpu_to_le16(log->digest_size),
+        .vendor_info_size = 0,
+    };
+
+    g_byte_array_append(log->content, (guint8 *)&header, sizeof(header));
+    g_byte_array_append(log->content, (guint8 *)&event, sizeof(event));
+    return 0;
+}
+
+int tpm_log_add_event(TpmLog *log, uint32_t event_type, const uint8_t *event,
+                      size_t event_size, const uint8_t *data, size_t data_size,
+                      Error **errp)
+{
+    int digests = 0;
+    size_t rollback_len;
+    struct TcgPcrEvent2Head header = {
+        .pcr_index = 0,
+        .event_type = cpu_to_le32(event_type),
+    };
+    struct TpmlDigestValues digest_header = {0};
+    struct TcgPcrEvent2Tail tail = {
+        .event_size = cpu_to_le32(event_size),
+    };
+
+    if (log->content == NULL) {
+        error_setg(errp, "event log is not initialized");
+        return -EINVAL;
+    }
+    rollback_len = log->content->len;
+
+    g_byte_array_append(log->content, (guint8 *)&header, sizeof(header));
+
+    if (data) {
+        QCryptoHashAlgo qc_algo;
+
+        digest_header.hash_alg = cpu_to_le16(log->tcg_algo);
+        switch (log->digest_algo) {
+        case TPM_LOG_DIGEST_ALGO_SHA256:
+            qc_algo = QCRYPTO_HASH_ALGO_SHA256;
+            break;
+        case TPM_LOG_DIGEST_ALGO_SHA512:
+            qc_algo = QCRYPTO_HASH_ALGO_SHA512;
+            break;
+        default:
+            g_assert_not_reached();
+        }
+        if (qcrypto_hash_bytes(qc_algo, (const char *)data, data_size,
+                               &log->digest, &log->digest_size, errp)) {
+            goto err_rollback;
+        }
+        digests = 1;
+    } else if (event_type == TCG_EV_NO_ACTION) {
+        /* EV_NO_ACTION contains empty digests for each supported algo */
+        memset(log->digest, 0, log->digest_size);
+        digest_header.hash_alg = 0;
+        digests = 1;
+    }
+
+    if (digests) {
+        digest_header.count = cpu_to_le32(digests);
+        g_byte_array_append(log->content, (guint8 *)&digest_header,
+                            sizeof(digest_header));
+        g_byte_array_append(log->content, log->digest, log->digest_size);
+    } else {
+        /* Add an empty digests list */
+        g_byte_array_append(log->content, (guint8 *)&digest_header.count,
+                            sizeof(digest_header.count));
+    }
+
+    g_byte_array_append(log->content, (guint8 *)&tail, sizeof(tail));
+    g_byte_array_append(log->content, event, event_size);
+
+    if (log->content->len > log->max_size) {
+        error_setg(errp, "event log exceeds max size");
+        goto err_rollback;
+    }
+
+    return 0;
+
+err_rollback:
+    g_byte_array_set_size(log->content, rollback_len);
+    return -1;
+}
+
+int tpm_log_write_and_close(TpmLog *log, Error **errp)
+{
+    int ret;
+
+    if (!log->content) {
+        error_setg(errp, "event log is not initialized");
+        return -1;
+    }
+
+    ret = address_space_write_rom(&address_space_memory, log->load_addr,
+                                  MEMTXATTRS_UNSPECIFIED, log->content->data,
+                                  log->content->len);
+    if (ret) {
+        error_setg(errp, "cannot load log into memory");
+        return -1;
+    }
+
+    tpm_log_destroy(log);
+    return ret;
+}
diff --git a/hw/tpm/Kconfig b/hw/tpm/Kconfig
index a46663288c..70694b14a3 100644
--- a/hw/tpm/Kconfig
+++ b/hw/tpm/Kconfig
@@ -30,3 +30,7 @@ config TPM_SPAPR
     default y
     depends on TPM && PSERIES
     select TPM_BACKEND
+
+config TPM_LOG
+    bool
+    default y
diff --git a/hw/tpm/meson.build b/hw/tpm/meson.build
index 6968e60b3f..81efb557f3 100644
--- a/hw/tpm/meson.build
+++ b/hw/tpm/meson.build
@@ -6,4 +6,5 @@ system_ss.add(when: 'CONFIG_TPM_CRB', if_true: files('tpm_crb.c'))
 system_ss.add(when: 'CONFIG_TPM_TIS', if_true: files('tpm_ppi.c'))
 system_ss.add(when: 'CONFIG_TPM_CRB', if_true: files('tpm_ppi.c'))
 
+system_ss.add(when: 'CONFIG_TPM_LOG', if_true: files('tpm_log.c'))
 specific_ss.add(when: 'CONFIG_TPM_SPAPR', if_true: files('tpm_spapr.c'))
-- 
2.47.0

[RFC PATCH v3 24/26] hw/core/loader: Add fields to RomLoaderNotify

[Jean-Philippe Brucker]

In order to write an event log, the ROM load notification handler needs
two more fields.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: New
---
 include/hw/loader.h | 2 ++
 hw/core/loader.c    | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/include/hw/loader.h b/include/hw/loader.h
index 0cd9905f97..73f317966d 100644
--- a/include/hw/loader.h
+++ b/include/hw/loader.h
@@ -355,6 +355,8 @@ ssize_t rom_add_option(const char *file, int32_t bootindex);
 
 typedef struct RomLoaderNotify {
     /* Parameters passed to rom_add_blob() */
+    const char *name;
+    uint8_t *data;
     hwaddr addr;
     size_t len;
     size_t max_len;
diff --git a/hw/core/loader.c b/hw/core/loader.c
index 759a62cf58..4ff40e8762 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -1278,6 +1278,8 @@ static void rom_reset(void *unused)
         trace_loader_write_rom(rom->name, rom->addr, rom->datasize, rom->isrom);
 
         notify = (RomLoaderNotify) {
+            .name = rom->name,
+            .data = rom->data,
             .addr = rom->addr,
             .len = rom->datasize,
             .max_len = rom->romsize,
-- 
2.47.0

[RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Jean-Philippe Brucker]

Create an event log, in the format defined by Trusted Computing Group
for TPM2. It contains information about the VMM, the Realm parameters,
any data loaded into guest memory before boot and the initial vCPU
state.

The guest can access this log from RAM and send it to a verifier, to
help the verifier independently compute the Realm Initial Measurement,
and check that the data we load into guest RAM is known-good images.
Without this log, the verifier has to guess where everything is loaded
and in what order.

Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: New
---
 qapi/qom.json        |   9 +-
 target/arm/kvm_arm.h |  27 +++
 target/arm/kvm-rme.c | 415 ++++++++++++++++++++++++++++++++++++++++++-
 target/arm/Kconfig   |   1 +
 4 files changed, 449 insertions(+), 3 deletions(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index 901ba67634..1de1b0d8af 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1094,11 +1094,18 @@
 # @measurement-algorithm: Realm measurement algorithm
 #     (default: sha512)
 #
+# @measurement-log: Enable a measurement log for the Realm. All events
+#     that contribute to the Realm Initial Measurement (RIM) are added
+#     to a log in TCG TPM2 format, which is itself loaded into Realm
+#     memory (unmeasured) and can then be read by a verifier to
+#     reconstruct the RIM.
+#
 # Since: 9.3
 ##
 { 'struct': 'RmeGuestProperties',
   'data': { '*personalization-value': 'str',
-            '*measurement-algorithm': 'RmeGuestMeasurementAlgorithm' } }
+            '*measurement-algorithm': 'RmeGuestMeasurementAlgorithm',
+            '*measurement-log': 'bool'} }
 
 ##
 # @ObjectType:
diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
index 77680f238a..44e95a034b 100644
--- a/target/arm/kvm_arm.h
+++ b/target/arm/kvm_arm.h
@@ -268,6 +268,24 @@ int kvm_arm_rme_vcpu_init(CPUState *cs);
  */
 void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size);
 
+/**
+ * kvm_arm_rme_get_measurement_log
+ *
+ * Obtain the measurement log object if enabled, in order to get its size and
+ * set its base address.
+ *
+ * Returns NULL if measurement log is disabled.
+ */
+Object *kvm_arm_rme_get_measurement_log(void);
+
+/**
+ * kvm_arm_rme_set_ipa_size
+ * @ipa_bits: number of guest physical address bits
+ *
+ * Set the GPA size, not counting the bit reserved for shared address range.
+ */
+void kvm_arm_rme_set_ipa_size(uint8_t ipa_bits);
+
 #else
 
 /*
@@ -298,6 +316,15 @@ static inline void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size)
 {
 }
 
+static inline Object *kvm_arm_rme_get_measurement_log(void)
+{
+    return NULL;
+}
+
+static inline void kvm_arm_rme_set_ipa_size(uint8_t ipa_size)
+{
+}
+
 /*
  * These functions should never actually be called without KVM support.
  */
diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
index bf0bcf9a38..f92cfdb5f3 100644
--- a/target/arm/kvm-rme.c
+++ b/target/arm/kvm-rme.c
@@ -10,10 +10,12 @@
 #include "hw/boards.h"
 #include "hw/core/cpu.h"
 #include "hw/loader.h"
+#include "hw/tpm/tpm_log.h"
 #include "kvm_arm.h"
 #include "migration/blocker.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
+#include "qemu/units.h"
 #include "qom/object_interfaces.h"
 #include "sysemu/kvm.h"
 #include "sysemu/runstate.h"
@@ -25,6 +27,8 @@ OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
 
 #define RME_MAX_CFG         2
 
+#define RME_MEASUREMENT_LOG_SIZE    (64 * KiB)
+
 struct RmeGuest {
     ConfidentialGuestSupport parent_obj;
     Notifier rom_load_notifier;
@@ -32,22 +36,344 @@ struct RmeGuest {
 
     uint8_t *personalization_value;
     RmeGuestMeasurementAlgorithm measurement_algo;
+    bool use_measurement_log;
 
+    size_t num_cpus;
+    uint8_t ipa_bits;
     hwaddr ram_base;
     size_t ram_size;
+
+    TpmLog *log;
+    GHashTable *images;
 };
 
 OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
                                           CONFIDENTIAL_GUEST_SUPPORT,
                                           { TYPE_USER_CREATABLE }, { })
 
+typedef struct RmeLogFiletype {
+    uint32_t event_type;
+    /* Description copied into the log event */
+    const char *desc;
+} RmeLogFiletype;
+
 typedef struct {
     hwaddr base;
     hwaddr size;
+    uint8_t *data;
+    RmeLogFiletype *filetype;
 } RmeRamRegion;
 
+typedef struct {
+    char        signature[16];
+    char        name[32];
+    char        version[40];
+    uint64_t    ram_size;
+    uint32_t    num_cpus;
+    uint64_t    flags;
+} EventLogVmmVersion;
+
+typedef struct {
+    uint32_t    id;
+    uint32_t    data_size;
+    uint8_t     data[];
+} EventLogTagged;
+
+#define EVENT_LOG_TAG_REALM_CREATE  1
+#define EVENT_LOG_TAG_INIT_RIPAS    2
+#define EVENT_LOG_TAG_REC_CREATE    3
+
+#define REALM_PARAMS_FLAG_SVE       (1 << 1)
+#define REALM_PARAMS_FLAG_PMU       (1 << 2)
+
+#define REC_CREATE_FLAG_RUNNABLE    (1 << 0)
+
 static RmeGuest *rme_guest;
 
+static int rme_init_measurement_log(MachineState *ms)
+{
+    Object *log;
+    gpointer filename;
+    TpmLogDigestAlgo algo;
+    RmeLogFiletype *filetype;
+
+    if (!rme_guest->use_measurement_log) {
+        return 0;
+    }
+
+    switch (rme_guest->measurement_algo) {
+    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA256:
+        algo = TPM_LOG_DIGEST_ALGO_SHA256;
+        break;
+    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA512:
+        algo = TPM_LOG_DIGEST_ALGO_SHA512;
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    log = object_new_with_props(TYPE_TPM_LOG, OBJECT(rme_guest),
+                                "log", &error_fatal,
+                                "digest-algo", TpmLogDigestAlgo_str(algo),
+                                NULL);
+
+    tpm_log_create(TPM_LOG(log), RME_MEASUREMENT_LOG_SIZE, &error_fatal);
+    rme_guest->log = TPM_LOG(log);
+
+    /*
+     * Write down the image names we're expecting to encounter when handling the
+     * ROM load notifications, so we can record the type of image being loaded
+     * to help the verifier.
+     */
+    rme_guest->images = g_hash_table_new_full(g_str_hash, g_str_equal, g_free,
+                                              g_free);
+
+    filename = g_strdup(ms->kernel_filename);
+    if (filename) {
+        filetype = g_new0(RmeLogFiletype, 1);
+        filetype->event_type = TCG_EV_POST_CODE2;
+        filetype->desc = "KERNEL";
+        g_hash_table_insert(rme_guest->images, filename, (gpointer)filetype);
+    }
+
+    filename = g_strdup(ms->initrd_filename);
+    if (filename) {
+        filetype = g_new0(RmeLogFiletype, 1);
+        filetype->event_type = TCG_EV_POST_CODE2;
+        filetype->desc = "INITRD";
+        g_hash_table_insert(rme_guest->images, filename, (gpointer)filetype);
+    }
+
+    filename = g_strdup(ms->firmware);
+    if (filename) {
+        filetype = g_new0(RmeLogFiletype, 1);
+        filetype->event_type = TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2;
+        filetype->desc = "FIRMWARE";
+        g_hash_table_insert(rme_guest->images, filename, filetype);
+    }
+
+    filename = g_strdup(ms->dtb);
+    if (!filename) {
+        filename = g_strdup("dtb");
+    }
+    filetype = g_new0(RmeLogFiletype, 1);
+    filetype->event_type = TCG_EV_POST_CODE2;
+    filetype->desc = "DTB";
+    g_hash_table_insert(rme_guest->images, filename, filetype);
+
+    return 0;
+}
+
+static int rme_log_event_tag(uint32_t id, uint8_t *data, size_t size,
+                             Error **errp)
+{
+    int ret;
+    EventLogTagged event = {
+        .id = id,
+        .data_size = size,
+    };
+    GByteArray *bytes = g_byte_array_new();
+
+    if (!rme_guest->log) {
+        return 0;
+    }
+
+    g_byte_array_append(bytes, (uint8_t *)&event, sizeof(event));
+    g_byte_array_append(bytes, data, size);
+    ret = tpm_log_add_event(rme_guest->log, TCG_EV_EVENT_TAG, bytes->data,
+                             bytes->len, NULL, 0, errp);
+    g_byte_array_free(bytes, true);
+    return ret;
+}
+
+/* Log VM type and Realm Descriptor create */
+static int rme_log_realm_create(Error **errp)
+{
+    int ret;
+    ARMCPU *cpu;
+    EventLogVmmVersion vmm_version = {
+        .signature = "VM VERSION",
+        .name = "QEMU",
+        .version = "9.1", /* TODO: dynamic */
+        .ram_size = cpu_to_le64(rme_guest->ram_size),
+        .num_cpus = cpu_to_le32(rme_guest->num_cpus),
+        .flags = 0,
+    };
+    struct {
+        uint64_t    flags;
+        uint8_t     s2sz;
+        uint8_t     sve_vl;
+        uint8_t     num_bps;
+        uint8_t     num_wps;
+        uint8_t     pmu_num_ctrs;
+        uint8_t     hash_algo;
+    } params = {
+        .s2sz = rme_guest->ipa_bits,
+    };
+
+    if (!rme_guest->log) {
+        return 0;
+    }
+
+    ret = tpm_log_add_event(rme_guest->log, TCG_EV_NO_ACTION,
+                            (uint8_t *)&vmm_version, sizeof(vmm_version),
+                            NULL, 0, errp);
+    if (ret) {
+        return ret;
+    }
+
+    /* With KVM all CPUs have the same capability */
+    cpu = ARM_CPU(first_cpu);
+    if (cpu->has_pmu) {
+        params.flags |= REALM_PARAMS_FLAG_PMU;
+        params.pmu_num_ctrs = FIELD_EX64(cpu->isar.reset_pmcr_el0, PMCR, N);
+    }
+
+    if (cpu->sve_max_vq) {
+        params.flags |= REALM_PARAMS_FLAG_SVE;
+        params.sve_vl = cpu->sve_max_vq - 1;
+    }
+    params.num_bps = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, BRPS);
+    params.num_wps = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, WRPS);
+
+    switch (rme_guest->measurement_algo) {
+    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA256:
+        params.hash_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
+        break;
+    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA512:
+        params.hash_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA512;
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    return rme_log_event_tag(EVENT_LOG_TAG_REALM_CREATE, (uint8_t *)&params,
+                             sizeof(params), errp);
+}
+
+/* unmeasured images are logged with @data == NULL */
+static int rme_log_image(RmeLogFiletype *filetype, uint8_t *data, hwaddr base,
+                          size_t size, Error **errp)
+{
+    int ret;
+    size_t desc_size;
+    GByteArray *event = g_byte_array_new();
+    struct UefiPlatformFirmwareBlob2Head head = {0};
+    struct UefiPlatformFirmwareBlob2Tail tail = {0};
+
+    if (!rme_guest->log) {
+        return 0;
+    }
+
+    if (!filetype) {
+        error_setg(errp, "cannot log image without a filetype");
+        return -1;
+    }
+
+    /* EV_POST_CODE2 strings are not NUL-terminated */
+    desc_size = strlen(filetype->desc);
+    head.blob_description_size = desc_size;
+    tail.blob_base = cpu_to_le64(base);
+    tail.blob_size = cpu_to_le64(size);
+
+    g_byte_array_append(event, (guint8 *)&head, sizeof(head));
+    g_byte_array_append(event, (guint8 *)filetype->desc, desc_size);
+    g_byte_array_append(event, (guint8 *)&tail, sizeof(tail));
+
+    ret = tpm_log_add_event(rme_guest->log, filetype->event_type, event->data,
+                            event->len, data, size, errp);
+    g_byte_array_free(event, true);
+    return ret;
+}
+
+static int rme_log_ripas(hwaddr base, size_t size, Error **errp)
+{
+    struct {
+        uint64_t base;
+        uint64_t size;
+    } init_ripas = {
+        .base = cpu_to_le64(base),
+        .size = cpu_to_le64(size),
+    };
+
+    return rme_log_event_tag(EVENT_LOG_TAG_INIT_RIPAS, (uint8_t *)&init_ripas,
+                             sizeof(init_ripas), errp);
+}
+
+static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], Error **errp)
+{
+    struct {
+        uint64_t flags;
+        uint64_t pc;
+        uint64_t gprs[8];
+    } rec_create = {
+        .flags = cpu_to_le64(flags),
+        .pc = cpu_to_le64(pc),
+        .gprs[0] = cpu_to_le64(gprs[0]),
+        .gprs[1] = cpu_to_le64(gprs[1]),
+        .gprs[2] = cpu_to_le64(gprs[2]),
+        .gprs[3] = cpu_to_le64(gprs[3]),
+        .gprs[4] = cpu_to_le64(gprs[4]),
+        .gprs[5] = cpu_to_le64(gprs[5]),
+        .gprs[6] = cpu_to_le64(gprs[6]),
+        .gprs[7] = cpu_to_le64(gprs[7]),
+    };
+
+    return rme_log_event_tag(EVENT_LOG_TAG_REC_CREATE, (uint8_t *)&rec_create,
+                             sizeof(rec_create), errp);
+}
+
+static int rme_populate_range(hwaddr base, size_t size, bool measure,
+                              Error **errp);
+
+static int rme_close_measurement_log(Error **errp)
+{
+    int ret;
+    hwaddr base;
+    size_t size;
+    RmeLogFiletype filetype = {
+        .event_type = TCG_EV_POST_CODE2,
+        .desc = "LOG",
+    };
+
+    if (!rme_guest->log) {
+        return 0;
+    }
+
+    base = object_property_get_uint(OBJECT(rme_guest->log), "load-addr", errp);
+    if (*errp) {
+        return -1;
+    }
+
+    size = object_property_get_uint(OBJECT(rme_guest->log), "max-size", errp);
+    if (*errp) {
+        return -1;
+    }
+
+    /* Log the log itself */
+    ret = rme_log_image(&filetype, NULL, base, size, errp);
+    if (ret) {
+        return ret;
+    }
+
+    ret = tpm_log_write_and_close(rme_guest->log, errp);
+    if (ret) {
+        return ret;
+    }
+
+    ret = rme_populate_range(base, size, /* measure */ false, errp);
+    if (ret) {
+        return ret;
+    }
+
+    g_hash_table_destroy(rme_guest->images);
+
+    /* The log is now in the guest. Free this object */
+    object_unparent(OBJECT(rme_guest->log));
+    rme_guest->log = NULL;
+    return 0;
+}
+
 static int rme_configure_one(RmeGuest *guest, uint32_t cfg, Error **errp)
 {
     int ret;
@@ -120,9 +446,10 @@ static int rme_init_ram(hwaddr base, size_t size, Error **errp)
         error_setg_errno(errp, -ret,
                          "failed to init RAM [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
                          start, end);
+        return ret;
     }
 
-    return ret;
+    return rme_log_ripas(base, size, errp);
 }
 
 static int rme_populate_range(hwaddr base, size_t size, bool measure,
@@ -158,23 +485,42 @@ static void rme_populate_ram_region(gpointer data, gpointer err)
     }
 
     rme_populate_range(region->base, region->size, /* measure */ true, errp);
+    if (*errp) {
+        return;
+    }
+
+    rme_log_image(region->filetype, region->data, region->base, region->size,
+                  errp);
 }
 
 static int rme_init_cpus(Error **errp)
 {
     int ret;
     CPUState *cs;
+    bool logged_primary_cpu = false;
 
     /*
      * Now that do_cpu_reset() initialized the boot PC and
      * kvm_cpu_synchronize_post_reset() registered it, we can finalize the REC.
      */
     CPU_FOREACH(cs) {
-        ret = kvm_arm_vcpu_finalize(ARM_CPU(cs), KVM_ARM_VCPU_REC);
+        ARMCPU *cpu = ARM_CPU(cs);
+
+        ret = kvm_arm_vcpu_finalize(cpu, KVM_ARM_VCPU_REC);
         if (ret) {
             error_setg_errno(errp, -ret, "failed to finalize vCPU");
             return ret;
         }
+
+        if (!logged_primary_cpu) {
+            ret = rme_log_rec(REC_CREATE_FLAG_RUNNABLE, cpu->env.pc,
+                              cpu->env.xregs, errp);
+            if (ret) {
+                return ret;
+            }
+
+            logged_primary_cpu = true;
+        }
     }
     return 0;
 }
@@ -194,6 +540,10 @@ static int rme_create_realm(Error **errp)
         return -1;
     }
 
+    if (rme_log_realm_create(errp)) {
+        return -1;
+    }
+
     if (rme_init_ram(rme_guest->ram_base, rme_guest->ram_size, errp)) {
         return -1;
     }
@@ -208,6 +558,10 @@ static int rme_create_realm(Error **errp)
         return -1;
     }
 
+    if (rme_close_measurement_log(errp)) {
+        return -1;
+    }
+
     ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
                             KVM_CAP_ARM_RME_ACTIVATE_REALM);
     if (ret) {
@@ -303,6 +657,20 @@ static void rme_set_measurement_algo(Object *obj, int algo, Error **errp)
     guest->measurement_algo = algo;
 }
 
+static bool rme_get_measurement_log(Object *obj, Error **errp)
+{
+    RmeGuest *guest = RME_GUEST(obj);
+
+    return guest->use_measurement_log;
+}
+
+static void rme_set_measurement_log(Object *obj, bool value, Error **errp)
+{
+    RmeGuest *guest = RME_GUEST(obj);
+
+    guest->use_measurement_log = value;
+}
+
 static void rme_guest_class_init(ObjectClass *oc, void *data)
 {
     object_class_property_add_str(oc, "personalization-value", rme_get_rpv,
@@ -317,6 +685,12 @@ static void rme_guest_class_init(ObjectClass *oc, void *data)
                                    rme_set_measurement_algo);
     object_class_property_set_description(oc, "measurement-algorithm",
             "Realm measurement algorithm ('sha256', 'sha512')");
+
+    object_class_property_add_bool(oc, "measurement-log",
+                                   rme_get_measurement_log,
+                                   rme_set_measurement_log);
+    object_class_property_set_description(oc, "measurement-log",
+            "Enable/disable Realm measurement log");
 }
 
 static void rme_guest_init(Object *obj)
@@ -359,6 +733,20 @@ static void rme_rom_load_notify(Notifier *notifier, void *data)
     region = g_new0(RmeRamRegion, 1);
     region->base = rom->addr;
     region->size = rom->len;
+    /*
+     * TODO: double-check lifetime. Is data is still available when we measure
+     * it, while writing the log. Should be fine since data is kept for the next
+     * reset.
+     */
+    region->data = rom->data;
+
+    /*
+     * rme_guest->images is destroyed after ram_regions, so we can store
+     * filetype even if we don't own the struct.
+     */
+    if (rme_guest->images) {
+        region->filetype = g_hash_table_lookup(rme_guest->images, rom->name);
+    }
 
     /*
      * The Realm Initial Measurement (RIM) depends on the order in which we
@@ -388,6 +776,13 @@ int kvm_arm_rme_init(MachineState *ms)
         return -ENODEV;
     }
 
+    if (rme_init_measurement_log(ms)) {
+        return -ENODEV;
+    }
+
+    rme_guest->ram_size = ms->ram_size;
+    rme_guest->num_cpus = ms->smp.max_cpus;
+
     error_setg(&rme_mig_blocker, "RME: migration is not implemented");
     migrate_add_blocker(&rme_mig_blocker, &error_fatal);
 
@@ -430,3 +825,19 @@ int kvm_arm_rme_vm_type(MachineState *ms)
     }
     return 0;
 }
+
+void kvm_arm_rme_set_ipa_size(uint8_t ipa_bits)
+{
+    if (rme_guest) {
+        /* We request one more bit to KVM as the NS flag */
+        rme_guest->ipa_bits = ipa_bits + 1;
+    }
+}
+
+Object *kvm_arm_rme_get_measurement_log(void)
+{
+    if (rme_guest) {
+        return OBJECT(rme_guest->log);
+    }
+    return NULL;
+}
diff --git a/target/arm/Kconfig b/target/arm/Kconfig
index 7f8a2217ae..ee3a2184d0 100644
--- a/target/arm/Kconfig
+++ b/target/arm/Kconfig
@@ -13,3 +13,4 @@ config AARCH64
     select ARM
     # kvm_arch_fixup_msi_route() needs to access PCIDevice
     select PCI if KVM
+    select TPM_LOG if KVM
-- 
2.47.0

[RFC PATCH v3 26/26] hw/arm/virt: Add measurement log for confidential boot

[Jean-Philippe Brucker]

Create a measurement log describing operations performed by QEMU to
initialize the guest, and load it into guest memory above the DTB.

Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v2->v3: New
---
 include/hw/arm/boot.h |  3 +++
 include/hw/arm/virt.h |  1 +
 hw/arm/boot.c         | 47 +++++++++++++++++++++++++++++++++++++++++++
 hw/arm/virt.c         | 23 +++++++++++++++++++++
 4 files changed, 74 insertions(+)

diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
index 5fcbaa2625..f2518c4e81 100644
--- a/include/hw/arm/boot.h
+++ b/include/hw/arm/boot.h
@@ -147,6 +147,9 @@ struct arm_boot_info {
      * Confidential guest boot loads everything into RAM so it can be measured.
      */
     bool confidential;
+    /* measurement log location in guest memory */
+    hwaddr log_start;
+    size_t log_size;
 };
 
 /**
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index e5e9f67f52..fe1c464be6 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -180,6 +180,7 @@ struct VirtMachineState {
     char *oem_id;
     char *oem_table_id;
     bool ns_el2_virt_timer_irq;
+    Object *event_log;
 };
 
 #define VIRT_ECAM_ID(high) (high ? VIRT_HIGH_PCIE_ECAM : VIRT_PCIE_ECAM)
diff --git a/hw/arm/boot.c b/hw/arm/boot.c
index e461137595..1ced008062 100644
--- a/hw/arm/boot.c
+++ b/hw/arm/boot.c
@@ -665,6 +665,24 @@ int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo,
 
     fdt_add_psci_node(fdt);
 
+    /* Add a reserved-memory node for the event log */
+    if (binfo->log_size) {
+        char *nodename;
+
+        qemu_fdt_add_subnode(fdt, "/reserved-memory");
+        qemu_fdt_setprop_cell(fdt, "/reserved-memory", "#address-cells", 0x2);
+        qemu_fdt_setprop_cell(fdt, "/reserved-memory", "#size-cells", 0x2);
+        qemu_fdt_setprop(fdt, "/reserved-memory", "ranges", NULL, 0);
+
+        nodename = g_strdup_printf("/reserved-memory/event-log@%" PRIx64,
+                                   binfo->log_start);
+        qemu_fdt_add_subnode(fdt, nodename);
+        qemu_fdt_setprop_string(fdt, nodename, "compatible", "cc-event-log");
+        qemu_fdt_setprop_sized_cells(fdt, nodename, "reg", 2, binfo->log_start,
+                                           2, binfo->log_size);
+        g_free(nodename);
+    }
+
     if (binfo->modify_dtb) {
         binfo->modify_dtb(binfo, fdt);
     }
@@ -943,6 +961,30 @@ static uint64_t load_aarch64_image(const char *filename, hwaddr mem_base,
     return kernel_size;
 }
 
+static void add_event_log(struct arm_boot_info *info)
+{
+    if (!info->log_size) {
+        return;
+    }
+
+    if (!info->dtb_limit) {
+        int dtb_size = 0;
+
+        if (!info->get_dtb(info, &dtb_size) || dtb_size == 0) {
+            error_report("Board does not have a DTB");
+            exit(1);
+        }
+        info->dtb_limit = info->dtb_start + dtb_size;
+    }
+
+    info->log_start = info->dtb_limit;
+    if (info->log_start + info->log_size >
+        info->loader_start + info->ram_size) {
+        error_report("Not enough space for measurement log and DTB");
+        exit(1);
+    }
+}
+
 static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
                                          struct arm_boot_info *info)
 {
@@ -990,6 +1032,7 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
             }
             info->dtb_start = info->loader_start;
             info->dtb_limit = image_low_addr;
+            add_event_log(info);
         }
     }
     entry = elf_entry;
@@ -1128,6 +1171,8 @@ static void arm_setup_direct_kernel_boot(ARMCPU *cpu,
                 error_report("Not enough space for DTB after kernel/initrd");
                 exit(1);
             }
+            add_event_log(info);
+
             fixupcontext[FIXUP_ARGPTR_LO] = info->dtb_start;
             fixupcontext[FIXUP_ARGPTR_HI] = info->dtb_start >> 32;
         } else {
@@ -1189,6 +1234,8 @@ static void arm_setup_confidential_firmware_boot(ARMCPU *cpu,
         error_report("could not load firmware '%s'", firmware_filename);
         exit(1);
     }
+
+    add_event_log(info);
 }
 
 static void arm_setup_firmware_boot(ARMCPU *cpu, struct arm_boot_info *info,
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 5247f53882..1e0f664af0 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1808,6 +1808,11 @@ void virt_machine_done(Notifier *notifier, void *data)
         exit(1);
     }
 
+    if (vms->event_log) {
+        object_property_set_uint(vms->event_log, "load-addr",
+                                 vms->bootinfo.log_start, &error_fatal);
+    }
+
     fw_cfg_add_extra_pci_roots(vms->bus, vms->fw_cfg);
 
     virt_acpi_setup(vms);
@@ -2149,6 +2154,21 @@ static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
     }
 }
 
+static void create_measurement_log(VirtMachineState *vms)
+{
+    Error *err = NULL;
+
+    vms->event_log = kvm_arm_rme_get_measurement_log();
+    if (vms->event_log == NULL) {
+        return;
+    }
+    vms->bootinfo.log_size = object_property_get_uint(vms->event_log,
+                                                      "max-size", &err);
+    if (err != NULL) {
+        error_report_err(err);
+    }
+}
+
 static void machvirt_init(MachineState *machine)
 {
     VirtMachineState *vms = VIRT_MACHINE(machine);
@@ -2499,6 +2519,9 @@ static void machvirt_init(MachineState *machine)
                                vms->fw_cfg, OBJECT(vms));
     }
 
+    kvm_arm_rme_set_ipa_size(64 - clz64(vms->highest_gpa));
+    create_measurement_log(vms);
+
     vms->bootinfo.ram_size = machine->ram_size;
     vms->bootinfo.board_id = -1;
     vms->bootinfo.loader_start = vms->memmap[VIRT_MEM].base;
-- 
2.47.0

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Stefan Berger]

On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> Create an event log, in the format defined by Trusted Computing Group
> for TPM2. It contains information about the VMM, the Realm parameters,
> any data loaded into guest memory before boot and the initial vCPU
> state.
> 
> The guest can access this log from RAM and send it to a verifier, to
> help the verifier independently compute the Realm Initial Measurement,
> and check that the data we load into guest RAM is known-good images.
 > Without this log, the verifier has to guess where everything is 
loaded> and in what order.

Typically these logs are backed by extensions of TPM PCRs and when you 
send a log to a verifier you send a TPM quote along with it for the 
verifer to replay the log and check the TPM quote. Also, early code in 
the firmware is typically serving as a root of trust that starts the 
chain of measurements of code and data, first measuring itself and then 
other parts of the firmware before it jumps into the other parts. Now 
here you seem to just have a log and no PCR extensions and therefore no 
quote over PCRs can be used. Then what prevents anyone from faking this 
log and presenting a completely fake log to the verifier?

Re: [PATCH v3 10/26] target/arm/kvm-rme: Add Realm Personalization Value parameter

[Markus Armbruster]

Jean-Philippe Brucker <jean-philippe@linaro.org> writes:

> The Realm Personalization Value (RPV) is provided by the user to
> distinguish Realms that have the same initial measurement.
>
> The user provides up to 64 hexadecimal bytes. They are stored into the
> RPV in the same order, zero-padded on the right.
>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> Cc: Eduardo Habkost <eduardo@habkost.net>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: Fix documentation
> ---
>  qapi/qom.json        |  15 ++++++
>  target/arm/kvm-rme.c | 111 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 126 insertions(+)
>
> diff --git a/qapi/qom.json b/qapi/qom.json
> index a8beeabf1f..f982850bca 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -1068,6 +1068,19 @@
>    'data': { '*cpu-affinity': ['uint16'],
>              '*node-affinity': ['uint16'] } }
>  
> +##
> +# @RmeGuestProperties:
> +#
> +# Properties for rme-guest objects.
> +#
> +# @personalization-value: Realm personalization value, as a 64-byte
> +#     hex string. This optional parameter allows to uniquely identify
> +#     the VM instance during attestation. (default: 0)

QMP commonly uses base64 for encoding binary data.  Any particular
reason to deviate?

Is the "hex string" to be mapped to binary in little or big endian?  Not
an issue with base64.

Nitpick: (default: all-zero), please, for consistency with similar
documentation in SevSnpGuestProperties.

> +#
> +# Since: 9.3
> +##
> +{ 'struct': 'RmeGuestProperties',
> +  'data': { '*personalization-value': 'str' } }
>  
>  ##
>  # @ObjectType:
> @@ -1121,6 +1134,7 @@
>      { 'name': 'pr-manager-helper',
>        'if': 'CONFIG_LINUX' },
>      'qtest',
> +    'rme-guest',
>      'rng-builtin',
>      'rng-egd',
>      { 'name': 'rng-random',

The commit message claims the patch adds a parameter.  It actually adds
an entire object type.

> @@ -1195,6 +1209,7 @@
>        'pr-manager-helper':          { 'type': 'PrManagerHelperProperties',
>                                        'if': 'CONFIG_LINUX' },
>        'qtest':                      'QtestProperties',
> +      'rme-guest':                  'RmeGuestProperties',
>        'rng-builtin':                'RngProperties',
>        'rng-egd':                    'RngEgdProperties',
>        'rng-random':                 { 'type': 'RngRandomProperties',
> diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
> index 83a29421df..0be55867ee 100644
> --- a/target/arm/kvm-rme.c
> +++ b/target/arm/kvm-rme.c
> @@ -23,11 +23,15 @@ OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
>  
>  #define RME_PAGE_SIZE qemu_real_host_page_size()
>  
> +#define RME_MAX_CFG         1
> +
>  struct RmeGuest {
>      ConfidentialGuestSupport parent_obj;
>      Notifier rom_load_notifier;
>      GSList *ram_regions;
>  
> +    uint8_t *personalization_value;
> +
>      hwaddr ram_base;
>      size_t ram_size;
>  };
> @@ -43,6 +47,48 @@ typedef struct {
>  
>  static RmeGuest *rme_guest;
>  
> +static int rme_configure_one(RmeGuest *guest, uint32_t cfg, Error **errp)
> +{
> +    int ret;
> +    const char *cfg_str;
> +    struct kvm_cap_arm_rme_config_item args = {
> +        .cfg = cfg,
> +    };
> +
> +    switch (cfg) {
> +    case KVM_CAP_ARM_RME_CFG_RPV:
> +        if (!guest->personalization_value) {
> +            return 0;
> +        }
> +        memcpy(args.rpv, guest->personalization_value, KVM_CAP_ARM_RME_RPV_SIZE);
> +        cfg_str = "personalization value";
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }

This function gets called with @cfg arguments 0 .. RME_MAX_CFG-1, from
rme_configure() right below.  RME_MAX_CFG is defined to 1 above.

The switch assumes KVM_CAP_ARM_RME_CFG_RPV is zero.  Such assumptions
are ideally avoided, and alternatively checked at build time.

> +
> +    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
> +                            KVM_CAP_ARM_RME_CONFIG_REALM, (intptr_t)&args);
> +    if (ret) {
> +        error_setg_errno(errp, -ret, "failed to configure %s", cfg_str);
> +    }
> +    return ret;
> +}
> +
> +static int rme_configure(Error **errp)
> +{
> +    int ret;
> +    int cfg;
> +
> +    for (cfg = 0; cfg < RME_MAX_CFG; cfg++) {
> +        ret = rme_configure_one(rme_guest, cfg, errp);
> +        if (ret) {
> +            return ret;
> +        }
> +    }
> +    return 0;
> +}
> +
>  static int rme_init_ram(hwaddr base, size_t size, Error **errp)
>  {
>      int ret;
> @@ -123,6 +169,10 @@ static int rme_create_realm(Error **errp)
>  {
>      int ret;
>  
> +    if (rme_configure(errp)) {
> +        return -1;
> +    }
> +
>      ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
>                              KVM_CAP_ARM_RME_CREATE_RD);
>      if (ret) {
> @@ -168,8 +218,69 @@ static void rme_vm_state_change(void *opaque, bool running, RunState state)
>      }
>  }
>  
> +static char *rme_get_rpv(Object *obj, Error **errp)
> +{
> +    RmeGuest *guest = RME_GUEST(obj);
> +    GString *s;
> +    int i;
> +
> +    if (!guest->personalization_value) {
> +        return NULL;
> +    }
> +
> +    s = g_string_sized_new(KVM_CAP_ARM_RME_RPV_SIZE * 2 + 1);
> +
> +    for (i = 0; i < KVM_CAP_ARM_RME_RPV_SIZE; i++) {
> +        g_string_append_printf(s, "%02x", guest->personalization_value[i]);
> +    }

The size of the destrination string is known at compile time.  Why grow
it dynamically?

Base64 would take less code.

> +
> +    return g_string_free(s, /* free_segment */ false);
> +}
> +
> +static void rme_set_rpv(Object *obj, const char *value, Error **errp)
> +{
> +    RmeGuest *guest = RME_GUEST(obj);
> +    size_t len = strlen(value);
> +    uint8_t *out;
> +    int i = 1;
> +    int ret;
> +
> +    g_free(guest->personalization_value);
> +    guest->personalization_value = out = g_malloc0(KVM_CAP_ARM_RME_RPV_SIZE);
> +
> +    /* Two chars per byte */
> +    if (len > KVM_CAP_ARM_RME_RPV_SIZE * 2) {
> +        error_setg(errp, "Realm Personalization Value is too large");
> +        return;
> +    }
> +
> +    /* First byte may have a single char */
> +    if (len % 2) {
> +        ret = sscanf(value, "%1hhx", out++);
> +    } else {
> +        ret = sscanf(value, "%2hhx", out++);
> +        i++;
> +    }
> +    if (ret != 1) {
> +        error_setg(errp, "Invalid Realm Personalization Value");
> +        return;
> +    }
> +
> +    for (; i < len; i += 2) {
> +        ret = sscanf(value + i, "%2hhx", out++);
> +        if (ret != 1) {
> +            error_setg(errp, "Invalid Realm Personalization Value");
> +            return;
> +        }
> +    }

Looks like this supports hex strings shorter than
KVM_CAP_ARM_RME_RPV_SIZE * 2.  How these get padded is not documented.
Fixable, but are you sure the convenience is worth the complexity?

Again, base64 would take less code.

> +}
> +
>  static void rme_guest_class_init(ObjectClass *oc, void *data)
>  {
> +    object_class_property_add_str(oc, "personalization-value", rme_get_rpv,
> +                                  rme_set_rpv);
> +    object_class_property_set_description(oc, "personalization-value",
> +            "Realm personalization value (512-bit hexadecimal number)");
>  }
>  
>  static void rme_guest_init(Object *obj)

Re: [PATCH v3 01/26] kvm: Merge kvm_check_extension() and kvm_vm_check_extension()

[Daniel P. Berrangé]

On Mon, Nov 25, 2024 at 07:56:00PM +0000, Jean-Philippe Brucker wrote:
> The KVM_CHECK_EXTENSION ioctl can be issued either on the global fd
> (/dev/kvm), or on the VM fd obtained with KVM_CREATE_VM. For most
> extensions, KVM returns the same value with either method, but for some
> of them it can refine the returned value depending on the VM type. The
> KVM documentation [1] advises to use the VM fd:
> 
>   Based on their initialization different VMs may have different
>   capabilities. It is thus encouraged to use the vm ioctl to query for
>   capabilities (available with KVM_CAP_CHECK_EXTENSION_VM on the vm fd)
> 
> Ongoing work on Arm confidential VMs confirms this, as some capabilities
> become unavailable to confidential VMs, requiring changes in QEMU to use
> kvm_vm_check_extension() instead of kvm_check_extension() [2]. Rather
> than changing each check one by one, change kvm_check_extension() to
> always issue the ioctl on the VM fd when available, and remove
> kvm_vm_check_extension().

The downside I see of this approach is that it can potentially
mask mistakes / unexpected behaviour.

eg, consider you are in a code path where you /think/ the VM fd
is available, but for some unexpected reason it is NOT in fact
available. The code silently falls back to the global FD, thus
giving a potentially incorrect extension check answer.

Having separate check methods with no fallback ensures that we
are checking exactly what we /intend/ to be checking, or will
see an error

> 
> Fall back to the global fd when the VM check is unavailable:
> 
> * Ancient kernels do not support KVM_CHECK_EXTENSION on the VM fd, since
>   it was added by commit 92b591a4c46b ("KVM: Allow KVM_CHECK_EXTENSION
>   on the vm fd") in Linux 3.17 [3]. Support for Linux 3.16 ended in June
>   2020, but there may still be old images around.
> 
> * A couple of calls must be issued before the VM fd is available, since
>   they determine the VM type: KVM_CAP_MIPS_VZ and KVM_CAP_ARM_VM_IPA_SIZE
> 
> Does any user actually depend on the check being done on the global fd
> instead of the VM fd?  I surveyed all cases where KVM presently returns
> different values depending on the query method. Luckily QEMU already
> calls kvm_vm_check_extension() for most of those. Only three of them are
> ambiguous, because currently done on the global fd:
> 
> * KVM_CAP_MAX_VCPUS and KVM_CAP_MAX_VCPU_ID on Arm, changes value if the
>   user requests a vGIC different from the default. But QEMU queries this
>   before vGIC configuration, so the reported value will be the same.
> 
> * KVM_CAP_SW_TLB on PPC. When issued on the global fd, returns false if
>   the kvm-hv module is loaded; when issued on the VM fd, returns false
>   only if the VM type is HV instead of PR. If this returns false, then
>   QEMU will fail to initialize a BOOKE206 MMU model.
> 
>   So this patch supposedly improves things, as it allows to run this
>   type of vCPU even when both KVM modules are loaded.
> 
> * KVM_CAP_PPC_SECURE_GUEST. Similarly, doing this check on a VM fd
>   refines the returned value, and ensures that SVM is actually
>   supported. Since QEMU follows the check with kvm_vm_enable_cap(), this
>   patch should only provide better error reporting.
> 
> [1] https://www.kernel.org/doc/html/latest/virt/kvm/api.html#kvm-check-extension
> [2] https://lore.kernel.org/kvm/875ybi0ytc.fsf@redhat.com/
> [3] https://github.com/torvalds/linux/commit/92b591a4c46b
> 
> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> Cc: Nicholas Piggin <npiggin@gmail.com>
> Cc: Daniel Henrique Barboza <danielhb413@gmail.com>
> Cc: qemu-ppc@nongnu.org
> Suggested-by: Cornelia Huck <cohuck@redhat.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>  include/sysemu/kvm.h     |  2 --
>  include/sysemu/kvm_int.h |  1 +
>  accel/kvm/kvm-all.c      | 41 +++++++++++++++++++---------------------
>  target/arm/kvm.c         |  2 +-
>  target/i386/kvm/kvm.c    |  6 +++---
>  target/ppc/kvm.c         | 36 +++++++++++++++++------------------
>  6 files changed, 42 insertions(+), 46 deletions(-)
> 
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index c3a60b2890..63c96d0096 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -437,8 +437,6 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cpu);
>  
>  int kvm_check_extension(KVMState *s, unsigned int extension);
>  
> -int kvm_vm_check_extension(KVMState *s, unsigned int extension);
> -
>  #define kvm_vm_enable_cap(s, capability, cap_flags, ...)             \
>      ({                                                               \
>          struct kvm_enable_cap cap = {                                \
> diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
> index a1e72763da..cb38085d54 100644
> --- a/include/sysemu/kvm_int.h
> +++ b/include/sysemu/kvm_int.h
> @@ -166,6 +166,7 @@ struct KVMState
>      uint16_t xen_gnttab_max_frames;
>      uint16_t xen_evtchn_max_pirq;
>      char *device;
> +    bool check_extension_vm;
>  };
>  
>  void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 801cff16a5..7ea016d598 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -1238,7 +1238,11 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
>  {
>      int ret;
>  
> -    ret = kvm_ioctl(s, KVM_CHECK_EXTENSION, extension);
> +    if (!s->check_extension_vm) {
> +        ret = kvm_ioctl(s, KVM_CHECK_EXTENSION, extension);
> +    } else {
> +        ret = kvm_vm_ioctl(s, KVM_CHECK_EXTENSION, extension);
> +    }
>      if (ret < 0) {
>          ret = 0;
>      }
> @@ -1246,19 +1250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
>      return ret;
>  }
>  
> -int kvm_vm_check_extension(KVMState *s, unsigned int extension)
> -{
> -    int ret;
> -
> -    ret = kvm_vm_ioctl(s, KVM_CHECK_EXTENSION, extension);
> -    if (ret < 0) {
> -        /* VM wide version not implemented, use global one instead */
> -        ret = kvm_check_extension(s, extension);
> -    }
> -
> -    return ret;
> -}
> -
>  /*
>   * We track the poisoned pages to be able to:
>   * - replace them on VM reset
> @@ -1622,10 +1613,10 @@ static int kvm_dirty_ring_init(KVMState *s)
>       * Read the max supported pages. Fall back to dirty logging mode
>       * if the dirty ring isn't supported.
>       */
> -    ret = kvm_vm_check_extension(s, capability);
> +    ret = kvm_check_extension(s, capability);
>      if (ret <= 0) {
>          capability = KVM_CAP_DIRTY_LOG_RING_ACQ_REL;
> -        ret = kvm_vm_check_extension(s, capability);
> +        ret = kvm_check_extension(s, capability);
>      }
>  
>      if (ret <= 0) {
> @@ -1648,7 +1639,7 @@ static int kvm_dirty_ring_init(KVMState *s)
>      }
>  
>      /* Enable the backup bitmap if it is supported */
> -    ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP);
> +    ret = kvm_check_extension(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP);
>      if (ret > 0) {
>          ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP, 0);
>          if (ret) {
> @@ -2404,7 +2395,7 @@ static void kvm_irqchip_create(KVMState *s)
>   */
>  static int kvm_recommended_vcpus(KVMState *s)
>  {
> -    int ret = kvm_vm_check_extension(s, KVM_CAP_NR_VCPUS);
> +    int ret = kvm_check_extension(s, KVM_CAP_NR_VCPUS);
>      return (ret) ? ret : 4;
>  }
>  
> @@ -2625,7 +2616,12 @@ static int kvm_init(MachineState *ms)
>  
>      s->vmfd = ret;
>  
> -    s->nr_as = kvm_vm_check_extension(s, KVM_CAP_MULTI_ADDRESS_SPACE);
> +    ret = kvm_vm_ioctl(s, KVM_CHECK_EXTENSION, KVM_CAP_CHECK_EXTENSION_VM);
> +    if (ret > 0) {
> +        s->check_extension_vm = true;
> +    }
> +
> +    s->nr_as = kvm_check_extension(s, KVM_CAP_MULTI_ADDRESS_SPACE);
>      if (s->nr_as <= 1) {
>          s->nr_as = 1;
>      }
> @@ -2683,7 +2679,7 @@ static int kvm_init(MachineState *ms)
>      }
>  
>      kvm_readonly_mem_allowed =
> -        (kvm_vm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
> +        (kvm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
>  
>      kvm_resamplefds_allowed =
>          (kvm_check_extension(s, KVM_CAP_IRQFD_RESAMPLE) > 0);
> @@ -2717,7 +2713,8 @@ static int kvm_init(MachineState *ms)
>          goto err;
>      }
>  
> -    kvm_supported_memory_attributes = kvm_vm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
> +    kvm_supported_memory_attributes =
> +        kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
>      kvm_guest_memfd_supported =
>          kvm_check_extension(s, KVM_CAP_GUEST_MEMFD) &&
>          kvm_check_extension(s, KVM_CAP_USER_MEMORY2) &&
> @@ -2743,7 +2740,7 @@ static int kvm_init(MachineState *ms)
>      memory_listener_register(&kvm_io_listener,
>                               &address_space_io);
>  
> -    s->sync_mmu = !!kvm_vm_check_extension(kvm_state, KVM_CAP_SYNC_MMU);
> +    s->sync_mmu = !!kvm_check_extension(kvm_state, KVM_CAP_SYNC_MMU);
>      if (!s->sync_mmu) {
>          ret = ram_block_discard_disable(true);
>          assert(!ret);
> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> index 7b6812c0de..8bdf4abeb6 100644
> --- a/target/arm/kvm.c
> +++ b/target/arm/kvm.c
> @@ -601,7 +601,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>      if (s->kvm_eager_split_size) {
>          uint32_t sizes;
>  
> -        sizes = kvm_vm_check_extension(s, KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES);
> +        sizes = kvm_check_extension(s, KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES);
>          if (!sizes) {
>              s->kvm_eager_split_size = 0;
>              warn_report("Eager Page Split support not available");
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 8e17942c3b..2f35e7468c 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -244,7 +244,7 @@ bool kvm_enable_hypercall(uint64_t enable_mask)
>  
>  bool kvm_has_smm(void)
>  {
> -    return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
> +    return kvm_check_extension(kvm_state, KVM_CAP_X86_SMM);
>  }
>  
>  bool kvm_has_adjust_clock_stable(void)
> @@ -3320,7 +3320,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>          }
>      }
>  
> -    if (kvm_vm_check_extension(s, KVM_CAP_X86_USER_SPACE_MSR)) {
> +    if (kvm_check_extension(s, KVM_CAP_X86_USER_SPACE_MSR)) {
>          ret = kvm_vm_enable_userspace_msr(s);
>          if (ret < 0) {
>              return ret;
> @@ -5936,7 +5936,7 @@ static bool __kvm_enable_sgx_provisioning(KVMState *s)
>  {
>      int fd, ret;
>  
> -    if (!kvm_vm_check_extension(s, KVM_CAP_SGX_ATTRIBUTE)) {
> +    if (!kvm_check_extension(s, KVM_CAP_SGX_ATTRIBUTE)) {
>          return false;
>      }
>  
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 3efc28f18b..8bcb0368ce 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -110,7 +110,7 @@ static uint32_t debug_inst_opcode;
>  static bool kvmppc_is_pr(KVMState *ks)
>  {
>      /* Assume KVM-PR if the GET_PVINFO capability is available */
> -    return kvm_vm_check_extension(ks, KVM_CAP_PPC_GET_PVINFO) != 0;
> +    return kvm_check_extension(ks, KVM_CAP_PPC_GET_PVINFO) != 0;
>  }
>  
>  static int kvm_ppc_register_host_cpu_type(void);
> @@ -127,11 +127,11 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>      cap_interrupt_unset = kvm_check_extension(s, KVM_CAP_PPC_UNSET_IRQ);
>      cap_segstate = kvm_check_extension(s, KVM_CAP_PPC_SEGSTATE);
>      cap_booke_sregs = kvm_check_extension(s, KVM_CAP_PPC_BOOKE_SREGS);
> -    cap_ppc_smt_possible = kvm_vm_check_extension(s, KVM_CAP_PPC_SMT_POSSIBLE);
> +    cap_ppc_smt_possible = kvm_check_extension(s, KVM_CAP_PPC_SMT_POSSIBLE);
>      cap_spapr_tce = kvm_check_extension(s, KVM_CAP_SPAPR_TCE);
>      cap_spapr_tce_64 = kvm_check_extension(s, KVM_CAP_SPAPR_TCE_64);
>      cap_spapr_multitce = kvm_check_extension(s, KVM_CAP_SPAPR_MULTITCE);
> -    cap_spapr_vfio = kvm_vm_check_extension(s, KVM_CAP_SPAPR_TCE_VFIO);
> +    cap_spapr_vfio = kvm_check_extension(s, KVM_CAP_SPAPR_TCE_VFIO);
>      cap_one_reg = kvm_check_extension(s, KVM_CAP_ONE_REG);
>      cap_hior = kvm_check_extension(s, KVM_CAP_PPC_HIOR);
>      cap_epr = kvm_check_extension(s, KVM_CAP_PPC_EPR);
> @@ -140,23 +140,23 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>       * Note: we don't set cap_papr here, because this capability is
>       * only activated after this by kvmppc_set_papr()
>       */
> -    cap_htab_fd = kvm_vm_check_extension(s, KVM_CAP_PPC_HTAB_FD);
> +    cap_htab_fd = kvm_check_extension(s, KVM_CAP_PPC_HTAB_FD);
>      cap_fixup_hcalls = kvm_check_extension(s, KVM_CAP_PPC_FIXUP_HCALL);
> -    cap_ppc_smt = kvm_vm_check_extension(s, KVM_CAP_PPC_SMT);
> -    cap_htm = kvm_vm_check_extension(s, KVM_CAP_PPC_HTM);
> -    cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
> -    cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
> -    cap_xive = kvm_vm_check_extension(s, KVM_CAP_PPC_IRQ_XIVE);
> -    cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
> +    cap_ppc_smt = kvm_check_extension(s, KVM_CAP_PPC_SMT);
> +    cap_htm = kvm_check_extension(s, KVM_CAP_PPC_HTM);
> +    cap_mmu_radix = kvm_check_extension(s, KVM_CAP_PPC_MMU_RADIX);
> +    cap_mmu_hash_v3 = kvm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3);
> +    cap_xive = kvm_check_extension(s, KVM_CAP_PPC_IRQ_XIVE);
> +    cap_resize_hpt = kvm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT);
>      kvmppc_get_cpu_characteristics(s);
> -    cap_ppc_nested_kvm_hv = kvm_vm_check_extension(s, KVM_CAP_PPC_NESTED_HV);
> +    cap_ppc_nested_kvm_hv = kvm_check_extension(s, KVM_CAP_PPC_NESTED_HV);
>      cap_large_decr = kvmppc_get_dec_bits();
> -    cap_fwnmi = kvm_vm_check_extension(s, KVM_CAP_PPC_FWNMI);
> +    cap_fwnmi = kvm_check_extension(s, KVM_CAP_PPC_FWNMI);
>      /*
>       * Note: setting it to false because there is not such capability
>       * in KVM at this moment.
>       *
> -     * TODO: call kvm_vm_check_extension() with the right capability
> +     * TODO: call kvm_check_extension() with the right capability
>       * after the kernel starts implementing it.
>       */
>      cap_ppc_pvr_compat = false;
> @@ -166,8 +166,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>          exit(1);
>      }
>  
> -    cap_rpt_invalidate = kvm_vm_check_extension(s, KVM_CAP_PPC_RPT_INVALIDATE);
> -    cap_ail_mode_3 = kvm_vm_check_extension(s, KVM_CAP_PPC_AIL_MODE_3);
> +    cap_rpt_invalidate = kvm_check_extension(s, KVM_CAP_PPC_RPT_INVALIDATE);
> +    cap_ail_mode_3 = kvm_check_extension(s, KVM_CAP_PPC_AIL_MODE_3);
>      kvm_ppc_register_host_cpu_type();
>  
>      return 0;
> @@ -1976,7 +1976,7 @@ static int kvmppc_get_pvinfo(CPUPPCState *env, struct kvm_ppc_pvinfo *pvinfo)
>  {
>      CPUState *cs = env_cpu(env);
>  
> -    if (kvm_vm_check_extension(cs->kvm_state, KVM_CAP_PPC_GET_PVINFO) &&
> +    if (kvm_check_extension(cs->kvm_state, KVM_CAP_PPC_GET_PVINFO) &&
>          !kvm_vm_ioctl(cs->kvm_state, KVM_PPC_GET_PVINFO, pvinfo)) {
>          return 0;
>      }
> @@ -2298,7 +2298,7 @@ int kvmppc_reset_htab(int shift_hint)
>          /* Full emulation, tell caller to allocate htab itself */
>          return 0;
>      }
> -    if (kvm_vm_check_extension(kvm_state, KVM_CAP_PPC_ALLOC_HTAB)) {
> +    if (kvm_check_extension(kvm_state, KVM_CAP_PPC_ALLOC_HTAB)) {
>          int ret;
>          ret = kvm_vm_ioctl(kvm_state, KVM_PPC_ALLOCATE_HTAB, &shift);
>          if (ret == -ENOTTY) {
> @@ -2507,7 +2507,7 @@ static void kvmppc_get_cpu_characteristics(KVMState *s)
>      cap_ppc_safe_bounds_check = 0;
>      cap_ppc_safe_indirect_branch = 0;
>  
> -    ret = kvm_vm_check_extension(s, KVM_CAP_PPC_GET_CPU_CHAR);
> +    ret = kvm_check_extension(s, KVM_CAP_PPC_GET_CPU_CHAR);
>      if (!ret) {
>          return;
>      }
> -- 
> 2.47.0
> 
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v3 02/26] target/arm: Add confidential guest support

[Daniel P. Berrangé]

On Mon, Nov 25, 2024 at 07:56:01PM +0000, Jean-Philippe Brucker wrote:
> Add a new RmeGuest object, inheriting from ConfidentialGuestSupport, to
> support the Arm Realm Management Extension (RME). It is instantiated by
> passing on the command-line:
> 
>   -M virt,confidential-guest-support=<id>
>   -object rme-guest,id=<id>[,options...]
> 
> This is only the skeleton. Support will be added in following patches.
> 
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> Cc: Eduardo Habkost <eduardo@habkost.net>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: remove some boilerplate with OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES
> ---
>  docs/system/confidential-guest-support.rst |  1 +
>  target/arm/kvm-rme.c                       | 40 ++++++++++++++++++++++
>  target/arm/meson.build                     |  7 +++-
>  3 files changed, 47 insertions(+), 1 deletion(-)
>  create mode 100644 target/arm/kvm-rme.c
> 
> diff --git a/docs/system/confidential-guest-support.rst b/docs/system/confidential-guest-support.rst
> index 0c490dbda2..acf46d8856 100644
> --- a/docs/system/confidential-guest-support.rst
> +++ b/docs/system/confidential-guest-support.rst
> @@ -40,5 +40,6 @@ Currently supported confidential guest mechanisms are:
>  * AMD Secure Encrypted Virtualization (SEV) (see :doc:`i386/amd-memory-encryption`)
>  * POWER Protected Execution Facility (PEF) (see :ref:`power-papr-protected-execution-facility-pef`)
>  * s390x Protected Virtualization (PV) (see :doc:`s390x/protvirt`)
> +* Arm Realm Management Extension (RME)
>  
>  Other mechanisms may be supported in future.
> diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
> new file mode 100644
> index 0000000000..67909349c1
> --- /dev/null
> +++ b/target/arm/kvm-rme.c
> @@ -0,0 +1,40 @@
> +/*
> + * QEMU Arm RME support
> + *
> + * Copyright Linaro 2024
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "exec/confidential-guest-support.h"
> +#include "hw/boards.h"
> +#include "hw/core/cpu.h"
> +#include "kvm_arm.h"
> +#include "migration/blocker.h"
> +#include "qapi/error.h"
> +#include "qom/object_interfaces.h"
> +#include "sysemu/kvm.h"
> +#include "sysemu/runstate.h"
> +
> +#define TYPE_RME_GUEST "rme-guest"
> +OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
> +
> +struct RmeGuest {
> +    ConfidentialGuestSupport parent_obj;
> +};
> +
> +OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
> +                                          CONFIDENTIAL_GUEST_SUPPORT,
> +                                          { TYPE_USER_CREATABLE }, { })
> +
> +static void rme_guest_class_init(ObjectClass *oc, void *data)
> +{
> +}
> +
> +static void rme_guest_init(Object *obj)
> +{
> +}
> +
> +static void rme_guest_finalize(Object *obj)
> +{
> +}

When you create a new "user creatable" object, there should also be a
corresponding addition to qapi/qom.json.

I believe you have the qom.json addition in a later patch, but it
should actually be here 


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v3 10/26] target/arm/kvm-rme: Add Realm Personalization Value parameter

[Daniel P. Berrangé]

On Tue, Nov 26, 2024 at 08:20:42AM +0100, Markus Armbruster wrote:
> Jean-Philippe Brucker <jean-philippe@linaro.org> writes:
> 
> > The Realm Personalization Value (RPV) is provided by the user to
> > distinguish Realms that have the same initial measurement.
> >
> > The user provides up to 64 hexadecimal bytes. They are stored into the
> > RPV in the same order, zero-padded on the right.
> >
> > Cc: Eric Blake <eblake@redhat.com>
> > Cc: Markus Armbruster <armbru@redhat.com>
> > Cc: Daniel P. Berrangé <berrange@redhat.com>
> > Cc: Eduardo Habkost <eduardo@habkost.net>
> > Acked-by: Markus Armbruster <armbru@redhat.com>
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > v2->v3: Fix documentation
> > ---
> >  qapi/qom.json        |  15 ++++++
> >  target/arm/kvm-rme.c | 111 +++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 126 insertions(+)
> >
> > diff --git a/qapi/qom.json b/qapi/qom.json
> > index a8beeabf1f..f982850bca 100644
> > --- a/qapi/qom.json
> > +++ b/qapi/qom.json
> > @@ -1068,6 +1068,19 @@
> >    'data': { '*cpu-affinity': ['uint16'],
> >              '*node-affinity': ['uint16'] } }
> >  
> > +##
> > +# @RmeGuestProperties:
> > +#
> > +# Properties for rme-guest objects.
> > +#
> > +# @personalization-value: Realm personalization value, as a 64-byte
> > +#     hex string. This optional parameter allows to uniquely identify
> > +#     the VM instance during attestation. (default: 0)
> 
> QMP commonly uses base64 for encoding binary data.  Any particular
> reason to deviate?
> 
> Is the "hex string" to be mapped to binary in little or big endian?  Not
> an issue with base64.

Agreed, using base64 is preferrable for consistency.

> 
> Nitpick: (default: all-zero), please, for consistency with similar
> documentation in SevSnpGuestProperties.
> 
> > +#
> > +# Since: 9.3

There is never any x.3 version of QEMU, as we bump the major
version once a year. IOW, next release you could target this
for will be 10.0

> > +##
> > +{ 'struct': 'RmeGuestProperties',
> > +  'data': { '*personalization-value': 'str' } }
> >  
> >  ##
> >  # @ObjectType:
> > @@ -1121,6 +1134,7 @@
> >      { 'name': 'pr-manager-helper',
> >        'if': 'CONFIG_LINUX' },
> >      'qtest',
> > +    'rme-guest',
> >      'rng-builtin',
> >      'rng-egd',
> >      { 'name': 'rng-random',
> 
> The commit message claims the patch adds a parameter.  It actually adds
> an entire object type.

The object should be added to qom.json earlier in this series when
the RmeGuest class is defined, then this patch would merely be adding
the parameter.

> 
> > @@ -1195,6 +1209,7 @@
> >        'pr-manager-helper':          { 'type': 'PrManagerHelperProperties',
> >                                        'if': 'CONFIG_LINUX' },
> >        'qtest':                      'QtestProperties',
> > +      'rme-guest':                  'RmeGuestProperties',
> >        'rng-builtin':                'RngProperties',
> >        'rng-egd':                    'RngEgdProperties',
> >        'rng-random':                 { 'type': 'RngRandomProperties',

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v3 11/26] target/arm/kvm-rme: Add measurement algorithm property

[Daniel P. Berrangé]

On Mon, Nov 25, 2024 at 07:56:10PM +0000, Jean-Philippe Brucker wrote:
> This option selects which measurement algorithm to use for attestation.
> Supported values are SHA256 and SHA512. Default to SHA512 arbitrarily.

I'd suggest that defaulting to sha256 is the better choice.

sha512 is overkill from a security POV, while doubling the size
of hash data that has to be passed around. It is unusual for
applications needing hashing to pick 512 over 256 IME.

Shorter config param data is a win for QEMU command line parameters.

> SHA512 is generally faster on 64-bit architectures. On a few arm64 CPUs
> I tested SHA256 is much faster, but that's most likely because they only
> support acceleration via FEAT_SHA256 (Armv8.0) and not FEAT_SHA512
> (Armv8.2). Future CPUs supporting RME are likely to also support
> FEAT_SHA512.

IMHO speed is largely tangential for this use case, as it is a
one time hash operation at VM startup.

> Cc: Eric Blake <eblake@redhat.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> Cc: Eduardo Habkost <eduardo@habkost.net>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: Rename measurement-algorithm
> ---
>  qapi/qom.json        | 20 +++++++++++++++++++-
>  target/arm/kvm-rme.c | 39 ++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 57 insertions(+), 2 deletions(-)
> 
> diff --git a/qapi/qom.json b/qapi/qom.json
> index f982850bca..901ba67634 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -1068,6 +1068,20 @@
>    'data': { '*cpu-affinity': ['uint16'],
>              '*node-affinity': ['uint16'] } }
>  
> +##
> +# @RmeGuestMeasurementAlgorithm:
> +#
> +# @sha256: Use the SHA256 algorithm
> +#
> +# @sha512: Use the SHA512 algorithm
> +#
> +# Algorithm to use for realm measurements
> +#
> +# Since: 9.3
> +##
> +{ 'enum': 'RmeGuestMeasurementAlgorithm',
> +  'data': ['sha256', 'sha512'] }


A design question for Markus....


We have a "QCryptoHashAlg" enum that covers both of these strings
and more besides.

We have a choice of using a dedicated enum strictly limited to
just what RME needs, vs using the common enum type, but rejecting
unsupported algorithms at runtime.  Do we prefer duplication for
improve specificity, or removal of duplication ?

> +
>  ##
>  # @RmeGuestProperties:
>  #
> @@ -1077,10 +1091,14 @@
>  #     hex string. This optional parameter allows to uniquely identify
>  #     the VM instance during attestation. (default: 0)
>  #
> +# @measurement-algorithm: Realm measurement algorithm
> +#     (default: sha512)
> +#
>  # Since: 9.3
>  ##
>  { 'struct': 'RmeGuestProperties',
> -  'data': { '*personalization-value': 'str' } }
> +  'data': { '*personalization-value': 'str',
> +            '*measurement-algorithm': 'RmeGuestMeasurementAlgorithm' } }
>  

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Daniel P. Berrangé]

On Mon, Nov 25, 2024 at 05:23:44PM -0500, Stefan Berger wrote:
> 
> 
> On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> > Create an event log, in the format defined by Trusted Computing Group
> > for TPM2. It contains information about the VMM, the Realm parameters,
> > any data loaded into guest memory before boot and the initial vCPU
> > state.
> > 
> > The guest can access this log from RAM and send it to a verifier, to
> > help the verifier independently compute the Realm Initial Measurement,
> > and check that the data we load into guest RAM is known-good images.
> > Without this log, the verifier has to guess where everything is loaded>
> and in what order.
> 
> Typically these logs are backed by extensions of TPM PCRs and when you send
> a log to a verifier you send a TPM quote along with it for the verifer to
> replay the log and check the TPM quote. Also, early code in the firmware is
> typically serving as a root of trust that starts the chain of measurements
> of code and data, first measuring itself and then other parts of the
> firmware before it jumps into the other parts. Now here you seem to just
> have a log and no PCR extensions and therefore no quote over PCRs can be
> used. Then what prevents anyone from faking this log and presenting a
> completely fake log to the verifier?

In addition, a measurement log is just one of the interesting features
that a TPM provides to OS. The other TPM features are still relevant
and useful to confidential VMs.

As a high level goal I think we should be aiming to make it possible for
users to move their existing VM workloads from non-confidentail to
confidential environments, simply as a choice at deployment time. To make
this as practical as possible, confidential VMs  need to be aiming to
match non-confidential VM features where ever it is practical to do so.
Users & vendors should not need to build & carry around 2 sets of disk
images - one setup for confidential and one setup for non-confidential.
Following existing standards will reduce the work both for OS developers,
app developers and users alike, to adopt the CVM world.

IOW, this is a long winded way of saying that we should be looking to
provide a complete *standards compliant*, trusted TPM implementation to
confidential VMs, not providing a cherry-picked selection of a few
TPM-like features.

On the x86 side of things, the route to providing a trusted TPM is via
SVSM, both for SNP and TDX. Microsoft's recently open sources openhcl
similarly provides a st

I don't know so much about RME. Is providing a trusted TPM a job for
the RMM ?

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v3 11/26] target/arm/kvm-rme: Add measurement algorithm property

[Markus Armbruster]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Mon, Nov 25, 2024 at 07:56:10PM +0000, Jean-Philippe Brucker wrote:

[...]

>> diff --git a/qapi/qom.json b/qapi/qom.json
>> index f982850bca..901ba67634 100644
>> --- a/qapi/qom.json
>> +++ b/qapi/qom.json
>> @@ -1068,6 +1068,20 @@
>>    'data': { '*cpu-affinity': ['uint16'],
>>              '*node-affinity': ['uint16'] } }
>>  
>> +##
>> +# @RmeGuestMeasurementAlgorithm:
>> +#
>> +# @sha256: Use the SHA256 algorithm
>> +#
>> +# @sha512: Use the SHA512 algorithm
>> +#
>> +# Algorithm to use for realm measurements
>> +#
>> +# Since: 9.3
>> +##
>> +{ 'enum': 'RmeGuestMeasurementAlgorithm',
>> +  'data': ['sha256', 'sha512'] }
>
>
> A design question for Markus....
>
>
> We have a "QCryptoHashAlg" enum that covers both of these strings
> and more besides.
>
> We have a choice of using a dedicated enum strictly limited to
> just what RME needs, vs using the common enum type, but rejecting
> unsupported algorithms at runtime.  Do we prefer duplication for
> improve specificity, or removal of duplication ?

With a dedicated enum, introspection shows precisely the accepted
values.

If we reuse a wider enum, introspection shows additional, invalid
values.

Say we later make one of these valid here.  If we reuse the wider enum
now, nothing changes in introspection then, i.e. introspection cannot
tell us whether this particular QEMU supports this additional algorithm.
With a dedicated enum, it can.  Whether that's needed in practice I find
hard to predict.

I lean towards lean towards dedicated enum.

Questions?

[...]

Re: [PATCH v3 11/26] target/arm/kvm-rme: Add measurement algorithm property

[Daniel P. Berrangé]

On Tue, Nov 26, 2024 at 04:11:19PM +0100, Markus Armbruster wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > On Mon, Nov 25, 2024 at 07:56:10PM +0000, Jean-Philippe Brucker wrote:
> 
> [...]
> 
> >> diff --git a/qapi/qom.json b/qapi/qom.json
> >> index f982850bca..901ba67634 100644
> >> --- a/qapi/qom.json
> >> +++ b/qapi/qom.json
> >> @@ -1068,6 +1068,20 @@
> >>    'data': { '*cpu-affinity': ['uint16'],
> >>              '*node-affinity': ['uint16'] } }
> >>  
> >> +##
> >> +# @RmeGuestMeasurementAlgorithm:
> >> +#
> >> +# @sha256: Use the SHA256 algorithm
> >> +#
> >> +# @sha512: Use the SHA512 algorithm
> >> +#
> >> +# Algorithm to use for realm measurements
> >> +#
> >> +# Since: 9.3
> >> +##
> >> +{ 'enum': 'RmeGuestMeasurementAlgorithm',
> >> +  'data': ['sha256', 'sha512'] }
> >
> >
> > A design question for Markus....
> >
> >
> > We have a "QCryptoHashAlg" enum that covers both of these strings
> > and more besides.
> >
> > We have a choice of using a dedicated enum strictly limited to
> > just what RME needs, vs using the common enum type, but rejecting
> > unsupported algorithms at runtime.  Do we prefer duplication for
> > improve specificity, or removal of duplication ?
> 
> With a dedicated enum, introspection shows precisely the accepted
> values.
> 
> If we reuse a wider enum, introspection shows additional, invalid
> values.
> 
> Say we later make one of these valid here.  If we reuse the wider enum
> now, nothing changes in introspection then, i.e. introspection cannot
> tell us whether this particular QEMU supports this additional algorithm.
> With a dedicated enum, it can.  Whether that's needed in practice I find
> hard to predict.
> 
> I lean towards lean towards dedicated enum.
> 
> Questions?

That's fine with me. Lets stick with the approach in this patch.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Jean-Philippe Brucker]

On Tue, Nov 26, 2024 at 01:45:55PM +0000, Daniel P. Berrangé wrote:
> On Mon, Nov 25, 2024 at 05:23:44PM -0500, Stefan Berger wrote:
> > 
> > 
> > On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> > > Create an event log, in the format defined by Trusted Computing Group
> > > for TPM2. It contains information about the VMM, the Realm parameters,
> > > any data loaded into guest memory before boot and the initial vCPU
> > > state.
> > > 
> > > The guest can access this log from RAM and send it to a verifier, to
> > > help the verifier independently compute the Realm Initial Measurement,
> > > and check that the data we load into guest RAM is known-good images.
> > > Without this log, the verifier has to guess where everything is loaded>
> > and in what order.
> > 
> > Typically these logs are backed by extensions of TPM PCRs and when you send
> > a log to a verifier you send a TPM quote along with it for the verifer to
> > replay the log and check the TPM quote. Also, early code in the firmware is
> > typically serving as a root of trust that starts the chain of measurements
> > of code and data, first measuring itself and then other parts of the
> > firmware before it jumps into the other parts. Now here you seem to just
> > have a log and no PCR extensions and therefore no quote over PCRs can be
> > used.

Indeed, in our case it's the trusted hypervisor (RMM) that provides the
equivalent to TPM quote and PCRs. In more details:

1. QEMU loads images into guest RAM by calling KVM, which calls RMM.
2. RMM calculates a hash of the image content, adds it to a rolling hash
   the "Realm Initial Measurement" (RIM), which I believe is equivalent to
   a PCR.
3. During remote attestation, the guest sends evidence containing this RIM
   signed by the root of trust, along with a signed token identifying the
   platform (hardware, firmware, RMM).
4. The verifier checks the signature and the platform token, so it trusts
   the RMM and the RIM.

> > Then what prevents anyone from faking this log and presenting a
> > completely fake log to the verifier?

Absolutely, the verifier does not trust the content of the log, it only
uses the log as helper to try to reconstruct the RIM. For example a log
event says "I loaded image XYZ at address A", then the verifier searches
image XYZ in its database of known-good images, calculates the hash that
would result from loading that image at address A. Any malformed event in
the log causes the hash to diverge from the trusted RIM value, and causes
an attestation error.

> In addition, a measurement log is just one of the interesting features
> that a TPM provides to OS. The other TPM features are still relevant
> and useful to confidential VMs.
> 
> As a high level goal I think we should be aiming to make it possible for
> users to move their existing VM workloads from non-confidentail to
> confidential environments, simply as a choice at deployment time. To make
> this as practical as possible, confidential VMs  need to be aiming to
> match non-confidential VM features where ever it is practical to do so.
> Users & vendors should not need to build & carry around 2 sets of disk
> images - one setup for confidential and one setup for non-confidential.
> Following existing standards will reduce the work both for OS developers,
> app developers and users alike, to adopt the CVM world.
> 
> IOW, this is a long winded way of saying that we should be looking to
> provide a complete *standards compliant*, trusted TPM implementation to
> confidential VMs, not providing a cherry-picked selection of a few
> TPM-like features.
> 
> On the x86 side of things, the route to providing a trusted TPM is via
> SVSM, both for SNP and TDX. Microsoft's recently open sources openhcl
> similarly provides a st
> 
> I don't know so much about RME. Is providing a trusted TPM a job for
> the RMM ?

Not directly, but I've heard of at least two options that are being
actively worked on:
* running payloads like SVSM and openhcl that emulate a TPM. In RMM 1.1
  there is a concept of "planes" that enables this.
* having edk2 in the VM provide a "fTPM".

I'm less familiar with these, but I think both need to connect the virtual
TPM to the root of trust by performing remote attestation via RMM (?). So
the problem of reconstructing a RIM on the verifier side remains, even if
it's not done by the application.

In addition I'm wondering about lighter, container-like workloads, which
will want to boot quickly and run the bare minimum software, and where
edk2 or SVSM seems superfluous. Maybe people will want to tailor those
workloads and avoid the extra layer?

Thanks,
Jean

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Stefan Berger]

On 11/26/24 11:21 AM, Jean-Philippe Brucker wrote:
> On Tue, Nov 26, 2024 at 01:45:55PM +0000, Daniel P. Berrangé wrote:
>> On Mon, Nov 25, 2024 at 05:23:44PM -0500, Stefan Berger wrote:
>>>
>>>
>>> On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
>>>> Create an event log, in the format defined by Trusted Computing Group
>>>> for TPM2. It contains information about the VMM, the Realm parameters,
>>>> any data loaded into guest memory before boot and the initial vCPU
>>>> state.
>>>>
>>>> The guest can access this log from RAM and send it to a verifier, to
>>>> help the verifier independently compute the Realm Initial Measurement,
>>>> and check that the data we load into guest RAM is known-good images.
>>>> Without this log, the verifier has to guess where everything is loaded>
>>> and in what order.
>>>
>>> Typically these logs are backed by extensions of TPM PCRs and when you send
>>> a log to a verifier you send a TPM quote along with it for the verifer to
>>> replay the log and check the TPM quote. Also, early code in the firmware is
>>> typically serving as a root of trust that starts the chain of measurements
>>> of code and data, first measuring itself and then other parts of the
>>> firmware before it jumps into the other parts. Now here you seem to just
>>> have a log and no PCR extensions and therefore no quote over PCRs can be
>>> used.
> 
> Indeed, in our case it's the trusted hypervisor (RMM) that provides the
> equivalent to TPM quote and PCRs. In more details:
> 
> 1. QEMU loads images into guest RAM by calling KVM, which calls RMM.
> 2. RMM calculates a hash of the image content, adds it to a rolling hash
>     the "Realm Initial Measurement" (RIM), which I believe is equivalent to
>     a PCR.

I am not familiar with RIM. A link to read more about it would be helpful.

> 3. During remote attestation, the guest sends evidence containing this RIM
>     signed by the root of trust, along with a signed token identifying the
>     platform (hardware, firmware, RMM).

Is this a well known manufacturer key that one would expect for 
signature verification or is it locally created?

> 4. The verifier checks the signature and the platform token, so it trusts
>     the RMM and the RIM.
> 
>>> Then what prevents anyone from faking this log and presenting a
>>> completely fake log to the verifier?
> 
> Absolutely, the verifier does not trust the content of the log, it only
> uses the log as helper to try to reconstruct the RIM. For example a log
> event says "I loaded image XYZ at address A", then the verifier searches
> image XYZ in its database of known-good images, calculates the hash that
> would result from loading that image at address A. Any malformed event in

Hopefully just calculating a hash over the image will do and the 
location an image was loaded to, like address A (relocation?), doesn't 
matter...

Re: [PATCH v3 01/26] kvm: Merge kvm_check_extension() and kvm_vm_check_extension()

[Jean-Philippe Brucker]

On Tue, Nov 26, 2024 at 12:29:35PM +0000, Daniel P. Berrangé wrote:
> On Mon, Nov 25, 2024 at 07:56:00PM +0000, Jean-Philippe Brucker wrote:
> > The KVM_CHECK_EXTENSION ioctl can be issued either on the global fd
> > (/dev/kvm), or on the VM fd obtained with KVM_CREATE_VM. For most
> > extensions, KVM returns the same value with either method, but for some
> > of them it can refine the returned value depending on the VM type. The
> > KVM documentation [1] advises to use the VM fd:
> > 
> >   Based on their initialization different VMs may have different
> >   capabilities. It is thus encouraged to use the vm ioctl to query for
> >   capabilities (available with KVM_CAP_CHECK_EXTENSION_VM on the vm fd)
> > 
> > Ongoing work on Arm confidential VMs confirms this, as some capabilities
> > become unavailable to confidential VMs, requiring changes in QEMU to use
> > kvm_vm_check_extension() instead of kvm_check_extension() [2]. Rather
> > than changing each check one by one, change kvm_check_extension() to
> > always issue the ioctl on the VM fd when available, and remove
> > kvm_vm_check_extension().
> 
> The downside I see of this approach is that it can potentially
> mask mistakes / unexpected behaviour.
> 
> eg, consider you are in a code path where you /think/ the VM fd
> is available, but for some unexpected reason it is NOT in fact
> available. The code silently falls back to the global FD, thus
> giving a potentially incorrect extension check answer.
> 
> Having separate check methods with no fallback ensures that we
> are checking exactly what we /intend/ to be checking, or will
> see an error

Yes I see your point, and I'm happy dropping this patch since I'm less
familiar with the other archs.

The alternative is replacing kvm_check_extension() with
kvm_vm_check_extension() wherever the Arm ioctl handler behaves
differently depending on the VM type. Simple enough though it does affect
kvm-all.c too:

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 801cff16a5..a56b943f31 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2410,13 +2410,13 @@ static int kvm_recommended_vcpus(KVMState *s)
 
 static int kvm_max_vcpus(KVMState *s)
 {
-    int ret = kvm_check_extension(s, KVM_CAP_MAX_VCPUS);
+    int ret = kvm_vm_check_extension(s, KVM_CAP_MAX_VCPUS);
     return (ret) ? ret : kvm_recommended_vcpus(s);
 }
 
 static int kvm_max_vcpu_id(KVMState *s)
 {
-    int ret = kvm_check_extension(s, KVM_CAP_MAX_VCPU_ID);
+    int ret = kvm_vm_check_extension(s, KVM_CAP_MAX_VCPU_ID);
     return (ret) ? ret : kvm_max_vcpus(s);
 }
 
@@ -2693,7 +2693,7 @@ static int kvm_init(MachineState *ms)
 
 #ifdef TARGET_KVM_HAVE_GUEST_DEBUG
     kvm_has_guest_debug =
-        (kvm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG) > 0);
+        (kvm_vm_check_extension(s, KVM_CAP_SET_GUEST_DEBUG) > 0);
 #endif
 
     kvm_sstep_flags = 0;
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 7b6812c0de..609c6d4e7a 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -618,11 +618,11 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
         }
     }
 
-    max_hw_wps = kvm_check_extension(s, KVM_CAP_GUEST_DEBUG_HW_WPS);
+    max_hw_wps = kvm_vm_check_extension(s, KVM_CAP_GUEST_DEBUG_HW_WPS);
     hw_watchpoints = g_array_sized_new(true, true,
                                        sizeof(HWWatchpoint), max_hw_wps);
 
-    max_hw_bps = kvm_check_extension(s, KVM_CAP_GUEST_DEBUG_HW_BPS);
+    max_hw_bps = kvm_vm_check_extension(s, KVM_CAP_GUEST_DEBUG_HW_BPS);
     hw_breakpoints = g_array_sized_new(true, true,
                                        sizeof(HWBreakpoint), max_hw_bps);
 
@@ -1764,7 +1764,7 @@ void kvm_arm_pvtime_init(ARMCPU *cpu, uint64_t ipa)
 
 void kvm_arm_steal_time_finalize(ARMCPU *cpu, Error **errp)
 {
-    bool has_steal_time = kvm_check_extension(kvm_state, KVM_CAP_STEAL_TIME);
+    bool has_steal_time = kvm_vm_check_extension(kvm_state, KVM_CAP_STEAL_TIME);
 
     if (cpu->kvm_steal_time == ON_OFF_AUTO_AUTO) {
         if (!has_steal_time || !arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
@@ -1799,7 +1799,7 @@ bool kvm_arm_aarch32_supported(void)
 
 bool kvm_arm_sve_supported(void)
 {
-    return kvm_check_extension(kvm_state, KVM_CAP_ARM_SVE);
+    return kvm_vm_check_extension(kvm_state, KVM_CAP_ARM_SVE);
 }
 
 bool kvm_arm_mte_supported(void)

Re: [PATCH v3 02/26] target/arm: Add confidential guest support

[Jean-Philippe Brucker]

On Tue, Nov 26, 2024 at 12:37:02PM +0000, Daniel P. Berrangé wrote:
> When you create a new "user creatable" object, there should also be a
> corresponding addition to qapi/qom.json.
> 
> I believe you have the qom.json addition in a later patch, but it
> should actually be here 

Yes, I'll move it here

Thanks,
Jean

Re: [PATCH v3 10/26] target/arm/kvm-rme: Add Realm Personalization Value parameter

[Jean-Philippe Brucker]

On Tue, Nov 26, 2024 at 08:20:42AM +0100, Markus Armbruster wrote:
> > +# @personalization-value: Realm personalization value, as a 64-byte
> > +#     hex string. This optional parameter allows to uniquely identify
> > +#     the VM instance during attestation. (default: 0)
> 
> QMP commonly uses base64 for encoding binary data.  Any particular
> reason to deviate?

No, I think base64 is fine for this

> 
> Is the "hex string" to be mapped to binary in little or big endian?  Not
> an issue with base64.

(It was big endian with padding on the right)

> 
> Nitpick: (default: all-zero), please, for consistency with similar
> documentation in SevSnpGuestProperties.
> 
> > +#
> > +# Since: 9.3
> > +##
> > +{ 'struct': 'RmeGuestProperties',
> > +  'data': { '*personalization-value': 'str' } }
> >  
> >  ##
> >  # @ObjectType:
> > @@ -1121,6 +1134,7 @@
> >      { 'name': 'pr-manager-helper',
> >        'if': 'CONFIG_LINUX' },
> >      'qtest',
> > +    'rme-guest',
> >      'rng-builtin',
> >      'rng-egd',
> >      { 'name': 'rng-random',
> 
> The commit message claims the patch adds a parameter.  It actually adds
> an entire object type.

Yes as Daniel noted this belongs in an earlier patch

> 
> > @@ -1195,6 +1209,7 @@
> >        'pr-manager-helper':          { 'type': 'PrManagerHelperProperties',
> >                                        'if': 'CONFIG_LINUX' },
> >        'qtest':                      'QtestProperties',
> > +      'rme-guest':                  'RmeGuestProperties',
> >        'rng-builtin':                'RngProperties',
> >        'rng-egd':                    'RngEgdProperties',
> >        'rng-random':                 { 'type': 'RngRandomProperties',
> > diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
> > index 83a29421df..0be55867ee 100644
> > --- a/target/arm/kvm-rme.c
> > +++ b/target/arm/kvm-rme.c
> > @@ -23,11 +23,15 @@ OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
> >  
> >  #define RME_PAGE_SIZE qemu_real_host_page_size()
> >  
> > +#define RME_MAX_CFG         1
> > +
> >  struct RmeGuest {
> >      ConfidentialGuestSupport parent_obj;
> >      Notifier rom_load_notifier;
> >      GSList *ram_regions;
> >  
> > +    uint8_t *personalization_value;
> > +
> >      hwaddr ram_base;
> >      size_t ram_size;
> >  };
> > @@ -43,6 +47,48 @@ typedef struct {
> >  
> >  static RmeGuest *rme_guest;
> >  
> > +static int rme_configure_one(RmeGuest *guest, uint32_t cfg, Error **errp)
> > +{
> > +    int ret;
> > +    const char *cfg_str;
> > +    struct kvm_cap_arm_rme_config_item args = {
> > +        .cfg = cfg,
> > +    };
> > +
> > +    switch (cfg) {
> > +    case KVM_CAP_ARM_RME_CFG_RPV:
> > +        if (!guest->personalization_value) {
> > +            return 0;
> > +        }
> > +        memcpy(args.rpv, guest->personalization_value, KVM_CAP_ARM_RME_RPV_SIZE);
> > +        cfg_str = "personalization value";
> > +        break;
> > +    default:
> > +        g_assert_not_reached();
> > +    }
> 
> This function gets called with @cfg arguments 0 .. RME_MAX_CFG-1, from
> rme_configure() right below.  RME_MAX_CFG is defined to 1 above.
> 
> The switch assumes KVM_CAP_ARM_RME_CFG_RPV is zero.  Such assumptions
> are ideally avoided, and alternatively checked at build time.

Ok, I'll change this

Thanks,
Jean

Re: [PATCH v3 10/26] target/arm/kvm-rme: Add Realm Personalization Value parameter

[Jean-Philippe Brucker]

On Tue, Nov 26, 2024 at 12:47:59PM +0000, Daniel P. Berrangé wrote:
> On Tue, Nov 26, 2024 at 08:20:42AM +0100, Markus Armbruster wrote:
> > Jean-Philippe Brucker <jean-philippe@linaro.org> writes:
> > 
> > > The Realm Personalization Value (RPV) is provided by the user to
> > > distinguish Realms that have the same initial measurement.
> > >
> > > The user provides up to 64 hexadecimal bytes. They are stored into the
> > > RPV in the same order, zero-padded on the right.
> > >
> > > Cc: Eric Blake <eblake@redhat.com>
> > > Cc: Markus Armbruster <armbru@redhat.com>
> > > Cc: Daniel P. Berrangé <berrange@redhat.com>
> > > Cc: Eduardo Habkost <eduardo@habkost.net>
> > > Acked-by: Markus Armbruster <armbru@redhat.com>
> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > ---
> > > v2->v3: Fix documentation
> > > ---
> > >  qapi/qom.json        |  15 ++++++
> > >  target/arm/kvm-rme.c | 111 +++++++++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 126 insertions(+)
> > >
> > > diff --git a/qapi/qom.json b/qapi/qom.json
> > > index a8beeabf1f..f982850bca 100644
> > > --- a/qapi/qom.json
> > > +++ b/qapi/qom.json
> > > @@ -1068,6 +1068,19 @@
> > >    'data': { '*cpu-affinity': ['uint16'],
> > >              '*node-affinity': ['uint16'] } }
> > >  
> > > +##
> > > +# @RmeGuestProperties:
> > > +#
> > > +# Properties for rme-guest objects.
> > > +#
> > > +# @personalization-value: Realm personalization value, as a 64-byte
> > > +#     hex string. This optional parameter allows to uniquely identify
> > > +#     the VM instance during attestation. (default: 0)
> > 
> > QMP commonly uses base64 for encoding binary data.  Any particular
> > reason to deviate?
> > 
> > Is the "hex string" to be mapped to binary in little or big endian?  Not
> > an issue with base64.
> 
> Agreed, using base64 is preferrable for consistency.

Ack

> 
> > 
> > Nitpick: (default: all-zero), please, for consistency with similar
> > documentation in SevSnpGuestProperties.
> > 
> > > +#
> > > +# Since: 9.3
> 
> There is never any x.3 version of QEMU, as we bump the major
> version once a year. IOW, next release you could target this
> for will be 10.0

Ok good to know

Thanks,
Jean

> 
> > > +##
> > > +{ 'struct': 'RmeGuestProperties',
> > > +  'data': { '*personalization-value': 'str' } }
> > >  
> > >  ##
> > >  # @ObjectType:
> > > @@ -1121,6 +1134,7 @@
> > >      { 'name': 'pr-manager-helper',
> > >        'if': 'CONFIG_LINUX' },
> > >      'qtest',
> > > +    'rme-guest',
> > >      'rng-builtin',
> > >      'rng-egd',
> > >      { 'name': 'rng-random',
> > 
> > The commit message claims the patch adds a parameter.  It actually adds
> > an entire object type.
> 
> The object should be added to qom.json earlier in this series when
> the RmeGuest class is defined, then this patch would merely be adding
> the parameter.
> 
> > 
> > > @@ -1195,6 +1209,7 @@
> > >        'pr-manager-helper':          { 'type': 'PrManagerHelperProperties',
> > >                                        'if': 'CONFIG_LINUX' },
> > >        'qtest':                      'QtestProperties',
> > > +      'rme-guest':                  'RmeGuestProperties',
> > >        'rng-builtin':                'RngProperties',
> > >        'rng-egd':                    'RngEgdProperties',
> > >        'rng-random':                 { 'type': 'RngRandomProperties',
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> 

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Jean-Philippe Brucker]

On Mon, Dec 02, 2024 at 10:58:01AM -0500, Stefan Berger wrote:
> 
> 
> On 11/26/24 11:21 AM, Jean-Philippe Brucker wrote:
> > On Tue, Nov 26, 2024 at 01:45:55PM +0000, Daniel P. Berrangé wrote:
> > > On Mon, Nov 25, 2024 at 05:23:44PM -0500, Stefan Berger wrote:
> > > > 
> > > > 
> > > > On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> > > > > Create an event log, in the format defined by Trusted Computing Group
> > > > > for TPM2. It contains information about the VMM, the Realm parameters,
> > > > > any data loaded into guest memory before boot and the initial vCPU
> > > > > state.
> > > > > 
> > > > > The guest can access this log from RAM and send it to a verifier, to
> > > > > help the verifier independently compute the Realm Initial Measurement,
> > > > > and check that the data we load into guest RAM is known-good images.
> > > > > Without this log, the verifier has to guess where everything is loaded>
> > > > and in what order.
> > > > 
> > > > Typically these logs are backed by extensions of TPM PCRs and when you send
> > > > a log to a verifier you send a TPM quote along with it for the verifer to
> > > > replay the log and check the TPM quote. Also, early code in the firmware is
> > > > typically serving as a root of trust that starts the chain of measurements
> > > > of code and data, first measuring itself and then other parts of the
> > > > firmware before it jumps into the other parts. Now here you seem to just
> > > > have a log and no PCR extensions and therefore no quote over PCRs can be
> > > > used.
> > 
> > Indeed, in our case it's the trusted hypervisor (RMM) that provides the
> > equivalent to TPM quote and PCRs. In more details:
> > 
> > 1. QEMU loads images into guest RAM by calling KVM, which calls RMM.
> > 2. RMM calculates a hash of the image content, adds it to a rolling hash
> >     the "Realm Initial Measurement" (RIM), which I believe is equivalent to
> >     a PCR.
> 
> I am not familiar with RIM. A link to read more about it would be helpful.

The "Learn the architecture" documentation might be a good introduction
https://developer.arm.com/documentation/den0127/0200/Overview
In particular the part about Realm creation:
https://developer.arm.com/documentation/den0127/0200/Realm-management/Realm-creation-and-attestation

The RMM specification describes exactly how the RIM is calculated, but
is less palatable:
https://developer.arm.com/documentation/den0137/1-0rel0/?lang=en
A7.1.1 Realm Initial Measurement

More specialized resource are the attestation token documentation:
[1] https://datatracker.ietf.org/doc/html/draft-ffm-rats-cca-token-00
and CCA Security Model:
https://developer.arm.com/documentation/DEN0096/latest/

> 
> > 3. During remote attestation, the guest sends evidence containing this RIM
> >     signed by the root of trust, along with a signed token identifying the
> >     platform (hardware, firmware, RMM).
> 
> Is this a well known manufacturer key that one would expect for signature
> verification or is it locally created?

It comes from a well known manufacturer key, although the signing can be
delegated in some models (like in the current demos):

The hardware RoT creates a key pair for the RMM, which the RMM uses to
sign the RIM. The RoT then signs the RMM pubkey, using the well-known key
(see [1] 4.10 Token Binding).

> 
> > 4. The verifier checks the signature and the platform token, so it trusts
> >     the RMM and the RIM.
> > 
> > > > Then what prevents anyone from faking this log and presenting a
> > > > completely fake log to the verifier?
> > 
> > Absolutely, the verifier does not trust the content of the log, it only
> > uses the log as helper to try to reconstruct the RIM. For example a log
> > event says "I loaded image XYZ at address A", then the verifier searches
> > image XYZ in its database of known-good images, calculates the hash that
> > would result from loading that image at address A. Any malformed event in
> 
> Hopefully just calculating a hash over the image will do and the location an
> image was loaded to, like address A (relocation?), doesn't matter...

In this case it does matter because we don't trust the host VMM which
instructs RMM where to load the images. So we need to verify that the
image is in the correct location, otherwise the VM could be executing the
wrong part of the code, or using the wrong part of the data loaded into
RAM.

When loading an image the RMM hashes its content, then hashes a structure
that contains both the content hash and the address, and it is that hash
that is added to the RIM.

Thanks,
Jean

Re: [PATCH v3 03/26] target/arm/kvm: Return immediately on error in kvm_arch_init()

[Philippe Mathieu-Daudé]

Hi Jean-Philippe,

On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> Returning an error to kvm_init() is fatal anyway, no need to continue
> the initialization.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   target/arm/kvm.c | 9 +++++----
>   1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> index 8bdf4abeb6..95bcecf804 100644
> --- a/target/arm/kvm.c
> +++ b/target/arm/kvm.c
> @@ -563,7 +563,7 @@ int kvm_arch_get_default_type(MachineState *ms)
>   
>   int kvm_arch_init(MachineState *ms, KVMState *s)
>   {
> -    int ret = 0;
> +    int ret;

With your change we can reduce this variable scope ...

>       /* For ARM interrupt delivery is always asynchronous,
>        * whether we are using an in-kernel VGIC or not.
>        */
> @@ -585,7 +585,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>           !kvm_check_extension(s, KVM_CAP_ARM_IRQ_LINE_LAYOUT_2)) {
>           error_report("Using more than 256 vcpus requires a host kernel "
>                        "with KVM_CAP_ARM_IRQ_LINE_LAYOUT_2");
> -        ret = -EINVAL;
> +        return -EINVAL;
>       }
>   
>       if (kvm_check_extension(s, KVM_CAP_ARM_NISV_TO_USER)) {
> @@ -607,13 +607,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>               warn_report("Eager Page Split support not available");
>           } else if (!(s->kvm_eager_split_size & sizes)) {
>               error_report("Eager Page Split requested chunk size not valid");
> -            ret = -EINVAL;
> +            return -EINVAL;
>           } else {
>               ret = kvm_vm_enable_cap(s, KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE, 0,
>                                       s->kvm_eager_split_size);

... by declaring it here.

Otherwise:

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

>               if (ret < 0) {
>                   error_report("Enabling of Eager Page Split failed: %s",
>                                strerror(-ret));
> +                return ret;
>               }
>           }
>       }
> @@ -626,7 +627,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>       hw_breakpoints = g_array_sized_new(true, true,
>                                          sizeof(HWBreakpoint), max_hw_bps);
>   
> -    return ret;
> +    return 0;
>   }
>   
>   unsigned long kvm_arch_vcpu_id(CPUState *cpu)

Re: [PATCH v3 08/26] hw/core/loader: Add ROM loader notifier

[Philippe Mathieu-Daudé]

On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> Add a function to register a notifier, that is invoked after a ROM gets
> loaded into guest memory.
> 
> It will be used by Arm confidential guest support, in order to register
> all blobs loaded into memory with KVM, so that their content is moved
> into Realm state and measured into the initial VM state.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   include/hw/loader.h | 15 +++++++++++++++
>   hw/core/loader.c    | 15 +++++++++++++++
>   2 files changed, 30 insertions(+)
> 
> diff --git a/include/hw/loader.h b/include/hw/loader.h
> index 7f6d06b956..0cd9905f97 100644
> --- a/include/hw/loader.h
> +++ b/include/hw/loader.h
> @@ -353,6 +353,21 @@ void *rom_ptr_for_as(AddressSpace *as, hwaddr addr, size_t size);
>   ssize_t rom_add_vga(const char *file);
>   ssize_t rom_add_option(const char *file, int32_t bootindex);
>   
> +typedef struct RomLoaderNotify {
> +    /* Parameters passed to rom_add_blob() */
> +    hwaddr addr;

This is the buffer (blob) address in guest physical memory.

> +    size_t len;

This is the buffer length.

> +    size_t max_len;

This is the size of the MemoryRegion ROM containing the buffer.

Do we need to notify it? You don't use it in your next patch.
If so, I'd rather have this API returns a MemoryRegion (Rom->mr),
max_len can be retrieved using memory_region_size(mr); but I
don't think we need this at all (at least for now).

> +} RomLoaderNotify;
> +
> +/**
> + * rom_add_load_notifier - Add a notifier for loaded images
> + *
> + * Add a notifier that will be invoked with a RomLoaderNotify structure for each
> + * blob loaded into guest memory, after the blob is loaded.
> + */
> +void rom_add_load_notifier(Notifier *notifier);
> +
>   /* This is the usual maximum in uboot, so if a uImage overflows this, it would
>    * overflow on real hardware too. */
>   #define UBOOT_MAX_GUNZIP_BYTES (64 << 20)
> diff --git a/hw/core/loader.c b/hw/core/loader.c
> index 31593a1171..759a62cf58 100644
> --- a/hw/core/loader.c
> +++ b/hw/core/loader.c
> @@ -67,6 +67,8 @@
>   #include <zlib.h>
>   
>   static int roms_loaded;
> +static NotifierList rom_loader_notifier =
> +    NOTIFIER_LIST_INITIALIZER(rom_loader_notifier);
>   
>   /* return the size or -1 if error */
>   int64_t get_image_size(const char *filename)
> @@ -1179,6 +1181,11 @@ MemoryRegion *rom_add_blob(const char *name, const void *blob, size_t len,
>       return mr;
>   }
>   
> +void rom_add_load_notifier(Notifier *notifier)
> +{
> +    notifier_list_add(&rom_loader_notifier, notifier);
> +}
> +
>   /* This function is specific for elf program because we don't need to allocate
>    * all the rom. We just allocate the first part and the rest is just zeros. This
>    * is why romsize and datasize are different. Also, this function takes its own
> @@ -1220,6 +1227,7 @@ ssize_t rom_add_option(const char *file, int32_t bootindex)
>   static void rom_reset(void *unused)
>   {
>       Rom *rom;
> +    RomLoaderNotify notify;
>   
>       QTAILQ_FOREACH(rom, &roms, next) {
>           if (rom->fw_file) {
> @@ -1268,6 +1276,13 @@ static void rom_reset(void *unused)
>           cpu_flush_icache_range(rom->addr, rom->datasize);
>   
>           trace_loader_write_rom(rom->name, rom->addr, rom->datasize, rom->isrom);
> +
> +        notify = (RomLoaderNotify) {
> +            .addr = rom->addr,
> +            .len = rom->datasize,
> +            .max_len = rom->romsize,
> +        };
> +        notifier_list_notify(&rom_loader_notifier, &notify);
>       }
>   }
>   

Re: [PATCH v3 16/26] hw/arm/virt: Disable DTB randomness for confidential VMs

[Philippe Mathieu-Daudé]

On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> The dtb-randomness feature, which adds random seeds to the DTB, isn't
> really compatible with confidential VMs since it randomizes the Realm
> Initial Measurement. Enabling it is not an error, but it prevents
> attestation. It also isn't useful to a Realm, which doesn't trust host
> input.
> 
> Currently the feature is automatically enabled, unless the user disables
> it on the command-line. Change it to OnOffAuto, and automatically
> disable it for confidential VMs, unless the user explicitly enables it.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   docs/system/arm/virt.rst |  9 +++++----
>   include/hw/arm/virt.h    |  2 +-
>   hw/arm/virt.c            | 41 +++++++++++++++++++++++++---------------
>   3 files changed, 32 insertions(+), 20 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [RFC PATCH v3 23/26] hw/tpm: Add TPM event log

[Philippe Mathieu-Daudé]

On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> Provide a library allowing the VMM to create an event log that describes
> what is loaded into memory. During remote attestation in confidential
> computing this helps an independent verifier reconstruct the initial
> measurements of a VM, which contain the initial state of memory and
> CPUs.
> 
> We provide some definitions and structures described by the Trusted
> Computing Group (TCG) in "TCG PC Client Platform Firmware Profile
> Specification" Level 00 Version 1.06 Revision 52 [1]. This is the same
> format used by UEFI, and UEFI could reuse this log after finding it in
> DT or ACPI tables, but can also copy its content into a new one.
> 
> [1] https://trustedcomputinggroup.org/resource/pc-client-specific-platform-firmware-profile-specification/
> 
> Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: New
> ---
>   qapi/tpm.json            |  14 ++
>   include/hw/tpm/tpm_log.h |  89 +++++++++++
>   hw/tpm/tpm_log.c         | 325 +++++++++++++++++++++++++++++++++++++++
>   hw/tpm/Kconfig           |   4 +
>   hw/tpm/meson.build       |   1 +
>   5 files changed, 433 insertions(+)
>   create mode 100644 include/hw/tpm/tpm_log.h
>   create mode 100644 hw/tpm/tpm_log.c


> +/*
> + * Defined in: TCG PC Client Platform Firmware Profile Specification
> + * Version 1.06 revision 52
> + */
> +#define TCG_EV_NO_ACTION                        0x00000003
> +#define TCG_EV_EVENT_TAG                        0x00000006
> +#define TCG_EV_POST_CODE2                       0x00000013
> +#define TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2      0x8000000A
> +
> +struct UefiPlatformFirmwareBlob2Head {
> +        uint8_t blob_description_size;
> +        uint8_t blob_description[];
> +} __attribute__((packed));

We use QEMU_PACKED.

Re: [RFC PATCH v3 24/26] hw/core/loader: Add fields to RomLoaderNotify

[Philippe Mathieu-Daudé]

On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> In order to write an event log, the ROM load notification handler needs
> two more fields.

IMHO it makes more sense to squash that in the "hw/core/loader:
Add ROM loader notifier" patch introducing that API.

> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: New
> ---
>   include/hw/loader.h | 2 ++
>   hw/core/loader.c    | 2 ++
>   2 files changed, 4 insertions(+)
> 
> diff --git a/include/hw/loader.h b/include/hw/loader.h
> index 0cd9905f97..73f317966d 100644
> --- a/include/hw/loader.h
> +++ b/include/hw/loader.h
> @@ -355,6 +355,8 @@ ssize_t rom_add_option(const char *file, int32_t bootindex);
>   
>   typedef struct RomLoaderNotify {
>       /* Parameters passed to rom_add_blob() */
> +    const char *name;

Description of the loaded ROM.

> +    uint8_t *data;

Or 'blob', blob_ptr. Maybe declare as 'const void *'?

>       hwaddr addr;

Now easier to document, where 'data' is addressed in guest memory.

>       size_t len;

Size of 'data'.

>       size_t max_len;

Still unused. Drop?

Re: [RFC PATCH v3 26/26] hw/arm/virt: Add measurement log for confidential boot

[Philippe Mathieu-Daudé]

On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> Create a measurement log describing operations performed by QEMU to
> initialize the guest, and load it into guest memory above the DTB.
> 
> Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: New
> ---
>   include/hw/arm/boot.h |  3 +++
>   include/hw/arm/virt.h |  1 +
>   hw/arm/boot.c         | 47 +++++++++++++++++++++++++++++++++++++++++++
>   hw/arm/virt.c         | 23 +++++++++++++++++++++
>   4 files changed, 74 insertions(+)
> 
> diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
> index 5fcbaa2625..f2518c4e81 100644
> --- a/include/hw/arm/boot.h
> +++ b/include/hw/arm/boot.h
> @@ -147,6 +147,9 @@ struct arm_boot_info {
>        * Confidential guest boot loads everything into RAM so it can be measured.
>        */
>       bool confidential;
> +    /* measurement log location in guest memory */
> +    hwaddr log_start;

One expects a stop/end after "start", maybe 'log_paddr'?

> +    size_t log_size;
>   };

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Stefan Berger]

On 12/5/24 7:33 AM, Jean-Philippe Brucker wrote:
> On Mon, Dec 02, 2024 at 10:58:01AM -0500, Stefan Berger wrote:
>>
>>
>> On 11/26/24 11:21 AM, Jean-Philippe Brucker wrote:
>>> On Tue, Nov 26, 2024 at 01:45:55PM +0000, Daniel P. Berrangé wrote:
>>>> On Mon, Nov 25, 2024 at 05:23:44PM -0500, Stefan Berger wrote:
>>>>>
>>>>>
>>>>> On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
>>>>>> Create an event log, in the format defined by Trusted Computing Group
>>>>>> for TPM2. It contains information about the VMM, the Realm parameters,
>>>>>> any data loaded into guest memory before boot and the initial vCPU
>>>>>> state.
>>>>>>
>>>>>> The guest can access this log from RAM and send it to a verifier, to
>>>>>> help the verifier independently compute the Realm Initial Measurement,
>>>>>> and check that the data we load into guest RAM is known-good images.
>>>>>> Without this log, the verifier has to guess where everything is loaded>
>>>>> and in what order.
>>>>>
>>>>> Typically these logs are backed by extensions of TPM PCRs and when you send
>>>>> a log to a verifier you send a TPM quote along with it for the verifer to
>>>>> replay the log and check the TPM quote. Also, early code in the firmware is
>>>>> typically serving as a root of trust that starts the chain of measurements
>>>>> of code and data, first measuring itself and then other parts of the
>>>>> firmware before it jumps into the other parts. Now here you seem to just
>>>>> have a log and no PCR extensions and therefore no quote over PCRs can be
>>>>> used.
>>>
>>> Indeed, in our case it's the trusted hypervisor (RMM) that provides the
>>> equivalent to TPM quote and PCRs. In more details:
>>>
>>> 1. QEMU loads images into guest RAM by calling KVM, which calls RMM.
>>> 2. RMM calculates a hash of the image content, adds it to a rolling hash
>>>      the "Realm Initial Measurement" (RIM), which I believe is equivalent to
>>>      a PCR.
>>
>> I am not familiar with RIM. A link to read more about it would be helpful.
> 
> The "Learn the architecture" documentation might be a good introduction
> https://developer.arm.com/documentation/den0127/0200/Overview
> In particular the part about Realm creation:
> https://developer.arm.com/documentation/den0127/0200/Realm-management/Realm-creation-and-attestation
> 
> The RMM specification describes exactly how the RIM is calculated, but
> is less palatable:
> https://developer.arm.com/documentation/den0137/1-0rel0/?lang=en
> A7.1.1 Realm Initial Measurement
> 
> More specialized resource are the attestation token documentation:
> [1] https://datatracker.ietf.org/doc/html/draft-ffm-rats-cca-token-00
> and CCA Security Model:
> https://developer.arm.com/documentation/DEN0096/latest/

Thanks for the links. I will have a look at them when I have time.

> 
>>
>>> 3. During remote attestation, the guest sends evidence containing this RIM
>>>      signed by the root of trust, along with a signed token identifying the
>>>      platform (hardware, firmware, RMM).
>>
>> Is this a well known manufacturer key that one would expect for signature
>> verification or is it locally created?
> 
> It comes from a well known manufacturer key, although the signing can be
> delegated in some models (like in the current demos):
> 
> The hardware RoT creates a key pair for the RMM, which the RMM uses to
> sign the RIM. The RoT then signs the RMM pubkey, using the well-known key
> (see [1] 4.10 Token Binding).

You should mention in the commit message that the log will be signed and 
user space can get the signature over the log from some filesystem or so.

    Stefan

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Stefan Berger]

On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> Create an event log, in the format defined by Trusted Computing Group

s/,//

> for TPM2. It contains information about the VMM, the Realm parameters,
> any data loaded into guest memory before boot and the initial vCPU

s/ and/ ,and/

[move comma from above to before 'and']

> state.
> 
> The guest can access this log from RAM and send it to a verifier, to
> help the verifier independently compute the Realm Initial Measurement,
> and check that the data we load into guest RAM is known-good images.
> Without this log, the verifier has to guess where everything is loaded
> and in what order.

Mention that the verifier can pull out the signature from somewhere as 
well...

> 
> Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: New
> ---
>   qapi/qom.json        |   9 +-
>   target/arm/kvm_arm.h |  27 +++
>   target/arm/kvm-rme.c | 415 ++++++++++++++++++++++++++++++++++++++++++-
>   target/arm/Kconfig   |   1 +
>   4 files changed, 449 insertions(+), 3 deletions(-)
> 
> diff --git a/qapi/qom.json b/qapi/qom.json
> index 901ba67634..1de1b0d8af 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -1094,11 +1094,18 @@
>   # @measurement-algorithm: Realm measurement algorithm
>   #     (default: sha512)
>   #
> +# @measurement-log: Enable a measurement log for the Realm. All events
> +#     that contribute to the Realm Initial Measurement (RIM) are added
> +#     to a log in TCG TPM2 format, which is itself loaded into Realm
> +#     memory (unmeasured) and can then be read by a verifier to
> +#     reconstruct the RIM.
> +#
>   # Since: 9.3
>   ##
>   { 'struct': 'RmeGuestProperties',
>     'data': { '*personalization-value': 'str',
> -            '*measurement-algorithm': 'RmeGuestMeasurementAlgorithm' } }
> +            '*measurement-algorithm': 'RmeGuestMeasurementAlgorithm',
> +            '*measurement-log': 'bool'} }
>   
>   ##
>   # @ObjectType:
> diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
> index 77680f238a..44e95a034b 100644
> --- a/target/arm/kvm_arm.h
> +++ b/target/arm/kvm_arm.h
> @@ -268,6 +268,24 @@ int kvm_arm_rme_vcpu_init(CPUState *cs);
>    */
>   void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size);
>   
> +/**
> + * kvm_arm_rme_get_measurement_log
> + *
> + * Obtain the measurement log object if enabled, in order to get its size and
> + * set its base address.
> + *
> + * Returns NULL if measurement log is disabled.
> + */
> +Object *kvm_arm_rme_get_measurement_log(void);
> +
> +/**
> + * kvm_arm_rme_set_ipa_size
> + * @ipa_bits: number of guest physical address bits
> + *
> + * Set the GPA size, not counting the bit reserved for shared address range.
> + */
> +void kvm_arm_rme_set_ipa_size(uint8_t ipa_bits);
> +
>   #else
>   
>   /*
> @@ -298,6 +316,15 @@ static inline void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size)
>   {
>   }
>   
> +static inline Object *kvm_arm_rme_get_measurement_log(void)
> +{
> +    return NULL;
> +}
> +
> +static inline void kvm_arm_rme_set_ipa_size(uint8_t ipa_size)
> +{
> +}
> +
>   /*
>    * These functions should never actually be called without KVM support.
>    */
> diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
> index bf0bcf9a38..f92cfdb5f3 100644
> --- a/target/arm/kvm-rme.c
> +++ b/target/arm/kvm-rme.c
> @@ -10,10 +10,12 @@
>   #include "hw/boards.h"
>   #include "hw/core/cpu.h"
>   #include "hw/loader.h"
> +#include "hw/tpm/tpm_log.h"
>   #include "kvm_arm.h"
>   #include "migration/blocker.h"
>   #include "qapi/error.h"
>   #include "qemu/error-report.h"
> +#include "qemu/units.h"
>   #include "qom/object_interfaces.h"
>   #include "sysemu/kvm.h"
>   #include "sysemu/runstate.h"
> @@ -25,6 +27,8 @@ OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
>   
>   #define RME_MAX_CFG         2
>   
> +#define RME_MEASUREMENT_LOG_SIZE    (64 * KiB)
> +
>   struct RmeGuest {
>       ConfidentialGuestSupport parent_obj;
>       Notifier rom_load_notifier;
> @@ -32,22 +36,344 @@ struct RmeGuest {
>   
>       uint8_t *personalization_value;
>       RmeGuestMeasurementAlgorithm measurement_algo;
> +    bool use_measurement_log;
>   
> +    size_t num_cpus;
> +    uint8_t ipa_bits;
>       hwaddr ram_base;
>       size_t ram_size;
> +
> +    TpmLog *log;
> +    GHashTable *images;
>   };
>   
>   OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
>                                             CONFIDENTIAL_GUEST_SUPPORT,
>                                             { TYPE_USER_CREATABLE }, { })
>   
> +typedef struct RmeLogFiletype {
> +    uint32_t event_type;
> +    /* Description copied into the log event */
> +    const char *desc;
> +} RmeLogFiletype;
> +
>   typedef struct {
>       hwaddr base;
>       hwaddr size;
> +    uint8_t *data;
> +    RmeLogFiletype *filetype;
>   } RmeRamRegion;
>   
> +typedef struct {
> +    char        signature[16];
> +    char        name[32];
> +    char        version[40];
> +    uint64_t    ram_size;
> +    uint32_t    num_cpus;
> +    uint64_t    flags;
> +} EventLogVmmVersion;
> +
> +typedef struct {
> +    uint32_t    id;
> +    uint32_t    data_size;
> +    uint8_t     data[];
> +} EventLogTagged;
> +


> +#define EVENT_LOG_TAG_REALM_CREATE  1
> +#define EVENT_LOG_TAG_INIT_RIPAS    2
> +#define EVENT_LOG_TAG_REC_CREATE    3
> +
If these are ARM-related structures and constants from a document you 
may want to mention the document you got them from.


> +#define REALM_PARAMS_FLAG_SVE       (1 << 1)
> +#define REALM_PARAMS_FLAG_PMU       (1 << 2)
> +
> +#define REC_CREATE_FLAG_RUNNABLE    (1 << 0)
> +
>   static RmeGuest *rme_guest;
>   
> +static int rme_init_measurement_log(MachineState *ms)
> +{
> +    Object *log;
> +    gpointer filename;
> +    TpmLogDigestAlgo algo;
> +    RmeLogFiletype *filetype;
> +
> +    if (!rme_guest->use_measurement_log) {
> +        return 0;
> +    }
> +
> +    switch (rme_guest->measurement_algo) {
> +    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA256:
> +        algo = TPM_LOG_DIGEST_ALGO_SHA256;
> +        break;
> +    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA512:
> +        algo = TPM_LOG_DIGEST_ALGO_SHA512;
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }
> +
> +    log = object_new_with_props(TYPE_TPM_LOG, OBJECT(rme_guest),
> +                                "log", &error_fatal,
> +                                "digest-algo", TpmLogDigestAlgo_str(algo),
> +                                NULL);
> +
> +    tpm_log_create(TPM_LOG(log), RME_MEASUREMENT_LOG_SIZE, &error_fatal);
> +    rme_guest->log = TPM_LOG(log);
> +
> +    /*
> +     * Write down the image names we're expecting to encounter when handling the
> +     * ROM load notifications, so we can record the type of image being loaded
> +     * to help the verifier.
> +     */
> +    rme_guest->images = g_hash_table_new_full(g_str_hash, g_str_equal, g_free,
> +                                              g_free);
> +
> +    filename = g_strdup(ms->kernel_filename);
> +    if (filename) {
> +        filetype = g_new0(RmeLogFiletype, 1);
> +        filetype->event_type = TCG_EV_POST_CODE2;
> +        filetype->desc = "KERNEL";
> +        g_hash_table_insert(rme_guest->images, filename, (gpointer)filetype);
> +    }
> +
> +    filename = g_strdup(ms->initrd_filename);
> +    if (filename) {
> +        filetype = g_new0(RmeLogFiletype, 1);
> +        filetype->event_type = TCG_EV_POST_CODE2;
> +        filetype->desc = "INITRD";
> +        g_hash_table_insert(rme_guest->images, filename, (gpointer)filetype);
> +    }
> +
> +    filename = g_strdup(ms->firmware);
> +    if (filename) {
> +        filetype = g_new0(RmeLogFiletype, 1);
> +        filetype->event_type = TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2;
> +        filetype->desc = "FIRMWARE";
> +        g_hash_table_insert(rme_guest->images, filename, filetype);
> +    }
> +
> +    filename = g_strdup(ms->dtb);
> +    if (!filename) {
> +        filename = g_strdup("dtb");
> +    }
> +    filetype = g_new0(RmeLogFiletype, 1);
> +    filetype->event_type = TCG_EV_POST_CODE2;
> +    filetype->desc = "DTB";
> +    g_hash_table_insert(rme_guest->images, filename, filetype);
> +
> +    return 0;
> +}
> +
> +static int rme_log_event_tag(uint32_t id, uint8_t *data, size_t size,
> +                             Error **errp)
> +{
> +    int ret;
> +    EventLogTagged event = {
> +        .id = id,
> +        .data_size = size,
> +    };
> +    GByteArray *bytes = g_byte_array_new();
> +
> +    if (!rme_guest->log) {
> +        return 0;
> +    }
> +
> +    g_byte_array_append(bytes, (uint8_t *)&event, sizeof(event));
> +    g_byte_array_append(bytes, data, size);
> +    ret = tpm_log_add_event(rme_guest->log, TCG_EV_EVENT_TAG, bytes->data,
> +                             bytes->len, NULL, 0, errp);
> +    g_byte_array_free(bytes, true);
> +    return ret;
> +}
> +
> +/* Log VM type and Realm Descriptor create */
> +static int rme_log_realm_create(Error **errp)
> +{
> +    int ret;
> +    ARMCPU *cpu;
> +    EventLogVmmVersion vmm_version = {
> +        .signature = "VM VERSION",
> +        .name = "QEMU",
> +        .version = "9.1", /* TODO: dynamic */

$ grep -r QEMU_VERSION_M build/
build/config-host.h:#define QEMU_VERSION_MAJOR 9
build/config-host.h:#define QEMU_VERSION_MICRO 93
build/config-host.h:#define QEMU_VERSION_MINOR 1

$ cat VERSION
9.1.93


> +        .ram_size = cpu_to_le64(rme_guest->ram_size),
> +        .num_cpus = cpu_to_le32(rme_guest->num_cpus),
> +        .flags = 0,
> +    };
> +    struct {
> +        uint64_t    flags;
> +        uint8_t     s2sz;
> +        uint8_t     sve_vl;
> +        uint8_t     num_bps;
> +        uint8_t     num_wps;
> +        uint8_t     pmu_num_ctrs;
> +        uint8_t     hash_algo;
> +    } params = {
> +        .s2sz = rme_guest->ipa_bits,
> +    };
> +
> +    if (!rme_guest->log) {
> +        return 0;
> +    }
> +
> +    ret = tpm_log_add_event(rme_guest->log, TCG_EV_NO_ACTION,
> +                            (uint8_t *)&vmm_version, sizeof(vmm_version),
> +                            NULL, 0, errp);
> +    if (ret) {
> +        return ret;
> +    }
> +
> +    /* With KVM all CPUs have the same capability */
> +    cpu = ARM_CPU(first_cpu);
> +    if (cpu->has_pmu) {
> +        params.flags |= REALM_PARAMS_FLAG_PMU;
> +        params.pmu_num_ctrs = FIELD_EX64(cpu->isar.reset_pmcr_el0, PMCR, N);
> +    }
> +
> +    if (cpu->sve_max_vq) {
> +        params.flags |= REALM_PARAMS_FLAG_SVE;
> +        params.sve_vl = cpu->sve_max_vq - 1;
> +    }
> +    params.num_bps = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, BRPS);
> +    params.num_wps = FIELD_EX64(cpu->isar.id_aa64dfr0, ID_AA64DFR0, WRPS);
> +
> +    switch (rme_guest->measurement_algo) {
> +    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA256:
> +        params.hash_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA256;
> +        break;
> +    case RME_GUEST_MEASUREMENT_ALGORITHM_SHA512:
> +        params.hash_algo = KVM_CAP_ARM_RME_MEASUREMENT_ALGO_SHA512;
> +        break;
> +    default:
> +        g_assert_not_reached();
> +    }
> +
> +    return rme_log_event_tag(EVENT_LOG_TAG_REALM_CREATE, (uint8_t *)&params,
> +                             sizeof(params), errp);
> +}
> +
> +/* unmeasured images are logged with @data == NULL */
> +static int rme_log_image(RmeLogFiletype *filetype, uint8_t *data, hwaddr base,
> +                          size_t size, Error **errp)
> +{
> +    int ret;
> +    size_t desc_size;
> +    GByteArray *event = g_byte_array_new();
> +    struct UefiPlatformFirmwareBlob2Head head = {0};
> +    struct UefiPlatformFirmwareBlob2Tail tail = {0};
> +
> +    if (!rme_guest->log) {
> +        return 0;
> +    }
> +
> +    if (!filetype) {
> +        error_setg(errp, "cannot log image without a filetype");
> +        return -1;
> +    }
> +
> +    /* EV_POST_CODE2 strings are not NUL-terminated */
> +    desc_size = strlen(filetype->desc);
> +    head.blob_description_size = desc_size;
> +    tail.blob_base = cpu_to_le64(base);
> +    tail.blob_size = cpu_to_le64(size);
> +
> +    g_byte_array_append(event, (guint8 *)&head, sizeof(head));
> +    g_byte_array_append(event, (guint8 *)filetype->desc, desc_size);
> +    g_byte_array_append(event, (guint8 *)&tail, sizeof(tail));
> +
> +    ret = tpm_log_add_event(rme_guest->log, filetype->event_type, event->data,
> +                            event->len, data, size, errp);
> +    g_byte_array_free(event, true);
> +    return ret;
> +}
> +
> +static int rme_log_ripas(hwaddr base, size_t size, Error **errp)
> +{
> +    struct {
> +        uint64_t base;
> +        uint64_t size;
> +    } init_ripas = {
> +        .base = cpu_to_le64(base),
> +        .size = cpu_to_le64(size),
> +    };
> +
> +    return rme_log_event_tag(EVENT_LOG_TAG_INIT_RIPAS, (uint8_t *)&init_ripas,
> +                             sizeof(init_ripas), errp);
> +}
> +
> +static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], Error **errp)
> +{

$ ./scripts/checkpatch.pl ./tmp/*.patch
[...]
Checking ./tmp/0002-target-arm-kvm-rme-Add-measurement-log.patch...
WARNING: line over 80 characters
#353: FILE: target/arm/kvm-rme.c:303:
+static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], 
Error **errp)

May want to run this on all patches.

Rest LGTM.


> +    struct {
> +        uint64_t flags;
> +        uint64_t pc;
> +        uint64_t gprs[8];
> +    } rec_create = {
> +        .flags = cpu_to_le64(flags),
> +        .pc = cpu_to_le64(pc),
> +        .gprs[0] = cpu_to_le64(gprs[0]),
> +        .gprs[1] = cpu_to_le64(gprs[1]),
> +        .gprs[2] = cpu_to_le64(gprs[2]),
> +        .gprs[3] = cpu_to_le64(gprs[3]),
> +        .gprs[4] = cpu_to_le64(gprs[4]),
> +        .gprs[5] = cpu_to_le64(gprs[5]),
> +        .gprs[6] = cpu_to_le64(gprs[6]),
> +        .gprs[7] = cpu_to_le64(gprs[7]),
> +    };
> +
> +    return rme_log_event_tag(EVENT_LOG_TAG_REC_CREATE, (uint8_t *)&rec_create,
> +                             sizeof(rec_create), errp);
> +}
> +
> +static int rme_populate_range(hwaddr base, size_t size, bool measure,
> +                              Error **errp);
> +
> +static int rme_close_measurement_log(Error **errp)
> +{
> +    int ret;
> +    hwaddr base;
> +    size_t size;
> +    RmeLogFiletype filetype = {
> +        .event_type = TCG_EV_POST_CODE2,
> +        .desc = "LOG",
> +    };
> +
> +    if (!rme_guest->log) {
> +        return 0;
> +    }
> +
> +    base = object_property_get_uint(OBJECT(rme_guest->log), "load-addr", errp);
> +    if (*errp) {
> +        return -1;
> +    }
> +
> +    size = object_property_get_uint(OBJECT(rme_guest->log), "max-size", errp);
> +    if (*errp) {
> +        return -1;
> +    }
> +
> +    /* Log the log itself */
> +    ret = rme_log_image(&filetype, NULL, base, size, errp);
> +    if (ret) {
> +        return ret;
> +    }
> +
> +    ret = tpm_log_write_and_close(rme_guest->log, errp);
> +    if (ret) {
> +        return ret;
> +    }
> +
> +    ret = rme_populate_range(base, size, /* measure */ false, errp);
> +    if (ret) {
> +        return ret;
> +    }
> +
> +    g_hash_table_destroy(rme_guest->images);
> +
> +    /* The log is now in the guest. Free this object */
> +    object_unparent(OBJECT(rme_guest->log));
> +    rme_guest->log = NULL;
> +    return 0;
> +}
> +
>   static int rme_configure_one(RmeGuest *guest, uint32_t cfg, Error **errp)
>   {
>       int ret;
> @@ -120,9 +446,10 @@ static int rme_init_ram(hwaddr base, size_t size, Error **errp)
>           error_setg_errno(errp, -ret,
>                            "failed to init RAM [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
>                            start, end);
> +        return ret;
>       }
>   
> -    return ret;
> +    return rme_log_ripas(base, size, errp);
>   }
>   
>   static int rme_populate_range(hwaddr base, size_t size, bool measure,
> @@ -158,23 +485,42 @@ static void rme_populate_ram_region(gpointer data, gpointer err)
>       }
>   
>       rme_populate_range(region->base, region->size, /* measure */ true, errp);
> +    if (*errp) {
> +        return;
> +    }
> +
> +    rme_log_image(region->filetype, region->data, region->base, region->size,
> +                  errp);
>   }
>   
>   static int rme_init_cpus(Error **errp)
>   {
>       int ret;
>       CPUState *cs;
> +    bool logged_primary_cpu = false;
>   
>       /*
>        * Now that do_cpu_reset() initialized the boot PC and
>        * kvm_cpu_synchronize_post_reset() registered it, we can finalize the REC.
>        */
>       CPU_FOREACH(cs) {
> -        ret = kvm_arm_vcpu_finalize(ARM_CPU(cs), KVM_ARM_VCPU_REC);
> +        ARMCPU *cpu = ARM_CPU(cs);
> +
> +        ret = kvm_arm_vcpu_finalize(cpu, KVM_ARM_VCPU_REC);
>           if (ret) {
>               error_setg_errno(errp, -ret, "failed to finalize vCPU");
>               return ret;
>           }
> +
> +        if (!logged_primary_cpu) {
> +            ret = rme_log_rec(REC_CREATE_FLAG_RUNNABLE, cpu->env.pc,
> +                              cpu->env.xregs, errp);
> +            if (ret) {
> +                return ret;
> +            }
> +
> +            logged_primary_cpu = true;
> +        }
>       }
>       return 0;
>   }
> @@ -194,6 +540,10 @@ static int rme_create_realm(Error **errp)
>           return -1;
>       }
>   
> +    if (rme_log_realm_create(errp)) {
> +        return -1;
> +    }
> +
>       if (rme_init_ram(rme_guest->ram_base, rme_guest->ram_size, errp)) {
>           return -1;
>       }
> @@ -208,6 +558,10 @@ static int rme_create_realm(Error **errp)
>           return -1;
>       }
>   
> +    if (rme_close_measurement_log(errp)) {
> +        return -1;
> +    }
> +
>       ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
>                               KVM_CAP_ARM_RME_ACTIVATE_REALM);
>       if (ret) {
> @@ -303,6 +657,20 @@ static void rme_set_measurement_algo(Object *obj, int algo, Error **errp)
>       guest->measurement_algo = algo;
>   }
>   
> +static bool rme_get_measurement_log(Object *obj, Error **errp)
> +{
> +    RmeGuest *guest = RME_GUEST(obj);
> +
> +    return guest->use_measurement_log;
> +}
> +
> +static void rme_set_measurement_log(Object *obj, bool value, Error **errp)
> +{
> +    RmeGuest *guest = RME_GUEST(obj);
> +
> +    guest->use_measurement_log = value;
> +}
> +
>   static void rme_guest_class_init(ObjectClass *oc, void *data)
>   {
>       object_class_property_add_str(oc, "personalization-value", rme_get_rpv,
> @@ -317,6 +685,12 @@ static void rme_guest_class_init(ObjectClass *oc, void *data)
>                                      rme_set_measurement_algo);
>       object_class_property_set_description(oc, "measurement-algorithm",
>               "Realm measurement algorithm ('sha256', 'sha512')");
> +
> +    object_class_property_add_bool(oc, "measurement-log",
> +                                   rme_get_measurement_log,
> +                                   rme_set_measurement_log);
> +    object_class_property_set_description(oc, "measurement-log",
> +            "Enable/disable Realm measurement log");
>   }
>   
>   static void rme_guest_init(Object *obj)
> @@ -359,6 +733,20 @@ static void rme_rom_load_notify(Notifier *notifier, void *data)
>       region = g_new0(RmeRamRegion, 1);
>       region->base = rom->addr;
>       region->size = rom->len;
> +    /*
> +     * TODO: double-check lifetime. Is data is still available when we measure
> +     * it, while writing the log. Should be fine since data is kept for the next
> +     * reset.
> +     */
> +    region->data = rom->data;
> +
> +    /*
> +     * rme_guest->images is destroyed after ram_regions, so we can store
> +     * filetype even if we don't own the struct.
> +     */
> +    if (rme_guest->images) {
> +        region->filetype = g_hash_table_lookup(rme_guest->images, rom->name);
> +    }
>   
>       /*
>        * The Realm Initial Measurement (RIM) depends on the order in which we
> @@ -388,6 +776,13 @@ int kvm_arm_rme_init(MachineState *ms)
>           return -ENODEV;
>       }
>   
> +    if (rme_init_measurement_log(ms)) {
> +        return -ENODEV;
> +    }
> +
> +    rme_guest->ram_size = ms->ram_size;
> +    rme_guest->num_cpus = ms->smp.max_cpus;
> +
>       error_setg(&rme_mig_blocker, "RME: migration is not implemented");
>       migrate_add_blocker(&rme_mig_blocker, &error_fatal);
>   
> @@ -430,3 +825,19 @@ int kvm_arm_rme_vm_type(MachineState *ms)
>       }
>       return 0;
>   }
> +
> +void kvm_arm_rme_set_ipa_size(uint8_t ipa_bits)
> +{
> +    if (rme_guest) {
> +        /* We request one more bit to KVM as the NS flag */
> +        rme_guest->ipa_bits = ipa_bits + 1;
> +    }
> +}
> +
> +Object *kvm_arm_rme_get_measurement_log(void)
> +{
> +    if (rme_guest) {
> +        return OBJECT(rme_guest->log);
> +    }
> +    return NULL;
> +}
> diff --git a/target/arm/Kconfig b/target/arm/Kconfig
> index 7f8a2217ae..ee3a2184d0 100644
> --- a/target/arm/Kconfig
> +++ b/target/arm/Kconfig
> @@ -13,3 +13,4 @@ config AARCH64
>       select ARM
>       # kvm_arch_fixup_msi_route() needs to access PCIDevice
>       select PCI if KVM
> +    select TPM_LOG if KVM

Re: [RFC PATCH v3 23/26] hw/tpm: Add TPM event log

[Stefan Berger]

On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> Provide a library allowing the VMM to create an event log that describes
> what is loaded into memory. During remote attestation in confidential
> computing this helps an independent verifier reconstruct the initial
> measurements of a VM, which contain the initial state of memory and
> CPUs.
> 
> We provide some definitions and structures described by the Trusted
> Computing Group (TCG) in "TCG PC Client Platform Firmware Profile
> Specification" Level 00 Version 1.06 Revision 52 [1]. This is the same
> format used by UEFI, and UEFI could reuse this log after finding it in

as used by

> DT or ACPI tables, but can also copy its content into a new one.

I thought it was going to be a completely independent log. If UEFI would 
do anything with it, I think it would have to replay the measurements 
into its own log and extend them into all PCRs of all active PCR banks 
of the TPM, but if I understand correctly then you do not use the TPM 
for this log at all since you have a signature over it and defined 
(somewhere -- where?) that only sha256 and sha512 are to be used for 
this log.

> 
> [1] https://trustedcomputinggroup.org/resource/pc-client-specific-platform-firmware-profile-specification/
> 
> Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: New
> ---
>   qapi/tpm.json            |  14 ++
>   include/hw/tpm/tpm_log.h |  89 +++++++++++
>   hw/tpm/tpm_log.c         | 325 +++++++++++++++++++++++++++++++++++++++
>   hw/tpm/Kconfig           |   4 +
>   hw/tpm/meson.build       |   1 +
>   5 files changed, 433 insertions(+)
>   create mode 100644 include/hw/tpm/tpm_log.h
>   create mode 100644 hw/tpm/tpm_log.c
> 
> diff --git a/qapi/tpm.json b/qapi/tpm.json
> index a16a72edb9..697e7150ee 100644
> --- a/qapi/tpm.json
> +++ b/qapi/tpm.json
> @@ -188,3 +188,17 @@
>   ##
>   { 'command': 'query-tpm', 'returns': ['TPMInfo'],
>     'if': 'CONFIG_TPM' }
> +
> +##
> +# @TpmLogDigestAlgo:
> +#
> +# @sha256: Use the SHA256 algorithm
> +#
> +# @sha512: Use the SHA512 algorithm
> +#
> +# Algorithm to use for event log digests
> +#
> +# Since: 9.3
> +##
> +{ 'enum': 'TpmLogDigestAlgo',
> +  'data': ['sha256', 'sha512'] }
> diff --git a/include/hw/tpm/tpm_log.h b/include/hw/tpm/tpm_log.h
> new file mode 100644
> index 0000000000..b3cd2e7563
> --- /dev/null
> +++ b/include/hw/tpm/tpm_log.h
> @@ -0,0 +1,89 @@
> +#ifndef QEMU_TPM_LOG_H
> +#define QEMU_TPM_LOG_H
> +
> +#include "qom/object.h"
> +#include "sysemu/tpm.h"
> +
> +/*
> + * Defined in: TCG Algorithm Registry
> + * Family 2.0 Level 00 Revision 01.34
> + *
> + * (Here TCG stands for Trusted Computing Group)
> + */
> +#define TCG_ALG_SHA256  0xB
> +#define TCG_ALG_SHA512  0xD
> +
> +/* Size of a digest in bytes */
> +#define TCG_ALG_SHA256_DIGEST_SIZE      32
> +#define TCG_ALG_SHA512_DIGEST_SIZE      64
> +
> +/*
> + * Defined in: TCG PC Client Platform Firmware Profile Specification
> + * Version 1.06 revision 52
> + */
> +#define TCG_EV_NO_ACTION                        0x00000003
> +#define TCG_EV_EVENT_TAG                        0x00000006
> +#define TCG_EV_POST_CODE2                       0x00000013
> +#define TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2      0x8000000A
> +
> +struct UefiPlatformFirmwareBlob2Head {
> +        uint8_t blob_description_size;
> +        uint8_t blob_description[];
> +} __attribute__((packed));
> +
> +struct UefiPlatformFirmwareBlob2Tail {
> +        uint64_t blob_base;
> +        uint64_t blob_size;
> +} __attribute__((packed));
> +
> +#define TYPE_TPM_LOG "tpm-log"
> +
> +OBJECT_DECLARE_SIMPLE_TYPE(TpmLog, TPM_LOG)
> +
> +/**
> + * tpm_log_create - Create the event log
> + * @log: the log object
> + * @max_size: maximum size of the log. Adding an event past that size will
> + *            return an error
> + * @errp: pointer to a NULL-initialized error object
> + *
> + * Allocate the event log and create the initial entry (Spec ID Event03)
> + * describing the log format.
> + *
> + * Returns: 0 on success, -1 on error
> + */
> +int tpm_log_create(TpmLog *log, size_t max_size, Error **errp);
> +
> +/**
> + * tpm_log_add_event - Append an event to the log
> + * @log: the log object
> + * @event_type: the `eventType` field in TCG_PCR_EVENT2
> + * @event: the `event` field in TCG_PCR_EVENT2
> + * @event_size: the `eventSize` field in TCG_PCR_EVENT2
> + * @data: content to be hashed into the event digest. May be NULL.
> + * @data_size: size of @data. Should be zero when @data is NULL.
> + * @errp: pointer to a NULL-initialized error object
> + *
> + * Add a TCG_PCR_EVENT2 event to the event log. Depending on the event type, a
> + * data buffer may be hashed into the event digest (for example
> + * TCG_EV_EFI_PLATFORM_FIRMWARE_BLOB2 contains a digest of the blob.)
> + *
> + * Returns: 0 on success, -1 on error
> + */
> +int tpm_log_add_event(TpmLog *log, uint32_t event_type, const uint8_t *event,
> +                      size_t event_size, const uint8_t *data, size_t data_size,
> +                      Error **errp);
> +
> +/**
> + * tpm_log_write_and_close - Move the log to guest memory
> + * @log: the log object
> + * @errp: pointer to a NULL-initialized error object
> + *
> + * Write the log into memory, at the address set in the load-addr property.
> + * After this operation, the log is not writable anymore.
> + *
> + * Return: 0 on success, -1 on error
> + */
> +int tpm_log_write_and_close(TpmLog *log, Error **errp);
> +
> +#endif
> diff --git a/hw/tpm/tpm_log.c b/hw/tpm/tpm_log.c
> new file mode 100644
> index 0000000000..e6183a6e70
> --- /dev/null
> +++ b/hw/tpm/tpm_log.c
> @@ -0,0 +1,325 @@
> +/*
> + * tpm_log.c - Event log as described by the Trusted Computing Group (TCG)
> + *
> + * Copyright (c) 2024 Linaro Ltd.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + * Create an event log in the format specified by:
> + *
> + *  TCG PC Client Platform Firmware Profile Specification
> + *  Level 00 Version 1.06 Revision 52
> + *  Family “2.0”
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "crypto/hash.h"
> +#include "exec/address-spaces.h"
> +#include "exec/memory.h"
> +#include "hw/tpm/tpm_log.h"
> +#include "qapi/error.h"
> +#include "qemu/bswap.h"
> +#include "qom/object_interfaces.h"
> +
> +/*
> + * Legacy structure used only in the first event in the log, for compatibility
> + */
> +struct TcgPcClientPcrEvent {
> +        uint32_t pcr_index;
> +        uint32_t event_type;
> +        uint8_t  digest[20];
> +        uint32_t event_data_size;
> +        uint8_t  event[];
> +} __attribute__((packed));
> +
> +struct TcgEfiSpecIdEvent {
> +        uint8_t  signature[16];
> +        uint32_t platform_class;
> +        uint8_t  family_version_minor;
> +        uint8_t  family_version_major;
> +        uint8_t  spec_revision;
> +        uint8_t  uintn_size;
> +        uint32_t number_of_algorithms; /* 1 */
> +        /*
> +         * For now we declare a single algo, but if we want UEFI to reuse this

You mean UEFI would reuse this struct here? I think UEFI will not use it 
nor will it look at the binary log...

> +         * header then we'd need to add entries here for all algos supported by
> +         * UEFI (and expand the digest field for EV_NO_ACTION).
> +         */
> +        uint16_t algorithm_id;
> +        uint16_t digest_size;
> +        uint8_t  vendor_info_size;
> +        uint8_t  vendor_info[];
> +} __attribute__((packed));

Apart from QEMU_PACKED I have not much else to say here.

Re: [RFC PATCH v3 24/26] hw/core/loader: Add fields to RomLoaderNotify

[Jean-Philippe Brucker]

On Thu, Dec 05, 2024 at 11:21:19PM +0100, Philippe Mathieu-Daudé wrote:
> On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> > In order to write an event log, the ROM load notification handler needs
> > two more fields.
> 
> IMHO it makes more sense to squash that in the "hw/core/loader:
> Add ROM loader notifier" patch introducing that API.

Yes I'd squash it if we decide that the patch 25, which needs this, is
useful. But it's possible that no one actually needs it so I left this
separate for the moment.

> 
> > 
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > v2->v3: New
> > ---
> >   include/hw/loader.h | 2 ++
> >   hw/core/loader.c    | 2 ++
> >   2 files changed, 4 insertions(+)
> > 
> > diff --git a/include/hw/loader.h b/include/hw/loader.h
> > index 0cd9905f97..73f317966d 100644
> > --- a/include/hw/loader.h
> > +++ b/include/hw/loader.h
> > @@ -355,6 +355,8 @@ ssize_t rom_add_option(const char *file, int32_t bootindex);
> >   typedef struct RomLoaderNotify {
> >       /* Parameters passed to rom_add_blob() */
> > +    const char *name;
> 
> Description of the loaded ROM.
> 
> > +    uint8_t *data;
> 
> Or 'blob', blob_ptr. Maybe declare as 'const void *'?
> 
> >       hwaddr addr;
> 
> Now easier to document, where 'data' is addressed in guest memory.
> 
> >       size_t len;
> 
> Size of 'data'.

Thanks, I'll fix those

> 
> >       size_t max_len;
> 
> Still unused. Drop?

Yes

Thanks,
Jean

Re: [RFC PATCH v3 26/26] hw/arm/virt: Add measurement log for confidential boot

[Jean-Philippe Brucker]

On Thu, Dec 05, 2024 at 11:23:09PM +0100, Philippe Mathieu-Daudé wrote:
> On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> > Create a measurement log describing operations performed by QEMU to
> > initialize the guest, and load it into guest memory above the DTB.
> > 
> > Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > v2->v3: New
> > ---
> >   include/hw/arm/boot.h |  3 +++
> >   include/hw/arm/virt.h |  1 +
> >   hw/arm/boot.c         | 47 +++++++++++++++++++++++++++++++++++++++++++
> >   hw/arm/virt.c         | 23 +++++++++++++++++++++
> >   4 files changed, 74 insertions(+)
> > 
> > diff --git a/include/hw/arm/boot.h b/include/hw/arm/boot.h
> > index 5fcbaa2625..f2518c4e81 100644
> > --- a/include/hw/arm/boot.h
> > +++ b/include/hw/arm/boot.h
> > @@ -147,6 +147,9 @@ struct arm_boot_info {
> >        * Confidential guest boot loads everything into RAM so it can be measured.
> >        */
> >       bool confidential;
> > +    /* measurement log location in guest memory */
> > +    hwaddr log_start;
> 
> One expects a stop/end after "start", maybe 'log_paddr'?

Sure

Thanks,
Jean

Re: [PATCH v3 03/26] target/arm/kvm: Return immediately on error in kvm_arch_init()

[Jean-Philippe Brucker]

On Thu, Dec 05, 2024 at 10:47:13PM +0100, Philippe Mathieu-Daudé wrote:
> Hi Jean-Philippe,
> 
> On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> > Returning an error to kvm_init() is fatal anyway, no need to continue
> > the initialization.
> > 
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> >   target/arm/kvm.c | 9 +++++----
> >   1 file changed, 5 insertions(+), 4 deletions(-)
> > 
> > diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> > index 8bdf4abeb6..95bcecf804 100644
> > --- a/target/arm/kvm.c
> > +++ b/target/arm/kvm.c
> > @@ -563,7 +563,7 @@ int kvm_arch_get_default_type(MachineState *ms)
> >   int kvm_arch_init(MachineState *ms, KVMState *s)
> >   {
> > -    int ret = 0;
> > +    int ret;
> 
> With your change we can reduce this variable scope ...
> 
> >       /* For ARM interrupt delivery is always asynchronous,
> >        * whether we are using an in-kernel VGIC or not.
> >        */
> > @@ -585,7 +585,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> >           !kvm_check_extension(s, KVM_CAP_ARM_IRQ_LINE_LAYOUT_2)) {
> >           error_report("Using more than 256 vcpus requires a host kernel "
> >                        "with KVM_CAP_ARM_IRQ_LINE_LAYOUT_2");
> > -        ret = -EINVAL;
> > +        return -EINVAL;
> >       }
> >       if (kvm_check_extension(s, KVM_CAP_ARM_NISV_TO_USER)) {
> > @@ -607,13 +607,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> >               warn_report("Eager Page Split support not available");
> >           } else if (!(s->kvm_eager_split_size & sizes)) {
> >               error_report("Eager Page Split requested chunk size not valid");
> > -            ret = -EINVAL;
> > +            return -EINVAL;
> >           } else {
> >               ret = kvm_vm_enable_cap(s, KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE, 0,
> >                                       s->kvm_eager_split_size);
> 
> ... by declaring it here.

Ah right, but next patch immediately reuses ret:

@@ -627,7 +627,12 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     hw_breakpoints = g_array_sized_new(true, true,
                                        sizeof(HWBreakpoint), max_hw_bps);

-    return 0;
+    ret = kvm_arm_rme_init(ms);
+    if (ret) {
+        error_report("Failed to enable RME: %s", strerror(-ret));
+    }
+
+    return ret;
 }


> 
> Otherwise:
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Thank you!

> 
> >               if (ret < 0) {
> >                   error_report("Enabling of Eager Page Split failed: %s",
> >                                strerror(-ret));
> > +                return ret;
> >               }
> >           }
> >       }
> > @@ -626,7 +627,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> >       hw_breakpoints = g_array_sized_new(true, true,
> >                                          sizeof(HWBreakpoint), max_hw_bps);
> > -    return ret;
> > +    return 0;
> >   }
> >   unsigned long kvm_arch_vcpu_id(CPUState *cpu)
> 

Re: [PATCH v3 08/26] hw/core/loader: Add ROM loader notifier

[Jean-Philippe Brucker]

On Thu, Dec 05, 2024 at 10:59:52PM +0100, Philippe Mathieu-Daudé wrote:
> On 25/11/24 20:56, Jean-Philippe Brucker wrote:
> > Add a function to register a notifier, that is invoked after a ROM gets
> > loaded into guest memory.
> > 
> > It will be used by Arm confidential guest support, in order to register
> > all blobs loaded into memory with KVM, so that their content is moved
> > into Realm state and measured into the initial VM state.
> > 
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> >   include/hw/loader.h | 15 +++++++++++++++
> >   hw/core/loader.c    | 15 +++++++++++++++
> >   2 files changed, 30 insertions(+)
> > 
> > diff --git a/include/hw/loader.h b/include/hw/loader.h
> > index 7f6d06b956..0cd9905f97 100644
> > --- a/include/hw/loader.h
> > +++ b/include/hw/loader.h
> > @@ -353,6 +353,21 @@ void *rom_ptr_for_as(AddressSpace *as, hwaddr addr, size_t size);
> >   ssize_t rom_add_vga(const char *file);
> >   ssize_t rom_add_option(const char *file, int32_t bootindex);
> > +typedef struct RomLoaderNotify {
> > +    /* Parameters passed to rom_add_blob() */
> > +    hwaddr addr;
> 
> This is the buffer (blob) address in guest physical memory.
> 
> > +    size_t len;
> 
> This is the buffer length.
> 
> > +    size_t max_len;
> 
> This is the size of the MemoryRegion ROM containing the buffer.
> 
> Do we need to notify it? You don't use it in your next patch.
> If so, I'd rather have this API returns a MemoryRegion (Rom->mr),
> max_len can be retrieved using memory_region_size(mr); but I
> don't think we need this at all (at least for now).

No I don't think we need it either, what matters is the size of the data
copied into guest memory. I'll remove this

Thanks,
Jean

> 
> > +} RomLoaderNotify;
> > +
> > +/**
> > + * rom_add_load_notifier - Add a notifier for loaded images
> > + *
> > + * Add a notifier that will be invoked with a RomLoaderNotify structure for each
> > + * blob loaded into guest memory, after the blob is loaded.
> > + */
> > +void rom_add_load_notifier(Notifier *notifier);
> > +
> >   /* This is the usual maximum in uboot, so if a uImage overflows this, it would
> >    * overflow on real hardware too. */
> >   #define UBOOT_MAX_GUNZIP_BYTES (64 << 20)
> > diff --git a/hw/core/loader.c b/hw/core/loader.c
> > index 31593a1171..759a62cf58 100644
> > --- a/hw/core/loader.c
> > +++ b/hw/core/loader.c
> > @@ -67,6 +67,8 @@
> >   #include <zlib.h>
> >   static int roms_loaded;
> > +static NotifierList rom_loader_notifier =
> > +    NOTIFIER_LIST_INITIALIZER(rom_loader_notifier);
> >   /* return the size or -1 if error */
> >   int64_t get_image_size(const char *filename)
> > @@ -1179,6 +1181,11 @@ MemoryRegion *rom_add_blob(const char *name, const void *blob, size_t len,
> >       return mr;
> >   }
> > +void rom_add_load_notifier(Notifier *notifier)
> > +{
> > +    notifier_list_add(&rom_loader_notifier, notifier);
> > +}
> > +
> >   /* This function is specific for elf program because we don't need to allocate
> >    * all the rom. We just allocate the first part and the rest is just zeros. This
> >    * is why romsize and datasize are different. Also, this function takes its own
> > @@ -1220,6 +1227,7 @@ ssize_t rom_add_option(const char *file, int32_t bootindex)
> >   static void rom_reset(void *unused)
> >   {
> >       Rom *rom;
> > +    RomLoaderNotify notify;
> >       QTAILQ_FOREACH(rom, &roms, next) {
> >           if (rom->fw_file) {
> > @@ -1268,6 +1276,13 @@ static void rom_reset(void *unused)
> >           cpu_flush_icache_range(rom->addr, rom->datasize);
> >           trace_loader_write_rom(rom->name, rom->addr, rom->datasize, rom->isrom);
> > +
> > +        notify = (RomLoaderNotify) {
> > +            .addr = rom->addr,
> > +            .len = rom->datasize,
> > +            .max_len = rom->romsize,
> > +        };
> > +        notifier_list_notify(&rom_loader_notifier, &notify);
> >       }
> >   }
> 

Re: [PATCH v3 00/26] arm: Run Arm CCA VMs with KVM

[Gavin Shan]

Hi Jean,

On 11/26/24 5:55 AM, Jean-Philippe Brucker wrote:
> This series enables running confidential VMs on Arm CCA. The host KVM
> support is progressing but still under discussion [1], so there is no
> urgency to upstream this series. I'm sending this new version to give a
> status update, and also to discuss remote attestation below.
> 
> Since v2 [2] I addressed comments on the QAPI patches. The support for
> running Linux in a Realm will be in Linux v6.13 [3], so the guest-facing
> interface is now stable. One important change since v2 is the requirement
> to initialize the whole GPA space in RMM before boot, which we do in patch
> 9. The 'earlycon' kernel parameter now requires an unprotected address
> parameter (see Documentation/arch/arm64/arm-cca.rst in Linux v6.13).
> 
> Documentation to try this series out:
> https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/Building+an+RME+stack+for+QEMU
> 

I followed the instructions, but encountering some issues. First of all, the
recommended TF-RMM (branch: cca/v4) fails to be built. After switching to the
latest upstream TF-RMM, it fails to be initialized.

Note: the combination of the upstream TF-A + upstream TF-RMM + upstream EDK2
worked fine for the 'fvp' emulator.

TF-RMM fails to be built
========================
$ git clone https://git.codelinaro.org/linaro/dcap/rmm.git tf-rmm
$ cd tf-rmm
$ git checkout origin/cca/v4 -b cca/v4
$ git submodule update --init --recursive
$ cmake -DCMAKE_BUILD_TYPE=Debug -DRMM_CONFIG=qemu_virt_defcfg -B build-qemu
-- The C compiler identification is GNU 11.5.0
-- The CXX compiler identification is GNU 11.5.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc - broken
CMake Error at /usr/share/cmake/Modules/CMakeTestCCompiler.cmake:67 (message):
   The C compiler

     "/usr/bin/gcc"

   is not able to compile a simple test program.

   It fails with the following output:

     Change Dir: /home/gshan/sandbox/qemu/host/tf-rmm/build-qemu/CMakeFiles/CMakeScratch/TryCompile-Aab6zP
     
     Run Build Command(s):/usr/bin/cmake -E env VERBOSE=1 /usr/bin/gmake -f Makefile cmTC_87ded/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_87ded.dir/build.make CMakeFiles/cmTC_87ded.dir/build
     gmake[1]: Entering directory '/home/gshan/sandbox/qemu/host/tf-rmm/build-qemu/CMakeFiles/CMakeScratch/TryCompile-Aab6zP'
     Building C object CMakeFiles/cmTC_87ded.dir/testCCompiler.c.obj
     /usr/bin/gcc   -fno-common -ffunction-sections -fdata-sections -Wall -Werror -gdwarf-4 -ffreestanding -mbranch-protection=standard -mgeneral-regs-only -mstrict-align -fpie  -o CMakeFiles/cmTC_87ded.dir/testCCompiler.c.obj -c /home/gshan/sandbox/qemu/host/tf-rmm/build-qemu/CMakeFiles/CMakeScratch/TryCompile-Aab6zP/testCCompiler.c
     gcc: error: unrecognized command-line option ‘-mbranch-protection=standard’
     gcc: error: unrecognized command-line option ‘-mstrict-align’; did you mean ‘-Wstrict-aliasing’?
     gmake[1]: *** [CMakeFiles/cmTC_87ded.dir/build.make:78: CMakeFiles/cmTC_87ded.dir/testCCompiler.c.obj] Error 1
     gmake[1]: Leaving directory '/home/gshan/sandbox/qemu/host/tf-rmm/build-qemu/CMakeFiles/CMakeScratch/TryCompile-Aab6zP'
     gmake: *** [Makefile:127: cmTC_87ded/fast] Error 2

   CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
   CMakeLists.txt:51 (project)


-- Configuring incomplete, errors occurred!

Upstream TF-RMM fails to be initialized
========================================
$ git clone git@github.com:TF-RMM/tf-rmm.git tf-rmm
$ cd tf-rmm
$ git submodule update --init --recursive
$ export CROSS_COMPILE=aarch64-none-elf-
$ cmake -DCMAKE_BUILD_TYPE=Debug -DRMM_CONFIG=qemu_virt_defcfg -B build-qemu
$ cmake --build build-qemu

$ cd ..
$ git clone git@github.com:tianocore/edk2.git edk2
$ cd edk2
$ git submodule update --init --recursive
$ source edksetup.sh
$ make -j -C BaseTools
$ export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
$ build -b RELEASE -a AARCH64 -t GCC5 -p ArmVirtPkg/ArmVirtQemuKernel.dsc

$ cd ..
$ git@github.com:ARM-software/arm-trusted-firmware.git tf-a
$ make -j CROSS_COMPILE=aarch64-linux-gnu-     \
   PLAT=qemu ENABLE_RME=1 DEBUG=1 LOG_LEVEL=40  \
   QEMU_USE_GIC_DRIVER=QEMU_GICV3               \
   RMM=../rmm/build-qemu/Debug/rmm.img          \
   BL33=../edk2/Build/ArmVirtQemuKernel-AARCH64/RELEASE_GCC5/FV/QEMU_EFI.fd all fip
$ dd if=build/qemu/debug/bl1.bin of=flash.bin
$ dd if=build/qemu/debug/fip.bin of=flash.bin seek=64 bs=4096

$ ${HOST_PATH}/qemu/build/qemu-system-aarch64                         \
-M virt,virtualization=on,secure=on,gic-version=3,acpi=off            \
-cpu max,x-rme=on -m 64G -smp 8                                       \
-serial mon:stdio -monitor none -nographic -nodefaults                \
-bios ${HOST_PATH}/tf-a/flash.bin                                     \
-kernel ${HOST_PATH}/linux/arch/arm64/boot/Image                      \
-initrd ${HOST_PATH}/buildroot/output/images/rootfs.cpio.xz           \
-device pcie-root-port,bus=pcie.0,chassis=1,id=pcie.1                 \
-device pcie-root-port,bus=pcie.0,chassis=2,id=pcie.2                 \
-device pcie-root-port,bus=pcie.0,chassis=3,id=pcie.3                 \
-device pcie-root-port,bus=pcie.0,chassis=4,id=pcie.4                 \
-device virtio-9p-device,fsdev=shr0,mount_tag=shr0                    \
-fsdev local,security_model=none,path=${GUEST_PATH},id=shr0           \
-netdev tap,id=tap1,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-device virtio-net-pci,bus=pcie.2,netdev=tap1,mac=78:ac:44:2b:43:f0
   :
   :
NOTICE:  Booting Trusted Firmware
NOTICE:  BL1: v2.12.0(debug):ad6310a66
NOTICE:  BL1: Built : 22:09:22, Dec  7 2024
INFO:    BL1: RAM 0xe0ee000 - 0xe0f6000
INFO:    BL1: Loading BL2
INFO:    Loading image id=1 at address 0xe05b000
INFO:    Image id=1 loaded: 0xe05b000 - 0xe0652b9
NOTICE:  BL1: Booting BL2
INFO:    Entry point address = 0xe05b000
INFO:    SPSR = 0x3cd
INFO:    GPT: Boot Configuration
INFO:      PPS/T:     0x2/40
INFO:      PGS/P:     0x0/12
INFO:      L0GPTSZ/S: 0x0/30
INFO:      PAS count: 6
INFO:      L0 base:   0xeefc000
INFO:    Enabling Granule Protection Checks
NOTICE:  BL2: v2.12.0(debug):ad6310a66
NOTICE:  BL2: Built : 22:09:23, Dec  7 2024
INFO:    BL2: Doing platform setup
INFO:    Reserved RMM memory [0x40100000, 0x418fffff] in Device tree
INFO:    BL2: Loading image id 3
INFO:    Loading image id=3 at address 0xe090000
INFO:    Image id=3 loaded: 0xe090000 - 0xe0a20c4
INFO:    BL2: Loading image id 35
INFO:    Loading image id=35 at address 0x40100000
INFO:    Image id=35 loaded: 0x40100000 - 0x40303a00
INFO:    BL2: Loading image id 5
INFO:    Loading image id=5 at address 0x60000000
INFO:    Image id=5 loaded: 0x60000000 - 0x60200000
NOTICE:  BL2: Booting BL31
INFO:    Entry point address = 0xe090000
INFO:    SPSR = 0x3cd
NOTICE:  BL31: v2.12.0(debug):ad6310a66
NOTICE:  BL31: Built : 22:09:23, Dec  7 2024
INFO:    GICv3 without legacy support detected.
INFO:    ARM GICv3 driver initialized in EL3
INFO:    Maximum SPI INTID supported: 287
INFO:    BL31: Initializing runtime services
INFO:    RMM setup done.
INFO:    BL31: Initializing RMM
INFO:    RMM init start.
ERROR:   RMM init failed: -7               <<<<< RMM initialization failed
WARNING: BL31: RMM initialization failed
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x60000000
INFO:    SPSR = 0x3c9
UEFI firmware (version  built at 22:03:35 on Dec  7 2024)
  :
EFI stub: Booting Linux Kernel...
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services...
ERROR:   RMM Failed to initialize. Ignoring for CPU1
ERROR:   RMM Failed to initialize. Ignoring for CPU2
ERROR:   RMM Failed to initialize. Ignoring for CPU3
ERROR:   RMM Failed to initialize. Ignoring for CPU4
ERROR:   RMM Failed to initialize. Ignoring for CPU5
ERROR:   RMM Failed to initialize. Ignoring for CPU6
ERROR:   RMM Failed to initialize. Ignoring for CPU7
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510]

Thanks,
Gavin

Re: [PATCH v3 00/26] arm: Run Arm CCA VMs with KVM

[Gavin Shan]

Hi Jean,

On 12/11/24 1:01 PM, Gavin Shan wrote:
> On 11/26/24 5:55 AM, Jean-Philippe Brucker wrote:
>> This series enables running confidential VMs on Arm CCA. The host KVM
>> support is progressing but still under discussion [1], so there is no
>> urgency to upstream this series. I'm sending this new version to give a
>> status update, and also to discuss remote attestation below.
>>
>> Since v2 [2] I addressed comments on the QAPI patches. The support for
>> running Linux in a Realm will be in Linux v6.13 [3], so the guest-facing
>> interface is now stable. One important change since v2 is the requirement
>> to initialize the whole GPA space in RMM before boot, which we do in patch
>> 9. The 'earlycon' kernel parameter now requires an unprotected address
>> parameter (see Documentation/arch/arm64/arm-cca.rst in Linux v6.13).
>>
>> Documentation to try this series out:
>> https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/Building+an+RME+stack+for+QEMU
>>
> 
> I followed the instructions, but encountering some issues. First of all, the
> recommended TF-RMM (branch: cca/v4) fails to be built. After switching to the
> latest upstream TF-RMM, it fails to be initialized.
> 

Please ignore the issue. The RMM service can be started successfully with your
recommended tf-a/tf-rmm repositories. I don't know why tf-rmm wasn't built
successfully and all looks good after I reconnect to my x86-64 server. So
it seems the environment was corrupted.

https://git.codelinaro.org/linaro/dcap/rmm.git (branch: cca/v4)
https://git.codelinaro.org/linaro/dcap/tf-a/trusted-firmware-a.git (branch: cca/v4)

Logs from the host side
========================
INFO:    BL31: Initializing runtime services
INFO:    RMM setup done.
INFO:    BL31: Initializing RMM
INFO:    RMM init start.
Booting RMM v.0.5.0(debug) 4e2e764 Built with GCC 13.3.1
RMM-EL3 Interface v.0.4
Boot Manifest Interface v.0.3
RMI/RSI ABI v.1.0/1.0 built: Dec 11 2024 02:52:28
INFO:    RMM init end.

Thanks,
Gavin

Re: [PATCH v3 17/26] hw/arm/virt: Reserve one bit of guest-physical address for RME

[Gavin Shan]

Hi Jean,

On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> When RME is enabled, the upper GPA bit is used to distinguish protected
> from unprotected addresses. Reserve it when setting up the guest memory
> map.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   hw/arm/virt.c | 14 ++++++++++++--
>   1 file changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 9836dfbdfb..eb94997914 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -3035,14 +3035,24 @@ static int virt_kvm_type(MachineState *ms, const char *type_str)
>       VirtMachineState *vms = VIRT_MACHINE(ms);
>       int rme_vm_type = kvm_arm_rme_vm_type(ms);
>       int max_vm_pa_size, requested_pa_size;
> +    int rme_reserve_bit = 0;
>       bool fixed_ipa;
>   
> -    max_vm_pa_size = kvm_arm_get_max_vm_ipa_size(ms, &fixed_ipa);
> +    if (rme_vm_type) {
> +        /*
> +         * With RME, the upper GPA bit differentiates Realm from NS memory.
> +         * Reserve the upper bit to ensure that highmem devices will fit.
> +         */
> +        rme_reserve_bit = 1;
> +    }
> +
> +    max_vm_pa_size = kvm_arm_get_max_vm_ipa_size(ms, &fixed_ipa) -
> +                     rme_reserve_bit;
>   

For realm, @max_vm_pa_size is decreased by 1 ...

>       /* we freeze the memory map to compute the highest gpa */
>       virt_set_memmap(vms, max_vm_pa_size);
>   
> -    requested_pa_size = 64 - clz64(vms->highest_gpa);
> +    requested_pa_size = 64 - clz64(vms->highest_gpa) + rme_reserve_bit;
>   

... For realm, @requested_pa_size is increased by 1, meaning there are two bits in
the gap.

>       /*
>        * KVM requires the IPA size to be at least 32 bits.

One bit instead of two bits seems the correct gap for the followup check?

     if (requested_pa_size > max_vm_pa_size) {
         error_report("-m and ,maxmem option values "
                      "require an IPA range (%d bits) larger than "
                      "the one supported by the host (%d bits)",
                      requested_pa_size, max_vm_pa_size);
         return -1;
     }

Thanks,
Gavin

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Jean-Philippe Brucker]

On Mon, Dec 09, 2024 at 05:08:37PM -0500, Stefan Berger wrote:
> >   typedef struct {
> >       hwaddr base;
> >       hwaddr size;
> > +    uint8_t *data;
> > +    RmeLogFiletype *filetype;
> >   } RmeRamRegion;
> > +typedef struct {
> > +    char        signature[16];
> > +    char        name[32];
> > +    char        version[40];
> > +    uint64_t    ram_size;
> > +    uint32_t    num_cpus;
> > +    uint64_t    flags;
> > +} EventLogVmmVersion;
> > +
> > +typedef struct {
> > +    uint32_t    id;
> > +    uint32_t    data_size;
> > +    uint8_t     data[];
> > +} EventLogTagged;
> > +
> 
> 
> > +#define EVENT_LOG_TAG_REALM_CREATE  1
> > +#define EVENT_LOG_TAG_INIT_RIPAS    2
> > +#define EVENT_LOG_TAG_REC_CREATE    3
> > +
> If these are ARM-related structures and constants from a document you may
> want to mention the document you got them from.

Agreed. At the moment they're just numbers and structures I made up [1].
I'm not certain in which standard they should go. TCG would seem
appropriate, or IETF is also used for protocols related to confidential
computing attestation. Or maybe it could live in the reference verifier
documentation. QEMU docs wouldn't be the best place because VMMs might
been reluctant to adopt it because they don't consider it a standard (like
cloud-hv and fw_cfg)

When researching this I found TCG event types and payloads that only seem
to be documented in their respective project:
* efistub [2] with
  * EV_EVENT_TAG, id=0x8F3B22EC, data="Linux initrd",
  * EV_EVENT_TAG, id=0x8F3B22ED, data="LOADED_IMAGE::LoadOptions"
* grub [3] with a few EV_IPL
* systemd [4] with various EV_EVENT_TAG and EV_IPL

I'm wondering if we could create a common registry somewhere for these,
like IANA or somewhere informal.


[1] https://github.com/veraison/cca-realm-measurements/blob/main/docs/measurement-log.md#rim-log
[2] https://lore.kernel.org/all/20211119114745.1560453-1-ilias.apalodimas@linaro.org/
[3] https://www.gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
[4] https://systemd.io/TPM2_PCR_MEASUREMENTS/


> > +/* Log VM type and Realm Descriptor create */
> > +static int rme_log_realm_create(Error **errp)
> > +{
> > +    int ret;
> > +    ARMCPU *cpu;
> > +    EventLogVmmVersion vmm_version = {
> > +        .signature = "VM VERSION",
> > +        .name = "QEMU",
> > +        .version = "9.1", /* TODO: dynamic */
> 
> $ grep -r QEMU_VERSION_M build/
> build/config-host.h:#define QEMU_VERSION_MAJOR 9
> build/config-host.h:#define QEMU_VERSION_MICRO 93
> build/config-host.h:#define QEMU_VERSION_MINOR 1
> 
> $ cat VERSION
> 9.1.93

Ah yes that would work, thank you

> > +static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], Error **errp)
> > +{
> 
> $ ./scripts/checkpatch.pl ./tmp/*.patch
> [...]
> Checking ./tmp/0002-target-arm-kvm-rme-Add-measurement-log.patch...
> WARNING: line over 80 characters
> #353: FILE: target/arm/kvm-rme.c:303:
> +static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], Error
> **errp)
> 
> May want to run this on all patches.
> 
> Rest LGTM.

Thank you!

Jean

Re: [RFC PATCH v3 23/26] hw/tpm: Add TPM event log

[Jean-Philippe Brucker]

On Mon, Dec 09, 2024 at 05:34:13PM -0500, Stefan Berger wrote:
> 
> 
> On 11/25/24 2:56 PM, Jean-Philippe Brucker wrote:
> > Provide a library allowing the VMM to create an event log that describes
> > what is loaded into memory. During remote attestation in confidential
> > computing this helps an independent verifier reconstruct the initial
> > measurements of a VM, which contain the initial state of memory and
> > CPUs.
> > 
> > We provide some definitions and structures described by the Trusted
> > Computing Group (TCG) in "TCG PC Client Platform Firmware Profile
> > Specification" Level 00 Version 1.06 Revision 52 [1]. This is the same
> > format used by UEFI, and UEFI could reuse this log after finding it in
> 
> as used by
> 
> > DT or ACPI tables, but can also copy its content into a new one.
> 
> I thought it was going to be a completely independent log. If UEFI would do
> anything with it, I think it would have to replay the measurements into its
> own log and extend them into all PCRs of all active PCR banks of the TPM,

UEFI does need an event log, because it will measure some images using the
RMM's Realm Extensible Measurement (REM) registers, but nothing forces us
to use the same log. To reuse the existing measurement infrastructure
those REM registers can be mapped to the PCR numbers already used by
UEFI's TPM support, like Intel did for TDX:

https://uefi.org/specs/UEFI/2.10/38_Confidential_Computing.html#intel-trust-domain-extension

So for Arm the RIM could map to PCR[0], and the four REMs could map to
PCR[1,7], PCR[2-6] etc.

> but if I understand correctly then you do not use the TPM for this log at
> all since you have a signature over it and defined (somewhere -- where?)
> that only sha256 and sha512 are to be used for this log.

The algorithm choice matches that of RMM, which only support sha256 and
sha512 at the moment. But it's arbitrary. We could use any TCG algorithm
for the log digests.

Thanks,
Jean

Re: [RFC PATCH v3 25/26] target/arm/kvm-rme: Add measurement log

[Stefan Berger]

On 12/13/24 9:21 AM, Jean-Philippe Brucker wrote:
> On Mon, Dec 09, 2024 at 05:08:37PM -0500, Stefan Berger wrote:
>>>    typedef struct {
>>>        hwaddr base;
>>>        hwaddr size;
>>> +    uint8_t *data;
>>> +    RmeLogFiletype *filetype;
>>>    } RmeRamRegion;
>>> +typedef struct {
>>> +    char        signature[16];
>>> +    char        name[32];
>>> +    char        version[40];
>>> +    uint64_t    ram_size;
>>> +    uint32_t    num_cpus;
>>> +    uint64_t    flags;
>>> +} EventLogVmmVersion;
>>> +
>>> +typedef struct {
>>> +    uint32_t    id;
>>> +    uint32_t    data_size;
>>> +    uint8_t     data[];
>>> +} EventLogTagged;
>>> +
>>
>>
>>> +#define EVENT_LOG_TAG_REALM_CREATE  1
>>> +#define EVENT_LOG_TAG_INIT_RIPAS    2
>>> +#define EVENT_LOG_TAG_REC_CREATE    3
>>> +
>> If these are ARM-related structures and constants from a document you may
>> want to mention the document you got them from.
> 
> Agreed. At the moment they're just numbers and structures I made up [1].

I looked through old TCG specs (likely there are newer ones that mention 
this as well, but don't currently know) that have some definitions for 
EV_EVENT_TAG with a specific structure for logging the event. You seem 
to use the same structure:
https://trustedcomputinggroup.org/wp-content/uploads/TCG_PCClientImplementation_1-21_1_00.pdf


11.3.2.1:

typedef struct tdTCG_PCClientTaggedEventStruct {
UINT32 EventID;
UINT32 EventDataSize;
BYTE[EventDataSize] EventData;
} TCG_PCClientTaggedEventStruct;

This seems to be your EventLogTagged.

EV_EVENT_TAG has tags defined for x86_64 defined in 11.3.2.3.{1 .. 13} 
in that spec with numbers 0 .. 0xe. You would be clashing with those 
numbers but otoh the are platform-specific.

> I'm not certain in which standard they should go. TCG would seem
> appropriate, or IETF is also used for protocols related to confidential

The log is from TCG, so it would probably have to go into some sort of 
TCG registry.

> computing attestation. Or maybe it could live in the reference verifier
> documentation. QEMU docs wouldn't be the best place because VMMs might
> been reluctant to adopt it because they don't consider it a standard (like
> cloud-hv and fw_cfg)
> 
> When researching this I found TCG event types and payloads that only seem
> to be documented in their respective project:
> * efistub [2] with
>    * EV_EVENT_TAG, id=0x8F3B22EC, data="Linux initrd",
>    * EV_EVENT_TAG, id=0x8F3B22ED, data="LOADED_IMAGE::LoadOptions"

Maybe the first id is just a random number or in a vendor-specific space 
that I don't know about. My guess is they likely try to avoid clashes 
with existing numbers.


> * grub [3] with a few EV_IPL
> * systemd [4] with various EV_EVENT_TAG and EV_IPL
> 
> I'm wondering if we could create a common registry somewhere for these,
> like IANA or somewhere informal.
 > >
> [1] https://github.com/veraison/cca-realm-measurements/blob/main/docs/measurement-log.md#rim-log
> [2] https://lore.kernel.org/all/20211119114745.1560453-1-ilias.apalodimas@linaro.org/
> [3] https://www.gnu.org/software/grub/manual/grub/html_node/Measured-Boot.html
> [4] https://systemd.io/TPM2_PCR_MEASUREMENTS/
> 
> 
>>> +/* Log VM type and Realm Descriptor create */
>>> +static int rme_log_realm_create(Error **errp)
>>> +{
>>> +    int ret;
>>> +    ARMCPU *cpu;
>>> +    EventLogVmmVersion vmm_version = {
>>> +        .signature = "VM VERSION",
>>> +        .name = "QEMU",
>>> +        .version = "9.1", /* TODO: dynamic */
>>
>> $ grep -r QEMU_VERSION_M build/
>> build/config-host.h:#define QEMU_VERSION_MAJOR 9
>> build/config-host.h:#define QEMU_VERSION_MICRO 93
>> build/config-host.h:#define QEMU_VERSION_MINOR 1
>>
>> $ cat VERSION
>> 9.1.93
> 
> Ah yes that would work, thank you
> 
>>> +static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], Error **errp)
>>> +{
>>
>> $ ./scripts/checkpatch.pl ./tmp/*.patch
>> [...]
>> Checking ./tmp/0002-target-arm-kvm-rme-Add-measurement-log.patch...
>> WARNING: line over 80 characters
>> #353: FILE: target/arm/kvm-rme.c:303:
>> +static int rme_log_rec(uint64_t flags, uint64_t pc, uint64_t gprs[8], Error
>> **errp)
>>
>> May want to run this on all patches.
>>
>> Rest LGTM.
> 
> Thank you!
> 
> Jean
> 

Re: [PATCH v3 17/26] hw/arm/virt: Reserve one bit of guest-physical address for RME

[Jean-Philippe Brucker]

Hi Gavin,

On Fri, Dec 13, 2024 at 10:03:08PM +1000, Gavin Shan wrote:
> Hi Jean,
> 
> On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> > When RME is enabled, the upper GPA bit is used to distinguish protected
> > from unprotected addresses. Reserve it when setting up the guest memory
> > map.
> > 
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> >   hw/arm/virt.c | 14 ++++++++++++--
> >   1 file changed, 12 insertions(+), 2 deletions(-)
> > 
> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> > index 9836dfbdfb..eb94997914 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -3035,14 +3035,24 @@ static int virt_kvm_type(MachineState *ms, const char *type_str)
> >       VirtMachineState *vms = VIRT_MACHINE(ms);
> >       int rme_vm_type = kvm_arm_rme_vm_type(ms);
> >       int max_vm_pa_size, requested_pa_size;
> > +    int rme_reserve_bit = 0;
> >       bool fixed_ipa;
> > -    max_vm_pa_size = kvm_arm_get_max_vm_ipa_size(ms, &fixed_ipa);
> > +    if (rme_vm_type) {
> > +        /*
> > +         * With RME, the upper GPA bit differentiates Realm from NS memory.
> > +         * Reserve the upper bit to ensure that highmem devices will fit.
> > +         */
> > +        rme_reserve_bit = 1;
> > +    }
> > +
> > +    max_vm_pa_size = kvm_arm_get_max_vm_ipa_size(ms, &fixed_ipa) -
> > +                     rme_reserve_bit;
> 
> For realm, @max_vm_pa_size is decreased by 1 ...
> 
> >       /* we freeze the memory map to compute the highest gpa */
> >       virt_set_memmap(vms, max_vm_pa_size);
> > -    requested_pa_size = 64 - clz64(vms->highest_gpa);
> > +    requested_pa_size = 64 - clz64(vms->highest_gpa) + rme_reserve_bit;
> 
> ... For realm, @requested_pa_size is increased by 1, meaning there are two bits in
> the gap.

I think it's a 1-bit gap: max_vm_pa_size is decreased by 1 for the purpose
of memory map calculation, and here we increase by 1 what comes out of
that calculation, for the KVM IPA size setting

> 
> >       /*
> >        * KVM requires the IPA size to be at least 32 bits.
> 
> One bit instead of two bits seems the correct gap for the followup check?

Yes this check seems wrong for realm, since (requested_pa_size ==
max_vm_pa_size + 1) should be valid in this case. I'll fix this.

Thanks,
Jean


> 
>     if (requested_pa_size > max_vm_pa_size) {
>         error_report("-m and ,maxmem option values "
>                      "require an IPA range (%d bits) larger than "
>                      "the one supported by the host (%d bits)",
>                      requested_pa_size, max_vm_pa_size);
>         return -1;
>     }
> 
> Thanks,
> Gavin
> 

Re: [PATCH v3 06/26] target/arm/kvm-rme: Initialize vCPU

[Gavin Shan]

On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> The target code calls kvm_arm_vcpu_init() to mark the vCPU as part of a
> Realm. For a Realm vCPU, only x0-x7 can be set at runtime. Before boot,
> the PC can also be set, and is ignored at runtime. KVM also accepts a
> few system register changes during initial configuration, as returned by
> KVM_GET_REG_LIST.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   target/arm/cpu.h     |  3 +++
>   target/arm/kvm_arm.h | 15 +++++++++++
>   target/arm/kvm-rme.c | 10 ++++++++
>   target/arm/kvm.c     | 61 ++++++++++++++++++++++++++++++++++++++++++++
>   4 files changed, 89 insertions(+)
> 
> diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> index d86e641280..f617591921 100644
> --- a/target/arm/cpu.h
> +++ b/target/arm/cpu.h
> @@ -961,6 +961,9 @@ struct ArchCPU {
>       OnOffAuto kvm_steal_time;
>   #endif /* CONFIG_KVM */
>   
> +    /* Realm Management Extension */
> +    bool kvm_rme;
> +
>       /* Uniprocessor system with MP extensions */
>       bool mp_is_up;
>   
> diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
> index 9d6a89f9b1..8b52a881b0 100644
> --- a/target/arm/kvm_arm.h
> +++ b/target/arm/kvm_arm.h
> @@ -245,6 +245,16 @@ int kvm_arm_rme_init(MachineState *ms);
>    */
>   int kvm_arm_rme_vm_type(MachineState *ms);
>   
> +/**
> + * kvm_arm_rme_vcpu_init
> + * @cs: the CPU
> + *
> + * If the user requested a Realm, setup the given vCPU accordingly. Realm vCPUs
> + * behave a little differently, for example most of their register state is
> + * hidden from the host.
> + */
> +int kvm_arm_rme_vcpu_init(CPUState *cs);
> +
>   #else
>   
>   /*
> @@ -339,6 +349,11 @@ static inline int kvm_arm_rme_vm_type(MachineState *ms)
>       g_assert_not_reached();
>   }
>   
> +static inline int kvm_arm_rme_vcpu_init(CPUState *cs)
> +{
> +    g_assert_not_reached();
> +}
> +
>   #endif
>   
>   #endif
> diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
> index 60d967a842..e3cc37538a 100644
> --- a/target/arm/kvm-rme.c
> +++ b/target/arm/kvm-rme.c
> @@ -137,6 +137,16 @@ int kvm_arm_rme_init(MachineState *ms)
>       return 0;
>   }
>   
> +int kvm_arm_rme_vcpu_init(CPUState *cs)
> +{
> +    ARMCPU *cpu = ARM_CPU(cs);
> +
> +    if (rme_guest) {
> +        cpu->kvm_rme = true;
> +    }
> +    return 0;
> +}
> +
>   int kvm_arm_rme_vm_type(MachineState *ms)
>   {
>       if (rme_guest) {
> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> index 0c80992f7c..a0de2efc41 100644
> --- a/target/arm/kvm.c
> +++ b/target/arm/kvm.c
> @@ -1926,6 +1926,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
>           return ret;
>       }
>   
> +    ret = kvm_arm_rme_vcpu_init(cs);
> +    if (ret) {
> +        return ret;
> +    }
> +
>       if (cpu_isar_feature(aa64_sve, cpu)) {
>           ret = kvm_arm_sve_set_vls(cpu);
>           if (ret) {
> @@ -2062,6 +2067,35 @@ static int kvm_arch_put_sve(CPUState *cs)
>       return 0;
>   }
>   
> +static int kvm_arm_rme_put_core_regs(CPUState *cs, Error **errp)
> +{
> +    int i, ret;
> +    struct kvm_one_reg reg;
> +    ARMCPU *cpu = ARM_CPU(cs);
> +    CPUARMState *env = &cpu->env;
> +
> +    /*
> +     * The RME ABI only allows us to set 8 GPRs and the PC
> +     */

Needn't to span for multiple lines.

> +    for (i = 0; i < 8; i++) {
> +        reg.id = AARCH64_CORE_REG(regs.regs[i]);
> +        reg.addr = (uintptr_t) &env->xregs[i];
> +        ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
> +        if (ret) {
> +            return ret;
> +        }
> +    }
> +
> +    reg.id = AARCH64_CORE_REG(regs.pc);
> +    reg.addr = (uintptr_t) &env->pc;
> +    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
> +    if (ret) {
> +        return ret;
> +    }
> +
> +    return 0;
> +}
> +

Nice place to use kvm_set_one_reg(). With it, @reg can be dropped.

>   static int kvm_arm_put_core_regs(CPUState *cs, int level, Error **errp)
>   {
>       uint64_t val;
> @@ -2072,6 +2106,10 @@ static int kvm_arm_put_core_regs(CPUState *cs, int level, Error **errp)
>       ARMCPU *cpu = ARM_CPU(cs);
>       CPUARMState *env = &cpu->env;
>   
> +    if (cpu->kvm_rme) {
> +        return kvm_arm_rme_put_core_regs(cs, errp);
> +    }
> +
>       /* If we are in AArch32 mode then we need to copy the AArch32 regs to the
>        * AArch64 registers before pushing them out to 64-bit KVM.
>        */
> @@ -2259,6 +2297,25 @@ static int kvm_arch_get_sve(CPUState *cs)
>       return 0;
>   }
>   
> +static int kvm_arm_rme_get_core_regs(CPUState *cs, Error **errp)
> +{
> +    int i, ret;
> +    struct kvm_one_reg reg;
> +    ARMCPU *cpu = ARM_CPU(cs);
> +    CPUARMState *env = &cpu->env;
> +
> +    for (i = 0; i < 8; i++) {
> +        reg.id = AARCH64_CORE_REG(regs.regs[i]);
> +        reg.addr = (uintptr_t) &env->xregs[i];
> +        ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, &reg);
> +        if (ret) {
> +            return ret;
> +        }
> +    }
> +
> +    return 0;
> +}
> +

Similiarly, kvm_get_one_reg() can be used.

>   static int kvm_arm_get_core_regs(CPUState *cs, Error **errp)
>   {
>       uint64_t val;
> @@ -2269,6 +2326,10 @@ static int kvm_arm_get_core_regs(CPUState *cs, Error **errp)
>       ARMCPU *cpu = ARM_CPU(cs);
>       CPUARMState *env = &cpu->env;
>   
> +    if (cpu->kvm_rme) {
> +        return kvm_arm_rme_get_core_regs(cs, errp);
> +    }
> +
>       for (i = 0; i < 31; i++) {
>           ret = kvm_get_one_reg(cs, AARCH64_CORE_REG(regs.regs[i]),
>                                 &env->xregs[i]);

Thanks,
Gavin

Re: [PATCH v3 09/26] target/arm/kvm-rme: Initialize Realm memory

[Gavin Shan]

On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> Initialize the IPA state of RAM. Collect the images copied into guest
> RAM into a sorted list, and issue POPULATE_REALM KVM ioctls once we've
> created the Realm Descriptor. The images are part of the Realm Initial
> Measurement.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: RIPAS is now initialized separately
> ---
>   target/arm/kvm_arm.h |  14 +++++
>   target/arm/kvm-rme.c | 128 +++++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 142 insertions(+)
> 
> diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h
> index 8b52a881b0..67db09a424 100644
> --- a/target/arm/kvm_arm.h
> +++ b/target/arm/kvm_arm.h
> @@ -255,6 +255,16 @@ int kvm_arm_rme_vm_type(MachineState *ms);
>    */
>   int kvm_arm_rme_vcpu_init(CPUState *cs);
>   
> +/*
> + * kvm_arm_rme_init_guest_ram
> + * @base: base address of RAM
> + * @size: size of RAM
> + *
> + * If the user requested a Realm, set the base and size of guest RAM, in order
> + * to initialize the Realm IPA space.
> + */
> +void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size);
> +
>   #else
>   
>   /*
> @@ -281,6 +291,10 @@ static inline bool kvm_arm_mte_supported(void)
>       return false;
>   }
>   
> +static inline void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size)
> +{
> +}
> +
>   /*
>    * These functions should never actually be called without KVM support.
>    */
> diff --git a/target/arm/kvm-rme.c b/target/arm/kvm-rme.c
> index e3cc37538a..83a29421df 100644
> --- a/target/arm/kvm-rme.c
> +++ b/target/arm/kvm-rme.c
> @@ -9,6 +9,7 @@
>   #include "exec/confidential-guest-support.h"
>   #include "hw/boards.h"
>   #include "hw/core/cpu.h"
> +#include "hw/loader.h"
>   #include "kvm_arm.h"
>   #include "migration/blocker.h"
>   #include "qapi/error.h"
> @@ -20,16 +21,85 @@
>   #define TYPE_RME_GUEST "rme-guest"
>   OBJECT_DECLARE_SIMPLE_TYPE(RmeGuest, RME_GUEST)
>   
> +#define RME_PAGE_SIZE qemu_real_host_page_size()
> +
>   struct RmeGuest {
>       ConfidentialGuestSupport parent_obj;
> +    Notifier rom_load_notifier;
> +    GSList *ram_regions;
> +
> +    hwaddr ram_base;
> +    size_t ram_size;
>   };
>   

s/size_t/hwaddr. To be consistent with RmeRamRegion, we may reuse
it like below.

struct RmeGuest {
     :
     GSlist *populate_ram_regions;
     RmeRamRegion init_ram_region;
};

>   OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
>                                             CONFIDENTIAL_GUEST_SUPPORT,
>                                             { TYPE_USER_CREATABLE }, { })
>   
> +typedef struct {
> +    hwaddr base;
> +    hwaddr size;
> +} RmeRamRegion;
> +
>   static RmeGuest *rme_guest;
>   
> +static int rme_init_ram(hwaddr base, size_t size, Error **errp)
> +{
> +    int ret;
> +    uint64_t start = QEMU_ALIGN_DOWN(base, RME_PAGE_SIZE);
> +    uint64_t end = QEMU_ALIGN_UP(base + size, RME_PAGE_SIZE);
> +    struct kvm_cap_arm_rme_init_ipa_args init_args = {
> +        .init_ipa_base = start,
> +        .init_ipa_size = end - start,
> +    };
> +
> +    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
> +                            KVM_CAP_ARM_RME_INIT_IPA_REALM,
> +                            (intptr_t)&init_args);
> +    if (ret) {
> +        error_setg_errno(errp, -ret,
> +                         "failed to init RAM [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
                                                      ^^^^^^^^^^        ^^^^^^^^^^^
The type for 'start' and 'end' would be 'hwaddr'.

> +                         start, end);
> +    }
> +
> +    return ret;
> +}
> +
> +static int rme_populate_range(hwaddr base, size_t size, bool measure,
> +                              Error **errp)
> +{
> +    int ret;
> +    uint64_t start = QEMU_ALIGN_DOWN(base, RME_PAGE_SIZE);
> +    uint64_t end = QEMU_ALIGN_UP(base + size, RME_PAGE_SIZE);
> +    struct kvm_cap_arm_rme_populate_realm_args populate_args = {
> +        .populate_ipa_base = start,
> +        .populate_ipa_size = end - start,
> +        .flags = measure ? KVM_ARM_RME_POPULATE_FLAGS_MEASURE : 0,
> +    };
> +
> +    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
> +                            KVM_CAP_ARM_RME_POPULATE_REALM,
> +                            (intptr_t)&populate_args);
> +    if (ret) {
> +        error_setg_errno(errp, -ret,
> +                   "failed to populate realm [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
> +                   start, end);
> +    }
> +    return ret;
> +}
> +
> +static void rme_populate_ram_region(gpointer data, gpointer err)
> +{
> +    Error **errp = err;
> +    const RmeRamRegion *region = data;
> +
> +    if (*errp) {
> +        return;
> +    }
> +
> +    rme_populate_range(region->base, region->size, /* measure */ true, errp);
> +}
> +
>   static int rme_init_cpus(Error **errp)
>   {
>       int ret;
> @@ -60,6 +130,16 @@ static int rme_create_realm(Error **errp)
>           return -1;
>       }
>   
> +    if (rme_init_ram(rme_guest->ram_base, rme_guest->ram_size, errp)) {
> +        return -1;
> +    }
> +
> +    g_slist_foreach(rme_guest->ram_regions, rme_populate_ram_region, errp);
> +    g_slist_free_full(g_steal_pointer(&rme_guest->ram_regions), g_free);
> +    if (*errp) {
> +        return -1;
> +    }
> +
>       if (rme_init_cpus(errp)) {
>           return -1;
>       }
> @@ -105,6 +185,43 @@ static void rme_guest_finalize(Object *obj)
>   {
>   }
>   
> +static gint rme_compare_ram_regions(gconstpointer a, gconstpointer b)
> +{
> +        const RmeRamRegion *ra = a;
> +        const RmeRamRegion *rb = b;
> +
> +        g_assert(ra->base != rb->base);
> +        return ra->base < rb->base ? -1 : 1;
> +}
> +
> +static void rme_rom_load_notify(Notifier *notifier, void *data)
> +{
> +    RmeRamRegion *region;
> +    RomLoaderNotify *rom = data;
> +
> +    if (rom->addr == -1) {
> +        /*
> +         * These blobs (ACPI tables) are not loaded into guest RAM at reset.
> +         * Instead the firmware will load them via fw_cfg and measure them
> +         * itself.
> +         */
> +        return;
> +    }
> +
> +    region = g_new0(RmeRamRegion, 1);
> +    region->base = rom->addr;
> +    region->size = rom->len;
> +
> +    /*
> +     * The Realm Initial Measurement (RIM) depends on the order in which we
> +     * initialize and populate the RAM regions. To help a verifier
> +     * independently calculate the RIM, sort regions by GPA.
> +     */
> +    rme_guest->ram_regions = g_slist_insert_sorted(rme_guest->ram_regions,
> +                                                   region,
> +                                                   rme_compare_ram_regions);
> +}
> +
>   int kvm_arm_rme_init(MachineState *ms)
>   {
>       static Error *rme_mig_blocker;
> @@ -132,11 +249,22 @@ int kvm_arm_rme_init(MachineState *ms)
>        */
>       qemu_add_vm_change_state_handler(rme_vm_state_change, NULL);
>   
> +    rme_guest->rom_load_notifier.notify = rme_rom_load_notify;
> +    rom_add_load_notifier(&rme_guest->rom_load_notifier);
> +
>       cgs->require_guest_memfd = true;
>       cgs->ready = true;
>       return 0;
>   }
>   
> +void kvm_arm_rme_init_guest_ram(hwaddr base, size_t size)
> +{
> +    if (rme_guest) {
> +        rme_guest->ram_base = base;
> +        rme_guest->ram_size = size;
> +    }
> +}
> +
>   int kvm_arm_rme_vcpu_init(CPUState *cs)
>   {
>       ARMCPU *cpu = ARM_CPU(cs);

Thanks,
Gavin

Re: [PATCH v3 08/26] hw/core/loader: Add ROM loader notifier

[Gavin Shan]

On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> Add a function to register a notifier, that is invoked after a ROM gets
> loaded into guest memory.
> 
> It will be used by Arm confidential guest support, in order to register
> all blobs loaded into memory with KVM, so that their content is moved
> into Realm state and measured into the initial VM state.
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   include/hw/loader.h | 15 +++++++++++++++
>   hw/core/loader.c    | 15 +++++++++++++++
>   2 files changed, 30 insertions(+)
> 
> diff --git a/include/hw/loader.h b/include/hw/loader.h
> index 7f6d06b956..0cd9905f97 100644
> --- a/include/hw/loader.h
> +++ b/include/hw/loader.h
> @@ -353,6 +353,21 @@ void *rom_ptr_for_as(AddressSpace *as, hwaddr addr, size_t size);
>   ssize_t rom_add_vga(const char *file);
>   ssize_t rom_add_option(const char *file, int32_t bootindex);
>   
> +typedef struct RomLoaderNotify {
> +    /* Parameters passed to rom_add_blob() */
> +    hwaddr addr;
> +    size_t len;
> +    size_t max_len;
> +} RomLoaderNotify;
> +

I would suggest to rename it to RomLoaderNotifyData since it's the
data passed to the notifier.

> +/**
> + * rom_add_load_notifier - Add a notifier for loaded images
> + *
> + * Add a notifier that will be invoked with a RomLoaderNotify structure for each
> + * blob loaded into guest memory, after the blob is loaded.
> + */
> +void rom_add_load_notifier(Notifier *notifier);
> +
>   /* This is the usual maximum in uboot, so if a uImage overflows this, it would
>    * overflow on real hardware too. */
>   #define UBOOT_MAX_GUNZIP_BYTES (64 << 20)
> diff --git a/hw/core/loader.c b/hw/core/loader.c
> index 31593a1171..759a62cf58 100644
> --- a/hw/core/loader.c
> +++ b/hw/core/loader.c
> @@ -67,6 +67,8 @@
>   #include <zlib.h>
>   
>   static int roms_loaded;
> +static NotifierList rom_loader_notifier =
> +    NOTIFIER_LIST_INITIALIZER(rom_loader_notifier);
>   
>   /* return the size or -1 if error */
>   int64_t get_image_size(const char *filename)
> @@ -1179,6 +1181,11 @@ MemoryRegion *rom_add_blob(const char *name, const void *blob, size_t len,
>       return mr;
>   }
>   
> +void rom_add_load_notifier(Notifier *notifier)
> +{
> +    notifier_list_add(&rom_loader_notifier, notifier);
> +}
> +
>   /* This function is specific for elf program because we don't need to allocate
>    * all the rom. We just allocate the first part and the rest is just zeros. This
>    * is why romsize and datasize are different. Also, this function takes its own
> @@ -1220,6 +1227,7 @@ ssize_t rom_add_option(const char *file, int32_t bootindex)
>   static void rom_reset(void *unused)
>   {
>       Rom *rom;
> +    RomLoaderNotify notify;
>   
>       QTAILQ_FOREACH(rom, &roms, next) {
>           if (rom->fw_file) {
> @@ -1268,6 +1276,13 @@ static void rom_reset(void *unused)
>           cpu_flush_icache_range(rom->addr, rom->datasize);
>   
>           trace_loader_write_rom(rom->name, rom->addr, rom->datasize, rom->isrom);
> +
> +        notify = (RomLoaderNotify) {
> +            .addr = rom->addr,
> +            .len = rom->datasize,
> +            .max_len = rom->romsize,
> +        };
> +        notifier_list_notify(&rom_loader_notifier, &notify);
>       }
>   }
>   

Thanks,
Gavin

Re: [PATCH v3 18/26] hw/arm/boot: Mark all guest memory as RIPAS_RAM.

[Gavin Shan]

On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> All Realm IPA states are by default RIPAS_EMPTY, and accessing them in
> that state causes injection of synchronous exception. Either the loader
> or the guest needs to set IPA state to RIPAS_RAM before accessing it.
> Since a Linux guest needs all memory ready at boot [1], initialize it
> here.
> 
> [1] https://docs.kernel.org/arch/arm64/booting.html
>      https://lore.kernel.org/all/20241004144307.66199-12-steven.price@arm.com/
> 
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v2->v3: New: the Linux guest does not initialize RIPAS itself anymore,
> and expects the loader to do it.
> ---
>   hw/arm/boot.c | 4 ++++
>   1 file changed, 4 insertions(+)
> 

I think the changes in this patch can be combined to the following one,
or move this after it. It logically to initialize guest's RAM covered
by PATCH[9]. I'm not sure if there is particular reasonable why we
don't do that.

PATCH[09/26] target/arm/kvm-rme: Initialize Realm memory

> diff --git a/hw/arm/boot.c b/hw/arm/boot.c
> index 5301d8d318..a2414b1f98 100644
> --- a/hw/arm/boot.c
> +++ b/hw/arm/boot.c
> @@ -26,6 +26,7 @@
>   #include "qemu/config-file.h"
>   #include "qemu/option.h"
>   #include "qemu/units.h"
> +#include "kvm_arm.h"
>   
>   /* Kernel boot protocol is specified in the kernel docs
>    * Documentation/arm/Booting and Documentation/arm64/booting.txt
> @@ -1238,6 +1239,9 @@ void arm_load_kernel(ARMCPU *cpu, MachineState *ms, struct arm_boot_info *info)
>       info->dtb_filename = ms->dtb;
>       info->dtb_limit = 0;
>   
> +    /* Mark all Realm memory as RAM */
> +    kvm_arm_rme_init_guest_ram(info->loader_start, info->ram_size);
> +
>       /* Load the kernel.  */
>       if (!info->kernel_filename || info->firmware_loaded) {
>           arm_setup_firmware_boot(cpu, info);

Thanks,
Gavin

Re: [PATCH v3 06/26] target/arm/kvm-rme: Initialize vCPU

[Jean-Philippe Brucker]

On Tue, Feb 04, 2025 at 03:02:41PM +1000, Gavin Shan wrote:
> > +    reg.id = AARCH64_CORE_REG(regs.pc);
> > +    reg.addr = (uintptr_t) &env->pc;
> > +    ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, &reg);
> > +    if (ret) {
> > +        return ret;
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> 
> Nice place to use kvm_set_one_reg(). With it, @reg can be dropped.

Ah indeed, that's nicer

Thanks,
Jean

Re: [PATCH v3 08/26] hw/core/loader: Add ROM loader notifier

[Jean-Philippe Brucker]

On Tue, Feb 04, 2025 at 03:33:10PM +1000, Gavin Shan wrote:
> > diff --git a/include/hw/loader.h b/include/hw/loader.h
> > index 7f6d06b956..0cd9905f97 100644
> > --- a/include/hw/loader.h
> > +++ b/include/hw/loader.h
> > @@ -353,6 +353,21 @@ void *rom_ptr_for_as(AddressSpace *as, hwaddr addr, size_t size);
> >   ssize_t rom_add_vga(const char *file);
> >   ssize_t rom_add_option(const char *file, int32_t bootindex);
> > +typedef struct RomLoaderNotify {
> > +    /* Parameters passed to rom_add_blob() */
> > +    hwaddr addr;
> > +    size_t len;
> > +    size_t max_len;
> > +} RomLoaderNotify;
> > +
> 
> I would suggest to rename it to RomLoaderNotifyData since it's the
> data passed to the notifier.

Agreed

Thanks,
Jean

Re: [PATCH v3 09/26] target/arm/kvm-rme: Initialize Realm memory

[Jean-Philippe Brucker]

On Tue, Feb 04, 2025 at 03:30:00PM +1000, Gavin Shan wrote:
> > +    hwaddr ram_base;
> > +    size_t ram_size;
> >   };
> 
> s/size_t/hwaddr. To be consistent with RmeRamRegion, we may reuse
> it like below.
> 
> struct RmeGuest {
>     :
>     GSlist *populate_ram_regions;
>     RmeRamRegion init_ram_region;
> };

Good idea, I'll make that init_ram

> 
> >   OBJECT_DEFINE_SIMPLE_TYPE_WITH_INTERFACES(RmeGuest, rme_guest, RME_GUEST,
> >                                             CONFIDENTIAL_GUEST_SUPPORT,
> >                                             { TYPE_USER_CREATABLE }, { })
> > +typedef struct {
> > +    hwaddr base;
> > +    hwaddr size;
> > +} RmeRamRegion;
> > +
> >   static RmeGuest *rme_guest;
> > +static int rme_init_ram(hwaddr base, size_t size, Error **errp)
> > +{
> > +    int ret;
> > +    uint64_t start = QEMU_ALIGN_DOWN(base, RME_PAGE_SIZE);
> > +    uint64_t end = QEMU_ALIGN_UP(base + size, RME_PAGE_SIZE);
> > +    struct kvm_cap_arm_rme_init_ipa_args init_args = {
> > +        .init_ipa_base = start,
> > +        .init_ipa_size = end - start,
> > +    };
> > +
> > +    ret = kvm_vm_enable_cap(kvm_state, KVM_CAP_ARM_RME, 0,
> > +                            KVM_CAP_ARM_RME_INIT_IPA_REALM,
> > +                            (intptr_t)&init_args);
> > +    if (ret) {
> > +        error_setg_errno(errp, -ret,
> > +                         "failed to init RAM [0x%"HWADDR_PRIx", 0x%"HWADDR_PRIx")",
>                                                      ^^^^^^^^^^        ^^^^^^^^^^^
> The type for 'start' and 'end' would be 'hwaddr'.

Right, I changed everything to hwaddr

Thanks,
Jean

Re: [PATCH v3 18/26] hw/arm/boot: Mark all guest memory as RIPAS_RAM.

[Jean-Philippe Brucker]

On Tue, Feb 04, 2025 at 05:27:17PM +1000, Gavin Shan wrote:
> On 11/26/24 5:56 AM, Jean-Philippe Brucker wrote:
> > All Realm IPA states are by default RIPAS_EMPTY, and accessing them in
> > that state causes injection of synchronous exception. Either the loader
> > or the guest needs to set IPA state to RIPAS_RAM before accessing it.
> > Since a Linux guest needs all memory ready at boot [1], initialize it
> > here.
> > 
> > [1] https://docs.kernel.org/arch/arm64/booting.html
> >      https://lore.kernel.org/all/20241004144307.66199-12-steven.price@arm.com/
> > 
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > v2->v3: New: the Linux guest does not initialize RIPAS itself anymore,
> > and expects the loader to do it.
> > ---
> >   hw/arm/boot.c | 4 ++++
> >   1 file changed, 4 insertions(+)
> > 
> 
> I think the changes in this patch can be combined to the following one,
> or move this after it. It logically to initialize guest's RAM covered
> by PATCH[9]. I'm not sure if there is particular reasonable why we
> don't do that.

If I understand correctly you mean I should merge patches
9 and 18, or at least bring them closer together in the series?

I'd like to keep the hw/arm and target/arm changes in separate patches,
because it's a nice way to have smaller patches and gives a clean commit
message. But I could interleave the hw/arm and target/arm changes within
the series (where it makes sense like here), it may be easier to review

For now I pushed the reworked series to branch cca/latest
https://git.codelinaro.org/linaro/dcap/qemu/-/tree/cca/latest

Thanks,
Jean

> 
> PATCH[09/26] target/arm/kvm-rme: Initialize Realm memory