Re: [PATCH v8 0/6] AWS Nitro Enclave emulation support

[Alexander Graf <graf@amazon.com>]

On 08.10.24 23:17, Dorjoy Chowdhury wrote:
> This is v8 submission for AWS Nitro Enclave emulation in QEMU. From the QEMU side
> the implementation for nitro enclaves is complete. v7 is at:
> https://lore.kernel.org/qemu-devel/20240922094441.23802-1-dorjoychy111@gmail.com/T/#t
>
> Changes in v8:
>      - create_default_memdev related changes have been made into a separate commit
>      - PCR states are now saved during migration
>      - some changes in virtio-nsm.c for making the implementation similar to real
> NSM in AWS which include returning the same error, ignoring unknown properties in
> nested map etc. I got a chance to test real NSM in AWS and I prepared a script to
> test various NSM operations and error conditions which I am maintaining a git repo
> here: https://github.com/dorjoy03/nsm-api . The README has some details for testing.
>
> Changes in v7:
>      - rebased with master as the first 3 crypto patches got in master now
>      - had to change some QCRYPTO_HASH_ALG refs to QCRYPTO_HASH_ALGO (with an 'O')
> as that got changed in master
>      - had to re-do the libvirt-ci update patch as I got a conflict from master
>      - renamed handle_* functions in virtio-nsm.c to be not camel case
>
> Changes in v6:
>      - updated MAINTAINERS in the commit that introduced eif.c, eif.h files
>      - used iov_to_buf and iov_from_buf in virtio-nsm.c handle_input
>      - used g_new0 for iovec_list instead of g_malloc
>      - updated documentation as now the vsock backend work[6] has been merged in
> rust-vmm's vhost-device-vsock
>
> Changes in v5:
>      - bunch of use of glib utilities like g_autofree, g_memdup2, GList etc
>      - updated libvirt-ci and added libcbor dependency
>      - files in this patch are built under libcbor and gnutls dependency check now and
> libcbor dependency has been moved to root meson.build file
>      - separated and re-ordered commits as suggested
>      - user_data and nonce are added as null to attestation when empty and payload_map_size is fixed
>      - variables in eif.c have been shortened for readability
>
> Changes in v4:
>      - fixed error_setv assertion failed. I could not reproduce this but I think
> this was happening because I did not set Error *err = NULL in x86_load_eif
>      - qemu_cbor.. helpers moved to a separate file now
>      - libcbor version requirement reduced from 0.8.0 to 0.7.0
>      - replaced GChecksum uses with qcrypto apis
>      - timestamp multiplied by 1000 in virtio-nsm
>      - user_data and nonce are now included in attestation even when they are empty
>      - added x509-utils in crypto
>      - added G_CHECKSUM_SHA384 support in hash-glib.c
>      - PCR3 and PCR4 can be set from nitro-enclave machine options. I did not add
> the options for virtio-nsm device though. I think the PCR states are set by
> machines so it made sense to add the options for machine only.
>
> Changes in v3:
>      - Support for virtio-nsm device
>      - The EIF related logic has been removed from microvm.c i.e., the logic is
> contained in enclave related code
>      - For vsock emulation in nitro-enclave, now vhost-user-vsock is being used
> instead of vhost-vsock (more details in the cover-letter below)
>      - updated documentation accordingly
>
> Changes in v2:
>      - moved eif.c and eif.h files from hw/i386 to hw/core
>
> Hi,
>
> Hope everyone is doing well. This is a patch series adding AWS Nitro Enclave[1]
> emulation support in QEMU. Alexander Graf is mentoring me on this work. I have
> a gitlab branch where you can view the patches in the gitlab web UI for each commit:
> https://gitlab.com/dorjoy03/qemu/-/tree/nitro-enclave-emulation
>
> AWS nitro enclaves is an Amazon EC2[2] feature that allows creating isolated
> execution environments, called enclaves, from Amazon EC2 instances, which are
> used for processing highly sensitive data. Enclaves have no persistent storage
> and no external networking. The enclave VMs are based on Firecracker microvm
> and have a vhost-vsock device for communication with the parent EC2 instance
> that spawned it and a Nitro Secure Module (NSM) device for cryptographic
> attestation. The parent instance VM always has CID 3 while the enclave VM gets
> a dynamic CID. The enclave VMs can communicate with the parent instance over
> various ports to CID 3, for example, the init process inside an enclave sends a
> heartbeat to port 9000 upon boot, expecting a heartbeat reply, letting the
> parent instance know that the enclave VM has successfully booted.
>
>  From inside an EC2 instance, nitro-cli[3] is used to spawn an enclave VM using
> an EIF (Enclave Image Format)[4] file. EIF files can be built using nitro-cli
> as well. The EIF specification can be found in the README of the github
> aws-nitro-enclaves-image-format repository[4]. An EIF file contains the kernel,
> cmdline and ramdisk(s) in different sections which are used to boot the enclave
> VM.
>
> Adding nitro enclave emulation support in QEMU will make the life of AWS Nitro
> Enclave users easier as they will be able to test their EIF images locally
> without having to run real nitro enclaves which can be difficult for debugging
> due to its roots in security. This will also make quick prototyping easier.
>
> In QEMU, the new nitro-enclave machine type is implemented based on the microvm
> machine type similar to how AWS Nitro Enclaves are based on Firecracker microvm.
>
> The vsock emulation support is added using vhost-user-vsock device. This is
> needed as nitro VMs always talk to parent VM (CID 3) but there is no support for
> sibling VM communication in vhost-vsock. So to run nitro-enclave, a process that
> does vsock emulation in user-space like vhost-device-vsock[5] from rust-vmm must
> be run. I am working on adding proxying using vsock (right now it uses unix
> domain socket) to the host machine in vhost-device-vsock which I have already
> posted a PR[6] in rust-vmm repo. This will allow users to run the necessary parent
> VM applications in the host machine instead of a separate VM with CID 3. Update:
> this has been merged now.
>
> A new device virtio-nsm support has been added to QEMU. This device is built-into
> the nitro-enclave VM. The virtio-nsm spec can be found here[7].
>
> For local testing you need to generate a hello.eif image by first building
> nitro-cli locally[8]. Then you can use nitro-cli to build a hello.eif image[9].
> More details about testing can be found in the docs/system/i386/nitro-enclave.rst
> file.


Reviewed-by: Alexander Graf <graf@amazon.com>


Alex





Amazon Web Services Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597