From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B38CAC27C4F for ; Wed, 26 Jun 2024 13:39:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cN78iJ8U/lje3aJBxX6xVzD3ryQxStuuR9tI07xBrl4=; b=Cp1wUi1d8A4S4Iv+QDgcIV7Ilj +1D0qIuC39olpdaxlJX7nRSTKG17chwOMDSLvSS3z/6htUsjHu6yXoq/s2kjqJaWYfUMAkMup6bMo 8ouLmSRJXZQI3hOf7YjSDyirV1paA/NJmbmZGGtIDl9eBfF94eIPaIuKSfSvvQsuTETAFeFZc8FmA Z1L38RPy5Ni4Gy+U7DvWgISJYe8Qb+/4AJQ1+peJ4JZ6izNnTMkLtD5suiFwpOv61xCvNTSJLu3k3 wpEkKQsIVM3/S2s6lDn5tPgAm+JJ6UgOe9Xx3wFRjV+TdgqeUFsJ/05FS4coogsxv8iChC4goohIm egErl2UA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sMSsD-000000070gk-1pCS; Wed, 26 Jun 2024 13:39:45 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sMSs4-000000070d1-3dSF for linux-arm-kernel@lists.infradead.org; Wed, 26 Jun 2024 13:39:39 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8295B339; Wed, 26 Jun 2024 06:39:58 -0700 (PDT) Received: from [10.57.42.86] (unknown [10.57.42.86]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B14803F73B; Wed, 26 Jun 2024 06:39:29 -0700 (PDT) Message-ID: Date: Wed, 26 Jun 2024 14:39:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [v2] Support for Arm CCA VMs on Linux To: Itaru Kitayama Cc: kvm@vger.kernel.org, kvmarm@lists.linux.dev, Catalin Marinas , Marc Zyngier , Will Deacon , James Morse , Oliver Upton , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , linux-coco@lists.linux.dev, Ganapatrao Kulkarni References: <20240412084056.1733704-1-steven.price@arm.com> From: Steven Price Content-Language: en-GB In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240626_063937_174902_C2408C2E X-CRM114-Status: GOOD ( 26.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 24/06/2024 07:13, Itaru Kitayama wrote: > Hi Steven, > On Fri, Apr 12, 2024 at 09:40:56AM +0100, Steven Price wrote: >> We are happy to announce the second version of the Arm Confidential >> Compute Architecture (CCA) support for the Linux stack. The intention is >> to seek early feedback in the following areas: >> * KVM integration of the Arm CCA; >> * KVM UABI for managing the Realms, seeking to generalise the >> operations where possible with other Confidential Compute solutions; >> * Linux Guest support for Realms. >> >> See the previous RFC[1] for a more detailed overview of Arm's CCA >> solution, or visible the Arm CCA Landing page[2]. >> >> This series is based on the final RMM v1.0 (EAC5) specification[3]. >> >> Quick-start guide >> ================= >> >> The easiest way of getting started with the stack is by using >> Shrinkwrap[4]. Currently Shrinkwrap has a configuration for the initial >> v1.0-EAC5 release[5], so the following overlay needs to be applied to >> the standard 'cca-3world.yaml' file. Note that the 'rmm' component needs >> updating to 'main' because there are fixes that are needed and are not >> yet in a tagged release. The following will create an overlay file and >> build a working environment: >> >> cat<cca-v2.yaml >> build: >> linux: >> repo: >> revision: cca-full/v2 >> kvmtool: >> repo: >> kvmtool: >> revision: cca/v2 >> rmm: >> repo: >> revision: main >> kvm-unit-tests: >> repo: >> revision: cca/v2 >> EOT >> >> shrinkwrap build cca-3world.yaml --overlay buildroot.yaml --btvar GUEST_ROOTFS='${artifact:BUILDROOT}' --overlay cca-v2.yaml >> >> You will then want to modify the 'guest-disk.img' to include the files >> necessary for the realm guest (see the documentation in cca-3world.yaml >> for details of other options): >> >> cd ~/.shrinkwrap/package/cca-3world >> /sbin/e2fsck -fp rootfs.ext2 >> /sbin/resize2fs rootfs.ext2 256M >> mkdir mnt >> sudo mount rootfs.ext2 mnt/ >> sudo mkdir mnt/cca >> sudo cp guest-disk.img KVMTOOL_EFI.fd lkvm Image mnt/cca/ >> sudo umount mnt >> rmdir mnt/ >> >> Finally you can run the FVP with the host: >> >> shrinkwrap run cca-3world.yaml --rtvar ROOTFS=$HOME/.shrinkwrap/package/cca-3world/rootfs.ext2 >> >> And once the host kernel has booted, login (user name 'root') and start >> a realm guest: >> >> cd /cca >> ./lkvm run --realm --restricted_mem -c 2 -m 256 -k Image -p earlycon >> >> Be patient and you should end up in a realm guest with the host's >> filesystem mounted via p9. >> >> It's also possible to use EFI within the realm guest, again see >> cca-3world.yaml within Shrinkwrap for more details. > > I am trying to see if libvirt can work with the CCA-aware KVM with minimal Ubuntu22.10 filesystem, however virt-install triggers a system failure: > > $ sudo virt-install -v --name f39 --ram 4096 --disk path=fedora40.img,cache=none --nographics --os-variant fedora38 --import --arch aarch64 --vcpus 4 > [sudo] password for realm: > [ 3694.176579] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000e00 > [ 3694.176687] Mem abort info: > [ 3694.176745] ESR = 0x0000000096000004 > [ 3694.176817] EC = 0x25: DABT (current EL), IL = 32 bits > [ 3694.176907] SET = 0, FnV = 0 > [ 3694.176978] EA = 0, S1PTW = 0 > [ 3694.177049] FSC = 0x04: level 0 translation fault > [ 3694.177132] Data abort info: > [ 3694.177189] ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000 > [ 3694.177276] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 > [ 3694.177370] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 > [ 3694.177544] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000880f6e000 > [ 3694.177649] [0000000000000e00] pgd=0000000000000000, p4d=0000000000000000 > [ 3694.177788] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP > [ 3694.177887] Modules linked in: > [ 3694.177966] CPU: 2 PID: 540 Comm: qemu-system-aar Not tainted 6.10.0-rc1-00058-gd901c27a1783 #149 > [ 3694.178105] Hardware name: FVP Base RevC (DT) > [ 3694.178180] pstate: 61400009 (nZCv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--) > [ 3694.178315] pc : kvm_vm_ioctl_check_extension+0x1fc/0x3c4 > [ 3694.178447] lr : kvm_vm_ioctl_check_extension_generic+0x34/0x12c > [ 3694.178587] sp : ffff800081523cb0 > [ 3694.178657] x29: ffff800081523cb0 x28: 0000000000000051 x27: 0000000000000000 > [ 3694.178840] x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 > [ 3694.179019] x23: 000000000000000a x22: 0000000000000051 x21: ffff000801075f00 > [ 3694.179200] x20: ffff000801075f01 x19: 000000000000ae03 x18: 0000000000000000 > [ 3694.179383] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 > [ 3694.179565] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 > [ 3694.179745] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000 > [ 3694.179923] x8 : 0000000000000000 x7 : ffff000801075f18 x6 : 00000000401c5820 > [ 3694.180106] x5 : 000000000000000a x4 : 0000000000000800 x3 : 0000000000000000 > [ 3694.180285] x2 : 000000000000000b x1 : 0000000100061001 x0 : 0000000000000001 > [ 3694.180465] Call trace: > [ 3694.180523] kvm_vm_ioctl_check_extension+0x1fc/0x3c4 > [ 3694.180656] kvm_vm_ioctl_check_extension_generic+0x34/0x12c > [ 3694.180794] kvm_dev_ioctl+0x3c8/0x8b8 > [ 3694.180938] __arm64_sys_ioctl+0xac/0xf0 > [ 3694.181079] invoke_syscall+0x48/0x114 > [ 3694.181220] el0_svc_common.constprop.0+0x40/0xe0 > [ 3694.181367] do_el0_svc+0x1c/0x28 > [ 3694.181507] el0_svc+0x34/0xd8 > [ 3694.181608] el0t_64_sync_handler+0x120/0x12c > [ 3694.181723] el0t_64_sync+0x190/0x194 > [ 3694.181865] Code: 17ffffbd 97fffc9d 12001c00 17ffff91 (39780060) > [ 3694.181955] ---[ end trace 0000000000000000 ]--- > > I'd appreciate it if you could take a look at it. Thanks for the bug report. I believe this is because kvm_vm_ioctl_check_extension() is being called with kvm==NULL and I've missed some checks. I believe the following should get things working - and it's probably better than attempting to remember to check with the NULL kvm at each call site. ---8<---- diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 27c58bbdf50b..c85e5f566506 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -602,7 +602,7 @@ static __always_inline void kvm_reset_cptr_el2(struct kvm_vcpu *vcpu) static inline bool kvm_is_realm(struct kvm *kvm) { - if (static_branch_unlikely(&kvm_rme_is_available)) + if (static_branch_unlikely(&kvm_rme_is_available) && kvm) return kvm->arch.is_realm; return false; } ---8<---- Thanks, Steve