From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1398C02199 for ; Thu, 6 Feb 2025 21:16:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6x8IkEHoGzEoV22BcxNT1P4XAV28qH88FMQRED/bjOk=; b=H5VB/GfOOJI6D4pyaUi+lep1Az Wxn0mV9JxnVt0lcufVhpkAShLopbhGBSgAUzYIQG+mAphHe8B0fxt5rfLjQea87ctwX9vSre2+Uzq QvjfStIjhuUxfv0gL4baGnb8ou0OhV9ctw+LBGAeCBBHZohI3TYn26rt0GFOZbDJJildkMVDFGw8z AGmKxLabM3zUgNhxwitLeVsgCWX99dbAR8UGimeVNJ6AzvOi5clEVN6URDEoLTRqX/hZcp10rBJoZ p84WqBpqUAKTVkkDo5ZREDNZv4ycz7lMP1DMu2HQEGTCqoMmtPg+8t37+Bl+Fz+S/mVbomgY91SF9 a77nyheA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tg9EI-00000007YnF-0rjX; Thu, 06 Feb 2025 21:16:10 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tg9Cu-00000007YgC-1Ahj for linux-arm-kernel@lists.infradead.org; Thu, 06 Feb 2025 21:14:45 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id D1EB15C6388; Thu, 6 Feb 2025 21:14:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2AD4FC4CEDD; Thu, 6 Feb 2025 21:14:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1738876483; bh=e2SOY182HjtRf1jcrCmT/yGU2J8GDiU314PnbVwrp7g=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=gd1alUST0IO6Lv5wd3A05bXQe2+4UMAsApIHndqvLWYKsFxSJzLnSIV1pH9/FJEqq Oi/vtKp55eWJktVqKpkXp4yTqD78gZ2JEtTIG56M//S0kfut3Q3jDQb5ptyEq5BacG +wKIIYrqqsV3d4LGFrh7nZXDTaDYVpD6MxHmJYSvB41BRRfYFWNdwmkSspuGzhqrPP eEUwS7WCmQfGgwo58hQ1FGiPp0H/tp/WuNVSAJ6mvbyo9ELleMKAqT3Vsoj/DDL52s q71wajBFFmlPiKz/UXxG1M4pIua9Wx016A9MJYiMBWeMuD8VSVqvPOhkbjHAu/dzBN gwiHw8Tl3Pjgw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1tg9Cq-001J8q-PU; Thu, 06 Feb 2025 21:14:40 +0000 Date: Thu, 06 Feb 2025 21:14:39 +0000 Message-ID: <86o6zeu668.wl-maz@kernel.org> From: Marc Zyngier To: Ganapatrao Kulkarni Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, darren@os.amperecomputing.com, scott@os.amperecomputing.com Subject: Re: [RFC PATCH 1/2] KVM: arm64: nv: selftests: Add guest hypervisor test In-Reply-To: <20250206164120.4045569-2-gankulkarni@os.amperecomputing.com> References: <20250206164120.4045569-1-gankulkarni@os.amperecomputing.com> <20250206164120.4045569-2-gankulkarni@os.amperecomputing.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gankulkarni@os.amperecomputing.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, oliver.upton@linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, darren@os.amperecomputing.com, scott@os.amperecomputing.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250206_131444_408963_3EC13496 X-CRM114-Status: GOOD ( 31.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 06 Feb 2025 16:41:19 +0000, Ganapatrao Kulkarni wrote: > > This patch adds the required changes to init vcpu in vEL2 context. > Also adds a KVM selftest to execute guest code as a guest hypervisor(L1). > > Signed-off-by: Ganapatrao Kulkarni > --- > tools/testing/selftests/kvm/Makefile.kvm | 1 + > .../selftests/kvm/arm64/nv_guest_hypervisor.c | 83 +++++++++++++++++++ > .../kvm/include/arm64/kvm_util_arch.h | 3 + > .../selftests/kvm/include/arm64/nv_util.h | 28 +++++++ > .../testing/selftests/kvm/include/kvm_util.h | 1 + > .../selftests/kvm/lib/arm64/processor.c | 59 +++++++++---- > 6 files changed, 161 insertions(+), 14 deletions(-) > create mode 100644 tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c > create mode 100644 tools/testing/selftests/kvm/include/arm64/nv_util.h > > diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm > index 4277b983cace..a85d3bec9fb1 100644 > --- a/tools/testing/selftests/kvm/Makefile.kvm > +++ b/tools/testing/selftests/kvm/Makefile.kvm > @@ -154,6 +154,7 @@ TEST_GEN_PROGS_arm64 += arm64/vgic_irq > TEST_GEN_PROGS_arm64 += arm64/vgic_lpi_stress > TEST_GEN_PROGS_arm64 += arm64/vpmu_counter_access > TEST_GEN_PROGS_arm64 += arm64/no-vgic-v3 > +TEST_GEN_PROGS_arm64 += arm64/nv_guest_hypervisor > TEST_GEN_PROGS_arm64 += access_tracking_perf_test > TEST_GEN_PROGS_arm64 += arch_timer > TEST_GEN_PROGS_arm64 += coalesced_io_test > diff --git a/tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c b/tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c > new file mode 100644 > index 000000000000..5aeefe43aff7 > --- /dev/null > +++ b/tools/testing/selftests/kvm/arm64/nv_guest_hypervisor.c > @@ -0,0 +1,83 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2025 Ampere Computing LLC > + */ > +#include > +#include > +#include > +#include > + > +static void guest_code(void) > +{ > + if (read_sysreg(CurrentEL) == CurrentEL_EL2) > + GUEST_PRINTF("Executing guest code in vEL2\n"); > + else > + GUEST_FAIL("Fail to run in vEL2\n"); > + > + GUEST_DONE(); > +} > + > +static void guest_undef_handler(struct ex_regs *regs) > +{ > + GUEST_FAIL("Unexpected exception far_el1 = 0x%lx", read_sysreg(far_el1)); > +} > + > +static void test_run_vcpu(struct kvm_vcpu *vcpu) > +{ > + struct ucall uc; > + > + do { > + vcpu_run(vcpu); > + > + switch (get_ucall(vcpu, &uc)) { > + case UCALL_ABORT: > + REPORT_GUEST_ASSERT(uc); > + break; > + case UCALL_PRINTF: > + printf("%s", uc.buffer); > + break; > + case UCALL_DONE: > + printf("Test PASS\n"); > + break; > + default: > + TEST_FAIL("Unknown ucall %lu", uc.cmd); > + } > + } while (uc.cmd != UCALL_DONE); > +} > + > +static void test_nv_guest_hypervisor(void) > +{ > + struct kvm_vcpu *vcpu; > + struct kvm_vm *vm; > + struct kvm_vcpu_init init; > + int gic_fd; > + > + vm = vm_create(1); > + vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &init); > + > + init.features[0] = 0; > + init_vcpu_nested(&init); > + vcpu = aarch64_vcpu_add(vm, 0, &init, guest_code); > + > + __TEST_REQUIRE(is_vcpu_nested(vcpu), "Failed to Enable NV"); > + > + vm_init_descriptor_tables(vm); > + vcpu_init_descriptor_tables(vcpu); > + gic_fd = vgic_v3_setup(vm, 1, 64); > + __TEST_REQUIRE(gic_fd >= 0, "Failed to create vgic-v3"); > + > + vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT, > + ESR_ELx_EC_UNKNOWN, guest_undef_handler); > + > + test_run_vcpu(vcpu); > + kvm_vm_free(vm); > +} > + > +int main(int argc, char *argv[]) > +{ > + TEST_REQUIRE(kvm_has_cap(KVM_CAP_ARM_EL2)); > + > + test_nv_guest_hypervisor(); > + > + return 0; > +} > diff --git a/tools/testing/selftests/kvm/include/arm64/kvm_util_arch.h b/tools/testing/selftests/kvm/include/arm64/kvm_util_arch.h > index e43a57d99b56..ab5279c24413 100644 > --- a/tools/testing/selftests/kvm/include/arm64/kvm_util_arch.h > +++ b/tools/testing/selftests/kvm/include/arm64/kvm_util_arch.h > @@ -2,6 +2,9 @@ > #ifndef SELFTEST_KVM_UTIL_ARCH_H > #define SELFTEST_KVM_UTIL_ARCH_H > > +#define CurrentEL_EL1 (1 << 2) > +#define CurrentEL_EL2 (2 << 2) > + > struct kvm_vm_arch {}; > > #endif // SELFTEST_KVM_UTIL_ARCH_H > diff --git a/tools/testing/selftests/kvm/include/arm64/nv_util.h b/tools/testing/selftests/kvm/include/arm64/nv_util.h > new file mode 100644 > index 000000000000..4fecf1f18554 > --- /dev/null > +++ b/tools/testing/selftests/kvm/include/arm64/nv_util.h > @@ -0,0 +1,28 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * Copyright (c) 2025 Ampere Computing > + */ > +#ifndef SELFTEST_NV_UTIL_H > +#define SELFTEST_NV_UTIL_H > + > +#include > + > +/* NV helpers */ > +static inline void init_vcpu_nested(struct kvm_vcpu_init *init) > +{ > + init->features[0] |= (1 << KVM_ARM_VCPU_HAS_EL2); > +} > + > +static inline bool kvm_arm_vcpu_has_el2(struct kvm_vcpu_init *init) > +{ > + unsigned long features = init->features[0]; > + > + return test_bit(KVM_ARM_VCPU_HAS_EL2, &features); > +} > + > +static inline bool is_vcpu_nested(struct kvm_vcpu *vcpu) > +{ > + return vcpu->nested; > +} > + > +#endif /* SELFTEST_NV_UTIL_H */ > diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h > index 4c4e5a847f67..8c53dbc17f8f 100644 > --- a/tools/testing/selftests/kvm/include/kvm_util.h > +++ b/tools/testing/selftests/kvm/include/kvm_util.h > @@ -58,6 +58,7 @@ struct kvm_vcpu { > struct kvm_dirty_gfn *dirty_gfns; > uint32_t fetch_index; > uint32_t dirty_gfns_count; > + bool nested; > }; > > struct userspace_mem_regions { > diff --git a/tools/testing/selftests/kvm/lib/arm64/processor.c b/tools/testing/selftests/kvm/lib/arm64/processor.c > index 7ba3aa3755f3..35ba2ace61a2 100644 > --- a/tools/testing/selftests/kvm/lib/arm64/processor.c > +++ b/tools/testing/selftests/kvm/lib/arm64/processor.c > @@ -10,6 +10,7 @@ > > #include "guest_modes.h" > #include "kvm_util.h" > +#include "nv_util.h" > #include "processor.h" > #include "ucall_common.h" > > @@ -258,14 +259,47 @@ void virt_arch_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent) > } > } > > +static void aarch64_vcpu_set_reg(struct kvm_vcpu *vcpu, uint64_t sctlr_el1, > + uint64_t tcr_el1, uint64_t ttbr0_el1) > +{ > + uint64_t fpen; > + > + /* > + * Enable FP/ASIMD to avoid trapping when accessing Q0-Q15 > + * registers, which the variable argument list macros do. > + */ > + fpen = 3 << 20; > + > + if (is_vcpu_nested(vcpu)) { > + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_CPTR_EL2), fpen); > + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_SCTLR_EL2), sctlr_el1); > + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_TCR_EL2), tcr_el1); > + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_MAIR_EL2), DEFAULT_MAIR_EL1); > + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_TTBR0_EL2), ttbr0_el1); > + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_TPIDR_EL2), vcpu->id); How about some of the basics such as HCR_EL2, MDCR_EL2? A bunch of things there do have an impact on how the guest behaves, and relying on defaults feels like a bad idea. This also assumes VHE, without trying to enforce it. Finally, how to you plan to make all the existing tests run as EL2 guests if TPIDR_EL1 isn't populated with the expected value? Surely you need to change the read side... M. -- Without deviation from the norm, progress is not possible.