From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 630A2C77B73 for ; Thu, 20 Apr 2023 12:51:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MQ5ySF+r9SC4VSJAHM6f4uJYsR8CePhA20Voppul5VM=; b=g6Gosywr0GTiap AvJYWjLIr305Es6Ej9DJsUgKn73ru+ZqvIZ021AKUSLhR0noeSQvQSDl+83nwo914A6sVsEr9PwB+ BSDb9kX/Exj1ZnmJRryP+UJcfpVjv9Y4jgiJUeypePQ0sXakW1MFwkD3l+jJkEa6UyH0/GixvNywl 7OIxwugv+SC4LXWmymAVoAOUq+vOgZ8EMEQs5jdToeS+VLKUd0kWb/5Mtseu+nb0F+H4XIiUnDPBz m3AHtFh0w8OFqzYVOPoPNi3iDr1/zGa0LY2Up5TR2oMoHmp3+G2wAAU3/MedkAr9Lejn+Rc4KRYyS o6QlTNyS50Q0JBVOPjYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1ppTkb-0087rm-15; Thu, 20 Apr 2023 12:51:01 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1ppTkY-0087rH-2M for linux-arm-kernel@lists.infradead.org; Thu, 20 Apr 2023 12:51:00 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4276161653; Thu, 20 Apr 2023 12:50:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A613AC433D2; Thu, 20 Apr 2023 12:50:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681995057; bh=GgGNnVFx+UdtJgLfOH0azYXRhTQ7mHUfj60HpM9nJYU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=VJrN0+67m/9vhvKvmzIVP+rLfKU/rmo9wYIrU6jd9sK/EFNJd2xKbWPtZglMzN67i Nd99ucTxSbbmvHz+vza8B9gq+OGz9QmrOpE6HyCBDHCEI4z6MT7AUZ1MF57PC+oYN9 FxXpu47SG8EBre5NZer16lKd2bi1tW3YkFZH526TsteIHydIHUlKKR2MH6Tt7LlNy6 zEFHk9GYSIr/cHNlnD/iZx2fuowZMeDDN1Z2R6NnWIbmmz/Kptx1SyomVpk2okY/Vt Rpnn/hXI1cVnUI3mVMfDkay7QQRRPlI8OtP1aSf36QwBQyg6uD0UHMJyGBsE6cO+wX 3CXyLJ77jrvsA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1ppTkV-009soP-Cx; Thu, 20 Apr 2023 13:50:55 +0100 Date: Thu, 20 Apr 2023 13:50:55 +0100 Message-ID: <86ttxak98w.wl-maz@kernel.org> From: Marc Zyngier To: Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, Oliver Upton , Fuad Tabba Subject: Re: [PATCH] KVM: arm64: Ensure CPU PMU probes before pKVM host de-privilege In-Reply-To: <20230420123356.2708-1-will@kernel.org> References: <20230420123356.2708-1-will@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, oliver.upton@linux.dev, tabba@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230420_055058_845685_1CE17B5B X-CRM114-Status: GOOD ( 25.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 20 Apr 2023 13:33:56 +0100, Will Deacon wrote: > > Although pKVM supports CPU PMU emulation for non-protected guests since > 722625c6f4c5 ("KVM: arm64: Reenable pmu in Protected Mode"), this relies > on the PMU driver probing before the host has de-privileged so that the > 'kvm_arm_pmu_available' static key can still be enabled by patching the > hypervisor text. > > As it happens, both of these events hang off device_initcall() but the > PMU consistently won the race until 7755cec63ade ("arm64: perf: Move > PMUv3 driver to drivers/perf"). Since then, the host will fail to boot > when pKVM is enabled: > > | hw perfevents: enabled with armv8_pmuv3_0 PMU driver, 7 counters available > | kvm [1]: nVHE hyp BUG at: [] __kvm_nvhe_handle_host_mem_abort+0x270/0x284! > | kvm [1]: Cannot dump pKVM nVHE stacktrace: !CONFIG_PROTECTED_NVHE_STACKTRACE > | kvm [1]: Hyp Offset: 0xfffea41fbdf70000 > | Kernel panic - not syncing: HYP panic: > | PS:a00003c9 PC:0000dbe04b0c66e0 ESR:00000000f2000800 > | FAR:fffffbfffddfcf00 HPFAR:00000000010b0bf0 PAR:0000000000000000 > | VCPU:0000000000000000 > | CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-rc7-00083-g0bce6746d154 #1 > | Hardware name: QEMU QEMU Virtual Machine, BIOS 0.0.0 02/06/2015 > | Call trace: > | dump_backtrace+0xec/0x108 > | show_stack+0x18/0x2c > | dump_stack_lvl+0x50/0x68 > | dump_stack+0x18/0x24 > | panic+0x13c/0x33c > | nvhe_hyp_panic_handler+0x10c/0x190 > | aarch64_insn_patch_text_nosync+0x64/0xc8 > | arch_jump_label_transform+0x4c/0x5c > | __jump_label_update+0x84/0xfc > | jump_label_update+0x100/0x134 > | static_key_enable_cpuslocked+0x68/0xac > | static_key_enable+0x20/0x34 > | kvm_host_pmu_init+0x88/0xa4 > | armpmu_register+0xf0/0xf4 > | arm_pmu_acpi_probe+0x2ec/0x368 > | armv8_pmu_driver_init+0x38/0x44 > | do_one_initcall+0xcc/0x240 > > Fix the race properly by deferring the de-privilege step to > device_initcall_sync(). This will also be needed in future when probing > IOMMU devices and allows us to separate the pKVM de-privilege logic from > the core hypervisor initialisation path. > > Cc: Oliver Upton > Cc: Fuad Tabba > Cc: Marc Zyngier > Fixes: 7755cec63ade ("arm64: perf: Move PMUv3 driver to drivers/perf") > Signed-off-by: Will Deacon > --- > > Marc, Oliver -- in practice, this issue only crops with the patches > moving the CPU PMU driver out into drivers/perf/ and so the arm64 > for-next/core branch is broken. Please can I queue this in the arm64 > tree for 6.4 with your Ack? Thanks. It doesn't conflict with the current state of kvmarm/next, and I actually like that this code is moved into pkvm.c, so: Acked-by: Marc Zyngier Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel