Re: [PATCH] KVM: arm64: Ensure CPU PMU probes before pKVM host de-privilege

From: Marc Zyngier <maz@kernel.org>
To: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
	Oliver Upton <oliver.upton@linux.dev>,
	Fuad Tabba <tabba@google.com>
Subject: Re: [PATCH] KVM: arm64: Ensure CPU PMU probes before pKVM host de-privilege
Date: Thu, 20 Apr 2023 13:50:55 +0100	[thread overview]
Message-ID: <86ttxak98w.wl-maz@kernel.org> (raw)
In-Reply-To: <20230420123356.2708-1-will@kernel.org>

On Thu, 20 Apr 2023 13:33:56 +0100,
Will Deacon <will@kernel.org> wrote:
> 
> Although pKVM supports CPU PMU emulation for non-protected guests since
> 722625c6f4c5 ("KVM: arm64: Reenable pmu in Protected Mode"), this relies
> on the PMU driver probing before the host has de-privileged so that the
> 'kvm_arm_pmu_available' static key can still be enabled by patching the
> hypervisor text.
> 
> As it happens, both of these events hang off device_initcall() but the
> PMU consistently won the race until 7755cec63ade ("arm64: perf: Move
> PMUv3 driver to drivers/perf"). Since then, the host will fail to boot
> when pKVM is enabled:
> 
>   | hw perfevents: enabled with armv8_pmuv3_0 PMU driver, 7 counters available
>   | kvm [1]: nVHE hyp BUG at: [<ffff8000090366e0>] __kvm_nvhe_handle_host_mem_abort+0x270/0x284!
>   | kvm [1]: Cannot dump pKVM nVHE stacktrace: !CONFIG_PROTECTED_NVHE_STACKTRACE
>   | kvm [1]: Hyp Offset: 0xfffea41fbdf70000
>   | Kernel panic - not syncing: HYP panic:
>   | PS:a00003c9 PC:0000dbe04b0c66e0 ESR:00000000f2000800
>   | FAR:fffffbfffddfcf00 HPFAR:00000000010b0bf0 PAR:0000000000000000
>   | VCPU:0000000000000000
>   | CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-rc7-00083-g0bce6746d154 #1
>   | Hardware name: QEMU QEMU Virtual Machine, BIOS 0.0.0 02/06/2015
>   | Call trace:
>   |  dump_backtrace+0xec/0x108
>   |  show_stack+0x18/0x2c
>   |  dump_stack_lvl+0x50/0x68
>   |  dump_stack+0x18/0x24
>   |  panic+0x13c/0x33c
>   |  nvhe_hyp_panic_handler+0x10c/0x190
>   |  aarch64_insn_patch_text_nosync+0x64/0xc8
>   |  arch_jump_label_transform+0x4c/0x5c
>   |  __jump_label_update+0x84/0xfc
>   |  jump_label_update+0x100/0x134
>   |  static_key_enable_cpuslocked+0x68/0xac
>   |  static_key_enable+0x20/0x34
>   |  kvm_host_pmu_init+0x88/0xa4
>   |  armpmu_register+0xf0/0xf4
>   |  arm_pmu_acpi_probe+0x2ec/0x368
>   |  armv8_pmu_driver_init+0x38/0x44
>   |  do_one_initcall+0xcc/0x240
> 
> Fix the race properly by deferring the de-privilege step to
> device_initcall_sync(). This will also be needed in future when probing
> IOMMU devices and allows us to separate the pKVM de-privilege logic from
> the core hypervisor initialisation path.
> 
> Cc: Oliver Upton <oliver.upton@linux.dev>
> Cc: Fuad Tabba <tabba@google.com>
> Cc: Marc Zyngier <maz@kernel.org>
> Fixes: 7755cec63ade ("arm64: perf: Move PMUv3 driver to drivers/perf")
> Signed-off-by: Will Deacon <will@kernel.org>
> ---
> 
> Marc, Oliver -- in practice, this issue only crops with the patches
> moving the CPU PMU driver out into drivers/perf/ and so the arm64
> for-next/core branch is broken. Please can I queue this in the arm64
> tree for 6.4 with your Ack? Thanks.

It doesn't conflict with the current state of kvmarm/next, and I
actually like that this code is moved into pkvm.c, so:

Acked-by: Marc Zyngier <maz@kernel.org>

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.