From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABFA7C433EF for ; Mon, 20 Sep 2021 10:24:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 920CD60FED for ; Mon, 20 Sep 2021 10:24:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234976AbhITKZ2 (ORCPT ); Mon, 20 Sep 2021 06:25:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:52892 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230495AbhITKZZ (ORCPT ); Mon, 20 Sep 2021 06:25:25 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EBBAC60FED; Mon, 20 Sep 2021 10:23:58 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mSGSq-00Bjnp-Un; Mon, 20 Sep 2021 11:23:57 +0100 Date: Mon, 20 Sep 2021 11:23:56 +0100 Message-ID: <87y27rv937.wl-maz@kernel.org> From: Marc Zyngier To: Will Deacon Cc: "Russell King (Oracle)" , linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, James Morse , Alexandru Elisei , Suzuki K Poulose Subject: Re: REGRESSION: Upgrading host kernel from 5.11 to 5.13 breaks QEMU guests - perf/fw_devlink/kvm In-Reply-To: <20210920095656.GA11961@willie-the-truck> References: <877dfcwutt.wl-maz@kernel.org> <20210920095656.GA11961@willie-the-truck> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: will@kernel.org, linux@armlinux.org.uk, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Mon, 20 Sep 2021 10:56:57 +0100, Will Deacon wrote: > > On Sun, Sep 19, 2021 at 02:36:46PM +0100, Marc Zyngier wrote: > > From 9c26e3e6bbcbc3a583b3974e7a9017029d31fe29 Mon Sep 17 00:00:00 2001 > > From: Marc Zyngier > > Date: Sun, 19 Sep 2021 14:09:49 +0100 > > Subject: [PATCH] KVM: arm64: Fix PMU probe ordering > > > > Russell reported that since 5.13, KVM's probing of the PMU has > > started to fail on his HW. As it turns out, there is an implicit > > ordering dependency between the architectural PMU probing code and > > and KVM's own probing. If, due to probe ordering reasons, KVM probes > > before the PMU driver, it will fail to detect the PMU and prevent it > > from being advertised to guests as well as the VMM. > > > > Obviously, this is one probing too many, and we should be able to > > deal with any ordering. > > > > Add a callback from the PMU code into KVM to advertise the registration > > of a host CPU PMU, allowing for any probing order. > > > > Fixes: 5421db1be3b1 ("KVM: arm64: Divorce the perf code from oprofile helpers") > > Reported-by: "Russell King (Oracle)" > > Signed-off-by: Marc Zyngier > > Link: https://lore.kernel.org/r/YUYRKVflRtUytzy5@shell.armlinux.org.uk > > Cc: stable@vger.kernel.org > > --- > > arch/arm64/kvm/perf.c | 3 --- > > arch/arm64/kvm/pmu-emul.c | 12 +++++++++++- > > drivers/perf/arm_pmu.c | 2 ++ > > include/kvm/arm_pmu.h | 3 --- > > include/linux/perf/arm_pmu.h | 6 ++++++ > > 5 files changed, 19 insertions(+), 7 deletions(-) > > > > diff --git a/arch/arm64/kvm/perf.c b/arch/arm64/kvm/perf.c > > index f9bb3b14130e..c84fe24b2ea1 100644 > > --- a/arch/arm64/kvm/perf.c > > +++ b/arch/arm64/kvm/perf.c > > @@ -50,9 +50,6 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = { > > > > int kvm_perf_init(void) > > { > > - if (kvm_pmu_probe_pmuver() != ID_AA64DFR0_PMUVER_IMP_DEF && !is_protected_kvm_enabled()) > > - static_branch_enable(&kvm_arm_pmu_available); > > - > > return perf_register_guest_info_callbacks(&kvm_guest_cbs); > > } > > > > diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c > > index f5065f23b413..588100c52f34 100644 > > --- a/arch/arm64/kvm/pmu-emul.c > > +++ b/arch/arm64/kvm/pmu-emul.c > > @@ -740,7 +740,17 @@ void kvm_pmu_set_counter_event_type(struct kvm_vcpu *vcpu, u64 data, > > kvm_pmu_create_perf_event(vcpu, select_idx); > > } > > > > -int kvm_pmu_probe_pmuver(void) > > +void kvm_host_pmu_init(struct arm_pmu *pmu) > > +{ > > + if (pmu->pmuver != 0 && > > + pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF && > > + !is_protected_kvm_enabled()) { > > + static_branch_enable(&kvm_arm_pmu_available); > > + kvm_info("PMU detected and enabled\n"); > > + } > > +} > > + > > +static int kvm_pmu_probe_pmuver(void) > > { > > struct perf_event_attr attr = { }; > > struct perf_event *event; > > diff --git a/drivers/perf/arm_pmu.c b/drivers/perf/arm_pmu.c > > index 3cbc3baf087f..295cc7952d0e 100644 > > --- a/drivers/perf/arm_pmu.c > > +++ b/drivers/perf/arm_pmu.c > > @@ -952,6 +952,8 @@ int armpmu_register(struct arm_pmu *pmu) > > pmu->name, pmu->num_events, > > has_nmi ? ", using NMIs" : ""); > > > > + kvm_host_pmu_init(pmu); > > Just a nit, but I think this will get called for each PMU we probe > on a big.LITTLE system which is probably harmless, but possible not > what you want? Yeah, it is a bit ugly, but harmless. In the future, it would be useful to track which PMU is used on which CPUs, and this will give us a decent hook. I'll tone the print down though. Thanks, M. -- Without deviation from the norm, progress is not possible.