From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3F731EE49A0 for ; Wed, 23 Aug 2023 07:16:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tZGk1wdEnblJAsrGzAaaoss6CswVFsnMJylV/8AqsjA=; b=L9ekIoxqefAZrB aYt73KEgr4QLScbG16B9pT9JXdMHbHsryhsfiqhARUyNtq7tveVB4zdTGkWdO26O14/egXRs2/cCt RDCs0HMaI1wxffwzWEoskjqMvO2C9s/nU+Cj6HSdLXhARLk2rSzqKuiVa9YT3Rc88AzrQYjn8jj8j e/kcM0pS6Fin6CSX8W1P4LxufhINnl9JeAoIGSAVe8bJADQCMFygPbnInRIiEiuOdSIrnxR/R6N/R v/9KxbA036MTuWuw+npWPjGVqGEYAOqUqlXGR75AR7p14ccBOk7IBpYKTCxJI6gtBbJ9lxvJI9CdD 8v0LsAJzD9n3DxgMFozw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYi5j-00HRmV-2y; Wed, 23 Aug 2023 07:15:47 +0000 Received: from mx1.tq-group.com ([93.104.207.81]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYi5g-00HRlT-1I for linux-arm-kernel@lists.infradead.org; Wed, 23 Aug 2023 07:15:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tq-group.com; i=@tq-group.com; q=dns/txt; s=key1; t=1692774944; x=1724310944; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DjzRqyjeWdMT4yTOwjQ1OB/ZRsANx6ueGhGD1k+8Wik=; b=SIE7HI701fWU3F3DWcnRwuCin9EXGY0GF+DBnwhwBZ4KzBNFn922RkkU 9jqQq9pyw1xFt4ZKpJ17im40TWeAc/+09pVCO9TmaFVP3yxcjCw/TZOst 5iPOxgUxYCqmAeuz2ZiqU2eOwyQyTgeZFt2WzQf5RbqBudamHGzgsGJCT Mhf1JIsaTW5JYOcl3UR5GUye7fEfnHR78DD5ckSLD5afm/du+8uV0XUmI V+TK5s8jdF/EqzKPqDWTv5GXm6AG7eS4QgM+9XZUDyoxRizQmlbzSi644 YDDqsL5R9+EXX/tT8/9lzMHpIIimIE0akM+7ZhOWozLFksNLRHmU4CeNx w==; X-IronPort-AV: E=Sophos;i="6.01,195,1684792800"; d="scan'208";a="32577938" Received: from vtuxmail01.tq-net.de ([10.115.0.20]) by mx1.tq-group.com with ESMTP; 23 Aug 2023 09:15:38 +0200 Received: from steina-w.localnet (steina-w.tq-net.de [10.123.53.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by vtuxmail01.tq-net.de (Postfix) with ESMTPSA id 708F3280075; Wed, 23 Aug 2023 09:15:38 +0200 (CEST) From: Alexander Stein To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Huang Shijie , Leo Yan , Mark Rutland , Will Deacon , Marc Zyngier Subject: Re: [PATCH v2] KVM: arm64: pmu: Resync EL0 state on counter rotation Date: Wed, 23 Aug 2023 09:15:38 +0200 Message-ID: <14141350.uLZWGnKmhe@steina-w> Organization: TQ-Systems GmbH In-Reply-To: <20230820090108.177817-1-maz@kernel.org> References: <20230820090108.177817-1-maz@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230823_001544_818131_7A86E718 X-CRM114-Status: GOOD ( 27.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, Am Sonntag, 20. August 2023, 11:01:08 CEST schrieb Marc Zyngier: > Huang Shijie reports that, when profiling a guest from the host > with a number of events that exceeds the number of available > counters, the reported counts are wildly inaccurate. Without > the counter oversubscription, the reported counts are correct. > = > Their investigation indicates that upon counter rotation (which > takes place on the back of a timer interrupt), we fail to > re-apply the guest EL0 enabling, leading to the counting of host > events instead of guest events. > = > In order to solve this, add yet another hook between the host PMU > driver and KVM, re-applying the guest EL0 configuration if the > right conditions apply (the host is VHE, we are in interrupt > context, and we interrupted a running vcpu). This triggers a new > vcpu request which will apply the correct configuration on guest > reentry. > = > With this, we have the correct counts, even when the counters are > oversubscribed. > = > Reported-by: Huang Shijie > Suggested-by: Oliver Upton > Tested_by: Huang Shijie > Signed-off-by: Marc Zyngier > Cc: Leo Yan > Cc: Mark Rutland > Cc: Will Deacon > Link: > https://lore.kernel.org/r/20230809013953.7692-1-shijie@os.amperecomputing= .c > om --- > = > Notes: > V2: Fixed 32bit compilation > = > arch/arm/include/asm/arm_pmuv3.h | 2 ++ > arch/arm64/include/asm/kvm_host.h | 1 + > arch/arm64/kvm/arm.c | 3 +++ > arch/arm64/kvm/pmu.c | 18 ++++++++++++++++++ > drivers/perf/arm_pmuv3.c | 2 ++ > include/kvm/arm_pmu.h | 2 ++ > 6 files changed, 28 insertions(+) > = > diff --git a/arch/arm/include/asm/arm_pmuv3.h > b/arch/arm/include/asm/arm_pmuv3.h index f3cd04ff022d..72529f5e2bed 100644 > --- a/arch/arm/include/asm/arm_pmuv3.h > +++ b/arch/arm/include/asm/arm_pmuv3.h > @@ -227,6 +227,8 @@ static inline bool kvm_set_pmuserenr(u64 val) > return false; > } > = > +static inline void kvm_vcpu_pmu_resync_el0(void) {} > + > /* PMU Version in DFR Register */ > #define ARMV8_PMU_DFR_VER_NI 0 > #define ARMV8_PMU_DFR_VER_V3P4 0x5 > diff --git a/arch/arm64/include/asm/kvm_host.h > b/arch/arm64/include/asm/kvm_host.h index d3dd05bbfe23..553040e0e375 1006= 44 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -49,6 +49,7 @@ > #define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4) > #define KVM_REQ_RELOAD_PMU KVM_ARCH_REQ(5) > #define KVM_REQ_SUSPEND KVM_ARCH_REQ(6) > +#define KVM_REQ_RESYNC_PMU_EL0 KVM_ARCH_REQ(7) > = > #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE= | > \ KVM_DIRTY_LOG_INITIALLY_SET) > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index 72dc53a75d1c..978b0411082f 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -803,6 +803,9 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu) > kvm_pmu_handle_pmcr(vcpu, > __vcpu_sys_reg(vcpu, = PMCR_EL0)); > = > + if (kvm_check_request(KVM_REQ_RESYNC_PMU_EL0, vcpu)) > + kvm_vcpu_pmu_restore_guest(vcpu); > + > if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) > return kvm_vcpu_suspend(vcpu); > = > diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c > index 121f1a14c829..0eea225fd09a 100644 > --- a/arch/arm64/kvm/pmu.c > +++ b/arch/arm64/kvm/pmu.c > @@ -236,3 +236,21 @@ bool kvm_set_pmuserenr(u64 val) > ctxt_sys_reg(hctxt, PMUSERENR_EL0) =3D val; > return true; > } > + > +/* > + * If we interrupted the guest to update the host PMU context, make > + * sure we re-apply the guest EL0 state. > + */ > +void kvm_vcpu_pmu_resync_el0(void) > +{ > + struct kvm_vcpu *vcpu; > + > + if (!has_vhe() || !in_interrupt()) > + return; > + > + vcpu =3D kvm_get_running_vcpu(); > + if (!vcpu) > + return; > + > + kvm_make_request(KVM_REQ_RESYNC_PMU_EL0, vcpu); > +} > diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c > index 08b3a1bf0ef6..6a3d8176f54a 100644 > --- a/drivers/perf/arm_pmuv3.c > +++ b/drivers/perf/arm_pmuv3.c > @@ -772,6 +772,8 @@ static void armv8pmu_start(struct arm_pmu *cpu_pmu) > = > /* Enable all counters */ > armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E); > + > + kvm_vcpu_pmu_resync_el0(); This breaks if CONFIG_HW_PERF_EVENTS is enabled but CONFIG_KVM is not. As i= n = this case arch/arm64/kvm/pmu.c is never compiled, but the dummy inline in = include/kvm/arm_pmu.h for kvm_vcpu_pmu_resync_el0() only depends on = CONFIG_HW_PERF_EVENTS. This results in this error: aarch64-v8a-linux-gnu-ld: Unexpected GOT/PLT entries detected! aarch64-v8a-linux-gnu-ld: Unexpected run-time procedure linkages detected! aarch64-v8a-linux-gnu-ld: drivers/perf/arm_pmuv3.o: in function = `armv8pmu_start': drivers/perf/arm_pmuv3.c:753: undefined reference to `kvm_vcpu_pmu_resync_e= l0' Best regards, Alexander > } > = > static void armv8pmu_stop(struct arm_pmu *cpu_pmu) > diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h > index 847da6fc2713..3a8a70a60794 100644 > --- a/include/kvm/arm_pmu.h > +++ b/include/kvm/arm_pmu.h > @@ -74,6 +74,7 @@ int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu); > struct kvm_pmu_events *kvm_get_pmu_events(void); > void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu); > void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu); > +void kvm_vcpu_pmu_resync_el0(void); > = > #define kvm_vcpu_has_pmu(vcpu) = \ > (test_bit(KVM_ARM_VCPU_PMU_V3, (vcpu)->arch.features)) > @@ -171,6 +172,7 @@ static inline u8 kvm_arm_pmu_get_pmuver_limit(void) > { > return 0; > } > +static inline void kvm_vcpu_pmu_resync_el0(void) {} > = > #endif -- = TQ-Systems GmbH | M=FChlstra=DFe 2, Gut Delling | 82229 Seefeld, Germany Amtsgericht M=FCnchen, HRB 105018 Gesch=E4ftsf=FChrer: Detlef Schneider, R=FCdiger Stahl, Stefan Schneider http://www.tq-group.com/ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel