From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EE4AC25B4E for ; Fri, 20 Jan 2023 12:12:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229985AbjATMMm (ORCPT ); Fri, 20 Jan 2023 07:12:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229851AbjATMMk (ORCPT ); Fri, 20 Jan 2023 07:12:40 -0500 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0375983E5 for ; Fri, 20 Jan 2023 04:12:38 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 42442CE2804 for ; Fri, 20 Jan 2023 12:12:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7BC5CC433D2; Fri, 20 Jan 2023 12:12:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1674216755; bh=0iRYbwf/9VfNZDvMm3r5D5V9SjGnLfb8zDeKKD726pQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=dYnc6eLdwwBG1cYTK3l0gV197Jwvcv3DfcQMD7C3ZDWPUVN1nEQJ7kdFEdGf7ARlo H9oQJSIJ8mDcFW58izgVRn4Ex5lRA5IHko2t3aOhY3pNhfvvw9DyZIS9aVjEDcLiju rxUqJF4Ts+JJZqaGxNqN/sF5zObR3gUkYMGgLAGlfRFQvlaDLrpehQ0xeyCybAefIJ lzHLWa6rOCSPYU6ACvOfapi5fFmwJspzCfJOc7cUXoIzi7aa69dQCZCXcQxw8bzytO HRQpLRONAz3jYQfLGBIj/qKXAVvKfVtePF/i/jdpEHrq/Pu/AjtvY7gbtdtk2cPCZr 1zok2UNyqvTvw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pIqG1-003P46-99; Fri, 20 Jan 2023 12:12:33 +0000 Date: Fri, 20 Jan 2023 12:12:32 +0000 Message-ID: <86pmb9mmkv.wl-maz@kernel.org> From: Marc Zyngier To: Oliver Upton Cc: Reiji Watanabe , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, James Morse , Alexandru Elisei , Zenghui Yu , Suzuki K Poulose , Paolo Bonzini , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata Subject: Re: [PATCH v2 3/8] KVM: arm64: PMU: Preserve vCPU's PMCR_EL0.N value on vCPU reset In-Reply-To: References: <20230117013542.371944-1-reijiw@google.com> <20230117013542.371944-4-reijiw@google.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: oliver.upton@linux.dev, reijiw@google.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, alexandru.elisei@arm.com, yuzenghui@huawei.com, suzuki.poulose@arm.com, pbonzini@redhat.com, ricarkol@google.com, jingzhangos@google.com, rananta@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Fri, 20 Jan 2023 00:30:33 +0000, Oliver Upton wrote: > > On Mon, Jan 16, 2023 at 05:35:37PM -0800, Reiji Watanabe wrote: > > The number of PMU event counters is indicated in PMCR_EL0.N. > > For a vCPU with PMUv3 configured, its value will be the same as > > the host value by default. Userspace can set PMCR_EL0.N for the > > vCPU to a lower value than the host value using KVM_SET_ONE_REG. > > However, it is practically unsupported, as reset_pmcr() resets > > PMCR_EL0.N to the host value on vCPU reset. > > > > Change reset_pmcr() to preserve the vCPU's PMCR_EL0.N value on > > vCPU reset so that userspace can limit the number of the PMU > > event counter on the vCPU. > > > > Signed-off-by: Reiji Watanabe > > --- > > arch/arm64/kvm/pmu-emul.c | 6 ++++++ > > arch/arm64/kvm/sys_regs.c | 4 +++- > > 2 files changed, 9 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c > > index 24908400e190..937a272b00a5 100644 > > --- a/arch/arm64/kvm/pmu-emul.c > > +++ b/arch/arm64/kvm/pmu-emul.c > > @@ -213,6 +213,12 @@ void kvm_pmu_vcpu_init(struct kvm_vcpu *vcpu) > > > > for (i = 0; i < ARMV8_PMU_MAX_COUNTERS; i++) > > pmu->pmc[i].idx = i; > > + > > + /* > > + * Initialize PMCR_EL0 for the vCPU with the host value so that > > + * the value is available at the very first vCPU reset. > > + */ > > + __vcpu_sys_reg(vcpu, PMCR_EL0) = read_sysreg(pmcr_el0); > > I think we need to derive a sanitised value for PMCR_EL0.N, as I believe > nothing in the architecture prevents implementers from gluing together > cores with varying numbers of PMCs. We probably haven't noticed it yet > since it would appear all Arm designs have had 6 PMCs. This brings back the question of late onlining. How do you cope with with the onlining of such a CPU that has a smaller set of counters than its online counterparts? This is at odds with the way the PMU code works. If you have a different set of counters, you are likely to have a different PMU altogether: [ 1.192606] hw perfevents: enabled with armv8_cortex_a57 PMU driver, 7 counters available [ 1.201254] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available This isn't a broken system, but it has two set of cores which are massively different, and two PMUs. This really should tie back to the PMU type we're counting on, and to the set of CPUs that implements it. We already have some infrastructure to check for the affinity of the PMU vs the CPU we're running on, and this is already visible to userspace. Can't we just leave this responsibility to userspace? Thanks, M. -- Without deviation from the norm, progress is not possible.