From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C82F7C433EF for ; Mon, 22 Nov 2021 14:42:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p/yop8IYZIucNmH4RD8AGgcRcHq36Dvj46hiePXhx60=; b=vHd+/64/YiPlue zhkCUJn0JCrsshlR2NrCJv+p6Ap3rO3jRsoQLVLcmQ+XD7vRZ75Scj4Y8p5kXTmy4GSy0I5oYVH8v ul7yx4ZUw02809Ht/kS8iY5g9VUMyplTYcCMYCW79Yo7UC3FpKx+9J8g4ux9zMd6NyLsjlNgALjsG gpRhQ6DxhWonyJj+u48kX0mwEhQEVZPOauc9SYqDc1wS5ut+YKF221p0b8D9U6RNqRA1t/AhRR2Pr DiNqrAhvhIuW7d52fAjqyDSCXuYLgl6oRTIByTvLHk5W4gLsUMkqqSbYn53faJ6EWGK4C5TY9QQCy 2r3OoZukz4x1ld/ZkSIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpAVm-00Gu53-4u; Mon, 22 Nov 2021 14:41:38 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mpAVc-00Gu2U-Ux for linux-arm-kernel@lists.infradead.org; Mon, 22 Nov 2021 14:41:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 189C81063; Mon, 22 Nov 2021 06:41:27 -0800 (PST) Received: from monolith.localdoman (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A02B23F66F; Mon, 22 Nov 2021 06:41:25 -0800 (PST) Date: Mon, 22 Nov 2021 14:43:17 +0000 From: Alexandru Elisei To: Marc Zyngier Cc: james.morse@arm.com, suzuki.poulose@arm.com, will@kernel.org, mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, peter.maydell@linaro.org Subject: Re: [PATCH 4/4] KVM: arm64: Refuse to run VCPU if the PMU doesn't match the physical CPU Message-ID: References: <20211115165041.194884-1-alexandru.elisei@arm.com> <20211115165041.194884-5-alexandru.elisei@arm.com> <87bl2ds3ny.wl-maz@kernel.org> <871r386zlf.wl-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <871r386zlf.wl-maz@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211122_064129_114135_FF862890 X-CRM114-Status: GOOD ( 37.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Marc, On Mon, Nov 22, 2021 at 02:21:00PM +0000, Marc Zyngier wrote: > On Mon, 22 Nov 2021 12:12:17 +0000, > Alexandru Elisei wrote: > > > > Hi Marc, > > > > On Sun, Nov 21, 2021 at 07:35:13PM +0000, Marc Zyngier wrote: > > > On Mon, 15 Nov 2021 16:50:41 +0000, > > > Alexandru Elisei wrote: > > > > > > > > Userspace can assign a PMU to a VCPU with the KVM_ARM_VCPU_PMU_V3_SET_PMU > > > > device ioctl. If the VCPU is scheduled on a physical CPU which has a > > > > different PMU, the perf events needed to emulate a guest PMU won't be > > > > scheduled in and the guest performance counters will stop counting. Treat > > > > it as an userspace error and refuse to run the VCPU in this situation. > > > > > > > > The VCPU is flagged as being scheduled on the wrong CPU in vcpu_load(), but > > > > the flag is cleared when the KVM_RUN enters the non-preemptible section > > > > instead of in vcpu_put(); this has been done on purpose so the error > > > > condition is communicated as soon as possible to userspace, otherwise > > > > vcpu_load() on the wrong CPU followed by a vcpu_put() could clear the flag. > > > > > > Can we make this something orthogonal to the PMU, and get userspace to > > > pick an affinity mask independently of instantiating a PMU? I can > > > imagine this would also be useful for SPE on asymmetric systems. > > > > I actually went this way for the latest version of the SPE series [1] and > > dropped the explicit userspace ioctl in favor of this mechanism. > > > > The expectation is that userspace already knows which CPUs are associated > > with the chosen PMU (or SPE) when setting the PMU for the VCPU, and having > > userspace set it explicitely via an ioctl looks like an unnecessary step to > > me. I don't see other usecases of an explicit ioctl outside of the above > > two situation (if userspace wants a VCPU to run only on specific CPUs, it > > can use thread affinity for that), so I decided to drop it. > > My problem with that is that if you have (for whatever reason) a set > of affinities that are not strictly identical for both PMU and SPE, > and expose both of these to a guest, what do you choose? > > As long as you have a single affinity set to take care of, you're > good. It is when you have several ones that it becomes ugly (as with > anything involving asymmetric CPUs). I thought about it when I decided to do it this way, my solution was to do a cpumask_and() with the existing VCPU cpumask when setting a VCPU feature that requires it, and returning an error if we get an empty cpumask, because userspace is requesting a combination of VCPU features that is not supported by the hardware. Going with the other solution (user sets the cpumask via an ioctl), KVM would still have to check against certain combinations of VCPU features (for SPE it's mandatory, so KVM doesn't trigger an undefined exception, we could skip the check for PMU, but then what do we gain from the ioctl if KVM doesn't check that it matches the PMU?), so I don't think we loose anything by going with the implicit cpumask. What do you think? Thanks, Alex > > M. > > -- > Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel