From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80323EB64DD for ; Wed, 9 Aug 2023 09:23:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=64n5hh85sFTdvgwbZ+TF9erjT/XBEV1UWJcIZYEhboQ=; b=iR4VOt0mS3q2A1 N6ljGG8fq0hB8WOYSldampKL0cHTcZYl2QHFDDoMSiQ2D+yJmc5ktmklzsXuM+lBD5FozSmnA0SKM ehyoPo8/qdhgeR3ljD8EE71+YIomYunIn2fQwzEHsqBPSacG3x7BIWSFcuWmdXtTBENAJil04Bgop A5Da99gH4o5XjSmjx8e/oJl3uCpBv+kTufNzSn3s20CsefN/yMk/F4tESbfaXuUBX08LOD1X2D29U mNm5r7mGItvjDJZaA1lp1MqTPd/iiR40QmJzcQKrviKa993BpgnGzEyFV6ERu2RKnGAN0tQgq+sxg XLF/pCPHlda1B4T3PhNg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qTfP0-004RlJ-1a; Wed, 09 Aug 2023 09:22:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qTfOx-004RjF-1a for linux-arm-kernel@lists.infradead.org; Wed, 09 Aug 2023 09:22:49 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 49989D75; Wed, 9 Aug 2023 02:23:22 -0700 (PDT) Received: from FVFF77S0Q05N (unknown [10.57.3.120]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 455C53F59C; Wed, 9 Aug 2023 02:22:36 -0700 (PDT) Date: Wed, 9 Aug 2023 10:22:25 +0100 From: Mark Rutland To: Oliver Upton Cc: Huang Shijie , maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, pbonzini@redhat.com, peterz@infradead.org, ingo@redhat.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org, irogers@google.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-perf-users@vger.kernel.org, patches@amperecomputing.com, zwang@amperecomputing.com Subject: Re: [PATCH] perf/core: fix the bug in the event multiplexing Message-ID: References: <20230809013953.7692-1-shijie@os.amperecomputing.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230809_022247_581675_368EC24B X-CRM114-Status: GOOD ( 27.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Aug 09, 2023 at 08:25:07AM +0000, Oliver Upton wrote: > Hi Huang, > > On Wed, Aug 09, 2023 at 09:39:53AM +0800, Huang Shijie wrote: > > 2.) Root cause. > > There is only 7 counters in my arm64 platform: > > (one cycle counter) + (6 normal counters) > > > > In 1.3 above, we will use 10 event counters. > > Since we only have 7 counters, the perf core will trigger > > event multiplexing in hrtimer: > > merge_sched_in() -->perf_mux_hrtimer_restart() --> > > perf_rotate_context(). > > > > In the perf_rotate_context(), it does not restore some PMU registers > > as context_switch() does. In context_switch(): > > kvm_sched_in() --> kvm_vcpu_pmu_restore_guest() > > kvm_sched_out() --> kvm_vcpu_pmu_restore_host() > > > > So we got wrong result. > > This is a rather vague description of the problem. AFAICT, the > issue here is on VHE systems we wind up getting the EL0 count > enable/disable bits backwards when entering the guest, which is > corroborated by the data you have below. Yep; IIUC the issue here is that when we take an IRQ from a guest and reprogram the PMU in the IRQ handler, the IRQ handler will program the PMU with appropriate host/guest/user/etc filters for a *host* context, and then we'll return back into the guest without reconfigurign the event filtering for a *guest* context. That can happen for perf_rotate_context(), or when we install an event into a running context, as that'll happen via an IPI. > > +void arch_perf_rotate_pmu_set(void) > > +{ > > + if (is_guest()) > > + kvm_vcpu_pmu_restore_guest(NULL); > > + else > > + kvm_vcpu_pmu_restore_host(NULL); > > +} > > + > > This sort of hook is rather nasty, and I'd strongly prefer a solution > that's confined to KVM. I don't think the !is_guest() branch is > necessary at all. Regardless of how the pmu context is changed, we need > to go through vcpu_put() before getting back out to userspace. > > We can check for a running vCPU (ick) from kvm_set_pmu_events() and either > do the EL0 bit flip there or make a request on the vCPU to call > kvm_vcpu_pmu_restore_guest() immediately before reentering the guest. > I'm slightly leaning towards the latter, unless anyone has a better idea > here. The latter sounds reasonable to me. I suspect we need to take special care here to make sure we leave *all* events in a good state when re-entering the guest or if we get to kvm_sched_out() after *removing* an event via an IPI -- it'd be easy to mess either case up and leave some events in a bad state. Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel