From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0680EE4996 for ; Tue, 22 Aug 2023 13:46:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xXmd4l6ah+V67xxB09Dc75uTUTN2inV5V7A+TbwIVDM=; b=dpZpDUszIH4VMc 4OJn7LNgjvmJCgOV5OBG6cCz4B4dnnoY5UAG6LvblKLi2qmVIPrc4PvYPcvzg40v2SLsuTyS/fo/p FLup3JpX2oAyUMmVavRWQ3k/KYQp3wOO1tHGxJ9wdD41J677Chh07FjWsy93pl1PtzWmXEdhQ6QV3 /1v7zhID1BFtKs9xaRXhYCcrvIh9AbroIzC2gdOmkJcbeF50LpisJvaM4GUSFO3iU4Q1PGOjLwG6X 8G106XM7ptxLuU0EkWjgkzwiT1uVPvFFFnMc2VAdHboBTpWn1P2TvMetQfgpAPxh7ebZ9VYts6F/C nriCy9hHlngW1TqyrKkA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYRhM-00G44j-0o; Tue, 22 Aug 2023 13:45:32 +0000 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYRhJ-00G43m-0F for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 13:45:30 +0000 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1bd9b4f8e0eso28681555ad.1 for ; Tue, 22 Aug 2023 06:45:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692711926; x=1693316726; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=2pdIJkwJb26+2W2iCXEOozbm0Klva0FzdGK7hVu8EGo=; b=QC+dl4RBMWFG+Vyg2wPTTMfqUXTo6EkdhfppCFSDR68s/OvCmsi5JCs8khGim0sZf2 4dZCiZEtW8FMDzuiRSXtrOL72pqZZxRYVFqeuYP1FX1Ed/J6M1/3+nKlSk949JRkna9F QXn/EuuQh5WtCkLuMFejtNJ7MRBFGg5i4tlHWBBtyuXhH8UefNbP//gx+H18OAMZRu+8 iINFffcfMOnh52lbWYeDN28QJIBzUX6uwJ6MIRCLVOAw87F8veXL6Try5aXKJPUg/eg/ DF4kAWjRFmJxoxS3tGPK1omR+96pR4yUKCgm1QBRb/QI5GwFMr9MCtfki1253diiSKea DInQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692711926; x=1693316726; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=2pdIJkwJb26+2W2iCXEOozbm0Klva0FzdGK7hVu8EGo=; b=QctfE2cu7XahIoh7MjaHzMhEgM4swwFpnHg9vFMVDP6AMVjZtingwwxCfZaKzenyAw i3xp+Qqv9hS2sMA9rva5Z596MaVrbSSWNVmCi8fxsIVzLlAOhmIfX7pSn5xsY3HCKtbU ayWCe3+lNED3Argor6GaIyhbUpJ0mndI44TWOXO8xRQZ9Cdz2po+j4HdQuKDSzVXrfiX UJpxQLWnK3335kGcTvIXvqH09f37JGZ/48aJNIPtsHkFzzb8VukfjcrQyELQEPZLUiMo 4OdvNmIqmdoCLidvGyTqNOnEZt6j1qLCGM42HRcozVxF01fDNzGD0+eusN/kKmVunIxV JX/A== X-Gm-Message-State: AOJu0Ywkj/Iy+BtAKHCVS4r3pp3RH13OCa//w51xk+Z7BTZNOLcUZk8S q29++UvL8990ceyvBqRN00K2Tw== X-Google-Smtp-Source: AGHT+IFNBAV4UqQj2/1i97hcCATnDPDrNLOLjdzcBiWaKzJmpiND5FXxF0VqsUjRbZsA2vCThmXkgA== X-Received: by 2002:a17:902:820d:b0:1bb:ee1b:3cdf with SMTP id x13-20020a170902820d00b001bbee1b3cdfmr6327318pln.67.1692711926394; Tue, 22 Aug 2023 06:45:26 -0700 (PDT) Received: from leoy-huanghe.lan ([94.177.131.100]) by smtp.gmail.com with ESMTPSA id v7-20020a170902b7c700b001c0a3df39a9sm19154plz.205.2023.08.22.06.45.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 06:45:25 -0700 (PDT) Date: Tue, 22 Aug 2023 21:45:16 +0800 From: Leo Yan To: Marc Zyngier Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Huang Shijie , Mark Rutland , Will Deacon Subject: Re: [PATCH v2] KVM: arm64: pmu: Resync EL0 state on counter rotation Message-ID: <20230822134516.GC57731@leoy-huanghe.lan> References: <20230820090108.177817-1-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230820090108.177817-1-maz@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_064529_131489_6C53C2EC X-CRM114-Status: GOOD ( 21.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi, On Sun, Aug 20, 2023 at 10:01:08AM +0100, Marc Zyngier wrote: > Huang Shijie reports that, when profiling a guest from the host > with a number of events that exceeds the number of available > counters, the reported counts are wildly inaccurate. Without > the counter oversubscription, the reported counts are correct. > > Their investigation indicates that upon counter rotation (which > takes place on the back of a timer interrupt), we fail to > re-apply the guest EL0 enabling, leading to the counting of host > events instead of guest events. > > In order to solve this, add yet another hook between the host PMU > driver and KVM, re-applying the guest EL0 configuration if the > right conditions apply (the host is VHE, we are in interrupt > context, and we interrupted a running vcpu). This triggers a new > vcpu request which will apply the correct configuration on guest > reentry. > > With this, we have the correct counts, even when the counters are > oversubscribed. I gave a test for this patch, It works well. However, I do see this patch can introduce huge amount invoking kvm_vcpu_pmu_restore_guest() when using 'perf record' command. As I mentioned in the patch v2, we can call kvm_vcpu_pmu_resync_el0() in the function kvm_set_pmu_events() rather than in armv8pmu_start(). With this change, the kernel only syncs PMU context when the host and the guest have different traceing for EL0. Just paste the suggested code for reference: @@ -46,6 +48,8 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) pmu->events_host |= set; if (!attr->exclude_guest) pmu->events_guest |= set; + + kvm_vcpu_pmu_resync_el0(); } Below is the comparison result for counting resync, the result is for counting how many times kvm_vcpu_pmu_restore_guest() is called for 'perf stat' and 'perf record' commands. | perf stat(*) | perf record(**) -----------------+------------------+----------------- Patch v3: | 2506 | 47325 Proposed change: | 2514 | 2504 (*): sudo ./perf stat -a -e cycles:G,cycles:H -d -d -d sleep 10 (**): sudo ./perf record -a -e cycles:G,cycles:H -d -d -d sleep 10 Thanks, Leo _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel