From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99730EE49AC for ; Tue, 22 Aug 2023 13:49:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Dka69bhCT9nL0FXeO9eBkYb7/Vv69OYbGxEnnHevYeE=; b=L6u219na50nImG ynzxY4a4OglOHqrF1SR0Vn9CeP/xS5tdtF0ZfgF+A1k22K7hWj0oHZye30yxnMR+MtCC0wvVXTua1 YjqKpsVqr2LCqyZIB/cAuSumawocIO8B/qmNVv5FsoMiJkyVAqvy3N2Cf9DfnwCzIcKoOZjw0R6qL DSFsg4vpEQMTCsgjKMK4AC7yvdLDLsJqV5j2M11GKDpFWOrL8hvnsAPAWdY4STzY4QO57RVZsJRR5 sPF9VXZZv7ZoAWeiPS+inRoLB0auxOTih8kbJoMz2Ip8tl4GDof6SNjHBe0AI+RZGoCG/2IK0GBml fBMQmbc7qLA79j9VCjAw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYRl6-00G4im-1R; Tue, 22 Aug 2023 13:49:24 +0000 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qYRl4-00G4gv-03 for linux-arm-kernel@lists.infradead.org; Tue, 22 Aug 2023 13:49:23 +0000 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-565e387000fso2883565a12.2 for ; Tue, 22 Aug 2023 06:49:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692712160; x=1693316960; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=b5h7+qUO0omnn5FGFlX6GxD/r2siYs9YlF+eLfFrmYU=; b=e6GkpzOu/lNcFEoI+f/yDK1Y7h/zRWksZa3DoLx9g9/wbXT/BcLwC6Qb5RBkiKH0j1 X5RRCK5nU8DCd8LXr9lFIFNakvfdwhgBvHaQasKYBMyGLnTmTLM7+qBoeUjP+OgcH8Qe tFTMCmFEeSXnxi/LzI279HMGu7gViPXFMx00DsmMS+zwXihGtbZF5afZMwraesDqVWs0 fPjTxue7MEb/Hkgg87O+gI9Qt7vr6a0zySkyaHDzWY/K6tEJWzvkmNca/Tq0AY0AJzW0 T59nMvtFs8K9cZr2J+lpFPS1aMQ4pZ6/zlhzSASiOTrBgtdp7CWcyowkzyu4eQid6OCz GzMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692712160; x=1693316960; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=b5h7+qUO0omnn5FGFlX6GxD/r2siYs9YlF+eLfFrmYU=; b=bxneYIWCqVl8dWappJMhxMlxHqDBPtIRutNfixSVkNSafsGr9OVF3Sc2ZQgtsloyiJ Z92bkgWC5NNQl9ayz3qDzJ82vubk71OLyUcNh9mwpi+rJUmzVtrmnJjv6N4/SKu0H22V NvRDdFKTD/Y9FqsXZoqDOqQNShZqSw86ImJWKIhbdXsZFmUjsU/Ow+F3YRSwxD0wM16I UIwSD3UOTF5VgyAqLnScY9Y3ccYYRauxOjGUMd8eOs1OeiGHRtPk4ES0dmLAcv/o+Pb0 eKO9gI8zzgEXydNzEvY03wu2JXVEMRHqBUppg/hsBq6U+mvwrzDyXDqLwgbEhLdToTHr kADg== X-Gm-Message-State: AOJu0YzO7c+GDpXlRok/Ip/dksVeydc+ZPYoiIN5PF2yPDlJR+DyHsoF JRBPWJub3N1ogWkG9xhEFI457jXmhIcmRJnmftDAeR+Z X-Google-Smtp-Source: AGHT+IEBMrd+myE8dEIe+4eFc+Nie0XvCrz3oRIZL8U+J8LpndkUKhfUH50RweUtGX6Acl/RLdowVQ== X-Received: by 2002:a05:6a20:429f:b0:140:fd3b:5d4c with SMTP id o31-20020a056a20429f00b00140fd3b5d4cmr12537889pzj.38.1692712159880; Tue, 22 Aug 2023 06:49:19 -0700 (PDT) Received: from leoy-huanghe.lan ([94.177.131.100]) by smtp.gmail.com with ESMTPSA id d190-20020a6336c7000000b00563da87a52dsm2116862pga.40.2023.08.22.06.49.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Aug 2023 06:49:19 -0700 (PDT) Date: Tue, 22 Aug 2023 21:49:12 +0800 From: Leo Yan To: Marc Zyngier Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Huang Shijie , Mark Rutland , Will Deacon Subject: Re: [PATCH v2] KVM: arm64: pmu: Resync EL0 state on counter rotation Message-ID: <20230822134912.GD57731@leoy-huanghe.lan> References: <20230820090108.177817-1-maz@kernel.org> <20230822134516.GC57731@leoy-huanghe.lan> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230822134516.GC57731@leoy-huanghe.lan> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230822_064922_075211_51D1BDD1 X-CRM114-Status: GOOD ( 28.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Aug 22, 2023 at 09:45:16PM +0800, Leo Yan wrote: > Hi, > > On Sun, Aug 20, 2023 at 10:01:08AM +0100, Marc Zyngier wrote: > > Huang Shijie reports that, when profiling a guest from the host > > with a number of events that exceeds the number of available > > counters, the reported counts are wildly inaccurate. Without > > the counter oversubscription, the reported counts are correct. > > > > Their investigation indicates that upon counter rotation (which > > takes place on the back of a timer interrupt), we fail to > > re-apply the guest EL0 enabling, leading to the counting of host > > events instead of guest events. > > > > In order to solve this, add yet another hook between the host PMU > > driver and KVM, re-applying the guest EL0 configuration if the > > right conditions apply (the host is VHE, we are in interrupt > > context, and we interrupted a running vcpu). This triggers a new > > vcpu request which will apply the correct configuration on guest > > reentry. > > > > With this, we have the correct counts, even when the counters are > > oversubscribed. > > I gave a test for this patch, It works well. > > However, I do see this patch can introduce huge amount invoking > kvm_vcpu_pmu_restore_guest() when using 'perf record' command. > > As I mentioned in the patch v2, we can call kvm_vcpu_pmu_resync_el0() > in the function kvm_set_pmu_events() rather than in armv8pmu_start(). > With this change, the kernel only syncs PMU context when the host and > the guest have different traceing for EL0. Just paste the suggested > code for reference: > > @@ -46,6 +48,8 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) > pmu->events_host |= set; > if (!attr->exclude_guest) > pmu->events_guest |= set; > + > + kvm_vcpu_pmu_resync_el0(); > } > > Below is the comparison result for counting resync, the result is for > counting how many times kvm_vcpu_pmu_restore_guest() is called for > 'perf stat' and 'perf record' commands. > > | perf stat(*) | perf record(**) > -----------------+------------------+----------------- > Patch v3: | 2506 | 47325 Here should be "Patch v2", sorry for typo. > Proposed change: | 2514 | 2504 > > (*): sudo ./perf stat -a -e cycles:G,cycles:H -d -d -d sleep 10 > (**): sudo ./perf record -a -e cycles:G,cycles:H -d -d -d sleep 10 > > Thanks, > Leo _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel