From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DFA2CC87FD1 for ; Wed, 6 Aug 2025 09:58:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gGmcOe3qZ6FTWLrDgajIMhIq39XDR1i6v2FE59lQbLc=; b=qM+wL5VdF1Bva1IPTPkhMwO8vj i368jVxWwXLy6S6yKpnwfG1B2GYnBIYcwg6uRjB8oeJSLiXiAwZhNdhyYrbHYKMywKBHNQtEr7KRh aOBz0FbqmGD4IFILCaKrao1ciTpQNf+N36IEe04dI9/YxLLc7zBPIcMqIlmftG4hczoXxIDwcLu08 Hos8x11JJ3EOpqGkOV2OwxL19z5r+esabQ/hGRFSYJt6TX+ovPeGjXqNoWuoSoJajT6GmECod9KwJ Pet8nz/M1f6/UjQZq7Zha0JcR7w/hkp1hm+26P+A4bHbbFJYqYzELjdNFhCbYMwYw0LXCWlUTypiW I8N8VWGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1ujauZ-0000000Eq3P-2HnG; Wed, 06 Aug 2025 09:58:19 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1ujas6-0000000EpkE-0UvC for linux-arm-kernel@lists.infradead.org; Wed, 06 Aug 2025 09:55:47 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 585BD1E8D; Wed, 6 Aug 2025 02:55:37 -0700 (PDT) Received: from arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 75E0B3F738; Wed, 6 Aug 2025 02:55:40 -0700 (PDT) Date: Wed, 6 Aug 2025 11:55:24 +0200 From: Beata Michalska To: Lifeng Zheng Cc: catalin.marinas@arm.com, will@kernel.org, rafael@kernel.org, viresh.kumar@linaro.org, sudeep.holla@arm.com, linux-arm-kernel@lists.infradead.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linuxarm@huawei.com, jonathan.cameron@huawei.com, vincent.guittot@linaro.org, yangyicong@hisilicon.com, zhanjie9@hisilicon.com, lihuisong@huawei.com, yubowen8@huawei.com, linhongye@h-partners.com Subject: Re: [PATCH v3 3/3] arm64: topology: Setup AMU FIE for online CPUs only Message-ID: References: <20250805093330.3715444-1-zhenglifeng1@huawei.com> <20250805093330.3715444-4-zhenglifeng1@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250805093330.3715444-4-zhenglifeng1@huawei.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250806_025546_238868_E222D0BB X-CRM114-Status: GOOD ( 40.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Aug 05, 2025 at 05:33:30PM +0800, Lifeng Zheng wrote: > When boot with maxcpu=1 restrict, and LPI(Low Power Idle States) is on, > only CPU0 will go online. The support AMU flag of CPU0 will be set but the > flags of other CPUs will not. This will cause AMU FIE set up fail for CPU0 > when it shares a cpufreq policy with other CPU(s). After that, when other > CPUs are finally online and the support AMU flags of them are set, they'll > never have a chance to set up AMU FIE, even though they're eligible. > > To solve this problem, the process of setting up AMU FIE needs to be > modified as follows: > > 1. Set up AMU FIE only for the online CPUs. > > 2. Try to set up AMU FIE each time a CPU goes online and do the > freq_counters_valid() check. If this check fails, clear scale freq source > of all the CPUs related to the same policy, in case they use different > source of the freq scale. > > Signed-off-by: Lifeng Zheng > --- > arch/arm64/kernel/topology.c | 54 ++++++++++++++++++++++++++++++++++-- > 1 file changed, 52 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c > index 9317a618bb87..b68621b3c071 100644 > --- a/arch/arm64/kernel/topology.c > +++ b/arch/arm64/kernel/topology.c > @@ -385,7 +385,7 @@ static int init_amu_fie_callback(struct notifier_block *nb, unsigned long val, > struct cpufreq_policy *policy = data; > > if (val == CPUFREQ_CREATE_POLICY) > - amu_fie_setup(policy->related_cpus); > + amu_fie_setup(policy->cpus); I think my previous comment still stands. This will indeed set the AMU FIE support for online cpus. Still, on each frequency change, arch_set_freq_scale will be called with `related_cpus`, and that mask will be used to verify support for AMU counters, and it will report there is none as 'related_cpus' won't be a subset of `scale_freq_counters_mask`. As a consequence, new scale will be set, as seen by the cpufreq. Now this will be corrected on next tick but it might cause disruptions. So this change should also be applied to cpufreq - if feasible, or at least be proven not to be an issue. Unless I am missing smth. > > /* > * We don't need to handle CPUFREQ_REMOVE_POLICY event as the AMU > @@ -404,10 +404,60 @@ static struct notifier_block init_amu_fie_notifier = { > .notifier_call = init_amu_fie_callback, > }; > > +static int cpuhp_topology_online(unsigned int cpu) > +{ > + struct cpufreq_policy *policy = cpufreq_cpu_get_raw_no_check(cpu); > + > + /* > + * If the online CPUs are not all AMU FIE CPUs or the new one is already > + * an AMU FIE one, no need to set it. > + */ > + if (!policy || !cpumask_available(amu_fie_cpus) || > + !cpumask_subset(policy->cpus, amu_fie_cpus) || > + cpumask_test_cpu(cpu, amu_fie_cpus)) > + return 0; This is getting rather cumbersome as the CPU that is coming online might belong to a policy that is yet to be created. Both AMU FIE support, as well as cpufreq, rely on dynamic hp state so, in theory, we cannot be certain that the cpufreq callback will be fired first (although that seems to be the case). If that does not happen, the policy will not exist, and as such given CPU will not use AMUs for FIE. The problem might be hypothetical but it at least deservers a comment I think. Second problem is cpumask_available use: this might be the very fist CPU that might potentially rely on AMUs for frequency invariance so that mask may not be available yet. That does not mean AMUs setup should be skipped. Not just yet, at least. Again more hypothetical. BTW, there should be `amu_fie_cpu_supported'. > + > + /* > + * If the new online CPU cannot pass this check, all the CPUs related to > + * the same policy should be clear from amu_fie_cpus mask, otherwise they > + * may use different source of the freq scale. > + */ > + if (!freq_counters_valid(cpu)) { > + topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_ARCH, > + policy->related_cpus); > + cpumask_andnot(amu_fie_cpus, amu_fie_cpus, policy->related_cpus); I think it might deserve a warning as this probably should not happen. --- BR Beata > + return 0; > + } > + > + cpumask_set_cpu(cpu, amu_fie_cpus); > + > + topology_set_scale_freq_source(&amu_sfd, cpumask_of(cpu)); > + > + pr_debug("CPU[%u]: counter will be used for FIE.", cpu); > + > + return 0; > +} > + > static int __init init_amu_fie(void) > { > - return cpufreq_register_notifier(&init_amu_fie_notifier, > + int ret; > + > + ret = cpufreq_register_notifier(&init_amu_fie_notifier, > CPUFREQ_POLICY_NOTIFIER); > + if (ret) > + return ret; > + > + ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, > + "arm64/topology:online", > + cpuhp_topology_online, > + NULL); > + if (ret < 0) { > + cpufreq_unregister_notifier(&init_amu_fie_notifier, > + CPUFREQ_POLICY_NOTIFIER); > + return ret; > + } > + > + return 0; > } > core_initcall(init_amu_fie); > > -- > 2.33.0 >