From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59972C54E58 for ; Wed, 13 Mar 2024 23:47:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4jgJnwEujZ0i0mb/PdB15SL1flODw/bGMA5fxJt0JR4=; b=TLD3wCfx/YdsSE U9nd87MrXJwHu8VKa+oRpPhJwDxCHJGXO6DnyU6MljIhjgOvyflAou+ESXS9stbuLGPdM2uNuhoPP wUI+G52zd98GbfDGzjDpCgwsZWxLI8fSKd9k34IAAAaAQoGt/q6m1Y1U+6zp+YKBm0fVubxCmxGuy sqQIiXFBPqnW7wHz+AYupO9TD+UUmzVLP4rkkG0IiymWwwVyCXwsxVkcWHNxiIosEyxaknJ3Mskpg te8Pd853a+xZnPUYm1RP9B8Ls9WhfuGWOFLS2PmclEeyMNM6s5lfJ/nbkJVmb94NsGHvjSejA7Hqb vhMA5NW3cdcAulDJ1p/w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rkYJQ-0000000CJ42-27GF; Wed, 13 Mar 2024 23:47:08 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rkYJL-0000000CJ39-0xzM for linux-arm-kernel@lists.infradead.org; Wed, 13 Mar 2024 23:47:06 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 392361007; Wed, 13 Mar 2024 16:47:36 -0700 (PDT) Received: from arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2D9B73F73F; Wed, 13 Mar 2024 16:46:55 -0700 (PDT) Date: Thu, 14 Mar 2024 00:46:19 +0100 From: Beata Michalska To: Ionela Voinescu Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, vanshikonda@os.amperecomputing.com, sudeep.holla@arm.com, will@kernel.org, catalin.marinas@arm.com, vincent.guittot@linaro.org, sumitg@nvidia.com, yang@os.amperecomputing.com, lihuisong@huawei.com Subject: Re: [PATCH v3 2/3] arm64: Provide an AMU-based version of arch_freq_get_on_cpu Message-ID: References: <20240312083431.3239989-1-beata.michalska@arm.com> <20240312083431.3239989-3-beata.michalska@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240313_164704_785962_B4021560 X-CRM114-Status: GOOD ( 45.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Mar 13, 2024 at 12:20:16PM +0000, Ionela Voinescu wrote: > Hi Beata, > > Thank you for the patches! > High time for those! > On Tuesday 12 Mar 2024 at 08:34:30 (+0000), Beata Michalska wrote: > > With the Frequency Invariance Engine (FIE) being already wired up with > > sched tick and making use of relevant (core counter and constant > > counter) AMU counters, getting the current frequency for a given CPU > > on supported platforms can be achieved by utilizing the frequency scale > > factor which reflects an average CPU frequency for the last tick period > > length. > > > > The solution is partially based on APERF/MPERF implementation of > > arch_freq_get_on_cpu. > > > > Suggested-by: Ionela Voinescu > > Signed-off-by: Beata Michalska > > --- > > arch/arm64/kernel/topology.c | 103 +++++++++++++++++++++++++++++++---- > > 1 file changed, 92 insertions(+), 11 deletions(-) > > > > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c > > index 1a2c72f3e7f8..42cb19c31719 100644 > > --- a/arch/arm64/kernel/topology.c > > +++ b/arch/arm64/kernel/topology.c > > @@ -17,6 +17,8 @@ > > #include > > #include > > #include > > +#include > > +#include > > > > #include > > #include > > @@ -88,18 +90,31 @@ int __init parse_acpi_topology(void) > > * initialized. > > */ > > static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) = 1UL << (2 * SCHED_CAPACITY_SHIFT); > > -static DEFINE_PER_CPU(u64, arch_const_cycles_prev); > > -static DEFINE_PER_CPU(u64, arch_core_cycles_prev); > > static cpumask_var_t amu_fie_cpus; > > > > +struct amu_cntr_sample { > > + u64 arch_const_cycles_prev; > > + u64 arch_core_cycles_prev; > > + unsigned long last_update; > > + seqcount_t seq; > > +}; > > + > > +static DEFINE_PER_CPU_SHARED_ALIGNED(struct amu_cntr_sample, cpu_amu_samples) = { > > + .seq = SEQCNT_ZERO(cpu_amu_samples.seq) > > +}; > > + > > void update_freq_counters_refs(void) > > { > > - this_cpu_write(arch_core_cycles_prev, read_corecnt()); > > - this_cpu_write(arch_const_cycles_prev, read_constcnt()); > > + struct amu_cntr_sample *amu_sample = this_cpu_ptr(&cpu_amu_samples); > > + > > + amu_sample->arch_core_cycles_prev = read_corecnt(); > > + amu_sample->arch_const_cycles_prev = read_constcnt(); > > } > > > > static inline bool freq_counters_valid(int cpu) > > { > > + struct amu_cntr_sample *amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu); > > + > > if ((cpu >= nr_cpu_ids) || !cpumask_test_cpu(cpu, cpu_present_mask)) > > return false; > > > > @@ -108,8 +123,8 @@ static inline bool freq_counters_valid(int cpu) > > return false; > > } > > > > - if (unlikely(!per_cpu(arch_const_cycles_prev, cpu) || > > - !per_cpu(arch_core_cycles_prev, cpu))) { > > + if (unlikely(!amu_sample->arch_const_cycles_prev || > > + !amu_sample->arch_core_cycles_prev)) { > > pr_debug("CPU%d: cycle counters are not enabled.\n", cpu); > > return false; > > } > > @@ -152,20 +167,27 @@ void freq_inv_set_max_ratio(int cpu, u64 max_rate) > > > > static void amu_scale_freq_tick(void) > > { > > + struct amu_cntr_sample *amu_sample = this_cpu_ptr(&cpu_amu_samples); > > u64 prev_core_cnt, prev_const_cnt; > > u64 core_cnt, const_cnt, scale; > > > > - prev_const_cnt = this_cpu_read(arch_const_cycles_prev); > > - prev_core_cnt = this_cpu_read(arch_core_cycles_prev); > > + prev_const_cnt = amu_sample->arch_const_cycles_prev; > > + prev_core_cnt = amu_sample->arch_core_cycles_prev; > > + > > + write_seqcount_begin(&amu_sample->seq); > > The critical section here does not need to be this extensive, right? > > The arch_freq_get_on_cpu() function only uses the frequency scale factor > and the last_update value, so this need only be placed above > "this_cpu_write(arch_freq_scale,..", if I'm not missing anything. You're not missing anything. The write side critical section could span only those two, but having it extended gives a chance for the readers to get in on the update and as those are not really performance sensitive I though it might be a good option, especially if we can save the cycles on not needing to poke the cpufeq driver. Furthermore, if the critical section is to span only the two, then it does not really change much and can be dropped. > > > > > update_freq_counters_refs(); > > > > - const_cnt = this_cpu_read(arch_const_cycles_prev); > > - core_cnt = this_cpu_read(arch_core_cycles_prev); > > + const_cnt = amu_sample->arch_const_cycles_prev; > > + core_cnt = amu_sample->arch_core_cycles_prev; > > > > + /* > > + * This should not happen unless the AMUs have been reset and the > > + * counter values have not been resroted - unlikely > > + */ > > if (unlikely(core_cnt <= prev_core_cnt || > > const_cnt <= prev_const_cnt)) > > - return; > > + goto leave; > > > > /* > > * /\core arch_max_freq_scale > > @@ -182,6 +204,10 @@ static void amu_scale_freq_tick(void) > > > > scale = min_t(unsigned long, scale, SCHED_CAPACITY_SCALE); > > this_cpu_write(arch_freq_scale, (unsigned long)scale); > > + > > + amu_sample->last_update = jiffies; > > +leave: > > + write_seqcount_end(&amu_sample->seq); > > } > > > > static struct scale_freq_data amu_sfd = { > > @@ -189,6 +215,61 @@ static struct scale_freq_data amu_sfd = { > > .set_freq_scale = amu_scale_freq_tick, > > }; > > > > +#define AMU_SAMPLE_EXP_MS 20 > > + > > +unsigned int arch_freq_get_on_cpu(int cpu) > > +{ > > + struct amu_cntr_sample *amu_sample; > > + unsigned long last_update; > > + unsigned int seq; > > + unsigned int freq; > > + u64 scale; > > + > > + if (!cpumask_test_cpu(cpu, amu_fie_cpus) || !arch_scale_freq_ref(cpu)) > > + return 0; > > + > > +retry: > > + amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu); > > + > > + do { > > + seq = raw_read_seqcount_begin(&amu_sample->seq); > > + last_update = amu_sample->last_update; > > + } while (read_seqcount_retry(&amu_sample->seq, seq)); > > Related to the point above, this retry loop should also contain > "scale = arch_scale_freq_capacity(cpu)", otherwise there's no much point > for synchronisation, as far as I can tell. I'm not entirely sure why we would need to include the scale factor within the read critical section. The aim here is to make sure we see the update if one is ongoing and that the update to the timestamp is observed along with one to the scale factor, which is what the write_seqcount_end will guarantee (although the latter is not a hard sell as the update happens under interrupts being disabled). If later on we fetch newer scale factor that's perfectly fine, we do not want to see the stale one. Again, I can drop the seqcount (which is slightly abused in this case I must admit) at a cost of potentially missing some updates. > > For x86, arch_freq_get_on_cpu() uses the counter deltas and it would be > bad if values from different ticks would be used. But here the only > benefit of synchronisation is to make sure that we're using the scale > factor computed at the last update time. For us, even skipping on the > synchronisation logic would still be acceptable, as we'd be ensuring that > there was a tick in the past 20ms and we'd always use the most recent > value of the frequency scale factor. How would we ensure there was a tick in last 20ms ? > > Hope it helps, It does, thank you. -- BR Beata > Ionela. > > > + > > + /* > > + * For those CPUs that are in full dynticks mode, > > + * and those that have not seen tick for a while > > + * try an alternative source for the counters (and thus freq scale), > > + * if available for given policy > > + */ > > + if (time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) { > > + struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); > > + int ref_cpu = nr_cpu_ids; > > + > > + if (cpumask_intersects(housekeeping_cpumask(HK_TYPE_TICK), > > + policy->cpus)) > > + ref_cpu = cpumask_nth_and(cpu, policy->cpus, > > + housekeeping_cpumask(HK_TYPE_TICK)); > > + > > + cpufreq_cpu_put(policy); > > + if (ref_cpu >= nr_cpu_ids || ref_cpu == cpu) > > + /* No alternative to pull info from */ > > + return 0; > > + cpu = ref_cpu; > > + goto retry; > > + } > > + /* > > + * Reversed computation to the one used to determine > > + * the arch_freq_scale value > > + * (see amu_scale_freq_tick for details) > > + */ > > + scale = arch_scale_freq_capacity(cpu); > > + freq = scale * arch_scale_freq_ref(cpu); > > + freq >>= SCHED_CAPACITY_SHIFT; > > + > > + return freq; > > +} > > + > > static void amu_fie_setup(const struct cpumask *cpus) > > { > > int cpu; > > -- > > 2.25.1 > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel