From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF5B61A00CE for ; Thu, 27 Jun 2024 21:22:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719523338; cv=none; b=PEMf24YTHnfDUu5P5s89zcFKLaKVNAy66zpw17+kTnXbxsBSKR33ktiZMV61nN3C2+uRv068mu5YFVG2N6bOCvO7gOGddPqeFMBlBSfgy9rkv0MTisxpAVDK6aqMNJQICqiF5KTlZ9Jz1SMhiq32WLP2taDMvAK+Pai9i1tUqCo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719523338; c=relaxed/simple; bh=qhtJNUhzDoON9I6jaUH0r0PsFEv2E3VTs8TwiviBK5M=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=idHewGEZWaluaiB/4WM6Vj7dB03DncFRvV/azL1zNJq1rRkW739qnQvrj5l4h9NhQ6QrzmM6yqMHNHEacnbemYawZmnympbYrKLR9KjUa/K7LyIPY2Ok1Rp3Nnp26zGNsA/NVQzV3PHm4IYa/8/iGutxMLm03XJjseyu/u0EtfY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UcY94Syc; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UcY94Syc" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-365663f51adso6083414f8f.1 for ; Thu, 27 Jun 2024 14:22:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1719523335; x=1720128135; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=TRQZdxqpT6x5YpRkgnwCvTH3y2IWn5FDxDsgzvSaC18=; b=UcY94Syc9HNKfoV1xlXEPyWVHThrm6m5dWgZMjsrf/bFQFf8+GVnNG+o1fhRgU1xDz 6DmhPJOBv5S89jmzAmlVFmUvkJg1YGDsMe4JqhwR9PrX44rGW2cbEeCUaTDg4JjOnN8s UEYEqinecyO75dr4TsLdtQQZCMxkobgs9CKVIP7Ydf0x6jbS1XA7ieu/59Gk/vs+ywGM 4yu/1LbDAjA9BXMQZbg1Z5EHqfoTpvrZ/P5wGx6sc1H2SaNArKIdWkGo9njtGH4AI4Gt k0SHpbMFJI9DAfUTB4RJPXMThCNAANvUkzrIYrl/XYe51oGRgGQ9mQA7n84VAfBXtwus 4EHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719523335; x=1720128135; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TRQZdxqpT6x5YpRkgnwCvTH3y2IWn5FDxDsgzvSaC18=; b=jF+T3SPKXL5WDQ8iVH6I0aqGfTh45cCXoyteyYFZx9MoOUapPw3GWyFFP5GIaiJq9R pANx6EOmtRETliwrs+9H/Xiz4Q6Ij4gyyS4aOWV/K0S2FLAADZEOh5jHNhTdYcvly43b Go4r7iuK0y55KyF7MHZ/2y+ZIUAfYM+10JAEz0kEz6hZctUsdbWjnv7TdP6GUFCxJPj2 +38YjuLYlyX7Om5dufL5bXM8OmKwtyNfj2CjinUaGs6KP3cccogFrz4KVzqMUetRGix8 fbfXdp92Hu/xEBxV4fU8h6YPw3cRx/82jYyxemm1YJhja7qLVk4E61iem7LYXrP30uQQ imCQ== X-Forwarded-Encrypted: i=1; AJvYcCX0bmjwBBF48bqHYAWKXsASktE+WwU35oWnWsMT3TtOQ6hCGKMYXuTZrMHyiBSgQBmbecJGn99TwsHwErZXkurZFD4iXqmB29HCoA== X-Gm-Message-State: AOJu0YxNqQyjm5gv7DqbdriMwhwHzv+faMUeiYmc6+x5NtGWb6Mqrw7C 9Ib7nzEY4NdQXGoMUZBU+08P1ErmDy15x2oR9f7p6NfyCm8difYWUxc4IfPVmMt4TAiJQE0aUk2 kl7cpzJpvOh8w3UsGyAj91rR0JdAQ5UcJ3HCt X-Google-Smtp-Source: AGHT+IFtToqmHA26slD6w71qcbxkHPK1BnScTMkqy7f6xNaC3JmrYokkOj7/EE2iPCrpyxax3GOxj2+k26WbktdKjSI= X-Received: by 2002:a5d:59a5:0:b0:366:e685:d0cb with SMTP id ffacd0b85a97d-366e685d27dmr12970392f8f.6.1719523334823; Thu, 27 Jun 2024 14:22:14 -0700 (PDT) Precedence: bulk X-Mailing-List: devicetree@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20240521043102.2786284-1-davidai@google.com> <20240521043102.2786284-3-davidai@google.com> In-Reply-To: <20240521043102.2786284-3-davidai@google.com> From: David Dai Date: Thu, 27 Jun 2024 15:22:02 -0600 Message-ID: Subject: Re: [PATCH v6 2/2] cpufreq: add virtual-cpufreq driver To: "Rafael J. Wysocki" , Viresh Kumar , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Sudeep Holla , David Dai , Saravana Kannan Cc: Quentin Perret , Masami Hiramatsu , Will Deacon , Peter Zijlstra , Vincent Guittot , Marc Zyngier , Oliver Upton , Dietmar Eggemann , Pavan Kondeti , Gupta Pankaj , Mel Gorman , kernel-team@android.com, linux-pm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi folks, Gentle nudge on this patch to see if there's any remaining concerns? Thanks, David On Mon, May 20, 2024 at 10:32=E2=80=AFPM David Dai wro= te: > > Introduce a virtualized cpufreq driver for guest kernels to improve > performance and power of workloads within VMs. > > This driver does two main things: > > 1. Sends the frequency of vCPUs as a hint to the host. The host uses the > hint to schedule the vCPU threads and decide physical CPU frequency. > > 2. If a VM does not support a virtualized FIE(like AMUs), it queries the > host CPU frequency by reading a MMIO region of a virtual cpufreq device > to update the guest's frequency scaling factor periodically. This enables > accurate Per-Entity Load Tracking for tasks running in the guest. > > Co-developed-by: Saravana Kannan > Signed-off-by: Saravana Kannan > Signed-off-by: David Dai > --- > drivers/cpufreq/Kconfig | 14 ++ > drivers/cpufreq/Makefile | 1 + > drivers/cpufreq/virtual-cpufreq.c | 335 ++++++++++++++++++++++++++++++ > include/linux/arch_topology.h | 1 + > 4 files changed, 351 insertions(+) > create mode 100644 drivers/cpufreq/virtual-cpufreq.c > > diff --git a/drivers/cpufreq/Kconfig b/drivers/cpufreq/Kconfig > index 94e55c40970a..9aa86bec5793 100644 > --- a/drivers/cpufreq/Kconfig > +++ b/drivers/cpufreq/Kconfig > @@ -217,6 +217,20 @@ config CPUFREQ_DT > > If in doubt, say N. > > +config CPUFREQ_VIRT > + tristate "Virtual cpufreq driver" > + depends on OF && GENERIC_ARCH_TOPOLOGY > + help > + This adds a virtualized cpufreq driver for guest kernels that > + read/writes to a MMIO region for a virtualized cpufreq device t= o > + communicate with the host. It sends performance requests to the= host > + which gets used as a hint to schedule vCPU threads and select C= PU > + frequency. If a VM does not support a virtualized FIE such as A= MUs, > + it updates the frequency scaling factor by polling host CPU fre= quency > + to enable accurate Per-Entity Load Tracking for tasks running i= n the guest. > + > + If in doubt, say N. > + > config CPUFREQ_DT_PLATDEV > tristate "Generic DT based cpufreq platdev driver" > depends on OF > diff --git a/drivers/cpufreq/Makefile b/drivers/cpufreq/Makefile > index 8d141c71b016..eb72ecdc24db 100644 > --- a/drivers/cpufreq/Makefile > +++ b/drivers/cpufreq/Makefile > @@ -16,6 +16,7 @@ obj-$(CONFIG_CPU_FREQ_GOV_ATTR_SET) +=3D cpufreq_gove= rnor_attr_set.o > > obj-$(CONFIG_CPUFREQ_DT) +=3D cpufreq-dt.o > obj-$(CONFIG_CPUFREQ_DT_PLATDEV) +=3D cpufreq-dt-platdev.o > +obj-$(CONFIG_CPUFREQ_VIRT) +=3D virtual-cpufreq.o > > # Traces > CFLAGS_amd-pstate-trace.o :=3D -I$(src) > diff --git a/drivers/cpufreq/virtual-cpufreq.c b/drivers/cpufreq/virtual-= cpufreq.c > new file mode 100644 > index 000000000000..59ce2bda3913 > --- /dev/null > +++ b/drivers/cpufreq/virtual-cpufreq.c > @@ -0,0 +1,335 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (C) 2024 Google LLC > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +/* > + * CPU0..CPUn > + * +-------------+-------------------------------+--------+-------+ > + * | Register | Description | Offset | Len | > + * +-------------+-------------------------------+--------+-------+ > + * | cur_perf | read this register to get | 0x0 | 0x4 | > + * | | the current perf (integer val | | | > + * | | representing perf relative to | | | > + * | | max performance) | | | > + * | | that vCPU is running at | | | > + * +-------------+-------------------------------+--------+-------+ > + * | set_perf | write to this register to set | 0x4 | 0x4 | > + * | | perf value of the vCPU | | | > + * +-------------+-------------------------------+--------+-------+ > + * | perftbl_len | number of entries in perf | 0x8 | 0x4 | > + * | | table. A single entry in the | | | > + * | | perf table denotes no table | | | > + * | | and the entry contains | | | > + * | | the maximum perf value | | | > + * | | that this vCPU supports. | | | > + * | | The guest can request any | | | > + * | | value between 1 and max perf | | | > + * | | when perftbls are not used. | | | > + * +---------------------------------------------+--------+-------+ > + * | perftbl_sel | write to this register to | 0xc | 0x4 | > + * | | select perf table entry to | | | > + * | | read from | | | > + * +---------------------------------------------+--------+-------+ > + * | perftbl_rd | read this register to get | 0x10 | 0x4 | > + * | | perf value of the selected | | | > + * | | entry based on perftbl_sel | | | > + * +---------------------------------------------+--------+-------+ > + * | perf_domain | performance domain number | 0x14 | 0x4 | > + * | | that this vCPU belongs to. | | | > + * | | vCPUs sharing the same perf | | | > + * | | domain number are part of the | | | > + * | | same performance domain. | | | > + * +-------------+-------------------------------+--------+-------+ > + */ > + > +#define REG_CUR_PERF_STATE_OFFSET 0x0 > +#define REG_SET_PERF_STATE_OFFSET 0x4 > +#define REG_PERFTBL_LEN_OFFSET 0x8 > +#define REG_PERFTBL_SEL_OFFSET 0xc > +#define REG_PERFTBL_RD_OFFSET 0x10 > +#define REG_PERF_DOMAIN_OFFSET 0x14 > +#define PER_CPU_OFFSET 0x1000 > + > +#define PERFTBL_MAX_ENTRIES 64U > + > +static void __iomem *base; > +static DEFINE_PER_CPU(u32, perftbl_num_entries); > + > +static void virt_scale_freq_tick(void) > +{ > + int cpu =3D smp_processor_id(); > + u32 max_freq =3D (u32)cpufreq_get_hw_max_freq(cpu); > + u64 cur_freq; > + unsigned long scale; > + > + cur_freq =3D (u64)readl_relaxed(base + cpu * PER_CPU_OFFSET > + + REG_CUR_PERF_STATE_OFFSET); > + > + cur_freq <<=3D SCHED_CAPACITY_SHIFT; > + scale =3D (unsigned long)div_u64(cur_freq, max_freq); > + scale =3D min(scale, SCHED_CAPACITY_SCALE); > + > + this_cpu_write(arch_freq_scale, scale); > +} > + > +static struct scale_freq_data virt_sfd =3D { > + .source =3D SCALE_FREQ_SOURCE_VIRT, > + .set_freq_scale =3D virt_scale_freq_tick, > +}; > + > +static unsigned int virt_cpufreq_set_perf(struct cpufreq_policy *policy, > + unsigned int target_freq) > +{ > + writel_relaxed(target_freq, > + base + policy->cpu * PER_CPU_OFFSET + REG_SET_PERF= _STATE_OFFSET); > + return 0; > +} > + > +static unsigned int virt_cpufreq_fast_switch(struct cpufreq_policy *poli= cy, > + unsigned int target_freq) > +{ > + virt_cpufreq_set_perf(policy, target_freq); > + return target_freq; > +} > + > +static u32 virt_cpufreq_get_perftbl_entry(int cpu, u32 idx) > +{ > + writel_relaxed(idx, base + cpu * PER_CPU_OFFSET + > + REG_PERFTBL_SEL_OFFSET); > + return readl_relaxed(base + cpu * PER_CPU_OFFSET + > + REG_PERFTBL_RD_OFFSET); > +} > + > +static int virt_cpufreq_target(struct cpufreq_policy *policy, > + unsigned int target_freq, > + unsigned int relation) > +{ > + struct cpufreq_freqs freqs; > + int ret =3D 0; > + > + freqs.old =3D policy->cur; > + freqs.new =3D target_freq; > + > + cpufreq_freq_transition_begin(policy, &freqs); > + ret =3D virt_cpufreq_set_perf(policy, target_freq); > + cpufreq_freq_transition_end(policy, &freqs, ret !=3D 0); > + > + return ret; > +} > + > +static int virt_cpufreq_get_sharing_cpus(struct cpufreq_policy *policy) > +{ > + u32 cur_perf_domain, perf_domain; > + struct device *cpu_dev; > + int cpu; > + > + cur_perf_domain =3D readl_relaxed(base + policy->cpu * > + PER_CPU_OFFSET + REG_PERF_DOMAIN_= OFFSET); > + > + for_each_possible_cpu(cpu) { > + cpu_dev =3D get_cpu_device(cpu); > + if (!cpu_dev) > + continue; > + > + perf_domain =3D readl_relaxed(base + cpu * > + PER_CPU_OFFSET + REG_PERF_DOM= AIN_OFFSET); > + > + if (perf_domain =3D=3D cur_perf_domain) > + cpumask_set_cpu(cpu, policy->cpus); > + } > + > + return 0; > +} > + > +static int virt_cpufreq_get_freq_info(struct cpufreq_policy *policy) > +{ > + struct cpufreq_frequency_table *table; > + u32 num_perftbl_entries, idx; > + > + num_perftbl_entries =3D per_cpu(perftbl_num_entries, policy->cpu)= ; > + > + if (num_perftbl_entries =3D=3D 1) { > + policy->cpuinfo.min_freq =3D 1; > + policy->cpuinfo.max_freq =3D virt_cpufreq_get_perftbl_ent= ry(policy->cpu, 0); > + > + policy->min =3D policy->cpuinfo.min_freq; > + policy->max =3D policy->cpuinfo.max_freq; > + > + policy->cur =3D policy->max; > + return 0; > + } > + > + table =3D kcalloc(num_perftbl_entries + 1, sizeof(*table), GFP_KE= RNEL); > + if (!table) > + return -ENOMEM; > + > + for (idx =3D 0; idx < num_perftbl_entries; idx++) > + table[idx].frequency =3D virt_cpufreq_get_perftbl_entry(p= olicy->cpu, idx); > + > + table[idx].frequency =3D CPUFREQ_TABLE_END; > + policy->freq_table =3D table; > + > + return 0; > +} > + > +static int virt_cpufreq_cpu_init(struct cpufreq_policy *policy) > +{ > + struct device *cpu_dev; > + int ret; > + > + cpu_dev =3D get_cpu_device(policy->cpu); > + if (!cpu_dev) > + return -ENODEV; > + > + ret =3D virt_cpufreq_get_freq_info(policy); > + if (ret) { > + dev_warn(cpu_dev, "failed to get cpufreq info\n"); > + return ret; > + } > + > + ret =3D virt_cpufreq_get_sharing_cpus(policy); > + if (ret) { > + dev_warn(cpu_dev, "failed to get sharing cpumask\n"); > + return ret; > + } > + > + /* > + * To simplify and improve latency of handling frequency requests= on > + * the host side, this ensures that the vCPU thread triggering th= e MMIO > + * abort is the same thread whose performance constraints (Ex. uc= lamp > + * settings) need to be updated. This simplifies the VMM (Virtual > + * Machine Manager) having to find the correct vCPU thread and/or > + * facing permission issues when configuring other threads. > + */ > + policy->dvfs_possible_from_any_cpu =3D false; > + policy->fast_switch_possible =3D true; > + > + /* > + * Using the default SCALE_FREQ_SOURCE_CPUFREQ is insufficient si= nce > + * the actual physical CPU frequency may not match requested freq= uency > + * from the vCPU thread due to frequency update latencies or othe= r > + * inputs to the physical CPU frequency selection. This additiona= l FIE > + * source allows for more accurate freq_scale updates and only ta= kes > + * effect if another FIE source such as AMUs have not been regist= ered. > + */ > + topology_set_scale_freq_source(&virt_sfd, policy->cpus); > + > + return 0; > +} > + > +static int virt_cpufreq_cpu_exit(struct cpufreq_policy *policy) > +{ > + topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_VIRT, policy->= related_cpus); > + kfree(policy->freq_table); > + return 0; > +} > + > +static int virt_cpufreq_online(struct cpufreq_policy *policy) > +{ > + /* Nothing to restore. */ > + return 0; > +} > + > +static int virt_cpufreq_offline(struct cpufreq_policy *policy) > +{ > + /* Dummy offline() to avoid exit() being called and freeing resou= rces. */ > + return 0; > +} > + > +static int virt_cpufreq_verify_policy(struct cpufreq_policy_data *policy= ) > +{ > + if (policy->freq_table) > + return cpufreq_frequency_table_verify(policy, policy->fre= q_table); > + > + cpufreq_verify_within_cpu_limits(policy); > + return 0; > +} > + > +static struct cpufreq_driver cpufreq_virt_driver =3D { > + .name =3D "virt-cpufreq", > + .init =3D virt_cpufreq_cpu_init, > + .exit =3D virt_cpufreq_cpu_exit, > + .online =3D virt_cpufreq_online, > + .offline =3D virt_cpufreq_offline, > + .verify =3D virt_cpufreq_verify_policy, > + .target =3D virt_cpufreq_target, > + .fast_switch =3D virt_cpufreq_fast_switch, > + .attr =3D cpufreq_generic_attr, > +}; > + > +static int virt_cpufreq_driver_probe(struct platform_device *pdev) > +{ > + u32 num_perftbl_entries; > + int ret, cpu; > + > + base =3D devm_platform_ioremap_resource(pdev, 0); > + if (IS_ERR(base)) > + return PTR_ERR(base); > + > + for_each_possible_cpu(cpu) { > + num_perftbl_entries =3D readl_relaxed(base + cpu * PER_CP= U_OFFSET + > + REG_PERFTBL_LEN_OFFSE= T); > + > + if (!num_perftbl_entries || num_perftbl_entries > PERFTBL= _MAX_ENTRIES) > + return -ENODEV; > + > + per_cpu(perftbl_num_entries, cpu) =3D num_perftbl_entries= ; > + } > + > + ret =3D cpufreq_register_driver(&cpufreq_virt_driver); > + if (ret) { > + dev_err(&pdev->dev, "Virtual CPUFreq driver failed to reg= ister: %d\n", ret); > + return ret; > + } > + > + dev_dbg(&pdev->dev, "Virtual CPUFreq driver initialized\n"); > + return 0; > +} > + > +static int virt_cpufreq_driver_remove(struct platform_device *pdev) > +{ > + cpufreq_unregister_driver(&cpufreq_virt_driver); > + return 0; > +} > + > +static const struct of_device_id virt_cpufreq_match[] =3D { > + { .compatible =3D "qemu,virtual-cpufreq", .data =3D NULL}, > + {} > +}; > +MODULE_DEVICE_TABLE(of, virt_cpufreq_match); > + > +static struct platform_driver virt_cpufreq_driver =3D { > + .probe =3D virt_cpufreq_driver_probe, > + .remove =3D virt_cpufreq_driver_remove, > + .driver =3D { > + .name =3D "virt-cpufreq", > + .of_match_table =3D virt_cpufreq_match, > + }, > +}; > + > +static int __init virt_cpufreq_init(void) > +{ > + return platform_driver_register(&virt_cpufreq_driver); > +} > +postcore_initcall(virt_cpufreq_init); > + > +static void __exit virt_cpufreq_exit(void) > +{ > + platform_driver_unregister(&virt_cpufreq_driver); > +} > +module_exit(virt_cpufreq_exit); > + > +MODULE_DESCRIPTION("Virtual cpufreq driver"); > +MODULE_LICENSE("GPL"); > diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.= h > index b721f360d759..d5d848849408 100644 > --- a/include/linux/arch_topology.h > +++ b/include/linux/arch_topology.h > @@ -49,6 +49,7 @@ enum scale_freq_source { > SCALE_FREQ_SOURCE_CPUFREQ =3D 0, > SCALE_FREQ_SOURCE_ARCH, > SCALE_FREQ_SOURCE_CPPC, > + SCALE_FREQ_SOURCE_VIRT, > }; > > struct scale_freq_data { > -- > 2.45.0.215.g3402c0e53f-goog >