[RESEND][PATCH v2 0/4] Update Energy Model after chip binning adjusted voltages

[RESEND][PATCH v2 0/4] Update Energy Model after chip binning adjusted voltages

[Lukasz Luba]

Hi all,

This is a follow-up patch aiming to add EM modification due to chip binning.
The first RFC and the discussion can be found here [1].

It uses Exynos chip driver code as a 1st user. The EM framework has been
extended to handle this use case easily, when the voltage has been changed
after setup. On my Odroid-xu4 in some OPPs I can observe ~20% power difference.
According to that data in driver tables it could be up to ~29%.

This chip binning is applicable to a lot of SoCs, so the EM framework should
make it easy to update. It uses the existing OPP and DT information to
re-calculate the new power values.


Changes:
v2:
- removed 'ret' from error message which wasn't initialized (Christian)
v1:
- exported the OPP calculation function from the OPP/OF so it can be
  used from EM fwk (Viresh)
- refactored EM updating function to re-use common code
- added new EM function which can be used by chip device drivers which
  modify the voltage in OPPs
RFC is at [1]

Regards,
Lukasz Luba

[1] https://lore.kernel.org/lkml/20231220110339.1065505-1-lukasz.luba@arm.com/

Lukasz Luba (4):
  OPP: OF: Export dev_opp_pm_calc_power() for usage from EM
  PM: EM: Change the em_adjust_new_capacity() to re-use code
  PM: EM: Add em_dev_update_chip_binning()
  soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

 drivers/opp/of.c                 |  17 +++--
 drivers/soc/samsung/exynos-asv.c |  11 +++-
 include/linux/energy_model.h     |   5 ++
 include/linux/pm_opp.h           |   8 +++
 kernel/power/energy_model.c      | 109 +++++++++++++++++++++++++------
 5 files changed, 125 insertions(+), 25 deletions(-)

-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RESEND][PATCH v2 1/4] OPP: OF: Export dev_opp_pm_calc_power() for usage from EM

[Lukasz Luba]

There are device drivers which can modify voltage values for OPPs. It
could be due to the chip binning and those drivers have specific chip
knowledge about it. This adjustment can happen after Energy Model is
registered, thus EM can have stale data about power.

Export dev_opp_pm_calc_power() which can be used by Energy Model to
calculate new power with the new voltage for OPPs.

Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
---
 drivers/opp/of.c       | 17 ++++++++++++-----
 include/linux/pm_opp.h |  8 ++++++++
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/drivers/opp/of.c b/drivers/opp/of.c
index f9f0b22bccbb4..282eb5966fd03 100644
--- a/drivers/opp/of.c
+++ b/drivers/opp/of.c
@@ -1494,20 +1494,26 @@ _get_dt_power(struct device *dev, unsigned long *uW, unsigned long *kHz)
 	return 0;
 }
 
-/*
- * Callback function provided to the Energy Model framework upon registration.
+/**
+ * dev_pm_opp_calc_power() - Calculate power value for device with EM
+ * @dev		: Device for which an Energy Model has to be registered
+ * @uW		: New power value that is calculated
+ * @kHz		: Frequency for which the new power is calculated
+ *
  * This computes the power estimated by @dev at @kHz if it is the frequency
  * of an existing OPP, or at the frequency of the first OPP above @kHz otherwise
  * (see dev_pm_opp_find_freq_ceil()). This function updates @kHz to the ceiled
  * frequency and @uW to the associated power. The power is estimated as
  * P = C * V^2 * f with C being the device's capacitance and V and f
  * respectively the voltage and frequency of the OPP.
+ * It is also used as a callback function provided to the Energy Model
+ * framework upon registration.
  *
  * Returns -EINVAL if the power calculation failed because of missing
  * parameters, 0 otherwise.
  */
-static int __maybe_unused _get_power(struct device *dev, unsigned long *uW,
-				     unsigned long *kHz)
+int dev_pm_opp_calc_power(struct device *dev, unsigned long *uW,
+			  unsigned long *kHz)
 {
 	struct dev_pm_opp *opp;
 	struct device_node *np;
@@ -1544,6 +1550,7 @@ static int __maybe_unused _get_power(struct device *dev, unsigned long *uW,
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(dev_pm_opp_calc_power);
 
 static bool _of_has_opp_microwatt_property(struct device *dev)
 {
@@ -1619,7 +1626,7 @@ int dev_pm_opp_of_register_em(struct device *dev, struct cpumask *cpus)
 		goto failed;
 	}
 
-	EM_SET_ACTIVE_POWER_CB(em_cb, _get_power);
+	EM_SET_ACTIVE_POWER_CB(em_cb, dev_pm_opp_calc_power);
 
 register_em:
 	ret = em_dev_register_perf_domain(dev, nr_opp, &em_cb, cpus, true);
diff --git a/include/linux/pm_opp.h b/include/linux/pm_opp.h
index 065a47382302c..31370deb9905f 100644
--- a/include/linux/pm_opp.h
+++ b/include/linux/pm_opp.h
@@ -476,6 +476,8 @@ struct device_node *dev_pm_opp_get_of_node(struct dev_pm_opp *opp);
 int of_get_required_opp_performance_state(struct device_node *np, int index);
 int dev_pm_opp_of_find_icc_paths(struct device *dev, struct opp_table *opp_table);
 int dev_pm_opp_of_register_em(struct device *dev, struct cpumask *cpus);
+int dev_pm_opp_calc_power(struct device *dev, unsigned long *uW,
+			  unsigned long *kHz);
 static inline void dev_pm_opp_of_unregister_em(struct device *dev)
 {
 	em_dev_unregister_perf_domain(dev);
@@ -539,6 +541,12 @@ static inline void dev_pm_opp_of_unregister_em(struct device *dev)
 {
 }
 
+static inline int dev_pm_opp_calc_power(struct device *dev, unsigned long *uW,
+			  unsigned long *kHz)
+{
+	return -EOPNOTSUPP;
+}
+
 static inline int of_get_required_opp_performance_state(struct device_node *np, int index)
 {
 	return -EOPNOTSUPP;
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 1/4] OPP: OF: Export dev_opp_pm_calc_power() for usage from EM

[Viresh Kumar]

On 22-03-24, 11:08, Lukasz Luba wrote:
> There are device drivers which can modify voltage values for OPPs. It
> could be due to the chip binning and those drivers have specific chip
> knowledge about it. This adjustment can happen after Energy Model is
> registered, thus EM can have stale data about power.
> 
> Export dev_opp_pm_calc_power() which can be used by Energy Model to
> calculate new power with the new voltage for OPPs.
> 
> Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
> ---
>  drivers/opp/of.c       | 17 ++++++++++++-----
>  include/linux/pm_opp.h |  8 ++++++++
>  2 files changed, 20 insertions(+), 5 deletions(-)

Acked-by: Viresh Kumar <viresh.kumar@linaro.org>

-- 
viresh

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 1/4] OPP: OF: Export dev_opp_pm_calc_power() for usage from EM

[Lukasz Luba]

On 3/26/24 02:51, Viresh Kumar wrote:
> On 22-03-24, 11:08, Lukasz Luba wrote:
>> There are device drivers which can modify voltage values for OPPs. It
>> could be due to the chip binning and those drivers have specific chip
>> knowledge about it. This adjustment can happen after Energy Model is
>> registered, thus EM can have stale data about power.
>>
>> Export dev_opp_pm_calc_power() which can be used by Energy Model to
>> calculate new power with the new voltage for OPPs.
>>
>> Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
>> ---
>>   drivers/opp/of.c       | 17 ++++++++++++-----
>>   include/linux/pm_opp.h |  8 ++++++++
>>   2 files changed, 20 insertions(+), 5 deletions(-)
> 
> Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
> 

Thanks Viresh for the ACK!

Regards,
Lukasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RESEND][PATCH v2 2/4] PM: EM: Change the em_adjust_new_capacity() to re-use code

[Lukasz Luba]

There is going to be a new update function addressing chip binning.
Therefore, some common code which can be refactored and called from
upcoming changes and em_adjust_new_capacity(). In this way the code
duplication can be avoided.

Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
---
 kernel/power/energy_model.c | 58 +++++++++++++++++++++++++------------
 1 file changed, 39 insertions(+), 19 deletions(-)

diff --git a/kernel/power/energy_model.c b/kernel/power/energy_model.c
index 9e1c9aa399ea9..6960dd7393b2d 100644
--- a/kernel/power/energy_model.c
+++ b/kernel/power/energy_model.c
@@ -674,23 +674,15 @@ void em_dev_unregister_perf_domain(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(em_dev_unregister_perf_domain);
 
-/*
- * Adjustment of CPU performance values after boot, when all CPUs capacites
- * are correctly calculated.
- */
-static void em_adjust_new_capacity(struct device *dev,
-				   struct em_perf_domain *pd,
-				   u64 max_cap)
+static struct em_perf_table __rcu *em_table_dup(struct em_perf_domain *pd)
 {
 	struct em_perf_table __rcu *em_table;
 	struct em_perf_state *ps, *new_ps;
-	int ret, ps_size;
+	int ps_size;
 
 	em_table = em_table_alloc(pd);
-	if (!em_table) {
-		dev_warn(dev, "EM: allocation failed\n");
-		return;
-	}
+	if (!em_table)
+		return NULL;
 
 	new_ps = em_table->state;
 
@@ -702,24 +694,52 @@ static void em_adjust_new_capacity(struct device *dev,
 
 	rcu_read_unlock();
 
-	em_init_performance(dev, pd, new_ps, pd->nr_perf_states);
-	ret = em_compute_costs(dev, new_ps, NULL, pd->nr_perf_states,
+	return em_table;
+}
+
+static int em_recalc_and_update(struct device *dev, struct em_perf_domain *pd,
+				struct em_perf_table __rcu *em_table)
+{
+	int ret;
+
+	ret = em_compute_costs(dev, em_table->state, NULL, pd->nr_perf_states,
 			       pd->flags);
-	if (ret) {
-		dev_warn(dev, "EM: compute costs failed\n");
-		return;
-	}
+	if (ret)
+		goto free_em_table;
 
 	ret = em_dev_update_perf_domain(dev, em_table);
 	if (ret)
-		dev_warn(dev, "EM: update failed %d\n", ret);
+		goto free_em_table;
 
 	/*
 	 * This is one-time-update, so give up the ownership in this updater.
 	 * The EM framework has incremented the usage counter and from now
 	 * will keep the reference (then free the memory when needed).
 	 */
+free_em_table:
 	em_table_free(em_table);
+	return ret;
+}
+
+/*
+ * Adjustment of CPU performance values after boot, when all CPUs capacites
+ * are correctly calculated.
+ */
+static void em_adjust_new_capacity(struct device *dev,
+				   struct em_perf_domain *pd,
+				   u64 max_cap)
+{
+	struct em_perf_table __rcu *em_table;
+
+	em_table = em_table_dup(pd);
+	if (!em_table) {
+		dev_warn(dev, "EM: allocation failed\n");
+		return;
+	}
+
+	em_init_performance(dev, pd, em_table->state, pd->nr_perf_states);
+
+	em_recalc_and_update(dev, pd, em_table);
 }
 
 static void em_check_capacity_update(void)
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 2/4] PM: EM: Change the em_adjust_new_capacity() to re-use code

[Dietmar Eggemann]

On 22/03/2024 12:08, Lukasz Luba wrote:

Maybe better : "PM: EM: Refactoring em_adjust_new_capacity()" ?

> There is going to be a new update function addressing chip binning.
> Therefore, some common code which can be refactored and called from
> upcoming changes and em_adjust_new_capacity(). In this way the code
> duplication can be avoided.

IMHO, that's hard to digest.

Extract em_table_dup() and em_recalc_and_update() from
em_adjust_new_capacity(). Both functions will be later reused by the
'update EM due to chip binning' functionality.

[...]

> +static int em_recalc_and_update(struct device *dev, struct em_perf_domain *pd,
> +				struct em_perf_table __rcu *em_table)
> +{
> +	int ret;
> +
> +	ret = em_compute_costs(dev, em_table->state, NULL, pd->nr_perf_states,
>  			       pd->flags);
> -	if (ret) {
> -		dev_warn(dev, "EM: compute costs failed\n");
> -		return;
> -	}
> +	if (ret)
> +		goto free_em_table;

There seems to be a subtle change in this patch. When em_compute_costs()
fails now em_table_free() is called. This wasn't the case before when
em_compute_costs() was directly called from em_adjust_new_capacity().

[...]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 2/4] PM: EM: Change the em_adjust_new_capacity() to re-use code

[Lukasz Luba]

On 3/26/24 10:51, Dietmar Eggemann wrote:
> On 22/03/2024 12:08, Lukasz Luba wrote:
> 
> Maybe better : "PM: EM: Refactoring em_adjust_new_capacity()" ?
> 
>> There is going to be a new update function addressing chip binning.
>> Therefore, some common code which can be refactored and called from
>> upcoming changes and em_adjust_new_capacity(). In this way the code
>> duplication can be avoided.
> 
> IMHO, that's hard to digest.
> 
> Extract em_table_dup() and em_recalc_and_update() from
> em_adjust_new_capacity(). Both functions will be later reused by the
> 'update EM due to chip binning' functionality.

That looks good, I'll update it.

> 
> [...]
> 
>> +static int em_recalc_and_update(struct device *dev, struct em_perf_domain *pd,
>> +				struct em_perf_table __rcu *em_table)
>> +{
>> +	int ret;
>> +
>> +	ret = em_compute_costs(dev, em_table->state, NULL, pd->nr_perf_states,
>>   			       pd->flags);
>> -	if (ret) {
>> -		dev_warn(dev, "EM: compute costs failed\n");
>> -		return;
>> -	}
>> +	if (ret)
>> +		goto free_em_table;
> 
> There seems to be a subtle change in this patch. When em_compute_costs()
> fails now em_table_free() is called. This wasn't the case before when
> em_compute_costs() was directly called from em_adjust_new_capacity().

Yes, I've refactored it to explicitly call to the same free_em_table
for both fails in the new code. It should have been done in old code.

> 
> [...]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RESEND][PATCH v2 3/4] PM: EM: Add em_dev_update_chip_binning()

[Lukasz Luba]

Add a function which allows to modify easily the EM after the new voltage
information is available. The device drivers for the chip can adjust
the voltage values after setup. The voltage for the same frequency in OPP
can be different due to chip binning. The voltage impacts the power usage
and the EM power values can be updated to reflect that.

Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
---
 include/linux/energy_model.h |  5 ++++
 kernel/power/energy_model.c  | 51 ++++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/include/linux/energy_model.h b/include/linux/energy_model.h
index 770755df852f1..d30d67c2f07cf 100644
--- a/include/linux/energy_model.h
+++ b/include/linux/energy_model.h
@@ -172,6 +172,7 @@ struct em_perf_table __rcu *em_table_alloc(struct em_perf_domain *pd);
 void em_table_free(struct em_perf_table __rcu *table);
 int em_dev_compute_costs(struct device *dev, struct em_perf_state *table,
 			 int nr_states);
+int em_dev_update_chip_binning(struct device *dev);
 
 /**
  * em_pd_get_efficient_state() - Get an efficient performance state from the EM
@@ -387,6 +388,10 @@ int em_dev_compute_costs(struct device *dev, struct em_perf_state *table,
 {
 	return -EINVAL;
 }
+static inline int em_dev_update_chip_binning(struct device *dev)
+{
+	return -EINVAL;
+}
 #endif
 
 #endif
diff --git a/kernel/power/energy_model.c b/kernel/power/energy_model.c
index 6960dd7393b2d..f7f7ae34ec552 100644
--- a/kernel/power/energy_model.c
+++ b/kernel/power/energy_model.c
@@ -808,3 +808,54 @@ static void em_update_workfn(struct work_struct *work)
 {
 	em_check_capacity_update();
 }
+
+/**
+ * em_dev_update_chip_binning() - Update Energy Model with new values after
+ *			the new voltage information is present in the OPPs.
+ * @dev		: Device for which the Energy Model has to be updated.
+ *
+ * This function allows to update easily the EM with new values available in
+ * the OPP framework and DT. It can be used after the chip has been properly
+ * verified by device drivers and the voltages adjusted for the 'chip binning'.
+ * It uses the "dynamic-power-coefficient" DT property to calculate the power
+ * values for EM. For power calculation it uses the new adjusted voltage
+ * values known for OPPs, which might be changed after boot.
+ */
+int em_dev_update_chip_binning(struct device *dev)
+{
+	struct em_perf_table __rcu *em_table;
+	struct em_perf_domain *pd;
+	int i, ret;
+
+	if (IS_ERR_OR_NULL(dev))
+		return -EINVAL;
+
+	pd = em_pd_get(dev);
+	if (!pd) {
+		dev_warn(dev, "Couldn't find Energy Model\n");
+		return -EINVAL;
+	}
+
+	em_table = em_table_dup(pd);
+	if (!em_table) {
+		dev_warn(dev, "EM: allocation failed\n");
+		return -ENOMEM;
+	}
+
+	/* Update power values which might change due to new voltage in OPPs */
+	for (i = 0; i < pd->nr_perf_states; i++) {
+		unsigned long freq = em_table->state[i].frequency;
+		unsigned long power;
+
+		ret = dev_pm_opp_calc_power(dev, &power, &freq);
+		if (ret) {
+			em_table_free(em_table);
+			return ret;
+		}
+
+		em_table->state[i].power = power;
+	}
+
+	return em_recalc_and_update(dev, pd, em_table);
+}
+EXPORT_SYMBOL_GPL(em_dev_update_chip_binning);
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 3/4] PM: EM: Add em_dev_update_chip_binning()

[Dietmar Eggemann]

On 22/03/2024 12:08, Lukasz Luba wrote:

> diff --git a/kernel/power/energy_model.c b/kernel/power/energy_model.c
> index 6960dd7393b2d..f7f7ae34ec552 100644
> --- a/kernel/power/energy_model.c
> +++ b/kernel/power/energy_model.c
> @@ -808,3 +808,54 @@ static void em_update_workfn(struct work_struct *work)
>  {
>  	em_check_capacity_update();
>  }
> +
> +/**
> + * em_dev_update_chip_binning() - Update Energy Model with new values after

s/with new values// ... IMHO this should be obvious ?

> + *			the new voltage information is present in the OPPs.
> + * @dev		: Device for which the Energy Model has to be updated.
> + *
> + * This function allows to update easily the EM with new values available in
> + * the OPP framework and DT. It can be used after the chip has been properly
> + * verified by device drivers and the voltages adjusted for the 'chip binning'.
> + * It uses the "dynamic-power-coefficient" DT property to calculate the power
> + * values for EM. For power calculation it uses the new adjusted voltage
> + * values known for OPPs, which might be changed after boot.

The last two sentences describe what dev_pm_opp_calc_power() is doing.
Maybe this can be made clearer here?

> + */
> +int em_dev_update_chip_binning(struct device *dev)

This is the old dev_pm_opp_of_update_em() right?

> +{
> +	struct em_perf_table __rcu *em_table;
> +	struct em_perf_domain *pd;
> +	int i, ret;
> +
> +	if (IS_ERR_OR_NULL(dev))
> +		return -EINVAL;

When do you use if '(IS_ERR_OR_NULL(dev))' and when 'if(!dev)' for EM
interface functions?

> +	pd = em_pd_get(dev);
> +	if (!pd) {
> +		dev_warn(dev, "Couldn't find Energy Model\n");
> +		return -EINVAL;
> +	}
> +
> +	em_table = em_table_dup(pd);
> +	if (!em_table) {
> +		dev_warn(dev, "EM: allocation failed\n");
> +		return -ENOMEM;
> +	}
> +
> +	/* Update power values which might change due to new voltage in OPPs */
> +	for (i = 0; i < pd->nr_perf_states; i++) {
> +		unsigned long freq = em_table->state[i].frequency;
> +		unsigned long power;
> +
> +		ret = dev_pm_opp_calc_power(dev, &power, &freq);
> +		if (ret) {
> +			em_table_free(em_table);
> +			return ret;
> +		}
> +
> +		em_table->state[i].power = power;
> +	}
> +
> +	return em_recalc_and_update(dev, pd, em_table);
> +}
> +EXPORT_SYMBOL_GPL(em_dev_update_chip_binning);

In the previous version of 'chip-binning' you were using the new EM
interface em_dev_compute_costs() (1) which is now replaced by
em_recalc_and_update() -> em_compute_costs().

https://lkml.kernel.org/r/20231220110339.1065505-2-lukasz.luba@arm.com

Which leaves (1) still unused.

That was why my concern back then that we shouldn't introduce EM
interfaces without a user:

https://lkml.kernel.org/r/8fc499cf-fca1-4465-bff7-a93dfd36f3c8@arm.com

What happens now with em_dev_compute_costs()?





_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 3/4] PM: EM: Add em_dev_update_chip_binning()

[Lukasz Luba]

On 3/26/24 10:09, Dietmar Eggemann wrote:
> On 22/03/2024 12:08, Lukasz Luba wrote:
> 
>> diff --git a/kernel/power/energy_model.c b/kernel/power/energy_model.c
>> index 6960dd7393b2d..f7f7ae34ec552 100644
>> --- a/kernel/power/energy_model.c
>> +++ b/kernel/power/energy_model.c
>> @@ -808,3 +808,54 @@ static void em_update_workfn(struct work_struct *work)
>>   {
>>   	em_check_capacity_update();
>>   }
>> +
>> +/**
>> + * em_dev_update_chip_binning() - Update Energy Model with new values after
> 
> s/with new values// ... IMHO this should be obvious ?

Make sense

> 
>> + *			the new voltage information is present in the OPPs.
>> + * @dev		: Device for which the Energy Model has to be updated.
>> + *
>> + * This function allows to update easily the EM with new values available in
>> + * the OPP framework and DT. It can be used after the chip has been properly
>> + * verified by device drivers and the voltages adjusted for the 'chip binning'.
>> + * It uses the "dynamic-power-coefficient" DT property to calculate the power
>> + * values for EM. For power calculation it uses the new adjusted voltage
>> + * values known for OPPs, which might be changed after boot.
> 
> The last two sentences describe what dev_pm_opp_calc_power() is doing.
> Maybe this can be made clearer here?

Or I can just remove this, since it's too detailed description.

> 
>> + */
>> +int em_dev_update_chip_binning(struct device *dev)
> 
> This is the old dev_pm_opp_of_update_em() right?

Yes, it is similar.

> 
>> +{
>> +	struct em_perf_table __rcu *em_table;
>> +	struct em_perf_domain *pd;
>> +	int i, ret;
>> +
>> +	if (IS_ERR_OR_NULL(dev))
>> +		return -EINVAL;
> 
> When do you use if '(IS_ERR_OR_NULL(dev))' and when 'if(!dev)' for EM
> interface functions?

Sometimes IS_ERR_OR_NULL is used, especially for API function other
that register function.

> 
>> +	pd = em_pd_get(dev);
>> +	if (!pd) {
>> +		dev_warn(dev, "Couldn't find Energy Model\n");
>> +		return -EINVAL;
>> +	}
>> +
>> +	em_table = em_table_dup(pd);
>> +	if (!em_table) {
>> +		dev_warn(dev, "EM: allocation failed\n");
>> +		return -ENOMEM;
>> +	}
>> +
>> +	/* Update power values which might change due to new voltage in OPPs */
>> +	for (i = 0; i < pd->nr_perf_states; i++) {
>> +		unsigned long freq = em_table->state[i].frequency;
>> +		unsigned long power;
>> +
>> +		ret = dev_pm_opp_calc_power(dev, &power, &freq);
>> +		if (ret) {
>> +			em_table_free(em_table);
>> +			return ret;
>> +		}
>> +
>> +		em_table->state[i].power = power;
>> +	}
>> +
>> +	return em_recalc_and_update(dev, pd, em_table);
>> +}
>> +EXPORT_SYMBOL_GPL(em_dev_update_chip_binning);
> 
> In the previous version of 'chip-binning' you were using the new EM
> interface em_dev_compute_costs() (1) which is now replaced by
> em_recalc_and_update() -> em_compute_costs().
> 
> https://lkml.kernel.org/r/20231220110339.1065505-2-lukasz.luba@arm.com
> 
> Which leaves (1) still unused.
> 
> That was why my concern back then that we shouldn't introduce EM
> interfaces without a user:
> 
> https://lkml.kernel.org/r/8fc499cf-fca1-4465-bff7-a93dfd36f3c8@arm.com
> 
> What happens now with em_dev_compute_costs()?
> 

For now it's not used, but modules which will create new EMs
with custom power values will use it. When such a module have
e.g. 5 EMs for one PD and only switches on one of them, then
this em_dev_compute_costs() will be used at setup for those
5 EMs. Later it won't be used.
I don't wanted to combine the registration of new EM with
the compute cost, because that will create overhead in the
switching to new EM code path. Now we have only ~3us, which
was the main goal.

When our scmi-cpufreq get the support for EM update this
compute cost will be used there.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 3/4] PM: EM: Add em_dev_update_chip_binning()

[Dietmar Eggemann]

On 26/03/2024 21:32, Lukasz Luba wrote:
> 
> 
> On 3/26/24 10:09, Dietmar Eggemann wrote:
>> On 22/03/2024 12:08, Lukasz Luba wrote:

[...]

>>> +    return em_recalc_and_update(dev, pd, em_table);
>>> +}
>>> +EXPORT_SYMBOL_GPL(em_dev_update_chip_binning);
>>
>> In the previous version of 'chip-binning' you were using the new EM
>> interface em_dev_compute_costs() (1) which is now replaced by
>> em_recalc_and_update() -> em_compute_costs().
>>
>> https://lkml.kernel.org/r/20231220110339.1065505-2-lukasz.luba@arm.com
>>
>> Which leaves (1) still unused.
>>
>> That was why my concern back then that we shouldn't introduce EM
>> interfaces without a user:
>>
>> https://lkml.kernel.org/r/8fc499cf-fca1-4465-bff7-a93dfd36f3c8@arm.com
>>
>> What happens now with em_dev_compute_costs()?
>>
> 
> For now it's not used, but modules which will create new EMs
> with custom power values will use it. When such a module have
> e.g. 5 EMs for one PD and only switches on one of them, then
> this em_dev_compute_costs() will be used at setup for those
> 5 EMs. Later it won't be used.
> I don't wanted to combine the registration of new EM with
> the compute cost, because that will create overhead in the
> switching to new EM code path. Now we have only ~3us, which
> was the main goal.
> 
> When our scmi-cpufreq get the support for EM update this
> compute cost will be used there.

OK, I see. I checked the reloadable EM test module and
em_dev_compute_costs() is used there like you described it.


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 3/4] PM: EM: Add em_dev_update_chip_binning()

[Dietmar Eggemann]

On 27/03/2024 13:55, Dietmar Eggemann wrote:
> On 26/03/2024 21:32, Lukasz Luba wrote:
>>
>>
>> On 3/26/24 10:09, Dietmar Eggemann wrote:
>>> On 22/03/2024 12:08, Lukasz Luba wrote:
> 
> [...]
> 
>>>> +    return em_recalc_and_update(dev, pd, em_table);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(em_dev_update_chip_binning);
>>>
>>> In the previous version of 'chip-binning' you were using the new EM
>>> interface em_dev_compute_costs() (1) which is now replaced by
>>> em_recalc_and_update() -> em_compute_costs().
>>>
>>> https://lkml.kernel.org/r/20231220110339.1065505-2-lukasz.luba@arm.com
>>>
>>> Which leaves (1) still unused.
>>>
>>> That was why my concern back then that we shouldn't introduce EM
>>> interfaces without a user:
>>>
>>> https://lkml.kernel.org/r/8fc499cf-fca1-4465-bff7-a93dfd36f3c8@arm.com
>>>
>>> What happens now with em_dev_compute_costs()?
>>>
>>
>> For now it's not used, but modules which will create new EMs
>> with custom power values will use it. When such a module have
>> e.g. 5 EMs for one PD and only switches on one of them, then
>> this em_dev_compute_costs() will be used at setup for those
>> 5 EMs. Later it won't be used.
>> I don't wanted to combine the registration of new EM with
>> the compute cost, because that will create overhead in the
>> switching to new EM code path. Now we have only ~3us, which
>> was the main goal.
>>
>> When our scmi-cpufreq get the support for EM update this
>> compute cost will be used there.
> 
> OK, I see. I checked the reloadable EM test module and
> em_dev_compute_costs() is used there like you described it.

I had a second look and IMHO the layout is like this:

Internal (1) and external (2,3) 'reloadable EM' use cases:
(EM alloc and free not depicted)

1. Late CPUs booting (CPU capacity adjustment)

 e3f1164fc9ee  PM: EM: Support late CPUs booting and capacity adjustment

 schedule_delayed_work(&em_update_work, ...)

  em_update_workfn()
   em_check_capacity_update()
    em_adjust_new_capacity()
     em_recalc_and_update()       <-- (i)
      em_compute_costs()          <-- (ii)
      em_dev_update_perf_domain() <-- external

2. Reload EM from driver

 22ea02848c07  PM: EM: Add em_dev_compute_costs()
 977230d5d503  PM: EM: Introduce em_dev_update_perf_domain() for EM
               updates

 em_dev_compute_costs()           <-- external
  em_compute_costs()              <-- (ii)
 em_dev_update_perf_domain()      <-- external

3. Chip binning

 this patchset  PM: EM: Add em_dev_update_chip_binning()

 em_dev_update_chip_binning()     <-- external
  em_recalc_and_update()          <-- (i)
   em_compute_costs()             <-- (ii)
   em_dev_update_perf_domain()    <-- external




_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 3/4] PM: EM: Add em_dev_update_chip_binning()

[Lukasz Luba]

On 3/28/24 07:21, Dietmar Eggemann wrote:
> On 27/03/2024 13:55, Dietmar Eggemann wrote:
>> On 26/03/2024 21:32, Lukasz Luba wrote:
>>>
>>>
>>> On 3/26/24 10:09, Dietmar Eggemann wrote:
>>>> On 22/03/2024 12:08, Lukasz Luba wrote:
>>
>> [...]
>>
>>>>> +    return em_recalc_and_update(dev, pd, em_table);
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(em_dev_update_chip_binning);
>>>>
>>>> In the previous version of 'chip-binning' you were using the new EM
>>>> interface em_dev_compute_costs() (1) which is now replaced by
>>>> em_recalc_and_update() -> em_compute_costs().
>>>>
>>>> https://lkml.kernel.org/r/20231220110339.1065505-2-lukasz.luba@arm.com
>>>>
>>>> Which leaves (1) still unused.
>>>>
>>>> That was why my concern back then that we shouldn't introduce EM
>>>> interfaces without a user:
>>>>
>>>> https://lkml.kernel.org/r/8fc499cf-fca1-4465-bff7-a93dfd36f3c8@arm.com
>>>>
>>>> What happens now with em_dev_compute_costs()?
>>>>
>>>
>>> For now it's not used, but modules which will create new EMs
>>> with custom power values will use it. When such a module have
>>> e.g. 5 EMs for one PD and only switches on one of them, then
>>> this em_dev_compute_costs() will be used at setup for those
>>> 5 EMs. Later it won't be used.
>>> I don't wanted to combine the registration of new EM with
>>> the compute cost, because that will create overhead in the
>>> switching to new EM code path. Now we have only ~3us, which
>>> was the main goal.
>>>
>>> When our scmi-cpufreq get the support for EM update this
>>> compute cost will be used there.
>>
>> OK, I see. I checked the reloadable EM test module and
>> em_dev_compute_costs() is used there like you described it.
> 
> I had a second look and IMHO the layout is like this:
> 
> Internal (1) and external (2,3) 'reloadable EM' use cases:
> (EM alloc and free not depicted)
> 
> 1. Late CPUs booting (CPU capacity adjustment)
> 
>   e3f1164fc9ee  PM: EM: Support late CPUs booting and capacity adjustment
> 
>   schedule_delayed_work(&em_update_work, ...)
> 
>    em_update_workfn()
>     em_check_capacity_update()
>      em_adjust_new_capacity()
>       em_recalc_and_update()       <-- (i)
>        em_compute_costs()          <-- (ii)
>        em_dev_update_perf_domain() <-- external
> 
> 2. Reload EM from driver
> 
>   22ea02848c07  PM: EM: Add em_dev_compute_costs()
>   977230d5d503  PM: EM: Introduce em_dev_update_perf_domain() for EM
>                 updates
> 
>   em_dev_compute_costs()           <-- external
>    em_compute_costs()              <-- (ii)
>   em_dev_update_perf_domain()      <-- external
> 
> 3. Chip binning
> 
>   this patchset  PM: EM: Add em_dev_update_chip_binning()
> 
>   em_dev_update_chip_binning()     <-- external
>    em_recalc_and_update()          <-- (i)
>     em_compute_costs()             <-- (ii)
>     em_dev_update_perf_domain()    <-- external
> 
> 
> 
> 

Yes, that's correct.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Lukasz Luba]

When the voltage for OPPs is adjusted there is a need to also update
Energy Model framework. The EM data contains power values which depend
on voltage values. The EM structure is used for thermal (IPA governor)
and in scheduler task placement (EAS) so it should reflect the real HW
model as best as possible to operate properly.

Based on data on Exynos5422 ASV tables the maximum power difference might
be ~29%. An Odroid-XU4 (with a random sample SoC in this chip lottery)
showed power difference for some OPPs ~20%. Therefore, it's worth to
update the EM.

Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
---
 drivers/soc/samsung/exynos-asv.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/soc/samsung/exynos-asv.c b/drivers/soc/samsung/exynos-asv.c
index d60af8acc3916..bd6bb2cab2cd8 100644
--- a/drivers/soc/samsung/exynos-asv.c
+++ b/drivers/soc/samsung/exynos-asv.c
@@ -11,6 +11,7 @@
 
 #include <linux/cpu.h>
 #include <linux/device.h>
+#include <linux/energy_model.h>
 #include <linux/errno.h>
 #include <linux/of.h>
 #include <linux/pm_opp.h>
@@ -97,9 +98,17 @@ static int exynos_asv_update_opps(struct exynos_asv *asv)
 			last_opp_table = opp_table;
 
 			ret = exynos_asv_update_cpu_opps(asv, cpu);
-			if (ret < 0)
+			if (!ret) {
+				/*
+				 * When the voltage for OPPs successfully
+				 * changed, update the EM power values to
+				 * reflect the reality and not use stale data
+				 */
+				em_dev_update_chip_binning(cpu);
+			} else {
 				dev_err(asv->dev, "Couldn't udate OPPs for cpu%d\n",
 					cpuid);
+			}
 		}
 
 		dev_pm_opp_put_opp_table(opp_table);
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Krzysztof Kozlowski]

On 22/03/2024 12:08, Lukasz Luba wrote:
> When the voltage for OPPs is adjusted there is a need to also update
> Energy Model framework. The EM data contains power values which depend
> on voltage values. The EM structure is used for thermal (IPA governor)
> and in scheduler task placement (EAS) so it should reflect the real HW
> model as best as possible to operate properly.
> 
> Based on data on Exynos5422 ASV tables the maximum power difference might
> be ~29%. An Odroid-XU4 (with a random sample SoC in this chip lottery)
> showed power difference for some OPPs ~20%. Therefore, it's worth to
> update the EM.
> 
> Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
> ---
>  drivers/soc/samsung/exynos-asv.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)

I assume there is dependency, even though cover letter did not mention
it, so:

Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Best regards,
Krzysztof


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Lukasz Luba]

On 3/25/24 19:17, Krzysztof Kozlowski wrote:
> On 22/03/2024 12:08, Lukasz Luba wrote:
>> When the voltage for OPPs is adjusted there is a need to also update
>> Energy Model framework. The EM data contains power values which depend
>> on voltage values. The EM structure is used for thermal (IPA governor)
>> and in scheduler task placement (EAS) so it should reflect the real HW
>> model as best as possible to operate properly.
>>
>> Based on data on Exynos5422 ASV tables the maximum power difference might
>> be ~29%. An Odroid-XU4 (with a random sample SoC in this chip lottery)
>> showed power difference for some OPPs ~20%. Therefore, it's worth to
>> update the EM.
>>
>> Signed-off-by: Lukasz Luba <lukasz.luba@arm.com>
>> ---
>>   drivers/soc/samsung/exynos-asv.c | 11 ++++++++++-
>>   1 file changed, 10 insertions(+), 1 deletion(-)
> 
> I assume there is dependency, even though cover letter did not mention
> it, so:
> 
> Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>

Thanks Krzysztof for the review. Yes, it touches driver in your tree
indeed. Hopefully it can goes smoothly via Rafael's tree with your
tag.

Regards,
Lukasz

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Dietmar Eggemann]

On 22/03/2024 12:08, Lukasz Luba wrote:

[...]

> @@ -97,9 +98,17 @@ static int exynos_asv_update_opps(struct exynos_asv *asv)
>  			last_opp_table = opp_table;
>  
>  			ret = exynos_asv_update_cpu_opps(asv, cpu);
> -			if (ret < 0)
> +			if (!ret) {
> +				/*
> +				 * When the voltage for OPPs successfully
> +				 * changed, update the EM power values to
> +				 * reflect the reality and not use stale data

At this point, can we really say that the voltage has changed?

  exynos_asv_update_cpu_opps()

    ...
    ret = dev_pm_opp_adjust_voltage()
    if (!ret)
      em_dev_update_chip_binning()
    ...

dev_pm_opp_adjust_voltage() also returns 0 when the voltage value stays
the same?

[...]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Lukasz Luba]

Hi Dietmar,

On 3/26/24 11:20, Dietmar Eggemann wrote:
> On 22/03/2024 12:08, Lukasz Luba wrote:
> 
> [...]
> 
>> @@ -97,9 +98,17 @@ static int exynos_asv_update_opps(struct exynos_asv *asv)
>>   			last_opp_table = opp_table;
>>   
>>   			ret = exynos_asv_update_cpu_opps(asv, cpu);
>> -			if (ret < 0)
>> +			if (!ret) {
>> +				/*
>> +				 * When the voltage for OPPs successfully
>> +				 * changed, update the EM power values to
>> +				 * reflect the reality and not use stale data
> 
> At this point, can we really say that the voltage has changed?
> 
>    exynos_asv_update_cpu_opps()
> 
>      ...
>      ret = dev_pm_opp_adjust_voltage()
>      if (!ret)
>        em_dev_update_chip_binning()
>      ...
> 
> dev_pm_opp_adjust_voltage() also returns 0 when the voltage value stays
> the same?
> 
> [...]

The comment for the dev_pm_opp_adjust_voltage() says that it
returns 0 if no modification was done or modification was
successful. So I cannot distinguish in that driver code, but
also there is no additional need to do it IMO (even framework
doesn't do this).

Regards,
Lukasz

[1] 
https://elixir.bootlin.com/linux/v6.9-rc1/source/drivers/opp/core.c#L2950

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Dietmar Eggemann]

On 26/03/2024 21:12, Lukasz Luba wrote:
> Hi Dietmar,
> 
> On 3/26/24 11:20, Dietmar Eggemann wrote:
>> On 22/03/2024 12:08, Lukasz Luba wrote:
>>
>> [...]
>>
>>> @@ -97,9 +98,17 @@ static int exynos_asv_update_opps(struct
>>> exynos_asv *asv)
>>>               last_opp_table = opp_table;
>>>                 ret = exynos_asv_update_cpu_opps(asv, cpu);
>>> -            if (ret < 0)
>>> +            if (!ret) {
>>> +                /*
>>> +                 * When the voltage for OPPs successfully
>>> +                 * changed, update the EM power values to
>>> +                 * reflect the reality and not use stale data
>>
>> At this point, can we really say that the voltage has changed?
>>
>>    exynos_asv_update_cpu_opps()
>>
>>      ...
>>      ret = dev_pm_opp_adjust_voltage()
>>      if (!ret)
>>        em_dev_update_chip_binning()
>>      ...
>>
>> dev_pm_opp_adjust_voltage() also returns 0 when the voltage value stays
>> the same?
>>
>> [...]
> 
> The comment for the dev_pm_opp_adjust_voltage() says that it
> returns 0 if no modification was done or modification was
> successful. So I cannot distinguish in that driver code, but
> also there is no additional need to do it IMO (even framework
> doesn't do this).

Precisely. That's why the added comment in exynos_asv_update_opps():
"When the voltage for OPPs successfully __changed__, ..." is somehow
misleading IMHO.


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RESEND][PATCH v2 4/4] soc: samsung: exynos-asv: Update Energy Model after adjusting voltage

[Lukasz Luba]

On 3/27/24 12:56, Dietmar Eggemann wrote:
> On 26/03/2024 21:12, Lukasz Luba wrote:
>> Hi Dietmar,
>>
>> On 3/26/24 11:20, Dietmar Eggemann wrote:
>>> On 22/03/2024 12:08, Lukasz Luba wrote:
>>>
>>> [...]
>>>
>>>> @@ -97,9 +98,17 @@ static int exynos_asv_update_opps(struct
>>>> exynos_asv *asv)
>>>>                last_opp_table = opp_table;
>>>>                  ret = exynos_asv_update_cpu_opps(asv, cpu);
>>>> -            if (ret < 0)
>>>> +            if (!ret) {
>>>> +                /*
>>>> +                 * When the voltage for OPPs successfully
>>>> +                 * changed, update the EM power values to
>>>> +                 * reflect the reality and not use stale data
>>>
>>> At this point, can we really say that the voltage has changed?
>>>
>>>     exynos_asv_update_cpu_opps()
>>>
>>>       ...
>>>       ret = dev_pm_opp_adjust_voltage()
>>>       if (!ret)
>>>         em_dev_update_chip_binning()
>>>       ...
>>>
>>> dev_pm_opp_adjust_voltage() also returns 0 when the voltage value stays
>>> the same?
>>>
>>> [...]
>>
>> The comment for the dev_pm_opp_adjust_voltage() says that it
>> returns 0 if no modification was done or modification was
>> successful. So I cannot distinguish in that driver code, but
>> also there is no additional need to do it IMO (even framework
>> doesn't do this).
> 
> Precisely. That's why the added comment in exynos_asv_update_opps():
> "When the voltage for OPPs successfully __changed__, ..." is somehow
> misleading IMHO.
> 

I got your point. Let me re-phrase that comment to reflect this
OPP function return properly.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel