Re: [PATCH v5 0/9] Add per-core RAPL energy counter support for AMD CPUs

[Dhananjay Ugwekar <Dhananjay.Ugwekar@amd.com>]

On 9/16/2024 12:19 PM, Oleksandr Natalenko wrote:
> Hello.
> 
> On pondělí 16. září 2024 6:18:13, SELČ Dhananjay Ugwekar wrote:
>> Hello Oleksandr,
>>
>> On 9/14/2024 2:48 AM, Oleksandr Natalenko wrote:
>>> Hello.
>>>
>>> On pátek 13. září 2024 17:21:40, SELČ Dhananjay Ugwekar wrote:
>>>> Currently the energy-cores event in the power PMU aggregates energy
>>>> consumption data at a package level. On the other hand the core energy
>>>> RAPL counter in AMD CPUs has a core scope (which means the energy 
>>>> consumption is recorded separately for each core). Earlier efforts to add
>>>> the core event in the power PMU had failed [1], due to the difference in 
>>>> the scope of these two events. Hence, there is a need for a new core scope
>>>> PMU.
>>>>
>>>> This patchset adds a new "power_per_core" PMU alongside the existing
>>>> "power" PMU, which will be responsible for collecting the new
>>>> "energy-per-core" event.
>>>>
>>>> Tested the package level and core level PMU counters with workloads
>>>> pinned to different CPUs.
>>>>
>>>> Results with workload pinned to CPU 1 in Core 1 on an AMD Zen4 Genoa 
>>>> machine:
>>>>
>>>> $ perf stat -a --per-core -e power_per_core/energy-per-core/ -- sleep 1
>>>>
>>>>  Performance counter stats for 'system wide':
>>>>
>>>> S0-D0-C0         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C1         1          5.72 Joules power_per_core/energy-per-core/
>>>> S0-D0-C2         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C3         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C4         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C5         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C6         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C7         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C8         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C9         1          0.02 Joules power_per_core/energy-per-core/
>>>> S0-D0-C10        1          0.02 Joules power_per_core/energy-per-core/
>>>>
>>>> v4 Link: https://lore.kernel.org/all/20240711102436.4432-1-Dhananjay.Ugwekar@amd.com/
>>>>
>>>> v5 changes:
>>>> * Rebase on top of Kan Liang's "PMU scope" patchset [2]
>>>> * rapl_cntr_mask moved to rapl_pmus struct in patch 8
>>>> * Patch 1 from v4 is merged separately, so removed from this series
>>>> * Add an extra argument "scope" in patch 5 to the init functions
>>>> * Add an new patch 2, which removes the cpu_to_rapl_pmu() function
>>>>
>>>> Base: tip/perf/core(currently has just 1-5 patches from [2]) + patch 6 from [2] + 
>>>>       diff [3] + patch 7 from [2] + revert [4] + apply [5] 
>>>>
>>>> [1]: https://lore.kernel.org/lkml/3e766f0e-37d4-0f82-3868-31b14228868d@linux.intel.com/
>>>> [2]: https://lore.kernel.org/all/20240802151643.1691631-1-kan.liang@linux.intel.com/
>>>> [3]: https://lore.kernel.org/all/8c09633c-5bf2-48a2-91a6-a0af9b9f2e8c@linux.intel.com/
>>>> [4]: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=perf/core&id=8d72eba1cf8cecd76a2b4c1dd7673c2dc775f514
>>>> [5]: https://lore.kernel.org/all/20240910085504.204814-1-Dhananjay.Ugwekar@amd.com/
>>>>
>>>> Dhananjay Ugwekar (8):
>>>>   perf/x86/rapl: Remove the cpu_to_rapl_pmu() function
>>>>   perf/x86/rapl: Rename rapl_pmu variables
>>>>   perf/x86/rapl: Make rapl_model struct global
>>>>   perf/x86/rapl: Add arguments to the cleanup and init functions
>>>>   perf/x86/rapl: Modify the generic variable names to *_pkg*
>>>>   perf/x86/rapl: Remove the global variable rapl_msrs
>>>>   perf/x86/rapl: Move the cntr_mask to rapl_pmus struct
>>>>   perf/x86/rapl: Add per-core energy counter support for AMD CPUs
>>>>
>>>> K Prateek Nayak (1):
>>>>   x86/topology: Introduce topology_logical_core_id()
>>>>
>>>>  Documentation/arch/x86/topology.rst   |   4 +
>>>>  arch/x86/events/rapl.c                | 406 ++++++++++++++++----------
>>>>  arch/x86/include/asm/processor.h      |   1 +
>>>>  arch/x86/include/asm/topology.h       |   1 +
>>>>  arch/x86/kernel/cpu/debugfs.c         |   1 +
>>>>  arch/x86/kernel/cpu/topology_common.c |   1 +
>>>>  6 files changed, 266 insertions(+), 148 deletions(-)
>>>
>>> With v6.11-rc7 + all the mentioned preparatory patches and this series:
>>>
>>> $ taskset -c 9 dd if=/dev/zero of=/dev/null &
>>>
>>> $ sudo perf stat -a --per-core -e power_per_core/energy-per-core/ sleep 5
>>>
>>>  Performance counter stats for 'system wide':
>>>
>>> S0-D0-C0              1               3,79 Joules power_per_core/energy-per-core/
>>> S0-D0-C1              1               5,65 Joules power_per_core/energy-per-core/
>>> S0-D0-C2              1               1,26 Joules power_per_core/energy-per-core/
>>> S0-D0-C3              1               3,18 Joules power_per_core/energy-per-core/
>>> S0-D0-C4              1               2,06 Joules power_per_core/energy-per-core/
>>> S0-D0-C5              1               3,51 Joules power_per_core/energy-per-core/
>>> S0-D0-C6              1               0,77 Joules power_per_core/energy-per-core/
>>> S0-D0-C7              1               0,55 Joules power_per_core/energy-per-core/
>>> S0-D0-C8              1               1,65 Joules power_per_core/energy-per-core/
>>> S0-D0-C9              1              47,85 Joules power_per_core/energy-per-core/
>>> S0-D0-C10             1               2,49 Joules power_per_core/energy-per-core/
>>> S0-D0-C11             1              11,85 Joules power_per_core/energy-per-core/
>>> S0-D0-C12             1               1,75 Joules power_per_core/energy-per-core/
>>> S0-D0-C13             1               0,74 Joules power_per_core/energy-per-core/
>>> S0-D0-C14             1               2,58 Joules power_per_core/energy-per-core/
>>> S0-D0-C15             1               4,67 Joules power_per_core/energy-per-core/
>>>
>>>        5,003391425 seconds time elapsed
>>>
>>> on the following CPU:
>>>
>>> AMD Ryzen 9 5950X 16-Core Processor
>>>
>>> If this behaves as expected, please add:
>>>
>>> Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
>>
>> Energy reported by core 9 is considerably higher than other cores, which is as expected.
>>
>> However, can you please post the core_id for CPU 9 just to be sure,
> 
> $ cat /sys/devices/system/cpu/cpu9/topology/core_id
> 9
> 
>> also I see that other 
>> cores are also showing considerable energy consumption(e.g. core 11), are there some other 
>> tasks running in the background?
> 
> Yes, this is a fully-featured desktop environment. Another run:
> 
> S0-D0-C0              1               7,68 Joules power_per_core/energy-per-core/
> S0-D0-C1              1               2,05 Joules power_per_core/energy-per-core/
> S0-D0-C2              1               0,96 Joules power_per_core/energy-per-core/
> S0-D0-C3              1               5,16 Joules power_per_core/energy-per-core/
> S0-D0-C4              1               1,11 Joules power_per_core/energy-per-core/
> S0-D0-C5              1               2,42 Joules power_per_core/energy-per-core/
> S0-D0-C6              1               1,11 Joules power_per_core/energy-per-core/
> S0-D0-C7              1               0,51 Joules power_per_core/energy-per-core/
> S0-D0-C8              1               1,61 Joules power_per_core/energy-per-core/
> S0-D0-C9              1              47,07 Joules power_per_core/energy-per-core/
> S0-D0-C10             1               1,59 Joules power_per_core/energy-per-core/
> S0-D0-C11             1               3,02 Joules power_per_core/energy-per-core/
> S0-D0-C12             1               1,43 Joules power_per_core/energy-per-core/
> S0-D0-C13             1               0,84 Joules power_per_core/energy-per-core/
> S0-D0-C14             1               0,73 Joules power_per_core/energy-per-core/
> S0-D0-C15             1               0,78 Joules power_per_core/energy-per-core/

Great!, the results are as expected, will add your tested-by,

Thanks,
Dhananjay

> 
> Thank you.
> 
>>
>> Thanks for testing!
>>
>> Regards,
>> Dhananjay
>>
>>>
>>> Thank you.
>>>
>>
> 
>