* DRAM power consumption with turbostat
@ 2022-01-30 10:27 Julia Lawall
2022-02-06 8:29 ` Zhang Rui
0 siblings, 1 reply; 4+ messages in thread
From: Julia Lawall @ 2022-01-30 10:27 UTC (permalink / raw)
To: Len Brown; +Cc: linux-pm
Hello,
I was wondering whether the DRAM power consumption reported by turbostat
is reliable on recent Intel machines?
In particular, I observed that turbostat reported a high DRAM energy
comsumption on a machine (Intel 5128) with persistent memory, but where
the persistent memory was not being used. A colleague did an experiemnt
on another machine with persistent memory, and reported:
-----
I didn't run the test on troll but on another server equipped with PM
where I was able to reproduce the bug and by reading directly the msr
registers, I see that:
CPU Energy units = 0.00006104J
DRAM Energy units = 0.00001526J
However turbostat assumes that the DRAM Energy units is 0.00006104J when
it runs the computation to obtain Joules (hence the too-high value
returned by turbostat)
-----
I see the code in turbostat that just uses the CPU energy units value
(rapl_dram_energy_units_probe), but I don't know what was the MSR used to
collect the above information. Overall, I am wondering if the DRAM energy
consumption values are reliable in cases with and without persistent
memory.
thanks,
julia
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: DRAM power consumption with turbostat
2022-01-30 10:27 DRAM power consumption with turbostat Julia Lawall
@ 2022-02-06 8:29 ` Zhang Rui
2022-02-06 8:34 ` Julia Lawall
2022-02-06 9:32 ` Julia Lawall
0 siblings, 2 replies; 4+ messages in thread
From: Zhang Rui @ 2022-02-06 8:29 UTC (permalink / raw)
To: Julia Lawall, Len Brown; +Cc: linux-pm
Hi, Julia,
Thanks for reporting this.
On Sun, 2022-01-30 at 11:27 +0100, Julia Lawall wrote:
> Hello,
>
> I was wondering whether the DRAM power consumption reported by
> turbostat
> is reliable on recent Intel machines?
>
> In particular, I observed that turbostat reported a high DRAM energy
> comsumption on a machine (Intel 5128)
are you sure it is 5128? google tells me that 5128 is pretty old, in
around 2006.
can you please paste the lscpu output?
> with persistent memory, but where
> the persistent memory was not being used. A colleague did an
> experiemnt
> on another machine with persistent memory, and reported:
>
> -----
>
> I didn't run the test on troll but on another server equipped with PM
> where I was able to reproduce the bug and by reading directly the msr
> registers, I see that:
> CPU Energy units = 0.00006104J
> DRAM Energy units = 0.00001526J
>
> However turbostat assumes that the DRAM Energy units is 0.00006104J
> when
> it runs the computation to obtain Joules (hence the too-high value
> returned by turbostat)
And can you please try latest version of turbostat?
The problem should be fixed by the below commit
commit abdc75ab53b7fd2ef42c79e88cf0caf2d007c4f2
Author: Zhang Rui <rui.zhang@intel.com>
AuthorDate: Thu Mar 11 10:05:13 2021 +0800
Commit: Len Brown <len.brown@intel.com>
CommitDate: Tue May 4 18:23:14 2021 -0400
tools/power turbostat: Fix DRAM Energy Unit on SKX
SKX uses fixed DRAM Energy Unit, just like HSX and BDX.
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
thanks,
rui
>
> -----
>
> I see the code in turbostat that just uses the CPU energy units value
> (rapl_dram_energy_units_probe), but I don't know what was the MSR
> used to
> collect the above information. Overall, I am wondering if the DRAM
> energy
> consumption values are reliable in cases with and without persistent
> memory.
>
> thanks,
> julia
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: DRAM power consumption with turbostat
2022-02-06 8:29 ` Zhang Rui
@ 2022-02-06 8:34 ` Julia Lawall
2022-02-06 9:32 ` Julia Lawall
1 sibling, 0 replies; 4+ messages in thread
From: Julia Lawall @ 2022-02-06 8:34 UTC (permalink / raw)
To: Zhang Rui; +Cc: Len Brown, linux-pm
On Sun, 6 Feb 2022, Zhang Rui wrote:
> Hi, Julia,
>
> Thanks for reporting this.
>
> On Sun, 2022-01-30 at 11:27 +0100, Julia Lawall wrote:
> > Hello,
> >
> > I was wondering whether the DRAM power consumption reported by
> > turbostat
> > is reliable on recent Intel machines?
> >
> > In particular, I observed that turbostat reported a high DRAM energy
> > comsumption on a machine (Intel 5128)
>
> are you sure it is 5128? google tells me that 5128 is pretty old, in
> around 2006.
Oops! 5218
Thanks for the pointer to the new version. I will try that.
julia
>
> can you please paste the lscpu output?
>
> > with persistent memory, but where
> > the persistent memory was not being used. A colleague did an
> > experiemnt
> > on another machine with persistent memory, and reported:
> >
> > -----
> >
> > I didn't run the test on troll but on another server equipped with PM
> > where I was able to reproduce the bug and by reading directly the msr
> > registers, I see that:
> > CPU Energy units = 0.00006104J
> > DRAM Energy units = 0.00001526J
> >
> > However turbostat assumes that the DRAM Energy units is 0.00006104J
> > when
> > it runs the computation to obtain Joules (hence the too-high value
> > returned by turbostat)
>
> And can you please try latest version of turbostat?
>
> The problem should be fixed by the below commit
>
> commit abdc75ab53b7fd2ef42c79e88cf0caf2d007c4f2
> Author: Zhang Rui <rui.zhang@intel.com>
> AuthorDate: Thu Mar 11 10:05:13 2021 +0800
> Commit: Len Brown <len.brown@intel.com>
> CommitDate: Tue May 4 18:23:14 2021 -0400
>
> tools/power turbostat: Fix DRAM Energy Unit on SKX
>
> SKX uses fixed DRAM Energy Unit, just like HSX and BDX.
>
> Signed-off-by: Zhang Rui <rui.zhang@intel.com>
> Signed-off-by: Len Brown <len.brown@intel.com>
>
> thanks,
> rui
> >
> > -----
> >
> > I see the code in turbostat that just uses the CPU energy units value
> > (rapl_dram_energy_units_probe), but I don't know what was the MSR
> > used to
> > collect the above information. Overall, I am wondering if the DRAM
> > energy
> > consumption values are reliable in cases with and without persistent
> > memory.
> >
> > thanks,
> > julia
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: DRAM power consumption with turbostat
2022-02-06 8:29 ` Zhang Rui
2022-02-06 8:34 ` Julia Lawall
@ 2022-02-06 9:32 ` Julia Lawall
1 sibling, 0 replies; 4+ messages in thread
From: Julia Lawall @ 2022-02-06 9:32 UTC (permalink / raw)
To: Zhang Rui; +Cc: Len Brown, linux-pm
On Sun, 6 Feb 2022, Zhang Rui wrote:
> Hi, Julia,
>
> Thanks for reporting this.
>
> On Sun, 2022-01-30 at 11:27 +0100, Julia Lawall wrote:
> > Hello,
> >
> > I was wondering whether the DRAM power consumption reported by
> > turbostat
> > is reliable on recent Intel machines?
> >
> > In particular, I observed that turbostat reported a high DRAM energy
> > comsumption on a machine (Intel 5128)
>
> are you sure it is 5128? google tells me that 5128 is pretty old, in
> around 2006.
>
> can you please paste the lscpu output?
>
> > with persistent memory, but where
> > the persistent memory was not being used. A colleague did an
> > experiemnt
> > on another machine with persistent memory, and reported:
> >
> > -----
> >
> > I didn't run the test on troll but on another server equipped with PM
> > where I was able to reproduce the bug and by reading directly the msr
> > registers, I see that:
> > CPU Energy units = 0.00006104J
> > DRAM Energy units = 0.00001526J
> >
> > However turbostat assumes that the DRAM Energy units is 0.00006104J
> > when
> > it runs the computation to obtain Joules (hence the too-high value
> > returned by turbostat)
>
> And can you please try latest version of turbostat?
>
> The problem should be fixed by the below commit
The result indeed looks better, thanks!
The following just spins one on core for 5 seconds:
root@troll-4:/home/jlawall/spin# turbostat --version
turbostat version 20.09.30 - Len Brown <lenb@kernel.org>
root@troll-4:/home/jlawall/spin# head troll_turbo
5.011506 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 Pkg_J RAM_J PKG_% RAM_%
- - - 61 1.59 3853 2294 5176 0 37 1493 153 2853 0.00 0.22 0.03 98.14 2.73 95.68 38 40 39.42 0.00 459.78 1336.73 0.00 0.00
0 0 0 4 0.36 1107 2294 298 0 0 0 0 564 0.00 0.00 0.00 99.69 4.86 94.78 36 40 78.85 0.00 228.32 664.98 0.00 0.00
0 0 32 1 0.06 1263 2294 49 0 0 1 14 46 0.00 0.00 0.69 99.26 5.15
0 1 4 0 0.03 1395 2294 29 0 0 8 0 44 0.00 0.59 0.00 99.36 1.03 98.94 34
0 1 36 0 0.02 1245 2294 22 0 0 0 1 25 0.00 0.00 0.02 99.95 1.04
0 2 8 0 0.02 1390 2294 22 0 0 0 0 27 0.00 0.00 0.00 99.97 0.37 99.61 32
0 2 40 0 0.03 1392 2294 24 0 0 0 0 36 0.00 0.00 0.00 99.96 0.36
0 3 12 0 0.02 1348 2294 23 0 0 0 0 26 0.00 0.00 0.00 99.97 0.23 99.76 36
root@troll-4:/home/jlawall/spin# myturbostat --version
turbostat version 21.05.04 - Len Brown <lenb@kernel.org>
root@troll-4:/home/jlawall/spin# head troll_new_turbo
5.014223 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IPC IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 Pkg_J RAM_J PKG_% RAM_%
- - - 61 1.58 3863 2294 1.00 2800 0 0 27 68 1927 0.00 0.02 0.02 98.37 2.14 96.28 37 39 46.29 0.00 457.55 335.42 0.00 0.00
0 0 0 5 0.44 1054 2295 0.63 351 0 0 0 0 668 0.00 0.00 0.00 99.63 4.53 95.04 34 39 92.52 0.00 225.67 166.64 0.00 0.00
0 0 32 1 0.05 1060 2295 0.12 25 0 0 0 1 25 0.00 0.00 0.02 99.97 4.91
0 1 4 0 0.01 1461 2295 0.38 6 0 0 0 1 6 0.00 0.00 0.01 99.97 0.19 99.80 33
0 1 36 0 0.02 1295 2295 0.56 19 0 0 0 2 15 0.00 0.00 0.04 99.94 0.18
0 2 8 0 0.01 1764 2295 0.31 9 0 0 0 1 10 0.00 0.00 0.01 99.98 0.18 99.81 30
0 2 40 0 0.01 1761 2295 0.32 10 0 0 0 2 15 0.00 0.00 0.03 99.95 0.18
0 3 12 0 0.03 1304 2295 0.41 32 0 0 0 21 42 0.00 0.00 0.62 99.35 0.95 99.02 35
It seems that the problem was not limited to machines with persuistent
memory. Here are the same results on a 4-socket 6130:
Old version:
root@yeti-4:/home/jlawall/spin# head yeti_turbo
5.031399 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 PkgWatt RAMWatt PKG_% RAM_%
- - - 29 0.80 3630 2096 4751 0 0 8 56 3585 0.00 0.00 0.01 99.23 1.23 97.97 30 31 72.53 0.00 193.12 111.17 0.00 0.00
0 0 0 4 0.41 1027 2095 290 0 0 0 10 536 0.00 0.00 0.66 98.98 4.19 95.40 28 31 92.84 0.00 44.66 25.49 0.00 0.00
0 0 64 0 0.04 1073 2095 9 0 0 0 0 9 0.00 0.00 0.00 99.98 4.56
0 1 8 0 0.01 1922 2095 9 0 0 0 0 10 0.00 0.00 0.00 99.99 0.23 99.76 29
0 1 72 0 0.02 1380 2095 27 0 0 0 0 29 0.00 0.00 0.00 99.98 0.22
0 2 16 0 0.02 1574 2095 24 0 0 0 0 26 0.00 0.00 0.00 99.98 0.91 99.07 29
0 2 80 0 0.03 1290 2095 55 0 0 8 1 45 0.00 0.58 0.04 99.36 0.90
0 3 24 0 0.02 1460 2095 34 0 0 0 8 30 0.00 0.00 0.59 99.39 0.83 99.15 29
New version:
root@yeti-4:/home/jlawall/spin# head yeti_new_turbo
5.030216 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IPC IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 PkgWatt RAMWatt PKG_% RAM_%
- - - 51 1.49 3427 2107 1.04 23135 0 8 1524 2664 23370 0.00 0.07 0.22 98.79 5.71 92.80 31 32 59.79 0.00 205.38 30.23 0.00 0.00
0 0 0 7 0.52 1312 2095 0.56 397 0 0 0 3 690 0.00 0.00 0.01 99.53 5.58 93.90 29 32 80.47 0.00 45.69 6.78 0.00 0.00
0 0 64 1 0.04 1278 2095 0.10 9 0 0 0 0 11 0.00 0.00 0.00 99.99 6.06
0 1 8 3 0.12 2865 2095 0.71 44 0 0 0 1 51 0.00 0.00 0.00 99.88 0.64 99.24 30
0 1 72 0 0.01 1835 2095 0.30 18 0 0 0 0 18 0.00 0.00 0.00 99.99 0.75
0 2 16 0 0.02 1537 2095 0.16 8 0 0 0 0 9 0.00 0.00 0.00 99.99 12.87 87.11 29
0 2 80 38 2.08 1847 2095 0.16 8262 0 3 1426 98 6552 0.00 6.45 0.12 91.60 10.81
0 3 24 0 0.01 2043 2095 0.24 9 0 0 0 0 13 0.00 0.00 0.00 99.99 1.41 98.58 30
On the other hand, the energy consumption is still much higher on the
machine with the persistent memory, even though the persistent memory is
not being used. But I guess it is reasonable that it should still consume
something.
thanks,
julia
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-02-06 9:32 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-01-30 10:27 DRAM power consumption with turbostat Julia Lawall
2022-02-06 8:29 ` Zhang Rui
2022-02-06 8:34 ` Julia Lawall
2022-02-06 9:32 ` Julia Lawall
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox