Re: DRAM power consumption with turbostat

Julia Lawall
2022-02-06T09:32:17Z

On Sun, 6 Feb 2022, Zhang Rui wrote:

> Hi, Julia,
>
> Thanks for reporting this.
>
> On Sun, 2022-01-30 at 11:27 +0100, Julia Lawall wrote:
> > Hello,
> >
> > I was wondering whether the DRAM power consumption reported by
> > turbostat
> > is reliable on recent Intel machines?
> >
> > In particular, I observed that turbostat reported a high DRAM energy
> > comsumption on a machine (Intel 5128)
>
> are you sure it is 5128? google tells me that 5128 is pretty old, in
> around 2006.
>
> can you please paste the lscpu output?
>
> > with persistent memory, but where
> > the persistent memory was not being used. A colleague did an
> > experiemnt
> > on another machine with persistent memory, and reported:
> >
> > -----
> >
> > I didn't run the test on troll but on another server equipped with PM
> > where I was able to reproduce the bug and by reading directly the msr
> > registers, I see that:
> > CPU Energy units = 0.00006104J
> > DRAM Energy units = 0.00001526J
> >
> > However turbostat assumes that the DRAM Energy units is 0.00006104J
> > when
> > it runs the computation to obtain Joules (hence the too-high value
> > returned by turbostat)
>
> And can you please try latest version of turbostat?
>
> The problem should be fixed by the below commit

The result indeed looks better, thanks!

The following just spins one on core for 5 seconds:

root@troll-4:/home/jlawall/spin# turbostat --version
turbostat version 20.09.30 - Len Brown <lenb@kernel.org>
root@troll-4:/home/jlawall/spin# head troll_turbo
5.011506 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 Pkg_J RAM_J PKG_% RAM_%
- - - 61 1.59 3853 2294 5176 0 37 1493 153 2853 0.00 0.22 0.03 98.14 2.73 95.68 38 40 39.42 0.00 459.78 1336.73 0.00 0.00
0 0 0 4 0.36 1107 2294 298 0 0 0 0 564 0.00 0.00 0.00 99.69 4.86 94.78 36 40 78.85 0.00 228.32 664.98 0.00 0.00
0 0 32 1 0.06 1263 2294 49 0 0 1 14 46 0.00 0.00 0.69 99.26 5.15
0 1 4 0 0.03 1395 2294 29 0 0 8 0 44 0.00 0.59 0.00 99.36 1.03 98.94 34
0 1 36 0 0.02 1245 2294 22 0 0 0 1 25 0.00 0.00 0.02 99.95 1.04
0 2 8 0 0.02 1390 2294 22 0 0 0 0 27 0.00 0.00 0.00 99.97 0.37 99.61 32
0 2 40 0 0.03 1392 2294 24 0 0 0 0 36 0.00 0.00 0.00 99.96 0.36
0 3 12 0 0.02 1348 2294 23 0 0 0 0 26 0.00 0.00 0.00 99.97 0.23 99.76 36

root@troll-4:/home/jlawall/spin# myturbostat --version
turbostat version 21.05.04 - Len Brown <lenb@kernel.org>
root@troll-4:/home/jlawall/spin# head troll_new_turbo
5.014223 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IPC IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 Pkg_J RAM_J PKG_% RAM_%
- - - 61 1.58 3863 2294 1.00 2800 0 0 27 68 1927 0.00 0.02 0.02 98.37 2.14 96.28 37 39 46.29 0.00 457.55 335.42 0.00 0.00
0 0 0 5 0.44 1054 2295 0.63 351 0 0 0 0 668 0.00 0.00 0.00 99.63 4.53 95.04 34 39 92.52 0.00 225.67 166.64 0.00 0.00
0 0 32 1 0.05 1060 2295 0.12 25 0 0 0 1 25 0.00 0.00 0.02 99.97 4.91
0 1 4 0 0.01 1461 2295 0.38 6 0 0 0 1 6 0.00 0.00 0.01 99.97 0.19 99.80 33
0 1 36 0 0.02 1295 2295 0.56 19 0 0 0 2 15 0.00 0.00 0.04 99.94 0.18
0 2 8 0 0.01 1764 2295 0.31 9 0 0 0 1 10 0.00 0.00 0.01 99.98 0.18 99.81 30
0 2 40 0 0.01 1761 2295 0.32 10 0 0 0 2 15 0.00 0.00 0.03 99.95 0.18
0 3 12 0 0.03 1304 2295 0.41 32 0 0 0 21 42 0.00 0.00 0.62 99.35 0.95 99.02 35

It seems that the problem was not limited to machines with persuistent
memory. Here are the same results on a 4-socket 6130:

Old version:
root@yeti-4:/home/jlawall/spin# head yeti_turbo
5.031399 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 PkgWatt RAMWatt PKG_% RAM_%
- - - 29 0.80 3630 2096 4751 0 0 8 56 3585 0.00 0.00 0.01 99.23 1.23 97.97 30 31 72.53 0.00 193.12 111.17 0.00 0.00
0 0 0 4 0.41 1027 2095 290 0 0 0 10 536 0.00 0.00 0.66 98.98 4.19 95.40 28 31 92.84 0.00 44.66 25.49 0.00 0.00
0 0 64 0 0.04 1073 2095 9 0 0 0 0 9 0.00 0.00 0.00 99.98 4.56
0 1 8 0 0.01 1922 2095 9 0 0 0 0 10 0.00 0.00 0.00 99.99 0.23 99.76 29
0 1 72 0 0.02 1380 2095 27 0 0 0 0 29 0.00 0.00 0.00 99.98 0.22
0 2 16 0 0.02 1574 2095 24 0 0 0 0 26 0.00 0.00 0.00 99.98 0.91 99.07 29
0 2 80 0 0.03 1290 2095 55 0 0 8 1 45 0.00 0.58 0.04 99.36 0.90
0 3 24 0 0.02 1460 2095 34 0 0 0 8 30 0.00 0.00 0.59 99.39 0.83 99.15 29

New version:
root@yeti-4:/home/jlawall/spin# head yeti_new_turbo
5.030216 sec
Package Core CPU Avg_MHz Busy% Bzy_MHz TSC_MHz IPC IRQ SMI POLL C1 C1E C6 POLL% C1% C1E% C6% CPU%c1 CPU%c6 CoreTmp PkgTmp Pkg%pc2 Pkg%pc6 PkgWatt RAMWatt PKG_% RAM_%
- - - 51 1.49 3427 2107 1.04 23135 0 8 1524 2664 23370 0.00 0.07 0.22 98.79 5.71 92.80 31 32 59.79 0.00 205.38 30.23 0.00 0.00
0 0 0 7 0.52 1312 2095 0.56 397 0 0 0 3 690 0.00 0.00 0.01 99.53 5.58 93.90 29 32 80.47 0.00 45.69 6.78 0.00 0.00
0 0 64 1 0.04 1278 2095 0.10 9 0 0 0 0 11 0.00 0.00 0.00 99.99 6.06
0 1 8 3 0.12 2865 2095 0.71 44 0 0 0 1 51 0.00 0.00 0.00 99.88 0.64 99.24 30
0 1 72 0 0.01 1835 2095 0.30 18 0 0 0 0 18 0.00 0.00 0.00 99.99 0.75
0 2 16 0 0.02 1537 2095 0.16 8 0 0 0 0 9 0.00 0.00 0.00 99.99 12.87 87.11 29
0 2 80 38 2.08 1847 2095 0.16 8262 0 3 1426 98 6552 0.00 6.45 0.12 91.60 10.81
0 3 24 0 0.01 2043 2095 0.24 9 0 0 0 0 13 0.00 0.00 0.00 99.99 1.41 98.58 30

On the other hand, the energy consumption is still much higher on the
machine with the persistent memory, even though the persistent memory is
not being used. But I guess it is reasonable that it should still consume
something.

thanks,
julia