All of lore.kernel.org
 help / color / mirror / Atom feed
From: Huang, Ying <ying.huang@intel.com>
To: lkp@lists.01.org
Subject: Re: [swap] 9c9c831c31: pmbench.latency.ns.average -33.9% improvement
Date: Wed, 27 May 2020 16:44:16 +0800	[thread overview]
Message-ID: <87pnaphin3.fsf@yhuang-dev.intel.com> (raw)
In-Reply-To: <20200527053346.GT12456@shao2-debian>

[-- Attachment #1: Type: text/plain, Size: 96295 bytes --]

kernel test robot <rong.a.chen@intel.com> writes:

> Greeting,
>
> FYI, we noticed a -33.9% improvement of pmbench.latency.ns.average due to commit:

Hi, Rong,

Usually, throughput is more important to report than latency.  There's
the throughput different in the comparison result below.

>     877972 ±  4%     +81.8%    1595757 ± 27%  pmbench.throughput.aps

Please report this instead in the future.

Best Regards,
Huang, Ying

> commit: 9c9c831c31a9315365df978af99f7ff315bbf7a7 ("swap: try to scan more free slots even when fragmented")
> https://github.com/hnaz/linux-mm master
>
> in testcase: pmbench
> on test machine: 96 threads Intel(R) Xeon(R) CPU @ 2.30GHz with 128G memory
> with following parameters:
>
> 	runtime: 1800s
> 	mode: raw
> 	nr_dev: 2
> 	nr_part: 2
> 	part_type: S
> 	all: 1
> 	thp_enabled: never
> 	thp_defrag: never
> 	nr_processes: 16
> 	nr_threads: 1
> 	pattern: uniform
> 	ratio: 80
> 	cold: 1
> 	initialize: 1
> 	setsize: 4800M
> 	cpu_node_bind: even
> 	cpufreq_governor: performance
> 	debug-setup: yhss9
> 	sc_numa_balancing: 0
> 	sc_swappiness: 100
> 	ucode: 0x400002c
>
> test-description: pmbench - paging and virtual memory benchmark
>
>
>
>
>
> Details are as below:
> -------------------------------------------------------------------------------------------------->
>
>
> To reproduce:
>
>         git clone https://github.com/intel/lkp-tests.git
>         cd lkp-tests
>         bin/lkp install job.yaml  # job file is attached in this email
>         bin/lkp run     job.yaml
>
> =========================================================================================
> all/cold/compiler/cpu_node_bind/cpufreq_governor/debug-setup/initialize/kconfig/mode/nr_dev/nr_part/nr_processes/nr_threads/part_type/pattern/ratio/rootfs/runtime/sc_numa_balancing/sc_swappiness/setsize/tbox_group/testcase/thp_defrag/thp_enabled/ucode:
>   1/1/gcc-7/even/performance/yhss9/1/x86_64-rhel-7.6/raw/2/2/16/1/S/uniform/80/debian-x86_64-20191114.cgz/1800s/0/100/4800M/lkp-csl-2sp4/pmbench/never/never/0x400002c
>
> commit: 
>   b80829e0bc ("mm/swapfile.c: omit a duplicate code by compare tmp and max first")
>   9c9c831c31 ("swap: try to scan more free slots even when fragmented")
>
> b80829e0bc2d6fed 9c9c831c31a9315365df978af99 
> ---------------- --------------------------- 
>        fail:runs  %reproduction    fail:runs
>            |             |             |    
>           1:4          -25%            :4     kmsg.Dev_pmem#:unable_to_read_RDB_block
>           0:4            9%           1:4     perf-profile.children.cycles-pp.error_entry
>          %stddev     %change         %stddev
>              \          |                \  
>      10.65            -1.8%      10.46        pmbench.init_time.avg
>      18758 ±  2%     -33.9%      12408 ±  4%  pmbench.latency.ns.average
>       0.03 ± 19%      +0.3        0.31 ± 21%  pmbench.read.latency.ns.128K-256K%
>       0.00 ± 46%      -0.0        0.00 ± 82%  pmbench.read.latency.ns.16M-32M%
>       0.00 ± 17%      -0.0        0.00 ± 21%  pmbench.read.latency.ns.1G-inf%
>       9.00 ±  7%      -1.4        7.60 ± 14%  pmbench.read.latency.ns.256-512%
>       0.00 ± 10%      +0.0        0.03 ± 22%  pmbench.read.latency.ns.256K-512K%
>       0.00 ± 44%      -0.0        0.00 ± 43%  pmbench.read.latency.ns.256M-512M%
>      26.10 ±  4%      -4.3       21.76 ± 20%  pmbench.read.latency.ns.2K-4K%
>       0.27 ±  2%      -0.2        0.02 ± 22%  pmbench.read.latency.ns.2M-4M%
>       0.75 ±  4%      -0.5        0.22 ± 21%  pmbench.read.latency.ns.32K-64K%
>      27.69 ±  3%      -6.0       21.72 ± 22%  pmbench.read.latency.ns.4K-8K%
>       0.04 ± 28%      -0.0        0.00 ± 23%  pmbench.read.latency.ns.4M-8M%
>       0.03 ± 24%      +0.1        0.17 ± 19%  pmbench.read.latency.ns.512K-1M%
>       0.00 ± 44%      -0.0        0.00 ± 61%  pmbench.read.latency.ns.8M-16M%
>     877972 ±  4%     +81.8%    1595757 ± 27%  pmbench.throughput.aps
>  9.738e+08           +44.1%  1.404e+09        pmbench.time.major_page_faults
>  1.045e+08           +36.2%  1.423e+08        pmbench.time.minor_page_faults
>       1548            -3.5%       1494 ±  3%  pmbench.time.percent_of_cpu_this_job_got
>      27681            -1.9%      27146        pmbench.time.system_time
>     575.21 ±  2%     +55.8%     896.14 ± 13%  pmbench.time.user_time
>    1458512 ±  3%     -21.1%    1150258 ±  4%  pmbench.time.voluntary_context_switches
>       0.03 ± 19%      +0.3        0.31 ± 21%  pmbench.write.latency.ns.128K-256K%
>       0.00 ± 47%      -0.0        0.00 ± 77%  pmbench.write.latency.ns.16M-32M%
>      18.27 ±  2%      -3.9       14.38 ± 23%  pmbench.write.latency.ns.1K-2K%
>       3.21 ±  4%      -0.5        2.66 ± 20%  pmbench.write.latency.ns.256-512%
>       0.00 ± 10%      +0.0        0.03 ± 22%  pmbench.write.latency.ns.256K-512K%
>       0.00 ± 46%      -0.0        0.00 ± 36%  pmbench.write.latency.ns.256M-512M%
>      25.94 ±  4%      -4.4       21.55 ± 20%  pmbench.write.latency.ns.2K-4K%
>       0.27 ±  3%      -0.2        0.02 ± 22%  pmbench.write.latency.ns.2M-4M%
>       0.76 ±  4%      -0.5        0.22 ± 21%  pmbench.write.latency.ns.32K-64K%
>      28.27 ±  3%      -6.1       22.16 ± 22%  pmbench.write.latency.ns.4K-8K%
>       0.04 ± 28%      -0.0        0.00 ± 23%  pmbench.write.latency.ns.4M-8M%
>       0.03 ± 23%      +0.1        0.17 ± 19%  pmbench.write.latency.ns.512K-1M%
>       0.00 ± 43%      -0.0        0.00 ± 61%  pmbench.write.latency.ns.8M-16M%
>   38900508 ±165%     -97.0%    1161040 ±  3%  cpuidle.C1.usage
>      95548 ±  2%     +15.0%     109893        meminfo.KReclaimable
>      95548 ±  2%     +15.0%     109893        meminfo.SReclaimable
>      22154 ± 13%     -99.5%     117.75 ±  2%  meminfo.SwapCached
>    2133104           +40.2%    2991446 ±  3%  vmstat.swap.si
>    2159159           +39.6%    3015086 ±  3%  vmstat.swap.so
>     403272           +15.6%     466175        vmstat.system.in
>      51216 ±  4%     +16.9%      59856 ±  3%  numa-meminfo.node0.KReclaimable
>     272326 ± 22%    +130.4%     627386 ± 53%  numa-meminfo.node0.MemFree
>      51216 ±  4%     +16.9%      59856 ±  3%  numa-meminfo.node0.SReclaimable
>      44329 ±  5%     +12.9%      50031 ±  4%  numa-meminfo.node1.KReclaimable
>      44329 ±  5%     +12.9%      50031 ±  4%  numa-meminfo.node1.SReclaimable
>       0.00 ± 45%      +0.0        0.01 ± 15%  mpstat.node.0.soft%
>       0.38 ±  3%      +0.2        0.56 ±  8%  mpstat.node.0.usr%
>       0.00 ± 58%      +0.0        0.01 ± 29%  mpstat.node.1.soft%
>       0.30 ±  2%      +0.2        0.47 ± 11%  mpstat.node.1.usr%
>       0.00 ± 51%      +0.0        0.01 ± 22%  mpstat.node.all.soft%
>       0.34 ±  2%      +0.2        0.51 ±  9%  mpstat.node.all.usr%
>       2836 ±  3%     -17.6%       2336 ±  7%  slabinfo.fsnotify_mark_connector.active_objs
>       2836 ±  3%     -17.6%       2336 ±  7%  slabinfo.fsnotify_mark_connector.num_objs
>      56887 ±  2%     +43.0%      81372        slabinfo.radix_tree_node.active_objs
>       1090 ±  3%     +36.1%       1483 ±  2%  slabinfo.radix_tree_node.active_slabs
>      61051 ±  3%     +35.7%      82864        slabinfo.radix_tree_node.num_objs
>       1090 ±  3%     +36.1%       1483 ±  2%  slabinfo.radix_tree_node.num_slabs
>  2.774e+08           +88.1%  5.217e+08 ±  3%  numa-numastat.node0.local_node
>  2.774e+08           +88.1%  5.217e+08 ±  3%  numa-numastat.node0.numa_hit
>   1.77e+08 ± 11%     -34.4%  1.162e+08 ± 12%  numa-numastat.node0.numa_miss
>   1.77e+08 ± 11%     -34.4%  1.162e+08 ± 12%  numa-numastat.node0.other_node
>  2.913e+08 ± 10%     +93.8%  5.646e+08        numa-numastat.node1.local_node
>   1.77e+08 ± 11%     -34.4%  1.162e+08 ± 12%  numa-numastat.node1.numa_foreign
>  2.913e+08 ± 10%     +93.8%  5.646e+08        numa-numastat.node1.numa_hit
>      34077 ± 19%     +88.3%      64163 ± 14%  sched_debug.cfs_rq:/.exec_clock.min
>     420702 ± 19%     +47.6%     620876 ± 17%  sched_debug.cfs_rq:/.min_vruntime.min
>    1141699 ±  5%     -11.1%    1014934 ±  8%  sched_debug.cfs_rq:/.min_vruntime.stddev
>    1141714 ±  5%     -11.1%    1014949 ±  8%  sched_debug.cfs_rq:/.spread0.stddev
>       9.40 ±  8%     -46.6%       5.02        sched_debug.cpu.clock.stddev
>       9.40 ±  8%     -46.6%       5.02        sched_debug.cpu.clock_task.stddev
>      53350 ± 30%     +52.0%      81088 ± 23%  sched_debug.cpu.nr_switches.min
>      14.62 ±  9%     -10.9%      13.02 ±  7%  sched_debug.cpu.nr_uninterruptible.stddev
>      51798 ± 32%     +54.2%      79853 ± 23%  sched_debug.cpu.sched_count.min
>      16360 ±  3%     -11.4%      14489 ±  2%  sched_debug.cpu.sched_goidle.avg
>      35124 ± 15%     -28.5%      25097 ±  4%  sched_debug.cpu.sched_goidle.max
>       5864 ±  7%     -39.1%       3574 ±  8%  sched_debug.cpu.sched_goidle.stddev
>      24835 ±  4%    +106.2%      51199 ±  4%  sched_debug.cpu.ttwu_local.avg
>      58889 ± 16%     +73.3%     102071 ±  5%  sched_debug.cpu.ttwu_local.max
>       6358 ± 21%    +134.0%      14880 ± 22%  sched_debug.cpu.ttwu_local.min
>      10665 ±  8%     +89.0%      20160 ±  7%  sched_debug.cpu.ttwu_local.stddev
>      67428 ± 23%    +131.8%     156312 ± 54%  numa-vmstat.node0.nr_free_pages
>     343073 ±  6%      -9.3%     311011 ±  5%  numa-vmstat.node0.nr_inactive_anon
>     175.75 ±  7%     -26.0%     130.00 ±  6%  numa-vmstat.node0.nr_isolated_anon
>      12806 ±  4%     +16.8%      14959 ±  3%  numa-vmstat.node0.nr_slab_reclaimable
>  2.228e+08 ±  5%     +45.6%  3.244e+08 ±  5%  numa-vmstat.node0.nr_vmscan_write
>  2.228e+08 ±  5%     +45.6%  3.244e+08 ±  5%  numa-vmstat.node0.nr_written
>     342900 ±  6%      -9.3%     310899 ±  5%  numa-vmstat.node0.nr_zone_inactive_anon
>  1.413e+08           +92.9%  2.726e+08 ±  5%  numa-vmstat.node0.numa_hit
>  1.412e+08           +93.0%  2.724e+08 ±  5%  numa-vmstat.node0.numa_local
>   87382887 ± 12%     -30.4%   60836611 ± 10%  numa-vmstat.node0.numa_miss
>   87569634 ± 12%     -30.3%   61038664 ± 10%  numa-vmstat.node0.numa_other
>     224.25 ±  6%     -21.0%     177.25 ±  4%  numa-vmstat.node1.nr_isolated_anon
>      11084 ±  5%     +12.4%      12462 ±  4%  numa-vmstat.node1.nr_slab_reclaimable
>  2.696e+08 ±  5%     +52.1%  4.101e+08 ±  4%  numa-vmstat.node1.nr_vmscan_write
>  2.696e+08 ±  5%     +52.1%  4.101e+08 ±  4%  numa-vmstat.node1.nr_written
>   87384914 ± 12%     -30.4%   60836973 ± 10%  numa-vmstat.node1.numa_foreign
>  1.484e+08 ±  9%     +98.4%  2.944e+08 ±  4%  numa-vmstat.node1.numa_hit
>  1.484e+08 ±  9%     +98.4%  2.943e+08 ±  4%  numa-vmstat.node1.numa_local
>    9317863 ±  7%     +52.6%   14215591        proc-vmstat.allocstall_movable
>     302829 ± 56%     -62.6%     113363 ±  9%  proc-vmstat.allocstall_normal
>     341221            -1.4%     336578        proc-vmstat.nr_file_pages
>     399.75           -23.1%     307.25 ±  2%  proc-vmstat.nr_isolated_anon
>      23880 ±  2%     +14.6%      27371        proc-vmstat.nr_slab_reclaimable
>  4.946e+08 ±  2%     +48.5%  7.346e+08 ±  3%  proc-vmstat.nr_vmscan_write
>  9.858e+08           +43.5%  1.415e+09        proc-vmstat.nr_written
>  4.295e+08 ±  3%     -18.3%  3.508e+08 ±  2%  proc-vmstat.numa_foreign
>  5.688e+08 ±  5%     +91.0%  1.086e+09        proc-vmstat.numa_hit
>  5.687e+08 ±  5%     +91.0%  1.086e+09        proc-vmstat.numa_local
>  4.295e+08 ±  3%     -18.3%  3.508e+08 ±  2%  proc-vmstat.numa_miss
>  4.295e+08 ±  3%     -18.3%  3.508e+08 ±  2%  proc-vmstat.numa_other
>  1.003e+09           +43.7%  1.441e+09        proc-vmstat.pgactivate
>   52861209 ±  5%     +65.9%   87707974 ±  4%  proc-vmstat.pgalloc_dma32
>  9.533e+08           +48.6%  1.416e+09        proc-vmstat.pgalloc_normal
>  1.016e+09           +43.1%  1.454e+09        proc-vmstat.pgdeactivate
>  1.083e+09           +43.3%  1.551e+09        proc-vmstat.pgfault
>  1.006e+09           +49.5%  1.504e+09        proc-vmstat.pgfree
>  9.739e+08           +44.2%  1.404e+09        proc-vmstat.pgmajfault
>  1.016e+09           +43.1%  1.454e+09        proc-vmstat.pgrefill
>  1.711e+09 ±  2%     +32.0%  2.259e+09        proc-vmstat.pgscan_direct
>  4.813e+08 ±  3%     +65.7%  7.977e+08 ±  3%  proc-vmstat.pgscan_kswapd
>  8.277e+08           +37.5%  1.138e+09        proc-vmstat.pgsteal_direct
>  1.581e+08           +75.2%  2.771e+08        proc-vmstat.pgsteal_kswapd
>  9.739e+08           +44.2%  1.404e+09        proc-vmstat.pswpin
>  9.858e+08           +43.5%  1.415e+09        proc-vmstat.pswpout
>     292.25 ± 20%   +1416.4%       4431 ±  8%  proc-vmstat.swap_ra
>      59.00 ±  7%   +4333.1%       2615 ±  9%  proc-vmstat.swap_ra_hit
>      28.93 ±  2%     -26.6%      21.24        perf-stat.i.MPKI
>  2.895e+09           +27.5%  3.692e+09 ±  3%  perf-stat.i.branch-instructions
>       0.87            -0.2        0.63 ±  2%  perf-stat.i.branch-miss-rate%
>   25073529            -9.9%   22581211 ±  3%  perf-stat.i.branch-misses
>      37.59           +10.8       48.39        perf-stat.i.cache-miss-rate%
>  1.615e+08 ±  2%     +24.9%  2.016e+08 ±  3%  perf-stat.i.cache-misses
>  4.297e+08            -3.6%  4.143e+08 ±  3%  perf-stat.i.cache-references
>       4.04           -25.9%       2.99        perf-stat.i.cpi
>  5.982e+10            -4.4%  5.722e+10 ±  3%  perf-stat.i.cpu-cycles
>     114.58            +9.5%     125.49        perf-stat.i.cpu-migrations
>     377.29 ±  3%     -20.4%     300.26 ±  4%  perf-stat.i.cycles-between-cache-misses
>       0.18 ±  4%      +0.0        0.23 ± 18%  perf-stat.i.dTLB-load-miss-rate%
>    6826244 ±  3%     +42.0%    9690079 ±  2%  perf-stat.i.dTLB-load-misses
>  3.818e+09           +30.8%  4.993e+09 ±  3%  perf-stat.i.dTLB-loads
>       0.07 ±  2%      +0.0        0.09 ± 23%  perf-stat.i.dTLB-store-miss-rate%
>    1463946 ±  2%     +43.4%    2099657        perf-stat.i.dTLB-store-misses
>  2.059e+09           +35.2%  2.784e+09 ±  3%  perf-stat.i.dTLB-stores
>    5166354           +14.7%    5927760 ±  2%  perf-stat.i.iTLB-load-misses
>    1840937            +1.9%    1875841        perf-stat.i.iTLB-loads
>  1.495e+10           +29.8%   1.94e+10 ±  3%  perf-stat.i.instructions
>       2903 ±  2%     +11.8%       3245 ±  2%  perf-stat.i.instructions-per-iTLB-miss
>       0.25           +34.4%       0.34        perf-stat.i.ipc
>     533993           +40.1%     748359 ±  3%  perf-stat.i.major-faults
>       0.62            -4.3%       0.60 ±  3%  perf-stat.i.metric.GHz
>       0.81 ±  3%     +22.8%       1.00 ±  3%  perf-stat.i.metric.K/sec
>      96.51           +29.2%     124.71 ±  3%  perf-stat.i.metric.M/sec
>      59652           +31.5%      78424 ±  2%  perf-stat.i.minor-faults
>      70.98            -8.0       62.97 ±  3%  perf-stat.i.node-load-miss-rate%
>   21974526           +15.9%   25469692 ±  3%  perf-stat.i.node-load-misses
>    8913392           +58.8%   14150101        perf-stat.i.node-loads
>      77.63           -12.1       65.55 ±  3%  perf-stat.i.node-store-miss-rate%
>   18858465           +20.4%   22703235 ±  4%  perf-stat.i.node-store-misses
>    5393043 ±  3%    +105.3%   11069950 ±  3%  perf-stat.i.node-stores
>     593645           +39.3%     826783 ±  3%  perf-stat.i.page-faults
>      28.75 ±  2%     -25.7%      21.35        perf-stat.overall.MPKI
>       0.87            -0.3        0.61        perf-stat.overall.branch-miss-rate%
>      37.58           +11.1       48.67        perf-stat.overall.cache-miss-rate%
>       4.00           -26.3%       2.95        perf-stat.overall.cpi
>     370.83 ±  3%     -23.5%     283.79        perf-stat.overall.cycles-between-cache-misses
>       0.18 ±  2%      +0.0        0.19 ±  3%  perf-stat.overall.dTLB-load-miss-rate%
>       0.07            +0.0        0.08 ±  2%  perf-stat.overall.dTLB-store-miss-rate%
>      73.72            +2.2       75.96        perf-stat.overall.iTLB-load-miss-rate%
>       2895 ±  2%     +13.0%       3273        perf-stat.overall.instructions-per-iTLB-miss
>       0.25           +35.7%       0.34        perf-stat.overall.ipc
>      71.14            -6.9       64.27        perf-stat.overall.node-load-miss-rate%
>      77.75           -10.6       67.19        perf-stat.overall.node-store-miss-rate%
>  2.892e+09           +27.6%  3.691e+09 ±  2%  perf-stat.ps.branch-instructions
>   25055439            -9.9%   22571336 ±  3%  perf-stat.ps.branch-misses
>  1.614e+08 ±  2%     +24.9%  2.016e+08 ±  3%  perf-stat.ps.cache-misses
>  4.294e+08            -3.5%  4.142e+08 ±  3%  perf-stat.ps.cache-references
>  5.979e+10            -4.3%  5.721e+10 ±  2%  perf-stat.ps.cpu-cycles
>     114.49            +9.6%     125.42        perf-stat.ps.cpu-migrations
>    6821115 ±  3%     +42.0%    9684339 ±  2%  perf-stat.ps.dTLB-load-misses
>  3.815e+09           +30.8%  4.992e+09 ±  2%  perf-stat.ps.dTLB-loads
>    1462610 ±  2%     +43.4%    2097904        perf-stat.ps.dTLB-store-misses
>  2.057e+09           +35.3%  2.784e+09 ±  3%  perf-stat.ps.dTLB-stores
>    5161712           +14.8%    5925351 ±  2%  perf-stat.ps.iTLB-load-misses
>    1839661            +1.9%    1874628        perf-stat.ps.iTLB-loads
>  1.494e+10           +29.8%   1.94e+10 ±  3%  perf-stat.ps.instructions
>     533681           +40.2%     748293 ±  3%  perf-stat.ps.major-faults
>      59546           +31.5%      78313 ±  2%  perf-stat.ps.minor-faults
>   21957854           +16.0%   25464108 ±  3%  perf-stat.ps.node-load-misses
>    8907729           +58.8%   14145004        perf-stat.ps.node-loads
>   18845579           +20.4%   22695308 ±  4%  perf-stat.ps.node-store-misses
>    5393487 ±  3%    +105.4%   11075863 ±  3%  perf-stat.ps.node-stores
>     593228           +39.3%     826607 ±  3%  perf-stat.ps.page-faults
>  2.726e+13           +33.5%  3.639e+13        perf-stat.total.instructions
>     443491 ±  5%    +212.9%    1387638 ±  5%  softirqs.CPU0.RCU
>     503276 ±  5%    +204.8%    1533957 ± 11%  softirqs.CPU1.RCU
>     458460 ± 10%    +203.1%    1389464 ±  3%  softirqs.CPU10.RCU
>     466020 ±  7%    +189.4%    1348723 ±  4%  softirqs.CPU11.RCU
>     490298 ± 11%    +179.3%    1369353 ±  5%  softirqs.CPU12.RCU
>     494696 ±  8%    +169.3%    1332016 ±  8%  softirqs.CPU13.RCU
>     481566 ± 11%    +168.0%    1290819 ±  6%  softirqs.CPU14.RCU
>     496854 ± 13%    +131.0%    1147735 ±  2%  softirqs.CPU15.RCU
>     531038 ± 10%    +161.4%    1387929 ±  7%  softirqs.CPU16.RCU
>     505272 ±  6%    +167.8%    1352880 ± 12%  softirqs.CPU17.RCU
>     560153 ±  8%    +130.4%    1290800 ±  5%  softirqs.CPU18.RCU
>     516032 ± 12%    +161.3%    1348456 ±  2%  softirqs.CPU19.RCU
>     520803 ± 15%    +162.3%    1365891 ± 12%  softirqs.CPU2.RCU
>     540294 ±  8%    +148.4%    1341956 ± 11%  softirqs.CPU20.RCU
>     518283 ± 10%    +167.3%    1385208 ±  4%  softirqs.CPU21.RCU
>     509050 ±  8%    +154.9%    1297700 ±  6%  softirqs.CPU22.RCU
>     532748 ±  4%    +161.7%    1394395 ± 13%  softirqs.CPU23.RCU
>     168912 ±  2%      +9.4%     184788 ±  2%  softirqs.CPU23.SCHED
>     529751 ± 11%    +176.9%    1466915 ±  5%  softirqs.CPU24.RCU
>     513625 ± 10%    +190.9%    1493952 ±  4%  softirqs.CPU25.RCU
>     534949 ±  6%    +170.0%    1444342 ±  9%  softirqs.CPU26.RCU
>     525891 ±  7%    +158.2%    1358030 ±  6%  softirqs.CPU27.RCU
>     527525 ±  2%    +167.3%    1410109 ±  7%  softirqs.CPU28.RCU
>     538612 ± 12%    +148.6%    1339044 ±  9%  softirqs.CPU29.RCU
>     501518 ± 11%    +154.7%    1277418 ± 18%  softirqs.CPU3.RCU
>     499873 ±  8%    +178.6%    1392722 ± 11%  softirqs.CPU30.RCU
>     541393 ± 15%    +148.7%    1346646 ± 13%  softirqs.CPU31.RCU
>     488032 ± 11%    +196.9%    1448790 ±  5%  softirqs.CPU32.RCU
>     487641 ±  5%    +199.3%    1459556 ±  2%  softirqs.CPU33.RCU
>     494297 ±  9%    +185.9%    1412967 ±  7%  softirqs.CPU34.RCU
>     477316 ±  7%    +151.4%    1199896 ± 11%  softirqs.CPU35.RCU
>     449119 ±  5%    +201.1%    1352432 ±  2%  softirqs.CPU36.RCU
>     492009 ± 12%    +176.8%    1361656 ±  4%  softirqs.CPU37.RCU
>     475731 ±  8%    +153.2%    1204751 ±  5%  softirqs.CPU38.RCU
>     482480 ± 10%    +156.7%    1238457 ±  3%  softirqs.CPU39.RCU
>     509260 ±  9%    +180.9%    1430351 ±  4%  softirqs.CPU4.RCU
>     454685 ±  6%    +174.9%    1250025 ±  5%  softirqs.CPU40.RCU
>     466072 ±  6%    +166.0%    1239804 ±  4%  softirqs.CPU41.RCU
>     454541 ± 10%    +144.8%    1112664 ± 20%  softirqs.CPU42.RCU
>     475468 ±  7%    +237.7%    1605708 ± 18%  softirqs.CPU43.RCU
>     618563 ±  3%     +39.1%     860217 ± 13%  softirqs.CPU43.TIMER
>     450517 ±  5%    +170.1%    1216896 ±  2%  softirqs.CPU44.RCU
>     454575 ± 12%    +161.0%    1186280 ±  7%  softirqs.CPU45.RCU
>     454771 ± 11%    +182.4%    1284239 ± 14%  softirqs.CPU46.RCU
>     449412 ±  6%    +149.5%    1121155 ±  7%  softirqs.CPU47.RCU
>     414849 ±  6%    +126.5%     939428 ±  6%  softirqs.CPU48.RCU
>     433016 ±  5%    +124.7%     972966 ±  4%  softirqs.CPU49.RCU
>     501748 ± 11%    +186.0%    1435214 ±  7%  softirqs.CPU5.RCU
>     423156           +97.1%     834000 ±  7%  softirqs.CPU50.RCU
>     450877 ± 10%     +86.9%     842810 ±  3%  softirqs.CPU51.RCU
>     418004 ±  7%     +97.7%     826189 ±  5%  softirqs.CPU52.RCU
>     441588 ±  9%    +101.6%     890348 ± 11%  softirqs.CPU53.RCU
>     453035 ±  9%    +105.0%     928589 ±  6%  softirqs.CPU54.RCU
>     420431 ±  7%    +121.2%     930137 ±  3%  softirqs.CPU55.RCU
>     438963 ± 12%    +105.8%     903501 ±  7%  softirqs.CPU56.RCU
>     410730 ± 13%    +113.0%     874769 ±  6%  softirqs.CPU57.RCU
>     420291 ± 11%    +107.2%     870797 ±  8%  softirqs.CPU58.RCU
>     407732 ±  4%    +127.5%     927618 ±  2%  softirqs.CPU59.RCU
>     513816 ±  9%    +205.0%    1567003 ±  6%  softirqs.CPU6.RCU
>     433723 ±  8%    +100.7%     870689 ±  8%  softirqs.CPU60.RCU
>     421901 ± 10%     +85.4%     782265 ±  7%  softirqs.CPU61.RCU
>     422623 ± 10%    +109.2%     883991 ± 10%  softirqs.CPU62.RCU
>     446480 ± 12%    +100.9%     896950 ± 13%  softirqs.CPU63.RCU
>     433999 ±  6%     +95.2%     847197 ± 10%  softirqs.CPU64.RCU
>     438890 ±  8%     +94.6%     854073 ±  4%  softirqs.CPU65.RCU
>     454182 ±  8%     +87.7%     852613 ±  4%  softirqs.CPU66.RCU
>     461925 ±  6%     +86.0%     859400 ±  8%  softirqs.CPU67.RCU
>     475795 ±  8%     +83.9%     874999 ±  5%  softirqs.CPU68.RCU
>     455685 ±  9%     +90.2%     866591 ±  4%  softirqs.CPU69.RCU
>     482041 ± 11%    +185.5%    1376404 ±  4%  softirqs.CPU7.RCU
>     463570 ±  8%    +100.2%     928244 ±  6%  softirqs.CPU70.RCU
>     459342 ±  6%     +83.8%     844048 ± 13%  softirqs.CPU71.RCU
>     433326 ±  8%    +106.0%     892758 ±  5%  softirqs.CPU72.RCU
>     443470 ±  9%    +111.3%     937255 ± 12%  softirqs.CPU73.RCU
>     456777 ±  6%    +102.4%     924332 ±  3%  softirqs.CPU74.RCU
>     426667 ±  6%    +109.6%     894288 ± 10%  softirqs.CPU75.RCU
>     459857 ±  5%     +94.8%     895808 ±  9%  softirqs.CPU76.RCU
>     459038 ± 11%    +113.8%     981209 ±  9%  softirqs.CPU77.RCU
>     428188 ±  7%    +102.6%     867397 ± 14%  softirqs.CPU78.RCU
>     437820 ±  8%    +112.9%     932211 ± 14%  softirqs.CPU79.RCU
>     483731 ±  9%    +186.0%    1383272 ±  9%  softirqs.CPU8.RCU
>     442268 ± 11%    +126.6%    1002224 ±  6%  softirqs.CPU80.RCU
>     436999 ±  6%    +136.0%    1031208 ± 11%  softirqs.CPU81.RCU
>     429103 ±  6%    +142.1%    1038942 ±  8%  softirqs.CPU82.RCU
>     430548 ±  5%    +125.4%     970384 ±  8%  softirqs.CPU83.RCU
>     436066 ±  8%    +120.9%     963403 ±  2%  softirqs.CPU84.RCU
>     456423 ±  8%     +94.9%     889464 ±  5%  softirqs.CPU85.RCU
>     439537 ±  6%    +109.2%     919640 ± 11%  softirqs.CPU86.RCU
>     431747 ±  8%    +123.5%     964856 ±  3%  softirqs.CPU87.RCU
>     430063 ±  6%    +127.5%     978424 ±  8%  softirqs.CPU88.RCU
>     434420 ±  8%    +107.6%     901876 ± 10%  softirqs.CPU89.RCU
>     454460 ± 17%    +217.2%    1441628 ±  7%  softirqs.CPU9.RCU
>     416830 ±  7%    +104.7%     853403 ± 17%  softirqs.CPU90.RCU
>     445821 ±  6%     +78.1%     794219 ± 16%  softirqs.CPU91.RCU
>     431634 ±  8%    +121.0%     953758 ±  3%  softirqs.CPU92.RCU
>     422717 ±  7%    +114.5%     906585 ±  8%  softirqs.CPU93.RCU
>     428269 ± 10%    +128.8%     979929 ±  4%  softirqs.CPU94.RCU
>     440717 ± 10%    +114.1%     943752 ±  7%  softirqs.CPU95.RCU
>   44732340 ±  7%    +142.2%  1.083e+08        softirqs.RCU
>       0.75 ±173%  +52900.0%     397.50 ±170%  interrupts.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78
>      97.50 ±165%     -99.7%       0.25 ±173%  interrupts.41:PCI-MSI.31981574-edge.i40e-eth0-TxRx-5
>     999.50 ±169%     -99.7%       3.25 ± 59%  interrupts.51:PCI-MSI.31981584-edge.i40e-eth0-TxRx-15
>  3.665e+08           +36.3%  4.994e+08        interrupts.CAL:Function_call_interrupts
>    4250237 ± 15%     +94.2%    8254579 ± 14%  interrupts.CPU0.CAL:Function_call_interrupts
>     180.25 ± 45%   +1426.5%       2751 ± 73%  interrupts.CPU0.NMI:Non-maskable_interrupts
>     180.25 ± 45%   +1426.5%       2751 ± 73%  interrupts.CPU0.PMI:Performance_monitoring_interrupts
>    4364426 ± 15%    +101.8%    8807883 ± 14%  interrupts.CPU0.TLB:TLB_shootdowns
>    6887147 ±  8%     +31.6%    9065212 ±  5%  interrupts.CPU1.CAL:Function_call_interrupts
>     320681 ± 10%     -22.2%     249429 ± 10%  interrupts.CPU1.RES:Rescheduling_interrupts
>    7069709 ±  8%     +36.9%    9678295 ±  5%  interrupts.CPU1.TLB:TLB_shootdowns
>    5319324 ± 18%     +61.7%    8599906 ±  7%  interrupts.CPU10.CAL:Function_call_interrupts
>    5459270 ± 18%     +68.1%    9178557 ±  6%  interrupts.CPU10.TLB:TLB_shootdowns
>    5438142 ±  8%     +51.2%    8224872 ±  7%  interrupts.CPU11.CAL:Function_call_interrupts
>    5573915 ±  8%     +57.4%    8774360 ±  7%  interrupts.CPU11.TLB:TLB_shootdowns
>    5514339 ± 20%     +51.6%    8357645 ±  7%  interrupts.CPU12.CAL:Function_call_interrupts
>    5655154 ± 20%     +57.7%    8919402 ±  7%  interrupts.CPU12.TLB:TLB_shootdowns
>     316268 ±  9%     -30.3%     220351 ± 12%  interrupts.CPU13.RES:Rescheduling_interrupts
>    5218746 ± 20%     +35.3%    7059981 ±  5%  interrupts.CPU14.CAL:Function_call_interrupts
>    5360749 ± 20%     +40.5%    7533508 ±  5%  interrupts.CPU14.TLB:TLB_shootdowns
>     999.00 ±169%     -99.7%       2.50 ± 60%  interrupts.CPU15.51:PCI-MSI.31981584-edge.i40e-eth0-TxRx-15
>    5268692 ± 21%     +27.4%    6714679 ±  4%  interrupts.CPU15.CAL:Function_call_interrupts
>    5406305 ± 21%     +32.6%    7167336 ±  4%  interrupts.CPU15.TLB:TLB_shootdowns
>     333420 ± 37%     -37.3%     208999 ± 10%  interrupts.CPU16.RES:Rescheduling_interrupts
>     273177 ± 23%     -33.2%     182441 ± 11%  interrupts.CPU18.RES:Rescheduling_interrupts
>    3894908 ± 33%     +85.0%    7204409 ± 10%  interrupts.CPU21.CAL:Function_call_interrupts
>    4000400 ± 33%     +92.1%    7686237 ± 10%  interrupts.CPU21.TLB:TLB_shootdowns
>    3311159 ± 16%    +122.7%    7372406 ± 18%  interrupts.CPU22.CAL:Function_call_interrupts
>    3407341 ± 16%    +130.9%    7867947 ± 18%  interrupts.CPU22.TLB:TLB_shootdowns
>     308340 ± 13%     -33.2%     206079 ± 25%  interrupts.CPU23.RES:Rescheduling_interrupts
>    4731053 ± 21%     +52.1%    7195614 ±  5%  interrupts.CPU24.CAL:Function_call_interrupts
>    4850668 ± 21%     +58.1%    7667084 ±  5%  interrupts.CPU24.TLB:TLB_shootdowns
>    5153629 ± 16%     +31.2%    6759913 ±  6%  interrupts.CPU25.CAL:Function_call_interrupts
>     769.75 ±101%    +469.3%       4382 ± 39%  interrupts.CPU25.NMI:Non-maskable_interrupts
>     769.75 ±101%    +469.3%       4382 ± 39%  interrupts.CPU25.PMI:Performance_monitoring_interrupts
>    5287312 ± 16%     +36.3%    7204382 ±  6%  interrupts.CPU25.TLB:TLB_shootdowns
>     331436 ± 17%     -31.6%     226848 ± 13%  interrupts.CPU26.RES:Rescheduling_interrupts
>     332756 ± 12%     -39.8%     200298 ± 10%  interrupts.CPU27.RES:Rescheduling_interrupts
>    4884898 ± 19%     +46.1%    7138724 ±  8%  interrupts.CPU28.CAL:Function_call_interrupts
>    5022338 ± 19%     +51.4%    7602363 ±  8%  interrupts.CPU28.TLB:TLB_shootdowns
>     935.50 ±142%    +332.3%       4044 ± 82%  interrupts.CPU29.NMI:Non-maskable_interrupts
>     935.50 ±142%    +332.3%       4044 ± 82%  interrupts.CPU29.PMI:Performance_monitoring_interrupts
>     304324 ± 19%     -30.8%     210555 ± 20%  interrupts.CPU29.RES:Rescheduling_interrupts
>     315423 ± 10%     -40.0%     189124 ± 30%  interrupts.CPU3.RES:Rescheduling_interrupts
>    4625033 ± 14%     +53.6%    7103505 ±  8%  interrupts.CPU32.CAL:Function_call_interrupts
>    4747343 ± 14%     +59.5%    7571224 ±  8%  interrupts.CPU32.TLB:TLB_shootdowns
>    4472750 ± 21%     +69.9%    7599668 ±  6%  interrupts.CPU33.CAL:Function_call_interrupts
>    4584043 ± 21%     +76.6%    8095690 ±  7%  interrupts.CPU33.TLB:TLB_shootdowns
>    4268226 ± 38%     +65.0%    7043245 ± 13%  interrupts.CPU34.CAL:Function_call_interrupts
>    4379970 ± 38%     +71.3%    7504681 ± 13%  interrupts.CPU34.TLB:TLB_shootdowns
>     255741 ± 19%     -31.9%     174271 ± 16%  interrupts.CPU35.RES:Rescheduling_interrupts
>    4833654 ± 15%     +28.6%    6217724 ±  9%  interrupts.CPU36.CAL:Function_call_interrupts
>    4953580 ± 15%     +33.9%    6634073 ±  9%  interrupts.CPU36.TLB:TLB_shootdowns
>    3812596 ± 18%     +78.8%    6817083 ± 10%  interrupts.CPU37.CAL:Function_call_interrupts
>    3904296 ± 18%     +86.0%    7263429 ± 10%  interrupts.CPU37.TLB:TLB_shootdowns
>    4570954 ±  4%     +26.3%    5772207 ±  6%  interrupts.CPU38.CAL:Function_call_interrupts
>     246444 ±  9%     -26.2%     181774 ±  4%  interrupts.CPU38.RES:Rescheduling_interrupts
>    4688436 ±  4%     +31.3%    6154828 ±  6%  interrupts.CPU38.TLB:TLB_shootdowns
>     257365 ± 18%     -30.8%     178063 ±  3%  interrupts.CPU39.RES:Rescheduling_interrupts
>    6333449 ± 20%     +44.8%    9168599 ±  8%  interrupts.CPU4.CAL:Function_call_interrupts
>    6509539 ± 20%     +50.3%    9783596 ±  8%  interrupts.CPU4.TLB:TLB_shootdowns
>    3861253 ± 24%     +56.7%    6049605 ± 10%  interrupts.CPU40.CAL:Function_call_interrupts
>       6082 ± 29%     -68.9%       1894 ± 60%  interrupts.CPU40.NMI:Non-maskable_interrupts
>       6082 ± 29%     -68.9%       1894 ± 60%  interrupts.CPU40.PMI:Performance_monitoring_interrupts
>    3955076 ± 25%     +63.0%    6446080 ±  9%  interrupts.CPU40.TLB:TLB_shootdowns
>    3936669 ± 17%     +43.8%    5660309 ± 13%  interrupts.CPU41.CAL:Function_call_interrupts
>       2429 ±121%     -95.5%     109.25 ± 26%  interrupts.CPU41.NMI:Non-maskable_interrupts
>       2429 ±121%     -95.5%     109.25 ± 26%  interrupts.CPU41.PMI:Performance_monitoring_interrupts
>    4040803 ± 17%     +49.3%    6031191 ± 13%  interrupts.CPU41.TLB:TLB_shootdowns
>    3837743 ± 14%    +134.3%    8992108 ± 24%  interrupts.CPU43.CAL:Function_call_interrupts
>    3925024 ± 14%    +143.8%    9569245 ± 23%  interrupts.CPU43.TLB:TLB_shootdowns
>    3723900 ± 18%     +51.1%    5628458 ±  8%  interrupts.CPU44.CAL:Function_call_interrupts
>    3812856 ± 18%     +57.4%    6002945 ±  8%  interrupts.CPU44.TLB:TLB_shootdowns
>    4018194 ± 13%     +41.9%    5701949 ± 14%  interrupts.CPU45.TLB:TLB_shootdowns
>       5920 ± 40%     -93.9%     359.00 ± 72%  interrupts.CPU46.NMI:Non-maskable_interrupts
>       5920 ± 40%     -93.9%     359.00 ± 72%  interrupts.CPU46.PMI:Performance_monitoring_interrupts
>    3095335 ± 10%     +34.1%    4151554 ± 14%  interrupts.CPU47.CAL:Function_call_interrupts
>    3177654 ± 10%     +39.3%    4426572 ± 14%  interrupts.CPU47.TLB:TLB_shootdowns
>    2333315 ± 23%     +49.8%    3496470 ± 17%  interrupts.CPU48.CAL:Function_call_interrupts
>     227.25 ± 40%    +594.5%       1578 ± 57%  interrupts.CPU48.NMI:Non-maskable_interrupts
>     227.25 ± 40%    +594.5%       1578 ± 57%  interrupts.CPU48.PMI:Performance_monitoring_interrupts
>    2388411 ± 23%     +56.4%    3734706 ± 17%  interrupts.CPU48.TLB:TLB_shootdowns
>     156678 ± 16%     -25.9%     116081 ±  3%  interrupts.CPU49.RES:Rescheduling_interrupts
>      97.00 ±166%    -100.0%       0.00        interrupts.CPU5.41:PCI-MSI.31981574-edge.i40e-eth0-TxRx-5
>    6422259 ± 17%     +40.0%    8994022 ±  4%  interrupts.CPU5.CAL:Function_call_interrupts
>    6588736 ± 17%     +45.7%    9599344 ±  4%  interrupts.CPU5.TLB:TLB_shootdowns
>    2754289 ±  8%     +20.4%    3314941 ±  7%  interrupts.CPU50.CAL:Function_call_interrupts
>     129863 ± 10%     -31.6%      88869 ± 12%  interrupts.CPU50.RES:Rescheduling_interrupts
>    2829118 ±  8%     +25.3%    3543789 ±  7%  interrupts.CPU50.TLB:TLB_shootdowns
>     140169 ± 12%     -41.9%      81381 ± 18%  interrupts.CPU51.RES:Rescheduling_interrupts
>     140297 ± 16%     -30.9%      96921 ± 16%  interrupts.CPU53.RES:Rescheduling_interrupts
>    2921981 ± 15%     +27.1%    3714944 ± 13%  interrupts.CPU53.TLB:TLB_shootdowns
>    3237400 ± 19%     +33.0%    4305778 ± 14%  interrupts.CPU54.CAL:Function_call_interrupts
>     166668 ±  6%     -33.2%     111304 ±  9%  interrupts.CPU54.RES:Rescheduling_interrupts
>    3321770 ± 19%     +38.5%    4599776 ± 14%  interrupts.CPU54.TLB:TLB_shootdowns
>    3007556 ± 25%     +29.2%    3886147 ±  5%  interrupts.CPU55.CAL:Function_call_interrupts
>    3087920 ± 25%     +34.5%    4154362 ±  5%  interrupts.CPU55.TLB:TLB_shootdowns
>     134219 ±  6%     -27.1%      97792 ± 12%  interrupts.CPU56.RES:Rescheduling_interrupts
>    1936189 ± 59%     +91.9%    3715887 ±  4%  interrupts.CPU57.CAL:Function_call_interrupts
>    1984477 ± 59%    +100.1%    3971841 ±  4%  interrupts.CPU57.TLB:TLB_shootdowns
>    2155219 ± 23%     +69.9%    3660679 ± 11%  interrupts.CPU59.CAL:Function_call_interrupts
>    2207465 ± 24%     +77.2%    3911795 ± 11%  interrupts.CPU59.TLB:TLB_shootdowns
>    6622562 ± 14%     +50.6%    9974322 ±  9%  interrupts.CPU6.CAL:Function_call_interrupts
>     331433 ± 14%     -19.7%     266042 ±  7%  interrupts.CPU6.RES:Rescheduling_interrupts
>    6791301 ± 14%     +56.7%   10639319 ±  9%  interrupts.CPU6.TLB:TLB_shootdowns
>    2319114 ± 23%     +62.6%    3770521 ± 10%  interrupts.CPU60.CAL:Function_call_interrupts
>     133595 ±  8%     -23.3%     102532 ± 14%  interrupts.CPU60.RES:Rescheduling_interrupts
>    2384205 ± 23%     +69.0%    4028121 ± 10%  interrupts.CPU60.TLB:TLB_shootdowns
>    1985143 ± 36%     +93.9%    3849410 ± 22%  interrupts.CPU64.TLB:TLB_shootdowns
>    1934320 ± 19%     +39.1%    2690966 ± 10%  interrupts.CPU65.CAL:Function_call_interrupts
>    1989414 ± 19%     +44.6%    2876877 ± 10%  interrupts.CPU65.TLB:TLB_shootdowns
>     144699 ±  6%     -41.0%      85346 ± 15%  interrupts.CPU67.RES:Rescheduling_interrupts
>    1781258 ± 54%     +83.6%    3270508 ±  8%  interrupts.CPU69.CAL:Function_call_interrupts
>    1826936 ± 54%     +91.4%    3496606 ±  8%  interrupts.CPU69.TLB:TLB_shootdowns
>    6036433 ± 10%     +45.9%    8805646 ±  6%  interrupts.CPU7.CAL:Function_call_interrupts
>    6189031 ± 10%     +51.8%    9396175 ±  6%  interrupts.CPU7.TLB:TLB_shootdowns
>    1627804 ± 44%    +121.9%    3612617 ± 15%  interrupts.CPU70.CAL:Function_call_interrupts
>    1674222 ± 44%    +130.5%    3859473 ± 16%  interrupts.CPU70.TLB:TLB_shootdowns
>     145373 ± 13%     -44.7%      80450 ± 32%  interrupts.CPU71.RES:Rescheduling_interrupts
>    1854119 ± 28%     +63.6%    3033881 ±  4%  interrupts.CPU72.CAL:Function_call_interrupts
>    1900533 ± 29%     +70.6%    3242000 ±  4%  interrupts.CPU72.TLB:TLB_shootdowns
>     573.25 ± 88%    +755.7%       4905 ± 14%  interrupts.CPU73.NMI:Non-maskable_interrupts
>     573.25 ± 88%    +755.7%       4905 ± 14%  interrupts.CPU73.PMI:Performance_monitoring_interrupts
>       5642 ± 54%     -85.5%     816.00 ±146%  interrupts.CPU74.NMI:Non-maskable_interrupts
>       5642 ± 54%     -85.5%     816.00 ±146%  interrupts.CPU74.PMI:Performance_monitoring_interrupts
>     156895 ± 19%     -31.4%     107653 ±  9%  interrupts.CPU74.RES:Rescheduling_interrupts
>     136591 ± 20%     -34.1%      90037 ± 12%  interrupts.CPU75.RES:Rescheduling_interrupts
>    1957783 ± 33%     +57.4%    3082257 ± 14%  interrupts.CPU76.CAL:Function_call_interrupts
>     134538 ±  9%     -27.1%      98018 ± 16%  interrupts.CPU76.RES:Rescheduling_interrupts
>    2007848 ± 33%     +63.8%    3289196 ± 15%  interrupts.CPU76.TLB:TLB_shootdowns
>    2077242 ± 20%     +46.9%    3052413 ±  5%  interrupts.CPU77.CAL:Function_call_interrupts
>    2130239 ± 20%     +53.0%    3259669 ±  5%  interrupts.CPU77.TLB:TLB_shootdowns
>       0.50 ±173%  +79250.0%     396.75 ±170%  interrupts.CPU78.114:PCI-MSI.31981647-edge.i40e-eth0-TxRx-78
>    5676981 ± 16%     +52.9%    8681781 ± 19%  interrupts.CPU8.CAL:Function_call_interrupts
>    5824519 ± 16%     +59.0%    9260770 ± 19%  interrupts.CPU8.TLB:TLB_shootdowns
>    2790943 ± 15%     +34.8%    3761999 ± 13%  interrupts.CPU80.CAL:Function_call_interrupts
>     166386 ± 15%     -25.1%     124544 ± 13%  interrupts.CPU80.RES:Rescheduling_interrupts
>    2860209 ± 15%     +40.4%    4016175 ± 14%  interrupts.CPU80.TLB:TLB_shootdowns
>    2068338 ± 52%     +99.1%    4118332 ± 10%  interrupts.CPU82.CAL:Function_call_interrupts
>    2117237 ± 52%    +107.5%    4393055 ± 10%  interrupts.CPU82.TLB:TLB_shootdowns
>     189027 ± 32%     -39.5%     114387 ± 16%  interrupts.CPU83.RES:Rescheduling_interrupts
>    2862990 ± 25%     +31.7%    3769327 ±  8%  interrupts.CPU84.CAL:Function_call_interrupts
>    2933064 ± 25%     +37.1%    4021916 ±  8%  interrupts.CPU84.TLB:TLB_shootdowns
>     159294 ± 24%     -31.6%     108891 ± 15%  interrupts.CPU86.RES:Rescheduling_interrupts
>     149416 ± 24%     -24.8%     112293 ±  4%  interrupts.CPU87.RES:Rescheduling_interrupts
>       2160 ±131%     -94.5%     118.00 ± 45%  interrupts.CPU89.NMI:Non-maskable_interrupts
>       2160 ±131%     -94.5%     118.00 ± 45%  interrupts.CPU89.PMI:Performance_monitoring_interrupts
>    5133940 ± 42%     +78.0%    9137868 ±  3%  interrupts.CPU9.CAL:Function_call_interrupts
>    5260647 ± 42%     +85.4%    9752902 ±  3%  interrupts.CPU9.TLB:TLB_shootdowns
>     147315 ± 20%     -43.6%      83131 ± 33%  interrupts.CPU91.RES:Rescheduling_interrupts
>    2765546 ± 15%     +30.5%    3610145 ±  4%  interrupts.CPU92.CAL:Function_call_interrupts
>     160350 ± 19%     -26.2%     118337 ±  5%  interrupts.CPU92.RES:Rescheduling_interrupts
>    2824612 ± 15%     +36.3%    3850288 ±  4%  interrupts.CPU92.TLB:TLB_shootdowns
>    1795118 ± 10%     +70.4%    3058852 ± 27%  interrupts.CPU94.CAL:Function_call_interrupts
>       4289 ± 53%     -91.7%     357.50 ±100%  interrupts.CPU94.NMI:Non-maskable_interrupts
>       4289 ± 53%     -91.7%     357.50 ±100%  interrupts.CPU94.PMI:Performance_monitoring_interrupts
>    1838265 ± 11%     +77.4%    3261908 ± 26%  interrupts.CPU94.TLB:TLB_shootdowns
>   19195824 ±  9%     -22.6%   14857933 ±  2%  interrupts.RES:Rescheduling_interrupts
>  3.759e+08           +41.8%  5.328e+08        interrupts.TLB:TLB_shootdowns
>      30.67 ±  9%     -29.9        0.79 ±  6%  perf-profile.calltrace.cycles-pp.get_swap_page.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec
>      29.50 ±  8%     -29.2        0.28 ±100%  perf-profile.calltrace.cycles-pp.get_swap_pages.get_swap_page.add_to_swap.shrink_page_list.shrink_inactive_list
>      32.44 ±  9%     -22.1       10.31 ± 12%  perf-profile.calltrace.cycles-pp.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
>      17.82 ±  5%     -17.8        0.00        perf-profile.calltrace.cycles-pp.scan_swap_map_slots.get_swap_pages.get_swap_page.add_to_swap.shrink_page_list
>      14.95 ±  4%     -14.9        0.00        perf-profile.calltrace.cycles-pp._raw_spin_lock.scan_swap_map_slots.get_swap_pages.get_swap_page.add_to_swap
>      13.56 ±  4%     -13.6        0.00        perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.scan_swap_map_slots.get_swap_pages.get_swap_page
>      39.98 ± 11%     -11.8       28.19 ± 11%  perf-profile.calltrace.cycles-pp.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
>      41.74 ± 11%     -10.6       31.14 ± 10%  perf-profile.calltrace.cycles-pp.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages
>      43.84 ± 11%      -9.8       34.07 ± 11%  perf-profile.calltrace.cycles-pp.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma
>      43.92 ± 10%      -9.8       34.16 ± 11%  perf-profile.calltrace.cycles-pp.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath
>       9.73 ± 15%      -9.7        0.00        perf-profile.calltrace.cycles-pp._raw_spin_lock.get_swap_pages.get_swap_page.add_to_swap.shrink_page_list
>      43.98 ± 11%      -9.7       34.29 ± 11%  perf-profile.calltrace.cycles-pp.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma.do_swap_page
>      44.32 ± 10%      -9.5       34.79 ± 11%  perf-profile.calltrace.cycles-pp.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath.__alloc_pages_nodemask
>      44.87 ± 11%      -9.5       35.39 ± 11%  perf-profile.calltrace.cycles-pp.__alloc_pages_slowpath.__alloc_pages_nodemask.alloc_pages_vma.do_swap_page.handle_pte_fault
>       8.60 ± 15%      -8.6        0.00        perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.get_swap_pages.get_swap_page.add_to_swap
>       0.97 ± 15%      -0.7        0.27 ±100%  perf-profile.calltrace.cycles-pp.put_swap_page.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       1.39 ± 14%      -0.7        0.71 ± 10%  perf-profile.calltrace.cycles-pp.swap_writepage.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       1.60 ± 13%      -0.6        0.97 ±  9%  perf-profile.calltrace.cycles-pp.try_to_unmap_one.rmap_walk_anon.try_to_unmap.shrink_page_list.shrink_inactive_list
>       1.76 ± 13%      -0.6        1.18 ±  9%  perf-profile.calltrace.cycles-pp.rmap_walk_anon.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       1.80 ± 13%      -0.6        1.23 ± 10%  perf-profile.calltrace.cycles-pp.try_to_unmap.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
>       0.73 ± 16%      +0.3        1.06 ± 14%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_active_list.shrink_lruvec.shrink_node
>       0.73 ± 17%      +0.3        1.07 ± 14%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_active_list.shrink_lruvec.shrink_node.do_try_to_free_pages
>       0.85 ± 17%      +0.4        1.25 ± 10%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node
>       0.88 ± 17%      +0.4        1.28 ± 10%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node.do_try_to_free_pages
>       0.43 ± 61%      +0.4        0.88 ± 10%  perf-profile.calltrace.cycles-pp.page_referenced.shrink_active_list.shrink_lruvec.shrink_node.do_try_to_free_pages
>       0.75 ± 18%      +0.5        1.22 ± 23%  perf-profile.calltrace.cycles-pp.page_vma_mapped_walk.page_referenced_one.rmap_walk_anon.page_referenced.shrink_page_list
>       0.69 ± 18%      +0.5        1.19 ±  9%  perf-profile.calltrace.cycles-pp.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work
>       0.72 ± 18%      +0.5        1.22 ±  9%  perf-profile.calltrace.cycles-pp.drain_local_pages_wq.process_one_work.worker_thread.kthread.ret_from_fork
>       0.72 ± 18%      +0.5        1.22 ±  9%  perf-profile.calltrace.cycles-pp.drain_pages.drain_local_pages_wq.process_one_work.worker_thread.kthread
>       0.71 ± 18%      +0.5        1.21 ±  9%  perf-profile.calltrace.cycles-pp.drain_pages_zone.drain_pages.drain_local_pages_wq.process_one_work.worker_thread
>       0.31 ±100%      +0.5        0.82 ± 12%  perf-profile.calltrace.cycles-pp.end_page_writeback.pmem_rw_page.bdev_write_page.__swap_writepage.pageout
>       0.13 ±173%      +0.5        0.66 ± 11%  perf-profile.calltrace.cycles-pp.page_referenced_one.rmap_walk_anon.page_referenced.shrink_active_list.shrink_lruvec
>       0.00            +0.6        0.56 ± 10%  perf-profile.calltrace.cycles-pp.xas_store.__delete_from_swap_cache.__remove_mapping.shrink_page_list.shrink_inactive_list
>       0.00            +0.6        0.59 ± 11%  perf-profile.calltrace.cycles-pp.shrink_slab.shrink_node.do_try_to_free_pages.try_to_free_pages.__alloc_pages_slowpath
>       0.00            +0.6        0.60 ± 11%  perf-profile.calltrace.cycles-pp.page_vma_mapped_walk.page_referenced_one.rmap_walk_anon.page_referenced.shrink_active_list
>       0.31 ±100%      +0.6        0.92 ± 10%  perf-profile.calltrace.cycles-pp.__delete_from_swap_cache.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       0.00            +0.6        0.60 ± 10%  perf-profile.calltrace.cycles-pp.mem_cgroup_swapout.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       0.00            +0.6        0.61 ± 14%  perf-profile.calltrace.cycles-pp.__lru_cache_add.do_swap_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
>       0.81 ± 18%      +0.6        1.46 ± 24%  perf-profile.calltrace.cycles-pp.page_referenced_one.rmap_walk_anon.page_referenced.shrink_page_list.shrink_inactive_list
>       1.39 ±  7%      +0.7        2.04 ± 14%  perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork
>       1.34 ±  7%      +0.7        1.99 ± 14%  perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork
>       0.17 ±173%      +0.7        0.82 ±  9%  perf-profile.calltrace.cycles-pp.rmap_walk_anon.page_referenced.shrink_active_list.shrink_lruvec.shrink_node
>       0.00            +0.7        0.67 ± 12%  perf-profile.calltrace.cycles-pp.swap_cgroup_record.mem_cgroup_uncharge_swap.swapcache_free_entries.free_swap_slot.__swap_entry_free
>       0.13 ±173%      +0.7        0.81 ± 10%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages
>       0.14 ±173%      +0.7        0.82 ± 10%  perf-profile.calltrace.cycles-pp._raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages.drain_local_pages_wq
>       0.81 ± 15%      +0.7        1.54 ± 12%  perf-profile.calltrace.cycles-pp.mem_cgroup_uncharge_swap.swapcache_free_entries.free_swap_slot.__swap_entry_free.do_swap_page
>       1.69 ± 13%      +0.8        2.46 ± 10%  perf-profile.calltrace.cycles-pp.__memcpy_mcsafe.pmem_do_read.pmem_rw_page.bdev_read_page.swap_readpage
>       1.71 ± 13%      +0.8        2.50 ± 10%  perf-profile.calltrace.cycles-pp.pmem_do_read.pmem_rw_page.bdev_read_page.swap_readpage.do_swap_page
>       0.00            +0.8        0.81 ± 11%  perf-profile.calltrace.cycles-pp.mem_cgroup_id_put_many.mem_cgroup_uncharge_swap.swapcache_free_entries.free_swap_slot.__swap_entry_free
>       2.67 ± 14%      +0.8        3.50 ± 11%  perf-profile.calltrace.cycles-pp.swap_readpage.do_swap_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
>       1.82 ± 13%      +0.9        2.67 ± 10%  perf-profile.calltrace.cycles-pp.pmem_rw_page.bdev_read_page.swap_readpage.do_swap_page.handle_pte_fault
>       1.85 ± 12%      +0.9        2.71 ± 10%  perf-profile.calltrace.cycles-pp.bdev_read_page.swap_readpage.do_swap_page.handle_pte_fault.__handle_mm_fault
>       1.69 ± 15%      +0.9        2.57 ± 12%  perf-profile.calltrace.cycles-pp.shrink_active_list.shrink_lruvec.shrink_node.do_try_to_free_pages.try_to_free_pages
>       1.13 ± 17%      +1.1        2.18 ± 18%  perf-profile.calltrace.cycles-pp.rmap_walk_anon.page_referenced.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       1.19 ± 17%      +1.1        2.29 ± 18%  perf-profile.calltrace.cycles-pp.page_referenced.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
>       1.53 ± 14%      +1.1        2.63 ± 17%  perf-profile.calltrace.cycles-pp.__memcpy_flushcache.write_pmem.pmem_do_write.pmem_rw_page.bdev_write_page
>       1.55 ± 14%      +1.1        2.68 ± 17%  perf-profile.calltrace.cycles-pp.write_pmem.pmem_do_write.pmem_rw_page.bdev_write_page.__swap_writepage
>       1.55 ± 14%      +1.1        2.69 ± 17%  perf-profile.calltrace.cycles-pp.pmem_do_write.pmem_rw_page.bdev_write_page.__swap_writepage.pageout
>       3.36 ± 18%      +1.3        4.62 ± 11%  perf-profile.calltrace.cycles-pp.__swap_writepage.pageout.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       8.54 ±  8%      +1.4        9.90 ± 11%  perf-profile.calltrace.cycles-pp.ret_from_fork
>       8.53 ±  8%      +1.4        9.90 ± 11%  perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
>       0.73 ± 20%      +1.4        2.15 ± 27%  perf-profile.calltrace.cycles-pp.smp_call_function_single.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list
>       0.00            +1.6        1.57 ±  2%  perf-profile.calltrace.cycles-pp.smp_call_function_many_cond.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list
>       2.12 ± 14%      +1.7        3.85 ± 11%  perf-profile.calltrace.cycles-pp.pmem_rw_page.bdev_write_page.__swap_writepage.pageout.shrink_page_list
>       2.54 ± 20%      +1.9        4.42 ± 11%  perf-profile.calltrace.cycles-pp.bdev_write_page.__swap_writepage.pageout.shrink_page_list.shrink_inactive_list
>       0.13 ±173%      +2.3        2.47 ±  7%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.swapcache_free_entries.free_swap_slot.__swap_entry_free
>       0.28 ±100%      +2.4        2.63 ±  7%  perf-profile.calltrace.cycles-pp._raw_spin_lock.swapcache_free_entries.free_swap_slot.__swap_entry_free.do_swap_page
>       1.62 ± 15%      +2.6        4.19 ± 16%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.mem_cgroup_commit_charge.do_swap_page.handle_pte_fault
>       1.71 ± 15%      +2.7        4.39 ± 16%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.mem_cgroup_commit_charge.do_swap_page.handle_pte_fault.__handle_mm_fault
>       1.15 ± 15%      +2.7        3.85 ± 11%  perf-profile.calltrace.cycles-pp.on_each_cpu_cond_mask.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list
>       1.16 ± 15%      +2.7        3.87 ± 11%  perf-profile.calltrace.cycles-pp.arch_tlbbatch_flush.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       1.16 ± 15%      +2.7        3.88 ± 11%  perf-profile.calltrace.cycles-pp.try_to_unmap_flush_dirty.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
>       1.81 ± 15%      +2.8        4.59 ± 16%  perf-profile.calltrace.cycles-pp.mem_cgroup_commit_charge.do_swap_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
>       1.79 ± 14%      +3.4        5.23 ±  9%  perf-profile.calltrace.cycles-pp.swapcache_free_entries.free_swap_slot.__swap_entry_free.do_swap_page.handle_pte_fault
>       2.15 ± 15%      +3.5        5.70 ±  9%  perf-profile.calltrace.cycles-pp.free_swap_slot.__swap_entry_free.do_swap_page.handle_pte_fault.__handle_mm_fault
>       2.34 ± 15%      +3.7        6.03 ±  9%  perf-profile.calltrace.cycles-pp.__swap_entry_free.do_swap_page.handle_pte_fault.__handle_mm_fault.handle_mm_fault
>       0.28 ±100%      +6.2        6.44 ± 13%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec
>       0.00            +6.2        6.25 ± 13%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__remove_mapping.shrink_page_list.shrink_inactive_list
>       0.29 ±100%      +6.7        7.02 ± 12%  perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.add_to_swap_cache.add_to_swap.shrink_page_list.shrink_inactive_list
>       0.13 ±173%      +6.7        6.87 ± 12%  perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irq.add_to_swap_cache.add_to_swap.shrink_page_list
>       2.51 ± 15%      +7.0        9.50 ± 12%  perf-profile.calltrace.cycles-pp.__remove_mapping.shrink_page_list.shrink_inactive_list.shrink_lruvec.shrink_node
>       0.94 ± 15%      +7.6        8.52 ± 12%  perf-profile.calltrace.cycles-pp.add_to_swap_cache.add_to_swap.shrink_page_list.shrink_inactive_list.shrink_lruvec
>      31.30 ±  9%     -30.1        1.19 ±  8%  perf-profile.children.cycles-pp.get_swap_page
>      30.10 ±  9%     -29.3        0.84 ±  9%  perf-profile.children.cycles-pp.get_swap_pages
>      33.04 ±  9%     -22.7       10.38 ± 12%  perf-profile.children.cycles-pp.add_to_swap
>      28.05 ±  9%     -22.3        5.79 ±  8%  perf-profile.children.cycles-pp._raw_spin_lock
>      18.25 ±  5%     -17.7        0.55 ± 10%  perf-profile.children.cycles-pp.scan_swap_map_slots
>      47.61 ± 10%     -12.3       35.27 ± 11%  perf-profile.children.cycles-pp.shrink_page_list
>      49.74 ± 10%     -11.1       38.64 ± 11%  perf-profile.children.cycles-pp.shrink_inactive_list
>      45.29 ± 11%     -10.2       35.09 ± 11%  perf-profile.children.cycles-pp.do_try_to_free_pages
>      45.44 ± 11%     -10.1       35.32 ± 11%  perf-profile.children.cycles-pp.try_to_free_pages
>      51.93 ± 10%     -10.1       41.87 ± 11%  perf-profile.children.cycles-pp.shrink_lruvec
>      46.34 ± 11%      -9.9       36.45 ± 11%  perf-profile.children.cycles-pp.__alloc_pages_slowpath
>       1.64 ± 13%      -0.7        0.91 ± 12%  perf-profile.children.cycles-pp.swap_writepage
>       1.06 ± 13%      -0.7        0.38 ± 10%  perf-profile.children.cycles-pp.page_swapcount
>       1.08 ± 13%      -0.7        0.41 ±  9%  perf-profile.children.cycles-pp.try_to_free_swap
>       1.60 ± 15%      -0.7        0.94 ± 10%  perf-profile.children.cycles-pp.page_swap_info
>       1.88 ± 13%      -0.7        1.23 ± 10%  perf-profile.children.cycles-pp.try_to_unmap_one
>       0.95 ± 13%      -0.6        0.35 ± 10%  perf-profile.children.cycles-pp.__swap_duplicate
>       0.95 ± 13%      -0.6        0.35 ±  9%  perf-profile.children.cycles-pp.swap_duplicate
>       2.11 ± 13%      -0.6        1.53 ± 10%  perf-profile.children.cycles-pp.try_to_unmap
>       1.14 ± 13%      -0.5        0.62 ± 12%  perf-profile.children.cycles-pp.put_swap_page
>       2.06 ± 15%      -0.5        1.54 ± 11%  perf-profile.children.cycles-pp._swap_info_get
>       0.74 ± 14%      -0.4        0.31 ±  7%  perf-profile.children.cycles-pp.swap_set_page_dirty
>       0.67 ± 49%      -0.4        0.29 ± 31%  perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
>       0.67 ± 49%      -0.4        0.29 ± 31%  perf-profile.children.cycles-pp.do_syscall_64
>       0.46 ± 15%      -0.4        0.09 ± 14%  perf-profile.children.cycles-pp.mutex_lock
>       1.15 ± 15%      -0.3        0.82 ±  9%  perf-profile.children.cycles-pp.get_swap_device
>       0.36 ±  9%      -0.3        0.05        perf-profile.children.cycles-pp.pagecache_get_page
>       0.44 ± 42%      -0.3        0.16 ± 39%  perf-profile.children.cycles-pp.forkshell
>       0.40 ± 47%      -0.2        0.15 ± 41%  perf-profile.children.cycles-pp.__libc_fork
>       0.35 ± 52%      -0.2        0.14 ± 42%  perf-profile.children.cycles-pp.__do_sys_clone
>       0.35 ± 52%      -0.2        0.14 ± 42%  perf-profile.children.cycles-pp._do_fork
>       0.35 ± 52%      -0.2        0.14 ± 42%  perf-profile.children.cycles-pp.copy_process
>       0.20 ± 71%      -0.1        0.06 ± 68%  perf-profile.children.cycles-pp.execve
>       0.20 ± 71%      -0.1        0.06 ± 68%  perf-profile.children.cycles-pp.__x64_sys_execve
>       0.20 ± 71%      -0.1        0.06 ± 68%  perf-profile.children.cycles-pp.__do_execve_file
>       0.15 ± 32%      -0.1        0.06 ± 20%  perf-profile.children.cycles-pp.pte_alloc_one
>       0.45 ± 15%      -0.1        0.35 ±  9%  perf-profile.children.cycles-pp.ktime_get
>       0.10 ± 27%      -0.1        0.03 ±100%  perf-profile.children.cycles-pp.do_wp_page
>       0.08 ± 15%      +0.0        0.11 ± 14%  perf-profile.children.cycles-pp.native_flush_tlb
>       0.08 ± 16%      +0.0        0.11 ±  7%  perf-profile.children.cycles-pp.llist_add_batch
>       0.07 ± 12%      +0.0        0.10 ± 10%  perf-profile.children.cycles-pp.xas_init_marks
>       0.08 ± 20%      +0.0        0.11 ±  6%  perf-profile.children.cycles-pp.__count_memcg_events
>       0.07 ± 19%      +0.0        0.10 ±  8%  perf-profile.children.cycles-pp.__mod_zone_page_state
>       0.11 ± 11%      +0.0        0.15 ±  8%  perf-profile.children.cycles-pp.sort_r
>       0.07 ± 30%      +0.0        0.11 ±  7%  perf-profile.children.cycles-pp.__perf_sw_event
>       0.04 ± 59%      +0.0        0.08 ±  5%  perf-profile.children.cycles-pp.mem_cgroup_charge_statistics
>       0.03 ±100%      +0.0        0.06 ±  6%  perf-profile.children.cycles-pp.inc_node_page_state
>       0.12 ± 10%      +0.0        0.16 ± 12%  perf-profile.children.cycles-pp.page_mapping
>       0.09 ± 13%      +0.0        0.13 ± 14%  perf-profile.children.cycles-pp.___might_sleep
>       0.04 ± 58%      +0.0        0.08 ± 15%  perf-profile.children.cycles-pp.super_cache_count
>       0.10 ± 11%      +0.0        0.14 ± 12%  perf-profile.children.cycles-pp.__mod_node_page_state
>       0.03 ±100%      +0.0        0.07 ± 12%  perf-profile.children.cycles-pp.___perf_sw_event
>       0.08 ± 15%      +0.0        0.13 ± 11%  perf-profile.children.cycles-pp.__mod_memcg_state
>       0.11 ± 13%      +0.1        0.16 ± 13%  perf-profile.children.cycles-pp.page_endio
>       0.00            +0.1        0.05 ±  8%  perf-profile.children.cycles-pp.ptep_clear_flush_young
>       0.00            +0.1        0.06 ±  9%  perf-profile.children.cycles-pp.vmacache_find
>       0.00            +0.1        0.06 ± 15%  perf-profile.children.cycles-pp.anon_vma_interval_tree_iter_first
>       0.17 ± 19%      +0.1        0.22 ± 10%  perf-profile.children.cycles-pp.flush_tlb_func_common
>       0.00            +0.1        0.06 ±  7%  perf-profile.children.cycles-pp.find_vma
>       0.00            +0.1        0.06 ± 14%  perf-profile.children.cycles-pp.release_pages
>       0.00            +0.1        0.06 ± 14%  perf-profile.children.cycles-pp.mutex_unlock
>       0.06 ± 14%      +0.1        0.12 ±  8%  perf-profile.children.cycles-pp.wake_all_kswapds
>       0.00            +0.1        0.07 ±  7%  perf-profile.children.cycles-pp.radix_tree_node_rcu_free
>       0.09 ± 11%      +0.1        0.16 ± 10%  perf-profile.children.cycles-pp.do_page_add_anon_rmap
>       0.16 ±  9%      +0.1        0.22 ±  9%  perf-profile.children.cycles-pp.smp_call_function_interrupt
>       0.14 ± 13%      +0.1        0.21 ± 12%  perf-profile.children.cycles-pp.up_read
>       0.20 ± 14%      +0.1        0.27 ± 11%  perf-profile.children.cycles-pp._find_next_bit
>       0.15 ±  5%      +0.1        0.22 ± 10%  perf-profile.children.cycles-pp.__isolate_lru_page
>       0.16 ±  7%      +0.1        0.23 ± 14%  perf-profile.children.cycles-pp.__mod_lruvec_state
>       0.27 ± 16%      +0.1        0.34 ±  9%  perf-profile.children.cycles-pp.__swap_count
>       0.17 ± 14%      +0.1        0.24 ± 10%  perf-profile.children.cycles-pp.cpumask_next
>       0.16 ± 14%      +0.1        0.24 ± 12%  perf-profile.children.cycles-pp.sync_regs
>       0.03 ±100%      +0.1        0.10 ± 10%  perf-profile.children.cycles-pp.wakeup_kswapd
>       0.14 ±  8%      +0.1        0.22 ± 15%  perf-profile.children.cycles-pp.throttle_direct_reclaim
>       0.14 ±  8%      +0.1        0.22 ± 15%  perf-profile.children.cycles-pp.allow_direct_reclaim
>       0.15 ±  5%      +0.1        0.23 ± 15%  perf-profile.children.cycles-pp.zone_reclaimable_pages
>       0.00            +0.1        0.08 ± 15%  perf-profile.children.cycles-pp.rcu_segcblist_enqueue
>       0.16 ± 17%      +0.1        0.26 ± 11%  perf-profile.children.cycles-pp.__pagevec_lru_add_fn
>       0.15 ± 16%      +0.1        0.24 ± 12%  perf-profile.children.cycles-pp.test_clear_page_writeback
>       0.00            +0.1        0.10 ± 30%  perf-profile.children.cycles-pp.radix_tree_node_ctor
>       0.22 ± 10%      +0.1        0.34 ±  7%  perf-profile.children.cycles-pp.call_function_interrupt
>       0.29 ± 18%      +0.1        0.41 ± 11%  perf-profile.children.cycles-pp.count_shadow_nodes
>       0.42 ± 12%      +0.1        0.54 ± 11%  perf-profile.children.cycles-pp.native_irq_return_iret
>       0.02 ±173%      +0.1        0.15 ± 23%  perf-profile.children.cycles-pp.new_slab
>       0.29 ±  6%      +0.1        0.42 ± 11%  perf-profile.children.cycles-pp.move_pages_to_lru
>       0.09 ± 16%      +0.1        0.22 ± 14%  perf-profile.children.cycles-pp.lru_add_drain_cpu
>       0.21 ± 13%      +0.1        0.35 ± 11%  perf-profile.children.cycles-pp.mem_cgroup_id_get_online
>       0.02 ±173%      +0.1        0.17 ± 21%  perf-profile.children.cycles-pp.___slab_alloc
>       0.02 ±173%      +0.2        0.17 ± 19%  perf-profile.children.cycles-pp.__slab_alloc
>       0.00            +0.2        0.16 ± 13%  perf-profile.children.cycles-pp.call_rcu
>       0.39 ± 18%      +0.2        0.57 ± 11%  perf-profile.children.cycles-pp.do_shrink_slab
>       0.41 ± 18%      +0.2        0.59 ± 11%  perf-profile.children.cycles-pp.shrink_slab
>       0.38 ± 11%      +0.2        0.57 ±  9%  perf-profile.children.cycles-pp.irq_exit
>       0.31 ± 14%      +0.2        0.51 ±  7%  perf-profile.children.cycles-pp.down_read_trylock
>       0.00            +0.2        0.20 ± 23%  perf-profile.children.cycles-pp.kmem_cache_alloc
>       0.32 ± 16%      +0.2        0.52 ± 10%  perf-profile.children.cycles-pp.__test_set_page_writeback
>       0.08 ± 19%      +0.2        0.28 ± 10%  perf-profile.children.cycles-pp.__set_page_dirty_no_writeback
>       0.00            +0.2        0.24 ± 20%  perf-profile.children.cycles-pp.xas_alloc
>       0.18 ± 16%      +0.3        0.46 ±  7%  perf-profile.children.cycles-pp.try_to_unmap_flush
>       0.50 ± 12%      +0.3        0.79 ± 11%  perf-profile.children.cycles-pp.page_lock_anon_vma_read
>       0.32 ± 12%      +0.3        0.61 ± 13%  perf-profile.children.cycles-pp.__lru_cache_add
>       0.12 ± 14%      +0.3        0.41 ±  9%  perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
>       0.51 ± 13%      +0.3        0.81 ± 10%  perf-profile.children.cycles-pp.mem_cgroup_swapout
>       0.91 ± 20%      +0.3        1.23 ±  8%  perf-profile.children.cycles-pp.get_page_from_freelist
>       0.23 ± 11%      +0.3        0.55 ± 10%  perf-profile.children.cycles-pp.swap_range_free
>       0.23 ± 12%      +0.3        0.56 ± 11%  perf-profile.children.cycles-pp.lru_add_drain
>       0.41 ± 12%      +0.3        0.75 ± 10%  perf-profile.children.cycles-pp.xas_store
>       0.66 ± 13%      +0.4        1.02 ± 12%  perf-profile.children.cycles-pp.end_page_writeback
>       1.05 ± 13%      +0.4        1.42 ±  7%  perf-profile.children.cycles-pp.__list_del_entry_valid
>       0.91 ±  9%      +0.4        1.27 ±  9%  perf-profile.children.cycles-pp.isolate_lru_pages
>       0.04 ± 58%      +0.4        0.42 ± 12%  perf-profile.children.cycles-pp.__slab_free
>       0.00            +0.4        0.41 ± 13%  perf-profile.children.cycles-pp.run_ksoftirqd
>       0.58 ± 16%      +0.4        1.00 ± 12%  perf-profile.children.cycles-pp.swap_cgroup_record
>       0.00            +0.4        0.42 ± 13%  perf-profile.children.cycles-pp.smpboot_thread_fn
>       0.39 ± 13%      +0.4        0.81 ± 11%  perf-profile.children.cycles-pp.mem_cgroup_id_put_many
>       0.06 ± 13%      +0.4        0.49 ± 13%  perf-profile.children.cycles-pp.kmem_cache_free
>       0.04 ±103%      +0.5        0.52 ± 80%  perf-profile.children.cycles-pp.start_kernel
>       0.66 ± 13%      +0.5        1.15 ± 11%  perf-profile.children.cycles-pp.__delete_from_swap_cache
>       0.72 ± 18%      +0.5        1.22 ±  9%  perf-profile.children.cycles-pp.drain_local_pages_wq
>       0.72 ± 18%      +0.5        1.22 ±  9%  perf-profile.children.cycles-pp.drain_pages
>       0.71 ± 18%      +0.5        1.21 ±  9%  perf-profile.children.cycles-pp.drain_pages_zone
>       0.71 ± 18%      +0.5        1.22 ±  9%  perf-profile.children.cycles-pp.free_pcppages_bulk
>       0.24 ± 14%      +0.5        0.76 ± 14%  perf-profile.children.cycles-pp.xas_create
>       0.24 ± 15%      +0.5        0.77 ± 14%  perf-profile.children.cycles-pp.xas_create_range
>       0.08 ± 13%      +0.6        0.63 ± 13%  perf-profile.children.cycles-pp.rcu_do_batch
>       0.14 ± 15%      +0.6        0.70 ± 12%  perf-profile.children.cycles-pp.rcu_core
>       0.27 ± 13%      +0.6        0.85 ± 12%  perf-profile.children.cycles-pp.__softirqentry_text_start
>       0.53 ± 12%      +0.6        1.15 ± 13%  perf-profile.children.cycles-pp.pagevec_lru_move_fn
>       1.39 ±  7%      +0.7        2.04 ± 14%  perf-profile.children.cycles-pp.worker_thread
>       1.34 ±  7%      +0.7        1.99 ± 14%  perf-profile.children.cycles-pp.process_one_work
>       0.82 ± 14%      +0.7        1.55 ± 12%  perf-profile.children.cycles-pp.mem_cgroup_uncharge_swap
>       1.69 ± 13%      +0.8        2.47 ± 10%  perf-profile.children.cycles-pp.__memcpy_mcsafe
>       1.71 ± 13%      +0.8        2.50 ± 10%  perf-profile.children.cycles-pp.pmem_do_read
>       2.67 ± 14%      +0.8        3.50 ± 11%  perf-profile.children.cycles-pp.swap_readpage
>       1.85 ± 12%      +0.9        2.71 ± 10%  perf-profile.children.cycles-pp.bdev_read_page
>       1.78 ±  9%      +0.9        2.66 ±  9%  perf-profile.children.cycles-pp.page_vma_mapped_walk
>       1.79 ±  9%      +0.9        2.71 ±  9%  perf-profile.children.cycles-pp.page_referenced_one
>       2.08 ±  9%      +1.0        3.03 ± 10%  perf-profile.children.cycles-pp.shrink_active_list
>       1.84 ± 13%      +1.0        2.80 ± 10%  perf-profile.children.cycles-pp.__memcpy_flushcache
>       1.86 ± 13%      +1.0        2.81 ± 11%  perf-profile.children.cycles-pp.write_pmem
>       1.86 ± 13%      +1.0        2.83 ± 10%  perf-profile.children.cycles-pp.pmem_do_write
>       3.53 ± 13%      +1.1        4.66 ± 11%  perf-profile.children.cycles-pp.__swap_writepage
>       0.59 ± 10%      +1.2        1.79 ±  2%  perf-profile.children.cycles-pp.smp_call_function_many_cond
>       2.51 ±  9%      +1.3        3.84 ±  9%  perf-profile.children.cycles-pp.page_referenced
>       8.54 ±  8%      +1.4        9.91 ± 11%  perf-profile.children.cycles-pp.ret_from_fork
>       8.53 ±  8%      +1.4        9.90 ± 11%  perf-profile.children.cycles-pp.kthread
>       1.01 ± 18%      +1.5        2.54 ± 19%  perf-profile.children.cycles-pp.smp_call_function_single
>       2.90 ± 13%      +1.6        4.46 ± 11%  perf-profile.children.cycles-pp.bdev_write_page
>       4.38 ± 13%      +2.2        6.56 ± 11%  perf-profile.children.cycles-pp.pmem_rw_page
>       1.44 ± 13%      +2.5        3.92 ± 11%  perf-profile.children.cycles-pp.try_to_unmap_flush_dirty
>       1.60 ± 13%      +2.7        4.35 ± 11%  perf-profile.children.cycles-pp.on_each_cpu_cond_mask
>       1.62 ± 14%      +2.8        4.38 ± 11%  perf-profile.children.cycles-pp.arch_tlbbatch_flush
>       1.82 ± 15%      +2.8        4.61 ± 16%  perf-profile.children.cycles-pp.mem_cgroup_commit_charge
>       1.79 ± 14%      +3.4        5.23 ±  9%  perf-profile.children.cycles-pp.swapcache_free_entries
>       2.16 ± 14%      +3.5        5.71 ±  9%  perf-profile.children.cycles-pp.free_swap_slot
>       2.34 ± 15%      +3.7        6.03 ±  9%  perf-profile.children.cycles-pp.__swap_entry_free
>       1.07 ± 15%      +6.6        7.66 ± 13%  perf-profile.children.cycles-pp._raw_spin_lock_irqsave
>       2.98 ± 14%      +6.6        9.57 ± 12%  perf-profile.children.cycles-pp.__remove_mapping
>       1.11 ± 15%      +7.5        8.64 ± 12%  perf-profile.children.cycles-pp.add_to_swap_cache
>       4.32 ± 12%     +10.2       14.49 ± 13%  perf-profile.children.cycles-pp._raw_spin_lock_irq
>       2.32 ±  7%      -2.2        0.12 ± 10%  perf-profile.self.cycles-pp.scan_swap_map_slots
>       3.43 ± 11%      -1.9        1.57 ± 10%  perf-profile.self.cycles-pp._raw_spin_lock
>       1.58 ± 15%      -0.7        0.92 ± 10%  perf-profile.self.cycles-pp.page_swap_info
>       2.04 ± 15%      -0.5        1.53 ± 11%  perf-profile.self.cycles-pp._swap_info_get
>       0.70 ± 15%      -0.5        0.20 ±  4%  perf-profile.self.cycles-pp.get_swap_page
>       0.44 ± 15%      -0.4        0.07 ± 12%  perf-profile.self.cycles-pp.mutex_lock
>       1.14 ± 15%      -0.3        0.80 ±  9%  perf-profile.self.cycles-pp.get_swap_device
>       0.34 ± 13%      -0.1        0.21 ± 13%  perf-profile.self.cycles-pp.__frontswap_store
>       0.41 ± 17%      -0.1        0.32 ± 11%  perf-profile.self.cycles-pp.ktime_get
>       0.08 ± 10%      -0.0        0.05 ±  8%  perf-profile.self.cycles-pp.__swap_duplicate
>       0.08 ± 12%      +0.0        0.11 ± 11%  perf-profile.self.cycles-pp.native_flush_tlb
>       0.07 ± 19%      +0.0        0.10 ±  8%  perf-profile.self.cycles-pp.__mod_zone_page_state
>       0.08 ± 16%      +0.0        0.11 ±  9%  perf-profile.self.cycles-pp.handle_pte_fault
>       0.07 ± 15%      +0.0        0.11 ±  7%  perf-profile.self.cycles-pp.__count_memcg_events
>       0.11 ± 11%      +0.0        0.15 ± 12%  perf-profile.self.cycles-pp.page_mapping
>       0.08 ± 13%      +0.0        0.12 ± 10%  perf-profile.self.cycles-pp.___might_sleep
>       0.09 ± 17%      +0.0        0.13 ± 12%  perf-profile.self.cycles-pp.__mod_node_page_state
>       0.03 ±100%      +0.0        0.07 ± 10%  perf-profile.self.cycles-pp.do_page_fault
>       0.15 ± 11%      +0.0        0.19 ± 10%  perf-profile.self.cycles-pp.do_swap_page
>       0.07 ± 20%      +0.0        0.12 ± 12%  perf-profile.self.cycles-pp.__handle_mm_fault
>       0.12 ±  7%      +0.0        0.16 ± 14%  perf-profile.self.cycles-pp.page_referenced_one
>       0.08 ± 14%      +0.0        0.12 ± 12%  perf-profile.self.cycles-pp.__mod_memcg_state
>       0.10 ± 10%      +0.0        0.15 ± 14%  perf-profile.self.cycles-pp.page_endio
>       0.07 ± 19%      +0.1        0.12 ± 15%  perf-profile.self.cycles-pp.test_clear_page_writeback
>       0.08 ± 19%      +0.1        0.14 ± 15%  perf-profile.self.cycles-pp.__pagevec_lru_add_fn
>       0.09 ±  4%      +0.1        0.14 ± 15%  perf-profile.self.cycles-pp.zone_reclaimable_pages
>       0.00            +0.1        0.06 ±  9%  perf-profile.self.cycles-pp.vmacache_find
>       0.00            +0.1        0.06 ±  9%  perf-profile.self.cycles-pp.mutex_unlock
>       0.03 ±100%      +0.1        0.08 ± 13%  perf-profile.self.cycles-pp.xas_store
>       0.00            +0.1        0.06 ± 15%  perf-profile.self.cycles-pp.anon_vma_interval_tree_iter_first
>       0.00            +0.1        0.06 ± 14%  perf-profile.self.cycles-pp.kmem_cache_free
>       0.08 ± 13%      +0.1        0.14 ±  9%  perf-profile.self.cycles-pp.do_page_add_anon_rmap
>       0.01 ±173%      +0.1        0.07 ± 17%  perf-profile.self.cycles-pp.mem_cgroup_commit_charge
>       0.00            +0.1        0.06 ± 11%  perf-profile.self.cycles-pp.xas_init_marks
>       0.18 ± 13%      +0.1        0.25 ± 11%  perf-profile.self.cycles-pp._find_next_bit
>       0.11 ±  9%      +0.1        0.18 ± 15%  perf-profile.self.cycles-pp.isolate_lru_pages
>       0.00            +0.1        0.06 ± 13%  perf-profile.self.cycles-pp.swapcache_free_entries
>       0.14 ± 12%      +0.1        0.21 ± 11%  perf-profile.self.cycles-pp.rmap_walk_anon
>       0.14 ± 11%      +0.1        0.21 ± 11%  perf-profile.self.cycles-pp.up_read
>       0.00            +0.1        0.07 ±  7%  perf-profile.self.cycles-pp.radix_tree_node_rcu_free
>       0.18 ± 16%      +0.1        0.25 ±  9%  perf-profile.self.cycles-pp.free_pcppages_bulk
>       0.15 ±  5%      +0.1        0.22 ± 10%  perf-profile.self.cycles-pp.__isolate_lru_page
>       0.16 ± 14%      +0.1        0.23 ± 12%  perf-profile.self.cycles-pp.sync_regs
>       0.16 ±  6%      +0.1        0.23 ± 10%  perf-profile.self.cycles-pp.move_pages_to_lru
>       0.00            +0.1        0.07 ± 11%  perf-profile.self.cycles-pp.call_rcu
>       0.18 ± 18%      +0.1        0.26 ± 12%  perf-profile.self.cycles-pp.count_shadow_nodes
>       0.00            +0.1        0.08 ± 15%  perf-profile.self.cycles-pp.rcu_segcblist_enqueue
>       0.08 ±  6%      +0.1        0.16 ±  9%  perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
>       0.14 ± 14%      +0.1        0.23 ±  6%  perf-profile.self.cycles-pp.pageout
>       0.01 ±173%      +0.1        0.10 ± 10%  perf-profile.self.cycles-pp.wakeup_kswapd
>       0.27 ±  9%      +0.1        0.37 ± 10%  perf-profile.self.cycles-pp.shrink_page_list
>       0.00            +0.1        0.10 ± 30%  perf-profile.self.cycles-pp.radix_tree_node_ctor
>       0.08 ± 10%      +0.1        0.19 ± 11%  perf-profile.self.cycles-pp.lookup_swap_cache
>       0.22 ± 12%      +0.1        0.34 ± 14%  perf-profile.self.cycles-pp.page_lock_anon_vma_read
>       0.42 ± 12%      +0.1        0.54 ± 11%  perf-profile.self.cycles-pp.native_irq_return_iret
>       0.12 ± 18%      +0.1        0.26 ±  9%  perf-profile.self.cycles-pp.__swap_count
>       0.21 ± 13%      +0.1        0.35 ± 11%  perf-profile.self.cycles-pp.mem_cgroup_id_get_online
>       0.07 ± 17%      +0.1        0.21 ± 12%  perf-profile.self.cycles-pp.__remove_mapping
>       0.26 ± 12%      +0.1        0.41 ± 11%  perf-profile.self.cycles-pp.__delete_from_swap_cache
>       0.25 ± 15%      +0.2        0.42 ± 10%  perf-profile.self.cycles-pp.get_page_from_freelist
>       0.24 ± 17%      +0.2        0.42 ± 10%  perf-profile.self.cycles-pp.__test_set_page_writeback
>       0.30 ± 15%      +0.2        0.49 ±  7%  perf-profile.self.cycles-pp.down_read_trylock
>       0.08 ± 19%      +0.2        0.28 ± 11%  perf-profile.self.cycles-pp.__set_page_dirty_no_writeback
>       0.28 ± 14%      +0.2        0.49 ±  9%  perf-profile.self.cycles-pp._raw_spin_lock_irq
>       0.37 ± 16%      +0.2        0.59 ± 13%  perf-profile.self.cycles-pp.swap_cgroup_record
>       0.51 ± 11%      +0.2        0.76 ± 12%  perf-profile.self.cycles-pp.end_page_writeback
>       0.35 ± 15%      +0.3        0.60 ± 12%  perf-profile.self.cycles-pp._raw_spin_lock_irqsave
>       0.22 ± 16%      +0.3        0.51 ± 11%  perf-profile.self.cycles-pp.xas_create
>       0.20 ± 11%      +0.3        0.50 ± 10%  perf-profile.self.cycles-pp.swap_range_free
>       0.25 ± 13%      +0.3        0.56 ± 11%  perf-profile.self.cycles-pp.add_to_swap_cache
>       1.05 ± 13%      +0.4        1.41 ±  8%  perf-profile.self.cycles-pp.__list_del_entry_valid
>       0.03 ±100%      +0.4        0.41 ± 11%  perf-profile.self.cycles-pp.__slab_free
>       0.38 ± 13%      +0.4        0.80 ± 12%  perf-profile.self.cycles-pp.mem_cgroup_id_put_many
>       1.67 ± 13%      +0.8        2.43 ± 10%  perf-profile.self.cycles-pp.__memcpy_mcsafe
>       1.54 ±  9%      +0.8        2.31 ±  9%  perf-profile.self.cycles-pp.page_vma_mapped_walk
>       1.81 ± 12%      +0.9        2.74 ± 11%  perf-profile.self.cycles-pp.__memcpy_flushcache
>       0.47 ± 12%      +1.1        1.60 ±  2%  perf-profile.self.cycles-pp.smp_call_function_many_cond
>       0.93 ± 18%      +1.5        2.42 ± 20%  perf-profile.self.cycles-pp.smp_call_function_single
>
>
>                                                                                 
>                               pmbench.time.user_time                            
>                                                                                 
>   1200 +--------------------------------------------------------------------+   
>        |                                            O                       |   
>   1000 |-+                                                                  |   
>        |  O          O                    O                                 |   
>        |      O  O                           O  O                           |   
>    800 |-+              O   O  O   O  O                O                    |   
>        |                                                                    |   
>    600 |..  ..+..+...+..+...+..+...+..+...+       ..+..+...+..+...+..+...  .|   
>        |  +.                              :     +.                       +. |   
>    400 |-+                                 :    :                           |   
>        |                                   :   :                            |   
>        |                                    :  :                            |   
>    200 |-+                                  : :                             |   
>        |                                     ::                             |   
>      0 +--------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                               pmbench.time.system_time                          
>                                                                                 
>   30000 +-------------------------------------------------------------------+   
>         |..+...+..+...+..+..+...+..+...+..+  O   +..+...+..+..+...+..+...+..|   
>   25000 |-+O   O  O   O                   :      :                          |   
>         |                                 :     :                           |   
>         |                                  :    :                           |   
>   20000 |-+                                :    :                           |   
>         |                                  :   :                            |   
>   15000 |-+                                :   :                            |   
>         |                                   :  :                            |   
>   10000 |-+                                 :  :                            |   
>         |                                   : :                             |   
>         |                                   : :                             |   
>    5000 |-+                                  ::                             |   
>         |                                    :                              |   
>       0 +-------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                             pmbench.time.major_page_faults                      
>                                                                                 
>   1.8e+09 +-----------------------------------------------------------------+   
>           |  O   O  O  O                                                    |   
>   1.6e+09 |-+                                                               |   
>   1.4e+09 |-+              O  O  O  O   O  O  O   O  O  O                   |   
>           |                                                                 |   
>   1.2e+09 |-+                                                               |   
>     1e+09 |..         .+...+..  .+..+...+..+        .+..+...  .+..+..+...  .|   
>           |  +...+..+.        +.           :      +.        +.           +. |   
>     8e+08 |-+                               :    :                          |   
>     6e+08 |-+                               :    :                          |   
>           |                                  :  :                           |   
>     4e+08 |-+                                :  :                           |   
>     2e+08 |-+                                : :                            |   
>           |                                   ::                            |   
>         0 +-----------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                             pmbench.time.minor_page_faults                      
>                                                                                 
>   1.8e+08 +-----------------------------------------------------------------+   
>           |  O   O  O  O                                                    |   
>   1.6e+08 |-+                                                               |   
>   1.4e+08 |-+              O  O  O  O   O  O  O   O  O  O                   |   
>           |                                                                 |   
>   1.2e+08 |-+                                                               |   
>     1e+08 |..+...+..+..+...+..+..+..+...+..+      +..+..+...+..+..+..+...+..|   
>           |                                :      :                         |   
>     8e+07 |-+                               :    :                          |   
>     6e+07 |-+                               :    :                          |   
>           |                                  :  :                           |   
>     4e+07 |-+                                :  :                           |   
>     2e+07 |-+                                : :                            |   
>           |                                   ::                            |   
>         0 +-----------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                          pmbench.read.latency.ns.32K-64K_                       
>                                                                                 
>   0.9 +---------------------------------------------------------------------+   
>       |                                                       +...          |   
>   0.8 |-+             ..+..+...+..                    .+... ..    +..      .|   
>   0.7 |-..+..      .+.            +...+..+      +...+.     +         +...+. |   
>       |.     +...+.                      :      :                           |   
>   0.6 |-+                                 :    :                            |   
>   0.5 |-+                                 :    :                            |   
>       |                                   :    :                            |   
>   0.4 |-+                                  :   :                            |   
>   0.3 |-+                                  :  :                             |   
>       |                 O  O   O  O   O     :O: O      O                    |   
>   0.2 |-+ O  O   O  O                    O  : :                             |   
>   0.1 |-+                                   : :     O                       |   
>       |                                      :                              |   
>     0 +---------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                           pmbench.read.latency.ns.2M-4M_                        
>                                                                                 
>    0.3 +--------------------------------------------------------------------+   
>        | +  ..           ..                     :  .  .+...+..  ..+..  ..+..|   
>   0.25 |++      .+...+..+      +...      .+     :   +.        +.     +.     |   
>        |      +.                   +.. .. :    :                            |   
>        |                              +    :   :                            |   
>    0.2 |-+                                 :   :                            |   
>        |                                   :   :                            |   
>   0.15 |-+                                 :   :                            |   
>        |                                    : :                             |   
>    0.1 |-+                                  : :                             |   
>        |                                    : :                             |   
>        |                                    : :                             |   
>   0.05 |-+                         O         :                              |   
>        |      O  O   O  O   O  O      O   O  :  O   O  O                    |   
>      0 +--------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                           pmbench.read.latency.ns.4M-8M_                        
>                                                                                 
>   0.06 +--------------------------------------------------------------------+   
>        |                                                                 +  |   
>   0.05 |-+                                +                             : : |   
>        |                                 +:                   +.        : : |   
>        |                                +  :                ..  ..     :   :|   
>   0.04 |-+  ..+..   .+         +       +   :               +           :   :|   
>        |  +.      ..  :       + +     +    :             ..       +.. :     |   
>   0.03 |..       +     :     +   +   +     :            .             :     |   
>        |               :  ..+     + +       :   +...+..+             +      |   
>   0.02 |-+              +.         +        :   :                           |   
>        |                                    :  :                            |   
>        |                                    :  :                            |   
>   0.01 |-+                                   ::                             |   
>        |  O   O  O   O      O             O  ::                             |   
>      0 +--------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                          pmbench.write.latency.ns.32K-64K_                      
>                                                                                 
>   0.9 +---------------------------------------------------------------------+   
>       |                                                       +...          |   
>   0.8 |-+             ..+..+...+..                    .+... ..    +..      .|   
>   0.7 |-..+..      .+.            +...+..+      +...+.     +         +...+. |   
>       |.     +...+.                      :      :                           |   
>   0.6 |-+                                 :    :                            |   
>   0.5 |-+                                 :    :                            |   
>       |                                   :    :                            |   
>   0.4 |-+                                  :   :                            |   
>   0.3 |-+                                  :  :                             |   
>       |                 O  O   O  O   O     :O: O      O                    |   
>   0.2 |-+ O  O   O  O                    O  : :                             |   
>   0.1 |-+                                   : :     O                       |   
>       |                                      :                              |   
>     0 +---------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                           pmbench.write.latency.ns.2M-4M_                       
>                                                                                 
>    0.3 +--------------------------------------------------------------------+   
>        | +  ..           ..                     :  .  .+...+..  ..+..  ..+..|   
>   0.25 |++      .+...+..+      +...      .+     :   +.        +.     +.     |   
>        |      +.                   +.. .. :    :                            |   
>        |                              +    :   :                            |   
>    0.2 |-+                                 :   :                            |   
>        |                                   :   :                            |   
>   0.15 |-+                                 :   :                            |   
>        |                                    : :                             |   
>    0.1 |-+                                  : :                             |   
>        |                                    : :                             |   
>        |                                    : :                             |   
>   0.05 |-+                         O         :                              |   
>        |      O  O   O  O   O  O      O   O  :  O   O  O                    |   
>      0 +--------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                           pmbench.write.latency.ns.4M-8M_                       
>                                                                                 
>   0.06 +--------------------------------------------------------------------+   
>        |                                                                 +  |   
>   0.05 |-+                                +                             : : |   
>        |                                 +:                   +.        : : |   
>        |                                +  :                ..  ..     :   :|   
>   0.04 |-++...+..   .+         +       +   :               +          :    :|   
>        | +        ..  :       + +     +    :             ..       +.. :     |   
>   0.03 |++       +     :     +   +   +     :            .            +      |   
>        |               :  ..+     + +       :   +...+..+                    |   
>   0.02 |-+              +.         +        :   :                           |   
>        |                                    :  :                            |   
>        |                                    :  :                            |   
>   0.01 |-+                                   ::                             |   
>        |  O   O  O   O      O             O  ::                             |   
>      0 +--------------------------------------------------------------------+   
>                                                                                 
>                                                                                                                                                                 
>                              pmbench.latency.ns.average                         
>                                                                                 
>   20000 +-------------------------------------------------------------------+   
>   18000 |.+   .  .+...+..+..+...+..      .+      :  +...+..+..+...+..+.     |   
>         |      +.                  +...+. :     :                           |   
>   16000 |-+                               :     :                           |   
>   14000 |-+                                :    :                           |   
>         |                O  O   O  O   O  O:    :O      O                   |   
>   12000 |-+    O  O   O                    : O :    O                       |   
>   10000 |-+O                               :   :                            |   
>    8000 |-+                                 :  :                            |   
>         |                                   : :                             |   
>    6000 |-+                                 : :                             |   
>    4000 |-+                                 : :                             |   
>         |                                    ::                             |   
>    2000 |-+                                  :                              |   
>       0 +-------------------------------------------------------------------+   
>                                                                                 
>                                                                                 
> [*] bisect-good sample
> [O] bisect-bad  sample
>
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
> Thanks,
> Rong Chen

      reply	other threads:[~2020-05-27  8:44 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-27  5:33 [swap] 9c9c831c31: pmbench.latency.ns.average -33.9% improvement kernel test robot
2020-05-27  8:44 ` Huang, Ying [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87pnaphin3.fsf@yhuang-dev.intel.com \
    --to=ying.huang@intel.com \
    --cc=lkp@lists.01.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.