* [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression
@ 2024-07-12 6:04 kernel test robot
2024-07-12 19:03 ` Roman Gushchin
0 siblings, 1 reply; 12+ messages in thread
From: kernel test robot @ 2024-07-12 6:04 UTC (permalink / raw)
To: Roman Gushchin
Cc: oe-lkp, lkp, Linux Memory Management List, Andrew Morton,
Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups,
ying.huang, feng.tang, fengwei.yin, oliver.sang
Hello,
kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on:
commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
testcase: aim7
test machine: 128 threads 2 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz (Ice Lake) with 256G memory
parameters:
disk: 1BRD_48G
fs: ext4
test: disk_rr
load: 3000
cpufreq_governor: performance
some further information.
we don't have CONFIG_MEMCG_V1 in our config which is uploaded to [1]
# CONFIG_MEMCG_V1 is not set
we also tried to capturing perf-c2c data in the whole process of this benchmark.
got below:
94b7e5bf09b08aa4 98c9daf5ae6be008f78c07b744b
---------------- ---------------------------
%stddev %change %stddev
\ | \
5252 ± 4% +86.6% 9801 ± 3% perf-c2c.DRAM.local
149160 ± 3% +95.7% 291886 ± 2% perf-c2c.DRAM.remote
98978 +114.6% 212395 perf-c2c.HITM.local
10458 ± 2% +47.2% 15397 ± 2% perf-c2c.HITM.remote
109437 +108.1% 227793 perf-c2c.HITM.total
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202407121335.31a10cb6-oliver.sang@intel.com
Details are as below:
-------------------------------------------------------------------------------------------------->
[1]
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20240712/202407121335.31a10cb6-oliver.sang@intel.com
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase:
gcc-13/performance/1BRD_48G/ext4/x86_64-rhel-8.3/3000/debian-12-x86_64-20240206.cgz/lkp-icl-2sp2/disk_rr/aim7
commit:
94b7e5bf09 ("mm: memcg: put memcg1-specific struct mem_cgroup's members under CONFIG_MEMCG_V1")
98c9daf5ae ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node")
94b7e5bf09b08aa4 98c9daf5ae6be008f78c07b744b
---------------- ---------------------------
%stddev %change %stddev
\ | \
4752 ± 13% -18.1% 3890 ± 11% sched_debug.cpu.avg_idle.min
76.42 +15.1% 87.99 ± 3% uptime.boot
53.28 ± 2% -27.5% 38.65 ± 2% iostat.cpu.idle
44.58 ± 2% +33.6% 59.58 iostat.cpu.system
2.13 -17.0% 1.77 iostat.cpu.user
49.85 ± 2% -14.7 35.17 ± 2% mpstat.cpu.all.idle%
47.74 ± 2% +15.1 62.82 mpstat.cpu.all.sys%
2.23 -0.4 1.82 mpstat.cpu.all.usr%
53.29 ± 2% -27.4% 38.70 ± 2% vmstat.cpu.id
65.83 ± 4% +59.1% 104.76 ± 3% vmstat.procs.r
8385 ± 4% +34.6% 11284 ± 3% vmstat.system.cs
245966 ± 2% +8.1% 265964 vmstat.system.in
778685 -29.4% 549435 aim7.jobs-per-min
23.31 +41.4% 32.96 aim7.time.elapsed_time
23.31 +41.4% 32.96 aim7.time.elapsed_time.max
47890 ± 7% +338.5% 210000 ± 3% aim7.time.involuntary_context_switches
6674 +26.7% 8455 aim7.time.percent_of_cpu_this_job_got
1510 +81.7% 2744 ± 2% aim7.time.system_time
19454 ± 3% +9.1% 21223 ± 3% aim7.time.voluntary_context_switches
49345 ± 6% +108.4% 102820 ± 10% meminfo.Active
22670 ± 11% +182.6% 64065 ± 15% meminfo.Active(anon)
26674 ± 4% +45.3% 38754 ± 4% meminfo.Active(file)
33695 +18.6% 39973 ± 3% meminfo.AnonHugePages
1360098 ± 3% +14.0% 1551107 ± 2% meminfo.Inactive
803759 +15.0% 924519 meminfo.Inactive(anon)
66977 ± 3% +115.7% 144485 ± 7% meminfo.Mapped
78152 ± 9% +188.4% 225431 ± 5% meminfo.Shmem
15327 ± 8% +52.6% 23389 ± 11% numa-meminfo.node0.Active
13455 ± 9% +46.0% 19642 ± 10% numa-meminfo.node0.Active(file)
19657 ± 14% +111.2% 41510 ± 18% numa-meminfo.node0.Mapped
6081 ± 16% +84.9% 11247 ± 42% numa-meminfo.node0.Shmem
33647 ± 8% +134.3% 78825 ± 15% numa-meminfo.node1.Active
20790 ± 14% +188.7% 60020 ± 18% numa-meminfo.node1.Active(anon)
12857 ± 7% +46.3% 18805 ± 9% numa-meminfo.node1.Active(file)
485228 ± 23% +85.5% 899983 ± 21% numa-meminfo.node1.Inactive
202089 ± 50% +192.9% 591917 ± 32% numa-meminfo.node1.Inactive(anon)
47991 ± 7% +112.0% 101755 ± 8% numa-meminfo.node1.Mapped
72431 ± 8% +194.1% 213055 ± 4% numa-meminfo.node1.Shmem
3290 ± 10% +50.9% 4965 ± 10% numa-vmstat.node0.nr_active_file
4793 ± 17% +113.8% 10249 ± 20% numa-vmstat.node0.nr_mapped
1519 ± 16% +72.6% 2622 ± 33% numa-vmstat.node0.nr_shmem
3294 ± 9% +50.9% 4970 ± 10% numa-vmstat.node0.nr_zone_active_file
4955 ± 11% +202.3% 14980 ± 18% numa-vmstat.node1.nr_active_anon
3376 ± 8% +40.1% 4729 ± 7% numa-vmstat.node1.nr_active_file
50152 ± 51% +194.6% 147731 ± 32% numa-vmstat.node1.nr_inactive_anon
12101 ± 7% +109.5% 25357 ± 8% numa-vmstat.node1.nr_mapped
17201 ± 7% +208.3% 53026 ± 4% numa-vmstat.node1.nr_shmem
4955 ± 11% +202.3% 14980 ± 18% numa-vmstat.node1.nr_zone_active_anon
3352 ± 8% +41.1% 4729 ± 7% numa-vmstat.node1.nr_zone_active_file
50150 ± 51% +194.6% 147730 ± 32% numa-vmstat.node1.nr_zone_inactive_anon
5776 ± 11% +175.1% 15893 ± 16% proc-vmstat.nr_active_anon
6595 ± 8% +46.7% 9677 ± 3% proc-vmstat.nr_active_file
187950 +2.0% 191760 proc-vmstat.nr_anon_pages
945653 +6.0% 1002646 proc-vmstat.nr_file_pages
201118 +14.8% 230842 proc-vmstat.nr_inactive_anon
17214 ± 3% +108.2% 35836 ± 6% proc-vmstat.nr_mapped
19951 ± 8% +180.4% 55933 ± 4% proc-vmstat.nr_shmem
40267 +2.7% 41335 proc-vmstat.nr_slab_reclaimable
86277 +1.8% 87792 proc-vmstat.nr_slab_unreclaimable
5776 ± 11% +175.1% 15893 ± 16% proc-vmstat.nr_zone_active_anon
6595 ± 8% +46.7% 9677 ± 3% proc-vmstat.nr_zone_active_file
201118 +14.8% 230842 proc-vmstat.nr_zone_inactive_anon
312.12 ±241% +1618.1% 5362 ±125% proc-vmstat.numa_pages_migrated
369792 +19.6% 442285 ± 3% proc-vmstat.pgfault
312.12 ±241% +1618.1% 5362 ±125% proc-vmstat.pgmigrate_success
2426 ± 2% +29.2% 3135 ± 3% proc-vmstat.pgpgout
1515 +3.8% 1572 proc-vmstat.unevictable_pgs_culled
0.63 ± 3% +36.5% 0.85 ± 2% perf-stat.i.MPKI
1.885e+10 -23.7% 1.437e+10 perf-stat.i.branch-instructions
2.78 ± 2% -0.5 2.29 ± 4% perf-stat.i.branch-miss-rate%
67232899 -17.4% 55553782 ± 3% perf-stat.i.branch-misses
13.70 ± 3% +2.5 16.17 ± 3% perf-stat.i.cache-miss-rate%
5.483e+08 -24.9% 4.118e+08 perf-stat.i.cache-references
8605 ± 4% +34.7% 11593 ± 4% perf-stat.i.context-switches
1.35 +65.4% 2.22 ± 3% perf-stat.i.cpi
1.616e+11 +22.1% 1.973e+11 perf-stat.i.cpu-cycles
8.537e+10 -24.8% 6.416e+10 perf-stat.i.instructions
0.96 -22.2% 0.75 ± 2% perf-stat.i.ipc
13455 -15.4% 11379 ± 3% perf-stat.i.minor-faults
13489 -15.5% 11396 ± 3% perf-stat.i.page-faults
0.85 ± 3% +34.8% 1.15 perf-stat.overall.MPKI
0.34 +0.0 0.36 ± 2% perf-stat.overall.branch-miss-rate%
13.23 ± 3% +4.6 17.78 perf-stat.overall.cache-miss-rate%
1.90 +62.8% 3.08 perf-stat.overall.cpi
2231 ± 3% +20.6% 2690 ± 2% perf-stat.overall.cycles-between-cache-misses
0.53 -38.5% 0.32 perf-stat.overall.ipc
1.86e+10 -23.0% 1.433e+10 perf-stat.ps.branch-instructions
64095803 -19.1% 51843320 ± 2% perf-stat.ps.branch-misses
5.417e+08 -23.9% 4.124e+08 perf-stat.ps.cache-references
8220 ± 4% +33.3% 10962 ± 4% perf-stat.ps.context-switches
1.597e+11 +23.5% 1.972e+11 ± 2% perf-stat.ps.cpu-cycles
8.426e+10 -24.1% 6.394e+10 perf-stat.ps.instructions
12730 -18.7% 10354 ± 3% perf-stat.ps.minor-faults
12762 -18.8% 10369 ± 3% perf-stat.ps.page-faults
3.93 ± 6% -3.2 0.69 ± 8% perf-profile.calltrace.cycles-pp.creat64
3.93 ± 6% -3.2 0.69 ± 8% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64
3.93 ± 6% -3.2 0.69 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.creat64
3.92 ± 6% -3.2 0.68 ± 8% perf-profile.calltrace.cycles-pp.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64
3.92 ± 6% -3.2 0.68 ± 8% perf-profile.calltrace.cycles-pp.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64
3.91 ± 6% -3.2 0.68 ± 8% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64
3.91 ± 6% -3.2 0.68 ± 8% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.87 ± 6% -3.2 0.65 ± 9% perf-profile.calltrace.cycles-pp.open_last_lookups.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat
3.66 -1.7 1.99 ± 2% perf-profile.calltrace.cycles-pp.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write
3.31 -1.5 1.77 ± 2% perf-profile.calltrace.cycles-pp.llseek
2.87 -1.3 1.62 ± 2% perf-profile.calltrace.cycles-pp.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write
2.25 -1.0 1.29 ± 2% perf-profile.calltrace.cycles-pp.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write
2.17 -0.9 1.25 ± 2% perf-profile.calltrace.cycles-pp.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter
1.93 -0.8 1.14 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.vfs_read.ksys_read.do_syscall_64
1.99 -0.8 1.23 ± 2% perf-profile.calltrace.cycles-pp.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill
1.79 -0.7 1.07 ± 2% perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.filemap_read.vfs_read.ksys_read
1.34 -0.7 0.68 ± 3% perf-profile.calltrace.cycles-pp.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter
1.30 -0.6 0.66 ± 3% perf-profile.calltrace.cycles-pp.ext4_da_map_blocks.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write
1.22 -0.6 0.65 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.llseek
1.17 -0.5 0.63 ± 2% perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.vfs_read.ksys_read.do_syscall_64
1.21 -0.5 0.67 ± 2% perf-profile.calltrace.cycles-pp.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write
0.96 -0.5 0.43 ± 47% perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.vfs_read.ksys_read
1.08 -0.5 0.57 ± 3% perf-profile.calltrace.cycles-pp.clear_bhb_loop.write
1.08 -0.5 0.57 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek
1.20 -0.5 0.70 ± 3% perf-profile.calltrace.cycles-pp.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter
1.05 -0.5 0.56 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write
0.91 -0.5 0.42 ± 47% perf-profile.calltrace.cycles-pp.__folio_mark_dirty.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end
1.15 -0.5 0.68 ± 2% perf-profile.calltrace.cycles-pp.memset_orig.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write
1.02 -0.5 0.55 ± 2% perf-profile.calltrace.cycles-pp.clear_bhb_loop.llseek
1.30 -0.5 0.83 ± 2% perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict
0.98 -0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.clear_bhb_loop.read
1.10 ± 3% -0.4 0.67 ± 3% perf-profile.calltrace.cycles-pp.workingset_activation.folio_mark_accessed.filemap_read.vfs_read.ksys_read
30.65 -0.4 30.25 perf-profile.calltrace.cycles-pp.read
0.98 -0.3 0.69 ± 2% perf-profile.calltrace.cycles-pp.__filemap_add_folio.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write
2.39 +0.2 2.61 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release
2.42 +0.2 2.64 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode
2.39 +0.2 2.61 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range
2.42 +0.2 2.64 perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict
2.38 +0.2 2.61 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu
2.44 +0.4 2.89 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict
2.38 +0.5 2.84 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode
2.38 +0.5 2.84 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range
2.38 +0.5 2.84 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release
28.52 +0.6 29.11 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
28.38 +0.7 29.04 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
4.86 +0.7 5.54 perf-profile.calltrace.cycles-pp.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill
27.89 +0.9 28.77 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
27.47 +1.1 28.54 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
25.90 +1.9 27.78 perf-profile.calltrace.cycles-pp.filemap_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.26 +4.0 34.31 perf-profile.calltrace.cycles-pp.write
21.06 +4.1 25.14 perf-profile.calltrace.cycles-pp.folio_mark_accessed.filemap_read.vfs_read.ksys_read.do_syscall_64
19.70 +4.6 24.33 perf-profile.calltrace.cycles-pp.folio_activate.folio_mark_accessed.filemap_read.vfs_read.ksys_read
19.63 +4.7 24.29 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read.vfs_read
18.85 +4.7 23.52 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read
18.84 +4.7 23.51 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed
18.83 +4.7 23.51 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate
28.38 +5.0 33.33 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
28.23 +5.0 33.25 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
27.72 +5.3 32.99 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
27.25 +5.5 32.74 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
25.87 +5.6 31.48 perf-profile.calltrace.cycles-pp.__close
25.86 +5.6 31.46 perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close
25.86 +5.6 31.47 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close
25.86 +5.6 31.47 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__close
25.84 +5.6 31.45 perf-profile.calltrace.cycles-pp.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close
25.82 +5.6 31.44 perf-profile.calltrace.cycles-pp.dput.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe
25.81 +5.6 31.44 perf-profile.calltrace.cycles-pp.__dentry_kill.dput.__fput.__x64_sys_close.do_syscall_64
25.79 +5.6 31.43 perf-profile.calltrace.cycles-pp.evict.__dentry_kill.dput.__fput.__x64_sys_close
25.78 +5.6 31.42 perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.__dentry_kill.dput.__fput
25.63 +5.7 31.33 perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill.dput
17.98 +6.1 24.07 perf-profile.calltrace.cycles-pp.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill
17.57 +6.2 23.81 perf-profile.calltrace.cycles-pp.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict
17.33 +6.3 23.66 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range
17.34 +6.3 23.66 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode
17.32 +6.3 23.65 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs
25.47 +6.5 31.92 perf-profile.calltrace.cycles-pp.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
23.72 ± 2% +7.3 31.06 perf-profile.calltrace.cycles-pp.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64
17.89 ± 3% +10.1 27.95 perf-profile.calltrace.cycles-pp.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write
13.68 ± 4% +12.1 25.73 perf-profile.calltrace.cycles-pp.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write
11.76 ± 5% +13.0 24.76 perf-profile.calltrace.cycles-pp.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter
9.80 ± 6% +13.4 23.16 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio
9.81 ± 6% +13.4 23.17 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio
9.79 ± 6% +13.4 23.16 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru
10.38 ± 6% +13.5 23.85 perf-profile.calltrace.cycles-pp.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write
10.32 ± 6% +13.5 23.82 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin
8.04 ± 8% -7.2 0.84 ± 9% perf-profile.children.cycles-pp.down_write
7.67 ± 8% -7.0 0.65 ± 12% perf-profile.children.cycles-pp.rwsem_down_write_slowpath
7.66 ± 8% -7.0 0.64 ± 12% perf-profile.children.cycles-pp.rwsem_optimistic_spin
7.28 ± 9% -6.8 0.53 ± 14% perf-profile.children.cycles-pp.osq_lock
4.35 ± 10% -4.1 0.30 ± 8% perf-profile.children.cycles-pp.unlink
4.34 ± 10% -4.0 0.29 ± 8% perf-profile.children.cycles-pp.__x64_sys_unlink
4.34 ± 10% -4.0 0.29 ± 8% perf-profile.children.cycles-pp.do_unlinkat
3.97 ± 6% -3.3 0.71 ± 8% perf-profile.children.cycles-pp.do_sys_openat2
3.95 ± 6% -3.2 0.70 ± 8% perf-profile.children.cycles-pp.path_openat
3.95 ± 6% -3.2 0.70 ± 8% perf-profile.children.cycles-pp.do_filp_open
3.93 ± 6% -3.2 0.69 ± 8% perf-profile.children.cycles-pp.creat64
3.92 ± 6% -3.2 0.68 ± 8% perf-profile.children.cycles-pp.__x64_sys_creat
3.87 ± 6% -3.2 0.66 ± 8% perf-profile.children.cycles-pp.open_last_lookups
3.70 -1.7 1.98 ± 2% perf-profile.children.cycles-pp.llseek
3.68 -1.7 2.00 ± 2% perf-profile.children.cycles-pp.ext4_block_write_begin
3.12 -1.4 1.67 ± 2% perf-profile.children.cycles-pp.clear_bhb_loop
2.90 -1.3 1.64 ± 2% perf-profile.children.cycles-pp.ext4_da_write_end
2.29 -1.0 1.31 ± 2% perf-profile.children.cycles-pp.block_write_end
2.20 -0.9 1.26 ± 2% perf-profile.children.cycles-pp.__block_commit_write
1.95 -0.8 1.15 ± 2% perf-profile.children.cycles-pp.copy_page_to_iter
1.99 -0.8 1.24 ± 2% perf-profile.children.cycles-pp.truncate_cleanup_folio
1.80 -0.7 1.08 ± 2% perf-profile.children.cycles-pp._copy_to_iter
1.54 -0.7 0.82 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64
1.34 -0.7 0.68 ± 3% perf-profile.children.cycles-pp.ext4_da_get_block_prep
1.32 -0.7 0.67 ± 3% perf-profile.children.cycles-pp.ext4_da_map_blocks
31.04 -0.6 30.46 perf-profile.children.cycles-pp.read
1.19 -0.6 0.64 ± 2% perf-profile.children.cycles-pp.filemap_get_pages
1.18 -0.5 0.63 ± 2% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
1.22 -0.5 0.68 ± 2% perf-profile.children.cycles-pp.mark_buffer_dirty
1.20 -0.5 0.70 ± 2% perf-profile.children.cycles-pp.zero_user_segments
1.20 -0.5 0.71 ± 2% perf-profile.children.cycles-pp.memset_orig
1.07 -0.5 0.58 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.95 -0.5 0.46 ± 3% perf-profile.children.cycles-pp.rw_verify_area
1.32 -0.5 0.84 ± 2% perf-profile.children.cycles-pp.try_to_free_buffers
0.98 -0.5 0.53 ± 2% perf-profile.children.cycles-pp.filemap_get_read_batch
0.98 ± 2% -0.4 0.53 ± 3% perf-profile.children.cycles-pp.__fdget_pos
0.77 ± 2% -0.4 0.34 ± 3% perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited_flags
1.10 ± 3% -0.4 0.67 ± 3% perf-profile.children.cycles-pp.workingset_activation
0.80 -0.4 0.37 ± 2% perf-profile.children.cycles-pp.file_modified
0.86 ± 4% -0.4 0.46 ± 3% perf-profile.children.cycles-pp.workingset_age_nonresident
0.77 ± 2% -0.4 0.37 ± 3% perf-profile.children.cycles-pp.security_file_permission
0.92 -0.4 0.52 ± 2% perf-profile.children.cycles-pp.__folio_mark_dirty
0.80 -0.4 0.42 ± 2% perf-profile.children.cycles-pp.xas_load
0.74 -0.4 0.37 ± 3% perf-profile.children.cycles-pp.folio_alloc_noprof
0.70 -0.4 0.33 ± 3% perf-profile.children.cycles-pp.touch_atime
0.60 ± 2% -0.4 0.24 ± 4% perf-profile.children.cycles-pp.percpu_counter_add_batch
0.71 -0.4 0.35 ± 3% perf-profile.children.cycles-pp.alloc_pages_mpol_noprof
0.80 -0.4 0.45 ± 2% perf-profile.children.cycles-pp.create_empty_buffers
0.68 ± 2% -0.3 0.36 ± 3% perf-profile.children.cycles-pp.fault_in_iov_iter_readable
0.62 -0.3 0.30 ± 3% perf-profile.children.cycles-pp.__alloc_pages_noprof
0.59 ± 2% -0.3 0.28 ± 3% perf-profile.children.cycles-pp.apparmor_file_permission
0.62 -0.3 0.32 ± 3% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state
0.64 -0.3 0.34 ± 3% perf-profile.children.cycles-pp.ksys_lseek
1.01 -0.3 0.71 ± 2% perf-profile.children.cycles-pp.__filemap_add_folio
0.62 -0.3 0.32 ± 3% perf-profile.children.cycles-pp.filemap_get_entry
0.57 -0.3 0.28 ± 3% perf-profile.children.cycles-pp.atime_needs_update
0.69 -0.3 0.40 ± 2% perf-profile.children.cycles-pp.folio_account_dirtied
0.61 -0.3 0.32 ± 2% perf-profile.children.cycles-pp.__cond_resched
0.60 ± 2% -0.3 0.32 ± 3% perf-profile.children.cycles-pp.fault_in_readable
0.62 -0.3 0.35 ± 2% perf-profile.children.cycles-pp.folio_alloc_buffers
0.58 -0.3 0.32 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.37 ± 3% -0.3 0.11 ± 4% perf-profile.children.cycles-pp.rwsem_spin_on_owner
0.40 ± 6% -0.3 0.14 ± 3% perf-profile.children.cycles-pp.ext4_file_write_iter
0.51 ± 3% -0.2 0.27 ± 3% perf-profile.children.cycles-pp.disk_rr
0.59 -0.2 0.34 ± 2% perf-profile.children.cycles-pp.kmem_cache_free
0.46 -0.2 0.22 ± 2% perf-profile.children.cycles-pp.get_page_from_freelist
0.55 ± 2% -0.2 0.31 ± 3% perf-profile.children.cycles-pp.alloc_buffer_head
0.53 -0.2 0.30 ± 2% perf-profile.children.cycles-pp.kmem_cache_alloc_noprof
0.36 ± 3% -0.2 0.14 ± 4% perf-profile.children.cycles-pp.__mark_inode_dirty
0.36 ± 6% -0.2 0.14 ± 5% perf-profile.children.cycles-pp.ext4_file_read_iter
0.40 -0.2 0.20 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.44 -0.2 0.25 ± 3% perf-profile.children.cycles-pp.xas_store
0.39 -0.2 0.19 ± 3% perf-profile.children.cycles-pp.inode_needs_update_time
0.52 -0.2 0.33 ± 2% perf-profile.children.cycles-pp.delete_from_page_cache_batch
0.29 ± 4% -0.2 0.10 ± 6% perf-profile.children.cycles-pp.generic_update_time
0.36 ± 2% -0.2 0.18 ± 4% perf-profile.children.cycles-pp.ext4_da_reserve_space
0.57 ± 2% -0.2 0.40 ± 3% perf-profile.children.cycles-pp.__folio_cancel_dirty
0.34 ± 2% -0.2 0.18 ± 3% perf-profile.children.cycles-pp.__mem_cgroup_charge
0.25 ± 4% -0.2 0.09 ± 5% perf-profile.children.cycles-pp.ext4_dirty_inode
0.33 -0.2 0.17 ± 2% perf-profile.children.cycles-pp.ext4_es_insert_delayed_block
0.34 -0.2 0.18 ± 2% perf-profile.children.cycles-pp.ext4_generic_write_checks
0.22 ± 5% -0.2 0.06 ± 9% perf-profile.children.cycles-pp.jbd2__journal_start
0.21 ± 5% -0.2 0.06 ± 8% perf-profile.children.cycles-pp.start_this_handle
0.31 -0.1 0.17 ± 3% perf-profile.children.cycles-pp._raw_spin_lock
0.40 -0.1 0.25 ± 2% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.30 ± 2% -0.1 0.17 ± 3% perf-profile.children.cycles-pp.__memcg_slab_post_alloc_hook
0.31 ± 2% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.block_invalidate_folio
0.29 ± 2% -0.1 0.16 ± 2% perf-profile.children.cycles-pp.x64_sys_call
0.49 -0.1 0.36 ± 3% perf-profile.children.cycles-pp.folio_account_cleaned
0.32 -0.1 0.19 ± 2% perf-profile.children.cycles-pp.lookup_open
0.27 -0.1 0.14 ± 2% perf-profile.children.cycles-pp.generic_write_checks
0.26 ± 3% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.rcu_all_qs
0.26 ± 2% -0.1 0.13 ± 4% perf-profile.children.cycles-pp.ext4_es_lookup_extent
0.27 ± 2% -0.1 0.14 ± 4% perf-profile.children.cycles-pp.up_write
0.27 -0.1 0.14 ± 3% perf-profile.children.cycles-pp.xas_start
0.35 ± 2% -0.1 0.22 perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.17 ± 4% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.ext4_nonda_switch
0.23 -0.1 0.12 ± 3% perf-profile.children.cycles-pp.folio_unlock
0.22 ± 2% -0.1 0.12 ± 3% perf-profile.children.cycles-pp.current_time
0.27 -0.1 0.16 ± 3% perf-profile.children.cycles-pp.__memcg_slab_free_hook
0.27 ± 2% -0.1 0.17 ± 3% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.18 -0.1 0.08 ± 5% perf-profile.children.cycles-pp.node_dirty_ok
0.21 ± 2% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.__slab_free
0.26 ± 3% -0.1 0.16 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt
0.93 ± 2% -0.1 0.84 ± 3% perf-profile.children.cycles-pp.__lruvec_stat_mod_folio
0.18 ± 2% -0.1 0.09 ± 3% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare
0.19 ± 2% -0.1 0.10 ± 5% perf-profile.children.cycles-pp.aa_file_perm
0.20 -0.1 0.11 ± 2% perf-profile.children.cycles-pp.__mod_node_page_state
0.21 -0.1 0.12 ± 2% perf-profile.children.cycles-pp.ext4_create
0.18 -0.1 0.09 ± 4% perf-profile.children.cycles-pp.rmqueue
0.20 ± 2% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.find_lock_entries
0.17 ± 5% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.19 ± 2% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.cgroup_rstat_updated
0.17 -0.1 0.09 ± 4% perf-profile.children.cycles-pp.__dquot_alloc_space
0.21 ± 2% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.17 -0.1 0.09 ± 4% perf-profile.children.cycles-pp.free_unref_folios
0.20 ± 3% -0.1 0.13 ± 3% perf-profile.children.cycles-pp.tick_nohz_handler
0.16 ± 2% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.vfs_unlink
0.19 -0.1 0.12 ± 4% perf-profile.children.cycles-pp.mod_objcg_state
0.14 ± 3% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__es_insert_extent
0.16 ± 3% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.__ext4_unlink
0.14 ± 4% -0.1 0.07 perf-profile.children.cycles-pp.__count_memcg_events
0.14 ± 2% -0.1 0.07 perf-profile.children.cycles-pp.__radix_tree_lookup
0.16 ± 3% -0.1 0.09 ± 5% perf-profile.children.cycles-pp.ext4_unlink
0.18 ± 2% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.update_process_times
0.15 ± 3% -0.1 0.08 ± 3% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers
0.12 ± 5% -0.1 0.05 ± 8% perf-profile.children.cycles-pp.ext4_claim_free_clusters
0.13 -0.1 0.07 ± 7% perf-profile.children.cycles-pp.generic_write_check_limits
0.12 ± 4% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.__xa_set_mark
0.12 ± 2% -0.1 0.06 perf-profile.children.cycles-pp.file_remove_privs_flags
0.13 ± 3% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.ext4_llseek
0.15 ± 2% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.__ext4_mark_inode_dirty
0.12 -0.1 0.06 ± 6% perf-profile.children.cycles-pp.amd_clear_divider
0.12 ± 2% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head
0.11 ± 5% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.inode_to_bdi
0.09 ± 5% -0.1 0.04 ± 47% perf-profile.children.cycles-pp.handle_softirqs
0.50 -0.1 0.45 ± 3% perf-profile.children.cycles-pp.folio_activate_fn
0.11 ± 2% -0.1 0.06 perf-profile.children.cycles-pp.__ext4_new_inode
0.12 ± 2% -0.1 0.07 perf-profile.children.cycles-pp.try_charge_memcg
0.11 ± 4% -0.1 0.06 ± 4% perf-profile.children.cycles-pp.timestamp_truncate
0.12 ± 3% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.12 ± 3% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.drop_buffers
0.13 ± 3% -0.0 0.08 ± 3% perf-profile.children.cycles-pp.__ext4_find_entry
0.11 ± 3% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.ext4_mark_iloc_dirty
0.12 ± 8% -0.0 0.08 ± 6% perf-profile.children.cycles-pp.mem_cgroup_update_lru_size
0.12 -0.0 0.08 ± 5% perf-profile.children.cycles-pp.ext4_dx_find_entry
0.13 ± 3% -0.0 0.09 perf-profile.children.cycles-pp.sched_tick
0.10 ± 4% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.ext4_do_update_inode
0.09 -0.0 0.05 perf-profile.children.cycles-pp.__mem_cgroup_uncharge_folios
0.08 ± 4% -0.0 0.04 ± 47% perf-profile.children.cycles-pp.ext4_reserve_inode_write
0.17 -0.0 0.13 ± 3% perf-profile.children.cycles-pp.filemap_unaccount_folio
0.09 ± 4% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.ext4_lookup
0.09 ± 4% -0.0 0.06 perf-profile.children.cycles-pp.task_tick_fair
0.09 ± 3% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.ext4_add_nondir
0.24 ± 2% -0.0 0.22 ± 2% perf-profile.children.cycles-pp.xas_find_conflict
0.08 ± 4% -0.0 0.05 perf-profile.children.cycles-pp.ext4_add_entry
0.08 ± 5% -0.0 0.05 perf-profile.children.cycles-pp.ext4_dx_add_entry
0.13 ± 3% -0.0 0.11 ± 5% perf-profile.children.cycles-pp.__mod_lruvec_state
0.07 -0.0 0.05 ± 31% perf-profile.children.cycles-pp.ext4_search_dir
0.34 ± 2% +0.0 0.37 perf-profile.children.cycles-pp.lru_add_fn
0.00 +0.1 0.05 ± 5% perf-profile.children.cycles-pp.lru_add_drain
0.08 ± 5% +0.1 0.15 ± 4% perf-profile.children.cycles-pp.__cmd_record
0.08 ± 5% +0.1 0.15 ± 4% perf-profile.children.cycles-pp.cmd_record
0.08 ± 4% +0.1 0.14 ± 4% perf-profile.children.cycles-pp.record__mmap_read_evlist
0.09 ± 3% +0.1 0.15 ± 4% perf-profile.children.cycles-pp.main
0.09 ± 3% +0.1 0.15 ± 4% perf-profile.children.cycles-pp.run_builtin
0.08 ± 6% +0.1 0.14 ± 4% perf-profile.children.cycles-pp.perf_mmap__push
0.07 ± 4% +0.1 0.13 ± 3% perf-profile.children.cycles-pp.record__pushfn
0.07 ± 4% +0.1 0.13 ± 3% perf-profile.children.cycles-pp.writen
0.06 +0.1 0.13 ± 5% perf-profile.children.cycles-pp.shmem_file_write_iter
0.00 +0.1 0.10 ± 4% perf-profile.children.cycles-pp.shmem_alloc_and_add_folio
0.00 +0.1 0.11 ± 6% perf-profile.children.cycles-pp.shmem_get_folio_gfp
0.00 +0.1 0.11 ± 6% perf-profile.children.cycles-pp.shmem_write_begin
2.45 +0.2 2.70 perf-profile.children.cycles-pp.lru_add_drain_cpu
4.86 +0.7 5.54 perf-profile.children.cycles-pp.__folio_batch_release
27.93 +0.9 28.79 perf-profile.children.cycles-pp.ksys_read
27.50 +1.1 28.56 perf-profile.children.cycles-pp.vfs_read
25.97 +1.8 27.81 perf-profile.children.cycles-pp.filemap_read
92.83 +3.3 96.09 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
92.42 +3.5 95.88 perf-profile.children.cycles-pp.do_syscall_64
30.73 +3.9 34.66 perf-profile.children.cycles-pp.write
21.08 +4.1 25.15 perf-profile.children.cycles-pp.folio_mark_accessed
19.70 +4.6 24.33 perf-profile.children.cycles-pp.folio_activate
27.83 +5.3 33.14 perf-profile.children.cycles-pp.ksys_write
27.36 +5.5 32.89 perf-profile.children.cycles-pp.vfs_write
25.87 +5.6 31.48 perf-profile.children.cycles-pp.__close
25.86 +5.6 31.46 perf-profile.children.cycles-pp.__x64_sys_close
25.84 +5.6 31.46 perf-profile.children.cycles-pp.__fput
25.83 +5.6 31.45 perf-profile.children.cycles-pp.dput
25.82 +5.6 31.44 perf-profile.children.cycles-pp.__dentry_kill
25.79 +5.6 31.43 perf-profile.children.cycles-pp.evict
25.78 +5.6 31.42 perf-profile.children.cycles-pp.ext4_evict_inode
25.64 +5.7 31.34 perf-profile.children.cycles-pp.truncate_inode_pages_range
18.31 +6.0 24.35 perf-profile.children.cycles-pp.folios_put_refs
17.78 +6.3 24.04 perf-profile.children.cycles-pp.__page_cache_release
25.54 +6.4 31.96 perf-profile.children.cycles-pp.ext4_buffered_write_iter
23.87 ± 2% +7.4 31.23 perf-profile.children.cycles-pp.generic_perform_write
17.93 ± 3% +10.0 27.97 perf-profile.children.cycles-pp.ext4_da_write_begin
13.75 ± 4% +12.0 25.77 perf-profile.children.cycles-pp.__filemap_get_folio
11.78 ± 5% +13.0 24.77 perf-profile.children.cycles-pp.filemap_add_folio
10.41 ± 6% +13.5 23.96 perf-profile.children.cycles-pp.folio_add_lru
34.89 ± 2% +18.9 53.82 perf-profile.children.cycles-pp.folio_batch_move_lru
51.11 +25.1 76.23 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
51.02 +25.2 76.18 perf-profile.children.cycles-pp.folio_lruvec_lock_irqsave
50.98 +25.3 76.24 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
7.25 ± 9% -6.7 0.53 ± 14% perf-profile.self.cycles-pp.osq_lock
3.09 -1.4 1.65 ± 2% perf-profile.self.cycles-pp.clear_bhb_loop
1.79 -0.7 1.07 ± 2% perf-profile.self.cycles-pp._copy_to_iter
1.14 -0.5 0.61 ± 2% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
1.20 -0.5 0.70 ± 2% perf-profile.self.cycles-pp.memset_orig
1.06 -0.5 0.57 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter_atomic
0.97 -0.5 0.51 ± 2% perf-profile.self.cycles-pp.filemap_read
0.94 ± 2% -0.4 0.51 ± 3% perf-profile.self.cycles-pp.__fdget_pos
0.86 ± 4% -0.4 0.45 ± 3% perf-profile.self.cycles-pp.workingset_age_nonresident
0.75 ± 2% -0.4 0.38 ± 3% perf-profile.self.cycles-pp.vfs_write
0.89 -0.3 0.55 ± 2% perf-profile.self.cycles-pp.__block_commit_write
0.54 ± 3% -0.3 0.21 ± 4% perf-profile.self.cycles-pp.percpu_counter_add_batch
0.50 ± 3% -0.3 0.20 ± 3% perf-profile.self.cycles-pp.balance_dirty_pages_ratelimited_flags
0.63 -0.3 0.33 ± 2% perf-profile.self.cycles-pp.vfs_read
0.57 -0.3 0.29 ± 2% perf-profile.self.cycles-pp.do_syscall_64
0.61 -0.3 0.33 ± 2% perf-profile.self.cycles-pp.filemap_get_read_batch
0.58 ± 2% -0.3 0.31 ± 3% perf-profile.self.cycles-pp.fault_in_readable
0.37 ± 3% -0.3 0.10 ± 4% perf-profile.self.cycles-pp.rwsem_spin_on_owner
0.56 -0.3 0.29 ± 3% perf-profile.self.cycles-pp.xas_load
0.38 ± 6% -0.3 0.13 ± 4% perf-profile.self.cycles-pp.ext4_file_write_iter
0.48 ± 2% -0.2 0.24 ± 4% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state
0.35 ± 5% -0.2 0.13 ± 5% perf-profile.self.cycles-pp.ext4_file_read_iter
0.46 ± 2% -0.2 0.24 ± 2% perf-profile.self.cycles-pp.write
0.45 ± 2% -0.2 0.23 ± 6% perf-profile.self.cycles-pp.disk_rr
0.39 ± 3% -0.2 0.18 ± 4% perf-profile.self.cycles-pp.apparmor_file_permission
0.36 ± 4% -0.2 0.15 ± 4% perf-profile.self.cycles-pp.ext4_da_write_begin
0.44 -0.2 0.23 ± 2% perf-profile.self.cycles-pp.ext4_da_write_end
0.44 -0.2 0.23 ± 2% perf-profile.self.cycles-pp.__filemap_get_folio
0.42 -0.2 0.21 ± 2% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.42 ± 2% -0.2 0.22 ± 3% perf-profile.self.cycles-pp.generic_perform_write
0.40 -0.2 0.20 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.43 ± 2% -0.2 0.24 ± 3% perf-profile.self.cycles-pp.read
0.40 ± 2% -0.2 0.21 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.41 -0.2 0.22 ± 3% perf-profile.self.cycles-pp.llseek
0.33 -0.2 0.17 ± 2% perf-profile.self.cycles-pp.ext4_block_write_begin
0.33 ± 2% -0.2 0.17 ± 3% perf-profile.self.cycles-pp.__cond_resched
0.28 ± 2% -0.2 0.13 ± 3% perf-profile.self.cycles-pp.atime_needs_update
0.30 ± 2% -0.1 0.16 ± 3% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.30 -0.1 0.16 ± 4% perf-profile.self.cycles-pp.filemap_get_entry
0.30 -0.1 0.16 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.28 -0.1 0.14 ± 4% perf-profile.self.cycles-pp.folio_mark_accessed
0.27 -0.1 0.14 ± 2% perf-profile.self.cycles-pp.mark_buffer_dirty
0.26 -0.1 0.14 ± 3% perf-profile.self.cycles-pp.ext4_da_map_blocks
0.25 -0.1 0.13 ± 3% perf-profile.self.cycles-pp.down_write
0.25 -0.1 0.14 ± 2% perf-profile.self.cycles-pp.x64_sys_call
0.27 ± 2% -0.1 0.16 ± 2% perf-profile.self.cycles-pp.block_invalidate_folio
0.24 ± 2% -0.1 0.12 ± 3% perf-profile.self.cycles-pp.up_write
0.16 ± 4% -0.1 0.06 ± 7% perf-profile.self.cycles-pp.ext4_nonda_switch
0.20 ± 2% -0.1 0.10 perf-profile.self.cycles-pp.inode_needs_update_time
0.21 ± 2% -0.1 0.11 perf-profile.self.cycles-pp.folio_unlock
0.23 -0.1 0.13 ± 3% perf-profile.self.cycles-pp.xas_store
0.21 ± 2% -0.1 0.11 perf-profile.self.cycles-pp.xas_start
0.20 ± 2% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.filemap_get_pages
0.20 ± 2% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.security_file_permission
0.20 ± 4% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.rcu_all_qs
0.22 -0.1 0.12 ± 3% perf-profile.self.cycles-pp.folios_put_refs
0.20 ± 2% -0.1 0.11 ± 3% perf-profile.self.cycles-pp.__slab_free
0.18 -0.1 0.09 ± 3% perf-profile.self.cycles-pp.__filemap_add_folio
0.17 ± 2% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.ext4_buffered_write_iter
0.18 ± 2% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.rw_verify_area
0.17 -0.1 0.09 ± 4% perf-profile.self.cycles-pp.__mod_node_page_state
0.16 ± 3% -0.1 0.08 ± 3% perf-profile.self.cycles-pp.aa_file_perm
0.15 -0.1 0.07 ± 6% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare
0.16 ± 2% -0.1 0.09 ± 3% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.15 ± 3% -0.1 0.08 ± 4% perf-profile.self.cycles-pp.current_time
0.15 ± 3% -0.1 0.08 ± 3% perf-profile.self.cycles-pp.ksys_write
0.15 ± 6% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.15 ± 2% -0.1 0.08 perf-profile.self.cycles-pp.generic_write_checks
0.15 ± 4% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.__memcg_slab_post_alloc_hook
0.15 ± 3% -0.1 0.08 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter
0.15 ± 3% -0.1 0.08 ± 4% perf-profile.self.cycles-pp.ksys_read
0.18 ± 2% -0.1 0.11 ± 3% perf-profile.self.cycles-pp.mod_objcg_state
0.13 -0.1 0.06 ± 7% perf-profile.self.cycles-pp.__radix_tree_lookup
0.13 ± 3% -0.1 0.06 ± 4% perf-profile.self.cycles-pp.__alloc_pages_noprof
0.15 ± 2% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.find_lock_entries
0.14 -0.1 0.08 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.13 ± 5% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.kmem_cache_alloc_noprof
0.12 ± 2% -0.1 0.06 perf-profile.self.cycles-pp.node_dirty_ok
0.12 -0.1 0.06 ± 6% perf-profile.self.cycles-pp.ksys_lseek
0.09 -0.1 0.03 ± 75% perf-profile.self.cycles-pp.ext4_es_lookup_extent
0.12 -0.1 0.06 ± 7% perf-profile.self.cycles-pp.create_empty_buffers
0.12 ± 4% -0.1 0.06 perf-profile.self.cycles-pp.__dquot_alloc_space
0.10 ± 3% -0.1 0.05 ± 31% perf-profile.self.cycles-pp.folio_account_dirtied
0.12 ± 4% -0.1 0.06 ± 4% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head
0.09 -0.1 0.04 ± 61% perf-profile.self.cycles-pp.file_modified
0.11 ± 4% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.ext4_llseek
0.10 ± 4% -0.1 0.05 perf-profile.self.cycles-pp.__count_memcg_events
0.11 ± 4% -0.1 0.06 ± 8% perf-profile.self.cycles-pp.generic_write_check_limits
0.10 ± 5% -0.1 0.05 perf-profile.self.cycles-pp.block_write_end
0.10 ± 5% -0.1 0.05 perf-profile.self.cycles-pp.file_remove_privs_flags
0.10 -0.1 0.05 perf-profile.self.cycles-pp.get_page_from_freelist
0.12 ± 4% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.12 -0.0 0.07 ± 5% perf-profile.self.cycles-pp.drop_buffers
0.10 ± 5% -0.0 0.05 perf-profile.self.cycles-pp.timestamp_truncate
0.10 ± 5% -0.0 0.05 ± 5% perf-profile.self.cycles-pp.folio_account_cleaned
0.10 ± 4% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.kmem_cache_free
0.11 ± 9% -0.0 0.07 ± 8% perf-profile.self.cycles-pp.mem_cgroup_update_lru_size
0.10 ± 4% -0.0 0.07 perf-profile.self.cycles-pp.__memcg_slab_free_hook
0.09 ± 5% -0.0 0.06 ± 4% perf-profile.self.cycles-pp.try_charge_memcg
0.10 ± 4% -0.0 0.07 ± 5% perf-profile.self.cycles-pp.__page_cache_release
0.23 ± 3% -0.0 0.21 ± 2% perf-profile.self.cycles-pp.xas_find_conflict
0.25 ± 2% +0.0 0.28 ± 4% perf-profile.self.cycles-pp.folio_activate_fn
0.17 ± 3% +0.1 0.26 perf-profile.self.cycles-pp.lru_add_fn
0.51 ± 2% +0.1 0.63 ± 3% perf-profile.self.cycles-pp.__lruvec_stat_mod_folio
0.30 ± 5% +0.2 0.51 ± 2% perf-profile.self.cycles-pp.folio_batch_move_lru
50.97 +25.3 76.24 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-12 6:04 [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression kernel test robot @ 2024-07-12 19:03 ` Roman Gushchin 2024-07-15 14:14 ` Oliver Sang 0 siblings, 1 reply; 12+ messages in thread From: Roman Gushchin @ 2024-07-12 19:03 UTC (permalink / raw) To: kernel test robot Cc: oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > Hello, > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master Hello, thank you for the report! I'd expect that the regression should be fixed by the commit "mm: memcg: add cache line padding to mem_cgroup_per_node". Can you, please, confirm that it's not the case? Thank you! ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-12 19:03 ` Roman Gushchin @ 2024-07-15 14:14 ` Oliver Sang 2024-07-15 22:18 ` Roman Gushchin 0 siblings, 1 reply; 12+ messages in thread From: Oliver Sang @ 2024-07-15 14:14 UTC (permalink / raw) To: Roman Gushchin Cc: oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin, oliver.sang hi, Roman Gushchin, On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > Hello, > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > Hello, > > thank you for the report! > > I'd expect that the regression should be fixed by the commit > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > Can you, please, confirm that it's not the case? > > Thank you! in our this aim7 test, we found the performance partially recovered by "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase: gcc-13/performance/1BRD_48G/ext4/x86_64-rhel-8.3/3000/debian-12-x86_64-20240206.cgz/lkp-icl-2sp2/disk_rr/aim7 commit: 94b7e5bf09 ("mm: memcg: put memcg1-specific struct mem_cgroup's members under CONFIG_MEMCG_V1") 98c9daf5ae ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") 9fa001cf3b ("mm: memcg: drop obsolete cache line padding in struct mem_cgroup") 6df13230b6 ("mm: memcg: add cache line padding to mem_cgroup_per_node") 94b7e5bf09b08aa4 98c9daf5ae6be008f78c07b744b 9fa001cf3bb0598aad09d15b289 6df13230b612af81ce04f20bb37 ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 76.42 +15.1% 87.99 ± 3% +15.4% 88.18 ± 3% +3.1% 78.77 uptime.boot 53.28 ± 2% -27.5% 38.65 ± 2% -27.0% 38.88 ± 3% -9.1% 48.45 ± 2% iostat.cpu.idle 44.58 ± 2% +33.6% 59.58 +33.1% 59.35 ± 2% +11.2% 49.57 ± 2% iostat.cpu.system 2.13 -17.0% 1.77 -17.0% 1.77 -7.1% 1.98 iostat.cpu.user 49.85 ± 2% -14.7 35.17 ± 2% -14.2 35.64 ± 4% -5.2 44.66 ± 3% mpstat.cpu.all.idle% 0.15 ± 7% +0.0 0.15 ± 3% -0.0 0.15 ± 3% -0.0 0.13 ± 4% mpstat.cpu.all.irq% 47.74 ± 2% +15.1 62.82 +14.6 62.36 ± 2% +5.4 53.14 ± 2% mpstat.cpu.all.sys% 2.23 -0.4 1.82 -0.4 1.82 -0.2 2.04 mpstat.cpu.all.usr% 4752 ± 13% -18.1% 3890 ± 11% +3.2% 4905 ± 18% -9.0% 4322 ± 19% sched_debug.cpu.avg_idle.min 0.00 ± 32% -49.2% 0.00 ±171% -43.2% 0.00 ± 81% -62.3% 0.00 ± 99% sched_debug.rt_rq:.rt_time.avg 0.19 ± 32% -49.2% 0.10 ±171% -43.2% 0.11 ± 81% -62.3% 0.07 ± 99% sched_debug.rt_rq:.rt_time.max 0.02 ± 32% -49.2% 0.01 ±171% -43.2% 0.01 ± 81% -62.3% 0.01 ± 99% sched_debug.rt_rq:.rt_time.stddev 53.29 ± 2% -27.4% 38.70 ± 2% -27.0% 38.92 ± 3% -9.4% 48.28 ± 2% vmstat.cpu.id 65.83 ± 4% +59.1% 104.76 ± 3% +55.5% 102.35 ± 3% +19.0% 78.35 ± 3% vmstat.procs.r 8385 ± 4% +34.6% 11284 ± 3% +33.9% 11228 ± 5% +20.5% 10101 ± 8% vmstat.system.cs 245966 ± 2% +8.1% 265964 +8.1% 265934 ± 2% +19.3% 293498 ± 2% vmstat.system.in 778685 -29.4% 549435 -29.4% 549581 -10.3% 698378 aim7.jobs-per-min 23.31 +41.4% 32.96 +41.3% 32.94 +11.4% 25.96 aim7.time.elapsed_time 23.31 +41.4% 32.96 +41.3% 32.94 +11.4% 25.96 aim7.time.elapsed_time.max 47890 ± 7% +338.5% 210000 ± 3% +343.2% 212238 ± 4% +117.1% 103949 ± 7% aim7.time.involuntary_context_switches 6674 +26.7% 8455 +26.6% 8447 +10.5% 7372 aim7.time.percent_of_cpu_this_job_got 1510 +81.7% 2744 ± 2% +81.4% 2740 +23.8% 1869 aim7.time.system_time 19454 ± 3% +9.1% 21223 ± 3% +8.3% 21069 ± 3% -3.5% 18782 ± 4% aim7.time.voluntary_context_switches 23.31 +41.4% 32.96 +41.3% 32.94 +11.4% 25.96 time.elapsed_time 23.31 +41.4% 32.96 +41.3% 32.94 +11.4% 25.96 time.elapsed_time.max 47890 ± 7% +338.5% 210000 ± 3% +343.2% 212238 ± 4% +117.1% 103949 ± 7% time.involuntary_context_switches 6674 +26.7% 8455 +26.6% 8447 +10.5% 7372 time.percent_of_cpu_this_job_got 1510 +81.7% 2744 ± 2% +81.4% 2740 +23.8% 1869 time.system_time 45.72 -4.9% 43.47 -4.7% 43.58 -1.7% 44.96 time.user_time 19454 ± 3% +9.1% 21223 ± 3% +8.3% 21069 ± 3% -3.5% 18782 ± 4% time.voluntary_context_switches 49345 ± 6% +108.4% 102820 ± 10% +97.2% 97294 ± 5% +32.3% 65261 ± 5% meminfo.Active 22670 ± 11% +182.6% 64065 ± 15% +157.0% 58258 ± 7% +57.3% 35667 ± 11% meminfo.Active(anon) 26674 ± 4% +45.3% 38754 ± 4% +46.3% 39035 ± 6% +10.9% 29594 ± 5% meminfo.Active(file) 33695 +18.6% 39973 ± 3% +18.7% 40010 ± 3% +3.8% 34971 meminfo.AnonHugePages 561012 ± 8% +14.6% 642770 ± 5% +18.1% 662691 ± 2% -2.1% 548999 ± 7% meminfo.Dirty 1360098 ± 3% +14.0% 1551107 ± 2% +16.2% 1580651 +1.6% 1381539 ± 3% meminfo.Inactive 803759 +15.0% 924519 +16.2% 933975 +4.7% 841839 meminfo.Inactive(anon) 66977 ± 3% +115.7% 144485 ± 7% +114.7% 143809 ± 5% +21.0% 81015 ± 3% meminfo.Mapped 78152 ± 9% +188.4% 225431 ± 5% +192.1% 228311 ± 3% +64.5% 128567 ± 5% meminfo.Shmem 15327 ± 8% +52.6% 23389 ± 11% +52.3% 23342 ± 7% +29.5% 19853 ± 14% numa-meminfo.node0.Active 13455 ± 9% +46.0% 19642 ± 10% +44.0% 19375 ± 8% +18.7% 15975 ± 13% numa-meminfo.node0.Active(file) 19657 ± 14% +111.2% 41510 ± 18% +132.9% 45776 ± 24% +76.5% 34690 ± 37% numa-meminfo.node0.Mapped 6081 ± 16% +84.9% 11247 ± 42% +108.9% 12705 ± 44% +38.8% 8442 ± 13% numa-meminfo.node0.Shmem 33647 ± 8% +134.3% 78825 ± 15% +115.1% 72392 ± 4% +35.8% 45703 ± 9% numa-meminfo.node1.Active 20790 ± 14% +188.7% 60020 ± 18% +155.6% 53149 ± 8% +52.8% 31760 ± 14% numa-meminfo.node1.Active(anon) 12857 ± 7% +46.3% 18805 ± 9% +49.7% 19243 ± 7% +8.4% 13942 ± 8% numa-meminfo.node1.Active(file) 285023 ± 5% +10.9% 316178 ± 5% +15.2% 328281 ± 6% -6.1% 267619 ± 11% numa-meminfo.node1.Dirty 485228 ± 23% +85.5% 899983 ± 21% +90.5% 924573 ± 26% +85.5% 900160 ± 25% numa-meminfo.node1.Inactive 202089 ± 50% +192.9% 591917 ± 32% +198.6% 603355 ± 41% +215.8% 638153 ± 34% numa-meminfo.node1.Inactive(anon) 283138 ± 6% +8.8% 308066 ± 5% +13.4% 321217 ± 6% -7.5% 262007 ± 11% numa-meminfo.node1.Inactive(file) 47991 ± 7% +112.0% 101755 ± 8% +93.7% 92947 ± 18% -2.6% 46748 ± 32% numa-meminfo.node1.Mapped 72431 ± 8% +194.1% 213055 ± 4% +191.3% 210966 ± 2% +65.2% 119653 ± 5% numa-meminfo.node1.Shmem 467.96 ± 53% +99.5% 933.35 ± 29% +112.2% 993.04 ± 29% +107.2% 969.49 ± 31% numa-vmstat.node0.nr_active_anon 3290 ± 10% +50.9% 4965 ± 10% +48.6% 4889 ± 6% +20.2% 3956 ± 7% numa-vmstat.node0.nr_active_file 4793 ± 17% +113.8% 10249 ± 20% +142.6% 11627 ± 25% +84.1% 8827 ± 36% numa-vmstat.node0.nr_mapped 1519 ± 16% +72.6% 2622 ± 33% +109.2% 3179 ± 44% +38.8% 2110 ± 13% numa-vmstat.node0.nr_shmem 467.96 ± 53% +99.4% 933.32 ± 29% +112.2% 993.04 ± 29% +107.2% 969.49 ± 31% numa-vmstat.node0.nr_zone_active_anon 3294 ± 9% +50.9% 4970 ± 10% +48.4% 4889 ± 6% +20.4% 3967 ± 7% numa-vmstat.node0.nr_zone_active_file 4955 ± 11% +202.3% 14980 ± 18% +173.1% 13536 ± 7% +61.8% 8017 ± 16% numa-vmstat.node1.nr_active_anon 3376 ± 8% +40.1% 4729 ± 7% +40.5% 4742 ± 6% +7.0% 3611 ± 13% numa-vmstat.node1.nr_active_file 72003 ± 5% +9.4% 78780 ± 5% +12.1% 80713 ± 5% -7.1% 66888 ± 12% numa-vmstat.node1.nr_dirty 50152 ± 51% +194.6% 147731 ± 32% +201.7% 151300 ± 41% +218.6% 159767 ± 34% numa-vmstat.node1.nr_inactive_anon 12101 ± 7% +109.5% 25357 ± 8% +98.5% 24026 ± 16% -1.6% 11908 ± 33% numa-vmstat.node1.nr_mapped 17201 ± 7% +208.3% 53026 ± 4% +211.7% 53610 ± 2% +75.7% 30214 ± 8% numa-vmstat.node1.nr_shmem 4955 ± 11% +202.3% 14980 ± 18% +173.1% 13536 ± 7% +61.8% 8017 ± 16% numa-vmstat.node1.nr_zone_active_anon 3352 ± 8% +41.1% 4729 ± 7% +41.8% 4753 ± 6% +7.8% 3612 ± 13% numa-vmstat.node1.nr_zone_active_file 50150 ± 51% +194.6% 147730 ± 32% +201.7% 151301 ± 41% +218.6% 159766 ± 34% numa-vmstat.node1.nr_zone_inactive_anon 72002 ± 5% +9.4% 78787 ± 5% +12.1% 80711 ± 5% -7.1% 66896 ± 12% numa-vmstat.node1.nr_zone_write_pending 5776 ± 11% +175.1% 15893 ± 16% +147.7% 14310 ± 6% +55.0% 8953 ± 11% proc-vmstat.nr_active_anon 6595 ± 8% +46.7% 9677 ± 3% +47.0% 9695 ± 4% +12.6% 7424 ± 5% proc-vmstat.nr_active_file 187950 +2.0% 191760 +2.2% 192015 +0.1% 188193 proc-vmstat.nr_anon_pages 139601 ± 8% +15.0% 160546 ± 5% +18.9% 165920 ± 2% -2.1% 136650 ± 8% proc-vmstat.nr_dirty 945653 +6.0% 1002646 +6.6% 1008161 +1.0% 955008 proc-vmstat.nr_file_pages 201118 +14.8% 230842 +15.8% 232804 +4.7% 210479 proc-vmstat.nr_inactive_anon 17214 ± 3% +108.2% 35836 ± 6% +103.7% 35070 ± 6% +20.0% 20660 ± 3% proc-vmstat.nr_mapped 19951 ± 8% +180.4% 55933 ± 4% +180.9% 56052 ± 4% +61.5% 32218 ± 5% proc-vmstat.nr_shmem 40267 +2.7% 41335 +3.2% 41567 -0.7% 39991 proc-vmstat.nr_slab_reclaimable 86277 +1.8% 87792 +1.7% 87706 +0.1% 86326 proc-vmstat.nr_slab_unreclaimable 5776 ± 11% +175.1% 15893 ± 16% +147.7% 14310 ± 6% +55.0% 8953 ± 11% proc-vmstat.nr_zone_active_anon 6595 ± 8% +46.7% 9677 ± 3% +47.0% 9695 ± 4% +12.6% 7424 ± 5% proc-vmstat.nr_zone_active_file 201118 +14.8% 230842 +15.8% 232804 +4.7% 210479 proc-vmstat.nr_zone_inactive_anon 139600 ± 8% +15.0% 160546 ± 5% +18.9% 165919 ± 2% -2.1% 136650 ± 8% proc-vmstat.nr_zone_write_pending 312.12 ±241% +1618.1% 5362 ±125% +1397.7% 4674 ±166% +521.3% 1939 ±171% proc-vmstat.numa_pages_migrated 369792 +19.6% 442285 ± 3% +16.6% 431187 ± 3% +2.9% 380386 proc-vmstat.pgfault 312.12 ±241% +1618.1% 5362 ±125% +1397.7% 4674 ±166% +521.3% 1939 ±171% proc-vmstat.pgmigrate_success 2426 ± 2% +29.2% 3135 ± 3% +31.4% 3189 ± 2% +16.3% 2821 ± 3% proc-vmstat.pgpgout 1515 +3.8% 1572 +3.4% 1566 +1.3% 1535 proc-vmstat.unevictable_pgs_culled 0.63 ± 3% +36.5% 0.85 ± 2% +35.9% 0.85 ± 3% +19.4% 0.75 ± 2% perf-stat.i.MPKI 1.885e+10 -23.7% 1.437e+10 -22.8% 1.455e+10 -7.4% 1.745e+10 ± 2% perf-stat.i.branch-instructions 2.78 ± 2% -0.5 2.29 ± 4% -0.5 2.31 ± 5% -0.2 2.55 ± 4% perf-stat.i.branch-miss-rate% 67232899 -17.4% 55553782 ± 3% -17.1% 55749070 ± 3% -5.8% 63348903 perf-stat.i.branch-misses 13.70 ± 3% +2.5 16.17 ± 3% +2.5 16.24 ± 2% +1.1 14.84 ± 2% perf-stat.i.cache-miss-rate% 72591570 ± 2% +0.6% 73035167 +1.8% 73898780 ± 3% +10.8% 80408900 ± 3% perf-stat.i.cache-misses 5.483e+08 -24.9% 4.118e+08 -23.7% 4.181e+08 ± 2% -5.7% 5.17e+08 ± 2% perf-stat.i.cache-references 8605 ± 4% +34.7% 11593 ± 4% +34.2% 11552 ± 4% +23.4% 10618 ± 7% perf-stat.i.context-switches 1.35 +65.4% 2.22 ± 3% +64.8% 2.22 ± 2% +20.9% 1.63 perf-stat.i.cpi 1.616e+11 +22.1% 1.973e+11 +23.4% 1.994e+11 +10.5% 1.785e+11 ± 2% perf-stat.i.cpu-cycles 2387 ± 4% +13.8% 2717 ± 4% +12.0% 2673 ± 3% +0.6% 2402 ± 3% perf-stat.i.cycles-between-cache-misses 8.537e+10 -24.8% 6.416e+10 -23.9% 6.494e+10 -8.0% 7.852e+10 ± 2% perf-stat.i.instructions 0.96 -22.2% 0.75 ± 2% -21.7% 0.75 ± 3% -9.4% 0.87 perf-stat.i.ipc 13455 -15.4% 11379 ± 3% -15.5% 11364 ± 2% -7.6% 12438 perf-stat.i.minor-faults 13489 -15.5% 11396 ± 3% -15.6% 11378 ± 2% -7.6% 12465 perf-stat.i.page-faults 0.85 ± 3% +34.8% 1.15 +34.7% 1.15 +20.2% 1.02 ± 3% perf-stat.overall.MPKI 0.34 +0.0 0.36 ± 2% +0.0 0.36 ± 4% +0.0 0.35 perf-stat.overall.branch-miss-rate% 13.23 ± 3% +4.6 17.78 +4.5 17.71 +2.3 15.52 ± 2% perf-stat.overall.cache-miss-rate% 1.90 +62.8% 3.08 +62.5% 3.08 +20.1% 2.28 perf-stat.overall.cpi 2231 ± 3% +20.6% 2690 ± 2% +20.6% 2689 ± 2% -0.2% 2227 ± 2% perf-stat.overall.cycles-between-cache-misses 0.53 -38.5% 0.32 -38.5% 0.32 -16.7% 0.44 perf-stat.overall.ipc 1.86e+10 -23.0% 1.433e+10 -22.3% 1.446e+10 ± 3% -7.6% 1.719e+10 perf-stat.ps.branch-instructions 64095803 -19.1% 51843320 ± 2% -18.9% 52013238 ± 3% -5.7% 60421424 perf-stat.ps.branch-misses 71639076 ± 2% +2.3% 73304464 ± 2% +3.1% 73894713 ± 4% +10.4% 79092736 ± 3% perf-stat.ps.cache-misses 5.417e+08 -23.9% 4.124e+08 -23.0% 4.171e+08 ± 3% -5.9% 5.096e+08 ± 2% perf-stat.ps.cache-references 8220 ± 4% +33.3% 10962 ± 4% +33.2% 10952 ± 4% +23.9% 10184 ± 8% perf-stat.ps.context-switches 1.597e+11 +23.5% 1.972e+11 ± 2% +24.3% 1.986e+11 ± 3% +10.2% 1.761e+11 ± 2% perf-stat.ps.cpu-cycles 8.426e+10 -24.1% 6.394e+10 -23.5% 6.449e+10 ± 3% -8.2% 7.737e+10 ± 2% perf-stat.ps.instructions 12730 -18.7% 10354 ± 3% -18.3% 10405 ± 3% -7.2% 11811 perf-stat.ps.minor-faults 12762 -18.8% 10369 ± 3% -18.4% 10417 ± 3% -7.3% 11836 perf-stat.ps.page-faults 4.35 ± 10% -4.4 0.00 -4.4 0.00 -3.5 0.81 ± 20% perf-profile.calltrace.cycles-pp.unlink 4.35 ± 10% -4.3 0.00 -4.3 0.00 -3.5 0.81 ± 19% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.35 ± 10% -4.3 0.00 -4.3 0.00 -3.5 0.81 ± 19% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink 4.34 ± 10% -4.3 0.00 -4.3 0.00 -3.5 0.80 ± 19% perf-profile.calltrace.cycles-pp.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.34 ± 10% -4.3 0.00 -4.3 0.00 -3.5 0.80 ± 20% perf-profile.calltrace.cycles-pp.do_unlinkat.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.14 ± 10% -4.1 0.00 -4.1 0.00 -3.5 0.65 ± 24% perf-profile.calltrace.cycles-pp.down_write.do_unlinkat.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.14 ± 10% -4.1 0.00 -4.1 0.00 -3.5 0.65 ± 24% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.do_unlinkat.__x64_sys_unlink.do_syscall_64 4.14 ± 10% -4.1 0.00 -4.1 0.00 -3.6 0.58 ± 45% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.do_unlinkat.__x64_sys_unlink 3.99 ± 11% -4.0 0.00 -4.0 0.00 -3.6 0.40 ± 83% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.do_unlinkat 3.53 ± 6% -3.4 0.10 ±212% -3.5 0.07 ±264% -2.4 1.12 ± 17% perf-profile.calltrace.cycles-pp.down_write.open_last_lookups.path_openat.do_filp_open.do_sys_openat2 3.53 ± 6% -3.4 0.10 ±212% -3.5 0.07 ±264% -2.4 1.12 ± 17% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.open_last_lookups.path_openat.do_filp_open 3.52 ± 6% -3.4 0.10 ±212% -3.5 0.07 ±264% -2.4 1.12 ± 17% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.open_last_lookups.path_openat 3.30 ± 7% -3.3 0.00 -3.3 0.00 -2.3 0.98 ± 19% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.open_last_lookups 3.93 ± 6% -3.2 0.69 ± 8% -3.3 0.65 ± 9% -2.5 1.44 ± 13% perf-profile.calltrace.cycles-pp.creat64 3.93 ± 6% -3.2 0.69 ± 8% -3.3 0.64 ± 10% -2.5 1.44 ± 13% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.93 ± 6% -3.2 0.69 ± 8% -3.3 0.64 ± 10% -2.5 1.44 ± 13% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.creat64 3.92 ± 6% -3.2 0.68 ± 8% -3.3 0.64 ± 10% -2.5 1.43 ± 13% perf-profile.calltrace.cycles-pp.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.92 ± 6% -3.2 0.68 ± 8% -3.3 0.64 ± 10% -2.5 1.43 ± 13% perf-profile.calltrace.cycles-pp.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.91 ± 6% -3.2 0.68 ± 8% -3.3 0.63 ± 10% -2.5 1.42 ± 13% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64 3.91 ± 6% -3.2 0.68 ± 8% -3.3 0.63 ± 9% -2.5 1.42 ± 13% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.87 ± 6% -3.2 0.65 ± 9% -3.3 0.61 ± 10% -2.5 1.39 ± 14% perf-profile.calltrace.cycles-pp.open_last_lookups.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat 3.66 -1.7 1.99 ± 2% -1.6 2.02 -0.6 3.06 perf-profile.calltrace.cycles-pp.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write 3.31 -1.5 1.77 ± 2% -1.5 1.78 -0.6 2.73 perf-profile.calltrace.cycles-pp.llseek 2.87 -1.3 1.62 ± 2% -1.2 1.63 -0.5 2.33 perf-profile.calltrace.cycles-pp.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 2.25 -1.0 1.29 ± 2% -1.0 1.30 -0.4 1.83 perf-profile.calltrace.cycles-pp.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write 2.17 -0.9 1.25 ± 2% -0.9 1.26 -0.4 1.76 perf-profile.calltrace.cycles-pp.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter 0.86 ± 4% -0.9 0.00 -0.9 0.00 -0.2 0.62 ± 4% perf-profile.calltrace.cycles-pp.workingset_age_nonresident.workingset_activation.folio_mark_accessed.filemap_read.vfs_read 0.79 -0.8 0.00 -0.8 0.00 -0.1 0.74 perf-profile.calltrace.cycles-pp.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.93 -0.8 1.14 ± 2% -0.8 1.14 -0.3 1.66 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.vfs_read.ksys_read.do_syscall_64 0.76 -0.8 0.00 -0.8 0.00 -0.2 0.58 perf-profile.calltrace.cycles-pp.file_modified.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64 1.99 -0.8 1.23 ± 2% -0.8 1.24 -0.4 1.63 perf-profile.calltrace.cycles-pp.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 0.74 -0.7 0.00 -0.7 0.00 -0.2 0.58 perf-profile.calltrace.cycles-pp.folio_alloc_noprof.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.79 -0.7 1.07 ± 2% -0.7 1.07 -0.2 1.55 perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.filemap_read.vfs_read.ksys_read 0.72 ± 2% -0.7 0.00 -0.7 0.00 -0.2 0.54 ± 4% perf-profile.calltrace.cycles-pp.balance_dirty_pages_ratelimited_flags.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 0.69 -0.7 0.00 -0.7 0.00 -0.2 0.54 perf-profile.calltrace.cycles-pp.alloc_pages_mpol_noprof.folio_alloc_noprof.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 0.68 -0.7 0.00 -0.7 0.00 -0.2 0.53 perf-profile.calltrace.cycles-pp.touch_atime.filemap_read.vfs_read.ksys_read.do_syscall_64 1.34 -0.7 0.68 ± 3% -0.7 0.67 -0.3 1.05 perf-profile.calltrace.cycles-pp.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 0.65 ± 2% -0.7 0.00 -0.7 0.00 -0.1 0.52 perf-profile.calltrace.cycles-pp.fault_in_iov_iter_readable.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 1.30 -0.6 0.66 ± 3% -0.6 0.65 -0.3 1.02 perf-profile.calltrace.cycles-pp.ext4_da_map_blocks.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 1.22 -0.6 0.65 ± 2% -0.6 0.65 -0.2 0.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.llseek 1.17 -0.5 0.63 ± 2% -0.5 0.65 -0.2 1.00 perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.vfs_read.ksys_read.do_syscall_64 1.21 -0.5 0.67 ± 2% -0.5 0.67 ± 2% -0.3 0.91 perf-profile.calltrace.cycles-pp.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write 0.96 -0.5 0.43 ± 47% -0.4 0.54 -0.1 0.83 perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.vfs_read.ksys_read 1.08 -0.5 0.57 ± 3% -0.5 0.56 -0.2 0.86 perf-profile.calltrace.cycles-pp.clear_bhb_loop.write 1.08 -0.5 0.57 ± 2% -0.5 0.57 -0.2 0.88 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek 1.20 -0.5 0.70 ± 3% -0.5 0.70 -0.2 1.02 perf-profile.calltrace.cycles-pp.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.05 -0.5 0.56 ± 2% -0.5 0.56 -0.2 0.85 perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 0.91 -0.5 0.42 ± 47% -0.4 0.52 ± 2% -0.2 0.67 perf-profile.calltrace.cycles-pp.__folio_mark_dirty.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end 1.15 -0.5 0.68 ± 2% -0.5 0.68 -0.2 0.99 perf-profile.calltrace.cycles-pp.memset_orig.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 1.02 -0.5 0.55 ± 2% -0.5 0.56 -0.2 0.86 perf-profile.calltrace.cycles-pp.clear_bhb_loop.llseek 1.30 -0.5 0.83 ± 2% -0.5 0.83 -0.2 1.05 perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict 0.98 -0.5 0.53 ± 2% -0.4 0.54 -0.2 0.82 perf-profile.calltrace.cycles-pp.clear_bhb_loop.read 1.10 ± 3% -0.4 0.67 ± 3% -0.4 0.66 ± 2% -0.3 0.80 ± 3% perf-profile.calltrace.cycles-pp.workingset_activation.folio_mark_accessed.filemap_read.vfs_read.ksys_read 30.65 -0.4 30.25 -0.4 30.21 +0.7 31.31 perf-profile.calltrace.cycles-pp.read 0.98 -0.3 0.69 ± 2% -1.0 0.00 -0.4 0.61 perf-profile.calltrace.cycles-pp.__filemap_add_folio.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 2.39 +0.2 2.61 +0.2 2.62 +0.1 2.47 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release 2.42 +0.2 2.64 +0.2 2.65 +0.1 2.50 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode 2.39 +0.2 2.61 +0.2 2.62 +0.1 2.48 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range 2.42 +0.2 2.64 +0.2 2.65 +0.1 2.50 perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict 2.38 +0.2 2.61 +0.2 2.62 +0.1 2.47 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu 2.44 +0.4 2.89 +0.5 2.91 +0.2 2.62 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict 2.38 +0.5 2.84 +0.5 2.86 +0.2 2.56 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode 2.38 +0.5 2.84 +0.5 2.86 +0.2 2.56 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range 2.38 +0.5 2.84 +0.5 2.85 +0.2 2.56 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release 28.52 +0.6 29.11 +0.5 29.06 +1.0 29.54 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 28.38 +0.7 29.04 +0.6 28.99 +1.0 29.42 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 4.86 +0.7 5.54 +0.7 5.56 +0.3 5.13 perf-profile.calltrace.cycles-pp.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 27.89 +0.9 28.77 +0.8 28.72 +1.1 29.02 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 27.47 +1.1 28.54 +1.0 28.49 +1.2 28.67 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 25.90 +1.9 27.78 +1.8 27.72 +1.5 27.42 perf-profile.calltrace.cycles-pp.filemap_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.26 +4.0 34.31 +3.9 34.16 +3.9 34.16 perf-profile.calltrace.cycles-pp.write 21.06 +4.1 25.14 +4.0 25.05 +2.3 23.39 perf-profile.calltrace.cycles-pp.folio_mark_accessed.filemap_read.vfs_read.ksys_read.do_syscall_64 19.70 +4.6 24.33 +4.6 24.25 +2.7 22.38 perf-profile.calltrace.cycles-pp.folio_activate.folio_mark_accessed.filemap_read.vfs_read.ksys_read 19.63 +4.7 24.29 +4.6 24.21 +2.7 22.32 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read.vfs_read 18.85 +4.7 23.52 +4.6 23.44 +2.7 21.52 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read 18.84 +4.7 23.51 +4.6 23.44 +2.7 21.52 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed 18.83 +4.7 23.51 +4.6 23.43 +2.7 21.51 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate 28.38 +5.0 33.33 +4.8 33.17 +4.3 32.65 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 28.23 +5.0 33.25 +4.9 33.09 +4.3 32.53 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 27.72 +5.3 32.99 +5.1 32.83 +4.4 32.13 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 27.25 +5.5 32.74 +5.3 32.58 +4.5 31.75 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 25.87 +5.6 31.48 +5.8 31.70 +2.2 28.03 perf-profile.calltrace.cycles-pp.__close 25.86 +5.6 31.47 +5.8 31.70 +2.2 28.02 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.86 +5.6 31.47 +5.8 31.70 +2.2 28.02 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__close 25.86 +5.6 31.46 +5.8 31.69 +2.2 28.01 perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.84 +5.6 31.45 +5.8 31.68 +2.2 28.00 perf-profile.calltrace.cycles-pp.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.82 +5.6 31.44 +5.8 31.67 +2.2 27.99 perf-profile.calltrace.cycles-pp.dput.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 25.81 +5.6 31.44 +5.9 31.67 +2.2 27.98 perf-profile.calltrace.cycles-pp.__dentry_kill.dput.__fput.__x64_sys_close.do_syscall_64 25.79 +5.6 31.43 +5.9 31.66 +2.2 27.96 perf-profile.calltrace.cycles-pp.evict.__dentry_kill.dput.__fput.__x64_sys_close 25.78 +5.6 31.42 +5.9 31.65 +2.2 27.95 perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.__dentry_kill.dput.__fput 25.63 +5.7 31.33 +5.9 31.56 +2.2 27.83 perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill.dput 17.98 +6.1 24.07 +6.3 24.28 +2.4 20.42 perf-profile.calltrace.cycles-pp.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 17.57 +6.2 23.81 +6.4 24.02 +2.5 20.06 perf-profile.calltrace.cycles-pp.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict 17.33 +6.3 23.66 +6.5 23.87 +2.5 19.86 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range 17.34 +6.3 23.66 +6.5 23.87 +2.5 19.86 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode 17.32 +6.3 23.65 +6.5 23.86 +2.5 19.85 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs 25.47 +6.5 31.92 +6.3 31.77 +4.9 30.41 perf-profile.calltrace.cycles-pp.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 23.72 ± 2% +7.3 31.06 +7.2 30.91 +5.3 29.06 perf-profile.calltrace.cycles-pp.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64 17.89 ± 3% +10.1 27.95 +9.9 27.79 +6.5 24.39 perf-profile.calltrace.cycles-pp.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 13.68 ± 4% +12.1 25.73 +11.9 25.55 +7.3 20.95 perf-profile.calltrace.cycles-pp.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write 11.76 ± 5% +13.0 24.76 +12.8 24.57 +7.7 19.42 perf-profile.calltrace.cycles-pp.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 9.80 ± 6% +13.4 23.16 +13.4 23.18 +8.1 17.87 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio 9.81 ± 6% +13.4 23.17 +13.4 23.18 +8.1 17.87 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio 9.79 ± 6% +13.4 23.16 +13.4 23.18 +8.1 17.86 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru 10.38 ± 6% +13.5 23.85 +13.5 23.87 +8.1 18.48 perf-profile.calltrace.cycles-pp.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 10.32 ± 6% +13.5 23.82 +13.5 23.84 +8.1 18.43 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin 8.04 ± 8% -7.2 0.84 ± 9% -7.2 0.80 ± 10% -6.0 2.06 ± 17% perf-profile.children.cycles-pp.down_write 7.67 ± 8% -7.0 0.65 ± 12% -7.1 0.62 ± 14% -5.9 1.77 ± 19% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 7.66 ± 8% -7.0 0.64 ± 12% -7.0 0.61 ± 14% -5.9 1.76 ± 19% perf-profile.children.cycles-pp.rwsem_optimistic_spin 7.28 ± 9% -6.8 0.53 ± 14% -6.8 0.50 ± 16% -5.7 1.54 ± 21% perf-profile.children.cycles-pp.osq_lock 4.35 ± 10% -4.1 0.30 ± 8% -4.0 0.31 ± 10% -3.5 0.81 ± 19% perf-profile.children.cycles-pp.unlink 4.34 ± 10% -4.0 0.29 ± 8% -4.0 0.30 ± 9% -3.5 0.80 ± 19% perf-profile.children.cycles-pp.__x64_sys_unlink 4.34 ± 10% -4.0 0.29 ± 8% -4.0 0.30 ± 9% -3.5 0.80 ± 20% perf-profile.children.cycles-pp.do_unlinkat 3.97 ± 6% -3.3 0.71 ± 8% -3.3 0.67 ± 9% -2.5 1.47 ± 13% perf-profile.children.cycles-pp.do_sys_openat2 3.95 ± 6% -3.2 0.70 ± 8% -3.3 0.66 ± 9% -2.5 1.45 ± 13% perf-profile.children.cycles-pp.path_openat 3.95 ± 6% -3.2 0.70 ± 8% -3.3 0.66 ± 9% -2.5 1.45 ± 13% perf-profile.children.cycles-pp.do_filp_open 3.93 ± 6% -3.2 0.69 ± 8% -3.3 0.65 ± 9% -2.5 1.44 ± 13% perf-profile.children.cycles-pp.creat64 3.92 ± 6% -3.2 0.68 ± 8% -3.3 0.64 ± 10% -2.5 1.43 ± 13% perf-profile.children.cycles-pp.__x64_sys_creat 3.87 ± 6% -3.2 0.66 ± 8% -3.3 0.61 ± 10% -2.5 1.39 ± 14% perf-profile.children.cycles-pp.open_last_lookups 3.70 -1.7 1.98 ± 2% -1.7 1.99 -0.7 3.05 perf-profile.children.cycles-pp.llseek 3.68 -1.7 2.00 ± 2% -1.6 2.03 -0.6 3.08 perf-profile.children.cycles-pp.ext4_block_write_begin 3.12 -1.4 1.67 ± 2% -1.4 1.69 -0.5 2.57 perf-profile.children.cycles-pp.clear_bhb_loop 2.90 -1.3 1.64 ± 2% -1.2 1.65 -0.5 2.36 perf-profile.children.cycles-pp.ext4_da_write_end 2.29 -1.0 1.31 ± 2% -1.0 1.32 -0.4 1.85 perf-profile.children.cycles-pp.block_write_end 2.20 -0.9 1.26 ± 2% -0.9 1.27 -0.4 1.78 perf-profile.children.cycles-pp.__block_commit_write 1.95 -0.8 1.15 ± 2% -0.8 1.15 -0.3 1.68 perf-profile.children.cycles-pp.copy_page_to_iter 1.99 -0.8 1.24 ± 2% -0.8 1.24 -0.4 1.64 perf-profile.children.cycles-pp.truncate_cleanup_folio 1.80 -0.7 1.08 ± 2% -0.7 1.08 -0.2 1.56 perf-profile.children.cycles-pp._copy_to_iter 1.54 -0.7 0.82 ± 2% -0.7 0.82 -0.3 1.27 perf-profile.children.cycles-pp.entry_SYSCALL_64 1.34 -0.7 0.68 ± 3% -0.7 0.68 -0.3 1.06 perf-profile.children.cycles-pp.ext4_da_get_block_prep 1.32 -0.7 0.67 ± 3% -0.7 0.66 -0.3 1.04 perf-profile.children.cycles-pp.ext4_da_map_blocks 31.04 -0.6 30.46 -0.6 30.42 +0.6 31.63 perf-profile.children.cycles-pp.read 1.19 -0.6 0.64 ± 2% -0.5 0.66 -0.2 1.01 perf-profile.children.cycles-pp.filemap_get_pages 1.18 -0.5 0.63 ± 2% -0.5 0.64 -0.2 0.97 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 1.22 -0.5 0.68 ± 2% -0.5 0.68 -0.3 0.92 perf-profile.children.cycles-pp.mark_buffer_dirty 1.20 -0.5 0.70 ± 2% -0.5 0.70 -0.2 1.02 perf-profile.children.cycles-pp.zero_user_segments 1.20 -0.5 0.71 ± 2% -0.5 0.71 -0.2 1.03 perf-profile.children.cycles-pp.memset_orig 1.07 -0.5 0.58 ± 2% -0.5 0.57 -0.2 0.87 perf-profile.children.cycles-pp.copy_page_from_iter_atomic 0.95 -0.5 0.46 ± 3% -0.5 0.47 ± 3% -0.2 0.78 ± 4% perf-profile.children.cycles-pp.rw_verify_area 1.32 -0.5 0.84 ± 2% -0.5 0.84 -0.3 1.06 perf-profile.children.cycles-pp.try_to_free_buffers 0.98 -0.5 0.53 ± 2% -0.4 0.56 -0.1 0.85 perf-profile.children.cycles-pp.filemap_get_read_batch 0.98 ± 2% -0.4 0.53 ± 3% -0.4 0.53 ± 2% -0.2 0.80 perf-profile.children.cycles-pp.__fdget_pos 0.77 ± 2% -0.4 0.34 ± 3% -0.4 0.34 -0.2 0.58 ± 4% perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited_flags 1.10 ± 3% -0.4 0.67 ± 3% -0.4 0.67 ± 2% -0.3 0.80 ± 3% perf-profile.children.cycles-pp.workingset_activation 0.80 -0.4 0.37 ± 2% -0.4 0.37 -0.2 0.60 perf-profile.children.cycles-pp.file_modified 0.86 ± 4% -0.4 0.46 ± 3% -0.4 0.45 ± 2% -0.2 0.62 ± 3% perf-profile.children.cycles-pp.workingset_age_nonresident 0.77 ± 2% -0.4 0.37 ± 3% -0.4 0.38 ± 3% -0.1 0.64 ± 5% perf-profile.children.cycles-pp.security_file_permission 0.92 -0.4 0.52 ± 2% -0.4 0.52 ± 2% -0.2 0.68 perf-profile.children.cycles-pp.__folio_mark_dirty 0.80 -0.4 0.42 ± 2% -0.4 0.42 -0.1 0.65 perf-profile.children.cycles-pp.xas_load 0.74 -0.4 0.37 ± 3% -0.4 0.37 -0.2 0.58 perf-profile.children.cycles-pp.folio_alloc_noprof 0.70 -0.4 0.33 ± 3% -0.4 0.33 -0.2 0.54 perf-profile.children.cycles-pp.touch_atime 0.60 ± 2% -0.4 0.24 ± 4% -0.4 0.24 ± 3% -0.2 0.42 perf-profile.children.cycles-pp.percpu_counter_add_batch 0.71 -0.4 0.35 ± 3% -0.4 0.35 -0.2 0.55 perf-profile.children.cycles-pp.alloc_pages_mpol_noprof 0.80 -0.4 0.45 ± 2% -0.3 0.48 ± 2% -0.1 0.74 perf-profile.children.cycles-pp.create_empty_buffers 0.68 ± 2% -0.3 0.36 ± 3% -0.3 0.36 -0.1 0.54 perf-profile.children.cycles-pp.fault_in_iov_iter_readable 0.62 -0.3 0.30 ± 3% -0.3 0.30 -0.1 0.48 perf-profile.children.cycles-pp.__alloc_pages_noprof 0.59 ± 2% -0.3 0.28 ± 3% -0.3 0.28 ± 4% -0.1 0.48 ± 7% perf-profile.children.cycles-pp.apparmor_file_permission 0.62 -0.3 0.32 ± 3% -0.3 0.32 ± 2% -0.1 0.49 ± 2% perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.64 -0.3 0.34 ± 3% -0.3 0.34 -0.1 0.52 perf-profile.children.cycles-pp.ksys_lseek 1.01 -0.3 0.71 ± 2% -0.5 0.50 ± 2% -0.4 0.62 perf-profile.children.cycles-pp.__filemap_add_folio 0.62 -0.3 0.32 ± 3% -0.3 0.32 -0.1 0.51 perf-profile.children.cycles-pp.filemap_get_entry 0.57 -0.3 0.28 ± 3% -0.3 0.28 -0.1 0.45 perf-profile.children.cycles-pp.atime_needs_update 0.69 -0.3 0.40 ± 2% -0.3 0.40 ± 3% -0.2 0.50 perf-profile.children.cycles-pp.folio_account_dirtied 0.61 -0.3 0.32 ± 2% -0.3 0.33 ± 2% -0.1 0.50 ± 3% perf-profile.children.cycles-pp.__cond_resched 0.60 ± 2% -0.3 0.32 ± 3% -0.3 0.32 ± 2% -0.1 0.48 perf-profile.children.cycles-pp.fault_in_readable 0.62 -0.3 0.35 ± 2% -0.2 0.39 ± 2% -0.0 0.59 perf-profile.children.cycles-pp.folio_alloc_buffers 0.58 -0.3 0.32 ± 2% -0.3 0.32 -0.1 0.48 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.37 ± 3% -0.3 0.11 ± 4% -0.3 0.10 ± 4% -0.2 0.22 ± 6% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.40 ± 6% -0.3 0.14 ± 3% -0.3 0.13 ± 4% -0.1 0.26 ± 5% perf-profile.children.cycles-pp.ext4_file_write_iter 0.51 ± 3% -0.2 0.27 ± 3% -0.3 0.26 ± 3% -0.1 0.40 ± 2% perf-profile.children.cycles-pp.disk_rr 0.59 -0.2 0.34 ± 2% -0.2 0.34 -0.1 0.49 perf-profile.children.cycles-pp.kmem_cache_free 0.46 -0.2 0.22 ± 2% -0.2 0.22 -0.1 0.36 perf-profile.children.cycles-pp.get_page_from_freelist 0.55 ± 2% -0.2 0.31 ± 3% -0.2 0.34 -0.0 0.52 perf-profile.children.cycles-pp.alloc_buffer_head 0.53 -0.2 0.30 ± 2% -0.2 0.34 -0.0 0.51 perf-profile.children.cycles-pp.kmem_cache_alloc_noprof 0.36 ± 6% -0.2 0.14 ± 5% -0.2 0.14 ± 4% -0.1 0.25 ± 4% perf-profile.children.cycles-pp.ext4_file_read_iter 0.36 ± 3% -0.2 0.14 ± 4% -0.2 0.14 -0.1 0.25 ± 2% perf-profile.children.cycles-pp.__mark_inode_dirty 0.40 -0.2 0.20 ± 2% -0.2 0.21 ± 2% -0.1 0.33 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.44 -0.2 0.25 ± 3% -0.2 0.22 ± 2% -0.1 0.34 perf-profile.children.cycles-pp.xas_store 0.39 -0.2 0.19 ± 3% -0.2 0.19 ± 2% -0.1 0.31 ± 3% perf-profile.children.cycles-pp.inode_needs_update_time 0.52 -0.2 0.33 ± 2% -0.2 0.33 ± 2% -0.1 0.42 perf-profile.children.cycles-pp.delete_from_page_cache_batch 0.29 ± 4% -0.2 0.10 ± 6% -0.2 0.10 ± 4% -0.1 0.19 ± 3% perf-profile.children.cycles-pp.generic_update_time 0.36 ± 2% -0.2 0.18 ± 4% -0.2 0.18 ± 2% -0.1 0.28 perf-profile.children.cycles-pp.ext4_da_reserve_space 0.57 ± 2% -0.2 0.40 ± 3% -0.2 0.41 -0.1 0.44 perf-profile.children.cycles-pp.__folio_cancel_dirty 0.34 ± 2% -0.2 0.18 ± 3% -0.2 0.19 -0.1 0.29 ± 2% perf-profile.children.cycles-pp.__mem_cgroup_charge 0.25 ± 4% -0.2 0.09 ± 5% -0.2 0.09 -0.1 0.16 ± 4% perf-profile.children.cycles-pp.ext4_dirty_inode 0.33 -0.2 0.17 ± 2% -0.2 0.16 ± 2% -0.1 0.25 perf-profile.children.cycles-pp.ext4_es_insert_delayed_block 0.34 -0.2 0.18 ± 2% -0.2 0.18 ± 2% -0.1 0.27 ± 2% perf-profile.children.cycles-pp.ext4_generic_write_checks 0.22 ± 5% -0.2 0.06 ± 9% -0.2 0.06 -0.1 0.13 ± 3% perf-profile.children.cycles-pp.jbd2__journal_start 0.21 ± 5% -0.2 0.06 ± 8% -0.2 0.05 ± 8% -0.1 0.12 ± 5% perf-profile.children.cycles-pp.start_this_handle 0.31 -0.1 0.17 ± 3% -0.1 0.17 ± 2% -0.1 0.25 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 0.40 -0.1 0.25 ± 2% -0.1 0.25 -0.1 0.30 ± 2% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.30 ± 2% -0.1 0.17 ± 3% -0.1 0.20 ± 2% -0.0 0.30 ± 3% perf-profile.children.cycles-pp.__memcg_slab_post_alloc_hook 0.31 ± 2% -0.1 0.18 ± 2% -0.1 0.18 -0.1 0.26 ± 2% perf-profile.children.cycles-pp.block_invalidate_folio 0.29 ± 2% -0.1 0.16 ± 2% -0.1 0.16 ± 2% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.x64_sys_call 0.49 -0.1 0.36 ± 3% -0.1 0.36 -0.1 0.38 perf-profile.children.cycles-pp.folio_account_cleaned 0.32 -0.1 0.19 ± 2% -0.1 0.18 ± 2% -0.1 0.25 ± 2% perf-profile.children.cycles-pp.lookup_open 0.27 -0.1 0.14 ± 2% -0.1 0.14 -0.1 0.22 ± 2% perf-profile.children.cycles-pp.generic_write_checks 0.26 ± 3% -0.1 0.13 ± 3% -0.1 0.14 ± 3% -0.1 0.21 ± 3% perf-profile.children.cycles-pp.rcu_all_qs 0.26 ± 2% -0.1 0.13 ± 4% -0.1 0.13 ± 2% -0.1 0.20 perf-profile.children.cycles-pp.ext4_es_lookup_extent 0.27 ± 2% -0.1 0.14 ± 4% -0.1 0.14 -0.1 0.22 ± 2% perf-profile.children.cycles-pp.up_write 0.27 -0.1 0.14 ± 3% -0.1 0.14 ± 2% -0.0 0.22 perf-profile.children.cycles-pp.xas_start 0.35 ± 2% -0.1 0.22 -0.1 0.22 -0.1 0.27 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.17 ± 4% -0.1 0.06 ± 6% -0.1 0.06 ± 5% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.ext4_nonda_switch 0.23 -0.1 0.12 ± 3% -0.1 0.12 ± 2% -0.0 0.18 perf-profile.children.cycles-pp.folio_unlock 0.22 ± 2% -0.1 0.12 ± 3% -0.1 0.12 ± 2% -0.0 0.18 ± 2% perf-profile.children.cycles-pp.current_time 0.27 -0.1 0.16 ± 3% -0.1 0.16 ± 2% -0.0 0.23 ± 2% perf-profile.children.cycles-pp.__memcg_slab_free_hook 0.27 ± 2% -0.1 0.17 ± 3% -0.1 0.17 -0.1 0.20 ± 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.18 -0.1 0.08 ± 5% -0.1 0.08 ± 5% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.node_dirty_ok 0.21 ± 2% -0.1 0.12 ± 4% -0.1 0.11 ± 4% -0.0 0.17 perf-profile.children.cycles-pp.__slab_free 0.26 ± 3% -0.1 0.16 ± 2% -0.1 0.16 ± 2% -0.1 0.20 ± 2% perf-profile.children.cycles-pp.hrtimer_interrupt 0.93 ± 2% -0.1 0.84 ± 3% -0.1 0.83 ± 2% -0.2 0.72 ± 2% perf-profile.children.cycles-pp.__lruvec_stat_mod_folio 0.18 ± 2% -0.1 0.09 ± 3% -0.1 0.09 ± 4% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.19 ± 2% -0.1 0.10 ± 5% -0.1 0.10 ± 5% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.aa_file_perm 0.20 -0.1 0.11 ± 2% -0.1 0.11 ± 3% -0.0 0.16 ± 2% perf-profile.children.cycles-pp.__mod_node_page_state 0.21 -0.1 0.12 ± 2% -0.1 0.12 ± 2% -0.0 0.16 ± 2% perf-profile.children.cycles-pp.ext4_create 0.18 -0.1 0.09 ± 4% -0.1 0.09 -0.0 0.14 perf-profile.children.cycles-pp.rmqueue 0.20 ± 2% -0.1 0.12 ± 4% -0.1 0.12 ± 3% -0.0 0.17 perf-profile.children.cycles-pp.find_lock_entries 0.17 ± 5% -0.1 0.08 ± 5% -0.1 0.08 ± 5% -0.0 0.14 ± 7% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.19 ± 2% -0.1 0.10 ± 4% -0.1 0.10 -0.0 0.15 ± 3% perf-profile.children.cycles-pp.cgroup_rstat_updated 0.17 -0.1 0.09 ± 4% -0.1 0.09 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.__dquot_alloc_space 0.08 -0.1 0.00 -0.1 0.00 -0.0 0.06 ± 5% perf-profile.children.cycles-pp.__mod_zone_page_state 0.21 ± 2% -0.1 0.13 ± 3% -0.1 0.13 ± 2% -0.1 0.16 perf-profile.children.cycles-pp.__hrtimer_run_queues 0.08 ± 5% -0.1 0.00 -0.1 0.00 -0.0 0.06 ± 7% perf-profile.children.cycles-pp._raw_read_lock 0.08 ± 5% -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.children.cycles-pp.rcu_core 0.17 -0.1 0.09 ± 4% -0.1 0.09 ± 5% -0.0 0.14 ± 2% perf-profile.children.cycles-pp.free_unref_folios 0.08 ± 5% -0.1 0.01 ±212% -0.0 0.04 ± 37% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.get_mem_cgroup_from_mm 0.09 ± 3% -0.1 0.01 ±163% -0.1 0.01 ±173% -0.0 0.07 ± 4% perf-profile.children.cycles-pp.generic_file_llseek_size 0.08 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.06 ± 5% perf-profile.children.cycles-pp.rcu_do_batch 0.20 ± 3% -0.1 0.13 ± 3% -0.1 0.13 ± 3% -0.0 0.15 ± 2% perf-profile.children.cycles-pp.tick_nohz_handler 0.16 ± 2% -0.1 0.09 ± 4% -0.1 0.09 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.vfs_unlink 0.08 ± 4% -0.1 0.00 ±316% -0.1 0.00 -0.0 0.06 ± 5% perf-profile.children.cycles-pp.xas_create 0.09 ± 5% -0.1 0.01 ±163% -0.1 0.00 -0.0 0.07 ± 4% perf-profile.children.cycles-pp.generic_file_read_iter 0.19 -0.1 0.12 ± 4% -0.1 0.11 ± 4% -0.0 0.16 ± 2% perf-profile.children.cycles-pp.mod_objcg_state 0.14 ± 3% -0.1 0.07 ± 6% -0.1 0.07 ± 4% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__es_insert_extent 0.16 ± 3% -0.1 0.08 ± 5% -0.1 0.09 ± 5% -0.0 0.12 ± 2% perf-profile.children.cycles-pp.__ext4_unlink 0.14 ± 4% -0.1 0.07 -0.1 0.07 -0.0 0.11 ± 2% perf-profile.children.cycles-pp.__count_memcg_events 0.07 ± 4% -0.1 0.00 -0.1 0.00 -0.0 0.06 ± 5% perf-profile.children.cycles-pp.__es_remove_extent 0.07 ± 4% -0.1 0.00 -0.1 0.00 -0.0 0.06 ± 5% perf-profile.children.cycles-pp.read@plt 0.14 ± 2% -0.1 0.07 -0.1 0.07 ± 4% -0.0 0.11 perf-profile.children.cycles-pp.__radix_tree_lookup 0.16 ± 3% -0.1 0.09 ± 5% -0.1 0.09 ± 5% -0.0 0.12 ± 2% perf-profile.children.cycles-pp.ext4_unlink 0.08 ± 6% -0.1 0.00 ±316% -0.1 0.00 -0.0 0.06 ± 5% perf-profile.children.cycles-pp.xas_clear_mark 0.08 ± 5% -0.1 0.01 ±163% -0.1 0.02 ±129% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.folio_mapping 0.07 -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.children.cycles-pp.node_page_state 0.07 -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.children.cycles-pp.ext4_fill_raw_inode 0.18 ± 2% -0.1 0.12 ± 4% -0.1 0.12 ± 2% -0.0 0.14 perf-profile.children.cycles-pp.update_process_times 0.15 ± 3% -0.1 0.08 ± 3% -0.1 0.08 -0.0 0.12 ± 4% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers 0.12 ± 5% -0.1 0.05 ± 8% -0.1 0.05 ± 9% -0.0 0.09 ± 3% perf-profile.children.cycles-pp.ext4_claim_free_clusters 0.07 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.children.cycles-pp.folio_wait_stable 0.07 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.children.cycles-pp.free_unref_page_commit 0.06 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 ± 6% perf-profile.children.cycles-pp.balance_dirty_pages 0.13 -0.1 0.07 ± 7% -0.1 0.07 ± 6% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.generic_write_check_limits 0.12 ± 4% -0.1 0.06 ± 7% -0.1 0.06 ± 6% -0.0 0.10 perf-profile.children.cycles-pp.__xa_set_mark 0.12 ± 2% -0.1 0.06 -0.1 0.06 ± 5% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.file_remove_privs_flags 0.13 ± 3% -0.1 0.07 ± 7% -0.1 0.06 ± 6% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.ext4_llseek 0.07 -0.1 0.01 ±212% -0.1 0.01 ±264% -0.0 0.06 ± 9% perf-profile.children.cycles-pp.irq_exit_rcu 0.06 -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.children.cycles-pp._raw_spin_trylock 0.06 -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.children.cycles-pp.bdev_getblk 0.06 -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.children.cycles-pp.crc32c_pcl_intel_update 0.06 -0.1 0.00 -0.1 0.01 ±264% -0.0 0.04 ± 37% perf-profile.children.cycles-pp.add_dirent_to_buf 0.15 ± 2% -0.1 0.09 ± 4% -0.1 0.09 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.__ext4_mark_inode_dirty 0.12 -0.1 0.06 ± 6% -0.1 0.06 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.amd_clear_divider 0.12 ± 2% -0.1 0.06 ± 6% -0.1 0.06 -0.0 0.09 ± 5% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head 0.11 ± 5% -0.1 0.06 ± 6% -0.1 0.06 ± 7% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.inode_to_bdi 0.09 ± 5% -0.1 0.04 ± 47% -0.0 0.04 ± 37% -0.0 0.07 perf-profile.children.cycles-pp.handle_softirqs 0.50 -0.1 0.45 ± 3% -0.0 0.46 +0.0 0.55 perf-profile.children.cycles-pp.folio_activate_fn 0.11 ± 2% -0.1 0.06 -0.1 0.06 -0.0 0.09 perf-profile.children.cycles-pp.__ext4_new_inode 0.12 ± 2% -0.1 0.07 -0.1 0.07 -0.0 0.10 perf-profile.children.cycles-pp.try_charge_memcg 0.11 ± 4% -0.1 0.06 ± 4% -0.1 0.06 -0.0 0.09 ± 3% perf-profile.children.cycles-pp.timestamp_truncate 0.12 ± 3% -0.1 0.07 ± 6% -0.0 0.07 ± 6% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.12 ± 3% -0.1 0.07 ± 6% -0.1 0.07 ± 5% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.drop_buffers 0.13 ± 3% -0.0 0.08 ± 3% -0.0 0.08 ± 8% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.__ext4_find_entry 0.11 ± 3% -0.0 0.06 ± 6% -0.0 0.06 -0.0 0.09 ± 3% perf-profile.children.cycles-pp.ext4_mark_iloc_dirty 0.12 ± 8% -0.0 0.08 ± 6% -0.0 0.08 ± 7% +0.1 0.20 ± 3% perf-profile.children.cycles-pp.mem_cgroup_update_lru_size 0.12 -0.0 0.08 ± 5% -0.0 0.08 ± 7% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.ext4_dx_find_entry 0.13 ± 3% -0.0 0.09 -0.0 0.09 -0.0 0.11 ± 4% perf-profile.children.cycles-pp.sched_tick 0.10 ± 4% -0.0 0.06 ± 7% -0.0 0.06 ± 8% -0.0 0.08 ± 4% perf-profile.children.cycles-pp.ext4_do_update_inode 0.09 -0.0 0.05 -0.0 0.05 -0.0 0.07 perf-profile.children.cycles-pp.__mem_cgroup_uncharge_folios 0.08 ± 4% -0.0 0.04 ± 47% -0.0 0.04 ± 37% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.ext4_reserve_inode_write 0.17 -0.0 0.13 ± 3% -0.0 0.13 ± 2% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.filemap_unaccount_folio 0.09 ± 4% -0.0 0.05 ± 9% -0.0 0.05 ± 9% -0.0 0.07 ± 4% perf-profile.children.cycles-pp.ext4_lookup 0.09 ± 4% -0.0 0.06 -0.0 0.06 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.task_tick_fair 0.09 ± 3% -0.0 0.06 ± 8% -0.0 0.05 ± 9% -0.0 0.07 perf-profile.children.cycles-pp.ext4_add_nondir 0.24 ± 2% -0.0 0.22 ± 2% -0.2 0.00 -0.2 0.06 perf-profile.children.cycles-pp.xas_find_conflict 0.08 ± 4% -0.0 0.05 -0.0 0.04 ± 37% -0.0 0.06 perf-profile.children.cycles-pp.ext4_add_entry 0.08 ± 5% -0.0 0.05 -0.0 0.04 ± 57% -0.0 0.06 perf-profile.children.cycles-pp.ext4_dx_add_entry 0.13 ± 3% -0.0 0.11 ± 5% -0.0 0.11 -0.0 0.11 perf-profile.children.cycles-pp.__mod_lruvec_state 0.07 -0.0 0.05 ± 31% -0.0 0.04 ± 38% -0.0 0.05 ± 6% perf-profile.children.cycles-pp.ext4_search_dir 0.26 ± 3% -0.0 0.24 ± 8% -0.0 0.24 ± 2% +0.1 0.36 ± 4% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.26 ± 3% -0.0 0.24 ± 8% -0.0 0.24 ± 2% +0.1 0.36 ± 4% perf-profile.children.cycles-pp.do_group_exit 0.26 ± 3% -0.0 0.24 ± 9% -0.0 0.25 ± 4% +0.1 0.36 ± 3% perf-profile.children.cycles-pp.do_exit 0.24 ± 4% -0.0 0.24 ± 8% -0.0 0.24 ± 2% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.exit_mm 0.25 ± 3% -0.0 0.25 ± 8% +0.0 0.25 ± 2% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.__mmput 0.25 ± 3% -0.0 0.25 ± 8% +0.0 0.25 ± 2% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.exit_mmap 0.20 ± 3% +0.0 0.22 ± 9% +0.0 0.22 ± 3% +0.1 0.30 ± 4% perf-profile.children.cycles-pp.__tlb_batch_free_encoded_pages 0.20 ± 3% +0.0 0.22 ± 9% +0.0 0.22 ± 3% +0.1 0.30 ± 4% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.20 ± 3% +0.0 0.22 ± 9% +0.0 0.22 ± 3% +0.1 0.31 ± 4% perf-profile.children.cycles-pp.tlb_finish_mmu 0.34 ± 2% +0.0 0.37 +0.0 0.36 -0.0 0.32 perf-profile.children.cycles-pp.lru_add_fn 0.00 +0.1 0.05 ± 5% +0.1 0.05 +0.1 0.05 perf-profile.children.cycles-pp.lru_add_drain 0.08 ± 5% +0.1 0.15 ± 4% +0.1 0.15 ± 2% +0.0 0.11 perf-profile.children.cycles-pp.__cmd_record 0.08 ± 5% +0.1 0.15 ± 4% +0.1 0.15 ± 2% +0.0 0.11 perf-profile.children.cycles-pp.cmd_record 0.08 ± 4% +0.1 0.14 ± 4% +0.1 0.14 ± 2% +0.0 0.10 ± 4% perf-profile.children.cycles-pp.record__mmap_read_evlist 0.09 ± 3% +0.1 0.15 ± 4% +0.1 0.15 ± 2% +0.0 0.11 ± 2% perf-profile.children.cycles-pp.main 0.09 ± 3% +0.1 0.15 ± 4% +0.1 0.15 ± 2% +0.0 0.11 ± 2% perf-profile.children.cycles-pp.run_builtin 0.08 ± 6% +0.1 0.14 ± 4% +0.1 0.14 ± 2% +0.0 0.10 perf-profile.children.cycles-pp.perf_mmap__push 0.07 ± 4% +0.1 0.13 ± 3% +0.1 0.13 ± 3% +0.0 0.09 ± 3% perf-profile.children.cycles-pp.record__pushfn 0.07 ± 4% +0.1 0.13 ± 3% +0.1 0.14 ± 3% +0.0 0.09 ± 3% perf-profile.children.cycles-pp.writen 0.06 +0.1 0.13 ± 5% +0.1 0.13 +0.0 0.09 ± 4% perf-profile.children.cycles-pp.shmem_file_write_iter 0.00 +0.1 0.10 ± 4% +0.1 0.11 ± 4% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.shmem_alloc_and_add_folio 0.00 +0.1 0.11 ± 6% +0.1 0.11 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.shmem_get_folio_gfp 0.00 +0.1 0.11 ± 6% +0.1 0.11 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.shmem_write_begin 2.45 +0.2 2.70 +0.3 2.70 +0.1 2.55 perf-profile.children.cycles-pp.lru_add_drain_cpu 4.86 +0.7 5.54 +0.7 5.56 +0.3 5.13 perf-profile.children.cycles-pp.__folio_batch_release 27.93 +0.9 28.79 +0.8 28.74 +1.1 29.05 perf-profile.children.cycles-pp.ksys_read 27.50 +1.1 28.56 +1.0 28.51 +1.2 28.70 perf-profile.children.cycles-pp.vfs_read 25.97 +1.8 27.81 +1.8 27.76 +1.5 27.48 perf-profile.children.cycles-pp.filemap_read 92.83 +3.3 96.09 +3.3 96.08 +1.3 94.11 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 92.42 +3.5 95.88 +3.4 95.87 +1.4 93.79 perf-profile.children.cycles-pp.do_syscall_64 30.73 +3.9 34.66 +3.8 34.50 +3.8 34.58 perf-profile.children.cycles-pp.write 21.08 +4.1 25.15 +4.0 25.07 +2.3 23.40 perf-profile.children.cycles-pp.folio_mark_accessed 19.70 +4.6 24.33 +4.6 24.25 +2.7 22.38 perf-profile.children.cycles-pp.folio_activate 27.83 +5.3 33.14 +5.2 32.98 +4.4 32.25 perf-profile.children.cycles-pp.ksys_write 27.36 +5.5 32.89 +5.4 32.73 +4.5 31.87 perf-profile.children.cycles-pp.vfs_write 25.87 +5.6 31.48 +5.8 31.71 +2.2 28.03 perf-profile.children.cycles-pp.__close 25.86 +5.6 31.46 +5.8 31.69 +2.2 28.01 perf-profile.children.cycles-pp.__x64_sys_close 25.84 +5.6 31.46 +5.8 31.68 +2.2 28.00 perf-profile.children.cycles-pp.__fput 25.83 +5.6 31.45 +5.8 31.68 +2.2 27.99 perf-profile.children.cycles-pp.dput 25.82 +5.6 31.44 +5.9 31.67 +2.2 27.98 perf-profile.children.cycles-pp.__dentry_kill 25.79 +5.6 31.43 +5.9 31.66 +2.2 27.96 perf-profile.children.cycles-pp.evict 25.78 +5.6 31.42 +5.9 31.65 +2.2 27.95 perf-profile.children.cycles-pp.ext4_evict_inode 25.64 +5.7 31.34 +5.9 31.57 +2.2 27.83 perf-profile.children.cycles-pp.truncate_inode_pages_range 18.31 +6.0 24.35 +6.3 24.56 +2.5 20.82 perf-profile.children.cycles-pp.folios_put_refs 17.78 +6.3 24.04 +6.5 24.25 +2.6 20.37 perf-profile.children.cycles-pp.__page_cache_release 25.54 +6.4 31.96 +6.3 31.81 +4.9 30.47 perf-profile.children.cycles-pp.ext4_buffered_write_iter 23.87 ± 2% +7.4 31.23 +7.2 31.09 +5.3 29.21 perf-profile.children.cycles-pp.generic_perform_write 17.93 ± 3% +10.0 27.97 +9.9 27.81 +6.5 24.43 perf-profile.children.cycles-pp.ext4_da_write_begin 13.75 ± 4% +12.0 25.77 +11.8 25.58 +7.3 21.00 perf-profile.children.cycles-pp.__filemap_get_folio 11.78 ± 5% +13.0 24.77 +12.8 24.57 +7.7 19.43 perf-profile.children.cycles-pp.filemap_add_folio 10.41 ± 6% +13.5 23.96 +13.6 23.98 +8.1 18.54 perf-profile.children.cycles-pp.folio_add_lru 34.89 ± 2% +18.9 53.82 +18.9 53.79 +11.1 46.00 perf-profile.children.cycles-pp.folio_batch_move_lru 51.11 +25.1 76.23 +25.3 76.40 +13.7 64.78 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 51.02 +25.2 76.18 +25.3 76.36 +13.7 64.70 perf-profile.children.cycles-pp.folio_lruvec_lock_irqsave 50.98 +25.3 76.24 +25.4 76.41 +13.7 64.69 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 7.25 ± 9% -6.7 0.53 ± 14% -6.8 0.50 ± 16% -5.7 1.54 ± 21% perf-profile.self.cycles-pp.osq_lock 3.09 -1.4 1.65 ± 2% -1.4 1.67 -0.5 2.55 perf-profile.self.cycles-pp.clear_bhb_loop 1.79 -0.7 1.07 ± 2% -0.7 1.07 -0.2 1.54 perf-profile.self.cycles-pp._copy_to_iter 1.14 -0.5 0.61 ± 2% -0.5 0.62 -0.2 0.94 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 1.20 -0.5 0.70 ± 2% -0.5 0.70 -0.2 1.02 perf-profile.self.cycles-pp.memset_orig 1.06 -0.5 0.57 ± 2% -0.5 0.56 ± 2% -0.2 0.86 perf-profile.self.cycles-pp.copy_page_from_iter_atomic 0.97 -0.5 0.51 ± 2% -0.5 0.51 -0.2 0.79 perf-profile.self.cycles-pp.filemap_read 0.94 ± 2% -0.4 0.51 ± 3% -0.4 0.51 -0.2 0.78 perf-profile.self.cycles-pp.__fdget_pos 0.86 ± 4% -0.4 0.45 ± 3% -0.4 0.44 ± 2% -0.2 0.62 ± 4% perf-profile.self.cycles-pp.workingset_age_nonresident 0.75 ± 2% -0.4 0.38 ± 3% -0.4 0.38 ± 2% -0.2 0.59 perf-profile.self.cycles-pp.vfs_write 0.89 -0.3 0.55 ± 2% -0.3 0.55 -0.1 0.80 perf-profile.self.cycles-pp.__block_commit_write 0.54 ± 3% -0.3 0.21 ± 4% -0.3 0.22 ± 3% -0.2 0.38 perf-profile.self.cycles-pp.percpu_counter_add_batch 0.50 ± 3% -0.3 0.20 ± 3% -0.3 0.20 ± 2% -0.1 0.37 ± 6% perf-profile.self.cycles-pp.balance_dirty_pages_ratelimited_flags 0.63 -0.3 0.33 ± 2% -0.3 0.33 -0.1 0.52 perf-profile.self.cycles-pp.vfs_read 0.57 -0.3 0.29 ± 2% -0.3 0.30 -0.1 0.46 perf-profile.self.cycles-pp.do_syscall_64 0.61 -0.3 0.33 ± 2% -0.3 0.36 -0.1 0.54 perf-profile.self.cycles-pp.filemap_get_read_batch 0.58 ± 2% -0.3 0.31 ± 3% -0.3 0.31 ± 2% -0.1 0.46 perf-profile.self.cycles-pp.fault_in_readable 0.37 ± 3% -0.3 0.10 ± 4% -0.3 0.10 ± 6% -0.2 0.22 ± 6% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.56 -0.3 0.29 ± 3% -0.3 0.29 -0.1 0.45 ± 2% perf-profile.self.cycles-pp.xas_load 0.38 ± 6% -0.3 0.13 ± 4% -0.3 0.12 ± 6% -0.1 0.25 ± 6% perf-profile.self.cycles-pp.ext4_file_write_iter 0.48 ± 2% -0.2 0.24 ± 4% -0.2 0.24 ± 2% -0.1 0.37 ± 2% perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.35 ± 5% -0.2 0.13 ± 5% -0.2 0.13 ± 4% -0.1 0.24 ± 4% perf-profile.self.cycles-pp.ext4_file_read_iter 0.46 ± 2% -0.2 0.24 ± 2% -0.2 0.24 ± 2% -0.1 0.36 ± 3% perf-profile.self.cycles-pp.write 0.45 ± 2% -0.2 0.23 ± 6% -0.2 0.23 ± 4% -0.1 0.35 ± 4% perf-profile.self.cycles-pp.disk_rr 0.39 ± 3% -0.2 0.18 ± 4% -0.2 0.18 ± 6% -0.1 0.33 ± 10% perf-profile.self.cycles-pp.apparmor_file_permission 0.36 ± 4% -0.2 0.15 ± 4% -0.2 0.15 ± 3% -0.1 0.26 ± 3% perf-profile.self.cycles-pp.ext4_da_write_begin 0.44 -0.2 0.23 ± 2% -0.2 0.24 ± 2% -0.1 0.37 perf-profile.self.cycles-pp.ext4_da_write_end 0.44 -0.2 0.23 ± 2% -0.2 0.24 ± 2% -0.1 0.36 perf-profile.self.cycles-pp.__filemap_get_folio 0.42 -0.2 0.21 ± 2% -0.2 0.22 -0.1 0.34 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.42 ± 2% -0.2 0.22 ± 3% -0.2 0.22 ± 2% -0.1 0.33 ± 2% perf-profile.self.cycles-pp.generic_perform_write 0.40 -0.2 0.20 ± 2% -0.2 0.21 -0.1 0.33 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.43 ± 2% -0.2 0.24 ± 3% -0.2 0.23 ± 2% -0.1 0.36 ± 2% perf-profile.self.cycles-pp.read 0.40 ± 2% -0.2 0.21 ± 3% -0.2 0.21 -0.1 0.32 ± 2% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.41 -0.2 0.22 ± 3% -0.2 0.22 -0.1 0.34 perf-profile.self.cycles-pp.llseek 0.33 -0.2 0.17 ± 2% -0.2 0.16 ± 2% -0.1 0.26 ± 2% perf-profile.self.cycles-pp.ext4_block_write_begin 0.33 ± 2% -0.2 0.17 ± 3% -0.2 0.17 -0.1 0.26 perf-profile.self.cycles-pp.__cond_resched 0.28 ± 2% -0.2 0.13 ± 3% -0.2 0.13 -0.1 0.22 ± 2% perf-profile.self.cycles-pp.atime_needs_update 0.30 ± 2% -0.1 0.16 ± 3% -0.1 0.16 ± 2% -0.1 0.24 perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.30 -0.1 0.16 ± 4% -0.1 0.16 ± 3% -0.1 0.25 perf-profile.self.cycles-pp.filemap_get_entry 0.30 -0.1 0.16 ± 3% -0.1 0.16 ± 2% -0.1 0.24 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 0.28 -0.1 0.14 ± 4% -0.1 0.14 ± 2% -0.1 0.22 perf-profile.self.cycles-pp.folio_mark_accessed 0.27 -0.1 0.14 ± 2% -0.1 0.14 -0.1 0.22 ± 2% perf-profile.self.cycles-pp.mark_buffer_dirty 0.12 ± 5% -0.1 0.00 -0.1 0.00 -0.0 0.07 ± 5% perf-profile.self.cycles-pp.start_this_handle 0.26 -0.1 0.14 ± 3% -0.1 0.13 ± 3% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.ext4_da_map_blocks 0.25 -0.1 0.13 ± 3% -0.1 0.13 ± 3% -0.1 0.19 ± 2% perf-profile.self.cycles-pp.down_write 0.25 -0.1 0.14 ± 2% -0.1 0.14 ± 2% -0.0 0.21 ± 2% perf-profile.self.cycles-pp.x64_sys_call 0.27 ± 2% -0.1 0.16 ± 2% -0.1 0.16 ± 3% -0.0 0.22 ± 2% perf-profile.self.cycles-pp.block_invalidate_folio 0.24 ± 2% -0.1 0.12 ± 3% -0.1 0.12 ± 3% -0.0 0.19 ± 3% perf-profile.self.cycles-pp.up_write 0.16 ± 4% -0.1 0.06 ± 7% -0.1 0.06 ± 7% -0.1 0.11 ± 4% perf-profile.self.cycles-pp.ext4_nonda_switch 0.20 ± 2% -0.1 0.10 -0.1 0.10 -0.0 0.16 ± 2% perf-profile.self.cycles-pp.inode_needs_update_time 0.21 ± 2% -0.1 0.11 -0.1 0.11 -0.0 0.17 perf-profile.self.cycles-pp.folio_unlock 0.23 -0.1 0.13 ± 3% -0.1 0.11 ± 2% -0.1 0.17 perf-profile.self.cycles-pp.xas_store 0.21 ± 2% -0.1 0.11 -0.1 0.11 ± 2% -0.0 0.18 ± 2% perf-profile.self.cycles-pp.xas_start 0.20 ± 2% -0.1 0.10 ± 4% -0.1 0.10 ± 4% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.filemap_get_pages 0.20 ± 2% -0.1 0.10 ± 4% -0.1 0.10 ± 4% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.security_file_permission 0.20 ± 4% -0.1 0.10 ± 4% -0.1 0.10 ± 4% -0.0 0.16 ± 3% perf-profile.self.cycles-pp.rcu_all_qs 0.22 -0.1 0.12 ± 3% -0.1 0.12 ± 2% -0.0 0.19 perf-profile.self.cycles-pp.folios_put_refs 0.20 ± 2% -0.1 0.11 ± 3% -0.1 0.11 -0.0 0.16 ± 2% perf-profile.self.cycles-pp.__slab_free 0.18 -0.1 0.09 ± 3% -0.1 0.08 ± 5% -0.0 0.13 perf-profile.self.cycles-pp.__filemap_add_folio 0.17 ± 2% -0.1 0.09 ± 4% -0.1 0.09 ± 5% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.ext4_buffered_write_iter 0.18 ± 2% -0.1 0.09 ± 5% -0.1 0.09 -0.0 0.14 ± 2% perf-profile.self.cycles-pp.rw_verify_area 0.08 ± 4% -0.1 0.00 -0.1 0.00 -0.0 0.07 ± 6% perf-profile.self.cycles-pp.fault_in_iov_iter_readable 0.08 ± 7% -0.1 0.00 -0.1 0.02 ±100% -0.0 0.07 ± 13% perf-profile.self.cycles-pp.get_mem_cgroup_from_mm 0.08 -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.self.cycles-pp.__es_insert_extent 0.08 -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.self.cycles-pp.ext4_es_insert_delayed_block 0.08 -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.self.cycles-pp.rmqueue 0.08 ± 8% -0.1 0.00 -0.1 0.01 ±264% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.inode_to_bdi 0.17 -0.1 0.09 ± 4% -0.1 0.09 -0.0 0.13 ± 3% perf-profile.self.cycles-pp.__mod_node_page_state 0.08 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.05 ± 9% perf-profile.self.cycles-pp._raw_read_lock 0.08 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.self.cycles-pp.generic_file_llseek_size 0.16 ± 3% -0.1 0.08 ± 3% -0.1 0.08 -0.0 0.12 ± 2% perf-profile.self.cycles-pp.aa_file_perm 0.15 -0.1 0.07 ± 6% -0.1 0.08 ± 5% -0.0 0.12 perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.16 ± 2% -0.1 0.09 ± 3% -0.1 0.08 ± 5% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.cgroup_rstat_updated 0.07 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.self.cycles-pp.__mark_inode_dirty 0.07 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.06 perf-profile.self.cycles-pp.generic_file_read_iter 0.15 ± 3% -0.1 0.08 ± 4% -0.1 0.08 -0.0 0.12 ± 2% perf-profile.self.cycles-pp.current_time 0.15 ± 3% -0.1 0.08 ± 3% -0.1 0.08 -0.0 0.12 ± 2% perf-profile.self.cycles-pp.ksys_write 0.15 ± 6% -0.1 0.07 ± 6% -0.1 0.07 ± 4% -0.0 0.12 ± 8% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.15 ± 2% -0.1 0.08 -0.1 0.08 -0.0 0.12 ± 2% perf-profile.self.cycles-pp.generic_write_checks 0.15 ± 4% -0.1 0.08 ± 6% -0.0 0.10 ± 4% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.__memcg_slab_post_alloc_hook 0.15 ± 3% -0.1 0.08 ± 4% -0.1 0.08 ± 6% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.copy_page_to_iter 0.15 ± 3% -0.1 0.08 ± 4% -0.1 0.08 -0.0 0.12 perf-profile.self.cycles-pp.ksys_read 0.07 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.__folio_cancel_dirty 0.07 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.ext4_generic_write_checks 0.07 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__mem_cgroup_charge 0.08 -0.1 0.01 ±163% -0.1 0.02 ±100% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.free_unref_folios 0.18 ± 2% -0.1 0.11 ± 3% -0.1 0.11 ± 3% -0.0 0.15 ± 3% perf-profile.self.cycles-pp.mod_objcg_state 0.13 -0.1 0.06 ± 7% -0.1 0.07 ± 7% -0.0 0.10 perf-profile.self.cycles-pp.__radix_tree_lookup 0.13 ± 3% -0.1 0.06 ± 4% -0.1 0.06 ± 7% -0.0 0.10 perf-profile.self.cycles-pp.__alloc_pages_noprof 0.06 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp._raw_spin_lock_irq 0.06 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.folio_activate 0.06 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.truncate_cleanup_folio 0.06 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 ± 9% perf-profile.self.cycles-pp.xas_clear_mark 0.15 ± 2% -0.1 0.09 ± 5% -0.1 0.09 -0.0 0.13 ± 2% perf-profile.self.cycles-pp.find_lock_entries 0.14 -0.1 0.08 ± 6% -0.1 0.08 ± 4% -0.0 0.12 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.06 ± 7% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.amd_clear_divider 0.13 ± 5% -0.1 0.07 ± 7% -0.1 0.08 ± 8% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.kmem_cache_alloc_noprof 0.06 ± 6% -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.try_to_free_buffers 0.12 ± 2% -0.1 0.06 -0.1 0.06 -0.0 0.09 ± 4% perf-profile.self.cycles-pp.node_dirty_ok 0.06 -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.__mod_zone_page_state 0.06 -0.1 0.00 -0.1 0.00 -0.0 0.05 perf-profile.self.cycles-pp.delete_from_page_cache_batch 0.12 -0.1 0.06 ± 6% -0.1 0.06 -0.0 0.10 ± 4% perf-profile.self.cycles-pp.ksys_lseek 0.09 -0.1 0.03 ± 75% -0.1 0.03 ± 77% -0.0 0.07 perf-profile.self.cycles-pp.ext4_es_lookup_extent 0.12 -0.1 0.06 ± 7% -0.1 0.06 ± 5% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.create_empty_buffers 0.12 ± 4% -0.1 0.06 -0.1 0.06 ± 5% -0.0 0.09 perf-profile.self.cycles-pp.__dquot_alloc_space 0.10 ± 3% -0.1 0.05 ± 31% -0.1 0.04 ± 57% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.folio_account_dirtied 0.12 ± 4% -0.1 0.06 ± 4% -0.1 0.06 -0.0 0.09 ± 5% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head 0.09 -0.1 0.04 ± 61% -0.1 0.02 ±100% -0.0 0.07 perf-profile.self.cycles-pp.file_modified 0.11 ± 4% -0.1 0.05 ± 8% -0.1 0.05 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ext4_llseek 0.10 ± 4% -0.1 0.05 -0.1 0.05 ± 6% -0.0 0.08 ± 6% perf-profile.self.cycles-pp.__count_memcg_events 0.11 ± 4% -0.1 0.06 ± 8% -0.1 0.06 ± 9% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.generic_write_check_limits 0.10 ± 5% -0.1 0.05 -0.1 0.05 -0.0 0.08 ± 4% perf-profile.self.cycles-pp.file_remove_privs_flags 0.10 ± 5% -0.1 0.05 -0.1 0.05 -0.0 0.08 perf-profile.self.cycles-pp.block_write_end 0.10 -0.1 0.05 -0.1 0.05 -0.0 0.08 perf-profile.self.cycles-pp.get_page_from_freelist 0.12 ± 4% -0.0 0.07 ± 6% -0.0 0.07 ± 6% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.12 -0.0 0.07 ± 5% -0.0 0.07 -0.0 0.10 perf-profile.self.cycles-pp.drop_buffers 0.10 ± 5% -0.0 0.05 -0.0 0.05 -0.0 0.08 ± 6% perf-profile.self.cycles-pp.timestamp_truncate 0.10 ± 5% -0.0 0.05 ± 5% -0.0 0.05 -0.0 0.08 perf-profile.self.cycles-pp.folio_account_cleaned 0.10 ± 4% -0.0 0.06 ± 8% -0.0 0.05 ± 6% -0.0 0.08 perf-profile.self.cycles-pp.kmem_cache_free 0.11 ± 9% -0.0 0.07 ± 8% -0.0 0.07 ± 9% +0.1 0.20 ± 3% perf-profile.self.cycles-pp.mem_cgroup_update_lru_size 0.10 ± 4% -0.0 0.07 -0.0 0.07 -0.0 0.10 ± 4% perf-profile.self.cycles-pp.__memcg_slab_free_hook 0.09 ± 5% -0.0 0.06 ± 4% -0.0 0.06 ± 5% -0.0 0.08 ± 4% perf-profile.self.cycles-pp.try_charge_memcg 0.10 ± 4% -0.0 0.07 ± 5% -0.0 0.07 -0.0 0.08 ± 4% perf-profile.self.cycles-pp.__page_cache_release 0.23 ± 3% -0.0 0.21 ± 2% -0.2 0.00 -0.2 0.05 perf-profile.self.cycles-pp.xas_find_conflict 0.23 ± 2% -0.0 0.22 ± 3% -0.0 0.22 ± 4% -0.0 0.18 ± 3% perf-profile.self.cycles-pp.workingset_activation 0.25 ± 2% +0.0 0.28 ± 4% +0.0 0.28 +0.0 0.30 perf-profile.self.cycles-pp.folio_activate_fn 0.17 ± 3% +0.1 0.26 +0.1 0.25 ± 3% -0.0 0.13 perf-profile.self.cycles-pp.lru_add_fn 0.51 ± 2% +0.1 0.63 ± 3% +0.1 0.63 ± 2% -0.1 0.40 ± 2% perf-profile.self.cycles-pp.__lruvec_stat_mod_folio 0.30 ± 5% +0.2 0.51 ± 2% +0.2 0.50 +0.0 0.34 ± 3% perf-profile.self.cycles-pp.folio_batch_move_lru 50.97 +25.3 76.24 +25.4 76.41 +13.7 64.69 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-15 14:14 ` Oliver Sang @ 2024-07-15 22:18 ` Roman Gushchin 2024-07-16 7:53 ` Oliver Sang 0 siblings, 1 reply; 12+ messages in thread From: Roman Gushchin @ 2024-07-15 22:18 UTC (permalink / raw) To: Oliver Sang Cc: oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > hi, Roman Gushchin, > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > Hello, > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > Hello, > > > > thank you for the report! > > > > I'd expect that the regression should be fixed by the commit > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > Can you, please, confirm that it's not the case? > > > > Thank you! > > in our this aim7 test, we found the performance partially recovered by > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully Thank you for providing the detailed information! Can you, please, check if the following patch resolves the regression entirely? Thanks, Roman -- diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 60418934827c..3aae347cda09 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -110,6 +110,7 @@ struct mem_cgroup_per_node { /* Fields which get updated often at the end. */ struct lruvec lruvec; unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; + CACHELINE_PADDING(_pad2_); struct mem_cgroup_reclaim_iter iter; }; ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-15 22:18 ` Roman Gushchin @ 2024-07-16 7:53 ` Oliver Sang 2024-07-19 17:14 ` Roman Gushchin 0 siblings, 1 reply; 12+ messages in thread From: Oliver Sang @ 2024-07-16 7:53 UTC (permalink / raw) To: Roman Gushchin Cc: oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin, oliver.sang hi, Roman, On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > hi, Roman Gushchin, > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > Hello, > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > Hello, > > > > > > thank you for the report! > > > > > > I'd expect that the regression should be fixed by the commit > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > Can you, please, confirm that it's not the case? > > > > > > Thank you! > > > > in our this aim7 test, we found the performance partially recovered by > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > Thank you for providing the detailed information! > > Can you, please, check if the following patch resolves the regression entirely? no. in our tests, the following patch has little impact. I directly apply it upon 6df13230b6 (if this is not the proper applyment, please let me know, thanks) ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase: gcc-13/performance/1BRD_48G/ext4/x86_64-rhel-8.3/3000/debian-12-x86_64-20240206.cgz/lkp-icl-2sp2/disk_rr/aim7 commit: 94b7e5bf09 ("mm: memcg: put memcg1-specific struct mem_cgroup's members under CONFIG_MEMCG_V1") 98c9daf5ae ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") 6df13230b6 ("mm: memcg: add cache line padding to mem_cgroup_per_node") 7b3274adca <---- your "_pad2_" patch 94b7e5bf09b08aa4 98c9daf5ae6be008f78c07b744b 6df13230b612af81ce04f20bb37 7b3274adcaac12369e13ac32e46 ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 210696 ± 6% -3.0% 204465 ± 7% -8.3% 193107 ± 3% -12.9% 183521 ± 2% cpuidle..usage 76.42 +15.1% 87.99 ± 3% +3.1% 78.77 +3.4% 79.00 ± 2% uptime.boot 53.28 ± 2% -27.5% 38.65 ± 2% -9.1% 48.45 ± 2% -10.3% 47.81 ± 2% iostat.cpu.idle 44.58 ± 2% +33.6% 59.58 +11.2% 49.57 ± 2% +12.6% 50.22 ± 2% iostat.cpu.system 2.13 -17.0% 1.77 -7.1% 1.98 -7.5% 1.97 iostat.cpu.user 49.85 ± 2% -14.7 35.17 ± 2% -5.2 44.66 ± 3% -5.7 44.18 ± 2% mpstat.cpu.all.idle% 0.15 ± 7% +0.0 0.15 ± 3% -0.0 0.13 ± 4% -0.0 0.13 ± 3% mpstat.cpu.all.irq% 47.74 ± 2% +15.1 62.82 +5.4 53.14 ± 2% +5.9 53.63 ± 2% mpstat.cpu.all.sys% 2.23 -0.4 1.82 -0.2 2.04 -0.2 2.04 mpstat.cpu.all.usr% 53.29 ± 2% -27.4% 38.70 ± 2% -9.4% 48.28 ± 2% -10.2% 47.86 ± 2% vmstat.cpu.id 2.242e+08 -0.0% 2.242e+08 +0.0% 2.243e+08 +14.7% 2.572e+08 vmstat.memory.free 65.83 ± 4% +59.1% 104.76 ± 3% +19.0% 78.35 ± 3% +19.0% 78.36 ± 3% vmstat.procs.r 8385 ± 4% +34.6% 11284 ± 3% +20.5% 10101 ± 8% +12.7% 9450 ± 5% vmstat.system.cs 245966 ± 2% +8.1% 265964 +19.3% 293498 ± 2% +19.6% 294290 ± 2% vmstat.system.in 4752 ± 13% -18.1% 3890 ± 11% -9.0% 4322 ± 19% -1.8% 4666 ± 16% sched_debug.cpu.avg_idle.min 552896 ± 10% +8.5% 599623 ± 10% +6.6% 589455 ± 16% +68.1% 929486 ± 18% sched_debug.cpu.max_idle_balance_cost.max 5295 ±109% +102.4% 10720 ± 68% +69.7% 8987 ±112% +796.0% 47452 ± 41% sched_debug.cpu.max_idle_balance_cost.stddev 0.00 ± 32% -49.2% 0.00 ±171% -62.3% 0.00 ± 99% -80.7% 0.00 ±173% sched_debug.rt_rq:.rt_time.avg 0.19 ± 32% -49.2% 0.10 ±171% -62.3% 0.07 ± 99% -80.7% 0.04 ±173% sched_debug.rt_rq:.rt_time.max 0.02 ± 32% -49.2% 0.01 ±171% -62.3% 0.01 ± 99% -80.7% 0.00 ±173% sched_debug.rt_rq:.rt_time.stddev 778685 -29.4% 549435 -10.3% 698378 -10.8% 694234 aim7.jobs-per-min 23.31 +41.4% 32.96 +11.4% 25.96 +12.1% 26.12 aim7.time.elapsed_time 23.31 +41.4% 32.96 +11.4% 25.96 +12.1% 26.12 aim7.time.elapsed_time.max 47890 ± 7% +338.5% 210000 ± 3% +117.1% 103949 ± 7% +107.4% 99316 ± 3% aim7.time.involuntary_context_switches 6674 +26.7% 8455 +10.5% 7372 +11.2% 7423 aim7.time.percent_of_cpu_this_job_got 1510 +81.7% 2744 ± 2% +23.8% 1869 +25.5% 1894 aim7.time.system_time 19454 ± 3% +9.1% 21223 ± 3% -3.5% 18782 ± 4% -1.1% 19246 ± 3% aim7.time.voluntary_context_switches 23.31 +41.4% 32.96 +11.4% 25.96 +12.1% 26.12 time.elapsed_time 23.31 +41.4% 32.96 +11.4% 25.96 +12.1% 26.12 time.elapsed_time.max 47890 ± 7% +338.5% 210000 ± 3% +117.1% 103949 ± 7% +107.4% 99316 ± 3% time.involuntary_context_switches 6674 +26.7% 8455 +10.5% 7372 +11.2% 7423 time.percent_of_cpu_this_job_got 1510 +81.7% 2744 ± 2% +23.8% 1869 +25.5% 1894 time.system_time 45.72 -4.9% 43.47 -1.7% 44.96 -2.6% 44.53 time.user_time 19454 ± 3% +9.1% 21223 ± 3% -3.5% 18782 ± 4% -1.1% 19246 ± 3% time.voluntary_context_switches 49345 ± 6% +108.4% 102820 ± 10% +32.3% 65261 ± 5% +31.2% 64743 ± 3% meminfo.Active 22670 ± 11% +182.6% 64065 ± 15% +57.3% 35667 ± 11% +55.9% 35345 ± 5% meminfo.Active(anon) 26674 ± 4% +45.3% 38754 ± 4% +10.9% 29594 ± 5% +10.2% 29398 ± 4% meminfo.Active(file) 33695 +18.6% 39973 ± 3% +3.8% 34971 +3.5% 34889 meminfo.AnonHugePages 1.154e+08 +0.0% 1.154e+08 +0.0% 1.154e+08 +14.3% 1.319e+08 meminfo.CommitLimit 2.244e+08 -0.1% 2.241e+08 -0.4% 2.236e+08 +14.4% 2.568e+08 meminfo.DirectMap1G 1360098 ± 3% +14.0% 1551107 ± 2% +1.6% 1381539 ± 3% -0.4% 1355108 ± 3% meminfo.Inactive 803759 +15.0% 924519 +4.7% 841839 +4.8% 842104 meminfo.Inactive(anon) 66977 ± 3% +115.7% 144485 ± 7% +21.0% 81015 ± 3% +20.8% 80939 ± 4% meminfo.Mapped 2.216e+08 -0.1% 2.214e+08 -0.1% 2.214e+08 +14.8% 2.544e+08 meminfo.MemAvailable 2.221e+08 -0.1% 2.219e+08 -0.0% 2.22e+08 +14.8% 2.551e+08 meminfo.MemFree 2.307e+08 +0.0% 2.307e+08 +0.0% 2.307e+08 +14.3% 2.637e+08 meminfo.MemTotal 78152 ± 9% +188.4% 225431 ± 5% +64.5% 128567 ± 5% +65.1% 129035 ± 5% meminfo.Shmem 15327 ± 8% +52.6% 23389 ± 11% +29.5% 19853 ± 14% +23.5% 18923 ± 6% numa-meminfo.node0.Active 13455 ± 9% +46.0% 19642 ± 10% +18.7% 15975 ± 13% +11.0% 14929 ± 6% numa-meminfo.node0.Active(file) 19657 ± 14% +111.2% 41510 ± 18% +76.5% 34690 ± 37% +25.3% 24630 ± 3% numa-meminfo.node0.Mapped 6081 ± 16% +84.9% 11247 ± 42% +38.8% 8442 ± 13% +31.3% 7984 ± 16% numa-meminfo.node0.Shmem 33647 ± 8% +134.3% 78825 ± 15% +35.8% 45703 ± 9% +40.3% 47191 ± 3% numa-meminfo.node1.Active 20790 ± 14% +188.7% 60020 ± 18% +52.8% 31760 ± 14% +50.9% 31364 ± 6% numa-meminfo.node1.Active(anon) 12857 ± 7% +46.3% 18805 ± 9% +8.4% 13942 ± 8% +23.1% 15827 ± 5% numa-meminfo.node1.Active(file) 485228 ± 23% +85.5% 899983 ± 21% +85.5% 900160 ± 25% +58.5% 768931 ± 35% numa-meminfo.node1.Inactive 202089 ± 50% +192.9% 591917 ± 32% +215.8% 638153 ± 34% +153.9% 513101 ± 51% numa-meminfo.node1.Inactive(anon) 47991 ± 7% +112.0% 101755 ± 8% -2.6% 46748 ± 32% +18.4% 56799 ± 4% numa-meminfo.node1.Mapped 93326924 +0.8% 94106109 +1.2% 94466709 +35.5% 1.265e+08 numa-meminfo.node1.MemFree 99028084 +0.0% 99028084 +0.0% 99028084 +33.4% 1.321e+08 numa-meminfo.node1.MemTotal 72431 ± 8% +194.1% 213055 ± 4% +65.2% 119653 ± 5% +67.5% 121337 ± 5% numa-meminfo.node1.Shmem 467.96 ± 53% +99.5% 933.35 ± 29% +107.2% 969.49 ± 31% +113.4% 998.53 ± 25% numa-vmstat.node0.nr_active_anon 3290 ± 10% +50.9% 4965 ± 10% +20.2% 3956 ± 7% +11.0% 3652 ± 7% numa-vmstat.node0.nr_active_file 4793 ± 17% +113.8% 10249 ± 20% +84.1% 8827 ± 36% +30.5% 6255 ± 3% numa-vmstat.node0.nr_mapped 1519 ± 16% +72.6% 2622 ± 33% +38.8% 2110 ± 13% +31.3% 1995 ± 16% numa-vmstat.node0.nr_shmem 467.96 ± 53% +99.4% 933.32 ± 29% +107.2% 969.49 ± 31% +113.4% 998.53 ± 25% numa-vmstat.node0.nr_zone_active_anon 3294 ± 9% +50.9% 4970 ± 10% +20.4% 3967 ± 7% +11.0% 3655 ± 7% numa-vmstat.node0.nr_zone_active_file 4955 ± 11% +202.3% 14980 ± 18% +61.8% 8017 ± 16% +59.8% 7921 ± 9% numa-vmstat.node1.nr_active_anon 3376 ± 8% +40.1% 4729 ± 7% +7.0% 3611 ± 13% +10.2% 3720 ± 8% numa-vmstat.node1.nr_active_file 23331085 +0.8% 23527208 +1.2% 23617254 +35.5% 31624945 numa-vmstat.node1.nr_free_pages 50152 ± 51% +194.6% 147731 ± 32% +218.6% 159767 ± 34% +155.9% 128355 ± 51% numa-vmstat.node1.nr_inactive_anon 12101 ± 7% +109.5% 25357 ± 8% -1.6% 11908 ± 33% +18.9% 14390 ± 4% numa-vmstat.node1.nr_mapped 393216 +0.0% 393216 +0.0% 393216 +33.3% 524288 numa-vmstat.node1.nr_memmap_boot 17201 ± 7% +208.3% 53026 ± 4% +75.7% 30214 ± 8% +77.7% 30558 ± 4% numa-vmstat.node1.nr_shmem 4955 ± 11% +202.3% 14980 ± 18% +61.8% 8017 ± 16% +59.8% 7921 ± 9% numa-vmstat.node1.nr_zone_active_anon 3352 ± 8% +41.1% 4729 ± 7% +7.8% 3612 ± 13% +11.0% 3721 ± 8% numa-vmstat.node1.nr_zone_active_file 50150 ± 51% +194.6% 147730 ± 32% +218.6% 159766 ± 34% +155.9% 128353 ± 51% numa-vmstat.node1.nr_zone_inactive_anon 5776 ± 11% +175.1% 15893 ± 16% +55.0% 8953 ± 11% +51.7% 8765 ± 8% proc-vmstat.nr_active_anon 6595 ± 8% +46.7% 9677 ± 3% +12.6% 7424 ± 5% +12.8% 7440 ± 3% proc-vmstat.nr_active_file 187950 +2.0% 191760 +0.1% 188193 +0.1% 188115 proc-vmstat.nr_anon_pages 5532779 -0.1% 5528733 -0.0% 5530018 +14.8% 6353558 proc-vmstat.nr_dirty_background_threshold 11079086 -0.1% 11070986 -0.0% 11073558 +14.8% 12722652 proc-vmstat.nr_dirty_threshold 945653 +6.0% 1002646 +1.0% 955008 +0.3% 948831 proc-vmstat.nr_file_pages 55532095 -0.1% 55470389 -0.0% 55507767 +14.8% 63777563 proc-vmstat.nr_free_pages 201118 +14.8% 230842 +4.7% 210479 +4.6% 210383 proc-vmstat.nr_inactive_anon 17214 ± 3% +108.2% 35836 ± 6% +20.0% 20660 ± 3% +18.7% 20433 ± 3% proc-vmstat.nr_mapped 916480 +0.0% 916480 +0.0% 916480 +14.3% 1047552 proc-vmstat.nr_memmap_boot 19951 ± 8% +180.4% 55933 ± 4% +61.5% 32218 ± 5% +60.6% 32031 ± 4% proc-vmstat.nr_shmem 40267 +2.7% 41335 -0.7% 39991 -1.1% 39828 proc-vmstat.nr_slab_reclaimable 86277 +1.8% 87792 +0.1% 86326 -0.1% 86210 proc-vmstat.nr_slab_unreclaimable 5776 ± 11% +175.1% 15893 ± 16% +55.0% 8953 ± 11% +51.7% 8765 ± 8% proc-vmstat.nr_zone_active_anon 6595 ± 8% +46.7% 9677 ± 3% +12.6% 7424 ± 5% +12.8% 7440 ± 3% proc-vmstat.nr_zone_active_file 201118 +14.8% 230842 +4.7% 210479 +4.6% 210383 proc-vmstat.nr_zone_inactive_anon 312.12 ±241% +1618.1% 5362 ±125% +521.3% 1939 ±171% +317.0% 1301 ±218% proc-vmstat.numa_pages_migrated 369792 +19.6% 442285 ± 3% +2.9% 380386 +3.7% 383301 proc-vmstat.pgfault 312.12 ±241% +1618.1% 5362 ±125% +521.3% 1939 ±171% +317.0% 1301 ±218% proc-vmstat.pgmigrate_success 2426 ± 2% +29.2% 3135 ± 3% +16.3% 2821 ± 3% +15.3% 2798 ± 4% proc-vmstat.pgpgout 1515 +3.8% 1572 +1.3% 1535 +8.5% 1643 ± 17% proc-vmstat.unevictable_pgs_culled 0.63 ± 3% +36.5% 0.85 ± 2% +19.4% 0.75 ± 2% +14.4% 0.72 ± 3% perf-stat.i.MPKI 1.885e+10 -23.7% 1.437e+10 -7.4% 1.745e+10 ± 2% -8.3% 1.729e+10 perf-stat.i.branch-instructions 2.78 ± 2% -0.5 2.29 ± 4% -0.2 2.55 ± 4% -0.1 2.65 ± 2% perf-stat.i.branch-miss-rate% 67232899 -17.4% 55553782 ± 3% -5.8% 63348903 -6.0% 63198323 perf-stat.i.branch-misses 13.70 ± 3% +2.5 16.17 ± 3% +1.1 14.84 ± 2% +0.8 14.45 ± 3% perf-stat.i.cache-miss-rate% 72591570 ± 2% +0.6% 73035167 +10.8% 80408900 ± 3% +7.0% 77651016 ± 2% perf-stat.i.cache-misses 5.483e+08 -24.9% 4.118e+08 -5.7% 5.17e+08 ± 2% -6.8% 5.109e+08 perf-stat.i.cache-references 8605 ± 4% +34.7% 11593 ± 4% +23.4% 10618 ± 7% +13.6% 9772 ± 6% perf-stat.i.context-switches 1.35 +65.4% 2.22 ± 3% +20.9% 1.63 +19.8% 1.61 perf-stat.i.cpi 1.616e+11 +22.1% 1.973e+11 +10.5% 1.785e+11 ± 2% +10.3% 1.782e+11 perf-stat.i.cpu-cycles 659.92 ± 3% -3.5% 636.65 ± 2% -5.9% 620.91 ± 3% -6.8% 615.12 ± 3% perf-stat.i.cpu-migrations 8.537e+10 -24.8% 6.416e+10 -8.0% 7.852e+10 ± 2% -8.8% 7.783e+10 perf-stat.i.instructions 0.96 -22.2% 0.75 ± 2% -9.4% 0.87 -7.8% 0.89 ± 2% perf-stat.i.ipc 13455 -15.4% 11379 ± 3% -7.6% 12438 -6.7% 12554 perf-stat.i.minor-faults 13489 -15.5% 11396 ± 3% -7.6% 12465 -6.6% 12595 perf-stat.i.page-faults 0.85 ± 3% +34.8% 1.15 +20.2% 1.02 ± 3% +17.1% 1.00 perf-stat.overall.MPKI 0.34 +0.0 0.36 ± 2% +0.0 0.35 +0.0 0.35 perf-stat.overall.branch-miss-rate% 13.23 ± 3% +4.6 17.78 +2.3 15.52 ± 2% +1.9 15.16 perf-stat.overall.cache-miss-rate% 1.90 +62.8% 3.08 +20.1% 2.28 +20.9% 2.29 perf-stat.overall.cpi 2231 ± 3% +20.6% 2690 ± 2% -0.2% 2227 ± 2% +3.2% 2302 perf-stat.overall.cycles-between-cache-misses 0.53 -38.5% 0.32 -16.7% 0.44 -17.3% 0.44 perf-stat.overall.ipc 1.86e+10 -23.0% 1.433e+10 -7.6% 1.719e+10 -8.4% 1.704e+10 perf-stat.ps.branch-instructions 64095803 -19.1% 51843320 ± 2% -5.7% 60421424 -5.8% 60408153 perf-stat.ps.branch-misses 71639076 ± 2% +2.3% 73304464 ± 2% +10.4% 79092736 ± 3% +6.6% 76347758 ± 2% perf-stat.ps.cache-misses 5.417e+08 -23.9% 4.124e+08 -5.9% 5.096e+08 ± 2% -7.0% 5.036e+08 perf-stat.ps.cache-references 8220 ± 4% +33.3% 10962 ± 4% +23.9% 10184 ± 8% +13.9% 9360 ± 5% perf-stat.ps.context-switches 1.597e+11 +23.5% 1.972e+11 ± 2% +10.2% 1.761e+11 ± 2% +10.0% 1.757e+11 perf-stat.ps.cpu-cycles 643.07 ± 2% -4.9% 611.59 ± 2% -5.6% 607.27 ± 3% -7.4% 595.33 ± 3% perf-stat.ps.cpu-migrations 8.426e+10 -24.1% 6.394e+10 -8.2% 7.737e+10 ± 2% -9.0% 7.668e+10 perf-stat.ps.instructions 12730 -18.7% 10354 ± 3% -7.2% 11811 -6.7% 11877 perf-stat.ps.minor-faults 12762 -18.8% 10369 ± 3% -7.3% 11836 -6.6% 11915 perf-stat.ps.page-faults 4.35 ± 10% -4.4 0.00 -3.5 0.81 ± 20% -3.5 0.84 ± 11% perf-profile.calltrace.cycles-pp.unlink 4.35 ± 10% -4.3 0.00 -3.5 0.81 ± 19% -3.5 0.84 ± 11% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.35 ± 10% -4.3 0.00 -3.5 0.81 ± 19% -3.5 0.84 ± 11% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink 4.34 ± 10% -4.3 0.00 -3.5 0.80 ± 19% -3.5 0.84 ± 11% perf-profile.calltrace.cycles-pp.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.34 ± 10% -4.3 0.00 -3.5 0.80 ± 20% -3.5 0.83 ± 11% perf-profile.calltrace.cycles-pp.do_unlinkat.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.14 ± 10% -4.1 0.00 -3.5 0.65 ± 24% -3.5 0.68 ± 13% perf-profile.calltrace.cycles-pp.down_write.do_unlinkat.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.14 ± 10% -4.1 0.00 -3.5 0.65 ± 24% -3.5 0.68 ± 13% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.do_unlinkat.__x64_sys_unlink.do_syscall_64 4.14 ± 10% -4.1 0.00 -3.6 0.58 ± 45% -3.5 0.68 ± 13% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.do_unlinkat.__x64_sys_unlink 3.99 ± 11% -4.0 0.00 -3.6 0.40 ± 83% -3.4 0.54 ± 40% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.do_unlinkat 3.53 ± 6% -3.4 0.10 ±212% -2.4 1.12 ± 17% -2.3 1.22 ± 8% perf-profile.calltrace.cycles-pp.down_write.open_last_lookups.path_openat.do_filp_open.do_sys_openat2 3.53 ± 6% -3.4 0.10 ±212% -2.4 1.12 ± 17% -2.3 1.22 ± 8% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.open_last_lookups.path_openat.do_filp_open 3.52 ± 6% -3.4 0.10 ±212% -2.4 1.12 ± 17% -2.3 1.22 ± 9% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.open_last_lookups.path_openat 3.30 ± 7% -3.3 0.00 -2.3 0.98 ± 19% -2.2 1.08 ± 9% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.open_last_lookups 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -2.4 1.54 ± 7% perf-profile.calltrace.cycles-pp.creat64 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -2.4 1.54 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -2.4 1.54 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.creat64 3.92 ± 6% -3.2 0.68 ± 8% -2.5 1.43 ± 13% -2.4 1.53 ± 7% perf-profile.calltrace.cycles-pp.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.92 ± 6% -3.2 0.68 ± 8% -2.5 1.43 ± 13% -2.4 1.53 ± 7% perf-profile.calltrace.cycles-pp.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.91 ± 6% -3.2 0.68 ± 8% -2.5 1.42 ± 13% -2.4 1.52 ± 7% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64 3.91 ± 6% -3.2 0.68 ± 8% -2.5 1.42 ± 13% -2.4 1.52 ± 7% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.87 ± 6% -3.2 0.65 ± 9% -2.5 1.39 ± 14% -2.4 1.49 ± 7% perf-profile.calltrace.cycles-pp.open_last_lookups.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat 3.66 -1.7 1.99 ± 2% -0.6 3.06 -0.7 3.01 perf-profile.calltrace.cycles-pp.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write 3.31 -1.5 1.77 ± 2% -0.6 2.73 -0.6 2.69 perf-profile.calltrace.cycles-pp.llseek 2.87 -1.3 1.62 ± 2% -0.5 2.33 -0.6 2.28 perf-profile.calltrace.cycles-pp.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 2.25 -1.0 1.29 ± 2% -0.4 1.83 -0.5 1.80 perf-profile.calltrace.cycles-pp.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write 2.17 -0.9 1.25 ± 2% -0.4 1.76 -0.4 1.73 perf-profile.calltrace.cycles-pp.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter 0.86 ± 4% -0.9 0.00 -0.2 0.62 ± 4% -0.3 0.61 ± 3% perf-profile.calltrace.cycles-pp.workingset_age_nonresident.workingset_activation.folio_mark_accessed.filemap_read.vfs_read 0.79 -0.8 0.00 -0.1 0.74 -0.1 0.72 perf-profile.calltrace.cycles-pp.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.93 -0.8 1.14 ± 2% -0.3 1.66 -0.3 1.61 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.vfs_read.ksys_read.do_syscall_64 0.76 -0.8 0.00 -0.2 0.58 -0.2 0.56 ± 2% perf-profile.calltrace.cycles-pp.file_modified.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64 1.99 -0.8 1.23 ± 2% -0.4 1.63 -0.4 1.59 perf-profile.calltrace.cycles-pp.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 0.74 -0.7 0.00 -0.2 0.58 -0.2 0.57 perf-profile.calltrace.cycles-pp.folio_alloc_noprof.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.79 -0.7 1.07 ± 2% -0.2 1.55 -0.3 1.50 perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.filemap_read.vfs_read.ksys_read 0.72 ± 2% -0.7 0.00 -0.2 0.54 ± 4% -0.2 0.52 ± 3% perf-profile.calltrace.cycles-pp.balance_dirty_pages_ratelimited_flags.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 0.69 -0.7 0.00 -0.2 0.54 -0.2 0.53 perf-profile.calltrace.cycles-pp.alloc_pages_mpol_noprof.folio_alloc_noprof.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 0.68 -0.7 0.00 -0.2 0.53 -0.2 0.52 perf-profile.calltrace.cycles-pp.touch_atime.filemap_read.vfs_read.ksys_read.do_syscall_64 1.34 -0.7 0.68 ± 3% -0.3 1.05 -0.3 1.04 perf-profile.calltrace.cycles-pp.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 0.65 ± 2% -0.7 0.00 -0.1 0.52 -0.1 0.52 ± 2% perf-profile.calltrace.cycles-pp.fault_in_iov_iter_readable.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 1.30 -0.6 0.66 ± 3% -0.3 1.02 -0.3 1.00 perf-profile.calltrace.cycles-pp.ext4_da_map_blocks.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 0.61 -0.6 0.00 -0.0 0.59 -0.0 0.58 perf-profile.calltrace.cycles-pp.folio_alloc_buffers.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 1.22 -0.6 0.65 ± 2% -0.2 0.99 -0.2 0.98 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.llseek 1.17 -0.5 0.63 ± 2% -0.2 1.00 -0.2 0.98 perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.vfs_read.ksys_read.do_syscall_64 0.54 ± 2% -0.5 0.00 -0.0 0.52 -0.1 0.44 ± 37% perf-profile.calltrace.cycles-pp.alloc_buffer_head.folio_alloc_buffers.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin 1.21 -0.5 0.67 ± 2% -0.3 0.91 -0.3 0.90 perf-profile.calltrace.cycles-pp.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write 0.96 -0.5 0.43 ± 47% -0.1 0.83 -0.1 0.81 perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.vfs_read.ksys_read 1.08 -0.5 0.57 ± 3% -0.2 0.86 -0.2 0.84 perf-profile.calltrace.cycles-pp.clear_bhb_loop.write 1.08 -0.5 0.57 ± 2% -0.2 0.88 -0.2 0.86 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek 1.20 -0.5 0.70 ± 3% -0.2 1.02 -0.2 1.00 perf-profile.calltrace.cycles-pp.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.05 -0.5 0.56 ± 2% -0.2 0.85 -0.2 0.83 perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 0.91 -0.5 0.42 ± 47% -0.2 0.67 -0.2 0.66 perf-profile.calltrace.cycles-pp.__folio_mark_dirty.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end 1.15 -0.5 0.68 ± 2% -0.2 0.99 -0.2 0.97 perf-profile.calltrace.cycles-pp.memset_orig.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 1.02 -0.5 0.55 ± 2% -0.2 0.86 -0.2 0.84 perf-profile.calltrace.cycles-pp.clear_bhb_loop.llseek 1.30 -0.5 0.83 ± 2% -0.2 1.05 -0.3 1.03 perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict 0.98 -0.5 0.53 ± 2% -0.2 0.82 -0.2 0.81 perf-profile.calltrace.cycles-pp.clear_bhb_loop.read 1.10 ± 3% -0.4 0.67 ± 3% -0.3 0.80 ± 3% -0.3 0.79 ± 3% perf-profile.calltrace.cycles-pp.workingset_activation.folio_mark_accessed.filemap_read.vfs_read.ksys_read 30.65 -0.4 30.25 +0.7 31.31 +0.7 31.35 perf-profile.calltrace.cycles-pp.read 0.98 -0.3 0.69 ± 2% -0.4 0.61 -0.4 0.59 perf-profile.calltrace.cycles-pp.__filemap_add_folio.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 2.39 +0.2 2.61 +0.1 2.47 +0.1 2.50 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release 2.42 +0.2 2.64 +0.1 2.50 +0.1 2.53 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode 2.39 +0.2 2.61 +0.1 2.48 +0.1 2.50 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range 2.42 +0.2 2.64 +0.1 2.50 +0.1 2.53 perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict 2.38 +0.2 2.61 +0.1 2.47 +0.1 2.50 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu 2.44 +0.4 2.89 +0.2 2.62 +0.2 2.63 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict 2.38 +0.5 2.84 +0.2 2.56 +0.2 2.58 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode 2.38 +0.5 2.84 +0.2 2.56 +0.2 2.58 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range 2.38 +0.5 2.84 +0.2 2.56 +0.2 2.57 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release 28.52 +0.6 29.11 +1.0 29.54 +1.1 29.60 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 28.38 +0.7 29.04 +1.0 29.42 +1.1 29.49 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 4.86 +0.7 5.54 +0.3 5.13 +0.3 5.16 perf-profile.calltrace.cycles-pp.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 27.89 +0.9 28.77 +1.1 29.02 +1.2 29.09 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 27.47 +1.1 28.54 +1.2 28.67 +1.3 28.75 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 25.90 +1.9 27.78 +1.5 27.42 +1.6 27.54 perf-profile.calltrace.cycles-pp.filemap_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.26 +4.0 34.31 +3.9 34.16 +3.8 34.05 perf-profile.calltrace.cycles-pp.write 21.06 +4.1 25.14 +2.3 23.39 +2.5 23.59 perf-profile.calltrace.cycles-pp.folio_mark_accessed.filemap_read.vfs_read.ksys_read.do_syscall_64 19.70 +4.6 24.33 +2.7 22.38 +2.9 22.60 perf-profile.calltrace.cycles-pp.folio_activate.folio_mark_accessed.filemap_read.vfs_read.ksys_read 19.63 +4.7 24.29 +2.7 22.32 +2.9 22.54 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read.vfs_read 18.85 +4.7 23.52 +2.7 21.52 +2.9 21.74 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read 18.84 +4.7 23.51 +2.7 21.52 +2.9 21.73 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed 18.83 +4.7 23.51 +2.7 21.51 +2.9 21.73 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate 28.38 +5.0 33.33 +4.3 32.65 +4.2 32.56 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 28.23 +5.0 33.25 +4.3 32.53 +4.2 32.45 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 27.72 +5.3 32.99 +4.4 32.13 +4.3 32.05 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 27.25 +5.5 32.74 +4.5 31.75 +4.4 31.68 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 25.87 +5.6 31.48 +2.2 28.03 +2.2 28.02 perf-profile.calltrace.cycles-pp.__close 25.86 +5.6 31.46 +2.2 28.01 +2.2 28.01 perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.86 +5.6 31.47 +2.2 28.02 +2.2 28.02 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.86 +5.6 31.47 +2.2 28.02 +2.2 28.02 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__close 25.84 +5.6 31.45 +2.2 28.00 +2.2 28.00 perf-profile.calltrace.cycles-pp.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.82 +5.6 31.44 +2.2 27.99 +2.2 27.98 perf-profile.calltrace.cycles-pp.dput.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 25.81 +5.6 31.44 +2.2 27.98 +2.2 27.98 perf-profile.calltrace.cycles-pp.__dentry_kill.dput.__fput.__x64_sys_close.do_syscall_64 25.79 +5.6 31.43 +2.2 27.96 +2.2 27.96 perf-profile.calltrace.cycles-pp.evict.__dentry_kill.dput.__fput.__x64_sys_close 25.78 +5.6 31.42 +2.2 27.95 +2.2 27.95 perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.__dentry_kill.dput.__fput 25.63 +5.7 31.33 +2.2 27.83 +2.2 27.83 perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill.dput 17.98 +6.1 24.07 +2.4 20.42 +2.5 20.43 perf-profile.calltrace.cycles-pp.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 17.57 +6.2 23.81 +2.5 20.06 +2.5 20.08 perf-profile.calltrace.cycles-pp.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict 17.33 +6.3 23.66 +2.5 19.86 +2.5 19.88 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range 17.34 +6.3 23.66 +2.5 19.86 +2.5 19.88 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode 17.32 +6.3 23.65 +2.5 19.85 +2.6 19.87 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs 25.47 +6.5 31.92 +4.9 30.41 +4.9 30.37 perf-profile.calltrace.cycles-pp.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 23.72 ± 2% +7.3 31.06 +5.3 29.06 +5.3 29.04 perf-profile.calltrace.cycles-pp.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64 17.89 ± 3% +10.1 27.95 +6.5 24.39 +6.6 24.46 perf-profile.calltrace.cycles-pp.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 13.68 ± 4% +12.1 25.73 +7.3 20.95 +7.4 21.07 perf-profile.calltrace.cycles-pp.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write 11.76 ± 5% +13.0 24.76 +7.7 19.42 +7.8 19.56 perf-profile.calltrace.cycles-pp.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 9.80 ± 6% +13.4 23.16 +8.1 17.87 +8.2 18.03 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio 9.81 ± 6% +13.4 23.17 +8.1 17.87 +8.2 18.04 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio 9.79 ± 6% +13.4 23.16 +8.1 17.86 +8.2 18.02 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru 10.38 ± 6% +13.5 23.85 +8.1 18.48 +8.3 18.65 perf-profile.calltrace.cycles-pp.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 10.32 ± 6% +13.5 23.82 +8.1 18.43 +8.3 18.60 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin 8.04 ± 8% -7.2 0.84 ± 9% -6.0 2.06 ± 17% -5.8 2.20 ± 9% perf-profile.children.cycles-pp.down_write 7.67 ± 8% -7.0 0.65 ± 12% -5.9 1.77 ± 19% -5.8 1.91 ± 10% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 7.66 ± 8% -7.0 0.64 ± 12% -5.9 1.76 ± 19% -5.8 1.90 ± 10% perf-profile.children.cycles-pp.rwsem_optimistic_spin 7.28 ± 9% -6.8 0.53 ± 14% -5.7 1.54 ± 21% -5.6 1.68 ± 11% perf-profile.children.cycles-pp.osq_lock 4.35 ± 10% -4.1 0.30 ± 8% -3.5 0.81 ± 19% -3.5 0.84 ± 11% perf-profile.children.cycles-pp.unlink 4.34 ± 10% -4.0 0.29 ± 8% -3.5 0.80 ± 19% -3.5 0.84 ± 11% perf-profile.children.cycles-pp.__x64_sys_unlink 4.34 ± 10% -4.0 0.29 ± 8% -3.5 0.80 ± 20% -3.5 0.83 ± 11% perf-profile.children.cycles-pp.do_unlinkat 3.97 ± 6% -3.3 0.71 ± 8% -2.5 1.47 ± 13% -2.4 1.57 ± 7% perf-profile.children.cycles-pp.do_sys_openat2 3.95 ± 6% -3.2 0.70 ± 8% -2.5 1.45 ± 13% -2.4 1.55 ± 7% perf-profile.children.cycles-pp.path_openat 3.95 ± 6% -3.2 0.70 ± 8% -2.5 1.45 ± 13% -2.4 1.55 ± 7% perf-profile.children.cycles-pp.do_filp_open 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -2.4 1.54 ± 7% perf-profile.children.cycles-pp.creat64 3.92 ± 6% -3.2 0.68 ± 8% -2.5 1.43 ± 13% -2.4 1.53 ± 7% perf-profile.children.cycles-pp.__x64_sys_creat 3.87 ± 6% -3.2 0.66 ± 8% -2.5 1.39 ± 14% -2.4 1.49 ± 7% perf-profile.children.cycles-pp.open_last_lookups 3.70 -1.7 1.98 ± 2% -0.7 3.05 -0.7 3.01 perf-profile.children.cycles-pp.llseek 3.68 -1.7 2.00 ± 2% -0.6 3.08 -0.7 3.03 perf-profile.children.cycles-pp.ext4_block_write_begin 3.12 -1.4 1.67 ± 2% -0.5 2.57 -0.6 2.53 perf-profile.children.cycles-pp.clear_bhb_loop 2.90 -1.3 1.64 ± 2% -0.5 2.36 -0.6 2.31 perf-profile.children.cycles-pp.ext4_da_write_end 2.29 -1.0 1.31 ± 2% -0.4 1.85 -0.5 1.82 perf-profile.children.cycles-pp.block_write_end 2.20 -0.9 1.26 ± 2% -0.4 1.78 -0.4 1.75 perf-profile.children.cycles-pp.__block_commit_write 1.95 -0.8 1.15 ± 2% -0.3 1.68 -0.3 1.63 perf-profile.children.cycles-pp.copy_page_to_iter 1.99 -0.8 1.24 ± 2% -0.4 1.64 -0.4 1.59 perf-profile.children.cycles-pp.truncate_cleanup_folio 1.80 -0.7 1.08 ± 2% -0.2 1.56 -0.3 1.51 perf-profile.children.cycles-pp._copy_to_iter 1.54 -0.7 0.82 ± 2% -0.3 1.27 -0.3 1.25 perf-profile.children.cycles-pp.entry_SYSCALL_64 1.34 -0.7 0.68 ± 3% -0.3 1.06 -0.3 1.04 perf-profile.children.cycles-pp.ext4_da_get_block_prep 1.32 -0.7 0.67 ± 3% -0.3 1.04 -0.3 1.02 perf-profile.children.cycles-pp.ext4_da_map_blocks 31.04 -0.6 30.46 +0.6 31.63 +0.6 31.66 perf-profile.children.cycles-pp.read 1.19 -0.6 0.64 ± 2% -0.2 1.01 -0.2 1.00 perf-profile.children.cycles-pp.filemap_get_pages 1.18 -0.5 0.63 ± 2% -0.2 0.97 -0.2 0.96 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 1.22 -0.5 0.68 ± 2% -0.3 0.92 -0.3 0.90 perf-profile.children.cycles-pp.mark_buffer_dirty 1.20 -0.5 0.70 ± 2% -0.2 1.02 -0.2 1.00 perf-profile.children.cycles-pp.zero_user_segments 1.20 -0.5 0.71 ± 2% -0.2 1.03 -0.2 1.01 perf-profile.children.cycles-pp.memset_orig 1.07 -0.5 0.58 ± 2% -0.2 0.87 -0.2 0.85 perf-profile.children.cycles-pp.copy_page_from_iter_atomic 0.95 -0.5 0.46 ± 3% -0.2 0.78 ± 4% -0.2 0.74 ± 2% perf-profile.children.cycles-pp.rw_verify_area 1.32 -0.5 0.84 ± 2% -0.3 1.06 -0.3 1.04 perf-profile.children.cycles-pp.try_to_free_buffers 0.98 -0.5 0.53 ± 2% -0.1 0.85 -0.1 0.84 perf-profile.children.cycles-pp.filemap_get_read_batch 0.98 ± 2% -0.4 0.53 ± 3% -0.2 0.80 -0.2 0.79 perf-profile.children.cycles-pp.__fdget_pos 0.77 ± 2% -0.4 0.34 ± 3% -0.2 0.58 ± 4% -0.2 0.56 ± 3% perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited_flags 1.10 ± 3% -0.4 0.67 ± 3% -0.3 0.80 ± 3% -0.3 0.79 ± 2% perf-profile.children.cycles-pp.workingset_activation 0.80 -0.4 0.37 ± 2% -0.2 0.60 -0.2 0.59 ± 2% perf-profile.children.cycles-pp.file_modified 0.86 ± 4% -0.4 0.46 ± 3% -0.2 0.62 ± 3% -0.2 0.61 ± 3% perf-profile.children.cycles-pp.workingset_age_nonresident 0.77 ± 2% -0.4 0.37 ± 3% -0.1 0.64 ± 5% -0.2 0.60 ± 2% perf-profile.children.cycles-pp.security_file_permission 0.92 -0.4 0.52 ± 2% -0.2 0.68 -0.2 0.67 perf-profile.children.cycles-pp.__folio_mark_dirty 0.80 -0.4 0.42 ± 2% -0.1 0.65 -0.2 0.64 perf-profile.children.cycles-pp.xas_load 0.74 -0.4 0.37 ± 3% -0.2 0.58 -0.2 0.57 perf-profile.children.cycles-pp.folio_alloc_noprof 0.70 -0.4 0.33 ± 3% -0.2 0.54 -0.2 0.54 perf-profile.children.cycles-pp.touch_atime 0.60 ± 2% -0.4 0.24 ± 4% -0.2 0.42 -0.2 0.42 ± 2% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.71 -0.4 0.35 ± 3% -0.2 0.55 -0.2 0.54 perf-profile.children.cycles-pp.alloc_pages_mpol_noprof 0.80 -0.4 0.45 ± 2% -0.1 0.74 -0.1 0.72 perf-profile.children.cycles-pp.create_empty_buffers 0.68 ± 2% -0.3 0.36 ± 3% -0.1 0.54 -0.1 0.55 ± 2% perf-profile.children.cycles-pp.fault_in_iov_iter_readable 0.62 -0.3 0.30 ± 3% -0.1 0.48 -0.1 0.48 perf-profile.children.cycles-pp.__alloc_pages_noprof 0.59 ± 2% -0.3 0.28 ± 3% -0.1 0.48 ± 7% -0.1 0.45 ± 2% perf-profile.children.cycles-pp.apparmor_file_permission 0.62 -0.3 0.32 ± 3% -0.1 0.49 ± 2% -0.1 0.49 perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.64 -0.3 0.34 ± 3% -0.1 0.52 -0.1 0.52 perf-profile.children.cycles-pp.ksys_lseek 1.01 -0.3 0.71 ± 2% -0.4 0.62 -0.4 0.61 perf-profile.children.cycles-pp.__filemap_add_folio 0.62 -0.3 0.32 ± 3% -0.1 0.51 -0.1 0.50 perf-profile.children.cycles-pp.filemap_get_entry 0.57 -0.3 0.28 ± 3% -0.1 0.45 -0.1 0.44 perf-profile.children.cycles-pp.atime_needs_update 0.69 -0.3 0.40 ± 2% -0.2 0.50 -0.2 0.48 perf-profile.children.cycles-pp.folio_account_dirtied 0.61 -0.3 0.32 ± 2% -0.1 0.50 ± 3% -0.1 0.49 ± 2% perf-profile.children.cycles-pp.__cond_resched 0.60 ± 2% -0.3 0.32 ± 3% -0.1 0.48 -0.1 0.48 ± 2% perf-profile.children.cycles-pp.fault_in_readable 0.62 -0.3 0.35 ± 2% -0.0 0.59 -0.0 0.58 perf-profile.children.cycles-pp.folio_alloc_buffers 0.58 -0.3 0.32 ± 2% -0.1 0.48 -0.1 0.47 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.37 ± 3% -0.3 0.11 ± 4% -0.2 0.22 ± 6% -0.2 0.22 ± 3% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.40 ± 6% -0.3 0.14 ± 3% -0.1 0.26 ± 5% -0.1 0.25 ± 6% perf-profile.children.cycles-pp.ext4_file_write_iter 0.51 ± 3% -0.2 0.27 ± 3% -0.1 0.40 ± 2% -0.1 0.40 ± 2% perf-profile.children.cycles-pp.disk_rr 0.59 -0.2 0.34 ± 2% -0.1 0.49 -0.1 0.47 perf-profile.children.cycles-pp.kmem_cache_free 0.46 -0.2 0.22 ± 2% -0.1 0.36 -0.1 0.35 perf-profile.children.cycles-pp.get_page_from_freelist 0.55 ± 2% -0.2 0.31 ± 3% -0.0 0.52 -0.0 0.51 perf-profile.children.cycles-pp.alloc_buffer_head 0.53 -0.2 0.30 ± 2% -0.0 0.51 -0.0 0.50 perf-profile.children.cycles-pp.kmem_cache_alloc_noprof 0.36 ± 3% -0.2 0.14 ± 4% -0.1 0.25 ± 2% -0.1 0.24 ± 3% perf-profile.children.cycles-pp.__mark_inode_dirty 0.36 ± 6% -0.2 0.14 ± 5% -0.1 0.25 ± 4% -0.1 0.25 ± 2% perf-profile.children.cycles-pp.ext4_file_read_iter 0.40 -0.2 0.20 ± 2% -0.1 0.33 ± 2% -0.1 0.33 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.44 -0.2 0.25 ± 3% -0.1 0.34 -0.1 0.33 ± 2% perf-profile.children.cycles-pp.xas_store 0.39 -0.2 0.19 ± 3% -0.1 0.31 ± 3% -0.1 0.30 perf-profile.children.cycles-pp.inode_needs_update_time 0.52 -0.2 0.33 ± 2% -0.1 0.42 -0.1 0.42 perf-profile.children.cycles-pp.delete_from_page_cache_batch 0.29 ± 4% -0.2 0.10 ± 6% -0.1 0.19 ± 3% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.generic_update_time 0.36 ± 2% -0.2 0.18 ± 4% -0.1 0.28 -0.1 0.28 ± 2% perf-profile.children.cycles-pp.ext4_da_reserve_space 0.57 ± 2% -0.2 0.40 ± 3% -0.1 0.44 -0.1 0.44 perf-profile.children.cycles-pp.__folio_cancel_dirty 0.34 ± 2% -0.2 0.18 ± 3% -0.1 0.29 ± 2% -0.1 0.28 ± 2% perf-profile.children.cycles-pp.__mem_cgroup_charge 0.25 ± 4% -0.2 0.09 ± 5% -0.1 0.16 ± 4% -0.1 0.16 ± 5% perf-profile.children.cycles-pp.ext4_dirty_inode 0.33 -0.2 0.17 ± 2% -0.1 0.25 -0.1 0.25 perf-profile.children.cycles-pp.ext4_es_insert_delayed_block 0.34 -0.2 0.18 ± 2% -0.1 0.27 ± 2% -0.1 0.26 perf-profile.children.cycles-pp.ext4_generic_write_checks 0.22 ± 5% -0.2 0.06 ± 9% -0.1 0.13 ± 3% -0.1 0.13 ± 7% perf-profile.children.cycles-pp.jbd2__journal_start 0.21 ± 5% -0.2 0.06 ± 8% -0.1 0.12 ± 5% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.start_this_handle 0.31 -0.1 0.17 ± 3% -0.1 0.25 ± 2% -0.1 0.24 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 0.40 -0.1 0.25 ± 2% -0.1 0.30 ± 2% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.30 ± 2% -0.1 0.17 ± 3% -0.0 0.30 ± 3% -0.0 0.29 perf-profile.children.cycles-pp.__memcg_slab_post_alloc_hook 0.31 ± 2% -0.1 0.18 ± 2% -0.1 0.26 ± 2% -0.1 0.26 perf-profile.children.cycles-pp.block_invalidate_folio 0.29 ± 2% -0.1 0.16 ± 2% -0.0 0.24 ± 2% -0.1 0.24 ± 2% perf-profile.children.cycles-pp.x64_sys_call 0.49 -0.1 0.36 ± 3% -0.1 0.38 -0.1 0.37 perf-profile.children.cycles-pp.folio_account_cleaned 0.32 -0.1 0.19 ± 2% -0.1 0.25 ± 2% -0.1 0.25 perf-profile.children.cycles-pp.lookup_open 0.27 -0.1 0.14 ± 2% -0.1 0.22 ± 2% -0.1 0.21 perf-profile.children.cycles-pp.generic_write_checks 0.26 ± 3% -0.1 0.13 ± 3% -0.1 0.21 ± 3% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.rcu_all_qs 0.26 ± 2% -0.1 0.13 ± 4% -0.1 0.20 -0.1 0.20 perf-profile.children.cycles-pp.ext4_es_lookup_extent 0.27 ± 2% -0.1 0.14 ± 4% -0.1 0.22 ± 2% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.up_write 0.27 -0.1 0.14 ± 3% -0.0 0.22 -0.0 0.22 ± 2% perf-profile.children.cycles-pp.xas_start 0.35 ± 2% -0.1 0.22 -0.1 0.27 ± 2% -0.1 0.27 ± 2% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.17 ± 4% -0.1 0.06 ± 6% -0.1 0.12 ± 4% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.ext4_nonda_switch 0.23 -0.1 0.12 ± 3% -0.0 0.18 -0.0 0.18 perf-profile.children.cycles-pp.folio_unlock 0.22 ± 2% -0.1 0.12 ± 3% -0.0 0.18 ± 2% -0.0 0.18 ± 2% perf-profile.children.cycles-pp.current_time 0.27 -0.1 0.16 ± 3% -0.0 0.23 ± 2% -0.0 0.22 perf-profile.children.cycles-pp.__memcg_slab_free_hook 0.27 ± 2% -0.1 0.17 ± 3% -0.1 0.20 ± 2% -0.1 0.20 ± 2% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.18 -0.1 0.08 ± 5% -0.0 0.14 ± 3% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.node_dirty_ok 0.21 ± 2% -0.1 0.12 ± 4% -0.0 0.17 -0.0 0.17 ± 2% perf-profile.children.cycles-pp.__slab_free 0.26 ± 3% -0.1 0.16 ± 2% -0.1 0.20 ± 2% -0.1 0.20 ± 3% perf-profile.children.cycles-pp.hrtimer_interrupt 0.93 ± 2% -0.1 0.84 ± 3% -0.2 0.72 ± 2% -0.2 0.70 perf-profile.children.cycles-pp.__lruvec_stat_mod_folio 0.18 ± 2% -0.1 0.09 ± 3% -0.0 0.15 ± 3% -0.0 0.14 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.19 ± 2% -0.1 0.10 ± 5% -0.0 0.14 ± 3% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.aa_file_perm 0.20 -0.1 0.11 ± 2% -0.0 0.16 ± 2% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.__mod_node_page_state 0.21 -0.1 0.12 ± 2% -0.0 0.16 ± 2% -0.0 0.16 ± 2% perf-profile.children.cycles-pp.ext4_create 0.18 -0.1 0.09 ± 4% -0.0 0.14 -0.0 0.14 perf-profile.children.cycles-pp.rmqueue 0.20 ± 2% -0.1 0.12 ± 4% -0.0 0.17 -0.0 0.17 perf-profile.children.cycles-pp.find_lock_entries 0.17 ± 5% -0.1 0.08 ± 5% -0.0 0.14 ± 7% -0.0 0.13 ± 6% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.19 ± 2% -0.1 0.10 ± 4% -0.0 0.15 ± 3% -0.0 0.15 perf-profile.children.cycles-pp.cgroup_rstat_updated 0.17 -0.1 0.09 ± 4% -0.0 0.13 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.__dquot_alloc_space 0.08 -0.1 0.00 -0.0 0.06 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.__mod_zone_page_state 0.08 ± 5% -0.1 0.00 ±316% -0.0 0.06 ± 10% -0.0 0.06 ± 6% perf-profile.children.cycles-pp.write@plt 0.21 ± 2% -0.1 0.13 ± 3% -0.1 0.16 -0.1 0.16 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.08 ± 5% -0.1 0.00 -0.0 0.06 ± 7% -0.0 0.06 ± 5% perf-profile.children.cycles-pp._raw_read_lock 0.08 ± 5% -0.1 0.00 -0.0 0.06 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.rcu_core 0.17 -0.1 0.09 ± 4% -0.0 0.14 ± 2% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.free_unref_folios 0.09 ± 3% -0.1 0.01 ±163% -0.0 0.07 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.generic_file_llseek_size 0.08 ± 6% -0.1 0.00 -0.0 0.06 ± 5% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.rcu_do_batch 0.20 ± 3% -0.1 0.13 ± 3% -0.0 0.15 ± 2% -0.0 0.15 ± 2% perf-profile.children.cycles-pp.tick_nohz_handler 0.16 ± 2% -0.1 0.09 ± 4% -0.0 0.13 ± 3% -0.0 0.13 ± 2% perf-profile.children.cycles-pp.vfs_unlink 0.08 ± 4% -0.1 0.00 ±316% -0.0 0.06 ± 5% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.xas_create 0.09 ± 5% -0.1 0.01 ±163% -0.0 0.07 ± 4% -0.0 0.07 ± 4% perf-profile.children.cycles-pp.generic_file_read_iter 0.19 -0.1 0.12 ± 4% -0.0 0.16 ± 2% -0.0 0.15 ± 2% perf-profile.children.cycles-pp.mod_objcg_state 0.14 ± 3% -0.1 0.07 ± 6% -0.0 0.11 ± 4% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.__es_insert_extent 0.16 ± 3% -0.1 0.08 ± 5% -0.0 0.12 ± 2% -0.0 0.12 ± 2% perf-profile.children.cycles-pp.__ext4_unlink 0.14 ± 2% -0.1 0.07 -0.0 0.11 -0.0 0.11 perf-profile.children.cycles-pp.__radix_tree_lookup 0.14 ± 4% -0.1 0.07 -0.0 0.11 ± 2% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.__count_memcg_events 0.07 ± 4% -0.1 0.00 -0.0 0.06 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.__es_remove_extent 0.07 ± 4% -0.1 0.00 -0.0 0.06 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.read@plt 0.16 ± 3% -0.1 0.09 ± 5% -0.0 0.12 ± 2% -0.0 0.12 ± 2% perf-profile.children.cycles-pp.ext4_unlink 0.08 ± 6% -0.1 0.00 ±316% -0.0 0.06 ± 5% -0.0 0.06 perf-profile.children.cycles-pp.xas_clear_mark 0.08 ± 5% -0.1 0.01 ±163% -0.0 0.07 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.folio_mapping 0.07 -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.children.cycles-pp.node_page_state 0.07 -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.children.cycles-pp.ext4_fill_raw_inode 0.18 ± 2% -0.1 0.12 ± 4% -0.0 0.14 -0.0 0.14 ± 2% perf-profile.children.cycles-pp.update_process_times 0.15 ± 3% -0.1 0.08 ± 3% -0.0 0.12 ± 4% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers 0.12 ± 5% -0.1 0.05 ± 8% -0.0 0.09 ± 3% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.ext4_claim_free_clusters 0.07 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.children.cycles-pp.folio_wait_stable 0.07 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.children.cycles-pp.free_unref_page_commit 0.06 ± 7% -0.1 0.00 -0.0 0.05 ± 6% -0.0 0.05 perf-profile.children.cycles-pp.balance_dirty_pages 0.13 -0.1 0.07 ± 7% -0.0 0.10 ± 3% -0.0 0.10 ± 3% perf-profile.children.cycles-pp.generic_write_check_limits 0.12 ± 4% -0.1 0.06 ± 7% -0.0 0.10 -0.0 0.10 perf-profile.children.cycles-pp.__xa_set_mark 0.12 ± 2% -0.1 0.06 -0.0 0.10 ± 5% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.file_remove_privs_flags 0.13 ± 3% -0.1 0.07 ± 7% -0.0 0.10 ± 3% -0.0 0.10 perf-profile.children.cycles-pp.ext4_llseek 0.07 -0.1 0.01 ±212% -0.0 0.06 ± 9% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.irq_exit_rcu 0.06 -0.1 0.00 -0.0 0.04 ± 37% -0.0 0.04 ± 57% perf-profile.children.cycles-pp.add_dirent_to_buf 0.06 -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.children.cycles-pp._raw_spin_trylock 0.06 -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.children.cycles-pp.bdev_getblk 0.06 -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.children.cycles-pp.crc32c_pcl_intel_update 0.15 ± 2% -0.1 0.09 ± 4% -0.0 0.12 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.__ext4_mark_inode_dirty 0.12 -0.1 0.06 ± 6% -0.0 0.10 ± 4% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.amd_clear_divider 0.12 ± 2% -0.1 0.06 ± 6% -0.0 0.09 ± 5% -0.0 0.09 ± 4% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head 0.11 ± 5% -0.1 0.06 ± 6% -0.0 0.09 ± 5% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.inode_to_bdi 0.09 ± 5% -0.1 0.04 ± 47% -0.0 0.07 -0.0 0.07 ± 4% perf-profile.children.cycles-pp.handle_softirqs 0.50 -0.1 0.45 ± 3% +0.0 0.55 +0.1 0.56 perf-profile.children.cycles-pp.folio_activate_fn 0.11 ± 2% -0.1 0.06 -0.0 0.09 -0.0 0.09 perf-profile.children.cycles-pp.__ext4_new_inode 0.12 ± 2% -0.1 0.07 -0.0 0.10 -0.0 0.10 perf-profile.children.cycles-pp.try_charge_memcg 0.11 ± 4% -0.1 0.06 ± 4% -0.0 0.09 ± 3% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.timestamp_truncate 0.12 ± 3% -0.1 0.07 ± 6% -0.0 0.10 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.12 ± 3% -0.1 0.07 ± 6% -0.0 0.10 ± 3% -0.0 0.10 perf-profile.children.cycles-pp.drop_buffers 0.13 ± 3% -0.0 0.08 ± 3% -0.0 0.10 ± 3% -0.0 0.10 perf-profile.children.cycles-pp.__ext4_find_entry 0.11 ± 3% -0.0 0.06 ± 6% -0.0 0.09 ± 3% -0.0 0.09 perf-profile.children.cycles-pp.ext4_mark_iloc_dirty 0.12 ± 8% -0.0 0.08 ± 6% +0.1 0.20 ± 3% +0.1 0.20 ± 3% perf-profile.children.cycles-pp.mem_cgroup_update_lru_size 0.12 -0.0 0.08 ± 5% -0.0 0.10 ± 5% -0.0 0.10 ± 5% perf-profile.children.cycles-pp.ext4_dx_find_entry 0.13 ± 3% -0.0 0.09 -0.0 0.11 ± 4% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.sched_tick 0.10 ± 4% -0.0 0.06 ± 7% -0.0 0.08 ± 4% -0.0 0.08 perf-profile.children.cycles-pp.ext4_do_update_inode 0.09 -0.0 0.05 -0.0 0.07 -0.0 0.07 perf-profile.children.cycles-pp.__mem_cgroup_uncharge_folios 0.08 ± 4% -0.0 0.04 ± 47% -0.0 0.06 ± 7% -0.0 0.06 ± 5% perf-profile.children.cycles-pp.ext4_reserve_inode_write 0.17 -0.0 0.13 ± 3% -0.0 0.13 ± 3% -0.0 0.13 ± 2% perf-profile.children.cycles-pp.filemap_unaccount_folio 0.09 ± 4% -0.0 0.05 ± 9% -0.0 0.07 ± 4% -0.0 0.07 ± 4% perf-profile.children.cycles-pp.ext4_lookup 0.09 ± 4% -0.0 0.06 -0.0 0.07 -0.0 0.07 perf-profile.children.cycles-pp.task_tick_fair 0.09 ± 3% -0.0 0.06 ± 8% -0.0 0.07 -0.0 0.07 ± 4% perf-profile.children.cycles-pp.ext4_add_nondir 0.24 ± 2% -0.0 0.22 ± 2% -0.2 0.06 -0.2 0.06 perf-profile.children.cycles-pp.xas_find_conflict 0.08 ± 4% -0.0 0.05 -0.0 0.06 -0.0 0.06 perf-profile.children.cycles-pp.ext4_add_entry 0.08 ± 5% -0.0 0.05 -0.0 0.06 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.ext4_dx_add_entry 0.13 ± 3% -0.0 0.11 ± 5% -0.0 0.11 -0.0 0.11 ± 3% perf-profile.children.cycles-pp.__mod_lruvec_state 0.07 -0.0 0.05 ± 31% -0.0 0.05 ± 6% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.ext4_search_dir 0.26 ± 3% -0.0 0.24 ± 8% +0.1 0.36 ± 4% +0.1 0.35 ± 5% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.26 ± 3% -0.0 0.24 ± 8% +0.1 0.36 ± 4% +0.1 0.35 ± 5% perf-profile.children.cycles-pp.do_group_exit 0.26 ± 3% -0.0 0.24 ± 9% +0.1 0.36 ± 3% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.do_exit 0.24 ± 4% -0.0 0.24 ± 8% +0.1 0.35 ± 4% +0.1 0.34 ± 5% perf-profile.children.cycles-pp.exit_mm 0.25 ± 3% -0.0 0.25 ± 8% +0.1 0.35 ± 4% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.__mmput 0.25 ± 3% -0.0 0.25 ± 8% +0.1 0.35 ± 4% +0.1 0.35 ± 4% perf-profile.children.cycles-pp.exit_mmap 0.20 ± 3% +0.0 0.22 ± 9% +0.1 0.30 ± 4% +0.1 0.30 ± 5% perf-profile.children.cycles-pp.__tlb_batch_free_encoded_pages 0.20 ± 3% +0.0 0.22 ± 9% +0.1 0.30 ± 4% +0.1 0.30 ± 5% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.20 ± 3% +0.0 0.22 ± 9% +0.1 0.31 ± 4% +0.1 0.30 ± 5% perf-profile.children.cycles-pp.tlb_finish_mmu 0.34 ± 2% +0.0 0.37 -0.0 0.32 -0.0 0.31 perf-profile.children.cycles-pp.lru_add_fn 0.00 +0.1 0.05 ± 5% +0.1 0.05 +0.1 0.05 perf-profile.children.cycles-pp.lru_add_drain 0.08 ± 5% +0.1 0.15 ± 4% +0.0 0.11 +0.0 0.11 ± 4% perf-profile.children.cycles-pp.__cmd_record 0.08 ± 5% +0.1 0.15 ± 4% +0.0 0.11 +0.0 0.11 ± 4% perf-profile.children.cycles-pp.cmd_record 0.08 ± 4% +0.1 0.14 ± 4% +0.0 0.10 ± 4% +0.0 0.10 ± 3% perf-profile.children.cycles-pp.record__mmap_read_evlist 0.09 ± 3% +0.1 0.15 ± 4% +0.0 0.11 ± 2% +0.0 0.11 ± 2% perf-profile.children.cycles-pp.main 0.09 ± 3% +0.1 0.15 ± 4% +0.0 0.11 ± 2% +0.0 0.11 ± 2% perf-profile.children.cycles-pp.run_builtin 0.08 ± 6% +0.1 0.14 ± 4% +0.0 0.10 +0.0 0.10 ± 5% perf-profile.children.cycles-pp.perf_mmap__push 0.07 ± 4% +0.1 0.13 ± 3% +0.0 0.09 ± 3% +0.0 0.09 ± 3% perf-profile.children.cycles-pp.record__pushfn 0.07 ± 4% +0.1 0.13 ± 3% +0.0 0.09 ± 3% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.writen 0.06 +0.1 0.13 ± 5% +0.0 0.09 ± 4% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.shmem_file_write_iter 0.00 +0.1 0.10 ± 4% +0.1 0.06 ± 6% +0.1 0.06 ± 5% perf-profile.children.cycles-pp.shmem_alloc_and_add_folio 0.00 +0.1 0.11 ± 6% +0.1 0.07 ± 7% +0.1 0.07 ± 6% perf-profile.children.cycles-pp.shmem_get_folio_gfp 0.00 +0.1 0.11 ± 6% +0.1 0.07 ± 7% +0.1 0.07 ± 6% perf-profile.children.cycles-pp.shmem_write_begin 2.45 +0.2 2.70 +0.1 2.55 +0.1 2.58 perf-profile.children.cycles-pp.lru_add_drain_cpu 4.86 +0.7 5.54 +0.3 5.13 +0.3 5.16 perf-profile.children.cycles-pp.__folio_batch_release 27.93 +0.9 28.79 +1.1 29.05 +1.2 29.12 perf-profile.children.cycles-pp.ksys_read 27.50 +1.1 28.56 +1.2 28.70 +1.3 28.77 perf-profile.children.cycles-pp.vfs_read 25.97 +1.8 27.81 +1.5 27.48 +1.6 27.59 perf-profile.children.cycles-pp.filemap_read 92.83 +3.3 96.09 +1.3 94.11 +1.4 94.19 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 92.42 +3.5 95.88 +1.4 93.79 +1.5 93.87 perf-profile.children.cycles-pp.do_syscall_64 30.73 +3.9 34.66 +3.8 34.58 +3.7 34.46 perf-profile.children.cycles-pp.write 21.08 +4.1 25.15 +2.3 23.40 +2.5 23.61 perf-profile.children.cycles-pp.folio_mark_accessed 19.70 +4.6 24.33 +2.7 22.38 +2.9 22.60 perf-profile.children.cycles-pp.folio_activate 27.83 +5.3 33.14 +4.4 32.25 +4.3 32.17 perf-profile.children.cycles-pp.ksys_write 27.36 +5.5 32.89 +4.5 31.87 +4.4 31.80 perf-profile.children.cycles-pp.vfs_write 25.87 +5.6 31.48 +2.2 28.03 +2.2 28.03 perf-profile.children.cycles-pp.__close 25.86 +5.6 31.46 +2.2 28.01 +2.2 28.01 perf-profile.children.cycles-pp.__x64_sys_close 25.84 +5.6 31.46 +2.2 28.00 +2.2 28.00 perf-profile.children.cycles-pp.__fput 25.83 +5.6 31.45 +2.2 27.99 +2.2 27.99 perf-profile.children.cycles-pp.dput 25.82 +5.6 31.44 +2.2 27.98 +2.2 27.98 perf-profile.children.cycles-pp.__dentry_kill 25.79 +5.6 31.43 +2.2 27.96 +2.2 27.96 perf-profile.children.cycles-pp.evict 25.78 +5.6 31.42 +2.2 27.95 +2.2 27.95 perf-profile.children.cycles-pp.ext4_evict_inode 25.64 +5.7 31.34 +2.2 27.83 +2.2 27.83 perf-profile.children.cycles-pp.truncate_inode_pages_range 18.31 +6.0 24.35 +2.5 20.82 +2.5 20.84 perf-profile.children.cycles-pp.folios_put_refs 17.78 +6.3 24.04 +2.6 20.37 +2.6 20.39 perf-profile.children.cycles-pp.__page_cache_release 25.54 +6.4 31.96 +4.9 30.47 +4.9 30.43 perf-profile.children.cycles-pp.ext4_buffered_write_iter 23.87 ± 2% +7.4 31.23 +5.3 29.21 +5.3 29.20 perf-profile.children.cycles-pp.generic_perform_write 17.93 ± 3% +10.0 27.97 +6.5 24.43 +6.6 24.49 perf-profile.children.cycles-pp.ext4_da_write_begin 13.75 ± 4% +12.0 25.77 +7.3 21.00 +7.4 21.12 perf-profile.children.cycles-pp.__filemap_get_folio 11.78 ± 5% +13.0 24.77 +7.7 19.43 +7.8 19.57 perf-profile.children.cycles-pp.filemap_add_folio 10.41 ± 6% +13.5 23.96 +8.1 18.54 +8.3 18.71 perf-profile.children.cycles-pp.folio_add_lru 34.89 ± 2% +18.9 53.82 +11.1 46.00 +11.5 46.42 perf-profile.children.cycles-pp.folio_batch_move_lru 51.11 +25.1 76.23 +13.7 64.78 +14.1 65.22 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 51.02 +25.2 76.18 +13.7 64.70 +14.1 65.14 perf-profile.children.cycles-pp.folio_lruvec_lock_irqsave 50.98 +25.3 76.24 +13.7 64.69 +14.2 65.15 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 7.25 ± 9% -6.7 0.53 ± 14% -5.7 1.54 ± 21% -5.6 1.67 ± 11% perf-profile.self.cycles-pp.osq_lock 3.09 -1.4 1.65 ± 2% -0.5 2.55 -0.6 2.50 perf-profile.self.cycles-pp.clear_bhb_loop 1.79 -0.7 1.07 ± 2% -0.2 1.54 -0.3 1.50 perf-profile.self.cycles-pp._copy_to_iter 1.14 -0.5 0.61 ± 2% -0.2 0.94 -0.2 0.92 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 1.20 -0.5 0.70 ± 2% -0.2 1.02 -0.2 1.00 perf-profile.self.cycles-pp.memset_orig 1.06 -0.5 0.57 ± 2% -0.2 0.86 -0.2 0.84 perf-profile.self.cycles-pp.copy_page_from_iter_atomic 0.97 -0.5 0.51 ± 2% -0.2 0.79 -0.2 0.78 perf-profile.self.cycles-pp.filemap_read 0.94 ± 2% -0.4 0.51 ± 3% -0.2 0.78 -0.2 0.76 perf-profile.self.cycles-pp.__fdget_pos 0.86 ± 4% -0.4 0.45 ± 3% -0.2 0.62 ± 4% -0.2 0.61 ± 3% perf-profile.self.cycles-pp.workingset_age_nonresident 0.75 ± 2% -0.4 0.38 ± 3% -0.2 0.59 -0.2 0.58 ± 2% perf-profile.self.cycles-pp.vfs_write 0.89 -0.3 0.55 ± 2% -0.1 0.80 -0.1 0.78 perf-profile.self.cycles-pp.__block_commit_write 0.54 ± 3% -0.3 0.21 ± 4% -0.2 0.38 -0.2 0.37 ± 2% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.50 ± 3% -0.3 0.20 ± 3% -0.1 0.37 ± 6% -0.1 0.35 ± 5% perf-profile.self.cycles-pp.balance_dirty_pages_ratelimited_flags 0.63 -0.3 0.33 ± 2% -0.1 0.52 -0.1 0.51 perf-profile.self.cycles-pp.vfs_read 0.57 -0.3 0.29 ± 2% -0.1 0.46 -0.1 0.45 perf-profile.self.cycles-pp.do_syscall_64 0.61 -0.3 0.33 ± 2% -0.1 0.54 -0.1 0.53 perf-profile.self.cycles-pp.filemap_get_read_batch 0.58 ± 2% -0.3 0.31 ± 3% -0.1 0.46 -0.1 0.47 ± 3% perf-profile.self.cycles-pp.fault_in_readable 0.37 ± 3% -0.3 0.10 ± 4% -0.2 0.22 ± 6% -0.2 0.22 ± 3% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.56 -0.3 0.29 ± 3% -0.1 0.45 ± 2% -0.1 0.45 perf-profile.self.cycles-pp.xas_load 0.38 ± 6% -0.3 0.13 ± 4% -0.1 0.25 ± 6% -0.1 0.24 ± 6% perf-profile.self.cycles-pp.ext4_file_write_iter 0.48 ± 2% -0.2 0.24 ± 4% -0.1 0.37 ± 2% -0.1 0.37 perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.35 ± 5% -0.2 0.13 ± 5% -0.1 0.24 ± 4% -0.1 0.24 ± 2% perf-profile.self.cycles-pp.ext4_file_read_iter 0.46 ± 2% -0.2 0.24 ± 2% -0.1 0.36 ± 3% -0.1 0.36 ± 2% perf-profile.self.cycles-pp.write 0.45 ± 2% -0.2 0.23 ± 6% -0.1 0.35 ± 4% -0.1 0.34 ± 5% perf-profile.self.cycles-pp.disk_rr 0.39 ± 3% -0.2 0.18 ± 4% -0.1 0.33 ± 10% -0.1 0.30 ± 4% perf-profile.self.cycles-pp.apparmor_file_permission 0.36 ± 4% -0.2 0.15 ± 4% -0.1 0.26 ± 3% -0.1 0.26 ± 2% perf-profile.self.cycles-pp.ext4_da_write_begin 0.44 -0.2 0.23 ± 2% -0.1 0.37 -0.1 0.35 perf-profile.self.cycles-pp.ext4_da_write_end 0.44 -0.2 0.23 ± 2% -0.1 0.36 -0.1 0.35 perf-profile.self.cycles-pp.__filemap_get_folio 0.42 -0.2 0.21 ± 2% -0.1 0.34 -0.1 0.33 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.42 ± 2% -0.2 0.22 ± 3% -0.1 0.33 ± 2% -0.1 0.33 perf-profile.self.cycles-pp.generic_perform_write 0.40 -0.2 0.20 ± 2% -0.1 0.33 ± 2% -0.1 0.32 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.43 ± 2% -0.2 0.24 ± 3% -0.1 0.36 ± 2% -0.1 0.35 ± 2% perf-profile.self.cycles-pp.read 0.40 ± 2% -0.2 0.21 ± 3% -0.1 0.32 ± 2% -0.1 0.32 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.41 -0.2 0.22 ± 3% -0.1 0.34 -0.1 0.34 perf-profile.self.cycles-pp.llseek 0.33 -0.2 0.17 ± 2% -0.1 0.26 ± 2% -0.1 0.25 perf-profile.self.cycles-pp.ext4_block_write_begin 0.33 ± 2% -0.2 0.17 ± 3% -0.1 0.26 -0.1 0.26 perf-profile.self.cycles-pp.__cond_resched 0.28 ± 2% -0.2 0.13 ± 3% -0.1 0.22 ± 2% -0.1 0.21 perf-profile.self.cycles-pp.atime_needs_update 0.30 ± 2% -0.1 0.16 ± 3% -0.1 0.24 -0.1 0.24 perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.30 -0.1 0.16 ± 4% -0.1 0.25 -0.1 0.25 perf-profile.self.cycles-pp.filemap_get_entry 0.30 -0.1 0.16 ± 3% -0.1 0.24 ± 2% -0.1 0.23 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 0.28 -0.1 0.14 ± 4% -0.1 0.22 -0.1 0.22 perf-profile.self.cycles-pp.folio_mark_accessed 0.27 -0.1 0.14 ± 2% -0.1 0.22 ± 2% -0.1 0.21 perf-profile.self.cycles-pp.mark_buffer_dirty 0.12 ± 5% -0.1 0.00 -0.0 0.07 ± 5% -0.1 0.07 ± 9% perf-profile.self.cycles-pp.start_this_handle 0.26 -0.1 0.14 ± 3% -0.1 0.21 ± 3% -0.1 0.20 ± 2% perf-profile.self.cycles-pp.ext4_da_map_blocks 0.25 -0.1 0.13 ± 3% -0.1 0.19 ± 2% -0.1 0.19 ± 3% perf-profile.self.cycles-pp.down_write 0.25 -0.1 0.14 ± 2% -0.0 0.21 ± 2% -0.0 0.21 ± 2% perf-profile.self.cycles-pp.x64_sys_call 0.27 ± 2% -0.1 0.16 ± 2% -0.0 0.22 ± 2% -0.1 0.22 perf-profile.self.cycles-pp.block_invalidate_folio 0.24 ± 2% -0.1 0.12 ± 3% -0.0 0.19 ± 3% -0.0 0.19 perf-profile.self.cycles-pp.up_write 0.16 ± 4% -0.1 0.06 ± 7% -0.1 0.11 ± 4% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.ext4_nonda_switch 0.20 ± 2% -0.1 0.10 -0.0 0.16 ± 2% -0.1 0.15 ± 2% perf-profile.self.cycles-pp.inode_needs_update_time 0.21 ± 2% -0.1 0.11 -0.0 0.17 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.folio_unlock 0.23 -0.1 0.13 ± 3% -0.1 0.17 -0.1 0.17 perf-profile.self.cycles-pp.xas_store 0.21 ± 2% -0.1 0.11 -0.0 0.18 ± 2% -0.0 0.17 ± 2% perf-profile.self.cycles-pp.xas_start 0.20 ± 2% -0.1 0.10 ± 4% -0.0 0.16 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.filemap_get_pages 0.20 ± 2% -0.1 0.10 ± 4% -0.0 0.16 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.security_file_permission 0.20 ± 4% -0.1 0.10 ± 4% -0.0 0.16 ± 3% -0.0 0.16 ± 3% perf-profile.self.cycles-pp.rcu_all_qs 0.22 -0.1 0.12 ± 3% -0.0 0.19 -0.0 0.19 ± 3% perf-profile.self.cycles-pp.folios_put_refs 0.20 ± 2% -0.1 0.11 ± 3% -0.0 0.16 ± 2% -0.0 0.16 perf-profile.self.cycles-pp.__slab_free 0.18 -0.1 0.09 ± 3% -0.0 0.13 -0.0 0.13 perf-profile.self.cycles-pp.__filemap_add_folio 0.17 ± 2% -0.1 0.09 ± 4% -0.0 0.13 ± 3% -0.0 0.13 ± 2% perf-profile.self.cycles-pp.ext4_buffered_write_iter 0.18 ± 2% -0.1 0.09 ± 5% -0.0 0.14 ± 2% -0.0 0.14 ± 2% perf-profile.self.cycles-pp.rw_verify_area 0.08 ± 4% -0.1 0.00 -0.0 0.07 ± 6% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.fault_in_iov_iter_readable 0.08 -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.self.cycles-pp.__es_insert_extent 0.08 -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.self.cycles-pp.ext4_es_insert_delayed_block 0.08 -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.self.cycles-pp.rmqueue 0.08 ± 8% -0.1 0.00 -0.0 0.06 ± 6% -0.0 0.06 ± 5% perf-profile.self.cycles-pp.inode_to_bdi 0.17 -0.1 0.09 ± 4% -0.0 0.13 ± 3% -0.0 0.13 ± 2% perf-profile.self.cycles-pp.__mod_node_page_state 0.08 ± 6% -0.1 0.00 -0.0 0.05 ± 9% -0.0 0.05 ± 8% perf-profile.self.cycles-pp._raw_read_lock 0.08 ± 6% -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.self.cycles-pp.generic_file_llseek_size 0.16 ± 3% -0.1 0.08 ± 3% -0.0 0.12 ± 2% -0.0 0.12 perf-profile.self.cycles-pp.aa_file_perm 0.15 -0.1 0.07 ± 6% -0.0 0.12 -0.0 0.12 ± 3% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.16 ± 2% -0.1 0.09 ± 3% -0.0 0.13 ± 3% -0.0 0.13 perf-profile.self.cycles-pp.cgroup_rstat_updated 0.07 ± 6% -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.self.cycles-pp.__mark_inode_dirty 0.07 ± 6% -0.1 0.00 -0.0 0.06 -0.0 0.06 perf-profile.self.cycles-pp.generic_file_read_iter 0.15 ± 3% -0.1 0.08 ± 4% -0.0 0.12 ± 2% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.current_time 0.15 ± 3% -0.1 0.08 ± 3% -0.0 0.12 ± 2% -0.0 0.12 ± 2% perf-profile.self.cycles-pp.ksys_write 0.15 ± 6% -0.1 0.07 ± 6% -0.0 0.12 ± 8% -0.0 0.12 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.15 ± 2% -0.1 0.08 -0.0 0.12 ± 2% -0.0 0.12 perf-profile.self.cycles-pp.generic_write_checks 0.15 ± 4% -0.1 0.08 ± 6% +0.0 0.16 ± 5% +0.0 0.16 ± 3% perf-profile.self.cycles-pp.__memcg_slab_post_alloc_hook 0.15 ± 3% -0.1 0.08 ± 4% -0.0 0.12 ± 3% -0.0 0.12 ± 2% perf-profile.self.cycles-pp.copy_page_to_iter 0.15 ± 3% -0.1 0.08 ± 4% -0.0 0.12 -0.0 0.12 ± 2% perf-profile.self.cycles-pp.ksys_read 0.07 ± 6% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.__folio_cancel_dirty 0.07 ± 6% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.ext4_generic_write_checks 0.07 ± 6% -0.1 0.00 -0.0 0.05 ± 8% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__mem_cgroup_charge 0.08 -0.1 0.01 ±163% -0.0 0.07 ± 6% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.free_unref_folios 0.18 ± 2% -0.1 0.11 ± 3% -0.0 0.15 ± 3% -0.0 0.14 ± 4% perf-profile.self.cycles-pp.mod_objcg_state 0.13 -0.1 0.06 ± 7% -0.0 0.10 -0.0 0.10 ± 3% perf-profile.self.cycles-pp.__radix_tree_lookup 0.13 ± 3% -0.1 0.06 ± 4% -0.0 0.10 -0.0 0.10 ± 3% perf-profile.self.cycles-pp.__alloc_pages_noprof 0.06 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp._raw_spin_lock_irq 0.06 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.folio_activate 0.06 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.truncate_cleanup_folio 0.06 ± 7% -0.1 0.00 -0.0 0.05 ± 9% -0.0 0.05 perf-profile.self.cycles-pp.xas_clear_mark 0.15 ± 2% -0.1 0.09 ± 5% -0.0 0.13 ± 2% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.find_lock_entries 0.14 -0.1 0.08 ± 6% -0.0 0.12 ± 3% -0.0 0.12 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.06 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.amd_clear_divider 0.06 ± 7% -0.1 0.00 -0.0 0.05 ± 6% -0.0 0.05 perf-profile.self.cycles-pp.folio_mapping 0.13 ± 5% -0.1 0.07 ± 7% -0.0 0.13 ± 9% -0.0 0.13 ± 7% perf-profile.self.cycles-pp.kmem_cache_alloc_noprof 0.06 ± 6% -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.try_to_free_buffers 0.12 ± 2% -0.1 0.06 -0.0 0.09 ± 4% -0.0 0.09 perf-profile.self.cycles-pp.node_dirty_ok 0.06 -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.__mod_zone_page_state 0.06 -0.1 0.00 -0.0 0.05 -0.0 0.05 perf-profile.self.cycles-pp.delete_from_page_cache_batch 0.12 -0.1 0.06 ± 6% -0.0 0.10 ± 4% -0.0 0.10 ± 5% perf-profile.self.cycles-pp.ksys_lseek 0.09 -0.1 0.03 ± 75% -0.0 0.07 -0.0 0.07 perf-profile.self.cycles-pp.ext4_es_lookup_extent 0.12 -0.1 0.06 ± 7% -0.0 0.10 ± 4% -0.0 0.10 ± 3% perf-profile.self.cycles-pp.create_empty_buffers 0.12 ± 4% -0.1 0.06 -0.0 0.09 -0.0 0.09 ± 3% perf-profile.self.cycles-pp.__dquot_alloc_space 0.10 ± 3% -0.1 0.05 ± 31% -0.0 0.08 ± 6% -0.0 0.07 perf-profile.self.cycles-pp.folio_account_dirtied 0.12 ± 4% -0.1 0.06 ± 4% -0.0 0.09 ± 5% -0.0 0.09 perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head 0.09 -0.1 0.04 ± 61% -0.0 0.07 -0.0 0.07 perf-profile.self.cycles-pp.file_modified 0.11 ± 4% -0.1 0.05 ± 8% -0.0 0.08 ± 5% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ext4_llseek 0.10 ± 4% -0.1 0.05 -0.0 0.08 ± 6% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__count_memcg_events 0.11 ± 4% -0.1 0.06 ± 8% -0.0 0.08 ± 5% -0.0 0.08 ± 4% perf-profile.self.cycles-pp.generic_write_check_limits 0.10 ± 5% -0.1 0.05 -0.0 0.08 ± 4% -0.0 0.08 ± 4% perf-profile.self.cycles-pp.file_remove_privs_flags 0.10 -0.1 0.05 -0.0 0.08 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.get_page_from_freelist 0.10 ± 5% -0.1 0.05 -0.0 0.08 -0.0 0.08 perf-profile.self.cycles-pp.block_write_end 0.12 ± 4% -0.0 0.07 ± 6% -0.0 0.10 ± 4% -0.0 0.10 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.12 -0.0 0.07 ± 5% -0.0 0.10 -0.0 0.10 perf-profile.self.cycles-pp.drop_buffers 0.10 ± 5% -0.0 0.05 -0.0 0.08 ± 6% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.timestamp_truncate 0.10 ± 5% -0.0 0.05 ± 5% -0.0 0.08 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.folio_account_cleaned 0.10 ± 4% -0.0 0.06 ± 8% -0.0 0.08 -0.0 0.08 perf-profile.self.cycles-pp.kmem_cache_free 0.11 ± 9% -0.0 0.07 ± 8% +0.1 0.20 ± 3% +0.1 0.20 ± 2% perf-profile.self.cycles-pp.mem_cgroup_update_lru_size 0.10 ± 4% -0.0 0.07 -0.0 0.10 ± 4% -0.0 0.09 perf-profile.self.cycles-pp.__memcg_slab_free_hook 0.09 ± 5% -0.0 0.06 ± 4% -0.0 0.08 ± 4% -0.0 0.08 perf-profile.self.cycles-pp.try_charge_memcg 0.10 ± 4% -0.0 0.07 ± 5% -0.0 0.08 ± 4% -0.0 0.08 perf-profile.self.cycles-pp.__page_cache_release 0.23 ± 3% -0.0 0.21 ± 2% -0.2 0.05 -0.2 0.05 perf-profile.self.cycles-pp.xas_find_conflict 0.23 ± 2% -0.0 0.22 ± 3% -0.0 0.18 ± 3% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.workingset_activation 0.25 ± 2% +0.0 0.28 ± 4% +0.0 0.30 +0.0 0.30 perf-profile.self.cycles-pp.folio_activate_fn 0.17 ± 3% +0.1 0.26 -0.0 0.13 -0.0 0.13 ± 2% perf-profile.self.cycles-pp.lru_add_fn 0.51 ± 2% +0.1 0.63 ± 3% -0.1 0.40 ± 2% -0.1 0.38 perf-profile.self.cycles-pp.__lruvec_stat_mod_folio 0.30 ± 5% +0.2 0.51 ± 2% +0.0 0.34 ± 3% +0.0 0.35 ± 2% perf-profile.self.cycles-pp.folio_batch_move_lru 50.97 +25.3 76.24 +13.7 64.69 +14.2 65.15 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath > > Thanks, > Roman > > -- > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 60418934827c..3aae347cda09 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -110,6 +110,7 @@ struct mem_cgroup_per_node { > /* Fields which get updated often at the end. */ > struct lruvec lruvec; > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > + CACHELINE_PADDING(_pad2_); > struct mem_cgroup_reclaim_iter iter; > }; ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-16 7:53 ` Oliver Sang @ 2024-07-19 17:14 ` Roman Gushchin 2024-07-19 22:38 ` Shakeel Butt 2024-07-23 3:42 ` [linux-next:master] [mm] : " Oliver Sang 0 siblings, 2 replies; 12+ messages in thread From: Roman Gushchin @ 2024-07-19 17:14 UTC (permalink / raw) To: Oliver Sang Cc: oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > hi, Roman, > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > hi, Roman Gushchin, > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > Hello, > > > > > > > > thank you for the report! > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > Thank you! > > > > > > in our this aim7 test, we found the performance partially recovered by > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > Thank you for providing the detailed information! > > > > Can you, please, check if the following patch resolves the regression entirely? > > no. in our tests, the following patch has little impact. > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > let me know, thanks) Hm, interesting. And thank you for the confirmation, you did everything correct. Because the only thing the original patch did was a removal of few fields from the mem_cgroup_per_node struct, there are not many options left here. Would you mind to try the following patch? Thank you and really appreciate your help! diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7e2eb091049a..0e5bf25d324f 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { /* Fields which get updated often at the end. */ struct lruvec lruvec; + CACHELINE_PADDING(_pad2_); unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; struct mem_cgroup_reclaim_iter iter; }; ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-19 17:14 ` Roman Gushchin @ 2024-07-19 22:38 ` Shakeel Butt 2024-07-22 15:37 ` Roman Gushchin 2024-07-23 3:42 ` [linux-next:master] [mm] : " Oliver Sang 1 sibling, 1 reply; 12+ messages in thread From: Shakeel Butt @ 2024-07-19 22:38 UTC (permalink / raw) To: Roman Gushchin Cc: Oliver Sang, oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin On Fri, Jul 19, 2024 at 05:14:16PM GMT, Roman Gushchin wrote: > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > hi, Roman, > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > hi, Roman Gushchin, > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > Hello, > > > > > > > > > > thank you for the report! > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > Thank you! > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > Thank you for providing the detailed information! > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > no. in our tests, the following patch has little impact. > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > let me know, thanks) > > Hm, interesting. And thank you for the confirmation, you did everything correct. > Because the only thing the original patch did was a removal of few fields from > the mem_cgroup_per_node struct, there are not many options left here. > Would you mind to try the following patch? > > Thank you and really appreciate your help! > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 7e2eb091049a..0e5bf25d324f 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { > > /* Fields which get updated often at the end. */ > struct lruvec lruvec; > + CACHELINE_PADDING(_pad2_); > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > struct mem_cgroup_reclaim_iter iter; > }; > > I suspect we need padding in the struct mem_cgroup instead of in struct mem_cgroup_per_node. I am planning to run some experiments and will report back once I have some convincing numbers. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-19 22:38 ` Shakeel Butt @ 2024-07-22 15:37 ` Roman Gushchin 2024-07-22 19:51 ` Shakeel Butt 0 siblings, 1 reply; 12+ messages in thread From: Roman Gushchin @ 2024-07-22 15:37 UTC (permalink / raw) To: Shakeel Butt Cc: Oliver Sang, oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin On Fri, Jul 19, 2024 at 03:38:26PM -0700, Shakeel Butt wrote: > On Fri, Jul 19, 2024 at 05:14:16PM GMT, Roman Gushchin wrote: > > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > > hi, Roman, > > > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > > hi, Roman Gushchin, > > > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > > > Hello, > > > > > > > > > > > > thank you for the report! > > > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > > > Thank you! > > > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > > > Thank you for providing the detailed information! > > > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > > > no. in our tests, the following patch has little impact. > > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > > let me know, thanks) > > > > Hm, interesting. And thank you for the confirmation, you did everything correct. > > Because the only thing the original patch did was a removal of few fields from > > the mem_cgroup_per_node struct, there are not many options left here. > > Would you mind to try the following patch? > > > > Thank you and really appreciate your help! > > > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 7e2eb091049a..0e5bf25d324f 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { > > > > /* Fields which get updated often at the end. */ > > struct lruvec lruvec; > > + CACHELINE_PADDING(_pad2_); > > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > > struct mem_cgroup_reclaim_iter iter; > > }; > > > > > > I suspect we need padding in the struct mem_cgroup instead of in struct > mem_cgroup_per_node. I am planning to run some experiments and will > report back once I have some convincing numbers. You mean the regression was attributed to the wrong commit? Because the blamed commit only removed few fields from the middle of mem_cgroup_per_node. Thanks ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression 2024-07-22 15:37 ` Roman Gushchin @ 2024-07-22 19:51 ` Shakeel Butt 0 siblings, 0 replies; 12+ messages in thread From: Shakeel Butt @ 2024-07-22 19:51 UTC (permalink / raw) To: Roman Gushchin Cc: Oliver Sang, oe-lkp, lkp, Linux Memory Management List, Andrew Morton, Johannes Weiner, Michal Hocko, Muchun Song, cgroups, ying.huang, feng.tang, fengwei.yin On Mon, Jul 22, 2024 at 03:37:27PM GMT, Roman Gushchin wrote: > On Fri, Jul 19, 2024 at 03:38:26PM -0700, Shakeel Butt wrote: > > On Fri, Jul 19, 2024 at 05:14:16PM GMT, Roman Gushchin wrote: > > > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > > > hi, Roman, > > > > > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > > > hi, Roman Gushchin, > > > > > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > thank you for the report! > > > > > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > > > > > Thank you! > > > > > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > > > > > Thank you for providing the detailed information! > > > > > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > > > > > no. in our tests, the following patch has little impact. > > > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > > > let me know, thanks) > > > > > > Hm, interesting. And thank you for the confirmation, you did everything correct. > > > Because the only thing the original patch did was a removal of few fields from > > > the mem_cgroup_per_node struct, there are not many options left here. > > > Would you mind to try the following patch? > > > > > > Thank you and really appreciate your help! > > > > > > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > > index 7e2eb091049a..0e5bf25d324f 100644 > > > --- a/include/linux/memcontrol.h > > > +++ b/include/linux/memcontrol.h > > > @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { > > > > > > /* Fields which get updated often at the end. */ > > > struct lruvec lruvec; > > > + CACHELINE_PADDING(_pad2_); > > > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > > > struct mem_cgroup_reclaim_iter iter; > > > }; > > > > > > > > > > I suspect we need padding in the struct mem_cgroup instead of in struct > > mem_cgroup_per_node. I am planning to run some experiments and will > > report back once I have some convincing numbers. > > You mean the regression was attributed to the wrong commit? > Because the blamed commit only removed few fields from the middle > of mem_cgroup_per_node. > The whole series changed the layout of multiple memcg structs, so focusing on a single patch is not necessarily correct. We need to accept the current status and work towards improving the charge and stats update path to similar or better than before performance. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] : aim7.jobs-per-min -29.4% regression 2024-07-19 17:14 ` Roman Gushchin 2024-07-19 22:38 ` Shakeel Butt @ 2024-07-23 3:42 ` Oliver Sang 2024-07-23 15:44 ` Shakeel Butt 2024-07-24 22:15 ` Roman Gushchin 1 sibling, 2 replies; 12+ messages in thread From: Oliver Sang @ 2024-07-23 3:42 UTC (permalink / raw) To: Roman Gushchin Cc: oe-lkp@lists.linux.dev, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups@vger.kernel.org, Huang, Ying, Tang, Feng, Yin, Fengwei, oliver.sang hi, Roman, On Sat, Jul 20, 2024 at 01:14:16AM +0800, Roman Gushchin wrote: > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > hi, Roman, > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > hi, Roman Gushchin, > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > Hello, > > > > > > > > > > thank you for the report! > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > Thank you! > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > Thank you for providing the detailed information! > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > no. in our tests, the following patch has little impact. > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > let me know, thanks) > > Hm, interesting. And thank you for the confirmation, you did everything correct. > Because the only thing the original patch did was a removal of few fields from > the mem_cgroup_per_node struct, there are not many options left here. > Would you mind to try the following patch? > > Thank you and really appreciate your help! you are welcome! though we saw there are further discussions, we still share our test results to you. in our tests, by your new version patch, the regression is entirely resoloved. ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/rootfs/tbox_group/test/testcase: gcc-13/performance/1BRD_48G/ext4/x86_64-rhel-8.3/3000/debian-12-x86_64-20240206.cgz/lkp-icl-2sp2/disk_rr/aim7 commit: 94b7e5bf09 ("mm: memcg: put memcg1-specific struct mem_cgroup's members under CONFIG_MEMCG_V1") 98c9daf5ae ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") 6df13230b6 ("mm: memcg: add cache line padding to mem_cgroup_per_node") 981204d280 <--- your v2 "_pad2_" patch 94b7e5bf09b08aa4 98c9daf5ae6be008f78c07b744b 6df13230b612af81ce04f20bb37 981204d28033e8b3a33c3f18861 ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 76.42 +15.1% 87.99 ± 3% +3.1% 78.77 +1.2% 77.31 ± 3% uptime.boot 53.28 ± 2% -27.5% 38.65 ± 2% -9.1% 48.45 ± 2% -0.4% 53.06 iostat.cpu.idle 44.58 ± 2% +33.6% 59.58 +11.2% 49.57 ± 2% +0.5% 44.80 iostat.cpu.system 2.13 -17.0% 1.77 -7.1% 1.98 +0.4% 2.14 iostat.cpu.user 49.85 ± 2% -14.7 35.17 ± 2% -5.2 44.66 ± 3% -0.4 49.48 mpstat.cpu.all.idle% 0.15 ± 7% +0.0 0.15 ± 3% -0.0 0.13 ± 4% -0.0 0.15 ± 4% mpstat.cpu.all.irq% 47.74 ± 2% +15.1 62.82 +5.4 53.14 ± 2% +0.4 48.10 mpstat.cpu.all.sys% 2.23 -0.4 1.82 -0.2 2.04 +0.0 2.24 mpstat.cpu.all.usr% 4752 ± 13% -18.1% 3890 ± 11% -9.0% 4322 ± 19% +5.5% 5014 ± 44% sched_debug.cpu.avg_idle.min 0.00 ± 32% -49.2% 0.00 ±171% -62.3% 0.00 ± 99% -34.2% 0.00 ± 89% sched_debug.rt_rq:.rt_time.avg 0.19 ± 32% -49.2% 0.10 ±171% -62.3% 0.07 ± 99% -34.2% 0.13 ± 89% sched_debug.rt_rq:.rt_time.max 0.02 ± 32% -49.2% 0.01 ±171% -62.3% 0.01 ± 99% -34.2% 0.01 ± 89% sched_debug.rt_rq:.rt_time.stddev 53.29 ± 2% -27.4% 38.70 ± 2% -9.4% 48.28 ± 2% -0.4% 53.07 vmstat.cpu.id 2.242e+08 -0.0% 2.242e+08 +0.0% 2.243e+08 +14.7% 2.573e+08 vmstat.memory.free 65.83 ± 4% +59.1% 104.76 ± 3% +19.0% 78.35 ± 3% -0.7% 65.39 ± 4% vmstat.procs.r 8385 ± 4% +34.6% 11284 ± 3% +20.5% 10101 ± 8% +2.2% 8573 ± 6% vmstat.system.cs 245966 ± 2% +8.1% 265964 +19.3% 293498 ± 2% +0.2% 246520 vmstat.system.in 778685 -29.4% 549435 -10.3% 698378 +0.2% 779959 aim7.jobs-per-min 23.31 +41.4% 32.96 +11.4% 25.96 -0.2% 23.27 aim7.time.elapsed_time 23.31 +41.4% 32.96 +11.4% 25.96 -0.2% 23.27 aim7.time.elapsed_time.max 47890 ± 7% +338.5% 210000 ± 3% +117.1% 103949 ± 7% +4.6% 50095 ± 6% aim7.time.involuntary_context_switches 6674 +26.7% 8455 +10.5% 7372 -0.2% 6657 aim7.time.percent_of_cpu_this_job_got 1510 +81.7% 2744 ± 2% +23.8% 1869 -0.4% 1504 aim7.time.system_time 19454 ± 3% +9.1% 21223 ± 3% -3.5% 18782 ± 4% -2.0% 19067 ± 3% aim7.time.voluntary_context_switches 23.31 +41.4% 32.96 +11.4% 25.96 -0.2% 23.27 time.elapsed_time 23.31 +41.4% 32.96 +11.4% 25.96 -0.2% 23.27 time.elapsed_time.max 47890 ± 7% +338.5% 210000 ± 3% +117.1% 103949 ± 7% +4.6% 50095 ± 6% time.involuntary_context_switches 6674 +26.7% 8455 +10.5% 7372 -0.2% 6657 time.percent_of_cpu_this_job_got 1510 +81.7% 2744 ± 2% +23.8% 1869 -0.4% 1504 time.system_time 45.72 -4.9% 43.47 -1.7% 44.96 -0.4% 45.53 time.user_time 19454 ± 3% +9.1% 21223 ± 3% -3.5% 18782 ± 4% -2.0% 19067 ± 3% time.voluntary_context_switches 49345 ± 6% +108.4% 102820 ± 10% +32.3% 65261 ± 5% -4.1% 47303 ± 3% meminfo.Active 22670 ± 11% +182.6% 64065 ± 15% +57.3% 35667 ± 11% -6.2% 21274 ± 7% meminfo.Active(anon) 26674 ± 4% +45.3% 38754 ± 4% +10.9% 29594 ± 5% -2.4% 26028 ± 5% meminfo.Active(file) 33695 +18.6% 39973 ± 3% +3.8% 34971 -1.2% 33306 meminfo.AnonHugePages 1.154e+08 +0.0% 1.154e+08 +0.0% 1.154e+08 +14.3% 1.319e+08 meminfo.CommitLimit 2.244e+08 -0.1% 2.241e+08 -0.4% 2.236e+08 +14.7% 2.573e+08 meminfo.DirectMap1G 1360098 ± 3% +14.0% 1551107 ± 2% +1.6% 1381539 ± 3% +3.3% 1405172 ± 3% meminfo.Inactive 803759 +15.0% 924519 +4.7% 841839 -0.5% 799481 meminfo.Inactive(anon) 66977 ± 3% +115.7% 144485 ± 7% +21.0% 81015 ± 3% -3.8% 64423 ± 2% meminfo.Mapped 2.216e+08 -0.1% 2.214e+08 -0.1% 2.214e+08 +14.9% 2.545e+08 meminfo.MemAvailable 2.221e+08 -0.1% 2.219e+08 -0.0% 2.22e+08 +14.8% 2.551e+08 meminfo.MemFree 2.307e+08 +0.0% 2.307e+08 +0.0% 2.307e+08 +14.3% 2.637e+08 meminfo.MemTotal 78152 ± 9% +188.4% 225431 ± 5% +64.5% 128567 ± 5% -9.6% 70624 ± 6% meminfo.Shmem 15327 ± 8% +52.6% 23389 ± 11% +29.5% 19853 ± 14% +16.0% 17781 ± 10% numa-meminfo.node0.Active 1872 ± 53% +100.1% 3747 ± 29% +107.1% 3877 ± 31% +127.8% 4264 ± 7% numa-meminfo.node0.Active(anon) 13455 ± 9% +46.0% 19642 ± 10% +18.7% 15975 ± 13% +0.5% 13516 ± 12% numa-meminfo.node0.Active(file) 601453 ± 17% -45.5% 327882 ± 59% -66.4% 202373 ±109% -52.6% 285131 ± 71% numa-meminfo.node0.AnonPages 875888 ± 10% -25.8% 649771 ± 29% -45.1% 480842 ± 47% -32.6% 590166 ± 35% numa-meminfo.node0.Inactive 602152 ± 17% -44.9% 331923 ± 58% -66.2% 203431 ±109% -52.6% 285685 ± 71% numa-meminfo.node0.Inactive(anon) 19657 ± 14% +111.2% 41510 ± 18% +76.5% 34690 ± 37% +24.3% 24440 ± 48% numa-meminfo.node0.Mapped 6081 ± 16% +84.9% 11247 ± 42% +38.8% 8442 ± 13% +38.5% 8422 ± 6% numa-meminfo.node0.Shmem 33647 ± 8% +134.3% 78825 ± 15% +35.8% 45703 ± 9% -10.2% 30214 ± 3% numa-meminfo.node1.Active 20790 ± 14% +188.7% 60020 ± 18% +52.8% 31760 ± 14% -16.7% 17318 ± 9% numa-meminfo.node1.Active(anon) 12857 ± 7% +46.3% 18805 ± 9% +8.4% 13942 ± 8% +0.3% 12896 ± 10% numa-meminfo.node1.Active(file) 150990 ± 70% +190.9% 439229 ± 44% +264.7% 550716 ± 40% +210.3% 468571 ± 43% numa-meminfo.node1.AnonPages 180780 ± 55% +160.4% 470756 ± 41% +220.7% 579702 ± 39% +173.1% 493698 ± 42% numa-meminfo.node1.AnonPages.max 485228 ± 23% +85.5% 899983 ± 21% +85.5% 900160 ± 25% +67.6% 813028 ± 26% numa-meminfo.node1.Inactive 202089 ± 50% +192.9% 591917 ± 32% +215.8% 638153 ± 34% +154.3% 513982 ± 39% numa-meminfo.node1.Inactive(anon) 47991 ± 7% +112.0% 101755 ± 8% -2.6% 46748 ± 32% -15.0% 40769 ± 26% numa-meminfo.node1.Mapped 93326924 +0.8% 94106109 +1.2% 94466709 +36.2% 1.271e+08 numa-meminfo.node1.MemFree 99028084 +0.0% 99028084 +0.0% 99028084 +33.4% 1.321e+08 numa-meminfo.node1.MemTotal 72431 ± 8% +194.1% 213055 ± 4% +65.2% 119653 ± 5% -12.7% 63207 ± 6% numa-meminfo.node1.Shmem 467.96 ± 53% +99.5% 933.35 ± 29% +107.2% 969.49 ± 31% +127.9% 1066 ± 7% numa-vmstat.node0.nr_active_anon 3290 ± 10% +50.9% 4965 ± 10% +20.2% 3956 ± 7% -0.4% 3277 ± 10% numa-vmstat.node0.nr_active_file 150399 ± 17% -45.5% 82004 ± 59% -66.3% 50675 ±109% -52.7% 71196 ± 71% numa-vmstat.node0.nr_anon_pages 150570 ± 17% -45.0% 82828 ± 59% -66.2% 50936 ±108% -52.6% 71333 ± 71% numa-vmstat.node0.nr_inactive_anon 4793 ± 17% +113.8% 10249 ± 20% +84.1% 8827 ± 36% +31.7% 6311 ± 48% numa-vmstat.node0.nr_mapped 1519 ± 16% +72.6% 2622 ± 33% +38.8% 2110 ± 13% +38.5% 2105 ± 6% numa-vmstat.node0.nr_shmem 467.96 ± 53% +99.4% 933.32 ± 29% +107.2% 969.49 ± 31% +127.9% 1066 ± 7% numa-vmstat.node0.nr_zone_active_anon 3294 ± 9% +50.9% 4970 ± 10% +20.4% 3967 ± 7% +0.1% 3296 ± 9% numa-vmstat.node0.nr_zone_active_file 150570 ± 17% -45.0% 82827 ± 59% -66.2% 50936 ±108% -52.6% 71333 ± 71% numa-vmstat.node0.nr_zone_inactive_anon 4955 ± 11% +202.3% 14980 ± 18% +61.8% 8017 ± 16% -10.0% 4458 ± 9% numa-vmstat.node1.nr_active_anon 3376 ± 8% +40.1% 4729 ± 7% +7.0% 3611 ± 13% -4.1% 3238 ± 8% numa-vmstat.node1.nr_active_file 38044 ± 69% +188.6% 109788 ± 44% +261.9% 137690 ± 40% +207.9% 117122 ± 43% numa-vmstat.node1.nr_anon_pages 23331085 +0.8% 23527208 +1.2% 23617254 +36.2% 31785125 numa-vmstat.node1.nr_free_pages 50152 ± 51% +194.6% 147731 ± 32% +218.6% 159767 ± 34% +156.6% 128700 ± 39% numa-vmstat.node1.nr_inactive_anon 12101 ± 7% +109.5% 25357 ± 8% -1.6% 11908 ± 33% -13.7% 10445 ± 27% numa-vmstat.node1.nr_mapped 393216 +0.0% 393216 +0.0% 393216 +33.3% 524288 numa-vmstat.node1.nr_memmap_boot 17201 ± 7% +208.3% 53026 ± 4% +75.7% 30214 ± 8% -6.1% 16157 ± 5% numa-vmstat.node1.nr_shmem 4955 ± 11% +202.3% 14980 ± 18% +61.8% 8017 ± 16% -10.0% 4458 ± 9% numa-vmstat.node1.nr_zone_active_anon 3352 ± 8% +41.1% 4729 ± 7% +7.8% 3612 ± 13% -3.7% 3228 ± 7% numa-vmstat.node1.nr_zone_active_file 50150 ± 51% +194.6% 147730 ± 32% +218.6% 159766 ± 34% +156.6% 128700 ± 39% numa-vmstat.node1.nr_zone_inactive_anon 5776 ± 11% +175.1% 15893 ± 16% +55.0% 8953 ± 11% -5.9% 5433 ± 7% proc-vmstat.nr_active_anon 6595 ± 8% +46.7% 9677 ± 3% +12.6% 7424 ± 5% -1.4% 6506 ± 7% proc-vmstat.nr_active_file 187950 +2.0% 191760 +0.1% 188193 +0.2% 188345 proc-vmstat.nr_anon_pages 5532779 -0.1% 5528733 -0.0% 5530018 +14.9% 6354839 proc-vmstat.nr_dirty_background_threshold 11079086 -0.1% 11070986 -0.0% 11073558 +14.9% 12725216 proc-vmstat.nr_dirty_threshold 945653 +6.0% 1002646 +1.0% 955008 +1.1% 956077 proc-vmstat.nr_file_pages 55532095 -0.1% 55470389 -0.0% 55507767 +14.8% 63769166 proc-vmstat.nr_free_pages 201118 +14.8% 230842 +4.7% 210479 -0.6% 199886 proc-vmstat.nr_inactive_anon 17214 ± 3% +108.2% 35836 ± 6% +20.0% 20660 ± 3% -3.9% 16541 ± 2% proc-vmstat.nr_mapped 916480 +0.0% 916480 +0.0% 916480 +14.3% 1047552 proc-vmstat.nr_memmap_boot 19951 ± 8% +180.4% 55933 ± 4% +61.5% 32218 ± 5% -9.8% 17996 ± 5% proc-vmstat.nr_shmem 40267 +2.7% 41335 -0.7% 39991 +0.9% 40618 proc-vmstat.nr_slab_reclaimable 86277 +1.8% 87792 +0.1% 86326 +0.8% 86995 proc-vmstat.nr_slab_unreclaimable 5776 ± 11% +175.1% 15893 ± 16% +55.0% 8953 ± 11% -5.9% 5433 ± 7% proc-vmstat.nr_zone_active_anon 6595 ± 8% +46.7% 9677 ± 3% +12.6% 7424 ± 5% -1.3% 6506 ± 7% proc-vmstat.nr_zone_active_file 201118 +14.8% 230842 +4.7% 210479 -0.6% 199886 proc-vmstat.nr_zone_inactive_anon 312.12 ±241% +1618.1% 5362 ±125% +521.3% 1939 ±171% -88.9% 34.75 ±131% proc-vmstat.numa_pages_migrated 369792 +19.6% 442285 ± 3% +2.9% 380386 -1.8% 363217 proc-vmstat.pgfault 312.12 ±241% +1618.1% 5362 ±125% +521.3% 1939 ±171% -88.9% 34.75 ±131% proc-vmstat.pgmigrate_success 2426 ± 2% +29.2% 3135 ± 3% +16.3% 2821 ± 3% +1.3% 2457 proc-vmstat.pgpgout 1515 +3.8% 1572 +1.3% 1535 -0.2% 1512 proc-vmstat.unevictable_pgs_culled 0.63 ± 3% +36.5% 0.85 ± 2% +19.4% 0.75 ± 2% -0.7% 0.62 ± 5% perf-stat.i.MPKI 1.885e+10 -23.7% 1.437e+10 -7.4% 1.745e+10 ± 2% -0.8% 1.869e+10 perf-stat.i.branch-instructions 2.78 ± 2% -0.5 2.29 ± 4% -0.2 2.55 ± 4% +0.1 2.84 ± 4% perf-stat.i.branch-miss-rate% 67232899 -17.4% 55553782 ± 3% -5.8% 63348903 -0.5% 66913803 perf-stat.i.branch-misses 13.70 ± 3% +2.5 16.17 ± 3% +1.1 14.84 ± 2% -0.3 13.40 ± 4% perf-stat.i.cache-miss-rate% 72591570 ± 2% +0.6% 73035167 +10.8% 80408900 ± 3% +0.5% 72971231 ± 3% perf-stat.i.cache-misses 5.483e+08 -24.9% 4.118e+08 -5.7% 5.17e+08 ± 2% -0.8% 5.441e+08 perf-stat.i.cache-references 8605 ± 4% +34.7% 11593 ± 4% +23.4% 10618 ± 7% +2.7% 8834 ± 8% perf-stat.i.context-switches 1.35 +65.4% 2.22 ± 3% +20.9% 1.63 -1.2% 1.33 perf-stat.i.cpi 1.616e+11 +22.1% 1.973e+11 +10.5% 1.785e+11 ± 2% -1.5% 1.592e+11 perf-stat.i.cpu-cycles 8.537e+10 -24.8% 6.416e+10 -8.0% 7.852e+10 ± 2% -0.8% 8.465e+10 perf-stat.i.instructions 0.96 -22.2% 0.75 ± 2% -9.4% 0.87 +1.5% 0.98 ± 2% perf-stat.i.ipc 13455 -15.4% 11379 ± 3% -7.6% 12438 +0.3% 13496 perf-stat.i.minor-faults 13489 -15.5% 11396 ± 3% -7.6% 12465 +0.2% 13522 perf-stat.i.page-faults 0.85 ± 3% +34.8% 1.15 +20.2% 1.02 ± 3% +1.2% 0.86 ± 3% perf-stat.overall.MPKI 0.34 +0.0 0.36 ± 2% +0.0 0.35 +0.0 0.35 perf-stat.overall.branch-miss-rate% 13.23 ± 3% +4.6 17.78 +2.3 15.52 ± 2% +0.2 13.38 ± 2% perf-stat.overall.cache-miss-rate% 1.90 +62.8% 3.08 +20.1% 2.28 -0.7% 1.88 perf-stat.overall.cpi 2231 ± 3% +20.6% 2690 ± 2% -0.2% 2227 ± 2% -1.9% 2189 ± 2% perf-stat.overall.cycles-between-cache-misses 0.53 -38.5% 0.32 -16.7% 0.44 +0.7% 0.53 perf-stat.overall.ipc 1.86e+10 -23.0% 1.433e+10 -7.6% 1.719e+10 -0.9% 1.844e+10 perf-stat.ps.branch-instructions 64095803 -19.1% 51843320 ± 2% -5.7% 60421424 -0.3% 63895014 perf-stat.ps.branch-misses 71639076 ± 2% +2.3% 73304464 ± 2% +10.4% 79092736 ± 3% +0.3% 71888849 ± 3% perf-stat.ps.cache-misses 5.417e+08 -23.9% 4.124e+08 -5.9% 5.096e+08 ± 2% -0.8% 5.372e+08 perf-stat.ps.cache-references 8220 ± 4% +33.3% 10962 ± 4% +23.9% 10184 ± 8% +2.4% 8418 ± 7% perf-stat.ps.context-switches 1.597e+11 +23.5% 1.972e+11 ± 2% +10.2% 1.761e+11 ± 2% -1.5% 1.573e+11 perf-stat.ps.cpu-cycles 8.426e+10 -24.1% 6.394e+10 -8.2% 7.737e+10 ± 2% -0.9% 8.353e+10 perf-stat.ps.instructions 12730 -18.7% 10354 ± 3% -7.2% 11811 +0.2% 12756 perf-stat.ps.minor-faults 12762 -18.8% 10369 ± 3% -7.3% 11836 +0.1% 12781 perf-stat.ps.page-faults 4.35 ± 10% -4.4 0.00 -3.5 0.81 ± 20% -0.2 4.14 ± 11% perf-profile.calltrace.cycles-pp.unlink 4.35 ± 10% -4.3 0.00 -3.5 0.81 ± 19% -0.2 4.13 ± 11% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.35 ± 10% -4.3 0.00 -3.5 0.81 ± 19% -0.2 4.13 ± 11% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlink 4.34 ± 10% -4.3 0.00 -3.5 0.80 ± 19% -0.2 4.12 ± 11% perf-profile.calltrace.cycles-pp.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.34 ± 10% -4.3 0.00 -3.5 0.80 ± 20% -0.2 4.12 ± 11% perf-profile.calltrace.cycles-pp.do_unlinkat.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlink 4.14 ± 10% -4.1 0.00 -3.5 0.65 ± 24% -0.2 3.93 ± 11% perf-profile.calltrace.cycles-pp.down_write.do_unlinkat.__x64_sys_unlink.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.14 ± 10% -4.1 0.00 -3.5 0.65 ± 24% -0.2 3.92 ± 11% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.do_unlinkat.__x64_sys_unlink.do_syscall_64 4.14 ± 10% -4.1 0.00 -3.6 0.58 ± 45% -0.2 3.92 ± 11% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.do_unlinkat.__x64_sys_unlink 3.99 ± 11% -4.0 0.00 -3.6 0.40 ± 83% -0.2 3.76 ± 12% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.do_unlinkat 3.53 ± 6% -3.4 0.10 ±212% -2.4 1.12 ± 17% -0.2 3.37 ± 8% perf-profile.calltrace.cycles-pp.down_write.open_last_lookups.path_openat.do_filp_open.do_sys_openat2 3.53 ± 6% -3.4 0.10 ±212% -2.4 1.12 ± 17% -0.2 3.37 ± 8% perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.open_last_lookups.path_openat.do_filp_open 3.52 ± 6% -3.4 0.10 ±212% -2.4 1.12 ± 17% -0.2 3.36 ± 8% perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.open_last_lookups.path_openat 3.30 ± 7% -3.3 0.00 -2.3 0.98 ± 19% -0.2 3.13 ± 8% perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.open_last_lookups 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -0.2 3.76 ± 7% perf-profile.calltrace.cycles-pp.creat64 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -0.2 3.76 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -0.2 3.76 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.creat64 3.92 ± 6% -3.2 0.68 ± 8% -2.5 1.43 ± 13% -0.2 3.75 ± 7% perf-profile.calltrace.cycles-pp.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.92 ± 6% -3.2 0.68 ± 8% -2.5 1.43 ± 13% -0.2 3.75 ± 7% perf-profile.calltrace.cycles-pp.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe.creat64 3.91 ± 6% -3.2 0.68 ± 8% -2.5 1.42 ± 13% -0.2 3.74 ± 7% perf-profile.calltrace.cycles-pp.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64 3.91 ± 6% -3.2 0.68 ± 8% -2.5 1.42 ± 13% -0.2 3.74 ± 7% perf-profile.calltrace.cycles-pp.do_filp_open.do_sys_openat2.__x64_sys_creat.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.87 ± 6% -3.2 0.65 ± 9% -2.5 1.39 ± 14% -0.2 3.70 ± 7% perf-profile.calltrace.cycles-pp.open_last_lookups.path_openat.do_filp_open.do_sys_openat2.__x64_sys_creat 3.66 -1.7 1.99 ± 2% -0.6 3.06 +0.0 3.70 perf-profile.calltrace.cycles-pp.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write 3.31 -1.5 1.77 ± 2% -0.6 2.73 +0.1 3.36 perf-profile.calltrace.cycles-pp.llseek 2.87 -1.3 1.62 ± 2% -0.5 2.33 +0.0 2.87 perf-profile.calltrace.cycles-pp.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 2.25 -1.0 1.29 ± 2% -0.4 1.83 +0.0 2.26 perf-profile.calltrace.cycles-pp.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter.vfs_write 2.17 -0.9 1.25 ± 2% -0.4 1.76 +0.0 2.18 perf-profile.calltrace.cycles-pp.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write.ext4_buffered_write_iter 0.86 ± 4% -0.9 0.00 -0.2 0.62 ± 4% +0.0 0.87 ± 3% perf-profile.calltrace.cycles-pp.workingset_age_nonresident.workingset_activation.folio_mark_accessed.filemap_read.vfs_read 0.79 -0.8 0.00 -0.1 0.74 +0.1 0.85 perf-profile.calltrace.cycles-pp.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.93 -0.8 1.14 ± 2% -0.3 1.66 -0.0 1.92 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.vfs_read.ksys_read.do_syscall_64 0.76 -0.8 0.00 -0.2 0.58 -0.0 0.76 ± 2% perf-profile.calltrace.cycles-pp.file_modified.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64 1.99 -0.8 1.23 ± 2% -0.4 1.63 +0.0 2.00 perf-profile.calltrace.cycles-pp.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 0.74 -0.7 0.00 -0.2 0.58 +0.0 0.74 perf-profile.calltrace.cycles-pp.folio_alloc_noprof.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.79 -0.7 1.07 ± 2% -0.2 1.55 -0.0 1.78 perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.filemap_read.vfs_read.ksys_read 0.72 ± 2% -0.7 0.00 -0.2 0.54 ± 4% -0.0 0.70 perf-profile.calltrace.cycles-pp.balance_dirty_pages_ratelimited_flags.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 0.69 -0.7 0.00 -0.2 0.54 +0.0 0.70 perf-profile.calltrace.cycles-pp.alloc_pages_mpol_noprof.folio_alloc_noprof.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 0.68 -0.7 0.00 -0.2 0.53 +0.0 0.69 perf-profile.calltrace.cycles-pp.touch_atime.filemap_read.vfs_read.ksys_read.do_syscall_64 0.67 -0.7 0.00 -0.7 0.00 -0.0 0.65 perf-profile.calltrace.cycles-pp.folio_account_dirtied.__folio_mark_dirty.mark_buffer_dirty.__block_commit_write.block_write_end 1.34 -0.7 0.68 ± 3% -0.3 1.05 -0.0 1.34 perf-profile.calltrace.cycles-pp.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 0.65 ± 2% -0.7 0.00 -0.1 0.52 -0.0 0.65 ± 2% perf-profile.calltrace.cycles-pp.fault_in_iov_iter_readable.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 1.30 -0.6 0.66 ± 3% -0.3 1.02 -0.0 1.30 perf-profile.calltrace.cycles-pp.ext4_da_map_blocks.ext4_da_get_block_prep.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 0.61 -0.6 0.00 -0.0 0.59 +0.1 0.67 perf-profile.calltrace.cycles-pp.folio_alloc_buffers.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 1.22 -0.6 0.65 ± 2% -0.2 0.99 +0.0 1.24 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.llseek 1.17 -0.5 0.63 ± 2% -0.2 1.00 +0.0 1.21 perf-profile.calltrace.cycles-pp.filemap_get_pages.filemap_read.vfs_read.ksys_read.do_syscall_64 0.54 ± 2% -0.5 0.00 -0.0 0.52 +0.0 0.59 perf-profile.calltrace.cycles-pp.alloc_buffer_head.folio_alloc_buffers.create_empty_buffers.ext4_block_write_begin.ext4_da_write_begin 1.21 -0.5 0.67 ± 2% -0.3 0.91 -0.0 1.19 perf-profile.calltrace.cycles-pp.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end.generic_perform_write 0.96 -0.5 0.43 ± 47% -0.1 0.83 +0.0 1.00 perf-profile.calltrace.cycles-pp.filemap_get_read_batch.filemap_get_pages.filemap_read.vfs_read.ksys_read 1.08 -0.5 0.57 ± 3% -0.2 0.86 -0.0 1.06 perf-profile.calltrace.cycles-pp.clear_bhb_loop.write 1.08 -0.5 0.57 ± 2% -0.2 0.88 +0.0 1.09 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek 1.20 -0.5 0.70 ± 3% -0.2 1.02 -0.0 1.19 perf-profile.calltrace.cycles-pp.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 1.05 -0.5 0.56 ± 2% -0.2 0.85 -0.0 1.05 perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 0.91 -0.5 0.42 ± 47% -0.2 0.67 -0.0 0.89 perf-profile.calltrace.cycles-pp.__folio_mark_dirty.mark_buffer_dirty.__block_commit_write.block_write_end.ext4_da_write_end 1.15 -0.5 0.68 ± 2% -0.2 0.99 -0.0 1.14 perf-profile.calltrace.cycles-pp.memset_orig.zero_user_segments.ext4_block_write_begin.ext4_da_write_begin.generic_perform_write 1.02 -0.5 0.55 ± 2% -0.2 0.86 +0.0 1.05 perf-profile.calltrace.cycles-pp.clear_bhb_loop.llseek 1.30 -0.5 0.83 ± 2% -0.2 1.05 +0.0 1.30 perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_folio.truncate_inode_pages_range.ext4_evict_inode.evict 0.98 -0.5 0.53 ± 2% -0.2 0.82 +0.0 1.00 perf-profile.calltrace.cycles-pp.clear_bhb_loop.read 1.10 ± 3% -0.4 0.67 ± 3% -0.3 0.80 ± 3% +0.0 1.10 ± 2% perf-profile.calltrace.cycles-pp.workingset_activation.folio_mark_accessed.filemap_read.vfs_read.ksys_read 30.65 -0.4 30.25 +0.7 31.31 +0.1 30.71 perf-profile.calltrace.cycles-pp.read 0.98 -0.3 0.69 ± 2% -0.4 0.61 -0.2 0.77 perf-profile.calltrace.cycles-pp.__filemap_add_folio.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 0.13 ±173% -0.1 0.00 -0.1 0.00 +0.4 0.53 perf-profile.calltrace.cycles-pp.kmem_cache_alloc_noprof.alloc_buffer_head.folio_alloc_buffers.create_empty_buffers.ext4_block_write_begin 2.39 +0.2 2.61 +0.1 2.47 +0.0 2.39 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release 2.42 +0.2 2.64 +0.1 2.50 -0.0 2.42 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode 2.39 +0.2 2.61 +0.1 2.48 +0.0 2.39 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range 2.42 +0.2 2.64 +0.1 2.50 +0.0 2.42 perf-profile.calltrace.cycles-pp.lru_add_drain_cpu.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict 2.38 +0.2 2.61 +0.1 2.47 +0.0 2.39 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.lru_add_drain_cpu 2.44 +0.4 2.89 +0.2 2.62 +0.0 2.45 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict 2.38 +0.5 2.84 +0.2 2.56 +0.0 2.39 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode 2.38 +0.5 2.84 +0.2 2.56 +0.0 2.39 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release.truncate_inode_pages_range 2.38 +0.5 2.84 +0.2 2.56 +0.0 2.39 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.__folio_batch_release 28.52 +0.6 29.11 +1.0 29.54 +0.0 28.55 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 28.38 +0.7 29.04 +1.0 29.42 +0.0 28.40 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 4.86 +0.7 5.54 +0.3 5.13 +0.0 4.87 perf-profile.calltrace.cycles-pp.__folio_batch_release.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 27.89 +0.9 28.77 +1.1 29.02 +0.0 27.90 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 27.47 +1.1 28.54 +1.2 28.67 +0.0 27.48 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 25.90 +1.9 27.78 +1.5 27.42 -0.0 25.90 perf-profile.calltrace.cycles-pp.filemap_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.26 +4.0 34.31 +3.9 34.16 +0.1 30.37 perf-profile.calltrace.cycles-pp.write 21.06 +4.1 25.14 +2.3 23.39 -0.0 21.02 perf-profile.calltrace.cycles-pp.folio_mark_accessed.filemap_read.vfs_read.ksys_read.do_syscall_64 19.70 +4.6 24.33 +2.7 22.38 -0.0 19.66 perf-profile.calltrace.cycles-pp.folio_activate.folio_mark_accessed.filemap_read.vfs_read.ksys_read 19.63 +4.7 24.29 +2.7 22.32 -0.0 19.59 perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read.vfs_read 18.85 +4.7 23.52 +2.7 21.52 -0.1 18.80 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed.filemap_read 18.84 +4.7 23.51 +2.7 21.52 -0.0 18.79 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate.folio_mark_accessed 18.83 +4.7 23.51 +2.7 21.51 -0.0 18.78 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_activate 28.38 +5.0 33.33 +4.3 32.65 +0.1 28.49 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write 28.23 +5.0 33.25 +4.3 32.53 +0.1 28.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 27.72 +5.3 32.99 +4.4 32.13 +0.1 27.83 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 27.25 +5.5 32.74 +4.5 31.75 +0.1 27.35 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 25.87 +5.6 31.48 +2.2 28.03 +0.1 26.02 perf-profile.calltrace.cycles-pp.__close 25.86 +5.6 31.47 +2.2 28.02 +0.1 26.01 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.86 +5.6 31.47 +2.2 28.02 +0.1 26.01 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__close 25.86 +5.6 31.46 +2.2 28.01 +0.1 26.00 perf-profile.calltrace.cycles-pp.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.84 +5.6 31.45 +2.2 28.00 +0.2 25.99 perf-profile.calltrace.cycles-pp.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe.__close 25.82 +5.6 31.44 +2.2 27.99 +0.1 25.97 perf-profile.calltrace.cycles-pp.dput.__fput.__x64_sys_close.do_syscall_64.entry_SYSCALL_64_after_hwframe 25.81 +5.6 31.44 +2.2 27.98 +0.1 25.96 perf-profile.calltrace.cycles-pp.__dentry_kill.dput.__fput.__x64_sys_close.do_syscall_64 25.79 +5.6 31.43 +2.2 27.96 +0.1 25.94 perf-profile.calltrace.cycles-pp.evict.__dentry_kill.dput.__fput.__x64_sys_close 25.78 +5.6 31.42 +2.2 27.95 +0.1 25.93 perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.__dentry_kill.dput.__fput 25.63 +5.7 31.33 +2.2 27.83 +0.1 25.78 perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill.dput 17.98 +6.1 24.07 +2.4 20.42 +0.1 18.10 perf-profile.calltrace.cycles-pp.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict.__dentry_kill 17.57 +6.2 23.81 +2.5 20.06 +0.1 17.69 perf-profile.calltrace.cycles-pp.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode.evict 17.33 +6.3 23.66 +2.5 19.86 +0.1 17.45 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range 17.34 +6.3 23.66 +2.5 19.86 +0.1 17.45 perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs.truncate_inode_pages_range.ext4_evict_inode 17.32 +6.3 23.65 +2.5 19.85 +0.1 17.44 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__page_cache_release.folios_put_refs 25.47 +6.5 31.92 +4.9 30.41 +0.1 25.58 ± 2% perf-profile.calltrace.cycles-pp.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 23.72 ± 2% +7.3 31.06 +5.3 29.06 +0.1 23.84 ± 2% perf-profile.calltrace.cycles-pp.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write.do_syscall_64 17.89 ± 3% +10.1 27.95 +6.5 24.39 +0.2 18.04 ± 3% perf-profile.calltrace.cycles-pp.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write.ksys_write 13.68 ± 4% +12.1 25.73 +7.3 20.95 +0.1 13.79 ± 4% perf-profile.calltrace.cycles-pp.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter.vfs_write 11.76 ± 5% +13.0 24.76 +7.7 19.42 +0.1 11.85 ± 5% perf-profile.calltrace.cycles-pp.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write.ext4_buffered_write_iter 9.80 ± 6% +13.4 23.16 +8.1 17.87 +0.3 10.09 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio 9.81 ± 6% +13.4 23.17 +8.1 17.87 +0.3 10.10 ± 6% perf-profile.calltrace.cycles-pp.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio 9.79 ± 6% +13.4 23.16 +8.1 17.86 +0.3 10.08 ± 6% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.folio_batch_move_lru.folio_add_lru 10.38 ± 6% +13.5 23.85 +8.1 18.48 +0.3 10.67 ± 5% perf-profile.calltrace.cycles-pp.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin.generic_perform_write 10.32 ± 6% +13.5 23.82 +8.1 18.43 +0.3 10.61 ± 5% perf-profile.calltrace.cycles-pp.folio_batch_move_lru.folio_add_lru.filemap_add_folio.__filemap_get_folio.ext4_da_write_begin 8.04 ± 8% -7.2 0.84 ± 9% -6.0 2.06 ± 17% -0.4 7.66 ± 9% perf-profile.children.cycles-pp.down_write 7.67 ± 8% -7.0 0.65 ± 12% -5.9 1.77 ± 19% -0.4 7.29 ± 10% perf-profile.children.cycles-pp.rwsem_down_write_slowpath 7.66 ± 8% -7.0 0.64 ± 12% -5.9 1.76 ± 19% -0.4 7.28 ± 10% perf-profile.children.cycles-pp.rwsem_optimistic_spin 7.28 ± 9% -6.8 0.53 ± 14% -5.7 1.54 ± 21% -0.4 6.90 ± 10% perf-profile.children.cycles-pp.osq_lock 4.35 ± 10% -4.1 0.30 ± 8% -3.5 0.81 ± 19% -0.2 4.14 ± 11% perf-profile.children.cycles-pp.unlink 4.34 ± 10% -4.0 0.29 ± 8% -3.5 0.80 ± 19% -0.2 4.12 ± 11% perf-profile.children.cycles-pp.__x64_sys_unlink 4.34 ± 10% -4.0 0.29 ± 8% -3.5 0.80 ± 20% -0.2 4.12 ± 11% perf-profile.children.cycles-pp.do_unlinkat 3.97 ± 6% -3.3 0.71 ± 8% -2.5 1.47 ± 13% -0.2 3.80 ± 7% perf-profile.children.cycles-pp.do_sys_openat2 3.95 ± 6% -3.2 0.70 ± 8% -2.5 1.45 ± 13% -0.2 3.78 ± 7% perf-profile.children.cycles-pp.path_openat 3.95 ± 6% -3.2 0.70 ± 8% -2.5 1.45 ± 13% -0.2 3.78 ± 7% perf-profile.children.cycles-pp.do_filp_open 3.93 ± 6% -3.2 0.69 ± 8% -2.5 1.44 ± 13% -0.2 3.76 ± 7% perf-profile.children.cycles-pp.creat64 3.92 ± 6% -3.2 0.68 ± 8% -2.5 1.43 ± 13% -0.2 3.75 ± 7% perf-profile.children.cycles-pp.__x64_sys_creat 3.87 ± 6% -3.2 0.66 ± 8% -2.5 1.39 ± 14% -0.2 3.70 ± 7% perf-profile.children.cycles-pp.open_last_lookups 3.70 -1.7 1.98 ± 2% -0.7 3.05 +0.1 3.76 perf-profile.children.cycles-pp.llseek 3.68 -1.7 2.00 ± 2% -0.6 3.08 +0.0 3.72 perf-profile.children.cycles-pp.ext4_block_write_begin 3.12 -1.4 1.67 ± 2% -0.5 2.57 +0.0 3.14 perf-profile.children.cycles-pp.clear_bhb_loop 2.90 -1.3 1.64 ± 2% -0.5 2.36 -0.0 2.90 perf-profile.children.cycles-pp.ext4_da_write_end 2.29 -1.0 1.31 ± 2% -0.4 1.85 +0.0 2.29 perf-profile.children.cycles-pp.block_write_end 2.20 -0.9 1.26 ± 2% -0.4 1.78 +0.0 2.20 perf-profile.children.cycles-pp.__block_commit_write 1.95 -0.8 1.15 ± 2% -0.3 1.68 -0.0 1.94 perf-profile.children.cycles-pp.copy_page_to_iter 1.99 -0.8 1.24 ± 2% -0.4 1.64 +0.0 2.00 perf-profile.children.cycles-pp.truncate_cleanup_folio 1.80 -0.7 1.08 ± 2% -0.2 1.56 -0.0 1.79 perf-profile.children.cycles-pp._copy_to_iter 1.54 -0.7 0.82 ± 2% -0.3 1.27 +0.0 1.56 perf-profile.children.cycles-pp.entry_SYSCALL_64 1.34 -0.7 0.68 ± 3% -0.3 1.06 +0.0 1.35 perf-profile.children.cycles-pp.ext4_da_get_block_prep 1.32 -0.7 0.67 ± 3% -0.3 1.04 -0.0 1.32 perf-profile.children.cycles-pp.ext4_da_map_blocks 31.04 -0.6 30.46 +0.6 31.63 +0.1 31.10 perf-profile.children.cycles-pp.read 1.19 -0.6 0.64 ± 2% -0.2 1.01 +0.0 1.24 perf-profile.children.cycles-pp.filemap_get_pages 1.18 -0.5 0.63 ± 2% -0.2 0.97 +0.0 1.20 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack 1.22 -0.5 0.68 ± 2% -0.3 0.92 -0.0 1.20 perf-profile.children.cycles-pp.mark_buffer_dirty 1.20 -0.5 0.70 ± 2% -0.2 1.02 -0.0 1.19 perf-profile.children.cycles-pp.zero_user_segments 1.20 -0.5 0.71 ± 2% -0.2 1.03 -0.0 1.20 perf-profile.children.cycles-pp.memset_orig 1.07 -0.5 0.58 ± 2% -0.2 0.87 -0.0 1.07 perf-profile.children.cycles-pp.copy_page_from_iter_atomic 0.95 -0.5 0.46 ± 3% -0.2 0.78 ± 4% +0.0 0.97 ± 4% perf-profile.children.cycles-pp.rw_verify_area 1.32 -0.5 0.84 ± 2% -0.3 1.06 +0.0 1.32 perf-profile.children.cycles-pp.try_to_free_buffers 0.98 -0.5 0.53 ± 2% -0.1 0.85 +0.1 1.04 perf-profile.children.cycles-pp.filemap_get_read_batch 0.98 ± 2% -0.4 0.53 ± 3% -0.2 0.80 +0.0 1.00 perf-profile.children.cycles-pp.__fdget_pos 0.77 ± 2% -0.4 0.34 ± 3% -0.2 0.58 ± 4% -0.0 0.74 perf-profile.children.cycles-pp.balance_dirty_pages_ratelimited_flags 1.10 ± 3% -0.4 0.67 ± 3% -0.3 0.80 ± 3% +0.0 1.11 ± 2% perf-profile.children.cycles-pp.workingset_activation 0.80 -0.4 0.37 ± 2% -0.2 0.60 -0.0 0.79 perf-profile.children.cycles-pp.file_modified 0.86 ± 4% -0.4 0.46 ± 3% -0.2 0.62 ± 3% +0.0 0.87 ± 3% perf-profile.children.cycles-pp.workingset_age_nonresident 0.77 ± 2% -0.4 0.37 ± 3% -0.1 0.64 ± 5% +0.0 0.79 ± 6% perf-profile.children.cycles-pp.security_file_permission 0.92 -0.4 0.52 ± 2% -0.2 0.68 -0.0 0.90 perf-profile.children.cycles-pp.__folio_mark_dirty 0.80 -0.4 0.42 ± 2% -0.1 0.65 +0.0 0.80 perf-profile.children.cycles-pp.xas_load 0.74 -0.4 0.37 ± 3% -0.2 0.58 +0.0 0.75 perf-profile.children.cycles-pp.folio_alloc_noprof 0.70 -0.4 0.33 ± 3% -0.2 0.54 +0.0 0.71 perf-profile.children.cycles-pp.touch_atime 0.60 ± 2% -0.4 0.24 ± 4% -0.2 0.42 -0.0 0.59 ± 2% perf-profile.children.cycles-pp.percpu_counter_add_batch 0.71 -0.4 0.35 ± 3% -0.2 0.55 -0.0 0.71 perf-profile.children.cycles-pp.alloc_pages_mpol_noprof 0.80 -0.4 0.45 ± 2% -0.1 0.74 +0.1 0.86 perf-profile.children.cycles-pp.create_empty_buffers 0.68 ± 2% -0.3 0.36 ± 3% -0.1 0.54 -0.0 0.68 perf-profile.children.cycles-pp.fault_in_iov_iter_readable 0.62 -0.3 0.30 ± 3% -0.1 0.48 +0.0 0.62 perf-profile.children.cycles-pp.__alloc_pages_noprof 0.59 ± 2% -0.3 0.28 ± 3% -0.1 0.48 ± 7% +0.0 0.60 ± 8% perf-profile.children.cycles-pp.apparmor_file_permission 0.62 -0.3 0.32 ± 3% -0.1 0.49 ± 2% -0.0 0.62 perf-profile.children.cycles-pp.__mod_memcg_lruvec_state 0.64 -0.3 0.34 ± 3% -0.1 0.52 +0.0 0.65 ± 2% perf-profile.children.cycles-pp.ksys_lseek 1.01 -0.3 0.71 ± 2% -0.4 0.62 -0.2 0.79 perf-profile.children.cycles-pp.__filemap_add_folio 0.62 -0.3 0.32 ± 3% -0.1 0.51 +0.0 0.64 perf-profile.children.cycles-pp.filemap_get_entry 0.57 -0.3 0.28 ± 3% -0.1 0.45 +0.0 0.58 perf-profile.children.cycles-pp.atime_needs_update 0.69 -0.3 0.40 ± 2% -0.2 0.50 -0.0 0.67 perf-profile.children.cycles-pp.folio_account_dirtied 0.61 -0.3 0.32 ± 2% -0.1 0.50 ± 3% +0.0 0.61 perf-profile.children.cycles-pp.__cond_resched 0.60 ± 2% -0.3 0.32 ± 3% -0.1 0.48 -0.0 0.59 ± 2% perf-profile.children.cycles-pp.fault_in_readable 0.62 -0.3 0.35 ± 2% -0.0 0.59 +0.1 0.67 perf-profile.children.cycles-pp.folio_alloc_buffers 0.58 -0.3 0.32 ± 2% -0.1 0.48 +0.0 0.59 perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.37 ± 3% -0.3 0.11 ± 4% -0.2 0.22 ± 6% -0.0 0.37 ± 3% perf-profile.children.cycles-pp.rwsem_spin_on_owner 0.40 ± 6% -0.3 0.14 ± 3% -0.1 0.26 ± 5% -0.0 0.38 ± 5% perf-profile.children.cycles-pp.ext4_file_write_iter 0.51 ± 3% -0.2 0.27 ± 3% -0.1 0.40 ± 2% -0.0 0.51 perf-profile.children.cycles-pp.disk_rr 0.59 -0.2 0.34 ± 2% -0.1 0.49 +0.0 0.60 perf-profile.children.cycles-pp.kmem_cache_free 0.46 -0.2 0.22 ± 2% -0.1 0.36 +0.0 0.46 perf-profile.children.cycles-pp.get_page_from_freelist 0.55 ± 2% -0.2 0.31 ± 3% -0.0 0.52 +0.0 0.59 perf-profile.children.cycles-pp.alloc_buffer_head 0.53 -0.2 0.30 ± 2% -0.0 0.51 +0.0 0.58 perf-profile.children.cycles-pp.kmem_cache_alloc_noprof 0.36 ± 3% -0.2 0.14 ± 4% -0.1 0.25 ± 2% -0.0 0.36 ± 2% perf-profile.children.cycles-pp.__mark_inode_dirty 0.36 ± 6% -0.2 0.14 ± 5% -0.1 0.25 ± 4% -0.0 0.36 ± 4% perf-profile.children.cycles-pp.ext4_file_read_iter 0.40 -0.2 0.20 ± 2% -0.1 0.33 ± 2% +0.0 0.42 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.44 -0.2 0.25 ± 3% -0.1 0.34 -0.0 0.42 perf-profile.children.cycles-pp.xas_store 0.39 -0.2 0.19 ± 3% -0.1 0.31 ± 3% -0.0 0.39 ± 2% perf-profile.children.cycles-pp.inode_needs_update_time 0.52 -0.2 0.33 ± 2% -0.1 0.42 +0.0 0.52 perf-profile.children.cycles-pp.delete_from_page_cache_batch 0.29 ± 4% -0.2 0.10 ± 6% -0.1 0.19 ± 3% -0.0 0.29 ± 3% perf-profile.children.cycles-pp.generic_update_time 0.36 ± 2% -0.2 0.18 ± 4% -0.1 0.28 +0.0 0.37 perf-profile.children.cycles-pp.ext4_da_reserve_space 0.57 ± 2% -0.2 0.40 ± 3% -0.1 0.44 +0.0 0.57 perf-profile.children.cycles-pp.__folio_cancel_dirty 0.34 ± 2% -0.2 0.18 ± 3% -0.1 0.29 ± 2% +0.0 0.36 perf-profile.children.cycles-pp.__mem_cgroup_charge 0.25 ± 4% -0.2 0.09 ± 5% -0.1 0.16 ± 4% -0.0 0.25 ± 3% perf-profile.children.cycles-pp.ext4_dirty_inode 0.33 -0.2 0.17 ± 2% -0.1 0.25 -0.0 0.32 ± 2% perf-profile.children.cycles-pp.ext4_es_insert_delayed_block 0.34 -0.2 0.18 ± 2% -0.1 0.27 ± 2% +0.0 0.34 ± 2% perf-profile.children.cycles-pp.ext4_generic_write_checks 0.22 ± 5% -0.2 0.06 ± 9% -0.1 0.13 ± 3% -0.0 0.21 ± 6% perf-profile.children.cycles-pp.jbd2__journal_start 0.21 ± 5% -0.2 0.06 ± 8% -0.1 0.12 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.start_this_handle 0.31 -0.1 0.17 ± 3% -0.1 0.25 ± 2% +0.0 0.31 perf-profile.children.cycles-pp._raw_spin_lock 0.40 -0.1 0.25 ± 2% -0.1 0.30 ± 2% +0.0 0.40 ± 3% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 0.30 ± 2% -0.1 0.17 ± 3% -0.0 0.30 ± 3% +0.0 0.34 perf-profile.children.cycles-pp.__memcg_slab_post_alloc_hook 0.31 ± 2% -0.1 0.18 ± 2% -0.1 0.26 ± 2% +0.0 0.31 perf-profile.children.cycles-pp.block_invalidate_folio 0.29 ± 2% -0.1 0.16 ± 2% -0.0 0.24 ± 2% +0.0 0.30 perf-profile.children.cycles-pp.x64_sys_call 0.49 -0.1 0.36 ± 3% -0.1 0.38 -0.0 0.48 perf-profile.children.cycles-pp.folio_account_cleaned 0.32 -0.1 0.19 ± 2% -0.1 0.25 ± 2% +0.0 0.32 ± 3% perf-profile.children.cycles-pp.lookup_open 0.27 -0.1 0.14 ± 2% -0.1 0.22 ± 2% -0.0 0.27 ± 2% perf-profile.children.cycles-pp.generic_write_checks 0.26 ± 3% -0.1 0.13 ± 3% -0.1 0.21 ± 3% -0.0 0.26 ± 2% perf-profile.children.cycles-pp.rcu_all_qs 0.26 ± 2% -0.1 0.13 ± 4% -0.1 0.20 +0.0 0.26 ± 3% perf-profile.children.cycles-pp.ext4_es_lookup_extent 0.27 ± 2% -0.1 0.14 ± 4% -0.1 0.22 ± 2% +0.0 0.27 perf-profile.children.cycles-pp.up_write 0.27 -0.1 0.14 ± 3% -0.0 0.22 +0.0 0.27 ± 2% perf-profile.children.cycles-pp.xas_start 0.35 ± 2% -0.1 0.22 -0.1 0.27 ± 2% +0.0 0.35 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt 0.17 ± 4% -0.1 0.06 ± 6% -0.1 0.12 ± 4% -0.0 0.17 ± 3% perf-profile.children.cycles-pp.ext4_nonda_switch 0.23 -0.1 0.12 ± 3% -0.0 0.18 -0.0 0.23 ± 2% perf-profile.children.cycles-pp.folio_unlock 0.22 ± 2% -0.1 0.12 ± 3% -0.0 0.18 ± 2% +0.0 0.23 ± 3% perf-profile.children.cycles-pp.current_time 0.27 -0.1 0.16 ± 3% -0.0 0.23 ± 2% +0.0 0.27 ± 2% perf-profile.children.cycles-pp.__memcg_slab_free_hook 0.27 ± 2% -0.1 0.17 ± 3% -0.1 0.20 ± 2% -0.0 0.27 ± 4% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt 0.18 -0.1 0.08 ± 5% -0.0 0.14 ± 3% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.node_dirty_ok 0.21 ± 2% -0.1 0.12 ± 4% -0.0 0.17 -0.0 0.21 perf-profile.children.cycles-pp.__slab_free 0.26 ± 3% -0.1 0.16 ± 2% -0.1 0.20 ± 2% +0.0 0.26 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt 0.93 ± 2% -0.1 0.84 ± 3% -0.2 0.72 ± 2% -0.0 0.92 perf-profile.children.cycles-pp.__lruvec_stat_mod_folio 0.18 ± 2% -0.1 0.09 ± 3% -0.0 0.15 ± 3% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.19 ± 2% -0.1 0.10 ± 5% -0.0 0.14 ± 3% +0.0 0.19 ± 2% perf-profile.children.cycles-pp.aa_file_perm 0.20 -0.1 0.11 ± 2% -0.0 0.16 ± 2% +0.0 0.20 perf-profile.children.cycles-pp.__mod_node_page_state 0.21 -0.1 0.12 ± 2% -0.0 0.16 ± 2% -0.0 0.21 ± 3% perf-profile.children.cycles-pp.ext4_create 0.18 -0.1 0.09 ± 4% -0.0 0.14 +0.0 0.18 ± 2% perf-profile.children.cycles-pp.rmqueue 0.20 ± 2% -0.1 0.12 ± 4% -0.0 0.17 +0.0 0.21 ± 2% perf-profile.children.cycles-pp.find_lock_entries 0.17 ± 5% -0.1 0.08 ± 5% -0.0 0.14 ± 7% -0.0 0.17 ± 5% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.19 ± 2% -0.1 0.10 ± 4% -0.0 0.15 ± 3% +0.0 0.19 ± 2% perf-profile.children.cycles-pp.cgroup_rstat_updated 0.17 -0.1 0.09 ± 4% -0.0 0.13 ± 3% +0.0 0.17 perf-profile.children.cycles-pp.__dquot_alloc_space 0.08 -0.1 0.00 -0.0 0.06 ± 5% +0.0 0.08 perf-profile.children.cycles-pp.__mod_zone_page_state 0.21 ± 2% -0.1 0.13 ± 3% -0.1 0.16 +0.0 0.21 ± 4% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.08 ± 5% -0.1 0.00 -0.0 0.06 ± 7% -0.0 0.08 ± 6% perf-profile.children.cycles-pp._raw_read_lock 0.08 ± 5% -0.1 0.00 -0.0 0.06 -0.0 0.07 ± 5% perf-profile.children.cycles-pp.rcu_core 0.17 -0.1 0.09 ± 4% -0.0 0.14 ± 2% +0.0 0.17 perf-profile.children.cycles-pp.free_unref_folios 0.09 ± 3% -0.1 0.01 ±163% -0.0 0.07 ± 4% +0.0 0.09 ± 3% perf-profile.children.cycles-pp.generic_file_llseek_size 0.08 ± 6% -0.1 0.00 -0.0 0.06 ± 5% -0.0 0.07 ± 5% perf-profile.children.cycles-pp.rcu_do_batch 0.20 ± 3% -0.1 0.13 ± 3% -0.0 0.15 ± 2% +0.0 0.20 ± 4% perf-profile.children.cycles-pp.tick_nohz_handler 0.16 ± 2% -0.1 0.09 ± 4% -0.0 0.13 ± 3% +0.0 0.16 ± 3% perf-profile.children.cycles-pp.vfs_unlink 0.08 ± 4% -0.1 0.00 ±316% -0.0 0.06 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.xas_create 0.09 ± 5% -0.1 0.01 ±163% -0.0 0.07 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.generic_file_read_iter 0.19 -0.1 0.12 ± 4% -0.0 0.16 ± 2% +0.0 0.19 perf-profile.children.cycles-pp.mod_objcg_state 0.14 ± 3% -0.1 0.07 ± 6% -0.0 0.11 ± 4% +0.0 0.15 ± 3% perf-profile.children.cycles-pp.__es_insert_extent 0.16 ± 3% -0.1 0.08 ± 5% -0.0 0.12 ± 2% -0.0 0.16 ± 3% perf-profile.children.cycles-pp.__ext4_unlink 0.14 ± 2% -0.1 0.07 -0.0 0.11 +0.0 0.14 ± 3% perf-profile.children.cycles-pp.__radix_tree_lookup 0.14 ± 4% -0.1 0.07 -0.0 0.11 ± 2% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.__count_memcg_events 0.07 ± 4% -0.1 0.00 -0.0 0.06 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.__es_remove_extent 0.07 ± 4% -0.1 0.00 -0.0 0.06 ± 5% +0.0 0.07 ± 6% perf-profile.children.cycles-pp.read@plt 0.16 ± 3% -0.1 0.09 ± 5% -0.0 0.12 ± 2% +0.0 0.16 ± 3% perf-profile.children.cycles-pp.ext4_unlink 0.08 ± 6% -0.1 0.00 ±316% -0.0 0.06 ± 5% +0.0 0.08 ± 4% perf-profile.children.cycles-pp.xas_clear_mark 0.08 ± 5% -0.1 0.01 ±163% -0.0 0.07 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.folio_mapping 0.07 -0.1 0.00 -0.0 0.05 +0.0 0.07 perf-profile.children.cycles-pp.node_page_state 0.07 -0.1 0.00 -0.0 0.06 +0.0 0.07 perf-profile.children.cycles-pp.ext4_fill_raw_inode 0.18 ± 2% -0.1 0.12 ± 4% -0.0 0.14 +0.0 0.19 ± 4% perf-profile.children.cycles-pp.update_process_times 0.15 ± 3% -0.1 0.08 ± 3% -0.0 0.12 ± 4% +0.0 0.15 ± 3% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers 0.12 ± 5% -0.1 0.05 ± 8% -0.0 0.09 ± 3% +0.0 0.12 ± 2% perf-profile.children.cycles-pp.ext4_claim_free_clusters 0.07 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.folio_wait_stable 0.07 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.free_unref_page_commit 0.07 ± 7% -0.1 0.00 -0.0 0.06 ± 5% +0.0 0.08 perf-profile.children.cycles-pp.obj_cgroup_charge 0.06 ± 7% -0.1 0.00 -0.0 0.05 ± 6% -0.0 0.06 ± 5% perf-profile.children.cycles-pp.balance_dirty_pages 0.13 -0.1 0.07 ± 7% -0.0 0.10 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.generic_write_check_limits 0.12 ± 4% -0.1 0.06 ± 7% -0.0 0.10 +0.0 0.12 ± 4% perf-profile.children.cycles-pp.__xa_set_mark 0.12 ± 2% -0.1 0.06 -0.0 0.10 ± 5% +0.0 0.12 ± 2% perf-profile.children.cycles-pp.file_remove_privs_flags 0.13 ± 3% -0.1 0.07 ± 7% -0.0 0.10 ± 3% -0.0 0.13 ± 3% perf-profile.children.cycles-pp.ext4_llseek 0.07 -0.1 0.01 ±212% -0.0 0.06 ± 9% +0.0 0.07 ± 4% perf-profile.children.cycles-pp.irq_exit_rcu 0.06 -0.1 0.00 -0.0 0.04 ± 37% +0.0 0.06 ± 5% perf-profile.children.cycles-pp.add_dirent_to_buf 0.06 -0.1 0.00 -0.0 0.05 +0.0 0.06 perf-profile.children.cycles-pp._raw_spin_trylock 0.06 -0.1 0.00 -0.0 0.05 +0.0 0.06 perf-profile.children.cycles-pp.bdev_getblk 0.06 -0.1 0.00 -0.0 0.05 +0.0 0.06 perf-profile.children.cycles-pp.crc32c_pcl_intel_update 0.15 ± 2% -0.1 0.09 ± 4% -0.0 0.12 ± 3% +0.0 0.15 ± 2% perf-profile.children.cycles-pp.__ext4_mark_inode_dirty 0.12 -0.1 0.06 ± 6% -0.0 0.10 ± 4% +0.0 0.12 perf-profile.children.cycles-pp.amd_clear_divider 0.12 ± 2% -0.1 0.06 ± 6% -0.0 0.09 ± 5% +0.0 0.12 ± 2% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head 0.11 ± 5% -0.1 0.06 ± 6% -0.0 0.09 ± 5% -0.0 0.11 ± 5% perf-profile.children.cycles-pp.inode_to_bdi 0.09 ± 5% -0.1 0.04 ± 47% -0.0 0.07 -0.0 0.09 ± 3% perf-profile.children.cycles-pp.handle_softirqs 0.50 -0.1 0.45 ± 3% +0.0 0.55 +0.0 0.51 perf-profile.children.cycles-pp.folio_activate_fn 0.11 ± 2% -0.1 0.06 -0.0 0.09 -0.0 0.11 perf-profile.children.cycles-pp.__ext4_new_inode 0.12 ± 2% -0.1 0.07 -0.0 0.10 -0.0 0.12 perf-profile.children.cycles-pp.try_charge_memcg 0.11 ± 4% -0.1 0.06 ± 4% -0.0 0.09 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.timestamp_truncate 0.12 ± 3% -0.1 0.07 ± 6% -0.0 0.10 ± 3% +0.0 0.12 ± 5% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack 0.12 ± 3% -0.1 0.07 ± 6% -0.0 0.10 ± 3% -0.0 0.12 ± 2% perf-profile.children.cycles-pp.drop_buffers 0.13 ± 3% -0.0 0.08 ± 3% -0.0 0.10 ± 3% +0.0 0.13 ± 4% perf-profile.children.cycles-pp.__ext4_find_entry 0.11 ± 3% -0.0 0.06 ± 6% -0.0 0.09 ± 3% +0.0 0.11 perf-profile.children.cycles-pp.ext4_mark_iloc_dirty 0.12 ± 8% -0.0 0.08 ± 6% +0.1 0.20 ± 3% -0.0 0.12 ± 9% perf-profile.children.cycles-pp.mem_cgroup_update_lru_size 0.12 -0.0 0.08 ± 5% -0.0 0.10 ± 5% +0.0 0.12 ± 5% perf-profile.children.cycles-pp.ext4_dx_find_entry 0.13 ± 3% -0.0 0.09 -0.0 0.11 ± 4% +0.0 0.13 ± 3% perf-profile.children.cycles-pp.sched_tick 0.10 ± 4% -0.0 0.06 ± 7% -0.0 0.08 ± 4% +0.0 0.10 ± 3% perf-profile.children.cycles-pp.ext4_do_update_inode 0.09 -0.0 0.05 -0.0 0.07 +0.0 0.09 ± 3% perf-profile.children.cycles-pp.__mem_cgroup_uncharge_folios 0.08 ± 4% -0.0 0.04 ± 47% -0.0 0.06 ± 7% +0.0 0.08 ± 4% perf-profile.children.cycles-pp.ext4_reserve_inode_write 0.17 -0.0 0.13 ± 3% -0.0 0.13 ± 3% -0.0 0.17 perf-profile.children.cycles-pp.filemap_unaccount_folio 0.09 ± 4% -0.0 0.05 ± 9% -0.0 0.07 ± 4% +0.0 0.09 ± 4% perf-profile.children.cycles-pp.ext4_lookup 0.09 ± 4% -0.0 0.06 -0.0 0.07 +0.0 0.10 ± 5% perf-profile.children.cycles-pp.task_tick_fair 0.09 ± 3% -0.0 0.06 ± 8% -0.0 0.07 +0.0 0.09 ± 3% perf-profile.children.cycles-pp.ext4_add_nondir 0.24 ± 2% -0.0 0.22 ± 2% -0.2 0.06 -0.2 0.08 ± 5% perf-profile.children.cycles-pp.xas_find_conflict 0.08 ± 4% -0.0 0.05 -0.0 0.06 +0.0 0.08 ± 4% perf-profile.children.cycles-pp.ext4_add_entry 0.08 ± 5% -0.0 0.05 -0.0 0.06 +0.0 0.08 ± 5% perf-profile.children.cycles-pp.ext4_dx_add_entry 0.13 ± 3% -0.0 0.11 ± 5% -0.0 0.11 +0.0 0.14 ± 3% perf-profile.children.cycles-pp.__mod_lruvec_state 0.07 -0.0 0.05 ± 31% -0.0 0.05 ± 6% -0.0 0.07 ± 8% perf-profile.children.cycles-pp.ext4_search_dir 0.26 ± 3% -0.0 0.24 ± 8% +0.1 0.36 ± 4% +0.0 0.26 ± 6% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.26 ± 3% -0.0 0.24 ± 8% +0.1 0.36 ± 4% +0.0 0.26 ± 6% perf-profile.children.cycles-pp.do_group_exit 0.26 ± 3% -0.0 0.24 ± 9% +0.1 0.36 ± 3% +0.0 0.27 ± 5% perf-profile.children.cycles-pp.do_exit 0.24 ± 4% -0.0 0.24 ± 8% +0.1 0.35 ± 4% +0.0 0.25 ± 6% perf-profile.children.cycles-pp.exit_mm 0.25 ± 3% -0.0 0.25 ± 8% +0.1 0.35 ± 4% +0.0 0.26 ± 6% perf-profile.children.cycles-pp.__mmput 0.25 ± 3% -0.0 0.25 ± 8% +0.1 0.35 ± 4% +0.0 0.26 ± 6% perf-profile.children.cycles-pp.exit_mmap 0.20 ± 3% +0.0 0.22 ± 9% +0.1 0.30 ± 4% +0.0 0.21 ± 7% perf-profile.children.cycles-pp.__tlb_batch_free_encoded_pages 0.20 ± 3% +0.0 0.22 ± 9% +0.1 0.30 ± 4% +0.0 0.21 ± 7% perf-profile.children.cycles-pp.free_pages_and_swap_cache 0.20 ± 3% +0.0 0.22 ± 9% +0.1 0.31 ± 4% +0.0 0.21 ± 7% perf-profile.children.cycles-pp.tlb_finish_mmu 0.34 ± 2% +0.0 0.37 -0.0 0.32 +0.0 0.34 perf-profile.children.cycles-pp.lru_add_fn 0.00 +0.1 0.05 ± 5% +0.1 0.05 +0.0 0.00 perf-profile.children.cycles-pp.lru_add_drain 0.08 ± 5% +0.1 0.15 ± 4% +0.0 0.11 +0.0 0.09 ± 5% perf-profile.children.cycles-pp.__cmd_record 0.08 ± 5% +0.1 0.15 ± 4% +0.0 0.11 +0.0 0.09 ± 5% perf-profile.children.cycles-pp.cmd_record 0.08 ± 4% +0.1 0.14 ± 4% +0.0 0.10 ± 4% +0.0 0.08 ± 4% perf-profile.children.cycles-pp.record__mmap_read_evlist 0.09 ± 3% +0.1 0.15 ± 4% +0.0 0.11 ± 2% +0.0 0.09 ± 3% perf-profile.children.cycles-pp.main 0.09 ± 3% +0.1 0.15 ± 4% +0.0 0.11 ± 2% +0.0 0.09 ± 3% perf-profile.children.cycles-pp.run_builtin 0.08 ± 6% +0.1 0.14 ± 4% +0.0 0.10 +0.0 0.08 ± 4% perf-profile.children.cycles-pp.perf_mmap__push 0.07 ± 4% +0.1 0.13 ± 3% +0.0 0.09 ± 3% +0.0 0.07 ± 4% perf-profile.children.cycles-pp.record__pushfn 0.07 ± 4% +0.1 0.13 ± 3% +0.0 0.09 ± 3% +0.0 0.07 ± 4% perf-profile.children.cycles-pp.writen 0.06 +0.1 0.13 ± 5% +0.0 0.09 ± 4% +0.0 0.06 ± 7% perf-profile.children.cycles-pp.shmem_file_write_iter 0.00 +0.1 0.10 ± 4% +0.1 0.06 ± 6% +0.0 0.00 perf-profile.children.cycles-pp.shmem_alloc_and_add_folio 0.00 +0.1 0.11 ± 6% +0.1 0.07 ± 7% +0.0 0.01 ±173% perf-profile.children.cycles-pp.shmem_get_folio_gfp 0.00 +0.1 0.11 ± 6% +0.1 0.07 ± 7% +0.0 0.01 ±173% perf-profile.children.cycles-pp.shmem_write_begin 2.45 +0.2 2.70 +0.1 2.55 +0.0 2.45 perf-profile.children.cycles-pp.lru_add_drain_cpu 4.86 +0.7 5.54 +0.3 5.13 +0.0 4.87 perf-profile.children.cycles-pp.__folio_batch_release 27.93 +0.9 28.79 +1.1 29.05 +0.0 27.95 perf-profile.children.cycles-pp.ksys_read 27.50 +1.1 28.56 +1.2 28.70 +0.0 27.51 perf-profile.children.cycles-pp.vfs_read 25.97 +1.8 27.81 +1.5 27.48 -0.0 25.96 perf-profile.children.cycles-pp.filemap_read 92.83 +3.3 96.09 +1.3 94.11 -0.1 92.76 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 92.42 +3.5 95.88 +1.4 93.79 -0.1 92.35 perf-profile.children.cycles-pp.do_syscall_64 30.73 +3.9 34.66 +3.8 34.58 +0.1 30.85 perf-profile.children.cycles-pp.write 21.08 +4.1 25.15 +2.3 23.40 -0.0 21.04 perf-profile.children.cycles-pp.folio_mark_accessed 19.70 +4.6 24.33 +2.7 22.38 -0.0 19.66 perf-profile.children.cycles-pp.folio_activate 27.83 +5.3 33.14 +4.4 32.25 +0.1 27.94 perf-profile.children.cycles-pp.ksys_write 27.36 +5.5 32.89 +4.5 31.87 +0.1 27.46 perf-profile.children.cycles-pp.vfs_write 25.87 +5.6 31.48 +2.2 28.03 +0.1 26.02 perf-profile.children.cycles-pp.__close 25.86 +5.6 31.46 +2.2 28.01 +0.1 26.00 perf-profile.children.cycles-pp.__x64_sys_close 25.84 +5.6 31.46 +2.2 28.00 +0.1 25.99 perf-profile.children.cycles-pp.__fput 25.83 +5.6 31.45 +2.2 27.99 +0.1 25.98 perf-profile.children.cycles-pp.dput 25.82 +5.6 31.44 +2.2 27.98 +0.1 25.96 perf-profile.children.cycles-pp.__dentry_kill 25.79 +5.6 31.43 +2.2 27.96 +0.1 25.94 perf-profile.children.cycles-pp.evict 25.78 +5.6 31.42 +2.2 27.95 +0.1 25.93 perf-profile.children.cycles-pp.ext4_evict_inode 25.64 +5.7 31.34 +2.2 27.83 +0.1 25.78 perf-profile.children.cycles-pp.truncate_inode_pages_range 18.31 +6.0 24.35 +2.5 20.82 +0.1 18.43 perf-profile.children.cycles-pp.folios_put_refs 17.78 +6.3 24.04 +2.6 20.37 +0.1 17.90 perf-profile.children.cycles-pp.__page_cache_release 25.54 +6.4 31.96 +4.9 30.47 +0.1 25.66 ± 2% perf-profile.children.cycles-pp.ext4_buffered_write_iter 23.87 ± 2% +7.4 31.23 +5.3 29.21 +0.1 24.00 ± 2% perf-profile.children.cycles-pp.generic_perform_write 17.93 ± 3% +10.0 27.97 +6.5 24.43 +0.2 18.08 ± 3% perf-profile.children.cycles-pp.ext4_da_write_begin 13.75 ± 4% +12.0 25.77 +7.3 21.00 +0.1 13.86 ± 4% perf-profile.children.cycles-pp.__filemap_get_folio 11.78 ± 5% +13.0 24.77 +7.7 19.43 +0.1 11.86 ± 5% perf-profile.children.cycles-pp.filemap_add_folio 10.41 ± 6% +13.5 23.96 +8.1 18.54 +0.3 10.71 ± 5% perf-profile.children.cycles-pp.folio_add_lru 34.89 ± 2% +18.9 53.82 +11.1 46.00 +0.3 35.15 ± 2% perf-profile.children.cycles-pp.folio_batch_move_lru 51.11 +25.1 76.23 +13.7 64.78 +0.4 51.49 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 51.02 +25.2 76.18 +13.7 64.70 +0.4 51.40 perf-profile.children.cycles-pp.folio_lruvec_lock_irqsave 50.98 +25.3 76.24 +13.7 64.69 +0.4 51.36 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 7.25 ± 9% -6.7 0.53 ± 14% -5.7 1.54 ± 21% -0.4 6.87 ± 10% perf-profile.self.cycles-pp.osq_lock 3.09 -1.4 1.65 ± 2% -0.5 2.55 +0.0 3.11 perf-profile.self.cycles-pp.clear_bhb_loop 1.79 -0.7 1.07 ± 2% -0.2 1.54 -0.0 1.77 perf-profile.self.cycles-pp._copy_to_iter 1.14 -0.5 0.61 ± 2% -0.2 0.94 +0.0 1.16 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack 1.20 -0.5 0.70 ± 2% -0.2 1.02 -0.0 1.19 perf-profile.self.cycles-pp.memset_orig 1.06 -0.5 0.57 ± 2% -0.2 0.86 -0.0 1.05 perf-profile.self.cycles-pp.copy_page_from_iter_atomic 0.97 -0.5 0.51 ± 2% -0.2 0.79 +0.0 0.97 perf-profile.self.cycles-pp.filemap_read 0.94 ± 2% -0.4 0.51 ± 3% -0.2 0.78 +0.0 0.96 perf-profile.self.cycles-pp.__fdget_pos 0.86 ± 4% -0.4 0.45 ± 3% -0.2 0.62 ± 4% +0.0 0.87 ± 3% perf-profile.self.cycles-pp.workingset_age_nonresident 0.75 ± 2% -0.4 0.38 ± 3% -0.2 0.59 +0.0 0.76 perf-profile.self.cycles-pp.vfs_write 0.89 -0.3 0.55 ± 2% -0.1 0.80 +0.0 0.91 perf-profile.self.cycles-pp.__block_commit_write 0.54 ± 3% -0.3 0.21 ± 4% -0.2 0.38 -0.0 0.54 ± 2% perf-profile.self.cycles-pp.percpu_counter_add_batch 0.50 ± 3% -0.3 0.20 ± 3% -0.1 0.37 ± 6% -0.0 0.48 ± 2% perf-profile.self.cycles-pp.balance_dirty_pages_ratelimited_flags 0.63 -0.3 0.33 ± 2% -0.1 0.52 +0.0 0.64 perf-profile.self.cycles-pp.vfs_read 0.57 -0.3 0.29 ± 2% -0.1 0.46 +0.0 0.58 perf-profile.self.cycles-pp.do_syscall_64 0.61 -0.3 0.33 ± 2% -0.1 0.54 +0.0 0.65 perf-profile.self.cycles-pp.filemap_get_read_batch 0.58 ± 2% -0.3 0.31 ± 3% -0.1 0.46 -0.0 0.58 ± 2% perf-profile.self.cycles-pp.fault_in_readable 0.37 ± 3% -0.3 0.10 ± 4% -0.2 0.22 ± 6% +0.0 0.37 ± 3% perf-profile.self.cycles-pp.rwsem_spin_on_owner 0.56 -0.3 0.29 ± 3% -0.1 0.45 ± 2% +0.0 0.56 perf-profile.self.cycles-pp.xas_load 0.38 ± 6% -0.3 0.13 ± 4% -0.1 0.25 ± 6% -0.0 0.37 ± 5% perf-profile.self.cycles-pp.ext4_file_write_iter 0.48 ± 2% -0.2 0.24 ± 4% -0.1 0.37 ± 2% -0.0 0.47 perf-profile.self.cycles-pp.__mod_memcg_lruvec_state 0.35 ± 5% -0.2 0.13 ± 5% -0.1 0.24 ± 4% -0.0 0.35 ± 4% perf-profile.self.cycles-pp.ext4_file_read_iter 0.46 ± 2% -0.2 0.24 ± 2% -0.1 0.36 ± 3% -0.0 0.46 ± 2% perf-profile.self.cycles-pp.write 0.45 ± 2% -0.2 0.23 ± 6% -0.1 0.35 ± 4% -0.0 0.44 ± 3% perf-profile.self.cycles-pp.disk_rr 0.39 ± 3% -0.2 0.18 ± 4% -0.1 0.33 ± 10% +0.0 0.40 ± 11% perf-profile.self.cycles-pp.apparmor_file_permission 0.36 ± 4% -0.2 0.15 ± 4% -0.1 0.26 ± 3% -0.0 0.36 ± 3% perf-profile.self.cycles-pp.ext4_da_write_begin 0.44 -0.2 0.23 ± 2% -0.1 0.37 -0.0 0.44 perf-profile.self.cycles-pp.ext4_da_write_end 0.44 -0.2 0.23 ± 2% -0.1 0.36 +0.0 0.45 perf-profile.self.cycles-pp.__filemap_get_folio 0.42 -0.2 0.21 ± 2% -0.1 0.34 +0.0 0.42 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.42 ± 2% -0.2 0.22 ± 3% -0.1 0.33 ± 2% +0.0 0.42 perf-profile.self.cycles-pp.generic_perform_write 0.40 -0.2 0.20 ± 2% -0.1 0.33 ± 2% +0.0 0.42 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.43 ± 2% -0.2 0.24 ± 3% -0.1 0.36 ± 2% +0.0 0.44 ± 2% perf-profile.self.cycles-pp.read 0.40 ± 2% -0.2 0.21 ± 3% -0.1 0.32 ± 2% +0.0 0.41 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.41 -0.2 0.22 ± 3% -0.1 0.34 +0.0 0.41 perf-profile.self.cycles-pp.llseek 0.33 -0.2 0.17 ± 2% -0.1 0.26 ± 2% -0.0 0.32 ± 2% perf-profile.self.cycles-pp.ext4_block_write_begin 0.33 ± 2% -0.2 0.17 ± 3% -0.1 0.26 -0.0 0.33 perf-profile.self.cycles-pp.__cond_resched 0.28 ± 2% -0.2 0.13 ± 3% -0.1 0.22 ± 2% +0.0 0.28 ± 2% perf-profile.self.cycles-pp.atime_needs_update 0.30 ± 2% -0.1 0.16 ± 3% -0.1 0.24 +0.0 0.31 perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.30 -0.1 0.16 ± 4% -0.1 0.25 +0.0 0.32 ± 2% perf-profile.self.cycles-pp.filemap_get_entry 0.30 -0.1 0.16 ± 3% -0.1 0.24 ± 2% -0.0 0.29 ± 2% perf-profile.self.cycles-pp._raw_spin_lock 0.28 -0.1 0.14 ± 4% -0.1 0.22 -0.0 0.27 perf-profile.self.cycles-pp.folio_mark_accessed 0.27 -0.1 0.14 ± 2% -0.1 0.22 ± 2% +0.0 0.27 perf-profile.self.cycles-pp.mark_buffer_dirty 0.12 ± 5% -0.1 0.00 -0.0 0.07 ± 5% +0.0 0.12 ± 6% perf-profile.self.cycles-pp.start_this_handle 0.26 -0.1 0.14 ± 3% -0.1 0.21 ± 3% +0.0 0.26 perf-profile.self.cycles-pp.ext4_da_map_blocks 0.25 -0.1 0.13 ± 3% -0.1 0.19 ± 2% -0.0 0.24 perf-profile.self.cycles-pp.down_write 0.25 -0.1 0.14 ± 2% -0.0 0.21 ± 2% +0.0 0.26 perf-profile.self.cycles-pp.x64_sys_call 0.27 ± 2% -0.1 0.16 ± 2% -0.0 0.22 ± 2% -0.0 0.27 perf-profile.self.cycles-pp.block_invalidate_folio 0.24 ± 2% -0.1 0.12 ± 3% -0.0 0.19 ± 3% +0.0 0.24 ± 2% perf-profile.self.cycles-pp.up_write 0.16 ± 4% -0.1 0.06 ± 7% -0.1 0.11 ± 4% +0.0 0.16 ± 3% perf-profile.self.cycles-pp.ext4_nonda_switch 0.20 ± 2% -0.1 0.10 -0.0 0.16 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.inode_needs_update_time 0.21 ± 2% -0.1 0.11 -0.0 0.17 +0.0 0.21 ± 2% perf-profile.self.cycles-pp.folio_unlock 0.23 -0.1 0.13 ± 3% -0.1 0.17 -0.0 0.21 perf-profile.self.cycles-pp.xas_store 0.21 ± 2% -0.1 0.11 -0.0 0.18 ± 2% +0.0 0.22 ± 2% perf-profile.self.cycles-pp.xas_start 0.20 ± 2% -0.1 0.10 ± 4% -0.0 0.16 ± 2% -0.0 0.20 ± 2% perf-profile.self.cycles-pp.filemap_get_pages 0.20 ± 2% -0.1 0.10 ± 4% -0.0 0.16 ± 2% +0.0 0.21 ± 4% perf-profile.self.cycles-pp.security_file_permission 0.20 ± 4% -0.1 0.10 ± 4% -0.0 0.16 ± 3% -0.0 0.20 perf-profile.self.cycles-pp.rcu_all_qs 0.22 -0.1 0.12 ± 3% -0.0 0.19 +0.0 0.22 ± 2% perf-profile.self.cycles-pp.folios_put_refs 0.20 ± 2% -0.1 0.11 ± 3% -0.0 0.16 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.__slab_free 0.18 -0.1 0.09 ± 3% -0.0 0.13 -0.0 0.17 perf-profile.self.cycles-pp.__filemap_add_folio 0.17 ± 2% -0.1 0.09 ± 4% -0.0 0.13 ± 3% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.ext4_buffered_write_iter 0.18 ± 2% -0.1 0.09 ± 5% -0.0 0.14 ± 2% +0.0 0.18 perf-profile.self.cycles-pp.rw_verify_area 0.08 ± 4% -0.1 0.00 -0.0 0.07 ± 6% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.fault_in_iov_iter_readable 0.08 -0.1 0.00 -0.0 0.06 -0.0 0.07 perf-profile.self.cycles-pp.ext4_es_insert_delayed_block 0.08 -0.1 0.00 -0.0 0.06 +0.0 0.08 perf-profile.self.cycles-pp.__es_insert_extent 0.08 -0.1 0.00 -0.0 0.06 +0.0 0.08 ± 5% perf-profile.self.cycles-pp.rmqueue 0.08 ± 8% -0.1 0.00 -0.0 0.06 ± 6% -0.0 0.08 ± 7% perf-profile.self.cycles-pp.inode_to_bdi 0.17 -0.1 0.09 ± 4% -0.0 0.13 ± 3% +0.0 0.17 perf-profile.self.cycles-pp.__mod_node_page_state 0.08 ± 6% -0.1 0.00 -0.0 0.05 ± 9% -0.0 0.07 ± 5% perf-profile.self.cycles-pp._raw_read_lock 0.08 ± 6% -0.1 0.00 -0.0 0.06 +0.0 0.08 ± 5% perf-profile.self.cycles-pp.generic_file_llseek_size 0.16 ± 3% -0.1 0.08 ± 3% -0.0 0.12 ± 2% +0.0 0.16 ± 3% perf-profile.self.cycles-pp.aa_file_perm 0.15 -0.1 0.07 ± 6% -0.0 0.12 +0.0 0.15 ± 2% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.16 ± 2% -0.1 0.09 ± 3% -0.0 0.13 ± 3% +0.0 0.17 ± 2% perf-profile.self.cycles-pp.cgroup_rstat_updated 0.07 ± 6% -0.1 0.00 -0.0 0.06 -0.0 0.07 ± 4% perf-profile.self.cycles-pp.generic_file_read_iter 0.07 ± 6% -0.1 0.00 -0.0 0.06 +0.0 0.07 ± 6% perf-profile.self.cycles-pp.__mark_inode_dirty 0.15 ± 3% -0.1 0.08 ± 4% -0.0 0.12 ± 2% +0.0 0.15 ± 2% perf-profile.self.cycles-pp.current_time 0.15 ± 3% -0.1 0.08 ± 3% -0.0 0.12 ± 2% +0.0 0.16 ± 2% perf-profile.self.cycles-pp.ksys_write 0.15 ± 6% -0.1 0.07 ± 6% -0.0 0.12 ± 8% +0.0 0.15 ± 6% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.15 ± 2% -0.1 0.08 -0.0 0.12 ± 2% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.generic_write_checks 0.15 ± 4% -0.1 0.08 ± 6% +0.0 0.16 ± 5% +0.0 0.17 ± 3% perf-profile.self.cycles-pp.__memcg_slab_post_alloc_hook 0.07 -0.1 0.00 -0.0 0.07 ± 7% +0.0 0.08 perf-profile.self.cycles-pp.folio_alloc_buffers 0.15 ± 3% -0.1 0.08 ± 4% -0.0 0.12 ± 3% +0.0 0.15 ± 2% perf-profile.self.cycles-pp.copy_page_to_iter 0.15 ± 3% -0.1 0.08 ± 4% -0.0 0.12 +0.0 0.15 ± 3% perf-profile.self.cycles-pp.ksys_read 0.07 ± 6% -0.1 0.00 -0.0 0.05 -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__folio_cancel_dirty 0.07 ± 6% -0.1 0.00 -0.0 0.05 +0.0 0.07 ± 6% perf-profile.self.cycles-pp.ext4_generic_write_checks 0.07 ± 6% -0.1 0.00 -0.0 0.05 ± 8% +0.0 0.07 ± 4% perf-profile.self.cycles-pp.__mem_cgroup_charge 0.08 -0.1 0.01 ±163% -0.0 0.07 ± 6% +0.0 0.08 ± 4% perf-profile.self.cycles-pp.free_unref_folios 0.18 ± 2% -0.1 0.11 ± 3% -0.0 0.15 ± 3% +0.0 0.18 perf-profile.self.cycles-pp.mod_objcg_state 0.13 -0.1 0.06 ± 7% -0.0 0.10 +0.0 0.13 ± 3% perf-profile.self.cycles-pp.__radix_tree_lookup 0.13 ± 3% -0.1 0.06 ± 4% -0.0 0.10 -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__alloc_pages_noprof 0.06 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.06 perf-profile.self.cycles-pp.folio_activate 0.06 ± 7% -0.1 0.00 -0.0 0.05 +0.0 0.07 ± 6% perf-profile.self.cycles-pp.truncate_cleanup_folio 0.06 ± 7% -0.1 0.00 -0.0 0.05 +0.0 0.07 ± 4% perf-profile.self.cycles-pp._raw_spin_lock_irq 0.06 ± 7% -0.1 0.00 -0.0 0.05 ± 9% +0.0 0.07 ± 4% perf-profile.self.cycles-pp.xas_clear_mark 0.15 ± 2% -0.1 0.09 ± 5% -0.0 0.13 ± 2% +0.0 0.16 ± 2% perf-profile.self.cycles-pp.find_lock_entries 0.14 -0.1 0.08 ± 6% -0.0 0.12 ± 3% +0.0 0.14 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.06 ± 7% -0.1 0.00 -0.0 0.05 -0.0 0.06 perf-profile.self.cycles-pp.amd_clear_divider 0.13 ± 5% -0.1 0.07 ± 7% -0.0 0.13 ± 9% +0.0 0.15 ± 4% perf-profile.self.cycles-pp.kmem_cache_alloc_noprof 0.06 ± 6% -0.1 0.00 -0.0 0.05 +0.0 0.06 ± 6% perf-profile.self.cycles-pp.try_to_free_buffers 0.12 ± 2% -0.1 0.06 -0.0 0.09 ± 4% +0.0 0.13 ± 3% perf-profile.self.cycles-pp.node_dirty_ok 0.06 -0.1 0.00 -0.0 0.05 +0.0 0.06 perf-profile.self.cycles-pp.__mod_zone_page_state 0.06 -0.1 0.00 -0.0 0.05 +0.0 0.06 perf-profile.self.cycles-pp.delete_from_page_cache_batch 0.12 -0.1 0.06 ± 6% -0.0 0.10 ± 4% +0.0 0.12 perf-profile.self.cycles-pp.ksys_lseek 0.09 -0.1 0.03 ± 75% -0.0 0.07 +0.0 0.09 perf-profile.self.cycles-pp.ext4_es_lookup_extent 0.12 -0.1 0.06 ± 7% -0.0 0.10 ± 4% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.create_empty_buffers 0.12 ± 4% -0.1 0.06 -0.0 0.09 -0.0 0.12 ± 4% perf-profile.self.cycles-pp.__dquot_alloc_space 0.10 ± 3% -0.1 0.05 ± 31% -0.0 0.08 ± 6% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.folio_account_dirtied 0.12 ± 4% -0.1 0.06 ± 4% -0.0 0.09 ± 5% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head 0.09 -0.1 0.04 ± 61% -0.0 0.07 +0.0 0.09 perf-profile.self.cycles-pp.file_modified 0.11 ± 4% -0.1 0.05 ± 8% -0.0 0.08 ± 5% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.ext4_llseek 0.10 ± 4% -0.1 0.05 -0.0 0.08 ± 6% -0.0 0.10 ± 3% perf-profile.self.cycles-pp.__count_memcg_events 0.05 ± 8% -0.1 0.00 -0.0 0.05 ± 6% +0.0 0.07 ± 6% perf-profile.self.cycles-pp.obj_cgroup_charge 0.11 ± 4% -0.1 0.06 ± 8% -0.0 0.08 ± 5% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.generic_write_check_limits 0.10 ± 5% -0.1 0.05 -0.0 0.08 ± 4% +0.0 0.10 perf-profile.self.cycles-pp.file_remove_privs_flags 0.10 ± 5% -0.1 0.05 -0.0 0.08 +0.0 0.10 perf-profile.self.cycles-pp.block_write_end 0.10 -0.1 0.05 -0.0 0.08 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.get_page_from_freelist 0.12 ± 4% -0.0 0.07 ± 6% -0.0 0.10 ± 4% +0.0 0.12 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack 0.12 -0.0 0.07 ± 5% -0.0 0.10 +0.0 0.12 perf-profile.self.cycles-pp.drop_buffers 0.10 ± 5% -0.0 0.05 -0.0 0.08 ± 6% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.timestamp_truncate 0.10 ± 5% -0.0 0.05 ± 5% -0.0 0.08 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.folio_account_cleaned 0.10 ± 4% -0.0 0.06 ± 8% -0.0 0.08 +0.0 0.10 perf-profile.self.cycles-pp.kmem_cache_free 0.11 ± 9% -0.0 0.07 ± 8% +0.1 0.20 ± 3% -0.0 0.11 ± 10% perf-profile.self.cycles-pp.mem_cgroup_update_lru_size 0.10 ± 4% -0.0 0.07 -0.0 0.10 ± 4% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.__memcg_slab_free_hook 0.09 ± 5% -0.0 0.06 ± 4% -0.0 0.08 ± 4% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.try_charge_memcg 0.10 ± 4% -0.0 0.07 ± 5% -0.0 0.08 ± 4% +0.0 0.10 ± 4% perf-profile.self.cycles-pp.__page_cache_release 0.23 ± 3% -0.0 0.21 ± 2% -0.2 0.05 -0.2 0.06 perf-profile.self.cycles-pp.xas_find_conflict 0.23 ± 2% -0.0 0.22 ± 3% -0.0 0.18 ± 3% -0.0 0.23 perf-profile.self.cycles-pp.workingset_activation 0.25 ± 2% +0.0 0.28 ± 4% +0.0 0.30 +0.0 0.25 perf-profile.self.cycles-pp.folio_activate_fn 0.17 ± 3% +0.1 0.26 -0.0 0.13 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.lru_add_fn 0.51 ± 2% +0.1 0.63 ± 3% -0.1 0.40 ± 2% -0.0 0.50 perf-profile.self.cycles-pp.__lruvec_stat_mod_folio 0.30 ± 5% +0.2 0.51 ± 2% +0.0 0.34 ± 3% +0.0 0.30 ± 3% perf-profile.self.cycles-pp.folio_batch_move_lru 50.97 +25.3 76.24 +13.7 64.69 +0.4 51.36 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 7e2eb091049a..0e5bf25d324f 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { > > /* Fields which get updated often at the end. */ > struct lruvec lruvec; > + CACHELINE_PADDING(_pad2_); > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > struct mem_cgroup_reclaim_iter iter; > }; > > > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] : aim7.jobs-per-min -29.4% regression 2024-07-23 3:42 ` [linux-next:master] [mm] : " Oliver Sang @ 2024-07-23 15:44 ` Shakeel Butt 2024-07-24 22:15 ` Roman Gushchin 1 sibling, 0 replies; 12+ messages in thread From: Shakeel Butt @ 2024-07-23 15:44 UTC (permalink / raw) To: Oliver Sang Cc: Roman Gushchin, oe-lkp@lists.linux.dev, lkp, Linux Memory Management List, Andrew Morton, Johannes Weiner, Michal Hocko, Muchun Song, cgroups@vger.kernel.org, Huang, Ying, Tang, Feng, Yin, Fengwei On Tue, Jul 23, 2024 at 11:42:33AM GMT, Oliver Sang wrote: > hi, Roman, > > On Sat, Jul 20, 2024 at 01:14:16AM +0800, Roman Gushchin wrote: > > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > > hi, Roman, > > > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > > hi, Roman Gushchin, > > > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > > > Hello, > > > > > > > > > > > > thank you for the report! > > > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > > > Thank you! > > > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > > > Thank you for providing the detailed information! > > > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > > > no. in our tests, the following patch has little impact. > > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > > let me know, thanks) > > > > Hm, interesting. And thank you for the confirmation, you did everything correct. > > Because the only thing the original patch did was a removal of few fields from > > the mem_cgroup_per_node struct, there are not many options left here. > > Would you mind to try the following patch? > > > > Thank you and really appreciate your help! > > you are welcome! > > though we saw there are further discussions, we still share our test results to > you. > > in our tests, by your new version patch, the regression is entirely resoloved. > [...] This is very interesting as this shows (possibly) there was false sharing between lruvec and lru_zone_size. I will check if these two fields were accidentally on different cacheline before the series. > > > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 7e2eb091049a..0e5bf25d324f 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -109,6 +109,7 @@ struct mem_cgroup_per_node { > > > > /* Fields which get updated often at the end. */ > > struct lruvec lruvec; > > + CACHELINE_PADDING(_pad2_); > > unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > > struct mem_cgroup_reclaim_iter iter; > > }; > > > > > > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [linux-next:master] [mm] : aim7.jobs-per-min -29.4% regression 2024-07-23 3:42 ` [linux-next:master] [mm] : " Oliver Sang 2024-07-23 15:44 ` Shakeel Butt @ 2024-07-24 22:15 ` Roman Gushchin 1 sibling, 0 replies; 12+ messages in thread From: Roman Gushchin @ 2024-07-24 22:15 UTC (permalink / raw) To: Oliver Sang Cc: oe-lkp@lists.linux.dev, lkp, Linux Memory Management List, Andrew Morton, Shakeel Butt, Johannes Weiner, Michal Hocko, Muchun Song, cgroups@vger.kernel.org, Huang, Ying, Tang, Feng, Yin, Fengwei On Tue, Jul 23, 2024 at 11:42:33AM +0800, Oliver Sang wrote: > hi, Roman, > > On Sat, Jul 20, 2024 at 01:14:16AM +0800, Roman Gushchin wrote: > > On Tue, Jul 16, 2024 at 03:53:25PM +0800, Oliver Sang wrote: > > > hi, Roman, > > > > > > On Mon, Jul 15, 2024 at 10:18:39PM +0000, Roman Gushchin wrote: > > > > On Mon, Jul 15, 2024 at 10:14:31PM +0800, Oliver Sang wrote: > > > > > hi, Roman Gushchin, > > > > > > > > > > On Fri, Jul 12, 2024 at 07:03:31PM +0000, Roman Gushchin wrote: > > > > > > On Fri, Jul 12, 2024 at 02:04:48PM +0800, kernel test robot wrote: > > > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > kernel test robot noticed a -29.4% regression of aim7.jobs-per-min on: > > > > > > > > > > > > > > > > > > > > > commit: 98c9daf5ae6be008f78c07b744bcff7bcc6e98da ("mm: memcg: guard memcg1-specific members of struct mem_cgroup_per_node") > > > > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master > > > > > > > > > > > > Hello, > > > > > > > > > > > > thank you for the report! > > > > > > > > > > > > I'd expect that the regression should be fixed by the commit > > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node". > > > > > > > > > > > > Can you, please, confirm that it's not the case? > > > > > > > > > > > > Thank you! > > > > > > > > > > in our this aim7 test, we found the performance partially recovered by > > > > > "mm: memcg: add cache line padding to mem_cgroup_per_node" but not fully > > > > > > > > Thank you for providing the detailed information! > > > > > > > > Can you, please, check if the following patch resolves the regression entirely? > > > > > > no. in our tests, the following patch has little impact. > > > I directly apply it upon 6df13230b6 (if this is not the proper applyment, please > > > let me know, thanks) > > > > Hm, interesting. And thank you for the confirmation, you did everything correct. > > Because the only thing the original patch did was a removal of few fields from > > the mem_cgroup_per_node struct, there are not many options left here. > > Would you mind to try the following patch? > > > > Thank you and really appreciate your help! > > you are welcome! > > though we saw there are further discussions, we still share our test results to > you. > > in our tests, by your new version patch, the regression is entirely resoloved. Hi Oliver, perfect! A patch with a fix landed to the mm tree (you're cc'ed). Once again thank you for the report and for the collaboration on testing fixes! Roman ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2024-07-24 22:15 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-07-12 6:04 [linux-next:master] [mm] 98c9daf5ae: aim7.jobs-per-min -29.4% regression kernel test robot 2024-07-12 19:03 ` Roman Gushchin 2024-07-15 14:14 ` Oliver Sang 2024-07-15 22:18 ` Roman Gushchin 2024-07-16 7:53 ` Oliver Sang 2024-07-19 17:14 ` Roman Gushchin 2024-07-19 22:38 ` Shakeel Butt 2024-07-22 15:37 ` Roman Gushchin 2024-07-22 19:51 ` Shakeel Butt 2024-07-23 3:42 ` [linux-next:master] [mm] : " Oliver Sang 2024-07-23 15:44 ` Shakeel Butt 2024-07-24 22:15 ` Roman Gushchin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).