From: edwin <edwintorok@gmail.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@elte.hu>,
rml@tech9.net, Linux Kernel <linux-kernel@vger.kernel.org>,
"Thomas Gleixner mingo@redhat.com" <tglx@linutronix.de>,
"H. Peter Anvin" <hpa@zytor.com>
Subject: Re: Quad core CPUs loaded at only 50% when running a CPU and mmap intensive multi-threaded task
Date: Mon, 25 Aug 2008 14:00:55 +0300 [thread overview]
Message-ID: <48B290E7.4070805@gmail.com> (raw)
In-Reply-To: <1219660582.8515.24.camel@twins>
[-- Attachment #1: Type: text/plain, Size: 1858 bytes --]
Peter Zijlstra wrote:
> On Mon, 2008-08-25 at 13:22 +0300, Török Edwin wrote:
>
>
>> Well, the real program (clamd) that this testprogram tries to simulate
>> does an mmap for almost every file, and I have lots of small files.
>> 6.5G, 114122 files, average size 57k.
>>
>> I'll run latencytop again, last time it has showed 100ms - 500ms latency
>>
Latencytop output attached.
There is 4 - 60 ms latency for mmap/munmap, and the more threads there
are the total latency gets higher (latencytop says sum was ~480ms).
Running with MaxThreads 4 gets me 300-400% CPU usage, but with
MaxThreads 8 CPU usage drops to around 120-250%.
Now, maxthreads 4 looks like a good choice from a CPU usage point of
view, but is actually bad because it means that threads gets stuck in
iowait, and the CPU won't have anything to do. MaxThreads 8 looked like
a good alternative to fill the iowait gaps, but we run into the mmap_sem
issue.
In a real world environment MaxThreads influences how many mails you can
process in parallel with your MTA, so generally it should be as high as
possible.
On 2.6.27-rc4:
MaxThreads 4 time, empty database (all cached, almost no I/O):
1m9s
MaxThreads 4 time, after echo 3>/proc/sys/vm/drop_caches:
1m29s
MaxThreads 8 time, empty database (all cached, almost no I/O):
2m16s
MaxThreads 8 time, after echo 3>/proc/sys/vm/drop_caches:
2m15s
Of course running with a full database will give different results, so
I'll do some timing with that too (will take a little longer though).
>> for clamd, and it was about mmap, I'll provide you with the exact output.
>>
>
> Right - does it make sense to teach clamav about pread() ?
If it is preferred over mmap, then maybe yes.
Peter Zijlstra wrote:
> OK, I'll poke a little more at is later today to see if I can spot
> something
Thanks!
Best regards,
--Edwin
[-- Attachment #2: latgather-1219660865 --]
[-- Type: text/plain, Size: 27285 bytes --]
---------
Latency Top version : v0.1
clamd (5571, #threads: 10)
---------------------------------------------------------
se.exec_start : 700869.502286
se.vruntime : 262417.157677
se.sum_exec_runtime : 0.000000
se.avg_overlap : 0.000000
se.wait_start : 0.000000
se.sleep_start : 700869.502286
se.block_start : 0.000000
se.sleep_max : 0.000000
se.block_max : 0.000000
se.exec_max : 0.000000
se.slice_max : 0.000000
se.wait_max : 0.000000
se.wait_sum : 0.000000
se.wait_count : 0
sched_info.bkl_count : 0
se.nr_migrations : 0
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 0
se.nr_failed_migrations_hot : 0
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 0
se.nr_wakeups : 0
se.nr_wakeups_sync : 0
se.nr_wakeups_migrate : 0
se.nr_wakeups_local : 0
se.nr_wakeups_remote : 0
se.nr_wakeups_affine : 0
se.nr_wakeups_affine_attempts : 0
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.000001
avg_per_cpu : 0.000001
nr_switches : 0
nr_voluntary_switches : 0
nr_involuntary_switches : 0
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 87
__skb_recv_datagram---------
---------
Latency Top version : v0.1
16 15062 4955 futex_wait do_futex sys_futex system_call_fastpath
clamd (6072, #threads: 10)
---------------------------------------------------------
se.exec_start : 890682.642007
se.vruntime : 329161.740000
se.sum_exec_runtime : 480.929023
se.avg_overlap : 0.929824
se.wait_start : 0.000000
se.sleep_start : 890682.642007
se.block_start : 0.000000
se.sleep_max : 14582.725673
se.block_max : 9.018512
se.exec_max : 6.499281
se.slice_max : 27.999520
se.wait_max : 39.792243
se.wait_sum : 167.376729
se.wait_count : 340
sched_info.bkl_count : 0
se.nr_migrations : 11
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 255
se.nr_failed_migrations_hot : 7
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 4
se.nr_wakeups : 306
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 7
se.nr_wakeups_local : 96
se.nr_wakeups_remote : 210
se.nr_wakeups_affine : 7
se.nr_wakeups_affine_attempts : 300
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.466247
avg_per_cpu : 43.720820
nr_switches : 328
nr_voluntary_switches : 302
nr_involuntary_switches : 26
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 84
futex_wait---------
---------
Latency Top version : v0.1
11 10912 3779 sys_munmap system_call_fastpath
8 46486 19495 sys_mmap system_call_fastpath
6 13265 5138 do_page_fault error_exit
4 4142 2263 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
2 0 0 sys_mmap system_call_fastpath
clamd (6073, #threads: 10)
---------------------------------------------------------
se.exec_start : 891089.502763
se.vruntime : 297039.691398
se.sum_exec_runtime : 38531.939735
se.avg_overlap : 0.257175
se.wait_start : 891089.502774
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 56.856987
se.block_max : 39.998814
se.exec_max : 7.361909
se.slice_max : 32.002390
se.wait_max : 64.478851
se.wait_sum : 28485.466170
se.wait_count : 134840
sched_info.bkl_count : 0
se.nr_migrations : 807
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 6305
se.nr_failed_migrations_hot : 19174
se.nr_forced_migrations : 86
se.nr_forced2_migrations : 304
se.nr_wakeups : 118434
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 357
se.nr_wakeups_local : 26408
se.nr_wakeups_remote : 92026
se.nr_wakeups_affine : 357
se.nr_wakeups_affine_attempts : 118134
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.303195
avg_per_cpu : 47.747137
nr_switches : 127086
nr_voluntary_switches : 118139
nr_involuntary_switches : 8947
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 78
do_page_fault---------
---------
Latency Top version : v0.1
8 33623 10075 sys_mmap system_call_fastpath
4 126 126 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
10 8978 6660 sys_munmap system_call_fastpath
6 6322 5011 do_page_fault error_exit
3 1879 1879 sys_mmap system_call_fastpath
clamd (6074, #threads: 10)
---------------------------------------------------------
se.exec_start : 891094.957709
se.vruntime : 282081.330593
se.sum_exec_runtime : 37529.609679
se.avg_overlap : 0.128584
se.wait_start : 891094.957729
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 56.001313
se.block_max : 52.999336
se.exec_max : 8.328602
se.slice_max : 61.695609
se.wait_max : 78.999644
se.wait_sum : 27798.394839
se.wait_count : 133319
sched_info.bkl_count : 0
se.nr_migrations : 843
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5888
se.nr_failed_migrations_hot : 20124
se.nr_forced_migrations : 83
se.nr_forced2_migrations : 307
se.nr_wakeups : 119224
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 362
se.nr_wakeups_local : 27409
se.nr_wakeups_remote : 91815
se.nr_wakeups_affine : 362
se.nr_wakeups_affine_attempts : 118907
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.296110
avg_per_cpu : 44.519109
nr_switches : 126742
nr_voluntary_switches : 118908
nr_involuntary_switches : 7834
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 78
0---------
---------
Latency Top version : v0.1
11 11662 6229 sys_munmap system_call_fastpath
10 48650 20169 sys_mmap system_call_fastpath
10 24384 7811 do_page_fault error_exit
clamd (6075, #threads: 10)
---------------------------------------------------------
se.exec_start : 891094.957715
se.vruntime : 325149.711380
se.sum_exec_runtime : 38581.104338
se.avg_overlap : 0.112403
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 891094.957715
se.sleep_max : 30.979799
se.block_max : 56.856984
se.exec_max : 7.517372
se.slice_max : 31.999387
se.wait_max : 60.001367
se.wait_sum : 28278.306752
se.wait_count : 133652
sched_info.bkl_count : 0
se.nr_migrations : 802
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5425
se.nr_failed_migrations_hot : 19955
se.nr_forced_migrations : 77
se.nr_forced2_migrations : 277
se.nr_wakeups : 118420
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 322
se.nr_wakeups_local : 27118
se.nr_wakeups_remote : 91302
se.nr_wakeups_affine : 322
se.nr_wakeups_affine_attempts : 118161
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.304837
avg_per_cpu : 48.106115
nr_switches : 126563
nr_voluntary_switches : 118164
nr_involuntary_switches : 8399
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 78
do_page_fault---------
---------
Latency Top version : v0.1
10 15502 5352 sys_munmap system_call_fastpath
9 38554 20265 sys_mmap system_call_fastpath
7 9083 5138 do_page_fault error_exit
1 5011 5011 sys_mmap system_call_fastpath
4 3762 2641 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6076, #threads: 10)
---------------------------------------------------------
se.exec_start : 891094.957721
se.vruntime : 329720.631055
se.sum_exec_runtime : 42341.424828
se.avg_overlap : 1.514241
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 891094.957721
se.sleep_max : 30.907363
se.block_max : 56.500704
se.exec_max : 7.507303
se.slice_max : 32.001234
se.wait_max : 59.986115
se.wait_sum : 29257.415262
se.wait_count : 133434
sched_info.bkl_count : 0
se.nr_migrations : 814
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5043
se.nr_failed_migrations_hot : 20066
se.nr_forced_migrations : 94
se.nr_forced2_migrations : 231
se.nr_wakeups : 111450
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 284
se.nr_wakeups_local : 26837
se.nr_wakeups_remote : 84613
se.nr_wakeups_affine : 284
se.nr_wakeups_affine_attempts : 111188
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.344127
avg_per_cpu : 52.016492
nr_switches : 123040
nr_voluntary_switches : 111189
nr_involuntary_switches : 11851
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 83
do_page_fault---------
---------
Latency Top version : v0.1
11 34222 20131 sys_munmap system_call_fastpath
10 20073 7812 sys_mmap system_call_fastpath
2 3344 3344 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
7 9696 2999 do_page_fault error_exit
1 487 487 sys_mmap system_call_fastpath
clamd (6077, #threads: 10)
---------------------------------------------------------
se.exec_start : 891098.952122
se.vruntime : 282078.697244
se.sum_exec_runtime : 40209.531524
se.avg_overlap : 0.116129
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 28.499495
se.block_max : 51.003639
se.exec_max : 8.000313
se.slice_max : 31.999072
se.wait_max : 70.679248
se.wait_sum : 28364.256448
se.wait_count : 135183
sched_info.bkl_count : 0
se.nr_migrations : 776
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5111
se.nr_failed_migrations_hot : 19975
se.nr_forced_migrations : 80
se.nr_forced2_migrations : 235
se.nr_wakeups : 114583
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 288
se.nr_wakeups_local : 26813
se.nr_wakeups_remote : 87770
se.nr_wakeups_affine : 288
se.nr_wakeups_affine_attempts : 114354
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.320261
avg_per_cpu : 51.816406
nr_switches : 125552
nr_voluntary_switches : 114359
nr_involuntary_switches : 11193
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 81
0---------
---------
Latency Top version : v0.1
10 8515 3778 sys_munmap system_call_fastpath
8 12692 3418 sys_mmap system_call_fastpath
2 0 0 do_page_fault error_exit shmem_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
7 9412 5506 do_page_fault error_exit
1 3239 3239 sys_mmap system_call_fastpath
1 1587 1587 sys_madvise system_call_fastpath
2 0 0 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6078, #threads: 10)
---------------------------------------------------------
se.exec_start : 891094.957730
se.vruntime : 325213.210983
se.sum_exec_runtime : 38004.560148
se.avg_overlap : 2.244092
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 891094.957730
se.sleep_max : 40.291386
se.block_max : 59.495329
se.exec_max : 7.937506
se.slice_max : 39.969289
se.wait_max : 67.500878
se.wait_sum : 28689.272253
se.wait_count : 133749
sched_info.bkl_count : 0
se.nr_migrations : 891
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 4931
se.nr_failed_migrations_hot : 20678
se.nr_forced_migrations : 81
se.nr_forced2_migrations : 323
se.nr_wakeups : 117985
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 370
se.nr_wakeups_local : 28551
se.nr_wakeups_remote : 89434
se.nr_wakeups_affine : 370
se.nr_wakeups_affine_attempts : 117741
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.300516
avg_per_cpu : 42.653827
nr_switches : 126464
nr_voluntary_switches : 117738
nr_involuntary_switches : 8726
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 78
do_page_fault---------
---------
Latency Top version : v0.1
9 14443 5011 sys_mmap system_call_fastpath
4 8480 5136 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
10 17139 5993 sys_munmap system_call_fastpath
7 6619 2352 do_page_fault error_exit
1 0 0 sys_mmap system_call_fastpath
clamd (6079, #threads: 10)
---------------------------------------------------------
se.exec_start : 891094.957727
se.vruntime : 325200.342156
se.sum_exec_runtime : 39871.746222
se.avg_overlap : 0.700269
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 891094.957727
se.sleep_max : 29.363157
se.block_max : 58.999622
se.exec_max : 8.000502
se.slice_max : 49.687478
se.wait_max : 50.879058
se.wait_sum : 28190.321613
se.wait_count : 135340
sched_info.bkl_count : 0
se.nr_migrations : 791
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 6292
se.nr_failed_migrations_hot : 19989
se.nr_forced_migrations : 74
se.nr_forced2_migrations : 278
se.nr_wakeups : 115205
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 343
se.nr_wakeups_local : 27413
se.nr_wakeups_remote : 87792
se.nr_wakeups_affine : 343
se.nr_wakeups_affine_attempts : 114949
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.316658
avg_per_cpu : 50.406758
nr_switches : 125914
nr_voluntary_switches : 114946
nr_involuntary_switches : 10968
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 90
sys_munmap---------
---------
Latency Top version : v0.1
10 4854 1643 sys_munmap system_call_fastpath
10 31224 20031 sys_mmap system_call_fastpath
3 8985 5011 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
1 0 0 futex_wait do_futex sys_futex system_call_fastpath
5 2736 1837 do_page_fault error_exit
2 3906 3906 do_page_fault error_exit shmem_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6080, #threads: 10)
---------------------------------------------------------
se.exec_start : 891094.957724
se.vruntime : 297044.689377
se.sum_exec_runtime : 39379.909583
se.avg_overlap : 0.264610
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 891094.957724
se.sleep_max : 31.055799
se.block_max : 58.999625
se.exec_max : 7.496151
se.slice_max : 72.000452
se.wait_max : 59.951360
se.wait_sum : 28778.446818
se.wait_count : 134612
sched_info.bkl_count : 0
se.nr_migrations : 817
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 4766
se.nr_failed_migrations_hot : 20436
se.nr_forced_migrations : 88
se.nr_forced2_migrations : 267
se.nr_wakeups : 116053
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 320
se.nr_wakeups_local : 27626
se.nr_wakeups_remote : 88427
se.nr_wakeups_affine : 320
se.nr_wakeups_affine_attempts : 115788
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.312553
avg_per_cpu : 48.200623
nr_switches : 125994
nr_voluntary_switches : 115786
nr_involuntary_switches : 10208
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 87
sys_mmap---------
Mon Aug 25 13:41:05 EEST 2008
[-- Attachment #3: latgather-1219660915 --]
[-- Type: text/plain, Size: 52120 bytes --]
[max 66.4msec] unmapping memory
- 4.01 msec (18.8%)
[max 58.6msec] Page fault
- 3.30 msec (59.0%)
[max 58.6msec] mmaping memory
- 4.35 msec (20.4%)
[max 27.3msec] sys_madvise system_call_fastpath - 5.40 msec (1.2%)
[max 25.3msec] Creating block layer request
- 25.27 msec (0.1%)
[max 8.8msec] mprotect() system call
- 8.79 msec (0.0%)
[max 4.5msec] Userspace lock contention
- 0.67 msec (0.3%)
[max 3.8msec] Submitting block IO
- 1.06 msec (0.0%)
[max 3.5msec] opening cdrom device
- 0.45 msec (0.0%)
[max 2.9msec] fsync() on a file
- 1.47 msec (0.0%)
[max 0.5msec] Removing directory
- 0.27 msec (0.0%)
[max 0.3msec] Closing a file
- 0.05 msec (0.0%)
[max 0.0msec] sys_brk system_call_fastpath - 0.00 msec (0.0%)
---------
Latency Top version : v0.1
clamd (5571, #threads: 19)
---------------------------------------------------------
se.exec_start : 931741.503387
se.vruntime : 366135.984717
se.sum_exec_runtime : 0.000000
se.avg_overlap : 0.000000
se.wait_start : 0.000000
se.sleep_start : 931741.503387
se.block_start : 0.000000
se.sleep_max : 0.000000
se.block_max : 0.000000
se.exec_max : 0.000000
se.slice_max : 0.000000
se.wait_max : 0.000000
se.wait_sum : 0.000000
se.wait_count : 0
sched_info.bkl_count : 0
se.nr_migrations : 0
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 0
se.nr_failed_migrations_hot : 0
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 0
se.nr_wakeups : 0
se.nr_wakeups_sync : 0
se.nr_wakeups_migrate : 0
se.nr_wakeups_local : 0
se.nr_wakeups_remote : 0
se.nr_wakeups_affine : 0
se.nr_wakeups_affine_attempts : 0
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.000001
avg_per_cpu : 0.000001
nr_switches : 0
nr_voluntary_switches : 0
nr_involuntary_switches : 0
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
__skb_recv_datagram---------
---------
Latency Top version : v0.1
clamd (6072, #threads: 19)
---------------------------------------------------------
se.exec_start : 921337.501698
se.vruntime : 357474.768314
se.sum_exec_runtime : 533.510071
se.avg_overlap : 5.630120
se.wait_start : 0.000000
se.sleep_start : 921337.501698
se.block_start : 0.000000
se.sleep_max : 16378.448918
se.block_max : 9.018512
se.exec_max : 6.499281
se.slice_max : 27.999520
se.wait_max : 39.792243
se.wait_sum : 190.393565
se.wait_count : 352
sched_info.bkl_count : 0
se.nr_migrations : 11
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 257
se.nr_failed_migrations_hot : 7
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 4
se.nr_wakeups : 317
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 7
se.nr_wakeups_local : 99
se.nr_wakeups_remote : 218
se.nr_wakeups_affine : 7
se.nr_wakeups_affine_attempts : 311
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.573776
avg_per_cpu : 48.500915
nr_switches : 339
nr_voluntary_switches : 313
nr_involuntary_switches : 26
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
futex_wait---------
---------
Latency Top version : v0.1
5 1944 907 futex_wait do_futex sys_futex system_call_fastpath
22 149759 57819 do_page_fault error_exit
1 0 0 sys_madvise system_call_fastpath
2 1928 964 sys_mmap system_call_fastpath
1 826 826 sys_munmap system_call_fastpath
clamd (6073, #threads: 19)
---------------------------------------------------------
se.exec_start : 941737.001727
se.vruntime : 369293.578220
se.sum_exec_runtime : 60400.545136
se.avg_overlap : 6.090918
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 157.455998
se.block_max : 59.207159
se.exec_max : 8.000502
se.slice_max : 103.917546
se.wait_max : 108.000687
se.wait_sum : 49523.405131
se.wait_count : 141028
sched_info.bkl_count : 0
se.nr_migrations : 867
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 6593
se.nr_failed_migrations_hot : 19659
se.nr_forced_migrations : 92
se.nr_forced2_migrations : 318
se.nr_wakeups : 122605
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 377
se.nr_wakeups_local : 27546
se.nr_wakeups_remote : 95059
se.nr_wakeups_affine : 377
se.nr_wakeups_affine_attempts : 122302
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.455221
avg_per_cpu : 69.666142
nr_switches : 132684
nr_voluntary_switches : 122307
nr_involuntary_switches : 10377
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 57
0---------
---------
Latency Top version : v0.1
4 4898 3906 futex_wait do_futex sys_futex system_call_fastpath
2 12205 12205 sys_mmap system_call_fastpath
19 52728 31249 do_page_fault error_exit
3 14773 7325 sys_munmap system_call_fastpath
2 4756 4391 sys_mmap system_call_fastpath
1 3904 3904 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6074, #threads: 19)
---------------------------------------------------------
se.exec_start : 941737.501664
se.vruntime : 328090.268953
se.sum_exec_runtime : 57843.294472
se.avg_overlap : 2.129179
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941737.501664
se.sleep_max : 139.916296
se.block_max : 52.999336
se.exec_max : 8.328602
se.slice_max : 61.695609
se.wait_max : 81.027965
se.wait_sum : 48893.020266
se.wait_count : 140200
sched_info.bkl_count : 0
se.nr_migrations : 897
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 6295
se.nr_failed_migrations_hot : 20615
se.nr_forced_migrations : 85
se.nr_forced2_migrations : 316
se.nr_wakeups : 123699
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 379
se.nr_wakeups_local : 28785
se.nr_wakeups_remote : 94914
se.nr_wakeups_affine : 379
se.nr_wakeups_affine_attempts : 123373
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.435478
avg_per_cpu : 64.485278
nr_switches : 132827
nr_voluntary_switches : 123376
nr_involuntary_switches : 9451
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
do_page_fault---------
---------
Latency Top version : v0.1
3 4896 3921 futex_wait do_futex sys_futex system_call_fastpath
3 7814 7814 sys_mmap system_call_fastpath
17 125178 58593 do_page_fault error_exit
5 11717 3907 sys_munmap system_call_fastpath
2 20017 19529 sys_mmap system_call_fastpath
1 0 0 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6075, #threads: 19)
---------------------------------------------------------
se.exec_start : 941737.001727
se.vruntime : 369339.322708
se.sum_exec_runtime : 59325.744093
se.avg_overlap : 0.176192
se.wait_start : 941737.001727
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 297.024248
se.block_max : 59.999974
se.exec_max : 8.500719
se.slice_max : 140.654297
se.wait_max : 107.301024
se.wait_sum : 49163.396194
se.wait_count : 140275
sched_info.bkl_count : 0
se.nr_migrations : 872
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5612
se.nr_failed_migrations_hot : 20612
se.nr_forced_migrations : 86
se.nr_forced2_migrations : 293
se.nr_wakeups : 122887
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 347
se.nr_wakeups_local : 28386
se.nr_wakeups_remote : 94501
se.nr_wakeups_affine : 347
se.nr_wakeups_affine_attempts : 122619
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.447768
avg_per_cpu : 68.034110
nr_switches : 132492
nr_voluntary_switches : 122622
nr_involuntary_switches : 9870
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
0---------
---------
Latency Top version : v0.1
1 0 0 futex_wait do_futex sys_futex system_call_fastpath
2 11718 8301 sys_mmap system_call_fastpath
28 86522 19040 do_page_fault error_exit
clamd (6076, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.501293
se.vruntime : 328405.149685
se.sum_exec_runtime : 63894.247770
se.avg_overlap : 1.548911
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941761.501293
se.sleep_max : 143.499199
se.block_max : 56.500704
se.exec_max : 8.000810
se.slice_max : 63.943081
se.wait_max : 124.000596
se.wait_sum : 49831.782520
se.wait_count : 140785
sched_info.bkl_count : 0
se.nr_migrations : 893
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5161
se.nr_failed_migrations_hot : 20439
se.nr_forced_migrations : 105
se.nr_forced2_migrations : 250
se.nr_wakeups : 114063
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 309
se.nr_wakeups_local : 27663
se.nr_wakeups_remote : 86400
se.nr_wakeups_affine : 309
se.nr_wakeups_affine_attempts : 113795
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.497537
avg_per_cpu : 71.550109
nr_switches : 128421
nr_voluntary_switches : 113795
nr_involuntary_switches : 14626
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 45
do_page_fault---------
---------
Latency Top version : v0.1
9 35633 19040 do_page_fault error_exit
5 51511 19511 sys_madvise system_call_fastpath
2 3620 3620 futex_wait do_futex sys_futex system_call_fastpath
4 12211 7814 sys_mmap system_call_fastpath
7 16104 3907 sys_munmap system_call_fastpath
3 23923 20505 sys_mmap system_call_fastpath
1 31249 31249 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6077, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502248
se.vruntime : 342292.045052
se.sum_exec_runtime : 62027.755009
se.avg_overlap : 16.009423
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941761.502248
se.sleep_max : 282.119134
se.block_max : 51.003639
se.exec_max : 8.000708
se.slice_max : 47.973916
se.wait_max : 115.500667
se.wait_sum : 49175.485984
se.wait_count : 142922
sched_info.bkl_count : 0
se.nr_migrations : 847
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5514
se.nr_failed_migrations_hot : 20415
se.nr_forced_migrations : 82
se.nr_forced2_migrations : 256
se.nr_wakeups : 119354
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 315
se.nr_wakeups_local : 28128
se.nr_wakeups_remote : 91226
se.nr_wakeups_affine : 315
se.nr_wakeups_affine_attempts : 119120
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.469263
avg_per_cpu : 73.232296
nr_switches : 132181
nr_voluntary_switches : 119127
nr_involuntary_switches : 13054
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
do_page_fault---------
---------
Latency Top version : v0.1
18 14999 12206 do_page_fault error_exit
12 23439 19044 sys_madvise system_call_fastpath
1 0 0 futex_wait do_futex sys_futex system_call_fastpath
clamd (6078, #threads: 19)
---------------------------------------------------------
se.exec_start : 941764.504558
se.vruntime : 342304.056898
se.sum_exec_runtime : 59142.145521
se.avg_overlap : 4.029801
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 130.026373
se.block_max : 59.495329
se.exec_max : 7.937506
se.slice_max : 47.987529
se.wait_max : 88.907165
se.wait_sum : 49416.291505
se.wait_count : 140949
sched_info.bkl_count : 0
se.nr_migrations : 960
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 5362
se.nr_failed_migrations_hot : 21172
se.nr_forced_migrations : 84
se.nr_forced2_migrations : 342
se.nr_wakeups : 122180
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 404
se.nr_wakeups_local : 29684
se.nr_wakeups_remote : 92496
se.nr_wakeups_affine : 404
se.nr_wakeups_affine_attempts : 121930
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.446133
avg_per_cpu : 61.606401
nr_switches : 132566
nr_voluntary_switches : 121927
nr_involuntary_switches : 10639
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 63
0---------
---------
Latency Top version : v0.1
1 2035 2035 do_page_fault error_exit
30 19092 3907 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
clamd (6079, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502217
se.vruntime : 342472.045047
se.sum_exec_runtime : 61547.035391
se.avg_overlap : 15.077962
se.wait_start : 941761.502217
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 168.500075
se.block_max : 58.999622
se.exec_max : 8.000502
se.slice_max : 49.687478
se.wait_max : 92.000401
se.wait_sum : 50062.444615
se.wait_count : 141533
sched_info.bkl_count : 0
se.nr_migrations : 847
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 6707
se.nr_failed_migrations_hot : 20548
se.nr_forced_migrations : 80
se.nr_forced2_migrations : 292
se.nr_wakeups : 119240
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 358
se.nr_wakeups_local : 28569
se.nr_wakeups_remote : 90671
se.nr_wakeups_affine : 358
se.nr_wakeups_affine_attempts : 118978
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.468291
avg_per_cpu : 72.664740
nr_switches : 131429
nr_voluntary_switches : 118976
nr_involuntary_switches : 12453
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
0---------
---------
Latency Top version : v0.1
2 3906 3906 futex_wait do_futex sys_futex system_call_fastpath
5 66701 27634 sys_mmap system_call_fastpath
1 0 0 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
5 38081 19040 sys_munmap system_call_fastpath
15 114649 58593 do_page_fault error_exit
2 8243 8243 sys_mmap system_call_fastpath
1 0 0 sys_brk system_call_fastpath
clamd (6080, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502244
se.vruntime : 372898.839430
se.sum_exec_runtime : 60636.590526
se.avg_overlap : 0.833244
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941761.502244
se.sleep_max : 176.489855
se.block_max : 59.999483
se.exec_max : 8.000795
se.slice_max : 95.765041
se.wait_max : 110.998843
se.wait_sum : 50024.195476
se.wait_count : 141436
sched_info.bkl_count : 0
se.nr_migrations : 864
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 4924
se.nr_failed_migrations_hot : 20880
se.nr_forced_migrations : 92
se.nr_forced2_migrations : 276
se.nr_wakeups : 120249
se.nr_wakeups_sync : 2
se.nr_wakeups_migrate : 335
se.nr_wakeups_local : 28648
se.nr_wakeups_remote : 91601
se.nr_wakeups_affine : 335
se.nr_wakeups_affine_attempts : 119981
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.459723
avg_per_cpu : 70.181239
nr_switches : 131898
nr_voluntary_switches : 119979
nr_involuntary_switches : 11919
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
do_page_fault---------
---------
Latency Top version : v0.1
clamd (6488, #threads: 19)
---------------------------------------------------------
se.exec_start : 932061.002356
se.vruntime : 361680.992179
se.sum_exec_runtime : 20.193691
se.avg_overlap : 1.778925
se.wait_start : 0.000000
se.sleep_start : 932061.002356
se.block_start : 0.000000
se.sleep_max : 47.981172
se.block_max : 0.453532
se.exec_max : 3.996571
se.slice_max : 14.231404
se.wait_max : 65.459499
se.wait_sum : 242.627365
se.wait_count : 21
sched_info.bkl_count : 0
se.nr_migrations : 1
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 0
se.nr_failed_migrations_hot : 0
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 0
se.nr_wakeups : 10
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 1
se.nr_wakeups_local : 4
se.nr_wakeups_remote : 6
se.nr_wakeups_affine : 1
se.nr_wakeups_affine_attempts : 10
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.009684
avg_per_cpu : 20.193691
nr_switches : 20
nr_voluntary_switches : 10
nr_involuntary_switches : 10
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
futex_wait---------
---------
Latency Top version : v0.1
12 6590 6348 sys_munmap system_call_fastpath
11 21773 15519 sys_mmap system_call_fastpath
5 47929 27559 do_page_fault error_exit
2 0 0 futex_wait do_futex sys_futex system_call_fastpath
1 0 0 sys_mmap system_call_fastpath
clamd (6489, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502244
se.vruntime : 328631.656634
se.sum_exec_runtime : 1134.871842
se.avg_overlap : 0.000552
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941761.502244
se.sleep_max : 46.675267
se.block_max : 59.999969
se.exec_max : 4.991282
se.slice_max : 24.002234
se.wait_max : 56.000427
se.wait_sum : 2612.148267
se.wait_count : 2598
sched_info.bkl_count : 0
se.nr_migrations : 69
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 66
se.nr_failed_migrations_hot : 78
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 37
se.nr_wakeups : 2355
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 56
se.nr_wakeups_local : 1135
se.nr_wakeups_remote : 1220
se.nr_wakeups_affine : 56
se.nr_wakeups_affine_attempts : 2353
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.449632
avg_per_cpu : 16.447418
nr_switches : 2524
nr_voluntary_switches : 2352
nr_involuntary_switches : 172
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 63
sys_munmap---------
---------
Latency Top version : v0.1
31 150289 25449 do_page_fault error_exit
clamd (6490, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502243
se.vruntime : 328571.656638
se.sum_exec_runtime : 2104.750095
se.avg_overlap : 0.004952
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941761.502243
se.sleep_max : 49.174184
se.block_max : 51.989146
se.exec_max : 6.745074
se.slice_max : 30.408170
se.wait_max : 76.001241
se.wait_sum : 3630.921868
se.wait_count : 2117
sched_info.bkl_count : 0
se.nr_migrations : 53
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 77
se.nr_failed_migrations_hot : 89
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 18
se.nr_wakeups : 1466
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 26
se.nr_wakeups_local : 700
se.nr_wakeups_remote : 766
se.nr_wakeups_affine : 26
se.nr_wakeups_affine_attempts : 1465
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.134024
avg_per_cpu : 39.712265
nr_switches : 1856
nr_voluntary_switches : 1463
nr_involuntary_switches : 393
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
sys_munmap---------
---------
Latency Top version : v0.1
10 27049 15616 sys_munmap system_call_fastpath
8 88470 27343 sys_madvise system_call_fastpath
5 15022 8788 sys_mmap system_call_fastpath
2 573 573 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
4 11834 11834 do_page_fault error_exit
1 4772 4772 sys_mmap system_call_fastpath
1 0 0 futex_wait do_futex sys_futex system_call_fastpath
clamd (6491, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502246
se.vruntime : 372937.561308
se.sum_exec_runtime : 1956.099660
se.avg_overlap : 0.288143
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941761.502246
se.sleep_max : 65.459500
se.block_max : 49.488078
se.exec_max : 4.371700
se.slice_max : 31.957703
se.wait_max : 73.001000
se.wait_sum : 3596.043882
se.wait_count : 2151
sched_info.bkl_count : 0
se.nr_migrations : 54
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 38
se.nr_failed_migrations_hot : 74
se.nr_forced_migrations : 8
se.nr_forced2_migrations : 17
se.nr_wakeups : 1445
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 30
se.nr_wakeups_local : 583
se.nr_wakeups_remote : 862
se.nr_wakeups_affine : 30
se.nr_wakeups_affine_attempts : 1443
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.046602
avg_per_cpu : 36.224067
nr_switches : 1869
nr_voluntary_switches : 1441
nr_involuntary_switches : 428
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
sys_munmap---------
---------
Latency Top version : v0.1
6 4899 3921 futex_wait do_futex sys_futex system_call_fastpath
9 43003 9351 sys_mmap system_call_fastpath
4 5065 3536 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
8 28817 9175 sys_munmap system_call_fastpath
4 0 0 do_page_fault error_exit
clamd (6492, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502203
se.vruntime : 342335.603991
se.sum_exec_runtime : 2616.754918
se.avg_overlap : 0.893570
se.wait_start : 941761.502233
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 37.463866
se.block_max : 67.997947
se.exec_max : 4.501504
se.slice_max : 29.002212
se.wait_max : 107.502983
se.wait_sum : 4099.174164
se.wait_count : 2148
sched_info.bkl_count : 0
se.nr_migrations : 39
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 30
se.nr_failed_migrations_hot : 84
se.nr_forced_migrations : 1
se.nr_forced2_migrations : 13
se.nr_wakeups : 1234
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 21
se.nr_wakeups_local : 584
se.nr_wakeups_remote : 650
se.nr_wakeups_affine : 21
se.nr_wakeups_affine_attempts : 1230
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.503882
avg_per_cpu : 67.096279
nr_switches : 1740
nr_voluntary_switches : 1228
nr_involuntary_switches : 512
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
0---------
---------
Latency Top version : v0.1
1 15881 15881 sys_munmap system_call_fastpath
5 0 0 futex_wait do_futex sys_futex system_call_fastpath
25 34887 19533 do_page_fault error_exit
clamd (6493, #threads: 19)
---------------------------------------------------------
se.exec_start : 941761.502196
se.vruntime : 373078.839406
se.sum_exec_runtime : 2096.383223
se.avg_overlap : 0.076622
se.wait_start : 941761.502196
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 43.995613
se.block_max : 39.551665
se.exec_max : 4.978118
se.slice_max : 31.999444
se.wait_max : 96.010967
se.wait_sum : 3744.592128
se.wait_count : 2131
sched_info.bkl_count : 0
se.nr_migrations : 49
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 106
se.nr_failed_migrations_hot : 94
se.nr_forced_migrations : 1
se.nr_forced2_migrations : 16
se.nr_wakeups : 1511
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 22
se.nr_wakeups_local : 728
se.nr_wakeups_remote : 783
se.nr_wakeups_affine : 22
se.nr_wakeups_affine_attempts : 1511
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.122260
avg_per_cpu : 42.783331
nr_switches : 1868
nr_voluntary_switches : 1506
nr_involuntary_switches : 362
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 54
0---------
---------
Latency Top version : v0.1
18 48286 12698 do_page_fault error_exit
3 4281 3795 sys_mmap system_call_fastpath
4 319 319 futex_wait do_futex sys_futex system_call_fastpath
4 17575 13181 sys_munmap system_call_fastpath
2 21 21 sys_mmap system_call_fastpath
clamd (6494, #threads: 19)
---------------------------------------------------------
se.exec_start : 941777.501631
se.vruntime : 373084.464784
se.sum_exec_runtime : 1563.026910
se.avg_overlap : 0.015511
se.wait_start : 941777.501631
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 27.054878
se.block_max : 59.999974
se.exec_max : 5.465053
se.slice_max : 20.613659
se.wait_max : 72.565150
se.wait_sum : 3116.116243
se.wait_count : 2314
sched_info.bkl_count : 0
se.nr_migrations : 50
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 103
se.nr_failed_migrations_hot : 87
se.nr_forced_migrations : 2
se.nr_forced2_migrations : 25
se.nr_wakeups : 1860
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 36
se.nr_wakeups_local : 760
se.nr_wakeups_remote : 1100
se.nr_wakeups_affine : 36
se.nr_wakeups_affine_attempts : 1859
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.728683
avg_per_cpu : 31.260538
nr_switches : 2145
nr_voluntary_switches : 1852
nr_involuntary_switches : 293
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 63
0---------
---------
Latency Top version : v0.1
4 488 488 futex_wait do_futex sys_futex system_call_fastpath
10 97064 21850 sys_mmap system_call_fastpath
4 3496 2032 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
10 1683 1325 sys_munmap system_call_fastpath
2 16167 15609 do_page_fault error_exit
1 0 0 sys_mmap system_call_fastpath
clamd (6495, #threads: 19)
---------------------------------------------------------
se.exec_start : 941777.501641
se.vruntime : 342351.048003
se.sum_exec_runtime : 2022.330325
se.avg_overlap : 0.724541
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 0.000000
se.sleep_max : 67.497250
se.block_max : 59.994716
se.exec_max : 4.481811
se.slice_max : 31.998247
se.wait_max : 76.998763
se.wait_sum : 3130.901319
se.wait_count : 2311
sched_info.bkl_count : 0
se.nr_migrations : 50
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 66
se.nr_failed_migrations_hot : 48
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 22
se.nr_wakeups : 1713
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 28
se.nr_wakeups_local : 851
se.nr_wakeups_remote : 862
se.nr_wakeups_affine : 28
se.nr_wakeups_affine_attempts : 1713
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.976971
avg_per_cpu : 40.446606
nr_switches : 2070
nr_voluntary_switches : 1703
nr_involuntary_switches : 367
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 63
0---------
---------
Latency Top version : v0.1
10 71007 31248 sys_mmap system_call_fastpath
3 28046 27559 do_page_fault error_exit generic_file_aio_read xfs_read xfs_file_aio_read do_sync_read vfs_read sys_read system_call_fastpath
11 63370 19505 sys_munmap system_call_fastpath
6 20488 19511 do_page_fault error_exit
1 19040 19040 sys_mmap system_call_fastpath
clamd (6496, #threads: 19)
---------------------------------------------------------
se.exec_start : 941777.501635
se.vruntime : 372958.839431
se.sum_exec_runtime : 1607.801008
se.avg_overlap : 0.238204
se.wait_start : 0.000000
se.sleep_start : 0.000000
se.block_start : 941777.501635
se.sleep_max : 40.001752
se.block_max : 51.326701
se.exec_max : 4.015841
se.slice_max : 29.956924
se.wait_max : 83.982555
se.wait_sum : 3023.210300
se.wait_count : 2241
sched_info.bkl_count : 0
se.nr_migrations : 66
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 32
se.nr_failed_migrations_hot : 79
se.nr_forced_migrations : 2
se.nr_forced2_migrations : 31
se.nr_wakeups : 1742
se.nr_wakeups_sync : 1
se.nr_wakeups_migrate : 47
se.nr_wakeups_local : 874
se.nr_wakeups_remote : 868
se.nr_wakeups_affine : 47
se.nr_wakeups_affine_attempts : 1739
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 0.779350
avg_per_cpu : 24.360621
nr_switches : 2063
nr_voluntary_switches : 1730
nr_involuntary_switches : 333
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 60
do_page_fault---------
Mon Aug 25 13:41:56 EEST 2008
next prev parent reply other threads:[~2008-08-25 11:03 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <48B1CC15.2040006@gmail.com>
2008-08-25 5:51 ` Quad core CPUs loaded at only 50% when running a CPU and mmap intensive multi-threaded task Peter Zijlstra
2008-08-25 7:04 ` edwin
2008-08-25 9:23 ` Peter Zijlstra
2008-08-25 9:49 ` Török Edwin
2008-08-25 10:02 ` Peter Zijlstra
2008-08-25 10:22 ` Török Edwin
2008-08-25 10:36 ` Peter Zijlstra
2008-08-25 11:00 ` edwin [this message]
2008-08-25 11:30 ` edwin
2008-08-25 11:41 ` Peter Zijlstra
2008-08-25 13:48 ` Theodore Tso
2008-08-26 8:12 ` Andi Kleen
2008-09-08 19:10 ` Török Edwin
2008-09-12 19:16 ` mmap/munmap latency on multithreaded apps, because pagefaults hold mmap_sem during disk read Török Edwin
2008-09-12 19:16 ` Török Edwin
2008-09-12 19:35 ` Mike Waychison
2008-09-12 19:35 ` Mike Waychison
2008-09-12 20:10 ` Török Edwin
2008-09-12 20:10 ` Török Edwin
2008-08-25 10:37 ` Quad core CPUs loaded at only 50% when running a CPU and mmap intensive multi-threaded task Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=48B290E7.4070805@gmail.com \
--to=edwintorok@gmail.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=peterz@infradead.org \
--cc=rml@tech9.net \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.