From: Richard Davies <richard@arachsys.com>
To: Rik van Riel <riel@redhat.com>
Cc: Avi Kivity <avi@redhat.com>, kvm@vger.kernel.org, qemu-devel@nongnu.org
Subject: Re: Windows slow boot: contractor wanted
Date: Sun, 26 Aug 2012 11:58:03 +0100 [thread overview]
Message-ID: <20120826105803.GA377@alpha.arachsys.com> (raw)
In-Reply-To: <50391564.30401@redhat.com>
Rik van Riel wrote:
> Can you get a backtrace to that _raw_spin_lock_irqsave, to see
> from where it is running into lock contention?
>
> It would be good to know whether it is isolate_freepages_block,
> yield_to, kvm_vcpu_on_spin or something else...
Hi Rik,
I got into a slow boot situation on 3.6.0-rc3, ran "perf record -g -a" for a
while, then ran perf report with the output below.
This trace looks more like the second perf top trace that I sent on Saturday
(there were two in my email and they were different from each other as well
as different from on 3.5.2).
The symptoms were a bit different too - the VM boots appeared to be
completely locked up rather than just slow, and I couldn't quit qemu-kvm at
the monitor - I had to restart the host.
So perhaps this one is actually a deadlock rather than just slow?
Cheers,
Richard.
# ========
# captured on: Sun Aug 26 10:08:28 2012
# os release : 3.6.0-rc3-elastic
# perf version : 3.5.2
# arch : x86_64
# nrcpus online : 16
# nrcpus avail : 16
# cpudesc : AMD Opteron(tm) Processor 6128
# cpuid : AuthenticAMD,16,9,1
# total memory : 131971760 kB
# cmdline : /home/root/bin/perf record -g -a
# event : name = cycles, type = 0, config = 0x0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, id = { 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 }
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# ========
#
# Samples: 2M of event 'cycles'
# Event count (approx.): 1040676441385
#
# Overhead Command Shared Object Symbol
# ........ ............... .................... ..............................................
#
90.01% qemu-kvm [kernel.kallsyms] [k] _raw_spin_lock_irqsave
|
--- _raw_spin_lock_irqsave
|
|--99.99%-- isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--54.91%-- 0x10100000002
| |
| --45.09%-- 0x10100000006
--0.01%-- [...]
4.66% qemu-kvm [kernel.kallsyms] [k] sub_preempt_count
|
--- sub_preempt_count
|
|--99.77%-- _raw_spin_unlock_irqrestore
| |
| |--99.99%-- compact_checklock_irqsave
| | isolate_migratepages_range
| | compact_zone
| | compact_zone_order
| | try_to_compact_pages
| | __alloc_pages_direct_compact
| | __alloc_pages_nodemask
| | alloc_pages_vma
| | do_huge_pmd_anonymous_page
| | handle_mm_fault
| | __get_user_pages
| | get_user_page_nowait
| | hva_to_pfn.isra.33
| | __gfn_to_pfn
| | gfn_to_pfn_async
| | try_async_pf
| | tdp_page_fault
| | kvm_mmu_page_fault
| | pf_interception
| | handle_exit
| | kvm_arch_vcpu_ioctl_run
| | kvm_vcpu_ioctl
| | do_vfs_ioctl
| | sys_ioctl
| | system_call_fastpath
| | ioctl
| | |
| | |--51.94%-- 0x10100000002
| | |
| | --48.06%-- 0x10100000006
| --0.01%-- [...]
--0.23%-- [...]
1.23% ksmd [kernel.kallsyms] [k] memcmp
|
--- memcmp
|
|--99.83%-- memcmp_pages
| |
| |--78.46%-- ksm_scan_thread
| | kthread
| | kernel_thread_helper
| |
| --21.54%-- try_to_merge_with_ksm_page
| ksm_scan_thread
| kthread
| kernel_thread_helper
--0.17%-- [...]
0.91% ksmd [kernel.kallsyms] [k] smp_call_function_many
|
--- smp_call_function_many
|
|--99.98%-- native_flush_tlb_others
| |
| |--99.86%-- flush_tlb_page
| | ptep_clear_flush
| | try_to_merge_with_ksm_page
| | ksm_scan_thread
| | kthread
| | kernel_thread_helper
| --0.14%-- [...]
--0.02%-- [...]
0.34% qemu-kvm [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
|
--- _raw_spin_unlock_irqrestore
|
|--96.08%-- compact_checklock_irqsave
| isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--65.19%-- 0x10100000006
| |
| --34.81%-- 0x10100000002
|
|--2.68%-- isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--52.08%-- 0x10100000002
| |
| --47.92%-- 0x10100000006
|
|--0.56%-- ntp_tick_length
| do_timer
| tick_do_update_jiffies64
| tick_sched_timer
| __run_hrtimer
| hrtimer_interrupt
| smp_apic_timer_interrupt
| apic_timer_interrupt
| compact_checklock_irqsave
| isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| 0x10100000002
--0.68%-- [...]
0.30% swapper [kernel.kallsyms] [k] default_idle
|
--- default_idle
|
|--99.95%-- cpu_idle
| start_secondary
--0.05%-- [...]
0.15% qemu-kvm [kernel.kallsyms] [k] isolate_migratepages_range
|
--- isolate_migratepages_range
|
|--97.41%-- compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--54.02%-- 0x10100000002
| |
| --45.98%-- 0x10100000006
|
--2.59%-- compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--56.10%-- 0x10100000002
|
--43.90%-- 0x10100000006
0.12% qemu-kvm [kernel.kallsyms] [k] compact_zone
|
--- compact_zone
compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--52.09%-- 0x10100000002
|
--47.91%-- 0x10100000006
0.11% qemu-kvm [kernel.kallsyms] [k] flush_tlb_func
|
--- flush_tlb_func
|
|--99.58%-- generic_smp_call_function_interrupt
| smp_call_function_interrupt
| call_function_interrupt
| |
| |--94.65%-- compact_checklock_irqsave
| | isolate_migratepages_range
| | compact_zone
| | compact_zone_order
| | try_to_compact_pages
| | __alloc_pages_direct_compact
| | __alloc_pages_nodemask
| | alloc_pages_vma
| | do_huge_pmd_anonymous_page
| | handle_mm_fault
| | __get_user_pages
| | get_user_page_nowait
| | hva_to_pfn.isra.33
| | __gfn_to_pfn
| | gfn_to_pfn_async
| | try_async_pf
| | tdp_page_fault
| | kvm_mmu_page_fault
| | pf_interception
| | handle_exit
| | kvm_arch_vcpu_ioctl_run
| | kvm_vcpu_ioctl
| | do_vfs_ioctl
| | sys_ioctl
| | system_call_fastpath
| | ioctl
| | |
| | |--78.04%-- 0x10100000006
| | |
| | --21.96%-- 0x10100000002
| |
| |--4.67%-- sub_preempt_count
| | _raw_spin_unlock_irqrestore
| | compact_checklock_irqsave
| | isolate_migratepages_range
| | compact_zone
| | compact_zone_order
| | try_to_compact_pages
| | __alloc_pages_direct_compact
| | __alloc_pages_nodemask
| | alloc_pages_vma
| | do_huge_pmd_anonymous_page
| | handle_mm_fault
| | __get_user_pages
| | get_user_page_nowait
| | hva_to_pfn.isra.33
| | __gfn_to_pfn
| | gfn_to_pfn_async
| | try_async_pf
| | tdp_page_fault
| | kvm_mmu_page_fault
| | pf_interception
| | handle_exit
| | kvm_arch_vcpu_ioctl_run
| | kvm_vcpu_ioctl
| | do_vfs_ioctl
| | sys_ioctl
| | system_call_fastpath
| | ioctl
| | |
| | |--78.18%-- 0x10100000006
| | |
| | --21.82%-- 0x10100000002
| --0.68%-- [...]
--0.42%-- [...]
0.09% qemu-kvm [kernel.kallsyms] [k] mod_zone_page_state
|
--- mod_zone_page_state
|
|--80.84%-- isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--53.90%-- 0x10100000002
| |
| --46.10%-- 0x10100000006
|
--19.16%-- compact_zone
compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--55.04%-- 0x10100000002
|
--44.96%-- 0x10100000006
0.09% qemu-kvm [kernel.kallsyms] [k] migrate_pages
|
--- migrate_pages
|
|--96.21%-- compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--52.94%-- 0x10100000002
| |
| --47.06%-- 0x10100000006
|
--3.79%-- compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--50.72%-- 0x10100000002
|
--49.28%-- 0x10100000006
0.09% qemu-kvm [kernel.kallsyms] [k] __zone_watermark_ok
|
--- __zone_watermark_ok
|
|--95.81%-- zone_watermark_ok
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--51.21%-- 0x10100000002
| |
| --48.79%-- 0x10100000006
|
--4.19%-- compact_zone
compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--50.00%-- 0x10100000006
|
--50.00%-- 0x10100000002
0.06% perf [kernel.kallsyms] [k] copy_user_generic_string
|
--- copy_user_generic_string
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call_fastpath
write
run_builtin
main
__libc_start_main
WARNING: multiple messages have this Message-ID (diff)
From: Richard Davies <richard@arachsys.com>
To: Rik van Riel <riel@redhat.com>
Cc: Avi Kivity <avi@redhat.com>, kvm@vger.kernel.org, qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] Windows slow boot: contractor wanted
Date: Sun, 26 Aug 2012 11:58:03 +0100 [thread overview]
Message-ID: <20120826105803.GA377@alpha.arachsys.com> (raw)
In-Reply-To: <50391564.30401@redhat.com>
Rik van Riel wrote:
> Can you get a backtrace to that _raw_spin_lock_irqsave, to see
> from where it is running into lock contention?
>
> It would be good to know whether it is isolate_freepages_block,
> yield_to, kvm_vcpu_on_spin or something else...
Hi Rik,
I got into a slow boot situation on 3.6.0-rc3, ran "perf record -g -a" for a
while, then ran perf report with the output below.
This trace looks more like the second perf top trace that I sent on Saturday
(there were two in my email and they were different from each other as well
as different from on 3.5.2).
The symptoms were a bit different too - the VM boots appeared to be
completely locked up rather than just slow, and I couldn't quit qemu-kvm at
the monitor - I had to restart the host.
So perhaps this one is actually a deadlock rather than just slow?
Cheers,
Richard.
# ========
# captured on: Sun Aug 26 10:08:28 2012
# os release : 3.6.0-rc3-elastic
# perf version : 3.5.2
# arch : x86_64
# nrcpus online : 16
# nrcpus avail : 16
# cpudesc : AMD Opteron(tm) Processor 6128
# cpuid : AuthenticAMD,16,9,1
# total memory : 131971760 kB
# cmdline : /home/root/bin/perf record -g -a
# event : name = cycles, type = 0, config = 0x0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, id = { 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 }
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# ========
#
# Samples: 2M of event 'cycles'
# Event count (approx.): 1040676441385
#
# Overhead Command Shared Object Symbol
# ........ ............... .................... ..............................................
#
90.01% qemu-kvm [kernel.kallsyms] [k] _raw_spin_lock_irqsave
|
--- _raw_spin_lock_irqsave
|
|--99.99%-- isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--54.91%-- 0x10100000002
| |
| --45.09%-- 0x10100000006
--0.01%-- [...]
4.66% qemu-kvm [kernel.kallsyms] [k] sub_preempt_count
|
--- sub_preempt_count
|
|--99.77%-- _raw_spin_unlock_irqrestore
| |
| |--99.99%-- compact_checklock_irqsave
| | isolate_migratepages_range
| | compact_zone
| | compact_zone_order
| | try_to_compact_pages
| | __alloc_pages_direct_compact
| | __alloc_pages_nodemask
| | alloc_pages_vma
| | do_huge_pmd_anonymous_page
| | handle_mm_fault
| | __get_user_pages
| | get_user_page_nowait
| | hva_to_pfn.isra.33
| | __gfn_to_pfn
| | gfn_to_pfn_async
| | try_async_pf
| | tdp_page_fault
| | kvm_mmu_page_fault
| | pf_interception
| | handle_exit
| | kvm_arch_vcpu_ioctl_run
| | kvm_vcpu_ioctl
| | do_vfs_ioctl
| | sys_ioctl
| | system_call_fastpath
| | ioctl
| | |
| | |--51.94%-- 0x10100000002
| | |
| | --48.06%-- 0x10100000006
| --0.01%-- [...]
--0.23%-- [...]
1.23% ksmd [kernel.kallsyms] [k] memcmp
|
--- memcmp
|
|--99.83%-- memcmp_pages
| |
| |--78.46%-- ksm_scan_thread
| | kthread
| | kernel_thread_helper
| |
| --21.54%-- try_to_merge_with_ksm_page
| ksm_scan_thread
| kthread
| kernel_thread_helper
--0.17%-- [...]
0.91% ksmd [kernel.kallsyms] [k] smp_call_function_many
|
--- smp_call_function_many
|
|--99.98%-- native_flush_tlb_others
| |
| |--99.86%-- flush_tlb_page
| | ptep_clear_flush
| | try_to_merge_with_ksm_page
| | ksm_scan_thread
| | kthread
| | kernel_thread_helper
| --0.14%-- [...]
--0.02%-- [...]
0.34% qemu-kvm [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
|
--- _raw_spin_unlock_irqrestore
|
|--96.08%-- compact_checklock_irqsave
| isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--65.19%-- 0x10100000006
| |
| --34.81%-- 0x10100000002
|
|--2.68%-- isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--52.08%-- 0x10100000002
| |
| --47.92%-- 0x10100000006
|
|--0.56%-- ntp_tick_length
| do_timer
| tick_do_update_jiffies64
| tick_sched_timer
| __run_hrtimer
| hrtimer_interrupt
| smp_apic_timer_interrupt
| apic_timer_interrupt
| compact_checklock_irqsave
| isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| 0x10100000002
--0.68%-- [...]
0.30% swapper [kernel.kallsyms] [k] default_idle
|
--- default_idle
|
|--99.95%-- cpu_idle
| start_secondary
--0.05%-- [...]
0.15% qemu-kvm [kernel.kallsyms] [k] isolate_migratepages_range
|
--- isolate_migratepages_range
|
|--97.41%-- compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--54.02%-- 0x10100000002
| |
| --45.98%-- 0x10100000006
|
--2.59%-- compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--56.10%-- 0x10100000002
|
--43.90%-- 0x10100000006
0.12% qemu-kvm [kernel.kallsyms] [k] compact_zone
|
--- compact_zone
compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--52.09%-- 0x10100000002
|
--47.91%-- 0x10100000006
0.11% qemu-kvm [kernel.kallsyms] [k] flush_tlb_func
|
--- flush_tlb_func
|
|--99.58%-- generic_smp_call_function_interrupt
| smp_call_function_interrupt
| call_function_interrupt
| |
| |--94.65%-- compact_checklock_irqsave
| | isolate_migratepages_range
| | compact_zone
| | compact_zone_order
| | try_to_compact_pages
| | __alloc_pages_direct_compact
| | __alloc_pages_nodemask
| | alloc_pages_vma
| | do_huge_pmd_anonymous_page
| | handle_mm_fault
| | __get_user_pages
| | get_user_page_nowait
| | hva_to_pfn.isra.33
| | __gfn_to_pfn
| | gfn_to_pfn_async
| | try_async_pf
| | tdp_page_fault
| | kvm_mmu_page_fault
| | pf_interception
| | handle_exit
| | kvm_arch_vcpu_ioctl_run
| | kvm_vcpu_ioctl
| | do_vfs_ioctl
| | sys_ioctl
| | system_call_fastpath
| | ioctl
| | |
| | |--78.04%-- 0x10100000006
| | |
| | --21.96%-- 0x10100000002
| |
| |--4.67%-- sub_preempt_count
| | _raw_spin_unlock_irqrestore
| | compact_checklock_irqsave
| | isolate_migratepages_range
| | compact_zone
| | compact_zone_order
| | try_to_compact_pages
| | __alloc_pages_direct_compact
| | __alloc_pages_nodemask
| | alloc_pages_vma
| | do_huge_pmd_anonymous_page
| | handle_mm_fault
| | __get_user_pages
| | get_user_page_nowait
| | hva_to_pfn.isra.33
| | __gfn_to_pfn
| | gfn_to_pfn_async
| | try_async_pf
| | tdp_page_fault
| | kvm_mmu_page_fault
| | pf_interception
| | handle_exit
| | kvm_arch_vcpu_ioctl_run
| | kvm_vcpu_ioctl
| | do_vfs_ioctl
| | sys_ioctl
| | system_call_fastpath
| | ioctl
| | |
| | |--78.18%-- 0x10100000006
| | |
| | --21.82%-- 0x10100000002
| --0.68%-- [...]
--0.42%-- [...]
0.09% qemu-kvm [kernel.kallsyms] [k] mod_zone_page_state
|
--- mod_zone_page_state
|
|--80.84%-- isolate_migratepages_range
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--53.90%-- 0x10100000002
| |
| --46.10%-- 0x10100000006
|
--19.16%-- compact_zone
compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--55.04%-- 0x10100000002
|
--44.96%-- 0x10100000006
0.09% qemu-kvm [kernel.kallsyms] [k] migrate_pages
|
--- migrate_pages
|
|--96.21%-- compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--52.94%-- 0x10100000002
| |
| --47.06%-- 0x10100000006
|
--3.79%-- compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--50.72%-- 0x10100000002
|
--49.28%-- 0x10100000006
0.09% qemu-kvm [kernel.kallsyms] [k] __zone_watermark_ok
|
--- __zone_watermark_ok
|
|--95.81%-- zone_watermark_ok
| compact_zone
| compact_zone_order
| try_to_compact_pages
| __alloc_pages_direct_compact
| __alloc_pages_nodemask
| alloc_pages_vma
| do_huge_pmd_anonymous_page
| handle_mm_fault
| __get_user_pages
| get_user_page_nowait
| hva_to_pfn.isra.33
| __gfn_to_pfn
| gfn_to_pfn_async
| try_async_pf
| tdp_page_fault
| kvm_mmu_page_fault
| pf_interception
| handle_exit
| kvm_arch_vcpu_ioctl_run
| kvm_vcpu_ioctl
| do_vfs_ioctl
| sys_ioctl
| system_call_fastpath
| ioctl
| |
| |--51.21%-- 0x10100000002
| |
| --48.79%-- 0x10100000006
|
--4.19%-- compact_zone
compact_zone_order
try_to_compact_pages
__alloc_pages_direct_compact
__alloc_pages_nodemask
alloc_pages_vma
do_huge_pmd_anonymous_page
handle_mm_fault
__get_user_pages
get_user_page_nowait
hva_to_pfn.isra.33
__gfn_to_pfn
gfn_to_pfn_async
try_async_pf
tdp_page_fault
kvm_mmu_page_fault
pf_interception
handle_exit
kvm_arch_vcpu_ioctl_run
kvm_vcpu_ioctl
do_vfs_ioctl
sys_ioctl
system_call_fastpath
ioctl
|
|--50.00%-- 0x10100000006
|
--50.00%-- 0x10100000002
0.06% perf [kernel.kallsyms] [k] copy_user_generic_string
|
--- copy_user_generic_string
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call_fastpath
write
run_builtin
main
__libc_start_main
next prev parent reply other threads:[~2012-08-26 10:58 UTC|newest] Thread overview: 101+ messages / expand[flat|nested] mbox.gz Atom feed top 2012-08-16 10:47 Windows slow boot: contractor wanted Richard Davies 2012-08-16 10:47 ` [Qemu-devel] " Richard Davies 2012-08-16 11:39 ` Avi Kivity 2012-08-16 11:39 ` [Qemu-devel] " Avi Kivity 2012-08-17 12:36 ` Richard Davies 2012-08-17 12:36 ` [Qemu-devel] " Richard Davies 2012-08-17 13:02 ` Robert Vineyard 2012-08-17 13:02 ` [Qemu-devel] " Robert Vineyard 2012-08-18 14:44 ` Richard Davies 2012-08-18 14:44 ` [Qemu-devel] " Richard Davies 2012-08-19 5:02 ` Brian Jackson 2012-08-19 5:02 ` [Qemu-devel] " Brian Jackson 2012-08-20 8:16 ` Richard Davies 2012-08-20 8:16 ` [Qemu-devel] " Richard Davies 2012-08-19 8:40 ` Avi Kivity 2012-08-19 8:40 ` [Qemu-devel] " Avi Kivity 2012-08-19 8:51 ` Richard Davies 2012-08-19 8:51 ` [Qemu-devel] " Richard Davies 2012-08-19 14:04 ` Avi Kivity 2012-08-19 14:04 ` [Qemu-devel] " Avi Kivity 2012-08-20 13:56 ` Richard Davies 2012-08-20 13:56 ` [Qemu-devel] " Richard Davies 2012-08-21 9:00 ` Avi Kivity 2012-08-21 9:00 ` [Qemu-devel] " Avi Kivity 2012-08-21 15:21 ` Richard Davies 2012-08-21 15:21 ` [Qemu-devel] " Richard Davies 2012-08-21 15:39 ` Troy Benjegerdes 2012-08-21 15:39 ` Troy Benjegerdes 2012-08-22 9:08 ` Avi Kivity 2012-08-22 9:08 ` [Qemu-devel] " Avi Kivity 2012-08-22 12:40 ` Richard Davies 2012-08-22 12:40 ` [Qemu-devel] " Richard Davies 2012-08-22 12:44 ` Avi Kivity 2012-08-22 12:44 ` [Qemu-devel] " Avi Kivity 2012-08-22 14:41 ` Richard Davies 2012-08-22 14:41 ` [Qemu-devel] " Richard Davies 2012-08-22 14:53 ` Avi Kivity 2012-08-22 14:53 ` [Qemu-devel] " Avi Kivity 2012-08-22 15:26 ` Richard Davies 2012-08-22 15:26 ` [Qemu-devel] " Richard Davies 2012-08-22 17:22 ` Troy Benjegerdes 2012-08-22 17:22 ` Troy Benjegerdes 2012-08-25 17:51 ` Richard Davies 2012-08-25 17:51 ` Richard Davies 2012-08-22 15:21 ` Rik van Riel 2012-08-22 15:21 ` [Qemu-devel] " Rik van Riel 2012-08-22 15:34 ` Richard Davies 2012-08-22 15:34 ` [Qemu-devel] " Richard Davies 2012-08-25 17:45 ` Richard Davies 2012-08-25 17:45 ` [Qemu-devel] " Richard Davies 2012-08-25 18:11 ` Rik van Riel 2012-08-25 18:11 ` [Qemu-devel] " Rik van Riel 2012-08-26 10:58 ` Richard Davies [this message] 2012-08-26 10:58 ` Richard Davies 2012-09-06 9:20 ` Richard Davies 2012-09-06 9:20 ` [Qemu-devel] " Richard Davies 2012-09-12 10:56 ` Windows VM slow boot Richard Davies 2012-09-12 10:56 ` [Qemu-devel] " Richard Davies 2012-09-12 10:56 ` Richard Davies 2012-09-12 12:25 ` Mel Gorman 2012-09-12 12:25 ` [Qemu-devel] " Mel Gorman 2012-09-12 12:25 ` Mel Gorman 2012-09-12 16:46 ` Richard Davies 2012-09-12 16:46 ` [Qemu-devel] " Richard Davies 2012-09-12 16:46 ` Richard Davies 2012-09-13 9:50 ` Mel Gorman 2012-09-13 9:50 ` [Qemu-devel] " Mel Gorman 2012-09-13 9:50 ` Mel Gorman 2012-09-13 19:47 ` [PATCH 1/2] Revert "mm: have order > 0 compaction start near a pageblock with free pages" Rik van Riel 2012-09-13 19:47 ` [Qemu-devel] " Rik van Riel 2012-09-13 19:47 ` Rik van Riel 2012-09-13 19:48 ` [PATCH 2/2] make the compaction "skip ahead" logic robust Rik van Riel 2012-09-13 19:48 ` [Qemu-devel] " Rik van Riel 2012-09-13 19:48 ` Rik van Riel 2012-09-13 19:54 ` [PATCH -v2 " Rik van Riel 2012-09-13 19:54 ` [Qemu-devel] " Rik van Riel 2012-09-13 19:54 ` Rik van Riel 2012-09-15 15:55 ` Richard Davies 2012-09-15 15:55 ` [Qemu-devel] " Richard Davies 2012-09-15 15:55 ` Richard Davies 2012-09-16 19:12 ` Richard Davies 2012-09-16 19:12 ` [Qemu-devel] " Richard Davies 2012-09-17 12:26 ` Mel Gorman 2012-09-17 12:26 ` [Qemu-devel] " Mel Gorman 2012-09-18 8:14 ` Richard Davies 2012-09-18 8:14 ` [Qemu-devel] " Richard Davies 2012-09-18 11:21 ` Mel Gorman 2012-09-18 11:21 ` [Qemu-devel] " Mel Gorman 2012-09-18 11:21 ` Mel Gorman 2012-09-18 17:58 ` Richard Davies 2012-09-18 17:58 ` [Qemu-devel] " Richard Davies 2012-09-17 13:50 ` Rik van Riel 2012-09-17 13:50 ` [Qemu-devel] " Rik van Riel 2012-09-17 14:07 ` Mel Gorman 2012-09-17 14:07 ` [Qemu-devel] " Mel Gorman 2012-09-17 14:07 ` Mel Gorman 2012-08-16 14:10 ` Windows slow boot: contractor wanted Benoît Canet 2012-08-16 14:10 ` [Qemu-devel] " Benoît Canet 2012-08-16 15:53 ` Troy Benjegerdes 2012-09-18 15:12 ` Windows slow boot Michael Tokarev 2012-09-18 15:12 ` [Qemu-devel] " Michael Tokarev
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120826105803.GA377@alpha.arachsys.com \
--to=richard@arachsys.com \
--cc=avi@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=qemu-devel@nongnu.org \
--cc=riel@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.