* RE: More network tests with xenoprofile this time
@ 2005-06-01 0:15 Santos, Jose Renato G
2005-06-07 21:47 ` William Cohen
0 siblings, 1 reply; 12+ messages in thread
From: Santos, Jose Renato G @ 2005-06-01 0:15 UTC (permalink / raw)
To: Andrew Theurer, xen-devel
Cc: Turner, Yoshio, Aravind Menon, G John Janakiraman, Ian Pratt
Andrew,
You may want to take a look at the folowing paper
which is being presented at VEE'05 (June 11 and 12, 2005).
http://www.hpl.hp.com/research/dca/system/papers/xenoprof-vee05.pdf
It presents network performance results using xenoprof.
This was done for xen 2.0.3. The profile you reported
has some similarities with our results although the
exact numbers are different. But that is expected, since
you are running a different version of Xen on a different
hardware.
We have seen that a significant amount of time was spent
on handling interrupts in Xen, as well.
We have also seen that a significant amount of time is
spent on the hypervisor (+/- 40%) for the dom1 <-> external
case, measured both at dom1 and at dom0.
(in our case we instrumented the receive side)
When we run the benchmark on dom0 the time spent on Xen
is reduced to (+/-20%).
Most of this extra Xen overhead when running a guest
seems to come from the page transfer between
domain 0 and the guest (see table 6 and discussion
on paper).
The paper omits the complete oprofile reports
for brevity. I will be happy to send you any
detailed oprofile report we have generated for the
paper, if you want to compare it with your results.
Just let me know ...
Renato
>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com
>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Ian Pratt
>> Sent: Tuesday, May 31, 2005 3:16 PM
>> To: Andrew Theurer; xen-devel@lists.xensource.com
>> Subject: RE: [Xen-devel] More network tests with xenoprofile
>> this time
>>
>>
>> > I had a chance to run a couple of the netperf tests with
>> > xenoprofile. I am still having some trouble with
>> > multi-domain profiles (probably user error), but I have been
>> > able to profile dom0 while running 2 types of tests. I was
>> > surprised to see as much as 50% cpu in hypervisor on these tests:
>> >
>> > netperf tcp_stream 16k msg size, dom1 -> dom2 dom0 is on
>> > cpu0, HT thread 0, dom1 is on cpu1, HT thread 0,
>> > dom2 is on cpu1, HT thread 1.
>>
>> Let's ignore the domU <-> domU results for the moment as we
>> know about the problem with lack of batching in this
>> scenario. Let's dig into the dom1 -> external.
>>
>> First off, are these figures just for CPU 0 HT 0? i.e. just
>> dom0 so we don't see where time goes in the domU? How is
>> idle time on the CPU reported?
>>
>> Spending 18% of the time handling interrupts in Xen is
>> surprisingy (at least to me).
>>
>> What interrupt rate are you observing? What are the default
>> tg3 interrupt coallescing settings? What interrupt rate do
>> you get on native? Also, what hypercall rate are you seeing?
>>
>> (It would be good to put this in context of the rx/tx packet rates).
>>
>> Is the Ethernet NIC sharing an interrupt with the USB
>> controller per chance?
>>
>> Seeing find_domain_by_id and copy_from_user so high up the
>> list is pretty surprising.
>>
>> Cheers,
>> Ian
>>
>> > netperf tcp_stream 16k msg size, dom1 -> external host dom0
>> > is on cpu0, HT thread 0, dom1 is on cpu1, HT thread 1.
>> >
>> >
>> > Throughput is ~940 Mbps, wire speed.
>> >
>> > xenoprofile opreport:
>> >
>> > 4244562 49.9375 xen-unstable-syms
>> > 4110594 48.3614 vmlinux-2.6.11-xen0-up
>> > 132643 1.5606 oprofiled
>> > 4212 0.0496 libc-2.3.3.so
>> > 2892 0.0340 libpython2.3.so.1.0
>> >
>> > xenoprofile opreport -l:
>> >
>> > 828587 9.75 xen-unstable-syms end_level_ioapic_irq
>> > 712035 8.38 xen-unstable-syms
>> mask_and_ack_level_ioapic_irq
>> > 370265 4.36 vmlinux-2.6.11-xen0-up net_tx_action
>> > 323797 3.81 vmlinux-2.6.11-xen0-up ohci_irq
>> > 282005 3.32 vmlinux-2.6.11-xen0-up tg3_interrupt
>> > 273161 3.21 xen-unstable-syms find_domain_by_id
>> > 234726 2.76 xen-unstable-syms hypercall
>> > 206693 2.43 xen-unstable-syms do_update_va_mapping
>> > 203758 2.40 xen-unstable-syms __copy_from_user_ll
>> > 201665 2.37 xen-unstable-syms do_mmuext_op
>> > 195020 2.29 vmlinux-2.6.11-xen0-up nf_iterate
>> > 184295 2.17 vmlinux-2.6.11-xen0-up nf_hook_slow
>> > 172110 2.02 vmlinux-2.6.11-xen0-up tg3_rx
>> > 164337 1.93 vmlinux-2.6.11-xen0-up net_rx_action
>> > 141999 1.67 xen-unstable-syms do_mmu_update
>> > 139120 1.64 vmlinux-2.6.11-xen0-up fdb_insert
>> > 122483 1.44 xen-unstable-syms mod_l1_entry
>> > 122017 1.44 xen-unstable-syms put_page_from_l1e
>> > 111159 1.31 xen-unstable-syms get_page_from_l1e
>> > 109921 1.29 xen-unstable-syms do_IRQ
>> > 99847 1.17 vmlinux-2.6.11-xen0-up br_handle_frame
>> > 99709 1.17 xen-unstable-syms get_page_type
>> > 93613 1.10 vmlinux-2.6.11-xen0-up kfree
>> > 90885 1.07 vmlinux-2.6.11-xen0-up end_pirq
>> >
>> >
>> > -Andrew
>> >
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>> >
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: More network tests with xenoprofile this time
2005-06-01 0:15 More network tests with xenoprofile this time Santos, Jose Renato G
@ 2005-06-07 21:47 ` William Cohen
2005-06-08 19:20 ` Andrew Theurer
0 siblings, 1 reply; 12+ messages in thread
From: William Cohen @ 2005-06-07 21:47 UTC (permalink / raw)
To: Santos, Jose Renato G
Cc: Ian Pratt, xen-devel, Turner, Yoshio, Andrew Theurer,
Aravind Menon, G John Janakiraman
Santos, Jose Renato G wrote:
> Andrew,
>
> You may want to take a look at the folowing paper
> which is being presented at VEE'05 (June 11 and 12, 2005).
>
> http://www.hpl.hp.com/research/dca/system/papers/xenoprof-vee05.pdf
>
> It presents network performance results using xenoprof.
> This was done for xen 2.0.3. The profile you reported
> has some similarities with our results although the
> exact numbers are different. But that is expected, since
> you are running a different version of Xen on a different
> hardware.
> We have seen that a significant amount of time was spent
> on handling interrupts in Xen, as well.
> We have also seen that a significant amount of time is
> spent on the hypervisor (+/- 40%) for the dom1 <-> external
> case, measured both at dom1 and at dom0.
> (in our case we instrumented the receive side)
> When we run the benchmark on dom0 the time spent on Xen
> is reduced to (+/-20%).
> Most of this extra Xen overhead when running a guest
> seems to come from the page transfer between
> domain 0 and the guest (see table 6 and discussion
> on paper).
>
> The paper omits the complete oprofile reports
> for brevity. I will be happy to send you any
> detailed oprofile report we have generated for the
> paper, if you want to compare it with your results.
> Just let me know ...
>
> Renato
Hi Renato,
The article was an interesting application of the xenoprof.
It seem like it would be useful to also have data collected using the
cycle counts (GLOBAL_POWER_EVENTS on P4) to give some indication of
areas with high overhead operations. There may be some areas with few
very expensive instructions. Calling attention to those areas would help
improve performance.
The increases in I-TLB and D-TLB events for Xen-domain0 shown in Figure
4 are surprising. Why would the working sets be that much larger for
Xen-domain0 than regular linux, particularly for code? Is there an table
similar to table 3 for I-TLB event sample locations?
Can't the VMM use a 4-MB page and the Xen-domain0 kernel shouldn't be
that much larger than regular linux kernel? How were TLB flushes ruled
out as a cause? Could the PERFCOUNTER_CPU counters in perfc_defn.h be
used to see if the VMM is doing a lot of TLB flushes?
Also how much of I-TLB and D-TLB events are due to the P4 architecture?
Are the results so dramatic for a Athlon or AMD64 processors?
-Will
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: More network tests with xenoprofile this time
2005-06-07 21:47 ` William Cohen
@ 2005-06-08 19:20 ` Andrew Theurer
0 siblings, 0 replies; 12+ messages in thread
From: Andrew Theurer @ 2005-06-08 19:20 UTC (permalink / raw)
To: William Cohen, Santos, Jose Renato G
Cc: Ian Pratt, Aravind Menon, xen-devel, G John Janakiraman,
Turner, Yoshio
> Hi Renato,
>
> The article was an interesting application of the xenoprof.
>
> It seem like it would be useful to also have data collected using the
> cycle counts (GLOBAL_POWER_EVENTS on P4) to give some indication of
> areas with high overhead operations. There may be some areas with few
> very expensive instructions. Calling attention to those areas would
> help improve performance.
>
> The increases in I-TLB and D-TLB events for Xen-domain0 shown in
> Figure 4 are surprising. Why would the working sets be that much
> larger for Xen-domain0 than regular linux, particularly for code? Is
> there an table similar to table 3 for I-TLB event sample locations?
>
> Can't the VMM use a 4-MB page and the Xen-domain0 kernel shouldn't be
> that much larger than regular linux kernel? How were TLB flushes
> ruled out as a cause? Could the PERFCOUNTER_CPU counters in
> perfc_defn.h be used to see if the VMM is doing a lot of TLB flushes?
I had the same concern as you, and IMO, it seemed unlikely that the
working set for dom0 would be so much larger to cause significant
amount of TLB miss. I also suspect TLB flushes to be the problem, but
I have not had a chance to look at it. I hope to very soon.
-Andrew
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: More network tests with xenoprofile this time
@ 2005-06-17 19:39 Santos, Jose Renato G
2005-06-22 2:22 ` Andrew Theurer
0 siblings, 1 reply; 12+ messages in thread
From: Santos, Jose Renato G @ 2005-06-17 19:39 UTC (permalink / raw)
To: William Cohen
Cc: Ian Pratt, xen-devel, Turner, Yoshio, Andrew Theurer,
Aravind Menon, G John Janakiraman
[-- Attachment #1: Type: text/plain, Size: 3190 bytes --]
William and Andrew
Sorry for the delay in replying. I have been traveling
and did not have email access while away.
>
> Hi Renato,
>
> The article was an interesting application of the xenoprof.
>
> It seem like it would be useful to also have data collected using the
> cycle counts (GLOBAL_POWER_EVENTS on P4) to give some indication of
> areas with high overhead operations. There may be some areas with few
> very expensive instructions. Calling attention to those areas
> would help
> improve performance.
Yes, you are right. We have in fact collected GLOBAL_POWER_EVENTS,
but did not include in the paper due to space limitations.
I have attached oprofile results for our ttcp like benchmark(receive
side) for the case with 1 NIC (both cycle counts and instructions).
As you can see there are some functions with very expensive
instructions.
For example "hypercall" add anly 0.6% additional instructions but
these consume 3.0% more clock cycles; "unmask_IO_APIC_irq" add
0.25% instructions but consume 5% more cycles. It would be
interesting to investigate these and see if we can optimize them.
>
> The increases in I-TLB and D-TLB events for Xen-domain0 shown
> in Figure
> 4 are surprising. Why would the working sets be that much larger for
> Xen-domain0 than regular linux, particularly for code? Is
> there an table
> similar to table 3 for I-TLB event sample locations?
>
Yes, we were also surprised by these results. I have attached
the complete I-TLB and D_TLB oprofile results (for the 3 NICs case)
(note these are on a different type of machine than the other
2 attached oprofile results)
Aravind instrumented the macros in xen/include/asm-x86/flushtlb.h.
I am not sure if he used PERFCOUNTER_CPU or if he included his
own instrumentation. With this instrumentation we did not observe
any TLB flush, but I suppose we could have missed TLB flushes
that did not use the macro... I think it would be a good idea to
investigate this further to confirm that TLB flushes are not
happening.
One additional observation is that in general the number of misses
in NOT proportional to the size of the working set. It is possible
that a small increase in the working set significantly increase the
number of misses. Therefore it is possible that the increase
in TLB misses is in fact due to a larger working set. But, I agree
we have to investigate this further to get confirmation ...
> Can't the VMM use a 4-MB page and the Xen-domain0 kernel shouldn't be
> that much larger than regular linux kernel?
> How were TLB flushes ruled
> out as a cause? Could the PERFCOUNTER_CPU counters in perfc_defn.h be
> used to see if the VMM is doing a lot of TLB flushes?
>
> Also how much of I-TLB and D-TLB events are due to the P4
> architecture?
> Are the results so dramatic for a Athlon or AMD64 processors?
>
We did not try this on any other architecture.
Right now xenoprof is only supported on P4.
Support for other architectures is not on top of our priority list.
Regards
Renato
> -Will
>
>
[-- Attachment #2: time_func_xen0.prof --]
[-- Type: application/octet-stream, Size: 20343 bytes --]
CPU: P4 / Xeon, speed 2799.36 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 1000000
samples % app name symbol name
16313 20.1224 vmlinux-syms-2.6.10-xen0 __copy_user_intel
13355 16.4736 bcm5700 (no symbols)
4065 5.0142 xen-syms-2.0.3 unmask_IO_APIC_irq
2884 3.5575 vmlinux-syms-2.6.10-xen0 eth_type_trans
2496 3.0789 xen-syms-2.0.3 hypercall
1496 1.8453 vmlinux-syms-2.6.10-xen0 kfree
1360 1.6776 vmlinux-syms-2.6.10-xen0 tcp_v4_rcv
1208 1.4901 vmlinux-syms-2.6.10-xen0 schedule
1152 1.4210 vmlinux-syms-2.6.10-xen0 __switch_to
1151 1.4198 vmlinux-syms-2.6.10-xen0 tcp_rcv_established
1138 1.4037 xen-syms-2.0.3 get_s_time
1108 1.3667 vmlinux-syms-2.6.10-xen0 alloc_skb_from_cache
823 1.0152 vmlinux-syms-2.6.10-xen0 kmem_cache_alloc
802 0.9893 xen-syms-2.0.3 __copy_from_user_ll
690 0.8511 xen-syms-2.0.3 do_multicall
666 0.8215 xen-syms-2.0.3 __enter_scheduler
632 0.7796 vmlinux-syms-2.6.10-xen0 netif_receive_skb
628 0.7746 vmlinux-syms-2.6.10-xen0 end_pirq
622 0.7672 vmlinux-syms-2.6.10-xen0 tcp_recvmsg
581 0.7167 xen-syms-2.0.3 copy_from_user
580 0.7154 xen-syms-2.0.3 __copy_to_user_ll
572 0.7056 vmlinux-syms-2.6.10-xen0 nf_iterate
556 0.6858 vmlinux-syms-2.6.10-xen0 ip_rcv
539 0.6649 xen-syms-2.0.3 switch_to
518 0.6390 xen-syms-2.0.3 stext
516 0.6365 vmlinux-syms-2.6.10-xen0 xen_idle
510 0.6291 vmlinux-syms-2.6.10-xen0 hypervisor_callback
510 0.6291 vmlinux-syms-2.6.10-xen0 ip_queue_xmit
509 0.6279 vmlinux-syms-2.6.10-xen0 kmem_cache_free
498 0.6143 vmlinux-syms-2.6.10-xen0 set_timeout_timer
497 0.6131 vmlinux-syms-2.6.10-xen0 system_call
481 0.5933 vmlinux-syms-2.6.10-xen0 ip_route_input
461 0.5687 vmlinux-syms-2.6.10-xen0 nf_hook_slow
452 0.5575 xen-syms-2.0.3 rem_ac_timer
447 0.5514 rcv __read
434 0.5353 vmlinux-syms-2.6.10-xen0 evtchn_do_upcall
425 0.5242 vmlinux-syms-2.6.10-xen0 tcp_transmit_skb
422 0.5205 vmlinux-syms-2.6.10-xen0 local_bh_enable
393 0.4848 vmlinux-syms-2.6.10-xen0 rt_hash_code
390 0.4811 vmlinux-syms-2.6.10-xen0 tcp_prequeue_process
389 0.4798 vmlinux-syms-2.6.10-xen0 preempt_schedule
376 0.4638 vmlinux-syms-2.6.10-xen0 sched_clock
371 0.4576 vmlinux-syms-2.6.10-xen0 ip_local_deliver
340 0.4194 vmlinux-syms-2.6.10-xen0 sock_aio_read
333 0.4108 rcv main
327 0.4034 vmlinux-syms-2.6.10-xen0 __kfree_skb
324 0.3997 vmlinux-syms-2.6.10-xen0 schedule_timeout
322 0.3972 vmlinux-syms-2.6.10-xen0 skb_release_data
321 0.3960 xen-syms-2.0.3 pirq_guest_unmask
307 0.3787 vmlinux-syms-2.6.10-xen0 recalc_task_prio
307 0.3787 vmlinux-syms-2.6.10-xen0 tcp_send_delayed_ack
295 0.3639 vmlinux-syms-2.6.10-xen0 tcp_copy_to_iovec
291 0.3590 vmlinux-syms-2.6.10-xen0 ack_dynirq
289 0.3565 vmlinux-syms-2.6.10-xen0 skb_copy_datagram_iovec
286 0.3528 vmlinux-syms-2.6.10-xen0 dev_queue_xmit
282 0.3479 vmlinux-syms-2.6.10-xen0 memcpy_toiovec
279 0.3442 vmlinux-syms-2.6.10-xen0 do_sync_read
274 0.3380 vmlinux-syms-2.6.10-xen0 __kmalloc
273 0.3368 vmlinux-syms-2.6.10-xen0 __mod_timer
246 0.3034 vmlinux-syms-2.6.10-xen0 dnotify_parent
241 0.2973 vmlinux-syms-2.6.10-xen0 tcp_event_data_recv
240 0.2960 vmlinux-syms-2.6.10-xen0 __tcp_select_window
239 0.2948 vmlinux-syms-2.6.10-xen0 syscall_exit
236 0.2911 vmlinux-syms-2.6.10-xen0 sk_wait_data
233 0.2874 xen-syms-2.0.3 bvt_do_schedule
232 0.2862 vmlinux-syms-2.6.10-xen0 alloc_skb
230 0.2837 vmlinux-syms-2.6.10-xen0 __do_IRQ
230 0.2837 vmlinux-syms-2.6.10-xen0 ip_finish_output2
224 0.2763 vmlinux-syms-2.6.10-xen0 timer_interrupt
222 0.2738 vmlinux-syms-2.6.10-xen0 ack_pirq
206 0.2541 vmlinux-syms-2.6.10-xen0 scrit
199 0.2455 vmlinux-syms-2.6.10-xen0 do_softirq
198 0.2442 vmlinux-syms-2.6.10-xen0 ip_rcv_finish
197 0.2430 vmlinux-syms-2.6.10-xen0 vfs_read
196 0.2418 vmlinux-syms-2.6.10-xen0 handle_IRQ_event
195 0.2405 vmlinux-syms-2.6.10-xen0 pfifo_fast_enqueue
190 0.2344 vmlinux-syms-2.6.10-xen0 mod_timer
187 0.2307 vmlinux-syms-2.6.10-xen0 copy_to_user
183 0.2257 xen-syms-2.0.3 do_stack_switch
182 0.2245 vmlinux-syms-2.6.10-xen0 pfifo_fast_dequeue
180 0.2220 vmlinux-syms-2.6.10-xen0 try_to_wake_up
174 0.2146 vmlinux-syms-2.6.10-xen0 fget_light
173 0.2134 vmlinux-syms-2.6.10-xen0 finish_task_switch
172 0.2122 vmlinux-syms-2.6.10-xen0 next_timer_interrupt
170 0.2097 vmlinux-syms-2.6.10-xen0 __wake_up_common
169 0.2085 xen-syms-2.0.3 do_sched_op
169 0.2085 xen-syms-2.0.3 test_all_events
168 0.2072 vmlinux-syms-2.6.10-xen0 prepare_to_wait
163 0.2011 vmlinux-syms-2.6.10-xen0 tcp_rcv_space_adjust
160 0.1974 vmlinux-syms-2.6.10-xen0 kfree_skbmem
156 0.1924 vmlinux-syms-2.6.10-xen0 tcp_v4_send_check
155 0.1912 xen-syms-2.0.3 restore_all_vm86
152 0.1875 xen-syms-2.0.3 remove_entry
148 0.1826 xen-syms-2.0.3 do_dom0_op
146 0.1801 vmlinux-syms-2.6.10-xen0 csum_partial
145 0.1789 vmlinux-syms-2.6.10-xen0 tcp_v4_do_rcv
143 0.1764 vmlinux-syms-2.6.10-xen0 end_dynirq
141 0.1739 vmlinux-syms-2.6.10-xen0 __dev_alloc_skb
140 0.1727 vmlinux-syms-2.6.10-xen0 MULTICALL_flush_page_update_queue
138 0.1702 vmlinux-syms-2.6.10-xen0 ip_sabotage_out
132 0.1628 vmlinux-syms-2.6.10-xen0 qdisc_restart
130 0.1604 vmlinux-syms-2.6.10-xen0 dev_queue_xmit_nit
129 0.1591 vmlinux-syms-2.6.10-xen0 cpu_idle
126 0.1554 xen-syms-2.0.3 ac_timer_softirq_action
121 0.1493 vmlinux-syms-2.6.10-xen0 __copy_to_user_ll
120 0.1480 xen-syms-2.0.3 do_physdev_op
108 0.1332 vmlinux-syms-2.6.10-xen0 irq_exit
108 0.1332 vmlinux-syms-2.6.10-xen0 net_tx_action
105 0.1295 xen-syms-2.0.3 add_entry
103 0.1271 xen-syms-2.0.3 copy_to_user
102 0.1258 vmlinux-syms-2.6.10-xen0 __get_time_values_from_xen
101 0.1246 vmlinux-syms-2.6.10-xen0 cleanup_rbuf
98 0.1209 vmlinux-syms-2.6.10-xen0 __do_softirq
98 0.1209 vmlinux-syms-2.6.10-xen0 sock_common_recvmsg
95 0.1172 vmlinux-syms-2.6.10-xen0 net_rx_action
93 0.1147 vmlinux-syms-2.6.10-xen0 get_offset_tsc
92 0.1135 vmlinux-syms-2.6.10-xen0 __wake_up
91 0.1123 vmlinux-syms-2.6.10-xen0 dequeue_task
89 0.1098 vmlinux-syms-2.6.10-xen0 sk_reset_timer
87 0.1073 vmlinux-syms-2.6.10-xen0 tcp_rcv_rtt_update
87 0.1073 xen-syms-2.0.3 do_softirq
85 0.1048 xen-syms-2.0.3 __constant_memcpy
84 0.1036 vmlinux-syms-2.6.10-xen0 sys_read
83 0.1024 vmlinux-syms-2.6.10-xen0 ip_sabotage_in
80 0.0987 vmlinux-syms-2.6.10-xen0 do_IRQ
74 0.0913 xen-syms-2.0.3 FLT14
68 0.0839 vmlinux-syms-2.6.10-xen0 enqueue_task
67 0.0826 xen-syms-2.0.3 do_set_timer_op
64 0.0789 vmlinux-syms-2.6.10-xen0 sock_wfree
63 0.0777 vmlinux-syms-2.6.10-xen0 effective_prio
63 0.0777 vmlinux-syms-2.6.10-xen0 restore_all
63 0.0777 xen-syms-2.0.3 reprogram_ac_timer
60 0.0740 vmlinux-syms-2.6.10-xen0 internal_add_timer
53 0.0654 xen-syms-2.0.3 FLT2
49 0.0604 vmlinux-syms-2.6.10-xen0 ip_output
48 0.0592 vmlinux-syms-2.6.10-xen0 dst_output
46 0.0567 vmlinux-syms-2.6.10-xen0 ip_finish_output
46 0.0567 vmlinux-syms-2.6.10-xen0 release_sock
46 0.0567 xen-syms-2.0.3 continue_nonidle_task
45 0.0555 vmlinux-syms-2.6.10-xen0 tcp_send_ack
43 0.0530 xen-syms-2.0.3 FLT15
41 0.0506 vmlinux-syms-2.6.10-xen0 profile_hit
41 0.0506 xen-syms-2.0.3 __constant_c_and_count_memset
39 0.0481 bash (no symbols)
34 0.0419 vmlinux-syms-2.6.10-xen0 lock_sock
34 0.0419 xen-syms-2.0.3 FLT4
30 0.0370 oprofile (no symbols)
30 0.0370 vmlinux-syms-2.6.10-xen0 neigh_resolve_output
29 0.0358 vmlinux-syms-2.6.10-xen0 mark_offset_tsc
29 0.0358 xen-syms-2.0.3 FLT3
29 0.0358 xen-syms-2.0.3 ptwr_flush
25 0.0308 vmlinux-syms-2.6.10-xen0 ret_from_intr
25 0.0308 vmlinux-syms-2.6.10-xen0 task_rq_lock
24 0.0296 xen-syms-2.0.3 mask_and_ack_level_ioapic_irq
23 0.0284 vmlinux-syms-2.6.10-xen0 eth_header
23 0.0284 xen-syms-2.0.3 page_fault
23 0.0284 xen-syms-2.0.3 restore_all_guest
22 0.0271 xen-syms-2.0.3 timer_interrupt
21 0.0259 vmlinux-syms-2.6.10-xen0 activate_task
21 0.0259 vmlinux-syms-2.6.10-xen0 do_wp_page
21 0.0259 xen-syms-2.0.3 do_iopl
19 0.0234 vmlinux-syms-2.6.10-xen0 note_interrupt
16 0.0197 oprofiled odb_insert
16 0.0197 vmlinux-syms-2.6.10-xen0 finish_wait
13 0.0160 xen-syms-2.0.3 nvm86_3
11 0.0136 ld-2.3.2.so strcmp
11 0.0136 oprofiled pop_buffer_value
11 0.0136 vmlinux-syms-2.6.10-xen0 autoremove_wake_function
11 0.0136 vmlinux-syms-2.6.10-xen0 scheduler_tick
11 0.0136 vmlinux-syms-2.6.10-xen0 zap_pte_range
10 0.0123 xen-syms-2.0.3 process_softirqs
10 0.0123 xen-syms-2.0.3 update_dom_time
9 0.0111 vmlinux-syms-2.6.10-xen0 do_no_page
9 0.0111 vmlinux-syms-2.6.10-xen0 syscall_call
8 0.0099 oprofiled sfile_log_sample
8 0.0099 vmlinux-syms-2.6.10-xen0 copy_page_range
8 0.0099 xen-syms-2.0.3 get_page
7 0.0086 ld-2.3.2.so do_lookup_versioned
7 0.0086 libc-2.3.2.so __GI___mbrtowc
7 0.0086 vmlinux-syms-2.6.10-xen0 page_fault
7 0.0086 vmlinux-syms-2.6.10-xen0 resume_kernel
7 0.0086 xen-syms-2.0.3 smp_send_event_check_mask
6 0.0074 vmlinux-syms-2.6.10-xen0 raise_softirq_irqoff
6 0.0074 xen-syms-2.0.3 FLT1
6 0.0074 xen-syms-2.0.3 ret_from_hypercall
5 0.0062 ld-2.3.2.so _dl_lookup_versioned_symbol_internal
5 0.0062 libc-2.3.2.so __gconv_transform_utf8_internal
5 0.0062 oprofiled opd_process_samples
5 0.0062 oprofiled sfile_find
5 0.0062 vmlinux-syms-2.6.10-xen0 deactivate_task
5 0.0062 vmlinux-syms-2.6.10-xen0 update_wall_time
5 0.0062 xen-syms-2.0.3 IRQ0x00_interrupt
5 0.0062 xen-syms-2.0.3 do_IRQ
5 0.0062 xen-syms-2.0.3 do_update_va_mapping
5 0.0062 xen-syms-2.0.3 end_level_ioapic_irq
5 0.0062 xen-syms-2.0.3 evtchn_set_pending
5 0.0062 xen-syms-2.0.3 mod_l1_entry
4 0.0049 gawk (no symbols)
4 0.0049 oprofiled odb_open_count
4 0.0049 vmlinux-syms-2.6.10-xen0 _atomic_dec_and_lock
4 0.0049 vmlinux-syms-2.6.10-xen0 i8042_interrupt
4 0.0049 vmlinux-syms-2.6.10-xen0 link_path_walk
4 0.0049 xen-syms-2.0.3 ret_from_intr
4 0.0049 xen-syms-2.0.3 t_timer_fn
3 0.0037 ld-2.3.2.so fixup
3 0.0037 libc-2.3.2.so strcpy
3 0.0037 libc-2.3.2.so strlen
3 0.0037 vmlinux-syms-2.6.10-xen0 __queue_work
3 0.0037 vmlinux-syms-2.6.10-xen0 default_wake_function
3 0.0037 vmlinux-syms-2.6.10-xen0 do_anonymous_page
3 0.0037 vmlinux-syms-2.6.10-xen0 do_timer
3 0.0037 vmlinux-syms-2.6.10-xen0 run_timer_softirq
3 0.0037 xen-syms-2.0.3 __do_IRQ_guest
3 0.0037 xen-syms-2.0.3 create_bounce_frame
3 0.0037 xen-syms-2.0.3 domain_unblock
3 0.0037 xen-syms-2.0.3 smp_apic_timer_interrupt
2 0.0025 ld-2.3.2.so _dl_fini
2 0.0025 ld-2.3.2.so _dl_relocate_object_internal
2 0.0025 libc-2.3.2.so _IO_flush_all_lockp
2 0.0025 libc-2.3.2.so __GI___errno_location
2 0.0025 libc-2.3.2.so __GI__dl_mcount_wrapper_check
2 0.0025 libc-2.3.2.so ___fxstat64
2 0.0025 libc-2.3.2.so __i686.get_pc_thunk.bx
2 0.0025 libc-2.3.2.so _int_malloc
2 0.0025 libc-2.3.2.so getc
2 0.0025 libc-2.3.2.so memcpy
2 0.0025 libc-2.3.2.so memset
2 0.0025 oprofiled code_ctx_switch
2 0.0025 oprofiled find_kernel_image
2 0.0025 oprofiled get_file
2 0.0025 oprofiled sfile_get
2 0.0025 vmlinux-syms-2.6.10-xen0 __d_lookup
2 0.0025 vmlinux-syms-2.6.10-xen0 copy_mm
2 0.0025 vmlinux-syms-2.6.10-xen0 find_get_page
2 0.0025 vmlinux-syms-2.6.10-xen0 get_task_mm
2 0.0025 vmlinux-syms-2.6.10-xen0 page_add_anon_rmap
2 0.0025 vmlinux-syms-2.6.10-xen0 profile_tick
2 0.0025 vmlinux-syms-2.6.10-xen0 pte_alloc_map
2 0.0025 vmlinux-syms-2.6.10-xen0 queue_pt_switch
2 0.0025 vmlinux-syms-2.6.10-xen0 raise_softirq
2 0.0025 vmlinux-syms-2.6.10-xen0 update_wall_time_one_tick
2 0.0025 xen-syms-2.0.3 FLT16
2 0.0025 xen-syms-2.0.3 FLT19
2 0.0025 xen-syms-2.0.3 FLT27
2 0.0025 xen-syms-2.0.3 IRQ0x0f_interrupt
2 0.0025 xen-syms-2.0.3 __constant_memcpy
2 0.0025 xen-syms-2.0.3 apic_timer_interrupt
2 0.0025 xen-syms-2.0.3 common_interrupt
2 0.0025 xen-syms-2.0.3 do_mmu_update
2 0.0025 xen-syms-2.0.3 dom_timer_fn
2 0.0025 xen-syms-2.0.3 nmi_sanity_check
2 0.0025 xen-syms-2.0.3 p4_check_ctrs
2 0.0025 xen-syms-2.0.3 ptwr_do_page_fault
1 0.0012 cat (no symbols)
1 0.0012 ld-2.3.2.so _dl_catch_error_internal
1 0.0012 ld-2.3.2.so mempcpy
1 0.0012 libc-2.3.2.so _IO_new_do_write
1 0.0012 libc-2.3.2.so __GI___fork
1 0.0012 libc-2.3.2.so __GI___fxstat
1 0.0012 libc-2.3.2.so __GI___open64
1 0.0012 libc-2.3.2.so __GI___strtod_internal
1 0.0012 libc-2.3.2.so __GI_close
1 0.0012 libc-2.3.2.so __GI_dup2
1 0.0012 libc-2.3.2.so __GI_exit
1 0.0012 libc-2.3.2.so __GI_getaddrinfo
1 0.0012 libc-2.3.2.so __gen_tempname
1 0.0012 libc-2.3.2.so __textdomain
1 0.0012 libc-2.3.2.so find_module_idx
1 0.0012 libc-2.3.2.so init
1 0.0012 libc-2.3.2.so malloc_consolidate
1 0.0012 libc-2.3.2.so ptmalloc_init
1 0.0012 libc-2.3.2.so utf8_internal_loop
1 0.0012 libdl-2.3.2.so anonymous symbol from section .plt
1 0.0012 ls (no symbols)
1 0.0012 ntpd (no symbols)
1 0.0012 op_help new_event
1 0.0012 op_help op_get_line
1 0.0012 oprofiled create_dir
1 0.0012 oprofiled enough_remaining
1 0.0012 oprofiled odb_open
1 0.0012 oprofiled opd_put_sample
1 0.0012 oprofiled sfile_hash
1 0.0012 oprofiled sfile_put
1 0.0012 sendmail.sendmail (no symbols)
1 0.0012 sshd (no symbols)
1 0.0012 vmlinux-syms-2.6.10-xen0 __do_page_cache_readahead
1 0.0012 vmlinux-syms-2.6.10-xen0 __find_get_block_slow
1 0.0012 vmlinux-syms-2.6.10-xen0 __group_complete_signal
1 0.0012 vmlinux-syms-2.6.10-xen0 __lookup_tag
1 0.0012 vmlinux-syms-2.6.10-xen0 __mark_inode_dirty
1 0.0012 vmlinux-syms-2.6.10-xen0 __pagevec_lru_add_active
1 0.0012 vmlinux-syms-2.6.10-xen0 _flush_page_update_queue
1 0.0012 vmlinux-syms-2.6.10-xen0 add_wait_queue
1 0.0012 vmlinux-syms-2.6.10-xen0 as_add_arq_hash
1 0.0012 vmlinux-syms-2.6.10-xen0 bad_range
1 0.0012 vmlinux-syms-2.6.10-xen0 cache_alloc_refill
1 0.0012 vmlinux-syms-2.6.10-xen0 clear_inode
1 0.0012 vmlinux-syms-2.6.10-xen0 clear_page_tables
1 0.0012 vmlinux-syms-2.6.10-xen0 copy_from_user
1 0.0012 vmlinux-syms-2.6.10-xen0 datagram_poll
1 0.0012 vmlinux-syms-2.6.10-xen0 del_timer
1 0.0012 vmlinux-syms-2.6.10-xen0 dentry_open
1 0.0012 vmlinux-syms-2.6.10-xen0 do_cciss_intr
1 0.0012 vmlinux-syms-2.6.10-xen0 do_get_write_access
1 0.0012 vmlinux-syms-2.6.10-xen0 do_mmap_pgoff
1 0.0012 vmlinux-syms-2.6.10-xen0 do_munmap
1 0.0012 vmlinux-syms-2.6.10-xen0 drain_array_locked
1 0.0012 vmlinux-syms-2.6.10-xen0 ext3_get_block_handle
1 0.0012 vmlinux-syms-2.6.10-xen0 ext3_get_inode_block
1 0.0012 vmlinux-syms-2.6.10-xen0 filemap_nopage
1 0.0012 vmlinux-syms-2.6.10-xen0 find_vma
1 0.0012 vmlinux-syms-2.6.10-xen0 follow_mount
1 0.0012 vmlinux-syms-2.6.10-xen0 free_block
1 0.0012 vmlinux-syms-2.6.10-xen0 free_page_and_swap_cache
1 0.0012 vmlinux-syms-2.6.10-xen0 group_send_sig_info
1 0.0012 vmlinux-syms-2.6.10-xen0 handle_mm_fault
1 0.0012 vmlinux-syms-2.6.10-xen0 i8042_timer_func
1 0.0012 vmlinux-syms-2.6.10-xen0 journal_get_write_access
1 0.0012 vmlinux-syms-2.6.10-xen0 kmem_freepages
1 0.0012 vmlinux-syms-2.6.10-xen0 load_elf_binary
1 0.0012 vmlinux-syms-2.6.10-xen0 mpage_writepages
1 0.0012 vmlinux-syms-2.6.10-xen0 page_waitqueue
1 0.0012 vmlinux-syms-2.6.10-xen0 prepare_to_copy
1 0.0012 vmlinux-syms-2.6.10-xen0 prio_tree_insert
1 0.0012 vmlinux-syms-2.6.10-xen0 proc_lookup
1 0.0012 vmlinux-syms-2.6.10-xen0 radix_tree_insert
1 0.0012 vmlinux-syms-2.6.10-xen0 radix_tree_lookup
1 0.0012 vmlinux-syms-2.6.10-xen0 restore_all_enable_events
1 0.0012 vmlinux-syms-2.6.10-xen0 save_i387
1 0.0012 vmlinux-syms-2.6.10-xen0 schedule_tail
1 0.0012 vmlinux-syms-2.6.10-xen0 shrink_icache_memory
1 0.0012 vmlinux-syms-2.6.10-xen0 shrink_list
1 0.0012 vmlinux-syms-2.6.10-xen0 sys_brk
1 0.0012 vmlinux-syms-2.6.10-xen0 sys_fcntl64
1 0.0012 vmlinux-syms-2.6.10-xen0 syscall_exit_work
1 0.0012 vmlinux-syms-2.6.10-xen0 update_one_process
1 0.0012 vmlinux-syms-2.6.10-xen0 update_process_times
1 0.0012 vmlinux-syms-2.6.10-xen0 vfs_stat
1 0.0012 vmlinux-syms-2.6.10-xen0 vma_link
1 0.0012 vmlinux-syms-2.6.10-xen0 vma_merge
1 0.0012 xen-syms-2.0.3 FLT17
1 0.0012 xen-syms-2.0.3 do_extended_command
1 0.0012 xen-syms-2.0.3 do_page_fault
1 0.0012 xen-syms-2.0.3 event_check_interrupt
1 0.0012 xen-syms-2.0.3 get_page_from_l1e
1 0.0012 xen-syms-2.0.3 get_page_type
1 0.0012 xen-syms-2.0.3 put_page_type
[-- Attachment #3: instr_func_xen0.prof --]
[-- Type: application/octet-stream, Size: 12671 bytes --]
CPU: P4 / Xeon, speed 2799.36 MHz (estimated)
Counted INSTR_RETIRED events (retired instructions) with a unit mask of 0x01 (count non-bogus instructions which are not tagged) count 1000000
samples % app name symbol name
4646 22.0786 vmlinux-syms-2.6.10-xen0 __copy_user_intel
1715 8.1500 bcm5700 (no symbols)
1087 5.1656 vmlinux-syms-2.6.10-xen0 tcp_v4_rcv
706 3.3550 vmlinux-syms-2.6.10-xen0 tcp_rcv_established
652 3.0984 vmlinux-syms-2.6.10-xen0 netif_receive_skb
487 2.3143 vmlinux-syms-2.6.10-xen0 ip_rcv
389 1.8486 vmlinux-syms-2.6.10-xen0 nf_hook_slow
386 1.8343 vmlinux-syms-2.6.10-xen0 nf_iterate
352 1.6728 vmlinux-syms-2.6.10-xen0 rt_hash_code
318 1.5112 vmlinux-syms-2.6.10-xen0 kmem_cache_alloc
307 1.4589 vmlinux-syms-2.6.10-xen0 ip_local_deliver
303 1.4399 vmlinux-syms-2.6.10-xen0 schedule
302 1.4352 vmlinux-syms-2.6.10-xen0 ip_route_input
291 1.3829 vmlinux-syms-2.6.10-xen0 preempt_schedule
264 1.2546 vmlinux-syms-2.6.10-xen0 kfree
253 1.2023 vmlinux-syms-2.6.10-xen0 alloc_skb_from_cache
251 1.1928 vmlinux-syms-2.6.10-xen0 tcp_send_delayed_ack
240 1.1405 vmlinux-syms-2.6.10-xen0 tcp_copy_to_iovec
240 1.1405 vmlinux-syms-2.6.10-xen0 tcp_event_data_recv
202 0.9599 vmlinux-syms-2.6.10-xen0 tcp_recvmsg
196 0.9314 vmlinux-syms-2.6.10-xen0 memcpy_toiovec
188 0.8934 vmlinux-syms-2.6.10-xen0 eth_type_trans
187 0.8887 vmlinux-syms-2.6.10-xen0 skb_copy_datagram_iovec
178 0.8459 vmlinux-syms-2.6.10-xen0 local_bh_enable
168 0.7984 vmlinux-syms-2.6.10-xen0 __mod_timer
149 0.7081 vmlinux-syms-2.6.10-xen0 __kfree_skb
145 0.6891 vmlinux-syms-2.6.10-xen0 tcp_transmit_skb
144 0.6843 vmlinux-syms-2.6.10-xen0 tcp_prequeue_process
143 0.6796 xen-syms-2.0.3 do_multicall
142 0.6748 vmlinux-syms-2.6.10-xen0 kmem_cache_free
142 0.6748 xen-syms-2.0.3 copy_from_user
139 0.6606 vmlinux-syms-2.6.10-xen0 ip_rcv_finish
137 0.6510 xen-syms-2.0.3 __copy_from_user_ll
136 0.6463 vmlinux-syms-2.6.10-xen0 skb_release_data
136 0.6463 xen-syms-2.0.3 get_s_time
134 0.6368 vmlinux-syms-2.6.10-xen0 ip_queue_xmit
130 0.6178 vmlinux-syms-2.6.10-xen0 recalc_task_prio
125 0.5940 xen-syms-2.0.3 hypercall
121 0.5750 vmlinux-syms-2.6.10-xen0 __switch_to
109 0.5180 xen-syms-2.0.3 stext
105 0.4990 xen-syms-2.0.3 rem_ac_timer
92 0.4372 xen-syms-2.0.3 bvt_do_schedule
90 0.4277 vmlinux-syms-2.6.10-xen0 copy_to_user
89 0.4229 vmlinux-syms-2.6.10-xen0 __dev_alloc_skb
89 0.4229 vmlinux-syms-2.6.10-xen0 tcp_rcv_rtt_update
86 0.4087 vmlinux-syms-2.6.10-xen0 __tcp_select_window
86 0.4087 vmlinux-syms-2.6.10-xen0 sched_clock
78 0.3707 vmlinux-syms-2.6.10-xen0 dequeue_task
78 0.3707 vmlinux-syms-2.6.10-xen0 tcp_v4_do_rcv
78 0.3707 xen-syms-2.0.3 test_all_events
77 0.3659 vmlinux-syms-2.6.10-xen0 pfifo_fast_dequeue
77 0.3659 vmlinux-syms-2.6.10-xen0 qdisc_restart
76 0.3612 vmlinux-syms-2.6.10-xen0 ip_finish_output2
76 0.3612 vmlinux-syms-2.6.10-xen0 tcp_rcv_space_adjust
75 0.3564 vmlinux-syms-2.6.10-xen0 handle_IRQ_event
73 0.3469 vmlinux-syms-2.6.10-xen0 __copy_to_user_ll
73 0.3469 vmlinux-syms-2.6.10-xen0 ip_sabotage_out
73 0.3469 vmlinux-syms-2.6.10-xen0 mod_timer
73 0.3469 xen-syms-2.0.3 __copy_to_user_ll
72 0.3422 vmlinux-syms-2.6.10-xen0 sk_reset_timer
71 0.3374 vmlinux-syms-2.6.10-xen0 __do_IRQ
70 0.3327 xen-syms-2.0.3 __enter_scheduler
68 0.3231 xen-syms-2.0.3 do_dom0_op
67 0.3184 xen-syms-2.0.3 pirq_guest_unmask
65 0.3089 vmlinux-syms-2.6.10-xen0 timer_interrupt
64 0.3041 xen-syms-2.0.3 remove_entry
63 0.2994 vmlinux-syms-2.6.10-xen0 kfree_skbmem
60 0.2851 vmlinux-syms-2.6.10-xen0 dev_queue_xmit
58 0.2756 vmlinux-syms-2.6.10-xen0 alloc_skb
58 0.2756 vmlinux-syms-2.6.10-xen0 cleanup_rbuf
58 0.2756 vmlinux-syms-2.6.10-xen0 end_pirq
57 0.2709 vmlinux-syms-2.6.10-xen0 sock_aio_read
56 0.2661 vmlinux-syms-2.6.10-xen0 enqueue_task
53 0.2519 vmlinux-syms-2.6.10-xen0 __kmalloc
52 0.2471 xen-syms-2.0.3 unmask_IO_APIC_irq
51 0.2424 vmlinux-syms-2.6.10-xen0 sk_wait_data
50 0.2376 vmlinux-syms-2.6.10-xen0 xen_idle
49 0.2329 vmlinux-syms-2.6.10-xen0 do_sync_read
48 0.2281 vmlinux-syms-2.6.10-xen0 __do_softirq
48 0.2281 vmlinux-syms-2.6.10-xen0 __wake_up
48 0.2281 vmlinux-syms-2.6.10-xen0 effective_prio
45 0.2138 vmlinux-syms-2.6.10-xen0 csum_partial
45 0.2138 xen-syms-2.0.3 do_stack_switch
44 0.2091 vmlinux-syms-2.6.10-xen0 ip_sabotage_in
43 0.2043 vmlinux-syms-2.6.10-xen0 prepare_to_wait
43 0.2043 vmlinux-syms-2.6.10-xen0 try_to_wake_up
43 0.2043 xen-syms-2.0.3 __constant_memcpy
42 0.1996 vmlinux-syms-2.6.10-xen0 evtchn_do_upcall
42 0.1996 vmlinux-syms-2.6.10-xen0 net_tx_action
42 0.1996 vmlinux-syms-2.6.10-xen0 schedule_timeout
41 0.1948 xen-syms-2.0.3 add_entry
39 0.1853 vmlinux-syms-2.6.10-xen0 tcp_v4_send_check
39 0.1853 vmlinux-syms-2.6.10-xen0 vfs_read
37 0.1758 vmlinux-syms-2.6.10-xen0 hypervisor_callback
37 0.1758 vmlinux-syms-2.6.10-xen0 net_rx_action
36 0.1711 vmlinux-syms-2.6.10-xen0 finish_task_switch
36 0.1711 vmlinux-syms-2.6.10-xen0 next_timer_interrupt
35 0.1663 xen-syms-2.0.3 copy_to_user
35 0.1663 xen-syms-2.0.3 switch_to
33 0.1568 vmlinux-syms-2.6.10-xen0 internal_add_timer
32 0.1521 xen-syms-2.0.3 do_physdev_op
30 0.1426 vmlinux-syms-2.6.10-xen0 dev_queue_xmit_nit
30 0.1426 vmlinux-syms-2.6.10-xen0 scrit
29 0.1378 vmlinux-syms-2.6.10-xen0 MULTICALL_flush_page_update_queue
29 0.1378 vmlinux-syms-2.6.10-xen0 autoremove_wake_function
29 0.1378 vmlinux-syms-2.6.10-xen0 sock_common_recvmsg
29 0.1378 vmlinux-syms-2.6.10-xen0 system_call
28 0.1331 vmlinux-syms-2.6.10-xen0 end_dynirq
28 0.1331 vmlinux-syms-2.6.10-xen0 irq_exit
27 0.1283 xen-syms-2.0.3 reprogram_ac_timer
26 0.1236 vmlinux-syms-2.6.10-xen0 dst_output
26 0.1236 xen-syms-2.0.3 restore_all_vm86
25 0.1188 vmlinux-syms-2.6.10-xen0 __wake_up_common
25 0.1188 vmlinux-syms-2.6.10-xen0 profile_hit
24 0.1141 vmlinux-syms-2.6.10-xen0 set_timeout_timer
23 0.1093 xen-syms-2.0.3 FLT2
23 0.1093 xen-syms-2.0.3 ac_timer_softirq_action
22 0.1045 vmlinux-syms-2.6.10-xen0 do_softirq
22 0.1045 xen-syms-2.0.3 __constant_c_and_count_memset
21 0.0998 vmlinux-syms-2.6.10-xen0 lock_sock
20 0.0950 vmlinux-syms-2.6.10-xen0 ack_pirq
20 0.0950 xen-syms-2.0.3 do_sched_op
19 0.0903 vmlinux-syms-2.6.10-xen0 fget_light
19 0.0903 vmlinux-syms-2.6.10-xen0 tcp_send_ack
18 0.0855 vmlinux-syms-2.6.10-xen0 __get_time_values_from_xen
18 0.0855 vmlinux-syms-2.6.10-xen0 pfifo_fast_enqueue
18 0.0855 xen-syms-2.0.3 do_iopl
17 0.0808 vmlinux-syms-2.6.10-xen0 do_IRQ
17 0.0808 vmlinux-syms-2.6.10-xen0 note_interrupt
17 0.0808 vmlinux-syms-2.6.10-xen0 sys_read
17 0.0808 xen-syms-2.0.3 FLT3
17 0.0808 xen-syms-2.0.3 ptwr_flush
16 0.0760 bash (no symbols)
16 0.0760 vmlinux-syms-2.6.10-xen0 dnotify_parent
14 0.0665 vmlinux-syms-2.6.10-xen0 ack_dynirq
14 0.0665 vmlinux-syms-2.6.10-xen0 sock_wfree
14 0.0665 vmlinux-syms-2.6.10-xen0 task_rq_lock
14 0.0665 xen-syms-2.0.3 do_softirq
13 0.0618 vmlinux-syms-2.6.10-xen0 ip_finish_output
12 0.0570 vmlinux-syms-2.6.10-xen0 get_offset_tsc
12 0.0570 vmlinux-syms-2.6.10-xen0 neigh_resolve_output
11 0.0523 libc-2.3.2.so __GI___mbrtowc
11 0.0523 libc-2.3.2.so __gconv_transform_utf8_internal
11 0.0523 vmlinux-syms-2.6.10-xen0 mark_offset_tsc
11 0.0523 vmlinux-syms-2.6.10-xen0 restore_all
10 0.0475 rcv main
10 0.0475 vmlinux-syms-2.6.10-xen0 syscall_exit
10 0.0475 xen-syms-2.0.3 FLT14
9 0.0428 vmlinux-syms-2.6.10-xen0 eth_header
9 0.0428 vmlinux-syms-2.6.10-xen0 release_sock
8 0.0380 vmlinux-syms-2.6.10-xen0 deactivate_task
8 0.0380 vmlinux-syms-2.6.10-xen0 ip_output
7 0.0333 rcv __read
7 0.0333 vmlinux-syms-2.6.10-xen0 activate_task
7 0.0333 xen-syms-2.0.3 continue_nonidle_task
6 0.0285 libc-2.3.2.so utf8_internal_loop
6 0.0285 vmlinux-syms-2.6.10-xen0 cpu_idle
6 0.0285 vmlinux-syms-2.6.10-xen0 default_wake_function
6 0.0285 vmlinux-syms-2.6.10-xen0 zap_pte_range
5 0.0238 oprofile (no symbols)
5 0.0238 xen-syms-2.0.3 do_set_timer_op
4 0.0190 vmlinux-syms-2.6.10-xen0 radix_tree_lookup
3 0.0143 ld-2.3.2.so strcmp
3 0.0143 oprofiled pop_buffer_value
3 0.0143 vmlinux-syms-2.6.10-xen0 do_no_page
3 0.0143 xen-syms-2.0.3 FLT4
3 0.0143 xen-syms-2.0.3 get_page
3 0.0143 xen-syms-2.0.3 restore_all_guest
2 0.0095 ld-2.3.2.so _dl_lookup_versioned_symbol_internal
2 0.0095 ld-2.3.2.so do_lookup_versioned
2 0.0095 libc-2.3.2.so __GI__dl_mcount_wrapper_check
2 0.0095 libc-2.3.2.so __i686.get_pc_thunk.bx
2 0.0095 oprofiled get_file
2 0.0095 oprofiled odb_insert
2 0.0095 oprofiled odb_open_count
2 0.0095 oprofiled opd_process_samples
2 0.0095 oprofiled sfile_log_sample
2 0.0095 vmlinux-syms-2.6.10-xen0 clear_page_tables
2 0.0095 vmlinux-syms-2.6.10-xen0 copy_page_range
2 0.0095 vmlinux-syms-2.6.10-xen0 raise_softirq_irqoff
2 0.0095 xen-syms-2.0.3 FLT1
2 0.0095 xen-syms-2.0.3 FLT15
2 0.0095 xen-syms-2.0.3 process_softirqs
1 0.0048 grep (no symbols)
1 0.0048 ld-2.3.2.so _dl_lookup_symbol_internal
1 0.0048 ld-2.3.2.so _dl_relocate_object_internal
1 0.0048 ld-2.3.2.so fixup
1 0.0048 libc-2.3.2.so _IO_str_init_static_internal
1 0.0048 libc-2.3.2.so _IO_vfscanf_internal
1 0.0048 libc-2.3.2.so __GI___strtoll_internal
1 0.0048 libc-2.3.2.so __GI_getenv
1 0.0048 libc-2.3.2.so __textdomain
1 0.0048 libc-2.3.2.so _int_free
1 0.0048 libc-2.3.2.so strlen
1 0.0048 op_help op_get_line
1 0.0048 oprofiled sfile_get
1 0.0048 oprofiled sfile_match
1 0.0048 vmlinux-syms-2.6.10-xen0 __do_page_cache_readahead
1 0.0048 vmlinux-syms-2.6.10-xen0 bio_alloc
1 0.0048 vmlinux-syms-2.6.10-xen0 buffered_rmqueue
1 0.0048 vmlinux-syms-2.6.10-xen0 convert_fxsr_to_user
1 0.0048 vmlinux-syms-2.6.10-xen0 do_anonymous_page
1 0.0048 vmlinux-syms-2.6.10-xen0 do_page_fault
1 0.0048 vmlinux-syms-2.6.10-xen0 filemap_nopage
1 0.0048 vmlinux-syms-2.6.10-xen0 force_evtchn_callback
1 0.0048 vmlinux-syms-2.6.10-xen0 free_page_and_swap_cache
1 0.0048 vmlinux-syms-2.6.10-xen0 free_pages_bulk
1 0.0048 vmlinux-syms-2.6.10-xen0 journal_stop
1 0.0048 vmlinux-syms-2.6.10-xen0 link_path_walk
1 0.0048 vmlinux-syms-2.6.10-xen0 page_fault
1 0.0048 vmlinux-syms-2.6.10-xen0 resume_kernel
1 0.0048 xen-syms-2.0.3 create_bounce_frame
1 0.0048 xen-syms-2.0.3 mod_l1_entry
1 0.0048 xen-syms-2.0.3 put_page_from_l1e
[-- Attachment #4: dtlb_3nic.prof --]
[-- Type: application/octet-stream, Size: 10120 bytes --]
CPU: P4 / Xeon, speed 2392.12 MHz (estimated)
Counted PAGE_WALK_TYPE events (page walks by the page miss handler) with a unit mask of 0x01 (page walk for data TLB miss) count 100000
samples % image name app name symbol name
217 9.4801 e1000.ko e1000 e1000_intr
176 7.6889 e1000.ko e1000 e1000_clean_rx_irq
133 5.8104 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 alloc_skb_from_cache
102 4.4561 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_rcv
84 3.6697 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 free_block
80 3.4950 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kfree
76 3.3202 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_prequeue_process
69 3.0144 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_rcv_established
62 2.7086 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 skb_release_data
51 2.2280 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 eth_type_trans
49 2.1407 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 process_backlog
45 1.9659 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __copy_user_intel
42 1.8349 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 schedule
40 1.7475 e1000.ko e1000 e1000_clean_tx_irq
39 1.7038 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 finish_task_switch
35 1.5291 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_v4_do_rcv
32 1.3980 e1000.ko e1000 e1000_xmit_frame
31 1.3543 e1000.ko e1000 e1000_alloc_rx_buffers
29 1.2669 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 MULTICALL_flush_page_update_queue
29 1.2669 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_recvmsg
28 1.2232 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ack_pirq
26 1.1359 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 skbuff_ctor
25 1.0922 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __switch_to
25 1.0922 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 add_timer_randomness
24 1.0485 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kmem_cache_free
23 1.0048 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_v4_rcv
21 0.9174 e1000.ko e1000 e1000_rx_checksum
21 0.9174 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 netif_receive_skb
20 0.8737 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 netif_rx
19 0.8301 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __kmalloc
19 0.8301 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 dnotify_parent
18 0.7864 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kmem_cache_alloc
18 0.7864 xen xen mask_and_ack_level_ioapic_irq
17 0.7427 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __copy_to_user_ll
17 0.7427 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __do_IRQ
17 0.7427 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __kfree_skb
17 0.7427 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __mod_timer
17 0.7427 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 nf_iterate
17 0.7427 xen xen __copy_to_user_ll
17 0.7427 xen xen unmask_IO_APIC_irq
16 0.6990 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __release_sock
16 0.6990 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 cache_alloc_refill
16 0.6990 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_route_input
16 0.6990 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 vfs_read
15 0.6553 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 net_tx_action
14 0.6116 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 evtchn_do_upcall
14 0.6116 xen xen copy_from_user
14 0.6116 xen xen hypercall
12 0.5242 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 fget_light
12 0.5242 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 handle_IRQ_event
12 0.5242 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_sabotage_out
12 0.5242 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 queue_pt_switch
12 0.5242 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 try_to_wake_up
11 0.4806 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __dev_alloc_skb
11 0.4806 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __wake_up_common
10 0.4369 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __do_softirq
10 0.4369 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_local_deliver
10 0.4369 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 system_call
9 0.3932 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 alloc_skb
9 0.3932 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 free_pages_bulk
9 0.3932 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sk_wait_data
9 0.3932 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_collapse
8 0.3495 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 dev_queue_xmit
8 0.3495 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_queue_xmit
8 0.3495 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_event_data_recv
8 0.3495 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_transmit_skb
8 0.3495 xen xen FLT2
7 0.3058 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 enqueue_task
6 0.2621 libc-2.3.2.so libc-2.3.2.so __GI_read
6 0.2621 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 add_interrupt_randomness
6 0.2621 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 buffered_rmqueue
6 0.2621 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sock_common_recvmsg
6 0.2621 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_v4_send_check
6 0.2621 xen xen IRQ0x18_interrupt
5 0.2184 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 alloc_slabmgmt
5 0.2184 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 do_IRQ
5 0.2184 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_output
5 0.2184 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 pfifo_fast_dequeue
5 0.2184 xen xen do_extended_command
4 0.1747 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 internal_add_timer
4 0.1747 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kfree_skbmem
4 0.1747 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 local_bh_enable
4 0.1747 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 pfifo_fast_enqueue
4 0.1747 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sock_wfree
4 0.1747 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sys_read
3 0.1311 a.out a.out main
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 cache_flusharray
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 hypervisor_callback
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kmem_freepages
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 lock_sock
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 net_rx_action
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 scrit
3 0.1311 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_copy_to_iovec
3 0.1311 xen xen IRQ0x16_interrupt
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __alloc_pages
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __rmqueue
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 dequeue_task
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 end_pirq
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 free_hot_cold_page
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_finish_output2
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 task_rq_lock
2 0.0874 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_rcv_space_adjust
2 0.0874 xen xen IRQ0x17_interrupt
1 0.0437 bash bash (no symbols)
1 0.0437 ld-2.3.2.so ld-2.3.2.so strcmp
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 activate_task
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 cleanup_rbuf
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 do_softirq
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_rcv_finish
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_sabotage_in
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 irq_exit
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 memcpy_toiovec
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 neigh_resolve_output
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 note_interrupt
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 profile_hit
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sk_stream_rfree
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 skb_copy_bits
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 syscall_exit
1 0.0437 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 zap_pte_range
[-- Attachment #5: itlb_3nic.prof --]
[-- Type: application/octet-stream, Size: 6900 bytes --]
CPU: P4 / Xeon, speed 2392.12 MHz (estimated)
Counted PAGE_WALK_TYPE events (page walks by the page miss handler) with a unit mask of 0x02 (page walk for instruction TLB miss) count 100000
samples % image name app name symbol name
25 6.7568 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __switch_to
20 5.4054 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 schedule
19 5.1351 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_recvmsg
14 3.7838 a.out a.out main
13 3.5135 e1000.ko e1000 e1000_intr
12 3.2432 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 evtchn_do_upcall
12 3.2432 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 finish_task_switch
12 3.2432 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 skb_copy_datagram_iovec
11 2.9730 e1000.ko e1000 e1000_clean_tx_irq
10 2.7027 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_prequeue_process
9 2.4324 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sk_wait_data
9 2.4324 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_rcv_established
8 2.1622 libc-2.3.2.so libc-2.3.2.so __GI_read
8 2.1622 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 MULTICALL_flush_page_update_queue
8 2.1622 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __kfree_skb
8 2.1622 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 skb_release_data
7 1.8919 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 nf_hook_slow
7 1.8919 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_copy_to_iovec
6 1.6216 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 hypervisor_callback
6 1.6216 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_queue_xmit
6 1.6216 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 nf_iterate
6 1.6216 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_transmit_skb
5 1.3514 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 local_bh_enable
5 1.3514 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sock_aio_read
5 1.3514 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_v4_rcv
5 1.3514 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 vfs_read
4 1.0811 e1000.ko e1000 e1000_clean_rx_irq
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ack_pirq
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 dnotify_parent
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 do_sync_read
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_route_input
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 memcpy_toiovec
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 pfifo_fast_enqueue
4 1.0811 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_v4_send_check
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 dev_queue_xmit
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_rcv
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_rcv_finish
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 net_tx_action
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 netif_receive_skb
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 pfifo_fast_dequeue
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 schedule_timeout
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sock_common_recvmsg
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 sys_read
3 0.8108 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_send_delayed_ack
2 0.5405 e1000.ko e1000 e1000_xmit_frame
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __copy_to_user_ll
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __copy_user_intel
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __do_IRQ
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __mod_timer
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __tcp_grow_window
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 add_interrupt_randomness
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 alloc_skb
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 dst_output
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 handle_IRQ_event
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_local_deliver
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_output
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_sabotage_out
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 need_resched
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 profile_hit
2 0.5405 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 scrit
1 0.2703 e1000.ko e1000 e1000_alloc_rx_buffers
1 0.2703 ld-2.3.2.so ld-2.3.2.so fixup
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __tcp_ack_snd_check
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __tcp_select_window
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 __wake_up_common
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 copy_to_user
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 do_softirq
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 eth_type_trans
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 free_block
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 internal_add_timer
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_finish_output
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 ip_finish_output2
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 irq_exit
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kfree
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 kmem_cache_alloc
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 lock_sock
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 neigh_resolve_output
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 qdisc_restart
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 restore_all
1 0.2703 vmlinux-2.6.10-xen0 vmlinux-2.6.10-xen0 tcp_ack
[-- Attachment #6: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: More network tests with xenoprofile this time
2005-06-17 19:39 Santos, Jose Renato G
@ 2005-06-22 2:22 ` Andrew Theurer
0 siblings, 0 replies; 12+ messages in thread
From: Andrew Theurer @ 2005-06-22 2:22 UTC (permalink / raw)
To: Santos, Jose Renato G
Cc: Ian Pratt, xen-devel, Turner, Yoshio, Aravind Menon,
G John Janakiraman, William Cohen
FWIW, I took a look at find_domain_by_id() and swapped out the rw lock
with a spin lock for domlist_lock. The time spent in the this function
was reduced by 18%. This function was certainly not the "hottest" one I
recorded while running netperf, but every little bit helps I suppose.
Below are before/after xenoprofile snippets:
before:
547036 6.32 xen-unstable-syms mask_and_ack_level_ioapic_irq
510448 5.90 xen-unstable-syms end_level_ioapic_irq
463386 5.35 vmlinux-2.6.11-xen0-up net_tx_action
371072 4.29 tg3.ko tg3_interrupt
261341 3.02 xen-unstable-syms find_domain_by_id
237601 2.75 xen-unstable-syms hypercall
228649 2.64 vmlinux-2.6.11-xen0-up nf_iterate
215634 2.49 xen-unstable-syms do_update_va_mapping
214077 2.47 vmlinux-2.6.11-xen0-up net_rx_action
after:
549276 6.35 xen-unstable-syms mask_and_ack_level_ioapic_irq
511693 5.91 xen-unstable-syms end_level_ioapic_irq
466873 5.39 vmlinux-2.6.11-xen0-up net_tx_action
375702 4.34 tg3.ko tg3_interrupt
239219 2.76 xen-unstable-syms hypercall
230641 2.67 vmlinux-2.6.11-xen0-up nf_iterate
220480 2.55 xen-unstable-syms do_update_va_mapping
217472 2.51 tg3.ko tg3_rx
217029 2.51 vmlinux-2.6.11-xen0-up net_rx_action
214271 2.48 xen-unstable-syms find_domain_by_id
Has anyone thought about using read-copy-update in Xen?
I plan on looking at the two irq functions next.
-Andrew
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: More network tests with xenoprofile this time
@ 2005-05-31 22:48 Ian Pratt
2005-06-01 20:03 ` Jon Mason
0 siblings, 1 reply; 12+ messages in thread
From: Ian Pratt @ 2005-05-31 22:48 UTC (permalink / raw)
To: Andrew Theurer, xen-devel
> I have cpu util from polling xc_domain_get_cpu_usage() for
> both domains, which is (an exerpt from the whole run, in 3
> second intervals):
>
> cpu0: [100.4] d0-0[100.4]
> cpu2: [045.1] d1-0[045.1]
OK, so you're confident idle time would be reported OK if there was any.
> > Is the Ethernet NIC sharing an interrupt with the USB
> controller per
> > chance?
>
> Not as far as I can tell:
>
> CPU0
> 11: 6764395 Phys-irq ohci_hcd
> 24: 6037311 Phys-irq eth0
> 260: 1688517 Dynamic-irq vif1.0
Anyone care to suggest hy ohci_hcd is taking so many interrupts? Looks
very fishy to me. I take it you're not using a USB Ethernet NIC? :-)
What happens if you boot 'nousb' ?
> > Seeing find_domain_by_id and copy_from_user so high up the list is
> > pretty surprising.
>
> Yes.
Definitely worth looking in to...
Sorry for all the questions. This work is much appreciated.
Ian
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: More network tests with xenoprofile this time
2005-05-31 22:48 Ian Pratt
@ 2005-06-01 20:03 ` Jon Mason
2005-06-01 20:21 ` Andrew Theurer
0 siblings, 1 reply; 12+ messages in thread
From: Jon Mason @ 2005-06-01 20:03 UTC (permalink / raw)
To: xen-devel; +Cc: Ian Pratt, Andrew Theurer
On Tuesday 31 May 2005 05:48 pm, Ian Pratt wrote:
> > I have cpu util from polling xc_domain_get_cpu_usage() for
> > both domains, which is (an exerpt from the whole run, in 3
> > second intervals):
> >
> > cpu0: [100.4] d0-0[100.4]
> > cpu2: [045.1] d1-0[045.1]
>
> OK, so you're confident idle time would be reported OK if there was any.
>
> > > Is the Ethernet NIC sharing an interrupt with the USB
> >
> > controller per
> >
> > > chance?
> >
> > Not as far as I can tell:
> >
> > CPU0
> > 11: 6764395 Phys-irq ohci_hcd
> > 24: 6037311 Phys-irq eth0
> > 260: 1688517 Dynamic-irq vif1.0
>
> Anyone care to suggest hy ohci_hcd is taking so many interrupts? Looks
> very fishy to me. I take it you're not using a USB Ethernet NIC? :-)
The bladecenters have a shared USB connected to all the blades. I would
imagine it is the keyboard/mouse or USB CDROM connected to this bus that is
generating all of these interrupts.
> What happens if you boot 'nousb' ?
This shouldn't hurt anything, unless Andrew needs access to kdb or cdrom.
Thanks,
Jon
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: More network tests with xenoprofile this time
2005-06-01 20:03 ` Jon Mason
@ 2005-06-01 20:21 ` Andrew Theurer
2005-06-02 14:53 ` Andrew Theurer
0 siblings, 1 reply; 12+ messages in thread
From: Andrew Theurer @ 2005-06-01 20:21 UTC (permalink / raw)
To: Jon Mason, xen-devel; +Cc: Ian Pratt
On Wednesday 01 June 2005 15:03, Jon Mason wrote:
> On Tuesday 31 May 2005 05:48 pm, Ian Pratt wrote:
> > > I have cpu util from polling xc_domain_get_cpu_usage() for
> > > both domains, which is (an exerpt from the whole run, in 3
> > > second intervals):
> > >
> > > cpu0: [100.4] d0-0[100.4]
> > > cpu2: [045.1] d1-0[045.1]
> >
> > OK, so you're confident idle time would be reported OK if there was
> > any.
> >
> > > > Is the Ethernet NIC sharing an interrupt with the USB
> > >
> > > controller per
> > >
> > > > chance?
> > >
> > > Not as far as I can tell:
> > >
> > > CPU0
> > > 11: 6764395 Phys-irq ohci_hcd
> > > 24: 6037311 Phys-irq eth0
> > > 260: 1688517 Dynamic-irq vif1.0
> >
> > Anyone care to suggest hy ohci_hcd is taking so many interrupts?
> > Looks very fishy to me. I take it you're not using a USB Ethernet
> > NIC? :-)
>
> The bladecenters have a shared USB connected to all the blades. I
> would imagine it is the keyboard/mouse or USB CDROM connected to this
> bus that is generating all of these interrupts.
>
> > What happens if you boot 'nousb' ?
>
> This shouldn't hurt anything, unless Andrew needs access to kdb or
> cdrom.
This is on a x336 system, P4 Xeon, not much USB really needed. I did
not see any difference in performace or the profile with nousb.
I also tried disbaling the locks in find_domain_by_id and saw no
difference. I'm curious to see how things differ with dom0 on CPU-0
HT-0 and dom1 on CPU-0 HT-1. I will probably try that next.
FWIW, baremetal linux used about 33% of one cpu to drive the same
throughput. int's/sec was 41k/sec for baremetal vs 59k/sec for dom0.
I don't have the breakdown of int/sec per interrupt number yet.
-Andrew
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: More network tests with xenoprofile this time
2005-06-01 20:21 ` Andrew Theurer
@ 2005-06-02 14:53 ` Andrew Theurer
0 siblings, 0 replies; 12+ messages in thread
From: Andrew Theurer @ 2005-06-02 14:53 UTC (permalink / raw)
To: Jon Mason, xen-devel; +Cc: Ian Pratt
On Wednesday 01 June 2005 15:21, Andrew Theurer wrote:
> On Wednesday 01 June 2005 15:03, Jon Mason wrote:
> > On Tuesday 31 May 2005 05:48 pm, Ian Pratt wrote:
> > > > I have cpu util from polling xc_domain_get_cpu_usage() for
> > > > both domains, which is (an exerpt from the whole run, in 3
> > > > second intervals):
> > > >
> > > > cpu0: [100.4] d0-0[100.4]
> > > > cpu2: [045.1] d1-0[045.1]
> > >
> > > OK, so you're confident idle time would be reported OK if there
> > > was any.
> > >
> > > > > Is the Ethernet NIC sharing an interrupt with the USB
> > > >
> > > > controller per
> > > >
> > > > > chance?
> > > >
> > > > Not as far as I can tell:
> > > >
> > > > CPU0
> > > > 11: 6764395 Phys-irq ohci_hcd
> > > > 24: 6037311 Phys-irq eth0
> > > > 260: 1688517 Dynamic-irq vif1.0
> > >
> > > Anyone care to suggest hy ohci_hcd is taking so many interrupts?
> > > Looks very fishy to me. I take it you're not using a USB Ethernet
> > > NIC? :-)
> >
> > The bladecenters have a shared USB connected to all the blades. I
> > would imagine it is the keyboard/mouse or USB CDROM connected to
> > this bus that is generating all of these interrupts.
> >
> > > What happens if you boot 'nousb' ?
> >
> > This shouldn't hurt anything, unless Andrew needs access to kdb or
> > cdrom.
>
> This is on a x336 system, P4 Xeon, not much USB really needed. I did
> not see any difference in performace or the profile with nousb.
>
> I also tried disbaling the locks in find_domain_by_id and saw no
> difference. I'm curious to see how things differ with dom0 on CPU-0
> HT-0 and dom1 on CPU-0 HT-1. I will probably try that next.
>
> FWIW, baremetal linux used about 33% of one cpu to drive the same
> throughput. int's/sec was 41k/sec for baremetal vs 59k/sec for dom0.
> I don't have the breakdown of int/sec per interrupt number yet.
Wanted to follow up, one correction, I did not have usb disabled
properly, and with properly removing usb, there is a slight reduction
in irq handling overhead as a result:
542129 6.2205 xen-unstable-syms mask_and_ack_level_ioapic_irq
506060 5.8067 xen-unstable-syms end_level_ioapic_irq
475786 5.4593 vmlinux-2.6.11-xen0-up net_tx_action
376309 4.3179 vmlinux-2.6.11-xen0-up tg3_interrupt
263008 3.0178 xen-unstable-syms find_domain_by_id
239789 2.7514 xen-unstable-syms hypercall
224547 2.5765 vmlinux-2.6.11-xen0-up nf_iterate
...vs about 8-9% each for the top two functions before. The interrupt
rate for the tg3 adapter is very high still, about 24k/sec. At that
rate it does not appear to have any interrupt coalescing going on, so I
am going to look into that.
-Andrew
^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: More network tests with xenoprofile this time
@ 2005-05-31 22:16 Ian Pratt
2005-05-31 22:38 ` Andrew Theurer
0 siblings, 1 reply; 12+ messages in thread
From: Ian Pratt @ 2005-05-31 22:16 UTC (permalink / raw)
To: Andrew Theurer, xen-devel
> I had a chance to run a couple of the netperf tests with
> xenoprofile. I am still having some trouble with
> multi-domain profiles (probably user error), but I have been
> able to profile dom0 while running 2 types of tests. I was
> surprised to see as much as 50% cpu in hypervisor on these tests:
>
> netperf tcp_stream 16k msg size, dom1 -> dom2 dom0 is on
> cpu0, HT thread 0, dom1 is on cpu1, HT thread 0,
> dom2 is on cpu1, HT thread 1.
Let's ignore the domU <-> domU results for the moment as we know about
the problem with lack of batching in this scenario. Let's dig into the
dom1 -> external.
First off, are these figures just for CPU 0 HT 0? i.e. just dom0 so we
don't see where time goes in the domU? How is idle time on the CPU
reported?
Spending 18% of the time handling interrupts in Xen is surprisingy (at
least to me).
What interrupt rate are you observing? What are the default tg3
interrupt coallescing settings? What interrupt rate do you get on
native? Also, what hypercall rate are you seeing?
(It would be good to put this in context of the rx/tx packet rates).
Is the Ethernet NIC sharing an interrupt with the USB controller per
chance?
Seeing find_domain_by_id and copy_from_user so high up the list is
pretty surprising.
Cheers,
Ian
> netperf tcp_stream 16k msg size, dom1 -> external host dom0
> is on cpu0, HT thread 0, dom1 is on cpu1, HT thread 1.
>
>
> Throughput is ~940 Mbps, wire speed.
>
> xenoprofile opreport:
>
> 4244562 49.9375 xen-unstable-syms
> 4110594 48.3614 vmlinux-2.6.11-xen0-up
> 132643 1.5606 oprofiled
> 4212 0.0496 libc-2.3.3.so
> 2892 0.0340 libpython2.3.so.1.0
>
> xenoprofile opreport -l:
>
> 828587 9.75 xen-unstable-syms end_level_ioapic_irq
> 712035 8.38 xen-unstable-syms mask_and_ack_level_ioapic_irq
> 370265 4.36 vmlinux-2.6.11-xen0-up net_tx_action
> 323797 3.81 vmlinux-2.6.11-xen0-up ohci_irq
> 282005 3.32 vmlinux-2.6.11-xen0-up tg3_interrupt
> 273161 3.21 xen-unstable-syms find_domain_by_id
> 234726 2.76 xen-unstable-syms hypercall
> 206693 2.43 xen-unstable-syms do_update_va_mapping
> 203758 2.40 xen-unstable-syms __copy_from_user_ll
> 201665 2.37 xen-unstable-syms do_mmuext_op
> 195020 2.29 vmlinux-2.6.11-xen0-up nf_iterate
> 184295 2.17 vmlinux-2.6.11-xen0-up nf_hook_slow
> 172110 2.02 vmlinux-2.6.11-xen0-up tg3_rx
> 164337 1.93 vmlinux-2.6.11-xen0-up net_rx_action
> 141999 1.67 xen-unstable-syms do_mmu_update
> 139120 1.64 vmlinux-2.6.11-xen0-up fdb_insert
> 122483 1.44 xen-unstable-syms mod_l1_entry
> 122017 1.44 xen-unstable-syms put_page_from_l1e
> 111159 1.31 xen-unstable-syms get_page_from_l1e
> 109921 1.29 xen-unstable-syms do_IRQ
> 99847 1.17 vmlinux-2.6.11-xen0-up br_handle_frame
> 99709 1.17 xen-unstable-syms get_page_type
> 93613 1.10 vmlinux-2.6.11-xen0-up kfree
> 90885 1.07 vmlinux-2.6.11-xen0-up end_pirq
>
>
> -Andrew
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: More network tests with xenoprofile this time
2005-05-31 22:16 Ian Pratt
@ 2005-05-31 22:38 ` Andrew Theurer
0 siblings, 0 replies; 12+ messages in thread
From: Andrew Theurer @ 2005-05-31 22:38 UTC (permalink / raw)
To: Ian Pratt, xen-devel
On Tuesday 31 May 2005 17:16, Ian Pratt wrote:
> > I had a chance to run a couple of the netperf tests with
> > xenoprofile. I am still having some trouble with
> > multi-domain profiles (probably user error), but I have been
> > able to profile dom0 while running 2 types of tests. I was
> > surprised to see as much as 50% cpu in hypervisor on these tests:
> >
> > netperf tcp_stream 16k msg size, dom1 -> dom2 dom0 is on
> > cpu0, HT thread 0, dom1 is on cpu1, HT thread 0,
> > dom2 is on cpu1, HT thread 1.
>
> Let's ignore the domU <-> domU results for the moment as we know
> about the problem with lack of batching in this scenario. Let's dig
> into the dom1 -> external.
>
> First off, are these figures just for CPU 0 HT 0? i.e. just dom0 so
> we don't see where time goes in the domU? How is idle time on the CPU
> reported?
Yes, this is just for CPU 0 HT 0. DomU is pinned to its own cpu, which
is CPU 1 HT 0.
I have cpu util from polling xc_domain_get_cpu_usage() for both domains,
which is (an exerpt from the whole run, in 3 second intervals):
cpu0: [100.4] d0-0[100.4]
cpu2: [045.1] d1-0[045.1]
cpu0: [100.0] d0-0[100.0]
cpu2: [045.1] d1-0[045.1]
cpu0: [099.6] d0-0[099.6]
cpu2: [045.1] d1-0[045.1]
cpu0: [101.3] d0-0[101.3]
cpu2: [045.3] d1-0[045.3]
cpu0: [099.7] d0-0[099.7]
cpu2: [045.1] d1-0[045.1]
cpu0: [099.7] d0-0[099.7]
cpu2: [045.0] d1-0[045.0]
This is fairly consistent for the whole test.
> Spending 18% of the time handling interrupts in Xen is surprisingy
> (at least to me).
>
> What interrupt rate are you observing? What are the default tg3
> interrupt coallescing settings? What interrupt rate do you get on
> native? Also, what hypercall rate are you seeing?
>
> (It would be good to put this in context of the rx/tx packet rates).
I don't have that data from this test, but I am queuing up another with
sar, so I should have it soon.
I will also queue up a test with just baremetal linux so we can compare
int rates, etc.
> Is the Ethernet NIC sharing an interrupt with the USB controller per
> chance?
Not as far as I can tell:
CPU0
1: 8 Phys-irq i8042
3: 0 Phys-irq acpi
4: 3031 Phys-irq serial
11: 6764395 Phys-irq ohci_hcd
12: 93 Phys-irq i8042
15: 38687 Phys-irq ide1
18: 39398 Phys-irq qla2300
22: 47905 Phys-irq ioc0
24: 6037311 Phys-irq eth0
256: 7 Dynamic-irq ctrl-if
257: 182396 Dynamic-irq timer0
258: 0 Dynamic-irq net-be-dbg
259: 83437 Dynamic-irq blkif-backend
260: 1688517 Dynamic-irq vif1.0
>
> Seeing find_domain_by_id and copy_from_user so high up the list is
> pretty surprising.
Yes.
-Andrew
^ permalink raw reply [flat|nested] 12+ messages in thread
* More network tests with xenoprofile this time
@ 2005-05-31 22:01 Andrew Theurer
0 siblings, 0 replies; 12+ messages in thread
From: Andrew Theurer @ 2005-05-31 22:01 UTC (permalink / raw)
To: xen-devel
I had a chance to run a couple of the netperf tests with xenoprofile. I
am still having some trouble with multi-domain profiles (probably user
error), but I have been able to profile dom0 while running 2 types of
tests. I was surprised to see as much as 50% cpu in hypervisor on
these tests:
netperf tcp_stream 16k msg size, dom1 -> dom2
dom0 is on cpu0, HT thread 0, dom1 is on cpu1, HT thread 0,
dom2 is on cpu1, HT thread 1.
Throughput is ~900 Mbps
xenoprofile opreport:
4914314 61.2189 vmlinux-2.6.11-xen0-up
3022609 37.6534 xen-unstable-syms
79516 0.9906 oprofiled
3602 0.0449 libc-2.3.3.so
2764 0.0344 libpython2.3.so.1.0
xenoprofile opreport -l:
1656571 20.64 vmlinux-2.6.11-xen0-up skb_copy_bits
457043 5.69 vmlinux-2.6.11-xen0-up net_tx_action
361259 4.50 xen-unstable-syms do_mmuext_op
325335 4.05 xen-unstable-syms find_domain_by_id
277331 3.45 xen-unstable-syms __copy_from_user_ll
242850 3.03 xen-unstable-syms do_update_va_mapping
208405 2.60 vmlinux-2.6.11-xen0-up kfree
200640 2.50 xen-unstable-syms do_mmu_update
199645 2.49 xen-unstable-syms get_page_from_l1e
189219 2.36 xen-unstable-syms put_page_from_l1e
185831 2.31 xen-unstable-syms get_page_type
172362 2.15 vmlinux-2.6.11-xen0-up make_rx_response
171329 2.13 vmlinux-2.6.11-xen0-up nf_iterate
165977 2.07 vmlinux-2.6.11-xen0-up net_rx_action
165341 2.06 xen-unstable-syms mod_l1_entry
156055 1.94 vmlinux-2.6.11-xen0-up nf_hook_slow
116876 1.46 xen-unstable-syms alloc_domheap_pages
116650 1.45 xen-unstable-syms evtchn_send
116215 1.45 vmlinux-2.6.11-xen0-up fdb_insert
111314 1.39 vmlinux-2.6.11-xen0-up make_tx_response
108480 1.35 xen-unstable-syms alloc_heap_pages
107896 1.34 xen-unstable-syms hypercall
99013 1.23 vmlinux-2.6.11-xen0-up netif_be_start_xmit
91792 1.14 vmlinux-2.6.11-xen0-up br_handle_frame
netperf tcp_stream 16k msg size, dom1 -> external host
dom0 is on cpu0, HT thread 0, dom1 is on cpu1, HT thread 1.
Throughput is ~940 Mbps, wire speed.
xenoprofile opreport:
4244562 49.9375 xen-unstable-syms
4110594 48.3614 vmlinux-2.6.11-xen0-up
132643 1.5606 oprofiled
4212 0.0496 libc-2.3.3.so
2892 0.0340 libpython2.3.so.1.0
xenoprofile opreport -l:
828587 9.75 xen-unstable-syms end_level_ioapic_irq
712035 8.38 xen-unstable-syms mask_and_ack_level_ioapic_irq
370265 4.36 vmlinux-2.6.11-xen0-up net_tx_action
323797 3.81 vmlinux-2.6.11-xen0-up ohci_irq
282005 3.32 vmlinux-2.6.11-xen0-up tg3_interrupt
273161 3.21 xen-unstable-syms find_domain_by_id
234726 2.76 xen-unstable-syms hypercall
206693 2.43 xen-unstable-syms do_update_va_mapping
203758 2.40 xen-unstable-syms __copy_from_user_ll
201665 2.37 xen-unstable-syms do_mmuext_op
195020 2.29 vmlinux-2.6.11-xen0-up nf_iterate
184295 2.17 vmlinux-2.6.11-xen0-up nf_hook_slow
172110 2.02 vmlinux-2.6.11-xen0-up tg3_rx
164337 1.93 vmlinux-2.6.11-xen0-up net_rx_action
141999 1.67 xen-unstable-syms do_mmu_update
139120 1.64 vmlinux-2.6.11-xen0-up fdb_insert
122483 1.44 xen-unstable-syms mod_l1_entry
122017 1.44 xen-unstable-syms put_page_from_l1e
111159 1.31 xen-unstable-syms get_page_from_l1e
109921 1.29 xen-unstable-syms do_IRQ
99847 1.17 vmlinux-2.6.11-xen0-up br_handle_frame
99709 1.17 xen-unstable-syms get_page_type
93613 1.10 vmlinux-2.6.11-xen0-up kfree
90885 1.07 vmlinux-2.6.11-xen0-up end_pirq
-Andrew
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2005-06-22 2:22 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-06-01 0:15 More network tests with xenoprofile this time Santos, Jose Renato G
2005-06-07 21:47 ` William Cohen
2005-06-08 19:20 ` Andrew Theurer
-- strict thread matches above, loose matches on Subject: below --
2005-06-17 19:39 Santos, Jose Renato G
2005-06-22 2:22 ` Andrew Theurer
2005-05-31 22:48 Ian Pratt
2005-06-01 20:03 ` Jon Mason
2005-06-01 20:21 ` Andrew Theurer
2005-06-02 14:53 ` Andrew Theurer
2005-05-31 22:16 Ian Pratt
2005-05-31 22:38 ` Andrew Theurer
2005-05-31 22:01 Andrew Theurer
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.