From: Simon Kirby <sim@netnation.com>
To: "David S. Miller" <davem@redhat.com>
Cc: Robert Olsson <Robert.Olsson@data.slu.se>,
ralph+d@istop.com, hadi@shell.cyberus.ca, xerox@foonet.net,
fw@deneb.enyo.de, netdev@oss.sgi.com, linux-net@vger.kernel.org
Subject: Re: Route cache performance tests
Date: Tue, 17 Jun 2003 15:50:36 -0700 [thread overview]
Message-ID: <20030617225036.GG25773@netnation.com> (raw)
In-Reply-To: <20030617210700.GE25773@netnation.com>
On Tue, Jun 17, 2003 at 02:07:01PM -0700, Simon Kirby wrote:
> 60.0104 seconds passed, avg forwarding rate: 259027.200 pps
>
> This is with standard juno (pseudo-random sources).
>
> This is with CONFIG_IP_MULTIPLE_TABLES still on, too.
Here is with CONFIG_IP_MULTIPLE_TABLES=n and CONFIG_NETFILTER=n
(rp_filter off and CONFIG_SMP=n in all tests):
60.0050 seconds passed, avg forwarding rate: 276893.102 pps
60.0046 seconds passed, avg forwarding rate: 257257.533 pps
60.0101 seconds passed, avg forwarding rate: 251852.843 pps
60.0106 seconds passed, avg forwarding rate: 248110.756 pps
60.0045 seconds passed, avg forwarding rate: 246280.066 pps
"rtstat -i 1" shows the pps rate decreasing because of the growing
rtcache:
size IN: hit tot mc no_rt bcast madst masrc OUT: hit tot mc GC: tot ignored goal_miss ovrf
16688 18 294882 0 0 0 0 0 0 0 0 294882 294880 2 0
16834 24 302376 0 0 0 0 0 0 0 0 302376 302374 2 0
16970 12 294288 0 0 0 0 0 0 0 0 294288 294286 2 0
17037 21 294278 0 0 0 0 0 0 0 0 294278 294276 1 0
17133 20 293080 0 0 0 0 0 0 0 0 293080 293078 1 0
17195 22 293978 0 0 0 0 0 0 0 0 293978 293976 1 0
17293 16 292184 0 0 0 0 0 0 0 0 292184 292182 1 0
17370 19 293681 0 0 0 0 0 0 0 0 293681 293679 1 0
17450 21 293079 0 0 0 0 0 0 0 0 293079 293077 1 0
17542 12 293388 0 0 0 0 0 0 0 0 293388 293386 1 0
17604 16 293684 0 0 0 0 0 0 0 0 293684 293682 1 0
17676 27 294573 0 0 0 0 0 0 0 0 294573 294571 0 0
17762 18 291582 0 0 0 0 0 0 0 0 291582 291580 1 0
...
21615 17 257683 0 0 0 0 0 0 0 0 257683 257681 1 0
21641 23 257077 0 0 0 0 0 0 0 0 257077 257075 0 0
21672 23 257077 0 0 0 0 0 0 0 0 257077 257075 1 0
Profile:
vma samples % symbol name
c025ee10 9228 15.1465 fn_hash_lookup
c02379a0 4025 6.60648 ip_route_input_slow
c0236650 3321 5.45096 rt_intern_hash
c0235d60 2775 4.55478 rt_hash_code
c012d750 2622 4.30365 kmem_cache_alloc
c012d950 2338 3.83751 kmem_cache_free
c0239f50 2323 3.81288 ip_rcv
c0238060 2277 3.73738 ip_route_input
c0233ac0 2190 3.59458 eth_header
c0231b50 2119 3.47805 neigh_resolve_output
c025ca50 2074 3.40419 fib_validate_source
c023aed0 1987 3.26139 ip_forward
c0230a70 1926 3.16126 neigh_lookup
c012d830 1845 3.02831 kmalloc
c0234270 1523 2.49979 pfifo_fast_dequeue
c025dde0 1407 2.3094 fib_semantic_match
c02376c0 1354 2.2224 rt_set_nexthop
c022a730 1322 2.16988 __kfree_skb
c022a480 1248 2.04842 alloc_skb
c012da00 1229 2.01723 kfree
c0259830 1171 1.92204 inet_select_addr
c0233ca0 1169 1.91875 eth_type_trans
c02304a0 994 1.63151 dst_destroy
c0230380 976 1.60197 dst_alloc
c010c910 806 1.32294 do_gettimeofday
c02319d0 793 1.3016 neigh_hh_init
c0236380 739 1.21297 rt_garbage_collect
c0233ef0 732 1.20148 qdisc_restart
c0234200 731 1.19984 pfifo_fast_enqueue
c022e590 695 1.14075 netif_receive_skb
c022dff0 623 1.02257 dev_queue_xmit
c023d6e0 575 0.943783 ip_finish_output
c02562a0 293 0.480919 arp_hash
Full route table:
60.0054 seconds passed, avg forwarding rate: 141888.209 pps
vma samples % symbol name
c025ee10 21133 42.5588 fn_hash_lookup
c02379a0 2219 4.46874 ip_route_input_slow
c0236650 1600 3.22217 rt_intern_hash
c0235d60 1471 2.96238 rt_hash_code
c012d750 1282 2.58176 kmem_cache_alloc
c012d950 1279 2.57572 kmem_cache_free
c0259830 1253 2.52336 inet_select_addr
c0239f50 1231 2.47906 ip_rcv
c0233ac0 1214 2.44482 eth_header
c025dde0 1165 2.34614 fib_semantic_match
c0231b50 1133 2.2817 neigh_resolve_output
c0238060 1126 2.2676 ip_route_input
c025ca50 1120 2.25552 fib_validate_source
c0230a70 1041 2.09642 neigh_lookup
c023aed0 1032 2.0783 ip_forward
c012d830 1002 2.01788 kmalloc
c02376c0 762 1.53456 rt_set_nexthop
c0234270 733 1.47616 pfifo_fast_dequeue
c022a730 709 1.42782 __kfree_skb
c022a480 633 1.27477 alloc_skb
c012da00 629 1.26671 kfree
c0233ca0 623 1.25463 eth_type_trans
c02304a0 549 1.10561 dst_destroy
c0230380 519 1.04519 dst_alloc
c02319d0 447 0.900193 neigh_hh_init
c0236380 426 0.857902 rt_garbage_collect
c010c910 425 0.855889 do_gettimeofday
c022e590 395 0.795473 netif_receive_skb
c0234200 392 0.789431 pfifo_fast_enqueue
c0233ef0 381 0.767279 qdisc_restart
c022dff0 345 0.69478 dev_queue_xmit
c025de90 298 0.600129 __fib_res_prefsrc
c023d6e0 294 0.592073 ip_finish_output
c02567d0 145 0.292009 arp_bind_neighbour
Here's 2.5.72 (which seems to have all of the patches already in), empty
routing table:
60.0085 seconds passed, avg forwarding rate: 166543.268 pps
60.0080 seconds passed, avg forwarding rate: 167055.912 pps
60.0051 seconds passed, avg forwarding rate: 166843.560 pps
vma samples % symbol name
c02c0020 5193 10.2685 fn_hash_lookup
c02930d0 3475 6.87139 ip_route_input_slow
c02222c0 2349 4.64486 tg3_start_xmit
c0291c10 2217 4.38385 rt_intern_hash
c02bde40 1910 3.77679 fib_validate_source
c0293900 1864 3.68583 ip_route_input
c02953d0 1646 3.25477 ip_rcv
c0288b40 1609 3.1816 netif_receive_skb
c0135210 1462 2.89093 kmem_cache_free
c0134fd0 1457 2.88104 free_block
c02216a0 1390 2.74856 tg3_rx
c0135150 1295 2.56071 kmem_cache_alloc
c02912d0 1275 2.52116 rt_hash_code
c028f040 1250 2.47172 eth_header
c0134e00 1250 2.47172 cache_alloc_refill
c028ca40 1223 2.41833 neigh_resolve_output
c028f740 1072 2.11975 pfifo_fast_dequeue
c0135190 1019 2.01495 __kmalloc
c028ba50 991 1.95958 neigh_lookup
c02bf1d0 843 1.66693 fib_semantic_match
c01adc10 816 1.61354 memcpy
c0284ba0 768 1.51863 alloc_skb
c0135250 724 1.43162 kfree
c028f1b0 702 1.38812 eth_type_trans
c02b8930 674 1.33275 inet_select_addr
c0288600 668 1.32089 dev_queue_xmit
c0297850 664 1.31298 ip_finish_output
c0221e20 654 1.29321 tg3_set_txd
c028b340 635 1.25564 dst_alloc
c01289e0 632 1.2497 call_rcu
c0296370 606 1.19829 ip_forward
c0297af0 567 1.12117 ip_output
c028c8c0 547 1.08163 neigh_hh_init
c028b470 547 1.08163 dst_destroy
c011f080 489 0.966938 local_bh_enable
c0221550 477 0.94321 tg3_recycle_rx
Erp, needed a new rtstat. And a wider console, apparently:
size IN: hit tot mc no_rt bcast madst masrc OUT: hit tot mc GC: tot ignored goal_miss ovrf HASH: in_search out_search
22233 18 329638 0 0 0 0 0 0 0 0 329638 329634 2 0 665742 0
20523 22 329074 0 0 0 0 0 0 0 0 329074 329070 2 0 665184 0
23510 26 331502 0 0 0 0 0 0 0 0 331502 331498 2 0 671610 0
22552 24 330464 0 0 0 0 0 0 0 0 330464 330460 4 0 669214 0
20359 8 329512 0 0 0 0 0 0 0 0 329512 329508 2 0 664428 0
19965 22 330090 0 0 0 0 0 0 0 0 330090 330086 2 0 663296 0
20081 20 332660 0 0 0 0 0 0 0 0 332660 332656 2 0 671912 0
21113 22 330458 0 0 0 0 0 0 0 0 330458 330454 2 0 666340 0
19864 14 329778 0 0 0 0 0 0 0 0 329778 329774 2 0 667324 0
20195 18 329702 0 0 0 0 0 0 0 0 329702 329698 2 0 670646 0
Route cache size does not increase on 2.5, so the problems in 2.4 are
probably the result of me hacking in the 2.5 patch.
2.5.72, full routing table:
60.0057 seconds passed, avg forwarding rate: 101800.795 pps
60.0045 seconds passed, avg forwarding rate: 101612.797 pps
60.0046 seconds passed, avg forwarding rate: 102004.873 pps
60.0044 seconds passed, avg forwarding rate: 102042.629 pps
60.0055 seconds passed, avg forwarding rate: 102135.224 pps
60.0057 seconds passed, avg forwarding rate: 102158.546 pps
60.0044 seconds passed, avg forwarding rate: 102200.430 pps
vma samples % symbol name
c02c0020 14206 33.0911 fn_hash_lookup
c02930d0 2103 4.89867 ip_route_input_slow
c02222c0 1436 3.34498 tg3_start_xmit
c0291c10 1328 3.09341 rt_intern_hash
c02bde40 1315 3.06313 fib_validate_source
c0293900 1122 2.61356 ip_route_input
c02953d0 1028 2.3946 ip_rcv
c02bf1d0 1013 2.35966 fib_semantic_match
c0288b40 957 2.22921 netif_receive_skb
c0134fd0 840 1.95667 free_block
c0135210 823 1.91707 kmem_cache_free
c02b8930 811 1.88912 inet_select_addr
c02216a0 804 1.87282 tg3_rx
c028ca40 801 1.86583 neigh_resolve_output
c02912d0 786 1.83089 rt_hash_code
c028f040 709 1.65153 eth_header
c0135190 700 1.63056 __kmalloc
c0134e00 692 1.61193 cache_alloc_refill
c028f740 644 1.50012 pfifo_fast_dequeue
c028ba50 597 1.39064 neigh_lookup
c0135150 591 1.37666 kmem_cache_alloc
c0284ba0 539 1.25553 alloc_skb
c01adc10 491 1.14372 memcpy
c0135250 449 1.04589 kfree
c028b340 437 1.01794 dst_alloc
c028f1b0 433 1.00862 eth_type_trans
c0297af0 422 0.982996 ip_output
c01289e0 402 0.936408 call_rcu
c0297850 400 0.931749 ip_finish_output
c0296370 386 0.899138 ip_forward
c0221e20 384 0.894479 tg3_set_txd
c0288600 365 0.850221 dev_queue_xmit
c011f080 364 0.847892 local_bh_enable
c02919c0 356 0.829257 rt_garbage_collect
c028c8c0 317 0.738411 neigh_hh_init
c028b470 313 0.729094 dst_destroy
c02212e0 299 0.696483 tg3_tx
size IN: hit tot mc no_rt bcast madst masrc OUT: hit tot mc GC: tot ignored goal_miss ovrf HASH: in_search out_search
18755 12 202740 0 0 0 0 0 0 0 0 202740 202736 2 0 405030 0
19945 20 203884 0 0 0 0 0 0 0 0 203884 203880 2 0 406726 0
18449 8 204152 0 0 0 0 0 0 0 0 204152 204148 0 0 409590 0
19637 10 205302 0 0 0 0 0 0 0 0 205302 205298 2 0 413004 0
19213 10 204022 0 0 0 0 0 0 0 0 204022 204018 2 0 411092 0
20182 8 204280 0 0 0 0 0 0 0 0 204280 204276 2 0 412044 0
19311 14 203378 0 0 0 0 0 0 0 0 203378 203374 2 0 411052 0
18790 16 202480 0 0 0 0 0 0 0 0 202480 202476 2 0 409440 0
18835 24 204776 0 0 0 0 0 0 0 0 204776 204772 0 0 414416 0
19830 8 204792 0 0 0 0 0 0 0 0 204792 204788 2 0 415514 0
Simon-
next prev parent reply other threads:[~2003-06-17 22:50 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-06-10 7:57 Route cache performance tests Simon Kirby
2003-06-10 11:23 ` Jamal Hadi
2003-06-10 20:36 ` CIT/Paul
2003-06-10 13:34 ` Ralph Doncaster
2003-06-10 13:39 ` Jamal Hadi
2003-06-13 6:20 ` David S. Miller
2003-06-16 22:37 ` Simon Kirby
2003-06-16 22:44 ` David S. Miller
2003-06-16 23:09 ` Simon Kirby
2003-06-16 23:08 ` David S. Miller
2003-06-16 23:27 ` Simon Kirby
2003-06-16 23:49 ` Simon Kirby
2003-06-17 15:59 ` David S. Miller
2003-06-17 16:50 ` Robert Olsson
2003-06-17 16:50 ` David S. Miller
2003-06-17 17:29 ` Robert Olsson
2003-06-17 19:06 ` Mr. James W. Laferriere
2003-06-17 20:12 ` Robert Olsson
2003-06-17 20:07 ` Simon Kirby
2003-06-17 20:17 ` Martin Josefsson
2003-06-17 20:37 ` Simon Kirby
2003-06-17 20:36 ` David S. Miller
2003-06-17 20:51 ` Simon Kirby
2003-06-17 20:49 ` David S. Miller
2003-06-18 5:50 ` Pekka Savola
2003-06-17 20:49 ` Robert Olsson
2003-06-17 21:07 ` Simon Kirby
2003-06-17 22:50 ` Simon Kirby [this message]
2003-06-17 23:07 ` David S. Miller
2003-06-17 22:11 ` Ralph Doncaster
2003-06-17 22:08 ` David S. Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20030617225036.GG25773@netnation.com \
--to=sim@netnation.com \
--cc=Robert.Olsson@data.slu.se \
--cc=davem@redhat.com \
--cc=fw@deneb.enyo.de \
--cc=hadi@shell.cyberus.ca \
--cc=linux-net@vger.kernel.org \
--cc=netdev@oss.sgi.com \
--cc=ralph+d@istop.com \
--cc=xerox@foonet.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).