Linux Route Cache performance tests

["Paweł Staszewski" <pstaszewski@itcare.pl>]

Hello



I make some networking performance tests for Linux 3.1

Configuration:

Linux (pktget) ----> Linux (router) ----> Linux (Sink)

pktgen config:
clone_skb 32
pkt_size 64
delay 0

pgset "flag IPDST_RND"
pgset "dst_min 10.0.0.0"
pgset "dst_max 10.18.255.255"
pgset "config 1"
pgset "flows 256"
pgset "flowlen 8"

TX performance for this host:
eth0:            RX: 0.00 P/s      TX: 12346107.73 P/s      TOTAL: 
12346107.73 P/s

On Linux (router):
grep . /proc/sys/net/ipv4/route/*
/proc/sys/net/ipv4/route/error_burst:500
/proc/sys/net/ipv4/route/error_cost:100
grep: /proc/sys/net/ipv4/route/flush: Permission denied
/proc/sys/net/ipv4/route/gc_elasticity:4
/proc/sys/net/ipv4/route/gc_interval:60
/proc/sys/net/ipv4/route/gc_min_interval:0
/proc/sys/net/ipv4/route/gc_min_interval_ms:500
/proc/sys/net/ipv4/route/gc_thresh:2000000
/proc/sys/net/ipv4/route/gc_timeout:60
/proc/sys/net/ipv4/route/max_size:8388608
/proc/sys/net/ipv4/route/min_adv_mss:256
/proc/sys/net/ipv4/route/min_pmtu:552
/proc/sys/net/ipv4/route/mtu_expires:600
/proc/sys/net/ipv4/route/redirect_load:2
/proc/sys/net/ipv4/route/redirect_number:9
/proc/sys/net/ipv4/route/redirect_silence:2048

For the first 30secs maybee more router is forwarding ~5Mpps to the 
Linux (Sink)
and some stats for this forst 30secs in attached image:

http://imageshack.us/photo/my-images/684/test1ih.png/

Left up - pktgen linux
left down - Linux router (htop)
Right up - Linux router (bwm-ng - showing pps)
Right down - Linux router (lnstat)


And all is good - performance 5Mpps until Linux router will reach ~1kk 
entries
What You can see on next attached image:

http://imageshack.us/photo/my-images/24/test2id.png/

Forwarding performance drops from 5Mpps to 1,8Mpps
And after 3 - 4 minutes it will stop on 0,7Mpps


After flushing the route cache performance increase from 0.7Mpps to 6Mpps
What You can see on next attached image:

http://imageshack.us/photo/my-images/197/test3r.png/

Is it possible to turn off route cache ? and see what performance will 
be without caching


Thanks
Pawel