Re: Problem wit route cache

[Eric Dumazet <eric.dumazet@gmail.com>]

Le lundi 08 février 2010 à 14:59 +0100, Paweł Staszewski a écrit :
> W dniu 2010-02-08 14:51, Eric Dumazet pisze:
> > Le lundi 08 février 2010 à 14:33 +0100, Paweł Staszewski a écrit :
> >
> >    
> >>>
> >>>        
> >> Yes this is x86_64 kernel
> >> i kernels  2.6.32.2 /  2.6.32.7 and now 2.6.33-rc6-git5 and on all
> >> kernels the same thing happens.
> >> grep . /proc/sys/net/ipv4/route/*
> >> /proc/sys/net/ipv4/route/error_burst:1250
> >> /proc/sys/net/ipv4/route/error_cost:250
> >> grep: /proc/sys/net/ipv4/route/flush: Permission denied
> >> /proc/sys/net/ipv4/route/gc_elasticity:2
> >> /proc/sys/net/ipv4/route/gc_interval:2
> >> /proc/sys/net/ipv4/route/gc_min_interval:0
> >> /proc/sys/net/ipv4/route/gc_min_interval_ms:500
> >> /proc/sys/net/ipv4/route/gc_thresh:65535
> >> /proc/sys/net/ipv4/route/gc_timeout:300
> >> /proc/sys/net/ipv4/route/max_size:524288
> >> /proc/sys/net/ipv4/route/min_adv_mss:256
> >> /proc/sys/net/ipv4/route/min_pmtu:552
> >> /proc/sys/net/ipv4/route/mtu_expires:600
> >> /proc/sys/net/ipv4/route/redirect_load:5
> >> /proc/sys/net/ipv4/route/redirect_number:9
> >> /proc/sys/net/ipv4/route/redirect_silence:5120
> >> /proc/sys/net/ipv4/route/secret_interval:2
> >>
> >> This happens not all the time.
> >> I have this info only when there are "internet rush hours" - thn there
> >> is about 700Mbit/s TX + 700Mbit/s RX forwarded traffic
> >>
> >>      
> > I dont understand your settings, they are very very small for your
> > setup. You want to flush cache every 2 seconds...
> >
> > With 12GB of ram, you could have
> >
> > /proc/sys/net/ipv4/route/gc_thresh:524288
> > /proc/sys/net/ipv4/route/max_size:8388608
> > /proc/sys/net/ipv4/route/secret_interval:3600
> > /proc/sys/net/ipv4/route/gc_elasticity:4
> > /proc/sys/net/ipv4/route/gc_interval:1
> >
> > That would allow about 2 million entries in your route cache, using 768
> > Mbytes of ram, and a good cache hit ratio.
> >
> >
> >    
> Yes as i write i change this settings after i see first info 
> "secret_interval" - from 3600 to 2
> To check if this resolve the problem.
> Also my normal settings are:
> 
> /proc/sys/net/ipv4/route/gc_thresh:256000
> /proc/sys/net/ipv4/route/max_size:1048576
> /proc/sys/net/ipv4/route/secret_interval:3600
> /proc/sys/net/ipv4/route/gc_interval:2
> /proc/sys/net/ipv4/route/gc_elasticity:2
> 
> And with this setting i was have this info:
> Route hash chain too long!
> Adjust your secret_interval!
> 
> 
> 
> Now i put Your settings as You suggest ... and we will see but i dont know it will help.
> Because i try many of different settings.
> 

One important point is the size of hash table, you want something big
for your router.

# dmesg | grep 'IP route'
 ... IP route cache hash table entries: 524288 (order: 10, 4194304
bytes)

Then if it is correctly sized, dont change gc_thresh or max_size, as
defaults are good.

I would only change gc_interval to 1, to perform a smooth gc

And eventually gc_elasticity to 4, 5 or 6 if I had less ram than your
machine.