Re: 0% cpu usasge after fresh boot or net restart but 10% CPU if kernel flush route cache

[cold cold <nedkonedev@gmail.com>]

On Wed, Jan 27, 2010 at 5:26 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mardi 26 janvier 2010 à 18:58 +0200, cold cold a écrit :
>> HI,
>>
>>
>> i have expiriance some CPU usage spikes up to 10% on each four cpus
>> after the first kernel route cache flush.
>> After mashine start first 20 min CPU is 0%si  300Mbits/s full duplex
>> and arount 100k pps forwarded traffic, without any firewall, just
>> plain routing.
>>
>> route -n |wc -l
>> 34
>>
>> ip route show cache | wc -l
>> 2140842
>>
>> cat /proc/sys/net/ipv4/route/secret_interval
>> 600
>> cat /proc/sys/net/ipv4/route/max_size
>> 33554432
>>
>> after kernel  flush i got 10% on all CPUs for 5-6 mins.  It's not from
>> rebuilding route cashe becouse after
>> fresh boot or network restrat there is no CPU usage until kernel flush
>> route cache.
>> I try to play with rhash_entries= 300000 to 2000000 same result.
>
> If you have one million dst entries to flush, it takes some time.
>
> You could try to not increase the rhash_entries
> (or keep it low, say 131072)
> but tune /proc/sys/net/ipv4/route settings.
>
> Try to reduce gc_elasticity from 8 to 2
> Try to reduce gc_interval from 60 to 1
>
> Important thing to consider is to irq affinities (so that one cpu
> handles network interrupts, to minimize cache ping poings )
>


btw atm i have 1k ip on that router but plan to put 10k so route cache
will grow 10 time fast.
i try to keep low  rhash_entries but garbage collector  is non stop
running and eat CPU
atm i have around 7k  entries per second for second for first minute.

i think it will be better if i flush cache on secret interval instead
of giving work to gc.
i test with 512MB cache and CPU is 0% and flushing entire hash dont take a lot,
I'm not sure is there some side effects from flushing.