From: cold cold <nedkonedev@gmail.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: netdev@vger.kernel.org
Subject: Re: 0% cpu usasge after fresh boot or net restart but 10% CPU if kernel flush route cache
Date: Thu, 28 Jan 2010 11:14:27 +0200 [thread overview]
Message-ID: <41ac0f9e1001280114x17951046oda3fddcae4b8b9d3@mail.gmail.com> (raw)
In-Reply-To: <1264627088.2892.7.camel@edumazet-laptop>
On Wed, Jan 27, 2010 at 11:18 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mercredi 27 janvier 2010 à 21:53 +0200, cold cold a écrit :
>
>>
>> btw atm i have 1k ip on that router but plan to put 10k so route cache
>> will grow 10 time fast.
>> i try to keep low rhash_entries but garbage collector is non stop
>> running and eat CPU
>> atm i have around 7k entries per second for second for first minute.
>>
>> i think it will be better if i flush cache on secret interval instead
>> of giving work to gc.
>> i test with 512MB cache and CPU is 0% and flushing entire hash dont take a lot,
>> I'm not sure is there some side effects from flushing.
>
> Flush is immediate, it only marks entries as invalid and they are
> cleaned up later.
>
> 512 MB cache is quite small for your needs, each entry uses 384 bytes
> (assuming you use a 64bit kernel)
>
> In my experiments, I found using gc (and no flushing) was the most
> reliable way to have an equilibrium.
>
> WHen setting gc_interval to 1, the garbage collector is fired every
> second and handles 1/300 of entries, from a work queue (thus doesnt stop
> packets to be handled by irqs), in a smooth way.
>
> You can post "perf top" results to check where cpu is consumed.
>
>
>
RX Kpps : 57 TX Kpps : 53 RX Kbits : 331184 TX Kbits : 306213
RX Kpps : 59 TX Kpps : 54 RX Kbits : 345517 TX Kbits : 304323
RX Kpps : 56 TX Kpps : 52 RX Kbits : 331418 TX Kbits : 296032
RX Kpps : 60 TX Kpps : 54 RX Kbits : 362007 TX Kbits : 297371
RX Kpps : 59 TX Kpps : 52 RX Kbits : 360455 TX Kbits : 280603
ON one cpu, gc_interval to 1, gc_elasticity 2
Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 72.0%id, 0.0%wa, 8.3%hi, 19.7%si, 0.0%st
Cpu1 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
------------------------------------------------------------------------------
PerfTop: 17064 irqs/sec kernel:98.0% [100000 cycles], (all, 4 CPUs)
------------------------------------------------------------------------------
samples pcnt kernel function
_______ _____ _______________
40388.00 - 27.8% : acpi_idle_do_entry
24651.00 - 17.0% : read_hpet
4271.00 - 2.9% : _spin_lock
3388.00 - 2.3% : pskb_expand_head
3288.00 - 2.3% : igb_poll [igb]
3246.00 - 2.2% : irq_entries_start
2868.00 - 2.0% : dev_gro_receive
2665.00 - 1.8% : igb_xmit_frame_adv [igb]
2513.00 - 1.7% : ip_route_input
2144.00 - 1.5% : igb_clean_tx_irq [igb]
1842.00 - 1.3% : __slab_free
1544.00 - 1.1% : dev_queue_xmit
1423.00 - 1.0% : igb_msix_rx [igb]
1353.00 - 0.9% : __alloc_skb
1285.00 - 0.9% : eth_type_trans
next prev parent reply other threads:[~2010-01-28 9:14 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-26 16:58 0% cpu usasge after fresh boot or net restart but 10% CPU if kernel flush route cache cold cold
2010-01-27 15:26 ` Eric Dumazet
2010-01-27 19:53 ` cold cold
2010-01-27 21:18 ` Eric Dumazet
2010-01-28 9:14 ` cold cold [this message]
2010-01-28 16:09 ` Eric Dumazet
2010-01-28 16:26 ` cold cold
2010-01-28 17:06 ` Eric Dumazet
2010-01-28 18:49 ` cold cold
2010-01-28 21:16 ` Eric Dumazet
2010-01-29 7:38 ` cold cold
2010-01-29 9:06 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=41ac0f9e1001280114x17951046oda3fddcae4b8b9d3@mail.gmail.com \
--to=nedkonedev@gmail.com \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).