netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Thomas Graf <tgraf@infradead.org>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>,
	David Miller <davem@davemloft.net>,
	rick.jones2@hp.com, therbert@google.com, wsommerfeld@google.com,
	daniel.baluta@gmail.com, netdev@vger.kernel.org
Subject: Re: SO_REUSEPORT - can it be done in kernel?
Date: Tue, 1 Mar 2011 07:18:29 -0500	[thread overview]
Message-ID: <20110301121829.GM9763@canuck.infradead.org> (raw)
In-Reply-To: <1298979909.3284.28.camel@edumazet-laptop>

On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
> ethtool -S eth0 | grep rx_pk
>      rxq0: rx_pkts: ??
>      rxq1: rx_pkts: ??
>      rxq2: rx_pkts: ??
>      rxq3: rx_pkts: ??
>      rxq4: rx_pkts: ??

It could do multiqueue but it doesnt:

[root@hp-bl460cg7-01 ~]# ethtool -S eth0 | grep rx_pk
     rxq0: rx_pkts: 1512
     rxq1: rx_pkts: 462
     rxq2: rx_pkts: 122
     rxq3: rx_pkts: 24751393
     rxq4: rx_pkts: 35

So, adding a third client making sure it would hit another queue:
     rxq0: rx_pkts: 3041
     rxq1: rx_pkts: 867
     rxq2: rx_pkts: 4610476
     rxq3: rx_pkts: 57418776
     rxq4: rx_pkts: 40

... makes it use CPU 5 for rxq2 and the qps goes up from 250kqps to 270kqps

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
   PerfTop:   18417 irqs/sec  kernel:50.2%  exact:  0.0% [1000Hz cpu-clock-msecs],  (all, CPU: 5)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------

             samples  pcnt function                      DSO
             _______ _____ _____________________________ ___________________________________________________________

            12712.00 18.5% _raw_spin_unlock_irqrestore   [kernel.kallsyms]                                          
             3697.00  5.4% isc_rwlock_lock               /usr/lib64/libisc.so.62.0.1                                
             1948.00  2.8% dns_rbt_findnode              /usr/lib64/libdns.so.69.0.1                                
             1809.00  2.6% isc_rwlock_unlock             /usr/lib64/libisc.so.62.0.1                                
             1631.00  2.4% __do_softirq                  [kernel.kallsyms]                                          
             1237.00  1.8% isc_stats_increment           /usr/lib64/libisc.so.62.0.1                                
             1106.00  1.6% clflush_cache_range           [kernel.kallsyms]                                          
              964.00  1.4% _raw_spin_lock                [kernel.kallsyms]                                          
              714.00  1.0% be_poll_rx                    /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
              630.00  0.9% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so                                  
              627.00  0.9% dns_name_fullcompare          /usr/lib64/libdns.so.69.0.1                                
              582.00  0.8% dns_rbtnodechain_init         /usr/lib64/libdns.so.69.0.1                                
              552.00  0.8% sk_run_filter                 [kernel.kallsyms]                                          
              527.00  0.8% dns_name_getlabelsequence     /usr/lib64/libdns.so.69.0.1                                
              525.00  0.8% __pthread_mutex_unlock        /lib64/libpthread-2.12.so                                  
              492.00  0.7% dns_name_equal                /usr/lib64/libdns.so.69.0.1                                
              468.00  0.7% isc___mempool_get             /usr/lib64/libisc.so.62.0.1                                
              462.00  0.7% __udp4_lib_lookup             [kernel.kallsyms]                                          
              457.00  0.7% dns_acl_match                 /usr/lib64/libdns.so.69.0.1                                
              453.00  0.7% dns_zone_attach               /usr/lib64/libdns.so.69.0.1                                
              451.00  0.7% fget_light                    [kernel.kallsyms]                                          
              443.00  0.6% dns_message_rendersection     /usr/lib64/libdns.so.69.0.1                                
              431.00  0.6% ipt_do_table                  [kernel.kallsyms]                                          
              429.00  0.6% nf_iterate                    [kernel.kallsyms]                                          
              422.00  0.6% __kmalloc_node_track_caller   [kernel.kallsyms]                                          
              408.00  0.6% __domain_mapping              [kernel.kallsyms]                                          
              387.00  0.6% dns_name_hash                 /usr/lib64/libdns.so.69.0.1                                
              353.00  0.5% copy_user_generic_string      [kernel.kallsyms]                                          
              349.00  0.5% dns_name_fromwire             /usr/lib64/libdns.so.69.0.1                          

  parent reply	other threads:[~2011-03-01 12:18 UTC|newest]

Thread overview: 91+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-27 10:07 SO_REUSEPORT - can it be done in kernel? Daniel Baluta
2011-01-27 15:55 ` Bill Sommerfeld
2011-01-27 21:32 ` Tom Herbert
2011-02-25 12:56   ` Thomas Graf
2011-02-25 19:18     ` Rick Jones
2011-02-25 19:20       ` David Miller
2011-02-26  0:57         ` Herbert Xu
2011-02-26  2:12           ` David Miller
2011-02-26  2:48             ` Herbert Xu
2011-02-26  3:07               ` David Miller
2011-02-26  3:11                 ` Herbert Xu
2011-02-26  7:31                   ` Eric Dumazet
2011-02-26  7:46                     ` David Miller
2011-02-27 11:02           ` Thomas Graf
2011-02-27 11:06             ` Herbert Xu
2011-02-28  3:45               ` Tom Herbert
2011-02-28  4:26                 ` Herbert Xu
2011-02-28 11:36               ` Herbert Xu
2011-02-28 13:32                 ` Eric Dumazet
2011-02-28 14:13                   ` Herbert Xu
2011-02-28 14:22                     ` Eric Dumazet
2011-02-28 14:25                       ` Herbert Xu
2011-02-28 14:53                   ` Eric Dumazet
2011-02-28 15:01                     ` Thomas Graf
2011-02-28 14:13                 ` Thomas Graf
2011-02-28 16:22                   ` Eric Dumazet
2011-02-28 16:37                     ` Thomas Graf
2011-02-28 17:07                       ` Eric Dumazet
2011-03-01 10:19                         ` Thomas Graf
2011-03-01 10:33                           ` Eric Dumazet
2011-03-01 11:07                             ` Thomas Graf
2011-03-01 11:13                               ` Eric Dumazet
2011-03-01 11:27                                 ` Thomas Graf
2011-03-01 11:45                                   ` Eric Dumazet
2011-03-01 11:53                                     ` Herbert Xu
2011-03-01 12:32                                       ` Herbert Xu
2011-03-01 13:04                                         ` Eric Dumazet
2011-03-01 13:11                                           ` Herbert Xu
2011-03-01 13:03                                       ` Eric Dumazet
2011-03-01 13:18                                         ` Herbert Xu
2011-03-01 13:52                                           ` Eric Dumazet
2011-03-01 13:58                                             ` Herbert Xu
2011-03-01 16:31                                           ` Eric Dumazet
2011-03-02  0:23                                             ` Herbert Xu
2011-03-02  2:00                                               ` Eric Dumazet
2011-03-02  2:39                                                 ` Herbert Xu
2011-03-02  2:56                                                   ` Eric Dumazet
2011-03-02  3:09                                                     ` Herbert Xu
2011-03-02  3:44                                                       ` Eric Dumazet
2011-03-02  7:12                                                   ` Tom Herbert
2011-03-02  7:31                                                     ` Herbert Xu
2011-03-02  8:04                                                       ` Eric Dumazet
2011-03-02  8:07                                                         ` Herbert Xu
2011-03-02  8:24                                                           ` Eric Dumazet
2011-03-01 12:01                                     ` Thomas Graf
2011-03-01 12:15                                       ` Herbert Xu
2011-03-01 13:27                                       ` Herbert Xu
2011-03-01 12:18                                     ` Thomas Graf [this message]
2011-03-01 12:19                                       ` Herbert Xu
2011-03-01 13:50                                         ` Thomas Graf
2011-03-01 14:06                                           ` Eric Dumazet
2011-03-01 14:22                                             ` Thomas Graf
2011-03-01 14:30                                               ` Thomas Graf
2011-03-01 14:52                                                 ` Eric Dumazet
2011-03-01 15:07                                                   ` Thomas Graf
2011-03-01  5:33                 ` Eric Dumazet
2011-03-01 12:35                 ` Herbert Xu
2011-03-01 12:36                   ` [PATCH 3/5] inet: Add ip_make_skb and ip_finish_skb Herbert Xu
2011-03-01 12:36                   ` [PATCH 2/5] inet: Remove explicit write references to sk/inet in ip_append_data Herbert Xu
2011-03-02  6:15                     ` inet: Replace left-over references to inet->cork Herbert Xu
2011-03-02  7:01                       ` David Miller
2011-03-01 12:36                   ` [PATCH 1/5] inet: Remove unused sk_sndmsg_* from UFO Herbert Xu
2011-03-01 12:36                   ` [PATCH 5/5] udp: Add lockless transmit path Herbert Xu
2011-03-01 12:36                   ` [PATCH 4/5] udp: Switch to ip_finish_skb Herbert Xu
2011-03-01 16:43                   ` SO_REUSEPORT - can it be done in kernel? Eric Dumazet
2011-03-01 20:36                     ` David Miller
2011-02-28 11:41               ` [PATCH 1/5] net: Remove unused sk_sndmsg_* from UFO Herbert Xu
2011-03-01  5:31                 ` Eric Dumazet
2011-02-28 11:41               ` [PATCH 2/5] net: Remove explicit write references to sk/inet in ip_append_data Herbert Xu
2011-03-01  5:31                 ` Eric Dumazet
2011-02-28 11:41               ` [PATCH 3/5] inet: Add ip_make_skb and ip_send_skb Herbert Xu
2011-03-01  5:31                 ` Eric Dumazet
2011-02-28 11:41               ` [PATCH 4/5] udp: Add lockless transmit path Herbert Xu
2011-02-28 11:41                 ` Herbert Xu
2011-03-01  5:30                 ` Eric Dumazet
2011-02-25 19:21       ` SO_REUSEPORT - can it be done in kernel? Eric Dumazet
2011-02-25 22:48       ` Thomas Graf
2011-02-25 23:15         ` Rick Jones
2011-02-25 19:51     ` Tom Herbert
2011-02-25 22:58       ` Thomas Graf
2011-02-25 23:33       ` Bill Sommerfeld

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110301121829.GM9763@canuck.infradead.org \
    --to=tgraf@infradead.org \
    --cc=daniel.baluta@gmail.com \
    --cc=davem@davemloft.net \
    --cc=eric.dumazet@gmail.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=netdev@vger.kernel.org \
    --cc=rick.jones2@hp.com \
    --cc=therbert@google.com \
    --cc=wsommerfeld@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).