From: Tim Chen <tim.c.chen@linux.intel.com>
To: David Miller <davem@davemloft.net>
Cc: eric.dumazet@gmail.com, netdev@vger.kernel.org, andi@firstfloor.org
Subject: Re: [PATCH net-next-2.6] inetpeer: lower false sharing effect
Date: Fri, 10 Jun 2011 15:33:10 -0700 [thread overview]
Message-ID: <1307745190.17300.85.camel@schen9-DESK> (raw)
In-Reply-To: <20110609.204330.2090335955971650557.davem@davemloft.net>
On Thu, 2011-06-09 at 20:43 -0700, David Miller wrote:
> From: Tim Chen <tim.c.chen@linux.intel.com>
> Date: Thu, 09 Jun 2011 17:03:55 -0700
>
> > When I retest with original 3.0-rc2 kernel, inet_putpeer no longer shows
> > up, wonder if dst->peer was not set for some reason.
>
> The overhead will only show up if an inetpeer entry exists for
> the destination IP address.
>
> You can force one to be created, for example, by making a TCP
> connection to that destination.
You're right. By adding the TCP connection, inet peer shows up now in
my profile of the patched kernel with Eric's two patches.
Eric's patches produced much better cpu utilization.
The addr_compare (used to consume 10% cpu) and atomic_dec_and_lock (used
to consume 20.5% cpu) in inet_putpeer is eliminated and inet_putpeer
uses only 10% cpu now. Though inet_getpeer and inet_putpeer still
consumes significant cpu compared to the other test case when peer is
not present.
Tim
Profile with Eric's two patches and peer forced to be present with TCP
added looks like this:
- 19.38% memcached [kernel.kallsyms] [k] inet_getpeer
- inet_getpeer
+ 99.97% inet_getpeer_v4
- 11.49% memcached [kernel.kallsyms] [k] inet_putpeer
- inet_putpeer
- 99.96% ipv4_dst_destroy
dst_destroy
+ dst_release
- 5.71% memcached [kernel.kallsyms] [k] rt_set_nexthop.clone.30
- rt_set_nexthop.clone.30
+ 99.89% __ip_route_output_key
- 5.60% memcached [kernel.kallsyms] [k] atomic_add_unless.clone.34
- atomic_add_unless.clone.34
+ 99.94% neigh_lookup
+ 3.02% memcached [kernel.kallsyms] [k] do_raw_spin_lock
+ 2.87% memcached [kernel.kallsyms] [k] atomic_dec_and_test
+ 1.45% memcached [kernel.kallsyms] [k] atomic_add
+ 1.04% memcached [kernel.kallsyms] [k] _raw_spin_lock_irqsave
+ 1.03% memcached [kernel.kallsyms] [k] bit_spin_lock.clone.41
next prev parent reply other threads:[~2011-06-10 22:32 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-06-09 6:26 [PATCH net-next-2.6] inetpeer: lower false sharing effect Eric Dumazet
2011-06-09 6:31 ` David Miller
2011-06-10 0:03 ` Tim Chen
2011-06-10 3:43 ` David Miller
2011-06-10 4:47 ` Eric Dumazet
2011-06-10 22:33 ` Tim Chen [this message]
2011-06-11 0:54 ` Changli Gao
2011-06-11 4:54 ` Eric Dumazet
2011-06-11 6:17 ` Changli Gao
2011-06-11 7:09 ` Andi Kleen
2011-06-10 4:31 ` Eric Dumazet
2011-06-10 17:05 ` Tim Chen
2011-06-10 17:17 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1307745190.17300.85.camel@schen9-DESK \
--to=tim.c.chen@linux.intel.com \
--cc=andi@firstfloor.org \
--cc=davem@davemloft.net \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox