From: Dimitri Sivanich <sivanich@sgi.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Christoph Lameter <cl@gentwo.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Mel Gorman <mel@csn.ul.ie>
Subject: Re: [PATCH] Reduce vm_stat cacheline contention in __vm_enough_memory
Date: Fri, 14 Oct 2011 08:50:55 -0500 [thread overview]
Message-ID: <20111014135055.GA28592@sgi.com> (raw)
In-Reply-To: <20111014122506.GB26737@sgi.com>
On Fri, Oct 14, 2011 at 07:25:06AM -0500, Dimitri Sivanich wrote:
> On Thu, Oct 13, 2011 at 02:24:34PM -0700, Andrew Morton wrote:
> > On Thu, 13 Oct 2011 16:02:58 -0500 (CDT)
> > Christoph Lameter <cl@gentwo.org> wrote:
> >
> > > On Thu, 13 Oct 2011, Andrew Morton wrote:
> > >
> > > > > If there are no updates occurring for a while (due to increased deltas
> > > > > and/or vmstat updates) then the vm_stat cacheline should be able to stay
> > > > > in shared mode in multiple processors and the performance should increase.
> > > > >
> > > >
> > > > We could cacheline align vm_stat[]. But the thing is pretty small - we
> > > > couild put each entry in its own cacheline.
> > >
> > > Which in turn would increase the cache footprint of some key kernel
> > > functions (because they need multiple vm_stat entries) and cause eviction
> > > of other cachelines that then reduce overall system performance again.
> >
> > Sure, but we gain performance by not having different CPUs treading on
> > each other when they update different vmstat fields. Sometimes one
> > effect will win and other times the other effect will win. Some
> > engineering is needed..
>
> I think the first step is to determine the role (if any) that false sharing may be playing in this, since that's a simpler fix (cacheline align and pad the array).
>
Testing on a smaller machine with 46 writer threads in parallel (my original
test used 120).
Looks as though cache-aligning and padding the end of the vm_stat array
results in a ~150 MB/sec speedup. This is a nice improvement for only 46
writer threads, though it's not the full ~250 MB/sec speedup I get from
setting OVERCOMMIT_NEVER.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-10-14 13:50 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20111012160202.GA18666@sgi.com>
2011-10-12 19:01 ` [PATCH] Reduce vm_stat cacheline contention in __vm_enough_memory Andrew Morton
2011-10-12 19:57 ` Christoph Lameter
2011-10-13 15:06 ` Mel Gorman
2011-10-13 15:59 ` Andi Kleen
2011-10-13 15:23 ` Dimitri Sivanich
2011-10-13 15:54 ` Christoph Lameter
2011-10-13 20:50 ` Andrew Morton
2011-10-13 21:02 ` Christoph Lameter
2011-10-13 21:24 ` Andrew Morton
2011-10-14 12:25 ` Dimitri Sivanich
2011-10-14 13:50 ` Dimitri Sivanich [this message]
2011-10-14 13:57 ` Christoph Lameter
2011-10-14 14:19 ` Dimitri Sivanich
2011-10-14 14:34 ` Christoph Lameter
2011-10-14 15:18 ` Christoph Lameter
2011-10-14 16:16 ` Dimitri Sivanich
2011-10-18 13:48 ` Dimitri Sivanich
2011-10-18 14:36 ` Christoph Lameter
2011-10-18 15:48 ` Andi Kleen
2011-10-19 1:16 ` David Rientjes
2011-10-19 14:54 ` Dimitri Sivanich
2011-10-19 15:31 ` Christoph Lameter
2011-10-24 14:59 ` Dimitri Sivanich
[not found] ` <CADE8fzrdMOBF1RyyEpMVi8aKcgOVKRQSKi0=c1Qvh3p6hHcXRA@mail.gmail.com>
2011-10-13 0:07 ` Tim Chen
2011-10-13 14:15 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111014135055.GA28592@sgi.com \
--to=sivanich@sgi.com \
--cc=akpm@linux-foundation.org \
--cc=cl@gentwo.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mel@csn.ul.ie \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).