From: Andrew Morton <akpm@linux-foundation.org>
To: Dimitri Sivanich <sivanich@sgi.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Christoph Lameter <cl@linux.com>
Subject: Re: [PATCH] Reduce vm_stat cacheline contention in __vm_enough_memory
Date: Wed, 12 Oct 2011 12:01:18 -0700 [thread overview]
Message-ID: <20111012120118.e948f40a.akpm@linux-foundation.org> (raw)
In-Reply-To: <20111012160202.GA18666@sgi.com>
On Wed, 12 Oct 2011 11:02:02 -0500
Dimitri Sivanich <sivanich@sgi.com> wrote:
> Tmpfs I/O throughput testing on UV systems has shown writeback contention
> between multiple writer threads (even when each thread writes to a separate
> tmpfs mount point).
>
> A large part of this is caused by cacheline contention reading the vm_stat
> array in the __vm_enough_memory check.
>
> The attached test patch illustrates a possible avenue for improvement in this
> area. By locally caching the values read from vm_stat (and refreshing the
> values after 2 seconds), I was able to improve tmpfs writeback performance from
> ~300 MB/sec to ~700 MB/sec with 120 threads writing data simultaneously to
> files on separate tmpfs mount points (tested on 3.1.0-rc9).
>
> Note that this patch is simply to illustrate the gains that can be made here.
> What I'm looking for is some guidance on an acceptable way to accomplish the
> task of reducing contention in this area, either by caching these values in a
> way similar to the attached patch, or by some other mechanism if this is
> unacceptable.
Yes, the global vm_stat[] array is a problem - I'm surprised it's hung
around for this long. Altering the sysctl_overcommit_memory mode will
hide the problem, but that's no good.
I think we've discussed switching vm_stat[] to a contention-avoiding
counter scheme. Simply using <percpu_counter.h> would be the simplest
approach. They'll introduce inaccuracies but hopefully any problems
from that will be minor for the global page counters.
otoh, I think we've been round this loop before and I don't recall why
nothing happened.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next parent reply other threads:[~2011-10-12 19:01 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20111012160202.GA18666@sgi.com>
2011-10-12 19:01 ` Andrew Morton [this message]
2011-10-12 19:57 ` [PATCH] Reduce vm_stat cacheline contention in __vm_enough_memory Christoph Lameter
2011-10-13 15:06 ` Mel Gorman
2011-10-13 15:59 ` Andi Kleen
2011-10-13 15:23 ` Dimitri Sivanich
2011-10-13 15:54 ` Christoph Lameter
2011-10-13 20:50 ` Andrew Morton
2011-10-13 21:02 ` Christoph Lameter
2011-10-13 21:24 ` Andrew Morton
2011-10-14 12:25 ` Dimitri Sivanich
2011-10-14 13:50 ` Dimitri Sivanich
2011-10-14 13:57 ` Christoph Lameter
2011-10-14 14:19 ` Dimitri Sivanich
2011-10-14 14:34 ` Christoph Lameter
2011-10-14 15:18 ` Christoph Lameter
2011-10-14 16:16 ` Dimitri Sivanich
2011-10-18 13:48 ` Dimitri Sivanich
2011-10-18 14:36 ` Christoph Lameter
2011-10-18 15:48 ` Andi Kleen
2011-10-19 1:16 ` David Rientjes
2011-10-19 14:54 ` Dimitri Sivanich
2011-10-19 15:31 ` Christoph Lameter
2011-10-24 14:59 ` Dimitri Sivanich
[not found] ` <CADE8fzrdMOBF1RyyEpMVi8aKcgOVKRQSKi0=c1Qvh3p6hHcXRA@mail.gmail.com>
2011-10-13 0:07 ` Tim Chen
2011-10-13 14:15 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111012120118.e948f40a.akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=sivanich@sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).