From: Andi Kleen <ak@linux.intel.com>
To: Christoph Lameter <cl@linux.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
linux-mm@kvack.org, Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
x86@kernel.org, Tim Chen <tim.c.chen@linux.intel.com>,
Alex Shi <alex.shu@intel.com>, Jan Beulich <jbeulich@novell.com>,
Robert Richter <robert.richter@amd.com>,
Andy Lutomirski <luto@amacapital.net>,
Andrew Morton <akpm@linux-foundation.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Hugh Dickins <hughd@google.com>,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
Mel Gorman <mgorman@suse.de>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH, RFC 0/6] Avoid cache trashing on clearing huge/gigantic page
Date: Wed, 25 Jul 2012 12:28:50 -0700 [thread overview]
Message-ID: <20120725192850.GA4952@tassilo.jf.intel.com> (raw)
In-Reply-To: <alpine.DEB.2.00.1207251346250.4995@router.home>
On Wed, Jul 25, 2012 at 01:51:01PM -0500, Christoph Lameter wrote:
> On Fri, 20 Jul 2012, Kirill A. Shutemov wrote:
>
> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> >
> > Clearing a 2MB huge page will typically blow away several levels of CPU
> > caches. To avoid this only cache clear the 4K area around the fault
> > address and use a cache avoiding clears for the rest of the 2MB area.
>
> why exempt the 4K around the fault address? Is there a regression if that
> is not exempted?
You would get an immediate cache miss when the faulting instruction
is reexecuted.
>
> I guess for anonymous huge pages one may assume that there will be at
> least one write to one cache line in the 4k page. Is it useful to get all
> the cachelines in the page in the cache.
We did some measurements -- comparing 4K and 2MB with some tracing
of fault patterns -- and a lot of apps don't use the full 2MB area.
The apps with THP regressions usually used less than others.
The patchkit significantly reduced some of the regressions.
>
> Also note that if we get later into hugepage use for the page cache we
> would want the cache to be cold because the contents have to come in from
> a storage medium.
Page cache is not cleared, so never runs this code.
-Andi
--
ak@linux.intel.com -- Speaking for myself only
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Andi Kleen <ak@linux.intel.com>
To: Christoph Lameter <cl@linux.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
linux-mm@kvack.org, Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
x86@kernel.org, Tim Chen <tim.c.chen@linux.intel.com>,
Alex Shi <alex.shu@intel.com>, Jan Beulich <jbeulich@novell.com>,
Robert Richter <robert.richter@amd.com>,
Andy Lutomirski <luto@amacapital.net>,
Andrew Morton <akpm@linux-foundation.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Hugh Dickins <hughd@google.com>,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
Mel Gorman <mgorman@suse.de>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH, RFC 0/6] Avoid cache trashing on clearing huge/gigantic page
Date: Wed, 25 Jul 2012 12:28:50 -0700 [thread overview]
Message-ID: <20120725192850.GA4952@tassilo.jf.intel.com> (raw)
In-Reply-To: <alpine.DEB.2.00.1207251346250.4995@router.home>
On Wed, Jul 25, 2012 at 01:51:01PM -0500, Christoph Lameter wrote:
> On Fri, 20 Jul 2012, Kirill A. Shutemov wrote:
>
> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> >
> > Clearing a 2MB huge page will typically blow away several levels of CPU
> > caches. To avoid this only cache clear the 4K area around the fault
> > address and use a cache avoiding clears for the rest of the 2MB area.
>
> why exempt the 4K around the fault address? Is there a regression if that
> is not exempted?
You would get an immediate cache miss when the faulting instruction
is reexecuted.
>
> I guess for anonymous huge pages one may assume that there will be at
> least one write to one cache line in the 4k page. Is it useful to get all
> the cachelines in the page in the cache.
We did some measurements -- comparing 4K and 2MB with some tracing
of fault patterns -- and a lot of apps don't use the full 2MB area.
The apps with THP regressions usually used less than others.
The patchkit significantly reduced some of the regressions.
>
> Also note that if we get later into hugepage use for the page cache we
> would want the cache to be cold because the contents have to come in from
> a storage medium.
Page cache is not cleared, so never runs this code.
-Andi
--
ak@linux.intel.com -- Speaking for myself only
next prev parent reply other threads:[~2012-07-25 19:28 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-07-20 12:50 [PATCH, RFC 0/6] Avoid cache trashing on clearing huge/gigantic page Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-20 12:50 ` [PATCH, RFC 1/6] THP: Use real address for NUMA policy Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-20 12:50 ` [PATCH, RFC 2/6] mm: make clear_huge_page tolerate non aligned address Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-20 12:50 ` [PATCH, RFC 3/6] THP: Pass real, not rounded, address to clear_huge_page Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-20 12:50 ` [PATCH, RFC 4/6] x86: Add clear_page_nocache Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-20 12:50 ` [PATCH, RFC 5/6] mm: make clear_huge_page cache clear only around the fault address Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-20 12:50 ` [PATCH, RFC 6/6] x86: switch the 64bit uncached page clear to SSE/AVX v2 Kirill A. Shutemov
2012-07-20 12:50 ` Kirill A. Shutemov
2012-07-23 23:30 ` [PATCH, RFC 0/6] Avoid cache trashing on clearing huge/gigantic page Andrew Morton
2012-07-23 23:30 ` Andrew Morton
2012-07-24 14:09 ` Kirill A. Shutemov
2012-07-25 18:51 ` Christoph Lameter
2012-07-25 18:51 ` Christoph Lameter
2012-07-25 19:28 ` Andi Kleen [this message]
2012-07-25 19:28 ` Andi Kleen
2012-07-25 19:38 ` Christoph Lameter
2012-07-25 19:38 ` Christoph Lameter
2012-08-09 15:18 ` H. Peter Anvin
2012-08-09 15:18 ` H. Peter Anvin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120725192850.GA4952@tassilo.jf.intel.com \
--to=ak@linux.intel.com \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=alex.shu@intel.com \
--cc=cl@linux.com \
--cc=hannes@cmpxchg.org \
--cc=hpa@zytor.com \
--cc=hughd@google.com \
--cc=jbeulich@novell.com \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@amacapital.net \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=robert.richter@amd.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.