From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Russell King - ARM Linux <linux@arm.linux.org.uk>
Cc: Mel Gorman <mel@csn.ul.ie>,
KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
Gilad Ben-Yossef <gilad@benyossef.com>,
linux-kernel@vger.kernel.org, Chris Metcalf <cmetcalf@tilera.com>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Frederic Weisbecker <fweisbec@gmail.com>,
linux-mm@kvack.org, Pekka Enberg <penberg@kernel.org>,
Matt Mackall <mpm@selenic.com>,
Sasha Levin <levinsasha928@gmail.com>,
Rik van Riel <riel@redhat.com>, Andi Kleen <andi@firstfloor.org>,
Andrew Morton <akpm@linux-foundation.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
linux-fsdevel@vger.kernel.org, Avi Kivity <avi@redhat.com>
Subject: Re: [PATCH v5 7/8] mm: Only IPI CPUs to drain local pages if they exist
Date: Thu, 5 Jan 2012 10:35:04 -0800 [thread overview]
Message-ID: <20120105183504.GF2393@linux.vnet.ibm.com> (raw)
In-Reply-To: <20120105163529.GA11810@n2100.arm.linux.org.uk>
On Thu, Jan 05, 2012 at 04:35:29PM +0000, Russell King - ARM Linux wrote:
> On Thu, Jan 05, 2012 at 04:17:39PM +0000, Mel Gorman wrote:
> > Link please?
>
> Forwarded, as its still in my mailbox.
>
> > I'm including a patch below under development that is
> > intended to only cope with the page allocator case under heavy memory
> > pressure. Currently it does not pass testing because eventually RCU
> > gets stalled with the following trace
> >
> > [ 1817.176001] [<ffffffff810214d7>] arch_trigger_all_cpu_backtrace+0x87/0xa0
> > [ 1817.176001] [<ffffffff810c4779>] __rcu_pending+0x149/0x260
> > [ 1817.176001] [<ffffffff810c48ef>] rcu_check_callbacks+0x5f/0x110
> > [ 1817.176001] [<ffffffff81068d7f>] update_process_times+0x3f/0x80
> > [ 1817.176001] [<ffffffff8108c4eb>] tick_sched_timer+0x5b/0xc0
> > [ 1817.176001] [<ffffffff8107f28e>] __run_hrtimer+0xbe/0x1a0
> > [ 1817.176001] [<ffffffff8107f581>] hrtimer_interrupt+0xc1/0x1e0
> > [ 1817.176001] [<ffffffff81020ef3>] smp_apic_timer_interrupt+0x63/0xa0
> > [ 1817.176001] [<ffffffff81449073>] apic_timer_interrupt+0x13/0x20
> > [ 1817.176001] [<ffffffff8116c135>] vfsmount_lock_local_lock+0x25/0x30
> > [ 1817.176001] [<ffffffff8115c855>] path_init+0x2d5/0x370
> > [ 1817.176001] [<ffffffff8115eecd>] path_lookupat+0x2d/0x620
> > [ 1817.176001] [<ffffffff8115f4ef>] do_path_lookup+0x2f/0xd0
> > [ 1817.176001] [<ffffffff811602af>] user_path_at_empty+0x9f/0xd0
> > [ 1817.176001] [<ffffffff81154e7b>] vfs_fstatat+0x4b/0x90
> > [ 1817.176001] [<ffffffff81154f4f>] sys_newlstat+0x1f/0x50
> > [ 1817.176001] [<ffffffff81448692>] system_call_fastpath+0x16/0x1b
> >
> > It might be a separate bug, don't know for sure.
Do you get multiple RCU CPU stall-warning messages? If so, it can
be helpful to look at how the stack frame changes over time. These
stalls are normally caused by a loop in the kernel with preemption
disabled, though other scenarios can also cause them.
I am assuming that the CPU is reporting a stall on itself in this case.
If not, then it is necessary to look at the stack of the CPU that the
stall is being reported for.
Thanx, Paul
> I'm not going to even pretend to understand what the above backtrace
> means: it doesn't look like what I'd expect from the problem which
> PeterZ's patch is supposed to address. It certainly doesn't do anything
> to address the cpu-going-offline problem you seem to have found.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Russell King - ARM Linux <linux@arm.linux.org.uk>
Cc: Mel Gorman <mel@csn.ul.ie>,
KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
Gilad Ben-Yossef <gilad@benyossef.com>,
linux-kernel@vger.kernel.org, Chris Metcalf <cmetcalf@tilera.com>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Frederic Weisbecker <fweisbec@gmail.com>,
linux-mm@kvack.org, Pekka Enberg <penberg@kernel.org>,
Matt Mackall <mpm@selenic.com>,
Sasha Levin <levinsasha928@gmail.com>,
Rik van Riel <riel@redhat.com>, Andi Kleen <andi@firstfloor.org>,
Andrew Morton <akpm@linux-foundation.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
linux-fsdevel@vger.kernel.org, Avi Kivity <avi@redhat.com>
Subject: Re: [PATCH v5 7/8] mm: Only IPI CPUs to drain local pages if they exist
Date: Thu, 5 Jan 2012 10:35:04 -0800 [thread overview]
Message-ID: <20120105183504.GF2393@linux.vnet.ibm.com> (raw)
In-Reply-To: <20120105163529.GA11810@n2100.arm.linux.org.uk>
On Thu, Jan 05, 2012 at 04:35:29PM +0000, Russell King - ARM Linux wrote:
> On Thu, Jan 05, 2012 at 04:17:39PM +0000, Mel Gorman wrote:
> > Link please?
>
> Forwarded, as its still in my mailbox.
>
> > I'm including a patch below under development that is
> > intended to only cope with the page allocator case under heavy memory
> > pressure. Currently it does not pass testing because eventually RCU
> > gets stalled with the following trace
> >
> > [ 1817.176001] [<ffffffff810214d7>] arch_trigger_all_cpu_backtrace+0x87/0xa0
> > [ 1817.176001] [<ffffffff810c4779>] __rcu_pending+0x149/0x260
> > [ 1817.176001] [<ffffffff810c48ef>] rcu_check_callbacks+0x5f/0x110
> > [ 1817.176001] [<ffffffff81068d7f>] update_process_times+0x3f/0x80
> > [ 1817.176001] [<ffffffff8108c4eb>] tick_sched_timer+0x5b/0xc0
> > [ 1817.176001] [<ffffffff8107f28e>] __run_hrtimer+0xbe/0x1a0
> > [ 1817.176001] [<ffffffff8107f581>] hrtimer_interrupt+0xc1/0x1e0
> > [ 1817.176001] [<ffffffff81020ef3>] smp_apic_timer_interrupt+0x63/0xa0
> > [ 1817.176001] [<ffffffff81449073>] apic_timer_interrupt+0x13/0x20
> > [ 1817.176001] [<ffffffff8116c135>] vfsmount_lock_local_lock+0x25/0x30
> > [ 1817.176001] [<ffffffff8115c855>] path_init+0x2d5/0x370
> > [ 1817.176001] [<ffffffff8115eecd>] path_lookupat+0x2d/0x620
> > [ 1817.176001] [<ffffffff8115f4ef>] do_path_lookup+0x2f/0xd0
> > [ 1817.176001] [<ffffffff811602af>] user_path_at_empty+0x9f/0xd0
> > [ 1817.176001] [<ffffffff81154e7b>] vfs_fstatat+0x4b/0x90
> > [ 1817.176001] [<ffffffff81154f4f>] sys_newlstat+0x1f/0x50
> > [ 1817.176001] [<ffffffff81448692>] system_call_fastpath+0x16/0x1b
> >
> > It might be a separate bug, don't know for sure.
Do you get multiple RCU CPU stall-warning messages? If so, it can
be helpful to look at how the stack frame changes over time. These
stalls are normally caused by a loop in the kernel with preemption
disabled, though other scenarios can also cause them.
I am assuming that the CPU is reporting a stall on itself in this case.
If not, then it is necessary to look at the stack of the CPU that the
stall is being reported for.
Thanx, Paul
> I'm not going to even pretend to understand what the above backtrace
> means: it doesn't look like what I'd expect from the problem which
> PeterZ's patch is supposed to address. It certainly doesn't do anything
> to address the cpu-going-offline problem you seem to have found.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
next prev parent reply other threads:[~2012-01-05 18:35 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-01-02 10:24 [PATCH v5 0/8] Reduce cross CPU IPI interference Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 1/8] smp: Introduce a generic on_each_cpu_mask function Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-03 7:51 ` Michal Nazarewicz
2012-01-03 7:51 ` Michal Nazarewicz
2012-01-03 7:51 ` Michal Nazarewicz
2012-01-03 8:12 ` Gilad Ben-Yossef
2012-01-03 8:12 ` Gilad Ben-Yossef
2012-01-03 8:57 ` Michal Nazarewicz
2012-01-03 8:57 ` Michal Nazarewicz
2012-01-03 8:57 ` Michal Nazarewicz
2012-01-03 22:26 ` Andrew Morton
2012-01-03 22:26 ` Andrew Morton
2012-01-05 13:17 ` Michal Nazarewicz
2012-01-05 13:17 ` Michal Nazarewicz
2012-01-08 16:04 ` Gilad Ben-Yossef
2012-01-08 16:04 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 2/8] arm: Move arm over to generic on_each_cpu_mask Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 3/8] tile: Move tile to use " Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 4/8] smp: Add func to IPI cpus based on parameter func Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-03 22:34 ` Andrew Morton
2012-01-03 22:34 ` Andrew Morton
2012-01-08 16:09 ` Gilad Ben-Yossef
2012-01-08 16:09 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 5/8] slub: Only IPI CPUs that have per cpu obj to flush Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 6/8] fs: only send IPI to invalidate LRU BH when needed Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 7/8] mm: Only IPI CPUs to drain local pages if they exist Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-03 17:45 ` KOSAKI Motohiro
2012-01-03 17:45 ` KOSAKI Motohiro
2012-01-03 18:58 ` Gilad Ben-Yossef
2012-01-03 18:58 ` Gilad Ben-Yossef
2012-01-03 22:02 ` KOSAKI Motohiro
2012-01-03 22:02 ` KOSAKI Motohiro
2012-01-05 14:20 ` Mel Gorman
2012-01-05 14:20 ` Mel Gorman
2012-01-05 14:40 ` Russell King - ARM Linux
2012-01-05 14:40 ` Russell King - ARM Linux
2012-01-05 15:24 ` Peter Zijlstra
2012-01-05 15:24 ` Peter Zijlstra
2012-01-05 16:17 ` Mel Gorman
2012-01-05 16:17 ` Mel Gorman
2012-01-05 16:35 ` Russell King - ARM Linux
2012-01-05 16:35 ` Russell King - ARM Linux
2012-01-05 18:35 ` Paul E. McKenney [this message]
2012-01-05 18:35 ` Paul E. McKenney
2012-01-05 22:21 ` Mel Gorman
2012-01-05 22:21 ` Mel Gorman
2012-01-06 6:06 ` Srivatsa S. Bhat
2012-01-06 6:06 ` Srivatsa S. Bhat
2012-01-06 10:46 ` Mel Gorman
2012-01-06 10:46 ` Mel Gorman
2012-01-06 13:28 ` Greg KH
2012-01-06 13:28 ` Greg KH
2012-01-06 14:09 ` Mel Gorman
2012-01-06 14:09 ` Mel Gorman
2012-01-05 22:06 ` Andrew Morton
2012-01-05 22:06 ` Andrew Morton
2012-01-05 22:31 ` Mel Gorman
2012-01-05 22:31 ` Mel Gorman
2012-01-05 23:19 ` Andrew Morton
2012-01-05 23:19 ` Andrew Morton
2012-01-09 17:25 ` Mel Gorman
2012-01-09 17:25 ` Mel Gorman
2012-01-07 16:52 ` Paul E. McKenney
2012-01-07 16:52 ` Paul E. McKenney
2012-01-07 17:05 ` Paul E. McKenney
2012-01-07 17:05 ` Paul E. McKenney
2012-01-05 15:54 ` Mel Gorman
2012-01-05 15:54 ` Mel Gorman
2012-01-08 16:01 ` Gilad Ben-Yossef
2012-01-08 16:01 ` Gilad Ben-Yossef
2012-01-02 10:24 ` [PATCH v5 8/8] mm: add vmstat counters for tracking PCP drains Gilad Ben-Yossef
2012-01-02 10:24 ` Gilad Ben-Yossef
2012-01-03 17:47 ` KOSAKI Motohiro
2012-01-03 17:47 ` KOSAKI Motohiro
2012-01-03 19:00 ` Gilad Ben-Yossef
2012-01-03 19:00 ` Gilad Ben-Yossef
2012-01-03 22:13 ` KOSAKI Motohiro
2012-01-03 22:13 ` KOSAKI Motohiro
2012-01-03 22:37 ` Andrew Morton
2012-01-03 22:37 ` Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120105183504.GF2393@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=avi@redhat.com \
--cc=cmetcalf@tilera.com \
--cc=fweisbec@gmail.com \
--cc=gilad@benyossef.com \
--cc=kosaki.motohiro@gmail.com \
--cc=levinsasha928@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux@arm.linux.org.uk \
--cc=mel@csn.ul.ie \
--cc=mpm@selenic.com \
--cc=penberg@kernel.org \
--cc=riel@redhat.com \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.