From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
LKML <linux-kernel@vger.kernel.org>,
Alessio Igor Bogani <abogani@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Avi Kivity <avi@redhat.com>, Chris Metcalf <cmetcalf@tilera.com>,
Christoph Lameter <cl@linux.com>,
Geoff Levand <geoff@infradead.org>,
Gilad Ben Yossef <gilad@benyossef.com>,
Hakan Akkan <hakanakkan@gmail.com>,
Ingo Molnar <mingo@kernel.org>,
Paul Gortmaker <paul.gortmaker@windriver.com>,
Peter Zijlstra <peterz@infradead.org>,
Thomas Gleixner <tglx@linutronix.de>,
Li Zhong <zhong@linux.vnet.ibm.com>
Subject: Re: [ANNOUNCE] 3.7-nohz1
Date: Sat, 29 Dec 2012 19:56:38 -0800 [thread overview]
Message-ID: <20121230035638.GM2542@linux.vnet.ibm.com> (raw)
In-Reply-To: <CAFTL4hy_k8rWg5EAFnh-owey7NJCQbK+ULrkNrSjgdynzEseTw@mail.gmail.com>
On Mon, Dec 24, 2012 at 12:43:25AM +0100, Frederic Weisbecker wrote:
> 2012/12/21 Steven Rostedt <rostedt@goodmis.org>:
> > On Thu, 2012-12-20 at 19:32 +0100, Frederic Weisbecker wrote:
> >> Let's imagine you have 4 CPUs. We keep the CPU 0 to offline RCU callbacks there and to
> >> handle the timekeeping. We set the rest as full dynticks. So you need the following kernel
> >> parameters:
> >>
> >> rcu_nocbs=1-3 full_nohz=1-3
> >>
> >> (Note rcu_nocbs value must always be the same as full_nohz).
> >
> > Why? You can't have: rcu_nocbs=1-4 full_nohz=1-3
>
> That should be allowed.
>
> > or: rcu_nocbs=1-3 full_nohz=1-4 ?
>
> But that not.
>
> You need to have: rcu_nocbs & full_nohz == full_nohz. This is because
> the tick is not there to maintain the local RCU callbacks anymore. So
> this must be offloaded to the rcu_nocb threads.
>
> I just have a doubt with rcu_nocb. Do we still need the tick to
> complete the grace period for local rcu callbacks? I need to discuss
> that with Paul.
The tick is only needed if rcu_needs_cpu() returns false. Of course,
this means that if you don't invoke rcu_needs_cpu() before returning to
adaptive-idle usermode execution, you are correct that a full_nohz CPU
would also have to be a rcu_nocbs CPU.
That said, I am getting close to having an rcu_needs_cpu() that only
returns false if there are callbacks immediately ready to invoke, at
least if RCU_FAST_NO_HZ=y.
Thanx, Paul
> > That needs to be fixed. Either with a warning, and/or to force the two
> > to be the same. That is, if they specify:
> >
> > rcu_nocbs=1-3 full_nohz=1-4
> >
> > Then set rcu_nocbs=1-4 with a warning about it. Or simply set
> > full_nohz=1-3.
>
> Yep, will do.
>
> Thanks!
>
> >
> > -- Steve
> >
> >>
> >> Now if you want proper isolation you need to:
> >>
> >> * Migrate your processes adequately
> >> * Migrate your irqs to CPU 0
> >> * Migrate the RCU nocb threads to CPU 0. Example with the above configuration:
> >>
> >> for p in $(ps -o pid= -C rcuo1,rcuo2,rcuo3)
> >> do
> >> taskset -cp 0 $p
> >> done
> >>
> >> Then run what you want on the full dynticks CPUs. For best results, run 1 task
> >> per CPU, mostly in userspace and mostly CPU bound (otherwise more IO = more kernel
> >> mode execution = more chances to get IPIs, tick restarted, workqueues, kthreads, etc...)
> >>
> >> This page contains a good reminder for those interested in CPU isolation: https://github.com/gby/linux/wiki
> >>
> >> But keep in mind that my tree is not yet ready for serious production.
> >>
> >
> >
>
next prev parent reply other threads:[~2012-12-30 3:57 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-12-20 18:32 [ANNOUNCE] 3.7-nohz1 Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 01/24] context_tracking: Add comments on interface and internals Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 02/24] cputime: Generic on-demand virtual cputime accounting Frederic Weisbecker
2012-12-21 5:11 ` Steven Rostedt
2012-12-26 8:19 ` Li Zhong
2012-12-29 13:15 ` Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 03/24] cputime: Allow dynamic switch between tick/virtual based " Frederic Weisbecker
2012-12-21 15:05 ` Steven Rostedt
2012-12-22 17:43 ` Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 04/24] cputime: Use accessors to read task cputime stats Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 05/24] cputime: Safely read cputime of full dynticks CPUs Frederic Weisbecker
2012-12-21 15:09 ` Steven Rostedt
2012-12-22 17:51 ` Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 06/24] nohz: Basic full dynticks interface Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 07/24] nohz: Assign timekeeping duty to a non-full-nohz CPU Frederic Weisbecker
2012-12-21 16:13 ` Steven Rostedt
2012-12-22 16:39 ` Frederic Weisbecker
2012-12-22 17:05 ` Steven Rostedt
2012-12-20 18:32 ` [PATCH 08/24] nohz: Trace timekeeping update Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 09/24] nohz: Wake up full dynticks CPUs when a timer gets enqueued Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 10/24] rcu: Restart the tick on non-responding full dynticks CPUs Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 11/24] sched: Comment on rq->clock correctness in ttwu_do_wakeup() in nohz Frederic Weisbecker
2012-12-20 18:32 ` [PATCH 12/24] sched: Update rq clock on nohz CPU before migrating tasks Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 13/24] sched: Update rq clock on nohz CPU before setting fair group shares Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 14/24] sched: Update rq clock on tickless CPUs before calling check_preempt_curr() Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 15/24] sched: Update rq clock earlier in unthrottle_cfs_rq Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 16/24] sched: Update clock of nohz busiest rq before balancing Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 17/24] sched: Update rq clock before idle balancing Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 18/24] sched: Update nohz rq clock before searching busiest group on load balancing Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 19/24] nohz: Move nohz load balancer selection into idle logic Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 20/24] nohz: Full dynticks mode Frederic Weisbecker
2012-12-26 6:12 ` Namhyung Kim
2012-12-26 7:02 ` Namhyung Kim
2012-12-29 13:21 ` Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 21/24] nohz: Only stop the tick on RCU nocb CPUs Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 22/24] nohz: Don't turn off the tick if rcu needs it Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 23/24] nohz: Don't stop the tick if posix cpu timers are running Frederic Weisbecker
2012-12-20 18:33 ` [PATCH 24/24] nohz: Add some tracing Frederic Weisbecker
2012-12-21 2:35 ` [ANNOUNCE] 3.7-nohz1 Steven Rostedt
2012-12-23 23:43 ` Frederic Weisbecker
2012-12-30 3:56 ` Paul E. McKenney [this message]
2013-01-04 23:42 ` Frederic Weisbecker
2013-01-07 13:06 ` Paul E. McKenney
2012-12-21 5:20 ` Hakan Akkan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121230035638.GM2542@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=abogani@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=avi@redhat.com \
--cc=cl@linux.com \
--cc=cmetcalf@tilera.com \
--cc=fweisbec@gmail.com \
--cc=geoff@infradead.org \
--cc=gilad@benyossef.com \
--cc=hakanakkan@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paul.gortmaker@windriver.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=zhong@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.