From: Steven Rostedt <rostedt@goodmis.org>
To: linux-kernel@vger.kernel.org,
linux-rt-users <linux-rt-users@vger.kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
Carsten Emde <C.Emde@osadl.org>, John Kacur <jkacur@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Clark Williams <clark.williams@gmail.com>,
Ingo Molnar <mingo@kernel.org>,
Frank Rowand <frank.rowand@am.sony.com>,
Mike Galbraith <bitbucket@online.de>
Subject: [RFC][PATCH RT 0/4 v2] sched/rt: Lower rq lock contention latencies on many CPU boxes
Date: Wed, 12 Dec 2012 14:27:33 -0500 [thread overview]
Message-ID: <20121212192733.221810086@goodmis.org> (raw)
This version I rearranged the patches a little so that the IPI patch
comes last. As the first 3 patches are less controversal, and should
probably be added now.
The difference in this version was that I added more comments, but
more importantly I also added a sched feature called RT_PUSH_IPI.
When this feature is enabled, it switches the pull logic to send
an IPI to the RT overloaded CPU to do a push instead of doing the
pull locally.
When the feature is disabled, the push/pull logic stays the same as
it always has (with rq lock contention).
Now I've discussed this with Clark, and we noticed that the contention
didn't show up until we tried it on a 24 CPU machine. On a 16 CPU machine
it ran fine. Thus, by default, machines with 16 or less CPUs will
have the RT_PUSH_IPI feature disabled. Machines with 17 or more
possible CPUs will have the feature enabled at boot up. Note, we haven't
tried it on a machine with 17 to 23 CPUs.
It is safe to enable or disable this feature at run time, although
you may cause latencies in doing so, but there shouldn't be any missed
wakeups or anything else that is serious. The worse that can happen
is that you miss a pull, and an RT task will stay on its CPU when it
could have migrated to another CPU that just lowered its priority.
Well, if you are worried about that, don't change it when you care :-)
The 16 CPUs is just a heuristic, and people may debate it. And perhaps
you may not like how the push/pull default changes between different
machines. I could also add a command line switch to force enable/disable
at boot, and/or I could add a config as well.
Right now, by default <= 16 CPU machines are as it always was, and
17 or more CPU machines have this new logic enabled. Either one can
change it at run time via the debugfs directory (unfortunately it's
not a sysctl).
Comments?
-- Steve
next reply other threads:[~2012-12-12 19:27 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-12-12 19:27 Steven Rostedt [this message]
2012-12-12 19:27 ` [RFC][PATCH RT 1/4 v2] sched/rt: Fix push_rt_task() to have the same checks as the caller did Steven Rostedt
2012-12-12 19:27 ` [RFC][PATCH RT 2/4 v2] sched/rt: Try to migrate task if preempting pinned rt task Steven Rostedt
2012-12-12 19:27 ` [RFC][PATCH RT 3/4 v2] sched/rt: Initiate a pull when the priority of a task is lowered Steven Rostedt
2012-12-12 19:27 ` [RFC][PATCH RT 4/4 v2] sched/rt: Use IPI to trigger RT task push migration instead of pulling Steven Rostedt
2012-12-12 20:44 ` Steven Rostedt
2012-12-13 19:53 ` Steven Rostedt
2012-12-21 15:42 ` Mike Galbraith
2013-02-13 16:49 ` John Kacur
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121212192733.221810086@goodmis.org \
--to=rostedt@goodmis.org \
--cc=C.Emde@osadl.org \
--cc=bitbucket@online.de \
--cc=clark.williams@gmail.com \
--cc=frank.rowand@am.sony.com \
--cc=jkacur@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).