* [PATCH RT 01/19] sched/workqueue: Only wake up idle workers if not blocked on sleeping spin lock
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
@ 2013-09-08 15:13 ` Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 02/19] x86/mce: fix mce timer interval Steven Rostedt
` (17 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:13 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0001-sched-workqueue-Only-wake-up-idle-workers-if-not-blo.patch --]
[-- Type: text/plain, Size: 1481 bytes --]
From: Steven Rostedt <rostedt@goodmis.org>
In -rt, most spin_locks() turn into mutexes. One of these spin_lock
conversions is performed on the workqueue gcwq->lock. When the idle
worker is worken, the first thing it will do is grab that same lock and
it too will block, possibly jumping into the same code, but because
nr_running would already be decremented it prevents an infinite loop.
But this is still a waste of CPU cycles, and it doesn't follow the method
of mainline, as new workers should only be woken when a worker thread is
truly going to sleep, and not just blocked on a spin_lock().
Check the saved_state too before waking up new workers.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
kernel/sched/core.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5051ca6..781e6c4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3632,8 +3632,10 @@ need_resched:
* If a worker went to sleep, notify and ask workqueue
* whether it wants to wake up a task to maintain
* concurrency.
+ * Only call wake up if prev isn't blocked on a sleeping
+ * spin lock.
*/
- if (prev->flags & PF_WQ_WORKER) {
+ if (prev->flags & PF_WQ_WORKER && !prev->saved_state) {
struct task_struct *to_wakeup;
to_wakeup = wq_worker_sleeping(prev, cpu);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 02/19] x86/mce: fix mce timer interval
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 01/19] sched/workqueue: Only wake up idle workers if not blocked on sleeping spin lock Steven Rostedt
@ 2013-09-08 15:13 ` Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 03/19] genirq: Set irq thread to RT priority on creation Steven Rostedt
` (16 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:13 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Mike Galbraith
[-- Attachment #1: 0002-x86-mce-fix-mce-timer-interval.patch --]
[-- Type: text/plain, Size: 1107 bytes --]
From: Mike Galbraith <bitbucket@online.de>
Seems mce timer fire at the wrong frequency in -rt kernels since roughly
forever due to 32 bit overflow. 3.8-rt is also missing a multiplier.
Add missing us -> ns conversion and 32 bit overflow prevention.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Mike Galbraith <bitbucket@online.de>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
[bigeasy: use ULL instead of u64 cast]
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
arch/x86/kernel/cpu/mcheck/mce.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/mcheck/mce.c b/arch/x86/kernel/cpu/mcheck/mce.c
index e31ea90..339bb4d0 100644
--- a/arch/x86/kernel/cpu/mcheck/mce.c
+++ b/arch/x86/kernel/cpu/mcheck/mce.c
@@ -1649,7 +1649,7 @@ static void __mcheck_cpu_init_timer(void)
__this_cpu_write(mce_next_interval, iv);
if (!iv)
return;
- hrtimer_start_range_ns(t, ns_to_ktime(jiffies_to_usecs(iv) * 1000),
+ hrtimer_start_range_ns(t, ns_to_ktime(jiffies_to_usecs(iv) * 1000ULL),
0, HRTIMER_MODE_REL_PINNED);
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 03/19] genirq: Set irq thread to RT priority on creation
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 01/19] sched/workqueue: Only wake up idle workers if not blocked on sleeping spin lock Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 02/19] x86/mce: fix mce timer interval Steven Rostedt
@ 2013-09-08 15:13 ` Steven Rostedt
2013-09-08 15:13 ` [PATCH RT 04/19] list_bl.h: make list head locking RT safe Steven Rostedt
` (15 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:13 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Ivo Sieben
[-- Attachment #1: 0003-genirq-Set-irq-thread-to-RT-priority-on-creation.patch --]
[-- Type: text/plain, Size: 2409 bytes --]
From: Ivo Sieben <meltedpianoman@gmail.com>
When a threaded irq handler is installed the irq thread is initially
created on normal scheduling priority. Only after the irq thread is
woken up it sets its priority to RT_FIFO MAX_USER_RT_PRIO/2 itself.
This means that interrupts that occur directly after the irq handler
is installed will be handled on a normal scheduling priority instead
of the realtime priority that one would expect.
Fix this by setting the RT priority on creation of the irq_thread.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Ivo Sieben <meltedpianoman@gmail.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/r/1370254322-17240-1-git-send-email-meltedpianoman@gmail.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
kernel/irq/manage.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 478ade0..2ecc232 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -824,9 +824,6 @@ static void irq_thread_dtor(struct callback_head *unused)
static int irq_thread(void *data)
{
struct callback_head on_exit_work;
- static const struct sched_param param = {
- .sched_priority = MAX_USER_RT_PRIO/2,
- };
struct irqaction *action = data;
struct irq_desc *desc = irq_to_desc(action->irq);
irqreturn_t (*handler_fn)(struct irq_desc *desc,
@@ -838,8 +835,6 @@ static int irq_thread(void *data)
else
handler_fn = irq_thread_fn;
- sched_setscheduler(current, SCHED_FIFO, ¶m);
-
init_task_work(&on_exit_work, irq_thread_dtor);
task_work_add(current, &on_exit_work, false);
@@ -938,6 +933,9 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
*/
if (new->thread_fn && !nested) {
struct task_struct *t;
+ static const struct sched_param param = {
+ .sched_priority = MAX_USER_RT_PRIO/2,
+ };
t = kthread_create(irq_thread, new, "irq/%d-%s", irq,
new->name);
@@ -945,6 +943,9 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
ret = PTR_ERR(t);
goto out_mput;
}
+
+ sched_setscheduler(t, SCHED_FIFO, ¶m);
+
/*
* We keep the reference to the task struct even if
* the thread dies to avoid that the interrupt code
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 04/19] list_bl.h: make list head locking RT safe
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (2 preceding siblings ...)
2013-09-08 15:13 ` [PATCH RT 03/19] genirq: Set irq thread to RT priority on creation Steven Rostedt
@ 2013-09-08 15:13 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 05/19] list_bl.h: fix it for for !SMP && !DEBUG_SPINLOCK Steven Rostedt
` (14 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:13 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Paul Gortmaker
[-- Attachment #1: 0004-list_bl.h-make-list-head-locking-RT-safe.patch --]
[-- Type: text/plain, Size: 3645 bytes --]
From: Paul Gortmaker <paul.gortmaker@windriver.com>
As per changes in include/linux/jbd_common.h for avoiding the
bit_spin_locks on RT ("fs: jbd/jbd2: Make state lock and journal
head lock rt safe") we do the same thing here.
We use the non atomic __set_bit and __clear_bit inside the scope of
the lock to preserve the ability of the existing LIST_DEBUG code to
use the zero'th bit in the sanity checks.
As a bit spinlock, we had no lockdep visibility into the usage
of the list head locking. Now, if we were to implement it as a
standard non-raw spinlock, we would see:
BUG: sleeping function called from invalid context at kernel/rtmutex.c:658
in_atomic(): 1, irqs_disabled(): 0, pid: 122, name: udevd
5 locks held by udevd/122:
#0: (&sb->s_type->i_mutex_key#7/1){+.+.+.}, at: [<ffffffff811967e8>] lock_rename+0xe8/0xf0
#1: (rename_lock){+.+...}, at: [<ffffffff811a277c>] d_move+0x2c/0x60
#2: (&dentry->d_lock){+.+...}, at: [<ffffffff811a0763>] dentry_lock_for_move+0xf3/0x130
#3: (&dentry->d_lock/2){+.+...}, at: [<ffffffff811a0734>] dentry_lock_for_move+0xc4/0x130
#4: (&dentry->d_lock/3){+.+...}, at: [<ffffffff811a0747>] dentry_lock_for_move+0xd7/0x130
Pid: 122, comm: udevd Not tainted 3.4.47-rt62 #7
Call Trace:
[<ffffffff810b9624>] __might_sleep+0x134/0x1f0
[<ffffffff817a24d4>] rt_spin_lock+0x24/0x60
[<ffffffff811a0c4c>] __d_shrink+0x5c/0xa0
[<ffffffff811a1b2d>] __d_drop+0x1d/0x40
[<ffffffff811a24be>] __d_move+0x8e/0x320
[<ffffffff811a278e>] d_move+0x3e/0x60
[<ffffffff81199598>] vfs_rename+0x198/0x4c0
[<ffffffff8119b093>] sys_renameat+0x213/0x240
[<ffffffff817a2de5>] ? _raw_spin_unlock+0x35/0x60
[<ffffffff8107781c>] ? do_page_fault+0x1ec/0x4b0
[<ffffffff817a32ca>] ? retint_swapgs+0xe/0x13
[<ffffffff813eb0e6>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[<ffffffff8119b0db>] sys_rename+0x1b/0x20
[<ffffffff817a3b96>] system_call_fastpath+0x1a/0x1f
Since we are only taking the lock during short lived list operations,
lets assume for now that it being raw won't be a significant latency
concern.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/list_bl.h | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/include/linux/list_bl.h b/include/linux/list_bl.h
index 31f9d75..ddfd46a 100644
--- a/include/linux/list_bl.h
+++ b/include/linux/list_bl.h
@@ -2,6 +2,7 @@
#define _LINUX_LIST_BL_H
#include <linux/list.h>
+#include <linux/spinlock.h>
#include <linux/bit_spinlock.h>
/*
@@ -32,13 +33,22 @@
struct hlist_bl_head {
struct hlist_bl_node *first;
+#ifdef CONFIG_PREEMPT_RT_BASE
+ raw_spinlock_t lock;
+#endif
};
struct hlist_bl_node {
struct hlist_bl_node *next, **pprev;
};
-#define INIT_HLIST_BL_HEAD(ptr) \
- ((ptr)->first = NULL)
+
+static inline void INIT_HLIST_BL_HEAD(struct hlist_bl_head *h)
+{
+ h->first = NULL;
+#ifdef CONFIG_PREEMPT_RT_BASE
+ raw_spin_lock_init(&h->lock);
+#endif
+}
static inline void INIT_HLIST_BL_NODE(struct hlist_bl_node *h)
{
@@ -117,12 +127,22 @@ static inline void hlist_bl_del_init(struct hlist_bl_node *n)
static inline void hlist_bl_lock(struct hlist_bl_head *b)
{
+#ifndef CONFIG_PREEMPT_RT_BASE
bit_spin_lock(0, (unsigned long *)b);
+#else
+ raw_spin_lock(&b->lock);
+ __set_bit(0, (unsigned long *)b);
+#endif
}
static inline void hlist_bl_unlock(struct hlist_bl_head *b)
{
+#ifndef CONFIG_PREEMPT_RT_BASE
__bit_spin_unlock(0, (unsigned long *)b);
+#else
+ __clear_bit(0, (unsigned long *)b);
+ raw_spin_unlock(&b->lock);
+#endif
}
/**
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 05/19] list_bl.h: fix it for for !SMP && !DEBUG_SPINLOCK
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (3 preceding siblings ...)
2013-09-08 15:13 ` [PATCH RT 04/19] list_bl.h: make list head locking RT safe Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 06/19] timers: prepare for full preemption improve Steven Rostedt
` (13 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Paul Gortmaker, Uwe Kleine-König
[-- Attachment #1: 0005-list_bl.h-fix-it-for-for-SMP-DEBUG_SPINLOCK.patch --]
[-- Type: TEXT/PLAIN, Size: 2052 bytes --]
From: =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de>
The patch "list_bl.h: make list head locking RT safe" introduced
an unconditional
__set_bit(0, (unsigned long *)b);
in void hlist_bl_lock(struct hlist_bl_head *b). This clobbers the value
of b->first. When the value of b->first is retrieved using
hlist_bl_first the clobbering is undone using
(unsigned long)h->first & ~LIST_BL_LOCKMASK
and so depending on LIST_BL_LOCKMASK being one. But LIST_BL_LOCKMASK is
only one if at least on of CONFIG_SMP and CONFIG_DEBUG_SPINLOCK are
defined. Without these the value returned by hlist_bl_first has the
zeroth bit set which likely results in a crash.
So only do the clobbering in the cases where LIST_BL_LOCKMASK is one.
An alternative would be to always define LIST_BL_LOCKMASK to one with
CONFIG_PREEMPT_RT_BASE.
Cc: stable-rt@vger.kernel.org
Acked-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/list_bl.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/list_bl.h b/include/linux/list_bl.h
index ddfd46a..becd7a6 100644
--- a/include/linux/list_bl.h
+++ b/include/linux/list_bl.h
@@ -131,8 +131,10 @@ static inline void hlist_bl_lock(struct hlist_bl_head *b)
bit_spin_lock(0, (unsigned long *)b);
#else
raw_spin_lock(&b->lock);
+#if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
__set_bit(0, (unsigned long *)b);
#endif
+#endif
}
static inline void hlist_bl_unlock(struct hlist_bl_head *b)
@@ -140,7 +142,9 @@ static inline void hlist_bl_unlock(struct hlist_bl_head *b)
#ifndef CONFIG_PREEMPT_RT_BASE
__bit_spin_unlock(0, (unsigned long *)b);
#else
+#if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
__clear_bit(0, (unsigned long *)b);
+#endif
raw_spin_unlock(&b->lock);
#endif
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 06/19] timers: prepare for full preemption improve
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (4 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 05/19] list_bl.h: fix it for for !SMP && !DEBUG_SPINLOCK Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 07/19] kernel/cpu: fix cpu down problem if kthreads cpu is going down Steven Rostedt
` (12 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Zhao Hongjiang
[-- Attachment #1: 0006-timers-prepare-for-full-preemption-improve.patch --]
[-- Type: text/plain, Size: 1747 bytes --]
From: Zhao Hongjiang <zhaohongjiang@huawei.com>
wake_up should do nothing on the nort, so we should use wakeup_timer_waiters,
also fix a spell mistake.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Zhao Hongjiang <zhaohongjiang@huawei.com>
[bigeasy: s/CONFIG_PREEMPT_RT_BASE/CONFIG_PREEMPT_RT_FULL/]
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
kernel/timer.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/kernel/timer.c b/kernel/timer.c
index 027ce4e..61b57c1 100644
--- a/kernel/timer.c
+++ b/kernel/timer.c
@@ -76,7 +76,9 @@ struct tvec_root {
struct tvec_base {
spinlock_t lock;
struct timer_list *running_timer;
+#ifdef CONFIG_PREEMPT_RT_FULL
wait_queue_head_t wait_for_running_timer;
+#endif
unsigned long timer_jiffies;
unsigned long next_timer;
unsigned long active_timers;
@@ -1006,7 +1008,7 @@ static void wait_for_running_timer(struct timer_list *timer)
base->running_timer != timer);
}
-# define wakeup_timer_waiters(b) wake_up(&(b)->wait_for_tunning_timer)
+# define wakeup_timer_waiters(b) wake_up(&(b)->wait_for_running_timer)
#else
static inline void wait_for_running_timer(struct timer_list *timer)
{
@@ -1251,7 +1253,7 @@ static inline void __run_timers(struct tvec_base *base)
spin_lock_irq(&base->lock);
}
}
- wake_up(&base->wait_for_running_timer);
+ wakeup_timer_waiters(base);
spin_unlock_irq(&base->lock);
}
@@ -1765,7 +1767,9 @@ static int __cpuinit init_timers_cpu(int cpu)
base = &boot_tvec_bases;
}
spin_lock_init(&base->lock);
+#ifdef CONFIG_PREEMPT_RT_FULL
init_waitqueue_head(&base->wait_for_running_timer);
+#endif
tvec_base_done[cpu] = 1;
} else {
base = per_cpu(tvec_bases, cpu);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 07/19] kernel/cpu: fix cpu down problem if kthreads cpu is going down
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (5 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 06/19] timers: prepare for full preemption improve Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 08/19] kernel/hotplug: restore original cpu mask oncpu/down Steven Rostedt
` (11 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0007-kernel-cpu-fix-cpu-down-problem-if-kthread-s-cpu-is-.patch --]
[-- Type: text/plain, Size: 2635 bytes --]
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
If kthread is pinned to CPUx and CPUx is going down then we get into
trouble:
- first the unplug thread is created
- it will set itself to hp->unplug. As a result, every task that is
going to take a lock, has to leave the CPU.
- the CPU_DOWN_PREPARE notifier are started. The worker thread will
start a new process for the "high priority worker".
Now kthread would like to take a lock but since it can't leave the CPU
it will never complete its task.
We could fire the unplug thread after the notifier but then the cpu is
no longer marked "online" and the unplug thread will run on CPU0 which
was fixed before :)
So instead the unplug thread is started and kept waiting until the
notfier complete their work.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
kernel/cpu.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 295bf9d..22063f3 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -83,6 +83,7 @@ struct hotplug_pcp {
int refcount;
int grab_lock;
struct completion synced;
+ struct completion unplug_wait;
#ifdef CONFIG_PREEMPT_RT_FULL
spinlock_t lock;
#else
@@ -180,6 +181,7 @@ static int sync_unplug_thread(void *data)
{
struct hotplug_pcp *hp = data;
+ wait_for_completion(&hp->unplug_wait);
preempt_disable();
hp->unplug = current;
wait_for_pinned_cpus(hp);
@@ -245,6 +247,14 @@ static void __cpu_unplug_sync(struct hotplug_pcp *hp)
wait_for_completion(&hp->synced);
}
+static void __cpu_unplug_wait(unsigned int cpu)
+{
+ struct hotplug_pcp *hp = &per_cpu(hotplug_pcp, cpu);
+
+ complete(&hp->unplug_wait);
+ wait_for_completion(&hp->synced);
+}
+
/*
* Start the sync_unplug_thread on the target cpu and wait for it to
* complete.
@@ -268,6 +278,7 @@ static int cpu_unplug_begin(unsigned int cpu)
tell_sched_cpu_down_begin(cpu);
init_completion(&hp->synced);
+ init_completion(&hp->unplug_wait);
hp->sync_tsk = kthread_create(sync_unplug_thread, hp, "sync_unplug/%d", cpu);
if (IS_ERR(hp->sync_tsk)) {
@@ -283,8 +294,7 @@ static int cpu_unplug_begin(unsigned int cpu)
* wait for tasks that are going to enter these sections and
* we must not have them block.
*/
- __cpu_unplug_sync(hp);
-
+ wake_up_process(hp->sync_tsk);
return 0;
}
@@ -590,6 +600,8 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
goto out_release;
}
+ __cpu_unplug_wait(cpu);
+
/* Notifiers are done. Don't let any more tasks pin this CPU. */
cpu_unplug_sync(cpu);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 08/19] kernel/hotplug: restore original cpu mask oncpu/down
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (6 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 07/19] kernel/cpu: fix cpu down problem if kthreads cpu is going down Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 09/19] gpu: i915: allow the user not to do the wbinvd Steven Rostedt
` (10 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0008-kernel-hotplug-restore-original-cpu-mask-oncpu-down.patch --]
[-- Type: text/plain, Size: 1818 bytes --]
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
If a task which is allowed to run only on CPU X puts CPU Y down then it
will be allowed on all CPUs but the on CPU Y after it comes back from
kernel. This patch ensures that we don't lose the initial setting unless
the CPU the task is running is going down.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
kernel/cpu.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 22063f3..16ec791 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -563,6 +563,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
.hcpu = hcpu,
};
cpumask_var_t cpumask;
+ cpumask_var_t cpumask_org;
if (num_online_cpus() == 1)
return -EBUSY;
@@ -573,6 +574,12 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
/* Move the downtaker off the unplug cpu */
if (!alloc_cpumask_var(&cpumask, GFP_KERNEL))
return -ENOMEM;
+ if (!alloc_cpumask_var(&cpumask_org, GFP_KERNEL)) {
+ free_cpumask_var(cpumask);
+ return -ENOMEM;
+ }
+
+ cpumask_copy(cpumask_org, tsk_cpus_allowed(current));
cpumask_andnot(cpumask, cpu_online_mask, cpumask_of(cpu));
set_cpus_allowed_ptr(current, cpumask);
free_cpumask_var(cpumask);
@@ -581,7 +588,8 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
if (mycpu == cpu) {
printk(KERN_ERR "Yuck! Still on unplug CPU\n!");
migrate_enable();
- return -EBUSY;
+ err = -EBUSY;
+ goto restore_cpus;
}
cpu_hotplug_begin();
@@ -639,6 +647,9 @@ out_cancel:
cpu_hotplug_done();
if (!err)
cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
+restore_cpus:
+ set_cpus_allowed_ptr(current, cpumask_org);
+ free_cpumask_var(cpumask_org);
return err;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 09/19] gpu: i915: allow the user not to do the wbinvd
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (7 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 08/19] kernel/hotplug: restore original cpu mask oncpu/down Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 10/19] drm/i915: drop trace_i915_gem_ring_dispatch on rt Steven Rostedt
` (9 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0009-gpu-i915-allow-the-user-not-to-do-the-wbinvd.patch --]
[-- Type: text/plain, Size: 2306 bytes --]
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
The wbinvd() renders the system with i915 unusable on RT. Using this
expensive instruction avoids GPU trouble according to
https://bugs.freedesktop.org/show_bug.cgi?id=62191
As a workaround for RT it is recommended to pin each GPU related process
to the same CPU and then disable this instruction via the module
paramter.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/gpu/drm/i915/i915_gem.c | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 18da42c..a3f72cb 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -36,6 +36,7 @@
#include <linux/swap.h>
#include <linux/pci.h>
#include <linux/dma-buf.h>
+#include <linux/module.h>
static __must_check int i915_gem_object_flush_gpu_write_domain(struct drm_i915_gem_object *obj);
static void i915_gem_object_flush_gtt_write_domain(struct drm_i915_gem_object *obj);
@@ -2525,6 +2526,10 @@ static inline int fence_number(struct drm_i915_private *dev_priv,
return fence - dev_priv->fence_regs;
}
+static bool do_wbinvd = true;
+module_param(do_wbinvd, bool, 0644);
+MODULE_PARM_DESC(do_wbinvd, "Do expensive synchronization. Say no after you pin each GPU process to the same CPU in order to lower the latency.");
+
static void i915_gem_write_fence__ipi(void *data)
{
wbinvd();
@@ -2548,8 +2553,16 @@ static void i915_gem_object_update_fence(struct drm_i915_gem_object *obj,
* on each processor in order to manually flush all memory
* transactions before updating the fence register.
*/
- if (HAS_LLC(obj->base.dev))
- on_each_cpu(i915_gem_write_fence__ipi, NULL, 1);
+ if (HAS_LLC(obj->base.dev)) {
+ if (do_wbinvd) {
+#ifdef CONFIG_PREEMPT_RT_FULL
+ pr_err_once("WARNING! The i915 invalidates all caches which increases the latency.");
+ pr_err_once("As a workaround use 'i915.do_wbinvd=no' and PIN each process doing ");
+ pr_err_once("any kind of GPU activity to the same CPU to avoid problems.");
+#endif
+ on_each_cpu(i915_gem_write_fence__ipi, NULL, 1);
+ }
+ }
i915_gem_write_fence(dev, fence_reg, enable ? obj : NULL);
if (enable) {
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 10/19] drm/i915: drop trace_i915_gem_ring_dispatch on rt
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (8 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 09/19] gpu: i915: allow the user not to do the wbinvd Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 11/19] sched: Distangle worker accounting from rqlock Steven Rostedt
` (8 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Joakim Hernberg
[-- Attachment #1: 0010-drm-i915-drop-trace_i915_gem_ring_dispatch-on-rt.patch --]
[-- Type: text/plain, Size: 2246 bytes --]
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
This tracepoint is responsible for:
|[<814cc358>] __schedule_bug+0x4d/0x59
|[<814d24cc>] __schedule+0x88c/0x930
|[<814d3b90>] ? _raw_spin_unlock_irqrestore+0x40/0x50
|[<814d3b95>] ? _raw_spin_unlock_irqrestore+0x45/0x50
|[<810b57b5>] ? task_blocks_on_rt_mutex+0x1f5/0x250
|[<814d27d9>] schedule+0x29/0x70
|[<814d3423>] rt_spin_lock_slowlock+0x15b/0x278
|[<814d3786>] rt_spin_lock+0x26/0x30
|[<a00dced9>] gen6_gt_force_wake_get+0x29/0x60 [i915]
|[<a00e183f>] gen6_ring_get_irq+0x5f/0x100 [i915]
|[<a00b2a33>] ftrace_raw_event_i915_gem_ring_dispatch+0xe3/0x100 [i915]
|[<a00ac1b3>] i915_gem_do_execbuffer.isra.13+0xbd3/0x1430 [i915]
|[<810f8943>] ? trace_buffer_unlock_commit+0x43/0x60
|[<8113e8d2>] ? ftrace_raw_event_kmem_alloc+0xd2/0x180
|[<8101d063>] ? native_sched_clock+0x13/0x80
|[<a00acf29>] i915_gem_execbuffer2+0x99/0x280 [i915]
|[<a00114a3>] drm_ioctl+0x4c3/0x570 [drm]
|[<8101d0d9>] ? sched_clock+0x9/0x10
|[<a00ace90>] ? i915_gem_execbuffer+0x480/0x480 [i915]
|[<810f1c18>] ? rb_commit+0x68/0xa0
|[<810f1c6c>] ? ring_buffer_unlock_commit+0x1c/0xa0
|[<81197467>] do_vfs_ioctl+0x97/0x540
|[<81021318>] ? ftrace_raw_event_sys_enter+0xd8/0x130
|[<811979a1>] sys_ioctl+0x91/0xb0
|[<814db931>] tracesys+0xe1/0xe6
Chris Wilson does not like to move i915_trace_irq_get() out of the macro
|No. This enables the IRQ, as well as making a number of
|very expensively serialised read, unconditionally.
so it is gone now on RT.
Cc: stable-rt@vger.kernel.org
Reported-by: Joakim Hernberg <jbh@alchemy.lu>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index c26a8f8a..14df3f6 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1259,7 +1259,9 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
goto err;
}
+#ifndef CONFIG_PREEMPT_RT_BASE
trace_i915_gem_ring_dispatch(ring, seqno);
+#endif
exec_start = batch_obj->gtt_offset + args->batch_start_offset;
exec_len = args->batch_len;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 11/19] sched: Distangle worker accounting from rqlock
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (9 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 10/19] drm/i915: drop trace_i915_gem_ring_dispatch on rt Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 12/19] rt,ntp: Move call to schedule_delayed_work() to helper thread Steven Rostedt
` (7 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, Peter Zijlstra, Tejun Heo, Jens Axboe, stable-rt
[-- Attachment #1: 0011-sched-Distangle-worker-accounting-from-rqlock.patch --]
[-- Type: text/plain, Size: 7903 bytes --]
From: Thomas Gleixner <tglx@linutronix.de>
The worker accounting for cpu bound workers is plugged into the core
scheduler code and the wakeup code. This is not a hard requirement and
can be avoided by keeping track of the state in the workqueue code
itself.
Keep track of the sleeping state in the worker itself and call the
notifier before entering the core scheduler. There might be false
positives when the task is woken between that call and actually
scheduling, but that's not really different from scheduling and being
woken immediately after switching away. There is also no harm from
updating nr_running when the task returns from scheduling instead of
accounting it in the wakeup code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: stable-rt@vger.kernel.org
Link: http://lkml.kernel.org/r/20110622174919.135236139@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/sched/core.c | 70 ++++++++++------------------------------------
kernel/workqueue.c | 59 +++++++++++++++++++-------------------
kernel/workqueue_sched.h | 5 ++--
3 files changed, 45 insertions(+), 89 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 781e6c4..1493f43 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1452,10 +1452,6 @@ static void ttwu_activate(struct rq *rq, struct task_struct *p, int en_flags)
{
activate_task(rq, p, en_flags);
p->on_rq = 1;
-
- /* if a worker is waking up, notify workqueue */
- if (p->flags & PF_WQ_WORKER)
- wq_worker_waking_up(p, cpu_of(rq));
}
/*
@@ -1714,42 +1710,6 @@ out:
}
/**
- * try_to_wake_up_local - try to wake up a local task with rq lock held
- * @p: the thread to be awakened
- *
- * Put @p on the run-queue if it's not already there. The caller must
- * ensure that this_rq() is locked, @p is bound to this_rq() and not
- * the current task.
- */
-static void try_to_wake_up_local(struct task_struct *p)
-{
- struct rq *rq = task_rq(p);
-
- if (WARN_ON_ONCE(rq != this_rq()) ||
- WARN_ON_ONCE(p == current))
- return;
-
- lockdep_assert_held(&rq->lock);
-
- if (!raw_spin_trylock(&p->pi_lock)) {
- raw_spin_unlock(&rq->lock);
- raw_spin_lock(&p->pi_lock);
- raw_spin_lock(&rq->lock);
- }
-
- if (!(p->state & TASK_NORMAL))
- goto out;
-
- if (!p->on_rq)
- ttwu_activate(rq, p, ENQUEUE_WAKEUP);
-
- ttwu_do_wakeup(rq, p, 0);
- ttwu_stat(p, smp_processor_id(), 0);
-out:
- raw_spin_unlock(&p->pi_lock);
-}
-
-/**
* wake_up_process - Wake up a specific process
* @p: The process to be woken up.
*
@@ -3627,21 +3587,6 @@ need_resched:
} else {
deactivate_task(rq, prev, DEQUEUE_SLEEP);
prev->on_rq = 0;
-
- /*
- * If a worker went to sleep, notify and ask workqueue
- * whether it wants to wake up a task to maintain
- * concurrency.
- * Only call wake up if prev isn't blocked on a sleeping
- * spin lock.
- */
- if (prev->flags & PF_WQ_WORKER && !prev->saved_state) {
- struct task_struct *to_wakeup;
-
- to_wakeup = wq_worker_sleeping(prev, cpu);
- if (to_wakeup)
- try_to_wake_up_local(to_wakeup);
- }
}
switch_count = &prev->nvcsw;
}
@@ -3685,6 +3630,14 @@ static inline void sched_submit_work(struct task_struct *tsk)
{
if (!tsk->state || tsk_is_pi_blocked(tsk))
return;
+
+ /*
+ * If a worker went to sleep, notify and ask workqueue whether
+ * it wants to wake up a task to maintain concurrency.
+ */
+ if (tsk->flags & PF_WQ_WORKER)
+ wq_worker_sleeping(tsk);
+
/*
* If we are going to sleep and we have plugged IO queued,
* make sure to submit it to avoid deadlocks.
@@ -3693,12 +3646,19 @@ static inline void sched_submit_work(struct task_struct *tsk)
blk_schedule_flush_plug(tsk);
}
+static inline void sched_update_worker(struct task_struct *tsk)
+{
+ if (tsk->flags & PF_WQ_WORKER)
+ wq_worker_running(tsk);
+}
+
asmlinkage void __sched schedule(void)
{
struct task_struct *tsk = current;
sched_submit_work(tsk);
__schedule();
+ sched_update_worker(tsk);
}
EXPORT_SYMBOL(schedule);
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index b973d66..6308d9b 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -148,6 +148,7 @@ struct worker {
unsigned long last_active; /* L: last active timestamp */
unsigned int flags; /* X: flags */
int id; /* I: worker id */
+ int sleeping; /* None */
/* for rebinding worker to CPU */
struct idle_rebind *idle_rebind; /* L: for idle worker */
@@ -691,52 +692,45 @@ static void wake_up_worker(struct worker_pool *pool)
}
/**
- * wq_worker_waking_up - a worker is waking up
- * @task: task waking up
- * @cpu: CPU @task is waking up to
+ * wq_worker_running - a worker is running again
+ * @task: task returning from sleep
*
- * This function is called during try_to_wake_up() when a worker is
- * being awoken.
- *
- * CONTEXT:
- * spin_lock_irq(rq->lock)
+ * This function is called when a worker returns from schedule()
*/
-void wq_worker_waking_up(struct task_struct *task, unsigned int cpu)
+void wq_worker_running(struct task_struct *task)
{
struct worker *worker = kthread_data(task);
+ if (!worker->sleeping)
+ return;
if (!(worker->flags & WORKER_NOT_RUNNING))
atomic_inc(get_pool_nr_running(worker->pool));
+ worker->sleeping = 0;
}
/**
* wq_worker_sleeping - a worker is going to sleep
* @task: task going to sleep
- * @cpu: CPU in question, must be the current CPU number
- *
- * This function is called during schedule() when a busy worker is
- * going to sleep. Worker on the same cpu can be woken up by
- * returning pointer to its task.
- *
- * CONTEXT:
- * spin_lock_irq(rq->lock)
- *
- * RETURNS:
- * Worker task on @cpu to wake up, %NULL if none.
+ * This function is called from schedule() when a busy worker is
+ * going to sleep.
*/
-struct task_struct *wq_worker_sleeping(struct task_struct *task,
- unsigned int cpu)
+void wq_worker_sleeping(struct task_struct *task)
{
- struct worker *worker = kthread_data(task), *to_wakeup = NULL;
- struct worker_pool *pool = worker->pool;
- atomic_t *nr_running = get_pool_nr_running(pool);
+ struct worker *next, *worker = kthread_data(task);
+ struct worker_pool *pool;
+ atomic_t *nr_running;
if (worker->flags & WORKER_NOT_RUNNING)
- return NULL;
+ return;
+
+ if (WARN_ON_ONCE(worker->sleeping))
+ return;
- /* this can only happen on the local cpu */
- BUG_ON(cpu != raw_smp_processor_id());
+ pool = worker->pool;
+ nr_running = get_pool_nr_running(pool);
+ worker->sleeping = 1;
+ spin_lock_irq(&pool->gcwq->lock);
/*
* The counterpart of the following dec_and_test, implied mb,
* worklist not empty test sequence is in insert_work().
@@ -748,9 +742,12 @@ struct task_struct *wq_worker_sleeping(struct task_struct *task,
* manipulating idle_list, so dereferencing idle_list without gcwq
* lock is safe.
*/
- if (atomic_dec_and_test(nr_running) && !list_empty(&pool->worklist))
- to_wakeup = first_worker(pool);
- return to_wakeup ? to_wakeup->task : NULL;
+ if (atomic_dec_and_test(nr_running) && !list_empty(&pool->worklist)) {
+ next = first_worker(pool);
+ if (next)
+ wake_up_process(next->task);
+ }
+ spin_unlock_irq(&pool->gcwq->lock);
}
/**
diff --git a/kernel/workqueue_sched.h b/kernel/workqueue_sched.h
index 2d10fc9..3bf73e2 100644
--- a/kernel/workqueue_sched.h
+++ b/kernel/workqueue_sched.h
@@ -4,6 +4,5 @@
* Scheduler hooks for concurrency managed workqueue. Only to be
* included from sched.c and workqueue.c.
*/
-void wq_worker_waking_up(struct task_struct *task, unsigned int cpu);
-struct task_struct *wq_worker_sleeping(struct task_struct *task,
- unsigned int cpu);
+void wq_worker_running(struct task_struct *task);
+void wq_worker_sleeping(struct task_struct *task);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 12/19] rt,ntp: Move call to schedule_delayed_work() to helper thread
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (10 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 11/19] sched: Distangle worker accounting from rqlock Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 13/19] hpsa: fix warning with smp_processor_id() in preemptible section Steven Rostedt
` (6 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0012-rt-ntp-Move-call-to-schedule_delayed_work-to-helper-.patch --]
[-- Type: text/plain, Size: 2578 bytes --]
From: Steven Rostedt <rostedt@goodmis.org>
The ntp code for notify_cmos_timer() is called from a hard interrupt
context. schedule_delayed_work() under PREEMPT_RT_FULL calls spinlocks
that have been converted to mutexes, thus calling schedule_delayed_work()
from interrupt is not safe.
Add a helper thread that does the call to schedule_delayed_work and wake
up that thread instead of calling schedule_delayed_work() directly.
This is only for CONFIG_PREEMPT_RT_FULL, otherwise the code still calls
schedule_delayed_work() directly in irq context.
Note: There's a few places in the kernel that do this. Perhaps the RT
code should have a dedicated thread that does the checks. Just register
a notifier on boot up for your check and wake up the thread when
needed. This will be a todo.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
kernel/time/ntp.c | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/kernel/time/ntp.c b/kernel/time/ntp.c
index bb1edfa..3e93516 100644
--- a/kernel/time/ntp.c
+++ b/kernel/time/ntp.c
@@ -10,6 +10,7 @@
#include <linux/workqueue.h>
#include <linux/hrtimer.h>
#include <linux/jiffies.h>
+#include <linux/kthread.h>
#include <linux/math64.h>
#include <linux/timex.h>
#include <linux/time.h>
@@ -529,10 +530,49 @@ static void sync_cmos_clock(struct work_struct *work)
schedule_delayed_work(&sync_cmos_work, timespec_to_jiffies(&next));
}
+#ifdef CONFIG_PREEMPT_RT_FULL
+/*
+ * RT can not call schedule_delayed_work from real interrupt context.
+ * Need to make a thread to do the real work.
+ */
+static struct task_struct *cmos_delay_thread;
+static bool do_cmos_delay;
+
+static int run_cmos_delay(void *ignore)
+{
+ while (!kthread_should_stop()) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ if (do_cmos_delay) {
+ do_cmos_delay = false;
+ schedule_delayed_work(&sync_cmos_work, 0);
+ }
+ schedule();
+ }
+ __set_current_state(TASK_RUNNING);
+ return 0;
+}
+
+static void notify_cmos_timer(void)
+{
+ do_cmos_delay = true;
+ /* Make visible before waking up process */
+ smp_wmb();
+ wake_up_process(cmos_delay_thread);
+}
+
+static __init int create_cmos_delay_thread(void)
+{
+ cmos_delay_thread = kthread_run(run_cmos_delay, NULL, "kcmosdelayd");
+ BUG_ON(!cmos_delay_thread);
+ return 0;
+}
+early_initcall(create_cmos_delay_thread);
+#else
static void notify_cmos_timer(void)
{
schedule_delayed_work(&sync_cmos_work, 0);
}
+#endif /* CONFIG_PREEMPT_RT_FULL */
#else
static inline void notify_cmos_timer(void) { }
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 13/19] hpsa: fix warning with smp_processor_id() in preemptible section
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (11 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 12/19] rt,ntp: Move call to schedule_delayed_work() to helper thread Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 14/19] hwlat-detector: Update hwlat_detector to add outer loop detection Steven Rostedt
` (5 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Stephen
[-- Attachment #1: 0013-hpsa-fix-warning-with-smp_processor_id-in-preemptibl.patch --]
[-- Type: text/plain, Size: 3376 bytes --]
From: John Kacur <jkacur@redhat.com>
On a 3.6-rt (real-time patch) kernel we are seeing the following BUG
However, it appears to be relevant for non-realtime (mainline) as well.
|hpsa 0000:03:00.0: hpsa0: <0x323a> at IRQ 67 using DAC
|scsi0 : hpsa
|BUG: using smp_processor_id() in preemptible [0000000000000000] code: kworker/u:0/6
|caller is enqueue_cmd_and_start_io+0x5a/0x100 [hpsa]
|Pid: 6, comm: kworker/u:0 Not tainted 3.6.11.5-rt37.52.el6rt.x86_64.debug #1
|Call Trace:
| [<ffffffff812abe83>] debug_smp_processor_id+0x123/0x150
| [<ffffffffa009043a>] enqueue_cmd_and_start_io+0x5a/0x100 [hpsa]
| [<ffffffffa00905cb>] hpsa_scsi_do_simple_cmd_core+0xeb/0x110 [hpsa]
| [<ffffffff812b09c8>] ? swiotlb_dma_mapping_error+0x18/0x30
| [<ffffffff812b09c8>] ? swiotlb_dma_mapping_error+0x18/0x30
| [<ffffffffa0090701>] hpsa_scsi_do_simple_cmd_with_retry+0x91/0x280 [hpsa]
| [<ffffffffa0093558>] hpsa_scsi_do_report_luns.clone.2+0xd8/0x130 [hpsa]
| [<ffffffffa00935ea>] hpsa_gather_lun_info.clone.3+0x3a/0x1a0 [hpsa]
| [<ffffffffa00963df>] hpsa_update_scsi_devices+0x11f/0x4f0 [hpsa]
| [<ffffffff81592019>] ? sub_preempt_count+0xa9/0xe0
| [<ffffffffa00968ad>] hpsa_scan_start+0xfd/0x150 [hpsa]
| [<ffffffff8158cba8>] ? rt_spin_lock_slowunlock+0x78/0x90
| [<ffffffff813b04d7>] do_scsi_scan_host+0x37/0xa0
| [<ffffffff813b05da>] do_scan_async+0x1a/0x30
| [<ffffffff8107c4ab>] async_run_entry_fn+0x9b/0x1d0
| [<ffffffff8106ae92>] process_one_work+0x1f2/0x620
| [<ffffffff8106ae20>] ? process_one_work+0x180/0x620
| [<ffffffff8106d4fe>] ? worker_thread+0x5e/0x3a0
| [<ffffffff8107c410>] ? async_schedule+0x20/0x20
| [<ffffffff8106d5d3>] worker_thread+0x133/0x3a0
| [<ffffffff8106d4a0>] ? manage_workers+0x190/0x190
| [<ffffffff81073236>] kthread+0xa6/0xb0
| [<ffffffff815970a4>] kernel_thread_helper+0x4/0x10
| [<ffffffff81082a7c>] ? finish_task_switch+0x8c/0x110
| [<ffffffff8158e44b>] ? _raw_spin_unlock_irq+0x3b/0x70
| [<ffffffff8158e85d>] ? retint_restore_args+0xe/0xe
| [<ffffffff81073190>] ? kthreadd+0x1e0/0x1e0
| [<ffffffff815970a0>] ? gs_change+0xb/0xb
-------------------
This is caused by
enqueue_cmd_and_start_io()->
set_performant_mode()->
smp_processor_id()
Which if you have debugging enabled calls debug_processor_id() and triggers the warning.
The code here is
c->Header.ReplyQueue = smp_processor_id() % h->nreply_queues;
Since it is not critical that the code complete on the same processor,
but the cpu is a hint used in generating the ReplyQueue and will still work if
the cpu migrates or is preempted, it is safe to use the raw_smp_processor_id()
to surpress the false positve warning.
Cc: stable-rt@vger.kernel.org
Signed-off-by: John Kacur <jkacur@redhat.com>
Acked-by: Stephen <scameron@beardog.cce.hp.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/scsi/hpsa.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
index f9823f2..be83a15 100644
--- a/drivers/scsi/hpsa.c
+++ b/drivers/scsi/hpsa.c
@@ -565,7 +565,7 @@ static void set_performant_mode(struct ctlr_info *h, struct CommandList *c)
c->busaddr |= 1 | (h->blockFetchTable[c->Header.SGList] << 1);
if (likely(h->msix_vector))
c->Header.ReplyQueue =
- smp_processor_id() % h->nreply_queues;
+ raw_smp_processor_id() % h->nreply_queues;
}
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 14/19] hwlat-detector: Update hwlat_detector to add outer loop detection
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (12 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 13/19] hpsa: fix warning with smp_processor_id() in preemptible section Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 15/19] hwlat-detect/trace: Export trace_clock_local for hwlat-detector Steven Rostedt
` (4 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Steven Rostedt
[-- Attachment #1: 0014-hwlat-detector-Update-hwlat_detector-to-add-outer-lo.patch --]
[-- Type: text/plain, Size: 3993 bytes --]
From: Steven Rostedt <rostedt@goodmis.org>
The hwlat_detector reads two timestamps in a row, then reports any
gap between those calls. The problem is, it misses everything between
the second reading of the time stamp to the first reading of the time stamp
in the next loop. That's were most of the time is spent, which means,
chances are likely that it will miss all hardware latencies. This
defeats the purpose.
By also testing the first time stamp from the previous loop second
time stamp (the outer loop), we are more likely to find a latency.
Setting the threshold to 1, here's what the report now looks like:
1347415723.0232202770 0 2
1347415725.0234202822 0 2
1347415727.0236202875 0 2
1347415729.0238202928 0 2
1347415731.0240202980 0 2
1347415734.0243203061 0 2
1347415736.0245203113 0 2
1347415738.0247203166 2 0
1347415740.0249203219 0 3
1347415742.0251203272 0 3
1347415743.0252203299 0 3
1347415745.0254203351 0 2
1347415747.0256203404 0 2
1347415749.0258203457 0 2
1347415751.0260203510 0 2
1347415754.0263203589 0 2
1347415756.0265203642 0 2
1347415758.0267203695 0 2
1347415760.0269203748 0 2
1347415762.0271203801 0 2
1347415764.0273203853 2 0
There's some hardware latency that takes 2 microseconds to run.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/misc/hwlat_detector.c | 32 ++++++++++++++++++++++++++------
1 file changed, 26 insertions(+), 6 deletions(-)
diff --git a/drivers/misc/hwlat_detector.c b/drivers/misc/hwlat_detector.c
index b7b7c90..f93b8ef 100644
--- a/drivers/misc/hwlat_detector.c
+++ b/drivers/misc/hwlat_detector.c
@@ -143,6 +143,7 @@ static void detector_exit(void);
struct sample {
u64 seqnum; /* unique sequence */
u64 duration; /* ktime delta */
+ u64 outer_duration; /* ktime delta (outer loop) */
struct timespec timestamp; /* wall time */
unsigned long lost;
};
@@ -219,11 +220,13 @@ static struct sample *buffer_get_sample(struct sample *sample)
*/
static int get_sample(void *unused)
{
- ktime_t start, t1, t2;
+ ktime_t start, t1, t2, last_t2;
s64 diff, total = 0;
u64 sample = 0;
+ u64 outer_sample = 0;
int ret = 1;
+ last_t2.tv64 = 0;
start = ktime_get(); /* start timestamp */
do {
@@ -231,7 +234,22 @@ static int get_sample(void *unused)
t1 = ktime_get(); /* we'll look for a discontinuity */
t2 = ktime_get();
+ if (last_t2.tv64) {
+ /* Check the delta from the outer loop (t2 to next t1) */
+ diff = ktime_to_us(ktime_sub(t1, last_t2));
+ /* This shouldn't happen */
+ if (diff < 0) {
+ printk(KERN_ERR BANNER "time running backwards\n");
+ goto out;
+ }
+ if (diff > outer_sample)
+ outer_sample = diff;
+ }
+ last_t2 = t2;
+
total = ktime_to_us(ktime_sub(t2, start)); /* sample width */
+
+ /* This checks the inner loop (t1 to t2) */
diff = ktime_to_us(ktime_sub(t2, t1)); /* current diff */
/* This shouldn't happen */
@@ -246,12 +264,13 @@ static int get_sample(void *unused)
} while (total <= data.sample_width);
/* If we exceed the threshold value, we have found a hardware latency */
- if (sample > data.threshold) {
+ if (sample > data.threshold || outer_sample > data.threshold) {
struct sample s;
data.count++;
s.seqnum = data.count;
s.duration = sample;
+ s.outer_duration = outer_sample;
s.timestamp = CURRENT_TIME;
__buffer_add_sample(&s);
@@ -738,10 +757,11 @@ static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf,
}
}
- len = snprintf(buf, sizeof(buf), "%010lu.%010lu\t%llu\n",
- sample->timestamp.tv_sec,
- sample->timestamp.tv_nsec,
- sample->duration);
+ len = snprintf(buf, sizeof(buf), "%010lu.%010lu\t%llu\t%llu\n",
+ sample->timestamp.tv_sec,
+ sample->timestamp.tv_nsec,
+ sample->duration,
+ sample->outer_duration);
/* handling partial reads is more trouble than it's worth */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 15/19] hwlat-detect/trace: Export trace_clock_local for hwlat-detector
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (13 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 14/19] hwlat-detector: Update hwlat_detector to add outer loop detection Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 16/19] hwlat-detector: Use trace_clock_local if available Steven Rostedt
` (3 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur
[-- Attachment #1: 0015-hwlat-detect-trace-Export-trace_clock_local-for-hwla.patch --]
[-- Type: text/plain, Size: 735 bytes --]
From: "Steven Rostedt (Red Hat)" <rostedt@goodmis.org>
The hwlat-detector needs a better clock than just ktime_get() as that
can induce its own latencies. The trace clock is perfect for it, but
it needs to be exported for use by modules.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
---
kernel/trace/trace_clock.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/kernel/trace/trace_clock.c b/kernel/trace/trace_clock.c
index 3947835..1bbb1b2 100644
--- a/kernel/trace/trace_clock.c
+++ b/kernel/trace/trace_clock.c
@@ -44,6 +44,7 @@ u64 notrace trace_clock_local(void)
return clock;
}
+EXPORT_SYMBOL_GPL(trace_clock_local);
/*
* trace_clock(): 'between' trace clock. Not completely serialized,
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 16/19] hwlat-detector: Use trace_clock_local if available
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (14 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 15/19] hwlat-detect/trace: Export trace_clock_local for hwlat-detector Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 17/19] hwlat-detector: Use thread instead of stop machine Steven Rostedt
` (2 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt, Steven Rostedt
[-- Attachment #1: 0016-hwlat-detector-Use-trace_clock_local-if-available.patch --]
[-- Type: text/plain, Size: 3093 bytes --]
From: Steven Rostedt <rostedt@goodmis.org>
As ktime_get() calls into the timing code which does a read_seq(), it
may be affected by other CPUS that touch that lock. To remove this
dependency, use the trace_clock_local() which is already exported
for module use. If CONFIG_TRACING is enabled, use that as the clock,
otherwise use ktime_get().
Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/misc/hwlat_detector.c | 34 +++++++++++++++++++++++++---------
1 file changed, 25 insertions(+), 9 deletions(-)
diff --git a/drivers/misc/hwlat_detector.c b/drivers/misc/hwlat_detector.c
index f93b8ef..13443e9 100644
--- a/drivers/misc/hwlat_detector.c
+++ b/drivers/misc/hwlat_detector.c
@@ -51,6 +51,7 @@
#include <linux/version.h>
#include <linux/delay.h>
#include <linux/slab.h>
+#include <linux/trace_clock.h>
#define BUF_SIZE_DEFAULT 262144UL /* 8K*(sizeof(entry)) */
#define BUF_FLAGS (RB_FL_OVERWRITE) /* no block on full */
@@ -211,6 +212,21 @@ static struct sample *buffer_get_sample(struct sample *sample)
return sample;
}
+#ifndef CONFIG_TRACING
+#define time_type ktime_t
+#define time_get() ktime_get()
+#define time_to_us(x) ktime_to_us(x)
+#define time_sub(a, b) ktime_sub(a, b)
+#define init_time(a, b) (a).tv64 = b
+#define time_u64(a) (a).tv64
+#else
+#define time_type u64
+#define time_get() trace_clock_local()
+#define time_to_us(x) div_u64(x, 1000)
+#define time_sub(a, b) ((a) - (b))
+#define init_time(a, b) a = b
+#define time_u64(a) a
+#endif
/**
* get_sample - sample the CPU TSC and look for likely hardware latencies
* @unused: This is not used but is a part of the stop_machine API
@@ -220,23 +236,23 @@ static struct sample *buffer_get_sample(struct sample *sample)
*/
static int get_sample(void *unused)
{
- ktime_t start, t1, t2, last_t2;
+ time_type start, t1, t2, last_t2;
s64 diff, total = 0;
u64 sample = 0;
u64 outer_sample = 0;
int ret = 1;
- last_t2.tv64 = 0;
- start = ktime_get(); /* start timestamp */
+ init_time(last_t2, 0);
+ start = time_get(); /* start timestamp */
do {
- t1 = ktime_get(); /* we'll look for a discontinuity */
- t2 = ktime_get();
+ t1 = time_get(); /* we'll look for a discontinuity */
+ t2 = time_get();
- if (last_t2.tv64) {
+ if (time_u64(last_t2)) {
/* Check the delta from the outer loop (t2 to next t1) */
- diff = ktime_to_us(ktime_sub(t1, last_t2));
+ diff = time_to_us(time_sub(t1, last_t2));
/* This shouldn't happen */
if (diff < 0) {
printk(KERN_ERR BANNER "time running backwards\n");
@@ -247,10 +263,10 @@ static int get_sample(void *unused)
}
last_t2 = t2;
- total = ktime_to_us(ktime_sub(t2, start)); /* sample width */
+ total = time_to_us(time_sub(t2, start)); /* sample width */
/* This checks the inner loop (t1 to t2) */
- diff = ktime_to_us(ktime_sub(t2, t1)); /* current diff */
+ diff = time_to_us(time_sub(t2, t1)); /* current diff */
/* This shouldn't happen */
if (diff < 0) {
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 17/19] hwlat-detector: Use thread instead of stop machine
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (15 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 16/19] hwlat-detector: Use trace_clock_local if available Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 18/19] genirq: do not invoke the affinity callback via a workqueue Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 19/19] Linux 3.6.11.8-rt41-rc1 Steven Rostedt
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0017-hwlat-detector-Use-thread-instead-of-stop-machine.patch --]
[-- Type: text/plain, Size: 6412 bytes --]
From: Steven Rostedt <rostedt@goodmis.org>
There's no reason to use stop machine to search for hardware latency.
Simply disabling interrupts while running the loop will do enough to
check if something comes in that wasn't disabled by interrupts being
off, which is exactly what stop machine does.
Instead of using stop machine, just have the thread disable interrupts
while it checks for hardware latency.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
drivers/misc/hwlat_detector.c | 59 +++++++++++++++++------------------------
1 file changed, 25 insertions(+), 34 deletions(-)
diff --git a/drivers/misc/hwlat_detector.c b/drivers/misc/hwlat_detector.c
index 13443e9..6f61d5f 100644
--- a/drivers/misc/hwlat_detector.c
+++ b/drivers/misc/hwlat_detector.c
@@ -41,7 +41,6 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/ring_buffer.h>
-#include <linux/stop_machine.h>
#include <linux/time.h>
#include <linux/hrtimer.h>
#include <linux/kthread.h>
@@ -107,7 +106,6 @@ struct data; /* Global state */
/* Sampling functions */
static int __buffer_add_sample(struct sample *sample);
static struct sample *buffer_get_sample(struct sample *sample);
-static int get_sample(void *unused);
/* Threading and state */
static int kthread_fn(void *unused);
@@ -149,7 +147,7 @@ struct sample {
unsigned long lost;
};
-/* keep the global state somewhere. Mostly used under stop_machine. */
+/* keep the global state somewhere. */
static struct data {
struct mutex lock; /* protect changes */
@@ -172,7 +170,7 @@ static struct data {
* @sample: The new latency sample value
*
* This receives a new latency sample and records it in a global ring buffer.
- * No additional locking is used in this case - suited for stop_machine use.
+ * No additional locking is used in this case.
*/
static int __buffer_add_sample(struct sample *sample)
{
@@ -229,18 +227,17 @@ static struct sample *buffer_get_sample(struct sample *sample)
#endif
/**
* get_sample - sample the CPU TSC and look for likely hardware latencies
- * @unused: This is not used but is a part of the stop_machine API
*
* Used to repeatedly capture the CPU TSC (or similar), looking for potential
- * hardware-induced latency. Called under stop_machine, with data.lock held.
+ * hardware-induced latency. Called with interrupts disabled and with data.lock held.
*/
-static int get_sample(void *unused)
+static int get_sample(void)
{
time_type start, t1, t2, last_t2;
s64 diff, total = 0;
u64 sample = 0;
u64 outer_sample = 0;
- int ret = 1;
+ int ret = -1;
init_time(last_t2, 0);
start = time_get(); /* start timestamp */
@@ -279,10 +276,14 @@ static int get_sample(void *unused)
} while (total <= data.sample_width);
+ ret = 0;
+
/* If we exceed the threshold value, we have found a hardware latency */
if (sample > data.threshold || outer_sample > data.threshold) {
struct sample s;
+ ret = 1;
+
data.count++;
s.seqnum = data.count;
s.duration = sample;
@@ -295,7 +296,6 @@ static int get_sample(void *unused)
data.max_sample = sample;
}
- ret = 0;
out:
return ret;
}
@@ -305,32 +305,30 @@ out:
* @unused: A required part of the kthread API.
*
* Used to periodically sample the CPU TSC via a call to get_sample. We
- * use stop_machine, whith does (intentionally) introduce latency since we
+ * disable interrupts, which does (intentionally) introduce latency since we
* need to ensure nothing else might be running (and thus pre-empting).
* Obviously this should never be used in production environments.
*
- * stop_machine will schedule us typically only on CPU0 which is fine for
- * almost every real-world hardware latency situation - but we might later
- * generalize this if we find there are any actualy systems with alternate
- * SMI delivery or other non CPU0 hardware latencies.
+ * Currently this runs on which ever CPU it was scheduled on, but most
+ * real-worald hardware latency situations occur across several CPUs,
+ * but we might later generalize this if we find there are any actualy
+ * systems with alternate SMI delivery or other hardware latencies.
*/
static int kthread_fn(void *unused)
{
- int err = 0;
- u64 interval = 0;
+ int ret;
+ u64 interval;
while (!kthread_should_stop()) {
mutex_lock(&data.lock);
- err = stop_machine(get_sample, unused, 0);
- if (err) {
- /* Houston, we have a problem */
- mutex_unlock(&data.lock);
- goto err_out;
- }
+ local_irq_disable();
+ ret = get_sample();
+ local_irq_enable();
- wake_up(&data.wq); /* wake up reader(s) */
+ if (ret > 0)
+ wake_up(&data.wq); /* wake up reader(s) */
interval = data.sample_window - data.sample_width;
do_div(interval, USEC_PER_MSEC); /* modifies interval value */
@@ -338,15 +336,10 @@ static int kthread_fn(void *unused)
mutex_unlock(&data.lock);
if (msleep_interruptible(interval))
- goto out;
+ break;
}
- goto out;
-err_out:
- printk(KERN_ERR BANNER "could not call stop_machine, disabling\n");
- enabled = 0;
-out:
- return err;
+ return 0;
}
/**
@@ -442,8 +435,7 @@ out:
* This function provides a generic read implementation for the global state
* "data" structure debugfs filesystem entries. It would be nice to use
* simple_attr_read directly, but we need to make sure that the data.lock
- * spinlock is held during the actual read (even though we likely won't ever
- * actually race here as the updater runs under a stop_machine context).
+ * is held during the actual read.
*/
static ssize_t simple_data_read(struct file *filp, char __user *ubuf,
size_t cnt, loff_t *ppos, const u64 *entry)
@@ -478,8 +470,7 @@ static ssize_t simple_data_read(struct file *filp, char __user *ubuf,
* This function provides a generic write implementation for the global state
* "data" structure debugfs filesystem entries. It would be nice to use
* simple_attr_write directly, but we need to make sure that the data.lock
- * spinlock is held during the actual write (even though we likely won't ever
- * actually race here as the updater runs under a stop_machine context).
+ * is held during the actual write.
*/
static ssize_t simple_data_write(struct file *filp, const char __user *ubuf,
size_t cnt, loff_t *ppos, u64 *entry)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 18/19] genirq: do not invoke the affinity callback via a workqueue
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (16 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 17/19] hwlat-detector: Use thread instead of stop machine Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
2013-09-08 15:14 ` [PATCH RT 19/19] Linux 3.6.11.8-rt41-rc1 Steven Rostedt
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur, stable-rt
[-- Attachment #1: 0018-genirq-do-not-invoke-the-affinity-callback-via-a-wor.patch --]
[-- Type: text/plain, Size: 4444 bytes --]
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Joe Korty reported, that __irq_set_affinity_locked() schedules a
workqueue while holding a rawlock which results in a might_sleep()
warning.
This patch moves the invokation into a process context so that we only
wakeup() a process while holding the lock.
Cc: stable-rt@vger.kernel.org
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/interrupt.h | 1 +
kernel/irq/manage.c | 79 +++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 77 insertions(+), 3 deletions(-)
diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
index fb81baa..16fbe2b 100644
--- a/include/linux/interrupt.h
+++ b/include/linux/interrupt.h
@@ -263,6 +263,7 @@ struct irq_affinity_notify {
unsigned int irq;
struct kref kref;
struct work_struct work;
+ struct list_head list;
void (*notify)(struct irq_affinity_notify *, const cpumask_t *mask);
void (*release)(struct kref *ref);
};
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 2ecc232..0fe99a4 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -163,6 +163,62 @@ int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask,
return ret;
}
+#ifdef CONFIG_PREEMPT_RT_FULL
+static void _irq_affinity_notify(struct irq_affinity_notify *notify);
+static struct task_struct *set_affinity_helper;
+static LIST_HEAD(affinity_list);
+static DEFINE_RAW_SPINLOCK(affinity_list_lock);
+
+static int set_affinity_thread(void *unused)
+{
+ while (1) {
+ struct irq_affinity_notify *notify;
+ int empty;
+
+ set_current_state(TASK_INTERRUPTIBLE);
+
+ raw_spin_lock_irq(&affinity_list_lock);
+ empty = list_empty(&affinity_list);
+ raw_spin_unlock_irq(&affinity_list_lock);
+
+ if (empty)
+ schedule();
+ if (kthread_should_stop())
+ break;
+ set_current_state(TASK_RUNNING);
+try_next:
+ notify = NULL;
+
+ raw_spin_lock_irq(&affinity_list_lock);
+ if (!list_empty(&affinity_list)) {
+ notify = list_first_entry(&affinity_list,
+ struct irq_affinity_notify, list);
+ list_del_init(¬ify->list);
+ }
+ raw_spin_unlock_irq(&affinity_list_lock);
+
+ if (!notify)
+ continue;
+ _irq_affinity_notify(notify);
+ goto try_next;
+ }
+ return 0;
+}
+
+static void init_helper_thread(void)
+{
+ if (set_affinity_helper)
+ return;
+ set_affinity_helper = kthread_run(set_affinity_thread, NULL,
+ "affinity-cb");
+ WARN_ON(IS_ERR(set_affinity_helper));
+}
+#else
+
+static inline void init_helper_thread(void) { }
+
+#endif
+
int __irq_set_affinity_locked(struct irq_data *data, const struct cpumask *mask)
{
struct irq_chip *chip = irq_data_get_irq_chip(data);
@@ -181,7 +237,17 @@ int __irq_set_affinity_locked(struct irq_data *data, const struct cpumask *mask)
if (desc->affinity_notify) {
kref_get(&desc->affinity_notify->kref);
+
+#ifdef CONFIG_PREEMPT_RT_FULL
+ raw_spin_lock(&affinity_list_lock);
+ if (list_empty(&desc->affinity_notify->list))
+ list_add_tail(&affinity_list,
+ &desc->affinity_notify->list);
+ raw_spin_unlock(&affinity_list_lock);
+ wake_up_process(set_affinity_helper);
+#else
schedule_work(&desc->affinity_notify->work);
+#endif
}
irqd_set(data, IRQD_AFFINITY_SET);
@@ -222,10 +288,8 @@ int irq_set_affinity_hint(unsigned int irq, const struct cpumask *m)
}
EXPORT_SYMBOL_GPL(irq_set_affinity_hint);
-static void irq_affinity_notify(struct work_struct *work)
+static void _irq_affinity_notify(struct irq_affinity_notify *notify)
{
- struct irq_affinity_notify *notify =
- container_of(work, struct irq_affinity_notify, work);
struct irq_desc *desc = irq_to_desc(notify->irq);
cpumask_var_t cpumask;
unsigned long flags;
@@ -247,6 +311,13 @@ out:
kref_put(¬ify->kref, notify->release);
}
+static void irq_affinity_notify(struct work_struct *work)
+{
+ struct irq_affinity_notify *notify =
+ container_of(work, struct irq_affinity_notify, work);
+ _irq_affinity_notify(notify);
+}
+
/**
* irq_set_affinity_notifier - control notification of IRQ affinity changes
* @irq: Interrupt for which to enable/disable notification
@@ -276,6 +347,8 @@ irq_set_affinity_notifier(unsigned int irq, struct irq_affinity_notify *notify)
notify->irq = irq;
kref_init(¬ify->kref);
INIT_WORK(¬ify->work, irq_affinity_notify);
+ INIT_LIST_HEAD(¬ify->list);
+ init_helper_thread();
}
raw_spin_lock_irqsave(&desc->lock, flags);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH RT 19/19] Linux 3.6.11.8-rt41-rc1
2013-09-08 15:13 [PATCH RT 00/19] 3.6.11.8-rt41-rc1 stable review Steven Rostedt
` (17 preceding siblings ...)
2013-09-08 15:14 ` [PATCH RT 18/19] genirq: do not invoke the affinity callback via a workqueue Steven Rostedt
@ 2013-09-08 15:14 ` Steven Rostedt
18 siblings, 0 replies; 20+ messages in thread
From: Steven Rostedt @ 2013-09-08 15:14 UTC (permalink / raw)
To: linux-kernel, linux-rt-users
Cc: Thomas Gleixner, Carsten Emde, Sebastian Andrzej Siewior,
John Kacur
[-- Attachment #1: 0019-Linux-3.6.11.8-rt41-rc1.patch --]
[-- Type: text/plain, Size: 299 bytes --]
From: "Steven Rostedt (Red Hat)" <rostedt@goodmis.org>
---
localversion-rt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/localversion-rt b/localversion-rt
index 2af6c89..ad263ce 100644
--- a/localversion-rt
+++ b/localversion-rt
@@ -1 +1 @@
--rt40
+-rt41-rc1
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread