2.6.26.3-rt2

2.6.26.3-rt2

[Steven Rostedt]

We are pleased to announce the 2.6.26.3-rt2 tree, which can be
downloaded from the location:

  http://rt.et.redhat.com/download/

Information on the RT patch can be found at:

  http://rt.wiki.kernel.org/index.php/Main_Page

Changes since 2.6.26-rt1

  - ported to 2.6.26.3

  - basic compile fixes (Steven Rostedt)

  - basic raw lock conversions (Steven Rostedt)

  - powerpc: irq radix tree updates (Sebastien Dugue)

  - call_rcu_sched (Paul E. McKenney)

  - rcu trace preemption fixes (Paul E. Mckenney)

  - remove lock in pm_qos_requirement (John Kacur)

  - rcu_process_callbacks (Paul E. McKenney)

  - rcu memory barriers update (Paul E. McKenney)

  - shorten posix_cpu_timers thread names (Arnaldo Carvalho de Melo)

  - account IRQ and RT load in SCHED_OTHER (Peter Zijlstra)

  - seq_lock updates/fixes (Gregory Haskins)

to build a 2.6.26.3-rt2 tree, the following patches should be applied:

  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.26.tar.bz2
  http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.26.3.bz2
  http://rt.et.redhat.com/download/patch-2.6.26.3-rt2.bz2



And like always, my RT version of Matt Mackall's ketchup will get this
for you nicely:

  http://people.redhat.com/srostedt/rt/tools/ketchup-0.9.8-rt3


The broken out patches are also available.


-- Steve

Re: 2.6.26.3-rt2

[Javier Sanz]

Hello,

I got the following problem compiling linux 2.6.23-rt2:

 <....>
  CC      kernel/ns_cgroup.o
  CC      kernel/utsname.o
  CC      kernel/user_namespace.o
  CC      kernel/pid_namespace.o
  CC      kernel/res_counter.o
  CC      kernel/stop_machine.o
  CC      kernel/audit.o
  CC      kernel/auditfilter.o
  CC      kernel/auditsc.o
  CC      kernel/audit_tree.o
  CC      kernel/kprobes.o
  CC      kernel/softlockup.o
  CC      kernel/seccomp.o
  CC      kernel/rcupreempt.o
  CC      kernel/rcupreempt-boost.o
kernel/rcupreempt-boost.c: In function '__rcu_preempt_unboost':
kernel/rcupreempt-boost.c:328: error: too few arguments to function
'rcu_trace_boost_unboost_called'
make[1]: *** [kernel/rcupreempt-boost.o] Error 1
make: *** [kernel] Error 2

am i doing something wrong ?

thank you

Re: 2.6.26.3-rt2

[Steven Rostedt]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1045 bytes --]



On Thu, 21 Aug 2008, Javier Sanz wrote:

> Hello,
> 
> I got the following compiling problem with 2.6.23-rt2:
> 
>  <....>
>   CC      kernel/ns_cgroup.o
>   CC      kernel/utsname.o
>   CC      kernel/user_namespace.o
>   CC      kernel/pid_namespace.o
>   CC      kernel/res_counter.o
>   CC      kernel/stop_machine.o
>   CC      kernel/audit.o
>   CC      kernel/auditfilter.o
>   CC      kernel/auditsc.o
>   CC      kernel/audit_tree.o
>   CC      kernel/kprobes.o
>   CC      kernel/softlockup.o
>   CC      kernel/seccomp.o
>   CC      kernel/rcupreempt.o
>   CC      kernel/rcupreempt-boost.o
> kernel/rcupreempt-boost.c: In function '__rcu_preempt_unboost':
> kernel/rcupreempt-boost.c:328: error: too few arguments to function
> 'rcu_trace_boost_unboost_called'
> make[1]: *** [kernel/rcupreempt-boost.o] Error 1
> make: *** [kernel] Error 2
> 
> am i doing something wrong ?

Probably not, I'll work on getting a -rt3 out today. There's several 
patches that need to go in.

Thanks,

-- Steve

Re: 2.6.26.3-rt2

[John Kacur]

[-- Attachment #1: Type: text/plain, Size: 3189 bytes --]

On Thu, Aug 21, 2008 at 7:18 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> We are pleased to announce the 2.6.26.3-rt2 tree, which can be
> downloaded from the location:
>
>  http://rt.et.redhat.com/download/
>
> Information on the RT patch can be found at:
>
>  http://rt.wiki.kernel.org/index.php/Main_Page
>
> Changes since 2.6.26-rt1
>
>  - ported to 2.6.26.3
>
>  - basic compile fixes (Steven Rostedt)
>
>  - basic raw lock conversions (Steven Rostedt)
>
>  - powerpc: irq radix tree updates (Sebastien Dugue)
>
>  - call_rcu_sched (Paul E. McKenney)
>
>  - rcu trace preemption fixes (Paul E. Mckenney)
>
>  - remove lock in pm_qos_requirement (John Kacur)
>
>  - rcu_process_callbacks (Paul E. McKenney)
>
>  - rcu memory barriers update (Paul E. McKenney)
>
>  - shorten posix_cpu_timers thread names (Arnaldo Carvalho de Melo)
>
>  - account IRQ and RT load in SCHED_OTHER (Peter Zijlstra)
>
>  - seq_lock updates/fixes (Gregory Haskins)
>
> to build a 2.6.26.3-rt2 tree, the following patches should be applied:
>
>  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.26.tar.bz2
>  http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.26.3.bz2
>  http://rt.et.redhat.com/download/patch-2.6.26.3-rt2.bz2
>
>
>
> And like always, my RT version of Matt Mackall's ketchup will get this
> for you nicely:
>
>  http://people.redhat.com/srostedt/rt/tools/ketchup-0.9.8-rt3
>
>
> The broken out patches are also available.
>

Hi Steve

Probably one of the most important patches that was missed is the one
from Chirag. I've taken it and updated it against 2.6.26.3-rt2 for
you, in case that will speed up -rt3 (see attached). (two hunks were
rejected because another applied fix fixed the same thing as part of
Chirag's patch.

I've compiled and done some limited testing, and among other things it
removes this kind of message.
BUG: sleeping function called from invalid context hald-runner(12244)
at kernel/rtmutex.c:743
in_atomic():1 [00000001], irqs_disabled():0
Pid: 12244, comm: hald-runner Tainted: G        W 2.6.26.3-rt2 #1

Call Trace:
 [<ffffffff802d3cda>] ? mnt_want_write+0x19/0xbc
 [<ffffffff802307b6>] __might_sleep+0x12d/0x132
 [<ffffffff8046d3de>] __rt_spin_lock+0x34/0x7d
 [<ffffffff8046d435>] rt_spin_lock+0xe/0x10
 [<ffffffff802d3cfc>] mnt_want_write+0x3b/0xbc
 [<ffffffff802cea88>] touch_atime+0x1c/0x101
 [<ffffffff802c426c>] __link_path_walk+0x484/0xf24
 [<ffffffff802c4d72>] path_walk+0x66/0xc9
 [<ffffffff802c51bf>] do_path_lookup+0x209/0x264
 [<ffffffff802c5509>] __path_lookup_intent_open+0x61/0xa4
 [<ffffffff802c555d>] path_lookup_open+0x11/0x13
 [<ffffffff802c61f1>] do_filp_open+0xb6/0x8f5
 [<ffffffff8046d3de>] ? __rt_spin_lock+0x34/0x7d
 [<ffffffff802b897d>] ? get_unused_fd_flags+0x119/0x12a
 [<ffffffff802b89e4>] do_sys_open+0x56/0xd6
 [<ffffffff8020c374>] ? system_call_after_swapgs+0xc4/0x168
 [<ffffffff802b8a97>] sys_open+0x20/0x22
 [<ffffffff8020c3ab>] system_call_after_swapgs+0xfb/0x168

---------------------------
| preempt count: 00000001 ]
| 1-level deep critical section nesting:
----------------------------------------
.. [<ffffffff802d3cda>] .... mnt_want_write+0x19/0xbc
.....[<ffffffff802cea88>] ..   ( <= touch_atime+0x1c/0x101)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fix-bug-messages.patch --]
[-- Type: text/x-patch; name=fix-bug-messages.patch, Size: 3502 bytes --]

From: chirag@linux.vnet.ibm.com Chirag Jog 
To: j.mell@t-online.de J?rgen Mell, tglx@linutronix.de Thomas Gleixner 
Date: Wed, 30 Jul 2008 19:18:42 +0200 
Subject: [PATCH] Fix Bug messages 

This patch should solve some of the bug messages.
It does two things:
1. Change rt_runtime_lock to be a raw spinlock as the comment above it
says: it is nested inside the rq lock.

2. Change mnt_writers to be a per_cpu locked variable.
This eliminates the need for the codepath to disable preemption and
then potentially sleep, leading to the BUG messages

Signed-off-by: Chirag <chirag@linux.vnet.ibm.com>
[updated Chirag's patch against 2.6.26.3-rt3]
Signed-off-by: John Kacur <jkacur at gmail dot com>

Index: linux-2.6.26.3/fs/namespace.c
===================================================================
--- linux-2.6.26.3.orig/fs/namespace.c
+++ linux-2.6.26.3/fs/namespace.c
@@ -178,13 +178,13 @@ struct mnt_writer {
 	unsigned long count;
 	struct vfsmount *mnt;
 } ____cacheline_aligned_in_smp;
-static DEFINE_PER_CPU(struct mnt_writer, mnt_writers);
+static DEFINE_PER_CPU_LOCKED(struct mnt_writer, mnt_writers);
 
 static int __init init_mnt_writers(void)
 {
 	int cpu;
 	for_each_possible_cpu(cpu) {
-		struct mnt_writer *writer = &per_cpu(mnt_writers, cpu);
+		struct mnt_writer *writer = &per_cpu_var_locked(mnt_writers, cpu);
 		spin_lock_init(&writer->lock);
 		lockdep_set_class(&writer->lock, &writer->lock_class);
 		writer->count = 0;
@@ -199,7 +199,7 @@ static void unlock_mnt_writers(void)
 	struct mnt_writer *cpu_writer;
 
 	for_each_possible_cpu(cpu) {
-		cpu_writer = &per_cpu(mnt_writers, cpu);
+		cpu_writer = &per_cpu_var_locked(mnt_writers, cpu);
 		spin_unlock(&cpu_writer->lock);
 	}
 }
@@ -251,8 +251,8 @@ int mnt_want_write(struct vfsmount *mnt)
 {
 	int ret = 0;
 	struct mnt_writer *cpu_writer;
-
-	cpu_writer = &get_cpu_var(mnt_writers);
+	int cpu = 0;
+	cpu_writer = &get_cpu_var_locked(mnt_writers, &cpu);
 	spin_lock(&cpu_writer->lock);
 	if (__mnt_is_readonly(mnt)) {
 		ret = -EROFS;
@@ -262,7 +262,7 @@ int mnt_want_write(struct vfsmount *mnt)
 	cpu_writer->count++;
 out:
 	spin_unlock(&cpu_writer->lock);
-	put_cpu_var(mnt_writers);
+	put_cpu_var_locked(mnt_writers, cpu);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(mnt_want_write);
@@ -273,7 +273,7 @@ static void lock_mnt_writers(void)
 	struct mnt_writer *cpu_writer;
 
 	for_each_possible_cpu(cpu) {
-		cpu_writer = &per_cpu(mnt_writers, cpu);
+		cpu_writer = &per_cpu_var_locked(mnt_writers, cpu);
 		spin_lock(&cpu_writer->lock);
 		__clear_mnt_count(cpu_writer);
 		cpu_writer->mnt = NULL;
@@ -332,8 +332,8 @@ void mnt_drop_write(struct vfsmount *mnt
 {
 	int must_check_underflow = 0;
 	struct mnt_writer *cpu_writer;
-
-	cpu_writer = &get_cpu_var(mnt_writers);
+	int cpu = 0;
+	cpu_writer = &get_cpu_var_locked(mnt_writers, &cpu);
 	spin_lock(&cpu_writer->lock);
 
 	use_cpu_writer_for_mount(cpu_writer, mnt);
@@ -360,7 +360,7 @@ void mnt_drop_write(struct vfsmount *mnt
 	 * __mnt_writers can underflow.  Without it,
 	 * we could theoretically wrap __mnt_writers.
 	 */
-	put_cpu_var(mnt_writers);
+	put_cpu_var_locked(mnt_writers, cpu);
 }
 EXPORT_SYMBOL_GPL(mnt_drop_write);
 
@@ -612,7 +612,7 @@ static inline void __mntput(struct vfsmo
 	 * can come in.
 	 */
 	for_each_possible_cpu(cpu) {
-		struct mnt_writer *cpu_writer = &per_cpu(mnt_writers, cpu);
+		struct mnt_writer *cpu_writer = &per_cpu_var_locked(mnt_writers, cpu);
 		if (cpu_writer->mnt != mnt)
 			continue;
 		spin_lock(&cpu_writer->lock);

Re: 2.6.26.3-rt2

[John Kacur]

[-- Attachment #1: Type: text/plain, Size: 3895 bytes --]

On Thu, Aug 21, 2008 at 9:33 PM, John Kacur <jkacur@gmail.com> wrote:
> On Thu, Aug 21, 2008 at 7:18 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>> We are pleased to announce the 2.6.26.3-rt2 tree, which can be
>> downloaded from the location:
>>
>>  http://rt.et.redhat.com/download/
>>
>> Information on the RT patch can be found at:
>>
>>  http://rt.wiki.kernel.org/index.php/Main_Page
>>
>> Changes since 2.6.26-rt1
>>
>>  - ported to 2.6.26.3
>>
>>  - basic compile fixes (Steven Rostedt)
>>
>>  - basic raw lock conversions (Steven Rostedt)
>>
>>  - powerpc: irq radix tree updates (Sebastien Dugue)
>>
>>  - call_rcu_sched (Paul E. McKenney)
>>
>>  - rcu trace preemption fixes (Paul E. Mckenney)
>>
>>  - remove lock in pm_qos_requirement (John Kacur)
>>
>>  - rcu_process_callbacks (Paul E. McKenney)
>>
>>  - rcu memory barriers update (Paul E. McKenney)
>>
>>  - shorten posix_cpu_timers thread names (Arnaldo Carvalho de Melo)
>>
>>  - account IRQ and RT load in SCHED_OTHER (Peter Zijlstra)
>>
>>  - seq_lock updates/fixes (Gregory Haskins)
>>
>> to build a 2.6.26.3-rt2 tree, the following patches should be applied:
>>
>>  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.26.tar.bz2
>>  http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.26.3.bz2
>>  http://rt.et.redhat.com/download/patch-2.6.26.3-rt2.bz2
>>
>>
>>
>> And like always, my RT version of Matt Mackall's ketchup will get this
>> for you nicely:
>>
>>  http://people.redhat.com/srostedt/rt/tools/ketchup-0.9.8-rt3
>>
>>
>> The broken out patches are also available.
>>
>
> Hi Steve
>
> Probably one of the most important patches that was missed is the one
> from Chirag. I've taken it and updated it against 2.6.26.3-rt2 for
> you, in case that will speed up -rt3 (see attached). (two hunks were
> rejected because another applied fix fixed the same thing as part of
> Chirag's patch.
>
> I've compiled and done some limited testing, and among other things it
> removes this kind of message.
> BUG: sleeping function called from invalid context hald-runner(12244)
> at kernel/rtmutex.c:743
> in_atomic():1 [00000001], irqs_disabled():0
> Pid: 12244, comm: hald-runner Tainted: G        W 2.6.26.3-rt2 #1
>
> Call Trace:
>  [<ffffffff802d3cda>] ? mnt_want_write+0x19/0xbc
>  [<ffffffff802307b6>] __might_sleep+0x12d/0x132
>  [<ffffffff8046d3de>] __rt_spin_lock+0x34/0x7d
>  [<ffffffff8046d435>] rt_spin_lock+0xe/0x10
>  [<ffffffff802d3cfc>] mnt_want_write+0x3b/0xbc
>  [<ffffffff802cea88>] touch_atime+0x1c/0x101
>  [<ffffffff802c426c>] __link_path_walk+0x484/0xf24
>  [<ffffffff802c4d72>] path_walk+0x66/0xc9
>  [<ffffffff802c51bf>] do_path_lookup+0x209/0x264
>  [<ffffffff802c5509>] __path_lookup_intent_open+0x61/0xa4
>  [<ffffffff802c555d>] path_lookup_open+0x11/0x13
>  [<ffffffff802c61f1>] do_filp_open+0xb6/0x8f5
>  [<ffffffff8046d3de>] ? __rt_spin_lock+0x34/0x7d
>  [<ffffffff802b897d>] ? get_unused_fd_flags+0x119/0x12a
>  [<ffffffff802b89e4>] do_sys_open+0x56/0xd6
>  [<ffffffff8020c374>] ? system_call_after_swapgs+0xc4/0x168
>  [<ffffffff802b8a97>] sys_open+0x20/0x22
>  [<ffffffff8020c3ab>] system_call_after_swapgs+0xfb/0x168
>
> ---------------------------
> | preempt count: 00000001 ]
> | 1-level deep critical section nesting:
> ----------------------------------------
> .. [<ffffffff802d3cda>] .... mnt_want_write+0x19/0xbc
> .....[<ffffffff802cea88>] ..   ( <= touch_atime+0x1c/0x101)
>

Another patch that should be included in -rt3 was discussed here.
http://marc.info/?l=linux-rt-users&m=121751295105462&w=2

After some attempts at fixing this myself, Peter Zijlstra offered the
above solution (in the link) which I've been running with ever since,
no changes required, and it solved the problem.
I'm attaching the patch again, I think it would be optimal to get
Peter's Signed-off-by: on it as he is the author of the patch.
No changes or refreshing were required against 2.6.26.3-rt2

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: sock_prot_inuse_add-fix.patch --]
[-- Type: text/x-patch; name=sock_prot_inuse_add-fix.patch, Size: 1068 bytes --]

This patch disables preemption for a very short time, so it doesn't hurt
preempt-latency. The author of this solution was Peter Zijlstra.

Signed-off-by: John Kacur <jkacur at gmail dot com>
Index: linux-2.6.26.2-rt1-jk/net/core/sock.c
===================================================================
--- linux-2.6.26.2-rt1-jk.orig/net/core/sock.c
+++ linux-2.6.26.2-rt1-jk/net/core/sock.c
@@ -1943,8 +1943,9 @@ static DECLARE_BITMAP(proto_inuse_idx, P
 #ifdef CONFIG_NET_NS
 void sock_prot_inuse_add(struct net *net, struct proto *prot, int val)
 {
-	int cpu = smp_processor_id();
+	int cpu = get_cpu();
 	per_cpu_ptr(net->core.inuse, cpu)->val[prot->inuse_idx] += val;
+	put_cpu();
 }
 EXPORT_SYMBOL_GPL(sock_prot_inuse_add);
 
@@ -1990,7 +1991,9 @@ static DEFINE_PER_CPU(struct prot_inuse,
 
 void sock_prot_inuse_add(struct net *net, struct proto *prot, int val)
 {
-	__get_cpu_var(prot_inuse).val[prot->inuse_idx] += val;
+	int cpu = get_cpu();
+	per_cpu(prot_inuse, cpu).val[prot->inuse_idx] += val;
+	put_cpu();
 }
 EXPORT_SYMBOL_GPL(sock_prot_inuse_add);