Re: 2.6.26.3-rt2

["John Kacur" <jkacur@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3895 bytes --]

On Thu, Aug 21, 2008 at 9:33 PM, John Kacur <jkacur@gmail.com> wrote:
> On Thu, Aug 21, 2008 at 7:18 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>> We are pleased to announce the 2.6.26.3-rt2 tree, which can be
>> downloaded from the location:
>>
>>  http://rt.et.redhat.com/download/
>>
>> Information on the RT patch can be found at:
>>
>>  http://rt.wiki.kernel.org/index.php/Main_Page
>>
>> Changes since 2.6.26-rt1
>>
>>  - ported to 2.6.26.3
>>
>>  - basic compile fixes (Steven Rostedt)
>>
>>  - basic raw lock conversions (Steven Rostedt)
>>
>>  - powerpc: irq radix tree updates (Sebastien Dugue)
>>
>>  - call_rcu_sched (Paul E. McKenney)
>>
>>  - rcu trace preemption fixes (Paul E. Mckenney)
>>
>>  - remove lock in pm_qos_requirement (John Kacur)
>>
>>  - rcu_process_callbacks (Paul E. McKenney)
>>
>>  - rcu memory barriers update (Paul E. McKenney)
>>
>>  - shorten posix_cpu_timers thread names (Arnaldo Carvalho de Melo)
>>
>>  - account IRQ and RT load in SCHED_OTHER (Peter Zijlstra)
>>
>>  - seq_lock updates/fixes (Gregory Haskins)
>>
>> to build a 2.6.26.3-rt2 tree, the following patches should be applied:
>>
>>  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.26.tar.bz2
>>  http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.26.3.bz2
>>  http://rt.et.redhat.com/download/patch-2.6.26.3-rt2.bz2
>>
>>
>>
>> And like always, my RT version of Matt Mackall's ketchup will get this
>> for you nicely:
>>
>>  http://people.redhat.com/srostedt/rt/tools/ketchup-0.9.8-rt3
>>
>>
>> The broken out patches are also available.
>>
>
> Hi Steve
>
> Probably one of the most important patches that was missed is the one
> from Chirag. I've taken it and updated it against 2.6.26.3-rt2 for
> you, in case that will speed up -rt3 (see attached). (two hunks were
> rejected because another applied fix fixed the same thing as part of
> Chirag's patch.
>
> I've compiled and done some limited testing, and among other things it
> removes this kind of message.
> BUG: sleeping function called from invalid context hald-runner(12244)
> at kernel/rtmutex.c:743
> in_atomic():1 [00000001], irqs_disabled():0
> Pid: 12244, comm: hald-runner Tainted: G        W 2.6.26.3-rt2 #1
>
> Call Trace:
>  [<ffffffff802d3cda>] ? mnt_want_write+0x19/0xbc
>  [<ffffffff802307b6>] __might_sleep+0x12d/0x132
>  [<ffffffff8046d3de>] __rt_spin_lock+0x34/0x7d
>  [<ffffffff8046d435>] rt_spin_lock+0xe/0x10
>  [<ffffffff802d3cfc>] mnt_want_write+0x3b/0xbc
>  [<ffffffff802cea88>] touch_atime+0x1c/0x101
>  [<ffffffff802c426c>] __link_path_walk+0x484/0xf24
>  [<ffffffff802c4d72>] path_walk+0x66/0xc9
>  [<ffffffff802c51bf>] do_path_lookup+0x209/0x264
>  [<ffffffff802c5509>] __path_lookup_intent_open+0x61/0xa4
>  [<ffffffff802c555d>] path_lookup_open+0x11/0x13
>  [<ffffffff802c61f1>] do_filp_open+0xb6/0x8f5
>  [<ffffffff8046d3de>] ? __rt_spin_lock+0x34/0x7d
>  [<ffffffff802b897d>] ? get_unused_fd_flags+0x119/0x12a
>  [<ffffffff802b89e4>] do_sys_open+0x56/0xd6
>  [<ffffffff8020c374>] ? system_call_after_swapgs+0xc4/0x168
>  [<ffffffff802b8a97>] sys_open+0x20/0x22
>  [<ffffffff8020c3ab>] system_call_after_swapgs+0xfb/0x168
>
> ---------------------------
> | preempt count: 00000001 ]
> | 1-level deep critical section nesting:
> ----------------------------------------
> .. [<ffffffff802d3cda>] .... mnt_want_write+0x19/0xbc
> .....[<ffffffff802cea88>] ..   ( <= touch_atime+0x1c/0x101)
>

Another patch that should be included in -rt3 was discussed here.
http://marc.info/?l=linux-rt-users&m=121751295105462&w=2

After some attempts at fixing this myself, Peter Zijlstra offered the
above solution (in the link) which I've been running with ever since,
no changes required, and it solved the problem.
I'm attaching the patch again, I think it would be optimal to get
Peter's Signed-off-by: on it as he is the author of the patch.
No changes or refreshing were required against 2.6.26.3-rt2

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: sock_prot_inuse_add-fix.patch --]
[-- Type: text/x-patch; name=sock_prot_inuse_add-fix.patch, Size: 1068 bytes --]

This patch disables preemption for a very short time, so it doesn't hurt
preempt-latency. The author of this solution was Peter Zijlstra.

Signed-off-by: John Kacur <jkacur at gmail dot com>
Index: linux-2.6.26.2-rt1-jk/net/core/sock.c
===================================================================
--- linux-2.6.26.2-rt1-jk.orig/net/core/sock.c
+++ linux-2.6.26.2-rt1-jk/net/core/sock.c
@@ -1943,8 +1943,9 @@ static DECLARE_BITMAP(proto_inuse_idx, P
 #ifdef CONFIG_NET_NS
 void sock_prot_inuse_add(struct net *net, struct proto *prot, int val)
 {
-	int cpu = smp_processor_id();
+	int cpu = get_cpu();
 	per_cpu_ptr(net->core.inuse, cpu)->val[prot->inuse_idx] += val;
+	put_cpu();
 }
 EXPORT_SYMBOL_GPL(sock_prot_inuse_add);
 
@@ -1990,7 +1991,9 @@ static DEFINE_PER_CPU(struct prot_inuse,
 
 void sock_prot_inuse_add(struct net *net, struct proto *prot, int val)
 {
-	__get_cpu_var(prot_inuse).val[prot->inuse_idx] += val;
+	int cpu = get_cpu();
+	per_cpu(prot_inuse, cpu).val[prot->inuse_idx] += val;
+	put_cpu();
 }
 EXPORT_SYMBOL_GPL(sock_prot_inuse_add);