* [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section
@ 2010-08-30 17:26 Paul E. McKenney
2010-08-30 19:51 ` Jiri Slaby
2010-08-31 13:02 ` David Howells
0 siblings, 2 replies; 9+ messages in thread
From: Paul E. McKenney @ 2010-08-30 17:26 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc,
niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells,
eric.dumazet, jslaby, jmorris
[ 23.584719]
[ 23.584720] ===================================================
[ 23.585059] [ INFO: suspicious rcu_dereference_check() usage. ]
[ 23.585176] ---------------------------------------------------
[ 23.585176] kernel/pid.c:419 invoked rcu_dereference_check() without protection!
[ 23.585176]
[ 23.585176] other info that might help us debug this:
[ 23.585176]
[ 23.585176]
[ 23.585176] rcu_scheduler_active = 1, debug_locks = 1
[ 23.585176] 1 lock held by rc.sysinit/728:
[ 23.585176] #0: (tasklist_lock){.+.+..}, at: [<ffffffff8104771f>] sys_setpgid+0x5f/0x193
[ 23.585176]
[ 23.585176] stack backtrace:
[ 23.585176] Pid: 728, comm: rc.sysinit Not tainted 2.6.36-rc2 #2
[ 23.585176] Call Trace:
[ 23.585176] [<ffffffff8105b436>] lockdep_rcu_dereference+0x99/0xa2
[ 23.585176] [<ffffffff8104c324>] find_task_by_pid_ns+0x50/0x6a
[ 23.585176] [<ffffffff8104c35b>] find_task_by_vpid+0x1d/0x1f
[ 23.585176] [<ffffffff81047727>] sys_setpgid+0x67/0x193
[ 23.585176] [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b
[ 24.959669] type=1400 audit(1282938522.956:4): avc: denied { module_request } for pid=766 comm="hwclock" kmod="char-major-10-135" scontext=system_u:system_r:hwclock_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclas
It turns out that the setpgid() system call fails to enter an RCU
read-side critical section before doing a PID-to-task_struct translation.
This commit therefore does rcu_read_lock() before the translation, and
also does rcu_read_unlock() after the last use of the returned pointer.
Located-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
sys.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/sys.c b/kernel/sys.c
index e9ad444..05a4b0c 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -938,6 +938,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid)
write_lock_irq(&tasklist_lock);
err = -ESRCH;
+ rcu_read_lock();
p = find_task_by_vpid(pid);
if (!p)
goto out;
@@ -983,6 +984,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid)
err = 0;
out:
/* All paths lead to here, thus we are safe. -DaveM */
+ rcu_read_unlock();
write_unlock_irq(&tasklist_lock);
return err;
}
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-08-30 17:26 [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section Paul E. McKenney @ 2010-08-30 19:51 ` Jiri Slaby 2010-08-30 20:32 ` Paul E. McKenney 2010-09-09 22:15 ` Oleg Nesterov 2010-08-31 13:02 ` David Howells 1 sibling, 2 replies; 9+ messages in thread From: Jiri Slaby @ 2010-08-30 19:51 UTC (permalink / raw) To: paulmck Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells, eric.dumazet, jmorris, Oleg Nesterov Ccing Oleg. On 08/30/2010 07:26 PM, Paul E. McKenney wrote: > [ 23.584720] =================================================== > [ 23.585059] [ INFO: suspicious rcu_dereference_check() usage. ] > [ 23.585176] --------------------------------------------------- > [ 23.585176] kernel/pid.c:419 invoked rcu_dereference_check() without protection! > [ 23.585176] > [ 23.585176] other info that might help us debug this: > [ 23.585176] > [ 23.585176] > [ 23.585176] rcu_scheduler_active = 1, debug_locks = 1 > [ 23.585176] 1 lock held by rc.sysinit/728: > [ 23.585176] #0: (tasklist_lock){.+.+..}, at: [<ffffffff8104771f>] sys_setpgid+0x5f/0x193 > [ 23.585176] > [ 23.585176] stack backtrace: > [ 23.585176] Pid: 728, comm: rc.sysinit Not tainted 2.6.36-rc2 #2 > [ 23.585176] Call Trace: > [ 23.585176] [<ffffffff8105b436>] lockdep_rcu_dereference+0x99/0xa2 > [ 23.585176] [<ffffffff8104c324>] find_task_by_pid_ns+0x50/0x6a > [ 23.585176] [<ffffffff8104c35b>] find_task_by_vpid+0x1d/0x1f > [ 23.585176] [<ffffffff81047727>] sys_setpgid+0x67/0x193 > [ 23.585176] [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b > [ 24.959669] type=1400 audit(1282938522.956:4): avc: denied { module_request } for pid=766 comm="hwclock" kmod="char-major-10-135" scontext=system_u:system_r:hwclock_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclas > > It turns out that the setpgid() system call fails to enter an RCU > read-side critical section before doing a PID-to-task_struct translation. > This commit therefore does rcu_read_lock() before the translation, and > also does rcu_read_unlock() after the last use of the returned pointer. > > Located-by: Andrew Morton <akpm@linux-foundation.org> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> > --- > > sys.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/kernel/sys.c b/kernel/sys.c > index e9ad444..05a4b0c 100644 > --- a/kernel/sys.c > +++ b/kernel/sys.c > @@ -938,6 +938,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) > write_lock_irq(&tasklist_lock); > > err = -ESRCH; > + rcu_read_lock(); > p = find_task_by_vpid(pid); AFAICT the missing lock doesn't harm due to the write_lock of tasklist above. But is probably a good thing to do anyway. regards, -- js suse labs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-08-30 19:51 ` Jiri Slaby @ 2010-08-30 20:32 ` Paul E. McKenney 2010-09-09 22:15 ` Oleg Nesterov 1 sibling, 0 replies; 9+ messages in thread From: Paul E. McKenney @ 2010-08-30 20:32 UTC (permalink / raw) To: Jiri Slaby Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells, eric.dumazet, jmorris, Oleg Nesterov On Mon, Aug 30, 2010 at 09:51:07PM +0200, Jiri Slaby wrote: > Ccing Oleg. > > On 08/30/2010 07:26 PM, Paul E. McKenney wrote: > > [ 23.584720] =================================================== > > [ 23.585059] [ INFO: suspicious rcu_dereference_check() usage. ] > > [ 23.585176] --------------------------------------------------- > > [ 23.585176] kernel/pid.c:419 invoked rcu_dereference_check() without protection! > > [ 23.585176] > > [ 23.585176] other info that might help us debug this: > > [ 23.585176] > > [ 23.585176] > > [ 23.585176] rcu_scheduler_active = 1, debug_locks = 1 > > [ 23.585176] 1 lock held by rc.sysinit/728: > > [ 23.585176] #0: (tasklist_lock){.+.+..}, at: [<ffffffff8104771f>] sys_setpgid+0x5f/0x193 > > [ 23.585176] > > [ 23.585176] stack backtrace: > > [ 23.585176] Pid: 728, comm: rc.sysinit Not tainted 2.6.36-rc2 #2 > > [ 23.585176] Call Trace: > > [ 23.585176] [<ffffffff8105b436>] lockdep_rcu_dereference+0x99/0xa2 > > [ 23.585176] [<ffffffff8104c324>] find_task_by_pid_ns+0x50/0x6a > > [ 23.585176] [<ffffffff8104c35b>] find_task_by_vpid+0x1d/0x1f > > [ 23.585176] [<ffffffff81047727>] sys_setpgid+0x67/0x193 > > [ 23.585176] [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b > > [ 24.959669] type=1400 audit(1282938522.956:4): avc: denied { module_request } for pid=766 comm="hwclock" kmod="char-major-10-135" scontext=system_u:system_r:hwclock_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclas > > > > It turns out that the setpgid() system call fails to enter an RCU > > read-side critical section before doing a PID-to-task_struct translation. > > This commit therefore does rcu_read_lock() before the translation, and > > also does rcu_read_unlock() after the last use of the returned pointer. > > > > Located-by: Andrew Morton <akpm@linux-foundation.org> > > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> > > --- > > > > sys.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/kernel/sys.c b/kernel/sys.c > > index e9ad444..05a4b0c 100644 > > --- a/kernel/sys.c > > +++ b/kernel/sys.c > > @@ -938,6 +938,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) > > write_lock_irq(&tasklist_lock); > > > > err = -ESRCH; > > + rcu_read_lock(); > > p = find_task_by_vpid(pid); > > AFAICT the missing lock doesn't harm due to the write_lock of tasklist > above. But is probably a good thing to do anyway. Or we can add the tasklist lock to the rcu_dereference_check() condition. Thanx, Paul > regards, > -- > js > suse labs > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-08-30 19:51 ` Jiri Slaby 2010-08-30 20:32 ` Paul E. McKenney @ 2010-09-09 22:15 ` Oleg Nesterov 2010-09-16 9:18 ` Jiri Slaby 1 sibling, 1 reply; 9+ messages in thread From: Oleg Nesterov @ 2010-09-09 22:15 UTC (permalink / raw) To: Jiri Slaby Cc: paulmck, linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells, eric.dumazet, jmorris On 08/30, Jiri Slaby wrote: > > Ccing Oleg. Sorry for delay... > > --- a/kernel/sys.c > > +++ b/kernel/sys.c > > @@ -938,6 +938,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) > > write_lock_irq(&tasklist_lock); > > > > err = -ESRCH; > > + rcu_read_lock(); > > p = find_task_by_vpid(pid); > > AFAICT the missing lock doesn't harm due to the write_lock of tasklist > above. But is probably a good thing to do anyway. The problem is, find_task_by_vpid() is not safe without RCU. It is not that the returned task_struct can't go away, find_pid_ns() itself is not safe. This is because the failing copy_process() calls free_pid() without tasklist_lock and modifies pid_hash[] list. Oleg. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-09-09 22:15 ` Oleg Nesterov @ 2010-09-16 9:18 ` Jiri Slaby 2010-09-16 16:39 ` Oleg Nesterov 0 siblings, 1 reply; 9+ messages in thread From: Jiri Slaby @ 2010-09-16 9:18 UTC (permalink / raw) To: Oleg Nesterov Cc: paulmck, linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells, eric.dumazet, jmorris, stable On 09/10/2010 12:15 AM, Oleg Nesterov wrote: > On 08/30, Jiri Slaby wrote: >>> --- a/kernel/sys.c >>> +++ b/kernel/sys.c >>> @@ -938,6 +938,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) >>> write_lock_irq(&tasklist_lock); >>> >>> err = -ESRCH; >>> + rcu_read_lock(); >>> p = find_task_by_vpid(pid); >> >> AFAICT the missing lock doesn't harm due to the write_lock of tasklist >> above. But is probably a good thing to do anyway. > > The problem is, find_task_by_vpid() is not safe without RCU. It is not > that the returned task_struct can't go away, find_pid_ns() itself is > not safe. This is because the failing copy_process() calls free_pid() > without tasklist_lock and modifies pid_hash[] list. That said, it (950eaaca681c4) should probably go into stable. (Apply to all 32-35 whichever are maintained currently.) thanks, -- js suse labs ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-09-16 9:18 ` Jiri Slaby @ 2010-09-16 16:39 ` Oleg Nesterov 0 siblings, 0 replies; 9+ messages in thread From: Oleg Nesterov @ 2010-09-16 16:39 UTC (permalink / raw) To: Jiri Slaby Cc: paulmck, linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, dhowells, eric.dumazet, jmorris, stable On 09/16, Jiri Slaby wrote: > > On 09/10/2010 12:15 AM, Oleg Nesterov wrote: > > On 08/30, Jiri Slaby wrote: > >>> --- a/kernel/sys.c > >>> +++ b/kernel/sys.c > >>> @@ -938,6 +938,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) > >>> write_lock_irq(&tasklist_lock); > >>> > >>> err = -ESRCH; > >>> + rcu_read_lock(); > >>> p = find_task_by_vpid(pid); > >> > >> AFAICT the missing lock doesn't harm due to the write_lock of tasklist > >> above. But is probably a good thing to do anyway. > > > > The problem is, find_task_by_vpid() is not safe without RCU. It is not > > that the returned task_struct can't go away, find_pid_ns() itself is > > not safe. This is because the failing copy_process() calls free_pid() > > without tasklist_lock and modifies pid_hash[] list. > > That said, it (950eaaca681c4) should probably go into stable. (Apply to > all 32-35 whichever are maintained currently.) Perhaps, but the race is mostly theoretical. To be honest, I think 950eaaca681c4 needs a comment to explain what rcu_read_lock() protects, or perhaps we can make it more explicit. Oleg. --- a/kernel/sys.c +++ b/kernel/sys.c @@ -931,7 +931,6 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid pgid = pid; if (pgid < 0) return -EINVAL; - rcu_read_lock(); /* From this point forward we keep holding onto the tasklist lock * so that our parent does not change from under us. -DaveM @@ -939,7 +938,9 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid write_lock_irq(&tasklist_lock); err = -ESRCH; + rcu_read_lock(); p = find_task_by_vpid(pid); + rcu_read_unlock(); if (!p) goto out; @@ -968,7 +969,9 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid if (pgid != pid) { struct task_struct *g; + rcu_read_lock(); pgrp = find_vpid(pgid); + rcu_read_unlock(); g = pid_task(pgrp, PIDTYPE_PGID); if (!g || task_session(g) != task_session(group_leader)) goto out; @@ -985,7 +988,6 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid out: /* All paths lead to here, thus we are safe. -DaveM */ write_unlock_irq(&tasklist_lock); - rcu_read_unlock(); return err; } ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-08-30 17:26 [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section Paul E. McKenney 2010-08-30 19:51 ` Jiri Slaby @ 2010-08-31 13:02 ` David Howells 2010-08-31 15:12 ` Paul E. McKenney 1 sibling, 1 reply; 9+ messages in thread From: David Howells @ 2010-08-31 13:02 UTC (permalink / raw) To: paulmck Cc: dhowells, linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, eric.dumazet, jslaby, jmorris Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote: > write_lock_irq(&tasklist_lock); > > err = -ESRCH; > + rcu_read_lock(); > ... > + rcu_read_unlock(); > write_unlock_irq(&tasklist_lock); I would generally put the rcu lock outside the IRQ disabled section on the basis that it's better to keep the amount of time we have interrupts disabled shortest. > Located-by: Andrew Morton <akpm@linux-foundation.org> 'Reported-by' might be more consistent with what others use. David ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-08-31 13:02 ` David Howells @ 2010-08-31 15:12 ` Paul E. McKenney 2010-08-31 15:31 ` David Howells 0 siblings, 1 reply; 9+ messages in thread From: Paul E. McKenney @ 2010-08-31 15:12 UTC (permalink / raw) To: David Howells Cc: linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, eric.dumazet, jslaby, jmorris On Tue, Aug 31, 2010 at 02:02:31PM +0100, David Howells wrote: > Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote: > > > write_lock_irq(&tasklist_lock); > > > > err = -ESRCH; > > + rcu_read_lock(); > > ... > > + rcu_read_unlock(); > > write_unlock_irq(&tasklist_lock); > > I would generally put the rcu lock outside the IRQ disabled section on the > basis that it's better to keep the amount of time we have interrupts disabled > shortest. > > > Located-by: Andrew Morton <akpm@linux-foundation.org> > > 'Reported-by' might be more consistent with what others use. > > David Good points, how about the following? Thanx, Paul ------------------------------------------------------------------------ pid: make setpgid() system call use RCU read-side critical section [ 23.584719] [ 23.584720] =================================================== [ 23.585059] [ INFO: suspicious rcu_dereference_check() usage. ] [ 23.585176] --------------------------------------------------- [ 23.585176] kernel/pid.c:419 invoked rcu_dereference_check() without protection! [ 23.585176] [ 23.585176] other info that might help us debug this: [ 23.585176] [ 23.585176] [ 23.585176] rcu_scheduler_active = 1, debug_locks = 1 [ 23.585176] 1 lock held by rc.sysinit/728: [ 23.585176] #0: (tasklist_lock){.+.+..}, at: [<ffffffff8104771f>] sys_setpgid+0x5f/0x193 [ 23.585176] [ 23.585176] stack backtrace: [ 23.585176] Pid: 728, comm: rc.sysinit Not tainted 2.6.36-rc2 #2 [ 23.585176] Call Trace: [ 23.585176] [<ffffffff8105b436>] lockdep_rcu_dereference+0x99/0xa2 [ 23.585176] [<ffffffff8104c324>] find_task_by_pid_ns+0x50/0x6a [ 23.585176] [<ffffffff8104c35b>] find_task_by_vpid+0x1d/0x1f [ 23.585176] [<ffffffff81047727>] sys_setpgid+0x67/0x193 [ 23.585176] [<ffffffff810029eb>] system_call_fastpath+0x16/0x1b [ 24.959669] type=1400 audit(1282938522.956:4): avc: denied { module_request } for pid=766 comm="hwclock" kmod="char-major-10-135" scontext=system_u:system_r:hwclock_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclas It turns out that the setpgid() system call fails to enter an RCU read-side critical section before doing a PID-to-task_struct translation. This commit therefore does rcu_read_lock() before the translation, and also does rcu_read_unlock() after the last use of the returned pointer. Reported-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> diff --git a/kernel/sys.c b/kernel/sys.c index e9ad444..7f5a0cd 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -931,6 +931,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) pgid = pid; if (pgid < 0) return -EINVAL; + rcu_read_lock(); /* From this point forward we keep holding onto the tasklist lock * so that our parent does not change from under us. -DaveM @@ -984,6 +985,7 @@ SYSCALL_DEFINE2(setpgid, pid_t, pid, pid_t, pgid) out: /* All paths lead to here, thus we are safe. -DaveM */ write_unlock_irq(&tasklist_lock); + rcu_read_unlock(); return err; } ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section 2010-08-31 15:12 ` Paul E. McKenney @ 2010-08-31 15:31 ` David Howells 0 siblings, 0 replies; 9+ messages in thread From: David Howells @ 2010-08-31 15:31 UTC (permalink / raw) To: paulmck Cc: dhowells, linux-kernel, mingo, laijs, dipankar, akpm, mathieu.desnoyers, josh, dvhltc, niv, tglx, peterz, rostedt, Valdis.Kletnieks, eric.dumazet, jslaby, jmorris Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote: > It turns out that the setpgid() system call fails to enter an RCU > read-side critical section before doing a PID-to-task_struct translation. > This commit therefore does rcu_read_lock() before the translation, and > also does rcu_read_unlock() after the last use of the returned pointer. > > Reported-by: Andrew Morton <akpm@linux-foundation.org> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Acked-by: David Howells <dhowells@redhat.com> ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2010-09-16 17:14 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-08-30 17:26 [PATCH RFC] pid: make setpgid() system call use RCU read-side critical section Paul E. McKenney 2010-08-30 19:51 ` Jiri Slaby 2010-08-30 20:32 ` Paul E. McKenney 2010-09-09 22:15 ` Oleg Nesterov 2010-09-16 9:18 ` Jiri Slaby 2010-09-16 16:39 ` Oleg Nesterov 2010-08-31 13:02 ` David Howells 2010-08-31 15:12 ` Paul E. McKenney 2010-08-31 15:31 ` David Howells
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox