Re: [patch] latency tracer, 2.6.15-rc7

[Lee Revell <rlrevell@joe-job.com>]

On Sat, 2005-12-31 at 12:14 -0800, Paul E. McKenney wrote:
> So it seems to me that Linus's patch is part of the solution, but
> needs to also have a global component, perhaps as follows:
> 
> 	if (unlikely(rdp->count > 100)) {
> 		set_need_resched();

In fact neither of these patches helps, because these RCU callbacks run
from a tasklet in softirq context, and softirqs are not preemptible
(unless they are running in threads, see below):

                 _------=> CPU#        
                / _-----=> irqs-off        
               | / _----=> need-resched    
               || / _---=> hardirq/softirq <---------***
               ||| / _--=> preempt-depth   
               |||| /        
               |||||     delay        
   cmd     pid ||||| time  |   caller      
      \   /    |||||   \   |   /        
  <idle>-0     0d.s2    1us : __trace_start_sched_wakeup (try_to_wake_up)
  <idle>-0     0d.s2    1us : __trace_start_sched_wakeup <<...>-21330> (73 0)
  <idle>-0     0d.s.    2us : wake_up_process (process_timeout)
  <idle>-0     0d.s.    3us : tasklet_action (__do_softirq)
  <idle>-0     0d.s.    4us : rcu_process_callbacks (tasklet_action)
  <idle>-0     0d.s.    5us : __rcu_process_callbacks (rcu_process_callbacks)
  <idle>-0     0d.s.    5us : rcu_check_quiescent_state (__rcu_process_callbacks)
  <idle>-0     0d.s.    6us : __rcu_process_callbacks (rcu_process_callbacks)
  <idle>-0     0d.s.    7us : rcu_check_quiescent_state (__rcu_process_callbacks)
  <idle>-0     0d.s.    7us : rcu_do_batch (__rcu_process_callbacks)
  <idle>-0     0d.s.    8us : dst_rcu_free (rcu_do_batch)

Fortunately softirq preemption is one of the simplest parts of the -rt
patch - since the kernel already runs all softirqs in threads under
heavy load, it's simply a matter of adding a .config option to always do
that.

Linus, would you accept CONFIG_PREEMPT_SOFTIRQS to always run softirqs
in threads (default N of course, it certainly has a slight throughput
cost) for mainline if Ingo were to submit it?

Lee