Re: v2.6.21.4-rt11

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Mon, Jun 11, 2007 at 10:18:06AM -0700, Paul E. McKenney wrote:
> On Mon, Jun 11, 2007 at 08:55:27AM -0700, Paul E. McKenney wrote:
> > On Mon, Jun 11, 2007 at 05:38:55PM +0200, Ingo Molnar wrote:
> > > 
> > > * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> > > 
> > > > > hm, what affinity do they start out with? Could they all be pinned 
> > > > > to CPU#0 by default?
> > > > 
> > > > They start off with affinity masks of 0xf on a 4-CPU system.  I would 
> > > > expect them to load-balance across the four CPUs, but they stay all on 
> > > > the same CPU until long after I lose patience (many minutes).
> > > 
> > > ugh. Would be nice to figure out why this happens. I enabled rcutorture 
> > > on a dual-core CPU and all the threads are spread evenly.
> > 
> > Here is the /proc/cpuinfo in case this helps.  I am starting up a test
> > on a dual-core CPU to see if that works better.
> 
> And this quickly load-balanced to put a pair of readers on each CPU.
> Later, it moved one of the readers so that it is now running with
> one reader on one of the CPUs, and the remaining three readers on the
> other CPU.
> 
> Argh...  this is with 2.6.21-rt1...  Need to reboot with 2.6.21.4-rt12...

OK, here are a couple of snapshots from "top" on a two-way system.
It seems to cycle back and forth between these two states.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
20126 root      39  19     0    0    0 R   47  0.0  11:38.62 rcu_torture_rea    
20129 root      39  19     0    0    0 R   47  0.0  13:28.06 rcu_torture_rea    
20127 root      39  19     0    0    0 R   43  0.0  12:39.83 rcu_torture_rea    
20128 root      39  19     0    0    0 R   43  0.0  11:50.58 rcu_torture_rea    
20121 root      39  19     0    0    0 R   10  0.0   2:59.69 rcu_torture_wri    
20123 root      39  19     0    0    0 D    2  0.0   0:28.52 rcu_torture_fak    
20125 root      39  19     0    0    0 D    2  0.0   0:28.47 rcu_torture_fak    
20122 root      39  19     0    0    0 D    1  0.0   0:28.38 rcu_torture_fak    
20124 root      39  19     0    0    0 D    1  0.0   0:28.41 rcu_torture_fak    

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
20129 root      39  19     0    0    0 R   80  0.0  14:46.56 rcu_torture_rea    
20126 root      39  19     0    0    0 R   33  0.0  12:52.70 rcu_torture_rea    
20128 root      39  19     0    0    0 R   33  0.0  13:01.50 rcu_torture_rea    
20127 root      39  19     0    0    0 R   33  0.0  13:49.68 rcu_torture_rea    
20121 root      39  19     0    0    0 R   13  0.0   3:16.82 rcu_torture_wri    
20122 root      39  19     0    0    0 R    2  0.0   0:31.16 rcu_torture_fak    
20123 root      39  19     0    0    0 R    2  0.0   0:31.25 rcu_torture_fak    
20124 root      39  19     0    0    0 D    2  0.0   0:31.23 rcu_torture_fak    
20125 root      39  19     0    0    0 R    2  0.0   0:31.25 rcu_torture_fak    
12907 root      20   0 12576 1068  796 R    1  0.0   0:08.55 top                

The "preferred" state is the first one.  But given that the readers
will consume all CPU available to them, the scheduler might not be
able to tell the difference.

Perhaps the fakewriters are confusing the scheduler, will try again on
a 4-CPU machine leaving them out.

						Thanx, Paul