Re: Possible bug from kernel 2.6.22 and above

[Jie Chen <chen@jlab.org>]

Simon Holm Thøgersen wrote:
> ons, 21 11 2007 kl. 20:52 -0500, skrev Jie Chen:

> There is a backport of the CFS scheduler to 2.6.21, see
> http://lkml.org/lkml/2007/11/19/127
> 
Hi, Simon:

I will try that after the thanksgiving holiday to find out whether the 
odd behavior will show up using 2.6.21 with back ported CFS.

>>>> Kernel 2.6.21
>>>> Number of Threads              2          4           6         8
>>>> SpinLock (Time micro second)   10.5618    10.58538    10.5915   10.643
>>>>                   (Overhead)   0.073      0.05746     0.102805 0.154563
>>>> Barrier (Time micro second)    11.020410  11.678125   11.9889   12.38002
>>>>                  (Overhead)    0.531660   1.1502      1.500112 1.891617
>>>>
>>>> Each thread is bound to a particular core using pthread_setaffinity_np.
>>>>
>>>> Kernel 2.6.23.8
>>>> Number of Threads              2          4           6         8
>>>> SpinLock (Time micro second)   14.849915  17.117603   14.4496   10.5990
>>>>                  (Overhead)    4.345417   6.617207    3.949435  0.110985
>>>> Barrier (Time micro second)    19.462255  20.285117   16.19395  12.37662
>>>>                  (Overhead)    8.957755   9.784722    5.699590  1.869518
>>>>

> 
> 
> Simon Holm Thøgersen
> 
> 
I just ran a simple test to prove that the problem may be related to 
load balance of the scheduler. I first started 6 processes using 
"taskset -c 2 donothing&; taskset -c 3 donothing&; ..., taskset -c 7 
donothing". These 6 processes will run on core 2 to 7. Then I started my 
test program using two threads bound to core 0 and 1. Here is the result:

Two threads on Kernel 2.6.23.8:
SpinLock (Time micro second)             10.558255
          (Overhead)                      0.068965
Barrier  (Time micro second)             10.865520
          (Overhead)                      0.376230

Similarly, I started 4 donothing processes on core 4, 5, 6 and 7, and 
ran the test program. I have the following result:

Four threads on Kernel 2.6.23.8:
SpinLock (Time micro second)             10.579413
          (Overhead)                      0.090023
Barrier  (Time micro second)             11.363193
          (Overhead)                      0.873803

Finally, here is the result for 6 threads with two donothing processes 
running on core 6 and 7:

Six threads on Kernel 2.6.23.8:
SpinLock (Time micro second)             10.590030
          (Overhead)                      0.100940
Barrier  (Time micro second)             11.977548
          (Overhead)                      1.488458

Now the above results are very much similar to the results obtained for 
the kernel 2.6.21. I hope this helps you guys in some ways. Thank you.

-- 
#############################################################################
# Jie Chen
# Scientific Computing Group
# Thomas Jefferson National Accelerator Facility
# Newport News, VA 23606
#
# chen@jlab.org
# (757)269-5046 (office)
# (757)269-6248 (fax)
#############################################################################