unusual scheduling performance

[William Lee Irwin III <wli@holomorphy.com>]

On 16x, 2.5.47 kernel compiles take about 26s when the machine is
otherwise idle.

On 32x, 2.5.47 kernel compiles take about 48s when the machine is 
otherwise idle.

When a single-threaded task consumes an entire cpu, kernel compiles
take 36s on 32s when the machine is idle aside from the task consuming
that cpu and the kernel compile itself.

I suspect the scheduler, because cpu reporting in top(1) shows that a
two or more cpu-intensive tasks are concentrated on the same cpu, and
some long-lived tasks appear to be "bouncing" across cpus. If someone
with knowledge and/or expertise with respect to scheduling semantics
could look into this, I would be much obliged. Resolving this would
likely address many SMP and/or NUMA scheduling performance issues.


Thanks,
Bill