Re: balance storm

Mike Galbraith
2014-05-27T09:56:00Z

On Tue, 2014-05-27 at 15:56 +0800, Libo Chen wrote:
> On 2014/5/26 22:19, Mike Galbraith wrote:
> > On Mon, 2014-05-26 at 20:16 +0800, Libo Chen wrote:
> >> On 2014/5/26 13:11, Mike Galbraith wrote:
> >
> >>> Your synthetic test is the absolute worst case scenario. There has to
> >>> be work between wakeups for select_idle_sibling() to have any chance
> >>> whatsoever of turning in a win. At 0 work, it becomes 100% overhead.
> >>
> >> not synthetic, it is a real problem in our product. under no load, waste
> >> much cpu time.
> >
> > What happens in your product if you apply the commit I pointed out?
>
> under no load, cpu usage is up to 60%, but the same apps cost 10% on
> susp sp1. The apps use a lot of timer.

Something is rotten. 3.14-rt contains that commit, I ran your test with
256 threads on 64 core box, saw ~4%.

Putting master/nopreempt config on box and doing the same test, box is
chewing up truckloads of CPU, but not from migrations.

perf top -g --sort=symbol

Samples: 7M of event 'cycles', Event count (approx.): 1316249172581
- 82.56% [k] _raw_spin_lock_irqsave ▒
- _raw_spin_lock_irqsave ▒
- 96.59% __nanosleep_nocancel ◆
100.00% __libc_start_main ▒
2.88% __poll ▒
0 ▒
+ 1.56% [k] native_write_msr_safe ▒
+ 1.21% [k] update_cfs_shares ▒
+ 0.92% [k] __schedule ▒
+ 0.88% [k] _raw_spin_lock ▒
+ 0.73% [k] update_cfs_rq_blocked_load ▒
+ 0.62% [k] idle_cpu ▒
+ 0.47% [.] usleep ▒
+ 0.41% [k] cpuidle_enter_state ▒
+ 0.37% [k] set_task_cpu

Oh, 256 * usleep(100) is not a great idea.

-Mike