Re: sched_domains and Stream benchmark

[Andi Kleen <ak@muc.de>]

On Tue, Apr 27, 2004 at 09:47:19AM -0700, Darren Hart wrote:
> On Mon, 2004-04-26 at 19:33, Andi Kleen wrote:
> > > I noticed your binary ran with N=2000000 which is only sufficient for a
> > > 2 proc 1 MB cache opteron box according to the documentation on the
> > 
> > It does not seem to make any difference. 
> 
> I was under the impression you didn't change the N value (array size)
> and ran the benchmark with someone else's precompiled binaries (the ones
> you sent me).  Did you have two binaries with different array sizes

Correct.  I always used 2000000. But it did not seem to make
any difference in showing the scheduler issues even when going
to 4 CPUs.

(there were some fluctuations, but much less than the 25% you reported)

> > > stream faq.  I also noticed wide variation in results (25% or so) when
> > > running with 4 threads on a 4 proc opteron on linux-2.6.5-mm5.  Can you
> > > provide me with the specs of the system you ran your tests on?
> > 
> > Yes, mm5 is still broken because it has the "tuned to numasaurus" numa
> > scheduler. Run it on a standard (non mm*) kernel or with Ingo's early 
> > load balance patch.
> 
> I ran it on 2.6.5, 2.6.5-mm5, and 2.6.5-mm5-flat-domains trying to
> reproduce the results you found (including the poor performance of
> virgin and mm) so that I can have some context while analyzing the
> sched_domains topology on x86_64 and its effects on performance.  So
> that I can see where the differences lie in our tests, could you please
> provide some of the specs of the system you ran on, such as number of
> procs, cache size, and amount of RAM.

I saw the issue on a wide range of systems, ranging from 2 CPUs to 4 CPUs.
All hard enough per CPU memory to fit the benchmark.

-Andi