Re: Does Solaris really scale this well?

Re: Does Solaris really scale this well?

[venom]

80% is quite possible, I have similar results with a E10K domain of
around 32 CPUs, with a 100mhz bus. Buf 80% is far from 94%...



On Tue, 20 Aug 2002, Peter Chubb wrote:

> Date: Tue, 20 Aug 2002 08:58:06 +1000
> From: Peter Chubb <peter@chubb.wattle.id.au>
> To: venom@sns.it
> Cc: Larry McVoy <lm@bitmover.com>,
>      Ruth Ivimey-Cook <Ruth.Ivimey-Cook@ivimey.org>,
>      Matti Aarnio <matti.aarnio@zmailer.org>, Dax Kelson <dax@gurulabs.com>,
>      "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
> Subject: Re: Does Solaris really scale this well?
>
> >>>>> "venom" == venom  <venom@sns.it> writes:
>
> venom> On Sat, 17 Aug 2002, Larry McVoy wrote:
> >> Date: Sat, 17 Aug 2002 17:55:17 -0700 From: Larry McVoy
>
> venom> And where reasonable != 94%. Seriously, 94% scalability could
> venom> be on a 8 CPUs 880, but, for example, I have a 64 CPUS domain
> venom> on a E10k which is far from 94% scalability (ok, an old E10k
> venom> with an 83Mhz bus).  For what I saw, maybe SGI Origin 3000 is
> venom> scaling a little better with a lot of CPUS, but I also never
> venom> had an E15000 around for now...
>
> I've played around with 8-way E10000 and a 128-way Origin.
> Both scaled reasonably from an OS perspective --- enabling more cpus on
> a mixed lots-of-small-jobs workload increased performance close to
> linearly --- from memory (and it was a couple of years ago) above
> 80%, and in some tests better than that.  Unfotunately, I no longer
> have access either to the machines or to the data, as I've changed jobs...
>
> Peter C
>

Re: Does Solaris really scale this well?

[Jakob Oestergaard]

On Tue, Aug 20, 2002 at 12:13:44PM +0200, venom@sns.it wrote:
> 
> 80% is quite possible, I have similar results with a E10K domain of
> around 32 CPUs, with a 100mhz bus. Buf 80% is far from 94%...
> 

[shortening CC: to save electrons]

Let's end this thread shall we.

Anyone talking about scalability without talking about workload and
measurement is just spreading BS.

Anyone can get 100% (or even superlinear) scalability on a 1000
processor i486 with a 1 KHz bus, if the workload is right and
performance is measured "right".

The same person can get negative scalability on any THz (my-privates-
are-longer-and-fatter-than-yours)-bus machine out there, if the workload
is again chosen "correctly".

The thread could have gone on-topic, but it didn't   :)

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: IDE?

[Andre Hedrick]

On Sat, 17 Aug 2002, Alexander Kellett wrote:

> [shortening cc:'s as they are probably not interested in me :) ]
> 
> IMO this is not something that you can see a positive, or as a 
> lesson learned. This to me just seems like a lot of hard work 
> going down the drain.

This one of my arguments against deleting Marcin work.
There are things he did which are good and real cleanups.

> So, pleeease Bartlomiej/Alan/Jens, whoever. Someone step up
> to get most/some of Marcin' cleanup patches into 2.5 again.

I have addressed most of his work which was sane and did not violate
transport protocols.

Regards,

Andre Hedrick
LAD Storage Consulting Group

Does Solaris really scale this well?

[Dax Kelson]

From:

http://www.itworld.com/Man/3828/020816mcnealy/

Scott McNealy:

"When you take a 99-way UltraSPARC III machine and add a 100th processor, 
you get 94 percent linear scalability. You can't get 94 percent linear 
scalability on your first Intel chip. It's very, very hard to do, and they 
have not done it."

Re: Does Solaris really scale this well?

[Matti Aarnio]

On Sat, Aug 17, 2002 at 11:53:16AM -0600, Dax Kelson wrote:
> From:
>   http://www.itworld.com/Man/3828/020816mcnealy/
> 
> Scott McNealy:
> 
> "When you take a 99-way UltraSPARC III machine and add a 100th processor, 
> you get 94 percent linear scalability. You can't get 94 percent linear 
> scalability on your first Intel chip. It's very, very hard to do, and they 
> have not done it."

  Conditionally...  I would like to know the exact architecture,
and the problem set running in the system to say.

When you have noncc-NUMA, you have a Beowulf-like setup.
when you have cc-NUMA ("cc" = cache coherent), things get
truly hairy...

Re: Does Solaris really scale this well?

[Ruth Ivimey-Cook]

On Sat, 17 Aug 2002, Matti Aarnio wrote:

>On Sat, Aug 17, 2002 at 11:53:16AM -0600, Dax Kelson wrote:
>> From:
>>   http://www.itworld.com/Man/3828/020816mcnealy/
>> 
>> Scott McNealy:
>> 
>> "When you take a 99-way UltraSPARC III machine and add a 100th processor, 
>> you get 94 percent linear scalability. You can't get 94 percent linear 
>> scalability on your first Intel chip. It's very, very hard to do, and they 
>> have not done it."
>
>  Conditionally...  I would like to know the exact architecture,
>and the problem set running in the system to say.
>
>When you have noncc-NUMA, you have a Beowulf-like setup.
>when you have cc-NUMA ("cc" = cache coherent), things get
>truly hairy...

I've seen scientific reports of scalability that good in non-shared memory
computers (mostly in transputer arrays) where (with a scalable algorithm)
unless you got >90% you were doing something wrong.  However, if you insist on
sharing main memory, I still don't believe you can get anywhere near that...
IMO 30% is doing very well once past the first few CPUs.

Regards,

Ruth

-- 
Ruth Ivimey-Cook
Software engineer and technical writer.

Re: Does Solaris really scale this well?

[Larry McVoy]

On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
> >> "When you take a 99-way UltraSPARC III machine and add a 100th processor, 
> >> you get 94 percent linear scalability. You can't get 94 percent linear 
> >> scalability on your first Intel chip. It's very, very hard to do, and they 
> >> have not done it."
> 
> I've seen scientific reports of scalability that good in non-shared memory
> computers (mostly in transputer arrays) where (with a scalable algorithm)
> unless you got >90% you were doing something wrong.  However, if you insist on
> sharing main memory, I still don't believe you can get anywhere near that...
> IMO 30% is doing very well once past the first few CPUs.

Please reconsider your opinion.  Both Sun and SGI scale past 100 CPUs on
reasonable workloads in shared memory.  Where "reasonable" != easy to do.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: Does Solaris really scale this well?

[David S. Miller]

   From: Larry McVoy <lm@bitmover.com>
   Date: Sat, 17 Aug 2002 17:55:17 -0700

   On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
   > >> "When you take a 99-way UltraSPARC III machine and add a 100th processor, 
   > >> you get 94 percent linear scalability. You can't get 94 percent linear 
   > >> scalability on your first Intel chip. It's very, very hard to do, and they 
   > >> have not done it."
   > 
   > I've seen scientific reports of scalability that good in non-shared memory
   > computers (mostly in transputer arrays) where (with a scalable algorithm)
   > unless you got >90% you were doing something wrong.  However, if you insist on
   > sharing main memory, I still don't believe you can get anywhere near that...
   > IMO 30% is doing very well once past the first few CPUs.
   
   Please reconsider your opinion.  Both Sun and SGI scale past 100 CPUs on
   reasonable workloads in shared memory.  Where "reasonable" != easy to do.

Also consider that if you start having performed so badly in the
uniprocessor case like Solaris does, it doesn't take so much effort to
get good scalability percentages as you add cpus because there isn't
much to scale. :-)

To Sun's credit, they have on their side the fact that in the x86
world there still has never has been a very good large scale SMP
backplane as of yet.  At least not on the order of what you'd find
on one of Sun's big boxes.

But in the same breath this is what will kill Sun in the end.  Over
time, the commodity stuff inches closer and closer to what Sun's "high
end" is.

Re: Does Solaris really scale this well?

[venom]

On Sat, 17 Aug 2002, Larry McVoy wrote:

> Date: Sat, 17 Aug 2002 17:55:17 -0700
> From: Larry McVoy <lm@bitmover.com>
> To: Ruth Ivimey-Cook <Ruth.Ivimey-Cook@ivimey.org>
> Cc: Matti Aarnio <matti.aarnio@zmailer.org>, Dax Kelson <dax@gurulabs.com>,
>      "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
> Subject: Re: Does Solaris really scale this well?
>
> On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
> > >> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> > >> you get 94 percent linear scalability. You can't get 94 percent linear
> > >> scalability on your first Intel chip. It's very, very hard to do, and they
> > >> have not done it."
> >
> > I've seen scientific reports of scalability that good in non-shared memory
> > computers (mostly in transputer arrays) where (with a scalable algorithm)
> > unless you got >90% you were doing something wrong.  However, if you insist on
> > sharing main memory, I still don't believe you can get anywhere near that...
> > IMO 30% is doing very well once past the first few CPUs.
>
> Please reconsider your opinion.  Both Sun and SGI scale past 100 CPUs on
> reasonable workloads in shared memory.  Where "reasonable" != easy to do.


And where reasonable != 94%. Seriously, 94% scalability could be on a
8 CPUs 880, but, for example, I have a 64 CPUS domain on a E10k which is
far from 94% scalability (ok, an old E10k with an 83Mhz bus).
For what I saw, maybe SGI Origin 3000 is scaling
a little better with a lot of CPUS, but I also never had an E15000 around
for now...

Re: Does Solaris really scale this well?

[Ruth Ivimey-Cook]

On Sat, 17 Aug 2002, Larry McVoy wrote:

>On Sun, Aug 18, 2002 at 12:03:24AM +0100, Ruth Ivimey-Cook wrote:
>> >> "When you take a 99-way UltraSPARC III machine and add a 100th processor, 
>> >> you get 94 percent linear scalability. You can't get 94 percent linear 
>> >> scalability on your first Intel chip. It's very, very hard to do, and they 
>> >> have not done it."
>> 
>> I've seen scientific reports of scalability that good in non-shared memory
>> computers (mostly in transputer arrays) where (with a scalable algorithm)
>> unless you got >90% you were doing something wrong.  However, if you insist on
>> sharing main memory, I still don't believe you can get anywhere near that...
>> IMO 30% is doing very well once past the first few CPUs.
>
>Please reconsider your opinion.  Both Sun and SGI scale past 100 CPUs on
>reasonable workloads in shared memory.  Where "reasonable" != easy to do.

Larry,

I wasn't disputing that Sun could have say 100 cpus in a box, but that the
100th shared-memory CPU didias much work as the first. That said, I _am_ out
of date on the performance of the Sun machines; what kind of measured
performance effeciency do you get with them?

A google search turned up:
    http://www.icg.tu-graz.ac.at/goller/publication/pers/node6.html (1997)
in which the author says:

  Utilizing more than half of all processors is commonly agreed to be an
acceptable efficiency for parallel applications. In all three diagrams, the
efficiency never drops below this 50%-level in the area of interest

-- well, I would disagree about the implied "any parallel app", but it does
seem to be true of many SMP systems...

In the following paper, the authors benchmarked IBM, Cray and SGI
supercomputers: http://citeseer.nj.nec.com/kang99benchmarking.html (1999)

If you look at pages numbered 53 & 54, you will see graphs of time vs num
processors that are a very long way from linear, indeed in one case time
increased as cum cpus increased.

It is also instructive to note that in many cases, the peak processor power is 
obtained by multiplying individual peak power by the number of processors, 
with no notice taken of the costs of synchronisation, memory access or 
communication. Consequently, many new owners of supercomputers are very 
disappointed with their new 'baby' when they find it's not nearly as powerful 
as they had been told.

Regards,

Ruth

-- 
Ruth Ivimey-Cook
Software engineer and technical writer.

Re: Does Solaris really scale this well?

[venom]

On Sat, 17 Aug 2002, Dax Kelson wrote:

> Date: Sat, 17 Aug 2002 11:53:16 -0600 (MDT)
> From: Dax Kelson <dax@gurulabs.com>
> To: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
> Subject: Does Solaris really scale this well?
>
>
> From:
>
> http://www.itworld.com/Man/3828/020816mcnealy/
>
> Scott McNealy:
>
> "When you take a 99-way UltraSPARC III machine and add a 100th processor,
> you get 94 percent linear scalability. You can't get 94 percent linear
> scalability on your first Intel chip. It's very, very hard to do, and they
> have not done it."

you can't get 94% linear scalability also on Sparc III, to say the
truth...