SMP question

SMP question

[Artemio]

Hello!

I have the following question.

My system is 2x 2.4Ghz Xeons.

Linux kernel 2.4.18 compiled with SMP support sees it as four processors. 
SMP-disabled kernel sees one, of course.

I would like to know, how will it influence the system performance, if I run a 
UP kernel?

What does the kernel SMP support add? Just some API for additinal 
multiprocessor control? Will the SMP-enabled kernel run faster than the UP 
one?

That's what I would like to know.



Thanks!


Artemio.

Re: SMP question

[Tigran Aivazian]

Hello Artemio,

SMP kernel doesn't run "faster" than a UP one, as such, but enabling SMP
support will allow you to make use of your 3 extra CPUs, e.g. by multiple
user applications at the same time, or by various tasks performed inside
the kernel. So, if you have more than one CPU definitely enable SMP
support, unless this causes your system to hang (in which case it's a bug
and you are expected to send a bugfix to this list)

Regards
Tigran

On Wed, 11 Jun 2003, Artemio wrote:

> Hello!
> 
> I have the following question.
> 
> My system is 2x 2.4Ghz Xeons.
> 
> Linux kernel 2.4.18 compiled with SMP support sees it as four processors. 
> SMP-disabled kernel sees one, of course.
> 
> I would like to know, how will it influence the system performance, if I run a 
> UP kernel?
> 
> What does the kernel SMP support add? Just some API for additinal 
> multiprocessor control? Will the SMP-enabled kernel run faster than the UP 
> one?
> 
> That's what I would like to know.
> 
> 
> 
> Thanks!
> 
> 
> Artemio.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: SMP question

[Samuel Flory]

Artemio wrote:

>Hello!
>
>I have the following question.
>
>My system is 2x 2.4Ghz Xeons.
>
>Linux kernel 2.4.18 compiled with SMP support sees it as four processors. 
>
  This is hyper threading.  This is intel's attempt to cram extra 
instructions on each cpu cyle.  It may make things faster, or slower.  
It really depends on what you are doing.  You can shut off hyper 
threading off in the bios.

>SMP-disabled kernel sees one, of course.
>
>I would like to know, how will it influence the system performance, if I run a 
>UP kernel?
>
  Well if you are generally only running a single program then it will 
speed things up.  If you run a number of programs all at once it will 
speed things up.  Hyperthreading tends to be a good thing if you are 
running 8 or more cpu hungry processes.

>
>What does the kernel SMP support add? Just some API for additinal 
>multiprocessor control? 
>
  For the most part.  It also enables io-apic and other stuff.


-- 
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@rackable.com>

Re: SMP question

[Artemio]

OK guys, thanks for all your replies.

What I have is the following.

I'm building a hard real-time Linux (RTLinux) system on a 2x Xeon machine. If 
I compile and run a 2.4.18 kernel with SMP support, rtlinux hangs the 
machine. However, with SMP disabled, rtlinux and all it's hard-realtime 
applications runs okay.

So, I have to deside between these two:

 - Run rtlinux and hard-realtime applications on a kernel without SMP support. 
How much performance will I loose this way? Is SMP *THAT* critical? 

 - Run all tasks in a usual way, no hard realtime, but with SMP support.


What would you suggest?


Also, if I turn hyperthreading off, how will it influence the system with SMP 
support? Without SMP support?



Thanks!


Artemio.

RE: SMP question

[David Schwartz]

> I'm building a hard real-time Linux (RTLinux) system on a 2x Xeon
> machine. If
> I compile and run a 2.4.18 kernel with SMP support, rtlinux hangs the
> machine. However, with SMP disabled, rtlinux and all it's hard-realtime
> applications runs okay.

> So, I have to deside between these two:

>  - Run rtlinux and hard-realtime applications on a kernel without
> SMP support.
> How much performance will I loose this way? Is SMP *THAT* critical?

	You will lose about half your CPU power.

>  - Run all tasks in a usual way, no hard realtime, but with SMP support.

> What would you suggest?

	If you don't install a kernel with SMP support, you might as well remove
one processor.

> Also, if I turn hyperthreading off, how will it influence the
> system with SMP
> support? Without SMP support?

	In a system with more than one physical CPU, hyperthreading is not that big
of a performance boost.

	DS

Re: SMP question

[Artemio]

Hello!

> > How much performance will I loose this way? Is SMP *THAT* critical?
>
> 	You will lose about half your CPU power.

Hmmm... So, you mean uni-processor Linux kernel can't see two processors as 
one "big" processor? 

> >  - Run all tasks in a usual way, no hard realtime, but with SMP support.
> >
> > What would you suggest?
>
> 	If you don't install a kernel with SMP support, you might as well remove
> one processor.

:-)

> > Also, if I turn hyperthreading off, how will it influence the
> > system with SMP
> > support? Without SMP support?
>
> 	In a system with more than one physical CPU, hyperthreading is not that
> big of a performance boost.

Okay, I will try turning hyperthreding off and see if RTLinux keeps hanging 
the machine.


Thanks for your reply!

Artemio.

RE: SMP question

[David Schwartz]

> Hello!

> > > How much performance will I loose this way? Is SMP *THAT* critical?
> >
> > 	You will lose about half your CPU power.

> Hmmm... So, you mean uni-processor Linux kernel can't see two
> processors as
> one "big" processor?

	Don't we wish. No, two processors is two processors.

> > > Also, if I turn hyperthreading off, how will it influence the
> > > system with SMP
> > > support? Without SMP support?

> > 	In a system with more than one physical CPU, hyperthreading
> > is not that
> > big of a performance boost.

> Okay, I will try turning hyperthreding off and see if RTLinux
> keeps hanging
> the machine.

	It sounds like you're experiencing a bug. You can do this kind of testing
to help determine the 'envelope' of the bug (that is, under what
circumstances it appears and under what circumstances it doesn't appear),
however this is not a substitute for fixing the bug. ;)

	Even if you find a configuration that doesn't show the bug, you still
should work with the RTLinux guys to track down the bug and get it fixed.
If, for example, it's due to unreliable hardware, turning off hyperthreading
might hide it but it will still be lurking there.

	RTLinux might be hanging because of problems with the code you're running.
Thinks like deadlock and priority inversion can become much more obvious in
an SMP machine, but can definitely still happen in a UP machine, just much
less often.

	DS

Re: SMP question

[J.A. Magallon]

On 06.11, Artemio wrote:
> Hello!
> 
> > > How much performance will I loose this way? Is SMP *THAT* critical?
> >
> > 	You will lose about half your CPU power.
> 
> Hmmm... So, you mean uni-processor Linux kernel can't see two processors as 
> one "big" processor? 
> 

No, but it will work as if it does...explain below.

You have 2 processor packages, each one is HyperThreading capable. This
means you have two 'CPUs' inside each package, so that sums up your 4 CPUs.
But there is a flaw. The 2 'CPUs' inside each processor package are not
full real CPUs, just two register sets that share cache, FP units, integer
units and so on. So let's say your Xeon has 8 FP units, and you want to
run a FPU intensive task with low or null disk IO. If you activate
hyperthreading each of the 2 'cpus' has 4 FP units, so half the computation
power. If you deactivate HT, you have 1 CPU with 8 FP units.

In short, for FP intensive tasks, hyperthreading is a big lie...
You can't run 2 computations in parallel.

-- 
J.A. Magallon <jamagallon@able.es>      \                 Software is like sex:
werewolf.able.es                         \           It's better when it's free
Mandrake Linux release 9.2 (Cooker) for i586
Linux 2.4.21-rc7-jam1 (gcc 3.3 (Mandrake Linux 9.2 3.3-1mdk))

Re: SMP question

[Artemio]

Hello!

Thanks for your reply!

> > Hmmm... So, you mean uni-processor Linux kernel can't see two processors
> > as one "big" processor?
>
> No, but it will work as if it does...explain below.
>
> You have 2 processor packages, each one is HyperThreading capable. This
> means you have two 'CPUs' inside each package, so that sums up your 4 CPUs.
> But there is a flaw. The 2 'CPUs' inside each processor package are not
> full real CPUs, just two register sets that share cache, FP units, integer
> units and so on. So let's say your Xeon has 8 FP units, and you want to
> run a FPU intensive task with low or null disk IO. If you activate
> hyperthreading each of the 2 'cpus' has 4 FP units, so half the computation
> power. If you deactivate HT, you have 1 CPU with 8 FP units.
>
> In short, for FP intensive tasks, hyperthreading is a big lie...
> You can't run 2 computations in parallel.

Thanks for such in-depth explanation! 

As I understood, with HT enabled, Linux-SMP sees four CPUs with 5000 bogo mips 
each (of course I've already seen this in /proc/cpuinfo).

So, if I deactivate HT, will a UP Linux see one CPU with 4x5000=20000 bogo 
mips?

Anyway, I will try what I mentioned on that machine today.


Thank you again and good luck!


Artemio.

Re: SMP question

[Matt Reppert]

On Thu, 12 Jun 2003 08:37:40 +0300
Artemio <artemio@artemio.net> wrote:

> As I understood, with HT enabled, Linux-SMP sees four CPUs with 5000 bogo mips 
> each (of course I've already seen this in /proc/cpuinfo).
> 
> So, if I deactivate HT, will a UP Linux see one CPU with 4x5000=20000 bogo 
> mips?

No. It will see one CPU with 5000 BogoMips.

Matt

Re: SMP question

[Artemio]

> > As I understood, with HT enabled, Linux-SMP sees four CPUs with 5000 bogo
> > mips each (of course I've already seen this in /proc/cpuinfo).
> >
> > So, if I deactivate HT, will a UP Linux see one CPU with 4x5000=20000
> > bogo mips?
>
> No. It will see one CPU with 5000 BogoMips.

Thanks, I've already checked this out myself.

We have decided not to use rtlinux on SMP - it seems to be some bug in rtlinux 
- but that's not for discussion here.

Thank you all for your replies.

Good luck!

Artemio.

Re: SMP question

[Scott Robert Ladd]

J.A. Magallon wrote:
> In short, for FP intensive tasks, hyperthreading is a big lie...
> You can't run 2 computations in parallel.

Yes and no; the benefit of HT depends on the application in question. 
I've seen everything from a 5% LOSS in performance to a 30% INCREASE in 
performance, for intensive floating-point code. This is with programs 
parallelized with OpenMP and Intel's C and Fortran compilers. I'm still 
analyzing the exact nature of the benefits, but they *do* exist.

While I much prefer multiple physical CPUs to "virtual" CPUs, HT *does* 
provide performance improvements for certain applications. To call HT a 
"big lie" is both provacative and inaccurate.

-- 
Scott Robert Ladd
Coyote Gulch Productions (http://www.coyotegulch.com)

Re: SMP question

[J.A. Magallon]

On 06.12, Scott Robert Ladd wrote:
> J.A. Magallon wrote:
> > In short, for FP intensive tasks, hyperthreading is a big lie...
> > You can't run 2 computations in parallel.
> 
> Yes and no; the benefit of HT depends on the application in question. 
> I've seen everything from a 5% LOSS in performance to a 30% INCREASE in 
> performance, for intensive floating-point code. This is with programs 
> parallelized with OpenMP and Intel's C and Fortran compilers. I'm still 
> analyzing the exact nature of the benefits, but they *do* exist.
> 
> While I much prefer multiple physical CPUs to "virtual" CPUs, HT *does* 
> provide performance improvements for certain applications. To call HT a 
> "big lie" is both provacative and inaccurate.
> 

I told that because non-techical people just think they are buying _two_
processors, but they really pay too much for 1.25 processors, and that
if you fine-tune your code...

-- 
J.A. Magallon <jamagallon@able.es>      \                 Software is like sex:
werewolf.able.es                         \           It's better when it's free
Mandrake Linux release 9.2 (Cooker) for i586
Linux 2.4.21-rc8-jam1 (gcc 3.3 (Mandrake Linux 9.2 3.3-1mdk))

Re: SMP question

[Zwane Mwaikambo]

On Wed, 11 Jun 2003, Artemio wrote:

> I'm building a hard real-time Linux (RTLinux) system on a 2x Xeon machine. If 
> I compile and run a 2.4.18 kernel with SMP support, rtlinux hangs the 
> machine. However, with SMP disabled, rtlinux and all it's hard-realtime 
> applications runs okay.

Sounds like an RTLinux bug, perhaps you should elaborate on that on the 
RTLinux mailing list.

> So, I have to deside between these two:
> 
>  - Run rtlinux and hard-realtime applications on a kernel without SMP support. 
> How much performance will I loose this way? Is SMP *THAT* critical? 

Depending on your load it could make a very significant difference.

>  - Run all tasks in a usual way, no hard realtime, but with SMP support.

If you have that option you didn't need hard realtime in the first place.

> Also, if I turn hyperthreading off, how will it influence the system with SMP 
> support? Without SMP support?

HT w/ SMP = You'll have to do your own tests with your applications
HT w/o SMP = Normal UP

	Zwane
-- 
function.linuxpower.ca

Re: SMP question

[Artemio]

Hello!

> > I'm building a hard real-time Linux (RTLinux) system on a 2x Xeon
> > machine. If
>
> then LKML is not the place for you.  RTLinux is a fork of "real" linux.

Well, those guys seem to have success on the same machines that I fail. 

I even used *their* .config file to compile a kernel, but I still fail.

> well, talk to the rtlinux people.  considering what rtlinux changes from
> real linux, this hang is not a big surprise.

Yes, rtlinux runs linux kernel as it's idle thread, and it messes up the 
system - I mean it becomes more difficult to make it work if it doesn't work.

> > So, I have to deside between these two:
> >
> >  - Run rtlinux and hard-realtime applications on a kernel without SMP
> > support. How much performance will I loose this way? Is SMP *THAT*
> > critical?
>
> huh?  it's not critical at all.  though if you have two processors,
> you've just wasted several hundred dollars unless you run SMP.

:-)

> >  - Run all tasks in a usual way, no hard realtime, but with SMP support.
>
> why do you think you need rtlinux-type realtime?

Hmm... Nice question.

We are to run many applications that will be mission-critical. That's why we 
decided to make them in hard-realtime.

The other way is probably making low-priority kernel threads by usual means.

But I think hard-realtime will give us more performance. 

> > Also, if I turn hyperthreading off, how will it influence the system with
> > SMP support? Without SMP support?
>
> HT is just hardware-supported multithreading - imagine that you have
> the same old processor, except that the CPU's instruction-dispatcher
> is reading from two instruction streams.  HT doesn't add any new resources,
> just shares a single CPU among two threads.

Got it.

> I suspect the design of rtlinux is inherently incompatible with HT.

I will try turning HT off and running RTLinux.



Thank you very much for your reply!


Artemio.