Kernel Compilation for Dual Xeon system

Kernel Compilation for Dual Xeon system

[Simon Oliver]

I am about to compile an optimized Kernel for my new servers.  I'm used to
compiling kernels but not for this specification of machine - these
servers are all Dual P4-Xeon 2.2GHz (512Kb cache) using Intel E7500
(Plumas) chipsets, ECC DDR RAM..  

- which processor type to select - I assume Pentium-4 since no Xeon
option.

- how much High Memory Support - some of these servers have 2GB, some 1GB
RAM.  I don't foresee expanding the RAM beyond 4GB in the future.

- what to set for User address space size - why?

Finally.  Some of these servers will be high-throughput number crunchers
and I have been advised by the software vendor to switch off
hyper-threading in the BIOS (which I have done) as their tests indicate it
degrades compute intensive tasks.  But one server will be a general
purpose machine and from Intel's pages it sounds like hyper-threading
would be good for this.  Anyone have experience with this?  Are there any
Linux specific optimizations?

TIA

-- 
  Simon Oliver

Re: Kernel Compilation for Dual Xeon system

[E. Robert Bogusta]

On Mon, 1 Jul 2002, Simon Oliver wrote:

> - which processor type to select - I assume Pentium-4 since no Xeon
> option.

Right.
 
> - how much High Memory Support - some of these servers have 2GB, some 1GB
> RAM.  I don't foresee expanding the RAM beyond 4GB in the future.

You can either cut to the memory you have now and recompile later, or give
up a few k of table space now and not have to remember later.
 
> - what to set for User address space size - why?

Unless you really need big user space, go small. An issue of table space
again, although it isn't a big one.
 
> Finally.  Some of these servers will be high-throughput number crunchers
> and I have been advised by the software vendor to switch off
> hyper-threading in the BIOS (which I have done) as their tests indicate it
> degrades compute intensive tasks.

You may want to try that and draw your own conclusion. It depends a lot on
how many processes you run and how active their data set might be. I would
assume that the vendor is right, but I would be curious enough to check
just in case.

>                                      But one server will be a general
> purpose machine and from Intel's pages it sounds like hyper-threading
> would be good for this.  Anyone have experience with this?  Are there any
> Linux specific optimizations?

This is another one you try yourself, but I think your idea is correct,
hyperthreading may help on a machine with lots of processes. You could see
5-30% better performance, although that's very hard to measure, since many
benchmarks would change the general purpose nature of the load. I'd try a
kernel compile with -jN, with N=number of processors, physical or logical.
You might run a tad faster adding 1 to that N, making it larger than that
is usually worse.

-- 
	-rob bogus

Re: Kernel Compilation for Dual Xeon system

[Simon Oliver]

Rob, 

thanks for the quick response

"E. Robert Bogusta" wrote:
> 
> > - what to set for User address space size - why?
> 
> Unless you really need big user space, go small. An issue of table space
> again, although it isn't a big one.
What is big user space?  I have one application that will use about 1GB
RAM, another (on another server) uses between 40MB and 500MB.

What is are the pros and cons of setting a large/small user space.

For example if I have a 1GB user space defined but a process needs to
address 1.5GB?  

Or conversly if I have configured for 1GB user space but never use more
than about 250MB user space.

Finally.  What will have happen if I compile the kernel with support for
4GB RAM (even though I only have 2GB) then set the user space to 2/2GB?

> You may want to try that and draw your own conclusion. It depends a lot on
> how many processes you run and how active their data set might be. I would
> assume that the vendor is right, but I would be curious enough to check
> just in case.
This is a memory mapped daemon application - it loads an entire database
(~1GB) into RAM and waits for jobs.

> I'd try a
> kernel compile with -jN, with N=number of processors, physical or logical.
> You might run a tad faster adding 1 to that N, making it larger than that
> is usually worse.
I've seen this advice in the smp readme with the kernel source.  I thought
that this only affected kernel compilation speed - I didn't realise it
affected overall system performance - how does it do that?

Finally, and good resources on this - I had a look around but couldn't
find much up-to-date stuff about?

Thanks again

--
  Simon Oliver

Re: Kernel Compilation for Dual Xeon system

[E. Robert Bogusta]

On Tue, 2 Jul 2002, Simon Oliver wrote:

> > > - what to set for User address space size - why?
> > 
> > Unless you really need big user space, go small. An issue of table space
> > again, although it isn't a big one.
> What is big user space?  I have one application that will use about 1GB
> RAM, another (on another server) uses between 40MB and 500MB.
> 
> What is are the pros and cons of setting a large/small user space.

The only con is using the 3.5GB patch, at least that I've seen. This
reduces address space to the kernel and can cause some overhead under some
loads. Other than that I just use the 3/1 config. Perhaps someone will
comment on what real case would make the slightest difference?
 > Finally.  What will have happen if I compile the kernel with support for
> 4GB RAM (even though I only have 2GB) then set the user space to 2/2GB?

Don't confuse physical memory support and user address space, you need
support for 4GB if you have more than 1GB you want to use, user space just
isn't an issue with a stock kernel.
 
> > I'd try a
> > kernel compile with -jN, with N=number of processors, physical or logical.
> > You might run a tad faster adding 1 to that N, making it larger than that
> > is usually worse.
> I've seen this advice in the smp readme with the kernel source.  I thought
> that this only affected kernel compilation speed - I didn't realise it
> affected overall system performance - how does it do that?

It does affect only the compilation, I thought that was one of the things
you wanted to know.

-- 
	-rob bogus

Re: Kernel Compilation for Dual Xeon system

[Simon Oliver]

Rob, thanks again.

Here's what the Kernel help system says on selecting "User address space
size":

	Typically there will 128 megabytes less "user memory" mapped    
	than the number in the configuration option. Saying that        
	another way, "high memory" will usually start 128 megabytes     
	lower than the configuration option.                            

	Selecting "05GB" results in a "3.5GB/0.5GB" kernel/user split:  
	On a system with 1 gigabyte of physical memory, you may get 384 
	megabytes of "user memory" and 640 megabytes of "high memory"   
	with this selection.

Why would I want that?  
What is the advantage of having a high ratio of "high memory"?
What is this "high memory" used for, the kernel and drivers?
How do I gauge how much high memory I need?

	Selecting "1GB" results in a "3GB/1GB" kernel/user split:       
	On a system with 1 gigabyte of memory, you may get 896 MB of    
	"user memory" and 128 megabytes of "high memory" with this
	selection. This is the usual setting.                     

And on a 2GB system would I get 1152 MB high and 896MB user?

	Selecting "2GB" results in a "2GB/2GB" kernel/user split:       
	On a system with less than 1.75 gigabytes of physical memory,   
	this option will make it so no memory is mapped as "high".

Does that mean I get 128/1920 with 2GB of physical RAM?
And 0/1024 with 1GB of physical RAM?
And what are the cons of having no high memory?

To summarize, how much does all this affect performance and in what way?

--
  Simon Oliver