10 GB in Opteron machine

10 GB in Opteron machine

[Christoph Pleger]

Hello,

I had a working kernel configuration for an Opteron machine. Since that
configuration was supposed to support many kinds of hardware, it
contained many settings that were not optimal for an Opteron machine. So
I created a new configuration especially for that machine. But the
resulting kernel could not be booted. To find the problem I took the
working configuration and changed and it in many small steps and after
every change compiled the kernel, installed it and rebooted to see if
the kernel still boots. 

At last I found out that setting HIGHMEM support to 64 GB is the
problem. But is it really not possible to use more than 4GB on an
Opteron machine?

I have set the processor type to Opteron and disabled SMP support. I am
using Kernel 2.6.11.12.

Christoph 

Re: 10 GB in Opteron machine

[Jeff Garzik]

Christoph Pleger wrote:
> At last I found out that setting HIGHMEM support to 64 GB is the
> problem. But is it really not possible to use more than 4GB on an
> Opteron machine?

Build and boot a 64-bit kernel, not a 32-bit kernel.

There is no highmem option for the 64-bit kernel, because it doesn't 
need one.

	Jeff

Re: 10 GB in Opteron machine

[Christoph Pleger]

Hello,

On Fri, 22 Jul 2005 05:05:40 -0400
Jeff Garzik <jgarzik@pobox.com> wrote:

> Christoph Pleger wrote:
> > At last I found out that setting HIGHMEM support to 64 GB is the
> > problem. But is it really not possible to use more than 4GB on an
> > Opteron machine?
> 
> Build and boot a 64-bit kernel, not a 32-bit kernel.
> 
> There is no highmem option for the 64-bit kernel, because it doesn't 
> need one.

I have two questions:

1. Is it possible to compile a 64-bit kernel on a 32-bit machine (or at
least on a 64-bit machine with 32-bit software) and if yes, how can I do
that?
2. All other software on the machine is 32-bit software. Will that
software work with a 64-bit kernel?

Christoph

Re: 10 GB in Opteron machine

[Bernd Petrovitsch]

On Fri, 2005-07-22 at 11:31 +0200, Christoph Pleger wrote:
[...]
> 2. All other software on the machine is 32-bit software. Will that
> software work with a 64-bit kernel?

Basically yes.
E.g. open-office does not exist natively for 64bit architectures ATM (at
least not on x86-compatibles).

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: 10 GB in Opteron machine

[Bernd Petrovitsch]

On Fri, 2005-07-22 at 11:31 +0200, Christoph Pleger wrote:
[...] 
> 2. All other software on the machine is 32-bit software. Will that
> software work with a 64-bit kernel?

Basically yes.
E.g. open-office does not exist natively for 64bit architectuires ATM.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: 10 GB in Opteron machine

[Jakob Oestergaard]

[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]

On Fri, Jul 22, 2005 at 11:31:38AM +0200, Christoph Pleger wrote:
> Hello,
...
> > There is no highmem option for the 64-bit kernel, because it doesn't 
> > need one.
> 
> I have two questions:
> 
> 1. Is it possible to compile a 64-bit kernel on a 32-bit machine (or at
> least on a 64-bit machine with 32-bit software) and if yes, how can I do
> that?

Yes. On Debian Sarge, I have a few wrapper scripts to accomplish it -
all attached to this mail - just untar them in /usr/local/bin on a
standard x86 32-bit Sarge distro.  Use 'kmake' instead of 'make' when
you are working with your kernel source (eg. 'kmake menuconfig', 'kmake
all')

Sarge comes with all the necessary toolchain support to build a 64-bit
kernel.

It should be equally possible on most other distros of course, I just
haven't felt the urge to go waste my time with them :)

> 2. All other software on the machine is 32-bit software. Will that
> software work with a 64-bit kernel?

Yes. You tell your 64-bit kernel to enable 'IA32 Emulation' (under
Executable file formats / Emulations).

This is really the clever way to run a 64-bit system - 99% of what is
commonly run on most systems only gains overhead from the 64-bit address
space - tools like postfix, cron, syslog, apache, ... will not gain from
being native 64-bit.

The kernel however will gain from being 64-bit - and it will easily run
your existing 32-bit apps.

Solaris has done this for ages - maintaining a mostly 32-bit user space,
a 64-bit kernel, and then allowing for certain memory intensive
applications to run natively 64-bit.

It's a nice way to run a Linux based system too, IMO.

-- 

 / jakob


[-- Attachment #2: kmake.tar --]
[-- Type: application/x-tar, Size: 10240 bytes --]

Re: 10 GB in Opteron machine

[Christoph Pleger]

Hello,

On Fri, 22 Jul 2005 12:39:55 +0200
Jakob Oestergaard <jakob@unthought.net> wrote:

> > 1. Is it possible to compile a 64-bit kernel on a 32-bit machine (or
> > at least on a 64-bit machine with 32-bit software) and if yes, how
> > can I do that?
> 
> Yes. On Debian Sarge, I have a few wrapper scripts to accomplish it -
> all attached to this mail - just untar them in /usr/local/bin on a
> standard x86 32-bit Sarge distro.  Use 'kmake' instead of 'make' when
> you are working with your kernel source (eg. 'kmake menuconfig',
> 'kmake all')
> 
> Sarge comes with all the necessary toolchain support to build a 64-bit
> kernel.
> 
> It should be equally possible on most other distros of course, I just
> haven't felt the urge to go waste my time with them :)

I am also using Debian sarge. I extracted the tarfile to /usr/local/bin
end executed "kmake menuconfig". Everything seemed fine so far. But a
few seconds after starting the compilation (kmake bzImage) I got this
error message:

In file included from <snip>
...
<snip>
include/asm/mpspec.h:6:25: mach_mpspec.h: No such file or directory



Hm. I understand why that file cannot be found: It only exists in the
asm-i386 directory. But why does the compilation process look for a file
that belongs to i386, but not to x86_64?

Christoph  

Re: 10 GB in Opteron machine

[Stephen Frost]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

* Jakob Oestergaard (jakob@unthought.net) wrote:
> This is really the clever way to run a 64-bit system - 99% of what is
> commonly run on most systems only gains overhead from the 64-bit address
> space - tools like postfix, cron, syslog, apache, ... will not gain from
> being native 64-bit.

For most 64-bit systems, sure.  For amd64 it's a little different
because there are additional changes to the architecture (as compared to
ia32/x86) which can more than make up for the difference for many
applications.  Then there's also things like encryption (postfix/tls,
apache/ssl, etc) which can benefit greatly from better handling of
64bit (and larger) types.

So, basically, it's not nearly so clear-cut as you portray it. :)

> Solaris has done this for ages - maintaining a mostly 32-bit user space,
> a 64-bit kernel, and then allowing for certain memory intensive
> applications to run natively 64-bit.

The differences between a 64bit sparc chip in 32bit and 64bit are quite
a bit less than the differences between an amd64 chip in 32bit and
64bit.  Thus, this makes alot more sense for sparc.

> It's a nice way to run a Linux based system too, IMO.

Perhaps on sparc or mips; it's much less clear-cut on amd64.

	Stephen

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 10 GB in Opteron machine

[Jakob Oestergaard]

On Fri, Jul 22, 2005 at 01:37:46PM +0200, Christoph Pleger wrote:
> Hello,
> 
...
> I am also using Debian sarge. I extracted the tarfile to /usr/local/bin
> end executed "kmake menuconfig". Everything seemed fine so far. But a
> few seconds after starting the compilation (kmake bzImage) I got this
> error message:
> 
> In file included from <snip>
> ...
> <snip>
> include/asm/mpspec.h:6:25: mach_mpspec.h: No such file or directory

Try a plain 2.6.11.11

> Hm. I understand why that file cannot be found: It only exists in the
> asm-i386 directory. But why does the compilation process look for a file
> that belongs to i386, but not to x86_64?

Kernel source screwed up?  :)

-- 

 / jakob

Re: 10 GB in Opteron machine

[Christoph Pleger]

Hello,

I now could compile the amd64-kernel successfully. I installed it on my
machine, rebooted and in the beginning everything seemed fine. But after
mounting the root (ext 3) filesystem (or before mounting, I do not know
exactly) the machine hangs. The last message I see is:

Mounting root filesystem, starting kjournald.


What can I do now?

Christoph