[RFC] A simple way to determine if the kernel needs HIGHMEM64G to be able to use all the installed memory

[RFC] A simple way to determine if the kernel needs HIGHMEM64G to be able to use all the installed memory

[Ozan Çağlayan]

Hi,

When the installed memory size is >= 4GB, kernel drops some messages like

Warning only 4GB of memory will be used
You have to enable HIGHMEM64G.

I checked that the message comes from arch/x86/init_32.c after checking
max_pfn.

I'm quite dumb about the internal structures of the kernel but, wouldn't it
be possible to create a simple read-only sysfs object like kexec_loaded
that will contain "1" when the kernel needs HIGHMEM64G to see all of the memory
and "0" when it doesn't?

I think that it would be a nice facility for distribution kernels to detect
the need for a PAE enabled 32-bit kernel by just reading some /sys/.. entry.

Thanks,

-- 

Ozan Çağlayan
<ozan_at_pardus.org.tr>

Re: [RFC] A simple way to determine if the kernel needs HIGHMEM64G to be able to use all the installed memory

[Alexey Dobriyan]

On Sat, Mar 07, 2009 at 01:04:38PM +0200, Ozan Çağlayan wrote:
> When the installed memory size is >= 4GB, kernel drops some messages like
> 
> Warning only 4GB of memory will be used
> You have to enable HIGHMEM64G.
> 
> I checked that the message comes from arch/x86/init_32.c after checking
> max_pfn.
> 
> I'm quite dumb about the internal structures of the kernel but, wouldn't it
> be possible to create a simple read-only sysfs object like kexec_loaded
> that will contain "1" when the kernel needs HIGHMEM64G to see all of the memory
> and "0" when it doesn't?

Why do you need a file? If kernel says to enable HIGHMEM64G, enable it.

> I think that it would be a nice facility for distribution kernels to detect
> the need for a PAE enabled 32-bit kernel by just reading some /sys/.. entry.

Re: [RFC] A simple way to determine if the kernel needs HIGHMEM64G to be able to use all the installed memory

[Ozan Çağlayan]

Alexey Dobriyan wrote:
> On Sat, Mar 07, 2009 at 01:04:38PM +0200, Ozan Çağlayan wrote:
>>
>> I'm quite dumb about the internal structures of the kernel but, wouldn't it
>> be possible to create a simple read-only sysfs object like kexec_loaded
>> that will contain "1" when the kernel needs HIGHMEM64G to see all of the memory
>> and "0" when it doesn't?
> 
> Why do you need a file? If kernel says to enable HIGHMEM64G, enable it.

It's not for me. We're not able to ship HIGHMEM64G enabled kernel packages with
distributions because it breaks some processors not having PAE flag.

So, the PAE enabled kernel should be installed by distribution installers or
package management systems if the processor honors the PAE bit. Because PAE
is an extension mechanism for 32bit processors, AFAIK it has some overhead.
So enabling it if the user doesn't have a memory size >=4GB would not make
much sense if a proper noexecute (NX) support is not intended.

Fedora installer parses /proc/iomem to see if any address listed is bigger than
0x100000000h. I checked it on a machine having 4GB of memory, and it failed.

If the linux kernel is able to drop such an advisory message into the log buffer,
it would be very easy to implement a read-only sysfs object. dmesg parsing
is another hacky, dirty solution to the problem that I would not prefer at all.

Regards,

-- 

Ozan Çağlayan
<ozan_at_pardus.org.tr>

Re: [RFC] A simple way to determine if the kernel needs HIGHMEM64G to be able to use all the installed memory

[Andi Kleen]

Ozan Çağlayan <ozan@pardus.org.tr> writes:
>
> So, the PAE enabled kernel should be installed by distribution installers or
> package management systems if the processor honors the PAE bit. Because PAE
> is an extension mechanism for 32bit processors, AFAIK it has some overhead.
> So enabling it if the user doesn't have a memory size >=4GB would not make
> much sense if a proper noexecute (NX) support is not intended.

The general recommendation is to always use PAE when the machine 
is NX capable. Or at least use it by default.

> Fedora installer parses /proc/iomem to see if any address listed is bigger than
> 0x100000000h. I checked it on a machine having 4GB of memory, and it failed.

Newer kernels have the memmap in /sys/firmware/memmap to export the memory
map. If anything is in there >4GB you need PAE.

But there's also the problem that a few systems have MMIO mappings above 4GB.
If you're lucky they are in the memmap too, but sometimes they are not.

For those you also always need PAE. The only way to check for that
would be to load all drivers and then check /proc/iomem or walk
the PCI resources (in /sys/bus/pci). That would be still not 100% fool
proof, but probably as good as you can get.

-Andi

[PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Peter Zijlstra]

On Sun, 2009-03-08 at 21:20 +0100, Andi Kleen wrote:
> Ozan Çağlayan <ozan@pardus.org.tr> writes:
> >
> > So, the PAE enabled kernel should be installed by distribution installers or
> > package management systems if the processor honors the PAE bit. Because PAE
> > is an extension mechanism for 32bit processors, AFAIK it has some overhead.
> > So enabling it if the user doesn't have a memory size >=4GB would not make
> > much sense if a proper noexecute (NX) support is not intended.
> 
> The general recommendation is to always use PAE when the machine 
> is NX capable. Or at least use it by default.

How about we do the below as well?

---
If the hardware is capable, its much better to run a 64bit kernel than a
32bit PAE kernel with lots of memory.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
 arch/x86/mm/init_32.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
index 34ef5c7..3d22baa 100644
--- a/arch/x86/mm/init_32.c
+++ b/arch/x86/mm/init_32.c
@@ -714,6 +714,9 @@ void __init lowmem_pfn_init(void)
 
 #define MSG_HIGHMEM_TRIMMED \
 	"Warning: only 4GB will be used. Use a HIGHMEM64G enabled kernel!\n"
+
+#define MSG_LM_TRIMMED \
+	"Warning: instead of using a 32bit PAE kernel, Use a 64bit kernel!\n"
 /*
  * We have more RAM than fits into lowmem - we try to put it into
  * highmem, also taking the highmem=x boot parameter into account:
@@ -750,6 +753,8 @@ void __init highmem_pfn_init(void)
 	}
 #endif /* !CONFIG_HIGHMEM64G */
 #endif /* !CONFIG_HIGHMEM */
+	if (boot_cpu_has(X86_FEATURE_LM))
+		printk(KERN_WARNING MSG_LM_TRIMMED);
 }
 
 /*

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Andi Kleen]

> How about we do the below as well?

Idea looks good, but with your patch it would print two conflicting
messages ("use highmem64. No, use 64bit instead"), wouldn't it? 
Only one would be better.

-Andi

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Peter Zijlstra]

On Mon, 2009-03-09 at 20:15 +0100, Andi Kleen wrote:
> > How about we do the below as well?
> 
> Idea looks good, but with your patch it would print two conflicting
> messages ("use highmem64. No, use 64bit instead"), wouldn't it? 
> Only one would be better.

Something like the below?


---
 arch/x86/mm/init_32.c |   26 ++++++++++++++++++++++----
 1 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
index 34ef5c7..e759107 100644
--- a/arch/x86/mm/init_32.c
+++ b/arch/x86/mm/init_32.c
@@ -713,13 +713,19 @@ void __init lowmem_pfn_init(void)
 	"only %luMB highmem pages available, ignoring highmem size of %luMB!\n"
 
 #define MSG_HIGHMEM_TRIMMED \
-	"Warning: only 4GB will be used. Use a HIGHMEM64G enabled kernel!\n"
+	"Warning: only 4GB will be used. Use a %s kernel!\n"
+
+#define MSG_LM_TRIMMED \
+	"Warning: instead of using a 32bit %s kernel, Use a 64bit kernel!\n"
+
+
 /*
  * We have more RAM than fits into lowmem - we try to put it into
  * highmem, also taking the highmem=x boot parameter into account:
  */
 void __init highmem_pfn_init(void)
 {
+	char *highmem;
 	max_low_pfn = MAXMEM_PFN;
 
 	if (highmem_pages == -1)
@@ -737,16 +743,28 @@ void __init highmem_pfn_init(void)
 #ifndef CONFIG_HIGHMEM
 	/* Maximum memory usable is what is directly addressable */
 	printk(KERN_WARNING "Warning only %ldMB will be used.\n", MAXMEM>>20);
+
 	if (max_pfn > MAX_NONPAE_PFN)
-		printk(KERN_WARNING "Use a HIGHMEM64G enabled kernel.\n");
+		highmem = "HIGHMEM64G";
 	else
-		printk(KERN_WARNING "Use a HIGHMEM enabled kernel.\n");
+		highmem = "HIGHMEM";
+
+	if (boot_cpu_has(X86_FEATURE_LM))
+		printk(KERN_WARNING MSG_LM_TRIMMED, highmem);
+	else 
+		printk(KERN_WARNING "Use a %s enabled kernel.\n", highmem);
+
 	max_pfn = MAXMEM_PFN;
 #else /* !CONFIG_HIGHMEM */
 #ifndef CONFIG_HIGHMEM64G
 	if (max_pfn > MAX_NONPAE_PFN) {
 		max_pfn = MAX_NONPAE_PFN;
-		printk(KERN_WARNING MSG_HIGHMEM_TRIMMED);
+		if (boot_cpu_has(X86_FEATURE_LM))
+			highmem = "64bit";
+		else
+			highmem = "HIGHMEM64G enabled";
+
+		printk(KERN_WARNING MSG_HIGHMEM_TRIMMED, highmem);
 	}
 #endif /* !CONFIG_HIGHMEM64G */
 #endif /* !CONFIG_HIGHMEM */

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Andi Kleen]

On Tue, Mar 10, 2009 at 10:37:08AM +0100, Peter Zijlstra wrote:
> On Mon, 2009-03-09 at 20:15 +0100, Andi Kleen wrote:
> > > How about we do the below as well?
> > 
> > Idea looks good, but with your patch it would print two conflicting
> > messages ("use highmem64. No, use 64bit instead"), wouldn't it? 
> > Only one would be better.
> 
> Something like the below?

Looks good from the logic. Only nit would be missing grepability.

-Andi

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Arjan van de Ven]

On Tue, 10 Mar 2009 10:37:08 +0100
Peter Zijlstra <peterz@infradead.org> wrote:
> diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
> index 34ef5c7..e759107 100644
> --- a/arch/x86/mm/init_32.c
> +++ b/arch/x86/mm/init_32.c
> @@ -713,13 +713,19 @@ void __init lowmem_pfn_init(void)
>  	"only %luMB highmem pages available, ignoring highmem size
> of %luMB!\n" 
>  #define MSG_HIGHMEM_TRIMMED \
> -	"Warning: only 4GB will be used. Use a HIGHMEM64G enabled
> kernel!\n"
> +	"Warning: only 4GB will be used. Use a %s kernel!\n"
> +
> +#define MSG_LM_TRIMMED \
> +	"Warning: instead of using a 32bit %s kernel, Use a 64bit
> kernel!\n" +

I would like to avoid the prefix "Warning:" for something like this ;-)

lets keep that prefix for Real Warnings(tm) instead please.


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Peter Zijlstra]

On Tue, 2009-03-10 at 06:21 -0700, Arjan van de Ven wrote:
> On Tue, 10 Mar 2009 10:37:08 +0100
> Peter Zijlstra <peterz@infradead.org> wrote:
> > diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
> > index 34ef5c7..e759107 100644
> > --- a/arch/x86/mm/init_32.c
> > +++ b/arch/x86/mm/init_32.c
> > @@ -713,13 +713,19 @@ void __init lowmem_pfn_init(void)
> >  	"only %luMB highmem pages available, ignoring highmem size
> > of %luMB!\n" 
> >  #define MSG_HIGHMEM_TRIMMED \
> > -	"Warning: only 4GB will be used. Use a HIGHMEM64G enabled
> > kernel!\n"
> > +	"Warning: only 4GB will be used. Use a %s kernel!\n"
> > +
> > +#define MSG_LM_TRIMMED \
> > +	"Warning: instead of using a 32bit %s kernel, Use a 64bit
> > kernel!\n" +
> 
> I would like to avoid the prefix "Warning:" for something like this ;-)
> 
> lets keep that prefix for Real Warnings(tm) instead please.

It is a Real warning, its issued when memory gets trimmed because you
run a wrong kernel. The only change is suggesting to run a 64bit kernel
when possible instead of a HIGHMEM/PAE kernel.

Esp on x86_64 machines with >=4G of memory its insane to run i386-PAE.

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Andi Kleen]

> Esp on x86_64 machines with >=4G of memory its insane to run i386-PAE.

Not sure I would apply insane to 4GB, but it definitely applies 
to >= 16GB, likely >= 8GB. Enough known deadlocks in this case.
I would consider >=16GB on 32bit at least a kerneloops worthy condition.

Perhaps the warnings should vary in urgency.  e.g. if it's >= 16GB and 
not 2:2 split we know that the system will have problems and it should
strongly suggest, otherwise weakly.

Or perhaps it should be measured in percentage of mem_map
in lowmem?

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: [PATCH] x86: suggest running a 64bit kernel on LM capable machines with plenty memory.

[Ingo Molnar]

* Peter Zijlstra <peterz@infradead.org> wrote:

> Something like the below?

> +#define MSG_LM_TRIMMED \
> +	"Warning: instead of using a 32bit %s kernel, Use a 64bit kernel!\n"

Sounds good, but please change the message to something like:

  INFO: running a 32-bit %s kernel on a 64-bit capable CPU!

To not overwhelm Arjan's kerneloops.org site with a hundred 
thousand bogus warnings ;-)

	Ingo