divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[William Lee Irwin III]

PAE is useful for more than supporting more than 4GB RAM. It supports
expanded swapspace and NX executable protections. Some users may want
NX or expanded swapspace support without the overhead or instability
of highmem. For these reasons, the following patch divorces
CONFIG_X86_PAE from CONFIG_HIGHMEM64G.

vs. 2.6.22-rc4-mm2

Cc: Mark Lord <lkml@rtr.ca>
Cc: Andi Kleen <ak@suse.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: William Irwin <wli@holomorphy.com>


Index: mm-2.6.22-rc4-2/arch/i386/Kconfig
===================================================================
--- mm-2.6.22-rc4-2.orig/arch/i386/Kconfig	2007-06-07 00:05:53.609599701 -0700
+++ mm-2.6.22-rc4-2/arch/i386/Kconfig	2007-06-07 17:02:24.333262965 -0700
@@ -544,6 +544,7 @@
 config HIGHMEM64G
 	bool "64GB"
 	depends on X86_CMPXCHG64
+	select X86_PAE
 	help
 	  Select this if you have a 32-bit processor and more than 4
 	  gigabytes of physical RAM.
@@ -573,12 +574,12 @@
 	config VMSPLIT_3G
 		bool "3G/1G user/kernel split"
 	config VMSPLIT_3G_OPT
-		depends on !HIGHMEM
+		depends on !X86_PAE
 		bool "3G/1G user/kernel split (for full 1G low memory)"
 	config VMSPLIT_2G
 		bool "2G/2G user/kernel split"
 	config VMSPLIT_2G_OPT
-		depends on !HIGHMEM
+		depends on !X86_PAE
 		bool "2G/2G user/kernel split (for full 2G low memory)"
 	config VMSPLIT_1G
 		bool "1G/3G user/kernel split"
@@ -598,10 +599,15 @@
 	default y
 
 config X86_PAE
-	bool
-	depends on HIGHMEM64G
-	default y
+	bool "PAE (Physical Address Extension) Support"
+	default n
+	depends on !HIGHMEM4G
 	select RESOURCES_64BIT
+	help
+	  PAE is required for NX support, and furthermore enables
+	  larger swapspace support for non-overcommit purposes. It
+	  has the cost of more pagetable lookup overhead, and also
+	  consumes more pagetable space per process.
 
 # Common NUMA Features
 config NUMA
Index: mm-2.6.22-rc4-2/arch/i386/kernel/setup.c
===================================================================
--- mm-2.6.22-rc4-2.orig/arch/i386/kernel/setup.c	2007-06-06 23:52:18.839168580 -0700
+++ mm-2.6.22-rc4-2/arch/i386/kernel/setup.c	2007-06-07 17:02:24.349263876 -0700
@@ -273,18 +273,18 @@
 		printk(KERN_WARNING "Warning only %ldMB will be used.\n",
 					MAXMEM>>20);
 		if (max_pfn > MAX_NONPAE_PFN)
-			printk(KERN_WARNING "Use a PAE enabled kernel.\n");
+			printk(KERN_WARNING "Use a HIGHMEM64G enabled kernel.\n");
 		else
 			printk(KERN_WARNING "Use a HIGHMEM enabled kernel.\n");
 		max_pfn = MAXMEM_PFN;
 #else /* !CONFIG_HIGHMEM */
-#ifndef CONFIG_X86_PAE
+#ifndef CONFIG_HIGHMEM64G
 		if (max_pfn > MAX_NONPAE_PFN) {
 			max_pfn = MAX_NONPAE_PFN;
 			printk(KERN_WARNING "Warning only 4GB will be used.\n");
-			printk(KERN_WARNING "Use a PAE enabled kernel.\n");
+			printk(KERN_WARNING "Use a HIGHMEM64G enabled kernel.\n");
 		}
-#endif /* !CONFIG_X86_PAE */
+#endif /* !CONFIG_HIGHMEM64G */
 #endif /* !CONFIG_HIGHMEM */
 	} else {
 		if (highmem_pages == -1)

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[Andrew Morton]

On Thu, 7 Jun 2007 19:35:51 -0700 William Lee Irwin III <wli@holomorphy.com> wrote:

> PAE is useful for more than supporting more than 4GB RAM. It supports
> expanded swapspace and NX executable protections. Some users may want
> NX or expanded swapspace support without the overhead or instability
> of highmem. For these reasons, the following patch divorces
> CONFIG_X86_PAE from CONFIG_HIGHMEM64G.

Do (CONFIG_X86_PAE && !CONFIG_HIGHMEM64G) and (!CONFIG_X86_PAE && CONFIG_HIGHMEM64G)
kernels actually work?  I wouldn't be surprised if there are places where we used
the incorrect one.

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[William Lee Irwin III]

On Thu, 7 Jun 2007 19:35:51 -0700 William Lee Irwin III <wli@holomorphy.com> wrote:
>> PAE is useful for more than supporting more than 4GB RAM. It supports
>> expanded swapspace and NX executable protections. Some users may want
>> NX or expanded swapspace support without the overhead or instability
>> of highmem. For these reasons, the following patch divorces
>> CONFIG_X86_PAE from CONFIG_HIGHMEM64G.

On Thu, Jun 07, 2007 at 07:41:56PM -0700, Andrew Morton wrote:
> Do (CONFIG_X86_PAE && !CONFIG_HIGHMEM64G) and (!CONFIG_X86_PAE && CONFIG_HIGHMEM64G)
> kernels actually work?  I wouldn't be surprised if there are places where we used
> the incorrect one.

!CONFIG_X86_PAE && CONFIG_HIGHMEM64G doesn't make sense and is not allowed
by this patch. CONFIG_X86_PAE && !CONFIG_HIGHMEM64G works here.


-- wli

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[H. Peter Anvin]

William Lee Irwin III wrote:
> 
> !CONFIG_X86_PAE && CONFIG_HIGHMEM64G doesn't make sense and is not allowed
> by this patch. CONFIG_X86_PAE && !CONFIG_HIGHMEM64G works here.
> 

But what's the point?

If you're going to divorce these, at least do it in a way that makes
sense, specifically the two independent variables are PAE and HIGHMEM.
PAE and !HIGHMEM does make (some amount of) sense, due to no kmap overhead.

	-hpa

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[William Lee Irwin III]

William Lee Irwin III wrote:
>> !CONFIG_X86_PAE && CONFIG_HIGHMEM64G doesn't make sense and is not allowed
>> by this patch. CONFIG_X86_PAE && !CONFIG_HIGHMEM64G works here.


On Thu, Jun 07, 2007 at 08:38:22PM -0700, H. Peter Anvin wrote:
> But what's the point?
> If you're going to divorce these, at least do it in a way that makes
> sense, specifically the two independent variables are PAE and HIGHMEM.
> PAE and !HIGHMEM does make (some amount of) sense, due to no kmap overhead.

Beg your pardon? Are you reading the patch description correctly?


-- wli

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[H. Peter Anvin]

William Lee Irwin III wrote:
> William Lee Irwin III wrote:
>>> !CONFIG_X86_PAE && CONFIG_HIGHMEM64G doesn't make sense and is not allowed
>>> by this patch. CONFIG_X86_PAE && !CONFIG_HIGHMEM64G works here.
> 
> 
> On Thu, Jun 07, 2007 at 08:38:22PM -0700, H. Peter Anvin wrote:
>> But what's the point?
>> If you're going to divorce these, at least do it in a way that makes
>> sense, specifically the two independent variables are PAE and HIGHMEM.
>> PAE and !HIGHMEM does make (some amount of) sense, due to no kmap overhead.
> 
> Beg your pardon? Are you reading the patch description correctly?
> 

I mean, with your patch CONFIG_HIGHMEM4G versus CONFIG_HIGHMEM64G really
don't make sense as separate selections anymore.

	-hpa

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[William Lee Irwin III]

William Lee Irwin III wrote:
>> Beg your pardon? Are you reading the patch description correctly?

On Thu, Jun 07, 2007 at 08:44:09PM -0700, H. Peter Anvin wrote:
> I mean, with your patch CONFIG_HIGHMEM4G versus CONFIG_HIGHMEM64G really
> don't make sense as separate selections anymore.

I thought about sweeping those up, but defaulted to minimal diffsize.
I can sweep them up given more votes in favor of doing so.


-- wli

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[Adrian Bunk]

On Thu, Jun 07, 2007 at 07:35:51PM -0700, William Lee Irwin III wrote:
>...
>  config X86_PAE
>...
> +	help
> +	  PAE is required for NX support, and furthermore enables
> +	  larger swapspace support for non-overcommit purposes. It
> +	  has the cost of more pagetable lookup overhead, and also
> +	  consumes more pagetable space per process.
>...


It's not specific to this help text, but I start becoming a bit picky 
about this issues:

If you understand this help text after reading it, you don't need a help 
text for this option...  ;-)

What is "NX support"?
What are "non-overcommit purposes"?
What is "pagetable lookup overhead"?

And if in doubt, should I say Y or N?

"System administrator who knows which hardware components he put into 
the computer and which filesystems his data is on" might be a good 
description for the average kconfig user, and these are the people who 
should understand this help text.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[William Lee Irwin III]

On Thu, Jun 07, 2007 at 07:35:51PM -0700, William Lee Irwin III wrote:
>> +	  PAE is required for NX support, and furthermore enables
>> +	  larger swapspace support for non-overcommit purposes. It
>> +	  has the cost of more pagetable lookup overhead, and also
>> +	  consumes more pagetable space per process.

On Tue, Jun 12, 2007 at 01:52:35AM +0200, Adrian Bunk wrote:
> It's not specific to this help text, but I start becoming a bit picky 
> about this issues:
> If you understand this help text after reading it, you don't need a help 
> text for this option...  ;-)
> What is "NX support"?
> What are "non-overcommit purposes"?
> What is "pagetable lookup overhead"?
> And if in doubt, should I say Y or N?
> "System administrator who knows which hardware components he put into 
> the computer and which filesystems his data is on" might be a good 
> description for the average kconfig user, and these are the people who 
> should understand this help text.

I would like to have some place to explain issues such as those, but
there are as of yet no designated places for tutorial-level information.

If such a place were provided, I would provide storybook commentary to
explain all those. Similarly actually holds for kernel function docbook.


-- wli

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[Adrian Bunk]

On Mon, Jun 11, 2007 at 05:00:26PM -0700, William Lee Irwin III wrote:
> On Thu, Jun 07, 2007 at 07:35:51PM -0700, William Lee Irwin III wrote:
> >> +	  PAE is required for NX support, and furthermore enables
> >> +	  larger swapspace support for non-overcommit purposes. It
> >> +	  has the cost of more pagetable lookup overhead, and also
> >> +	  consumes more pagetable space per process.
> 
> On Tue, Jun 12, 2007 at 01:52:35AM +0200, Adrian Bunk wrote:
> > It's not specific to this help text, but I start becoming a bit picky 
> > about this issues:
> > If you understand this help text after reading it, you don't need a help 
> > text for this option...  ;-)
> > What is "NX support"?
> > What are "non-overcommit purposes"?
> > What is "pagetable lookup overhead"?
> > And if in doubt, should I say Y or N?
> > "System administrator who knows which hardware components he put into 
> > the computer and which filesystems his data is on" might be a good 
> > description for the average kconfig user, and these are the people who 
> > should understand this help text.
> 
> I would like to have some place to explain issues such as those, but
> there are as of yet no designated places for tutorial-level information.
>  
> If such a place were provided, I would provide storybook commentary to
> explain all those. Similarly actually holds for kernel function docbook.

There's no 4 line limit for Kconfig entries. If it takes a 10 line 
paragraph for a short explanation what the NX bit is and when enabling 
NX support makes sense (because it will be used) then that's completely 
appropriate here. The same goes for the other parts.

The kconfig help should give anyone running "make oldconfig" a rough 
understanding what this option is about and a clear understanding what 
to answer for this option ("If unsure, say Y/N." is a standard text we 
use for the latter).

> -- wli

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[Mikael Pettersson]

On Thu, 7 Jun 2007 19:35:51 -0700, William Lee Irwin III wrote:
> @@ -573,12 +574,12 @@
>  	config VMSPLIT_3G
>  		bool "3G/1G user/kernel split"
>  	config VMSPLIT_3G_OPT
> -		depends on !HIGHMEM
> +		depends on !X86_PAE
>  		bool "3G/1G user/kernel split (for full 1G low memory)"
>  	config VMSPLIT_2G
>  		bool "2G/2G user/kernel split"
>  	config VMSPLIT_2G_OPT
> -		depends on !HIGHMEM
> +		depends on !X86_PAE
>  		bool "2G/2G user/kernel split (for full 2G low memory)"
>  	config VMSPLIT_1G
>  		bool "1G/3G user/kernel split"

Is this really needed? I can see why VMSPLIT_{2,3}G_OPT would
depend on !HIGHMEM, but why would they depend on !X86_PAE?

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[William Lee Irwin III]

On Fri, Jun 08, 2007 at 10:07:52AM +0200, Mikael Pettersson wrote:
> Is this really needed? I can see why VMSPLIT_{2,3}G_OPT would
> depend on !HIGHMEM, but why would they depend on !X86_PAE?

The only reason they depend on !HIGHMEM is because handling for
1GB-unaligned splits is unimplemented for PAE, which formerly only
occurred in conjunction with HIGHMEM64G. That said, they were oriented
toward avoiding highmem on laptops, hence the broader !HIGHMEM
constraint. The entire point of the patch is to add an option to use
PAE without highmem for the purposes of NX and secondarily expanded
swapspace, at which point CONFIG_VMSPLIT_[23]G_OPT need some other way
besides !HIGHMEM to exclude PAE, such as specifying !X86_PAE directly.


-- wli

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[linux]

Given that incomprehensible help texts are a bit of a pet peeve of mine
(I just last weekend figured out that you don't need to select an I2C
algorithm driver to have working I2c - I had thought it was a "one from
column A, one from column B" thing), let me take a crack...

	PAE doubles the size of each page table entry, increasing
	kernel memory consumption and slowing page table access.
	However, it enables:
	- Addressing more than 4G of physical RAM (CONFIG_HIGHMEM is
	  also required)
	- Marking pages as readable but not executable using the NX
	  (no-execute) bit, which protects applications from stack
	  overflow attacks.
	- Swap files or partitions larger than 64G each.
	  (Only needed with >4G RAM or very heavy tmpfs use.)

	A kernel compiled with this option cannot boot on a processor
	without PAE support.  Enabling this also disables the
	(expert use only) CONFIG_VMSPLIT_[23]G_OPT options.

Does that seem reasonably user-oriented?

Re: divorce CONFIG_X86_PAE from CONFIG_HIGHMEM64G

[Bodo Eggert]

Since your (William) patch will change the kconfig files my proposed patch
would change, could you please add those changes? 

I hand-updated the patch below as recommended by the original discussion 
on LKML. It won't aply as-is because of that (and because of your changes).


--- 2.6.21/arch/i386/Kconfig.ori	2007-06-06 13:41:09.000000000 +0200
+++ 2.6.21/arch/i386/Kconfig	2007-06-06 14:07:40.000000000 +0200
@@ -495,8 +495,8 @@ config NOHIGHMEM
 	bool "off"
 	depends on !X86_NUMAQ
 	---help---
-	  Linux can use up to 64 Gigabytes of physical memory on x86 systems.
-	  However, the address space of 32-bit x86 processors is only 4
+	  Linux can use up to 64 Gigabytes of physical address space on x86
+	  systems. However, the address space of 32-bit x86 processors is only 4
 	  Gigabytes large. That means that, if you have a large amount of
 	  physical memory, not all of it can be "permanently mapped" by the
 	  kernel. The physical memory that's not permanently mapped is called
@@ -510,8 +510,15 @@ config NOHIGHMEM
 	  by the kernel to permanently map as much physical memory as
 	  possible.
 
-	  If the machine has between 1 and 4 Gigabytes physical RAM, then
+
+	  If the machine has between 1 and about 3 Gigabytes physical RAM, 
+         then
 	  answer "4GB" here.
+	  
+	  The PCI address space will usually take 512 MB or 1 GB of address
+	  space. This address space is unavailable to RAM, but depending on the
+	  chipset (and BIOS settings), memory overlapping the PCI address space
+	  may be mapped beyond the 4 GB limit and be available using "64GB".
+
 
 	  If more than 4 Gigabytes is used then answer "64GB" here. This
 	  selection turns Intel PAE (Physical Address Extension) mode on.
@@ -520,6 +527,10 @@ config NOHIGHMEM
 	  processors (Pentium Pro and better). NOTE: If you say "64GB" here,
 	  then the kernel will not boot on CPUs that don't support PAE!
 
+	  An additional benefit of the 64GB-Mode is the availability of the
+	  no-execute-pageflag, which can be used to prevent some attacks from
+	  injecting malicious code into applications.
+
 	  The actual amount of total physical memory will either be
 	  auto detected or can be forced by using a kernel command line option
 	  such as "mem=256M". (Try "man bootparam" or see the documentation of
@@ -532,14 +543,14 @@ config HIGHMEM4G
 	bool "4GB"
 	depends on !X86_NUMAQ
 	help
-	  Select this if you have a 32-bit processor and between 1 and 4
+	  Select this if you have a 32-bit processor and between 1 and 3
 	  gigabytes of physical RAM.
 
 config HIGHMEM64G
-	bool "64GB"
+	bool "64GB (enables no-execute memory protection if available)"
 	depends on X86_CMPXCHG64
 	help
-	  Select this if you have a 32-bit processor and more than 4
+	  Select this if you have a 32-bit processor and more than 3
 	  gigabytes of physical RAM.
 
 endchoice
-- 
RAM DISK is not an installation procedure!