Re: Have ever checked in your mips sparsemem code into mips-linux tree?

Re: Have ever checked in your mips sparsemem code into mips-linux tree?

[C Michael Sundius]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

adding patch 2  containing Documentation:




     - - - - -                              Cisco                            - - - - -         
This e-mail and any attachments may contain information which is confidential, 
proprietary, privileged or otherwise protected by law. The information is solely 
intended for the named addressee (or a person responsible for delivering it to 
the addressee). If you are not the intended recipient of this message, you are 
not authorized to read, print, retain, copy or disseminate this message or any 
part of it. If you have received this e-mail in error, please notify the sender 
immediately by return e-mail and delete it from your computer.

Re: Have ever checked in your mips sparsemem code into mips-linux tree?

[Ralf Baechle]

On Mon, Oct 06, 2008 at 01:15:49PM -0700, C Michael Sundius wrote:

Btw, I'm planning to rip support for discontig memory from MIPS.  IP27
is the only platform using it and it also would be better off using
sparsemem instead.

  Ralf

Re: Have ever checked in your mips sparsemem code into mips-linux tree?

[Ralf Baechle]

On Mon, Oct 06, 2008 at 01:15:49PM -0700, C Michael Sundius wrote:

Btw, I'm planning to rip support for discontig memory from MIPS.  IP27
is the only platform using it and it also would be better off using
sparsemem instead.

  Ralf

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Have ever checked in your mips sparsemem code into mips-linux tree?

[John]

Thank you Michael! I will try it out, and will post results later, but not next couple of days, since I have some stuff on hands approaching the deadline.

John

--- On Mon, 10/6/08, C Michael Sundius <Michael.sundius@sciatl.com> wrote:

> From: C Michael Sundius <Michael.sundius@sciatl.com>
> Subject: Re: Have ever checked in your mips sparsemem code into mips-linux tree?
> To: "Andy Whitcroft" <apw@shadowen.org>, "Dave Hansen" <dave@linux.vnet.ibm.com>, linux-mm@kvack.org, linux-mips@linux-mips.org, "VomLehn, David" <dvomlehn@cisco.com>, me94043@yahoo.com
> Date: Monday, October 6, 2008, 1:15 PM
> adding patch 2  containing Documentation:
> 
> 
> 
> 
>      - - - - -                              Cisco          
>                  - - - - -         
> This e-mail and any attachments may contain information
> which is confidential, 
> proprietary, privileged or otherwise protected by law. The
> information is solely 
> intended for the named addressee (or a person responsible
> for delivering it to 
> the addressee). If you are not the intended recipient of
> this message, you are 
> not authorized to read, print, retain, copy or disseminate
> this message or any 
> part of it. If you have received this e-mail in error,
> please notify the sender 
> immediately by return e-mail and delete it from your
> computer.From e01ad377b29c0e5c39289bece382e1f78f6e7e2c Mon
> Sep 17 00:00:00 2001
> From: Sundis <sundism@CUPLXSUNDISM01.corp.sa.net>
> Date: Mon, 6 Oct 2008 10:31:08 -0700
> Subject: [PATCH] mips sparsemem howto
> 
> ---
>  Documentation/sparsemem.txt |   92
> +++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 92 insertions(+), 0 deletions(-)
>  create mode 100644 Documentation/sparsemem.txt
> 
> diff --git a/Documentation/sparsemem.txt
> b/Documentation/sparsemem.txt
> new file mode 100644
> index 0000000..0b36412
> --- /dev/null
> +++ b/Documentation/sparsemem.txt
> @@ -0,0 +1,92 @@
> +Sparsemem divides up physical memory in your system into N
> sections of M
> +bytes. Page tables are created for only those sections
> that
> +actually exist (as far as the sparsemem code is
> concerned). This allows
> +for holes in the physical memory without having to waste
> space by
> +creating page descriptors for those pages that do not
> exist.
> +When page_to_pfn() or pfn_to_page() are called there is a
> bit of overhead to
> +look up the proper memory section to get to the
> page_table, but this
> +is small compared to the memory you are likely to save.
> So, it's not the
> +default, but should be used if you have big holes in
> physical memory.
> +
> +Note that discontiguous memory is more closely related to
> NUMA machines
> +and if you are a single CPU system use sparsemem and not
> discontig. 
> +It's much simpler. 
> +
> +1) CALL MEMORY_PRESENT()
> +Existing sections are recorded once the bootmem allocator
> is up and running by
> +calling the sparsemem function "memory_present(node,
> pfn_start, pfn_end)" for each
> +block of memory that exists in your physical address
> space. The
> +memory_present() function records valid sections in a data
> structure called
> +mem_section[].
> +
> +2) DETERMINE AND SET THE SIZE OF SECTIONS AND PHYSMEM
> +The size of N and M above depend upon your architecture
> +and your platform and are specified in the file:
> +
> +      include/asm-<your_arch>/sparsemem.h
> +
> +and you should create the following lines similar to
> below: 
> +
> +	#ifdef CONFIG_YOUR_PLATFORM
> +	 #define SECTION_SIZE_BITS       27	/* 128 MiB */
> +	#endif
> +	#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
> +
> +if they don't already exist, where: 
> +
> + * SECTION_SIZE_BITS            2^M: how big each section
> will be
> + * MAX_PHYSMEM_BITS             2^N: how much memory we
> can have in that
> +                                     space
> +
> +3) INITIALIZE SPARSE MEMORY
> +You should make sure that you initialize the sparse memory
> code by calling 
> +
> +	bootmem_init();
> +  +	sparse_init();
> +	paging_init();
> +
> +just before you call paging_init() and after the
> bootmem_allocator is
> +turned on in your setup_arch() code.  
> +
> +4) ENABLE SPARSEMEM IN KCONFIG
> +Add a line like this:
> +
> +	select ARCH_SPARSEMEM_ENABLE
> +
> +into the config for your platform in
> arch/<your_arch>/Kconfig. This will
> +ensure that turning on sparsemem is enabled for your
> platform. 
> +
> +5) CONFIG
> +Run make *config, as you like, and turn on the sparsemem
> +memory model under the "Kernel Type" -->
> "Memory Model" and then build your
> +kernel.
> +
> +
> +6) Gotchas
> +
> +One trick that I encountered when I was turning this on
> for MIPS was that there
> +was some code in mem_init() that set the
> "reserved" flag for pages that were not
> +valid RAM. This caused my kernel to crash when I enabled
> sparsemem since those
> +pages (and page descriptors) didn't actually exist. I
> changed my code by adding
> +lines like below:
> +
> +
> +	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
> +		struct page *page = pfn_to_page(tmp);
> +
> +   +		if (!pfn_valid(tmp))
> +   +			continue;
> +   +
> +		if (!page_is_ram(tmp)) {
> +			SetPageReserved(page);
> +			continue;
> +		}
> +		ClearPageReserved(page);
> +		init_page_count(page);
> +		__free_page(page);
> +		physmem_record(PFN_PHYS(tmp), PAGE_SIZE,
> physmem_highmem);
> +		totalhigh_pages++;
> +	}
> +
> +
> +Once I got that straight, it worked!!!! I saved 10MiB of
> memory.  
> -- 
> 1.5.4.1


      

Re: Have ever checked in your mips sparsemem code into mips-linux tree?

[John]

Thank you Michael! I will try it out, and will post results later, but not next couple of days, since I have some stuff on hands approaching the deadline.

John

--- On Mon, 10/6/08, C Michael Sundius <Michael.sundius@sciatl.com> wrote:

> From: C Michael Sundius <Michael.sundius@sciatl.com>
> Subject: Re: Have ever checked in your mips sparsemem code into mips-linux tree?
> To: "Andy Whitcroft" <apw@shadowen.org>, "Dave Hansen" <dave@linux.vnet.ibm.com>, linux-mm@kvack.org, linux-mips@linux-mips.org, "VomLehn, David" <dvomlehn@cisco.com>, me94043@yahoo.com
> Date: Monday, October 6, 2008, 1:15 PM
> adding patch 2  containing Documentation:
> 
> 
> 
> 
>      - - - - -                              Cisco          
>                  - - - - -         
> This e-mail and any attachments may contain information
> which is confidential, 
> proprietary, privileged or otherwise protected by law. The
> information is solely 
> intended for the named addressee (or a person responsible
> for delivering it to 
> the addressee). If you are not the intended recipient of
> this message, you are 
> not authorized to read, print, retain, copy or disseminate
> this message or any 
> part of it. If you have received this e-mail in error,
> please notify the sender 
> immediately by return e-mail and delete it from your
> computer.From e01ad377b29c0e5c39289bece382e1f78f6e7e2c Mon
> Sep 17 00:00:00 2001
> From: Sundis <sundism@CUPLXSUNDISM01.corp.sa.net>
> Date: Mon, 6 Oct 2008 10:31:08 -0700
> Subject: [PATCH] mips sparsemem howto
> 
> ---
>  Documentation/sparsemem.txt |   92
> +++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 92 insertions(+), 0 deletions(-)
>  create mode 100644 Documentation/sparsemem.txt
> 
> diff --git a/Documentation/sparsemem.txt
> b/Documentation/sparsemem.txt
> new file mode 100644
> index 0000000..0b36412
> --- /dev/null
> +++ b/Documentation/sparsemem.txt
> @@ -0,0 +1,92 @@
> +Sparsemem divides up physical memory in your system into N
> sections of M
> +bytes. Page tables are created for only those sections
> that
> +actually exist (as far as the sparsemem code is
> concerned). This allows
> +for holes in the physical memory without having to waste
> space by
> +creating page descriptors for those pages that do not
> exist.
> +When page_to_pfn() or pfn_to_page() are called there is a
> bit of overhead to
> +look up the proper memory section to get to the
> page_table, but this
> +is small compared to the memory you are likely to save.
> So, it's not the
> +default, but should be used if you have big holes in
> physical memory.
> +
> +Note that discontiguous memory is more closely related to
> NUMA machines
> +and if you are a single CPU system use sparsemem and not
> discontig. 
> +It's much simpler. 
> +
> +1) CALL MEMORY_PRESENT()
> +Existing sections are recorded once the bootmem allocator
> is up and running by
> +calling the sparsemem function "memory_present(node,
> pfn_start, pfn_end)" for each
> +block of memory that exists in your physical address
> space. The
> +memory_present() function records valid sections in a data
> structure called
> +mem_section[].
> +
> +2) DETERMINE AND SET THE SIZE OF SECTIONS AND PHYSMEM
> +The size of N and M above depend upon your architecture
> +and your platform and are specified in the file:
> +
> +      include/asm-<your_arch>/sparsemem.h
> +
> +and you should create the following lines similar to
> below: 
> +
> +	#ifdef CONFIG_YOUR_PLATFORM
> +	 #define SECTION_SIZE_BITS       27	/* 128 MiB */
> +	#endif
> +	#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
> +
> +if they don't already exist, where: 
> +
> + * SECTION_SIZE_BITS            2^M: how big each section
> will be
> + * MAX_PHYSMEM_BITS             2^N: how much memory we
> can have in that
> +                                     space
> +
> +3) INITIALIZE SPARSE MEMORY
> +You should make sure that you initialize the sparse memory
> code by calling 
> +
> +	bootmem_init();
> +  +	sparse_init();
> +	paging_init();
> +
> +just before you call paging_init() and after the
> bootmem_allocator is
> +turned on in your setup_arch() code.  
> +
> +4) ENABLE SPARSEMEM IN KCONFIG
> +Add a line like this:
> +
> +	select ARCH_SPARSEMEM_ENABLE
> +
> +into the config for your platform in
> arch/<your_arch>/Kconfig. This will
> +ensure that turning on sparsemem is enabled for your
> platform. 
> +
> +5) CONFIG
> +Run make *config, as you like, and turn on the sparsemem
> +memory model under the "Kernel Type" -->
> "Memory Model" and then build your
> +kernel.
> +
> +
> +6) Gotchas
> +
> +One trick that I encountered when I was turning this on
> for MIPS was that there
> +was some code in mem_init() that set the
> "reserved" flag for pages that were not
> +valid RAM. This caused my kernel to crash when I enabled
> sparsemem since those
> +pages (and page descriptors) didn't actually exist. I
> changed my code by adding
> +lines like below:
> +
> +
> +	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
> +		struct page *page = pfn_to_page(tmp);
> +
> +   +		if (!pfn_valid(tmp))
> +   +			continue;
> +   +
> +		if (!page_is_ram(tmp)) {
> +			SetPageReserved(page);
> +			continue;
> +		}
> +		ClearPageReserved(page);
> +		init_page_count(page);
> +		__free_page(page);
> +		physmem_record(PFN_PHYS(tmp), PAGE_SIZE,
> physmem_highmem);
> +		totalhigh_pages++;
> +	}
> +
> +
> +Once I got that straight, it worked!!!! I saved 10MiB of
> memory.  
> -- 
> 1.5.4.1


      

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

sparsemem support for mips with highmem

[C Michael Sundius]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

Hi

I just got sparsemem working on our MIPS 32 platform. I'm not sure if 
anyone
has done that before since there seems to be a couple of problems in the 
arch specific code.

Well I realize that it is blazingly simple to turn on sparsemem, but for 
the idiots (like myself)
out there I created a howto file to put in the Documentation directory 
just because I thought
it would be a good idea to have some official info on  it written down 
somewhere.

it saved me a ton of space by the way.  it seems to work great.

Mike

Re: sparsemem support for mips with highmem

[Dave Hansen]

On Thu, 2008-08-14 at 15:05 -0700, C Michael Sundius wrote:
> I just got sparsemem working on our MIPS 32 platform. I'm not sure if 
> anyone
> has done that before since there seems to be a couple of problems in the 
> arch specific code.
> 
> Well I realize that it is blazingly simple to turn on sparsemem, but for 
> the idiots (like myself)
> out there I created a howto file to put in the Documentation directory 
> just because I thought
> it would be a good idea to have some official info on  it written down 
> somewhere.
> 
> it saved me a ton of space by the way.  it seems to work great.

Cool!  Thanks for writing all that up.

>  arch/mips/kernel/setup.c     |   18 +++++++++++++++++-
>  arch/mips/mm/init.c          |    3 +++
>  include/asm-mips/sparsemem.h |    6 ++++++
>  3 files changed, 26 insertions(+), 1 deletions(-)

Wow!  25 lines of code.  Sparsemem is a pig! :)

> diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
> index f8a535a..6ff0f72 100644
> --- a/arch/mips/kernel/setup.c
> +++ b/arch/mips/kernel/setup.c
> @@ -405,7 +405,6 @@ static void __init bootmem_init(void)
> 
>  		/* Register lowmem ranges */
>  		free_bootmem(PFN_PHYS(start), size << PAGE_SHIFT);
> -		memory_present(0, start, end);
>  	}
> 
>  	/*
> @@ -417,6 +416,23 @@ static void __init bootmem_init(void)
>  	 * Reserve initrd memory if needed.
>  	 */
>  	finalize_initrd();
> +
> +	/* call memory present for all the ram */
> +	for (i = 0; i < boot_mem_map.nr_map; i++) {
> +		unsigned long start, end;
> +
> +		/*
> + * 		 * memory present only usable memory.
> + * 		 		 */

There's a wee bit of whitespace weirdness in here.  You might want to go
double-check it.

> +		if (boot_mem_map.map[i].type != BOOT_MEM_RAM)
> +			continue;
> +
> +		start = PFN_UP(boot_mem_map.map[i].addr);
> +		end   = PFN_DOWN(boot_mem_map.map[i].addr
> +				    + boot_mem_map.map[i].size);
> +
> +		memory_present(0, start, end);
> +	}
>  }

Is that aligning really necessary?  I'm just curious because if it is,
it would probably be good to stick it inside memory_present().

<snip>
> +Sparsemem divides up physical memory in your system into N section of M
> +bytes. Page tables are created for only those sections that
> +actually exist (as far as the sparsemem code is concerned). This allows
> +for holes in the physical memory without having to waste space by
> +creating page discriptors for those pages that do not exist.

descriptors

> +When page_to_pfn() or pfn_to_page() are called there is a bit of overhead to
> +look up the proper memory section to get to the page_table, but this
> +is small compared to the memory you are likely to save. So, it's not the
> +default, but should be used if you have big holes in physical memory.
> +
> +Note that discontiguous memory is more closely related to NUMA machines
> +and if you are a single CPU system use sparsemem and not discontig. 
> +It's much simpler. 
> +
> +1) CALL MEMORY_PRESENT()
> +Existing sections are recorded once the bootmem allocator is up and running by
> +calling the sparsemem function "memory_present(node, pfn_start, pfn_end)" for each
> +block of memory that exists in your physical address space. The
> +memory_present() function records valid sections in a data structure called
> +mem_section[].

I might reword this a bit, but it's not big deal:

Once the bootmem allocator is up and running, you should call the
sparsemem function "memory_present(node, pfn_start, pfn_end)" for each
block of memory that exists on your system.

> +6) Gotchas
> +
> +One trick that I encountered when I was turning this on for MIPS was that there
> +was some code in mem_init() that set the "reserved" flag for pages that were not
> +valid RAM. This caused my kernel to crash when I enabled sparsemem since those
> +pages (and page descriptors) didn't actually exist. I changed my code by adding
> +lines like below:
> +
> +
> +	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
> +		struct page *page = pfn_to_page(tmp);
> +
> +   +		if (!pfn_valid(tmp))
> +   +			continue;
> +   +
> +		if (!page_is_ram(tmp)) {
> +			SetPageReserved(page);
> +			continue;
> +		}
> +		ClearPageReserved(page);
> +		init_page_count(page);
> +		__free_page(page);
> +		physmem_record(PFN_PHYS(tmp), PAGE_SIZE, physmem_highmem);
> +		totalhigh_pages++;
> +	}
> +
> +
> +Once I got that straight, it worked!!!! I saved 10MiB of memory.  

Note: this would be a bug on both DISCONTIG and SPARSEMEM systems.  It
is a common one where ranges of physical memory are walked without
regard for whether there are 'struct page's backing those ares.  These
kinds of coding errors are perhaps the most common when converting from
FLATMEM to DISCONTIG/SPARSEMEM.

-- Dave

Re: sparsemem support for mips with highmem

[C Michael Sundius]

[-- Attachment #1: Type: text/plain, Size: 14 bytes --]

fixed patch



[-- Attachment #2: mypatchfile.1 --]
[-- Type: text/plain, Size: 5260 bytes --]

diff --git a/Documentation/sparsemem.txt b/Documentation/sparsemem.txt
new file mode 100644
index 0000000..6aea0d1
--- /dev/null
+++ b/Documentation/sparsemem.txt
@@ -0,0 +1,93 @@
+Sparsemem divides up physical memory in your system into N section of M
+bytes. Page descriptors are created for only those sections that
+actually exist (as far as the sparsemem code is concerned). This allows
+for holes in the physical memory without having to waste space by
+creating page discriptors for those pages that do not exist.
+When page_to_pfn() or pfn_to_page() are called there is a bit of overhead to
+look up the proper memory section to get to the descriptors, but this
+is small compared to the memory you are likely to save. So, it's not the
+default, but should be used if you have big holes in physical memory.
+
+Note that discontiguous memory is more closely related to NUMA machines
+and if you are a single CPU system use sparsemem and not discontig. 
+It's much simpler. 
+
+1) CALL MEMORY_PRESENT()
+Once the bootmem allocator is up and running, you should call the
+sparsemem function "memory_present(node, pfn_start, pfn_end)" for each
+block of memory that exists on your system.
+
+2) DETERMINE AND SET THE SIZE OF SECTIONS AND PHYSMEM
+The size of N and M above depend upon your architecture
+and your platform and are specified in the file:
+
+      include/asm-<your_arch>/sparsemem.h
+
+and you should create the following lines similar to below: 
+
+	#ifdef CONFIG_YOUR_PLATFORM
+	 #define SECTION_SIZE_BITS       27	/* 128 MiB */
+	#endif
+	#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
+
+if they don't already exist, where: 
+
+ * SECTION_SIZE_BITS            2^M: how big each section will be
+ * MAX_PHYSMEM_BITS             2^N: how much memory we can have in that
+                                     space
+
+3) INITIALIZE SPARSE MEMORY
+You should make sure that you initialize the sparse memory code by calling 
+
+	bootmem_init();
+  +	sparse_init();
+	paging_init();
+
+just before you call paging_init() and after the bootmem_allocator is
+turned on in your setup_arch() code.  
+
+4) ENABLE SPARSEMEM IN KCONFIG
+Add a line like this:
+
+	select ARCH_SPARSEMEM_ENABLE
+
+into the config for your platform in arch/<your_arch>/Kconfig. This will
+ensure that turning on sparsemem is enabled for your platform. 
+
+5) CONFIG
+Run make menuconfig or make gconfig, as you like, and turn on the sparsemem
+memory model under the "Kernel Type" --> "Memory Model" and then build your
+kernel.
+
+
+6) Gotchas
+
+One trick that I encountered when I was turning this on for MIPS was that there
+was some code in mem_init() that set the "reserved" flag for pages that were not
+valid RAM. This caused my kernel to crash when I enabled sparsemem since those
+pages (and page descriptors) didn't actually exist. I changed my code by adding
+lines like below:
+
+
+	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
+		struct page *page = pfn_to_page(tmp);
+
+   +		if (!pfn_valid(tmp))
+   +			continue;
+   +
+		if (!page_is_ram(tmp)) {
+			SetPageReserved(page);
+			continue;
+		}
+		ClearPageReserved(page);
+		init_page_count(page);
+		__free_page(page);
+		physmem_record(PFN_PHYS(tmp), PAGE_SIZE, physmem_highmem);
+		totalhigh_pages++;
+	}
+
+
+Once I got that straight, it worked!!!! I saved 10MiB of memory.  
+
+
+
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index c6a063b..5b1af87 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -408,7 +408,6 @@ static void __init bootmem_init(void)
 
 		/* Register lowmem ranges */
 		free_bootmem(PFN_PHYS(start), size << PAGE_SHIFT);
-		memory_present(0, start, end);
 	}
 
 	/*
@@ -420,6 +419,23 @@ static void __init bootmem_init(void)
 	 * Reserve initrd memory if needed.
 	 */
 	finalize_initrd();
+
+	/* call memory present for all the ram */
+	for (i = 0; i < boot_mem_map.nr_map; i++) {
+		unsigned long start, end;
+
+		/*
+		 * memory present only usable memory.
+		 */
+		if (boot_mem_map.map[i].type != BOOT_MEM_RAM)
+			continue;
+
+		start = PFN_UP(boot_mem_map.map[i].addr);
+		end   = PFN_DOWN(boot_mem_map.map[i].addr
+				    + boot_mem_map.map[i].size);
+
+		memory_present(0, start, end);
+	}
 }
 
 #endif	/* CONFIG_SGI_IP27 */
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 137c14b..31496a1 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -414,6 +414,9 @@ void __init mem_init(void)
 	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
 		struct page *page = pfn_to_page(tmp);
 
+		if (!pfn_valid(tmp))
+			continue;
+
 		if (!page_is_ram(tmp)) {
 			SetPageReserved(page);
 			continue;
diff --git a/include/asm-mips/sparsemem.h b/include/asm-mips/sparsemem.h
index 795ac6c..9faaf59 100644
--- a/include/asm-mips/sparsemem.h
+++ b/include/asm-mips/sparsemem.h
@@ -6,8 +6,14 @@
  * SECTION_SIZE_BITS		2^N: how big each section will be
  * MAX_PHYSMEM_BITS		2^N: how much memory we can have in that space
  */
+
+#ifndef CONFIG_64BIT
+#define SECTION_SIZE_BITS       27	/* 128 MiB */
+#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
+#else
 #define SECTION_SIZE_BITS       28
 #define MAX_PHYSMEM_BITS        35
+#endif
 
 #endif /* CONFIG_SPARSEMEM */
 #endif /* _MIPS_SPARSEMEM_H */

Re: sparsemem support for mips with highmem

[Andy Whitcroft]

On Thu, Aug 14, 2008 at 04:52:34PM -0700, C Michael Sundius wrote:
> fixed patch
>
>

Typically I was on holiday when you posted, how does that always happen.

> diff --git a/Documentation/sparsemem.txt b/Documentation/sparsemem.txt
> new file mode 100644
> index 0000000..6aea0d1
> --- /dev/null
> +++ b/Documentation/sparsemem.txt
> @@ -0,0 +1,93 @@
> +Sparsemem divides up physical memory in your system into N section of M
> +bytes. Page descriptors are created for only those sections that
> +actually exist (as far as the sparsemem code is concerned). This allows
> +for holes in the physical memory without having to waste space by
> +creating page discriptors for those pages that do not exist.
> +When page_to_pfn() or pfn_to_page() are called there is a bit of overhead to
> +look up the proper memory section to get to the descriptors, but this
> +is small compared to the memory you are likely to save. So, it's not the
> +default, but should be used if you have big holes in physical memory.
> +
> +Note that discontiguous memory is more closely related to NUMA machines
> +and if you are a single CPU system use sparsemem and not discontig. 
> +It's much simpler. 
> +
> +1) CALL MEMORY_PRESENT()
> +Once the bootmem allocator is up and running, you should call the
> +sparsemem function "memory_present(node, pfn_start, pfn_end)" for each
> +block of memory that exists on your system.
> +
> +2) DETERMINE AND SET THE SIZE OF SECTIONS AND PHYSMEM
> +The size of N and M above depend upon your architecture
> +and your platform and are specified in the file:
> +
> +      include/asm-<your_arch>/sparsemem.h
> +
> +and you should create the following lines similar to below: 
> +
> +	#ifdef CONFIG_YOUR_PLATFORM
> +	 #define SECTION_SIZE_BITS       27	/* 128 MiB */
> +	#endif
> +	#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */

This example is slightly out of step with the reality of what you add.
I would have expected the two defines to cary together?

> +
> +if they don't already exist, where: 
> +
> + * SECTION_SIZE_BITS            2^M: how big each section will be
> + * MAX_PHYSMEM_BITS             2^N: how much memory we can have in that
> +                                     space
> +
> +3) INITIALIZE SPARSE MEMORY
> +You should make sure that you initialize the sparse memory code by calling 
> +
> +	bootmem_init();
> +  +	sparse_init();
> +	paging_init();
> +
> +just before you call paging_init() and after the bootmem_allocator is
> +turned on in your setup_arch() code.  
> +
> +4) ENABLE SPARSEMEM IN KCONFIG
> +Add a line like this:
> +
> +	select ARCH_SPARSEMEM_ENABLE
> +
> +into the config for your platform in arch/<your_arch>/Kconfig. This will
> +ensure that turning on sparsemem is enabled for your platform. 

One other thing to to worry about here is turning any of the _ENABLEs
on tends to turn off the default models; particularly FLATMEM tends to
turn off if you don't explicitly ask for it.  So you may also need to
add entries for all of your models if none are already specified.

> +
> +5) CONFIG
> +Run make menuconfig or make gconfig, as you like, and turn on the sparsemem
> +memory model under the "Kernel Type" --> "Memory Model" and then build your
> +kernel.
> +
> +
> +6) Gotchas
> +
> +One trick that I encountered when I was turning this on for MIPS was that there
> +was some code in mem_init() that set the "reserved" flag for pages that were not
> +valid RAM. This caused my kernel to crash when I enabled sparsemem since those
> +pages (and page descriptors) didn't actually exist. I changed my code by adding
> +lines like below:
> +
> +
> +	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
> +		struct page *page = pfn_to_page(tmp);
> +
> +   +		if (!pfn_valid(tmp))
> +   +			continue;
> +   +
> +		if (!page_is_ram(tmp)) {
> +			SetPageReserved(page);
> +			continue;
> +		}
> +		ClearPageReserved(page);
> +		init_page_count(page);
> +		__free_page(page);
> +		physmem_record(PFN_PHYS(tmp), PAGE_SIZE, physmem_highmem);
> +		totalhigh_pages++;
> +	}
> +
> +
> +Once I got that straight, it worked!!!! I saved 10MiB of memory.  

That documentation is good whether the mips part is merged or not.  It
is probabally worth making it a separate patch.

> +
> +
> +
> diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
> index c6a063b..5b1af87 100644
> --- a/arch/mips/kernel/setup.c
> +++ b/arch/mips/kernel/setup.c
> @@ -408,7 +408,6 @@ static void __init bootmem_init(void)
>  
>  		/* Register lowmem ranges */
>  		free_bootmem(PFN_PHYS(start), size << PAGE_SHIFT);
> -		memory_present(0, start, end);
>  	}
>  
>  	/*
> @@ -420,6 +419,23 @@ static void __init bootmem_init(void)
>  	 * Reserve initrd memory if needed.
>  	 */
>  	finalize_initrd();
> +
> +	/* call memory present for all the ram */
> +	for (i = 0; i < boot_mem_map.nr_map; i++) {
> +		unsigned long start, end;
> +
> +		/*
> +		 * memory present only usable memory.
> +		 */
> +		if (boot_mem_map.map[i].type != BOOT_MEM_RAM)
> +			continue;
> +
> +		start = PFN_UP(boot_mem_map.map[i].addr);
> +		end   = PFN_DOWN(boot_mem_map.map[i].addr
> +				    + boot_mem_map.map[i].size);
> +
> +		memory_present(0, start, end);
> +	}
>  }
>  
>  #endif	/* CONFIG_SGI_IP27 */
> diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
> index 137c14b..31496a1 100644
> --- a/arch/mips/mm/init.c
> +++ b/arch/mips/mm/init.c
> @@ -414,6 +414,9 @@ void __init mem_init(void)
>  	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
>  		struct page *page = pfn_to_page(tmp);
>  
> +		if (!pfn_valid(tmp))
> +			continue;
> +
>  		if (!page_is_ram(tmp)) {
>  			SetPageReserved(page);
>  			continue;
> diff --git a/include/asm-mips/sparsemem.h b/include/asm-mips/sparsemem.h
> index 795ac6c..9faaf59 100644
> --- a/include/asm-mips/sparsemem.h
> +++ b/include/asm-mips/sparsemem.h
> @@ -6,8 +6,14 @@
>   * SECTION_SIZE_BITS		2^N: how big each section will be
>   * MAX_PHYSMEM_BITS		2^N: how much memory we can have in that space
>   */
> +
> +#ifndef CONFIG_64BIT
> +#define SECTION_SIZE_BITS       27	/* 128 MiB */
> +#define MAX_PHYSMEM_BITS        31	/* 2 GiB   */
> +#else
>  #define SECTION_SIZE_BITS       28
>  #define MAX_PHYSMEM_BITS        35
> +#endif
>  
>  #endif /* CONFIG_SPARSEMEM */
>  #endif /* _MIPS_SPARSEMEM_H */

Otherwise it looks good to me.  I see from the rest of the thread that
there is some discussion over the sizes of these, with that sorted.

Acked-by: Andy Whitcroft <apw@shadowen.org>

-apw

Re: Have ever checked in your mips sparsemem code into mips-linux tree?

[C Michael Sundius]

[-- Attachment #1: Type: text/plain, Size: 2001 bytes --]

John wrote:


> Hi Michael,
>
> After I read this link, noticed that you have the following patch, but when I check up the mips-linux, the patch is not there.
>
> I wonder if you could explain to me a little bit?
>
> Thank you!
>
> John
> P.S.: I also worked at SciAtl a few years ago in IPTV division.
>   
John,

I *think* I got tentative signoff from Dave and Any below as per the 
copied snipits below.
I made the modifications that they suggested. please see the attached 
for two patches:
a) the code
b) the sparsemem.txt doc

not sure if the mips powers that be were ok w/ it. pardon my ignorance, 
not sure if I am
required to do anymore. There was some comment to try this out w/ the 
CONFIG_SPARSEMEM_VMEMMAP
which I believe should "just work", but we've never tried it as of yet, 
so by my rule I can't
say it is so.. (has anyone tried that?)

Mike

====================================================


Dave Hansen wrote:

Looks great to me.  I can't test it, of course, but I don't see any
problems with it.

Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>

-- Dave


Andy Whitcroft wrote:
>
>
> Otherwise it looks good to me.  I see from the rest of the thread that
> there is some discussion over the sizes of these, with that sorted.
>
> Acked-by: Andy Whitcroft <apw@shadowen.org>
>
> -apw
>   
adding patch 1 containing code only:





     - - - - -                              Cisco                            - - - - -         
This e-mail and any attachments may contain information which is confidential, 
proprietary, privileged or otherwise protected by law. The information is solely 
intended for the named addressee (or a person responsible for delivering it to 
the addressee). If you are not the intended recipient of this message, you are 
not authorized to read, print, retain, copy or disseminate this message or any 
part of it. If you have received this e-mail in error, please notify the sender 
immediately by return e-mail and delete it from your computer.