[PATCH] Fix ppc64 max_pfn issue

[PATCH] Fix ppc64 max_pfn issue

[Anton Blanchard]

Hi,

I noticed excessive time in the pid hash functions on a ppc64 box. It
turns out the pid hash is being sized way too small, eg on a 16GB box:

PID hash table entries: 16 (order 4: 256 bytes)

The reason is that the pid hash init function uses max_pfn before it was
setup on ppc64. With the following patch things are good again:

PID hash table entries: 4096 (order 12: 65536 bytes)

Signed-off-by: Anton Blanchard <anton@samba.org>

diff -puN arch/ppc64/mm/init.c~fix_max_pfn arch/ppc64/mm/init.c
--- gr_work/arch/ppc64/mm/init.c~fix_max_pfn	2004-07-23 23:13:18.501206270 -0500
+++ gr_work-anton/arch/ppc64/mm/init.c	2004-07-23 23:22:05.691934376 -0500
@@ -555,6 +555,8 @@ void __init do_init_bootmem(void)
 	unsigned long total_pages = lmb_end_of_DRAM() >> PAGE_SHIFT;
 	int boot_mapsize;
 
+        max_pfn = max_low_pfn;
+
 	/*
 	 * Find an area to use for the bootmem bitmap.  Calculate the size of
 	 * bitmap required as (Total Memory) / PAGE_SIZE / BITS_PER_BYTE.
@@ -651,7 +653,6 @@ void __init mem_init(void)
 
 	num_physpages = max_low_pfn;	/* RAM is assumed contiguous */
 	high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
-	max_pfn = max_low_pfn;
 
 #ifdef CONFIG_DISCONTIGMEM
 {
diff -puN arch/ppc64/mm/numa.c~fix_max_pfn arch/ppc64/mm/numa.c
--- gr_work/arch/ppc64/mm/numa.c~fix_max_pfn	2004-07-23 23:16:29.491011782 -0500
+++ gr_work-anton/arch/ppc64/mm/numa.c	2004-07-23 23:20:47.684429879 -0500
@@ -429,6 +429,7 @@ void __init do_init_bootmem(void)
 
 	min_low_pfn = 0;
 	max_low_pfn = lmb_end_of_DRAM() >> PAGE_SHIFT;
+	max_pfn = max_low_pfn;
 
 	if (parse_numa_properties())
 		setup_nonnuma();

_

Re: [PATCH] Fix ppc64 max_pfn issue

[Chris Wedgwood]

On Sat, Jul 24, 2004 at 02:47:20PM +1000, Anton Blanchard wrote:

> I noticed excessive time in the pid hash functions on a ppc64
> box. It turns out the pid hash is being sized way too small, eg on a
> 16GB box:

This reminds me (that someone pointed out to me, I forget who) that in
pid.c we have:

 void __init pidhash_init(void)
 {
         int i, j, pidhash_size;
         unsigned long megabytes = max_pfn >> (20 - PAGE_SHIFT);

         pidhash_shift = max(4, fls(megabytes * 4));
         pidhash_shift = min(12, pidhash_shift);
         pidhash_size = 1 << pidhash_shift;

which isn't strictly correct for machines with sparse/discontiguous
memory as max_pfn may have very little bearing on the overall memory
size.

Since we cap it at 1<<12 I guess the platform affected might be ARM
where you would otherwise get a smaller hash size.  That said, I
wonder if a shift of 12 suffices for really large machines with many
many processes.


  --cw

Re: [PATCH] Fix ppc64 max_pfn issue

[Anton Blanchard]

 
> Since we cap it at 1<<12 I guess the platform affected might be ARM
> where you would otherwise get a smaller hash size.  That said, I
> wonder if a shift of 12 suffices for really large machines with many
> many processes.

Yeah it does seem low considering how easy it is to create 100,000
threads these days.

Anton

Re: [PATCH] Fix ppc64 max_pfn issue

[Anton Blanchard]

 
> I noticed excessive time in the pid hash functions on a ppc64 box. It
> turns out the pid hash is being sized way too small, eg on a 16GB box:

It turns out in the non NUMA case, max_low_pfn doesnt get initialised
until init_bootmem so we need to move initialisation of max_pfn below
it.

Signed-off-by: Anton Blanchard <anton@samba.org>

diff -u linux-2.5/arch/ppc64/mm/init.c foobar2/arch/ppc64/mm/init.c
--- linux-2.5/arch/ppc64/mm/init.c	2004-07-25 00:40:04.263577944 +1000
+++ foobar2/arch/ppc64/mm/init.c	2004-07-25 01:32:48.030807187 +1000
@@ -533,8 +533,6 @@
 	unsigned long total_pages = lmb_end_of_DRAM() >> PAGE_SHIFT;
 	int boot_mapsize;
 
-        max_pfn = max_low_pfn;
-
 	/*
 	 * Find an area to use for the bootmem bitmap.  Calculate the size of
 	 * bitmap required as (Total Memory) / PAGE_SIZE / BITS_PER_BYTE.
@@ -547,6 +545,8 @@
 
 	boot_mapsize = init_bootmem(start >> PAGE_SHIFT, total_pages);
 
+        max_pfn = max_low_pfn;
+
 	/* add all physical memory to the bootmem map. Also find the first */
 	for (i=0; i < lmb.memory.cnt; i++) {
 		unsigned long physbase, size;