[PATCH] migration cost tune up in sparc smp

[PATCH] migration cost tune up in sparc smp

[Krzysztof Helt]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

This patch sets the max_cache_size value required to tune up
scheduler in SMP systems. Otherwise, the calculated
migration_cost is too high and task scheduling may lock up.
Signed-off-by: Krzysztof Helt <krzysztof.h1@wp.pl>
---

This patch is for 2.6.17-rc5 kernel. This patch helps with Bob
Breuer's patch. Default cache size (20MB) leads to calculation of
the migration cost 20000 (20msecs).  The calculated cost of
migration is 4000 (4msecs) on dual SM81 with the patch.

The same trick is used with setting up max_cache_size  on i386
architecture. I assume the value should be set to on sparc64 as
well to correctly tune up the scheduler. 

Regards,
Krzysztof

----------------------------------------------------
Nowe serie superksiążek o Czarodziejkach WITCH!
Poznaj świat bohaterek WITCH! 
Will, Irma, Taranee, Cornelia i Hay-Lin zapraszają do księgarni.
http://klik.wp.pl/?adr=http%3A%2F%2Fadv.reklama.wp.pl%2Fas%2Fwitch.html&sid=781

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: sparc32-migration-cost.patch --]
[-- Type: text/x-patch; name="sparc32-migration-cost.patch", Size: 948 bytes --]

diff -urp linux-2.6.16/arch/sparc/kernel/smp.c linux-2.6.17/arch/sparc/kernel/smp.c
--- linux-2.6.16/arch/sparc/kernel/smp.c	2006-06-05 08:56:06.000000000 +0200
+++ linux-2.6.17/arch/sparc/kernel/smp.c	2006-06-05 09:15:28.000000000 +0200
@@ -69,6 +69,17 @@ void __init smp_store_cpu_info(int id)
 						     "clock-frequency", 0);
 	cpu_data(id).prom_node = cpu_node;
 	cpu_data(id).mid = cpu_get_hwmid(cpu_node);
+
+	/* this is required to tune the scheduler correctly */
+	/* is it possible to have CPUs with different cache sizes? */
+	if (id == boot_cpu_id) {
+		int cache_line,cache_nlines;
+		cache_line = 0x20;
+		cache_line = prom_getintdefault(cpu_node, "ecache-line-size", cache_line);
+		cache_nlines = 0x8000;
+		cache_nlines = prom_getintdefault(cpu_node, "ecache-nlines", cache_nlines);
+		max_cache_size = cache_line * cache_nlines;
+	}
 	if (cpu_data(id).mid < 0)
 		panic("No MID found for CPU%d at node 0x%08d", id, cpu_node);
 }

Re: [PATCH] migration cost tune up in sparc smp

[David Miller]

From: "Krzysztof Helt" <krzysztof.h1@wp.pl>
Date: Mon, 05 Jun 2006 09:21:15 +0200

> This patch sets the max_cache_size value required to tune up
> scheduler in SMP systems. Otherwise, the calculated
> migration_cost is too high and task scheduling may lock up.
> Signed-off-by: Krzysztof Helt <krzysztof.h1@wp.pl>

Thanks for this, yes sparc64 needs this too.

But why in the world does the scheduler "lock up" just because the
migration cost is too large?

It should continue to function properly even if the migration cost is
very large.

Re: [PATCH] migration cost tune up in sparc smp

[Bob Breuer]

David Miller wrote:
> But why in the world does the scheduler "lock up" just because the
> migration cost is too large?

There is an address space collision between the vmalloc area and where
the prom maps the framebuffer.

The vmalloc range is 0xfe600000 to 0xffc00000.  I have a cg6 that gets
mapped to 0xfee00000, and a cg14 with 8MB vsimm that gets mapped to
0xfe700000.  These leave only 8MB and 1MB of usable vmalloc space before
bad things happen.

We may want to find a different vm hole to put vmalloc into.  Is there
any documentation on what address spaces are reserved with the different
prom versions?

Bob

Re: [PATCH] migration cost tune up in sparc smp

[David Miller]

From: Bob Breuer <breuerr@mc.net>
Date: Sun, 11 Jun 2006 20:21:09 -0500

> David Miller wrote:
> > But why in the world does the scheduler "lock up" just because the
> > migration cost is too large?
> 
> There is an address space collision between the vmalloc area and where
> the prom maps the framebuffer.

And this is what locks up the scheduler when a bad migration cost
calculation is made?

Please start a new thread if you are talking about some new topic,
it gets confusing :-/

> The vmalloc range is 0xfe600000 to 0xffc00000.  I have a cg6 that gets
> mapped to 0xfee00000, and a cg14 with 8MB vsimm that gets mapped to
> 0xfe700000.  These leave only 8MB and 1MB of usable vmalloc space before
> bad things happen.
> 
> We may want to find a different vm hole to put vmalloc into.  Is there
> any documentation on what address spaces are reserved with the different
> prom versions?

I don't know, but from your report it seems that OBP takes
0xfe000000 up to 0xfff00000.

Actually, you can look at the "available" property of the
virtual-memory node.  Or, you can see if there is a "translations"
node in the /virtual-memory node like there is on sparc64.