[Linux-ia64] Pre-emption patch for IA-64

[Linux-ia64] Pre-emption patch for IA-64

[Mario Smarduch]

Hi,
    we're trying to build a reliable low-latency system for use
in telco call processing carrier grade environments (on rx2600
NEBS like servers) . One of the patches we're heavily relying on
is the 'Kernel Preemption' patch. First of we're interested in the
(i) status of this patch on  IA-64 (ii) any issues that you may
have ran into.

Our understanding is that the 'Preemption Patch' offers a systematic
scheduling checkpoint vs. 'low latency patch' which examines
the kernel for any long running code  and based on that
preemption checks are inserted in the kernel proper only (kernel.org)
but addon packages may introduce long latencies. While we plan to
run with 1mS clock tick, low latency  and the preemption
paches, the preemption patch is viewed as a must to gurantee
a bound on 'soft-realtime response'.  Please point out if these
assumption are somewhat correct or is too much emphasis
being placed on the preemption patch, the soft-realtime response
we're looking for is 1mS.

thanks,
    Mario.

Re: [Linux-ia64] Pre-emption patch for IA-64

[David Mosberger]

Peter Chubb has been working on adding preemption support to 2.5.  I
applied his patch in the 2.5.64 patch.  I'm sure he would appreciate
more people beating on it.

	--david

>>>>> On Fri, 14 Mar 2003 11:21:21 -0600, Mario Smarduch <cms063@email.mot.com> said:

  Mario> Hi, we're trying to build a reliable low-latency system for
  Mario> use in telco call processing carrier grade environments (on
  Mario> rx2600 NEBS like servers) . One of the patches we're heavily
  Mario> relying on is the 'Kernel Preemption' patch. First of we're
  Mario> interested in the (i) status of this patch on IA-64 (ii) any
  Mario> issues that you may have ran into.

  Mario> Our understanding is that the 'Preemption Patch' offers a
  Mario> systematic scheduling checkpoint vs. 'low latency patch'
  Mario> which examines the kernel for any long running code and based
  Mario> on that preemption checks are inserted in the kernel proper
  Mario> only (kernel.org) but addon packages may introduce long
  Mario> latencies. While we plan to run with 1mS clock tick, low
  Mario> latency and the preemption paches, the preemption patch is
  Mario> viewed as a must to gurantee a bound on 'soft-realtime
  Mario> response'.  Please point out if these assumption are somewhat
  Mario> correct or is too much emphasis being placed on the
  Mario> preemption patch, the soft-realtime response we're looking
  Mario> for is 1mS.

Re: [Linux-ia64] Pre-emption patch for IA-64

[Jesse Barnes]

Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok
with the following exceptions:
  o the AT keyboard support seems broken (characters are slow to show
    up on the screen and are sometimes repeated)
  o agp support for 460GX causes a link failure

But by using a USB keyboard and turning off agp support, I was able to
get an SMP+preempt kernel going, and it seems fine.

Jesse

On Fri, Mar 14, 2003 at 01:24:44PM -0800, David Mosberger wrote:
> Peter Chubb has been working on adding preemption support to 2.5.  I
> applied his patch in the 2.5.64 patch.  I'm sure he would appreciate
> more people beating on it.
> 
> 	--david
> 
> >>>>> On Fri, 14 Mar 2003 11:21:21 -0600, Mario Smarduch <cms063@email.mot.com> said:
> 
>   Mario> Hi, we're trying to build a reliable low-latency system for
>   Mario> use in telco call processing carrier grade environments (on
>   Mario> rx2600 NEBS like servers) . One of the patches we're heavily
>   Mario> relying on is the 'Kernel Preemption' patch. First of we're
>   Mario> interested in the (i) status of this patch on IA-64 (ii) any
>   Mario> issues that you may have ran into.
> 
>   Mario> Our understanding is that the 'Preemption Patch' offers a
>   Mario> systematic scheduling checkpoint vs. 'low latency patch'
>   Mario> which examines the kernel for any long running code and based
>   Mario> on that preemption checks are inserted in the kernel proper
>   Mario> only (kernel.org) but addon packages may introduce long
>   Mario> latencies. While we plan to run with 1mS clock tick, low
>   Mario> latency and the preemption paches, the preemption patch is
>   Mario> viewed as a must to gurantee a bound on 'soft-realtime
>   Mario> response'.  Please point out if these assumption are somewhat
>   Mario> correct or is too much emphasis being placed on the
>   Mario> preemption patch, the soft-realtime response we're looking
>   Mario> for is 1mS.
> 
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

Re: [Linux-ia64] Pre-emption patch for IA-64

[Bjorn Helgaas]

On Friday 14 March 2003 2:32 pm, Jesse Barnes wrote:
> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok
> with the following exceptions:
>   o the AT keyboard support seems broken (characters are slow to show
>     up on the screen and are sometimes repeated)
>   o agp support for 460GX causes a link failure

2.4 had a problem that seems to be a compiler issue with some AGP
code.  I don't know whether this is the same problem, but if it is, I
applied the following patch as a workaround:

diff -Nru a/drivers/char/agp/agpgart_be.c b/drivers/char/agp/agpgart_be.c
--- a/drivers/char/agp/agpgart_be.c	Fri Mar 14 14:22:45 2003
+++ b/drivers/char/agp/agpgart_be.c	Fri Mar 14 14:22:45 2003
@@ -341,26 +341,18 @@
 
 	temp = agp_bridge.current_size;
 
-	switch (agp_bridge.size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge.size_type = U8_APER_SIZE)
 		current_size = A_SIZE_8(temp)->size;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge.size_type =  U16_APER_SIZE)
 		current_size = A_SIZE_16(temp)->size;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge.size_type =  U32_APER_SIZE)
 		current_size = A_SIZE_32(temp)->size;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge.size_type =  LVL2_APER_SIZE)
 		current_size = A_SIZE_LVL2(temp)->size;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge.size_type =  FIXED_APER_SIZE)
 		current_size = A_SIZE_FIX(temp)->size;
-		break;
-	default:
+	else
 		current_size = 0;
-		break;
-	}
 
 	current_size -= (agp_memory_reserved / (1024*1024));
 

Re: [Linux-ia64] Pre-emption patch for IA-64

[Peter Chubb]

>>>>> "Jesse" = Jesse Barnes <jbarnes@sgi.com> writes:

Jesse> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok
Jesse> with the following exceptions:
Jesse> o the AT keyboard support seems broken (characters are slow to show
Jesse> up on the screen and are sometimes repeated)
Jesse> o agp support for 460GX causes a link failure

AT keyboard and PS/2 mouse both!

Jesse> But by using a USB keyboard and turning off agp support, I was able to
Jesse> get an SMP+preempt kernel going, and it seems fine.


On a dual McKinley prototype machine I was able to get lockups with
preempt under heavy I/O and cpu load (e.g., make -j4 in glibc).  I'm
waiting for production model McKinleys SMP to retry, as I couldn't
reproduce the problem on production-model Itanium 1, and there were no
symptoms that I could debug.

Also FYI, if you're using posix threads in userland, the port of the
next generation library is almost complete, and, with the kernel
preemption patch, will give much lower latencies for heavily threaded
code, depending on exactly what you're doing.

--
Dr Peter Chubb				    peterc@gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories, all almost the same.

Re: [Linux-ia64] Pre-emption patch for IA-64

[David Mosberger]

>>>>> On Sat, 15 Mar 2003 08:57:55 +1100, Peter Chubb <peter@chubb.wattle.id.au> said:

  Peter> On a dual McKinley prototype machine I was able to get
  Peter> lockups with preempt under heavy I/O and cpu load (e.g., make
  Peter> -j4 in glibc).  I'm waiting for production model McKinleys
  Peter> SMP to retry, as I couldn't reproduce the problem on
  Peter> production-model Itanium 1, and there were no symptoms that I
  Peter> could debug.

Do you know what on the zx1-based machines you can trigger INIT via
the BMC?  That's quite handy when the kernel locks up (you can dump
the collected state at the next boot).

	--david

Re: [Linux-ia64] Pre-emption patch for IA-64

[Jesse Barnes]

On Fri, Mar 14, 2003 at 02:38:55PM -0700, Bjorn Helgaas wrote:
> On Friday 14 March 2003 2:32 pm, Jesse Barnes wrote:
> > Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok
> > with the following exceptions:
> >   o the AT keyboard support seems broken (characters are slow to show
> >     up on the screen and are sometimes repeated)
> >   o agp support for 460GX causes a link failure
> 
> 2.4 had a problem that seems to be a compiler issue with some AGP
> code.  I don't know whether this is the same problem, but if it is, I
> applied the following patch as a workaround:

Thanks for the pointer.  Here's a patch against 2.5.64.  It builds and
boots, but my XFree dri/glx/whatever drivers seem broken, so I'm not
sure if it's really working like it should.

Btw Peter, thanks for doing the preemption patch.  No hangs yet for
me...

Jesse


diff -Naur -X /usr/people/jbarnes/dontdiff linux-2.5.64-ia64-stock/drivers/char/agp/generic.c linux-2.5.64-ia64/drivers/char/agp/generic.c
--- linux-2.5.64-ia64-stock/drivers/char/agp/generic.c	2003-03-04 19:29:15.000000000 -0800
+++ linux-2.5.64-ia64/drivers/char/agp/generic.c	2003-03-14 14:09:03.000000000 -0800
@@ -34,6 +34,7 @@
 #include <linux/miscdevice.h>
 #include <linux/pm.h>
 #include <linux/agp_backend.h>
+#include <linux/vmalloc.h>
 #include "agp.h"
 
 __u32 *agp_gatt_table; 
@@ -169,26 +170,18 @@
 
 	temp = agp_bridge->current_size;
 
-	switch (agp_bridge->size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge->size_type = U8_APER_SIZE)
 		current_size = A_SIZE_8(temp)->size;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge->size_type = U16_APER_SIZE)
 		current_size = A_SIZE_16(temp)->size;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge->size_type = U32_APER_SIZE)
 		current_size = A_SIZE_32(temp)->size;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge->size_type = LVL2_APER_SIZE)
 		current_size = A_SIZE_LVL2(temp)->size;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge->size_type = FIXED_APER_SIZE)
 		current_size = A_SIZE_FIX(temp)->size;
-		break;
-	default:
+	else
 		current_size = 0;
-		break;
-	}
 
 	current_size -= (agp_memory_reserved / (1024*1024));
 	if (current_size <0)
@@ -203,26 +196,18 @@
 
 	temp = agp_bridge->current_size;
 
-	switch (agp_bridge->size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge->size_type = U8_APER_SIZE)
 		num_entries = A_SIZE_8(temp)->num_entries;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge->size_type = U16_APER_SIZE)
 		num_entries = A_SIZE_16(temp)->num_entries;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge->size_type = U32_APER_SIZE)
 		num_entries = A_SIZE_32(temp)->num_entries;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge->size_type = LVL2_APER_SIZE)
 		num_entries = A_SIZE_LVL2(temp)->num_entries;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge->size_type = FIXED_APER_SIZE)
 		num_entries = A_SIZE_FIX(temp)->num_entries;
-		break;
-	default:
+	else
 		num_entries = 0;
-		break;
-	}
 
 	num_entries -= agp_memory_reserved>>PAGE_SHIFT;
 	if (num_entries<0)
@@ -546,27 +531,19 @@
 
 	temp = agp_bridge->current_size;
 
-	switch (agp_bridge->size_type) {
-	case U8_APER_SIZE:
+	if (agp_bridge->size_type = U8_APER_SIZE)
 		page_order = A_SIZE_8(temp)->page_order;
-		break;
-	case U16_APER_SIZE:
+	else if (agp_bridge->size_type = U16_APER_SIZE)
 		page_order = A_SIZE_16(temp)->page_order;
-		break;
-	case U32_APER_SIZE:
+	else if (agp_bridge->size_type = U32_APER_SIZE)
 		page_order = A_SIZE_32(temp)->page_order;
-		break;
-	case FIXED_APER_SIZE:
+	else if (agp_bridge->size_type = FIXED_APER_SIZE)
 		page_order = A_SIZE_FIX(temp)->page_order;
-		break;
-	case LVL2_APER_SIZE:
+	else if (agp_bridge->size_type = LVL2_APER_SIZE)
 		/* The generic routines can't deal with 2 level gatt's */
 		return -EINVAL;
-		break;
-	default:
+	else
 		page_order = 0;
-		break;
-	}
 
 	/* Do not worry about freeing memory, because if this is
 	 * called, then all agp memory is deallocated and removed

Re: [Linux-ia64] Pre-emption patch for IA-64

[jim]

>2.4 had a problem that seems to be a compiler issue with some AGP
>code.  I don't know whether this is the same problem, but if it is, I
>applied the following patch as a workaround:

There was a patch from Jakub Jelinek last week to address this problem.
 I don't know if it has been approved or checked in yet, but this should
be fixed soon.

    http://gcc.gnu.org/ml/gcc-patches/2003-03/msg00823.html

Jim

Re: [Linux-ia64] Pre-emption patch for IA-64

[Mario Smarduch]

So it appears like drivers are having issues and that's
to be expected if they're not reentrant. But what about
the kernel proper, has anyone observed issues there or
knows of any outstanding issues?

    - Mario.


Peter Chubb wrote:

> >>>>> "Jesse" = Jesse Barnes <jbarnes@sgi.com> writes:
>
> Jesse> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok
> Jesse> with the following exceptions:
> Jesse> o the AT keyboard support seems broken (characters are slow to show
> Jesse> up on the screen and are sometimes repeated)
> Jesse> o agp support for 460GX causes a link failure
>
> AT keyboard and PS/2 mouse both!
>
> Jesse> But by using a USB keyboard and turning off agp support, I was able to
> Jesse> get an SMP+preempt kernel going, and it seems fine.
>
> On a dual McKinley prototype machine I was able to get lockups with
> preempt under heavy I/O and cpu load (e.g., make -j4 in glibc).  I'm
> waiting for production model McKinleys SMP to retry, as I couldn't
> reproduce the problem on production-model Itanium 1, and there were no
> symptoms that I could debug.
>
> Also FYI, if you're using posix threads in userland, the port of the
> next generation library is almost complete, and, with the kernel
> preemption patch, will give much lower latencies for heavily threaded
> code, depending on exactly what you're doing.
>
> --
> Dr Peter Chubb                              peterc@gelato.unsw.edu.au
> You are lost in a maze of BitKeeper repositories, all almost the same.
>
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

Re: [Linux-ia64] Pre-emption patch for IA-64

[Jesse Barnes]

My box is still running strong, so no kernel issues that I know of
(aside from the kernel eating all my CPU time whenever someone mounts
an exported nfs volume)

[jbarnes@spamtin jbarnes]$ uname -a;uptime
Linux spamtin 2.5.64 #7 SMP Fri Mar 14 14:10:36 PST 2003 ia64 unknown
 15:33:01  up 13 days, 51 min,  7 users,  load average: 0.70, 0.30, 0.14

Thanks,
Jesse


On Thu, Mar 27, 2003 at 05:31:15PM -0600, Mario Smarduch wrote:
> So it appears like drivers are having issues and that's
> to be expected if they're not reentrant. But what about
> the kernel proper, has anyone observed issues there or
> knows of any outstanding issues?
> 
>     - Mario.
> 
> 
> Peter Chubb wrote:
> 
> > >>>>> "Jesse" = Jesse Barnes <jbarnes@sgi.com> writes:
> >
> > Jesse> Just FYI, I tried 2.5.64 out on my Bug Sur, and it seems to work ok
> > Jesse> with the following exceptions:
> > Jesse> o the AT keyboard support seems broken (characters are slow to show
> > Jesse> up on the screen and are sometimes repeated)
> > Jesse> o agp support for 460GX causes a link failure
> >
> > AT keyboard and PS/2 mouse both!
> >
> > Jesse> But by using a USB keyboard and turning off agp support, I was able to
> > Jesse> get an SMP+preempt kernel going, and it seems fine.
> >
> > On a dual McKinley prototype machine I was able to get lockups with
> > preempt under heavy I/O and cpu load (e.g., make -j4 in glibc).  I'm
> > waiting for production model McKinleys SMP to retry, as I couldn't
> > reproduce the problem on production-model Itanium 1, and there were no
> > symptoms that I could debug.
> >
> > Also FYI, if you're using posix threads in userland, the port of the
> > next generation library is almost complete, and, with the kernel
> > preemption patch, will give much lower latencies for heavily threaded
> > code, depending on exactly what you're doing.
> >
> > --
> > Dr Peter Chubb                              peterc@gelato.unsw.edu.au
> > You are lost in a maze of BitKeeper repositories, all almost the same.
> >
> > _______________________________________________
> > Linux-IA64 mailing list
> > Linux-IA64@linuxia64.org
> > http://lists.linuxia64.org/lists/listinfo/linux-ia64

Re: [Linux-ia64] Pre-emption patch for IA-64

[Peter Chubb]

Mario> So it appears like drivers are having issues and that's to be
Mario> expected if they're not reentrant. But what about the kernel
Mario> proper, has anyone observed issues there or knows of any
Mario> outstanding issues?

There are no issues I'm aware of, even in drivers, with the preeption
patch.  The driver issues with the PS/2 mouse and AT keyboard seem
to be generic, not related to preemption support.

--
Dr Peter Chubb				    peterc@gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories, all almost the same.