[patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[David Mosberger]

$SUBJECT says it all...

	--david

Signed-off-by: David Mosberger-Tang <davidm@hpl.hp.com>

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2005/01/18 17:46:04-08:00 davidm@tiger.hpl.hp.com 
#   ia64: Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.
#   	Drops normal getpid() from 275 down to 271 cycles.
# 
# arch/ia64/kernel/entry.S
#   2005/01/18 17:45:52-08:00 davidm@tiger.hpl.hp.com +26 -16
#   (skip_rbs_switch): Reschedule to run 4 cycles faster on McKinley-type cores
#   	(drops normal getpid() from 275 down to 271 cycles).
# 
diff -Nru a/arch/ia64/kernel/entry.S b/arch/ia64/kernel/entry.S
--- a/arch/ia64/kernel/entry.S	2005-01-18 20:58:43 -08:00
+++ b/arch/ia64/kernel/entry.S	2005-01-18 20:58:43 -08:00
@@ -1032,23 +1032,33 @@
 	loadrs
 	;;
 skip_rbs_switch:
-(pLvSys)	mov r19=r0		// clear r19 for leave_syscall, no-op otherwise
-	mov b0=r21
-	mov ar.pfs=r26
-(pUStk)	mov ar.bspstore=r23
-(p9)	mov cr.ifs=r30
-(pLvSys)mov r16=r0		// clear r16 for leave_syscall, no-op otherwise
-	mov cr.ipsr=r29
-	mov ar.fpsr=r20
-(pLvSys)mov r17=r0		// clear r17 for leave_syscall, no-op otherwise
-	mov cr.iip=r28
+	mov ar.unat=r25		// M2
+	nop.i 0			// I0
+(pLvSys)mov r19=r0		// A  clear r19 for leave_syscall, no-op otherwise
 	;;
-(pUStk)	mov ar.rnat=r24		// must happen with RSE in lazy mode
-(pLvSys)mov r18=r0		// clear r18 for leave_syscall, no-op otherwise
-	mov ar.rsc=r27
-	mov ar.unat=r25
-	mov pr=r31,-1
-	rfi
+(pUStk)	mov ar.bspstore=r23	// M2
+	nop.i 0			// I0
+(pLvSys)mov r16=r0		// A  clear r16 for leave_syscall, no-op otherwise
+	;;
+	mov cr.ipsr=r29		// M2
+	mov ar.pfs=r26		// I0
+(pLvSys)mov r17=r0		// A  clear r17 for leave_syscall, no-op otherwise
+
+(p9)	mov cr.ifs=r30		// M2
+	mov b0=r21		// I0
+(pLvSys)mov r18=r0		// A  clear r18 for leave_syscall, no-op otherwise
+
+	mov ar.fpsr=r20		// M2
+	mov cr.iip=r28		// M2
+	nop 0
+	;;
+(pUStk)	mov ar.rnat=r24		// M2 must happen with RSE in lazy mode
+	nop 0
+	nop 0
+
+	mov ar.rsc=r27		// M2
+	mov pr=r31,-1		// I0
+	rfi			// B
 
 	/*
 	 * On entry:

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[Keith Owens]

Damn, I was in the middle of developing a patch against this area :(.

David, since you are redoing some of this code, I probably need to
discuss my changes.  My aim is to allow the kernel unwinder to work on
the per-cpu MCA and INIT stacks that Russ Anderson (rja) recently
added.  This has never been an issue before because we only had one
stack and could not handle INIT over MCA.

The problem with unwinding from the MCA or INIT stacks is handling the
change of rbs.  The current code assumes that all kernel work is done
on the same rbs, which is not true for the MCA and INIT stacks.  I plan
to change the meaning of pt_regs ar_bspstore and ar_rnat.  Instead of
being garbage when the interrupted code was in the kernel, ar_bspstore
and ar_rnat are set to zero.  The test for "do I skip_rbs_switch"
changes from pUStk to "is saved ar_bspstore non-zero".

Interrupts in kernel context save a zero ar_bspstore and skip the rbs
swicth on return from interrupt.  Interrupts in user context save a
valid ar_bspstore and switch rbs on exit.  MCA and INIT events will
synthesize and save struct pt_regs on the MCA/INIT stack, this pt_regs
will contain non-zero ar_bspstore pointing back to the previous bsp,
which could be in a kernel or user space stack.

Comments?

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[David Mosberger]

>>>>> On Thu, 20 Jan 2005 09:25:35 +1100, Keith Owens <kaos@sgi.com> said:

  Keith> Damn, I was in the middle of developing a patch against this
  Keith> area :(.

Oops.

  Keith> Instead of being garbage when the interrupted code was in the
  Keith> kernel, ar_bspstore and ar_rnat are set to zero.  The test
  Keith> for "do I skip_rbs_switch" changes from pUStk to "is saved
  Keith> ar_bspstore non-zero".

Wouldn't this prevent user-level code from setting ar.bspstore to
zero?  I'm not sure we should nail that into the kernel (by default
the first page is a NaT page, so it's not likely to be a real problem
as of today).

  Keith> Comments?

Libunwind does support multiple register backing-stores, but since we
cannot unwind across firmware, I'm not sure whether this by itself
would make your life easier.  If it would, let me know.

	--david

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[Keith Owens]

On Wed, 19 Jan 2005 22:50:07 -0800, 
David Mosberger <davidm@napali.hpl.hp.com> wrote:
>>>>>> On Thu, 20 Jan 2005 09:25:35 +1100, Keith Owens <kaos@sgi.com> said:
>
>  Keith> Instead of being garbage when the interrupted code was in the
>  Keith> kernel, ar_bspstore and ar_rnat are set to zero.  The test
>  Keith> for "do I skip_rbs_switch" changes from pUStk to "is saved
>  Keith> ar_bspstore non-zero".
>
>Wouldn't this prevent user-level code from setting ar.bspstore to
>zero?  I'm not sure we should nail that into the kernel (by default
>the first page is a NaT page, so it's not likely to be a real problem
>as of today).

Good point, use ar_bspstore = 1 to mean no rbs switch required.

>  Keith> Comments?
>
>Libunwind does support multiple register backing-stores, but since we
>cannot unwind across firmware, I'm not sure whether this by itself
>would make your life easier.  If it would, let me know.

It would help, if libunwind were in the kernel.

Unwinding across firmware is not an issue here, we have two views of
the MCA/INIT context.  One view is "how to return to SAL?", that data
is stored outside the stack.  The other view is "what was the cpu doing
when the event occurred?".  It is the latter that I am working on.

From the point of view of unwind, I want the MCA/INIT to look like a
normal interrupt, with the addition of separate backing store.  That
will let us backtrace during MCA/INIT processing to see what the cpu
was really doing.  SAL will not appear in the backtrace at all.

One side effect of this change is that the unwinder needs an MCA/INIT
safe allocator.  The existing code uses kmalloc which works most of the
time, but some MCA/INIT traces have been killed by the unwinder calling
kmalloc when the slab structures were inconsistent.  I recently added
an MCA/INIT safe, last ditch allocator to kdb, so we can get
diagnostics even when GFP_ATOMIC has been exhausted and in MCA/INIT.

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[David Mosberger]

>>>>> On Thu, 20 Jan 2005 21:26:18 +1100, Keith Owens <kaos@sgi.com> said:

  Keith> It would help, if libunwind were in the kernel.

OK, it does sound like I need to get going with this.  Fortunately,
it's nearing the top of my to-do-list.  I'll see if I can get this
going next week.

  Keith> One side effect of this change is that the unwinder needs an
  Keith> MCA/INIT safe allocator.

Libunwind already has that.  Need to double-check that it's MCA/INIT safe,
but if it isn't already, it should be real close.

	--david

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[David Mosberger]

>>>>> On Thu, 20 Jan 2005 21:26:18 +1100, Keith Owens <kaos@sgi.com> said:

  Keith> Unwinding across firmware is not an issue here, we have two
  Keith> views of the MCA/INIT context.  One view is "how to return to
  Keith> SAL?", that data is stored outside the stack.  The other view
  Keith> is "what was the cpu doing when the event occurred?".  It is
  Keith> the latter that I am working on.

How is this different from the "unw_init_from_interruption" call that
we're doing in mca.c:init_handler_platform()?

	--david

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[Keith Owens]

On Thu, 27 Jan 2005 18:08:18 -0800, 
David Mosberger <davidm@napali.hpl.hp.com> wrote:
>>>>>> On Thu, 20 Jan 2005 21:26:18 +1100, Keith Owens <kaos@sgi.com> said:
>
>  Keith> Unwinding across firmware is not an issue here, we have two
>  Keith> views of the MCA/INIT context.  One view is "how to return to
>  Keith> SAL?", that data is stored outside the stack.  The other view
>  Keith> is "what was the cpu doing when the event occurred?".  It is
>  Keith> the latter that I am working on.
>
>How is this different from the "unw_init_from_interruption" call that
>we're doing in mca.c:init_handler_platform()?

That only works for the INIT call which uses the current bspstore, MCA
uses its own bspstore.  The kernel unwinder is not set up to switch
bspstore between two kernel stacks, only between kernel and user space.
The INIT call will change to its own bspstore once we use the separate
INIT and MCA stacks for each cpu.

Re: [patch] Resched skip_rbs_switch to run 4 cycles faster on McKinley-type cores.

[David Mosberger]

>>>>> On Fri, 28 Jan 2005 14:04:35 +1100, Keith Owens <kaos@sgi.com> said:

  >>  How is this different from the "unw_init_from_interruption" call
  >> that we're doing in mca.c:init_handler_platform()?

  Keith> That only works for the INIT call which uses the current
  Keith> bspstore, MCA uses its own bspstore.

Ah, OK, I understand now what you're after.

Thanks,

	--david

PS: For what it's woth, I did start work on a libunwind-enabled kernel.