[PATCH] Make sched_clock() work in early boot

[PATCH] Make sched_clock() work in early boot

[Jason Uhlenkott]

Turning on PRINTK_TIME causes us to die in early boot when sched_clock()
tries to access per-cpu data which isn't set up yet.

We can avoid this by doing a speculative load on the per-cpu data and
just returning zero if we fail.  This matches the way sched_clock()
appears to behave on most other arches if it gets called before we
know the cpu frequency.

Signed-off-by: Jason Uhlenkott <jasonuhl@sgi.com>

Index: linux/arch/ia64/kernel/head.S
=================================--- linux.orig/arch/ia64/kernel/head.S	2005-08-15 15:54:56.130313231 -0700
+++ linux/arch/ia64/kernel/head.S	2005-08-16 12:49:23.843747550 -0700
@@ -983,9 +983,10 @@
 	addl r8=THIS_CPU(cpu_info) + IA64_CPUINFO_NSEC_PER_CYC_OFFSET,r0
 	mov.m r9=ar.itc		// fetch cycle-counter				(35 cyc)
 	;;
-	ldf8 f8=[r8]
+	ldf8.s f8=[r8]
 	;;
 	setf.sig f9=r9		// certain to stall, so issue it _after_ ldf8...
+	chk.s r8, .recover
 	;;
 	xmpy.lu f10ù,f8	// calculate low 64 bits of 128-bit product	(4 cyc)
 	xmpy.hu f11ù,f8	// calculate high 64 bits of 128-bit product
@@ -995,6 +996,10 @@
 	;;
 	shrp r8=r9,r8,IA64_NSEC_PER_CYC_SHIFT
 	br.ret.sptk.many rp
+.recover:			// per-cpu data isn't set up yet,
+				// so just return 0.
+	mov r8=r0
+	br.ret.sptk.many rp
 END(sched_clock)
 
 GLOBAL_ENTRY(start_kernel_thread)

Re: [PATCH] Make sched_clock() work in early boot

[david mosberger]

I'm somewhat nervous about *requiring* that the per-CPU page be pinned
in the TLB.  Sure, that's how it works today, but if it ever changed,
it would lead to subtle sporadic failures in sched_clock().  Also, I
believe that with your patch applied, sched_clock() would fail (return
0) if a data-debug breakpoint was set on the area that it's reading
and debug-faults are deferred.

  --david

On 8/16/05, Jason Uhlenkott <jasonuhl@sgi.com> wrote:
> Turning on PRINTK_TIME causes us to die in early boot when sched_clock()
> tries to access per-cpu data which isn't set up yet.
> 
> We can avoid this by doing a speculative load on the per-cpu data and
> just returning zero if we fail.  This matches the way sched_clock()
> appears to behave on most other arches if it gets called before we
> know the cpu frequency.
> 
> Signed-off-by: Jason Uhlenkott <jasonuhl@sgi.com>
> 
> Index: linux/arch/ia64/kernel/head.S
> =================================> --- linux.orig/arch/ia64/kernel/head.S  2005-08-15 15:54:56.130313231 -0700
> +++ linux/arch/ia64/kernel/head.S       2005-08-16 12:49:23.843747550 -0700
> @@ -983,9 +983,10 @@
>         addl r8=THIS_CPU(cpu_info) + IA64_CPUINFO_NSEC_PER_CYC_OFFSET,r0
>         mov.m r9=ar.itc         // fetch cycle-counter                          (35 cyc)
>         ;;
> -       ldf8 f8=[r8]
> +       ldf8.s f8=[r8]
>         ;;
>         setf.sig f9=r9          // certain to stall, so issue it _after_ ldf8...
> +       chk.s r8, .recover
>         ;;
>         xmpy.lu f10ù,f8       // calculate low 64 bits of 128-bit product     (4 cyc)
>         xmpy.hu f11ù,f8       // calculate high 64 bits of 128-bit product
> @@ -995,6 +996,10 @@
>         ;;
>         shrp r8=r9,r8,IA64_NSEC_PER_CYC_SHIFT
>         br.ret.sptk.many rp
> +.recover:                      // per-cpu data isn't set up yet,
> +                               // so just return 0.
> +       mov r8=r0
> +       br.ret.sptk.many rp
>  END(sched_clock)
> 
>  GLOBAL_ENTRY(start_kernel_thread)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
Mosberger Consulting LLC, voice/fax: 510-744-9372,
http://www.mosberger-consulting.com/
35706 Runckel Lane, Fremont, CA 94536

RE: [PATCH] Make sched_clock() work in early boot

[Luck, Tony]

>Turning on PRINTK_TIME causes us to die in early boot when sched_clock()
>tries to access per-cpu data which isn't set up yet.

When did this break?  I tried out PRINTK_TIME when it was first added
and it worked for me then (but since it didn't look very useful to me
I left it off in the config files that I use ... so I haven't tried it
since.

-Tony

Re: [PATCH] Make sched_clock() work in early boot

[Jason Uhlenkott]

On Tue, Aug 16, 2005 at 01:17:33PM -0700, Luck, Tony wrote:
> >Turning on PRINTK_TIME causes us to die in early boot when sched_clock()
> >tries to access per-cpu data which isn't set up yet.
> 
> When did this break?  I tried out PRINTK_TIME when it was first added
> and it worked for me then (but since it didn't look very useful to me
> I left it off in the config files that I use ... so I haven't tried it
> since.

It's been broken for me since at least April (which was the first time
I tried it).

RE: [PATCH] Make sched_clock() work in early boot

[Luck, Tony]

>> When did this break?  I tried out PRINTK_TIME when it was first added
>> and it worked for me then (but since it didn't look very useful to me
>> I left it off in the config files that I use ... so I haven't tried it
>> since.
>
>It's been broken for me since at least April (which was the first time
>I tried it).

Weird.  The timestamp code in printk() has been using sched_clock()
since the day it was added.  And the implementation of sched_clock()
has used the per-cpu area that whole time too.  But there are clearly
many printk's done before we setup the per-cpu area ... so I wonder
how this ever worked [I have a very clear memory of trying this!].

Also interesting is that even with your fix, there is something
wrong with the code ... the first few lines of output are lost
output starts with:

 of memory at 0x85000 due to granule hole at 0x0
[   0.000000] efi.trim_bottom: ignoring ...


Finally, while your patch is compact and neat, I have to agree
with David that depending on the locked TR entry for the per-cpu
area is an assumption that I'd prefer not to hardwire into more
places.

-Tony

Re: [PATCH] Make sched_clock() work in early boot

[Jason Uhlenkott]

On Tue, Aug 16, 2005 at 01:56:10PM -0700, Luck, Tony wrote:
> Also interesting is that even with your fix, there is something
> wrong with the code ... the first few lines of output are lost
> output starts with:
> 
>  of memory at 0x85000 due to granule hole at 0x0
> [   0.000000] efi.trim_bottom: ignoring ...

That's odd.  I'm getting full output on sn2 (on both real hardware and
medusa).


> Finally, while your patch is compact and neat, I have to agree
> with David that depending on the locked TR entry for the per-cpu
> area is an assumption that I'd prefer not to hardwire into more
> places.

I don't necessarily disagree.  Using a speculative load for this seemed
a little sleazy to me.  I just didn't know of any other cheap way to
test for the existence of the per-cpu area...

RE: [PATCH] Make sched_clock() work in early boot

[Luck, Tony]

>I don't necessarily disagree.  Using a speculative load for this seemed
>a little sleazy to me.  I just didn't know of any other cheap way to
>test for the existence of the per-cpu area...

Perhaps look to see if ar.k3 has been set up with the physical
address of the per-cpu area?  You'd need to clear it early in
head.S (as I'm not sure what garbage it might contain) and delay
the initialization in cpu_init() until after the per-cpu area is
fully set up [which might create new challenges :-)].

-Tony