Bogomips and loops_per_jiffy

Bogomips and loops_per_jiffy

[Jim Thompson]

I've noticed that about half the time, the kernels I build take what I
think is a long time (> 20 seconds) in calibrate_delay().

That is, there is quite a pause between
    Calibrating delay loop...
and (the answer, of sorts)
    133.12 BogoMIPS

    => bootm ff800000
    ## Booting image at ff800000 ...
       Image Name:   Musenki kernel 2.4.19pre1
       Image Type:   PowerPC Linux Kernel Image (gzip compressed)
       Data Size:    674330 Bytes = 658 kB = 0 MB
       Load Address: 00000000
       Entry Point:  00000000
       Verifying Checksum ... OK
       Uncompressing Kernel Image ... OK
    Memory BAT mapping: BAT2=32Mb, BAT3=0Mb, residual: 0Mb
    Linux version 2.4.19pre1 (jim@ford) (gcc version 2.95.4 20011002 (Debian prerelease)) #4 Fri Apr 19 05:58:00 CDT 2002
    Musenki M3/M1: See www.musenki.com for details
    On node 0 totalpages: 8192
    zone(0): 8192 pages.
    zone(1): 0 pages.
    zone(2): 0 pages.
    Kernel command line: root=/dev/nfs rw nfsroot=192.168.0.8:/usr/local/m1_nfsroot ip=192.168.0.42:192.168.0.8:192.168.0.1:255.255.255.0:fuckme::off
    OpenPIC Version 1.2 (1 CPUs and 138 IRQ sources) at fc040000
    time_init: decrementer frequency = 25.000000 MHz
    Calibrating delay loop... 133.12 BogoMIPS
    Memory: 30452k available (1200k kernel code, 480k data, 88k init, 0k highmem)
    Dentry-cache hash table entries: 4096 (order: 3, 32768 bytes)
    Inode-cache hash table entries: 2048 (order: 2, 16384 bytes)
    Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
    Buffer-cache hash table entries: 1024 (order: 0, 4096 bytes)
    Page-cache hash table entries: 8192 (order: 3, 32768 bytes)
    POSIX conformance testing by UNIFIX
    PCI: Probing PCI hardware
    [...]

I, as on so many other machines in arch/ppc/platforms/, set

	loops_per_jiffy = 50000000 / HZ;

as the first line of musenki_setup_arch(), though I've tried
"100000000 / HZ;" with no difference in effect.

Interestingly, loops_per_jiffy is apparently over-written when
calibrate_delay() is called:

    void __init calibrate_delay(void)
    {
	    unsigned long ticks, loopbit;
	    int lps_precision = LPS_PREC;

	    loops_per_jiffy = (1<<12);

	    printk("Calibrating delay loop... ");
	    while (loops_per_jiffy <<= 1) {
		    /* wait for "start of" clock tick */
		    ticks = jiffies;
		    while (ticks == jiffies)
			    /* nothing */;
		    /* Go .. */
		    ticks = jiffies;
		    __delay(loops_per_jiffy);
		    ticks = jiffies - ticks;
		    if (ticks)
			    break;
	    }

    /* Do a binary approximation to get loops_per_jiffy set to equal one clock
       (up to lps_precision bits) */
	    loops_per_jiffy >>= 1;
	    loopbit = loops_per_jiffy;
	    while ( lps_precision-- && (loopbit >>= 1) ) {
		    loops_per_jiffy |= loopbit;
		    ticks = jiffies;
		    while (ticks == jiffies);
		    ticks = jiffies;
		    __delay(loops_per_jiffy);
		    if (jiffies != ticks)   /* longer than 1 tick */
			    loops_per_jiffy &= ~loopbit;
	    }

    /* Round the value and print it */
	    printk("%lu.%02lu BogoMIPS\n",
		    loops_per_jiffy/(500000/HZ),
		    (loops_per_jiffy/(5000/HZ)) % 100);
    }



anyone have a clue to offer?

Jim


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

Jim

> I've noticed that about half the time, the kernels I build take what I
> think is a long time (> 20 seconds) in calibrate_delay().

I have exactly the same problem with my custom board using an 8260.
At every 5-10 times I start the kernel, it hangs on
'Calibrating delay loop...' for about 1 minute, and then yields
the same result (130.66 BogoMips).

I begun to try to debug that problem, but without any success yet. :-(
I've searched through the mailing lists, but did not find anything of
interest.

This problem is very annoying and I really want to fix it!


--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

Jim,

I have found a fix for the hanging "Calibrating delay loop...".

I added the following line at the beginning of
the function "time_init" in arch/ppc/kernel/time.c

  set_tb(0, 0);

This resets the time base register and avoids the timer interrupt
to set the second decrementer value to a very large one.
It happens only when the TBL is large.

This is a quick fix. Because I think it should work
for any value of TB. It looks like that part of the code
does not correctly support the overflow.

Looking through the code, I wonder why the return value of
the function "tb_delta" is signed.


--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------


> -----Original Message-----
> From: Jean-Denis Boyer [mailto:jdboyer@mediatrix.com]
> Sent: 22 avril, 2002 11:58
> To: 'Jim Thompson'
> Cc: linuxppc-embedded@lists.linuxppc.org
> Subject: RE: Bogomips and loops_per_jiffy
>
>
>
> Jim
>
> > I've noticed that about half the time, the kernels I build
> take what I
> > think is a long time (> 20 seconds) in calibrate_delay().
>
> I have exactly the same problem with my custom board using an 8260.
> At every 5-10 times I start the kernel, it hangs on
> 'Calibrating delay loop...' for about 1 minute, and then yields
> the same result (130.66 BogoMips).
>
> I begun to try to debug that problem, but without any success yet. :-(
> I've searched through the mailing lists, but did not find anything of
> interest.
>
> This problem is very annoying and I really want to fix it!
>
>
> --------------------------------------------
>  Jean-Denis Boyer, B.Eng., System Architect
>  Mediatrix Telecom Inc.
>  4229 Garlock Street
>  Sherbrooke (Québec)
>  J1L 2C8  CANADA
>  (819)829-8749 x241
> --------------------------------------------
>

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Bogomips and loops_per_jiffy

[Troy Benjegerdes]

On Mon, May 06, 2002 at 05:17:40PM -0400, Jean-Denis Boyer wrote:
>
> Jim,
>
> I have found a fix for the hanging "Calibrating delay loop...".
>
> I added the following line at the beginning of
> the function "time_init" in arch/ppc/kernel/time.c
>
>   set_tb(0, 0);
>
> This resets the time base register and avoids the timer interrupt
> to set the second decrementer value to a very large one.
> It happens only when the TBL is large.
>
> This is a quick fix. Because I think it should work
> for any value of TB. It looks like that part of the code
> does not correctly support the overflow.
>
> Looking through the code, I wonder why the return value of
> the function "tb_delta" is signed.

Hrrm... that definitely seems wrong.

I think this is a signed vs unsigned overflow.

Can you try the following and see if it fixes the problem without the
'set_tb()' quick fix? This should be a more 'correct' fix.


===== time.c 1.42 vs edited =====
--- 1.42/arch/ppc/kernel/time.c	Fri Apr  5 12:39:36 2002
+++ edited/time.c	Tue May  7 19:40:39 2002
@@ -99,8 +99,8 @@
 EXPORT_SYMBOL(rtc_lock);

 /* Timer interrupt helper function */
-static inline int tb_delta(unsigned *jiffy_stamp) {
-	int delta;
+static inline unsigned tb_delta(unsigned *jiffy_stamp) {
+	unsigned delta;
 	if (__USE_RTC()) {
 		delta = get_rtcl();
 		if (delta < *jiffy_stamp) *jiffy_stamp -= 1000000000;
@@ -150,7 +150,7 @@
  */
 int timer_interrupt(struct pt_regs * regs)
 {
-	int next_dec;
+	unsigned next_dec;
 	unsigned long cpu = smp_processor_id();
 	unsigned jiffy_stamp = last_jiffy_stamp(cpu);
 	extern void do_IRQ(struct pt_regs *);
@@ -258,7 +258,7 @@
 void do_settimeofday(struct timeval *tv)
 {
 	unsigned long flags;
-	int tb_delta, new_usec, new_sec;
+	unsigned tb_delta, new_usec, new_sec;

 	write_lock_irqsave(&xtime_lock, flags);
 	/* Updating the RTC is not the job of this code. If the time is

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]


Troy,
with the patch, now, it completely hangs.
The reason is that in timer_interrupt()

	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0)
{  

Now that next_dec is unsigned, this condition is always false.

Here is another patch I suggest to replace yours.
Its in attachement to this email.
I tested it, and it seems OK.

I did not include your modification made to the type of local variables in
function do_settimeofday().
I do not see how it is related to our problem.


--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------


[-- Attachment #2: patch.time.2 --]
[-- Type: application/octet-stream, Size: 1113 bytes --]

--- Kernel.orig/arch/ppc/kernel/time.c	Mon Apr 22 12:19:05 2002
+++ Kernel/arch/ppc/kernel/time.c	Wed May  8 12:46:57 2002
@@ -99,8 +99,8 @@
 EXPORT_SYMBOL(rtc_lock);
 
 /* Timer interrupt helper function */
-static inline int tb_delta(unsigned *jiffy_stamp) {
-	int delta;
+static inline unsigned tb_delta(unsigned *jiffy_stamp) {
+	unsigned delta;
 	if (__USE_RTC()) {
 		delta = get_rtcl();
 		if (delta < *jiffy_stamp) *jiffy_stamp -= 1000000000;
@@ -150,7 +150,7 @@
  */
 int timer_interrupt(struct pt_regs * regs)
 {
-	int next_dec;
+	unsigned next_dec;
 	unsigned long cpu = smp_processor_id();
 	unsigned jiffy_stamp = last_jiffy_stamp(cpu);
 	extern void do_IRQ(struct pt_regs *);
@@ -160,7 +160,13 @@
 
 	hardirq_enter(cpu);
 	
-	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0) {
+	while(1) {
+		unsigned delta = tb_delta(&jiffy_stamp);
+		if(tb_ticks_per_jiffy >= delta) {
+			next_dec = tb_ticks_per_jiffy - delta;
+			break;
+		}
+
 		jiffy_stamp += tb_ticks_per_jiffy;
 		if (!user_mode(regs))
 			ppc_do_profile(instruction_pointer(regs));

RE: Bogomips and loops_per_jiffy

[Benjamin Herrenschmidt]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

Hi Paul & Gabriel !

We indeed have some weird stuff mixing signed & unsigned in the
arch/ppc/kernel/time.c code. The patch enclosed, from Jean-Denis
Boyer, seems the right fix, except for one thing: I'd rather
change the test from >= to > (which in the old could would mean
changing the while (... < 0 to <= 0) as I see no reason to trigger
a new decrementer interrupt in ... 0 ticks !

Can you confirm it's ok to push ?

Regards,
Ben.

>
>Troy,
>with the patch, now, it completely hangs.
>The reason is that in timer_interrupt()
>
>	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0)
>{
>
>Now that next_dec is unsigned, this condition is always false.
>
>Here is another patch I suggest to replace yours.
>Its in attachement to this email.
>I tested it, and it seems OK.
>
>I did not include your modification made to the type of local variables in
>function do_settimeofday().
>I do not see how it is related to our problem.
>
>
>--------------------------------------------
> Jean-Denis Boyer, B.Eng., System Architect
> Mediatrix Telecom Inc.
> 4229 Garlock Street
> Sherbrooke (Québec)
> J1L 2C8  CANADA
> (819)829-8749 x241
>--------------------------------------------
>

[-- Attachment #2: patch.time.2 --]
[-- Type: application/binary, Size: 1076 bytes --]

Re: Bogomips and loops_per_jiffy

[Troy Benjegerdes]

On Wed, May 08, 2002 at 12:53:39PM -0400, Jean-Denis Boyer wrote:
>
> Troy,
> with the patch, now, it completely hangs.
> The reason is that in timer_interrupt()
>
> 	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0)
> {
>
> Now that next_dec is unsigned, this condition is always false.

Yeah, I was a little too quick to go messing with tb_delta. Benh and I
were talking on IRC and the original code does a bunch of signed vs
unsigned stuff and should probably be made more explicit as to what it's
actually doing. (Especially since I don't completely understand the
intention of this code yet)

> Here is another patch I suggest to replace yours.
> Its in attachement to this email.
> I tested it, and it seems OK.
>
> I did not include your modification made to the type of local variables in
> function do_settimeofday().
> I do not see how it is related to our problem.
>

I got carried away cleaning up stuff ;)

--
Troy Benjegerdes | master of mispeeling | 'da hozer' |  hozer@drgw.net
-----"If this message isn't misspelled, I didn't write it" -- Me -----
"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's
why I draw cartoons. It's my life." -- Charles Schulz

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

Benjamin,

> I'd rather
> change the test from >= to > (which in the old could would mean
> changing the while (... < 0 to <= 0) as I see no reason to trigger
> a new decrementer interrupt in ... 0 ticks !

I think I did not change the previous 'while' logic.
It said:
    while ( diff < 0 )

Now it says:
    while ( 1 ) {
        if ( diff >= 0 )
            break;

Previously, the while could be left for a next_dec of 0.
But I agree with you: it could be changed.
Why leaving the loop just to set the decrementer to 0!?

--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Gabriel Paubert]

On Wed, 8 May 2002, Benjamin Herrenschmidt wrote:

> Hi Paul & Gabriel !
>
> We indeed have some weird stuff mixing signed & unsigned in the
> arch/ppc/kernel/time.c code. The patch enclosed, from Jean-Denis
> Boyer, seems the right fix, except for one thing: I'd rather
> change the test from >= to > (which in the old could would mean
> changing the while (... < 0 to <= 0) as I see no reason to trigger
> a new decrementer interrupt in ... 0 ticks !

This was done on purpose in case the decrementer is way slower than the
processor. It's nitpicking but I believed that some processors have a very
slow decrementer (in this case a true fractional phaselock would be
necessary to get the HZ you want with some precision).

> Can you confirm it's ok to push ?

I'm not yet sure. I don't understand what happened in the first place.

> >	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0)
> >{
> >
> >Now that next_dec is unsigned, this condition is always false.

Yes I wrote this in the most possible compact way.  But I don't understand
(yet) how this can cause problems. You have a security margin of about 2
billion timbase ticks.

> >
> >Here is another patch I suggest to replace yours.
> >Its in attachement to this email.
> >I tested it, and it seems OK.
> >
> >I did not include your modification made to the type of local variables in
> >function do_settimeofday().
> >I do not see how it is related to our problem.

That one is better for symmetry between gettimeofday and settimeofay,
but OTOH the types in the timeval structures are defined as signed.
(Wrongly IMHO)

Could you please decribe exactly the problem or forward me the messages.
(Which machine, processor and compiler just in case). It seems that I have
been automagically dropped from linuxppc-embedded, unfortunately!

	Regards,
	Gabriel.

P.S: please try to get patches attached as text. It appears as
Application/BINARY which is quite annoying.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Bogomips and loops_per_jiffy

[Troy Benjegerdes]

>
> > >	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0)
> > >{
> > >
> > >Now that next_dec is unsigned, this condition is always false.
>
> Yes I wrote this in the most possible compact way.  But I don't understand
> (yet) how this can cause problems. You have a security margin of about 2
> billion timbase ticks.

The problem occurs when calibrating the delay loop, and the timebase is
very large.

http://lists.linuxppc.org/linuxppc-embedded/200205/msg00040.html

We could probably just avoid all this and do 'set_tb(0,0)' in time_init
(since we go and try to sync the timebase on SMP systems anyway)

>
> > >
> > >Here is another patch I suggest to replace yours.
> > >Its in attachement to this email.
> > >I tested it, and it seems OK.
> > >
> > >I did not include your modification made to the type of local variables in
> > >function do_settimeofday().
> > >I do not see how it is related to our problem.
>
> That one is better for symmetry between gettimeofday and settimeofay,
> but OTOH the types in the timeval structures are defined as signed.
> (Wrongly IMHO)
>
> Could you please decribe exactly the problem or forward me the messages.
> (Which machine, processor and compiler just in case). It seems that I have
> been automagically dropped from linuxppc-embedded, unfortunately!
>
> 	Regards,
> 	Gabriel.
>
> P.S: please try to get patches attached as text. It appears as
> Application/BINARY which is quite annoying.
>
>
>
>

--
Troy Benjegerdes | master of mispeeling | 'da hozer' |  hozer@drgw.net
-----"If this message isn't misspelled, I didn't write it" -- Me -----
"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's
why I draw cartoons. It's my life." -- Charles Schulz

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Bogomips and loops_per_jiffy

[Gabriel Paubert]

On Thu, 9 May 2002, Troy Benjegerdes wrote:

> >
> > > >	while ((next_dec = tb_ticks_per_jiffy - tb_delta(&jiffy_stamp)) < 0)
> > > >{
> > > >
> > > >Now that next_dec is unsigned, this condition is always false.
> >
> > Yes I wrote this in the most possible compact way.  But I don't understand
> > (yet) how this can cause problems. You have a security margin of about 2
> > billion timbase ticks.
>
> The problem occurs when calibrating the delay loop, and the timebase is
> very large.

The fact that the TB is large or not should be irrelevanet, after all
the TB wraps around every 2 to 5 minutes or so on most processors. Do the
math, it's rather simple modulo 2^32 arithmetic. If I lost decrementer
interrupts every time the TB wraps around, I would see strange things
under top for example.

Now if you do a set_tb after the time_init at some point, you have to
update the timekeeping variables (jiffy_stamp or whatever) in sync.
Otherwise you are going to have problems.

Note that there is a very simple possible patch: truncate the new value
you load into the decrementer to tb_ticks_per_jiffy. And for debugging,
keep the last few values of the timebase at the decrementer interrupt
around and print them when you have to truncate. I'm afraid that this is a
symptom of lost timekeeping after reading the RTC and that you simply
cure the symptom.


>
> http://lists.linuxppc.org/linuxppc-embedded/200205/msg00040.html
>
> We could probably just avoid all this and do 'set_tb(0,0)' in time_init
> (since we go and try to sync the timebase on SMP systems anyway)

Better set the TB to a large value to force the problem and see if you can
reproduce it.

	Regards,
	Gabriel.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

Grabiel,

> I'm afraid that this is a symptom of lost timekeeping
> after reading the RTC and that you simply cure the symptom.

My platform uses an 8260 processor, and AFAIK there is no RTC. The function
__USE_RTC() returns 0.

> Better set the TB to a large value to force the problem and
> see if you can reproduce it.

That's exactly the way I reproduced it!... and why I initially proposed the
fix set_tb(0,0).

If you add a set_tb(0, 0x90000000) at the beginning of time_init, you will
fall right into it.

The problem occurs only when the TBL is initially over 0x80000000. Then,
when compared against jiffy_stamp (unsigned, initially 0), and transformed
into a signed integer, it returns a negative delta, causing the while loop
in the time_interrupt to be left, and the decrementer set to a very high
value.

Yes, pratically, it will never happen at run time. Anyway, the decrementer
and the time base count at the same rate, so they share the same
constraints.

At boot time, however, it is platform specific... usually, after a cold
boot, the TB is reset to 0. But after a warm reset, this is not necessarily
true. Does the boot load resets the time base? The boot loader can also take
a long time to get the kernel in place and jump to it, for example after a
heavy hardware testing (RAM and peripherals), or an inefficient TFTP
download... Well, any weird reason you can imagine that slows down the boot
time only by a few minutes. ;-)

Regards,

--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Petersen, David (MED, GEMS-IT)]

	On the 8xx processors that I use the tb is not reset by Hard or
Soft reset (per Motorla manual), therefore it is required to set the tb
to achieve consistant operation. My system will hang periodically and
whenever it does the initial decrementer value has been set to a huge
value eg 0xFD842356.



-----Original Message-----
From: Jean-Denis Boyer [mailto:jdboyer@mediatrix.com]
Sent: Friday, May 10, 2002 9:11 AM
To: 'Gabriel Paubert'
Cc: Troy Benjegerdes; Benjamin Herrenschmidt;
linuxppc-embedded@lists.linuxppc.org; Paul Mackerras
Subject: RE: Bogomips and loops_per_jiffy



Grabiel,

> I'm afraid that this is a symptom of lost timekeeping
> after reading the RTC and that you simply cure the symptom.

My platform uses an 8260 processor, and AFAIK there is no RTC. The
function
__USE_RTC() returns 0.

> Better set the TB to a large value to force the problem and
> see if you can reproduce it.

That's exactly the way I reproduced it!... and why I initially proposed
the
fix set_tb(0,0).

If you add a set_tb(0, 0x90000000) at the beginning of time_init, you
will
fall right into it.

The problem occurs only when the TBL is initially over 0x80000000. Then,
when compared against jiffy_stamp (unsigned, initially 0), and
transformed
into a signed integer, it returns a negative delta, causing the while
loop
in the time_interrupt to be left, and the decrementer set to a very high
value.

Yes, pratically, it will never happen at run time. Anyway, the
decrementer
and the time base count at the same rate, so they share the same
constraints.

At boot time, however, it is platform specific... usually, after a cold
boot, the TB is reset to 0. But after a warm reset, this is not
necessarily
true. Does the boot load resets the time base? The boot loader can also
take
a long time to get the kernel in place and jump to it, for example after
a
heavy hardware testing (RAM and peripherals), or an inefficient TFTP
download... Well, any weird reason you can imagine that slows down the
boot
time only by a few minutes. ;-)

Regards,

--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Allen Curtis]

FYI: There is a 1 second timer in the 8260 that could be used. (Assuming
that I understand what you are referring to) I am not sure why m8260_setup.c
just sets/returns a global variable but there is/was a note there mentioning
the 1 second timer. This is what we are using. It is actually started in our
BIOS but could be done just as easily in the setup routine.

> My platform uses an 8260 processor, and AFAIK there is no RTC.
> The function


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Gabriel Paubert]

On Fri, 10 May 2002, Jean-Denis Boyer wrote:

>
> Grabiel,
>
> > I'm afraid that this is a symptom of lost timekeeping
> > after reading the RTC and that you simply cure the symptom.
>
> My platform uses an 8260 processor, and AFAIK there is no RTC. The function
> __USE_RTC() returns 0.

Yes the RTC is a 601 specific thing which makes everything more complex.

>
> > Better set the TB to a large value to force the problem and
> > see if you can reproduce it.
>
> That's exactly the way I reproduced it!... and why I initially proposed the
> fix set_tb(0,0).
>
> If you add a set_tb(0, 0x90000000) at the beginning of time_init, you will
> fall right into it.
>
> The problem occurs only when the TBL is initially over 0x80000000. Then,
> when compared against jiffy_stamp (unsigned, initially 0), and transformed
> into a signed integer, it returns a negative delta, causing the while loop
> in the time_interrupt to be left, and the decrementer set to a very high
> value.

I don't yet understand this explanation. Jiffy stamp is initialized in
time_init, in the line:

 last_jiffy_stamp(0) = tb_last_stamp = stamp;

and then jiffy_stamp is simply a temporary copy of last_jiffy_stamp in
the interrupt handler.

>
> Yes, pratically, it will never happen at run time. Anyway, the decrementer
> and the time base count at the same rate, so they share the same
> constraints.
>
> At boot time, however, it is platform specific... usually, after a cold
> boot, the TB is reset to 0. But after a warm reset, this is not necessarily
> true. Does the boot load resets the time base? The boot loader can also take
> a long time to get the kernel in place and jump to it, for example after a
> heavy hardware testing (RAM and peripherals), or an inefficient TFTP
> download... Well, any weird reason you can imagine that slows down the boot
> time only by a few minutes. ;-)

I almost only use machines which boot via tftp and take quite a long time
to boot, yet I have never seen this.

	Regards,
	Gabriel.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Gabriel Paubert]

On Fri, 10 May 2002, Jean-Denis Boyer wrote:

>
> Grabiel,
>
> > I'm afraid that this is a symptom of lost timekeeping
> > after reading the RTC and that you simply cure the symptom.
>
> My platform uses an 8260 processor, and AFAIK there is no RTC. The function
> __USE_RTC() returns 0.

OOPs, sorry for the confusion, by RTC I meant the chip with a battery
which keeps time across reboots and that we read to initially set the
system time.

	Gabriel.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Allen Curtis]

I thought that this discussion was regarding BogoMips. If so, then you do
not need the RTC, use the 1 second timer. If your concern is returning and
maintaining TOD, then that is RTC.

Right?

> >
> > > I'm afraid that this is a symptom of lost timekeeping
> > > after reading the RTC and that you simply cure the symptom.
> >
> > My platform uses an 8260 processor, and AFAIK there is no RTC.
> The function
> > __USE_RTC() returns 0.
>
> OOPs, sorry for the confusion, by RTC I meant the chip with a battery
> which keeps time across reboots and that we read to initially set the
> system time.
>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

Gabriel,


Warning: real time clock seems stuck!
> -----Original Message-----
> From: Gabriel Paubert [mailto:paubert@iram.es]
> Sent: 10 mai, 2002 10:29
> To: Jean-Denis Boyer
> Cc: Troy Benjegerdes; Benjamin Herrenschmidt;
> linuxppc-embedded@lists.linuxppc.org; Paul Mackerras
> Subject: RE: Bogomips and loops_per_jiffy
>
>
>
>
> On Fri, 10 May 2002, Jean-Denis Boyer wrote:
>
> >
> > Grabiel,
> >
> > > I'm afraid that this is a symptom of lost timekeeping
> > > after reading the RTC and that you simply cure the symptom.
> >
> > My platform uses an 8260 processor, and AFAIK there is no
> RTC. The function
> > __USE_RTC() returns 0.
>
> OOPs, sorry for the confusion, by RTC I meant the chip with a battery
> which keeps time across reboots and that we read to initially set the
> system time.
>
> 	Gabriel.
>
>

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jean-Denis Boyer]

Gabriel, (sorry to all for the previous incomplete mail!)

> I don't yet understand this explanation. Jiffy stamp is
> initialized in time_init, in the line:
>
> last_jiffy_stamp(0) = tb_last_stamp = stamp;
>
> and then jiffy_stamp is simply a temporary copy of
> last_jiffy_stamp in the interrupt handler.

Mmmm! You are right! I was annoyed of always getting the message,
 "Warning: real time clock seems stuck!"
and it also caused a 1-2 seconds delay before getting something printed to
the serial port on boot up. So I set the value of ppc_md.get_rtc_time to
NULL, since I have no RTC. That is why I never enter this code in time_init.
It correctly removed the annoying message, but brought this insidious bug.

Regards,

--------------------------------------------
 Jean-Denis Boyer, B.Eng., System Architect
 Mediatrix Telecom Inc.
 4229 Garlock Street
 Sherbrooke (Québec)
 J1L 2C8  CANADA
 (819)829-8749 x241
--------------------------------------------

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Jerry Van Baren]

The 8260's 1 second timer is only useful if your hardware clocks the "RTC"
clock input at the proper rate.  A lot of embedded boards don't bother
because the usefulness of having another crystal on the board just to clock
a counter is very tenuous.

gvb


At 07:23 AM 5/10/2002 -0700, Allen Curtis wrote:

>FYI: There is a 1 second timer in the 8260 that could be used. (Assuming
>that I understand what you are referring to) I am not sure why m8260_setup.c
>just sets/returns a global variable but there is/was a note there mentioning
>the 1 second timer. This is what we are using. It is actually started in our
>BIOS but could be done just as easily in the setup routine.
>
> > My platform uses an 8260 processor, and AFAIK there is no RTC.
> > The function
>
>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Gabriel Paubert]

On Fri, 10 May 2002, Jean-Denis Boyer wrote:

>
> Gabriel, (sorry to all for the previous incomplete mail!)
>
> > I don't yet understand this explanation. Jiffy stamp is
> > initialized in time_init, in the line:
> >
> > last_jiffy_stamp(0) = tb_last_stamp = stamp;
> >
> > and then jiffy_stamp is simply a temporary copy of
> > last_jiffy_stamp in the interrupt handler.
>
> Mmmm! You are right! I was annoyed of always getting the message,
>  "Warning: real time clock seems stuck!"
> and it also caused a 1-2 seconds delay before getting something printed to
> the serial port on boot up.

Yes, but it's basically impossible to detect problems in a time shorter
than the RTC resolution of one second. I'd just love to be able to detect
this condition faster, believe me. I also wrote it to get subsecond
precision initial system time (I don't reach the jiffy, but I don't
remember being off by over 0.1 second very often either).

> So I set the value of ppc_md.get_rtc_time to
> NULL, since I have no RTC. That is why I never enter this code in time_init.
> It correctly removed the annoying message, but brought this insidious bug.

Of course, if you break the initial conditions required by the algorithms,
you run into problems :-).

This explains that I never had this problem since all my boards have a
lithium battery powered timekeeping device called the RTC, not to be
mistaken for the RTC register in the 8260 or the RTCU/RTCL registers
of the PPC601 (actually inherited from IBM's Power AFAICT).

But in any case the solution may be as simple as either set_tb(0,0) as you
did or initializing last_jiffy_stamp(0) and tb_last_stamp with the
timebase value systematically at some earlier point. Which way is a matter
of taste, but at least it won't complicate the existing algorithms which
behave well even in the case of:

- early interrupts which can happen since the decrementer is used as a
self IPI in pmac_pic.c (actually it could be set_dec(0) instead of
set_dec(1) since the dec interrupt triggers on the rising transition of
the MSB).

- late interrupts in which you have to catch up several jiffies, which
happened quite frequently at some point when switching consoles...

I don't have the time to study whether the patches proposed would work
well in all these weird cases.

BTW, how do you initially set the system time in your RTC less machines ?

	Regards,
	Gabriel.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Benjamin Herrenschmidt]

>I don't have the time to study whether the patches proposed would work
>well in all these weird cases.
>
>BTW, how do you initially set the system time in your RTC less machines ?

I spent some time studying the latest of the patches and it seems ok.

Ben.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

RE: Bogomips and loops_per_jiffy

[Gabriel Paubert]

On Fri, 10 May 2002, Benjamin Herrenschmidt wrote:

> >I don't have the time to study whether the patches proposed would work
> >well in all these weird cases.
> >
> >BTW, how do you initially set the system time in your RTC less machines ?
>
> I spent some time studying the latest of the patches and it seems ok.

Yes, however if it can be fixed by just adding an initialization of the
timebase or of the timekeeping values even on the machines on which you
can't initialize the system time, I'd prefer to do it that way.

The decrementer interrupt code is not executed frequently enough to stay
in the cache, so making it bigger means perhaps one more cache miss (to
memory, not to L2 or L3) HZ times per second. Not a big deal I know, but
you know that I'm a fanatic of minimal code size (as everybody should be
for non looping code at least) and the loop exit expression I wrote maps
exactly to:

	sub. next_dec,tb_ticks_per_jiffy,tb_delta()


Besides that, the proposed patch which only exits the loop when the
decrememter count falls between 0 and tb_ticks_per_jiffy means that in the
missing initialization case you are going to execute do_timer() a few
thousand times. Although it should not cause any problem since I don't
expect any timer to be running at this early boot stage, I find this
possibility extremely ugly.

In short, I still prefer fixing the initialization of tb_last_stamp, which
could be done with the following simple (but untested and not even
compiled) patch, which essentially shrinks what we skip when
ppc_md.get_rtc_time is not defined.

That's from my 2.4 BK repository but it should apply almost trivially to
any 2.4/2.5 kernel.

===== time.c 1.32 vs edited =====
--- 1.32/arch/ppc/kernel/time.c	Fri Apr  5 19:17:22 2002
+++ edited/time.c	Sat May 11 03:46:08 2002
@@ -316,8 +316,9 @@
 	 * makes things more complex. Repeatedly read the RTC until the
 	 * next second boundary to try to achieve some precision...
 	 */
+	sec = 0;
+	stamp = get_native_tbl();
 	if (ppc_md.get_rtc_time) {
-		stamp = get_native_tbl();
 		sec = ppc_md.get_rtc_time();
 		elapsed = 0;
 		do {
@@ -331,14 +332,14 @@
 		if (sec==old_sec) {
 			printk("Warning: real time clock seems stuck!\n");
 		}
-		write_lock_irqsave(&xtime_lock, flags);
-		xtime.tv_sec = sec;
-		last_jiffy_stamp(0) = tb_last_stamp = stamp;
-		xtime.tv_usec = 0;
-		/* No update now, we just read the time from the RTC ! */
-		last_rtc_update = xtime.tv_sec;
-		write_unlock_irqrestore(&xtime_lock, flags);
 	}
+	write_lock_irqsave(&xtime_lock, flags);
+	xtime.tv_sec = sec;
+	last_jiffy_stamp(0) = tb_last_stamp = stamp;
+	xtime.tv_usec = 0;
+	/* No update now, we just read the time from the RTC ! */
+	last_rtc_update = xtime.tv_sec;
+	write_unlock_irqrestore(&xtime_lock, flags);

 	/* Not exact, but the timer interrupt takes care of this */
 	set_dec(tb_ticks_per_jiffy);


Oh, BTW I just had to reboot 6 machines (correlation spectrometers) after
some hardware modifications I had to perform today, that's what I have in
the logs showing the absolute time error on boot of these machines:

May 10 21:25:02 vcorr1 ntpdate[203]: step time server 150.214.224.210
offset -0.010231 sec
May 10 21:25:03 vcorr2 ntpdate[203]: step time server 150.214.224.210
offset 0.014724 sec
May 10 21:25:06 vcorr3 ntpdate[203]: step time server 150.214.224.210
offset 0.035770 sec
May 10 21:25:09 vcorr4 ntpdate[203]: step time server 150.214.224.210
offset 0.045062 sec
May 10 21:25:09 vcorr5 ntpdate[203]: step time server 150.214.224.210
offset 0.034193 sec
May 10 21:25:21 vcorr6 ntpdate[203]: step time server 150.214.224.210
offset 0.034062 sec

I believe we can do better but since what I have some doubts on the
absolute precision of the RTC chips when you store a new value. But the
documentation of the chips on this kind of details sucks. The only
conclusion is that I have to write to the clock as close as possible from
the second boudnary and not on the half second like PC clocks.

	Regards,
	Gabriel.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/