IA64 implementation of timesource for new time of day subsystem

IA64 implementation of timesource for new time of day subsystem

[Christoph Lameter]

On Fri, 13 May 2005, john stultz wrote:

> I look forward to your comments and feedback.

Here is the implementation of the IA64 timesources for the new time of 
day subsystem.

This is quite straighforward. Thanks John. However, the ITC
interpolator can no longer use MMIO in SMP situations since there is no 
provision for jitter compensation in the new time of day subsystem. I have
implemented that via a function now which will slow down clock access
for non SGI IA64 hardware significantly since it will not be able to use
the fastcall anymore.

I am working on the fastcall but I would need a couple of changes
to the core code to make the following symbols non-static since they
will need to be accessed from the fast syscall handler:

timesource
system_time
wall_time_offset
offset_base

The asm code is going to be simplified because there will be no need
to support jitter compensation and most values are now single 64 bit values
instead of two 64 bit values with separate seconds and nanoseconds.

However, the asm code is also is going to be a bit more complicated since
the split from 64 bit nanoseconds into seconds and 
nanoseconds/microseconds for gettimeofday and clock_gettime
has to be done in asm as well.

I would recommend to add jitter compensation to the time sources. Otherwise
each ITC/TSC like timesource will have to implement that on its own.

Signed-off-by: Christoph Lameter <clameter@sgi.com>

Index: linux-2.6.12-rc4/drivers/timesource/Makefile
=================================--- linux-2.6.12-rc4.orig/drivers/timesource/Makefile	2005-05-14 11:21:46.000000000 -0700
+++ linux-2.6.12-rc4/drivers/timesource/Makefile	2005-05-14 12:15:08.000000000 -0700
@@ -4,9 +4,8 @@ obj-$(CONFIG_PPC64) += ppc64_timebase.o
 obj-$(CONFIG_PPC) += ppc_timebase.o
 obj-$(CONFIG_ARCH_S390) += s390_tod.o
 
-# XXX - Untested/Uncompiled
-#obj-$(CONFIG_IA64) += itc.c
-#obj-$(CONFIG_IA64_SGI_SN2) += sn2_rtc.c
+obj-$(CONFIG_IA64) += itc.o
+obj-$(CONFIG_IA64_SGI_SN2) += sn2_rtc.o
 obj-$(CONFIG_X86) += tsc.o
 obj-$(CONFIG_X86) += i386_pit.o
 obj-$(CONFIG_X86) += tsc-interp.o
Index: linux-2.6.12-rc4/drivers/timesource/itc.c
=================================--- linux-2.6.12-rc4.orig/drivers/timesource/itc.c	2005-05-14 11:21:46.000000000 -0700
+++ linux-2.6.12-rc4/drivers/timesource/itc.c	2005-05-14 12:20:00.000000000 -0700
@@ -1,31 +1,83 @@
-/* XXX - this is totally untested and uncompiled
- * TODO:
- *		o cpufreq issues
- *		o unsynched ITCs ?
+/*
+ * drivers/timesource/itc.c
+ *
+ * Use of the ITC register on Itanium processors as a time source
+ *
+ * Copyright (C) 2005 Silicon Graphics, Inc.
+ *	Christoph Lameter, <clameter@sgi.com>
  */
+#include <linux/config.h>
 #include <linux/timesource.h>
+#include <linux/jiffies.h>
 
-/* XXX - Other includes needed for:
- *		sal_platform_features, IA64_SAL_PLATFORM_FEATURE_ITC_DRIFT,
- *		local_cpu_data->itc_freq
- * See arch/ia64/kernel/time.c for ideas
- */
+#include <asm/machvec.h>
+#include <asm/sal.h>
+#include <asm/system.h>
 
 static struct timesource_t timesource_itc = {
 	.name = "itc",
 	.priority = 25,
 	.type = TIMESOURCE_CYCLES,
 	.mask = (cycle_t)-1,
-	.mult = 0, /* to be set */
 	.shift = 22,
 };
 
+#ifdef CONFIG_SMP
+static int nojitter;
+
+static __init int nojitter_setup(char *str)
+{
+	nojitter = 1;
+	printk(KERN_INFO "ITC timesource: Jitter checking bypassed.\n");
+	return 1;
+}
+
+__setup("itc_nojitter", nojitter_setup);
+
+cycle_t last_itc;
+
+/*
+ * Insure that ITC is monotonically increasing by comparing
+ * to the last value encountered. Do this in an atomic fashion
+ * by using cmpxchg for synchronization between processors
+ * and at the same time for the updating of the last_itc value;
+ */
+static cycle_t itc_filtered(void) {
+	cycle_t now, last;
+
+	do {
+		last = last_itc;
+		smb_rmb();
+		now = get_cycles();
+		if (time_before(now, last))
+			return last_itc;
+	} while (cmpxchg(&last_itc, last, now) != last);
+	return now;
+}
+#endif
+
 static int __init init_itc_timesource(void)
 {
 	if (!(sal_platform_features & IA64_SAL_PLATFORM_FEATURE_ITC_DRIFT)) {
-		/* XXX - I'm not really sure if itc_freq is in cyc/sec */
 		timesource_itc.mult = timesource_hz2mult(local_cpu_data->itc_freq,
 									timesource_itc.shift);
+#ifdef CONFIG_SMP
+		/* ITCs are never accurately synchronized in an SMP configuration
+		 * even if the ITC_DRIFT bit is not set.
+		 * Jitter compensation requires a cmpxchg which may limit
+		 * the scalability of the syscalls for retrieving time.
+		 * ITC synchronization is usually successful to within a few
+		 * ITC ticks but this is not a sure thing. If you need to improve
+		 * timer performance in SMP situations then boot the kernel with the
+		 * "itc_nojitter" option. However, doing so may result in time fluctuating
+		 * (maybe even appearing to go backward!) if the ITC offsets between the
+		 * individual CPUs are too large.
+		 */
+		if (!nojitter) {
+			timesource_itc.type = TIMESOURCE_FUNCTION;
+			timesource_itc.read_fnct = itc_filtered;
+		}
+#endif
 		register_timesource(&timesource_itc);
 	}
 	return 0;
Index: linux-2.6.12-rc4/arch/ia64/kernel/time.c
=================================--- linux-2.6.12-rc4.orig/arch/ia64/kernel/time.c	2005-05-14 11:21:46.000000000 -0700
+++ linux-2.6.12-rc4/arch/ia64/kernel/time.c	2005-05-14 12:15:08.000000000 -0700
@@ -139,6 +139,7 @@ ia64_cpu_local_tick (void)
 	ia64_set_itm(local_cpu_data->itm_next);
 }
 
+#ifndef CONFIG_NEWTOD
 static int nojitter;
 
 static int __init nojitter_setup(char *str)
@@ -150,6 +151,7 @@ static int __init nojitter_setup(char *s
 
 __setup("nojitter", nojitter_setup);
 
+#endif
 
 void __devinit
 ia64_init_itm (void)
Index: linux-2.6.12-rc4/drivers/timesource/sn2_rtc.c
=================================--- linux-2.6.12-rc4.orig/drivers/timesource/sn2_rtc.c	2005-05-14 11:21:46.000000000 -0700
+++ linux-2.6.12-rc4/drivers/timesource/sn2_rtc.c	2005-05-14 12:15:08.000000000 -0700
@@ -1,29 +1,38 @@
-#include <linux/timesource.h>
-/* XXX this will need some includes
- * to find: sn_rtc_cycles_per_second and RTC_COUNTER_ADDR
- * See arch/ia64/sn/kernel/sn2/timer.c for likely suspects
+/*
+ * linux/drivers/timesource/sn2_rtc.c
+ *
+ * Use the RTC on the SN2 on an Altix system as the time source
+ *
+ * (C) 2005 Silicon Graphics, Inc.
+ *	Christoph Lameter <clameter@sgi.com>
  */
 
+
+#include <linux/timesource.h>
+#include <asm/system.h>
+#include <asm/sn/leds.h>
+#include <asm/sn/shub_mmr.h>
+#include <asm/sn/clksupport.h>
+
+extern unsigned long sn_rtc_cycles_per_second;
+
 #define SN2_RTC_MASK ((1LL << 55) - 1)
 #define SN2_SHIFT 10
 
 struct timesource_t timesource_sn2_rtc = {
 	.name = "sn2_rtc",
-	.priority = 300, /* XXX - not sure what this should be */
+	.priority = 999,
 	.type = TIMESOURCE_MMIO_64,
-	.mmio_ptr = NULL,
 	.mask = (cycle_t)SN2_RTC_MASK,
 	.mult = 0, /* set below */
 	.shift = SN2_SHIFT,
 };
 
-static void __init init_sn2_timesource(void)
+static __init int init_sn2_timesource(void)
 {
-	timesource_sn2_rtc.mult = timesource_hz2mult(sn_rtc_cycles_per_second,
-												SN2_SHIFT);
+	timesource_sn2_rtc.mult = timesource_hz2mult(sn_rtc_cycles_per_second,	SN2_SHIFT);
 	timesource_sn2_rtc.mmio_ptr = RTC_COUNTER_ADDR;
-
-	register_time_interpolator(&timesource_sn2_rtc);
+	register_timesource(&timesource_sn2_rtc);
 	return 0;
 }
 module_init(init_sn2_timesource);

Re: IA64 implementation of timesource for new time of day subsystem

[James Courtier-Dutton]

Christoph Lameter wrote:
> On Fri, 13 May 2005, john stultz wrote:
> 
> 
>>I look forward to your comments and feedback.
> 
> 
> Here is the implementation of the IA64 timesources for the new time of 
> day subsystem.
> 
> This is quite straighforward. Thanks John. However, the ITC
> interpolator can no longer use MMIO in SMP situations since there is no 
> provision for jitter compensation in the new time of day subsystem. I have
> implemented that via a function now which will slow down clock access
> for non SGI IA64 hardware significantly since it will not be able to use
> the fastcall anymore.
> 
> I am working on the fastcall but I would need a couple of changes
> to the core code to make the following symbols non-static since they
> will need to be accessed from the fast syscall handler:
> 
> timesource
> system_time
> wall_time_offset
> offset_base
> 

Will this mean that Linux will have a monotonic time source?
For media players we need a timesource that does not change under any
circumstances. e.g. User changes the clock time, the monotonic time
source should not change. The monotonic time source should just start at
0 at power on, and continually increase accurately over time. I.e. A
very accurate "uptime" measurement.

James

Re: IA64 implementation of timesource for new time of day subsystem

[Andi Kleen]

> Will this mean that Linux will have a monotonic time source?

2.6 has had one for a long time (posix_gettime(CLOCK_MONOTONIC))

-Andi

Re: IA64 implementation of timesource for new time of day subsystem

[Chris Friesen]

Andi Kleen wrote:
>>Will this mean that Linux will have a monotonic time source?
> 
> 2.6 has had one for a long time (posix_gettime(CLOCK_MONOTONIC))

I think that's clock_gettime(), no?

Chris

Re: IA64 implementation of timesource for new time of day subsystem

[john stultz]

On Sat, 2005-05-14 at 12:55 -0700, Christoph Lameter wrote:
> On Fri, 13 May 2005, john stultz wrote:
> 
> > I look forward to your comments and feedback.
> 
> Here is the implementation of the IA64 timesources for the new time of 
> day subsystem.

Great!

> This is quite straighforward. Thanks John. However, the ITC
> interpolator can no longer use MMIO in SMP situations since there is no 
> provision for jitter compensation in the new time of day subsystem. I have
> implemented that via a function now which will slow down clock access
> for non SGI IA64 hardware significantly since it will not be able to use
> the fastcall anymore.
> 
> I am working on the fastcall but I would need a couple of changes
> to the core code to make the following symbols non-static since they
> will need to be accessed from the fast syscall handler:
> 
> timesource
> system_time
> wall_time_offset
> offset_base


Actually that shouldn't be necessary. Look at my arch-x86-64 patch or
vsyscall-i386 patch for how the arch_vsyscall_gtod_update() function is
used. It provides an arch specific hook called by the timeofday core to
provide the information you desire. 

Please let me know if it is not sufficient for some reason. 

[snip]

> I would recommend to add jitter compensation to the time sources. Otherwise
> each ITC/TSC like timesource will have to implement that on its own.

Just to clarify for others, this is the same unsynced cpu cycle counter
problem that affects the TSC on i386 and x86-64. ia64 gets around the
problem by checking on every call to gettimeofday() if the ITC value is
less then the ITC value used on the previous call to gettimeofday(). If
the value is less (ie: would result in time going backwards) it just
uses the last value to calculate time. It then uses cmpxchg to
atomically update the last ITC value. 

The problem I have with this is it that if the ITCs are not synced, they
really are not good timesources. If one cpu's ITC is behind another, the
net result of the above algorithm is cpu 2 will always just use cpu 1's
last calculated time. This could cause jumps in time when a process
moves from cpu2 to cpu1.

Since it only affects the TSC and ITC, I think keeping the decision to
use cmpxchg in the timesource code, as you've implemented with the ITC
is the best way to go. If you really want to you can special case the
arch specific fsyscall code by switching on the time source .name, and
that would allow you to use a similar cmpxchg algorithm there as well. 


thanks
-john

Re: IA64 implementation of timesource for new time of day subsystem

[Christoph Lameter]

On Mon, 16 May 2005, john stultz wrote:

> Actually that shouldn't be necessary. Look at my arch-x86-64 patch or
> vsyscall-i386 patch for how the arch_vsyscall_gtod_update() function is
> used. It provides an arch specific hook called by the timeofday core to
> provide the information you desire. 
> 
> Please let me know if it is not sufficient for some reason. 

Obviously this wont work since you cannot execute C code nor functions in 
an ia64 fastcall. I need the variables exported.

> > I would recommend to add jitter compensation to the time sources. Otherwise
> > each ITC/TSC like timesource will have to implement that on its own.
> 
> Just to clarify for others, this is the same unsynced cpu cycle counter
> problem that affects the TSC on i386 and x86-64. ia64 gets around the
> problem by checking on every call to gettimeofday() if the ITC value is
> less then the ITC value used on the previous call to gettimeofday(). If
> the value is less (ie: would result in time going backwards) it just
> uses the last value to calculate time. It then uses cmpxchg to
> atomically update the last ITC value. 

Nope. You are way off here. Unsynched cpu cycle counters lead to the ITC 
timesource not being registered.

> The problem I have with this is it that if the ITCs are not synced, they
> really are not good timesources. If one cpu's ITC is behind another, the
> net result of the above algorithm is cpu 2 will always just use cpu 1's
> last calculated time. This could cause jumps in time when a process
> moves from cpu2 to cpu1.

Note again that the use of cmpxchg is NOT covering the case of ITCs not 
being synced. If the ITCs are not synced then no timesource will be 
established for ITC!

This is the case of ITC's running synchronous but at a tiny offset. The 
startup on IA64 syncs the ITCs but cannot guarantee a complete sync. There 
may be a small offset of a few clock ticks. The cmpxchg is 
needed to compensate for that small offset. I imagine that other 
architectures have similar issues.

> Since it only affects the TSC and ITC, I think keeping the decision to
> use cmpxchg in the timesource code, as you've implemented with the ITC
> is the best way to go. If you really want to you can special case the
> arch specific fsyscall code by switching on the time source .name, and
> that would allow you to use a similar cmpxchg algorithm there as well. 

Again this will not work on IA64 since it does the fast system calls in a 
different way.

Clock jitter can affect multiple clock sources that may fluctuate
in a minor way due to a variety of influences. Jitter compensation may 
help in these situations.

Re: IA64 implementation of timesource for new time of day subsystem

[john stultz]

On Mon, 2005-05-16 at 11:09 -0700, Christoph Lameter wrote:
> On Mon, 16 May 2005, john stultz wrote:
> 
> > Actually that shouldn't be necessary. Look at my arch-x86-64 patch or
> > vsyscall-i386 patch for how the arch_vsyscall_gtod_update() function is
> > used. It provides an arch specific hook called by the timeofday core to
> > provide the information you desire. 
> > 
> > Please let me know if it is not sufficient for some reason. 
> 
> Obviously this wont work since you cannot execute C code nor functions in 
> an ia64 fastcall. I need the variables exported.

No. Look at the x86-64 code. The generic timeofday core calls
arch_update_vsyscall_gtod() (sorry for the function name confusion
above) any time the timekeeping variables change. 

All you need is to do is define  implement an ia64 version of
arch_update_vsyscall_gtod() which can then export the values passed to
it in whatever form you desire so it can be used by the fastcall.


> > > I would recommend to add jitter compensation to the time sources. Otherwise
> > > each ITC/TSC like timesource will have to implement that on its own.
> > 
> > Just to clarify for others, this is the same unsynced cpu cycle counter
> > problem that affects the TSC on i386 and x86-64. ia64 gets around the
> > problem by checking on every call to gettimeofday() if the ITC value is
> > less then the ITC value used on the previous call to gettimeofday(). If
> > the value is less (ie: would result in time going backwards) it just
> > uses the last value to calculate time. It then uses cmpxchg to
> > atomically update the last ITC value. 
> 
> Nope. You are way off here. Unsynched cpu cycle counters lead to the ITC 
> timesource not being registered.
> 
> > The problem I have with this is it that if the ITCs are not synced, they
> > really are not good timesources. If one cpu's ITC is behind another, the
> > net result of the above algorithm is cpu 2 will always just use cpu 1's
> > last calculated time. This could cause jumps in time when a process
> > moves from cpu2 to cpu1.
> 
> Note again that the use of cmpxchg is NOT covering the case of ITCs not 
> being synced. If the ITCs are not synced then no timesource will be 
> established for ITC!
> 
> This is the case of ITC's running synchronous but at a tiny offset. The 
> startup on IA64 syncs the ITCs but cannot guarantee a complete sync. There 
> may be a small offset of a few clock ticks. The cmpxchg is 
> needed to compensate for that small offset. I imagine that other 
> architectures have similar issues.

Just per-cpu cycle counters like the TSC and ITC to my knowledge. The
PPC timebase increments off of a global bus-signal, so it is not
affected. I'd be interested in other examples, though.


> > Since it only affects the TSC and ITC, I think keeping the decision to
> > use cmpxchg in the timesource code, as you've implemented with the ITC
> > is the best way to go. If you really want to you can special case the
> > arch specific fsyscall code by switching on the time source .name, and
> > that would allow you to use a similar cmpxchg algorithm there as well. 
> 
> Again this will not work on IA64 since it does the fast system calls in a 
> different way.

I think you'll find otherwise. The arch_update_vsyscall_gtod() interface
gives each arch quite a bit of flexibility in how to implement their own
accelerated timeofday. 

In pseudo code, all you would need to do is something like:

arch_update_vsyscall_gtod(wall_time, offset_base, timesource, ntp_adj):

	fastcall_data.wall = wall_time
	fastcall_data.base = offset_base
	fastcall_data.ts = timesource
	fastcall_data.ntpadj = ntp_adj


fastcall_gtod(): [I understand this would be done in asm]

	switch(fastcall_data.ts.type):
	case TIMESOURCE_MMIO:
		now = <mmio read code>
	case TIMESOURCE_CYCLE:	
		bow = <cycle read code>
	case TIMESOURCE_FUNCTION:
		# special case for itc
		if (fastcall_data.ts.name = "itc")
			now = <cycle jitter read>
		else
			return gettimeofday()

	offset = (now - fastcall_data.base)
	offset *= fastcall_data.ts.mult 
	offset += fastcall_data.ntpadj
	offset >>= fastcall_data.ts.shift 

	return fastcall_data.wall + offset


> Clock jitter can affect multiple clock sources that may fluctuate
> in a minor way due to a variety of influences. Jitter compensation may 
> help in these situations.

Forgive me as I'm just not aware of these, and am thus hesitant to
change the core code for two known cases that can be cleanly dealt with
in the timesource driver code.

thanks
-john

Re: IA64 implementation of timesource for new time of day subsystem

[john stultz]

On Mon, 2005-05-16 at 11:45 -0700, john stultz wrote:
> All you need is to do is define  implement an ia64 version of
> arch_update_vsyscall_gtod() which can then export the values passed to
> it in whatever form you desire so it can be used by the fastcall.


Sorry, that first sentence wasn't complete, I went to go look up the
reference and got distracted with something else.

It should be "All you need to do is define NEWTOD_VSYSCALL and
implement.."

time for more coffee :)
-john

Re: IA64 implementation of timesource for new time of day subsystem

[Christoph Lameter]

On Mon, 16 May 2005, john stultz wrote:

> In pseudo code, all you would need to do is something like:
> 
> arch_update_vsyscall_gtod(wall_time, offset_base, timesource, ntp_adj):
> 
> 	fastcall_data.wall = wall_time
> 	fastcall_data.base = offset_base
> 	fastcall_data.ts = timesource
> 	fastcall_data.ntpadj = ntp_adj


Ahh. Thanks.

> > Clock jitter can affect multiple clock sources that may fluctuate
> > in a minor way due to a variety of influences. Jitter compensation may 
> > help in these situations.
> 
> Forgive me as I'm just not aware of these, and am thus hesitant to
> change the core code for two known cases that can be cleanly dealt with
> in the timesource driver code.

I am happy to leave the situation as is since it does not affect SGI. 
We have a memory mapped timer that does not need this jitter compensation.

Other IA64 vendors will see that their timer performance drops 
significantly after the new timer subsystem is in. IBM no longer 
has IA64 systems that rely on ITC?

Re: IA64 implementation of timesource for new time of day subsystem

[David Mosberger]

>>>>> On Mon, 16 May 2005 12:24:08 -0700 (PDT), Christoph Lameter <clameter@engr.sgi.com> said:

  Christoph> Other IA64 vendors will see that their timer performance
  Christoph> drops significantly after the new timer subsystem is
  Christoph> in. IBM no longer has IA64 systems that rely on ITC?

Would that somehow make it ok to break existing and working code?

	--david

Re: IA64 implementation of timesource for new time of day subsystem

[john stultz]

On Mon, 2005-05-16 at 12:29 -0700, David Mosberger wrote: 
> >>>>> On Mon, 16 May 2005 12:24:08 -0700 (PDT), Christoph Lameter <clameter@engr.sgi.com> said:
> 
>   Christoph> Other IA64 vendors will see that their timer performance
>   Christoph> drops significantly after the new timer subsystem is
>   Christoph> in. IBM no longer has IA64 systems that rely on ITC?
> 
> Would that somehow make it ok to break existing and working code?

No. I intend to preserve the existing functionality (and performance) of
the current code. The current timeofday core should allow for this (as I
described in my last mail), so really its just a matter of either me or
someone else getting around to properly converting that arch with the
help of the arch maintainer. Until the arch is really ready to use the
new timeofday core, no changes are necessary.

Christoph's patch is just a step in the right direction. That is, a much
appreciated step, I haven't yet had the time to implement or test the
ia64 timesources. Any notable regressions introduced will need to be
resolved before the arch specific patch is finally submitted.

What I'm trying to shake out, with Christoph's help, is any major
limitations in the core timeofday code that would keep an arch from
being able to use it.  I feel Christoph's concerns have been addressed,
but please let me know if you disagree.

thanks
-john

Re: IA64 implementation of timesource for new time of day subsystem

[Christoph Lameter]

On Mon, 16 May 2005, john stultz wrote:

> 
> No. I intend to preserve the existing functionality (and performance) of
> the current code. The current timeofday core should allow for this (as I
> described in my last mail), so really its just a matter of either me or
> someone else getting around to properly converting that arch with the
> help of the arch maintainer. Until the arch is really ready to use the
> new timeofday core, no changes are necessary.

Its not an arch specific issue. The time sources need to have a field that 
specifies that jitter protection is needed and there needs to be some 
logic to implement it. Otherwise we have to develop special functions for 
each timesource that deal with jitter protection. Function will make a 
fastcall for the clocks that use jitter protection not possible and thus 
timer access will slow down.

> What I'm trying to shake out, with Christoph's help, is any major
> limitations in the core timeofday code that would keep an arch from
> being able to use it.  I feel Christoph's concerns have been addressed,
> but please let me know if you disagree.

Please add jitter protection to the arch independent code.

Re: IA64 implementation of timesource for new time of day subsystem

[john stultz]

On Mon, 2005-05-16 at 13:27 -0700, Christoph Lameter wrote:
> On Mon, 16 May 2005, john stultz wrote:
> 
> > 
> > No. I intend to preserve the existing functionality (and performance) of
> > the current code. The current timeofday core should allow for this (as I
> > described in my last mail), so really its just a matter of either me or
> > someone else getting around to properly converting that arch with the
> > help of the arch maintainer. Until the arch is really ready to use the
> > new timeofday core, no changes are necessary.
> 
> Its not an arch specific issue. The time sources need to have a field that 
> specifies that jitter protection is needed and there needs to be some 
> logic to implement it. Otherwise we have to develop special functions for 
> each timesource that deal with jitter protection. 

You've only pointed out two timesources that could want this (ITC and
TSC), so I think its reasonable to do your jitter handling in the
timesource driver. If there are other arches that have non hardware
synced per-cpu counters, then it would be something to consider.

> Function will make a 
> fastcall for the clocks that use jitter protection not possible and thus 
> timer access will slow down.


I disagree. I already explained how this can be done via the
arch_update_vsyscall_gtod() interface by special casing for this
specific well known time source.


> > What I'm trying to shake out, with Christoph's help, is any major
> > limitations in the core timeofday code that would keep an arch from
> > being able to use it.  I feel Christoph's concerns have been addressed,
> > but please let me know if you disagree.
> 
> Please add jitter protection to the arch independent code.

If more timesources need that functionality, then I'll be happy to.
Until then it should stay in the ia64 specific itc driver and fastcall
code.

thanks
-john

Re: IA64 implementation of timesource for new time of day subsystem

[David Mosberger]

>>>>> On Mon, 16 May 2005 13:53:44 -0700, john stultz <johnstul@us.ibm.com> said:


  John> You've only pointed out two timesources that could want this
  John> (ITC and TSC), so I think its reasonable to do your jitter
  John> handling in the timesource driver. If there are other arches
  John> that have non hardware synced per-cpu counters, then it would
  John> be something to consider.

I think Christopher's point is that _all_ time-sources which require
software syncing will need this since it is not possible to sync
perfectly, even if there is no drift.

	--david

Re: IA64 implementation of timesource for new time of day subsystem

[john stultz]

On Mon, 2005-05-16 at 13:58 -0700, David Mosberger wrote:
> >>>>> On Mon, 16 May 2005 13:53:44 -0700, john stultz <johnstul@us.ibm.com> said:
>   John> You've only pointed out two timesources that could want this
>   John> (ITC and TSC), so I think its reasonable to do your jitter
>   John> handling in the timesource driver. If there are other arches
>   John> that have non hardware synced per-cpu counters, then it would
>   John> be something to consider.
> 
> I think Christopher's point is that _all_ time-sources which require
> software syncing will need this since it is not possible to sync
> perfectly, even if there is no drift.

Yes, but to my knowledge it is only the ITC that does software syncing. 
The TSC could use it as well, but doesn't. Other then that I haven't
heard of any other timesource that would use such functionality.

Since its possible to do jitter compensation within the itc timesource
driver (and within the fastcall code to preserve the existing
performance), would it be reasonable to deffer making the jitter
compensation code generic until another timesource needs it? It should
be a fairly simple change.

Or is this just something I'm being hard-headed about?

thanks
-john

Re: IA64 implementation of timesource for new time of day subsystem

[Christoph Lameter]

On Mon, 16 May 2005, john stultz wrote:

> Since its possible to do jitter compensation within the itc timesource
> driver (and within the fastcall code to preserve the existing
> performance), would it be reasonable to deffer making the jitter
> compensation code generic until another timesource needs it? It should
> be a fairly simple change.

Well looks that we will start out with the new time subsystem by putting 
some hacks in. I need to check in the funky routine (for setting up the 
fastcall configuration) if the function pointer passed = jitter-compensated-itc
and depending on that set a special flag that makes the asm code do jitter
compensation.

> Or is this just something I'm being hard-headed about?

Looks like it. Its not that difficult. Add a jitter compensation flag to
timesource. Check on retrieving from a timesource if its less than last if 
flag is set. Pass the field to the funky function to setup the 
vsyscalls.

Maybe add a general flags field? There may be other things that need to be 
added in the future.

Re: IA64 implementation of timesource for new time of day subsystem

[Ulrich Windl]

On 16 May 2005 at 12:29, David Mosberger wrote:

> >>>>> On Mon, 16 May 2005 12:24:08 -0700 (PDT), Christoph Lameter <clameter@engr.sgi.com> said:
> 
>   Christoph> Other IA64 vendors will see that their timer performance
>   Christoph> drops significantly after the new timer subsystem is
>   Christoph> in. IBM no longer has IA64 systems that rely on ITC?
> 
> Would that somehow make it ok to break existing and working code?

AFAIR the design goal was not to make a new time implementation, but to make a 
more precise one ;-)

Regards,
Ulrich