[Linux-ia64] provide /proc/sal/itc_drift through AUX?

[Linux-ia64] provide /proc/sal/itc_drift through AUX?

[Jes Sorensen]

Hi

I was wondering what people think about providing the information of
/proc/sal/itc_drift as an AUX vector?

The problem is that on some NUMA boxen (such as the SGI boxes), the ITC
isn't synchronized across nodes and we can't rely on ar.itc in userland
for implementing the high-precision timing. I believe the IBM NUMA-Q
team has a similar problem that could be solved in a similar way on
ia32?

Instead one can switch to using gettimeofday() for the timing, which
with the new fast syscalls should be quite pleasant.

I have a patch which implements this for glibc-2.2 (will do 2.3 later),
however what I don't like about it is that one ends up opening and
reading /proc/sal/itc_drift in every single binary executed. To avoid
the overhead of this it seems a good idea to me to provide this
information via an AUX vector.

If anybody is interested in the glibc patch, feel free to grab it from
http://www.wildopensource.com/~jes/glibc/itc-drift-patch.diff
For now it's a test patch, though it does seem to behave as expected.

Thoughts?

Cheers,
Jes

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[David Mosberger]

>>>>> On Wed, 19 Mar 2003 16:39:31 -0500, Jes Sorensen <jes@wildopensource.com> said:

  Jes> Hi I was wondering what people think about providing the
  Jes> information of /proc/sal/itc_drift as an AUX vector?

  Jes> The problem is that on some NUMA boxen (such as the SGI boxes),
  Jes> the ITC isn't synchronized across nodes and we can't rely on
  Jes> ar.itc in userland for implementing the high-precision
  Jes> timing. I believe the IBM NUMA-Q team has a similar problem
  Jes> that could be solved in a similar way on ia32?

  Jes> Instead one can switch to using gettimeofday() for the timing,
  Jes> which with the new fast syscalls should be quite pleasant.

  Jes> I have a patch which implements this for glibc-2.2 (will do 2.3
  Jes> later), however what I don't like about it is that one ends up
  Jes> opening and reading /proc/sal/itc_drift in every single binary
  Jes> executed. To avoid the overhead of this it seems a good idea to
  Jes> me to provide this information via an AUX vector.

  Jes> If anybody is interested in the glibc patch, feel free to grab
  Jes> it from
  Jes> http://www.wildopensource.com/~jes/glibc/itc-drift-patch.diff
  Jes> For now it's a test patch, though it does seem to behave as
  Jes> expected.

  Jes> Thoughts?

Why do you need to export it if you just use a light-weight
gettimeofday()?  The light-weight syscall can readily access the
ITC-drift info via kernel memory.

BTW: I think someone should explore using an NTP-like approach to keep
ITC drift small enough that it's still usable for ITC-based
interpolation.  Granted, this assumes that hw drifts are reasonably
small and you'd still need to worry about different clock-frequencies,
but I suspect that in practice, this would work extremely well.  It
would be nice because it would be much faster than an HPET-based
approach, would work on all ia64 machines, and would provide better
resolution.  Furthermore, we already have all the logic in the
time-interpolation to ensure that there is never an observable time
discontinuity.

	--david

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[Jes Sorensen]

>>>>> "David" = David Mosberger <davidm@napali.hpl.hp.com> writes:

David> Why do you need to export it if you just use a light-weight
David> gettimeofday()?  The light-weight syscall can readily access
David> the ITC-drift info via kernel memory.

Let me get sure I understand you right here, you are suggesting I add
a new syscall to provide the info currently found in
/proc/sal/itc_drift? Right now it's just returning true/false (1/0) as
to whether the ITC drifts or not.

The reason I was thinking about using the AUX was that it's still
faster than making a system call and the dynamic linker will always be
calling it once. It doesn't make a big difference of course whether we
do it one way or another.

David> BTW: I think someone should explore using an NTP-like approach
David> to keep ITC drift small enough that it's still usable for
David> ITC-based interpolation.  Granted, this assumes that hw drifts
David> are reasonably small and you'd still need to worry about
David> different clock-frequencies, but I suspect that in practice,
David> this would work extremely well.  It would be nice because it
David> would be much faster than an HPET-based approach, would work on
David> all ia64 machines, and would provide better resolution.
David> Furthermore, we already have all the logic in the
David> time-interpolation to ensure that there is never an observable
David> time discontinuity.

I think this would work if it wasn't for the different clock frequency
problem. What real life ITC drifts are like I must admit I have no
idea.

Pardon my ignorance, what are you referring to by HPET? I am sure it's
something obvious and I know I have heard it before, but it's just not
ringing a bell here right now.

Cheers,
Jes

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[David Mosberger]

>>>>> On 19 Mar 2003 21:06:20 -0500, Jes Sorensen <jes@wildopensource.com> said:

>>>>> "David" = David Mosberger <davidm@napali.hpl.hp.com> writes:

  David> Why do you need to export it if you just use a light-weight
  David> gettimeofday()?  The light-weight syscall can readily access
  David> the ITC-drift info via kernel memory.

  Jes> Let me get sure I understand you right here, you are suggesting
  Jes> I add a new syscall to provide the info currently found in
  Jes> /proc/sal/itc_drift? Right now it's just returning true/false
  Jes> (1/0) as to whether the ITC drifts or not.

No, not at all.  I was simply suggesting to always use gettimeofday().
The light-weight implementation of gettimeofday() runs in ~130 cycles,
which isn't much slower than what you could do in user-level.

  Jes> what are you referring to by HPET?

HPET = High Precision Event Timer.  See:

  http://www.intel.com/labs/platcomp/hpet/hpetspec.htm

	--david

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[Rich Altmaier]

Jes, regarding your query on real life ITC drifts, I have this input.
Typical crystals used for the CPU clock today have an accuracy
of 250ppm.  That is 250 parts per million.  Which means for each
million ticks, it may actually be 250 realtime units too fast or too slow.
So in one second's worth of 1GHz ticks, it may be wrong versus
realtime by 250 microseconds.  
(not to belabor the point, but then a 1 millisecond error accumulates
each 4 seconds).

In case you ask "how can this be, my digital watch is better than that".
Your watch is better designed, plus it has more temperature stability
on your arm.  Plus nobody is willing to buy expensive crystals
for the computer...

FYI, Rich
Rich Altmaier
SGI


Jes Sorensen wrote:
...
> 
> I think this would work if it wasn't for the different clock frequency
> problem. What real life ITC drifts are like I must admit I have no
> idea.
>

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[David Mosberger]

>>>>> On Thu, 20 Mar 2003 15:55:02 -0800, Rich Altmaier <richa@engr.sgi.com> said:

  Rich> Jes, regarding your query on real life ITC drifts, I have this
  Rich> input.  Typical crystals used for the CPU clock today have an
  Rich> accuracy of 250ppm.  That is 250 parts per million.  Which
  Rich> means for each million ticks, it may actually be 250 realtime
  Rich> units too fast or too slow.  So in one second's worth of 1GHz
  Rich> ticks, it may be wrong versus realtime by 250 microseconds.
  Rich> (not to belabor the point, but then a 1 millisecond error
  Rich> accumulates each 4 seconds).

Good point.

  Rich> In case you ask "how can this be, my digital watch is better
  Rich> than that".  Your watch is better designed, plus it has more
  Rich> temperature stability on your arm.  Plus nobody is willing to
  Rich> buy expensive crystals for the computer...

Well, most watches don't run at 1GHz either! ;-)

	--david

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[Jes Sorensen]

>>>>> "David" = David Mosberger <davidm@napali.hpl.hp.com> writes:

>>>>> On 19 Mar 2003 21:06:20 -0500, Jes Sorensen <jes@wildopensource.com> said:
Jes> Let me get sure I understand you right here, you are suggesting I
Jes> add a new syscall to provide the info currently found in
Jes> /proc/sal/itc_drift? Right now it's just returning true/false
Jes> (1/0) as to whether the ITC drifts or not.

David> No, not at all.  I was simply suggesting to always use
David> gettimeofday().  The light-weight implementation of
David> gettimeofday() runs in ~130 cycles, which isn't much slower
David> than what you could do in user-level.

Ok, that makes it even simpler, I will do that then.

Cheers,
Jes

Re: [Linux-ia64] provide /proc/sal/itc_drift through AUX?

[Jes Sorensen]

>>>>> "Jes" = Jes Sorensen <jes@wildopensource.com> writes:

>>>>> "David" = David Mosberger <davidm@napali.hpl.hp.com> writes:
David> No, not at all.  I was simply suggesting to always use
David> gettimeofday().  The light-weight implementation of
David> gettimeofday() runs in ~130 cycles, which isn't much slower
David> than what you could do in user-level.

Jes> Ok, that makes it even simpler, I will do that then.

Thinking about this, I still think we need the patch for glibc-2.2 and
maybe glibc-2.3 for compiling with kernels < 2.5.x? Though I'd
probably just say we should take the hit for people using 2.4 kernels
with glibc-2.3. The fast syscall support isn't going into glibc-2.2.

I assume some of the BuzzwordCompliantServer packages from various
distributions are going to stick to glibc-2.2 and kernel-2.4.x for a
while.

Back to mangling glibc-2.3.

Cheers,
Jes