+ ntp-adjust-shift_pll-to-improve-ntp-convergence.patch added to -mm tree

+ ntp-adjust-shift_pll-to-improve-ntp-convergence.patch added to -mm tree

[akpm]

The patch titled
     ntp: adjust SHIFT_PLL to improve NTP convergence.
has been added to the -mm tree.  Its filename is
     ntp-adjust-shift_pll-to-improve-ntp-convergence.patch

Before you just go and hit "reply", please:
   a) Consider who else should be cc'ed
   b) Prefer to cc a suitable mailing list as well
   c) Ideally: find the original patch on the mailing list and do a
      reply-to-all to that, adding suitable additional cc's

*** Remember to use Documentation/SubmitChecklist when testing your code ***

See http://userweb.kernel.org/~akpm/stuff/added-to-mm.txt to find
out what to do about this

The current -mm tree may be found at http://userweb.kernel.org/~akpm/mmotm/

------------------------------------------------------
Subject: ntp: adjust SHIFT_PLL to improve NTP convergence.
From: john stultz <johnstul@us.ibm.com>

The conversion to the ntpv4 reference model
f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
reference model") in 2.6.19 added nanosecond resolution the adjtimex
interface, but also changed the "stiffness" of the frequency adjustments,
causing NTP convergence time to greatly increase.

SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
designed to be inversely linked to HZ, and the reference value of 4 was
designed for Unix systems using HZ=100.  However Linux's clock steering
code mostly independent of HZ.

So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
behavior to match kernels prior to 2.6.19, greatly reducing convergence
times, and improving close synchronization through environmental thermal
changes.

The patch also changes some l's to L's in nearby code to avoid misreading
50l as 501.

Signed-off-by: John Stultz <johnstul@us.ibm.com>
Acked-by: Rik van Riel <riel@redhat.com>
Cc: Roman Zippel <zippel@linux-m68k.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/timex.h |   44 +++++++++++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 12 deletions(-)

diff -puN include/linux/timex.h~adjust-shift_pll-to-improve-ntp-convergence include/linux/timex.h
--- a/include/linux/timex.h~adjust-shift_pll-to-improve-ntp-convergence
+++ a/include/linux/timex.h
@@ -170,17 +170,37 @@ struct timex {
 #include <asm/timex.h>
 
 /*
- * SHIFT_KG and SHIFT_KF establish the damping of the PLL and are chosen
- * for a slightly underdamped convergence characteristic. SHIFT_KH
- * establishes the damping of the FLL and is chosen by wisdom and black
- * art.
- *
- * MAXTC establishes the maximum time constant of the PLL. With the
- * SHIFT_KG and SHIFT_KF values given and a time constant range from
- * zero to MAXTC, the PLL will converge in 15 minutes to 16 hours,
- * respectively.
+ * SHIFT_PLL is used as a dampening factor to define how much we
+ * adjust the frequency correction for a given offset in PLL mode.
+ * It also used in dampening the offset correction, to define how
+ * much of the current value in time_offset we correct for each
+ * second. Changing this value changes the stiffness of the ntp
+ * adjustment code. A lower value makes it more flexible, reducing
+ * NTP convergence time. A higher value makes it stiffer, increasing
+ * convergence time, but making the clock more stable.
+ *
+ * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 4.
+ * However this seems to increase convergence time much too long.
+ *
+ * https://lists.ntp.org/pipermail/hackers/2008-January/003487.html
+ *
+ * In the above mailing list discussion, it seems the value of 4
+ * was appropriate for other Unix systems with HZ=100, and that
+ * SHIFT_PLL should be decreased as HZ increases. However, Linux's
+ * clock steering implementation is HZ independent.
+ *
+ * Through experimentation, a SHIFT_PLL value of 2 was found to allow
+ * for fast convergence (very similar to the NTPv3 code used prior to
+ * v2.6.19), with good clock stability.
+ *
+ *
+ * SHIFT_FLL is used as a dampening factor to define how much we
+ * adjust the frequency correction for a given offset in FLL mode.
+ * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 2.
+ *
+ * MAXTC establishes the maximum time constant of the PLL.
  */
-#define SHIFT_PLL	4	/* PLL frequency factor (shift) */
+#define SHIFT_PLL	2	/* PLL frequency factor (shift) */
 #define SHIFT_FLL	2	/* FLL frequency factor (shift) */
 #define MAXTC		10	/* maximum time constant (shift) */
 
@@ -192,10 +212,10 @@ struct timex {
 #define SHIFT_USEC 16		/* frequency offset scale (shift) */
 #define PPM_SCALE ((s64)NSEC_PER_USEC << (NTP_SCALE_SHIFT - SHIFT_USEC))
 #define PPM_SCALE_INV_SHIFT 19
-#define PPM_SCALE_INV ((1ll << (PPM_SCALE_INV_SHIFT + NTP_SCALE_SHIFT)) / \
+#define PPM_SCALE_INV ((1LL << (PPM_SCALE_INV_SHIFT + NTP_SCALE_SHIFT)) / \
 		       PPM_SCALE + 1)
 
-#define MAXPHASE 500000000l	/* max phase error (ns) */
+#define MAXPHASE 500000000L	/* max phase error (ns) */
 #define MAXFREQ 500000		/* max frequency error (ns/s) */
 #define MAXFREQ_SCALED ((s64)MAXFREQ << NTP_SCALE_SHIFT)
 #define MINSEC 256		/* min interval between updates (s) */
_

Patches currently in -mm which might be from johnstul@us.ibm.com are

linux-next.patch
time-move-calc_load-call-out-from-xtime_lock-protection.patch
time-avoid-possible-endless-loop-when-using-jiffies-clocksource-and-oneshot-mode-clockevent.patch
clockevent-export-register_device-and-delta2ns.patch
ntp-adjust-shift_pll-to-improve-ntp-convergence.patch
clocksource-sanity-check-sysfs-clocksource-changes.patch

[tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[tip-bot for john stultz]

Commit-ID:  22cfbbfd9f67b67fe073010f51cb71d3632387d5
Gitweb:     http://git.kernel.org/tip/22cfbbfd9f67b67fe073010f51cb71d3632387d5
Author:     john stultz <johnstul@us.ibm.com>
AuthorDate: Wed, 6 May 2009 11:43:57 +0200
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Wed, 6 May 2009 11:44:02 +0200

ntp: adjust SHIFT_PLL to improve NTP convergence

The conversion to the ntpv4 reference model
f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
reference model") in 2.6.19 added nanosecond resolution the adjtimex
interface, but also changed the "stiffness" of the frequency adjustments,
causing NTP convergence time to greatly increase.

SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
designed to be inversely linked to HZ, and the reference value of 4 was
designed for Unix systems using HZ=100.  However Linux's clock steering
code mostly independent of HZ.

So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
behavior to match kernels prior to 2.6.19, greatly reducing convergence
times, and improving close synchronization through environmental thermal
changes.

The patch also changes some l's to L's in nearby code to avoid misreading
50l as 501.

[ Impact: tweak NTP algorithm for faster convergence ]

Signed-off-by: John Stultz <johnstul@us.ibm.com>
Acked-by: Rik van Riel <riel@redhat.com>
Cc: zippel@linux-m68k.org
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
LKML-Reference: <200905051956.n45JuVo9025575@imap1.linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>


---
 include/linux/timex.h |   42 +++++++++++++++++++++++++++++++-----------
 1 files changed, 31 insertions(+), 11 deletions(-)

diff --git a/include/linux/timex.h b/include/linux/timex.h
index aa3475f..0daf961 100644
--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -170,17 +170,37 @@ struct timex {
 #include <asm/timex.h>
 
 /*
- * SHIFT_KG and SHIFT_KF establish the damping of the PLL and are chosen
- * for a slightly underdamped convergence characteristic. SHIFT_KH
- * establishes the damping of the FLL and is chosen by wisdom and black
- * art.
+ * SHIFT_PLL is used as a dampening factor to define how much we
+ * adjust the frequency correction for a given offset in PLL mode.
+ * It also used in dampening the offset correction, to define how
+ * much of the current value in time_offset we correct for each
+ * second. Changing this value changes the stiffness of the ntp
+ * adjustment code. A lower value makes it more flexible, reducing
+ * NTP convergence time. A higher value makes it stiffer, increasing
+ * convergence time, but making the clock more stable.
  *
- * MAXTC establishes the maximum time constant of the PLL. With the
- * SHIFT_KG and SHIFT_KF values given and a time constant range from
- * zero to MAXTC, the PLL will converge in 15 minutes to 16 hours,
- * respectively.
+ * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 4.
+ * However this seems to increase convergence time much too long.
+ *
+ * https://lists.ntp.org/pipermail/hackers/2008-January/003487.html
+ *
+ * In the above mailing list discussion, it seems the value of 4
+ * was appropriate for other Unix systems with HZ=100, and that
+ * SHIFT_PLL should be decreased as HZ increases. However, Linux's
+ * clock steering implementation is HZ independent.
+ *
+ * Through experimentation, a SHIFT_PLL value of 2 was found to allow
+ * for fast convergence (very similar to the NTPv3 code used prior to
+ * v2.6.19), with good clock stability.
+ *
+ *
+ * SHIFT_FLL is used as a dampening factor to define how much we
+ * adjust the frequency correction for a given offset in FLL mode.
+ * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 2.
+ *
+ * MAXTC establishes the maximum time constant of the PLL.
  */
-#define SHIFT_PLL	4	/* PLL frequency factor (shift) */
+#define SHIFT_PLL	2	/* PLL frequency factor (shift) */
 #define SHIFT_FLL	2	/* FLL frequency factor (shift) */
 #define MAXTC		10	/* maximum time constant (shift) */
 
@@ -192,10 +212,10 @@ struct timex {
 #define SHIFT_USEC 16		/* frequency offset scale (shift) */
 #define PPM_SCALE ((s64)NSEC_PER_USEC << (NTP_SCALE_SHIFT - SHIFT_USEC))
 #define PPM_SCALE_INV_SHIFT 19
-#define PPM_SCALE_INV ((1ll << (PPM_SCALE_INV_SHIFT + NTP_SCALE_SHIFT)) / \
+#define PPM_SCALE_INV ((1LL << (PPM_SCALE_INV_SHIFT + NTP_SCALE_SHIFT)) / \
 		       PPM_SCALE + 1)
 
-#define MAXPHASE 500000000l	/* max phase error (ns) */
+#define MAXPHASE 500000000L	/* max phase error (ns) */
 #define MAXFREQ 500000		/* max frequency error (ns/s) */
 #define MAXFREQ_SCALED ((s64)MAXFREQ << NTP_SCALE_SHIFT)
 #define MINSEC 256		/* min interval between updates (s) */

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[john stultz]

On Wed, 2009-05-06 at 09:46 +0000, tip-bot for john stultz wrote:
> Commit-ID:  22cfbbfd9f67b67fe073010f51cb71d3632387d5
> Gitweb:     http://git.kernel.org/tip/22cfbbfd9f67b67fe073010f51cb71d3632387d5
> Author:     john stultz <johnstul@us.ibm.com>
> AuthorDate: Wed, 6 May 2009 11:43:57 +0200
> Committer:  Ingo Molnar <mingo@elte.hu>
> CommitDate: Wed, 6 May 2009 11:44:02 +0200
> 
> ntp: adjust SHIFT_PLL to improve NTP convergence
> 
> The conversion to the ntpv4 reference model
> f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
> reference model") in 2.6.19 added nanosecond resolution the adjtimex
> interface, but also changed the "stiffness" of the frequency adjustments,
> causing NTP convergence time to greatly increase.
> 
> SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
> designed to be inversely linked to HZ, and the reference value of 4 was
> designed for Unix systems using HZ=100.  However Linux's clock steering
> code mostly independent of HZ.
> 
> So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
> behavior to match kernels prior to 2.6.19, greatly reducing convergence
> times, and improving close synchronization through environmental thermal
> changes.
> 
> The patch also changes some l's to L's in nearby code to avoid misreading
> 50l as 501.
> 
> [ Impact: tweak NTP algorithm for faster convergence ]


Bernhard Schiffner noticed I had a few comment typos in this patch,
(note: to save embarrassment, when making typos, avoid copying and
pasting them) so this patch corrects them. 

Signed-off-by: John Stultz <johnstul@us.ibm.com>

diff --git a/include/linux/timex.h b/include/linux/timex.h
index 0daf961..9910e3b 100644
--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -179,7 +179,7 @@ struct timex {
  * NTP convergence time. A higher value makes it stiffer, increasing
  * convergence time, but making the clock more stable.
  *
- * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 4.
+ * In David Mills' nanokernel reference implementation SHIFT_PLL is 4.
  * However this seems to increase convergence time much too long.
  *
  * https://lists.ntp.org/pipermail/hackers/2008-January/003487.html
@@ -196,7 +196,7 @@ struct timex {
  *
  * SHIFT_FLL is used as a dampening factor to define how much we
  * adjust the frequency correction for a given offset in FLL mode.
- * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 2.
+ * In David Mills' nanokernel reference implementation SHIFT_FLL is 2.
  *
  * MAXTC establishes the maximum time constant of the PLL.
  */

[tip:timers/ntp] ntp: fix comment typos

[tip-bot for john stultz]

Commit-ID:  e13cf6e04ebc231653e03ebde4799dc55bf62849
Gitweb:     http://git.kernel.org/tip/e13cf6e04ebc231653e03ebde4799dc55bf62849
Author:     john stultz <johnstul@us.ibm.com>
AuthorDate: Mon, 11 May 2009 18:13:13 -0700
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Tue, 12 May 2009 10:35:15 +0200

ntp: fix comment typos

Bernhard Schiffner noticed I had a few comment typos in this patch,
(note: to save embarrassment, when making typos, avoid copying and
pasting them) so this patch corrects them.

[ Impact: cleanup ]

Reported-by: Bernhard Schiffner <bernhard@schiffner-limbach.de>
Signed-off-by: John Stultz <johnstul@us.ibm.com>
Cc: riel@redhat.com
Cc: akpm@linux-foundation.org
LKML-Reference: <1242090794.7214.131.camel@localhost.localdomain>
Signed-off-by: Ingo Molnar <mingo@elte.hu>


---
 include/linux/timex.h |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/linux/timex.h b/include/linux/timex.h
index 0daf961..9910e3b 100644
--- a/include/linux/timex.h
+++ b/include/linux/timex.h
@@ -179,7 +179,7 @@ struct timex {
  * NTP convergence time. A higher value makes it stiffer, increasing
  * convergence time, but making the clock more stable.
  *
- * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 4.
+ * In David Mills' nanokernel reference implementation SHIFT_PLL is 4.
  * However this seems to increase convergence time much too long.
  *
  * https://lists.ntp.org/pipermail/hackers/2008-January/003487.html
@@ -196,7 +196,7 @@ struct timex {
  *
  * SHIFT_FLL is used as a dampening factor to define how much we
  * adjust the frequency correction for a given offset in FLL mode.
- * In David Mills' nanokenrel reference implmentation SHIFT_PLL is 2.
+ * In David Mills' nanokernel reference implementation SHIFT_FLL is 2.
  *
  * MAXTC establishes the maximum time constant of the PLL.
  */

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[John Stultz]

On Wed, 2009-05-06 at 09:46 +0000, tip-bot for john stultz wrote:
> Commit-ID:  22cfbbfd9f67b67fe073010f51cb71d3632387d5
> Gitweb:     http://git.kernel.org/tip/22cfbbfd9f67b67fe073010f51cb71d3632387d5
> Author:     john stultz <johnstul@us.ibm.com>
> AuthorDate: Wed, 6 May 2009 11:43:57 +0200
> Committer:  Ingo Molnar <mingo@elte.hu>
> CommitDate: Wed, 6 May 2009 11:44:02 +0200
> 
> ntp: adjust SHIFT_PLL to improve NTP convergence
> 
> The conversion to the ntpv4 reference model
> f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
> reference model") in 2.6.19 added nanosecond resolution the adjtimex
> interface, but also changed the "stiffness" of the frequency adjustments,
> causing NTP convergence time to greatly increase.
> 
> SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
> designed to be inversely linked to HZ, and the reference value of 4 was
> designed for Unix systems using HZ=100.  However Linux's clock steering
> code mostly independent of HZ.
> 
> So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
> behavior to match kernels prior to 2.6.19, greatly reducing convergence
> times, and improving close synchronization through environmental thermal
> changes.
> 
> The patch also changes some l's to L's in nearby code to avoid misreading
> 50l as 501.
> 
> [ Impact: tweak NTP algorithm for faster convergence ]

Hey Ingo,
	So I've been speaking with Miroslav (cc'ed) who maintains the RH ntpd
packages, and he's concerned that this patch takes us out of NTP's
expected behavior, which may cause problems when dealing with non-linux
systems using NTP.

We're both still trying to pin down exactly what NTP's expected behavior
should be with respect to convergence time, and there may be fixes to
the ntpd userland component that may resolve the slow convergence times
users are experiencing that the SHIFT_PLL change was trying to fix.

So in the interest of keeping the kernel's adjtimex() interface behavior
stable, I'd like to put a hold on this patch until Miroslav and I have
come to agreement about how to best resolve the issue.

So again, please don't push this when 2.6.31 opens.

thanks
-john

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[Ingo Molnar]

* John Stultz <johnstul@us.ibm.com> wrote:

> On Wed, 2009-05-06 at 09:46 +0000, tip-bot for john stultz wrote:
> > Commit-ID:  22cfbbfd9f67b67fe073010f51cb71d3632387d5
> > Gitweb:     http://git.kernel.org/tip/22cfbbfd9f67b67fe073010f51cb71d3632387d5
> > Author:     john stultz <johnstul@us.ibm.com>
> > AuthorDate: Wed, 6 May 2009 11:43:57 +0200
> > Committer:  Ingo Molnar <mingo@elte.hu>
> > CommitDate: Wed, 6 May 2009 11:44:02 +0200
> > 
> > ntp: adjust SHIFT_PLL to improve NTP convergence
> > 
> > The conversion to the ntpv4 reference model
> > f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
> > reference model") in 2.6.19 added nanosecond resolution the adjtimex
> > interface, but also changed the "stiffness" of the frequency adjustments,
> > causing NTP convergence time to greatly increase.
> > 
> > SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
> > designed to be inversely linked to HZ, and the reference value of 4 was
> > designed for Unix systems using HZ=100.  However Linux's clock steering
> > code mostly independent of HZ.
> > 
> > So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
> > behavior to match kernels prior to 2.6.19, greatly reducing convergence
> > times, and improving close synchronization through environmental thermal
> > changes.
> > 
> > The patch also changes some l's to L's in nearby code to avoid misreading
> > 50l as 501.
> > 
> > [ Impact: tweak NTP algorithm for faster convergence ]
> 
> Hey Ingo,
>
> 	So I've been speaking with Miroslav (cc'ed) who maintains 
> the RH ntpd packages, and he's concerned that this patch takes us 
> out of NTP's expected behavior, which may cause problems when 
> dealing with non-linux systems using NTP.

I might be missing something here - but Linux converging faster 
seems like a genuinely good thing. What non-Linux problem could 
there be? Linux's convergence is really Linux's private issue. 

(since things like the PIT calibration are random noise for which 
there can be no external expecation about our convergence speed 
anyway.)

	Ingo

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[John Stultz]

On Tue, 2009-06-02 at 01:22 +0200, Ingo Molnar wrote:
> * John Stultz <johnstul@us.ibm.com> wrote:
> > On Wed, 2009-05-06 at 09:46 +0000, tip-bot for john stultz wrote:
> > > ntp: adjust SHIFT_PLL to improve NTP convergence
> > > 
> > > The conversion to the ntpv4 reference model
> > > f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
> > > reference model") in 2.6.19 added nanosecond resolution the adjtimex
> > > interface, but also changed the "stiffness" of the frequency adjustments,
> > > causing NTP convergence time to greatly increase.
> > > 
> > > SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
> > > designed to be inversely linked to HZ, and the reference value of 4 was
> > > designed for Unix systems using HZ=100.  However Linux's clock steering
> > > code mostly independent of HZ.
> > > 
> > > So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
> > > behavior to match kernels prior to 2.6.19, greatly reducing convergence
> > > times, and improving close synchronization through environmental thermal
> > > changes.
> > > 
> > > 
> > > [ Impact: tweak NTP algorithm for faster convergence ]
> >
> > 	So I've been speaking with Miroslav (cc'ed) who maintains 
> > the RH ntpd packages, and he's concerned that this patch takes us 
> > out of NTP's expected behavior, which may cause problems when 
> > dealing with non-linux systems using NTP.
> 
> I might be missing something here - but Linux converging faster 
> seems like a genuinely good thing. What non-Linux problem could 
> there be? Linux's convergence is really Linux's private issue. 

Yea. It does seem that way. Miroslav can likely expand on the issue to
help clarify, but as I understand it, the example is if you have a
number of systems that are peers in an NTP network. All of them are
using the same userland NTP daemon. However, if the rate of change that
corrections are applied is different in half of them, you will have
problems getting all the systems to converge together. 

An rough analogy might be creating a an automatic control system to
drive an RC car, and then letting it control a Ferrari. 

Now, that said, I have yet to actually see or hear of any negative
effects of the patch in testing, but I'd  prefer to try to keep to the
NTP spec if we can.

So Miroslav is helping by looking for similar changes we can possibly
make from the userland side. This not only lets us keep the adjtimex()
interface stable while we sort out what options we have, but also, by
trying to tune the NTP knobs in userland instead of kernel space, we are
more likely to get feedback and hopefully constructive solutions by the
NTP gurus who have in the past have maybe been less interested in
Linux's behavior.

So yea, we'll see what comes of it. In the meantime, I know some folks
who will continue to use this patch on their systems because it really
improves things for them. So I'm not abandoning the patch just yet, but
I want to really make sure we're not needlessly changing the kernel
behavior.

thanks
-john

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[Rik van Riel]

John Stultz wrote:
> On Tue, 2009-06-02 at 01:22 +0200, Ingo Molnar wrote:

>> I might be missing something here - but Linux converging faster 
>> seems like a genuinely good thing. What non-Linux problem could 
>> there be? Linux's convergence is really Linux's private issue. 
> 
> Yea. It does seem that way. Miroslav can likely expand on the issue to
> help clarify, but as I understand it, the example is if you have a
> number of systems that are peers in an NTP network. All of them are
> using the same userland NTP daemon. However, if the rate of change that
> corrections are applied is different in half of them, you will have
> problems getting all the systems to converge together. 

Would this not be true already, because the convergence
of Linux system suddenly became a lot slower in 2.6.19?

Damned if we do, damned if we don't - except the new
behaviour introduced by your patches is nicer.

-- 
All rights reversed.

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[Ingo Molnar]

* Rik van Riel <riel@redhat.com> wrote:

> John Stultz wrote:
>> On Tue, 2009-06-02 at 01:22 +0200, Ingo Molnar wrote:
>
>>> I might be missing something here - but Linux converging faster  
>>> seems like a genuinely good thing. What non-Linux problem could  
>>> there be? Linux's convergence is really Linux's private issue. 
>>
>> Yea. It does seem that way. Miroslav can likely expand on the 
>> issue to help clarify, but as I understand it, the example is if 
>> you have a number of systems that are peers in an NTP network. 
>> All of them are using the same userland NTP daemon. However, if 
>> the rate of change that corrections are applied is different in 
>> half of them, you will have problems getting all the systems to 
>> converge together.
>
> Would this not be true already, because the convergence of Linux 
> system suddenly became a lot slower in 2.6.19?
>
> Damned if we do, damned if we don't - except the new behaviour 
> introduced by your patches is nicer.

Not just that - but there's calibration noise during bootup that can 
cause randomly distributed recalibrations as well. So other hosts in 
a mixed environment will see inconsistencies anyway, after every 
bootup.

NTP is all about being able to be resilient against time noise and 
being able to sync up to a common time base ASAP.

	Ingo

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[Miroslav Lichvar]

On Tue, Jun 02, 2009 at 02:20:39AM +0200, Ingo Molnar wrote:
> > Would this not be true already, because the convergence of Linux 
> > system suddenly became a lot slower in 2.6.19?
> >
> > Damned if we do, damned if we don't - except the new behaviour 
> > introduced by your patches is nicer.
> 
> Not just that - but there's calibration noise during bootup that can 
> cause randomly distributed recalibrations as well. So other hosts in 
> a mixed environment will see inconsistencies anyway, after every 
> bootup.
> 
> NTP is all about being able to be resilient against time noise and 
> being able to sync up to a common time base ASAP.

There has to be a compromise between frequency and offset noise. When
SHIFT_PLL is set to 2 the frequency noise will be higher and that will
have a negative impact on the long-term ability to keep the clock
accurate. The error will grow faster when network connection is
suspended.

The PLL response can be configured to be the same as the proposed
SHIFT_PLL 2 by decreasing the time constant value in adjtimex
structure, so I'd rather keep following the NTP specification and
control it from userspace if necessary.

As for the calibration issue, would it be possible to export the
information that an instable clocksource is used and when was the last
time it was calibrated? Then we'd know when the drift file should not
be trusted and let NTP calculate the frequency directly (it takes
about 15 minutes).

-- 
Miroslav Lichvar

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[john stultz]

On Tue, 2009-06-02 at 18:22 +0200, Miroslav Lichvar wrote:
> On Tue, Jun 02, 2009 at 02:20:39AM +0200, Ingo Molnar wrote:
> > 
> > Not just that - but there's calibration noise during bootup that can 
> > cause randomly distributed recalibrations as well. So other hosts in 
> > a mixed environment will see inconsistencies anyway, after every 
> > bootup.
[snip]
> As for the calibration issue, would it be possible to export the
> information that an instable clocksource is used and when was the last
> time it was calibrated? Then we'd know when the drift file should not
> be trusted and let NTP calculate the frequency directly (it takes
> about 15 minutes).

Just to de-thread the issues here, the calibration noise issue really is
separate from the SHIFT_PLL convergence issue.

I'd really prefer the calibration noise issue to be resolved by the
kernel, as its really only an issue on a subset of x86 machines. The
tsc_khz= boot option I proposed earlier for folks who really care seems
to me like a good route.

The only NTPd side change to help the calibration issue that might be
useful, would be a explicit ntp option to force NTP to always calculate
the freq on startup if the drift file was present or not. Anything else
would be way too much of a hack to get around bad kernel behavior.

thanks
-john

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[John Stultz]

On Mon, 2009-06-01 at 20:06 -0400, Rik van Riel wrote:
> John Stultz wrote:
> > On Tue, 2009-06-02 at 01:22 +0200, Ingo Molnar wrote:
> 
> >> I might be missing something here - but Linux converging faster 
> >> seems like a genuinely good thing. What non-Linux problem could 
> >> there be? Linux's convergence is really Linux's private issue. 
> > 
> > Yea. It does seem that way. Miroslav can likely expand on the issue to
> > help clarify, but as I understand it, the example is if you have a
> > number of systems that are peers in an NTP network. All of them are
> > using the same userland NTP daemon. However, if the rate of change that
> > corrections are applied is different in half of them, you will have
> > problems getting all the systems to converge together. 
> 
> Would this not be true already, because the convergence
> of Linux system suddenly became a lot slower in 2.6.19?

Yes, this is true. But some folks have considered Linux to have had a
faulty NTP implementation up until 2.6.19.

> Damned if we do, damned if we don't - except the new
> behaviour introduced by your patches is nicer.

It would seem this way, so I'm not throwing the patch out yet. I'm just
suggesting we hold off including it until we've tried attacking the
issue from a few other angles. Miroslav understands the details behind
the NTP protocol much better then I, so I'd like to try to address them
before going out on our own.

I just want to avoid the kernel from oscillating between fast(and maybe
incorrect)convergence and ntp-spec-compliance.

thanks
-john

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[Ray Lee]

On Mon, Jun 1, 2009 at 4:58 PM, John Stultz <johnstul@us.ibm.com> wrote:
> On Tue, 2009-06-02 at 01:22 +0200, Ingo Molnar wrote:
>> * John Stultz <johnstul@us.ibm.com> wrote:
>> > On Wed, 2009-05-06 at 09:46 +0000, tip-bot for john stultz wrote:
>> > > ntp: adjust SHIFT_PLL to improve NTP convergence
>> > >
>> > > The conversion to the ntpv4 reference model
>> > > f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
>> > > reference model") in 2.6.19 added nanosecond resolution the adjtimex
>> > > interface, but also changed the "stiffness" of the frequency adjustments,
>> > > causing NTP convergence time to greatly increase.
>> > >
>> > > SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
>> > > designed to be inversely linked to HZ, and the reference value of 4 was
>> > > designed for Unix systems using HZ=100.  However Linux's clock steering
>> > > code mostly independent of HZ.
>> > >
>> > > So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
>> > > behavior to match kernels prior to 2.6.19, greatly reducing convergence
>> > > times, and improving close synchronization through environmental thermal
>> > > changes.
>> > >
>> > >
>> > > [ Impact: tweak NTP algorithm for faster convergence ]
>> >
>> >     So I've been speaking with Miroslav (cc'ed) who maintains
>> > the RH ntpd packages, and he's concerned that this patch takes us
>> > out of NTP's expected behavior, which may cause problems when
>> > dealing with non-linux systems using NTP.
>>
>> I might be missing something here - but Linux converging faster
>> seems like a genuinely good thing. What non-Linux problem could
>> there be? Linux's convergence is really Linux's private issue.
>
> Yea. It does seem that way. Miroslav can likely expand on the issue to
> help clarify, but as I understand it, the example is if you have a
> number of systems that are peers in an NTP network. All of them are
> using the same userland NTP daemon. However, if the rate of change that
> corrections are applied is different in half of them, you will have
> problems getting all the systems to converge together.

Your point is clear, however -- reasonably speaking --  how many
instances will there be out there of networks of peers partially
upgraded versus lone systems slowly or never converging off of
masters?

By my naive understanding, the latter would strongly outnumber the former.

Re: [tip:timers/ntp] ntp: adjust SHIFT_PLL to improve NTP convergence

[Mr. James W. Laferriere]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3191 bytes --]

 	Hello All ,

On Mon, 1 Jun 2009, Ray Lee wrote:
> On Mon, Jun 1, 2009 at 4:58 PM, John Stultz <johnstul@us.ibm.com> wrote:
>> On Tue, 2009-06-02 at 01:22 +0200, Ingo Molnar wrote:
>>> * John Stultz <johnstul@us.ibm.com> wrote:
>>>> On Wed, 2009-05-06 at 09:46 +0000, tip-bot for john stultz wrote:
>>>>> ntp: adjust SHIFT_PLL to improve NTP convergence
>>>>>
>>>>> The conversion to the ntpv4 reference model
>>>>> f19923937321244e7dc334767eb4b67e0e3d5c74 ("ntp: convert to the NTP4
>>>>> reference model") in 2.6.19 added nanosecond resolution the adjtimex
>>>>> interface, but also changed the "stiffness" of the frequency adjustments,
>>>>> causing NTP convergence time to greatly increase.
>>>>>
>>>>> SHIFT_PLL, which reduces the stiffness of the freq adjustments, was
>>>>> designed to be inversely linked to HZ, and the reference value of 4 was
>>>>> designed for Unix systems using HZ=100.  However Linux's clock steering
>>>>> code mostly independent of HZ.
>>>>>
>>>>> So this patch reduces the SHIFT_PLL value from 4 to 2, which causes NTPd
>>>>> behavior to match kernels prior to 2.6.19, greatly reducing convergence
>>>>> times, and improving close synchronization through environmental thermal
>>>>> changes.
>>>>>
>>>>>
>>>>> [ Impact: tweak NTP algorithm for faster convergence ]
>>>>
>>>>     So I've been speaking with Miroslav (cc'ed) who maintains
>>>> the RH ntpd packages, and he's concerned that this patch takes us
>>>> out of NTP's expected behavior, which may cause problems when
>>>> dealing with non-linux systems using NTP.
>>>
>>> I might be missing something here - but Linux converging faster
>>> seems like a genuinely good thing. What non-Linux problem could
>>> there be? Linux's convergence is really Linux's private issue.
>>
>> Yea. It does seem that way. Miroslav can likely expand on the issue to
>> help clarify, but as I understand it, the example is if you have a
>> number of systems that are peers in an NTP network. All of them are
>> using the same userland NTP daemon. However, if the rate of change that
>> corrections are applied is different in half of them, you will have
>> problems getting all the systems to converge together.
>
> Your point is clear, however -- reasonably speaking --  how many
> instances will there be out there of networks of peers partially
> upgraded versus lone systems slowly or never converging off of
> masters?
 	A site with three or four differant system types ,  ie: sparc running 
sloaris ,  pc running openbsd ,  Dec(hp) running VMS ,  Dec(hp) Alpha running 
Linux , ...
 	This moving the Hardware Arch & OS differencews is sometimes done to 
limit hacks & software glicthes by NOT using the same hardware arch or OS .

> By my naive understanding, the latter would strongly outnumber the former.
 	You'd be VERY unhappily suprised .

 		Hth ,  JimL
-- 
+------------------------------------------------------------------+
| James   W.   Laferriere | System    Techniques | Give me VMS     |
| Network&System Engineer | 2133    McCullam Ave |  Give me Linux  |
| babydr@baby-dragons.com | Fairbanks, AK. 99701 |   only  on  AXP |
+------------------------------------------------------------------+