All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Paul E. McKenney" <paulmck@kernel.org>
To: tglx@linutronix.de
Cc: linux-kernel@vger.kernel.org, john.stultz@linaro.org,
	sboyd@kernel.org, corbet@lwn.net, Mark.Rutland@arm.com,
	maz@kernel.org, kernel-team@meta.com, neeraju@codeaurora.org,
	ak@linux.intel.com, feng.tang@intel.com, zhengjun.xing@intel.com,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Chris Mason <clm@meta.com>, John Stultz <jstultz@google.com>,
	Waiman Long <longman@redhat.com>
Subject: [PATCH clocksource 1/3] clocksource: Reject bogus watchdog clocksource measurements
Date: Mon, 14 Nov 2022 15:28:24 -0800	[thread overview]
Message-ID: <20221114232827.835599-1-paulmck@kernel.org> (raw)
In-Reply-To: <20221114232807.GA834337@paulmck-ThinkPad-P17-Gen-1>

One remaining clocksource-skew issue involves extreme CPU overcommit,
which can cause the clocksource watchdog measurements to be delayed by
tens of seconds.  This in turn means that a clock-skew criterion that
is appropriate for a 500-millisecond interval will instead give lots of
false positives.

Therefore, check for the watchdog clocksource reporting much larger or
much less than the time specified by WATCHDOG_INTERVAL.  In these cases,
print a pr_warn() warning and refrain from marking the clocksource under
test as being unstable.

Reported-by: Chris Mason <clm@meta.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: John Stultz <jstultz@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Stephen Boyd <sboyd@kernel.org>
Cc: Feng Tang <feng.tang@intel.com>
Cc: Waiman Long <longman@redhat.com>
---
 kernel/time/clocksource.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index 8058bec87acee..dcaf38c062161 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -386,7 +386,7 @@ EXPORT_SYMBOL_GPL(clocksource_verify_percpu);
 
 static void clocksource_watchdog(struct timer_list *unused)
 {
-	u64 csnow, wdnow, cslast, wdlast, delta;
+	u64 csnow, wdnow, cslast, wdlast, delta, wdi;
 	int next_cpu, reset_pending;
 	int64_t wd_nsec, cs_nsec;
 	struct clocksource *cs;
@@ -440,6 +440,17 @@ static void clocksource_watchdog(struct timer_list *unused)
 		if (atomic_read(&watchdog_reset_pending))
 			continue;
 
+		/* Check for bogus measurements. */
+		wdi = jiffies_to_nsecs(WATCHDOG_INTERVAL);
+		if (wd_nsec < (wdi >> 2)) {
+			pr_warn("timekeeping watchdog on CPU%d: Watchdog clocksource '%s' advanced only %lld ns during %d-jiffy time interval, skipping watchdog check.\n", smp_processor_id(), watchdog->name, wd_nsec, WATCHDOG_INTERVAL);
+			continue;
+		}
+		if (wd_nsec > (wdi << 2)) {
+			pr_warn("timekeeping watchdog on CPU%d: Watchdog clocksource '%s' advanced an excessive %lld ns during %d-jiffy time interval, probable CPU overutilization, skipping watchdog check.\n", smp_processor_id(), watchdog->name, wd_nsec, WATCHDOG_INTERVAL);
+			continue;
+		}
+
 		/* Check the deviation from the watchdog clocksource. */
 		md = cs->uncertainty_margin + watchdog->uncertainty_margin;
 		if (abs(cs_nsec - wd_nsec) > md) {
-- 
2.31.1.189.g2e36527f23


  reply	other threads:[~2022-11-14 23:28 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-14 23:28 [PATCH clocksource 0/3] Reject bogus watchdog clocksource measurements Paul E. McKenney
2022-11-14 23:28 ` Paul E. McKenney [this message]
2022-11-17 21:57   ` [PATCH clocksource 1/3] clocksource: " Thomas Gleixner
2022-11-17 23:09     ` Paul E. McKenney
2022-11-21  0:55       ` Feng Tang
2022-11-21 15:21         ` Paul E. McKenney
2022-11-21 18:14         ` Paul E. McKenney
2022-11-22 15:55           ` Feng Tang
2022-11-22 22:07             ` Paul E. McKenney
2022-11-23  2:36               ` Feng Tang
2022-11-23 21:23                 ` Paul E. McKenney
2022-11-28  2:15                   ` Feng Tang
2022-11-29 19:29                     ` Paul E. McKenney
2022-11-30  1:38                       ` Feng Tang
2022-11-30  4:12                         ` Paul E. McKenney
2022-11-30  4:49                           ` Feng Tang
2022-11-30  5:16                             ` Paul E. McKenney
2022-11-30  5:35                               ` Feng Tang
2022-11-30  5:50                                 ` Paul E. McKenney
2022-11-30  6:00                                   ` Feng Tang
2022-12-01 17:24                                     ` Paul E. McKenney
2022-12-02  1:10                                       ` Feng Tang
2022-12-02  1:44                                         ` Paul E. McKenney
2022-12-02  2:02                                           ` Feng Tang
2022-12-02 22:24                                             ` Paul E. McKenney
2022-12-03  2:51                                               ` Feng Tang
2022-11-14 23:28 ` [PATCH clocksource 2/3] clocksource: Add comments to classify bogus measurements Paul E. McKenney
2022-11-14 23:28 ` [PATCH clocksource 3/3] clocksource: Exponential backoff for load-induced bogus watchdog reads Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221114232827.835599-1-paulmck@kernel.org \
    --to=paulmck@kernel.org \
    --cc=Mark.Rutland@arm.com \
    --cc=ak@linux.intel.com \
    --cc=clm@meta.com \
    --cc=corbet@lwn.net \
    --cc=feng.tang@intel.com \
    --cc=john.stultz@linaro.org \
    --cc=jstultz@google.com \
    --cc=kernel-team@meta.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longman@redhat.com \
    --cc=maz@kernel.org \
    --cc=neeraju@codeaurora.org \
    --cc=sboyd@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=zhengjun.xing@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.