* [PATCH] clocksource: Print clocksource name when clocksource is tested unstable @ 2022-11-16 8:22 Yunying Sun 2022-11-16 20:31 ` Paul E. McKenney 2022-11-17 6:46 ` Feng Tang 0 siblings, 2 replies; 6+ messages in thread From: Yunying Sun @ 2022-11-16 8:22 UTC (permalink / raw) To: tglx, peterz, paulmck, longman, x86, linux-kernel; +Cc: feng.tang, yunying.sun Some "TSC fall back to HPET" cases are seen on systems that have more than 2 numa nodes. When this happens, in kernel log it has: clocksource: timekeeping watchdog on CPU168: hpet read-back delay of 4296200ns, attempt 4, marking unstable The "hpet" here is misleading since it prints only the name of watchdog, where actually it's measuring the delay of 3 reads: wd-clocksource-wd. Signed-off-by: Yunying Sun <yunying.sun@intel.com> --- kernel/time/clocksource.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c index 8058bec87ace..fac8c0d90e61 100644 --- a/kernel/time/clocksource.c +++ b/kernel/time/clocksource.c @@ -257,8 +257,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, goto skip_test; } - pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n", - smp_processor_id(), watchdog->name, wd_delay, nretries); + pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd read-back delay of %lldns, attempt %d, marking unstable\n", + smp_processor_id(), cs->name, wd_delay, nretries); return WD_READ_UNSTABLE; skip_test: -- 2.17.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable 2022-11-16 8:22 [PATCH] clocksource: Print clocksource name when clocksource is tested unstable Yunying Sun @ 2022-11-16 20:31 ` Paul E. McKenney 2022-11-17 1:12 ` Sun, Yunying 2022-11-17 6:46 ` Feng Tang 1 sibling, 1 reply; 6+ messages in thread From: Paul E. McKenney @ 2022-11-16 20:31 UTC (permalink / raw) To: Yunying Sun; +Cc: tglx, peterz, longman, x86, linux-kernel, feng.tang On Wed, Nov 16, 2022 at 04:22:21PM +0800, Yunying Sun wrote: > Some "TSC fall back to HPET" cases are seen on systems that have more > than 2 numa nodes. When this happens, in kernel log it has: > > clocksource: timekeeping watchdog on CPU168: hpet read-back delay of 4296200ns, attempt 4, marking unstable > > The "hpet" here is misleading since it prints only the name of watchdog, > where actually it's measuring the delay of 3 reads: wd-clocksource-wd. > > Signed-off-by: Yunying Sun <yunying.sun@intel.com> Good eyes! I queued this with wordsmithing as shown below. Please let me know if I messed something up. Thanx, Paul ------------------------------------------------------------------------ commit 72ff63e2d3f19963a4ef401e5f8c1bd16646b587 Author: Yunying Sun <yunying.sun@intel.com> Date: Wed Nov 16 16:22:21 2022 +0800 clocksource: Print clocksource name when clocksource is tested unstable Some "TSC fall back to HPET" messages appear on systems having more than 2 NUMA nodes: clocksource: timekeeping watchdog on CPU168: hpet read-back delay of 4296200ns, attempt 4, marking unstable The "hpet" here is misleading the clocksource watchdog is really doing repeated reads of "hpet" in order to check for unrelated delays. Therefore, print the name of the clocksource under test, prefixed by "wd-" and suffixed by "-wd", for example, "wd-tsc-wd". Signed-off-by: Yunying Sun <yunying.sun@intel.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c index 4015ec6503a52..6f681d56a843f 100644 --- a/kernel/time/clocksource.c +++ b/kernel/time/clocksource.c @@ -257,8 +257,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, goto skip_test; } - pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n", - smp_processor_id(), watchdog->name, wd_delay, nretries); + pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd read-back delay of %lldns, attempt %d, marking unstable\n", + smp_processor_id(), cs->name, wd_delay, nretries); return WD_READ_UNSTABLE; skip_test: ^ permalink raw reply related [flat|nested] 6+ messages in thread
* RE: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable 2022-11-16 20:31 ` Paul E. McKenney @ 2022-11-17 1:12 ` Sun, Yunying 0 siblings, 0 replies; 6+ messages in thread From: Sun, Yunying @ 2022-11-17 1:12 UTC (permalink / raw) To: paulmck@kernel.org Cc: tglx@linutronix.de, peterz@infradead.org, longman@redhat.com, x86@kernel.org, linux-kernel@vger.kernel.org, Tang, Feng Hi Paul, Thank you for the quick review. The commit message looks more accurate and clear after your wordsmithing. -Yunying -----Original Message----- From: Paul E. McKenney <paulmck@kernel.org> Sent: Thursday, 17 November, 2022 04:31 To: Sun, Yunying <yunying.sun@intel.com> Cc: tglx@linutronix.de; peterz@infradead.org; longman@redhat.com; x86@kernel.org; linux-kernel@vger.kernel.org; Tang, Feng <feng.tang@intel.com> Subject: Re: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable On Wed, Nov 16, 2022 at 04:22:21PM +0800, Yunying Sun wrote: > Some "TSC fall back to HPET" cases are seen on systems that have more > than 2 numa nodes. When this happens, in kernel log it has: > > clocksource: timekeeping watchdog on CPU168: hpet read-back delay of > 4296200ns, attempt 4, marking unstable > > The "hpet" here is misleading since it prints only the name of > watchdog, where actually it's measuring the delay of 3 reads: wd-clocksource-wd. > > Signed-off-by: Yunying Sun <yunying.sun@intel.com> Good eyes! I queued this with wordsmithing as shown below. Please let me know if I messed something up. Thanx, Paul ------------------------------------------------------------------------ commit 72ff63e2d3f19963a4ef401e5f8c1bd16646b587 Author: Yunying Sun <yunying.sun@intel.com> Date: Wed Nov 16 16:22:21 2022 +0800 clocksource: Print clocksource name when clocksource is tested unstable Some "TSC fall back to HPET" messages appear on systems having more than 2 NUMA nodes: clocksource: timekeeping watchdog on CPU168: hpet read-back delay of 4296200ns, attempt 4, marking unstable The "hpet" here is misleading the clocksource watchdog is really doing repeated reads of "hpet" in order to check for unrelated delays. Therefore, print the name of the clocksource under test, prefixed by "wd-" and suffixed by "-wd", for example, "wd-tsc-wd". Signed-off-by: Yunying Sun <yunying.sun@intel.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c index 4015ec6503a52..6f681d56a843f 100644 --- a/kernel/time/clocksource.c +++ b/kernel/time/clocksource.c @@ -257,8 +257,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, goto skip_test; } - pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n", - smp_processor_id(), watchdog->name, wd_delay, nretries); + pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd read-back delay of %lldns, attempt %d, marking unstable\n", + smp_processor_id(), cs->name, wd_delay, nretries); return WD_READ_UNSTABLE; skip_test: ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable 2022-11-16 8:22 [PATCH] clocksource: Print clocksource name when clocksource is tested unstable Yunying Sun 2022-11-16 20:31 ` Paul E. McKenney @ 2022-11-17 6:46 ` Feng Tang 2022-11-17 8:03 ` Sun, Yunying 1 sibling, 1 reply; 6+ messages in thread From: Feng Tang @ 2022-11-17 6:46 UTC (permalink / raw) To: Yunying Sun; +Cc: tglx, peterz, paulmck, longman, x86, linux-kernel On Wed, Nov 16, 2022 at 04:22:21PM +0800, Yunying Sun wrote: > Some "TSC fall back to HPET" cases are seen on systems that have more > than 2 numa nodes. When this happens, in kernel log it has: Hi Yunying, If the system's physical sockests number <= 2, I would suggest you to try the patch https://lore.kernel.org/lkml/20221021062131.1826810-1-feng.tang@intel.com/, which is still under review and discussion. Thanks, Feng > clocksource: timekeeping watchdog on CPU168: hpet read-back delay of 4296200ns, attempt 4, marking unstable > > The "hpet" here is misleading since it prints only the name of watchdog, > where actually it's measuring the delay of 3 reads: wd-clocksource-wd. > > Signed-off-by: Yunying Sun <yunying.sun@intel.com> > --- > kernel/time/clocksource.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c > index 8058bec87ace..fac8c0d90e61 100644 > --- a/kernel/time/clocksource.c > +++ b/kernel/time/clocksource.c > @@ -257,8 +257,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, > goto skip_test; > } > > - pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n", > - smp_processor_id(), watchdog->name, wd_delay, nretries); > + pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd read-back delay of %lldns, attempt %d, marking unstable\n", > + smp_processor_id(), cs->name, wd_delay, nretries); > return WD_READ_UNSTABLE; > > skip_test: > -- > 2.17.0 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable 2022-11-17 6:46 ` Feng Tang @ 2022-11-17 8:03 ` Sun, Yunying 2022-11-17 8:35 ` Feng Tang 0 siblings, 1 reply; 6+ messages in thread From: Sun, Yunying @ 2022-11-17 8:03 UTC (permalink / raw) To: Tang, Feng Cc: tglx@linutronix.de, peterz@infradead.org, paulmck@kernel.org, longman@redhat.com, x86@kernel.org, linux-kernel@vger.kernel.org Hi Feng, For the "tsc fall back to hpet" issue, thanks a lot for pointing me to the fix proposal patch earlier: https://lore.kernel.org/lkml/20221021062131.1826810-1-feng.tang@intel.com/ . I shared this patch set to our customer yesterday. Just got feedback that customer has verified the "tsc fall back to hpet" issue does not appear anymore with this patch set, after 24hrs stress test on 8 servers. -Yunying -----Original Message----- From: Tang, Feng <feng.tang@intel.com> Sent: Thursday, 17 November, 2022 14:47 To: Sun, Yunying <yunying.sun@intel.com> Cc: tglx@linutronix.de; peterz@infradead.org; paulmck@kernel.org; longman@redhat.com; x86@kernel.org; linux-kernel@vger.kernel.org Subject: Re: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable On Wed, Nov 16, 2022 at 04:22:21PM +0800, Yunying Sun wrote: > Some "TSC fall back to HPET" cases are seen on systems that have more > than 2 numa nodes. When this happens, in kernel log it has: Hi Yunying, If the system's physical sockests number <= 2, I would suggest you to try the patch https://lore.kernel.org/lkml/20221021062131.1826810-1-feng.tang@intel.com/, which is still under review and discussion. Thanks, Feng > clocksource: timekeeping watchdog on CPU168: hpet read-back delay of > 4296200ns, attempt 4, marking unstable > > The "hpet" here is misleading since it prints only the name of > watchdog, where actually it's measuring the delay of 3 reads: wd-clocksource-wd. > > Signed-off-by: Yunying Sun <yunying.sun@intel.com> > --- > kernel/time/clocksource.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c > index 8058bec87ace..fac8c0d90e61 100644 > --- a/kernel/time/clocksource.c > +++ b/kernel/time/clocksource.c > @@ -257,8 +257,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, > goto skip_test; > } > > - pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n", > - smp_processor_id(), watchdog->name, wd_delay, nretries); > + pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd read-back delay of %lldns, attempt %d, marking unstable\n", > + smp_processor_id(), cs->name, wd_delay, nretries); > return WD_READ_UNSTABLE; > > skip_test: > -- > 2.17.0 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable 2022-11-17 8:03 ` Sun, Yunying @ 2022-11-17 8:35 ` Feng Tang 0 siblings, 0 replies; 6+ messages in thread From: Feng Tang @ 2022-11-17 8:35 UTC (permalink / raw) To: Sun, Yunying, tglx@linutronix.de, peterz@infradead.org, dave.hansen Cc: paulmck@kernel.org, longman@redhat.com, x86@kernel.org, linux-kernel@vger.kernel.org On Thu, Nov 17, 2022 at 04:03:07PM +0800, Sun, Yunying wrote: > Hi Feng, > > For the "tsc fall back to hpet" issue, thanks a lot for pointing me to the fix proposal patch earlier: > https://lore.kernel.org/lkml/20221021062131.1826810-1-feng.tang@intel.com/ . > > I shared this patch set to our customer yesterday. > Just got feedback that customer has verified the "tsc fall back to hpet" issue does not appear anymore with this patch set, after 24hrs stress test on 8 servers. Thanks for the testing! So your case is a real case that current 'nr_online_nodes' check cannot cover, which Dave concerned about. Thanks, Feng > -Yunying > > -----Original Message----- > From: Tang, Feng <feng.tang@intel.com> > Sent: Thursday, 17 November, 2022 14:47 > To: Sun, Yunying <yunying.sun@intel.com> > Cc: tglx@linutronix.de; peterz@infradead.org; paulmck@kernel.org; longman@redhat.com; x86@kernel.org; linux-kernel@vger.kernel.org > Subject: Re: [PATCH] clocksource: Print clocksource name when clocksource is tested unstable > > On Wed, Nov 16, 2022 at 04:22:21PM +0800, Yunying Sun wrote: > > Some "TSC fall back to HPET" cases are seen on systems that have more > > than 2 numa nodes. When this happens, in kernel log it has: > > Hi Yunying, > > If the system's physical sockests number <= 2, I would suggest you to try the patch https://lore.kernel.org/lkml/20221021062131.1826810-1-feng.tang@intel.com/, > which is still under review and discussion. > > Thanks, > Feng > > > clocksource: timekeeping watchdog on CPU168: hpet read-back delay of > > 4296200ns, attempt 4, marking unstable > > > > The "hpet" here is misleading since it prints only the name of > > watchdog, where actually it's measuring the delay of 3 reads: wd-clocksource-wd. > > > > Signed-off-by: Yunying Sun <yunying.sun@intel.com> > > --- > > kernel/time/clocksource.c | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c > > index 8058bec87ace..fac8c0d90e61 100644 > > --- a/kernel/time/clocksource.c > > +++ b/kernel/time/clocksource.c > > @@ -257,8 +257,8 @@ static enum wd_read_status cs_watchdog_read(struct clocksource *cs, u64 *csnow, > > goto skip_test; > > } > > > > - pr_warn("timekeeping watchdog on CPU%d: %s read-back delay of %lldns, attempt %d, marking unstable\n", > > - smp_processor_id(), watchdog->name, wd_delay, nretries); > > + pr_warn("timekeeping watchdog on CPU%d: wd-%s-wd read-back delay of %lldns, attempt %d, marking unstable\n", > > + smp_processor_id(), cs->name, wd_delay, nretries); > > return WD_READ_UNSTABLE; > > > > skip_test: > > -- > > 2.17.0 > > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-11-17 8:39 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-11-16 8:22 [PATCH] clocksource: Print clocksource name when clocksource is tested unstable Yunying Sun 2022-11-16 20:31 ` Paul E. McKenney 2022-11-17 1:12 ` Sun, Yunying 2022-11-17 6:46 ` Feng Tang 2022-11-17 8:03 ` Sun, Yunying 2022-11-17 8:35 ` Feng Tang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.